tag:blogger.com,1999:blog-8546074595093882670.post6604390087619266618..comments2018-02-14T13:02:13.062-08:00Comments on School Data Updates: KS2 progress measures 2017: a guide to what has and hasn't changedJames Pembrokenoreply@blogger.comBlogger16125tag:blogger.com,1999:blog-8546074595093882670.post-42474258745578446542017-10-11T04:21:08.118-07:002017-10-11T04:21:08.118-07:00In a word: yes! If your pupils from same start poi...In a word: yes! If your pupils from same start points achieved the same as last year, your VA would definitely be lower unless your cohort comprised very low prior attainers, whose estimates actually dropped this year due to inclusion of special school data. Most schools would however see a drop of 1-1.5 as you say, because other pupils nationally have done quite a bit better this year. It’s a zero sum game unfortunately. Heard a few HTs half jokingly say ‘I wish we hadn’t done so well last year’. James Pembrokehttps://www.blogger.com/profile/11839642769644656596noreply@blogger.comtag:blogger.com,1999:blog-8546074595093882670.post-52923032314910596662017-10-11T03:47:01.510-07:002017-10-11T03:47:01.510-07:00Hi James
Our overall combined progress measure fo...Hi James<br /><br />Our overall combined progress measure for 2017 was +2.6 (sig +) as oppose to +4.8 (sig +) in 2016.<br />The FFT aspire dashboard denotes a downward arrow for progress on the 2017 dashboard front page.<br /><br />Given that nationally each PA group made on average about 1 to 1.5 scaled points more in 2017 than they did in 2016, would it be correct to say that if my pupils attainment remained in line with that of last year, and from their starting points they reached the same end points as they had in 2016, then the school VA, would in fact be approx 1 to 1.5 points lower than last year. <br />Given that VA is a measure of the difference between scaled score achieved at the school, and that found nationally on average, then in my case the gap has narrowed not necessarily because my children made significantly less progress, but because nationally the children made more progress - hence the gap has narrowed.<br /><br />I am finding myself having to argue that our progress is not significantly worse than last year, ( although it is slightly worse) but the gap has narrowed and thus VA fallen because the nation has done better on average, not that our children have done worse.<br /><br />I refer back to a previous blog of yours in which you clearly argue that 2016 data is not easily compared to 2017 data, yet a down arrow on FFT dashboard implies one is being judged against the other.<br /><br />If the 2016 VA were to be recalculated on the basis of the 2017 calibration scale, then I am aware that my 2106 VA would drop from the dizzy heights of +4.6, but maybe the difference between 2016 and 2107 would then be not quite such a chasm, that I find myself having to justify.<br /><br />Im sure it is not often that schools ask for their VA to be reduced, but it seems if data from one year to another is to be compared then the same calibration scale needs to be used.<br /><br />Would be interested to know your thoughts and I keep coming back to trying to find a way of calculating VA, using a stable measure of expected progress, that does not vary year to year according to how well or how bad the national cohort does.<br /><br />Thanks<br />saint petershttps://www.blogger.com/profile/07928474554657743737noreply@blogger.comtag:blogger.com,1999:blog-8546074595093882670.post-11617081026099878832017-09-27T12:57:21.601-07:002017-09-27T12:57:21.601-07:00Ok, thanks for your quick reply.Ok, thanks for your quick reply.KMachttps://www.blogger.com/profile/17129493258302452801noreply@blogger.comtag:blogger.com,1999:blog-8546074595093882670.post-68272908088289763402017-09-27T12:50:39.568-07:002017-09-27T12:50:39.568-07:00There's no guidance for this so your guess is ...There's no guidance for this so your guess is as good as mine. FFT have come up with a logical method. Alternatively devise your own. I wouldn't be surprised if DfE assign KS1 assessments a nominal score in order to calculate APS. Guess we'll have to wait 2 years to find out.James Pembrokehttps://www.blogger.com/profile/11839642769644656596noreply@blogger.comtag:blogger.com,1999:blog-8546074595093882670.post-90012770794080244882017-09-27T12:47:30.538-07:002017-09-27T12:47:30.538-07:00Hi James. I'm currently working out our low, m...Hi James. I'm currently working out our low, middle and high attainers from year 3 and 4. Can I just clarify that if a child's results were: WTS, WTS and ARE, they would still fall into the 'middle' bracket. <br />Thanks KMachttps://www.blogger.com/profile/17129493258302452801noreply@blogger.comtag:blogger.com,1999:blog-8546074595093882670.post-63849854859409809942017-09-19T13:04:37.508-07:002017-09-19T13:04:37.508-07:00You mean at KS1? Prior attainment bands as follows...You mean at KS1? Prior attainment bands as follows:<br />Low: <12 APS<br />Middle: 12-17.99 APS<br />High: 18+ APS<br /><br />Hope that helps James Pembrokehttps://www.blogger.com/profile/11839642769644656596noreply@blogger.comtag:blogger.com,1999:blog-8546074595093882670.post-40612407127777526452017-09-19T12:10:11.130-07:002017-09-19T12:10:11.130-07:00Thank you James for your help and clarity.
Is the...Thank you James for your help and clarity.<br /><br />Is the measure of 17+ still deemed as above average?<br />What is 'above above average'?<br />ThanksUnknownhttps://www.blogger.com/profile/06393981198857148908noreply@blogger.comtag:blogger.com,1999:blog-8546074595093882670.post-42565402841335026072017-09-14T04:32:36.184-07:002017-09-14T04:32:36.184-07:00No. It just means less than 5% are below floor (it...No. It just means less than 5% are below floor (it was approx 5% last year). Surprised that they kept them the same but this way you can show improvement due to fewer schools below floor.<br /><br />Genius. James Pembrokehttps://www.blogger.com/profile/11839642769644656596noreply@blogger.comtag:blogger.com,1999:blog-8546074595093882670.post-10672800786667756842017-09-14T04:23:46.158-07:002017-09-14T04:23:46.158-07:00Interesting Point to make about floor targets at R...Interesting Point to make about floor targets at Reading: -5<br />Writing: -7 Maths: -5 is the Bottom 5% is Reading: -3.9 & below Writing: -4 & below Maths: -4.3 & below. <br /><br />Does that mean no school is below floor?<br /><br />Julian Wood (@ideas_factory)Unknownhttps://www.blogger.com/profile/02362551615609120965noreply@blogger.comtag:blogger.com,1999:blog-8546074595093882670.post-9001022463360028042017-09-10T10:29:59.621-07:002017-09-10T10:29:59.621-07:00Yes, in theory it's possible: if the national ...Yes, in theory it's possible: if the national average score for 2b pupils was 98 then any 2b pupil achieving above that will have a positive VA score. However it is highly unlikely. Last year the national estimates for pupils with KS1 APS of 15 were around 100. This year slightly higher reflecting the increasing KS2 attainment of that prior attainment group. It will no doubt only rise as years go on. <br /><br />The other thing to consider is whether 2b really represents the national expectation at KS1. 1) KS1 assessments are highly subjective and therefore not especially reliable (one school's 2b is another school's 2c or 2a); and 2) 2b ceased to be national average many years ago. The last time sublevels were used, the KS1 APS was around 16.8 so much closer to a 2a. <br /><br />Let's arrange a call. James Pembrokehttps://www.blogger.com/profile/11839642769644656596noreply@blogger.comtag:blogger.com,1999:blog-8546074595093882670.post-14967847232747500162017-09-10T09:53:58.722-07:002017-09-10T09:53:58.722-07:00"But that's how it works now: it 2b child..."But that's how it works now: it 2b children on average score 98, then 98 is the estimate, so if your 2b child scores 99 then they make positive progress." <br />Yes, I agree with this - I think I'm understanding the current process now, however, I am wondering whether this is appropriate? A child who was age related at ks1, converting to below age related at ks2, yet still getting a +ve VA?<br /><br />I shall email you and book a conversation - thank you.<br /><br />Maybe even a chance for you to come up to Bolton and address our head teacher cluster?<br /><br />saint petershttps://www.blogger.com/profile/07928474554657743737noreply@blogger.comtag:blogger.com,1999:blog-8546074595093882670.post-51676556086372824972017-09-10T09:24:10.016-07:002017-09-10T09:24:10.016-07:00But that's how it works now: it 2b children on...But that's how it works now: it 2b children on average score 98, then 98 is the estimate, so if your 2b child scores 99 then they make positive progress. <br /><br />The phrase 'expected progress' concerns me. The removal of levels has resulted in the end of expected progress measure. Even Ofsted have stated that inspectors must not use the phrase. By fixing estimated outcomes to a specific scaled score from each start point, we have effectively reinstalled the measure. <br /><br />It's a complex issue. Probably best discussing in person or by phone. Email me and we can arrange a phone call. James Pembrokehttps://www.blogger.com/profile/11839642769644656596noreply@blogger.comtag:blogger.com,1999:blog-8546074595093882670.post-8375485042738694232017-09-10T06:37:41.825-07:002017-09-10T06:37:41.825-07:00Thanks - it's all very fascinating.
I now hav...Thanks - it's all very fascinating.<br /><br />I now have so many questions.<br /><br />I find myself wondering how the system would change if the expected progress where determined by a static attainment outcome I.e., 15 APS convert to 100 scaled score, as oppose to the current system where expected progress is determined by how well the national cohort does.<br />Theoretically if the national cohort do badly, then expected progress in that year would drop? A PA 2b child could fail to achieve ARE at ks2, yet still have a positive VA, if they exceed national average for the PA group? Hypothetical, I realise, yet an interesting thought.<br /><br />Thank you for your insight - always appreciated.saint petershttps://www.blogger.com/profile/07928474554657743737noreply@blogger.comtag:blogger.com,1999:blog-8546074595093882670.post-21912870409785278172017-09-09T14:59:21.192-07:002017-09-09T14:59:21.192-07:00It perhaps makes more sense if you think about KS4...It perhaps makes more sense if you think about KS4. Perhaps the average GCSE result in maths for a pupil that was 4.5 at KS2 is a grade 4 this year. This becomes the VA benchmark for all 4.5 pupils. Next year that may increase to a grade 5, so the benchmark (expectation) increases to a grade 5. If we implemented the system you proposed for KS2, we would always expect a grade 4 for 4.5 pupils, but that the value of a grade 4 changes over time. This couldn't work because grades are supposed to be comparable (stable). Scaled scores are also supposed to be comparable year on year. James Pembrokehttps://www.blogger.com/profile/11839642769644656596noreply@blogger.comtag:blogger.com,1999:blog-8546074595093882670.post-3839482337461594322017-09-09T14:50:32.127-07:002017-09-09T14:50:32.127-07:00I sort of get where you're coming from. Howeve...I sort of get where you're coming from. However, doing that goes against the concept of VA: that pupils are compared against the average score for their prior attainment group in that year. <br /><br />The scaled score conversion already does fluctuate each year to reflect changes in difficulty of the tests. The raw score required to achieve expected standard is set using results of sample and live tests. Once the expected standard has been set, they can then establish the scale. If you say in advance that all 2b pupils (bear in mind we only have 2 more years of them) need to get 100 to make average progress, then the underlying the raw score will change considerably. it doesn't really change anything: what children have to get to make average progress still changes every year. Either it's a raw score, or the scaled score. <br /><br />Also I think what you're proposing changes the meaning and value of the expected standard. At the moment it represents an acabdemic standard we expect all pupils to meet; if we went with this idea, it would become the standard we expect 2b pupils to meet.<br /><br />It would also probably result in the return of expected progress measures, progress matrices, and points-based progress measures.<br /><br />15 to 100 is 85 points over the key stage.<br /><br />That's approx 20 points per year. <br /><br />I worry about stuff like that. James Pembrokehttps://www.blogger.com/profile/11839642769644656596noreply@blogger.comtag:blogger.com,1999:blog-8546074595093882670.post-39227086552824217652017-09-09T14:27:01.608-07:002017-09-09T14:27:01.608-07:00Fantastic post - thank you.
Just to clarify thou...Fantastic post - thank you.<br /><br />Just to clarify though - why do the estimates have to change each year? <br />Could the estimates not be kept the same, but the scaled score conversion be adjusted to account for the change in average raw score achieved for that PA group.<br />So the raw score may fluctuate each year, needed to achieve the scaled score required, but the scaled score estimate could remain static - thus allowing the 2b PA group of 15 APS+ to always be expected to achieve 100 scaled score? <br /> saint petershttps://www.blogger.com/profile/07928474554657743737noreply@blogger.com