Monday, 24 February 2014

Key Stage 2 Value Added: Fact and Fiction

Let’s start with the obvious question: what is value added? Well, at its most basic – and at the risk of teaching people to suck eggs here – it is a comparison of pupils’ actual and expected outcomes. The ‘actual’ bit is fairly straightforward: it’s what a pupil gets in a test, obviously. Well, it’s quite straightforward except it’s actually the test score converted to a fine grade, and this is subject to change every year (more on this in a bit). But what is ‘expected’? Well, there is the common misconception that expected progress equates to 12 points of progress from KS1 to KS2; and value-added is therefore the difference between a pupil’s actual progress and the expected rate of 12 points. Unfortunately this is very wrong and I know of schools that been significantly below for VA despite the cohort having made just over 12 points of progress on average. This has come as a bit of a shock.

So, start again: what is expected progress? First, it should be pointed out that ‘the DfE does not define expected progress in terms of APS’ (see p6, paragraph 10 of Ofsted Subsidiary Guidance). However, this is not entirely true. Ofsted, in the same document (see p33) go on to define attainment gaps in terms of differences in ability i.e. a 3 point gap between groups of pupils (e.g. pupil premium/non-Pupil Premium) is stated as being a year’s difference. This implies that 3 points is equivalent to a year’s progress, thus reinforcing the misconception that 12 points equates to expected progress across KS2.

More pertinent to this discussion is the fact that VA involves the calculation of an APS estimate for the end of the key stage against which the actual KS2 outcome will be compared. This calculation is based on the prior attainment (APS) of the pupil at KS1 and the resulting difference (KS2 APS estimate - KS1 APS actual) is essentially the expected progress for that pupil. In other words, pupils with different prior attainment have different expectations. Essentially VA is the comparison of a pupil’s progress against that of pupils with similar prior attainment nationally, and variations can be rather surprising.

https://www.raiseonline.org/documentlibrary/ViewDocumentLibrary.aspx

To address this, I’ve built an excel-based VA calculator tool, which will calculate KS2 estimates for reading, writing and maths for all pupils in a cohort. It will also calculate potential VA scores for a cohort and significance if end of KS2 targets/predictions are entered into the columns provided. Please get in touch if you’d like a copy of this tool.

So, what does this mean for target setting? Well, as mentioned previously, Ofsted do not overtly define expected progress in terms of APS (except they do when it comes to VA estimates!). Instead, they now expect schools to talk about percentages on track to make two and three levels of progress (again, see subsidiary guidance p6). However, many schools continue to set APS-based targets, often at 4 points per year (16 points across the key stage), and this is commonly prescribed by consultants et al. It is, therefore, worth pointing out, from my own analysis of schools’ VA pupil lists, that 16 points of progress on average would result in the school being ranked at the 2nd or 3rd percentile (a school at the 1st percentile had a cohort which made 17 points of progress on average). 16 points of progress is therefore highly aspirational and 97% of schools do not reach this lofty bar. Instead it may be more appropriate to aim a sublevel above the estimates generated by my VA calculator tool. Alternatively, considering schools ranked at or above the 20th percentile in RAISE are usually significantly above national average for VA, schools could use FFT PA 15-20th percentile estimates (or SE estimates if preferred, but be warned that this may be somewhat lower depending on your school context). Both of these methods are VA based and link to the prior attainment of the pupil, which is perhaps preferable to a one-size fits all approach to target setting currently employed in many schools.

So, those are some thoughts on KS2 value-added. What the future is for this measure is anyone’s guess but the Primary Assessment Consultation does suggest a VA floor standard in future, and a baseline at the start of reception as a possibility. With the removal of levels on the horizon, it may be that some sort of VA measure becomes a more useful measure of progress. Watch this space!

And what about KS4? Well, that’s a topic for another day.