Let’s start with the obvious
question: what is value added? Well, at its most basic – and at the risk of
teaching people to suck eggs here – it is a comparison of pupils’ actual and
expected outcomes. The ‘actual’ bit is fairly straightforward: it’s what a
pupil gets in a test, obviously. Well, it’s quite straightforward except it’s
actually the test score converted to a fine grade, and this is subject to change
every year (more on this in a bit). But what is ‘expected’? Well, there is the
common misconception that expected progress equates to 12 points of progress
from KS1 to KS2; and value-added is therefore the difference between a pupil’s
actual progress and the expected rate of 12 points. Unfortunately this is very
wrong and I know of schools that been significantly below for VA despite the cohort
having made just over 12 points of progress on average. This has come as a bit
of a shock.

So, start again: what is expected
progress? First, it should be pointed out that ‘the DfE does not define
expected progress in terms of APS’ (see p6, paragraph 10 of Ofsted Subsidiary
Guidance). However, this is not entirely true. Ofsted, in the same document
(see p33) go on to define attainment gaps in terms of differences in ability
i.e. a 3 point gap between groups of pupils (e.g. pupil premium/non-Pupil Premium)
is stated as being a year’s difference. This implies that 3 points is
equivalent to a year’s progress, thus reinforcing the misconception that 12
points equates to expected progress across KS2.

More pertinent to this discussion
is the fact that VA involves the calculation of an APS estimate for the end of
the key stage against which the actual KS2 outcome will be compared. This calculation
is based on the prior attainment (APS) of the pupil at KS1 and the resulting
difference (KS2 APS estimate - KS1 APS actual) is essentially the expected
progress for that pupil. In other words, pupils with different prior attainment
have different expectations. Essentially VA is the comparison of a pupil’s
progress against that of pupils with similar prior attainment nationally, and
variations can be rather surprising.

So how can you find out VA
estimates for your pupils? The best place to start is RAISEonline. Most users
tend to log in to RAISE, download the summary report and don’t log in again
until the following autumn (by which time they’ve forgotten their password or
the account has been locked). However, there is other useful stuff in RAISE if
you know where to look, and top of the list is the KS1-2 VA Pupil list. Scroll
down through that long list of interactive/dynamic reports and you’ll find it
about two thirds of the way down the list in the VA section. Click on it and
when it opens, export it to Excel. The report lists all pupils in the previous
Year 6 cohort alongside pupil characteristics, their KS1 outcomes, and, most
importantly in this case, their actual and expected KS2 outcomes in terms of
APS. To calculate actual progress, subtract KS1 APS (Column Y) from the actual KS2
overall fine grade (column Z); to calculate expected progress, subtract KS1 APS
(column Y) from the estimated KS2 overall point score (column AD). Do this for
each child in columns at the end of the spreadsheet and then calculate the
column averages. These will provide the average actual and average expected
rates of progress. The difference between these two averages (actual –
estimated) added on to 100 is essentially the school’s VA score (give or take
some adjustments relating to cohort size (known as shrinkage factor)). The
average expected progress rate is, for most schools, between 13.2-13.4 points,
but could be lower or higher than this.

What shocks headteachers the
most, upon performing the above exercise, are those pupils for whom VA expected
progress is greater than 14 points. In extreme cases expected progress may be
higher than 15 e.g. a pupil with 2c in Reading and writing, and L3 in maths at
KS1 has expected progress rates of just 11.9 points in maths yet 15.98 points
in reading. The next question is usually “how do I calculate estimates for my
current pupils?” There is a tool in the RAISE library (the 2013 Key Stage 1 to
Key Stage 2 pupil ready reckoner – see link below), which will do this but it’s
a bit of a pain as you can only calculate estimates for one pupil and subject at
a time, which is a tedious and time consuming process; and makes it difficult
to calculate cohort averages. However, it is useful for converting test scores
to fine grades (right click on any of the tabs and ‘unhide’ the fine grade
lookup table). To download this tool, click on link below and go to the ‘How
Ofsted analyse your data section’:

https://www.raiseonline.org/documentlibrary/ViewDocumentLibrary.aspx

To address this, I’ve built an
excel-based VA calculator tool, which will calculate KS2 estimates for reading,
writing and maths for all pupils in a cohort. It will also calculate potential
VA scores for a cohort and significance if end of KS2 targets/predictions are
entered into the columns provided. Please get in touch if you’d like a copy of
this tool.

So, what does this mean for
target setting? Well, as mentioned previously, Ofsted do not overtly define
expected progress in terms of APS (except they do when it comes to VA estimates!).
Instead, they now expect schools to talk about percentages on track to make two
and three levels of progress (again, see subsidiary guidance p6). However, many
schools continue to set APS-based targets, often at 4 points per year (16
points across the key stage), and this is commonly prescribed by consultants et
al. It is, therefore, worth pointing out, from my own analysis of schools’ VA
pupil lists, that 16 points of progress on average would result in the school
being ranked at the 2

^{nd}or 3^{rd}percentile (a school at the 1^{st}percentile had a cohort which made 17 points of progress on average). 16 points of progress is therefore highly aspirational and 97% of schools do not reach this lofty bar. Instead it may be more appropriate to aim a sublevel above the estimates generated by my VA calculator tool. Alternatively, considering schools ranked at or above the 20^{th}percentile in RAISE are usually significantly above national average for VA, schools could use FFT PA 15-20^{th}percentile estimates (or SE estimates if preferred, but be warned that this may be somewhat lower depending on your school context). Both of these methods are VA based and link to the prior attainment of the pupil, which is perhaps preferable to a one-size fits all approach to target setting currently employed in many schools.So, those are some thoughts on KS2 value-added. What the future is for this measure is anyone’s guess but the Primary Assessment Consultation does suggest a VA floor standard in future, and a baseline at the start of reception as a possibility. With the removal of levels on the horizon, it may be that some sort of VA measure becomes a more useful measure of progress. Watch this space!

And what about KS4? Well, that’s
a topic for another day.

## No comments:

## Post a Comment