Monday, 9 July 2018

What I think a primary tracking system should do

I talk and write a lot about the issues with primary tracking systems: that many have reinvented levels, and are often inflexible and overly complicated. Quite rightly this means I get challenged to define what a good system looks like, and it's a tricky question to answer, but I think I'm getting there now.

I've already written a post on five golden rules of tracking, which summarised my talk at the inaugural Learning First conference. I still stand by all these, but have since added a sixth rule: don't compromise your approach to fit the rules of a system. Essentially, whatever software you use, it needs to be flexible so you can adapt it to accommodate your approach to assessment as it develops. I hear too many teachers say "the system doesn't really work for us" but they persevere, finding workarounds, ignoring vast swathes of data, focussing on the colours, navigating numerous reports to find something useful, and fuzzy-matching ill-fitting criteria that is out of alignment with their own curriculum. It's not necessarily the tracking systems I have a problem with, it's the approach within the system. If your system can't be adapted to be more meaningful and better suited to the needs of your school, don't struggle on with it, change the system.

Thankfully most systems now offer some level of customisation.

So this is what I think a primary tracking system needs to offer:

1) A flexible approach to tracking objectives
Some schools want to track against objectives, some schools don't. Some schools want a few KPIs, some schools want more. Some schools want something bespoke, some schools are happy with national curriculum objectives. Whatever your approach, ensure your system accommodates it and can be modified as and when you change your mind.

Personally, I think too many schools are tracking against far too many objectives and this needs paring back drastically. It is counter-productive to have teachers spending their weekends and evenings ticking boxes. Chances are it's not informing anything and is highly likely to be having a negative impact if it's sucking up teachers' time and eroding morale. It's important that you have a system that quickly allows you to reduce the objectives you track against. Or delete them entirely.

Whilst we're on the subject, think very carefully before extending this process into foundation subjects. Ask yourself: why do you need this data? Will it improve outcomes? Will it tell you anything you didn't already know? What impact will it have on workload?

Be honest!

2) Bespoke summative assessment descriptors and point-in-time assessment
Systems should be designed from classroom upwards as tools for teaching and learning, not from the head's office downwards as tools for accountability. With this in mind, ensure those assessment descriptors reflect the language of the classroom. On-track, secure, expected, achieved, at age-related - whatever you use on a day to day basis to summarise learning should be reflected in the system. This again means we need systems that are flexible.

And don't reinvent levels. I'm referring to those steps that pupils apparently progress through, where they're all emerging because it's autumn, are apparently developing in the spring, and magically become secure after Easter. This was never useful, never linked to reality, and was all about having a neat, linear point scale in order to measure progress. I believe that to get tracking right we need to put our obsession with progress measures to one side. It drives everything in the wrong direction.

If we don't reinvent levels, what should we do? More and more schools are adopting a simple 'point in time' assessment i.e. if a pupil is keeping pace with the demands of the curriculum, and gets what has been taught so far, then they are 'secure' or 'on-track' and are therefore making good progress. We don't need any point scores or arbitrary thresholds, we just need that simple overall descriptor. Yes, it means they are likely to be in the same 'band' all year, which means we can kiss goodbye to our flightpath and associated points, but honestly that's fine.

And finally, the overall assessment should be based purely on a teacher's judgement, not on some dubious algorithm linked to how many objectives have been ticked. For too long we have relied on systems for answers - an assessment by numbers approach - and it's time teachers were given back this responsibility and regained their confidence.

3) Assessment out of year group and tracking interventions
Tricky to do in many systems, and perhaps somewhat controversial, but I think it's important that teachers can easily track pupils against previous (or even next!) year's objectives (if the school is tracking against objectives, of course). I also think systems should allow users to create their own lists of objectives for specific, supported groups of children, rather than limiting tracking to national curriculum statements. In fact, this may be the only objective-level tracking a school chooses to do: just for those pupils that are working below their curriculum year. One thing's for sure: I don't see how it's useful to describe, say, a year 4 pupil that is working well below as Year 4 Emerging for the entire year. Greater system flexibility will allow that pupil to have a more appropriate assessment, and one school I visited recently used the term 'personal curriculum' instead of 'well below' or 'emerging'. I rather like that.

4) Handling test scores and other data
Many schools use tests, and systems need to be able to store and analyse that data, whether it be standardised scores, raw marks, percentages, or reading ages. This should be straightforward to enter onto the system and, if the school so chooses, easily integrated into reports. It seems crazy to spend a lot of money on a system only to have to store test scores or other assessment data in a spreadsheet, where it can't analysed alongside the teacher assessment data.

5) A few simple reports
I think there are only three reports that primary schools need:
  1. A quick overview of attainment showing percentages/numbers of pupils that are below, at, or above where you expected them to be in reading, writing and maths at a given point in time, based either on teacher assessment or a test if desired. Users should be able to drill down to identify individual pupils, in each category, and this will be enough to answer many of the questions that are likely to get asked by external agencies.
  2. A progress matrix. I'm a fan of these because they are simple, easily understood by all, and summarise progress visually without quantifying it so they get away from the need for points and levels. Essentially it's a grid with rows and columns, with the vertical axis usually used for a previous assessment and the horizontal axis used for the current assessment. We can then talk about those five pupils that were 'secure' but are now 'working towards'; or those 6 pupils that were struggling last term but are now 'above expectations'. Rather than talking about abstract concepts of points and measures, we are talking about pupils, which is all teachers want to do anyway. And don't forget that matrices can also be used to compare other types of data eg standardised test compared to teacher assessment at one assessment point; EYFS or KS1 prior attainment to latest teacher assessment, or results in one subject against another. 
  3. A summary table that pulls all key data together in one place - prior attainment, teacher assessment, or test scores - and groups it by year group and/or pupil characteristic groups (if statistically meaningful!). Whatever the school deems necessary for the intended purpose, whether that be a governor meeting, SIA visit, or Ofsted inspection, the system should quickly provide it in an accessible, bespoke format. Many if not most schools produce such tables of data; unfortunately all too often this is an onerous manual exercise, which involves running numerous reports, noting down figures and transferring them to a template in Word or Excel. And the next term, they do it all again. A huge waste of time and something that needs to stop.
These are only suggestions and many schools will have already gone beyond this. For example, I know plenty of schools that do not require teachers to record assessments against objectives; they simply make an overall assessment three times per year. Then there is the pupil group-level data that many schools spend a great deal of time producing. The usefulness of such data is certainly questionable (I think we've always known this) and it was encouraging to hear Amanda Spielman address this issue recently. Ultimately, the less the powers that be insist on abstract and low quality data, the less data schools will need to produce, the less complicated systems need to be, and the more we can focus on teaching and learning.

I think we are moving in the right direction.

Now we just need our systems to catch up. 

VA Calculator: excel version free to download

With KS2 results about to be released on NCA Tools (10th July, 07.30) I thought I'd publish a link to the latest excel version of my VA calculator. This version, with pupil group tables for progress and average scores, can be downloaded here

To use the VA calculator, you will need to download first (it will be read only in your browser). To download, click on the 3 dots found top right of browser tab window and select 'download'. This will open the tool in full excel mode on your laptop. Recommend reading the notes page first and also worth reading the primary accountability guidance for more information about nominal scores, p-scales and pre-key stage pupils. Progress measures were quite bit more complicated in 2017 with more prior attainment groups (24 instead of 21), closing of the loophole (pupils not scoring on test receive a nominal score of 79), and nominal scores assigned to individual p-scales (rather than blanket score of 70 assigned to BLW).

Which, in my opinion, means this year's progress data is not comparable with last year's, but hey ho.....

Hopefully we won't see too much change to methodology in autumn 2018 but we can't assume anything. 

and please, please note that this tool is for guidance only. The official progress data will benchmark pupils' scores against the national average score for pupils with the same prior attainment in the same year. Essentially, the estimated scores shown in the VA calculator WILL change. 

and if you want the online tool, which is very neat, easy to use, and GDPR compliant (i.e. it does not collect the data, the data is only stored in the cache memory of your browser and can't be accessed by Insight or anyone else), then you can find it here:


and let me know asap if you find any errors