Wednesday, 25 June 2014

Review: Insight Tracking system for primary schools

There are loads of tracking systems available for schools now; and like the Ofsted judgements they seek to achieve (or avoid) they range from outstanding to inadequate. Many primary schools are now turning away from desktop-based systems (usually Excel) in favour of online solutions. Some of these online tools started life as a spreadsheet and it shows: they are clunky, messy and unintuitive. Other systems are extraordinarily complex and powerful but perhaps lack the support required for many primary schools, particularly smaller ones, to cope. And some are a bit odd, providing data that doesn't really seem to make much sense. The best systems are simple, easy to use, visually arresting and built ground up for the web. Insight definitely fits into the latter group.

Before I go any further with this review, I should point out that I have no link to the company other than getting to know Andrew and Sarah, the people behind Insight. I'm just a fan. It came to my attention over a year ago when I was searching for a system that would meet the needs of some of our smaller schools. It certainly does that. In fact it would meet the needs of a 2 form entry school as well as it does a village primary with 50 pupils. What I like about it is its elegance and ease of use. It's neat and I admire that. Insight is a new kid on the block and has the look and feel of a startup product: bold, different, innovative, cool even. Not exactly punk - Andrew and Sarah are too polite for that - but if Insight deserves a musical analogy it'd be Bowie circa 1972. I don't give that comparison lightly, believe me.

Right, that's enough gushing. You get the idea. On to the functionality. What does it look like? What does it do? Some screen shots are required. Data is made up, by the way. No pupils were harmed in this review. 

Once you've contacted Insight and got your log in, you can access the admin page above. Here you can upload all sorts of stuff, including the all important SIMS file with all your pupils, their year groups, classes and characteristics. You can also send Insight excel files containing historical assessments, which they can import into the system and match to the relevant pupils, saving you time. I've been in schools recently where we've managed to get the system up and running in an afternoon. That's fairly amazing. Here you can also easily set up additional users, tweak the system and send secure messages to the team (see top right). 

Next up, analysis! Here's a view of a data table. Not hugely exciting and much like others, but colour coding relates to age-related expectation comparison, rather than progress, so ahead of the game in terms of the new curriculum and assessment without levels.

This table is for Y5 maths and it shows progress for each pupil per year and then gives more detail for current year. You can filter the table for particular groups, or split the table to compare groups. Average progress is provided at the bottom of the table but this is also provided in other reports. One neat feature is the ability to add in extra columns, to modify the table to meet your needs. For example you could add in end of key stage predictions, targets and an on track indicator. If you wanted these to appear by default, Insight will set that up for you.

Teacher assessments, by the way, are entered, surprise surprise, by clicking the Enter Data button. Select teacher assessments, then the year, the term and subject, and enter the data using simple dropdowns as follows:

As you start to type the level, it will filter down to available options that match, so type 1 and only L1 options will be visible. Again, a neat touch.

Now, onto some reporting tools. First up, the key groups summary report. This report provides a fantastic summary of headline data for a cohort; and is great for Governors, staff and Ofsted. As a fit for purpose test, see how quickly and easily your current tracking system provides the following information:

I've known schools spend days trying to put together a table like this from their systems. Insight provides it at the click of a button. Invaluable. And it includes RAISE-defined ability bands, too. A real bonus.

The next screen shot needs no introduction but these progress matrices are based on end of key stage 2 predictions, in this case for Y4:

The prediction uses a least squares regression model (I think that's right), and so is based on an extrapolation of actual progress to date, which will recalculate as and when new data is entered. It is not based on an assumption of progress, which some systems use.

Now onto attainment and progress overviews. These provide data, in terms of percentage below, at or above age-related level; or making less than expected, expected or better than expected progress, and present it as a simple horizontal bar chart. Segments of the bar can be clicked on to reveal the associated pupils, i.e:

This screenshot shows the attainment breakdown for Year 3 in reading, writing, maths and science at the end of the Spring 2 term. It will also allow the user to look back at the cohort's attainment at any assessment point. A really useful feature is the ability to select more than one year group and compare them. You could, for example compare the end of Y4 attainment of current Y5 against that of the current Y4, so you are comparing 2 or more cohorts at the same assessment point. Useful for school improvement evidence. Note the drill down to pupil level, by the way. Pupil names can be clicked on to access the individual pupil data. More on that in a bit.

The following image shows the progress overview report:

Here, I'm looking at the progress of Year 4 from the end of key stage 1 to summer 1 term, Year 4; and I've selected 6 points as my expected rate of progress over that period. Again, the user can drill down by clicking on the segments of the bar. A wonderfully simple yet powerful report.

And finally, ladders. I love these and encourage schools to make use of them. They show a cohort's in year progress on one page and pupils can be clicked on to reveal their progress journey:

This is a Y4 ladder with a pupil selected. You can instantly see the bands the pupil has moved up through; and again, you can drill down to pupil level here as you can elsewhere in the system. The next shot shows what happens when you click on a pupil name and access their page:

This shows the progress over the year for the pupil in maths (reading and writing are just beneath), and provides additional data in simple statements. You can access previous years' data by clicking the relevant tab, and get the entire KS1 and KS2 journey by clicking the KS1&2 tab (obvious, huh?). You can add assessments and targets using the dropdown and switch between pupils via the list on the right. Simple!

So, that pretty much wraps up this review/tour of Insight. As stated above, I'm a fan of this system, and get rather excited about it. It's simple, intuitive, backed up by first class support, and is relatively inexpensive, too. Feedback from schools is excellent and it's very Ofsted friendly. Definitely worth a closer look.

Monday, 16 June 2014

Primary accountability reforms, assessment beyond levels, and the implications for tracking pupil progress

In March 2014, the DfE published their response to the consultation on primary school assessment and accountability1. Since then, more information has been published on assessment principles and proposed headline performance measures, as well as proprietary tracking systems promoting their developments to tackle assessment without levels. Now that the dust has started to settle (a bit) after the barrage of information, and before we hit the inevitable chaos and confusion next term, I thought this would be an ideal time to attempt to summarise and discuss these important changes.

Summary of main points of primary assessment and accountability reforms

This has been much discussed in the blogosphere, but in case you missed it, the main points are as follows:

The baseline:
  • New reception baseline to measure progress from 2015 (KS1 baseline can also be used for 2022 cohort)
  • From 2016 the reception baseline will be the only measure used to assess progress in 2023
  • Schools choosing not to use approved baseline will be judged on attainment alone
  • From Sept 2016, EYFSP no longer compulsory but EYFS still statutory

Changes to KS1 and KS2 assessment:
  • KS1 teacher assessments in maths and reading informed by externally-set, internally marked tests
  • Also, an externally-set GPS test to inform TA of writing
  • KS1 expressed as a scaled score
  • End KS2 pupils will continue to sit externally-set and marked tests in maths, reading and GPS
  • Teacher assessments to continue
  • KS2 results reported as scaled score alongside average for school, LA and nationally (no decile banding)
  • New assessments to take place in summer 2016

And of course there are new floor measures, too:

Primary schools
  • Pupils make sufficient progress from reception baseline (or from KS1 baseline for 2015 cohort, if higher); or
  • 85% or more meet expected standard

Junior Schools
  • Pupils make sufficient progress from KS1; or
  • 85% or more meet expected standard

Infant Schools (not officially a floor measure)
  • Infant schools will be measured on the average progress made by pupils between the reception year and when they leave at age 7, with the reception baseline assessment and the teacher assessments in key stage 1 used to show this

It’s going to be interesting to see how many schools choose not to carry out a reception baseline and thus decide to be judged on attainment alone. There is also the issue of accuracy of the reception baseline. The STA’s May 2014 document ‘Reception Baseline: criteria for potential assessments’2 states that reception baselines that have a poor relationship with KS1 assessments will be considered for removal from the approved list. If a school uses such a baseline, which is considered inaccurate and is removed from the approved list, does this mean that the cohort will be judged on its attainment alone? Or will progress be measured from KS1 instead? and what happens if an infant school chooses not to use a reception baseline? Will there be a KS1 attainment floor measure announced at some point?

What constitutes sufficient progress is yet to be defined but with the removal of levels, and therefore the expected levels of progress measure, we can expect the floor measure to be a VA score. One interesting and notable omission from the report is the phrase ‘secondary ready’ which featured in the consultation. However, primary schools are expected to ‘equip the vast majority of their pupils for life at secondary school’ and pupils ‘are well prepared for secondary school’, so ‘secondary ready’ is there in all but name.

The DfE provided us with more information on headline performance measures in their recently published consultation3 and accompanying example dashboard4. These tell us that the proposed main headline indicators are as follows:
  • Progress: A VA score centred on 0
  • Attainment: APS pupils in Y6 assessments (where 100 is national average)
  • English & maths: % achieving the expected standard (100) in reading, writing and maths
  • High achievers: % achieving a ‘very high’ score in their Y6 assessments

It is safe to assume that the first three indicators form the floor standards due to the fact they are grouped together and highlighted, whilst the high achievers measure appears separate on the dashboard. As stated above, we were already aware that there would be a progress floor measure, and the 85% threshold has been much discussed5, the inclusion of an APS floor measure is an interesting development, especially considering the statement in paragraph 10 of current Ofsted subsidiary guidance, that APS ‘masks the detail’, especially in smaller schools.

Assessment Principles

In April 2014, The DfE published a summary of core principles6, which aim to underpin effective assessment systems in schools. Such systems should:
  • Allow meaningful tracking of pupils towards end key stage expectations in new curriculum, including regular feedback to parents
  • Provide transferable information that is both quantitative and qualitative
  • Differentiate attainment between pupils of different abilities and identify those falling behind and excelling
  • Are reliable and free from bias
  • Are closely linked to improving the quality of teaching
  • Ensure feedback contributes to improved learning and focus on specific objectives
  • Produce recordable measures that compare against expected standards and show progress over time
  • Take account of local best practice and are benchmarked against international best practice

Whilst the DfE are keen for schools to be innovative in their approach to assessment and tracking, it is clear from these core principles that any method must be underpinned by a quantitative system that allows clear and accurate measuring of achievement.

The transferability of data is perhaps the major concern and will clearly be a key issue when pupils change schools. It is therefore highly likely that certain systems will ‘win out’ and gain popularity, particularly within a local authority area. Proprietary tracking systems are now setting out their stalls with regards assessment from September 2014 onwards; and schools that have invested in these systems will no doubt be attempting to make their new curriculum fit the system on offer in the short to medium term. However, we can probably expect to see far greater turbulence in the tracking system market over the next year or two as schools trial various systems; ultimately leading to particular systems becoming widely adopted across LAs and forming near monopolies.

So what are proprietary systems offering? Without naming names, most are going down the route of measuring pupils’ attainment or ability against a definition of age-related expectation derived from the new curriculum. Some are expanding the EYFS concept, further subdividing emerging, expected and exceeding into narrower low, mid and high bands (C, B, A?). Others are assigning a value (or decimal value) based on the curriculum year that the pupil is working at, e.g. a pupil in Year 3 working securely at Year 4 level could be categorised as 4S. Similar to this, some systems may use values based on, for example reading age, which will assign an age-related code to the pupil related to their ability, not their actual age. Some systems will be offering quite a bit of freedom, allowing schools to select their assessment system from a range of options, like those listed above.

2B or not 2B?

For the next year or so it is most likely, inevitable even, that many, if not most, schools will wish to maintain the status quo, continuing with the existing system of levels, sublevels and points scores. This is understandable as schools want to wait and see what emerges, find out  what works and what doesn’t, before committing to a new approach: no one wants to invest their time and money in a ‘betamax’ tracking system.  However, there are some important points to consider if this is your approach.

Next year’s Y2 and Y6 pupils will be on the old curriculum so current system of levels, sublevels and point scores will apply. No big problem there. All other cohorts, however, will be on the new curriculum, which does not necessarily translate well to levels. Schools should therefore consider adapting or changing their tracking system to suit the new curriculum, not vice versa. Some schools may even adopt a dual approach to tracking using separate approaches for different cohorts until 2015, whilst others will attempt to convert all pupils’ levels to a new system.

On the subject of conversion, how feasible is it? As stated above, the new curriculum does not necessarily translate well into levels and sublevels. If, for example, you have a year 2 pupil awarded a L3 (3B) at the end of KS1, and 3B is supposedly the current age-related expectation at the end of Year 4, would that pupil be converted to a 4S, or given an ability age of 9, using two of the examples given above? In reality, the 4S should only be assigned to a pupil working securely within the Year 4 curriculum, having completed all or nearly all the objectives for that year. This is highly unlikely for a pupil in year 2 as they won’t have accessed it; and because of this many schools are predicting an apparent fall in ability as they convert data from old levels to a new system. It is also probable that an increasing number schools will consider having mixed-age ability classes to address the demands of new curriculum and give higher ability pupils access to more advanced topics.

So, that just about wraps up this blog. We are certainly going through a period of considerable change in education, and keeping pace with these changes is proving difficult for headteachers and others, many of whom are understandably getting anxious. The most common question I get asked these days is definitely ‘what are other schools doing when levels go?’ Many are therefore unprepared, opting for a ‘wait and see’ approach. But it is critical that schools address these changes, paying particular attention to the capabilities of incumbent tracking systems: if they are ineffective now, consider how well they’ll cope in a year when levels are history. If you have developed your own excel-based system, do you have the time and expertise to adapt it? If you struggle to get the data you need from your current system, is that going to change for the better? If you are unimpressed with the support provided by your current supplier, is it likely to improve with the increased needs of its users and demands placed on the system?

If you’ve been thinking about implementing a new system, now is a good time. You could take the opportunity to trial a new system alongside your existing one, to find which approach to assessment you prefer, or tap into the experience of local schools. Do bear in mind though that what’s right for one school may not work in another. This has always been the case but is probably even more relevant now that schools have the freedom to develop and adopt systems of assessment and tracking that suit their curriculum. There are many good tracking systems on the market: shop around, ask the right questions, take your time, but don’t brush these impending issues under the carpet.

Whatever happens, one thing’s for certain: 2015 is going to be a very interesting and challenging year.

Links to documents used in this blog: