The Cambridge Primary Review Trust

by

Compliant, collegial or Clint? What type of school leaders do we need?

In its chapter on professional leadership and workforce reform, the Cambridge Primary Review final report described a working environment in which ‘head teachers, once the undisputed and independent leaders of their schools, now operate in a culture of compliance and one that, borrowing the language of business, exhorts them by turns to be “visionary”, “invitational”, “democratic”, “strategic”, “instructional” or “transformational”’. Reflecting on the conceptual shift in primary school leadership over the last 50 years, the report highlighted the move from an absolutist, paternalistic (or even grand-paternalistic, with the head teacher leading a ‘three-generation family’) model of headship to the more collegial professional relations of recent times (Children, their World, their Education, pp 437-9).

The six years since the final report was published have seen both a continuation of these trends and the introduction of new dimensions into the school leadership landscape. The pressure on head teachers to be all things to all people has certainly not gone away – and to the list above we can now add, courtesy of Sir Michael Wilshaw, the suggestion that head teachers should be ‘bruisers’, ‘battleaxes’ and, as if that weren’t enough, ‘more like Clint Eastwood’.

The concept of compliance, though, is an interesting one. The Association of School and College Leaders (ASCL) is unequivocal in its belief that, if we are to create a truly self-improving school system, we must move away from a model based on compliance with central direction to one in which schools work together to build capacity and drive continuing improvement.

In many ways, we have made important strides over the last six years away from a compliance-based model of school leadership. The hour-by-hour, minute-by-minute direction of the school day through the primary national strategies is no more. National Curriculum levels have gone, with schools now able to assess attainment and track progress between key stages as they see fit. Ofsted is at pains to point out that its inspectors do not expect teachers to plan or mark in a particular way, or school leaders to evaluate their schools using a specific format. And schools that convert to academy status can choose to reject central prescription in more radical ways, such as moving away from the National Curriculum or adopting different term times.

Increased autonomy (along with its joined-at-the-hip bedfellow, accountability) is, we are constantly reminded, the holy grail of current education policy. Nick Gibb, minister of state for school standards, rarely misses an opportunity to extol the benefits of autonomy. He spoke at a conference earlier this year of the emergence of a system of education in England in which ‘autonomous schools are able to break free from the intellectual and bureaucratic constraints of the past, allowing school leaders to beat a new path of previously unimaginable success’. The government has, apparently, taken ‘clear and purposeful action to free heads from … meddling’, liberating them to ‘focus on what is best for raising pupil outcomes’.

The lived experience of teachers and school leaders, of course, doesn’t always match high-flown political rhetoric. Levels and the primary strategy ‘lunchboxes’ may have gone, but in their place are detailed ‘interim’ assessment frameworks, and statutory national assessments in three out of the six years of primary education.

The dangled carrot of autonomy through embracing academy status may also be something of a mirage. The National Curriculum may not be compulsory for the 20 per cent of primary schools that are now academies, but it’s a brave head teacher who strays too far from the ‘expected standards’ against which both children and schools are held to account. And some head teachers are finding they are actually more closely managed in a multi-academy trust than they ever were by their shrinking local authority. There are many good reasons to consider joining a MAT, but increased autonomy is not one of them. (See Warwick Mansell’s CPRT report, Academies: autonomy, accountability, quality and evidence, for more in-depth analysis of the dimensions and realities of academy autonomy.)

Compliance is clearly not what we should be aiming for. As one of the most often quoted phrases from the Cambridge Primary Review puts it, how can we expect children to think for themselves if their teachers simply do what they are told? And, by extension, how can we expect teachers to think for themselves if school leaders simply do what they are told?

But neither, perhaps, is autonomy all it’s cracked up to be. There is an increasing body of evidence that genuine improvement, whether at an individual school or a system level, happens when schools work together to plan learning, solve problems and create the right solutions to local needs.

In our increasingly fragmented system, the answer to tackling the culture of compliance is not a proliferation of ‘hero heads’, but school leaders coming together to shape not only their own schools, but the education system as a whole. Lone warriors need not apply.

Julie McCulloch is Primary and Governance Specialist at the Association of School and College Leaders (ASCL). She is also a member of the Board of CPRT.

Embodying the ‘collegial’ leader are the two winners of the CPRT ASCL award for evidence-informed leadership, presented at CPRT’s national conference. Find out more about Graham Chisnell and Iain Erskine here.

by

Free schools and free markets

Why isn’t policy done better in this country? We have some of the world’s best-known and most prestigious universities, churning out hundreds if not thousands of social science graduates every year, each of them educated to understand their subjects to a decent level of depth, sophistication and nuance.

And yet policymaking in England so often comes via papers which are so full of holes that I can only imagine they would be covered in the red ink of a lecturer’s corrections if produced, for example, by an undergraduate student as part of a research project. How must it feel for individuals to give up what they have learned as they progress towards influence at the heart of government?

Those were my thoughts a few months ago on surveying the atrocious white paper ‘Education Excellence Everywhere’. They surfaced again on coming across a report of a recent speech on free schools which may also have formed the basis for the government’s current super-controversial move to allow more grammar schools.

This latest document was a report of a speech in May by Nick Timothy, who at the time was the director of the free schools support group the New Schools Network but who is now Theresa May’s joint chief of staff and is widely credited with having had heavy influence over the formulation of the grammar school policy. I wrote about it in the Guardian last month.

In the speech, as reported by the Centre for the Study of Market Reform of Education (CMRE), the organisation to which it was given, Mr Timothy sets out a vision whereby free schools – the government’s name for new schools, overseen by the Department for Education – would return to their original mission.

This, the speech suggests, was not simply to provide new classroom places where demographics meant they were desperately needed. This has become the role of many recent free schools in the face of England’s surging pupil numbers. But no, the distinctive original rationale of the policy, Mr Timothy argued, was to open frees where there were sufficient places, but where the schools on offer to parents were not good enough. Mr Timothy reportedly told his invited audience:

The original logic of the free schools policy was that new schools should be set up in the communities served by failing schools: this would improve standards, give parents more choice, and allow new schools to innovate,

According to the CMRE report of his speech, he added:

The government is trying to create a market in the education system. This … is the right track for reform, but at the moment there’s a risk that we’re building in the potential for market failures too. A functioning market needs enough genuinely new entrants to challenge existing providers, enough capacity for competition to be meaningful, enough information for providers and users alike, ways of breaking up failing or monopolistic providers, and exit points for providers that aren’t doing a good enough job. The direction of travel is the right one, but there’s a lot that still needs to be done.

Underlying this talk, then, is a view of the free schools policy being used to set off market mechanisms which, it is envisaged, both help parents by responding to a demand for high quality education in the free schools which are created and spur other schools to improve for fear of failing to compete effectively for parental ‘custom’, and perhaps then having to close.

To be fair, it is an interesting model. If free schools were to work as described above, it sounds as if they would be a positive influence on the quality of English education. Wouldn’t it be great if there were this unending supply of ‘good schools’, funded by the government and set up where any group of parents wanted them? More seriously, the notion of teachers and possibly parents throwing themselves into free school projects to put their own stamp on innovative education provision also seems to me to have some face-value merit. And Mr Timothy has clearly thought through a few possible problems in the detail of how such market mechanisms need to work if they are to function effectively.

Yet the speech as reported was undermined by a basic failure to consider some of the more fundamental difficulties facing any avowedly free market approach to schools reform such as this.

The first problem is affordability. The premise of the talk was to create more school places, with the speech raising the possibility that this could include grammar schools.

Mr Timothy said:

If you can prove parental demand for your proposed school, then subject to all the other quality checks, you should be able to open it.

In the logic of this system, without this creation of new institutions, instigating surplus places in the system as a whole, the market mechanism he envisages would not work properly, since schools need to face a genuine risk of closure through failing to recruit enough pupils. And that is only possible when there are not enough pupils for the classroom spaces available in local schools.

But providing this surplus capacity – opening up more classrooms than is strictly necessary to ensure every local pupil has a classroom seat – is expensive. Keeping places empty is, rightly, a tough sell to taxpayers. Why not simply concentrate on making sure that the limited number of places available to parents are all good, it could be argued.

The extra day-to-day cost of providing unfilled places is not the only financial issue. The capital costs of opening new institutions and closing those which fail to attract pupils also seem likely to be very expensive, as experience is increasingly telling us.

A second, practical, problem is the availability of sites to allow new schools to be built. A string of investigations I have done on the proposed siting of frees in a variety of strange and often expensive locations mainly in and around London suggest this a serious issue, as is the general environmental impact of a choice policy which presumably assumes pupils are able to travel to a range of potential institutions competing for their ‘custom’.

A third problem may be the experience of pupils being taught in schools which either are on the verge of closure having been forced into a fight for scarce resources, or in new schools which are similarly faced with a struggle with their rolls. In an article last year, Fiona Millar gave a vivid example of two schools in Suffolk which were competing in what one commentator described as a ‘race to die’ and which reportedly led to a reduced curriculum and staff redundancies in one of the schools.

The experience of school closure itself can be traumatic and disruptive for the young people who must go through it. Yet these are the very ‘consumers’ which free market education reform advocates presumably want to help. The system advocated here seems to embrace school failures as part of its model with Mr Timothy’s detached insistence that ‘exit points for providers which aren’t doing a good enough job,’ are vital.

The sense of imposed market reform trampling over the history of a school and pupils’ experiences was one I felt profoundly after interviewing a group of parents and students for a feature on the closure of Woodlands comprehensive in Coventry in July.

A fourth consideration should be the likely effect on the teaching force of creating, as seems the aim, a system built on perpetual fear that institutions must improve or close.

The fifth question for all of this is what the alternatives are. To read a speech such as this is to get the sense that this rather complicated market apparatus is the only way that institutions might improve. Yet consideration of alternatives surely might prompt a different view. Given the costs of oversupply and the creation and abolition of schools, and the risk of a bad experience for pupils as some schools are deliberately rendered unviable, simply providing more government support, including leadership resources, to struggling existing institutions will strike many as a better approach. Put another way, is it better to put possibly hundreds of millions of pounds into creating more empty school places through free schools, in the hope that the market mechanisms in which Mr Timothy seems to have so much unquestioning faith might kick in, or simply to invest the cash directly in improving existing provision?

None of these issues seemed to be considered in this speech, leaving it vulnerable to accusations of a naïve pro-market fundamentalism. This is staggering given that thinking through the possible downsides as well as the potential of market mechanisms in various policy areas, including impacts on users of services and the public purse, would surely feature in any respectable undergraduate economics course.

Thinking this over took me back to a talk I gave to a group of public policy economists, many of them not working in education, earlier this year.

One offered this insight:

I’ve always been puzzled by this drive to try to impose market principles on education. There are surely some basic problems, such as the fact that competition effects don’t seem to work unproblematically, and closing schools will be difficult for pupils. Yet it seems to persist. †

Yes, indeed it does.

† – This quote is paraphrased from memory; I wasn’t taking notes.

Warwick Mansell is a freelance journalist and author of ‘Education by Numbers: the tyranny of testing’ (Methuen, 2007). Read more CPRT blogs by Warwick here.

by

Academies: statisticians need to raise their game

Two major reports on the effectiveness of the government’s central education policy – turning schools into academies, preferably in chains – have been published in the past two weeks. But do they get to the truth of the policy? Not remotely, I think. I say that even though the reports serve a useful public interest function in holding ministers to account.

The central problem with these reports is that they see the success or not of the academies scheme entirely through the lens of the test and exam results either of individual institutions, or of institutions grouped together in chains or, more loosely, in local authorities. Although this approach purports to offer an ‘objective’ insight into the quality of academies, and by extension the success of the policy itself, in fact it has some serious problems.

The methodology

The two studies I highlight here are, first, one for the Sutton Trust charity, called Chain Effects: the impact of academy chains on low-income students. This is the third in a series which seeks to gauge the success of multi-academy trusts (MATs) by the exam results of disadvantaged pupils on their books. The second, School Performance in multi-academy trusts and local authorities – 2015, is an analysis of results in academy and local authority schools published by a newly-named think tank, the Education Policy Institute (EPI).

The Sutton Trust study produces five exam result measures for 39 MATs, all using the results of each of ‘their’ disadvantaged pupils to pronounce on how well each chain does for these pupils. The EPI paper offers a verdict on the overall performance of academy chains, this one using two exam result measures for pupils which count in official DfE statistics as being educated in these chains.

Both studies, which are statistically much more impressive, say, than a DfE press release – though that may be setting the bar very low indeed – found that the chains varied considerably in terms of their ‘performance’. They therefore garnered media attention for some findings which will not have been welcomed by ministers.

The reports may also be invaluable in another sense. Ministers – and this seems likely to remain the case even with Justine Greening replacing Nicky Morgan as Secretary of State – tend to justify their academies programme largely in terms of institutional exam results. If research considers the academies project on ministers’ own terms and raises serious questions, then that is an important finding.

Problems: teaching to test and inclusion

However, there are two main problems. The first is well-known. It is simply that focusing on exam results as the sole arbiter of success may tell us how effective the institution is at concentrating on performance metrics, but not much about other aspects of education. It may encourage narrow teaching to tests.

Despite the multiple measures used, both of these reports seem to encourage one-dimensional verdicts on which are the ‘best’ academy trusts: the ones which manage to see the pupils who are included in the indicators which the research uses – in the case of the Sutton Trust research, disadvantaged pupils, and in the EPI study, pupils as a whole – achieving the best results.

Yet the reality, it seems to me, is much more complex. A prominent academy chain, which runs schools near where I live, has been known to do well in statistical assessments of its results. Yet some parents I speak to seem not to want to go near it, because of a hard-line approach to pupil discipline and a reportedly test-obsessed outlook. This may generate the results prized in studies such as these, but are these schools unequivocally better than others? I think researchers should at least acknowledge that their results may not be the final word on what counts as quality. My hunch is that these studies may be picking up on academy trusts which are more successful in managing the process of getting good results for their institutions. But is that the same as providing a generally good, all-round education for all those they might educate? The reports offer no answers because they are purely statistical exercises which do not investigate what might be driving changes in results. So we need at least to be cautious with interpretation.

This is especially the case when we move on to perhaps the less obvious concern about these studies. It is that both investigations focus entirely on results at institutional level, counting the success of schools in getting good results out of those pupils who are on their books at the time the statistical indicators are compiled. However, this ignores a potentially serious perverse incentive of England’s results-based, increasingly deregulated education system.

The studies seem entirely uncurious about what is often put to me, by observers of its effects on the ground, as a very serious risk inherent in the academies scheme as currently understood. This is that in deregulating such that each academy trust is given a degree of autonomy, coupled with the pressure on each trust to improve its results, a perverse incentive is created for trusts to become less inclusive.

In other words, they either use admissions to take on more pupils who are likely to help their results, or they try to push out students who are already on their books but less likely to help their results. This concern is referenced in the research review I carried out for CPRT. This quotes a finding from the Pearson/RSA 2013 review of academies which said: ‘Numerous submissions to the Commission suggest some academies are finding methods to select covertly’. The commission’s director was Professor Becky Francis, who is a co-author of the Sutton Trust study, so it is surprising that the latter paper did not look at changing student composition in MATs.

A statistical approach summing up the effectiveness of individual academy chains entirely through the results of individual chains without any way of checking whether they are becoming more selective does not address this issue.

I admit, here, that I have more reasons to be concerned at the secondary, rather than at the primary, level. Since 2014, I have carried out simple statistical research showing how a small minority of secondary schools have seen the number of pupils in particular year groups dropping sharply between the time they arrive in year seven and when they complete their GCSEs, in year 11.

Indeed, one of the top-performing chains in both these reports – the Harris Federation – has recently seen secondary cohort numbers dropping markedly. Harris’s 2013 GCSE year group was 12 per cent smaller than the same cohort in year 8. The 2015 Harris GCSE cohort was 8 per cent smaller than when the same cohort was in year 7. This data is publicly available yet neither report investigates shrinking cohort size. That is not to say anything untoward has gone on – Harris is also very successful in Ofsted inspections, and has said in the past that pupils have left to go to new schools, to leave the UK or to be home-educated – but it certainly would seem worth looking into.

When the Sutton Trust study mentions ‘[academy] chains that are providing transformational outcomes to their disadvantaged pupils’, its figures are based only on those actually in the chains in the immediate run-up to taking exams. Would the analysis change if it included all those who started out at the schools? We don’t know. The fact that DfE data is available suggesting major changes in pupil cohorts but it seems not to have been looked at is remarkable.

In addition, the fact that high-profile research studies purporting to show the success of organisations are not considering alternative readings of their statistics may incentivise organisations not to think about students which they may consider to be harder to educate. Results measures currently provide an incentive to push such students out.

The lack of curiosity is extra surprising, given that the issue of ‘attrition rates’ – schools losing students – has been live in the debate over the success of one of the largest charter school operators in the US, KIPP schools.

As I’ve said: I don’t think this is just a secondary school issue. It is also a potential problem for any research which seeks to judge the success of primary academies solely with reference to the test results of pupils who remain in schools at the time of calculation of ‘performance’ indicators.

For, with reference to the academies scheme in general, as a journalist delving into goings-on at ground level, I frequently come across claims of schools, for example, not being keen to portray themselves as focusing on special needs pupils – and therefore not to attract such youngsters in the first place – or even trying to ease out children who might present behavioural challenges.

These two reports paint a simple picture of ‘more effective’ and ‘less effective’ academy chains. But the reality I see, based on both published evidence and many conversations on the ground, is rather different. I see a system which incentivises leaders to focus on the need to generate results that are good for the school. But is that always in the best interests of pupils? Should a school which sees rising results, but which also seems to be trying to make itself less attractive to what might be termed harder-to-educate pupils, be seen as a success?

These are very important questions. Sadly, the reports provide no answers.

This is the latest in a series of CPRT blogs in which Warwick Mansell, Henry Stewart and others have tested the government’s academies policy, and the claims by which it is so vigorously pursued, against the evidence. Read them here, and download Warwick’s more detailed CPRT research report Academies: autonomy, accountability, quality and evidence.

Warwick has also written extensively about the side-effects of results pressures in schools, most notably in his book ‘Education by Numbers: the tyranny of testing’ (Methuen, 2007).

by

A bridge over troubled waters?

In a recent post on this blog, After primary, the ‘wasted years’?, Robin Alexander questioned some of the assumptions behind Ofsted’s recent tendency to compare secondary schools unfavourably with primary. Citing the different educational and professional cultures of the two phases, the challenges of adolescence and the narrow focus of inspections in primary schools, Robin questioned the bluntness of the instrument with which Ofsted is trying to beat secondary schools.

Whether or not we accept the picture of Key Stage 3 painted by Ofsted, however, genuine continuity of learning between primary and secondary schools is, as Robin put it, ‘a historic problem’. The Cambridge Primary Review found that ‘curriculum discontinuity and variations in teaching practice tripped pupils up while they were adjusting to the new social environment of secondary school’, recommending that ‘The sudden curricular and pedagogical changes that mark [pupils’] moves between schools and between key stage “compartments” need to be eased. Transition must become a process, rather than an event.’ (Children, their World, their Education, pp 371-2)

Historic problems are rarely solved overnight, particularly through structural changes, and we should beware politicians who claim otherwise. But might the changes to our school system heralded by the recent education White Paper, concerning though they are to many primary teachers and leaders, provide new opportunities to bridge the primary-secondary divide, and new incentives and mechanisms for schools to work together?

It’s much too soon to make any grand claims about the benefits of academisation, particularly in the primary sector (a point robustly argued by Warwick Mansell in his recent CPRT research report, Academies: autonomy, accountability, quality and evidence). Where things get more interesting, in my view, is in the emerging evidence behind the benefits of formal school partnerships (including multi-academy trusts, but also hard federations of maintained schools), and in the ways in which school leaders and teachers are working within such partnerships.

Two recent reports by the House of Commons Education Committee (a cross-party group of 11 MPs) examined in some detail the impact of strong school-to-school collaboration. The first report, School Partnerships and Cooperation, found that ‘school partnerships and cooperation have become an increasingly important part of a self-improving or school-led system’, and that ‘such collaboration has great potential to continue driving improvement to the English education system’. The report cited substantial evidence for this claim, including a report by the National College of Teaching and Leadership which found that schools in federations appeared to perform better than schools with apparently similar characteristics that had not federated. Digging into the reasons behind the apparent success of such partnerships, the report identified shared accountability as a crucial factor, concluding that ‘school partnerships with clear lines of accountability and some element of obligation are more likely to be successful in achieving gains from collaboration.’

The second report, Academies and Free Schools, set out to explore the impact of these new types of school. In common with other research in this area, it found no evidence so far that academisation in itself raises standards. What it strongly identified, though, was a relationship between school-to-school collaboration and improved outcomes, quoting evidence from the Sutton Trust on the stronger performance of academies in multi-academy trusts over standalone academies, and from Ofsted on the higher likelihood of academies in multi-academy trusts maintaining good or outstanding judgements, compared to standalone academies.

This report found that the benefits of being part of a formal group were particularly strong for primary schools, whose smaller size and greater reliance on local authority support often made standalone academy status more problematic. Again, these benefits were down to the impact of working together, rather than the effect of academisation. For primary schools, the report concluded, ‘the model of partnership … is less important than the level of commitment of the heads and teachers involved’. As one primary head quoted in the report put it, ‘We are accountable for each other, and therefore it is imperative we support each other to improve.’

Simply being part of a group, of course, makes little difference in itself. What matters is what school leaders and teachers do with that partnership. And that’s where the benefits of primary and secondary schools being part of the same structure, accountable for each other, can start to pay off. I’ve spent a lot of time recently talking to, and working with, school leaders and teachers in cross-phase multi-academy trusts and federations, and have seen some interesting practices start to develop.

One multi-academy trust, for example, is restructuring its curriculum to bring together Years 5 to 8 into a single phase. Children still move from primary to secondary school halfway through, but their learning continues seamlessly. In another group, teachers have come together to design a common approach to assessment, which throws into sharp relief any instances of Year 7 dip. In another group, Year 7 teachers ask departing Year 6s to bring their best pieces of work with them when they start secondary school, stick them in their shiny new exercise books, and look back at them to remind themselves of the standard of work of which they were capable last year. In yet another group, Year 6s all take with them to secondary school a ‘pupil passport’, designed to showcase their academic achievements, but also the broader knowledge, skills and dispositions they’ve developed during their seven years at primary school, alongside their sporting and artistic achievements. How much more helpful is that to their Year 7 teachers than simply being told they scored 98 on their SATs, and so haven’t met the expected standard?

None of these ways of working is impossible when schools are separate institutions, of course, and many primary and secondary schools already work effectively together. But a school in a multi-academy trust or federation not only has an added incentive to work more closely with its partners; it also, crucially, can use the structure of the formal partnership to develop new approaches that can genuinely transform children’s experiences, beyond its own four walls.

Primary schools are right to approach the proposals in the White Paper with caution, and not to rush into any decisions. Advice against marrying in haste is as wise as it ever was. But the opportunity for schools to harness the policy agenda to develop collaborative ways of working that could significantly benefit children is, in my view, real and exciting. It won’t close the primary-secondary divide overnight, but it might just build some bridges.

Julie McCulloch is Primary and Governance Specialist at the Association of School and College Leaders (ASCL). She is also a member of the Board of CPRT.

by

Do sponsored primaries perform better than maintained schools?

The education white paper proposes the conversion of all schools in England to academies by 2022. If this takes place, the vast majority of schools affected will be primaries. Of the 15,343 mainstream schools that are not currently academies, 13,822 (90 per cent) are primary schools.

This is a massive change in the structure of the primary sector, with very little evidence of any benefits. The Education Select Committee report Academies and free schools stated in January 2015 that ‘there is at present no convincing evidence of the impact of academy status on attainment in primary schools’.

The Department for Education has carried out very little research on the subject, as Secretary of State Nicky Morgan confirmed in a letter to NUT Deputy General Secretary Kevin Courtney in April 2016:

We have not undertaken a ‘similar schools’ analysis for primary schools as, to date, there have been a relatively small number of schools with results for more than one academic year.

This prompts the question of why all primary schools should be forced to become academies when even the government admits that it lacks evidence to support its claim that the policy will produce school improvement. In fact there is growing evidence that it may do the opposite.

The one claim for primary school standards in the white paper is that:

2015 results show that primary sponsored academies open for two years have improved their results, on average, by 10 percentage points since opening, more than double the rate of improvement in local authority maintained schools over the same period.

This is an odd statement. It is not saying that all sponsored primaries perform better than the average. It is not even saying that sponsored primaries that have been open for two years or more perform better. It is only claiming that the specific subset of sponsored primaries that have been open for two years, and no more than two years, performed well.

The statement also compares two very different sets of schools. The results of sponsored academies tend to start from a lower base and so they have more room to grow. Many primary schools already have SAT results at 80 per cent or 90 per cent and so are unlikely to grow at the same pace.

The key question is whether a sponsored academy will improve more or less than a non-academy that starts from the same point in terms of results. Despite the Secretary of State’s claim, there is now a significant amount of data to enable us to explore that question, with 416 sponsored primaries having at least two years of SAT results.

In December 2015, DfE released data showing SAT results, for 2015 and previous years, for every primary school in the country. This analysis is based on dividing sponsored academies into quintiles, or five equal groups, based on their prior year 2014 results. The growth from 2014 to 2015 has then been compared to local authority maintained schools with results in the same ranges.

The chart below is based on the new Level 4b benchmark but the results are the same if the old Level 4 benchmark is used, or if Level 5 is used. The first thing that is clear is that schools with a lower starting point in their 2014 results do indeed grow at a faster pace. Those with fewer than 41 per cent achieving a 4b in reading, writing and maths in 2014 saw their results grow on average over 15 per cent in one year. In contrast those with results between 59 per cent and 67 per cent grew at less than 5 per cent and those with results of 68 per cent or more actually saw their results, on average, fall.

However what is also clear is that, in each of the five comparisons, it was the maintained schools that grew at a faster pace. On average maintained schools increased their Level 4b SAT benchmark by 6.4 per cent more than similar sponsored academies. In a one form entry primary that is equivalent to two extra pupils achieving the benchmark.

The difference is very clear. Regression analysis shows that the data demonstrating that maintained schools perform better than similar sponsored academies is very robust, being statistically significant at the 99 per cent level.

This analysis is freely available for checking at http://bit.ly/KS2Regression.

The white paper has already come in for substantial criticism. As well as Labour and trade union opposition many Conservative MPs and local authorities have objected. Currently 84 per cent of primary schools are rated good or outstanding by Ofsted. It is not clear why the government feels the need to change a structure that seems to be working well.

Indeed the evidence indicates that the mass conversion of primary schools, as well as being extremely disruptive, could lead to a slower improvement in results. In 2014 Ofsted noted the difference between secondary and primary schools after the main period of academy conversion in the secondary sector:

Children in primary schools have a better chance than ever of attending an effective school. Eighty-two per cent of primary schools are now good or outstanding, which means that 190,000 more pupils are attending good or outstanding primary schools than last year. However, the picture is not as positive for secondary schools: only 71 per cent are good or outstanding, a figure that is no better than last year. Some 170,000 pupils are now in inadequate secondary schools compared with 100,000 two years ago. (Ofsted annual report 2014, 8)

Ofsted’s evidence on the disparity in standards between primary and secondary schools also featured in the CPRT blog on 15 April. On that basis, using secondary school results to support the drive to turn all primary schools into academies is hardly convincing.

The available evidence provides no justification for a policy of forced conversion of primary schools to academies. Indeed it suggests that it could lead to the same slowdown in improvement, with more students in schools rated ‘inadequate’, that has occurred in the secondary sector.

Henry Stewart is Co-founder of the Local Schools Network

henry@happy.co.uk

Readers may also wish to read Warwick Mansell’s recent blog on the government’s academies drive. His full-length CPRT report reviewing the evidence for the government’s structural changes will be published within the next few weeks.

by

We need to talk about structures

It has been a seductive slogan, for several reasons. But ‘standards, not structures,’ the oft-invoked rallying cry of those who want to cast themselves as fair-minded pragmatists in the now-very-inflamed academies debate, has been an error, I think. For, as has been becoming clearer in recent days, last week’s white paper spelling out the policy of forcing schools towards academy status has at last pushed serious questions about the detail of the academies policy to the fore, and we do need to talk about structures.

The phrase ‘standards, not structures’ – first made popular during Tony Blair’s first term in office – is an attempt to take what is seen as ideology out of the debate as to how state-funded schools should be run. Instead of viewing one type of organisational arrangement – local authority versus academy – as superior and then defending it to the end, the argument goes that we should be agnostic on that. Instead, we should worry only about the quality of education provided to pupils; acknowledge the obvious truth that good practice, or not, exists on either side of this ideological debate; and then move on. In terms of what kind of organisational structure we have in English education, basically we should join the majority of the public and not care: what happens in the classroom is all that matters.

As I say, this argument, set out in those terms, is very powerful. My perception is that ‘standards, not structures’ is used principally, and to a certain extent very effectively, as a weapon against ministers who have been seen to favour academies as an end in themselves. It is very difficult to argue that this is not their position, when a white paper has been published which says all schools are to be turned into academies, but when there is no clear research evidence in favour of the policy. (My last blog discussed this, and it will be set out in detail in my forthcoming research review for the Cambridge Primary Review Trust, especially in relation to primary schools.)

‘Standards, not structures’, was deployed again in Sunday’s Observer newspaper by David Blunkett, who introduced the original academies policy back in 2000. Here, in a well-argued analysis of many of the central problems of this extraordinary white paper, Blunkett said it was part of an ‘ideological agenda that put the structure of our school system before classroom standards’.

He’s right, of course. A government which really cared above all about the quality of what went on in schools, took seriously all the evidence it had and genuinely put children rather than politics first, as the white paper claims to do, would not be proceeding in this manner. Before pushing thousands of schools through a costly and energy-diverting change such as this, it would want to know for sure that improvement would follow. If you want a further insight into the fragility of the evidence base, by the way, just consider Nicky Morgan’s first response when asked about it on last Thursday’s BBC Question Time. The main piece of evidence she could muster on academy quality was a set of statistics embracing changing Ofsted grades for all schools, academies or not.

So the implementation of this policy is, of course, ideological. But that does not mean that arguments about it should stop at a consideration of supposedly ideology-neutral statistics. In fact, we do need to consider arguments away from pure ‘standards’ questions, too.

A personal view is that the obsession over, say, whether school test and exam results are better on the academy or non-academy side, or whether either is improving Ofsted results faster, though important, has obscured real debate about the detail of the really quite fundamental structural changes schools go through in moving to academy status.

And I find myself increasingly thinking about structures – is this the best way of setting up our schools system, irrespective of often small movements in data? – when fielding calls from whistleblowers as I do when writing news stories about the academies system. I would highlight a few structural issues now.

Structure of control

The academies policy, of course, originated under Labour as the suggested answer to usually long-standing problems in inner-city secondary schools. Where institutions had struggled for many years, if not decades, the thinking was that something bold and new had to be tried. The answer was to give great influence to an outside sponsor who, initially usually in response to a promise to donate £2 million, would be given effective control over the school, with only the Secretary of State, overseeing matters from Whitehall, as a democratic backstop.

This was controversial, as it took schools away from local democratic influence and gave great power to sometimes controversial individuals who might have been seen by the ministers backing the scheme to be dynamic. However, if there were worries about an over-concentration of power, they might have been viewed by ministers as a price worth paying in the hope of finally bringing about improvement.

Fast forward to 2016, though, and this, effectively, is the model being proposed for every state-funded school in England by 2022. Academy trusts can be set up with a very small number of ‘members’ – sometimes, only three – at the apex of their governance structures. They can appoint and dismiss the other governors.

It is true that academy trusts can be set up in a much more democratic manner. Yet some of the larger current multi-academy trusts clearly are run as described above, with a small number of individuals having great power. This is made possible because the essential overarching philosophy of the way they are set up has not changed from the original scheme under Labour.

This is not just an abstract debate, either, in my experience. In recent years, as a journalist contacted by people raising concerns, I’ve heard about: a prominent couple running an academy chain, who have particular views as to what should be in the curriculum, imposing that curriculum on schools despite opposition from professionally-trained teachers; an American firm which is influential in running a school ensuring that ‘its’ curriculum is taught in that school; and high remuneration packages finding their way to two individuals who are both among only three or four controlling ‘members’ of the academy chain paying their salaries. This looks to me to suggest an over-concentration of power with regard to taxpayer-funded bodies, serving many pupils.

A key structural question might, then, be: is the original architecture of academy governance, set up for the very particular circumstances of a small number of secondary schools which had struggled, now right for all English  primary and secondary schools?

Autonomy for individual schools

This is probably a key one for school governing bodies considering how to react. The white paper effectively spells the end of the settlement between local authorities and individual institutions, ironically  set up by the Conservatives in 1988, whereby autonomy was given to headteachers and governing bodies, but with the local authority influencing in the background.

Now, the favoured multi-academy trusts can run a whole chain of schools in a top-down manner if they choose. Schools contemplating joining one would be well advised to try to pin down MATs on precisely what freedoms they might be allowed if they join them.

Complaints when things go wrong

It’s a fast-solidifying view of mine that worrying about ‘standards, not structures’, is fine so long as all is well in an institution. It is when things start to go wrong that there are problems. For, over the past four years I’ve been contacted by many people concerned about various goings-on within academies. These include staff bullying, inappropriate spending, the ‘gaming’ of Ofsted inspections, pupils going missing from the system and institutionalised exam cheating.

A refrain of many of these whistleblowers has been concern as to who academies are accountable to. In theory, central government, through the Education Funding Agency and Department for Education, investigates. But we have often found that these remote Whitehall agencies, who, after all, now have thousands of institutions to oversee, are not interested. Nor, by the way, generally is the new intermediate tier of academy oversight, the Regional Schools Commissioner.  To be sure, local authorities, a natural first port of call for a whistleblower in the past, are far from perfect. Yet the ability of an individual to complain, for example, to their local councillor about a particular issue with a local authority school, will be lost in a move to an all-academy system. The general concept of an appeal to a truly local body outside of the instititution itself has fallen by the wayside. The white paper promises that local authorities will focus on protecting, for example, the needs of ‘vulnerable’ children. But without real power, how are they to do this?

These are just a few structural issues. I could mention more, such as questions about the merits of teacher pay and conditions deregulation – is it really best for the taxpayer to have a kind of ‘race to the top’ going on in terms of academy chief executive pay, with salaries in the range of £200-£400,000 now not unheard-of? – the now-well-discussed removal of parents from academy governance structures or the fact that much education law can now be formulated privately, away from the Parliamentary gaze, in the form of academy funding agreements with the Secretary of State.

The bigger issue is that all of these structural changes, which may centre on the de-democratisation and deregulation of state schooling, are important. They should not be seen as subservient to questions about often small changes in test and exam results, for example, or Ofsted outcomes. The country needs to ask itself whether these structural reforms are really in the best interest of pupils. In making this whole issue much more contentious, by proclaiming that all schools are to be forced into the status, ministers  may actually have done this debate a favour. At least now these questions might get more attention.

Warwick Mansell is a freelance journalist and author of ‘Education by Numbers: the tyranny of testing’ (Methuen, 2007). His CPRT research report on recent systemic reforms in the primary sector will be published next term. This blog, a sequel to the one posted on 4 March, was prompted by the publication on 17 March of the White Paper ‘Educational Excellence Everywhere’.

by

The end of primary education as we know it?

The government seems seriously to be considering requiring all schools – and yes, that includes all primary schools – to become academies. A white paper setting out such a change may come later this month while, as Robin Alexander pointed out last month, the government has been arguing that ‘all schools should become academies or free schools’ for more than a year.

This would be the biggest change to institutional oversight in the primary sector since the 1988 Education Reform Act, at least. It would enable the Conservative party at long last to achieve what it set out to do less successfully 25 years ago through grant maintained schools: the bypassing of local authorities.

But is it going to herald change for the better, or is it a risky break with what has been a reasonably successful post-war primary education framework in favour of something much less democratic, potentially much more commercialised, arguably much more at risk of capture by vested interests, and certainly much less stable?

A research report I have written for the Cambridge Primary Review Trust, to be published this spring, considers the academies policy in detail. Here is what I found:

Evidence that academy status has improved primary schools since the first primary academies came into existence in 2010 is patchy-going-on-non-existent.

This is perhaps the most obvious finding. The headline is that there is little firm evidence that academy status in itself has improved primary schools. This doubt about evidence perhaps unsurprisingly includes that provided by the Department for Education, which last year was taken to task by the UK Statistics Authority for presenting official test data on sponsored academies without context and was asked by the cross-party Commons Education Select Committee to stop ‘exaggerating’ academy success.

The committee itself said that, as of January 2015:

We have sought but not found convincing evidence of the impact of academy status on attainment in primary schools.

Meanwhile, eight of the nine larger academy chains which have been inspected by Ofsted have come in for criticism.

It is true that the academies policy has been credited by the Commons Education Select Committee with injecting extra dynamism into state education. But the evidence for this seems very indirect.

There is little evidence that academies have used the autonomy which the government says is the centrepiece of the policy on the things that the government usually talks about in relation to it.

Although ministers have tended to argue that academies have greater freedoms over the curriculum, teachers’ pay and conditions and the length of the school day and term, independent studies have tended to find that such theoretical liberties are often not taken up.

For example, although academies can depart from national curriculum arrangements, the existence of an assessment regime which focuses on the national curriculum and against which schools’ futures are decided severely limits their scope for manoeuvre.  Indeed, professionals surveying the current highly prescriptive stipulations of primary English assessments might greet suggestions of professional teaching freedom anywhere within the state-funded sector with a hollow laugh.

The academies policy, with the freedom it gives to multi-academy trusts to control what goes on within a chain of schools, provides no guarantee of meaningful autonomy of any kind for individual schools run by an overarching trust.

At the level of the individual school operating as one of several within a multi-academy trust (MAT), the very concept of school-centred autonomy may not exist as currently understood. Decision-making can be controlled by the MAT board, with the governors of each school having little or no power. If this as a trend continues and many or all schools enter into multi-academy trusts run in such a centralised way – and several leaders of large MATS have confirmed this is what happens in their organisations in the past year– it will be highly significant. Effectively it will bring to an end nearly 30 years of local management of schools, which, ironically, the Conservatives introduced in 1988.

To put it another way, we will have replaced a system whereby heads and governing bodies were given some freedom to manage their own affairs, but subject to oversight and influence by a locally democratic body, with one where a central body which is not subject to local democracy can run a group of schools in a top-down manner.

The academies policy does give meaningful autonomy to academy trusts, at the level of the trust, rather than to single schools within a MAT, in relation to some aspects of what they do. But the question then becomes whether such freedoms are appropriate. Such freedoms come in several categories.

Admissions. For example, the movement of many institutions from ‘community school’ status, run under the auspices of local authorities, to semi-independent academies makes each such academy trust its own admissions authority, rather than having admissions controlled by its local authority.

Yet why would a non-selective school need to control its own admissions? It seems difficult to identify any benefits, from the perspective of a system which sought to treat all local children fairly. However, the potential downsides have been powerfully voiced by, for example, the independent Pearson/RSA Academies Commission. It found that

Academies’ autonomy over admissions has attracted controversy and fuelled concerns that the growth of academies may entrench rather than mitigate social inequalities.

Governance. The academies scheme genuinely gives trusts great freedom in terms of how they set themselves up. At the top of academies’ governance structure sit ‘members’. Members have the ability to appoint and dismiss the other powerful figures within the trust’s governance structure, the trustees. Yet there can be as few as three members controlling multi-academy trusts, which in turn can run large numbers of schools. In some cases, these members seem to know each other well, sometimes as husband and wife. In others, a major figure involved in the day-to-day running of the MAT – say, the chief executive – is also a member sitting at the top of its governance structure.

This seems at odds with established good corporate governance practice and appears to create the clear risk that only a small number of people can be in effective control of large sums of public money and of important public institutions.

Finance. The large sums of taxpayers’ money now finding their way to some senior academy executives – and thus away from the classroom – combined with the payment of public funds to companies in which those governing academy trusts sometimes have an interest, raise additional concerns. Specifically, they highlight the clearly debatable freedoms which have been given to academy trusts over financial affairs.

As DfE’s Academies Financial Handbook puts it, the academies system hands trusts ‘wide discretion’ over the spending of public money, with auditors and the government then checking afterwards that this has been done correctly. In the non-academy sector, the local authority retains ultimate responsibility for spending. Readers will no doubt have views on which procedure is better, but financial goings-on in the academies sector at the very least give rise to concerns.

The system for deciding who runs academies is lacking in transparency and democracy

Decisions such as which organisation gets to take over a school under the new Regional Schools Commissioner (RSC) system are not taken in public; no detailed decision-making documents are available to taxpayers, parents, pupils or staff; there is no detail even on the rules by which such decisions are made, if they exist. Virtually no education stakeholders have any role in the decision-making process. The system is effectively controlled from Whitehall, as RSCs report ultimately to ministers. Though taking decisions with big implications for local communities, they are not subject to any local accountability.

The academies system has also introduced a controversial new feature on education’s landscape: the school takeover. This is when an outside academy trust can be brought in to take over the running of an institution, sometimes whether or not the school community – the parents, staff and pupils – want it.

Conclusion

Moving all schools to primary academy status would seem to herald a major increase in bureaucracy – and a field day for lawyers and auditors – as for thousands of institutions the legal structures on which schools are based are changed. This stands to take resources and the energy of school leaderships away from the core business of what goes on in classrooms.

It also stands to move England towards a system where the claimed greater autonomy of academies over, for example, the curriculum – on which the teaching profession has real expertise –  often seems more rhetorical than real. But it would hand great power to academy trusts over areas such as admissions, where gameplaying in search of a ‘better’ intake seems a clear risk, and finance, where the misuse of public funds is an obvious worry.

It would move the management oversight of schools from governing bodies and local authorities  rooted in their local communities to potentially very small groups of individuals, sometimes friends of each other, supervised essentially from Whitehall. It would allow the writing-out of most parents and taxpayers from meaningful influence and information in relation to major decisions on the future of state-funded schools, including a potential takeover by an outside organisation.

Faced with all of these considerations, one more is worth bearing in mind. As shown by the extensive evidence presented in the final report of the Cambridge Primary Review, considering as it did the system before primary academies:

Primary schools appear to be under intense pressure but are in good heart. They are highly valued by children and parents and in general are doing a good job … [They] may be the one point of stability and positive values in a world where everything else is changing and uncertain. (p 488)

That being so, and taking into account the less attractive aspects of what is being proposed, one question begs itself in relation to this possible change.

It consists of five words: why would you do it?

Warwick Mansell is a freelance journalist and author of ‘Education by Numbers: the tyranny of testing’ (Methuen, 2007). His CPRT research report on recent systemic reforms in the primary sector will be published in the spring.

by Robin Alexander

An ideological step too far

Secretary of State Nicky Morgan is reportedly looking to recruit the next head of Ofsted from the United States.

Even if she were to locate, with due objectivity and rigour (words much used by ministers but seldom exemplified in their actions), a variety of American educators with the requisite expertise and professional standing, her quest would be perplexing. For it would signal that no home-grown British talent can match that imported from an education system which reflects a national culture very different from ours, is mired in controversy, and, though it has individual teachers, schools and school districts of matchless quality, performs as a system below the UK on international measures of pupil achievement.

But that is not all. A check on the touted names makes it clear that the search is less about talent than ideology.  The reputation of every US candidate in which the Secretary of State is said to be interested rests on their messianic zeal for the universalisation of charter schools (the US model for England’s academies), against public schools (the equivalent of our LA-maintained schools), and against the teaching unions.  This, then, is the mission that the government wants the new Chief Inspector to serve.

Too bad that the majority of England’s primary schools are not, or not yet, academies. Too bad that Ofsted, according to its website, is supposed to be ‘independent and impartial’; and that Her Majesty’s Chief Inspector of Education, Children’s Services and Skills is required to report to Parliament, not to the political party in power; and that he/she must do so without fear or favour, judging the performance of all schools, whether maintained or academies, not by their legal status or political allegiance but by the standards they achieve and the way they are run.  Too bad that on the question of the relative efficacy of academies and maintained schools the jury is still out, though Ofsted reports that while some truly outstanding schools are academies, many are not. And too bad that the teaching unions are legally-constituted organisations that every teacher has a right to join and that, by the way, they have an excellent track record in assembling reliable evidence on what works and what does not.

When we consider the paragons across the pond who are reportedly being considered or wooed in Morgan’s search for Michael Wilshaw’s successor, mere ideology descends into dangerous folly. One of them runs a charter school chain in which the brutal treatment of young children in the name of standards has been captured on a video that has gone viral. Another leads a business, recently sold by the Murdoch empire (yes, he’s there too), that having failed to generate profits in digital education is now trying to make money from core curriculum and testing. A third is the union-bashing founder of a charter school chain that has received millions of dollars from right-wing foundations and individuals but whose dubious classroom practices have been exposed not just as morally unacceptable but, in terms of standards, educationally ineffective. A fourth, yet again a charter chain leader, has published a proselytising set text for the chain’s teachers tagged ‘the Bible of pedagogy for no-excuses charter schools’ that, according to critics, makes teaching uniform, shallow, simplistic and test-obsessed. Finally, the most prominent member of the group has been feted by American and British politicians alike for ostensibly turning round one of America’s biggest urban school systems by closing schools in the teeth of parental protest, imposing a narrow curriculum and high stakes tests, and making teacher tenure dependent on student scores; yet after eight years, fewer than a quarter of the system’s students have reached the ‘expected standard’ in literacy and numeracy.

As head of Ofsted, every one of these would be a disaster. As for the US charter school movement for which such heroic individuals serve as models and cheerleaders, we would do well to pay less attention to ministerial hype and more to the evidence. In England we are familiar with occasional tales of financial irregularity and faltering accountability, and of DfE using Ofsted inspections to bludgeon academy-light communities into submission. But this is as nothing compared with the widely-documented American experience of lies, fraud, corruption, rigged student enrolment, random teacher hiring and firing and student misery in some US school districts and charters, all of which is generating growing parental and community opposition.  Witness the Alliance to Reclaim Our Schools and this week’s nationwide ‘walk-in’ in defence of  public education.  Yet the culture that American parents, teachers, children and communities are combining to resist is the one the UK government wishes, through Ofsted, to impose. Ministers believe in homework: have they done their own?

However, as prudent fallback Nicky Morgan is said to have identified five British candidates. While these don’t hail from the wilder shores of US charter evangelism, their affiliations confirm the mission ‘to make local authorities running schools a thing of the past’ (Prime Minister Cameron last December), and, to avoid any lingering ambiguity, ‘The government believes that all schools should become academies or free schools’ (from the DfE website).

In pursuit of this agenda, the reported British candidates have immaculate academy and/or Teach First credentials (Teach First is the British teacher training cousin of the evangelistic Teach for America, like charter schools an essential part of the package of corporate reform). Most take home eye-watering salaries. All are within the inner ministerial circle of school leaders whose politically compliant views are rewarded with access, patronage, gongs, and seats on this or that DfE ‘expert group’ whose job is to dress up as independent advice what the government wishes to hear.

Home-spun this second list may be, but it is hardly likely to meet the Ofsted criterion of ‘independent and impartial.’

It should not be like this, and it does not need to be. Like the United States, England has many more outstanding schools, talented teachers and inspirational educational leaders than those few who are repeatedly praised in party conference speeches and with which ministers assiduously pack their ‘expert groups’.  The talent worthy of celebration and reward is not located exclusively in academies or Teach First any more than in individual schools it resides solely in the office of the head (for these days rank and file teachers barely merit a mention even though without their unsung dedication and skill all schools would be in special measures).

The problem with the much longer list of potential candidates for the top Ofsted post is that those who ought to be on it – and they come from maintained schools, academies and other walks of life – don’t necessarily toe the ministerial line. They are not, in Thatcher’s still resonant words, ‘one of us’. Such independent-minded and genuinely talented people may conclude from inspection or research evidence that flagship policy x, on which minister y’s reputation depends, isn’t all it is cracked up to be. They put children before their own advancement. They dare to speak truth to power.

Yet isn’t this exactly what an ‘independent and impartial’ Ofsted is required to do, and what, give or take the odd hiatus, most HM Chief Inspectors have done – so far? And isn’t it exactly what a genuine democracy needs in order that well-founded policies gain a hearing, ill-founded policies are abandoned before they do lasting damage, and the education system is ‘reformed’ in the ameliorative sense rather than merely reorganised as part of the latest ministerial vanity project?

But no, for by politicising public education to the extent heralded by the 1988 Education ‘Reform’ Act and entrenched ever more deeply by each successive government since then, ministers are signalling that power matters more than improvement, compliance more than honesty, dogma more than reasoned argument; and that in the battle between ideology and evidence – a battle in which the Cambridge Primary Review and CPRT have been strenuously engaged for the past ten years, often to their cost – ideology trumps every time. The government’s attempt to ‘fix’ the agenda of England’s independent inspectorate by appointing one of its own persuasion as chief inspector is not just an ideological step too far. It is an indefensible abuse of political power.

Talking of Trump, is he on Nicky Morgan’s bucket list too?  Go on, Secretary of State – in for a penny, in for a trillion dollars.

www.robinalexander.org.uk

If you would like to learn more about educational ‘reform’ in the United States, try the blogs of Diane Ravitch  and Gene Glass, and recent books by Ravitch and Berliner and Glass. For a catalogue of US charter school irregularity see Charter School Scandals.  For Jeff Bryant’s reflections on this week’s ‘walk-ins’ in support of US public schooling, click here.

by

Test of truth

Are ministers continuing to misuse data in promoting their favoured policies?

Nick Gibb, the schools minister, seems not to have taken on board the implications of a recent letter from the statistics watchdog, stemming from one of my previous CPRT blogs, about how primary schools’ test results should be interpreted and presented.

When the provisional 2015 Key Stage 2 results for England were published last week, Mr Gibb was quoted in the DfE’s press release celebrating big gains overall in average results since Labour left office in 2010. The minister also highlighted, again, the performance of academies, and in particular that of sponsored academies – typically struggling schools whose management is transferred to an outside body which signs a contract with the Secretary of State – as improving faster than the national average.

However, in doing so he ignored a warning from the UK Statistics Authority (UKSA) about over-interpretation of data. The DfE release also seemed to be heavily skewed in favour of a particular narrative, when, as I suggest below, other interpretations are available. And the national data themselves seem to beg questions about what, in reality, has driven the big recent jumps in pupil performance.

That UKSA intervention was prompted after I wrote my CPRT blog in February and followed this up with one for NAHT which argued that seemingly big improvements in sponsored academy KS2 results last year may have been nothing to do with academy status. Rather, I argued, they seemed to follow a national trend, whereby schools of all types with low statistical starting points had improved faster than the national average.

I wrote to the UKSA citing the two blogs and arguing that a DfE statistical release published in December 2014, on which ministers had relied to support their academies policy, should have investigated whether improvements in sponsored academy results came not as a result of the schools’ governance structures, but simply reflected a broader statistical trend for all types of schools.

Ed Humpherson, UKSA director general for regulation, wrote to DfE in July to suggest that while ministers were entitled to use the contents of DfE’s December 2014 statistical release when they commented on the academies policy, the paper itself should have made clear that ‘the differences in the rates of improvement [of academies versus other schools] were not necessarily caused by school type.’ He also recommended that future statistical publications should see DfE ‘commenting on limitations’ when interpreting these statistics, in order to ‘make it clearer to Ministers and to other users that the statistics could not be used to infer a causal link between school type and either attainment or rates of improvement.’

Last week came the first test of how DfE and ministers would react to this advice, with the first statistical publication revealing this year’s KS2 results, and the accompanying press release. Mr Humpherson’s warning seems to have been taken on board to some extent in the DfE statistical release, but – perhaps unsurprisingly – not at all by the minister.

The new DfE statistical release  has a section on academy performance, as was the case last year. Again, it notes how sponsored academies improved faster than the average for all schools. This time, though, it says  that when interpreting differential rates of improvement between types of school ‘it should be noted that the extent to which a school improves is related to a range of factors. Schools with the lowest previous outcomes tend to see the largest improvements…’

For me, this does not go far enough in stating clearly, in line with UKSA, that differences in improvement rates between schools of different types may be nothing to do with whether the institution is an academy or not.

Yes, this extra line of interpretation is an improvement on last year, and in that sense should be welcomed. However, it appears not to have been clear enough for Mr Gibb, whose press release claims: ‘The results…show that sponsored primary academies…are improving more quickly than those run by local authorities.’ Most controversially, Mr Gibb is also quoted as saying: ‘These results vindicate our decision to expand the valuable academies programme into primary schools.’

So, Mr Gibb is inferring a causal link between school type and results, seemingly against the advice of the UKSA.

As mentioned in previous blogs, this is not a purely political or statistical debate with only abstract implications. No, this possibly erroneous and misleading interpretation is likely to have profound implications on the ground, as struggling primary schools are pushed, often controversially, towards sponsored academy status on evidential grounds which still seem dubious.

Of course it may be that this year’s sponsored academy results do not fit the statistical pattern of previous years. It may be that they have improved substantially, while other previously low-performing local authority schools have not. We will not know for sure if that is the case until all school-by-school results are published towards Christmas. But such a phenomenon seems unlikely, based on what has happened in the recent past.

We do also already have further data for 2015 which cast Mr Gibb’s pronouncements in the press release in a somewhat different light from that intended. In the DfE release, Mr Gibb talks not only of major improvements since 2010, with 90,000 more pupils achieving the expected levels in maths and literacy, but of the results in different local authority areas. The narrative with regard to the latter is almost entirely negative. In fact, throughout this release, the only messages to come through are that ministers and their policies are proving successful; that the types of schools favoured by ministers in their reforms are proving successful; and that particular local authorities – yes, that’s government, but not the national government presided over by ministers  – are underperforming and so are facing a ‘crackdown’.

Remarkably, there is no mention at all that other actors in this annual statistical drama – children, their schools and teachers, and their parents – may have played a part in improving results.

In relation to local authorities, the release features a table of ‘best performing local authority areas’ and ‘worst performing local authority areas’, but the text focuses only on the latter, with Mr Gibb promising to write to directors of LAs at the bottom of the rankings to get them to ‘explain how they intend to improve the teaching of reading and arithmetic in the primary schools under their control’.

There are several ways to unpick that last phrase, by the way. For example, do local authority directors really have much influence over teaching content? Is ‘arithmetic’ all that mathematics amounts to now? Have local authorities really ‘controlled’ schools since the 1988 Education Reform Act, introduced by the Conservatives supposedly to stop LA control happening? But we must move on.

The interesting thing is that, within these latest statistics, DfE did publish LA-by-LA figures which point to some large improvements in recent years. So, two authorities have improved their headline percentage of pupils achieving level four in reading, writing and mathematics by 12 points since 2012. In Hull, the figure rose from well below the national average, at 67 per cent, in 2012, to 79 per cent, just below the national figure. In Portsmouth, the gain was also by 12 points, from 65 to 77 per cent. Another five authorities – Redcar, Herefordshire, Suffolk, East Sussex and Hounslow – improved by at least nine percentage points across the three years. Overall, five of the top seven fastest rising authorities, on this measure, had below-average results in 2012 so have either closed the gap with the national average or have surpassed it.

Some of them, including Hull, it is true, do have a higher than average numbers of academies. Yet outside one very small authority – Rutland, where performance tends to jump around every year – the fastest-rising LA from 2014 to 2015 on this headline measure was South Tyneside, where results surged by seven percentage points. DfE data reveals that South Tyneside has only one sponsored primary academy. Meanwhile, the academy chain widely seen as the most successful in England – Ark Schools – posted average headline results which, at 72 per cent, were a point lower than the lowest-performing local authorities nationally. Will Mr Gibb now be writing to Ark?

It is possible, then, to see from the above statistics how an alternative narrative could have been crafted, perhaps based on ministerial praise for local authority areas which have risen on the Government’s chosen measures. As ever, interpretation of statistics can depend on what the interpreter chooses not to highlight.

One final set of questions present themselves from the press release’s statistics. What do the last few years of generally improving national data actually mean?

Of course, the implications of the press release, as voiced by Mr Gibb, are clear. Results have improved strongly since 2010. This shows, said Mr Gibb, that ‘the government is delivering on its one nation vision for education’ and that ministerial policies are paying off. The national data behind this claim show that the proportion of pupils achieving the expected level 4 in all of reading, writing and maths rose from 62 per cent in 2009 to 80 per cent this year.

But to repeat: why has this happened? I’m not convinced that any of the three policies listed in the DfE press release – introducing higher floor targets, banning calculators from maths tests and introducing a spelling, punctuation and grammar test – have been entirely behind it.

And perhaps the most obvious change that a government can make to teaching and learning – the introduction of a new national curriculum – cannot have contributed here as none of the pupils taking the 2015 tests have experienced the national curriculum introduced by the previous government.

So it is a bit of a mystery. Perhaps readers of this blog can explain why the figures have jumped. I am certainly curious about them, and would like to investigate further. For if anything is to be underlined from recent ministerial interpretations of figures, it is the need continually to ask questions.

Warwick Mansell, one of CPRT’s regular bloggers, is a freelance journalist and author of ‘Education by Numbers: the tyranny of testing’ (Methuen, 2007).

This is not the first time that our bloggers have had cause to challenge the government’s use of evidence. Click here for further comment.

For other blogs by Warwick Mansell click here and/or download CPRT’s book Primary Colours.

by

How well are primary academies doing? And how well is DfE doing with the evidence?

Has DfE, including its supposedly public-minded official statisticians, been misusing data in its drive to force on primary schools its favoured policy of academy status?

The question arises since I performed an analysis that seems to raise serious difficulties about a key statistic used by a minister to defend the academies scheme.

On February 2nd, education minister Nick Gibb was confronted on BBC Radio 4’s Today programme with the findings of a report by the cross-party House of Commons Education Select Committee. The committee, following an inquiry on academies and free schools, had concluded the previous week: ‘We have sought but not found convincing evidence of the impact of academy status on attainment in primary schools.’

The minister responded that sponsored academies – generally previously struggling schools which are taken over by a ‘sponsor’ entering into a contract with the Secretary of State to run the school – were improving faster than the national average.

He said: ‘We do know sponsored academies do improve standards of education in our schools. If you look at the primary sponsored academies, they’ve seen their reading, writing and maths results improve at double the rate seen across all schools.’ He added: ‘Primary [sponsored academies]…have seen their reading, writing and maths results improve at double the rate of local authority schools.’

This seemed to mark a change of position for the DfE, which less than a year ago concluded, in its publication Academies: research priorities and questions that ‘The research evidence [is] primarily based on secondary schools and with more and more primary schools becoming academies, further evidence is needed on what drives those schools to become academies and what makes them viable and sustainable.’

So was Mr Gibb’s statement accurate? Investigating, it became clear that the source was DfE’s Statistical First Release which accompanied the publication of primary league tables on December 12th, 2014. The document is headed with the reassuring logo ‘National Statistics’. It says: ‘Attainment in sponsored academies increased by 7 percentage points between 2013 and 2014, compared to 3 percentage points in converter academies and LA maintained schools.’

This statement seemed factual enough. But doubts began to surface in my mind after digging a tiny bit further into the data.

So, the 420 sponsored academies included in the statistic did indeed improve at faster than the national rate for other schools between 2013 and 2014, rising seven percentage points in the proportion of their pupils achieving level 4 in all of reading, writing and maths, from 61 to 68 per cent. By contrast, the 13,396 non-academy (local authority) schools rose three points, from 77 to 80 per cent, while, among 1,006 converter academies – generally previously successful schools choosing to take on the status – the rise was from 80 to 83 per cent.

The immediate question, though, was whether like was really being compared with like. With both types of school, other than sponsored academies, starting with higher average scores in 2013, sponsored academies would appear to have had more room for improvement.

Another way of looking at that is to say that, clearly, the closer a school gets to 100 per cent of its children achieving level 4s in all three subjects, the less scope it has to improve on this measure; at 100 per cent, it has no scope at all.

This would seem to be a basic statistical point. Yet it was not acknowledged anywhere in this statistical release that the higher rate of progress might be at least in part a product of sponsored academies starting from a lower base. The comparison used in the release, then, might be deemed invalid. Without further information it certainly looked potentially misleading.

The fairer comparison, then, would be to look at schools with the same statistical starting points. In other words, among schools averaging 61 per cent in 2013, did sponsored academies or non-sponsored academies improve faster?

Again, there is no mention of this potential statistical comparison in the release. So I have now performed this data analysis myself, based on the DfE’s official underlying school-by-school assessment data.

Staggeringly, this seems to show that, when schools with the same starting points in 2013 are compared, sponsored academies fared worse than a comparison group of primaries in 2014.

I am not a professional statistician, and the analysis below is rudimentary. But I did it in two ways. First, I decided to look at all schools which, in 2013, had exactly 61 per cent of their pupils achieving at least level four in reading, writing and maths. Remember, this was the average figure for 2013 sponsored academies.

This yielded 113 primaries: six sponsored academies, two converter academies and 105 non-academy state schools. Among the 107 that were not sponsored academies, results improved to 70.7 per cent in 2014, a rise of 10 percentage points.

Second, I widened the comparison group to include a much larger number of primaries: those which had results, in 2013, ranging from 56 to 66 per cent. Again, I made sure that this sample, of 1,650 schools, had an average result of 61 per cent in 2013.

What was the outcome? Well, the schools which were not sponsored academies improved on average to 72 per cent. So that’s an 11 point improvement, compared to a seven point gain in sponsored academies. (The 11 point gain included figures for academy converters; removing them from the sample, non-academy maintained schools – ‘local authority schools’ in Mr Gibb’s phrase – went up 10 percentage points, which again is higher than the seven points of sponsored academies).

So my research seems to point to an opposite conclusion – sponsored academy results rising less quickly than those of a comparison group – to that of the DfE’s official statistical publication.

It would have been easy for the DfE’s professional statisticians to have published a similar assessment. But they did not. Nor did they publish any statistical caveats about the sponsored academy-to-national-average improvement comparison they chose to use.

Why does this matter? It seems to me to be very important on the ground, where I hear regularly of communities struggling with campaigns against academy status being forced on them by DfE, in the face of claims by ministers that this should be the only option for school improvement.

Indeed, DfE guidance says that, in schools deemed inadequate, ministers’ ‘expectation’ is always that they should become sponsored academies.

Last month, David Cameron went further, proposing that thousands of schools deemed by Ofsted not to be inadequate but merely to ‘require improvement’ should become sponsored academies in the event of a new Conservative government.

But if statistical evidence on an area absolutely central to the current political debate about education is being made to say the opposite of what a reasonable person might think the data actually tell us, acknowledging the need to compare like with like, we have serious problems. Is evidence being made to fit policy, rather than vice-versa?

I’d make one final point.  In a recent article, Cambridge Primary Review Trust chair Robin Alexander, wrote: ‘Deep and lasting improvements in England’s education system will be secured only when, in their discourse and their handling of evidence, policymakers practise the best that has been thought and said rather than preach it, exemplify the educated mind rather than demean it.’

It is staggering that the DfE’s statistical publication was first released without the basic caveats and checks which would be expected of statistics students completing their assignments, and was then endorsed by a minister of education. And a minister of education with an enthusiasm for mathematics, at that. What kind of an example does this set for pupils? We all deserve better.

I invited DfE to comment on the content of this blog but it did not respond. 

Warwick Mansell is a freelance journalist and author of Education by Numbers: the tyranny of testing (Methuen, 2007).

The stance of ministers and DfE towards evidence has been a constant concern of CPRT and CPR, as it must be for any organisation that cares about the probity and efficacy of education policy and seeks to generate the kind of evidence that a well-founded education system requires.  CPRT’s concern is shared by the House of Commons Education Select Committee, which in 2014 launched an online enquiry into DfE’s use of evidence. Robin Alexander’s submission to this enquiry was published as the CPRT blog on 19 December.

Contact

Cambridge Primary Review Trust - Email: administrator@cprtrust.org.uk