It is laughable that John Banks told Cabinet in July 2012 that “A strong evaluation programme will be put in place that thoroughly examines the impact and effectiveness of the first such schools. This will enable us to make informed decisions about whether or not to open further such schools in the future.”
The 3-part evaluation of charter schools has failed in key respects to deliver on what Banks promised.
First, there is absolutely no attempt in the final report to evaluate the most important outcome, which is student achievement.
Instead, we get some wishy washy statement that : “MartinJenkins has worked with the Ministry to refocus the final year of the evaluation (away from a primary focus on outcomes) because it was still too early to determine “success”: schools/kura were still becoming established, numbers of students that had received a “full dose” of the PSKH intervention were low, and efforts were ongoing by the Ministry to define and agree contracted outcomes.”
Charter schools were touted as having “freedom from constraints imposed on regular state schools in exchange for rigorous accountability for performance against agreed objectives.”
So, if these schools had agreed objectives in their contracts from the outset, where is the rigorous analysis of how they have performed? And why would the Ministry of Education still be “defining and agreeing contracted outcomes” if the schools are in their 3rd or 4th year of operation?
The real answer is simple: they have not performed as expected.
The primary content of this expensive exercise was not a strong evaluation of impact and effectiveness but instead they turned to a weak gathering of “survey” data from students and whanau.
But even this part was an embarrassment for charter school supporters.
“Low response rates to surveys and selection bias meant we were not able to examine student and whanau perspectives from all angles or across all schools/kura.”
Five schools (out of eight) were included in the surveys of students, but the responses from three of them were so low that they were excluded. So, instead, they resorted to merely including the responses from the two Villa Education Trust middle schools as a “case study”. Wow!
But the problems did not stop there.
“Limitations in the administrative data meant:
The comment about the attendance data was interesting, given David Seymour’s press release only last month that charter schools outperform state schools on attendance.
There are numerous other problems and gaps with the MartinJenkins report and they can be discussed more fully in due course.
Overall, it is clear that this exercise has simply not produced the thorough examination that was promised by John Banks, let alone enabling informed decisions before opening further such schools.
It was clear from the outset that the charter school policy was driven solely by political ideology, and Mister of Education, Chris Hipkins, was right to dismiss the evaluation report as being of no real value to policy makers.
Maybe the very poor survey response rates – even from those closely involved in these schools – send a clear message: it is time to move on.
~ Bill Courtney, Save Our Schools NZ
The cover up of the true picture of student achievement in charter schools continued today with the belated release of the second Martin Jenkins Evaluation Report.
The report, with a final publication date of 28 November 2016, was released on Friday 5 May 2017, a delay of over 5 months.
However, even now, the report contains a massive caveat in the section discussing student achievement, which indicates there are still major problems behind the scenes.
Here is the footnote set out under the Evaluation Report’s analysis of Student Achievement:
The ratings in the May 2016 advice were based on the best information available to the Ministry at that time (and are indicative of the reports that the Ministry had received from schools/kura by then). They reflect the most up-to-date information provided to the evaluation team at the time of writing this report, but are not the Ministry’s final assessments of schools’/kura performance for 2015.
Source: Ministry of Education (2016) Education Report: Partnership Schools/Kura Hourua: 2015 Quarter Four and Annual Reports, 30 May 2016
So, a formal policy evaluation signed off in November 2016, cannot go to print in May 2017 with a clear statement of exactly what represents the “Ministry’s final assessments of schools’/kura performance for 2015”?
The same problem is holding back the Minister of Education’s decision on whether or not to release the retained operational funding that is performance related, in respect of the 2015 school year. And this is now May 2017!
The major problem relates to the issue which surfaced last year, when the Ministry acknowledged that the interpretation of the secondary schools’ contract performance standards had been incorrect. As a consequence, the schools had also reported incorrectly against their contracts.
These incorrect figures had been used to determine the Ministry’s ratings in its May 2016 advice, referred to in the footnote. While the Ministry has now acknowledged that these figures are incorrect, nothing further has since been released.
The poor performance of the primary and middle schools is also evident in the Evaluation Report. Of the five primary and middle schools, which have contract targets set against National Standards, only one school, the Rise Up Academy, was assessed as having met its contract targets.
And problems are also clearly evident in the assessment of performance against the Student Engagement standards. Vanguard Military School and Middle School West Auckland performed very poorly against the standards for Stand-downs, Suspensions, Exclusions and Expulsions.
Overall, the main takeaway from the Evaluation Report is a fairly damning indictment of performance to date.
But the continued cover up of the true picture should not be tolerated any longer.
~ Bill Courtney
For a Minister so obsessed with data and, in particular, the sharing of data, it is interesting how little we know about charter schools.
Bill Courtney writes:
The game of delaying the release of a vast range of information on the charter schools continues.
The Ministry has promised to release a lot of material, including the formal evaluation of 2015 student achievement, in “April” but has refused to state exactly when. They also need to release all of the 2016 quarterly reports, the 2016 contract variations and the second “annual” installment of the Martin Jenkins evaluation of the charter school initative.
In short, lots of information is being withheld for no apparent reason.
When it is finally released, we will go through it and post our thoughts on what it reveals.
In the meantime, propaganda and marketing material fills the void.
The evaluation, carried out by consultancy firm Martin Jenkins and released publicly yesterday, was a narrow study of three of the first five charter schools, which are part of the National Government’s charter schools experiment.
“The report was meant to focus on innovation but the only educational ‘innovation’ clearly identified is that smaller class sizes work better,” says Green Party education spokesperson Catherine Delahunty.
“Given that we knew this already, it’s even more apparent that National’s charter schools experiment is a giant waste of money – money which would have been better spent reducing class sizes for all children in state schools.
“The report also identifies ‘risk taking’ as a strength of charter schools, even though we know the risks taken in allowing the Whangaruru charter school were disastrous for the children enrolled there.
“The worst thing about this evaluation is its failure to carry out any real comparison between state schools which innovate and charter schools, which supposedly help priority learners via innovation.
“All we know from this report is that the charter schools like the extra money from the Government, as well as their reduced accountability to the Government, and that small classes help students.
“There is also no mention of any innovation to assist students with special needs, who the Government says are a target for the charter schools model.
“There is already considerable innovation in many state schools and the Green Party’s School Hubs policy would build on that, as well as reduce inequality.
“Introducing School Hubs would mean every decile one to decile four school would have a nurse, food in schools and real community involvement – all of which will lead to a more inclusive education experience for our children,” said Ms Delahunty.