It is disappointing to see Fairfax has published a new round of National Standards data and advocacy on the Stuff website. Last year I wrote urging Fairfax not to continue with publishing the data but it seems they could not resist.
The Fairfax approach encourages comparison but National Standards are not nationally moderated. They are affected by far too many sources of variation to use for comparing the performance of schools. Children rated ‘at’ at one school will often be rated ‘below’ or ‘above’ at other schools.
The Ministry of Education is aware of this problem so it has been trialling a national online tool to bring more consistency to the National Standards judgements – the Progress and Consistency Tool (PaCT).
But PaCT is only due to be introduced next year. So why would Fairfax publish the existing flawed data for all schools in a way that encourages comparison? The rows of figures may be tidy but the emperor has no clothes.
My concern about PaCT is that as it attempts to solve the moderation issue it will bring its own problems in schools and classrooms. It will be a bit like how stoats and ferrets were introduced into New Zealand to control the rabbit population.
Back to National Standards, there are many other good reasons for not giving the results any publicity. The language of the National Standards, especially the ‘below’ and ‘well below’ labels, is crude and stigmatising rather than developmental.
The National Standards approach is not a ‘value-added’ one and it tends to fail children with disadvantages. These include children with various special needs, children with English as a second language, and children from deprived backgrounds.
There are also some toxic effects of the National Standards on the culture of primary schools including curriculum narrowing and a wasteful use of precious teacher time. Ironically, it is often where teachers and schools are doing their best to take the National Standards seriously that they will be most harmful.
All in all the National Standards policy has little to recommend it. There are better alternatives to getting national information about student achievement such as an approach that samples across schools. But at the moment the public is being encouraged by Fairfax to take the National Standards seriously.
Of course some will insist that ‘at the end of the day’ we must have standards in schools. My response is that in education the cry of ‘standards’ is the last refuge of the scoundrel. I want standards, you want standards, the monkeys in the zoo surely want standards!
The point is that the Key Government’s National Standards are not just standards, they are a particular and idiosyncratic assessment system. They are also complete nonsense, at least for the comparative purposes that Fairfax is promoting.
Martin Thrupp is Professor of Education at the University of Waikato firstname.lastname@example.org
I am curious whether parents and educators and academics are on the same page or not, so comments are most welcome.
Newspaper editors, you should be just as ashamed.
Publishing shonky data and interpreting it badly does nothing to help our children or improve education for anyone. The vast majority of teachers know exactly where their students are at and where they need to go next. Nothing about this data collection exercise will inform them or their practice. It won’t help principals plan where to focus their limited resources. It won’t inform children about their skills and goals.
What it will do is panic parents. They will pore over newspaper articles and try to make sense of it all, with no real understanding of what they mean, how the data was collected, and what it does or doesn’t say about their children’s schools.
Newspapers will, of course, be sensationalist in their interpretation and reporting, because that’s what makes good headlines. Never mind who it worries or panics, and never mind how accurate or responsible their reporting is. The Herald have already started it all with this shabby journalism that chooses to focus on the lowest of the three collected figures (figures that I reiterate are not even reliable) and make totally unstartling non-revelations that the English language changes and moves over time and that this affects how people write. Quelle Surprise.
And is it really news that students with English as a second language are playing catch-up? I wonder how many journalists or National Party MPs would do well in a writing test in Urdu or Dutch or Spanish?
This is why the data is so ridiculous. Without looking at an individual student’s circumstances, it’s all sound bites and spin, and it means nothing much to anyone.
John Key says “National Standards in education are a critical part of the National-led plan for securing a brighter future for New Zealand children”. How, Mr Key? Tell us how shonky data, poorly reported helps us towards a brighter future. Or is the term ‘brighter future’ just a euphemism for the equally shonky charter schools you are planning to foist on NZ?
“There is a unanimous expert opinion – even among those
championing the potential of the National Standards – that
it would be very foolish indeed to make judgments about any school
on the basis of their results.” (Stuff)
Journalists publishing poor, sensationalist reports on National Standards, and National Party for using them to promote a faulty educational ideology, shame on you all.
On 8th August, something terrible happened in New Zealand. Unfortunately most of us didn’t notice because it was during the Olympics. On that day the Government revealed its plans for how National Standards achievement data will be released and how it intends to create more consistency between schools in their National Standards judgements and reporting. There is now a ‘Public Achievement Information Plan’ with numerous steps towards ‘incrementally improving the quality of the data’.
In this paper I describe the impending release of primary school achievement data as disingenuous, destructive and deluded. By disingenuous I am not trying to engage in conspiracy theories but I do point out that the Government can’t wash its hands of media-developed league tables and that print media coverage of the potential for harm is suspiciously thin. I then discuss why the decision to release the national standards data can be expected to be destructive to many schools, communities, families and children, indeed truly devastating for some. Following that, I argue that any notion that National Standards data that will get more meaningful over time because of the various consistency measures the Government is putting into place is deluded, as is the idea that many parents will have a better choice of schools because of the release of the data. Lastly, I suggest the situation is still recoverable if there was the political will to change direction.
Is there anything disingenuous about the Government’s treatment of the release of National Standards data?
Absolutely, because back in 2009 then Minister of Education Anne Tolley said repeatedly that the Government would not be creating league tables. This is likely to be why some schools were prepared to go with the National Standards rather than resisting them as many schools did. For instance in the RAINS research I am doing, the principal of the school most supportive of National Standards said last year that she wouldn’t have supported them if they were about league tables and performance pay.
Then in June this year the Prime Minister and current Education Minister Hekia Parata signalled their support for league tables. The current stance is that ‘Public Achievement Information’ is not about creating National Standards league tables and strictly speaking that is true. But it is not preventing them either, rather it is facilitating the media being able to create league tables by pulling data together from New Zealand’s 2300 primary and intermediate schools and making it all available on the ‘Education Counts’ website.
It is also requiring schools to present their National Standards data in increasing standardised fashion that will make league tables easier to construct. So the branding of this exercise as ‘Public Achievement Information’ is little more than a distraction when the Government is doing all that is needed to make league tables of primary school performance except actually rank the schools.
As for the media, the print media (as a whole, there are some notable exceptions) seems reluctant to engage with the arguments against league tables, perhaps even shutting them down.
Based on overseas experience, it will create an annual spectacle that will sell a lot of newspapers. Perhaps it is this, as much as their general political stance, that explains why newspaper editors seem to prefer to engage in relatively general and abstract arguments about whether or not parents have the right to the information rather than focus on the important issue of harm, of why releasing the data is likely to be destructive.
To me the Public Achievement Information plans are the educational equivalent of introducing a nasty disease into every primary and intermediate school in the country. Actually the disease is already there but relatively contained. Now it is about to go viral and there is not a school or school community in the country that will be unscathed, although as we will see, some will be afflicted more than others, in fact for some it is likely to be fatal.
I am talking about powering up a culture of performativity within schools. What this means in this situation is that the new professionalism that can be expected to grow up in the primary sector now will be to get as many children ‘at’ or ‘above’ in the National Standards since these will become the headline statistics in the league tables of school performance that the media will pull together.
“No bad thing” you might say, but note I did not say that the new professionalism will be to “authentically teach the children so well in reading, writing and maths that they can all genuinely achieve ‘at’ or ‘above’ in the National Standards as well as carry on doing all the other things that we expect in a rich and full primary curriculum”.
No, that is difficult, indeed impossible, in most schools so there are quicker and easier methods that schools will be pushed to use to increase the proportions of children ‘at’ and ‘above’. They can be expected to include:
I don’t see the scenario I am painting being already in place in schools through the introduction of National Standards. There are some elements of it because schools have been mindful their data is going to the Ministry of Education and many have been considering the prospect of the data becoming public. But at the moment school staff still seem prepared to be pretty open about what is going on in their schools.
For instance in the RAINS project we have a school where a lot of children were put ‘above’ in reading but the staff there are happy to admit they believe it’s an artefact of the test that was used and that if they had used another test many fewer would have been ‘above’ and they are making that change. Another school has lots of kids put at ‘well below’ for writing and the DP there just says that is realistically where those children should be. But I think all that will change, that we will see more children being put ‘at’ or ‘above’ standard and schools becoming more insistent that the achievement is at those levels.
This is the logic of the situation.
We could say it shouldn’t happen but it will because schools are being incentivised to take these ‘shortcuts’ and play these ‘tricks’. And as professional cultures shift, it will no longer be a trick or a shortcut; it will just become what you do to try to ensure your school prospers. Professional identities will change as responding to league tables becomes the ‘bread and butter’ of the job. Schools and teachers will come to measure their worth and value by their National Standards achievement and the position of their schools in the league tables. There will be greater anxiety around National Standards performance in classrooms, staffrooms, senior management and board meetings. A kind of ‘economy’ will develop where the energies and funding in each schools is directed to doing well in the league tables and away from other areas.
These patterns are well supported in the international literature and we can see them in other education sectors here too. In the tertiary sector, where staff and institutions are responding to various performance indicators, there is a lot of trickiness. For instance I understand the Tertiary Education Union has been getting complaints from staff who are being told what their pass rates need to be while others are having their grades modified by senior managers who are chasing particular targets and performance indicators.
There is also the game of people being hired or ‘moved on’ or given new roles in order to do better in the PBRF, the Performance Based Research Fund. We can also see some of the same patterns with NCEA. One issue which has been repeatedly raised in the past is that some schools have offered more ‘easy’ unit standards than others, allowing them to look more successful when they achieve higher pass rates than schools doing more difficult ‘achievement standards’. Another concern often raised is the problem of ‘credit cleansing’, schools boosting their pass rates by cancelling the NCEA enrolments of students who are unlikely to be successful. But the difference between NCEA and the situation at primary levels is that NCEA goes across many subject areas and levels and the resulting league tables are complicated. Whereas at primary the headline statistic of the percentage ‘at’ and ‘above’ standard in just reading, writing and maths will be deceptively straightforward and just the kind of simplistic approach that will make those league tables really take off.
The destructive effects could occur in schools anywhere on the socio-economic spectrum, even in schools that are much sought after by the middle classes, because the fear of falling in the league tables will be everywhere and there is always the problem of a ‘rogue’ year group that will upset the pattern of year-on-year improvement that all schools will want to see. But it is in low socio-economic schools serving mainly the country’s more vulnerable families that the release of National Standards data will be most devastating.
These schools will be on the back foot because their children typically come in less ready for school and there are continuing issues related to poverty and transience and many also have lots of issues with children with special needs and English Language Learners.
There will be a regional and national comparison and based on present practices with the NCEA the media will probably try to contextualise by decile but that crude comparison will just makes things worse in some ways. So I’m expecting intensified local hierarchies of high performing ‘star’ schools and others demonized as ‘failing’ with these hierarchies mainly reflecting socio-economic and other contextual differences. The ‘star’ schools will find it even easier to recruit new staff, whereas a low position on the league tables will tend to further discourage applicants for lower socio-economic schools. Wider criteria for choosing schools will become increasing ignored. League tables will also affect teacher expectations. Frustration about ‘underachieving’ children will translate to them even if nothing is said.
Unfortunately there is a lot of evidence that being told you or your school is not up to standard undermines students’ identities as learners because these are constructed through assessment processes.
Wider changes can be expected to occur as well. Numeracy and literacy will come to further dominate teacher education as providers respond to the demands from schools. This will be another pressure changing what it means to be a teacher. Schools will encourage parents to seek and pay for outside tutoring for their children: we are already seeing some tutoring services being explicitly targeted at children who have been assessed as ‘below National Standards’). The league tables will inevitably become the focus of target-setting and other policy and political commentary, an easy thing to focus on rather than digging into what is actually happening behind the figures. And of course the new charter schools will be waiting in the wings wherever existing provision is deemed to be failing.
At the moment schools’ approaches to the National Standards are all over the place. The RAINS research illustrates that this is because they involve different trajectories that reflect school-specific contexts including student intakes and the history of assessment and curriculum development in each school. The Government clearly knows this variability is a problem too as it is now using the various means at its disposal to try to create more consistency between schools in both judgements and reporting. So they are introducing and making compulsory the Progress and Consistency Tool, an online platform that will help teachers to line up different kinds of assessment tools and other data and create an OTJ. They are also tightening up reporting requirements so that everyone provides data in the same format, not in different ways as they did this year.
Now the Minister’s expectation is that through these measures the data “will get better and better and be more useful over time”. Becoming more ‘meaningful’ is the expression often used. But of course at the same time the Government is taking these steps, it is creating more incentive for schools to be gaming the system. So talk of the data becoming meaningful is nonsense, it may well become less meaningful in real terms. The problem for the Government is that while the PACT tool might remove the benefits to be gained by choosing one assessment tool over another, it won’t be able to be able to control all the variation in assessment processes, the turning away of students, the educational triage and the random instances of obvious cheating. So long as the incentive is there, every time a loophole is closed with new directives, the people out in schools will find another one. And if the Government becomes more serious about policing schools’ processes, the costs of trying to ensure compliance will get very high.
Also deluded is any notion that school choices will be improved for most people. On the one hand the release of data will encourage consumerist behaviour by parents and yet the system will be unable to deliver to most of them because they won’t be able to get their children into the top schools in the league tables. So there will be many needlessly dissatisfied parents who will have been encouraged to exit their local school whereas had they engaged with it more positively, would probably find it works just fine for their child. It is also worth recalling that the data is only in reading, writing and maths and is already out of date by the time it gets to parents. Someone using the data to choose a school is actually interested in a prediction of future achievement but in practice there is little certainty because their child will typically go through with a different student cohort, quite possibly with different teachers or board members, and maybe even a different principal.
I have argued that the impending release of primary school achievement data is disingenuous and will prove destructive and deluded.
When it comes to ‘meaningful’ data, the emperor will have no clothes because of all that will be going on behind the scenes.
So where to now?
We are told there is no alternative to releasing the data because of the Official Information Act and the Ombudsman’s recent ruling would seem to confirm this. So this year’s ‘ropey’ data will have to be published although anyone who wants to take it seriously should be laughed off the planet. But then the problem should be fixed by removing the reporting requirements around the National Standards or, better still, getting rid of National Standards altogether and letting schools just work with the underlying curriculum levels and assessment tools. In my view the situation without repeated public release of the data would still be recoverable. I’m basing this on the preference to be honest and frank that I see in the RAINS schools.
But the outlook for the culture of New Zealand primary and intermediate schools without shutting down the release of data is bleak.
What the politicians and the general public need to grapple with is the paradox that the more high stakes pressure is placed on teachers, the less authentic their teaching will become and that there is no easy way to get around this problem.
Martin Thrupp is Professor of Education at the University of Waikato.
In this article, much emphasis and layout is my own. You can see the original article here.
Instinctively, people like the idea of being able to see results, measure things, to compare. We like to know where we stand. We like to know who’s doing well and who might need a kick in the pants. That’s human nature – we’re all about categorising and judging. That’s not unreasonable.
But those instinctive behaviours fall down when we don’t understand what we are measuring, why we are measuring, how we are measuring, or what the measurements will be constructively used for.
Simply weighing the pig does not make it more tasty.
So here’s the problem. Most parents want to know how their kids are doing and whether the school they are at is giving them the best possible education. National Standards capitalise on that desire and promise to keep parents more fully informed. But is that really true? Do National Standards really help Mrs Jo Bloggs know more about how wee Joseph Bloggs is doing?
Not a bit of it.
In order to know how Joseph is doing, Mrs Bloggs needs to talk to his teacher, keep an eye on anything she thinks he is not grasping or is talented at so that she can tell the teacher and make sure it’s been noted and catered for, she needs to check his school books and look for improvement, she needs to encourage him to think and question and learn outside of school, she needs to scrutinise his school report and ask questions of anything that isn’t clear or doesn’t make sense. She should read with him, cook with him, let him build and play and explore and experiment.
In other words, the best way for Mrs Bloggs to ensure wee Joseph is doing okay and is getting a decent education is to be involved.
If Mrs Bloggs expects data on a National Standards website to inform her about her own child’s learning, she will be sadly disappointed, not to mention a year out of date.
Mrs Bloggs also might want to ponder how National Standards will help her or Joseph if he has special needs, or English as a Second Language, or has a history of challenging behaviour. Because truth be known, those are the children who often find it hard to get into a school when the schools are so scared of being judged by test scores that they avoid the more challenging children who are likely to score low on the tests.
Because National Standards do not tell you anything about how far a child has progressed, they merely give a snapshot of that child now. According to National Standards, if your child is 12 and was reading at age 7 a year ago and is now reading at age 9.5, they are failing. They are not average or above average, so the fabulous gain they have made is made to look like a failure.
How will Joseph feel about that, when it’s published for the world to see?
If Joseph is a talented sportsman, artist, musician, leader, environmentalist, mechanic or IT guru it will not matter to National Standards. Maths and English, that’s all that counts. Sure, Joseph wants to read and write well, and wants to be able to do his maths, but I’m thinking maybe he and Mrs Bloggs won’t want his talents to be over-ridden or enthusiasm quashed if his maths and English results are lower than they would like. Focusing so heavily on two areas starts to make the rest of the curriculum look unimportant. And that’s simply not the case.
Joseph will be very disappointed when his music classes stop or his art lessons have shoddy equipment, or the school cannot fund entry to the regional sports championships this year, and he won’t understand or care that the money’s been diverted to pay for extra coaching for maths and English for borderline students. All he will know is that his talent is being left on the kerbside.
Joseph’s sister, Josephine is rather good at maths, as it happens. In fact she is above average and really has a talent for it. Well National Standards means a teacher is more likely to disproportionately use his or her time to teach (or even coach) those children who are borderline so that they meet the average score – after all jobs are on the line and the school might look bad in the tables, leading to a falling role and to concern in the community. Josephine’s glad her friends are being helped, but kind of feels maybe she should be being stretched too., and her spark is dimming as no-one has time to fan the flames with her.
So, Mrs Bloggs can see her children are average and above average in National Standards. She can see the children in the school are generally average according to National Standards. What has that told her about her children? What has it told her about the leaps their classmates might have made? And what about the children who didn’t move forward much at all but were already average – is that okay?
Mrs Bloggs is starting to think maybe, just maybe, National Standards are good only for one thing – making sensationalist headlines in newspapers. Meanwhile, she is off to read with her children, and talk to their teachers about how they are doing, and maybe look into their school books each day to see what they are up to. Mrs Bloggs thinks that is the best way to judge how well her children, their teachers and their schools are doing.
Educators and parents want their children to achieve to their potential, but National Standards and particularly the reporting of them does not progress that goal in any way whatsoever.
The data is unreliable.
The data will be used to compare schools.
The data will be used to judge how a school is performing.
The data only shows a snapshot of two curriculum areas (numeracy and literacy), ignoring all other possible areas of learning an achievement.
In small schools individual pupils can or could be identified by the data.
(** note, since writing this post, schools do not have to show data that would identify individual students, although my understanding is that this is creating other problems leading to Ministry recommending that principals rejig data to make it fit! ~ Dianne)
Also worth remembering:
Again: Educators and parents want their children to achieve to their potential, but National Standards and particularly the reporting of them does not progress that goal in any way whatsoever.
READ & WATCH MORE ON NATIONAL STANDARDS:
The main thing to consider with any new initiative is what benefit it will bring – what new or clearer information? Will it improve anything?
It seems that league tables for schools have a lot to prove in that case, as research far and wide tells us they add little to nothing, instead adding pressures and causing a divide where none is needed, and unwittingly encourage schools to become more narrow in their focus, meaning a less rich learning environment for children. It can also lead to schools cherry-picking their children – and what that does for those that are deemed not good enough to get into a school just boggles the mind.
NZEI had this to say:
NZEI Te Riu Roa National President Ian Leckie says he welcomes the Prime Minister’s acknowledgement that National Standards information is too “ropey” to give parents accurate information about schools.
“The Government would be wise to act now to stop the release of dodgy data and avoid another education botch up that ultimately will be bad for children and for schools. National Standards league tables will not give a fair and accurate assessment of a school’s performance. It would be irresponsible, knowing this, to release information that could mislead parents and wrongly label schools and their communities as “failing”.
“We would be very happy to talk to the Government about the ways in which the data could be safely used for its original purpose of supporting teaching and learning. But we are not prepared to take part in any face-saving measure that would involve public release of this unreliable data.”
“At the end of the day we are concerned about the impact on our children and their schools. A system that sets primary schools up to compete with each other rather than focuses on ensuring all schools succeed would be disastrous for New Zealand’s quality public education system.
“It is vital we maintain what makes our education system world-leading: schools and teaching professionals working collaboratively and in the best interests of all New Zealand children.”
And here’s a round up of what’s being said out there on the web:
“League tables work well in sports. The way the competition is defined means that ‘games won’ really is the dominant factor in ordering teams, it matters who is at the top, and people don’t try to use the table for inappropriate purposes such as deciding which team to support. For schools and hospitals, not so much.” Read more here
“One of our worst fears the last term of Parliament was that National Standards in primary schools would be turned into League tables by the media. Now it’s about to happen as newspapers have requested the information so they can publish it. There was never a legal way to prevent this happening so the Government should have prevented it by not pretending the assessment model was a national standard. The media will use the information they have and it will be an inaccurate description of the quality of education in any given school.” Read more here
“The Government’s mooted plan to introduce league tables to rank schools is a blunt instrument that would have no impact on educational achievement and could leave parents with a blurred picture says Labour’s Education Spokesperson Nanaia Mahuta. League tables, drawn from National Standards, publically rank schools on a few select criteria, but don’t give the full story. Without proper context and moderation some schools may be unfairly labelled as underachieving and parents will be fed misleading information.” Read more here
“National standards data provided by schools is too “ropey” to show parents how well a school is doing in reading, writing and maths and may not be released this year, the Prime Minister says. Data the Ministry of Education had received from schools was “patchy”, making it difficult to create anything coherent for parents and needed more time, John Key said at a post-Cabinet press conference yesterday.” Read more here