“PISA shock” is the term that has been coined for the sense of political crisis and knee-jerk policy reaction that typically occurs when a country drops in the Organisation for Economic Co-operation and Development’s international education rankings.
Almost 100 education experts, including several of us from New Zealand, recently sent an open letter to the OECD’s chief education spokesperson, Andreas Schleicher, pointing out that the Programme of International Student Assessment (PISA) rankings have become more of a problem than a solution.
In many ways the concern of New Zealand educators is also around the wider influence of the OECD education programme on New Zealand educational politics and policy – perhaps as much “OECD hangover” as “PISA shock”.
New Zealand’s government led by prime minister John Key often draws on the authority of the OECD to endorse its policy direction, using both PISA findings and the related arguments of Schleicher. New Zealand’s minister of education Hekia Parata regularly quotes Schleicher, saying, “Without data, you are just another person with an opinion.”
But Schleicher is not close enough to the New Zealand context to correct any misuse of the OECD’s results by the Key government, and this causes problems. One example has involved the impact of poverty on student achievement.
Shortly after the latest PISA results came out in 2013, Parata started to say that New Zealand’s PISA results showed that socio-economic status accounted for only 18% of student achievement. This was surprising, to say the least, when a powerful relationship between social class and student achievement has been a theme of international research in the sociology of education for more than 50 years.
Further investigation revealed that the 18% claim was based only on PISA’s narrow definition of family socio-economic influence. Using PISA’s wider criteria that include neighbourhood and school socio-economic factors, about 78% of New Zealand’s latest results became explained by socio-economic conditions. But it is worth noting that some academics, such as Harvey Goldstein at Bristol, point out that PISA is not a long-term study, and so can’t estimate factors like this.
But downplaying the impact of poverty in order to emphasise the responsibility of teachers to raise achievement has been a regular strategy of the Key government. And when faced with the corrected figure by opposition parties in parliament in January, the prime minister still fell back on the authority of Schleicher to argue for the greater importance of teachers and schools.
Adding insult to injury, Schleicher himself started arguing that “poverty isn’t destiny” and arrived in New Zealand for the International Summit on the Teaching Profession in March stressing the power of high expectations in the face of social contexts.
Some of the points Schleicher has been making might be useful if the arguments were employed carefully. Unfortunately, in the national politics of New Zealand – and probably in many other countries – any such subtleties are quickly lost. Instead the OECD/Schleicher arguments become fertile ground for the politics of blaming teachers for the underperformance of students from poor backgrounds.
The OECD hangover in New Zealand goes far beyond PISA. In January, the Key government’s latest school policy proposal called “Investing in Educational Success” was announced. This is intended to introduce a number of new teaching and leadership roles into New Zealand’s schools, providing extra payments for carrying out the required roles as part of a NZ$359m (£184.4m) investment plan.
By February, a four-minute video of Schleicher endorsing the policy had appeared on Parata’s National Party website. This was concerning as although the OECD tries to be non-partisan, here was Schleicher, featuring on a party-political website and endorsing the governing coalition’s announcement of new education spending in an election year.
Watching the highly scripted video clip it also becomes apparent that Schleicher was willing to endorse the new policy without entering into the controversies it would cause in New Zealand.
He leaves out how the policy was announced by the Key government after a cabinet decision, without prior consultation and only subsequent input into the detail rather than the thrust of the policy. Not mentioned are worrying shifts in the power relations between the New Zealand government and teachers and between New Zealand teachers themselves that are likely to be caused by the policy.
Also not mentioned is the involvement and reinforcement of other New Zealand education policies that have been causing concern such as the new National Standards for primary schools, as well as many practical considerations. Instead, Schleicher discusses the policy only in an abstract, non-contextualised way. As the Quality Public Education Coalition pressure group said, his endorsement evokes the “best of all worlds”.
With Schleicher’s endorsement of the policy there can be no claim of misinterpretation by the Key government. It is more that Schleicher is not being careful enough about how the OECD’s support would be used in a local setting.
Our open letter concluded by suggesting the OECD had become the “global arbiter of the means and ends of education around the world”. In New Zealand, the OECD risks becoming known as a stick to beat educators with. Its reputation is unlikely to improve until it starts genuinely listening and acting on local concerns.
Sure, the politics drive me batty, so why do I carry on teaching?
I could tell you so many stories of special, amazing, beautiful children I have taught and what they have taught me.
But this video sums it up better than anything I could write.
Make sure you watch until the end, not just for the boy’s achievement, but for the reactions of the staff and his peers.
Well done, Mushy.
My 5 year old is presently “Behind the standard” for reading (although he is doing fantastically and is building speed). He is bright, and engaged and I won’t be surprised if he zooms past the “national standards” a few years down the track.
My question is, if he does start out behind, but arrive well ahead of “national standards” will this be taken as evidence that all children learn at different rates, or will it be celebrated as a victory for “national standards”, “proof” that the standards work to lift achievement??
Your thoughts on this are welcome here or on the Facebook page…
Who is to blame that some students achieve less than others?
Is apportioning blame and pointing fingers actually helpful for anything other than head-line grabbing?
Admit it – did you click on this because of the headline, hoping for an easy answer?
Well there isn’t one. It’s a complex issue.
Why we need to consider this
If you see an easy head-line friendly fix for these issues, prepare to sound the alarm.
The truth is, until we put vote-grabbing solutions aside, try to avoid the blame game, and look for genuine research on the issue and unpolitical, unhysterical, practical, research-based solutions, we won’t get very far.
It’s particularly pertinent given some are arguing that there is a long tail of under achievement comprising predominantly Maori and Pasifika students and that schools and teachers are to blame for this. This argument is then used to promote policy changes such as the introduction of National Standards, Charter (Partnership) Schools, and soon to justify performance pay. But whether the original statement has any real basis in fact is debatable. Could other issues be at play as well as teaching?
Until we know what the real issue is, we cannot begin to find good solutions.
So let’s begin to look at what we know.
Disadvantaged from the start
An American study found that “inequalities in children’s cognitive abilities are substantial from the beginning, with disadvantaged children starting kindergarten with significantly lower cognitive skills than their more advantaged counterparts.” Students are arriving at their first place of education, kindy, already on the back foot.
The study argues that the “same disadvantaged children are then placed in low-resource schools, magnifying the initial inequality” which certainly has bearing on New Zealand schools with their un-level playing field in terms of funds.(2)(3)
The report has conclusions relevant to education policy:
This is certainly worth considering carefully.
Decile as an indicator
Do students really achieve lower results at lower deciles? And if so, then why?
Robyn Caygill & Sarah Kirkham looked at mathematics for year 5 Kiwi students, and argue that the decile of the student’s school does indeed correlate to the average level of achievement reached by students. They point out that it “is indicative of a trend demonstrating that students with lower levels of disadvantage in terms of family background and socio-economic background and living in wealthier communities have higher achievement.”(1)
They concluded that, in general, students at lower decile schools tend to have access to fewer resources, stating that “the decile of the school [students] attend, [is] indicative of the level of economic disadvantage in the community in which they live, [and] was positively related to mathematics achievement.”
Does funding also have an impact on achievement? And if it does, does that link to the socioeconomic position of the school’s community in any way?
A book just published looks an inequality in New Zealand. In a survey, it found that decile 10 schools’ total budgets averaged $8,653 per student, whereas it was $7,518 per student in the decile 1 schools. Can wealthier schools afford more teacher aides, more specialists, better resources, small class sizes and so on, all contributing to a slightly better chance for their students? If so, what should this mean for the future funding of schools in poorer socioeconomic areas?
If students achieve less because of the socioeconomic status of their family, then this surely needs to be a focus for future research and action.
Parents as a factor
A Danish study last year found that in that country, a student’s parents are a huge indicator of future achievement, being five times stronger than the effect of teachers. The report was said to”raise questions over the extent to which schools can be expected to make significant improvements to pupils’ results without the necessary backing from mothers and fathers.” It stated that“[h]alf of the variation in test scores is attributable to shared family factors, while schools only account for 10 per cent,” It went on to say that the remaining variation was down to pupils themselves. Notably, researchers said the effect of families on test scores remained the same irrespective of household income. (4)
However, after looking at the research, the headline grabbing here seems to outweigh the scope of the research, which only looked at 16-17 year old students who changed schools at that age. Another case where the headline doesn’t help us learn much at all.
Whilst I am very sure indeed that parents are a factor, this particular paper is not the one to show the link, at least not for NZ and not for primary schools.
My search for more rigorous local research continues. If you know of any, please message me below.
Caygill and Kirkham (1) also noted that for mathematics, “books in the home, items in the home, household size and mobility” were indicators of students’ maths scores.
It will surprise no teacher that the more a child moves school, the lower their achievement is.
And consider the results for books: 34% of students reported having 25 -100 books in their homes while 28% said they have 25 or fewer books in their homes. Guess which group got the higher scores?
In essence, the poorer your family is, the lower your maths score.
So we are back to the socioeconomic status issue again.
For my part, I will continue to search out research that will inform the situation and ponder what it tells us. In this I stand on the shoulders of others, as people wiser and better placed than me are out there researching.
If you are one of those people, I would love to hear from you.
One thing I will leave you with is this – beware easy solutions sold to you in spintastic headlines. They rarely tell the whole story, let alone a fair one.
Sources and further reading:
Dear Dr Sharples,
“Assumption is the mother of all f*** ups,” Travis Dane
“…a spokeswoman for the minister said a website and a booklet were part of the plan and emphasised schools would not be compared with each other in league table fashion”(1)
I’m all for parents being well informed about how their children are doing – in fact it’s absolutely necessary that the students, parents and teacher all share information and know how the student is doing, what their next steps need to be, and where strengths and weaknesses lie.
I just really don’t see how the league tables, oops, I mean Public Achievement Information, will do that.
Most parents are already mighty flustered with the education system’s double speak, getting to grips with personal goal setting, inquiry learning, thinking hats, tidy numbers, and lordie knows what other mysteries we throw at them. It must seem to them that we are conducting some kind of alchemy or learning another language, at times. And now they are going to be given a set of data that most will have no idea how to interpret, that the media will twist for a headline or two or six, and that leaves them none the wiser.
It will not – make sure you realise this – NOT – give any parent any information they didn’t already have about their own child/ren.
What it WILL do, however, is allow people to make comparisons between schools.
“The spokeswoman for the minister said achievement data would not be used by the ministry to compare schools.”(2)
But it doesn’t need to. The media will do it for them. And the whole saga will add to the damage we already have due to the decile ratings, where people judge a school’s merit by something they have fundamentally misunderstood. I only hope that at least someone takes into account where the children were when they entered a school, so that people can get an idea of the value added by that school, rather than just a snapshot of where they are now with no context. That would be something.
My fear is that this is really just another stick with which to beat schools with lower achieving pupils, without any fundamental help for those schools to change things. Nothing to change poverty, to help with adult literacy or numeracy, nothing to show parents how to learn with and teach their kids, nothing to help boost training for teachers with weak areas. No, nothing that would help.
I think those who bring in these policies should take note of something my tutor, Bob, told me at teacher training college:
You can weigh the pig as often as you want, but if you don’t adapt what and how you feed it nothing will change.