Tory minister failed to mention England cheated in Pisa tests

She was referring to last week’s publication of an OECD report on international standards in educational performance among 15-year-olds – the so-called Pisa study.

Pisa stands for “Programme for International Student Assessment”. The study is conducted every three years. The latest (for 2022) appears to show a deterioration in average attainment levels in maths and science in Scotland, though the score in reading actually improved.

Who is Gillian Keegan? Well, for one thing, she is in charge of English education. Her reference to devolved Scottish education, as well as being erroneous, was pure political grandstanding.

We might remind Ms Keegan that under her watch England’s Pisa attainment scores in mathematics and reading have declined significantly since the previous 2018 report. As for science, England’s performance in 2022 was roughly equivalent to 2018 but was significantly down on that of 2015. Perhaps Ms Keegan should be doing her own job rather than trying to shift attention elsewhere.

READ MORE: Keegan blames ‘nationalist’ policies for Scotland’s falling education scores

For the record, the way the Pisa study is compiled is that pupils in selected schools sit a standard, two-hour exam. Supposedly, under the OECD rules, each national educational authority has to select the pupils sitting the exam according to fixed rules. In particular, the students should be of average academic background.

However, countries who want to cheat can fiddle their Pisa scores by putting forward pupils from schools and backgrounds that ensure a better performance. This appears to be the case in the latest results for England.

I quote from the official UK Department for Education “Pisa 2022: National Report for England”, executive summary, page eight: “It is important to keep in mind that the sample of participating pupils may not be entirely representative of all 15-year-old pupils in England … the sample for England did not meet two of the 82 Pisa technical standards … the final sample had somewhat higher academic attainment on average than the general population and a somewhat lower proportion of pupils who had been eligible for free school meals in the past six years. In other words, higher performing pupils may be over-represented …”

So Ms Keegan’s department actually cheated in the Pisa exam!

And she has the temerity to talk about Scottish education “collapsing”. As in previous Pisa cycles, the highest performing education systems tended to be in East Asia, with Singapore significantly outperforming all other education systems in all subjects. Japan, Taiwan, Macao and South Korea were also among the top performing systems.

Again, there were methodology issues involved. Singapore is a rich city state. Is it fair to compare its schools with much larger, more socially diverse countries? There is also the issue of countries deliberately fiddling their scores. There have been repeated accusations that China, Hong Kong, Macau and Argentina have selected Pisa samples from only the best or above-average schools. Pisa is very political.

My view is that the media and politicians are too hung up on the Pisa exercise. It has become a political football here and everywhere. According to the OECD, in the latest test round 12 participating countries failed to meet reporting standards. Call that accident or call that political duplicity. Besides, the latest Pisa scores are completely distorted by the pandemic. Ostensibly, Pisa learning outcomes have deteriorated in an exceptional number of countries. Mathematical literacy fell in as many as 41 comparable countries, of which 35 are OECD countries. Has education collapsed everywhere instantaneously, Ms Keegan?

BY the way, if you fancy doing the Pisa tests yourself, you’ll find them on the OECD website. You’ll discover that they have something of a cultural bias which may disadvantage students in some parts of the world. India pulled out of the Pisa assessment programme precisely because it thought this cultural bias was too strongly in favour of pupils in highly industrialised countries.

Back to Scotland: How would we test if education here is failing? One way might be to look at the numbers going on from school to higher education. If standards are collapsing, we would expect to see the number of pupils able to proceed to higher and further education (universities and college courses) start to decline. They haven’t. In 2021/22 (the latest stats published) there were more students on HE courses in Scotland than at any point in the last decade, at 346,280. This represented a year-on-year increase of 13,465 students, or +4%.

Critics are likely to argue that there is a high drop-out rate. But more students in Scotland qualified from HE courses in 2021/22 than at any point in the previous decade, with 117,195 students receiving an award. And many of these qualifications were achieved despite the impact of the pandemic (data from the Scottish Funding Council.) In particular, we should note that the number of students on taught postgraduate courses has doubled over the last decade, from 41,530 to an impressive 83,065 last year. True, that includes foreign postgrads. But the fact Scottish education attracts so many postgrad students from abroad is hardly a sign our education system has collapsed. Think on, Ms Keegan.

This is not to say there aren’t problems with Scottish education, not least with the so-called Curriculum for Excellence (CfE) experiment, introduced into Scottish schools in 2010-11. This was a very laudable attempt to shift the focus of schooling from rote acquisition of knowledge towards helping pupils “learn to learn” and towards encouraging wellbeing and self-confidence.

For many pupils, certainly those of less academic ability or interest, the CfE approach was revolutionary. We should note there are many countries – particularly in Asia – where rigorous rote learning produces high Pisa scores but in which creativity and independent-thinking are underdeveloped as result.

The evidence to date suggests that CfE has not dented university access or attainment. Has it reduced competence for non-academic pupils in basic skills, such as reading and maths? Since the last Pisa assessment in 2018, outcomes in Scotland have certainly declined. In maths, the score it is down by the equivalent of nearly a whole year of schooling. A typical 15-year-old now is doing no better than a 14-year-old in 2018.

There are two responses. One is that the 2022 results are so heavily biased by the impact of the lockdown as to be highly unreliable. And second, if learning skills, creativity and self-confidence – which are not measured in the Pisa study – have improved in Scotland, the long-term results may be positive rather than negative.

That said, I think there is a consensus that the CfE experiment needs to be reassessed urgently. For one thing, implementation of the CfE approach is inconsistent across schools and local authorities due to lack of resources and staffing. It is also the case that core reading and maths skills need to be reinforced at primary school level. Again, there are resource implications.

Scrapping CfE overnight and subjecting Scottish education to yet more trauma is perhaps not the right way to go. But the Scottish Government should not retreat into being defensive. We need a serious revaluation of the CfE to date.