The Folly of data fascination

Release from Bill Courtney, National Chairperson of the Quality Public Education Coalition (“QPEC”):

“Not everything that can be counted counts;

and not everything that counts can be counted”

Albert Einstein

Unfortunately, neither Education Minister, Hekia Parata, nor the editors of the Fairfax Media School Report database, understand the difference between data, information and knowledge.

Raw data of the type contained in recent Ministry and Fairfax reports is almost useless to anyone seeking to choose schools (the supposed purpose of the Fairfax database) or evaluate the New Zealand education system (as with Hekia’s Report Card).

What is desperately missing in this fascination with data, are real insight and knowledge into what impacts on student achievement and what we might do about it!

Fairfax Media’s School Report database is a modern version of the school league tables that are especially common in England. Proponents believe that publishing achievement data is a valid method of holding teachers and schools accountable.

Ian Schagen, formerly an educational statistician with England’s National Foundation for Educational Research, and now on contract to the Ministry of Education, dismissed this flawed approach. “League tables based only on student achievement tell virtually nothing about how well schools are doing. The schools that come out top are those with the best intake, not those providing the best education.”

League tables, and the punitive accountability measures that often accompany them, seem to be features of those countries which have adopted the competitive model of education. This approach assumes that schools should compete against each other, with parents playing the role of “informed consumers”.

OECD analysis of the PISA assessment results reveals that various system-level design features, such as posting achievement data publicly, comparing your school with other schools and schools competing with other schools in the same area, all show extremely low correlations with actual student outcomes.

Ironically, the public posting of achievement data does not feature at all in highly successful systems overseas, such as Finland. They simply concentrate on the quality of teaching and learning taking place in their schools and do not encompass the competitive model at all!

After the publication of the first batch of National Standards data by school last year, Fairfax published an excellent op-ed by Paul Duignan, an expert in the field of “outcomes measurement” (2 October 2012).

Mr Duignan rightly noted that raw data based on outcomes tells us nothing about the quality of the school, the quality of the teaching delivered, the resources available or the support and motivation provided to students by their parents.

All of these “count” for a great deal in influencing student achievement but none are “counted” in the Fairfax league tables.

Just as worrying, Hekia Parata’s “Education Report Card” was a tangled, confused montage of no less than 60 sets of numbers, covering everything from early childhood education participation to 18 year olds’ qualifications. All sandwiched literally into a single A4 page!

Hekia Parata said that releasing data was an important part of her government’s approach to raising achievement. Ms Parata even commented that “without data you are just another person with an opinion”.

Supporters of the data-driven approach also accuse critics of being soft on assessment and accountability. But raw data tells us nothing of value about how well our schools have performed, given the widely varying quality of their intakes.

Students should be assessed and schools should be accountable. The questions, however, are How? and, in relation to What?

QPEC has long argued that the overseas fashion of league tables together with the media practice of naming and shaming has done nothing but harm to students and has negatively impacted on the quality and breadth of education they receive.

What would impress QPEC is an insightful piece of analysis from the Ministry of Education revealing deep knowledge about the real factors that impact on student achievement.

Sound policy development based on real evidence and research is what we need from the Minister - not more clichés.”

ENDS

© Scoop Media