27, March, 2014

Evidence Counts, but what counts as evidence ?

Evidence Counts, but what counts as evidence ? In this blog , Dr Andrew Morris , Chair of EMC Trustees, charts how a prolonged debate around research in public life seems to be moving on.

The arrival of the world’s first Education Media Centre marks a significant moment for the evidence movement. The media are buying into it, researchers are contributing to it and evidence-based stories are resulting from it. But the EMC is more than just a useful tool. Along with parallel initiatives it is a sign of a significant change in public affairs.

Evidence is on the agenda at last; whether it’s childcare or flooding, neuroscience or university fees, evidence now gets a look in – on the news, in documentaries and at public enquiries. Public bodies everywhere are eager to declare their “evidence-based” credentials. This change has significance for public life and it’s been a while coming.

In 1995 the OECD published a powerful report on the economic and social urgency of building education systems around evidence rather than opinion. In 1998 the incoming government commissioned the seminal Hillage report into the quality and impact of  research into schools and effectively raised consciousness about evidence in education through a host of small initiatives linking the work of researchers and leaders of public services.

These efforts and many others were highly contentious, involving debate and arguments, sometimes acrimonious. As the evidence movement in healthcare had witnessed twenty years earlier, resistance was strong. Very little of direct value to teachers, students, parents or employers resulted from these debates. Yet, with hindsight, we can see awareness was heightened and attitudes changed.

Today the academic community is incentivised to work with the real world and the Cabinet Office has set up What Works Centres, focusing on the evidence base. Major organisations like the Education Endowment Foundation and NFER are synthesising high quality evidence in user friendly formats and university institutes and centres at Durham, York and elsewhere are creating practical tools such as the EEF Toolkit and Best Evidence Encyclopedia to get it into use.

A host of new networks and structures have sprung into existence to support the use of high quality evidence by practitioners, policymakers and journalists. So the new and long overdue prominence of evidence is to be celebrated.

But as one battle finishes, another begins. The disputed territory now is not so much “Should we ask for evidence in education?” but instead, “What kinds of evidence should we ask for?” For example, what does “evidence-based’ actually mean? Does based imply we will ground our practices and policies scientifically in evidence? Will we start to focus on how to engineer change as much as on studying how things should be?  We’ll also need to think carefully about the type of evidence we have in mind? Do we look to science or to the law – criminal trial or drug trial? We must also anticipate potential misuse of evidence: cherry-picking studies to suit your cause, and the retrospective game of policy-based evidence.

The debate must move on to these deeper questions of the nature of evidence. In science as in the law, evidence is mostly incomplete and often contradictory. The evidence-base is an assembly of evidence culled from numerous studies using differing methods in differing contexts. Sorting out which pieces can be relied upon and putting together messages consistent with all of them is no mean task. Turning these into tools and practices that actually improve learning is tougher still.

So it’s clear we are entering a phase of new challenges. What makes evidence sound – a hierarchy of methods; replication of studies; professional consensus or peer review? How can frontline practitioners use evidence if it’s inconclusive or, more likely, missing?

There are no easy answers, but facing these questions is a welcome sign of maturation. My hope is that it signals a move from abstract and often sterile disputes about epistemology, poor practice and failed policies towards more practical and collective thinking about the issues that matter most to teachers and learners : how to create knowledge that’s useful,needed and leads to better education and training.

Education Media Centre is a registered charity No. 1153567

All worldwide rights reserved for all content on this site.
Copyright © Education Media Centre 2013

learnedly broadcast by