INTERVENTIONS ARE NECESSARY BUT NOT SUFFICIENT

Peter Tymms

Outline• A quick look backwards• The need for interventions• Two recent eye openers• Alternative perspectives• Conclusions– For teaching– For policy making

Historically• Education was an evidence free zone• Experience, opinion and good ideas ruled• Two example examples– Kenneth Baker– Estelle Morris

Interview of Lord Baker with Giles Dinot on 18 February 2011

• Q: One criticism was that the National Curriculum just “taught to test”; is there a problem with that?

• A: I went to a Church of England primary school during the war in Southport. I’ve still got the little report books, and there they are, marks out of 100. I was quite clever in those days. I was being tested every term. And I took it home and showed my mother and father.

Estelle Morris: Education Secretary

When in office she did not base any action on research but took considerable notice of the Daily Mail

The need for interventions• Consider

– £500 million on the National Literacy Strategy with no detectable impact

– Etc etc etc• World wide recognition of the issue

– Campbell Collaboration– USA

• What Works Clearing House• Slavin’s Best Evidence

– John Hattie– Tool kit– CEM, Durham

Meta-analysis

• Example: use of digital technology to increase literacy in middle schools. (2005)

Forest Plot

Points to note

• Overall: ES=0.49• Variation of impact of interventions• Need to know more …. • But we now have a best estimate

First eye opener

• Slavin et al 2014• Cooperative Learning in mathematics has

been shown to be effective in the USA• Tried a large RCT in England– And failed “most surprising”

• Tried again with a revitalised approach– And failed again

Second eye opening

• Lemons et al 2014• Five RCTs of the efficacy of KG Peer-Assisted

Learning on reading program. • Over 9 years involving 2,591 students.• Early results very positive and widely reported• Most recent results– Failed

• Team: “shocked and depressed”

Explanations

• Slavin et al– “Differences between traditional teaching practices in

England and in North America”– “Teaching methods proven to be effective in one culture and

system cannot be assumed to be effective in another.”• Lemons et al

– “The changed context”. The bar had been raised in KG.– “the change agent - a no-nonsense Chief Instructional Officer” – The controls were different

A Further Complication

• We live in a complex interactive world.• Even deterministic worlds are not predictable

– http://www.math24.net/double-pendulum.html

• But patterns appear (strange attractors)– https://www.youtube.com/watch?v=aAJkLh76QnM

• Error propagation is the cause and it is summed up in the Dynamic linear model.

The nature of teaching• Great teaching involves– Planning– The ability to ditch Plan A and go to Plan B– Dealing with idiosyncratic

• children • classes

• Integrating professional judgement with scientific evidence

• The bottom line– Academic progress– Positive non-cognitive progress

Some features of policy making• Making big decisions.• John Maynard Keynes– "When the facts change, I change my mind. What

do you do, sir?"• But U-turns are bad news– “The lady’s not for turning”– Are “Reforms as Experiments” conceivable?

• Time scales are long

Propositions• Teachers:– must have up-to-date evidence– must interact and collect feedback continuously – must not be held to account for processes– progress is the bottom line

• Policy makers should:– accumulate & generate evidence– run pilot projects for proposed policy initiatives– spend large sums only when with evidence– must monitor– be held to account for consequences

Thank you