USING EVIDENCE TO INFORM YOUR LEADERSHIP APPROACH AND SUPPORT SCHOOL IMPROVEMENT ROB CARPENTER 26 TH SEPTEMBER 2013 [email protected]

@carpenter_rob robcarpenter1971.tumblr.com

USING EVIDENCE TO INFORM YOUR LEADERSHIP APPROACH AND SUPPORT SCHOOL IMPROVEMENT

ROB CARPENTER26TH SEPTEMBER 2013

[email protected]

@carpenter_rob

http://robcarpenter1971.tumblr.com


The Triple whammy


The work of EEF

• Building the evidence of what works to raise the attainment of disadvantaged pupils in schools.

• Sharing the evidence with schools by providing independent and accessible information.

• Promoting the use of evidence-based practice both through our projects, events and resources.


Problems with research• Quality varies, but a lot is not very good• Quality really matters• How do you know who or what to trust?• Academic papers are inaccessible• Academic debates are (mostly) pointless• Peer review doesn’t work


Spending and outcomes

$4000-$5000 $5000-$6000 $6000-$7000 $7000-$8000 $8000+450

470

490

510

530

550

Public spend per student (PPP US$)

PIS

A 2

009

Rea

din

g S

core

Capturing the benefits of increased spending is difficult. Internationally, the relationship between spending and outcomes is very weak.


Spending and outcomes

In England, spending has increased by 47% since 2001, but over this period improvements in pupil outcomes have been marginal.

2001-02 2002-03 2003-04 2004-05 2005-06 2006-07 2007-08 2008-09 2009-10 2010-113500

4000

4500

5000

5500

6000

450

490

530

570

610

650

Sp

end

ing

per

pu

pil

(20

10 p

rice

s)

PIR

LS

Sco

re


Scenario: Spending the Pupil Premium• A large secondary school receives £140,000 from the

Pupil Premium in 2012-13.• How should the school decide to use this money?• Would you rather spend your Pupil Premium on

professional development to improve the quality of feedback pupils receive, or small group tuition, or class size reduction? (WHAT? WHY? HOW?)

The Toolkit doesn’t tell you what to do, but we hope that it will help you make a more informed decision.


Impact vs cost

Cost per pupil

Eff

ect

Siz

e (

mon

ths

gain

)

£00

8

£1000

Meta-cognitive

Peer tutoringEarly Years

1-1 tuitionHomework (Secondary)

Mentoring

Summer schools After

school

AspirationsPerformance pay

Teaching assistants

Smaller classes

Ability grouping

Promising

May be worth it

Notworth it

Feedback

Phonics

Homework (Primary)

CollaborativeSmall gp

tuition Parental involvement

Individualised learning

ICT

Behaviour

Social

www.educationendowmentfoundation.org.uk/toolkit

http://www.educationendowmentfoundation.org.uk/toolkit

http://www.educationendowmentfoundation.org.uk/toolkit


The Sutton Trust-EEF Teaching and Learning Toolkit

• The Toolkit is an accessible, teacher-friendly summaries of educational research

• Practice focused: giving schools in the information they need to make informed decisions and narrow the gap

• Based on meta-analyses provided by Durham University


Toolkit overview


Feedback

ApproachAverage impact

CostEvidence estimate

Summary

Feedback 9 months ££Very high impact for

low cost

Research suggests that providing effective feedback is challenging. To be effective, it should be:• About challenging tasks or goals rather than easy ones.• Given sparingly so that it is meaningful.• About what is right more often than about what is wrong.• Specific, accurate and clear, e.g. not just “correct” or “incorrect”.• Provide examples of what is correct and not just tell students when they

are wrong.• Encouraging and supportive of further effort without threatening a

learner’s self-esteem.


An evidence informed improvement process

Identify school priorities using internal data and professional judgement.

Step 1: What do you want to achieve?

External evidence summarised in the Toolkit can be used to inform decision-making.

Step 2: How can you get there?

Evaluate the impact of your decisions and identify potential improvements for the future.

Step 3: Did it work?

Mobilise the knowledge and use the findings to inform the work of the school to grow or stop

the intervention.

Step 4: Taking it to scale?


An evidence informed improvement process – leadership matters

Identify school priorities using internal data and professional judgement.

Step 1: What do you want to achieve?

External evidence summarised in the Toolkit can be used to inform decision-making.

Step 2: How can you get there?

Evaluate the impact of your decisions and identify potential improvements for the future.

Step 3: Did it work?

Mobilise the knowledge and use the findings to inform the work of the school to grow or stop

the intervention.

Step 4: Taking it to scale?

Generating your hypothesis and designing an intervention – linked to values and school priorities

Disciplined innovation– a rigorous and transparent process – building trust and confidence

Publish the findings (positive or negative) and set out the scale up options and strategy.

Using the DIY guide to establish the impact or effect of the change. What difference did it make and for what cost?


Evidence can help• If used in the right way, internal and external evidence can help us capture the maximum possible benefit from effort and spending.

• Collected over time• Gathered from a range of indicators • Includes an estimate of impact and cost• Includes an estimate of confidence

Useful internal evidence

• Summarises all available studies• Doesn’t rely on anecdote• Includes an estimate of impact and cost• Includes an estimate of confidence

Useful external evidence


Four steps to improvement

• Think hard about learning• Invest in good professional development• Evaluate teaching quality• Evaluate impact of changes

How do we get students to learn hard things?

Eg• Place value• Persuasive

writing• Music

composition• Balancing

chemical equations

• Explain what they should do• Demonstrate it• Get them to do it (with

gradually reducing support)• Provide feedback • Get them to practise until it is

secure• Assess their skill/

understanding


How do we get teachers to learn hard things?

Eg• Using formative

assessment• Assertive

discipline• How to teach

algebra

• Explain what they should do


Summary …• A lot of educational research is poor, but some is very

good: relevant and rigorous

• Four steps to improve practice:• Think hard about learning• Invest in good CPD• Evaluate teaching quality (but not with dodgy observation)• Evaluate impact of changes

Documents

USING EVIDENCE TO INFORM YOUR LEADERSHIP APPROACH AND SUPPORT SCHOOL IMPROVEMENT ROB CARPENTER 26 TH SEPTEMBER 2013 [email protected]