joining the dots 1

Closing the Gap

Getting a better understanding of the data

Mike Treadaway, FFT

joining the dots

Changes….2014

Wolf Review implementedFirst entry only counted for performance tables

2015Some schools opt for Progress 8Details of KS2 accountability (and KS1?)New Baseline (September 2015)

20161-8 Scale used for Attainment 8 / Progress 8New KS2 assessments (September 2016 Y7)

2017 First reformed GCSEs taken (English and mathematics)1-9 Scale used for new GCSEsNew Points Scale for ‘legacy’ GCSEs

20182019 More reformed GCSEs

2021 Pupils with new KS2 assessment from 2016 are now Y11

2022 Pupils with new baseline in 2015 are now Y6

joining the dots

Issues• What’s the best way to measure the gap?

• Is looking ONLY at attainment gaps good enough?

• Are all ‘disadvantaged’ pupils the same?

• What resources can help us to do the above?

• How will we measure the gap when end of key stage assessments change?

joining the dots

Headlines

joining the dots

What if we measured things differently?

joining the dots

Measuring the Gap• Threshold

– Measure only changes around the threshold

• Points– Measure changes for all pupils

Change L4+ APS

L2 -> L3 0 +1

L3 -> L4 +1 +1

L4 -> L5 0 +1

Change A*C APS

E -> D 0 +1

D -> C +1 +1

C -> B 0 +1

Key Stage 2 Key Stage 4

• Threshold measures are important– But to focus ONLY on them can be misleading

Pupil Premium (PP) vs not Pupil Premium (Attainment)

Similar Gaps ?

School A School B3

3.5

4

4.5

5

5.5

6

4.9

3.7

5.4

4.2

0.5 Level Gap

0.5 Level GapPP

Not PP

PP

Not PP

Avera

ge K

S2

Level

Pupil Premium vs not Pupil Premium (Expected Attainment)

Similar Gaps?

School A School B3

3.5

4

4.5

5

5.5

6

4.64.5

5.4

4.7

0.8 ‘Expected’ Level gap

0.2 ‘Expected’ Level gap

PP

Not PP

PPNot PP

Est

imate

d K

S2

Level (f

rom

K

S1

)

Pupil Premium vs not Pupil Premium (Expected Attainment)

Similar Gaps?

School A School B0

0.25

0.5

0.75

1

0.8

0.2

0.5 0.5

Level gap 0.3 HIGHER than expected

Actual Gap

Expec Gap

KS

2 L

evel G

ap

Actual Gap

Expec Gap

Level gap 0.3 LOWER than expected

joining the dots

Measuring the Gap• Looking at attainment gaps is important

• Can be misleading if gap at entry is not taken into account

• Look at gaps in attainment AND in value-added

joining the dots

C

B

D

E

Year 11 Attainment in 2013 (Average GCSE Grade)

Year Group when last FSM

The “Invisible Group”FSM ‘Ever not 6’

Never FSM

Last FSMin Year 11

Pupil Premium‘FSM6’

joining the dots

Closing the Gap - FFT Resources• Estimates don’t take account of pupils’ FSM status

• This provides greater level of challenge for disadvantaged pupils.

• FFT Aspire• Pupil target-setting reports show some pupil characteristics

(FSM,EAL, SEN)• Reports can be filtered to show, for example, white

disadvantaged pupils only.• Pupil Premium report (June)

• FFT Student Explorer• Can filter pupils by range of characteristics.• Can show complete history (e.g. schools attended,

attainment, FSM status) for individual pupils.• Currently in FFT Live, Aspire version in April / May.

joining the dots

Closing the Gap - EEF Toolkit

• FFT worked with Education Endowment Foundation (EEF) to help in the design and production of ‘Families of Schools’ toolkit

• Key aspects:• Enables schools to compare with 50 similar schools.

• Focus is mainly upon 3 year summaries – particularly important for schools with low % of disadvantaged pupils.

• Enables assessment of gaps in both attainment and value-added

• Currently 2013 data, will be updated with 2014 data (March / April)

• Currently KS4, working with EEF on KS2 version

joining the dots 14

Closing the Gap - Tracking

• Many schools use tracking systems such as ‘Target Tracker’

• Whatever system is in use, ensure that important gaps are picked up as early as possible:• Example from Wirral• Principle of ‘notional prior attainment’

joining the dots 15

Measuring the Gap .. In the future

• End of KS assessments are going to change• Scaled Scores at KS2• Attainment 8 and Reformed GCSEs at KS4

• Does this mean that we can’t compare gaps over time?

• Possible solutions:• DfE publication (for feedback) December

2014• FFT Pilot with Essex

joining the dots 16

Possible Solutions

• DfE proposal• Ranks pupils based on attainment in English and

mathematics• Compares average ranks• Only for data at national level – DfE considering

whether approach will work for schools and LAs

• FFT Pilot with Essex• Uses standardised scores derived from measures

such as average NC level, average GCSE grade• In each year, national average (all pupils) is 100• Looks at both attainment and value-added

joining the dots 17

How is Essex doing?

• Uses FFT Pilot methodology• Also uses FFT ‘Similar Schools’

methodology• Initial analyses compare:• Essex• Regions within Essex• Individual schools

• With• National• Similar Schools outside Essex

joining the dots 18

Key Stage 1

• Gap of 10 points = 1 sublevel• Value-added gap about 25% of total gap• Trend (for individual years):

Not EAL EAL

Closing slower than national, about same rate as for similar schools.

Widening. Same pattern in similar schools but widening faster in Essex.

joining the dots 19

Key Stage 2

• Gap of 10 points = 0.5 NC Level• Value-added gap about 25% of total gap• Trend (for individual years):

Not EAL EAL

Closing 3 times faster than national and much faster than for similar schools

Attainment gap widening, VA gap closing. National stable, similar schools closing gap.

joining the dots 20

Key Stage 4 – Average GCSE Grade

• Gap of 8 points = 1 GCSE Grade• Value-added gap about 40% of total gap• Trend (for individual years):

Not EAL EAL

Attainment gap stable – but didn’t suffer from widening seen in national for 2014

caused by impact of first entry.

Attainment gap closing, VA gap variable. Same pattern as for similar schools.

joining the dots 21

Next stages …

• Explore variations within Essex• Between areas• Between schools

• Provide Essex with underlying data to support detailed evaluation of vulnerable groups

• Extend to cover wider range of indicators

• Refine methodology based upon feedback