Heading
Using the NPD: Some pitfalls and issues
Presentation to PLUG13 September 2006
Andrew RaySchools Analysis and Research Division, DfES
Heading
Statistical bulletins based on
analysis of the NPD
3
PLASC variablesPLASC
• FSM
• Ethnicity
• SEN
• Looked after children
• EAL
• Not covered here: variables linked to postcode
4
FSM is limited as a proxy for social class
FSM
Proportions achieving 5+ A*-C at GCSE in 2003
0
10
20
30
40
50
60
70
80
PR
OF
ES
SIO
NA
L
Hig
her
prof
essi
onal
Low
erpr
ofes
sion
al
NO
N-
PR
OF
ES
SIO
NA
L
Inte
rmed
iate
Low
ersu
perv
isor
y
Rou
tine
Oth
er: n
eith
erpa
rent
em
ploy
ed
Oth
er: 1
+ pa
rent
empl
oyed
ALL
PR
OF
ES
SIO
NA
L
Hig
her
prof
essi
onal
Low
erpr
ofes
sion
al
NO
N-
PR
OF
ES
SIO
NA
L
Inte
rmed
iate
Low
ersu
perv
isor
y
Rou
tine
Oth
er: n
eith
erpa
rent
em
ploy
ed
Oth
er: 1
+ pa
rent
empl
oyed
ALL
Non-FSM
Non-FSM
Non-FSM
Non-FSM
Non-FSM
Non-FSM
Non-FSM
Non-FSM
Non-FSM
Non-FSM
FSM FSM FSM FSM FSM FSM FSM FSM FSM FSM
Grey lines are 95% confidence intervals. Source: YCS linked to the NPD. See Table 5.7 in DfES Bulletin 'Trends in Attainment Gaps: 2005' (June 2006)
Non-FSM FSM
5
Code changes 2002 03Ethnicity
2002 classifications of the 2003 PLASC 'any other mixed background' ethnic group
WHITEUK
WHITEEURO PEAN
WHITE OTHER
WHITE NOT KNOWN
INDIAN
PAKI STANI
BANGLADESHI
BLACKCARIBBEAN
BLACKAFRICAN
BLACKOTHER
CHINESE
OTHER
NOT GIVEN
NOT OBTAINED
UNCLASSIFIED
Data taken from Ray Godfrey’s Changes in Ethnicity Codes in the Pupil Level Annual Schools Census 2002-2003
6
Local Authority variation in special school provision
SEN
Statemented Pupils in Local Authorities2005 Key Stage 2 cohort
0%
10%
20%
30%
40%
50%
60%
70%
0% 1% 2% 3% 4% 5% 6% 7%
Overall percentage with statements
Pro
port
ion
of s
tate
men
ted
SE
N in
spe
cial
sc
hool
s
7
Under-reporting of looked after children
Looked after children
Comparison of data sources on looked after children
0
500
1000
1500
2000
2500
3000
3500
4000
4500
5000
Year 2 Year 6 Year 9 Year 11
Nu
mb
er
of
loo
ked
aft
re p
up
ils
Data from OC2
Data from PLASC
Source: PLASC quality report on pupils in care
8
Complex relationship with ethnicity
EAL
Pupils under the PLASC heading of 'Black African' by EAL status(2006 key stage 4, all schools)
64 327 748 67 1349 36 176114
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Angolan Congolese Ghanaian Nigerian Sierra Leonian Somali Sudanese Other BlackAfrican
EAL
Non-EAL
9
Attainment dataAttainment
• Changing test structures
• Changing qualifications
• Changing point systems
• Ages and Key Stages
• Ceiling effects
• Use of marks data for sublevels
10
Changes to KS3 English levelsLevels and
tests
Available levels in years before current NPDAvailable levels in years within current NPD
Year A D T B N W 1 2 3 4 5 6 7 8 EP19961997199819992000200120022003200420052006
W, 1 and 2 dropped in 1997 and replaced by B
Extension paper and Level 8 dropped in 2003
L3 dropped in 2004 but reintroduced in
2005
Dispplied replaced
by 'T' in 2004
11
Key Stage 2: artificial increase in ‘Bs’
Levels and tests
0.0
0.5
1.0
1.5
2.0
2.5
3.0
3.5
4.0
4.5
1998 1999 2000 2001 2002 2003 2004 2005
% a
chie
vin
g le
vel
A T/D B N Level 2
Pupils achieving below Level 3 at Key Stage 2 English
12
Key Stage 1: the move to teacher assessment
Levels and tests
Key Stage 1 time series
0
10
20
30
40
50
60
70
80
90
100
1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006
Pe
rce
nta
ge
ach
ievi
ng
leve
ls
Maths (L2+)
Maths (L2B+)
Maths (L3+)
Trial TA dataTest data
13
Additional qualificationsQuals
Some of the new qualifications only collected from 2004
-
10,000
20,000
30,000
40,000
50,000
60,000
70,000
80,000
90,000
2002 2003 2004 2005
Nu
mb
er
take
n
Entry Level Qualifications
Level 1 VRQ
Level 2 Key Skills
Level 3 AS Level?
14
New GCSE points system introduced in 2004
Points
Grade Old Points New Points
A* 8 58
A 7 52
B 6 46
C 5 40
D 4 34
E 3 28
F 2 22
G 1 16
U/X/Q 0 0
Gap between “fail” and G has increased
New Points = (Old Points x 6) + (10 x number of passes at G or above)
(but points not awarded for all qualifications under the old system)
15
Age 15 or KS4Ages and
stages
56
.3%
44
.3%
89
.0%
96
.4%
57
.1%
44
.9%
90
.2%
97
.4%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
5+ A*-C 5+ A*-C inc E&M 5+ A*-G Any Passes
At age 15 At end of KS4
• End of KS4 reporting allows for more flexible rates of learning.
• 98.8% of pupils at end of KS4 are 15, 0.3% were 14 and 0.9% were 16.
16
Test scores are truncatedCeilings
0
0.25
0.5
0.75
1
1.25
1.5
15 17 19 21 23 25 27 29 31 33 35
Key Stage 2 Average Point Score
% o
f p
up
ils
usi
ng
fin
e g
rad
es
0
5
10
15
20
25
30
% o
f p
up
ils
usi
ng
wh
ole
lev
els
17
Marks and levelsSublevels
2006 Key stage 2 thresholds
Marks
Levels Points lower upper lower upperN 15 0 19 0 152 15 20 22 16 183 21 23 42 19 454 27 43 69 46 775 33 70 100 78 100
For example: Level 3 marks range: 20 for English, 27 for Maths
English Maths
18
Sublevels don’t directly correspond to marks
Sublevels
Comparison of marks and fine grade points
15.0
20.0
25.0
30.0
35.0
40.0
20 30 40 50 60 70 80 90 100
Mark
Fin
e gr
ade
poin
ts
English
Maths
19
Coverage of the NPDThe NPD
• Pupils with incomplete records (2 examples)
Not covered here:
• the extent of coverage of the independent sector
• pitfalls in choosing coverage, e.g. when to include selective schools or schools that are semi-independent (e.g. City Technology Colleges)
• whether or not to impute missing values
20
Pupils with no PLASC or KS4 records
The NPD
8
26
48
1719
31
38
11
0
10
20
30
40
50
60
Below Level 3 Level 3 Level 4 Level 5 or above
KS2 English Level
Per
cen
tag
e o
f p
up
ils
(%
)
1998 national KS2 figures Pupils with no PLASC in 2002 or 2003 and no GCSEs
Source: Trends in Attainment Gaps bulletin Appendix D
21
Pupils with no PLASC recordsThe NPD
8
26
48
1716
3739
9
3
9
47
42
0
10
20
30
40
50
60
Below level 3 Level 3 Level 4 Level 5 or above
KS2 English level
Per
cen
tag
e o
f p
up
ils
(%)
1998 national KS2 figures Non Independent schools Independent schools
Source: Trends in Attainment Gaps bulletin Appendix D
22
Issues for PLUG
• Future speakers: experts on PLASC or test data?
• Documentation: what improvements can be made?
• Further investigation: are there useful analyses that could illuminate some data issues?
•Can any common standards be agreed, or should everything be left to individual analyses?
23
Useful links
Bulletins:
Trends in Attainment Gaps http://www.dfes.gov.uk/rsgateway/DB/SBU/b000665/index.shtml
Characteristics of Low Attainers http://www.dfes.gov.uk/rsgateway/DB/SBU/b000588/index.shtml
Variation in pupil progress http://www.dfes.gov.uk/rsgateway/DB/SBU/b000481/index.shtml
Progress by pupil characteristics http://www.dfes.gov.uk/rsgateway/DB/SBU/b000402/index.shtm
Other key publications:
Results by pupil characteristics 2005 http://www.dfes.gov.uk/rsgateway/DB/SFR/s000640/index.shtml
Ethnicity and education topic note http://publications.teachernet.gov.uk/default.aspx?PageFunction=productdetails&PageMode=publications&ProductId=DFES+0208+2006&