Embed Size (px)
Citation preview
OECD EMPLOYER
BRAND
Playbook
1
PISA 2012 Evaluating school systems
to improve education
Andreas Schleicher
2 PISA in brief
• Over half a million students… – representing 28 million 15-year-olds in 65 countries/economies
… took an internationally agreed 2-hour test… – Goes beyond testing whether students can
reproduce what they were taught…
… to assess students’ capacity to extrapolate from what they know and creatively apply their knowledge in novel situations
– Mathematics, reading, science, problem-solving, financial literacy
– Total of 390 minutes of assessment material
… and responded to questions on… – their personal background, their schools
and their engagement with learning and school
• Parents, principals and system leaders provided data on… – school policies, practices, resources and institutional factors that
help explain performance differences .
Singapore
Hong Kong-China Chinese Taipei
Korea
Macao-China Japan Liechtenstein Switzerland
Netherlands Estonia Finland Canada
Poland Belgium Germany Viet Nam
Austria Australia Ireland Slovenia
Denmark New Zealand Czech Republic France
United Kingdom Iceland
Latvia Luxembourg Norway Portugal Italy Spain
Russian Fed. Slovak Republic United States Lithuania Sweden Hungary
Croatia Israel
Greece Serbia Turkey
Romania
Bulgaria U.A.E. Kazakhstan Thailand
Chile Malaysia
Mexico 410
420
430
440
450
460
470
480
490
500
510
520
530
540
550
560
570
580
Mean score
High mathematics performance
Low mathematics performance
… Shanghai-China performs above this line (613)
… 12 countries perform below this line
Average performance
of 15-year-olds in
Mathematics Fig I.2.13
US
Massachusetts
Connecticut
Florida
26% of American 15-year-olds do not reach PISA Level 2
(OECD average 23%, Shanghai 4%, Japan 11%, Canada 14%, Some estimate
long-term economic cost to be US$72 trillion )
Socially equitable
distribution of learning
opportunities
High mathematics performance
Low mathematics performance
Average performance
of 15-year-olds in
mathematics
Strong socio-economic
impact on student
performance
Singapore
Hong Kong-China Chinese Taipei
Korea
Macao-China Japan Liechtenstein Switzerland
Netherlands Estonia Finland Canada
Poland Belgium Germany Viet Nam
Austria Australia Ireland Slovenia
Denmark New Zealand Czech Republic France
United Kingdom Iceland
Latvia Luxembourg Norway Portugal Italy Spain
Russian Fed. Slovak Republic United States Lithuania Sweden Hungary
Croatia Israel
Greece Serbia Turkey
Romania
Bulgaria U.A.E. Kazakhstan Thailand
Chile Malaysia
Mexico
Australia Austria
Belgium Canada
Chile
Czech Rep.
Denmark
Estonia
Finland
France
Germany
Greece
Hungary
Iceland Ireland
Israel
Italy
Japan
Korea
Luxembourg
Mexico
Netherlands
New Zealand
Norway
Poland
Portugal
Slovak Rep.
Slovenia
Spain Sweden
Switzerland
Turkey
UK
US
Singapore
Hong Kong-China Chinese Taipei
Macao-China
Liechtenstein
Viet Nam
Latvia
Russian Fed. Lithuania
Croatia
Serbia Romania
Bulgaria United Arab Emirates
Kazakhstan
Thailand
Malaysia
0 2 4 6 8 10 12 14 16 18 20 22 24 26
2012 Shanghai-China
Socially equitable
distribution of learning
opportunities
Strong socio-economic
impact on student
performance
Australia Austria
Belgium Canada
Chile
Czech Rep.
Denmark
Estonia
Finland
France
Germany
Greece
Hungary
Iceland Ireland
Israel
Italy
Japan
Korea
Luxembourg
Mexico
Netherlands
New Zealand
Norway
Poland
Portugal
Slovak Rep.
Slovenia
Spain Sweden
Switzerland
Turkey
UK
US
Singapore
Hong Kong-China Chinese Taipei
Macao-China
Liechtenstein
Viet Nam
Latvia
Russian Fed. Lithuania
Croatia
Serbia Romania
Bulgaria United Arab Emirates
Kazakhstan
Thailand
Malaysia
0 2 4 6 8 10 12 14 16 18 20 22 24
Australia Austria
Belgium Canada
Chile
Czech Rep.
Denmark
Estonia
Finland
France
Germany
Greece
Hungary
Iceland Ireland
Israel
Italy
Japan
Korea
Luxembourg
Mexico
Netherlands
New Zealand
Norway
Poland
Portugal
Slovak Rep.
Slovenia
Spain Sweden
Switzerland
Turkey
UK
US
Australia
Austria
Belgium
Canada
Chile
Czech Rep.
Denmark
Estonia
Finland
France
Germany
Greece
Hungary
Iceland
Ireland
Israel
Italy
Japan
Korea
Luxembourg
Mexico
Netherlands
New Zealand
Norway
Poland
Portugal
Slovak Rep.
Slovenia
Spain
Sweden
Switzerland
Turkey
UK
US
2012
Socially equitable
distribution of learning
opportunities
Strong socio-economic
impact on student
performance
Australia Austria
Belgium Canada
Chile
Czech Rep.
Denmark
Estonia
Finland
France
Germany
Greece
Hungary
Iceland Ireland
Israel
Italy
Japan
Korea
Luxembourg
Mexico
Netherlands
New Zealand
Norway
Poland
Portugal
Slovak Rep.
Slovenia
Spain Sweden
Switzerland
Turkey
UK
US
Australia
Austria
Belgium
Canada
Chile
Czech Rep.
Denmark
Estonia
Finland
France
Germany
Greece
Hungary
Iceland
Ireland
Israel
Italy
Japan
Korea
Luxembourg
Mexico
Netherlands
New Zealand
Norway
Poland
Portugal
Slovak Rep.
Slovenia
Spain
Sweden
Switzerland
Turkey
UK
US
Australia Austria
Belgium Canada
Chile
Czech Rep.
Denmark
Estonia
Finland
France
Germany
Greece
Hungary
Iceland Ireland
Israel
Italy
Japan
Korea
Luxembourg
Mexico
Netherlands
New Zealand
Norway
Poland
Portugal
Slovak Rep.
Slovenia
Spain Sweden
Switzerland
Turkey
UK
US
Australia
Austria
Belgium
Canada
Chile
Czech Rep.
Denmark
Estonia
Finland
France
Germany
Greece
Hungary
Iceland
Ireland
Israel
Italy
Japan
Korea
Luxembourg
Mexico
Netherlands
New Zealand
Norway
Poland
Portugal
Slovak Rep.
Slovenia
Spain
Sweden
Switzerland
Turkey
UK
US
Singapore
Shanghai
Singapore
2003 - 2012
Hong Kong-China
Brazil
Uruguay
Croatia
Latvia
Chinese Taipei
Thailand
Bulgaria
Jordan
Macao-China
UAE
Argentina
Indonesia
Kazakhstan
Peru
Costa Rica Montenegro
Tunisia
Qatar
Singapore
Colombia
Malaysia Serbia
Romania
Viet Nam
Shanghai-China
USA
Poland
New Zealand
Greece
UK
Estonia
Finland
Slovak Rep.
Luxembourg
Germany
Austria France
Japan
Turkey Sweden Hungary
Australia Israel
Canada
Ireland
Chile
Belgium
Spain Denmark
Switzerland
Iceland
Slovenia
Portugal Norway
Mexico
Korea
Italy
R² = 0.19
300
350
400
450
500
550
600
650
700
-0.5 0 0.5 1 1.5
Ma
the
ma
tics
perf
orm
an
ce
(sc
ore
po
ints
)
Equity in resource allocation (index points)
Countries with better performance in mathematics tend
to allocate educational resources more equitably
Greater
equity Less
equity
Adjusted by per capita GDP
Fig IV.1.11
30% of the variation in math performance across OECD countries is explained by the degree of similarity of
educational resources between advantaged and disadvantaged schools
OECD countries tend to allocate at least
an equal, if not a larger, number of
teachers per student to disadvantaged
schools; but disadvantaged schools tend
to have great difficulty in attracting qualified teachers.
The American dream of social mobility
In some countries it is close to a reality
10
0
2
4
6
8
10
12
14
16
18
20
Sh
an
gh
ai-
Chin
a
Hon
g K
on
g-C
hin
a
Ma
ca
o-C
hin
a
Vie
t N
am
S
ing
ap
ore
K
ore
a
Ch
ine
se
Ta
ipe
i Ja
pa
n
Lie
ch
ten
ste
in
Sw
itze
rla
nd
E
sto
nia
N
eth
erl
an
ds
Po
lan
d
Can
ad
a
Fin
lan
d
Be
lgiu
m
Po
rtu
ga
l G
erm
an
y
Tu
rke
y
OE
CD
ave
rag
e
Ita
ly
Sp
ain
L
atv
ia
Ire
lan
d
Au
str
alia
T
ha
ilan
d
Au
str
ia
Lu
xe
mb
ou
rg
Cze
ch
Re
pu
blic
S
love
nia
U
nite
d K
ing
do
m
Lith
ua
nia
F
ran
ce
N
orw
ay
Ice
lan
d
Ne
w Z
ea
lan
d
Ru
ssia
n F
ed
. U
nite
d S
tate
s
Cro
atia
D
en
ma
rk
Sw
ed
en
H
un
ga
ry
Slo
va
k R
ep
ub
lic
Me
xic
o
Se
rbia
G
ree
ce
Is
rae
l T
un
isia
R
om
an
ia
Ma
laysia
In
do
ne
sia
B
ulg
ari
a
Ka
za
kh
sta
n
Uru
gu
ay
Bra
zil
Costa
Ric
a
Chile
C
olo
mb
ia
Mo
nte
ne
gro
U
.A.E
. A
rge
ntin
a
Jo
rda
n
Pe
ru
Qa
tar
%
Percentage of resilient students
More than 10
% resilient Between 5%-10% of resilient students Less than 5%
Fig II.2.4 11
Socio-economically disadvantaged students not
only score lower in mathematics, they also report
lower levels of engagement, drive, motivation and
self-beliefs. Resilient students break this link and
share many characteristics of advantaged high-achievers.
A resilient student is situated in the bottom quarter of
the PISA index of economic, social and cultural
status (ESCS) in the country of assessment and
performs in the top quarter of students among all
countries, after accounting for socio-economic status.
Performance of countries in a level playing field
How the world would look if students around the world were living in similar social and economic conditions
12
340
360
380
400
420
440
460
480
500
520
540
560
580
600
Sh
an
gh
ai-
Chin
a
Sin
ga
po
re
Hon
g K
on
g-C
hin
a
Ch
ine
se
Ta
ipe
i V
iet N
am
M
aca
o-C
hin
a
Ko
rea
Ja
pa
n
Lie
ch
ten
ste
in
Po
lan
d
Sw
itze
rla
nd
E
sto
nia
N
eth
erla
nd
s
Ge
rma
ny
Be
lgiu
m
Fin
lan
d
Can
ad
a
Po
rtuga
l A
ustr
ia
Cze
ch
Rep
ub
lic
Ne
w Z
ea
lan
d
La
tvia
F
ran
ce
Slo
ve
nia
Ir
ela
nd
A
ustr
alia
O
EC
D a
ve
rage
T
urk
ey
Slo
va
k R
ep
ub
lic
Sp
ain
H
un
ga
ry
Lu
xe
mb
ou
rg
Ita
ly
Ru
ssia
n F
ed
era
tio
n
Un
ite
d K
ingd
om
D
en
ma
rk
Lith
ua
nia
C
roa
tia
U
nite
d S
tate
s
Norw
ay
Sw
ed
en
Ic
ela
nd
R
om
an
ia
Isra
el
Se
rbia
T
ha
ilan
d
Gre
ece
B
ulg
aria
C
hile
U
rugu
ay
Ma
laysia
K
aza
kh
sta
n
Cyp
rus5
, 6
M
exic
o
Costa
Ric
a
Unite
d A
rab
Em
ira
tes
Bra
zil
Mo
nte
ne
gro
T
un
isia
In
do
ne
sia
P
eru
A
rge
ntin
a
Colo
mb
ia
Jo
rda
n
Qa
tar
Me
an
ma
the
ma
tic
s s
co
re
Mean score at the country level before adjusting for socio-economic status
Mean score at the country level after adjusting for socio economic status
Mathematics performance in a level playing field Mean mathematics performance after accounting for socio-economic status
Fig II.3.3 13
US would rank lower in a level playing field: • Ranks 3rd in wealth per person (explains 12%) • Ranks 5th in spending per student • Has average share of disadvantaged students • Ranks 6th in parental attainment • Has 6th largest share of immigrant students
(explains 4%)
It is not just about poor kids in poor neighbourhoods…
…but about many kids in many neighbourhoods
14
0
10
20
30
40
50
60 S
ha
ngh
ai-
Chin
a
Sin
ga
po
re
Ch
ine
se
Ta
ipe
i H
on
g K
on
g-C
hin
a
Ko
rea
L
iech
ten
ste
in
Ma
ca
o-C
hin
a
Ja
pa
n
Sw
itze
rla
nd
B
elg
ium
N
eth
erla
nd
s
Ge
rma
ny
Po
lan
d
Ca
na
da
F
inla
nd
N
ew
Ze
ala
nd
A
ustr
alia
E
sto
nia
A
ustr
ia
Slo
ve
nia
V
iet N
am
F
ran
ce
C
ze
ch
Rep
ub
lic
OE
CD
ave
rage
U
nite
d K
ingd
om
L
uxe
mb
ou
rg
Ice
lan
d
Slo
va
k R
ep
ub
lic
Irela
nd
P
ort
uga
l D
en
ma
rk
Ita
ly
Norw
ay
Isra
el
Hun
ga
ry
Unite
d S
tate
s
Lith
ua
nia
S
we
de
n
Sp
ain
L
atv
ia
Ru
ssia
n F
ed
era
tio
n
Cro
atia
T
urk
ey
Se
rbia
B
ulg
aria
G
ree
ce
U
nite
d A
rab
Em
ira
tes
Rom
an
ia
Th
aila
nd
Q
ata
r C
hile
U
rugu
ay
Ma
laysia
M
on
ten
egro
K
aza
kh
sta
n
Alb
an
ia
Tu
nis
ia
Bra
zil
Me
xic
o
Pe
ru
Co
sta
Ric
a
Jo
rda
n
Colo
mb
ia
Ind
on
esia
A
rge
ntin
a
%
Percentage of top performers
in mathematics 15 Tab I.2.1a
UK
Across OECD, 13% of students are top performers (Level 5 or 6). They can develop and work with models for complex situations, and work strategically with advanced thinking and reasoning skills
Hong Kong-China
Brazil
Uruguay
Albania
Croatia
Latvia
Lithuania
Chinese Taipei
Thailand Bulgaria
Jordan
Macao-China
UAE Argentina
Indonesia
Kazakhstan
Peru
Costa Rica
Tunisia
Qatar
Singapore
Colombia
Malaysia
Serbia
Romania
Viet Nam
Shanghai-China
USA
Poland
New Zealand
Greece
UK
Estonia
Finland
Slovak Rep.
Luxembourg
Germany Austria
Czech Rep.
France
Japan
Turkey
Sweden
Hungary Australia
Israel
Canada
Chile
Belgium
Netherlands Spain
Denmark
Switzerland
Iceland
Slovenia
Portugal
Norway
Korea
Italy
R² = 0.13
300
350
400
450
500
550
600
650
-1.5 -1 -0.5 0 0.5 1 1.5
Ma
the
ma
tics
perf
orm
an
ce
(sc
ore
po
ints
)
Index of school responsibility for curriculum and assessment (index points)
Countries that grant schools autonomy over curricula and
assessments tend to perform better in mathematics Fig IV.1.15
No standardised math policy
Standardised math policy 455
460
465
470
475
480
485
Less school autonomy
More school autonomy
Schools with more autonomy perform better than schools with
less autonomy in systems with standardised math policies
Score points
School autonomy for curriculum and assessment
x system's extent of implementing a standardised math policy (e.g. curriculum and
instructional materials)
Fig IV.1.16
Schools with more autonomy perform better than schools with
less autonomy in systems with more collaboration
Teachers don't participate in management
Teachers participate in management 455
460
465
470
475
480
485
Less school autonomy
More school autonomy
Score points
School autonomy for resource allocation x System's level of teachers
participating in school management
Across all participating countries and economies
Fig IV.1.17
Schools with more autonomy perform better than schools with
less autonomy in systems with more accountability arrangements
School data not public
School data public 464
466
468
470
472
474
476
478
Less school autonomy
More school autonomy
Score points
School autonomy for curriculum and assessment
x system's level of posting achievement data publicly
Fig IV.1.16
0 20 40 60 80 100
Written specification of the school's curriculum and educational goals
Written specification of student-performance standards
Systematic recording of data, including teacher and student attendance and graduation rates, test results …
Internal evaluation/self-evaluation
External evaluation
Written feedback from students (e.g. regarding lessons, teachers or resources)
Teacher mentoring
Regular consultation with one or more experts over a period of at least six months with the aim of improving …
Implementation of a standardised policy for mathematics
%
Percentage of students in schools whose principal reported that their schools have the following for quality assurance and improvement:
Shanghai-China OECD average
Quality assurance and school improvement Fig IV.4.14 20
21 21 L
essons f
rom
hig
h p
erf
orm
ers
Catching up with the top-performers
Low impact on outcomes
High impact on outcomes
Low feasibility High feasibility
Money pits
Must haves
Low hanging fruits
Quick wins
22 22 L
essons f
rom
hig
h p
erf
orm
ers
Low impact on outcomes
High impact on outcomes
Low feasibility High feasibility
Money pits
Must haves
Low hanging fruits
Quick wins
Commitment to universal achievement
Gateways, instructional systems
Capacity at point of delivery
Incentive structures and accountability
Resources where they yield most
A learning system Coherence
23 23 L
essons f
rom
hig
h p
erf
orm
ers
Some students learn at high levels All students need to learn at high levels
Student inclusion
Routine cognitive skills, rote learning Learning to learn, complex ways of thinking, ways of working
Curriculum, instruction and assessment
Few years more than secondary High-level professional knowledge workers
Teacher quality
‘Tayloristic’, hierarchical Flat, collegial
Work organisation
Primarily to authorities Primarily to peers and stakeholders
Accountability
What it all means
Average performers Top performers
Thank you !
Find out more about PISA at www.pisa.oecd.org
• All national and international publications
• The complete micro-level database
Email: [email protected]
Twitter: SchleicherEDU
and remember:
Without data, you are just another person with an opinion
