Upload
others
View
1
Download
0
Embed Size (px)
Citation preview
Learning for the Future: The
performance of 15-year-olds in
Ireland on reading literacy, science
and mathematics on PISA 2018
3rd December 2019
Overview
PISA 2018 results – key findings and trend comparisons
Reading literacy
Science
Mathematics
Selected key findings from PISA questionnaires
Conclusions
PISA 2018, national team:
Caroline McKeown, National Project Manager
Sylvia Denner, National Data Manager
National team: Gerry Shiel, Sarah McAteer, Lynsey O’Keeffe, and Mary Delaney.
PISA 2018 – Introduction (1)
79 countries, and almost 600,000 students participated
In Ireland, 5,577 students in 157 schools
In Ireland, students were born in 2002, and were aged 15- and 16-years-old
Students were drawn from across five year levels, First-Fifth Year
Third Year was the modal grade, with over three-fifths of students.
In Ireland, PISA was administered entirely on laptops hired and transported to
schools
PISA 2018 – Introduction (2)
Students sat a 2-hour test and completed a 55-minute questionnaire session
In addition:
School principals (or their nominee) completed a school questionnaire (CBA)
Parents of selected students completed a questionnaire (PBA)
Teachers of Junior Cycle English were asked to complete a national questionnaire
(PBA)
Ireland did not participate in the cognitive assessment of Global Competence,
but students in Ireland did respond to questionnaire items on the topic.
PISA 2018 – some data notes
79 countries participated, but:
Data for Vietnam has not been fully validated for international comparability –
applies to all three domains
Reporting on data for Spain in reading literacy is deferred until sub-optimal
response patterns (in the test of reading fluency) are investigated
Comparability of data for Austria in 2009 cannot be ensured, therefore trend
comparisons for Austria are excluded 2009-2018. Only applies to reading literacy.
Reading literacy:
2018 results for 36 OECD countries, 2009-2018 results for 35 OECD countries
Mathematics and science:
2018 results and trend comparisons include all 37 OECD countries
PISA 2018 – interpretation of results
To aid interpretation of results, a longer-term trend is provided in the figures. However, in the report, the focus is on the last time the subject was a major domain:
Reading literacy, 2009-2018
Science, 2015-2018
Mathematics, 2012-2018.
Interpreting scores: not like percent correct
OECD mean was originally set at 500, with SD of 100.
5-point difference is not really meaningful, 10-point difference is small, 15 is moderate, 20-25+ large.
Studies like PISA always involve uncertainty in measurement due to assessment of samples rather than populations, along with use of rotated test design. For example, the measurement error for Ireland for reading literacy is 2.2.
PISA 2018 results – an overview
Ireland’s performance in reading literacy, science and mathematics was
relatively stable between 2015 and 2018, with small but not statistically
significant changes in each domain.
On average across OECD countries between 2015 and 2018, mean scores on
reading literacy and mathematics increased by small non-significant amounts,
while there was a small non-significant drop in science.
Students in Ireland are significantly above the OECD average on all 3 domains
of reading literacy, science and mathematics.
PISA 2018 – reading literacy
On reading literacy, students in Ireland are ranked 4th out of 36 OECD
countries, and 3rd out of 27 EU countries
Ireland’s true rank is between 1st and 5th across OECD countries.
Student performance on PISA reading literacy in Ireland is characterised by:
An above-average percentage of high performers (12.1%)
A small and below-average percentage of low performers (11.8%)
Ireland’s mean score of 518.1 is significantly higher than the OECD average of
487.1
Ireland’s mean score is not significantly different from Hong Kong (China),
Estonia, Canada, Finland, Korea, Poland
Three regions significantly outperform Ireland, B-S-J-Z (China), Singapore and
Macao (China).
8
PISA reading literacy mean scores,
Ireland and the OECD, 2000-2018
526.7
515.5517.3
495.6
523.2 520.8518.1
494.4
493.8
486.1491.2
493.5490.2
487.2
470
480
490
500
510
520
530
2000 2003 2006 2009 2012 2015 2018
Ireland OECD Average
Large drop in PISA 2009; subsequently attributed to statistical model used
to account for missing responses; otherwise stable trend; slight downward
trend in line with OECD average.
PISA reading literacy proficiency levels,
Ireland, 2009-2018
17.2
9.6 10.2
11.8
7.0
11.410.7
12.1
4
6
8
10
12
14
16
18
2009 2012 2015 2018
Below Level 2 At or above level 5
Small, non-significant upwards trend in percentage below Level 2, and Levels 5-6
PISA reading literacy mean scores,
Ireland by gender, 2009-2018
476.3
509.2515.0
506.4
515.5
537.7526.9
529.6
400
420
440
460
480
500
520
540
560
2009 2012 2015 2018
Male Female
Unusually low gender gap in 2015; lower than average gap in 2018
compared to OECD.
PISA 2018 reading literacy proficiency
levels by gender, Ireland and the OECD
15.1
8.510.3
13.8
27.7
17.5
7.1
10.4
0
5
10
15
20
25
30
Male Female Male Female
Below Level 2 Levels 5-6
Ireland OECD
Strong performance by Irish females; room for improvement among Irish
males; both genders do well compared with OECD averages.
PISA 2018 - science
On science, students in Ireland ranked 17th among 37 OECD countries and 11th
out of 28 EU countries.
Ireland’s true rank is between 13th and 21st across the OECD.
Student performance on PISA science in Ireland is characterised by:
An average percentage of high performers (5.8%)
Lower than average percentage of low-performing students (17.0%).
Ireland’s mean score of 496.1 is significantly higher than the OECD average of
488.7
Ireland’s mean score is not significantly different from 11 countries (Germany,
the United States, Sweden, Belgium, Czech Republic, Switzerland, France,
Denmark, Portugal, Norway and Austria)
PISA science mean scores, Ireland and
the OECD, 2006-2018
508.3
508.0
522.0
502.6
496.1
494.8497.8 498.3
490.6488.7
480
485
490
495
500
505
510
515
520
525
2006 2009 2012 2015 2018
Ireland OECD Average
Drop in science scores in 2015 coincided with introduction of interactive
questions (simulated experiments)
PISA science proficiency levels, Ireland,
2015-2018
15.3% 17.0%
7.1%5.8%
0
2
4
6
8
10
12
14
16
18
2015 2018
Below Level 2 Levels 5-6
Trends in performance at proficiency levels mirror overall mean
performance 2015-2018 (small, non-significant changes)
PISA science mean scores, Ireland by
gender, 2015-2018
507.7
495.4
497.2496.9
450
460
470
480
490
500
510
520
2015 2018
Male Female
Narrowing of gender gap due to decline in the mean score of male
students in Ireland
PISA 2018 science, proficiency levels by
gender, Ireland and the OECD
18.1
16.0
6.8
4.9
23.2
20.8
7.36.2
0
5
10
15
20
25
Males Females Males Females
Below level 2 Levels 5-6
Ireland OECD average
PISA 2018 - mathematics
On mathematics, students in Ireland ranked 16th among 37 OECD countries,
and 11th out of 28 EU countries.
Ireland’s true rank is between 12th and 21st across the OECD.
Student performance on PISA mathematics in Ireland is characterised by:
A below average percentage of high performers (8.2%) than on average across
OECD countries.
A below average percentage of low-performing students (15.7%).
Ireland’s mean score of 499.6 is significantly higher than the OECD average of
489.3
Ireland’s mean score in mathematics does not differ significantly from ten
countries/economies (Sweden, United Kingdom, Norway, Germany, Czech
Republic, Austria, Latvia, France, Iceland, and New Zealand).
PISA mathematics mean scores, Ireland
and the OECD, 2012-2018
502.8
501.5
487.1
501.5503.7
499.6
499.2
490.0
491.6
490.4
487.2
489.3
475
480
485
490
495
500
505
2003 2006 2009 2012 2015 2018
Ireland OECD
As with reading, performance in mathematics is relatively stable across
cycles except for a drop in 2009.
PISA mathematics, Ireland by proficiency
levels, 2012-2018
16.9%
15.0%15.7%
10.7%9.8%
8.2%
0
5
10
15
20
2012 2015 2018
Below Level 2 At or above level 5
PISA mathematics mean scores, Ireland
by gender, 2012-2018
509.0 511.6 502.6
493.7 495.4 496.7
440.0
450.0
460.0
470.0
480.0
490.0
500.0
510.0
520.0
2012 2015 2018
Males Females
As with science, narrowing of gender gap due to decline in mean
score of male students in 2018.
PISA 2018 mathematics, proficiency
levels by gender, Ireland and the OECD
15.7 15.7
9.9
6.6
23.9 24.0
12.3
9.5
0
5
10
15
20
25
Male Female Male Female
Below Level 2 Levels 5-6
Ireland OECD
Use of digital technology by students in
PISA 2018
On the index of subject-related digital technology use in class, students in
Ireland had a mean score (-0.37), which was significantly and substantially
below the average across OECD countries (0.00).
Students’ reports of time spent using digital devices for subjects outside class
(this could be at home or in school). The mean score for Ireland was -0.30,
was significantly below the OECD average.
Students in Ireland also reported that across selected subjects (English,
Science, and Mathematics), it was mainly the teacher who used digital
devices in the classroom.
In 2018, under half of participating students had taken a test on computer
prior to PISA, which was up slightly from 2015 (46.5% in 2018 and 42.8% in
2015).
Enhancing teaching and learning with
digital technology in PISA 2018
Principals’ views on the capacity of their schools to enhance teaching and learning through digital technology are generally less favourable in Ireland than on average across the OECD, with some relative strengths.
The percentage of students’ whose principals ‘agreed’ the school’s Internet bandwidth was sufficient (75.9% v 67.5%), adequate software was available (71.9% v 71.3%) and digital devices were sufficiently powerful (73.2% v 68.5%) were about the same or higher than the OECD average.
However, principals in Ireland highlighted relative weaknesses relating to levels of sufficient technical support or assistance (20.7% v 54.1%), the availability of effective professional resources for teachers to learn how to use digital technology (47.4% v 64.7%), and the skill levels of teachers (49.3% v 64.6%) as challenges to successful integration of digital technology in teaching and learning.
Note, perentages: Ireland v OECD
PISA 2018 - Time spent using digital devices
on a normal school day by reading
achievement539.5
461.5
512.0
484.6
528.1
463.9
450.0
460.0
470.0
480.0
490.0
500.0
510.0
520.0
530.0
540.0
550.0
None Less than 1 hour 1 to 3 hours 3 to 5 hours More than 5 hours
Playing computer games (on a digital device)
Chatting or interacting with friends on social networking sites
Watching TV (including online)
Aspects of student well-being in PISA
2018 – Life satisfaction
Well-being in PISA 2018 will be reported on by the ERC in more detail in 2020.
‘Overall, how satisfied are you with your life as a whole these days?’
Slider from ‘0 - not all satisfied’ to ‘10 – completely satisfied’.
Three-fifths (61%) of students in Ireland reported that they were satisfied
(score of 7-10) with their lives, which is significantly lower than the overall
average across OECD countries (67%).
On average, significantly fewer female students in Ireland reported that they
were satisfied with their life (55.5%) compared to male students (67.3%).
Aspects of student well-being in PISA
2018 – Exam stress (national question)
22.6% of students reported they always feel under pressure from teachers to
do well in exams
Students who reported never feeling under pressure from their teachers to do well
scored significantly lower in reading literacy compared to students who reported
sometimes feeling pressure (517.3), often feeling pressure (535.2) or always
feeling pressure (530.1).
10.7% of students reported they always feel physically unwell thinking about
or doing exams and tests
Students who reported that they often or always felt physically unwell thinking
about exams performed less well on average on reading literacy (498.9 and 509.1
respectively), compared to students who never feel physically sick thinking about
exams (546.9).
Conclusions – reading literacy
The relative stability in overall mean scores, notwithstanding the revised PISA
assessment framework (to better reflect how reading has evolved in recent
years) and new test content, highlights the consistently strong performance
of students in Ireland on PISA reading literacy.
This steady performance includes the incorporation digital navigation and
other interactive reading skills and competencies, both in traditional and
newer texts.
The new English specification was introduced in 2014, and thus all
participating students in PISA 2018 studied under it.
Students in Ireland in 2018 reported strong awareness of reading strategies. In
particular, students in Ireland were significantly above the OECD average
(0.00) at 0.21 on an OECD-developed metacognition index, where students
are asked to evaluate the appropriateness of various strategies for assessing
the credibility of an email.
Conclusions - science
A small drop in student performance in science in PISA 2018 was not-statistically significant.
However, between 2012-2018, the decline in performance in science is large and significant (-25.9 score points). This coincided with the introduction of computer-based testing and the introduction of new interactive items.
There is a below average percentage of high-achieving students in science in PISA 2018, even though Ireland’s mean score is significantly higher than the OECD average.
As in mathematics, the gender difference for science is small and non-significant in 2018. This differs from 2015 when male students performed significantly better than females in mathematics and science.
The new science specification was introduced in 2016, with the first round of students sitting that subject as part of the Junior Cycle Profile of Achievement (JCPA) in 2019. Therefore, only a small proportion of students (2%) participating in PISA 2018 studied under the new specification for science.
Conclusions - mathematics
Ireland’s mean score for mathematics remains significantly above the OECD
average in 2018.
However, as in science, there is small but incremental decline in the
proportion of students performing at the highest levels in mathematics (at or
above Proficiency Levels 5 and 6) 2012-2018.
The new specification for mathematics was introduced to First Year students
in 2018; therefore, almost all PISA 2018-eligible students studied under the
previous specification, often referred to as Project Maths.
Mathematics becomes the major assessment domain in PISA 2021, when the
framework will be revised, and items that capitalise on the affordances of
computer-based assessment will be introduced.