Embed Size (px)
Citation preview
Stability and Change in Adults’
Literacy and Numeracy Skills:
Recent Insights from Longitudinal Studies
Clemens Lechner & Beatrice Rammstedt
5th PIAAC International Conference November 28, 2018 • Bratislava, Slovakia
Our Project Team @GESIS and our Collaborators
2
Core Project Team
Clemens Lechner
GESIS
Principal Investigator
Ai Miyamoto
GESIS
Research Associate
Beatrice Rammstedt
GESIS
Co-PI
Stephen Reder
University of Portland
Collaborator
Britta Gauly
GESIS
Research Associate
Daniel Danner
University of Applied
Science Mannheim
Co-PI
Collaborators
Cordula Artelt
(University of Bamberg)
Prof. Bernd Fitzenberger
(Humboldt University Berlin)
Corinna Kleinert
(University of Bamberg)
Anke Grotlüschen
(University of Hamburg)
Wider Network Alexandra Wicht
GESIS
Collaborator
This presentation is mainly based on four papers
Lechner, C. M., Gauly, B., Miyamoto, A., & Wicht, A. (2018). Stability and
Change in Adults’ Literacy and Numeracy Skills: A Longitudinal Perspective.
Manuscript in preparation.
Gauly, B. & Lechner, C. M. (2018). Self-perfection or self-selection? Unraveling
the relationship between job-related training and adults’ literacy skills.
Manuscript submitted for publication.
Gauly, B. , Lechner, C. M., & Reder, S. (2018). The relationship between job-
related training and adult numeracy: Evidence from a German panel study.
Manuscript submitted for publication.
Reder, S., Gauly B., & Lechner, C.M. (2018). Practice Makes Perfect: The
Effects of Skill Use in Literacy Development. Manuscript in preparation.
3
4
? In a nutshell, what is this
presentation about?
We present first insights from two longitudinal
studies into adults’ literacy and numeracy skills
5
Stability and change in
adults’ literacy and
numeracy skills
Patterns and predictors
Unique large-scale,
multi-wave data (3 to 6
years of adulthood)
What we present
New insights into the
malleability of skills in
adulthood
Identifying factors that
can foster skill growth
Identifying groups at a
heightened need for
interventions and policy
measures
Why it matters
6
? What are literacy and numeracy
skills – and why do they matter?
Technological and demographic change
call for lifelong learning
Globalization enhances international competition and
accelerates technological change (Blossfeld et al., 2011)
The workforce needs to continuously adapt their skills and
knowledge (Autor, Levy, & Murnane, 2003; Jarvis, 2007)
Lifelong learning gains in importance
For individuals: employability and productivity
For nation states: economic growth, innovation,
competitiveness
Technological and demographic change
call for lifelong learning
8
Developed countries face a major demographic transition,
with declining fertility rates and a growing share of older people
An ageing workforce reinforces the need for lifelong learning
Workforce needs to be more productive (skilled) (Bloom & Sousa-
Poza, 2013)
Individuals need to work longer before retiring (Murray, 2009)
In the context of lifelong learning, foundational
literacy and numeracy skills gain currency
9
Literacy: ability to
understand, use, and
interpret written texts
Numeracy: ability to
access, use and
interpret
mathematical
information
Definitions Prerequisites to handling any type
of written or digital material
(Jäckle & Himmler, 2012)
Prerequisites to acquiring job-
specific skills
Contribute to important life
outcomes (e.g., income, health,
well-being, social participation;
Rammstedt, Danner, & Lechner,
2017; Hanushek & Woessmann,
2015)
Literacy and numeracy are foundational skills
In the Cattell–Horn–Carrol (CHC) model of intelligence:
Stratum II of Broad Abilities
Literacy ~ Reading & Writing Ability (Grw)
Numeracy ~ Quantitative knowledge (Gq)
10
Grw Gq
Social relationships
11
Our increasingly digital world demands high
levels of literacy and numeracy skills
Workplace Political participation
• Digitalization increasingly
pervades all life areas
• Dealing with symbolic verbal
and numerical material
becomes the norm
• This requires foundational
literacy and numeracy skills
12
70
Skill
s
Age 20
?
Skills Life
outcomes
? Skills
t1
?
???
Skills t2
The growing importance of these skills poses a
number of policy-relevant questions
Age profiles of skills Skill gains in late adulthood
Returns on skill investments Drivers of skill development
70
Skill
s
Age 50
13
? What do we know about these
questions?
Skills develop across the entire lifespan
14
Childhood as a sensitive period (Cunha & Heckman, 2007)
Lifelong plasticity (gains and losses)
Literacy and numeracy show inverted u-shaped
age profiles (Paccagnella, 2016)
Continued
education and
training
Job content /
demands
Literacy and
numeracy skills
Skill use at and
outside of work
Labor market
participation
Occupational factors are among the key
influences on skill development in adulthood
15
Sources: Bynner & Parsons, 1998; Desjardins & Warnke, 2012;
Grotlüschen et al., 2016; Paccagnella, 2016)
There are two comprehensive reviews of the this
literature on ageing and skills
16
Desjardins & Warnke (2012) Paccagnella (2016)
17
? What are the limitations of the
current body of evidence?
18
Existing evidence on adults’ skills has
several limitations
Almost exclusively based
on cross-sectional data
Few longitudinal studies
mostly based on small-
scale, selective samples
Studies used different
skill measures
Previous studies
Age and cohort effects
confounded
Cause and effect unclear
Generalizability limited
Relevant subgroups not
adequately covered
Psychometric quality varies
Not all studies are pertinent
to literacy and numeracy
The few existing longitudinal studies conform to
Cattell‘s predicted age trajectories
19
30 40 50 60 70 80 Age
Schaie (1994;2005)
(USA) Verbal Memory
Verbal Ability
McArdle et al. (2000)
(USA) Digit Span Forward Vocabulary
Zelinski & Burnright
(1997) (USA) Vocabulary
Giambra et al. (1995)
(USA)
Alder et al. (1990)
(USA)
Vocabulary
Vocabulary
gc – related measures (literacy domain)
gf -related measures
Vocabulary
Large-scale longitudinal data on adults‘ literacy
and numeracy skills are in very short supply
small & selective
large & representative
Cross-
sectional
National Child Development
Study (NCDS )
(Bynner & Parson, 1998)
OECD-initiated studies:
• IALS (1994 &1998)
• ALL (2003 & 2008)
• PIAAC (2012)
Multi-
wave
Longitudinal Study of Adult
Learning (LSAL) (Reder,
2009)
20
Sample
De
sig
n
21
? What are the research
opportunities offered
by longitudinal data?
There are questions that only longitudinal studies
can conclusively answer
22
Skill
s
Time
?
Unravel age-related
changes in skills
Disentangle age vs.
birth cohort effects
Improve causal
inference (LDV, FE/FD
models)
Longitudinal data can…
Two German panel studies offer repeated
measures of adults‘ skills
23
Follow-up to the German
PIAAC 2012 study
Random sample of adults
(age 16–65) and their
household members
Three annual waves (2014-
2016)
Sample size: ~ 3.000 adults
National Educational Panel
Study in Germany
Multi-cohort longitudinal
design (Starting Cohort 6:
age 22–65)
Nine waves (2008–2016),
ongoing
Sample size: ~ 12.000 adults
NEPS SC6 PIAAC-L
PIAAC and NEPS assessed literacy and numeracy
skills twice, spaced 3 to 6 years apart
24
PIAAC
2012
Wave 3
2016
Literacy
Numeracy Literacy
Numeracy
3 years
Wave 3
2010/11
Wave 5
2012/13 …
Wave 9
2016/17
Reading
Math
Reading
Math
6 years
4 years
Reading
(subsample)
Wave 1
2014
Wave 2
2015
Wave 4
2011/12
The PIAAC and NEPS measures of literacy and
numeracy are highly similar to each other
25
correlation
r
PIAAC literacy w/ PIAAC numeracy .87
NEPS reading w/ NEPS math .81
PIAAC literacy w/ NEPS reading .87
PIAAC numeracy w/ NEPS math .90
Source: PIAAC-L/NEPS linking study, based on PIAAC-L wave 2016
(Carstensen et al., in preparation)
Our project leverages the unique analytical
potential of PIAAC-L and NEPS
New insights on:
Longitudinal findings on the stability and malleability of skills
Identifying factors that drive skill growth or prevent decline
Identifying groups at a heightened need for interventions
26
Identifying factors that
foster skill development,
e.g.
Skill use on the job
Participation in training
Predictors of Change
Quantifying stability and
change in skills
Sub-group differences in
skill change
Patterns of Change
27
1 Patterns of stability and change
in adults’ literacy and numeracy
We analyzed these indicators of stability and
change in parallel in both samples
28
PIAAC
(3-year period)
NEPS
(6-year period)
Gender (%)
Male 51.0 49.6
Female 49.0 50.4
Age in years (M) 45.0 47.0
Education (%)
Low (ISED 1&2) 6.9 4.6
Medium (ISCED 3&4) 56.0 45.3
High (ISCED 5&6) 37.1 50.2
Number of respondents 2,490 3,071 (literacy)
3,010 (numeracy)
We approached the question of stability and
change from two complementary perspectives
29
By how much do
individuals’ skills change
over time on average?
Difference scores
between skills at two
time points
∆ t2, t1
Intraindividual stability
How consistent is
individuals’ relative
standing in the skill
distribution over time?
Correlation of skills at two
time points
r t1, t2
Rank-order stability
We approached the question of stability and
change from two complementary perspectives
30
By how much do
individuals’ skills change
over time on average?
Difference scores
between skills at two
time points
∆ t2, t1
Intraindividual stability
How consistent is
individuals’ relative
standing in the skill
distribution over time?
Correlation of skills at two
time points
r t1, t2
Rank-order stability
Rank-order change (r t1, t2)
31
Literacy Numeracy
3 years
(PIAAC)
6 years
(NEPS)
3 years
(PIAAC)
6 years
(NEPS)
Full
Sample
0.85
[0.72]
0.62
[0.38]
0.82
[0.67]
0.71
[0.50]
Brackets: Amount of variance (%) shared by the two time points
Rank-order change (r t1, t2) in literacy
32
r = 0.62 r = 0.85
3 years 6 years
Rank-order change (r t1, t2) in numeracy
33
r = 0.70 r = 0.82
3 years 6 years
Rank-order change (correlations)
by age group
34
Literacy Numeracy
Age
group
3 years
(PIAAC)
6 years
(NEPS)
3 years
(PIAAC)
6 years
(NEPS)
24–34 0.86 0.64 0.84 0.71
35–54 0.85 0.61 0.82 0.70
> 55 0.81 0.56 0.80 0.66
Rank-order change (correlations)
by age group
35
0
0.2
0.4
0.6
0.8
1
24–34 35–54 > 55
Literacy
PIAAC (3 years) NEPS (6 years)
0
0.2
0.4
0.6
0.8
1
24–34 35–54 > 55
Numeracy
PIAAC (3 years) NEPS (6 years)
Rank-order change (correlations)
by Educational Attainment
36
Literacy Numeracy
Education 3 years
(PIAAC)
6 years
(NEPS)
3 years
(PIAAC)
6 years
(NEPS)
Low 0.83 0.59 0.82 0.40
Medium 0.83 0.60 0.79 0.65
High 0.81 0.59 0.77 0.71
Rank-order change (correlations)
by Educational Attainment
37
0
0.2
0.4
0.6
0.8
1
Low Medium High
Literacy
PIAAC (3 years) NEPS (6 years)
0
0.2
0.4
0.6
0.8
1
Low Medium High
Numeracy
PIAAC (3 years) NEPS (6 years)
We approached the question of stability and
change from two complementary perspectives
38
By how much do
individuals’ skills change
over time on average?
Difference scores
between skills at two time
points
∆ t2, t1
Intraindividual stability
How consistent is
individuals’ relative
standing in the skill
distribution over time?
Correlation of skills at two
time points
r t1, t2
Rank-order stability
The majority of adults’ skills change by less than
one standard deviation
39
< 1 SD
70%
< 1 SD
70%
< 0.5 SD
41%
< 0.5 SD
40%
< 1 SD
53%
< 0.5 SD
32%
< 1 SD
51%
< 0.5 SD
27%
3 years 6 years
Numeracy
Literacy
% of respondents who change +/- 0.5 SD or less
% of respondents who change +/- 1 SD or less
Intra-individual Change in literacy (∆ t2, t1) is
roughly normally distributed
40
3 years 6 years
Even among adults aged 40+, gains and losses
are almost equally likely!
41
Age 25–39 Age 40+
Intra-individual Change in numeracy (∆ t2, t1) is
roughly normally distributed
42
3 years 6 years
Literacy over age (cross-sectional, t1 only)
43
Chance in literacy (∆ t2, t1) over age
44
3 years 6 years
Numeracy over age (cross-sectional, t1 only)
45
Chance in numeracy (∆ t2, t1) over age
46
3 years 6 years
Change in Literacy (∆ t2, t1) over Education
47
3 years 6 years
Change in Numeracy (∆ t2, t1) over Education
48
3 years 6 years
Our findings portray skills as highly but not
perfectly stable
49
Literacy and numeracy are
roughly equally stable
Stability similar across
educational groups
Exception: 6-year stability of
numeracy lower among the
lower-educated
Further findings
Skills are highly rank-order
stable over 3 – but less so
over 6 years
Average skill gains during
young adulthood, losses
after age 35 to 40 years
Skills are not set like
plaster but show
considerable plasticity
Key findings
50
2 Predictors of change:
Participation in job-related training
Studies suggest that participation in job-related
training might foster adults‘ skills
Job-related training usually designed to foster job-specific skills
(rather than literacy or numeracy)
Still, cross-sectional findings suggest a substantial correlation between
(job-related) training and literacy or numeracy (OECD, 2013, Cegolon,
2015, Desjardins 2017)
51
289 294
276 277
250
260
270
280
290
300
2012 2015
Lite
racy S
co
res (
0-5
00)
Cross-sectional findings: PIAAC-L
Training No Training
Are there positive
spill-over effects of
job-related training
on more general
skills?
Purported training effects may arise from self-
selection or unobserved confounders
52
Literacy
t1
Literacy
t2
Job-related training
Possible confounders time-invariant or time-variant
observed or unobserved
What longitudinal data contributes to these
problems
Correct temporal ordering: Possible cause (training) precedes
purported effect (changes in literacy skills)
Control for previous level of literacy skills (t1) and estimate effect
of training on residual change in skills
Control for time-invariant third variable confounders by using
fixed-effects-methods
53
The relationship between job-related training and
literacy using longitudinal data
54
Data & Sample Methods
PIAAC 2012 & PIAAC-L
2015 two-wave panel
data
Native-speaking
employed individuals
N = 1,773
Relationship between
literacy and job-related
training (within the last
12 months)
Lagged-dependent-
variable (LDV) model to
control for t1 literacy
Fixed-effects (FE) model
to control for time-
invariant confounders
Instrumental-variables
(IV) approach to control
for other confounders
Selection models to
check reverse causality
Our analyses suggest no causal effect of job-
related training on literacy or numeracy skills
55
Literacy Positive cross-sectional association
confirmed (+ 0.14 SD)
No training effects on residual changes in
literacy (i.e. controlling for T1 literacy)
No training effects on changes in literacy
after accounting for unobserved confounders
and reverse causality
Reverse causality confirmed: higher literacy
skills increase likelihood to participate in
training (7 pp)
Possible confounders
Training
∆ Literacy Training
∆ Literacy Training
Training Literacy
What about training type, training intensity, and
subgroups?
56
Types of training: self-financed vs. employer–
financed training
employer–financed training has stronger
associations in OLS but not in panel models
Training intensity: number of trainings taken
No differential effects
Subgroups: „skills beget skills“ vs. „catch-up
effect“? (Cunha & Heckman, 2007; Blossfeld & von
Maurice, 2011)
Higher training effect for low-skilled individuals
in OLS but not in panel models
Again, no evidence for causal training effects
Our findings suggest that the link between job-
related training and skills reflects self-selection
57
Skill predict subsequent
participation in training
Higher skills also predict
higher training intensity
…but selection effects
No spill-over effects of
training on literacy or
numeracy
No effects of training type
or intensity
No differential effects for
subgroups (save skill level)
No training effects…
Training specifically designed to foster literacy / numeracy
skills likely a more viable option
58
3 Predictors of change:
Skill use
If training does not foster skills – does skill use?
59
Extent to which different literacy-
related and numeracy-related
skills are used
At work / on the job
Outside of work (e.g. at
home)
Examples:
Reading documents
Writing documents
Carrying out calculations
Skill use
Lagged-dependent-variable (LDV)
model predicting T2 literacy
Gender, age, level of education
(ISCED-97)
Skill use at home and at work
T1 literacy
Analyses
Literacy
skills t2
Literacy
skills t1
Skill Use
t1
More frequent use of skills prevents skill loss
60
PIAAC NEPS
Literacy t1 0.761*** (0.017) 0.567*** (0.021)
Female –0.013 (0.027) –0.031 (0.037)
Age (Ref:)
35-55 years –0.086* (0.036) –0.231*** (0.056)
> 55 years –0.153** (0.050) –0.478*** (0.070)
Education (Ref: high)
Low –0.193 (0.100) –0.195*** (0.040)
Medium –0.189*** (0.033) –0.299** (0.106)
Skill Use @home (Ref:low)
Medium 0.073 (0.038) 0.098 (0.068)
High 0.139*** (0.039) 0.146* (0.072)
Skill Use @work (Ref:low)
Medium 0.084 (0.042) 0.033 (0.044)
High 0.104* (0.042) 0.130** (0.047)
More frequent use of skills prevents skill loss
61
PIAAC NEPS
Skill Use @home
(Ref:low)
Medium 0.073 (0.038) 0.098 (0.068)
High 0.139*** (0.039) 0.146* (0.072)
Skill Use @work
(Ref:low)
Medium 0.084 (0.042) 0.033 (0.044)
High 0.104* (0.042) 0.130** (0.047)
More frequent use of skills prevents skill loss
62
0 0.05 0.1 0.15 0.2
high
medium
skill use@homereference: low
high
medium
skill use@workreference: low
NEPS
PIAAC
Our findings suggest that skill use matters for
skill retention (or growth)
63
Skill use may not fully offset
negative age effects
Results might also
represent reverse
relationship: high levels of
skills drive skill use
…but
Both, skill use at and
outside of work, are
positively related to skill
development
Incremental effects
Similar effect sizes for skills
use outside of work and at
work
Positive effects of
skill use…
64
4 Implications and outlook
Summary of key findings
65
No positive spill-over
effects from participation in
job-related training
Some evidence for practice
engagement effects
Predictors
Literacy and numeracy
skills are not set like plaster
in adulthood
Beyond age 35–40, skills
decline on average
Few subgroup differences
Substantial inter-
individual variability
Patterns
There are several open questions
we yet need to answer
Whence the variability in skill development?
General slowing hypothesis / biological ageing?
Practice engagement effects?
What are the consequences of skill loss?
For life outcomes (income, social participation, well-being, etc.)?
For learning ability?
Linear functions vs. critical thresholds?
What are the returns on investments in adults’ literacy and numeracy skills?
How and when to deliver interventions?
For what subgroups?
66
More longitudinal data would greatly benefit
research on adults’ skills
… more measurement occasions
Three or more measurement occasions would be ideal
Some factors might only affect skills over a longer time span
… additional skill domains
Digital /technological skills
More job-specific skill and skill use measures
…multiple countries
Comparing changes over time across country (diff-in-diff)
Role of educational systems and labor market regimes
67
Only the German follow-up study contains a re-
assessment of literacy and numeracy skills
68
?
? ?
?
Thank you for your attention!
References
70
Autor, D. H., Levy, F., & Murnane, R. J. (2003). The skill content of recent technological change: An empirical exploration. The Quarterly Journal of Economics,
118(4), 1279–1333.
Bloom, D. E., & Sousa-Poza, A. (2013). Ageing and productivity: Introduction. Retrieved from https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2219130
Blossfeld, H.-P., von Maurice, J., & Schneider, T. (2011). 1 The National Educational Panel Study: Need, main features, and research potential. Zeitschrift Für
Erziehungswissen-schaft, 14(2), 5–17.
Bynner, J., & Parsons, S. (1998). Use It or Lose It? The impact of time out of work on literacy and numeracy skills. ERIC. Retrieved from
http://eric.ed.gov/?id=ED430160
Carstensen, C.H., Gaasch, J. C., Rothaug, E. (2018). Scaling PIAAC-L cognitive data. Technical Report.
Cegolon, A. (2015). Determinants and Learning Effects of Adult Education-Training: a Cross-National Comparison Using PIAAC Data. U. I. o. Education.
University College London. Working Paper No. 15-11.
Cunha, F. and J. Heckman (2007). "The technology of skill formation." American Economic Review 97(2): 31-47.
Desjardins, R. (2015). Participation in adult education opportunities: Evidence from PIAAC and policy trends in selected countries.
Desjardins, R., & Warnke, A. J. (2012). Ageing and skills: a review and analysis of skill gain and skill loss over the lifespan and over time. OECD education
working papers. Retrieved from https://www.econstor.eu/handle/10419/57089
Grotlüschen, A., Mallows, D., Reder, S., & Sabatini, J. (2016). Adults with low proficiency in lit-eracy or numeracy (OECD Education Working Papers No. 131).
Hanushek, E. A., & Woessmann, L. (2015). The knowledge capital of nations: Education and the economics of growth. MIT Press.
Himmler, O., & Jaeckle, R. (2018). Literacy and the Migrant–Native Wage Gap. Review of Income and Wealth, 64(3), 592-625.
Jarvis, P. (2007). Globalization, lifelong learning and the learning society: Sociological perspec-tives. Routledge.
Murray, S. (2009). Longitudinal research related to the acquisition and maintenance of literacy. In S. Reder & J. Bynner (Eds.), Tracking adult literacy and
numeracy skills: Findings from longitudinal research (pp. 85–106). New York: Routledge.
OECD. (2013). OECD Skills Outlook 2013. Paris: Organisation for Economic Co-operation and Development. Retrieved from http://www.oecd-
ilibrary.org/content/book/9789264204256-en
Paccagnella, M. (2016). Age, ageing and skills. OECD Education Working Papers, 132. Re-trieved from http://www.oecd-ilibrary.org/education/age-ageing-and-
skills_5jm0q1n38lvc-en
Rammstedt, B., Danner, D., & Lechner, C. M. (2017). Personality, competencies, and life out-comes: Results from the German PIAAC longitudinal study. Large-
Scale Assessments in Education, 5(2), 1–19.
Reder, S. (2009). "Scaling up and moving in: Connecting social practices views to policies and programs in adult education." Literacy and Numeracy Studies
16(2): 35-50.
Weinert, S., Artelt, C., Prenzel, M., Senkbeil, M., Ehmke, T., & Carstensen, C. H. (2011). 5 De-velopment of competencies across the life span. Zeitschrift Für
Erziehungswissenschaft, 14(S2), 67–86.
