The CEFR for teaching and assessing young learners

LTF, 21 Nov 2009CRELLA, University of Bedfordshire, Luton

Szilvia Papp, Hanan Khalifa, Nick ChargeUniversity of Cambridge ESOL Examinations

2Aim of Talk

3Questions1. the construct behind YLE

a) linguistic progressionb) cognitive developmentc) educational, social, psychological aspects

2. the link between YLE and the CEFR3. CEFR proficiency levels reflected in YLE4. salient features of YLE performance in CEFR5. additional salient features of YLE performance not

in CEFR

4Tests for YLLs need to be supporting but challenging enough to cater for all

cognitive and emotional needs on both ends of the spectrum

build on and reflect previous experience of language learning

test language use rather than language knowledge provide children with

a range of social situations and sufficient breadth of experience with language

cover a wide enough sample of language

(Simon Smith 2009, University of York)

5Question 1What is the construct behind Young Learners of

English tests?a) What linguistic progression do the YLE tests reflect?b) How do the tasks reflect the cognitive development of YLE candidature?c) What educational, social, psychological aspects of young language learners are catered for in YLE tests?

(Papp 2007, 2008a)

6Linguistic progression Limitation to lexical, structural, pragmatic

development due to age Similar route, but different mechanisms, rate and

ultimate attainment among child L2 learners, depending on age of onset of learning

Communication strategies Language learning strategies Basic interpersonal communication skills (BICS) Cognitive academic language proficiency (CALP)

(Cummins 2001)

7Linguistic progressionDifferences in language development based on age

can manifest themselves in variations in: accuracy, fluency, complexity use of formulaic sequences interactional patterns and interactional moves accent content, topic, range of notions that the candidate is able to,

wants to or needs to encode different goals the presence of specific grammatical features in the ILG

(Nicholas and Lightbown 2008, Philp, Oliver and Mackey 2008)

8

9Cognitive development Strategies for working memory use Abstract concept formation Reasoning abilities Metalinguistic awareness Executive control Mind reading abilities

(Meadows 2006)

10

11

Educational, social, psychological variables

Amount & quality of input, exposure to, experience with L2

Social, cultural and educational context of interaction Development of literacy in L1 (different writing

systems) Theory of mind (use of complex discourse &

narrative skills) Individual differences Scaffolding

(Cameron 2003)

12

Task demands and scaffolding

13

SummaryAge-related differences between young learners which

have an impact on their L2 development: improved cognitive abilities related to growing attentional and

memory capacities changing social and psychological make-up, such as

fluctuating self-confidence, self-generated and peer-influenced interests, motivation, related affective schemata

unique experiential characteristics, such as differential exposure to the target language, differential acceptance and participation in activities and task types, etc.

(McKay 2006)

14

Question 2What is the demonstrable link between YLE test and

task specifications, YLE candidate performance and the relevant CEFR scales and descriptors in terms of a) proficiency levelsb) activities, strategies and competences YLE candidates

can be expected to do or have?

15

Link between YLE and CEFRYLE linked to CEFR by design through test

and task specifications (1992-1997 test development, 2002-2007 revision):

Flyers at A2 (KET) Movers at A1 Starters below A1

The link has been demonstrated empirically and qualitatively:

equivalence and equating studies formal alignment process in May 2008 next equivalence study in March 2010

(Gardiner 2000, Flux 2001, Jones 2001, Jones 2002a,b,c, Burrow 2004, Barker and Shaw 2007, Papp and Salamoura 2009)

16

-3.74

1-3.41

-3.19

-2.97

-2.75

-2.53

-2.31

2-2.09

Fail-1.87

Narrow Fail-1.65

-1.32

-0.99

Pass3-0.66

-0.33

0.22

40.77

Merit51.65

KET gradeFlyers Listening

shields

Logits

-1.54

Narrow Fail

-1.43

-1.32

-1.21

-1.10

-0.88

-0.77

-0.66

Pass3-0.44

-0.33

-0.11

0.00

0.33

0.55

0.88

41.32

Merit52.09

KET grade

Flyers Reading

& WritingShields

Logits

(Gardiner 2000, Flux 2001)

171.32

1.43

21.54

1.65

1.76

1.87

1.98

2.09

2.20

2.31

2.42

2.53

2.64

432.86

2.97

3.19

3.41

3.74

4.07

54.18

44.51

55.16

Movers Reading &

Writing shields

Flyers Reading &

Writing shields

Logits

-1.21

-0.99

-0.77

-0.55

-0.33

21-0.11

0.11

0.33

0.44

0.66

0.88

321.10

1.32

1.54

1.76

2.09

432.31

2.75

3.19

543.96

Movers Listening

shields

Flyers Listening

shields

Logits

(Gardiner 2000, Flux 2001)

18

Activities, strategies and competences of YLE candidates

The relationship between Starters and Movers and Breakthrough level investigated by a consultant in 2003.

In this project, the lexical content and topic areas of YLE tests were mapped against the preliminary Breakthrough objectives.

Results: substantial areas of overlap between Starters and Movers &

Breakthrough in terms of lexis and topic areas, however, YLE topic areas are expressed in more concrete

terms such as School & the Classroom rather than Education.

(Burrow 2004)

19

Verifying YLE alignment to the CEFR

A2A1+Pre-A1/A1CEFR level

3.062.311.67Average (all skills)

3.102.631.79Reading3.132.462.17Listening2.941.851.06SpeakingYLE FlyersYLE MoversYLE StartersSkill

Formal alignment/linking exercise May 2008: CEFR ratings for levels& skills

(Papp and Salamoura 2009)

20

Question 3How do the YLE tests embody and reflect the CEFR?

21

Describing YLE exams using CEFR? trade-off between control of candidate output and

authenticity of tasks, not reflected in CEFR scaffolding of tasks absent in CEFR learning to learn, enabling skills very sparse in CEFR limits to childrens cognition, linguistic progress,

psychological, emotional, social development not acknowledged in CEFR: no task should tax childrens cognition by requiring them to

deal with multiple perspectives other than their own effect of writing on the reader should be taken into account

in marking criteria (Papp and Salamoura 2009)

22

Question 4What are the salient features of YLE candidate

performance that reflect the CEFR in terms ofa) proficiency levelsb) activities, strategies and competences YLE candidates

can be expected to do or have?

23

Salient features of YLE candidatesperformance: listening and speaking

Starters (pre-A1) candidates performance: Ability to rely purely on a very finite, rehearsed repertoire of phrases, frequently used routines and patterns

limited to the performance of isolated tasks in specific situations, i.e. a list of pedagogic tasks in a primary school setting.

Movers (A1) candidates performance: Ability to interact in a simple way, initiate and respond to simple statements in areas of immediate need or on

very familiar topics (ask and answer simple questions about themselves, where they live, people they know, and things they have, etc.).

Flyers (A2) candidates performance: Ability to handle social functions (greet people, ask how they are and react to news; handle very short social

exchanges; ask and answer questions about what they do at school and in free time; make and respond to invitations; discuss what to do, where to go and make arrangements to meet; make and accept offers).

Ability to perform simple classroom tasks in English.

Strong Flyers (A2+) candidates performance: Ability to more actively participate in conversation. Ability to sustain monologues.

(CEFR 2001)

24

Question 5Which additional salient features can be identified in

YLE candidate performances that are not currently covered in the CEFR?

25

Childrens literacy: reading and writingThere is a need to list childrens L2 reading abilities, activities, strategies and

competences investigate emerging L2 writing abilities (e.g. copying,

handwriting, spelling and other enabling skills) list childrens specific L2 writing activities, strategies and

competences in order to identify salient features of typical/likely candidate

performance at each level of YLE not currently in CEFRThis requires additional work

on sample performances from YLE reading & writing tests with children in the classroom (ECML) review of materials used in classrooms across the world

26

Recommendations for future researchFor Cambridge ESOL further explore salient features of YLE performance, esp for

childrens reading and writing typical in & outside classroom develop Can Do statements for YLE candidates (aged 712)

to complement those for 1114 year-old school learners taking KET/PET for Schools examinations

further work on link between summative and formative assessment (assessment for learning)

For other YLL test providers carry out a similar linking exercise in order to

define and explicitly state construct tested generate evidence of CEFR relatedness identify qualities of YLL tests claiming links to the CEFR raise awareness of good testing practices

27

Recommendations for future researchFor all YLL test providers provide feedback to CoE on linking process for relating

exams such as the YLE tests to the framework of reference provide calibrated samples to CoE of childrens performance

in tests thus related to the CEFR answer the question that remains whether it is necessary to

carry out a whole-scale adaptation of current CEFR scales to reflect young learner needs and contexts or

start developing scales from scratch to reflect the nature of second language learning and use among children?

28

Selected referencesBarker, Fiona and Stuart Shaw (2006) YLE Vertical Linking Project. Validation Project No 171. Cambridge ESOL internal

document.Barker, Fiona and Stuart Shaw (2007) Linking language assessments for younger learners across proficiency levels (Phase 1).

Research Notes 28: 14-18. UCLES: Cambridge ESOL. Burrow, Trish (2004) Exploring the relationship between YLE Starters and Movers and Breakthrough level. Research Notes 16:

6-7. UCLES: Cambridge ESOL.Gardiner, Kerri (2000) KET / Flyers Research Project: Analysis of data. Research and Validation Report No 106/806. Cambridge

ESOL internal document. Flux, Tracy (2001) KET / YLE Link Project 2001. Research and Validation Report No 246. Cambridge ESOL internal document.Jones, Neil (2001) Reliability of YLE: Dependability of grades for a high-facility exam. Research and Validation Report No 316.

Cambridge ESOL internal document.Jones, Neil (2002a) Equating YLE levels: A study based on candidates taking two exams. Research and Validation Report No

318. Cambridge ESOL internal document.Jones, Neil (2002b) Linking YLE levels into a single framework. Research Notes 10: 14-15. UCLES: Cambridge ESOL. Jones, Neil (2002c) A Framework for YLE: Proposals for an integrated grade system and implications for calibration and grading.

Research and EFL Validation Report No 439. Cambridge ESOL internal document.Papp, Szilvia (2007) The Cambridge YLE tests in the light of cognitive, linguistic, educational, social-psychological and cultural

aspects of childrens L2 development and the CEFR, internal Cambridge ESOL Research and Validation report.Papp, Szilvia (2008a) Factors influencing L2 development and use in the 814 age group towards defining the construct,

Cambridge ESOL internal Research and Validation report.Papp, Szilvia (2008b) Skills profiles by age in KET and PET, Cambridge ESOL internal Research and Validation report.Papp, Szilvia (2008c) Coverage of Waystage and Threshold learning objectives within KET and PET and KET and PET for

Schools, Cambridge ESOL internal Research and Validation report.Papp, Szilvia (2008) Quantitative linking YLE to the CEFR summary of empirical studies to date, internal Cambridge ESOL

Research and Validation report.Papp, Szilvia (2009) Development of Can-do statements for KET and PET for Schools, Research Notes 36, 812, Cambridge:

Cambridge ESOL.Papp, Szilvia and Salamoura, Angeliki (2009) An exploratory study linking young learners examinations to the CEFR, Research

Notes 37, 1522,Cambridge: Cambridge ESOL.Trim, J L M (2001b) Breakthrough, unpublished manuscript, now available on the English Profile website.van Ek, J and Trim, J L M (1990a/1998a) Threshold 1990, Cambridge: Cambridge University Press.van Ek, J and Trim, J L M (1990b/1998b) Waystage 1990, Cambridge: Cambridge University Press.