Upload
abigayle-edwards
View
215
Download
2
Embed Size (px)
Citation preview
Kaftandjieva & Takala
Kaftandjieva & Takala
What is the structure of What is the structure of the space (if it exists) of the space (if it exists) of ‘Can Do’ statements?‘Can Do’ statements?
Kaftandjieva & Takala
The Universe: The Universe: Reading ‘Can Dos’Reading ‘Can Dos’
ALTE ALTE –– 126 126ALTE ALTE –– 126 126ELP – 123ELP – 123ELP – 123ELP – 123
YKI YKI –– 60 60YKI YKI –– 60 60
DIALANG DIALANG –– 36 36DIALANG DIALANG –– 36 36
EOI EOI –– 40 40EOI EOI –– 40 40
CEF – 56CEF – 56CEF – 56CEF – 56
New Set – 54New Set – 54New Set – 54New Set – 54
Kaftandjieva & Takala
The Universe: Levels of Language Proficiency
0102030405060708090
100110120130
A1 A2 B1 B2 C1 C2
New Set
EOI
YKI
DIALANG
ALTE
ELP
CEF
Kaftandjieva & Takala
The GuideThe Guide
Facet theoryprovides an approach for proposing
a "definitional system“, or a model, to describe a universe of observations
a taxonomic device a test of structural hypotheses
Kaftandjieva & Takala
Facet – "a classification of the item domains of a given content universe according to some rule" (Shye,1978)the population facet – foreign language
learnersthe common range facet – CEF levelsthe facets of the content domain
Mapping sentence – “a verbal statement of the domain and of the range of a mapping including connectives between facets as in ordinary language” (Shye, 1978)
The GuideThe Guide
Kaftandjieva & Takala
The HitchhikersThe Hitchhikers
Exact agreement:61% – 88%Mean = 78%
Kappa0.40 – 0.87Mean = 0.60
Contingency coefficient0.58 – 0.77Mean = 0.70
Can
Can
Can
Can
Can
Can
Can
Can
Can CanCan
Can
Can
Can
Can
Can
Can
Can
Kaftandjieva & Takala
Facets: Initial SetFacets: Initial Set
PurposePurpose
Text LengthText
Length
InformationInformation ComprehensionComprehensionFieldField
ModifierModifier
Text Complexity
Text Complexity
VerbVerb
HelpHelp Text Source
Kaftandjieva & Takala
PurposePurpose
Text LengthText
Length
InformationInformation ComprehensionComprehensionFieldField
ModifierModifier
Text Complexity
Text Complexity
VerbVerb
HelpHelp Text Source
Facets: Transitional SetFacets: Transitional Set
Kaftandjieva & Takala
The SpaceThe Space
Homogeneity analysisDimension 1: 64%Dimension 2: 33%
Correlation with CEFComprehension: 0.51Field: 0.57Text length: 0.71 Text complexity: 0.80
Discrimination Measures
0
0,1
0,2
0,3
0,4
0,5
0,6
0,7
0,8
0 0,2 0,4 0,6 0,8 1
Comprehension
Field
Complexity
Length
Kaftandjieva & Takala
Space StructureSpace Structure
-1,4 -1,2 -1,0 -0,8 -0,6 -0,4 -0,2 0,0 0,2 0,4 0,6 0,8 1,0 1,2 1,4 1,6 1,8 2,0-1,0
-0,8
-0,6
-0,4
-0,2
0,0
0,2
0,4
0,6
0,8
1,0
1,2
1,4
1,6
A1
A2
B1
B2
C1C2
In Part
In General
InDetail
Comprehension
Dimension 1
Dim
ensi
on 2
-1,4 -1,2 -1,0 -0,8 -0,6 -0,4 -0,2 0,0 0,2 0,4 0,6 0,8 1,0 1,2 1,4 1,6 1,8 2,0-1,0
-0,8
-0,6
-0,4
-0,2
0,0
0,2
0,4
0,6
0,8
1,0
1,2
1,4
1,6
A1
A2
B1
B2
C1C2
Short
Average
Long/any
Text Length
Dimension 1
Dim
en
sio
n 2
-1,4 -1,2 -1,0 -0,8 -0,6 -0,4 -0,2 0,0 0,2 0,4 0,6 0,8 1,0 1,2 1,4 1,6 1,8 2,0-1,0
-0,8
-0,6
-0,4
-0,2
0,0
0,2
0,4
0,6
0,8
1,0
1,2
1,4
1,6
A1
A2
B1
B2
C1C2 Simple
Average
Compl/any
Text Complexity
Dimension 1
Dim
en
sio
n 2
-1,4 -1,2 -1,0 -0,8 -0,6 -0,4 -0,2 0,0 0,2 0,4 0,6 0,8 1,0 1,2 1,4 1,6 1,8 2,0-1,0
-0,8
-0,6
-0,4
-0,2
0,0
0,2
0,4
0,6
0,8
1,0
1,2
1,4
1,6
A1
A2
B1
B2
C1C2
Everyday
Own field
Any field
Field
Dimension 1
Dim
ensi
on 2
Kaftandjieva & Takala
Space CoordinatesSpace CoordinatesA1A2B1B2C1C2
Kaftandjieva & Takala
Lost in SpaceLost in SpaceA1A2B1B2C1C2
Can understand a restaurant menu including a wide range of culinary terms. (ALTE 035 – C2)
CAN understand the general meaning of a simplified textbook or article, reading very slowly. (ALTE 035 – A1)
Kaftandjieva & Takala
Found in SpaceFound in Space
Ordinal regressionObjects
Reading ‘Can Dos’Dependent variable
CEF levelsFactors
(x, y) coordinates
PredictionR2 = 0.82r = 0.85 (Spearman)Kappa = 0.43Exact agreement – 53%
The best – A2The worst – A1
Agreement (±1) – 94%Mean difference – 0.56
Kaftandjieva & Takala
Lost and FoundLost and FoundALTE Can Do Project:
http://www.alte.org/can_do/alte_cando.pdf
Can read quickly enough to cope with the demands of an academic course. (p. 15: C1)
Can read quickly enough to cope with the demands of an academic course. (p. 90: C2)
Can understand the general meaning of a simplified textbook or article, reading very slowly. (p. 90: A1)
Can understand the general meaning of a simplified textbook or article, reading very slowly. (p. 15: A2)
Kaftandjieva & Takala
Difference between actual and predicted CEF levels
0%
25%
50%
75%
100%
CEFELP
ALTE
DIALA
NGYKI
EOINew
4
3
2
1
0
0: 42%1: 91%
Lost and FoundLost and Found
0: 67%1: 98%
0: 46%1: 83%
Can understand a restaurant menu including a wide range of culinary terms. (ALTE 035 – C2)
Kaftandjieva & Takala
Text LengthText
LengthComprehensionComprehension
FieldFieldText
ComplexityText
Complexity
Facets: Transitional SetFacets: Transitional Set
Kaftandjieva & Takala
Mapping Sentence: Mapping Sentence: Transitional Stage Transitional Stage
A person {p} who can understand information in1. in part2. in general3. in detail
complexity on topics concerning 1. everyday situations2. his/her own field3. any field
is at least on
Range:A1A2B1B2C1C2
CEF level of language proficency.
length, in a language of
1. short2. average3. long / any
a text of 1. simple2. average3. high / any
Kaftandjieva & Takala
A person {p} who can understand information1. in part2. in general3. in detail
on a topic concerning 1. everyday situations2. my own field of interest3. any field
is at least
in terms of length and/or complexity 1. restricted2. not restricted
in a text
Range:A1A2B1B2C1C2
CEF level.on
Mapping Sentence: Mapping Sentence: Final Stage Final Stage
Kaftandjieva & Takala
I {p} can understand information in a text 1. in part2. in general3. in detail
in terms of length and/or complexity 1. restricted2. not restricted
on a topic concerning 1. everyday situations2. my own field of interest3. any field
Range:
0. NO1. YES
Mapping Sentence: Mapping Sentence: Final Stage Final Stage
Kaftandjieva & Takala
Mapping Sentence: Mapping Sentence: Final Stage Final Stage
Samplen1 = 346 (Bg)n2 = 524 (Fi)
Reliability1 = 0.942 = 0.95
Rasch modelr12 = 0.85
Eta: bi dependentComprehension: 0.50Text: 0.62Field: 0.52CEF: 0.87 (R = 0.86)
-1,5 -1,0 -0,5 0,0 0,5 1,0 1,5
-1,0
-0,5
0,0
0,5
1,0
1,5
2,0Pattern Disposion
323
322
321313
312
311223
222
221
213
212
123
122
121
113
112
211
111
Dim
en
sio
n 2
Dimension 1
Kaftandjieva & Takala