Task-internal and task-external readiness:

Task-internal and task-external readiness:

A report of the effects of topic familiarity and strategic planning on task performance by L2 learners of

different proficiency levels

Gavin Bei Xiaoyue

The Chinese University of Hong Kong

[email protected]

3rd Biennial International Conference on Task-Based Language Teaching, Lancaster University, Sept 15, 2009

Part 1

Research background

Contextualizing

• Task-based instruction research looks at:• 1. Task characteristics: • Subjective or objective, structural or non-structural,

familiar or unfamiliar…• 2. Task conditions:• Monologic or interactive, Pre/Post task activities,

planning or non-planning…• 3. Participants:• Gender, motivation, learning style, proficiency…

Research background 1: topic familiarity

• 1. Comprehension (many)• facilitative• e.g., Shimioda, 1993; Barry and Lazarte, 1995; Bügel &

Buunk, 1996; Chen and Donin, 1997; Johnson, 1982; Lee, 1986; and Chang, 2006

• no effect on comprehension • e.g., Hammadou, 1991; Peretz & Shoham, 1990; and Carrell

(1983)

• 2. Production (few)

• Mostly in L1 research by psychologists

• Higher fluency, but inconsistent in accuracy or complexity

Research Background 2: planning types

• Two macro and four micro types of planning (Ellis, 2005)• 1. pre-task• 1) rehearsal 2) strategic planning• 2. within-task• 1) pressured 2) unpressured

• Or simply three micro types (Ellis, in press)• 1) rehearsal 2) strategic planning 3) within-task planning

Research background 3: strategic planning

• Ample studies (e.g., Skehan, Foster, Ellis, Crookes, Wigglesworth, etc.) with quite some consistent results.

• Planning raises: Fluency + Complexity• (sometimes, but usually not) Accuracy• Skehan: Trade-off of between Comp. and Accu. • Robinson: Planning does not lead to Comp, no

trade-off.• Is proficiency important here?

Research background 4: Proficiency and Familiarity

• Hudson (1982):

• In Reading: Familiarity > Proficiency.

• Schmidt-Rinehart (1994):

• In Listening: Familiarity > Proficiency.

• Carrell (1983):• In Reading: Proficiency > Familiarity (NS:NNS)

• Chern (1993):

• In Reading: Proficiency > Familiarity.

Research background 5: proficiency and planning

• Wigglesworth (1997): low proficiency did not benefit from planning.

• Tavokoli and Skehan (2005): planning drove high and low learners for better performance.

• Kawauchi (2005): more Flu. and Comp. for higher learners, more Accu. for the lower. The advanced gained the least.

• Most other studies did not consider proficiency.

Part 2

The study design

and methodology

1.Participants and proficiency test

• Participants: 80 HK Cantonese-speaking undergraduates volunteers to participate.

• A C-test as proficiency test to group participants

------- borrowed from Dornyei and Katona (1992).--------The validity and reliability are good in the

literature and in the present context. See appendix 1

2. Tasks

• Topic 1: Natural Viruses.

• Topic 2: Computer Viruses.

Topic 1 Topic 2

Medicine Majors (N=40)

+ familiar – familiar

Computer Majors (N=40)

– familiar + familiar

3. Independent Variables

• 1. Topic familiarity (within): 2 levels

• familiar VS unfamiliar task

• 2. Planning (between): 2 levels

• non-planning VS 10-min planning

• 3. Proficiency (between): 2 levels

• intermediate VS high

Study design

Planning(between)

Proficiency(between)

Topic familiarity (within)

Familiar Unfamiliar

Planners

High 20 20

Intermediate 20 20

Non-planners

High 20 20

Intermediate 20 20

Each cell consists of 10 computer majors and 10 medicine majors as counterbalancing to rule out the topic effect.

4. Dependent Variables

• Fluency: pausing, speech rate, MLR, phonation time, repairs, etc.• Accuracy: error-free clauses ratio, length of correct clause, and

errors per 100 words.• Complexity: Clauses per AS unit, AS unit length, and clause

length• Lexis: lexical diversity, lexical sophistication, and lexical density.• Formality: F-score, DB-score• Totally 21 measure were employed. See Appendix 2 for a detailed

description.

• P value: the significance level to tell whether there is an effect.• Cohen’s D value: the effect size to tell how big the effect is.

5. Statistical procedures

• A repeated measures analysis of variance (ANOVA) was employed.

Part 3

Results

Breakdown fluency main effects 1

T. Familiarity Planning Proficiency

p D p D p D

Speech rate .000 .26 .000 .58 ns /

Phonation time .000 .35 .000 .62 ns /

Mean length of run .016 .17 .046 .32 ns /

No. Mid-clause pauses .000 .38 .001 .54 ns /

No. End-of-clause pauses ns / ns / .027 .48

Table 1. p and Cohen’s D value

Means omitted due to the space limit. The means show that the familiar topics and the planning time improve fluency. Same directions below unless there is a note.

Breakdown fluency main effects 2


p D p D p D

Mid-clause silence total .000 .38 .000 .61 ns /

End-of-clause silence ttl. .014 .19 .007 .59 ns /

Av. mid-clause pause .019 .28 .000 .71 ns /

Av. End-of-clause pause ns / .004 .64 ns /

No. filled pauses .085 / ns / ns /


• T. Familiarity has significant interactions with Planning in:

• 1) speech rate• 2) phonation time• 3) No. Mid-clause pauses• 4) Mid-clause silence total (per 100 words)• 5) End-of-clause silence total (per 100 words)

Breakdown fluency: interactions

All showing one pattern: planning compensates for the unfamiliar topics.

Measures: speech rate and mid-clause silence total

Breakdown fluency: a summary

1) T. Familiarity affects fluency in a strikingly similar way as Planning does.

2) Approximately, the effect sizes of T. Familiarity is half as big as those of Planning.

3) Planning mitigate the difference between familiar and unfamiliar topics.

4) The effects of Proficiency is marginal, and probably overridden by T. F. and Planning.

Repair fluency


p D p D p D

False starts ns / .000 1.02 ns /

Reformulations .088 / .001 .53 ns /

Replacements .077 / .008 .43 ns /

Repetitions .001 .40 .000 .75 ns /


Note: planning induced more replacements, though reducing others.

Accuracy



p D p D p D

Error-free clauses ratio

.02 .22 ns / .000 .69

70% accuracy clause length

ns / ns / .000 .57

Errors per 100 words

.000 .38 ns / .000 .77

Complexity



p D p D p D

Clauses per AS unit

ns / .018 .39 .067 /

Words per AS unit ns / .000 .81 .000 .52

Words per clause ns / ns / ns /

Lexis



p D p D p D

Lexical diversity .018 .29 ns / ns /

Lexical sophistication .000 .41 ns / ns /

Lexical density ns / .008 .43 .031 .39

Formality



p D p D p D

F-score .000 .49 .003 .48 ns /

DB-score ns / .019 .38 ns /

Part 4

Some conclusions

Conclusions 1

• 1. Planning is more powerful in driving fluency than T. Familiarity. It can reduce the differences between familiar and unfamiliar topics in breakdown fluency.

• 2. Topic familiarity and planning seem to be more concerned with meaning expression (similar).

• 3. T. familiarity and planning affect different syntactic areas (different).

• 4. Proficiency affects mostly forms, esp. accuracy, but not so much meaning expression (fluency and lexis).

• 5. Higher proficiency does not appear to remove the trade-off effects. So L2 learners are L2 learners!

Conclusion 2

• 6. Factor analyses of all measures show:• 1) there’s probably an end-of-clause fluency different

from breakdown and repair fluency.• (Av. Pause and total silence at the end of a clause, and

phonation time. )• 2) there’s probably a noun-phrase complexity as

compared to the syntactic complexity. • (words per clause, F-score, DB-score, and Lambda)

• 6. A broader perspective on planning stems from the similarities and differences between T. familiarity and strategic planning in this study, in which I argue that T.F. can be regarded as a kind of implicit planning (see next page).

A general framework of task-readiness

Macro-dimension

Micro-dimension Sample studies

Learnerreadiness

fora task

Task-internal readiness (implicit planning)

Topic familiarity (prior domain knowledge)

This study

Schematic familiarity (story structure)

Skehan and Foster (1999)

Task familiarity (task types)

Bygate (2001)

Task-external readiness (explicit planning)

Rehearsal ( content repetition)

Bygate (1996)

Strategic (pre-task) planning

Foster and Skehan (1996)

Within-task (on-line) planning

Yuan and Ellis (2003)

Thank you!

Q and A

Appendix 1: reliability and validity of C-test

Dornyei and Katona (1992) found that the C-test is reliable (the internal consistency coefficients are very consistent, .75 and .77 respectively, for university English majors and secondary students) and valid (C-test is significantly highly correlated with different other proficiency tests like the General Language Proficiency and TOEIC). Cronbach’s alpha reached .84 in Daller and Phelan (2006). Klein-Braley and Raatz (1984), Klein-Braley (1985), Cohen, Segal and Bar-Siman-Tov (1984), Klei-Braley (1997), and Grotjahn, 1995 generally supported such a claim on written tasks. More importantly here, the C-test was reported to be highly correlated with oral tasks as well in recent studies (e.g., r=.64 in Arras, Eckes and Grotjahn, 2002, and also in oral lexical performance in Daller and Xue, 2007). More recently Dai (this conference) reported in Chinese context, Cronbach’s Alpha=.770, Concurrent validity r= .633, p<0.01 (correlated with CET-4). In this study, the Cronbach Alpha is=.64 in the pilot study, but =.74 in the main study.

back

Appendix 2: description of dependent variables

General Category Variable Name Description

Fluency

Pausing The number of pauses and the amount of silence. In the present study it is operationalized as any break of 0.4 second or longer.

Repair Fluency This measure is orthogonal to breakdown fluency and should be treated separtely. In the present study it is operationalized as the total number of repetitions, replacements, false starts and reformulations.

Speech Rate A pruned speech rate is investigated here because it shows the ‘real’ speed of the speaker. It is operationalized as the total words per minute after deletion of reformulations, replacements, false starts, repetitions, pauses and silence total.

Mean Length of Run The number of words uttered before any breakdown or repair fluency is encountered.

Phonation time The ratio of voicing time to the total time of utterance.

Accuracy

Error-free Ratio The ratio of error-free clauses to all clauses.

Errors per 100 Words The number of errors in every pruned one hundred words.

Length Accuracy The length of a clause with 50% of all clauses of the same length correct is set as the cut-off point beyond which the participant cannot produce correct clause at 50% level.

Complexity

Subordination Ratio The ratio of subordinate clauses per AS unit.

Words Per AS Unit The average word number in all AS units.

Words Per Clause The Average word number in all clauses

Lexis

Lexical diversity: the D value

Corrected Type-token ratio, an index of the extent to which the speaker avoid returning to the same set of words.

Lexical sophistication: the Lambda value

The extent to which speech contains difficult or rare words.

Lexical Density The ratio of content words to the total words.

Formality F-score (Noun frequency+adjective freq.+preposition freq.+article freq.-pron. Freq. –verb freq.-adverb freq. – interjection freq. +100) / 2

From F. Heylighen and J. Dewaele (1999).

DB-score The ‘involved’ style words in Biber, Conrad and Reppen (1998).

back

ReferencesBarry, S. & Lazarte, A. (1995). Embedded clause effects on recall: Does high prior knowledge of

content domain overcome syntactic complexity in students of Spanish? Modern Language Journal, 79, 491–504.

Biber, D., Conrad, S. & Reppen, R. (1998). Corpus Linguistics: Investigating Language Structure and Use. Cambridge: Cambridge University Press.

Bügel, K. & Buunk, B. (1996). Sex differences in foreign language text comprehension: The role of interests and prior knowledge. Modern Language Journal, 80, 15–31.

Bygate, M. (1996). Effects of task repetition: appraising the developing language of learners. In Jane Willis & Dave Willis (Eds). Challenge and change in language teaching. Oxford : Heinemann.

Bygate, M. 2001. Effects of task repetition on the structure and control of oral language [A]. In M. Bygate, P. Skehan & M. Swain (eds). Researching Pedagogical Tasks: Second Language Learning, Teaching and Testing [C]. Harlow, England: Longman.

Carrell, P. L. (1983). Three components of background knowledge in reading comprehension. Language Learning, 33, 183-207.

Chang, C. (2006). Effects of topic familiarity and linguistic difficulty on the reading strategies and mental representations of non-native readers of Chinese. Journal of Language and Learning, 4, 172-198.

Chen, Q. & Donin, J. (1997). Discourse processing of first and second language biology texts: Effects of language proficiency and domain-specific knowledge. Modern Language Journal, 81, 209–227.

Chern, C. (1993). Chinese students’ word-solving strategies in reading in English. In T. Huckin, M. Haynes, & J. Coady (Eds). Second language reading and vocabulary learning. Pp. 67–82. Westport, CT: Ablex.

Crookes, G. (1989). Planning and interlanguage variation. SSLA, 11, 367-383.Ellis, R. (2005). Planning and task-based performance: Theory and research. In R. Ellis. (Ed).

Planning and task performance in a second language. Amsterdam: Johan Benjamins.

Ellis, R. (in press). The Differential Effects of Three Types of-Task Planning on the Fluency, Complexity and Accuracy in L2 Oral Production. Applied Linguistics.

Hudson, T. (1982). The effects of induced schemata on the “short-circuit” in L2 reading: non-decoding factors in L2 reading performance. Language Learning, 32/1, 1-31.

Hammadou, J. (1991). Interrelationships among prior knowledge, inference, and language proficiency in foreign language reading. The Modern Language Journal,75: 27-39.

Johnson, P. (1982). Effects on reading comprehension of language complexity and cultural background of text. TESOL Quarterly, 16, 169–181.

Kawauchi, C. (2005). The effects of strategic planning on the oral narratives of learners with low and high intermediate proficiency. In R. Ellis. (Ed). Planning and task performance in a second language. Amsterdam: Johan Benjamins.

Lee, J. F. (1986). Background knowledge and L2 reading. Modern Language Journal, 70, 350–354.Peretz, A., & Shoham, M. (1990). Testing reading comprehension in LSP: Does topic familiarity

affect assessed difficulty and actual performance? Reading in a Foreign Language, 7, 447–455.Shimoda, T. A. (1993). The effects of interesting examples and topic familiarity on text

comprehension, attention, and reading speed. Journal of Experimental Education, 61, 93-103.Schmidt-Rinehart, B. C. (1994). The effects of topic familiarity on second language listening

comprehension. The Modern Language Journal, 78, 179-189.Skehan, P. & Foster, P. (1999). The influence of task structure and processing conditions in narrative

retellings. Language Learning, 49 (1): 93-120.Tavakoli, P. & Skehan, P. (2005). Strategic planning, task structure, and performance testing. In R.

Ellis. (Ed). Planning and task performance in a second language. Amsterdam: Johan Benjamins. Foster, P. & Skehan, P. (1996). The influence of planning and task type in second language

performance. SSLA, 18, 299-323.Wigglesworth, G. (1997). An investigation of planning time and proficiency level on oral test

discourse. Language Testing, 14 (1): 85-106.Yuan, F. & Ellis, R. (2003). The effects of pre-task and on-line planning on fluency, complexity and

accuracy in L2 monologic oral production. Applied Linguistics 24(1): 1–27.

Documents

Task-internal and task-external readiness: