Embed Size (px)
Citation preview
Assessment for English Speaking & Listening
Dr. Jessie Huang
Goal
Micro skills vs. Macro skillsForms vs. MessagesAccuracy vs. FluencyCommunicative competence
Elements of Oral Proficiency
• Fluency and coherence • Lexical resource • Grammatical range and accuracy• Phonology: intonation, rhythm, pronunciation,
discourse prosody
Elements IELTS TOFEL
Fluency and
coherence
Speak at length without
loss of coherence
Mostly coherent expression
of relevant ideas
Lexical resource
1.enough vocabulary to
discuss at length
2.use paraphrase
effectively
Effective use of vocabulary
Grammatical range
and accuracy
1.use complete structures
2.produce error-free
sentence
Effective use of grammar
Phonology:
Intonation
Rhythm
pronunciation
can generally be
understood throughout
1. speech is generally clear
with some fluidity of expression
2. minor difficulties with
pronunciation and may require
listener effort at time
Testing: Oral Ability
Basic Types of Speaking
1. Imitative-the ability to simply parrot back (imitate) a word
or phrase or possibly a sentence-a focus only on ‘pronunciation’; no inferences are
made about the test-taker’s ability to understand or convey meaning or to participate in an interactive conversation
Basic Types of Speaking
2. Intensive-the production of short stretches of oral language
in a narrow band of grammatical, phrasal, lexical, or phonological relationships
-assessment tasks of speaking: directed response tasks, reading aloud, sentence and dialogue completion, limited picture-cued tasks
Basic Types of Speaking
3. Responsive-assessment tasks of speaking: very short
conversations, standard greetings and small talk, simple requests and comments and the like.
Basic Types of Speaking
4. Interactive-the difference between responsive and interactive
speaking is the length and complexity of the interaction
-interaction including two formsa.Transactional languageb.Interpersonal exchanges
Example of ‘transactional language’
A. Mary: Excuse me, do you have the time? Doug: Yeah. Nine-fifteen.
• T: What is the most urgent environmental problem today?
S : I would say massive deforestation.
Example of ‘interpersonal exchanges’
• Jeff : Hey, Stef, how’s it going? Stef: Not bad, and yourself? Jeff: I’m good. Stef: Cool. Okay, gotta go.
Basic Types of Speaking
5. Extensive (monologue)- speeches, oral presentations and story-telling-language style: deliberative and foraml but
certain informal monologues cannot be ruled out, such as casually delivered speech (jokes)
Designing Assessment Tasks
1. Imitative Speaking
1. Word repetition task2. PhonePass test
Example of ‘word repetition task’
Test-takers hear: Repeat after me:beat [pause] bit [pause] bat [pause] vat [pause] etc.I bought a boat yesterday.The glow of the candle is growing. etc.When did they go on vacation?Do you like coffee? etc.
Test-takers repeat the stimulus.
PhonePass test specifications
Part A:Test-takers read aloud selected sentences from among those
printed on the test sheet. Examples:1. Traffic is a huge problem in Southern California.2. The endless city has no coherent mass transit system.3. Sharing rides was going to be the solution to rush-hour traffic.4. Most people still want to drive their own cars, though. Part B:Test-takers repeat sentences dictated over the phone. Examples:
‘Leave town on the next train’.
PhonePass test specifications
Part C:Test-takers answer questions with a single word or a short phrase of
two or three words. Example: ‘Would you get water from a bottle or a newspaper?’
Part D:Test-takers hear three word groups in random order and must link
them in a correctly ordered sentence. Example: was reading/ my mother/ a magzine.
Part E:Test-takers have 30 seconds to talk about their opinion about some
topic that is dictated over the phone. Topics center on family, preferences, and choices.
2. Intensive Speaking
▲ Test-takers are prompted to produce short stretches of discourse through which they demonstrate linguistic ability at a specified level of language
▲Many tasks are ‘cued’ tasks in that they lead the test-taker into a narrow band of posibilities
2. Intensive Speaking
a. Directed Response Tasksb. Read-Aloud Tasksc.
Sentence/Dialogue Completion Tasks and Oral Questionnairs
d. Picture-Cued Taskse. Translation (of limited stretches of discourse)
a. Directed response tasks
Test-takers will hear: Tell me he went home.Tell me that you like rock music.Remind him what time it is.(Note: Such tasks are clearly mechanical and not
communicative, but they do require minimal processing of meaning in order to produce the correct grammatical output.)
b. Read-Aloud Tasks
Read-aloud stimulus, paragragh length ‘Despite the decrease in size—and, some would say, quality– of our cultural world,
there still remain strong differences between the usual British and American writing styles. The question is, how do you get your message across? English prose conveys its most novel ideas as if they were timeless truths, while American writing exaggerates; if you believe half of what is said, that’s enough. The former uses understatement; the later, overstatement. There are also disadvantages to each characteristic approach. Readers who are used to being screamed at may not listen when someone chooses to whisper politely. At the same time, the individual who is used to a quiet manner may reject a series of loud imperatives.’
(Note: Reading aloud calls on certain specialized oral abilities that may not indicate one’s pragmatic ability to communicate orally in face-to-face contexts.)
c. Sentence/Dialogue Completion Tasks and Oral Questionnairs
Directed response tasksTest-takers see:Interviewer: What did you do last weekend?Test-taker : ___________________________.
Interviewer: What will you do after you graduate from this program?Test-taker : ___________________________.
Interviewer: ___________________________?Test-taker : I was in Japan for two weeks.Test-takers respond with appropriate lines.(Note: Its reliance on literacy and an ability to transfer easily from written to spoken
English.)
d. Pictured-Cued Tasks
It can be very simple, designed to elicit a word or a phrase.
Picture-cued elicitation of minimal pairs
Test-takers see:
Test-takers hear: What is this?
d. Pictured-Cued Tasks
Grammatical categories may be cued by pictures.
Test-takers hear: Use a comparative form to compare these objects.
e. Translationa. Test-taker is given a native language word,
phrase or sentence and is asked to translate it.b. Conditions may be varied from expecting an
instant translation of an orally elicited linguistic target to allowing more thinking time before producing a translation of somewhat longer texts, which may optionally be offered to the test-taker in written form.
3. Responsive Speaking
1. Question and answer• Display questions• Referential questions
• Giving Instructions and Directions• Paraphrasing• Test of Spoken English (TSE)-used by many
North American institutions or higher education to select international teaching assistants
4. Interactive Speaking
a. Interview (contains a number of mandatory stages, Michael Canale, 1984)• Warm-up• Level check• Probe• Wind-down
b. Role Playc. Discussions and Conversationsd. Gamese. Oral Proficiency Interview (OPI)
Oral interview content specificationsWarm-up:1. Small talkLevel check:The test-taker...2. Answers wh-questions 3. Produces a narrative without interruptions.4. Reads a passage aloud.5. Tells how to make something or do something.6. Engages in a brief, controlled, guided role play.Prob:The test-taker...7. Responds to interviewer’s questions about something the test-taker doesn’t know and is planning to include in an
article or paper. 8. Talks about his or her own field of study or profession.9. Engages in a longer, more open-ended role play (for example, simulates a difficult or embarrasing circumstance)
with the interviweer.10. Gives an impromptu presentation on some aspect of test-taker’s field.Wind-down:11. Feelings about the interview, information on results, further questions.
Assessment games Example
City maps are distributed to class members. Predetermined map directions are given to one student who, with a city map in front of him or her, describes the route to a partner, who must then trace the route and get to the correct final destination.
Oral Proficiency Interview (OPI)American Council of Teaching Foreign Languages (ACTFL) OPI
Producer: American Council on Teaching Foreign Languages, Yonkers, NY
Objective: To test oral production skills of speakers in 37 different foreign languages
Primary market: Certification of speakers for government personnel and employees in the workplace; evaluation of students in language programs
Type: Oral interview– telephoned or in person
Response modes: Oral production in a variety of genres and tasks
Specifications: Personalized question geared to the test-taker’s interests and experiences; a variety of communication tasks designed to gauge the test-taker’s upper limits; role play
Time allocation: 30-40 minutes
Internet access: Http://www.actfl.org/
5. Extensive Speaking
a. Oral Presentationsb. Picture-Cued Story-Tellingc. Retelling a Story, News Eventd. Translation (of Extended Prose)
Oral Presentations
Rules for effective assessment must be involved:(1) Specify the criterion (2) Set appropriate tasks(3) Elicit optimal output(4) Establish practical, reliable scoring procedure
Picture-Cued Story-Telling
1. Through visual pictures, photographs, diagrams, and charts
2. At this level a picture or a series of pictures as a stimulus for a longer story or description is considered.
Retelling a Story, News Event
1. Test-takers hear or read a story or news event thta they are asked to recall.
Translation (of Extended Prose)
1. Text forms: Dialogue, directions for assembly of a product, a synopsis of a story or play or movie, directions on how to find something on a map, and other genres.
Testing Listening
What makes listening difficult?
• Clustering• Redundancy• Reduced forms• Performance variables• Colloquial language• Rate of delivery• Stress, rhythm, and intonations• Interaction
Process of Listening• Recognize speech sounds and hold a
temporary ‘imprint’ of them in short-term memory
• Simultaneously determine the type of speech event
• Use ‘bottom-up’ liguistic decoding skills and/or (top-down) background schemata to bring a plausible interpretation to the message, and assign in a literal and intended meaning to the utterance
Each of these stages represents a potential assessment objective
• Comprehending of surface structure elements• Understanding of pragmatic context• Determining meaning of auditory input• Developing the gist, a global or comprehensive
understanding
Types of Listening
Types of listening
1. Intensive-Listening for perception of the components of a
larger stretch of language2. Responsive-Listening to a relatively short stretch of language
in order to make an equally short response
Types of listening
3. Selctive-Processing stretches of discourse such as
monologues for several minutes in order to ‘scan’ for certain information
-be able to comprehend designated information in a context of longer stretches of spoken language
Types of listening
4. Extensive-listening to develop a top-down, global
understanding od spoken language-ranges from listening to lengthy lectures to
listening to a conversation and deriving a comprehensive message or purpose
-listening for the gist, for the main idea, and making inferences
Designing Assessment Tasks
1. Intensive listening
a. Recognizing Phonological and Morphological Elements
b. Paraphrase Recognition
a. Recognizing Phonological and Morphological Elements
Phonemic pair, consonants
Test-takers hear: He’s from California.
Test-takers read: (a) He’s from California.
(b) She’s from California.
Phonemic pair, vowels
Test-takers hear: Is he living?
Test-takers read: (a) Is he leaving?
(b) Is he living?
a. Recognizing Phonological and Morphological Elements
Morphological pair, -ed ending
Test-takers hear: I missed you very much.
Test-takers read: (a) I missed you very much.
(b) I miss you very much.
b. Paraphrase Recognition
Sentence paraphrase
Test-takers hear: Hello, my name’s Keilo. I come from Japan.
Test-takers read: (a) Keiko is comfortable in Japan.
(b) Keiko wants to come to Japan.
(c) Keiko is Japanese.
(d) Keiko likes Japan.
b. Paraphrase Recognition
Dialogue paraphrase
Test-takers hear: Man: Hi, Maria, my name’s George.
Woman: Nice to meet you, George. Are you American?
Man: No, I am Canadian.
Test-takers read: (a) George lives in the United States.
(b) George is American
(c) George comes from Canada.
(d) Maria is Canadian.
2. Responsive listening
a. Question-and-answerb. Open-ended Framework
a. Question-and-answer
Test-takers hear: How much time did you take to do your homework?
Test-takers read: (a) In about an hour.
(b) About an hour.
(c) About $10.
(d) Yes, I did.
Appropriate response to a question
b. Open-ended Framework
Open-ended response to a question
Test-takers hear: How much time did you take to do your homework?
Test-takers write or speak: ________________________________________
3. Selective listening
a. Listening Clozeb. Information Transferc. Sentence Repetition
4. Extensive listening
1. Dictation2. Communicative Stimulus-Response Tasks3. Authentic Listening Tasks
a. Note-takingb. Editingc. Interpretation tasksd. Retelling
