Facultad de Educación
Pedagogía Media en Inglés
Speaking Test
Bryan Abos-Padilla V.
Camila Muñoz F.
June 12th, 2015
Index
I. Introduction
II. Test Specifications
III. Task Descriptions1) Task I2) Task II3) Task III
IV. Rubrics1) Rubric Task I2) Rubric Task II3) Rubric Task III
V. Piloting1) Fixed instructions2) Fixed rubric
VI. Appendixes1) Appendix I2) Appendix II3) Appendix III
2
3
578
91011
121313
141516
1
I. Introduction
The present assignment deals with the design of a test which assesses the productive skill of speaking of a sample group of students who attend to an Industrial High School from Biobio Region. The examination aims to evaluate the test takers oral competence trough the performance of three different tasks related to a common topic that is Sports, Healthy life and Physical activity. The assignment includes the main specifications of a test which follows Luoma and Fulcher’s structures, besides of the rubric to the scoring process, in a way of supporting the correct design of a speaking test.
2
II. Test Specifications
Test Purpose The purpose of this test is to assess the speaking ability of a particular sample group of students through different tasks. The student group is divided in pairs, where the test takers have to perform the tasks answering to personal questions, describing pictures and making decisions. All the previously mentioned activities are related to a common topic.
Description of the examinees
The examinees are students from the 9th grade sample class of an Industrial High School from Bio-Bio Region. The course is composed mainly by males, due to the technical oriented education of the institution, and the level of English of the learners is generally low, showing particular weakness in the productive rather than receptive skills.
Test Level The examination is a face-to-face paired interaction test, designed for Upper Basic English as a Foreign Language students, taking into account the contents and their proficiency level in the test design.
Definition of construct
The test takers have to use the language ability acquired in their English lessons, using a correct structure and syntax in an organized way when speaking and referring to a particular topic. Students have to be able to speak cohesively, using an appropriate register in order to accomplish the main purpose of this test.
Definition of suitable language course book or textbook
The sample group of students to which is oriented this test attend to an Industrial High School, whose lessons and contents are coherent with the national curriculum of the Chilean Ministry of Education (MINEDUC) and do not support the English subject with a language course or textbook, but rather in worksheet prepared by the teacher and class activities.
Number of sections
The test consists of three tasks. In the first task, the main aim is that students have to answer personal questions given by the teacher individually. In the second task, students have to describe a specific sport based on a picture, and in the third task, students have to discuss and reach an agreement on two different options related to the topic of healthy life and physical activity.
Time for each section
The total time for the test is ten minutes, divided into three different tasks. Test takers have to conduct the test in pairs, having two minutes each to perform the first task, one minute and thirty seconds each for the second and both have a total of three for the final task.
Weighting for each section
The test counts with a total of 96 points, considering the three speaking tasks. The ideal score that test takers should obtain in each task is thirty two points, according to the specifications and elements that will be assess in the students’ final performance.
Target Language situation
There is a common topic to each task that is related to Sports, Physical activity and Healthy life. The target language connects with the ability to speak about these topics, in different context and situations, using the vocabulary and expressions acquired in previous lessons, formulating complete responses, describing in detail a specific topic and giving opinion as well as expression personal ideas.
3
Text types The test counts with different text types as pictures related to different sports and physical activities, and it also includes role play, question and answer, description and discussion or conversation.
Language ability This test will assess the oral competence of the students, focusing on the speaking skill, through their performance in the three tasks.
Language elements to be tested
The main elements that will be evaluated by conducting this test are related to the development of speaking skills such as Fluency, Pronunciation and Grammar, focusing primarily on the ability of communicating the message effectively and vocabulary related to the topic, which have higher scores in relation to the rest of the language elements.
Test Tasks Before starting the examination, there is a warm up activity where the teacher asks some questions to the test takers in order to reduce their anxiety and to provide a confident environment to conduct the test. The first section consists of a role play where students have to randomly choose a famous Chilean athlete and answer some personal questions about it. The second section is about the description of a particular sport based on a picture, where test takers have to talk about the main characteristics of that sport. Finally, in the third section, both test takers have to discuss the pros and cons of two options shown on a set of pictures and come to a final agreement.
Test Methods In the first task, test takers speak individually and in turns, answering to the questions given by the teacher. In the second task, test takers speak for a fixed period of time describing a particular topic and finally, in the third task, test takers speak together, discussing a common theme and reaching a final agreement is compulsory.
Rubrics The assessing tool used to evaluate the test takers productive skill of speaking is an analytic rubric, since this type of device best describes the ideal outcome that test takers have to show when conducting the oral examination, providing detailed information of their performance. Moreover, analytic rubrics facilitate the assessor duty, because its structure is easy to follow and to apply, if they are well known at the moment of scoring the test.
4
III. Tasks Description
1) Task I: Personal Information
o Context: Television show interview
o Topic: Sports – Athletes
o Task Instruction: You will be randomly assigned with a famous athlete to personify (refer
to appendix I). Imagine that you are the interviewee at a television show, where I (the
teacher) will be the host and I will ask you some questions regarding your personal
information and the sport you practice. If you do not know any piece of information about
the athlete you were given, you are allowed to make up your answers.
o Athletes Options:
Gymnastics – Tomas Gonzales (Chilean)
Tennis – Fernando Gonzales (Chilean)
Tennis – Nicolas Massu (Chilean)
Soccer – Mark Gonzales (Chilean)
Soccer – Claudio Bravo (Chilean)
Boxing – Carolina Crespita Rodriguez (Chilean)
Swimming – Kristel Köbrich (Chilean)
o Questions: There are four general and equal question for both test takers, and two
different for each one.
1. What is your name?
2. What is your profession?
3. What is your sport about?
4. How long have you been practicing this sport?
Student A: Why did you choose to practice this sport? / What other sport would you
practice? Why?
Student B: What is your bigger achievement in your career? / What aspect do you like the
most from your sport?
5
o Task Specifications
Task Orientation The task orientation is guided. The test takers have to listen to
the questions given by the teacher and answer creatively,
making up their answers if necessary.
Interactional Relationship The task is one way interactional. It is an individual performance
and there is an evident communication between the teacher
(the host of the television show) and the test taker (the
interviewee).
Goal Orientation The goal orientation is convergent. The test taker has to be able
to answer the questions given by the teacher regarding the topic
and without losing their role, in this case a famous Chilean
athlete.
Interlocutor Status and
Familiarity
In the case of this task, even though there is an examination
where the test takers are in front of the teacher and the context
is an imaginary TV show interview, the interlocutor status is
high, whether the familiarity is completely low.
6
2) Task II: Descriptiono Context: Describe a sport based on a picture
o Topic: Sports characteristics
o Task Instruction: Describe the main characteristics of the sport that appear in the
picture. Refer to the number of players, the place where it is played, general rules and
equipment used to practice the sport.
o Sport Options: There are different sports for each test taker (refer to appendix II).
Basketball
Soccer
Tennis
Baseball
Hockey
Volleyball
o Task Specifications
Task Orientation The task orientation is open, because the test takers are
provided with a specific topic to talk about, communicating their
knowledge and ideas, using the known vocabulary and
expressions of the topic. The final outcome depends directly
upon the speaker.
Interactional Relationship Non-interactional. The test takers in turns have to describe
the characteristics of a particular sport in a limited period of
time. There is no interaction among the students or with the
teacher, being a kind of monologue in this task.
Goal Orientation The orientation of the goal is communicative or convergent,
since the test takers have to speak referring to a definite topic,
which in this case is a particular sport.
Interlocutor Status and
Familiarity
There is no interlocutor in this task, due to the test takers have
to speak in turns, without interacting either with the other
students or with the teacher. The familiarity level is low.
7
3) Task III: Agreemento Topic: Physical activity
o Task Instruction: In pairs, you have to reach an agreement on two different options
shown on the pictures (refer to appendix III), discussing the pros and cons of each one.
Option A: The nutritionist suggests you to do physical activity to improve your health
condition. Discuss: Would you prefer to exercise in the gym or to play a sport? Why?
Option B: The physical education teacher offers you to join a school team to increase
your credits in the subject. Discuss: Would you prefer to join a team that practice indoor
or outdoor? Why?
Option C: You win a radio contest where two tickets were offered to next year’s Olympic
Games. Discuss: If you have to choose, would you prefer to assist to the summer or
winter Olympic Games? Why?
o Task Specifications:
Task Orientation The orientation in this task is guided when the test takers have to listen
carefully to the instructions given by the teacher. However, the
orientation becomes open when test takers discuss with their partner
the pros and cons on two different options to finally reach an
agreement on the pictures given.
Interactional Relationship This task is two ways interactional. Both test takers have to interact by
discussing and sharing their ideas and opinions on the topic, in order to
get to an agreement and chose only one option.
Goal Orientation The orientation is convergent, since the test takers have to speak to
each other, discuss their opinions and interact among them. The task is
a pair performance by an agreement.
Interlocutor Status and
Familiarity
There is high status when the teacher set the task to the test takers.
However, there is a medium interlocutor status when the students
have to interact with each other, expressing their opinions on the
topic, since they know each other before taking the test. The familiarity
level is high in relation to the student who is interacting with, and low
in relation to the teacher.
8
IV. Rubrics
1) Task 1
Rating Unratable Minimal performance
Basic performance Average performance
Appropriate performance
0 1 2 3 4Task Completion
No attempt to complete the task
Student was not able to answer personal information questions about the character or the sport he/she practice
Student put a minimal effort in order to complete the task, answers are frequently inappropriate
Student’s completion of the task is incomplete; answers about the character are mostly appropriate but undeveloped
Student completes the task; answers about the character are appropriate and adequately developed.
Communication of Message
(x 2)
No attempt to convey message
Responses barely comprehensible and frequently inappropriate
Responses mostly comprehensible, requiring some interpretation on the part of the listener
Responses comprehensible, requiring minimal interpretation on the part of the listener
Responses readily comprehensible, requiring no interpretation on the part of the listener
Fluency No attempt to speak in the target language
Speech halting and uneven with long pauses or incomplete thoughts
Speech choppy and/or slow with frequent pauses; little attempt to keep conversation or presentation flowing
Some hesitation but manages to continue and complete thoughts
Speech continuous with few pauses or stumbling
Pronunciation No attempt to speak in the target language
Major pronunciation errors, may prevent comprehension
Frequent pronunciation errors, may hinder comprehension
Occasional pronunciation errors which do not impede comprehension
Very little or no pronunciation errors.
Vocabulary
(x 2)
No attempt to use the target language
Inadequate and inaccurate use of vocabulary. Lack of competence about the sport-related vocabulary
Somewhat inadequate and inaccurate use of vocabulary and too basic for this level. Little competence about the sport-related vocabulary
Adequate and accurate use of vocabulary for this level. Adequate knowledge about the sport-related vocabulary
Rich use of vocabulary with frequent attempts at elaboration. Clear competence about the sport-related vocabulary
9
Grammar No attempt to speak in the target language
Almost no correct grammatical structures, impeding comprehension
Frequent grammatical errors
Some minor grammatical errors that doesn’t interfere with comprehension
No or almost no grammatical errors
Adapted from: Loveland High School World Languages and Cultures
lhswlc.wikispaces.com/file/view/Level+1+Speaking+Rubric
2) Task 2
Rating Unratable Minimal performance
Basic performance
Average performance
Appropriate performance
0 1 2 3 4Task Completion
No attempt to complete the task
Student was not able to provide characteristics of the sport
Student put a minimal effort in order to complete the task, answers are frequently inappropriate
Student’s completion of the task is partial; characteristics of the sport provided are mostly appropriate but undeveloped
Student completes the task; characteristics of the sport provided are appropriate and adequately developed.
Communication of Message
(x 2)
No attempt to convey message
Responses barely comprehensible and frequently inappropriate
Responses mostly comprehensible, requiring some interpretation on the part of the listener
Responses comprehensible, requiring minimal interpretation on the part of the listener
Responses readily comprehensible, requiring no interpretation on the part of the listener
Fluency No attempt to speak in the target language
Speech halting and uneven with long pauses or incomplete thoughts
Speech choppy and/or slow with frequent pauses; little attempt to keep conversation or presentation flowing
Some hesitation but manages to continue and complete thoughts
Speech continuous with very few pauses or stumbling
Pronunciation No attempt to speak in the target language
Major pronunciation errors, may prevent comprehension
Frequent pronunciation errors, may hinder comprehension
Occasional pronunciation errors which do not impede comprehension
Very little or no pronunciation errors.
Vocabulary
(x 2)
No attempt to use the target language
Inadequate and inaccurate use of vocabulary. Lack of
Somewhat inadequate and inaccurate use of vocabulary and too
Adequate and accurate use of vocabulary for this level. Adequate
Rich use of vocabulary with frequent attempts at elaboration. Clear
10
competence about the sport-related vocabulary
basic for this level. Little competence about the sport-related vocabulary
knowledge about the sport-related vocabulary
competence about the sport-related vocabulary
Grammar No attempt to speak in the target language
Almost no correct grammatical structures, impeding comprehension
Frequent grammatical errors, requiring some interpretation on the part of the listener
Some minor grammatical errors that doesn’t interfere with comprehension
No or almost no grammatical errors. No comprehension interference
Adapted from: Loveland High School World Languages and Cultures
lhswlc.wikispaces.com/file/view/Level+1+Speaking+Rubric
3) Task 3
Rating Unratable Minimal performance
Basic performance
Average performance
Appropriate performance
0 1 2 3 4Task Completion
No attempt to complete the task
Rudely interrupts his interlocutor. Student does not wait for his/her turn to speak. No agreement reached.
Sometimes interrupts his interlocutor. Not often wait for his/her turn to speak. Does not provide his interlocutor with chances to reply. No agreement reached
Foment the dialogue with his interlocutor. Rarely interrupts him/her, but when it happens, it does not feel like a smooth transition. Agreement reached
Foment the dialogue with his interlocutor. Rarely interrupts him/her, but when it happens, it feels like a smooth and polite transition. Agreement reached
Communication of Message
(x 2)
No attempt to convey message
Interaction barely comprehensible and frequently inappropriate
Interaction mostly comprehensible, requiring some interpretation on the part of the listener
Interaction comprehensible, minimal misinterpretation observed on the part of the listener
Interaction readily comprehensible, requiring no apparent misinterpretation on the part of the listener
11
Fluency No attempt to speak in the target language
Speech halting and uneven with long pauses or incomplete thoughts
Speech choppy and/or slow with frequent pauses; little attempt to keep conversation flowing
Some hesitation but manages to continue and complete thoughts
Speech continuous with very few pauses or stumbling
Pronunciation No attempt to speak in the target language
Major pronunciation errors, may prevent comprehension
Frequent pronunciation errors, may hinder comprehension
Occasional pronunciation errors which do not impede comprehension
Very little or no pronunciation errors.
Vocabulary
(x 2)
No attempt to use the target language
Inadequate and inaccurate use of vocabulary
Somewhat inadequate and inaccurate use of vocabulary and too basic for this level
Adequate and accurate use of vocabulary for this level.
Rich use of vocabulary with frequent attempts at elaboration.
Grammar No attempt to speak in the target language
Almost no correct grammatical structures, impeding comprehension
Frequent grammatical errors, requiring some interpretation on the part of the listener
Some minor grammatical errors that doesn’t interfere with comprehension
No or almost no grammatical errors. No comprehension interference
Adapted from: Loveland High School World Languages and Cultures
lhswlc.wikispaces.com/file/view/Level+1+Speaking+Rubric
12
V. Piloting
The speaking test was applied to two students from the sample 9 th grade from an Industrial High School from Biobío Region. Both during and after the appliance of the test presented some problems regarding the designing and the content of its instructions. These problems were founded particularly on task 1 and 3 and also in the evaluation device that is an analytic rubric.
The main issue regarding the first task was that the sample test takers did not knew some of the athletes featured in the task. One of the students got “Carolina Crespita Rodriguez” as the athlete she had to portray, and did not knew who she was. Despites we give her the opportunity to make up the background story for the character, she was not really sure of whether her answers made any sense, which made her nervous and affect her final performance. Therefore, we decided to let them know in advance the options they were going to be given, in order to eliminate two that they might not be sure about. That way, the test takers’ probabilities of choosing an unknown athlete will be diminished.
On the other hand, the third task presented an issue in terms of agreement. Communication was very well developed, but the students did not reached agreement, since they had very different opinions on the topic that they were given. In spite of their remarkable performance on the speaking task, according to the rubric test takers should have been punished with low score on the task completion part. In order to achieve the main aim of this evaluation, it is important to note that not reaching an agreement is not result of any linguistic problem, but rather a matter of opinion that does not concern speaking ability. Therefore, both the instruction and the rubric need to be modified, in order to allow discrepancies to be accepted as a possibility.
As stated before, in order to fix the issues regarding the task performance segment of the rubric, it was deleted the statements that refer to reaching an agreement, focusing only on turn taking and conversational skills. In contrast, and since the problem with task 1 was related to the instructions and not with the rubric, it was left as it was initially planned, along with the rubric for task 2.
13
1) Fixed instructions
Task 1:
Seven different Chilean athletes will be shown to you, from which you have to eliminate
two that you might have trouble with when explaining the sport or his/her personal information.
You will be randomly assigned with one of the remaining options. Imagine that you are the
interviewee at a television show, where I (the teacher) will be the host and I will ask you some
questions regarding your personal information and the sport you practice. If you do not know any
piece of information about the athlete you were given, you are allowed to make up your answers.
Task 3:
In pairs, you have to reach an agreement on two different options shown on the pictures
(refer to appendix III), discussing the pros and cons of each one. It is not compulsory to reach an
agreement, so you can agree to disagree. The task aim is to see your conversational skills and turn
taking abilities, therefore you can have different opinions.
2) Fixed rubric
Task 3:
Rating Unratable Minimal performance
Basic performance
Average performance
Appropriate performance
0 1 2 3 4Task Completion
No attempt to complete the task
Rudely interrupts his interlocutor. Student does not wait for his/her turn to speak.
Sometimes interrupts his interlocutor. Not often wait for his/her turn to speak. Does not provide his interlocutor with chances to reply.
Foment the dialogue with his interlocutor. Rarely interrupts him/her, but when it happens, it does not feel like a smooth transition.
Foment the dialogue with his interlocutor. Rarely interrupts him/her, but when it happens, it feels like a smooth and polite transition.
Adapted from: Loveland High School World Languages and Cultures
lhswlc.wikispaces.com/file/view/Level+1+Speaking+Rubric
14
VI. Appendixes
1) Appendix I
Samples of the famous Chilean athletes and the sports they play that will be used to perform task one, related to personal questions.
15
Tomas Gonzalez - Gymnastics Fernando Gonzalez - Tennis
Mark Gonzalez - SoccerCrespita Rodriguez - Boxing
2) Appendix II
Samples of the pictures of different sports that will be used to perform task two.
16
3) Appendix III
Samples of pictures that show imaginary situations that will be used in task three about reaching an agreement.
o Option A
o Option B
o Option C
17