Testing Speaking Ability EFL

Facultad de Educación

Pedagogía Media en Inglés

Speaking Test

Bryan Abos-Padilla V.

Camila Muñoz F.

June 12th, 2015

Index

I. Introduction

II. Test Specifications

III. Task Descriptions1) Task I2) Task II3) Task III

IV. Rubrics1) Rubric Task I2) Rubric Task II3) Rubric Task III

V. Piloting1) Fixed instructions2) Fixed rubric

VI. Appendixes1) Appendix I2) Appendix II3) Appendix III

2

3

578

91011

121313

141516

1

I. Introduction

The present assignment deals with the design of a test which assesses the productive skill of speaking of a sample group of students who attend to an Industrial High School from Biobio Region. The examination aims to evaluate the test takers oral competence trough the performance of three different tasks related to a common topic that is Sports, Healthy life and Physical activity. The assignment includes the main specifications of a test which follows Luoma and Fulcher’s structures, besides of the rubric to the scoring process, in a way of supporting the correct design of a speaking test.

2

II. Test Specifications

Test Purpose The purpose of this test is to assess the speaking ability of a particular sample group of students through different tasks. The student group is divided in pairs, where the test takers have to perform the tasks answering to personal questions, describing pictures and making decisions. All the previously mentioned activities are related to a common topic.

Description of the examinees

The examinees are students from the 9th grade sample class of an Industrial High School from Bio-Bio Region. The course is composed mainly by males, due to the technical oriented education of the institution, and the level of English of the learners is generally low, showing particular weakness in the productive rather than receptive skills.

Test Level The examination is a face-to-face paired interaction test, designed for Upper Basic English as a Foreign Language students, taking into account the contents and their proficiency level in the test design.

Definition of construct

The test takers have to use the language ability acquired in their English lessons, using a correct structure and syntax in an organized way when speaking and referring to a particular topic. Students have to be able to speak cohesively, using an appropriate register in order to accomplish the main purpose of this test.

Definition of suitable language course book or textbook

The sample group of students to which is oriented this test attend to an Industrial High School, whose lessons and contents are coherent with the national curriculum of the Chilean Ministry of Education (MINEDUC) and do not support the English subject with a language course or textbook, but rather in worksheet prepared by the teacher and class activities.

Number of sections

The test consists of three tasks. In the first task, the main aim is that students have to answer personal questions given by the teacher individually. In the second task, students have to describe a specific sport based on a picture, and in the third task, students have to discuss and reach an agreement on two different options related to the topic of healthy life and physical activity.

Time for each section

The total time for the test is ten minutes, divided into three different tasks. Test takers have to conduct the test in pairs, having two minutes each to perform the first task, one minute and thirty seconds each for the second and both have a total of three for the final task.

Weighting for each section

The test counts with a total of 96 points, considering the three speaking tasks. The ideal score that test takers should obtain in each task is thirty two points, according to the specifications and elements that will be assess in the students’ final performance.

Target Language situation

There is a common topic to each task that is related to Sports, Physical activity and Healthy life. The target language connects with the ability to speak about these topics, in different context and situations, using the vocabulary and expressions acquired in previous lessons, formulating complete responses, describing in detail a specific topic and giving opinion as well as expression personal ideas.

3

Text types The test counts with different text types as pictures related to different sports and physical activities, and it also includes role play, question and answer, description and discussion or conversation.

Language ability This test will assess the oral competence of the students, focusing on the speaking skill, through their performance in the three tasks.

Language elements to be tested

The main elements that will be evaluated by conducting this test are related to the development of speaking skills such as Fluency, Pronunciation and Grammar, focusing primarily on the ability of communicating the message effectively and vocabulary related to the topic, which have higher scores in relation to the rest of the language elements.

Test Tasks Before starting the examination, there is a warm up activity where the teacher asks some questions to the test takers in order to reduce their anxiety and to provide a confident environment to conduct the test. The first section consists of a role play where students have to randomly choose a famous Chilean athlete and answer some personal questions about it. The second section is about the description of a particular sport based on a picture, where test takers have to talk about the main characteristics of that sport. Finally, in the third section, both test takers have to discuss the pros and cons of two options shown on a set of pictures and come to a final agreement.

Test Methods In the first task, test takers speak individually and in turns, answering to the questions given by the teacher. In the second task, test takers speak for a fixed period of time describing a particular topic and finally, in the third task, test takers speak together, discussing a common theme and reaching a final agreement is compulsory.

Rubrics The assessing tool used to evaluate the test takers productive skill of speaking is an analytic rubric, since this type of device best describes the ideal outcome that test takers have to show when conducting the oral examination, providing detailed information of their performance. Moreover, analytic rubrics facilitate the assessor duty, because its structure is easy to follow and to apply, if they are well known at the moment of scoring the test.

4

III. Tasks Description

1) Task I: Personal Information

o Context: Television show interview

o Topic: Sports – Athletes

o Task Instruction: You will be randomly assigned with a famous athlete to personify (refer

to appendix I). Imagine that you are the interviewee at a television show, where I (the

teacher) will be the host and I will ask you some questions regarding your personal

information and the sport you practice. If you do not know any piece of information about

the athlete you were given, you are allowed to make up your answers.

o Athletes Options:

Gymnastics – Tomas Gonzales (Chilean)

Tennis – Fernando Gonzales (Chilean)

Tennis – Nicolas Massu (Chilean)

Soccer – Mark Gonzales (Chilean)

Soccer – Claudio Bravo (Chilean)

Boxing – Carolina Crespita Rodriguez (Chilean)

Swimming – Kristel Köbrich (Chilean)

o Questions: There are four general and equal question for both test takers, and two

different for each one.

1. What is your name?

2. What is your profession?

3. What is your sport about?

4. How long have you been practicing this sport?

Student A: Why did you choose to practice this sport? / What other sport would you

practice? Why?

Student B: What is your bigger achievement in your career? / What aspect do you like the

most from your sport?

5

o Task Specifications

Task Orientation The task orientation is guided. The test takers have to listen to

the questions given by the teacher and answer creatively,

making up their answers if necessary.

Interactional Relationship The task is one way interactional. It is an individual performance

and there is an evident communication between the teacher

(the host of the television show) and the test taker (the

interviewee).

Goal Orientation The goal orientation is convergent. The test taker has to be able

to answer the questions given by the teacher regarding the topic

and without losing their role, in this case a famous Chilean

athlete.

Interlocutor Status and

Familiarity

In the case of this task, even though there is an examination

where the test takers are in front of the teacher and the context

is an imaginary TV show interview, the interlocutor status is

high, whether the familiarity is completely low.

6

2) Task II: Descriptiono Context: Describe a sport based on a picture

o Topic: Sports characteristics

o Task Instruction: Describe the main characteristics of the sport that appear in the

picture. Refer to the number of players, the place where it is played, general rules and

equipment used to practice the sport.

o Sport Options: There are different sports for each test taker (refer to appendix II).

Basketball

Soccer

Tennis

Baseball

Hockey

Volleyball

o Task Specifications

Task Orientation The task orientation is open, because the test takers are

provided with a specific topic to talk about, communicating their

knowledge and ideas, using the known vocabulary and

expressions of the topic. The final outcome depends directly

upon the speaker.

Interactional Relationship Non-interactional. The test takers in turns have to describe

the characteristics of a particular sport in a limited period of

time. There is no interaction among the students or with the

teacher, being a kind of monologue in this task.

Goal Orientation The orientation of the goal is communicative or convergent,

since the test takers have to speak referring to a definite topic,

which in this case is a particular sport.


Familiarity

There is no interlocutor in this task, due to the test takers have

to speak in turns, without interacting either with the other

students or with the teacher. The familiarity level is low.

7

3) Task III: Agreemento Topic: Physical activity

o Task Instruction: In pairs, you have to reach an agreement on two different options

shown on the pictures (refer to appendix III), discussing the pros and cons of each one.

Option A: The nutritionist suggests you to do physical activity to improve your health

condition. Discuss: Would you prefer to exercise in the gym or to play a sport? Why?

Option B: The physical education teacher offers you to join a school team to increase

your credits in the subject. Discuss: Would you prefer to join a team that practice indoor

or outdoor? Why?

Option C: You win a radio contest where two tickets were offered to next year’s Olympic

Games. Discuss: If you have to choose, would you prefer to assist to the summer or

winter Olympic Games? Why?

o Task Specifications:

Task Orientation The orientation in this task is guided when the test takers have to listen

carefully to the instructions given by the teacher. However, the

orientation becomes open when test takers discuss with their partner

the pros and cons on two different options to finally reach an

agreement on the pictures given.

Interactional Relationship This task is two ways interactional. Both test takers have to interact by

discussing and sharing their ideas and opinions on the topic, in order to

get to an agreement and chose only one option.

Goal Orientation The orientation is convergent, since the test takers have to speak to

each other, discuss their opinions and interact among them. The task is

a pair performance by an agreement.


Familiarity

There is high status when the teacher set the task to the test takers.

However, there is a medium interlocutor status when the students

have to interact with each other, expressing their opinions on the

topic, since they know each other before taking the test. The familiarity

level is high in relation to the student who is interacting with, and low

in relation to the teacher.

8

IV. Rubrics

1) Task 1

Rating Unratable Minimal performance

Basic performance Average performance

Appropriate performance

0 1 2 3 4Task Completion

No attempt to complete the task

Student was not able to answer personal information questions about the character or the sport he/she practice

Student put a minimal effort in order to complete the task, answers are frequently inappropriate

Student’s completion of the task is incomplete; answers about the character are mostly appropriate but undeveloped

Student completes the task; answers about the character are appropriate and adequately developed.

Communication of Message

(x 2)

No attempt to convey message

Responses barely comprehensible and frequently inappropriate

Responses mostly comprehensible, requiring some interpretation on the part of the listener

Responses comprehensible, requiring minimal interpretation on the part of the listener

Responses readily comprehensible, requiring no interpretation on the part of the listener

Fluency No attempt to speak in the target language

Speech halting and uneven with long pauses or incomplete thoughts

Speech choppy and/or slow with frequent pauses; little attempt to keep conversation or presentation flowing

Some hesitation but manages to continue and complete thoughts

Speech continuous with few pauses or stumbling

Pronunciation No attempt to speak in the target language

Major pronunciation errors, may prevent comprehension

Frequent pronunciation errors, may hinder comprehension

Occasional pronunciation errors which do not impede comprehension

Very little or no pronunciation errors.

Vocabulary

(x 2)

No attempt to use the target language

Inadequate and inaccurate use of vocabulary. Lack of competence about the sport-related vocabulary

Somewhat inadequate and inaccurate use of vocabulary and too basic for this level. Little competence about the sport-related vocabulary

Adequate and accurate use of vocabulary for this level. Adequate knowledge about the sport-related vocabulary

Rich use of vocabulary with frequent attempts at elaboration. Clear competence about the sport-related vocabulary

9

Grammar No attempt to speak in the target language

Almost no correct grammatical structures, impeding comprehension

Frequent grammatical errors

Some minor grammatical errors that doesn’t interfere with comprehension

No or almost no grammatical errors

Adapted from: Loveland High School World Languages and Cultures

lhswlc.wikispaces.com/file/view/Level+1+Speaking+Rubric

2) Task 2


Basic performance

Average performance




Student was not able to provide characteristics of the sport

Student put a minimal effort in order to complete the task, answers are frequently inappropriate

Student’s completion of the task is partial; characteristics of the sport provided are mostly appropriate but undeveloped

Student completes the task; characteristics of the sport provided are appropriate and adequately developed.


(x 2)


Responses barely comprehensible and frequently inappropriate

Responses mostly comprehensible, requiring some interpretation on the part of the listener

Responses comprehensible, requiring minimal interpretation on the part of the listener

Responses readily comprehensible, requiring no interpretation on the part of the listener



Speech choppy and/or slow with frequent pauses; little attempt to keep conversation or presentation flowing


Speech continuous with very few pauses or stumbling






Vocabulary

(x 2)


Inadequate and inaccurate use of vocabulary. Lack of

Somewhat inadequate and inaccurate use of vocabulary and too

Adequate and accurate use of vocabulary for this level. Adequate

Rich use of vocabulary with frequent attempts at elaboration. Clear

10

competence about the sport-related vocabulary

basic for this level. Little competence about the sport-related vocabulary

knowledge about the sport-related vocabulary

competence about the sport-related vocabulary



Frequent grammatical errors, requiring some interpretation on the part of the listener


No or almost no grammatical errors. No comprehension interference



3) Task 3


Basic performance

Average performance




Rudely interrupts his interlocutor. Student does not wait for his/her turn to speak. No agreement reached.

Sometimes interrupts his interlocutor. Not often wait for his/her turn to speak. Does not provide his interlocutor with chances to reply. No agreement reached

Foment the dialogue with his interlocutor. Rarely interrupts him/her, but when it happens, it does not feel like a smooth transition. Agreement reached

Foment the dialogue with his interlocutor. Rarely interrupts him/her, but when it happens, it feels like a smooth and polite transition. Agreement reached


(x 2)


Interaction barely comprehensible and frequently inappropriate

Interaction mostly comprehensible, requiring some interpretation on the part of the listener

Interaction comprehensible, minimal misinterpretation observed on the part of the listener

Interaction readily comprehensible, requiring no apparent misinterpretation on the part of the listener

11



Speech choppy and/or slow with frequent pauses; little attempt to keep conversation flowing


Speech continuous with very few pauses or stumbling






Vocabulary

(x 2)


Inadequate and inaccurate use of vocabulary

Somewhat inadequate and inaccurate use of vocabulary and too basic for this level

Adequate and accurate use of vocabulary for this level.

Rich use of vocabulary with frequent attempts at elaboration.



Frequent grammatical errors, requiring some interpretation on the part of the listener


No or almost no grammatical errors. No comprehension interference



12

V. Piloting

The speaking test was applied to two students from the sample 9 th grade from an Industrial High School from Biobío Region. Both during and after the appliance of the test presented some problems regarding the designing and the content of its instructions. These problems were founded particularly on task 1 and 3 and also in the evaluation device that is an analytic rubric.

The main issue regarding the first task was that the sample test takers did not knew some of the athletes featured in the task. One of the students got “Carolina Crespita Rodriguez” as the athlete she had to portray, and did not knew who she was. Despites we give her the opportunity to make up the background story for the character, she was not really sure of whether her answers made any sense, which made her nervous and affect her final performance. Therefore, we decided to let them know in advance the options they were going to be given, in order to eliminate two that they might not be sure about. That way, the test takers’ probabilities of choosing an unknown athlete will be diminished.

On the other hand, the third task presented an issue in terms of agreement. Communication was very well developed, but the students did not reached agreement, since they had very different opinions on the topic that they were given. In spite of their remarkable performance on the speaking task, according to the rubric test takers should have been punished with low score on the task completion part. In order to achieve the main aim of this evaluation, it is important to note that not reaching an agreement is not result of any linguistic problem, but rather a matter of opinion that does not concern speaking ability. Therefore, both the instruction and the rubric need to be modified, in order to allow discrepancies to be accepted as a possibility.

As stated before, in order to fix the issues regarding the task performance segment of the rubric, it was deleted the statements that refer to reaching an agreement, focusing only on turn taking and conversational skills. In contrast, and since the problem with task 1 was related to the instructions and not with the rubric, it was left as it was initially planned, along with the rubric for task 2.

13

1) Fixed instructions

Task 1:

Seven different Chilean athletes will be shown to you, from which you have to eliminate

two that you might have trouble with when explaining the sport or his/her personal information.

You will be randomly assigned with one of the remaining options. Imagine that you are the

interviewee at a television show, where I (the teacher) will be the host and I will ask you some

questions regarding your personal information and the sport you practice. If you do not know any

piece of information about the athlete you were given, you are allowed to make up your answers.

Task 3:

In pairs, you have to reach an agreement on two different options shown on the pictures

(refer to appendix III), discussing the pros and cons of each one. It is not compulsory to reach an

agreement, so you can agree to disagree. The task aim is to see your conversational skills and turn

taking abilities, therefore you can have different opinions.

2) Fixed rubric

Task 3:


Basic performance

Average performance




Rudely interrupts his interlocutor. Student does not wait for his/her turn to speak.

Sometimes interrupts his interlocutor. Not often wait for his/her turn to speak. Does not provide his interlocutor with chances to reply.

Foment the dialogue with his interlocutor. Rarely interrupts him/her, but when it happens, it does not feel like a smooth transition.

Foment the dialogue with his interlocutor. Rarely interrupts him/her, but when it happens, it feels like a smooth and polite transition.



14

VI. Appendixes

1) Appendix I

Samples of the famous Chilean athletes and the sports they play that will be used to perform task one, related to personal questions.

15

Tomas Gonzalez - Gymnastics Fernando Gonzalez - Tennis

Mark Gonzalez - SoccerCrespita Rodriguez - Boxing

2) Appendix II

Samples of the pictures of different sports that will be used to perform task two.

16

3) Appendix III

Samples of pictures that show imaginary situations that will be used in task three about reaching an agreement.

o Option A

o Option B

o Option C

17