Analyzing the English Test

LANGUAGE TEACHING EVALUATION

MID TERM PROJECT

Analyzing senior high school Mid term test

By:SYILVIA WENNY J.

1103757

ENGLISH EDUCATIONPOST GRADUATED PROGRAMPADANG STATE UNIVERSITY

PADANG2012

A. Introduction

Education aims to develop students’ ability, knowledge, skills, etc. In order

to reach those, education needs curriculum that contains a set of plan and role of

content, material and teaching learning method. Curriculum is used as a basis of

teaching learning process. It also contains a tool that is used to know how far

students reach the goal. It is called evaluation. In evaluating students, teacher needs

some tests that have some purposes, such as for placement, for continuing to the

higher level of education, etc.

Test consists of written test and spoken test. Written test is usually used to

know students ability in mastering writing skill, grammar, etc. While spoken is

sometimes used to know students’ ability in speaking and responding the listener.

Students prefer in doing written test to spoken test. For them, written test does not

make them more nervous than spoken test. Then, when they do not know how to

answer written test, they may not answer although the grades will be decreased. It

does not happen when they meet spoken test, they feel hard to say “I don’t know” in

front of their teacher. For teachers, written test has some weaknesses. Firstly, they

have to have much money in order to copy the question and answer sheet. Then,

they have to have much time in correcting the answers made by students. And,

sometimes some students are not appropriate with the grade they got as a result of

the test to the process their learning process. It does not happen when teachers have

the spoken test. Based on the issues above, the paper focuses on written test.

In designing test, especially written test, the teacher needs some

considerations in order to fulfill their needs in grading students. The test must be

shown as clear as possible in order to avoid misunderstanding of student. It is one of

the considerations. And, the principles in designing test are practicality, reliability,

validity, authenticity and washback.

In this paper, the writer will analyze a test that is used for mid term test of

Private SMA Padang for English subject. The analyzing will show whether the test

has accommodated all aspects of English language learning and assessment that

have been mentioned before.

Syilvia Wenny J (1103757). MID TERM TEST OF LTE

B. Review of the Related Literature

1. Definition of Test

Test is usually used when learning process has been done. It is used to know

whether students have mastered the lessons that have been taught or not. Brown

(2010) states that test are prepared administrative procedures that occur at

identifiable times in a curriculum when learners master all their faculties to offer

peak performance. There are some components of a test, first, method is a set of

techniques, procedures, or items that requires performance on the part of test-taker.

The method must be explicit and structured in order to make teachers are easy to

analyze the result of test. And students are easy to understand what must be

answered.

Then, test must measure as a process of quantifying a test-taker’s

performance according to explicit procedures or rules. Test can measure the general

ability of students, and also measure specific competencies or specific objectives.

For example, when students want to face the last examination in order to continue

the higher level, they will be tested their general ability on each subject. If the

subjects of test is English, they will be tested their ability in listening, speaking,

reading and writing. It also happens to others subject on different ability. Test can

measure the specific competencies or objectives, such as writing test. After having

the test, the results will be analyzed whether they have the coherent, unity, and other

aspects that must be measured in writing test.

Moreover, a test measures an individual’s ability, knowledge or

performance. Before measuring the students, teachers must be sure that the students

have been taught the knowledge and ability. If the teachers do not know what have

been taught, it could be harmful in scoring the students, because students have their

own perception of knowledge, and it will not have synchronize between the

knowledge given to knowledge that students have before. In testing students’

performance, teachers must keep attention to their performance. Teachers must be

aware students’ words choice, students’ grammar, etc. Those can be a consideration

in scoring them.


Test measures a given domain. Test has to have the particular section to

measure which particular criteria. For example, a vocabulary test may focus on only

the set of words covered in a particular lesson or unit. Another example is writing

test that focus only particular genre of a text. It will help students to be focused in

studying.

2. Types and Purposes of the Test

Before designing test, it is better for teachers to determine the purpose and

types of the test. The impact of doing this is making teachers easy in making

decisions for the students. Bachman (1990) states that since the basic purpose of

tests in educational programs is to provide information for making decisions, the

various specific uses of tests can be understood by considering the types of decision

to be made. Brown and Abeywickrama (2010) classify the types and the purposes of

the test include:

a. Achievement Test

Achievement test is measuring students’ ability within a classroom lesson

unit, or even total curriculum. This test is used to determine whether course

objectives have mastered at the end of a given period in instruction. This is a

reason why this test is often summative. The achievement test should be

determined by the objective of the lesson or course, the relative importance

assigned to each objective, the tasks employed in classroom lessons during

the unit of time, the time frame for the test and turn around time and its

potential for formative feedback.

b. Diagnostic Test

Diagnostic test is used to diagnose aspects of a language that a student needs

to develop or that a course should include. For example in vocabulary test,

teachers may diagnose the choice of words that the students made in

sentence. Diagnostic test seems similar to the achievement test. In fact,

achievement test analyze the extent to which students have acquired

language features that have already been taught, while diagnostic test should

elicit information on what students need to work in the future.


c. Placement Test

Placement is to place student into a certain level or section of a language

curriculum or school. Bachman (1990) states that in many language

programs students are grouped homogeneously according to factors such as

level of language ability, language aptitude, language use needs, and

professional or academic specialization. Decisions regarding the placement

of students into appropriate groups must be made.

d. Proficiency Test

Proficiency test is to know students’ global competence in a language. It

tests overall ability. It is always summative and norm-referenced.

e. Aptitude Test

Aptitude test is designed to measure capacity or general ability to learn a

foreign language a priori (before taking a course) and ultimate predicted

success in that undertaking.

3. Principle of Language Assessment

Before administering the test to the students, teachers have to determine the

principles of language assessment. It is done to know how effective, appropriate or

useful for the students and teachers itself. Those principles are (Brown, 2010):

a. Practicality

Practicality refers to the logistical, down to earth, administrative

issues involved in making, giving, and scoring an assessment instrument.

These include costs, the amount of time it takes to construct and to

administer, ease of scoring and ease of interpreting/reporting the results.

There are some examples of impractical test. First, if the test needs students

to have five or more hours in doing that, it can be said impractical.

Then, when the result of test can be analyzed by the computer only, it

can be said impractical. Moreover, if teachers need much money in

designing test, it also can be said impractical. And there are still many more

examples that show impractical test.


b. Reliability

A reliable test is consistent and dependable. If teachers give students

the same test in different time, and the result is similar, it can be said that the

test is reliable. Some factors regarding reliability are the student, the scoring,

the test administration and the test itself.

c. Validity

Validity is the extent to inference made from assessment results is

appropriate, meaningful and useful in terms of the purpose of the assessment.

A valid test measures exactly what it purposes to measure. It does not

measure irrelevant or contaminating variables. It relies as much as possible

on empirical evidence. It involves performance that samples the test’s

criterion. It offers useful, meaningful information about a test-taker’s ability.

And it is supported by a theoretical rationale or argument. There are some

evidences in measuring validity test; content related evidence, criterion-

related evidence, construct related evidence, consequential validity and face

validity.

d. Authenticity

Authenticity is the degree of correspondence of the characteristics of

a given language test task to the features of a target language task and then

suggested an agenda for identifying those target language tasks and for

transforming them into valid test items. Authentic test contains language that

is as natural as possible. It has items that are contextualized rather than

isolated. It includes meaningful, relevant, interesting topic. It also provides

some thematic organization to items, such as through a story line or episode.

Then it offers tasks that replicate real world tasks.

e. Wash back

Washback is the effect of testing on teaching and learning. Washback

influences what and how teachers teach. It influences what and how learners

learn. Then, it offers learners a chance to adequately prepare. It also gives

learners feedback that enhances their language development. Moreover, it is


more formative in nature than summative. And, it provides conditions for

peak performance by the students.

4. Standard Based Assessment

Standard based assessment refers to procedures that have been specifically

designed to test such competencies. Indonesia has the standard based education. It

makes all school have to follow those standards that is shown on curriculum. The

test will be designed based on the curriculum so that teachers and students are easy

to master certain objective or competencies. Standardized test offers high levels of

practicality and reliability and are often supported by impressive construct validation

studies. It is capable of accurately placing hundreds of students onto a norm-

referenced scale with high reliability ratios.

5. Design Classroom Language Test

When deigning, administering and making maximum tests in the classroom,

teachers need to considers some considerations. Those considerations are formed

into several questions, which are:

a. What is the purpose?

b. What are the objectives of the test?

c. How will the test specifications reflect both the purpose and the objectives?

d. How will the test item types (tasks) be selected and the separate items

arranged?

e. In administering the test, what details should teacher attend to in order to

help students achieve optimal performance?

f. What kind of scoring, grading and/or feedback is expected?

There some considerations in designing the communicative test. Weirr (1990)

mentions that there are at least seven items that should be considered significantly in

design of communicative test.

a. Realistic context –the test tasks should be regarded as appropriate to the

candidates’ situation.

b. Relevant information gap—candidates should have to process new

information as they might in the real life situation


c. Intersubjectivity—tasks should involve candidate both as language receiver

and language producers. In addition, the language produced by the candidate

should be modified in accordance with what their expectations of the

addressee are perceived to be.

d. Scope for development of activity by the candidates—the tasks should give

candidates the chance to assert their community independence and allowance

should also be made for the creative unpredictability of communication in

the tasks set and the marketing schemes that are applied.

e. Allowance for self monitoring by candidates—the tasks should allow

candidates to use their discourse processing strategies to evaluate their

communicative effectiveness and make any necessary adjustment in the

course of an event.

f. Processing of appropriately sized input—the size and scope of the activities

should be such that they are processing the kinds of input they would

normally be expected to.

g. Normal time constraint operative—the tasks should be accomplished under

time constraints.

Besides, Weir (1990) also gives suggestions about the test task dimensions which

may be useful into consideration by the teachers or test administer:

a. Size of text—the length of text, receptive or productive, that involved in the

event.

b. Grammatical complexity and range of cohesion devices required in the

event.

c. Functional range—the degree of variety of illocutionary acts involved in the

event

d. Referential range—the breadth and depth of lexical knowledge required to

handle activities in the event

6. Communicative Language Ability

Bachman (1990) states that communicative language ability (CLA) can be

described as consisting of both knowledge or competence, and the capacity for

implementing or executing that competence in appropriate, contextualized language


use. In designing test, teachers need to complete the communicative language. CLA

consists of language competence, strategic competence and psycho-physiological

mechanisms.

Language competence includes organizational competence, which consists of

grammatical competence and textual competence, and pragmatic competence, which

consists of illocutionary and sociolinguistic competence. Strategic competence is

seen as the capacity that relates language competence, or knowledge of language, to

the language user’s knowledge structures and the features of the context in which

communication takes place. Strategic competence performs assessment, planning,

and execution functions in determining the most effective means of achieving a

communicative goal.


C. Test Analysis

1. Purpose and Types of the Test

The Heading Form of Mid Semester Test of Senior High School

The writer will analyze the mid semester test of senior high school. The

test is for the second grade students. The test consists of listening test; match what

students hear to the appropriate options in question sheets and reading test. There is

no writing and speaking test. The kind of this test is achievement test. The function

of this test is to measure students’ achievement and ability. Moreover, this test is

used to know whether the students’ have reached what they have been taught.

2. Principles of Language Test Assessment

Before designing the test, teacher must consider the six principles of test that

has been explained above. They are practicality, reliability, validity, authenticity,

wash back and efficiency. The test will be analyzed based on those principles from

Brown and Abeywickrama (2010).

a. Practicality

Practicality is regarded by teachers (and the students’) time constraints,

costs, and administrative details and to some extent by what occurs before and

after the test. In determining whether the test is practicality, teachers need to

consider the following considerations:

- Are administrative details all carefully attended to before the test?

- Can students complete the test reasonably within the set time frame?

- Can the test be administered smoothly, without procedural ‘glitches’?


- Are all printed materials accounted for?

- Has equipment been pre-tested?...

- Is the cost of the test within the budgeted limits?

- Is the scoring/evaluation system feasible in the teacher’s time frame?

- Are methods for reporting results determined in advance?

Based on those considerations, administrative details have been attended

before the test. It makes the time is not wasteful for the students in completing

the test. In order words, students also can complete the test within the set time

frame. This test provides 75 minutes for 40 items. Therefore, the students have

approximately 2 minutes in doing each item. The test is administered well, and

there is no any problem in procedural. After that, all materials have been

accounted smoothly. School budgets the cost test well, so there would not be out

of budgeted spent. Also, teachers are feasible in scoring the test. On the other

hand, the test did not pre-test, but the test was designed based on the curriculum.

And for reporting result, the analyzer did not find any information.

Based on those considerations, it can be concluded that the test is generally

practical. As an additional information, this test is written in two columns, the

font used is times new roman, and the font size is 11 and use A4 paper.

b. Reliability

Reliability applies to the students, the test administration, the test itself, and

the teacher. The guidance in determining whether the test is reliable is:

- Does every student have a cleanly photocopied test sheet?

- Is sound amplification clearly audible to everyone in the room?

- Is video input clearly and uniformly visible to all?

- Are lighting, temperature, extraneous noise, and other classroom conditions

equal (and optimal) for all students?

- For closed-ended responses, do scoring procedures leave little debate about

correctness of an answer?

The test is photocopied cleanly and clearly. It makes students easy to read

the test. Based on the teacher who teaches this subject, the sound for listening is

not clear enough but students can still understand the audio. The test does not


use video, so this question is not used. The test uses multiple choices only, so

there is no debate about the correctness of an answer that is usually derived from

close-ended response. To sum up, this test is reliability.

c. Validity

Validity is extent to which the assessment requires students to perform tasks

that was included in the previous classroom lessons and that directly represent

the objectives of the unit on which the assessment is based. Several steps that

determine the validity, they are:

- Are unit objectives clearly identified?

- Are unit objectives represented in the form of test specifications?

- Do the test specifications include tasks that have already been performed as

part of the course procedures?

- Do the test specifications include tasks that represent all (or most) of the

objectives for the unit?

- Do those tasks involve actual performance of the target task?

This test clearly identifies the objectives. Those objectives understand the

narrative text, note, announcement, private letter and job application. The

specification of the task also shows in the test. Then, the test generally is valid to

measure the students’ competence. The strategies on how to answers are also

shown in the first form and it is written in Indonesian. To sum up, the test is

valid.

d. Authenticity

The considerations in determining the authenticity of a test are:

- Is the language in the test as natural as possible?

- Are items as contextualized as possible rather than isolated?

- Are topics and situations interesting, enjoyable, and/or humorous?

- Is some thematic organization provided, such as through a story line

or episode?

- Do tasks represent, or closely approximate, real-world tasks?


This test uses natural language in telling the narrative text, note,

announcement, private letter and job application. Those kinds of text are

commonly used in the real life. Then, the topics that is written is enjoyable so

that students can read it well while relate them into the students’ imagination. By

those explanations, it can be said that this test has the authenticity.

e. Washback

Washback provides on how tests influence both teaching and learning.

Washback usually can be done after having the result test. Here are some

questions in analyzing the washback of a test:

- Is the test designed in such a way that you can give feedback that will

be relevant to the objectives of the unit being tasted?

- Have the test-maker given students sufficient pre-test opportunities to

review the subject matter of the test?

- In the written feedback to each student, do you include comments

that will contribute to students’ formative development?

- After returning tests, do you spend class time ‘going over’ the test

and offering advice on what students should focus on in the future?

- After returning tests, do you encourage questions from students?

- If time and circumstances permit, do you offer students a chance to

discuss results in an office hour?

All of the questions above are not available for this test. Based on the

teacher’s information, this test is designed by teacher’s forum. It makes a hard to

get the washback that needs interaction between the test-maker and students. The

information is only got if students feel unsatisfied in completing the test. In

addition teacher sometimes reads the test and asks the students what are the

problems that found on the test. When the problems have been collected,

teachers just give some comments to the forum, and sometimes the forum

considers the problem and sometimes not all problems are considered.


3. Designing Classroom Language Test

Brown and Abeywickrama (2010) state that for the purpose of making practical

applications, test-makers need to consider four scenarios as they proceed through the

six steps explained before for designing an assessment. Those scenarios are reading

quiz, grammar unit test, midterm test and listening/speaking final exam.

According to Brown and Abeywickrama, there are some considerations, which

are:

- Provide clear directions.

- Write a prompt on either a narrative or a descriptive essay.

- The prompt must be on familiar topic that students will, with a

reasonable level of confidence, be able to write coherently about.

- Assign an expectation of a full page, handwritten, and no more than

two full pages.

- Students will be allowed a 90 minute time limit.

- In the prompt, include evaluation criteria: content, organization,

rhetorical discourse and grammar/mechanics.

- The final grade is to include four sub scores: content, organization,

rhetorical discourse and grammar/mechanics.

Based on those specifications, this test has clear directions that use

Indonesian so that students would understand the directions. The writer doesn’t

write directly the prompt on either narrative or description, but students can identify

the text because they have been taught before. The text is familiar for them. The

time is allocated for 75 minutes. This test can be said that it is having been fulfilled

to be a good midterm test.

The content of this test are made in the form of reading test. The test maker

used a multiple-choice. The test consists of 40 questions and shows five choices of

answer (a, b, c, d and e). The test includes two skills of language, listening and

reading.

There is no information how to score the result of this test made by students.

By analyzing this test, we may make assumption that they final score will be 100, so

that the score of each item is 2.5.


For standard based assessment, we analyze the test based on KTSP

(Kurikulum Tingkat Satuan Pendidikan). The competence standards on this test are

short functional text. The test-maker made this test based on those competence

standards. We can see each items relate to all materials which can be seen in

curriculum and competence standards.


D. Conclusion and Suggestion

After analyzing the test of mid term test for the second grade of senior high

school, it can be concluded that:

1. The test is an achievement test that measures the ability and capacity of the

students.

2. There are components that relate to principle of language assessment;

practicality, reliability, validity, authenticity and washback. The test almost

has all the components. But the washback does not complete those

principles.

3. The design of language test is appropriate so that the test can be distributed.

4. The test is based on KTSP.

5. The test-maker made the test appropriate with communicative language test.


REFERENCES

Bachman, Lyle F. 1990. Fundamental Consideration in Language Testing. Oxford:

Oxford University Press

Brown, H. Douglas and Abeywickrama. 2010. Language Assessment Principles and

Classroom Practices. Pearson Education.

Weir. Cyril J. 1990. Communicative Language Testing. Prantice Hall


Documents

Analyzing the English Test