Item Item writing: art or writing: art or

science?science?

Item Item writing: art or writing: art or

science?science?Maj. Corina ISPASMaj. Corina ISPAS

NDFLTCNDFLTC

Chief of SLP SectionChief of SLP Section

Item writing

• a matter of precision more akin to a computer programming than to writing purpose

Items vs question

• not always in question format• they should be discrete• one item is part of a larger whole • they occur gradually

Item development

• Where does test-writing expertise come from?

Test specificationsSTANAG Level 1 - Test specifications – Listening [draft 1] Oct 05

General purpose To determine whether the test taker is at Level 1 (STANAG 6001) Specific purpose 1. can understand common familiar phrases and short simple sentences.

2. can understand concrete utterances, simple questions and answers and very simple conversations.

3. there are many misunderstandings of both the main idea and supporting facts. 4. can only understand spoken language from the media or among native speakers if

content is completely unambiguous and predictable. Source of texts

media, e.g. radio, TV; the internet; people; promotional materials conversations among native speakers

Type of input

phrases short simple sentences concrete utterances simple questions and answers very simple conversations spoken language from the media or among native speakers

Length of texts minimum 10 seconds - maximum 60 seconds/excerpt = 20 – 100 words Speed of delivery 80-100 words/per minute Quality of input Speech delivered at normal rate with some repetition and rewording, hesitations,

pauses, asides, backtracking and repairs. Level of background noise; semi-scripted. Language should be original and/or natural

Form of texts prose, columns (newspaper articles), bullets, graphs/ charts, handwritten (use computer fonts)

Topic of input a] military b] general

everyday needs related to survival areas such as workplace requirements everyday needs related to personal and survival areas such as minimum courtesy,

travel, meals, lodging, transportation, time, simple directions and instructions relationships/family/friends; holidays/special occasions; town/country/city;

famous people; food/drink; directions; time; season/weather; places /home /cinema /hotel; leisure activities/ hobbies/sports; countries/ languages; transport /airport/ trains/travel; shops/shopping/prices/money; clothes; people/ feelings /personal information; daily routine; occupations

Speaker characteristics

1 or 2 at any one time NS or NNS, male and female using standard dialect Normal/natural speed of delivery (80-100 wpm)

Language features

simple, common sentence patterns spoken language unambiguous and predictable basic, common vocabulary

Time allowed 30 minutes for levels 1/2 Test format mcq, matching, short answer questions, table/sentence completion, gap-filling,

T/F No. of items 10 items No. of times heard

once

details 30% gist (main idea of a simple, short conversation) 70%

Weighting GE 30% GME 70% 1 point per item

Item types or format• all items use stimulus material • all items have at least one “right”

answer• all items will have inadequate or

wrong answers

Item writing guidelines

•Format•Content

•Structure•Response development

What item serves best?• MCQ• SAQ• Essay length written discourse• Role-play• Other…

Characteristics of a good item?

• it tests what it is supposed to be testing

• it co vibrates with the profile of the target population

• content is relevant to the objective• it is not biased or completely

unknown/unavailable to candidates

• comes in a variety of shapes and forms

• facilitates the item writer enough time to ponder

• pre-set scoring procedures and assessment criteria

• co-professional input: “the more the merrier”

A good item is the result of

creative actpaneling process

public acceptance

Other characteristics

Stem

item as a whole

item as a discrete

test element

item in relation to other items

the integrity of the test as a whole

Keyed answer

Distracter

Test/Item Length

Independence

But, is it a good item?

BALANCE

SPECIFICITY

OBJECTIVITY

• Provide enough information of what it is expected

poor: Write about what you did in your summer holiday. better: Write about your summer holiday. Include details and information about places you visited and things you did.

• Indicate the amount expected in the response

poor: Tell us your opinion about globalization. better: Give 2-3 arguments in sustaining your

opinion about globalization.

• Keep items of similar format/content together

• Avoid ambiguity poor: Do you know what the word “enunciate”

means? better: The word “enunciate” means…..

• Designate specific units in the instructions.

poor: What is the total time of the journey? better: The journey took…..hours and ……

minutes.

• Indicate the criteria of assessment

• Introduce the new/unfamiliar tasks by samples or explanation

• Indicate the expected structure/ organization

• Provide any useful advice• Train your invigilators• Train the scoring team

Questions?