WRITING QUALITY MULTIPLE CHOICE QUESTIONS Barbara Allen, Vernon-Verona-Sherrill Barbara Patterson,...

WRITING QUALITY MULTIPLE CHOICE QUESTIONS

Barbara Allen, Vernon-Verona-Sherrill Barbara Patterson, Liverpool

MATCH OBJECTIVEMATCH OBJECTIVE

•Assessment measures if students mastered desired objectives

QUALITY MULTIPLE CHOICE QUALITY MULTIPLE CHOICE QUESTIONS AREQUESTIONS ARE• A complete concept

• Clear and concise

• Reflect the main points or significant details of the passage

• Longer than the answer choices

QUALITY MULTIPLE CHOICE QUALITY MULTIPLE CHOICE QUESTIONS ARE NOTQUESTIONS ARE NOT

• Answerable without listening to or reading the passage

• Dependent on one word

• Written in the negative

• Grammar-based

QUALITY MULTIPLE CHOICE QUALITY MULTIPLE CHOICE ANSWERS AREANSWERS ARE

• Phrased positively • Approximately equal in

lengthoSimilar in construction, vocabulary or contexto Internally consistent

• Mutually exclusive• Distractors should be plausible and

unquestionably the wrong answer

QUALITY MULTIPLE CHOICE QUALITY MULTIPLE CHOICE ANSWERS ARE NOTANSWERS ARE NOT

• Identical in wording from the passage

• Contradictory

• Irrelevant

• Dependent on prior knowledge

• Written in the negative

• Written using all of the above or none of the above

What is the topic of this news item?

1. The environment2. Pro sports3. International politics4. Domestic travel

VALID OR INVALID?

WHY?

\\

Qual è il tempo nel nord-ovest d'Italia a domenica?

1. nevica2. piove 3. è nuvoloso4. c’è il sole

VALID OR INVALID?

WHY?

At what time do the doors open for the game?

1. 7:00 pm2. 7:00 am3. 17:004. 11:00

VALID OR INVALID?

WHY?

Welches Problem hat er nicht Donald Duck?1.Er muss seine Rechnungen zu bezahlen.2.Er hat nicht genug Geld.3.Er hat kein Papier mehr. 4.Seine Miete ist fallig.

VALID OR INVALID?

WHY?

POSSIBLE READING POSSIBLE READING COMPREHENSION ITEMS:COMPREHENSION ITEMS:

Forms, signs, labels, tickets, programs, timetables, maps, menus, celebration invitations and announcements, advertisements, short articles, informal notes, brief correspondence, hotel literature, telephone books

LISTENING COMPREHENSION LISTENING COMPREHENSION SETTINGSETTING

• Provides students with the background information they would have in an authentic listening/speaking/responding situation in the target language.

• Usually includes where they are and to whom they are listening.

LISTENING STIMULUS SHOULD LISTENING STIMULUS SHOULD BEBE

• Spoken in the target language

• One consistent message, not “tricky”

• Appropriate to the age level and interest of students

• Length and complexity appropriate to comprehension level of students

ACCEPTABLE SOURCES FOR TEST ACCEPTABLE SOURCES FOR TEST ITEMSITEMS

• Authentic cultural material

• Articles of general interest to students

• Appropriate for students’ age level

• From current magazines, newspapers, Internet, notes, texts, radio and television broadcasts

UNACCEPTABLE SOURCES FOR UNACCEPTABLE SOURCES FOR TEST ITEMSTEST ITEMS

• Textbooks and ancillary materials

• Exam review books

• Commercially published tests

• Materials frequently used in class work

• Teachers’ imaginations

• Controversial or objectionable

• Sensitive subjects (e.g. disasters)

• Humor

PERCENTAGE OF PERCENTAGE OF DIFFICULTYDIFFICULTY

• Include type of questioning used throughout the unit

• Easiest question first

• Within each section, place questions easiest to the most difficult

• End with an easy question

Arrange questions so there isn’t a pattern of answers

Vary the ratio of answers

LEVEL OF DIFFICULTYLEVEL OF DIFFICULTYUsing Bloom’s Taxonomy

•Focus of the stem:Is it about the main idea? (easier)

Is it regarding a significant detail? (more difficult)

Is it inferential? (very challenging)

•Options with a high degree of similarity require greater understanding in order to identify a correct response

CALCULATING ITEM CALCULATING ITEM DIFFICULTYDIFFICULTY

The percentage of students that correctly answered the item.

• Calculation: Divide number of students who got an item correct by total number of students who answered it.Higher the value, easier the item90+ answered correctly - very easy, shouldn’t be reused Below 20 answered correctly - very difficult & should be reviewed for validity or students didn’t understand conceptIdeal value is around 60%

QUESTIONS FOR QUESTIONS FOR REMEMBERINGREMEMBERING

What happened after...?How many...?What is...?Who was it that...?Can you name ...?Describe what happened after…Who spoke to...?Which is true/false …?SelectChoose

QUESTIONS FOR QUESTIONS FOR UNDERSTANDINGUNDERSTANDING

Can you explain why…?How would you explain…?Who do you think...?What was the main idea...?What do you think could have happened

next ...?Does everyone act in the way that ……..

does?Describe …

QUESTIONS FOR APPLYINGQUESTIONS FOR APPLYING

Do you know of another instance where…?Group by characteristics such as…Which factors would you change if…?What questions would you ask of…?Compute/calculate …

QUESTIONS FOR ANALYZINGQUESTIONS FOR ANALYZING

How is...similar to...?Which are other possible outcomes?Why did...changes occur?An explanation for what must have happened when...?What are some or the problems of...?What were some of the motives behind..?What was the turning point?What was the problem with...?Which events could not have happened? If. ..happened, what might the ending have been?Sequence/order/arrange …

QUESTIONS FOR QUESTIONS FOR EVALUATINGEVALUATING

What is the better solution to/for...?Why is … a good/bad thing?How would an ?/American have handled...?What changes to..?What are the consequences..?What influence will....have on our lives?What are the pros/cons of....?Why is ....of value? What are the alternatives of..?Who will gain/loose ….? 

QUESTIONS FOR CREATINGQUESTIONS FOR CREATING

A possible solution to…How would … deal with...?What would happen if ...?What could improve …?

QUESTIONSQUESTIONS