Portage Collage Item Objektif

7/27/2019 Portage Collage Item Objektif

1/47

Multiple Choice Item

ConstructionD. Michele Jacobsen

Assistant Professor

[email protected]


2/47

Test Development Process

1. Statement of Goals

2. Content Outline

3. Table of Specifications

4. Item Selection 5. Item Construction

6. Composition of Instructions

7. Development of Answer Sheets

8. Construction of Answer Keys

9. Test Administration

10. Test Revision


3/47

Todays Focus

3. Table of Specifications

4. Item Selection5. Item Construction


4/47

Two Important Characteristics of Tests

Reliability

consistency

free from extraneous sources of error

Validity

how well a test measures what it is

supposed to measure


5/47

Formative vs. Summative Tests

Formative

monitor progress toward goals within a

course of study

Summative

assess overall achievement of course

goals


6/47

A. Table of Specifications

Blue print for test

Purpose

ensure proper emphasis given to all

elements of a course of study

content validity

Guide for writing items


7/47

Table of Specifications (2)

Start with Instructional Objectives

Biology 30: Circulation Unit Test

1. Identifies parts of circulation system.

2. Explains function of parts in relationto whole system.

3. Distinguishes between circulationsystem and other bodily systems.


8/47

Biology 30: Circulation

Levels Of Understanding

Content Knowledge Comprehension Application

Parts

Function

Relationto othersystems


9/47

Need to Consider and Decide:

Length of Test

Weight to be given to each objective

Weight to be given to each level oftaxonomy

Estimate number of items in each

cell


10/47

Biology 30: Circulation

Levels Of Understanding

Content Knowledge Comprehension Application

Parts 10 5 0 15

Function

5 5 5 15

Relationto other

systems

0 5 10 15

15 15 15 45


11/47

B. Item Selection

Types of Items

Objectively Scored (Selection)

true/false completion

matching

multiple choice

Subjectively Scored (Supply) interpretive exercises

essay


12/47

C. Item Construction

Selected Response Test Items

Item construction skills for valid and

reliable measures of student

achievement.

Guidelines which apply to all types

Specific Suggestions for writing each type

Advantages and Disadvantages of Each


13/47

Guidelines for Writing Objective Items

1. Construct at appropriate level ofdifficulty for examinees

2. Include Items at appropriate level ofdifficulty for purpose of test.

3. Test significant elements of acourse.

4. Write independent items.


14/47

5. Construct questions free fromextraneous reasons for problems.

6. Communicate the question in clear,

concise language.

7. In the correct alternatives,paraphrase statements from the text.

8. Exclude clues to correct answer.

II. Guidelines for Writing Objective Items


15/47

III. Guidelines for Writing Objective Items

9. Provide one correct answer.

10. Edit the Items.


16/47

Alternate Response Items

Involves the selection of one of twoalternatives

true / false

yes / no

right / wrong

fact / opinion

Mainly for Knowledge & Comprehension

Can be written at higher levels


17/47

True / False

Word statements clearly. Vague or ambiguous wording

will confuse students.

Avoid overgeneralizing.

Poor: Heavy smokin g causes lung cancer. T F

Better: Heavy smokin g often causes lung cancer. T F

Avoid Trick questions.

i .e., General Wolseley led Canadian tro op s to Mani tob a in

1870.

3 tr icks: Colon el, Br i t ish , Not yet a pro vinc e

Do not use trivial statements to pad out the number of

questions and marks to arrive at a predetermined level.


18/47

True / False

Statements should be entirely true, or

entirely false:

Unacceptable:

In King Lear, Regan ordered Gloucesters eyes to beplucked out and Gloucester died when he jumped off

the cliff of Dover.

Acceptable:

In King Lear, Regan ordered Gloucesters eyes to beplucked out. (T)

In King Lear, Gloucester died when he jumped off

the cliff at Dover. (F)


19/47

True / False

Avoid using universal descriptors such as

never, none, always, and all.

Testw ise stud ents w i l l recog nize that there are few

absolutes.

Avoid negative words, as they are often

overlooked by students. Poor: It was no t un heard of for Henry VIII to close

monaster ies in England. T F Better: Henry VIII clos ed som e monaster ies in England.

(T)


20/47

True / False

Do not include two ideas in one statement unless you areevaluating students understanding of cause and effect

relationships.

Poor: Porpo ises are able to commun icate because

they are mammals. T F

Better: Porpo ises are mammals. T F

Porpo ises are able to commun icate. T F

Provide a T and F beside each statement and ask

students to circle correct answer.

Avoids problem of stud ents wr i t ing i l leg ib le let ters .


21/47

True / False & Variations

Include more false than true statments in any given testand vary the number of false statements from test to test.

tendency to mark more statements true than false.

discrimination between those who know the content

and those who do not is greater for false expressions.

Avoid using negative statements.

Under the demands of the testing situation, students

may fail to see the negative qualifier.


22/47

Matching Items

Consist of

a column of premises

a column of responses

directions for matching the two.

Similar to multiple choice, but easier and more

efficient to construct

Can be written to assess Knowledge,Comprehension, Application, Analysis level

behaviors


23/47

Guidelines for Writing Matching Items

Provide clear instructions on how to indicate thecorrect answers.

Indicate whether the same response can be used

more than once.

Maintain grammatical consistency within and betweencolumns.

within a column: either sentence or point form

between columns: one or the other

Ensure that any matching question appears entirely

on one page.


24/47

2. Guidelines for Writing Matching Items

Provide an unequal number of premises and

responses

reduces guessing and elimination

increases measure of comprehension

Avoid designing questions which require

students to draw lines between premise and

response.

confusing for student and marker provide space for letter or number answers


25/47

3. Guidelines for Writing Matching Items

Make sure lists are homogeneous.

i.e., do not include items testing names, dates, and

events.

Instead, make every response plausible

Make the wording of the premises longer than

the wording of the responses.

Identify the items in one list with numbers andthose in the second list with letters.


26/47


27/47

Short Answer Test Items

Typically, the student is asked to reply with aword, phrase, name, or sentence, rather than a

more extended response.

Direct Questions / Short Answer

Who is the current Prime Minister of Canada?

Incomplete Sentences / Fill In the Blanks

The current Prime Minister of Canada is _____?

Items are fairly easy to construct and mark

Assess mainly knowledge, comprehension, and

some application.


28/47


29/47


30/47

III. Guidelines for Writing Short Answer Items

When an answer is to be expressed in numerical units, theunit should be stated.

Poor: If a room measures 7 metres by 4 metres, theperimeter is _____ .

Better: If a room measures 7 metres by 4 metres, theperimeter is _____ metres (or m).

Do not use too many blanks in completion items. Theemphasis should be on knowledge and comprehension,not mind reading!

Consider:In the year _____ , Prime Minister _________ signed the

__________ , which led to a ___________ which was__________ .


31/47

Multiple Choice Items

Interpretive Exercise

Guidelines for Writing

Advantages & Disadvantages


32/47

1. The capital city of Canada isa. Vancouverb. Montrealc. Toronto

*d. Ottawa

Terminology: Multiple Choice

1. Stem:presents the problem

2. Keyed Response:correct or best answer

3. Distracters: appear to be reasonable answers to the

examinee who does not know the content

4. Options:include the distracters and the keyed

response.

4

1

2

3


33/47

Ambiguity

Extrinsic

Desirable quality in

multiple choiceitems

Outside the item

Allows

discrimination

between those who

know material and

those who do not.

Intr insic

Undesirable

Shou ld be avo ided Ins ide the item

poor word ing

more than on e answer

Even those who know

content have dif f icu l ty

choo sing correct answer


34/47

Types of Multiple Choice Items

Correct Answer*

Only one correct response

Best Answer

requires examinee to select alternative

closest to being correct

fine distinctions

Multiple Answer

More than one correct or best answer


35/47

Interpretive Exercise

Usually begins with verbal, tabular or graphic

information which is the basis for 1 or more

multiple choice questions.

map, passage from a story, a poem, a cartoon

Can challenge students at various levels of

understanding

appl icat ion, analysis , syn thesis, evaluat ion

Exercise contains all information needed toanswer questions

Readily adaptive to the more important

outcomes of disciplines.


36/47

Interpretive Exercises (cont)

Examples

If student answers incorrectly it is because

they have not mastered the thinking or

reasoning required by the question, NOT

because they failed to memorize background

information.

Math questions: give students the formulas,

test ability to apply concepts, rather than ability

to memorize formulas.


37/47

Guidelines for Writing:

Multiple Choice Items

State stem in the form of a question. Weak

Canada is

a) a country

b) where you live

c) between the Pacific and Atlantic oceans

d) all of the above

Better

Between which two oceans is Canada located?

a) Atlantic and Indian

b) Atlantic and Pacific

c) Pacific and Indian


38/47

Guidelines for Writing: Stems

Place most of the subject matter in the Stem ensures full statement of problem

Eliminate extraneous material from the Stem

goal is to measure student achievement, not to present

new material maximize use of time for demonstrating understanding,

not reading ability

Avoid Negatively phrased Stems

students may miss the qualifier

use only when learning outcome requires this type of

differentiation


39/47


40/47

Example: Length of Alternatives

Neurotics are more likely than psychotics to

a. be dangerous to society

b. have delusional symptoms

c. be dangerous to themselves

*d. have insight into their own inappropriatebehavior but nevertheless feel rather

helpless in terms of dealing with theirdifficulties


41/47

II. Guidelines for Writing: MC

Make one of the alternatives the most clearly

correct or best answer

exception: multiple answer form

reduces intrinsic ambiguity reduces f rustrat ionduring test

Make distracters plausible

desire to attract students who really do NOT know the

answer to the question create distracters from elements of the correct response

improves reliability of item


42/47


43/47

III. Guidelines for Writing: MC

Avoid parallel language between the Stem andthe Correct Response

gives clues to keyed response

emphasizes testwiseness, not knowledge

Randomly distribute answers across the

alternative positions

inexperienced test writers emphasize b and c

alternatives (hide the answer!!)

do NOT use an interpretable order of keyed responses


44/47

IV. Guidelines for Writing: MC

Use qualifiers such as all of the

above and none of the above

sparingly

testwise students w i l l use proc ess ofel im inat ion to select answer

do NOT use to pad out the distracters

because you cannot th ink of another

one.


45/47

Advantages of Multiple Choice Items

allow more adequate sampling of content.

tend to more effectively structure the problem

to be addressed

items can be more efficiently and reliably

scored than supply items

different response alternatives can provide

diagnostic feedback (item analysis) items can be constructed to address various

levels of cognitive complexity


46/47

Disadvantages of Mult.Choice Items

difficult & time consuming to construct gooditems

leads to emphasis on other selected response item types

can lead the instructor to favour simple recall

of facts high degree of dependence on students

reading and instructors writing ability

can be difficult to achieve clarity of expression

measuring synthesis and evaluation can bedifficult

inappropriate for measuring outcomes that

require skilled performance


47/47

Wrap-Up

Documents

Portage Collage Item Objektif