itemanalysis with spss software-

8/16/2019 itemanalysis with spss software-

http://slidepdf.com/reader/full/itemanalysis-with-spss-software- 1/44

ITEMANALYSISWITH SPSS

SOFTWARE



ITEM ANALYSIS

Technique to improve testitems an instruction



11. Administer Tests12. Conduct Item Analysis

1. ReviewNational

andProfessionalStandards

7. Develo New Test!uestions". Review Test uestions

1#. Standard Settin$Study1%. Set Passin$ Standard

&. Assem'le ( erational Test)orms

1*. Produce Printed Tests +at.

2. Convene NationalAdvisory Committee

#. Develo Domain,

-nowled$e and S illsStatements

Conduct NeedsAnalysis

/. Construct Ta'le ofS eci0cations

. Develo TestDesi$n

TEST DEVELOPMENT PROCESS



WHAT IS ITEM ANALYSIS ?process that e!amines stu entresponses to in ivi ua" test itemsassessin# qua"it$ items an the test as a%ho"e

va"ua&"e in improvin# items %hich %i"" &euse a#ain in "ater tests an e"iminateam&i#uous or mis"ea in# items

va"ua&"e 'or increasin# instructors( s)i""sin test construction* an

i enti'$in# speci+c areas o' coursecontent %hich nee #reater emphasis or

c"arit$,



SEVERAL PURPOSES

-, More ia#nostic in'ormation onstu ents

• ."assroom "eve"/• etermine questions most 'oun ver$

i0cu"t1 #uessin# on -• reteach that concept

• questions a"" #ot ri#ht 2• don't waste more time on this area

•

+n %ron# ans%ers stu ents arechoosin#3• identify common misconceptions

• In ivi ua" "eve"/•

iso"ate speci+c errors the stu entsma e



4, 5ui" 'uture tests* revisetest items to make thembetter

• )no% ho% much %or) in %ritin# #ooquestions

• SHO6L7 NOT RE6SE WHOLE TESTS 338ia#nostic teachin# means respon in#

to nee s o' stu ents* so a'ter a 'e%$ears a test &an) is &ui" up anchoose a tests 'or the c"ass

•

can sprea i0cu"t$ "eve"s across$our &"ueprint 9TOS:



;, Part o' continuin# pro'essiona"eve"opment

• oin# occasiona" item ana"$sis %i"" he"p&ecome a &etter test %riter

•

ocumentin# <ust ho% #oo $our eva"uationis

• use'u" 'or ea"in# %ith parents ora ministrators i' there(s ever a ispute

• once $ou start &rin#in# out a"" theseimpressive "oo)in# statistics* parents ana ministrators %i"" &e"ieve %h$ somestu ents 'ai"e ,



CLASSICAL ITEM ANALYSISSTATISTICS

• Re"ia&i"it$ 9test "eve" statistic:

• 7i0cu"t$ 9item "eve" statistic:

•

7iscrimination 9item "eve"statistic:



TEST LEVEL STATISTIC

=ua"it$ o' the Test

• Re"ia&i"it$ an >a"i it$

• Re"ia&i"it$ .onsistenc$ o' measurement

• >a"i it$ Truth'u"ness o' response

• Overa"" Test =ua"it$

• In ivi ua" Item =ua"it$



RELIABILITY

• re'ers to the e!tent to %hich the test is "i)e"$to pro uce consistent scores,

.haracteristics/

-, The intercorrelations amon# the items 33the #reater1stron#er the re"ative num&er

o' positive re"ationships are* the #reaterthe re"ia&i"it$,

4, The length of the test 2a test %ith more items %i"" have a hi#herre"ia&i"it$* a"" other thin#s &ein# equa",



;, The content of the test 33

#enera""$* the more iverse thesu&<ect matter teste an thetestin# techniques use * the

"o%er the re"ia&i"it$,

?, Heterogeneous groups o' test

ta)ers



TYPES OF RELIABILITY

• Sta&i"it $-, Test 2 Retest



• Sta&i"it$

4, Inter 2 rater 1 O&server1 Scorer• app"ica&"e 'or most"$ essa$ questions• 6se .ohen@s appa Statistic



• Equiva"ence;, Para""e"3Forms1 Equiva"ent

6se to assess the consistenc$ o' theresu"ts o' t%o tests constructe in thesame %a$ 'rom the same content omain,



• Interna" .onsistenc$• 6se to assess the consistenc$ o' resu"ts

across items %ithin a test,

?, Sp"it 2 Ha"'



• B, u er3Richar son

Formu"a 4C 1 4-.orre"ation is etermine 'rom

a sin#"e a ministration o' a

test throu#h a stu $ o' scorevariances



• D, .ron&ach(s A"pha 9a:



Re"ia&i"it$In ices Interpretation

, - an a&ove E!ce""ent re"ia&i"it$ at the "eve" o' the &eststan ar iGe tests

, - 3 , C >er$ #oo 'or a c"assroom test

, - 3 , CJoo 'or a c"assroom test in the ran#e o' most,

There are pro&a&"$ a 'e% items %hich cou" &eimprove ,

,D- 3 , C

Some%hat "o%, This test nee s to &esupp"emente &$ other measures 9e,#,* moretests: to etermine #ra es, There are pro&a&"$some items %hich cou" &e improve ,

,B- 3 ,DCSu##ests nee 'or revision o' test* un"ess it is

quite short 9ten or 'e%er items:, The teste+nite"$ nee s to &e supp"emente &$ other

measures 9e,#,* more tests: 'or #ra in#,

,BC or &e"o%=uestiona&"e re"ia&i"it$, This test shou" not

contri&ute heavi"$ to the course #ra e* an itnee s revision,



TEST ITEM STATISTIC

Item 7i0cu"t$Percent ans%erin# correct"$

Item 7iscriminationHo% %e"" the item K'unctionsHo% va"i the item is &ase onthe tota" test score criterion



WHAT IS A WELL-FUNCTIONINGTEST ITEM?

• ho% man$ stu ents #ot it correct

97IFFI.6LTY: • %hich stu ents #ot it correct

97E.RIMINATION:





ITEM DIFFICULTY

• Item i0cu"t$ is simp"$ the percenta#eo' stu ents %ho ans%er an itemcorrect"$, In this case* it is a"so equa" to

the item mean,

7i Q o' stu ents choosin# correct"$

tota" Q o' stu ents• The item i0cu"t$ in e! ran#es 'rom C

to -CC the hi#her the va"ue* the easier

the question,



ITEM DIFFICULTY LEVEL: DEFINITION

The percenta#e o' stu ents %hoans%ere the item correct"$,

High(Difficult)

Medium(Moderate)

Low(Easy

)

<= 30% > 30% !D < "0% >="0%

C -C 4C ;C ?C BC DC C C C -CC



ITEM DIFFICULTY LEVEL: SAMPLE

#tem!o$

!o$ orrect&swers

%orrect

DifficultyLe'el

* *

3 3

+ +

Num&er o' stu ents %ho ans%ere each item BC

;C Hi#h

BC Me ium

C Me ium

C Lo%



ITEM DIFFICULTY LEVEL:QUESTIONS/DISCUSSION

• Is a test that no&o $ 'ai"etoo eas$

• Is a test on %hich no&o $ #ot-CC too i0cu"t

•

Shou" items that are tooeas$ or too i0cu"t &ethro%n out



ITEM DISCRIMINATION• Tra itiona""$* usin# hi#h an "o% scorin#

#roups 9upper 4 an "o%er 4 :• .omputeriGe ana"$ses provi e more

accurate assessment o' theiscrimination po%er o' items since itaccounts a"" responses rather than <usthi#h an "o% scorin# #roups,

• Equiva"ent to point3&iseria" corre"ation,It provi es estimate the e#ree anin ivi ua" item is measurin# the samethin# as the rest o' the items,



WHAT IS ITEMDISCRIMINATION?

• Jenera""$* stu ents %ho i %e"" onthe e!am shou" se"ect the correctans%er to an$ #iven item on the

e!am,• The 7iscrimination In e!

istin#uishes 'or each item &et%eenthe per'ormance o' stu ents %ho i%e"" on the e!am an stu ents %ho

i poor"$,



INDICES OF DIFFICULTY ANDDISCRIMINATION(BY HOPKINS AND ANTES)

In e! 7i0cu"t$ 7iscrimination

C, D a&ove >er$ Eas$ To &e iscar eC, - 2 C, B Eas$ To &e reviseC,;C 2 C, C Mo erate >er$ Joo

itemsC,-B 2 C,4 7i0cu"t To &e reviseC,-? &e"o% >er$

7i0cu"tTo &e iscar e



ITEM DISCRIMINATION: QUESTIONS / DISCUSSION

• What 'actors cou"contri&ute to "o% item

iscrimination &et%een thet%o #roups o' stu ents

•

What is a "i)e"$ cause 'or ane#ative iscriminationin e!





SAMPLE TOS

Remember Understand Apply TotalSectio

nA

?9-*;* * :

D -C 4C

Section5

B94*B* *--*-B

:

B ? -?

Section.

;9D*- *4-: D -D

Tota"-4 - 4C BC



STEPS IN ITEM ANALYSIS

-, .o e the test items/- - 'or correct an C 'or

incorrect- >ertica" 2 co"umns 9item

num&ers:- HoriGonta" 2 ro%s

9respon ents1stu ents:



TEST ITEMSNo. - 4 ; ? B D - C -- -4 -; -? , , , , BC

1 1 * 1 1 1 * * * * 1 1 1 * * * * * 1 1

2 1 1 * 1 1 1 * 1 1 1 * 1 1 1 1 * 1 1 1

# * * * 1 * * * 1 * * * 1 1 1 1 1 1 1 *

% * 1 * * * 1 * * * 1 * * 1 * * * 1 * *

/ 1 * 1 1 1 * 1 1 1 * 1 1 * 1 1 1 * 1 *

1 1 1 1 1 1 1 1 1 1 1 1 1 * 1 1 1 * 1

7 * * 1 * * * 1 * * * 1 * * * 1 * * * 1

" 1 1 * 1 1 1 * 1 1 1 * 1 1 * * * 1 * *



2. IN SPSS:

Ana"$Ge3Sca"e3Re"ia&i"it$ana"$sis 2 9 ra#1p"ace

varia&"es to Item &o!: 2Statistics 2 Sca"e i' item

e"ete 2 o),

h h $



•

Metho - 9space saver: %i"" &e use 'or this ana"$sis • R E L I A 5 I L I T Y A N A L Y S I S 3 S . A L E 9A L P H A:• Item3tota" Statistics• Sca"e Sca"e .orrecte•

Mean >ariance Item3 A"pha• i' Item i' Item Tota" i' Item• 7e"ete 7e"ete .orre"ation 7e"ete• >ARCCCC- -?,?4-- -4 ,-CB; , ?C- , BC4• >ARCCCC4 -?,D;-D -;D, ??C , ;;4 , B?4

• >ARCCC44 -?,?4-- -4 ,-?-C , ;-- , B-;• >ARCCC4; -?,?4-- -4 ,-CB; ,??C- , BC4• >ARCCC4? -?,D;-D -;D, ??C 3,C;;4 , B?4• >ARCCC? -?,? ; -4 ,D-C , B-- , BC•

>ARCCC? -?,? ; -4 , 4B4 , 4 ? , BC• >ARCCC? -?,CB4D -;C,DB ,B4;D , B4B• >ARCCCBC -?,4-CB -4 , ;B , B;; , B--• Re"ia&i"it$ .oe0cients• N o' .ases B ,C N o' Items BC•

A"pha , B;;



;, In the output ia"o# &o!/

• A"pha p"ace at the &ottom

• the correcte item tota"corre"ation is the point&iseria" corre"ation as &ases'or in e! o' test re"ia&i"it$



?, .ount thenum&er o' items

iscar e an +""up summar$ itemana"$sis ta&"e,



TEST ITEM RELIABILITY ANALYSISSUMMARY (SAMPLE)

Test Leve" o'7i0cu"t$

Num&ero' Items

ItemNum&er

Math >er$ Eas$ - 4 1/*

items3Eas$ 4 ? 2,/

Mo erate -C 4C #,%,1*,1/47i0cu"t

;C DC ,7,",&,11,4>er$

7i0cu"t-? 1 ,2%,#24



B, .ount the num&er

o' items retaine&ase on theco#nitive omainsin the TOS, .omputethe percenta#e per"eve" o' i0cu"t$,



Remem&

er

6n erstan App"$

N Ret N Ret N Ret

A ? - D ; -

C

;

5 B ; B ; ? 4. ; 4 ? D ;

Tota" -4 D - -C 4C

BC BD ?C



• Rea"istica""$/ 7o item ana"$sis to$our most important tests

• en o' unit tests* +na" e!ams 338summative eva"uation

• common e!ams %ith other teachers

9 epartmenta"iGe e!am:• common e!ams #ives &i##ersamp"e to %or) %ith* %hich is #oo

• ma)es sure that questions otherteacher s prepare are %or)in#'or $our c"ass



ITEM ANALYSIS is one area %hereeven a "ot o' other%ise ver$#oo c"assroom teachers 'a""

o%n/

• they think they're doing a good job;

• they think they've doing good

evaluation;• but without doing item analysis • They don!t really know"



ITEM ANALYSIS is not

an en in itse"'*• no point un"ess $ou

use it to revise items*an• he"ps stu ents on the&asis o' in'ormation$ou #et out o' it,



EN7 OF PRESENTATIONU

THAN 6 FOR

LISTENINJU

HA>E A RELIA5LE AN7ENVOYA5LE 7AYU,

Documents

itemanalysis with spss software-