Embed Size (px)
Citation preview
Test Item Influences and Dependencies
1
Test Item Influences and Dependencies
G. Donald Allen
Department of Mathematics
Texas A&M University
Test Item Influences and Dependencies
2
Given and n item test
• We can arrange the items in any order we wish.
• Problem. What is the best strategy for arranging problems on a test? – Placing the easy items first followed by the more
difficult ones?– Randomly placing items?– Cluster difficult and easy items separately?
Test Item Influences and Dependencies
3
The Situation
• Additional assumptions are needed, the foremost being some model of the effect of one question upon succeeding ones.
• For example, if the test begins with a very difficult item, one which most students will not solve, will this event affect their state of mind, or confidence, toward solving subsequent problems?
Test Item Influences and Dependencies
4
Goal
• What we intend to show is that if we do allow test item dependencies, there can results a substantial variation on test scores depending on the ordering of the test item.
Test Item Influences and Dependencies
5
Toward a model
Test Item Influences and Dependencies
6
This means…
Test Item Influences and Dependencies
7
The grade
Test Item Influences and Dependencies
8
Influence and DependenciesHypotheses
Test Item Influences and Dependencies
9
Parameters of Influence
Test Item Influences and Dependencies
10
The Threshold
Test Item Influences and Dependencies
11
Positive or Negative?
Test Item Influences and Dependencies
12
The factors
Test Item Influences and Dependencies
13
Boost & Fade
C on f id e n ce
Con
diti
noal
Prob
abili
ty
1
11 c
p2
1 1 p 2
1 1 1 p 2
Test Item Influences and Dependencies
14
Conditional Probabilities
PP2 | P1 p2 if c i i
1 1 1p2 if c i 1
0 1 1 1p2 if c i 0
Test Item Influences and Dependencies
15
Psychological factors
• In fact α and β are psychological factors of emotional responses to test questions.
• For poor test takers α may be larger than β, and vice-versa for good test takers.
• Using different α and β here makes the problem very difficult.
Assumption 1 ai i, i 1, . . . ,n
Test Item Influences and Dependencies
16
Rewriting the model
Test Item Influences and Dependencies
17
Conditional probability
Test Item Influences and Dependencies
18
The Two Question Test
Test Item Influences and Dependencies
19
Two Cases
Test Item Influences and Dependencies
20
What to compare?
Quantities are not comparable as is, without more information
Test Item Influences and Dependencies
21
More Assumptions
Test Item Influences and Dependencies
22
Easy precedes Difficult
Test Item Influences and Dependencies
23
Many Question Test
• Multiple Questions
• Multiple difficulties
• Constant β’s
• Constant γ’s
Test Item Influences and Dependencies
24
Many Question Test
• Two possible models– Simplistic. Question i influences Question i+1
only– More realistic. The influence for question i+1
comes from all preceding questions, though perhaps with fading memory.
Test Item Influences and Dependencies
25
Simple Model Grade
Test Item Influences and Dependencies
26
Optimize
Test Item Influences and Dependencies
27
Strategies and Observations
Test Item Influences and Dependencies
28
Expanded version
Test Item Influences and Dependencies
29
An Interesting Connection
Traveling Salesman problem
Test Item Influences and Dependencies
30
Traveling Salesman
Test Item Influences and Dependencies
31
The Two Difficulties Test
Test Item Influences and Dependencies
32
Main Result
Test Item Influences and Dependencies
33
Proof (sketch)
Test Item Influences and Dependencies
34
Test Item Influences and Dependencies
35
Test Item Influences and Dependencies
36
Step two.
Test Item Influences and Dependencies
37
Test Item Influences and Dependencies
38
Step Three
Test Item Influences and Dependencies
39
Test Item Influences and Dependencies
40
Avoiding every single detail, we have…
Test Item Influences and Dependencies
41
Multiple Item Difficulties
• Here there are no clear answers.
• It seems somewhat dependent on gamma, unless
• Essentially, we can demonstrate there is no universal algorithm for a priori arranging the test items.
Test Item Influences and Dependencies
42
Ordering the Other Test Items
Test Item Influences and Dependencies
43
Local Strategy
Test Item Influences and Dependencies
44
A Weaker Local Result
Test Item Influences and Dependencies
45
Case 1
Test Item Influences and Dependencies
46
Case 2
Test Item Influences and Dependencies
47
Case 3
Test Item Influences and Dependencies
48
Test Item Influences and Dependencies
49
Test Item Influences and Dependencies
50
