Upload
eugene-sutton
View
220
Download
0
Embed Size (px)
Citation preview
1智慧型系統實驗室 iLab南台資訊工程
Evaluation for the Test Quality of Dynamic Question Generation by Particle Swarm
Optimization for Adaptive Testing
Department of Computer Science and Information EngineeringSouthern Taiwan University of Science and Technology, Taiwan
Shu-Chen CHENG, I-Chun PEN & Yu-Chih LIN
智慧型系統實驗室 iLab南台資訊工程 2
Outline1. Introduction
2. Literature
3. Method
4. Experiment
5. Conclusion
智慧型系統實驗室 iLab南台資訊工程 3
Introduction(3/4)• Item Response Theory(IRT)
– Item characteristics (difficulty, discrimination, guessing)– Testee’s ability (latent trait)
• Divided three models– Single-parameter IRT (1PL)– Two- parameter IRT (2PL)– Three-parameter IRT (3PL)
智慧型系統實驗室 iLab南台資訊工程 4
Literature(1/8)Table 1. Computerized Test Comparison
Type Computer Based Tests(CBT)
Computerized Adaptive Tests(CAT)
Rationale Classical Test Theory(CTT)
Item Response Theory(IRT)
Advantage
① Computer-aided test and scoring
② Reduce test error
① Depending on the ability to give tests
② Fewer number of quiz will be able to estimate the ability and precision
Disadvantage① Test presentation method
is fixed① Can’t go back to check and
modify the answer
智慧型系統實驗室 iLab南台資訊工程 5
Literature(3/8)• Three-Parameter Logistic Model (3PL)
ini
iniiiiiinni
ba
bacccbaXP
exp1
exp1,,,1
niX ni: correct possibility as testee’s answers itemn : testee’s ability
ia : discrimination of item i
ib : difficulty of item i
ic : pseudo-chance of item i
智慧型系統實驗室 iLab南台資訊工程 6
Experiment(1/4)• Item Exposure Rate Control
− Select 100, 500, 1000 testees and different-sized databases
智慧型系統實驗室 iLab南台資訊工程 7
Experiment(2/4)• Test overlap rate under the condition of the add
exposure control
智慧型系統實驗室 iLab南台資訊工程 8
Experiment(3/4)• Test Quality
• Select 100, 500, 1000 testees
智慧型系統實驗室 iLab南台資訊工程 9
Experiment(4/4)• Test Quality Comparison
智慧型系統實驗室 iLab南台資訊工程 10
Conclusion(1/1)• Added exposure factor slightly improved the
repetition rate of the test questions exposure control help
• Exam size greater than 300 questions, there will be better papers in the greater quality, exam greater the test quality
智慧型系統實驗室 iLab南台資訊工程 11
Thank you all for listeningThank you