智慧型系統實驗室 iLab 南台資訊工程 1 Evaluation for the Test Quality of Dynamic...

1智慧型系統實驗室 iLab南台資訊工程

Evaluation for the Test Quality of Dynamic Question Generation by Particle Swarm

Optimization for Adaptive Testing

Department of Computer Science and Information EngineeringSouthern Taiwan University of Science and Technology, Taiwan

Shu-Chen CHENG, I-Chun PEN & Yu-Chih LIN

智慧型系統實驗室 iLab南台資訊工程 2

Outline1. Introduction

2. Literature

3. Method

4. Experiment

5. Conclusion

智慧型系統實驗室 iLab南台資訊工程 3

Introduction(3/4)• Item Response Theory(IRT)

– Item characteristics (difficulty, discrimination, guessing)– Testee’s ability (latent trait)

• Divided three models– Single-parameter IRT (1PL)– Two- parameter IRT (2PL)– Three-parameter IRT (3PL)

智慧型系統實驗室 iLab南台資訊工程 4

Literature(1/8)Table 1. Computerized Test Comparison

Type Computer Based Tests(CBT)

Computerized Adaptive Tests(CAT)

Rationale Classical Test Theory(CTT)

Item Response Theory(IRT)

Advantage

① Computer-aided test and scoring

② Reduce test error

① Depending on the ability to give tests

② Fewer number of quiz will be able to estimate the ability and precision

Disadvantage① Test presentation method

is fixed① Can’t go back to check and

modify the answer

智慧型系統實驗室 iLab南台資訊工程 5

Literature(3/8)• Three-Parameter Logistic Model (3PL)

ini

iniiiiiinni

ba

bacccbaXP

exp1

exp1,,,1

niX ni: correct possibility as testee’s answers itemn : testee’s ability

ia : discrimination of item i

ib : difficulty of item i

ic : pseudo-chance of item i

智慧型系統實驗室 iLab南台資訊工程 6

Experiment(1/4)• Item Exposure Rate Control

− Select 100, 500, 1000 testees and different-sized databases

智慧型系統實驗室 iLab南台資訊工程 7

Experiment(2/4)• Test overlap rate under the condition of the add

exposure control

智慧型系統實驗室 iLab南台資訊工程 8

Experiment(3/4)• Test Quality

• Select 100, 500, 1000 testees

智慧型系統實驗室 iLab南台資訊工程 9

Experiment(4/4)• Test Quality Comparison

智慧型系統實驗室 iLab南台資訊工程 10

Conclusion(1/1)• Added exposure factor slightly improved the

repetition rate of the test questions exposure control help

• Exam size greater than 300 questions, there will be better papers in the greater quality, exam greater the test quality

智慧型系統實驗室 iLab南台資訊工程 11

Thank you all for listeningThank you