Embed Size (px)
Citation preview
1
The Development of A Computer The Development of A Computer Assisted Design, Analysis and Testing Assisted Design, Analysis and Testing
System for Analysing Students’ System for Analysing Students’ PerformancePerformance
Q. He & P. Tymms, CEM CENTRE, UNIVERSITY OF DURHAM, UK
Qingping He and Peter TymmsQingping He and Peter Tymms
((CEM Centre, University of Durham, UKCEM Centre, University of Durham, UK))
2
The Context
• Classic and Modern Test Theories
• Limitations of Classic Test Theory• Advantages of Item Response Theory (IRT)
• Progress in Computer Assisted Assessment Which Incorporates Test Theories
• Existing Computer Software Systems
• Expensive• Expertise• Functionalities
Q. He & P. Tymms, CEM CENTRE, UNIVERSITY OF DURHAM, UK
3
The Aim
Q. He & P. Tymms, CEM CENTRE, UNIVERSITY OF DURHAM, UK
Develop A System That is Easy and Economic for Schools (and other test organisations) to Use and Can Provide a Range of Useful Functionalities
4
• Build Items and Construct Item Banks
• Design Tests Effectively (both Classic and IRT Model – based Tests)
• Analyse Test Results – Detailed Diagnostic Analysis of Students’ Performance:• Individual Student Level • School/Class level
• Analyse Test Items and Equate Different Tests (Rasch)
• Conduct Classic and IRT- based Tests Including Computer Adaptive Tests (CATs) on Computers
CADATS – Computer Assisted Design, Analysis and Testing System
Q. He & P. Tymms, CEM CENTRE, UNIVERSITY OF DURHAM, UK
5
The Architecture
Item and Bank Building, Test Designing and Results Analysis
Component
Testing Component
Test XML FilesResponse XML Files
Q. He & P. Tymms, CEM CENTRE, UNIVERSITY OF DURHAM, UK
6
System Implementation
Both Components Have Been Developed as A Microsoft Excel Visual Basic for Application Project Embedded with Macromedia Flash Player for Easy Use and Provision of Intuitive Graphic Representation of Test Results
Q. He & P. Tymms, CEM CENTRE, UNIVERSITY OF DURHAM, UK
7
Item and Bank Building, Test Designing and Results Analysis Component – Excel Workbook: The Main Menu
Q. He & P. Tymms, CEM CENTRE, UNIVERSITY OF DURHAM, UK
8
Writing Items and Building an Item Bank
9
Designing Tests Effectively
10
Analysing Test Results: Report on Individual/School Overall Scores + More
11
Item Calibration and Equating Different Tests
12
The Test Delivery Component – Excel Workbook: The Test Interface
Q. He & P. Tymms, CEM CENTRE, UNIVERSITY OF DURHAM, UK
13
Q. He & P. Tymms, CEM CENTRE, UNIVERSITY OF DURHAM, UK
Analysis of Students’ Performance – The Hong Kong Year 7 Baseline Maths
Test Case Study
• Hong Kong Schools Involved in Middle Years Information System (MidYIS) Project
• The Year 7 Baseline Test For Hong Kong Schools
• Analysing Maths Test Results for Students from 5 Schools in Hong Kong Taking the Baseline Test in 2003
14
Q. He & P. Tymms, CEM CENTRE, UNIVERSITY OF DURHAM, UK
Maths Questions: Different Subject Areas
15
Q. He & P. Tymms, CEM CENTRE, UNIVERSITY OF DURHAM, UK
Item Difficulty and Student Ability Distributions: All 5 Schools
16
Individual Level: Individual Performance and Diagnosis Analysis
Q. He & P. Tymms, CEM CENTRE, UNIVERSITY OF DURHAM, UK
17
Individual Level: Individual Performance and Diagnosis Analysis
Q. He & P. Tymms, CEM CENTRE, UNIVERSITY OF DURHAM, UK
18
Individual Level: Individual Performance and Diagnosis Analysis
Q. He & P. Tymms, CEM CENTRE, UNIVERSITY OF DURHAM, UK
19
Individual Level: Individual Performance and Diagnosis Analysis – Comparison within School
Q. He & P. Tymms, CEM CENTRE, UNIVERSITY OF DURHAM, UK
20
Individual Level: Individual Performance and Diagnosis Analysis – Comparison with A Norm (e.g. National Average)
Q. He & P. Tymms, CEM CENTRE, UNIVERSITY OF DURHAM, UK
21
School Level: School/Class Performance and Diagnosis Analysis- Comparison with A Norm (e.g. National Average)
Q. He & P. Tymms, CEM CENTRE, UNIVERSITY OF DURHAM, UK
22
Conclusions
• The system is easy to use and provides a variety of functions
• The system can be used to generate information on the performance of students at both individual and school/class levels and the performance of test items
• The information generated by the system can be used to identify curriculum areas where students are under performing
• The use of the system will help schools improve students’ performance
Q. He & P. Tymms, CEM CENTRE, UNIVERSITY OF DURHAM, UK
