Test Security Considerations in the Dynamic Learning Maps System:
Consortium Perspective
Meagan Karvonen
The present publication was developed under grant 84.373X100001 from the U.S. Department of Education, Office of Special Education Programs. The views expressed herein are solely those of the author(s), and no official endorsement by the U.S. Department should be inferred.
Consortium Considerations18 states = 18 approaches:
• Defining “irregularity”• Concerns about
exposure• Policy / legislation
around cheating• Procedures for
investigation
3
Topics
• System design to prevent problems and minimize impact
• Training• Administration• Trust but verify
4
System Design
Exposure• Many testlets to make a test• Multiple linkage levels & adaptive• Very few materials need to be
printed, even for teacher-administered testlets
• Secure browser
5
System Design
Disincentive• Instructionally embedded:
teacher choice of Essential Element & linkage level
• Adaptive in response to student performance
6
Training
• All test administrators complete training annually
• Entire module on test security– Consortium standards (shall nots)– Integrity during the assessment
process– Post-test quiz (80% to pass)
• Other modules that promote fidelity– Procedures for administering testlets– Accessibility decisions
7
Administration
No administration without these:• Signed security agreement• Successful completion of all required
training modules
8
Administration
• Instructions in Test Administration Manual (e.g., appropriate setting for testing)
• Testlet Information Pages (TIPs)• Instructions inside teacher-
administered testlets
9
Trust but Verify
• Test administration observations– Check fidelity of implementation
BUT includes some unintended behaviors
– Crowdsourced
• Statistical detection – Item responses– Student history