Test Security Considerations in the Dynamic Learning Maps System: Consortium Perspective Meagan...

Test Security Considerations in the Dynamic Learning Maps System:

Consortium Perspective

Meagan Karvonen

The present publication was developed under grant 84.373X100001 from the U.S. Department of Education, Office of Special Education Programs. The views expressed herein are solely those of the author(s), and no official endorsement by the U.S. Department should be inferred.

Consortium Considerations18 states = 18 approaches:

• Defining “irregularity”• Concerns about

exposure• Policy / legislation

around cheating• Procedures for

investigation

3

Topics

• System design to prevent problems and minimize impact

• Training• Administration• Trust but verify

4

System Design

Exposure• Many testlets to make a test• Multiple linkage levels & adaptive• Very few materials need to be

printed, even for teacher-administered testlets

• Secure browser

5

System Design

Disincentive• Instructionally embedded:

teacher choice of Essential Element & linkage level

• Adaptive in response to student performance

6

Training

• All test administrators complete training annually

• Entire module on test security– Consortium standards (shall nots)– Integrity during the assessment

process– Post-test quiz (80% to pass)

• Other modules that promote fidelity– Procedures for administering testlets– Accessibility decisions

7

Administration

No administration without these:• Signed security agreement• Successful completion of all required

training modules

8

Administration

• Instructions in Test Administration Manual (e.g., appropriate setting for testing)

• Testlet Information Pages (TIPs)• Instructions inside teacher-

administered testlets

9

Trust but Verify

• Test administration observations– Check fidelity of implementation

BUT includes some unintended behaviors

– Crowdsourced

• Statistical detection – Item responses– Student history