How to spot a quality screening measure

How to spot a quality screening measure

Frances Page Glascoe

www.pedstest.com

www.forepath.org

STANDARDIZATION

Children with disabilities ( 16% - 18% of total)

Naturalistic sample

Large, nationally representativeProportionate Demographics: minorities, parents’ level of education, broad geographic locations, Spanish speakers, etc.

Inter-rater--at least 85%

Test-retest--at least 85%

Internal consistency > .80

RELIABILITY

How well does a screen actually measure what

the authors claim?

Good

Poor

Good

Poor

SCREEN DX TEST

?

?

DX TESTGood

Poor

SCREEN

VALIDITYConcurrent: battery should sample same domains as screen Discriminant: information on how well various kinds of problems are detected

Predictive: optional but helpful for showing that items sample enduring aspects of development

Criterion Measure/Gold Standard

SCREEN

CRITERION-RELATED VALIDITY

=

ACCURACY

Screening sorts those who probably have

problems from those who probably don’t

ACCURACYSPECIFICITY = % of children without problems correctly detected (e.g., by passing, above cutoffs scores)SENSITIVITY = % of children with problems correctly detected

70% to 80% is the standard

SPECIFICITY

SENSITIVITY

Accuracy of the Denver-II

Denver-II PASS 86

FAIL 27

69 8

Specificity = 69/86 = 80%

17 10

86 18

Developmental DX

NO YES

Sensitivity = 10/18 = 56%

ACCURACY:ACROSS AGE RANGES

AGE SENSITIVITY SPECIFICITY

N % N %

0 - 18 mos 3/4 75 66/82 80

18 mos - 3 yrs 27/34 79 117/149 79

3 - 4 1/2 yrs 26/35 74 118/165 72

4 1/2 - 8 yrs 42/57 74 172/245 70

TOTAL 98/130 75 473/641 74

Rigorously Peer-reviewed

Because of the enormous expense involved in standardization, reliabilty and validity research and accuracy studies, tests are expensive to create, keep current, provide translations, and to ensure that users and researchers have support and guidance. For these reasons, tests costs money and it is why few measures can be photocopied. The money, however small, you spend when purchasing measures goes to their continued development.

Detection rates WITH Screening Tests

70% to 80% of children with developmental disabilities correctly identified

Squires et al, JDBP. 1996;17:420 - 427

80% to 90% of children with mental health problems correctly identified

Sturner, JDBP . 1991; 12: 51-64

Most over-referrals on standardized screens are children with below average development and psychosocial risk factors

Glascoe, APAM. 2001; 155:54-59. -