Upload
magdalene-phelps
View
217
Download
0
Tags:
Embed Size (px)
Citation preview
Dichotomous Tests
Thomas B. Newman, MD, MPH
September 27, 2012
Thanks to Josh Galanter and Michael Shlipak
1
Overview Clarifications, chapter 1, chapter 2
material Definitions: sensitivity, specificity, prior
and posterior probability, predictive value, accuracy
2 x 2 table method Likelihood ratios - WOWO Probability and odds FP/FN confusion Test/treat thresholds 2
Clarifications
EBD errata on book website SLUBI= Self limited undiagnosed
benign illness – not a term I use with parents
3
Definitions: Sensitivity and Specificity
4
Disease status
Has disease
No disease
Total
Test Result
Positive
A B A + B
Negative
C D C + D
Total A + C B + D A + B + C + D
Sensitivity = A/ (A+C)
Specificity = D/ (B+D)
P.I.D. = Positive in Disease
N.I.H.= Negative in Health
Definitions: Positive and Negative Predictive value, Accuracy
5
Disease status
Has disease
No disease
Total
Test Result
Positive
A B A + B
Negative
C D C + D
Total A + C B + D A + B + C + D
PPV=A/(A+B)NPV=D/(C+D)
Accuracy = (A + D)/(A + B + C + D) = (A+D)/N Accuracy demonstration: screening for brain tumors
Definitions: Pretest (prior) and post-test (posterior) probability
6
Disease status
Has disease
No disease
Total
Test Result
Positive
A B A + B
Negative
C D C + D
Total A + C B + D A + B + C + D
Pretest probability =
ONLY IF SAMPLING IS “CROSS-SECTIONAL”!
(A+C)/(A+B+C+D)
Posttest probability = A/(A+B) or C/(C+D)
“Cross-sectional” sampling
7
Disease status
Has disease
No disease
Total
Test Result
Positive
A B A + B
Negative
C D C + D
Total A + C B + D A + B + C + D
PPV=A/(A+B)NPV=D/(C+D)
Subjects are sampled randomly or consecutively, so that the proportion with disease (pretest probability, prevalence) is clinically meaningful
“Case-control” sampling
8
Disease status
Has disease
No disease
Total
Test Result
Positive
A B A + B
Negative
C D C + D
Total A + C B + D A + B + C + D
PPV=A/(A+B)NPV=D/(C+D)
Subjects with and without disease are sampled separately
Proportion with disease is determined by investigator
Disease status
Has disease
No disease
Total
Test Result
Positive
A B A + B
Negative
C D C + D
Total A + C B + D A + B + C + D
Prevalence vs Pretest probability
Pretest probability is the more general term
For screening tests, pretest probability = prevalence
For diagnostic tests, pretest probability incorporates history and physical exam items
9
Post-test probability vs. Predictive value Posttest probability after a + test is the
same as positive predictive value Posttest probability after a – test is
1– negative predictive value
10
2 2 Table Method
Research vignette
“Tom, you need to call this mother. She’s really upset.”
11
Choroid Plexus Cyst
12
Fill in table
13
Pretest probability 0.0003 Sensitivity 33% Specificity 98.5%
Disease status
Trisomy 18
No Trisomy 18
Total
ChoroidPlexus Cyst
Present
Absent
Total
Likelihood Ratios
14
Likelihood ratios
A ratio of likelihoods:P(Result|Disease)P(Result|No Disease)
WOWO = With Over WithOut
Pretest odds x LR = Posttest odds (Prior odds x LR = Posterior odds)
15
What Tests Do•Their results change the probability of disease
Negative test Positive test
Reasurance TreatmentOrder a Test
•A good test moves us across action thresholds.
0% 100%
HIV+HIV-
16
Likelihood of Disease Depends on 2 Things
1. Where you started from (low, medium, high risk)
2. Length and direction of the “arrow”
Basic paradigm: – What we thought before test result
what we think now
17
Likelihood ratio Effect of test result
Very small (0.01) Greatly decreases P(disease)
Less than 1 (0.5) Decreases P(disease)
One No effect on P(disease)
More than 1 (2) Increases P(disease)
Very big (100) Greatly increases P(disease)
18
Likelihood Ratios Advantages
– Calculation of post-test probability easier (especially when disease is rare)
– Capture information for multi-level and continuous tests (next week)
Disadvantages– If either pretest or posttest probability is high
(~> 10%) you need to use odds (or a slide rule or calculator)
19
Switch to board
LR for the choroid plexus cyst example– Dichotomous test def of LR
Probability and odds
20
Can Use Slide Rule
21
False-negative confusion
Sensitivity of rapid strep test is 85% Therefore, false negative rate is 15% 15% is too high, so always culture to
confirm negative rapid strep tests
22
What’s wrong?Strep No Strep Total
Rapid Test + TP FP TP+FPRapid Test - FN TN TN+FN
TP+FN FP+TN
2 definitions of “false negative rate”– Def #1: 1-sensitivity = FN/(TP+FN). This one is easier because
it’s (assumed to be) constant.– Def #2: 1 - negative predictive value = FN/(FN+TN). This one is
harder because it depends on prior probability, but it is the one that should determine clinical decisions.
23
If prior probability of strep = 20% and specificity is 98%
False negative rate (def #2) = 15/407 = 3.7% NNC (number needed to culture) = 1/.037 = 27
to identify 1 false negative rapid test. (Pre-test probability of 20%)
At some prior probability of strep, culture after negative quick test is not indicated.
Strep No Strep TotalRapid test + 85 8 95Rapid test - 15 392 407Total 100 400 500
24
25
Sensitivity 85% Specificity 98% Prior probability = 20% Rapid test is NEGATIVE LR =
Try it with slide rule!
Similar examples:
Sensitivity of UA for UTI is only 80%, therefore always culture after a negative UA
Sensitivity of CT scan for subarachnoid hemorrhage is only 90%, therefore always do LP after a negative CT
False positive confusion is similar: 1-specificity vs. 1-positive predictive value
26
Test/Treat Thresholds
No test TreatTest
27
“X-Graph”
28
New “X-Graph”
29
Questions?
30