Epidemiology Kept Simple Chapter 4 Screening for Disease

Epidemiology Epidemiology Kept SimpleKept Simple

Chapter 4Chapter 4

Screening for Screening for DiseaseDisease

§4.1 Introduction §4.1 Introduction (comment)(comment)

• Accurate identification of true Accurate identification of true health status -- essential for all epi. health status -- essential for all epi. workwork

• The term The term diseasedisease refers to any refers to any health condition or health health condition or health determinantdeterminant

Sources of Epi Sources of Epi InformationInformation

• Interview (direct, surrogate)Interview (direct, surrogate)• QuestionnaireQuestionnaire• Pre-existing recordsPre-existing records• Direct examinationDirect examination

Signs, Symptoms, and Signs, Symptoms, and TestsTests

• Sign = observation by trained Sign = observation by trained observerobserver

• Symptom = observation by patient Symptom = observation by patient • Tests = any objective result Tests = any objective result

Reproducibility and Reproducibility and ValidityValidity

• Reproducibility = agreement upon Reproducibility = agreement upon repetitionrepetition

• Validity = ability to discriminate Validity = ability to discriminate those with and without diseasethose with and without disease

• A test can be reproducible but not A test can be reproducible but not validvalid

• A test can be valid but not A test can be valid but not reproduciblereproducible

§4.2 Reproducibility §4.2 Reproducibility StatisticsStatistics

• Overall agreement Overall agreement • Percentage of diagnoses that agreePercentage of diagnoses that agree• Some agreement due to chance (e.g., Some agreement due to chance (e.g.,

two coins flipped simultaneously will two coins flipped simultaneously will agree 50% of time)agree 50% of time)

• Kappa statistic - quantifies percent Kappa statistic - quantifies percent agreement above chanceagreement above chance

Kappa Statistic Kappa Statistic (Formula)(Formula)

Rater 2Rater 2

Rater Rater 11 ++ -- TotalTotal

++ aa b b pp11

-- cc dd qq11

TotalTotal pp22 qq22 NN

Text says to convert counts to proportions, but this is not necessary

1221

)(2

qpqp

bcad

Kappa Statistic Kappa Statistic (Illustration)(Illustration)

Rater 2Rater 2

Rater Rater 11 ++ -- TotalTotal

++ 2020 44 2424

-- 55 7171 7676

TotalTotal 2525 7575 100100

76.0

)76)(25()75)(24(

)5)(4()71)(20(2

Interpretation of KappaInterpretation of Kappa• Percent agreement above chancePercent agreement above chance• The closer to 1, better agreementThe closer to 1, better agreement

Range of KappaRange of Kappa InterpretationInterpretation

> .75> .75 Excellent agreementExcellent agreement

.40 to .75.40 to .75 Good agreementGood agreement

< .40< .40 Poor agreementPoor agreement

§4.3 Validity§4.3 Validity• Compare test results to definitive Compare test results to definitive

diagnostic procedure (“gold diagnostic procedure (“gold standard”)standard”)

• Each case classified asEach case classified as• TP = true positive TP = true positive • TN = true negativeTN = true negative• FP = false positiveFP = false positive• FN = false negativeFN = false negative

Organize Data in TableOrganize Data in Table

Gold StandardGold Standard

Test Test ResultResult ++ -- TotalTotal

++ TP FP TP+FP

-- FN TN FN+TN

TotalTotal TP+FN FP+TN n

DefinitionsDefinitions

• Notation: Pr(T|D) = probability test Notation: Pr(T|D) = probability test result result given given disease statusdisease status

• Sensitivity = Pr(T+|D+)Sensitivity = Pr(T+|D+)• Specificity = Pr(T-|D-)Specificity = Pr(T-|D-)• Predictive value positive = Pr(D+|Predictive value positive = Pr(D+|

T+)T+)• Predictive value negative = Pr(D-|T-)Predictive value negative = Pr(D-|T-)• Prevalence = Pr(D+)Prevalence = Pr(D+)

FormulasFormulas

• SEN = TP / (TP + FN)SEN = TP / (TP + FN)• SPEC = TN / (TN + FP)SPEC = TN / (TN + FP)• PVP = TP / (TP + FP)PVP = TP / (TP + FP)• PVN = TN / (TN + FN)PVN = TN / (TN + FN)

Example (HIV Screening Example (HIV Screening Test)Test)

Prevalence = 1000 / 1,000,000 = Prevalence = 1000 / 1,000,000 = .001.001

• SEN = 990 / 1000 = .99SEN = 990 / 1000 = .99• SPEC = 989,010 / 999,000 = .99SPEC = 989,010 / 999,000 = .99• PVP = 990 / 10,980 = .09PVP = 990 / 10,980 = .09• PVN = 989,010 / 989,020 PVN = 989,010 / 989,020 1.000 1.000

HIVHIV

ScreeninScreening Testg Test ++ -- TotalTotal

++ 990 9,990 10,980

-- 10 989,010 989,020

TotalTotal 1000 999,0001,000,000

Example (HIV Screening Example (HIV Screening Test)Test)

Prevalence = 10,000 / 1,000,000 Prevalence = 10,000 / 1,000,000 = .10= .10

• SEN = 99000 / 100,000 = .99SEN = 99000 / 100,000 = .99• SPEC = 891,000 / 900,000 = .99SPEC = 891,000 / 900,000 = .99• PVP = 99,000 / 108,000 = .92PVP = 99,000 / 108,000 = .92• PVN = 891,000 / 900,000 = .99PVN = 891,000 / 900,000 = .99

HIVHIV

ScreeninScreening Testg Test ++ -- TotalTotal

++ 99,000 9,000 108,000

-- 1,000 891,000 892,000

TotalTotal 100,000 900,0001,000,000

Conclusion Conclusion

• A test that is 99% sensitive and 99% A test that is 99% sensitive and 99% specific is used in two populationsspecific is used in two populations

• In low prevalence population PVP In low prevalence population PVP = .09= .09

• In high prevalence population, PVP = In high prevalence population, PVP = .92.92

• Therefore, PVP is a function of Therefore, PVP is a function of prevalenceprevalence

Relation Between Relation Between Predictive and Predictive and

Prevalence ValuePrevalence Value• PVP is a function of PVP is a function of

• Test SENTest SEN• Test SPECTest SPEC• prior probability (prevalence) of prior probability (prevalence) of

disease disease

• Bayesian formulas on pp. 88 - 92 Bayesian formulas on pp. 88 - 92 (NR)(NR)

Selecting a Cutoff for a Selecting a Cutoff for a TestTest

• HIV screening test detects color HIV screening test detects color change (Optical Density Ratio)change (Optical Density Ratio)

• Can change cutoff for amount of Can change cutoff for amount of color change to classify as color change to classify as “positive”“positive”

Use Low Cutoff Use Low Cutoff • no false negatives (high SEN)no false negatives (high SEN)• many false positive (low SPEC)many false positive (low SPEC)

Use High Cutoff Use High Cutoff • no false positive (high SPEC)no false positive (high SPEC)• many false negative (low SEN)many false negative (low SEN)

Use In-Between Cutoff Use In-Between Cutoff • medium no. of false positive (intermediate medium no. of false positive (intermediate

SPEC)SPEC)• medium no. of false negative (intermediate medium no. of false negative (intermediate

SEN)SEN)