Petch Rawdaree

Assessing diagnostic test Assessing diagnostic test s in Endocrinology s in Endocrinology

Special characteristic of endocrine Special characteristic of endocrine disordersdisorders

Properties of diagnostic testsProperties of diagnostic tests Probability of having diseasesProbability of having diseases Example of diagnostic tests and Example of diagnostic tests and

assessment of the testsassessment of the tests

Petch Rawdaree, MD. MSc, DLSHTMPetch Rawdaree, MD. MSc, DLSHTM

77th th July, 2007July, 2007

Petch Rawdaree

The Endocrine patients The Endocrine patients

General considerations General considerations Special features of endocrine illness Special features of endocrine illness

Discovery through screening Discovery through screening Quantitative rather than qualitative abn Quantitative rather than qualitative abn

ormalitiesormalities Overlap with other diseases Overlap with other diseases Unique features of reproductive disorde Unique features of reproductive disorde

rsrs

Petch Rawdaree

Laboratory testing of endocr Laboratory testing of endocr ine function ine function

Pulsatile hormone secretion Pulsatile hormone secretion Diurnal variation Diurnal variation

Cyclic variation Cyclic variationAgeAge

Sleep entrainment Sleep entrainment Hormone antagonism Hormone antagonism

Dynamic testing Dynamic testing Hormone and metabolite interaction Hormone and metabolite interaction

Protein binding Protein binding Laboratory error Laboratory error

Petch Rawdaree

Evaluation of patients with Evaluation of patients with Endocrine disordersEndocrine disorders

HistoryHistory Details of timingDetails of timing SequenceSequence Changes of diet or avtivityChanges of diet or avtivity Relationship to menstrual cycleRelationship to menstrual cycle Changes in weight or sizeChanges in weight or size Alterations in mood or sleep patternAlterations in mood or sleep pattern Family historyFamily history

Physical examinationPhysical examination GeneralGeneral Targeted examinationTargeted examination

Petch Rawdaree

Attributes of a testAttributes of a test Validity Validity Reliability Reliability Clinical relevance Clinical relevance Feasibility Feasibility CostCost

Petch Rawdaree

Degree to which the data measure Degree to which the data measure what they were intended to measure what they were intended to measure (i.e. the true state of the phenomenon)(i.e. the true state of the phenomenon)

Extent to which repeated Extent to which repeated measurements of a stable phenomenon measurements of a stable phenomenon (by different people, times,places, (by different people, times,places, instruments) get similar resultsinstruments) get similar results

Validity (accuracy)

Reliability (Reproducibility, precision)

Petch Rawdaree

“normal” diseased

false positives

false negatives

Petch Rawdaree

Properties of testProperties of test

SensitivitySensitivity SpecificitySpecificity Predictive value of positive testPredictive value of positive test Predictive value of negative testPredictive value of negative test Likelihood ratio of positive testLikelihood ratio of positive test Likelihood ratio of negative testLikelihood ratio of negative test Pretest probabilityPretest probability Posttest probabilityPosttest probability

Petch Rawdaree

Sensitivity and Sensitivity and SpecificitySpecificity

SensitivitySensitivity=ability of a test to detect the disease =ability of a test to detect the disease among persons who have itamong persons who have it=proportion of people with disease who =proportion of people with disease who have positive testhave positive test

specificityspecificity= ability of a test to confirm normal status = ability of a test to confirm normal status among people without diseaseamong people without disease= proportion of people without disease = proportion of people without disease who have negative test resultwho have negative test result

Petch Rawdaree

Relationship between test’s Relationship between test’s result and truthresult and truth

Disease

Present Absent

Test result

positive True-positive

(a)False-positive

(b)

negativeFalse-negative

(c)True-negative

(d)

a + c b + d

Sensitivity = a/(a+c)Specificity = d/(b+d)

Positive predictive value = a/(a+b)Negative predictive value = d/(c+d)

Prevalence = (a+c)/(a+b+c+d)Odds = (a+c)/(b+d)

Petch Rawdaree

Test result in primary care Test result in primary care settingsetting

hypothyroidismhypothyroidism

PresentPresent AbsentAbsent

TSH resultTSH result

highhigh99 99

(a)(a)

9999

(b)(b)

normalnormal11

(c)(c)

98019801

(d)(d)

100100 99009900

Sensitivity = 99/(99+1)=99%Specificity = 9801/(99+9801)=99%

PPV = 99/(99+99)=50%NPV = 9801/(9801+1)=100%

Prevalence = (a+c)/(a+b+c+d)=1%Odds = (a+c)/(b+d)=100/9900=1:99

N= 10,000

Petch Rawdaree

Test results in an Test results in an endocrinologist’s officeendocrinologist’s office

hypothyroidismhypothyroidism

PresentPresent AbsentAbsent

TSH resultTSH result

highhigh 990990

(a)(a)

9090

(b)(b)

normalnormal1010

(c)(c)

89108910

(d)(d)

10001000 90009000

Sensitivity = 990/(990+10)=99%Specificity = 8910/(90+8910)=99%

PPV = 990/(990+90)=91.6%NPV = 8910/(10+8910)=99.9%

Prevalence = (a+c)/(a+b+c+d)=10%Odds = (a+c)/(b+d)=1:9

N= 10,000

Petch Rawdaree

Relationship wetween UAER ( gold Relationship wetween UAER ( gold standard) and UACR(test) and two standard) and UACR(test) and two

levels of test’s resultslevels of test’s results UAER

> 28.8mg/d < 28.8mg/d

UACR> 15 mg/g 69 14

< 15 mg/g 0 40

69 54

Sensitivity = 69/(69+0)=100%Specificity = 40/(14+40)=74%

Positive PV = 69/(69+14)=83.1%Negative PV = 40/(0+40)=100%

Prevalence = (a+c)/(a+b+c+d)=56%Odds = (a+c)/(b+d)=69:54

N= 123

Petch Rawdaree

Relationship between UAER(gold Relationship between UAER(gold standard) and UACR(test) and two standard) and UACR(test) and two

levels of test’s resultslevels of test’s resultsUAERUAER

> > 28.8mg/28.8mg/

dd

< < 28.8mg/28.8mg/

dd

UACRUACR

> 26.8 > 26.8 mg/gmg/g 6161 66

< 26.8 < 26.8 mg/gmg/g 88 4848

6969 5454

Sensitivity = 61/(61+8)=88.4%Specificity = 48/(6+48)=88.9%

PPV = 61/(61+6)=91%NPV = 48/(8+48)=85.7%

Prevalence = (a+c)/(a+b+c+d)=56%Odds = (a+c)/(b+d)=69:54

N= 123

Petch Rawdaree

Relationship between prevalence Relationship between prevalence of disease, sensitivity, specificity, of disease, sensitivity, specificity,

and predictive valueand predictive value

DiseaseDisease

PresenPresentt

AbsentAbsent

Test Test resultresult

+ ve+ ve a a 2b2b

- veve c c 2d 2d

a + ca + c 2(b + d)2(b + d)

Petch Rawdaree

Sensitivity and specificity Sensitivity and specificity do not answer clinical do not answer clinical

questions:questions: If a patient’s test result is positive, If a patient’s test result is positive,

what is the probability that he or she what is the probability that he or she has the disease being tested?has the disease being tested?

If the result is negative, what is the If the result is negative, what is the probability that the patient does not probability that the patient does not have the disease?have the disease?

Petch Rawdaree

Predictive value of testPredictive value of test Predictive value of positive testPredictive value of positive test

= probability of the person having the = probability of the person having the disease when the test is positivedisease when the test is positive= proportion of the subjects who had = proportion of the subjects who had positive test results had the diseasepositive test results had the disease

Predictive value of negative testPredictive value of negative test= probability of the person not having = probability of the person not having disease when the test is negativedisease when the test is negative= proportion of the subject who had = proportion of the subject who had negative test results were free of the negative test results were free of the diseasedisease

Petch Rawdaree

Test result in primary care Test result in primary care settingsetting

hypothyroidismhypothyroidism

PresentPresent AbsentAbsent

TSH resultTSH result

highhigh99 99

(a)(a)

9999

(b)(b)

normalnormal11

(c)(c)

98019801

(d)(d)

100100 99009900

Sensitivity = 99/(99+1)=99%Specificity = 9801/(99+9801)=99%

PPV = 99/(99+99)=50%NPV = 9801/(9801+1)=100%

Prevalence = (a+c)/(a+b+c+d)=1%Odds = (a+c)/(b+d)=100/9900=1:99

N= 10,000

Petch Rawdaree

Test results in an Test results in an endocrinologist’s officeendocrinologist’s office

hypothyroidismhypothyroidism

PresentPresent AbsentAbsent

TSH resultTSH result

highhigh 990990

(a)(a)

9090

(b)(b)

normalnormal1010

(c)(c)

89108910

(d)(d)

10001000 90009000

Sensitivity = 990/(990+10)=99%Specificity = 8910/(90+8910)=99%

PPV = 990/(990+90)=91.6%NPV = 8910/(10+8910)=99.9%

Prevalence = (a+c)/(a+b+c+d)=10%Odds = (a+c)/(b+d)=1:9

N= 10,000

Petch Rawdaree

ValidityValidity

In a clinical settingIn a clinical setting

Likelihood ratio (LR) better

LR= Probability of result in diseased people Probability of result in non-dis. people

In tests measuring dicotomous variables (i.e.yes/no)

LR+ = Sensitivity/(1-Specificity)LR- = (1-Sensitivity)/Specificity

Petch Rawdaree

Likelihood ratiosLikelihood ratios Likelihood ratio positive (LR+)Likelihood ratio positive (LR+)โอกาสที่� โอกาสที่� test test จะให้�ผล บวก ในคนที่�เป็�นโรค เที่�ยบ ก�บคนที่�ไม่�เป็�นโรคจะให้�ผล บวก ในคนที่�เป็�นโรค เที่�ยบ ก�บคนที่�ไม่�เป็�นโรค

= sensitivity/false-positive= sensitivity/false-positive= sensitivity/(1-specificity)= sensitivity/(1-specificity)

““The higher ratio (>1) , the better the test”The higher ratio (>1) , the better the test”

Likelihood ratio negative (LR –)Likelihood ratio negative (LR –)โอกาสที่� โอกาสที่� test test จะให้�ผล ลบ ในคนที่�เป็�นโรค เที่�ยบก�บ คนที่�ไม่�เป็�นโรคจะให้�ผล ลบ ในคนที่�เป็�นโรค เที่�ยบก�บ คนที่�ไม่�เป็�นโรค

= false-negative/specificity= false-negative/specificity= (1-sensitivity)/specificity= (1-sensitivity)/specificity

““The smaller ratio ( close to 0 ), the better the test”The smaller ratio ( close to 0 ), the better the test”““no effect of prevalence on likelihood no effect of prevalence on likelihood

ratios”ratios”

Petch Rawdaree

Likelihood ratiosLikelihood ratios

Disease

Present Absent

Test result

positive True-positive

(a)False-positive

(b)

negativeFalse-negative

(c)True-negative

(d)

a + c b + d

Sensitivity = a/(a+c)Specificity = d/(b+d)

LR+ = [a/(a+c)]/[b/(b+d)LR- = [c/(a+c)/d(b+d)]

Prevalence = (a+c)/(a+b+c+d)Odds = (a+c)/(b+d)

Petch Rawdaree

Test results in an Test results in an endocrinologist’s officeendocrinologist’s office

hypothyroidismhypothyroidism

PresentPresent AbsentAbsent

TSH resultTSH result

highhigh 990990

(a)(a)

9090

(b)(b)

normalnormal1010

(c)(c)

89108910

(d)(d)

10001000 90009000

Sensitivity = 990/(990+10)=99%Specificity = 8910/(90+8910)=99%

LR+ = [990/(990+10)]/[90/(90+8910)=99LR- = [10/(990+10)/[8910/(90+8910)]=0.01

Prevalence = (a+c)/(a+b+c+d)=10%Odds = (a+c)/(b+d)=1:9

N= 10,000

Petch Rawdaree

Test results in an Test results in an endocrinologist’s officeendocrinologist’s office

hypothyroidismhypothyroidism

PresentPresent AbsentAbsent

TSH resultTSH result

highhigh 990990

(a)(a)

9090

(b)(b)

normalnormal1010

(c)(c)

89108910

(d)(d)

10001000 90009000

Sensitivity = 990/(990+10)=99%Specificity = 8910/(90+8910)=99%

LR+ = [990/(990+10)]/[90/(90+8910)=99LR- = [10/(990+10)]/[8910/(90+8910)]=0.01

Prevalence = (a+c)/(a+b+c+d)=10%Odds = (a+c)/(b+d)=1:9

N= 10,000

Petch Rawdaree

Interpretation of Likelihood Interpretation of Likelihood RatiosRatios

Likelihood ratio +Likelihood ratio +The higher, the betterThe higher, the better LR+ >10 conclusive LR+ >10 conclusive

changes from pre- to changes from pre- to post-test probabilitypost-test probability

5-10 moderate shift5-10 moderate shift 2-5 small changes in 2-5 small changes in

probabilityprobability 1-2 rarely important 1-2 rarely important

changeschanges

Likelihood ratio –Likelihood ratio –Closing to 0 is betterClosing to 0 is better LR- <0.1 conclusive LR- <0.1 conclusive

changes from pre- to changes from pre- to post-test probabilitypost-test probability

0.1-0.2 moderate 0.1-0.2 moderate shiftshift

0.2-0.5 small changes 0.2-0.5 small changes in probabilityin probability

0.5-1 rarely 0.5-1 rarely important changesimportant changes

Petch Rawdaree

Likelihood ratios for the diagnosis of Likelihood ratios for the diagnosis of malignancy in euthyroid patients with a single malignancy in euthyroid patients with a single

or dominant thyroid noduleor dominant thyroid nodule

Prevalence (pretest probability)(%)

No. of patients included

Test Result LR(95%CI)

20 132 FNAB guided with U/S

Malignant 226(4.4-11.7)

Suspicious 1.3 (0.52-3.2)

Insufficient

2.7 (0.52-15)

benign 0.24 (0.11-0.52)

7-22 868 FNAB not guided

Malignant 34 (15-74)

Suspicious 1.7 (0.94-3)

Insufficient

0.5 (0.27-0.76)

benign 0.23 (0.13-0.42)

Petch Rawdaree

Characteristics of laboratory Characteristics of laboratory tests in thyroid disorderstests in thyroid disorders

testtest SeSe SpSp LRLR++

LRLR--

hyperthyroidismhyperthyroidism

Total TTotal T44 90.90.00

90.90.00

9.09.0 0.10.111

TT33 RIA RIA 96.96.00

96.96.00

24.24.00

0.00.044

TSHTSH 99.99.00

99.99.00

99.99.00

0.00.011

Primary hypothyroidismPrimary hypothyroidism

Total TTotal T44 90.90.00

85.85.00

6.06.0 0.10.122

TT33 RIA RIA 95.95.00

95.95.00

19.19.00

0.00.055

TSHTSH 99.99.00

99.99.00

99.99.00

0.00.011

Petch Rawdaree

Example: calculation LRs Example: calculation LRs from Sensitivity and from Sensitivity and

specificityspecificity LR+ LR+

=[sensitivity]/[false positive error]=[sensitivity]/[false positive error]

= [sensitivity]/[1-specificity]= [sensitivity]/[1-specificity]

=90/[100-85]=90/[100-85]

=90/15 = 6=90/15 = 6 LR-LR-

=[false negative error] /[specificity]=[false negative error] /[specificity]

=[100-90]/85=[100-90]/85

=10/85 = 0.12=10/85 = 0.12

Petch Rawdaree

Characteristics of laboratory Characteristics of laboratory tests in pheochromocytomatests in pheochromocytoma

testtest SeSe SpSp LR+LR+ LR-LR-

Urine VMAUrine VMA 42.42.00

99.99.00

42.42.00

0.50.599

Urine metanephrinesUrine metanephrines 79.79.00

93.93.00

11.11.33

0.20.233

Plasma normetanephrine Plasma normetanephrine >0.66 nmol/L >0.66 nmol/L

or metanephrine > 0.30 or metanephrine > 0.30 nmol/lnmol/l

100100.0.0

85.85.00

6.76.7 0.00.0

Colnidine suppression testColnidine suppression test 98.98.22

97.97.22

35.35.11

0.00.022

Plasma catecholamine after Plasma catecholamine after supine 30 min.supine 30 min.

94.94.00

97.97.00

31.31.33

0.00.066

Petch Rawdaree

Characteristics of tests for Characteristics of tests for thyroid nodulethyroid nodule

TestTest SeSe SpSp LR+LR+ LR-LR-

Thyroid nodule (benign vs. mal.)Thyroid nodule (benign vs. mal.)

Local painLocal pain 33 9494 0.50.5 1.01.0

Hard to palpationHard to palpation 4242 8989 3.83.8 0.540.54

Fix to surrounding tissueFix to surrounding tissue 3131 9494 5.25.2 0.730.73

Cold nodule on iodine131Cold nodule on iodine131 8383 2525 1.11.1 0.680.68

Solid or mixed lesion on Solid or mixed lesion on ultrasoundultrasound

9595 1818 1.21.2 0.280.28

No change in size after T4 No change in size after T4 suppressionsuppression RxRx

8585 2525 1.11.1 0.60.6

FNBA: positive for malignancyFNBA: positive for malignancy -- -- 74.074.0 --

FNBA: suspiciousFNBA: suspicious -- -- 1.21.2 --

FNBA: benignFNBA: benign 0.090.09 --

Petch Rawdaree

Characteristics of tests for Characteristics of tests for Cushing’s syndromeCushing’s syndrome

TestTest SeSe SpSp LR+LR+ LR-LR-

Cushing’s syndromeCushing’s syndrome

Plasma cortisol 8.00 am >13-20 Plasma cortisol 8.00 am >13-20 ug/100mlug/100ml

83.083.0 67.067.0 2.52.5 0.160.16

Plasma cortisol midnight > 6-15 Plasma cortisol midnight > 6-15 ug/mlug/ml

96.096.0 96.096.0 24.024.0 0.040.04

24-hr urine free cortisol >20-181 24-hr urine free cortisol >20-181 ug/dayug/day

94.094.0 91.091.0 10.010.0 0.070.07

Low-dose dexamethasone Low-dose dexamethasone suppression: urine free cortisol > suppression: urine free cortisol > 0.019-0.025mg/day0.019-0.025mg/day

95.095.0 97.097.0 32.032.0 0.050.05

Cushing’s diseaseCushing’s disease

High-dose dexa- suppression testHigh-dose dexa- suppression test

Urine free cortisol suppressed > Urine free cortisol suppressed > 50%50%

90.090.0 79.079.0 4.34.3 0.130.13

High-dose dexa- suppression testHigh-dose dexa- suppression test

Urine free cortisol suppressed > Urine free cortisol suppressed > 80%80%

81.081.0 92.092.0 10.110.1 0.210.21

Petch Rawdaree

Prevalence (probability) Prevalence (probability) ≈ Odds≈ Odds

probaprobabilitybility

PrevalePrevalence (%)nce (%)

DiseasDiseasee

Total Total populatiopopulatio

nn

No No diseasedisease OddsOdds

0.20.2 2020 2020 100100 8080 0.250.25

0.10.1 1010 1010 100100 9090 0.110.11

0.050.05 55 55 100100 9595 0.05260.0526

0.010.01 11 11 100100 9999 0.0100.010

0.0010.001 0.10.1 11 1,0001,000 999999 0.00100.001000

Petch Rawdaree

Probability of having Probability of having diseasedisease

Pre-test probabilityPre-test probability= prevalence of disease= prevalence of disease

≈ ≈ pre-test odds of diseasepre-test odds of disease Post-test probabilityPost-test probability=probability of disease if test is positive=probability of disease if test is positive

≈ ≈ post-test odds of disease if test is post-test odds of disease if test is positivepositive

= Pre-test odds of disease = Pre-test odds of disease ×× LR+ LR+

Petch Rawdaree

Calculation of post-test Calculation of post-test probabilityprobability

Post-test probabilityPost-test probability≈ ≈ post-test odds of disease if test is positivepost-test odds of disease if test is positive

= Pre-test odds of disease = Pre-test odds of disease ×× LR+ LR+

ExampleExample: if LR+ = 8 : if LR+ = 8 Prevalence = 20% Prevalence = 20% pre-test odds = 20/80 = pre-test odds = 20/80 =

1:4 1:4

Post-test odds = 1:4 x 8 = 8:4Post-test odds = 1:4 x 8 = 8:4

post-test probability = 8/(8+4) = 66.6 %post-test probability = 8/(8+4) = 66.6 %

Petch Rawdaree

Calculation of post-test Calculation of post-test probability of diseaseprobability of disease

1.1. Find the symptom, Find the symptom, disease and test of disease and test of interestinterest

2.2. Estimate the probability Estimate the probability of disease before testingof disease before testing

3.3. Convert the probability of Convert the probability of disease to an odds of disease to an odds of diseasedisease

4.4. Multiply the pre-test odds Multiply the pre-test odds by the likelihood ratioby the likelihood ratio

5.5. Covert the post-test odds Covert the post-test odds to a post-test probabilityto a post-test probability

1.1. Hypothyroidism: TSH Hypothyroidism: TSH levellevel

2.2. Pretest probability = Pretest probability = 20%20%

3.3. Odds of hypothyroidism Odds of hypothyroidism = 20:80 = 1:4 = 20:80 = 1:4

4.4. LR+ for TSH in LR+ for TSH in hypothyroidism = 99 hypothyroidism = 99 1:4 x 99 = 99:4 1:4 x 99 = 99:4

5.5. 99/(99+4) = 96%99/(99+4) = 96%

Petch Rawdaree

How can we improve How can we improve accuracy of our diagnosisaccuracy of our diagnosis

Seek for other evidence Seek for other evidence supporting the diagnosissupporting the diagnosis Increase pre-test probabilityIncrease pre-test probability

Other diagnostic tests which Other diagnostic tests which have more sensitivity/specificity have more sensitivity/specificity and higher predictive value of and higher predictive value of positive testpositive test

Gold standard diagnostic testGold standard diagnostic test

Petch Rawdaree

conclusionsconclusions Clinical diagnosis is fundamental is endocrine Clinical diagnosis is fundamental is endocrine

disordersdisorders Selection of appropriate tests guided by tests Selection of appropriate tests guided by tests

propertiesproperties Diagnosis is integration of clinical experiences Diagnosis is integration of clinical experiences

and tests interpretationsand tests interpretations

Suggest reading: Suggest reading:

Endocrinology and Metabolism Clinics of Endocrinology and Metabolism Clinics of North North America: vol 26 No 1 March 1999America: vol 26 No 1 March 1999