The Cost-effectiveness The Cost-effectiveness AnalysisAnalysis

Introduction/MotivationIntroduction/Motivation Two measures in a cost-effectiveness Two measures in a cost-effectiveness

analysisanalysis

ScenarioScenario You are the pharmacy director for a 1.5 million You are the pharmacy director for a 1.5 million

member health insurance organization. The detail member health insurance organization. The detail person from Merp Inc. comes calling with news that person from Merp Inc. comes calling with news that their new pain relief drug “Arafex” has just been their new pain relief drug “Arafex” has just been approved for persons with osteoarthritis. It’s reported approved for persons with osteoarthritis. It’s reported benefits are lower incidence of GI bleeding and (an benefits are lower incidence of GI bleeding and (an unexpected outcome of the clinical trial) an apparent unexpected outcome of the clinical trial) an apparent modest improvement in LDL cholesterol. Given these modest improvement in LDL cholesterol. Given these purported benefits, the detail person states that at purported benefits, the detail person states that at the price, Arafex “blows away” the other Cox II the price, Arafex “blows away” the other Cox II inhibitor products, such as celecoxib and rofecoxibinhibitor products, such as celecoxib and rofecoxib

The cost of Arafex is $95 per 30 day supplyThe cost of Arafex is $95 per 30 day supply The cost of celecoxib is $82The cost of celecoxib is $82 The cost of rofecoxib is $74The cost of rofecoxib is $74 Question: Should you add Arafex to the formularyQuestion: Should you add Arafex to the formulary

(Taken from S. Ramsey)

IntroductionIntroduction

Scarcity of resourcesScarcity of resources Choices need to be madeChoices need to be made Decision need to be made based Decision need to be made based

costs and effectscosts and effects Cost-effectiveness analysis (CEA) is a Cost-effectiveness analysis (CEA) is a

way to mingle way to mingle costcost and and effectivenesseffectiveness of a study of a study

Examples Of Effectiveness Examples Of Effectiveness Measures Used In Cost-Measures Used In Cost-effectiveness Analyseseffectiveness Analyses

Study Reference

Clinical Field

Effectiveness measure

Logan et al. (1981) Treatment of hypertension MmHg blood pressure reduction

Schulman et al. (1990) Treatment of hypercholesterolaemia

% serum cholesterol reduction

Hull et al. (1981) Diagnosis of deep vein thrombosis

Cases of DVT detected

Sculpher and Buxton (1993)

Asthma Episode-free days

Mark et al (1995) Thrombosis Years of life gained

Two Commonly Used Two Commonly Used StatisticsStatistics

Incremental cost-effectiveness ratio Incremental cost-effectiveness ratio (ICER):(ICER):

Net health benefit (NHB):Net health benefit (NHB):

Where Where λλ=Willingness to pay=Willingness to pay

01

01

ee

ccICER

0101

1cceeNHB

0 treatmentofeffect avg.

1 treatmentofeffect avg.

0 treatmentofcost avg.

1 treatmentofcost avg.

0

1

0

1

c

e

c

c

Graphical RepresentationGraphical Representation

01 cc

01 ee

The ΔE- ΔC plane

Graphical RepresentationGraphical Representation

001 cc

01 cc

001 ee

The ΔE- ΔC plane

01 ee

Slope = ICER

Graphical RepresentationGraphical Representation

001 cc

01 cc

001 ee

The ΔE- ΔC plane

01 ee

Graphical RepresentationGraphical Representation

001 cc

01 cc

001 ee

The ΔE- ΔC plane

01 ee

001 cc

001 ee

Graphical RepresentationGraphical Representation0

01 cc

01 cc 0

01 ee

The ΔE- ΔC plane

01 ee

Slope=λ=30,000

ICER=50,000

EstimationEstimation Incremental cost-effectiveness ratio Incremental cost-effectiveness ratio

(ICER):(ICER):

Net health benefit (NHB):Net health benefit (NHB):

01

01ˆEE

CCREIC

01011ˆ CCEEBHN

PropertiesPropertiesICER NHBICER NHB

Natural interpretation Natural interpretation (price/unit of product)(price/unit of product)

Analysis independent Analysis independent of of λλ

Economic foundationEconomic foundation

Negative ICER is Negative ICER is problematicproblematic

Undefined value Undefined value possible for CIspossible for CIs

Biased, not sufficientBiased, not sufficient

NHB properly orderedNHB properly ordered Interpretation is not Interpretation is not

ambiguous like ambiguous like negative ICERnegative ICER

UnbiasedUnbiased Easy extension to Easy extension to

more than two more than two comparatorscomparators

Dependent on λDependent on λ No natural No natural

interpretationinterpretation

Inference: Existing Inference: Existing TechniquesTechniques ICER NHB ICER NHB Normal theory via Normal theory via

CLTCLT BootstrapBootstrap Confidence box Confidence box Fieller’s method Fieller’s method

(1954)(1954) Hinkley’s method Hinkley’s method

(1969)(1969)

Normal theory via Normal theory via CLTCLT

BootstrapBootstrap

Inference: Existing TechniquesInference: Existing TechniquesConfidence BoxConfidence Box

01 cc

The ΔE- ΔC plane

01 ee

Inference: Existing Inference: Existing TechniquesTechniques

Fieller’s method (1954):Fieller’s method (1954):

The C.I. can be obtained by equating the The C.I. can be obtained by equating the formula and solve for R:formula and solve for R:

)1,0(~)r(av

NERC

ERC

ICERR

2/1)r(av

z

ERC

ERC

Inference: Existing Inference: Existing TechniquesTechniques

Hinkley’s method (1969):Hinkley’s method (1969): Distribution of ratio of two correlated random Distribution of ratio of two correlated random

normal variablesnormal variables Can be applied to the ICER if we assume the Can be applied to the ICER if we assume the

numerator and the denominator of the ICER numerator and the denominator of the ICER are bivariate normalare bivariate normal

Has not been applied to inference for the Has not been applied to inference for the ICERICER

Previous StudiesPrevious Studies

Briggs, et al. (1999)Briggs, et al. (1999) CLT-based (Taylor’s), Fieller’s, CLT-based (Taylor’s), Fieller’s,

confidence box, bootstrap (normal confidence box, bootstrap (normal approximation, percentile, BC, BCa, approximation, percentile, BC, BCa, parametric bootstrap)parametric bootstrap)

Conclusion: Fieller’s appears best; Conclusion: Fieller’s appears best; parametric bootstrap and BCa are best parametric bootstrap and BCa are best among the bootstrap methodsamong the bootstrap methods

Previous StudiesPrevious Studies

Fan & Zhou (Outcome Research Fan & Zhou (Outcome Research Methodology, 2006)Methodology, 2006) CLT-based (Taylor’s), Fieller’s, confidence CLT-based (Taylor’s), Fieller’s, confidence

box, bootstrap (normal approximation, box, bootstrap (normal approximation, percentile, bootstrap-t, BCa, parametric percentile, bootstrap-t, BCa, parametric bootstrap)bootstrap)

Conclusion: nonparametric bootstrap-t is best Conclusion: nonparametric bootstrap-t is best in term of coverage accuracy; next are the in term of coverage accuracy; next are the Fieller’s and among the bootstrap methods, Fieller’s and among the bootstrap methods, parametric bootstrap and BCa are bestparametric bootstrap and BCa are best

Limitation of Existing Limitation of Existing TechniquesTechniques

CLT-based: may not be appropriate for small CLT-based: may not be appropriate for small sample skewed-data (Briggs, 1999; Fan & sample skewed-data (Briggs, 1999; Fan & Zhou, 2006)Zhou, 2006)

Bootstrap: may be time-consuming, may not Bootstrap: may be time-consuming, may not provide proper coverage (Fan & Zhou, 2005)provide proper coverage (Fan & Zhou, 2005)

Confidence box: does not provide proper Confidence box: does not provide proper coverage (Briggs, et al, 1999)coverage (Briggs, et al, 1999)

Fieller’s and Hinkley’s: based on bivariate Fieller’s and Hinkley’s: based on bivariate normal, Fieller’s may not be closed intervalnormal, Fieller’s may not be closed interval

Inference: Alternative Inference: Alternative ApproachApproach

Purpose:Purpose: To explain what play a role in the To explain what play a role in the

normal approximationnormal approximation To help improve inferenceTo help improve inference

Edgeworth Expansion: NHBEdgeworth Expansion: NHB

NHB: LetNHB: Let

where,where,

22 ˆ

ˆ

NHBBHN

T

12

2/12 )()()()( NOxxqNxxTP

3

2222

2

3

1111

1

2

21

12

221

2/32

2

,

),var(

,)1(

1),12(

6)(

EC

EPEC

EP

TAsympQ

nn

nPP

QAx

Axq

EC

EC

Edgeworth Expansion: ICEREdgeworth Expansion: ICER ICER: LetICER: Let

wherewhere

AA11 and A and A22 depends on the asymptotic depends on the asymptotic variance of Tvariance of T11 and skewness of costs and and skewness of costs and effectseffects

11

2/11 )()()()( NOxxqNxxTP

11 ˆ

ˆ

ICERREIC

T

)1(

6

64)( 221

211 xAA

AAxq

Edgeworth Expansion for ICER Edgeworth Expansion for ICER and NHBand NHB

ICER:ICER:

NHB:NHB:

N

OxxqN

xxTP1

)()(1

)()( 11

N

OxxqN

xxTP1

)()(1

)()( 22

Alternative Confidence Alternative Confidence IntervalsIntervals

Similar to the two-sample problem, we can Similar to the two-sample problem, we can introduce three new transformational introduce three new transformational intervalsintervals

3. 2, 1,ifor

,ˆ)(ˆ

ˆ)(ˆ

2/2/112/1

02/12/112/1

zNhN

zNhN

i

i

SimulationSimulation Bivariate normalBivariate normal

Bivariate lognormalBivariate lognormal

Bivariate mixtureBivariate mixture

Simulation: ICERSimulation: ICERCoverage of 95% Confidence Coverage of 95% Confidence

intervalsintervalsNormal Fieller Hinkley Boot-t New1 New2 New3

.8556(247.53)

.9037(266.87)

.9037(265.69)

.9030(385.72)

.8961(363.12)

.8609(244.59)

.8944(245.40)

.8583(111.36)

.8636(116.91)

.8636(116.90)

.8956(170.36)

.8731(136.81)

.8637(110.53)

.8937(110.10)

.8520(242.38)

.8980(256.34)

.8979(256.30)

.9059(394.79)

.8957(355.36)

.8557(239.85)

.8829(238.10)

.8875(183.21)

.9171(190.09)

.9172(190.27)

.9171(243.31)

.9084(244.94)

.8924(181.59)

.9175(181.67)

Simulation: NHBSimulation: NHB Coverage of 95% Confidence intervalsCoverage of 95% Confidence intervals

Normal Boot-t BCa New1 New2 New3

0.9025 (60.88)

0.9262 (76.77)

0.9052(64.40)

0.8767 (85.87)

0.8971 (61.06)

0.9355(64.61)

0.8965(61.05)

0.9229 (77.24)

0.9027(64.57)

0.8704 (86.23)

0.8913 (61.23)

0.9304 (64.79)

0.8930(60.21)

0.9272(77.03)

0.9021(63.85)

0.8621(86.55)

0.8864(60.35)

0.9204(62.35)

0.9192 (45.55)

0.9269 (52.50)

0.9142(47.51)

0.9008 (57.61)

0.9159 (45.63)

0.9388 (47.29)

SimulationSimulation The modelThe model

Specify Specify λλ, , ββ, , γγ Generate X ~ uniformGenerate X ~ uniform εε ~ normal(0, 1) ~ normal(0, 1) gg22(Z(Zii, , γγ) = exp(Z) = exp(Zii

TTγγ)) Objective: estimate E(Y|xObjective: estimate E(Y|x00))

iiTii ZgXyh );();(

SimulationSimulationnn λλ CoveragCoverag

ee

95% CI.95% CI.

5050 11 1616 15.9315.93 0.9480.948

5050 .5.5 72.9372.93 72.0472.04 0.9270.927

200200 .5.5 72.9372.93 72.5572.55 0.8940.894

20002000 .5.5 72.9372.93 72.7972.79 0.9480.948

5050 .2.2 1094.1094.33

1011.1011.99

0.8440.844

200200 .2.2 1094.1094.33

1040.1040.88

0.8310.831

20002000 .2.2 1094.1094.33

1081.1081.77

0.9350.935

)|( 0xYE )|(ˆ0xYE

Application to a real Application to a real exampleexample

IN 2002, Katon et al. (2002) conducted a randomized IN 2002, Katon et al. (2002) conducted a randomized trial to evaluate the cost effectiveness of a trial to evaluate the cost effectiveness of a collaborative care intervention, compared to the collaborative care intervention, compared to the usual primary care setting, in patients with panic usual primary care setting, in patients with panic disorder. disorder.

Panic disorder occurs in 4–6% of patients in primary Panic disorder occurs in 4–6% of patients in primary care. This severe anxiety disorder is associated with care. This severe anxiety disorder is associated with high use of medical services, high costs, and a high use of medical services, high costs, and a variety of unexplained medical symptoms. variety of unexplained medical symptoms.

Patients with panic disorder often do not receive an Patients with panic disorder often do not receive an accurate diagnosis in primary care and even when accurate diagnosis in primary care and even when diagnosis is assigned, few patients receive diagnosis is assigned, few patients receive appropriate treatment or psychotherapy (Katon et appropriate treatment or psychotherapy (Katon et al., 2002). al., 2002).

Application to real example, Application to real example, contcont

One objective of the study is to determine One objective of the study is to determine the incremental cost effectiveness of a the incremental cost effectiveness of a collaborative care program for primary care collaborative care program for primary care patients with panic disorder compared with patients with panic disorder compared with the usual primary care setting.the usual primary care setting.

To demonstrate our methods, we consider To demonstrate our methods, we consider total outpatient nonmental health costs, and total outpatient nonmental health costs, and for measure of effectiveness, we consider for measure of effectiveness, we consider the number of days a patient experiences the number of days a patient experiences panic disorder during the 1-year study panic disorder during the 1-year study period. period.

The summary statisticsThe summary statistics

  Estimated ICER = 15.33, NHB = 77.08Estimated ICER = 15.33, NHB = 77.08 Control group (Control group (nn1 = 54) Intervention group (1 = 54) Intervention group (nn2 = 53) 2 = 53)

Mean SD Skewness Mean SD Mean SD Skewness Mean SD

SkewnessSkewness Cost(U.S.$) 2507.42 4460.44 4.93 1325.48 1785.67 3.46Cost(U.S.$) 2507.42 4460.44 4.93 1325.48 1785.67 3.46 Effectiveness (days with anxiety attack) Effectiveness (days with anxiety attack) 211.52 139.68 −0.30 134.42 134.55 0.71211.52 139.68 −0.30 134.42 134.55 0.71

Application, contApplication, cont

Costs in both groups are highly skewed Costs in both groups are highly skewed with the coefficient of skewness 4.93 for with the coefficient of skewness 4.93 for the control group and 3.46 for the the control group and 3.46 for the intervention group. intervention group.

On average, the control group incurred On average, the control group incurred $1181.95 and 77.1 days of panic attack $1181.95 and 77.1 days of panic attack more than the intervention group. more than the intervention group.

The estimated ICER is 15.33, indicating The estimated ICER is 15.33, indicating that the intervention arm is dominant. that the intervention arm is dominant.

confidence intervals for the ICER and the confidence intervals for the ICER and the NHBNHB

Methods Confidence intervals Interval lengthMethods Confidence intervals Interval length ICER  Taylor's (−3.44, 34.10) 37.54  ICER  Taylor's (−3.44, 34.10) 37.54   Fieller's (−1.43, 53.86) 55.28 Fieller's (−1.43, 53.86) 55.28  Hinkley's (−1.68, 52.44) 54.12  Hinkley's (−1.68, 52.44) 54.12   Bootstrap-Bootstrap-tt   (4.50, 49.55)  45.05     (4.50, 49.55)  45.05   GG1   (3.33, 81.87)  78.54  1   (3.33, 81.87)  78.54   GG2   (−1.99, 35.22)   37.21  2   (−1.99, 35.22)   37.21   GG3   (0.35, 38.38)  38.033   (0.35, 38.38)  38.03 NHB  Taylor's  (25.12, 129.03) 103.92   NHB  Taylor's  (25.12, 129.03) 103.92    Bootstrap-Bootstrap-tt  (21.68, 132.33) 110.65    (21.68, 132.33) 110.65     BCa  (20.21, 125.52) 105.31   BCa  (20.21, 125.52) 105.31    GG1  (26.78, 130.87) 104.10   1  (26.78, 130.87) 104.10    GG2  (25.46, 129.39) 103.92   2  (25.46, 129.39) 103.92    GG3  (32.84, 144.42) 111.58 3  (32.84, 144.42) 111.58

Results, contResults, cont

Both bootstrap-Both bootstrap-tt and and GG3 intervals are positive, 3 intervals are positive, showing that the intervention is significantly showing that the intervention is significantly dominant (the control group incurred more dominant (the control group incurred more cost and more days of panic disorder). cost and more days of panic disorder).

As anticipated, both Fieller's and Hinkley's As anticipated, both Fieller's and Hinkley's intervals are similar and the intervals are similar and the GG3 interval has 3 interval has shorter length than bootstrap-shorter length than bootstrap-tt. .

Because the estimated qBecause the estimated q11/sqrt(N)=0.9 , /sqrt(N)=0.9 , Taylor's interval is inadequate. Based on our Taylor's interval is inadequate. Based on our simulation results, we would recommend simulation results, we would recommend using the using the GG3 interval as the confidence 3 interval as the confidence interval for the ICER. interval for the ICER.

A common willingness-to-pay is λ = $50,000; A common willingness-to-pay is λ = $50,000; using this value, the estimated NHB is 77.08. using this value, the estimated NHB is 77.08.

Confidence intervals for the NHB are Confidence intervals for the NHB are presented in Table 6. All intervals are presented in Table 6. All intervals are relatively similar, especially relatively similar, especially GG1, 1, GG2, and 2, and Taylor's intervals. The coefficient qTaylor's intervals. The coefficient q22/sqrt(N) is /sqrt(N) is 0.05 in this setting, suggesting that Taylor's 0.05 in this setting, suggesting that Taylor's interval is adequate. interval is adequate.

All of these confidence intervals are strictly All of these confidence intervals are strictly positive indicating, again, that the positive indicating, again, that the intervention arm is cost effective.intervention arm is cost effective.

Conclusions on ICERConclusions on ICER

For the ICER, when data were generated from a For the ICER, when data were generated from a skewed distribution, our new intervals gave skewed distribution, our new intervals gave better coverages than Taylor's interval. better coverages than Taylor's interval.

They were comparable and sometimes better They were comparable and sometimes better than Fieller's and Hinkley's intervals. We found than Fieller's and Hinkley's intervals. We found that Hinkley's method, which has not been that Hinkley's method, which has not been adopted for the ICER, was similar to Fieller's adopted for the ICER, was similar to Fieller's method in terms of coverage accuracy and method in terms of coverage accuracy and interval length. interval length.

Intervals based on Intervals based on GG3 transformation were 3 transformation were comparable to the boott in terms of coverage but comparable to the boott in terms of coverage but were about one third narrower.were about one third narrower.

Conclusions on NHBConclusions on NHB

For the NHB, we saw that intervals based on the For the NHB, we saw that intervals based on the GG3 3 transformation gave good coverages in all cases transformation gave good coverages in all cases considered and were comparable to the boott. considered and were comparable to the boott. However, our intervals were narrower than the boott However, our intervals were narrower than the boott and required less computing in terms of bootstrap and required less computing in terms of bootstrap resampling.resampling.

When dealing with highly skewed data, our intervals When dealing with highly skewed data, our intervals based on the based on the GG3 transformation should be 3 transformation should be recommended.recommended.

The remaining question is what one should choose The remaining question is what one should choose between the ICER and the NHB. between the ICER and the NHB.

Each measure has its own advantages as well as Each measure has its own advantages as well as disadvantages. We leave it to the readers to decide disadvantages. We leave it to the readers to decide which measure is more appropriate for their purposes.which measure is more appropriate for their purposes.