Lecture 3 2015

Deakin Health Economics1

HSH719 Economic Evaluation 1

Topic 3:CONTEXT: Methods and Stages:

The Key Stages of an Economic Evaluation


Economic Evaluation: Context, Method and Stages

Learning objectives:1. Know the logical stages of an economic evaluation2. Be familiar and know how to use quality assessment criteria for economic evaluations3. Understand the different ways in which the results of economic evaluations can be presented. Know how to calculate a simple ICER


Fig 1: The Steps in Undertaking an Economic Appraisal

Step One: Deciding upon the study question

Step Two: Clear statement of alternatives to be appraised

Step Three: Assessment of costs and benefits of both alternatives

a) Identification of the appropriate costs and benefits to include in the

appraisal

b) Measurement of resources used and saved by the program alternatives,

and the outcomes produced by each

c) valuing resources used (and saved) and valuing outcomes

Step Four: Adjusting for timing

Discounting for the time stream of costs and outcomes

Step Five: Adjusting for risk and uncertaintyModeling and sensitivity analysis

Step Six: Making a decisionCalculating and using decision rules

a) Net present value of programme

b) Comparing cost-effectiveness ratios


Figure 2: Basic Types of Economic Evaluation

COSTS CONSEQUENCES

HEALTH CARE

PROGRAM

Patient and family

(C2 )

Other sectors (C3 )

Healthcare sector

(C1 )

Identification Measurement Valuation

Health state

changed

Other value

created

Resources saved

Effects (E)

Health state preferences

(U)

Healthcare sector (S1 )

Patient and family

(S2 )

Other sectors (S3 )

Resources saved

Other value created

(V)

Willingness-to-pay

(W)

or

Global

willingness

-to-pay

(W)

Resources

consumed

Deakin Health Economics5 5

Update of Figure 2

COSTS

CONSEQUENCES

HEALTH CARE

PROGRAM

Patient & family (C3)

Other sectors (C2)

Healthcare sector

(C1 )

Identification Measurement Valuation

Health state changed

Other value created

Resources

saved

Effects (E)

Health state preferences (U)

Health sector (S1 )

Patient & family (S3)

Other sectors (S2)

Other value created (V)

Willingness-to-pay (W)

or

Global

willingness-

to-pay (W)

Resources

consumed

Drummond textbook p19 Figure 2.1: Components of economic evaluation in health care

Productivity

losses (C4)

Productivity gains

(S4)


Key Questions

Is the methodology employed in the study appropriate and are the results valid?

If the results are valid, would they apply to my setting?

Reference: Drummond et al (2005) CHEERS checklist


Study Frame

Gold et al 1995 (Washington Panel) outline two key aspects on an economic appraisal The Study Frame and the Study Design

The Study Frame covers the vital conceptualization of the evaluation purpose and methods this is really about a GOOD study question Objectives; decision context; intended audience

Perspective and choice of evaluation techniques (CEA; CUA; CBA; CCA; PBMA: etc) and ICERs

Choice of intervention(s) and comparators

Target population

Study boundaries

Time horizon (for intervention and tracking costs/outcomes)


Study Design

The Study Design specifies the key elements of data collection and analysis

There are four basic decisions Appropriateness of the analyses used to describe the intervention and

its effects on health outcomes (modelled versus a within trial design)

Appropriateness of data collections for activities, costs and outcomes for the intervention and comparators

Appropriateness of the analysis used to combine the information (modelling assumptions; uncertainty analysis; sensitivity analysis; discounting; shadow pricing)

Appropriateness of the interpretation/ conclusions, given the above


How can we decide if the study is any good?

Criteria checklists

There are a few in the literature

Best known are: Drummond 10 point Checklist (Chapter 3

textbook)

CHEERs checklist (in your readings)

Fundamentally cover similar ground CHEERS includes more explicit criteria around

modelling studies


Drummonds Checklist

Did study examine both costs and effects of the service(s) or programme(s)?

Did the study involve a comparison of alternatives?

Was a viewpoint for the analysis stated and was the study placed in any particular decision-making content?

1. Was a well-defined question posed in answerable

form?


Drummonds Checklist

Were any important alternatives omitted?

Was (Should) a do-nothing alternative (be) considered?

2. Was a comprehensive description of the competing

alternatives given?

(ie can you tell who did what to whom, where and how

often?)


Drummonds Checklist

Was this done through a RCT? If so, did the trial protocol reflect what would happen in regular practice?

Was effectiveness established through an overview of clinical studies?

Were observational data or assumptions used to established effectiveness? If so, what are the potential biases in results?

3. Was the effectiveness of the programmes or services

established?


Drummonds Checklist

Was the range wide enough for the research question at hand?

Did it cover all relevant viewpoints?

Were capital costs, as well as operating costs, included?

4. Were all the important and relevant costs and

consequences for each alternative identified?


Drummonds Checklist

Were any of the identified items omitted from measurement? If so, does this mean that they carried no weight in the subsequent analysis?

Were there any special circumstances (eg joint use of resources) that made measurement difficult? Were these circumstances handled appropriately?

5. Were costs and consequences measured accurately in appropriate physical units?(eg hours of nursing time, number of physician visits, lost work-days, gained life-years?)


Drummonds Checklist

Were the sources of all values clearly identified? (Possible sources include market values, patient or client preferences and views, policy-makers views and health professionals judgements).

Were market values employed for changes involving resources gained or depleted?

Where market values were absent (eg volunteer labour) what valuation sources were used?

Was the valuation of consequences appropriate for the question posed. CEA vs CUA vs CBA

6. Were costs and consequences valued credibly?


Drummonds Checklist

Were costs and consequences which occur in the future discounted to their present values?

Was any justification given for the discount rate used?

7. Were costs and consequences adjusted for differential

timing?


Drummonds Checklist

Were the additional (incremental) costs generated by one alternative over another compared to the additional effects, benefits or utilities generated?

More on this latter!!

8. Was an incremental analysis of costs and

consequences of alternatives performed?


Drummonds Checklist

If data on costs or consequences were stochastic, were appropriate statistical analyses performed?

If a sensitivity analysis was employed, was justification provided for the ranges of values (for key study parameters)?

Were study results sensitive to changes in the values (within the assumed range for sensitivity analysis, or within the confidence interval around the ratio of costs to consequences)?

9. Was allowance made for uncertainty in the estimates

of costs and consequences?


Drummonds Checklist

Were the conclusions of the analysis based on some overall index or ratio of costs to consequences (eg cost-effectiveness ratio)? If so, was the index interpreted intelligently or in a mechanistic fashion?

Were the results compared with those of others who have investigated the same question? If so, were allowances made for potential differences in study methodology?

Did the study discuss the generalisability of the results to other settings and patient/client groups?

10. Did the presentation and discussion of study results

include all issues of concern to users?


Drummonds Checklist

Did the study allude to, or take account of, other important factors in the choice or decision under consideration (eg distribution of costs and consequences, or relevant ethical issues)?

Did the study discuss issues of implementation, such as the feasibility of adopting the preferred programmegiven existing financial or other constraints, and whether any freed resources could be redeployed to other worthwhile programmes?

10. Did the presentation and discussion of study results

include all issues of concern to users? cont/d..


Economic Evaluation: Traps for Beginners

1. No comparatorThe essential question economic appraisal asks is what difference does it make?

2. Study perspective not specified1 and 2 lead to poorly specified research question, and problems with identification/measurement of costs and benefits.

3. Inadequate description of programme and comparatorWho does what to who, when and where?Leads to inadequate measurement of costs and benefits, and poorly identified data sources.


4. Failure to specify inclusion/exclusion criteria for costs and benefits

Leads to internal and external validity issues. Financial costing rather than economic approach. Failure to consider if outcome measure captures all relevant benefits.

5. Failure to undertake marginal analysis

Reliance on average C/E results can hide important information.

6. Failure to undertaken sensitivity analysis

No provision for uncertainty.



7. Failure to incorporate discounting

No provision for when costs and benefits are experienced.

8. Failure to consider evaluability

Is the project ready to be evaluated from output/outcome perspective?

9. Insufficient thought given to time period of the study

10. Double counting of benefits

(Life years saved plus forgone productivity).



Presentation of Results

Economic Evaluation has become a lot more sophisticated over the last 15 years

When I first started:

ICER =12

12

ICER = Comparative, costs, benefits

Now also have: Cost-Effectiveness Planes (CEP)

Cost-Effectiveness Acceptability Curves (CEAC)

Net-Monetary Benefits (NMB)


Example a simple ICER

Societal Cost of Drug 1 = $10,000

Societal Cost of Drug 2 = $5,000

Benefit of Drug 1 = 10 life years

Benefit of Drug 2 = 8 life years

ICER = 10,0005,000

108

ICER = 5000

2

ICER = $2,500/LY saved

Is this good value for money???


MORE SAME LESS

MORE ? X

SAME X

LESS X ?

Incremental

cost of

Program B

compared

with

Program A

X = study reduces from CEA to a cost-minimisation analysis.

Incremental benefit of Program B vis-

-vis Program A

Dominant

Dominant

Dominated

Dominated

Decision rules


Problems with League Tables

Comparability of methods, for example:

Perspective

Time Horizon

Settings (USA vs Aust)

Outcomes

Uncertainty in results

Rarely just one single point estimate!


Example

Mihalopoulos et al, 2011, ANZJP, 45, 36-44 This study evaluated 2 interventions designed to

prevent depression in adults Brief bibliotherapy Group based psychological therapy

Comparator was treatment as usual for both Perspective was health sector Time horizon was 5 years Modelled economic evaluation Part of a larger project called ACE-Prevention

which had a detailed protocol of methods


Mihalopoulos et al, 2011 (ANZJP, 45, 36-44)

Median point estimate (95% uncertainty interval)

Brief bibliotherapy

intervention

Group-based psychological

treatment

DALYs averted 2,600 (-1,500 6,700) 1,700 (518 3,000)

Cost of the intervention

Govt: $760,000 ($520,000 -

$1,000,000)

Private patient :$21M ($13M -

$31M)

Private other: $15M ($10M-

20M)

Govt: $20M ($13M - $30M)

Private patient :$3M ($2M -

$5M)

Private other: $14M ($9M-

20M)

Total costs $37M ($24M - $52M) $38M ($26M - $53M)

Cost offsets $6M ($-5M - $15M) $4M ($1M - $8M)

ICER (with cost-offsets)1 $8,600$20,000 ($12,000 -

$66,000)

ICER (without cost-offsets) 1 $11,000$23,000 ($14,000 - $

68,000)

Median point estimate (95% uncertainty interval)

Brief bibliotherapy intervention Group-based psychological treatment

DALYs averted 2,600 (-1,500 6,700) 1,700 (518 3,000)

Cost of the intervention

Govt: $760,000 ($520,000 -$1,000,000)

Private patient :$21M ($13M -$31M)

Private other: $15M ($10M-20M)

Govt: $20M ($13M - $30M)

Private patient :$3M ($2M - $5M)

Private other: $14M ($9M-20M)

Total costs $37M ($24M - $52M) $38M ($26M - $53M)

Cost offsets $6M ($-5M - $15M) $4M ($1M - $8M)

ICER (with cost-offsets)1 $8,600 $20,000 ($12,000 - $66,000)

ICER (without cost-offsets) 1 $11,000 $23,000 ($14,000 - $ 68,000)

Results from paper


Box 3.2: The Cost-effectiveness

PlaneIn the diagram the horizontal axis represents the difference in effect between the intervention of interest (A)

and the relevant alternative (O), and the vertical axis represents the difference in cost. The alternative (O)

could be the status quo or a competing program.

If point A is in quadrants II or IV the choice between the programs is clear. In quadrant II the intervention of

interest is both more effective and less costly than the alternative. That is, it dominates the alternative. In

quadrant IV the opposite is true. In quadrants I and III the choice depends on the maximum cost-

effectiveness ratio one is willing to accept. The slope of the line OA gives the cost-effectiveness ratio.

-

+

+O

-

A

Intervention less effective and

more costly than O

Intervention more effective and

more costly than O

Intervention more effective and

less costly than O

Intervention less effective and less

costly than O

EFFECT DIFFERENCE

IV I

IIIII

COST DIFFERENCE


Previous example as a CE Plane

Mihalopoulos et al, 2011

http://basecase.com/articles/cost-effectiveness-plane-explanation/


Thresholds

Threshold values to denote value for money E.g. $50,000/QALY, 30,000/QALY in UK These can change in different jurisdictions

PBAC does not have a stated threshold NICE does McKie et al (2011) found that in Australia societal values changed

according to constructs such as hope. (Health Economics, (20), 945-957)

Are largely subjective value judgements What is something worth

WHO Commission on Macroeconomics developed a rule of thumb 1 DALY for less than the average per capita GDP for a given country is

very cost-effective Even up to 3 times per GDP is still cost-effective Aust per capita GDP is: $37,000


Acceptability Curve

http://bjp.rcpsych.org/content/187/2/106

Excellent description of CEAC written for non-economists

Shows the probability that an intervention is cost-effective over a range of cost-effectiveness value for money thresholds

Are constructed using the same data used in the CEP


Acceptability Curves


Net Monetary Benefits

This is a composite measure (part CEA and part CBA)

Uses a threshold value for money E.G. $50,000/QALY

W>

E.g. ig C = $5000 & B = 2QALYs

= 50,000*2>5,000 Therefore intervention is deemed cost-effective Studies will often adopt different thresholds and

calculate probability that intervention is CE


Further Reading

Drummond et al (2005), Chapter 3

Weblinks in the lecture

CHEERS Checklist in Unit Readings

Documents

Lecture 3 2015