Upload
andres-delgado-montezuma
View
215
Download
0
Embed Size (px)
Citation preview
8/13/2019 Health Economics an Introduction Kobelt 2013
1/108
HealthEconomics:
An Introductionto EconomicEvaluation
Third Edition
Gisela Kobelt
8/13/2019 Health Economics an Introduction Kobelt 2013
2/108
Health Economics: An Introduction to
Economic Evaluation
Third Edition
Gisela Kobelt
2013
8/13/2019 Health Economics an Introduction Kobelt 2013
3/108
Ofce of Healh Ecoomic
Southside, 7th Floor
105 Victoria Street
London SW1E 6QT
United Kingdom
www.ohe.org
2013
All rights reserved
Printed in the United Kingdom
ISBN 978-1-899040-44-5
This monograph has undergone a rigorous peer review by the independent OHE Editorial Board and
other eperts i the eld. The vies epressed i this publicatio are those of the author ad do otecessaril represet those of OHE.
Abou he Ofce of Healh Ecoomic
Fouded i 1962, the OHEs terms of referece are to:
Commissio ad udertake research o the ecoomics of health ad health care
Collect ad aalse health ad health care data for the UK ad other coutries
Dissemiate the results of this ork ad stimulate discussio of them ad their polic implicatios.
The OHEs ork is supported b research grats ad cosultac reveues from a ide rage of UK
and international sources.
8/13/2019 Health Economics an Introduction Kobelt 2013
4/108
ii
Abou he Auho
Giela Kobelis Maagig Director of Europea Health Ecoomics Frace, a compa specialisig
i ecoomic evaluatio of health itervetios ad traiig courses i health ecoomics ad
economic evaluation.
Util the ed of 1997, Gisela as Vice Presidet, Corporate Health Ecoomics, at the Pharmacia &
Upjoh Corporate Maagemet Ceter i Stockholm ad Lodo. Prior to her resposibilities ith
Pharmacia, she created and headed the health economics department at Sandoz in Basel. Since 2007,
has bee a visitig professor at the Departmet of Orthopedics/Rheumatolog at the Uiversit of
Lund, Sweden.
Gisela holds a masters degree from the Uiversit of Strasbourg, a MBA from the Istitute for
Maagemet Developmet i Lausae, ad a PhD i health ecoomics from the Karoliska Istitute.
Author of over 90 scietic publicatios i the eld of health ecoomics, several book chapters ad
numerous reports, Gisela serves on several editorial boards.
Ackolegeme
I am grateful to Professor Begt Jsso of the Stockholm School of Ecoomics ad to the
OHE revieers for their advice ad helpful commets o the mauscript.
Gisela Kobelt
8/13/2019 Health Economics an Introduction Kobelt 2013
5/108
FOREwORD
iii
FOrEwOrd
The Ofce of Health Ecoomics is pleased to offer the third editio of this valuable guide to
health economics.
The secod editio of this publicatio appeared i 2002. I the decade sice, the use of ecoomic
evaluatio of e medical techologies as a basis for decisios about access to ad reimbursemet
of medicies ad medical services has epaded to a icreasig umber of coutries ad tpes of
technology. At the same time, the methods themselves have evolved in response to experience and to
chages i the abilit to capture ad aalse data. This e editio reects those chages.
This book presets a comprehesive overvie of approaches to health ecoomic evaluatio, illustrated
throughout with examples and with guidance about what methods are appropriate in which situations.
writte i a accessible stle, the book offers importat backgroud both for those ho ill udertake
evaluatios ad those ho ill use them as the bases for decisios. The author, Gisela Kobelt, has
etesive eperiece i ecoomic evaluatio, makig her perspective particularl isightful.
Professor Adria Tose, Director
Ofce of Health Ecoomics
8/13/2019 Health Economics an Introduction Kobelt 2013
6/108
COnTEnTS
iv
COntEnts
Foreword ...........................................................................................................iii
Illuaio ......................................................................................................vi
suy Example ...............................................................................................viii
1 Healh Ecoomic: Geeal Iue ...................................................................1
The ecoomics of health ad health care........................................................................1
Challeges i health care: the cotet.....................................................................2
The role of health ecoomic evaluatio studies i market access........................................4
The importace of ecoomic evaluatio for the developmet of e techologies.................9
2 Fom of Healh Ecoomic Evaluaio ............................................................12
Introduction ..............................................................................................................12
Tpes of ecoomic evaluatio...................................................................................... 13
Outcome measurement in economic evaluation..............................................................14
Physiological measures and clinical events ............................................................. 15
Survival.............................................................................................................16
Quality-adjusted survival ..................................................................................... 16
Monetary outcomes ............................................................................................ 18
Patient-reported outcomes ................................................................................... 18
Cost data for ecoomic evaluatio............................................................................... 19
Perspectives .............................................................................................................19
Steps in cost assessment ............................................................................................20
Cost-of-illess studies................................................................................................ 23
Prevalence-based studies .....................................................................................24
Incidence-based studies ...................................................................................... 27
Costig approaches............................................................................................. 28
3 Aalyical Appoache o Ecoomic Evaluaio ..............................................32
Decisio aalsis ad modellig techiques................................................................... 33
Decisio trees.................................................................................................... 33
Markov chai aalsis......................................................................................... 34
Estimatig cost effectiveess...................................................................................... 36
Cost-miimisatio aalsis................................................................................... 36
Cost-effectiveess aalsis.................................................................................. 38
8/13/2019 Health Economics an Introduction Kobelt 2013
7/108
COnTEnTS
v
Technical issues ......................................................................................................... 46
Time perspective or discounting ............................................................................ 46
Cost of added ears of life................................................................................... 47
Patiet groups, straticatio of risk ad sub-aalsis............................................... 48
Ucertait: sesitivit aalsis ad codece itervals......................................... 49
Meta-aalsis ad etork meta-aalsis: idirect compariso................................. 50
Cost-utilit aalsis.................................................................................................... 53
Utilities ............................................................................................................. 54
Cost-beet aalsis.................................................................................................. 75
Cotiget valuatio........................................................................................... 76
4 Guielie fo Ecoomic Evaluaio ..............................................................81
5 Cocluio ...................................................................................................83
Gloay ...........................................................................................................86
refeece .......................................................................................................90
Fuhe reaig ...............................................................................................94
8/13/2019 Health Economics an Introduction Kobelt 2013
8/108
ILLUSTRATIOnS
vi
ILLUstrAtIOns
Figure 1.1. Major cotributors to the groth of health care costs...............................................2
Figure 1.2. Eamples of measures for cotaiig spedig o prescriptio drugs i Europe...........3
Figure 1.3. Assessmet criteria for e therapies....................................................................3
Figure 1.4. Deitio ad forms of ecoomic evaluatio............................................................4
Figure 1.5. Bodies ivolved i determiig market access.........................................................6
Figure 1.6. Use of ecoomic evaluatio i various coutries.......................................................7
Figure 1.7. Documetig value for moe.............................................................................10
Figure 1.8. Ecoomic evaluatio durig ad after developmet................................................11
Figure 2.1. Structure of ecoomic evaluatio.........................................................................13
Figure 2.2. Effectiveess measures used i ecoomic aalses................................................14
Figure 2.3. Etrapolatig from itermediate to al outcome...................................................15
Figure 2.4. Etrapolatig from ithi-trial mortalit to life ears saved.....................................16
Figure 2.5. The cocept of qualit-adjusted life ears.............................................................17
Figure 2.6. Dimesios i patiet-reported outcomes.............................................................18
Figure 2.7. Established outcomes instruments .......................................................................19Figure 2.8.Tpical items of resource use i a ecoomic evaluatio..........................................21
Figure 2.9. Dimesios of costs ad prices............................................................................21
Figure 2.10. Associatio of severit of rheumatoid arthritis ad severit of disease.....................24
Figure 2.11. Associatio of cost ad severit of multiple sclerosis.............................................26
Figure 2.12. Relatioship betee utilit ad severit of multiple sclerosis................................27
Figure 2.13. Direct medical costs for metastatic breast cacer.................................................28
Figure 2.14. Differeces i the choice of treatmet ad patiet maagemet across coutries..... 30
Figure 2.15. Differeces i costs depedig o the use of estimates based o isuracetariffs or full opportuit costs...........................................................................31
Figure 3.1. Eample of a decisio tree..................................................................................33
Figure 3.2. Illustratio of a Markov chai aalsis..................................................................34
Figure 3.3. Markov state trasitio diagram..........................................................................35
Figure 3.4. Markov ccle tree..............................................................................................36
Figure 3.5. Retrospective cost aalsis of maagig de ovo trasplat patiets (1994 CHF).......38
Figure 3.6. Icremetal cost-effectiveess ratio.....................................................................39
Figure 3.7. Effectiveess of four treatmets for ochomcosis: proportio of patietsachievig cure ad eperiecig reifectio after oe course of treatmet................40
Figure 3.8. Cliical decisio tree for treatig fugal ifectio of the toeail (to ears)...............41
8/13/2019 Health Economics an Introduction Kobelt 2013
9/108
ILLUSTRATIOnS
vii
Figure 3.9. Structure of the Markov model............................................................................42
Figure 3.10. Distributio of costs b categor of resource use: Frace ad the UK
as examples. ................................................................................................... 43
Figure 3.11. Cost-effectiveess acceptabilit curve.................................................................50
Figure 3.12. Meta-aalsis of trials comparig beta-blockers to placebo.................................... 51
Figure 3.13. Estimatig the icidece of diabetes from studies of atihpertesive drugs............52
Figure 3.14. The ve dimesios of the EQ-5D......................................................................56
Figure 3.15. The visual aalogue scale i the EQ-5D: the EQ-VAS............................................ 56
Figure 3.16. Standard gamble .............................................................................................57
Figure 3.17. Time trade-off................................................................................................. 58
Figure 3.18. Visual aalogue scale (VAS)..............................................................................58
Figure 3.19. Model structure for hormoe replacemet therap................................................59
Figure 3.20. Structure of the RA models...............................................................................63
Figure 3.21. Average developmet of HAQ scores i the model, adjusted for differet
effectiveess after a oe-ear trial.....................................................................66
Figure 3.22. Ucertait i ICERs........................................................................................68
Figure 3.23. Probabilistic sensitivity analysis .........................................................................70
Figure 3.24. Structure of the Markov model..........................................................................72
Figure 3.25. Etrapolatio of disease progressio...................................................................74
Figure 3.26. Proportio of idividuals wTP as a fuctio of moe (SEK). Pooled data for all
attack rate reductios......................................................................................78
Figure 3.27. Proportio of patiets wTP for a reductio i smptoms as a fuctio of the
price of treatmet............................................................................................80
Figure 5.1. British Medical Jourals checklist for revieers or referees..................................... 84
8/13/2019 Health Economics an Introduction Kobelt 2013
10/108
STUDy ExAMPLES
viii
stUdY EXAMPLEs
Stud Eample 1. Prevalece-based cost of illessmultiple sclerosis....................................... 25
Stud Eample 2. Icidece-based cost of illessglaucoma................................................... 29
Stud Eample 3. Cost-miimisatio aalsiskide trasplatatio...................................... 37
Stud Eample 4. Icremetal cost-effectiveess aalsisochomcosis............................... 39
Stud Eample 5. Markov models i cost-effectiveess aalsisglaucoma............................... 42
Stud Eample 6. Combiig data i cost-effectiveess aalsischroic heart failure................ 44
Stud Eample 7. Cost-utilit aalsis: icorporatig multiple evetsosteoporosis................... 59
Stud Eample 8. Cost-utilit aalsis i chroic progressive diseasesrheumatoid arthritis........ 62
Stud Eample 9. Cost-utilit aalsis i chroic progressive diseasesmultiple sclerosis........... 71
Stud Eample 10. Cost-beet aalsis ad illigess to paagia pectoris....................... 77
Stud Eample 11. Cost-beet aalsis ad illigess to paicotiece........................... 78
8/13/2019 Health Economics an Introduction Kobelt 2013
11/108
HEALTH ECOnOMICS: GEnERAL ISSUES
1
Chape 1
HEALtH ECOnOMICs: GEnErAL IssUEs
the Ecoomic of Healh a Healh Cae
Over the past decades, the abilit to provide treatmet for a icreasigl ide rage of diseases
has icreased epoetiall ith the itroductio of e techologies. Demad for care also has
icreased, partl i respose to this, but also for other reasos. The resultig rise i health care costs
has put cosiderable strai o ite resources, a situatio that has orseed i the face of the curret
global economic slowdown.
Ecoomic issues i health care are o discussed ideli public polic forums, the medical ad
scietic literature, ad the la press. This is a smptom of a importat chage i health care
markets. Attetio has shifted from the passive fudig ad admiistratio of sstems to active
cocer about the cost of care ad the health outcomes achieved. The health ecoomic thikig that
o permeates health polic ad health care sstems is raisig questios such as: Ho much should
e sped o health care ad ho do e esure it is spet efcietl? Ho ad he should e assess
the outcome of usig health techologies i cliical practice to esure resources are used efcietl?
Box 1.1 deiio of healh ecoomic
Health ecoomics is the applicatio of the theories, tools ad cocepts of the disciplie of ecoomics to the topicsof health ad health care.
Ecoomics as a sciece is cocered ith the allocatio of scarce resources; health ecoomics is cocered ith
the allocatio of scarce resources to improve health. This icludes both resource allocatio ithi the ecoom to
the health care sstem ad ithi the health care sstem to differet activities ad idividuals.
A rage of approaches to ecoomic evaluatio has bee developed to help address these importat
questios of efciec. This guide provides a itroductio to them. The rst chapter revies cotetual
backgroud, illustratig the icreased level of iterest i the use of ecoomics b polic makers, paers,
ad health care providers. Chapter 2 itroduces the various tpes of ecoomic evaluatio ad discusses
ho the approach the to compoets of ecoomic evaluatio: hat effect a treatmet has o health
ad hat it costs. The challeges are illustrated ith eamples of cost-of-illess studies, hich seek
to quatif the aggregate costs of a disease ad its treatmet. Chapter 3 eplores the methods of
ecoomic evaluatio i greater detail, focusig particularl o the use of modellig techiques that
sthesise data from a rage of sources. The chapter illustrates these techiques usig a umber of
Box 1.2 deiio of healh echology aeme
Health techolog assessmet (HTA) is a multidiscipliar process that summarises iformatio about the medical,
social, ecoomic ad ethical issues related to the use of a health techolog throughout its life spa i a sstematic,
transparent, unbiased and robust manner.
The aim of HTA is to iform the formulatio of safe ad effective health policies that are patiet focused ad seekto achieve value for moe.
8/13/2019 Health Economics an Introduction Kobelt 2013
12/108
HEALTH ECOnOMICS: GEnERAL ISSUES
2
eamples, primaril from evaluatios of drugs. Importat aspects of each methodolog are eplaied
ad particular challeges idetied. Chapter 4 discusses methodological guidelies for the coduct of
the ecoomic evaluatios that are required or suggested i several coutries. Chapter 5 cocludes.
Challege i healh cae: he coex
Total health care spedig as a proportio of gross domestic product (GDP) has steadil icreased i
all OECD coutries, albeit startig from differet levels. Spedig i the Europea Uio as betee
7.5% ad 12% of GDP i 2010 (9.5% to 12% i wester Europe, 7.5% to 9.5% i Cetral/Easter
Europe). I the US, it reached over 17% of GDP (see Table 1.1).
table 1.1. Healh cae expeiue a peceage of GdP
Source: OECD (2013), wHO (2011)
numerous iterdepedet factors cotribute to icreased health care costs, as idicated i Figure 1.1.
I the idustrialised orld, the elderl populatio ofte is sigled out for cocer as it cosumes a
substatial ad icreasig share of health care resources. Health care epeditures have rise less
because of demographic chage, hoever, tha because of the availabilit of a greater umber of
treatmet optios ad cotiuous improvemet i the qualit ad itesit of care. More ca be doe,
so more is done.
Figue 1.1. Majo coibuo o he goh of healh cae co
Cocers about the acig of health care are high o ever govermets ageda, particularl i
coutries here health care is predomiatl fuded ith public moe via taes, social isurace or a
combiatio of the to (see Table 1.2). Amog the OECD coutries, the US is a eceptio, ith most
health care beig aced b private isurace, although public acig is icreasig steadil. The
private portio of the market i Lati America coutries is substatial ad groig.
Governments around the world, and particularly in Europe, have attempted to contain costs using a
variet of measures aimed at both the demad for ad the suppl of health care. Figure 1.2 shos
those that have bee aimed at the prescriptio pharmaceutical market. These measures, hoever,have bee less successful tha hoped, partl because groth i spedig is drive primaril b the
availabilit of e ad improved techolog to hich cost-cotaimet measures are less easil applied.
Couy o regio 1970 1980 1990 2000 2010
OECd aeage 5.8% 7.3% 8.7% 8.4% 9.3%
Us 6.9% 8.7% 11.9% 13.2% 17.7%
Japa 4.6% 6.5% 6.1% 7.7% 9.6%
wee Euope 4.7% 6.7% 7.1% 8.4% 9.5%
Ceal/Eae Euope NA NA NA 6.1% 7.9%
Lai Ameica NA NA NA NA 7.0%
Information(Educated consumer)
Demographics(Ageing population)
Innovation(Technology)
Lifestyle(Abuse)
Structure(Incentives)
Relative price effects(Skill intensity)
Standard of living(Quality of life expectations) Costs
8/13/2019 Health Economics an Introduction Kobelt 2013
13/108
HEALTH ECOnOMICS: GEnERAL ISSUES
3
table 1.2. Public healh expeiue a pece of oal healh expeiue
* Data for 2009
Source: OECD (2012), wHO (2010)
Figue 1.2. Example of meaue fo coaiig peig o pecipio ug i Euope
Picig Clusterig (same price for similar treatmets)
Price cuts, price freezes
Referece pricig
Liig De-listig (removal from eligibilit for reimbursemet)
Positive or egative lists of products eligible for coverage
shapig ue Greater use of geerics ad/or cotrol of geeric prices
Increased patient co-payment
Prescribig budgets ad/or guidelies for doctors
Puchaig Tendering
Volume contracts
Iiec co cool Prot limits for maufacturers
Promotioal budget limits for maufacturers
Reductios i holesale ad retail pharmac margis
The acial crisis that bega i the late 2000s has eacerbated the situatio b makig further
icreases i public spedig o health care more difcult. Discussios ad decisios about prices
ad purchasig, as a result, are o takig place i a eviromet characterised more b cocer
about cost ad value tha about demad for iovatio. Health care decisio makers everhere are
focusig more arrol o efciec ad ithi tighter budgets. ne, more epesive, therapies must
carr a clear additioal health beet to be deemed orth of a additioal epediture. Decisios
makers, the, ill icreasigl require that iovative therapiesmedicies ad other itervetios
be assessed for relative effectiveess ad cost-effectiveess, rather tha ol efcac ad safet
(see Figure 1.3).
Figue 1.3. Aeme cieia fo e heapie
Couy o egio 1970 1980 1990 2000 2010
OECd aeage 73% 73% 73% 72% 72%
Us 36% 41% 39% 43% 48%
Japa 70% 71% 78% 81% 81%*
wee Euope 76% 76% 77% 76% 76%
Ceal/Eae Euope NA NA NA 75% 72%
Lai Ameica NA NA NA NA 52%
safey Does it have side effects ad are these acceptable ad maageable?
Efcacy Does it ork i a cotrolled eviromet (cliical trials)?
relaie efcacy Ho ell does it ork i a cotrolled eviromet compared to oe or more
alteratives (stadard treatmet)?
Effeciee Does it ork i ormal cliical practice?
relaie effeciee Ho ell does it ork i ormal cliical practice compared to other alteratives
(stadard treatmet)?
Co effeciee Is it a efciet use of resources, i.e. is a additioal beet orth a
additioal cost?
8/13/2019 Health Economics an Introduction Kobelt 2013
14/108
HEALTH ECOnOMICS: GEnERAL ISSUES
4
A umber of Europea coutries log have requested cost-effectiveess assessmets as a aid
i decidig about the reimbursemet status or price of a e techolog. Demad is groig for
comparative trials that ca better dee the icremetal beet of a e treatmet. Cost-effectiveess,
ad eve comparative aalses, hoever, are based o models created before the product reaches the
market. Util a product has bee used i routie cliical practice, cosiderable ucertait remais
about both cliical outcome ad resource use. As a result, authorities icreasigl are requestig
additioal evaluatios usig eperiece from actual cliical practice. I some cases, the results calead to a reegotiatio of the price ad also ma be used to shape cliical practice.
the role of Healh Ecoomic Ealuaio suie i Make Acce
A ecoomic evaluatio is a tool for assessig the beets ad costs of competig uses of scarce
resources. It provides data i a structured format that is comparable across diseases, but does ot
i itself offer a decisio. Sice value for moe is o a core cocer, aalses of the cosequeces
of the use of e ad eistig therapies, i terms of both beets ad costs, have become essetial
to decisios about resource allocatio. Cost-effectiveess has become a importat criterio ot ol
for decidig hich therapies ought to be fuded or reimbursed, but also for idetifig the patiet
populations that should have access.
Figue 1.4. deiio a fom of ecoomic ealuaio
Ma coutries have ofcial or quasi-ofcial specialised groups that assess the value of both curretand new health care technologies. These may be independent reimbursement agencies or specialised
HTA agecies. Ecoomic evaluatios are a itegral part of their assessmets.
A ecoomic evaluatio provides a comparative aalsis of alterative courses of actio i terms of
costs ad cosequeces (see Figure 1.4). This etails comparig alterative treatmet strategies
over the etire course of a disease, or deed disease episode, i order to idetif the best optio
for specic patiet groups, give epected costs. Such evaluatios use aggregate measuremets ad
provide iformatio for groups of patiets, rather tha idividual patiets. All evaluatios use similar
techiques to estimate cost, although differet techiques are used for measurig cosequeces,
depending on the disease or the desired result.
whe to itervetios have the same outcome, the less costl oe domiates ad is preferred.Interest is greater in products that improve outcomes compared to existing treatments that are only
equivalet i outcome. But more efcacious techologies geerall come at a higher cost. Thus, a
deiio of ecoomic ealuaio:A comparative aalsis of to or more optios i terms of their costs
ad cosequeces
type of ecoomic ealuaio
Co-miimiaio aalyi (CMA) Compariso of costs of alteratives that have the
same health outcome
Allows comparison within a clinical indication
Co-effeciee aalyi (CEA) Compariso of costs ad disease-specic health outcomes
(e.g. life-ears saved, patiets cured, evets avoided)
Allows comparison within a clinical indication
Co-uiliy aalyi (CUA) Compariso of costs ad geeric health outcomes
(e.g. qualit-adjusted life ears)
Allows comparison across clinical indications
Co-bee aalyi (CBA) Compariso of costs ad health outcomes valued i moetar terms
(e.g. illigess to pa)
Allos compariso to other sectors of the ecoom
8/13/2019 Health Economics an Introduction Kobelt 2013
15/108
HEALTH ECOnOMICS: GEnERAL ISSUES
5
icremetal cost-effectiveess ratio (ICER), i.e. the etra ivestmet required for the additioal health
beet, is computed. The more costl itervetio ill be adopted if the icremetal cost per uit of
health effect is less tha the purchasers illigess to pa for such a health gai.
Box 1.3. deiio of a ICEr
[ Cost (B) Cost (A) ] Differece i Cost
___________________ or _________________
[ Effect (B) Effect (A) ] Differece i Effect
here B is more effective ad more epesive tha A
(if B is more effective ad less epesive tha A, B domiates A ad the ICER is ot calculated)
Health-related costs ma be icurred i a rage of social spheres, makig it importat to iclude all
costs for a relevat time period, eve if the fall uder differet budgets. For istace, a e treatmetma icrease the pharmaceutical budget, but over time produce eough savigs i other parts of the
sstem to partl or full offset this icrease, such as loer hospitalisatio costs or feer moitorig
requiremets. Savigs also ma occur i other sectors of the ecoom, for eample, he sickess
abseces, earl retiremet due to disease, or premature deaths are avoided. For efciet resource
allocatio, decisios should cosider the full impact of therapies, regardless of here effects occur.
Ecoomic evaluatios, the, must start from a societal perspective to capture all potetial beets.
Adoptig a societal perspective to assessig the value of treatmets matters ad makes sese
(Jsso, 2009; Johaesso et al, 2009). Regulator authorities take a societal perspective i
licesig a drug, eighig risks agaist ider beets. Ecoomic aalses, similarl, eed to iclude
both costs ad beets to societ overall. This ca help decisio makers avoid a overl arro,
budget-specic perspective, hich ma miss the importat beets accrued outside that budget ad
produce suboptimal decisions about resource allocation. Narrow decisions may inappropriately restrict
access b ot fudig the treatmet at all or b iappropriatel limitig it to ol some groups
of patiets. I such cases, the paer ma achieve the objective of cotrollig the budget (static
efciec), but the greater beet to societ, particular patiets, ill be missed (damic efciec).
Despite the rather obvious potetial beet of usig health ecoomic evaluatios, decisio makers
across Europe var i ho ad ho much the are used. The remits of decisio makig orgaisatios
also differ: HTA agecies are geerall cocered ith hether or ot to recommed treatmets, but
lack the poer to decide o access ad price; some reimbursemet agecies ca ol accept or refuse
to fud a treatmet at the proposed price, hile others have the poer to egotiate price (Figure 1.5).
I recet ears, HTA agecies have become icreasigl ivolved i decisios about earl market
access, blurrig the distictio betee their activities ad those of traditioal reimbursemetassessmets. For eample, the natioal Istitute for Health ad Care Ecellece (nICE) ithi
the natioal Health Service (nHS) i the UK assesses selected e treatmets earl o ad its
recommedatios are bidig. Decisios b the Scottish Medicies Cosortium, a HTA bod ithi
the Scottish nHS, are full bidig. I Frace, the Haute Autorit de Sat (HAS) icludes bodies
that assess the absolute ad relative beet of a e techolog ad its reimbursemet status, ad
those that perform full assessmets of techologies after the have etered the market. I Germa,
the Istitute for Qualit ad Efciec i Health Care (IQwiG) assesses the effectiveessad, if
requested, cost-effectiveessof e treatmets oe ear after their itroductio. Clearl, the, the
timig ad impact of cost-effectiveess studies varies across coutries ad orgaisatios.
Guidelies for performig ecoomic evaluatios have bee produced i ma coutries. These fall ito
two categories.
8/13/2019 Health Economics an Introduction Kobelt 2013
16/108
HEALTH ECOnOMICS: GEnERAL ISSUES
6
1. Reimbursemet guidelies, i.e. guidelies issued b authorities that make the submissio of
ecoomic evaluatios madator for listig a e product o the reimbursemet formular,
ad that dee the format of such submissios
2. Methodological guidelies, i.e. guidelies proposed b researchers or groups of researchers
ith the aim of improvig the techiques ad methods used ad makig studies more
transparent
Figue 1.5. Boie iole i eemiig make acce
The rst coutr to make submissio of ecoomic studies a ofcial requiremet for listig medicies
o the atioal drug formular for reimbursemet as Australia, i 1993. Sice the, the guidelies
for submissios to Australias Pharmaceutical Beets Advisor Committee (PBAC) have bee updatedseveral times to icorporate eperiece gaied (PBAC, 2008).
The secod coutr to require ecoomic studies as Caada, based o a iitiative i the provice
of Otario. Detailed methodological guidelies ere developed i collaboratio ith all stakeholders:
govermet, isurace compaies, providers associatios (hospitals, pharmacists, phsicias),
academia ad the pharmaceutical idustr. Revised editios ere published i 1997 ad 2006, ith
addenda in 2009 that covered indirect treatment comparisons and evaluations in oncology. The
Caadia documet is idel cosidered authoritative i terms of methodological stadards ad
most of the guidelies published subsequetl b other agecies have relied heavil o the Caadia
guidelies (CADTH, 2006 ad 2009).
Iitiall, Europea coutries took a somehat differet approach. while guidelies as a epressio ofmethodological stadards ere elaborated ad published i most coutries, the ere ot at rst tied
to reimbursemet decisios. no, hoever, the majorit of coutries have made ecoomic evaluatios
madator for reimbursemet decisios ad require studies to follo ofcial guidelies produced b
the reimbursemet authorities. (See Figure 1.6 for a o-ehaustive list of guidelies.) Differeces
among the guidelines are limited, with the most important being the perspective that submissions
are epected to adopt. Other differeces relate to discout rate, time horizo, ad level of detail i
forecastig use of a e product, i.e. the aticipated budget impact. As ma of the coutries that
have made these studies madator are rather small, the miimize additioal effort b acceptig the
results of studies from other coutries, ith appropriate adaptatio to local eeds.
I the US, the Departmet of Health ad Huma Services commissioed a pael of academic eperts,
the washigto Pael, to elaborate a set of guidelies for good practice. The effort produced a
idel-quoted book (Gold et al, 1996) that has sparked itese scietic discussio aimed at further
developmet of the methods. Sice the, the Academ of Maaged Care Pharmac has published a
more specic set of guidelies for submissios:AMCP guidance for submission of clinical and economic
regulaoy agecie reimbueme agecie HtA agecie
role
Eiece
ue
Poe
Market authorisatio;
subsequet revie of
beet-risk prole,
if arrated
At lauch: safet ad
efcac (potetiall rela-
tive efcac) data from
randomised clinical trials
Post launch:
safet follo-up
Decisio
Coverage decisio ithi a
health care system, given
resource constraints
At lauch: relative efcac/
effectiveess ad budget
impact, formal cost-
effectiveess aalsesin most countries
Post launch: relative
effectiveess ad
cost-effectiveess
Decisio (ith/ithout
price egotiatio)
Provide best evidence
to iform coverage
decisios (e.g. cliical
practice guidelies)
At launch:
seldom involved
Post launch: relative
efcac/effectiveess,
cost-effectiveess
Recommedatio
8/13/2019 Health Economics an Introduction Kobelt 2013
17/108
HEALTH ECOnOMICS: GEnERAL ISSUES
7
Figue 1.6. Ue of ecoomic ealuaio i aiou couie
evaluation data to support formulary listing in US health plans and pharmacy benets management
organisations (Sulliva et al, 2001).
I additio to the documets ad guidelies produced b idividual coutries, a group of academic
researchers published a report on researcher independence in 1995 that attempts to deal with problems
of bias i ecoomic evaluatio (Task Force, 1995). The report suggests that evaluatios ought ol to
be performed b idepedet researchers ith o direct acial lik to the sposor or, if a stud is
sposored, researchers should have complete freedom to publish a ad all results. This is based i
part o cocers about iappropriate modicatio, at a later stage, of elemets such as effectiveess
measures ad aaltical methods. Ulike protocols for cliical trials, those for the ecoomic evaluatio
of e drugs are ot alas deed i detail at the outset. Hoever, the solutio to potetial ethical
problems such as this surely must lie in adherence to good practices by all participants in this evolving
eld, rather tha i cotractual arragemets.
Fomal reeach
Guielie Guielie
year of 1st year of 1st
Couy Ue of ecoomic ealuaio publication publication
Aualia Required for all e drugs 1993 nA
Auia Required for all outpatiet drugs, ith focus o nA 2006
comparison budget impact and price
Belgium Required for all outpatiet drugs, ith focus o nA 2002
added beet assessmet
Caaa Required at atioal ad provicial level 1995 nA
demak Voluntary submission NA 1997
Fila Required for all outpatiet drugs 1999 nA
Face Reimbursemet ol based o added beet; 2011 2004
re-assessment by HTA agency
Gemay Upo request, oe ear after lauch 2010 1995
Hugay Required for all drugs 2002 nA
Ialy Authorit to request at atioal ad regioal level nA 2001
nehela Required for all e drugs outside eistig clusters 1999 nA
ne Zeala Required for all e drugs 1993 nA
noay Required for all prescriptio drugs 2002 nA
Pola Required for iovative drugs 2007 nA
Pougal Required for all e outpatiet ad ipatiet drugs 1999 nA
spai not required at atioal level; ca be used at 2010 1995
regional level
see Required for all e drugs 2003 nA
UK (Egla Submissios requested o deed drugs 1999 nA
& wale) ad devices, either for revie of class or for
single technology appraisal
UK (scola) Required for all e drugs ad devices 2000 nA
UsA Icosistetl used for listig nA 1996; 2001
8/13/2019 Health Economics an Introduction Kobelt 2013
18/108
HEALTH ECOnOMICS: GEnERAL ISSUES
8
Similarl, allegatios that ol studies ith positive results are published idicate a fudametal
misuderstadig of the purpose of ecoomic evaluatio. First ad foremost, ecoomic evaluatio
studies are a tool to support decisios about resource allocatio. The primar purpose of such studies,
the, is ot to achieve publicatio, but to iform decisio makig. B ature, the are ot hpothesis
testig i the a that cliical trials are, but istead seek scearios here the product uder
evaluatio ca be epected to be cost effective. The scearios ma ivolve specic patiet populatios
(subgroups), specic admiistrative coditios, specic positioig (rst-lie or secod-lie therap,last resort), ad so oall variables that are iformed b the cliical trial results ad hece ofte
caot be specied i a geeral set of guidelies beforehad. The goal of paers is to make treatmets
available i a efciet a, i.e. to those patiets most i eed ad i those settigs here the are
cost effective. negative results (i.e. high ICERs) are thus of o iterest ecept for rejectig that
particular sceario. The ol a to esure both credibilit of the claims of value for moe ad
usefuless of the studies to decisio makers is to use soud methodolog ad relevat data, ad to
report results in a complete and transparent manner.
Among those countries where economic analysis must be considered prior to deciding on reimbursement
for e products, ecoomic submissios also are required he approval is sought for a e
idicatio for a eistig treatmet. But as is apparet from Figure 1.5, cosiderable differeces
eist i the etet to hich ecoomic aalsis is used. Sede ad Filad iformall used ecoomic
evaluatios i decisio makig eve prior to the sstematic assessmet of all e techologies. I Thenetherlads, a ecoomic criterio is applied to reimbursemet decisios ol for drugs that caot
be icluded i a eistig therapeutic cluster uder the referece pricig scheme. I nora, all e
products for geeral prescriptio (schedule 2) require a ecoomic submissio, hile i Portugal both
outpatiet ad hospital drugs are subject to ecoomic evaluatio. Belgium ad Austria both require
ecoomic evaluatios, but Belgium appears to have a strog focus o added beet hile Austria
appears to focus o price comparisos ad budget impact. I Scotlad, fudig decisios based o
cost-effectiveess, amog other parameters, are bidig.
Amog the large coutries i wester Europe, ol the UK has trul formalised its requiremets. nICE
as set up i 1999 b the Departmet of Health to assess e ad eistig health techologies ad
recommend whether and how these technologies should be used within the NHS in England and Wales.
Sice the begiig of 2002, i a effort to limit regioal differeces i access, it has bee obligatorfor the nHS to fud prescriptios based o nICEs recommedatios.
The orgaisatio ad fuctioig of nICE are differet from similar agecies i other coutries, i part
due to the log traditio of academic research i health ecoomics i the UK, coupled ith a drive for
greater transparency and public discussion.
NICEs role is to improve outcomes for people using the NHS and other public health and social
care services by:
Producing evidence-based guidance and advice for health, public health and social care
practitioners;
Developing quality standards and performance metrics for those providing and commissioning
health, public health and social care services;
Providing a range of information services for commissioners, practitioners and managers
across the spectrum of health and social care (nICE, 2013b).
A sizeable compoet of nICEs ork has bee made up of techolog appraisals, hich ma eamie
complete idicatios, sigle techologies, or etire classes of drugs.
Is the techolog likel to result i a sigicat health beet, take across the nHS as a
hole, if give to all patiets for hom it is idicated?
Is the techolog likel to result i a sigicat impact o other health-related Govermetpolicies (for eample, reductio i health iequalities)?
8/13/2019 Health Economics an Introduction Kobelt 2013
19/108
HEALTH ECOnOMICS: GEnERAL ISSUES
9
Is the techolog likel to have a sigicat impact o nHS resources (acial or other) if
give to all patiets for hom it is idicated?
Is there sigicat iappropriate variatio i the use of the techolog across the coutr?
Is the Istitute likel to be able to add value b issuig atioal guidace? For eample, i the
absece of such guidace is there likel to be sigicat cotrovers over the iterpretatioor sigicace of the available evidece o cliical ad cost effectiveess? (nICE, 2013c)
whe developig techolog appraisals guidace, nICE commissios a idepedet academic cetre
to review the existing published evidence on each technology and, in some cases, the evidence contained
i the maufacturers submissio. It also ma ask the academic group to perform a idepedet
ecoomic evaluatio. A specic guidace for maufacturers has bee developed to esure that all
submissios have the same format (the referece case).
the Impoace of Ecoomic Ealuaio fo he deelopme of
ne techologie
Ecoomic evaluatios have become a ke elemet, ad i ma coutries, a madator requiremet,
in supporting reimbursement submissions. In most pharmaceutical companies, these studies are an
itegral part of research portfolio maagemet iteded to developed products for the market that
the market ats. A similar developmet is udera i the medical devices idustr. Hoever, oce
reimbursement status has been achieved, little attention has been given to ensuring that products
still offer value for moe he used i actual cliical practice. This is chagig graduall ad a
icreasig umber of coutries o revie reimbursemet decisios at regular itervals (e.g. Caada,
Frace) or periodicall (e.g. Sede, Eglad/wales).
Reassessmet ma be a particular challege i some casesfor istace, i chroic diseases here
the treatmet goal is to dela progressio to severe disease states ith high costs ad lo qualit of
life (e.g. multiple sclerosis, rheumatoid arthritis), or i disease areas here most treatmets aim to
prevet mortalit (e.g. heart disease, cacer). For such diseases, it ma take a umber of ears beforeit is possible to observe the effect of a e treatmet i the real orld.
Ecoomic evaluatios at lauch are b deitio based mostl o relative efcac observed o the
cotrolled eviromet of the cliical trial. Assessig relative effectiveess ad hece cost-effectiveess
i the ucotrolled cliical practice eviromet presets quite differet challeges. Hoever,
coditioal reimbursemet approvals are becomig commo, tig iitial reimbursemet to subsequet
proof of cost effectiveess i cliical practice. Cotractual agreemets here the acial risk is
shared betee maufacturers ad the health care sstem also require ecoomic evaluatio based o
cliical use. Perhaps the greatest challege preseted is availabilit of relevat cost ad outcome data
from cliical practice. Observatioal follo-up, cohort studies ad patiet registries ca suppl such
data, provided they are set up do so.
The demad for comparative data alread eerts a substatial impact o the cliical developmet
of e treatmets, a situatio that is likel to itesif. Marketig authorisatio traditioall has
bee based ol o efcac ad safet evidece for the particular product. The curret demad for
improved, rather tha similar, outcomes, hoever, requires comparative studies that cosider relative
efcac. The choice of comparator ca have crucial implicatios: i additio to the difcult of choosig
a comparator that is deemed a appropriate alterative treatmet optio i the largest umber of
markets, the choice made also ma drive the positioig ad/or the price of the e product.
while requiremets for comparisos are beig better deed, reimbursemet authorities ad HTA
agecies usuall accept idirect comparisos betee treatmets. To give a simple eample, if
treatmet A has bee compared to treatmet B, ad treatmet C has also bee compared to treatmet
B, it is possible to statisticall estimate the compariso of A ad C. This clearl is less certai tha
direct compariso, particularl if the studies of B versus C ere performed some ears prior to those
of A versus B (or vice versa). Despite this limitatio, agecies ted to take the pragmatic vie that it
is better to have at least some supportig evidece available for decisio makig.
8/13/2019 Health Economics an Introduction Kobelt 2013
20/108
HEALTH ECOnOMICS: GEnERAL ISSUES
10
Integrating comparative research into the development process and combining clinical and economic
objectives presets a umber of challeges.
Ho ca efcac be traslated ito effectiveess?
what is a appropriate outcome measure? Ho ca a patiets health outcome be trasformed
ito a quatiable measure, e.g. qualit of life (utilit)?
what is a appropriate time frame for such ecoomic aalsis, compared to cliical proof of
efcac?
what is the appropriate product or other itervetio for compariso? Ad ho ca
comparisos agaist placebo be icorporated?
where ad ho ca resource-use data be collected?
Some of these poits are addressed i methodological guidelies. More ofte, hoever, the chose
approach is guided b feasibilit based o time frame, resource costraits, data availabilit,
ad the idicatio ad positioig of the e treatmet. Cliical trials geerall are regarded as
iadequate vehicles for collectig data o resource because cosumptio i a trial is madated adheavil iueced b the protocol. A illustratio of the overall combiatio of cliical ad ecoomic
developmet is provided i gures 1.7 ad 1.8. Figure 1.7 shos ho the accumulatio of iformatio
produces evidece of value for moe; Figure 1.8 provides details about the sequece ad phase
timig of ecoomic evaluatio.
Figue 1.7. documeig alue fo moey
The evaluatio process spas the etire developmet time for e products. It ill be more successful
if performed ith due regard to the aticipated iformatio eeds of providers ad paers, ad if full
itegrated ito the cliical developmet process. I the earlier stages of developmet, activities mostl
ivolve basic research about the disease, its ecoomic cosequeces ad the costs of treatmets.
I later stages, ecoomic data are collected hile Phase III cliical trials are takig place. Because
ecoomic evaluatios tpicall cosider a ider frame ad loger time horizo tha cliical trials,
data from differet sources ma eed to be combied: data o the disease ad its developmet
(epidemiological data), data o patiet maagemet ad resource cosumptio (ecoomic data), ad
outcomes data (cliical trials, registries). Most ecoomic evaluatios thus are modellig studies b
default ad such studies are o accepted as the rule, rather tha the eceptio, b reimbursemet
authorities and HTA agencies.
Coductig etesive ecoomic evaluatio at all stages of developmet ca be epesive ad the
koledge gaied limited, both because of the ature of cliical trials ad because data about the most
effective use of a product accumulates ol over time i actual use. Studies coducted after lauch
certail are ot ithout cost; icetives for such epediture are ol o developig. A balace
must be struck betee the costs ad the beets of preparig ecoomic evaluatios throughout aproducts life ccle. Geeratig ecoomic iformatio that ill ot be used or that could be misleadig
is pointless.
Clinical
hypothesis
testing
Decision oncomparator(s)
Simulation of
cost-
effectiveness
Unmet
needs
analysis
Burden of
disease
Pivotal
clinical
trials
Economic
data
collection
Outcome
data
Economic
evaluation
(model)
Value forMoney
8/13/2019 Health Economics an Introduction Kobelt 2013
21/108
HEALTH ECOnOMICS: GEnERAL ISSUES
11
Pre-
clinical Phase I Phase II Phase IIIA Launch
Phase IIIB Marketing
Disease analysis (unmet need)
Feasibility/Simulation
Post-launch studies(clinical practice)
Data collection (resource use)
Modelling cost-effectiveness
Burden of disease (quality of life, costs)
Input intodevelopment
Figue 1.8. Ecoomic ealuaio uig a afe eelopme
8/13/2019 Health Economics an Introduction Kobelt 2013
22/108
FORMS OF HEALTH ECOnOMIC EVALUATIOn
12
Chape 2
FOrMs OF HEALtH ECOnOMIC EvALUAtIOn
Ioucio
A health ecoomic evaluatio is a meas of establishig the value for moe of health care techolog
ad as such is a itegral part of HTA. Takig as our startig poit the deitio of a ecoomic
evaluatio i health care as a comparative aalsis of alterative courses of actio i terms of both
their costs ad cosequeces (Drummod et al, 2005), ecoomic aalses are alas comparative
and are applied to explicit alternatives. One pharmaceutical product can be compared more or less
ith itself (differet dosages or modes of admiistratio), or ith aother pharmaceutical product, orith aother tpe of itervetio such as surger, or ith a atchful aitig approach hereb the
patiet receives o form of medical itervetio, but istead is moitored for a chage i health
status. A treatmet caot be cost-effective b itself, but ol i relatio to oe or more relevat
alteratives ad for deed patiet groups. whatever the alterative, at a miimum all the costs
related to each method of treatig a relevat disease episode must be cosidered ad related to the
beets i terms of improvemet i the legth or qualit of life.
All forms of ecoomic evaluatio ivolve assessmet of both the iputs (the use or loss of resources)
ad outputs (health beets) of the health care programmes to be compared so as to facilitate the
process of choosig the most appropriate alterative. The decisio criterio is to maimise health
outcome for the populatio as a hole (social utilit), give resource costraits. If a treatmet
strateg geerates better outcomes ad is less costl, it domiates the alteratives. More ofte,
however, a treatment strategy that generates better outcomes also will be more expensive and, as
as oted earlier, a judgemet ill have to be made as to hether the icremetal beet is orth
the incremental cost.
Bo 2.1 illustrates the structure of ecoomic evaluatio. The iputs, or costs, are deed as the costs
related to the use of the treatmet mius the costs that are avoided as a result of its use, compared
to costs ithout the treatmet or ith a differet treatmet. Costs are a fuctio of the quatit of
resources used ad their price. Detailed data o prevailig treatmet strategies i cliical practice,
hoever, are seldom readil available. Idetifig the relevat resources, quatifig ad valuig
them, the, is geerall ecessar, but it is a rather straightforard process. Outputs are more difcult
to estimate for several reasos. Treatmets ofte affect multiple smptoms or evets ad at differet
poits i time. It ma ot be obvious ho to combie these effects ito a sigle comprehesive
outcome measure.
Box 2.1. Compoe of ecoomic ealuaio
INPUTSdeed as resources used or lost.
Direct costsare deed as costs related to the use of resources due to either the disease or its treatmet.
We generally distinguish between
Costs to the health care sstem (direct medical costs)and
Costs to social services ad to patiets themselves or to their relatives
(direct non-medical costs).
8/13/2019 Health Economics an Introduction Kobelt 2013
23/108
FORMS OF HEALTH ECOnOMIC EVALUATIOn
13
Indirect costs or loss of productionare deed as costs that occur to societ related to loss of productio,
due either to the disease or its treatment. We generally distinguish among
Short term losses due to sickess absece
Log term losses due to premature death or earl retiremet due to the disease (ivalidit)
Losses due to reduced productivit hile at ork due to the disease (e.g. because of fatigue,
migraie attacks).
Figue 2.1. sucue of ecoomic ealuaio
Other costs that occur due to illess ad ma be iueced b treatmet are intangible costs.These relate to the
sufferig ad loss of qualit of life eperieced b the patiet, ad sometimes are icluded i descriptive studies
(cost-of-illess studies). I the frameork of a cost-utilit aalsis, the effects of a treatmet o qualit of life
are icluded i the health outputs (as part of qualit-adjusted life ears [QALys]). Itagible costs are particularl
difcult to measure ad value. Several approaches eist, icludig the use of qualit of life istrumets, direct
measuremets ithi the frameork of illigess to pa assessmets, or a valuatio here the loss of QALys
compared to the ormal populatio is valued ith a assumed illigess to pa for makig up this loss.
OUTCOMESare measured as health improvements expressed as
1. Disease measures such as evets avoided or delaed (e.g. hip fractures i osteoporosis; mocardial
ifarctio, stroke or death i cardiolog), patiets successfull treated (e.g. umber of cacer patiets
i complete remissio; umber of ifectios cured ithi a give time)
2. Survival measured i terms of lives saved or life-ears saved
3. Qualit-adjusted survival, epressed as QALys
4. Moetar value, epressed as illigess to pa for the improvemet
type of Ecoomic Ealuaio
Economic evaluations are categorised by type, distinguished primarily by how outcomes are treated.
The appropriate meas of evaluatig outcomes ill deped o a umber of factors, the most importat
beig the medical ad ecoomic problem addressedi.e. hether the evaluatio seeks to iform the
selectio of a treatmet for patiets ith the same disease, or to iform the prioritisatio of treatmets
for differet diseases. The medical questio ill determie hat effectiveess measure is used, hile
the ecoomic questio ill iuece both the effectiveess measure ad the tpe of evaluatio to be
used. In general:
1. If the ecoomic questio is hether a treatmet is a good use of resources ithi the
disease area, the comparison is with similar treatments and the outcome measure can be
disease specic. The tpe of evaluatio ill be a cost-effectiveess aalsis if there is ol
a single outcome. With multiple outcomes, it is necessary to choose one, or to construct an
ide. For eample, outcomes i hpertesio ca be stroke or chroic heart disease; i
Inputs OutputsTherapeutic Action
Cost of deliveringthe treatment
minuscosts saved bythe treatment
Health benefitsin terms of
reduced mortality,reduced morbidity,
improved quality of life
8/13/2019 Health Economics an Introduction Kobelt 2013
24/108
FORMS OF HEALTH ECOnOMIC EVALUATIOn
14
osteoporosis, several differet tpes of fractures ca happe; ad i cacer, outcomes ca
be measured i terms of survival, remissios, side-effects, qualit of life, etc.
2. If the ecoomic questio is hether a treatmet represets a good ivestmet cosiderig
the etire spectrum of diseases, the compariso ill be ith treatmets for other diseases
ad the outcome measure ill eed to be geeric, such as the QALy, hich is a combiatio
of life epectac ad qualit of life. This ill eable a cost-utilit aalsis, a specic tpe ofcost-effectiveess aalsis. It is appropriate to coduct a cost-utilit aalsis he qualit
of life is a importat compoet of the effect of the disease ad its treatmet, or he
there are a large umber of differet smptoms ad effects to cosider.
A somehat differet form of ecoomic aalsis is the cost-of-illess stud, described i detail later
o. A cost-of-illess stud is ot evaluative, but purel descriptive; it aims to establish ad quatif
the burden that a particular disease places on society. Since these studies do not consider the outcome
of treatmet, the are of limited value to decisio makers cocered ith achievig value for moe i
health care. Hoever, the provide importat backgroud iformatio o the disease ad its costa
overall ecoomic assessmet of the curret situatio. As such, the ca provide much of the basic
data for a ecoomic evaluatio that ivestigates the outcome he somethig i that situatio
changes, such as a new treatment being introduced.
B far the most importat questio to ask before embarkig o a ecoomic evaluatio is hether or
ot clear ad ell-documeted cliical evidece is available for the techolog to be compared to the
available alterative(s). A ecoomic evaluatio ca ol be as good as the uderlig effectiveess
data, ad the highest qualit ecoomic data ill ot be able to overcome a deciec i the
effectiveess data. Data qualit has become oe of the most importat topics i the curret debate
surroudig HTA here the demad is o for comparative effectiveess data usig patiet-relevat
outcome measures, rather tha data o efcac agaist placebo.
Figure 2.2 summarises the effectiveess measures used i the differet tpes of evaluatio ad
idicates hat questios each tpe of evaluatio tpicall addresses. Each of these aalses is
discussed in detail later and illustrated using examples.
Figue 2.2. Effeciee meaue ue i ecoomic aalye
Oucome Meaueme i Ecoomic Ealuaio
I cliical trials, as i cliical practice, several differet measures ca be used to epress health
outcomes because a variet of treatmet effects ma be importat i terms of cliical maagemet. I
type of aalyi Effeciee meaue deciio uppo
Co-miimiaio
aalyi
Co-effeciee
aalyi
Co-uiliy
aalyi
Co-bee
aalyi
No measurement, as cost-minimisation
analysis applies to alternatives with no
differece i outcome
Oe disease-specic patiet-relevat
measure, such as events avoided
(e.g. stroke, relapses), cure, disease-free
time, or a more general measure such
as life-ears saved
A summary measure combining
survival ad qualit of life
(i.e. qualit-adjusted life ears)
Effectiveess epressed as a
moetar beet (e.g. illigess
to pa for a give effect)
Compariso of treatmets for the
same disease
Compariso of treatmets for the
same disease
Compariso of treatmets for
differet diseases
Compariso of ivestmets i the
health sectors with those in other
sectors (e.g. educatio, road safet)
8/13/2019 Health Economics an Introduction Kobelt 2013
25/108
FORMS OF HEALTH ECOnOMIC EVALUATIOn
15
ecoomic evaluatio, o the other had, outcomes eed to be epressed usig a sigle effectiveess
measure that is easy to understand and to relate to the disease, and that ultimately can be compared
to outcomes across diseases. The measure should also epress the overall ad al outcome, rather
than intermediate ones.
I acute ad curable diseases, such as ifectios, it is rather straightforard to dee the al outcome
i a dichotomous a, such as cure or o cure. The ecoomic evaluatio the ill estimate adcompare the costs of achievig the cure usig differet treatmet strategies. For eample, if a e
treatmet cures a additioal 10% of patiets tha is curretl the case at a additioal cost of 100
per patiet, the the cost per etra cure achieved is 1000 (100/0.1).
I disease areas here the risk of a udesirable evet is cotiuous, such as cardiac disease, the
outcome ma be deed as avoidig or postpoig that evet. Hoever, the ultimate objective of
prevetig serious cliical evets is to avoid the cosequeces of the evet (such as death or serious
disabilit), rather tha the evet itself. Ecoomic evaluatio thus ill preferabl attempt to capture the
cosequeces of avoidig such cliical evets b estimatig chages i survival ad qualit of life.
I chroic diseases, o the other had, particularl i chroic progressive diseases, deig a overall
al outcome is more difcult ad efcac is ofte assessed based o itermediate edpoits ol.
Some of these edpoits are patiet related, eve if the do ot epress the al outcome, such aseacerbatios, relapses ad recurreces of the disease. Some of the edpoits assessed i cliical
trials are iadequate for traslatio ito effectiveess, such as a relative improvemet of 20%, 50%
or 70% i a groupig of multiple smptomsas i rheumatic diseases, for eample.
Phyiological meaue a cliical ee
Phsiological measures (or surrogate edpoits) such as mmHg i hpertesio, mMol cholesterol i
hyperlipidaemia, or bone mineral density in osteoporosis, are routinely used in clinical management
as outcome measures as the are liked to cliical evets such as stroke, mocardial ifarctio ad
fractures. I these cases, ecoomic evaluatio ca the estimate the value of avoidig (or postpoig)
a evet, provided that epidemiological data likig the surrogate measure to the udesirable evet
are available. The cost-effectiveess of treatmet toda hich aims to avoid a future evet ca be
estimated if it is possible to derive a risk fuctio for the aual riskfor eample, of a hip fracture at
a give level of boe mieral desit ad at a give age, or of a mocardial ifarctio at a give level
of cholesterol, cotrolled for age, geder ad other ko risk factors (such as smokig).
Figure 2.3 illustrates this cocept. Here, a risk fuctio for the aual risk of a serious cliical evet at
give levels of a surrogate measure ad uder differet coditios (age, se, risk factors) is derived
from epidemiological data. This liks short-term itermediate edpoits ith al outcomes, eablig
a calculatio of the cost-effectiveess of treatmets that reduce this risk.
Figue 2.3. Exapolaig fom iemeiae o al oucome
Economic evaluationof the benefit of avoidinga serious event
Clinical dataefficacy
Epidemiological datarisk function
Surrogateendpoint
Seriousclinical event
Treatment
8/13/2019 Health Economics an Introduction Kobelt 2013
26/108
FORMS OF HEALTH ECOnOMIC EVALUATIOn
16
suial
As oted above, ma evets ma ot represet the true al outcome i a disease as the ma be
importat ol isofar as the are liked to mortalit risk. I diseases ithout clearl deed evets,
survival is ofte the relevat edpoit.
Survival ca be epressed i differet as, for eample: the proportio of patiets alive i eachgroup at the ed of a cliical trial, the umber of deaths avoided, the umber of patiets alive after
ve ears, or overall survival. I ecoomic evaluatio, survival is geerall measured i terms of ears
of life, ad is represeted b a area uder the survival curve that ca be related to both costs ad
qualit of life. Hoever, cliical trials are seldom log eough to provide the data ecessar to estimate
directl the umber of life-ears saved (LyS) b oe treatmet compared to aother. Epidemiological
data are agai required to etrapolate from the short-term perspective of lives saved to the log-term
perspective of life epectac.
Figure 2.4 illustrates the cocept of LyS ad shos that the effects of a treatmet achieved ithi
trials carr over to the period after the trial. A differece i the umber of patiets alive at the ed of
a cliical trial ill lead to a differece durig the ears after the trial. For istace, if e assume that
5% of patiets survivig at ear ve die ever ear after the ed of the trial i both the cotrol ad
itervetio groups, all patiets ill be dead after 20 ears. Mea ad media survival after the trialill be te ears. If e further assume that survival at the ed of the trial as 80% i the cotrol
group ad 90% i the itervetio group, the gai i life epectac i the itervetio group ill be
0.25 ears durig the trial ((50.1)/2) hile the gai after the trial ill be oe ear ((200.1)/2).
The life epectac at the start of the trial ill be 12.5 ears i the cotrol group ad 13.75 ears i
the itervetio group, ith the majorit of the differece achieved after the cliical trial. The area
betee the to curves i the Figure 2.4 represets the differece i life epectac of the to groups.
Figue 2.4. Exapolaig fom ihi-ial moaliy o life yea ae
Qualiy-ajue uial
Outcome measuremet i chroic or progressive diseasessuch as Alzheimers disease, Parkisos
disease, multiple sclerosis (MS), rheumatoid arthritis (RA)is more difcult, as ofte o distict
evets have a impact o survival. Istead, patiets eperiece a declie i phsical ad/or metal
abilities over time. Ofte such diseases affect several fuctios ad produce a umber of differet
smptoms, leadig researchers to seek a outcome that ecompasses all effects. The most frequetlused such measure i ecoomic evaluatio is the qualit-adjusted life ear (QALy), hich captures
the overall effect of a disease o qualit of life over a give period of time, ad combies the quatit
100%
%o
fpatientsalive
05 25Clinical Trial Normal Survival
Lives saved(difference in survivors)
Life-years saved (LYS)
(area between the curves)
Time
8/13/2019 Health Economics an Introduction Kobelt 2013
27/108
FORMS OF HEALTH ECOnOMIC EVALUATIOn
17
ad qualit of life gaied from treatmet. QALys ca be compared across diseases ad thus support
choices for resource allocatio ithi a overall health care budget. As a cosequece, QALys are the
outcome measure preferred b ma govermet bodies ad other authorities that require ecoomic
evaluatio prior to recommedig that a treatmet be provided usig public fuds.
QALys are calculated b adjustig time (ears of life) ith a ide that epresses global qualit of life
(utilit) o a scale achored at 0 (death) ad 1 (full health). Utilit ca be measured usig techiquesfrom decisio aalsis that are eplaied later i this book. For eample, if beig blid has a utilit
of 0.4, spedig 10 ears as a blid perso ould give four QALys, hich is equivalet to spedig
four ears i full health. Thus, treatmets that prolog life (e.g. life-etedig cacer treatmets)
ca be assessed i the same a as oes that improve qualit of life (e.g. treatmets for rheumatic
diseases). Figure 2.5 illustrates this cocept.
Figue 2.5. the cocep of qualiy-ajue life yea
I order to compare QALys from differet studies, the same methods eed to be used i measuremet.
This is ot alas doe i practice ad is oe of the reasos h the use of QLAys has bee met ith
some scepticism.
without attemptig to do justice to the vast literature o QALys, it is useful to metio the gist of the
criticisms made agaist their use. Some of these cetre aroud the idea that QALys do ot accuratel
reect prefereces about survival ad qualit of life. Cosider the eample described above, here
blidess as valued at 0.4. This suggests that the idividual achieves four QALys he this health
state is eperieced over te ears, eight QALys he it is eperieced over 20 ears, ad so o.
However, it is possible either that some health states become either less tolerable over timeor more
tolerable for people ho adjust to the coditio. For a particular idividual, the, 20 ears of blidess
ma seem orth less tha, or more tha, the QALy value attached to spedig te ears i the same
state. I additio, some health states ma be preferable to death ol for a period of timesurvival
beod that poit ma seem less desirable tha immediate death. A shorter period of survival i those
health states, the, is preferred to a loger period.
From a equit perspective, it is sometimes argued that QALys discrimiate agaist certai groups,
such as the elderl. The potetial umber of life ears that ca be saved b treatig a 80-ear-old
patiet is feer tha the umber of life ears that ca be saved b treatig a 40-ear-old patiet. This
seems to udervalue the elderl patiet. Maimisig use of QALys to distribute resources, i additio,
implies that all QALys are of equal social value, o matter ho beets. Societ, hoever, ma ish
to give priorit to certai groups ad esure that those patiets have access to treatmet eve if the
cost-per-QALy is high.
Although the QALy is ot a perfect measure, its use is idespread because o clearl superior alterativecurretl eists for makig comparisos across diseases. Most decisio makers icorporate cocers
about QALys ito their decisios b ot applig a strict moetar threshold to their illigess to
Full health
Death
1.0
0.4
0
0 4 10Years
10 years at 0.4 = 4 QALYs
4 years at 1.0 = 4 QALYs
8/13/2019 Health Economics an Introduction Kobelt 2013
28/108
FORMS OF HEALTH ECOnOMIC EVALUATIOn
18
pa for a QALy. The ma be illig to pa more for treatmets for certai patiet populatios or rare
diseases. I other ords, the cost-per-QALy estimate ill ot be the ol decisio criterio.
Moeay oucome
I cost-beet aalsis, the outcome of a treatmet is epressed as the illigess of idividuals or
societ to pa for it. Moetar outcomes have bee met ith some scepticism i the medical eld,
mostl due to the reluctace to dee a threshold value that societ should be paig for a give
outcome, such as a life-ear or QALy. Furthermore, the techiques for measurig illigess to pa
have ot bee as ell tested ithi the health care eviromet as techiques for measurig utilities.
Paie-epoe oucome
The interest in measuring patient-reported outcome, i.e. patients subjective well-being, has increased
i recet ears. Oe eplaatio for this is the icreasig umber of people ith chroic diseases, hich
predomiatl affect patiets qualit of life. The objective of treatmet here is primaril to improve
patiets phsical, metal ad social fuctioig. The classical cliical measures are ofte iadequate
for describig ad evaluatig these effects, so a umber of istrumets to measure health-relatedqualit of life have bee developed, both geeric ad disease-specic. These istrumets are desiged
to elicit patiets subjective evaluatios of the effects of a disease or a treatmet ad have become
a importat tool for the assessmet of outcomes. Hoever, for the purposes of cost-utilit aalsis,
these measuremets ca be used ol if the are epressed as a ide, or eight, ith clearl-deed
achors betee the orst ad the best health states.
Health is deed b the world Health Orgaisatio as a state of complete phsical, metal ad
social ell-beig ad ot merel the absece of disease or irmit (wHO, 1948). I geeral,
measuremets of health-related qualit of life are carried out alog these three dimesios. Figure
2.6 lists some of the cocepts tpicall measured.
Figue 2.6. dimeio i paie-epoe oucome
Istrumets used to measure patiet reported outcomes fall ito three basic categories, used i
differet circumstaces ad for differet purposes:
1. Generic measures
2. Disease-specic measures
3. Preferece-based measures (utilit measures).
Generic measures were developed to assess health status across all diseases and are relevant to all
health problems. The have the advatage that the impact of a treatmet for oe disease ca becompared ith that of treatmet for aother disease. A potetial draback of geeric istrumets is
that the ma fail to capture small, but importat, effects that are specic to a particular disease. To
dimeio Cocep Iclue
Phyical Phsical fuctio Mobilit, activities of dail livig, self-care
Smptoms Pai, fatigue, ausea
Phsical role work, household tasks
Meal Psychological well-being Happiness, depression, anxiety
Persoal costructs Spiritualit, life satisfactio
Cogitive fuctioig Memor, cocetratio
social Social role Famil life, social cotacts, friedship
Social well-being Stigma, isolation
Oeall Global judgemet of health Overall ratig of curret health
Satisfactio ith care Satisfactio ith treatmet
8/13/2019 Health Economics an Introduction Kobelt 2013
29/108
FORMS OF HEALTH ECOnOMIC EVALUATIOn
19
address this limitatio, disease-specic istrumets have bee developed for ma diseases. These
measure the distictive aspects of diseases that are tpicall missed b geeric measures, thus
providig valuable iformatio i cliical trials, assessmet of specic eeds or patiet moitorig.
Hoever, the are ot useful for compariso betee diseases ad hece caot be used i decisios
relating to resource allocation across therapy areas.
The third categor of istrumet, preferece-based measures, is of particular iterest to ecoomistsbecause it ields a set of eights (utilities) o hich QALy calculatios ca be based. Some geeric
istrumets ill ield a overall qualit-of-life score as a ide ad therefore ca be used as utilit
measures suitable for geeratig QALys. The EuroQol Groups EQ-5D is a tpical eample of this,
as its descriptive health states are liked to preferece-based assessmets. I cotrast, aother
frequetl used geeric measure, the SF-36, does ot produce a overall ide, but rather to
summar scores for metal ad phsical domais ad therefore caot directl be used to geerate
QALys. More recetl, a algorithm etractig domais from the SF-36 to calculate a utilit ide has
bee developed (SF-6D) ad uder certai circumstaces ca be used to geerate QALys.
All outcome istrumets must stad up to scruti for reliabilit, reproducibilit, validit, feasibilit ad
sesitivit to chage ad ca be assessed agaist these criteria usig pschometric techiques. Figure
2.7 presets some of the better ko istrumets.
Figue 2.7. Eablihe oucome iume
The developmet of a qualit-of-life istrumet is a complicated process that ca spa several ears.
The Medical Outcomes Stud SF-36, for eample, as developed over a period of te ears, usig
questioaires ad data from the RAnD Medical Outcomes Stud i the US. It as traslated, adapted
ad validated i a large umber of coutries. Acceptabilit of a e istrumet ill deped o its use
i several differet ivestigatios, addig further dela to its idespread use. Thus, developmet of
e disease-specic istrumets should be udertake ol he o adequate istrumet is available
ad this lack of availabilit caot be overcome b, for istace, usig a combiatio of eistig
istrumets that together address the cocepts required.
Co daa fo Ecoomic Ealuaio
Pepecie
I order to capture all costs that are of relevace to societ, ecoomic evaluatios should be performed
from a societal perspective. Hoever, a umber of jurisdictios make decisios about e treatmets
from the perspective a public paer, coverig health care, social services, ad pesios, or a health
care payer, covering only health care and related services. The perspective used will determine which
resources are included in the analysis.
A societal perspective icludes all costs, regardless of ho icurs them. Thus, costs to the health
care service, social services, patiets ad the rest of societ (for eample, i the form of productio
losses) are icluded, but trasfer pamets are igored. Eamples of trasfer pamets are taes
ad reimbursemet for icome loss due to illess. For societ as a hole, taes ad reimbursemetrepreset a moe o from oe part of societ to aother, but o resources (labour, capital) are beig
used up. The relevat cocept of cost i ecoomics is that of opportuit cost, i.e. the beet foregoe
type of iume Example iume
Geeal healh pole Short Form 36 (SF-36), nottigham Health Prole (nHP),
Sickess Impact Prole (SIP), Geeral well-Beig Scale
Geeal healh iice Ide of well-Beig, EQ-5D, Health Utilities Ide (HUI), SF-6D
dieae-pecic cale Arthritis Impact Measuremet Scale (AIMS), Miesota Livig
ith Heart Disease Scale, Multiple Sclerosis Qualit of Life Ivetor
(MSQLI), Beck Depressio Ivetor (BDI)
8/13/2019 Health Economics an Introduction Kobelt 2013
30/108
FORMS OF HEALTH ECOnOMIC EVALUATIOn
20
from usig resources for oe purpose, rather tha for their best alterative use. This deitio serves
to remid us that costs ill be icurred eve he the use of a resource is ot associated ith a
acial os, such as i the case of a volutar caregiver.
From the perspective of a third-part paere.g. a govermet, isurace compa, or maaged care
orgaisatiool resources paid for b that orgaisatio are icluded as costs. For istace, a
reimbursemet to patiets for icome loss is a actual cost to the third-part paer. A good eampleof the effect of differet perspectives is sho i Table 2.1., take from a cost of illess stud for MS.
I the perspective of the health care paer, ol medical costs are icluded.
table 2.1. Mea co pe Ms paie i Gemay
Source: Kobelt et al (2006a)
sep i co aeme
Assessig the costs i a ecoomic evaluatio ivolves four steps, hich are idetical i all forms of
economic analyses.
1. Idetif the relevat resources used
2. Quatif these resources i phsical uits, such as hospital das, admissios, surgical
procedures, physician visit, tests, etc.
3. Value the differet resources used i terms of their opportuit costs
4. Adjust valuatios to accout for the differetial timig at hich resource use ca
occur (discoutig)
Identication of resources. Relevat resources ill be deed b the stud objective. I a
cost-of-illess stud, this ill iclude all resources related to the disease, its cosequeces ad its
treatmet. I a ecoomic evaluatio, relevat resources ca be deed as those that are related
to the admiistratio ad cosequeces of the treatmet durig the disease episode cocered. For
eample, if to differet surgical itervetios for the same problem are to be compared, such as
open surgery and laparoscopic surgery, resources related to the original disease diagnosis are not
relevat, as these are idetical for both alteratives.
Resource quantication.The a i hich resources are quatied ill deped o hat eeds to be
measured ad hether a uit cost ca be assiged to it. If a itervetio reduces hospital das, oe
ill logicall collect hospitalisatio data i the form of legth of sta. If it reduces the umber of hospital
Co pe peo a yea (2005 EUr)
type of Co socieal pepecie Public paye pepecie
Ipaie cae 3,203 3,133
Coulaio 3,096 1,860
te 368 368
Phamaceuical 10,498 9,588
seice 525 241
Aopio (ieme) 989 393
Ifomal cae 4,407
Poucio loe 16,911
tafe co (peio) 3,404
Total costs 39,998 18,988
8/13/2019 Health Economics an Introduction Kobelt 2013
31/108
FORMS OF HEALTH ECOnOMIC EVALUATIOn
21
admissios, oe ould collect data o admissios (or discharges). But if the itesit of care ithi
the hospitalisation is reduced by the intervention, it will be necessary to collect all details on resources
used during the stay.
Figue 2.8. typical iem of eouce ue i a ecoomic ealuaio
Resource valuation. The quatit of uits used is multiplied b their uit cost (price) to obtai
the total cost. The a i hich resources have bee quatied ill determie hat uit costs are
assiged to them. Admissios or discharges ill be costed usig aggregate measures of resource use
(macro-costig), hile costs icurred durig the hospitalisatio itself ill require uit costs for each
idividual resource (micro-costig).
Figue 2.9. dimeio of co a pice
Co ype Example of eouce
diec meical co Hospitalisation
Das of hospitalisatio
Discharges
Outpatient visits
Outpatiet cliic attedace
Visit to private practitioer
Visit to paramedic
Procedures and tests
Tests (blood aalsis, x-ra, scas, gastroscopies, etc.)
Surgical itervetios
Devices Medical devices (heelchairs, hearig aid, pacemakers, etc.)
Services
Home care (hours or das)
nursig care (hours or das)
diec o-meical co Transportation
For outpatiet visits (ambulace, tai, etc.)
For dail activities
Services
Home help (hours or das)
Meals o heels
Social assistace (hours or das)
Devices ad ivestmets
Adaptatio to house or car
Special kitche ad bathroom utesils
Iformal care
Care b relatives (is sometimes also cosidered a idirect cost)
Iiec co Sick leave (das or eeks)
Reduced productivit hile at ork (percetage or hours)
Earl retiremet due to illess (ears to ormal retiremet)
Premature death (ears to ormal retiremet)
Oppouiy co Cost of the et best alterative foregoe
taiff Price deed b or egotiated ith a third part paer
Chage Billings to third party payers or patients
8/13/2019 Health Economics an Introduction Kobelt 2013
32/108
FORMS OF HEALTH ECOnOMIC EVALUATIOn
22
Uit costs should represet opportuit costs (Figure 2.9). Bearig i mid the cocept of opportuit
cost as beet foregoe, a simple eample of opportuit cost is that of a phsicias time durig
a cosultatio. The time used durig oe cosultatio caot be used for aother cosultatio, ad
hece has a cost. The opportuit cost i this case is the value lost for the cosultatio that as
ot udertake. I ormal ell-fuctioig markets, market prices provide a good represetatio of
the opportuit costs of resources, but i health care this is ot alas the case. I coutries ith a
atioal health service, such as the UK or Sede, resources ma ot be subject to market valuatios.
I some coutries, the ol eas source of costs is tariffs, i.e. prices set b a govermet or a public
isurer for pamet to health care providers such as hospitals or phsicias. I ma cases, hoever,
tariffs do ot represet the actual opportuit costs. I fee-for-service sstems here each service is
paid for separatel, tariffs ma be set to iclude icetives for the level of suppl of a give resource,
ith high tariffs set to ecourage provisio ad lo tariffs to discourage it. A eample is sho i
Table 2.2, hich summarises a stud of the cost of glaucoma i Germa.
table 2.2. taiff a oppouiy co i a co of ille uy (glaucoma) i Gemay
Source: Kobelt et al (1998)
I other coutries, the most readil available uit costs ma be billigs (charges) from providers to
differet paers, geerall isurers or health plas. Such charges ofte are used to subsidise other
activities, e.g. within the hospital, and will hence be higher than the opportunity costs. This is the case
for istace i the US, here a cost-to-charge ratio of 1:2 is ofte applied.
Challeges arise i applig appropriate valuatios to resources that have a opportuit cost, but oclear market price, such as iformal care b famil members or frieds. These costs ca be importat
in disabling diseases and chronic diseases prevalent in the elderly, in particular. Agreement has not yet
developed on whether and how to include such resources in economic evaluation. Two methods are
geerall used: replacemet cost (i this case, the cost of a professioal providig the care i lieu of
the famil), or the loss of leisure time hile providig care, commol valued as disposable icome.
Hoever, these costs do ot ecessaril have to be valued i moetar terms for decisio makers to
take them ito accout.
The role plaed b idirect costs (productio losses) ill to some etet deped o the patholog
being analysed. In diseases such as asthma, depression, schizophrenia, MS and migraine, indirect
costs ted to make up a sizeable proportio of the total cost of the illess because these diseases
affect age groups ith high labour force participatio. I diseases that affect predomiatl elderl
people, indirect costs would be less important.
dM pe ui (1997)
reouce Iuace aiff Oppouiy co
(quaely billig) (ime, upplie, oehea)
Coulaio
ii/quae 19.11 34.62
ubeque ii/quae 3.56 34.62
elephoe 3.56 3.68
te
Gol