J.1756 5391.2010.01087.x

Journal of Evidence-Based Medicine ISSN 1756-5391

METHODOLOGY

A methodological framework of preparing economicevidence for selection of medicines in the Chinese settingXin Sun1 and Thomas Alured Faunce2

1 Chinese Evidence-Based Medicine Center, West China Hospital, Sichuan University, Chengdu 610041, China2 Associate Professor and Australian Research Council Future Fellow, College of Medicine, Biology and the Environment and College of Law(joint appointment), Australian National University, Australia

Correspondence

Xin Sun, Chinese Evidence-Based MedicineCenter, West China Hospital, SichuanUniversity, No. 37 GuoXueXiang, Chengdu610041, China.Tel: +86-28-85423458Fax: +86-28-85422253Email: [email protected]

Received 22 June 2010; accepted forpublication 5 July 2010.

doi: 10.1111/j.1756-5391.2010.01087.x

Abstract

Medicines are becoming a major component of health expenditure in China. Se-lection of effective and cost-effective medicines represents an important effort toimprove medicines use. A guideline on cost-effectiveness studies has been avail-able in China. This guideline, however, fails to be a practical tool to prepare andcritically appraise economic evidence. This article discusses, in the Chinese con-text, the approach to integrating economic component into the medicines selection,and elaborates the methods of producing economic evidence, including conducingeconomic reviews and primary economic studies.

Medicines are an increasingly significant component ofhealth expenditure in China. The medicines expenditureof hospitalized patients, for example, usually accounts foraround 45% of their total medical bill (1). Rapid increasesin price are rendering medicines unaffordable to manyChinese patients. The demand for more efficient, quality useof medicines has become increasingly strident, and the im-plementation of an Essential Medicines List in patient carerepresents one of the efforts to improve medicines use.

Selection of effective and cost-effective medicines is thekey to implementing an efficient Essential Medicines List.In the last decade, extensive methodological work has beenundertaken in China to improve the assessment of effective-ness of medicines (2, 3). However, little was done to explorebest approaches to addressing cost-effectiveness. Despiteincreasing publication of cost-effectiveness studies in theChinese medical literature (4, 5). the methodological qualityvaries substantially.

In an effort to improve the conduct of cost-effectivenessstudies, China has developed a guideline on economic evalua-tion of medicines (6). This endeavor—which follows a streamof similar attempts in other countries (7–11)—offers method-ological standards of conducting economic studies. However,it fails to provide policy makers and researchers with prac-

tical tools for preparing and using economic evidence forselection of medicines. This article offers a methodologi-cal framework to address issues about economic evidence inthe context of medicines selection for a Chinese EssentialMedicines List.

Integrating economic evidence intothe selection of medicines

Integrating economic evidence in the process of medicinesselection for an Essential Medicines List typically starts withformulation of a focused economic question that reflects pol-icy maker’s perspective. It is followed with a systematic re-view of existing studies that assesses the strength of eco-nomic evidence. The subsequent decisions will then be madeon the basis of strength of economic evidence, which can becategorized into three levels.� Strong evidence without uncertainty. The existing evi-dence provides precise and unbiased estimates of compar-ative cost effectiveness, and the study settings—includingpatients, interventions, outcomes, and clinical practicepatterns—are applicable to the question being evaluated. Insuch case, a decision about acceptance or refutation is ableto be made.

156 JEBM 3 (2010) 156–161 c© 2010 Blackwell Publishing Asia Pty Ltd and Chinese Cochrane Center, West China Hospital of Sichuan University

X Sun and TA Faunce Economic evidence for selection of medicines

Develop the research question

Search the economic studies

Screen the economic studies Primary inclusion, and justify the exclusion

Assess the methodological quality

Assess the transferability

Extract and adjust data

Analyze and summarize data

Adequate – secondary inclusion, and move to the next step; Inadequate – exclusion

Acceptable – final inclusion Poor – exclusion

Adjustable – results applicable for drug decision-making Not adjustable – justify the results

Target population, interventions, and outcomes

Foreign and Chinese medical databases, Specialized database, health economics databases

Figure 1 The flowchart of systematic review of economic evidence.

� Moderate evidence suggesting cost-effectiveness pro-files. The estimates of comparative cost-effectiveness areeither imprecise, involve moderate likelihood of bias, orthe study settings differ in part of components affectingtransferability of results. In such case, a decision will in-volve uncertainty, and conduct of new economic studies isdesirable.� Week evidence involving substantial uncertainty. Theestimates of results involve significant uncertainty (either theestimates is imprecise, the methodological quality is verylow, or both), and/or the study settings are substantially dif-ferent from question being evaluated. New economic studiesare required in such circumstances.

Systematic review of economicevidence

The framework for conducting systematic review ofeconomic evidence on a medicine’s comparative cost-effectiveness is presented in Figure 1. Empirical studies areundertaken elsewhere (12–14). We discuss points with par-ticular interest to the selection of medicines for subsidy undera Chinese Essential Medicines List.

Develop the research question andeligibility criteria

A single most important step of an economic review is to de-velop the research question. Typically, this question has threeminimal components, including study population, interven-tion strategies, and health outcomes of interest. Collaborationbetween researchers and policy makers is crucial in devel-oping a well-defined research question. An example mightbe a study comparing the cost-effectiveness of a new heartfailure medication against an existing marketed product atthe standard dose and price in rural or urban populations.

Corresponding to the research questions, eligibility criteriashould be established. Eligibility criteria may be wide ornarrow, however, should be reflected with policy makers’perspective. That is, the patients and interventions defined,and outcomes of interest should be relevant to the policymaking. For example, the finding that a new drug gives apatient increased quality adjusted life years (QALYs) maybe more important than the finding it alters physiologicalparameters (like blood pressure).

In defining the criteria, the systematic review should in-clude all types of study designs (eg, modeling, trial-basedstudies, database study, and observational cohort), and allforms of evaluation (eg, cost-effectiveness analyses, cost-utility analyses, cost-consequence analyses, and cost analy-ses), unless otherwise indicated by policy makers.

Methodological quality

Methodological quality, which provides the extent to whichthe results are biased, is subject to study design, data collec-tion, and data analysis, and reporting (15–17). Tools address-ing methodological quality of economic studies differ (18–20), and two checklists—Drummond’s checklist (28). andChiou’s checklist (21)—are commonly used, both consistingof similar items. Chiou’s instrument also weighs items, andis validated externally (22). A modified Drummond’s check-list incorporating three set of weights (23). demonstrated itsability in rating methodological quality, while not validated.We recommend using both checklists to examine consis-tency of results; in the presence of discrepancy, one mayconsider exploring details. In presenting the risk of bias ofestimates, the overall methodological quality may be ratedand categorized into three levels (ie, high, moderate, andlow).

Transferability

The extent to which the study is applicable (or transferrable)to the question of interest depends on a number of potentialfactors (24–27). Currently, neither consensus nor instrumentis available to rate the impact of on the transferability ofresults. Here, we suggest key factors and propose methodsof assessment (Table 1).

Methodological characteristics are likely to be adjustable,including currency, base year, and discount rate, Methods areavailable for adjusting these characteristics, and the adjust-ment requires availability of sufficient primary data. The im-pact of patient characteristics on transferability is subtle, andempirical evidence manifesting this does not exist. Healthpreference is among the major factor influencing transfer-ability. In assessing the potential impact, patients character-istics should be listed, and the judgment about the impactshould be justified.

JEBM 3 (2010) 156–161 c© 2010 Blackwell Publishing Asia Pty Ltd and Chinese Cochrane Center, West China Hospital of Sichuan University 157

Economic evidence for selection of medicines X Sun and TA Faunce

Table 1 Factors influencing transferability of economic evidence

Factors Appraisal guide

Methodological characteristicsPerspective Applicable if conducted from the healthcare sector perspective, or if disaggregated data were available

from the societal perspective.Possibly inapplicable if applying a narrower perspective, or if aggregated data are used.

Clinical setting Desirable if including study population of diverse socio-economic conditionsPotentially problem for a single healthcare centre study and international study.

Discount rate Applicable if using the same discount rate or similar range.Costing methods Questionable if micro-costing methods are not used.

Patient characteristicsDemographics Minor impact on the transferability in the presence of age or sex differences only.Case mix May significantly influence transferability of results. Potentially inapplicable in the presence of differential

case mix groups across countries/regions.Comorbidity Small to moderate effect on transferability, particularly when its contribution to total costs and effects

are minor.Health preferences Its impact on transferability varies. Sensitivity analyses may be considered to examine its impact.Compliance Particular relevance to trial-based studies. Its impact depends on the extent of compliance. Small to

moderate influence on transferability.Healthcare system characteristics

Unit costs/prices Large impact on transferability. Adequate adjustment might be appropriate to transfer the publishedresults to the Chinese setting.

Resources use Its impact depends on clinical practice variations and patient characteristics. Questionable transferabilityif resources use pattern differ across regions. Adjustment is usually not sufficient. Transferable if onlya small part of resources use differs and have minimal impact on the total costs.

Clinical practice variations Unlikely transferable if clinical practice pattern differs substantially.

Unit costs, resources use, and clinical practice patternsrepresent the key components affecting transferability. Unitcosts usually differ across regions, and may be adjustable.Clinical practice patterns, which also underlie resources use,have the great potential of influencing transferability, andmay be least adjustable. In the presence of differences ofclinical practice patterns, a summary detailing the differencesand potential impact is desirable.

Data Analysis and Presentation

Economic studies are often heterogeneous, and may differqualitatively (28). Four approaches are available to ana-lyze results in systematic reviews (19, 29–31), and narra-tive summaries and permutation matrix represent two mostwidely used methods. However, all these methods have dif-ficulties with handling discrepant results that equally spreadin the matrix. A possible solution is to use the single bestevidence.

Conducting primary studies

In the context of medicines selection, the absence of reliableand transferable economic studies necessitates the conductof new studies.

Audiences and perspective

Typically, policy makers form the primary audience of eco-nomic studies, and health care sector perspective may beused as the primary perspective of study, supplemented withan analysis with the societal perspective (37).

Indications

Ideally, the study population should cover all that are pre-scribed with major and minor indications. Most often,economics studies focus on major registered drug indica-tions. Nevertheless, the selection of indications needs to bejustified.

Comparators

In the context of competing medications, three treatmentstrategies—including most widely existed practice, mini-mal practice/or doing-nothing practice, and the drugs itmight replace—may be included as comparators. In the pres-ence of large number of competing medicines, compara-tors should at least include a reference drug listed in theformulary, and the most widely used alternatives (includ-ing “do nothing” or “minimal practice”). The decision anduse of reference drug is usually based on decision-makers’perspective.



Time horizon

The choice of time horizon is typically made on the basis ofquestion of interest. A desirable time horizon is able to cap-ture important health outcomes and costs. Surrogate markersare often used in clinical studies, in which case modelingrepresents an alternative to acquire outcomes important topatients.

Study design

Five study designs can inform cost-effectiveness studies(33). Pragmatic trials provide reliable elements for assess-ing economic efficiency of medicines, and offer advantagesof achieving desirable internal and external validity, while itis associated with substantial costs and time. Piggy-backedstudies enjoy the high internal validity, but are susceptible tolow external validity. Probably, multicenter trial offers an al-ternative to addressing the drawback. Observational studiesoften yield “real-world” data, but are more prone to bias. Suchdesign is time and cost consuming. Retrospective databaseanalysis provides a relatively inexpensive and expedient ap-proach. Its major advantages include a wide and large pop-ulation over long time, and coverage of current practice. Itskey limitation is low internal validity. Modeling techniqueoffers a flexible and timely framework. However, integra-tion of data from various studies and populations mandatessophisticated expertise, and may render biased results.

Health outcomes

Study outcomes of cost-effectiveness studies include naturalclinical outcomes and health-related adjusted quality of life.The single best natural outcomes may be based on a synthe-sis of multicenter trials, secondary to observational studiesand expert opinions (34). In the case of cost-utility analyses,the approaches to valuing outcomes differ (35, 36). Standardgamble and time trade-off may be desirable through a deliv-ery of patient survey. Quality of life instruments may also beused for valuing utility (37,38).

Valuing costs

The approaches to valuing costs differ (39), and can be con-ducted in four steps, including identification of cost compo-nents, measurement of resources use, identification of unitcosts, and valuing resources use. Identification of cost com-ponents is usually associated with the choice of perspective.In the context of health care sector, direct medical costs aremeasured, and resources use is collected associated with dis-ease management. The impact of each cost-generating eventon the total costs differs. We suggest documenting details ofthose events that have significant contributions.

Four approaches are used to measure resources use, thelevel of measurement ranging from primary data to expertopinions. The choice of approach is based on study per-spective, contribution to total costs, availability of data, andbalance between internal and external (40). We recommendthat that resources use of significant contributions to the totalcosts should be measured. Expert opinions are the last resortof measuring resources use.

Unit costs can be obtained through national healthcareprice reports, charges or tariffs, published data, and calcu-lation of unit costs. National Healthcare Price Report rep-resents a common source of unit costs, and may be goodrepresentation of price from societal perspective. Chargeand tariff is potentially useful to value price from health-care sector perspective. Naive charge is likely to over rep-resent unit costs, and an adjustment for charge-to-priceratio may be desirable. Published data, although readilyaccessible, may be invalid. In any case, the choice ofunit costs is associated with measurement of resourcesuse (41).

Discounting

Most analysts agree both the health outcomes and costs bediscounted at the same rate. A discount rate of 5% can beused as the reference case, and a range of 3%–6% used forsensitivity analysis.

Dealing with uncertainty

The approaches to handling uncertainty depends on the na-ture of data. A CI around cost-effectiveness ratios may beestimated, using one of the following methods including theFiedler’s theorem method, the Taylor series or delta method,nonparametric bootstrapping estimation techniques. A sen-sitivity analysis is desirable, and the choice of types of sen-sitivity analyses depends. One rule of thumb is to conductsensitivity analysis on those variables that have importantinfluence on results. The development of cost-effectivenessratios comparing cost with equality of life years gained willbecome an important factor in government price negotiationover what it will pay manufacturers whose drugs are listedon the Essential Medicines List.

Data analysis and presentation

Information about resources use, unit costs, total costs, andhealth outcomes should be fully reported. In the presence ofa dominant strategy, there is no need to report incrementalratio. The incremental ratio is reported, otherwise. The resultsof analyses should be presented in disaggregated manner, sothat the results could be recalculated and examined whenadapting to different perspectives.


Economic evidence for selection of medicines X Sun and TA Faunce

Conclusion

The development of a Chinese Essential Medicines List inwhich the cost of listed medicines is subsidized is an impor-tant public health initiative. Value for government expendi-ture on such a list will be enhanced if pharmaco-economicexperts recommend listing after rigorous cost-effectivenessanalysis of new medications against existing products andinitiate a related price negotiation with the manufacturer.

Acknowledgment

This study is funded by NSFC (70703025).

References

1. Health statistics from the Ministry of Health, China. Availablefrom: URL: http://61.49.18.68/statistics/digest04/s34.htm andhttp://61.49.18.68/statistics/digest04/s35.htm (accessed 20December 2005).

2. Li SL, Cao LY, Xiao AL,Yan L, Lan F. Evidence-basedevaluation of essential medicines. Chinese Pharmaceutical

Issues 2003; 17(6): 337–9 (in Chinese).3. The Centre for Drug Re-evaluation, State Food and Drug

Administration. The evaluation process of the essentialmedicines. Available from: URL:http://www.cdr.gov.cn/work-prog/1.htm (accessed 11 March2006).

4. Sun X, Cao LY, Chen XZ, Lan F, Li X, Xiao AL, et al. Fournon-sedating antihistamines for allergic diseases: healthtechnology assessment (protocol). Chinese Journal of

Evidence-based Medicine 2003; 3(1): 35–40.5. Li YP, Sun X. An evidence-based approach to evaluation and

selection of post-marketing medicines. In:Zheng XY, Liao QJ,eds. Annual Advance of Pharmacy. Beijing: Chinese MedicalScience and Technology Press, 2004.

6. ISPOR Chinese Medical Doctor AssociationPharmacoeconomics Chapter (ISPOR Chinese MDA-PEChapter). China Pharmacoeconomic Evaluation Guidelines (theSolicitation Draft), Beijing, China, 2006.

7. Commonwealth Department of Human Services and Health.Guidelines for the Pharmaceutical Industry on Preparation ofSubmissions to the Pharmaceutical Benefits AdvisoryCommittee. Commonwealth Department of Human Services andHealth, Canberra, 1995.

8. Garattini L, Grilli R, Scopelliti D, Mantovani L. A proposal forItalian guidelines in pharmacoeconomics. Pharmacoeconomics

1995; 7(1): 1–6.9. Canadian Coordinating Office for Health Technology

Assessment. Guidelines for Economic Evaluation ofPharmaceuticals, Canadian Coordinating Office for HealthTechnology Assessment, Ottawa, 1997.

10. Oostenbrink JB, KoopmanschapMA, Rutten FFH.Standardization of Costs: The Dutch Manual for Costing inEconomic Evaluations. Pharmacoeconomics 2002; 20(7):443–54.

11. Canadian Coordinating Office for Health TechnologyAssessment. A guidance document for the costing process(Version 1.0), Canadian Coordinating Office for HealthTechnology Assessment, Ottawa, 1996.

12. Brady BK, McGahan L, Skidmore B. Systematic review ofeconomic evidence on stroke rehabilitation services.International Journal of Technology Assessment in Health Care

2005; 21(1): 15–21.13. Holloway RG, Benesch CG, Rahilly CR, Courtright CE. A

systematic review of cost-effectiveness research of strokeevaluation and treatment. Stroke 1999; 30(7): 1340–9.

14. Siebert U, Bornschein B, Walbert T, Dodel RC. Systematicassessment of decision models in Parkinson’s disease. Value

Health 2004; 7(5): 610–26.15. Jefferson T, Demicheli V. Quality of economic evaluations in

health care: it is time for action to ensure higher methodologicalquality. British Medical Journal 2002; 324(7333): 313–4.

16. Udvarhelyi IS, Colditz GA, Rai A, Epstein AM.Cost-effectiveness and cost-benefit analyses in the medicalliterature. Are methods being used correctly?. Annals of Internal

Medicine 1992; 116(3): 238–44.17. Jefferson T, Demicheli V. Is vaccination against hepatitis B

efficient? A review of world literature. Health Economics 1994;3(1): 25–37.

18. Drummond MF, Jefferson TO. Guidelines for authors and peerreviewers of economic submissions to the BMJ. British Medical

Journal 1996; 313(7052): 275–83.19. Jefferson T, Demicheli V, Vale L. Quality of systematic reviews

of economic evaluations in health care. Journal of the American

Medical Association 2002; 287: 2809–12.20. Evers S, Goossens M, de Vet H, van Tulder M, Ament A.

Criteria list for assessment of methodological quality ofeconomic evaluations: consensus on Health Economic Criteria.International Journal of Technology Assessment in Health Care

2005; 21(2): 240–5.21. Chiou CF, Hay JW, Wallace JF, Bloom BS, Neumann PJ,

Sullivan SD, et al. Development and validation of a gradingsystem for the quality of cost-effectiveness studies. Medical

Care 2003; 41(1): 32–44.22. Spiegel BM, Targownik LE, Kanwal F, Derosa V, Dulai GS,

Gralnek IM, et al. The quality of published health economicanalyses in digestive diseases: a systematic review andquantitative appraisal. Gastroenterology 2004; 127(2):403–11.

23. Gonzalez-Perez JG. Developing a scoring system to qualityassess economic evaluations. European Journal of Health

Economics 2002; 3(2): 131–6.24. Sculpher MJ, Pang FS, Manca A, Drummond MF, Golder S,

Urdahl H, et al. Generalisability in economic evaluation studiesin healthcare: a review and case studies. Health Technology

Assessment 2004; 8(49): iii–iv, 1–192.25. Welte R, Feenstra T, Jager H, Leidl R. A decision chart for

assessing and improving the transferability of economicevaluation results between countries. Pharmacoeconomics 2004;22(13): 857–76.



26. Barbieri M, Drummond M, Willke R, Chancellor J, Jolain B,Towse A. Variability of cost-effectiveness estimates forpharmaceuticals in Western Europe: lessons for inferringgeneralizability. Value Health 2005; 8(1): 10–23.

27. Greiner W, Schofski O, Graf v. d., Schulenburg JM, et al. Thetransferability of international economic health-economic resultsto national study questions. HEPAC 2000; 1: 94–102.

28. Nixon J, Khan KS, Kleijnen J. Summarising economicevaluations in systematic reviews: a new approach. British

Medical Journal 2001; 322(7302): 1596–8.29. NHS Centre for Reviews and Dissemination. Undertaking

systematic reviews of research on effectiveness. CRD’s

Guidance for Carrying Out or Commissioning Reviews. 2nd ed.York: NHS Centre for Reviews and Dissemination, 2000.

30. Black WC. The CE plane: a graphic representation ofcosteffectiveness. Medical Decision Making 1990; 10(3):212–4.

31. Birch S, Gaffni A. Cost-effectiveness and cost utility analyses:methods for the non-economic evaluation of healthcareprograms and how we can do better. In:Geisler E, Heller O, eds.Managing Technology in Healthcare. Norwell, MA: KluwerAcademic, 1996.

32. Drummond MF, O’Brien B, Stoddart GL. Methods for theeconomic evaluation of health care programs. 2nd ed. Oxford:Oxford University Press, 1997.

33. Gold M, Siegel J, Russell L, Weinstein M, editors. Cost-

effectiveness in health and medicine. New York: OxfordUniversity Press, 1996.

34. Levels of Evidence – March 2002. Evidence-Based on Call.Available from: URL: http://www.eboncall.org/content/levels.html

35. Arnesen T, Trommald M. Are QALYs based on time trade-offcomparable? A systematic review of TTO methodologies.Health Economics 2005; 14(1): 39–53

36. Dolan P. Valuing health-related quality of life: issues andcontroversies. Pharmacoeconomics 1999; 15(2): 119–27.

37. Bala MV, Wood LL, Zarkin GA, Norton EC, Gafni A, O’BrienB. Valuing outcomes in health care: a comparison of willingnessto pay and quality-adjusted Life-years. Journal of Clinical

Epidemiology 1998; 51(8): 667–76.38. Perez DJ, McGee R, Campbell AV, Christensen EA, Williams S.

A comparison of time trade-off and quality of life measures inpatients with advanced cancer. Quality of Life Research 1997;6(2): 133–8.

39. Jacobs P, Ohinmaa A, Brady B. Providing systematic guidancein pharmacoeconomic guidelines for analysing costs.Pharmacoeconomics 2005; 23(2): 143–53.

40. Jacobs P, Baladi J-F. Biases in cost measurement for economicevaluation studies in health care. Health Economics 1996; 5(6):525–9.

41. Horngren CT. Cost accounting. 5th ed. Englewood Cliffs, NJ:Prentice-Hall, 1987.


Education

J.1756 5391.2010.01087.x