1©2007 ECRI Institute

The Role of Comparative Effectiveness in Health Reform

Vivian H. Coates, Vice President

ECRI Institute

National Congress 0n Health Reform23rd September, 2008 Washington, DC

What ECRI does:

Evidence-based Practice Center since 1997AHRQ National Guideline Clearinghouse, National Quality

Measures ClearinghouseConduct health technology assessment and systematic

reviews, including comparative effectiveness reviewsMethodology research Support for clinical practice guideline development and

evidence-based policy making (coverage decisions, national and state policy decisions)

ECRI Organizational Experience40 years’ experience in laboratory evaluation of healthcare technology,

devices and equipment20 years’ experience in HTA and systematic review of drugs, devices,

procedures and other interventions and practicesHTA clients include: Hospitals, health plans, federal and state agencies,

drug and device manufacturers worldwideWorld Health Organization Collaborating CenterNonprofit heath services research agency300 full time staff in 4 offices (Phila, London, Dubai, KL)

Neither ECRI nor any of its staff has a financial interest in the sale of any medical technology. ECRI and its staff accept no

royalties, gifts, finder’s fees, or commissions from the medical device or pharmaceutical

industries and are not permitted to own stock in or undertake consulting work for

such industries.

Conflict-of-Interest Rules

What is Comparative Effectiveness?

No standard definition existsMultiple methodologiesVariable quality and sophistication of comparative effective

researchExpensive, resource-intensiveBut: essential for determining what works best in healthcare

5©2008 ECRI Institute

What is Comparative Effectiveness?

The comparison of one diagnostic or therapeutic intervention to one or more othersPrimary comparative effectiveness research: generation of clinical evidence on

the outcomes of one intervention in comparison to one or more others (head-to-head trials), prospective observational studies

Secondary comparative effectiveness research involves analyzing the results of multiple primary studies to reach conclusions to be drawn.

Secondary CER can also utilize analyses of outcomes or claims databases and modeling (Tunis, 2008)

Can observational studies be useful in CER?

Yes…becauseThey include “real-world” patients, higher risk

patients, patients with comorbiditiesHave external validity--results more

generalizable to routine clinical practice

Can observational studies be useful?

Yes….whenEvaluating infrequent treatment-related adverse

events/side effects

Large clinical registries may be best source for such data e.g. Stent thrombosis in patients with drug-eluting stents

or drug-related MI

Secondary CER typically includes:

Direct analysis of the results of multiple head-to-head studies Indirect analysis, in which the diagnostic or treatment options have

not been directly compared to each other in a trial Conclusions utilize inferences based on the relative effect of each

intervention to a specific comparison, often a placebo or sham intervention (eg, multiple drug or device trials each compared to a placebo or active comparator).

Combinations of direct and indirect analysis

9©2008 ECRI Institute

Secondary Comparative Effectiveness Research: Where to find itAHRQ EPC Evidence ReportsHealth technology assessments (HTA) from many sources (eg, ECRI)Systematic reviews, including meta-analysesPublished analyses of outcomes databasesPeer-reviewed journal articles, proprietary reports and monographs,

government reports (gray literature)Can be hard to find – no single place to look

10©2008 ECRI Institute

ECRI uses best available evidence approach in CE

Pragmatic, used when ideal studies desired (ie, best possible evidence) are not available

Uses highest available level of published evidence with appropriate caveats, discussion about reliability

Presents the current state of knowledge, even if it is inconclusive

Strength of Evidence

Domains:

ConsistencySize of Effect

Quality

Robustness Quantity

Strength of Evidence “Strength of evidence” characterizes a body of evidence, NOT

individual studiesA system for rating the stability and strength of medical evidence

Jonathan R Treadwell1 , Stephen J Tregear1 , James T Reston1 and Charles M Turkelson2 1ECRI Evidence-Based Practice Center and Health Technology Assessment Group, 5200 Butler Pike, Plymouth Meeting, Pennsylvania 19462, USA2American Academy of Orthopaedic Surgeons, 6300 North River Road, Rosemont, Illinois 60018, USABMC Medical Research Methodology 2006, 6:52 doi:10.1186/1471-2288-6-52Published 19 October 2006

Healthcare Reform: The Current Environment

Health technology changing faster than clinical practicePressure for information on the value of new approachesPressure on providers to deliver care cost effectivelyPressure on manufacturers to get reimbursement for new

products and servicesPressure from govt./payers for less costly servicesPressure from reformers to define a benefits packaged that

can be offered to all, including uninsured and underinsured

Implications of Improper Decision Making (No CER):

Inappropriate practices Diffusion of ineffective health technologyExcess technologyUnnecessary technologyDistorted consumer expectationsPatient harmIncreased costs without improved quality – diminished

value

Why Payers and Providers Need CER for Health Reform:

To help control wide variations in practices among clinicians locally, regionally and nationally

To control costs and better match resources with needsTo make informed purchase decisions about health

technologyTo reduce duplication of servicesTo ensure access to what works best

Current Challenges for Assessing Comparative Effectiveness of New Technologies:Hospitals/clinicians offering more and more technology-based servicesManufacturers developing more and more productsChoices are overwhelming, gaps in evidence hugeComparative effectiveness review/health technology assessment output

never enough Large number of topics means that some will be evaluated in lesser

depthHow to support decision-making when the evidence isn’t there?

Case Study: ECRI Comparative Effectiveness Review of Lumbar Artificial Disc vs Spinal Fusion

18©2007 ECRI Institute

Artificial Intervertebral Discs

Insufficient evidence to establish the long-term safety and efficacy of lumbar artificial disc replacement (expected life: 20 years)

2 RCTs flawed (eg, failure to use intent-to-treat, only 2 year follow up))

Patients eligible for lumbar artificial disc replacement only a subset of patients who are candidates for spinal fusion surgery (ie, many contraindications)

Revisions/replacement operations for patients with complications are more risky, costly, and debilitating than spinal fusion

Implications for Evidence-based Policy:

For Charite and other artificial intervertebral discs, need to wait for the results of several ongoing trials (5 year follow up)

Comparative effectiveness analysis limited by available dataArtificial intervertebral discs may be eclipsed by a less invasive

newer procedure: lumbar disc nucleus replacement (if the evidence materializes)

Challenges for Health Reform:Many of us already do CER – there is demand and capabilityNeed to support ramped-up output – we salute Gail Wilensky!But – strong countervailing forces and vested interests (eg, manufacturers

of products already on the market have no incentive to do head-to-head trials)

To what extent will comparative effectiveness be encouraged as part of health reform?

Is there really the will to provide what CER findings say should be offered?

If not, can health reform be achieved in our society?

21©2008 ECRI Institute