Comparative Effectiveness Research :Rethinking Therapeutic Evaluation in

Chronic Diseases

Ph Ravaud

Therapeutic Evaluation of Chronic diseases

• Today : mainly RCts and Meta-Analysis (one drug )

• Tomorrow

- RCTs

- Meta-analysis and Network Meta-analysis (all drugs for a specific disease)

- Observational Data

Too slow Too expensive Doesn’t answer many critical questions ( or

doesn’t answer questions relevant for physicians ( Short term , inadequate comparator, side effects)

Otherwise its great ( RCTs are the best way to obtain groups of patients comparable for known and unknown prognostic factors )

Applicability or generalizability of trials

• Patients included are not representative of the patients treated in usual care (trial patients are younger and with less co-morbidities)

• Setting is not representative (centers are highly selected)

• Treatments are evaluated mainly according to the principle “one size fits all”

Critical Knowledge Gaps

• The paradox

- 18,000 RCTs published each year

- more than 350,000 RCTs available

• Despite that available evidence remains limited or of poor quality

Much of Care Today is Not Based on Scientific Evidence

Robert Califf, IOM Meeting on Evidence-based Medicine, December 2007

Less than 20% of AHA/ACC heart disease management recommendations are based on a high level of evidence and over 40% are based on the lowest level of evidence AND proportionof recommendations with high evidence levels has not increased over time

From Meta-analysis to Network Meta-analysis

One treatment All treatments available for a disease

Thousands/yLess than 50/y

Network meta-analysis

Intervention C

Intervention A

Intervention E

Intervention F

Intervention B

Intervention D

Combining direct and indirect evidence

Biologic Treatment in Rheumatoid Arthritis: Ongoing Trials

-Only 5 head to Head trials - Ongoing Trials recruiting patients failing to respond to Methotrexate and with high disease activity

Cost of RCTs: an example

• 18,000 patients

• Total crf pages 1.8 millions

• Total crf variables 2.5 Billions

• Total number of queries 600,000

• Cost 700 millions Euros

• Treatment effects decrease over time , number of patients required mecanically increase

Unrealistic to expect head-to-head RCTs addressing all 2-by-2 comparisons

Number of competing interventions

Number of pair-wise comparisons

4 65 106 157 218 289 3610 4511 5512 66

• It is an unrealistic expectation that we will have randomized trials for every intervention and its combinations in every patient subgroup ( for example if for a disease we have 20 different treatment options and 3 different subgroups of patients, we need theoritically at least 470 head to head trials !)

•We need Effectiveness evidence in a timely manner. Randomized trials take time to conduct

•Therefore, 85% of the CER evidence is from non-experimental data!*

* Academy Health Report June 2009

Transparency of clinical trials

• Reporting guidelines

• Clinicaltrial.gov 2005

• FDA amendment act 2007

Consort Extensions

• Too many extensions

• Too many reporting guidelines

• Editors do not really implement the guidelines

• Quality of reporting remains poor

Trial registration mandatory since 2005( International Committee of Medical Journal Editors)

15

16

Impact of dissemination bias

Separate meta-analyses of the FDA and journal data sets show that the increase in effect size ranged from 11 to 69% for individual drugs and was 32% overall

Turner et al. NEJM 200816

Selective Dissemination Bias

FDA Amendment Act

• US Federal law enacted in 2007 mandates registration and results reporting for clinical trials of drugs , biological products and devices at clinicaltrial.gov

• Study sponsors or PI are requires to report summary results information within 1 year of completing data collection for teh prespecified primary outcome

21

Levels of “Transparency”

Zarin DA, Tse T.. Science. 2008 Mar 7;319(5868):1340-2.21