Are we using available evidence at the best? Nicola Magrini AIFA Spring Seminar 30 Marzo 2007

Are we using available evidence at the best?

Nicola Magrini

AIFA Spring Seminar30 Marzo 2007

Contents of the presentation

• Are regulatory agencies using available evidence at the best?

• Are journals and the scientific community using evidence at the best?

• How are we presenting available evidence to prescribers?

• Conclusions

Are regulatory agencies using available evidence at the best?

Surely NO

Why regulatory agencies are not using available evidence at the best

• Loose criteria for approval (scanty data or just a single trial sufficient for registration)

• Lack of transparency and lack of full access to existing information and knowledge

• Lack of laws and regulation protecting public interest giving full access in Europe (vs Freedom of Information Act in the US)

• False myths: GCP role and their usefulness (a solution worse than the problem?)

FDA approval process

S. Okie NEJM 2005

Almost never are now requested 2

indipendent pivotal studies

How is FDA faring?

Why regulatory agencies are not using available evidence at the best

• Loose criteria for approval (scanty data or just a single trial)

• Lack of transparency and lack of full access to existing information and knowledge

• Lack of laws and regulation protecting public interest giving full access in Europe (vs Freedom of Information Act in the US)

• False myths: GCP role and their usefulness (a solution worse than the problem?)

Lack of transparency/access to EMEA data: a recent example

• Torcetrapib, a nove antidyslipidemic drug (which also increases HDL levels) was shown to increase myocardial events in association with atorvastatin – FDA press release - trial not yet published

• Where are the data for a more detailed evaluation?

EMEA and … novel torcetrapib

New “anonymous” productEMEA - Scientific Advice Background Package

November 2006Pp 1-224

Lack of transparency/access to EMEA data: a recent example

• Torcetrapib, a nove antidylipidemic drug (which also increases HDL levels) was shown to increase myocardial events in association with atorvastatin – FDA press release - trial not yet published

• Similar drugs are currently evaluated: how should they be studied? Why the scientific community was not involved in this debate?

• Several phase III “Worldwide, Multicenter, Double-Blind, Randomized, Parallel, Placebo-Controlled Studies” are proposed: but only with surrogate (LDL cholesterol) short-term (maximum 1 year) end-points

• Why EMEA is not changing its guideline for evaluating cholesterol drugs effectiveness? Why is EMEA so little present in the scientific journals?

Researchers contest how (regulatory

agencies accept) new drugs are developed

Are journals and the scientific community using evidence

at the best?

Surely improving …

Available evidence and Journals:the achievements

• RCT registration (to reduce publication bias and outcome reporting bias)

• Improving post-publication criticism and the future of medical publishing (open access movement)

• Putting the latest clinical trials in the context of available evidence

• What this study adds / what is already known (BMJ)

• Speak badly of GCP

A great achievement to reduce publication and outcome reporting

bias: RCT registration


• RCT registration (against publication bias and outcome reporting bias)

• Improving post-publication criticism and the future of medical publishing (open access movement)




Papers and postpublication criticism

R. Horton, JAMA 2002

An example of postpublication criticism flaws

R. Horton, JAMA 2002

A structured format for discussion section?

R. Horton. JAMA 2002



• The limits of post-publication criticism and the future of medical publishing (open access movement)




RCT … improper without a systematic review

Lancet, July 9, 2005

An interesting option:the latest trial and its metanalysis Lancet trial (alteplase in stroke)

SITS-MOST, Lancet, 2007

BMJ (slight) improvement of latest trial and previous RCTs





• Improve study results presentation and what this study adds / what is already known (BMJ)


How to improve study results presentation (I):some issues (from BMJ editorial board 2004)

(Wilson T, Sackett D, Altman D)

• Researchers prefer relative measures• Clinical decision need absolute measures (how

many patients will benefit from the intervention)• A reader needs thrre types of information to make

sense of relative measures:– The context

– The results

– The implications

An example:• For the patients included in the present study

(baseline characteristics), the control group (receiving standard therapy …) showed an event rate of XX,X%, the tretament has produced a reduction in the event rate of YY,Y% (ARR), which correspnds to a NNT of ZZZ (with a 95% CI …) which means a relative risk reduction of VV% (RRR)

How to express benefits and risks (II) some issues (from BMJ editorial board 2004)

(Wilson T, Sackett D, Altman D)

• Context: meningococcal septicaemia has a high mortality. Despite intrevenous antibiotics about 10% die.

• Results: Wondermycin is better than penicillin and reduces rates by 20%

• Implications: The NNT for patients in ITU is 50: for every fifty people with meningococcal septicaemia in ITU treated with wondermycin in place of penicillin one more would survive.

• Cost implications: in the UK a course of wondermycin in ITu would cost £300 whilst penicillin would cost £15. This represents a £14.250 for every life saved.

How to improve study results presentation (III):

an example with all essential informations(Wilson T, Sackett D, Altman D: BMJ editorial board 2004)

How to put the latest trial in the

context of available

knowledge:

BMJ’sWIAK & WTSA







GCP: gold or bronze standard?

GCP: drop them?

Grimes DA et al. Lancet 2005

Are journals and the scientific community using evidence

at the best?

Surely improving … tough often below the optimal standard

GRADEGRADE

GGrades of rades of RRecommendation ecommendation

AAssessment, ssessment, DDevelopment and evelopment and

EEvaluationvaluation

What is the essential information What is the essential information to to do more good than harm?do more good than harm?

BMJ, 2004

GRADE – qualità delle evidenze GRADE – qualità delle evidenze e forza delle raccomandazioni e forza delle raccomandazioni

1.1. Choose and rate the clinically relevant Choose and rate the clinically relevant outcomesoutcomes

2.2. Evaluate the quality of evidence and define a Evaluate the quality of evidence and define a balance-sheetbalance-sheet

3.3. Evalute the benefit/risk ratio also considering:Evalute the benefit/risk ratio also considering: FeasibilityFeasibility AccessAccess CostsCosts

4.4. Define the strength of the recommendaion Define the strength of the recommendaion (strong or weak) (strong or weak)

GRADE – quality of evidenceGRADE – quality of evidencedimensions to be considereddimensions to be considered

Tipe of study Tipe of study

Quality of study:Quality of study:

Consistency: etherogeneity Consistency: etherogeneity or variability across studiesor variability across studies

Directness: transferabilty Directness: transferabilty (external validity) (external validity)

CommentsComments

Critical appraisal - often done Critical appraisal - often done through accompanying editorial through accompanying editorial and lettersand letters

Putting the latest trial into the Putting the latest trial into the context of available evidencecontext of available evidence

Discuss the problems of patients Discuss the problems of patients selection and transferabiltyselection and transferabilty

COMMENT:For most drug approved the available evidence base does not

allow an evaluation considering all the dimensions recommended by GRADE

Sono le linee-guida ATP-III evidence-based?

Lancet 20 january 2007

Methodological quality of cholesterol guideline: NCEP ATP-III• Multidisciplinary Panel• Systematic Review• Benefits and harms quantified for each

recommendation

• Grading of the recommendations• Transferability and implications

• Full disclosure of Conflict of interests

• NO• YES• YES/NO

• NO• YES/NO

• ?

LG ATP-III – dove sono le evidenze?

Lancet 20 january 2007

How are we presenting available evidence to prescribers?

Improving but … mostly in format for researchers

Important achievements … for researchersClinical evidence & Cochrane SR

Do we need more explicit formats? See Prescrire

Should we tell prescribers the whole story?

Is the New York Times more informative than scientific journals

about FDA decisions?Why is EMEA surrounded by

silence?

NEJM

Regulators and Newspapers: NYT and NEJMwho is leading the game?

NY Times 8 febbraio 2005




Conclusions

Conclusions (1)

• A single study should be put in the context of available knowledge

• Protocols should be registered and made accessible• Formats and ways of presenting RCT results to

prescribers should be encouraged • We should try to have a symmetry of information

between producers (researchers and industry) and final users (doctors and patients)

Conclusions (2)

Priorities• Improve/change present laws to improve

transparency and access• Improve our “knowledge brokering” capacity• Create network of indipendent researchers to

perform pragmatic, simple large scale RCTs in a real practice setting

• Scientific Journals and newspapers should have a common approach and similar objectives

Documents

Are we using available evidence at the best? Nicola Magrini AIFA Spring Seminar 30 Marzo 2007