Avoiding waste in research:the role of public involvement

Iain ChalmersCoordinator, James Lind Initiative

‘Putting people first in research’INVOLVE Conference

Nottingham, 13 November 2012

The skeletons in academic medicine’s cupboards

Low priority questions addressed

Important outcomes not assessed

Clinicians and patients not involved in setting research agendas

Questions relevantto clinicians &

patients?

Over 50% studies designed without reference to systematic reviews of existing evidence

Over 50% of studies fail to take adequate steps to reduce biases, e.g. unconcealed treatment allocation

Appropriate design and methods?

Over 50% of studies never published in full

Biased under-reporting of studies with disappointing results

Accessible full publication?

Over 30% of trial interventions not sufficiently described

Over 50% of planned study outcomes not reported

Most new research not interpreted in the context of systematic assessment of other relevant evidence

Unbiased and usable report?

50%

85% Research waste = over $85 Billion / year

50%

50%

Low priority questions addressed

Important outcomes not assessed

Clinicians and patients not involved in setting research agendas

Questions relevantto clinicians &

patients?

Over 50% studies designed without reference to systematic reviews of existing evidence

Over 50% of studies fail to take adequate steps to reduce biases, e.g. unconcealed treatment allocation

Appropriate design and methods?

Over 50% of studies never published in full

Biased under-reporting of studies with disappointing results

Accessible full publication?

Over 30% of trial interventions not sufficiently described

Over 50% of planned study outcomes not reported

Most new research not interpreted in the context of systematic assessment of other relevant evidence

Unbiased and usable report?

50%

85% Research waste = over $85 Billion / year

50%

50%

Mismatch of patients’ and researchers’ priorities for osteoarthritis of the knee

Tallon, Chard and Dieppe. Lancet, 2000.

Priority treatment outcome from a survey of patients with rheumatoid arthritis was not pain

Priority treatment outcome from a survey of patients with rheumatoid arthritis was not pain

It was fatigue

New Healthtalkonline

interview study

A chance to share views/experiences of patient and public involvement in

research

If you’d like to contribute, contact louise.locock@phc.ox.ac.uk

01865 289303

How can patients and the public help to reduce

waste in research?

Chalmers I. Lancet 2000;356:774

Conclusions of a 2002 study commissioned by INVOLVE

“There was widespread support for the concept of a publicly available resource containing information about ongoing clinical trials designed for use by potential participants.”

Campbell M, Entwistle V, Langston A, Skea Z (2002). Scoping study to explore the most appropriate way to produce and disseminate information on the quality of randomised controlled trials for potential participants. http://www.invo.org.uk/pdfs/pact_reportfinal0305.pdf

INVOLVE 6TH National Conference

11 November 2008

The UK Clinical Trials Gateway: should provide information

about ongoing clinical research in the NHS

Iain ChalmersJames Lind Initiative

For every ongoing trial being conducted within the NHS, the UK Clinical Trials Gateway should aim to provide access to: a lay summary the patient information sheet the WHO 20-item dataset the protocol, with links to the systematic review(s) showing why the trial is needed the trial website (if one exists)

Reliable, user-friendly information about specific ongoing clinical trials is

still NOT generally available

What is the James Lind Initiative and what has it

done to try to address some of these problems?

2003-The James Lind Initiative

Funded by the National Institute of Health Research and the Medical Research Council

“to promote acknowledgement of uncertainties about the effects of treatments, and

research to address them.”

Programme of work of The James Lind Initiative

1. Identifying and publishing uncertainties about the effects of treatments: UK Database of Uncertainties about the Effects of Treatments

2. Identifying patients’ and clinicians’ shared priorities for research about the effects of treatments: James Lind Alliance

3. Explaining and illustrating the development of fair tests of treatments in health care: James Lind Library and Testing Treatments

The UK Database of Uncertainties about the Effects of Treatments

Established to publish uncertainties about the effects of treatments which cannot currently be answered by referring to relevant and reliable, up-to-date systematic reviews of existing

research evidence

Mark FentonEditor, DUETs

The UK DUETs Development Team

Hazim TimimiUpdate Software

UK DUETs draws on three main sources

•Patients', carers' and clinicians' unanswered

questions about the effects of treatments

•Research recommendations in reports of

systematic reviews and clinical guidelines

•Ongoing research, both systematic reviews in

preparation and new 'primary' studies

Programme of work of The James Lind Initiative

1. Identifying and publishing uncertainties about the effects of treatments: UK Database of Uncertainties about the Effects of Treatments

2. Identifying patients’ and clinicians’ shared priorities for research about the effects of treatments: James Lind Alliance

3. Explaining and illustrating the development of fair tests of treatments in health care: James Lind Library and Testing Treatments

Established in 2004to see if it would work

Iain Chalmers (James Lind Library)John Scadding (Royal Society of Medicine)

Nick Partridge (INVOLVE)

To increase the focus of the therapeutic research agenda on questions and priorities shared by patients and clinicians.

To promote Priority Setting Partnerships involving patients and clinicians to identify and promote their shared priorities for therapeutic research.

To increase general awareness and understanding of the need to refocus the therapeutic research agenda.

Sally CroweChair, JLA Monitoring & Implementation Group

Lester FirkinsChair, JLA Strategy

& Development Group

Patricia AtkinsonAdministrator,JLA Secretariat

Katherine CowanEditor, JLA Guidebook

Involving patients, carers and clinicians in research priority setting

The JLA’s principles•Inclusive

•Balance of perspectives•Accessible to all

•Supportive•Recognising a range of capacities and skills

•Transparent and democratic•Data sharing•Agreed protocol•Declaration of interests•Neutral facilitation •Communication and feedback

JLA Priority Setting Partnerships

Completed Current •Asthma•Urinary incontinence•Vitiligo•Prostate cancer•Schizophrenia•Type 1 diabetes•ENT aspects of balance•Life after stroke•Eczema•Tinnitus•Cleft lip and palate

•Acne•Childhood disability •Dementia•Dialysis•Head and neck cancer•Inflammatory bowel disease•Lyme disease•Multiple sclerosis •Pressure ulcers•Pre-term birth•Sight loss and vision

Stage 2

Agree shortlist of highest priority uncertainties

Notify highest priority uncertaintiesto research funders

Stage 3

Stage 5

Derive ‘indicative uncertainties’

Identifying and prioritising uncertainties about the effects of treatment

Stage 1

Publish ‘indicative uncertainties’ in UK DUETs

Stage 4

Prioritize uncertainties,James Lind Alliance

Priority Setting Partnerships

Harvest ‘raw uncertainties’ 19,551

2555

1678

264

Research priority themes [across asthma,

incontinence, vitiligo, eczema, stroke, prostate cancer, schizophrenia, aspects of balance, and type 1 diabetes]

• Assessment of long-term effects (wanted and unwanted) of treatments

• Assessment of safety and adverse effects of treatments

• Assessment of complementary and non-prescribed treatments

• Assessment of strategies to improve early diagnosis and treatments, and harmonisation of practice

• Assessment of the effectiveness and safety of self-care

For more information…www.lindalliance.org – become an Affiliate

www.JLAguidebook.org – practical guidance

@LindAlliance

From 1 April 2013, the JLA will be incorporated in the NIHR

Evaluation, Trials and Studies Coordinating Centre (NETSCC)

Tom Kenny, Pamela Young, Sarah Fryett

Low priority questions addressed

Important outcomes not assessed

Clinicians and patients not involved in setting research agendas

Questions relevantto clinicians &

patients?

Over 50% studies designed without reference to systematic reviews of existing evidence

Over 50% of studies fail to take adequate steps to reduce biases, e.g. unconcealed treatment allocation

Appropriate design and methods?

Over 50% of studies never published in full

Biased under-reporting of studies with disappointing results

Accessible full publication?

Over 30% of trial interventions not sufficiently described

Over 50% of planned study outcomes not reported

Most new research not interpreted in the context of systematic assessment of other relevant evidence

Unbiased and usable report?

50%

85% Research waste = over $85 Billion / year

50%

50%

Publication (2007) after registration (1999)

Ross JS, Mulvey GK, Hines EM, Nissen SE, Krumholz HM (2009). Trial publication after registration in ClinicalTrials.Gov: a cross-sectional analysis. PLoS Med 6(9): e1000144.

Country

Size

Phase

Funder

“Studies that report positive or significant resultsare more likely to be published and outcomes that are statistically significant have higher oddsof being fully reported.”

PLoS ONE, August 2008;3:e3081

Alessandro Liberati

Because research results have not been made public…

…patients have suffered and died unnecessarily and resources for health care and health research have been wasted.

TGN1412

TGN 1412

Why is medical academia content to acquiesce in biased

under-reporting of research?

What is the position of the Academy of Medical Sciences?

2006 letter to Prof John Bell, President, Academy of Medical Sciences

What should be done?

The public needs to be made aware of how the resources they provide for research are being wasted.

What should be done?

The public needs to be made aware of how the resources they provide for research are being wasted.

The public needs to hold the research community to account, and be critically involved in research, from agenda setting to dissemination of results.

Programme of work of The James Lind Initiative

1. Identifying and publishing uncertainties about the effects of treatments: UK Database of Uncertainties about the Effects of Treatments

2. Identifying patients’ and clinicians’ shared priorities for research about the effects of treatments: James Lind Alliance

3. Explaining and illustrating the development of fair tests of treatments in health care: James Lind Library and Testing Treatments

www.jameslindlibrary.org

2006 2011

Write your suggestions on the postcards in your

conference packs.

Pin them up on the ‘Burning Issues’

noticeboard.

Promote research on the effects of treatments…

…but only if it meets scientific and ethical principles

Promote research on the effects of treatments…

www.testingtreatments.org

“Bad Science introduces the basic scientific principles to help everyone become a more effective bullshit

detector.”