Medical thinking or clinical action ?

Medical thinking or clinical action ?

Dr Jeremy Wyatt FRCP DM

Professor of health informatics, University of Dundee, Scotland

[email protected]

Acknowledgements: Prof Susan Michie, UCL

The problem

• The NHS costs £80Bn per annum; there are severe workforce pressures

• For some tests and therapies, we know enough about what helps patients to recommend that their use should be reduced or increased

• Despite this evidence, there is much geographical variation in clinical practice and patient outcomes

How can we narrow the gap between what clinicians know and what they do ?

What evidence do we have ?

Source: 2329 therapies reviewed for Clinical Evidence issue 12, 2005

Known to benefit patients

15%

Likely to do nothing / harm

4%

Likely to benefit21%

Benefit / harm trade off

7%

Unlikely to benefit

5%

Not known48%

Why medical thinking ?

• In the UK in 2006, more prescribers are non-medical than medical: nurses, pharmacists

• Patients & carers responsible for key health related decisions: – Are my symptoms serious ?

– Who to go to ?

– Do I believe them ?

– Shall I take this therapy ?

– Is it better now ?

• Strong pressures toward multi-skilled clinical practitioners, shared decision making, team based care

• Clinical, not medical, is what matters

Why clinical thinking ?

• Mismatches between what people know, remember, think, decide, intend, say and do

• Limited skills, confidence, self efficacy to put decision into action

• Pressures from peers, patients, NHS Trusts, NICE…• Action slips• Limited resources to carry out action: staff, time,

equipment…

Result: gulf between the good intentions resulting from clinical thinking and actual clinical actions

Practice guidelines

Evidence

Practice guideline

recommendation 1Expert opinions

recommendation 2

• Cost of tests, drugs

• Health system reality

• Society’s values• Lobbying from

patients, industry

Constraints

recommendation 3

What can we do with guidelines ?

Guideline

Clinician

Library

provide access

searches

Newsletter

disseminate

receives

Innovation method

use as basis of

participates in

Clinical practice innovation

Aim: to narrow the gap between what we know and do

Synonyms: implementation of research, behaviour change, getting research into practice, change management

At least 63 methods available, including: – Paper / computer reminders– Audit and feedback– Patient information leaflets– Decision support systems– Outreach visits– Opinion leaders...

Impact of outreach visits on care given to 4500 pregnant women

25 Eligible obstetric units

Baseline data collection

Randomisation

Follow-up data collection

Follow-up data collection

CCPC: Cochrane module on pregnancy & childbirth (Wyatt, BMJ 1998)

Outreach visit:•Guideline feedback•Discuss EBM, give video, CCPC, train

•Discuss innovation methods, give slides

control units

outreach units

Control clinicians

Innovation clinicians

Difference in practices

Difference in outcomes

Recommendation

Innovation method

Summarising evidence on innovation methods

Evidence

about

innovation

methods

Systematic review of innovation studies

Control clinicians




Recommendation

Innovation method

Control clinicians




Recommendation

Innovation method

Example: review of manual paper reminders

Definition: reminder, decision support or audit & feedback ?

Finding studies: 324, spread across 101 journals

Study quality: only 22 RCTs of 82 relevant studies of manual reminders

Other issues:– 17 studies from USA (largely fee-for-service)– 3 “positive” studies had unit of analysis errors & 1 showed 2%

change - so 10 clinically positive studies– Poor reporting – able to examine only 5 of 10 effect modifiers of

interest– Success rate varied by reminder type and targeted clinical

practice; need meta-regression

(Wyatt et al 2001)

Cochrane EPOC review group

Founded 1994 by JW as “Cochrane Collaboration on Behaviour Change” (!)

Now “Effective Practice & Organisation of Care” group

Base in Canada: Google “Cochrane epoc”

Editors: Jeremy Grimshaw, Andy Oxman, Merrick Zwarenstein, Lisa Bero

Output: 26 completed reviews, further 17 in progress

How likely are disseminated guidelines etc. to improve clinical practice ?

Systematic reviews Improved practices

Evidence summaries(Oxman CMAJ 1995)

11% 1/9

Printed educational materials(Freemantle CL 1997)

14% 2/14

Educational materials(Davis JAMA 1995)

36% 4/11

Guidelines in general practice(Wensing BJGP 1998)

11% 2/18

Average (duplicate studies): 17% 9/52

How likely are lectures to improve practice ?

Improved practices

Lectures only 0 0/4

Interactive sessions 67% 4/6

Mixed interactive / lecture sessions 71% 5/7

Single session 28% 2/7

Multiple sessions 70% 7/10

Students choose session content 80% 4/5

Overall 53% 9/17

(Davis, JAMA 1999)

How likely are clinical innovation methods to improve practice ?

Improved practices Paper reminders Wyatt 2001 46% 10/22

Audit & feedback Thomson CL ‘98 62% 24/39

Computer decision support Garg JAMA ‘05 64% 62/97

Patient materials Davis JAMA ‘95 78% 7/9

Opinion leaders Thomson CL ‘98 86% 6/7

Outreach visits Thomson CL ‘98 94% 15/16

Computer drug dosing Walton BMJ ‘99 100% 17/17

GP computing Sullivan BMJ ‘95 100% 21/21

Telemedicine Currell CL 2000 ? ?/7

Overall 73% 143/196

How likely are different methods to improve patient outcomes ?

Improved outcomes

Outreach visits (94%) Thomson CL ‘98 0% 0/1

Audit & feedback (62%) Thomson CL ‘98 18% 1/6

Paper reminders (46%) Wyatt 2001 18% 1/6

Telemedicine (?) Currell CL 2000 20% 1/5

GP computing (100%) Sullivan BMJ ‘95 33% 1/3

Opinion leaders (86%) Thomson CL ‘98 33% 1/3

Computer decision support (66%) Garg JAMA ‘05 13% 7/52

Computer drug dosing (100%) Walton BMJ ‘99 83% 5/6

Patient materials (78%) Davis JAMA ‘95 ? ?/9

Overall (73%) 19% 17/91

Low numbers !

Which clinical practices are easiest to improve ?

Improved practices

Procedural skills 25% 1/4

Diagnosis 50% 2/4

Disease management 55% 32/58

Ordering tests, procedures 71% 17/24

Preventive care 74% 40/54

Prescribing 80% 11/14

Overall 65% 103/158

Davis D. JAMA 1995; 274:700-5

The gulf between recommendations and outcomes

Improved clinical practice

Improved patient outcome

Guideline recommendation

Clinician wants to improve their practice

motivation, incentives

Clinician able to improve their practice

resources, skills, support

commitment, good memory

Clinician knows recommendation

knowledge access, dissemination

Matching innovation methods to barriers


Clinician knows recommendation

knowledge access, dissemination

Clinician wants to improve their practice

motivation, incentives

Clinician able to improve their practice

resources, skills, support

commitment, good memory

Improved clinical practice

Not knowing Outreach visit, TM

Not caring Audit and feedback

No drugs, equipment Provide drugs, equipment

No skills, support Train with opinion leader, TM

Fear of consequences Target peer group, patients

Forgetting, action slips Reminder, decision support

Guideline recommendation

Barrier to change Innovation method

Should we bother with barriers ?

Improved practices

Innovators ignored barriers 42% 5/12

Checked literature 53% 18/34

Obtained local concensus 58% 26/45

Checked national guideline 61% 25/41

Local study to check barriers 90% 25/28

Davis D. JAMA 1995; 274:700-5

How long might innovation take ?

Time

Uptake %

100%

0

Rogers EM. The diffusion of innovations. New York Free Press 1993

innovators

early adopters

late adopters

laggards

What might make change more likely ?

The change is compatible with participant needs, norms, beliefs

The change is relevant to user’s work, provides an advantage for participants

Benefits can be easily observed, limited risk

The change is simple, can be broken down into steps, has a core with fuzzy boundaries

The change can be re-invented locally

The change is easy to try out – no infrastructure needed

All necessary knowledge or support available

(Rogers ’93, Greenhalgh Millbank Q 2005)

Relevant theories

1. PRECEDE (Green L, 1988):• Predispose the person to change• Enable the change• Reinforce the change

2. Theory of planned behaviour (Ajzen I, 1991)

Theory of planned behaviour

Human action is guided by three considerations:

• Behavioural beliefs about the likely outcomes of the behaviour and evaluations of these outcomes

• Normative beliefs about the expectations of others and motivation to comply with these

• Control beliefs about factors that may facilitate or impede performance of the behaviour and perceived behavioural control

In combination, these lead to intention to perform the behaviour in question

Intention assumed to be the immediate antecedent of behaviour. Given a sufficient degree of actual control over the behaviour, people

are expected to carry out intentions when the opportunity arises.

Ajzen, I. (1991). The theory of planned behavior. Organizational Behavior and Human Decision Processes, 50, 179-211.

Relevant psychological constructs

Consensus study of 62 psychologists etc.:1. Knowledge2. Skills3. Professional role and identity4. Beliefs about capabilities 5. Beliefs about consequences6. Motivation and goals7. Memory, attention and decision processes8. Environmental context and resources9. Social influences 10. Emotion11. Action plans12. Nature of the behaviour

Michie S, Johnston M, et al (2005) Making psychological theory useful for implementing evidence based practice: a consensus approach, Quality and Safety in Health Care, 14, 26-33

Wording of recommendations

The challenge is to ensure: • Comprehension – understanding what needs be done• Recall – remembering what to do in the relevant context• Planning of this behaviour• Performance of behaviour

Specificity of instructions is associated with attaining goals (Locke and Latham, 1990), so specify:

• Who should do • What• How• Where, and• WhenUse “if-then" plans and active verbs (do) Avoid general exhortations (should), passive descriptions (may)

Michie, S. and Johnston, M. Changing clinical behaviour by making guidelines specific. BMJ 2004, 328: 343–5

Evidence about specific recommendation wording

Grol’s study: 61 GPs & 47 recommendations from 10 national Dutch clinical guidelines

GPs followed recommendations: – on 2/3 of occasions when it was concrete and specific

– on 1/3 of occasions when it was vague and non-specific

NB. Specificity accounted for only 17% of the variance

Grol et al. Attributes of clinical guidelines that influence use of guidelines in general practice: observational study. BMJ 1998, 317:858-61

Some research issuesBarriers to change:

– Are the methods to elicit them reliable, valid ?– Are some barriers more important than others ?– Are barriers generic, or must we always tailor innovation ?

Practice innovation methods:– Is there evidence to match methods to barriers ?– How can psychological theory help in designing methods ?– Are some innovation methods more effective in specific:

• combinations ?• clinical practices or settings ?• professional groups ?

General: – Can we develop a valid “intention to implement” scale ?– Can an “innovation toolkit” & web site help clinicians change ?

Make the change

UCL KMC clinical practice innovation model

Isinnovationneeded ?

Audit local practice

Analyse barriers to change

Select & apply innovation method(s)

No

Choose a clinical practice

recommendation

Identify & engage all participants

Yes

Wait for a while

Loop A

Loop B

Examples of participants

GP prescribing: GPs, patients, pharmacists, nursing home staff…

Lab test ordering: junior doctors, senior doctors, medical schools, lab staff, patients, phlebotomists…

Cardiovascular risk reduction: patients, friends, relatives, manager of workplace / private gym / pub, local council, regional government, food industry…

Some practical conclusions

1. Guidelines do not themselves change practice

2. An innovation programme is needed:– Choose a clinical practice from the guideline that really does improve (or

worsen) patient outcomes– Obtain high-level support for an innovation programme

3. Implement the programme:– Carry out a careful local audit– Identify all participants (including opinion leaders)– Search for barriers to change– Select appropriate innovation method(s)– Monitor progress; consider a rigorous trial

4. Collaboration with psychologists, management scientists & ethnographers is likely to be useful

Some research questions

• How to use psychological theory to design and evaluate innovation methods ?

• Do different theories apply to increasing and decreasing actions ?

• Are barriers generic and enduring or person and episode specific ?

• How to validate theories eg. functional imaging• How much clinical variation depends on individual

cognition vs. team dynamics / environment ?

Does diagnosis matter ?

• At least 1/3 of encounters associated with long term illness – diagnosis made years ago

• Most advances in 20th century associated with quicker, more accurate diagnostic tests

• Challenging diagnoses rare in routine clinical practice• Challenges for health services are:

– which test in what order

– assembling patient data from multiple sources

– test interpretation

– monitoring of long term illness

– improving teamwork (work force)

– shifting services closer to home – self testing

One innovation method, or many ?

Improved practices

One method 60% 49/81

Two methods 64% 25/39

Three or more 79% 31/39

a) Indirect, between-study comparison (Davis, 1995):

b) Direct, within-study comparisons (Wensing, 1994): Multi-part interventions led to a larger effect in general practice than a single intervention in 4 / 10 RCTs

How long does the impact last ?

Thomson O’Brien. CL 1999

Few RCTs follow-up after audit & feedback stop:

3 months: both groups improved test usage (Martin A ‘80)

12 months: generic prescribing declined, but still better than control

group (Gehlbach ‘84)

14 months: improved management of cystitis (Norton P ‘85)

Conclusion: “There is insufficient data to clarify when the effects are most

likely to deteriorate after feedback stops”

How much does innovation cost and save ?

Cost of outreach visit: 1400 euro (1995 prices; Wyatt BMJ ‘98)

Cost to deliver paper reminders (Wyatt unpub.): – measured in 4 studies

– ranged from 10c to 75 euro per patient

– cost one sixth of the cost of pt. invitation letters for cervical screening

Savings from paper reminders: – cut inflation in asthma treatment costs to one third

– saved 1100 euro per inpatient (earlier discharge)

What might help guidelines succeed ?

Opinion survey of 1500 US internists - features they claim would cause them to comply (Hayward JGIM ‘96):

– Guidelines as short pamphlet: 86% in favour– Summary of supporting evidence: 85%– Benefits quantified: 77%– Endorsed by respected colleague (72%) or major organisation

(69%)

Study of actual practice - guideline features & actual use by 61 Dutch GPs (12 features, 12900 decisions - Grol BMJ ‘98):

– Positive correlation: recommendation was specific, uncontroversial, required no change in existing routine

– However, these accounted for only 17% of the variance

Systematic reviews of innovation studies

Special problems:– Innovation methods poorly defined, indexed, reported– Studies heterogeneous: different clinicians, settings, practices...– Few direct comparisons within the same study– Study designs poor, with bias and confounding (co-interventions,

Hawthorne Effect, contamination…)– Multiple measures of clinical practice - often subjective

Consequences:– Hard to identify studies, few studies eligible– Qualitative review (vote counting) usually more appropriate than

quantitative (meta-analysis)– Meta regression rarely possible: too many variables, too few

studies

From knowledge to outcome

External knowledge

Internalised knowledge

MotivationPerceived self efficacyClinical

decision

Opportunity

Distraction

Forgetting

Action slipsCognitive biases

Framing effects

Perception

Peer pressure

Tacit knowledge and skills

Action


Patient concordance

Documents

Medical thinking or clinical action ?