Searching for ‘do not do’ recommendations from NICE

guidance: a pilot study

Health Technology Assessment international (HTAi) Annual Meeting26 June 2012: 14.00-15.30

(Abstract ID: 606)

Presenter: Mendwas Dzingina

Authors: Moni Choudhury, Tarang Sharma, Sarah Garner, Grant Hill-Cawthorne (NICE)

Concept: disinvestment

• The term disinvestment in health care is gaining prominence internationally. It relates to:

“the processes of (partially or completely) withdrawing health resources from any existing health care practices, procedures, technologies or pharmaceuticals that are deemed to deliver little or no health gain for their cost, and thus are not efficient health resource allocations.”1

(Elshaug, Moss, Littlejohns et al, 2009).• Disinvestment remains a controversial activity

particularly when it is based on an assessment of cost-effectiveness.

Context: NHS efficiency plan

• The NHS is challenged with saving £20 billion by March 2015. Controlling costs without compromising quality of care, initiated by the quality, innovation, productivity and prevention (QIPP) programme, was reaffirmed by the Coalition Government.

• The National Institute for Health and Clinical Excellence (NICE) is supporting this challenge by identifying opportunities for cost savings through disinvestment of ineffective practices.

Background: pilot study

• Since 1999 NICE has been focusing attention to produce recommendations on interventions that should not be used (‘do not do’).

• A review of guidance indicated that NICE was producing these recommendations, but not publicising them.

• From 2006, NICE started highlighting ‘optimal practice’ from its guidance, and re-publishing them with supporting costing templates.

• In 2009 a project was initiated to explore and identify the best method in retrieving ‘do not do’ recommendations. Accessibility and format of recommendations via a web-based, searchable database were considered.

• Since 2010, NICE publishes a public list of ‘do not do’ recommendations on its website.

Study sources

Pilot study Source

1 NICE Clinical Guideline 71: Familial hypercholesterolaemia2

2 NICE Clinical Guideline 83: Critical illness rehabilitation3

•At the end of September 2009, NICE had published 89 clinical guidelines•Not possible to review them all, to test methods (follows)•2 selected randomly:

Methods

Search terms: Pilot study 1

1. Discontinued 13. Insufficient evidence

2. Should not 14. No evidence

3. Do not 15. Not useful

4. Not recommended 16. Cannot be recommended

5. Not currently recommended 17. Rarely indicated

6. Does not 18. Not routine

7. Not currently indicated 19. Should not be routine

8. Not indicated 20. Uncertain

9. Not an effective 21. Avoided

10. Not effective 22. Not usually

11. Ineffective 23. Only be used

12. Cost 24. Not required

Search terms: Pilot study 2

Changes: Additions (from NICE information services team):

12. Cost effective 29. Disinvest

Additions: 30. Low value (service)

25. Disadvantage(s) 31. Ineffectual

26. Feasibility 32 Only in research

27. Limited evidence

28. Should stop

Results: Pilot study 1

PILOT STUDY 1: CG71 Overall: Method 1:

Method 2:

Method 3:

Number of actual ‘do not do’ recommendations:

10 10 9 9

Number disregarded*: 37 29 8 19

Number of uncertain** ‘do not do’ recommendations:

1 1 1 1

Total number of potential ‘do not do’ recommendations:

48 40 18 29

*includes those that are statements about the guidance**unclear if it is a ‘do not do’ recommendation

Results: Pilot study 2

PILOT STUDY 2: CG83 Overall: Method 1:

Method 2:

Method 3:

Number of actual ‘do not do’ recommendations:

0 0 0 0

Number disregarded*: 64 36 58 47

Number of uncertain** ‘do not do’ recommendations:

0 0 0 0

Total number of potential ‘do not do’ recommendations:

64 36 58 47

*includes those that are statements about the guidance**unclear if it is a ‘do not do’ recommendation

Results

• Feedback from the three reviewers highlighted that due to the lack of standardised terminology method 1 was the best in capturing all relevant ‘do not do’ recommendations within context.

• For example, a recommendation identified by this method only:

“In asymptomatic children and young people with heterozygous [familial hypercholesterolaemia] FH, evaluation of coronary heart disease is unlikely to detect clinically significant disease and referral should not be routinely offered”.

• A database of all identified ‘do not do’ recommendations is now available on the NICE website.

• Feedback from the NHS is positive.

Conclusion

• Lack of consistency in terminology, not always easy to find recommendations.

• Many recommendations are redundant, they reinforce current best practice.

• Recommendations arise from the evidence base (for example, exploratory research) and would reflect strength of evidence.

• NICE guidance contains ‘do not do’ recommendations, and they can be highlighted as standalone.

• Many more areas of uncertainty are identified but where there is a lack of evidence, experts are reluctant to recommend that something should not be done.

Current output

Do not do database launched on NICE website October 2010.

Currently contains 769 recommendations.

References

1. Elshaug AG, Moss JR, Littlejohns P, Karnon J, Merlin TL, Hiller JE (2009) Identifying existing health care services that do not provide value for money. Med J Aust;190:269-73.

2. National Institute for Health and Clinical Excellence (2008). Identification and management of familial hypercholesterolaemia. Clinical guideline 71. London: National Institute for Health and Clinical Excellence.

3. National Institute for Health and Clinical Excellence (2009) Rehabilitation after critical illness. Clinical guideline 83. London: National Institute for Health and Clinical Excellence.

Links/Contact detailsLinks:NICE ‘Do not do’ recommendations database:http://www.nice.org.uk/usingguidance/donotdorecommendations/index.jsp

Research and Development, NICE:http://www.nice.org.uk/aboutnice/howwework/researchanddevelopment/about.jsp

Authors/Contributors: • Miss Moni Choudhury, Analyst• Ms Tarang Sharma, Senior Analyst• Dr Sarah Garner, Associate Director• Dr Grant Hill-Cawthorne, Clinical Fellow• Ms Bindweep Kaur, Analyst

Contact:• Moni Choudhury, Tel: Tel: 44 (0)20 7045 2094; • Email: Moni.Choudhury@nice.org.uk