www.fuse.ac.uk

How to make evidence fit? The artful use of research in public health decision making in England and Scotland; a

comparative case study analysis

1st Int Conference on Realist Approaches to Evaluation and Synthesis: Successes, Challenges & the Road Ahead, 2014, 27-30 October,

Liverpool

Peter van der Graaf Teesside University Karen McCabe University of SunderlandMandy Cheetham Teesside University Rosemary Rushmer Teesside University (PI)

NIHR:HS&DR funded call 259

How to get research evidence into Public Health?

• Research utilisation: what is the problem?

• A realist study approach: exploring mechanisms

• Findings from two case study sites (UK and Scotland)

• Localising and tailoring: the art of discounting evidence?

Context and BackgroundChallenges for research utilisation in PH•Multi-factorial issues•patchy, contested evidence-base•limited access to certain evidence•contextual churn

Knowledge management rolesFacilitate and enable use of research evidence in commissioning and joint planning processes

How do they work?

Study designResearch questions Methods

How, when, where and by whom is research evidence (and other information) used in commissioning & planning PH interventions and to what effect?

Case study sites • Interviews• Observations• Documentary analysis• Local workshops• Joint interpretation

meetings

What do ‘knowledge managers’ do? Interviews

Are findings transferable? Delphi process & national workshop

What is involved in working in co-creation? Reflections

Realist approach

Findings English Case Study Site

• National data is localised to make it fit for commissioning decisions

• Social marketing methodology applied for deciding on interventions (actionable messages)

• Flow of evidence is person dependent (valued sources and preferred providers)

• Exclusion of evidence and providers (particularly Public Health impact evaluations)

Quotes

“Because we’re just different [..] we have a lot more issues I think than some places. Somewhere down south is maybe doing a piece of work on this, it would probably look quite different to what it’s saying here. Yes, so it wouldn’t work looking at somebody in another area, it’s not the same problem” (local authority participant 3).

“Evidence for commissioning is less likely to appear in your sort of peer review materials. Which are all very good will probably tell you what you need to know if you’re interested in a particular treatment, but aren’t very good when it comes to detailing how you might design a service” (NHS participant 4).

Quotes

“If you’re a councillor, you don’t have time to read a 20-30 page report. You want the headline, what are the key points that I need to know about this topic? So things like a fact sheet essentially. [..] And if they’re not going to read it, it’s not going to do any good. We need something that’s kind of responsive and short and snappy” (local authority participant 2).

Findings Scottish Case Study Site

• Business intelligence approach using local crime data to visualise hotspots for immediate action

• Fragmentation of evidence not resolved

• Restrained by legal requirements (Licensing Act)• Need for premise-level data (bad fit with PH data)• Preference for economic outcomes (and tacit

knowledge)

Quotes

“The licensing board is a very statutory type role, they are only concerned with licensing issues in terms of the License in Scotland Act, [..] at the end of the day, they are there to apply the licensing act in terms of the business and that's what they do” (Crime and Safety Partnership participant 6).

“We deal with the other side of it [..] less about enforcement and more about encouraging and working with those businesses to say look, there is a problem here, what can we do, why don't you become part of the solution, why don't we get the environmental community action team to have a look whether you want to do a clean-up. So we kind of compliment this enforcement and licensing type information” (Crime and Safety Partnership participant 6).

Quotes

“There’s a famous quote that says policy makers will continue to make decisions based on anecdotal evidence, if we can’t as analysts, can’t bring that to life. We’ve got to bring the data to life, not make it complex, not get caught in all the statistical, just make it simple. What works and feed that back into the operational and strategic environments, so resources just can be targeted better” (Crime and Safety Partnership participant 10 ).

Discussion

Conclusion

• Decision makers look for external validity• What works is not only about the nature of

evidence but its mobilisation • Localising as mechanism to fit evidence to

different contexts• Tailoring as mechanism to develop interventions

(actionable messages)

Developing the CMO model• What counts as evidence tied to where it is used and by

whom• More dynamic (constructionist) view of context needed• Preferred models for localising and tailoring are prone to

change• Additional skills: in-depth understanding of local decision

making processes and ability to adapts mechanism to changing contexts

Acknowledgements

The work was undertaken by Fuse, a UKCRC Public Health Research: Centre of Excellence. Funding from the British Heart Foundation, Cancer Research UK, Economic and Social Research council, Medical Research Council, and the National Institute for Health Research, under the auspices of the UK Clinical Research Collaboration, is greatly acknowledged.

Opinions expressed in this presentation do not necessarily represent those of the funders.

This research has been funded by the NIHR HS&DR programme.

PI: Professor Rosemary Rushmer (Teesside University)

Co-applicants: Lynda Cox (NHS England),Professor Ann Crosland (University of Sunderland), Dr Joanne Gray (Northumbria University), Mr Liam Hughes (Local Government Group, retired), Professor David Hunter (Durham University), Dr Pete Seaman (Glasgow Centre for Population Health), Professor Carol Tannahill.

Researchers: Mandy Cheetham (Teesside University), Karen McCabe (University of Sunderland), Peter van der Graaf (Teesside University). HS&DR Funding Acknowledgement: This project was funded by the National Institute for Health Research Health Services and Delivery Research Programme (NIHR:HS&DR funded call 259).Department of Health Disclaimer: The views and opinions expressed therein are those of the authors and do not necessarily reflect those of the HS&DR Programme, NIHR, NHS or the Department of Health.