Upload
madeline-russell
View
216
Download
1
Tags:
Embed Size (px)
Citation preview
Involving the Public in Priority Involving the Public in Priority SettingSetting
A case study in NHS Dumfries and Galloway
Verity Watson1
Andrew Carnon2
Mandy Ryan1 Derek Cox2
1 Health Economics Research Unit, University of Aberdeen2 Directorate of Public Health and Strategic Planning, NHS Dumfries and Galloway
StructureStructure
1. Introduction to priority setting
2. Overview of priority-setting process in NHS Dumfries & Galloway
3. How discrete choice experiments can be used in priority setting
4. Applying the discrete choice experiment in NHS Dumfries & Galloway
5. How successful was the priority-setting process?
How Do We Decide?How Do We Decide?
IntroductionIntroduction
• Options for health care are constrained by limited funding
• Choices imply priorities
• To make priorities explicit requires a priority- setting framework
• Options appraisal:
– identified projects are scored based on pre-defined criteria
– the relative importance of criteria are defined by weights
• Weights can lack transparency and accountability
Portsmouth Scorecard (Sandwell modification)Portsmouth Scorecard (Sandwell modification)
Factor weighting Very low
Low Mid High Top
Evidence < 3 10 20 30 40
Benefit number < 3 10 20 30 40
Cost < 3 10 20 30 40
Acceptability < 3 10 20 30 40
National requirement < 3 10 20 30 40
Addressing inequality <3 5 10 15 20
Wider society benefit <3 5 10 15 20
Only alternative <3 5 10 15 20
Local feeling 0 5 5 7 10
(Austin et al, 2007)
Priority-Setting Principles in Priority-Setting Principles in NHS Dumfries & GallowayNHS Dumfries & Galloway
• Focus on Delivering for Health
• Transparency and rigour of process
• Public involvement
• Acute services CHP/Long term conditions
• Annual event
• Learning process
Priority-Setting CriteriaPriority-Setting Criteria
Ten Criteria were chosen based on Delivering for Health
1. Location of care2. Public consultation while developing project3. Use of latest technology4. Service availability5. Patient involvement in own care6. Management of care7. Evidence of clinical effectiveness8. Health gain9. Risk avoidance10. Priority area
NHS Dumfries & Galloway NHS Dumfries & Galloway ProgrammesProgrammes
• Acute Services• Cancer• CHD/Stroke/Diabetes• Child Health• Corporate• Healthcare-Acquired
Infection
• Learning Disabilities• Local Health
Partnership/Primary Care
• Long Term Conditions• Mental Health• Older People• Public Health
Process in Process in NHS Dumfries & GallowayNHS Dumfries & Galloway • Public involvement event
• 12 programme leads to submit bids
• Panel to take overview
• Bids scored on weighted criteria
• Ranked list of bids produced (Health Intelligence Unit)
• Corporate Management Team decisions
Public involvement event:Public involvement event:
Discrete Choice ExperimentDiscrete Choice Experiment
Applying DCEs to Priority SettingApplying DCEs to Priority Setting
To investigate the relative importance of criteria to public in Dumfries and Galloway
Attributes are the 10 priority setting criteria
Describe the criteria by a number of levels
Define all possible combinations of attributes and levels
Create choice sets
Ask respondents to choose between different hypothetical health services for Dumfries and Galloway.
Attributes and LevelsAttributes and Levels1. Location of care
• at home• at GP• at Local Health Partnership• at D&G Royal Infirmary• outside D&G
2. Public involvement• no consultation• consultation at final stage• consultation at some but not all stages• consultation at all stages
Attributes and LevelsAttributes and Levels3. Use of Technology
• does not use latest technology• uses latest technology• uses cutting edge technology
4. Service availability• office hours only • office hours and outside office hours
5. Patient involvement in own care• decision by health professional• patient shares decision
Attributes and LevelsAttributes and Levels6. Management of care
• group of health professionals not working as a team• an individual health professional• group of health professionals working as team
7. Evidence of clinical effectiveness• no evidence• number of clinical studies• at least one RCT• at least three RCTs
8. Health Gain• small gain to a small number• large gain to a small number• large gain to a large number
Attributes and LevelsAttributes and Levels9. Risk avoidance
• low risk to lower than low risk• medium risk to low risk• medium risk to lower than low risk• high risk to medium risk• high risk to low risk• high risk to lower than low risk
10. Priority area• none• local priority• national priority• local and national priority
Experimental designExperimental design
207,360 possible combinations (51x61x43x33x22)
Used experimental design techniques and reduce to 64 profiles.
These were paired with a mirror image (foldover method)
Ensured orthogonality, minimum overlap and level balance
Respondents were presented with the choice of two health services and asked to choose their most preferred.
Example choiceExample choice A B
Location of care Dumfries and Galloway Royal
Infirmary Outside Dumfries and Galloway
Public consultation in decision making
No consultation Public and Patients were consulted at the final stage
Use of latest technology
Not using the latest technology Using the latest technology
Service Availability Office Hours only Office hours and out-with office
hours Patient’s involvement in own care decision
No opportunity Has the opportunity
Management of care Care is managed by a single
individual A group of professionals working as a
team Evidence of clinical effectiveness
Clinical Studies At least 1 RCT
Health Gain Large Gain to a Small Number
Small Gain to a Large Number Large gain to a large number
Risk Avoidance Reduction from a high risk to a
low risk Reduction from a medium risk to a
low risk Priority Area National priority Local and National Priority
Service A Service B Which service do you prefer?
Sample and SettingSample and Setting
A random sample of 100 members of public from Dumfries and Galloway
Invited to attend a half day meeting
Lunch and refreshments and travel expenses
68 respondents attended.
Sample and SettingSample and Setting
Age of Sample and Age of Sample and Dumfries & Galloway PopulationDumfries & Galloway Population
05
10152025303540
18-35 36 - 50 51 - 65 66+
Age
Per
cen
t
Sample D&G Population
Rurality of Sample and Rurality of Sample and Dumfries & Galloway PopulationDumfries & Galloway Population
05
10152025303540
Largeurban
Otherurban
Accessiblesmalltowns
Remotesmalltowns
Accessiblerural
Remoterural
Rurality
Per
cen
t
Sample D&G Population
Deprivation of Sample and Deprivation of Sample and Dumfries & Galloway PopulationDumfries & Galloway Population
0
10
20
30
40
50
1 2 3 4 5
Scottish Index of Multiple Deprivation Quintiles
Per
cen
t
Sample D&G Population
Advantaged Disadvantaged
AnalysisAnalysis
Responses are dichotomous, =1 for the option chosen and 0 for the option not chosen
Analysis using a logit regression:
Benefit = βiDxl
All criteria are qualitativethese are coded as dummy variablesOne level is omittedCoefficients are interpreted relative to the omitted level
Applying DCEs to Priority SettingApplying DCEs to Priority Setting
Criteria Coefficient/Weight
1. Location of care: At GP’s office 0.2523549*** At local health partnership 0.1182278 At Dumfries and Galloway Royal Infirmary 0.3395357*** Outside Dumfries and Galloway -0.5770746*** At home Comparator 2. Public involvement in decision making: Consultation at the FINAL stage 0.1165849* Consultation at SOME but NOT ALL stages 0.2253108*** Consultation at ALL stages 0.4921577*** No consultation Comparator 3. Use of technology: Uses the latest technology 0.7056297*** Uses cutting edge technology 0.7087127*** Does not use the latest technology Comparator 4. Service Availability: Office hours only -0.5708215*** Office hours and outside office hours Comparator
Applying DCEs to Priority SettingApplying DCEs to Priority SettingCriteria Coefficient/Weight
5. Patient involvement in own care: Decision made by health professional -0.474334*** Patient shares in decision Comparator 6. Management of Care: By a individual health professional 0.1576166*** By a group of health professionals working as a team 0.7264206*** By a group of health professionals NOT working as a team Comparator 7. Evidence of Clinical Effectiveness A number of clinical studies -0.0117717 At least 1 RCT 0.0667422 At least 3 RCT 0.4146794*** No evidence Comparator 8. Health Gain Small gain to a large number or large gain to a small 0.4328322*** Large gain to a large number 0.9661512*** Small gain to a small number Comparator 9. Risk Avoidance Medium risk reducing to a low risk 0.1112207 Medium risk reducing to lower than low risk 0.2056392** High risk reducing to a medium risk 0.0775806 High risk reducing to a low risk 0.1491032 High risk reducing to lower than low risk 0.1032418 Low risk reducing to lower than low risk Comparator 10. Priority Area Local priority 0.2644803*** National priority 0.0711403 Local and national priority 0.1950561*** No priority Comparator
How were the How were the Discrete Choice Experiment Discrete Choice Experiment
results used?results used?
Example: Community/Voluntary Sector Example: Community/Voluntary Sector Health & Wellbeing DatabaseHealth & Wellbeing DatabaseCriterion Level Weighted
score
Location Regional 0.34
Public involvement All stages 0.49
Use of technology Up to date 0.71
Availability All hours 0.57
User involvement in treatment/service
Has opportunity 0.47
Management of care Not applicable -
Evidence of effectiveness Not applicable -
Health gain Large gain to a large number 0.97
Priority area Both local and national 0.19
Weighted bid score =100 x Σ criterion weighted scores
no of criteria scored
53.43
0
10
20
30
40
50
60
1 3 7 10 13 16 19 22 25 28 31 34 37 40 44 46 48 52 54 58 61 64 67 70 73 76 79 82 85 88 91 94
Ranked Order
Wei
gh
ted
Bid
Sco
re
Acute Services CHP/LTC Other Bids
Weighted Bid ScoresWeighted Bid Scores
33.31
37.71
37.83
39.1
42.62
43.28
49.12
0 10 20 30 40 50
Average Bid Score
Corporate
Child Health
Long Term Conditions
CHP/Primary Care
Acute Services
CHD/Stroke/Diabetes
Public Health
Programme Bid ScoresProgramme Bid Scores
Top and Bottom BidsTop and Bottom Bids
Top Scoring Bids
1. DGH capital developments
2. Additional consultant in elderly medicine
3. Expand anaesthetic services
4. Rapid access chest pain service
5. Community/voluntary sector health and wellbeing database
6. Extra IT training staff
Bottom Scoring Bids
90. Set up neurology/genetic clinic
91. Increase paediatric clinics
92. Orthoptic vision screening
93. Expand school nursing
94. Appoint NHS Board business continuity manager
95. Provide extra secretarial input for paediatric consultants
Outputs of Priority-Setting ProcessOutputs of Priority-Setting Process
• Good spread of weighted bid scores
• Public involvement, transparency and accountability intrinsic to process
• Uncertain whether Delivering for Health principles fully reflected in weights (acute services and CHP/ long term conditions)
• Some anomalies (e.g. school nursing, IT training)
• Bid scores used as central aid to decision making, along with other factors (directives, risk, available finance)
How Successful was the Priority-How Successful was the Priority-Setting Process?Setting Process?
Limitations
• Sample size
• Complex concepts for public
• Cost of exercise
• Decision making still difficult
Benefits
• Planned approach to decision making
• Public response positive
• Transparency and defensibility
• Workable process
Organisational Priority-setting DecisionsOrganisational Priority-setting Decisions
Relative Benefits
• Location• Patient involvement• Health gain, etc
Difficulty
• Current state• Availability of skills required• Scale of change
Urgency
• Risk if not done• Quick wins possible• Current versus avoided future costs
Dependencies
• Funding available • Fit with existing projects• Basket of projects