Upload
tarika
View
22
Download
0
Embed Size (px)
DESCRIPTION
Systematising the process : the role of PSMs in informing model structure. Jim Chilcott Technical Director, ScHARR-TAG Suzy Paisley DoH Research Scientist in Evidence Synthesis Paul Tappenden Senior OR Analyst Consensus Working Group on the Use of Evidence in Economic Decision Models - PowerPoint PPT Presentation
Citation preview
Systematising the process : the role of PSMs in informing model structure
Jim ChilcottTechnical Director, ScHARR-TAG
Suzy PaisleyDoH Research Scientist in Evidence Synthesis
Paul TappendenSenior OR Analyst
Consensus Working Group on the Use of Evidence in Economic Decision Models MRC HSRC Workshop 26 September 2005
Before we start…
Consensus Working Group on the Use of Evidence in Economic Decision Models
Today’s presentation
Define ‘structuring’ within the modelling process
Describe current practice– Evidence used– Tasks undertaken
Discuss ‘What role problem structuring methods (PSMs)?’
Consensus Working Group on the Use of Evidence in Economic Decision Models
Defining model structuring
Disease and treatment pathway
Model scope Modelling method
Consensus Working Group on the Use of Evidence in Economic Decision Models
Current practice:HTA Guidelines
HTA guidelines (n=32)
None 17 (53%)
Description 2 (6%)
Justification 4 (13%)
Sensitivity analysis 5 (16%)
Not available 4 (13%)
Consensus Working Group on the Use of Evidence in Economic Decision Models
Current practice:Modelling guidelines (Philips et al.,2004)
Statement of problem
Justification of approach
Scope / perspective Structural
assumptions Consistency
Strategies / comparators
Model type Time horizon Health states Cycle length Parsimony
Consensus Working Group on the Use of Evidence in Economic Decision Models
Current practice:Reported (Cooper et al., 2005)
HTA monographs (n=42)
Diagram of model 26(62%)
Model structure
and assumptions discussed 5 (12%)
Consensus Working Group on the Use of Evidence in Economic Decision Models
Current practice:Observed
8 case study projects Description model structuring process Short, semi-structured interviews Examination of corresponding study
reports
Consensus Working Group on the Use of Evidence in Economic Decision Models
Evidence consulted
Subject Clinical literature
Clinical expert (one-off)
Clinical expert (ongoing)
Existing models
1 2 3 4 5 6 7 8
TOTAL 8 (100%) 1 (13%) 5 (63%) 8 (100%)
Consensus Working Group on the Use of Evidence in Economic Decision Models
Reporting
Subject Methodology justified Health states /pathways justified?
Source of structural assumptions reported?
1 Report not yet available
2 3 4 5 6 7 8 TOTAL 5 (71%) 2 (29%) 3 (43%)
Consensus Working Group on the Use of Evidence in Economic Decision Models
Summary of themes
Model structure amended according to data availability
Clinician input (1st order validation) Existing models consulted as 1st step Iterative structuring process Complexity of structure shifted both ways Very limited reporting process of development MSc dissertation provided most transparent
reporting
Consensus Working Group on the Use of Evidence in Economic Decision Models
Current practice:Evidence used
Clinical expertise Published evidence Existing models
Natural history Treatment pathway Modelling method Effectiveness
evidence
Consensus Working Group on the Use of Evidence in Economic Decision Models
Current practice:Tasks undertaken
Iteration Consultation Evaluation and comparison Feedback and clarification Decisions regarding simplicity,
complexity, feasibility and inconsistency
Consensus Working Group on the Use of Evidence in Economic Decision Models
Why PSMs?
Lack of transparency Data modelling vs problem modelling Validity of model (unquestioned
question) Issues that ‘can’t be modelled’ ‘Not qualified to comment’
Consensus Working Group on the Use of Evidence in Economic Decision Models
Why PSMs?
Data led models vs evidence informed models
Use of expert opinion Implicit use of PSM methods Observed usefulness of action research
process
Consensus Working Group on the Use of Evidence in Economic Decision Models
PSMs / MSMs
Messes Problems
SODA
SSM
SCA
Robustness Analysis
Drama Theory
Decision Conferencing
?
Decision analysis
Markov
Queuing theory
Simulation
Game Theory
Consensus Working Group on the Use of Evidence in Economic Decision Models
Characteristics of PSMs
Techniques to facilitate exploration / definition of a problem situation
Group based Uncovering options Enabling group action Accepting uncertainty Managing conflict Facilitator - analyst
Consensus Working Group on the Use of Evidence in Economic Decision Models
SODA
Individual interviews =>
Merge maps =>
Group workshops =>
Commitment to action
Cognitive mapsGroup mapExpand/refine map Group ownership
Consensus Working Group on the Use of Evidence in Economic Decision Models
SSM
Enter situation Take action to improve theconsidered problematic problem situation
Define possible changes which areboth desirable and feasible
Express the problem situation
Compare models with the real worldReal world
Systems thinking about the real world
Formulate root Build conceptual models definitions of the systems named in
the root definitions
Consensus Working Group on the Use of Evidence in Economic Decision Models
Pros Problem structuring, not problem
solving Gathering information, generating ideas Retain flexibility in early stages Demonstrate understanding of
complexity, diversity and inconsistency Prioritising and simplifying
Consensus Working Group on the Use of Evidence in Economic Decision Models
Cons
Methods development required Adaptation to HTA context New skills Ownership Facilitation of group decision making Reproducibility
Consensus Working Group on the Use of Evidence in Economic Decision Models
Developing a mixed method
‘Soft’ qualitative structuring process
‘Hard’ analytic
methods
Qualitative model structuring process to inform quantitative decision-making process.