Upload
lizbeth-park
View
228
Download
0
Embed Size (px)
DESCRIPTION
Topics More on Markov models versus decision trees More examples of Markov models Calculating annual transition probabilities Time independent (Markov chains) Time dependent (Markov processes) Temporary and tunnel states Half-cycle corrections
Citation preview
Markov Models II
Brennan Spiegel, MD, MSHS
VA Greater Los Angeles Healthcare SystemDavid Geffen School of Medicine at UCLA
UCLA School of Public HealthCURE Digestive Diseases Research Center
UCLA/VA Center for Outcomes Research and Education (CORE)
HS 249T Spring 2008
Topics• More on Markov models versus decision trees
• More examples of Markov models
• Calculating annual transition probabilities– Time independent (Markov chains)– Time dependent (Markov processes)
• Temporary and tunnel states
• Half-cycle corrections
Disadvantages of Traditional Decision Trees
• Limited to one-way progression without opportunity to “go back”
• Can become unwieldy in short order
• Difficult to capture the dynamic path of moving between health states over time
• Often fails to accurately reflect clinical reality
Markov Models• Allow dynamic movement between
relevant health states
• Allow enhanced flexibility to better emulate clinical reality
• Acknowledge that different people follow different paths through health and disease
Example Markov Model
Inadomi et al. Ann Int Med 2003
Markov ModelAlive
No BarrettAlive
Barrett
DeadBarrett
DeadNo Barrett
Year 0
AliveNo Barrett
AliveBarrett
DeadNo Barrett
DeadBarrett
Markov Model
DeadNo Barrett
AliveNo Barrett
AliveBarrett
DeadBarrett
Markov Model
Year 1
DeadNo Barrett
AliveNo Barrett
AliveBarrett
DeadBarrett
Markov Model
DeadNo Barrett
AliveNo Barrett
AliveBarrett
DeadBarrett
Markov Model
DeadNo Barrett
AliveNo Barrett
AliveBarrett
DeadBarrett
Markov Model
End
DeadNo Barrett
AliveNo Barrett
AliveBarrett
DeadBarrett
Markov Model
Decision Trees and Markov Models may Co-Exist
• Both provide different types of information
• Information from both is not mutually exclusive
• Markov model can be “tacked” onto end of a traditional decision tree
No Therapy
Chronic HBV
Inteferon
Lamivudine
Adefovir Salvage
Adefovir
Virological ResponseNormal Lifespan
No Response
No Cirrhosis
CirrhosisMarkov Model
Normal Lifespan
No Cirrhosis
CirrhosisMarkov Model
Normal Lifespan
Response
No ResponseStart Adefovir
No Resistance
Resistance
Con’t Lamivudine
Chronic HBV
Chronic HBV on Treatment
Virological Response
Markov Model #1
Virological Resistance
Virological Relapse
Uncomplicated Cirrhosis
To Cirrhosis Markov Model
Uncomplicated Cirrhosis
Complicated Cirrhosis
Liver Transplant Death
HepatocellularCarcinoma
Markov Model #2
No GI or CV Complications
Dyspepsia
Myocardial Infarction
Post Myocardial Infarction
Death
Post GI Bleed
GI Bleed
Clinical Response
Hepatocellular Cancer
Liver Transplantation
Death
START
Sub-Clinical HE
Overt HE
Non-HE Complication
Annual Probability Estimates
Annual Probability EstimateCirrhosis in HBeAg(-) 4.0% Cirrhosis in HBeAg(+) 2.2% Chronic HBV liver cancer 1.0% Cirrhosis liver cancer 2.1% Compensated cirrhosis decompensated 3.3% Decompensated cirrhosis liver transplant 25% Liver cancer liver transplant 30% Death in compensated cirrhosis 4.4% Death in decompensated cirrhosis 30% Death in liver cancer 43%
Converting Data Into Annual Probability Estimates
Cannot simply divide long-term data by number of years
Example:If 5-year risk of an event is 40%, then annual risk does not amount to:
40 5
= 8%
Converting Data Into Annual Probability Estimates
General rule for converting long-term data into annual probabilities:
1-(1-x)Y = Probability at Y Years
Example of Converting Long Term Data into Annual Probability
If probability of bleed at 5 years = 0.40, then the annual probability = x, as follows:
1- (1-x)5 = 0.40 (1-x)5 = 1 – 0.40(1-x)5 = 0.60
x = 0.097
(1-x) = 0.902
… or 9.7%
Example of Converting Long Term Data into Annual Probability
Check for errors by back calculating using the inverse equation:
1-(1-annual probability)Y = probability at Y years
1-(1- 0.097)5 = 0.40 1-(0.903)5 = 0.40
0.40 = 0.40
Markov Cycle Converter Forward Calculator
Enter Percentage to be Converted 40
Enter Number of Cycles 5
Cycle Probability= 0.09711955
Backwards Calculator
Enter Cycle Probability for Conversion 0.097
Enter Number of Cycles 5
Converted Probability= 0.39960267
Converted Percentage= 39.96026709
Steps to Combining Time-Independent Transition Probabilities
Step 1 Collect and abstract relevant studies
Step 2 Select common cycle length
Step 3 Convert all studies to common cycle length units
Step 4 Calculate common cycle transition probabilities
Step 5 Combine common cycle probabilities
StudyStudy
DurationNumber of 12 Mo
CyclesEnd
PercentageCalculated 12-Month
Probability
Jones 6 months 0.5 12% 0.23
James 12 months 1 19% 0.19
Johnson 18 months 1.5 22% 0.15
Marshall 3 months 0.25 8% 0.28
Example
Mean = 21.3% / 12-month cycle
Many Probabilities are Time Dependent
• Time independence is usually a simplifying assumption
• Progress though many systems in health care (biological, organizational, psychosocial, etc) are erratic and non-linear
• May need to account for time-dependent transitional probabilities using:– Tables– Tunnels
Using Tables for Time-Dependent Probabilities
• Tables allow transition probabilities to vary cycle-by-cycle
• Allow greater precision for processes that are non-linear
0.058
0.057
0.056
0.055
0.054
0.053
0.052
0.051
ProbabilityCycle
0.028
0.037
0.046
0.055
0.064
0.073
0.082
0.11
ProbabilityCycle
Time Independent Time Dependent
Time Independent: Linear Curve
Time Dependent: Non-linear Diminishing Returns
Time Dependent: Non-linear Accelerating Returns
Cycle
Prob
abili
ty
• Some events can interfere with otherwise orderly Markov chains
• Can get “stuck in a rut” that removes subjects from the usual flow of events– e.g. developing cancer
• Tunnel states add flexibility to Markov models:– Model getting “stuck in the rut”– Compartmentalize processes into component states– Can model various “recovery states” from the “rut”– Can incorporate time-dependent transitions
Using Tunnels States
Example Prior to Tunnel State
Example With Tunnel State
Half-Cycle Corrections• In “real life,” events can occur anytime during a
given cycle – it is usually a random event
• The default setting for Markov models is for events to occur at the exact end of each cycle
• Yet the default setting can lead to errors in the calculation of average values– Will tend to overestimate benefits (e.g. life
expectancy) by about half of a cycle
Rationale for Half-Cycle Corrections
“In whatever cycle a ‘member’ of the cohort analysis dies, they have already received a full cycle’s worth of state reward, at the beginning of the cycle. In reality, however, deaths will occur halfway through a cycle on average. So, someone that dies during a cycle should lose half of the reward they received at the beginning of the cycle.”
- TreeAge Pro Manual, p476
0 1 2 3 4
Cycle
Prop
ortio
n A
live
1.0
0.8
0.6
0.4
0.2
0.0
AUC=2
0 1 2 3 4
Prop
ortio
n A
live
1.0
0.8
0.6
0.4
0.2
0.0
Cycle
AUC=2.5
0 1 2 3 4
Prop
ortio
n A
live
1.0
0.8
0.6
0.4
0.2
0.0
Cycle
AUC=2.0ish