Embed Size (px)
Citation preview
MMI 406: Decision Support Systems and Health Care Hunink, H. & Glasziou, P. (2009). Decision making in health and medicine: Integrating evidence and values (7th printing or later). Cambridge, England: Cambridge University Press.
Contents Chapter 1: Elements of decision making in health care ............................................................................... 2
Chapter 2: Managing Uncertainty............................................................................................................... 10
Chapter 3: Choosing the best treatment .................................................................................................... 17
Chapter 4: Valuing Outcomes ..................................................................................................................... 23
Chapter 5: Interpreting diagnostic information .......................................................................................... 32
Page 1.1
Chapter 1: Elements of decision making in health care - Decisions in health care involves a complex web of diagnostic and therapeutic uncertainties,
patient preferences and values and costs
- A combination of a broad range of illness and imperfect treatment options increases our
potential to help, but it also increases costs and makes decision making more complex and
difficult
Page 1.3
- Decision analysis is a systematic, explicit, quantitative way of making decisions in health care
that can lead to both enhanced communication about clinical controversies and better
decisions
Decision Making and uncertainty
- Uncertainties may be about
o the diagnosis,
o the accuracy of available diagnostic tests,
o the natural history of the disease
o the effects of treatment in an individual patient
o the effects of an intervention in a group or population as a whole
- A major purpose of decision analysis is to assist in comprehension of the problem and to
give us insight into what variables or features of the problem should have a major impact on
our decision
Page 1.4
- Medical decisions must be made, and they are often made under conditions of uncertainty
- Uncertainty may arise from
o Erroneous observation
o inaccurate recordings of clinical findings
o Misinterpretation of the data by the clinician
o Ambiguity of the data
o Variation in interpretation of the information
o The effects of treatment are uncertain
Page 1.5
o An important uncertainty is the natural history of the disease
- Decision analysis process
o Make the problem and its objectives explicit
o List the alternative actions and how these alter subsequent events with their
probabilities, values, and trade-offs
o Synthesize the balance of benefits and harms of each alternative
- PROACTIVE approach to health care decision making:
o Problem
o Reframe
o Objects
o Alternatives
o Consequences and changes
o Trade offs
o Integrate
o Value
o Explore
o Evaluate
- Step 1: PROactive: the problem and objectives
o Begin by making sure you are addressing the right problem
o Make explicit what the possible consequences are that you are seeking to avoid or
achieve
o After initial attempt at defining the problem, reframe the problem from other
perspectives
o Identify the fundamental objectives for any course of action
Page 1.6
o Define the problem:
What are your principal concerns?
What would happen if you took no immediate action?
Lead to a description of the possible sequences of events in the natural
history of the condition
Followup by asking 'and what then'? several times
Each problem has a complex sequence of uncertain but potentially serious
consequences
Visual aids that help describe the problem include decision trees, state
transition diagrams, influence diagrams, and survival plots
These visual aids help chart the possible course of events
they are helpful in describing and communicating the consequences and
help navigate the decision making process
Consequence table:
A tabulation of the principal concerns for various options
Page 1.7
o Reframe from multiple perspectives
Broaden focus from a disease framework to one that includes the concerns
for the patient
Broaden perspective to include the aggregate limits on resources
Broaden perspectives of the patient, the provider, the payer, and the public
policy maker
o Focus on the objective
Page 1.8
If you framed and reframed the problem appropriately, the pivotal concerns
and objectives should have become apparent.
Check that you have a clear idea of the objectives
What elements are of most concern to the patient or population
What are the short term and long term objectives and concerns
How do these vary between patients
Distinguish between means objectives and fundamental objectives
Means objective: An intermediate goal but which is only a stepping stone to
what we truly value
The nature of objectives may be clarified by repeatedly asking 'why'
Understanding the fundamental objectives can help us generate options
that achieve such objectives through different means
Page 1.9
- Step 2: proACTive: the alternatives, consequences, and trade-offs
o Consider all relevant alternatives
All alternatives may be placed in one of 3 categories
A wait and see, watchful waiting, or a do-nothing policy
initiate an intervention (eg treatment now)
obtain more information before deciding
Decision tree:
A decision tree is a visual representation of all the possible options
and consequences that may follow each option
The initial line is labeled with the population or problem you are
considering
The Square represents a decision node
Subsequent lines: alternative actions
From each alternative action, there will usually be a subsequent
chance node (the circles), with branches representing the possible
outcomes of each option
Page 1.10
Wait and see, watchful waiting, or do nothing policy
You may decide to do nothing about the condition
Usually you will have a contingent policy that requires action
depending on the disease course over time
o Monitoring: a regular check is made at fixed times to see
whether the condition has improved, remained the same,
or become worse
o Triggering: Wait for a change in the type or severity of
symptoms
Intervention
List the active intervention alternatives, refraining from any
evaluation of their merit at this point
Page 1.11
Source of List of alternatives:
o Current knowledge
o Discussion with colleagues and experts
o Textbooks
o literature searches
o A search of controlled trials – Cochrane Controlled Trials
Registry
Obtain information
Types of information:
o Determining the prevalence of disease
o doing a population survey
o measuring the level of a toxin
o Symptoms
o Signs
o Lab test
o imaging test
Testing may help to clarify the prognosis or the responsiveness to
treatment
Page 1.12
o Model the consequences and estimate the chances
Need to think through the sequence of consequences of each decision
option and the chances of each event
Both short term and long term consequences should be considered
For each consequences find the best available evidence to support your
arguments
After listing the alternatives, need to consider the consequences of each
Page 1.13
Each alternative will lead to a different distribution of outcomes which need
to be quantified
The relevant outcomes depend on the particular problem at hand
Chance Tree:
A chance tree is a visual representation of a series of random
discrete linked events. It visualizes the chance that each event can
occur
Round circles (chance nodes) are used to indicate time points at
which there are 2 or more possible outcomes
Page 1.14
The decision tree assists in structuring the sequence of choices and
outcomes over time
Balance sheet:
A balance sheet tabulates the consequences of different options
and considers all relevant perspectives and important dimensions
Usually, the first alternative will be a wait and see strategy and the
balance sheet will then incorporate the consequence table
The subsequent columns will show the consequences of each
alternative
The probability of uncertain outcomes are also included
(spontaneous resolution and complication rates)
Assembled by describing the outcomes with each alternative or by
describing the relative effects of each alternative
The table will also describe the potential harms and resource costs
of treatment alternatives:
o the direct burden or discomfort from the intervention
o the complications and adverse effects of the intervention
o the cost to the health care system, patients, and their
families
Page 1.15
o Identify and estimate the value trade-offs
Page 1.16
Such trade offs require clarification of the values involved
Some values can be clarified by trying out one of the alternatives
Many decisions do not allow trial period
A common dilemma is a treatment that offers relief of symptoms but at a
small risk of serious adverse consequences
The balance will depend on the individual's prognosis and severity as well
as the magnitude of the potential harms and the strength of each
individual's outcome preferences
Page 1.17
- Step 3: proactIVE: Integration and Exploration
o Which option is best
o Calculate the expected value: The average value gained from choosing a particular
alternative
o The option with the highest expected value will be chosen
o Integrate the evidence and values
If there are multiple dimensions, a useful next step is to focus on important
differences between options
In the clinical balance sheet, rank the issues in order of importance
Rankings done separately within the benefits and harms
Rows for which the consequences are fairly even may be struck out
For some dimensions, the sequence of events is complex, and will be better
represented by a chance tree
Require a formal calculation of the expected value of each option
Decision can be aided by calculating the expected value
Expected Value: the sum of the values of all the consequences of that
option, each value weighed by the probability that the consequence will
occur
Page 1.18
o Optimize the expected value
Decision analysis employs an explicit principle for making choices: maximize
expected utility
Page 1.19
The probability of each outcome is multiplied by its value, and for each
alternative, these products are added
In some situations, some decision makers prefer to minimize the chance of
the worst outcome (a minimax strategy)
Precedent, authority, habit, religious considerations, or local consensus may
also play a part in making decision
o Explore the assumptions and evaluate uncertainty
Page 1.20
To understand the effects of uncertainties on our decision, you should
perform a 'what if' analysis, also known as a sensitivity analysis
By varying the uncertain variables over the range of values considered
plausible, you can calculate what the effect of that uncertainty is on the
decision
If the decision is not sensitive to a plausible change in a parameter value,
then the precise value of that parameter is irrelevant
If the decision does change, this warrants further study to find out more
precisely what the value is
If the key variables causing changes are probabilities, we say the decision is
'probability-driven'. More research may be needed to get better evidence
If the decision hinges on values and preferences, it is said to be 'utility-
driven'. these uncertainties cannot be resolve by better evidence, because
they are not about facts
- Using the results
o It is analysis process that is reapplied
o Elements of the problem that are likely to change the decision are the critical factors
in applying the decision analysis more broadly
Page 1.21
o Probabilities such as the likelihood of having a disease, the likelihoods of observing
various test results given the presence or absence of a disease, and the responses to
treatments, are among the most important factors
o Also important are the values attached to the various dimensions of outcome, such
as survival, functional status, and symptoms
o Consideration of these factors is assisted by sensitivity and threshold analysis
o Guidelines for specific clinical decisions
A clinically useful decision guide should meet 2 requirements
It should give the clinician information about how outcomes of a
recommended practice are likely to vary with different patient
characteristics
The outcomes should be presented in a way that permits
incorporating patients' preferences
A decision analyst needs to be mindful of the practicalities and
constraint of using a guide in clinical practice: The more time it
provides for individually tailoring to an individual patient, the more
time it will take to use it, and so it may be less used as its complexity
increases
o Clinical Algorithms
A clinical algorithm consists of a structured sequence of questions and
recommended actions based on the answers to those questions
Page 1.22
They are called clinical protocols, clinical pathways, or flow charts
The questions will divide patients into subgroups based on features such as
disease severity, allergies, other diseases, etc., which will then lead to a
sequence of actions such as investigation or treatment
From a decision tree, prune away the suboptimal alternatives, the result
would be a clinical algorithm
Clinical algorithms are particularly useful to assist rapid and consistent
decision making when patient preferences are not crucial
However, we cannot readily adapt the steps to different circumstances and
patient values
An algorithm is usually devised for a well defined set of circumstances, and
it is difficult to broaden it to cover others
Page 1.23
o Clinical Balance sheets
The aim is to present quantitative estimates of the consequences of the
different reasonable alternatives
o Patient oriented decision aids
Use paper, video, or interactive computer guided information that describe
the problem, the alternatives, and the consequences
The informed patient and clinician can then meet to make a final decision
Page 1.24
- Why are these tools useful
o The tools enable us to lay on our assumptions, the evidence, and our goals explicitly
and systematically and they help us overcome some well-documented cognitive
limitation
- But are they practical?
Page 1.25
o For 'once-only' decisions, it is advisable to check the evidence and draw a rough
consequence table
Page 1.27
- Summary
o The process is a recursive circular process with feedback loops
o A complete decision model is generally developed only for commonly recurring
problems
o A detailed analysis is necessary if there are competing diagnostic or treatment
strategies, where consensus has not be established, and where there is considerable
uncertainty
Page 2.33
Chapter 2: Managing Uncertainty Page 2.34
- Regardless of uncertainties, we must make a choice
- even doing nothing is a choice
- Doctors' agreements about decisions to treat hypothetical cases was improved when given
numerical rather than verbal expressions of probability
- Patients generally express a desire for risk communication and most prefer this to be
quantitative
Page 2.35
- Using a range still provides a better picture than a verbal expression
Page 2.36
- Types of probability
Page 2.37
o Frequentist – Probability discussed in terms of empirical frequencies in a sample of
observation
o Subjectivist / Bayesian – Probability is fundamentally a degree of belief
o A rate is an instantaneous change in the cumulative probability of an outcome per
unit of time, rather than an average change
Page 2.38
- Diagnostic uncertainty
o Diagnosis is a very uncertain art
o Good diagnosis depends on both knowing all the possibilities and accurately
assessing their relative frequency
o Diagnostic probabilities express our uncertainty about the list of differential
diagnoses
o The summation principle
The differential diagnosis should include all possible single diseases and
combinations of disease and the sum of the probabilities of all possibilities
must add up to 1
One and only one possibility must be true
General requirement for the analysis of chance outcomes: they must be
structured to be mutually exclusive (only one can occur) and collectively
exhaustive (one must occur)
Page 2.39
If multiple diseases are possible, we need to be explicit about this
P(A) + P(B) + P(A and B) + P(neither) = 1
o Conditional Probabilities
Conditional probability of event E given event F = P(E|F) = The probability
that event E occurs, given the event F has occurred
If the conditional information makes no difference to the probability, then
we say the 2 factors are independent
o Sources of data
Page 2.40
The principal empirical information will be studies of consecutive patients
with a particular presenting complaint
Subjective probabilities – Estimates based on personal experience
Objective / Data based / Frequency based probabilities – Estimates based
on data
- Prognostic uncertainty
o Prognostic uncertainty is uncertainty about future health states
Page 2.41
o Prognosis involves probability over time
o Prognosis is often expressed through incidence and hazard rates, which measure
the probability per unit time (e.g. percent per year)
o The most complete description of prognosis will usually be a survival curve, which
shows the effects of risk over time
o Survival Curves
It plots the probability of being alive over a period of time
There is a progressive decrease in survival
Proportion Alive – 5 year survival
Proportion Dead – 5 year mortality
Median survival – the point at which 50% of patients have died and 50% are
still alive
Page 2.42
The expected value or mean (life expectancy) – the area under the survival
curve
o Probability tree
Prognosis may be defined as the chance tree facing an individual given
particular conditions
These conditions include prognostic factors (also called risk factors)
A survival curve is a concise format for visually representing the chance tree
of successive event rates (e.g. mortality rate) over time
The probability of dying during the first year = Probability of being alive at
year 0 (100%) - Probability of being alive at year 1 ( 91%) = 0.09
o Prognostic Factor
Everyone with the same disease does not have the same prognosis
Risk may be modified by many other factors, such as the stage of disease,
and the patient's age and gender
Such prognostic factors enable us to refine our individual prediction
Page 2.43
Survival curves are useful for presenting any data that involve the time to an
event, not only mortality
Page 2.44
o Sources of data
Use inception cohort: to have a large cohort of patients with all prognostic
factors measured at the beginning of the disease, followed up to the end
stage of the disease
Studies will describe the prognosis of patients given their particular
treatments
If they had no specific treatment, the prognosis is known as the natural
history
if they had specific treatment that modified the disease process, then the
study provides evidence on the prognosis conditional on those treatments
natural history gives us the probabilities we need to estimate along the "no-
intervention" branch of a decision tree
By comparing it with the prognosis in the absence of the disease, the
natural history also enables us to calculate the individual potential benefit
from treatment
Most treatments provide a chance of cure, but usually with the risk of some
adverse effects
This mean that the net benefit a patient derives from treatment is usually
less than the potential benefit
Integrating prognostic information as a function of patient characteristics is
necessary when developing a guideline
Page 2.46
Potential benefit – the difference between the expected outcome based on
an individual's prognosis if a harmless curative treatment is available and
the expected outcome based on his / her current prognosis without specific
treatment
- Treatment Uncertainty
o For those treatments that are not miracle cures, we need an accurate assessment of
their incremental benefit for comparison with possible harms
o Results from clinical trials may need to be adapted for application to individual
patients
Any individual's prognosis and potential benefit may be quite different from
the average patient in the trial
The individual's concomitant illnesses and risk factors may be different
The effectiveness and cost of the intervention may differ by setting
Page 2.47
o Sources of data
For controlled trials, there are two principal design problems: establishing 2
comparable groups and unbiased observation of the outcome
The imbalanced prognostic factors are said to confound the treatment
comparison
A confounder is a prognostic factor that is associated both with the
exposure to an intervention (or with another determinant of outcome) and
with the outcome (or disease) but is not an intermediate in the causal chain
Randomization is the only secure method of obtaining balance between the
treatment groups in both known and unknown prognostic factors
Ideally, both patient and clinician will be unaware of the treatment
allocation, known as double blind
Page 2.48
Levels of evidence used for preventive interventions
Level 1: Evidence obtained from at least one properly randomized
controlled trial
Level 2-I: Evidence obtained from well designed controlled trials without
randomization
Level 2-II: Evidence obtained from well designed cohort or case control
analytic studies, preferably from more than one center or research group
Level 2-III: Evidence obtained from multiple time series with or without the
intervention. Dramatic results in uncontrolled experiments could also be
regarded as this type of evidence
Level 3: Opinions of respected authorities, based on clinical experience;
descriptive studies and case reports; or reports of expert committees
A case control study compares a group of individuals who have experienced
an outcome of interest with a comparable group who have not, to
determine the differences between their previous prognostic factor
A cohort study compares a group of individual who have exposed to an
intervention with a comparable group who have not, to determine the
difference between their outcomes
Page 2.49
- Combining Probabilities
o The overall probability is the sum of all paths in the chance tree that result in
disease
o Probability multiplication rules
Page 2.50
o Conditional probability
P(A | B) = P (A and B) / P (B)
It expresses the probability of an outcome under the condition that the
other outcome has occurred
Page 2.51
o Dependence and independence
Probabilistic independence – When the conditional probability of an event
E, given another event F, is the same as the unconditional probability of
event E, we say that events E and F are probabilistically independent
P (E | F) = P(E)
If the p-value is less than 0.05, or the confidence interval does not include
zero, then the difference in the conditional probabilities is not explained by
chance
Statistical independence in a data set can be tested using the chi-squared
test for independence
Page 2.52
o Multiplying probabilities
If the events are independent, we can multiply the probabilities of each of
the events in the sequence
If the events are dependent, we need to know the probabilities for each
event conditional on the previous events in the sequence
Joint probability of those events – The probability of the concomitant
occurrence of any number of events
The joint probability of two events, E and F = P(E and F) = P(E, F)
P(E and F) = P(F) x P (E | F) = P(E) x P (F |E)
If E and F are independent, P(E and F) = P(E) x P(F)
Page 2.53
- Expected Value
o Averaging out – Events are combined by applying the basic laws of conditional and
joint probability
o If cost of prophylaxis is $10,000 and the cost of treatment is $500,000. Patient has
15% probability of HIV. Effectiveness of prophylaxis is 80%. Doctor has 0.5% of
developing HIV
o The expected cost of prophylaxis E[X] = 0.15 x ( 0.005 * (1 – 0.8) * ($500,000 +
$10,000) + 0.995 x (10,000)) + 0.85 x (0 * ($500,000 + $10,000) + 1 * (10,000)) =
$10,075
Page 2.54
o Averages, expected values, and the law of large numbers
The law of large numbers - If you observe enough patients and the
probabilities are correct, the average will tend to be very close to the
expected value
Page 2.55
- Summary
o Verbal expressions of uncertainties create 2 kinds of problems:
There is wide variation in the interpretation of probabilistic terms
If verbal expressions are assigned to different uncertainties in a complex
problem, there is no method for combining them into a single expression
o Using probabilities and related numerical expressions to talk about uncertainty
solves both problems
The numbers are more precise than words
There are well-defined rules for combining probabilities mathematically
o 3 major types of uncertainties in health care:
Diagnostic uncertainty = True underlying causes of illness
Prognostic uncertainty = future course of events
Treatment uncertainty = the effects of treatment are imperfect
o All of these uncertainties can be expressed as chance events in a balance sheet or
chance nodes in a decision tree
Page 2.56
- Checklist A: Assessing studies of diagnostic probabilities
o The ideal study of diagnostic probabilities examines a consecutive series (or random
sample) of persons with the clinical presentation of interest and applies a
comprehensive diagnostic workshop with adequate followup of those initially
undiagnosed.
o The specific design features to check are:
Did the study population represent the full spectrum of those who present
with this problem?
Were the criteria for each final diagnosis explicit and credible?
For initially undiagnosed patients, was the follow-up sufficiently long and
complete?
Was the diagnostic workup evaluated and described in sufficient detail?
- Checklist B: Assessing studies of prognosis
o In the ideal study of prognosis, a large representative sample of patients with the
condition is followed to the end stage of the condition.
o The specific design features to check are:
Was an inception cohort of persons, all initially free of the outcome of
interest, followed?
Were at least 80% of patients followed until either a major study end point
or completion of the study?
Were all relevant outcomes reported and done so accurately?
Page 2.57
- Checklist C: Assessing studies of treatment or prevention
o In the ideal study of an intervention, the only difference between 2 groups of
patients would be the use or nonuse of the intervention. All patient characteristics,
co-interventions, follow-up, and outcome measurement methods should be similar
in the groups.
o The specific design features to check are:
Was allocation of participants to the different interventions random and
concealed?
Were outcomes measured for at least 80% of participants?
Were all relevant outcomes reported and done so accurately?
Were outcomes measured blinded or were they objective, when feasible?
Page 3.61
Chapter 3: Choosing the best treatment Page 3.62
- At some critical level of risk, the inoculation becomes the better strategy. This risk is known
as an action threshold
- Choosing the better risky option
Page 3.64
o The decision between treating now versus expectant management ("watchful
waiting")
o The recurring trade off problem is: if we treat now, there is a small but measurable
risk of harm that might be avoided by expectant management
o If the expectant management is selected (in this case, no inoculation), there is a
chance that the patient will not become infected and will survive
o Under these circumstances, the patient will be better off with watchful waiting
o Usually, it may not be immediately apparent whether a risky preventive measure is
preferable to watchful waiting.
- Best treatment option under diagnostic uncertainty
o Sometimes treatment must be initiated before a clear diagnosis is reached
Page 3.65
In Emergency conditions
Invasive diagnostic procedures
Residual uncertainty
o First understanding the treatment decisions is important in devising a diagnostic
strategy
o When analyzing diagnostic strategies, treatment preceded diagnosis
Page 3.66
o PROactive
Problem: need to make a decision whether or not to treat the patient with
anticoagulation
Main objective of management is to avoid a recurrent PE which may be
fatal. He or she would try to maximize the survival chance for both the
patient and her unborn child
We need to take into account that we are uncertain whether PE is present
and that anticoagulation has a small risk of fatal hemorrhage
Page 3.67
o proACTive
Management alternatives:
Intervention (treatment with anticoagulation)
Wait and see (withholding anticoagulation)
Getting more diagnostic information
o Too risky because of pregnancy
Structure the problem in the form of a decision tree
Page 3.68
The chance node is used to represent the uncertainty of the underlying true
disease status and this probability reflects one of the unknown for this
problem
After formulating the problem, considering all the possible alternatives, and
structuring the consequences in the form of a decision tree, we need to
assign probabilities to the events and values to the outcome
Each probability should be determined conditional on all the events that
preceded it
Page 3.69
Construct a balance sheet with all the alternatives
From the balance sheet we see there are risks and benefits to both options
Outcomes:
Survival
Life expectancy
Quality-adjusted life years
costs
The unit of outcome should represent the outcome we wish to optimize and
should include any tradeoffs we wish to capture in the outcomes
Page 3. 70
o proactIVE
Integrate the evidence of the event probabilities and the value of the
outcomes
To find the expected value we work backwards from the right hand side of
the tree successively averaging out at each chance note until we have
folded back the entire tree to the decision node
Averaging out refers to the process of multiplying the probability by the
outcome value for each of the events leading from a chance node and doing
this successively from right to left
Averaging out calculates the weighted average of the outcome values (the
numbers at the tips of the branches) with each outcome value weighed for
the probability that it will occur (the path probability of that branch)
The process of removing less optimal alternatives from further
consideration is called folding back
Averaging out and folding back are together referred as rolling back
to be really confident about our decision we need first to explore how our
assumptions will affect the decision in a 'what-if' analysis – a sensitivity
analysis
A sensitivity analysis is any test of the stability of the conclusions of an
analysis over a range of structural assumptions, probability estimates, or
outcome values
Page 3.71
To evaluate whether our results apply under other assumptions, we can
repeat the analysis substituting a range of estimates for the probabilities in
question to see whether this alters the conclusion of the analysis
There is a probability where we switch between the options of AC versus no
AC – the treatment threshold (treat – do not treat threshold)
Below this threshold, withholding treatment is better
Above the threshold, treatment is better
At the threshold, treatment and no treatment are exactly equal
The treatment threshold for diagnostic uncertainty is the probability of
disease at which the expected value of treatment and no treatment are
exactly equal, and neither option is clearly preferable.
Page 3.73
Numerical sensitivity and threshold analysis
Sensitivity analysis can be performed numerically by constructing a
decision tree and inserting the uncertain variable, in this case, the
disease probability, as a variable
We then plot or tabulate the expected values for all values of this
probability in a table
Page 3.74
Algebraic sensitivity and threshold analysis
Construct the decision tree and insert the disease probability as a
variable
Expected value (AC) = 0.990 x P(PE) + 0.992 x (1 – P(PE))
Expected value (No AC) = 0.75 x P(PE) + 1.0 x ( 1- P(PE))
Set Expected value (AC) = Expect value (No AC), solve for P(PE)
Graphical sensitivity and threshold analysis
Compare the benefits and harms of treatment directly and visualize
them graphically
Page 3.75
The smaller the harm relative to the benefit, the lower the
treatment threshold should be
The larger the harm relative to the benefit, the higher the treatment
threshold should be
In a graph, the harms are indicated on the left axis, the benefits on
the right axis, and the treatment threshold is the pivot point
Benefit of a treatment is the difference in outcome in patients with
the disease who receive treatment and similar patients who do not
receive treatment
Benefit = utility (treatment | disease) – utility (No treatment |
disease)
Utility is the value of the outcomes to the patient
A utility with a positive sign implies the outcome is desirable
A utility with a negative sign implies it is an undesirable outcome
The benefit of treatment is the difference in outcome between AC
vs withholding AC
Page 3.76
(1 – 0.01) – (1 – 0.25) = 0.99 – 0.75 = 0.24 = an increase in the
survival chances of 0.24
In terms of disutility, the benefit is 0.01 – 0.25 = -0.24
The harm of a treatment is the difference in outcome between
patients without a disease who do not receive treatment and similar
patients who do receive the treatment
Harm = utility (no treatment | no disease) – utility (treatment | no
disease)
(1 – 0) – (1 – 0.008) = 0.008
Benefit is marked on the right hand axis by indicating the expected
outcomes with and without treatment in a group with the disease
(P(PE) = 1)
Harm is marked on the left hand vertical axis by indicating the
expected outcomes with and without the treatment in those
without the disease (P(PE) = 0)
The line joining the value of treating the nondiseased (at probability
0) and the values of treating the diseased (at probability 1)
represents the expected value of treatment over the range of values
from 0 to 1
The expected value of no treatment is the line joining the value of
withholding treatment from the nondiseased (at probability 0) and
the value of withholding treatment from the diseased (at probability
1)
These 2 lines cross at the treatment threshold where the expected
value of the 2 options are equal.
Treatment threshold = harm / (harm + benefit)
If harm is much smaller than the benefit, the treatment threshold
should be low
Page 3.77
If harm is large compared with the benefit, the treatment threshold
should be high
If benefits and harms are equal, the treatment threshold is0.5
The harms and benefit are in the same ratio as the threshold
probability P and its complement (1 – P)
P / (1 – P) = harm / benefit
P = harm / (harm + benefit)
Treatment threshold = 0.008 / (0.008 + 0.240) = 1 / 31 = 0.32
Subjective Treatment threshold estimates
A quick approximate and subject alternative should be to ask: "How
many times worse is not treating a case of true disease compared to
unnecessarily treating a case without the disease?
If you answer is N times, then the compounding treatment
threshold is 1 / (N+ 1)
Page 3.78
o On way, two way, three way, and n-way sensitivity analysis
In 2 way sensitivity analysis, the effect of simultaneous changes in 2 variable
values is evaluated
E.g . What the threshold probability of PE would be if the risk of a fatal
recurrent PE without anticoagulation were to be lower than the initially
estimated value of 0.25
Page 3.79
In n-way sensitivity analysis we vary multiple variable values at the same
time
An n-way sensitivity analysis is useful to evaluate the results for a different
setting, for different types of patients, and for best-worst case scenarios
- The decision to obtain diagnostic information and the do's and don'ts of tree building
Page 3.80
o The aim of obtaining more diagnostic information would be to shift our probability
assessment across the treatment threshold
o Performing a diagnostic test to obtain additional information is worthwhile only if at
least one decision would change by the test results and if the risk to the patient
associated with the test is less than the expected benefit that would be gained from
the subsequent change in the decision
o After performing the diagnostic test, we will know which option to take, we can
prune the tree by eliminating the branches that would never apply
o Changed from an extensive form to one in strategic form
o The options are expressed as strategies in terms of 'do A. if X, then do B. If Y, then
do C'
o We can redraw a tree in strategic form by bringing all the decision nodes up front
and instead of only letting the immediate decision lead from the initial decision
node, we define the entire set of diagnostic strategies up front
o A decision tree in embedded form contains no embedded decision nodes but
instead all options are strategies
o In building decision trees one needs to be careful about sequencing decision nodes
and chances nodes in the correct order
o Going from left to right, a decision tree generally depicts the sequence of events as
they occur over time (in chronological order)
Page 3.81
o Sometimes it can be convenient to model it the other way around: Model disease
status first followed by the test result
o Permissible only if there are no intervening decisions or events that may influence
the course of the disease or affect probabilities thereafter
o You need to let the next management decision depend on the test result, which you
observe, and not on the true disease status
o We model both the observable reality and the underlying truth but our decisions
can only be based on the observable world
Page 4.88
Chapter 4: Valuing Outcomes - In most cases, decisions between alternative strategies require not only estimates of
probabilities of the associated outcomes, but also value judgments about how to weigh the
benefits versus the harms
Page 4.89
- Value judgments about the quality of life are especially important when a disease cannot be
definitively cured and a patient may lie many years in a state of considerably less than
perfect health, or when a treatment carries some risk of severe side effects
- Decision making paradigms
Page 4.90
o Questions to consider:
Who is the decision maker
What information does the decision maker need
How can the decision maker be helped to clarify his / her values
o The clinical encounter
Page 4.92
Presented with all the requisite information, active decision makers are in a
position to apply their own values to choose the best treatment for them
This process can be facilitated with decision aids, that clearly presented the
trade-offs involved in choosing between treatments
In this paradigm the choice is essentially made in a 'black box'.
The process of combining the relevant probabilities and values is intuitive
(in the black box), not explicit
To give recommendations, several directed questions to patients to elicit
their feelings about the key outcomes affected by the choice may be
enough to allow the physician to make a tailored recommendation
Page 4.95
o Societal decision making
Medical decisions are made for classes or groups of patients, at a level
removed from the encounter between the individual patient and physician
E.g. clinical practice guidelines specify how patients in particular clinical
circumstances should be treated
Guidelines are designed to eliminate variation in patterns of care that
represent deviations from what is believed to be the most effective therapy
for a given disease
Medical decisions are made in the formulation of both guidelines and
resource allocation decisions, but they are made on behalf of groups of
people
- Attributes of outcomes
o Recognize that some decisions involve more complicated outcomes than others
Page 4.96
o Two possible outcomes
The criteria for decision making is simply to choose the strategy that gives
the highest probability of the better outcome, or the lower probability of
the worse outcome
Use the method of average out
The process of combining probabilities
o Many possible outcomes: the single attribute case
Often there is an underlying scale associated with the outcomes
The most commonly used single attribute outcome is survival time
We might wish to modify the underlying scale to account for the possibility
that a person might be risk averse or place greater importance on outcomes
occurring in the near term than later in the future
o Many possible outcome: the multiattribute case
There are 2 or more dimension or values
Page 4.97
one must decide how to make trade-offs between the competing values
associated with the 2 dimensions or attributes
Need a scale that reflects the importance of both attributes
- Quality Adjusted survival
o In a quality of life scores, the area under the curve is a function of both the length
and quality of life of a patient
o The area under the curve might function as a metric for valuing the 2 attributes of
life on a single scale
o Life needs to be measured in a way that the product of length of life and quality of
life is meaningful
o Characteristics of quality of life measures:
Page 4.98
A global evaluation of a state of health: Should reflect all aspects of the
state of health being assessed
Measured on a ratio scale between extremes of perfect health and death
Use length of life as the metric for measuring the subject's preference for
the quality of life in a given health state
Utility – the quantitative measure of the strength of a person's preference
for an outcome
Quality adjusted survival, measured in quality adjusted life years (QALYs)
- Techniques for valuing outcomes
o Generic quality of life instruments capture information on the nature of the quality
of life impairments of respondents. These scales are often summarized into scores
for several domains, like physical functioning, emotional functioning, pain, and
others. Most do not measure global quality of life directly or indirectly. They do not
capture preferences for a given state of health on a scale that lends itself to being
averaged out
o Utility measures reflect how a respondent values a state of health, not just the
characteristics of that health state
Page 4.99
o A utility scale (or utility function) is an assignment of numerical values to each
member of a set of outcomes, such that if the expected value of the utilities
assigned to the outcomes in one chance tree is greater than the expected value of
the utilities assigned to the outcomes in another chance tree, then the first chance
tree is preferred to the second chance tree
o A true utility scale is one that can be averaged out in a decision tree without
distorting the preferences of the individual whose preferences are represented
o If 20 years = 20 QALYS, 10 year = 10, 0 years = 0. A 50-50 gamble between 20 years
and 0 years is 10. If patient prefers 9 years over the gamble, this cannot be a utility
scale for the patient. A utility scale which assigns to each life span the square root
of the length of life would be consistent with the ranking of options because 0.5 x
sqrt(20) + 0.5 x sqrt(0) < sqrt(9)
o A scale of QALYs is frequently used in decision analysis and in economic evaluations
as a utility measure
o But it is not guaranteed that quality adjusted life expectancy will reflect a decision
maker's preferences regarding decisions under uncertainty
o Several different strategies for capturing such preferences based measures of
quality of life
Page 4.100
Rating Scale:
E.g. on a scale where 0 represents death and 100 represents
excellent health, what number would you say best describes your
current state of health over just the past 2 weeks
The rating scale is a global measure that captures a subject's
valuation of a particular state of health
It is easily explained to most people and it is easy to administer
It is not a true utility because it is not a ratio scale between perfect
health and death (A person who rates a state of health at 50 would
not trade away half of his life expectancy to be relieved of that
impairment
The rating scale does not satisfy the criterion of expected value
Standard gamble
It assesses the utility for a health state by asking how high a risk of
death one would accept to improve it
Choose between life in a given clinical state and a gamble between
death and perfect health
The utility of the health state is given by the probability of perfect
health in the gamble such that the respondent is indifferent
between the gamble and the certain intermediate outcome
Page 4.102
In the standard gamble, the iterative process is repeated, varying
the probabilities in the gamble, until the respondent feels that he 2
options are equally desirable. This is called the point of indifference
At the point of indifference, the respondent's utility for the health
state is given by the probability that the treatment will work
If you reached the point of indifference when offered a treatment
with a 95% probability of permanent relief and a 5% of immediate
death, your utility for the health state would be 0.95
The disutility of the health state = 1 – utility = 0.05
For A choice between life in a given clinical state H and a gamble
between death (with probability = 1 – P) and perfect health (with
probability P). A value of 1 to perfect health and 0 to death. The
expected value of the gamble = P x 1.0 + (1 – P) x 0 = P
Page 4.103
This expected value is the probability of getting perfect health in the
gamble
If the respondent considers the gamble equally desirable as a health
state H, then the utility of that health state must be P: u(H) = P
By varying the probability P in the standard gamble and the health
state, we can find the utility of any health state between perfect
health and death by a series of choices between gambles
A chained gamble – An anchor health state be evaluated in relation
to perfect health and the anchor health state
Indifferent – A situation in which an individual is equally happy with
2 outcomes or gambles.
Standard gamble reflects decision making under uncertainty
Your utility measured with a standard gamble reflects not only your
preferences about life in that state of health, but also your attitudes
toward risk
Time trade off
The utility for a health state is assessed by asking how much time
one would give up to improve it
Choose between a given compromised health state and a shorter
length of life in perfect health
Page 4.104
The respondent's utility for the compromised health state is given
by the ratio of the shorter to the longer life expectancy at which the
respondent finds the 2 health states equally desirable
In time trade off, the iterative process is repeated varying the length
of life in perfect health, until the respondent feels that the 2 options
are equally desirable – the point of indifference
The respondent's utility for the health state, is given by the ratio of
the length of life in perfect health to the length of life in the
compromised health state
A the point of indifference, the subject believes that life
characterized by time t spent in health state with utility u is
equivalent to the life with length x, spent in perfect health with
utility '1' : u = x/t
E.g. if a treatment reduce life expectancy from 40 to 38, the utility =
38/40 = 0.95
Page 4.105
The time trade off represents decision making under certainty
The utility is unaffected by your attitude toward risk
Other techniques for valuing outcomes
Willingness to pay: how much the respondent would be willing to
pay in financial terms to improve a state of health = cost benefit
analysis
Cost effectiveness analyses – health outcomes are expressed in
terms of quality adjusted life year
Magnitude estimation – ask how many times better or worse one
health state is than another
Equivalence measures (person trade off) – ask the respondent to
indicate how many people have to cured of one health state to be
equivalent to curing 100 people in another impaired health state
- Comment on nomenclature
Page 4.106
- Relationships among techniques for valuing outcomes
o When the same subjects are asked to evaluate health status using each of the
measures, the results are not identical
o Utilities elicited with standard gamble are the highest
o Most subjects are unwilling to accept much risk of immediate death
o Utilities elicited with the time trade of tend to be lower – a known, limited decrease
in life expectancy is more acceptable price to pay for being relieved of a quality of
life impairment than a low probability of immediate death
o Rating scale values tend to be considerable lower than utilities generated by either
of the other 2 techniques
Page 4.107
o To transform the values in a rating scale to a utility score, use power function
Utility = 1 – (1 – value)^r, where r ranges from 1.6 to 2.3
- Health indexes
o Assessment of utilities that is in some ways a hybrid between descriptive quality of
life measurement and utility measurement
o Multi-attribute utility measures - E.g. the Health Utilities Index (HUI) and the
EuroQol
o 2 components:
a health state classification instrument
a formula for assigning a utility to any unique set of responses to that
instrument
o The health state classification instrument measures health related quality of life
Generates descriptive data regarding the quality of life of the patient who
completes it
o the special feature of a health index is the mapping rule. by polling members of a
reference population to elicit their responses for all of the states of health
Page 4.108
- Off the shelf utilities
o BeaverDam Health Outcomes Study
o National health interview survey
- Health state worse than death
Page 4.109
o It is possible to have utilities less than 0
- Practical considerations in utility measurement
o Utility techniques are challenging to measure
o Best done in an interactive format
o Provide subjects with visual aids
Page 4.110
- Risk Aversion and time preference
o Risk averse – many individuals would opt for a smaller amount of money, offered
with certainty, over a gamble with a higher value on average
Page 4.111
o In a utility cure illustrating risk aversion and risk neutrality
The utilities of an individual who is risk neutral would lie in the straight
diagonal line
The utilities of an individual who is risk-averse would form a concave curve
above and to the left of the diagonal
An individual who prefers a gamble with a small probability of a very large
payoff over a certain outcome with the same expected value is said to be
risk-seeking. The utility curve would lie below the diagonal
o For risk neutral utility function, the certainty equivalent is equal to the expected
value
o For a risk averse utility function, the certainty equivalent is greater than the
expected value
o For a risk seeking utility function, the certainty equivalent is smaller than the
expected value
o The certainty equivalent of a gamble is the outcome along the scale such that the
decision maker is indifferent between the gamble and that certain outcome
o For time preference, most people would favor an intervention that increased life
expectancy from 1 year to 2, over one that increased life expectancy from 9 years to
10
Page 4.112
o Future years are valued less than those in the near term
o The longer the life expectancy in impaired health, the longer the proportion of that
life expectancy one might be willing to give up in exchange for perfect health
o The longer the life offered in the time trade off, the lower the utility for that
impaired state of health
- Quality adjusted life expectancy as a utility
o To be amenable to being averaged out, Health state utilities must reflect
preferences under uncertainty
o To be amenable to being used as the weights in quality adjusted survival, they must
reflect time trade offs
o To use averaged out life span as a utility (i.e. Life expectancy), preferences must be
risk neutral with regard to longevity
o All 3 criteria must be met in order to be able to use quality adjusted survival to
reflect preferences in a decision analysis
o If an individual's preferences satisfy only 2 conditions, then quality –adjusted
survival can represent his preferences:
Constant proportional trade-off: the proportion of life span that an
individual would give up in order to improve health from a health state to
perfect health does not depend on the length of life
Risk neutrality on survival
- Other psychological issues in utility assessment
o Eliciting and clarifying a patient's preference is inherently a psychological process
o Decision making must be made prospectively, before the patient has experience
with the outcome of the treatment intervention
o A utility assessment of a yet to be experienced outcome may judge to be worse than
it will subsequently prove to be, or the new reality may turn out to be worse than
they had feared
o People may not have well-formed preferences. Their responses to questions may
be generated on the fly
o Utility elicitation is subject to framing effects
o Procedural invariance: Choices should be stable over minor changes in the wording
of the problem
Page 4.114
o Expected utility theory is a prescriptive theory
o It is a small world theory
- Discussion: decision making paradigms revisited
o The clinical encounter with the active decision maker
Unusual decision should be accepted and honored
However, it is the physician's responsibility to attempt to help patients
avoid making decisions that are inconsistent with their underlying goals and
values
Page 4.115
o The clinical encounter with the patient wanting guidance
The idea of tailored medical decision making can only be achieved if the
individual patient's preferences are somehow brought to bear on the choice
among alternatives
Formal utility elicitation with formal preference assessment
Experiences of other patients who have faced similar choices may provide
some guidance
Page 4.116
o Societal decision making: clinical guidelines
Judgments should reflect the preference of the population
o Societal decision making: resource allocation
An international panel of experts has identified quality-adjusted survival as
the preferred outcome for use in cost effective analysis
All cost effectiveness analyses should include a 'reference case' conducted
from a societal perspective
Page 4.117
- Summary
o Many decisions require value judgment
o To compare strategies we need a scale that combines the important attributes in
one metric
o Quality adjusted survival, measured in QALYs, provides such a scale
o QALYs fulfill the criteria of a utility – they can be considered a quantitative measure
of the strength of a person's preference for an outcome
Page 5.128
Chapter 5: Interpreting diagnostic information - Diagnostic information and probability revision
o We must know how to interpret and select information to minimize the impact of
such errors
Page 5.130
o The pretest probability of disease is the probability of the presence of the target
disease conditional on the available information prior to performing the test under
consideration
o The post test probability of disease is the probability of the presence of the target
disease conditional on the pretest information and the test result
o Probability revision is the process of converting the pretest probability to the
posttest probability taking the test result into account
o Prevalence and pretest probability
If a patient were chosen at random from a given population, the pretest
probability of disease for the patient would be the disease prevalence in the
population
However, patients are not selected at random
Page 5.131
Disease prevalence is the frequency of existing disease in the population of
interest at a given point in time
Specific characteristics, including history, physical findings, and previous test
results, along with the disease prevalence, determine the probability that an
individual has any given disease at any point in time
This probability is conditional upon already available information and may
be taken as the pretest probability with respect to a subsequent test
o The 2 x 2 table for the FOBT and colorectal cancer
Page 5.132
o 2 important conditional probabilities: sensitivity and specificity
Sensitivity = True positive ratio = TPR = P(T+ | D+) = The probability of a
positive test result given that the disease is present
Page 5.133
Specificity = True negative ratio = TNR = P(T- | D-) = The probability of a
negative test result given that the disease is absent
False Negative ratio = FNR = 1 – TPR = 1 – sensitivity = P(T- | D+) = The
proportion of patients with disease who have a negative test result
False positive ratio = FPR = 1 – TNR = 1 – specificity = P(T+ | D-) = The
proportion of patients without the disease who have a positive test result
A sensitive test, one with a high true positive (and low false negative) ratio
is very good at detecting patients with the target disease (sensitive to the
presence of disease)
A specific test, one with high true negative (and low false positive) ratio, is
very good at screening out patients who do not have the disease (specific to
that disease)
A test may have a high sensitivity and a low specificity
Page 5.134
An ideal test has a true positive ratio of 1.0 and a true negative ratio of 1.0
o Post test probabilities: the postpositive test and post negative test probabilities
Postpositive test probability of disease = Predictive value positive of a test =
PV+ = P(D+ | T+) = TP / (TP + FP) = The conditional probability of a disease
given a positive test result
Postnegative test probability of disease = = P(D+ | T-) = FN / (FN + TN) = The
conditional probability of having the disease given a negative test result
Predictive value negative = PV- = P(D- | T-) = the probability that a patient
with a negative test does not have the target disease
Page 5.136
The predictive value positive and the Predictive value negative are both
examples of posttest probabilities
To estimate the posttest probabilities for our patients, we need an
independent estimate of the probability of the disease in the population
from which our patient is selected, an estimate of the pretest probability of
disease
Page 5.138
o Probability Revision
Start with a pretest probability of a disease, observe a test result, revise the
probability to obtain a posttest probability of a disease given the positive
test result
Page 5.139
o The effect of prevalence in screening
The posttest probability depends strongly on the pretest probability of the
group we apply the test to, and strongly influence both how the person
should be managed and what he should be told
Page 5.142
- Bayes formula
- P(D+ | T+) = 𝑃 𝑇+ 𝐷+)∗𝑃(𝐷+)
𝑃 𝑇+ 𝐷+)∗𝑃 𝐷+ + 𝑃 𝑇+ 𝐷−)∗𝑃(𝐷−)
- Bayes' formula for a dichotomous (+ or -) test and disease states
- Postpositive-test probability =
𝑆𝑒𝑛𝑠𝑖𝑡𝑖𝑣𝑖𝑡𝑦 𝑥 𝑝𝑟𝑒𝑡𝑒𝑠𝑡 𝑝𝑟𝑜𝑏𝑎𝑏𝑖𝑙𝑖𝑡𝑦
(𝑆𝑒𝑛𝑠𝑖𝑡𝑖𝑣𝑖𝑡𝑦 𝑥 𝑝𝑟𝑒𝑡𝑒𝑠𝑡 𝑝𝑟𝑜𝑏𝑎𝑏𝑖𝑙𝑖𝑡𝑦 + 1−𝑠𝑝𝑒𝑐𝑖𝑓𝑖𝑐𝑖𝑡𝑦 𝑥 1−𝑝𝑟𝑒𝑡𝑒𝑠𝑡 𝑝𝑟𝑜𝑏𝑎𝑏𝑖𝑙𝑖𝑡𝑦 )
- P(D+ | R+) = 𝑃 𝑅 𝐷+)∗𝑃(𝐷+)
𝑃 𝑅 𝐷+)∗𝑃 𝐷+ + 𝑃 𝑅 𝐷−)∗𝑃(𝐷−)
- P(D+ | T-) = (1− 𝑃 𝑇+ 𝐷+))∗𝑃(𝐷+)
(1− 𝑃 𝑇+ 𝐷+))∗𝑃 𝐷+ + (1− 𝑃 𝑇+ 𝐷−))∗𝑃(𝐷−)
- Postnegative-test probability =
(1− 𝑆𝑒𝑛𝑠𝑖𝑡𝑖 𝑣𝑖𝑡𝑦 ) 𝑥 𝑝𝑟𝑒𝑡𝑒𝑠𝑡 𝑝𝑟𝑜𝑏𝑎𝑏𝑖𝑙𝑖𝑡𝑦
((1− 𝑆𝑒𝑛𝑠𝑖𝑡𝑖𝑣𝑖𝑡𝑦 ) 𝑥 𝑝𝑟𝑒𝑡𝑒𝑠𝑡 𝑝𝑟𝑜𝑏𝑎𝑏𝑖𝑙𝑖𝑡𝑦 + 𝑠𝑝𝑒𝑐𝑖𝑓𝑖𝑐𝑖𝑡𝑦 𝑥 1−𝑝𝑟𝑒𝑡𝑒𝑠𝑡 𝑝𝑟𝑜𝑏𝑎𝑏𝑖𝑙𝑖𝑡𝑦 )
Page 5.143
- Bayes's theorem with tree inversion
o A chance tree first divides into D+ vs D- with the associated pretest probability of
disease
o The probability of a positive vs negative test results are depicted
o The chance tree represents the pretest probability of disease and the test sensitivity
and specificity
o To calculate the posttest probabilities of disease we need to invert the chance tree
so that the tree first models the test result and then the disease status conditional
on the test result
o The numbers are copied to the ends of the branches of the inverted chance tree
o The frequency of a positive test result is calculated by summing the true positive
and false positive results
o The frequency of a negative test result is calculated by summing the true negative
and false negative results
o we can calculate the posttest probabilities by dividing the path frequencies by the
test result totals
Page 5.145
- The odds likelihood ratio form of Bayes formula
o Odds
odds favoring the occurrence of an event = P / (1 – P)
Odds against the occurrence of an event = ( 1- P) / P
As probability varies from 0.0 to 1.0, the corresponding odds favoring range
from 0 to infinity
The odds against range from infinity to 0
P = O / (1 + O) (O = Odds favoring the event)
P = 1 / ( 1 + OA) (OA = Odds against the event)
o Probability revision using odds
Page 5.146
Pretest odds favoring disease = 𝑃 (𝐷+)
𝑃(𝐷−)
Posttest odds given the test result = odds corresponding to the posttest
probability = 𝑃 𝐷+ 𝑅)
𝑃 𝐷− 𝑅)
Posttest odds = 𝑃 𝐷+ ∗𝑃 𝑅 𝐷+)
𝑃 𝐷− ∗𝑃 𝑅 𝐷−)
Likelihood ratio for the test result R = 𝑃 𝑅 𝐷+)
𝑃 𝑅 𝐷−)
The likelihood ratio associated with a test result is the ratio of its probability
of occurrence if the disease is present to its probability of occurrence if the
disease is absent
Likelihood ratio for a positive test result = LR+ = 𝑃 𝑇+ 𝐷+)
𝑃 𝑇+ 𝐷−) =
𝑇𝑟𝑢𝑒 𝑝𝑜𝑠𝑖𝑡𝑖𝑣𝑒 𝑟𝑎𝑡𝑖𝑜
𝐹𝑎𝑙𝑠𝑒 𝑝𝑜𝑠𝑖𝑡𝑖𝑣𝑒 𝑟𝑎𝑡𝑖𝑜
LR+ = sensitivity / (1 – specificity) = TPR / FPR
Likelihood ratio for a negative test result = LR- = 𝑃 𝑇− 𝐷+)
𝑃 𝑇− 𝐷−) =
𝐹𝑎𝑙𝑠𝑒 𝑛𝑒𝑔𝑎𝑡𝑖𝑣𝑒 𝑟𝑎𝑡𝑖𝑜
𝑇𝑟𝑢𝑒 𝑛𝑒𝑔𝑎𝑡𝑖𝑣𝑒 𝑟𝑎𝑡𝑖𝑜
LR- = (1- sensitivity) / (specificity) = FNR / TNR
LR+ = (sensitivity) / (1 –specificity) = 0.83 / (1 – 0.96) = 20.8
The pretest probability = 0.08
The pretest odds = P / (1 – P) = 0.08 / ( 1- 0.08) = 0.087
Post test odds = pretest odds * LR+ = 0.087 * 20.8 = 1.8
Post test probability = O / (1 + O) = 1.8 / (1 + 1.8) = 0.64
Page 5.149
A nomogram : log(posttest odds) = log (pretest odds) + log(LR)
Given the likelihood ratios and the pretest probability, the posttest
probability can be read off with a ruler.
Page 5.149
- Finding subjective information about pretest probabilities
o Where did the pretest probability came from
o Clinicians sometimes have to rely upon subjective probabilities – personal opinions
formulated as probabilities
Page 5.150
o 3 heuristic principles for subjective probability estimates
Availability: Reliance on the easily recalled
Memory is affected by factors other than frequency and probability
More recent events are often better remembered than more
distant one
Memory is also affected by how strange and unusual an event is
Page 5.151
Representativeness: focusing on features at the neglect of prevalence
The probability of a disease is judged by how closely the clinical
picture resembles a larger class of events
Representativeness heuristic is insensitive to pretest probabilities
Anchoring and adjustment: under adjustment for new information
The published estimates serve as an anchor, and the subjective
probabilities of the prevalence are the result of adjustment
Page 5.152
Value induced bias
In decision analysis, estimates of probability (the likelihood of an
event) and utility (which reflects its value) should be made
independently and kept in separate accounts, to be combined
during the stage of evaluation
In medicine, the probability of serious illness may be overestimated
- Finding and assessing the quality of studies of test accuracy
o In MEDLINE, using keywords such as diagnosis and sensitivity-and-specificity
Page 5.153
- Summary
o Probability revision is the process of converting the pretest probability of disease to
the posttest probability of disease taking the test result into account
o Probability revision can be performed with a 2x2 table, Bayes' formula, tree
inversion, odds likelihood ratio form of Bayes' theorem, or with a nomogram
o The estimate of the probability of disease prior to performing the test is combine
with the information from the test result to derive the probability of disease after
performing the test
