Tutorial on Risk Adjusted P-chart

Farrokh Alemi, Ph.D.

Have Changes Led to Improvement?

Common cause variation (changes in outcomes because of chance) is everywhere.

Decision makers often mistakenly attribute positive outcomes to their own skills and

negative outcomes to others, while in reality both could be a chance outcome

Why Chart Data?To discipline intuitionsTo communicate data in vivid graphical ways

Purpose of Risk Adjustment

Purpose of p-chart Purpose of risk-adjusted p-chart

To detect if the process has improved beyond

historical levels. Assumes historically we have been serving the same type of

patients as now

To detect if the process has improved beyond what can be expected from patient conditions

Data NeededData collected over timeRisk (expected outcomes) for each patientOutcomes for each patient

The purpose is to improve not to get so lost in measurement to loose sight of improvement.

What Is Risk?A patient’s condition or characteristics that affects the expected outcomes for the patient A severity index used to predict patient

outcomesClinicians’ consensus regarding expected

outcomesPatient’s self rating of expected outcomes

Time 1 Time 2 Time 3 Time 4 Time 5 Time 6 Time 7 Time 8# deaths 2 3 1 3 2 2 4 2

8 9 7 7 7 7 7 8

Case1 0.18 0.97 0.85 0.27 0.12 0.07 0.96 0.05

2 0.88 0.88 0.61 0.71 0.44 0.05 0.05 0.96

3 0.33 0.04 0.27 0.07 0.18 0.93 0.75 0.96

4 0.29 0.29 0.28 0.74 0.67 0.24 0.04 0.14

5 0.14 0.03 0.8 0.08 0.51 0.14 0.96 0.05

6 0.24 0.19 0.71 0.04 0.62 0.58 0.71 0.58

7 0.15 0.14 0.85 0.76 0.67 0.05 0.15 0.07

8 0.04 0.74 0.16

9 0.07

Mortality Risks for Inidvidual Patients

MI Patients Over 8 Months in One Hospital

Observed mortality during this time period

Num

ber

of c

ases

Expected probability of mortality for case 8 in time period 1. Estimated from severity indices or experts’ consensus.

Elements of a Control Chart

X axis shows timeY axis shows probability of adverse eventsObserved rates are plotted against time sequenceUpper control limit is a line drawn so that points above it are rare to be observed by mere chanceLower control limit is a line drawn so that points below it are rare to observe by mere chance

Lets take a look

An Example of P-chart

Example p-chart

00.20.40.60.8

0 1 2 3 4 5 6 7

Time

Prob

abili

ty o

f ad

vers

e ou

tcom

eUpper control limit

Lower control limit

Observed rate

Steps in Creating P-chart for Mortality

1. Check assumptions2. Calculate observed rates and plot them3. Calculate expected rates and plot them4. Calculate expected deviation5. Calculate control limits and plot them6. Interpret findings7. Distribute chart and interpretation

Step One: Check Assumptions

We are examining discrete events that either happen or do not happen, e.g. mortality among MI patients, falls among nursing home patients, error in medical record entry, etc.The event is not rare, meaning the probability of it occurring exceeds 5% for each time period.Observed events are independent from each other. The probability of the event occurring does not change over time. This assumption is violated if one patient’s outcomes affects the outcomes for others, e.g. when dealing with infectious diseases.

Step Two: Calculate Observed Rates and Plot

Pi = Mortality rate in time period “i”

Oi = Mortality in period “i”

ni = Number of cases in time period “i”

Pi = Oi / ni

Number Observed Mortalityof cases Mortality rate

8 2 0.25

9 3 0.33

7 1 0.14

7 3 0.43

7 2 0.29

7 2 0.29

7 4 0.57

8 2 0.25

Observed Mortality Rates for All Time Periods

Plot of mortality rates

PeriodNumber of cases

Observed mortality

Expected mortality

Expected deviation

Upper limit

Lower limit

T1 8 0.25

T2 9 0.33

T3 7 0.14

T4 7 0.43

T5 7 0.29

T6 7 0.29

T7 7 0.57

T8 8 0.25

Plot of the Observed RatesTime period 7 and 3 seem different but don’t rush to judgment.Wait, until you see control limits of what could have been expected.

0.000.100.200.300.400.50

1 2 3 4 5 6 7 8

Time periodM

orta

lity

rate

Step Three: Calculate Expected Mortality

Eij = Expected mortality of case ‘j’ in time period “i”

Ei = Expected mortality for time period “i”

Ei = (j=1,…,ni Eij ) / ni

Sample calculation:E1= (.18+.88+.33+.29

+.14+.24+.15+.04)/8

Expected Mortality Rates for All Time Periods

PeriodNumber of cases

Observed mortality

Expected mortality

Expected deviation

Upper limit

Lower limit

T1 8 0.25 0.28

T2 9 0.33 0.37

T3 7 0.14 0.62

T4 7 0.43 0.38

T5 7 0.29 0.46

T6 7 0.29 0.29

T7 7 0.57 0.52

T8 8 0.25 0.37

Plot of Expected Mortality

Plotting expected mortality helps interpret the observed rates but does not settle the question of whether differences are due to chance.

0.00

0.20

0.40

0.60

0.80

T1 T2 T3 T4 T5 T6 T7 T8Pr

obab

ility

of m

orta

lity

Observed mortality Expected mortality

Step Four: Calculate Expected Deviation

Eij = Expected mortality of case ‘j’ in time period “i”

Di = Standard deviation of expected mortality in time period “i”, called by us as the Expected Deviation

Di = ( j=1,…,ni Eij (1-Eij))0.5 / ni

See sample calculation

Expected Deviation for Time Period 1

Case

Expected mortality E1j E1j *(1-E1j)

1 0.18 0.15

2 0.88 0.11

3 0.33 0.22

4 0.29 0.21

5 0.14 0.12

6 0.24 0.18

7 0.15 0.13

8 0.04 0.04

1.15

1.07

0.13Divided by # of casesSquare root of sum

SumExpected deviation

A

BC

D

Results: Expected Deviations for All Time

PeriodsPeriod

Number of cases

Observed mortality

Expected mortality

Expected deviation

Upper limit

Lower limit

T1 8 0.25 0.28 0.13

T2 9 0.33 0.37 0.11

T3 7 0.14 0.62 0.16

T4 7 0.43 0.38 0.14

T5 7 0.29 0.46 0.17

T6 7 0.29 0.29 0.13

T7 7 0.57 0.52 0.05

T8 8 0.25 0.37 0.11

Step Five: Calculate Control Limits

UCLi = Upper control limit for time period “i”

LCLi = Lower control limit for time period “i”

t = Constant based on t-student distribution UCLi = Ei + t * Di

LCLi = Ei - t * Di

Where for 95% confidence intervals:Degrees of

freedom

95% student t

value7 2.37

8 2.31

9 2.26

10 2.23

Calculation of Control Limits for Time Period 1

UCL = .28 + 2.37 * .13 = .59

LCL = .28 –2.37 * .13 = -.03t-value

Negative limits are set to zero as negative probabilities are not possible

Results: Control Limits for All Time Periods

PeriodNumber of cases

Observed mortality

Expected mortality

Expected deviation

Upper limit

Lower limit

T1 8 0.25 0.28 0.13 0.60 0.00

T2 9 0.33 0.37 0.11 0.62 0.12

T3 7 0.14 0.62 0.16 1.02 0.23

T4 7 0.43 0.38 0.14 0.72 0.04

T5 7 0.29 0.46 0.17 0.88 0.04

T6 7 0.29 0.29 0.13 0.60 0.00

T7 7 0.57 0.52 0.05 0.64 0.39

T8 8 0.25 0.37 0.11 0.63 0.12

Plot Control Limits

UCL

LCL

Step Six: Interpret Findings

There are no points above UCL. There is one point below LCL.In time period 3, mortality is lower than what can be expected from patient’s conditions.All other time periods are within expectations, even time period 7 with its high mortality rate is within expectation.

Step Seven: Distribute Control Chart

Include in the information:How was severity measured and expected

mortality anticipated?Why are assumptions met?What does the control chart look like?What is the interpretation of the findings?

Index of ContentClick on the Slide You Wish to Review

1. Check assumptions2. Calculate and plot observed mortality3. Calculate expected mortality4. Calculate expected deviation5. Calculate and plot control limits6. Interpret findings7. Distribute control chart