Comparative Rankings of Hospital Quality – Comparative Rankings of Hospital Quality – Does the Data Source Matter?Does the Data Source Matter?

Anne Elixhauser, Ph.D.Anne Elixhauser, Ph.D.Bernard Friedman, Ph.D. Bernard Friedman, Ph.D.

June 26, 2005June 26, 2005AcademyHealth Research MeetingAcademyHealth Research Meeting

BackgroundBackground

Hospitals are being compared based on readily Hospitals are being compared based on readily available dataavailable data

Data on Medicare patients are available from Data on Medicare patients are available from virtually all U.S. hospitalsvirtually all U.S. hospitals

Convenient – but do we know how the Convenient – but do we know how the Medicare experience reflects hospital quality Medicare experience reflects hospital quality overall?overall?

Purpose of StudyPurpose of Study

Examine the extent to which information Examine the extent to which information on Medicare patients can be on Medicare patients can be extrapolated to the general population extrapolated to the general population when comparing hospital qualitywhen comparing hospital quality

Methods – Data SourceMethods – Data Source

2001 Healthcare Cost and Utilization Project (HCUP) Nationwide 2001 Healthcare Cost and Utilization Project (HCUP) Nationwide Inpatient Sample (NIS)Inpatient Sample (NIS)

Sample of 986 hospitals from 33 statesSample of 986 hospitals from 33 states– All discharges from each hospital are includedAll discharges from each hospital are included

No weighting for this study – Used NIS as a convenience sample of No weighting for this study – Used NIS as a convenience sample of hospitals hospitals

Study population: hospital inpatients from short-term, non-Federal, Study population: hospital inpatients from short-term, non-Federal, acute care hospitalsacute care hospitals– Limited to hospitals with Medicare patientsLimited to hospitals with Medicare patients

Methods – Quality MeasuresMethods – Quality Measures

AHRQ Quality IndicatorsAHRQ Quality Indicators– Based on hospital administrative data Based on hospital administrative data – 15 Patient Safety Indicators (PSIs) 15 Patient Safety Indicators (PSIs)

Risk adjusted using gender, age, comorbidities, and Risk adjusted using gender, age, comorbidities, and collapsed DRGscollapsed DRGs

– 12 in-hospital mortality measures from the Inpatient 12 in-hospital mortality measures from the Inpatient Quality Indicators (IQIs)Quality Indicators (IQIs) Risk adjusted using APR-DRGsRisk adjusted using APR-DRGs

Methods – RankingsMethods – Rankings

For each measure:For each measure:– Dropped the 10% of hospitals with the fewest Dropped the 10% of hospitals with the fewest

Medicare casesMedicare cases– Using Medicare discharges onlyUsing Medicare discharges only

Ranked hospitals and ordered into decilesRanked hospitals and ordered into deciles

– Using all patients (including Medicare)Using all patients (including Medicare) Ranked hospitals and ordered into decilesRanked hospitals and ordered into deciles

Comparison of RankingsComparison of Rankings

How many hospitals changed from the highest How many hospitals changed from the highest or lowest rank by at least two deciles?or lowest rank by at least two deciles?

Compared rank using Medicare-only data to rank using Compared rank using Medicare-only data to rank using all-patient dataall-patient data

What % of poorest performing hospitals increased their What % of poorest performing hospitals increased their ranking?ranking?

What % of best-performing hospitals fell in their What % of best-performing hospitals fell in their ranking? ranking?

Percent of hospitals in Percent of hospitals in lowestlowest decile that decile that increasedincreased rank by at least 2 deciles rank by at least 2 deciles

05

101520253035404550

Cx anesth

Death-low m

ort DRG

Decubitus ulcer

Failure to

rescue

Foreign body

Iatro pneumothorax

Infx due to m

ed care

Postop hip fx

Postop hemorrhage

Postop derange

Postop resp fa

il

Postop PE DVT

Postop sepsis

Postop dehiscence

Accidental laceratio

n

Patient Safety Indicators

Percent of hospitals in Percent of hospitals in highesthighest decile decile that that fellfell in rank by at least 2 deciles in rank by at least 2 deciles

05

101520253035404550

Cx anesth

Death-low m

ort DRG

Decubitus ulcer

Failure to

rescue

Foreign body

Iatro pneumothorax

Infx due to m

ed care

Postop hip fx

Postop hemorrhage

Postop derange

Postop resp fa

il

Postop PE DVT

Postop sepsis

Postop dehiscence

Accidental laceratio

n

Patient Safety Indicators

Percent of hospitals in Percent of hospitals in lowestlowest decile that decile that increasedincreased rank by at least 2 deciles rank by at least 2 deciles

05

101520253035404550

AAA repair

CABG

Craniotomy

Hip replacement

AMICHF

Stroke

GI hemorrh

age

Hip fracture

PneumoniaPTCA

CEA

In-hospital Mortality Indicators

Percent of hospitals in Percent of hospitals in highest highest decile decile that that fellfell in rank by at least 2 deciles in rank by at least 2 deciles

05

101520253035404550

AAA repair

CABG

Craniotomy

Hip replacement

AMICHF

Stroke

GI hemorrh

age

Hip fracture

PneumoniaPTCA

CEA

In-hospital Mortality Indicators

Summary of Findings:Summary of Findings:Patient Safety IndicatorsPatient Safety Indicators

Among the top 10% of hospitals (best, or lowest PSI rates):Among the top 10% of hospitals (best, or lowest PSI rates):At least 40% of hospitals fell to At least 40% of hospitals fell to 33rdrd decile or lower for: decile or lower for:

Postop hip fracturePostop hip fracture

Postop physiologic and metabolic derangementPostop physiologic and metabolic derangement

Postop respiratory failurePostop respiratory failure

Wound dehiscenceWound dehiscence

1/3 of hospitals fell to 31/3 of hospitals fell to 3rdrd decile or lower for:decile or lower for:

Anesthesia complicationsAnesthesia complications

Death in low mortality DRGsDeath in low mortality DRGs

Foreign body after procedureForeign body after procedure

1/4 of hospitals fell to 31/4 of hospitals fell to 3rdrd decile or lower for:decile or lower for:

Iatrogenic pneumothoraxIatrogenic pneumothorax

Infection due to medical careInfection due to medical care

Postop hemorrhagePostop hemorrhage

Postop PE and DVTPostop PE and DVT

Postop sepsisPostop sepsis

Summary of Findings:Summary of Findings:In-hospital Mortality IndicatorsIn-hospital Mortality Indicators

Among top 10% of hospitals (best, or lowest mortality rates):Among top 10% of hospitals (best, or lowest mortality rates):38% of hospitals fell to 338% of hospitals fell to 3rdrd decile or lower for:decile or lower for:

CraniotomyCraniotomy

12-20% of hospitals fell to 312-20% of hospitals fell to 3rdrd decile or lower for:decile or lower for:

GI hemorrhageGI hemorrhage

Carotid endarterectomyCarotid endarterectomy

Abdominal aortic aneurysm repairAbdominal aortic aneurysm repair

Among bottom 10% of hospitals (worst, or highest mortality rates):Among bottom 10% of hospitals (worst, or highest mortality rates):24% of hospitals rose to 824% of hospitals rose to 8thth decile or higherdecile or higher

CABGCABG

21% of hospitals rose to 821% of hospitals rose to 8thth decile or higherdecile or higher

CraniotomyCraniotomy

14% of hospitals rose to 814% of hospitals rose to 8thth decile or higherdecile or higher

PTCAPTCA

10% of hospitals rose to 810% of hospitals rose to 8thth decile or higherdecile or higher

Abdominal aortic aneurysm repairAbdominal aortic aneurysm repair

LimitationsLimitations

Findings may be unique to these specific indicatorsFindings may be unique to these specific indicators Findings may not hold for cutpoints other than Findings may not hold for cutpoints other than

decilesdeciles Convenience sample of hospitals – not nationally Convenience sample of hospitals – not nationally

representativerepresentative– But hospitals are drawn from a sampling frame that But hospitals are drawn from a sampling frame that

comprises 80% of U.S. dischargescomprises 80% of U.S. discharges

ConclusionsConclusions

When comparing Medicare and all-payer analysesWhen comparing Medicare and all-payer analyses– Found loose overlap of the top-ranking and poorest-ranking Found loose overlap of the top-ranking and poorest-ranking

hospitals hospitals – Saw largest shifts for PSIs among the top-ranking hospitalsSaw largest shifts for PSIs among the top-ranking hospitals

Using Medicare-only data may carry a greater risk of Using Medicare-only data may carry a greater risk of incorrectly labeling a hospital as a top-ranking incorrectly labeling a hospital as a top-ranking performerperformer

Pre-test ranking approaches Pre-test ranking approaches