Vincent Mor, Ph.D. Brown University

Do Good Nursing Homes Achieve Good Outcomes? Measurement Considerations Associated with Public Reporting

Vincent Mor, Ph.D. Brown University

Do Good Nursing Homes Achieve Good Outcomes?

2

Purpose

Review Long Term Care Assessments Using Nursing Home as Example, Review Long Term

Care Outcome Domains and Measures Present Empirical Data Addressing Select Nursing

Home Quality Measurement Issues How Correlated are Quality Measures? How Stable are Quality Measures? How Variable are Quality Measures?

Implications for Public Reporting


3

Background Long Term Care, while relatively deficient in

resources, had head start with uniform clinical assessment tools in Rehab Hospitals, nursing homes and home health agencies

FIM for Rehab Hospitals, MDS for nursing homes and OASIS for home health emerged from acknowledged need to improve care quality.

MDS and OASIS implemented and applied to policies ranging from case mix reimbursement to quality review


The Nursing Home Resident Assessment Instrument (RAI)

1986 Institute of Medicine Report on Nursing Home Quality Recommended a Uniform Resident Assessment Instrument to Guide Care Planning

OBRA ‘87 Contained Nursing Home Reform Act Including RAI Requirement

A 300 Item, Multi-Dimensional RAI – MDS was Tested for 2 Years

Mandated Implementation in 1991 Fully Computerized in 1998 Case Mix Reimbursement in 1999 Quality Reporting in 2002


5

The Outcome and Assessment Information Set (OASIS)

Initially designed as an outcome measurement tool for home health agency staff to track how well patients progressing

Transformed into a multi-purpose tool for: patient assessment and care planning for individual adult

patients agency-level case mix reports internal HHA performance improvement Case-Mix Reimbursement mandated by BBA National Home Health Quality Compare 2003


6

Functional Impact Measure (FIM) Initiated as a way to document functional

progress during rehabilitation Widely used in Rehab Hospitals around the

country Measures Mobility, ADL dependence and

Cognition Summary scores created and differences

between admission & discharge calculated Some provider performance profiling

underway


7

Nursing Home Measurement Issues

What does quality mean and to whom? Hard to know how much inter-state variation

is due to measurement vs. real processes Differentiating between providers; how much

of a difference is important? Inter-relationship among the domains and

measures; implications of composite scores Stability of measures over time


8

Domains of Nursing Home Quality Measures Regulatory Compliance

Deficiencies, Complaints, etc. Staffing

Hours/Resident day, Skill Mix, contractors, etc. Clinical Processes

Restraints, Psychotropics, etc. Clinical Outcomes

ADL decline, Unmet needs, hospitalization Quality of Life

Mood, Food, Activity, Social Interaction Satisfaction

Service meets Expectations? Getting what you want? etc.


9

Regulatory Based Quality Measures State surveyors conduct annual inspections

following national protocol to assess compliance with Conditions of Participation

Substantial inter & intra-state variation in number, type, severity and distribution of regulatory infractions

Factors associated with deficiencies also differ somewhat by state

Termination of Medicare/Medicaid due to regulatory violations rare


10

TX

CA

MT

AZ

ID

NV

NM

CO

IL

OR

UT

KS

WY

IA

SD

NE

MN

ND

FL

OK

WI

WA

MO

AL GA

AR

LA

MI

IN

PA

NY

NC

MS

TN

VA

KY

OH

SC

ME

WV

MI VT NH

NJ

MA

CT

MD DE

RI

10.8 - 19.4

8.9 - 10.2

7.0 - 8.6

4.7 - 6.5

State Average Total Number of Deficiencies Per Nursing Home, 2004

Source: OSCAR 2004.


11

TX

CA

MT

AZ

ID

NV

NM

CO

IL

OR

UT

KS

WY

IA

SD

NE

MN

ND

FL

OK

WI

WA

MO

AL GA

AR

LA

MI

IN

PA

NY

NC

MS

TN

VA

KY

OH

SC

ME

WV

MI VT NH

NJ

MA

CT

MD DE

RI

13.3% - 35%

10.1% - 13.2%

7.7% - 10.1%

4.7% - 7.5%

Percentage of NHs Cited for Causing Actual Harm orImmediate Jeopardy to Redisents in Each State, 2004

Source: OSCAR 2004.


12

02

04

06

0

%

1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 Source: OSCAR.

Trends and Inter-State Variations in the Percentage of FacilitiesCited for Actual Harm or Immediate Jeopardy to Residents, 1995-2004


13

Staffing as Structural Quality Measures

Historically, most measures of nursing home quality revolved around staffing levels

The mix of staffing has also been considered, particularly the ratio of RNs to practical nurses and to nursing assistants

The availability of Medical Directors and Nurse Practitioners is a more recent focus

Data Quality poor since reported by home; we apply longitudinal cleaning algorithms


14

Trends in Total Staffing Levels

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

1996 1997 1998 1999 2000 2001 2002 2003 2004

Hartford Panel(4.44+ HPRD)

CMS Optimum(3.90+ HPRD)

CMS Preferred(3.00+ HPRD)

CMS Minimum(2.75+ HPRD)

Below Minimum(< 2.75 HPRD)


15

TX

CA

MT

AZ

ID

NV

NM

CO

IL

OR

UT

KS

WY

IA

SD

NE

MN

ND

FL

OK

WI

WA

MO

AL GA

AR

LA

MI

IN

PA

NY

NC

MS

TN

VA

KY

OH

SC

ME

WV

MI VT NH

NJ

MA

CT

MD DE

RI

78.7 - 96.5

66.8 - 78.6

56.8 - 66.7

32.3 - 56.7

22.4 - 32.2

Percent Nursing Homes Exceeding CMS PreferredTotal Direct Care Staffing Level (3.0+ HPRD)

Source: OSCAR 2004.


16

TX

CA

MT

AZ

ID

NV

NM

CO

IL

OR

UT

KS

WY

IA

SD

NE

MN

ND

FL

OK

WI

WA

MO

AL GA

AR

LA

MI

IN

PA

NY

NC

MS

TN

VA

KY

OH

SC

ME

WV

MI VT NH

NJ

MA

CT

MD DE

RI

11.7 - 30.0

8.5 - 11.6

5.1 - 8.4

3.6 - 5.0

0.0 - 3.5

Percent Nursing Homes Exceeding Hartford RecommendedTotal Direct Care Staffing Level (4.44+ HPRD)

Source: OSCAR 2004.


17

Proportion of facilities using 5% or more

contract RNs or LPNs, 1992-2001

0

2

4

6

8

10

12

14

1992 1993 1994 1995 1996 1997 1998 1999 2000 2001

%

RN LPN


18Note: Difference between unadjusted & adjusted means is statistically significant (at p<.001) by paired T-test for all years 2000-2004.

Mean Staffing HPPD, 1999-2004

2.88

2.91

2.95

3.043.02

3.10

2.88 2.88

2.90

2.96

2.93

2.97

2.75

2.80

2.85

2.90

2.95

3.00

3.05

3.10

3.15

1999 2000 2001 2002 2003 2004

HP

PD

Unadjusted RUG-NCMI Adjusted


19

MDS Resident Assessment Data Quality Issues

Although inter-rater reliability studies show good results, some studies suggest data MDS not consistently collected across providers.

Inconsistency of data collection undermines comparability of quality measures

Analyses of largest reliability trial done reveals large inter-state differences in the “direction” of disagreements; this is related to inter-state variation in quality indicators


20

Process Measures of Quality

Calculating the rate at which treatments that should be delivered to certain patients are and the rate at which certain things are done that shouldn’t be done Restraints Anti-psychotics Advanced Directives in Cognitively Impaired Feeding Tubes in the Cognitively Impaired


21

TX

CA

MT

AZ

ID

NV

NM

CO

IL

OR

UT

KS

WY

IA

SD

NE

MN

ND

FL

OK

WI

WA

MO

AL GA

AR

LA

MI

IN

PA

NY

NC

MS

TN

VA

KY

OH

SC

ME

WV

MI VT NH

NJ

MA

CT

MD DE

RI

0.070 - 0.141

0.050 - 0.064

0.036 - 0.047

0.015 - 0.035

State Averaged Quality Measure, 2004:Prevalence of Restraint Use


22

Changing Rate of restraint use in facilities 1999-2004

0.1

.2.3

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21

Restraints


23

Outcome Measures of Quality Clinical measures related to “sentinel”

health events; rare but should be near “0” Functional decline rates calculated as

changes between assessments Incidence of clinical events that are not

desired but hard to prevent - hospitalization “Unmet” need for care Measures of Satisfaction from family or

resident


24

TX

CA

MT

AZ

ID

NV

NM

CO

IL

OR

UT

KS

WY

IA

SD

NE

MN

ND

FL

OK

WI

WA

MO

AL GA

AR

LA

MI

IN

PA

NY

NC

MS

TN

VA

KY

OH

SC

ME

WV

MI VT NH

NJ

MA

CT

MD DE

RI

0.177 - 0.244

0.154 - 0.177

0.140 - 0.153

0.126 - 0.140

State Averaged Quality Measure, 2004:Incidence of Late-loss ADL Worsening

Double the rate between lowest & highest state.


25

Hospitalization Rate within 6 Months in the Long-Stay NH Population per State in 2000

0

5

10

15

20

25

30

LA

MS

NJ

OK

TX

KY

AR

WV

GA IL FL

MO AL

TN

MD

OH

PA IN MI

NY IA SD

CA

VA

MA

NC

SC RI

NV

DE

KS

NE

MN

WY

CT

WI

AZ

CO

WA

ND

MT

VT ID OR

NH

ME

NM

UT

% H

ospitaliz

ed

Source: MDS 2000; Medicare inpatient claims 2000.


26

How have clinical quality measures changed over the last half decade?

Pressure Ulcer Prevalence remained stable Anti-psychotic medication use increased Restraint use declined

HOWEVER Lots of variation; Facilities in the Top Quintile in 1999 converged to

the middle; and vice versa Restraint Improvement resulted from reductions

in restraint use among highest users


27

Changing Rate of ADL decline in nursing homes 1999-20040

.1.2

.3

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21

ADL Decline


28

Change in the Rate of Restraint Use: Stratified by Quintile Ranking at Baseline (1999Q3)

0.000

0.050

0.100

0.150

0.200

0.250

0.300

19

99

Q3

19

99

Q4

20

00

Q1

20

00

Q2

20

00

Q3

20

00

Q4

20

01

Q1

20

01

Q2

20

01

Q3

20

01

Q4

20

02

Q1

20

02

Q2

20

02

Q3

20

02

Q4

20

03

Q1

20

03

Q2

20

03

Q3

20

03

Q4

20

04

Q1

20

04

Q2

20

04

Q3

20

04

Q4

Me

an

Ra

te

QUINTILE 5

QUINTILE 4

QUINTILE 3

QUINTILE 2

QUINTILE 1


29

Do Nursing Homes that start out with good clinical outcomes remain good?

Split all US nursing homes into quintiles in 1999 based upon their clinical quality

Tracked facilities’ clinical quality measures each quarter over next 5 years

Observed substantial “regression to the mean”

Most homes beginning in the top quintile ended up very similar to those beginning at the bottom


30

Change in the Rate of ADL Decline: Stratified by Quintile Ranking at Baseline (1999Q3)

0.000

0.050

0.100

0.150

0.200

0.250

1999

Q3

1999

Q4

2000

Q1

2000

Q2

2000

Q3

2000

Q4

2001

Q1

2001

Q2

2001

Q3

2001

Q4

2002

Q1

2002

Q2

2002

Q3

2002

Q4

2003

Q1

2003

Q2

2003

Q3

2003

Q4

2004

Q1

2004

Q2

2004

Q3

2004

Q4

Mea

n R

ate

QUINTILE 5

QUINTILE 4

QUINTILE 3

QUINTILE 2

QUINTILE 1


31

Measuring Differences between Providers Variation is reduced after aggregating

measures to the level of the provider Number of residents per facility means little

power in standard statistical tests Many measures are skewed Comparing averages versus rates proportion

meeting a threshold; basis for setting thresholds?

Are we seeing differences when there are none?


32

Facility ADL Decline Rates: 1999-20050

24

68

Density

0 .2 .4 .6 .8qi: adl(28) decline 4 points--adj rate


33

Facility Restraint Rates: 1999-20050

510

15

Density

0 .2 .4 .6 .8 1qi: restraints --unadj rate


34

60% of customers satisfiedIn homes in bottom 20%


35

Facility level distribution of average resident satisfaction scores: RI 2006

.5 range out of 5 points


36

Facility level distribution of average family satisfaction scores: RI 2006

.5 range out of 5 points


37

How do Quality Measures Inter-relate?

Nearly as many providers with low rates of physical restraints have high levels of pressure ulcers as have low levels;

The average correlation among MDS based measures of quality across all US nursing homes is low;

Structural and Regulatory based measures of quality are only minimally related to outcome based measures of quality


38

Average within-MSA rank-order correlation coefficients between Clinical & Reputation Quality in 2004 (weighted by # NHs in MSA):

OSCAR “Reputational” Measures MDS Clinical Measures

N(MSAs)[A] [B] [C] [D] [E] [F] G] [H]

[A] OCCUPANCY 1.000 296

[B] % PRIVATE PAY 0.126 1.000 296

[C] DEFICIENCIES -0.133 -0.138 1.000 296

[D] TOTAL NURSE HPPD -0.079 0.165 -0.125 1.000 296

[E] ADL DECLINE -0.053 0.082 0.063 0.031 1.000 296

[F] RESTRAINTS -0.021 -0.055 0.094 -0.052 -0.012 1.000 296

[G] PU WORSENING -0.120 0.045 0.097 0.025 0.213 0.045 1.000 296

[H] PERSISTENT PAIN -0.081 -0.082 0.104 -0.038 0.017 0.060 0.065 1.000 296


39

How well do the “best” homes perform?

“Best” in terms of the fewest regulatory deficiencies, most staff, etc.

Classified facilities into thirds on Deficiencies, Staffing, etc.

Then, compared them on clinical quality measures over multiple years


40

Change in Rate of ADL Decline, Stratified by Baseline (1999) Ranking of Total Deficiencies (LOW/MIXED/HIGH)

0.0000.0200.0400.0600.0800.1000.1200.1400.160

1999

Q3

1999

Q4

2000

Q1

2000

Q2

2000

Q3

2000

Q4

2001

Q1

2001

Q2

2001

Q3

2001

Q4

2002

Q1

2002

Q2

2002

Q3

2002

Q4

2003

Q1

2003

Q2

2003

Q3

2003

Q4

2004

Q1

2004

Q2

2004

Q3

2004

Q4

Mea

n R

ate

LOW MIXED HIGH


41

Change in Rate of ADL Decline, Stratified by Baseline (1999) Ranking of Total Nurse HPPD (LOW/MIXED/HIGH)

0.0000.0200.0400.0600.0800.1000.1200.1400.160

1999

Q3

1999

Q4

2000

Q1

2000

Q2

2000

Q3

2000

Q4

2001

Q1

2001

Q2

2001

Q3

2001

Q4

2002

Q1

2002

Q2

2002

Q3

2002

Q4

2003

Q1

2003

Q2

2003

Q3

2003

Q4

2004

Q1

2004

Q2

2004

Q3

2004

Q4

Mea

n R

ate

LOW MIXED HIGH


42

Does Clinical Quality Predict Nursing Home Termination?

NHs with fewest deficiencies less likely to be terminated

NHs with lowest ADL decline and restraint rate less likely to be terminated

BUT, pressure ulcer worsening or persistent pain performance not related to future termination & staffing barely predictive


43

Quality Predicting Nursing Home Closure w/in 2 years

0

0.1

0.2

0.3

0.4

0.5

0.6

Percent Closed

WorstQuality

MixedQuality

BestQuality

StaffingDeficienciesADL DeclineRestraintsVery Rare Event


44

Implications for Public Reporting and Pay for Performance

Cross-sectional Comparisons are suspect Comparisons across state lines suspect Volatility in measures within facility Measures aren’t correlated; P4P needs a

priori value based weighting Multi-dimensional quality measures mean

consumers will need support in using the data to inform decisions


45

Implications for Quality Measurement

Improving Data Quality is essential; applies to regulatory, staffing, MDS based data and patient/family surveys

Addressing Methodological issues essential before applying these data to P4P

Findings are likely applicable to Home Health Agencies and Rehab Hospitals

Documents

Vincent Mor, Ph.D. Brown University