Edwin Huff, Ph.D., M.A., Science Officer, CMS, Boston Regional Office

6/12/02 Tri Regional presentation 1

In Search of Inpatient Quality Improvement Intelligence:

Preliminary Analyses from Remeasurement Results and TQIP Event Reporting (the Tracking Quality Improvement Projects (TQIP) database).

Edwin Huff, Ph.D., M.A.,Science Officer, CMS, Boston Regional Office

Division of Clinical Standards & Quality

6/12/02 Tri-Regional presentation 2

Alternate Local Subtitle:

“TQIP & CMS Performance Data -

Onramp to the QI Information (hopefully Knowledge (real soon))

Superhighway, or Road Kill?”


Presentation Goals:

Describe from Remeasurement and TQIP Event data, “What Happened?” during the Sixth Scope of Work, with emphasis on Task 1 Improvement Results, and QIO activities

Begin to explore “What things influenced what happened?”, including both “Nature” and “Nurture” factors, with a focus on the inpatient setting.


What are “Nature” and “Nurture” factors? Nature refers to:

History: • performance baselines

Structural factors:• # of hospitals/state, • average hospital bed size/state

Exogenous factors:• state specific healthcare initiatives

Nurture refers to what QIOs contributed: Provider Education Data dissemination and collection Face-to-face project recruitment Toolkits and Guideline dissemination, etc.


Other Forces “To Be Reckoned With” that Influenced “What Happened?”

C H A N G E HCFA/CMS

Reorganization New Leadership OCSQ Staff Turnover

and Reorganization Use of QIO

Subcontractors as new infrastructure - CASPROs/QIOSCs

New Contract Requirements & Evaluation Standards Performance Based

Contracting Statewide weighted

quality improvement expected

Payment Error Prevention Program

Disparity Reduction in Disadvantaged


So, “Did anything happen?”Remember in the movie “Contact” where

the young scientist poses the question: “Is there intelligent life somewhere out there in

space?” And she answers, as her father had once said to her as a child, “Well, if there weren’t, it sure would be a big waste of space, wouldn’t it?”

Is there anything useful from TQIP? Well, like in Contact, if there weren’t, it sure would be a tremendous waste of resources, wouldn’t it?


Other Emerging Influences ...

Expressed OCSQ interest to try to be more of a “Learning Organization”, trying to use some “Knowledge Management” ideas to accelerate dissemination of “What Works?” knowledge more effectively.

Also, Communities of Practice, and the Institute of Healthcare Improvement’s Breakthrough Series College


Principal Findings Inpatient and Outpatient Healthcare

Quality for Medicare beneficiaries improved, with much variation across clinical topics and geography

History, Structure & Exogenous factors predispose healthcare quality improvement

QIOs significantly influenced system change activity in hospitals


“What Happened?”

Remeasurement Overall

• Inpatient Setting (Rel. Imp. Mdn = 12.7%, lower 25% = 8.3%)

• Outpatient Setting (Rel. Imp. Mdn = 20.6%, lower 25% = 16.7%)

• Specific Clinical Topics QIO Improvement Activity (TQIP Events)

Inpatient Setting• AMI• Pneumonia


Aggregate Baseline Average and Remeasurement Average

y = 0.83x + 16.40R2 = 0.88

60

65

70

75

80

55 60 65 70 75Baseline Average

Rem

easu

rem

ent A

vera

ge


What Happened? Relatively Speaking...

y = 0.92x + 2.08R² = 0.85

0

10

20

30

40

50

0 10 20 30 40 50

Rem

eas.

Rank

Baseline Rank

Rank on Unweighted Average of 21 Task One (Inpatient) Measures at Baseline

and Rank at Remeasurement


What Happened in the Outpatient Setting?

Median Aggregate Topic Measures

Outpatient Improvement: Baseline RemeasurementDiabetes 65.3 74.1Mammography 55.4 60.3Immunization 61.0 68.8 Overall (Outpatient) 60.6 67.7


Outpatient Baseline and Remeasurement

y = 0.90x + 13.95R2 = 0.89

60

65

70

75

80

50 60 70

Baseline

Rem

easu

rem

ent


Inpatient Improvement Baseline RemeasurementAMI 69.5 72.7CHF 78.8 80.8Stroke 77.4 80.1Pneumonia 82.1 84.6 Overall (Inpatient) 76.9 79.5

What Happened in the Inpatient Setting?

Average Aggregate Topic Measures


Aggregate Unweighted Inpatient Baseline Average and Inpatient

Remeasurement

y = 0.73x + 22.15R2 = 0.79

55

60

65

70

75

80

55 60 65 70 75

Baseline

Rem

easu

rem

ent


Aggregate Unweighted Inpatient Baseline Average and Inpatient

Relative Improvement

y = -0.44x + 40.75R2 = 0.13

-5

0

5

10

15

20

25

55 60 65 70 75

Baseline

Rela

tive

Impr

ovem

ent


Aggregate AMI Baseline Average and Relative Improvement

y = -0.62x + 53.42R2 = 0.10

-30

-20

-10

0

10

20

30

40

45 50 55 60 65 70 75 80

AMI Baseline for 52 states & territories

Rela

tive

Impr

ovem

ent R

ate


Aggregate AMI Baseline & Absolute Improvement

y = -0.31x + 24.96R2 = 0.25

-10

-5

0

5

10

15

45 50 55 60 65 70 75 80

AMI Baseline for 52 states and territories

Abso

lute

Impr

ovem

ent


50

60

70

80

aavg1 aavg2

What Happened in AMI?

(Average of 6 QIs)


30

40

50

60

70

p1avg p2avg

What Happened in Community Acquired Pneumonia?


y = -0.48x + 38.90R² = 0.10

-505

101520253035

45 50 55 60 65

Rel

ativ

e Im

prov

emen

t

Pneumonia Baselines for 52 States and Territories

Pneumonia Guideline PerformanceBaselines Associated with Statewide Relative

Improvement


Pneumonia Baselines Associated with Statewide Absolute Improvement

y = -0.38x + 26.72R2 = 0.27

-5

0

5

10

15

45 50 55 60 65

Pneumonia Baselines for 52 States and Territories

Abso

lute

Impr

ovem

ent


Baseline Pneumonia Early Antibiotic Treatment and Improvement

y = -0.3617x + 32.525R2 = 0.3585

y = -1.9165x + 172.71R2 = 0.2433

-40

-30

-20

-10

0

10

20

30

40

50

75 80 85 90 95

Baseline % within 8 hours

Impr

ovem

ent Absolute

Improvement

RelativeImprovementLinear (AbsoluteImprovement)

Linear (RelativeImprovement)


“What Influenced What Happened?”Nature

Average # hospitals/state Average hospital bed size/state State Initiatives (go back to CAP) CASPRO influence

Nurture TQIP Events (Overall & Topic Specific) System Change influence What influenced System Changes?


AMI Baseline Quartiles across 52 States and Territories Stratified by Median Split on #

Hospitals in State (p< .056).

0

1020

30

4050

60

7080

90

1 2 3 4

AMI Baseline Quartiles

Perc

ent

<80

>80

Low High


Timely Antibiotic Treatment for CAP and Average Hospital Bed Size in US

p2an

ti8

(p 50) beds

p2anti8 Fitted values

0 100 200 300 400

80

85

90

95


12%

11%

8%

20%9%

20%

20%

QIO TQIP Inpatient Reported Events (n=105,888)

201- Face-to-face

202-Commit

203-Data Collect

205-Data Report

206-Sys. Change

207- Provider Educ.

208- Toolkits


Distribution across Inpatient Topics of Most Frequently Reported Event "208- Toolkits" (n=21,705)

FIB20%

NHF20%

PNE20%

STR20%

AMI20%


Distribution across Inpatient Topics of Event "205 - Data Reporting." (n=20,942)

FIB20%

NHF20%

PNE21%

STR19%

AMI20%


Distribution across Inpatient Topics of Next Most Frequently Reported Event "207- Provider Educ." (n = 20,749)

FIB20%

NHF20%

PNE21%

STR20%

AMI19%


Distribution across Inpatient Topics of Event "201 - Face-to-Face Recruitment." (n=12,608)

FIB20%

NHF20%

PNE20%

STR20%

AMI20%


Distribution across Inpatient Topics of Event "202 - Commitment to a Project." (n=12,181)

FIB17%

NHF22%

PNE24%

STR17%

AMI20%


Distribution across Inpatient Topics of Event "203 - Data Collection" (n=8,619)

FIB16%

NHF22%

PNE25%

STR15% AMI

22%


Distribution across Inpatient Topics of Event "206 - System Change" (n = 9,084)

FIB13%

NHF21%

PNE27%

STR16% AMI

23%


System Change Category Frequencies across Inpatient Topics

0200400600800

1000120014001600

System Change Category

Freq

uenc

y AMI

FIB

NHF

PNE

STR


CASPRO QIO Inpatient System Change Profiles

0

50

100

150

200

250

300Frequency

CO

CT

IA

OK


Percent of Hospitals reporting any Systems Changes using Computer Aids Systems Changes in AMI

Projects (P<.01)

7.3 8.1

13.711.3

0

5

10

15

Sm rural (728) Sm urban (484) Medium (606) Large (240)

Hospital Peer Group

Perc

ent


Percent of Hospitals reporting making system changes implementing Standing Orders Systems

Changes in AMI Projects (P<.01)

35.2

25.6 27.2 25

0

10

20

30

40


Hospital Peer Group

Perc

ent


Percent of Hospitals reporting Systems Changes implementing Critical Pathway Systems Changes in

AMI Projects (P<.01)

43.852.9 52.8 55

0

20

40

60


Hospital Peer Group

Perc

ent


Percent of Teaching Affiliated Hospitals reporting Systems Changes implementing Standing Order

Systems Changes in AMI Projects (P<.04)

30.59 32.9624.38 23.08

0

10

20

30

40

1 2 3 4

Quartiles of Intern/Beds Teaching Hospital Peer Groups

Perc

ent


Data Collection Activity (Total n = 4,282)

201 - PRO devel.46%

204 - other source20%

203 - Provider Devel.27%

202 - Medquest7%


Additional Structural Findings:

The larger the hospital, the more likely they are to conduct more projects or all projects, and to implement more changes.

The more total system changes made, the more likely that standing orders will eventually be implemented.

The fewer total systems changes made, the lower the level of planned treatment action will be.


relimp Coef. Std. Err. t P>|t| [95% Conf. Interval]

a206 6.06076 5.261026 3.05 0.004 5.470862 26.65066hospeer 8.96421 9.256963 0.97 0.338 -9.669089 27.59751aavg1 -.6269483.2264048 -2.77 0.008 -1.082678 -.1712191round 2.762943 1.640861 1.68 0.099 -.5399385 6.065824beds -.12036 .0626043 -1.92 0.061 -.2463759 .0056559_cons 37.85167 20.27627 1.87 0.068 -2.962338 78.66569

Regression on AMI Relative Improvement


regress a206 a208 a203 a202 a201

Source SS df MS Number of obs = 52 F( 4, 47) = 7.07

Model 1.25019503 4 .312548757 Prob > F = 0.0002Residual 2.07707067 47 .044192993 R-squared = 0.3757

Adj R-squared = 0.3226Total 3.3272657 51 .065240504 Root MSE = .21022

a206 Coef. Std. Err. t P>|t| [95% Conf. Interval]

a208 -.2891213 .2033471 -1.42 0.162 -.6982029 .1199604a203 .3074823 .1343849 2.2 0.027 .0371347 .5778299a202 .3775345 .1340076 2.82 0.007 .1079459 .647123a201 -.1533781 .085411 -1.80 0.079 -.3252029 .0184467_cons .4674229 .203479 2.30 0.026 .0580759 .8767698

What “Nurtures” System Change?


Multivariate Regression on Relative Improvement of AMI Discharge Medication

reldmedimp | Coef. Std. Err. t P>|t| [95% Conf. Interval]

avgdmed1 | -.0183014 .0039391 -4.65 0.000 -.0262453 -.0103574sca209 | .5577032 .2425894 2.30 0.026 .0684751 1.046931sca211 | .4847441 .2042759 2.37 0.022 .0727825 .8967058beds | -.0044496 .0008979 -4.96 0.000 -.0062604 -.0026388cvpeer | .2242144 .1288632 1.74 0.089 -.035663 .4840917teach2 | .9618056 .3567328 2.70 0.010 .2423854 1.681226amisysmax | -.0608676 .0541503 -1.12 0.267 -.170072 .0483368ami | .0057085 .0016384 3.48 0.001 .0024043 .0090127_cons | 1.169179 .3516684 3.32 0.002 .4599719 1.878386


Partial correlation of hospital with

Variable | Corr. Sig.

a201 | -0.0390 0.797 a202 | -0.3116 0.035 a203 | -0.0395 0.794 a205 | -0.0854 0.573 a206 | 0.0033 0.983 a207 | 0.1630 0.279 a208 | -0.0733 0.628


Partial correlation of aavg1 with

Variable | Corr. Sig.

a201 | -0.2646 0.076 a202 | -0.0505 0.739 a203 | 0.1198 0.428 a205 | 0.1306 0.387 a206 | 0.0822 0.587 a207 | -0.1580 0.294 a208 | -0.0257 0.865


Conclusions

Nature matters, and should also be “nurtured.”

Given expected public release of performance measures in the near future, high performers should be encouraged to stay engaged, to sustain historical achievements, and low performers should be encouraged to learn from earlier adapters, QIO experts, and to improve.


Conclusions, continued...

CMS and the QIO Community need to learn more effectively from QIO performance experience to: better identify performance as well as theoretical experts; better understand how breakthrough performance is

achieved, and sustained; better grapple with what QI really costs; surmise why low performance occurs; and anticipate whether there are limits to performance

improvement that require new types of measures, and new contract evaluation standards: such as rewarding sustained high performance.


Conclusions about What to “Nurture”

Foster “Higher” Anticipatory Clinical Treatment System Changes, like Standing Orders;

Foster Commitment to Participate in Projects through: effective technical assistance with data, best practices, sharing examples of effective system change within similar providers.

Documents

Edwin Huff, Ph.D., M.A., Science Officer, CMS, Boston Regional Office