Hospital Heterogeneity: What Drives the Quality of Health Care for Stroke

Hospital Heterogeneity: What Drives Quality of Care?

University of Manchester

May 29, 2015

Introduction

1 The need to improve the quality of healthcare in UK and in other developedcountries is well documented.

2 The conundrum of rising healthcare costs and disappointing quality exists in everydeveloped country including the NHS (Palmer, 2012)

3 Healthcare organizations are increasingly scrutinized by external agencies such asHealth Care Commission in England.

4 Such agencies are increasingly concern themselves with the quality of care.

5 Health systems around the world are placing increasing demands on health careorganisations to deliver improvements in the performance and quality of the ser-vices.

6 A central observation of the healthcare systems is the existence of substantialheterogeneity or variations in healthcare quality across hospitals which are majorproviders of healthcare services, treat the most critically ill patients and accruesa substantial amount of health expenditure.

(University of Manchester) May 29, 2015 2 / 39

Hetrogeneity across NHS trustsQuality of Care for Stroke Services

0

5

10

20 40 60 80score

coun

t

2004

0

5

10

40 60 80score

coun

t

2006

0

5

10

15

40 60 80 100score

coun

t

2008

05

1015

50 60 70 80 90 100score

coun

t

2010


IntroductionPossible Explanations

7 Differences in the quality of healthcare suggest opportunities to improve qualityexists throughout the healthcare system.

8 These in turn have generated huge academic and policy interest in understandingthe causes of these variations and what could be done to increase quality at underperforming hospitals.

9 Possible Explanations:

1 Resources

Physical CapitalHuman Capital

2 Optimal use of resources

Organizational factorsManagement practicesIncentive system (for instance clinical Best practice tariffs in 2010, Payment byresults, readmission penalties etc.)

3 Structural Characteristics4 External/Environmental factors

Regional factors—median wage, prevalence, mortality etc.Competition and other market–oriented policies


IntroductionPossible Explanations

7 Differences in the quality of healthcare suggest opportunities to improve qualityexists throughout the healthcare system.

8 These in turn have generated huge academic and policy interest in understandingthe causes of these variations and what could be done to increase quality at underperforming hospitals.

9 Possible Explanations:

1 Resources

Physical CapitalHuman Capital

2 Optimal use of resources

Organizational factorsManagement practicesIncentive system (for instance clinical Best practice tariffs in 2010, Payment byresults, readmission penalties etc.)

3 Structural Characteristics4 External/Environmental factors

Regional factors—median wage, prevalence, mortality etc.Competition and other market–oriented policies


IntroductionManagerial and Organisational factors

1 An emerging trend of literature focuses on managerial and organisational as-pects of healthcare

2 A number of hospitals have turned to management practices for example,“Lean” methodology that was originally developed by Toyota which focusedcontinuous improvement and team work (Bloom et al., 2015; McConnell etal., 2013 and 2014)

3 Problems of variations in and lower healthcare quality can be mitigated if clin-icians and hospitals have help in identifying the best strategies in evaluatingand treating patients, and if they work in teams characterized by good struc-tural organisation, management leadership, patient centred and coordination& communication tools which are understood by all involved and that helppatients avoid hospitalisations and stay as healthy as possible.

4 Challenges are not clinical, but organisational (Lee and Mongan, 2009; Ra-manujam and Rousseau, 2006; West 2001)


IntroductionManagerial and Organisational factors

1 An emerging trend of literature focuses on managerial and organisational as-pects of healthcare

2 A number of hospitals have turned to management practices for example,“Lean” methodology that was originally developed by Toyota which focusedcontinuous improvement and team work (Bloom et al., 2015; McConnell etal., 2013 and 2014)

3 Problems of variations in and lower healthcare quality can be mitigated if clin-icians and hospitals have help in identifying the best strategies in evaluatingand treating patients, and if they work in teams characterized by good struc-tural organisation, management leadership, patient centred and coordination& communication tools which are understood by all involved and that helppatients avoid hospitalisations and stay as healthy as possible.

4 Challenges are not clinical, but organisational (Lee and Mongan, 2009; Ra-manujam and Rousseau, 2006; West 2001)


IntroductionResearch Aims and Question

Research Question: What are the factors that determine the quality ofcare as measured by process of care?

Syverson (2011)—internal and external drivers of firm productivity

Stroke process of care data from National Sentinel Stroke Audit 2004 to 2010from Royal College of Physicians

Process of measure includes:

1 Brain scan within 24 hours of stroke2 Screening for swallowing within 24 hours of admission3 Aspirin within 24 hours4 Physio assessment within 72 hours5 Patients weighed atleast once6 Mood assessed7 Rehabilitation goals agreed within discharge

Process measures summarized by a numerical score ranging from 0 to 100 usingthe indicator average method


IntroductionThree Dimensions of Quality

Under the health services research framework, healthcare quality can be definedor categorized by structure, process and outcomes.

Donabedian (1980) hypothesized that structure influences process which inturn influences outcomes.

Process measures illuminate the complicated process of delivering healthcareand describe the specific actions associated with healthcare delivery.

Structural measures focus on the characteristics of resources of the healthcaresystem, including institutional capacity (e.g. hospital size), system resources(e.g. stroke units, foundation trust status, QI participation)

Outcome measures focus on the end result of care or the effect of the careprocesses on the health and well–being of patients and population.


IntroductionProcess of Care Measures

Using process measures as quality indicators has several advantages (Rubin etal. 2001; Lilford et al. 2007)

Can more easily be used to provide feedback for quality improvement initiatives

Sensitivity and responsiveness to intervention

Generally requires less adjustment for patient severity and case–mix than most

outcome measures (Ukawa et al. 2014)

There is a lack of evidence to support a direct association between the qualityof care in process measures and improved performance in outcomes (Rubin etal. 2001)

Indicators to evaluate improvements of healthcare quality and standardization,as well as for public disclosure for accountability (Ukawa et al. 2014)


IntroductionShortcomings and Research Contributions

Importance of organizational and managerial factors in determining hospitalproductivity or quality

Process measures of quality over outcomes in health care

Interactions between the variables — Machine Learning algorithms

Predictive accuracy and out–of–sample predictions — Statistical Modelling: TwoCultures by Brieman (2001)

Longitudinal Data

Problematic causal inference and inappropriate policy recommendations

Time trend analyses

Geographic variations


IntroductionStroke Care

Focus on the quality of care for stroke care, also known as cerebral infarction

Stroke is one of the major causes of non–accidental deaths worldwide (Feiginet al. 2013).

In western countries, stroke is currently second biggest cause of death rankingafter heart disease and before cancer.

Stroke is a major cause of mortality and morbidity in the UK. In England 2011,there has been 110,000 incidences of stroke (Bray et al. 2013).


IntroductionDeaths in 2010 from circulatory diseases, England & Wales. Source: The Guardian, 2011


Theoretical frameworkRole and Importance of Organisational Quality

1 Theory of Complementarity in Organisations

(i) Brynjolfsson and Milgrom (2013); Milgrom and Roberts, (1995)

(ii) The interaction of two or agents or forces to produce an effect than the sum of

their individual effects.

(iii) The sets of factors or inputs when used together in the production process can be

mutually reinforcing in their effects of quality and performance

(iv) Dranove et al. (2014)—hospital technology adoption and costs

DefinitionTwo organisational practices for example x1 and x2 are complementary to each otherif their second derivative with respect to hospital’s objective function that is∂2f /∂x1∂x2 is ≥ 0. for all values of (x1, x2) with strict inequality for at least onevalue of the organisational practices.


Theoretical frameworkRole and Importance of Organisational Quality

1 Theory of Complementarity in Organisations

(i) Brynjolfsson and Milgrom (2013); Milgrom and Roberts, (1995)

(ii) The interaction of two or agents or forces to produce an effect than the sum of

their individual effects.

(iii) The sets of factors or inputs when used together in the production process can be

mutually reinforcing in their effects of quality and performance

(iv) Dranove et al. (2014)—hospital technology adoption and costs

DefinitionTwo organisational practices for example x1 and x2 are complementary to each otherif their second derivative with respect to hospital’s objective function that is∂2f /∂x1∂x2 is ≥ 0. for all values of (x1, x2) with strict inequality for at least onevalue of the organisational practices.


DataOrganisational quality and other variables

Royal College of Physicians National Sentinel Stroke Audit

Total organisational score for each trust ranging from 0 to 100 calculated asan average from 8 domains:

1 Acute Stroke Care Organisation

2 Organisation of Care

3 Specialist Roles

4 Inter Disciplinary Services (for sites with a stroke unit)

5 TIA/Neurovascular Service

6 Quality Improvement and Research

7 Team Working–Team Meetings

8 Communication with Patients and Carer

Total no of neurologists, teaching status, no. of hospital beds (size), regionaletc.


Exploratory Analysis

Table 1: Descriptive Statistics for Clinical Process Score

Years

2004 2006 2008 2010

Minimum 25.00 31.00 40.00 52.00

First Quartile 51.75 58.25 63.00 73.00

Median 61.00 67.00 71.00 79.00

Mean 60.48 66.13 69.74 78.75

Third Quartile 68.25 75.75 77.00 85.00

Maximum 93.00 93.00 96.00 97.00

Standard Deviation 12.36 13.33 11.32 8.57


Exploratory Analaysis

Table 2: Descriptive Statistics for Organisational Score

Years

2006 2008 2010

Minimum 23.00 32.00 47.00

First Quartile 57.25 63.75 62.00

Median 64.00 71.00 69.00

Mean 63.74 70.59 69.65

Third Quartile 72.00 79.00 76.75

Maximum 89.00 95.00 96.00

Standard Deviation 11.96 11.37 10.20



Figure 1: Score against the year

40

60

80

100

2003 2005 2007 2009 2011Year

scor

e

Year

2004

2006

2008

2010



Figure 2: Organisational Performance and Process of Score Quality

40

60

80

100

lower quartile middle half upper quartileOrganisational Position

scor

e

orgposition

lower quartile

middle half

upper quartile



Figure 3: Regional Variations

40

60

80

100

scor

e

Regions

East

East Midlands

London

North East

North West

South Central

South East Coast

South West

West Midlands

Yorkshire & the Humber


MethodologyMachine Learning for Longitudinal Data

1 (Unbiased) Regression Trees for longitudinal data—Siminoff and Wu (2014)

2 Mixed effects model for longitudinal data with regression tree based estimationmethods—RE–EM tree (Random Effects–Expectation Maximization); Sela andSiminoff (2012)

3 Linear Mixed Effects Model (LME)

4 Model performance/assessment:

(i) In–sample fit/predictions—AIC & BIC

(ii) Out–sample predictions: Leave–one out and k–fold cross validation

5 Endogeneity Issues

6 Importance of making predictions—Friedman (1953) and Chetty (2015)


Models and Hypotheses

Table 3: Models

Models Variables

Model 1: Physical Capital

Beds

Operating theatres

Day Case theatres

Model 2: Human Capital

Total Clinical (medically qualified) staff

Total non–medical staff

Professionally qualified clinical staff

Nurses

Healthcare Scientists

Allied health professionals

General Medicine Group

Neurologists

Neurophysiology

Neurosurgeons

Model 3: Hospital Characteristics and Organisational

quality

Teaching status

Foundation Trust Status

Organisational performance

Model 4: Regional Health Variables

Standardised Stroke Mortality 30–day rate

Emergency stroke admissions

all SMR (area)

Model 5: Socio–Economic Variables

Median wage

Inequality

%regional population with no qualifications


Regression Tree Results

Figure 4: Unbiased REEM–Tree for Model 3

orgscorep < 0.001

1

≤ 67 > 67

Node 2 (n = 155)

40

50

60

70

80

90

100

gmgStaffp < 0.001

3

≤ 0.041 > 0.041

Node 4 (n = 144)

40

50

60

70

80

90

100

teachingp = 0.053

5

≤ 0 > 0

Node 6 (n = 9)

40

50

60

70

80

90

100Node 7 (n = 20)

40

50

60

70

80

90

100




orgscorep < 0.001

1

≤ 67 > 67

Node 2 (n = 155)

40

50

60

70

80

90

100

gmgStaffp < 0.001

3

≤ 0.041 > 0.041

Node 4 (n = 144)

40

50

60

70

80

90

100

all_smrp = 0.045

5

≤ 109.62 > 109.62

Node 6 (n = 21)

40

50

60

70

80

90

100Node 7 (n = 8)

40

50

60

70

80

90

100




orgscorep < 0.001

1

≤ 67 > 67

Node 2 (n = 155)

40

50

60

70

80

90

100

gmgStaffp < 0.001

3

≤ 0.041 > 0.041

Node 4 (n = 144)

40

50

60

70

80

90

100

all_smrp = 0.045

5

≤ 109.62 > 109.62

Node 6 (n = 21)

40

50

60

70

80

90

100Node 7 (n = 8)

40

50

60

70

80

90

100




orgscorep < 0.001

1

≤ 67 > 67

median_wagep < 0.001

2

≤ 486.5 > 486.5

Node 3 (n = 111)

40

50

60

70

80

90

100Node 4 (n = 44)

40

50

60

70

80

90

100

median_wagep < 0.001

5

≤ 524.8 > 524.8

median_wagep = 0.02

6

≤ 420.5 > 420.5

Node 7 (n = 47)

40

50

60

70

80

90

100Node 8 (n = 103)

40

50

60

70

80

90

100

profqStaffp = 0.031

9

≤ 0.506 > 0.506

Node 10 (n = 11)

40

50

60

70

80

90

100Node 11 (n = 12)

40

50

60

70

80

90

100


Model Performance and AssessmentIn Sample Fits for models 1 to 5

Table 4: In Sample fits for models without year and ratios

Unbiased REEM–Tree REEM–Tree LME

(1) (2) (3) (4) (5) (6) (7) (8) (9)

Log Likelihood AIC BIC Log Likelihood AIC BIC Log Likelihood AIC BIC

Models

Model 1 -1234.497 2474.99 2430.04 -1234.497 2474.99 2486.36 -1238.517 2489.03 2511.72

Model 2 -1197.678 2407.36 2457.86 -1192.093 2396.19 2418.87 -1164.889 2359.78 2416.07

Model 3 -1187.768 2387.54 2410.22 -1158.047 2332.10 2362.29 -1133.805 2303.61 2370.98

Model 4 -1187.121 2386.24 2408.93 -1179.302 2370.60 2393.85 -1139.121 2320.24 2398.64

Model 5 -1162.276 2340.55 2370.75 -1156.683 2329.37 2359.56 -1117.162 2282.32 2371.69


Regression Tree ResultsOut–sample–fits for models 1 to 5

Table 5: Out of Sample Predictions without year and ratios

LOOCV k–FOLD

Unbiased REEM–Tree REEM–Tree LME Unbiased REEM–Tree REEM–Tree LME

(1) (2) (3) (4) (5) (6)

Models

Model 1 9.220 9.264 9.222 9.314 9.601 9.319

Model 2 9.166 8.902 8.999 9.542 9.050 9.050

Model 3 8.800 9.057 8.348 8.857 8.991 8.330

Model 4 8.763 9.118 8.178 8.727 9.259 8.150

Model 5 7.982 8.210 7.634 8.133 8.498 7.682


Robustness AnalysisYear Robustness

Figure 8: Yearly Analysis

60

70

80

2004 2006 2008 2010Years

Val

ue

variable

score

overall


Regression Tree ResultsYear Robustness

Figure 9: Unbiased REEM–Tree for Model 5 with year (ratios)

yearp < 0.001

1

≤ 2008 > 2008

orgscorep < 0.001

2

≤ 67 > 67

Node 3 (n = 97)

50

60

70

80

90

gmgStaffp < 0.001

4

≤ 0.037 > 0.037

Node 5 (n = 55)

50

60

70

80

90

Node 6 (n = 12)

50

60

70

80

90

clinicalStaffp < 0.001

7

≤ 0.134 > 0.134

Node 8 (n = 149)

50

60

70

80

90

Node 9 (n = 15)

50

60

70

80

90


In sample fits

Figure 10: In–sample fit plot

40 50 60 70 80 90

6070

8090

Actual Score

Fitt

ed V

alue

s


Diagnostic Plots

Figure 11: Diagnostic Analysis

60 70 80 90

−15

−10

−5

05

10Residuals against Fitted values

Fitted Score

Res

idua

ls

−3 −1 0 1 2 3

−15

−10

−5

05

10

Normal Q−Q Plot

Theoretical Quantiles

Sam

ple

Qua

ntile

s


Robustness AnalysisFurther Robustness

Figure 12: Further Robustness

yearp < 0.001

1

≤ 2008 > 2008

orgscorep < 0.001

2

≤ 67 > 67

Node 3 (n = 97)

50

60

70

80

90

100

gmgStaffp < 0.001

4

≤ 0.037 > 0.037

Node 5 (n = 55)

50

60

70

80

90

100Node 6 (n = 12)

50

60

70

80

90

100

orgscorep = 0.003

7

≤ 77 > 77

Node 8 (n = 112)

50

60

70

80

90

100

gmgStaffp < 0.001

9

≤ 0.037 > 0.037

Node 10 (n = 39)

50

60

70

80

90

100Node 11 (n = 13)

50

60

70

80

90

100


Robustness AnalysisFurther Robustness: Cross Section

Figure 13: Further Robustness: 2010 Cross Section

orgscorep < 0.001

1

≤ 77 > 77

Node 2 (n = 112)

60

70

80

90

100

gmgStaffp = 0.003

3

≤ 0.042 > 0.042

Node 4 (n = 42)

60

70

80

90

100Node 5 (n = 10)

60

70

80

90

100


Model Assessment and PerformanceYear Robustness

Table 6: Year Robustness — In sample and out of sample performance (Ratios)

In–Sample Fits Out of Sample Predictions

Log Lik AIC BIC LOOCV k–FOLD

(1) (2) (3) (4) (5)

Unbiased REEM–Tree -1155.66 2325.32 2351.76 7.540 7.469

REEM–Tree -1141.89 2299.77 2329.97 7.728 7.519

LME -1113.739 2277.48 2370.487 7.527 7.568


Limitations & Extensions

1 Data

2 Randomised Experiments


Summary & Conclusions

The empirical findings are consistent with the productivity literature and cor-roborates with previous research on managerial and organisational determinantsof healthcare quality that have used different designs, data and methods and,they offer predictive support for the theory used in this study as well as in theeconomics literature on the role of institutions and productivity.

Robust

Regression Trees and machine learning methods

Healthcare exceptionalism—Chandra et al. (2013)

Can inform and improve the decision making process for healthcare qualityimprovement and also in general contributes to data driven decision making inhealthcare.

Limitations

Roadmap


Summary & Conclusions

The empirical findings are consistent with the productivity literature and cor-roborates with previous research on managerial and organisational determinantsof healthcare quality that have used different designs, data and methods and,they offer predictive support for the theory used in this study as well as in theeconomics literature on the role of institutions and productivity.

Robust

Regression Trees and machine learning methods

Healthcare exceptionalism—Chandra et al. (2013)

Can inform and improve the decision making process for healthcare qualityimprovement and also in general contributes to data driven decision making inhealthcare.

Limitations

Roadmap


References

1 Bloom, N., Propper, C., Seiler, S and Van Reenen J. (2015). The Impact of Competition

on Management Quality: Evidence from Public Hospitals. The Review of Economic

Studies, 0:1–33

2 Bloom, N. and Van Reenen, J. (2007). Measuring and explaining management practices

across rms and countries. The Quarterly Journal of Economics, 122(4):1351–1408.

3 Bray, D. B., Ayis, S., Campbell, J., Hoffman, A., Roughton, M., Tyrrell, P.J., Wolfe, C.

and Rudd, A. 2013. Associations between the organisation of stroke services, process

of care, and mortality in England: prospective cohort study. BMJ 346:f2827

4 Breiman, L. 2001. Statistical Modeling: The Two Cultures. Statistical Science. 16(3),

pp. 199–231 Brynjolfsson, E. and Milgrom, P. (2013). Complementarity In Orga-

nization. In Gibbons, R. and Roberts, J., editors, The Handbook of Organizational

Economics. Princeton University Press.

5 Chandra, A., Finkelstein, A., Sacarny, A., and Syverson, C. (2013). Healthcare ex-

ceptionalism? Productivity and allocation in the us healthcare sector. NBER Working

Paper, National Bureau of Economic Research.

6 Chetty, R. (2015). Behavioral economics and public policy: A pragmatic perspective.

7 Donabedian, A., 1980. The Definition of Quality and Approaches to Its Assessment.

Ann Arbor, MI: Health Administration Press.


References

8 Friedman, M. (1953). The methodology of positive economics. Essays in Positive

Economics, 3(3).

9 McConnell, K. J., Chang, A. M., Maddox, T. M., Wholey, D. R., and Lindrooth, R. C.

(2014). An Exploration of Management Practices in Hospitals. Healthcare, 2(2):121–

129.

10 McConnell, K. J., Lindrooth, R. C., Wholey, D. R., Maddox, T. M., and Bloom, N.

(2013). Management practices and the quality of care in cardiac units. JAMA Internal

Medicine, 173(8):684–692.

11 Milgrom, P. and Roberts, J. (1995). Complementarities and fit strategy, structure,

and organizational change in manufacturing. Journal of Accounting and Economics,

19(2):179–208.

12 Palmer, K. (2012). Stronger incentives for quality improvement needed in NHS in

England. Journal of Health Services Research & Policy, 17(2):65–67.

13 Ramanujam, R. and Rousseau, D. M. (2006). The challenges are organizational not

just clinical. Journal of Organizational Behavior, 27(7):811–827.

14 Dranove, D., Forman, C., Goldfrab, A. and Greenstein, S. (2014). The Trillion Dollar

Conundrum: Complementarities and Health Information Technology. American Eco-

nomic Journal: Economic Policy, 6(4): 239-70


References

15 Rubin, H. R., Pronovost, P., and Diette, G. B. (2001). The advantages and disadvan-

tages of process-based measures of health care quality. International Journal for Quality

in Health Care, 13(6):469–474.

16 Sela, R. J. and Simonoff, J. S. (2012). RE–EM trees: a data mining approach for

longitudinal and clustered data. Machine Learning, 86(2):169–207.

17 Simonoff, J. S. and Fu, W. (2014). Unbiased Regression Trees for Longitudinal Data.

SSRN Working Paper.

18 Syverson, C. 2011. What Determines Productivity? Journal of Economic Literature,

49(2), pp. 326–365

19 Ukawa, N., Ikai, H., and Imanaka, Y. (2014). Trends in hospital performance in acute

myocardial infarction care: a retrospective longitudinal study in japan. International

Journal for Quality in Health Care, 26(5):516–523.

20 West, E. (2001). Management matters: the link between hospital organisation and

quality of patient care. Quality in Health Care, 10(1):40–48.


AppendixData Sources

1 National Sentinel Stroke Audit 2004 to 2010, Royal College of Physicians

2 Hospital Estate and Facilities Data

3 NHS Workforce Statistics

4 Department of Health QMCO

5 Health and Social Care Information Centre (HSCIC)

6 Office for National Statistics; ASHE

7 NOMIS

8 QOF
