Leeds Institute of Health Sciences

Regional variation in health care system performance: A dual-level efficiency approach applied to NHS pathology in

England

John Buckell1,2, Andrew Smith2,3, Phill Wheat2, Roberta Longo1, David Holland4

1. Academic Unit of Health Economics University of Leeds2. Institute for Transport Studies, University of Leeds3. Leeds University Business School, University of Leeds4. Keele Benchmarking Unit, Keele University

Leeds Institute of Health Sciences

Outline: Regional Performance

1. Motivation

2. Methods

3. Data and Results

4. Interpretation of Mundlak group mean variables and model selection

Leeds Institute of Health Sciences

1. Motivation – why are we interested in regional performance?

(a) Policy interest: (i) expenditure savings (NHS, 2010; 2013) and (ii) regional inequalities (horizontal equity) (Marmot et al. 2010; NHS 2013)

(b) Literature gap: no regional performance measure per se

Apply a dual-level model to pathology services in the NHS in England

Leeds Institute of Health Sciences

2. Methods

• Dual-level stochastic frontier (Smith and Wheat, 2012) (DLSF)

SHA1

Laboratory 1

Laboratory 2

Laboratory 3

SHA2

Laboratory 4

Laboratory 5

Laboratory 6

• Upper level (SHA): Central management, configuration of

laboratories

• Lower level (laboratories): Local management, degree of

autonomy, local conditions

Where does inefficiency reside within the organisation?

Then, regional inefficiency is simply the product of the SHA (upper tier) and laboratory (lower tier) inefficiency

Leeds Institute of Health Sciences

DLSF based on a simple cost SF:

Where c are costs, x are regressors, u is inefficiency and v is noise.

Can decompose uil into SHA- and laboratory-specific components:

Then, we can construct our regional measure as:

For estimation, we use the two-stage RE FGLS/SF procedure outlined in Smith and Wheat (2012)

Leeds Institute of Health Sciences

Unobservable heterogeneity: Mundlak transform

Following Mundlak (1978) and Farsi et al. (2005), it is possible to partition a fixed effect into two components: one which is correlated with the regressors, and one which varies randomly,

If it is assumed that the UOH is correlated with the regressors, then delta represents inefficiency.

By averaging over SHA,

And we can then add group means of Xs directly into the regression and estimate RE FGLS

Leeds Institute of Health Sciences

Strategy

• First, compare DL models to single level models

• Second, compare Mundlak-adjusted model to non-adjusted model

• Third, discuss interpretation and model selection

Leeds Institute of Health Sciences

3. Data and Results

• We use pathology benchmarking data from Keele University combined with labour force survey data and NHS trust data

• 57 laboratories, 10 SHAs, 5 years (unbalanced panel)

• Costs and wage data are adjusted using the consumer prices index

Variable Mean S.D. Min Max

Operating costs (adjusted) 3617320 2058358 963875 11741895

Number of tests 5037362 2990846 1380384 30199502

Number of requests 714125 465535 191078 4423531

Input prices (Labour) (adjusted) 24551 4160 15834 49955

Leeds Institute of Health Sciences

Results: DLSF vs single level (SL) models

DLSF versus single level models

• Parameter estimates similar between models• Parameter estimates match theory and previous work (Buckell et al., 2013;

Gutacker et al., 2013)• Inefficiency is detected at both levels of the DLSF

Dual-level (stage 1) Single-level SHA SF Single-level laboratory SF

Variable

Constant (-)5.275 (1.74)*** (-)5.275 (1.74)*** 2.315 (2.831)OUTPUT 0.857 (0.043)*** 0.857 (0.043)*** 0.506 (0.043)***INPUT PRICES 0.775 (0.157)*** 0.775 (0.157)*** 0.484 (0.289)*TESTS:REQUESTS 0.520 (0.070)*** 0.520 (0.070)*** 0.284 (0.054)***TIME 0.009 (0.013) 0.009 (0.013) 0.017 (0.010)*METROPOLITAN 0.193 (0.050)*** 0.193 (0.050)*** 0.202 (0.111)*FOUNDATION (-)0.084 (0.044)* (-)0.084 (0.044)* (-) 0.055 (0.082)TEACHING 0.013 (0.042) 0.013 (0.042) 0.079 (0.078)

Moulton-Randolph 3.11 3.11Lambda 3.24***

Lambda (2nd stage) 1.58***

     

Leeds Institute of Health Sciences

  Mean CE SHA Mean CE LAB  Mean CE Region 

SHA-level frontier 0.795 n/a 0.795

LAB-level frontier n/a 0.699 0.699

DLSF 0.795 0.809 0.643

• Single level SHA SF model cannot measure inefficiency at lower (laboratory) level

• Single level laboratory SF cannot disentangle laboratory and SHA inefficiency; ascribes both levels’ performances to lower level

• Both single level models underestimate inefficiency at regional level (in keeping with Smith and Wheat, 2012)

Leeds Institute of Health Sciences

DLSF with Mundlak vs. DLSF without Mundlak

• Lower level estimates are identical for both models (because we have removed SHA-specific effects in the first stage)

• Significantly different estimates with/without Mundlak transform applied• Ranking of SHAs/regions sensitive to transformation

SHA Dual-level SF (Mundlak) Dual-level SF 

SHA CE Laboratory CE (weighted) Regional CE SHA CE Laboratory CE (weighted) Regional CE

1 0.973 0.793 0.772 0.908 0.793 0.6122 0.976 0.830 0.810 1.000 0.830 0.6733 0.966 0.784 0.757 0.795 0.784 0.5944 0.974 0.809 0.788 0.789 0.809 0.6385 0.973 0.821 0.798 0.658 0.821 0.6556 0.969 0.793 0.769 0.714 0.793 0.6107 0.968 0.832 0.806 0.719 0.832 0.6718 0.952 0.774 0.737 0.808 0.774 0.5709 0.969 0.823 0.797 0.780 0.823 0.656

10 1.000 0.784 0.784 0.781 0.784 0.615      

Avg. 0.972 0.804 0.781 0.795 0.804 0.629s.e. 0.012 0.021 0.023 0.098 0.021 0.034

Leeds Institute of Health Sciences

4. Interpretation of Mundlak group mean variables

Interpretation of Mundlak (group mean) variables

Literature appears divided on the issue

• Earlier applications ascribed interpretation of the variables as the realisation of unobservable heterogeneity (Farsi et al., 2005a; Farsi et al., 2005b; Farsi et al., 2007)

• Later, papers seemed not to comment on the interpretation of the group means (Abdulai et al., 2007; Last et al., 2010; Titus et al., 2012; Emvalomatis 2012; Menagaki, 2013)

• Recent papers have explicitly stated that these variables have no meaning (Hunt et al., 2012 – citing Baltagi, 2006)

Does ‘meaning’ refer to elasticity or to the decomposition of inefficiency?

Leeds Institute of Health Sciences

Results: Mundlak transform

DLSF with Mundlak retrieves within estimatesDual-level RE Mundlak Dual level FE

Variable

Constant No constantOUTPUT 0.852 (0.042)*** 0.852 (0.045)***INPUT PRICES 0.910 (0.163)*** 0.910 (0.175)***TESTS:REQUESTS 0.522 (0.067)*** 0.522 (0.072)***TIME 0.007 (0.013) 0.007 (0.013)METROPOLITAN 0.187 (0.048)*** 0.187 (0.052)***FOUNDATION (-)0.074 (0.043)* (-)0.074 (0.046)TEACHING 0.029 (0.040) 0.029 (0.043)

REQBAR 0.637 (0.346)*INPBAR (-)0.132 (0.338)TESBAR 0.812 (0.737)YEABAR 0.095 (0.107)AREBAR 0.107 (0.160)FOUBAR (-)0.507 (0.276)*TEABAR 1.151 (0.395)***     Wu test (Wald Chi-squared) 29.57***       

Leeds Institute of Health Sciences

Thank You