Estimation of the Impact of Contextual Variables in Stochastic Frontier

Production Function Models Using Data Envelopment Analysis: Two-

Stage versus Bootstrap Approaches

Rajiv Banker

Fox School of Business and Management, Temple University, Philadelphia, Pennsylvania 19122, banker@temple.edu

Ram Natarajan

School of Management, The University of Texas at Dallas, Richardson, Texas 75083, nataraj@utdallas.edu

Daqun Zhang

College of Business, Texas A&M University-Corpus Christi, Corpus Christi, Texas 78412, david.zhang@tamucc.edu

Banker and Natarajan (Operations Research 2008) identified sufficient conditions in a stochastic

framework to justify the popular two-stage DEA+OLS approach to estimate the impact of

contextual variables on productivity. Simar and Wilson (2007) published an alternative approach

involving a second-stage truncated regression model and bootstrap method to estimate confidence

intervals. We show that the effectiveness of the Simar-Wilson approach critically depends on the

assumption that the actual data generating process (DGP) exactly matching their assumed DGP and

their approach does not yield correct inferences in environments characterized by stochastic

frontiers of the type first proposed by Aigner, Lovell and Schmidt (1977). Extensive simulations

from a stochastic frontier data generating process document that the simple two-stage DEA+OLS

model outperforms the more complex Simar-Wilson model with lower mean absolute deviation

(MAD), lower median absolute deviation (MEAD) as well as higher coverage rates when the

contextual variables significantly impact productivity.

Subject classifications: organizational studies: productivity, effectiveness/performance;

statistics: nonparametric; probability: stochastic model applications; simulation: applications.

Area of review: Optimization.

1

1. Introduction

Hundreds of research studies have employed Data Envelopment Analysis (DEA) to estimate

the efficiency of decision making units (DMUs) and then evaluate the impact of contextual (or

environmental) variables on efficiency in a second stage analysis. Most of these studies use either

ordinary least squares (OLS) or Tobit regression (Hoff 2007; McDonald 2009; Simar and Wilson

2011) without specifying a data generating process (DGP) to justify their two-stage approach.

Banker and Natarajan (BN, 2008) describe a set of sufficient conditions under which OLS in the

second stage yields consistent estimators of the impact of contextual variables. Simar and Wilson

(SW, 2007) also describe a DGP to promote an alternative bootstrap approach with a truncated

regression model.

Simar and Wilson (SW, 2011) attempt to argue why the SW (2007) bootstrap approach is the

best. However, their model relies on a key assumption about the manner in which contextual

variables impact production. This assumption precludes statistical noise and, more importantly,

lacks economic content, which render it inconsistent with the long tradition of empirical research

modeling production functions, such as the so-called composed error stochastic frontier estimation

(SFE) method introduced by Aigner, Lovell and Schmidt (ALS, 1977) and Meeusen and van den

Broeck (MvB, 1977). The main objective of our paper is to explain why the BN approach is more

attractive for researchers who conceptualize the production function in the ALS-MvB composed

error tradition.

In this study, we assume the true DGP is the traditional composed error stochastic frontier

production model. We compare and contrast the performance of the BN and SW approaches both

of which assume DGPs that are different from the traditional composed error stochastic frontier

production model to develop their estimation approaches. Simulation results indicate that, overall,

the simple DEA+OLS method and to a lesser extent the DEA+Tobit method dominate both of SW’s

approaches in terms of lower mean absolute deviation (MAD), lower median absolute deviation

2

(MEAD) as well as higher coverage rates when the contextual variables significantly impact

productivity. This result holds whether contextual variables and inputs are independent or

correlated, whether there is only one input or multiple inputs and even when there is no statistical

noise. Even the two-limit censored Tobit model that lacks theoretical justification in the BN

framework outperforms the truncated regression + bootstrap approach in SW (2007). These results

indicate that the SW bootstrap method is not appropriate if researchers believe the true DGP is from

a composed error model and question the validity of inferences drawn in many empirical studies

that rely solely on the SW approach.

The remainder of this paper is organized as follows. In the next section, we provide a brief

overview of alternative approaches to estimating the impact of contextual variables in noisy

production environments. We also discuss the difference in the assumptions underlying the BN and

the SW models to assess their relative suitability for efficiency analysis in stochastic environments.

In Section 3, we describe the design of our Monte Carlo experiments and in Section 4 we present

the results. We conclude in Section 5 with a discussion of implications for empirical research in

productivity analysis.

2. Estimating the impact of contextual variables in noisy production environments

The idea that contextual or environmental factors play a significant role in explaining the

deviation of actual values from the frontier has long been an underlying motivation in frontier

models in production economics. Early models of production frontiers such as those in Aigner and

Chu (1968), Afriat (1972) and Richmond (1974) focused on only the detrimental, output-decreasing

effects of these environmental or contextual variables. However, in their more generalized and

pioneering SFE models that featured composed error terms Aigner, Lovell and Schmidt (1977),

ALS hereafter, and Meeusen and van den Broeck (1977), MvB hereafter, allowed for both

detrimental effects as well as beneficial effects from the contextual or environmental variables. In

their discussion justifying about the composed error term ALS appeal to the insights from several

3

prior studies such as Marschak and Andrews (1944), Zellner et al. (1966), Aigner and Chu (1968)

and Timmer (1971). They assert that contextual or environmental variables can influence the output

through both the two-sided noise term and the one-sided inefficiency term. Further, in motivating

the inclusion of the two-sided random disturbance term in the composed error structure, ALS also

explicitly mention that the stochasticity of the frontier arises as a result of “favorable and

unfavorable external events such as luck, climate, topography and machine performance” (p. 25,

ALS). In a similar vein, MvB describe the two-sided error term as a mechanism to account for the

“randomness in the real sense and to specification and measurement errors” (p. 436, MvB). Clearly,

the main takeaway from these studies is that any systematic analysis of the deviation of output from

the frontier requires a well-specified DGP that explicitly takes into account the two-directional

stochastic nature of the frontier.

Much of the early research on impact of contextual variables on productivity in different

industries was based on the ALS stochastic frontier framework. Using this framework, Ruggiero

and Vitaliano (1999) for instance, assert that the effect of single-parent households, poverty rate

and school-age children on school district productivity is stochastic and a significant component of

the stochastic frontier production function. Rosko (2001) examines the effect of contextual

variables such as demand for HMOs, regulatory pressures of public payers, competitive pressure

and profit-orientation on hospital costs using a SFE framework. His results suggest that the hospital

production function exhibits a significant degree of stochastic behavior and that environmental

factors explain a significant degree of variation in cost efficiency across hospitals. A number of

other studies in various manufacturing and service industries such as agriculture, education

institutions, banks, insurance and utilities have also used the SFE approach (see for e.g., Izadi 2002,

Li and Rosenman 2001, Fenn at al. 2008). A common feature of the empirical results documented

in these studies is the support for the argument that factors other than pure inefficiency may affect

output favorably or unfavorably.

4

We performed an extensive search of studies during the 1980s and early 1990s that examined

factors affecting productivity in different industries using the ALS framework. There were two

different types of variables identified by researchers as (a) exogenous variables or factors out of

management control that directly contributed to the output and the existence of which was needed

in order to describe why the “noise” term was required in the research specification, and (b)

endogenous variables under management control affecting output indirectly through the one-sided

inefficiency term. In appendix 1 we list, by industry, both exogenous and endogenous types of

contextual variables identified in studies based on the ALS model.

A popular alternative to the SFE approach in analyzing the effect of contextual variables has

been the two-stage DEA method which was used by many researchers in the 1990s and early 2000s,

and continues to be used at present. In this approach the efficiency scores are first estimated using

DEA and in a second stage either OLS or Tobit is used in regressions of these efficiency estimates

on a number of contextual variables.1 The implicit argument in many of these studies is that the

benefit of using an apparently deterministic method such as DEA in a stochastic setting

characterized by an unknown technology outweighs the cost of ignoring the noise in the production

environment.2

The lack of articulation of a DGP (data generating process) consistent with the two-stage DEA

and the ad hoc speculations underlying the explanations of productivity differences led to

theoretical investigation of the statistical underpinnings the two-stage DEA method (Grosskopf

1996; Forsund 1999). In two influential studies, Banker and Natarajan (BN 2008) and Simar and

Wilson (SW 2007), have adopted substantially different approaches to argue for (BN) and against

1 Simar and Wilson (2008) list 40-50 published studies and assert that they found “hundreds of working papers” that used the two-stage DEA approach in their internet search using the phrases “data envelopment analysis” and “two-stage”. 2 Ondrich and Ruggiero (2001) point out that the rankings for firm-specific inefficiency estimates produced by traditional SFE models are identical to those based on estimates of composed errors. So if the objective is only to rank-order the cross-section of firms in terms of efficiency then it does not matter whether a deterministic or SFE method is used.

5

(SW) the statistical validity of the two-stage DEA approach. We explain these two approaches in

greater detail in the next two sub-sections. The important and fundamental difference between the

two studies is that BN justify DEA followed by OLS as a valid approach to analyze the effect of

contextual variables in stochastic production environments under certain specified conditions

whereas SW rule out the applicability of the DEA+OLS approach and instead recommend

DEA+truncated regression+bootstrapping as their recommended method. Moreover, SW

completely ignore the role of two-sided noise in the production environment which raises critical

doubts about the applicability of their method in stochastic environments.

Because of its applicability in stochastic environments, the two-stage DEA+OLS approach

continues to be the method of choice in estimating the impact of contextual variables. However, a

significant minority of researchers appears to have bought into the strident criticism of the DEA +

OLS method by SW and have started reporting results only with SW’s DEA + truncated regression

approach.3 Interestingly, almost all of these studies have conducted their analyses in industries

such as educational institutions, farms, hospitals, banks, insurance companies (see, for example,

Afonso and Aubyn 2006, Latruffe et al. 2008, Blank and Valdmanis 2010, Barros et al. 2010). As

discussed earlier, these same industries have been modeled using the composed error stochastic

frontier approach in the past, many with extensive discussion motivating the two-sided exogenous

noise and one-sided inefficiency factors. It is questionable whether the assumptions justifying the

SW approach (especially the assumption of no two-sided noise) are valid in these industries or

environments. The citation counts reported in Table 1 indicate that the SW approach is now almost

as common as the SFE approach of ALS and MvB.4 Almost all of the studies using the SW

3 An internet search on Google Scholar for the period 2007-2015 with search criteria “DEA”+ “Second Stage” + “OLS” returned 1750 hits while a corresponding search for “DEA”+ “truncated regression” returned 766 hits. A more focused search that also included “contextual variables” to the above strings returned 227 and 112 hits, respectively. (Search date: September 27, 2015) 4 As of September 27, 2015, forty four studies citing SW are published in the Journal of Productivity Analysis based on the search on Google Scholar.

6

approach do not report corresponding SFE or DEA + OLS estimates, so the conclusions they draw

about the impact of contextual variables remain questionable.

Table 1: Citation results from Google Scholar5

Type of Environment Published articles and working

papers citing ALS

Published articles and working

papers citing SW

Before 2007 After 2007 Before 2007 After 2007

Hospitals 81 208 10 135

Banks 158 487 9 180

Insurance 149 392 4 147

Farms 337 696 3 114

Education Institutions 126 585 15 173

Utility 124 331 0 119

Search date: September 27, 2015.

We summarize in Appendix 2, by industry, the various contextual factors examined by the

studies following the SW approach. The important point to note is that many of these variables

(shown in italics) are identical to those identified by the early ALS researchers as exogenous

variables that contribute to the two-sided stochastic “noise” term. Since the SW approach cannot

include factors that affect output directly because of the absence of the “noise” term, studies using

5 The key words for studies citing ALS are, “Stochastic Frontier” + “Environmental” OR “Contextual” + “Hospitals”, “Stochastic Frontier” + “Environmental” OR “Contextual” + “Banks”, “Stochastic Frontier” + “Environmental” OR “Contextual” + “Insurance”, “Stochastic Frontier” + “Environmental” OR “Contextual” + “Farms”, “Stochastic Frontier” + “Environmental” OR “Contextual” + “Education Institutions" OR “Universities” OR “Schools” OR “Colleges”, “Stochastic Frontier” + “Environmental” OR “Contextual” + “Utility” The key words for studies citing SW are, “Environmental” OR “Contextual” + “Two Stage” +“Bootstrap”+ “Hospitals” , “Environmental” OR “Contextual” + “Two Stage” + “Bootstrap”+ “Banks”, “Environmental” OR “Contextual” + “Two Stage” + “Bootstrap”+ “Insurance”, “Environmental” OR “Contextual” + “Two Stage” + “Bootstrap” + “Farms”, “Environmental” OR “Contextual” + “Two Stage” + “Bootstrap” + “Education Institutions" OR “Universities” OR “Schools” OR “Colleges”, “Environmental” OR “Contextual” + “Two Stage” + “Bootstrap” + “Utility”

7

this approach constrain the cumulative effect of the included endogenous variables to be a part of

the one-sided inefficiency effect. Any miscategorization of exogenous variables by including them

in the one-sided inefficiency term leads to erroneous estimation of the impact of contextual factors

on productivity.

In this paper, we examine the performance of both DEA+OLS and DEA+truncated regression

methods in terms of their ability to estimate and draw inferences regarding the impact of contextual

variables on efficiency when the data generating process is typical of a noisy production

environment. In the subsections that follow we first compare and contrast the BN and SW

approaches and then in Sections 3 and 4 present estimation and inference results from extensive

simulations.

2.1 Banker and Natarajan’s (2008) approach to modelling the effect of contextual variables

in stochastic environments

Based on the composed error model in ALS (1977) and MvB (1977), BN (2008) first develop

a set of sufficient conditions to justify the widely used OLS estimator in a second stage analysis

following DEA estimation in the first stage. Suppose there are N decision making units (DMUs)

with a vector of inputs, 1( ,..., )mX x x , and a vector of outputs 1( ,..., )sY y y . The production

possibility set T is defined as {( , ) | can be produced from }T Y X Y X , and T satisfies the

regularity conditions of monotonicity and convexity. The inefficiency of a DMUi can be measured

by,

max{ | ( , ) }i i iY X T , (1)

where i ranges over [1, ∞], 1( ,..., )i i miX x x and 1( ,..., )i i siY y y . Correspondingly, i can be

estimated by a DEA inefficiency estimator ( i ) as follows,

1 1 1

ˆ max{ | , , 1, 0, 1,..., }n n n

i i j j i j j j jj j j

Y Y X X j n

. (2)

8

Because of the constraint 1

1n

jj

, the production possibility set T exhibits variable returns to

scale (Banker, Charnes and Cooper 1984).

The DEA estimator in equation (2) used to be thought of as deterministic that lacked statistical

foundation (Schmidt 1985). To derive statistical properties for the DEA estimator, Banker (1993)

models the deviation of the actual output from the production frontier as a one-sided inefficiency

term, which is distributed independently of the inputs. Furthermore, he requires the probability

density function of the inefficiency term in the neighborhood of the production frontier to be

positive so that at least some observations are likely to be realized arbitrarily close to the frontier

in large samples. Under these regularity conditions, he proves that the DEA estimator exhibits the

asymptotic property of consistency, even though it is biased for a finite sample size. BN (2008)

extend Banker’s (1993) model and assume the deviation consists of not only a one-sided

inefficiency term (u) but also a two-sided noise term (v). That is the inefficiency of the DMUi can

be represented as,

ln( )i i i i iu v , (3)

where the noise term iv is bounded above at MV for technical reasons.6 BN’s specification of v

and u is analogues to the composed error model in ALS (1977) and MvB (1977).

As the transformation of inputs to outputs could be affected by a DMU’s external operating

environment, the deviation ( i ) is specified by the following equation to incorporate exogenous

contextual variables,

ln( ) ( ; )i i i i ih Z u v , (4)

6 In extensive simulations they document that the model performance is equally good when v is distributed as Normal without any bounds. Our simulations confirm this assertion.

9

where Zi refers to a vector of contextual variables and the impact of Zi on i is characterized by

the function ( ; )ih Z (Yu 1998; Fried et al. 2002; SW 2007). If ( ; )ih Z is a linear function in Z,

equation (4) can be written as,

ln( )i i i i iZ u v . (5)

The i iZ v terms above can be interpreted as a part of the random disturbance term in the

ALS (1977) composed error model. That is, the contextual variables ( Z ) affect the stochastic

production frontier rather than the managerial inefficiency. The term iv in equation (5) represents

the impact of various omitted variables not included in Z, as well as measurement errors of output.

To link the composed error model in equation (5) with the usual DEA model, BN (2008)

redefine the deviation in equation (5) by scaling it up by a constant MV so that it is bounded below

at 0. That is,

( ) 0Mi i i iZ u V v . (6)

As iv is bounded above at MV we have ( ) 0MiV v .7 To simplify exposition, BN (2008)

assume all contextual variables (Z) are measured to be positive and higher values of the contextual

variables mean greater inefficiency to guarantee 0iZ in equation (6). 8 As Johnson and

Kuosmanen (2012) show in Theorem 1, this assumption can be easily relaxed if we assume that the

domain of Z is bounded and iZ is finite. The inefficiency, i , in equation (5) can be scaled up by

a constant, V, so that 0i i i iZ u V v for i = 1, …, N. Note, analogous to the composed

error model, iu and iv are assumed to be homogenous. 9

7 The bounded noise does not affect the estimation of , as MV is a constant. The simulation results in

Section 4 show that OLS estimator works well even when the noise term is unbounded. 8 It is straightforward to accommodate the opposite orientation in this model and hence omitted here. 9 SW (2007) also assume homogeneous error term ( i ) in their DGP described in Section 2.2.

10

As the logarithm of DEA inefficiency in equation (2) is a consistent estimator of i , BN (2008)

show that the impact of contextual variables can be estimated by regressing the logarithm of DEA

estimate, ˆlog( )i , on the exogenous contextual variables. To derive the consistency property of

the OLS estimator of , BN (2008) identify a regularity condition requiring that contextual

variables are independent of the inputs, but these contextual variables are allowed to be correlated

with each other.10 Their Monte Carlo simulation results show that two-stage DEA approaches,

using either OLS or MLE in the second stage, outperform both one and two stage parametric SFE

method if the production function is misspecified, even when the distribution of two-sided noise

term is unbounded.

The main contribution of BN (2008) is to identify sufficient conditions under which the DEA

+ OLS generates consistent estimators of the impact of contextual variables. BN do not advocate

that DEA+OLS should always be used when these sufficient conditions are severely violated. For

instance, if researchers have prior knowledge that a contextual variable Z affects the selection of

inputs X, then Z should be included and modeled in the first stage, but not be separated from X and

modeled in the second stage.11 If Z affects the managerial effort ( u ), then the maximum likelihood

estimation (MLE) should be used in the second stage to properly model the joint distribution of u

and Z. In the presence of a heteoskedastic noise term, the OLS estimator of is still consistent,

though it may not be efficient. If researchers know the variance structure of the noise term, they

can easily use generalized least squares (GLS) instead of OLS to handle the issue of

heteroskedasticity.

10 Contextual variables are required to be independent of input variables, otherwise the finite sample bias of DEA estimator in the first stage will lead to biased coefficient estimates on contextual variables in the second stage. However, such bias disappears when the bias of DEA estimator in the first stage vanishes as sample size gets large. 11 This independence condition applies to SW’s (2007) DGP as well.

11

BN’s specification for i in equation (5) can be easily extended to include a vector of

variables Ri that impact one-sided inefficiency so that,

ln( )i i i i i iZ R u v (7)

The above discussion applies directly and equally well to the estimation of and hence

omitted.

2.2 Simar and Wilson’s (2007) truncated regression and bootstrap model

Simar and Wilson (2007) propose an alternative DGP. They assume that the deviation of the

actual output from the production frontier arises from a one-sided inefficiency term whose

distribution is affected by a vector of contextual variables. The inefficiency that cannot be explained

by contextual variables has a left truncated normal distribution with a firm-specific truncation point

that is determined by realizations of contextual variables. Based on this DGP, SW(2007) claim that

both the popular OLS and two-limit censored Tobit models in the existing literature are not valid

ways to estimate the impact of contextual variables. Instead, they advocate a truncated regression

model to estimate the impact ( ) of contextual variables in the second stage. Furthermore, they

propose two bootstrap procedures to make statistical inferences about .

SW (2007) maintain eight assumptions to justify their two-stage DEA approach. Their

assumptions A1 to A3 specify inputs, outputs and contextual variables are identically and

independently distributed (i.i.d.), and describe how they are sequentially generated. Assumptions

A4 to A6 about the production possibility set are adopted from Shephard (1970) and Fare (1988).

For the consistent estimation of production possibility set and technical efficiency, SW (2007)

invoke two additional assumptions A7 and A8 from Banker (1993) and Kneip et al (1998).

Assumption A7 requires the likelihood of observing DMUs in a neighborhood of the production

frontier to be strictly positive. Assumption A8 says the production frontier is smooth so that the

technical efficiency is differentiable with respect to inputs and outputs.

12

SW’s assumption A2 postulates contextual variables iZ directly affect the inefficiency of DMU

i in the following manner:

1i i iZ , (8)

where error term i represents the part of inefficiency that cannot be explained by contextual

variables. SW (2007) assume the unexplained term i is independently and identically distributed,

and is also independent of contextual variables iZ .12 SW assume the error term i in equation (8)

is normally distributed as 2(0, )N left truncated at 1 iZ .

The above DGP is inconsistent with the composed error model in parametric SFA literature. To

attempt a connection, SW (2007) modify their specification to,

exp( ) 1i i iZ , (9)

where the error term i once again is iid and independent of iZ . Then the logarithm of inefficiency

is affected by contextual variables through

log( ) 0i i i iZ . (10)

SW assume the error term i is normally distributed as 2(0, )N left-truncated at iZ so

that the conditional probability density function ( | )i ip Z is,

1( )

, 0,( | ) 1 ( )

0, 0,

i i

iii i

i

Z

ifZp Z

if

12 Strictly speaking, i are not identically distributed as stated by SW (2007), because the left-truncation

point of their distribution varies across DMUs as defined in equation (10).

13

where (.) refers to the standard normal density function, and (.) refers to the standard normal

cumulative density function.

Under the null hypothesis that contextual variables do not impact the inefficiency ( 0 ),

equation (10) reduces to 0i i and the probability density function of i becomes

2( ) ( ), for 0i

i ip

, which is a half normal distribution. This means that SW’s

modified DGP under the null hypothesis corresponds to the DGP in ALS without their noise term.

However, when the null hypothesis is not true ( 0 ), the conditional density function,

( | )i if Z , is always left truncated at 0, but the shape of the distribution varies with the realizations

of contextual variables, iZ .

Even a casual comparison with the SFE composed error model reveals an obvious and

fundamental weakness of SW’s DGP: SW cannot accommodate the two-sided noise term.

Furthermore, SW require that contextual variables affect production process only through technical

inefficiency, a strong separability assumption which is unlikely to be satisfied in reality. SW (2011)

admit, “The null hypothesis constitutes a strong assumption, and we expect that in many samples,

the null will be rejected. As an example, Daraio et al. (2010) revisit the empirical example based

on Aly et al. (1990) that was presented in SW, and easily reject separability. (P.4)”

Another issue is that SW’s DGP lacks managerial effort does not play an explicit role in

technical inefficiency according to Assumption A2 and A2a. The unexplained error i in equation

(8) and (10) is difficult to interpret. If the error term i reflects managerial slacks after filtering the

effect of contextual variables, it is not clear why i should have a normal distribution left truncated

at DMU-specific points depending on realizations of contextual variables. When 0 , the

expectation of i conditional on contextual variables Zi is,

14

[( / )]( | )

1 [( / )]i

i i ii

ZE Z

Z

.

That is the expected managerial inefficiency, ( | )i i iE Z , is determined by the realizations

of contextual variables. Clearly, ( | )i i iE Z is monotonically decreasing in iZ and

asymptotically converges to 0 as iZ increases to infinity, implying that expected inefficiency

associated with managerial effort decreases to 0. In contrast, ( | )i i iE Z increases as iZ

decreases, implying that the expected inefficiency attributed to managerial slack increases as

operating environments become favorable. If the contextual variables of DMUs are measured

positive and higher values mean higher inefficiency, SW’s DGP implicitly assumes that DMUs in

unfavorable environment (with greater iZ ) on average attribute lower inefficiency (

( | )i i iE Z ) to managerial slack than those in favorable environment (with smaller iZ ).

However, there is no economic or behavioral theory to justify this underlying assumption.

To draw statistical inferences from the truncated regression they propose in the second stage,

SW (2007) argue, “Conventional approaches to inference employed in these papers are invalid due

to complicated, unknown serial correlation among the estimated efficiencies. (P.31)” Based on this

argument, they specify two bootstrap approaches to construct the confidence intervals for . The

first approach regresses the technical inefficiency obtained by DEA models on contextual variables

directly, and then uses a bootstrap procedure to estimate the confidence interval of (referred to

as “algorithm #1” and “single bootstrap approach” in SW 2007). Their second approach is a double

bootstrap procedure. The bias-corrected inefficiency is estimated by a first bootstrap procedure

derived from their DGP and then the confidence interval of is estimated by the second bootstrap

procedure (referred to as “algorithm #2” and “double bootstrap approach” in SW 2007). The point

to be noted is that SW’s bootstrap approaches are valid only if their DGP assumptions are valid.

15

3. Design of Monte Carlo Experiments

3.1 Data Generating Process

We conduct 1000 Monte Carlo experiments to evaluate the performance of the simple DEA +

OLS method against SW’s bootstrap approaches when the true DGP is the widely used composed

error model. In this section, we describe the simulation experiments in detail. We specify a

contextual variable z, allow for the possibility that it may be correlated with the input variables x,

include both the one-sided error term u and the two-sided error-term v and specify the output-

generating process as ( ) v uy g x z e ; That is

ln( ) ln( ( )) ln( )y g x z v u . (11)

We consider both one input and two inputs scenarios. When there is a single input, we assume

( )g x can be represented by following translog function,

21 0 1 1 2 1( ) ln( ) (ln( ))g x x x .

We draw the input variable 1x from a uniform distribution over the interval [1, 5]. The properties

of 1( )g x are determined by the coefficients 0 , 1 and 2 . Here, we set values for these

coefficients ( 0 1 , 1 0.4 , 2 0.1 ) such that ( )g x is continuous, monotone increasing

and concave in the range [1,5].

When there are two inputs, ( )g x is specified as

2 21 2 0 1 1 2 1 3 2 4 2 5 1 2( , ) ln ( ) (ln( )) ln( ) (ln( )) ln( ) ln( )g x x x x x x x x .

In the two-input case, we draw the input variables 1x and 2x from independent uniform

distributions over the interval [1, 5]. Here, we set coefficients 0 1 , 1 0.4 , 2 0.1 ,

16

3 0.4 , 4 0.1 and 5 0.1 , so that ( )g x is continuous, monotone increasing and

concave in inputs 1x and 2x over the interval [1, 5].

Similar to Wang and Schmidt (2002), we generate the contextual variable z as:

21[( 1)] 1 1z x w ,

when there is a single input. We specify

21 2[( 1) / 2] [( 1) / 2] 1 1z x x w when there are two inputs.

The parameter is a pre-specified correlation coefficient and w is drawn independently from the

uniform distribution [0, 4]. In our Monte Carlo experiments, we consider two situations: (1) when

the input variables and the contextual variable are independent, and (2) when they are moderately

correlated, by assigning the values 0 and 0.3 respectively. When the variables are

uncorrelated i.e., 0 , the contextual variable z, has a uniform distribution over [1, 5].

We follow prior empirical studies based on ALS (1977) and generate the inefficiency term u

from a half-normal distribution with parameter σu = 0.2, so that the expected value of efficiency

( )uE e for our Monte Carlo experiments is 0.858. 13 We draw the noise variable, v, from a normal

distribution with mean zero and standard deviation σv = 0.1. 14 To benchmark the performance of

the various approaches to a noise-less setting, we also consider an alternative DGP with no noise

(σv = 0) to favor SW’s approach.

13 In addition, as in MvB (1977), we assume the inefficiency term, u, is generated from an exponential distribution with the probability density function, f(u)= ue , where the parameter 8 . Correspondingly, the expected value of efficiency ( )uE e for the Monte Carlo experiments is 0.889. The untabulated

simulation results are similar to those reported in Section 4. 14 In BN (2008), the random noise variable, v, is generated from a two-side truncated normal distribution with upper and lower bounds, at 6 v and 6 v , so that OLS generates a consistent estimator of the

coefficient on contextual variable z . However, in this paper, we raise the ante relaxing the assumption of “bounded noise” in BN (2008) and use the untruncated normal distribution to compare the performance of the four estimation methods, including DEA+OLS, DEA+Tobit, and SW’s two approaches.

17

Table 2 presents the percentages of variance of ln( / ( ))y g x from three sources, including

technical inefficiency (u), random noise (v), and contextual variable (z) for the case when 0.2 .

When there is no noise (σv = 0), the technical inefficiency accounts for about 65.6% of variance of

ln( / ( ))y g x , while the contextual variable z accounts for about 34.4%. When there is moderate

noise (σv = 0.1), technical inefficiency, random noise and contextual variable account for 45.2%,

31.1%, and 23.7% respectively. To compare the ability of the different approaches in evaluating

the influence of the contextual variables, we consider two scenarios one when 0 i.e. the

contextual variable has no effect on the output, and 0.2 . We also examine the performance

of the various approaches to different sample sizes by considering small, medium and large samples

(n=50, 200 and 400). In total, we conduct 2 x 2 x 2 x 2 x 3= 48 sets of Monte Carlo experiments

each with 1,000 trials as summarized in Table 3.

Table 2: The percentage of the variance of ln( / ( ))y g x from technical inefficiency (u), noise (v)

and contextual variable (z) when the null hypothesis is false ( 0 .2)

Inefficiency (u) Noise (v) Percentages of variance of ln( / ( ))y g x

U V Z

Normal

( ( )uE e =0.858)

σv=0 65.6% 0 34.4%

σv=0.10 45.2% 31.1% 23.7%

Table 3: Different settings for the Monte Carlo experiments

Correlation between x

and z

Input Null hypothesis Standard deviation of v

Sample size

Independent One True ( 0 ) 0 50

Mildly correlated

Two False ( 0.2 ) 0.1 200

400

18

3.2 Estimation methods

We estimate the technical inefficiency () by an output-oriented BCC model (Banker, Charnes

and Cooper 1984),

01

01

1

Subject to:

ˆ = max

, 1,..., ,

, 1,..., ,

1, 0, 1,..., .

n

ij j ij

n

rj j rj

n

jj

x x i m

y y r s

j j n

(12)

After obtaining the DEA inefficiency scores in the first stage, we use OLS, Tobit, and SW’s

single and double bootstrap methods to estimate the impact of contextual variables. These

estimation methods in the second stage are described below in detail.

Method 1 (DEA + OLS):

As in BN (2008), we regress the logarithm of technical inefficiency on contextual variable (z)

using ordinary least squares. That is

0ˆln( ) ln( )i i iz , (13)

where i is the DEA estimate of technical inefficiency obtained from Model (12).15

Method 2 (DEA+ Censored Tobit regression):

As the range of DEA efficiency estimate ˆi , the reciprocal of the inefficiency estimate i

obtained from the BCC model in (12), is over the bounded interval (0, 1], many prior studies have

used two limit censored Tobit model to estimate the impact of contextual variables (Hoff 2007).

15 BN (2008) also propose an alternative maximum likelihood estimation in the second stage. However, as it does not show much advantage over OLS in their subsequent Monte Carlo experiments, we do not include it here for parsimony.

19

Suppose the true efficiency i is a latent variable, which can be represented as in the following

equation:

0 ln( )i i iz , (14)

If 0i , the observed DEA efficiency score i equals 0.

If 0 1i , then i equals i ,

And if 1i , then i equals 1.

Then we can use MLE to estimate the regression model 0ˆ ln( )i i iz .

Method 3 (SW’s single bootstrap approach):

According to the DGP in SW (2007), the correct regression model is,

0ˆln ln( ) 0i i iz , (15)

where i is distributed as 2(0, )N with left-truncation at 0 ln( )iz for each i.

We first use the single bootstrap approach (or “algorithm #1”) in SW (2007) to construct the

estimated confidence intervals at 0.80, 0.90 and 0.99 significance levels to make inference about

: 16

[1] Run truncated regression of ˆln i on contextual variable ln( )iz to obtain the coefficient

estimate , and the estimate ˆ , using the m DEA inefficient observations ( ˆ 1i ).

[2] Loop over Steps [2.1]-[2.3] L (=2000) times to obtain a set of bootstrap estimates

1ˆ{ }SW L

b b : 17

[2.1] For each i=1, …, m, draw i from the distribution 2ˆ(0, )N with left truncation at

0ˆ ˆ ln( )iz .

16 To facilitate the replication of our simulation results, we set the initial seed of simulation as 50000. 17 Following SW (2007), we set L = 2000.

20

[2.2] For each i=1, …, m, calculate 0ˆ ˆ ˆexp( ln( ) )bi i iz .

[2.3] Run truncated regression of ˆln bi on ln( )iz , and obtain the bootstrap estimate ˆ SW

b .

[3] Use the bootstrap estimates in 1ˆ{ }SW L

b b to construct the estimated confidence

intervals for .

Following SW (2007), we use the bootstrap estimates 1ˆ{ }SW L

b b to find values ˆ ˆ,b a such

that, ˆ ˆ ˆ ˆPr[ ( ) ] 1SWbb a . Then we get an estimated confidence interval

ˆˆ ˆˆ[ , ]a b for based on the 2000 bootstrap estimates for each trial. These confidence

intervals are then used to determine the percentage of times the true value of 0 or 0.2, as the

case may be, is inside the estimated confidence interval.

Method 4 (SW’s double bootstrap approach)

We also use the double bootstrap approach (or “algorithm #2”) to compute bias corrected

inefficiency scores in the first bootstrap procedure and then construct estimated confidence

intervals in the second bootstrap procedure:

[1] Run truncated regression of ˆln i on contextual variable ln( )iz to estimate the coefficient

, and the standard deviation ˆ , using the m DEA inefficient observations ( ˆ 1i ).

[2] Loop over Steps [2.1]-[2.4] L1 (=100)18 times to compute L1 bootstrap estimates of the

inefficiency score for DMU i, 1*1

ˆ{ }Li ib b , i = 1,…, n:

[2.1] For each DMU i =1, …, n, draw i from the distribution 2ˆ(0, )N with left

truncation at 0ˆ ˆ ln( )iz .

18 Regarding the choice of L1, SW (2007) wrote, “We and other have found that 100 replications are typically sufficient for this purpose (P.44)”.

21

[2.2] Compute the bootstrap inefficiency *ib 0

ˆ ˆexp( ln( ) )i iz , i=1,…, n.

[2.3] Obtain a bootstrap reference set {( *ix , *

iy ), i=1, …,n} by setting *i ix x , and

* *ˆ /i i i iy y .

[2.4] Compute the bootstrap estimate *ib by evaluating the inefficiency score of the

original sample observation ( ix , iy ) based on the bootstrap reference set {( *ix , *

iy ), i=1, …,n}.

[3] Compute the bias-corrected inefficiency estimate ˆi = )ˆ(ˆ

ii Bias , where )ˆ( iBias

1* *

11

ˆ( ) /L

ib ibb

L

.

[4] Run truncated regression of ˆ

ln i on the contextual variable ln( )iz to obtain the

coefficient estimate ˆ , and the estimate ˆ

.

[5] Loop over Steps [5.1]-[5.3] L2 (=2000) times to obtain a set of bootstrap estimates

21}

ˆ{ L

bSW

b :

[5.1] For each i=1, …, n, draw i from the distribution 2ˆ(0, )N with left truncation

0

ˆ ˆˆ ˆ ln( )iz .

[5.2] For each i=1, …, n, calculate 0

ˆ ˆˆ ˆexp( ln( ) )bi i iz .

[5.3] Run truncated regression of ln bi on ln( )iz , and obtain the bootstrap estimate

ˆ SWb .

[6] Use the bootstrap estimates in 21}

ˆ{ L

bSW

b to construct the estimated confidence

intervals for ˆ .

22

Following SW (2007), we use the bootstrap estimates 1

ˆ{ }SW L

b b to find values ˆ ˆ,b a such

that, ˆ ˆˆ ˆ ˆ ˆPr[ ( ) ] 1SW

bb a . Then we get an estimated confidence interval

ˆ ˆ ˆˆ ˆˆ[ , ]a b for ˆ based on the 2000 bootstrap estimates for each trial. These confidence

intervals are then used to determine the percentage of times the true value of 0 or 0.2, as the

case may be, is inside the estimated confidence interval.

3.3 Procedure for the Monte Carlo experiments

For each experimental setting we perform 1000 Monte Carlo trials, and then run 2000 bootstrap

replications for each Monte Carlo trial to construct confidence intervals for the coefficient estimate,

. Specifically, for each Monte Carlo trial t, t = 1, …., 1000, we use the following steps:

1. We generate n observations of ( , ,i i ix y z ), i=1, …, n according to the DGP described above

in Section 3.1. Then we run the BCC model in (13) to estimate the DEA inefficiency score i .

2. We run the OLS regression to obtain the coefficient estimate ( ˆOLSt ) for Monte Carlo trial t,

and the p-value of ˆOLSt using conventional inference methods.

3. We run the Tobit regression model to obtain the coefficient estimate ˆTobitt , and the p-value

of ˆTobitt using the conventional inference method.

4. We run the truncated regression based on SW’s single and double bootstrap approaches to

obtain the coefficient estimate ˆ SWt and the bootstrap distribution 1

ˆ{ }SW SW Lb b . Then we

construct the confidence intervals for ˆ SWt at three different significance levels, 99%, 95%, and

80%, under both single and double bootstrap approaches.

23

After the completion of 1000 Monte Carlo trials, we calculate the mean absolute deviation

(MAD) and the median absolute deviation (MEAD) for each estimation method (under every

experimental setup) using the 1000 estimated values of , where 1000

1

1 ˆ| |1000 t

t

MAD

and

MEAD is the sample median of the empirical distribution of the absolute deviation of from its

true value. We also compute the estimated coverages of confidence intervals under each method

for each experimental set up where the coverage is defined as the percentage of Monte Carlo trials

for which the true value of falls within the confidence interval at a specific significance level.

4. Simulation results

We employ the FEAR 1.15 package in R to estimate the BCC model in (12) and run truncated

regression in equation (15) (Wilson 2008). In Tables 4 and 5, we first present and discuss the

simulation results when the contextual variable has no effect on the output i.e., 0 and the input

x and the contextual variable z are independent.19 Specifically, Table 4 presents the mean estimates

of , MADs, and MEADs of 1000 Monte Carlo experiments when sample size equals to 50, 200

and 400 separately. We first discuss the results in the first four columns which correspond to the

case of zero noise. This is the case where given the true value of 0 , the left truncation point

under SW specification is zero for all observations and is independent of the realization of the

contextual variable. However, the operationalization of the truncated regression in the SW analysis

always assumes observation-specific left truncation point and as a result, almost always, generate

non-zero estimates. The question is, how exactly does this estimation issue affect the

19 For parsimony, we only present the simulation results for the subset of experiments with independent x and z in this section. The untabulated results for the moderately correlated x and z ( = 0.3) setup are similar

to those reported here. DEA+OLS approach still dominates SW’s single and double bootstrapping approaches in terms with lower MADs and MEADs.

24

performance of the SW methods. We find that while these methods, when compared to the

DEA+OLS and DEA+Tobit methods, generate comparable mean values (especially under large

sample sizes (N=400)) they have significantly higher MAD and MEAD values. This is likely due

to significantly higher frequency of non-zero values under the SW methods. Consistent with the

results in BN (2008), the DEA + OLS approach more consistently estimates the impact of the

contextual variable in larger samples. For instance, with the experimental setting of single input

and zero noise, the MAD of DEA + OLS decreases from 0.031 to 0.010, as sample size increases

from 50 to 400. Interestingly, the DEA + Tobit approach performs as well as DEA + OLS approach,

although the Tobit regression in the second stage does not have theoretical justification. While the

MAD and MEAD values for the SW approaches also go down with larger samples, the performance

of these methods is consistently inferior to the DEA+OLS and DEA+Tobit approaches for any

given sample size. We also don’t observe any systematic difference in performance between the

single and double bootstrap SW approaches. We believe the reason for this is that the benefits from

the efficiency bias correction procedure in the double bootstrap approach depends critically on the

actual DGP and SW’s assumed DGP being similar.

Next, we examine the results in columns 5 to 8 in Table 4. These results correspond to the case

when there is noise in the DGP. Clearly, this is a case where the assumed DGP in SW with no

provision for two-sided noise will significantly differ from the actual ALS-type DGP used for the

Monte Carlo simulations even with 0 . As a result, one would expect the performance of the

SW approaches under noise to be inferior compared to when there is no noise. The MAD and

MEAD values under the noisy setup for all sample sizes are generally higher than the counterparts

under the zero noise case. In contrast, there is hardly any difference in the MAD and MEAD

metrics between the no-noise and noise setups under both DEA+OLS and DEA+TOBIT. This

results in a bigger performance gap between DEA+ approaches and SW approaches being observed

under the noisy setup when compared to the zero noise setup.

25

Table 5 presents the coverage rates under nominal significance levels of 90%, 95% and 99%

under DEA+OLS, SW’s single and double bootstrap methods. The coverage rate, i.e. the

percentage of trials for which zero was part of the confidence interval corresponding to the chosen

significance level, is found to be higher and closer to the chosen significance level in almost all

cases under the DEA+OLS approach when compared to the SW approaches. The double bootstrap

actually has a lower coverage rate than the single bootstrap approach especially in the case where

noise is present in the DGP. In a couple of setups, especially when the sample size is large, the SW

approaches are found to have over-coverage relative to the chosen significance level. Overall, the

results in Tables 4 and 5 suggest the following: a) DEA+OLS approach does as good if not

sometimes better job than the SW approaches when the contextual variable does not have any

impact on the output b) The effectiveness of SW approaches is negatively related to the presence

of two-sided noise in the DGP and c) there appears to be no incremental benefits from the double

bootstrap approach when compared the single bootstrap approach when the actual DGP differs

from the DGP assumed in SW (2007).

Next, we examine the effectiveness of SW approaches when the contextual variables actually

have material impact on the output. Table 6 presents the results for the case when 0.2 . The

results indicate that DEA+OLS dominates the other methods and has the lowest MAD and MEAD

values. It also shows the desirable property of better estimation when sample sizes increase. Both

the single and double bootstrap procedures suffer from large outliers as is evident from the high

mean and high MAD values. The mismatch between SW’s assumed DGP and the actual DGP

appears to trigger greater inaccuracies in the ML estimation of SW truncated regression in larger

samples. While using the median absolute error (MEAD) as the metric of comparison somewhat

reduces the performance gap between DEA+OLS and the SW approaches, DEA+OLS has the

lowest of MEAD of all approaches for every experimental setup (differing sample sizes, different

26

number of inputs, with or without noise). DEA+Tobit ranks second and there is little difference

between the two SW approaches.

Table 7 presents the coverage rates under nominal significance levels of 90%, 95% and 99%

under DEA+OLS, SW’s single and double bootstrap methods for the case when 0.2 . The

coverage rate, i.e. the percentage of trials for which 0.2 was part of the confidence interval

corresponding to the chosen significance level, is found to be similar under DEA+OLS and the two

SW approaches when sample size is small i.e. N=50. When sample size increases, coverage rates

improve for DEA+OLS and approach closer to the chosen significance level. However, there is a

precipitous drop in coverage rates for the two SW approaches when sample sizes increase from

N=50 to N=200 and there is another significant drop when sample size increases to N=400. The

double bootstrap actually has a lower coverage rate than the single bootstrap approach especially

in the case where noise is present in the DGP. Overall, the results in Tables 6 and 7 suggest the

following: a) DEA+OLS approach does a significantly better job than the SW approaches when the

contextual variable does not have any impact on the output b) The mismatch between SW’s

assumed DGP and the actual DGP compounds the errors in ML estimation of SW truncated

regression in larger samples and as a result coverage rates suffer significantly as the sample size

becomes larger c) The presence of two-sided noise negatively affects the effectiveness of SW

approaches is negatively related to the presence of two-sided noise in the DGP and d) the double

bootstrap approach actually performs worse than the single bootstrap approach, when the

contextual variable significantly affects the output and when the actual DGP differs from the DGP

assumed in SW (2007).

27

Table 4: Mean estimates of , MADs and MEADs when null hypothesis is true ( 0 )

No. Inputs Sample size

Statistics Zero Noise (σv=0) Moderate Noise (σv=0.1)

DEA+ OLS

DEA+ Tobit

SW-Single

SW-Double

DEA+ OLS

DEA+ Tobit

SW-Single

SW-Double

One n = 50 Mean( ) 0.000 0.000 -0.020 0.018 0.001 0.000 -0.049 -0.005 MAD 0.031 0.030 0.221 0.175 0.042 0.035 0.163 0.164 MEAD 0.027 0.025 0.081 0.074 0.036 0.031 0.046 0.051

n = 200 Mean( ) 0.002 0.001 0.016 -0.007 0.001 0.000 0.008 -0.016 MAD 0.016 0.013 0.089 0.080 0.021 0.016 0.090 0.130 MEAD 0.014 0.011 0.039 0.039 0.018 0.013 0.035 0.045

n = 400 Mean( ) 0.001 0.001 0.097 -0.007 0.001 0.000 -0.002 0.001 MAD 0.010 0.009 0.211 0.058 0.014 0.010 0.160 0.174 MEAD 0.008 0.007 0.025 0.026 0.011 0.008 0.070 0.069

Two n = 50 Mean( ) 0.000 0.001 0.005 -0.005 0.001 0.001 0.002 -0.001 MAD 0.030 0.034 0.248 0.132 0.043 0.041 0.151 0.115 MEAD 0.025 0.028 0.081 0.063 0.037 0.035 0.059 0.055

n = 200 Mean( ) 0.000 0.000 -0.023 0.008 0.001 0.001 -0.010 -0.019 MAD 0.016 0.015 0.098 0.079 0.023 0.018 0.064 0.120 MEAD 0.014 0.013 0.038 0.035 0.020 0.015 0.031 0.046

n = 400 Mean( ) -0.001 -0.001 0.001 -0.005 0.000 0.000 -0.007 -0.011 MAD 0.011 0.010 0.060 0.069 0.015 0.011 0.142 0.182 MEAD 0.009 0.008 0.027 0.027 0.013 0.010 0.058 0.093

28

Table 5: Coverage rates of 1000 Monte Carlo experiments when null hypothesis is true ( 0 )

No. Inputs Sample Size

Sig. level

Zero Noise (σv=0) Strong Noise (σv=0.1)

DEA+OLS SW-Single SW-Double DEA+OLS SW-Single SW-Double

One

n = 50

99% 99.2% 97.9% 98.2% 99.5% 98.9% 98.2% 95% 95.0% 93.4% 91.3% 96.1% 92.9% 93.6% 90% 90.3% 86.9% 85.5% 91.5% 88.0% 87.8%

n = 200

99% 98.8% 98.9% 98.4% 98.7% 98.9% 99.1% 95% 94.6% 94.5% 93.4% 94.5% 95.3% 96.1% 90% 90.2% 89.5% 88.2% 88.7% 91.6% 93.4%

n = 400

99% 99.3% 99.8% 99.7% 99.3% 99.6% 99.4% 95% 95.6% 96.9% 96.3% 95.6% 97.9% 97.9% 90% 90.9% 91.4% 90.8% 91.2% 95.5% 96.3%

Two

n = 50

99% 99.1% 98.6% 96.8% 99.5% 98.2% 97.2% 95% 94.9% 93.7% 89.4% 95.7% 93.9% 89.9% 90% 90.2% 89.6% 83.6% 89.7% 88.8% 83.8%

n = 200

99% 98.8% 98.4% 98.7% 99.2% 98.5% 98.8% 95% 94.7% 94.5% 93.8% 95.0% 94.9% 95.4% 90% 90.0% 88.7% 87.4% 88.5% 89.8% 91.7%

n = 400

99% 99.2% 99.6% 99.3% 99.2% 99.7% 99.8% 95% 95.5% 95.6% 95.2% 95.7% 98.3% 98.5% 90% 92.0% 91.0% 90.0% 91.2% 95.4% 96.1%

29

30

Table 6: Mean estimates of , MADs and MEADs when null hypothesis is false ( 0.2 )

No. Inputs Sample size

Statistics Zero Noise (σv=0) Strong Noise (σv=0.1)

DEA+ OLS

DEA+ Tobit

SW-Single

SW-Double

DEA+ OLS

DEA+ Tobit

SW-Single

SW-Double

One n = 50 Mean( ) 0.185 0.164 0.225 0.254 0.185 0.150 0.213 0.244 MAD 0.037 0.042 0.072 0.073 0.046 0.055 0.077 0.075 MEAD 0.031 0.038 0.053 0.047 0.038 0.049 0.055 0.051

n = 200 Mean( ) 0.197 0.157 0.331 0.511 0.196 0.141 0.742 0.913 MAD 0.017 0.043 0.146 0.319 0.022 0.059 0.553 0.718 MEAD 0.014 0.043 0.042 0.076 0.019 0.060 0.578 0.790

n = 400 Mean( ) 0.198 0.153 1.190 1.303 0.197 0.136 1.755 1.886 MAD 0.011 0.047 0.992 1.103 0.015 0.064 1.559 1.688 MEAD 0.010 0.046 1.233 1.359 0.014 0.064 1.945 2.110

Two n = 50 Mean( ) 0.162 0.164 0.244 0.247 0.165 0.152 0.244 0.240 MAD 0.047 0.046 0.122 0.078 0.055 0.059 0.145 0.083 MEAD 0.041 0.039 0.067 0.048 0.047 0.052 0.071 0.056

n = 200 Mean( ) 0.186 0.158 0.220 0.408 0.187 0.144 0.420 0.786 MAD 0.021 0.042 0.039 0.220 0.025 0.056 0.241 0.592 MEAD 0.018 0.042 0.028 0.052 0.021 0.056 0.053 0.661

n = 400 Mean( ) 0.191 0.156 0.943 1.131 0.192 0.140 1.447 1.721 MAD 0.014 0.044 0.747 0.932 0.018 0.060 1.253 1.523 MEAD 0.012 0.044 0.919 1.141 0.015 0.060 1.585 1.910

31

Table 7: Coverage rates of 1000 Monte Carlo experiments when null hypothesis is false ( 0.2 )

No. Inputs

Sample Size

Sig. level

Zero Noise (σv=0) Strong Noise (σv=0.1)

DEA+OLS SW-Single SW-Double DEA+OLS SW-Single SW-Double One n = 50 99% 98.4% 99.0% 99.9% 98.4% 99.1% 100%

95% 93.6% 93.8% 97.8% 92.9% 94.0% 97.6% 90% 87.2% 88.2% 87.9% 88.1% 87.5% 90.2%

n = 200

99% 98.7% 91.1% 66.7% 98.8% 79.1% 67.0% 95% 94.5% 76.2% 53.4% 95.1% 49.2% 36.5% 90% 88.8% 70.8% 45.1% 90.2% 39.6% 28.2%

n = 400

99% 98.9% 26.2% 24.0% 98.7% 76.5% 76.1% 95% 95.4% 16.0% 10.7% 94.7% 23.7% 21.3% 90% 90.6% 10.8% 6.0% 89.4% 20.1% 17.6%

Two n = 50 99% 95.3% 99.7% 99.8% 97.2% 99.4% 99.9% 95% 86.7% 92.6% 96.4% 89.8% 92.9% 96.7% 90% 77.8% 87.8% 88.5% 82.2% 87.1% 88.5%

n = 200

99% 97.5% 98.9% 80.1% 98.6% 94.8% 69.2% 95% 90.1% 94.3% 65.7% 93.1% 77.0% 39.7% 90% 81.4% 86.8% 59.7% 87.9% 66.2% 32.2%

n = 400

99% 98.1% 29.7% 27.7% 99.0% 61.0% 68.3% 95% 90.5% 22.6% 12.9% 92.7% 25.5% 21.8% 90% 84.4% 16.5% 9.1% 86.1% 22.4% 18.0%

32

5. Conclusion

BN (2008) and SW (2007) propose two very different methods in the second stage to evaluate the

contextual variables affecting productivity. While BN (2008) propose that the popular two stage DEA +

OLS method can generate consistent estimates of impact of contextual variables under their DGP

assumptions, SW (2007) advocate a truncated regression model to estimate the impact of contextual

variables along with two bootstrap approaches to make inferences. Both of these papers show through

simulation evidence that their approaches work well when the actual DGP used in the simulation is also the

one used to develop the estimation model.

In this paper, we compare and contrast the performance of these two approaches to understand which

one is capable of correctly inferring the impact of contextual variables on productivity when the true DGP

is the traditional composed error stochastic frontier production model. We conduct 48 different experiments

each with 1000 Monte Carlo trials to compare the performance of the simple DEA+OLS method against

the computationally demanding bootstrap approaches of SW(2008). Simulation results indicate that, overall,

the simple DEA+OLS method and to a lesser extent the DEA+Tobit method dominate both of SW’s

approaches in terms of lower mean absolute deviation (MAD), lower median absolute deviation (MEAD)

as well as higher coverage rates when the contextual variables significantly impact productivity. The results

also suggest that the DEA+OLS method does as well, sometimes better than the two SW approaches when

the contextual variables do not impact productivity.

We hope that productivity researchers will take a careful look at these results and understand the

performance implications of the alternative approaches for productivity analysis. We conclude by

emphasizing that any model is by necessity a simple abstraction of reality, and thus the maintained

assumptions are rarely true in their entirety in practice. The key question is to what extent the assumptions

deviate from reality and reasonable expectations based on prior research. The models in both BN (2008)

and SW (2007) have certain set of maintained assumptions. Therefore, empirical researchers should make

a choice among the various two-stage models appropriate for their context.

33

Acknowledgments

Helpful comments and suggestions from many colleagues and seminar participants at various DEA and

Productivity conferences are gratefully acknowledged.

34

References

Afonso, A., M.S. Aubyn. 2006. Cross-country efficiency of secondary education provision: A semi-parametric analysis with non-discretionary inputs. Economic Modelling, 23, 476–491

Afriat, S. N. 1972. Efficiency Estimation of Production Functions. International Economic Review, 13, 568-598.

Aigner, D.J., S.F. Chu. 1968. On Estimating the Industry Production Function. The American Economic Review, 58(4), 826-839.

Aigner, D., C.A.K. Lovell, P. Schmidt. 1977. Formulation and Estimation of Stochastic Frontier Production Function Models. Journal of Econometrics, 6, 21-37.

Banker, R. D., A. Charnes, W.W. Cooper. 1984. Models for the Estimation of Technical and Scale Inefficiencies in Data Envelopment Analysis. Management Science, 30, 1078-1092.

Banker, R. D. 1993. Maximum Likelihood, Consistency and Data Envelopment Analysis: A Statistical Foundation. Management Science, 39, 1265-1273.

Banker, R. D., R. Natarajan. 2008. Evaluating Contextual Variables Affecting Productivity Using Data Envelopment Analysis. Operations Research, 56, 48-58.

Battese, G. E., T. J. Coelli. 1995. A Model for Technical Inefficiency Effects in a Stochastic Frontier Production Function for Panel Data. Empirical Economics, 20, 325-332.

Barros, C. P., M. Nektarios, A. Assaf. 2010. Efficiency in the Greek insurance industry. European Journal of Operational Research, 205, 431–436

Blank, J. T., V.G. Valdmanis. 2010. Environmental factors and productivity on Dutch hospitals: a semi-parametric approach. Health Care Management Science, 13, 27-34.

Daraio, C., L. Simar, P. Wilson, 2010. Testing whether Two-stage Estimation is Meaningful in Non-parametric Models of Production. Discussion paper #1031. Institut de Statistique, Universite Catholique de Louvain, Louvain-la-Neuve, Belgium.

Fenn, P., D. Vencappa, S. Diacon, P. Klumpes, C. O’Brien. 2008. Market structure and the efficiency of European insurance companies: A stochastic frontier analysis. Journal of Banking and Finance, 32, 86-100.

Fried, H.O., C.A.K. Lovell, S.S. Schmidt, S. Yaisawarng. 2002. Accounting for Environmental Effects and Statistical Noise in Data Envelopment Analysis. Journal of Productivity Analysis, 17, 157-174.

Forsund, F. R., 1999. The evolution of DEA -- The economics perspective. Working paper, University of Oslo, Oslo, Norway.

Gstach, D., 1998. Another Approach to Data Envelopment Analysis in Noisy Environments: DEA+. Journal of Productivity Analysis, 9, 161-176.

Grosskopf, S. 1996. Statistical Inference and Nonparametric Efficiency: A Selective Survey, Journal of Productivity Analysis, 7, 161-176.

Hoff, A. 2007. Second Stage DEA: Comparison of Approaches for Modeling the DEA Score. European Journal of Operational Research, 181, 425-435.

Izadi, H., G. Johnes, R. Oskrochi, R. Crouchley. 2002. Stochastic Frontier Estimation of a CES Cost Function: the Case of Higher Education in Britain. Economics of Education Review, 21, 63-71.

Johnson, A., T. Kuosmanen. 2012. One-stage and Two-stage DEA Estimation of the Effects of Contextual Variables. European Journal of Operational Research, 220, 559-570.

35

Latruffe, L., S. Davidova, K. Balcombe. 2008. Application of a double bootstrap to investigation of determinants of technical efficiency of farms in Central Europe. Journal of Productivity Analysis, 29, 183-191.

Li, T., R. Rosenman. 2001. Cost Inefficiency in Washington hospitals: a Stochastic Frontier Approach Using Panel Data. Health Care Management Science, 4, 73-81.

McDonald, J. 2009. Using Least Squares and Tobit in Second Stage DEA Efficiency Analyses. European Journal of Operational Research, 197, 792-798.

Meeusen, W., J. Broeck. 1977. Efficiency Estimation from Cobb-Douglas Production Functions with Composed Error. International Economic Review, 18(2), 435-444.

Marschak, J., W.J. Andrews. 1944. Random Simultaneous Equations and the Theory of Production. Econometrica, 12, 143-205.

Ruggiero, J., D.F. Vitaliano. 1999. Assessing the efficiency of public schools using data envelopment analysis and frontier regression. Contemporary Economic Policy, 17(3), 321–331. Rosko, M. D. 2001. Impact of HMO Penetration and Other Environmental Factors on Hospital X-Inefficiency. Medical Care Research and Review, 58, 430-454.

Richmond, J. 1974. Estimating the Efficiency of Production. International Economic Review, 15(2), 515-521.

Schmidt, P. 1985. Frontier Production Functions. Econometric Reviews, 4, 289-328.

Simar, L., P. Wilson. 2007. Estimation and Inference in Two-stage, Semiparametric Models of Production Processes. Journal of Econometrics, 136, 31-64.

Simar, L., P. Wilson. 2011. Two-stage DEA: Caveat Emptor. Journal of productivity analysis, 36, 205-218.

Timmer, C.P., 1971. Using a Probabilistic Frontier Production Function to Measure Technical Efficiency. Journal of Political Economy, 79, 776-794.

Wang, H., P. Schmidt. 2002. One-step and Two-step Estimation of the Effects of Exogenous Variables on Technical Efficiency Levels. Journal of Productivity Analysis, 18, 129-144.

Wilson, P. 2008. FEAR: A Software Package for Frontier Efficiency Analysis with R, Socio-Economic Planning Sciences, 42, 247-245.

Yu, C. 1998. The Effects of Exogenous Variables in Efficiency Measurement – A Monte Carlo Study. European Journal of Operational Research, 105, 569-580.

Zellner, A., J. Kmenta, J. Dreze, 1966. Specification and Estimation of Cobb-Douglas Production Functions. Econometrica, 34, 784-795.

36

37

Appendix 1

Examples of contextual and environmental factors cited in stochastic frontier estimation studies

Industry Exogenous variables or factors out of management control contributing to the stochastic “noise” term

Endogenous variables under management control affecting output indirectly through the one-sided inefficiency term

Hospitals Patient characteristics to capture the variation in outputs and complexity of care (1) Age (Hughes 1988; Bradford et al. 2001) (2) Gender (Hughes 1988; Bradford et al. 2001) (3) Patient health status prior to the treatment (Bradford et al. 2001) (4) Smoker (Bradford et al. 2001) (5) Unanticipated adverse events(Bradford et al. 2001) (6) Casemix (e.g., %Internal medicine, %General Surgery, %Intensive care, % Gynaecology; % Paediatrics; % mentally handicapped; %emergency visits; %long-term admissions; %admissions on weekend; %admissions from out-of-state; %patients transferred from other hospitals or a long term facility) (Hughes 1988; Wagstaff 1989; Zuckerman et al. 1994; Vitaliano and Toren 1996; Koop et al. 1997; Rosko 2001; Deily and McKay 2006; Carey et al. 2008) Hospital characteristics (7) Location (Vitaliano and Toren 1994, 1996; Crivelli et al. 2002) (8) Ownership type (i.e. government, for-profit, proprietorship and partnership): portmanteau variable, proxy for measurement errors in opportunity costs, care quality and production technology (Vitaliano and Toren 1994)

Factors that affect rent-seeking behavior and managerial incentives on profit maximization/cost minimization (1) Size (Vitaliano and Toren 1994, 1996; Carey et al. 2008) (2) Ownership type (Vitaliano and Toren 1996; Koop et al. 2007; Carey et al. 2008) (3) Unionization (Vitaliano and Toren 1994) (4)Managerial and Supervisory personnel to total labor employed: proxy for rent-seeking activity (Vitaliano and Toren 1994) (5) Outside Purchases/total costs (Vitaliano and Toren 1994); (6) Payment policies (e.g., %Medicare, %Medicaid, %Blue Cross): proxy for fiscal/regulatory pressures of public payers (Vitaliano and Toren 1996; Rosko 2001) (7) Industry concentration (Herfindahl index): proxy for Competitive pressure in hospital market and incentives to compete on the basis of price (Vitaliano and Toren 1996; Rosko 2001) (8) Malpractice payments: proxy for inept medical management (Vitaliano and Toren 1996) (9) Member of multihospital system (Carey et al. 2008)

38

(9) Teaching hospital: proxy for complex, innovative or rare medical procedures (Vitaliano and Toren 1996; Rosko 2001; Carey et al. 2008) (10) Scope of care/service (e.g., high technology services, numbers of services provided) (Zuckerman et al 1994; Vitaliano and Toren 1994, 1996; Crivelli et al. 2002) (11) Quality of care/service (e.g., average assistance time, no. of Deficiencies, patient safety indicators, mortality rates) (Vitaliano and Toren 1994; Crivelli et al. 2002; Carey et al. 2008) (12) Resident to bed ratio (Zuckerman et al. 1994; Carey et al. 2008) (13) Medical and nursing staff to resident ratio (Crivelli et al. 2002) (14) HMO penetration (i.e. %population enrolled in HMOs) (Rosko 2001)

Banks Macro socio-economic variables (1) Demographics (i.e., population density, no. of households) (Kaparakis et al 1994) (2) Financial sophistication of customers (i.e., Per capita income, Per capita deposits) (Kaparakis et al 1994) (3) Weighted average of nonperforming loans to total loans for the state: proxy for losses caused by negative economic shocks (Berger and Mester 1997) (4) Regional economic condition (e.g. %growth of taxable sales) (Chang et al. 1998) (5) Out-of-state entry permission (DeYoung et al. 1998) (6) % of deposits held by out-of-state banking institutions (DeYoung et al. 1998) (7) Unemployment rate (DeYoung et al. 1998) Bank-specific variables: (8) Length of banking experience (Mester 1996; Hasan and Marton 2003) (9) Location (e.g., headquarter, branch) (Mester 1993, 1996; Cebenoyan et al. 1993; Altunbas et al. 2000)

(1) Market competition/concentration (i.e., No. of saving institutions, % deposits/assets held by saving institutions) (Cebenoyan et al. 1993; Kaparakis et al 1994; Jeff et al. 1994; DeYoung et al. 1998) (2) Local market share (Cebenoyan et al. 1993; Jeff et al. 1994) (3) Ownership type (e.g., foreign-owned, US-owned) (Chang et al. 1998; Hasan and Marton 2003) (4) Organizational and regulatory structure (i.e., state or federally chartered, a member of holding company, a member of the Federal Reserve System, mutual or stock S&Ls) (Mester 1993, 1996; Jeff et al. 1994; Cebenoyan et al. 1993; Chang et al. 1998; DeYoung et al. 1998) (5) Bank size (Kaparakis et al 1994; Mester 1993, 1996; Cebenoyan et al. 1993; Jeff et al. 1994; Chang et al. 1998; Altunbas et al. 2000; Hasan and Marton 2003) (6) Bank performance (e.g., net income/total assets, nonperforming loans/total assets, repossessed

39

(10) Bank branching network/status (e.g., no. of domestic/foreign branches, statewide and unit branching) (Kaparakis et al 1994; Jeff et al. 1994) (11) Quality of banks’ output (i.e., average volume of nonperforming loans, average ratio of non-performing loans to total loans in bank’s state): non-performing loans are exogenous if they are caused by negative shocks, but endogenous if caused by bad management. The state average is exogenous to individual banks (Mester 1996; Altunbas et al. 2000; Berger and Mester 1997)

assets/total assets) (Mester 1993, 1996; Altunbas et al. 2000; Jeff et al. 1994) (7) Bank risk or safety characteristics (Nonaccrual loans/ Total loans, Uninsured deposits/total deposits; liquid assets to total assets ratio) (Kaparakis et al 1994; Altunbas et al. 2000; Hasan and Marton 2003) (8) Portfolio/business mix (e.g., %construction and land development loans, %real estate loans, %individual loans) (Mester 1996; Hasan and Marton 2003) (9) Business mix (e.g., net loans to assets ratio, off-balance sheet items to assets ratio, customer and short-term funds to total funds ratio) (Altunbas et al. 2000) (10) Capital adequacy (i.e. total qualifying capital/total assets, equity to asset ratio) (Kaparakis et al 1994; Mester 1996; Altunbas et al. 2000; Hasan and Marton 2003) (11) Capital-labor ratio: proxy for managerial discretion (Kaparakis et al 1994) (12) Purchased funds to total deposits: proxy for aggressive management behavior (Kaparakis et al. 1994) (13) Reliance on non-core deposits (e.g., brokered deposits/total deposits, uninsured deposits/total deposits) (Mester 1993, 1996) (14) Failed bank assets to total assets: to control for the effect of out-of-state competition (DeYoung et al. 1998)

Insurance (1) Location (e.g. country or city dummies) (Rai 1996; Hardwick 1997; Eling and Luhnen 2010) (2) Line of business dummies (e.g., fire, health, motor, transport) (Bikker and Gorter 2011) (3) Legal systems (e.g., common law, continental European law, mixed) (Eling and Luhnen 2010) (4) New York Regulation (Gardner and Grace 1993)

(1) Market share (Gardner and Grace 1993) (2) Market concentration (Herfindahl and Hirschman index) (Kasman and Turgutlu 2009) (3) Ownership type (e.g., foreign, domestic) (Kasman and Turgutlu 2009; Choi and Elyasiani 2011) (4) Organizational form (e.g., Mutual or stock) (Gardner and Grace 1993; Hardwick 1997; Greene and Segal 2004; Bikker and Gorter 2011; Eling and Luhnen 2010; Choi and Elyasiani 2011) (5) Group affiliation (membership in an insurance group)(Bikker and Gorter 2011; Choi and Elyasiani 2011)

40

(6) Distribution systems (Gardner and Grace 1993; Choi and Elyasiani 2011) (7) Specialization (e.g., life, non-life, line of business concentration) (Rai 1996; Bikker and Gorter 2011; Choi and Elyasiani 2011) (8) Rent seeking activities (e.g., bureau fees, litigation expenses, advertising expenses) (Gardner and Grace 1993) (9) Size (Rai 1996; Kasman and Turgutlu 2009; Eling and Luhnen 2010; Choi and Elyasiani 2011) (10) Complexity (% of complex lines) (Choi and Elyasiani 2011) (11) Reinsurance utilization (Choi and Elyasiani 2011) (12) Leverage (the ratio of net premiums written to policyholders’ surplus) (Choi and Elyasiani 2011) (13) Solvency/Capital structure (ratio of equity to total assets) (Gardner and Grace 1993; Rai 1996; Kasman and Turgutlu 2009; Eling and Luhnen 2010) (14) Advertising intensity (Choi and Elyasiani 2011) (15) Performance (i.e., total costs/net income, net income/total assets) (Rai 1996; Kasman and Turgutlu 2009) (16) Product composition (i.e., life premiums/total assets, nonlife premiums/total assets) (Rai 1996)

Farms/ Agriculture

Farm specific characteristics (1) Size (Ali and Flinn 1989; Kumbhakar et al 1989; Bravo-Ureta and Rieger 1991; Parikh and Shah, 1994) (2) Soil (Ali and Flinn 1989) (3) Location: proxy for effects of soil, terrain, and weather (Kalirajan 1982; Bravo-Ureta and Rieger 1991; Kumbhakar 1994) (4) Degree of land fragmentation (Parikh and Shah 1994) (5) Water constraints (Ali and Flinn 1989) Farmer-specific characteristics

(1) Ownership (Kalirajan 1984; Ali and Flinn 1989) (2) Extension service/assistance (Kalirajan 1984; Parikh and Shah 1994) (3) Credit availability (Ali and Flinn 1989; Ekayanakea 1987; Parikh and Shah 1994) (4) Weed/pest damage (Ekayanakea 1987) (5) Cropping practice (e.g. late crop establishment, late fertilizer application, early established long or short aged varieties) (Ekayanakea 1987; Ali and Flinn, 1989) (6) Use of technology (e.g., tractor use for rice cultivation, manual weeding, modern milking equipment, use of artificial insemination, use of feed concentrates) (Ali and

41

(6) farmers’ age and years of farming (Kalirajan, 1984; Bravo-Ureta and Rieger 1991; Ekayanakea, 1987; Parikh and Shah, 1994) (7) farmers’ education (Kalirajan, 1984; Ekayanakea, 1987; Kumbhakar et al 1989; Bravo-Ureta and Rieger 1991; Parikh and Shah, 1994) (8) Off-farm employment (Ali and Flinn 1989; Ekayanakea 1987; Kumbhakar et al 1989; Parikh and Shah, 1994)

Flinn, 1989; Bravo-Ureta and Rieger 1991; Ekayanakea 1987; Bailey et al. 1989)

Education Characteristics and family backgrounds of students: (1) Students’ innate ability (e.g., % of pupils obtaining 5 or more graded passes at ‘O’ level, % of students with incomplete grades, % of handicapped) (Barrow 1991; Duncombe et al. 1995; Heshmati and Kumbhakar 1997; Cooper and Cohn 1997) (2) Demographic make-up of classrooms (e.g., % of female students; % of black students, % of nonwhite student) (Cooper and Cohn 1997; Duncombe et al. 1995) (3) % of pupils with additional educational needs (e.g., % of student from one parent families) (Barrow 1991) (4) % of migrant students or % of limited English proficiency (Duncombe et al. 1995; Heshmati and Kumbhakar 1997) (5) % of handic (6) Poverty (e.g., % of pupils receiving free school, % of pupils from low socio-economic group households, parents’ income, net assessed real property value per student, official poverty rate) (Barrow 1991; Deller and Rudnicki 1993; Duncombe et al. 1995; Cooper and Cohn 1997; Heshmati and Kumbhakar 1997; Chakraborty et al. 2001; Ruggiero and Vitaliano 1999) (7) Parents’ education (Deller and Rudnicki 1993; Cooper and Cohn 1997; Chakraborty et al. 2001; Heshmati and Kumbhakar 1997) (8) Parents’ occupation (Deller and Rudnicki 1993; Cooper and Cohn 1997) (9) No. of books at home (Cooper and Cohn 1997) Community/Municipality characteristics

Institutional factors: (1) Ownership type (e.g., private or public school) (Cooper and Cohn 1997) (2) Merit-based compensation/teacher incentive plans (Cooper and Cohn 1997) Teaching and administrative variables: (3) Per pupil expenditure in teaching, administration, operations and busing (Deller and Rudnicki 1993; Duncombe et al. 1995) (4) Class size (e.g., No. of students in class) (Cooper and Cohn 1997; Heshmati and Kumbhakar 1997) (5) Space per student (Heshmati and Kumbhakar 1997) (6) Teacher-student ratio (Cooper and Cohn 1997; Heshmati and Kumbhakar 1997; Chakraborty et al. 2001) (7) Teacher’s education (Cooper and Cohn 1997; Chakraborty et al. 2001) (8) Teachers’ experience (Chakraborty et al. 2001) (9) Teachers’ absence hours (Heshmati and Kumbhakar 1997) (10) % of trained teachers (Heshmati and Kumbhakar 1997) (11) Library size (Cooper and Cohn 1997) (12) No. of computers (Cooper and Cohn 1997) (13) Audio-visual equipment (Cooper and Cohn 1997) (14) No. and quality of laboratories (Cooper and Cohn 1997)

42

(10) % of Single-parent female-headed households (Ruggiero and Vitaliano 1999) (11) % of households with school-age children (Ruggiero and Vitaliano 1999) (12) % of young population (Heshmati and Kumbhakar 1997) (13) Community income (or tax) per capita (Deller and Rudnicki 1993; Heshmati and Kumbhakar 1997) (14) Pupil density (pupils per square mile) (Duncombe et al. 1995) (15) Availability of jobs with the community or surrounding area (e.g., unemployment rate) (Deller and Rudnicki 1993) (16) fiscal capacity (e.g., weighted average of property wealth and adjusted gross income, per pupil non-operating state aid) (Duncombe et al. 1995) School and teacher characteristics: (17) School size (e.g., student enrollment, average daily attendance)(Deller and Rudnicki 1993; Duncombe et al. 1995; Heshmati and Kumbhakar 1997) (18) School district type (e.g., metropolitan, suburban, rural areas) (Barrow 1991; Ruggiero and Vitaliano 1999) (19) Medical school (e.g., whether or not a institution has a medical school) (Agasisti and Johnes 2010) (20) Carnegie classification (e.g., research, doctoral, masters, bachelors institutions) (Robst 2001) (21) Teacher’s sex (Cooper and Cohn 1997) (22) Teacher’s race (Cooper and Cohn 1997)

(15) building layout (Cooper and Cohn 1997) (16) Research expenditures (Robst 2001) (17) State + tuition revenue (Robst 2001) (18) State share of revenue (Robst 2001)

Utility (1) Population/customer density (Button and Weyman-Jones 1993; Bhattacharyya et al. 1995; Burns and Weyman-Jones 1996; Fabbri et al. 2000; Farsi et al. 2007; Farsi and Filippini 2009) (2) Maximum demand (peak load) on the system (Button and Weyman-Jones 1993; Burns and Weyman-Jones 1996; Rossi 2001; Farsi et al. 2007) (3) Market structure (i.e., %industrial, %commercial, %residential kWh sold )

(1) Ownership type (e.g., privately, co-operative, municipally owned)(Cote 1989; Button and Weyman-Jones 1993; Bhattacharyya et al. 1995; Rossi and Ruzzier 2000; Fabbri et al. 2000) (2) System loss of water (Bhattacharyya et al. 1995) (3) % of metered connection (Bhattacharyya et al. 1995) (4) Water treatment before delivery (Bhattacharyya et al. 1995) (5) Regulatory incentive programs (Knittel 2002)

43

(Button and Weyman-Jones 1993; Burns and Weyman-Jones 1996; Rossi 2001) (4) Size of concession area (Burns and Weyman-Jones 1996; Rossi 2001; Farsi et al. 2007) (5) Geographical characteristics (e.g., location, terrain, average altitude, slope, vegetation) (Rossi and Ruzzier 2000; Fabbri et al. 2000). (6) Urbanization (Fabbri et al. 2000) (7) Sources of water (i.e., surface water, groundwater) (Bhattacharyya et al 1995) (8) Scope of service (e.g. water and sewer services) (Bhattacharyya et al 1995) (9) Quality of service (Rossi and Ruzzier 2000) (10) Plant vintage (Knittel 2002)

44

Appendix 2

Examples of contextual and environmental factors cited in SW-based studies after 2007. Variables in italics refer to those out of management control as stochastic frontier type of studies do in Appendix 1

Type of Environment

Contextual variables

Hospitals (1) Market competition (e.g. Herfindahl-Hirschman index, concentration) (Puenpatom and Rosenman 2008; Nedelea and Fannin 2013a, 2003b; Fragkiadakis et al. 2014) (2) GDP per capita or median family income (Puenpatom and Rosenman 2008; Nedelea and Fannin 2013a, 2013b) (3) Location/Regional dummies (Bernet et al 2008; Puenpatom and Rosenman 2008; Tsekouras et al 2010; Kounetas and Papathanassopoulos 2013; Falavigna et al. 2013) (4) University/Teaching hospital (Tsekouras et al 2010; Kounetas and Papathanassopoulos 2013) (5) Market/service area (Bernet et al 2008) (6) Patient characteristics/demographics (Bernet et al 2008; Puenpatom and Rosenman 2008; Tsekouras et al 2010; Kounetas and Papathanassopoulos 2013; Fragkiadakis et al. 2014) (7) Casemix (Nedelea and Fannin 2013) (8) % Medicare/Medicaid admissions (Nedela and Fannin 2013a, 2013b) (8) Length of stay (Puenpatom and Rosenman 2008; Mitropoulos et al. 2013; Fragkiadakis et al. 2014) (9) Bed occupancy rate (Puenpatom and Rosenman 2008; Mitropoulos et al. 2013; Kounetas and Papathanassopoulos 2013; Fragkiadakis et al. 2014) (10) Doctor-to-nurse ratio (Tsekouras et al 2010) (11) Administrative staff ratio(Fragkiadakis et al 2014) (11) Physician intensity (No. physician per admission) (Blank and Valdmanis 2010) (12) Manager’s characteristics (e.g., years of experience, education) (Assaf and Matawie 2010) (13) Hospital governance characteristics (e.g., management board, supervisory board) (Blank and Hulst 2010) (14) Ownership type (e.g., Government, for-profit hospital) (Nedelea and Fannin 2013a, 2013b; Araujo et al. 2014) (15) Hospital Size (Mitropoulos et al. 2013; Araujo et al. 2014) (16) Specialization (Falavigna et al 2013; Fragkiadakis et al. 2014; Araujo et al. 2014)

Banks Macroeconomic variables: (1) Average wage in the banking sector (labor factor price in SFA studies) (Kenjegalieva et al. 2009) (2) GDP per capita (Kenjegalieva et al. 2009; Chronopoulos et al. 2011; Chortareas et al. 2012; Curi et al. 2013; Chronopoulos et al. 2011) (3) GDP growth (Chronopoulos et al. 2011; Chortareas et al. 2012; Zhang and Matthews 2012; Hou et al. 2014; Chronopoulos et al. 2011) (4) Corruption perception index or corruption control (Kenjegalieva et al. 2009; Chortareas et al. 2012, 2013)

45

(5) Unemployment rate (Kenjegalieva et al. 2009) (6) Inflation (Kenjegalieva et al. 2009; Chronopoulos et al. 2011) (7) Loan/Deposit market concentration (e.g., Herfindahl index) (Fukuyama and Matousek 2011; Assaf et al. 2011b; Chortareas et al. 2012; Zhang and Matthews 2012; Hou et al. 2014; Chronopoulos et al. 2011) (8) Location (Chronopoulos et al. 2011; Glass et al. 2010) (9) Ownership (e.g., foreign, domestic, government) (Kenjegalieva et al. 2009; Assaf et al. 2011a; Chortareas et al. 2012; Zhang and Matthew 2012; Hou et al. 2014; Wanke et al. 2015; Curi et al. 2013; Chronopoulos et al. 2011) (10) Liquidity risk (e.g., loan-to-deposit ratio, cash and current account deposits to assets) (Assaf et al. 2011a; Chortareas et al. 2012; Hou et al. 2014; Chronopoulos et al. 2011) (11) Profitability (e.g., net profit margin, return on asset/equity) (Assaf et al. 2011a, 2011b; Fukuyama and Matousek 2011; Chortareas et al. 2013; Hou et al. 2014; Chronopoulos et al. 2011) (12) Net interest margin (Assaf et al. 2011b; Fukuyama and Matousek 2011) (13) Payout ratio (Assaf et al. 2011a; Glass et al. 2010) (14) Credit risk (e.g., loan-to-asset ratio) (Chortareas et al. 2013; Glass et al. 2010; Chronopoulos et al. 2011) (15) Capitalization (e.g., equity-to-asset ratio, capital-to-equity ratio) (Fukuyama and Matousek 2011; Chortareas et al. 2012, 2013; Glass et al. 2010; Hou et al. 2014; Curi et al. 2013; Chronopoulos et al. 2011) (16) Write-off ratio or credit ratio of loan loss provisions to total loans (Glass et al. 2010; Hou et al. 2014) (17) Size (Assaf et al 2011a; Chortareas et al. 2012; Zhang and Matthews 2012; Hou et al. 2014; Wanke et al. 2015; Curi et al. 2013; Chronopoulos et al. 2011) (18) Specialization or diversification (Chortareas et al. 2012; Zhang and Matthews 2012; Curi et al. 2013; Chronopoulos et al. 2011) (19) Capital requirements (Chortareas et al. 2012; Curi et al. 2013) (20) Official supervisory power (Chortareas et al. 2012; Curi et al. 2013) (21) Freedom (e.g., banking/financial, business, economic freedom) (Chronopoulos et al. 2011; Chortareas et al. 2013) (22) Governmental spending (Chortareas et al. 2013) (23) Political stability (Chortareas et al. 2013) (24) Protection of property rights (Chortareas et al. 2013) (25) Rule of law (Chortareas et al. 2013) (26) Business cycle (Hodrick-Prescott filter) (Curi et al. 2013)

Insurance (1) Specialization (Life, health insurance) (Barros et al. 2010; Biener and Eling 2011) (2) Size (Barros et al. 2010; Biener and Eling 2011) (3) Mergers and acquisitions (Barros et al. 2010) (4) Foreign ownership (Barros et al. 2010) (5) Quoted in the stock market (Barros et al. 2010) (6) GDP per Capita (Huang and Eling 2013) (7) GDP growth (Huang and Eling 2013)

46

(8) Consumer price index (Huang and Eling 2013) (9) Market demand (e.g., vehicles per 1000 persons) (Huang and Eling 2013) (10) Market concentration (Huang and Eling 2013) (11) deposit interest rate (Huang and Eling 2013) (12) Policy dummy (i.e., individual or group policies) (Biener and Eling 2011) (13) Financial freedom (Huang and Eling 2013) (14) Capital requirement (e.g., equity to assets ratio) (Huang and Eling 2013) (15) Liquidity requirements (e.g., liabilities to liquid assets ratio) (Huang and Eling 2013)

Farms/ Agriculture

(1) Location (Latruffe et al 2008) (2) Soil, temperature, precipitation, biodiversity (Skevas et al. 2012) (3) Farm size (Latruffe et al 2008; Skevas et al. 2012; Buckley and Carney 2013; Balcombe et al. 2008; Picazo-Tadeo et al. 2011; Watto and Mugera 2014; Ullah and Perret 2014) (4) Farm fragmentation (Buckley and Carney 2013) (5) Technology (e.g. ratio of capital to labor) (Latruffe et al 2008) (6) Ownership (e.g. % of rented land) (Latruffe et al 2008; Balcombe et al. 2008; Olsen and Vu 2009; Watto and Mugera 2014) (7) Employment (e.g., % of hired labor, off-farm employment, income from agriculture) (Latruffe et al 2008; Buckley and Carney 2013; Balcombe et al. 2008; Picazo-Tadeo et al. 2011; Olsen and Vu 2009; Watto and Mugera 2014) (8) Rotation (Skevas et al. 2012) (9) Farmer’s age or years of farming (Skevas et al. 2012; Buckley and Carney 2013; Balcombe et al. 2008; Picazo-Tadeo et al. 2011; Olsen and Vu 2009; Wouterse 2010; Watto and Mugera 2014; Ullah and Perret 2014) (10) Farmer’s education (Skevas et al. 2012; Buckley and Carney 2013; Balcombe et al. 2008; Picazo-Tadeo et al. 2011; Wouterse 2010; Watto and Mugera 2014; Ullah and Perret 2014) (11) Crop Subsidies (Skevas et al. 2012) (12) Technology adoption (i.e., milk recordings) (Buckley and Carney 2013) (13) Sowing method (e.g., raised-bed, flatbed) (Ullah and Perret 2014) (14) Financial stress (e.g. interest plus rentals to total output, debt to asset ratio) (Latruffe et al 2008; Olsen and Vu 2009) (15) Credit access (Balcombe et al. 2008; Watto and Mugera 2014) (16) Extension Services/ Agricultural training (Balcombe et al. 2008; Watto and Mugera 2014; Picazo-Tadeo et al. 2011; Ullah and Perret 2014)

Education (1) Location (Naper 2010; Lee 2011; Alexander et al 2010; Agasisti et al 2014; Santin and Sicilia 2015) (2) Size (Naper 2010; Alexander et al. 2010) (3) Parent education (e.g., % of population with higher education or at least upper secondary education) (Naper 2010; Coco and Lagravinese 2014) (4) Immigrant status (e.g., % of student from immigrant background) (Coco and Lagravinese 2014)

47

(5) Unemployment (Coco and Lagravinese 2014) (6) % of student with special needs (Naper 2010) (7) Real GDP per capita or Local revenue per capita (Naper 2010; Wolszczak-Derlacz and Parteka 2011) (8) Relative earnings of the population with income from employment (Coco and Lagravinese 2014) (9) % of population living in rural areas (Naper 2010) (10) Socialist share (Naper 2010) (11) Cronyism (Coco and Lagravinese 2014) (12) Herfindahl index of (inverse) party fragmentation (Naper 2010) (13) Proportion of associate professors and professors to total academic staff (Lee 2011) (14) % of women academic staff (Wolszczak-Derlacz and Parteka 2011) (15) Ownership (state or private) (Alexander et al. 2010; Foltz et al. 2012) (16) Socio-economic environment status of the community (Alexander et al. 2010) (17) Households’ living situation (i.e., location, use of electricity, construction materials of housing) (Miningou and Vierstraete 2013) (18) Type of school (e.g., composite, secondary) (Alexander et al. 2010) (19) Boys’ or Girls school (Alexander et al. 2010) (20) Teachers’ qualification (Alexander et al. 2010) (21) Teaching pedagogy (e.g., assessment through test, quizzes and exams more than once a month, homework assignment, extra reading ) (Santin and Sicilia 2015) (22) Vocational technical school (Santin and Sicilia 2015) (23) Teacher to student ratio (Santin and Sicilia 2015) (24) Land and non-land grants (Foltz et al. 2012) (25) Medical school or university related hospital (whether a university has medical and pharmacy faculty) (Wolszczak-Derlacz and Parteka 2011; Curi et al. 2012; Foltz et al. 2012) (26) Years of foundation (Wolszczak-Derlacz and Parteka 2011) (27) Post-docs to faculty ratio (Foltz et al. 2012) (28) Total amount of research funding (Foltz et al. 2012) (29) Sources of funds (e.g., % of federal or industry funds) (Foltz et al. 2012; Wolszczak-Derlacz and Parteka 2011 ) (30) Responsibility of national authorities (e.g., curriculum, student disciplinary policies, budget distribution, student assessment policies) (Santin and Sicilia 2015)

Utility (1) Regional mean salary or GDP per capita (Simoes et al. 2010; Witte and Marques 2010; Mbuvi et al. 2012; Leme et al. 2014; See 2015) (2) Precipitation (Prado-Lorenzo and García-Sánchez 2010; Leme et al 2014)

48

(3) Average maximum temperature (See 2015) (4) Terrain (Witte and Marques 2010) (4) Complexity in combating non-technical losses (Leme et al 2014) (5) Population or Consumer density (Lorenzo and Sanchez 2007; Prado-Lorenzo and García-Sánchez 2010; Simoes et al. 2010; Mbuvi et al. 2012; Leme et al 2014; Ananda 2014; See 2015) (6) Production density (i.e., water or electric power consumption per capita) (Witte and Marques 2010; Ananda 2014) (7) Sources of water (e.g., % of surface water, % of underground water) (Witte and Marques 2010; Ananda 2014; See 2015) (8) Market structure (i.e. % of water/electricity delivered to industrial customers, % of residential consumption) (Witte and Marques 2010; Prado-Lorenzo and García-Sánchez 2010; Ananda 2014; ) (9) Specialization (i.e. water-only) (Witte and Marques 2010) (10) Service area (Lorenzo and Sanchez 2007; Simoes et al. 2010; Witte and Marques 2010; Prado-Lorenzo and García-Sánchez 2010; Simoes et al. 2010) (11) Independent regulation (Witte and Marques 2010; Mbuvi et al. 2012) (12) Benchmarking dummy (Witte and Marques 2010) (13) Performance contract use (Mbuvi et al. 2012) (13) Corporatization dummy (Witte and Marques 2010) (14) Infrastructure leakage index (Ananda 2014) (15) Ownership (private or state ownership) (Zhao and Ma 2013; Ma and Zhao 2015; See 2015; Prado-Lorenzo and García-Sánchez 2010; Simoes et al. 2010) (16) Day hours (i.e., hours of darkness) (Lorenzo and Sanchez 2007) (17) Vandalism (Lorenzo and Sanchez 2007) (8) Water service quality (i.e., non-revenue water) (See 2015) (10) Plant capacity/size (Ma and Zhao 2015) (11) Regulatory reform dummy (Zhao and Ma 2013; Ma and Zhao 2015) (12) Age of plant (Barros and Peypoch 2008; Zhao and Ma 2013; Ma and Zhao 2015) (13) market competition (i.e. Herfindhal index) (Barros and Peypoch 2008) (14) Air or Water pollution (Barros and Peypoch 2008; ) (15) Types of plants (i.e., coal, gas, oil, mixed) (Barros and Peypoch 2008; Ma and Zhao 2015) (16) Foreign direct investment (Zhao and Ma 2013; Ma and Zhao 2015)

49

References

Agasisti, T., Bonomi, F., Sibiano, P., 2014. Measuring the “managerial” efficiency of public schools: a case study in Italy. International Journal of Educational Management, 28(2), 120-140.

Alexander, W.R.J., Haug, A. A., Jaforullah, M., 2010. A two-stage double-bootstrap data envelopment analysis of efficiency differences of New Zealand secondary schools. Journal of Productivity Analysis, 34, 99-110.

Ali, M., Flinn, J. C., 1989. Profit Efficiency among Basmati Rice Producers in Pakistan Punjab. American Journal of Agricultural Economics, 71(2), 303-310.

Altunbas, Y., Liu, M., Molyneux, P., Seth, R., 2000. Efficiency and risk in Japanese banking. Journal of Banking and Finance, 24(10), 1605–1628.

Ananda, J. 2014. Evaluating the Performance of Urban Water Utilities: Robust Nonparametric Approach. Journal of Water Resources Planning and Management, 140, 0401 -4021.

Assaf, A., Matawie, K. M.,2010. Improving the accuracy of DEA efficiency analysis: a bootstrap application to the health care foodservice industry. Applied Economics, 42, 3547-3558.

Assaf, A. G., Barros, C.P., Matousek, R., 2011. Technical efficiency in Saudi banks. Expert Systems with Applications, 38, 5781–5786.

Assaf, A.G., Barros, C. P., Matousek, R., 2011. Productivity and efficiency analysis of Shinkin banks: Evidence from bootstrap and Bayesian approaches. Journal of Banking and Finance, 35, 331–342.

Araújo, C., Barros, C.P., Wanke, P., 2014. Efficiency determinants and capacity issues in Brazilian for-profit hospitals. Health Care Management Science, 17, 126-138.

Bagi, F. S., 1984. Stochastic Frontier Production Function and Farm-Level Technical Efficiency of Full-Time and Part-Time Farms in West Tennessee. North Central Journal of Agricultural Economics, 6, 48-55. Bailey, D.V., Biswas, B., Kumbhakar, S.C., Schulthies, B.K., 1989. An Analysis of Technical, Allocative, and Scale Inefficiency: The Case of Ecuadorian Dairy Farms. Western Journal of Agricultural Economics, 14(1), 30-37.

Balcombe, K., Fraserb, I., Latruffe, L., Rahmand, M., Smith, L., 2008. An application of the DEA double bootstrap to examine sources of efficiency in Bangladesh rice farming. Applied Economics, 40, 1919-1925.

Barros, C.P., Peypoch, N., 2008. Technical efficiency of thermoelectric power plants. Energy Economics, 30, 3118–3127.

Barros, C. P., Nektarios, M., Assaf, A., 2010. Efficiency in the Greek insurance industry. European Journal of Operational Research, 205, 431–436.

Barrow, M.M., 1991. Measuring local education authority performance: A frontier approach. Economics of Education Review, 10, 19–27.

50

Bernet, P. M., Michael D. Rosko, Valdmanis, V. G., Pilyavsky, A., Aaronson, W. E. , 2008. Productivity efficiencies in Ukrainian polyclinics: lessons for health system transitions from differential responses to market changes. Journal of Productivity Analysis, 29, 103-111.

Berger, A.N., Mester, L.J., 1997. Inside the black box: What explains differences in the efficiencies of financial institutions? Journal of Banking and Finance, 21, 895-947.

Bhattacharyya, A., Harris, T. R., Narayanan, R., Raffiee, K.,1995. Technical Efficiency of Rural Water Utilities. Journal of Agricultural and Resource Economics, 20(2), 373-391.

Biener, C., Eling, M., 2011. The Performance of Microinsurance Programs: A Data Envelopment Analysis. Journal of Risk and Insurance, 78, 83–115.

Bikker, J., Gorter, J., 2011. Restructuring of the Dutch nonlife insurance industry: consolidation, organizational form, and focus. Journal of Risk and Insurance, 78(1), 163-184.

Blank, J.L.T., Hulst, B., 2010. Governance and Performance: The Performance of Dutch Hospitals Explained by Governance Characteristics. Journal of Medical Systems, 35, 991-999.

Blank, J.L.T., Valdmanis, V.G., 2010. Environmental factors and productivity on Dutch hospitals: a semi-parametric approach. Health Care Management Science, 13, 27-34.

Bonaccorsi, A., Daraio, C., Simar, L., 2006. Advanced indicators of productivity of universities. An application of robust nonparametric methods to Italian data. Scientometrics, 66(2), 389–410.

Buckley, C., Carney, P., 2013. The potential to reduce the risk of diffuse pollution from agriculture while improving economic performance at farm level. Environmental Science & Policy, 25, 118–126.

Burns, P., Weyman-Jones, T., 1996. Cost drivers and cost efficiency in electricity distribution: a stochastic frontier approach. Bulletin of Economic Research, 48 (1), 0307-3378. Button, K., Weyman-Jones, T., 1993. X-inefficiency and regulatory regime shift in the UK. Journal of Evolutionary Economics, 3(4): 269-284.

Bradford, D., Kleit, A., Krousel-Wood, M., Re, R., 2001. Stochastic frontier estimation of cost models within the hospital. Review of Economics and Statistics, 82(2): 302-309.

Bravo-Ureta, B.E., Rieger, L., 1991. Dairy Farm Efficiency Measurement Using Stochastic Frontiers and Neoclassical Duality. American Journal of Agricultural Economics, 73, 421-428.

Cambini, C., Croce, A., Fumagalli, E.,2014. Output-based incentive regulation in electricity distribution: Evidence from Italy. Energy Economics, 45, 205–216.

Carey, K., Burgess, J.F., Young, G.J., 2008. Specialty and full-service hospitals: a comparative cost analysis. Health Services Research, 43(5), 1869-1887.

Cebenoyan, A.S., Cooperman, E., Register, C.A., 1993. The relative efficiency of stock versus Mutual S&L. Journal of Financial Services Research, 7(2), 151-170.

51

Chakraborty, K., Biswas, B., Lewis, W., 2001. Measurement of Technical Efficiency in Public Education: A stochastic and non-stochastic production function approach. Southern Economic Journal, 67, 889-905.

Chang, C. E., Hasan, I., Hunter, W.C., 1998. Efficiency of multinational banks: an empirical investigation. Applied Financial Economics, 8(6), 689-696

Choi, B., E. Elyasiani. 2011. Foreign-owned insurer performance in the US property-liability markets. Applied Economics, 43, 291-306.

Chortareas, G.E., Girardone, C., Ventouric, A., 2012. Bank supervision, regulation, and efficiency: Evidence from the European Union. Journal of Financial Stability, 8, 292–302.

Chortareasa, G. E., Girardoneb, C., Ventouric, A., 2013. Financial freedom and bank efficiency: Evidence from the European Union. Journal of Banking and Finance, 37, 1223–1231.

Chronopoulos, D.K., Girardone, C., Nankervis, J.C., 2011. Are there any cost and profit efficiency gains in financial conglomeration? Evidence from the accession countries. European Journal of Finance, 17, 603-621.

Cozad, M., Wichmann, B., 2013. Efficiency of health care delivery systems: effects of health insurance coverage. Applied Economics, 45, 4082-4094.

Crivelli, L., Filippini, M., Lunati, D., 2002. Regulation, ownership and efficiency in the Swiss nursing home industry. International Journal of Health Care Finance and Economics, 2, 79-97.

Curi, C., Daraio, C., Llerena, P., 2012. University technology transfer: how (in)efficient are French universities? Cambridge Journal of Economics, 1 of 26.

Curi, C., Guarda, P., Lozano-Vivas, A., Zelenyuk, V., 2013. Is foreign-bank efficiency in financial centers driven by home or host country characteristics? Journal of Productivity Analysis, 40:367–385.

Coco, G., Lagravinese, R., 2014. Cronyism and education performance. Economic Modelling, 38, 443-450.

Cooper, S.T., Cohn, E., 1997. Estimation of a frontier production function for the South Carolina educational process. Economics of Education Review, 16, 313-327.

Daniel O. COTE, 1989. Firm efficiency and ownership structure: the case of U.S. Electric Utilities Using Panel Data. Annals of Public and Cooperative Economics, 60, 431–450.

Deller, S. C., Rudnicki, E., 1993. Production efficiency in elementary education: The case of Maine public schools. Economics of Education Review, 12, 45-57.

DeYoung, R., Hasan, I., Kirchhoff, B., 1998. The Impact of Out-of-State Entry on the Cost Efficiency of Local Commercial Banks. Journal of Economics and Business, 50(2), 191–203. Duncombe, W., Miner, J., Ruggiero, J., 1995. Potential cost savings from school district consolidation: a case study of New York. Economies of Education Review, 14(3), 265-284. Eling, M., M. Luhnen, 2010. Efficiency in the international insurance industry: A cross-country comparison. Journal of Banking and Finance, 34, 1497-1509.

52

Ekayanakea, S.A.B., 1987. Location specificity, settler type and productive efficiency: A study of the Mahaweli project in Sri Lanka. Journal of Development Studies, 23, 509-521. Fabbri, P., Fraquelli, G., Giandrone, R., 2000. Costs, Technology and Ownership of Gas Distribution in Italy. Managerial and Decision Economics, 21(2), 71-81.

Falavigna, G., Ippoliti, R., Manello, A., 2013. Hospital organization and performance: a directional distance function approach. Health Care Management Science, 16, 139-151. Farsi, M., Filippini, M., Kuenzle, M., 2007. Cost efficiency in the Swiss gas distribution sector. Energy Economics, 29, 64-78. Farsi, M., Filippini, M., 2009. An analysis of cost efficiency in Swiss multi-utilities. Energy Economics, 31, 306-315.

Fukuyama, H., Matousek, R., 2011. Efficiency of Turkish banking: Two-stage network system. Variable returns to scale model. Journal of International Financial Markets, Institutions and Money, 21 (1), 75–91.

Fragkiadakis, G., Doumpos, M., Zopounidis, C., Germain, C., 2014. Operational and economic efficiency analysis of public hospitals in Greece. Annals of Operations Research, DOI: 10.1007/s10479-014-1710-7. Gardner, L., M.F. Grace. 1993. X-Efficiency in the U.S. Life Insurance Industry. Journal of

Banking and Finance, 17, 497-510.

Glass, J.C., McKillop, D.G., Rasaratnamc, S., 2010. Irish credit unions: Investigating performance determinants and the opportunity cost of regulatory compliance. Journal of Banking and Finance, 34, 67–76.

Greene, W., D. Segal. 2004. Profitability and Efficiency in the U.S. Life Insurance Industry. Journal of Productivity Analysis, 21, 229-247.

Hall, M.J.B., Kenjegalieva, K.A., Simper, R., 2008. Environmental factors affecting Hong Kong banking: A Post-Asian financial crisis efficiency analysis. HKIMR Working Paper.

Hardwick, P., 1997. Measuring cost inefficiency in the UK life insurance industry. Applied Financial Economics, 7, 37-44.

Hasan, I., Marton, K., 2003. Development and efficiency of the banking sector in a transitional economy: Hungarian experience. Journal of Banking and Finance, 27, 2249-2271

Heshmati, A., Kumbhakar, S. C., 1997. Efficiency of the Primary and Secondary Schools in Sweden. Scandinavian Journal of Educational Research, 41, 33-51.

Hou, X., Wang, Q., Zhang, Q., 2014. Market structure, risk taking, and the efficiency of Chinese commercial banks. Emerging Markets Review, 20, 75–88.

Huang, W., Eling, M., 2013. An efficiency comparison of the non-life insurance industry in the BRIC countries. European Journal of Operational Research, 226, 577–591.

Hughes, M.D., 1988. A stochastic frontier cost function for residential child care provision. Journal of Applied Econometrics, 3, 203–214.

Kalirajan, K., 1982. On measuring yield potential of the high yielding varieties technology at farm level. Journal of Agricultural Economics, 33, 227–235.

53

Kalirajan, K., 1984. Farm-Specific Technical Efficiencies and Development Policies. Journal of Economic Studies, 11, 3 – 13.

Kaparakis, E.I., Miller, S.M., Noulas, A., 1994. Short-run cost efficiency of commercial banks: A flexible stochastic frontier approach. Journal of Money, Credit, and Banking, 26, 875-893. Kasman, A., E. Turgutlu. 2009. Cost efficiency and scale economies in the Turkish insurance industry. Applied Economics, 41, 3151–3159. Knittel, C., 2002. Alternative Regulatory Methods and Firm Efficiency: Stochastic Frontier Evidence from the U.S. Electricity Industry, Review of Economics and Statistics, 84(3), 530-540.

Kounetas, K., Papathanassopoulos, F., 2013. How efficient are Greek hospitals? A case study using a double bootstrap DEA approach. European Journal of Health Economics, 14, 979-994.

Koop, G., Osiewalski, J., Steel, M., 1997. Bayesian efficiency analysis through individual effects: Hospital cost frontiers. Journal of Econometrics, 76, 77-105.

Kumbhakar, S.C., Biswas, B., Bailey, D., 1989. A Study of Economic Efficiency of Utah Dairy Farmers: A System Approach. Review of Economics and Statistics, 71, 595-604.

Kumbhakar, S. C., 1994. Efficiency estimation in a profit maximising model using flexible production function. Agricultural Economics, 10, 143–152.

Kenjegalieva, K.A., Simper, R., Weyman-Jones, T.G., 2009. Efficiency of transition banks: inter-country banking industry trends. Applied Financial Economics, 19, 1531–1546.

Larsen, K., 2010. Effects of machinery-sharing arrangements on farm efficiency: evidence from Sweden. Agricultural Economics, 41, 497–506.

Latruffe, L., Davidova, S., Balcombe, K., 2008. Application of a double bootstrap to investigation of determinants of technical efficiency of farms in Central Europe. Journal of Productivity Analysis, 29:183–191.

Leme, R. C., Paiva, A.P., Santos, P.E.S., Balestrassi , P.P., Galvao, L. L., 2014. Design of experiments applied to environmental variables analysis in electricity utilities efficiency: The Brazilian case. Energy Economics, 45, 111–119.

Lee, B. L., 2011. Efficiency of Research Performance of Australian Universities: A Reappraisal using a Bootstrap Truncated Regression Approach. Economic Analysis and Policy, 41, 195–203.

Lorenzo, J. M. P., Sánchez, I. M. G., 2007. Efficiency evaluation in municipal services: an application to the street lighting service in Spain. Journal of Productivity Analysis, 27, 149-162.

Ma, C., Zhao, X., 2015. China's electricity market restructuring and technology mandates: Plant-level evidence for changing operational efficiency. Energy Economics, 47, 227–237.

Mbuvi , D., Witte, K., Perelman, S., 2012. Urban water sector performance in Africa: A step-wise bias-corrected efficiency and effectiveness analysis. Utilities Policy, 22, 31–40.

54

Mester, L., 1993. Efficiency in the savings and loan industry. Journal of banking and Finance, 17, 267-286.

Mester, L.J., 1996. A study of bank efficiency taking into account risk-preferences. Journal of Banking and Finance, 20, 1025-1045.

Miningou, E. W., Vierstraete, V., 2013. Households' living situation and the efficient provision of primary education in Burkina Faso. Economic Modeling, 35, 910-917.

Mitropoulos, P., Mitropoulos, I., Sissouras, A., 2013. Managing for efficiency in health care: the case of Greek public hospitals. European Journal of Health Economics, 14, 929-938.

Naper, L.R., 2010. Teacher hiring practices and educational efficiency. Economics of Education Review, 29, 658–668.

Nedelea, I.C., Fannin, J. M., 2013. Impact of conversion to Critical Access Hospital status on hospital efficiency. Socio-Economic Planning Sciences, 47, 258–269.

Nedelea, I. C., Fannin, J. M., 2013. Technical efficiency of Critical Access Hospitals: an application of the two-stage approach with double bootstrap. Health Care Management Science, 16, 27-36.

Nedelea, C., Fannin, J. M., 2013. Analyzing cost efficiency of Critical Access Hospitals. Journal of Policy Modeling, 35, 183–195.

Neff, D.L., Dixon, B.L., Zhu, S., 1994. Measuring the efficiency of agricultural banks. American Journal of Agricultural Economics, 76(3), 662-668.

Olson, K., Vu, L., 2009. Economic efficiency in farm households: trends, explanatory factors, and estimation methods. Agricultural Economics, 40, 587–599.

Parikh, A., Shah, K., 1994. Measurement of Technical Efficiency in the North-West Frontier Province of Pakistan. Journal of Agricultural Economics, 45(1), 132–138.

Picazo-Tadeo, A.J., Gomez-Limonb, J. A., Reig-Martinez, E., 2011. Assessing farming eco-efficiency: A Data Envelopment Analysis approach. Journal of Environmental Management, 92, 1154–1164.

Prado‐Lorenzo, J.M., García‐Sánchez, I., 2010. Effect of operation size, environmental conditions and management on municipal sewerage services. International Journal of Productivity and Performance Management, 59(3), 206 – 228.

Puenpatom, R. A., Rosenman, R., 2008. Efficiency of Thai provincial public hospitals during the introduction of universal health coverage using capitation. Health Care Management Science, 11, 319-338.

Rai, A., 1996. Cost Efficiency of International Insurance Firms. Journal of Financial Services Research, 10, 213-233.

Rosko, M., 2001. Cost efficiency of US hospitals: a stochastic frontier approach. Health Economics, 10, 539-551.

Robst, J., 2001. Cost Efficiency in Public Higher Education Institutions. The Journal of Higher Education, 72(6), 730-750.

55

Rossi, M. A., Ruzzier, C. A., 2000. On the regulatory application of efficiency measures. Utilities Policy, 9(2), 81–92.

Ruggiero, J., Vitaliano, D.F., 1999. Assessing the efficiency of public schools using data envelopment analysis and frontier regression. Contemporary Economic Policy, 17(3), 321–331.

Santin, D., Sicilia, G., 2015. Measuring the efficiency of public schools in Uruguay: main drivers and policy implications. Latin American Economic Review, DOI 10.1007/s40503-015-0019-5.

See, K.F., 2015. Exploring and analysing sources of technical efficiency in water supply services: Some evidence from Southeast Asian public water utilities. Water Resources and Economics, 9, 23-44.

Skevas, T., Lansink, A.O., Stefanou, S. E., 2012. Measuring technical efficiency in the presence of pesticide spillovers and production uncertainty: The case of Dutch arable farms. European Journal of Operational Research, 223, 550–559.

Simões, P., Witte, K., Marques, R.,, 2010, Regulatory structures and operational environment in the Portuguese waste sector. Waste Management, 30, 1130-1137.

Tsekouras, K., Papathanassopoulos , F., Kounetas , K., Pappous, G., 2010. Does the adoption of new technology boost productive efficiency in the public sector? The case of ICUs system. International Journal of Production Economics, 128, 427–433.

Ullah, A., Perret, S. R., 2014. Technical- and Environmental-Efficiency Analysis of Irrigated Cotton-Cropping Systems in Punjab, Pakistan Using Data Envelopment Analysis. Environmental Management, 54, 288-300.

Vitaliano, D.F., Toren, M., 1994. Cost and efficiency in nursing homes: a stochastic frontier approach. Journal of Health Economics, 13, 281-300.

Vitaliano, D.F., Toren, M., 1996. Hospital Cost and Efficiency in a Regime of Stringent Regulation. Eastern Economic Journal. 22, 161-175.

Wagstaff, A., 1989. Estimating efficiency in the hospital sector: a comparison of three statistical cost frontiers. Applied Economics, 21, 659-672.

Wanke, P., Barros, C.P., Faria, J.R., 2015. Financial distress drivers in Brazilian banks: A dynamic slacks approach. European Journal of Operational Research, 240, 258–268.

Watto, M.A., Mugera, A. W., 2014. Measuring Production and Irrigation Efficiencies of Rice Farms: Evidence from the Punjab Province, Pakistan. Asian Economic Journal, 28, 301–322.

Witte, K., Marques, R.C. 2010. Designing performance incentives, an international benchmark study in the water sector. Central European Journal of Operations Research, 18, 189-220.

Wolszczak-Derlacz, J., Parteka, A., 2011. Efficiency of European public higher education institutions: a two-stage multicountry approach. Scientometrics, 887-917.

Wouterse, F., 2010. Migration and technical efficiency in cereal production: evidence from Burkina Faso. Agricultural Economics, 41, 385–395.

56

Zhang, T., Matthews, K., 2012. Efficiency convergence properties of Indonesian banks 1992–2007. Applied Financial Economics, 22, 1465-1478.

Zhao, X., Ma, C., 2013. Deregulation, vertical unbundling and the performance of China's large coal-fired power plants. Energy Economics, 40, 474–483.

Zuckerman, S., Hadley, J., Iezzoni, L., 1994. Measuring hospital efficiency with frontier cost functions. Journal of Health Economics, 13, 255-280.