Per J. AGRELLPeter BOGETOFT

2001-06-06

DEA - A Fresh Cure for Health Care Reimbursement?

Plenar talk at XXI Spanish Congress of Health Economics, Oviedo, June 6-8, 2001

© SUMICSID 2

Outline

1. Who Are We ?2. DEA3. Widespread Concerns About DEA4. The Consultant’s Answer5. The Theorist’s Answer6. Lessons from Theory7. Conclusions8. Literature9. Appendix

© SUMICSID 3

The SUMICSID Team ?

Dr. Per J. Agrell, CORE/UCL– pja@sumicsid.com

Prof. dr. merc. Peter Bogetoft, KVL– pb@sumicsid.com

Decision Theory (MCDM), Efficiency Evaluation (DEA) and Incentive Theory (Agency, Contracts)

© SUMICSID 4

Wide Use of Dea

• CCR(1978); 1000 papers 2000

• Regulators use DEA to estimate industry-wide or individual productivity improvement potentials.

• E.g. electricity distribution often use DEACountry Reg.App. Eval.Meth. Development / In useAustralia Ex ante CPI-DEA/SFA/Stat UDenmark Ex ante CPI-COLS D/UEngland Ex ante CPI-DEA/COLS U Finland Ex post DEA? DNetherlands Ex ante CPI-DEA UNew Zealand Ex ante CPI-DEA UNorway Ex ante CPI-DEA USpain Ex ante Ideal-Net DSweden Ex post DEA D

• Is DEA useful in health regulation ?

© SUMICSID 5

Performance Evaluation

PROCESSPROCESS

Exogenous factors(Non-discretionary resources or products)

Resources(Inputs)

Products(Outputs)

Management(Effort / Ability)

© SUMICSID 6

Rational Ideal Evaluation

TEACHING

RESEACH

E

B

C

A

T

Ideal

Pref.U(.)

Effectiveness

=

=

Actual Performance

Ideal Performance

U (D)

U (Ideal)D

© SUMICSID 7

Practical Problems

IN NON-IDEAL REALITY:

LACK OF INFORMATION:Neither preferences nor possibilities are known.Information is at best decentralized and not immediately available.We need full investigation, communication, choice procedures (DEA, MCDM etc)

STRATEGIC BEHAVIOR:Agents may not choose desired output unless given proper incentivesAgents may misrepresent information.We need full mechanism design procedures (Game, Agency, Auction etc theory)

© SUMICSID 8

Basic Ideas

U n kn ow n p e f. U

U n kn ow n p oss . T

R e la tive E ffic ien cy

E ffic ien cy

E ffec tiven ess i, iU x y

max U (x,y) s.t. (x,y) T

Produce more with less

Estimate empirical reference technology T*

i i, iE min E Ex y T

i i iE min E Ex ,y T *

© SUMICSID 9

Basic Efficiency Measures

Farrell (1957)

INPUT BASED : E = find largest proportional contractions of all inputs E

= minimal input/actual inputsE= 0.7 means that all inputs could be reduced by 30%

OUTPUT BASED: F = find largest proportional expansions of all outputs F

= maximal feasible output / actual outputF=1.4 means that all outputs could be expanded by 40%

ALTERNATIVE MEAURES:non-radial, directional, additive,… but Farrell’s easiest to interpret.

© SUMICSID 10

(Relative) Technical Efficiency 1

INPUT,fte

OUTPUT, DRG x

A

B

C

D

120

75

1 200

2 000

TE-OUT

TE-IN

TE-IN = 75/120 = 62,5%

TE-OUT = 2000/1200 = 167 %

© SUMICSID 11

(Relative) Technical Efficiency 2

TEACHING

RESEARCH

E

B

C

A

D

Fixed input

O

D’

| ' |1.3

| |

ODF

OD

ASS PROF

FULL PROF

F

I

J

H

G

Fixed output

I’

| ' |0.6

| |

OIE

OI

O

© SUMICSID 12

Other Efficiency Concepts

TECHNICAL EFFICIENCY (TE)Right methods, procedures etc given input and output mix

ALLOCATIVE EFFICIENCY (AE)Right input mix given prices

COST EFFICIENCY (CE)Technical and allocative efficiency: CE=TE•AECorresponds to TE in a cost model

SCALE EFFICIENCY (SE)Right scale of operation (max output per input, min average cost)TE(crs)=TE(vrs) • SE

PROFIT EFFICIENCY (PE)Right input-output mix given prices

© SUMICSID 13

Alternative Estimation Principles

FRONTIER ANALYSIS

Resource, M$, Staff budget

Performance, DRG x, Health care

AnotherHospital Co

100

1 000

2 000

Smalll HMO

50

THEORETICAL NORMS

STATISTICAL METHODSCathCH

GIANT CH

© SUMICSID 14

DEA Estimation

• Weak regularity assumptions– Free disposability– Convexity– Return to Scale

• Minimal Extrapolation Principle • Very flexible, best practice frontier

© SUMICSID 15

Basic DEA Models

INPUT

OUTPUT

A

B

C

ED

OUTPUT

A

B

C

E

D

INPUT

OUTPUT

A

BC

E

D

INPUT

FDH VRS

CRS

OUTPUT

A

B

C

E

D

INPUT

DRS

FDH

DRS CRS

VRS

© SUMICSID 16

DEA Pros and Cons

PROS– Requires no or little preference, price or priority information– Requires no or little technological information– Makes weak a priori assumptions– Handles multiple inputs and multiple outputs– Provides reel peers– Identifies best practice– Cautious or conservative evaluations (minimal

extrapolation)– Supports learning and planning and motivation

 

CONS– Relatively weak theory of significance testing (sensitivity,

resampling, bootstrapping, asymptotic theory)– Lack of focus of goals

© SUMICSID 17

Why evaluate?

Applications of Evaluations

Plan, reallocate,...MotivateLearn, explore, stat,...

© SUMICSID 18

Why Is DEA So Popular ?

• EASY TO USEminimize regulator’s effort

• EASY TO DEFENDYes:

easy to explain mild regularity assumptionshandles multiple inputs and outputsNo:explicit peers can be challengedslack and noise possibly entangled

© SUMICSID 19

Widespread Concerns

Regulators, firms and researchers:

THE MEASUREMENT PROBLEM– Is it possible to capture complex output like in health ?– Would it not be better to trust parties and use

retrospective (cost-plus) payment ?

THE NOISE PROBLEM– Is DEA the appropriate procedure given its sensitivity to

noise ?– Would it not be better to use econometric methods, SFA

etc ?

© SUMICSID 20

The Consultant’s Answer

“DEA puts everyone in their best light”

Correct ?Yes:

Minimal Extrapolation Principle and weak a priori regularity on technology

In TE multiplicity of goals allowedNo:

Noise and Best Practice not distinguished.

© SUMICSID 21

The Theorist’s Answer (I)

The measurement problem:

– Not unique to health – e.g. university evalution extremely complex!

– DEA work well with multiple inputs and outputs– Is the problem many outputs – or a complex

functional form ?– Conflicts of interest/ Asymmetric information makes

cost-pus / pure retrospective payment problematic

© SUMICSID 22

The Theorist’s Answer (II)

The appropriateness of DEA depends on:

How it is performed– METHODOLOGY

What it is used for– OBJECTIVES

When/where it is used– CONTEXT

© SUMICSID 23

Methodology

To be well-executed, it might involve:

• Careful data collection• Sensitivity analysis

• Monte Carlo, peeling techniques, alt. technology assumptions

• Stochastic programming• Hypothesis test

– Boot strapping, re-sampling, asymp. theory

© SUMICSID 24

Objectives

• DEA can– improve efficiency, distribution, social welfare– support concession granting, monitoring and information

dissemination– reduce administrative workload

• Noise may not matter– large impact on few units and small impact on many units– counteracted by regulator’s discretion (40% red.over 3

years) – some DEA estimates (e.g. MPSS) are more unstable than

others

© SUMICSID 25

Context

• Important aspects:– Technology (general assumptions plus impact of effort)– Information (noise, uncertainty, asymmetry)– Preferences (firms, customers, regulator, society)

• DEA is most appropriate when – Uncertainty about the structure of the technology (rates of

substitution etc) is as significant as individual noise

• Hence:– Noisy data, simple technology -> use SFA, Econometrics– Better data, complex technology -> use DEA

• See more details below

© SUMICSID 26

Lessons from Theory

Some models and results connecting incentive and productivity analysis techniques:

– Research Approach– Super- Efficiency– Static Incentives– Dynamic Incentives– Structural Developments

© SUMICSID 27

•Linkage of two literatures:

Production theoryDEA etc.

•Performance Eval.

Incentives theoriesAgency etc.

•See appendix 1 for more on this.

Research Approach (I)

Org. model

DEA

© SUMICSID 28

Research Approach (II)

THE BASIC PROBLEM:

Given cross section, time series or panel information:

(input, output) for DMUs i=1,…,nwhat should we ask an agent to do and

how should we reimburse him/her ?

© SUMICSID 29

Research Approach (III)

Reject

Accept

Technology perhapslearned by agents

Effort and Slackselected

(Inputs, Outputs)observed

Compensationpaid

Possiblereporting

Historicaldata

Incentive schemesproposed

..... .....

Private Costs and Slack

Production, costs etc

Incentives

DMU DMU DMU

REGULATOR

© SUMICSID 30

Super Efficiency

• Efficiency– can provide incentives to match others, but not

to surpass norm– multiple dim. model further facilitates shirking– Nash Equilibria involve minimal effort

• Super Efficiency– exclude the evaluated unit from the technology

definition– can support the implementation of most plans

© SUMICSID 31

Efficiency and Shirking

D

C B

A

OUTPUT 2

OUTPUT 1

Max Effort Output

© SUMICSID 32

Static Incentives (I)

• Situation:– Technological uncertainty,– Risk aversion– Individual noise

• Result:– DEA frontiers are incentive efficient (supports optimal

contracts) when noise is exponential or truncated

• Result:– DEA frontiers asymptotically incentive efficient when noise

is monotonic

• Payment:– B = B(x,y,CDEA)

© SUMICSID 33

Static Incentives (II)

• Situation– Technological uncertainty,– Risk neutrality– DMU maximizes {Profit + •slack}

where 0< <1 is the relative value of slack

• Result:– Optimal revenue cap under non-verifiable costs is

k + CDEA(y)

Constant + DEA-Estimated Cost Norm

© SUMICSID 34

Static Incentives (III)

• Result:– Optimal revenue cap with verifiable costs:

k + c+ •( CDEA(y) –c )

Constant + Actual Costs+ of DEA-est. cost savings

• Extensions:– Similar schemes work under varying demand assumptions,

genuine social benefit function, etc.

• Hence: DEA provides an optimal revenue cap !

© SUMICSID 35

Static Incentives (IV)

Cost

Production y

DEA Estimated Cost Norm CDEA(y)

Actual Cost

Yardstick Cost

Savings

Payment = Lump Sum + Actual Cost + Savings

DEA-based Yardstick Competition

© SUMICSID 36

Dynamic Incentives (I)

• Additional dynamic issues– Accumulate and use new information– Avoid ratchet effect

• Result:

k + ct+ •( C1-tDEA(y) –c )

Constant + Actual Costs+ of DEA-Est. Cost Savings

© SUMICSID 37

• Situation:– Limited catch-up capability

• Result:– Optimal revenue cap with limited cath-up capability:

k + ct+ •( (1-(1-E0))tC1-tDEA(y)/E0 –ct )

Constant + Actual Costs+ of adjusted DEA-est. cost savings

Dynamic Incentives (II)

© SUMICSID 38

Dynamic Incentives (III)

Dynamic, DEA based yardstick schemes solve many of the usual CPI-x problems:

• Risk of bankruptcy with too high x• Risk of excessive rents with to low x• Ratchet effect when updating x• Arbitrariness of the CPI measure• Arbitrariness of the x parameter• Inability to include changing output profiles

© SUMICSID 39

Dynamic Incentives (IV)• Situation:

– Single dimensional output– Constant return to scale– Fixed relative factor prices– Exogenous constant frontier shift of – No difference between profit and slack value =1

• Result:– The CPI-DEA scheme used in electricity distribution in

Norway (see appendix 2) is optimal

© SUMICSID 40

Dynamic Incentives (V)

• Situation:– Support innovation (frontier movements),– Support info dissemination (sharing)

• Result:– An operational scheme with innovation and dissemination is:

k + ct+ •( C1-tDEA(y) –ct) + bt

I+btD

Incentive = Cost+Profitshare+Innovation+Disseminationbt

I = innovation premium

btD = dissemination premium •(Ct-1–Ct )

© SUMICSID 41

Structural Developments

• Final concerns:Scale adaptationScope adaptationthrough incentives and concession granting

• Mergers:Adjust DEA based yardstick to share scale and scope gains

• Auctions:DEA based yardstick to aggregate multi-dimensional bids

© SUMICSID 42

Conclusions (I)

DEA frontiers – sufficient for exponential noise, truncated noise and– asymptotically sufficient for monotone noise

DEA based revenue cap optimal under considerable technological uncertainty

SFA, Econometric revenue cap useful under considerable individual uncertainty

Dynamic re-estimation, ex ante commitment to ex post regulation, solves many CPI-x problems

© SUMICSID 43

Conclusions (II)

DEA useful technique in yardstick regimes – supports– Complex technology– Partially undefined objectives– Conflicts of interest among agents– Asymmetric information– Organizational learning and innovation

DEA makes minimal assumptions and its results are hard to refute, even under varying objectives.

Theory combines DEA and agency theory.

Health care regimes complemented with DEA may be beneficial.

© SUMICSID 44

Some Current Events

Sixth European Workshop on Efficiency and Productivity Analysis,

Copenhagen, Denmark, October 29-31, 1999

– www.flec.kvl.dk/6ewepa

Seventh European Workshop on Efficiency and Productivity Analysis, Oviedo, Spain, September 25-27, 2001.– www19.uniovi.es/7ewepa

INFORMS Conference, Dynamic DEA Regulation session,

Hawaii, June 17-20, 2001.

© SUMICSID 45

Literature (1)

Some are downloadable at www.sumicsid.com

Agrell, P., P. Bogetoft and J.Tind, Multi-period DEA Incentive Regulation in Electricity Distribution, Working Paper, 2000.Agrell, P., P. Bogetoft and J.Tind, Incentive Plans for Productive Efficiency, Innovation and Learning, Int.Journal of Production Economics, to appear, 2000.Bogetoft, P., Strategic Responses to DEA Control, Working Paper, 1990.Bogetoft, P. Non-Cooperative Planning Theory, Springer-Verlag, 1994.Bogetoft, P , Incentive Efficient Production Frontiers: An Agency Perspective on DEA, Management Science, 40, pp.959-968, 1994.

© SUMICSID 46

Literature (2)

Bogetoft, P, Incentives and Productivity Measurements, International Journal of Production Economics, 39, pp. 67-81, 1995.Bogetoft, P, DEA-Based Yardstick Competition: The Optimality of Best Practice Regulation, Annals of Operations Research, 73, pp. 277-298, 1997.Bogetoft, P., DEA and Activity Planning under Asymmetric Information, 13, pp. 7-48, Journal of Productivity Analysis, 2000.Bogetoft, P. and D. Wang, Estimating the Potential Gains from Mergers, Working Paper, 1999.

© SUMICSID 47

Appendix 1:Approach (1)

ContextMultiple, rational, intelligent agents with private info and action

DEA

1) Set up an explicit contextual model using agency theory

2) Assume planner uses DEA 3) Find agents’ response 4) Viability: Prevails incentive

compatibility, will players be obedient and honest ?

5) Performance: Does proposal lead to efficient outcome ?

© SUMICSID 48

Appendix:1Approach (2)

Pick a model with a view towards:

– Conservatism - put DEA in best possible light

– Realism - use relevant context

– Faithfulness- use DEA modification and motivation that are fair to original purposes.

© SUMICSID 49

ECO - general insight, description/ understanding OR - specific proposal, prescription/ normative

•Bad match? Overkill?

• Applied

• Theoretical

foresee regulated firm behaviourprovide appropriate motivation/ prescription

Performance measurement (OR-) - disciplineProvides rich description of production for economic theory

Appendix:1Approach (III)

© SUMICSID 50

Appendix:1Approach (IV)

A Naive Solution:– Estimate cost function: C(output)– Find Benefit Function: B(output),– Choose to maximize {Benefit - Costs}– Pay estimated costs, actual costs, yardstick costs or similar

New questions:– How estimate C(.) ? Use DEA ? Econometrics ?– What is the optimal payment ?– How should additional information feed into the process ?– etc

© SUMICSID 51

Appendix 2The Norwegian Scheme (I)

Cost Model:– DEA cost model to estimate individual inefficiencies

and general productivity development

Payment Scheme:– Revenue cap with rate-of-return restrictions and an

efficiency incentive.– 2 year review period– 5 year regulation period– Deviations (+/-) accounted for in next regulation

period

© SUMICSID 52

Appendix 2The Norwegian Scheme (II)

Core of the regulatory scheme:Rt=PIt,t-1•QIt,t-1 •(1--•Gt) •Rt-1

ct+min •Xt Rt ct+max •Xt whereR revenuec costsPI price indexQI quantity indexG truncated DEA efficiency min{(1-E0)/(1-Elow),1} general productivity improvement (1,5%, Malmquist based) catch up coefficient (max 38.24% eliminated in 4 years) rate-of-return bounds (2%-15%)