An Outcome-based Resource Allocation Model: Local Education Services in Wales

An Outcome-based Resource Allocation Model: Local Education

Services in Wales

SG ‘ Tackling Multiple Deprivation’ Conference2 June 2009

Prof Glen Bramley

School of the Built Environment

What’s this paper about?

• ‘Outcome-based resource allocation’ appears to be in vogue – as in ‘Local Outcome Agreements’

• But such approaches, based on targeting outcomes, may only apply to marginal elements of total spending

• Mainstream service funding streams still often governed by more traditional formulae based on demographics and ‘need indicators’

• These formulae tend to replicate past patterns of spending

• They may give more to deprived areas, but do they give enough? And do they reflect different disadvantages evenly?


A Fresh Approach

• Paper looks at what a systematic ‘outcome-based’ approach to funding a mainstream service could look like

• Takes case of local education in Wales• Asks ‘what are we trying to achieve?’• Identifies key things to be modelled – attainment,

costs, special needs, (provision structure)• Models whole national system at indiv pupil,

neighbourhood, school and LEA levels• Highlights key decisions/options in process• Demonstrates how far you could go towards more

desirable (equitable) outcomes, with given resources• Discusses issues arising, & linked policies


The Research Base

• Treasury/NRU/Scot Exec ‘Mainstream Services & Neighbourhood Deprivation’ (Bramley, Evans, Noble 2005)

• Scot Exec Educ Dept ‘Home ownership and educational achievement’(Bramley & Karley, Housing Studies, 2007)

• Fife Council ‘Developing a Social Justice Analysis System for Fife’, (Bramley & Watkins 2005-06)

• Welsh Assembly Government ‘Alternative Resource Allocation Methods for Local Government’ (outcome-based funding model for schools; Bramley, Karley & Watkins forthcoming)


The Literature

• Large body of work on educational attainment and outcomes• Increasingly sophisticated methodologies used e.g. multi-

level modelling• Much of this focussed on ‘school effectiveness’• We are rather more interested in the contextual ‘control’

variables in these studies – the influence of family background, neighbourhood, peer group effects, etc.

• Common interest in effect of school resources – some earlier studies rather negative, more recent work tending to find positive effects (but requiring more sophisticated modelling)

• Also issues of structure of school provision, choice/selection• Dominant finding that poverty and social background

strongest factors• Interesting sub-themes around area/school concentration

effects and housing tenure effects


What are we trying to achieve?

• *Minimum standards approach - a ‘floor’ level of attainment for all areas/schools

• *A convergence approach – a certain proportional reduction in the spread of attainment between most and least deprived areas/school

• *Equal attainment for individual pupils with equivalent initial individual endowment/disadvantage (i.e. trying to neutralise the school or area effect of disadvantage)

• Equal entitlement to (lifetime) educational resources– attainment is mainly relevant via progression, or later participation in adult, further or higher education

• Maximise percentage attaining (say) 5+ A*-C at KS4 across Wales – implies allocating resources at margin where marginal productivity, in terms of this percentage, is highest– social efficiency vs equity

• Incentives approach, whereby schools/LEAs get some bonus for attaining above a (need-related?) threshold level


Operationalising the Concepts

• In order to give effect to these we need a - robust model of determinants of attainment which can distinguish factors we want to take account ofb - robust model linking £ resources to outcomes taking account of environment, provision structure, etc.

• a can be easier than b, and there are issues about getting the best fit-for-purpose model

• These models have been enabled by massive development of administrative data systems linking pupil characteristics, including attainment, to their school and neighbourhood characteristics


Data Sources

• Attainment levels x indiv pupils at 3 key stages (11, 14, 16)• Pupil level census (PLASC) inc age, stage, gender, language,

ethnicity, SEN, FSM + school and post codes• Distance home-school and pupil mobility derived from this using

GIS• School budgets & spend; also size (pupils), type (e.g. denom)• School-level measures of teaching workforce

number/qualifs/turnover/recruitment diffs• Measures of settlement pattern & characteristics based on 216

urban settlements & rural remainders• Census data on socio-demographic characteristics at COA level,

some quite specific (e.g. children with low occup class parents; adults >35 no qualifs)

• Wales Indices of Multiple Deprivation (WIMD)• (No data on school building stock, capacity or quality available)


Table 1: Regression models for attainment in primary and secondary schools in Wales 2006

Primary Secondary Level Explanatory Coeffic t stat Coeffic t stat Variable B B (Constant) 13.36695 102.3 81.920 16.3 Indiv Prior attain KS2 score 4.929 128.9 Indiv Distance from school km -0.00695 -2.2 -0.018 -1.1 Indiv In Care -0.75150 -5.8 -5.421 -4.4 Indiv Fluent Welsh 0.21463 8.2 3.487 13.4 Indiv Age -1.195 -4.6 Indiv Girl 3.765 25.1 Indiv Non-white/non-British -0.06435 -1.9 2.283 6.9 Indiv Any SEN -1.44589 -25.6 -5.132 -8.0 Indiv SEN Stage -0.78429 -20.8 -1.381 -3.0 Indiv Statement -0.27625 -2.5 7.399 5.5 Indiv EBD 1.10482 17.9 -7.505 -14.1 Indiv Severe -2.64023 -18.1 5.152 2.7 Indiv FSM eligible -0.53155 -20.9 -4.577 -19.1 Indiv Moved home -0.07204 -2.1 -3.027 -10.0 Indiv Moved school -0.17047 -2.5 -14.284 -9.9 Indiv Moved both -0.63923 -9.4 -14.102 -10.7 School MeanSized06 9.9815 4.5 School Recipsize06m -0.03650 -1.6 2.646 3.5 School Sizesquare06m -3.953 -4.3 School Denom 0.16431 6.1 3.078 12.2 School Pred exp /pupil 0.000107 2.3 0.00476 8.2 School Applications/teaching post 0.00267 5.9


LEA SEN non-deleg exp/pupil -0.0017 -1.8 LSOA WIMD housing score -0.0175 -4.0 LSOA WIMD access score 0.012 2.5 COA Owner occupiers % 0.00253 4.1 0.035 5.8 COA Flats % 0.00230 3.0 COA Cohab Cpl w/kids % -0.01192 -3.4 COA Lone parent hhld % -0.086 -5.3 COA No Qualifs (35+) % -0.01110 -11.9 -0.121 -15.0 COA Children Low NS-SEC % -0.00315 -4.0 -0.056 -8.4 COA Children Caring % -0.058 -2.1 School Travel >10km 0.041 6.5 School Minority Ethnic 0.041 3.9 School anysens 7.407 3.9 School statements -1.26213 -2.7 -40.032 -7.1 School ebds -20.028 -4.5 School sfsm -0.82554 -8.3 -14.647 -10.1 School pmvhome 0.091 4.4 School pmvscl 0.207 9.6 School School resid effectiveness 1.00451 68.2 1.006 55.1 Dep Var: vKS2Score Mean S D Dep Var: ks4pts100 mean & Std Dev 12.333 2.256 43.227 21.223 Adjusted r-sq 0.495 0.591 Std Err Est 1.588 13.535 F Ratio 1263.8 1260.203 N of Pupils 32155 33139 N of Schools 1351 218


Key Findings - Attainment

• Indiv pupil factors explain most, esp prior attainment (+ secondary),SEN -, FSM (- poverty), in care -, mobility +/-

• Varying school size effects (small schools less effective)• Spending /pupil positive, (stronger /clearer in secondary)• Concentrations of FSM & SEN in schools –ve• Neighbourhood factors: housing depriv (-), lone or cohab

parents (-), no qualifs -, low SEG –, children caring (-)• Owner occupation +, rural +, flats + (city centre?@)


Cost Models

• Models estimated for school costs per pupil at school level• Reasonable fit (r-sq 62%-73%)• Small primaries cost much more• Size, composition, quality of teaching force• Transport/distance effects

(but note separate model developed for transport costs)• SEN factors• Pupil mobility• School level FSM x weighting in LEA’s funding formula for

deprivation


Special Needs

• Tried also to model incidence of composite measure of special needs

• Similar to attainment models• Fit generally poorer (random variations in

assessment practice, or in LD incidence?)• Included LEA ‘dummy’ variables to proxy local

policy variation• Some systematic variation with similar social

disadvantage variables as in attainment models- in care, FSM, renters, no qualifs, low occups, lone or cohab parents, poor health, overcrowding


Outcome –based funding model

• Analysis at school (‘virtual catchment’) level• Standardize school size for settlement size • Standardize costs given size, spec needs, etc.• Measure relative disadvantage due to social/other

(?) factors (in terms of attainment)• Allocate enough extra money to bring predicted

attainment x% closer to mean (or to a minimum standard level y s.d. below mean)

• Given minimum school £ allocation, >=: lowest observed, feasible degree of redistribution is determined


Table 4: Factors Taken account of in the Different Spending Need Estimates

Factor (Source of Variation) Standard Cost

Outcome Crit A

Outcome Crit B

Outcome Crit C

School Size -Actual No No No No

School Size - Modelled Yes No Yes No

Structural features e.g. nursery, teacher recruitment

Yes No No No

Other structural e.g. denominational

No No Yes No

Pupil/neighbourhood social need inc FSM, in care

Yes Yes Yes Yes

Other pupil attribs, e.g. gender, language, ethnic, distance

n/a No Yes No

SEN Social element Yes Yes Yes Yes (Cii)

SEN Policy element No No No No

SEN remaining element Yes Yes Yes No

Individual pupil mobility n/a Yes Yes Yes

Pupil turnover at school level Yes No Yes No

Prior Attainment (secondary) n/a Yes Yes No

School Effectiveness n/a No No No

Criterion B compensates for‘almost everything’, to a degree;

Criterion C compensates for specified social disadvantages,

to a (higher) degree;Criterion A brings up to minimum

attainment standard, allowingfor more factors than C but less

than B.


Actual Base Exp Need Exp Need Exp Need Exp Need LEA Name Expend Expend Crit A Crit B Crit Ci Crit Cii

Min 2531 2597 1971 2175 2450 2350 Max 3785 3153 3561 3360 3339 3472 Std Dev 266 155 407 269 234 287 C V 9.5 5.5 14.5 9.6 8.3 10.2 Ratio 1.50 1.21 1.81 1.54 1.36 1.48 Parameters Delta 0.30 0.40 0.40 Min Std -0.4 SDs Levy 0.70 Min School Expend 1650 1650 1650 1650

Outcome based allocations are not overall (much) more variable than existing expenditure.The most redistributive is Criterion A ‘Minimum Standard’.

Criterion B can only go 30% towards full equalization of outcomesCriterion B can go 40% of way to equalizing for social disadvantages


Outcome-based needs for primary schoolsLEA Name Actual Need CiiBlaenau Gwent 2913 3472Merthyr Tydfil 2857 3277Neath P T 3026 3209Rhondda C T 2654 3023Carmarthen 3181 2989Torfaen 2752 2928Caerphilly 2581 2917Newport 2790 2876Swansea 2693 2856Cardiff 2857 2852Pembroke 2941 2849

Ceredigion 3785 2838Powys 2924 2813Gwynnedd 2875 2717Bridgend 2578 2716Wrexham 2678 2696Anglesey 2822 2664Denbigh 2744 2563Monmouth 2572 2441Conwy 2781 2432Flint 2531 2367Vale of Glam 2719 2350

Wales Ave 2801 2812

Outcome-based expenditure need (partial convergence) by current expenditure per pupil

2000

2500

3000

3500

4000

2000 2500 3000 3500 4000

Actual Expenditure per pupil

Expe

nditu

re N

eed

(crit

erio

n Ci

i) Need Cii

Outcome-based funding distribution (partial convergence)

0

1000

2000

3000

4000

1 3 5 7 9 11 13 15 17 19 21

LEA

£ pe

r pup

il

Series1

Note: needs formula based on standardized costs and compensating for 40% of social disadvantage





Patterns of Redistribution

Actual Expenditure and Outcome-Based Need by Type of LEA

(Primary, Criterion A, High Floor)

0

1000

2000

3000

4000

AffRur AffUrb DepRur DepUrb

Category of LEA

£ pe

r pup

il

Act Exp

OB Exp Need

• Outcome based formulae generally redistribute from rural to urban and from affluent to deprived areas

• The main gaining group are deprived urban

• Rural losses are partly due to provision rationalisation and cost standardisation

• There is considerable variation in the impact among relatively deprived LEAs (some were already quite well funded)

Difference of Outcome-based assessment

versus existing SSA (Primary, Criterion A, High Floor, percent)

-30.0-20.0-10.0

0.010.020.030.0

0.50 0.70 0.90 1.10 1.30 1.50

Composite Deprivation

Perc

ent D

iffer

ence

pdiffSSA


Secondary Schools Summary

Actual Base A B Ci Cii Min 3123 3367 2921 3220 3129 3096 Max 4199 3813 4135 3966 3996 4013 Std Dev 204 114 357 209 246 260 C V 5.8 3.2 10.2 6.0 7.0 7.4 Ratio 1.34 1.13 1.42 1.23 1.28 1.30 Parameters Delta 0.30 0.60 0.60 Min Std -0.4 SDs Levy 0.80 Min School Expend 2750 2750 2750 2750


Secondary Schools - Comments

• Range of existing spend and new formula allocations tend to be narrower

• Similar high/gaining LA’s (Blaenau Gwent, Merthyr Tydfil) and low/losing Las (Monmouth, Vale of Glam, Ceredigion)

• Criteria A and B achieve similar degree of levelling up /partial (30%) equalization, as in primary case; Criterion C achieves higher degree of equalization (60%)

• Results not that dramatic due to- high floor of minimum spend per school- larger size of schools, less scope for polarisation- stronger expenditure->attainment effect- prior attainment excluded from factors allowed for in Crit C


Conclusions 1

• Feasibility outcome-based approach demonstrated• Shows value of rich admin databases• Care needed over

- specification of what reasons for difference are compensated- modelling of cost and of school provision structure- understanding and modelling special needs provision

• Outcome-based resource allocation is not a utopian fantasy- even with minimum allocation floors, substantial progress could be made- variation in allocations not significantly greater than current actuals

• But, you can’t fully equalize, or level up to mean, with current resources

• Needs formulae could be simplified, with more current/updating indicators


Conclusions 2

• Initial reaction to this report mixed – LA’s find it difficult to agree – zero sum game

• Disparities between schools (& neighbourhoods) greater, but LEA formulae allocating to schools typically even less redistributive

• Small rural schools get most funding per pupil, and are of dubious educational value, but this issue is sensitive

• Would have been easier to apply progressive redistribution in 2001-07 with increasing real resources

• Still possible to aim for gradualist move towards outcome-oriented spending targets – as in NHS


Reflections on Resource Allocation

• ‘Poor’ areas tend to get poorer service outcomes, across quite diverse kinds of service

• Poverty/social deprivation makes the service provision task more difficult and potentially costly

• Poor areas get more resources of some kinds but less or the same of others

• They do not get enough extra resources to make a decisive difference to outcomes

• Therefore it may appear that there is a perverse negative relationship of resources with outcomes

• Local political resistance to re-allocation of resources likely to be formidable


Complementary approaches to improving school outcomes

• Reduction in poverty thru’ e.g. tax/benefits, labour market, minimum wage, etc. (poverty the strongest predictor of poor outcomes)

• Reduction in concentrations of poverty, e.g. thru’ planning/regeneration including tenure diversification*(* Bramley & Karley article in Housing Studies 2007 argues that owner occupation at indiv/nhood/school levels raises attainment)

• Focused use of ‘special needs’ resources e.g. special units for disturbed pupils

• Close or amalgamate failing schools• Earlier intervention, preschool/nursery; after school

clubs• Changing curriculum (addressing motivation,

engagement)

Documents

An Outcome-based Resource Allocation Model: Local Education Services in Wales