Impact conference 2 8 2012

Pathways of Progression Journeys through a Theory of Change

Nicola KassamCentre of Equity, School of Education, University of Manchester

‘Children growing up in poverty and disadvantage are less likely to do well at school. This feeds into disadvantage in later life and in turn affects their children. To break this cycle we need to address the attitudes and experiences that lie behind social differences in education’

(Hirsch, JRF 2007)

The relationship between child poverty & educational attainment in Cornwall*The charts plot the relationship between child poverty and the educational attainment of children in Cornwall at 3 key stages:

Early Years Foundation Stage at the end of reception year 2010

Key Stage 2: proportion of pupils achieving level 4 or above in English & Maths 2009

Key Stage 4: proportion of pupils achieving 5 or more GCSE’s A*-C including English & Maths 2010

*Child Poverty Needs Assessment 2012

http://www.cornwall.gov.uk/default.aspx?page=27214

Addressing Disadvantage Moves towards

Multi-agency Intervention

History of School as Community Hub

International Case Extended Service

Core Offer

Towards Extended Services

History of Extended ProvisionVillage CollegesPathfinder Project Full Service School Extended Schools

Towards Extended Services II Previous Core Offer

A wrap around childcare A varied menu of activitiesSwift and easy referral to targeted and

specialist services including parenting support and family learning

Community access to school facilities and adult learning

Towards Extended Services III

Complexity of Different Initiatives led to Mixed RationalesTargeting attainment & attendance ‘ fluffy’ enrichment activities, ‘add on’ or integrated to the curriculumAddressing wider community social issuesRegeneration Inclusion

Aims & Objectives

Explore school delivery of Extended Service provision to identify each school’s rationale.

Draw out the theories that underpin expected outcomes

Identify if the outcomes occur as predicted in the school context

C+M=O?

Black Box

Measures or conditions before

Measures or conditions

after

Realist Terms

Condition (C) + Mechanism (M) = Outcome (O)

Research Method

Horizontal complexity number of ES activities drawn together, Staff arrangement,

Cluster arrangement, Funding, Vertical Complexity

Individual pathways of progression, Links between ES provison, Influencing variables

Flexible & Evolving adaptable to different situations and stages, Adaptable to the

data produced Contextual Issues

Local community conerns, wider influencing factors beyond the community

Broad range of outcomes, Expected changes and ownership of change

Theory Based Method‘Theory of Change’

Black box is filled with theories, assumptions, predictions (Weiss 1995)

Identify situation, expected outcomes and outputs to produce particular outcomes.

Illustrate predicted pathways and seek out evidence

Test whether the action is having the impacts predicted by the theory

Theory of Change ModelIssues the school wish to address

Background of the school and community

Issues addressed

Action required by school & agencies

Systems required for action, steps and outcome to be achieved

The steps required by target group

Black Box

Study

Three case study schools A, B, C A Theory of Change methodology

Identify situation, context, actions and outputs, steps, outcomes

Uncover evidence of activityReveal pathways of progressionEstablish evidence of impact on social and

educational outcome

School A

School A

Large Primary school in highly deprived area Transient community, large Eastern

European Traveller population. 'Outstanding' Ofsted report. ES integrated into school provision,

coordinated and delivered by school staff ToC Pupil Centred

Rationale Address welfare and

social issues Encourage parental

engagement and empowerment

Improve attainment and pupil enjoyment

Provide opportunities for employment

Generate more active integrated community members

Theory of Change Predicted Pathways

Parents will receive support

Pupil EnrichmentParental Engagement &

EmpowermentPupil Welfare & SupportCommunity Engagement

& Learning

Increase citizenship

Improve thinking skills & behaviour

All parents will have access to support

All pupils will have access to safe secureenvironment

Community will have access to ES provision

& learning

Pupils will receive support for basic needs

All children will have access to activities

Pupils will access these activities

Pupils will learn to navigate school system

Pupils will develop transferable skills and

gain confidence

Pupils will gain new friendships that will

assist transition

Pupils will gain further confidence and a sense

of achievement

Pupils will increase attendance and

punctuality

Pupil attainment and confidence will increase

Pupils will receive support from services

Pupils will develop good relations with school

and staff

Pupils will learn to trust school service

Pupils’ life chances will improve

Pupils will gain confidence with school

Parents will learn to trust school service

Parents will gain confidence and

self esteem

Parents and Community will access learning opportunities

Parents will improve parenting skills

Community will gain self esteem from their peers

Parents will gain knowledge of parental expectations

Community will build relationship with the

school

Community will gain confidence with the school

Community will become more active and provide

further opportunities with the school

Community will access volunteering

opportunities with the school

Parents and community will attain employment skills and become more active citizens

Parents will gain access to employment. Children will become more engaged in

schooling

Parents will gain employment and child and adult life chances will improve

Individual Pathways

Student 1

Male, joined the school in year 3, this was his first experience of education.

Staff have identified that he requires support for his ‘Slovakian cultural difference’, attendance and parenting issues.

He has little contact with most ES support, occasionally accessed play scheme.

He is predicted to be working towards level 1 by the end of the key stage.

In 2 years he had attended the play scheme for only 6 months.

Student 1 Progress

Student G

-1

-0.5

0

0.5

1

1.5

2

0 1 2 3 4 5 6 7

Students School Years Engaged with ES Activities

Asse

ssme

nt Le

vel

student G predictedlevelstudent G actuallevelLinear (student Gpredicted level)Linear (student Gactual level)

Table 1.1 Student 1 Actual and Predicted Assessment Level

Table 1.2 Student 1Pathway

Pupil Enrichment Pupil Welfare & Support



All pupils will have access to safe secureEnvironment






gain confidence


assist transition

Pupils will gain further confidence and a sense of

achievement


punctuality

Pupil attainment and confidence will

increase


Within this short period of time, his Within this short period of time, his attendance increased from 33% to 50% attendance increased from 33% to 50% and attainment from the predicted and attainment from the predicted working towards to above level 1working towards to above level 1

Student A Progression

y = x - 1

y = 0.6667x - 0.6667

0

1

2

3

4

5

6

0 2 4 6 8

Years

Assessm

en

t L

evel

student A predictedlevelstudent A actuallevelLinear (student Aactual level)Linear (student Apredicted level)

Student B Progression

y = 1.6x

y = 0.8x

0

1

2

3

4

5

6

0 1 2 3 4

Years

Assessm

en

t L

evel student B predicted

level

student B actuallevel

Linear (student Bactual level)

Linear (student Bpredicted level)

Student C Progression

y = x - 1

y = x

-1-0.5

00.5

11.5

22.5

33.5

0 1 2 3 4

Years

Ass

essm

ent

Lev

el student C predictedlevel

student C actual level

Linear (student Cpredicted level)

Linear (student Cactual level)

Student D Progression

y = 1.5x - 1.5

y = 1.25x - 1.25

-2

0

2

4

6

8

10

0 2 4 6 8

Years

Ass

essm

ent

Lev

el

student D predictedlevel

student D actuallevel

Linear (student Dactual level)

Linear (student Dpredicted level)

Student E Progression

y = 1.0976x

y = 1.3415x

0

1

2

3

4

5

6

7

8

0 2 4 6

Years

Ass

essm

ent

Lev

el

student E predictedlevel

student E actuallevel

Linear (student Epredicted level)

Linear (student Eactual level)

Student F Progression

y = 1.5976x

y = 1.439x

0

2

4

6

8

10

12

0 2 4 6 8

Years

Ass

essm

ent

Lev

el

student F predictedlevelstudent F actuallevelLinear (student Factual level)Linear (student Fpredicted level)

Student G Progression

y = 0.3x - 1

y = 0.5x - 1

-1

-0.5

0

0.5

1

1.5

2

0 2 4 6 8

Years

Ass

essm

ent

Lev

el student Gpredicted levelstudent G actuallevelLinear (student Gpredicted level)Linear (student Gactual level)

Student H Progression

y = x - 1

y = 0.6x - 1

-1

0

1

2

3

4

5

6

0 2 4 6 8

Years

Ass

essm

ent

Lev

el student Hpredicted levelstudent H actuallevelLinear (student Hactual level)Linear (student Hpredicted level)

Student I Progression

y = 1.75x - 1.75

y = 1.25x - 1.25

-2

0

2

4

6

8

10

0 2 4 6 8

Years

Assessm

en

t L

evel student I predicted

levelstudent I actuallevelLinear (student Iactual level)Linear (student Ipredicted level)

Student 2 Male joined the school in year 4, with his sister year 1. Known to

social services; the family experienced bereavement, drug issues and concerns of a paedophile offender living in the home.

Engaging with his mother resulted in abusive behaviour. The school used ES as a way to feed and deter him from

stealing food for his sister. When he came to the school he could not read or write and was

working towards level 1. He had substantial attendance issues and attended one term in

total since nursery. He received staff support, 1 to1, collected from home and called

everyday. In year 6, he been accessing ES continuously. He was

predicted to be working towards level 1 by the end of the key stage.

Student 2 Progress

By year 6 he achieved level 2 Maths and Level 1 English and was attending 90% of the time.

At secondary school his needs were not met, he disengaged with education.

Maintained relationship with primary staff who advocated for his place

After 2 years staff managed to reinstate his place.

Provision limited to half a day a week, availability of staff limited to cater for his needs.

Table 1.3 Student 2 and Parent Pathway


Pupil EnrichmentParental Engagement

& EmpowermentPupil Welfare & Support










gain confidence


assist transition


achievement


punctuality



Pupils will develop good relations with

school and staff





Parent 1 British Asian female of Bangladeshi origin, in her mid

thirties. She had been engaged with the school for over 10 years

and had brought each of her children to family learning, to assist their development before beginning school.

For her family, achieving a good education was a high priority and central to improving life chances

She played a central role in the community, voicing group opinion, recommending possible courses the group could attend.

Her pathway links to learning and community, playing a significant role in the promotion of ES activity.

However, this did not extend beyond attaining employment skills and becoming more active citizens

Table 1.4 Parent 1 and her Children’s Pathway




& Learning





Community will have access to ES provision &

learning





Pupils will develop transferable skills and gain

confidence

Pupils will gain new friendships that will assist

transition


achievement

Pupils will increase attendance and punctuality



Pupils will develop good relations with school and

staff





Parents will gain

confidence and self esteem






school


Community will become more active and provide further opportunities with

the school

Community will access volunteering opportunities

with the school

Parents and community will attain employment skills and become more

active citizens

Parents will gain access to employment. Children will become

more engaged in schooling




& Learning






learning





Pupils will develop transferable skills and gain

confidence


transition


achievement

Pupils will increase attendance and punctuality




staff





Parents will gain







school



the school


with the school


active citizens



Parent 2

A single parent requiring additional support for child care when first engaging with the school.

After witnessing changes in her own children, she decided to volunteer

After a number of years, gained the skills and experience to become a member of staff.

‘I’ve been here about 6 years, and I absolutely love it. I really do.’

(Play scheme worker 11/11/10)


Parental Engagement & Empowerment

Community Engagement & Learning







gain confidence


transition


achievement


punctuality




staff




Parents will gain





school



the school


with the school


active citizens



Pupil Welfare & Support

Table 1.5 Parent 2 and Child Pathway

Community Member 1 A female pensioner from a white

British heritage. She was single and lived alone. Through the cluster school she had heard of the cookery class and signed up.

‘most days I would be… say…, at home looking for something to do. It can get very lonely…and you don’t want to cook just for yourself… With this I can get to see other people, young people and bake things that I wouldn’t do for just myself.’

(Community Member 11/11/10)



Community will have access to ES provision

& learning




school


Community will become more active and provide

further opportunities with the school

Community will access volunteering

opportunities with the school

Parents and community will attain employment skills and become

more active citizens

Parents will gain access to employment. Children will become more engaged in

schooling

Table 1.6 Community Member 1 Pathway

Community Member 2 Community Member 2 had

been made redundant in his late 50’s and wanted to change his career.

He had enrolled on a graphics design course at the local college on the school site.

After successfully completing the course he volunteered his services to the community café.

Through the community links with the Local Authority, he designed graphics and posters for the Local Authority.




learning



Community will build relationship with the school



the School Cluster


with the School Cluster


active citizens

Parents and community will gain access to employment.

Table 1.7 Community Member 2 Pathway

What have we learnt

ES can have an impact but effect can ‘wash out’ if not maintained

Differences ‘reinforced’ by variation between schools and gaps in provision

New approaches, new defined roles Outcomes must be defined and shared by

all (institution, agency, parent and community)

Future Possibilities

Harlem Children's Zone project pipeline*

*http://www.hcz.org/

Harlem Children Zone Project The goal is to create a "tipping point"

in the neighbourhood.

The HCZ Project expanded from 24 to 60 blocks, now to nearly 100 blocks of Central Harlem.

For children to do well, their families have to do well. And for families to do well, their community must do well.

The HCZ Project are provided to children and families absolutely free of charge.

Thank You

References BALL, M. (1998) School Inclusion: the School, the Family and the Community, York, Joseph Rowntree

Foundation. CONNELL, J., KUBISH, A., SCHORR, L. & WEISS, C. (1995a) New Approaches to Evaluating Community

Initiatives: Concepts, Methods and Context, Washington DC, The Aspen Institute. CONNELL, J. P., ABER, J. L. & WALKER, G. (1995b) ‘How do Urban Communities Affect Youth? Using Social

Science Research to Inform the Design and Evaluation of Comprehensive Community Initiatives’. IN CONNELL, J. P., KUBISCH, A. C., SCHORR, L. B. & WEISS, C. H. (Eds.) New Approaches to Evaluating Community Initiatives, vol. 2, Theory, Measurement and Analysis. Washington, DC, Aspen Institute.

CONNELL, J. P. & KUBISCH, A. C. (1998) Applying A Theory Of Change Approach To The Evaluation Of Comprehensive Community Initiatives: Progress, Prospects, And Problems, Washington DC, Aspen Institute.

CONNELL, J. P. & KUBISCH, A. C. (1998 ) Applying a Theory of Change Approach to the Evaluation of Comprehensive Community Initiatives: Progress, Prospects and Problems. IN FULBRIGHT-ANDERSON, K., KUBISCH, A. C. & CONNELL, J. P. (Eds.) Theory, Measurement and Analysis, New Approaches to Evaluating Community Initiatives 2. Washington, DC, The Aspen Institute.

CUMMINGS, C., TODD, L. & DYSON, A. (2003) Extended schools pathfinder evaluation: issues for schools and local education authorities. DfES research brief. London.

CUMMINGS, C., TODD, L. & DYSON, A. (2004) Evaluation of the Extended Schools Pathfinder Projects. London, Department for Education and Skills.

DAVIES, R. (2004) Scale, Complexity and the Representation of Theories of Change (Part I). Evaluation . 2004. DFEE (1999) Schools Plus: Building Learning Communities. Improving The Educational Chances Of Children And

Young People From Disadvantaged Areas. A Report From The Schools Plus Policy Action Team 11. DFES (2002) Extended Schools: Providing Opportunities and Services for All. London, DfES. DFES (2003a) Every Child Matters. London, The Stationery Office. DFES (2003b) Full Service Extended Schools: Requirements and Specifications. London, DfES. DFES (2003c) Full-Service Extended Schools Planning Documents. London, DfES. DFES (2004) Extended Schools prospectus, Access to opportunities and services for all London, DfES. DFES (2005) Extended Schools: Access to Opportunities and Services for all. A Prospectus. London, DfES. DFES (2007) Evaluation Of The Full Service Extended Schools Initiative. Final Report. DONALDSON, S. I. & GOOLER, L. E. (2003) Theory-Driven Evaluation in Action: Lessons from a $20 million Statewide Work and Health

Initiative. Evaluation and Program Planning 26, 355-66. DRYFOOS, J. (1994) Full-Service Schools. A revolution in health and social services for children, youth and families, San Francisco,

Jossey-Bass. DRYFOOS, J. G. (1998) A Look at Community Schools in 1998. National Center for Schools and Communities. New York.

References DRYFOOS, J. G. & MAGUIRE, S. (2002) Inside Full-Service Community Schools, Corwin Press. FOUNDATION, W. K. K. (2000) Logic Model Development Guide: Using Logic Models to Bring Together

Planning, Evaluation and Action. Battle Creek, MI, W. K. Kellogg Foundation. HANAFIN, J. & LYNCH, A. (2002) Peripheral Voices: Parental involvement, social class, and educational

disadvantage. British Journal of Sociology of Education, 23, 35-49. HIRSCH, D. (2007) Experiences of poverty and educational disadvantage. Joseph Rowntree Foundation. York,

North Yorkshire, UK. Retrieved, 27. MACHIN, S. (2006) Social Disadvantage and Education Experiences. OECD Social, Employment and Migration

Working Papers. Paris, OECD. MORRIS, H. (1925) The Village College. Being a Memorandum on the Provision of Educational and Social

Facilities for the Countryside, with Special Reference to Cambridgeshire. London, infed. RAFFO, C. & DYSON, A. (2007) Full service extended schools and educational inequality in urban

contextsâ€”new opportunities for progress? Journal of Education Policy, 22, 263-282. RAFFO, C., DYSON, A., GUNTER, H., HALL, D., JONES, L. & KALAMBOUKA, A. (2007) Education and Poverty.

Joseph Rowntree Foundation. York, University of Manchester. WEISS, C. (1987) Where Politics and Evaluation Research Meet. IN PALUMBO, D. (Ed.) The Politics of Program

Evaluation. Newbury Park, CA, Sage. WEISS, C. (1995) Nothing as Practical as Good Theory. Exploring Theory-based Evaluation for Comprehensive

Community Initiatives for Children and Families. IN CONNELL, J. P., KUBISCH, A. C., SCHORR, L. B. & WEISS, C. H. (Eds.) Concepts, Methods, and Contexts, New Approaches to Evaluating Community Initiatives 1. Washington, DC, Aspen Institute.

WEISS, C. (1997a) How Can Theory-Based Evaluation Make Greater Headway? . Eval Rev 501. WEISS, C. (1997b) Theory-Based Evaluation: Past, Present, and Future New Directions for Evaluation 76 41-55. YIN, R. (2003) Case Study Research, London, Sage Publications.