Maths & Science Community of Practice

Key Learnings from social investment interventions in maths and science

2 June 2015

BRIDGE’s role in supporting systems performance

A combination of:

Reviewing, documenting and segmenting

knowledge shared in reflective engagements

(communities of practice)

Reviewing, documenting and segmenting

knowledge shared through research and

evaluations

Developing integrated frameworks, systems, processes, tools, guidelines

How can our system improve maths and science performance?

BRIDGE has identified the drivers supporting success.

Working practice

(from

communities of

practitioners)

Gauteng Education

Development Trust

36 evaluations

7 projects

Thematic lessons in

repository of learnings and

tools

National Education

Collaboration Trust

Evaluations, reports and interviews

A Framework

for Success

Guidelines for the sector

Dimensions of the issue

Where is the Return on Investment? Government, private sector and donors have invested huge amounts of time, energy & money on Maths & Science with little (apparent) ROI in terms of learner outcomes.

Crisis in Maths & Science education Quality of passes Quality of teaching Dropping enrolments Drop out rate Gateway and foundation

subjects: impact on employment and skills development

Problem Areas

• Issues in Foundation Phase (FRF Chairs)

• Learners unable to transition from unit counting to abstract methods (FRF chairs)

• Transition points: uneven levels on entry into High School

• Multigrade classrooms [Dinaldi schools]: • Lack of CAPs coverage

• Falling further behind as you go up the pipeline

• Teacher content knowledge & PCK (FRF; JET ITE)

• Language (NEEDU): • Transition point in medium of instruction

• Learner understanding

• Teacher language

Illustrative Projects

Government Private

Dinaledi Schools Project FRF Maths Chairs (6)

GPLMS Public Schools Maths Challenge

WCED Lit Num Primary Maths Research Project

SSIP Maths Leadership Development Programme

GEDT: DIPIP Primary Science Programme

GEDT: 3 small ICT projects Science Education Project

GEDT: Maths & Science Grants RADMASTE

Projects: Interventions with specified inputs and outcomes

Themes from general literature

Need for an explicit theory of change

Context is key: no one size fits all

Identify ‘performance stage’ [poor/fair/good/excellent]

Targeting under-skilled teachers

“…. systems would do well to learn from those at a similar stage of the journey, rather than those that are at significantly different levels of performance.” (McKinsey 2012).

“Scaffolding that provides guidance on both what teachers should teach and how they should do it has proven effective in enhancing the skills of low-performing students.” (NBER 2014) [GPLMS]

General themes contd.

Unexamined assumptions/ theoretical starting points (e.g. re: how kids learn; acquisition of reading skills; pedagogical practices ….)

Approaches to maths education (e.g. importance of language; role of streaming; error analsyis; PCK ….)

Perceptions & confidence i.r.o maths

Importance of evaluations & meta-evaluations

7 Critical Drivers

CRITICAL DRIVERS FOR MAXIMISING IMPACT

Intervention Areas

Systemic Impact

Going to scale

Sustainability

Monitoring & Evaluation

Collaboration

Cost Effectiveness

Intervention Areas

Systems scope

National, provincial, district, cluster, school type: under-performing schools, high-performing schools, rural, urban, township, special needs schools …..

Phase / Level Foundation phase, Intermediate phase, Senior phase or FET phase; multigrade or a specific grade ….

Project Type Leadership or school management team development, teacher development (ITE or INSET), learner support, physical or materials resources (including ICTs), whole school interventions, or a combination of these elements …..

Focus Some examples: methodology, pedagogical content knowledge, content knowledge, study skills, leadership, school management ………

Outcome improvement type

Quality of passes, numbers of passes, number of entrants, number of capacitated teachers …….

Key Learnings on Intervention Areas

• Use a conceptual or decision framework to decide on scope, level, target audience, project type, dosage and project outcomes.

• Dosage must be appropriate and must be monitored [GEDT ICT projects]

• Understand where an intervention is situated in relation to the bigger picture.

• Be informed by lessons learned from previous projects, in relation to planning and implementation of different project types. [Teacher development lessons]

Zenex 2013

Systemic Impact

The notion of ‘impact’ is related to bringing about change in school systems and sub-systems. …. interventions need to be formalised into support from national, provincial and district levels in key areas (e.g. curriculum, institutional support) in order to have long term systemic impact.

The ongoing involvement and support of a province as part of normal practice is the only way in which interventions can be judged as systemic and sustainable, as they then become part of the norm. The constant use of external facilitators or support personnel does not represent systemic impact, and is not a long-term solution.

Key Learnings on Systemic Impact

Have a thorough understanding of the components and inter-

relationships of the schooling system.

Clearly define roles and responsibilities of the different levels of

the department and other partners and stakeholders at the start of

a project.

Clearly define accountability processes.

Plan for long term institutionalization and embedding of processes

which support positive outcomes. [SSIP]

Build in handover and transfer procedures to support post-project

systemic sustainability.

Sustainability

Plan for sustainability, in that the outcomes of a successful intervention continue to be

implemented in the school/s concerned long after the

project ends.

Key Learnings on Sustainability

• Make sure that the beneficiaries see value in the intervention, so that positive attitudes and commitment support sustainability (GEDT M&Science].

• Build in leadership for continuity.

• Allow time for embedding and internalizing changes in practice. [GPLMS]

• Plan for some form of information sharing and monitoring after the project has officially closed. [PSP]

Going to Scale

Increasing the number of

beneficiaries

Increasing the number impact sites

Replicating projects in different

contexts

There is more than one way of achieving scale.

Social franchising models

Pilot and evaluate before going to scale. [FRF]

Roll out in stages, using an iterative process to understand the

success and barrier factors when a project is scaled up.

Grow leaders and champions from within the project as you go to

scale.

Don’t limit understanding of scale to increasing the number of sites

only; consider alternative models such as collaboration or

replication of small projects.

Take systemic factors and relationships into account.

Key Lessons on going to Scale

Monitoring & Evaluation

‘Clearly, ‘more of the same’ is a waste of resources, and we need to focus on how to maximise the effect of education interventions in the fields of mathematics and science. In order to do this we need more data, and an enhanced understanding of the nature of evaluation data and how to interpret it.’

Lack of evidence-based impact & evaluation data tracking what works and what doesn’t from interventions and innovations over the last twenty years.

- What is impact?

- When can it be seen?

- How do you measure it?

- How do we learn from evaluations?

Key Learnings on M&E

Bring in professional evaluation experts at the design stage.

Use the right data gathering mechanisms for monitoring and impact evaluation pre-, during and post-project.

Allocate sufficient budget for monitoring and evaluation.

Build a research component into projects.

Contribute to evaluation repositories or databases

Understand types, levels and options for collaboration.

Identify successful NGOs or other social partners with a track

record in your area of interest.

Learn from other interventions.

Share information and resources.

Key Learnings on Collaboration

Key Learnings on Cost Effectiveness

• Invest in systemic and sustainable interventions - become part of normal delivery.

• Use criteria for selection i.r.o. school functionality

• Use criteria for the appointment of implementation partners or service providers.

• Be informed: avoid repeating ineffective interventions or duplicating existing projects.

• Use good practice costing and budgeting models for financial management and project accountability.

Tools & Guidelines Guidelines on Key Lessons for:

Teacher Development interventions

Learner Support interventions

ICTs interventions

School Leadership and Management

Guidelines for the Management of Stakeholders

Guidelines for Determining School Functionality

Guidelines for Evaluation

Databases / Information Repositories

Guidelines for Collaboration

A Checklist of First Principles for School Interventions

Guidelines for Selecting Intervention Phase or Level

Guidelines on working with the Department

Facts and statistics about maths and science

Practical Tools & Guidelines for Social Investors