OPEN AND DISTANCE LEARNING TEACHER

TRAINING PROGRAMME-MALAWI

IMPACT EVALUATION DESIGNSHORT TERM STRATEGY FOR IMPROVING STUDENTS’ LEARNING OUTCOMES

THROUGH REDUCING PUPIL QUALIFIED:TEACHER RATIOS

Presentation by: The Malawi team

APEIE Workshop,

Accra,Ghana

14 May 2010

Vision statement

Improve students’ learning outcomes through the reduction of pupil:qualified teacher ratios

Teacher training through Open and Distance Learning one of the means for reducing pupil:qualified ratios

Goal and Objective of Evaluation To compare the quality of teaching using ODL

teachers vs. Conventionally trained teachers. Quality of teaching, which is probably best measured,

in an objective way, by standardized tests scores. Test scores to be measured regularly using

standardized tests,not only in standard/grade 8, but also in lower standards/grades, in particular grades 3-7.

Other measurable outcomes to include as enrollment and attendance .

Today’s situation

5,462 primary schools (EMIS 2008). 3.6 million students (EMIS 2008) and projected

enrolment of 4.8million students by 2017. 3,500 teachers trained annually through conventional

means(1+1 or commonly called IPTE) since 2007. High pupil :qualified teacher ratio(90:1), worse in

some districts 135:1or above. 45,000 primary school teachers currently employed. Preferred pupil :teacher ratio of 60:1 by 2013.

Today’s situation(2) Objective of ODL teacher training project is to have a group of

16,000 teachers over 3 years (29/34 education districts). ODL teachers more attached to the community and therefore

potentially more stable and motivated. ODL teachers oriented on pedagogical skills for 3 weeks

before teaching..2.5 years on the job training plus distance learning.

IPTE teachers trained for 1 year (face to face) then 1 year teaching practice in schools.

Both IPTE and ODL teachers recruited via adverts and aptitude tests (age range-19 to 35 years).

Impact evaluation design Idea of randomizing ODL teachers across schools more appealing but

a key practical issue with that approach. Randomization at the school level, by which schools will be allocated

randomly between “Type 1 schools” and “type 2 schools”. Type 1 schools would assign ODL teachers to grades 3, 5 and 7 and

NEW conventionally trained (IPTE) teachers to grades 4 and 6. Type 2 schools would assign ODL teachers to grades 4 and 6 and

NEW conventionally trained (IPTE) teachers to grades 3, 5 and 7. Strict enforcement of teacher placements in schools taking part in the

evaluation for the entire length of the evaluation.

School level randomisation

Std. Type 1 schools Type 2 schools

1

2

3

4

5

6

7

8

Experienced teacher

Experienced teacher

Experienced teacher or ODL

Experienced or IPTE teacher

Experienced teacher or ODL

Experienced or IPTE teacher

Experienced teacher or ODL

Experienced teacher

Experienced teacher

Experienced teacher

Experienced or IPTE

Experienced teacher or ODL

Experienced or new IPTE

Experienced teacher or ODL

Experienced or IPTE teacher

Experienced teacher

Data collection

One important issue with the suggested impact evaluation-the absence of a baseline.

Options to alleviate this issue.1. EMIS data, collected on a routine basis every year, will provide

useful information on variables such as enrollment, teacher qualifications, infrastructure, etc.

2. Possibility to use results from the primary school leaving exam at the end of grade 8 as a proxy for overall school quality.

3. Possibility to use some of the standardized tests administered to a sample of schools in Malawi in grades 2, 3, 5 and 7 in 2008.

Data collection(2) To administer a standardized test EARLY (September

preferably, maybe October) at the beginning of the 2010-2011 school year.

To have an “almost baseline” in September-early October 2010,

Yearly testing, in grades 3-7, at the end of each school year (May?).

Computerized enrollment and attendance records and precise, computerized information about the allocation each teacher (ODL and new conventionally trained) to each school and each grade.

Length of evaluation IPTE (new conventionally trained) teachers start their

job fully trained while the ODL teachers will start teaching at the same time as they will start their training.

On the job distance learning for ODL teachers to take 2 ½ years. It is only after that period that it would be fair to compare ODL and newly qualified IPTE teachers .

3 batches of ODL teachers, 2010, 2011 and 2012. Implication: evaluation should continue at least until the 2015-2016 school year.

Length of evaluation(2)

School Year

2010-11 Sep.–Oct.2010

2010-2011 May 2011

2011-2012 May 2012

2012-2013 May 2013

2013-2014 May 2014

2014-2015 May 2015

2015-2016 May 2016

Standardized tests

At least 2 subjects in grades 3-7

At least 2 subjects in grades 3-7

At least 2 subjects in grades 3-7

Potentially more extended testing, grades 3-7

More extended testing, grades 3-7

More extended testing, grades 3-7

More extended testing, grades 3-7

Thank you