Effective Schooling in Ethiopia: Who

Benefits?

Caine Rolleston and Zoe James

UCL Institute of Education

• Analysis of YL data as part of World Bank Public Expenditure Review for Education

– YL data alongside EMIS, SDI etc.– Focus on efficiency and effectiveness of

spending in Ethiopia’s education sector– Efficiency includes attendance, progression – Effectiveness focuses on outcomes

INTRODUCTION

YOUNG LIVES SITES ETHIOPIA

• 2012-2013 school year• Site-level school census• All pupils in all G4 & G5 classes

• Start and end of year survey (W1 & W2)• Child, class, teacher, principal and school data• Assessments in maths and reading comprehension

• Allows identification of learning progress over school year

ETHIOPIA SCHOOL SURVEY 2012-13

SCHOOL SURVEY SAMPLE• 7 regions• 30 sites• 94 schools

• 19 non-government• 280 classes

• 142 grade 4• 134 grade 5

• 11,982 pupils at Wave 1• 10,068 pupils at Wave 2

CONTEXT: STRONG PROGRESS IN TERMS OF ON-TIME ENROLMENT & PROGRESSION

  Enrolled in 1st cycle primary by age 7

Older Cohort(born 1994/95)

Younger Cohort(born 2000/01)

Total 27.7 50.0

Location Urban (R2) 43.2 71.2

Rural (R2) 18.2 39.5

Poverty Least poor quintile (R2)

47.2 74.7

Poorest quintile (R2)

15.8 32.4

BUT…. ISSUES PERSIST IN TERMS OF GRADE REPETITION, DROP-OUT AND PUPIL

ABSENCE  Has dropped out, %

Absence

Total 17.3 4.6Pastoralist livelihood

Pastoralist 24.0 6.9Non-pastoralist 17.2 4.4

Location Urban 16.1 3.7Rural 20.4 7.1

Poverty Least poor quintile 13.0 3.8

Poorest quintile 19.2 6.9

Pastoralist children, rural children, and poor children are particularly disadvantaged, raising serious equity concerns

ASSESSMENT DESIGN AND COMPETENCY LEVELS• Development of test items in relation to MLCs & textbooks (maths

and reading)

• Inclusion of items from grades 1-3 plus 4 & 5 (all MC)

• Competency levels arrived at using empirical difficulty of items in test data, and grade-level expectations – linking norm-referenced (IRT) scores to criterion referenced scale

• Items allocated to competency levels 1-4 with level 0 denoting a score below level 1. Level 4 is ‘grade-expectation’ level

• Pupils who score correctly on 2/3 of items in a particular competency level are considered to be at that level providing they also reach required competency of levels below this

DISTRIBUTION OF W2 MATHS SCORES IN SELECTED SITES

0.0

01.0

02.0

03D

ensi

ty

300 400 500 600 700 800Score

Addis Amhara Urban (Typical)SNNP Rural Somali RuralSomali Jijiga Afar Rural

Maths Scores R2: Selected Sites

0.0

02.0

04.0

06D

ensi

ty

300 400 500 600 700 800x

Below Level 1 Level 1Level 2 Level 3Level 4

Maths Scores (Wave 2) by Competency Level

DISTRIBUTION OF NORM-REFERENCED (IRT) SCORES BY COMPETENCY LEVEL IN MATHS

Competency Level

Grade Level

Level Description Reading Comprehension Competencies

0Below Early Foundational

N/A Below early foundational reading comprehension

Pupils have not reached level 1

1Early Foundational

1 Early foundational reading comprehension

Pupils can read and understand simple familiar words and very short simple sentences illustrated by pictures.

2Foundational 

2 Foundational reading comprehension

Pupils can read and understand short sentences without illustration and to complete the meaning of sentences with simple missing words. Pupils can read a short passage with basic understanding of simple events.

3Emerging Basic 

2-4 Emerging basic reading comprehension skills

Pupils can read longer sentences and passages containing some less familiar words, with an understanding of simple events and characters.  Pupils have a basic and emerging ability to interpret events and characters.

4Basic 

4-5 Basic reading comprehension

Pupils can read passages with understanding of the main points communicated and a basic understanding of intention and purpose. Pupils have an emerging ability to use inference and deduction to interpret texts in terms of message.

Competency Level

Example Mathematics items

0 - Below Early Foundational1- Early Foundational

- How many dots are there? - Which is a triangle? - 9 X 2 = - 2 + 3 =

2Foundational 

- Put numbers in ascending order: 19, 6, 2, 11- How many minutes in 1 hour?- Tamiru has  5 Birr. His mother takes 4 Birr. How many Birr does Tamiru have?- Which is half of 6?

3Emerging Basic Numeracy

- 85 x 5 = - Which difference is closest to 300,000?- 30m = ____ cm- What is the value of the number 2 in 928?

4Basic Numeracy

- 2.34 +7.65 = - What is the average of 10, 12, 18, 24?- Calculate the perimeter of the rectangle- Fill in the appropriate number in the sequence: 1, 3, ___, 27

COMPETENCY LEVELS ARE LOW - MATHSCompetency level Number of Pupils Percentage of 

PupilsCumulative Percentage

0 (Below Level 1)  853 8.66 8.661 (Early Foundational) (~ G1) 2,121 21.54 30.212 (Foundational) (~G2-3) 5,152 52.33 82.533 (Emerging) (~G3-4) 1,473 14.96 97.494 (Grade level) (~G4-5) 247 2.51 100.00Total 9,846 100.00

020

4060

8010

0pe

rcen

t

Rural Urban5 4 5 4

Level 0 Level 1Level 2 Level 3Level 4

COMPETENCY LEVELS ARE ALSO LOW IN READING

020

4060

8010

0pe

rcen

t

Rural Urban5 4 5 4

Level 0 Level 1Level 2 Level 3Level 4

Competency level Number of Pupils Percentage of Pupils Cumulative Percentage

0 (Below Level 1) 466 6.05 6.051 (Early Foundational) 1,437 18.67 24.722 (Foundational) 840 10.91 35.643 (Emerging) 4,902 63.69 99.324 (Grade level) 52 0.68 100.00Total 7,697 100.00

DO PUPILS IN GRADE 4 AND GRADE 5 MAKE PROGRESS OVER THE SCHOOL YEAR?

Mathematics Reading Comprehension

October 2012

May 2013

Gain October 2012

May 2013

Gain

Mean 

500.0 530.0 30.0 500.0 530.6 30.6

Gender  

Boy 502.1 532.5 30.4 498.0 527.5 29.5

Girl 497.9 527.6 29.7 501.9 533.6 31.7

Difference 4.2 4.9 0.7 -3.9 -6.1 -2.2

Location  

Urban 517.6 548.5 30.9 521.1 551.1 30.0

Rural 456.4 481.2 24.8 447.3 476.6 29.3

Difference 61.2 67.3 6.1 73.8 74.5 0.7

WHILE SCHOOL AND CLASS FACTORS CONTRIBUTE TO LEARNING, PUPILS BACKGROUNDS AND HOUSEHOLD

CHARACTERISTICS REMAIN VERY IMPORTANTVARIABLES Maths T1 Maths T2 Reading T1 Reading T2Girl -10.2031 *** -4.4330 *** 0.1650 4.8107 ***Has 3+ meals per day 18.2975 *** 6.6820 *** 12.6032 *** 2.1339PCA pupil durable assets 4.5648 *** 1.7210 *** 4.9427 *** 0.8929 *% days absence W1-W2 -3.3064 *** -1.8387 *** -3.1255 *** -1.4661 ***Orphan (single or double) 2.8872 -3.2279 * 2.6308 -3.3137 **No-one in household literate -9.1339 ** -5.7685 * -14.3049 *** -1.8481Attended pre-school 2.6854 -0.3207 6.3172 *** 3.3394 **Ever repeated a grade -39.6920 *** -5.0135 *** -38.7614 *** -4.2894 ***Ever dropped-out -6.1516 ** -2.1293 -13.4413 *** -3.7113 **Reads books at home 17.5734 *** 6.2704 *** 17.3460 *** 3.6314 *Child learns in home language 3.8083 -2.2792 14.5838 *** 5.4488 *Pastoralist -24.1437 *** -4.3839 -37.9640 *** -8.0455 **Pupil spends time on paid work -10.4830 *** 0.2447 -12.1998 *** -1.0128

Pupils  backgrounds  and  household  experience  continue  to influence  their  learning  progress  (T2),  in  addition  to  their achievement at the start of the school year, and overcoming these factors is a significant challenge to effective schooling. 

ARE ALL SCHOOLS EQUAL IN ADDING VALUE IN TERMS OF PUPIL LEARNING

IN MATHS?

• In maths urban schools do not add more value than rural schools

-100

-50

050

100

Sch

ool V

alue

-Add

ed

0 20 40 60 80 100School Rank

Urban Rural

IN READING –VARIATION BETWEEN RURAL AND URBAN SCHOOLS

• In reading, urban schools do have an advantage, likely linked to languages of instruction

-100

-50

050

100

Sch

ool V

alue

-Add

ed

0 20 40 60 80 100School Rank

Urban Rural

SCHOOL, CLASS AND TEACHER FACTORS ASSOCIATED WITH HIGH AND LOW ‘VALUE-

ADDED’ CLASSESCharacteristic Maths Reading

High VA Low VA Sig High VA Low VA Sig

School teaches only shift classes 0.58 0.73 * 0.49 0.87 ***

Class Assets -0.01 -0.40 * -0.06 -0.30

Class has electric light 0.45 0.26 ** 0.60 0.24 ***

Class has radio 0.39 0.12 *** 0.27 0.13 *

Teacher score on maths test % 63.13 55.79 ***

Teacher education Post-Sec Dip 0.46 0.50 0.49 0.30 **

Teacher education University 0.15 0.03 ** 0.21 0.01 ***

www.younglives.org.uk

FINDING OUT MORE