Understanding ‘Learning’ in Pakistan: Building a Comprehensive Picture of

School Functioning in PakistanMonazza Aslam, Centre for the Study of

African Economies (CSAE), Dept. Of Economics, University of Oxford, UK and Idara-Taleem-o-Aagahi, Lahore.

Education is Key...• “Education is central to development and a key

to attaining the Millennium Development Goals. It is one of the most powerful instruments for reducing poverty and inequality….” (The World Bank);

• Access to education is high on the development agenda;

• Desire to achieve MDGs has galvanised policymakers worldwide into action.

• In many countries progress in school enrolment is phenomenal.

‘Learning’ is critical!

• It is now a well known fact – it is not years of schooling that matter to important outcomes but what is actually learnt that matters.

• Hanushek and WoBmann (2007) show that differences in learning achievements matter more in explaining cross-country differences in productivity growth than differences in the average number of years of schooling or in enrollment rates.

• An effective strategy should focus not solely on sending more children to school but also on enhancing the quality of schooling.

• Hanushek and WoBmann (2007) show using PISA surveys that disparities in secondary education between developing countries and OECD countries are even larger when one considers not only access but also learning achievements.

• Things are not much better at the primary level. In recent surveys in Ghana and Zambia, fewer than 60 percent of young women who complete six years of primary school could read a sentence in their own language.

‘Learning’ is critical for:

• growth and for pro-poor growth;• For poverty reduction;• For reduced inequality;• Fertility, population control;• For access to more lucrative occupations;• Higher earnings during the lifetime;• Women’s empowerment;• Children’s health outcomes and hence future life

outcomes.

Schooling and ‘Learning’ in Pakistan

• While Pakistan has made substantial headway recently in achieving some dimensions of the ‘Education for All’ Millennium Development Goal (MDG2), it still has a long way to go.

• although the net primary enrolment rate increased from 45% in 2002 to 54% in 2006 (World Bank), it is pitiably short of universality.

• More importantly, the MDG doesn’t focus enough on ‘learning’ outcomes.

But...

• Enrolment is the very first rung of the ‘access’ ladder. • Meaningful access requires – in addition to school enrolment

–good attendance rates, timely progression through grades, and mastery of basic cognitive skills.

• Not all enrolled children attend school regularly • not all who attend regularly complete primary schooling;• not all who complete primary schooling acquire adequate

learning to be literate and numerate. • Stable and regular attendance of both students and teachers

is necessary for learning.

What ‘needs’ doing according to common knowledge

• Improve curriculum;• Improve teaching;• Improve facilities for ‘learning’;

But research on most aspects is limited.

What do we know so far?(will talk about work I am associated with)

PREVIOUS RESEARCH IN PAKISTAN SHOWS:1. We know that students in private schools may achieve more and that girls may be

worse off in terms of access to private schools;2. We also know potentially how student A may be targeted through improving weak

teaching;3. ASER (2008) – builds a picture of basic learning and tells us that 65% 5-16 year olds

in 11 districts in Pakistan cannot read standard 2 story level text and more than 70 % cannot do two digit division (again standard 2 curriculum).

4. We also know that there are striking gender inequalities in occupational choice and in earnings;

5. Finally, we also know that the labour market rewards women’s ‘skills’ substantially.

If today's children are to become tomorrows earning adults, we need to know how they are learning today to ensure they are able to achieve their full potential

tomorrow.

The private schooling and learning story

• Aslam (2009), ‘The Relative Effectiveness of Government and Private Schools in Pakistan: Are Girls Worse Off?”, Education Economics, Vol 17 (3), pp. 329-353.

• If you look at raw PIHS data, girls are found more likely to be enrolled in private schools.

• But when we condition on household observables/unobs. Picture changes! BOYS are more likely to be enrolled in private schools.

• A major channel of lower educational expenditure on girls schooling than boys is because boys are more likely to be sent to private school;

• Although average cost of private schools low, relatively more expensive than government school.

• Are girls condemned to poor ‘quality’ schooling?

Achievement Data used to answer the question: Are girls condemned to poor quality schooling in Pakistan?

• Data – one district (Lahore);• School-based sample survey of 65 schools (40

pvt, 25 govt);• Data collected in 2002-2003;• Detailed info on 1800+ students, 300+

teachers – 8th grade students;• Tests administered – literacy, mathematics,

ravens (discuss);

Raw and standardised Achievement, Reading and Maths scores by school type

Hence girls lose out in terms of the quantity and quality of education acquired! Achievement data helps us identify a major inequality.

• Estimates based on School survey conducted by author in 65 schools (40 private and 25 government) in Lahore district in 2002-2003.

• A random student with average characteristics would score 9.63 points in maths if in private school compared to 8.10 points in government school (19 % higher)

• Predicted reading score of student in private school 8 % higher;

What is it about teachers that matters for student achievement?

• Aslam and Kingdon (2008, RECOUP Working Paper 19, under review). ‘What can teachers do to raise pupil achievement?’

• This study probes the ‘black box’ of teaching to explore the teacher characteristics that impact student learning;

• How important are the teaching practices adopted by teachers in determining student achievement?

What the study does…• This study takes the direct approach linking teacher characteristics

to student outcomes in an achievement production function, but with two innovations.

• Firstly, we use a methodology that overcomes the biases associated with the non-random matching of students to schools and, within schools, to teachers.

• Secondly, we also test the importance of classroom practices and teaching techniques rather than confining attention only to teachers’ resumè characteristics i.e. qualifications, experience and training.

Teacher characteristics data• Standard teacher chars such as age, gender, certification,

training, tenure, experience etc.• ‘Teaching methods and techniques’ information gathered by

asking each individual teacher to answer a series of questions such as record minutes/week spent on average in giving surprise tests/quizzes, maintaining discipline and order, writing material on the black board etc.

• Teachers also given series of questions on use of lesson plans, whether they read out loud from books, how well they engaged students when teaching etc.

Methodology• Achievement production function – ‘direct approach’• Estimate: Aik = α + βXik + δSk + μi + ηk (1)

• Where i = individual and k = school• X = individual and family characteristics• S = school and teacher characteristics• μi = individual-level unobservables• ηk = school-level unobservables

Summary of results• Resume’ – i.e. teacher qualification, training, etc. - characteristics of

teachers don’t matter to student learning – highly controversial claim but supported by studies from India, USA etc.;

• Girls benefit from being taught by female teachers; Why? 1) Preferences hypothesis...same-sex teachers prefer same-sex students , 2) stereotypes influence teachers evaluations; We gave tests and marked them so can rule out the ‘evaluation’ stemming from these 2 but can’t rule out possibility that knowledge transmission not affected by either route.

• Teaching ‘process’ – lesson-planning, quizzing and involving them in classroom discussions clearly matters to achievement.

G and P school differences• We know from recent literature that private schools more

‘effective’ than govt. schools (World Bank researchers, Alderman’s work etc. on primary and my work on Middle school students).

• Why more effective?- Private school student body superior quality?- Infrastructure superior?- Teacher quality superior?- In terms of apparent observed charachtersitics, G school teachers

more certified, experienced, better paid.- Know nothing about ‘unobserved’ motivation etc. – indeed contract

stuctures in G schools generate perverse incentives.- We know nothing about ‘process’ differences or even how the

production functions differ across school types.

Conclusions of ‘Teachers’ study• Delved into the ‘black box’ representing teaching;

• Identification strategy was unique;

• Importance of class room practices;

• Separate estimates for govt. and private schools;

• Teacher unobservables still remain as source of endogeneity undermining confidence in attributing causality to observed teacher variables.

Conclusions continued…• Standard teacher resume’ characteristics have no bearing on

student’s standardised mark;

• Teacher pay schedules are ‘inefficient’;

• Girls in 8th grade benefit being taught by female teachers;

• Teaching classroom practices (lesson planning, involving students by asking questions, quizzing etc.) substantially benefit student learning.

• Private schools teachers more ‘efficient’ and use the more innovative teaching methods in imparting learning.

What does ASER (2008) tell us?

• Annual Status of Education Report (ASER).• The purpose of the ASER rapid assessment survey and its planned annual

rounds in rural areas is twofold:

To get reliable estimates of the status of children’s schooling and basic learning (reading and arithmetic) up to grade II at the district level

To measure change in these basic learning and school statistics over time- annually

• In ASER Pakistan 2008 , 16737 children and 6520 households were surveyed in the rural areas in 11 districts

How ASER is conducted?

• ASER is a citizens-led initiative conducted in a campaign mode. This means that ASER is not conducted by any single organization. Instead, ASER is done in collaboration with a variety of civil society organizations, universities, research institutions government organizations and citizens’ groups. Collaborators voluntary supporters, teachers and students devote their time and resources to conduct ASER in their region/districts.

• Pratham and ASER India, provide technical support at various stages of the survey to the ASER Pakistan.

Scale & Scope of Survey

Coverage : In all five provinces i.e. in Sindh, Balochistan, Punjab, Pakhton Khawa, Gilgit Baltistan, FATA and AJK.

Only rural areas will be sampled.

Phase I : Year I 2010 – 30 districts across Pakistan

Phase II: Year II 2011 – 70 districts across Pakistan

Phase III : Years III, IV , V all districts across Pakistan (138 districts)

Sample: 600 households per district. Two-stage sample, stratified in the first stage. Obtained by selecting 30 villages per district & 20 households per village.

  Villages will be selected randomly using the village directory of the latest Census. The Probability Proportional to Size Sampling (PPS) technique will be adopted as an appropriate one when the sampling units are of different sizes.

Focus on nuances between public and private . In each village profile of 1 govt. and 1 private school will be collected .

ASER Pakistan Assessment Tools

ASER 2008 in Pakistan

Sr # Province District

1

Punjab

Lahore

2 Sheikhupura

3 Multan

4 Jhang

5 Rawalpindi

6 Rahim Yar Khan

7 Mianwali

8 Faisalabad

Sub total 8

9 ICT Islamabad

Sub total 1

10 Khair Pur

11 Dadu

Sub total 2

TOTAL 11

ASER Pakistan 2008 only completed in 11 districts of 2 provinces: Punjab, Sindh and the Islamabad Capital Territory. Unfortunately, ASER Pakistan 2008 could not be undertaken in Baluchistan and NWFP due to the Oct. 2008 earthquake/floods & rapidly deteriorating law and order situation /conflict ..local collaborators got engaged in multiple emergency responses

Districts’ Summary ASER 2008

Sr # Province District Villages HH Children Surveyed

Schools

1

Punjab

Lahore 30 600 1,367 302 Sheikhupura 30 600 1,199 303 Multan 30 600 1,430 304 Jhang 30 600 1,823 305 Rawalpindi 30 600 1,419 30

6 Rahim Yar Khan 30 600 1,565 30

7 Mianwali 30 600 1,631 308 Faisalabad 30 600 1,631 30

Sub total 8 240 4,800 12,065 240

9 ICT Islamabad 30 600 1,232 0

Sub total 1 30 600 1,232 010

SindhKhair Pur 28 560 1,539 28

11 Dadu 28 560 1,888 19Sub total 2 56 1120 3,427 47TOTAL 11 326 6,520 16,737 287

In ASER Pakistan 2008 , 16737 children and 6520 households were surveyed in the rural areas during the survey in 11 districts

Enrolment is still a problem – children aged 5-16

-More girls are out of school – no surprise here;-Enrolment is the very first rung of the access ladder – we stumble on the very first one.

Enrolment by School Type

•Apparently a larger proportion of girls enrolled in private schools (46% versus 41% boys).•This finding is NOT different from what the PIHS data tells you when you do not condition on household observables.•If we were to condition on observables etc., I suspect ASER 2008 finding would be identical to Aslam (2009) – girls lose out in the quality of schooling available to them!

What about ‘meaningful access’?

• Good Attendance rates?• Timely progression through grade?• Mastery of basic cognitive skills?

Attendance

School withStd 1-5 Std 1-8 Others Average

Children attendance 79% 72% 72% 74%Teachers Attendance

75% 82% 76% 78%

PTR 49 39 29 39%• Teachers Attendance is over all 78% Primary level 75%;

Elementary : 82%• Children’s Attendance overall : 74%

Primary level 79%; Elementary level 72%

Think about it!

• 79% children attendance in grades 1-5 means that in an average class of 49 children, 10 children were not attending school on the day visited.

• If there are 15 teachers in a primary school, 4 out of the 15 teachers are not available on the day visited.

• Aslam (2003), ‘Determinants of Student Achievement in Government and Private Schools in Pakistan’, The Pakistan Development Review, 42:4, Part II (Winter), pages 841-876: finds that student absenteeism has a detrimental affect on student learning.

School Facilities: Provision & Use

 

Schools withStd 1-5 Std 1-8 Others

Number of Schools Visited 220 44 23

Average No. of Rooms Available for Classes 4 6 9Average No. of Rooms Used for Classes 3 5 7Water Facility (working) % 69% 77% 74%Water Facility (not working / not Available) % 31% 23% 26%Toilet Facility (Working) % 58% 73% 71%Toilet Facility (not working/ not available) % 42% 27% 29%

Primary Schools•31% schools at primary level are without water facility, •42% schools are without toilet facilityElementary Schools•23% schools at Elementary level are without water facility, •27% schools are without toilet facility

What about mastery of basic cognitive skills?

• Standard 2/3 instruments designed and all children between 5-16 years tested.

• Children found to be performing way below curricular standard.

Standard 2 Reading ability of 3-16 year olds

Of the 16, 737 children tested, only about 30% are able to read a story with fluency in Urdu or local language.

Standard 3 Mathematics ability of 3-16 year olds

Of the more than 16,000 children tested, only about a quarter can do double digit division.

Learning Levels (Reading)Reading (%)

AgesBeginning /

NothingLetter Word Para Story Total

05-0925 24 23 15 12 100

(Class 1-5)10-12

7 9 15 21 49 100(Class 6-7)

13-147 3 8 15 66 100

(Class 8-9)15-16

8 2 6 11 73 100(Class 10-11)

05-16 16 15 17 17 35 100

36

Learning Ability of the Age Group 5-9 Years: •27% of the children read para or level - I text •12% of the children in this age group read story text level-II

Learning Ability of the Age Group 10-12 Years: •70% of the children in this age group can read para-or level 1 text •49% of the children in this age group can read story or level II

Learning Ability of the Age Group 15-16 Years: •84% of the children read para or level - I text •73% of the children in this age group read story text level-II

Learning Levels (Arithmetic)Arithmetic (%)

Age GroupBeginning /

NothingNumbers

1-9Numbers

11-99Subtraction Division Total

5-0926 25 29 14 6 100(Class 1-5)

10-1213 8 21 26 32 100(Class 6-7)

13-148 3 12 18 58 100(Class 8-9)

15-16

9 3 9 15 64 100(Class 10-11)

05-16 18 14 22 19 27 100

37

Learning Ability of the Age Group 05-09 Years: •20% do subtraction or Level-I(arithmetic) •6% of the children in this age group solve division (level-II)Learning Ability of the Age Group 10-12 Years: •58% do subtraction or Level-I(arithmetic) •26% of the children in this age group solve division (level-II)Learning Ability of the Age Group 15-16 Years: •79% do subtraction or Level-I(arithmetic) •64% of the children in this age group solve division (level-II)

Learning - National

Age GroupWho Cannot Read (%) Who cannot do Arithmetic (%)

Paragraph Story Subtraction Division

5-9 72 88 80 94

10-12 30 51 42 68

13-14 19 34 23 42

15-16 16 27 21 36

5-16 49 65 54 73

38

Going back to the ‘WHY’ in Pakistan

• Research further shows that even the most basic literacy and numeracy skills among adults in Pakistan:

1. Improve women’s access to more lucrative occupations;

2. Increase earnings for both men and women once in the labour market;

3. Reduce fertility;4. Improve mortality outcomes;5. Increase agricultural productivity.6. Etc.

Measure ‘basic’ skills

• Focus on ‘Public/Private’ /other school types differences;

• Rural/Urban divides;• Gender differences;• Socio-economic groups;• School/village location;• Individual level data – allows us to calculate

achievement differences b/w different groups;

We also need to go BEYOND existing efforts

We come back full circle:• Many children still not enrolled in school;• Not all enrolled children attend school regularly;• Not all who attend regularly complete primary schooling• finally not all who complete primary schooling acquire

adequate learning to be literate and numerate. • Stable and regular attendance of both students and teachers

is necessary for learning (among other factors)

Pakistan needs more research

• In a new project I am proposing to carry out (with ITA collaboration) two (linked) unique surveys to enable a better understanding of the constraints on (a) children’s schooling participation and (b) student learning, in Pakistan.

• Apart from testing children at two points in time in order to measure gain in achievement, the survey will measure factors that might be behind poor child learning but which have hardly received research or policy attention thus far, e.g.:

• (i) teacher effort (attendance rate, time-on-task);

• (ii) teacher competence to teach material in the primary texts (teacher’s knowledge, ability to explain and ability to spot the mistakes that children make); and

• (iii) aspects of school functioning such as existence of/ adherence to a school/class time-table; stability/flux in multi-grade class configurations in the school over the year; schooling resources and teacher characteristics; and the extent of community involvement in the school, inter alia.

• The proposed school survey will be unique in Pakistan because it will visit each sample school four times in the school year. This will permit observing the dynamic functioning of schools, in contrast to a snapshot picture.

Then what?

• Planned interventions can be undertaken once we know the areas that need targeting;

• Informed national policy decisions;• Align classroom curricula with actual

instruction in class;• Motivate the masses – increase public

awareness, mobilize communities, parents, educators, policymakers into ACTION!

Importance of having this disaggregated data

• Identify the schools/students/areas/groups in most need of educational policy targeting;

• Guided decisions on how to allocate resources/mobilize resources;

• How to tailor curricula and teaching to specific needs of different groups;