EVALUATING THE IMPACT OF SCHOOL EVALUATING THE IMPACT OF SCHOOL RESTAURANTS AND DEWORMING RESTAURANTS AND DEWORMING

PROGRAMSPROGRAMS ON RURAL PRIMARY SCHOOLS ON RURAL PRIMARY SCHOOLS ACHIEVEMENTS ACHIEVEMENTS

IN IN SENEGALSENEGAL

Research proposal presented toPEP - AusAid Policy Impact Evaluation Research Initiative

By :Abdoulaye DiagneFrançois J. Cabral

Amen DovoédoMarème Ndoye

Khady Kane

Dakar – 22 mai 2008

PLAN

• I. Objectives of the study• 1. Overview• 2. Most important research questions• and objectives • II. Background of the project: Literature review• III. Methodology• 1. Overview• 2. Data• 3. Modeling and tests • IV. Ethic, social, gender and environmental problems• V. Consulting strategy and dissemination of the

outcomes

I. OBJECTIVES OF THE PROJECT

- The great poverty and vulnerability of rural areas induce hunger and under-nutrition of children, which contribute largely their small enrollment rates and their high level of dropouts.

- Significant increase in the school enrollment rate, from 71,6% in 2002 to 82,5% in 2006. But a great difference exist between the regions, and rural areas, particularly, experienced the smallest progress.

I.1. Overview

- Important progress in primary education in Senegal since the beginning of the new century, due to the so called « PDEF » ( Programme Décennal pour l’Education et la Formation).

- The Government is aware of demand side constraints of rural education, and tries to solve the problem by experimenting schools restaurants where free breakfasts are served, and intestinal deworming with the help of international organizations like World Food Program or NGOs.

-Reports released by the Ministry of Education and NGOs reveal the positive impact of these school restaurants on pupils results. - Many positive educational effects are reported also about the deworming interventions.

-But the generalization of these school restaurants, coupled with a systematic deworming of the pupils, must be justified by a rigorous evaluation.

-That is the purpose of this research proposal.

I.2. The most important research questions and objectives

• The main research question Do the expected positive educational effects of the school restaurants and/or

deworming exceed the additional costs they will generate?

• Specific research questions– Do school restaurants really boost school enrollment and attendance ?– Do they contribute to attract more girls than boys to school ?– Do they prevent schools from experimenting severe decrease in enrollment

or significant increase of dropouts, which oblige them to close?– Do they reduce absenteeism, so increase effective learning time? – Do they increase concentration capacity of pupils? Do they help pupils to

obtain better scores in the standardized evaluation tests?– Does deworming alone lead to the same results?– What intervention is more cost-effective?

1.2 Objectives• The main objective is to evaluate short term educational impacts of

school restaurants or/and deworming programs in rural primary schools areas in Senegal, and in the long term, the private and social benefit of these impacts.

• Specific objectives are to:• Identify the channels by which better nutrition improves school

attendance, drops out, retention to school and academic test scores.• Establish robust empirical relations empirical between school

performances and the existence of a school restaurant or/and a deworming program.

• Analyze the variability of these impacts with the intensity to exposition to the programs.

• Estimate costs and additional revenues resulting from better nutrition due to school restaurants and/or deworming programs.

• Disseminate results in scientific journals and in the Senegalese education sector.

• Help the Ministry of Education to implement an impact evaluation Unit.

II. Background of the project : Litterature review of impact

evaluation studies• First generation studies :The impact evaluation is realized on the

same sample of individuals, observed before and after the program implementation. The approach is non experimental, and it is difficult to obtain an unbiased estimation of the mean impact of the intervention.

• Second generation studies : Experimental or quasi-experimental approaches.

• Experimental studies select randomly the treated and the control groups, and observe them at the end of the program implementation.

• Quasi-experimental studies build ex post an artificial control group from the treated sample, and estimate the impact for non randomized samples of treated and control individuals.

II. Background of the project : Literature review of impact

evaluation studies (continuation)

Experimental studies on nutrition and deworming effects in developing countries primary schools

• Different types of programs are evaluated in these studies :

– Cash transfer to households who have as counterpart enrollment and regular health examination of their children (Mexican PROGRESSA program, South Africa by Aguiro, Carter and Woohard (2007)).

– Free breakfast programs in poor rural schools, evaluated in Cuerto et alii (2000), or health package programs like deworming, studied in Kenya by Miguel and Kremer (2004).

• Nutrition evaluation studies generally report positive impact on children school enrollment, attendance, dropouts and sometimes on pupils test scores results (Miguel and Kremer (2004) for Kenya, and Cuerto and Chinen (2007) for Peru).

• The studies conducted are different in many aspects, such as targeted population, sampling strategies, sampling size, duration of exposure to treatment, types of effects analyzed (nutritional, sanitarian, educational, etc.), impact evaluation methods applied, type of treatment effect (mean or marginal), etc.

Gaps in the literature

• Few studies on school restaurants, on the combination of school restaurants and deworming.

• Studies in school restaurants don’t take account of quality of meals.

• Duration of exposure not analyzed (except Behrman, Cheng et Tood (2004)).

III. Methodology

• III.1. Overview of the methodology

• The program : Introduction of free restaurants and implementation of periodic deworming and health control in poor rural primary schools of Senegal, with the support of the Ministry of Education, international organizations concerned by nutrition (World Food Program) and NGOs.

• The population concerned : Primary schools of rural areas, without restaurant or deworming and health control program for their pupils, and having from 50 to 600 pupils.

III. Methodology (continuation)

• III.1. Overview of the methodology

• Sampling strategy : Four random samples of schools will be drawn : a control sample, and three treatment samples receiving different packages of treatment : only school restaurants for one sample, deworming and health control for a second sample, and the combination of the two packages for a third sample of schools. The three packages will be implemented by the Ministry of Education agencies responsible for school restaurants and deworming, by the World Food Program and some NGOs, all aware of the project and supporting it.

• Results expected by the research : To obtain solid empirical proofs of the positive effects of a better nutrition (in quantity and quality) on children school achievements, estimation of the economic returns of an investment in such a project, and contribution to the popularization of impact evaluation methods to support better education policies in Senegal.

III. Methodology (continuation)

III.2. Data to be gathered

• Administrative data supplied by the Ministry of Education and its agencies.

• Experimental data gathered before, during, and at the end of the experimentation, concerning socio-demographic characteristics of the pupils and their households, their school and its environment, etc. Standard descriptive and inferential statistics tools will be applied to these data, and appropriate econometric methods used to estimate the final impact of the interventions.

III. Methodology (continuation)

III.3. Data gathering methodology

Population studied and sampling strategy

• The population studied consists of children attending a primary school and living in poor rural areas of Senegal. The intervention area will be defined by the major actors of the experimentation.

• A two- level sampling strategy is adopted : Schools will first be randomly drawn, then a number of pupils will be selected randomly in these schools.

III. Methodology (continuation)

III.3. Data gathering methodology (continuation)

• Precisely, 160 schools randomly will be drawn and assigned randomly into four sub-samples of equal size (40 schools). One sub-sample is the control group and the three others the treated groups. All their pupils receive the traitement.

• At the pupil level, data will be collect on 20 pupils randomly drawn in each school,

resulting in a sample size of 800 for each group and a total sample size of 3200. The grade of pupils is third year.

• This sample size can be justified on the grounds of cluster-sampling results. Supposing that the between-class variance ratio of an impact variable of interest to its total variance is very low (0,05 for example), as is required for a good cluster-sampling, and an effect size equal to 0,3, any good design package (e;g. « O.D » by X. Liu, S. Raudenbush ), compute the optimal size of a cluster as equal to n=20, for a test power equal to 85%, in the test of the hypothesis of 30% annual decrease of the under-nutrition rate (as revealed for Senegal by a recent World Bank report on a nutrition pilot project).

III. Methodology (continuation)

III.3. Data gathering methodology (continuation)

Three Stages for the data collection

• First stage (pre-experimental stage) : Initial information on the schools and children. The random nature of the sampling will be tested in this stage, by comparing the main characteristics of the children in the four groups (matching methods).

• Second stage : Data on the experimentation and its results for the children are collected continuously, throughout the process.

• Final stage : Similar to that organized before the beginning of the experimentation for the four groups of schools.

• Data on the school restaurants, the different components of their menus and their quantity, their stock, the number of children served, number of days where they are open, etc.

• Children health and deworming data (deficiencies or any trouble affecting the children cognitive and intellectual development like vitamin deficiencies, their weight and height by age, illness occurrences and duration, etc.).

Data collected and organized in a data base will concern the following topics :

III. Methodology (continuation)

III.3. Data gathering methodology (continuation)

• Data on the school and its achievements : number of pupils, infrastructures, teaching methods, financing sources, number of teachers, number of new entrees, enrollments rates, dropouts, retention, etc., for the different grades.

• Data on the pupils and their households characteristics

• Pupils’ school achievements : Particularly their scores for standardized tests organized by the Ministry of Education.

• Community data : Information on the villages where the selected schools are located and their environment (public infrastructures, markets, health centers, etc.).

III. Methodology (continuation)

• III.4. Tests and Modelling

Hypotheses to be tested in the study

• A better nutrition through school restaurants or a systematic deworming of children favors the increase of school enrollment in poor rural areas.

• Schools with a restaurant and/or a deworming programs obtain better educational results in terms of internal efficiency and pupils achievements than school which have not.

• The impact of school restaurants programs on educational performances of pupils is greater than that of deworming programs, the comparison being made both relatively to the control group and inside the treated groups.

• The binary or discrete treatment method : Estimates the value of the impact variables of interest before and after the treatment for both the treated and control groups, and takes the double difference of the results. This is equivalent to the estimation of a regression model, explaining the impact variable by appropriate dummy variables for the groups and periods, and with exogenous variables for control.

Estimation of the programs impact

• Continuous treatment approaches : Take into account the treatment intensity or duration for the impact evaluation, using results developed by Hirano and Imbens (2004), and others. The impact estimator introduced by these authors is a function of the treatment duration, supposed to be normally distributed with parameters estimated by the maximum likelihood method, and is computed with a two stages procedure applied to a flexible specification of the impact function.

III. Methodology (end)

• An other approach to estimate continuously the treatment impact and to compute the mean or the marginal impact is the non parametric method of local regression used by Behrman, Cheng and Todd (2004).

• Estimate of the net economic return of each program : A cost-benefice analysis will compare, for a treated child, the costs of each intervention to the estimate of the present value of the total future incomes it will generate. Future incomes will be estimated for the whole activity period of the child, supposed to begin at 15 years and to end at 65 years.

IV. Ethic, social, gender and environment problems related to

the program• No ethic problem in perspective.

• No social problems projected, as the program will adopt a participative approach, implying all the local, political and education actors.

• The program will take in account the gender gap in education, by measuring its potential effects for both the girls and the boys, and will investigate the channels of reduction of this gap.

V. Consulting strategy and dissemination of the outcomes

• The project will be supervise by a national Steering Comity where are represented the main actors appointed by the Planning and Reform Direction of the Ministry of Education. The different stages of the experimentation and of the research will be monitored by this comity.

• A workshop launching the project will be organized, and a dissemination seminar of the results will be held at the end of the program.

- An extensive research report, posted on the websites of CRES and PEP.

• The outcomes will consist of :

- An executive report for the education authorities and the NGOs concerned by the project.

-A non technical summary will be written for the sampled schools authorities, the pupils’ parents association, etc.

- An unique experimental database on nutrition, health and education will be available for years of research.

THANK YOU FOR YOUR ATTENTION