PROGRESA and Its Impacts on the Welfare

of Rural Households in Mexico

Emmanuel Skoufias

RESEARCHREPORT 139INTERNATIONAL FOOD POLICY RESEARCH INSTITUTEWASHINGTON, DC

Copyright © 2005 International Food Policy Research Institute. All rights reserved. Sections of this material may be reproduced for personal and not-for-profit use without theexpress written permission of but with acknowledgment to IFPRI. To reproduce the mate-rial contained herein for profit or commercial use requires express written permission. To obtain permission, contact the Communications Division <ifpri-copyright@cgiar.org>.

International Food Policy Research Institute2033 K Street, NWWashington, DC 20006-1002 USATelephone +1-202-862-5600www.ifpri.org

Library of Congress Cataloging-in-Publication Data

Skoufias, Emmanuel.PROGRESA and its impacts on the welfare of rural households in Mexico /

Emmanuel Skoufias.p. cm. — (Research report ; 139)

Includes bibliographical references.ISBN 0-89629-142-1 (alk. paper)1. Rural poor—Government policy—Mexico. 2. Programa de Educación,

Salud y Alimentación (Mexico City, Mexico)—Evaluation. 3. Poverty—Government policy—Mexico. 4. Mexico—Social policy. I. Title. II. Researchreport (International Food Policy Research Institute) ; 139.HC140.P6.S58 2005362.5′56′0972—dc22 2005006221

Contents

List of Tables iv

List of Figures v

Acronyms and Abbreviations vi

Foreword vii

Acknowledgments viii

Summary ix

1. Background and Program Description 1

2. PROGRESA Seen through an Economic Lens 13

3. The Experimental Design and Information Sources Used to Evaluate PROGRESA 26

4. An Evaluation of PROGRESA’s Targeting and Its Impact on Poverty 34

5. Summary of the Methods Used to Evaluate the Impact of PROGRESA 40

6. A Summary of the Findings of the Impact Evaluation and a Cost Analysis of the Program 48

7. Conclusions and Future Policy Considerations 65

Appendix A: Summary of Mexican Anti-Poverty Programs 68

Appendix B: Characteristics of the Localities in the Evaluation Sample 72

Appendix C: On the Impact of PROGRESA on Poverty 75

References 81

iii

Tables

1.1 PROGRESA monthly cash transfer schedule (nominal pesos) 4

1.2 Composition of the basic health services package 5

1.3 Annual frequency of health care visits required by PROGRESA 6

1.4 Micronutrients contained in the food supplements 8

1.5 PROGRESA transfers to beneficiary households from November 1998 to October 1999 (pesos) 10

3.1 Decomposition of the sample of all households in treatment and control villages 29

3.2 Number of households and individual members covered in each survey round 32

5.1 Key outcome indicators and impact estimators used in the quantitative evaluation of PROGRESA 45

C.1 Poverty measures and difference-in-difference tests for total income per capita using canasta basica as poverty line 78

C.2 Poverty measures and difference-in-difference tests for total income per capita using 50th percentile of per capita value of consumption as poverty line 79

iv

Figures

1.1 Average cash transfer paid out by PROGRESA by month 9

2.1 Effect of conditional cash transfers on children’s school attendance and work 19

4.1 Percentages of households in treatment localities that receive transfers from other programs and PROGRESA 37

6.1a School enrollment and labor force participation of boys in PROGRESA communities prior to program implementation 49

6.1b School enrollment and labor force participation of girls in PROGRESA communities prior to program implementation 49

6.2a School attendance of children 8–11 years old 50

6.2b School attendance of children 12–17 years old 50

6.3a Incidence of illness for newborns to two-year-olds 57

6.3b Incidence of illness for three- to five-year-olds 57

6.4 Daily visits to public clinics 58

C.1a Mean household total income from childhood (excluding PROGRESA cash transfers) among beneficiary households with children 8–17 years of age 77

C.1b Mean household labor income from children among beneficiary households with children 8–17 years of age 77

C.1c Mean household other income from children among beneficiary households with children 8–17 years of age 77

v

Acronyms and Abbreviations

CIMO Programa Calidad Integral y ModernizacionCONAFE Consejo Nacional de Fomento EducativoCONASUPO Compania Nacional de Subsistencias PopularesCOPUSI Programa de Cocinas Polulares y Unidades de Servicios IntegralesDICONSA Distribuidora Comercial CONASUPOENCASEH Encuesta de Caracteristicas Socioeconomicas de los HogaresENCEL Encuesta de Evaluacion de los HogaresENIGH Encuesta Nacional de Ingreso-Gasto de los HogaresFISM Fondo para la Infraestructura Social MunicipalFONAES Fondo Nacional de Apoyo a Empresas SocialesIFPRI International Food Policy Research InstituteIMSS Instituto Mexicano del Seguro SocialINI Instituto Nacional IndigenistaLICONSA Leche Industrial CONASUPOPASAF Programa de Asistencia Social Alimentaria a FamiliasPET Programa de Empleo TemporalPROBECAT Programa de Becas de Capacitacion para DesempleadosPROCAMPO Programa de Apoyos Directos al CampoPROGRESA Programa Educación, Salud AlimentacionPRONASOL Programa Nacional de SolidaridadSEDESOL Secretaria de Desarrollo SocialSEP Secretaria de EducaciónTORTIBONO Subsidio a la Tortilla

vi

Foreword

In the second half of the twentieth century, many developing countries adopted broad socialassistance programs, like food subsidies, ostensibly designed to help poor people. Their ef-fectiveness was mixed and, unfortunately, many of these expensive programs did not make

much difference in the lives of poor people, much less help them climb permanently out ofpoverty. In the 1990s Mexico took a completely new approach. It launched a social program—PROGRESA—that was revolutionary in two ways. First, PROGRESA aimed to integrate in-terventions in health, education, and nutrition simultaneously, based on an understanding thatthese dimensions of human welfare are interdependent and that poor health, education, and nu-trition are both causes and consequences of poverty. Second, PROGRESA was designed fromthe beginning to be continually evaluated and improved, so that it would become ever moreeffective at improving the well-being of Mexico’s poorest people.

From 1998 to 2000 the International Food Policy Research Institute (IFPRI) assisted thePROGRESA administration in evaluating the program. This research resulted in a series ofIFPRI reports, synthesized here, on aspects of PROGRESA’s performance. The evaluation notonly highlighted areas of success, but also suggested needed improvements in the program. Onthe one hand, for example, the research showed that PROGRESA has helped keep poor chil-dren in school longer, improved the health of young children and adults, increased women’suse of prenatal care, and improved child nutrition. On the other hand, the evaluation revealedthat PROGRESA could have a greater impact on school enrollment by focusing on attendancein secondary schools—the stage at which many poor children drop out.

In the election of 2000, the people of Mexico voted a new party into power. Yet, faced withevidence of PROGRESA’s effectiveness, the new government decided to keep the program(renamed Oportunidades) and to make needed improvements in its operation.

IFPRI’s research on PROGRESA has advanced our knowledge about policy steps thatgovernments can take to improve the capacities of poor people, who may require interventionsin several areas to make real headway in overcoming poverty. In the meantime comparableprograms have been tested or implemented in other countries, including Brazil, Honduras, andNicaragua, and it would be promising to explore their adaptation in African and Asian coun-tries as well. It is our hope that research of this kind will encourage replications, adaptationsto country circumstances, and rigorous evaluation of social programs in many developingcountries.

Joachim von BraunDirector General, IFPRI

vii

viii

Acknowledgments

Iam deeply grateful to Lawrence Haddad, director of the Food Consumption and NutritionDivision of IFPRI, for providing me the opportunity to lead the PROGRESA evaluation.As a team leader I had the honor and pleasure to work closely with all the members of the

superb IFPRI-PROGRESA research team and affiliated institutions. Specifically, the IFPRIteam members include Dr. Michelle Adato, Dr. David Coady, Dr. Benjamin Davis, Dr. Béné-dicte de la Brière, Dr. Sudhanshu Handa, Dr. Rebecca Lee Harris, Dr. John Hoddinott, Dr.Agnes Quisumbing, and Dr. Marie Ruel. Habiba Djebbari, Ryan Washburn, and Lyla Kuriyanprovided excellent research assistance, while Lourdes Hinayon and Bonnie McClafferty pro-vided superb administrative and editorial support. A number of academics from U.S. and Mex-ican universities have also played a critical role in the design and implementation of the PRO-GRESA evaluation. These include Professor Jere Behrman (University of Pennsylvania),Professor Paul J. Gertler (University of California, Berkeley), Professor T. Paul Schultz (YaleUniversity), Professor Petra E. Todd (University of Pennsylvania), Dubravka Mindek (EscuelaNacional de Antropología y Historia), and Professor Graciela Teruel (Universidad Iberoamer-icana). Finally, I acknowledge my deep gratitude to the late director of the PROGRESA pro-gram, Dr. José Gómez de León, and to the members of his wonderful team: Daniel Hernan-dez, Monica Orozco, Patricia Muniz, Humberto Soto, Sergio de la Vega, Mari-Carmen Huerta,Raul Perez, Daniela Sotres, Beatriz Straffon, and Hadid Vera.

Summary

In early 1998, the International Food Policy Research Institute (IFPRI) was asked to assistthe PROGRESA administration to “determine if PROGRESA is functioning in practice asit is intended to by design.” This document synthesizes the findings contained in a series of

reports prepared by IFPRI for PROGRESA between November 1998 and November 2000. Amore detailed description of the research, rationale, and methods appears in the original IFPRIreports, which are provided in English and in Spanish on the CD enclosed with this publication.

PROGRESA is one of the major programs of the Mexican government aimed at develop-ing the human capital of poor households. Targeting its benefits directly to the population inextreme poverty in rural areas, PROGRESA aims to alleviate current and future poverty levels through cash transfers to mothers in households. The cash transfers provided are condi-tioned on regular school attendance and visits to health care centers. At the end of 1999, PROGRESA covered approximately 2.6 million families, representing one ninth of all fami-lies in Mexico; the beneficiaries comprised about 40 percent of all rural families. At that time,the program operated in almost 50,000 localities in more than 2,000 municipalities and 31states. PROGRESA’s budget of approximately US$777 million in 1999 was equivalent to 0.2percent of Mexico’s gross domestic product (GDP).

For Mexico, the design of PROGRESA represents a significant change in the provision ofsocial programs. First, in contrast to previous poverty alleviation programs in Mexico, PROGRESA applies targeting at the household level to ensure that the resources of the pro-gram are directed and delivered to households in extreme poverty, that is, the households thatcan most benefit from the program. General food subsidies, such as the tortilla price subsidy[Subsidio a la Tortilla (TORTIBONO)], are widely acknowledged to have had a high cost inthe government budget and a negligible effect on poverty because of the leakage of benefits tonon-poor households. In addition, more decentralized, community-based, demand-driven pro-grams such as the earlier anti-poverty program PRONASOL, in place during 1988 and 1994,were thought to be susceptible to local political influences and not very effective at reachingthose in extreme poverty.1 Under PROGRESA, communities are first selected using a mar-ginality index based on census data. Then, within the selected communities, households arechosen using socioeconomic data collected for all households in the community.

Second, unlike earlier social programs in Mexico, PROGRESA contains a multisectoralfocus. By design, the program intervenes simultaneously in health, education, and nutrition.The integrated nature of the program reflects a belief that addressing all dimensions of humancapital simultaneously has greater social returns than their implementation in isolation. Im-proved health and nutritional status not only are desirable in themselves, but also have an in-direct impact through enhancing the effectiveness of education programs, since, for example,school attendance and performance are often adversely affected by poor health and nutrition.Poor health is therefore both a cause as well as a consequence of poverty. Also by design,

1See Yaschine (1999) and Levy (1994) for a description of the program.

ix

PROGRESA differs in the mechanism of delivering its resources. Recognizing the potentialof mothers to use resources effectively and efficiently in a manner that reflects the immediateneeds of the family, PROGRESA gives benefits exclusively to mothers.

Specifically, the education component of PROGRESA is designed to increase school en-rollment among youth in Mexico’s poor rural communities by making education grants avail-able to pupils’ mothers, who then are required to have their children attend school regularly.In localities where PROGRESA currently operates, households that have been characterizedas poor, and have children enrolled in grades 3–9, are eligible to receive these educationalgrants every two months. The levels of these grants were determined taking into account,among other factors, what a child would earn in the labor force or contribute to family pro-duction. The educational grants are slightly higher at the secondary level for girls, given theirpropensity to drop out at earlier ages. Every two months, confirmation of whether children of beneficiary families attend school more than 85 percent of the time is submitted to PROGRESA by schoolteachers and directors, and this triggers the receipt of bimonthly cashtransfers for school attendance.

In the area of health and nutrition, PROGRESA brings basic attention to health issues andpromotes health care through free preventive interventions, such as distribution of nutritionalsupplements, and education on hygiene and nutrition as well as monetary transfers for the pur-chase of food. Receipt of monetary transfers and nutritional supplements is tied to mandatoryhealth care visits to public clinics. This aspect of the program emphasizes targeting benefits tochildren under five, and pregnant and lactating women, and is administered by the Ministry ofHealth and by IMSS-Solidaridad, a branch of the Mexican Social Security Institute, whichprovides benefits to uninsured individuals in rural areas.

Nutritional supplements are given to children between the ages of four months and twoyears, and to pregnant and breastfeeding women. If signs of malnutrition are detected in chil-dren between two and five years of age, nutritional supplements will also be administered. Thenutritional status of beneficiaries is monitored by mandatory visits to the clinic and is morefrequently monitored for children five years of age and under and pregnant and lactatingwomen. At each visit, young children and lactating women are measured for wasting (weight-for-height), stunting (height-for-age), and weight-for-age. An appointment monitoring systemis set up and a nurse or doctor verifies adherence. Every two months the health care profes-sionals submit certification of beneficiary visits to PROGRESA, which triggers the receipt ofthe cash transfer for food support.

The average monthly payment (received every two months) by a beneficiary familyamounts to 20 percent of the value of monthly consumption expenditures prior to the initia-tion of the program. One additional requirement of the PROGRESA program is that house-holds benefiting from PROGRESA were to stop receiving benefits from other programs in ef-fect, such as Niños de Solidaridad, Abasto Social de Leche, de Tortilla, and the NationalInstitute of Indigenous People (INI). This requirement of the PROGRESA program representsthe short-run objective of the new poverty alleviation strategy of the Mexican government to minimize duplication of benefits to poor families. A longer run objective is to absorb the variety of poverty alleviation programs within one program such as PROGRESA that repre-sents an integrated approach to poverty alleviation.

Poverty alleviation programs such as PROGRESA are an important component of the setof instruments that government has at its disposal for redistributing income and assets amonghouseholds. Program evaluation can improve the design and implementation of programs sothat they can have a greater effect on household welfare. In addition, program evaluation,when applied consistently across a wide spectrum of programs, allows governments to have astronger effect on social welfare with the same budget by reallocating resources from less to

x SUMMARY

more effective programs. In addition to these economic considerations, there are also socialand political reasons for justifying program evaluations. Primary among these is that programevaluation can serve as a means of increasing the accountability of governments toward theircitizens by providing a template for comparing sensibly whether public funds are used effec-tively toward poverty alleviation.

In the case of PROGRESA the national elections that were forthcoming in the year 2000and the increasing public support toward the opposition parties contributed to an unprece-dented willingness by the Zedillo administration to support a rigorous and politically neutralevaluation of the program. It is hard to dismiss the interpretation that the design of the pro-gram, with its careful targeting of the benefits to poor rural households, along with the deci-sion to evaluate PROGRESA, also served political purposes. For example, the targeting of theprogram to households in extreme poverty and the provision of the cash transfers directly tothe beneficiary households signaled a break from the wasteful practices of the past. At thesame time, the decision to evaluate the program established a precedent that any future ad-ministration could hardly afford not to imitate.

Until the implementation of PROGRESA in Mexico, as in most developing countries, national poverty alleviation programs were not customarily subjected to rigorous evaluations.In the rare instances in which programs were evaluated, the decision to do so was usually takenyears after the implementation of the program. In such situations, it is typically too late to eval-uate the program because the program has already achieved wide coverage and it is practicallyimpossible to construct a reliable comparison group that is essential for the credible evaluationof the program’s impact.

PROGRESA distinguishes itself further by the fact that the elements essential for a rigor-ous evaluation of the program’s impact were taken into consideration since the very earlystages of the implementation of the program. The strategy adopted for the evaluation of PROGRESA consisted of the following two critical elements:

1. The adoption of an experimental design in the early stage of the implementation of theprogram, which allows the measurement of program impact by comparing the mean values of key outcome indicators among beneficiary households (treatment group) withsimilar households that were not yet covered by the program (comparison/control group).The experimental evaluation design of PROGRESA offers the opportunity to evaluate theimpact of the program on beneficiary households by systematically removing the influ-ence of other factors that might have contributed to the observed changes.

2. The collection of information from these two groups of households (treatment group and comparison groups) before and after the implementation of the program.

Specifically, the full sample used in the evaluation of PROGRESA consists of repeated ob-servations (panel data) collected for 24,000 households from 506 localities in the seven statesof Guerrero, Hidalgo, Michoacan, Puebla, Queretaro, San Luis Potosi, and Veracruz. Of the506 localities, 320 localities assigned to the treatment group (where PROGRESA was in op-eration) and 186 localities were assigned as controls. As originally planned, the localities serv-ing the role of a control group started receiving PROGRESA benefits by December 1999. A total of 24,000 households from 506 localities in these states were interviewed periodicallybetween November 1997 and November 1999. Focus groups and workshops with beneficiar-ies, local leaders, PROGRESA officials, health clinic workers, and schoolteachers were alsocarried out. The following are some key highlights beginning to emerge from IFPRI’s evalua-tion of the impact of PROGRESA on its target group, Mexico’s rural poor:

SUMMARY xi

• After three years poor children in rural areas of Mexico where PROGRESA is currentlyoperating are more likely to enroll in school. Mexico’s primary school children typicallymaintain a primary school enrollment rate of 93 percent but generally begin dropping out of school after completing the sixth grade. Enrollment rates in general witness an-other steep decline as children transition to senior high school. Research reveals thatPROGRESA has had the largest impact on children who enter secondary school and represents a percentage increase of enrollment of more than 20 percent for girls and 10percent for boys. The research revealed that much of the positive impact on enrollment isattributable to increasing continuation rates rather than on getting children who were outof school to return.

• The accumulated effect of increased schooling from grades 1–9 suggests that the pro-gram can be expected to increase educational attainment for the poor by 0.66 years ofadditional schooling by grade 9 (0.72 years of additional schooling for girls, 0.64 yearsfor boys). Given that the average 18-year-old youth typically achieved 6.2 years of com-pleted schooling, PROGRESA effectively can be expected to increase educational attain-ment of poor Mexican rural children by 10 percent.

• Improved livelihood security for the poor depends on improving early childhood healthcare. Frequency and duration of illness have profound effects on the development andproductivity of populations. The IFPRI analysis indicates that improved nutrition andpreventive health care in PROGRESA areas have made younger children more robustagainst illness. Specifically, PROGRESA children one to five years of age have a 12 percent lower incidence of illness than non-PROGRESA children.

• Adult PROGRESA beneficiaries on average have 19 percent fewer days of difficultywith daily activities, 17 percent fewer days incapacitated by illness, 22 percent fewerdays in bed, and are able to walk about 7 percent longer than nonbeneficiaries.

• In January 1996, more than a year before PROGRESA began, average visits to healthclinics were identical in PROGRESA and non-PROGRESA localities. In 1998, the firstfull year in which PROGRESA was operational in all treatment localities, visit rates inPROGRESA areas were shown to grow faster than in non-PROGRESA areas.

• PROGRESA increased the number of first visits in the first trimester of pregnancy by about8 percent. This shift to early prenatal care significantly reduced the number of first visits inthe second and third trimesters of pregnancy. This positive change in behavior is docu-mented to have a significant improvement in the health of infants and pregnant mothers.

• In 1999, median food expenditures were 13 percent higher in PROGRESA householdswhen compared with control households. This increase was driven largely by higher expenditures on fruit, vegetables, meats, and animal products. By November 1999, median caloric acquisition had risen by 10.6 percent. Beneficiaries felt that since the initiation of PROGRESA, poor households are eating better.

• The nutrition of preschool children is of considerable importance not only because ofconcern over their immediate welfare, but also because their nutrition in the formativestages of life is widely perceived to have a substantial and persistent impact on theirphysical and mental development and on their health status as adults. Stunting—lowheight-for-age—is a major form of protein-energy malnutrition. In 1998, survey resultsindicated that 44 percent of 12- to 36-month-old children in PROGRESA regions werestunted.

• Data suggest that PROGRESA has had a significant impact on increasing child growthand in reducing the probability of child stunting; an increase of 16 percent in meangrowth rate per year (corresponding to 1 cm) for children who received treatment in thecritical 12- to 36-month age range.

xii SUMMARY

• The analysis suggests that PROGRESA may be having a fairly substantial effect on lifetime productivity and potential earnings of currently small children in poor house-holds. IFPRI estimates that the impact from the nutritional supplements alone and theireffect on productivity into adulthood could account for a 2.9 percent increase in lifetimeearnings.

• The administrative costs employed in getting transfers to poor households appear to besmall relative to the costs incurred in previous programs and for targeted programs inother countries. According to the program cost analysis, for every 100 pesos allocated to the program, 8.9 pesos are “absorbed” by administration costs. Dropping householdtargeting would reduce program costs from 8.9 pesos to 6.2 pesos per 100 pesos trans-ferred, while dropping conditioning would reduce the program costs from 8.9 pesos to 6.6 pesos per 100 pesos transferred. Dropping both would reduce these costs to 3.9 pesosper 100 pesos transferred.

One of the most important contributions of IFPRI’s evaluation of PROGRESA has beenthe continuation of the program in spite of the historic change in the government of Mexicoin the 2000 elections. The overwhelming (and unprecedented) evidence that a poverty alle-viation program shows strong signs of having a significant impact on the welfare and humancapital investment of poor rural families in Mexico has contributed to the decision of the Foxadministration to continue with the program and to expand its coverage in the poor urbanareas of the country after some improvements in the design of the program.

The majority of the improvements in the design of PROGRESA (renamed Oportunidadesby the Fox administration) were based on findings of the evaluation of PROGRESA that re-vealed areas of needed improvements in some of the structural components and the operationof the program. For example, the evaluation revealed a larger program impact only on theschooling attendance of children of secondary school age. This suggests that it would bepreferable to reorient the funds from primary school to families with children of secondaryschool age. Oportunidades did exactly that by extending the benefits of the program to chil-dren attending high school (preparatoria) rather than just junior high school, as it was in theearlier PROGRESA. Also, initially the award of PROGRESA’s educational benefits was con-ditional on regular school attendance but not performance. Oportunidades improved on thisdesign feature by linking benefits to performance, such as granting bonuses to encourage suc-cessful completion of a grade, or linking benefits with participation in other programs. For ex-ample, the creation of a related program, Jovenes con Oportunidades, aims to create income-generating opportunities for poor households through preferential access to microcredit,housing improvements, adult education, and social insurance.

Yet in spite of these improvements in the program, the evaluation findings suggest thatsome issues remain to be resolved. For example, the program was found to have no measur-able impact on the achievement test scores of children in beneficiary localities or on their reg-ular school attendance. This implies that if the program is to have a significant effect on thehuman capital of children, more attention needs to be directed to the quality of education pro-vided in schools. Enrolling in and attending school regularly are only necessary conditions forthe improvement of children’s human capital. Finally, it is also important to find ways to main-tain and improve the quality of the information provided in the pláticas.

The opportunity to conduct a rigorous evaluation of a program such as PROGRESA hasset a higher set of standards for the design and conduct of social policy in Mexico and in LatinAmerica in general. As policymakers now have a better sense of the basic elements of a pro-gram that can be effective toward alleviating poverty in the short run and, perhaps, in the longrun, the list of questions and concerns about program choices and design cannot help but grow

SUMMARY xiii

larger. For example, is it possible for unconditional cash transfers without any “strings” at-tached to have similar or higher impacts on human capital investments of poor rural families?Is the amount of the cash transfer given to families too high? Perhaps a lower cash transfercould achieve the same impact. Is the simultaneous intervention in the areas of education,health, and nutrition areas preferable to intervening in each of these sectors separately?

PROGRESA has been accompanied by complementary efforts and resources directed atstrengthening the supply and quality of educational and health capacity constraints that mightarise as a result of the more intensive use of existing facilities and resources. Perhaps this fea-ture of the program plays a critical role for the success of PROGRESA, and programs that donot pay sufficient attention to the capacity constraints that might arise as a result of the condi-tionality of cash transfers may be less effective. Is it not possible that similar or even highereffects on school attendance can be achieved through alternative programs, such as buildingnew schools or improving the quality of educational services? What if the benefits were givento the fathers rather than the mothers in the household? Are programs aimed toward youngerchildren to be preferred over programs aimed toward older children?

The nature of the program and the scope of the evaluation of the program’s impact can pro-vide only a tentative answer to some of these questions. More definite answers can be obtainedthrough the analysis and evaluation of programs that incorporate all or some of these featuresas part of their structure. Hopefully, early involvement of researchers in the design and evalu-ation of programs implemented in other countries, such as Brazil (Bolsa Familia), Colombia(Familias en Accion), Honduras (PRAF), Jamaica (PATH), Nicaragua (RPS), and Turkey, canshed more light on these critical questions for policy.

Finally, the critical question of whether the vicious cycle of poverty and its intergenera-tional transmission are indeed broken can be determined only by following the cohorts of chil-dren currently in the program. At least in Mexico, the evaluation of PROGRESA’s impact inthe short term has provided a solid foundation for determining whether the program had a sig-nificant difference in the welfare and earnings of these children as adults.

xiv SUMMARY

C H A P T E R 1

Background and Program Description

In 1997, the federal government of Mexico introduced the Programa de Educación, Salud yAlimentación (the Education, Health, and Nutrition Program), known by its Spanishacronym, PROGRESA, as part of its renewed effort to break the intergenerational trans-

mission of poverty. The program has a multiplicity of objectives, aimed primarily at improv-ing the educational, health, and nutritional status of poor families, particularly of children andtheir mothers. PROGRESA provides cash transfers linked to children’s enrollment and regu-lar school attendance and to clinic attendance. The program also includes in-kind health ben-efits and nutritional supplements for children up to age five and for pregnant and lactatingwomen.

The expansion of the program across localities and over time was determined by a plannedstrategy that involved the annual budget allocations and logistical complexities associated withthe operation of the program in very small and remote rural communities (such as verificationthat the localities to be covered by the program had the necessary educational and health fa-cilities). In consequence, the expansion of the program took place in 11 phases.2 In phase 1,which began in August 1997, 140,544 households in 3,369 localities were incorporated. Phase2 of the program began in November 1997 when a further 160,161 households in 2,988 lo-calities were incorporated. The greatest expansion occurred in 1998 (i.e., phases 3–6) whennearly 1.63 million families in 43,485 localities were incorporated. By phase 11, the finalphase of the program in early 2000, the program included nearly 2.6 million families in 72,345localities in all 31 states. This constitutes around 40 percent of all rural families and one ninthof all families in Mexico. The total annual budget of the program in 1999 was around $777million, equivalent to just under 20 percent of the federal poverty alleviation budget or 0.2 per-cent of gross domestic product (GDP).

As part of an overall strategy for poverty alleviation in Mexico, PROGRESA works in con-junction with other programs that are aimed toward developing employment and income op-portunities (such as Programa de Empleo Temporal [PET]) and facilitating the formation ofphysical capital, such as the Fondo para la Infraestructura Social Municipal (FISM) (for amore detailed description of the various anti-poverty programs in Mexico, see Appendix A).

For Mexico, the design of PROGRESA represents a significant change in the provision ofsocial programs. First, in contrast to previous poverty alleviation programs in Mexico, PRO-GRESA applies targeting at the household level to ensure that the resources of the programare directed and delivered to households in extreme poverty, that is, the households that canmost benefit from the program. General food subsidies, such as the tortilla price subsidy (Sub-

2For more details see Section 4 and Table 1 in Coady (2000).

1

sidio a la Tortilla [TORTIBONO]), arewidely acknowledged to have had a highcost on the government budget and a negli-gible effect on poverty because of the leak-age of benefits to non-poor households. Inaddition, more decentralized, community-based, demand-driven programs such as theearlier anti-poverty program Programa Na-cional de Solidaridad (PRONASOL), inplace during 1988 and 1994, were thoughtto be susceptible to local political influencesand not very effective at reaching the ex-treme poor.3 Under PROGRESA, commu-nities are first selected using a marginalityindex based on census data. Then, withinthe selected communities, households arechosen using socioeconomic data collectedfor all households in the community.

Second, unlike earlier social programsin Mexico, PROGRESA contains a multi-sectoral focus. By design, the program in-tervenes simultaneously in health, educa-tion, and nutrition. The integrated nature ofthe program reflects a belief that addressingall dimensions of human capital simultane-ously has greater social returns than theirimplementation in isolation. Improvedhealth and nutritional status are desirablenot only in themselves, but also have an in-direct impact through enhancing the effec-tiveness of education programs since, forexample, school attendance and perfor-mance are often adversely affected by poorhealth and nutrition. Poor health is thereforeboth a cause as well as a consequence ofpoverty. Also by design, PROGRESA dif-fers in the mechanism of delivering its resources. Recognizing the potential ofmothers to use resources effectively and ef-ficiently in a manner that reflects the imme-diate needs of the family, PROGRESAgives benefits exclusively to mothers.

These features of the program, in com-bination with its enormous scale, suggest

that the program has the potential to have asignificant impact on current and futurepoverty in Mexico. PROGRESA distin-guishes itself further by the fact that the el-ements essential for a rigorous evaluation ofthe program’s impact were taken into con-sideration since the very early stages of theimplementation of the program. For exam-ple, the PROGRESA administration tookadvantage of the sequential expansion of theprogram and adopted an experimental de-sign for its evaluation. This permitted thecollection of repeated observations frombeneficiary households surveyed before andafter the implementation of the program aswell as the collection of similar data fromcomparable households that were not yetcovered by the program. This experimentalevaluation design of PROGRESA offers theopportunity to evaluate the impact of theprogram on beneficiary households bymeasuring the changes that have taken placein the indicators of household investment inhuman capital and other economic and so-cial measures while systematically isolatingthe influence of other factors that mighthave contributed to the observed changes.4

This document synthesizes 24 monthsof extensive research by International Food Policy Research Institute (IFPRI) re-searchers, academic collaborators, andPROGRESA staff, designed to evaluate theimpact of PROGRESA and the extent towhich the measured impacts are delivered ina cost-effective manner. The impact evalua-tion focuses primarily on three poverty re-duction areas: improving school enrollment,improving health and nutrition outcomes,and increasing household consumption forpoor rural families. Other topics such as theimpact of PROGRESA on women’s status,intrahousehold transfers, and work incen-tives are also examined. The synthesis presented here builds on a series of reports

2 CHAPTER 1

3See Yaschine (1999) and Levy (1994) for a description of the program.

4For a more detailed discussion of the variety of quasi-experimental designs available in the evaluation literature,see Valadez and Bamberger (1994) and Ravallion (1999).

presented by IFPRI to PROGRESA fromNovember 1998 through November 2000. A more detailed description of the research,rationale, and methods appears in the origi-nal IFPRI reports, which are provided inEnglish and in Spanish in a CD enclosedwith this publication.

This analysis of the PROGRESA pro-gram comes at a crucial time as other LatinAmerican countries (such as Honduras,Nicaragua, Colombia, Brazil, and Argen-tina) are in the process of revising their so-cial program along lines similar to those ofthe PROGRESA program in Mexico (Rawl-ings and Rubio 2002).

To provide readers with a commonknowledge about the program, the require-ments and the benefits of the program, aswell as some of its operational aspects, aredescribed in detail. Most of the presentationin this report is drawn from documents pre-pared by the PROGRESA administration as well as from discussions of IFPRI re-searchers with PROGRESA administrationofficials.

Description of theEducational Benefits andProgram RequirementsEducation is seen as a pivotal component ofPROGRESA, reflecting the strong empiri-cal link between human capital, productiv-ity, and growth, but especially because it isseen as a strategic factor in breaking the vi-cious cycle of poverty. Investments in edu-cation are therefore viewed as a way of fa-cilitating growth while simultaneouslyreducing inequality and poverty.

The stated objectives of the program areto improve school enrollment, attendance,and educational performance. This is in-tended to be achieved through four channels:

1. A system of educational grants2. Monetary support for the acquisition of

school material

3. Strengthening the supply and quality ofeducation services

4. Cultivation of parental responsibility for,and appreciation of the advantages stem-ming from, their children’s education.

These are obviously interrelated in thateach is thought to enhance the effectivenessof the others in improving attendance andperformance.

The system of educational grants is in-tended to encourage regular and continuousattendance, especially for females. This isreflected in two crucial design features (seeTable 1.1). First, the size of the grant in-creases through grade levels. Second, at thesecondary level, grants are higher for fe-males. The latter is meant to address the cul-tural gender bias against female social par-ticipation as well as being an attempt tointernalize education externalities that ac-crue to other families after the marriage ofwomen. The level of the grants was set withthe aim of compensating for the opportunitycost of children’s school attendance.

The program tries to maintain the realvalue of the cash benefits stable over time.The nominal value of the educational cashbenefits and the cash benefit granted forfood consumption is adjusted every sixmonths to account for changes in the cost ofliving. The program design also tries toavoid diluting a household’s incentives forself-help. The total monthly monetary trans-fer (i.e., from education grants and foodsupport) a family can receive is capped (forthe period July–December 1999) at 750pesos (including 125 pesos for food). Thismay possibly impact on family educationdecisions, for example, how many andwhich eligible children to enroll. Also, asstated in PROGRESA documents, in orderto avoid adverse fertility incentives, onlychildren over the age of seven years (thestandard age of third-year primary students)are eligible for education grants.5

BACKGROUND AND PROGRAM DESCRIPTION 3

5As it is outlined in the model of Chapter 2, as long as families consider the full lifetime costs and benefits of hav-ing an additional child, this feature of the program is unlikely to leave the fertility decisions of families unaffected.

The grants are awarded to mothers everytwo months during the school calendar andall children over the age of 7 years andunder the age of 18 years are deemed eligi-ble. To receive the grant parents must enrolltheir children in school and ensure regularattendance (i.e., students must have a mini-mum attendance rate of 85 percent, bothmonthly and annually). Failure to fulfill thisresponsibility will lead to the loss of thebenefit, at first temporarily, but eventuallypermanently.

There are two forms that contain regis-tration and attendance information. Benefi-ciaries are provided with a form (E1) at thegeneral assembly that contains a list of thenames of eligible children. This has to betaken to the specific school where eachchild is to be registered and must be signedby a schoolteacher/director to certify enroll-

ment. This form is then returned to, and re-tained by, the district level PROGRESArepresentatives (UAEP) when the first pay-ment is collected. The second form (E2), formaintenance of detailed attendance records,is sent directly to the schools: one form perschool with names of registered childrentaken from the E1 forms returned by benefi-ciaries. Also, valid justification for absences(e.g., sickness) is to be maintained by theschool authorities with the cooperation ofparents’ associations.6

The amounts for the support of schoolmaterials differ according to educationallevel. For example, for the period of July toDecember 1999, for primary school stu-dents from beneficiary families, the supportconsists of 165 pesos, of which 110 pesosare paid at the beginning of the school yearand 55 pesos are paid halfway through the

4 CHAPTER 1

6Recent changes now mean that schools will return details only for those who do not meet attendance requirements.

Table 1.1 PROGRESA monthly cash transfer schedule (nominal pesos)

January–June July–December January–June July–DecemberGrant 1998 1998 1999 1999

Educational grant per child (conditioned on child school enrollment and regular attendance)Primary

Third grade 65 70 75 80Fourth grade 75 80 90 95Fifth grade 95 100 115 125Sixth grade 130 135 150 165

SecondaryFirst—male 190 200 220 240Second—male 200 210 235 250Third—male 210 220 245 265First—female 200 210 235 250Second—female 220 235 260 280Third—female 240 255 285 305

Grant for school materials per childPrimary—September — In-kind — 110Primary—January 40 — 45 —Secondary—September — 170 — 205

Grant for consumption of food per household (conditioned on attending scheduled visits to health centers)Cash transfer 95 100 115 125

Maximum grant per household 585 625 695 750

Source: Hernandez, Gomez de Leon, and Vasquez (1999).

school year (i.e., in January/February 2000),for the replacement of materials, as long aschildren continue to attend school. For sec-ondary school students, this support in-creases to 205 pesos and is delivered in asingle payment, at the beginning of theschool year, once pupils have enrolled.Children attending primary schools that aresupplied by the state-run Consejo Nacionalde Fomento Educativo (CONAFE) suppli-ers (under the Ministry of Education), thatis, essentially all schools except those lo-cated in very marginal communities receiveschool materials directly from their schoolsrather than a cash transfer. These are deliv-ered at the beginning of the school year andCONAFE informs PROGRESA whichschools received the school materials andhow much they received.

Description of the Healthand Nutrition ComponentThe health and nutrition component can beseen as a collection of a number of interre-lated subcomponents, namely:

1. A basic package of primary health careservices

2. Nutrition and health education andtraining for families and communities

3. Improved supply of health services (including annual refresher courses fordoctors and nurses)

4. Nutrition supplements for pregnant andlactating mothers and young children.

Although the general focus is on improvingthe health and nutritional status of all house-hold members, special emphasis is placedon the welfare of mothers and children.Some components are more important thanothers in this regard.

Primary Health Care ServicesThe basic approach of PROGRESA is thatof preventive health care that enables house-holds to anticipate both the causes and pres-ence of illnesses, with the objective of de-

creasing the incidence and duration of theseillnesses. This is reflected in the nature ofthe package of health services provided (seeTable 1.2). The most important actions arerelated to maternal and child health (e.g.,pre- and postnatal health care) and familyplanning services. A crucial ingredient inthe program is the emphasis placed on reg-ular visits to health centers and the settingup and monitoring of a schedule of appoint-ments. This includes the setting of appropri-ate health-center timetables that minimizethe inconvenience associated with the mak-ing and keeping of appointments. To facili-tate this, on registration at a health clinicbeneficiaries are given an appointmentsbooklet containing a specified schedule ofappointments for each household member,with particular attention placed on visits byvulnerable members, according to Table1.3. This information is entered on the S1form brought to the clinic by the benefici-ary, ensuring that a record of attendance byhousehold members is kept at the clinic.The other part of the form ( formato CRUS)is returned to the beneficiary, who uses it asproof of registration in order to receive cashgrants for food. For the period between Julyand December 1999 the value of the cashgrant for food consumption was 125 pesosper month.

BACKGROUND AND PROGRAM DESCRIPTION 5

Table 1.2 Composition of the basichealth services package

1. Basic sanitation at the family level2. Family planning3. Prenatal, childbirth, and puerperal care4. Supervision of nutrition and children’s growth5. Vaccinations6. Prevention and treatment of outbreaks of

diarrhea in the home7. Antiparasite treatment8. Prevention and treatment of respiratory

infections9. Prevention and control of tuberculosis

10. Prevention and control of high blood pressureand diabetes mellitus

11. Accident prevention and first aid for injuries12. Community training for health care self-help

Beneficiaries are also asked to attendhealth and nutrition talks (referred to aspláticas) at the clinic. Each clinic7 receivesan S2 form from the UAEP every twomonths that contains the names of benefici-aries as compiled from the CRUS form. TheS2 form, which contains only the benefi-ciary’s name with two columns (one forhealth center visits, another for attendanceat pláticas) for registering compliance ornoncompliance by the household, must befilled out by a nurse or a doctor at the healthunit every two months, certifying whetherfamily members visited the health units asrecommended (and presumably scheduled).This form is then submitted to the UAEPs,via the state health authorities (JuridicionSanitaria), in order to trigger the receipt ofthe bimonthly food support. In principle, ifat least one member did not comply withscheduled visits then the household is con-sidered not to have complied and thus willnot receive food support. However, sinceadults are asked to comply with only onevisit per year, if the appointment date ischanged in advance, the health center willfocus only on the compliance of women andchildren. Very often, though, adult memberscomplete their required visit at the time of

registration. Also, since a household mayvisit a clinic other than the one at which it isregistered, the UAEPs require informationfrom more than one clinic in order to regis-ter compliance correctly. This informationis entered onto a computer and a computer-ized file sent to CONPROGRESA.

Nutrition and Health EducationAn underlying assumption in PROGRESAis that effective health care requires activecommunity participation and a culture ofpreventive care. To empower individualsand communities to take control over theirown health, beneficiaries are required to at-tend nutrition and health education lectures(pláticas). Up to 25 themes are discussed inthe lectures, including nutrition, hygiene,infectious diseases, immunization, familyplanning, and detection and prevention ofchronic diseases. Because mothers are theprimary caretakers, the pláticas are directedmainly to them, but other members of bene-ficiary families as well as non-beneficiariesare invited to attend. Participants are trainedin various aspects of health and nutrition,with a special emphasis on preventivehealth care; more specifically they aretaught about: (1) ways to prevent and reduce

6 CHAPTER 1

7Regarding mobile clinics (unidad mobiles) that already existed in some localities, PROGRESA reached agree-ment with another program (Programas de Ampliacion de Cobertura) on a new frequency of visits to beneficiarylocalities in order to facilitate the expected increase in demand.

Table 1.3 Annual frequency of health care visits required by PROGRESA

Age group Frequency of check-ups

ChildrenYounger than 4 months Three check-ups: 7 and 28 days, and 2 months4 months to 24 months Eight check-ups: 4, 6, 9, 12, 15, 18, 21, and 24 months, with

one additional monthly weight and height check-up2–4 years Three check-ups a year: one every 4 months5–16 years Two check-ups a year: one every 6 months

WomenPregnant Five check-ups: prenatal periodDuring puerperium and lactation Two check-ups: in immediate puerperium and one during lactation

Adults and youths17–60 years One check-up per yearOver 60 years One check-up per year

health risks (e.g., through prenatal care,early detection of malnutrition, childhoodimmunizations, safe food and water treat-ment); (2) how to recognize signs or symp-toms of sickness; and (3) how to follow appropriate primary-care procedures (e.g.,treatment of diarrhea by means of oral rehydration). Participants are also trained in the use of the nutritional supplementsprovided by the program, as well as in optimal breastfeeding and complementaryfeeding of young children. Efforts are also made to broaden the information foradolescents and young people, particularlywomen, to favor the adoption of appropriatebehaviors to protect their health from anearly age.

Supply of Health ServicesAll public-sector health institutions are toprovide the package of basic health careservices. To facilitate this, especially in theface of anticipated increased demand, re-sources will be devoted to strengthening thesupply of health services as follows:

1. Ensuring adequate supply of equipmentto units

2. Encouraging staff working in remoterural areas to remain there on a long-term basis

3. Ensuring that health care units have thenecessary medicines and materials (including educational health materialsto distribute to families)

4. Providing extra training to improve both the quality of the medical attentionand the operational dimensions of theservice.

These resources are deemed necessary if thepublic health sector is to meet the additionaldemands placed on it by the program andprovide an efficient and high-quality ser-vice. Although the greatest efforts made bythe institutions involved will concentrate on

primary care, mechanisms will also be es-tablished for the timely detection and refer-ral (free of charge) of the beneficiaries whoneed attention in units at the second andthird levels of health care.

Nutritional SupplementsSpecial attention is given to the preventionof malnutrition in infants and small chil-dren, which is a crucial determinant of theirfuture development. Therefore, an addi-tional component of the program is the pro-vision of food (nutritional) supplements topregnant and lactating women and to chil-dren between the ages of four months andtwo years. These supplements will also begiven to children between two and fiveyears of age if any signs of malnutrition aredetected or to non-PROGRESA householdsunder similar circumstances.

Two different supplements were formu-lated specifically for the program: one forpregnant or lactating women and the otherone for young children. Both supplementscontain whole dry milk, sugar, maltodex-trin, vitamins, minerals, and artificial fla-vors and colors, but their specific macro-and micronutrient content is adapted tomeet the specific nutritional needs of moth-ers and children, respectively. The supple-ments are distributed in 240-gram packagesand are ready to eat after they are rehy-drated. The child supplement produces atype of pap and is available in banana,vanilla, and chocolate flavors. A 40-gramdaily ration (of dry product) supplies 194kilocalories, 5.8 grams of protein, and ap-proximately one recommended daily al-lowance (RDA) of selected micronutrients(see Table 1.4). The supplement for womenis intended to be consumed as a beverageafter rehydration, and is available in banana,vanilla, or natural flavor. The daily ration is 52 grams and provides 250 kilocalories of energy, 12–15 grams of protein, and se-lected vitamins and minerals.8

BACKGROUND AND PROGRAM DESCRIPTION 7

8A complete description of the design, formulation, and composition of the supplement is available in Rosado etal. (2000) and Rivera et al. (2000).

The supplements are prepared at oneproduction plant devoted solely to this taskand then distributed to health centersthrough Distribuidora Comercial CONA-SUPO (Compania Nacional de Subsisten-cias Populares) (DICONSA), which is anoperational arm of the Ministry of SocialDevelopment (Secretaria de Desarrollo So-cial [SEDESOL]) and also the largest dis-tributor of food in rural areas. There areabout 18,000 DICONSA stores in ruralareas. The supplements have a long shelf-life of about one year.

Mothers visit the clinic at least once amonth (more if they are pregnant or havesmall children) and are expected to pick upa one-month supply of the supplement foreach targeted household member. Appropri-ate use of the supplements and other con-cepts of optimal child feeding and feedingduring pregnancy and lactation are rein-forced during the nutrition and health pláti-cas provided in the clinics.

PROGRESA and Benefitsfrom Other ProgramsOne additional requirement of the PRO-GRESA program is that households benefit-ing from PROGRESA are supposed to stopreceiving benefits from other preexisting

programs. For example, according to theoperational guidelines of PROGRESA,households receiving PROGRESA benefitsshould not be receiving other similar bene-fits from programs such as Niños de Soli-daridad, Abasto Social de Leche, de Tortilla,and the National Institute of IndigenousPeople (Insituto Nacional Indigenista[INI]). This requirement of the PROGRESAprogram represents the short-run objectiveof the new poverty alleviation strategy ofthe Mexican government to minimize dupli-cation of benefits to poor families. A longerrun objective is to absorb the variety ofpoverty alleviation programs within oneprogram such as PROGRESA that repre-sents an integrated approach to poverty alleviation. Before the establishment ofPROGRESA, previous government inter-ventions in the areas of education, health,and nutrition in the rural sector of the coun-try consisted of many programs each inter-vening separately in health, education, ornutrition with little prior coordination orconsideration of the potential synergies thatcould result from a better coordinated andsimultaneous intervention.

Size of Monetary TransfersReceived by PROGRESABeneficiary HouseholdsFigure 1.1, based on the administrativerecords of PROGRESA containing the payments sent out to beneficiary families,reveals that there is a substantial variation inthe average cash transfer received permonth by beneficiary families.9 For exam-ple, the average cash transfer paid out tofamilies in December 1998, July 1999, andDecember 1999 were between 400 and 500pesos, amounts that are considerably higherthan those paid out in most other months. Inthe initial months of program implementa-tion, there were considerable delays in the

8 CHAPTER 1

9The sample used is the sample of beneficiary households in the 506 localities used for the evaluation of PROGRESA.

Table 1.4 Micronutrients contained in the food supplements

Pregnant and lactating women Children

Iron IronZinc ZincVitamin B12 Vitamin AVitamin C Vitamin CVitamin E Vitamin EFolic acid RiboflavinIodine Vitamin B12

Folic acid

processing of the forms necessary for pay-ment authorization. The unusually highpayments made during the months of De-cember and July are a consequence of PRO-GRESA’s efforts to catch up with the distri-bution of payments owed to the beneficiaryfamilies.

Figure 1.1 also reveals that the firstmonetary benefits associated with participa-tion in PROGRESA started in May 1998,covering, in principle, the first two monthsof participation in the program (i.e., Marchand April 1998). However, the fact that thefirst payments that were sent out to somehouseholds in May 1998 exceeded the max-imum bimonthly amount suggests that somehouseholds were incorporated before March1998 (e.g., in January 1998).10 Taking intoconsideration these factors, it was deter-mined that it would be more appropriate toderive the estimate of the average cash

transfer received based on the 12-month in-terval between November 1998 and October1999.

Actual average payments, in total and bycomponent, received over the 12-month pe-riod between November 1998 and October1999, along with data on household con-sumption averaged across all three rounds,are reported in Table 1.5. The averagemonthly transfers are around 197 pesos perbeneficiary household per month (expressedin November 1998 pesos). The calculationof this average includes households that did not receive any benefits because of non-adherence to the conditions of the program,or delays in the verification of the require-ments of the program or in the delivery ofthe monetary benefits. These transfers are19.5 percent of the mean value of consump-tion of poor households in control localities.On average, households receive 99 pesos for

BACKGROUND AND PROGRAM DESCRIPTION 9

10There is no record of the specific date that a household was incorporated into the program.

Figure 1.1 Average cash transfer paid out by PROGRESA by month

10 CHAPTER 1

Tabl

e 1.

5PR

OGRE

SA tr

ansf

ers

to b

enef

icia

ry h

ouse

hold

s fro

m N

ovem

ber 1

998

to O

ctob

er 1

999

(pes

os)

Ben

efic

iary

hou

seho

lds

Poo

r ho

useh

olds

res

idin

g in

con

trol

loca

litie

s

Tota

l val

ueA

vera

geT

rans

fers

ofA

vera

geA

vera

geA

vera

gem

onth

lyTo

tal

as a

cons

umpt

ion

mon

thly

mon

thly

mon

thly

scho

olex

pend

itur

espe

rcen

tage

of

Hou

seho

ld(f

ood)

tran

sfer

sal

imen

tobe

caut

iliti

esH

ouse

hold

(foo

d)no

n-be

nefi

ciar

ies

Rec

ipie

nt

size

[non

-foo

d]re

ceiv

edtr

ansf

ertr

ansf

ertr

ansf

ersi

ze[n

on-f

ood]

expe

ndit

ures

All

poor

hou

seho

lds

5.81

1190

197

9991

85.

4710

3919

.54%

(947

)(8

06)

[242

][2

33]

Hou

seho

lds

with

pre

-sch

ool-

aged

chi

ldre

n6.

5812

8920

210

193

86.

4110

9218

.7%

Hou

seho

lds

with

sch

ool-

aged

chi

ldre

n6.

5913

1123

910

112

811

6.40

1155

20.9

%H

ouse

hold

s w

ith h

eads

age

d 60

or

olde

r4.

3593

613

893

413

4.23

880

16.5

%

Sour

ce:

Cal

cula

tions

bas

ed o

n tr

ansf

er d

ata

prov

ided

by

PRO

GR

ESA

ave

rage

d ac

ross

the

12-

mon

th p

erio

d be

twee

n N

ovem

ber

1998

and

Oct

ober

199

9 (d

efla

ted

to N

ovem

ber

1998

pric

es).

Con

sum

ptio

n an

d fa

mily

siz

e av

erag

ed a

cros

s th

e th

ree

roun

ds o

f th

e E

NC

EL

sur

veys

in N

ovem

ber

1998

, Jun

e 19

99, a

nd N

ovem

ber

1999

.

food support (alimento), and 91 pesos forthe educational grant (beca). The alimentoaccounts for 68 percent of the transfers re-ceived by households headed by individuals60 years or older, a finding not surprising,given that such households will tend to havefewer children of school age.

Scope of EvaluationThe structure of the benefits and require-ments of the program naturally pose somelimitations on the kinds of questions that theevaluation can and cannot address. First, theevaluation of PROGRESA, as well as of anyother social program, requires a clear defi-nition of its objectives. Clearly specified ob-jectives provide a benchmark against whichthe performance of the program can be eval-uated. PROGRESA has multiple and inter-linked objectives. At the risk of oversimpli-fying, the objectives of PROGRESA are toalleviate poverty by inducing householdsthrough conditional cash transfers to investin their human capital, such as health, edu-cation, and nutrition.11 Clearly, the main ob-jectives of the program are long-run goalsthat can be evaluated only over the lifetimesof program participants. The PROGRESAevaluation data are limited to only two years of observations since the start of theprogram. This implies that the evaluation results presented herein can provide little information about the long-term conse-quences of the program on the human capi-tal and lifetime welfare of beneficiaries. Theevaluation of PROGRESA conducted byIFPRI is based on more short-term indica-tors of program impact on human capital,such as whether children from beneficiaryhouseholds are more likely to enroll or re-main in school or exhibit higher attendancerates and improved scores in educationalachievement examinations; whether benefi-ciaries make more frequent use of the health

services provided by the program; whethermorbidity among beneficiaries decreases;whether food consumption and nutrition atthe household level increases; and whetherthe intervention, especially on the nutri-tional side, has any measurable impact onthe nutritional status of children. In addi-tion, given that this is certainly an implicitobjective of PROGRESA, IFPRI’s evalua-tion includes the potential impact of thecash transfer component of the program onshort-run poverty measures and householdwelfare.

Second, it is important to note that theeducational and health services of the pro-gram as well as the nutritional supplementand pláticas are all provided as a package.This feature of the program makes it impos-sible to evaluate the impact of individualprogram components (e.g., on the impact ofthe health component of the program onschool attendance) or shed any light on pro-gram design (e.g., what if the cash transferswere awarded to fathers instead of mothers).It is certainly possible that households canchoose to comply with some of the require-ments of the program such as visiting healthcenters and not with others, such as en-rolling their children of eligible age intoschool. Although selective take-up of spe-cific program components is a real possibil-ity, this is an issue not directly addressed inthis evaluation but left for analyses of theprogram in the future.

Lastly, although PROGRESA is pri-marily a demand-side program, meaning thatits main objective is to induce households(through cash transfers and conditions asso-ciated with the receipt of these cash trans-fers) to make more intensive use of the ex-isting educational and health facilities, it isimportant to keep in mind that it is also ac-companied by complementary efforts andresources directed at the supply and qualityof the educational and health services. Thus

BACKGROUND AND PROGRAM DESCRIPTION 11

11See Skoufias, Davis, and Behrman (1999a,b) for a more detailed presentation of the stated objectives of thePROGRESA program.

although the program does not aim to in-crease the quantity of educational andhealth facilities (such as building newschools and health centers), it does try to an-ticipate and ease potential capacity con-straints that might arise as a result of themore intensive use of the existing facilities.Since these increased resources related tothe quality of services are part of the overallPROGRESA benefit package provided, theevaluation of the program can provide littledirect evidence on whether a demand-sideintervention is more effective (in terms ofimpact and/or in terms of cost) relative to asupply-side intervention.

Chapter 2 of this synthesis report con-tains an economic framework that is usefulin understanding the potential impacts of

the program. Chapter 3 presents the experi-mental design and the information sourcesused in the evaluation of PROGRESA.Chapter 4 discusses the evaluation of PRO-GRESA’s targeting and its impact onpoverty. Chapter 5 summarizes the econo-metric methods used to evaluate the impactof the program, while Chapter 6 contains asummary of the quantitative and qualitativeresults of the evaluation of PROGRESAalong with the cost analysis of the program.Chapter 7 contains a summary of the policyconsiderations derived from the evaluationof the program. Readers seeking a more de-tailed description of the research, rationale,and methods may consult the original IFPRIreports that are contained in the CD en-closed with this publication.

12 CHAPTER 1

C H A P T E R 2

PROGRESA Seen through an Economic Lens

The major component of IFPRI’s evaluation of PROGRESA focuses on the identifica-tion of the impact of the program (i.e., reductions in poverty levels, increased schoolenrollment and attendance, increased use of health services for preventive care, and im-

proved nutritional status). Knowledge of program impacts is an essential component of anyeconomic evaluation. However, in isolation impact evaluation provides limited guidance forpolicy. For this reason, an analysis of the costs and the cost effectiveness of the program is alsocarried out. A number of policy instruments could be employed to generate a given impact,and these may differ substantially in terms of cost. Cost-effectiveness analysis quantifies thecosts associated with bringing about a given impact. This aspect of policy choice is particu-larly important when budget allocations are tight.

In general, a complete economic evaluation of a program of the nature of PROGRESA re-quires not only the identification of the impacts of the program, and the costs of bringing aboutthese impacts, but also a comparison of these two key factors in order to determine the overallwelfare impact of the program and how effectively the program achieves these welfare impactsrelative to alternative policy instruments. This immense task typically requires the measure-ment of the benefits associated with higher investments in human capital. Assigning a mone-tary value to the increased nutrition, health, and education of a child over his or her lifetime asa result of the social program requires a series of assumptions that stretch the limits of credi-bility. Nevertheless, in some instances, assumptions of this nature are made in order to providereaders and policymakers with a rough quantitative estimate of the benefits of the program.

With these caveats in mind, the first part of this chapter outlines the economic frameworkthat summarizes some of the key determinants of household investment in human capital andthe ways in which participation in PROGRESA may influence these decisions. In very simpleterms, households have preferences that are summarized by a welfare function; a set of con-straints, such as expenditures cannot exceed income; and a set of variables, some of which areunder the control of the agent (endogenous or choice variables) and some are taken as given(exogenous variables or parameters). The main objective of a household is to determine thevalues for the variables that are under its control so as to get the maximum level of welfare aspossible while at the same time satisfying the constraints faced. The key feature of this eco-nomic framework is that a household will determine all its choice variables so that the ratio ofthe marginal benefit (MB) to the marginal cost (MC) associated with a small change in eachof its choice variable is equated across all choice variables.

In the remainder of the chapter, the main insights derived from this economic frameworkabout the direct as well as indirect impacts of the program, the nature and the size of these im-pacts, as well as some of the factors that could limit the impact of the program are discussed.

13

An Economic Model ofHuman Capital Investmentwithin HouseholdsThe design of the PROGRESA program andthe structure of its cash benefits and re-quirements suggest that the program is wellaware of the direct costs involved in induc-ing households to invest in human capital.For example, the size of the educationalgrant varies with child gender and age and isbased on the labor income children con-tribute to households. In addition, the factthat the educational benefits are given forchildren older than seven years of age sug-gests that the design of the program is alsocognizant of the possible indirect effects ofthe program on fertility.

This section presents a model of house-hold decision making that highlights thevarious costs and benefits associated withthe decision to invest in the human capital ofchildren. The model is sufficiently flexibleto embody the production of human capitalby heterogeneous households (Rosenzweigand Schultz 1983; Rosenzweig 1988), therole of the mother’s time (Willis 1974), theinteraction between child quantity and qual-ity in the household budget constraint(Becker 1981), the economic value of chil-dren (Rosenzweig and Evenson 1977), andthe biology of reproduction (Rosenzweigand Schultz 1983) emphasized in prior stud-ies formulating models of the household.

To simplify the presentation, it is as-sumed that households have full informa-tion and collapse all the decisions of thehousehold made early in life and the out-comes of these choices in the adult life ofchildren into one period. Fertility is initiallytreated as exogenous. Later the model isamended to allow households to make deci-sions about the number of children theyhave and considers the possible interaction

effects of PROGRESA with fertility. Themodel is also a unitary model, which meansthat it treats the household as if it were max-imizing a single welfare function withoutspecifying exactly whether this welfarefunction reflects the preferences of the adultmale or the mother in the household.

Cleary, most of these assumptions maybe questioned on the grounds that they im-pose some strong or unrealistic restrictionson household behavior. For example, thecollapsing of the life cycle of the householdinto one period may be less acceptable forpoor rural economies characterized by im-perfect credit markets, liquidity constraints,and limited possibilities of ensuring house-hold consumption (Jacoby and Skoufias1997, 1998). In addition, the assumption ofa unitary household may be subject to criti-cism as attested by the amount of theoreticaland empirical work that has been conductedon the alternative model of collective deci-sion making within families (e.g., Haddad,Hoddinott, and Alderman 1997).

Although it is important to acknowledgethe limitations of this model, it is also quiteinstructive to present it as a means of obtain-ing a better understanding of the pathwaysthrough which PROGRESA might influencehousehold behavior and its investments onthe human capital of children.

For the purposes of keeping the modelsimple, the term human capital will be usedto summarize the investments of families inboth education and health. One essentialfeature of the model is that human capital(H) per child is produced by the householdusing as inputs the time of family membersand other goods and services purchasedfrom the market.12 The function describingthe effects of changes in household re-sources on the level of human capital in-vested in each child is given by

14 CHAPTER 2

12In reality, since families produce more than one form of human capital simultaneously, there may be some im-portant feedbacks or synergies involved in the production of education and health. The health status of a child, forexample, may be an important factor in the child’s school attendance rate. In order to keep the model simple, thesetypes of synergies are left out of the model but are discussed in more detail later.

H = h(tcH, tm

H, X; Z, µ, K). (1)

The first partial derivatives for the first threearguments of the human capital productionfunction are assumed to be positive (i.e., h1,h2, h3 > 0). These restrictions on derivativesof the production function are equivalent toassuming that as children or their mothersdevote more time to schooling the stock ofhuman capital embodied in children in-creases. Here, three important human capi-tal inputs are highlighted: the time of thechild tc

H (in school, medical care), the timeof the mother tm

H, and purchased goods andservices X (e.g., books, medical care). Thehuman capital production function, equa-tion (1), also contains the terms Z, µ, and K.The term Z summarizes observable childcharacteristics such as gender or the birthorder, which also directly but exogenouslyinfluence H. The term µ captures, for exam-ple, the influence of biological factors, pos-sibly genetically transmitted, such as childability or health endowment, which also di-rectly but exogenously influence H. Typi-cally, the term µ can be observed by the par-ents of the child but is unobservable tooutsiders. The third term, K, reflects the roleof parental education; community charac-teristics such as distance from the market,health, or educational center; environmentalfactors; and the general availability ofknowledge and information about the pro-duction of human capital. It is possible thatsome of the components of K may act assubstitutes or complements for each other.For example, parental education may be asubstitute for the lack of information avail-able about sanitary practices. Thus both thehuman capital “endowment” and increasedaccess to relevant information about humancapital production may influence householddecisions. For example, increased aware-ness about sanitation, proper cooking meth-

ods that retain the nutrients in food, andother health maintenance practices can af-fect the productivity of the other inputs.

The income of an adult child is assumedto be determined by the stock of human cap-ital accumulated through parental invest-ments. Thus child earnings when he or shebecomes an adult denoted by E are

E = αµ + βH = αµ + βh(tc

H, tmH, X; Z, µ, K), (2)

where α is the market return to the geneticendowment of an individual and β is themarket rental rate on accumulated humancapital.

The budget constraint incorporates thepossibility that children contribute incometo the household when not engaged inhuman capital accumulation (e.g., inschool) and parents receive some fraction θof the earnings of “grown” children. Specif-ically, the budget constraint of the house-holds is

V + Wc (Ω – TcH)N + Wm(Ω – Ntm

H) + θNE= NpxX + Y, (3)

where N denotes the number of children in the household, V is non-employmentsources of income including the labor in-come of adult men in the household, Wc iswage rate of children, Wm is the wage rateof the mother, Ω is time available, px is theprice of X, and Y is household consumption(assumed to be the numeraire) excluding thepurchased goods and services for humancapital accumulation.13

Finally, parents are assumed to “careabout” the number and adult earnings oftheir children, and the level of householdconsumption.14 These parental preferencescan be summarized by the parental welfarefunction,

PROGRESA SEEN THROUGH AN ECONOMIC LENS 15

13Note that the health of the family members may also be modeled as increasing the amount of the time endow-ment of the family.

U = U(E,Y), (4)

which is assumed to possess the usual neo-classical properties.15

Assuming that parents maximize (4)subject to (1)–(3) by choosing the levels ofX, Y and by allocating parental (tm

H ) and childtime (tc

H) across activities, the first-order nec-essary conditions from the optimizationproblem of the household for each of its con-trol variables are (in addition to the budgetconstraint described by equation [3]):

UE Wc

MRSEY = ––– = N––– – θ = MCt cH

. (5)UY βh1

UE Wm

MRSEY = ––– = N––– – θ = MCt mH

. (6)UY βh2

UE PxMRSEY = ––– = N––– – θ = MCx . (7)UY βh3

Expressions (5), (6), and (7) highlight thefact that at the optimum households equatethe marginal rate of substitution betweenadult children’s earnings and house-hold consumption (denoted by the ratio ofthe partial derivatives of the utility func-tion with respect to E and Y ) with the marginal cost (MC) or “shadow price” ofinvesting in the human capital of a child.In addition, the combination of these three equations implies that householdswill allocate child time (tc

H), parental (tmH )

time, and market resources (X) so as toequalize the marginal costs associatedwith each activity and resource (i.e., MCt c

H

= MCt mH

= MCX).

For example, expression (5) impliesthat the marginal cost of children’s time inhuman capital production depends posi-tively on Wc, the wage rate children couldearn (opportunity cost of time in school),and negatively on the marginal increases inearnings associated with a unit increase in school time. Moreover, with all elseequal for households with a larger numberof children (higher N), the marginal cost ofinvesting in child human capital is higher.Along similar lines, the MC of the time amother allocates to human capital produc-tion depends on the wage rate of the motherand the marginal productivity of her time

in the production of human capital. Incombination, expressions (5) and (6) implythat at the optimum the household will al-locate the time children and mothers spendin human capital production so as to equal-ize the marginal costs associated with thesetwo activities.

Changes in non-employment income Valone leave the shadow prices of the re-sources unchanged since V does not enterdirectly into any of the expressions (5)–(7).Provided that E and Y are “normal” com-modities, increases in V result in “pure in-come effects” that increase human capitaland consumption. In contrast, changes inany of the factors that affect the marginalcost of time and goods used in producinghuman capital can trigger substitutionsamong the resources used in human capitalproduction as the household minimizes itsproduction costs and maximizes its welfareby using more of the input whose shadowprice decreased and less of the input whoseshadow price increased.16

16 CHAPTER 2

14In this specification of parental preferences, parents value child human capital only by its effect on the adultearnings of children. Another feasible specification is that parents care about the stock of their children’s humancapital directly (e.g., parents derive direct pleasure from having healthier or more educated children).

15Meaning that it has positive partial derivatives for each of its arguments and that it is strictly concave.

16See Behrman and Knowles (1999) for a similar approach to the determination of human investments withinfamilies.

The Conditionality of Cash TransfersAt the risk of oversimplifying, one of the keyfeatures of PROGRESA is that the cashtransfers paid by the program are condi-tioned on school attendance and visits tohealth centers. Simple economic theory sug-gests that a household is generally better offreceiving an unconditional cash transfer thana conditional cash transfer. The reason forthis lies in the fact that conditionalities in-duce households to make choices that aredifferent from those that they would make ifthey were given the cash transfer uncondi-tionally and allowed to use it as they pleased.One possible justification for the impositionof these requirements may be that significantmarket failures in rural economies tend tomake poor households invest less in educa-tion or in health than would be best in the so-ciety’s point of view.17 In the presence ofmarket failures and other externalities, theconditionality of cash transfer schemes canbe considered as an effective means of im-proving efficiency. Requiring that benefici-ary households fulfill some minimum re-quirement in school attendance and visits tohealth centers may result in a gain in socialwelfare that is greater than that obtainedfrom an unconditional cash transfer.

Based on the model outlined in the pre-vious section, it is instructive to followthrough some of the pathways in which thiskey characteristic of PROGRESA can im-pact on the investments of families in thehuman capital of their children. Such an ex-ercise, at a minimum, provides useful guid-ance about the cases or types of householdsin which impact can be expected.

Consider, first, the cash transfers bythemselves, ignoring for the moment that

these transfers are awarded conditionally. Inthis very simple example, participation inthe program increases the term V in equa-tion (3) while leaving the determinants ofthe marginal costs unaffected. Then the cashtransfers of PROGRESA act as an incomeeffect that tends to increase the human cap-ital invested in children.

Next consider the requirements associ-ated with the program. Participation in and compliance with the conditions ofPROGRESA are likely to result in changesin the shadow price or marginal cost of in-vestment in human capital.18 For example,consider a household with a child enrolledin school and with an attendance rate less than the 85 percent rate required byPROGRESA. Assuming full compliancewith the requirements of the program, thechanges in the amount of time the child andthe mother devote to schooling, and in theamount of the school supplies X (such astextbooks, pencils, and paper) made avail-able by the program are likely to change themarginal costs or shadow prices of thehousehold. Specifically, consider the impactof the program on the MC of tc

H (see equa-tion [5]). Even though the extra time thechild devotes to schooling has a cost interms of the lost child wage Wc, what mat-ters to the household is the ratio of childwage to the marginal increase in earningsgiven by the term βh1(tc

H, tmH, X; Z, µ, K) that

enters in the denominator. Thus the impactof PROGRESA on the MC of tc

H is deter-mined by how the marginal product of thetime of a child in human capital is affectedby various components and requirements ofthe program (i.e., the signs of the secondown and cross-partial derivatives (i.e., h11,h12, h13). The higher amount of tc

H required

PROGRESA SEEN THROUGH AN ECONOMIC LENS 17

17See Das, Do, and Ozler (2004) for an interesting discussion of the equity and efficiency trade-offs involved inconditional cash transfer programs.

18In principle, an eligible household’s decision of whether to participate in the program or not can be modeled ascomparing its maximum utility level outside the program with the utility level associated with participating in theprogram, adhering to all the conditions of the program imposed on the time allocation of children and mothersand receiving the cash transfers provided by the program.

by the program is likely to decrease h1in ex-pression (5) given the diminishing marginalproductivity of own time in human capitalproduction (i.e., h11 < 0). However, this neg-ative effect on h1is likely to be counteractedby the enhanced productivity of the child’stime resulting from the increased time spentby the mother in producing child educationand the larger number of textbooks avail-able (i.e., h12 > 0, h13 > 0).19

Whether the marginal cost of time de-creases or increases depends on how strongthese effects are in relation to each other.Since most of the program componentswork to increase the marginal product oftime in human capital production it is safe tosay that the program is likely to decrease theMC of time and thus result in further reallo-cation of inputs within the household. Inother words, the requirements of the pro-gram tend to generate additional shadowprice effects that lead to further substitutionand income effects that have the potential ofreinforcing the income effect resulting fromthe receipt of monetary benefits.

Figure 2.1 illustrates some of these ef-fects graphically. The vertical axis of thegraph depicts the quantity of other goodsavailable for consumption in the household,whereas the horizontal axis measures thetime a child devotes to schooling (or inhuman capital investment). Full or 100 per-cent attendance rate occurs when the childdevotes all non-leisure time in school atten-dance (including school-related homework)(i.e., S = T where T denotes the amount oftime available after excluding leisure timewhich for simplicity is assumed to be fixed).The vertical line of height V at the value ofS = T denotes the maximum amount ofother goods available in the household when

a child devotes all of his or her time toschooling and not working. When a childdivides his or her time between work andschooling then the line TVA describes theopportunity set of the household. The nega-tive slope of this line is given by the realmarket wage W for child labor, which de-scribes the tradeoff in the market betweenthe consumption of other goods and school-ing (or work).20 By devoting one hour lessin schooling and working one extra hour inmarket work the household can earn W ad-ditional units of other goods.

Let Smin denote the 85 percent atten-dance rate required by the PROGRESA program. Eligibility for the benefits of PROGRESA causes the budget line in theregion between points T and Smin to shift upwithout changing its slope and increases thenonlabor component of income upward tothe point V′. To the extent that the house-hold fulfills all the requirements of the program then V′ – V equals the maximumamount of benefits that the household canobtain from the program. In consequence,the feasible budget constraint of an eligiblefamily is now described by the line TV′A′BAthat is discontinuous at the point Smin.

Of course, differences in family non-earned income and market opportunitiesmay be one important reason why somechildren are enrolled or not enrolled inschool. To keep the exposition simple, weassume that the income opportunities ofhouseholds are identical and consider thecase when we have two different types ofhouseholds represented by two different in-difference curves. The household denotedby the tangency at point C represents house-holds with a child that has an attendancerate close to 100 percent (S > Smin) and

18 CHAPTER 2

19Note that in the case of health production the pláticas may enhance the marginal productivity of time further(h1K > 0).

20It is assumed that the opportunity cost of child schooling is the fixed market wage for child labor. The as-sumption of a perfectly competitive labor market can be replaced by (or combined with) the assumption that chil-dren work at home producing home-produced commodities that are perfectly substitutable with market-purchasedcommodities with no additional complications (see Skoufias 1994a).

works only a very small fraction of his orher time. The indifference curve that crossesthe vertical axis at point A represents house-holds with a child who does not attendschool at all (S = 0) and devotes all of his orher free time to market work. Although itdoes not have to be so, for simplicity, pointA is depicted as a tangency point betweenthe indifference curve of the household andthe real wage line W.

The discontinuity of the budget con-straint of the household, in combinationwith the assumption of utility maximiza-tion, implies that there is a minimum condi-tional cash transfer that will induce thehousehold to send its child to school. Let B′ denote the point of intersection of the in-difference curve of household A with the

vertical line at Smin. Then the vertical differ-ence B′ – B represents the minimum cashtransfer that will make household A just indifferent between complying with the 85percent attendance requirement and keep-ing their child out of school. A conditional cash transfer less than B′ – B is insuffi-cient to induce the household to enroll itschild in school. This is because by having its child work, the family gets a higher levelof utility compared to sending the child toschool.

In Figure 2.1, it is implicitly assumedthat the size of the conditional cash transferV′ – V is greater than the minimum amountB′ – B needed to induce household A to en-roll the child in school and comply with the85 percent attendance requirement. In con-

PROGRESA SEEN THROUGH AN ECONOMIC LENS 19

Other goods

Schooling

Work

A

W

B

A

BC

V

W*

V*

C

V

T SSmin

Figure 2.1 Effect of conditional cash transfers on children’s school attendance and work

Note: A, Initially not attending; C, initially attending full time; T, maximum amount of time available excludingleisure; Smin, program’s required school attendance.

sequence, household A finds it to its advan-tage to enroll the child in school. As can beinferred from this figure, participation in theprogram is likely to affect households dif-ferently depending on their location on thebudget line before the administration of theprogram. Consider household C, for exam-ple. Such a household can represent house-holds with children of primary school agewhere enrollment rate is close to 95 percentor the households with children of second-ary school age who are regularly attendingschool even before the administration of theprogram. Since the conditions are not bind-ing, the program is likely to have only apure income effect represented in Figure 2.1by the parallel upward shift in the portion ofthe budget constraint between points T andSmin.21 For these households the impact ofthe program may be concentrated at in-creasing the time they devote to schoolingsuch as spending more time studying ratherthan enrollment.22

For a contrast, consider household A.With the cash transfer conditioned on an 85percent attendance rate, and with the amountof the cash transfer greater than the mini-mum cash transfer B′ – B such a householdwill choose to send (enroll) its child toschool. This new equilibrium is representedby the point A′. The same household, giventhe same cash transfer of V′ – V, and withoutthe minimum requirement of an 85 percentattendance rate, might choose an attendancerate that is lower than 85 percent and achievea higher level of welfare (i.e., would be on ahigher indifference curve) than at point A′.One plausible justification for introducing“distortions” on the choices of poor house-holds is the role of significant market fail-ures and externalities that create differences

between individual and social welfare. Aslong as such market failures are prevalent,the social gains from making cash transfersconditional are likely to exceed the sum ofthe individual welfare losses from the impo-sition of these distortions.

Continuing with the discussion of house-hold A, at first sight it would appear that for this household it is very hard to attributeincome and substitution effects to the pro-gram since the final equilibrium point A′ isnot a tangency point. Yet, one can still applythe familiar concepts of income and substi-tution effects using the analytical frame-work of “linearizing the budget constraint”(discussed in detail in Killingsworth 1983).Linearizing the budget constraint amountsto transforming point A′ into a tangencypoint by drawing a line tangent to the indif-ference curve at A′ (i.e., finding the shadowwage W*) and finding the correspondinglevel of non-earned income (or shadow in-come) V* that corresponds to the shadowwage W*. As it becomes apparent, house-hold A’s participation in the program resultsin both substitution and income effects thattend to reinforce each other. The cash trans-fer component of the program leads to apure income effect that increases schooling,while the condition that the child devote atleast 85 percent of his or her time in schoolleads to a price effect. Based on standardeconomic theory the price effect may be further decomposed into a substitution andincome effect. At the final equilibrium point A′ the lower shadow wage W* (<W)represents the lower price of schooling as aresult of the program while the total in-crease in household income as a result of theprogram may be considered to be the cashtransfer V′ – V plus the implicit extra in-

20 CHAPTER 2

21In terms of the more detailed human capital model discussed earlier, the program will have negligible effectson the marginal product of the child’s time at school. Insofar as the cross-effects of the other program inputs arenegligible (i.e., h12 = 0, h13 = 0) then the MC of its time would be unchanged.

22It should be noted that the program may also have important dynamic effects by increasing the probability thatchildren continue on to higher grades in school. These dynamic effects of PROGRESA are explored by Behrman,Sengupta, and Todd (2001).

come V* – V′ earned as a result of the lowerprice of schooling.23

To summarize, the economic frameworkpresented above implies that participation inthe program is likely to affect householdsdifferently depending on their constraintsand preferences (or location on the budgetline) before the implementation of the pro-gram. For households for which the programconstraints are binding, the program islikely to result in income and substitutioneffects that tend to reinforce its impact. Incontrast, for households for which the con-straints of the program are nonbinding, theprogram is likely to have only income ef-fects. Given the heterogeneity of house-holds’ preferences and constraints, the ex-tent to which the program has a significantimpact on the human capital and work ofchildren can be determined only throughempirical analysis.

Additional Considerationsand Topics in the Evaluationof PROGRESA

SynergyOne important assumption in the design ofPROGRESA was that positive synergiesamong interventions affecting differenttypes of human capital, nutrition, health,and schooling are important. Two distinc-tions are useful in considering possible syn-ergies among human resource investments.First, there is the distinction between pro-duction function synergies and total syner-gies. The former refers only to whether theproduction function technology implies thattwo inputs are complements (positive syner-gies) or substitutes (negative synergies).The latter incorporates all behavioral adjust-ments to a change affecting one human re-

source investment and, depending on all pro-duction technologies and preferences thatare relevant for a household’s decisions, mayimply larger or smaller synergies than thepure production function synergies. Second,there is the distinction between synergiesamong human resources that are more orless concurrent (e.g., current nutrient intakesmight increase the effectiveness of currenttime in school in learning) and lagged effectsover the life cycle (e.g., infant malnourish-ment might affect adult productivities).

The PROGRESA evaluation data arenot well suited to investigate much aboutsuch possible synergies. Given that the nu-tritional intervention focused on children0–5 years of age, and the educational inter-vention is focused on children 8–18 years ofage, there is no way to determine or quan-tify the impact of the nutritional interven-tion at an early age on the educational andcognitive achievement of these children.The PROGRESA data also do not includecritical information about various possiblyrelevant human resources and related out-comes for the same individuals. For exam-ple, for infants and small children they in-clude some measures of nutrition, but not ofcognitive development. For children inschool, they include information on schoolenrollment, attendance, and test scores, butnot on longer-run health and nutrition statusor on short-run nutrient intakes. For adultsthey include information on school attain-ment and, for those who receive them, wagerates, but not on longer-run cognitive devel-opment or longer-run health and nutrition oron shorter-run nutrient intakes. Also, ofcourse, given that individuals are followedover three years at most, effectively individ-uals cannot be followed across life-cyclestages. Therefore, although analysis of

PROGRESA SEEN THROUGH AN ECONOMIC LENS 21

23In terms of the model above the attendance requirements of the program will affect the marginal product of thechild’s time at school. Assuming that the positive cross-effects on the household, the productivity of the child’stime resulting from the longer time spent by the mother in production of child education, and the larger numberof textbooks available (i.e., h12 > 0, h13 > 0) are greater than the negative effect of the higher attendance require-ment on h1 (h11 < 0), then the MC of its time is likely to decrease; see equation (5).

the PROGRESA data can provide useful information about some pieces of human resource effects that may be helpful in understanding possible synergies, the PROGRESA data in themselves cannot pro-vide much insight into the importance ofsuch synergies or even whether most ofthem exist.

However, an extensive review of the cur-rently available nutrition/epidemiologicaland socioeconomic literatures by Behrman(2000) reveals that human resource invest-ments in nutrition, health, and schooling doreflect considerably behavioral decisions atthe household level. Therefore, preferencesand other constraints matter, not just pureproduction function characteristics. In fact,the few available estimates directed to thisissue indicate that parental preferences aresuch as to reinforce differentials amongtheir children so that the total synergistic ef-fects are likely to be greater than the pureproduction function effects. This literaturedoes not include much persuasive evidenceon more-or-less concurrent synergies duringthe preschool and school-age stages. Butthere does seem to be evidence of signifi-cant positive synergies between concurrentshort-run nutrition and schooling in terms ofadult wages and productivities. More im-portant from the aspect of the human re-source emphasis in PROGRESA, there alsoseem to be cross-life-cycle-stage positivesynergies, particularly regarding the impactof preschool nutrition on schooling successand possibly on adult wages and productiv-ities. Illustrative simulations based on theavailable estimates of the impact of humanresources on outcomes of interest and on the persistence of human resources for indi-viduals over their life cycles suggest thatsuch synergies may importantly increase the returns of human resource investments,through a number of channels, of the typesemphasized by PROGRESA beyond the effects of the individual human resource

investments. Indeed the whole impact of the combined PROGRESA interventions in nutrition, health, and schooling is likelyto be significantly more than the sum of theparts.

FertilityImprovements in the “quality” or the humancapital embodied in children may also havean effect on the “quantity” or the number ofchildren families would like to have. For ex-ample, changes in fertility could be one ofthe unanticipated consequences of the pro-gram (e.g., Rosenzweig and Wolpin 1982).It is quite straightforward to extend the sim-ple human capital model presented earlier toallow for households to determine their fer-tility endogenously (i.e., the number of chil-dren N enters as a direct argument of theutility function of the household) as in

U = U (E, Y, N). (8)

Fertility is a biological process; resourcesmust be used by households to limit the sup-ply of births rather than to increase supply,as for most other “goods.” This can be ex-pressed in its most basic form by using theconstruct of a reproduction function, as in

N = φ + n(Z), n′ < 0, (9)

where N = number of births (children), Z =resources used to control births, with n′< 0, and φ = fecundity, the number of birthsthat would occur in the absence of control(Z = 0).24 The household chooses its level ofcontrol Z, but fecundity is biologically de-termined.

With the addition of the reproductionfunction and its determinants, the budgetconstraint equations changes to

V + Wc(Ω – tcH)N + Wm(Ω – Ntm

H ) + θNE= NpN + NpxX + pzZ + Y, (10)

22 CHAPTER 2

24Other reproductive inputs, for example, age and breastfeeding, can readily be incorporated.

where pN is the direct cost of having a childand pZ is the price of the Z good. Then par-ents may be modeled as maximizing theirwelfare function (8) subject to (1)–(3) and(9) and (10) by choosing the levels of Z, X,and Y and by allocating parental and childtime across activities.

In this revised maximization problem,the same MC expressions presented above,that is, expressions (6) and (7), still remainvalid, whereas the marginal cost or shadowprice of having an additional child is givenby the expression

MCN = πN = PN + Wmtmh + pxX

– θE – Wc(Ω – tcH)+ pz /n′ (11)

Expression (11) indicates that the resourcecosts associated with the addition of onechild include the direct costs of a child, thevalue of the mother’s time in child care, the value of the purchased human capital in-puts X, the child’s contributions to thehousehold when young and when grown.Also in the numerator is the ratio of the per-unit cost of the fertility control resource tothe “effectiveness” of control denoted bythe derivative of the control function (9)with respect to Z.

Increases in the direct costs of children,adult female wages, or in the prices ofhuman capital inputs increase the marginalcost of having a child, whereas increases inthe children’s wages or in their potentialcontribution to the family as adults act to reduce the marginal cost of having a child.Moreover, reductions in the “costliness” offertility control—decreases in the purchaseprice of contraception, pz, and/or increasesin the effectiveness with which a given in-crease in the fertility control resource re-duces fertility (a change in the absolutevalue of n′)—influence in the same way themarginal cost of fertility.

The revised model suggests that in-creases in the quality or the human capitalof children are likely to have an effect on themarginal cost of having an additional childand thus on the number of children desiredby families. For example, the basic healthpackage offered to households participatingin the PROGRESA program increases theeffectiveness with which a given increase inthe fertility control resource reduces fertil-ity, which tends to increase the MC of anadditional child. On the other hand, thehigher earnings of children with more edu-cation when they become adults tend to de-crease the MC of having an additional child.Which of the two effects dominates can bedetermined only empirically by observingfamilies over long periods of time. The em-pirical evidence available to date (Rosen-zweig and Wolpin 1982), however, suggeststhat rural households view schooling andchild health as complements, while thesetwo human capital characteristics of chil-dren are viewed by households as substi-tutes for fertility. This implies that the in-centives provided by PROGRESA forfamilies to invest in the health and educationof their children are mutually reinforcing al-ternatives and that they will over time tendto decrease fertility and population growthin rural areas.

Intrahousehold ResourceAllocation and Power and Status of Women within the HouseholdBy design PROGRESA gives transfers di-rectly to mothers. This decision is motivatedby growing evidence that resources con-trolled by women are more likely to mani-fest greater improvements in child healthand nutrition than resources placed in thehands of men (e.g., Haddad et al. 1997).25

PROGRESA SEEN THROUGH AN ECONOMIC LENS 23

25Duflo (2000) provides some of the first rigorous evidence that the impact of a cash transfer on children’s nutri-tional status is affected by the gender of its recipient. Specifically, she finds that pensions received by women inSouth Africa had a large impact on the anthropometric status of girls but little effect on that of boys. The pensionsreceived my men had no effects on the anthropometric status of either boys or girls.

The allocation of resources withinhouseholds is intricately related to the ques-tion of whether resources in the householdare controlled by adult men or women. Forexample, both unitary (e.g., Becker andTomes 1976) and collective models empha-size that a household is likely to allocate re-sources differentially among its children.Without introducing more complex nota-tion, in the context of the unitary model out-lined earlier, in families with more than onechild the amount of resources allocated toeach child is likely to depend on its healthendowment and ability summarized by theterm µ. As long as two children in a familyare endowed with different amounts of µ,households are likely to allocate differentresources to them even if all the other vari-ables affecting the decisions of householdsare the same for both children. As a conse-quence, policies and government interven-tions aimed at having a positive impact onspecific target groups such as girls versusboys may be weakened or neutralizedthrough adjustments in the distribution ofresources within the households. It is possi-ble, for example, that the availability of thenutritional supplements to younger childrenin households that are PROGRESA benefi-ciaries induces these households to decreasethe share of food allocated to children.Along similar lines, the cash transfers re-ceived by PROGRESA households maydisplace or “crowd-out” the remittancesbeneficiary households received from olderchildren and relatives working in the UnitedStates. Also, the loss of time in householdand on-farm productive time incurred by en-rolling eligible boys and girls in school mayplace serious time constraints on mothersand other household members as they try toreplace the time lost from children.

Incentive Effects and Impact on PovertyCash transfers, whether they are condi-tioned on some kind of household behavioror not, can have “incentive effects” on the

income obtained from work by adult house-hold members as well as “general equilib-rium” effects, meaning that the actual trans-fer of cash and the method used to financethese transfers may have secondary effectsthat could work to reinforce or weaken theeffects of the program. A naive approach tocash transfers is that they lead unambigu-ously to increases in household income andwelfare and reductions in poverty. The de-scription of the program requirements andthe model above suggest that the effect ofconditional cash transfers such as those as-sociated with PROGRESA may be morecomplex. The “pure income effect” of thecash transfers needs to be contrasted againstthe income losses or marginal cost increasesassociated with adhering to the requirementsof the program. The cash transfers house-holds receive may be just compensating forthe income lost by beneficiary householdsending their participation in other programssuch as Niños de Solidaridad, Abasto Socialde Leche, or de Tortilla. In addition, house-holds incur time costs when they adhere tothe requirements of the program. To the ex-tent that these costs are high, there is a pos-sibility that the cash transfers of the programhave no measurable effect on the income ofparticipating households or the poverty ratein these communities.

Another possibility is that the cashtransfers associated with the program arehigh enough to cover the income house-holds forego when they satisfy the require-ments of the program. In this case, house-holds may experience an increase in theirincome that in turn may affect the willing-ness of adult members to accept low-payingwork or physically demanding work. Incen-tive effects of this type have been empiri-cally documented in program evaluations inother countries (Sahn and Alderman 1995).One important implication of the precedingdiscussion concerns the impact of the pro-gram on measured poverty. Poverty mea-sures are typically based on measured in-come or consumption. It is possible thatpoverty measures based on income or con-

24 CHAPTER 2

sumption may show little or no impact onpoverty as long as households choose to usetheir cash transfers to “buy” more leisure.Under these circumstances, it is importantnot to ascribe any negative connotations tothe incentive effects of the program sincehouseholds may simply be choosing to in-crease their welfare by having more leisurerather than having higher consumption ofgoods and services. The report of Parkerand Skoufias (2000) makes an explicit effortto investigate the possibility of such incen-tive effects as a result of participation inPROGRESA.

Community and GeneralEquilibrium EffectsThe discussion so far has been limited toevaluating the effects of the program bysimply focusing on the behavior and humancapital outcome indicators of beneficiaryhouseholds. PROGRESA, however, canalso affect non-beneficiary households re-siding in the same community as well ashouseholds in other communities, urban orrural, where PROGRESA does not operate.The presence of PROGRESA in a commu-nity, for example, may affect the behavior ofnon-beneficiary households in that commu-nity through the “demonstration” or “peer-group” effect. It is also possible that theavailability of pláticas in localities coveredby PROGRESA may have spillover effectson the types of food consumed by non-ben-eficiary households as information abouthealthier foods and diets and better sanitarypractices spreads in the community. At thecommunity level, the selection of specifichouseholds into PROGRESA and the exclu-sion of others may introduce a new type ofsocial differentiation within communitiesthat could diminish social cohesion withinthese communities.

In addition to the effects of PROGRESAon communities covered by the program, it

is also important to recognize that PRO-GRESA may also have an indirect effect onthe welfare of households living in commu-nities not covered by PROGRESA. Whenone takes into consideration the fact that thecash transfers distributed by PROGRESAhave to be financed domestically, as they arein the case of Mexico, through the elimina-tion of distortionary price subsidies orvalue-added tax reforms, then the possibil-ity of a variety of indirect or multiplier ef-fects arises. A closer consideration of theseindirect effects in measuring program im-pacts in overall social welfare raises thepossibility that the first-round positive ef-fects of the program may be offset by thesecond-round negative indirect effects of theprogram.

To summarize, the economic frameworkpresented in the preceding paragraphs im-plies that participation in the program re-sults in income effects and in a multitude ofsubstitution effects that can reinforce theimpact of the program on participatinghouseholds. Depending on the specific cir-cumstances of the household; the con-straints it faces; and its preferences towardhuman capital, fertility, and consumption ofgoods and services, the substitution effectsinduced by the program may work againstthe positive income effect resulting from thecash transfer component of the program.The theoretical model also makes it clearthat it is necessary to adopt an empirical ap-proach to evaluating the impact of a pro-gram such as PROGRESA. Ultimately, thequestion of whether the program has a sig-nificant impact on the investments of house-holds in the education, health, and nutritionof their children can be determined only by observing the behavior of householdsparticipating in the program. The next chapter describes the quantitative and quali-tative methods and information sourcesused to evaluate empirically the impact ofPROGRESA.

PROGRESA SEEN THROUGH AN ECONOMIC LENS 25

C H A P T E R 3

The Experimental Design and InformationSources Used to Evaluate PROGRESA

T he central problem in the evaluation of any social program is the fact that householdsparticipating in the program cannot be simultaneously observed in the alternative stateof no treatment. To illustrate, let Y1 be the outcome for a given individual or household

in the treated state (i.e., during or after participation in the program) and Y0 be the outcome inthe untreated state (i.e., without participating in the program). Then the gain for any given in-dividual or household from being treated by the program is ∆ = (Y1 – Y0). At any given time,however, a person is either in the treated state, in which case Y1 is observed and Y0 is not ob-served, or in the untreated state, in which case Y1 is not observed and Y0 is observed. Giventhat missing Y1 or Y0 preclude measurement of this gain for any given individual, one has toresort to statistical methods as a means of addressing this problem (e.g., see Heckman,LaLonde, and Smith 1999). The statistical approach to this problem replaces the missing dataon persons using group means or other group statistics, such as medians.

For example, the majority of the studies on evaluation of social programs focus on thequestion of whether the program changes the mean value of an outcome variable among par-ticipants compared to what they would have experienced if they had not participated. The an-swer to this question is summarized by one parameter called “the mean direct effect of treat-ment on the treated.” Using formal notation, the mean effect (denoted by the expectationoperator E) of treatment on the treated (denoted by T = 1) with characteristics X may be ex-pressed as

E(∆ | T = 1,X) = E(Y1 – Y0 | T = 1,X) = E(Y1 |T = 1,X) – E(T0 | T = 1,X). (12)

The term E(Y1 | T = 1, X) can be reliably estimated from the experience of program partici-pants. What is missing is the mean counterfactual term E(Y0 | T = 1, X) that summarizes whatparticipants would have experienced had they not participated in the program.

The variety of solutions to the evaluation problem differ in the method and data used to con-struct the mean counterfactual term E(Y0 | T = 1, X). For example, one approach used frequentlyto evaluate social programs is based on the notion that all that is needed is repeated observa-tions on a set of households before and after the start of a program. Thus observations on thesame households before the implementation of a social program can be used to estimate E(Y0| T = 1, X). Another approach is that of social experimentation or randomization of individualsinto treatment and control groups. Experimental designs use information from individuals orhouseholds in the control group to construct an estimate of what participants would have ex-

26

perienced had they not participated in theprogram, that is, the term E(Y0 | T = 1, X).26

The empirical framework adopted bythe PROGRESA administration for the pur-poses of evaluating the program’s impactoffers a very flexible approach to solvingthe evaluation problem. Its advantages arederived from two key features. First, it is anexperimental design with randomization oflocalities, rather than households or individ-uals, into treatment and control groups.27

Second, data are collected from all house-holds in both treated and control localitiesbefore and after the start of the treatment.The combination of these two features per-mits researchers to evaluate the “mean di-rect effect of treatment on the treated,” or inother words the impact of the program onprogram participants using any of the esti-mators available in the evaluation literature,including the before–after estimator, the difference-in-differences estimator, and thefirst-difference or (cross-sectional) estimatordiscussed in more detail later in this chapter.

The expansion of the program across lo-calities and over time was determined by aplanned strategy that involved the annualbudget allocations and logistical complexi-ties associated with the operation of the pro-gram in very small and remote rural com-munities (such as verification that thelocalities to be covered by the program hadthe necessary educational and health facili-ties). In consequence, the expansion of theprogram took place in phases.28 In phase 1,

which began in August 1997, 140,544households in 3,369 localities were incorpo-rated. Phase 2 of the program began in No-vember 1997 when a further 160,161 house-holds in 2,988 localities were incorporated.The greatest expansion occurred in 1998(i.e., phases 3–6) when nearly 1.63 millionfamilies in 43,485 localities were incorpo-rated. By phase 11, the final phase of theprogram in early 2000, the program in-cluded nearly 2.6 million families in 72,345localities in all 31 states.

The experimental design used for theevaluation of PROGRESA takes advantageof the sequential expansion of the programin order to come up with a set of localitiesthat serve the role of controls. The sampleused in the evaluation of PROGRESA con-sists of repeated observations (panel data)collected for 24,000 households from 506localities in the seven states of Guerrero, Hi-dalgo, Michoacan, Puebla, Queretaro, SanLuis Potosi, and Veracruz. Of the 506 local-ities, 320 localities were assigned to thetreatment group (T = 1) and 186 localitieswere assigned as controls (T = 0).29 Specif-ically, according to Hernandez et al. (1999),the 320 treatment localities were randomlyselected using probabilities proportional tosize from a universe of 4,546 localities thatwere covered by phase 2 of the program inthe seven states mentioned previously.Using the same method, the 186 control lo-calities were selected from a universe of1,850 localities in these seven states that

EXPERIMENTAL DESIGN AND INFORMATION SOURCES 27

26For a more thorough discussion of the various solutions to the evaluation problem, see Heckman et al. (1999),Ravallion (1999), and Baker (2000).

27Valadez and Bamberger (1994) provide a detailed description of the elements of experimental and quasi-exper-imental approaches to program evaluation. For a review of evaluations of social sector programs using random-ized control designs in Mexico and other countries, see Newman, Rawlings, and Gertler (1994). In fact, themethod used by PROGRESA to select the eligible households also offers the opportunity to evaluate the impactof the program using a quasi-experimental design such as the Regression Discontinuity Design (Hahn, Todd, andVan der Klaauw 2001). See Buddelmeyer and Skoufias (2004) for an evaluation of the impact of PROGRESA onchild work and schooling using the RD design.

28For more details see Section 4 and Table 1 in Coady (2000).

29More details on the geographic distribution of the evaluation sample of localities and their characteristics areprovided in Appendix B.

were to be covered by PROGRESA in laterphases.30 As originally planned, the locali-ties serving the role of a control groupstarted receiving PROGRESA benefits byDecember 2000.

It is important to clarify that there is avery important difference between makingthe best use of the constraints involved in thecoverage of households and the “deliberatewithholding of benefits for the purposes ofthe evaluation.” Annual fiscal constraintsand logistical complexities associated withthe operation of a social program such asPROGRESA in very small and remote ruralcommunities typically do not permit theprogram to cover all of the eligible house-holds at once. Instead, households have tobe covered by the program in phases, as wasdone in the case of PROGRESA. Ratherthan purposefully depriving households ofprogram benefits, experimental or quasi-experimental designs simply take advantageof the sequential expansion of the programto select a comparable or control groupfrom the set of households that are eligiblefor the program but have yet to be coveredby the program. This practice offers the op-portunity to conduct a scientifically rigor-ous evaluation of whether the program hasan impact or not, and if so the size of thisimpact on beneficiary households. A scien-tifically rigorous evaluation is the best wayof determining whether the scarce publicfunds are used effectively and efficiently to-ward the achievement of the short-run andlong-run objectives of the program.

As discussed in more detail later in thischapter, all households were initially sur-veyed in October/November 1997 and,based on this first survey, the eligibility

status of households was determined. Based on PROGRESA’s beneficiary selectionmethod, all households in both treatmentand control communities were classified aseligible or non-eligible for participation inthe program. On average in our sample, 78percent of the households were classified aseligible for program benefits.31 A secondsurvey took place in March 1998 before theinitiation of payments in July 1998. Thethird round of the survey took place in Oc-tober 1998, which was well after mosthouseholds received some benefits as partof their participation in the program. Thenext round of the survey took place in June1999, and the fifth round took place in No-vember 1999. After that round, the benefitsof the program started being distributed inthe control communities.

A useful description of the generalmethodology and estimators used to evalu-ate the impact of PROGRESA on any givenoutcome indicator denoted by the letter Y isprovided in Table 3.1. Within any surveyround before (t = 0) or after the start of theprogram (t = 1, 2, 3, . . .), the average valueof the outcome indicator Y within the totalsurvey population, denoted by [Y(t)], can bedivided into four different components de-pending on whether an individual child oradult belongs in a household classified as el-igible to receive PROGRESA benefits (E =1 for eligible households and E = 0 for non-eligible households) and according towhether the household that the individualbelongs in resides in a locality where PRO-GRESA is in operation (treatment localityor T = 1) or not (control locality or T = 0).Given this decomposition of the sample onemay then construct all of the estimators com-

28 CHAPTER 3

30IFPRI’s active involvement in the evaluation of PROGRESA started in July 1998, more than one year after theselection of the evaluation sample by PROGRESA authorities. The only way of verifying the integrity of the ran-domization process was by using an “ex-post” approach as in Behrman and Todd (1999a). The pool of localitiesused to select the evaluation sample did not include localities from the states of Campeche, Chiapas, Chihuahua,Coahuila, Guanajuato, and Oaxaca for a variety of socioeconomic reasons, including the potential safety prob-lems for interviewers (e.g., in Chiapas).

31As is explained later, in reality the percentage of beneficiary households in treatment localities turned out to beless than the numbers of eligible households due to some administrative errors.

EXPERIMENTAL DESIGN AND INFORMATION SOURCES 29

monly used in program evaluation. Theseare:

1. The cross-sectional difference estimator(CSDIF) compares differences in the meansof the outcome variable Y between groups A and B during the periods after the implementation of the program (i.e., t = 1, 2, 3, . . .):

CSDIF = E(Y(t)|T = 1, E = 1) – E(Y(t)|T = 0, E = 1)for t = 1, 2, 3, . . . . (13)

2. The before and after estimator (BADIF)compares differences in the means of theoutcome variable Y between group A duringthe periods after (t ≥ 1) and before (t = 0) theimplementation of the program, that is:

BADIF = E(Y(t = 1)| T = 1, E = 1)– E(Y(t = 0)| T = 1, E = 1).

(14)

3. The double differences or difference-in-differences estimator (2DIF) measures pro-gram impact by comparing differences inthe means of the outcome between group Aand B in post-survey rounds with the differ-ences in the means of the outcome betweengroup A and B in the pre-program round.Formally,

2DIF = [E(Y(t = 1)| T = 1, E = 1) – E(Y(t = 1)| T = 0, E = 1)] –[E(Y(t = 0)| T = 1, E = 1) –E(Y(t = 0)| T = 0, E = 1)].

(15)

Each of these estimators has some advan-tages and shortcomings associated with it.However, the 2DIF estimator, in compari-son to either the BADIF or CSDIF estima-tor, is the preferred estimator for programevaluation. For example, one major advan-tage of the 2DIF estimator over CSDIF inevaluating the mean direct effect of treat-ment on the treated is that the former con-trols for any preexisting differences in theexpected value of Y between households intreatment and control localities. Measuringprogram impact based exclusively on post-program difference in the mean level of theoutcome indicator between treatment andcontrol localities, as done by the first differ-ence estimator, may lead to potentially mis-leading conclusions about program impact.For example, consider the case where thereare preprogram differences in the levels of Ybetween treatment and control localities.For example, suppose that the mean valueof the outcome indicator is lower among el-igible households in treatment localitiesthan in eligible households in control local-ities, that is,

[E(Y(t = 0)| T = 1, E = 1)] <[E(Y(t = 0)| T = 0, E = 1)].

In addition, suppose that the program is suc-cessful at bringing the level of Y in the treat-ment localities up to the level of Y in controllocalities in the period after the start of theprogram. Then a simple comparison ofmeans between treatment and control local-ities after the start of the program is likely to

Table 3.1 Decomposition of the sample of all households in treatment and control villages

Treatment locality where Control locality wherePROGRESA is in operation PROGRESA operations are delayed

Description (T = 1) (T = 0)

Eligible for PROGRESA A Bbenefits (E = 1) E = 1, T = 1 E = 1, T = 0

Non-eligible for PROGRESA C Dbenefits (E = 0) E = 0, T = 1 E = 0, T = 0

show no impact whereas the program hashad a significant impact.32

Ultimately, the extent to which theCSDIF estimator may lead to biased resultsdepends critically on whether the selectionof treatment and control localities was in-deed random. Pure and proper randomiza-tion of the selection of localities would en-sure that there are no significant preprogramdifferences in the outcome variable of inter-est between treatment and control localities,that is,

[E(Y(t = 0)| T = 1, E =1)] =[E(Y(t = 0)| T = 0, E = 1)]. (16)

Satisfaction of condition (16) also ensuresthat CSDIF = 2DIF. In other words, ran-domization implies that focusing exclu-sively on post-program comparisons be-tween treatment and controls yieldsunbiased conclusions about the impact ofthe program. The extent to which the selec-tion of localities into treatment and controlgroups can be considered as random is in-vestigated in detail in one of the early re-ports of the evaluation project (see Behrmanand Todd 1999a). Randomized assignmentto treatment implies that the distribution ofall the variables for treatments and controlsshould be equal prior to the administrationof the program. To check whether random-ization has been successfully implemented,the treatment and control samples werecompared in two key dimensions: first, bycomparing the means of key variables trans-formed into locality means in control andtreatment localities; and second, by compar-ing the means of the same variables withhousehold level data.

When these comparisons and tests wereperformed at the locality level (i.e., com-paring locality means of age, education, income, access to health care, etc.) the hy-

pothesis that the means are equal betweentreatment and control localities is not re-jected. When the same comparison was per-formed using household level data, it wasfound that the null hypothesis was rejectedmore frequently than would be expected bychance given standard significance levels.Although this rejection of random assign-ment into control and treatment is some-what alarming, the researchers interpreted itas being due to the fact that the samples arelarge, which means that even minor differ-ences could lead to rejection.

Which of these two estimators is feasi-ble in practice depends on whether data onan outcome indicator are available not onlyafter but also before the start of the program.For most of the key outcome indicators ofinterest such as school enrollment and at-tendance, child nutrition, incidence of ill-ness, and labor force participation, data areavailable before and after the start of theprogram that permit implementation of the 2DIF estimator. For some indicators,however, such as household consumption,caloric availability, and individual time allo-cation, observations are available only forone or more rounds after the start of theprogram. For these outcome indicators, the CSDIF estimator provides the best avail-able option for evaluating PROGRESA.

Evaluation Tools/InformationSourcesTo evaluate impact, researchers conductedformal surveys and structured and semi-structured observations and interviews,focus groups, and workshops with a seriesof stakeholders, including beneficiaries,local leaders, local PROGRESA officials,central PROGRESA officials, health clinicdoctors, nurses and assistants, and school-teachers.

30 CHAPTER 3

32Along parallel lines of reasoning the 2DIF estimator relative to the BADIF is able to yield an estimate of theprogram effect that is net of any time trends or aggregate effects present in the data (for more details see the dis-cussion later and Heckman et al. 1999).

In November 1997 PROGRESA con-ducted a survey of the socioeconomic condi-tions of rural Mexican households (Encuestade Caracteristicas Socioeconomicas de losHogares [ENCASEH]) in the evaluationcommunities to determine which house-holds would be eligible for benefits. Basedon PROGRESA’s beneficiary selectionmethods, households were then classified aseligible or non-eligible for participation inthe program in both treatment and controlcommunities. On average in the evaluationsample, 78 percent of the households wereclassified as eligible for program benefits.The first evaluation survey (Encuesta deEvaluation de los Hogares [ENCEL]) tookplace in March 1998 before the initiation ofbenefits distribution in May 1998.33 In com-bination, these two surveys provide the base-line observations available for all house-holds before the initiation of the distributionof cash benefits in the treatment villages.

The rest of the evaluation surveys wereconducted after beneficiary householdsstarted receiving benefits from PRO-GRESA.34 One round of surveys took placein November 1998, which was well aftermost households received some benefits aspart of their participation in the program.The next two waves took place in June 1999and November 1999.35 A number of corequestions about the demographic composi-tion of households and their socioeconomicstatus were applied in each round of the sur-vey. These core questions were accompa-nied by specific questionnaires, focused oncollecting information critical to a thoroughevaluation of the impact of the program.The topics of these modules included col-

lecting information about family back-ground, assets brought to marriage, school-ing indicators, health status and utilization,parental attitudes and aspirations towardchildren’s schooling, consumption of foodand nonfood items, the allocation of time ofhousehold members in various activities,and self-employment activities. Table 3.2presents the number of households and indi-vidual members covered in each surveyround. It is important to keep in mind thatthe March 1998 questionnaire did not con-tain a detailed household roster based oneach household member interviewed in theNovember 1997 round. In the March 1998round the mother of the child was simplyasked to give her child’s name followed byquestions about the child’s health andschool attendance. Member codes werelater assigned to children in the March 1998round after the completion of the survey aslong as a child’s name in a family could besafely matched with a name in the samefamily in the November 1997 round. Givendifferences in the spelling, many childrencould not be matched across rounds. As aresult, the March 1998 round contains a sig-nificant number of children with a newmember code that did not exist in the previ-ous round. This problem was corrected inthe later rounds, when interviewers weregiven a household roster based on the No-vember 1997 household roster. This prob-lem with the member codes in March 1998implies that panels of children constructedby including this survey round are likely tobe problematic.

The preceding surveys were supple-mented by school and clinic surveys, com-

EXPERIMENTAL DESIGN AND INFORMATION SOURCES 31

33In principle, the first payments in May 1998 were for the two-month period elapsed since incorporating fami-lies into PROGRESA (i.e., in March 1998). Note, however, there is no record kept for the exact date of incorpo-rating families into the program.

34IFPRI researchers and academic collaborators had a significant contribution in the design of the evaluationquestionnaires applied in November 1998 and later. IFPRI researchers were not allowed to contribute in the train-ing of the interviewers and in the household survey process.

35An additional survey took place in June 2000. From that survey, only the fertility module has been utilized forthe evaluation of PROGRESA.

munity questionnaires, data on studentachievement test scores, and other schooland clinic administrative data. The evalua-tion surveys (ENCEL) collected by PRO-GRESA did not allow for an evaluation ofthe nutritional component of the program.For the purposes of evaluating the nutri-tional component of PROGRESA, separatesurveys of the same families were carriedout by the National Institute of PublicHealth (INSP) in Cuernavaca. These sur-veys included collection of data on anthro-pometric measures (weight and height) dataof children and collection of blood samplesfor tests for anemia and other deficiencies.Note, however, that IFPRI researchers wereable to merge the child-specific anthropo-metric data collected and made available bythe INSP with the evaluation data collectedby PROGRESA in order to conduct an earlyevaluation of the impact of PROGRESA onthe height of preschool children (Behrmanand Hoddinott 2000).

In measuring the impact of a large andadministratively complex program such asPROGRESA it is very important to take intoconsideration the role that operational fac-

tors can play. Delays in the delivery, com-pletion, and/or processing of the variousforms required to prove compliance withthe program requirements can lead to delaysin the delivery of the cash benefits associ-ated with the program. To the extent thatsuch delays are not accompanied by seriousefforts by the PROGRESA administrationto solve the problems involved, they can re-sult in loss of confidence by householdscomplying with the requirements of the pro-gram. Such factors could result in weakerprogram impacts with the passage of time.In contrast, initial delays in the processingof forms and delivery of benefits that areimproved on over time could lead tostronger program impacts with the passageof time. It is thus crucial that a thoroughevaluation of PROGRESA must also exam-ine the operational process of the program,identify potential bottlenecks in the process,and offer constructive suggestions for im-proving the operation and overall effective-ness of the program.

The evaluation of PROGRESA byIFPRI has also included an evaluation ofthe operational aspects of the program.

32 CHAPTER 3

Table 3.2 Number of households and individual members covered in each survey round

Non-eligible (E = 0) Eligible (E = 1)

Control Treatment Control TreatmentSurvey round Coverage (T = 0) (T = 1) (T = 0) (T = 1) All

Pre-program/baseline census/surveyENCASEH Nov 97 Households 2,048 3,233 7,173 11,623 24,077

Individuals 5,791 8,765 17,114 27,366 59,036ENCEL-Mar 98 Households 1,925 3,048 6,567 10,549 22,059

Individuals n.a. n.a. n.a. n.a. n.a.

Post-program surveysENCEL-Nov 98 Households 2,058 3,272 7,158 11,585 24,073

Individuals 6,147 9,290 17,793 28,258 61,488ENCEL-Jun 99 Households 1,837 2,932 6,655 10,682 22,106

Individuals 5,361 8,090 16,406 25,775 55,632ENCEL-Nov 99 Households 1,921 2,902 6,818 10,475 22,116

Individuals 5,804 8,421 17,219 26,000 57,444

Notes: The terms eligible (E = 1) or non-eligible (E = 0) are based on the final list of eligible households con-structed by the PROGRESA administration (see Chapter 5 for more details).The March 1998 ENCEL survey collected information at the individual level only for children betweenbirth and six years of age. No information was collected at the individual level for adult members.

The evaluation used both quantitative and qualitative surveys. The quantitativesurveys included repeated surveys of bene-ficiary households, schools, and healthclinics. The qualitative surveys conductedin 1999 and in early 2000 included semi-structured interviews with stakeholders inPROGRESA including secondary schooland health clinic directors and nurses from18 communities, and focus group discus-sions with PROGRESA liaisons (promo-toras), beneficiaries, and non-beneficiaries.In total, 23 focus groups were held involv-

ing 230 participants: 80 beneficiaries from8 communities, 80 non-beneficiaries from 8communities, and 70 promotoras from 70communities.

Although the information collected aspart of the qualitative surveys is not in-tended to be statistically representative ortrue for the majority of the population, theresearch produces information that broad-ens the field of inquiry to include questions,issues, and factors that may have been pre-viously missed, and additional levels of ex-planatory and interpretive power.

EXPERIMENTAL DESIGN AND INFORMATION SOURCES 33

C H A P T E R 4

An Evaluation of PROGRESA’s Targeting and Its Impact on Poverty

The implementation of PROGRESA has involved three distinct stages (for more detailssee Skoufias, Davis, and Behrman 1999a and Skoufias, Davis, and de la Vega 1999b,2001). The first stage involved the identification of the most marginal rural localities

where the extremely poor are likely to be found. The identification of the marginal rural localities used a specially constructed “marginality index” based mainly on data from the na-tional census. Additional considerations included geographical location, locality size (locali-ties with fewer than 50 and more than 2,500 inhabitants were excluded), distance between lo-calities, and access to some supporting infrastructure such as the presence of a primary schoolwithin the locality and access to a secondary school and a health clinic within a certain dis-tance from the locality. The second stage involved the selection of households within eligiblelocalities. Using detailed socioeconomic data collected by the program from all the house-holds in the eligible localities, households were classified as “poor” or “non-poor” using a dis-criminant analysis of household income and other characteristics.

MethodologyThe evaluation of PROGRESA’s targeting is based on a framework consisting of three key el-ements: (1) a social objective; (2) a set of economic, political, and social constraints underwhich policy has to operate; and (3) a range of instruments available to attain these objectives.Although PROGRESA has a number of interlinked objectives with respect to health, educa-tion, and nutrition, the benefits of PROGRESA’s targeting are measured solely in terms of its potential impact on poverty alleviation.36 The economic, social, and political constraintsunder which policy has to operate are partly reflected in the size of the budget available forPROGRESA. The budget is assumed to be fixed and limited in the sense that it is not suffi-cient to eliminate poverty completely.

Policy instruments for poverty alleviation range from uniform transfers that apply no se-lection criteria to other schemes involving more strict selection criteria. Each of these instru-ments has different costs and benefits associated with it. The primary benefit derived from tar-geting at the household level is that classifying households into those eligible and ineligible

36It is also possible to evaluate PROGRESA’s targeting in terms of alternative objectives such as whether the pro-gram selects families with children who have a higher risk of dropping out of school (e.g., de Janvry and Sadoulet2002).

34

for receiving benefits from PROGRESAand giving benefits to those who are eligibleis a more effective way of using the limitedfunds toward the achievement of the socialobjective. This, however, is done at a cost.As discussed in the report, the PROGRESAtargeting mechanism involves the collectionof a household survey within all the locali-ties selected as marginal (or as more likelyto contain poor households). Such costs are taken into account by appropriately re-ducing the fixed budget available forpoverty alleviation.

Within this framework the evaluation ofPROGRESA’s targeting can be formulatedas providing an answer to the followingquestion: How well does PROGRESA’s tar-geting perform in terms of its objectiveafter taking into account the costs and theconstraints (financial and political) ofachieving these objectives? This question isanswered in two steps. First, PROGRESA’saccuracy in targeting is evaluated both atthe community level and at the house-hold level by comparing PROGRESA’s selection to an alternative selection ofhouseholds based on consumption. Second,PROGRESA’s targeting performance isevaluated in terms of its impact on povertyalleviation relative to other feasible target-ing and transfer schemes assuming thesame total budget.

The evaluation adopts an indicator thatis considered sensible for classifying house-holds into poor and non-poor, while beingcareful to point out that this is not necessar-ily the perfect poverty indicator. The indica-tor used to examine PROGRESA’s targetingis predicted household consumption. Con-sumption for households contained in PROGRESA’s sample (beneficiaries andnon-beneficiaries) is estimated using the1996 Encuesta Nacional de Ingreso-Gastode los Hogares (ENIGH). Based on this indicator, the accuracy of PROGRESA’s targeting is assessed using the concepts ofundercoverage (exclusion error) and leak-age (inclusion error) used frequently in thetargeting evaluation literature.

Evaluation of Targeting AccuracyThe conclusion regarding the accuracy ofPROGRESA’s targeting is that overall it isan effective method of selecting householdsinto the program. The evaluation analysisshows that the accuracy of PROGRESA’stargeting, in terms of both selecting locali-ties where poor households are more likelyto be found and selecting poorest house-holds within these localities, is good (Skoufias et al. 1999a,b, 2001). However,this accuracy fades when it comes to distin-guishing between localities in the moder-ate level of marginality. A similar conclu-sion is derived from the evaluation of thetargeting of households within localities.PROGRESA’s targeting is not perfect, butrelatively more effective at identifying theextremely poor households within localitiesand less so when it comes to selectinghouseholds that are moderately poor.

Household Targeting versusOther Feasible AlternativesBased on simulations using quantitativedata that take into account the costs of tar-geting, PROGRESA’s targeting as practicedduring the second phase of the program isfound to be the most effective among the setof feasible targeting and transfer schemes inreducing the depth of poverty and the sever-ity of poverty in Mexico (Skoufias et al.2001).

In short, PROGRESA performed closerto the ideal of “perfect” targeting than anyof the alternative feasible transfer and tar-geting schemes examined such as uniformtransfers (i.e., no targeting at all), targetingbased on consumption or “perfect” target-ing, and targeting at the locality level ratherthan at the household level. The researchfinds that PROGRESA’s method of select-ing households outperforms alternativemethods in terms of reducing poverty mea-sures, weighting extremely poor householdsmore heavily (Skoufias et al. 1999a,b). Asimilar conclusion is drawn when one ex-

PROGRESA’S TARGETING AND IMPACT ON POVERTY 35

amines the impact of PROGRESA’s target-ing on social welfare instead of the standardpoverty measures (Coady 2000).

The research also finds that the non-eco-nomic costs associated with targeting de-serve serious consideration in the overalldecision to pursue a household level target-ing strategy. The targeting evaluation studyfinds that PROGRESA’s method of target-ing households outperforms alternativemethods in terms of reducing the povertygap and severity of poverty indices, evenafter taking into account the economic costsof targeting. However, the reduction in thehigher order measures of poverty accom-plished by household targeting over andabove those accomplished by simply in-cluding all the households in the locality arerelatively small (only 3.05 percentagepoints higher than the reduction in povertyachieved by including all households in thelocality). Whether these marginal successesof targeting at the household level is aworthwhile effort depends on the size of thenoneconomic, or political and social costsof targeting, all of which are very difficult toquantify. As the qualitative surveys fromPROGRESA’s evaluation discussed in thefollowing section indicate, these costs oftargeting in rural, often indigenous, commu-nities may not be negligible.

PROGRESA and Its Impact on PovertyIn assessing the impact of the PROGRESAcash transfers on short-run poverty indica-tors two approaches were adopted. The firstapproach relies on simulations based on thepredicted consumption of each household inthe evaluation sample in November 1997and adding the maximum amount of PROGRESA cash transfers an eligiblehousehold could receive assuming full com-pliance with the program’s requirements(Skoufias et al. 1999a,b). The second ap-proach relies on reported household incomeand household consumption using the infor-mation collected by the household socio-

economic census (ENCASEH) and the eval-uation surveys (ENCEL) and the amount ofcash benefits received by beneficiary house-holds in treatment areas (Appendix C). Al-though each one of these approaches has anumber of shortcomings associated with it,in combination they offer the opportunity tocheck on the robustness of the measured im-pact of PROGRESA.

The results obtained from the simulatedimpact of PROGRESA’s cash transfersshow that the headcount ratio, which simplymeasures the percentage of the populationwith income levels below the poverty levelin a community, is reduced by about 10 per-cent through the supports of PROGRESA.The poverty gap and severity of povertymeasures that place greater weight on thepoorest households within the population inpoverty show that the level of poverty ac-cording to the poverty gap is reduced by 30 percent whereas the severity of thepoverty index is reduced by 45 percent. Forcomparison, an untargeted or uniform transfer is found to reduce the poverty gap by 28 percent and the severity ofpoverty by 36 percent. Given that these in-dicators put greater weight on the poorestof the poor, the simulation results suggestthat the largest reductions in poverty of PROGRESA are being achieved in thepoorest of the poor population.

One potential shortcoming of using simulations to measure the impact of PROGRESA on poverty is the fact that theincome households receive from other gov-ernment programs and children working inthe labor market are both assumed to beconstant. As discussed earlier, householdsreceiving PROGRESA benefits should not,in principle, be receiving other similar ben-efits from programs such as Abasto Socialde Leche, de Tortilla, and the National Insti-tute of Indigenous People (INI). In addition,the school attendance requirements of PROGRESA may force children to with-draw from paid activities and devote moreof their time to school. Both of these factorsmay work to negate the positive effect of the

36 CHAPTER 4

PROGRESA cash transfers on total house-hold income.

Figure 4.1 demonstrates in more detailthat among PROGRESA beneficiary house-holds in treatment localities the percent-age of households receiving governmenttransfers from other programs besides PROGRESA appears to decrease dramati-cally after the start of the PROGRESA pro-gram. In addition, among PROGRESA beneficiary households with children be-tween 8 and 17 years of age the total in-come received from children in this agegroup decreased.37

Relying on reported household incomeallows one to obtain the difference-in-differences (2DIF) estimate of the impact ofthe program on poverty which compares thechange in a poverty measure in treatmentvillages to the changes in the correspondingpoverty measure in control villages. In addi-tion to controlling for macroeconomicshocks common to both treatment and control localities, this estimate allows one toaccount for any preexisting differences inpoverty between control and treatment lo-

calities and thus yields a “cleaner” estimateof the impact of the program on poverty.

Irrespective of the measure of welfareused (per capita income or per capita con-sumption) and irrespective of the povertyline used (value of basic food basket or median of the value of household con-sumption) the 2DIF estimates imply thatPROGRESA had a significant impact in re-ducing poverty between November 1997and November 1999. For example, using in-come per capita as a measure of welfare andthe 50th percentile of the value of consump-tion per capita as a poverty line suggeststhat the headcount poverty rate declined by17 percent in treatment areas between No-vember 1997 and November 1999. Over thesame period, the poverty gap and the sever-ity of poverty measures declined by 36 per-cent and 46 percent (see Appendix C).These estimates are very much in line withthe estimates obtained using simulations,and provide further confirmation that theimpact of PROGRESA is concentrated atimproving the welfare of the poorest of thepoor households in marginal rural areas.

PROGRESA’S TARGETING AND IMPACT ON POVERTY 37

37For more details see Appendix C.

Figure 4.1 Percentages of households in treatment localities that receive transfers fromother programs and PROGRESA

Perceptions of StakeholdersRegarding the Selection ofBeneficiary HouseholdsQuantitative and qualitative data indicatethat there are perceived problems with theselection process—mainly, that there arepoor people who need the benefits and donot receive them and, less frequently men-tioned, that there are people receiving bene-fits who do not need them (Adato, Coady,and Ruel 2000a). Although not statisticallyrepresentative, the qualitative data collectedfrom focus groups indicate some problemswith the original socioeconomic survey(i.e., the ENCASEH survey). For example,in some cases people were not home whenthe enumerator came to call and the enu-merators did not return, or people overstatedtheir resources because they were ashamedto admit their poverty. Most respondents inthe qualitative research did not disagreewith targeting in the sense that they did notbelieve that professionals, shop owners, orother relatively rich people should receivebenefits; rather they believe that the mis-takes should be corrected. Also, focusgroups indicated that aside from these moreobviously richer people, in these rural com-munities people perceive themselves as “allpoor” and all in need, and thus did not agreewith the finer distinctions made in the selec-tion process. However, they did indicate thatthe selection did not appear to be politicallymotivated.

At the community level, focus groupsand interviews with doctors and school di-rectors indicated that there has not been anopportunity to have an input into the selec-tion process. In addition, these stakeholdersindicated that PROGRESA’s household tar-geting strategy has in some communitiesbeen associated with social divisions, mostoften manifested in non-beneficiaries notwanting to participate with beneficiaries incommunity work (Adato 2000; Adato et al.2000a). Responses from these stakeholderssuggest that these problems could be re-duced through a more systematic imple-mentation of PROGRESA’s policy proposal

to provide an opportunity for communitiesto review and improve the selection so thatthey are in agreement with its fairness.

Impact on Community Social RelationshipsThe overall conclusion of this research is thatPROGRESA’s system of household target-ing involves social costs that should be takeninto account in evaluations of this systemand consideration of alternative targetingsystems. Communities exhibit social solidar-ity in terms of the common ways in whichbeneficiaries and non-beneficiaries evaluatethe beneficiary selection process, outcomes,and impacts. At the same time, there is evi-dence of problems that the targeting has introduced into community social relation-ships. Although the percentage of communi-ties in Mexico that have experienced theseproblems is not known from a statistical per-spective, the frequent and similar statementsof beneficiaries, non-beneficiaries, promo-toras, and doctors in the majority of focusgroups and interviews conducted across sixstates provide strong evidence that there is aproblem that should be addressed.

PROGRESA has also strengthened socialrelationships between beneficiary women, potentially building new forms of social capi-tal. This is a valuable second-round effect of the program, and suggests that these typesof approaches to PROGRESA activities that promote social capital could be encour-aged. At the same time, the creation of a group of “PROGRESA women” who partici-pate in separate activities can reinforce social divisions, so these problems related tohousehold targeting need to be addressed simultaneously.

PROGRESA, Work Incentives forAdults, and the Allocation ofResources within HouseholdsPROGRESA also does not appear to createnegative incentives for work (Parker and Skoufias 2000). Analysis of before and afterprogram data shows no reduction in laborforce participation rates either for men or

38 CHAPTER 4

for women. These results may in part reflectthe design of PROGRESA, in which benefitsare provided to families for three years, irrespective of family income, so that there isno disincentive effect on work, as opposed to transfer programs in other countries whichoften reduce benefits with work income. The conventional wisdom is that there aretrade-offs between providing benefits to apopulation in need and stimulating work; the analysis here shows that, thus far, there is not necessarily any such trade-off in PROGRESA.

There are no significant differences be-tween treatment and control groups by yearand over time with regard to the receipt ofmonetary transfers from individuals orfriends not living in the household, includ-ing transfers from relatives working abroad,such as in the United States. The analysisfinds that after 19 months of receipt of ben-efits, the selection into the PROGRESAprogram has no influence over the incidenceor level of either monetary or nonmonetaryprivate transfers within households (Terueland Davis 2000).

PROGRESA’S TARGETING AND IMPACT ON POVERTY 39

C H A P T E R 5

Summary of the Methods Used to Evaluatethe Impact of PROGRESA

A ll of the reports evaluating the impact of PROGRESA did so using either the differ-ence-in-differences (2DIF) estimator or the cross-sectional differences (CSDIF) esti-mator of the program impact (discussed in Chapter 3). However, rather than simply

comparing unconditional means between treated and control groups, all of the reports used re-gression methods. The estimation of the 2DIF and CSDIF estimates of program impactthrough regression methods provides the advantage of controlling for the role of observedcharacteristics of the individual, the household, and the locality on the variation in the ob-served value of the outcome indicator of interest. Without any adjustment for the role of suchconfounding factors, all the differences in the mean value of the outcome indicator are attrib-uted to the program. As a matter of principle, it is preferable to have an estimate of the impactof the program net of the influence of these observed characteristics on the difference in themean value between treatment and control households.

The discussion that follows provides a summary of the regression methods that one canuse to estimate the impact of eligibility in the PROGRESA program on a generic outcome in-dicator. It should be kept in mind that the nature of the outcome indicator (i.e., whether it is acontinuous variable, such as household consumption, or a binary variable, such as child schoolenrollment) necessitates an appropriate econometric method for the estimation of the programimpact (such as ordinary least-squares [OLS], probit, or logit).38

To begin, consider the case where data are available for treatment and control householdsbefore and after the start of the program.39 Restricting the sample to eligible households only(E = 1), the various estimators for program evaluation discussed earlier that control for indi-

38However, it should be kept in mind that when nonlinear methods (such as probit or logit) are employed, the es-timated parameters of the model must be transformed into marginal effects that depend on the values of the re-gressors used in the model. The common practice in these circumstances is to estimate marginal effects at themeans of the regressors.

39It is assumed that there is only one observation after the start of the program for expositional simplicity. The avail-ability of more than one round of observations after the start of the program can be easily accommodated by includ-ing an additional binary variable (say R3) together with its interaction with the treatment dummy (R3*T). Thenthe coefficient of the (R3*T) term is an estimate of the 2DIF program impact estimate in the third round of thesurvey and can yield information on whether the impact of the program is strengthened or weakened over time(e.g., Schultz 2000a; Parker and Skoufias 2000).

40

vidual, household, and locality observedcharacteristics can be obtained by estimat-ing a regressions equation of the form40

Y(i, t) = α + βTT(i) + βRR2 + βTR(T(i)*R2) +Σj θj Xj + η (i,v,t), (17)

where Y(i,t) denotes the value of the out-come indicator in household (or individual)i in period t; α, β, γ, and θ are fixed param-eters to be estimated; T(i) is an binary vari-able taking the value of 1 if the householdbelongs in a treatment community and 0otherwise (i.e., for control communities);R2 is a binary variable equal to 1 for the sec-ond round of the panel (or the round afterthe initiation of the program) and equal to 0for the first round (the round before the ini-tiation of the program); X is a vector ofhousehold (and possibly village) character-istics; and η is an error term summarizingthe influence of random disturbances.

To better understand the preceding spec-ification it is best to divide the parametersinto two groups: one group summarizingdifferences in the conditional mean of theoutcome indicator before the start of the pro-gram (i.e., α, βT) and another group sum-marizing differences after the start of theprogram (i.e., βR, βTR). Specifically, the co-efficient βT allows the conditional mean ofthe outcome indicator to differ between eli-gible households in treatment and controllocalities before the initiation of the pro-gram whereas the rest of the parametersallow the passage of time to have a differenteffect on households in treatment and con-trol localities. For example, the combinationof parameters βR and βTR allows the differ-ences between eligible households in treat-ment and control localities to be differentafter the start of the program.

One advantage of this specification isthat the t-values associated with some ofthese parameters provide direct tests of anumber of interesting hypotheses. For ex-ample, the t-value associated with the esti-mated value βT provides a direct test of theequality in the conditional mean of Y be-tween treatment and control before the initi-ation of the program and serves the role of atest of the randomness in selection of local-ities. For if there were a truly random selec-tion of localities into control and treatment,then the conditional mean of the outcomeindicator should be identical across treat-ment and control households/individuals.

Specifically, given the preceding specifi-cation, the conditional mean values of theoutcome indicator for treatment and controlgroups before and after the start of the pro-gram are as follows:

[E(Y | T = 1, R2 = 1,X)] = α + βT + βR + βTR + Σ j θj Xj (18a)

[E(Y | T = 1, R2 = 0,X)] = α + βT + Σ j θj Xj (18b)

[E(Y | T = 0, R2 = 1,X)] = α + βR + Σ j θj Xj (18c)

[E(Y | T = 0, R2 = 0,X)] = α + Σ j θj Xj (18d)

According to the preceding specification,the cross-sectional difference estimator isgiven by the expression

CSDIF = (18a – 18c) = [E(Y | T = 1, R2 = 1,X) – E(Y | T = 0, R2 = 1, X)] = βT + βTR, (19)

SUMMARY OF METHODS USED TO EVALUATE IMPACT OF PROGRESA 41

40It should be noted that a slightly more restrictive specification is to pool all eligible and non-eligible householdsand include an additional variable denoting eligibility (E) along with a full set of its interactions with the binaryvariables T and R2. This alternative specification, in addition to estimates of CSDIF and 2DIF, yields the tripledifference (3DIF) or difference-in-difference-in-differences estimator which compares changes in the inequalityof the outcome indicator between eligible and non-eligible households.

while the before-and-after difference esti-mator is given by

BADIF = (18a – 18b) = [E(Y | T = 1, R2 = 1,X) – E(Y | T = 1, R2 = 0, X)] = βR + βTR. (20)

Expression (19) describing the CSDIF esti-mator highlights the fact that the estimatedimpact of the program is inclusive of anypre-program differences between treatmentand control groups (summarized by thepresence of the βT term). Along similarlines, expression (20) indicates that theBADIF estimator is inclusive of any trend oraggregate effects in the changes of the out-come indicator Y (summarized by the pres-ence of the βR term).

The advantage offered by the difference-in-differences (2DIF) estimator is that itprovides an estimate of the impact of theprogram that is net of any pre-program dif-ferences between treatment and controlhouseholds and/or any time trends or aggre-gate effects in changes of the values of theoutcome indicator.41 By comparing beforeand after differences between treatment andcontrol households (or differences betweentreatment and control households after andbefore the program), one is able to get an estimate of the impact of the program (sum-marized by the single parameter βTR):

2 DIF = (18a – 18b) – (18c – 18d) = (18a – 18c) – (18b – 18d) = βTR. (21)

Using the terminology of Heckman et al.(1999), the parameter βTR provides an esti-mate of the “mean direct effect of treatment

on those who take the treatment.” It shouldalso be noted the program effect summa-rized by the parameter βTR is inclusive ofthe role of the operational efficiency or in-efficiency with which the program operates.It is likely that persistent delays in the pro-cessing of forms in some states or munici-palities and other administrative bottlenecksmay lead to weaker impacts of the programon households residing in these states rela-tive to those in states where the program isoperating more efficiently.42 This questionwas dealt with as part of the operations eval-uation of PROGRESA rather than as part ofthe impact evaluation of the program.

In the majority of the reports, the vectorX typically consists of variables characteriz-ing the age and gender composition of thehousehold, household size, and the age andeducation level of the household head andhis or her spouse. Given that the vector Xdoes not contain any supply-related vari-ables, βTR is an estimate of the impact of the conditional cash transfers (demand-sideeffects) and the improvements in the quan-tity and quality (or supply-side effects) ofeducational and health services and facili-ties associated with the PROGRESA pro-gram. Efforts to distinguish between the de-mand and supply effects of the programrequire the inclusion of supply-related vari-ables as additional regressors in equations(18).43 However, the extent to which one isable to isolate sufficiently the demand effectfrom the supply effect depends on whetherthe observed supply-related variables cap-ture sufficiently all the supply effects of theprogram.

The availability of repeated observationsbefore and after the start of the program on

42 CHAPTER 5

41It is important to note, however, that these advantages of the 2DIF estimator are based on some implicit as-sumptions. For example, the 2DIF estimator assumes that the time trend present among control households is anadequate representation of the trend that would have prevailed among treated households in the absence of theprogram.

42For example, such regional inefficiencies in the operation of the program could be captured empirically by al-lowing the coefficient of treatment dummy variable T in expression (17) to vary by region or state.

43For example, see Schultz (2000a) and Coady (2000).

non-eligible households in treatment areasalso offers the opportunity to examine thepotential effects of the program on the non-eligible households residing in the treatmentcommunities. For example, improvementsin the quantity and quality of supply ofhealth and educational services may alsobenefit the non-eligible households in thetreatment communities. Non-eligible house-holds may also benefit by attending volun-tarily the monthly pláticas offered in the vil-lages covered by PROGRESA. In addition,non-eligible households in treatment locali-ties may alter their behavior (such as workless or withdraw their children from school)in anticipation that such actions may qualifythem for the program. An evaluation of theextent to which the program has had someindirect effects on the outcome indicatoramong non-eligible households in treatmentareas can also be conducted by estimating a regression similar to equations (18) but re-stricted to the sample of non-eligible house-holds (E = 0).

When data for the outcome variable ofinterest are available for one round (or morerounds) after the start of the program, thenthe evaluation of the impact of the programreduces to whether the coefficient of the T(i)binary variable identifying the householdsresiding in treatment localities, is positiveand significantly different from zero. Usingthe sample of eligible households only (E =1), then a specification that could be esti-mated is of the form

Y(i) = α + γTT(i) + Σ j θj Xj

+ η (i,v). (22)

In this case, an estimate of the cross-sectional difference estimator (CSDIF) isprovided by

CSDIF = E(Y | T = 1,X) – E(Y | T = 0,X) = γT . (23)

The availability of observations for morethan one round after the start of the programallows one to examine whether the impactof the program is strengthened or weakenedby the passage of time. For example, denot-ing observations from the third round of theevaluation survey by R3 (that is the secondround after the start of the program) is a re-gression of the form

Y(i) = α + γTT(i) + δRR3 + δRT(R3*T(i)) + Σ j θ jXj + η (i,v). (24)

With this specification, a statistical compar-ison of the relative sizes of the coefficientsδRT and γT (using one-tailed tests) can re-veal whether the impact of the program inthe third round is significantly greater orsmaller than the impact of the program inthe second round (e.g., see Hoddinott, Skoufias, and Washburn 2000; Skoufias andParker 2001).

The discussion so far did not address therole of unobserved heterogeneity summa-rized by the error term η(i, v, t) in the pre-ceding regressions. One primary implica-tion of the clustering of the households withvillages is that the household-specific errorterms η(i, v, t) are likely to be correlatedwithin each village (as well as across time).Failure to account for such a correlationmay lead to a considerable bias in the esti-mated standard error of the program impact(e.g., see Murray 1998). For this reason, allof the impact evaluation reports account forthe clustered nature of the sample and reportrobust standard error estimates for the im-pact of the program.44

Some Important DetailsBefore continuing with the presentation ofthe impact evaluation results it is necessaryto state upfront some of the caveats associ-ated with the evaluation of the impact of the

SUMMARY OF METHODS USED TO EVALUATE IMPACT OF PROGRESA 43

44Robust standard error estimates were obtained using the “robust” option in STATA v7.0.

PROGRESA program.45 First, and perhapsmore importantly, a key assumption is thatthe indirect effects or spillover effects of thePROGRESA program from treatment local-ities to control localities are negligible. Thisassumption is necessary in order for the “notreatment” state to approximate the “no pro-gram” state. Certainly in the early rounds ofthe program when the census and the first orsecond evaluation surveys were conductedit would be safe to say that such spillover ef-fects were likely to be insignificant. How-ever, the extent to which the spillover effectswere still insignificant during the laterrounds of the evaluation is a matter of debate.In fact, in most of the states covered by theevaluation sample, the control localities are surrounded by localities covered byPROGRESA. In addition, control locali-ties were eventually incorporated within PROGRESA and started receiving cash ben-efits in December 1999.46 Although neitherof these two facts necessarily invalidates theevaluation of PROGRESA, one should beaware of the possibility that PROGRESAhas had indirect or spillover or anticipationeffects on households in control localities.

Second, the variable used to identifyhouseholds eligible for PROGRESA bene-fits varies across some of the reports. Sincethis is a potential source of misunderstand-ing among readers of the individual evalua-tion reports, this point deserves more de-tailed elaboration. In the early stages of theprogram the PROGRESA beneficiary selec-tion method led to approximately 52 percentof the households in the evaluation sampleto be classified as eligible for the program

benefits (identified by the variable E1).47

By July 1999, PROGRESA had added newhouseholds to the list of beneficiaries sinceit was felt that the original selection methodwas biased against the elderly poor who nolonger lived with their children.48 As a re-sult of the revised selection process, thefraction of households classified as eligiblefor program benefits increased from 52 per-cent of the evaluation sample to 78 percentof the sample. Accordingly, a new variableidentifying the new and “final” list of eligi-ble households was provided for both treat-ment and control areas (denoted by E2).49

Use of the variable E2 to identify the eligi-ble households for PROGRESA benefits al-lows program evaluators to estimate the ef-fect of “treatment on the treated” wherebythe term treatment is used to represent the“offer to treat” or the “intent to treat” effectof PROGRESA (Heckman et al. 1999).

Given the differences across reports inthe variable used to identify the eligibilitystatus of a household, Table 5.1 provides aguide of the key outcome indicators used inthe quantitative evaluation of PROGRESA,as well as the econometric estimator used tomeasure the impact of PROGRESA. Themajority of the evaluation reports by IFPRIrely on the variable E2 to identify the households eligible to receive PROGRESAbenefits. The two evaluation reports thatrely exclusively on the variable E1 to iden-tify household eligibility status are Schultz(2000a,b,c) and Behrman et al. (2000).

To the extent that there is significantdropout or attrition from the program amongbeneficiary households or administrative

44 CHAPTER 5

45More detailed discussion of these and related issues can be found in Heckman (1992), Heckman and Smith(1995), and Heckman et al. (1999).

46IFPRI researchers were unable to determine whether households in control villages were given any specific rea-sons as to why PROGRESA did not cover their locality. It is not unlikely, however, that promises were made bylocal PROGRESA officials about possible inclusion of the control localities into the program in the future.

47E1 denotes the variable named “pobre_1” provided with the original data sets.

48The Spanish term used to describe this revised selection process is densification.

49E2 denotes the variable named pobreden provided with the original data sets.

SUMMARY OF METHODS USED TO EVALUATE IMPACT OF PROGRESA 45

Tabl

e 5.

1Ke

y ou

tcom

e in

dica

tors

and

impa

ct e

stim

ator

s us

ed in

the

quan

titat

ive

eval

uatio

n of

PRO

GRES

A

Est

imat

or u

sed

Var

iabl

e us

edIn

dica

tor

used

to

eval

uate

to m

easu

reto

iden

tify

U

sed

othe

rpr

ogra

m im

pact

prog

ram

impa

ctel

igib

ility

sta

tus

cont

rol v

aria

bles

?D

ata

sour

ces

Aut

hors

Edu

catio

nE

nrol

lmen

t in

scho

ol2D

IFE

1Y

esE

NC

ASE

H-N

ov 9

7Sc

hultz

(20

00a,

b)E

NC

EL

-Mar

98

EN

CE

L-N

ov 9

8E

NC

EL

-Jun

99

EN

CE

L-N

ov 9

9Pr

opor

tion

of s

choo

l day

s at

tend

ed2D

IFE

1Y

esE

NC

ASE

H-N

ov 9

7Sc

hultz

(20

00c)

EN

CE

L-M

ar 9

8E

NC

EL

-Nov

98

EN

CE

L-J

un 9

9E

NC

EL

-Nov

99

Beh

rman

, Sen

gupt

a,C

hild

sch

ool a

chie

vem

ent t

est s

core

s2D

IFE

1Y

esE

NC

ASE

H-N

ov 9

7To

dd (

2000

)E

NC

EL

-Nov

98

EN

CE

L-N

ov 9

9M

inis

try

of P

ublic

E

duca

tion

(Sec

reta

ria

de

Edu

caci

ón [

SEP]

)te

st s

core

s 19

97,

1998

, 199

9H

ealth

Dai

ly c

onsu

ltatio

ns p

er c

linic

2DIF

n.a.

Yes

IMM

S So

lidar

idad

G

ertle

r (2

000)

adm

inis

trat

ive

reco

rds

1996

, 199

7, 1

998

Vis

its b

y pr

ovid

er ty

pe (

publ

ic v

ersu

s pr

ivat

e)C

SDIF

E2

Yes

EN

CE

L-J

un 9

9G

ertle

r (2

000)

EN

CE

L-N

ov 9

9N

utri

tion

mon

itori

ng v

isits

(0-

to 5

-yea

r-ol

d ch

ildre

n)2D

IFE

2Y

esE

NC

ASE

H-N

ov 9

7G

ertle

r (2

000)

EN

CE

L-M

ar 9

8E

NC

EL

-Nov

98

EN

CE

L-J

un 9

9E

NC

EL

-Nov

99

Chi

ld il

lnes

s (0

- to

5-y

ear-

old

child

ren)

2DIF

E2

Yes

EN

CA

SEH

-Nov

97

Ger

tler

(200

0)E

NC

EL

-Mar

98

EN

CE

L-N

ov 9

8E

NC

EL

-Jun

99

EN

CE

L-N

ov 9

9A

dole

scen

t and

adu

lt he

alth

sta

tus

CSD

IFE

2Y

esE

NC

EL

-Jun

99

Ger

tler

(200

0)E

NC

EL

-Nov

99

(con

tinue

d)

46 CHAPTER 5

Tabl

e 5.

1Co

ntin

ued

Est

imat

or u

sed

Var

iabl

e us

edIn

dica

tor

used

to

eval

uate

to m

easu

reto

iden

tify

U

sed

othe

rpr

ogra

m im

pact

prog

ram

impa

ctel

igib

ility

sta

tus

cont

rol v

aria

bles

?D

ata

sour

ces

Aut

hors

Nut

ritio

nC

hild

hei

ght (

12-

to 3

6-m

onth

-old

chi

ldre

n)2D

IFE

1/E

2Y

esIN

SP E

valu

atio

n da

taB

ehrm

an a

nd

Hod

dino

tt (2

000)

Con

sum

ptio

nTo

tal v

alue

of

cons

umpt

ion

(foo

d an

d no

n-fo

od)

CSD

IFE

2Y

esE

NC

EL

-Nov

98

Hod

dino

tt, S

kouf

ias,

and

tota

l cal

oric

ava

ilabi

lity

Was

hbur

n (2

000)

EN

CE

L-J

un 9

9E

NC

EL

-Nov

99

Intr

afam

ily a

lloca

tion

of ti

me

Part

icip

atio

n in

pai

d an

d un

paid

wor

k ac

tiviti

es

2DIF

E2

Yes

EN

CA

SEH

-Nov

97

Park

er a

nd S

kouf

ias

(mea

sure

exc

lude

s do

mes

tic a

ctiv

ities

)E

NC

EL

-Nov

98

(200

0)E

NC

EL

-Jun

99

EN

CE

L-N

ov 9

9T

ime

spen

t in

a w

ide

rang

e of

act

iviti

es (

incl

udin

g C

SDIF

E2

Yes

EN

CE

L-J

un 9

9Pa

rker

and

Sko

ufia

s do

mes

tic a

ctiv

ities

) du

ring

pre

viou

s da

y(2

000)

Wom

en’s

sta

tus

and

intr

ahou

seho

ld r

elat

ions

hips

Dec

isio

ns r

egar

ding

chi

ldre

n (s

uch

as w

hen

to ta

ke

CSD

IFE

2Y

esE

NC

EL

-Nov

98

de la

Bri

ere

and

child

to th

e do

ctor

, tel

ling

child

to g

o to

sch

ool,

EN

CE

L-J

un 9

9Q

uisu

mbi

ng in

gi

ving

chi

ld p

erm

issi

on to

go

out,

and

expe

nses

A

dato

et a

l. (2

000b

)on

chi

ldre

n’s

clot

hing

)H

ouse

hold

exp

endi

ture

dec

isio

ns (

such

as

food

, C

SDIF

E2

Yes

EN

CE

L-N

ov 9

8de

la B

rier

e an

ddu

rabl

es, a

nd h

ouse

rep

airs

)E

NC

EL

-Jun

99

Qui

sum

bing

in

Ada

to e

t al.

(200

0b)

Dec

isio

ns o

n ho

w to

spe

nd w

omen

’s e

xtra

inco

me

CSD

IFE

2Y

esE

NC

EL

-Nov

98

de la

Bri

ere

and

EN

CE

L-J

un 9

9Q

uisu

mbi

ng in

A

dato

et a

l. (2

000b

)In

terh

ouse

hold

tran

sfer

s an

d ot

her

prog

ram

eff

ects

Inci

denc

e an

d am

ount

of

priv

ate

tran

sfer

s (m

onet

ary

CSD

IFE

1/E

2Y

esE

NC

EL

-Nov

98

Teru

el a

nd D

avis

an

d in

kin

d) a

mon

g ho

useh

olds

EN

CE

L-N

ov 9

9(2

000)

Pove

rty,

ineq

ualit

y, s

choo

l con

tinua

tion,

nut

ritio

n D

IF2

n.a.

Yes

EN

CA

SEH

-Nov

97

Han

da e

t al.

(200

0)su

rvei

llanc

e, a

nd in

flat

ion

rate

s at

the

loca

lity

leve

lE

NC

EL

-Nov

98

Soci

al w

elfa

re (

at th

e na

tiona

l lev

el)

Com

puta

ble

n.a.

Yes

EN

IGH

199

6C

oady

and

Har

ris

Gen

eral

(2

000)

Equ

ilibr

ium

m

odel

inefficiencies/delays in including all eligiblehouseholds on the final list of householdsthat actually receive program benefits, the“treatment of the treated” effect provides an underestimate of the mean effect of the program on those who actually receivedthe benefits of the program (e.g., see Heck-man et al. 1999). Data on the records of payments made out by the PROGRESA ad-ministration could not be obtained until lateinto the evaluation process. After the releaseof these payment records in late August2000, it was found that in the evaluationsample, many of the households that weresupposed to be added to the updated list ofbeneficiaries had never received any cashbenefits since the start of the distribution ofprogram benefits in these localities. Specif-ically, of the 12,291 households in treatmentlocalities eligible to receive PROGRESAbenefits, 3,350 or 27 percent of the total el-

igible population had not received any ben-efits by March 2000. After cross-checkingthis finding with the PROGRESA adminis-tration, it was confirmed that the explana-tion for this discrepancy is that 2,872 house-holds (or 85.7 percent of the eligiblehouseholds not receiving any benefits) hadnot been incorporated into the program. Allof these “forgotten” households werehouseholds with a revised eligibility statusfrom non-beneficiary to eligible beneficiaryas a result of the revision of the selectionprocess (densification). The remaining 478households not receiving any benefits (or14.3 percent of the forgotten eligible house-holds and 3.9 percent of the total eligiblepopulation in treatment areas) were house-holds that were incorporated during theearly months of 1998, and chose either notto participate or to move out of the localitycovered by PROGRESA.50

50There is no official record as to whether these households formally declined the opportunity to participate inthe program.

SUMMARY OF METHODS USED TO EVALUATE IMPACT OF PROGRESA 47

C H A P T E R 6

A Summary of the Findings of the Impact Evaluation and a Cost Analysis of the Program

Impact of PROGRESA on School EnrollmentStudies have shown that the economic returns to children from continuing to enroll in sec-ondary school are relatively large and provide children with opportunities to escape frompoverty. Mexico’s children typically maintain a high enrollment rate in primary school ofabout 93 percent. For the rural poor, however, education often stops there.

There appear to be two critical dips in enrollment rates among rural children in Mexico.Children generally begin dropping out of school after completing the sixth grade, when en-rollment rates decline to 55 percent (see Figure 6.1). But the trend in enrollment once againwitnesses a steep decline during the transition to senior secondary school or tenth grade, whereenrollment once again falls, to 58 percent for those qualified to enter.

The most critical objective of PROGRESA’s education program is to increase the transi-tion of poor rural youth into junior secondary school (seventh to ninth grade). By design, ed-ucational grants for enrolling in the first year of junior secondary school increase by 50 per-cent, with a small advantage to girls over boys in the first three years of secondary school.

MethodologyPROGRESA’s effect on school enrollment is evaluated at two levels: first, by comparing foreach grade completed simple differences in average enrollment rates of children in treatment(i.e., PROGRESA) and control localities; and second, by comparing differences in enrollmentoutcomes at the level of the individual child between those who are program eligible and thosewho are not receiving benefits. Family and community factors are controlled for in this lateranalysis. To ensure confidence in the results, the robustness of the estimated impact of PRO-GRESA is also examined by comparing the impact of PROGRESA using two different sam-ples of children. One sample consists of the children who are present in all five rounds of thesurveys; the other consists of all observations on all children for whom data are available.

Impact on Enrollment RatesAfter an exhaustive series of statistical tests, it was concluded that in all cases PROGRESAhad a positive enrollment effect for both boys and girls, primary and secondary levels, and ir-respective of the sample used. Figure 6.2 displays the impact of the program on the meanschool attendance of children.

48

At the primary school level, in whichenrollment rates before PROGRESA werebetween 90 and 94 percent, statistical meth-ods that control for the age and family back-ground of children as well as community

characteristics revealed that PROGRESAsucceeds at increasing the enrollment rate ofboys by 0.74 to 1.07 percentage points andof girls by 0.96 to 1.45 percentage points(Schultz 2000a).51

SUMMARY OF FINDINGS AND A COST ANALYSIS 49

Figure 6.1a School enrollment and labor force participation of boys in PROGRESAcommunities prior to program implementation

Source: Parker and Skoufias (2000).

Figure 6.1b School enrollment and labor force participation of girls in PROGRESAcommunities prior to program implementation

Source: Parker and Skoufias (2000).

51For clarification, all the impact estimates discussed in this chapter are statistically significant (i.e., have associ-ated t-values that are greater than or equal to 2). Impact estimates with t-values less than 2 are referred to as in-significant or not statistically significant.

At the secondary school level, in which the initial enrollment rates beforePROGRESA were 67 percent for girls and73 percent for boys, the increase in enroll-ment effects for girls ranged from 7.2 to 9.3percentage points and for boys from 3.5 to5.8 percentage points. This represents a proportional increase of boys from 5 to 8percent and of girls from 11 to 14 percent(Schultz 2000a).

If these program effects could be sus-tained over the period in which a child is ofschool age, the accumulated effect on edu-cational attainment for the average childfrom a poor household would be the sum ofthe estimated change for each grade level.Summing these values for grades 1–9 sug-gests that the program can be expected to in-crease educational attainment of the poor ofboth genders by 0.66 years of additional

50 CHAPTER 6

Figure 6.2a School attendance of children 8–11 years old

Source: Author’s calculation.

Figure 6.2b School attendance of children 12–17 years old

Source: Author’s calculation.

schooling. Girls in particular are gaining0.72 years of additional schooling by theninth grade while boys gain 0.64 years ofadditional schooling (Schultz 2000a). Giventhat the average 18-year-old youth achievedabout 6.2 years of completed schoolingprior to the program, these data are sugges-tive of an overall increase in educational attainment of about 10 percent.

Potential Impact ofPROGRESA on the AdultEarnings of ChildrenIf current urban wages approximate whatPROGRESA’s beneficiaries can expect toearn from their schooling in terms of futurepercentage increases in their wages, the in-ternal rate of return, taking into account thecosts of the grants, to PROGRESA’s educa-tional benefits is roughly 8 percent per year(Schultz 2000a). Children, when they reachadulthood, will have permanently higherearnings of 8 percent as a result of the in-creased years of schooling. Thus, in addi-tion to improving beneficiaries’ currentlivelihood by reducing current poverty andraising consumption, PROGRESA is hav-ing a significant impact on raising overallhuman capital into the future.

Comparing the Effect ofPROGRESA to IncreasedAccess to SchoolsIt should be emphasized that PROGRESAmight have additional impacts on increasingeducation beyond the level of secondaryschool if children are more likely to go on tohigher levels of schooling, implying the es-timates here are lower bounds of the im-pacts of PROGRESA on schooling. Notethat there are higher returns to education in Mexico for high school education thanfor junior high school. These possible im-

pacts would increase the overall impact ofPROGRESA on schooling and should beevaluated in the future.

Increased access to schooling may beconsidered as an alternative to providing educational grants to poor families. For ex-ample, 12 percent of the children in thePROGRESA evaluation sample currentlyhave to travel more than 4 kilometers to ajunior secondary school. The evaluation re-search shows that when access to secondaryschooling is measured in terms of distance,if additional schools were to be built andstaffed so that all children reside only 4 kilo-meters from their junior secondary school,secondary school enrollments would in-crease by 0.46 percentage points for girlsand by 0.34 for boys, impacts less than onetenth the size of those from PROGRESA. Incomparison to the impact of PROGRESA’stargeted educational grants to poor families,the effect of increased access to schoolingappears to be a relatively less effectivemeans of increasing school enrollments.52

Furthermore, a more detailed investi-gation taking into consideration the costsassociated with the options of buildingschools against the alternative of providingcash transfers conditional on enrollment as in PROGRESA revealed that in terms of the objective of getting more childreninto school a demand-side intervention such as PROGRESA is more cost effectivethan a supply-side one (Coady 2000). Inother words, the cost incurred in generat-ing one extra year of schooling is lower inPROGRESA than the alternative of buildingnew schools.

Impact on the Transitionfrom Primary to SecondarySchooling LevelThe impact of PROGRESA on enrollmentrates is largest for children who have com-

SUMMARY OF FINDINGS AND A COST ANALYSIS 51

52It is interesting to note, however, that in the case of Indonesia, Duflo (1999) finds that a primary school con-struction program had a significant impact on increasing school enrollments and earnings.

pleted the sixth grade and are thus qualifiedto enroll in junior secondary school, in-creasing 11.1 percentage points for bothgenders combined or 14.8 percentage pointsfor girls and 6.5 percentage points for boys,representing percentage increases of over20 percent for girls and about 10 percent for boys (Schultz 2000a). These resultsimply that, whereas many children beforePROGRESA would leave school after com-pleting the primary level, an important frac-tion, particularly girls, are now going on tosecondary school.

The available evidence also shows thatmuch of the positive impact on enrollmentis due to increasing continuation rates ratherthan on getting children who were out of theschool system to return to school. For in-stance, for girls (boys) who were attendingschool prior to the program, the impact ofPROGRESA is to increase enrollment ratesby 11 (7–8.5) percentage points. For boyswho were out of school, this impact wasonly 5.4 percentage points. Furthermore,those children who do return to school tendto return only for a year, whereupon theydrop out again, suggesting that the pro-gram’s impact is primarily to increase con-tinuation rates rather than return rates. It isperhaps not surprising that many childrendo not return, given that most of these chil-dren had been out of school several years al-ready at the time PROGRESA was imple-mented. With new generations of children, itis likely that PROGRESA will reducedropout rates, and thus reinforce the effectof PROGRESA to increase continuationrates (Coady 2000).

Impact on Dropout Rates,Progression through Grades, Grade Repetition Rates, and School Reentry RatesThese questions are explicitly addressed ina study by Behrman et al. (2001). Theirfindings show that the participation in theprogram is associated with earlier ages ofschool entry, less grade repetition and bettergrade progression, lower dropout rates, and

higher school reentry rates among dropouts.The program is especially effective in re-ducing dropout rates during the transitionfrom primary to secondary school. In addi-tion, at the secondary level the program ap-pears to be more effective in inducing boysto enroll in the second and third years ofsecondary school, despite the fact that thebenefit levels are slightly higher for girls.The study also finds the program to be ef-fective in inducing children who droppedout prior to the initiation of the program toreenter school. However, it should be notedthat a related analysis by Coady and Parkerin Coady (2000) found that the impacts ofthe program on children who were previ-ously out of school are not sustainable overtime. This suggests that those children whodo return to school tend to do so for only ayear and then drop out again.

PROGRESA and Child LaborThe results show very clear negative im-pacts of PROGRESA on children’s labormarket participation. Estimates based ondouble-difference models of labor forceparticipation before and after the implemen-tation of PROGRESA show important re-ductions in children’s labor force participa-tion for both boys and girls, in both salariedand non-salaried activities. Labor force par-ticipation for boys shows reductions as largeas 15–25 percent relative to the probabilityof participating prior to the program. Forgirls, in spite of their overall lower partici-pation level prior to the program, there arealso significant reductions associated withPROGRESA. In addition, the lower inci-dence of child work due to the PROGRESAprogram is found to account for 65 percent(in November 1999) to 82 percent (in No-vember 1998) of the increase in the enroll-ment of boys in school. In other similar programs, such as the Food for Educationprogram in Bangladesh, the lower incidenceof child labor was found to account for 25percent of the increase in the enrollment ofboys in school (Parker and Skoufias 2000).These estimates suggest that a PROGRESA-

52 CHAPTER 6

like program has the potential of combatingthe problem of child labor.

Impact on Time Spent Doing School HomeworkWhereas PROGRESA has a significant im-pact on the number of children who enrollin school, it thus far does not show a signif-icant impact on the time children spend inschool or on the time they spend afterschool on assigned homework. This sug-gests that the impacts of PROGRESA areprimarily to increase the number of childrenin school and to reduce the number of chil-dren who are working, but not necessarily,for instance, to reduce the hours worked ofchildren who attend school. A substantialnumber of children continue to combineboth work and school under the program(Parker and Skoufias 2000). In addition,analysis of school-standardized tests did notshow any significant impact of PROGRESAin improving student scores on achievementtests (Behrman et al. 2000). Whereas addi-tional years of data are needed to providemore conclusive evidence, the possibility ofincluding bonuses or prizes to provide in-centives for achieving high grades could beexplored.

Impact on School AttendanceA panel sample of data using children ages6–16, some who benefit from PROGRESAscholarships and some who do not, indi-cates that for the school year of 1998/99, attendance rates in schools are higher in localities that are further removed frommajor urban areas but the evaluation re-search clearly shows that PROGRESA has amore pronounced effect on school enroll-ment rates than on attendance rates. Be-cause enrollment does not guarantee atten-dance, this question deserves fullerinvestigation (Schultz 2000b).

Impact of FertilityBy design, the educational benefits ofPROGRESA are targeted to children be-tween 8 and 17 years of age. For these ben-

efits to have a significant effect on the fer-tility decisions of rural men and women itis necessary for households to have confi-dence that these benefits will be continuedfor at least eight years into the future. As ofNovember 1999 there is no statistical evi-dence that female PROGRESA beneficiar-ies had higher fertility than poor females incontrol localities.

Perceptions of StakeholdersRegarding the Operation of theEducational Component of the ProgramAnalysis of the quantitative and qualitativedata revealed that delays in the receipt of ed-ucational grants were common in the earlystages of the program, in part because of thecumbersome nature of the form design usedto register school attendance (Adato et al.2000a). The collection, filling out, and re-turning of forms involved substantial timecosts often incurred personally by school di-rectors. The simplification of the forms ap-pears to have reduced the time it takes to fillthem out and teachers and school directorsseem to be in agreement with the objectivesof the program and the conditioning oftransfers on attendance. Beneficiaries mayhave experienced a lag in the receipt of ed-ucational grants and indeed PROGRESA’sown records reveal that significant delaystook place at the early stages of the pro-gram, primarily owing to delays in the veri-fication of school attendance.

Analysis of the beneficiary surveys sug-gests that, on the supply side, the increaseddemands generated by the program has atleast not led to a degeneration in the qualityof education services, suggesting that re-sources have been increased. In many cases,there seems to have been an improvement.This view is also consistent with evidencefrom the quantitative survey of directors,with most schools reporting some improve-ments in infrastructure and other resources,albeit from a poor initial position. It is clearfrom the qualitative interviews that theprocess of acquiring extra resources is time

SUMMARY OF FINDINGS AND A COST ANALYSIS 53

and resource intensive for teachers andschool directors. But some teachers still com-plain that they lack such basic resources astelevisions for telesecondary schools. It willbe interesting to compare this picture of thesupply side with other data sources. Althoughmost directors in the qualitative interviewsreport improvements in education outcomes,they attribute most of this to improved atten-dance, student interest, and nutrition, ratherthan improvements in the supply side.

Both the quantitative analysis of theschool directors’ survey and the qualitativeanalysis of the focus group interviews sup-port the general perception that PROGRESAhas led to improvements in the attitude ofbeneficiary students and their families to-ward education. The program is viewed asallowing parents whose children were al-ways motivated to acquire education, butwho faced severe economic hardship thatmade it impossible to incur travel and othereducational expenses and who needed anyincome that children could contribute, tocontinue to send their children to school.The fact that lack of resources (or poverty)seems to be a major factor in explainingnonattendance at school, especially forolder children, is consistent with the program design and initial estimates of pro-gram impact (Schultz 2000a) since the education subsidy (or scholarship) seems tohave been effective in increasing demand.

Particularly from the focus-group analy-sis, there is evidence that families place astrong emphasis on school attendance andhomework and that, where possible, parentsattempt to adjust to these demands if chil-dren attend school. This was seen to be anacceptable tradeoff, with others in the familywillingly substituting for schoolgoing chil-dren’s time, especially during the week. Butchildren, in general, appear to have to con-tinue to contribute to household chores, es-pecially at the weekend and during the peakagricultural season. For some children, pos-sibly those from the poorest families or thosewho have long distances to travel to second-

ary school, the balancing of the demands ofschool and work is very demanding.

But children’s lack of interest in schoolis also an important factor in explainingnonattendance at school, especially forolder children, although this appears to be atleast in part indirectly motivated by povertyand the desire of older children to contributeto the family, and the lure of migrationwhich is seen as “progress.” In the case ofolder female children, concern for theirsafety when they have to travel long dis-tances is also an issue.

One of the common complaints in thequalitative interviews with school directorswas that teachers were never consultedabout the objectives and design of the pro-gram or informed how it would function. Inparticular, many could not understand whysome “deserving” students were excluded,why some who need it do not receive it, andwhy they could not have had a role in the se-lection of beneficiaries. Also, parents oftenblame teachers for their children not beingincluded, for delays in transfers, or for theirchild not receiving transfers because of poor attendance. Non-beneficiaries in somecommunities are reluctant to contribute toward school resources, arguing that bene-ficiary families should be relied upon more. They also argue that the demands onthem for school supplies should be less than for beneficiaries. Finally in some casesthe school directors point out that the in-crease in demand has brought in some stu-dents from remote areas who were givenpoor quality education and thus requiremore input from teachers.

In the qualitative interviews with teach-ers we asked them for their overall view ofthe program. Their answers suggested that,on the whole, teachers saw the program asbeing beneficial for the communities andwere in favor of greater participation. Theyinvariably agreed with the objectives of the program as well as the conditioning of transfers. Some even suggested attachingextra conditions such as linking scholar-

54 CHAPTER 6

ships to academic performance. Most werein favor of money transfers, although con-cern for how households spent their moneywere behind some suggestions that food oreducation coupons be introduced. The gen-eral perception was that the supply side wasnot sufficient to deal with the increase in de-mand, although better attendance and atti-tudes to schooling made teaching easier andmore rewarding. Also some schools thatwould have shut down because of insuffi-cient demand could now remain open.While in some cases the promotoras wereviewed as an asset to the school, in othersthere seemed to be some friction, possiblybecause of her perceived “interference” ineducational matters.

The Impact of PROGRESAon Health, Nutrition, and Health Care UseThe use of health care in rural Mexico is ex-tremely low compared to other Latin Amer-ican countries. On average, rural Mexicansmake less than one visit to a medicalprovider per year. The non-poor make about0.8 visits and the poor make about 0.65 vis-its per year.

The nutrition of preschool children is ofconsiderable importance not only because ofconcern over their immediate welfare, butalso because their nutrition in the formativestage of life is widely perceived to have asubstantial and persistent impact on theirphysical and mental development and ontheir health status as adults. Stunting, definedas having a z-score of height-for-age less than–2, is a major form of protein–energy mal-nutrition. Survey results for 1998 indicatethat 44 percent of 12- to 36-month-old chil-dren in PROGRESA regions were stunted.

MethodologiesThe effect of PROGRESA on health is eval-uated at two levels: first, at the level ofhealth clinics based on the administrativerecords of public clinics; second, at the

individual level using data from the PROGRESA evaluation surveys. The analy-sis of the impact of PROGRESA on healthcare centers investigates whether the serviceand incentive provided by the program ledto improved health care and maintenance byexploring the impact on the use of facilitiesin terms of number of visits, and on the pur-pose of these visits, such as the monitoringof the nutritional status of children and theuse of prenatal care.

The facility-level data were obtainedfrom surveys of 3,541 clinics operated byInstituto Mexicano del Seguro Social(IMSS)–Solidaridad from January 1996 toDecember 1998. This information, com-piled from the records of PROGRESA, per-tains to the number of beneficiary familiesincorporated to the program every month ineach clinic. About two thirds of these clinicsare in PROGRESA areas, while the remain-ing one third operates in control areas.

As is the case for the PROGRESA eval-uation survey, the availability of repeatedobservations at the same clinic over time,before and after the start of the program,permitted analysis of the changes over timewithin treatment and control clinics.

The individual level data from the PROGRESA evaluation surveys includedinformation on the utilization of public clin-ics, public hospitals, private providers, theincidence and type of illness, children’s vis-its to clinics for nutritional monitoring, andwhether children have received differenttypes of immunization. Analysis of bloodtests for anemia and other deficiencies didnot form part of this evaluation, althoughthe National Institute of Public Health inCuernavaca has carried out analysis in thisarea. In the last two rounds of the survey,adolescent and adult health status wasmeasured by collecting information for thelast four weeks on the days of difficultywith daily activities due to illness, days in-capacitated due to illness, days in bed due toillness, and the number of kilometers theywere able to walk without getting tired.

SUMMARY OF FINDINGS AND A COST ANALYSIS 55

Impact of Children’s HealthImproving livelihood security for the poordepends on improving early childhoodhealth care. Frequency and duration of ill-ness have profound effects on the develop-ment and productivity of populations. Theanalysis indicates that improved nutritionand preventive care in PROGRESA areashave made younger children more robustagainst illness. Specifically, PROGRESAchildren from birth to five years of age havea 12 percent lower incidence of illness thannon-PROGRESA children do (Gertler2000, 2004).

Impact on Adult HealthThe analysis also finds that adult members inbeneficiary households are significantlyhealthier (see Figure 6.3; Gertler 2000). Onaverage, PROGRESA beneficiaries have 19percent fewer days of difficulty with dailyactivities, 17 percent fewer days incapaci-tated, 22 percent fewer days in bed, and areable to walk about a 7 percent longer dis-tance than non-beneficiaries. Prime agePROGRESA adults (ages 18–50) had a sig-nificant reduction in the number of days ofdifficulty with daily activities due to illnessand a significant increase in the number ofkilometers able to walk without getting tired.Specifically, PROGRESA beneficiaries have19 percent fewer days of difficulty due to ill-ness than non-PROGRESA individuals, andare able to walk about 7.5 percent morewithout getting tired. For those older than50, PROGRESA beneficiaries have signifi-cantly fewer days of difficulty with daily ac-tivities, days incapacitated, and days in beddue to illness than non-beneficiaries. As withyounger adults, they are able to walk morekilometers without getting tired.

Impact on Utilization of Health FacilitiesIn January 1996, more than a year beforePROGRESA began, average visits to clinicswere identical in control and treatment lo-calities. In 1998, the first full year in whichPROGRESA was operational in all treat-

ment localities, visit rates in PROGRESAcommunities were shown to grow faster inPROGRESA villages than in control areas(see Figure 6.4; Gertler 2000). In addition,there was a significant increase in nutritionmonitoring visits, immunization rates, and prenatal care. Regarding prenatal care, the evaluation analysis indicates thatPROGRESA increased the number of firstvisits in the first trimester of pregnancy byabout 8 percent. This shift to early prenatalcare significantly reduced the number of firstvisits in the second and third trimesters ofpregnancy. Thus as a result of PROGRESA,pregnant women make their first visit to theclinic much earlier than before, a positivechange in behavior that is documented tohave a significant improvement in the healthof babies and pregnant mothers.

The analysis of the individual-level dataon health care use by type of provider con-firms that for 18- to 50-year-olds and forthose older than 50, there was no impact onvisits to private providers (Gertler 2000).This suggests that the increase in the use ofpublic clinics was not from substitution outof the private sector, but rather new partici-pation for preventive purposes, from house-holds previously not using public services.

Nutritional Supplements and Child GrowthThe data suggest that PROGRESA has hada significant impact on increasing childgrowth and in reducing the probability ofchild stunting for children in the critical agerange of 12–36 months (Behrman and Hod-dinott 2000). These estimates imply an in-crease of about a sixth (16 percent) in meangrowth per year, corresponding to about 1cm for these children per year. The effectsmay be somewhat larger for children frompoorer households and poorer communitiesbut who come from households with moreeducated household heads. Overall, the ef-fects suggest that PROGRESA had an im-portant impact on growth for the childrenwho received treatment in the critical 12- to36-month age range.

56 CHAPTER 6

SUMMARY OF FINDINGS AND A COST ANALYSIS 57

Figure 6.3a Incidence of illness for newborns to two-year-olds

Figure 6.3b Incidence of illness for three- to five-year-olds

There is evidence that a significant frac-tion of children in PROGRESA are not reg-ularly receiving the supplements (Behrmanand Hoddinott 2000). Furthermore, in somecases, supplements were not fully con-sumed and in several households the sup-plement was shared among other familymembers, suggesting that its effects mayhave been diluted. Increased and more ac-curate distribution of the supplement may

increase the impact of PROGRESA on nu-trition indicators, such as height.

The analysis of the data suggests thatPROGRESA may be having a fairly sub-stantial effect on lifetime productivity andpotential earning of currently small childrenin poor households. IFPRI estimates that theimpact from the nutrition supplements alonecould account for a 2.9 percent increase inlifetime earnings (Behrman and Hoddinott

58 CHAPTER 6

Figure 6.4 Daily visits to public clinics

2000). In addition, there are likely to beother effects through increased cognitivedevelopment, increased schooling, and low-ered age of completing given levels ofschooling through starting when youngerand passing successfully grades at a higherrate. Since the nutrition supplement (papilla)constitutes only a small fraction of the pro-gram costs given full compliance, the benefit–cost ratio of the nutrition supple-ment is likely to be high.

Perceptions of StakeholdersRegarding the Operation of theHealth and Nutritional Componentof the ProgramAnalysis of the quantitative and qualitativedata revealed that the administration of thehealth and nutrition component of the pro-gram has improved considerably (Adato et al. 2000a). In 1999, registration of bene-ficiaries was reported to have reached 97percent and health care professionals reportlittle problems with filling out forms. Appointment books have proven to be an effective mechanism for ensuring compli-ance to scheduled visits despite the reportedlack of time, transportation, and aware-ness of the benefits of preventive healthcare. The health education seminars (pláti-

cas) were found to be widely available, ef-fective, and very popular among beneficiar-ies, promotoras, and health professionals.Problems reported with pláticas in somecases were that male doctors giving talks towomen about family planning and the Papsmear is culturally problematic, and that the participation of non-beneficiaries varieswidely.

Nutritional supplements for the motherand child are very popular among benefici-aries, yet some receive only a fraction of thedaily ration they are supposed to receivefrom the program. Surveys reveal that fam-ilies either run out of supplements; share thesupplements with other household mem-bers; or the supplements are diluted, whichdiminishes their effectiveness. It also ap-pears that the supplements are being distrib-uted to non-beneficiaries, regardless of theirnutritional status.

Impact of PROGRESA’sMonetary Transfers onHousehold ConsumptionExpenditure-based or consumption-basedstandard of living measures are preferable toincome-based measures because estimatesof current consumption are likely to provide

a more reliable estimate of a household’spermanent income than estimates of currentincome that is subject to peaks and troughs.Consumption measures what people actu-ally consume and thus provides a bettermeasurement of a household’s standard ofliving.

Measuring consumption is not straight-forward. Households rarely know howmuch they have spent over a given referenceperiod, and experiments in survey design in-dicate that questions about broad categoriesof expenditures tend to lead to underesti-mates of consumption. Thus, the questionsthe evaluation exercise posed to householdsrelated to consumption were narrowed andthen the results were aggregated up.

In each of the evaluation surveys, house-holds were asked a set of questions on ex-penditures for food and non-food goods.The “most knowledgeable individual” in thehousehold was asked, “In the last sevendays, how much did you spend on the fol-lowing foods?” Thirty-six different foodswere queried.

Non-food expenditures are reportedbased on weekly expenditures, monthly ex-penditures, and expenditures made over theprevious six months. These were all con-verted to monthly expenditures and thenconverted into November 1997 prices forcomparable analysis.

The connection between PROGRESA’ssubsidy and both monetary and nonmone-tary private transfers from individuals out-side the household was investigated usingtwo methods of empirical analysis. Descrip-tive statistics compared the frequency andlevel of interhousehold transfers betweennon-beneficiaries and beneficiary groups attwo points in time for which the data wereavailable. Other characteristics of the house-holds that received and did not receive werealso compared. Second, selection into PROGRESA was analyzed econometricallyto determine whether the selection itself hada significant impact on the incidence and levels of existing private transfers, such as re-mittances from individuals working abroad.

Lastly, it is worth noting that the largeincrease in cash that these communities re-ceive as a result of having PROGRESAbeneficiaries is likely to have an effect onlocal economies and the development ofnew markets. Whereas this was not an as-pect that was evaluated, it is an importanttopic that deserves further examination infuture evaluations of PROGRESA and otherconditional cash transfer programs.

Impact on Household Consumption and DietUsing data from the three surveys after thestart of PROGRESA, the average level ofconsumption (including purchases and con-sumption out of own production) increasesby approximately 14.53 percent. (Hoddinottet al. 2000). The rest of the transfers werelikely used for savings or other purchasessuch as durable goods.

In November 1998, median food expen-ditures were only 2 percent higher. How-ever, in November 1999 median food ex-penditures were 10.6 percent higher inPROGRESA households when comparedwith comparable control households (Hod-dinott et al. 2000).

Not only are PROGRESA householdsincreasing overall acquisition of food, theyare choosing to improve dietary quality overcaloric intake. The increase in householdconsumption is driven largely by higher ex-penditures on fruits, vegetables, meats, andanimal products. By November 1999, me-dian caloric acquisition rose by 7.1 percent.There is also clear evidence that dietaryquality has improved in PROGRESAhouseholds (Hoddinott et al. 2000). The im-pact is greatest on the acquisition of caloriesfrom vegetable and animal products. Thesequantitative findings from the seven-day re-call surveys reinforce the views of benefici-aries that access to PROGRESA has meantthat they “eat better.”

Participation in PROGRESA is found tohave a significant impact on the acquisitionof calories from fruits, vegetables, and ani-mal products even after controlling for the

SUMMARY OF FINDINGS AND A COST ANALYSIS 59

effect of increased household income frommonetary transfers (Hoddinott et al. 2000).There is also some evidence that informa-tion conveyed during the pláticas spillsover, and alters, in a positive fashion, the behavior of non-beneficiaries in treatmentlocalities.

A possible concern is that the provisionof the papilla may cause households to di-vert expenditures on food to other items,thus undermining efforts to increase caloricavailability in these households. If thepapilla is truly “crowding out” householdacquisition of calories, we would expect tosee lower measures of impact for benefici-ary households, especially among thosewith preschool children. Statistical analysisof the caloric acquisition in households con-taining at least one child below the age offive revealed that such concerns are un-founded (Hoddinott et al. 2000). The impactof participation in PROGRESA on caloricacquisition is, if anything, slightly higherfor these households.

Impact of PROGRESA on Women’s Status andHousehold Relationships

MethodologyMeasuring the impact of PROGRESA onwomen’s status and household relationshipsis challenging. In general, household sur-veys are blunt instruments in this regard be-cause gender-based decision making isoften understated; without adequate under-standing of the sociocultural context, prob-ing questions can easily be misinterpreted.Thus, this section of the evaluation takes atwo-pronged approach using quantitativeand qualitative surveys to ascertain the posi-tion of women within the household (Adatoet al. 2000b). The analysis seeks to ascertain(1) whether PROGRESA has influencedhousehold relationships and the impact ofwomen’s status and (2) the extent to whichPROGRESA has influenced the attitudes to-ward the education of girls and women.

Several rounds of qualitative surveysconducted over a two-year period asked aseries of questions related to women’s statusand intrahousehold relationships. In addi-tion, related questions were exploredthrough focus groups and interviews con-ducted by IFPRI’s researchers. An addi-tional qualitative research effort took placein 1999 to further investigate questionsraised during the previous surveys. Focusgroups rather than semistructured interviewswere chosen in order to enrich responses.

Impact on Decision Making within HouseholdsPROGRESA’s monetary transfers are a cru-cial aspect of the program with respect tobringing about changes in patterns of deci-sion making within households. While re-siding in a PROGRESA locality is shown tonot have an effect on patterns of decisionmaking, being in PROGRESA decreasesthe probability that the husband is the soledecision maker in five out of the eight decision-making outcomes. In PROGRESAfamilies, over time husbands have shownthey are less likely to make decisions bythemselves, particularly as they affect thechildren. The surveys also indicate that,through time, the probability that womensolely decide on the use of their extra in-come increases.

Impact on Men’s Attitudes toward WomenResearch has shown that by giving moneyto women, PROGRESA forces recognitionamong men, and within the community as awhole, of women’s importance and of thegovernment’s recognition of women’s levelof responsibility in caring for the family.The survey shows that most men do nothave problems with their wives’ participa-tion in PROGRESA. Men see the benefitsas good for the entire family, since salaries,in general, are very low.

In focus group discussions, when asked,respondents indicated that, with a few exceptions, men do not take women’s

60 CHAPTER 6

PROGRESA income. In general, men aresaid to work as hard and still give the sameamount of money as they did before thefamily received PROGRESA.

Impact on Women’s TimeStatistical analysis of time use of womenshows that participation in the programyielded some evidence that the time de-mands on women associated with satisfyingprogram obligations are significant (Parkerand Skoufias 2000). Women in PROGRESAare more likely to report spending time inboth taking household members to schools,clinics, and so forth as well as having agreater participation in community workand faenas. Overall, however, there is nosignificant impact of PROGRESA on theleisure time of both male and female adults.This again provides reinforcing evidencethat adult beneficiaries do not use the bene-fits to work less and increase their leisure, asmay be predicted by some economic mod-els. These results would also seem to sup-port the hypothesis that PROGRESA doesnot create dependency on its benefits, in the sense that it does not appear to reducethe work incentives of adults.

In general accordance with the results ofthe quantitative analysis, focus group dis-cussions revealed that women were evenlydivided as to whether PROGRESA was toodemanding on their time. Those who said itwas demanding referred to the time de-mands of meetings. Women also discussedhow they and sometimes their husbands hadto do additional work that used to be doneby their children. However, they were quickto point out that this was worthwhile inorder for their children to study.

Impact on Women’s Empowermentand Bargaining PowerThe vast majority of responses indicatedthat women have benefited in ways that canbe seen as “empowerment”—defined as in-creased self confidence, awareness, andcontrol over their movements and house-hold resources. Women report that they

leave the house more often; have the oppor-tunity to speak to each other about concerns,problems, and solutions related to thehousehold; are more comfortable speakingout in groups; are becoming more educatedthrough the health pláticas; and have morecontrol over household expenditures.

Impact on Attitudes toward Girls’ EducationPROGRESA’s educational incentives forgirls are based on the belief that the in-creased education of girls is fundamental toimproving their living standards and socialparticipation. In an exploration of attitudestoward girls’ education, the survey foundoverwhelming support among women forgirls’ education.

Yet when faced with the hypotheticaldilemma of sending a boy or a girl to school,most respondents chose the boy. It isthought that boys are favored because ofmen’s responsibility as breadwinners andheads of households and the fact that girlsget married. That said, the main reason toencourage girls’ enrollment in school was toenable girls to get employment, or betteremployment. In general, women in the pro-gram do not understand the concept ofPROGRESA’s incentive to keep girls inschool. Most think that the benefit for girlsis higher than for boys because girls havehigher expenses.

Because responses about girls’ educa-tion were far stronger than statements aboutPROGRESA’s effect on women’s positionwithin the household, it is thought thatPROGRESA will have a far stronger sec-ondary effect on household relationshipsthrough the next generation than the pro-gram is having on this one.

Cost Analysis of PROGRESA

MethodologyIn conducting an economic analysis ofPROGRESA it is necessary to highlight twoof the complicating factors involved. First,

SUMMARY OF FINDINGS AND A COST ANALYSIS 61

in the absence of being able to attach mon-etary valuations to the human-capital im-pacts generated by the program, one is un-able to aggregate across the range ofimpacts in order to undertake unifiedcost–benefit analysis of the program. Sec-ond, on the cost side one faces the concep-tually difficult problem of allocating jointcosts to the various program components.

For these reasons and in order to applycost–benefit (or cost effectiveness) analysisto the evaluation of the program, IFPRI’sevaluation can be characterized as makingtwo types of comparisons:

• Comparisons across different programs• Comparisons across different policy

questions.

In making comparisons across different pro-grams, one can think of a number of differ-ent program designs. Each component ofPROGRESA (i.e., current poverty, educa-tion, and health) may be considered as astand-alone program. Then one can dealwith each of the impacts separately andidentify the costs that would have to be in-curred to generate these impacts in isola-tion. For example, one can focus on the costof transferring income to householdsthrough the program, or the cost of generat-ing the observed human-capital impacts. Allof these hypothetical programs will incurthe joint costs but certain costs will be spe-cific to individual components, for example,the supply-side costs or the costs of moni-toring attendance at schools and health cen-ters. These can then be compared to thecosts that would have to be incurred to gen-erate the same impacts using an alternativeinstrument.

When comparing across different policyquestions one can distinguish between thecosts associated with implementing the pro-gram from scratch (i.e., the actual program),the costs associated with expanding the pro-gram to incorporate more localities (i.e.,program expansion), and the costs associ-ated with continuing the existing program

unchanged (i.e., continuation of the pro-gram). The relevant costs are generallylower in moving from the actual program toprogram expansion to program continua-tion, reflecting the presence of sunk costs.

As explained in more detail in the reportof Coady (2000), the total costs of a pro-gram of the nature of PROGRESA can becategorized as program costs and privatecosts. Program costs capture all the costsassociated with the delivery of cash trans-fers to households such as (1) targetingcosts associated with the targeting of trans-fers to the most marginal localities as wellas only to the poorest households withinthese localities; (2) conditioning costs asso-ciated with ensuring that households meettheir responsibilities by ensuring atten-dance of children at school and householdmembers at scheduled regular preventativecheck-ups; and (3) operation costs associ-ated with the actual operation of the program. Private costs are the costs thathouseholds incur in order to receive cashtransfers. For example, private costs includethe time and financial costs of traveling toschools and health clinics (i.e., due to theconditioning of the program) as well as tocollect the transfers from distributionpoints.

Although information on total privatecosts is in general a useful input into policyanalysis, for the purposes of evaluatingPROGRESA it is only the incremental costsdue to the introduction of the program thatare relevant. For example, in order to qual-ify for the food transfer, household mem-bers must make a series of visits to healthclinics for check-ups and health lectures.One estimate of the private costs incurredby households is that households incur travelcosts of 6.38 pesos per 100 pesos receivedthrough the food transfer (Coady 2000).Such an estimate, however, is substantiallyhigher than the incremental private costs incurred by the household as a result ofPROGRESA. The incremental private costincurred by the household is the cost of theextra trips brought about by the program.

62 CHAPTER 6

According to Gertler (2000), PROGRESAbrought about a 30–50 percent increase inthe number of trips. Using an estimate of a40 percent increase, this implies that only28.6 percent of total trips are additional. Thisin turn implies that the incremental privatecosts of receiving the food transfer are 1.82pesos per 100 pesos received. Approxi-mately the same cost ratio is estimated forthe incremental travel costs incurred byhouseholds sending their older children tosecondary schools outside their locality (1.5pesos per 100 pesos received) and the travelcosts incurred for collecting the bimonthlyPROGRESA cash transfer (1.2 pesos per100 pesos received).

The Program Costs and theTotal Costs of PROGRESAIFPRI’s analysis of PROGRESA’s programcosts consisted of calculating cost–benefitratios that summarize the program cost in-curred in transferring monies to beneficiar-ies. According to the program costs analysis,for every 100 pesos allocated to the pro-gram, 8.2 pesos are administration or program costs. Given the complexity of theprogram, this level of program costs appearsto be quite small. It is definitely relativelylow compared to the numbers given byGrosh (1994) for the Leche IndustrialCONASUPO (LICONSA) and the Subsidioa la Tortilla (TORTIBONO) programs, whichimply program costs of 40 pesos and 14 pesosper 100 pesos transferred, respectively.

By comparing the cost–benefit ratiosacross the different hypothetical programsto that for the actual program, which is tar-geted and provides cash transfers condition-ally, one can also identify the relative im-portance of the different activity costs (seeTable 13 in Coady 2000). For example, thelargest cost component is that associatedwith targeting at the household level. Thisactivity accounts for nearly 30 percent ofthe program cost. This is followed by thecosts associated with conditioning the pro-gram, which account for 26 percent of the

program cost. Thus the costs associatedwith both the targeting and the conditioningof the program make up 56 percent of theprogram’s costs. This also implies that it isimportant to ensure that there is a return tothese activities.

When the incremental private costs dis-cussed above are added to the program costsit is found that the total cost–benefit ratio in-creases by about 27 percent (from 0.089 to0.113). So, for every 100 pesos transferredto households, 11.3 pesos are incurred inadministrative and private costs. The costanalysis also reveals that private costs asso-ciated with participating in the program areas important as household targeting andconditioning costs.

Overall, the administrative costs em-ployed in getting transfers to poor house-holds appear to be small relative to the costsincurred in previous programs and for tar-geted programs in other countries. This is inspite of the program being quite complex,involving both the targeting and condition-ing of transfers and all the costs that suchactivities entail. Although this partly reflectsoperational efficiency, it is important tokeep in mind that the size of the programalso plays an important role in keepingthese numbers low. In combination, thelarge number of households covered by theprogram and the size of the transfers tend toreduce the unit fixed costs of the program.

The Financing of PROGRESAand Its Impact on WelfareThe preceding cost analysis and the evalua-tion of the impact of the program on povertyfocus exclusively either on the costs of op-erating the program or on the direct effectsof the program on beneficiaries. Such par-tial equilibrium analyses may provide onlya limited view of the potential costs or ef-fects of the program since they ignore theindirect effects arising from the need to fi-nance the program domestically. As a mat-ter of principle, in evaluating a program ofthe size and nature of PROGRESA it is alsonecessary to adopt a broader perspective.

SUMMARY OF FINDINGS AND A COST ANALYSIS 63

PROGRESA, for example, may be consid-ered as being financed by the elimination ofsubsidies and various reforms in the struc-ture of value-added taxes. The removal offood subsidies is likely to have a negativeimpact on the welfare of poor households inurban areas where PROGRESA is not yet inoperation; yet, their removal will also haveefficiency gains.

These potential indirect effects of thePROGRESA program are examined using acomputable general equilibrium model ofthe Mexican economy (Coady and Harris2000). Their results show that financing the

program through the elimination of distor-tionary food subsidies is associated with asubstantial welfare gain. The simulation re-sults suggest that there are clear welfaregains from introducing a new efficiently tar-geted program such as PROGRESA; thebenefits from more efficient targeting ofhouseholds is substantial and they are rein-forced by the welfare gains from being ableto reform the existing system of subsidiesand taxes. The results also clearly indicatesubstantial welfare gains from the expan-sion of the PROGRESA program to includethe urban poor.

64 CHAPTER 6

C H A P T E R 7

Conclusions and Future Policy Considerations

Poverty alleviation programs such as PROGRESA are an important component of theset of instruments that governments have at their disposal for redistributing income andassets among households. The availability of a variety of programs and instruments

that can be used gives rise to the need for an evaluation of the impacts of these programs inrelation to their stated objectives.

Program evaluation can improve the design and implementation of programs so that theycan have a larger effect on household welfare. In addition, program evaluation, when appliedconsistently across a wide spectrum of programs, allows governments to have a greater effecton social welfare with the same budget by reallocating resources from less effective to moreeffective programs. In addition to these economic considerations, there are also social and po-litical reasons for justifying program evaluations. Primary among these is that program evalu-ation can serve as a means of increasing the accountability of governments toward their citi-zens by providing a template for comparing sensibly whether public funds are used effectivelytoward poverty alleviation.

In the case of PROGRESA the national elections that were forthcoming in the year 2000and the increasing public support in favor of the opposition parties contributed to an unprece-dented willingness by the Zedillo administration to support a rigorous and politically neutralevaluation of the program. Undoubtedly, the decision to evaluate PROGRESA was meant toserve a political purpose by signaling a break from the wasteful practices of the past and thusindirectly increasing the administration’s chances of reelection. However, at the same time thedecision to evaluate the program established a precedent that any future administration couldhardly afford not to imitate.

With these considerations in mind, the majority of the evaluation findings suggest thatPROGRESA’s combination of education, health, and nutrition interventions into one inte-grated package has a significant impact on the welfare and human capital of poor rural fami-lies in Mexico. The initial analysis of PROGRESA’s impact on education shows that the pro-gram has significantly increased the enrollment of boys and girls, particularly of girls, andabove all at the secondary school level (Schultz 2000a). In addition, most of the increase inschool attendance is attributable to children, especially boys, working less. The results implythat children will have, on average, about 0.7 years of extra schooling because of PROGRESA,although this effect may increase if children are more likely to go on to senior high school asa result of PROGRESA. Taking into account that higher schooling is associated with higherlevels of income, the estimations imply that children have lifetime earnings that are 8 percenthigher due to the education benefits they have received through PROGRESA. As a result of

65

PROGRESA, both children and adults are also experiencing improvements inhealth. Specifically, children receivingPROGRESA’s benefits have a 12 percentlower incidence of illness as a result of theprogram’s benefits and adults report a de-crease in 19 percent of sick or disabilitydays (Gertler 2000). In the area of nutrition,PROGRESA has had a significant effect onreducing the probability of stunting for chil-dren 12–36 months of age (Behrman andHoddinott 2000). PROGRESA has also had important impacts on food consump-tion. Program beneficiaries report highercalorie consumption and are eating a morediverse diet, including more fruits, vegeta-bles, and meat. The program is also found tohave no apparent effects on the work incen-tives of adults, while the award of the cashbenefits to mothers in beneficiary house-holds appears to have led to the empower-ment of women.

A detailed cost analysis of the programalso provides strong evidence that the pro-gram is generally administered in a cost-effective manner. For example, for every100 pesos allocated to the program, 9 pesosare “absorbed” by administration costs(Coady 2000). Given the complexity of theprogram, this level of program costs appearsto be quite small and definitely relativelylow compared to the numbers for roughlycomparable programs.

One of the most important contributionsof IFPRI’s evaluation of PROGRESA wasthat the program was continued in spite ofthe historic change in the government ofMexico after the 2000 election. The over-whelming (and unprecedented) evidencethat a poverty alleviation program showssigns of having a significant impact on thewelfare and human capital investment ofpoor rural families in Mexico has con-tributed to the decision of the Fox adminis-tration to continue with the program and toexpand its coverage in the poor urban areasof the country after some improvements inthe design of the program.

The majority of the improvements in thedesign of PROGRESA (renamed Oportu-nidades by the Fox administration) werebased on findings of the evaluation of PROGRESA that revealed areas of neededimprovements in some of the structuralcomponents and the operation of the pro-gram. For example, the evaluation revealeda larger program impact only on the school-ing attendance of children of secondaryschool age. This suggests that it would bepreferable to reorient the funds from pri-mary school to families with children ofsecondary school age. Oportunidades didexactly that by extending the benefits of the program to children attending highschool (preparatoria) rather than just juniorhigh school, as it was in the earlier PROGRESA. Also, initially the award ofPROGRESA’s educational benefits wasconditional on regular school attendance butnot performance. Oportunidades improvedon this design feature by linking benefits toperformance, such as granting bonuses toencourage successful completion of a grade,or linking benefits with participation inother programs. For example, the creationof a related program, Jovenes con Oportu-nidades, aims to create income-generatingopportunities for poor households throughpreferential access to microcredit, housingimprovements, adult education, and socialinsurance.

Yet in spite of these improvements in theprogram, the evaluation findings suggestthat some issues remain to be resolved. Forexample, the qualitative analysis revealedthat although the program strengthened social relationships between beneficiarywomen, the targeting of the program withinthese communities has introduced some social divisions between beneficiaries andnon-beneficiaries. The quantitative evalua-tion of PROGRESA’s targeting method sug-gests that in poor rural communities it maybe preferable to include all residents into theprogram instead of discriminating amonghouseholds. Also, the program was found to

66 CHAPTER 7

have no measurable impact on the achieve-ment test scores of children in beneficiarylocalities or on their regular school atten-dance. This implies that if the program is to have a significant effect on the humancapital of children more attention needs tobe directed to the quality of education pro-vided in schools. Enrolling in and attendingschool regularly are only necessary condi-tions for the improvement of children’shuman capital. Last, it is also important to find ways to maintain and improve the quality of the information provided inthe pláticas.

The opportunity to conduct a rigorousevaluation of the PROGRESA program hasset a higher set of standards for the designand conduct of social policy in Mexico andin Latin America in general. As policy-makers now have a better sense of the basicelements of a program that can be effectivetoward alleviating poverty in the short runand, perhaps, in the long run, the list ofquestions and concerns about programchoices and design cannot help but growlonger. For example, is it possible for uncon-ditional cash transfers without any “strings”attached to have similar or higher impact onhuman capital investments of poor ruralfamilies? Is the amount of the cash transfergiven to families too high? Perhaps a lowercash transfer could achieve the same im-pact. Is the simultaneous intervention in theareas of education, health, and nutritionareas preferable to intervening in each ofthese sectors separately? PROGRESA hasbeen accompanied by complimentary ef-forts and resources directed at strengtheningthe supply and quality of educational andhealth capacity constraints that might ariseas a result of the more intensive use of ex-

isting facilities and resources. Perhaps thisfeature of the program plays a critical role,as PROGRESA and programs that do notpay sufficient attention to the capacity con-straints that might arise as a result of theconditionality of cash transfers may be lesseffective. Is it not possible that similar oreven higher effects on school attendancecan be achieved through alternative pro-grams, such as building new schools or im-proving the quality of educational services?What if the benefits were given to fathersrather than the mothers in the household?Are programs aimed toward younger chil-dren to be preferred over programs aimedtoward older children?

The nature of the program and the scopeof the program’s impact evaluation can pro-vide only a tentative answer to some ofthese questions. More definite answers canbe obtained through the analysis and evalu-ation of programs that incorporate all orsome of these features as part of their struc-ture. It is hoped that early involvement of researchers in the design and evaluation of programs implemented in other LatinAmerican countries, such as Brazil, Hon-duras, Nicaragua, Colombia, Jamaica, andArgentina, can shed some light on thesecritical questions for policy.

Finally, the critical question of whetherthe vicious cycle of poverty and its inter-generational transmission are indeed brokencan be determined only by following the co-horts of children currently in the program.At least in Mexico, the evaluation of PROGRESA’s impact in the short term hasprovided a solid foundation for determiningwhether the program was able to have a sig-nificant difference in the welfare and earn-ings of these children as adults.

CONCLUSIONS AND FUTURE POLICY CONSIDERATIONS 67

A P P E N D I X A

Summary of Mexican Anti-Poverty Programs

It is important to note that the Mexican government distinguishes between three types ofanti-poverty programs.53 These include programs aimed at (1) human capital development,(2) income earning opportunities, and (3) infrastructure development. The first two types

of programs are benefits provided at the individual or household level whereas the third groupis aimed at the community or regional level. This document covers principally programs in thefirst two groups. It is important to note that neither Programa de Becas de Capacitacion paraDesempleados (PROBECAT) nor Programa de Apoyos Directos al Campo (PROCAMPO) isclassified as an anti-poverty program by the Mexican government although we describe theseprograms in the following paragraphs.

During the past few years, anti-poverty programs have undergone several important tran-sitions. First, overall spending has increased in real terms by about 20 percent over the pastfive years (Poder Ejecutivo Federal 2000). Second, there has been an increasing tendency to-ward giving states and municipalities greater control over resources and some consequent de-centralization of programs. Third, there has been a gradual transition toward a relativelygreater participation of rural areas in terms of spending. For instance, in food and nutritionsubsidies, whereas in 1994 rural areas received only 31.4 percent of spending, by the year2000 rural areas were receiving 76.4 percent of all spending on these programs. Overall spend-ing on anti-poverty programs shows similar trends. By the year 2000, 76 percent of all anti-poverty spending was dedicated to rural areas whereas in 1994 only 48 percent of all anti-povertyspending was spent in rural areas (Poder Ejecutivo Federal 2000). Finally, there has also beena gradual transition away from general subsidies and toward targeted programs. Again, refer-ring to spending on food and nutrition subsidies, in 1994 targeted programs received only 39percent of overall spending, whereas by the year 2000 95.5 percent of all spending was on tar-geted programs (Poder Ejecutivo Federal 2000; Subsecretaría de Egresos 2000a).

Human Capital Development

DICONSAAn important social supply program, DICONSA provides basic consumer goods, includingmilk, tortillas, and other food items, at low prices in 23,200 stores in rural areas, benefiting29.2 million individuals in 2000. DICONSA helps to guarantee that basic products are avail-able in isolated areas at an affordable price. Its objective communities are those with high andvery high margination with community size between 500 and 4,000 inhabitants.

68

53This appendix was prepared with the help of Susan W. Parker.

TORTIBONO and LICONSATwo other important food subsidy programsinclude TORTIBONO, which currently pro-vides approximately 1.7 million poor fami-lies with one free kilogram of tortillas dailythrough producers affiliated with the pro-gram, and LICONSA, which operates amilk subsidy program, supplying milk at areduced price to, in 2000, 2.4 million poorhouseholds with children younger than 12years of age, corresponding to 4.2 millionchildren. The average subsidy of LICONSAresults in a savings of approximately 52 per-cent per liter of milk with respect to equiv-alent brands of milk. It is important to note that both TORTIBONO and LICONSAcannot operate in the communities wherePROGRESA benefits are received. Never-theless, both TORTIBONO and LICONSAare in the process of transition toward usingthe same selection mechanism as that ofPROGRESA in terms of choosing house-holds in eligible communities.

DIFThe DIF (National System for Integral Family Development) operates three differ-ent nutritional programs. The largest is itsschool breakfast program which during theyear 2000 gave a total of 3 million freebreakfasts daily to children in preschoolsand primaries. DIF has two other sub-pro-grams that include Programa de AsistenciaSocial Alimetaria a Familias (PASAF) andProgram de Cocinas Polulares y Unidadesde Servicios Integrales (COPUSI). PASAFprovides a monthly despensa (package ofbasic food products) to families in mar-ginated urban and rural areas, benefiting inthe year 2000 1.7 million families. COPUSIalso provides hot breakfasts, and assisted571,000 individuals in 2000 (Poder Ejecu-tivo Nacional 2000).

CONAFE (National Council to Promote Education)CONAFE is part of the Secretary of PublicEducation (SEP) and distributes school sup-plies to children in isolated and marginated

areas as well as didactic materials, schoolequipment, and resources to support parent–teacher associations. It is important to notethat the benefits of CONAFE have beenlargely concentrated in the same communi-ties that are served by PROGRESA. Theoverall budget of CONAFE in the year 2000was about 400 million dollars, benefitingabout 4.5 million children (Subsecretaría deEgresos 2000b).

INIThe general objective of Instituto NacionalIndigenista (INI, National Indigenous Insti-tute) is to design and implement public poli-cies oriented toward indigenous communi-ties. In practice, INI has a wide range ofobjectives ranging from cultural research,social and economic development, andhuman rights. As part of its actions in pro-moting investment in human capital, INIprovides albergues escolares, which are res-idences that provide lodging and food to in-digenous students from communities whereeducation services are not available or in-sufficient. They also provide educationgrants to promote students at the junior highand high school levels. Its operation is supported by community committees thatsupervise and participate in resource alloca-tion. In 1999, the coverage of INI included1,082 albergues with a total of 59,823 stu-dents. It also provided 12,000 educationgrants to students at junior and senior highschool.

Niños de SolidaridadOther grants received by children in isolatedand highly marginalized regions derivefrom Programa Estímulos a la EducaciónBásica (Program Incentives to Basic Educa-tion), financed by Fondo para la Infraestruc-tura Social Municipal (FISM) (Fund forMunicipal Social Infrastructure), whichconsists of funds decentralized to munici-palities under the spending areas of Ramo33. Note that this program was formallycalled Niños de Solidaridad. These grantsare given to children who are not receiving

SUMMARY OF MEXICAN ANTI-POVERTY PROGRAMS 69

PROGRESA grants; nevertheless it is per-mitted that communities and even house-holds with PROGRESA benefits have chil-dren receiving these grants. The onlyrestriction is that the same child is not receiving an education grant from bothPROGRESA and from Programa Estímulosat the same time (PROGRESA 2000). About560,000 children received these grants in theyear 2000 (Subsecretaría de Egresos 2000b).

Income-GeneratingOpportunities

FONAESFondo Nacional de Apoyo a Empresas Sociales (FONAES) (National Social Enter-prise Fund) contributes to generating em-ployment and income opportunities throughthe financing of productive projects withrisk capital and other forms of credit (5,000projects last year with a budget of about 80million dollars). The most common activi-ties that have been financed include com-merce and fishing projects.

PET The Temporary Employment Program(PET), begun in 1995, is another importantsource of income for families that focusesresources in rural areas in Mexico. During2000, through the Secretariat of Social De-velopment, 518,996 temporary jobs werecreated in Mexico, with work involving im-provements in basic infrastructure, roadsand highways, irrigation, and reforestingprojects. Through the Secretariat of Com-munications and Transportation, another278,000 temporary jobs were created. Fi-nally, the Secretariat of Agriculture andRural Development began operating withinthe Temporary Employment Program sup-porting 228,000 producers. In all, more than one million temporary jobs were cre-ated with PET in the year 2000. The objec-tive of the program is to respond to possiblelack of work in rural areas due to different

farming seasons and differences in produc-tive activity. About 90 percent of all jobswere created in rural zones.

Other Programs PotentiallyReceived by PROGRESABeneficiaries

PROBECATOne of the most important training pro-grams in Mexico is the Program of TrainingScholarships for Unemployed Workers(PROBECAT). Here, unemployed individu-als receive short training courses (generallylasting less than 3 months), in accordancewith the economic activities common totheir region and requirements of firms withvacancies in the area. During the period inwhich they receive the training, they aregiven a grant equivalent to one minimumsalary. Coverage in this program grew sig-nificantly between 1995 and 1997, from412,318 recipients in 1995 to 552,186 re-cipients in 1999, corresponding to about25,000 training courses. It is important toemphasize that almost half of the coursesand grants given correspond to the Projectof Local Initiatives on Temporary Employ-ment, a program that tries to incorporateproductive projects to the population livingin marginalized areas.

CIMOThe Modernization and Integral QualityProgram (Programa Calidad Integral yModernizacion [CIMO]) gives trainingcourses on systems to improve productivity,mainly in very small firms. During the year1999, it benefited 760,000 workers in about418,000 firms.

PROCAMPO (Programa de Apoyos Directos al Campo)This cash transfer program is provided bythe secretary of agriculture to producer/farmers who produce any of the following

70 APPENDIX A

crops: corn, beans, wheat, rice, soy, cotton.The farming area (number of hectares) de-termines the amount of the cash transfer,which currently ranges from 700 to 800

pesos per hectare depending on the season.In the year 2000, approximately 2.9 millionproducers benefited, covering a square areaof approximately 14 million hectares.

SUMMARY OF MEXICAN ANTI-POVERTY PROGRAMS 71

A P P E N D I X B

Characteristics of the Localities in the Evaluation Sample

All Control Treatment

Committees present in the localityMunicipal president 0.04 0.02 0.05Municipal agent 0.40 0.42 0.38Municipal sub-delegado 0.35 0.37 0.34Ejidal marshal 0.37 0.47 0.32Communal property marshal 0.08 0.11 0.07Committee of municipal development 0.14 0.14 0.14Health committee 0.63 0.61 0.64Education committee 0.75 0.73 0.76Agriculture committee 0.14 0.12 0.15Cattle ranching committee 0.07 0.07 0.07DICONSA clerk 0.20 0.20 0.21Indigenous language–speaking inhabitants 0.42 0.32 0.48

Locality infrastructureWater from community well 1.00 1.00 0.99Flowing water 0.98 0.96 0.99Stagnant water 0.98 0.95 0.99Water truck 0.98 0.95 0.99Potable water 0.98 0.95 0.99Garbage is burned 0.99 0.99 0.99Garbage is buried 0.98 0.95 0.99Garbage is put in open fields 0.97 0.94 0.99Garbage is put in public facility 0.97 0.94 0.99Garbage is left for a truck to collect it 0.97 0.94 0.99Electricity 0.76 0.97 0.64Drainage system 0.16 0.23 0.12Public phone 0.25 0.33 0.21Private phone 0.02 0.02 0.02Movie theater 0.00 0.00 0.00Post office 0.03 0.05 0.02Telegraph office 0.01 0.02 0.01

Educational facilities in the localityPreschool 0.82 0.83 0.82Primary school 0.97 0.95 0.98Telesecondary 0.17 0.25 0.13Secondary school 0.01 0.01 0.01High school 0.01 0.02 0.00

72

All Control Treatment

Higher education (CONALEP) 0.00 0.00 0.00Higher education (CETA) 0.00 0.00 0.00Higher education (CETIS) 0.00 0.00 0.00Higher education (CEBTA) 0.00 0.00 0.00Higher education (CEBTIS) 0.00 0.00 0.00Higher education (other) 0.00 0.00 0.00

Health facilities in the localityHealth ministry clinic 0.10 0.13 0.08IMSS-SOLIDARIDAD clinic 0.04 0.05 0.03IMMS clinic 0.00 0.00 0.00ISSSTE clinic 0.00 0.00 0.00Private doctors 0.00 0.00 0.00Medical aids 0.60 0.62 0.58Dispensary 0.07 0.09 0.06Midwives 0.32 0.25 0.36Witch doctors 0.12 0.12 0.13Other health care workers 0.03 0.01 0.05Mobile health centers 0.75 0.76 0.74Visits of private doctor to locality 0.06 0.03 0.07Pregnancy supervision 0.28 0.26 0.29Delivery supervision available 0.25 0.24 0.25Baby checkups available 0.32 0.26 0.35Immunizations available 0.79 0.77 0.81Diarrhea supervision 0.50 0.42 0.55Family planning 0.44 0.39 0.47Hospitalization 0.05 0.03 0.06

SalariesDaily official minimum salary of agricultural

workers in locality 25.21 24.79 28.17Actual daily salary 25.34 24.69 29.81

Economic activitiesMain first activity is agriculture. 0.97 0.99 0.97Main first activity is commerce. 0.01 0.01 0.01Main first activity is cattle ranching. 0.01 0.00 0.01Main first activity is art and crafts production. 0.00 0.00 0.00Main first activity is construction. 0.00 0.00 0.00Main first activity is industrial production. 0.00 0.00 0.00Main first activity is services. 0.00 0.01 0.00Main first activity is mining. 0.00 0.00 0.00Main second activity is agriculture. 0.02 0.01 0.03Main second activity is commerce. 0.12 0.15 0.10Main second activity is cattle ranching. 0.22 0.15 0.26Main second activity is art and crafts production. 0.02 0.01 0.03Main second activity is construction. 0.03 0.02 0.03Main second activity is industrial production. 0.00 0.01 0.00Main second activity is services. 0.01 0.01 0.00Main second activity is mining. 0.00 0.00 0.00Main third activity is agriculture. 0.00 0.00 0.00Main third activity is commerce. 0.03 0.03 0.03Main third activity is cattle ranching. 0.02 0.00 0.03Main third activity is art and crafts production. 0.01 0.00 0.02Main third activity is construction. 0.02 0.01 0.03Main third activity is industrial production. 0.00 0.00 0.00

(continued )

CHARACTERISTICS OF LOCALITIES IN EVALUATION SAMPLE 73

All Control Treatment

Main third activity is services. 0.01 0.00 0.01Main third activity is mining. 0.00 0.00 0.00

Markets and product suppliesPublic market 0.00 0.00 0.00DICONSA shop 0.19 0.19 0.19Supply warehouse 0.00 0.00 0.00Grocery shop 0.36 0.43 0.32Weekly market 0.01 0.01 0.02Regional market 0.00 0.00 0.01Traveling market (1) 0.01 0.01 0.01Traveling market (2) 0.03 0.03 0.02Household commerce 0.39 0.40 0.38Traveling vendor 0.18 0.24 0.14Pharmacy 0.00 0.00 0.01DICONSA (filter) 0.19 0.19 0.19Can buy maize in locality? 0.39 0.40 0.38Can buy maize flour in locality? 0.47 0.46 0.47Can buy beans in locality? 0.56 0.57 0.55Can buy rice in locality? 0.64 0.68 0.62Can buy sugar in locality? 0.70 0.74 0.68Can buy milk in locality? 0.47 0.52 0.43Can buy eggs in locality? 0.62 0.68 0.58Can buy oil or lard in locality? 0.71 0.74 0.68Can buy meat in locality? 0.06 0.09 0.04Can buy chicken in locality? 0.12 0.13 0.11Can buy soap, toothpaste, etc. in locality? 0.67 0.73 0.64Can buy medicines in locality? 0.09 0.10 0.09Can buy school supplies in locality? 0.34 0.38 0.32

Government programs in localityCommunity kitchens program 0.04 0.04 0.04Distribution of DICONSA milk 0.08 0.06 0.10Provisions? 0.45 0.48 0.43Tortilla de Solidaridad Program 0.00 0.00 0.01Scholarships from Solidaridad 0.66 0.67 0.66Scholarships from PROBECAT 0.02 0.01 0.02Temporary employment program (PET) 0.12 0.12 0.12

Source: Locality Socio-Economic Characteristics Survey (ENCASEL Nov-97).

74 APPENDIX B

A P P E N D I X C

On the Impact of PROGRESA on Poverty

This appendix discusses in more detail the estimated impact of PROGRESA on povertyand provides some background discussion for the income per capita measure used as an indicator of poverty.54 The November 1997 ENCASEH survey as well as the

November 1998, June 1999, and November 1999 ENCEL surveys collected detailed informa-tion on income earned or received from a variety of sources for each individual in the house-hold. The survey instrument used to collect individual and household income for these varioussources changed significantly beginning with the November 1998 survey. With this caveat in mind, it should be noted that a serious effort was made to maintain comparability of in-come by source across the survey rounds. The various sources of income (excluding the PROGRESA cash transfers) were transformed into monthly income and then aggregated intofour main sources of income:

1. Labor income2. Income from self-employment (such as income from sewing, food preparation, construc-

tion or carpentry, commerce, produce transportation, repairs, and laundry or cooking)3. Other income (such as pensions, interest income, rents, and community profits)4. Government transfers (such as educational scholarships from Niños de Solidaridad,

benefits from Instituto Nacional Indigenista (INI), PROBECAT, Empleo Temporal, andProcampo)

For households in treatment villages receiving PROGRESA cash transfers, total income permonth was adjusted upwards by the cash transfer per month received by the household. Theactual amount of cash transfers received per month was obtained from the records of paymentssent out each month since May 1998 by the PROGRESA administration headquarters in Mex-ico City. The monthly income measure calculated for each round of the survey was then ex-pressed into November 1998 pesos by dividing by the corresponding adjustment ratio of thenational consumer price index.

The first item examined concerned the incidence of receipt of benefits from other govern-ment programs. Households receiving PROGRESA benefits should not, in principle, be re-ceiving other similar benefits from programs such as Abasto Social de Leche, de Tortilla, andthe National Institute of Indigenous People (INI). Figure 4.1 suggests that among beneficiaryhouseholds (i.e., those that received any PROGRESA benefits between May 1998 and No-vember 1999) the incidence of benefits received from DIF, Ninos de Solidaridad, and Abasto

54This appendix was prepared with the help of Claudia Aburto-Szekely.

75

Social de Leche decreased dramatically. Asdiscussed in the first part of Chapter 5, theset of beneficiary households is not identicalto the set of eligible households. Benefi-ciary households are defined to be the (eli-gible) households that actually receivedPROGRESA benefits. These householdswere identified based on the payment recorddata. Specifically, for households in treat-ment localities, a household is classified as a beneficiary (BEN = 1) as long as thehousehold received a positive amount ofcash benefits since the start of PROGRESAin March 1998 and the November 1999round of the evaluation survey (BEN = 0otherwise).

Second, we examined how the incomecontributed to beneficiary families by chil-dren between ages 8 and 17 evolved acrossdifferent survey rounds. Children can con-tribute income to families by working forwages or by being recipients of cash trans-fers from other government transfer pro-grams excluding PROGRESA. Figure C.1areveals that the total (labor + other) income(excluding PROGRESA cash transfers)beneficiary families received from childrendeclined in both treatment and control lo-calities since the initiation of PROGRESAin November 1998. Note that for compari-son purposes we use the set of all eligiblehouseholds in control localities (E2 = 1).The mean total income reported in Novem-ber 1998 is slightly lower among treatmenthouseholds compared to control house-holds and the gap gets even bigger by theJune 1999 round. By November 1999 thisgap is completely eliminated as controlhouseholds are already incorporated intoPROGRESA.55

Figures C.1b and C.1c break down totalincome into its two components, that is, in-come from labor and other income that con-

sists mainly of government transfers. Thesegraphs reveal that the differences in meantotal income from children in beneficiaryhouseholds and eligible households in con-trol localities are primarily due to drops inthe child-related income beneficiary fami-lies received from other government pro-grams. It also appears that there are no sig-nificant differences in the labor income ofchildren from beneficiary households intreatment localities and the labor incomecontribution of children in eligible house-holds in control villages.

Next the impact of the PROGRESA cashtransfers on poverty was estimated using theincome per capita as reported in (and con-structed from) the various household sur-veys. For this purpose we used two differentpoverty lines. The first one is the value of thestandard food basket (canasta basica) in No-vember 1997 pesos. The second poverty lineused is the median or 50th percentile of thevalue of consumption in November 1998(expressed in November 1997 pesos).

The availability of poverty estimates intreatment and control localities before andafter the start of the PROGRESA programprovides the opportunity to calculate a dif-ference-in-differences (2DIF) estimate ofthe impact of PROGRESA’s cash transferson the poverty rate in the sample. Such estimates allow for the possibility of pre-existing differences in poverty betweentreatment and control localities as well asfor the role of aggregate or macroeconomicshocks that affected all localities during thetime period between the first survey roundin November 1997 and subsequent surveyrounds.

Tables C.1 and C.2 present the estimatedpoverty rates in treatment and control local-ities in each survey round as well as a 2DIFestimate of the impact of PROGRESA’s

76 APPENDIX C

55Note that control households started receiving cash benefits in December 1999. Households are first incorpo-rated into PROGRESA, meaning that they are given all the necessary forms and informed of all the program re-quirements. A few months later, the cash benefits are sent out by the PROGRESA administration headquarters.

IMPACT OF PROGRESA ON POVERTY 77

Figure C.1a Mean household total income from childhood (excluding PROGRESA cashtransfers) among beneficiary households with children 8–17 years of age

Figure C.1b Mean household labor income from children among beneficiary householdswith children 8–17 years of age

Figure C.1c Mean household other income from children among beneficiary householdswith children 8–17 years of age

78 APPENDIX C

Table C.1 Poverty measures and difference-in-difference tests for total income per capita using canasta basica as poverty line

Standard StandardMeasure Mean error t 2DIF error t

Head count ratioOct-97 Control 0.927 0.003 302.889Oct-97 Treatment 0.926 0.002 383.197Oct-98 Control 0.935 0.003 325.836Oct-98 Treatment 0.932 0.002 406.049 –0.002 0.005 –0.418Jun-99 Control 0.946 0.003 356.493Jun-99 Treatment 0.937 0.002 416.080 –0.008 0.005 –1.602Nov-99 Control 0.940 0.003 347.807Nov-99 Treatment 0.925 0.002 378.144 –0.014 0.005 –2.594

Poverty gapOct-97 Control 0.575 0.003 174.386Oct-97 Treatment 0.598 0.003 223.418Oct-98 Control 0.610 0.003 186.859Oct-98 Treatment 0.594 0.003 233.816 –0.038 0.006 –6.496Jun-99 Control 0.658 0.003 191.019Jun-99 Treatment 0.624 0.003 232.768 –0.057 0.006 –9.280Nov-99 Control 0.593 0.003 189.077Nov-99 Treatment 0.527 0.003 209.916 –0.089 0.006 –15.273

Squared poverty gapOct-97 Control 0.409 0.003 118.037Oct-97 Treatment 0.441 0.003 153.006Oct-98 Control 0.450 0.004 125.878Oct-98 Treatment 0.430 0.003 158.585 –0.052 0.006 –8.129Jun-99 Control 0.518 0.004 130.365Jun-99 Treatment 0.473 0.003 158.554 –0.077 0.007 –11.486Nov-99 Control 0.428 0.003 126.858Nov-99 Treatment 0.350 0.003 138.566 –0.110 0.006 –17.789

cash transfers. The standard errors reportedfor the Foster-Greer-Thorbecke (FGT)poverty indices are calculated using themethod proposed by Kakwani (1993).

Specifically, the 2DIF estimate of theimpact of PROGRESA on the poverty ratedemoted by P between round R, where R = 2, 3, 4, and the first round of the survey(R = 1) is calculated as

2DIF = (PTREAT(R) – PTREAT (1)) – (PCONTROL(R) – PCONTROL(1)).

Inspection of Tables C.1 and C.2 revealsthat irrespective of the poverty line used(i.e., the value of basic food basket or the

median value of household consumption)the 2DIF estimates imply that PROGRESAhad a significant impact in reducing povertybetween November 1997 and November1999. For example, using the 50th per-centile of the value of consumption percapita as a poverty line suggests that theheadcount poverty rate declined by 17 per-cent in treatment areas between November1997 and November 1999 (using as base the67.4 percent poverty rate in treatment local-ities in November 1997). Over the same pe-riod, and using as base the correspondingvalue of the poverty gap and squaredpoverty gap indices in treatment areas inNovember 1997, the poverty gap measure

declined by 36 percent, and the severity ofpoverty measure declined by 46 percent.These estimates are very much in line withthe estimates obtained using simulations

and provide further confirmation that theimpact of PROGRESA is concentrated atimproving the welfare of the poorest of thepoor households in marginal rural areas.

IMPACT OF PROGRESA ON POVERTY 79

Table C.2 Poverty measures and difference-in-difference tests for total income per capita using 50th percentile of per capita value of consumption as poverty line

Standard StandardMeasure Mean error t 2DIF error t

Head count ratioOct-97 Control 0.652 0.006 116.437Oct-97 Treatment 0.674 0.004 156.024Oct-98 Control 0.698 0.005 130.130Oct-98 Treatment 0.681 0.004 160.306 –0.039 0.010 –3.922Jun-99 Control 0.758 0.005 150.238Jun-99 Treatment 0.712 0.004 170.030 –0.068 0.010 –7.011Nov-99 Control 0.694 0.005 132.232Nov-99 Treatment 0.599 0.005 131.543 –0.117 0.010 –11.783

Poverty gapOct-97 Control 0.319 0.004 82.296Oct-97 Treatment 0.357 0.003 110.179Oct-98 Control 0.364 0.004 89.969Oct-98 Treatment 0.343 0.003 111.617 –0.060 0.007 –8.378Jun-99 Control 0.444 0.005 98.214Jun-99 Treatment 0.392 0.003 115.398 –0.090 0.008 –11.921Nov-99 Control 0.339 0.004 89.219Nov-99 Treatment 0.248 0.003 89.104 –0.129 0.007 –18.622

Squared poverty gapOct-97 Control 0.211 0.004 59.182Oct-97 Treatment 0.252 0.003 81.895Oct-98 Control 0.253 0.004 65.227Oct-98 Treatment 0.231 0.003 82.093 –0.063 0.007 –9.439Jun-99 Control 0.344 0.005 74.477Jun-99 Treatment 0.288 0.003 87.587 –0.097 0.007 –13.182Nov-99 Control 0.226 0.004 –63.378Nov-99 Treatment 0.152 0.002 62.201 –0.115 0.006 18.065

81

References

Adato, M. 2000. Final report: The impact of PROGRESA on community social relationships. Septem-ber. Report submitted to PROGRESA. Washington, D.C.: International Food Policy ResearchInstitute (on enclosed CD in English and in Spanish).

Adato, M., D. Coady, and M. Ruel. 2000a. Final report: An operations evaluation of PROGRESA fromthe perspective of beneficiaries, promotoras, school directors, and health staff. August. Reportsubmitted to PROGRESA. Washington, D.C.: International Food Policy Research Institute (onenclosed CD in English and in Spanish).

Adato, M., B. de la Brière, D. Mindek, and A. Quisumbing. 2000b. Final report: The impact of PROGRESA on women’s status and intrahousehold relations. August. Report submitted toPROGRESA. Washington, D.C.: International Food Policy Research Institute (on enclosed CDin English and in Spanish).

Baker, J. 2000. Evaluating the impact of development projects on poverty: A handbook for practition-ers. Washington, D.C.: World Bank.

Becker, G. S. 1981. A treatise on the family. Cambridge, Mass.: Harvard University Press.

Becker, G. S., and N. Tomes. 1976. Child endowments and the quantity and quality of children. Journalof Political Economy 84 (4, pt. 2): S143–S162.

Behrman, J. 2000. Literature review on interactions between health, education and nutrition and the potential benefits of intervening simultaneously in all three. September. Washington, D.C.: International Food Policy Research Institute.

Behrman, J., and J. Hoddinott. 2000. An evaluation of the impact of PROGRESA on pre-school childheight. July. Report submitted to PROGRESA. Washington, D.C.: International Food Policy Research Institute (on enclosed CD in English and in Spanish).

Behrman, J., and J. C. Knowles. 1999. Household income and child schooling in Vietnam. The WorldBank Economic Review 13 (2): 211–256.

Behrman, J., and P. E. Todd. 1999a. Randomness in the experimental samples of PROGRESA (educa-tion, health, and nutrition program). February. Report submitted to PROGRESA. Washington,D.C.: International Food Policy Research Institute (on enclosed CD in English and in Spanish).

———. 1999b. A preliminary evaluation of the sample sizes used for the evaluation of the education,health, and nutrition program (PROGRESA) of Mexico. January. Report submitted to PROGRESA. Washington, D.C.: International Food Policy Research Institute (on enclosed CDin English and in Spanish).

Behrman, J., P. Sengupta, and P. Todd. 2000. Final report: The impact of PROGRESA on achievementtest scores in the first year. September. Report submitted to PROGRESA. Washington, D.C.: International Food Policy Research Institute (on enclosed CD in English and in Spanish).

———. 2001. Progressing through PROGRESA: An impact assessment of a school subsidy experi-ment. April. Washington, D.C.: International Food Policy Research Institute (on enclosed CD inEnglish).

Bergstrom, T. 1997. A survey of theories of the family. In Handbook of population and family econom-ics, ed. M. R. Rosenzweig and O. Stark. Amsterdam: North-Holland.

Buddelmeyer, H., and E. Skoufias. 2004. An evaluation of the performance of regression discontinuitydesign on PROGRESA. Policy Research Working Paper No. 3386 (September). Washington,D.C.: World Bank.

Coady, D. 2000. Final report: The application of social cost–benefit analysis to the evaluation of PROGRESA. November. Report submitted to PROGRESA. Washington, D.C.: InternationalFood Policy Research Institute (on enclosed CD in English and in Spanish).

Coady, D., and R. L. Harris. 2000. Final report: A general equilibrium analysis of the welfare impact ofPROGRESA transfers. April. Report submitted to PROGRESA. Washington, D.C.: Interna-tional Food Policy Research Institute (on enclosed CD in English and in Spanish).

Das, J., Do, Q. T., and B. Ozler. 2004. Conditional cash transfers and the equity-efficiency debate. Policy Research Working Paper No. 3280 (April). Washington, D.C.: World Bank.

De Janvry, A., and E. Sadoulet. 2002. Targeting and calibrating educational grants: Focus on poverty oron risk? Paper presented at the First Meeting of the Social Policy Monitoring Network of theInter-American Development Bank.

Duflo, E. 1999. Schooling and labor market consequences of school construction in Indonesia: Evidencefrom an unusual policy experiment. Mimeo, MIT.

———. 2000. Grandmothers and granddaughters: Old age pension and intra-household allocation inSouth Africa. Mimeo, MIT.

Gertler, P. J. 2000. Final report: The impact of PROGRESA on health. November. Report submitted toPROGRESA. Washington, D.C.: International Food Policy Research Institute (on enclosed CDin English and in Spanish).

———. 2004. Do conditional cash transfers improve child health? Evidence from PROGRESA’s controlrandomized experiment. American Economic Review Papers and Proceedings 94 (2): 336–341.

Grosh, M. 1994. Administering targeted social programs in Latin America: From platitudes to practice.Washington, D.C.: World Bank.

Haddad, L., J. Hoddinott, and H. Alderman, eds. 1997. Intrahousehold resource allocation in develop-ing countries: Methods, models, and policy. Baltimore, Md.: The Johns Hopkins UniversityPress for the International Food Policy Research Institute.

Hahn, J., P. Todd, and W. Van der Klaauw. 2001. Identification and estimation of treatment effects witha regression-discontinuity design. Econometrica 69 (1): 201–209.

Handa, S., M.-C. Huerta, R. Perez, and B. Straffon. 2000. Final report: Poverty, inequality, and “spill-over” in Mexico’s education, health, and nutrition program. April. Report submitted to PROGRESA. Washington, D.C.: International Food Policy Research Institute (on enclosed CDin English and in Spanish).

Heckman, J. J. 1992. Randomization and social policy evaluation. In Evaluating welfare and trainingprograms, ed. C. Manski and I. Garfinkel. Cambridge, Mass.: Harvard University Press.

Heckman, J. J., and J. Smith. 1995. Assessing the case for social experiments. Journal of Economic Per-spectives 9 (2) Spring: 85–110.

Heckman, J. J., R. LaLonde, and J. Smith. 1999. The economics and econometrics of active labor mar-ket programs. In Handbook of labor economics, vol. 3A, ed. O. Ashenfelter and D. Card. Amsterdam: North-Holland.

Hernandez, D., J. Gomez de Leon, and G. Vasquez. 1999. El Programa de Educacion, Salud y Ali-mentacion: orientaciones y componentes. Chapter 1 in Mas Oportunidades para las FamiliasPobres: Evaluacion de Resultados del Programa de Educacion, Salud y Alimentacion, PrimerosAvances, 1999. Secretaria de Desarollo Social, Mexico City.

Hoddinott, J., E. Skoufias, and R. Washburn. 2000. The impact of PROGRESA on consumption: A finalreport. September. Report submitted to PROGRESA. Washington, D.C.: International Food Policy Research Institute (on enclosed CD in English and in Spanish).

82 REFERENCES

Jacoby, H., and E. Skoufias. 1997. Risk, financial markets, and human capital in a developing country.Review of Economic Studies 64 (3): 311–335.

———. 1998. Testing theories of consumption behavior using information on aggregate shocks: Incomeseasonality and rainfall in rural India. American Journal of Agricultural Economics 80 (1):1–14.

Kakwani, N. 1993. Statistical inference in the measurement of poverty. Review of Economics and Statistics 75 (4): 632–639.

Killingsworth, M. R. 1983. Labor Supply. Cambridge Surveys of Economic Literature. Cambridge:Cambridge University Press.

Levy, S. 1994. La pobreza en Mexico. In Causas y políticas para combatirla, ed. F. Vélez. La pobrezaen Mexico. ITAM y FCE Mexico City, pp. 15–112.

Murray, D. M. 1998. Design and analysis of group-randomized trials. New York: Oxford UniversityPress.

Newman, J., L. Rawlings, and P. Gertler. 1994. Using randomized control designs in evaluating socialsector programs in developing countries. The World Bank Research Observer 9 (2): 181–201.

Parker, S., and E. Skoufias. 2000. The impact of PROGRESA on work, leisure and time allocation. October. Report submitted to PROGRESA. Washington, D.C.: International Food Policy Research Institute (on enclosed CD in English and in Spanish).

Poder Ejecutivo Federal. 2000. Sexto Informe de Gobierno. September.

PROGRESA. 2000. Reglas Generales para la Operación del Programa de Educación, Salud y Ali-mentación. Documentos de Divulgación.

Ravallion, M. 1999. The mystery of the vanishing benefits: Ms Speedy Analyst’s introduction to evalu-ation. Policy Research Working Paper No. 2153. Washington, D.C.: World Bank.

Rawlings, L., and G. Rubio. 2002. Evaluating the Impact of Conditional Cash Transfer Programs:Lessons from Latin America. Washington, D.C.: World Bank.

Rivera, J. A., G. Rodríguez, T. Shamah, J. L. Rosado, E. Casanueva, I. Maulén, G. Toussaint, and A. García-Aranda. 2000. Implementation, monitoring, and evaluation of the nutrition componentof the Mexican Social Programme (PROGRESA). Food and Nutrition Bulletin 21 (1): 35–42.

Rosado, J. L., J. Rivera, G. Lopez, and L. Solano. 2000. Development, production, and quality controlof nutritional supplements for a national supplementation program in Mexico. Food and Nutri-tion Bulletin 21 (1): 30–34.

Rosenzweig, M. 1988. Human capital, population growth, and economic development: Beyond correla-tions. Journal of Policy Modeling 10 (1): 83–111.

Rosenzweig, M. R., and R. E. Evenson. 1977. Fertility, schooling, and the economic contribution of chil-dren in rural India: An econometric analysis. Econometrica 45: 1065–1079.

Rosenzweig, M. R., and T. P. Schultz. 1983. Estimating a household production function: Heterogene-ity, the demand for health inputs, and their effects on birthweight. Journal of Political Economy92 (October): 723–746.

Rosenzweig, M. R., and K. I. Wolpin. 1982. Governmental interventions and household behavior in adeveloping country: Anticipating the unanticipated consequences of social programs. Journal ofDevelopment Economics 10: 209–225.

Sahn, D., and H. Alderman. 1995. Incentive effects on labor supply of Sri Lanka’s rice subsidy. In Pub-lic spending and the poor: Theory and evidence, ed. D. van de Walle and K. Nead. Baltimore,Md.: The Johns Hopkins University Press, published for the World Bank.

Schultz, T. P. 2000a. School subsidies for the poor: Evaluating a Mexican strategy for reducing poverty.June. Report submitted to PROGRESA. Washington, D.C.: International Food Policy ResearchInstitute (on enclosed CD in English only).

REFERENCES 83

———. 2000b. Final report: The impact of PROGRESA on school enrollments. April. Report submit-ted to PROGRESA. Washington, D.C.: International Food Policy Research Institute (on en-closed CD in English and in Spanish).

———. 2000c. Impact of PROGRESA on school attendance rates in the sampled population. February.Report submitted to PROGRESA. Washington, D.C.: International Food Policy Research Insti-tute (on enclosed CD in English and in Spanish).

Secretary of Social Development. 2000. www.sedesol.gob.mx, accessed March 3, 2005.

Skoufias, E. 1994a. Using shadow wages to estimate labor supply of agricultural households. AmericanJournal of Agricultural Economics 76 (2) May: 215–227.

———. 1994b. Market wages, family composition and the time allocation of children in agriculturalhouseholds. Journal of Development Studies 30 (2) January: 335–360.

Skoufias, E., and S. W. Parker. 2001. Conditional cash transfers and their impact on child work andschooling: Evidence from the PROGRESA program in Mexico. Economia 2 (1) Fall: 45–96.

Skoufias, E., B. Davis, and J. Behrman. 1999a. Final report: An evaluation of the selection of benefici-ary households in the education, health, and nutrition program (PROGRESA) of Mexico. June.Report submitted to PROGRESA. Washington, D.C.: International Food Policy Research Insti-tute (on enclosed CD in English and in Spanish).

Skoufias, E., B. Davis, and S. de la Vega. 1999b. An addendum to the final report: An evaluation of the selection of beneficiary households in the education, health, and nutrition program (PROGRESA) of Mexico. December. Report submitted to PROGRESA. Washington, D.C.: International Food Policy Research Institute (on enclosed CD in English and in Spanish).

———. 2001. Targeting the poor in Mexico: Evaluation of the selection of beneficiary households intoPROGRESA. World Development 29 (10): 1769–1784.

Subsecretaría de Egresos. 2000a. El Presupuesto de Egresos de la Federación 1995–2000. September.

———. 2000b. Proyecto de Preupuesto de Egresos de la Federación para el Ejercicio Fiscal.

Teruel, G., and B. Davis. 2000. Final report: An evaluation of the impact of PROGRESA cash paymentson private inter-household transfers. August. Report submitted to PROGRESA. Washington,D.C.: International Food Policy Research Institute (on enclosed CD in English and Spanish).

Valadez, J., and M. Bamberger. 1994. Monitoring and evaluating social programs in developing coun-tries: A handbook for policymakers, managers, and researchers. EDI Development Studies.Washington, D.C.: World Bank.

Willis, R. 1974. A new approach to the economic theory of fertility behavior. Journal of Political Econ-omy 81 (2, pt. 2): S14–S64.

Yaschine, I. 1999. The changing anti-poverty agenda: What can the Mexican case tell us? IDS Bulletin30 (2): 47–60.

84 REFERENCES