Building Capacity on Program Evaluation in Latin America: The Experience of the Partnership

between Mexico’s National Institute of Public Health (INSP) and the MEASURE Evaluation Project

Gustavo Angeles, Jose Urquieta, Bernardo Hernandez, Mara Tellez-Rojo, Aurelio Cruz

April 2009

The situation in LAC:

Shortage of professionals with M&E skills, but few places offer M&E training

Increasing demand for M&E services, but few places offer M&E technical assistance to programs

Our Response:

A partnership to build a Center of Reference in M&E

Why a Partnership? INSP recognized the need to strengthen internal capacity to

better respond to demand for M&E services and training

MEASURE Evaluation is only a 5-year project, need to work at institutional-level to achieve sustainability

Establish long-term relationship

Instituto Nacional de Salud Pública (INSP) In Cuernavaca, Morelos, Mexico

Leading center of training, research, & practice in public health in Mexico and Latin America

Closely linked to public health programs

In late 1990s, INSP participated in evaluation of impact of PROGRESA; it took leading role in 2001. Evaluation portfolio has expanded to other social and health programs.

MEASURE Evaluation

USAID-funded; since 1997

Goal : To improve collection, analysis, and presentation of data to promote better use of data for decision making in PHN programs

Implemented by the Carolina Population Center, University of North Carolina at Chapel Hill, with Futures Group, John Snow, Inc., Macro International Inc., Management Sciences for Health, and Tulane University

Areas: development of M&E methods and applications; capacity building; coordination & collaboration

The Partnership

Established in 2004

Objectives:

Strengthen INSP’s capacity to:

offer M&E training programs in a sustainable way

provide M&E technical assistance

conduct evaluation studies

Institutionalization of M&E

Building a cadre of trained M&E professionals in the Latin America region

Range of activities Engage leadership

Assessment of institutional M&E capacity

Training of key faculty

Collaborative development of curricula and training materials

Collaboratively offering training programs: Workshops & Master’s degree training

Provision of instructors, materials, and fellowships

Engage INSP faculty in evaluation research and technical assistance

Work on sustainability (technical, logistical and financial)

Some Results: Training Programs

Short-Term Training in M&E

- 8 Regional Workshops: HIV/AIDS, Impact, PHN, RHIS

- 159 professional trained from 16 countries (515 applicants)

Master’s – level Degree Training in M&E

- Diploma in Program M&E as part of two Master’s Degrees: Master of Public Health (MPH) and Masters of Health Sciences (MHSc)

- 59 students graduated/pursuing diploma in M&E, from 405 applicants

Institutionalization of M&E

Partnership contributed to ongoing process of INSP defining its role in M&E:

INSP adopted “Health Program and Policy Evaluation” as a Strategic Line of Action

Creation of a new “Center for Program Evaluation and Surveys”

Creation of M&E Tracks in 2 Master’s degree programs

Diploma programs in Economic Evaluation of Health Programs and M&E of Nutrition Programs

Increased technical and logistical capacity to offer M&E training programs

INSP Involvement in M&E Practice and Research

Evaluating Mexican government health programs: Oportunidades (formerly Progresa), Seguro Popular

Conducting regional workshops in Latin America

Leading UNAIDS/WB-funded “AIDS Strategic and Action Plan (ASAP)” Consortium of 11 universities to provide TA on HIV Strategic Planning, mainly in Africa

Conducting evaluation studies, published in peer-reviewed journals

Lessons Learned: Keys to Success Identify a “Champion” in M&E

M&E identified as an institutional strategic priority

Identify that there is need for M&E in the health system (a market) and the institution is flexible enough to respond to it

Facilitate incentives for faculty participation in M&E

Link teaching and practice: best way to learn and teach about evaluation is by doing evaluations

Technical aspects are not enough, it is necessary to establish administrative capacity and secure funding sources for long term sustainability

Long-term commitment

Seek institutionalization of M&E trainings

Next steps

Are we meeting the demand for training in M&E?

Review M&E needs in the field

Diversify range of M&E workshop options:

Adapt workshop content

Reach a larger audience: distance learning, “mobile” workshops, virtual Master’s Degree

Improve follow-up with participants, provide support after workshop; forum/community of practice

Formalize a plan for institutionalizing M&E training programs

But, all these require funding, patience, and long-term commitment

MEASURE Evaluation is funded by the U.S. Agency for

International Development (USAID) through Cooperative

Agreement GHA-A-00-08-00003-00 and is

implemented by the Carolina Population Center at the

University of North Carolina at Chapel Hill, in partnership

With Futures Group International, John Snow, Inc., Macro

International Inc., Management Sciences for Health, and

Tulane University. The views expressed in this presentation

do not necessarily reflect the views of USAID or the United

States government.