Session8

Monitoring and Evaluation (M&E)

1

Session Objective

• Devise a simple monitoring plan for a PHE activity or project

2

Monitoring & Evaluation

• PHE projects complex

• Different timeframes

• Need to demonstrate impact

• Contribute to data in field

3

Monitoring & Evaluation

• M&E also builds greater transparency and accountability in use of project resources

• Information generated through M&E provide project staff with a clearer basis for decision-making

• Future project planning and development is improved when guided by lessons learned from project experience

4

Relationship Between M&E

• M & E are two different management tools that are closely related, interactive and mutually supportive

• Through routine tracking of project progress, monitoring can provide quantitative and qualitative data useful for designing and implementing project evaluation exercises

• Through the results of periodic evaluations, monitoring tools and strategies can be refined and further developed

5

Comparison Between M&EItem Monitoring Evaluation

Frequency Regular, ongoing Episodic

Main action Keeping track/oversight Assessment

Basic purpose

Improving efficiency Adjusting work plan

Improve effectiveness, impact, future programming

Focus Inputs/outputs, process outcomes, work plans

Effectiveness, relevance, efficiency, impact, sustainability

Information sources

Routine systems, field visits, stakeholder meetings, output reports, rapid assessments

Same plusSurveys (pre-post project)Special studies

Undertaken by

Project/program managersCommunity workersSupervisorsCommunity (beneficiaries)Funders Other Stakeholders

External evaluatorsCommunity (beneficiaries)Project/program managersSupervisors Funders

Adapted from UNICEF, A UNICEF Guide for Monitoring and Evaluation: Making a Difference? New York, 1991, p.3

6

Commonalities of M&E

• Both monitoring and evaluation must be planned at the program/ project level

• Baseline data and appropriate indicators of performance and results must be established

7

Monitoring• In practice, covers a wide range of activities

and requires data collection, but data collection is not synonymous with monitoring

• Monitoring also implies analysis and use of the data

• Generally, the level that records information should be able to use it

• Designing data collection systems with this principle in mind helps improve chances that the data will be collected carefully and put to use.

8

Planning a Monitoring System

1.What should be monitored?– Keep information requirements to a bare minimum– Collect info that will be most helpful to those who

will use it

2.How? – Select methods to track indicators/report on

progress

• Observations, interviews, routine reporting, sentinel sites• Piggyback on existing data collection systems• Both formal/informal and quantitative/qualitative methods • Decide how information will be recorded systematically

and reported clearly• Consider the time and skills of those who will collect the

data• Pretest new monitoring instruments

9

Planning a Monitoring System

3. Who should be involved when?– Clearly identifying who will collect information on

indicators, when (frequency) and who will receive it

– The monitoring plan should also identify who will be involved in reviewing progress and providing feedback

4. What resources are needed and available?– The human and financial cost of gathering,

reporting and reviewing data should be identified

– Needed funding and time should be set aside for this work

10

Planning a Monitoring System

5. Consultation and Training– Discuss the monitoring program with a

representative group from each level before it is put into effect

– Provide training to those who will be using the monitoring systems

6. Prepare a workplan– for each year – listing the main activities to be carried out,

their output, timing and parties involved

11

Example : IPOPCORM Monitoring

Purpose:

• Serve as the basis for establishing the project’s information systems

• Generate information that will allow project managers to:– track achievement of project outputs

(immediate results of project activities) , and – monitor progress towards achievement of

objectives and desired outcomes (interim results created by outputs)

12

Table 1. Plan to monitor achievement of IPOPCORM outputs (immediate results of project activities)

Component

OutputMonitoring Indicator

Means of Collection (HOW)

Frequency

(WHEN)

Responsible Agency/Person

(WHO)

Policy1. No. of Municipal CRM Plan with

RH strategies approved by local governments

Municipal Planning and

Development Offices

Annual LGU partner

Planning and

Management

1. % of barangays with functioning MPA management committees

Barangay council record +

NGO Output report

Quarterly NGO Supervisor

Training

1. No. of NGO personnel trained in PHE integration/outreach service delivery

2. Change in knowledge on PHE and RH/FP and CRM among trained NGO personnel

3. No. education sessions delivered to fisherfolk on alternative livelihood

Training Reports

Pre-post training test

scores/Training Reports

NGO output report

Semi-annually

Semi-Annually

Quarterly

PATH Foundation Trainer

PATH Foundation Trainer

NGO Supervisor

Services

1. No. of people’s organizations with CBD outlet

2. No. FP clients served by CBDs, by method

3. Net Peso value of socially marketed products

NGO output report

CBD service statistics

CBD service statistics

Quarterly

Monthly

Monthly

NGO Outreach Worker

(for all indicators)

Integrated IEC

1. No. of linked RH-CRM messages and materials created

2. No. of PHE campaign events delivered in the community during fiesta periods

3. Audience reaction to PHE campaigns and linked RH-CRM messages

NGO output report and

portfolio of IEC materials

NGO output reports

FGD

Quarterly

Quarterly

Annually

NGO Outreach Worker

NGO Outreach Worker

NGO Supervisor

13

Table 2. Plan to monitor progress toward IPOPCORM objectives and desired outcomes

Objective IndicatorMeans of

Collection Frequenc

yResponsible Agency/Person

Objective 1: Improve reproductive health outcomes among people living in coastal communities

1. Contraceptive prevalence among WRA

2. Correct knowledge about emergency contraception (EC) among WRA

3. % of youth who used any method of contraception during first sex

Behavior Monitoring Survey (BMS)

FGD

BMS

Every 24 mo.

Every 24 mo.

Every 24 mo.

NGO Supervisor

Outreach Workers (OW)

NGO Supervisor

Objective 2: Enhance management of marine and coastal resources at the community level

Biophysical Indicators:1.percent live coral coverage 2.hectares of mangrove coverageManagement Indicators1.No. and size (km2) of Marine Protected Areas (MPA) under improved management 2.No. of MPA management committees (MMC) established and active3.No. of fish wardens deputized/active in surveillance and enforcement work4.No. of validated infractions reported

MPA monitoring survey(for both biophysical

Municipal Coastal Database (MCD)

NGO output report

MMC logbookMMC logbook

Annually

Annually

Quarterly

QuarterlyQuarterly

Peoples Organization (MPA management unit)

LGU partner

NGO Outreach Worker

NGO Outreach WorkerNGO Outreach Worker

Objective 3: Increase public and policymakers’ awareness and support for PHE

1.Number of policymakers knowledgeable

about the inter-relationships betweenPopulation and Environment and Food security

2.Total value of resources leveraged from local governments and communities for implementation of PHE activities

Policy makers’ survey

Audit of NGO’s financial records and statements

Annual

Every 24 mo.

NGO Project Director

PATH Foundation Philippines auditor

14

TASK• Draft a monitoring plan for your project

• Make sure that it is tied to your– Conceptual Model– Results Chain Factors– Objectives– Indicators– Interventions/Activities

• Use the Worksheet entitled “PHE Monitoring Plan Tool” (see next slide)

Time: 60 minutes15

PHE Monitoring Plan Tool

Objective

Intervention/Results Chain Factor

Indicator Target(Projected Result by EOP)

Means of Collection

Frequency

ResponsibleParty

16

1st Column – insert your pre-determined time-bound objective

2nd Column – insert your pre-determined Intervention and the associated Result Chain Factor

3rd Column – list 2 or more of the indicators you selected for the specific intervention-results chain factor set. Try to include an outcome indicator as well as process indicators.

4th Column - insert a performance target for each indicator – e.g. quantitative estimate of expected result by EOP

5th Column – insert the Means (HOW) you will gather the information

6th Column – insert the Frequency of data collection (WHEN)

7th Column – Insert the name of the party/agency/person who will be primarily responsible for collecting/ reporting the information

How to Use the Tool

17

Objective

Intervention/Results Chain Factor

Indicator Target(Projected Result by EOP)

Means of Collection

Frequency

ResponsibleParty

Improved RH outcomes among people living in coastal areas by 2004

Establish CBD outlets and services

RCF#1Increase FP access

RCF#2Increase FP method use among women or reproductive age (WRA) and sexually active youth

No. of CBD points established

Increase from 0 in 2001 to at least 500 by 2004

NGO output reports

Quarterly NGO Outreach workers

No. FP clients

served by CBDs

Increase from 0 in 2001 to at least 8000 by 2004

CBD records and sales data

Monthly NGO Outreach workers

CPR among WRA

Increase from 20% in 2001 to 35% by 2004

Behavior Monitoring Survey (BMS)

Annually NGO Supervisor and Outreach Workers

% of youth (15-24) whoused anycontraceptiv

e method

duringfirst sex

Increase from 5% in 2001 to 20% by 2004

BMS Annually NGO Supervisor and Outreach Workers

Sample Worksheet – Sectoral Intervention

18

Objective Intervention/Results Chain Factor

Indicator Target(Projected Result by EOP)

Means of Collection

Frequency ResponsibleParty

Increase awareness among policy makers and the general public about linkages between population, environment and consumption dynamics and food security outcomes in coastal Philippines

IEC and advocacy targeting local policymakers and the general public with messages about PHE linkages and CRM-FP approaches

RCF#1Increase support for integration of FP into CRM policies/agendas at the local level

# Municipal Mayors reached with policy advocacy communication messages

# of Municipal CRM plans that include FP activities and budget allocations

50 Municipal Mayors by EOP

25 plans by EOP

Minutes of convergence meetings

NGO output report

Municipal CRM plans + NGO output report

Annually

Annually

PHEField Supervisor

PHE Field Supervisor

RCF #2Increase public awareness and involvement in CRM-FP activities

# of PHE public education campaigns conducted

20 by EOP

NGO output report

Annually PHEOutreachWorker

# villages that develop and implement linked CRM-FP action plans & activities

100 villages by EOP

Village Development Plans

Annually PHE Outreach Worker

Sample Worksheet – Integrated IEC Intervention

19

Sample Worksheet - Value-Added InterventionObjective Interventio

n/Results Chain Factor

Indicator Target(Projected Result by EOP)

Means of Collection

Frequency

Responsible

Party

Community capacitated to manage forest resources in sustainable fashion

Promote ICS

RCF#1Reduce consumption of firewood

RCF#2Reduce time spent on firewood collectionRCF #3Reduce vulnerability to violence among women/girls who gather firewood

# of ICS units installed

100 ICS/yr @ 4yrs=400 ICS units by EOP

NGO service statistics

Every six months

PHE Field Supervisor

Amount ofFirewoodsaved

Yr1=100mt savedYr 2=100+100Yr 3=100+100+100Yr 4=100+100+100+100Total =1000mt saved

Standard calculation1 ICS = saves 1 metric tons of firewood/yr

Every six months

PHE FieldSupervisor

Average # hours/day spent collecting

wood

# of violations

reported bywomen/

girls while Collectingfirewood

Decreasing trend

Decreasing trend

Household survey

Police record; CFUG record

Annually

Annually

CFUG Chief

PHE FieldSupervisor

20

Sample Worksheet – Linked InterventionObjective Interventio

n/Results Chain Factor

Indicator Target(Projected Result by EOP)

Means of Collection

Frequency

Responsible

Party

Community capacitated to implement linked coastal resource management and family planning (CRM-FP) strategies and action

Capacity building for CRM-FP implementation at the sub-village level

RCF#1Increase FP practice among PO members

RCF#2Increase PO involvement in CRM

RCF#3Improve food security at the household level

# Peoples Organizations trained in both CRM and FP service delivery

50 Peoples Organizations by EOP

NGO report & informantinterviews with village captains

Annually PHE FieldSupervisor

% of couples currently using FP methods

No. and area of MPAs establishedBy POs

% of survey respondents who say their family sometimes lacks for food

50% of couples by EOP

5 fish sanctuaries (10 hectares each) established and co-managed by PO by EOP

Reducing trend

Survey among POs

MunicipalDatabase onMPAs

Household survey

Annually

Annually

At least 3 survey rounds conducted every 18 mo.

PO executive committee

PHE FieldSupervisor

PATHFoundationM&E Officer and PHEField Supervisor

21