Performance Monitoring Tools

Performance Monitoring Tools

M&E training for MoHSW & Business

Coalition on HIV/AIDSMaterial adapted from USDOL/ILO

HIV/AIDS Workplace Education programme

Session Objectives

Participants will be able to …1. Explain relationship between the strategic

framework and tools for data collection2. Develop a performance monitoring plan

(PMP)3. Identify tools for data collection4. Develop tools for data collection

Session Overview

Performance measurement Tools and Data Analysis

The Performance Measurement PlanSector specific measurement tools?How is Data Collected, Analyzed and Reported?

Monitoring Tools

• Reporting/analysis - entails obtaining and analyzing documentation from the project that provides information on progress;

• Validation - entails checking or verifying whether or not the reported progress is accurate;

• Participation - entails obtaining feedback from partners and beneficiaries on progress and proposed actions.

Selecting the right mix of monitoring mechanisms

Recap …Selecting the right mix of monitoring

mechanismsReporting & Analysis Validation ParticipationAnnual Project ReportProgress and/or quarterly reportsWork plansProject delivery reportsSubstantive project documentation

Field VisitsSpot-checksClient surveysResearchExternal assessments

Stakeholder meetingsAnnual ReviewSteering Committees

Learning takes place through all monitoring tools or mechanisms

Key Tools of National M&E for the WorkPlace

Tool

National Draft M&E

Framework

Strategic Framework

Performance Monitoring

Plan

Function

Provides guidance in developing strategic framework for M&E and selection of performance indicators.

Develop/capture project strategy (hypothesis)

Defines roles and responsibilities for data collection, sources and tools to use

Key Tools of NERCHA M&E for the Workplace

ToolData Collection

Tools

Data Tracking Table

Participatory Mechanisms

FunctionPeriodic collection of data

Track actual data against baselines and across time

Facilitate analysis and reporting

Communicate project progress and problems to stakeholders

Encourage and inform management reviews

National Draft M&E Framework

Features

1. General principles of the national monitoring and evaluation framework

2. National Indicators by Programme Area

3. The National Monitoring and Evaluation Framework

General Principles of M&EDefinition of monitoring and evaluation

Programme functions that can be monitored and evaluated

Monitoring and evaluation components

Levels of monitoring and evaluation

National Indicators byProgramme Area

Overall goals andprogramme areas of the national HIV/AIDSprogramme

Indicators by categoy

ImpactOutcome, output and process

The National Monitoring and Evaluation Framework

Data Collection StrategyNational surveysDesign checklistsInterview schedules


Data Flow ChannelsData collected by MOHData collected by NERCHAData collected by projects/programmes, business coalition, etc.


Data Management and DisseminationAnnual national review meetingPeriodic newsletterPress releasesWebsite


Stakeholders and PartnershipsNERCHANational M&E CommitteeRegional HIV/AIDS OfficesMoHSW/Other Govt. ministriesNGOsDonors

Strategic Framework

Strategic frameworks are diagrams that identify steps

or levels of results and illustrate the causal

relationships linking all levels of a program’s

objectives.

Strategic FrameworkPurposes:

•Provides a clarified focus on the causal relationships that connect incremental achievement of results to the comprehensive program impact

•Clarifies project/program mechanics and factors’ relationships that suggest ways and means of objectively measuring the achievement of desired ends

Strategic FrameworkReduced level of Employment-related

Discrimination against persons living with or affected by HIV/AIDS

Increased knowledge of HIV/AIDS workplace policy or guidelines

Reduced Stigma against people living with HIV/AIDS

Improved HIV/AIDS workplace policy or guidelines

Increased capacity of workplace to offer comprehensive HIV/AIDS policy andprogrammes on a sustainable basis

Population Level

ProgrammeLevel

Performance Monitoring Plan

Functions of the PMP:Facilitate the collection over time of comparable data Support the management process for data collection and analysisInform the analysis and review of data


In order to compare our plans to performance we need data. And if we are to have data, we need a plan for

acquiring them.

Performance Monitoring PlanIn what units will we collect the data?

Who has the data –what source?

How will we actually gather the data?

How frequently?

Who will do it?

What will it cost?

Data will be collected for each firm 1 year after seed funds are received. One year is counted after the last disbursement of funds. Data will be collected from all qualifying firms (i.e., not a sample survey).

Survey will include questions about net revenues. This data may be used ultimately in this indicator in lieu of expansion question.

Project surveyDefinition: Firms included are those receiving training and/or seed funds directly under LED or QS programs and those vendors/suppliers who are indirectly involved in LED, LMAC/RR or EC.

Business expansion is self reported using a survey that asks Y/N if expansion has occurred.

Disaggregated by LED, LMAC/RR and EC, based on direct and indirect involvement.

Unit: # of assisted firms that report business expansion; among firms assisted, # that report expansion as a % of total firms assisted.

1. Number and percent of existing firms that expandbusinesses over the past year

Immediate Objective 1: Increased business sector activity in target areas“Areas” may include regions, communities, groups, administrative units, associations, organizations, enterprises, countries, or special populations.

RESPONSIBLE PERSON(S) &

TEAM

SCHEDULE BY MANAGEMENT

EVENT

RESPONSIBLE PERSON(S) &

TEAMSCHEDULE/FREQUENCY

ANALYSIS,USE & REPORTINGDATA ACQUISITIONMETHOD/

APPROACHOF DATA

COLLECTION ORCALCULATIONDATA SOURCE

INDICATOR DEFINITION ANDUNIT OF MEASUREMENT

PERFORMANCE INDICATOR

COMMENTS/NOTES:

Quarterly, to capture all results from firms whose on year post-service delivery period terminates in that period.

Local specialists to administer survey to be reviewed by Regional Coordinators.

Periodic management reviews (semi-annual)

Technical Reports (semi-annual)

Regional Coordinator in conjunction with Project Director

Performance Monitoring PlanImmediate Objective 1

Performance Monitoring PlanImmediate Objective 1

Immediate Objective 1



PMP Process – involve the full project team

Improved quality of final indicators and related data

Greater practicality

Increased project level ownership

Sample Data Collection ToolsWorker Survey

Checklist Attributes for Quality Workplace HIV/AIDS Services

Workplace Monitoring Form

Worker Survey

This is a questionnaire for collecting data on behavioural, attitude and knowledge indicators at the population level.

The questions on the survey questionnaire are reflected on the PMP under the data source column.

Worker Survey (extract) …Q Can a person become infected by having

unprotected sex with a person who is infected with HIV?

Yes 1No 2Don’t Know 88No Response 99

Q205 Can a person become infected from a transfusion of blood and its products?


Q206 Can a person become infected by sharing needles with a person infected with HIV?


Q207 Can HIV positive mother infect her unborn child? Yes 1No 2Don’t Know 88No Response 99

Checklist of Quality AttributesThis tool helps define quality attributes for HIV/AIDS Workplace programme components such as …

Workplace Condom SupplyWorkplace VCTProvision of Information on HIV/AIDS ServicesWorkplace HIV/AIDS policyEtc.

Workplace Monitoring Form

This tool is used to collect data for indicators on programme level objectives.

The questions on the form are reflected on the PMP under the data source column.

Workplace Monitoring FormHIV/AIDS WORKPLACE EDUCATION PROJECT WORKPLACE:_______GENERIC WORKPLACE MONITORING FORM FOCAL POINT:_______

Form to be filled in every three months by workplace HIV/AIDS focal point. For further detail regarding the definition of indicators below, refer to the HIV/AIDS workplace Education Project Performance Monitoring Plan (PMP)

Indicator Baseline 1Qtr 2Qtr Comments

Programme Level Objective: HIV/AIDS workplace policy or guidelines

Q101 Does your workplace have a written HIV/AIDS policy or guidelines (Yes/No)?

Q102 If so, is your workplace policy or guidelines communicated to workers through sessions organised that explain the policy or guidelines to workers? (Yes/No)?

Participatory MechanismsParticipatory mechanisms include …

Networks, Consortia, Partnerships

Stakeholder meetings

Steering committees

Focus group interviews

Collecting InformationSelecting instruments for data collection

The types of instruments required will depend on the information you need to collect

All data collection instruments should be thought of as a package of inter-related tools that help you to obtain the information needed to make good management decisions

Collecting Information

Instructions for data collection

It is advisable to prepare a short manual on how to use each instrument for data collectionYour programme indicators will help guide you in developing your instruments

Collecting InformationInstructions for data collection

The manual should describe:Who should fill out the forms

How often and under what circumstances

How the forms should be filled out

Reporting format, frequency of reports

Who to report to

Examples of how information is used

Documents

Performance Monitoring Tools