May 3, 2023
MONITORING & EVALUATION OF HIV & AIDS PROGRAMS - FEBRUARY 2011
DEVELOPING M & E PLANS by DR DC TSHIBANGU
Objectives of the Session
By the end of the session, participants should be able to:
Describe the functions of an M&E plan
Identify the main elements of an M&E plan
Describe the process of developing an M&E plan
Understand how to go about implementing an M&E plan
Overview
Definition: What is an M&E Plan? Function: What is the Plan supposed to
do? Elements: What comprises an M&E
Plan? Standards: What constitutes a good M&E
Plan? Developing and implementing an M&E
plan Introduction to workshop group work:
Developing an M&E plan
M&E Plans – DefinitionComprehensive document that describes all
M&E activities in an M&E system, including: Program objectives, interventions developed
to achieve these objectives, & procedures to be implemented to determine whether or not the objectives are met
Expected results of the program and how they relate to goals and objectives
Data needs, how they will be collected & analyzed
Information use, including resources needed to do so
How the program will be accountable to stakeholders
M&E Plan: Functions States how program will measure
achievements Accountability
Documents consensus Transparency Responsibility
Guides M&E implementation Standardization Coordination
Preserves institutional memoryAn M&E plan is a living document that needsto be adjusted when a program is modified
M&E Plan: ElementsI. IntroductionII. Description of the overall program- including
problem statement and framework(s)III. Indicators- including definitions (presented in
indicator matrix and/or indicator reference sheets: very detailed!)
IV. Data sources and reporting systems (including management/roles and responsibilities)
V. Plans for demonstrating program outcome/impact
VI. Plans for dissemination and use of informationVII. Analysis of data quality constraints & potential
solutionsVIII. Implementation plan (aka- M&E action plan or
road map-should include budget and timeline)
Elements of an M&E Plan: Introduction
A. Purpose of the planB. Description of how it was developed
Stakeholders involved Consensus process
Elements of an M&E Plan: Program Description
A. Problem Statement What is the nature of the HIV-related issue
being addressed?B. Conceptual FrameworkC. Goal and Objectives
What is the ultimate outcome of the program (goal)
What are the shorter-term aims (objectives)D. Program Description:
Interventions Geographic scope Target population Duration
E. Logical Framework/Results Framework
Elements of an M&E Plan: Indicators
A. Selection of indicators based on: Conceptual and logic frameworks Strategic information needed for decision
making at appropriate level (country/state/local) Donor requirements Existing data (really needs to be based in reality) Funding (available and dedicate to M&E)
B. Presented in 2 ways: Indicator Matrix- a table presenting indicators
including information on data source, frequency and who is responsible
Indicator Reference Sheets- detailed sheets describing each indicator, how to measure it, underlying assumptions & interpretation considerations (may be included as appendices)
Elements of an M&E Plan:Data Sources and Reporting Systems
A. Sources of data for indicatorsB. Diagram of data collection, processing,
analysis, and reporting systemC. Data collection tools
Patient records or registers Survey instruments Commodity management forms (e.g.,
condoms) Others?
D. Management Roles and responsibilities of each
group/member of the M&E system1. Data Flow
Elements of an M&E Plan: Plan for demonstrating program outcome/impact
A methodology for measuring program impact (the evaluation)
Protocols for special studies
Elements of an M&E Plan: Dissemination and Use Plan
A. Clearly defined usersB. Databases for information storageC. Dissemination methods
Reports (schedule and audience) Media Speaking events Others?
Elements of an M&E Plan: Data Quality Constraints
Describe known constraints to data quality and/or system performance and what will be done to address these
Elements of an M&E Plan: Implementation Plan
A. Assessment of feasibility to implement plan
B. A detailed work plan for the M&E Plan to include: Each M&E activity (including
update of M&E Plan) Timing of each activity Party responsible for each activity Budget necessary for each activity
M&E Plan: Standards
Utility - serve practical information needs of intended users
Feasibility - be realistic, prudent, diplomatic and frugal
Propriety - conducted legally, ethically, and with regard to those involved in and affected by the evaluation(s)
Accuracy - reveal and convey technically accurate information
Developing an M&E Plan: Inputs
Authority and mandate to implement the M&E plan
Human resources with M&E technical capacity
Understanding of the program Commitment to accountability (not only to
donors, but to the people whom the interventions are to serve) and program improvement
Infrastructure (Technology, space, etc..)
Developing an M&E Plan: Process
Advocate for the need for M&E Assess strategic information needs Assess existing information systems’ capabilities to
address strategic information needs Achieve consensus and commitment among
stakeholders on: Indicators and their definitions Reporting structure, roles and responsibilities Establish a common “language”
Develop mechanism for M&E plan review Prepare document for final approval for all
stakeholders
M&E plan should be developed during theinitial stages of program development
Developing an M&E Plan: Output
A comprehensive document that describes the M&E system and that: Includes all the necessary elements; Has the approval of the governing
authority; Has the consensus of primary
stakeholders.
Implementing the M&E System: Costs
How much will it cost? Need to budget: Costs of information systems (costs of
data collection, processing, and analyzing)
Costs of information dissemination and use
Costs of the data quality control system Costs of coordination and capacity
building
Sample Costs for Selected Data Sources (example from a national system)
Information System Total Annual Costs (2001 US$)
Frequency of Data Collection
National Sentinel System $513,682 Continuous
HMIS $2,119,941 Continuous Integrated Disease Sample Costs for Selected Data Sources (example from a national system) Surveillance
$4,270,943 Continuous
Vital Registration $719,427 Continuous
DHS $854,164 Periodic (every 4 years)
National Census $8,244,114 Periodic (every 10 years)
Source: Rommelmann, et. al. Costs and Results of Information Systems for Poverty Monitoring, Health Sector Reform, and Local Government Reform In Tanzania.
2003
Developing & Implementing an M&E Plan: Role of the M&E Unit
Consensus building – among all stakeholders (donors to program managers to clients)
Coordination Data management and manipulation
(might include data entry, data formatting, data analysis and interpretation…)
Reporting Data auditing Information dissemination Training and capacity building
Consensus Building Tips
Make sure everyone understands what you are trying to achieve consensus on
Ensure that all stakeholders are involved early in the process
Provide stakeholders with the opportunity to provide input and receive feedback
Consistently promote the message that M&E provides the means to demonstrate the extent to which a program is achieving its objectives (“It is everyone’s job and to everyone’s benefit’)
Developing & Implementing an M&E Plan: Dos and Don’ts
Do: Start early Involve stakeholders at all stages in the
process Assess current capacity and use what is
already available Avoid duplication of data collection and
reporting
Do not: Collect information that will not be used Underestimate the importance of stakeholder
buy-in and ownership at every juncture
M&E for Program Improvement- An Iterative Process
Data collection methods and toolsHuman resources
and training
Indicators
Source of data
Implementation
Data use
May 3, 2023
WHAT IS PLANNING?
Planning is a process concerned with: Developing and achieving a vision Understanding some of the wider drivers of change A process that is flexible in light of these changes A process that is achievable in light of available
resources A process that is continuously reviewed as part of a
learning process A vision that is, importantly, implementable
May 3, 2023
PRINCIPLES FOR A SUCCESSFUL PLANNING PROCESS
Include at least the following four principles:
The problem identified (ensure that the right question is the focus of the planning)
The people involved (ensure that the right people are in the team)
The process used (ensure that this is a learning process that helps key players debate things that matter)
The resources available (it’s essential to match the size of the problem with the resources needed to address it)
May 3, 2023
WHAT IS A PLAN? It is a course of action one intends to follow in order to solve a
problem It is a purposeful, detailed outline of actions to allocate
resources and coordinate efforts, to achieve desired change (e.g. solve a problem, achieve an objective).
It is also regarded as a document that describes a system that links strategic information obtained from various data collection systems to decisions that will improve programs.
It ensures that objectives are set to deal with problems and to make best use of available resources
PRACTICALLY WHAT DOES THIS MEAN?
This implies the use of conceptual framework to develop a clear understanding of all the factors likely to positively or negatively influence the problem under investigation
Able to derive few models/scenarios from the conceptual framework at hand
CONCEPTUAL FRAMEWORK FOR HIV/AIDS CHALLENGES
Individual
Characteristics
Community
Characteristics
Programmatic
Characteristics
Not Infected
Infected
Not Infected
Sick
Infected
Death
THIS CONCEPTUAL FRAMEWORK SHOWS
SOME FACTORS THAT ARE LIKELY TO INFLUENCE THE TRANSMISSION OF HIV
THE OPTIMUM LEVEL OF APPLICATION OF SOME PREVENTION STRATEGIES:1. Primary Prevention: ABC; VCT; School-based life skill
education; PMTCT (infant); BCC.
2. Secondary Prevention: PMTCT (mother); ART
3. Tertiary Prevention: Palliative Treatment/Community- based support and care
PURPOSES OF STRATEGIC FRAMEWORK
To provide a clarified focus on the causal relationships that connect incremental achievement of results to the comprehensive program impact
To clarify project/program mechanics and factors’ relationships that suggest ways and means of objectively measuring the achievement of desired ends
Strategic/Results FrameworkYouth HIV Prevention Program
SO1 : Increased use of VCT services among young people
IR1 : Availability of youth friendly VCT services
IR2 : Demand for VCT services
IR1.1 : Information and services increased for youth
IR1.2 : Practioners’ skills and knowledge increased for counseling and testing youth
IR1.3 : Sustainable effective management
IR2.1 : Customer knowledge of HIV/AIDS transmission improved
WHO/WHAT DETERMINES THE AREAS OF THE PROGRAM TO BE MEASURED?
The program goals/results indicate the areas in which progress can be expected, hence, the areas in which it can be measured.
How to measure progress in these areas? By selecting appropriate and relevant indicators (country’s goals, objectives, activities, local epidemiology, nature of problem and risk behaviors)
May 3, 2023
WHAT IS THE PLAN MAIN PURPOSE?
To help monitoring and evaluating activities through:
M&E BUY IN Efficiency – saves time Effectiveness – doing what we want to do Data management Data Quality & Data Use
May 3, 2023
Some References
Adamchak S et al. (2000). A Guide to Monitoring and Evaluating Adolescent Reproductive Health Programs. Focus on Young Adults, Tool Series 5. Washington D.C.: Focus on Young Adults.
Bertrand J et al. (1996). Evaluating Family Planning Programs. The Evaluation Project.
Rommelmann V, P Setel, Y Hemed, H Mponezya, G Angeles, T Boerma (2003). Costs and Results of Information Systems for Poverty Monitoring, Health Sector Reform, and Local Government Reform In Tanzania.
Lusthaus, et. al. (1999) Enhancing Organizational Performance: A toolbox for Self-Assessment.
Loos G.P. (1996). Field guide for international health project planners & managers. Janus Publishing Company, London England.
Hadridge P (2004). Strategic approaches to planning health care. Oxford Handbook of Public Health Practice. Oxford University Press.
May 3, 2023
THANK YOU