View
448
Download
2
Tags:
Embed Size (px)
Citation preview
VALUE CHAIN MONITORING AND EVALUATION GUIDE
MODULE 8
Integrating Other Design Features into the M&E
System
April 11, 2023
2
GETTING STARTED
OBJECTIVE OF MODULE 8This module focuses on developing select support functions for the system including putting in place data management plans and checks to ensure data quality.
April 11, 2023
3
GETTING STARTED
A good M&E system includes a number of design features:
•Develop data collection instruments.
•Pilot test data collection instruments.
•Arrange data collection logistics.
•Establish data quality oversight procedures.
•Develop electronic database.
•Establish data management procedures.
•Establish data analysis procedures.
•Establish reporting procedures.
•Assign clearly defined roles to everyone involved.
April 11, 2023
4
All M&E system design
pieces developed to this
point.
Senior M&E staff and/or
country program M&E staff
working on related
initiatives
MATERIALS / INPUTS RECOMMENDED
April 11, 2023
5
STEP-BY-STEP GUIDE
1
•Develop data collection instruments
2
•Pilot test data collection instruments
3
•Arrange data collection logistics
4
•Establish data quality procedures
5
•Develop Project Database and Data Entry Procedures
6
•Establish data management procedures
7
•Establish data analysis procedures
8
•Establish information dissemination procedures
9
•Assign clearly defined roles
April 11, 2023
6
STEP-BY-STEP GUIDE: STEP 1Data Collection Instruments:
•Must be developed for every level - observation & measurement
•Must be developed in a participatory way, in collaboration with future users:
• Front-line staff members using them routinely• Project evaluators designing baseline, midterm and endline
•Important that these tools are developed in local languages and tested for accuracy
•Must be accessible to project participants, staff and partners – whoever will be using them
April 11, 2023
7
STEP-BY-STEP GUIDE
1
•Develop data collection instruments
2
•Pilot test data collection instruments
3
•Arrange data collection logistics
4
•Establish data quality procedures
5
•Develop Project Database and Data Entry Procedures
6
•Establish data management procedures
7
•Establish data analysis procedures
8
•Establish information dissemination procedures
9
•Assign clearly defined roles
April 11, 2023
8
STEP-BY-STEP GUIDE: STEP 2
Pilot Test•Should be done prior to full rollout•Identifies inappropriate, poorly worded, or mistranslated questions•Validates the existing coding scheme•Identifies omitted questions•Trains field researchers in tool implementation• Varies in importance
April 11, 2023
9
STEP-BY-STEP GUIDE
1
•Develop data collection instruments
2
•Pilot test data collection instruments
3
•Arrange data collection logistics
4
•Establish data quality procedures
5
•Develop Project Database and Data Entry Procedures
6
•Establish data management procedures
7
•Establish data analysis procedures
8
•Establish information dissemination procedures
9
•Assign clearly defined roles
April 11, 2023
10
Ensure all necessary logistical arrangements, to include:
•Selecting research team members•Transportation to and from the research site•Meals and lodging•Security arrangements•Communication with respondents or potential respondents•Means of communication between research teams and with the head office•Protocols for addressing problems or questions as they arise•Safeguarding of blank and completed information gathering instruments•Storage and transfer of research findings
STEP-BY-STEP GUIDE: STEP 3
April 11, 2023
11
STEP-BY-STEP GUIDE
1 •Develop data collection instruments
2 •Pilot test data collection instruments
3 •Arrange data collection logistics
4 •Establish data quality procedures
5 •Develop Project Database and Data Entry Procedures
6 •Establish data management procedures
7 •Establish data analysis procedures
8 •Establish information dissemination procedures
9 •Assign clearly defined roles
April 11, 2023
12
Establish data quality procedures to prevent:
STEP-BY-STEP GUIDE: STEP 4
Data collection errors
Data tampering
Security lapses
April 11, 2023
13
Quality control procedures that mitigate threats:
STEP-BY-STEP GUIDE: STEP 4
Quality M&E System Training
Random Quality Checks
Routine Refreshers
Collaboration with data collectors
In-field Supervision
Reviews of Data Collection Forms
April 11, 2023
14
STEP-BY-STEP GUIDE
1 •Develop data collection instruments
2 •Pilot test data collection instruments
3 •Arrange data collection logistics
4 •Establish data quality procedures
5 •Develop Project Database and Data Entry Procedures
6 •Establish data management procedures
7 •Establish data analysis procedures
8 •Establish information dissemination procedures
9 •Assign clearly defined roles
April 11, 2023
15
Electronic databases for survey/questionnaire results:
•Common format is to list questions across
columns, and respondents down rows
•Data management programs most often used:
-Microsoft Excel -SPSS
-Microsoft Access
•Should capture info in the order it was asked
•Should include built in data controls
•Should include a ‘codebook’
STEP-BY-STEP GUIDE: STEP 5
April 11, 2023
16
Consolidation of Qualitative Data:•Should be compiled and managed in a central space•Carefully consider data collection tools
STEP-BY-STEP GUIDE: STEP 5
Challenges Lessons learned
Anecdotes Success stories
Easy Access for
the Reader
April 11, 2023
17
STEP-BY-STEP GUIDE
1 •Develop data collection instruments
2 •Pilot test data collection instruments
3 •Arrange data collection logistics
4 •Establish data quality procedures
5 •Develop Project Database and Data Entry Procedures
6 •Establish data management procedures
7 •Establish data analysis procedures
8 •Establish information dissemination procedures
9 •Assign clearly defined roles
April 11, 2023
18
Critical components of data management:
STEP-BY-STEP GUIDE: STEP 6
Data Entry
Data Access
Data Backup
Data Storage
Data Revision
April 11, 2023
19
STEP-BY-STEP GUIDE
1 •Develop data collection instruments
2 •Pilot test data collection instruments
3 •Arrange data collection logistics
4 •Establish data quality procedures
5 •Develop Project Database and Data Entry Procedures
6 •Establish data management procedures
7 •Establish data analysis procedures
8 •Establish information dissemination procedures
9 •Assign clearly defined roles
April 11, 2023
20
Raw Data
STEP-BY-STEP GUIDE: STEP 7
Useful Informati
on
FINAL M&E EVALUATION
AWESOME PROJECT
KENYA
April 11, 2023
21
STEP-BY-STEP GUIDE
1 •Develop data collection instruments
2 •Pilot test data collection instruments
3 •Arrange data collection logistics
4 •Establish data quality procedures
5 •Develop Project Database and Data Entry Procedures
6 •Establish data management procedures
7 •Establish data analysis procedures
8 •Establish information dissemination procedures
9 •Assign clearly defined roles
April 11, 2023
22
Disseminating Data to Internal/External Users:
•Procedures include timing, content and format of information
•Externally, determined by donor requirements
•Internally, co-developed with staff who will use theinformation
STEP-BY-STEP GUIDE: STEP 8
April 11, 2023
23
STEP-BY-STEP GUIDE
1
•Develop data collection instruments
2
•Pilot test data collection instruments
3
•Arrange data collection logistics
4
•Establish data quality procedures
5
•Develop Project Database and Data Entry Procedures
6
•Establish data management procedures
7
•Establish data analysis procedures
8
•Establish information dissemination procedures
9
•Assign clearly defined roles
April 11, 2023
24
Effective operation requires that all involved understand roles and responsibilities clearly, which should be included in:
•Staff job descriptions
•MOUs with implementing partners
•Contracts with external researcher
STEP-BY-STEP GUIDE: STEP 9
April 11, 2023
25
COMMON PITFALLS•Failure to consider all the features necessary for
a functioning and effective M&E system.•Projects do not pilot test data collection tools and instruments.•Projects do not establish data quality procedures or conduct periodic assessments of data gathering activities and data quality.•Projects do not establish data management procedures or plan for staff turnover•Projects do not plan to and/or invest in conducting sufficient staff training
• Including follow up mentoring•Too much focus on donor needs, not enough on other M&E system clients.
April 11, 2023
26
QUESTIONS?
COMMENTS?
April 11, 2023
27
Multiple ways to continue the discussion and continue learning:• Initiate a monthly session on the M&E guide
and case studies from across CARE. Contact [email protected]
• Join the Market Engagement Community of Practice on LinkedIn.
• Join a task force to review and refine the universal indicators. Contact [email protected]
Want to Learn More?