ANNEX BOX 1 & BOX 2 · Annex 1: Definitions of Key Terms Nutrition Specific Interventions: a) Support for exclusive breastfeeding up to 6 months of age and continued breastfeeding,

Page 1

ANNEX BOX 1 & BOX 2

EVALUATION REPORT

UNITED NATIONS MATERNAL AND CHILD STUNTING REDUCTION

PROGRAMME IN THREE TARGET DISTRICTS IN SINDH,

PAKISTAN

Page 2

Annex Box 1:

Annex 1: Definitions of Key Terms

Annex 2: Global Impetus for Sector-wide Programming

Annex 2A: ToR of Evaluation

Annex 3: Performance Monitoring Plan (PMP)

Annex 4: Integrated Action Plan (IAP)

Annex 5: Evaluation Scope

Annex 6: Evaluation Matrix

Annex 7: Overview of Qualitative Methods

Annex 8: FGD and Observation Tool Kit

Annex 8A: IDI Topic Guide

Annex 9: List of Persons Interviewed and Area Visited

Annex 10A: Ethical Review Approval

Annex 10: Analytical Approaches and Quality Assurance

Annex 11: Integration Elements and Scaling Framework

Annex 12: Examples of Iterative Management

Annex 13: IEC Material and Fragmented Messages

Annex 14: Key Documents Reviewed

Annex 15: Team Composition and Responsibilities

Annex 16: Bibliography

Annex Box 1:

Annex A: Case Study Guide

Annex B: Consent Form Case Study

Annex C: Guide of the Spot Checks & Observation

Annex D: Consent Form Spot Checks

Annex E: Data Protection Guidelines

Annex F: Assent Form School Children

Annex G: Consent Form School Teachers

Annex H: Consent Form FGDs

Annex I: Consent Form IDIs

Page 3

Annex 1: Definitions of Key Terms

Nutrition Specific Interventions: a) Support for exclusive breastfeeding up to 6 months of age and

continued breastfeeding, together with appropriate and nutritious food, up to 2 years of age b) Fortification

of foods c) Micronutrient supplementation d) Treatment of severe malnutrition.

Nutrition-Sensitive Approaches: a) Agriculture: Making nutritious food more accessible to everyone, and

supporting small farms as a source of income for women and families, b) Clean Water and Sanitation:

Improving access to reduce infection and disease, c) Education and Employment: Making sure children

have the energy that they need to learn and earn sufficient income as adults, d) Healthcare: Improving

access to services to ensure that women and children stay healthy, e) Support for Resilience: Establishing

a stronger, healthier population and sustained prosperity to better endure emergencies and conflicts, f)

Women‘s Empowerment: At the core of all efforts, women are empowered to be leaders in Nutrition-

Sensitive Approaches.

Stunting or Chronic Malnutrition: Stunting, or low height for age, is caused by long-term insufficient

nutrient intake and frequent infections which generally occurs before age two. Wasting, or low weight for

height is usually the result of acute significant food shortage (UNICEF (2012). WHO defines stunting as

―the impaired growth and development that children experience from poor nutrition, repeated infection, and

inadequate psychosocial stimulation. Children are defined as stunted if their height-for-age is more than

two standard deviations below the WHO Child Growth Standards median.‖

Acute Malnutrition – Also known as ‗wasting‘, acute malnutrition is characterized by a rapid deterioration

in nutritional status over a short period of time. In children, it can be measured using the weight-for-height

nutritional index or mid-upper arm circumference. There are different levels of severity of acute malnutrition:

moderate acute malnutrition (MAM) and severe acute malnutrition (SAM).

Global Acute Malnutrition (GAM) – The total number of children aged between 6 and 59 months in a

given population who have moderate acute malnutrition, plus those who have severe acute malnutrition.

(The word ‗global‘ has no geographic meaning.) When GAM is equal to or greater than 15 per cent of the

population, then the nutrition situation is defined as ‗critical‘ by the World Health Organization (WHO). In

emergency situations, the nutritional status of children between 6 and 59 months old is also used as a

proxy to assess the health of the whole population.

Malnutrition – A broad term commonly used as an alternative to ‗under-nutrition‘, but which technically

also refers to over-nutrition. People are malnourished if their diet does not provide adequate nutrients for

growth and maintenance or if they are unable to fully utilize the food they eat due to illness (under-

nutrition). They are also malnourished if they consume too many calories (over-nutrition)

Moderate Acute Malnutrition – Defined as weight-for-height between minus two and minus three

standard deviations from the median weight-for-height for the standard reference population.

Severe Acute Malnutrition – Acute deficiency of protein, energy, minerals and vitamins leading to loss of

body fats and muscle tissues. Acute malnutrition presents with wasting (low weight-for-height) and/or the

presence of oedema (i.e., retention of water in body tissues). Defined for children aged 6–60 months, as a

weight-for-height below – 3 standard deviations from the median weight-for-height for the standard

reference population or a mid-upper arm circumference of less than 115 mm or the presence of nutritional

oedema or marasmic-kwashiorkor

Mid-Upper-Arm-Circumference (MUAC) – The circumference of the mid-upper arm is measured on a

straight left arm (in right-handed people) midway between the tip of the shoulder (acromion) and the tip of

Page 4

the elbow (olecranon). It measures acute malnutrition or wasting in children aged 6–59 months. The mid-

upper-arm circumference (MUAC) tape is a plastic strip, marked with measurements in millimetres. MUAC

< 115mm indicates that the child is severely malnourished; MUAC < 125mm indicates that the child is

moderately malnourished

Community-based Management of Acute Malnutrition (CMAM) – This approach aims to maximize

coverage and access of the population to treatment of severe acute malnutrition by providing timely

detection and treatment of acute malnutrition through 5 community outreach and outpatient services, with

inpatient care reserved for more critical cases. CMAM includes: inpatient care for children with SAM with

medical complications and infants under 6 months of age with visible signs of SAM; outpatient care for

children with SAM without medical complications; and community outreach for early case detection and

treatment.

Height-for-age Nutritional Index – A measure of stunting or chronic malnutrition.

Blended foods - Mixtures of milled cereals and other ingredients such as pulses, dried skimmed milk and

possibly sugar and oil. Blended foods are produced by dry-blending of milled ingredients; toasting or

roasting and milling of ingredients; extrusion cooking, which results in a ―pre-cooked‖ food. The final

product is milled into a fine powder and fortified with a mineral and vitamin premix and is subject to specific

requirements. Examples of blended foods include wheat-soy blend and corn-soy blend.

Nutritional Index – Different nutritional indices measure different aspects of growth failure (wasting,

stunting and underweight) and thus have different uses. The main nutritional indices for children are weight-

for-height, MUAC-for-age, sex and height, height-for-age, weight-for-age, all compared to values from a

reference population. In emergency situations, weight-for-height (wasting) is commonly used for nutritional

assessments.

Outpatient Therapeutic Care (OTP) – Outpatient care for treatment and management of severe acute

malnutrition that connects treatment in the health facility but does not require admission to the health

facility. Treatment is carried out while 10 patients remain at home, and involves intermittent health facility

visits and/or community outreach

Ready-to-use Supplementary Foods – Specialized ready-to-eat, portable, shelf-stable products, available

as pastes, spreads or biscuits, that meet the supplementary nutrient needs of those who are not severely

malnourished. They are increasingly used for the management of moderate acute malnutrition.

Ready-to-use Therapeutic Foods – Specialized ready-to-eat, portable, shelf-stable products, available as

pastes, spreads or biscuits that are used in a prescribed manner to treat children with severe acute

malnutrition.

Stabilization Centre – Inpatient care facility established for the treatment of severe acute malnutrition

with complications.

Supplementary Feeding Program (SFP) – There are two types of supplementary feeding programs.

Blanket supplementary feeding programs target a food supplement to all members of a specified at-risk

group, regardless of whether they have moderate acute malnutrition or not. Targeted supplementary

feeding programs provide nutritional support to individuals with moderate acute malnutrition. To be

effective, targeted supplementary feeding programs should always be implemented when there is sufficient

food supply or an adequate general ration for the general population, while blanket supplementary feeding

programs are often implemented when general food distribution for the household has yet to be established

or is inadequate for the level of food security in the population. The supplementary ration is meant to be

additional to, and not a substitute for, the general ration.

Page 5

Under-nutrition – An insufficient intake and/or inadequate absorption of energy, protein or micronutrients

that in turn leads to nutritional deficiency.

Improved drinking-water source - is one that by its construction is adequately protected from outside

contamination, in particular, from faecal matter

Improved sanitation facility - is one that hygienically separates human excreta from human contact.

Open defecation refers to when people defecate in fields, forests, bushes, open bodies of water, beaches,

and other open spaces

Definition of ‗additional‘ for cost-effectiveness and cost-benefit analysis: In this report any cost is

defined as ‗additional‘ if it is a direct result of the project intervention; in other words, the cost would not

have been incurred in if the project had not been implemented. The term ‗additional‘ is used instead of

‗marginal‘ or ‗incremental‘ to make the report more readable to lay audiences less familiar with health

economics terminology. For example, the cost of providing RUTF will be ‗additional‘ because without the

project the service would not have been provided and the expense not incurred for that purpose. Applying

the same rule, the salary cost of existing government officials/health workers who were involved in service

provision will not be treated as ‗additional‘, as he/she would have received the salary in any case even if

the project had not been implemented.

Attribution for cost-effectiveness and cost-benefit analysis: The results in this analysis assume 100%

cost attribution towards the supply chain. All activities in the supply chain beginning from advocacy,

meetings, marketing towards the end user service delivery were funded by the project. We considered any

change between the baseline and the end line 100% attributable to the project. However, we have used

sensitivity analysis to see the impact on cost-effectiveness if 100% of the results are not attributable to the

project.

Cost-effectiveness conclusions: In order to comment on the cost-effectiveness of the project, we used

the Commission on Macroeconomics & Health and the World Health Organization (WHO) recommendation.

According to this recommendation an intervention is considered to be very cost-effective and cost-effective

if the cost per Disability Adjusted Life Year (DALY) is less than per capita Gross Domestic Product (GDP)

or between 1- and 3-times per capita GDP, respectively. If the cost per DALY is more than three times the

GDP per capita, then the intervention is regarded as not cost-effective. We were not able to find any similar

studies with which the cost per DALY averted could be compared.

Valuation of benefits for cost-effectiveness and cost-benefit analysis: Only incremental benefits were

valued against the counterfactual. For the purposes of this analysis, the counterfactual is that if the

intervention is not implemented there would be no additional services provided. There are several gains

that will arise because of implementing the project. Only those gains which can be valued with reliable

estimation and are within the scope of the project have been accounted for. The following gains have been

valued as part of the benefits valuation:

Reduction in disability adjusted life years by improved nutrition and WASH services.

There are other gains which have not been accounted for in this study, as it would be too complicated and

beyond the scope of the project to do so. A few of them are listed below:

Environmental effects

Increased productivity

Fewer orphans due to decreased mortality and as a result, lower social costs to the society

Increased per capita household expenditure on different activities due to reduced expenditure on

health, leading to more savings.

Page 6

Annex 2: Global Impetus for Sector-wide

Programming

Large scale evidence indicates a close relationship between stunting and poor WASH. Many international organizations including SUN movement and UN agencies have implemented nutrition sensitive and nutrition specific interventions in many settings. The empirical evidence from these programmes, calls for sector-wide integrated programming and breaking down silos to combat the menace of malnutrition. The global nutrition community has repeatedly called for greater attention to and investments in WASH as a means to improve nutrition outcomes. Several developing countries, donors and organizations made public commitments to improve nutrition as part of the 2014 High Level Meeting of Sanitation and Water for All, a global partnership of over 90 developing country governments, donors, development banks and civil society organizations that tracks commitments made towards universal access to safe water and adequate sanitation services. The international impetus for sector-wide programming is discussed in the sub-sections below.

SDGs and SUN Movement: SUN is based on 1,000 Days partnership, which was launched by the USA and Ireland in 2010, focuses on the critical window of opportunity between a woman‘s pregnancy and her child‘s second birthday. This is the critical period to improve maternal and child nutrition, using the life cycle approach, which can have its greatest impact on reducing death and disease, increasing intellectual and physical work capacity and lowering the risk of under-nutrition, obesity and non-communicable diseases. SUN builds high-level support at the country level to foster collaboration and coordination across sectors and stakeholders to position nutrition in all development efforts. The movement promotes scaling up both specific nutrition interventions as well as nutrition-sensitive approaches, including clean drinking-water, improved sanitation facilities and hygiene. The global practitioners and experts consider nutrition essential for the success of all SDGs. To this end, SUN movement has developed an SDG advocacy tool kit that supplements the UN Decade of Action on Nutrition to support the adaptation of sector-wide and integrated approaches. The figure 1

1 demonstrates the links between nutrition and SDGs

WASH and Nutrition in UNICEF Global Strategy: UNICEF global approach supports and advocates for evidence-based nutrition-specific interventions and nutrition-sensitive approaches. The global approach acknowledges the need for multi-sectoral programming and UNICEF is working towards integration of nutrition actions with other sectors, including health, early childhood development, social protection, WASH and education. Figure 2 shows the UNICEF strategic priorities. Similarly, UNICEF WASH strategy 2016—2030 also highlight the need for integrated Nutrition and WASH programming as a step forward to address malnutrition. The UNICEF strategy has clearly spelled out WASH contributions to key outcomes on maternal, child and adolescents, across the life course and continuum of care.

FIGURE 2: WASH AND NUTRITION IN THE UNICEF GLOBAL STRATEGY2

1 Source: Adapted from SUN-SDG Advocacy Tool Kit 2 Source: Adapted from: UNICEF‘s Approach to Scaling-up Nutrition for Mothers and their Children, discussion paper, Programme Division,

UNICEF, New York City, June 2015 and UNICEF Nutrition-WASH Tool Kit – Guide for Particle Joint Action

FIGURE 1: THE LINKS BETWEEN NUTRITION AND

SDGS

Page 7

Nutrition Specific and Sensitive Approaches in the Lancet Series: Lancet highlights that achieving 90% coverage of the set of 10 nutrition-specific interventions will reduce the prevalence of stunting by only 20%

3, highlighting the

importance of combining nutrition-specific interventions with nutrition-sensitive interventions, including WASH. On the other spectrum combining the nutrition-specific intervention with 99% coverage of WASH will reduce diarrhoea incidence by 30%

4. This will break the vicious circle that exist between diarrhoea and under-nutrition and help the

children to absorb better the nutritious diet. Better understanding environmental enteropathy (subclinical inflammation) helps explain why purely nutrition-specific interventions have failed to reduce under nutrition in many contexts. The lancet notes that the return on investment in nutrition has high cost-benefit ratio. For every dollar spent on nutrition under ―First 1,000 days,‖ the government can save up to $166. Figure 3 provides the framework of action to achieve optimal nutrition outcomes.

FIGURE 3: FRAMEWORK OF ACTION TO ACHIEVE OPTIMAL FOETAL AND CHILD NUTRITION5

Annex 2A: ToR of Evaluation

This is annexed separately

3 Bhutta ZA, Das JK, Rizvi A, Gaffey MF, Walker N, Horton S et al. (2013). Evidence-based interventions for improvement of maternal and child nutrition: what can be done and at what cost? Lancet. 382(9890):452–77. 4 Source: Lancet 2008, WASHPlus 2015, Checkley at al 2008, Horton and Hoddinott, 2014; Ngure al 2014 5 Source: Adopted from Black, Robert E.et al., Maternal and Child Under-nutrition, Lancet Series Paper 1, 2013 and UNICEF Nutrition-WASH Tool

Kit – Guide for Particle Joint Action

Page 8

Annex 3: Performance Monitoring Plan (PMP)

Y1 Y2 Y1 Y2

Will be captured through post

KAP

will be captured through post

KAP

will be captured through post

KAP

2. Exclusive breast feeding rates of infants

under 6 months of age

30.90% 1% 1% XXWill be captured through post

KAPKAP & MICS Post project

3. % of children(ages 6-24 months) receiving

MM

107% 20% 40% 0% 124%NIS/NWG consolidated

databasemonthly

- A total of 249,779 out of 200,666 (124 percent) children aged 6-23 months (boys: 133,133 and girls: 116,646) were provided with micronutrients powder/sachet for

mimum of three month.

4. Proportion of children under 2/5 years of age

who had diarrhea in the two weeks preceding the

post KAP and WASH evaluation study 60% 20% 40% 0%

Post KAP/End

Programme Evaluation End of project The information will be gathered post project implementation

8. Number of communities certified as open

defecation free as a result of USG assistance

(3.1.6.8-5)

114,285 HHs/

800,000 people

22,857 HHs/

160,000

people

91,428 HHs/

640,000 people0%

96% (2,196 villages certified

as ODF. Post ODF

certification 2,150 villages)

reaching 1,192,425 people

(298,106 women, 286,182

men and 608,137 children)

MICS Sindh 2014 Quarterly

As a result of extensive social mobilization and triggering to the rural communities, all the 2,288 targeted villages were declared as ODF by the Village WASH

Committees and CRPs. However, as per ODF criteria, 2,268 of these villages were verified by the partners and UNICEF’s third-party monitors. As per ODF

certification guidelines, 2,196 villages (778 in Khairpur, 665 in Ghotki and 753 in Nausheroferoz) were certified as ODF by the committees notified by district

government. To minimize the slippages, post ODF follow ups were also conducted and during the first and second round of verification during which a total 2,150

villages were certified as ODF. Therefore, a total of 1,192,425 people (298,106 women, 286,182 men and 608,137 children) are living in ODF certified communities.

Outcome Indicator 1.1:

Number of projects and interventions with

resource allocations by Government of Sind as a

result of Nutrition sensitive WASH policies and

specific action plans

Numerator: N/A

Denominator: N/A

10 10 0%

UNICEF Progress

Report WASH Sector

Development Plan

Quarterly

reporting

1. More than 20 projects/ interventions have been identified in the Draft Sector Development Plan where resource allocations will be made by the GoS. This requires

continuous follow up and advocacy with Government of Sindh and UNICEF will prioritize the key interventions under rolling work plan (2018-19) under the new

country programme cycle (2018-2022).

Outcome indicator 2.1:

Number of households using an improved

drinking water source (3.1.8.1-1) (source DO

5.2 c)

N= No of households have access to water

D= Total number of households in targeted

areas

13,714 HHs 54,858 HHs 0% 109% MICS Sindh 2014 Quarterly

As a result of community water supply interventions, 75,189 households / total 526,324 people (126,318 men, 136,844 women and 263,162 children) gained access

to safe drinking water through the provision of 2,311 functional handpumps. Out of these, 1,168 handpumps were rehabilitated and 1,143 small water systems (including

handpumps) were newly installed. Additionally, seven water supply schemes were also rehabilitated and 116,800 people (28,032 men, 30,368 women and 58,400

children) gained access to safe drinking water in their local communities. All the water supply schemes are now operational and maintained by PHE&RDD, Government

of Sindh.

Water quality testing of identified water sources was completed by WASH implementing partners in all targeted talukas. UNICEF, through implementing partners,

installed 1,143 new water supply sources and rehabilitated 1,168 existing water sources by adopting pre-testing protocols to determine whether the water source is safe

for drinking or not. The existing or old hand pumps were tested before the rehabilitation to determine whether the hand pump is free of any chemical contamination or

not and only those pumps chemically fit for drinking purposes were rehabilitated. It was important that the selected hand pump was free of any chemical contamination

and fell under WHO recommended drinking water standards. All water quality testing reports were submitted to district government authorities in Ghotki, Khairpur and

Nausheroferoz.


Number of communities certified as “open

defecation free” as a result of USG assistance

(3.1.6.8-5)

N=No. of communities living in ODF

environment

D= Total number of communities in targeted

areas 22,857 HHs/

160,000

people

91,428 HHs/

640,000 people0%

96% (2,196 villages certified

as ODF. Post ODF

certification 2,150 villages)

reaching 1,192,425 people

(298,106 women, 286,182

men and 608,137 children)

MICS Sindh 2014 Quarterly

As a result of extensive social mobilization and triggering to the rural communities, all the 2,288 targeted villages were declared as ODF by the Village WASH

Committees and CRPs. However, as per ODF criteria, 2,268 of these villages were verified by the partners and UNICEF’s third-party monitors. As per ODF certification

guidelines, 2,196 villages (778 in Khairpur, 665 in Ghotki and 753 in Nausheroferoz) were certified as ODF by the committees notified by district government. To

minimize the slippages, post ODF follow ups were also conducted and during the first and second round of verification during which a total 2,150 villages were certified

as ODF. Therefore, a total of 1,192,425 people (298,106 women, 286,182 men and 608,137 children) are living in ODF certified communities.

Outcome indicator 2.3: Number of children

under five reached by USG-supported nutrition

program (3.1.9-15)

N= No of Children 6-23 months received

micronutrients.

D= No of Children 6-23 months in target

children

20% 40% 0% 124% NIS Quarterly As mentioned above in indicator 3

Outcome Indicator 3.1: Percentage of

households with soap and water at a hand

washing station commonly used by family

members (3.1.6.8-2)

N: No of women and children practicing

handwashing with soap

D: No of people (including women and children)

in target districts30% 30% 0% MICS Sindh 2014 Quarterly

To be determined in end evaluation and MICS 2018-19


Number of school children reached with specific

hygiene related messages

N: Number of school children reached with

specific hygiene related messages

D: Total number of children in targeted areas12,000 12,000 0%

273%

School Children: 75,529

Schools: 605

Project Progress Reports QuarterlyAs of reporting period: Total 605 schools have been provided with WASH facilities benefiting approximately 76,529 school children ((45,917 boys and 30,612 girls)

reached with specific hygiene related messages in all three targeted districts. Target over achieved as a result of actual enrollment in targeted schools.

>100%


Number of Health care providers/IPs receiving

nutrition related professional trainings through

USG-supported programs

N: Number of Health care providers/IPs

received trainings

D: Number of Health care provider/Individuals

from IPs working in the catchment health

facilities

40% 40% 68%

3618 LHW trained on

Stunting prevention

1488 CRPs trained on

SBCC

Training reports Quarterly

- 322/389 project staff trained on IYCF, MM supplementation, community mobilization, and OTP standard protocols so far.

- 3618 Lady Health Workers (LHWs) trained at field level on Stunting prevention and to sustain key health and hygiene behaviours in targeted districts

- 350 Master Trainers (including Doctors & Lady Health Supervisors ) trained at district level by LHW program to conduct cascade training for at facility level

-140 Pediatricians, Gynecologists, Medical Officers and Nurses) from Department of Health and 250 Community Midwives from MCH program trained on lactation

management in targeted three districts

- 1488 Community Resource Persons trained on WASH and Nutrition Integrated Social Behaviour Communication Campaign (SBCC)

- 24 participants from Department of Health including District Health Officers received master training (ToT) on essential nutrition modules in target three districts

- 35 staff from DoH and NGOs received training on online NMIS

0 2 1

0%52% sanitation coverage in schools

47% water coverage in schoolsMonthly

As of reporting period: Total 605 schools have been provided with WASH facilities benefiting approximately 76,529 school children having access to WASH facilities

including gender disaggregated toilets, group handwashing facilities and improved drinking water in their learning environment. results are over achieved as per actual

enrollment in the schools in three targeted districts

Quarterly

reporting

480,000 people

(including

women and

children)

96,000 384,000

1. Drinking Water Policy Sindh approved by Chief Minister Sindh and dessiminated to sector stakeholders.

2. Sanitation Policy approved by Chief Minister Sindh and dessiminated to sector stakeholders.

3. Nutrition sensitive WASH Sector Development Plan approved by Chief Minister Sindh and dessiminated to sector stakeholders.

4. Sindh WASH BCC Strategy approved by Chief Minister Sindh and dessiminated to sector stakeholders.

(target 100% achieved) .

28,000 14000 14000Sindh Education Sector

Plan 2014-18

273%

School Children: 76,529

Schools: 605

1 UNICEF progress report

MICS Sindh, 2014

RemarksTargets Achieved

Goal:

Enhanced Nutrition and WASH Practices contribute to a reduction in stunting by 2017 in three target districts of Sindh

1. Percentage of women who initiated early

breastfeeding within one hour of delivery

20.7% (Sindh) 22.70% 1% 1%

Numerator: # of women with a live birth in the

last 2 years who put their last newborn to the

breast within one hour of birth

Denominator: # of women gave live birth in last

2 years

Indicators Definition: Numerator and Denominator Baseline End of 2 year

Project

TargetsData Source Frequency

The information will be gathered follow up KAP survey

KAP results:

-Early initiation of Breast feeding : overall three districts (baseline:50.2%, endline : 51.2%) Ghotki (baseline: 49.5%, endline:50.8%), Khairpur (baseline: 48.4%,

endline:47.2%) and NausheroFeroz (baseline: 52.6%, endline:55.6% )

-Exclusive Breast Feeding : overall three districts (47.6%), Ghotki (baseline:45%, endline:TBD ), Khairpur (baseline: 54.5%, endline: TBD) and NausheroFeroz

(baseline: 41.9%, endline: TBD)

Note: The data currently reported is provisional as the report is undergoing a final quality review and verification of analysis. The post KAP results will be shared after

this process has been completed.

XX KAP & MICS Post project

28.9% (Sindh)

N: # of infants under 6 months of age who are

exclusively breast fed

D: # of infant born over a period of last 2 years


Number of mothers/caregivers of children 0-23

months who received direct maternal infant and

young child nutrition counselling at least once in

the past quarter through USG supported

programs

N:Number of mothers/care givers received

counselling at least once in the past quarter

D: Number of mothers/caregiver 0-23 present in

the targeted area

Numerator: N/A

Denominator: N/A

114,285 HHs/

800,000 people

6. # of children with access to WASH facilities

(toilets, group hand washing facilities &

improved drinking water) in schools

N: Number of children with access to WASH

facilities

D: Total number of children in targeted areas

N: Number of people (including women and

children) gaining access to improved drinking

water source

D: Total number of people (including women

and children) in targeted areas

154%

TBD 60%

0


Number of nutrition specific health programs

reflected in the draft ADP scheme by P&DD,

Sindh. 0 1

0 60%

60%60 % of the targeted

population

0 24,000

20


Percentage of audience who recall hearing or

seeing a specific USG-supported nutrition or

WASH related message (DO 5.2.1 b)

N: No of mothers and caregivers aware of IYCF

practices / WASH messages

D: No of mothers and caregivers in target

districts

0%7. Number of people gaining access to an

improved drinking water source (3.1.8.1-2; PPR)

Overall Sindh: 90.5%

Ghotki: Improved sources:

Piped Water: 4.4%

Other improved sources: 95.6%

Khairpur: Improved sources

Piped Water: 5%


Nausheroferoz: Improved sources

Piped Water: 0%


N: Number of certified ODF communities as a

result of USG assistance

D: Total number of communities in targeted

areas

0

→ 123,899 (>100%) mothers/ and care givers counselled/ reached for messages on IYCF and among them, 86,968 (98.7%) mothers were assessed through simple rapid

technique to address issue related to BF and or complementary feeding

30%NIS, KAP, Knowledge

retention survey

During

evaluation

phase

To asses progress on this indicators, Knowledge retention survey has been conducted as part of follow up KAP survey. Questions related knowledge of mothers and care

takers on key message were integrated into KAP questionnaires. 57% of surveyed population recalled the specific USG-supported nutrition or WASH related messages.

73.8% of those recalled 1 to 3 key messages, 19% 4 to 5 key messages and 6.4% recalled 6 and above.

Source of Messages received on mother and child nutrition, safe water, sanitation and hygiene;

1. LHWS= 35%

2. Mother (Mother-in-Law), Grandmother & friends= 19.8%

3.Local Doctors & Nurse=11.7%

4. Misali Maa CRPs=8.4%

5. Tv/Radio = 8.4%

Most recalled messages were;

1.Do not feed Babies under the age of 6 MONTHS anything, BUT Breast milk (33%)

2. WASH hands with soap before handling food and after using the toilet (28%)

3. Keep utensils, food, children CLEAN; away from flies and keep the house clean (20%)

-Following are the key results on program inputs to increase mothers and community knowledge on stunting prevention.

A total of 261,349 out of 91,879 mothers reached for six key messages by the community outreach teams

- a total of 118 Partners staff, and 3,618 LHW and 436 Couple Health Workers were trained and engaged in IYCF awareness session at community and HFs level.

- Global breastfeeding week (1-7 August) 2017-18 celebrated in three intervention districts.

7. Total 9152 Nutrition sensitive WASH related BCC campaigns launched and approximately 1,220,219 people (including women and children) reached out in

intervention talukas of District Ghotki, Khairpur and Nausheroferoz.

8. Nearly, 1,220,219 people reached through community based (Nutrition Sensitive WASH) health and hygiene sessions.

9.Total 311,168 people reached through SBCC campaigns to further reinforce the integrated (WASH + Nutrition) behavior change communication among rural

communities particularly for pregnant and lactating women under stunting reduction programme.

0%100%

Nutrition Sensitive WASH

BCC Campaigns

526,324 people (126,318

men, 136,844 women and

263,162 children)

Outcome 1: IR 1 Improved enabling environment in provincial government to address stunting in Sindh province and in particular the 3 target districts by 2017

Accelerated Action Plan (AAP), a multisectoral pan (consisted of eight sectors) has been developed and implemented by the Govt; of Sindh with the support of World

Bank. The AA Plan has been guided by integrated multisectoral strategy .

UNICEF is supporting Taskforce Secretariat in the implementation of AA plan through capacity building on SBCC, development of M & E framework and Dashboard

and strengthening of coordination mechanism.

N= No. PLW received micronutrients.

D= No. of PLW in 3 target districts.

Quarterly

As a result of community water supply interventions, 75,189 households / total 526,324 people (126,318 men, 136,844 women and 263,162 children) gained access

to safe drinking water through the provision of 2,311 functional handpumps. Out of these, 1,168 handpumps were rehabilitated and 1,143 small water systems (including

handpumps) were newly installed. Additionally, seven water supply schemes were also rehabilitated and 116,800 people (28,032 men, 30,368 women and 58,400

children) gained access to safe drinking water in their local communities. All the water supply schemes are now operational and maintained by PHE&RDD, Government

of Sindh.

Water quality testing of identified water sources was completed by WASH implementing partners in all targeted talukas. UNICEF, through implementing partners,

installed 1,143 new water supply sources and rehabilitated 1,168 existing water sources by adopting pre-testing protocols to determine whether the water source is safe

for drinking or not. The existing or old hand pumps were tested before the rehabilitation to determine whether the hand pump is free of any chemical contamination or

not and only those pumps chemically fit for drinking purposes were rehabilitated. It was important that the selected hand pump was free of any chemical contamination

and fell under WHO recommended drinking water standards. All water quality testing reports were submitted to district government authorities in Ghotki, Khairpur and

Nausheroferoz.

0%

Overall Sindh: 90.5%

Ghotki: Improved sources:

Piped Water: 4.4%


Khairpur: Improved sources

Piped Water: 5%


Nausheroferoz: Improved sources

Piped Water: 0%


68,572 HHs

20%

A total of 557,535 out of 312,147 or 179 percent women reached for MN supplementation for prevention and treatment of anemia and other micronutrient deficiencies

- Pre-Pregnant: 107,340 out of 49,999 (215%)

- Lactating women: 174,675 out of 133,778 (131%)

- Pregnant women: 155,696 out of 133,778 (116%)

-Adolescent girls: 119,824 out of 77,630 (154%)

0%

UN Maternal and Child Nutrition Stunting Reduction Program

Performance Monitoring Plan

Outcome 2: IR 2 WASH-Nutrition service delivery strengthened in three target districts through public and CSOs partners

Outcome 3: IR 3 Improved WASH and Nutrition awareness in communities of target districts by 2017

10%

107%

0

0 1 UNICEF progress report

Quarterly

reporting on

process


Number of pregnant & Lactating women reached

with nutrition interventions through USG-

supported programs

40% NIS Quarterly

0

0

N: # of children aged 6 to 24 months received

MM

D: # of children aged 06 to 24months in a

catchment population

N: # of children under 2/5 reduced incidents of

diarrhea

D: # of children aged 2/5 of total targeted

population

Numerator: N/A

Denominator: N/A

60% 100%

5. Number of water, sanitation and hygiene

polices made “Nutrition Sensitive” and

incorporated into WASH Programming

154%0

0 80%

30% 30%Session reports,

Nutrition data base

0%

30%

Page 9

Annex 4: Integrated Action Plan (IAP)

No Output/ Activity description Resposnsibility Reporting Frequency Remarks

I II III IV

1 Joint WASH/ Nutrition Coordination Mechanism Strengthened

1.1 Hiring of district coordinators and joint induction session Plan International 2nd February, 2017

Two district coordinators are hired and 3rd recruitment is under process -

Plan international will ensure to share the ToRs and contact details with all

implementing partners working under stunting reduction programme. In

addition, district coordinators will be held accountable and responsible to

organize the district steering committee meetings and laison between the

DCO office and implementing partners to address the issues in timely

manner.

1.2 Hold Monthly Coordination Meetings at District level Plan International + HANDS+ ACF

(rotational basis) Monthly

The agenda of this meeting will be shared with UNICEF along with the names

of Committee members. Minutes to be shared by IPs.

1.3 Hold Monthly Coordination Meetings at Taluka levelPlan International + HANDS+ ACF (SCF to

take lead)Monthly

Taluka level meetings will be organized before District level meetings. Names

of the Committee memebers to be shared with UNICEF. Minutes will be

shared with UNICEF.

1.4

Hold Monthly Coordination Meetings at Communityt level (Village WASH

Committee, Father Support Groups, Mother Support Groups, LHWs, Social

Organizers, CRPs)

Plan International + HANDS+ ACF MonthlyVillage WASH Committee will have at least 3 male and female memebers

who are also part of the Father & Mothers support groups

1.5Hold District Steering Meetings in District Ghotki, Khairpur and

Nausheroferoz

UNICEF + IPs (Plan International, HANDS,

SAFWCO, NRSP, ACF, and HELP including

district coordinators

Quarterly

Under the leadership of ADCs/DCO office, district steering committee

meentings will be organized on regular basis and UNICEF all implementing

partners will ensure to participate in planned meetings to share the

progress, key lessons learnt, best practices, challanges and way forward to

reach out to desired results planned and targetted for UN Maternal and

Child Stunting Reduction Programme.

1.6 Hold Provincial Steering Committee Meetings P&DD, LG, PHE&RDD, Nutrition Cell, and

UNICEF Bi-Annual

For technical review and oversight of stunting reduction programme,

provincial steering committee meetings will be organized and attended by

members technical working groups ( WASH and Nutrition). UNICEF will

ensure to participate in planned meetings to share progress with the

government and create the enabling enviornment for smooth exection of

integrated implementation startegies at the field level by all partners.

2 Convergence Mechanism at Field Level Exists and Implemented

2.1

Integrated WASH/ Nutrion Micro Plan prepared and shared with UNICEF

A. Micro plan to include joint training sessions for frontline health workers

(LHWs, CRPs, Teachers) on Health, Hygiene & Nutrition at the community

level

B. Micro plan Joint to include mechanism hold joint celebrations during all

campaigns (GBF week, MCW, GIDD Day, World Toilet Day, Hand Washing Day,

World Water day)

C. Micro plan to include number of combined WASH/ Nutrion Sessions held

which includes IYCF key messages along with hygiene promotion (at

community and school levels)

D. Micro plan to include combined cooking demonstrations and hygiene

promotion sessions

All implementing partners including; Plan

International, NRSP, SAFWCO, HANDS, ACF

and HELP

Monthly Micro-plans will be prepared on monthly basis and shared with UNICEF

2.2 All activities carried out as per the Micro Plan



and HELP

Monthly A status report will be shared after completion of the monthly micro plan.

2.3 Include MSG & FSG memebers in all village WASH Committees



and HELP

Regular A status report for each of the Talukas will be finalized by WASH & Nutrition

IPs by end March, 2017 providing composition of Village WASH Committee in

all the villages and its alignment with MSG and FSG memebers

2.4 LHW training sessions to be held jointly by WASH & Nutrition IPs



and HELP

As per WP

3 Joint Monitoring Mechanism prepared and finalized

3.1Joint monitoring framework finalized in relation to common indicators

(touch points on convergence)



and HELP

February, 2017

The first draft of the joint monitoring framework should be finalized by

February 15th.

3.2WASH & Nutrition IPs to share the monthly report on the indicators part of

joint monitoring framework



and HELP

Monthly

List of final joint indicators will be shared with WASH and Nutrition

implementing partners.

3.3 Conduct Quartery Joint monitoring visist



and HELP

Quarterly

Joint monitoring visits will be conducted on quarterly basis to cover the gaps

jointly and improve the quality of interventions. It will also create the

mechanism for accountability.

3.4 Conduct Bi-Annual Joint monitoring visits with UNICEF



and HELP

Bi-Annual

3.5 Involvement of DOH and SC in monitoring and supervision

UNICEF and all implementing partners

including; Plan International, NRSP,

SAFWCO, HANDS, ACF and HELP

Regular when

required

3.6 Development of 'Joint Checklist' for third party UNICEF ( WASH + Nutrition) February, 2017

4Roll out of Integrated BCC Campaigns and Finalization of LHWs training

material as well as Capacity Building trainings for LHWs

4.1 Finalize training material (PPT) including translation into Sindhi ACF/UNICEF 3rd February, 2017

4.2 Finalize manuals (ToT and participants) for LHW UNICEF 28th February, 2017

4.3 DCT to LHW program for LHW training on manual UNICEF To be finlaized

4.4 Conduct ToT and roll-out trainings at provincial and district level UNICEF, LHW Programme and

Implementing PartnersTo be finlaized

4.5 Completion of Behavioural Analysis UNICEF CO February, 2017

Currently behavioural analayis is underway, partners are being consulted

and data colleaction will be started in 1st week of Feb and will be completed

by end Feb, 2017

4.6 Integrated Stunting Reduction BCC Strategy finalized UNICEF CO Mar-17

4.7 Integrated WASH/ Nutrition Stunting Reduction IEC material finalized UNICEF CO Apr-17

4.8Integrated Stunting Reduction IEC material rolled out by WASH & Nutrition

IPsWASH & Nutrition IPs Apr-17

5 KM Mechanism exists and implemented

5.1

WASH & Nutrion IPs to share atleast one/or two Human Interest Stories

every month to generate the evidence around integrated efforts for

convergence of WASH and Nutrition interventions under stunting reduction

programme.



and HELP

Monthly

All implementing partners will be held accountable to create the evidence

gneration for joint results through documentation of 'Human Interest Stories'

around integration and convergence of programme (WASH and Nutrition)

5.2WASH and Nutrition implementing partners to document the key lessons

learnt on regular basis and report on quarterly basis



and HELP

Quarterly

All implementing partners will be held accountable to create the evidence

gneration for joint results through documentation of key lessons learnt

around integration and convergence of programme (WASH and Nutrition)

5.3 Nutrion IPs to share data of SAM children and PLWs with WASH partners ACF and HELP February, 2017 Nutrition implementing partners (ACF and HELP) to share the information

with WASH partners

6 Programme Reviews Stunting Reduction Programme

6.1 Hold Quarterly Joint Programme Review Meetings

UNICEF and All implementing partners



Quarterly

Next quarterly review meeting with partners will be organized in March,

2017.

6.2 Proper documentation on delays in implementation to be shared with USAID UNICEF (Nutrition & WASH) To be determined

Iram and Junaid will document the jusitifications and resaons of dalays

6.3 Annual Review of UN Maternal and Child Stunting Reduction Programme

UNICEF and All implementing partners



Annual

Joint Annual review will be conducted by end December, 2017 with all

implementing partners working under 'UN Maternal and Child Stunting

Reduction Programme'.

6.4 Request to Steering committee to relocate fund from HF to SchoolsWASH implementing partners ( Plan

International, NRSP, SAFWCO and HANDS)

Upcoming steering

committee meeting

During the upcoming steering committee meeting, UNICEF and WASH

implementing partners will inform ADC II about the revised list of health

facilities and issue of duplication and justification to divert funds under

WASH in Schools.

Integrated WASH/ Nutrition Action Plan UN Maternal & Child Stunting Reduction Programme

(To be implemented and Reviewed on Regular Basis)

Quarters

(2017)

Page 10

Annex 5: Evaluation Scope

The Scope of Evaluation defining characteristics included:

Programme Timeframe: As discussed in table 1, the evaluation covered a period between May 2016 and December 2018. However, the WASH and Nutrition Implementing Partners began with implementation on roll-on bases. The WASH IPs signed PCAs in May 2016 while the nutrition IPs signed the PCAs in July. Hence, the WASH IPs became at least 3 months earlier than the nutrition partners Geographic Coverage: The evaluation covered the programme impact areas in the three target districts i.e. Noushero Feroze, Ghotki and Khairpur in the Sindh province of Pakistan Stakeholder‘s Consultations: Evaluation covered the involved male and female stakeholders form public institutions, rights holders (services beneficiaries), community-led structures, civil society organizations, nutrition/WASH practitioner in technical working groups and private entities. These stakeholders were stretched across at national, provincial, districts and community levels Desk Review of the Key Project Documents and Scoping Literature Review: UNICEF country office Islamabad, field office Karachi and Implementing Partners shared essential documents with the evaluation team which were reviewed thoroughly by the evaluation team. The evaluation team also conducted scoping literature review and accessed reference documents from multiple sources including online scientific databases and grey literature, national & international key strategic and policy documents, and UNICEF evaluation documents standards etc. The evaluation team extensive reviewed of all the shared documents and reference materials which paved the way to undertake the evaluation and structured the methodology to gather and synthesize relevant information including findings, conclusions and recommendations Gathering Primary Qualitative Data: The evaluators undertook 49 IDIs with key stakeholders at national (9), provincial level (18) and district (22) levels. At the community level the evaluators conducted 31 FGDs with 220 participants and carried out direct observations of OTP centres, SC centres, latrines, hand pumps, hand washing stations at households and ODF status during the field sites visits. Statistical Analysis of Pre and Post KAP Data: The evaluators ran statistical analysis on the pre and post KAP raw data to ascertain the progress made overtime Evaluation Kick-off Meetings, Evaluation Findings Presentation and Discussions: After signing the contract, evaluation team held in-depth discussions at several occasions with UNICEF and Implementing partners at Islamabad and Karachi. In addition to face-to-face meetings, the evaluation team held telephonic conversations with UNICEF and IPs. The team also remained in touch with each other on emails and to share the updates and follow-up on the agreed tasks and deliverables at both ends. The evaluators also presented the evaluation findings at national and provincial levels to UNICEF and IP‘s staff. Level of Analysis: Programmatically, the evaluation examined all programme aspects under UNEG/OECD-DAC evaluation criteria for the entire programme timeframe. The evaluation team used the OECD/DAC criteria to assess relevance, efficiency, effectiveness, impact and sustainability comprehensively, using mixed method approach. Additionally, the evaluation team also gathered information and ascertained the state of the cross-cutting themes of gender, rights, equity and DRR

Page 11

Annex 6: Evaluation Matrix

The evaluation ToR listed DAC and non-DAC criteria-based questions (see annex 2 for the original ToR), which guided the formulation of Evaluation Matrix with few changes. In total there were 24 major questions in the ToR (2 in relevance, 6 in efficiency, 6 in effectiveness, 6 in impact, 2 in sustainability and 2 in cross-cutting themes).

The evaluation team assessed all these 24 questions and noted that: a) some questions were repetitive and transversal in nature; hence, they were repeated with slight difference in wordings under more than one DAC criteria and regrouped several criteria, b) some of the questions were of loaded nature that were having more than one question to explore multiple aspects, c) parts of some questions were of probing nature which could be captured during interviews and FGDs with participants, and d) aspects of cross-cutting questions (gender, equity, human rights and DRR) were covered under two questions only in the ToR.

In the inception phase, the evaluators modified a few of these listed questions in the ToR, which is being reflected in the evaluation matrix below. Additionally, the evaluators converted some of the questions into indicators of broader questions or made them part of the probing questions within the IDIs and FGDs topic guides. Subsequently, the evaluators added new questions and indicators to cover the important aspects of the programme in a coherent manner. For example, the evaluators developed 5 questions (instead of 2 questions in the ToR) for the cross-cutting themes of gender, equity, human rights and DRR under relevance and effectiveness sections of the criteria with more probing questions in the IDIs and FGDs topic guides. Based on these considerations, the evaluators rephrased and consolidated major questions into ‗Key Evaluation Questions and ‗Sub-questions‘ in the Evaluation Matrix.

The evaluation matrix contained a total of 25 questions (5 key questions and 20 sub-questions). These questions have been unpacked further in the qualitative data collection tools (IDIs and FGDs interview guides) through many probing questions to gather relevant information. For each evaluation question, specific data sources, indicators6, data collection methods were identified and included in the Evaluation Matrix. The selection of these questions was based on the following considerations to:

Inform the analysis of relevance, effectiveness, efficiency, impact and sustainability and cross-cutting themes of gender, equity, participation, HRBA and DRR

Increase the accuracy of information and mitigate the information gaps and limitations when used in a combined manner

Substantiate the diversity of the perspectives in the analysis of the planners, implementers and final beneficiaries including policy makers at national, provincial and district level, UNICEF, and implementing partners

6 Evaluators modified the indicators based on the feedback received from UNICEF on the 2

nd Inception report

Page 12

Evaluation Criteria & Key

Questions

Sub Questions Indicators Data/Information Sources

Data Collection & Methods

1. Relevance ―To what degree the objectives & interventions of the integrated WASH-Nutrition program were in line with needs and priorities of the target communities, government plans and international commitments‖?

1.1. To what extent the program was in line to the government and international priorities and directions?

Ascertain the extent to which the program objectives and activities are aligned to national, provincial and international priorities

Determine the extent to which the program interventions are aligned and complement the GoS efforts in addressing stunting through sector-wide integrative programming, particularly joint nutrition and WASH intervention packages

Assess the extent to which program is avoiding duplication of efforts through sector-wide coordination and collaboration with other key actors and their interventions don‘t overlap with the UN maternal and child stunting reduction program

National Health Vision

2025

Sindh government‘s

provincial plans &

policies

Provincial nutrition

sector strategic plans

SDGs agenda &

priorities

Scoping literature

review

Target districts

stunting levels & data

PDHS & other

surveys

Program Needs

Assessments

Integrated action

plans for Nutrition and

WASH teams and IPS

Baseline surveys

Key project

documents

Periodic progress and

monitoring reports

Primary and

secondary data and

information

Desk review of the

key documents

(internal project docs

& external national &

international docs)

IDIs with key

stakeholders

(UNICEF & IPs staff

and govt officials)

FGDs with community

& providers

Analysis of the qualitative data, using manual framework approach

Primary quantitative

data review,

tabulation and

analysis (baseline

and end line), using

statistical process

control

Secondary data

gathering and review

1.2. Was the program based on adequate needs assessment, analysis of the target communities (particularly to cater the needs of most vulnerable and marginalized groups) and relevant to the program objectives?

Determine the extent to which the program objectives and activities are consistent to systemic, human and geographical needs and vulnerabilities

Ascertain the extent to which the program is aligned specifically to the needs of women, children, disabled, vulnerable and deprived populations

Determine whether the program undertook rapid need assessment of the target communities prior to the start of the activities to guide the program design and delivery

1.3. Was information disaggregated according to gender and other relevant social differences (ethnicity, income levels, rural/urban split, DRR etc.) in the program design and delivery?

Assess the extent to which program interventions are consistent with the program needs of gathering and synthesizing disaggregated information on gender and social differences and vulnerabilities (income levels, ethnicity, rural/urban split, marginalized groups and DRR)

Identify the key aspects and interventions of the program that are designed and implemented to promote access to and availability of equitable services, particularly to the women, children, vulnerable and disadvantaged groups in the target communities

1.4. How relevant the target communities perceived the program in addressing their priority

Assess the extent to which the program adopted measures to gather beneficiaries feedback on the provided services, particularly from women, adolescents, poor and vulnerable segments of the population who

Page 13

WASH and Nutrition needs including measures to protect the rights of access to services in non-discriminatory way?

could raise and escalate their voices on access to and availability of services

Identify program activities that support the provision of services to women, children, disabled, vulnerable and deprived populations in non-discriminatory way

2. Efficiency

―Were resources (human, financial, material, technical) used in the best possible way to attain results‖?

2.1. How well aligned were the multiple WASH and Nutrition partnerships and to what extent they supported program convergence and integration?

Determine how the capacities of the implementing partners and their skill-base is contributing in the implementation of the integrated program

Ascertain the drivers/factors that supports or constrains the convergence or integration of the services

Examine how well UNICEF and IPs planned and applied integrative approaches and adaptive management

Key project

documents (proposal,

program reports,

financial reports,

program operational

plans, stakeholder

mapping, minutes of

steering committee

meetings, baseline

survey, existing case

studies etc.)

Nutrition and WASH

program data and

monitoring reports

Minutes of UNICEF &

Partner meetings

Minutes of steering

committee meetings

Program case studies

regarding changes in

policies and practices

Primary and

secondary data and

information

Performance

Management Plan

Desk review of the key documents

Nutrition and WASH Management Information System

Case studies

IDIs with key stakeholders (UNICEF & IPs staff and govt officials)

FGDs with community & providers


Primary quantitative data review, tabulation and analysis (baseline and end line), using statistical process control

Triangulating primary data with beneficiaries and stakeholders‘ interviews

Secondary data gathering and review

2.2. How well the allocated resources were managed by partners either directly or jointly to ensure timely, cost efficient attainment of results?

Analyze how efficiently UNICEF and IPs planned and

applied the implementation approaches (operational

plans, M&E, supply chain, IEC materials etc.) to manage

the program

Ascertain the measures that partners and UNICEF put in

place to avoid misuse of funds

Assess how efficiently UNICEF disburse funds in timely

manner per agreed modalities

2.3. To what extent costs incurred can be justified by the results achieved?

Determine the cost-benefit outcomes for integrated program by conducting cost-benefit analysis

Analyze key program interventions to assess the DALYs (disability adjusted life years) by useing LiST (Life Save Tool)

2.4. To what extent the program was kept flexible to adapt lessons from implementation and from other similar interventions to attain the results in more efficient manner?

List the changes made in the program ToC over time due to learning from implementation or another similar program

3. Effectiveness

―To what extent the program was able

3.1. To what extent the program produced the intended outputs and outcomes per the ToC?

Analyze the program data including pre and post KAP survey and qualitative data to assess the extent of reaching the set targets against the program result chain in effective manner

Ascertain the changes in nutrition and hygiene practices

Key project

documents (proposal,

program reports,

financial reports,

Desk review of the key documents

Nutrition and WASH Management Information System

Page 14

to achieve the intended objectives and application of the planned strategies‖?

at the household and community level program operational

plans, stakeholder

mapping, minutes of

steering committee

meetings, baseline

survey, existing case

studies etc.)

Nutrition and WASH

program data and

monitoring reports

Minutes of UNICEF &

Partner meetings

Minutes of steering

committee meetings

Program case studies

regarding changes in

policies and practices

Primary and

secondary data and

information

Bottleneck analysis

Formative Research

report

Performance

Management Plan

Case studies

IDIs with key stakeholders (UNICEF & IPs staff and govt officials)

FGDs with community & providers




Secondary data gathering and review

3.2. How were the program implementation strategies contributed in achieving better synergies and complementarities among the partners as part of the integrated programming towards stunting reduction?

Determine how effectively UNICEF and partners

operationalize the integrated strategy that resulted in

program‘s adaptations to constraints, ultimately helping

to achieve program objectives

3.3. To what extent has the application of integrated approach and implementation strategies worked as intended in influencing policies, strategies, plans, collaboration and integrated community engagements?

Ascertain program‘s contribution in influencing changes in government plans, policies and practices

Analyse the extent to which program contributed in improving nutrition and WASH service delivery at the community level

Asses the community engagement in the program through women, children, vulnerable population participation in the community-led structures that the program established during implementation

3.4. To what extent gender was mainstreamed in the program delivery and how well the identified barriers and bottlenecks were mitigated?

Assess how effectively the program adopt activities to address the specific needs of women, girls and children

Ascertain to which extent the program engaged and capacitate women in the community-based and community-led structures and the network of community-based providers

Gather views from women and program implementers on the factors that contributed to or constrained gender inclusion in the program delivery

3.5. What type of approaches and interventions were implemented and were effective to yield results in reducing stunting in disadvantaged, marginalized and less reachable areas/groups?

Assess how effectively the program targeted the most vulnerable and disadvantage groups by ensuring access to and availability of equitable nutrition and WASH services

Assess how the program effectively identified and engaged the most vulnerable families for targeted assistance such as provision of community grants

Assess how the program manage to provide services in hard to reach areas

3.6. To what degree human rights-based

Ascertain how effectively the program ensured inclusivity

Page 15

approaches were mainstreamed in the program design and delivery?

of different target groups, particularly women, children, adolescents, disabled and other vulnerable populations in non-discriminatory way

Assess how well the program ensured women, girls, children and vulnerable population participation in the program activities

3.7. How was DRR incorporated into the programme design and mainstreamed in programme implementation?

Ascertain which risks and hazards were identified and address in the program delivery

4. Sustainability ―Are the positive effects of the program sustainable‖?

4.1. What are the key factors and drivers (internal & external) that contribute to or constrain the continuity of the program effects and interventions after the intervention period?

Ascertain the key drivers/factors that contribute to or constrain the longer term effects of the program

PC1 including policy

briefs and advocacy

plans

Govt plans for scale-

up of the intervention

to other areas

Key project

documents that reflect

plans for future

programming

Primary and

secondary data and

information

Program data and

monitoring reports

Minutes of UNICEF &

Partner meetings

Minutes of steering

committee meetings

Performance

Management Plan

ODF committee

minutes

Desk review of the

key documents

IDIs with key

stakeholders

(UNICEF & IPs staff

and govt officials)

FGDs with community

& providers




4.2. What aspects of the program activities and the induced behaviors would likely to continue and last beyond the project life?

Asses how well UNICEF and IPs planned and implemented program exit strategy

Ascertain program contributions in institutionalization of policies and practices at the institutional level

Determine the extent of lasting the behaviour changes at individual, household and community level

Assess the likelihood of continuum of nutrition and WASH services after the exit of the program

Assess the extent to which community will maintain ODF village status

5. Impact

5.1. Has the program contributed in adaptation of improved health, nutrition,

Determine the level of change in beneficiaries‘ knowledge and/or practices, based on the analysis of quantitative and qualitative data (pre and post KAP, FGDs, direct observations)

Key project

documents

Baseline and end line

Desk review of the

key documents

IDIs with key

Page 16

―To what degree the program contributed in achieving the intended long-term outcomes and impact of the program‖?

sanitation and hygiene practices and behaviors?

KAP surveys

Bottleneck analysis

Monitoring reports

and case studies

Minutes of UNICEF &

Partner meetings

Minutes of steering

committee meetings

Primary and

secondary data and

information

Formative Research

report

Performance

Management Plan

stakeholders

(UNICEF & IPs staff

and govt officials)

FGDs with community

& providers

Primary quantitative

data review

Secondary data

gathering and review

Triangulation of the

collated data

5.2. To what extent the program contributed in creating an enabling environment for improved Nutrition and WASH service delivery?

Assess the extent to which the program contributed in changes of policies, practices and enabling environment to support the integrated services delivery

Ascertain program contribution in access to and availability of nutrition and WASH services, particularly women, girls, children and vulnerable population

5.3. Have any unintended, unexpected or negative effects been observed during program implementation as result of this program?

Determine factors within or outside the program that influenced positively or negatively the program outcomes

Page 17

Annex 7: Overview of Qualitative Methods

Page 18

Annex 8: FGD and Observation Tool Kit

FGD Guide—Lady Health Workers

Location ________________________ Date ___________________________

Ground rules for the FGD

Start by explaining the ground rules as follows:

Before we start, I would like to remind you that there is no right or wrong answers in this discussion. We are

interested in knowing what each of you think, so please feel free to be frank and to share your point of view,

regardless of whether you agree or disagree with what you hear.

It is very important that we hear all your opinions.

You probably prefer that your comments not be repeated to people outside of this group. Please treat others

in the group as you want to be treated by not telling anyone about what you hear in this discussion today.

Don‘t feel like you must respond to me all the time. Feel free to have a conversation with one another about

these questions

Feel free to get up and get more refreshments if you would like.

INFORMED CONSENT

My name is _________________ and I am a member of the Evaluation Team, commissioned by UNICEF to conduct evaluation of UN

maternal and child stunting program. I thank you for taking time for this discussion today. I will be asking questions about how the

integrated Nutrition and WASH programme interventions were implemented and what changes this programme brought in your lives. We

want to emphasize that there is no right or wrong answer to any questions. We are keen to know your point of view, reflection, experiences

and learning of this program. The findings will be used to refine and scale-up the integrated Nutrition and WASH approaches at the

National and International levels.

The discussion usually takes 60 minutes to complete. Whatever information you provide will be kept strictly confidential and will be dealt

with anonymity. To keep the anonymity of the information, we will follow steps as: a) we will assign pseudonyms against your names and

will use them to refer to you, b) no personal information will be sought from you, c) nothing will be stated in the evaluation report with your

names, d) No one will be able to track you from your responses, e) the information collected will be destroyed after a certain time, and f)

the group participants are requested not to share any information or views outside the group.

Additionally, you can refuse to participate in the discussion, leave the discussion at any time or refuse to answer any specific question.

Furthermore, this information is not collected to grade anyone‘s performance or to provide incentives or grade your community/vil lage,

hence, there is no associated risk or threat or impact on you.

We will be taking notes to help us remember what is said during the discussion. If you allow we will use recorder so that we do not miss

any important information. We will destroy the recording after certain time. If you have any question or concerns, even after we leave, you

can ask those from the lead evaluator Dr Asmatullah on [email protected] or 03340090944. At this point, I want your permission to

record the interview, take notes to avoid any loss of information and seek your consent to record the discussion. We also seek your

permission to take photos to be used in the report.

Prior to the start of interview, do you have any question about the evaluation or the interview process?

Do you agree to participate in the discussion? YES NO

Do you allow recording the discussion? YES NO

Do you allow taking photos during the discussion YES NO

THANK YOU

mailto:[email protected]

Page 19

and I will both be taking notes to help us remember what is said. We would also like to ask for your names,

but we will not include them in the report. We also wanted to take permission for recording the session so that

we do not miss any important information but only if you permit

Let's start by going around the circle and having each person introduce herself (members of the ET shall

introduce themselves again during introduction circle, if needed)

Points to remember for the moderators

Throughout the focus group interview, moderators shall focus to elicit responses from participants who

may be reluctant to contribute to the discussion through the pause and the probe. Here are some examples of

probes. • Would you explain further? • Would you give me an example of what you mean? • Would you say

more? • Is there anything else? • I don‘t understand.

As participants speak, the moderators must use active listening techniques such as a forward lean, head

nodding, or short verbal responses, like ―go on,‖ to let participants know that their contributions are welcome.

It is important, however, not to communicate a judgment of the participant‘s contribution by using words like

―correct‖ or ―good.‖

Participants list

Pseudonyms Age Pseudonyms Age

1. 2.

3. 4.

5. 6.

7. 8.

9. 10.

Questions

Introductory Questions

1. Tell us your name and how long you have been participating in the programme. How was it that you first learned

about the programme?

Transition questions

2. Think back to when you first became involved with the programme. What were your first impressions?

Probe for: level of their engagement within the programme, addressing community needs particularly most

vulnerable groups, also who are the most vulnerable groups and how the programme was geared to address their

needs

Key Questions Responses

3. According to you what is the intended role of the LHW in this programme? To what extent do you think you have been able to deliver on the expected role?

Probe for: community mobilization process, engagement of

mother/father/grandmother forums, village WASH and health committee,

engagement in PATS plus OTP staff, coordination with IYCF counsellors,

referrals, hygiene sessions with school kids etc)

4. What were the key messages /behaviors/interventions and target groups

you were focusing on in your catchment population?

Probe for: if they were disseminating messages on both health and WASH,

who were their target groups (mothers, fathers health committee WASH

Page 20

committee etc), engagement in WASH activities including PATS plus,

knowledge about community investment grants

5. In your opinion/experience which interventions were most appropriate

and yielded the desired results?

Probe: on the anticipated and actual time for delivery of interventions,

response from community, impact on changed behaviours

6. Do you think that the support and resources provided to you was

adequate and timely for the delivery of desired interventions?

Probe for quality of training, provision of adequate and timely supplies, post

training /field support, incentives, tools/IEC materials, feedback, supervision

7. Did the processes undertaken for Social Mobilization help identify and

target the poorest (lowest quintiles), the vulnerable including the disabled

as a priority? If so, what was done to address their WASH/Nutrition

related needs? How was the equity focus maintained during the

identification, selection and implementation?

8. How familiar are you with other cadres of health workers supporting the

programme interventions in your catchment area?

Probe: if they understand the role of social organizers/ Couple health

workers, village health and WASH committee, any coordination mechanism

established among them, linkages with WASH activities and community

investment grants

9. In your opinion what impact change this programme has been able to

bring in the lives of women and children especially with regard to

changing their key behaviours and stunting?

Probe: if they feel that practices for nutrition, breast feeding, hygiene,

disposal of faeces, treatment of drinking water and heath seeking behaviours

have been changed

10. How many children did you to refer to OTP/SC and how do you think

they were treated there?

Probe: Explore for any human case study

11. What do you think were the key gaps and challenges in the delivery of

the programme interventions? Is there any suggestion for what could be

done differently in future programming?

Probe: gaps in terms of field activities supplies, expectations from them,

enabling environment, coordination gaps?

12. According to you who shall take the responsibility of bringing up /feeding

the children and why? (Mother/father/grandmother /all). What is the

common practice in your community?

Probe for what role can father and grandmother etc. can help in feeding the

children

13. How do you think you will be able to continue these interventions in the

field after the closure of project?

Probe: why do they think their work is important? How they feel motivated to

continue? Do they identify any bottlenecks?

Concluding questions

Let‘s summarize some of the key points from our discussion. Is there

anything else?

Page 21

Do you have any questions, comments or suggestions for us?

FGD Guide—Mothers Group

Location ________________________ Date ___________________________



Before we start, I would like to remind you that there are no right or wrong answers in this discussion. We are







these questions







INFORMED CONSENT

My name is _________________ and I am a member of the Evaluation Team, commissioned by UNICEF to conduct evaluation of UN maternal and child stunting program. I thank you for taking time for this discussion today. I will be asking questions about how the integrated Nutrition and WASH programme interventions were implemented and what changes this programme brought in your lives. We want to emphasize that there is no right or wrong answer to any questions. We are keen to know your point of view, reflection, experiences and learning of this program. The findings will be used to refine and scale-up the integrated Nutrition and WASH approaches at the National and International levels.

The discussion usually takes 60 minutes to complete. Whatever information you provide will be kept strictly confidential and will be dealt with anonymity. To keep the anonymity of the information, we will follow steps as: a) we will assign pseudonyms against your names and will use them to refer to you, b) no personal information will be sought from you, c) nothing will be stated in the evaluation report with your names, d) No one will be able to track you from your responses, e) the information collected will be destroyed after a certain time, and f) the group participants are requested not to share any information or views outside the group.

Additionally, you can refuse to participate in the discussion, leave the discussion at any time or refuse to answer any specific question. Furthermore, this information is not collected to grade anyone‘s performance or to provide incentives or grade your community/village, hence, there is no associated risk or threat or impact on you.

We will be taking notes to help us remember what is said during the discussion. If you allow we will use recorder so that we do not miss any important information. We will destroy the recording after certain time. If you have any question or concerns, even after we leave, you can ask those from the lead evaluator Dr Asmatullah on [email protected] or 03340090944. At this point, I want your permission to record the interview, take notes to avoid any loss of information and seek your consent to record the discussion. We also seek your permission to take photos to be used in the report. Prior to the start of interview, do you have any question about the evaluation or the interview process? Do you agree to participate in the discussion? YES NO Do you allow recording the discussion? YES NO


THANK YOU


Page 22










Participants list


1. 6.

2. 7.

3. 8.

4. 9.

5. 10.

Questions


1. Tell us your name and how long you have been participating in the programme. How was it that you first learned

about the programme?



Probe for: level of their engagement within the programme, addressing community needs particularly most vulnerable

groups, also who are the most vulnerable groups and how the programme was geared to address their needs

Key questions Responses

3. According to you what are the reasons that the children get sick

and their growth is stunted?

Probe for unsafe water, sanitation, hygiene as leading causes of child sickness

4. In the last two weeks, was any child in your family got sick and

how often does he/she get sick?

Probe for:

Diarrhoea and ARI

Days of hospitalization for any disease, particularly diarrhoea

and ARI

Costs of treatment

5. According to you how important is breast feeding for a child‘s

health and how shall a mother breastfeed a child?

Page 23

Probe for when to start breastfeeding, exclusive and continuation breastfeeding, advantages and disadvantages of breast milk, associated problems with formula milk

6. If the child is not adequately fed with nutritious food and

breastfed, how do you think it will impact his/her health?

Probe for

Stunting, growth retardation, multiple infection etc

If your child is week and not growing well, where would you go to

seek healthcare?

What do you think of the services that were given to your

children for not growing well?

7. When do you think supplemental food shall be introduced to a

child‘s diet and how does it impact a child‘s overall health?

Probe for: adequate amounts of nutritious, age appropriate food to children from 6 to 24 months of age

8. How is hand washing important for the health and prevention of

illness? When do you think hand must be washed? How often do

you practice hand washing?

Probe for

Hand washing at five critical times: after using the toilet, after

handling baby's faces, before eating, before feeding a child, and

before cooking or preparing food,

Observe hand washing area at their home, observe if hand

washing area has soap and being used at the end of FGD

explore what are the barriers which have been addressed and

the barriers which still exist to adapt hand washing practice

9. Do you know about the role of LHW, CHW, CRP in your

community and were they ever visited you? What was the

purpose of their visit?

10. Do you know where to take your child for screening and growth

monitoring?

Probe if they are familiar with OPT mobile and static sites and SC

11. What diseases can be caused by unsafe drinking water? Where

do you get drinking water from and is it safe to drink? How do

you store it? Are you familiar with water treatment to make it safe

for drinking?

Probe for safe handling and storage of drinking water, non-availability or scarcity of drinking water, poor water infrastructure

12. Why is important to safely dispose of faces? What methods are

commonly being used in your community?

Probe for: for ODF status, harmful effects of contamination, sewerage systems/infrastructure and who is managing it, management of solid and liquid waste practices if any, Waste water handling and treatment

13. What has been your interaction with UN stunting project? How

were you engaging and how do you think it has impacted your

lives?

Probe for the interventions of project, engagement level and knowledge and practice change

14. How do you think you can continue to work in the community

Page 24

after the closure of project?

Probe for: why do they think their work is important? How they feel motivated to continue? Do they identify any bottlenecks

Concluding Question Let‘s summarize some of the key points from our discussion. Is there anything else?


Guide—Village WASH Committee

Location ________________________ Date ___________________________







INFORMED CONSENT






THANK YOU


Page 25




these questions
















Participants list


1. 6.

2. 7.

3. 8.

4. 9.

5. 10.

Questions


1. Tell us your name and how long you have been participating in the programme. How was it that you first

learned about the programme?




3. In your opinion why has the village WASH committee been formed?

Probe for selection of committee members, scope of work, frequency of meetings, and understanding about the role as WASH committee members

4. To what extent do you think the committee has achieved its intended

role?

Probe: any achievement of the committee so far

Page 26

5. What do you understand by the village action plan and how did you

prepare and implement the action plan for your village?

Probe for progress against the action plans, resources required to implement action plans

6. What were the bottle necks/gaps and challenges for programme

implantation? How did u mitigate them? What role UNICEF / partners

played in this regard?

Probe for suggestions for what could be done differently in future for better implementation for such programme?

7. What trainings and resources you received from UNICEF/IPs to be

able to implement the interventions?

Probe: adequacy of resources, duration of trainings, IEC material supplies and timely delivery

8. How was the committee engage with the other interventions being

implemented by UNICEF in area

Probe: for linkage with CRP, LHW, LG rep, School WASH, mother support group, OTP/SC centres etc. Mechanism for working with them, integration with nutrition

9. What were the key messages /behaviours/interventions and target

groups you were focusing on in your catchment population?

Probe: if they were disseminating messages on both health and WASH, who were their target groups (mothers, fathers health committee WASH committee etc.), engagement in WASH activities including PATS plus, knowledge about community investment grants

10. In your opinion/experience which interventions were most

appropriate and yielded the desired results?

Probe: on the anticipated and actual time for delivery of interventions, response from community, impact on changed behaviours

11. Did the processes undertaken for Social Mobilization help identify

and target the poorest (lowest quintiles), the vulnerable including the

disabled as a priority? If so, what was done to address their

WASH/Nutrition related needs? How was the equity focus

maintained during the identification, selection and implementation of

community investment grants?

12. Are you familiar with other cadres of health/WASH workers

supporting the programme interventions in your catchment area?

Probe: if they understand the role of social organizers/ Couple health workers, village health and WASH committee, any coordination mechanism established among them, linkages with WASH activities and community investment grants

13. In your opinion what impact change this programme has been able

to bring in the lives of women and children especially about changing

their key behaviours and stunting?

Probe: if they feel that practices for nutrition, breast feeding, hygiene, disposal of faeces, treatment of drinking water and heath seeking behaviours have been changed


they were treated there


15. How do you think you will be able to continue these interventions in

Page 27

the field after the closure of project?

Probe: why do they think their work is important? How they feel motivated to continue? Do they identify any bottlenecks?

Concluding questions Let‘s summarize some of the key points from our discussion. Is there anything else?

Do you have questions or suggestion for us?

Community Resource Persons

Location ________________________ Date ___________________________










these questions


INFORMED CONSENT






THANK YOU


Page 28















Participants list


1. 6.

2. 7.

3. 8.

4. 9.

5. 10.

Questions


1. Tell us your name and how long you have been participating in the program. How was it that you first learned

about the program?


2. Think back to when you first became involved with the program. What were your first impressions?


1. According to you what is the intended role of the CRP in this

programme? To what extent do you think you have been able to

deliver on the expected role?

Probe for: community mobilization process, engagement of mother/father/grandmother forums, village WASH and health committee, engagement in PATS plus OTP staff, coordination with IYCF counsellors, referrals, hygiene sessions with school kids etc.)

2. What were the key messages /behaviours/interventions and target

groups you were focusing on in your catchment population?

Probe for: if they were disseminating messages on both health and WASH, who were their target groups (mothers, fathers health committee WASH committee etc.), engagement in WASH activities including PATS plus, knowledge about community investment grants

Page 29

3. In your opinion/experience which interventions were most appropriate

and yielded the desired results?

Probe: on the anticipated and actual time for delivery of interventions, response from community, impact on changed behaviours

4. Do you think that the support and resources provided to you was

adequate and timely for the delivery of desired interventions?

Probe for quality of training, provision of adequate and timely supplies, post training /field support, incentives, tools/IEC materials, feedback, supervision

5. Did the processes undertaken for Social Mobilization help identify and

target the poorest (lowest quintiles), the vulnerable including the

disabled as a priority? If so, what was done to address their

WASH/Nutrition related needs? How was the equity focus maintained

during the identification, selection and implementation?

6. How familiar are you with other cadres of health workers supporting

the programme interventions in your catchment area?

Probe: if they understand the role of social organizers/ Couple health workers, village health and WASH committee, any coordination mechanism established among them, linkages with WASH activities and community investment grants

7. In your opinion what impact change this programme has been able to

bring in the lives of women and children especially with regard to

changing their key behaviours and stunting?

Probe: if they feel that practices for nutrition, breast feeding, hygiene, disposal of faeces, treatment of drinking water and heath seeking behaviours have been changed


they were treated there?


9. What do you think were the key gaps and challenges in the delivery of

the programme interventions? Is there any suggestion for what could

be done differently in future programming?

Probe: gaps in terms of field activities supplies, expectations from them, enabling environment, coordination gaps?

10. According to you who shall take the responsibility of bringing up

/feeding the children and why? (Mother/father/grandmother /all). What

is the common practice in your community?

Probe for what role can father and grandmother etc can help in feeding the children

11. How do you think you will be able to continue these interventions in

the field after the closure of project?

Probe: why do they think their work is important? How they feel motivated to continue? Do they identify any bottlenecks?

Concluding questions Let‘s summarize some of the key points from our discussion. Is there anything else?


Page 30

FGD Guide—Fathers Group

Location ________________________ Date ___________________________










these questions








INFORMED CONSENT






THANK YOU


Page 31









Participants list


1. 6.

2. 7.

3. 8.

4. 9.

5. 10.

Questions


1. Tell us your names and how long you have been participating in the programme. How was it that you first



2. Think back to when you first became involved with the programme. What was your first impressions?

Probe for: level of their engagement within the programme, addressing community needs particularly most

vulnerable groups, also who are the most vulnerable groups and how the programme was geared to address their

needs



and their growth is stunted?

Probe for unsafe water, sanitation, hygiene as leading causes of child sickness/growth faltering

4. In the last two weeks, was any child in your family got sick and

how often does he/she get sick?

Probe for:

Diarrhoea and ARI

Days of hospitalization for any disease, particularly diarrhoea and ARI

Costs of treatment


illness? When do you think hand must be washed? How often do

you practice hand washing?

Probe for

Page 32

Hand washing at five critical times: after using the toilet, after handling

baby's faces, before eating, before feeding a child, and before

cooking or preparing food,

Observe hand washing area at their home, observe if hand washing

area has soap and being used at the end of the FGD

Explore what are the barriers which have been addressed and the

barriers which still exist to adapt hand washing practice

6. Do you know about the role of LHW, CHW, CRP in your

community and were they ever visited you? What was the

purpose of their visit?

7. Do you know where to take your child for screening and growth

monitoring?

Probe if they are familiar with OPT mobile and static sites and SC


do you get drinking water from and is it safe to drink? How do you

store it? Are you familiar with water treatment to make it safe for

drinking?

Probe for safe handling and storage of drinking water, non-availability or scarcity of drinking water, poor water infrastructure



Probe for: for ODF status, harmful effects of contamination, sewerage systems/infrastructure and who is managing it, management of solid and liquid waste practices if any, Waste water handling and treatment

10. What has been your interaction with UN stunting project? How

were you engaging and how do you think it has impacted your

lives?

Probe for the interventions of project, engagement level and knowledge and practice change

11. How do you think you can continue to work in the community after

the closure of project?

Probe for: why do they think their work is important? How they feel motivated to continue? Do they identify any bottlenecks

Concluding Question Let‘s summarize some of the key points from our discussion. Is there anything else?


Page 33

FGD Guide—School Teachers

Location ________________________ Date ___________________________










these questions








INFORMED CONSENT






THANK YOU


Page 34









Participants list


1. 6.

2. 7.

3. 8.

4. 9.

5. 10.

Questions






Key questions

Focus Group Questions: Please listen carefully to the questions and provide your feedback as per the

guidelines provided.



and their growth is stunted


4. How you and your school engaged in the UNICEF were

supported stunting reduction programme? What interventions

have been introduced in your school to improve the health

hygiene practices and facilities?

5. Did you and your fellow teachers receive training on WASH

programme? How do you think the training was helpful to for

implementing school WASH services? What was the duration of

training?

6. Do you think that the resources provided for health hygiene

promotion in your school were adequate and relevant? When did

you receive the interventions /supplies and was there enough

time to implement the interventions?

Page 35

Probe: IEC material was available, number of latrines and hand washing area and when were they built?

7. Are you familiar with the 3-start approach for school WASH?

Which category do you think your school fits in?

Probe for their understanding of the 3-start approach and ask for the interventions and results according to the category of school?

8. In your opinion what were the bottlenecks in the implementation

of the school WASH? What do you suggest could be done

differently in future for such programmes?

9. Are you familiar with the idea of WASH clubs in school? Does

your school has a WASH club? If yes, why do you think WASH

club has been formed in the school? What has it done?

Prob for: who are the members, what is its scope, are all children aware of it, what were its achievements

10. In the last few years, what changes has occurred to improve

WASH facilities in the school?

Probe: were latrine constructed? Was hand area established?

11. Are you familiar with 3-star approach? If yes in which category

your school falls under 3-star approach and why?


illness? When do you think hand must be washed?

Probe for:

Hand washing at five critical times: after using the toilet, after

handling baby's faces, before eating, before feeding a child, and

before cooking or preparing food,

Observe hand washing area at their home, observe if hand washing

area has soap and being used at the end of FGD

explore what are the barriers which have been addressed and the


13. How many of you were engaged in activities/games to improve

hygiene? How often were they conducted?

Probe: daily group activities to improve the effectiveness, hand washing days other interventions to promote health hygiene

14. How many of you have adopted health and hygiene learning

(messages/practices) in school? If yes how? If not, why?

15. Have you passed on this information to your families/village, and

did you see any changes in family/neighbor‘s health and hygiene

practices?

Probe: if students shared information with family members, any change in the practice of their family members and the community


do you get drinking water from and is it safe to drink?



Probe: for ODF status, harmful effects of contamination, sewerage systems/infrastructure and who is managing it, management of solid and liquid waste practices if any, Waste water handling and treatment Are their children at school and adults in your community who still continue to defecate in open? Why?

18. Do you think that the school WASH interventions had any impact

on the student‘s health? including number of day taken off due to

sickness/class attendance

Probe: Has there decrease in the number of off days taken by students due to illness? Was MHM results in better attendance by girls

19. Can you please share your impressions (to assess child friendly

facilities)? Ask few of these to teacher and observe the rest

a) Are the facilities sufficient in number (or are there long queues)?

Page 36

b) Do the facilities have soap, running water, and water/ablution jug/mug? c) Is the height of the water stand post accessible for children? d) Are the latrines properly ventilated and clean at all time? e) Can all children (girls, boys as well as disabled children) access the Water and Sanitation facilities equitably? f) Is the area around the latrine and water stand post kept clean? g) Do the latrine facilities provide privacy? h) Are all facilities still functional and in use of school children? i) Are there any which are either non-functional or inaccessible to you (used by teachers or others)?

20. Concluding question

Let‘s summarize some of the key points from our discussion. Is there anything else

Do you have questions, comments and suggestions for us?

FGD Guide—School Children

Assent Form School Children (WASH Club Member)

Location ________________________ (Boys or Girl School) Date ____________________

INTRODUCTION AND INFORMED ASSENT

My name is ___________ and my colleagues‘ names are ________________ and ____________________

We are member of an evaluation team who are visiting your school to meet you and collect some information. Let us

just share more about the evaluation and our purpose of visit so that you can decide if you want to help us or not with

this study. Please feel free to ask me more questions about the purpose of visit. Our telephone number and e-mail

address is listed at the bottom of this page. If you have questions and would like to discuss them with someone else,

you can speak to Dr Asmtullah and Dr Javeria who sit in the main office whose number is also listed at the end of this

page. They will be able to answer your questions. I would like you to help us because you are a member of the school

WASH club.

We would like to ask you questions about how the WASH club was formed, what your role as WASH club members,

what changes you can see in the WASH infrastructure of school over last two years, hygiene and cleanliness

practices of yourself and other children in the school and your opinions about the health and hygiene practices in the

village where you live in.

Helping us with this study will take about 30 minutes of your time. You will not miss any important class time. Your

teachers have granted us permission to talk to you and will be standing outside of room and you can call them inside

anytime you want .We will not record your names and will assign you pseudonym names. Only this number will be left

on the questionnaire, so nobody, not even me, will know how you answers. It will work the same way for the teachers.

Nothing in this study will be graded.

Nobody will be rude or trick you in any way. No one will know about your responses and further more your responses

are not being used to grade you, your teachers or your school. Even, if you do not understand any questions then ask

for clarification. If you may not want to answer any question so that is your right and you can say no to it, this will have

no implications on you, your teachers and school.

Your teachers know about this and agree that it is okay for you to help us if you want to. You may find the activity a

fun. The things we will learn from this study will help us and others to improve school health and hygiene and WASH

programs designed for schools in future.

Page 37

No one is making you help us, and you don‘t have to if you don‘t want to. If you don‘t want to help us with the study

nothing bad will happen to you. No one will be mad at you. If you decide later that you don‘t want to be part of this

study, you or your parent/guardian /teacher can tell us by calling, emailing, or writing to us and I will put all of the

answer sheets in the garbage and will not include you in my study. If you do want to be in my study, nobody will know

your answers, including me. I am asking children of WASH clubs of other schools, many mothers and fathers in

community and other people so the information from you and your teacher will just be a little part of the big study.

When we finish the study, we might talk about what we learned from other people, or write it down so other people can

read it, but we will always talk about groups of kids, never about you and will never take your name.

You will be given two copies of this page. If you would like to help us and answer the questions please select ―Yes‖ on

this form and return, if you don‘t want to participate select ―No‖ and return the paper. We will not ask you any reason

for deciding not to participate in the discussion. You can keep the second copy of the form for your record

Evaluation Team members

Ms________________________

Contact No___________________

Address ________________________

E-mail:

If you want to ask someone else any questions about this assignment you can contact:

Dr Asmtaullah and Dr Javeria

Lead Evaluators

Contact: 03340090944

Email: info@the visiontomorrow.com

Postal Address : House 141 Steet 23 G10/2 Islamabad Pakistan

Assent

Select by circling YES or NO from below and return

Yes: I want to participate in the discussion

No: I don‘t agree to participate in the discussions







Page 38




these questions
















Participants list


1. 6.

2. 7.

3. 8.

4. 9.

5. 10.

Questions






Key questions

Focus Group Questions: Please listen carefully to the questions and provide your feedback as per the

guidelines provided.


3. According to you what are the reasons that the children get sick and

their growth is stunted


Page 39

4. Are you familiar with the idea of WASH clubs in school? Does your

school has a WASH club? If yes, why do you think WASH club has

been formed in the school? What has it done?

5. In the last few years, what changes has occurred to improve WASH

facilities in the school?

Probe: were latrine constructed? Was hand area established?

6. Are you familiar with 3-star approach? If yes in which category your

school falls under 3-star approach and why?


illness? When do you think hand must be washed?

Probe for:

Hand washing at five critical times: after using the toilet, after handling

baby's faces, before eating, before feeding a child, and before cooking or

preparing food,

Observe hand washing area at their home, observe if hand washing area

has soap and being used at the end of FGD

explore what are the barriers which have been addressed and the


8. How many of you were engaged in activities/games to improve

hygiene? How often were they conducted?

Probe: daily group activities to improve the effectiveness, hand washing days other interventions to promote health hygiene

9. How many of you have adopted health and hygiene learning

(messages/practices) in school? If yes how? If not, why?

10. Have you passed on this information to your families/village, and did

you see any changes in family/neighbor‘s health and hygiene

practices?

Probe: if students shared information with family members, any change in the practice of their family members and the community

11. What diseases can be caused by unsafe drinking water? Where do

you get drinking water from and is it safe to drink?



13. Concluding question

Let‘s summarize some of the key points from our discussion. Is there anything else

Do you have questions, comments and suggestions for us?

Page 40

FGD Guide—Entrepreneurs

Location ________________________ Date ___________________________










these questions








INFORMED CONSENT






THANK YOU


Page 41









Participants list


1. 6.

2. 7.

3. 8.

4. 9.

5. 10.

Transitioning Question: Think back, when you first became involved with the program. What was your first

impression?


1. What has been your interaction with UN stunting project?

Probe for:

How were you engaged?

How do you think it has impacted your lives?

What you know about the project interventions?

Was the project resulted in improving your knowledge? What aspects

did you learn?

What practice and skills you have adapted as result of the programme?

2. How do you think you can continue to work in the community after the

closure of project?

Probe:

Why do they think their work is important?

How they feel motivated to continue?

Do they identify any bottlenecks

Concluding question Let‘s summarize some of the key points from our discussion. Is there anything else?


Page 42

FGD Guide—Masons

Location ________________________ Date ___________________________










these questions








INFORMED CONSENT






THANK YOU


Page 43









Participants list

Name Age Name Age

1. 6.

2. 7.

3. 8.

4. 9.

5. 10.

Questions





2. What do you know about this stunting reduction programme? How were you engaged in the programme?

Probe: How building latrines/safe disposal of faeces can contribute in health benefits?

3. Why do you think Masons were trained by UNICEF? How were

the masons identified for trainings? How do you feel masons

training was helpful in addressing in constructing low costs

latrines? How many days of trainings did you receive?

Probe: Have they or their peers-built latrines for marginalized family? Are the trained masons constructing low cost latrines in their community? How do people contact them?

4. Do you know about any new low-cost material manufacturers who

are providing materials to Sanitary Marts?

Probe: if they know about the entrepreneurs trained by UNICEF and the low-cost manufacturers, how to contact them

5. Are you aware of the Village wash committee and CRPs? What

role they play in your community? Did you ever participate in the

WASH committee meetings?

Probe: Explore the linkages between masons, WASH committee, CRPs for identification of poor and marginalized families for community investment grants

Page 44

Spot Checks/Observation Guide

Instructions for Enumerators

The enumerators are expected to record their observation while being in field. The observations limited

to noticing general environment when the enumerators walk through the village and street and spot

checks of the hand washing stations in schools and households, latrines and drinking water storage

To observe the communal hand pumps the enumerators ill not need any informed consent but are

expected to respect the cultural norms such as not visiting the pumps if females are using it at the time

of observation

The spot checks at households can be done only by female enumerators and male enumerators will not

try to go inside the house to respect the local culture

IF the FGD with mother support group is conducted in a household level the female enumerator will

request the owner of the house who is expectedly member of support group to visit the hand washing

station and latrines within the household. The request will be made in Privacy to the owner of house

explaining the purpose of visiting the hand washing stations as to understand how the hand washing

stations in the village are set up and used. The female observer will further explain that information

collected will be anonymous and will have no implications of any type. If the owner agrees with the

enumerators, then she will have brief spot check and leave after thanking her.

In Schools and Health facilities the enumerators will seek permission from senior teachers/Facility in-

charge to observe the hand washing stations and latrine. The observations will be done only in the

schools and health facilities where the enumerators will visit for FGD or IDI. The observer will further

explain that information collected will be anonymous and will have no implications of any type. If the

head teacher and HF In-charge agree with the enumerators, then they will have brief spot check and

leave after thanking them. They will observe hand washing stations and latrines in the presence of

teachers and HF In-charges.

The enumerators are not required to record any address or collected any other personal information

from anyone during the spot check

The Enumerators and ET team will discuss the observations and use the double entry notebook to further reduce

the bias

Observation Thoughts

Checklist/Marker of the Observation/Spot-Checks:

A few markers for observation areas have been listed below as an example while the enumerators are expected to

record all observations they come across which are relevant thematic areas of the programme intervention

Observe the household Compound and Courtyard or any area where FGD is being conducted for

if animal or child feces are visible

if there is evidence of sweeping

Observing whether hand washing is practiced correctly based on a list of criteria

Page 45

Observing the latrines Cleanliness

Latrine has a lid

Does not have flies

Has soap or ash nearby, etc.

Observing the type of water storage container for

What is the height of the container,

Whether it is covered,

Whether there is a separate clean mug to use, etc.

Observing of hand pump area Time, place, number and type of people there

Details of what happens at the site, sights, sounds, smells,

Observation regarding the certain age group or gender comes to this hand pump, or that it differs at certain times of day etc

Observation during WASH and Safe Water supply in Health facility or School

Existence, quality, and capacity of WASH infrastructure

Functional latrines

Open defecation in and near the school/HF is eliminated

Hand washing station with soap and water is available

Observation Checklist for OTP/SFP/SC

Instructions

The enumerators shall introduce themselves and explain the purpose of visit

They shall seek permission from the facility staff to observe the service delivery point

The enumerator will follow the steps in the spot-checks/observation guide and will take

informed consent from the health facility in-charge per annex C and D

Service Delivery Points Observation Checklist

Area Yes No NA Remarks/comments

Infrastructure: Observe the following facilities

Adequate waiting area for clients?

Breastfeeding corners?

In-patient facility for severely malnourished, only applicable for SC?

Are there appropriate facilities for mothers to bath and do laundry? Applicable only to SC

Are the rooms and courtyard clean (no litter, no faeces)?

Is hand washing Facility available in all areas for mothers and children

Water and Sanitation: Are the following services available?

Safe drinking water?

Functional Latrines available for both patients and attendants?

Waste collection system for non- medical waste?

Page 46

Waste collection system for medical waste?

Material: Are the following printed material present?

Are there adequate number of necessary forms available at the service delivery point?

OTP/SFP ration cards for all beneficiaries

Home treatment card

Follow-up card

Register book

Medicine including antibiotics in the SC?

Referral /Admission

Are the referral forms available in the facility?

Is the referral facility known to the staff and the name displayed?

Guidelines for referral system available?

Is there adequate equipment available in the center?

Weighing scales

Measuring board

MUAC tapes

Salter scale

Thermometer

Beds in SC

Clean water for drinking (jug and cups)

Water and soap for hand-washing

Page 47

Annex 8A: IDI Topic Guide

In-depth Interview Guide—UNICEF & Implementing Partners (IPs)

Location ________________________ Date ___________________________

INFORMED CONSENT

My name is _________________ and I am a member of the Evaluation Team, commissioned by UNICEF to conduct evaluation of UN maternal and child stunting programme. I thank you for taking time for this interview today. I will be asking questions about how the integrated Nutrition and WASH programme interventions were designed and implemented that contributed towards the reduction in maternal and child stunting. Prior to start the interview, just wanted to emphasize that there is no right or wrong answer to any questions. We are keen to know your point of view, reflection, experiences and learning in various phases of this programme (design, inception, implementation and exit). This will help us in assessing, analysing and triangulating the results for this programme. The findings will be used to refine and scale-up the integrated Nutrition and WASH approaches at the National and International levels.

The interview usually takes 1-2 hours to complete. Whatever information you provide will be kept strictly confidential and will be dealt with anonymity. To this end, we will follow steps as: a) we will assign pseudonyms against your names and will use them to refer to you, b) no personal information will be sought from you, c) nothing will be stated in the evaluation report with your names, d) No one will be able to track you from your responses, and e) the information collected will be destroyed after a certain time. Additionally, you can refuse to participate in the discussion, leave the discussion at any time or refuse to answer any specific question. Furthermore, this information is not collected to grade anyone‘s performance or to provide incentives or grade your community/village, hence, there is no associated risk or threat or impact on you.

We will be taking notes to help us remember what is said during the discussion. If you allow we will use recorder so that we do not miss any important information. We will destroy the recording after certain time. If you have any question or concerns, even after we leave, you can ask those from the lead evaluator Dr Asmatullah on [email protected] or 03340090944. At this point, I want your permission to record the interview, take notes to avoid any loss of information and seek your consent to record the discussion. We also seek your permission to take photos to be used in the report. At this point, I want your permission to record the interview, take notes to avoid any loss of information and seek your consent to record our discussion. We also seek your permission to take photos to be used in the report. Prior to the start of interview, do you have any question about the evaluation or the interview process? Do you agree to participate in the interview? YES NO



THANK YOU

Page 48

Note about the IDI Tool: This IDI guide has been designed to ask questions from UNICEF and Implementing Partners staff, who were part of the programme design and

implementation. The guide offers flexibility to the interviewers to adapt and tweak questions to the specific role of the organization (UNICEF or IP) and the respondent‘s role in

the programme.

Key Questions Sub-Questions Responses

Relevance (To what degree the objectives & interventions of the integrated WASH-Nutrition programme were in line with needs and priorities of the target communities, government plans and international commitments‖?

1.1. To what extent the

programme was in

line to the

government and

international

priorities and

directions?

In the programme design and/or implementation, what measures did you take to list, identify and include national and international priorities and commitments

7?

Probe for:

How did you conduct the situational analysis regarding national and international commitments?

What were the priority areas that were considered in the programme design and/or implementation?

Were there some areas left out? If so what were those and why they were not considered8?

Considering the national/international priorities, how did you ensure the involvement of concerned stakeholders?

Probe for:

Who were the key stakeholders engaged in the programme (design and implementation phase) and what was their role in each phase?

Were they remained engaged to the extent as was envisaged in the programme design phase?

Were new stakeholders identified and engaged during programme implementation who were not considered in the design phase?

What were the challenges / bottlenecks in engaging them? How were those challenges addressed and mitigated?

1.2. Was the programme

based on adequate

needs assessment,

analysis of the

target communities

(particularly to cater

the needs of most

In your opinion, how relevant was the programme to the perceived and expressed needs of the target population? Probe for:

Was the programme based on adequate needs assessment & analysis of the target communities? o How did the target communities and districts were identified and mapped to undertake needs

assessment? Why certain districts and communities were selected than the others? o What tools and approaches were used to undertake situation analysis and conducting needs

7 Was the programme based on nutrition sector strategic plan and in line with local and national Health Vision 2025 & international commitments such as SDG

agenda 8 Note for Evaluation Team: Prepare the list of the areas which were not made part of the programme, if any, based on the review of the key documents and

then probe for the reasons why they were not made part of the programme

Page 49

vulnerable and

marginalized

groups) and

relevant to the

programme

objectives?

assessments? o What steps were taken to engage the relevant stakeholders and beneficiaries in conducting

situational analysis or needs assessment? o How were the findings of the need‘s assessment used in the programme design or

implementation?

What specific measures were taken to identify the most vulnerable (women and children) and marginalized groups and neglected communities?

o Probe for vulnerability criteria o Probe for marginalized groups composition o Probe for neglected communities within the programme catchment population in all three districts

At the design phase, what barriers were identified to consider in the programme design?

1.3. Was information

disaggregated

according to gender

and other relevant

social differences

(ethnicity, income

levels, rural/urban

split, DRR etc.) in

the programme

design and

delivery?

Probe for:

Was information desegregated by social differences as: o Gender (design or delivery and/or both), if yes how the info was synthesized and utilized? o Ethnicity (design or delivery and/or both), if yes how the info was synthesized and utilized? o Income level (design or delivery and/or both), if yes how the info was synthesized and utilized? o Rural/Urban Split – what was the ratio? Hoe the info was synthesized and utilized? o DRR prone areas and potential health risks including stunting and outbreaks?

What tools and mechanisms were adapted to gather disaggregated info on social differences in the programme implementation phase? What bottlenecks/barriers were identified in gathering disaggregated info on social differences in the programme implementation phase? How were those challenges mitigated?

1.4. How relevant the

target communities

perceived the

program in

addressing their

priority WASH and

Nutrition needs

including measures

to protect the rights

of access to

services in non-

discriminatory way?

How did the target communities and beneficiaries perceive the program in addressing their priority needs? Probe for:

What mechanisms were adopted to gather client feedback?

How did you ensure to provide feedback to the beneficiaries and establish a two-way dialogue between providers and clients? Also probe for:

o Establishment and functionality of nutrition, WASH and schools‘ committees o Engagements and linkages of CRPs, LHWs, CHWs and the community they serve o Mechanisms for community outreach and linkages including facilitation by the committees in

undertaking outreach activities Based on the client feedback and analysis of the programme data, what measures did you take to ensure that the programme cater the needs of the different target groups/sub-groups in non-discriminatory manner and ensure equitable access to services to all, particularly the disadvantaged?

Efficiency: Were resources (human, financial, material, technical) used in the best possible way to attain results‖?

2.1. How well aligned How were IPs mapped and selected?

Page 50

were the multiple

WASH and Nutrition

partnerships and to

what extent they

supported

programme

convergence and

integration?

Probe for:

How was the IPs capacity assessment carried out?

What was the IPs organizational set-up, niche and programme portfolio?

Have there been any concerns over the IPs capacities to deliver such integrated WASH and Nutrition programme?

To what extent these concerns (lags in IPs capacities) affected the programme delivery? What measures were taken to build the IPs capacities?

In your opinion, how efficient was the coordination and intersectoral collaboration among:

Implementing partners

Government partners

UNICEF Probe for:

What mechanisms were adopted for coordination and intersectoral collaboration?

How far the coordination and intersectoral collaboration supported or constrained the delivery of outputs and results?

To what extent intersectoral collaboration supported the convergence/integration of services? In your opinion, what were the promising approaches or interventions that resulted in mobilizing and strengthening partnership with community and government structures?

2.2. How well the

allocated resources

were managed by

partners either

directly or jointly to

ensure timely, cost

efficient attainment

of results?

How well the programme implementation was managed? Probe for:

What measures (monitoring and reporting systems and tools) were put in place to track programme delivery against set milestone and targets? How did these measures contribute to the programme management process?

Was programme operational plan developed and reviewed on periodic basis?

What measures were put in place to recruit and deploy human resources per programme need in timely manner?

What measures were put in place to develop and/or adopt materials to provide services?

What measures were put in place to maintain efficient supply-chain?

What measures were put in place to disburse funds to the partners in timely and efficient manner?

What measures were put in place to utilize funds per agreed schedule?

What measures were put in place to avoid misuse and wastage of funds by tracking them against set standards—what were those standards?

What bottlenecks and lesson learned were identified during implementation and how those were managed?

2.3. To what extent

costs incurred can

be justified by the

results achieved?

Were financial resources used appropriately to attain value for money against the benefits/outputs that programme interventions have generated? Probe for:

What factors substantiate that the associated costs were proportionate to the intended outputs/benefits?

Page 51

What factors point towards that costs were not proportionate to certain output/s? How do these factors link

to the intervention/s?

Could the same results have been achieved with fewer resources?

Above all, Evaluation Team will conduct cost-benefit analysis to answer the question in quantifiable manner


programme was

kept flexible to

adapt lessons from

implementation and

from other similar

interventions to

attain the results in

more efficient

manner?

Have perceptions of what is relevant and appropriate changed over time? If yes what were those major aspects, and how it was adapted and integrated into the programme design and delivery? Probe for:

What changes were made in the programme ToC overtime based on learning from implementation or

changes in perception? What programme elements were added or taken out and why? How often the ToC

was reviewed?

What other changes were made in the programme delivery (deviation from the original plan such as

geographical locations, workplans, alternative plans for supplies/commodities, human resource, allocation

of funds etc.)

To what extent the programme learned from other similar interventions and whether there were other alternative, more cost-effective strategies available to reach intended results?

Effectiveness: ――To what extent the programme was able to achieve the intended objectives and application of the planned strategies‖?


programme

produced the

intended outputs

and outcomes per

the ToC

How effective were the integrated Nutrition & WASH interventions in terms of coverage and inducing change, needed for stunting reduction? Probe for:

In your opinion, how successfully was the programme in reaching the target groups in provision of Nutrition and WASH services and achieving planned targets?

o What was the coverage and utilization of nutrition services? What were the most promising intervention that helped the programme in reaching intended outcomes (IYCF Counsellors, LHWs, CHWs, Mother groups, CRPs etc.)?

o To what extent the programme was successful in implementing PATS Plus approach including adaptation and maintenance of ODF Status and access to safe water by the target communities? What were the most promising intervention that helped the programme in reaching intended outcomes (CRPs, ODF committees, Steering Committees, Community Investment grants, water treatments, hands pumps, and other elements of PATS Plus approach etc.)

o To what extent, the programme was able to improve WASH services in schools and health facilities? What were the most promising interventions that helped the programme in reaching intended outcomes (elements of 3-star approach, trainings and capacity building, school clubs, ownership of the concerned line department etc.)?

What aspects of the BCC approaches were more effective in improving health & hygiene practices in the cultural and the given context? (selective entry points and means, IEC materials and channels of communications)?

3.2. How were the

programme

How effectively the programme was able to implement the envisaged integrated strategy to achieve the intended results?

Page 52

implementation

strategies

contributed in

achieving better

synergies and

complementarities

among the partners

as part of the

integrated

programming

towards stunting

reduction?

Probe for:

How was the integrated framework developed and to which extent it guided the programme implementation? Was it developed at the design phase or during implementation? Was it reviewed periodically to accommodate the changes overtime, based on the learning from implementation?

What measures were taken for joint planning, budgeting and undertaking programme reviews?

Were resources (human or financial or materials) shared as part of the integrated programming among partners?

Were joint outreach activities planned and carried out?

Were joint supportive supervision, monitoring and evaluation planned and carried out?

What were the bottlenecks in integrated programming? How they were mitigated? What lessons did you learn from implementation of integrated programming and what would you do differently in future?

3.3. To what extent has

the application of

integrated approach

and implementation

strategies worked

as intended in

influencing policies,

strategies, plans,

collaboration and

integrated

community

engagements?

Probe for:

How far the integrated programme helped in addressing silos and supported the implementation of UNICEF programme plans?

Did application of integrated approach influence changes in government plans or policies or practices? If yes what are those?

Did the application of integrated approaches result in improved and effective service delivery mechanism for health, nutrition and WASH sectors? If yes what are those?

Did the application of integrated approaches result in improved and effective community engagement and buy-in?

To what extent has the programme been able to develop capacity of programme planners and decision makers in effective nutrition planning; develop capacities of frontline workers?

To what extent the programme has been able to contribute to ownership and leadership of the provincial departments for achieving open defecation free environment? What is the duty bearers‘ level of buy-in of the application of integrated programme approaches?

3.4. To what extent

gender was

mainstreamed in

the programme

delivery and how

well the identified

barriers and

bottlenecks were

mitigated?

Probe for:

Was gender awareness incorporated in the trainings conducted for IPs staff and management?

Were men and women provided with opportunities to participate in project activities? How it was done?

What were some of challenges or constraints in achieving the desired level of gender dimension integration? What did you do to mitigate them?

Learning from this experience, what would you do differently in future programming?

3.5. What type of

approaches and

interventions were

How did the programme target and reach the disadvantaged and marginalized groups and hard to reach areas?

Page 53

implemented and

were effective to

yield results in

reducing stunting in

disadvantaged,

marginalized and

less reachable

areas/groups?

Probe for:

How were these vulnerable groups and hard to reach areas mapped?

What approaches, and interventions were adapted to reach them out?

How did you ensure to engage them in the programme delivery?

Was a feedback mechanism put in place to gather their feedback on the provision of services?

In your opinion what are the key achievements that in proxy indicate the contribution of the programme to reduce stunting in the disadvantaged and marginalized groups?

What were the key barriers and bottlenecks in reaching them out? How were those mitigated?

3.6. To what degree

human rights-based

approaches were

mainstreamed in

the programme

design and

delivery?

Probe for

Was staff trained on the rights-based approach?

Were beneficiaries and communities mobilized and sensitized on their health rights and entitlements (access to and availability of health, nutrition and WASH services)?

What were some of challenges or constraints in mainstreaming HRBA in the programme? How did you mitigate them?

3.7. How was DRR

incorporated into

the programme

design and

mainstreamed in

programme

implementation?

Probe for:

Was hazard mapping and vulnerability assessment carried out in the targeted locations? What risks were identified to be made part of the programme delivery?

What DRR or protection measures were applied to address the identified risks? o How the capacities of service providers, district health system and PHE&RDD were strengthened

to gather data and monitor trends/thresholds for diseases under surveillance (diarrhoea, ARI and other infectious diseases)?

o Were they able to interpret such data to know the potential health risks and to use such information for response planning to avert outbreaks?

o In your opinion, how well the communities are prepared to avoid potential health risks based on the programme contribution (construction of latrines, use of safe drinking water, ODF, positive behaviours etc.)?

What were the key challenges or constraints in achieving the desired level of integration? How did you mitigate them?

Sustainability: ―Are the positive effects of the programme sustainable‖?

4.1. What are the key

factors and drivers

(internal & external)

that contribute to or

constrain the

continuity of the

programme effects

To what extent did the programme contribute in strengthening local ownership and leadership? Probe for:

How did the programme contribute in building local level leadership (LGD Reps and district administration and district line departments)?

Do you think inter-sectoral collaboration will continue after programme comes to an end?

What measures the local leaderships have taken to institutionalize the programme key practices to support the implementation of the integrated approach beyond the project (notifications of committees, adaptation of 3-star approach, continuation of any interventions such as uptake of capacity building modules, Working with LHWs etc.)?

Page 54

and interventions

after the

intervention period?

Was a reasonable exit strategy developed?

Probe for: Schedule and guidelines for the transfer of responsibility and activities to government departments?

What level of resources the government has allocated for integrated Nutrition and WASH programming? Probe for:

Were the partners abled to motivate concerned government departments to allocate funds for continuation of this programme?

Were changes made in practices and policies within the government such as PC1, AAA framework? Are the allocated financial (financial and human) resources adequate?

What role the programme played in developing capacities of the providers, duty bearers and stakeholders? Is it adequate and will it sustain beyond the project life? Probe for:

Adaptation of training packages Pool of Master trainers

Is government better prepared to address Nutrition and WASH related diseases in target areas? If yes, how can this be attributed to the programme interventions?

4.2. What aspects of the

programme

activities and the

induced behaviors

would likely to

continue and last

beyond the project

life?

What aspects of the programme have the probability to continue and last beyond UNICEF and IPs presence? Probe for:

government institutional support and ownership and/or expansion

maintaining status of ODF villages

maintaining improved health, hygiene and sanitation practices improved access to and availability of nutrition care and service delivery

What you think are the critical enablers or barriers for sustainability? Explain why? Probe for:

systems and structures, capacities and behavioural changes that this programme has been generated

Impact: ―To what degree the programme contributed in achieving the intended long-term outcomes and impact of the programme‖?

5.1. Has the programme

contributed in

adaptation of

In which ways programme was able to bring changes in people‘s lives (with focus on adaptation of healthy behaviours)? Probe for:

Page 55

improved health,

nutrition, sanitation

and hygiene

practices and

behaviors?

To what extent the targeted communities adapted improved health, nutrition, hygiene and sanitation practices (planned vs set targets)?

How did the target communities and beneficiaries perceive the programme in addressing their priority needs? Also Probe for examples of beneficiary‘s statements and documented case studies, if any

Which factors contributed positively in adaptation of healthy behaviours and creating lasting changes?

In your opinion, was the programme able to cater the needs of the most vulnerable (women and Children) and marginalized groups? If yes, why do you think so?

Has the programme resulted in uptake of improved adaption and practices about the intended and planned level of change in Knowledge, Attitude and Practice (health, nutrition, hygiene and sanitation) as compared to the Pre-KAP/Post KAP? How far the programme has been able to increase awareness and inculcate practice among pregnant and lactating women on key behavioural changes such as Infant and Young Child Feeding (IYCF) practices, exclusive breastfeeding, use of Iron Folic Acid (IFA), multi-micronutrient powder (MNP) for children and multi-micronutrient tablets for women, use of better foods and cooking methods and key hygiene behaviours such as hand washing and giving up open defecation by using latrines?

Has anticipated impact varied for different target groups (men, women, children, older people, poor, marginalized etc.) or for programme impact areas and if so, why?


programme

contributed in

creating an

enabling

environment for

improved Nutrition

and WASH service?

What were the major factors influencing the achievement or non-achievement of planned outcomes? Probe for:

Which factors (within the project or outside the project) resulted changes in practices, polices and enabling environment (institutional levels) to support the services delivery? Were those changes intended or unintended?

In your opinion, was nutrition services uptake improved? If yes, what were the key factors that contributed in improved services uptake? Would the health system sustain these efforts in future?

In your opinion, did the targeted communities attain the status of ODF? If yes what were the key factors that contributed in its attainment? Would the certified ODF communities sustain the status? Has the programme gathered and analysed the diarrhoea and ARI data for comparable periods? If yes what was the disease trend over those comparable periods? Was the incidence reduced for these diseases?

5.3. Have any

unintended,

unexpected or

negative effects

been observed

during programme

implementation as

result of this

programme?

Probe for:

Have any unintended, unexpected or negative effects been observed during programme implementation as result of this programme? If yes, what are they and which factors aggravated these negative results? What measures were taken to mitigate the negative effects?

Page 56

In-depth Interview Guide—Provincial Level Line Departments Officials

Location ________________________ Date ___________________________

INFORMED CONSENT






THANK YOU

Page 57

Note about the IDI Tool: This IDI guide has been designed to ask questions from Provincial Line Departments Officials and Counterparts, who were part of the programme

design and implementation were. The guide offers flexibility to the interviewers to adapt and tweak questions to the specific role of the organization (Health, PHE&RDD,

Education, LHW Programme, Social Welfare etc.) and the respondent‘s role and profile in the programme.


Relevance (To what degree the objectives & interventions of the integrated WASH-Nutrition programme were in line with needs and priorities of the target communities, government plans and international commitments‖?)


programme was in line to

the government and

international priorities

and directions?

In your opinion the UN maternal and child stunting reduction programme design was based on the provincial and national needs and priorities? Probe for

What is the priority WASH and Nutrition needs in your province?

Were the priority needs made part of the integrated WASH and Nutrition programme?

How familiar are you with integrated programme approaches?

Were these approaches relevant to address the WASH and Nutrition needs in your province?

To what degree do you think that you were engaged in shaping the programme?

Probe:

Were you engaged in design and implementation phase and exit phase?

If yes what was their role and how they were engaged?

Did the engagement continued throughout the programme cycle?


based on adequate needs

assessment, analysis of

the target communities

(particularly to cater the

needs of most vulnerable

and marginalized groups)

and relevant to the

programme objectives?

In your opinion, how relevant was the programme to the perceived and expressed needs of the target population? Probe For:

How do you think the target districts and communities were selected for the programme?

Were you engaged in selecting the target districts (situational analysis or needs assessment) for the programme?

What specific measures were taken to identify the most vulnerable (women and children) and marginalized groups and neglected communities

Efficiency: Were resources (human, financial, material, technical) used in the best possible way to attain results?

2.1. How well aligned were the

multiple WASH and

Nutrition partnerships

and to what extent they

supported programme

convergence and

integration?

In your opinion, how efficient was the coordination and inter-sectoral collaboration among:

Implementing partners

Government partners

UNICEF Probe for:

What mechanisms were adopted for coordination and inter sectoral collaboration at the provincial levels?

Page 58

How far the coordination and inter sectoral collaboration supported or constrained the delivery of outputs and results?

To what extent inter sectoral collaboration supported the convergence/integration of services?

How successful was the provincial steering committee and TAG about overseeing the programme?

2.2. How well the allocated

resources were managed

by partners either directly

or jointly to ensure

timely, cost efficient

attainment of results?

How well do you think the programme implementation was managed? Probe for:

Were the resources (financial and human resources) adequate for programme delivery?

Are you aware of any delays /bottlenecks in the programme implementation?

Effectiveness: ―To what extent the programme was able to achieve the intended objectives and application of the planned strategies‖?


programme produced the

intended outputs and

outcomes per programme

Theory of Change?



To what extent, the programme was able to improve Nutrition and WASH services in the target districts/communities

3.2. How were the programme

implementation strategies

contributed in achieving

better synergies and

complementarities among

the partners as part of the

integrated programming

towards stunting

reduction?

How effectively the programme was able to implement the envisaged integrated strategy to achieve the intended results? Probe for:

Do you think that the programme was able to build the complementarities and synergies among the partners and provincial departments?

Are you aware of mechanism to promote these complementarities at the provincial level (steering committee, joint reporting, dissemination workshops, lobby meetings etc.)?

Were joint outreach activities planned and carried out?

Do you have any specific learning/recommendations to implement the integrated programming in future?

3.3. To what extent has the

application of integrated

approach and

implementation strategies

worked as intended in


strategies, plans,

Probe for:

Did application of integrated approach influence changes in government plans or policies or practices? If yes what are those?

Did the application of integrated approaches result in improved and effective service delivery mechanism for health, nutrition and WASH sectors? If yes what are those?

To what extent has the programme been able to develop capacity of programme planners and decision makers in effective nutrition planning; develop capacities of frontline workers?

To what extent the programme has been able to contribute to ownership and leadership of the

Page 59

collaboration and

integrated community

engagements?

provincial departments for achieving open defecation free environment?

What is the duty bearers‘ level of buy-in of the application of integrated programme approaches?

3.4. How was DRR

incorporated into the

programme design and

mainstreamed in

programme

implementation?

Probe for: How was DRR incorporated into the programme design and mainstreamed in programme implementation? Probe for

How the capacities of service providers, district health system and line departments were strengthened to gather data and monitor trends/thresholds for diseases under surveillance (diarrhoea, ARI and other infectious diseases)?

Were they able to interpret such data to know the potential health risks and to use such information for response planning to avert outbreaks


4.1. What are the key factors

and drivers (internal &

external) that contribute

to constrain the

continuity of the

programme effects and

interventions?

To what extent, did the programme contribute in strengthening duty bearer‘s ownership and leadership? Probe for:

How did the programme contribute in building Provincial level leadership including all line departments (PHE&RDD, LGD, etc.)?

Do you think inter-sectoral collaboration will continue after programme comes to an end?

What measures have taken to institutionalize the programme key practices beyond the project (NMIS, WASH MIS, Sindh policy etc.?

Were you engaged for the exit strategy of the programme? Probe For:

Do you think you will be able to continue the responsibility even after the programme ends?

What level of resources the government has allocated for integrated Nutrition and WASH programming?

Probe for:

Were there any changes made in practices and policies within the government such as PC1, AAA framework?

Are the allocated financial (financial and human) resources adequate?


programme activities and

the induced behaviours

would likely to continue

and last beyond the

project life?

What aspects of the programme have the probability to continue and last beyond UNICEF and IPs presence? Probe for:

Is government better prepared to address Nutrition and WASH related diseases in target

areas? If yes, how can this be attributed to the programme interventions?

Page 60

Government institutional support and ownership and/or expansion for improved access to and

availability of nutrition care and service delivery such as AA framework, MIS, NMIS Sindh

policies, training materials, BCC etc.



Impact: ―To what degree the programme contributed in achieving the intended long-term outcomes and impact of the program‖?


programme contributed

in creating an enabling

environment for improved

Nutrition and WASH

service delivery?


Which factors (within the project or outside the project) resulted changes in practices, polices and enabling environment (institutional levels) to support the services delivery?

In your opinion, was nutrition services uptake improved? If yes, what were the key factors that contributed in improved services uptake? Would the health system sustain these efforts in future?

In your opinion, did the targeted communities attain the status of ODF? If yes what were the key factors that contributed in its attainment?

In-depth Interview Guide—DCO, DHO, DEO, District PHE&RDD Officer, LHW Coordinator, Social Welfare Department

Location ________________________ Date ___________________________

INFORMED CONSENT


The interview usually takes 1-2 hours to complete. Whatever information you provide will be kept strictly confidential and will be dealt with anonymity. To this end, we will follow

Page 61

steps as: a) we will assign pseudonyms against your names and will use them to refer to you, b) no personal information will be sought from you, c) nothing will be stated in the evaluation report with your names, d) no one will be able to track you from your responses, and e) the information collected will be destroyed after a certain time. Additionally, you can refuse to participate in the discussion, leave the discussion at any time or refuse to answer any specific question. Furthermore, this information is not collected to grade anyone‘s performance or to provide incentives or grade your community/village, hence, there is no associated risk or threat or impact on you.




THANK YOU

Note about the IDI Tool: This IDI guide has been designed to ask questions from District Officials (DCO, EDO, DHO, PHE&RDD Officer, LHW Coordinator, Social Welfare)

UNICEF and Implementing Partners staff, who were part of the program design and implementation. The guide offers flexibility to the interviewers to adapt and tweak

questions to the specific role of the line department and the respondent‘s role and profile in the program.


Relevance (To what degree the objectives & interventions of the integrated WASH-Nutrition program were in line with needs and priorities of the target communities, government plans and international commitments‖?)


program was in line to

the government and


and directions?

In your opinion the UN maternal and child stunting reduction program design was based on the district and provincial needs and priorities? Probe for

What is the priority WASH and Nutrition needs in your district?

Were the priority needs made part of the integrated WASH and Nutrition program?

How familiar are you with integrated program approaches?

Were these approaches relevant to address the WASH and Nutrition needs in your district?

To what degree do you think that you were engaged in shaping/adapting the programme approaches for your district? Probe for:

Were you engaged in implementation and exit phases of program?

Page 62

What was your level of engagement?

How often were you consulted by the program? implementers and the beneficiaries?

Were you provided with an opportunity to provide your feedback with regard to program implementation?


based on adequate

needs assessment,

analysis of the target

communities


the needs of most

vulnerable and

marginalized groups)

and relevant to the

program objectives?

In your opinion, how relevant was the program to the perceived and expressed needs of the target population in your districts? Probe For:

How do you think the target communities/UCs were selected for the program?

Were you consulted in selecting the target UCs/Schools/ HFs (situational analysis or needs assessment) for the programme?

What specific measures were taken to identify the most vulnerable (women and children) and marginalized groups and neglected communities in your district?

Were the neglected communities within your district included in program catchment population?

1.3. How relevant the

target communities

perceived the

program in

addressing their

priority WASH and

Nutrition needs

including measures to

protect the rights of

access to services in

non-discriminatory

way?

How did the target communities and beneficiaries perceive the program in addressing their priority needs? Probe for:

Are you familiar with any mechanisms adopted to gather client/ beneficiary feedback and feedback including establishing a two-way dialogue between providers and clients?

Are you familiar with the establishment of the two-way dialogue between provider and clients such as, WASH and schools‘ committees, Engagements and linkages of CRPs, LHWs, CHWs, LGD rep and the community they serve, school wash clubs etc.

Based on the client feedback and analysis of the program data, do you know what measures were taken to ensure that the program cater the needs of the different target groups/sub-groups in non-discriminatory manner and ensure equitable access to services to all, particularly the disadvantaged?


2.1. How well aligned were

the multiple WASH

and Nutrition

partnerships and to

what extent they

supported

programme

convergence and

integration?

In your opinion, how efficient was the coordination and inter-sectoral collaboration among Implementing partners District Administration and District Line departments? Probe for:

What mechanisms were adopted for coordination and inter-sectoral collaboration?

To what extent inter-sectoral collaboration supported the delivery of results and the convergence/integration of services?

In your opinion, what were the promising approaches or interventions implemented in the district that resulted in mobilizing and strengthening partnership with community and government structures/District administration?

Page 63

Probe for

What were different programme approaches in terms of monitoring the programme implementation such as District and Taluqa steering committed, Village WASH committee, coordination with the district Line departments etc.)?

2.2. How well the

allocated resources

were managed by

partners either

directly or jointly to

ensure timely, cost

efficient attainment of

results?

How well do you think the Programme implementation was managed? Probe for:

Were the resources (financial and human resources) adequate for programme delivery?

Are you aware of any delays in the Programme implementation

Effectiveness: ―To what extent the program was able to achieve the intended objectives and application of the planned strategies‖?


Programme

produced the

intended outputs and

outcomes per

program Theory of

Change?


In your opinion, how successful was the Programme in reaching the target groups with Nutrition and WASH services?

o Do you think that the coverage and utilization of nutrition services in your district improved? o Did the adaptation and maintenance of ODF status and access to safe water supply in your

district improved? o To what extent, the program was able to improve WASH services in schools and health

facilities?

3.2. How were the

Programme

implementation

strategies contributed

in achieving better

synergies and

complementarities

among the partners

as part of the

integrated

programming towards

stunting reduction?

How effectively the Programme was able to implement the envisaged integrated strategy to achieve the intended results? Probe for:

Are you aware of any mechanism established to promote the complementarities between the implementing partners and Government line departments at district and UC level?

Probe for steering committee, joint monitoring visits, integrated planning, engaging all line departments etc.

Which in your opinion was most promising interventions I delivery of project to improve nutrition and WASH services in your district?

What were the bottlenecks/challenges in integrated programming?

What would you recommend being done differently in future for designing and implementing integrated Programme s

To what extent the application of integrated approach has created an enabling environment needed for stunting reduction?

Page 64

Probe for: Did application of integrated approach influence any practice changes in your district (such as notification of steering committee, adaptation of 3-star approach etc.)?

3.3. To what extent has

the application of

integrated approach

and implementation

strategies worked as

intended in


strategies, plans,

collaboration and

integrated community

engagements?

Probe for:

Did application of integrated approach influence changes in government plans or policies or practices?

If yes what are those?

Did the application of integrated approaches result in improved and effective service delivery

mechanism for health, nutrition and WASH sectors? If yes what are those?

Did the application of integrated approaches result in improved and effective community engagement

and buy-in?

To what extent has the programme been able to develop capacity of programme planners and decision

makers in effective nutrition planning; develop capacities of frontline workers?

To what extent the programme has been able to contribute to ownership and leadership of the district

departments for achieving open defecation free environment?

What is the duty bearers‘ level of buy-in of the application of integrated Programme approaches?


How did the program target and reach the disadvantaged and marginalized groups and hard to reach areas? Probe for:



How did the Programme ensure to engage them in the Programme delivery?



3.5. How was DRR

incorporated into the

programme design

and mainstreamed in

programme

implementation?

How was DRR incorporated into the programme design and mainstreamed in programme implementation? Probe for

How the capacities of service providers, district health system and line departments were strengthened to gather data and monitor trends/thresholds for diseases under surveillance (diarrhoea, ARI and other infectious diseases)?

Were they able to interpret such data to know the potential health risks and to use such information for response planning to avert outbreaks

Sustainability: ―Are the positive effects of the Programme sustainable‖?

4.1. What are the key

factors and drivers

To what extent did the Programme contribute in strengthening District and UC level ownership and leadership?

Page 65

(internal & external)

that contribute to

constrain the

continuity of the

Programme effects

and interventions?

Probe for:

How did the Programme contribute in building local level leadership (LGD Reps and district administration and district line departments?

Do you think inter-sectoral collaboration will continue after Programme comes to an end?

What measures the local leaderships have taken to institutionalize the Programme key practices to support the implementation of the integrated approach beyond the project (notifications of committees, adaptation of 3-star approach, continuation of any interventions such as uptake of capacity building modules, Working with LHWs etc.)?

Were you engaged for the exit strategy of the programme? Probe For: Do you think you will be able to continue the responsibility even after the programme ends?

What level of resources the government has allocated for integrated Nutrition and WASH programming? Probe for:

Were there any changes made in practices and policies within the government such as PC1, AAA framework, notification of committees, adaptation of 3-start approach, LHW manuals etc.)

Do you think the district administration is better prepared to address Nutrition and WASH related diseases in target areas? If yes, how can this be attributed to the programme interventions? Probe for:

Adaptation of training packages Pool of Master trainers?

Were capacity of members of ODF Committee, Steering committee, WASH Committee, School Clubs etc developed?


Programme activities

and the induced

behaviours would

likely to continue and

last beyond the

project life?

What aspects of the program have the probability to continue and last beyond UNICEF and IPs presence? Probe for:

government institutional support and ownership and/or expansion

maintaining status of ODF villages

maintaining improved health, hygiene and sanitation practices improved access to and availability of nutrition care and service delivery



Impact: ―To what degree the Programme contributed in achieving the intended long-term outcomes and impact of the program‖?

5.1. Has the Programme

contributed in

adaptation of

improved health,

In which ways Programme was able to bring changes in people‘s lives (with focus on adaptation of healthy behaviours)? Probe for:

To what extent the targeted communities adapted improved health, nutrition, hygiene and sanitation

Page 66

nutrition, sanitation

and hygiene practices

and behaviours?

practices?

How did beneficiaries perceive the Programme in addressing their priority needs? Also Probe for examples of beneficiary‘s statements and documented case studies, if any?

Was the programme able to cater the needs of the most vulnerable (women and Children) and marginalized groups? If yes, why do you think so?


Programme

contributed in

creating an enabling

environment for

improved Nutrition

and WASH service

delivery?


Which factors (within the project or outside the project) resulted changes in practices, polices and enabling environment (institutional levels) to support the services delivery?

In your opinion, was nutrition services uptake improved? Would the health system sustain these efforts in future?

In your opinion, did the targeted communities attain the status of ODF? If yes what were the key factors that contributed in its attainment?

Did the School and health facility WASH and safe water supply improved? Will it sustain?

In-depth Interview Guide-LGD Reps (Local Government Representatives)

Location ________________________ Date ___________________________

INFORMED CONSENT



We will be taking notes to help us remember what is said during the discussion. If you allow we will use recorder so that we do not miss any important information. We will destroy the recording after certain time. If you have any question or concerns, even after we leave, you can ask those from the lead evaluator Dr Asmatullah on [email protected]

Page 67

or 03340090944. At this point, I want your permission to record the interview, take notes to avoid any loss of information and seek your consent to record the discussion. We also seek your permission to take photos to be used in the report. At this point, I want your permission to record the interview, take notes to avoid any loss of information and seek your consent to record our discussion. We also seek your permission to take photos to be used in the report. Prior to the start of interview, do you have any question about the evaluation or the interview process? Do you agree to participate in the interview? YES NO



THANK YOU

Note about the IDI Tool: This IDI guide has been designed to ask questions from Local Government Representatives, who were engaged in the programme design and

implementation. The guide offers flexibility to the interviewers to adapt and tweak questions to the specific role of the LGD Reps (Local Government Representatives) and the

respondent‘s role and profile in the programme.


Relevance (To what degree the objectives & interventions of the integrated WASH-Nutrition programme were in line with needs and priorities of the target communities, government plans and international commitments‖?)


programme was in line

to the government and


and directions?

To what degree were you engaged during the programme implementation? Probe for:

Were you provided with an opportunity to provide feedback with regard to programme implementation and planning for exit?


based on adequate

needs assessment,

analysis of the target

communities


the needs of most

vulnerable and

marginalized groups)

and relevant to the

programme

objectives?

In your opinion, did the programme targeted the most vulnerable / neglected communities in your district? Probe for:

Are you familiar with any specific approaches that the programme used to target the most vulnerable population?

Are you familiar about the criteria to identify marginalized and vulnerable communities

1.3. How relevant the target How did the target communities and beneficiaries perceive the programme in addressing their priority

Page 68

communities

perceived the

programme in

addressing their

priority WASH and

Nutrition needs

including measures to

protect the rights of

access to services in

non-discriminatory

way?

needs? Probe for:

Are you familiar with any mechanisms adopted to gather client / beneficiary feedback including

establishing a two-way dialogue between providers and clients? For example: WASH and schools‘

committees, Engagements and linkages of CRPs, LHWs, CHWs and the community they serve


2.1. How well the allocated

resources were

managed by partners

either directly or jointly

to ensure timely, cost

efficient attainment of

results?

In your opinion, were the programme activities implemented in a timely manner? Are you aware of any delays?



programme produced

the intended outputs

and outcomes per

programme Theory of

Change?



To what extent, the programme was able to improve Nutrition and WASH services in the target districts/communities


How did the programme target and reach the disadvantaged and marginalized groups and hard to reach areas? Probe for:



How did the programme ensure to engage them in the programme delivery?



Page 69


4.1. What are the key factors and drivers (internal & external) that contribute to constrain the continuity of the programme effects and interventions?

To what extent did the programme contribute in strengthening District and UC level ownership and

leadership?

Probe for:

How did the programme contribute in LGD reps Capacity building local?

Adaptation of training packages Pool of Master trainers

Do you think as LGD rep you will continue to work for the cause after the programme ends?

Impact: ―To what degree the programme contributed in achieving the intended long-term outcomes and impact of the programme‖?

5.1. Has the program contributed in adaptation of improved health, nutrition, sanitation and hygiene practices and behaviors?

In which ways program was able to bring changes in people‘s lives (with focus on adaptation of healthy behaviors)?

Probe for:

To what extent the targeted communities adapted improved health, nutrition, hygiene and sanitation practices

How did beneficiaries perceive the programme in addressing their priority needs? Also Probe for examples of beneficiary‘s statements and documented case studies, if any

Was the programme able to cater the needs of the most vulnerable (women and Children) and marginalized groups? If yes, why do you think so?

In-depth Interview Guide-USAID

Location ________________________ Date ___________________________

INFORMED CONSENT


The interview usually takes 1-2 hours to complete. Whatever information you provide will be kept strictly confidential and will be dealt with anonymity. To this end, we will follow steps as: a) we will assign pseudonyms against your names and will use them to refer to you, b) no personal information will be sought from you, c) nothing will be stated in the evaluation report with your names, d) No one will be able to track you from your responses, and e) the information collected will be destroyed after a certain time. Additionally, you

Page 70

can refuse to participate in the discussion, leave the discussion at any time or refuse to answer any specific question. Furthermore, this information is not collected to grade anyone‘s performance or to provide incentives or grade your community/village, hence, there is no associated risk or threat or impact on you.




THANK YOU

Note about the IDI Tool: This IDI guide has been designed to ask questions from USAID staff who oversee the programme design and implementation. The guide offers

flexibility to the interviewers to adapt and tweak questions to the specific role of the respondent‘s role and profile in the programme.


Relevance (To what degree the objectives & interventions of the integrated WASH-Nutrition program were in line with needs and priorities of the target communities, government plans and international commitments‖?)

To what extent the programme was in line to the government and international priorities and directions? Probe for:

Was the programme based on nutrition sector strategic plan and in line with local and national Health Vision 2025 &

international commitments such as SDG agenda

Was the programme based on adequate needs assessment and analysis of the target communities, particularly to cater the

needs of most vulnerable and marginalized groups?


To what extent the programme produced the intended outputs and outcomes per programme Theory of Change? Probe for:

How successful was the integrated Nutrition & WASH interventions in terms of coverage and inducing change, needed for stunting reduction?

How well the allocated resources were managed to ensure timely, cost efficient attainment of results? o Operational plans, Programme Critical review, Monitoring System, Adaptive management, Flexible Budgeting o Were financial resources used appropriately to attain value for money against the benefits/outputs that programme

Page 71

interventions have generated? o What factors substantiate that the associated costs were proportionate to the intended outputs/benefits? o What factors point towards that costs were not proportionate to certain output/s? How do these factors link to the

intervention/s? o Could the same results have been achieved with fewer resources?


How were the programme implementation strategies contributed in achieving better synergies and complementarities among the partners as part of the integrated programming towards stunting reduction? Probe for:

In your opinion, how efficient was the coordination and intersectoral collaboration among: o UNICEF and Implementing partners o Government partners o USAID funded other health projects (MCHIP, Health System Strengthening etc.) o WFP

How effectively the programme was able to implement the envisaged integrated strategy to achieve the intended results?

Sustainability: ―Are the positive effects of the programme sustainable‖? and Impact: ―To what degree the programme contributed in achieving the intended long-term outcomes and impact of the programme‖?

What are the key factors and drivers (internal & external) that contribute to or constrain the continuity of the programme effects and interventions? Probe for:

To what extent did the programme contribute in strengthening duty bearer‘s ownership and leadership?

What level of resources the government has allocated for integrated Nutrition and WASH programming (such as AA framework,

PC1, NMIS, WASH MIS, notifications of committees, Sindh Sanitation Policy etc.)?

What you think are the critical enablers or barriers for sustainability? Explain why?

Have any unintended, unexpected or negative effects been observed during programme implementation as result of this

programme? If yes, what are they and which factors aggravated these negative results?

Page 72

Annex 9: List of Persons Interviewed and Area

Visited

This Annex includes the following

List of Key Informant Interviewed at the National, Provincial and District Level

List of focused Group Discussions and areas visited

Key Informant Interviews

S.No Name Organization Level

1 Dr Wisal UNICEF National

2 Dr Saba UNICEF National

3 Theo UNICEF National

4 Mubasshra Irum UNICEF National

5 Asim UNICEF(PMER) National

6 Asim Plan International National

7 Umer DCD ACF National

8 Mohammad Ali Senior MEAL ACF National

9 Sidra USAID National

10 Dr Umer UNICEF Provincial

11 Lodhi SAFCOW Provincial

12 Dr Sahib Jan- Program Corrdinator AAP- H Provincial

13 Sayfoor Training Coordinator AAP H Provincial

14 Mr Imran MIS Coordinator AAP H Provincial

15 Shoukat AAP Secretariat Provincial

16 Fahim Junejo Local government Provincial

17 Dr Paras NSP Provincial

18 Dr Zahoor Baloch PD program/X NSP PM Provincial

19 Dr Umer UNICEF Provincial

20 Barkat Ali HANDS Provincial

21 Maqbool Rung HANDS Provincial

22 Dr Sarwal Mirza HANDS Provincial

23 Habibullah Mustafa HANDS Provincial

24 Dr DS Akram HELP Provincial

Page 73

25 Dr Yasmeen HELP Provincial

26 Mr Raheel CEO White Rice Provincial

27 Ms Sara Team lead Research Provincial

28 Zahid HANDS District

29 Haq Nawaz Haqnawakalro ACF Program Manager ghotki +Khairpur)

District

30 Samina NRSP District

31 Nasir Baloch PHED District

32 Dr Dur Mohammad Memom MS RHC Peer ghot district Khairpur District

33 Abdul Qadir Nutrition supervisor OTP RHC Peerghot Kahirpur

District

34 Dr Nawabdin Mahar Nutrition focal person District Kahirpur(X ACF)

District

35 Dr Barkat Ali SMO RHC Peerghot Khairpur District

36 Dr Saleem Somor SMO, DHQ/Teaching Hospital Khairpur

District

37 Dr Fayaz Ali SMO, DHQ/Teaching Hospital Khairpur

District

38 Rizwan ahmed Staff Nurse ,DHQ/Teaching Hospital Khairpur

District

39 Dr Altaf Ahmad Somro DHO Ghotki District

40 Dr Abud Razaq AAP Focal Person Ghotki District

41 Dr Imdad Lashri MS DHQ District

42 Najema Laghari ADC (Additional district coordinator for Sindh) LHW Program

District

43 Mohammad Khalid Saleem ADCC-2Ghotki District

44 Ms Shahana Shah Stunting Coordinator District

45 Taj Mohammad Qureshi PHED Ghotki District

46 Soubal Bhutto LHV , Nutrition Stablization Centre DHQt Noushero Feroze

District

47 Azam Ali Bhutto Staff Nurse, Nutrition Stablization Centre DHQt Noushero Feroze

District

48 M Iqbal Ansari Medical Suprintendant DHQ Noushero Feroze

District

49 Liaqat Ali Khshkheli DEO Primary Education District Khairpur

District

Focussed Group discussions

Page 74

S. No Focused Group Discussions

Village Taluka UC District

1 Mothers Tajdin Arain Kingri Peer badal Khairpur

2 Fathers Mahesar Kingri Peer badal Khairpur

3 LHW/CHW/Couple workers

Khawaja Kingri Peerbadal Khairpur

4 Masons Wada mahesar Kingri Peerbadal Khairpur

5 Entrepreneurs Khawaja Kingri Peerbadal Khairpur

6 School children Khaleefa Mohammad Ramzan

Kingri Peer badal Khairpur

7 School children Tajdin Arain Kingri Peer badal Khairpur

8 School teachers Tajdin Arain Kingri Peerbadal Khairpur

9 School teachers Khallefa Mohammad Ramzan

Kingri Peerbadal Khairpur

10 CRP Khuwaja Kingri Peerbadal Khairpur

11 Village WASH committee

Khuwaja Kingri Peerbadal Khairpur

12 Mothers Deenpur Mehrabpur Halani Nousheroferoze

13 Mothers Saifal Khan Mangrio

Mehrabpur Beelani Nousheroferoze

14 Fathers Saiful mangrio Mehrabpur Beelani Nousheroferoze

15 Fathers Pirpur Mohhala Kandyaroo Halani Nousheroferoze

16 Fathers Soomar Khaskheli Mehrabpur Behlani Nousheroferoze

17 Masons Soomar Kaskheli Mehrabpur Behlani Nousheroferoze

18 CRP Moahammad Ramzan Haleepoto

Mehrabpur Behlani Nousheroferoze


Sono Khan Mehrabpur Behlani Nousheroferoze


Unar Village Mehrabpur Behlani Nousheroferoze

21 Fathers Gul muhammad Malik

Obauro Wasti Jeevan Shah

Ghotki

22 Fathers Gharangin Obauro Obauro Ghotki


Magri Kot Obauro Wasti Jeevan Shah

Ghotki

24 Masons Obauro City Obauro Tigh Ghotki

25 Entrepreneurs Obauro City Obauro Tigh Ghotki

Page 75

26 CRP Gul Mohammad Malik


Ghotki


Qaim din Kalhoro Obauro Wasti Jeevan Shah

Ghotki

28 School children GBPS Tigh Obauro Wasti Jeevan Shah

Ghotki

29 School teachers GBPS Tigh Obauro Wasti Jeevan Shah

Ghotki

30 Mothers Gul muhammad Malik


Ghotki

31 Mothers Gharangin Meerpur Obauro Ghotki

Annex 10A: Ethical Review Approval

This is annexed in Separate

Annex 10: Analytical Approaches and Quality

Assurance

Analytical Approaches & Quality Assurance The evaluation assessed the DAC criteria of relevance, efficiency, effectiveness, impact and sustainability and non-DAC cross-cutting themes of stakeholder participation, equity, HRBA, gender and DRR. The key elements of evaluation assessment included: programme design, programme reports, ToC, result framework, PMP, IAP and data used for monitoring performance and attainment of the results. Additionally, the evaluators used qualitative and quantitative data including scoping literature review to draw inferences against the DAC and non-DAC criteria. In order to operationalize the evaluation, the evaluators developed an evaluation matrix (section C.1 and annex 5) that served as a guide to gather and analyse the data/information to answer the evaluation questions. The evaluation matrix specified indicators, data collection methods and sources of information to answer the questions under each criterion. The key aspects of the analytical approaches and quality assurance included:

Analysis of Quantitative Pre and Post KAP Surveys: As discussed in C.2.2 above, the evaluators analysed the pre and post KAP survey to assess change over time between base and end line programme estimates for key indicators. The pre and post KAP results are presented in the finding sections of effectiveness, impact and sustainability of this report. Management and Analysis of Qualitative Data: The evaluation gathered a large set of qualitative data as discussed in C.2.7 section above. The evaluators adopted a mixed of the approaches to manage this large dataset, using manual framework and qualitative data analysis software (QDA Minor Lite) to tabulate and summarized the data. The process of data management and analysis included: a) transcribed recorded statements, b) applied deductive codes for organizing dataset under themes and coding the transcription9, c) analysed and interpreted qualitative data, using group than individuals so that to analyse the data in disruptive manner 10, and d) held debrief sessions with survey team to address challenges and assess whether new ideas are still emerging or if saturation has been reached on the key topics, as discussed in section C.2.7 Programme Data and Secondary Sources of Information: The evaluators also reviewed and analysed programme on-going dataset to objectively verify the programme results against the PMP and indicators. The programme data

9 Codes are ‗labels‘ that summarize or bookmark short fragments of text, and therefore help to sort and structure the data 10 For example, ―the consensus achieved by the group was …‖, ―the majority of participants agreed that …‖, ―there were several contradictory opinions about …‖, ―almost no one mentioned ...‖ Similarly, the analysis identified patterns and themes, map a problem, identified differences and similarities within the data, and making comparisons between different groups involved in the topic.

Page 76

analysis is presented in effectiveness section of the report against the intended programme outputs and outcome objectives. Similarly, the evaluators also gathered and review secondary sources of information, notably PDHS 2017-18, PDHS 2013, NNS 2011, and MICS 2014-15. Data Validity and Reliability: The collated data was analysed against the questions and criteria listed in the evaluation matrix. In the process, the evaluators triangulated the key findings with the gathered data from various sources, particularly KAP findings with the qualitative data. This helped to gather common trends, contradictions and differences against each criteria and evaluation question and gaps in information. As a result of triangulation and complementarities between quantitative and qualitative methods ensured the validity and reliability of the findings and drawn inferences of the changes that happened overtime against the key programme indictors. Checklist for Conducting Field Work: The evaluation team developed a checklist for conducting FGDs, IDIs and direct Observations in the field. The checklist described step-by-step approach to plan, prepare and implement the FGDs, IDIs, observations and case studies in the field. The checklist also identified the logistic support needs for conducting field activities Evaluation Reference Group (EVG): UNICEF has a functional EVG with membership from UNICEF Pakistan and ROSA, who guided the process of evaluation including feedback and approval of the inception report and methodology. The UNICEF focal point in Pakistan worked closely with the evaluators and ensured compliance on the guidance received from the group. The PMER unit in Pakistan kept a track record of all the activities and the evaluators provided periodic updates on the progress made on evaluation Coordination and Meetings with UNICEF and IPs: The evaluators ensured a close coordination with the UNICEF and IPs staff on the various aspects of the evaluation. The evaluators held several face-to-face meetings with the UNICEF and IP staff at national, provincial and district levels to understand further the program. The evaluators also kept them in loop for undertaking evaluation activities through UNICEF focal points including reaching out to the key stakeholders at national, provincial and district level.

Annex 11: Integration Elements and Scaling

Framework

Page 77

Annex 12: Examples of Iterative Management Integration is an evolving and iterative process to reflect back and learn from implementation experiences including continuous sharing, not only of successes, but also of challenges

11. Integrated programming can happen at many

levels (national, subnational, between or within agencies), and opportunities to better integrate interventions at varying level/s may arise as programmes evolve. This evolving nature of integrated programming underpins the need to keep program design flexible and deploy the principles of ―adaptive program management‖ to enable implementers to identify and rectify gaps and challenges in timely manner.

Cross-referencing to the adaptive management principles, review of key project documents and IDIs with key informants, UNICEF took various corrective measures as presented in table 8 below.

TABLE 1: EXAMPLES OF ITERATIVE ADAPTIVE MANAGEMENT

Aspect Bottlenecks/Challenge Adaptive Management

Programme Inception Phase and IPs selection

In the inception phase, the programme lost considerable time in finalization of the intervention districts. At the design phase, it was planned to implement activities in the southern parts of Sindh but due to the Nutrition PC1 which was already covering those districts and consultation P&DB the plan was changed to the three districts in north. This resulted in identification of new intervention districts based on district mapping exercise including MCHIP presence, stunting prevalence and poor sanitation and hygiene indictors and water quality issues. As a result, the inception phase remained inadequate, which is necessary to undertake appropriate preparations to launch the programme. Moreover, the short program lifespan (only 18 months) posed additional pressure on UNICEF to begin implementation of activities swiftly at the start of the programme

UNICEF speeded up the process of IPs selection and identify partners from a pool of pre-qualified agencies

IP‘s Placement and Commencement of Programme Activities

Both sectors (WASH and Nutrition) signed standalone PCAs with IPs on-roll-on basis, specifying sector-specific interventions. Hence, WASH partners began their activities much earlier than the Nutrition partners at the district and community levels.

The WASH partners recruited CRPs and established Village WASH Committees prior to the placement of Nutrition partners in many locations.

In the start, WASH and Nutrition partners were having their specific work plans and limited coordination at the district and community level

After placement of all IPs, UNICEF and partners took several steps to integrate activities as:

Developed joint work plans for integrated outreach activities

Developed joint PMP

Established mechanism of periodic joint program review

Planned and implement joint capacity building activities

Adapted PATS plus approach by integrating nutrition messages in the WASH packages

Implemented joint SBCC activities

Sectoral Prioritization within UNICEF

As pointed above, the IPs became operationalized on roll-on basis in the start of the programme; hence, both sectors worked in silos and remained engaged with their concerned IPs. This resulted in fragmentation of activities

UNICEF established interdisciplinary management team to break the silos, as discussed under SR-CIG above

11 https://www.unicef.org/media/files/IntegratingWASHandNut_WHO_UNICEF_USAID_Nov2015.pdf

Page 78

Coordination with District Authorities and Line Departments at District level

After inception, IPs were facing challenges to coordinate and work in collaborative and efficient manner with district authorities and line departments

UNICEF and IPs established District Steering Committee (DSC), led by DCs. The members included: representative from all line departments, UNICEF-IPs and other CSOs

UNICEF with support of PLAN International seconded a coordination officer in each district to organizing DSC‘s meetings and following on the agreed actions

SBCC After inception, the partners were using different SBCC approaches including IEC materials

UNICEF led the process and equip the teams with standardized messages, adopted from NSP

Legal Binding The PCAs neither mentioned integrated approaches nor made the partners bind to implement integrated activities

UNICEF advocated and worked jointly with partners to adopt mechanisms for integration and identified common touch points to support the process of integration

Water Quality Testing

Water quality testing was not budgeted in the PCAs

After signing PCAs and during implementation, UNICEF re-appropriate budget for undertaking pre and post-tests of water supply systems (e.g. water supply schemes and hand pumps) water quality tests

Annex 13: IEC Material and Fragmented Messages

Screenshots of the Pectoral IEC materials by incorporating stylistic

components of most watched media into the illustration (figure 1) and

Fragmented Hand Washing Messages (figure 2)

FIGURE 1: SCREENSHOT OF THE IEC MATERIALS INCORPORATING STYLISTIC COMPONENTS FROM THE MOST WATCHED

MEDIA

Page 79

FIGURE 1: SCREENSHOT OF THE MESSAGES OF THE MISAALI MAAN CAMPAIGN

Page 80

Annex 14: Key Documents Reviewed

1. Action Against Hunger/ACF International PCA 2016-2017

2. HELP Program Cooperation Agreement (PCA)

3. SAFWCO Program Cooperation Agreement (PCA)

4. NRSP Program Cooperation Agreement (PCA)

5. Plan International Program Cooperation Agreement (PCA)

6. HANDS Program Cooperation Agreement (PCA)

7. UNMCSRP _Social Behaviour change Communication Strategy _White Rice _2018

8. M&E Report SBCC

9. Misaali Maa _Urdu

10. Participatory evaluation of Misaali Maa Campaign _2018-participatory video with most significant

change initiative for UNMCSRP

11. UNMSCRP Social Behaviour Change communication Strategy

12. Stop Stunting SBCC Presentation by White Rice

13. Institutional/Corporate Contract White Rice

14. Formative Research Report on Behavioural determinants

15. UNMCSRP ToR for implementation of SBCC interventions

16. SBCC 3 day Training Handbook

17. Status of Funds Transferred to WASH IPs(2016-2018)

18. Tools and Checklist:

o Growth monitoring card o Monthly Report of LHWs format o Checklist PATS PLUS Program o Field Monitoring checklist of PATS Program o PHC Monthly report format o Facility staff meeting notes format

19. Third Party Monitoring Checklist

o Field Visit Report 4-7 July o Field Visit Report 10-13 July o Field Visit Report 8-20 July o Field Visit Report 30 July-3 August o Checklist PHE updated

20. Cumulative Action Point Matrix for the Month of Feb 2018 (002)

21. Cumulative Action Point Matrix NRSP Feb – March 2018 (002

22. WASH E tools _CSO partnerships

23. HANDS Action Point Matric Feb 18

24. HANDS Detail of CIG Beneficiaries

25. Summary of Action Points in Program Review meetings

26. Nutrition Case studies

27. PATS PLUS Program Training Manual for CRPS Sindhi

28. Training Manual on Stunting Reduction for Lady Health workers and Lady Health supervisors

English

29. Training Manual on Stunting Reduction for Lady Health workers and Lady Health supervisors Sindhi

30. PATS PLUS training manual for LGD and PHE&RDD, Sindhi

31. PATS PLUS Program Training Manual for LHWs-Sindhi

32. PATS PLUS Program Training manual for social Organizer Sindhi

33. PATS PLUS Program Training manual for teacher Sindhi

Page 81

34. PATS PLUS Program Training manual for mason Sindhi

35. PATS PLUS Program Training manual for Village WASH committee Sindhi

36. Joint Partnership Review HANDS

37. Joint Partnership Review NRSP

38. Joint Partnership Review Plan International

39. Joint Partnership Review SAFWCO

40. UNMCSRP Monitoring and Evaluation Plan –Revised March 2016

41. Revised M&E Matrix AAP

42. Notifications

o Notification of Provincial Nutrition steering Committee-Sindh

o Notification Provincial Task Force -Sindh

o Notification for District Coordination Committee for Nutrition-Sindh

o Notification for Sub divisional coordination committee-Sindh

43. ToRs for Multi-sectoral coordination committee

44. Minutes of 2nd Provincial Nutrition Steering committee July 16

45. Minutes of Provincial Nutrition Steering committee Dec 15

46. UNMCSRP Final Progress and Utilization Report (October 2015 – December 2018) For USAID

47. UNMCSR Program Contribution Agreement USAID and UNICEF

48. UNMCSRP Progress and Utilization Report for USAID Oct 17-Sept 18

49. UNMCSRP Monitoring and Evaluation Plan –Revised Feb 2016

50. Key Highlights of UNMCSRP presented in Round table Conference

51. UN Maternal & Child Stunting Reduction Programme (Utilization of Community Investment Grant)

52. Integrated WASH and Nutrition Action Plan

53. Memorandum of Understanding by and among UNICEF AND Department of Health, Government of

Sindh (―GOS/DOH‖) AND Local Government Department, Government of Sindh (―GOS/LGD‖) AND

Public Health Engineering & Rural Development Department, Government of Sindh

(―GOS/PHERDD‖) AND Planning and Development Department, Government of Sindh

(―GOS/P&D‖)

54. District Mapping Matrix _USAID "The UN Maternal and Child Stunting Reduction Program"

55. Union Council Wise population of Districts Ghotki, Noushero Feroze and Khairpur

56. Work plan USAID Stunting Reduction Program, Annex

57. Pre KAP Survey INFANT YOUNG CHILD FEEDING & WASH PRACTICES AND BEHAVIORS in

three districts of Sindh

58. Pre and Post KAP raw data

59. Post KAP Questionnaire

60. Details of Fathers support Group Noushero Feroze

61. Details of Mothers Support Group Noushero Feroze

62. List of trained CHWs District Khairpur

63. List of trained CHWs District Ghotki

64. List of trained CHWs District Noushero Feroze

65. Master Data base District Ghotki

66. WASH Data base District Khairpur

67. List of status OTP sites established by HELP in District Noushero Feroze

68. Detailed Nutrition indictors for all three districts for 3 years( 2016. 2017, 2018)

69. Nutrition Coordination Platforms in sindh

70. Program and donor mapping Sindh

71. Coverage Matrix Nutrition Who, What, When 3W's 2017-18

72. District-Wise Nutrition Caseload

73. Nutrition stakeholders Contact List

74. Nutrition Working Group Contact List

Page 82

75. District Wise mapping of nutrition interventions Sindh

76. Module on Nutrition for LHW training Manual

77. Job description IYCF Counsellors

78. Job Description of CHWs

79. Presentation of the program by HELP in inception meeting

80. Presentation of Program by HANDS in inception meeting

81. Presentation of Program by NRSP in inception meeting

82. BCC Strategy May 2016

83. Sindh Multi -Sectoral Action For Nutrition (MSAN)

84. Inter sectoral Nutrition strategy Sindh , Pakistan

85. Drinking Water Supply Policy (2016) Sindh Pakistan

86. Sindh Sanitation Policy- November 2016

87. Drinking Water, Sanitation and Hygiene Sindh Strategic Sector Plan 2016 – 2026 (Final Draft 6.0

dated 29th December 2016)

88. Final Consolidated Summary from May 2016 to April 2018 Nutrition Sensitive WASH Interventions

under PATS Plus Districts: Ghotki, Khairpur and Naoshero Feroz

89. WASH Services under UN Maternal and child stunting reduction Programme complete Database

90. 400 stories SBCC –UNMCSRP in Sindh

91. Water Quality analysis of drinking water sources in three districts of Sindh under UN Maternal and

Child Stunting Reduction Program, Sindh, Pakistan

92. UN Maternal & Child Stunting Reduction Programme (Utilization of Community Investment Grant

HANDS

o HANDS Project Completion Report 15th May 16-15th April 2018 o Integrated WASH/ Nutrition Action Plan o Comparative Analysis of Rapid Need Assessment 2015 and Multiple Indicator Cluster Survey 2014-

15 Ghotki, Khairpur and Nosheroferoz Districts of Sindh –prepared by HANDS o Case studies by HANDS o Installation and rehabilitation of hand pump in community installation District Khairpur o List of Beneficiaries under Community Investment Grant o list of new construction and rehabilitation wash facilities in heath care centres in district Khairpur o NOC (No Objection Certificate )from District Administration Khairpur o NOC (No Objection Certificate )from Education Department for WINS o NOC (No Objection Certificate) from Health Department for WASH District Khairpur o ODF Committee notification from District Administration Khairpur o List of ODF Villages District Khairpur o Post Water Quality test results of hand pumps- District Khairpur o Project Exit strategy –HANDs o Monitoring Framework WASH Services under ‗UN Maternal and Child Stunting Reduction

Programme‘ o Project sustainability plan at village level by HANDS

NRSP

o Baseline and End line Survey Report by NRSP

o Beneficiaries of Behavior change campaign Ghotki –NRSP

o Exit strategy UN-Maternal & Child Stunting Reduction Taluka ubauro District Ghotki Sindh

o List of Intervention Villages in Tehsil Uboro District Ghotki

o Overview of results and achievements –NRSP

o UNMCSRP List of Social Organizers Taluka Ubaro District Ghotki byNRSP

o UNMCSRP Village WASH Committees (VEC) Report _Ubaro District Ghotki by NRSP

o Water and Sanitation Hygiene Club (WASH-CLUB) Report District Ghotki by NRSP

SAFWCO

o Project Completion Report 15th May 2017-15th April 2018

o ODF committee Training Rpeot

Page 83

o LG UC Reps and PATS Plus Training Report

o 02 Days Training L.G, UC Secretaries & Govt. Officials on ODF sustainability

o Village WASH Committee Details of District Noushero Feroze Taluka Mehrabpur

o ODF /Non ODF Assessment Report by SAFWCO

o PATS Plus –social Organizer Training report by SAFWCO

o PATS Plus –SAFWCO engineers training report

o PATS Plus –CRP Training report by SAFWCO

o PATS Plus –Masons Training report by SAFWCO

o PATS Plus –Teachers Training report by SAFWCO

o PATS Plus –School Management committee Training report by SAFWCO

o Community Partnership in the project for utilization of Community Investment Grant WASH Services

under UN Maternal and Child Stunting Reduction Programme SAFWCO

o List of villages declared, verified and certified for ODF in Noushero Feroze-SAFWCO

o List of self-made latrines –SAFWCO

o ODF Reward Ceremony Report by SAFWCO

o Report on Global Hand washing day 15Sept 2017 by SAFWCO

o Report on World toilet day 19th November 2016 by SAFWCO

o List of entrepreneur and Masons trained in Noushro Feroze

o Report on Conclave for Community Resources persons to Exchange Knowledge –

SAFWCO

o Water Biological Training report by SAFWCO

o List of WASH Club Students Taluqa Mehrabpur District Nosheroferoze by SAFWCO

o List of WASH Facilities in School and Health Centres in Taluqa Mehrabpur District Noshero

Feroze by SAFWCO

o Sample Short Term Employment Agreement for CRP

Annex 15: Team Composition and Responsibilities

The senior leadership and lead consultants included ,Lead Evaluator, Evaluation Coordinator, Qualitative

Research Lead, Programmer & Data Analysis Expert and Health Economist while the Team lead was

responsible for coordination among the consortium partners and UNICEF The lead consultants trained the

field data enumerators, who gathered data at the community level. The lead consultants provided technical

back stopping and ensured quality assurance in the data collection and management. Additionally, the lead

consultants undertook the in-depth interviews with key stakeholders at national, provincial and district

levels.

The evaluation followed open communication and collaborative approaches with all involved actors and

UNICEF. The evaluation team coordinated with UNICEF for issues and barriers in the implementation of

activities and to address them optimally, should they arise.

Evaluation Team Organogram

Page 84

Team Responsibility Matrix

Keeping in view the requirements of the project, the core team comprised of Sector/Thematic Specialists

assisted by respective Associates. The matrix gives overview of core team‘s roles and responsibilities:

Name Role

Dr Fayaz Team Lead

Key responsibilities

Ensure smooth flow of information and close coordination among the consortium members/partners who are jointly conducting the evaluation

Coordinate with UNICEF for all contract and technical/programme related matters

Dr Asmatullah M&E Advisor

Key responsibilities:

Develop the overall Evaluation framework in consultation with UNICEF

Provide technical oversight to develop methodology for evaluation

Conduct desk review and scooping literature review

Lead the development of the evaluation tool kit including interviews guides for FGD, IDI and observation checklist

Finalize sampling framework (gathering relevant information from UNICEF and then selection of respondents for the FGDs)

Provide technical oversight for statistical analysis on the primary/secondary data

Validate and triangulate the finding of quantitative and qualitative information gathered for evaluation

Co-facilitate the training for enumerators

Conduct IDI at National, Provincial and District level

Ensure that the evaluation tool and approaches are of high quality and adhered to UNICEF quality standards

Organize preliminary findings, present it to the concerned stakeholders

Incorporate feedback into the report and lead the process of suggesting recommendations

Lead the process of evaluation report write up

Dr Javeria Majeed Swathi Qualitative Research Lead


Page 85

Conduct scooping literature review

Conduct desk review of all key programme and other relevant documents

Develop study tool kit (FGD, IDI, observation tools)

Shortlist the case studies and gather information for compiling them

Develop training packages for training the field enumerators

Facilitate the training for the field enumerators

Conduct interviews with the key informants at National, Provincial and district level

Lead qualitative data gathering, compilation, interpretation and analysis

Triangulate the qualitative findings with primary and secondary data

Work on the case studies and documentation of the programme outcomes

Contribute in the presentation of preliminary findings

Contribute in the final evaluation report write up

Afeef Health Economist


Gather relevant financial and programme data for cost-benefit-analysis

Gather information on nutrition specific and sensitive programme interventions for DALY calculation

Conduct consultation with UNICEF and partners staff with regard to cost-benefit-analysis

Synthesize information of cost-benefit-analysis including DALYs and cost ratio for the programme

Analyse data and calculate cost effective and cost benefit analysis

Contribute in validation and triangulation of collated data and to synthesise findings for evaluation and inception report

Dr Mohammad Rafiq Data Analysis expert


Import the raw data of pre-and-post KAP data set for analysis

Listing of indicators and run statistical analysis on the quantitative data

Prepare preliminary findings of pre and post data sets

Triangulate the quantitative data results with qualitative findings

Gather secondary data from different sources from PDHS, NNS, PCLM etc

Prepare pre and post KAP survey findings including tabulation and graphical presentation

Contribute in the evaluation report write up

Amjad District Coordinators


Support the implementation of field work plans

Support the process of selection of the enumerators for field data collection

Coordinate with field teams and ensure that data collections is carried out as planned and all standards are being followed

Provide logistic support within the districts

Document proceedings and compile field-based report if needed

Participate in daily de brief session and ensure compliance to action points

Update teams of evolving security context and take effective steps to avert security incidents

Page 86

Annex 16: Bibliography

1. UNICEF The State of the World’s Children 2009 - Full report [Internet]. [cited 2019 Jun 26]. Available from:

https://www.unicef.org/sowc09/report/report.php

2. Aguayo VM, Menon P. Stop stunting: improving child feeding, women’s nutrition and household sanitation in

South Asia. Matern Child Nutr [Internet]. 2016 [cited 2019 Jun 26];12 (Suppl Suppl 1):3. Available from:

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5084809/

3. WHO | World Health Statistics 2014. WHO [Internet]. 2015 [cited 2019 Jun 26]; Available from:

https://www.who.int/gho/publications/world_health_statistics/2014/en/

4. Chambers R, Medeazza G Von. Reframing Undernutrition: Faecally- Transmitted Infections and the 5 As

[Internet]. Vol. 2014, IDS Working Paper. 2014. 36 pp. Available from:

http://www.communityledtotalsanitation.org/sites/communityledtotalsanitation.org/files/media/Wp450_C

hambers_Medeazza_Reframing_Undernutrition.pdf

5. UNICEF ROSA. http://www.unicefrosa-progressreport.org/stopstunting.html

6. Bhutta ZA, Das JK, Rizvi A, Gaffey MF, Walker N, Horton S, et al. Evidence-based interventions for

improvement of maternal and child nutrition: what can be done and at what cost? Lancet [Internet]. 2013

Aug 3 [cited 2019 Jun 26];382(9890):452–77. Available from:

http://www.ncbi.nlm.nih.gov/pubmed/23746776

7. SBCC Strategy UNMCSRP 2018, p3

8. Reproduced from PDHS 2017-18 Report, p37

9. ICF. NI of PS (NIPS) [Pakistan] and. Pakistan Demographic and Health Survey 2017-18. [Internet]. Pakistan;

2003. 1–18 p. Available from: http://nips.org.pk/PDHS.htm

10. World Bank. World development report. [Internet]. Oxford University Press; [cited 2019 Jun 26]. Available

from:http://documents.worldbank.org/curated/en/435331468127174418/World-development-report-2006-

equity-and-development

11. UNICEF Annual Report 2016 | UNICEF Publications | UNICEF [Internet]. [cited 2019 Jun 26]Available from:

https://www.unicef.org/publications/index_96412.html

12. Chambers R, Medeazza G Von. Reframing Undernutrition: Faecally- Transmitted Infections and the 5 As

[Internet]. Vol. 2014, IDS Working Paper. 2014. 36 pp. Available from:

http://www.communityledtotalsanitation.org/sites/communityledtotalsanitation.org/files/media/Wp450_C

hambers_Medeazza_Reframing_Undernutrition.pdf

13. Spring Nutrition: https://www.spring-nutrition.org/stories/impact-water-sanitation-and-hygiene-wash-

nutrition

14. WHO/UNICEF. Annual Report. 2016;108. Available from:

https://cms.qut.edu.au/__data/assets/pdf_file/0017/631502/2016-annual-report.pdf

15. Cumming O, Cairncross S. Can water, sanitation and hygiene help eliminate stunting? Current evidence and

policy implications. Matern Child Nutr [Internet]. 2016 May [cited 2019 Jun 26];12:91–105. Available from:


16. Smith T, Shively G. Multilevel analysis of individual, household, and community factors influencing child

growth in Nepal. BMC Pediatr [Internet]. 2019 Dec 5 [cited 2019 Jun 26];19(1):91. Available from:

https://bmcpediatr.biomedcentral.com/articles/10.1186/s12887-019-1469-8

17. Raju D, D’Souza R. Child Undernutrition in Pakistan. What Do We Know? 2017;(May).

https://www.unicef.org/sowc09/report/report.php


https://www.who.int/gho/publications/world_health_statistics/2014/en/

http://www.communityledtotalsanitation.org/sites/communityledtotalsanitation.org/files/media/Wp450_Chambers_Medeazza_Reframing_Undernutrition.pdf


http://nips.org.pk/PDHS.htm

http://documents.worldbank.org/curated/en/435331468127174418/World-development-report-2006-equity-and-development

http://documents.worldbank.org/curated/en/435331468127174418/World-development-report-2006-equity-and-development

https://www.unicef.org/publications/index_96412.html



https://cms.qut.edu.au/__data/assets/pdf_file/0017/631502/2016-annual-report.pdf


https://bmcpediatr.biomedcentral.com/articles/10.1186/s12887-019-1469-8

Page 87

18. Aguayo VM, Menon P. Stop stunting: improving child feeding, women’s nutrition and household sanitation

in South Asia. Matern Child Nutr [Internet]. 2016 [cited 2019 Jun 26];12(Suppl Suppl 1):3. Available from:


19. Shroff MR, Griffiths PL, Suchindran C, Nagalla B, Vazir S, Bentley ME. Does maternal autonomy influence

feeding practices and infant growth in rural India? Soc Sci Med [Internet]. 2011 Aug [cited 2019 Jun

26];73(3):447–55. Available from: http://www.ncbi.nlm.nih.gov/pubmed/21742425

20. UNICEF Pakistan: https://www.unicef.org/pakistan/

21. Global Nutrition Report 2018

22. SUN-SDG Advocacy Tool Kit

23. UNICEF’s Approach to Scaling-up Nutrition for Mothers and their Children, discussion paper, Programme

Division, UNICEF, New York City, June 2015 and UNICEF Nutrition-WASH Tool Kit – Guide for Particle Joint

Action

24. Bhutta ZA, Das JK, Rizvi A, Gaffey MF, Walker N, Horton S et al. (2013). Evidence-based interventions for

improvement of maternal and child nutrition: what can be done and at what cost? Lancet. 382(9890):452–

77.

25. Lancet 2008, WASHPlus 2015, Checkley at al 2008, Horton and Hoddinott, 2014; Ngure al 2014

26. Adopted from Black, Robert E.et al., Maternal and Child Under-nutrition, Lancet Series Paper 1, 2013 and

UNICEF Nutrition-WASH Tool Kit – Guide for Particle Joint Action

27. UNEG Integrating Human Rights and Gender Equality in Evaluations

28. Scaling Up Nutrition (SUN). https://scalingupnutrition.org/news/national-nutrition-survey-2018-launched-

pakistan/

29. Pakistan Institutional Transformations 2016-2017. http://docs.scalingupnutrition.org/wp-

content/uploads/2017/10/Pakistan-SUN-Movement-Annual-Progress-Report-2017.pdf

30. SUN. http://docs.scalingupnutrition.org/wp-content/uploads/2017/10/Pakistan-SUN-Movement-Annual-

Progress-Report-2017.pdf

31. Ministry of Health Services, Regulation and Coordination Dashboard. http://nhsrc.pk/

32. Pakistan 6th Population and Housing Census 2017

33. Pakistan Census 2017. http://www.pbscensus.gov.pk/

34. National University of Science and Technology

35. Organisation for Economic Coordination and Development – Development Assistance Committee (OECD

DAC), 2010. Glossary of Key Terms in Evaluation and Results Based Management. [pdf] Paris: OECD DAC.

Available at: http://www.oecd.org/dac/evaluation/2754804.pdf [Accessed: 12 January 2019].

36. United Nations Evaluation Group (UNEG), 2014. Integrating Human Rights and Gender Equality in

Evaluations. [pdf] New York: UNEG. Available at: http://www.unevaluation.org/document/download/2107

[Accessed: 12 January 2019].

37. R Studio, Inc., Boston, MA, USA

38. WHO (2010) Indicators for Assessing Infant and Young Child Feeding Practices, World Health Organization.

doi: ISBN 978 92 4 159975 7.

39. United Nations Children’s Fund and World Health Organization (2016) Core questions and indicators for

monitoring WASH in Schools in the Sustainable Development Goals. Available at:

http://www.who.int/about/licensing/copyright_form/en/.

40. Mchugh, M. L. (2013) ‘The Chi-square test of independence Lessons in biostatistics’, Biochemia Medica, 23(2),

pp. 143–9. doi: 10.11613/BM.2013.018

41. UNICEF. Equity-focused evaluations. 2014;

42. Nick. Microsoft Word - IJQM 8.3 Onwuegbuzie. 2009;1–21. Available from:

http://research.apc.org/images/2/2f/A_Qualitative_Framework_for_Collecting_and_Analyzing_Data_in_Foc

us_Group_Research.pdf%5Cnpapers3://publication/uuid/97441C12-D3E5-4116-BE31-58D5E04A3915



https://www.unicef.org/pakistan/

http://www.who.int/about/licensing/copyright_form/en/

http://research.apc.org/images/2/2f/A_Qualitative_Framework_for_Collecting_and_Analyzing_Data_in_Focus_Group_Research.pdf%5Cnpapers3:/publication/uuid/97441C12-D3E5-4116-BE31-58D5E04A3915

http://research.apc.org/images/2/2f/A_Qualitative_Framework_for_Collecting_and_Analyzing_Data_in_Focus_Group_Research.pdf%5Cnpapers3:/publication/uuid/97441C12-D3E5-4116-BE31-58D5E04A3915

Page 88

43. Arena P, Brunno F, Caponetto R. A genetic algorithm optimized SDT for state feedback control. Int Symp

Power Electron Electr Drives, Autom Motion, 2006 SPEEDAM 2006. 2006;2006:722–6.

44. Networker EMC, Guidance T, Networker EMC. Technical Note. 2010;(November):1–25.

45. Eeuwijk P Van, Zuzana A. How to Conduct a Focus Group Discussion ( FGD ) Methodological Manual By Peter

van Eeuwijk and Zuzanna Angehrn. 2017;

46. USAID. C-3: Evaluation stakeholder analysis. 2013; Available from:

https://usaidlearninglab.org/sites/default/files/resource/files/23od4_evaluation_stakeholder_analysis.pdf

47. United Nations Evaluation Group (UNEG), 2016. Norms and Standards for Evaluation. [online] New York;

UNEG. Available at: http://www.uneval.org/document/download/2787 [Accessed: 12 January 2019].

48. United Nations Children’s Fund (UNICEF), 2015. UNICEF PROCEDURE FOR ETHICAL STANDARDS IN RESEARCH,

49. EVALUATION, DATA COLLECTION AND ANALYSIS. [online] New York: UNICEF. Available at:

https://www.unicef.org/supply/files/ATTACHMENT_IV UNICEF_Procedure_for_Ethical_Standards.PDF

[Accessed: 26 April 2018].

50. www.unicef.org/eapro/Reporting_on_children_and_young_pp.pdf and www.childethics.com

51. United Nations Economic and Social Council, Statistical Commission, Report of the Inter-Agency and Expert

Group on Sustainable Development Goal Indicators, February 19, 2016 (New York)—2016 Global Nutrition

Report

52. http://scalingupnutrition.org/wp-content/uploads/2017/06/EXE_Black_SCORECARD-ACF_ENG.pdf

53. http://documents.worldbank.org/curated/en/893571483783978617/pdf/ITM00194-P158769-01-07-2017-

1483783975398.pdf

54. https://extranet.who.int/nutrition/gina/sites/default/files/PAK_Intersectoral%20Nutrition%20Strategy%20fo

r%20Sindh%202015.pdf

55. UNEG Integrating Human Rights and Gender Equality in Evaluations

56. SIDA, WHO and OHCHR, ‘Human Rights and Gender Equality in Health Sector Strategies: How to Assess Policy

Coherence’, WHO 2011, p.15<whqlibdoc.who.int/publications/2011/9789241564083_eng.pdf>.

57. Caren Levy, ‘Gender Justice and Development Policy: Is “Gender Mainstreaming” Up To The Challenge?’UCL

Development Planning Unit, <www.ucl.ac.uk/~ucugw3i/files/ISID6/ISID_Caren Levy_Gender Justice and

Policy.pdf>.

58. Source: UNICEF, WHO and USAID guidance on improving nutrition outcomes with better water, sanitation

and hygiene: practical solutions for policies and programs:

ttps://www.unicef.org/media/files/IntegratingWASHandNut_WHO_UNICEF_USAID_Nov2015.pdf

59. Lancet Maternal and Child Under Nutrition, June 2013 http://www.lancet-

journals.com/NutritionSeries/files/assets/common/downloads/publication.pdf

60. Adnew, B. and Statz, J. (2007) ‘Bamboo Market Study in Ethiopia’. Available at:

https://www.unido.org/sites/default/files/2009-09/IP_ETH_ebook_0.PDF.

61. Funds, C. H. (no date) ‘Prospects for Integrating a Programme-Based Approach in CAPs and Common

Humanitarian Funds A SCOPING STUDY’, pp. 1–37.

62. Improving the evaluation of integrated programs: A guide for multisector evaluation | FHI 360 (no date).

Available at: https://www.fhi360.org/news/improving-evaluation-integrated-programs-guide-multisector-

evaluation (Accessed: 19 April 2019).

63. Making Islamkot a Model SDG Taluka – SDGs Support Unit Sindh (no date). Available at:

http://sindhsdgs.gov.pk/insight-newsandupdates-makingislamkot-a-model-sdg-taluka/ (Accessed: 23 April

2019).

64. Medicines, V. and Note, G. (2011) ‘Guidance for’, (3), pp. 1–46. doi: 10.2776/31954.

65. Metcalfe, V. and Giffen, A. (2011) ‘UN Integration and Humanitarian Space’, Humanitarian Policy Group,

(December). Available at: http://www.stimson.org/images/uploads/research-pdfs/Integration_final.pdf.

https://usaidlearninglab.org/sites/default/files/resource/files/23od4_evaluation_stakeholder_analysis.pdf

https://www.unicef.org/supply/files/ATTACHMENT_IV

http://www.childethics.com/

http://scalingupnutrition.org/wp-content/uploads/2017/06/EXE_Black_SCORECARD-ACF_ENG.pdf

http://documents.worldbank.org/curated/en/893571483783978617/pdf/ITM00194-P158769-01-07-2017-1483783975398.pdf

http://documents.worldbank.org/curated/en/893571483783978617/pdf/ITM00194-P158769-01-07-2017-1483783975398.pdf

https://extranet.who.int/nutrition/gina/sites/default/files/PAK_Intersectoral%20Nutrition%20Strategy%20for%20Sindh%202015.pdf

https://extranet.who.int/nutrition/gina/sites/default/files/PAK_Intersectoral%20Nutrition%20Strategy%20for%20Sindh%202015.pdf

http://www.lancet-journals.com/NutritionSeries/files/assets/common/downloads/publication.pdf

http://www.lancet-journals.com/NutritionSeries/files/assets/common/downloads/publication.pdf

Page 89

66. New scorecard shows mixed picture on national nutrition targets - Nutrition for Growth (no date). Available

at: https://nutritionforgrowth.org/new-scorecard-shows-mixed-picture-national-nutrition-targets/

(Accessed: 23 April 2019).

67. Report, F. (2018) ‘Evaluation of the Integrated Programmes Final Report NWO-WOTRO Evaluation of the

Integrated Programmes Final Report’, (January), pp. 1–56.

68. Taking the first step: Prioritizing SDGs for Sindh – SDGs Support Unit Sindh (no date). Available at:

http://sindhsdgs.gov.pk/taking-the-first-step-prioritizing-sdgs-for-sindh/ (Accessed: 23 April 2019).

69. UNDP (2017) UNDG | Delivering Together For Development, Standard Operating Procedures for Delivering as

One. Available at: https://undg.org/programme/undaf-guidance/key-approaches-for-integrated-

programming/ (Accessed: 18 April 2019).

70. Unicef (2018) ‘Evaluation of UNICEF’s Response to the Rohingya Crisis in Bangladesh Terms of Reference’,

(November). Available at:

https://www.unicef.org/evaluation/files/Terms_of_Reference_for_the_Evaluation_of_UNICEFs_repsonse_to

_the_Rohingya_Crisis_in_Bangladesh.pdf.

71. Unicef (no date) ‘Social Protection Strategic Framework, Integrated Social Protection Systems : Enhancing

equity for children’, Strategy, pp. 1–27.

72. Use an integrated rather than single-issue approach | Communicating with children | UNICEF (no date).

Available at: https://www.unicef.org/cwc/cwc_58608.html (Accessed: 18 April 2019).

73. WASH-Nutrition linkages (no date). Available at:

http://www.unicef.org/eapro/UNICEF_program_guidance_on_manangement_of_SAM_2015.pdf[ (Accessed:

23 April 2019).

Page 90

Annex Box 2

Annex A: Case Study Guide

The topics for the case studies will be identified and selected based on the review of the project

documents, ToC, secondary data, findings during field work and in-depth discussions with UNICEF and

Implementing Partners.

Case study sampling techniques: Considering the scope of Evaluation, the team will deploy purposeful

sampling to select the case studies

USING APPROPRIATE DATA COLLECTION METHODS AND PROCEDURES FOR CASE STUDIES:

Evaluators will use a range of data collection methods/tools to construct a case study, including desk

reviews of the key programme documents, ToC, consultation with UNICEF and IPs etc. Considering the

scope of evaluation ad balancing data need against time and resources, semi structured

interviews/FGDs will be deployed as the primary data collection methods for case studies

Though the nature of the case studies (deployed management approaches and assessing equity

aspects in services delivery) are such that no risk or threat is associated to any respondent, however

special precautionary measures will be followed to ensure that there is no harm or risk to the subject

RECORDING AND ORGANIZING DATA: During the data collection stage, the case study data will be

collected, organized and stored as:

The Senior Evaluation team members being experts in qualitative studies will be engaged in the

information collection for the case studies

The Evaluators will seek permission from relevant authorities (if needed) and will visit the subject for

collection of information

The Evaluators will ensure to keep the privacy and the confidentiality of the subjects being introduced

by following all the ethical principles including Informed consent, explain the purpose of visit, assuring

that the subject understand the reason for why information is being collected and feels secure from any

associated threat or harm to them

All the steps for the data handling, storage and destruction will be followed as stated in Annex E

No Photo or recordings will be done without the permission and if the case study engaged other people

than the subject, the evaluators will take an informed consents from them

The findings /information collected from the subjects will be shared with him/her at the end of any data

collection exercise to make sure that they agree with what has been recorded and are comfortable with

it. The subject/respondents will have a right to change any information or refuse to report something at

any stage

The contact number and addresses will be available with the subject/ respondent for him/her to contact

if he/she has any queries at later stage

The evaluation team will comply with all the ethical principles for data collection, analysis storage

and destruction as stated in Annex E

PROTECTION OF HUMAN SUBJECTS: Protecting the privacy and the rights of those involved will be

particularly taken care. The evaluators will make sure that the necessary steps are taken to respect privacy

and do no harm

Page 91

ANALYZING CASE STUDY DATA: Data analysis will begin during the data collection stage of case study,

Data protection will be ensured at all stages of handling of data

Coding: The Evaluator will deploy codes which involve labelling or categorizing passages or parts of text

so that the data can be readily retrieved, searched, compared and contrasted. The responses from the

subject will be narrated in a way that no one can track them to the respondents. The filled data collection

tools will be labelled such that no PII information can be seen there

TRIANGULATION: The data collected will be triangulated with multiple sources to compare and contrast

the findings. It will also help to reduce potential bias and increase confidence in the evaluation‘s overall

findings and conclusions.

GENERALIZATION Of FININDINGs FROM EVALUATIVE CASE STUDIES: The evaluators will not use

case studies to generalize the findings about the entire population, as such inferences for such

generalization can be drawn from a statistically representative sample surveys. However, case studies still

can help to advance or refute general ideas about how a project is working or not working in the given

scope.

The Evaluators will use the following case study checklist criteria12 for ensuring quality in the evaluation:

Questions YES Or NO

Is the rationale clearly presented for use of the evaluation case study method?

Is the unit of analysis clear?

Is the rationale provided for the selection of the specific case(s) or site(s)?

Is the justification clear for the number of case studies conducted?

Is the context in which the intervention is embedded described?

Are data collection methods within the case study clearly described?

Annex B: Consent Form Case Study

Location_________________________ Date_________________

Consent Form Case Study

My name is _________________ and I am a member of the Evaluation Team who has been hired by

UNICEF to know how the integrated Nutrition and WASH program implemented in your area.

You might know that about half of the children in Sindh are stunted which means that they are prevented

from growing or developing properly due to malnourishment and sickness which in most cases result from

low dietary intake, infections, poor Water and Sanitation infrastructure, unhygienic conditions and risky

behaviours. In this regard UNICEF implemented the project in your area which aimed to reduce stunting by

making the nutrition specific services including identification and treating of malnourished children in your

community and also working to improve the Water and Sanitation infrastructure and help people adopt

healthy behaviours.

12

USAID Technical Note for Evaluative Case studies

Page 92

You are being requested to participate in the case study as you were a beneficiary of the

services/participated in design and implementation of the interventions. We are taking information from you

as your experiences will be used by people in the country and across the world to design such projects in

better ways to help people live healthy lives. This information will have no negative impact on you and will

take up to 2 hours of your time.

To participate in this study, you do not need to do anything other than consent. We will be asking you

questions about your experiences with the program such as how did you come to know about this

programme? How did you interact with this programme? Do you think that the programme addressed any

specific needs?

No Personal Identification Information such as your National ID card number, bank details will be asked

from you. Additionally, we will be using pseudonyms in the report instead of your real name to keep the

anonymity of the gathered information. Furthermore, you will not be required to leave your home and visit

anywhere. If you feel you are busy, we can come and meet you at a time more convenient for you.

Participation in this study is voluntary and we will not provide any compensation /incentives to you.

The information being gathered from you is just for learning purpose and is not associated with bringing any

risk or harm to you. We respect your confidentially and we will take all necessary measures to protect it.

You will not benefit directly from participating in this study. We hope that the information learned from this

study can be used in the future to help other people or communities who are living in similar conditions to

achieve better health and nutrition status.

You can change your mind about participation at any time during the case study. You do not need to give a

reason to withdraw from the study. Withdrawal from the study will not have any effect on you or your family.

If you decide to leave the study at a later stage, you can ask that the information that was collected about

you not be used for the study, unless the study results are already published.

If you have any questions about this case study, you can contact the Lead Evaluator Dr Asmatullah at

0092-3340090944. We will hand over a copy of the consent form to you for your record.

Consent to Participate in the Case Study

By signing this consent form, I understand and confirm that:

1. All of my questions have been answered,

2. I understand the information within this informed consent form,

3. The Information will be reported in the final evaluation report which can be published or available on

website and can be read by many across the world

4. The information will be published without my name/child‘s name or fake names will be used and

every attempt will be made to ensure anonymity.

5. I can withdraw my consent at any time before publication of the report, but once the Information has

been published it will not be possible to withdraw the consent

Page 93

6. I do not give up any of my legal rights by signing this consent form,

Do you agree to participate in the case study? YES NO

Do you allow recording of the discussion? YES NO

Do you allow taking photos during the discussion? YES NO

Annex C: Guide of the Spot Checks & Observation

Spot Checks/Observation Guide

Instructions for Enumerators

The enumerators are expected to record their observation while being in field. The observations limited

to noticing general environment when the enumerators walk through the village and street and spot

checks of the hand washing stations in schools and households (if FGD is conducted in the HH),

latrines and drinking water storage

To observe the communal hand pumps the enumerators ill not need any informed consent but are

expected to respect the cultural norms such as not visiting the pumps if females are using it at the time

of observation

The spot checks at households can be done only by female enumerators and male enumerators will not

try to go inside the house to respect the local culture

IF the FGD with mother support group is conducted in a household level the female enumerator will

request the owner of the house who is expectedly member of support group to allow her to visit the

hand washing station and latrines within the household. The request will be made in Privacy to the

owner of house explaining the purpose of visiting the hand washing stations as to understand how the

hand washing stations in the village are set up and used. The female observer will further explain that

information collected will be anonymous and will have no implications of any type. If the owner agrees

with the enumerators, then she will have brief spot check and leave after thanking her.

In Schools the enumerators will seek permission from senior teachers to observe the hand washing

stations and latrine. The observations will be done only in the schools where the enumerators will visit

for FGD or IDI. The observer will further explain that information collected will be anonymous and will

have no implications of any type. They will observe hand washing stations and latrines in the presence

of teachers.

Similarly, the enumerators will take permission from the health facility in charge to observe OTP/SC site

infrastructure, WASH facilities in the OTP/SC centre and key materials/supplies in their presence per

the markers in the observation checklist.

The enumerators are not required to record any address or collected any other personal information

from anyone during the spot check

Page 94

The enumerators will take an informed consent from HH member, School senior/head teacher and

Health Facility in-charge as and where the observation is carried out. The informed consent is attached

as annex D

The Enumerators and ET team will discuss the observations and use the double entry notebook to further reduce

the bias

Observation Thoughts

Checklist/Marker of the Observation/Spot-Checks:

A few markers for observation areas have been listed below as an example while the enumerators are expected to

record all observations they come across which are relevant thematic areas of the programme intervention

Observe the household Compound and Courtyard or any area where FGD is being conducted for

if animal or child faeces are visible

if there is evidence of sweeping

Observing the latrines Cleanliness

Latrine has a lid

does not have flies

has soap or ash nearby, etc

Observing the type of water storage container for

What is the height of the container,

Whether it is covered,

Whether there is a separate clean mug to use, etc.

Observing of hand pump area Time, place, number and type of people there

Details of what happens at the site, sights, sounds, smells,

Observation during WASH and Safe Water supply in Health facility or School

existence, quality, and capacity of WASH infrastructure

Functional latrines

Open defecation in and near the school/HF is eliminated

Hand washing station with soap and water is available

Area Yes No NA Remarks/comments

Infrastructure: Observe the following facilities

Adequate waiting area for clients?

Breastfeeding corners?

In-patient facility for severely malnourished, only applicable for SC?

Are there appropriate facilities for mothers to bath and do laundry? Applicable only to SC

Are the rooms and courtyard clean (no litter, no

Page 95

faeces)?

Is hand washing Facility available in all areas for mothers and children

Water and Sanitation: Are the following services available?

Safe drinking water?

Functional Latrines available for both patients and attendants?

Waste collection system for non- medical waste?

Waste collection system for medical waste?

Material: Are the following printed material present?

Are there adequate number of necessary forms available at the service delivery point?

OTP/SFP ration cards for all beneficiaries

Home treatment card

Follow-up card

Register book

Medicine including antibiotics in the SC?

Referral /Admission

Are the referral forms available in the facility?

Is the referral facility known to the staff and the name displayed?

Guidelines for referral system available?

Is there adequate equipment available in the center?

Weighing scales

Measuring board

MUAC tapes

Salter scale

Thermometer

Beds in SC

Clean water for drinking (jug and cups)

Water and soap for hand-washing

Annex D: Consent Form Spot Checks

Consent Form for Spot Checks

Location: HH/HF/School and district______________________________ (Encircle one Option)

Date ___________________________

INTRODUCTION AND INFORMED CONSENT

Greetings in traditional way and Salaam. I would like to introduce myself and colleagues. My name is

_________________ and the colleagues names are______________ and _______________.

We are member of the Evaluation Team, commissioned by UNICEF to conduct evaluation of UN maternal and child

stunting program. We conducted FGD in your HH/School/HF today. The informed consent for observation/spot check

will be taken at three levels (as appropriate and relevant) and the enumerators will take the consent in separate from

the subject as:

Page 96

Household level: The purpose of the observation/spot check at household level is to understand what

changes the program brought in the WASH infrastructure (hand washing stations and latrines) at community

level

Schools: The purpose of the observation/spot check at school level is to understand what changes the

program brought in the WASH infrastructure at school level (hand washing stations, running water, soap

availability, cleanliness of latrines etc.)

Health Facilities: The purpose of the observation/spot check at health facilities is to understand what

changes the program brought in establishing OTP/SC centers (WASH infrastructure, availability of

materials/supplies required for management of malnourished children)

The observation/spot check will take around 10 minutes to complete. Whatever information we will observe will be

kept strictly confidential and will be dealt with anonymity. To keep the anonymity of the information, we will follow

steps as: a) we will assign pseudonyms instead of the real names and will use them to refer for the findings of the

observations, b) no personal information will be sought from you, c) nothing will be stated in the evaluation report with

your names or schools or health facility, d) No one will be able to track you or your school or your health facility from

the observations listed in the report, e) the information collected will be destroyed after a certain time, and f) the

collated information will have no implications on you or your staff (in case of schools and HF).

In this regard, I want to request your permission to visit the hand washing stations and latrine in the school and

households while the OTP/SC centre to observe how these have been set up. You can refuse the request even

without giving any reason. You or your team members (teachers/staff/respondent themselves) can accompany me

while I visit these spots.

If you have any questions about this case study, you can contact the Lead Evaluator Dr Asmatullah at 0092-

3340090944. We will hand over a copy of the consent form to you for your record.

Do you allow us to visit following?

1. Hand washing station : Yes No

2. Latrines Yes No

3. OPT/SC center Yes No

Annex E: Data Protection Guidelines

The evaluation will deploy special data security procedures in collecting, transferring, storing and

destroying as:

The study's location, tools, and planned interactions will ensure every possible effort to maintain the

respondents /participant's privacy.

Evaluation will ensure to maintain confidentiality of information collected during the field work. To this

end, the lead evaluators and the senior co-evaluators will identify the responses of individual subjects;

however, all necessary precautions will be put in place to prevent anyone outside of the evaluation

team from reaching the individual subjects with their responses. These measures will included:

o During the data collection, pseudonyms will be recorded in the FGDs and IDIs tools in the

column for the names in the FGD tools to keep the identity of the respondents anonymous. No

other personal information will be collected.

o Once the data collection tools are completed, a label/subject code will be written on the

completed IDI, FGD and case studies tool (e.g. completed questionnaire) and face sheets, if

any, will be removed. Additionally, we will prepare a separate document that links the code to

subjects‘ identifying information.

Page 97

o All hard copies such as interview notes, audio tapes and pictures will be kept securely locked to

prevent it from tampering, loss, theft, or physical damage. It will have restricted access and only

the lead evaluator and co evaluator (qualitative data expert) will have access to these files.

o All computer files will be encrypted, or password protected with limited access to the agreed

members of the team. The lead evaluator will limit administrator access rights to the senior co-

evaluators. The lead evaluators will change passwords often to ensure that only current project

members can access data. The evaluation team will also ensure to maintaining an updated anti-

virus protection on every computer including up-to-date versions of all software and media

storage devices. The evaluation team will use encryption, electronic signatures, or watermarking

to keep track of authorship and changes made to data file, as required. Additionally, the

evaluation team will take regular back up of electronic data files (both on and offsite) including

creation of protected copies.

o Evaluation team will ensure to take precautionary measures whilst sharing files within the

evaluation team such as sharing data through shared computer drives or by email. The

evaluation team members will not be allowed to take data home for work, and if it‘s unavoidable

then the files on the data storage device will be password protected including use unique user

IDs and passwords that cannot be easily guessed

o The evaluation team including all members who will work with the confidential data will be

trained on basic data security practices. The Lead evaluator will be directly responsible to train

evaluation team members who will work on confidential data.

The data will be thoroughly disposed, destroyed, or deleted after a period of 3 years of completion of the

evaluation. In a special case and upon UNICEF request, the evaluation team can consider providing a de-

identified data files for use to UNICEF. All paper files or CDs with any PII will be shredded and any

electronic files on memory drives, PCs, laptops and file servers will be permanently deleted.

Annex F: Assent Form School Children

Assent Form School Children (WASH Club Member)

Location ________________________ (Boys or Girl School) Date ____________________

INTRODUCTION AND INFORMED ASSENT

My name is ___________ and my colleagues‘ names are ________________ and ____________________

We are member of an evaluation team who are visiting your school to meet you and collect some information. Let us

just share more about the evaluation and our purpose of visit so that you can decide if you want to help us or not with

this study. Please feel free to ask me more questions about the purpose of visit. Our telephone number and e-mail

address is listed at the bottom of this page. If you have questions and would like to discuss them with someone else,

you can speak to Dr Asmtullah and Dr Javeria who sit in the main office whose number is also listed at the end of this

page. They will be able to answer your questions. I would like you to help us because you are a member of the school

WASH club.

We would like to ask you questions about how the WASH club was formed, what your role as WASH club members,

what changes you can see in the WASH infrastructure of school over last two years, hygiene and cleanliness

practices of yourself and other children in the school and your opinions about the health and hygiene practices in the

village where you live in.

Helping us with this study will take about 30 minutes of your time. You will not miss any important class time. Your

teachers have granted us permission to talk to you and will be standing outside of room and you can call them inside

anytime you want .We will not record your names and will assign you pseudonym names. Only this number will be left

Page 98

on the questionnaire, so nobody, not even me, will know how you answers. It will work the same way for the teachers.

Nothing in this study will be graded.

Nobody will be rude or trick you in any way. No one will know about your responses and further more your responses

are not being used to grade you, your teachers or your school. Even, if you do not understand any questions then ask

for clarification. If you may not want to answer any question so that is your right and you can say no to it, this will have

no implications on you, your teachers and school.

Your teachers know about this and agree that it is okay for you to help us if you want to. You may find the activity a

fun. The things we will learn from this study will help us and others to improve school health and hygiene and WASH

programs designed for schools in future.

No one is making you help us, and you don‘t have to if you don‘t want to. If you don‘t want to help us with the study

nothing bad will happen to you. No one will be mad at you. If you decide later that you don‘t want to be part of this

study, you or your parent/guardian /teacher can tell us by calling, emailing, or writing to us and I will put all of the

answer sheets in the garbage and will not include you in my study. If you do want to be in my study, nobody will know

your answers, including me. I am asking children of WASH clubs of other schools, many mothers and fathers in

community and other people so the information from you and your teacher will just be a little part of the big study.

When we finish the study, we might talk about what we learned from other people, or write it down so other people can

read it, but we will always talk about groups of kids, never about you and will never take your name.

You will be given two copies of this page. If you would like to help us and answer the questions please select ―Yes‖ on

this form and return, if you don‘t want to participate select ―No‖ and return the paper. We will not ask you any reason

for deciding not to participate in the discussion. You can keep the second copy of the form for your record


Ms________________________

Contact No___________________

Address ________________________

E-mail:



Lead Evaluators

Contact: 03340090944


Postal Address : House 141 Steet 23 G10/2 Islamabad Pakistan

Assent


Yes: I want to participate in the discussion

No: I don‘t agree to participate in the discussions

Annex G: Consent Form School Teachers

Page 99

Consent Form Responsible Persons/Teachers for Undertaking FGD with Children (School

WASH Club Members Children)

Location ________________________________: (boy school or girls school and district only)

Date ___________________________

INTRODUCTION AND INFORMED CONSENT

Greetings in traditional way and Salaam. I would like to introduce myself and colleagues. My name is

_________________ and the colleagues names are______________ and _______________.

I will be facilitating the discussion and my colleagues will help me in noting down the discussions. We are member of

the Evaluation Team, commissioned by UNICEF to conduct evaluation of UN maternal and child stunting program.

The purpose of today discussion is to know how the WASH and Nutrition program activities were implemented and

what changes this program brought in school infrastructure and health and hygiene practices of kids. The information

that you share will help to design better services in the future. In this regard, we want to meet the school children who

are members of the School WASH Club and ask them some questions regarding their role of WASH club members

and their understanding of health and hygiene practices. Sample question that we will ask from children is as: Does

your school have a WASH club? If yes, why do you think WASH club has been formed in the school? What has it

done?

We will not ask them any question about their personal identity, teacher performance or any other information.

Similarly, these responses will not be used to grade them, the teachers or school in any manner. No other sensitive

information will be collected from the children. The information will help us and others to design better program to

improve School WASH in future only.

The discussion with children will take about 30 minutes and we can wait to start the discussion with them on

appropriate time as advised by you so that they don‘t miss any important class to attend. During the discussion with

them, we will request the teachers to leave the room as to avoid any stress on children and help them express their

views freely. We assure you and will also educate children to leave the discussion any time they may prefer or refuse

to participate in the discussion and decide not to respond to the questions. Additionally, if you feel concerned with

anything and want to stop the discussion between us and them, you can communicate to us at any time. We will not

record children‘s names and will assign them with pseudonym. Only this name will be left on the data collection tool,

so nobody, not even us, will know who answered which question. Nothing in this study will be graded. We will not

even record your school address with the Reponses and no one will be able to track your school from the responses.

The information gathered from you and students will just be a little part of the big study. When we will finish the study,

we might talk about what we learned with other people or write it down so other people can read it, but we will always

talk about groups of kids and and will never take any name. If you or the parents/guardians of the children have more

questions and want to know further information after we leave you can contact on our contact mentioned below

Moreover, we have shared the purpose of evaluation with the District Administration including District Education

authorities and took their approval to proceed with FGDs with the children of WASH Club members. They informed

that we need your permission to meet the children of school WASH club members, so we are asking for your consent.

If you do not allow the children to participate in the study, please select ―No‖ and if you allow them please select ―Yes‖

below. We will give you an extra copy of the consent form for your record


Ms________________________

Contact No___________________

Address ________________________

E-mail:

Page 100



Lead Evaluators

Contact: 03340090944


Postal Address: House 141 Steet 23 G10/2 Islamabad Pakistan

Consent


Yes: I allow children who are members of WASH club to participate in the discussion

No: I don‘t allow the children who are members of WASH club to participate in the discussions

We are highly thankful for your time and cooperation to fill the consent from

Annex H: Consent Form FGDs

Fathers Group, Mothers Group, Social organizer, CRPs, Teachers, Masons, Village WASH Committee, Entrepreneurs, LHWs

Location ________________________

Date ___________________________

INFORMED CONSENT


The discussion usually takes 60 minutes to complete. Whatever information you provide will be kept strictly confidential and will be dealt with anonymity. To this end, we will follow steps as: a) we will assign pseudonyms against your names and will use them to refer to you, b) no personal information will be sought from you, c) nothing will be stated in the evaluation report with your names, d) No one will be able to track you from your responses, e) the information collected will be destroyed after a certain time, and f) the group participants are requested not to share any information or views outside the group.


We will be taking notes to help us remember what is said during the discussion. If you allow we will use recorder so that we do not miss any important information. We will destroy the recording after certain time. If you have any question or concerns, even after we leave, you can ask those from the lead evaluator Dr Asmatullah on [email protected] or 03340090944. At this point, I want your permission to record the interview, take notes to avoid any loss of information and seek your consent to record the discussion. We also seek your permission to take photos to be used in the report. Prior to the start of interview, do you have any question about the evaluation or the interview process? Do you agree to participate in the discussion? YES NO


Page 101



THANK YOU

Annex I: Consent Form IDIs

In-depth Interview Guide—UNICEF, Implementing Partners (IPs), Provincial Line Departments, District Line Departments, Local Government Representatives and Donor

Location ________________________

Date ___________________________

INFORMED CONSENT






THANK YOU