Upload
others
View
7
Download
0
Embed Size (px)
Citation preview
Achieving sustainable and equitable ODF at scale -evidence from sanitation and hygiene behaviour
change programmes
Hannah ChirgwinRadhika MenonWit Wichaidit
Matteus van der Velden
Water & Health Conference at UNC1 November 2018
Overview of the session
1. Introduction & background - WSSCC
2. WASH Sector Evidence gap map – 3IE
3. Outcome survey methodology for GSF programmes – UB
4. Systematic reviews & Impact study - 3IE
5. Discussion & closure
• Q&A after each presentation
2
Water, sanitation, and hygiene promotion for households, schools, and
health facilities – an evidence gap map update
Hannah Chirgwin
Research Associate
Water and Health 2018, UNC
Date of presentation: 01 November 2018
Hugh Waddington Hannah Chirgwin Duae Zehra John EyersSandy Cairncross
Authors and acknowledgements
• Thematic evidence collection on programmes, e.g. on a range of interventions
• Presents a matrix of policy relevant interventions, intermediate outcomes, and impacts
• Impact evaluations and systematic reviews
• Additional filters for region/country, study design, population, etc.
• A tool to navigate the evidence base
• A global public good
What is an evidence gap map?
Birte Snilstveit, Martina Vojtkova, Ami Bhavsar, Marie Gaarder 2013
Democratising evidence for accountability and learning
PRISMA flow chart
18, 037records identified through academic and trial registry database searching
430records identified through other
sources (grey literature searching, inclusion in 2014 map, etc.)
Title & abstract screening criteria:- Date- Country- Intervention/relevance- Study design- Relevant outcomes - Duplicate
13, 458records after duplicates
removed
1, 246 records manually excluded
11, 460 records excluded by machine
learning
Full-text screening criteria:- Date- Country- Intervention- Study design- Relevant outcomes - Excluded version (where
multiple publications)- Full-text unavailable
752records screened at full-text
Exclude on country: 7Exclude on intervention: 78
Exclude on study design: 140Exclude on relevant outcomes: 34
Exclude on version: 44Exclude as full-text unavailable:41
320 impact evaluations, 41 systematic reviews, and 47
protocols/trial registries included in the map
What vs. How
What vs. How
Source: Wager Lanoix1958, https://water1st.org/problem/f-diagram/
http://gapmaps.3ieimpact.org/evidence-maps/water-sanitation-and-hygiene-wash-evidence-gap-map-2018-update
• Sector-wide evidence map covering interventions to improve WASH access in households, communities, schools and health facilities
• 320 completed impact evaluations
• 47 ongoing impact evaluations
• 42 completed systematic reviews
• 1 ongoing systematic review
• 39 IEs in schools
• 1 IE in health facilities
Key statistics
Evidence base in L&MICs
Interventions mechanisms
0.00%
10.00%
20.00%
30.00%
40.00%
Direct hardwareprovision
Health messaging Psychosocial'triggering'
Systems basedapproaches
Until 2008
Since 2009
Intervention technologies
0.00%
5.00%
10.00%
15.00%
20.00%
25.00%
30.00%
35.00%
40.00%
45.00%
50.00%
Water Sanitation Hygiene Combined
Until 2008
Since 2009
Frequency of outcomes
0 20 40 60 80 100 120 140 160 180 200
Hygiene behaviour
Construction, use, and maintenance of latrines
Open defecation
Sustainability and slippage
Diarrhoeal disease
Nutrition and anthropometry
Mortality
Drudgery, pain, and musculoskeletal disorders
Income, consumption, and poverty
Safety and vulnerability
Changes in what is being reported on?
0.00% 20.00% 40.00% 60.00% 80.00% 100.00%
Behavioural
Health
Socioeconomic
Post-2008
Pre-2008
Populations
0
50
100
150
200
250
300
350
Rural Urban Slum(informal
settlement)
Refugeecamp
People livingwith HIV
Humanitariancrisis
People withdisabilities
Outcomes
- Time use (22 studies)
- Psychosocial health (7 studies)
- Safety and vulnerability (4 studies)
BUT gender analysis rarely used to understand programme effects and most studies don’t even report sex-disaggregated outcomes!
- 20% of IEs and SRs report sex disaggregation
- Outcomes disaggregated include: psychosocial health (43%), education and cognitive development (40%), open defecation (33%), time use (26%)
Gender-sensitivity
Study designs
62%
33%
5%
Randomized ControlledTrial (RCT)
Non-Randomized Design
Natural Experiment
What are we learning from SRs?
Topic area High confidence
Disposal of excreta Clasen 2010De Buck 2015
Hand hygiene Aiello 2008Ejemot-Nwadjaro 2015De Buck 2015
Water quality Clasen 2015
Trachoma Rabiu 2012
MHM Hennegan 2015
WASH in humanitarian settings Ramesh 2015
Nutrition Dangour 2013
People living with HIV/AIDS Peletz 2013
Slums Turley 2013
Schools ?
Community-driven approach ?
• Sustainability and slippage
• Psychosocial health
• Menstrual care
• Vulnerable populations
• Health facilities
• Synthesis gaps
Gaps?
Outcome Surveys
WSSCC’s Global Sanitation Funds Outcome Surveys in
Malawi, India and Tanzania
Wit Wichaidit, MS and Pavani K. Ram, MD
University at Buffalo
11/16/2018 23
Global Sanitation Fund
• UN Human Development Report 2006 raised the issue of global sanitation crisis
• GSF established in 2008 by Water Supply & Sanitation Collaborative Council (WSSCC )
• The world's only fund dedicated solely to improving sanitation
• $117 million USD allocated to 13 countries
11/16/2018 24
Key Results & GSF Outcome Surveys
• To be conducted every two years
• Independent verification of programme results
• Measure sustainability of the results achieved
• Measure other information not part of the routine monitoring system 25
Theoretical Framework for the Outcome Survey
(Based on theory of change for collective behavior change programs, with individual- and household-level effects)
Learning beyond ODF: Objectives of core analyses of outcome surveys
To describe key sanitation and hygiene outcomes of Global Sanitation Fund in study households and public facilities
To describe emerging process and outcome indicators of the program, including exposure to program activities, social norms, and habits of latrine use
To describe the needs of marginal and vulnerable households/population, and assess equity and non-discrimination by disaggregation of the key indicators by equity variables (e.g., geographic area, demographics, wealth status)
To assess sustainability of open-defecation free (ODF) status among ODF-verified communities
Core Methods
• Cross-sectional survey
• Multi-stage cluster sampling
• Data collection sites: households, schools, health facilities, and communities
• Data collection methods• Structured interviews: 1) heads of households;
2) female household members; 3) elderly/persons with disabilities
• Visual inspection of WaSH facilities and surroundings
• Structured observation in subset of households
• Community inspection for ODF verification
11/16/2018 28
Survey Description
11/16/2018 29
Malawi India Tanzania
Research firm The Malawi Polytechnic, Blantyre
Centre for Media Studies, New Delhi
FXBT Health, Dar-Es-Salaam
Data collection dates
April - May 2017 April - May 2018 June - August 2018
Geographicdistribution
All 6 programmedistricts
3 states, 8 districts, 56 program blocks
All 3 programmedistricts
Sources of data 100 villages1594 households54 schools18 health facilities
65 villages1051 households 66 schools25 health facilities
73 villages 1185 households 46 schools31 health facilities
Findings from household data collection
11/16/2018 30
Learning beyond ODF: Objectives of core analyses of outcome surveys
To describe key sanitation and hygiene outcomes of Global Sanitation Fund in study households and public facilities
To describe emerging process and outcome indicators of the program, including exposure to program activities, social norms, and habits of latrine use
To describe the needs of marginal and vulnerable households/population, and assess equity and non-discrimination by disaggregation of the key indicators by equity variables (e.g., geographic area, demographics, wealth status)
To assess sustainability of open-defecation free (ODF) status among ODF-verified communities
Access to sanitation facilities in survey populations
11/16/2018 32
*Data from Malawi did not allow for differentiation between Basic and Limited sanitation facilities
Reported information on latrine use (> 5 years old) and child feces disposal
11/16/2018 33
38
50
4
15
97
7
53
41
94
14
76
15
0 20 40 60 80 100
Person >5 years always uses the latrine
Person >5 always always or sometimesdefecate in the open
Feces of child <5 put in toilet
Child <5 uses the toilet
Tanzania Malawi India
Learning beyond ODF: Objectives of core analyses of outcome surveys
To describe key sanitation and hygiene outcomes of Global Sanitation Fund in study households and public facilities
To describe emerging process and outcome indicators of the program, including exposure to program activities, social norms, and habits of latrine use
To describe the needs of marginal and vulnerable households/population, and assess equity and non-discrimination by disaggregation of the key indicators by equity variables (e.g., geographic area, demographics, wealth status)
To assess sustainability of open-defecation free (ODF) status among ODF-verified communities
Exposure to GSF program activities
Malawi India Tanzania
(N=1065) (N=1835) (N=1485)
Participated in transect walk 53% 7% 6%
Participated in creating map of feces in the environment
39% 3% 2%
Reported that someone came to the home to talk about using a toilet
82% 34% 70%
Participated in at least one programme activity
88% 44% 85%
11/16/2018 35
Social norms of sanitation
Personal normative beliefs
Empirical expectations
Normative expectations
Bicchieri, Norms in the Wild, 2017
11/16/2018 36
Empirical Expectations: I see other peopleusing the latrines
11/16/2018 37
Malawi India Tanzania
Males
(N=675)
Females
(N=1328)
Males
(N=1051)
Females
(N=784)
Males
(N=352)
Females
(N=1132)
Neighbors regularly defecate in the
open/field
Some/most/all of them 21% 23% 89% 88% 53% 29%
None of them 78% 74% 11% 12% 47% 71%
How many of your neighbors use a
toilet / latrine?
All of them 81% 82% 15% 14% 47% 54%
Most of them 19% 18% 85% 86% 53% 46%
Empirical expectations score (IQR)
(total possible score of 2)2 (2,2) 1 (0, 2) 1 (0,2)
11/16/2018 38
Empirical expectations of sanitation behavior
Normative Expectations: Other people care strongly about me using the latrine
11/16/2018 39
Malawi India Tanzania
Male
responden
ts (N=675)
Female
respondents
(N=1328)
Male respondents
(N=1051)
Female
respondents
(N=764)
Male
respondents
(N=352)
Female
respondents
(N=1132)
Where do your neighbors think you should defecate? In the latrine / toilet
98% 97% 90% 88% 97% 97%
How many of your neighbors would agree with the statement: Everyone should use a toilet / latrine?
All of them
87% 84% 50% 52% 64% 70%
How many of your neighbors would agree with the statement: It is fine for everyone to defecate in the
open? None of them
86% 81% 63% 61% 77% 79%
Median normative expectations
score (IQR)
(total possible score of 5)
5 (5,5) 3 (2, 4) 5 (4,5)
11/16/2018 40
Normative expectations of sanitation behavior
Habit Definition and Measurement
• Questions developed based on work in social psychology (Verplanken & Orbell, 2003)
• A habit is a regular tendency or practice that a person:
1. Typically does (i.e, "repetition");
2. Does so without planning (i.e., "automaticity"), and;
3. Will feel uncomfortable not doing so (i.e., "identity").
11/16/2018 41
https://images.mentalfloss.com/sites/default/files/styles/mf_image_16x9/public/534067-istock-637130216.jpg?itok=WYcdFb71&resize=1100x1100
Verplanken, B., & Orbell, S. (2003). Reflections on Past Behavior: A Self-Report Index of Habit Strength. Journal of Applied Social Psychology, 33(6), 1313–1330. https://doi.org/10.1111/j.1559-1816.2003.tb01951.x
Latrine use habit
Malawi India Tanzania
N=2003 N=1835 N=
Where do you usually go when you need to defecate? Always use
the latrine (repetition)
97% 42% 98%
Is using the latrine when you need to defecate something you
have been doing for many years? Yes (repetition)
98% 76% 96%
Is using the latrine when you need to defecate something that
you do without having to plan or think about? Yes
(automaticity)
60% 77% N/A
Is going to the bush when you need to defecate something that
you do without having to plan or think about? Yes
(automaticity)
25% 76% N/A
Is using the latrine when you need to defecate something you
would find hard not to do? Yes (identity)
89% 54% N/A
Median habit score (IQR) (total possible score of 6) 5 (5,6) 2 (1.3) N/A11/16/2018 42
Learning beyond ODF: Objectives of core analyses of outcome surveys
To describe key sanitation and hygiene outcomes of Global Sanitation Fund in study households and public facilities
To describe emerging process and outcome indicators of the program, including exposure to program activities, social norms, and habits of latrine use
To describe the needs of marginal and vulnerable households/population, and assess equity and non-discrimination by disaggregation of the key indicators by equity variables (e.g., geographic area, demographics, wealth status)
To assess sustainability of open-defecation free (ODF) status among ODF-verified communities
Assessing equity and non-discrimination
• Gender
• Age
• Mobility and/or visual limitations
• Wealth
• Sub-national region
Latrine access to different categories of household members: Malawi
Region Wealth
Ntchisi District Rumphi District Lowest quintile Highest quintile
Girls 56% 75% 59% 70%
Boys 91% 88% 82% 95%
Men 96% 89% 87% 95%
Women 99% 97% 98% 97%
Latrine access to different categories of household members: India
Region Wealth
Assam Bihar JharkhandLowest
quintile
Highest
quintile
Girls 63% 85% 80% 69% 79%
Boys 70% 88% 91% 81% 81%
Men 84% 89% 96% 82% 88%
Women 85% 91% 98% 88% 89%
Equity and non-discrimination: persons with disability do not report difficulty in access in Malawi
• A household member with limited mobility or vision reported in 64 households (4%)
• 26 persons available and consented for interview• 23 reported being able to use the latrine every time necessary
• 24 reported privacy when using the toilet
• 15 reported being involved in toilet type decision of the household
Equity and non-discrimination: elderly and persons with disability do not report difficulty in access in India
ReportedPersons with visual or mobility
limitations (N=28)
Persons > 65 years old
(N=197)
Being able to use the latrine every
time necessary14 103 (92%)
Feeling very safe when using the
latrine during the night12 80 (72%)
Privacy when using the toilet 14 105 (94%)
Being involved in toilet type decision
of the household6 72 (59%)
Can use latrine without help from any
other person12 (75%) 102 (91%)
Learning beyond ODF: Objectives of core analyses of outcome surveys
To describe key sanitation and hygiene outcomes of Global Sanitation Fund in study households and public facilities
To describe emerging process and outcome indicators of the program, including exposure to program activities, social norms, and habits of latrine use
To describe the needs of marginal and vulnerable households/population, and assess equity and non-discrimination by disaggregation of the key indicators by equity variables (e.g., geographic area, demographics, wealth status)
To assess sustainability of open-defecation free (ODF) status among ODF-verified communities
Sustained ODF status among ODF-verified communities: findings from Malawi
92 of 100 villages had been previously declared ODF, comprising 1467 households
Criteria for ODF:
• Households with access to a sanitation facility belonging to the respondent
• Households with a facility that ensures privacy
• No human feces observed in household area / compound
Adherence to ODF criteria, among households in previously ODF-declared villages, Malawi
Access to sanitationfacility
Facility ensuresprivacy*
No human fecesobserved
0
10
20
30
40
50
60
70
80
90
100
Overall Poorest Second Middle Fourth Wealthiest
Sustained ODF status among ODF-verified communities: findings from India
35 of 65 (54%) villages had been previously declared ODF, comprising 574 (55%) households
June 2015: Ministry of Drinking Water and Sanitation definition of ODF"ODF is the termination of fecal-oral transmission, defined by • a) no visible feces found in the environment/village; and • b) every household as well as public/community institutions using safe technology
option for disposal of feces”*
*”Safe technology option means no contamination of surface soil, ground water or surface water; excreta inaccessible to flies or animals; no handling of fresh excreta; and freedom from odour and unsightly condition)”
http://swachhbharatmission.gov.in/sbmcms/ writereaddata/images/pdf/Guidelines/Guidelines-ODF-Verification.pdf
Adherence to ODF criteria, among households in previously ODF-declared villages, India
Access to latrine Fly proof latrine Latrine with safeseptage disposal
Soap and water in ornear the latrine
0
10
20
30
40
50
60
70
80
90
100
Overall Assam Bihar Jharkhand
A high bar for ODF status is difficult to sustain
Malawi
India
0
10
20
30
40
50
60
70
80
90
100
All households meetnational ODF criteria
<1 non-adherenthousehold
<2 non-adherenthouseholds
<3 non-adherenthouseholds
211
17
70
0
46
8694
% o
f co
mm
un
itie
s m
eeti
ng
OD
F ad
her
ence
cri
teri
a
Summary of the Findings
Item Malawi India Tanzania
Access to Basic Sanitation
15% of population*
48% of population 69% of population
ReportedLatrine Use
97% of household members
38% of household members
94%
Exposure** 88% 44% 85%
SocialNorms***
EE score = 2 (2,2)NE score = 5 (5,5)
EE score = 1 (0,2)NE score = 3 (2,4)
EE score = 1 (0,2)NE score = 5 (4,5)
Habits Scores Median (IQR) = 5 (5,6)
Median (IQR) = 2 (1,3)
N/A
11/16/2018 55
*Included both Limited and Basic sanitation **Participation in at least one programme activity among respondents*** Median (IQR); EE = Empirical Expectation; NE = Normative Expectation;
Summary of the Findings
Item Malawi India Tanzania
Latrine access equity and non-discrimination
Girls had lowest level of access;
Persons with disability do not
report difficulty in access
Girls had lowest level of access
Elderly and persons with disability do not
report difficulty in access
Analysis on-going
Sustained ODF status among ODF-verified communities
>= 98% of interviewed
households had no human feces in
the vicinity
Access to latrine in previously ODF-
declared villages ~ 55%
Analysis on-going
11/16/2018 56
Implications
•Developing and validating indicators is a process, not an endpoint•We’re still learning what are the best questions
to ask
•Construct validity of the data•How much do the answers actually tell? •Use of hypothetical scenarios and vignettes•Feasible methods for quantitative assessment
11/16/2018 57
Evaluation and Research Agenda
•Evaluation agenda•Multiple-countries comparisons•Comparison with previous outcome surveys•Collaboration with other actors in the WASH sector (e.g., UNICEF)
• Research agenda•Harvest the potential of multiple-countries harmonized surveys•Social norms in 13 countries•Structured observation in 13 countries
11/16/2018 58
CollaboratorsGlobal Sanitation Fund – Water Supply and Sanitation Collaborative CouncilRhiannon James, Carolien Van der Voorden, Matteus Van Der Velden, Rita Bonomally, Valerie Varela
Plan MalawiMike Khoza, Thoko Kaitane
The Malawi PolytechnicKondwani Chidziwisano, Limbani Kalumbi, Save Kumwenda, Khumbo Kalulu
Centre for Media StudiesAnisur Rahman, Vinod Mishra, Alok
Plan TanzaniaNyanzobe Malimi, Joyce Massile,
FXBT ResearchCharles Matiko, Donata Didas
University at BuffaloMoshood (Lanre) Omotayo, Kimmy Giacalone, Jia Hua, Megan Yoerg, Ryan Muldoon, Kimberly Giacalone,Pavani K. Ram 59
THANK YOU. ANY QUESTION?
11/16/2018 60
Achieving sustainable and equitable WASH outcomes at
scale
WSSCC-3ie supported evaluations and systematic reviews: what does the
evidence say?
Radhika Menon, 3ie
UNC Water and Health conference, Chapel Hill
1 November 2018
Assessing the impact of sanitation on psycho-social stress of women in Bihar, India
• Evaluation: Before and after ethnographic study of GSF-supported programme (phased out) in Gopalganj and PaschimChamparam districts, Bihar
• -1200 women surveyed at baseline, 41 interviews at baseline and 33 at endline
Context: Government led Swachh Bharat Mission (SBM) aims to end open defecation in India
Authors: Kathleen O’ Reilly et al., 2018
Conceptual framework
Findings: Bihar mixed-method study
Study showed that when intervention was successful there was impact on sanitation coverage and use, and women’s pyscho-social stress
In line with baseline findings which showed a 48% reduction in sanitation-related psycho-social stress scores among women with access to a sanitation facility
Findings
Implementation-related findings
• Within larger context of SBM:
Differences in contractor-built vs. self-financed latrine quality
Mukhiya-contractor nexus that may help build toilets but also encourages corruption, poor service delivery, focus on selected areas
Implications for policy: going to scale- Contractor-built latrines require standards
for usability that need to be monitored and enforced
- Improved information about self-financed latrines reimbursement requirements, role of SHGs
- More consistent community-based mobilisation and demand-generation activities for different groups
•
- Programme design and M&E should incorporate elements related to women’s satisfaction with latrine design and construction, and look at sanitation-related psycho social stress
What are systematic reviews?
Systematic reviews examine all the available existing evidence on the effectiveness of a particular intervention or programme.
WSSCC-3ie supported systematic reviews
Systematic review of handwashing and sanitation behavior change interventions
• Interventions to promote sanitation and handwashing behavior-change include community-based approaches, social marketing, messaging and theory-based approaches – De Buck et al. 2017
• Review synthesises evidence from 42 quantitative studies on effectiveness of behavior change approaches and 28 qualitative studies on implementation, focused on progs in 24 L&MICs
Systematic review findings
• Community-based approaches were effective in reducing open defecation and improving latrine use.
• Social marketing approaches in combined handwashing and sanitation programmes may improve latrine use but effect on handwashing unclear
• Implementation factors: facilitator, attitude of implementer, trust and cooperation
Systematic review findings• Sanitation and hygiene messaging may
improve handwashing in the short term, but no impact on open defecation behaviour or safe faeces disposal.
Further research is needed on theory-based approaches using elements of psychosocial theory.
Using interpersonal communication is effective in certain circumstances
Systematic review of the life-cycle approach in WASH
• Examines evidence from WASH policies, programmes and projects in 11 countries in South Asia and Sub-Saharan Africa during the MDGs – Annamalai et al. 2017
• The authors synthesised evidence from 59 policy documents and 131 programme and project documents
• Review uses a life-cycle lens to examine WASH portfolio, in comparison to geographic and socio-economic targettingstrategies
Life cycle approach to sanitation
Systematic review findings
• Shift towards using the life-cycle approach occurred during the MDG period in Africa and Asia
• Women were the focus of a relatively large number of policies, followed by children
However, life cycle segmentation is not always translated from policies to programmes and projects
Life cycle systematic review: findings• Robustness index higher for
geographic and socio-economic than life-cycle approach
• Life-cycle benefits were included more often in policies related to sanitation and hygiene than for water, despite impact on women
Programme and projects funded by multi-lateral agencies aided adoption of life-cycle segments during implementation, community participation also played an enabling role
What does the evidence say?
More WASH programmes and evaluations that draw on behavioural science
Evidence suggests we need a multi-pronged intervention strategy to improve outcomes
More evaluations looking at long-term use
More implementation research on programmes at scale
Systematic steps needed to incorporate life-cycle and gendered principles in targetting approach, programmedesign