Summative Evaluation of the Enhanced and Accelerated WASH ... · Summative Evaluation of the Enhanced and Accelerated WASH Programmes ACCELERATED Programme Evaluation – Final Report

18-093236-01 | Version 4 | Internal Use Only | This work was carried out in accordance with the requirements of the international quality standard for Market Research, ISO 20252:2012, and with the Ipsos MORI Terms and Conditions which can be found at http://www.ipsos-mori.com/terms. © UNICEF 2020

Summative Evaluation of the Enhanced and Accelerated WASH Programmes ACCELERATED Programme Evaluation –

Final Report

Scope: March 2015 to June 2019 30th January 2020, commissioned by UNICEF in Ghana

Ipsos MORI & FCG Sweden

Ipsos MORI & FCG Sweden | Accelerated Programme Evaluation Final Report





Contents Executive Summary ........................................................................................................................ 6 1 Introduction .............................................................................................................................. 13

1.1 The Accelerated WASH programme ............................................................................................................. 13 1.2 Background to the programme ..................................................................................................................... 14 1.3 Revised Theory of Change .............................................................................................................................. 15 1.4 Evaluation purpose, objectives and scope ................................................................................................... 18 1.5 Evaluation Criteria ........................................................................................................................................... 18 1.6 Evaluation Questions ....................................................................................................................................... 19 1.7 Methodology .................................................................................................................................................... 20 1.8 Challenges and limitations ............................................................................................................................. 25

2 Evaluation Findings & Preliminary Conclusions (by Criterion) ........................................... 28 2.1 Relevance .......................................................................................................................................................... 29 2.2 Effectiveness ..................................................................................................................................................... 35 2.3 Efficiency ........................................................................................................................................................... 78

3 Final Conclusions ...................................................................................................................... 86 4 Lessons Learned ......................................................................................................................... 88 5 Recommendations ..................................................................................................................... 90 Annexes .......................................................................................................................................... 99

List of Figures

Figure 1: Programme timeline ....................................................................................................................................... 13 Figure 2: Revised Theory of Change ............................................................................................................................. 17

List of Tables

Table 1: Summary of recommendations ...................................................................................................................... 12 Table 2: Key evaluation questions for the Accelerated programme ........................................................................ 19 Table 3: Interviews completed by stakeholder type for the Accelerated programme ......................................... 23 Table 4: Focus groups for the Accelerated programme ............................................................................................. 24 Table 5: Key evaluation questions for the Accelerated programme related to relevance ................................... 29 Table 6: Key evaluation questions for the Accelerated programme related to effectiveness ............................. 35 Table 7: ODF communities in Ghana by implementing organisation ...................................................................... 61 Table 8: Key evaluation questions for the Accelerated programme related to efficiency ................................... 78 Table 9: Strategic recommendations ............................................................................................................................ 91 Table 10: Operational recommendations ..................................................................................................................... 97



Table of Acronyms

BaSIS – Basic Sanitation Information System

CLTS – Community-led total sanitation

CONIWAS - Coalition of NGOs in Water and Sanitation

CSO – Civil Society Organisation

CWSA – Community Water and Sanitation Agency

DAC – Development Assistance Committee

DCD – Department for Community Development

DCE - District Chief Executive

DICCS – District Inter-Agency Coordinating Committee on Sanitation

DHS – Demographics and Health Surveys

DRP – District Resource Person

EHSD – Environmental, Health and Sanitation Directorate

EMIS – Education Management Information System

GAC – Global Affairs Canada

GES – Ghana Education Service

GHS – Ghana Health Service

GoG – Government of Ghana

HHETPS – Health and Hygiene Education Through Play and Sports

HWWS – Handwashing with Soap

JMP - WHO/UNICEF Joint Monitoring Programme

KEQ – Key Evaluation Question

KNUST – Kwame Nkrumah University of Science and Technology

KPM – Key Performance Measurement

MDGs – Millennium Development Goals

MICS - Multiple Indicator Cluster Surveys

MHM – Menstrual Hygiene Management



MMDA – Metropolitan, Municipal and District Assemblies

NCWSP – National Community Water and Sanitation Programme

NDPC – National Development Planning Commission

NESSAP – National Environmental Sanitation Strategy and Action Plan

NGO – Non-governmental organisation

OD – Open defecation

ODF – Open defecation free

OECD – Organisation for Economic Cooperation and Development

O&M – Operations and Maintenance

RICCS – Regional Inter-Agency Coordinating Committee on Sanitation

RSMS – Rural Sanitation Model and Strategy

SDGs – Sustainable Development Goals

SHEP – School Health Education Programme

SIS – Sector Information System

SKMI – Sanitation Knowledge Management Initiative

STSM – Small Town Sanitation Model

ToR – Terms of Reference

ToC – Theory of Change

WASH – Water, Sanitation and Hygiene

WSUP - Water and Sanitation for the Urban Poor

WinS – WASH in Schools

Ipsos MORI & FCG Sweden | Accelerated Programme Evaluation - Final Report 6


1. Introduction

UNICEF in Ghana supported implementation of the Accelerated Water, Sanitation and Hygiene (WASH) programme from March 2015 to June 2019 in Ghana’s Northern region, where access to WASH-related services and related health outcomes are among the lowest in the country. At the start of the programme, only 68% of households and 37% of schools had access to improved water services in the Northern region. Additionally, open defecation was widely practiced (by 72% of the population). Subsequently, the Accelerated programme proposal identified three specific objectives: 1) to increase equitable access to suitable household and institutional sanitation facilities; 2) to facilitate the adoption of good hygiene practices for improved health; and 3) to increase capacity at the community, district and regional levels to support sanitation service delivery and engage the private sector in the sanitation market.

The UNICEF Ghana Country Office commissioned Ipsos MORI, together with its partner FCG Sweden, to undertake an independent evaluation of the Accelerated WASH programme.

2. Purpose and objectives

This evaluation has a double-fold purpose: accountability (to both donors and expected beneficiaries) and organisational learning (both amongst UNICEF staff and their governmental counterparts). With that in mind, the objectives of this evaluation are to determine the extent and quality of the programme’s achievements as well as to identify the lessons learned and put forward key recommendations to the evaluation key users to support future programming, including: government at all levels. UNICEF’s Ghana country office, Global Affairs Canada (GAC), project implementing partners, and the WASH sector in general (including other donors and international organisations, civil society, and the private sector).

3. Scope (thematic, geographical and chronological)

As set out in the Terms of Reference (ToR) for the evaluation, the scope of this evaluation is the Accelerated Programme. The geographic scope of the evaluation encompasses all areas targeted by the Accelerated programme; namely, Ghana’s Northern region. It encompasses all WASH activities delivered under the programme but assesses results of the programme as a whole, rather than each activity individually, and primarily considers the results at a programme-wide level, as well as by urbanity (i.e. rural communities versus small towns). It covers programme implementation through its entire timeline, i.e. from March 2015 to June 2019.

4. Criteria and Evaluation Questions

This evaluation was guided by three of the five Organisation for Economic Cooperation and Development’s Development Assistance Committee (OECD DAC) evaluation criteria (relevance, effectiveness, and efficiency) to address the 26 evaluation questions set out in the evaluation framework. Equity, Gender and Human Right issues were taken into consideration throughout the evaluation, from the inception phase until the data analysis and write-up of the recommendations.

Executive Summary



5. Methodology

The evaluation followed a mixed-methods approach and rested on the combination of five different data collection methods, namely: programme document review, secondary data review, household survey and structured observation, stakeholder interviews and school focus groups. The evaluation team also accessed available secondary databases to gather information on WASH indicators. For the household survey and structured observation, the evaluation team surveyed 543 households, selected using a two-stage clustered sample design. Key informant interviews were carried out with a purposive sample of 38 stakeholders including government bodies at national, regional and district levels; civil society organisations; UNICEF staff; health centre and school staff, and other WASH sector stakeholders. Informant categories and quota ranges for each category were identified by the evaluation team, and key contacts with the greatest knowledge of the programme were identified by UNICEF for each category. Six focus groups were carried out in three schools with students and teachers. Primary and secondary data were triangulated to answer the key evaluation questions as set out in the evaluation framework, using a ‘weight of evidence’ based principle to consider the reliability and validity of each piece of evidence to minimise bias. A contribution analysis approach was applied to assess the programme’s contribution to results relative to other explanations and contextual factors.

Key limitations of the evaluation include a lack of programme targets for some of the outputs and outcomes included in the Theory of Change, against which progress could have been assessed, as well as a lack of baseline data in some cases. Some stakeholder groups were unavailable for interview. Data has been disaggregated by gender and by small towns and rural communities, although more granular geographic analysis was not possible given the survey’s sample size.

6. Key findings

Relevance

The evaluation team found that the Accelerated programme has been designed following the key provisions of the country’s national WASH policies and strategies, which aligned to the targeted outcomes of the Sustainable Development Goals (SDGs) and the specific needs of the country to meet the SDGs. Similarly, programme design had a deliberate emphasis on improving access to gender-friendly WASH services and encouraging women’s social and economic empowerment – through which it was aligned with Goal 5 of the SDGs.

Stakeholders, too, felt that programme design has been well aligned with national priorities and Ghana’s key WASH sector strategies and that the programme as a whole has raised the importance of WASH in Ghana’s political agenda. Furthermore, it has been well aligned with the guidelines stipulated in the Rural Sanitation Model and Strategy (RSMS) and has adapted these for application in small towns through creation of the Small-Town Sanitation Model (STSM). Stakeholders also noted that programme design and delivery have been adapted throughout programme implementation in response to lessons learned.

Effectiveness

Evidence from this evaluation suggests that the Accelerated programme has contributed to a notable change in behaviour around open defecation, led to a drive of latrine building, and subsequently contributed to reduced rates of open defecation in both small towns and rural communities. These findings suggest that the Community-Led Total Sanitation approach – which had thus far mostly been implemented in rural communities – can be effectively applied to small towns through the STSM.



Whilst the programme did not reach its target of five districts achieving Open Defecation Free (ODF) status, significant progress was made in this area: over 70% of communities in five districts became certified ODF. The evaluation’s endline household survey found that the proportion of communities practicing open defecation had halved since baseline in both small towns (32%, down from 63%) and rural communities (33%, down from 68%). In this vein, the programme contribution is quite apparent given that open defecation has reduced more substantively in the communities that participated in it than across the Northern region as a whole. The programme also contributed to a certain increase in access to facilities being accompanied by a change in behaviour. This is attested by the fact that the majority of survey participants who used a toilet (63% in small towns and 84% in rural communities) reported that their toilets were built after the programme commenced its activities. The evaluation team also observed that 80% of observed household or compound latrines in small towns and 90% in rural communities are still in active use. To the contrary, most of those who still practiced open defecation declared that they do so mainly due to lack of access to public or private toilets.

The data showed that gender and disability friendliness, use of public latrines and access to improved facilities was higher in small towns than in rural communities. Access to private facilities (where the household has exclusive use of a facility) was higher in rural communities than in small towns1.. In order to move people up the sanitation ladder, support is therefore needed to improve affordability of household toilets especially in small towns – where some community members without access to artisan-supplied latrines still use public toilets, compared to rural communities, where many build their own latrines.

Progress is mixed in terms of the number of people who are aware of, and practicing, good hygiene behaviours. Although the programme achieved 86% of its target for the number of people reached with messaging on good hygiene, the survey evidence found a gap between knowledge and practice – particularly in rural communities. Participants largely understood that soap was effective in killing germs, however fewer participants knew about the link between germs, handwashing, and disease. Endline participants in rural communities were more aware of the importance of handwashing than those in small towns. Despite this reported knowledge base, upon observation over a third of participants who claimed to have washed their hands with water on the previous day had no water or soap available at the designated place for handwashing. Even though there appears to be good understanding of the most effective mode of handwashing – with most participants using running water rather than a jug or bowl, few of them washed their hands at critical times, for example after disposing of a child’s stools.

There has been no improvement from baseline in the proportion of women safely managing their menstruation in small towns, though there has been a notable improvement in rural communities – where the baseline figure was particularly low. While a majority of female participants were using appropriate materials for menstrual hygiene management, women and girls rarely disposed of their sanitary products safely, with only 3% of female participants in small towns and no female participants in rural communities disposing of their menstrual hygiene products in a special bin.2

Targets for provision of WASH infrastructure in schools and health centres were largely met; in schools, 93 sanitation facilities were constructed and 42 rehabilitated (with reference to a target of 120), and in health centres, 58 sanitation facilities were constructed (with reference to a target of 60). Handwashing facilities were provided in 106 schools and 56 health centres (with reference to a combined target of 180). Despite separate facilities being for boys and girls, the evaluation team found that these were deteriorated in some cases. Interviewed health centre staff attributed this to outsider use of their WASH facilities, causing damage and making it difficult for students and staff to maintain them. This, in turn, was due to a lack of

1 There was no statistically significant difference in the number of households in rural communities and small towns using privately managed shared facilities. 2 With reference to the UNICEF and Ghana Education Service guide to menstrual hygiene management.



private or public facilities within the communities participating in the programme, as confirmed by the significant portion of the community that still practice open defecation.

Schools and health centres were also reached with hygiene behaviour-change activities. Staff in 84 health centres were trained in hygiene promotion (with reference to a target of 80), and in schools, 130 hygiene promotion programmes have been established (with reference to a target of 160). Staff interviewed from health centres, however, had not received training and found it difficult to encourage visitors to change their hygiene practices.

More positively, school health clubs were active in all three schools visited and were regularly sharing hygiene and sanitation information within the school, as well as being engaged in maintaining the facilities. They also disseminated information in their communities, although there were some challenges in feeling empowered to do so without support of authority figures. The programme exceeded its targets in terms of the number of school girls reached with training on menstrual hygiene management, with 20,066 schoolgirls receiving training (with reference to a target of 16,000), and girls who participated in focus groups described that in some cases, their schools were now going beyond the programme’s expectations by providing girls with sanitary materials. However, issues around knowledge and attitude about menstruation persisted: girls who participated in focus groups seemed largely unaware about how to calculate menstrual cycles, and in some cases, boys reportedly made fun of girls during their menstruation.

Although water access activities mainly focussed on advocacy and awareness among policymakers (with no target for direct water supply activities for communities), it is important to note that access to drinking water is poor in the Northern region due also to issues related to the hydrogeological conditions that make access to groundwater difficult, thus limiting availability of clean drinking water – particularly in the dry season. As a result, a significant proportion of survey participants still relied on surface water – especially in rural communities and sources of drinking water are generally located far away from people’s houses with the majority of survey participants having to travel half an hour or more to access clean water, and women and girls remaining the primary collectors of drinking water in both small towns and rural communities.

The programme implemented the Community-Led Total Sanitation approach in small towns for the first time, developing a manual for this approach and training government staff at all levels to implement this. The training provided was considered effective. In particular, capacity at district level was reinforced through the provision of District Resource Persons, who helped the district with programme management and monitoring. In general, the programme was considered to have been successful in motivating government stakeholders at all levels to reach ODF targets and giving them the necessary technical, management and monitoring skills to do so. However, district authorities still face constraints in available human resources and logistical support to ensure the continuity of the CLTS approach. Government stakeholders highlighted the information and knowledge shared among stakeholders involved in the implementation of the Accelerated programme, through the programme’s efforts to strengthen existing platforms.

There have been limited changes in the health of people living the targeted communities. The household survey showed a slight decrease in the percentage of children suffering from diarrhoea in the targeted communities (not significant). Moreover, stakeholders highlighted that the programme’s sector-wide impact in making policies available for the wider sector and that it has raised the importance of WASH in the political agenda.

Efficiency

Most programme targets have been met within the programme timeframe, although the programme experienced delays in bringing new districts into the programme, building capacity to implement the CLTS approach in small towns, and building



capacity to implement the ODF verification protocol. Some of the targets for beneficiary reach – particularly reaching an additional 500,000 people with improved sanitation – were considered in retrospect to have been based on a poor estimation of the population in the target area. Targets for provision of water facilities in institutions were missed by a small margin, but this is due to challenging conditions in the Northern region that make borehole drilling difficult. Capacity-development targets have also been mostly achieved as well.

The programme’s overall approach of strengthening government and the private sector to deliver WASH services is cost-effective and supports longer term sustainability of results. It has been particularly cost-effective when it comes to the construction of household latrines, with the number of latrines constructed delivered for around the same cost that it is estimated would have been required for private construction. Construction of WASH facilities in schools and health facilities and handwashing activities were also cost-effective when compared to available benchmarks. There may have been under-investment, however, in training for operations and maintenance of health centre WASH facilities.

7. Key Conclusions

The Accelerated programme remained relevant to the national and international policy context and the needs of the target population. The programme helped shape the policy environment by building the knowledge base on CLTS, contributing new policies to support WASH sector programming, and raising the prominence of WASH in the national policy dialogue. It did so by engaging government at all levels and developing guidance and policies that could be used more broadly. The shift to the SDGs, however, has also raised the importance of water access as a development priority which future programmes will need to address, particularly in the Northern region. There is also a need for better faecal sludge management systems in small towns, although it was not an aim of the programme to address either water access or faecal sludge management.

The majority of programme targets were met, accompanied by positive changes in behaviour at community level, especially a reduction in open defecation to around half the level at baseline. The CLTS approach efficiently mobilised household funding for latrine construction. Although hygiene knowledge has improved, behaviour has not improved to the same extent. In terms of the programme’s ultimate impact on health, regional-level health targets were not achieved, although the survey found evidence of a decrease in children suffering from diarrhoea; external factors mean this cannot be attributed solely to the programme.

The programme’s consideration of the specific sanitation and hygiene needs of women and girls was an important and commendable innovation in WASH programme delivery in Ghana. However, marginalised groups continue to experience unmet needs for WASH services, and there are variations in access between small towns and rural communities.

The programme was effective in providing WASH facilities in institutions through new constructions and rehabilitation of existing facilities, meeting its targets for new facilities, and it has done so efficiently in comparison to benchmark costs. These were mainly provided in small towns rather than rural communities. Poor maintenance of facilities is a key risk to the current benefits of the programme as well as the longer-term sustainability of the facilities, and there may have been an inefficient under-investment in training on operations and maintenance for health institutions’ staff.

The programme has also been effective in strengthening government’s capacity and motivation to meet Ghana’s WASH needs. Ongoing reinforcement of capacity built and further strengthening of accountability among MMDAs are likely to be required to ensure the sustainability of the programme’s results.



8. Lessons Learned

The evaluation team identified the following key learnings for future programming:

1. The CLTS approach can be applied effectively in small towns, although local stakeholders may have less prior knowledge of this approach, requiring additional time to build awareness and capability to deliver it.

2. Achievement of progress under the CLTS approach can be hindered by the detrimental impact of blanket subsidies for WASH infrastructure.

3. Strengthening government capacity to deliver WASH has been a central activity of the programme, and significant progress has been made, but continued strengthening and maintenance of capacity is required to build on this and to sustain the progress that has been achieved, especially where authorities were starting from a lower base of awareness on CLTS.

4. WASH facilities constructed in institutions are vulnerable to misuse and damage by community members who lack access to household latrines, or whose latrines are not high-quality, in communities that where open defecation is practiced. Particularly in this context, maintenance is a challenge, and training of institutions’ staff on maintenance as well as handover need to be adequately supported.

5. Accountability among duty-bearers is an important precursor for achieving success from the CLTS approach, and this can be built while at the same time their capacities to deliver the approach is strengthened.

6. The ability of the evaluation team – and UNICEF – to measure the results of the programme was limited by the lack of a comprehensive results framework that fully reflected the programme’s Theory of Change.

7. Water access remains an important issue in the Northern region, although provision of water services was not one of the key activities of the Accelerated programme, and this is exacerbated both by the Northern region’s hydrogeological conditions and inconsistent quality of drilling feasibility studies.

9. Recommendations

Based on the different findings and conclusions, the evaluation team put forward a number of recommendations (strategic and operational) whose objective is to contribute to programme future improvements. These recommendations were validated during several exchanges held between the evaluation team and UNICEF Country Office staff as well as other stakeholders. These exchanges focused on two particular aspects of the recommendations, namely their feasibility and relevance to the implementation context. This process also contributed to the prioritization of the recommendations in question.



Table 1: Summary of recommendations

Strategic recommendations Operational recommendations

1. Future programmes should continue building government capacity and accountability to deliver the CLTS approach. (All stakeholders)

2. Future programmes should continue the joint roll-out of supply-side and demand-side interventions and should continue to focus on the affordability of household toilets in small towns. (All stakeholders)

3. There needs to be a clear national policy set by government against provision of blanket subsidies. (Government and other WASH sector stakeholders)

4. The WASH sector should intensify efforts to improve access to clean water at the community and institutional levels in the Northern region. (All stakeholders, especially government)

5. There is a need for a systematic approach to ensure continuous messaging on hygiene, with greater follow-up of training delivered. (All stakeholders)

6. Future WASH programming in Ghana should incorporate a gender- and disability-inclusive approach in CLTS. (All stakeholders)

7. Future programmes should contribute to piloting and tailoring the faecal sludge management strategy, ensuring its final implementation as a national approach. (All stakeholders)

1. The handover of constructed facilities from contractors to GHS/GES and finally to institutions should be improved. Handovers should take place as soon as facility construction is complete. (GHS, GES, CSWA and contractors)

2. Logistical support (particularly at district level) should be strengthened in future WASH programmes to ensure hard-to-reach populations are included in programming. (MMDAs and donors)

3. Future UNICEF WASH programming under the GoG-UNICEF Programme should include a more holistic set of indicators at the output and outcome levels. (UNICEF and donors)



UNICEF commissioned Ipsos MORI, together with its partner FCG Sweden, to undertake an independent evaluation of the Enhanced and Accelerated Water, Sanitation and Hygiene (WASH) programmes in Ghana after six years of implementation. This final report sets out the preliminary findings of the evaluation of the Accelerated WASH Programme only (a second report was produced to present the key findings, conclusions and recommendation related to the Enhanced portion of the programme) The evaluation was carried out following development of an Inception Report, which sets out the context and methodology for the evaluation in detail, and which was agreed with UNICEF in Ghana.

0.1 The Accelerated WASH programme

UNICEF’s cooperation with the Government of Ghana (GoG) is set out in the Basic Cooperation Agreement and the UNICEF Country Programme of Cooperation. The Accelerated WASH Services in Schools and Communities in Ghana Programme (the Accelerated programme) lies under the WASH component of the Country Programme of Cooperation and was implemented with financial support from Global Affairs Canada (GAC). The programme’s objectives are to increase the use of safe water supply, to improve sanitation facilities, and promote hygiene-related behaviour change in Ghana.

The government agencies involved in the implementation of the Accelerated programme include: the School Health Education Programme (SHEP) of the Ghana Education Service, the Environmental Health and Sanitation Directorate (EHSD), Department of Community Development, Community Water and Sanitation Agency (CWSA), and Metropolitan, Municipal and District Assemblies (MMDAs).

Implementation of the Accelerated programme began in March 2015 in Ghana’s Northern region, where access to WASH related services and related health outcomes were among the lowest levels in the country. The programme ran until the end of June 2019, and the evaluation ran from December 2018 to December 2019, as demonstrated by the timeline in Figure 1 below.

Figure 1: Accelerated programme timeline

The Accelerated programme proposal had the following objectives:

To increase equitable access to suitable household and institutional sanitation facilities; To facilitate the adoption of good hygiene practices for improved health (including maternal, new-born and child

health); and To increase capacity at the community, district and regional levels to support sanitation service delivery and engage

the private sector in the sanitation market.

March 2015 – Accelerated programme begins

implementation

December 2018 -Accelerated programe

evaluation starts

30th June 2019 –Accelerated programme

finishes

December 2019 -Accelarted programme

evalution finishes

0 Introduction



Overall, the programme aims to support communities to become open-defecation free (ODF) through several activities: the provision and rehabilitation of sanitation and hygiene infrastructure in schools and health centres; behaviour change activities around sanitation, hygiene and clean water at the community level (including schools and health facilities); building the capacity of government and non-government stakeholders; and engaging the private sector in the sanitation market.

To promote behaviour change and the construction of latrines at the household level, the programme used a Community-Led Total Sanitation (CLTS) approach, which incorporates both demand-side and supply-side activities to spur households to construct latrines (rather than subsidising and/or constructing latrines directly for households). Demand for latrines is triggered through communications campaigns, as well as behaviour-change activities, in both communities and schools, and finance-focussed activities strengthened the provision of finance for households to purchase latrine products and services. The programme also focused on the supply side by building the capacities of stakeholders in sanitation and hygiene service delivery at the national, regional, district and community levels. Government departments at the district level are trained to implement and coordinate CLTS activities in communities and institutions by Civil Society Organisations (CSOs). For instance, a business-focussed approach to the delivery of sanitation facilities was adopted, which involved training local artisans on good business practices and delivering sanitation products and services.

Governance and accountability activities were also a key component of the programme and focussed on building government’s capacity at all levels to implement the CLTS approach. These included the incorporation of WASH indicators into government monitoring systems for the health and education sectors, the development and scaling-up of Basic Sanitation Information System (BaSIS) and helping government to track better budgeting and expenditure on WASH.

Further, the programme constructed and rehabilitated latrines in health facilities and schools and provided associated training in operations and maintenance of facilities, as well as in WASH behaviours. The Health and Hygiene Education Through Play and Sport (HHETPS) approach, for example, engages children in edutainment activities around handwashing with soap.

Gender, human rights and equity considerations are considered by UNICEF to be an important feature in the design and delivery of the programmes. There are significant WASH-related gender inequities in Ghana, and the Accelerated programme aimed to address this by promoting gender mainstreaming in the WASH sector and embedding gender equity considerations in its key target outcomes.

0.2 Background to the programme

Annex 6 provides a detail background on the context for the evaluation, including the international and national context. The Accelerated programme commenced when the Sustainable Development Goals (SDGs) were being developed by the international community as a new framework for international development cooperation; in which WASH was given its own goal: SGD6. There was broad consensus among experts and organisations working in the fields of WASH and human rights that the overall vision is universal access to safe drinking water, sanitation and hygiene, seeking to ensure availability and sustainable management of water and sanitation for all by 20303. According to latest SDG progress report 2018, the international community is not on track to achieve SDG 6 by 2030.4

3 UNDP’ Goal 6: Clean water and sanitation’, available at https://www.undp.org/content/undp/en/home/sustainable-development-goals/goal-6-clean-water-and-sanitation.html. 4 SGD Knowledge Platform’ Sustainable Development Goal 6’, available at https://sustainabledevelopment.un.org/sdg6

https://www.undp.org/content/undp/en/home/sustainable-development-goals/goal-6-clean-water-and-sanitation.html


https://sustainabledevelopment.un.org/sdg6



The UNICEF-supported Enhanced WASH programme had been active in five regions of Ghana, including the Northern region, for over three years by the time the Accelerated programme commenced its activities. Although the Enhanced programme was found to be creating tangible benefits for those in (and surrounding) its target communities, coverage did not extend to the whole of the Northern region. Access to WASH services in the Northern region of Ghana remained among the lowest in the country at the outset of the Accelerated programme, while rates of open defecation were among the highest. Subsequently, WASH related health outcomes were poor, illustrating a clear unmet need among those living in the region. At the programme outset, only 68% of households and 37% of schools had access to improved water services in the Northern region.5 Additionally, open defecation (OD) was widely practiced (by 72% of the population) and the under-five mortality rate was higher than in any other region of the country.6

The Accelerated programme was introduced to address these issues, targeting districts of the Northern region that were not covered by the Enhanced programme’s activities. For the Accelerated programme, the CLTS approach was adapted for implementation in small towns with populations of between 2,000 to 30,000 people. These small towns often lacked the clearly defined community governance structures found in rural communities and manage their own water and sanitation systems.

Further, there were significant WASH-related gender inequities in the Northern region. Prior to the implementation of the Accelerated programme, WASH interventions rarely sought to address these differential needs, and gender mainstreaming was largely absent from WASH programming, although the Enhanced programme had begun to address this. Subsequently, in the Northern region, as elsewhere in Ghana, women and girls were often responsible for WASH related domestic duties such as fetching, storing and treating water, while simultaneously being excluded from WASH-related decision making in their communities. The Accelerated programme therefore embedding gender equity considerations – including around promotion gender mainstreaming in the WASH sector – in its key programme targets.

In Ghana, the National Water Policy (2007), Environmental Sanitation Policy (revised 2010) and the Water Sector Strategic Development Plan (WSSDP) 2012-2025 and the National Environmental Sanitation Strategy and Action Plan (NESSAP) (2011), are the key policy and strategic documents that guide the small towns and rural communities WASH sector. WASH service delivery in small towns and rural communities is decentralized in Ghana, with national level institutions providing policy and monitoring frameworks, while local government and communities plan, implement, coordinate, monitor and evaluate services.7 Yet prior to the programme, many MMDAs in Northern Ghana did not have the resources or capacity to implement an effective WASH service delivery, and many did not consider WASH as political priority. The programme therefore sought to improve sanitation and hygiene practices by partnering with the government and strengthening government capacity to deliver WASH policies and strategies.

0.3 Revised Theory of Change

A Theory of Change (ToC) for the Accelerated programme was reconstructed in the Inception Phase for this evaluation and included in the Inception Report. It provides an overall narrative that explains how the programme aims to achieve its intended long-term objectives. It traces the programme inputs through to its intended impacts. This ToC was developed by the evaluation team based on an initial desk-based review of programme documentation and monitoring records and a

5 Accelerated programme proposal (2015) 6 Ghana Statistical Service. Multiple Indicator Cluster Survey 2011 7 The National Community water and Sanitation Programme (NCWSP)



series of familiarisation interviews with key programme stakeholders. It should be noted that most of the targets set in the Key Performance Metrics (KPM) database are at an ‘output’ level. However, as this is a summative evaluation, the evaluation goes beyond this to assess the results i.e. the outcomes and impacts of the programme.

As part of the evaluation activities, the ToC has been revised to better reflect the programme delivery. Community members’ time is recognised as an input. More detailed outputs have been added (e.g. Establishment of Village Savings and Loan Associations - VSLAs). Increased understanding of WASH service implementation in Ghana is linked to increased number of communities with ODF status (outcomes). Increased knowledge of community around sanitation, hygiene and safe water is linked with the number of communities with ODF status (outcomes). This logic model below provides an illustration of the ToC. The full, updated ToC is contained in Annex 1.



Figure 2: Revised Theory of Change



0.4 Evaluation purpose, objectives and scope

0.4.1 Purpose

This evaluation has two purposes: accountability and learning. Specifically, as set out in the Terms of Reference (ToR), the evaluation aims to determine the achievements and effectiveness of the Accelerated Programme and the related Enhanced WASH Programme, in relation to the programmes’ overall aim of providing equitable and sustainable access to water and sanitation services, although this report focusses on the Accelerated Programme only. For both programme evaluations, the purposes of the evaluation are to identify which of the programme’s envisaged objectives were achieved (to support organisational learning), prioritise future activities and strategies for the implementation of more gender-responsive and gender transformative WASH programming in the future (targeted learning), and provide evidence of the effectiveness of the programme for donors and beneficiaries (accountability).

The expected primary users of this evaluation include government at all levels. UNICEF’s Ghana country office, GAC, project implementing partners, and the WASH sector in general. Such users may be interested in identifying learnings for improving WASH programming.

0.4.2 Objectives

As set out in the ToR, this final evaluation report focuses on assessing the relevance, effectiveness, and efficiency of the Accelerated Programme. The evaluation answers whether the programme has contributed to improving sanitation and hygiene for community people and school children, if targets are met, and what could have been done differently to meet targets in the future. In doing so, the evaluation identifies lessons learned, and the evaluation team has developed a series of strategic and operational recommendations for future WASH programme design within and outside of UNICEF support.

0.4.3 Scope

The geographic scope of the evaluation covers rural communities and small towns targeted by the Accelerated programme in the Northern region. UNICEF also indicated an interest in the Inception Phase to understand differences between rural communities and small towns, so these are included where statistically significant differences in survey results were found. The thematic scope covers all activities included within the programme but looks at the results of the programme as a whole, rather than each activity individually, and primarily considers the results at a programme-wide level. The chronological scope is the programme’s entire timeline, from March 2015 to June 2019.

0.5 Evaluation Criteria 0.5.1 Criteria

The evaluation is guided by the Organisation for Economic Cooperation and Development’s (OECD’s) Development Assistance Committee (DAC) criteria of relevance, effectiveness, and efficiency. In the Inception Phase, the evaluation team explored whether impact and sustainability criteria could also be added to the evaluation framework (although these were not required under the ToR). However, it was considered too soon for the evaluation to find evidence of some of the long-term outcomes and impacts set out in the ToC, and that the programme design did not include experimental or quasi-experimental allocation of treatment to communities which would have enabled a robust assessment of impact and attribution of effects to the programmes. In line with the ToR, therefore, the evaluation did not seek to address the DAC criteria of impact or sustainability. Nevertheless, the evaluation team identified findings regarding how sustainability of results can be achieved; these are incorporated under the effectiveness criterion in the findings and preliminary conclusions chapter.



0.6 Evaluation Questions

The evaluation team developed an evaluation framework which is structured around the three evaluation criteria and includes revised evaluation questions from the ToR. The evaluation framework is provided in Annex 2, and the relevant DAC criteria and 26 key evaluation questions (KEQs) are referenced throughout this report and shown in Table 2 below.

Table 2: Key evaluation questions for the Accelerated programme KEQ number Evaluation

framework reference8

Key Evaluation Question (KEQ)

Relevance

1 26 How relevant is the design of the programmes to national and international policies and strategies for WASH?

2 25 Has research been appropriately designed to answer sector knowledge gaps?

Effectiveness - WASH in rural communities and small towns

3 6 What is the access to household sanitation in the targeted regions and districts? Are the facilities in use? Are there variations in access in rural communities versus small towns?

4 7 What do community members and school children know about the hygiene practices which the programme promoted? Are there variations in knowledge in rural communities versus small towns?

5 8 To what extent do community members and school children have appropriate HWWS facilities? Are there variations in access in rural communities versus small towns?

6 9 What is the access to safe water supply (including HWTS) in the targeted rural communities and small towns?

7 10 How is faecal sludge managed in the targeted small towns?

Effectiveness - WASH in institutions (schools and health centres)

8 11 To what extent do school boys and girls have access to school sanitation facilities? What is the functionality status of WASH facilities in schools? Are there variations between rural communities and small towns?

9 12 To what extent are the school sanitation facilities gender- and disability- friendly? Are there variations between rural communities and small towns?

10 13 What proportion of schools have facility management plans in place?

11 14 What is the level of knowledge on menstrual hygiene management and hygiene behaviours?

12 15 To what extent are WASH facilities in health centres and CHPS compounds available? What is the functionality status of WASH facilities in health facilities?

Effectiveness - District level WASH access

13 16 How many certified ODF communities are there within each of the targeted districts, and to what extent have the number and proportion of the population living in ODF communities increased?

14 17 How does access to WASH facilities in schools in rural communities compare with schools in small towns? And what impact does it have on ODF status?

Effectiveness - Governance and knowledge management

15 18 What plans exist at the regional level for WASH implementation in the 5 regions?

16 19 What systems for accountability exist at district levels?

17 20 What monitoring systems exist for tracking WASH results at the regional and district level?

8 KEQ numbers in this column refer to those contained in the evaluation framework.



18 21 Have the knowledge management components of the project improved understandings?

19 22 Are appropriate systems for sanitation law enforcement established?

Effectiveness in general

20 1 To what extent have the projects effectively addressed the specific needs of women and girls?

21 2 In which cases were project targets not met, and what could have been done differently to enable them to have been reached? How should future programming be designed in order to overcome experienced challenges?

22 3 How effectively has the programme responded to learnings and recommendations during delivery, and to what extent have key learnings been disseminated more widely?

23 4 What were (if any) the unintended effects (positive and negative) produced by the two projects?

24 5 To what extent have health outcomes improved in the project areas?

Efficiency

25 23 To what extent have the two projects met the intended targets in terms of service delivery in the project duration? In which cases were project targets not met, and what could have been done differently to enable them to have been reached?

26 24 To what extent have the projects delivered the desired targets whilst maximizing the human and financial resources available efficiently?

Source: Evaluation framework (Annex 2)

0.7 Methodology

0.7.1 Evaluation Framework

The overall evaluation approach is theory-based focused and consists in reviewing and testing the different components of the ToC developed for the Accelerated Programme. The evaluation assessed the programme’s contribution to achieving intended outcomes, by adopting Mayne’s contribution analysis approach.9 Contribution analysis was centred on the ToC and explored the attribution of observed outcomes to the Enhanced programme as opposed to other explanations and contextual factors. It is an appropriate design option for evaluating programmes that are not founded on the random selection and assignment of communities to treatment and control (experimental design). This process is detailed further in the Inception Report.

In order to address the evaluation questions, the evaluation team has used a mixed-methods approach, drawing on a range of data sources, including: internal document review, secondary data review, household survey and observation, interviews, and focus groups. The evaluation utilised a range of data sources to ensure the reliability of results, promote impartiality, reduce bias, and ensure that the evaluation is based on the most comprehensive and relevant information possible. Quantitative data sources were used to assess the programme’s achievements, and qualitative sources were used to triangulate quantitative data and to explore the programme’s implementation and contextual factors that have contributed to the achievement or non-achievement of results. To account for the possible influence of different contextual factors, data collection tools gathered information not just on the role of the programme but also to address the effects of other factors, such as other WASH programmes operating in the same region.

9 John Mayne (2008). Contribution analysis: An approach to exploring cause and effect. ILAC Briefing 16.



The evaluation framework includes the assessment criteria used to answer each evaluation question, as well as the data sources used to answer the questions. These data sources are described in further detail in the subsequent sub-sections.

0.7.2 Internal document review

The evaluation team reviewed key programme documents related to the planning, implementation and monitoring of the Accelerated programme, in order to support an understanding of the programme delivery and to evaluate the quality of key policies and guidelines developed. Annex 3 lists all documents that have been reviewed; this includes progress reports, the programme proposal, guidance documents developed, the baseline study, the KPM database, etc. All literature provided by UNICEF was mapped in a grid that contains information such as: source, year, programme, type of document (evaluation, progress report, strategic document, etc.) and relevance to the evaluation criteria to assist the team’s analysis.

0.7.3 Secondary data review

The evaluation team used secondary datasets to gather information on WASH indicators and to analyse differences in achievements by district and rural community/small town. The databases consulted include: Multiple Indicator Cluster Surveys (MICS), Basic Sanitation Information System (BaSIS), Demographics and Health Survey (DHS), UNICEF’s internal CLTS database, CWSA national water coverage statistics, and data from UNICEF and WHO/UNICEF Joint Monitoring Programme (JMP) reports. However, data from BaSIS were not used in the final evaluation as multiple discrepancies were found between the data in the various published league tables, and because UNICEF staff informed the evaluation team that BaSIS was a work in progress.

The MICS database is a useful tool for monitoring change across the Northern region between 2015 and 2019, as it uses indicators that are aligned with global indicators and targets, such as the MDGs and SDGs. However, given that MICS is based on data from many districts and communities in which the UNICEF programme was not active, it is difficult to directly attribute any identified changes or to quantify the programme’s contribution. As such, where MICS data is used, it is compared with data from the household endline survey and qualitative data where possible.

The Accelerated programme baseline report does not provide overall combined figures and does not mention any population level weighting, rather it only reports data for small towns and rural communities. Thus, the evaluation team cannot make overall comparisons across areas between baseline and endline. The baseline only provides separate findings for the rural sample and for the small-town sample. Thus, household data focuses solely on data disaggregated by urbanity (small town vs rural community) and gender.

0.7.4 Household survey and structured observation

Ipsos Ghana carried out a face-to-face household survey and structured observation of WASH facilities. Fieldwork commenced on the 2nd August 2019 and was completed in the week of the 9th September 2019. The final number of survey participants in the Accelerated programme communities was 543, from which 302 were men and 241 women (total for both Enhanced and Accelerated programmes: 1,088 participants). The number of participants in rural communities was 272 and 271 for small towns. The survey was conducted in Dagbane.

The sampling was based on a standard two-stage clustered sample design. Primary sampling units (PSUs) were sampled randomly, with stratification by region/district. The field team were provided with GPS coordinates of the programme communities, which allowed the field team to get familiar with the communities’ boundaries and identify a starting point for



random walks, which were carried out so that the interviews were spread across each community. The interviewees were carried out with only one household representative at a time. The gender target was 50% females and 50% males.

Based on the sample size, the confidence interval around a survey estimate of 50%, for the Accelerated programme with corrective weighting based on the population distribution, given a sample size of n=500 and 95% confidence level, is +/- 5.9 percentage points.10

One of the key aims of the programme is to ensure equity in access to improved sanitation facilities. The endline household survey data has therefore been disaggregated by key variables of interest throughout the report, namely gender and urbanity (rural community versus small town). Findings are only presented for these subgroups where there are statistically significant differences between them.11

During implementation of the household questionnaire, the enumerators asked to observe the household’s WASH facilities and recorded observations using the observation protocol contained within the household survey questionnaire. The questionnaire is based on the baseline questionnaire for the Accelerated programme, and to an extent based on the baseline questionnaire for the Enhanced programme12, for comparability between baseline and endline. It is included in Annex 5. Data from the endline household survey were also recoded into variables that are aligned with both the MDG and SDG WASH indicators. These can be found in Annex 5.

0.7.5 Key Informant Interviews

As part of the Accelerated programme evaluation, the evaluation team interviewed a total of 38 stakeholders. These included government bodies at national and district levels, civil society organisations, UNICEF staff, health centre and school staff, and other stakeholders. While the interviews were conducted as part of the broader programme of interviews covering both the Enhanced and Accelerated programme evaluations, the table below summarises only the interview participants relevant to the Accelerated programme (including those with knowledge of both programmes). The team also contacted Northern region government representatives, fieldwork facilitators and natural leaders but was unable to secure interviews with them.

A purposive sampling method was used, and contact details were provided by UNICEF. The evaluation team contacted stakeholders via email, phone and text message to schedule interviews. The project manager and team leader carried out face-to-face, Skype and telephone interviews with national and regional government stakeholders, CSOs, UNICEF staff and other stakeholders in Accra. The rest of the face-to-face interviewees were carried out by Ipsos Ghana’s qualitative research team.

The interview topic guides were prepared by the evaluation team during the preparation for fieldwork phase and are included in Annex 7. The topic guides were refined based on comments from UNICEF. The interview topic guides are semi-structured with a framework of key topics to explore, and specific tailored questions for each stakeholder type. The aim of

10 For disaggregated data, given n=272 were in rural areas, the confidence interval is +/- 6.7 percentage points, and with n=271 in small towns, the confidence interval is +/- 7.2 percentage points. Comparing results from the baseline survey and endline survey, given the respective sample sizes (n=1,000 at baseline and n=543 at endline), a change of 12.5 percentage points for the small towns sample (i.e. end-line estimate of 32.5%) and 11 percentage points for the rural villages sample (i.e. end-line estimate of 31%). can be detected as significantly different. The confidence intervals presented will be slightly narrower for survey estimates larger or smaller than the 50% level. 11 The Accelerated baseline survey provides separate results for small towns and rural communities but does not provide combined findings for all areas. Thus, to enable comparison, disaggregated results are reported throughout this evaluation unless such disaggregation is not statistically significant. 12 It is important to note that the baseline questionnaire of the Enhanced programme covered the whole GoG-UNICEF WASH programme (2012-2016), thus we only consider questions that we believed were relevant to the activities carried out under the Enhanced and Accelerated programmes.



these interviews was to gather views to address the evaluation questions including the following aspects: 1) the context for the programme, and the alignment of the programme design to local, regional and national needs and strategies; 2) the programmes’ effectiveness and efficiency in implementing its activities and achieving the targeted outcomes; 3) the programme’s effectiveness in addressing the needs of women and girls and any challenges in delivery; and 4) opportunities for improvement of programme design.

Table 3: Interviews completed by stakeholder type for the Accelerated programme

Stakeholder type Number of interviews

Total Male Female Metropolitan, Municipal and District Assemblies (MMDAs) (including DRPs)

9 6 3

National government 4 3 1 CSOs 11 9 2 School administrators 3 3 0 Health centre/Community-Based Health Planning and Services (CHPS) compound staff

2 2 0

UNICEF staff 6 2 4 Other stakeholders (WASH national expert, GAC, other agencies)

3 2 1

Total 38 27 11

0.7.6 Focus groups

The evaluation team carried out six school focus group discussions with students in communities where the Accelerated programme was implemented. In each school, separate focus groups with girls and boys were carried out, and each group consisted of eight pupils and one teacher. To mitigate the risk of bias in pupils’ responses due to teachers’ participation, participants were instructed that the role of teachers was to help ensure students are comfortable and to encourage students to respond, with questions primarily directed at students and only posed to teachers following students’ responses. Table 4 below presents the districts where the focus groups took place; to preserve participant anonymity, community and school names are not included.

The visited schools were purposively selected by UNICEF.13 These were located in communities where health centres also received programme intervention in order to allow the evaluation team to better understand the community context and results at the community level. All three communities were CLTS-triggered but none had achieved basic ODF status. The focus group guide was amended from the Inception Report version following comments received by UNICEF staff in September 2019 and is included in Annex 8.

13 Initially, the schools were selected by performance (e.g. number of school toilets available) but this approach had to be dropped to ensure the schools were in the same communities as health centres.



Table 4: Focus groups for the Accelerated programme

District School Reference Age of participants East Mamprusi School A Primary School:

12-15 years (Girls & Boys) East Mamprusi School B Junior High School:

12-17 years (Girls & Boys) East Gonja School C Junior High School:

14-15 years (Girls) 11-15 years (Boys)

0.7.7 Data analysis

The evaluation team triangulated the primary and secondary data from across all sources in order to answer the KEQs as set out in the evaluation framework. Focus groups and interviews were recorded and transcribed, and qualitative data was entered into an Excel-based coding frame aligned to the evaluation framework; qualitative analysis consists of both content analysis (to draw out findings from individual sources to help identify common content and subject matter) and thematic analysis (developing descriptive themes from the primary data and the generation of analytical themes to provide greater context and interpretation of the key findings). Thematic analysis was carried out to the extent possible at the level of community (through focus groups and interviews with community level stakeholders), activity in the ToC, or higher-level themes to ensure a complete picture of the programmes and understanding of the context in which they operate.

A ‘weight of evidence’ based principle was applied, in which the evaluation team considered the reliability and validity of each piece of evidence to minimise bias. The weight given to evidence depended on the type of data collection method, the level of stakeholder engagement, and the evaluation question being addressed. Acknowledging possible biases before and during the consultations, as well as during the analysis phase, allowed the evaluation team to make objective judgments of the information collected. Where triangulation of sources was not possible due to lack of additional sources to compare with, it is indicated as such within the reporting. In this instance, evidence is presented as the opinions of a certain group of stakeholders, rather than factual information (e.g. ‘According to X type of interviewee…’).

0.7.8 Ethical considerations

The evaluation was conducted in line with the UNEG Norms and Standards (2016); the UNEG Ethical Guidelines for Evaluation (2008); the UNICEF Procedure for Ethical Standards in Research, Evaluation, Data Collection and Analysis (2015); and UNICEF’s Ethical Research Involving Children guidance (2013). Ipsos is a member of the World Association for Social, Opinion and Market Research (ESOMAR). Ipsos Ghana has a Government research permit that allows Ipsos to conduct research in all the districts in Ghana. Ipsos obtained ethical approval to conduct this specific study from Ghana Health Service Ethics Review Committee on 9th July 2019. More information on the ethical conduct, including the consent process, confidentiality of responses and safeguarding practices, can be found in Annex 9.

0.7.9 Gender, human rights, and equity considerations

The evaluation team followed UNEG Guidance on Integrating Human Rights and Gender Equality in Evaluation (2014) throughout the design and delivery of the evaluation.



The evaluation team comprised an appropriate mix of women and men, local and international evaluators/research teams, and understanding of relevant approaches and methods; the team included both gender and WASH specialists. The majority of the data collected – including all household survey and school data – was collected by a local research team from Ipsos Ghana, who conducted the research in local languages where relevant. Please see Annex 11 for short biographies of each member of the evaluation team.

The evaluation’s rights-based approach started from the premise that all people have the right to participate in and benefit from all development actions; it considered the attitudinal, environmental and institutional barriers that limit and exclude women, girls, and people with disabilities from access to and use of WASH services. In particular, the Inception Report sought to understand the context of WASH-related inequities in Ghana, including gendered division of household chores such as water collection, the specific hygiene and sanitation needs of women and girls, and inequities in access to WASH services faced by marginalised groups.

The evaluation framework mainstreamed gender and incorporated a rights-based approach through the evaluation questions and selection of appropriate indicators. The evaluation questions sought to understand both the extent to which human rights and gender equity considerations were integrated into the programme design, as well as the extent to which results were equitable.

The evaluation adopted a mixed-methods approach to ensure both quantitative and qualitative evidence of equity considerations was sought and to ensure the voices of all stakeholder groups were heard. Inclusion of women and girls in the evaluation process was ensured through setting 50/50 male/female targets for the survey sample, and by holding gender-segregated focus group discussions for students.

Survey data generated was disaggregated by gender, and all data was triangulated. Evaluation findings focus on equity and gender equality rather than needs and problems, and the rights of rights-holders and obligations of duty-bearers are recognised.

See Annex 9 on ethical considerations for more information on this topic.

0.7.10 Technical review meeting

A workshop was held with UNICEF in Ghana staff by Skype on 10th January 2020 to discuss preliminary findings. This workshop highlighted the most important learnings for UNICEF and the wider WASH sector and provided an opportunity for stakeholders to provide their feedback on the draft report. UNICEF and evaluation team representatives were in attendance.

0.8 Challenges and mitigation measures

During the evaluation process, the team has identified a number of challenges and limitations to the overall evaluation, which are together with the mitigation measures described below:

Programme implementation. The programme implementation was ongoing while the evaluation was taking place, only finishing in June 2019 while the evaluation started in December 2018. This means that the programme’s results cannot be fully captured by the evaluation. However, the ream received the programme’s final KPM data in January 2020, which allowed the evaluation to consider the final achievement of the programme against its targets.



Comparability of baseline data: The evaluation makes comparisons to the baseline indicators and also provides combined endline figures that are aligned with global indicators. However, for the latter, there are no directly comparable baselines. Questions in the endline survey have been designed to reflect the baseline in order to allow for some level of comparability. The endline survey also asked additional questions that are useful in order to understand the WASH situation within the target communities and to answer the KEQs. However, for some of these questions, there is no baseline available from which to monitor change. Furthermore, the baseline survey report does not provide data for a combined sample of rural villages and small towns, and no information is provided on population level weighting. The raw data file provided to the evaluation team is also empty, meaning that the evaluation team have not been able to combine rural and small-town data to create combined baseline population estimates. Nevertheless, it is useful for this evaluation to present the findings from the endline household survey, as any improvements identified are confined to communities and small towns in which the programme was active (which is not necessarily the case in the secondary data).

Some outputs and outcomes did not have clear targets: Some of the outputs and outcomes included in the ToC did not have concrete targets in the KPM database, which hindered the assessment of the results of related activities.14 These missing targets were identified in the Inception Report, and the evaluation framework set out evaluation indicators and sources to assess these. In addition, targets set in the KPM database for handwashing with soap (HWWS) and ODF verified districts are regional targets, although the programme does not cover the entire region; the programme’s target is therefore described as a ‘contribution‘. While acknowledging that the achievement or non-achievement of KPM targets based on these agreed definitions, the evaluation team therefore used qualitative evidence and available quantitative evidence to assess the extent to which the Accelerated programme actually contributed to these regional results. Throughout the report, the evaluation team provides an explanation on how the success of each activity was assessed.

Difficulties contacting and reaching some stakeholders: The evaluation team was unable to interview some of the intended stakeholder interviewees, although multiple contact attempts were made for each invitee through all available means, and multiple contacts were requested for each target interview set out in the Inception Report. The team also added artisans and natural leaders to the target interview list during fieldwork at UNICEF’s request but was unable to secure these interviews. This is mitigated by use of secondary sources and interviews with other stakeholders with knowledge of these topics, such as MMDAs, national government representatives, and CSOs, although it is acknowledged that these are likely to have less direct knowledge of barriers to behavioural change among community members

Limited ability to conduct analysis at district level: As highlighted in the Inception Report, the sample size was selected to enable disaggregation by rural communities versus small towns and by gender, but this sample size does not permit disaggregation across districts or communities. At reporting stage, UNICEF has requested greater visibility into results at district level. Therefore, secondary data, for example from UNICEF’s WASH in Institutions database, have been incorporated where relevant.

Disentangling results of individual activities: The Accelerated programme carries out a large variety of activities, for which is not possible to disentangle their individual effects quantitatively. Further, not all activities were delivered in

14 Some targets use the same numbers e.g. there is a target for people reached with messages around safe hygiene and another target for people who have been made aware of safe hygiene practices. The same numbers have been used to record progress towards each - 432,000 people.



all communities or districts. Qualitative evidence – and where possible, disaggregated quantitative evidence – are therefore used to identify which factors were the most important and assess the results of these factors.

Selection of schools for qualitative data collection: UNICEF selected the schools and health centres for the qualitative data collection, rather than the evaluation team, to ensure that data collection was carried out in communities where both schools and communities were present. Subsequently, all three communities were CLTS-triggered but none had achieved basic ODF status. Findings are therefore not generalisable but have been used to highlight prominent contextual factors and risks to the programme’s Theory of Change.



This section provides the list of the evaluation findings and preliminary conclusions that will allow addressing the different evaluation questions. It is structured around the DAC criteria of relevance, effectiveness and efficiency. Each key evaluation question is addressed individually; in some cases, the order of the KEQs has been altered to enable a better logical flow of findings. Each paragraph pertaining to a finding is numbered, and each preliminary conclusion (placed in boxes throughout the section) clearly indicates the number of the findings paragraphs on which it is based. For the sake of precision, the evaluation final conclusions, which build and expand on the preliminary conclusions displayed in this section, are presented in the next section of the report (Section 3).

1 Evaluation Findings & Preliminary

Conclusions (by Criterion)



1.1 Relevance

The OECD DAC’s ‘relevance’ criterion refers to ‘the extent to which [an] aid activity is suited to the priorities and policies of the target group, recipient and donor.’ In this evaluation, relevance considers whether programme design and delivery align with national and international policies and strategies for WASH, and whether the knowledge produced by the programme answers knowledge gaps for the WASH sector in Ghana. The KEQs related to relevance are presented in the table below.

Table 5: Key evaluation questions for the Accelerated programme related to relevance KEQ number


1 How relevant is the design of the programmes to national and international policies and strategies for WASH?

2 Has research been appropriately designed to answer sector knowledge gaps?


Source: @UNICEF/WASH/GHANA



KEQ 1: How relevant is the design of the programmes to national and international policies and strategies for WASH?

International relevance

1. At the time the programme proposal was prepared, Ghana had achieved the MDG for access to improved water sources (a target of 78% of the population). However, the percentage was lower in the Northern region - only 68% of households having access to improved water sources. As highlighted in the Accelerated programme proposal and stakeholders, Ghana’s Northern region did not reach the MDG target due to the challenging conditions of the groundwater. Moreover, Ghana was not on track to reach the MDG target of 54% of the population with access to improved sanitation facilities, at 15% nationally. Access to improved sanitation facilities had only increased by 7% since 1990, with nearly 1 out 4 households (23%) practicing OD or having no toilet facility nationally.15 The situation in the Northern region was worse than national levels, with 72% of the population practicing OD.

2. According to stakeholders involved in the implementation of the CLTS approach in the Northern region under the Enhanced programme, sanitation and hygiene promotion activities were more difficult to implement in the Northern region than in other regions due to socio-cultural practices and the proliferation of different approaches to sanitation implementation by different non-governmental organisations (NGOs). Thus, to see significant change in the overall sanitation results, the Accelerated programme targeted the critical sanitation situation in the Northern region, aligning its targets to an extent to the SDG global indicators.

3. SDG 6 includes concrete targets and indicators on hygiene, wastewater treatment and water scarcity, and provides more emphasis on equitable access to water and sanitation facilities that its predecessor. Due to the SDGs being signed in 2015 when the programme commenced, the programme had a focus on helping households to achieve at least basic sanitation (i.e. acceptable sanitation facilities that are not shared).

4. Similarly to the Enhanced programme, the Accelerated programme did not aim to directly improve access to clean water through community-level activities, as Ghana by 2015 had already met its clean water target, but sought to improve access more indirectly through government engagement, advocating for government to commit to reaching the final 20% of people across the country without access to potable water, and through institutional-level activities (in schools and health centres).

5. The inclusion of gender quality objectives in the programme was also highlighted by the stakeholders consulted. They said that the programme had a deliberate focus to ensure women and girls have access to gender-friendly sanitation and hygiene services and facilities. They noted the programme’s focus on enabling women to access and use facilities in schools and health centres and its aims to improve technical expertise on gender equality and support the implementation of gender mainstreaming guidelines and toolkits. All these programme activities are aligned with SDG 5 targets16; particularly to ‘adopt and strengthen sound policies and enforceable legislation for the promotion of gender equality and the empowerment of all women and girls at all levels’. According to national stakeholders, the programme covered activities that were directly contributing to the achievement of SDGs, namely: the strengthening of the private sector in the provision of sanitation services and building government capacity (particularly at district level) to deliver water and sanitation services, which accommodate the sanitation needs of all. These stakeholders felt that building government capacity and getting local

15 Accelerated programme proposal 16 https://www.unwomen.org/en/news/in-focus/women-and-the-sdgs/sdg-5-gender-equality

https://www.unwomen.org/en/news/in-focus/women-and-the-sdgs/sdg-5-gender-equality



communities involved in improving water and sanitation management has been at the core of the programme since the beginning. The establishment of the country’s Rural Sanitation Model and Strategy (RSMS) in 2012 strengthened this need, which aligns with SDG 6.B: ‘support and strengthen the participation of local communities in improving water and sanitation management’, and the Accelerated programme has continuously refined the approach to implementation of the RSMS, including through development of the Small-Town Sanitation Model (STSM).

6. For example, the programme supported the new Ministry of Sanitation and Water Resources (MSWR) in developing the Sector Information System (SIS) for performance monitoring of the WASH sector. This support lead to the establish of set of performance indicator, known as the ‘Golden Indicators’, 14 key indicators for the WASH sector which include a range of indicators from access to water and sanitation to the enabling environment as well as two indicators on gender issues in WASH. Moreover, the programme (as well as the Enhanced programme) supported the inclusion of WASH indicators in the Education Management Information System (EMIS) and District Health Information Management System (DHIMS) for improved monitoring nationally. For schools, this includes monitoring indicators such as access to gender-segregated toilets and changing rooms for MHM purposes. The enhanced monitoring and evaluation structures will provide data on to track Ghana’s performance related to SDG 6.2, which is measured by the proportion of population using safely managed sanitation services, including a handwashing facility with soap and water. The programme monitoring tools reflect the updated sanitation description under SDG 6 when it comes to improved access to sanitation facilities but has no specific targets on the number of households with access to basic, limited and unimproved latrines. Thus, they are not fully aligned with the SDG 6 indicators. However, as highlighted by UNICEF staff, UNICEF worked under the Accelerated programme and related Enhanced programme with the GoG to establish a Sector Information System (SIS) with golden indicators that capture the full range of indicators in the sanitation chain, and the SIS model and golden indicators to measure SDG 6 have been approved by the GoG and WASH sector stakeholders.

Relevance to UNICEF policies and strategies

7. Under the UNICEF in Ghana country programme 2018-2022, the overall strategic objective of UNICEF in the WASH sector in Ghana is ‘to support the sector to ensure universal access to safe drinking water and sanitation, supported by good hygiene practices, with the aim of increasing child survival and reducing morbidity and mortality caused by WASH-related diseases and poor sanitation’. This objective is clearly aligned to UNICEF’s Regional Office for West and Central Africa’s Key Results for Children agenda, which was developed in 2018 and includes ‘ending open defecation to allow children to grow up in a safe and clean environment’ as one of its eight priorities.

8. The Accelerated programme was designed before the latest UNICEF Gender Action Plan (2018-2021) was published. However, interviewees highlighted that the programme has been in direct alignment with UNICEF Gender Action Plan through two aspects of WASH programming: supporting gender-responsive WASH systems across government – designing sectoral policies and programmes which facilitate the achievement of gender-responsive results and gender mainstreaming (where WASH systems are included) – and girls’ empowerment, where the programme has strong focused on MHM and creating a supportive environment for girls. Thus, the programme aligns and contributes to achieving the broader goals of the gender policies and throughout implementation, the programme implementers have remained open to adjusting it to international policies and strategies.

National relevance

9. UNCIEF-supported WASH programmes in Ghana aim to contribute to the GoG political commitment to achieve nationwide ODF status by 2020 and universal safe sanitation and water access supported by good hygiene practices by



2025, with 98% functionality of water supplies. However, as highlighted in this report, the Accelerated programme focused more on achieving nationwide ODF status as Ghana is closer to meeting the universal safe water access targets than its ODF targets.

10. The Accelerated programme aims to reflect the WASH priorities set out by the GoG in the national medium-term development plan called the Ghana Shared Growth and Development Agenda (GSGDA).17 According to the Accelerated programme proposal and national government stakeholders, the programme design aligns with the priorities of the WASH sector in Ghana and the following strategic national policies and strategies: National Water Policy, Environmental Sanitation Policy, National Environmental Sanitation Strategy and Action Plan, Strategic Environmental Sanitation Investment Plan and the MDG Acceleration Framework for Sanitation, and the RSMS. All these policies and strategies are from the MDG era but are currently being review by GoG to reflect SDG targets.

11. Stakeholders agreed that the programme was designed following existing national policies and strategies and that the government supported the implementation because it falls under its mandate to deliver WASH. An important indicator of the relevance of the programme is the level of ownership shown by stakeholders interviewed, particularly by national government authorities.

‘It is our role to implement and coordinate the CLTS approach at all levels of government, to change people’s behaviour towards OD and become an ODF Ghana in the near future’. – National government representative

12. Regional and district-level government, as well as CSO representatives interviewed, agreed that the Accelerated programme follows national policies and highlighted the RSMS as a key policy being followed and implemented. The RSMS, developed before the start of the programme by the GoG in collaboration with UNICEF, provides details on how the CLTS approach can be used as a vehicle to scale up rural sanitation at decentralised levels. Programme documentation provides detailed information on how the Accelerated programme has implemented the RSMS by training stakeholders on the CLTS approach, building district level capacity to monitor and report on WASH activities, working on the supply side of WASH services by training artisans on latrine techniques and entrepreneur skills, etc. While many of these documents were built on or shared with the Enhanced programme, stakeholders also highlighted how the Accelerated programme has facilitated the implementation of the RSMS in small towns by developing the CLTS in Small Towns Implementation Manual (i.e. the Small-Town Sanitation Model, or STSM),18 which was developed to guide district level staff of the Environmental Health and Sanitation Units in small towns on CLTS implementation.

13. Some stated that ‘UNICEF has aligned the design and delivery of the programme to the needs and policy goals of government departments’, building the capacity of government at different levels to improve the implementation of WASH services. In other words, the programme has enabled the government to develop a national approach to rural sanitation adapted to the country’s needs.19 CSOs agree that the programme activities which reinforce the implementation of the CLTS approach, particularly the financing of sanitation through the establishment of the District Sanitation Fund for rural areas, aligns with Ghana’s priority to strengthen and scrutinise the financial sector so they play a stronger role in the provision of

17 Terms of Reference 18 CSOs explained how the CLTS approach is implemented under the STSM: small towns are divided into sections so different communities within the towns can become ODF at different times. 19 7th Enhanced progress report



WASH services in Ghana. However, even though the programme has aligned with the RSMS and contributed to the promotion and implementation of the CLTS approach, according to CSOs and District Resource Persons (DRPs), many WASH sector stakeholders still questioned this approach and still today some do not yet see the value of it against the provision of toilets through subsidies or communal toilets.

14. Apart from considering the relevance of the programme’s design to national policies and strategies for WASH, stakeholders highlighted that the programme has remained valid to the government’s WASH priorities over the course of its implementation and has supported sustainability of the WASH sector in Ghana. Moreover, UNICEF played a role in the creation of the Ministry of Sanitation and Water Resources by building the capacity of WASH development partners and advocating for sanitation to become a national priority.

15. The programme’s design and delivery has been adapting to the lessons of rolling out the RSMS in rural areas but also in small towns. Interviewees highlighted that the programme adaptive learning approach required research activities, and strong collaboration across the implementing organisations to carry out the needed research as well as sharing the knowledge obtained. KEQ 2 explains in detail some of the research carried out to improve the programme activities and strategies/polices to implement WASH in Ghana; for example, the research on the cost effectiveness of school hygiene approaches and a survey of school hardware costs to develop the Wash in Schools (WinS) Costed Strategy.

KEQ 2: Has research been appropriately designed to answer sector knowledge gaps?

16. Through the programme, several research and guidelines documents have been developed by UNICEF and research organisations (including CSOs) to support planning, implementation and management of WASH programmes. For example, the Accelerated programme supported research on the cost effectiveness of school hygiene approaches and a survey of school hardware costs starting to develop the WinS Costed Strategy which helps identify investments needed to enable Ghana to achieve the SDG target of universal access to water and sanitation in schools by 2030. Some elements of the WinS Costed Strategy were also adopted into the Education Sector Plan for the country. Other examples of research and documents that were developed include documents to support delivery of WASH services include such as the manual for small town CLTS and research to understand the impact of the national social norm campaign. The programme has used and reviewed many financial and sanitation technical support documents developed under the Enhanced programme to inform key stakeholders on how to implement WASH policies and strategies but also for planning, implementation and management of future WASH programmes. Some of the most important guidance documents reviewed or updated by the Accelerated programme highlighted by stakeholders include: Latrine Artisans Training Manual for Basic Training, Quality Assurance for Toilet Construction for Implementation Guide, Provision of Technology Support to CLTS Communities - A Guide for District Facilitators and Minimum Guidelines for Household Toilets.

17. Moreover, according to stakeholders consulted, the programme also advocated to include new guidelines and tools into exiting government policies to ensure WASH services are adequately delivered by all government agencies; namely the inclusion of WASH priorities in the Education Sector Medium-Term Development Plan and Education Strategic Plan. Implementing organisations highlighted that guidance documents have not only been produced or updated to implement concrete strategies and policies but also as a response to challenges and sector knowledge gaps encountered when implementing the programme. This facilitated the implementation of programme activities and increased, to an extent, knowledge across WASH sector stakeholders. For example, under the Enhanced programme, research was conducted to address challenges encountered related to the lack of technical capacity to construct toilet facilities, such as research related to sanitation technologies and costs. CSOs continue to carry out research under the Accelerated programme to ensure artisans and community technical volunteers have the most up to date and detailed manuals to construct sustainable toilets.



CSOs developed a more advanced latrines manual and are working on an optimisation training guide. CSOs highlighted that to develop the optimisation training guide, they have designed sanitation challenges at the artisan level and ask artisans to submit ideas to improve these sanitation challenges. This research allowed CSOs delivering SanMark activities to develop financial and sanitation technical support documents mentioned above under both programmes (Enhanced and Accelerated). Some of CSOs delivering SanMark activities highlighted that they are continuing to develop manuals and guidance documents for more sustainable latrines and handwashing facilities in collaboration with UNICEF. These could be used in future WASH programme, but also will generally be use by WASH sector stakeholders if disseminated correctly.

18. According to interviewees, referring to both programmes (Enhanced and Accelerated), these supporting documents ensure the sustainability of the sanitation progress made so far by the GoG in collaboration with UNICEF

19. Moreover, according to stakeholders, the development of new and more updated guidance documents is driven by community and institutional demand for more sustainable toilets; stakeholders supplying and financing household latrines want to understand which products are more profitable and how households will be able to finance them (e.g. through district sanitation funds or VSLAs). In general, stakeholders believe that the knowledge generated by the programme answers some of the sector knowledge gaps; however, many gaps still exist, for example, how to include more female entrepreneurs into the supply of sanitation facilities. It was recognised that these challenges cannot be addressed by one programme alone but by all WASH sector stakeholders.

PRELIMINARY CONCLUSIONS: RELEVANCE:

KEQ 1: The Accelerated programme has been designed following the country’s national WASH policies and strategies, which aligned to the targeted outcomes of the SDGs and the specific needs of the country to meet the SDGs (Paragraphs 2, 3 and 5). Although scope for inclusion of SDGs in programme monitoring has been limited due to concurrent development of the golden indicators, monitoring of future programming could be better aligned to the SDGs by including the number of households with access to basic, limited and unimproved latrines (Paragraph 6). Government representatives considered the Accelerated programme to be aligned to existing national policies and strategies, particularly to the RSMS, and helped government build its knowledge base on CLTS, particularly in its application to small towns (Paragraphs 9-13).

KEQ 2: Research and guideline documents have been updated or being reviewed by UNICEF and research organisations (including CSOs) to facilitate the implementation of RSMS and other policies and particularly to adapt the RSMS to small towns (Paragraphs 16 and 17). These have not only been drafted to implement concrete strategies and policies but also as a response to challenges and sector knowledge gaps encountered when implementing the programme (Paragraphs 18). The programme has contributed to filling in sector knowledge gaps, but there are still remaining gaps that need to be addressed by future programmes and more generally by WASH sector stakeholders (Paragraph 19).



1.2 Effectiveness

The OECD DAC’s ‘effectiveness’ criterion refers to ‘the extent to which [an] aid activity attains its objectives.’20 It assesses the extent to which the objectives of an activity have been achieved or are likely to be achieved as well as the factors influencing this. ‘Effectiveness’ in this evaluation seeks to understand to the extent to which the Accelerated programme has achieved its intended objectives (i.e., to what extent have the intended outcomes set out in the ToC been achieved?). The KEQs related to effectiveness are presented in the table below.

Table 6: Key evaluation questions for the Accelerated programme related to effectiveness KEQ number


Effectiveness - WASH in rural communities and small towns

3 What is the access to household sanitation in the targeted regions and districts? Are the facilities in use? Are there variations in access in rural communities versus small towns?

4 What do community members and school children know about the hygiene practices which the programme promoted? Are there variations in knowledge in rural communities versus small towns?

5 To what extent do community members and school children have appropriate HWWS facilities? Are there variations in access in rural communities versus small towns?

6 What is the access to safe water supply (including HWTS) in the targeted rural communities and small towns?

7 How is faecal sludge managed in the targeted small towns?

Effectiveness - WASH in institutions (schools and health centres)

8 To what extent do school boys and girls have access to school sanitation facilities? What is the functionality status of WASH facilities in schools? Are there variations between rural communities and small towns?

9 To what extent are the school sanitation facilities gender- and disability- friendly? Are there variations between rural communities and small towns?

10 What proportion of schools have facility management plans in place?

11 What is the level of knowledge on menstrual hygiene management and hygiene behaviours?

12 To what extent are WASH facilities in health centres and CHPS compounds available? What is the functionality status of WASH facilities in health facilities?

Effectiveness - District level WASH access

13 How many certified ODF communities are there within each of the targeted districts, and to what extent have the number and proportion of the population living in ODF communities increased?

14 How does access to WASH facilities in schools in rural communities compare with schools in small towns? And what impact does it have on ODF status?

Effectiveness - Governance and knowledge management

15 What plans exist at the regional level for WASH implementation in the 5 regions?

16 What systems for accountability exist at district levels?

17 What monitoring systems exist for tracking WASH results at the regional and district level?

18 Have the knowledge management components of the project improved understandings?

19 Are appropriate systems for sanitation law enforcement established?

Effectiveness in general

20 To what extent have the projects effectively addressed the specific needs of women and girls?

20 DAC criteria are available at http://www.oecd.org/dac/evaluation/daccriteriaforevaluatingdevelopmentassistance.htm



21 In which cases were project targets not met, and what could have been done differently to enable them to have been reached? How should future programming be designed in order to overcome experienced challenges?

22 How effectively has the programme responded to learnings and recommendations during delivery, and to what extent have key learnings been disseminated more widely?

23 What were (if any) the unintended effects (positive and negative) produced by the two projects?

24 To what extent have health outcomes improved in the project areas?





1.2.1 - Effectiveness - WASH in rural communities and small towns

KEQ 3: What is the access to household sanitation in the targeted regions and districts? Are the facilities in use? Are there variations in access in rural communities versus small towns?

20. To improve community sanitation, the Accelerated programme sought to improve people’s knowledge of good sanitation practices and change behaviours around open defecation, generate demand for sanitation facilities within communities, and improve access to and use of sanitation facilities in households. Activities to support these objectives included helping households to save towards sanitation facilities through VSLAs and conditional grants; training to help households construct and maintain latrines; educational activities in communities and in schools – encouraging children to become leaders promoting good sanitation practices within communities; and SanMark activities to train artisans/community technical volunteers on latrine technologies and entrepreneurial skills – with the Accelerated programme focusing especially on engaging the private sector to scale-up sanitation development across the Northern region.

21. According to the final Accelerated programme KPM database, the Accelerated programme achieved the following relative to its targets:

None of the targeted districts have achieved ODF certified status. (Target: Five districts achieving ODF certified status.)

411 rural communities and 34 small towns have been declared ODF. (Target: 80 small towns across ten districts; no target was set for rural communities.)

204,655 additional people are living in households that use improved sanitation. (Target: 500,000 people).

22. Thus, no target has been achieved in terms of household and community sanitation. However, clear progress has also been made towards achieving district-level ODF status. Although none of the targeted districts have yet achieved ODF status, 90% of communities in Tatale Sanguli are ODF, more than 80% of communities in East Mamprusi are ODF, and over 70% of communities in Kpandai, Karaga, West Mamprusi and Chereponi are ODF. Staff from UNICEF in Ghana expect to see at least one district achieve ODF status by March 2020, and if progress continues at the current pace, this appears likely. Moreover, it was found in discussion with UNICEF staff that the number of people living in ODF communities was not reached as the number of people living in these communities was overestimated at the programme design phase and that greater involvement of districts in the programme planning phase would have allowed UNICEF to develop more accurate targets.

23. Comparing the endline household survey to available baseline data, the evaluation team also assessed the extent to which the population in the target rural communities and small towns have moved up the sanitation ladder across programme implementation, using global indicators defined in JMP reports and in the SDGs. The JMP defines an at least ‘basic sanitation service’ as the use of improved sanitation facilities that are not shared between other households. It defines a ‘limited sanitation’ service as the use of otherwise improved sanitation facilities that are shared between two or more households. Comparisons between baseline and endline data highlight the following:

OD declined significantly. At baseline, 68% of rural communities were practicing OD. By endline, just 33% were practicing OD. A slightly lower proportion were practicing OD in small towns at baseline (63%), while at endline 32% of small-town participants were practicing OD.



According to the MICS database, the percentage of the overall population practicing OD in the Northern region fell from 72% at baseline (2011) to 57% in 2017. This suggests that OD reduced more substantially in Accelerated programme rural communities and small towns than across the region as a whole.

Thirty-six percent of small town endline survey participants have access to ‘at least basic’ sanitation facilities (improved facilities that are not shared between households)21. In comparison, 27% of endline survey participants in rural communities have access to ‘at least basic’ sanitation facilities. Across the Accelerated programme intervention area, 34% of participants have access to at least basic sanitation facilities – significantly higher than the proportion of the population with access to at least basic sanitation across the Northern region as a whole (12%).22 These findings are not directly comparable to data from the programme’s baseline survey.

In both rural communities and small towns, a significant majority of participants say that the latrines they use were built after 2015, after the Accelerated programmes activities had commenced (63% in small towns and 84% in rural communities).

Importantly, it seems that the latrines are in active use, with 80% of observed household or compound latrines in small towns and 90% in rural communities displaying evidence of use. These findings suggest that, not only has there been a notable increase in latrine building in recent years, but that it has been complemented by the necessary change in behaviour.

There are no significant differences between men and women in terms of access to public or private sanitation facilities or in OD rates.

24. Among those endline household survey participants who say they practice OD, none say that they do so because they ‘simply prefer open defecation’. This differs notably from baseline, when 16% in small-towns and 23% in rural communities said they did so because they simply preferred OD, reaffirming a change in norms around OD from baseline to endline. Rather, 73% of those who openly defecate in small towns and 79% in rural communities say their reason for doing so is that there are no other options available to them (no toilet in the house and no public toilet).

25. Just five per cent of those who practice OD in small towns and three per cent in rural communities say that they cannot afford to use a public toilet and, interestingly, 87% and 83% respectively say they would be willing to pay to use a private latrine. Hence, it appears that a lack of skills and knowledge to construct latrines among those who continue to defecate openly, rather than a lack of financial resources, is a key barrier to ending OD in the target communities. According to interviewees, the lack of skills and knowledge to construct latrines is a recurrent issue, not only because the supply-side activities are not available at the right time to meet the demand for household latrines but also because households are not aware of available support.

26. Indeed, the Accelerated programme sought to address these supply-side barriers to latrine construction from the outset. In the Accelerated programme both supply and demand activities were included from the start of the programme; SanMark teams (which supervise and regulate activities of private sector actors in the sanitation supply chain) were established in the targeted districts while communities were being triggered under the CLTS approach. The programme had

21At endline, in line with JMP definitions, improved facilities refer to those connected to a public sewer, those connected to a septic tank (e.g. water closet with septic tank), pour flush toilet to pit latrines or sewers (with water seal) or access to a hygienic pit toilet (KVIP, VIP, basic clean pit latrine, Biofil digester/ micro flush toilet, urine diverting). 22 UNICEF and Ghana Statistical Service. The Ghana Multiple Indicator Cluster Survey 2017/2018: Snapshots of key findings. January 2019. Available at https://www.unicef.org/ghana/media/576/file/Ghana%20Multiple%20Cluster%20Indicator%20Survey.pdf

https://www.unicef.org/ghana/media/576/file/Ghana%20Multiple%20Cluster%20Indicator%20Survey.pdf



numerous related KPM targets, all of which were achieved: 14 districts implementing framework for PPP in sanitation (target: 10), 240 private sector businesses engaged in sanitation services (target: 200), 14 districts with district-level framework for private sector engagement in place (target: 10), and 240 sanitation-related private sector business mobilized (target: 200). Stakeholders believe that the programme has contributed to strengthening the engagement of districts with private sector actors, developing the private sector, and supporting its promotion of sanitation services.

27. The programme has also addressed demand-side barriers to latrine access. For example, the establishment of more VSLAs has allowed households to save so they can pay for latrines. A total of 493 VSLAs have been formed through the programme, with over 70% of members being women. A total of 134,397 GHS (ranging from 3,760 GHS at the lowest, and 39,803 GHS at the highest) have been saved, from which 74,129 GHS have gone to community sanitation funds. The programme also worked to identify financial institutions to manage the District Sanitation Funds in three MMDAs.

28. Additionally, the programme successfully contributed to raising awareness around sanitation and hygiene, not only by training natural leaders and community champions but establishing natural leader networks to boost sanitation results. According to CSOs, the formation and support of these network required extensive research and support from by CSOs; but have allowed natural leader to share experience and ensure a coordinated approach when implementing CLTS activities. According to fourth progress report, a total of 230 NLNs have been formed across 13 districts, with a total of 625 natural leaders (384 males and 241 females).

29. Student focus groups highlighted the role of the Accelerated programme’s activities in schools in generating demand for latrines in the wider community, including the programme’s establishment of health clubs and appointment of student ambassadors (discussed further under KEQ 4), although there were some challenges encountered.

‘If we see anybody going to the toilet and not using the facilities during classes, they are reported to the teachers and either advised or sanctioned. This helps to stop OD in the school community.’ – Student

30. Stakeholders were positive that the CLTS/STSM approach has worked well; however, the main implementation challenge identified by interviewees was inadequate capacity at district level. This was particularly the case of districts that had not worked with UNICEF before in the implementation of the RSMS. District Resource Persons (DRPs) were provided by the programme and appointed to districts to strengthen their technical and monitoring capacity. MMDAs confirmed that the process of implementing the CLTS approach in small towns was been more difficult than anticipated, and to ensure that district assemblies meet their ODF targets, additional capacity building activities and manpower should be provided. They stressed that field facilitators need additional logistical support, including phones and motorbikes, to be able to target the programme activities towards hard-to-reach communities. In addition, sociocultural attitudes and an individualistic culture in the Northern region presented challenges for the implementation of the CLTS approach and subsequent achievement of ODF status, as the CLTS approach is dependent upon communities mobilising collectively around the issue of OD.

31. Other WASH programmes were active in some of the Northern region while the Accelerated programme was active, so the above improvements cannot be attributed solely to the Accelerated programme, although stakeholders interviewed felt the programme had made an important contribution.



Use of public latrines23

32. Eighteen percent of endline survey participants in small towns use a public latrine, compared to just 3% in rural communities, which may be due to greater public investment in larger communities. There was no significant change in public latrines use between baseline to endline. The fact that there is greater use of public latrines in small towns as well as higher access to improved private sanitation facilities in small towns may also be explained by the available skills in small towns versus rural communities, i.e. access to improved unshared sanitation facilities is higher in small towns because there are more artisans working there. According to UNICEF staff, people in rural communities tend to build their own latrines or rely on the services of community technical volunteers (indeed, access to private latrines, both improved or unimproved, is higher in rural communities, where 50% of households use a private latrine, compared to 34% in small towns).24 By contrast, in small towns, wealthier households have procured the services of private artisans (trained as part of the programme) to construct higher quality, more advanced latrines. Among those small-town participants who did use a public toilet, the most commonly cited reasons for not having a private latrine is the cost, mentioned by 68% of participants.25community technical volunteers

33. Together, this suggests that in small towns, not enough financial support was available to those who cannot afford to purchase household latrines from private artisans, resulting in greater reliance on public facilities. Conversely, in rural communities, while household latrines are being constructed at scale, standards are inconsistent. Indeed, both MMDAs interviewed as part of the evaluation and Accelerated progress reports note that there are persisting construction issues around household latrines falling apart during the rain. When asked if there was anything they would like to add at the end of the household survey, many endline survey participants also mentioned that they were struggling to maintain their household latrines.

Gender and disability-friendly latrines

34. The evaluation also identified areas of focus for future interventions to build on the progress made under the Accelerated programme – notably around gender and disability friendliness of household sanitation facilities. Women and girls have different risks and requirements to men and boys, both in terms of biological needs (around menstruation, for instance) and around social needs (women are often at a greater risk of harassment or sexual violence). Ensuring that women and girls have access to latrines that are gender-friendly can increase their use and uptake. As such, the endline household survey also gathered information related to the features of the toilet and found that:

Almost three quarters of participants (74%) in small towns and 64% in rural communities say that their toilets are segregated (meaning they are in a private space where nobody else can see who is inside).

Just 20% of participants in small towns and 14% of participants in rural communities say that their toilets are well lit.

23 For the purposes of the endline survey, a public latrine is defined as one that ‘is used by the public’ rather than a shared toilet that is ‘shared between households in a compound’. 24 Private latrines refers to exclusive use of a household toilet, rather than privately managed shared facilities. There was no significant difference between rural communities and small towns in access to privately managed shared facilities. 25 It should be noted that cost is also the most commonly cited reason among users of public toilets in rural communities, however the base size is too low to allow for any meaningful analysis (n=8 ).



Over half (51%) of small-town participants said that their toilets had locks, which aligns with the results of the observation (where 61% were found to be lockable). In contrast, while 37% of participants in rural communities say their toilets have locks, just 17% of rural toilet facilities were found to be lockable and at the time of the survey. It should be noted, however, that there is no standard for toilets to be lockable and inclusion of locks is a household-level decision.

35. Among those participants who say that someone with a disability lives in their household, half in small towns (50%) and just less than half in rural communities (45%) say that the toilet is disability-friendly.26 This compares to 0% and 27% respectively at baseline. While care should be taken when interpreting this result due to the low base size, observations of private household and shared-compound toilet facilities corroborate the finding, whereby only 26% and 9% (respectively) were found to be disability friendly, and this raises equity concerns that merit further investigation

KEQ 4: What do community members and school children know about the hygiene practices which the programme promoted? Are there variations in knowledge in rural communities versus small towns?

36. The Accelerated programme sought to improve knowledge of, and change behaviours around, hygiene within communities, specifically around HWWS and MHM. Some of the activities performed in schools and communities include:

1. Hygiene education through HHETPS, as well as through the implementation of MHM plans and a focus on hygiene pertaining to the health of women and children in schools and health centres collectively.

2. Capacity building of a core group of children called Children of Youth Ambassadors; in this model the CSOs have identified children interested in hygiene promotion and worked with them to build a network of ‘agents of change in schools’. These are involved in influencing their peers around WASH practices.

3. Training schools to construct tippy taps in their surrounding communities to promote HWWS. 4. Training of natural leaders and community health promoters to disseminate the importance of sanitation and

hygiene practices at community level, and subsequent implementation of a natural leaders’ network and Electoral Area Management Teams.

Knowledge among community members

37. The Accelerated programme aimed to reach 500,000 people with hygiene messages and for the same number to be aware of safe hygiene practices. While the indicators are separate and disaggregate between men, women, children and girls, a combined target is provided. According to the KPM database, a total of 432,000 people were reached and aware of safe hygiene practices. As there is a distinct difference between reaching community members with information on HWWS and increased awareness, these targets are insufficient for assessing knowledge of HWWS.

38. The evaluation team therefore used the endline household survey to assess knowledge and to identify gaps between awareness and practice, in particular in relation to the mode of handwashing performed and key times for handwashing.

26 Small base sizes – n=21 in small towns; n=11 in rural communities.



39. At endline, household survey participants were read a number of statements relating to HWWS and were asked whether they agreed or disagreed on a scale of ‘strongly agree’ to ‘strongly disagree’, to evaluate their knowledge on the importance of HWWS. There was no directly comparable baseline data to compare to. The survey found that:

Over half of participants (54%) in small towns either agree or strongly agree that ‘hands that look clean can have germs on them’ compared to over three quarters (77%) of participants in rural communities.

Around nine in 10 participants in both small towns and rural communities agree or strongly agree that ‘washing hands with soap kills more germs than water alone’ (87% and 93% respectively).

Despite this, just under a third of participants in small towns (32%) and almost a quarter in rural communities (24%) agree or strongly agree that ‘washing hands with water prevents diseases, it’s not necessary to use soap as well’.

40. These finding suggest that participants in rural communities are more aware than those in small towns of the importance of HWWS. While participants understand the importance of soap for killing germs, the link between soap and disease prevention is less widely understood in both sub-groups. Further, knowledge does not indicate that behaviour has changed in practice; findings related to changes in behaviour are explored further below in KEQ 5 given that behaviour is also a function of access to HWWS facilities.

41. MMDAs reported that there has been an increased in community knowledge around hygiene practices and sanitation. They highlighted the role of the natural leaders and community champions to increase knowledge of, and change behaviours around, hygiene within communities. Natural leaders and community champions do not only provide information on where to find artisans/community technical volunteers, they first inform household on the importance around hygiene, and its impact on health, and how easy it is to construct low-cost handwashing facilities.

42. Both health centre staff interviewed reported that their health centres had not been offered training, and one staff faced challenges in their efforts to educate the community. Subsequently the health centre official suggested that including men in trainings would be more beneficial in future attempts to educate the community.

‘The women are not really heard and when they even tell them (the community) anything they are tagged as liars.’ – Health centre staff

Knowledge in schools

43. The Accelerated programme had KPM targets around reaching schoolchildren with information on good hygiene:

Through establishing school health clubs and CYA programmes, 160 schools were targeted to have hygiene promotion programmes in place. While no baseline data is available in the KPM database against which to monitor change, 130 schools are shown to have hygiene promotion programmes in place, so this target has not been met.

16,000 schoolgirls and 16,000 schoolboys were targeted to be reached with hygiene messages (including MHM, which is covered in further detail in KEQ 11). The KPM database shows that 42,890 pupils were reached with hygiene messaging. Assuming this is an even split between boys and girls, then the programme met this target, although the data is not disaggregated by gender.



44. Focus group participants in all three schools studied mentioned that health clubs were active in their schools – providing messaging around good hygiene and sanitation practice to fellow pupils and community members, and some pupils mentioned that they were part of the clubs. Although pupils in School A mentioned that there were currently no ‘health ambassadors’ enrolled in the school, they did have sanitation prefects who worked to uphold improvements in person hygiene and sanitation. In School C, health ambassadors said that school health club activities involve advising pupils on how to keep their environment clean, teaching pupils to wash their hands properly, explaining the negative effects of poor sanitation, and providing advice around MHM to girls. In addition, pupils in the Schools B and C elect Ministers of Media, Water and Sanitation who talk to the pupils about hygiene and sanitation on a weekly basis. The health club in Community C also holds pop quizzes around handwashing and fields volleyball and football teams.

‘They are boys who will come and talk about causes and effect of poor sanitation […] on Tuesday, […]at closing assembly, the Minister of Media will come and talk about defecation, then Wednesday, Minister for Sanitation will come and talk about how to dress well concerning how to keep their hair, and dresses; then on Thursday, Minister of Water will come and talk about how to wash hands with soap. Then on Friday, Minister for Hygiene and Sanitation will talk about how to dispose sanitary pad.’ – Student

45. Focus group participants across all three schools studied reported receiving training on good hygiene practices. According to the school administrators and head teachers, pupils are taught by teachers who attend workshops supported by the Accelerated programme before cascading what they have learnt to the pupils, as well as by members of the school health club.

‘After the training we started cleaning our toilets and the urinary and then the children now wash their hands after visiting the toilet.’ – School administrator

46. In focus group discussions, students noted that the training provided by the Accelerated programme had resulted in an increase in their knowledge about hygiene as well as sanitation.

‘I also learnt that after visiting the toilet you have to wash your hands. So, it means that when you go there, you don’t have to touch the faeces because the faeces are organisms that can transmit diseases which can easily affect anybody. You can even get a disease like cholera. That means after going to toilet you have to wash your hands.’ – Student

47. Pupils in the health clubs are taught to cascade their learning to their families and other community members. Pupils in the focus groups in Community C provided some examples of impact outside the school, including one pupil who taught her grandmother the good hygiene practices she had learnt in school, which she said had resulted in her grandmother now washing her hands every day before preparing food. However, focus group participants in School B mentioned that some community members were not receptive to receiving advice from school children – which deterred the pupils from wanting to educate their wider community. Other members of the school health club mentioned that they did want to go out and educate their community but felt they needed the backing of their teachers and chiefs of their communities.



‘We are in need of support from our teachers. So, if they allow us and the people are aware or if the Chief is aware and they announce, then we can go around and there wouldn’t be any problem. But if somebody is sitting at home and you […] go there and you start talking the person will not understand you. So, we are in need of support from our teachers and from the Chief so that they will give us the chance to go round.’ – Student

48. CSOs highlighted that the design of the hygiene promotion activities has been crucial for an effective implementation. The design included regular meeting with teachers on how to implement schools’ activities which helped teachers to speak freely and in a relaxed setting about personal hygiene and menstruation. CSOs also mentioned examples on how children’s attitudes towards sanitation had improved and even exceeded expectations, with children recycling plastic bags and making bins to throw away sanitary towels.

KEQ 5: To what extent do community members and school children have appropriate HWWS facilities? Are there variations in access in rural communities versus small towns?

49. The programme’s activities with regards to HWWS are detailed in KEQ 4. Further, the programme provided water points to schools; it had aimed to provide 180 water facilities in schools and health centres (a combined target) and achieved 106 water points (including 60 boreholes and 46 pipe extensions) in schools alone. Here, the results of the programme in terms of availability of facilities are analysed.

Access to soap in households

50. Targets were set for the proportion of the population with HWWS facilities available, with the programme again aiming for 500,000 people with water and soap available at the designated place for handwashing. According to the KPM database, 204,655 people had water and soap available at the designated place for handwashing by the end of the programme, so the programme met less than half its target.

51. According to MICS 2017, 31% of the population in the Northern region had access to a hygiene facility with soap and water available on the premises at the time of the survey, although this includes areas where the programme was not active. The evaluation’s endline survey therefore also assessed access to soap. Interviewers asked participants if they could see ‘the area where you wash your hands’ and noted the availability of water, soap, ash, mud, or sand.

52. MMDA representatives said that they had seen an improvement in hygiene practices in their districts. Observations of the areas where endline survey participants washed their hands found that 57% of participants in both small towns and rural communities had water and soap/ash available at the time of the survey27. Conversely, 39% of participants in small towns and 40% of participants in rural communities had no water/soap/ash or only water available at the time of the survey. This compares to 44% and 63% respectively who had no water and/or soap available at baseline – a clear improvement.

53. Although the programme’s target refers to availability of soap at the handwashing facility, it should be noted that among those who had no soap by their handwashing facility, only 2% said that there was no soap in their household. Ninety-

27 There is no statistically significant difference between the percentage of observed households with soap available in small towns and rural communities.



eight percent reported that there was soap in the household, but not at the handwashing facility, with no differences between rural communities and small towns. Existing research shows that families in Ghana sometimes hide soap in inaccessible places to stop children finding it. That same research, however, found that lack of immediate access to soap at key moments can negatively impact the practice of HWWS.28 The evaluation team did not collect information on the availability of alternative products like ash to wash their hands elsewhere in the home.

Behaviour in households

54. According to household survey data, at endline, 71% of participants in small towns and 64% in rural communities reported washing their hands with soap on the previous day. This represents no significant change from baseline, when 69% of participants in small towns and 64% in rural communities reported washing their hands with soap and water the previous day.

55. As might be expected, those who practice OD are less likely to say they wash their hands with soap than those who use an improved latrine (69% of those who practice OD reported washing their hands with soap on the previous day compared with 79% of those who use an improved latrine). This likely relates to access to sanitation facilities, with HWWS facilities usually constructed alongside newly constructed toilets.

56. There was no significant difference between the proportion of participants from ODF verified communities who reported washing their hands with soap the previous day when compared to the overall endline sample. This is despite the programme having prioritised handwashing activities in communities where rates of open defecation had been substantially reduced, as poor faecal management can contaminate the water used for handwashing.

Timing and mode of handwashing in households

57. The baseline survey does not provide data on the mode or time of handwashing. However, data from the endline household observation found that more participants in both small towns and rural communities washed their hands under running water (69% and 71%, respectively) than jugs or basins (29% and 31%, respectively).

58. In terms of the times of HWWS, 31% of participants in small towns and 37% in rural communities reported washing their hands with soap before cooking food, 67% and 81% before eating, 45% and 47% after eating, 91% and 94% after visiting the toilet, 30% and 35% after disposing of a child’s stools.

59. Women are more likely than men to say they wash their hands after disposing of a child’s stool (45% versus 16% respectively), after cleaning a baby (49% versus 16%, respectively) and before food preparation (45% versus 19%). This likely reflects the gendered division of childcare and household responsibilities.

Menstrual hygiene management (MHM) in households

60. There are no direct KPM targets for improvements to knowledge or practice of menstrual hygiene at the community and household level. While many of the activities related to improving knowledge of MHM took place within schools and

28 Scott, B. Rabie, T. and Garbah-Aidoo, N. Health in our hands: understanding hygiene motivation in Ghana. Health Policy and Planning. 22/4. 2007.



are discussed in detail in KEQ 11, they have broader implications for MHM knowledge and practice in communities and households.

61. Interviewees repeatedly highlighted the MHM promotion activities of both the Enhanced and Accelerated programmes as being some of its most successful. Both those involved in programme implementation and those in advisory roles spoke of the impact that MHM activities in schools have had in improving knowledge and awareness of MHM in communities outside of the classroom, particularly among men and boys.

62. However, findings from the household survey suggest that improvements in MHM have not been universal. Around three quarters of female endline survey participants in both small towns and rural communities (75% and 72%, respectively) report using a sanitary pad to manage their menstrual cycle when it occurs. Among women in small towns, this represents no significant change from baseline when 73% used a sanitary pad; however, this represents a marked improvement in rural communities where only 37% of women used a sanitary pad at baseline. Ten per cent of women in small towns and 17% of women in rural communities use old clothes rather than sanitary products at endline, compared to 19% and 49% respectively at baseline. More women at endline are using cotton wool than at baseline, however (14% at endline compared with 1% at baseline in small towns and 10% compared with 4% in rural communities).

63. Nevertheless, despite a majority of participants using appropriate materials for MHM, women and girls rarely dispose of their sanitary products safely. The UNICEF and Ghana Education Service guide to menstrual hygiene management stipulates that sanitary pads should be wrapped and disposed of in latrine bins in the school, community, or at home. The guidelines state that sanitary materials should not be disposed of in toilet bowls or toilet pits, on the toilet floor, and that they should not be thrown into piles of refuse. While plastic buckets with lads are provided in school changing rooms as per the WASH in Schools National Implementation Model, there may be a lack of available bins in the wider community for use by adult women. Indeed, only 3% of female participants in small towns and no female participants in rural communities reported disposing of their menstrual hygiene products in a special bin. Around a fifth (21% in small towns and 23% in rural communities) in each used a latrine; 23% in small towns and 45% in rural communities buried their materials; 29% and 14% respectively disposed of them in a field, bush, water body, beach or other open space; while 7 and 8% respectively burnt their materials after use. This may relate to deeply held norms and taboos around menstruation – which can lead to a culture of silence and secrecy around MHM, compounded by a lack of availability of latrine bins for private and proper disposal.29 Indeed, around a fifth (18%) of female endline survey participants say there are work days or social events that they missed due to their last menstruation. Unfortunately, there is no comparable baseline data available against which to assess change.

Access to handwashing facilities in schools

64. The Accelerated programme aimed to provide 180 water facilities in schools and health centres (a combined target) and achieved 106 water points (including 60 boreholes and 46 pipe extensions) in schools alone (as well as 58 water points in health centres, including 54 boreholes and 4 pipe extensions), according to the KPM database. Borehole drilling had high failure rates in the Northern region; this posed a challenge for provision of water supply to institutions.

65. Across the schools studied, focus group participants described that tippy taps and/or Veronica buckets are available. In School C, students described that tippy taps are located near the toilet facilities – to encourage handwashing after use – and elsewhere on the school compound. Pupils in School B use both tippy taps and Veronica buckets; however,

29 See ‘Notes on Cultural Aspects of Menstruation in Ghana’, Christin Oppong, World Bank, 1974; and ‘Overcoming the Taboo: Advancing the Global Agenda for Menstrual Hygiene Management for Schoolgirls. Sommer, RN. And Sahin, M. American Journal of Public Health. 2013; and



in School A, they rely solely on Veronica buckets. While handwashing facilities are available, tippy taps are yet to be constructed.

66. A further challenge raised was the functionality of the handwashing facilities in School C, where the administrator noted in interview that the taps that were provided by the programme are not functional and that neither the contractor who built them nor UNICEF handed these over to the school. The school administrator in School A also noted that provision of soap at the school is a challenge which impacts handwashing; it is difficult to provide soap to students year-round because they receive capitation funds on a yearly basis. When soap finishes, they must wait to receive capitation funds again to buy soap.

‘We also have this challenge of the taps. In fact, there has not been any handing over of that thing and those taps were not even working. […] So, the contractor didn’t hand it over to us and UNICEF too didn’t hand it over to us.’ – School administrator

67. Pupils in both focus groups in Community B noted that, during the dry season there is a considerable decrease in water supply. Subsequently, drinking water is scarce and water for handwashing is deprioritised. Pupils are concerned, therefore, that the new handwashing facilities may not be maintained during the dry season and progress will not be sustained. Some students said their families resort to using water from rivers or streams to wash their hands during this period (though the household survey found no significant difference between use of improved/unimproved water for handwashing during the rainy season and the dry season).

Behaviour in schools

68. While the KPM database provides no targets directly relating to hygiene behaviours among schoolchildren, it is important to understand how children’s behaviours have changed so as to identify the results of the programme’s activities related to its objective of facilitating the adoption of good hygiene practices for improved health.

69. Focus group participants in all three schools described that hygiene activities in schools had facilitated behaviour change, and they attributed this directly to the school health clubs (though pupils were less clear on the role that UNICEF and the Accelerated programme have played in establishing those clubs). In Community B, pupils in both focus groups emphasised that as a result of the school health club, there is now widespread awareness of good hygiene and sanitation practices and a strong desire among pupils for both their schools and communities to become ODF. Pupils are now reportedly using soap, Omo or Dettol to wash their hands and practice handwashing after using the toilet facilities. In School A, students mentioned that they regularly report their peers to the teachers if they see them defecating in the open.

‘I wasn’t washing my hands after using the toilet. Previously after visiting the toilet, when I get food, I eat without washing my hands. And I started experiencing stomach ache, and I didn’t know the cause of that. But now since I wash my hands after using the toilet, I don’t experience the pains again.’ – Student



KEQ 6: What is the access to safe water supply (including HWTS) in the targeted rural communities and small towns?

70. The programme did not aim to directly improve access to clean water through community-level activities but sought to improve access more indirectly through government engagement, advocating for government to commit to reaching the final 20% of people across the country without access to potable water, and through institutional-level activities (in schools and health centres). There were nevertheless a number of activities carried out at the community-level to improve access to safe drinking water, including education and behavioural change activities around safe water, including WASHSPLASH, water management training provided to households by the CWSA, training individuals to maintain and repair institutional water points, and national level advocacy. Some MMDAs mentioned that the community-level water supply component of the programme was limited in comparison to sanitation and hygiene, with district staff mainly focussing on improving knowledge and changing behaviours around sanitation hygiene, rather than on provision of safe water infrastructure. Although it should be again noted that the programme did not have a goal of improving community water supply or significant activities related to water, the evaluation nevertheless considered access to safe water supply in order to address the KEQ and build the evidence base regarding water access for future programming guidance.

71. These activities were particularly challenging in the Accelerated programme compared to the Enhanced programme, due to the geographical remit of the programme’s interventions; the programme’s Northern region focus meant that activities were hindered by contextual factors (hydrogeological conditions) which impact access to groundwater. Borehole drilling had high failure rates, and some stakeholders including government and CSOs, noted that there were issues with inconsistent quality in feasibility studies for borehole drilling.

72. The KPM database for the Accelerated programme lacks indicators and targets related to water supply and household water treatment and storage (HWTS) given the limited programme activities around water access. There is also a lack of robust comparable data on access to safe drinking water between baseline and endline in small towns and rural communities that is aligned with current global indicators and targets.30 The indicators used in the SDGs represent a shift in focus from the location of water sources (which defined the MDG targets for WASH) to the time spent collecting water, differentiating between ‘basic drinking water services’ – defined in SDG 6.1.1 as those that are improved but for which collection time is no more than 30 minutes, ‘limited’ – referring to improved services that are more than 30 minutes away, and ‘unimproved services’. Nonetheless, the endline survey found that:

A third of endline survey participants in small towns (33%) and almost half of participants in rural communities (46%) still rely on surface water.

In both small towns and rural communities, around half of participants (53% and 50% respectively) are using improved sources of drinking water during the dry season.31 This compares to 78% in small towns and 61% in rural

30 For example, with relation to the time participants have to spend travelling to collect water from improved sources. 31 At endline, improved sources of drinking water refers to water from the following sources: pipe-borne inside dwelling, pipe-borne outside dwelling, public tap/standpipe, tube well/borehole/pump, protected spring, rainwater harvesting, bottled water/sachet water (if the secondary source used by the household for cooking and personal hygiene is improved). These criteria are in line with JMP criteria. The baseline report did not specify its definitions for improved water sources.



communities at baseline. There was no significant difference between male and female endline survey participants in access to acceptable/improved sources of drinking water during the dry season.

In both small towns and rural communities, 75% of endline survey participants spend 30 minutes or more on water collection (round trip, including travelling, queueing and returning, for either improved or unimproved water) during the dry season. This compares to 38% in small towns and 51% in rural communities at baseline.

Women have primary responsibility for collecting water in both rural communities and small towns (where 91% and 92% of participants, respectively, said women collect water for family consumption)32. Women are also primarily responsible for storing and managing water, with 96% of participants in both rural communities and small towns reporting that women store and manage water.

73. The CWSA also provides official statistics on regional district level rural coverage of improved water sources for the years 2014-2017. From 2014 to 2017, largely during the years that the Accelerated programme was active, the number of households with access to improved water (rural population served) was increasing year-on-year in all the Accelerated districts. However, this has not been matched by an increase in population coverage, especially from 2016 to 2017; when looking at the rural population for each district, it can be seen that the population has increased annually. This suggests that population growth is outstripping the capacity of current interventions (both government and NGO-led) and technologies to meet the growing demand of the rural population – which may be an issue affecting the Accelerated rural communities.

Progress towards global targets and indicators

74. The endline household survey provides figures for access to safe water supply in the target regions and districts that can be aligned to global targets and indicators (although there is no directly comparable baseline data available against which to monitor change). The survey shows that:

Just 1% of participants in small town and 0% of participants in rural communities have access to improved sources of water that are piped into their household.

Around half of participants in both small towns (52%) and rural communities (50%) rely on other improved/acceptable sources that are not piped into the dwelling, while 47% and 51% respectively use unimproved sources (either household or public).

75. SDG indicator 6.1.1 relates to the ‘proportion of population using safely managed drinking water services’. The Goal aims for countries to achieve ‘universal and equitable access to safe and affordable drinking water’ for all by 2030. The household survey found that:

Around a quarter of participants in small towns and rural communities (23% and 27% respectively) have access to at least basic sources of drinking water (defined as an improved source of water for which collection time is no longer than a thirty-minute round trip). This is very low when compared with the percentage of the total rural population that have access to at least basic drinking water in Ghana (68%).

32 There is no statistically significant difference between rural communities and small towns in terms of the percentage of participants who say women are the primary collectors of drinking water.



Twenty-nine percent of endline household participants in small towns have access to ‘limited’ sources of drinking water – referring to improved sources of water that are more than a thirty-minute round trip away, compared to 22% in rural communities.

KEQ 7: How is faecal sludge managed in the targeted small towns?

76. The programme proposal highlights the precarious situation when it comes to sustainable management of faecal sludge in Ghana and the need to develop the urban sanitation framework by carrying out a sustainable environmental assessment and reforming Ghana’s national environmental policies, plans and programmes. Despite this, the programme did not have any concrete activities around faecal sludge management nor associated targets, and this subsequently has not been included in the ToC for the programme.

77. According to the stakeholders interviewed, the faecal management in small town as well as in rural areas is poorly controlled and it is having significant impacts on the environment. There are districts that have sceptic tanks but lack treatments plants or have treatment plants that lack technology to adequately dispose of waste; other districts do not yet have the infrastructure and so dispose of faecal waste by digging holes or dumping it in nearby water streams. Stakeholders highlighted that the treatment of faecal matter should be a priority for small towns and that small towns. At rural level, the need to treat faecal waste is less, as many households use salt to dissolve this, and CSOs considered this to be an appropriate mid-term solution until effective faecal sludge management systems are in place in small towns.

78. To have a functioning faecal sludge management system in Ghana, substantial funding and a national strategy would be required. Stakeholders highlighted that the rural sanitation strategy does not place much emphasis on faecal management, and there is currently is no clear national strategy on urban sanitation. CSOs mentioned the work carried out under the Greater Accra Metropolitan Area Sanitation project with funding from the World Bank, which contributed to the development of urban sanitation in Accra. Other agencies (e.g. African Development Bank) are also working together with the government of Ghana on developing an urban sanitation strategy for Accra which could potentially be replicated in other cities and consequently in small towns. Further, UNICEF is also working with the government under a separate programme funded by the government of the Netherlands to develop a liquid waste strategy, which is now being validated and should be completed in 2020.

PRELIMINARY CONCLUSIONS: EFFECTIVENESS WASH IN RURAL COMMUNITIES AND SMALL TOWNS

KEQ 3: The programme missed its targets for household sanitation, although this target may have been ambitious or based on incorrect population estimates (Paragraphs 21 and 22). However, there has been a notable change in behaviour around OD, with the proportion of communities practicing OD almost halved since baseline in both small towns and rural communities. OD has reduced more substantively in Accelerated programme small towns and rural communities than across the region as a whole (Paragraph 23). Most survey participants who use a toilet reporting that their toilets were built after the programme commenced its activities (Paragraph 23). Among those who still practice OD, none admitted to doing so by choice. Rather, most defecate in the open due to lack of access to public or private toilets (Paragraphs 24 and 25). Use of public latrines is higher in small towns than rural communities, while use of private facilities is higher in rural communities (Paragraph 32). However, access to improved unshared facilities is highest in small towns (Paragraph 23) – where those who can afford to do so purchase advanced toilets from artisans, and where those who cannot afford to do so may lack financial support to construct a latrine (Paragraph 32). Toilet facilities in small towns are more gender-



and disability-friendly than in rural communities, however gender and disability friendly features are generally lacking in both (Paragraphs 34 and 35).

KEQ 4: Although the programme did not fully achieve its target for community members reached with hygiene messages (Paragraph 37), the survey provides evidence of knowledge of some hygiene messages. Participants in rural communities are more aware than those in small towns of the importance of HWWS, but in both sub-groups, there is a lack of knowledge of the link between soap and disease prevention (Paragraphs 39 and 40). The programme has been effective in establishing hygiene programmes including health clubs and ambassador programmes in schools, supporting members’ knowledge about hygiene and dissemination of learnings within the school (Paragraphs 43-46). In some cases, members had also cascaded learnings to the community, although students did not feel fully empowered to do so (Paragraph 47).

KEQ 5: Although the programme fell short of its target for people with water and soap available at the designated place for handwashing (Paragraph 50), the endline household survey found evidence that access to soap has increased (Paragraph 52). Where this is not present at the handwashing facility, it is generally reported to be available elsewhere in the household (Paragraph 53). However, there is still a gap in HWWS practice. Particularly concerning are the low levels of HWWS before cooking and after disposing of a child’s stools, which are both gendered activities disproportionately carried out by women. MHM has also improved markedly in rural communities (58 and 59). The programme’s provision of HWWS facilities and development of health clubs in schools also supported improvements in hygiene (Paragraph 69), although there are some concerns regarding schools whose facilities have not yet been constructed and lack of soap for students (Paragraphs 55 and 56).

KEQ 6: Activities to improve access to clean water at the community-level were limited, and the programme’s contribution to changes are therefore not easily quantified (Paragraphs 70-72); however, the evaluation team considered the current access to safe water in response to the KEQ and in order to build the current evidence base regarding water access.

Access to drinking water is limited in the Northern region, with only around a quarter of endline participants in rural communities and small towns having access to at least basic drinking water (Paragraph 75). This is exacerbated by hydrogeological conditions in the region which make drilling success low, and there were some concerns about inconsistent quality in drilling feasibility studies (Paragraph 71). Sources of drinking water are generally located far away from people’s houses (Paragraph 72). The high proportion of women with primary responsibility for collecting and managing water is especially concerning given the large proportion of households travelling long distances to access clean water and could have a detrimental impact on women’s involvement in social and economic life, although again, it should be noted that the programme did not aim to address safe water access (Paragraph 72).

KEQ 7: There are few cities and towns that have treatment plants and technology to dispose of waste adequality in the Northern region (Paragraph 77). There is therefore a need for a comprehensive urban sanitation strategy. Although the Accelerated programme was not involved in faecal sludge management activities, UNICEF is working under a separate programme with the government to develop a liquid waste strategy (Paragraph 78).



1.2.2 Effectiveness - WASH in institutions (schools and health centres)

KEQ 8: To what extent do school boys and girls have access to school sanitation facilities? What is the functionality status of WASH facilities in schools? Are there variations between rural communities and small towns?

79. The Accelerated programme aimed to provide 120 schools with gender sensitive, disability friendly and environmentally friendly sanitation facilities. Activities were targeted at those schools with the lowest recorded standards of sanitation and hygiene – including those with few or no existing facilities. By the programme end, 93 sanitation facilities had been constructed and 42 rehabilitated, according to the KPM database. According to a database shared by UNICEF, the Accelerated programme’s provision of WASH facilities in schools was concentrated mostly in small towns.

80. SDG targets 4.a 6.1 and 6.2 include indicators for monitoring WASH in schools – reflecting increased global recognition of the central role that schools can play in achieving universal WASH access and establishing social norms around good hygiene and sanitation practices from an early age. The key indicator relating to sanitation focuses on the proportion of schools with single-sex basic sanitation. These indicators are based on those defined in the UNICEF/WHO JMP multi-level service ladders for monitoring WASH in schools. In these ladders, schools with ‘basic sanitation’ are defined as having improved facilities that are single-sex and useable (accessible – with doors that are unlocked functional – with water available for flushing, and private - with closable doors that lock from the inside), those with a ‘limited service’ are those that have improved facilities but where the facilities are not gender-segregated or are not useable, and those with ‘no service’ are schools that have no toilets or latrines or unimproved toilets’.

81. Further, in 2014 under the Enhanced programme, UNICEF supported development of national guidelines, serving as a blueprint for Government to use for all WinS interventions in the country; taking into consideration the provision of child-friendly sanitation facilities in schools.33 Thus, UNICEF’s targets for child-, disability- and gender-friendly toilets are based on the definitions set out in these guidelines, and these are the same definitions used in this evaluation.

82. The evaluation team interviewed school administrators and carried out focus group discussions with male and female pupils in three schools to discuss the availability and usability of the WASH facilities that had been constructed at endline.34 Positive improvements have been noted in all three schools and, while there are some issues (discussed below), all three schools are providing a basic sanitation service, with reference to JMP/SDG indicators.

83. In School A, a primary school, separate toilets have been constructed for girls and rehabilitated for boys; however, there is no changing room for girls. Despite the availability of facilities, pupils said they frequently avoid using the toilets – largely because the facilities are reportedly dirty and unpleasant. Focus group participants attributed this lack of cleanliness

33 These guidelines stipulate that ‘a maximum of 50 pupils should be using one drop hole/toilet cubicle; separate blocks should be provided for boys’ and girls’ toilets; changing rooms for adolescent girls should be provided in each toilet block; toilets should be lockable from the inside; toilets for children should be appropriately sized to ensure the children are comfortable and feel safe in the use of the toilet. Foot rests for squatting should be appropriately sized and positioned for age and sex; toilets should be located at reasonable distances from school buildings to enable ease of access; school toilets should be designed and constructed for ease of use by all categories of pupils including the very small and those who are physically challenged; and all toilets should be disability-friendly, with an access ramp provided to the toilet, at least one cubicle or drop hole for use by children with disabilities, railings at the side and back of the cubicles and special grips, guiding systems and proper lighting for those pupils who are poor-sighted.’ 34 It should be noted that observations did not take place to verify any statements made during the qualitative research. As such, the findings mentioned below relate to the perceptions of the interviewees and indicate potential areas of interest, rather than generalisable conclusions.



to kindergarten pupils who misuse the toilet as well as to community misuse of the facilities. (Challenges of community misuse of toilet facilities are detailed further under KEQ 10.)

84. Progress in School B, a junior high school, is more positive. Both focus groups reported improvements in toilet facilities and attributed these improvements directly to the activities of the Accelerated programme, with UNICEF building gender-segregated toilets and a changing room for girls. There is also a separate toilet for teachers. However, there were challenges in this school due to misuse and vandalism of the facilities by members of the community. As a result of damage and community misuse, the school administer reported that they now simply leave one of the broken teachers’ toilets unlocked so that the community can use this instead of vandalising and misusing the rest.

85. In School C, a junior high school, a toilet facility for girls has been built as part of the Accelerated programme, as has a changing room. The old, previously shared, cubicle is currently being used for the boys until it is renovated. The head teacher stated that UNICEF has promised to carry out this renovation, however at the time of the focus group, this facility has not been rehabilitated. Again, challenges due to misuse by the community were described.

‘When you put a lock, they will break it, even the girls’ own they will jump the wall to come and use it. If you talk, they want to beat you.’ – Student

KEQ 9: To what extent are the school sanitation facilities gender- and disability- friendly? Are there variations between rural communities and small towns?

86. KEQ 8 sets out the programme’s targets and achievements for construction of toilets, which were targeted to be child-, gender- and disability-friendly. In this section, the gender- and disability-friendliness of the facilities is specifically discussed. Again, to address this question, the evaluation team relied on the evidence collected from the focus group discussions.

87. Progress reports provide examples of latrines constructed in targeted schools. These are considered gender friendly as they are equipped with changing rooms, and they are also described as disability friendly, although there is no mention of the construction of ramps, and these are not visible in the pictures provided. Evidence collected from the focus group discussions provides more details on the how facilities provided are gender- and disability- friendly

88. As noted under KEQ 8, in the three schools, gender-friendly toilets have been constructed; plus, in School C the new female facility is disability-friendly and has a separate cubicle for people with disabilities to access. No information has been provided around access to sanitation facilities for pupils with disabilities in the schools studied in Schools A or B.

89. As the newly constructed toilet facilities appear improved and gender-friendly, the schools can be considered to be providing an ‘at-least-basic’ sanitation service with reference to JMP/SDG targets. In School A, although the toilets are gender-segregated, there are no changing rooms for girls. In Schools B and C, girls were confident about their period and acknowledge using the available changing rooms when they had the period. In School B, sanitary pads are provided in the girls changing room at school and girls are advised to wash three times a day; MHM is discussed further in KEQ 11.

‘We tell the girls that if they are menstruating, they have to bath three time a day morning afternoon and evening.’ – School administrator



90. It should be noted that as of 2018, the provision of changing rooms alongside female toilets is a standardised requirement for the construction of school toilet facilities for adolescent girls in Ghana. Although the standard refers to ‘adolescent girls’, UNICEF noted that the standard applies to both primary and junior high schools; hence, all school toilet facilities constructed in secondary schools from 2018 onwards should include female changing rooms. This may contribute to a further reduction in school absenteeism and may already have contributed to the improvements mentioned by school administrators, although it is too early to be able to monitor any direct impact. It remains important that changing rooms are also constructed in schools where toilet facilities were built prior to 2018, however.

KEQ 10: What proportion of schools have facility management plans in place?

91. No KPM targets were set around the maintenance of facilities, nor around training staff or pupils to look after the newly constructed facilities. Proper maintenance of WASH facilities is essential to ensure that facilities remain operational and any improvements in sanitation practices are sustained in the long term. Indeed, according to the focus group discussions and school administrator interviews, maintenance appears to be an issue in all three schools.

92. In all the schools studied, focus group participants described the maintenance schedules in place and students’ role in carrying out cleaning. According to pupils in the focus groups, in School A, pupils are responsible for maintaining the facilities. Pupils reportedly sweep the toilets and clean the area, and certain classes are responsible for washing the toilets three days a week. The school administrator also explained that maintenance activities were introduced as a result of the programme. Nevertheless, as mentioned in KEQ 8, the students in the focus group in School A also reported that the school toilet facilities often become dirty and unpleasant, leading to pupils feeling uncomfortable using the facilities.

‘What has changed here is usually we were not cleaning our toilets. After the training, we started cleaning our toilets and the urinary and then the children now wash their hands after visiting the toilet.’ – School administrator

93. Similarly, in School B, focus group participants reported that pupils are responsible for cleaning the toilet facilities, with this responsibility sometimes being allocated as a punishment for bad behaviour, but they also noted the responsibility of the health club members to clean the facilities and provide water.

‘[It] is the responsibility of the Minister for Hygiene and the Minister for Sanitation. These are the two people who are to supervise that. We have a group who are under the Ministers. They help to fetch the water. And also, at the end of the day they go round and visit the toilet and the environment and come back to give a report whether is clean or not. […] In the school here the juniors are always responsible for the cleaning of the toilet. They always let the juniors clean it.’ - Student

94. In School C, focus group participants and the administrator described a plan for rotating responsibility for cleaning the facilities among students and noted that they receive ongoing training about hygiene and sanitation, as well as reminders to clean the facilities. They also described the responsibilities of health club members, referred to as the Minister of Water and the Minister of Health, who refill the tippy tap and collect sanitary waste.

‘Every Monday they advise use how to keep our environment clean and how to wash our hands. They also talk to us about to dispose anything bad.’ – Student



95. Focus group participants provided considerable detail on the issues around outsider use in the schools studied. In School C, for instance, community members reportedly jump the school wall or break the lock to use both the girls’ facilities, due to a lack of community facilities. Specifically, some reportedly defecate on the veranda or the floor of the toilet instead of the hole, others defecate at the entrance of the toilet, and some community members also vandalise the toilet, writing on the walls and dumping their rubbish into the hole.

96. Pupils in School B also mentioned that community members sometimes jump the wall to use the school toilet facilities, even when the gates are locked. Similarly, pupils in School A spoke of community members breaking locks and removing gates to gain access to the school toilet facilities.

97. This issue is apparently compounded by the significant number of community members who still practice OD in these communities. Focus group participants mentioned that those community members who do practice OD break into the school to defecate around the compound.

‘Especially during vacation they will come and use the toilets, but they will not clean; they even defecate around the compound and the entrance of the toilets. Some will go to the bush where we dump our rubbish; some will do it outside of the toilet.’ – Student

98. This likely reflects the low overall access to improved sanitation facilities in this region, and in particular, to high quality household sanitation facilities and/or public facilities in the rural communities and lack of access to affordable household facilities for a significant portion of the small-town population. Indeed, according to UNICEF’s CLTS Communities 2019 database, all three of the communities where the schools studied are located have been CLTS triggered, but none have achieved ODF basic status.

99. Some focus group participants across schools also mentioned that more soap, toiletries and litter bins needed to be provided in order to maintain the facilities.

KEQ 11: What is the level of knowledge on menstrual hygiene management and hygiene behaviours?

100. The Accelerated programme aimed for 16,000 schoolgirls to receive MHM training or support within their school and for 16,000 girls to be reached with messages on MHM, and the KPM database shows that the target was exceeded, with 20,066 schoolgirls having received training on MHM. According to UNICEF staff, the programme was implemented the same way across schools, but some schools went beyond the training provided, for example by offering emergency sanitation supplies to pupils.

101. MHM training programmes were active in all three of the schools studied, though the quality and content offered varied from school-to-school.

102. School C appears to have in place a well-developed MHM plan. The plan seeks to educate girls on how to manage their period. In addition, the school provides female pupils with sanitary products for MHM. Despite this, girls in the focus groups mentioned that their parents still provide them with less effective and unhygienic materials for managing their period, including toilet roll, cloth and cotton wool. The school health club teaches both boys and girls about menstruation, encourage girls to take appropriate medication if they are in pain, and teaches boys to respond appropriately if they see girls with stains on their clothes. Overall, the boys in the focus group appeared well educated in how to respond sensitively



to girls staining their clothes, but the girls in the focus group knew little about calculating their monthly cycles. The school administrator said that these activities have already had a positive impact on improving attendance rates among schoolgirls, as girls who would otherwise stay home during their menstruation are confident to be in school.

103. Girl participants in the School B focus group reported having been taught how to hygienically deal with their cycle, advised on bathing frequency and taught to use a sanitary pad. However, the school does not provide materials for MHM. Subsequently, some girls who do not have the money to buy sanitary pads use old cloth instead. Although the girls showed evidence of being educated about personal hygiene around their menstruation, they do not know how to calculate their monthly cycles, which can leave them feeling embarrassed if they stain their school uniform, for example.

104. In School A, the boy participants in the focus group added that they had been educated around menstruation and taught to advise girls to wear a pad. However, activities in this school seemed limited relative to the other schools and the boys expressed less knowledge of how to handle these situations sensitively.

105. Despite the MHM programmes also educating boys, girls in the focus group discussion in School B mentioned that boys often still laugh at girls who menstruate, causing embarrassment for the girls (which can lead to absenteeism). However, boys in the focus groups mentioned that most of this behaviour tends to come from junior pupils, and that senior pupils try to intervene. Similarly, in School A, boys noted that bullying of girls is an issue which is responded to with punishment.

‘I remember Ione girl had her menses and a stupid boy laughed at her and our […] seniors punished him and asked him to bring a flower to plant on the compound.’ – Student

106. Though there is no baseline figure to compare to, around a fifth of female endline survey participants in small towns and rural communities (17% and 22%, respectively) say that there are social activities, school days or work days that they did not attend due to their last menstruation.

KEQ 12: To what extent are WASH facilities in health centres and CHPS compounds available? What is the functionality status of WASH facilities in health facilities?

107. According to the KPM database, the Accelerated programme achieved the following with regards to WASH facilities in health centres:

Sanitation facilities were constructed in 58 health centres/CHPS compounds (target: 60 health centres and CHPS compounds with gender sensitive sanitation facilities);

Water facilities were constructed in 56 health centres/CHPS compounds, comprised of 52 boreholes and 4 pipe extensions (target: 180 schools and health centres with water facilities); and

1,200 staff were trained in 84 institutions on hygiene promotion (target: 80 health centres with staff trained in hygiene promotion).

108. Thus, the target for trained staff was significantly exceeded, while the target for construction of WASH facilities was nearly achieved (97%). Achievement against the target for water facilities cannot be measured since the target was combined for schools and health institutions.



109. According to national government stakeholders, the programme’s support was welcome, as construction of WASH facilities in health centres is normally carried out by these offices and UNICEF provided funding. UNICEF’s funding enabled construction where no facilities were available, ensuring that people will not defecate around the health centres. Government representatives also mentioned how the WASH facilities constructed were disability-friendly.

110. To gather a more in depth understanding of the programme’s activities in health centres, this evaluation interviewed staff from health centres in two communities. According to both interviewees, the programme’s activities in these health centres were particularly limited. Water and sanitation facilities have reportedly been constructed in both health centres as part of the Accelerated programme, however the staff member interviewed felt the quality was poor. The toilet constructed in one location is not disability-friendly, and the toilet compound lacks structural support. The interviewee added that their toilet needed fundamentally restructuring, the metal doors fixing, and a fence placed around the facility. Although the staff member in the other facility considered the necessary repairs to be minor, they felt that there is little support in place for such repairs to be carried out. The health centre has Veronica buckets and a tippy tap, but the latter has not been set up yet.

111. Both interviewees reported issues around the handing over of the facilities and strained relationships with UNICEF. UNICEF staff explained that protocols are in place for contractors to hand over responsibility for health centre WASH facilities to GHS, who then hands over to the health institution. However, in one community, the health centre staff felt that UNICEF itself was responsible for handover and that this did not occur for a long time, by which point, the toilet had been already damaged.

112. Health centres face similar issues to schools in terms of outsider use. Interviewees in both health centres detailed their boreholes being used and damaged by community members, even when locked. Community members reportedly removed the gates in order to access the facilities. As was the case for the schools visited, these health centres were in communities that are not yet ODF, which is likely contributing to misuse of facilities; toilets have become damaged and unhygienic after use.

113. UNICEF staff noted that operations and maintenance training is provided by GHS, who owns the facilities, but that this was not instituted at the outset of the programme, and that an operations and maintenance manual is currently being prepared to support ongoing maintenance.

114. In terms of hygiene practice, while one interviewee reported seeing improvements in hygiene practices among patients, the interviewee in the other community mentioned that they have found it particularly challenging to encourage visitors to wash their hands with soap. This may relate to possible a lack of training for health centre staff, or the fact that the community is not yet ODF and so hygiene promotion activities have not yet been rolled out in the community. According to the interviewees, neither health centre had been offered training in hygiene promotion.



Preliminary conclusions – EFFECTIVENESS – WASH IN INSTITUTIONS:

KEQ 8: The programme exceeded its target for provision of sanitation facilities in schools, having constructed 93 facilities and rehabilitated 42 (a total of 135 schools against a target of 120) (Paragraph 79). However, evidence shows that some students may still be lacking access to new facilities (Paragraph 85). Outsider use is damaging the facilities and making it difficult for the children to maintain them (Paragraphs 83, 84, 95 and 96). This was generally attributed to a lack of private or public facilities within communities and to the significant portion of the community that still defecate in the open (Paragraphs 97 and 98).

KEQ 9: The programme exceeded its target for provision of gender- and disability-friendly toilets in schools (Paragraph 79). However, changing rooms were not provided in one of the schools studied, which was a primary school; given, national guidelines stipulate that changing rooms must be provided for adolescent girls, there may remain an unmet need for younger girls, and this could be a point of interest for future research (Paragraphs 83, 89 and 90).

KEQ 10: No KPM targets were set around the maintenance of school sanitation facilities, nor around training staff or pupils to look after the newly constructed facilities (Paragraph 91). However, qualitative data collected shows that in the schools visited, students are actively involved in facility maintenance (Paragraphs 92 and 93). While this is seen as the collective responsibility of students, health club members had specific leadership roles in maintaining facilities. Nevertheless, outsider use is a major challenge and has resulted in misuse and vandalism of the facilities (Paragraphs 95-98).

KEQ 11: The programme achieved its targets in terms of the number of schools providing training on MHM (Paragraph 100), and MHM programmes were active in all three schools studied (Paragraph 101). The quality and content various from school-to-school, however, and additional work is needed to ensure that a consistent high-quality service is available to all pupils (Paragraphs 101-105). Some schools provide girls with sanitary materials for managing their menstruation (Paragraph 102), but where this is not provided, those who do not have the money to buy sanitary pads use cloth or old clothes (Paragraph 103). Girl students across all focus groups were largely unaware about how to calculate their monthly cycles, despite the training they receive from the MHM programmes (Paragraphs 101-105). In all three schools, boys are also taught about MHM. While this has proven effective in some schools, in other boys reportedly still make fun of girls during their menstruation (Paragraph 105).

KEQ 12: The programme nearly achieved its target for construction and rehabilitation of WASH facilities in health facilities and significantly exceeded its target for training of health centre staff (Paragraphs 107-108). While it is difficult to make any broad claims about the effectiveness of these activities, due to limitations around primary data collection, findings from the interviews with health centre staff raise concerns for further investigation, regarding the quality and maintenance of the facilities and the handover of the facilities to the institutions (Paragraphs 110-113). Reportedly, health centre facilities are used and damaged by community members who lack access to private or public toilets, which is likely a challenge because the institutions were in communities that are not yet ODF (Paragraph 112). Limited available support for maintenance and a lack of training in hygiene promotion or maintenance are concerns for sustainability of the intervention in health centres.



1.2.3 Effectiveness - District level WASH access

KEQ 13: How many certified ODF communities are there within each of the targeted districts, and to what extent have the number and proportion of the population living in ODF communities increased?

115. Achievement of ODF certified community status was intended to be reached both through supporting behaviour change through the CLTS model and provision of WASH infrastructure in institutions, as well as capacity building activities to support the verification and certification process. Achievement of ODF verification at district level was intended to be supported by roll-out of the STSM and RSMS in rural communities through a district-wide approach, targeting both small towns and rural communities in the same districts; the district-wide approach is assumed to create a ripple effect as ODF achievements in rural communities surrounding small towns puts pressure on the small towns to achieve same status35. The KPM database includes the following targets related to the achievement of ODF status: 1) 80 ODF small towns in 10 districts, 2) Five ODF districts, 3) 500,000 people living in ODF small towns and rural communities, and 4) 10 districts with evidence-based ODF plans. UNICEF staff also noted that the programme aims to ‘contribute to’ these targets, noting the role of other potential factors in driving ODF certification as well that are beyond the programme’s remit.

116. The KPM database shows that 411 rural communities and 34 small towns have achieved ODF status (against a target of 80 small towns). The target of five ODF districts has also not been met; though as of 2019, six districts were at least 70% ODF, a promising step towards the target figure. The database does not provide the current number of people living in ODF small towns and rural communities but does show that 204,655 people live in households with access to improved sanitation, significantly below the target of 500,000. However, the programme did make significant steps toward supporting ODF certification at district level, including establishing evidence-based ODF plans in 10 districts, and as discussed in KEQ 3, OD rates have declined.

117. Even though the ODF district target was not met, district authorities believe the programme has had significant impact in the decline of OD rates. Most of the MMDAs mentioned that there had been an increase in ODF communities in their districts and, importantly, one mentioned that there had been a spillover effect whereby communities that had not yet been triggered were changing their behaviours in response to neighbouring CLTS triggered communities.

‘There is evidence of ODF due to the trigger and even surrounding communities without the triggers are learning from their neighbours and building latrines'. – MMDA representative

118. As discussed in KEQ 24, MMDAs also felt that this reduction in open defecation has resulted in noticeable improvements in health across their districts. However, MMDAs highlighted that the process of implementing the CLTS approach in small towns has been more difficult than anticipated due to the lower prior awareness of CLTS in small towns, and to ensure that district assemblies meet their ODF targets, additional capacity building activities and man-power should be provided to MMDAs to reach ODF targets. Use of the ODF verification protocol was also considered challenging.

35 Accelerated programme proposal



‘Due to ODF, when it rains the communities no longer smell and they also don’t spend a lot of money going to the hospital anymore’. – MMDA representative

119. Programme implementers developed and disseminated a manual to implement the STSM. This manual was accompanied by training activities on the CLTS approach, as well as follow-up mentoring and M&E activities to ensure operational management continued and that as new staff join MMDAs, they are trained on the CLTS methodology from their peers. Stakeholders interviewed for this evaluation were satisfied with the training activities provided under the programme. MMDAs also highlighted the importance of the CSO-supported DRPs providing capacity-building support directly to districts; they focussed on programme management and monitoring, including training of field facilitators, natural leaders, and community technical volunteers. However, some DRPs highlighted that it was not easy to convince districts of DRP role and that they were there to support the district in meeting their ODF targets; rather, MMDAs perceived them as a ‘police’ sent to see how they utilise programme funds. They also felt that some district management teams were disengaged with the CLTS activities.

120. Nevertheless, in interviews, all stakeholders consulted seem motivated to eliminate OD, not only to achieve ODF status at community and district level but also nationally. Some national government stakeholders emphasised the support provided by the programme to strengthen the Sanitation League Table, which is prepared at the regional level to show the status of ODF in each district. Several MMDAs also mentioned the Sanitation League Table as an important source to measure how they are doing compared to other MMDAs.

‘The table put peer pressure on regions to support the ODF goal by providing support to their respective districts, so they are not at the bottom of the sanitation league table’. – National government representative

121. Another international agency implementing WASH programmes in Ghana highlighted that getting communities to reach ODF status is not an easy task; regional and district authorities do not only need to be motivated but have the capacity to provide WASH services to their population while also understand how to verify and certify their communities as ODF. The programme therefore adopted an incremental approach to facilitating CLTS, achieving ODF status at community level and building up to district level. Many stakeholders consulted considered that UNICEF is ‘the front runner’ in building the district-level capacity to provide WASH services using the CLTS approach, and that strengthening the government capacity to use the CLTS approach is the first step to achieve ODF status, but it is important to keep government stakeholders motivated to reach ODF status and using the ODF verification protocol, which was developed under the Enhanced programme. According to another international agency implementing WASH programmes in Ghana, UNICEF has contributed to all these three factors effectively by developing the ODF verification protocol and maintaining the momentum with tools such as the Sanitation League Table. Certainly, compared to other organisations, UNICEF is leading in ODF certification (as shown in the table below).



Table 7: ODF communities in Ghana by implementing organisation

Partner 2015 ODF May 2019 ODF USAID and partners (Global communities, RING and SPRING projects) - 1,267 UNICEF and partners 151 2,170 CRS - 21 SNV - 237 WaterAid - 9 World Vision - 228 Plan Ghana 19 34 World Bank / CWSA - 156 DA - 3 Prewater - 3 Care International - 17 GoG Pilot - 4 Total 170 4,149

Source: UNICEF

KEQ 14: How does access to WASH facilities in schools in rural communities compare with schools in small towns? And what impact does it have on ODF status?

122. The Accelerated programme’s provision of WASH facilities in schools was concentrated in small towns. According to UNICEF data, the Accelerated project constructed sanitation facilities in 59 schools in small town and four rural communities.

Work to improve water access was completed in 93 small town schools by August 2019, providing children with access to water from either boreholes with handpumps or piped water. Attempts to improve access to water had reportedly been unsuccessful in 27 schools where this was attempted. Work to improve water access also commenced in three schools in rural communities. All three of the schools in rural communities were given access to water through boreholes with hand pumps. HWWS facilities were constructed in 14 small town schools and in five rural community schools. The number of individual HWWS facilities completed ranged from two to 11 in schools in small towns (median – 3.5 facilities) and two to four in schools in rural communities (median - 2 facilities).

123. UNCIEF data shows that none of the schools in which handover of sanitation facilities has been completed following construction are located in communities that have since been verified as ODF. Further, in many schools, WASH facilities have only been constructed in the last year. Both factors limit the evaluation’s ability to compare the success of the programme in rural communities and small towns.

124. The programme ToC assumes that availability of WASH facilities in schools will support a reduction in OD in the community as children serve as positive ambassadors for hygiene and sanitation behaviours and cascade learnings regarding how to construct WASH facilities (specifically tippy taps, which students receive training to construct.) This assumption was challenged by evidence gathered in focus groups, however. As discussed in KEQ 4, some students provided examples of successfully changing behaviour of adults in the community, but students also faced challenges speaking to wider community members and felt that they needed greater backing from teachers or community leaders. Further, as discussed in KEQ 8 and KEQ 10, location of schools in non-ODF communities also appears to be having a detrimental effect



on the facilities themselves – and hence the benefits received directly by students from the facilities – as communities are using and damaging the facilities. These findings suggest that the schools programme may be more effective when complemented by CLTS and community-level hygiene promotion activities and where OD is already reduced.

125. The impact that improved access to WASH facilities in schools had on the achievement of ODF status, and any differences in effects between small towns and rural communities, cannot be quantified from the available data; further experimental or quasi-experimental research would be needed in order to robustly determine the contribution of school WASH facilities to reducing the overall OD rate within communities.

Preliminary conclusions – EFFECTIVENESS – DISTRICT LEVEL WASH ACCESS:

KEQ 13: The programme did not achieve its targets for ODF verified small towns or districts (Paragraphs 115-116). The programme developed an approach and supporting materials to implement the CLTS approach in small towns and built capacity at district levels to implement the RSMS and STSM through provision of training to district staff, as well as direct support from DRPs (Paragraph 119). This has motivated stakeholders to achieve their ODF targets, although DRPs felt that there are still some challenges at the district level in terms of capacity and buy-in for the CLTS approach (Paragraph 120).

KEQ 14: None of the schools in which handover of sanitation facilities has been completed following construction are in a community that has since been verified as ODF (Paragraph 123). While students can serve as positive ambassadors of hygiene and sanitation behaviours in their communities, students also face barriers in engaging with the community around sanitation, and OD also seems to be linked to misuse and vandalism of school WASH facilities (Paragraph 124).

1.2.4 Effectiveness - Governance and knowledge management

KEQ 15: What plans exist at the regional level regional level for WASH implementation in the 5 regions?

126. Although the programme did not have any set targets for regional-level staff training or plan development, one of the intended outcomes of the programme is to create an enabling environment for sustainable WASH services by strengthening the capacity for effective leadership, coordination and facilitation of WASH service delivery in accordance with the RSMS at national, regional and district levels. This included capacity building to deliver CLTS activities as well as to enhance WASH monitoring, evaluation and reporting. The RSMS is the core national plan for implementing CLTS; it was developed with aim of eliminating the practice of open defecation and promoting the use of improved latrines. Under the Accelerated programme, this adapted for small towns through creation of the STSM.

127. The programme provided technical support to the to the Regional Interagency Coordinating Committee on Sanitation (RICCS) in the Northern region, with services being provided by a regional consultant and a national consultant under a co-funding arrangement with the Enhanced programme grant. This technical support made it possible for regional WASH institutions such as the EHSD, Department for Community Development (DCD) and National Board for Small Scale Industries to effectively roll out the STSM. This was confirmed by MMDAs. MMDAs highlighted the good collaboration and support being provided by the regional teams.

128. National authorities also emphasised how the programme contributed to bringing WASH into the National Development Planning Commission (NDPC) plan and WASH guidelines to the MMDAs. As a result of this, MMDAs are obliged to contribute to the national WASH plans, resulting in the Ministry of Finance including a budget for WASH in their national planning. National government stakeholders commented that action plans are in active use and that receipt of



funds to implement WASH activities is now contingent on submission of their environmental management plans and when results are achieved, as described further below under KEQ 20. Nevertheless, there is no ODF plan at national level which, according to stakeholders, hindered the programme’s promotion at regional and district level to develop ODF plans

129. As part of the national plans to implement WASH in schools, the National Costed Strategy for WASH in schools was developed under the programme in collaboration with the Ghana Education Service. This strategy was developed following the existing WASH in schools’ national implementation model. However, the National Costed Strategy for WASH in schools has not yet been adopted by Ghana Education Service. Moreover, as highlighted by UNICEF staff, the cost included in the strategy still need to scrutinise by WASH sector stakeholders.

130. At the national level, numerous agencies are involved in planning of WASH activities, including the EHSD, DCD and CWSA, who oversee all WASH interventions at national, regional and district levels. According to national government representatives, the coordination of partners involved in the programme delivery was a difficult task for which resources and time required were not anticipated to the full extent at the programme outset, but which became visible during programme implementation. The coordination required a strong strategy including the different roles and responsibilities of each stakeholder. Moreover, their responsibilities within the programme needed to be aligned with their overall mandate; this led to conflict as people challenged each other over where each other’s mandates lie. Overall, stakeholders believe UNICEF coordinated stakeholders well, but coordination is a key risk to the programme (and has been added as such in the revised ToC).

KEQ 16: What systems for accountability exist at district levels?

131. As part of the RSMS roll out, the programme invested heavily in the building of capacity at district level, which included training on CLTS implementation, planning, and M&E. For the Accelerated programme, all these activities aimed to ensure the roll out of the CLTS approach not only in rural communities but small towns as well; district level staff were trained on how to implement the STSM using a manual developed under this programme. Moreover, the programme contributed to developing the artisan-led business model and strengthening private sector engagement.

132. The programme had numerous targets for accountability at the district level, all of which were achieved or even exceeded. These included: 14 districts adopting and using accountability mechanisms, and having accountability systems in place (target: 10); 14 districts implementing a framework for PPP in sanitation (target: 10); 14 districts implementing demand-responsive approaches (e.g. having funds set aside for sanitation) (target: 10); 113 coordination meetings held (target: 36); 10 review meetings held (target: 10). A few MMDAs mentioned that they attend coordination meetings on a quarterly basis; in which they share feedback on how the progress made and challenges experienced. One MMDA highlighted the use of weekly reports to track progress made on the implementation of CLTS activities at the quarterly meetings.

133. Further, the programme supported districts to establish various planning policies; these included 10 Memoranda of Understanding signed (target: 10), 14 districts with district-level frameworks for private sector engagement in place (target: 10), and 14 districts with results-based plans for sanitation (target: 10 – discussed further in KEQ 20). However, the programme did not meet its target for districts with frameworks in place for regulating private sector involvement in sanitation, with no districts having these yet (target: 10).

134. Stakeholders, including MMDAs, emphasised the impact that the Accelerated programme has had in terms of galvanising and building the capacity of district authorities to act around sanitation. According to MMDAs, districts are more



accountable and readier to provide WASH services because of the extensive trainings they have received. MMDAs noted the monitoring systems (detailed under KEQ 20) as being helpful to this.

‘We have received extensive training through the programme which has allow us to train other districts members and implement the community-led activities’. – MMDA representative

135. All 14 participating MMDAs have submitted proposals to participate in the programme to the Northern Regional Coordinating Council, signing and rolling out Memoranda of Understanding on responsibilities of all parties and developing district-level ODF plans. Most of the MMDA representatives from the Northern region who were interviewed mentioned they have District Environmental Sanitation Strategies and Action Plans in place which include detailed work plans and targets. They also highlighted that the training received on how to develop their plans was essential to understand the processes and tools to inform UNICEF and national and regional authorities on the WASH activities carried out at district level.

136. CSOs and MMDAs stated that there have been improvements related to the reporting, monitoring and coordination WASH activities at district and national levels. In particular, they noted improvements in the level of involvement of District Inter-Agency Coordinating Committees on Sanitation (DICCS). Before the DRPs started supporting the districts, DICCS were not very functional, but at programme end, DRPs reported that participants are active and provide inputs to ensure WASH service delivery. Another improvement is the increased level of commitment of field facilitators, whose mandate includes contributing actively to communities becoming ODF.

137. The programme’s provision of regional consultant support and DRPs were seen as a fundamental component to ensuring that MMDAs achieve results. DRPs encountered several challenges working with district assemblies that have not yet been fully addressed. One challenge was ensuring that district management teams release funds for field facilitators and other district staff involved in the WASH service delivery, which affected their commitment to reaching ODF status. According to CSOs, in districts where DPRs have been placed, field facilitators receive funds on time, improving motivation. Secondly, DRPs felt that the support of district management teams to field facilitators is not adequate and their interactions with communities are very limited. Field facilitators are involved in many activities (e.g. social norms, construction of latrines facilities, ODF achievements) for which district-level support is not always available.

138. Similarly, CSOs highlighted that they struggled to get district management boards to support the WASH programme activities. It takes time to get district management level staff on board and sustain their commitment to their programme and to RSMS overall. CSOs said that district management staff might promise to build communal toilets or provide subsidies to obtain votes, going against the CLTS approach, which required further awareness building activities at district level.

139. Thus, while governance and accountability structures have been put in place through the Enhanced programme’s support, MMDAs are not yet fully resourced to implement CLTS activities. However, it is important to note that this is the first programme that strengthen existing government structures to implement the CLTS approach at district level, which means that it tested if and how the RSMS could be (if at all) implemented. Moreover, district management staff might need further sensitisation on the importance of CLTS activities and the need to release funds to execute these on time – either through the DRPs or other channels.



KEQ 17: What monitoring systems exist for tracking WASH results at the regional and district level?

140. The programme contributed to strengthening existing government structures by supporting existing M&E systems such as BaSIS and helping to establish new ones like the RBF. While this success largely built on the work of the Enhanced programme, the Accelerated programme brought new districts into the systems developed as part of the Enhanced programme.

141. Targets relating to M&E systems strengthening were exceeded: 13 districts have functional information systems in place (target: 10), all target districts have data on ODF communities and disaggregated information on populations (target: 10) and a database is in place to capture sex-disaggregated data for WASH and health; 67 male and 67 female were trained on budgets and results monitoring (target: 20 male and 20 female staff), and 14 districts have monitoring formats developed and in use (target: 10); and as noted in KEQ 19, 14 districts have results-based plans for sanitation.

142. A key achievement of the programme’s monitoring systems activities was roll out of the RBF, which took place in 2018. This links the demand-responsive approach to achievement of results, with participating districts only qualifying for further funding based on achievement of at least 50% of planned results. This required the engagement and oversight of government agencies beyond those delivering WASH services including the Ministry of Finance, who is in charge of overseeing the implementation of the RBF. For the RBF to work, all national, regional and district level authorities should develop environmental management plans including a budget to deliver WASH activities, thus this initiative involved intervention at multiple levels. National government stakeholders commented that the RBF and action plans are in active use and that receipt of funds to implement WASH activities is now contingent on submission of their environmental management plans and when results are achieved. Nonetheless, UNICEF acknowledged that while the system was designed to deal with delays to provision of funds, there have been some continued issues when using the system. Even though national stakeholders mentioned the RBF is in active use, MMDAs are still getting used to working with RBF. Among Accelerated programme MMDAs, the RBF is perceived as a core strategy for WASH service implementation, but many MMDAs would like the system to be strengthened and clarity on the role of districts and how to use it.

143. Government stakeholders indicated that monitoring and reporting on WASH at district level have improved as a result of training provided by the programme, enabling more effective monitoring at national level. According to some stakeholders, this improvement is particularly linked to the strengthening of the BaSIS M&E system. Several MMDAs highlighted the training received on BaSIS and felt that it is a good tool to monitor and report on progress related to WASH service delivery. Between April 2018 and March 2019, there was a more than 300% rise in the number of communities recorded on BaSIS, which both the Enhanced and Accelerated programmes contributed to. As of March 2019, the system had data on over 131,893 households and more than 121,017 toilets. The system also had complete data of over 300 ODF communities recorded within the year and varied data on over 2,000 CLTS communities and includes gender-disaggregated data, reflecting the effort of district authorities to use BaSIS.

144. However, an ongoing concern is the quality of the data, and the evaluation team identified several inconsistencies in the database which meant that it could not be used as a secondary data source for the evaluation. For instance, figures that have been inputted for ‘population living in ODF communities’ in the District League Table have been used for ‘number of toilets constructed’ in the Regional ODF achievement reports. Similarly, there are discrepancies between the figures inputted for the ‘total population living in ODF communities’, ‘total number of toilets constructed’, and ‘population using



newly constructed toilets’ across the two tables.36 UNICEF envisages to improve BaSIS data quality and functionality in future programmes. The inconsistencies in BaSIS are understandable considering that it has been operational since January 2019. Moreover, UNICEF staff noted that BaSIS is being upgraded to facilitate the data capture and updates by field facilitators.

145. The programme also supported the inclusion of WASH indicators in the Education Management Information System (EMIS) and District Health Information Management System (DHIMS) for improved monitoring nationally. For schools, this includes monitoring such indicators as access to gender segregated toilets and changing rooms for MHM purposes. It seems that the indicators in EMIS have been used to the develop the National Costed Strategy for WinS, which suggests that the programme has contributed to improving the coherence of WASH monitoring; aligning strategies with existing monitoring systems.

146. Government at the national level, including through the National Technical Working Committee, is primarily involved in reviewing reports received from lower levels of government and reviewing resource utilisation. In addition, national government stakeholders are also involved in collection of monitoring information; for example, the Ministry of Education undertakes monitoring missions. However, one national government stakeholder noted that programme trainings focussed on regional and district level staff only and felt that national government’s M&E staff should also be included in training and could attend international capacity-building events.

KEQ 18: Have the knowledge management components of the programme improved understandings?

147. The Accelerated programme targeted strengthening of existing learning platforms, the KPM database shows that two learning platform were strengthened. However, progress reports state that the programme has mainly contributed to improve the national learning platform through a series of national workshops and meetings in Accra and Tamale, and consultation with implementing partners. The programme aimed to support the EHSD to strengthen sector capacity for documentation through the Sanitation Knowledge Management Initiative (SKMI), a knowledge generation and sharing platform between government, academic and research institutions. This was intended to include support to strengthen the management of the GoG website for sanitation and the publication of national ODF league tables as well as a quarterly newsletter on key sanitation issues and initiatives. The programme also aimed to improve coordination and functioning of the DICCS and RICCS.

148. As part of the Enhanced programme a partnership through IRC and government, engaging universities (Kwame Nkrumah University of Science and Technology (KNUST) and Emory University) and bodies of technical expertise (IRC and TREND), was formed to work on the SKMI. The partnership continued working on strengthening the WASH knowledge management under the Accelerated programme. However, according to national level stakeholders, the CSOs involved in establishing the needed knowledge management platform and tools were not very coordinated and did not manage to encourage WASH stakeholders to use and access information on the available knowledge management platforms. MMDAs consulted were not aware of the SKMI or activities under it. Although awareness of the SKMI was low among interviewees, stakeholders did highlight the role of the programme in publishing the ODF League Tables, which were considered a useful tool (as discussed in KEQ 16).

149. Stakeholders’ views on how Enhanced and Accelerated programmes have contributed to WASH knowledge management is similar. Government stakeholders highlighted the information and knowledge shared among stakeholders

36 See East Gonja for an example of these discrepancies.



involved in the implementation of the programmes. However, this seems to have taken place primarily through the structures that existed prior to the programme (and which were strengthened by the programme), and the Accelerated programme’s targets for strengthening existing structures reflects a learning from the Enhanced programme in this regard. However, MMDAs explained that they were processing information and disseminating WASH knowledge, particularly through participating at RICCS, DICCS, and in field meetings. These platforms existed prior to the programme (to coordinate not only WASH service delivery but other activities) but have been strengthened through the programme. As part of the programme, government stakeholders were encouraged to attend and share experiences at RICCS and DICCS meetings, thus reinforcing these coordinating bodies. As a result, these were described as functional bodies in which ‘stakeholders involved are active and provide good contributions to ensure WASH service delivery’. Similarly, the National Level Learning Alliance platform was seen as effective at the national level. However, the sustainability of these platforms is unclear; other agencies believe that as soon as funding is not available these bodies will weaken again, while the positive feedback from stakeholders involved regarding how attending these meetings has helped them to share experiences and coordinate activities suggests that, at least in targeted districts, DICCS will remain active.

150. The CLTS Stock-Taking Forums (conferences where WASH stakeholders share knowledge around WASH and discuss WASH sector priorities) were also considered a key event in which academia and practitioners actively participate. Stakeholders were more positive about the role of CLTS Stock-Taking Forums for knowledge coordination and sharing at national level. These forums have been ‘a game changer’ as various stakeholders including academia and practitioners actively participate in these. The 7th CLTS Stock-Taking Forum was held in July 2019 in Kumasi.

151. Many stakeholders also highlighted the role of the Coalition of NGOs in Water and Sanitation (CONIWAS) as good disseminator of WASH knowledge outside of the programme. According to UNICEF staff, CONIWAS was supported through the Enhanced and Accelerated programmes for disseminating sectoral strategies, providing information and tools to the members and raising issues at the national level (e.g. advocacy with Parliamentary select committee on WASH) on needs and gaps in the sector.

152. Overall, as a result of the programme’s intervention, MMDAs are now more knowledgeable around WASH service delivery than they were before and have learned from each other’s experiences and best practice. However, some stakeholders believed that more effort should be put into acting on the lessons learnt through programme interventions and the sharing of experiences with sector stakeholders to ensure faster collective progress towards national targets. Stakeholders envisaged an ongoing role for existing platforms to meet this need.

KEQ 19: Are appropriate systems for sanitation law enforcement established?

153. According to the Accelerated programme proposal, the programme aimed to support the capacity of the regional and district environmental health officers to develop and enforce sanitation by-laws in accordance with the Expanded Sanitary Inspection and Compliance Enforcement (ESICOME) manual. Furthermore, the programme was supposed to develop environmental management plans and guidelines based on requirements of the Strategic Environmental Assessment of the RSMS by to ensure that the programme has no negative impacts on the environment. However, the KPM does not have concrete targets related to the enforcement of sanitation by-laws.

154. A few interviewees provided insights as to how the programme has contributed to strengthening the enforcement mechanisms in Ghana. As part of the CLTS approach, the programme developed community rules and regulations to enforce sanitation and hygiene. Further, programme implementers worked with MMDAs to intensify the roll out of the ESICOME component of the RSMS and the enforcement of sanitation by-laws. UNICEF staff mentioned that, when carrying



out the ODF verification protocol, communities are using community by-laws. For example, two MMDAs in the Northern region mentioned their strategies for enforcement. In one district, the MMDA gazettes the community by-laws and punishes those who practice open defecation; this includes charging a penalty of 30 GHS for persons practicing open defecation.

‘Some communities have used the recently developed rules and regulation and written out in their books, this is certainly a start to enforcing sanitation at community level’. – National government representative

155. However, the enforcement of the sanitation policies and by-laws is still limited in Ghana, particularly sanitation law enforcement at district level, even if it is considered to be crucial to achieve national sanitation targets. Interviewees also noted that future WASH programmes in Ghana should support government unblocking the current non-enforcement of sanitation law and build on existing policies and by-laws including penalisations for defecating in the open. However, stakeholders cautioned that awareness and knowledge around WASH services and practices should be firmly reached, to enable different levels of government to enforce sanitation law policies and by-laws adequately.

Preliminary conclusions – EFFECTIVENESS – GOVERNANCE AND KNOWLEDGE MANAGEMENT:

KEQ 15: The programme has invested heavily in building the capacity of government at all levels to ensure effective leadership, coordination and facilitation of WASH service delivery (Paragraphs 126-127). National government stakeholders have action plans for WASH implementation. However, only regional authorities and MMDAs have ODF plans. Stakeholders highlighted the need to have a national ODF plan (Paragraph 128)

KEQ 16: The programme invested heavily in the building of capacity at district level, providing a model for systems strengthening at the decentralised level (Paragraph 131). The trainings provided to districts have made them more accountable and readier to provide WASH services and resulted in tangible outputs such as Memoranda of Understanding and ODF plans (Paragraph 135). However, challenges in implementing the CLTS approach at district level persist, including support from district management, timely release of funds to field facilitators, and use of subsidies and funding of community latrines for political reasons (Paragraphs 137-139).

KEQ 17: The programme built on the success of the Enhanced programme in strengthening national monitoring systems by rolling these out to the Accelerated programme districts and building capacity, particularly at district level, to implement these systems (Paragraph 140). The roll out of the RBF took place in 2018 and involved government at all levels; the RBF and action plans are in active use and receipt of funds to implement WASH activities is now contingent on submission of districts’ environmental management plans and when results are achieved, although there are still some challenges in using this new system (Paragraph 142). Moreover, government stakeholders reported that monitoring and reporting on WASH at district level has improved, allowing for more effective monitoring at national level (Paragraph 143). This improvement is also linked to the strengthening of the BaSIS, which district authorities and other organisations are using, although there are still some issues with the quality of the data (Paragraph 144). Monitoring at national level has been supported by the inclusion of WASH indicators EMIS and DHIMS and the development of National Costed Strategy for WinS using indicators from EMIS (Paragraph 145).

KEQ 18: The programme knowledge management and sharing activities built to an extent on the lessons learnt in the implementation of the SKMI under the Enhanced programme, focussing on getting WASH stakeholders to use and access information on the available knowledge management platforms like the GoG website and WASH learning alliance



platforms instead of creating new platforms (Paragraphs 147-149). Stakeholders at national and regional levels were more aware of the WASH learning alliance platforms and the advocacy work of CONIWAS. They also were coordinated and shared WASH knowledge at least on yearly basis at the CLTS Stock-taking forums, although they felt that knowledge-sharing could be further strengthened (Paragraphs 150 and 151).

KEQ 19: The Accelerated programme contributed somewhat to improving WASH sanitation law enforcement in Ghana by working on community rules and regulations to enforce sanitation and hygiene as part of the CLTS approach and ODF verification protocol. This includes rolling out the ESICOME component of the RSMS (Paragraph 154)

1.2.5 Effectiveness – in general

KEQ 20: In which cases were project targets not met, and what could have been done differently to enable them to have been reached? How should future programming be designed in order to overcome experienced challenges?

156. This section summarises the overall effectiveness of the Accelerated programme in achieving its intended targets, drawing on the KPM database as well as evidence presented earlier in this report (excluding targets related to health outcomes, which are discussed in KEQ 24). Annex 4 includes information from the KPM database on programme targets and achievement.

157. As discussed in KEQ 3 and KEQ 13, the programme did not achieve its targets for improved community-level sanitation under ‘increased equitable access to suitable household and institutional sanitation facilities’. The programme reached less than half of its intended target for household latrines constructed (17,488 vs a target of 50,000), nor did it achieve its targets for additional people living in households that use improved sanitation (204,655 vs a target of 500,000), ODF districts (0 vs a target of 5), or ODF communities (411 rural communities and 34 small towns vs a target of 80 small towns). In this context, these targets were deemed too ambitious by stakeholders, and particularly the targeted number of additional people living in households that use improved sanitation was considered to be due to an overestimation of the target population.

158. The survey evidence, discussed in KEQ 3, also shows that OD has declined significantly in the targeted communities and that most survey participants who use a toilet also reported that their toilets were built after the programme commenced. Evidence in KEQ 3 also shows that those who still practice OD typically do not do so by choice but due to a lack of access to public or private toilets and that cost is a barrier. Greater provision of financial support and awareness raising could therefore help overcome these challenges.

159. In schools and health centres, the programme achieved most of its targets for construction of WASH facilities, with the exception of the target for provision of water facilities (162 vs a target of 180). As explained in KEQ 6, borehole drilling had high failure rates in the Northern region; this posed a challenge for provision of water supply to institutions. Some stakeholders including government and CSOs, noted that there were issues with inconsistent quality in feasibility studies for borehole drilling; they highlighted that in future programmes, there should be a better control on feasibility studies. The challenge of drilling in the Northern region is an issue well-known by the WASH sector, and there are not many contractors that work in the Northern region as a result. Thus, UNICEF staff suggested other future programmes should investigate alternative approaches to harvesting water and to minimise loss incurred through dry boreholes. Further, it should be noted



that the evaluation team found qualitative evidence that maintenance of WASH facilities is poor, especially due to use of facilities by community members and lack of support from authorities to maintain them (see KEQs 8, 10 and 12). To address these challenges, support from GHS and MMDAs to maintain facilities is needed, as well as an increase in community access to household latrines through a concurrent implementation of the CLTS approach.

160. Finally, the programme also aimed to increase the capacity to deliver the STSM and RSMS, and again, the majority of targets under ‘increased capacity at the community, district and regional levels’ have been achieved, with the exception of strengthening lessons learning platforms (2 vs a target of 6) and putting in place district frameworks for regulating private sector investment in sanitation (0 vs a target of 10). As the challenges encountered in achieving these two targets relate primarily to timing and the efficiency of delivery rather than effectiveness, these are discussed in KEQ 25. Nonetheless, stakeholders consulted considered the programme to be broadly effective in building the capacity of government to ensure effective leadership, coordination and facilitation of WASH service delivery as well as supporting government to set M&E and reporting frameworks. However, some stakeholders pointed to some persisting limitations in the capacity of districts to deliver WASH services under the CLTS approach and to use the ODF verification protocol (see section KEQ 13). Several interviewees highlighted the following factors as critical to sustaining the programme’s results, which should be addressed by future programmes:

1. Continued investment is required to continue to build capacity and accountability, making use of the existing structures. DRPs are an important resource for continuing to build capacity, but they should ensure they act in a supportive function rather than a leading one.

2. One of the most important legacies of the programme is all the information generated to provide WASH services and facilities across the targeted communities and, these should be disseminated more widely across other communities not targeted by the programme.

3. The political commitment and ownership to deliver effective WASH services exists and needs to continue so all the efforts put into delivering this programme and other WASH programmes result in improved health across Ghana. This is especially important at the district level, where commitment to CLTS rather than approaches such as blanket subsidies, needs to continue to be encouraged.

KEQ 21: To what extent have the projects effectively addressed the specific needs of women and girls?

161. The Accelerated programme aimed to address inequities in WASH by promoting gender mainstreaming in the WASH sector and embedding gender and other equity considerations in its programming. The inclusion of gender quality objectives in the programme was also highlighted by duty-bearers consulted as part of the interview process, who noted that the programme had a deliberate focus to ensure women and girls have access to gender-friendly sanitation and hygiene services and facilities in schools and health centres. The proposal also includes some analysis of equity considerations, such as that ‘the lack of sanitation and hygiene has more negative impacts on girls and women, girls in basic schools and women particularly pregnant women and women with children under the age of five’ and also states that ‘the gender mainstreaming guidelines will be implemented to achieve results on empowerment of women and girls to participate in the WASH programme’. The programme aimed to achieve its gender-related programme goals by improving the availability of gender- and disability-friendly WASH facilities in schools and health centres, improving awareness of MHM, and strengthening gender-responsive WASH programming through increasing the capacity of duty-bearers to address gender inequity, by mainstreaming gender issues in operational instruments at the district level.



162. Evidence of the programme’s effort to mainstream gender and other equity dimensions is also found in the inclusion of some specific KPM targets for ‘gender-sensitive, disability and environmentally-friendly sanitation facilities’ in schools and health centres, ‘districts with gender disaggregated data and evidence available’ and ‘database in place to capture sector-specific sex disaggregated data for WASH and health’. KEQ 9 found that gender-friendly toilets had been constructed, but that changing rooms were not provided for girls in all schools. As shown in Annex 4 and discussed in KEQ 18, targets related to implementing monitoring systems were achieved. Women and girls are also mentioned in KPM targets specifically in regards to practicing HWWS and receiving training on MHM in schools, and inclusion of women-managed businesses in private sector engagement activities. As shown in Annex 4, the target for reaching schoolgirls with MHM training was significantly exceeded (discussed further in KEQ 11), although the KPM database does not include gender-disaggregated results regarding private sector engagement. Further, the only logframe target that references disability relates to the facilities in schools and health centres.

163. The results of programme activities targeting women’s needs is mixed. As discussed in KEQ 3, while almost three quarters of survey participants (74%) in small towns and two thirds (64%) in rural communities say that their toilets are segregated (meaning they are in a private space where nobody else can see who is inside), participants’ responses may have been biased. Upon observation, just 24% of small-town toilet facilities and 10% of rural facilities were found to be segregated. In terms of privacy, 51% of small-town participants said that their toilets had locks, which aligns with the results of the observation (where 61% were found to be lockable and private). In contrast, while 37% of participants in rural communities say their toilets have locks, just 17% of rural toilet facilities were found to be lockable and private at the time of the survey when observed. This means that more women than in the baseline, particularly in small towns, now have a private place to wash themselves when needed during their last menstrual period. It should be noted, however, that inclusion of locks is a household-level decision and there is not a standard or requirement to include locks in household latrines.

164. When it comes to MHM, significant improvements have been made in rural communities, but these are not yet universal, as discussed in KEQ 8. Around three quarters of female endline survey participants in both small towns and rural communities (75% and 72%) report using a sanitary pad to manage their menstrual cycle when it occurs. Among women in small towns, this represents no significant change from baseline when 73% used a sanitary pad, however this represents a marked improvement in rural communities where only 37% of women used a sanitary pad at baseline.

165. The endline survey also found that women and girls remain the primary collectors of drinking water in both small towns and rural communities. Given the large numbers of households travelling long distances to collect water and the gendered division of WASH duties – this may be having a detrimental impact on women’s involvement in social and economic life, as discussed in KEQ 6. However, the programme did not address gendered issues around water access and management given its limited focus on water activities in general.

166. The endline household survey has been used to assess the programme’s effectiveness in achieving gender-equitable outcomes by disaggregating data by gender. The survey found the following:

There are no significant differences between the proportion of male and female participants who practice open defecation.

Female participants are notably less likely than male participants to agree that ‘hands that look clean can have germs on them (49% either strongly agree or tend to agree compared with 66% on males). Females are also less likely than males to agree that ‘washing hands with soap kills more germs than water alone (83% compared with 94%).



Women are more likely than men to say they wash their hands after disposing of a child’s stool (45% versus 16% respectively), after cleaning a baby (49% versus 16%, respectively) and before food preparation (45% versus 19%). This likely reflects the gendered division of childcare and household responsibilities.

Women are responsible for collecting water for family consumption in over three quarters of surveyed households (92%). Women are also overwhelmingly responsibly for storing and managing water (96%) and cleaning the household toilet (76%). Again, however, it should be noted that water access issues in general were not a focus of the programme.

167. Among those participants who say that someone with a disability lives in their household, half in small towns (50%) and just less than half in rural communities (45%) say that the toilet is disability-friendly.37 While care should be taken when interpreting this result due to the low base size, observations of private household and shared-compound toilet facilities corroborate the finding, whereby only 26% and 9% (respectively) were found to be disability friendly, and this raises equity concerns that merit further investigation.

168. When it comes to WASH in schools, as discussed in KEQs 9 and 10, focus group discussions highlighted that in all three schools studied, there are separate facilities for boys and girls. However, schools are experiencing issues around outsider use, which is damaging the facilities and making it difficult for the children to maintain them. MHM programmes were also active, but girls across all focus groups were largely unaware about how to calculate their monthly cycles, despite the training they receive from the MHM programmes. This can result in girls staining their clothes, leading to embarrassment and, subsequently, poor school attendance. Further, interviewees and focus group participants highlighted that girls are experiencing difficulties because of the limited affordability of pads and other material to use for their personal menstrual hygiene, thus some might resort to low-quality and unsafe materials for their own reproductive health. In some schools, girls are being provided with sanitary materials for managing their menstruation and in others not. UNICEF staff suggested that a comprehensive assessment on MHM materials should be carried out to understand what materials are available and how much these are being used. Finally, while stakeholders felt that the programme has been particularly effective in changing boys’ attitudes around MHM, according to focus groups findings many boys still laugh at girls when they are unable to move because of menstrual pain or when their pants get stained.

169. In terms of participation of women and girls in programme delivery, evidence shows that progress is mixed. CSOs highlighted the limited role of women in SanMark activities; the programme managed to train only a limited number of female artisans although data on women’s engagement in private sector activities is not included in the KPM database. This reflects a lack of prior technical skills, linked to preconceived ideas around gender roles and stereotypes which will take time to change, and CSOs made an effort to encourage women to obtain these skills. However, the programme has managed to support many women-led VSLAs which not only supports construction of latrines but also provides capital for women’s businesses, according to stakeholders.

170. The Accelerated programme also built on policy and guidance documents that mainstreamed gender which were developed under the Enhanced programme such as the WASH sector gender mainstreaming guidelines, and this was a specific focus of the programme, as set out in its proposal. Apart from building on the work carried out under Enhanced programme, the Accelerated programme aimed to improve technical expertise on gender equality to support the implementation of the gender mainstreaming guidelines and toolkits, such as contributing to development of low-cost,

37 Small base sizes – n=21 in small towns; n=11 in rural communities.



gender-friendly toilet design as part of its role in developing the WinS National Costed Strategy. However, the programme does not include concrete target for this activity, and the National Costed Strategy not include specific gender issues, needs, analysis and/or impact. Further, the strategy still requires adoption by Ghana Education Service, but inclusion of WASH targets in the Education Sector Plan is a step forward in changing the institutional context. The evaluation team carried out a gender-assessment of the Education Sector Medium-Term Development Plan 2018–2021 (ESMTDP) and the Education Strategic Plan 2018–2030 (ESP) which are well aligned to SDG4 but do not include a situational gender analysis. The ESMTDP only states low participation of females in science, technology, education and mathematics (STEM) subjects, and lower educational outcomes for girls as one of the key issues of the sector. However, the ESMTDP and the ESP do have targets on gender parity, for example: targets on boys and girls in Basic Education and Secondary education and participation of women in TVET (technical and vocational education and training).

171. Another result of the programme is the development of the first equity index score. This was developed by government stakeholders and consultants which covers both the time spent collecting water by women/girls versus men/boys, and the access to gender-disaggregated toilets in schools. According to the fourth progress report, there was challenge in obtaining nationwide data for schools, but the work on updating EMIS WASH indicators will enable it to be populated. However, there are some gaps in the monitoring system used by the programme. Since 2016, the data in the programme monitoring tool is gender-disaggregated except for sanitation data at the national level. According to UNICEF staff, further training on BaSIS should be carried out in future programmes to ensure that reliable gender-disaggregated data is available at national level.

KEQ 22: How effectively has the programme responded to learnings and recommendations during delivery, and to what extent have key learnings been disseminated more widely?

172. UNICEF displayed a commitment to adaptive learning from the early stages of the programme. This was demonstrated through the programme’s support of new and existing knowledge-sharing and M&E platforms, and documentation of lessons learned and recommendations for improvement which were published annually in the Accelerated programme progress reports, based on monitoring data received from fieldwork facilitators and feedback from implementing organisations. These sections allowed programme staff and implementers to review their activities and take corrective actions.

173. The evaluation team found several good examples of recommendations being made and associated corrective actions being taken in response to challenges faced. First, the hydrogeological conditions in the Northern region limited the programme activities that aim to provide access of water supply (in schools and health centres). For example, in the schools where sanitation was provided, these new facilities could not be reasonably supported by the inconsistent water supply, unless a mechanised system was nearby. To work around this challenge, UNICEF targeted schools with existing water supply systems, focusing on providing support on how to handle and manage the existing water supply to ensure it is not contaminated and the limited water is used most effectively.

174. Second, challenges surrounding latrine construction were consistently noted in all progress reports from the programme outset. They identified concerns that the prevalence of household wells in some communities made household latrines a potential source of groundwater pollution. In addition to risks of water pollution, the presence of public latrines in some communities also made the uptake for household latrine construction very slow. In response, the SanMark activities introduced suitable technologies to enable appropriate, simple low-cost lining materials to be used. Alongside greater access to more affordable materials, the concerns over groundwater pollution were alleviated by the presence of field facilitators emphasising the adherence of a minimum distance for situating latrines from water sources.



175. Third, the progress reports repeatedly recognised a broad sectoral preference for very high standard school sanitation facilities. The programme therefore increased advocacy on the importance of interim low-cost facilities to ensure that all school children have access to school sanitation facilities and on constructing facilities in communities with existing water systems. This response occurred in line with the WASH in Schools Costed Strategy and budget monitoring exercises to ensure that strategic decisions were made with resourcing considerations in mind.

176. Fourth, several the districts involved in the Accelerated programme had not previously partnered with UNICEF. As a result, they required additional time to clarify and implement a cohesive partnership and resolve collaboration-related issues including logistics for the transfer of funds and their management. While the challenge of building new relationships between districts and UNICEF was anticipated, the length of time that was required to train new district departments on specific WASH processes took longer than anticipated. There was specific reference in the 2016 progress report to the impact of the 2016 Election Campaign which weakened the engagement of the district level with UNICEF management as their efforts re-centred on politically sensitive issues rather than the WASH programme. Subsequently, CLTS progress was slowed. In response, UNICEF consistently engaged with district management officials on several platforms, carrying out training for the districts to optimise a smooth collaboration and minimise any negative impacts on the delivery of the results of the Accelerated programme. Additional support from financial consultants in relation to the RBF approach has helped to strengthen financial management and accountability within the districts. A stronger enabling environment was facilitated in light of the 2016 election with the appointment of a new Ministry of Sanitation and Water Sources which aimed to provide support for the attainment of sanitation goals in the districts. As a result, by December 2018, all MMDAs signed Memoranda of Understanding with UNICEF. Evidence also suggest that districts capacity has been strengthened the financial management and accountability within the district as well as the delivery of WASH services. Thus, the constant engagement and training provided by UNICEF has successfully ensured the implementation of the programme at district-level.

177. Fifth, the gender ratio in WASH trainings was found to be skewed more towards males than females. Through a partnership with CONIWAS, focus groups were held amongst women’s groups to shed more light on female reluctance to become more involved in construction related activities.

178. However, the evaluation team also found some examples of recommendations that were not implemented or issues that have not yet been resolved. First, despite the RSMS stipulating that blanket subsidies should not be provided for latrine construction, some political leaders (and other NGOs) continued to promise financial support to communities, which can negate the achievements of the CLTS approach and can negatively affect the prioritisation of the CLTS approach at the district-level. This was also highlighted by interviewees; many communities within the Accelerated targeted districts receive subsidies to build household toilets, which limited the impact of the support provided to build awareness and capacity to deliver CLTS activities. Progress reports recommended engaging political authorities at different levels to develop understanding and generate support for the CLTS approach; however, interviewees highlighted that further political engagement is still necessary.

179. Second, the construction of boreholes in the Northern region was not successful due to the groundwater conditions of the region but also due to low-quality feasibility studies carried out before starting the construction. According to stakeholders, CSWA follows the available protocol on hydrogeological investigations and can only provide contracts to businesses that follow the protocol and standards included in this. Nonetheless, issues persist.



KEQ 23: To what extent have health outcomes improved in the project areas?

180. Achievement of targets for improved health and well-being of children, women and men in the Northern region were not favourable. DHS data from 2014 shows that 16% of children under five in the Northern region had diarrhoea in the last 2 weeks. In 2018, the proportion had increased to 27%, according to MICS data. As the programme does not operate across all communities in the Northern region, it is important to note that other external factors will have contributed to the broad generally negative health outcomes reported in the secondary data, including the population increase in recent years.

181. To evaluate health outcomes in the target small towns and rural communities specifically, the evaluation team also asked relevant questions around diarrhoea in children under five in the household survey. According to the Accelerated programme baseline survey, 22% of household heads in small towns and 20% of those in rural communities said that children under five suffered no cases of diarrhoea in the last month. Endline survey data shows an improvement in the percentage of children who are free from diarrhoea in rural communities, where 33% of participants said their children has not suffered from diarrhoea. There was no statistically significant difference between baseline and endline in small towns.

182. A smaller proportion of households in both small towns (10%) and rural communities (13%) cited at least four diarrhoea cases a month for children under five at baseline.38 This is not surprising, as the baseline survey results also show relatively high rates of OD among children under five and indiscriminate excreta disposal practices. The endline household survey results illustrate a decrease in the percentage of children suffering from diarrhoea at least four times a month in small towns – with 0% of endline participants in small towns saying that their children suffered at least four cases per month, although this must be interpreted cautiously as the rate at baseline was already low and the change between baseline and endline is within the range at which change cannot be interpreted as significant. There was no statistically significant change between baseline and endline in the number of children suffering at least four cases per month in rural communities.

183. Across the overall endline survey sample, those who used an improved toilet were more likely than those who practiced OD to say their children had suffered no cases of diarrhoea (30% compared with 20% respectively); however, there was no difference in the percentage of respondents reporting that their children suffered at least four cases in a month.

184. According to a few MMDAs, the overall health of communities targeted by the programme has improved. National government representatives were also positive about the reduced number of sanitation-related diseases in the Northern region. They highlighted conversations with community members in which their satisfaction with overall improved health outcomes in their communities was discussed.

‘There is less sickness in the communities that have been ODF certified in the Northern region’. – MMDA representative

185. Stakeholders mentioned that improvements in the health of community members cannot be attributed to the Accelerated programme intervention alone but that it has contributed to these improvements. Other factors such as the increased government funding on health education, including food hygiene, and other WASH programmes operating in the regions need to be considered.

38 There was no significant difference between rural communities and small towns.



186. There are also several WASH programmes operating in the same districts as the Accelerated programme, which could potentially have influenced the health outcomes of the population targeted by the programme. Some national government stakeholders believe that WASH programmes active in Ghana do not overlap in terms of the support provided to the districts and to other WASH stakeholders. They highlighted that national agencies try to share programme support according to the needs of the population and existing support available. While this might be the case, CSOs said that other programmes are being implemented in some of the districts where the Accelerated programmes was operating. For example, a World Bank WASH programme provides support to districts in the Northern region financing household toilets, including roofing, through 70% subsidies. A full dataset of other WASH programmes active in Ghana was not available to the evaluation team.

KEQ 24: What were (if any) the unintended effects (positive and negative) produced by the Accelerated programme?

187. The programme had more reported positive unintended effects than negative unintended effects. Many stakeholders highlighted the following two unintended positive effects:

1. The programme contributed to bringing sanitation and hygiene to the top of the Ghanaian political agenda. According to national level stakeholders, UNICEF and other implementing partners advocated for the NDPC to include sanitation and hygiene expenditure monitoring at district level in the national monitoring and reporting guidelines for MMDAs. This means that MMDAs need to include sanitation and hygiene strategies in their medium and annual development plans which further triggered the Ministry of Finance to have budget lines for sanitation and hygiene in national planning and financial allocation for it. Moreover, even though stakeholders do not directly attribute it to the programme, the creation of the Ministry of Sanitation and Water Resources illustrates how sanitation and hygiene has become a political priority for Ghana.

2. Some guidance documents developed under the Enhanced programme and used in the Accelerated programme are being used as guidance documents by other agencies implementing WASH programmes in Ghana. According to the CSOs, some of the materials updated and reviewed by the programme have been adopted and used not only by the government and WASH sector stakeholders but also by other agencies working on WASH in Ghana. The government ownership of the guidance documents and other documents produced in collaboration with UNICEF has been crucial for the use of these by other agencies.

188. In relation to negative unintended effects, several stakeholders highlighted that community members who are now aware of the need of sanitation and hygiene facilities want their latrines to be constructed with the best technologies. However, they cannot afford to pay for more technologically advanced toilets because these are still quite expensive for them. Thus, the programme might be a cause of frustration. Linked to the issue of wanting more technologically advanced toilets (normally flush toilets), many households do not understand that they will need to pay for the water supply to flush their toilets. According to some stakeholders, many households in Ghana do not pay for their water bills, because they believe they do not have to as they contribute to public services in other ways. Wanting more technologically advanced toilets brings to the table issues around how national government will provide all households with a water supply by 2030.



Preliminary conclusions – EFFECTIVENESS – IN GENERAL:

KEQ 20: Overall, the programme achieved most of its targets, with key exceptions being the targets for ODF communities and districts, household latrines constructed, number of additional people living in households that use improved sanitation, awareness of hygiene practices and additional people practicing HWWS (Paragraph 157). Key improvements which should be considered for future programmes include greater provision of household financing (and greater awareness-raising of those support structures that exist), better quality control of groundwater feasibility assessments and investigation of alternative technologies to provide water to institutions, and continual messaging on hygiene to facilitate behaviour change (Paragraph 160). Further, capacity built among government should be continually reinforced to ensure sustainability of results achieved (Paragraph 160).

KEQ 21: The programme has made progress in addressing gender equity in WASH programming (Paragraphs 161, 162, 169, 170), both in supporting rights-holders to access WASH services and in building the capacity of duty-bearers to deliver these (Paragraphs 163-166, 168). However, it was less effective in mainstreaming disability, and barriers for women, girls and people with disability in accessing WASH services persist (Paragraph 167).

KEQ 22: UNICEF displayed a commitment to adaptive learning from the early stages of the programme (Paragraph 172). Lessons learned and recommendations for improvement were published annually in the Accelerated programme progress reports, based on monitoring data received from fieldwork facilitators and feedback from implementing organisations (Paragraphs 173-177). These sections allowed ongoing strengthening of the programme.

KEQ 23: There is mixed evidence related to changes in health outcomes, although the household survey shows a slight decrease in the number of children suffering diarrhoea in the targeted communities (Paragraphs 181 and 182). Stakeholders were positive that the programme had contributed to improved health outcomes, however (Paragraphs 184 and 185). Improvements can not solely be attributed to the programme due to other policies and programmes addressing health outcomes in the targeted districts (Paragraph 186).

KEQ 24: Stakeholders highlighted that the programme contributed to bringing sanitation and hygiene to the top of the Ghanaian political agenda and that there were positive spill overs, with others in the sector using the guidance produced (Paragraph 187). However, the programme may have caused frustration around lack of access to more technologically sophisticated toilets and lack of water supply (Paragraph 188).



1.3 Efficiency

189. This section assesses the efficiency of the Accelerated programme. ‘Efficiency’ refers to the processes through which programme activities are carried out, and the resources – in terms of time and finance – used; it assesses the cost effectiveness and timeliness of these processes. The evaluation team reviewed the programme’s final expenditure data to assess its cost-effectiveness; this is provided in Annex 10. Expenditure data has been categorised to match the activities in the ToC to the extent possible. The KEQs related to efficiency are presented in the table below.

Table 8: Key evaluation questions for the Accelerated programme related to efficiency KEQ number


KEQ 25 To what extent have the two projects met the intended targets in terms of service delivery in the project duration? In which cases were project targets not met, and what could have been done differently to enable them to have been reached?

KEQ 26 To what extent have the projects delivered the desired targets whilst maximizing the human and financial resources available efficiently?





KEQ 25: To what extent have the two projects met the intended targets in terms of service delivery in the project duration? In which cases were project targets not met, and what could have been done differently to enable them to have been reached?

190. As highlighted in KEQ 21 and summarised in Annex 4, the programme met most of its targets within the project duration.

191. At community level, there was mixed success in relation to targets for ‘increased equitable access to suitable household and institutional sanitation facilities’, as discussed in KEQ 3 and KEQ 13. The programme reached less than half of its intended target for additional people living in households that use improved sanitation and did not achieve its target for ODF districts; however, it significantly exceeded the target for the number of ODF communities. In this context, these first two targets were deemed too ambitious by stakeholders, and particularly the targeted number of additional people living in households that use improved sanitation was considered to be due to an overestimation of the target population.

192. Nonetheless, as highlighted in KEQ 3, there has been a notable change in behaviour around OD, with the proportion of people practicing OD having almost halved since baseline in both small towns and rural communities, and most household latrines having been constructed since the programme began. This significant achievement in latrine construction was facilitated by implementation of both supply-side and demand-side activities as part of the CLTS approach. Thus, demand for new latrines was able to be met from the moment demand was triggered, and financial support was available, to an extent, from the start of the programme through VSLAs, although many community members are still unaware of the available financing support. Failure to reach the target for ODF districts is likely due to limitations in district level capacity to complete the ODF verification protocol, given the high number of communities verified.

193. Targets related to access to sanitation facilities in institutions were exceeded. However, the focus groups did reveal some continued need for investment in facilities, despite this target being reached. As explained in KEQs 8 and 9, one school lacked a changing room for girls, and another was waiting for the facilities for boys to be upgraded. The programme follows the national technical guide for WASH in school facilities, under which all schools should have changing room for girls. Thus, the failure to include a changing room in one school is potentially due to the fact that the construction of changing rooms in schools has only been compulsory since 2018. Thus, schools which have been invested in already may require an additional investment, although the extent of this gap is uncertain. Similarly, as discussed in KEQ 12, there were some concerns about the quality of the facility in one of the health centres visited.

194. The target for provision of water facilities for institutions was only partially achieved, however. This is due to challenges encountered in accessing groundwater in the Northern region, which limited access. Inconsistent quality of feasibility studies for groundwater drilling are an ongoing issue, which UNICEF is already working with the GoG to address.

195. With regards to targets for ‘adoption of good hygiene practices for improved health (including maternal, new-born and child health)’, all of the targets related to engaging health institutions were achieved. There was a slight shortfall in the number of schools with hygiene promotion programmes in place and in reaching community members with hygiene messages. As discussed in KEQ 4, the programme aimed to reach 500,000 people with hygiene message and make 500,000 people aware of safe hygiene practices, although only 432,000 were reached. This is again likely linked to over-estimation of community size at the programme outset. As discussed in KEQs 4 and 7, the target for hygiene promotion programmes in schools has not been achieved.



196. Almost all targets for ‘increased capacity at the community, district and regional levels to support sanitation service delivery and engage the private sector in [the] sanitation market’ were achieved, with the exception of strengthening lessons learning platforms and putting in place district frameworks for regulating private sector investment in sanitation. As highlighted in KEQ 13, the programme has accelerated the implementation of the RSMS in small towns, including through developing a manual to implement the small-town approach. However, stakeholders noted that implementing the approach in small towns has taken longer than in rural communities where government and other stakeholders were already more familiar with the CLTS approach at the programme start. Under the Accelerated programme, UNICEF developed new relationships with districts as well. Implementation of a regulatory structure for private sector investment in sanitation is also a longer-term activity, considering the challenge already faced by the programme in developing a private sector market in the first place.

197. With regards to the target for strengthening lessons learning platforms, as highlighted in KEQ 18, the Enhanced and Accelerated programmes found greater success working with existing lessons learning platforms than in developing new ones, and its contribution to strengthening existing platforms was substantial in terms of results achieved.

KEQ 26: To what extent have the projects delivered the desired targets whilst maximizing the human and financial resources available efficiently?

198. As shown in Annex 10, UNICEF spent a total of 13,943,801 USD in the implementation of the Accelerated programme: 36% of the programme expenditure was used for the construction WASH facilities in institutions and households, 23% in education and behavioural change activities at community and school level, 33% in enabling environment activities – including capacity building, research and knowledge management - and 8% in operational and administrative tasks.

Water and Sanitation activities

199. Thirty-six percent of the programme budget was spent on the construction and rehabilitation of WASH facilities in institutions and households. Of this, most (53%, constituting 19% of the total programme expenditure) was spent on the construction and rehabilitation of schools’ WASH facilities, 23% (constituting 8% of the total programme expenditure) was spent on providing household sanitation facilities (i.e. the establishment of VSLAs and latrine artisans’ selection and training) and 24% (constituting 9% of the total programme expenditure) on WASH infrastructure in health centres. As highlighted in KEQ 20, the programme did not reach some of the targets aiming to increase equitable access to sanitation facilities at household level and at district level.

Household sanitation facilities in communities

200. Nine percent of the total programme budget (as noted above, 24% of the budget for water and sanitation activities) was spent on supporting facilitation of household sanitation facilities in communities. Delivery of the CLTS approach was also supported by line items contained under ‘enabling environment activities’ (improving financing options to households



and engaging with the private sector) and under ‘behavioural change activities’ (CLTS implementation); combining all activities related to construction of household latrines using the CLTS approach39, the budget is 4.7 million USD.

201. The programme’s approach of leveraging household financing for latrine construction is cost efficient, as it led to mobilisation of financial resources and supports ownership and therefore sustainability of the latrines constructed. To benchmark the costs of private latrine construction, the evaluation team used the average cost of 212 USD per latrine, based on the findings of a study by KNUST in 2014.40 To achieve the 17,488 constructed latrines indicated in the KPM database, this would have cost 3.7 million USD. Considering that 4.7 million USD was spent by the programme on activities that directly or indirectly supported latrine construction and that many of these activities would expect to result in additional construction of latrines in the future (including strengthening of the private sector, establishing community financing schemes and providing financing for households through these schemes), the provision of WASH facilities at community level can be considered relatively cost-effective.

WASH infrastructure in schools

202. Nineteen percent of the total programme budget (as noted above, 53% of the budget for water and sanitation activities) was spent on WASH infrastructure in schools, which includes costs of new and refurbished latrine blocks, handwashing facilities, and boreholes installed. Stakeholders highlighted how children have been involved in the construction of tippy taps in schools and in the community, a result of school behaviour change activities which accounted for 2% of the total budget (discussed further below), although these have been emitted from the analysis in this sub-section given the school behaviour change activities were not solely related to WASH infrastructure in schools.

203. To benchmark these costs, the evaluation team used data from a systematic review of WASH in schools construction costs41, which identified unit costs of 8,965 USD for VIP latrine blocks with four cabins in Kenya, 7,606 USD per borehole and handpump in Ghana, and 57 USD per handwashing station in Kenya. UNICEF data shows that, on average approximately seven toilets were provided per school; therefore, assuming two VIP latrine blocks and two handwashing stations are constructed per school, with water provided via borehole to all schools, a benchmark cost for the infrastructure constructed in schools is 2.8 million USD. As these are international comparisons and may have assumed different technologies, these costs are not necessarily directly comparable, particularly considering some of the facilities in the programme were rehabilitated rather than newly constructed and that in 46 schools, water connections were provided via pipe rather than borehole. Nevertheless, the programme may have incurred additional costs such as engaging with the schools, procurement, and building of associated features. Still, it appears that the programme budget of 2.7 million USD for WASH infrastructure in schools was efficient.

WASH Infrastructure in Health Facilities

204. 1.2 million USD was spent on WASH in health facilities (8%) for the construction of 58 sanitation facilities and 58 water facilities (54 boreholes and 4 pipe extensions). Benchmark costs of 356,000 GHC (71,200 USD) per facility were

39 Annex 10: activities 1.3, 2.1, 3.4 and 3.5 40 https://core.ac.uk/download/pdf/234646531.pdf 41 McGinnis, et al. (2017), ‘A Systematic Review: Costing and Financing of Water, Sanitation and Hygiene (WASH) in Schools’, International Journal of Environmental Research and Public Health 14(4): 442. Available online at https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5409642/.

https://core.ac.uk/download/pdf/234646531.pdf

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5409642/



obtained from WaterAid42. Thus, the programme’s 58 facilities would have cost approximately 4.1 million USD under these cost assumptions. This cost does not include the cost of the structure around the facility if it were needed, which is approximately 70,321 USD per facility43. The estimated cost of building full WASH facilities for 58 CHPS compounds is therefore approximately 8.2 million USD.

205. The Accelerated programme constructed WASH facilities in both health centres and CHPS compounds, and it is normally more inexpensive to build these in health centres as they tend to have water access and better infrastructure than CHPS compounds; therefore, the Accelerated programme costs are not fully comparable to the benchmarks available from WaterAid. Nevertheless, the construction or rehabilitation of 58 WASH facilities can be considered comparatively cost effective.

Behavioural change activities

CLTS implementation

206. Thirteen per cent of the total programme budget was spent on promotion of sanitation and hygiene activities by the MMDAs as well as providing the necessary information to MMDAs to promote and carry out broader hygiene and behaviour change interventions. The national campaigns on social norms were also part of the costs incurred for the CLTS implementation.

207. As discussed in KEQ 13, these activities were considered effective. It was also noted by stakeholders consulted that the promotion of the CLTS approach at district levels is a time and resource consuming task, which requires good coordination among programme implementers and commitment from government stakeholders, particularly MMDAs. Moreover, stakeholders also highlighted that a substantial amount of time was spent in getting small towns familiarised with RSMS in comparison to rural communities that knew of this before. Under the Accelerated programme, new districts started collaborating with UNICEF, which also required more time and resources. Other agencies working on WASH in Ghana consider UNICEF the ‘frontrunner’ in involving government in the implementation of the CLTS approach. This strategy is considered efficient because it avoids the creation of parallel systems by using the existing structures. Other agencies often face challenges in directly involving MMDAs in the implementation of the CLTS approach even if they work closely with them.

208. Thus, the evaluation team considers that the Accelerated programme has been cost-effective in providing the right support, as well as building capacity, to promote sanitation and hygiene activities in the targeted communities and institutions.

Handwashing and school behaviour change

42 https://washmatters.wateraid.org/blog/doing-the-maths-financing-wash-in-health-care-facilities-in-ghana 43 Bongo district assembly estimates

https://washmatters.wateraid.org/blog/doing-the-maths-financing-wash-in-health-care-facilities-in-ghana



209. One percent of the programme’s budget was spent on handwashing behaviour change activities, such as HHETPS and training on how to build tippy taps and the promotion of HWWS practices at community level. A further 2% of the programme budget was spent on the implementation of school health clubs, child and youth programmes, and MHM programmes. KEQ 4 found evidenced of mixed success of these activities, with some continued gaps between knowledge and behaviours around HWWS.

210. UNICEF staff highlighted that handwashing activities at community level only took place in communities with UNICEF-supported CLTS activities, given that in communities where open defecation is practiced, faecal contamination of water can undermine the effectiveness of HWWS. Thus, the programme avoided spending resources in communities that were not yet ready for HWWS – a cost-effective approach.

211. Stakeholders, including other WASH programme representatives, considered school behavioural change activities (including handwashing promotion) the most cost-effective activities to reach communities with information on sanitation and hygiene practices. Stakeholders highlighted how children have been involved in the construction of tippy taps in schools and in the community, a cost-effective approach as it mobilised students as agents of change. Moreover, the data from the focus groups suggest that children have been promoting sanitation and hygiene outside of the schools; meaning that the activities in schools have a spillover effect when it comes to promoting sanitation and hygiene at community-level. Further, an economic study carried out by Global Handwashing Partnership in India also suggest that HWWS promotion is shown to the most cost- effective way to avert disability-adjusted life years associated with diarrheal diseases.44

212. Comparing the costs of delivery to KPM data on beneficiaries reached (42,890 children in schools and 433,000 adults) with HWWS messages, the cost of the intervention is 1.14 USD per person, which is lower than the benchmark cost for the promotion of handwashing45. Thus, the promotion of HWWS appears to have been cost-effective.

Enabling environment

213. Thirty-three per cent of the budget was spent on activities to ensure WASH stakeholders had the knowledge and capacity to implement the RSMS and small town CLTS manual by providing training to national, regional and district level staff and supporting M&E systems and practices. Moreover, the enabling environment activities also included the knowledge management and dissemination, coordination of WASH activities and gender mainstreaming activities. The enabling environment activities (33%) comprise six sub-activities: capacity building and training (15%), support and establishment of M&E system and practices (1%), Research, knowledge building and management, dissemination and coordination (10%), engagement with private sector (5%), improving financing options (2%) and gender mainstreaming activities at community level (1%).

Capacity building and training

214. Most of the budget under capacity building and training was spent to provide expertise to government representatives at all levels, particularly at district level, to implement the RSMS based on the CLTS approach in rural communities and small towns. This comprises 15% of the total programme expenditure. These activities provided capacity and expertise to SHEP to implement CLTS activities in schools, to district-level staff (including field facilitators) to establish

44 https://globalhandwashing.org/about-handwashing/why-handwashing/economic-impact/ 45 Global Handwashing Partnership study (3.35 USD)

https://globalhandwashing.org/about-handwashing/why-handwashing/economic-impact/



natural leader networks, to the private sector, and finally to regional government staff to oversee the CLTS implementation of districts.

215. To assess the cost-effectiveness of these activities, we have calculated the unit cost of some capacity building and training activities carried out. The roll out of the RSMS and small-town CLTS approach accounted for 4% of the total programme budget, and the unit cost of rolling this out is 38,397 USD per district. Adding in the roll out of the RSMS and capacity building activities at district level (excluding SHEP capacity building activities); the unit cost is 78,385 USD per district. The unit cost for SHEP training and capacity building activities cannot be calculated as the evaluation team does not have the total number of SHEP staff trained by the end of the programme. The total spent in supporting SHEP staff capacity is 30,692 USD (a total of 0.2% of the programme budget).

216. The remaining budget under capacity building and training (3%) was spent in technical assistance to implement all activities related to capacity building and training, UNICEF staff salaries and travels and other small activities.

217. Although comparable benchmarks are not available for these capacity building costs, the unit costs seem proportionate to their aims and so are considered cost-effective.

Support and establishment of M&E systems and practices

218. To support government’s monitoring systems for tracking WASH results at regional and district level, 1% of the programme’s expenditure went to supporting existing M&E systems and establishing new ones such as the RBF. As mentioned in KEQ 17, government stakeholders indicated that monitoring and reporting on WASH at district level have improved, enabling more effective monitoring at national level. The activities supporting M&E systems and practices could be considered cost-effective; the overall budget for this seems proportionate given the number of systems developed and agencies involved, and the approach to engaging stakeholders supports long-term sustainability of monitoring and accountability.

Research, knowledge building and management, dissemination and coordination

219. Ten percent of the programme budget was spent on research activities as well as activities to manage and disseminate knowledge and coordinate WASH service delivery at all levels. This equates to 103,342 USD per district, which can be considered proportionate given the volume of research produced and number of knowledge exchange events supported, although notably, this is much higher than the cost per district of similar activities for the Enhanced programme and likely reflects economies of scale in the Enhanced programme.

220. As highlighted in KEQ 2, these activities contributed to the adaptive programming and dissemination of knowledge across the WASH sector in Ghana. Adaptive programming activities were cost effective and led to cost savings. Moreover, learning from the Enhanced programme, UNICEF continued strengthening the role of existing coordination structures (DICCS and RICCS) to share knowledge around WASH and coordinate WASH activities, and these have indeed been improved, which is a cost-effective approach.

Gender mainstreaming activities

221. One per cent of the total programme budget was spent on gender mainstreaming activities undertaken at community level. These included training of natural leaders and government partners on gender issues and gender baseline data collection in communities. As highlighted in KEQ 20, the programme has made progress in addressing gender equity



in WASH programming but was less effective in mainstreaming disability, and barriers for women, girls and people with disability in accessing WASH services persist. Nevertheless, there is no budget line-item for ensuring mainstreaming disability around WASH in the same way that there is for gender, and monitoring of disability-friendly features in construction is not carried out. Further resources would be required in future programming to address these persistent barriers.

Cross sectoral

222. Eight percent of the total budget was spent on coordination and operational support (including field monitoring), administration, and supporting the social norms campaign. National government representatives highlighted in KEQ 15 that coordination required a strong strategy and noted that there were some inefficiencies in coordination as a result. UNICEF was considered a front runner in implementing the CLTS approach in collaboration with government, without creating parallel structures. Moreover, it is important to note that the Accelerated programme, together with the Enhanced programme, are the first programmes of its kind, rolling out the RSMS and building capacity at such a large scale. Additionally, in the Accelerated programme, the CLTS approach was rolled out to small town, which required coordination with stakeholders who were not familiar with CLTS. Stakeholders also emphasised how the programme strengthen existing systems by building capacity and streamlining process as well as using existing platforms to share knowledge. Nevertheless, it had a relatively low budget assigned for operational and administration of the programme. Thus, the programme can be considered efficiently managed and well-run.

Preliminary conclusions – EFFICIENCY:

KEQ 25: Most programme targets have been met (Paragraph 190), although there are some exceptions in each of the programme objective areas. In particular, the targets of reaching an additional 500,000 people with improved sanitation and hygiene messages were not met; these were ambitious targets considering the number of people living in the targeted areas (Paragraph 191). Further, the target for ODF verified districts have not yet been achieved (Paragraph 191). This reflects the time required to build capacity at district-level to implement the ODF verification protocol. Targets for access to sanitation facilities in institutions were exceeded (Paragraph 193), although provision of water facilities is a challenge due to the hydrogeological conditions in the Northern region and inconsistent quality of feasibility studies for groundwater drilling (Paragraph 194). Capacity-development targets have also been mostly achieved, although additional time was required to build relationships with new districts (Paragraph 196).

KEQ 26: The programme has been particularly cost-effective when it comes to the construction of household latrines (Paragraphs 200 and 201), as well as WASH facilities for schools (Paragraphs 202 and 203) and health centres (Paragraphs 204 and 205), having spent less on these activities than benchmark costs predict. Activities aimed at improving the enabling environment for WASH were conducted at relatively low cost and bolstered existing structures, which is an efficient approach (Paragraph 213).



Relevance

The Accelerated programme remained relevant to the national and international policy context and the needs of the target population throughout programme implementation. Indeed, the programme helped shape the policy environment by building the knowledge base on CLTS – particularly on how to implement the CLTS approach in small towns – contributing new policies to support WASH sector programming, and raising the prominence of WASH in the national policy dialogue (KEQ 2 and KEQ 23). The programme’s focus on engaging government at all levels and developing guidance and policies that could be used more broadly have therefore been strengths of the programme on which future programmes can continue to build.

The shift to the SDGs, however, has also raised the importance of water access as a development priority which future programmes will need to address, particularly in the Northern region (KEQ 6). Further, the programme did not contribute directly to the management of faecal waste or to policy on faecal management systems, but there is a need for policy guidance on faecal management systems to address sanitation in small towns (KEQ 7).

Effectiveness

The evaluation team has found that the Accelerated programme has been effective in meeting the majority of its targets, and this has been accompanied by a marked reduction in OD with the proportion of communities practicing OD reducing by half since baseline in both small towns and rural communities (KEQ 3). This approach has also been efficient; by leveraging household funding for latrine construction, the programme led to an increase in household sanitation access at a lower cost than a programme of direct provision or full subsidisation would have (KEQ 26). Learnings from the implementation of the Enhanced programme were applied effectively, so the supply-side and demand-side CLTS activities took place in parallel, which is an efficient approach. Funding support was available to households from the start of the programme, and the private sector was strengthened to meet the created demand for household latrines.

However, household-level improvements in sanitation are not universal, and there are some differences in sanitation access between small towns and rural communities. In rural communities, use of private facilities is higher overall, but use of improved unshared activities is higher in small towns, where community members are more able to access artisan-constructed latrines. Conversely in rural communities, standards are inconsistent (KEQ 3). Community knowledge of hygiene practices improved in the communities targeted by the programme; however, the practice of handwashing activities is still inadequate. While participants generally have a good understanding of the importance of HWWS, many of those who said they washed their hands with soap had no water and/or soap available at their handwashing facility on further inspection (KEQ 4 and KEQ 5). In terms of the programme’s ultimate impact on health, regional-level health targets were not achieved, although the survey found evidence of a decrease in children suffering from diarrhoea; external factors mean this cannot be attributed solely to the programme (KEQ 24).

The programme’s consideration of the specific sanitation and hygiene needs of women and girls was an important innovation in WASH programme delivery in Ghana and so was, even if to a lesser extent, that of the specific needs of people with disabilities (KEQ 20). The programme’s aims to support inclusion of marginalised groups, especially poor households,

2 Final Conclusions



rural communities, and women and girls, are commended. However, there continue to be gaps in the access of marginalised groups to WASH infrastructure, and policies and logistical support are not fully in place in the Enhanced programme to meet these needs. First, in small towns, cost remains a barrier to household latrine access, although the programme’s demand-side financial support activities were effective in spurring construction of household latrines among the wider target population. Second, toilets built at the household level rarely met the differential needs of women, girls and people with disabilities – even in households that were home to someone with a disability. Third, logistical barriers presented challenges for field facilitators to reach remote communities (KEQ 3). Fourth, although the programme’s activities around access to clean water at the community level were limited, access to improved water sources appears to have decreased from baseline, and significant numbers of respondents still rely on surface water – especially in rural communities. The precarious water situation in the Northern region merits a focus from future programmes.

Efficiency

The programme was effective in providing WASH facilities in schools and health centres through new constructions and rehabilitation of existing facilities, meeting its targets for new facilities (KEQ 8 and KEQ 12), and it has done so efficiently in comparison to benchmark costs (KEQ 26). WASH in schools activities were primarily focussed on small towns, however (KEQ 14). Poor maintenance of facilities is a key risk to the current benefits of the programme as well as the longer-term sustainability of the facilities, particularly in the context of community use and vandalism of facilities (KEQ 10 and KEQ 12).

Even if some of the programme targets were not met – including the ODF community target – the programme has also been largely effective in strengthening government’s capacity and motivation to meet Ghana’s WASH needs through its advocacy, capacity development, policy development, and knowledge-sharing activities. By strengthening monitoring and accountability mechanisms, government engagement in CLTS will hopefully continue into the future, although government capacity will need to be continually reinforced, and accountability among MMDAs will need to continue to improve to ensure the sustainability of the programme’s results (KEQs 13, 15, 16, 17 and 18).



This evaluation team identified a certain number of lessons learned from the Accelerated programme throughout this report, many o, which would be relevant for future WASH programming in Ghana and in other development contexts:

1. The CLTS approach can be applied effectively and efficiently in small towns, and there is greater availability of artisans in small towns than in rural communities, which can contribute to construction of higher-standard latrines (as found in KEQ 3). However, as the CLTS approach has previously been applied primarily in rural communities, implementing partners including local government in small towns are less likely to be familiar with CLTS, so roll out may take more time there (as found in KEQ 25). This learning may extend more broadly to CLTS programmes in other countries, or across sectors where rural community interventions are transferred to more urbanised areas.

2. Achievement of progress under the CLTS approach can be hindered by the detrimental impact of blanket subsidies for WASH infrastructure, as found in KEQ 22. Although the evaluation has not investigated the prevalence of subsidies, implementers identified that blanket subsidies are sometimes being applied across communities and not necessarily targeting poor and vulnerable communities following the government’s guidelines. District authorities may also be resistant to the CLTS approach due to a preference for building toilets in communities for political favour. This can negate achievements made under the CLTS approach by disincentivising household investment in latrines. Previous studies and national guidelines have identified subsidies as an issue; the CWSA Study of Community Contributions Toward the Capital Cost of Water Facilities supported the principle of beneficiary communities contributing to the cost of WASH facilities, and the 2018 Ministry of Sanitation and Water Resources’ Guidelines for Targeting the Poor and Vulnerable for Basic Sanitation Services in Ghana provides guidelines for targeting the poor and vulnerable in provision of subsidies . These national guidelines, however, seem to have been insufficient, suggesting a need for reinforcement through policy as well as advocacy and capacity-building interventions for the WASH sector.

3. Strengthening government capacity to deliver WASH has been a central activity of the programme, and significant progress has been made, but continued strengthening and maintenance of capacity is required to build on this and to sustain the progress that has been achieved, as found in KEQs 15-17. There has not yet been sufficient time for authorities at district level to fully adopt some of the systems which were developed under the programme, such as the ODF verification protocol and BaSIS, particularly in districts that were less familiar with the CLTS approach at the programme outset. Future capacity building programmes across sectors should ensure sufficient time is planned for to fully test and roll out new systems before programme completion.

4. WASH facilities constructed in schools (as found in KEQ 14) and health centres (as found in KEQ 12) are vulnerable to misuse, damage, and by community members who lack access to household latrines, or whose latrines are not high-quality, in communities that where open defecation is practiced. This jeopardises the sustainability of investments in facility construction, especially as the institutions often lack the funds and skills to repair them. Sufficient attention (and budget) needs to be paid in programme implementation to ensure institutions receive operations and maintenance training, but even where this takes place and institutions take on responsibility for ongoing maintenance, this is insufficient and can place burdens on staff and students, for example to provide cleaning materials. Linked to this, timely and clear handover of constructed facilities is important to ensure clarity

4 Lessons Learned



as to ownership of the facilities as well as responsibility for maintenance and repair. This lesson is particularly of relevance to WASH programmes, which should therefore consider how best to link investments in institutional facilities with a broader suite of interventions at community-level and among duty-bearers to support accountability for maintenance of facilities while ensuring the needs of community members and rights-holders are met, but it is more broadly relevant for development interventions in institutions that assume the capacity of rights-holders to maintain constructed facilities.

5. Accountability among duty-bearers is an important precursor for achieving success from the CLTS approach, and accountability has been built effectively while at the same time those duty-bearers’ capacities to deliver the approach was built, although further strengthening of accountability and capacities is required. Intensive engagement of government at all levels in advocacy, capacity-building, delivery, accountability, and knowledge-sharing has been an effective approach, although this can be strengthened by involving local authorities at the programme outset to support target-setting. While long-term results of the programme have not yet been measured, development practitioners in other contexts may be able to learn from and replicate this type of model across sectors and contexts.

6. The ability of the evaluation team – and UNICEF – to measure the results of the programme was limited by the lack of a comprehensive results framework that fully reflected the programme’s Theory of Change. In particular, the results framework focussed on activities and outputs only, rather than outcomes; as a result, baseline data is missing against which the evaluation could have measured progress toward the intended results of the programme. In addition, the results framework lacked inclusion of some of the activities carried out by the programme, meaning there were not output targets against which progress could have been measured.

7. Although provision of water services was not one of the key activities of the Accelerated programme, water access remains an important issue in Ghana, especially in the Northern region. The shift from the MDGs to SDGs further raises the need for the WASH sector to prioritise water access, and in the Northern region, there are problems of both limited access to safe water and gendered issues around water collection and management, which places a particular burden on women and girls. This is exacerbated by hydrogeological conditions in the region which make the drilling success rate low, which is a further problem when drilling relies on drilling feasibility studies of inconsistent quality. WASH programmes in the future will need to incorporate water access in order to address current WASH inequities comprehensively.



The evaluation team developed recommendations based on the findings and conclusions of this evaluation and based on recommendations raised by interviewees. Recommendations were also discussed at the technical review meeting with UNICEF in Ghana in order to obtain feedback and improve the feasibility and prioritisation of recommendations. These are separated into strategic recommendations, which are likely to be of interest not just to UNICEF in Ghana but to the broader WASH sector, and more operational recommendations which are primarily targeted toward UNICEF in Ghana.

5 Recommendations



Table 9: Strategic recommendations

Number of the KEQ on which the recommendation builds

Recommendations

Recipient(s) of the recommendation

Level of priority

Comments

KEQ 21 1. Future programmes should continue building government capacity and accountability to deliver the CLTS approach.

a. Government at all levels should continue to provide a supportive policy environment for WASH through its expressed commitments to WASH and its collaboration with UNICEF and others in the sector.

b. Use of BaSIS should be intensified as a monitoring mechanism to continue to hold government to account.

c. Future programmes should continue to include capacity building elements and should intensify efforts to share knowledge across the WASH sector in Ghana.

d. This evaluation supports the review of the ODF verification protocol, which has now been completed. Further, authorities should be supported to carry out verification. The government is currently working to develop a national ODF plan, and this should incorporate training on ODF verification. District ODF plans would also benefit from having a national

All WASH sector stakeholders, including UNICEF, government, and other WASH programmes

1



costed ODF plan, as well as associated guidance and financial commitments.

e. Future WASH programming should continue to develop community rules and regulations, and the ODF verification process should refer to these. However, district-level enforcement of these rules is needed. Government should consider what enforcement mechanisms or incentives are possible to promote compliance and develop national guidelines on this, and districts should enforce these.

KEQs 3 2. Joint roll-out of supply-side and demand-side interventions works well, and future programmes should adopt this approach.

b. However, affordability of household toilets in small towns is an issue that future programmes should continue to focus on, and better publicity of available financing support should be communicated. More should be done to ensure the poorest households have access to finance to construct latrines.

c. Community-level sensitisation activities should also be carried out to deter community members from using, damaging, and vandalising school and health centre sanitation facilities. It is important for there to be a stronger link between delivery of CLTS interventions in communities and provision of school and health centre WASH programmes.

All WASH sector stakeholders, including UNICEF, government, and other WASH programmes; CSOs and financial institutions

1 Sensitisation is unlikely to be sufficient to deter public use and damage of the facilities, and community members need to be able to access sanitation facilities outside of these institutions. Withholding school support until communities have achieved ODF status is not recommended.



KEQs 3 and 23 3. There needs to be a clear national policy set by government against provision of blanket subsidies.

a. National government should build on its existing guidelines to formalise a national policy against provision of blanket subsidies.

b. Future WASH programmes should reinforce this policy through advocacy and accountability activities.

Government and other WASH sector stakeholders, including CSOs and regional and district governments, have a responsibility for ensuring blanket subsidies are not applied. National government should ensure all stakeholders are sensitised on the importance of this policy.

1 Such a policy would be consistent with the recommendations of the CWSA Study of Community Contributions Toward the Capital Cost of Water Facilities, which supported the principle of beneficiary communities contributing to the cost of WASH facilities, and the 2018 Ministry of Sanitation and Water Resources’ Guidelines for Targeting the Poor and Vulnerable for Basic Sanitation Services in Ghana, which provides guidelines for targeting the poor and vulnerable in provision of subsidies.

KEQs 6 and 20 4. Intensify efforts to improve access to clean water at the community and institutional levels, especially in the Northern region, if Ghana is to achieve the SDG target for universal and equitable access to safe and affordable drinking water.

b. Future UNICEF WASH programmes should put greater emphasis on programme activities targeting improved access to clean water, including advocating for sustained government commitment to achieving universal access to clean drinking water.

c. Stricter controls need to be put in place for hydrogeological investigations to ensure the protocol for groundwater feasibility studies is carried out before drilling begins. Alternative approaches should be explored to harvesting water and to minimise the loss of dry boreholes.

d. To improve economic and social opportunities for women and girls, future WASH programme should invest

All WASH sector stakeholders, especially government at national and regional levels

2



more in campaigns to change social norms and gender roles around water collection and management and to improve access to basic drinking water.

KEQ 4 5. There is a need for a systematic approach to ensure continuous messaging on hygiene, with greater follow-up of training delivered.

b. For women and girls, taboos and stigma may be a barrier to safe disposal of menstrual materials, so social norms campaigns to shape these attitudes should be developed and rolled out, beyond schools, as part of WASH sector programming. For example, UNICEF in Ghana should build on UNICEF’s new Behavioural Drivers Model to develop this and incorporate it in CLTS messaging.

c. Future programmes should also advocate for better relationships between schools and communities, so the learnings from sanitation and hygiene promotion activities in schools can be cascaded to communities. Learnings can be communicated at the School Management Committees and Parent-Teacher Associations, and these could be orientated in supporting WASH facilities in schools and ensuring that community members do not use the facilities.


2

KEQs 4, 8, 9 and 20 6. Future WASH programming in Ghana should incorporate a gender- and disability-inclusive approach in CLTS.


3 For example, at the household level, although not every household may require these features, incorporating them in training materials and guidance



b. First, technical documents related to latrine construction should incorporate gender-friendly features that promote the safety of women and girls, such as locks and lighting, as well as more disability-inclusive features such as grab rails – currently, only ramps are included in the Latrine Technology Manual. Sensitisation materials should also address the unique needs of women, girls and people with disabilities.

c. This should include advocating for the WASH in Schools standard to specify that changing rooms need to be constructed in all schools, regardless of student age.

d. Second, the strategies and policy documents developed by UNICEF could be enhanced to take disability into account and to be more gender-inclusive.

will ensure that they are offered to households and the benefits are explained. As another example, in schools, school staff and other duty-bearers should be trained on disability inclusion to ensure disability-friendly facilities are both built and used for their intended purpose For example, the National Costed Strategy for WASH in Schools could: a. Describe the situation of women and men, girls and boys (and different sub-groups such as youth and people with disabilities) in the sector and address them in a targeted way. b. Check that adequate budget is allocated to implement hidden gender-specific issues such as: women and girls’ participation in training, women’s inclusion in contracting process, women’s access to contracts and the labour force in construction of the WASH facilities; and women’s participation in decision making in WASH Schools and community projects. c. Check whether the expenditure is spent as planned on the gender specific items identified in the strategy. d. Finally, examine the impact of the strategy and expenditure i.e. whether it has promoted gender equity as intended; and allocate some resources towards the impact assessment post project completion. At the moment, the standard says that changing rooms should be available in schools for adolescents’ girls. All types of schools should be included so there



is no confusion among WASH sector stakeholders when using the standard.

KEQs 1 and 7 7. Future programmes should contribute to piloting and tailoring the faecal sludge management strategy, ensuring its final implementation as a national approach to manage faecal waste in small towns.


3 The GoG is in the process of agreeing a finalised national strategy to manage faecal waste in small towns, which is expected to be finished by the first quarter of 2020. The faecal sludge management strategy will be piloted in five MMDAs under a separate UNICEF urban WASH programme. After two years of piloting the strategy, it should potentially become a national approach to manage faecal waste in small towns. Substantial funding commitment will be needed for its implementation.



Table 10: Operational recommendations Number of the KEQ on which the recommendation builds

Recommendations

Recipient(s) of the recommendation

Level of priority

KEQs 10 and 12 1. The handover of constructed facilities from contractors to GHS/GES and finally to institutions should be improved. Handovers should take place as soon as facility construction is complete.

b. CWSA could be involved in handover given its stronger local presence.

c. Handover kits and maintenance checklist should be developed to ensure the three-stage handover is well structured and carried out smoothly.

d. Subsequently, more support should be given to institutions to maintain and repair handwashing and sanitation facilities over time to enable sustained improvements in WASH practices. While support to maintenance and repairs does occur, this is insufficient; district authorities should commit further resources to this.

e. More training should be offered to health centre staff on how to maintain and operate WASH facilities and on how to promote good hygiene and sanitation practices among visitors. Handover processes should clarify the responsibilities of institutions’ staff as well as GHS in maintaining facilities.

GHS, GES, CWSA and contractors, MMDAs and donors. GHS is change of the operation and maintenance of WASH facilities, and they provide training to health centre staff. An O&M manual is being prepared by UNICEF to assist GHS with their O&M training.

1

KEQ 13 2. Logistical support (particularly at district level) should be strengthened in future WASH programmes to ensure hard-to-reach populations are included in programming.

b. Fieldwork facilitators should be offered more transportation and communication resources to ensure that programming reaches remote areas, and this should be funded by WASH sector donors.

MMDAs and donors 3

See ‘Methodology’ section

3. Future UNICEF-supported WASH programming should include a more holistic set of indicators at the output and outcome levels.

b. All programme activities should have a clear rationale rooted in the programme’s Theory of Change and should be included in the results framework. Indicators should include accompanying baseline data, and targets should be aligned to the SDG framework. Donors supporting WASH programming

UNICEF and donors 3



through UNICEF should support this and work collaboratively to define the Theory of Change and results framework at the programme design phase.



This Annex provides a Theory of Change (ToC) for the Accelerated programme. It provides an overall narrative that explains how the programme aims to achieve their intended objectives. It traces the programme inputs through to its intended impacts, (including the key risks and assumptions underpinning the programme) and contextual factors that may influence the programme in achieving its objectives. Throughout, it discusses a set of wider issues and hypotheses that are relevant to the programme which will be explored in the evaluation. This TOC was initially developed during the Inception Phase by the evaluation team based on an initial desk-based review of programme documentation and monitoring records and a series of familiarisation interviews with key programme stakeholders; it has been subsequently refined during the evaluation. The ToC is described below and illustrated in a logic model in the main report.

7.4.1 Inputs

Inputs are the financial and human resources that are used to deliver the Accelerated programme:

Global Affairs Canada funding and human resources: GAC provides the overall budget of the programme, funding UNICEF’s activities and contributing its resources and networks. It also facilitates policy dialogue, knowledge generation and scale up of interventions across WASH stakeholders in Ghana

UNICEF human resources: Human resources provided by UNICEF include staff working in UNICEF Ghana offices and UNICEF Headquarters. UNICEF’s team in Ghana working in WASH programmes, comprises around 23 employed staff46, with a wide range of expertise such as gender policy mainstreaming, WASH facility management etc. UNICEF also hires consultants (between 5 to 10) to work alongside regional and local government bodies in the implementation of the programme at local levels. UNICEF ensures the implementation of the programme, from the programme planning, coordination of partners, progress monitoring to carrying out programme activities and mitigate risks and challenges in programme delivery.

Funded project partners resources: Civil society organisations and private sector representatives receive funding from UNICEF and contribute human resources to carry out many of the education and behavioural change activities around sanitation, hygiene and safe water access. They also ensure presence at both national and at decentralised level.

Human resources from government bodies at local, regional and national level. Government partners have several roles in the WASH programme implementation including leading the CLTS implementation and activities around building capacity for operation and maintenance of the facilities constructed.

46 Familiarisation interview

Annexes

Annex 1: Updated ToC



7.4.2 Activities

Activities are the tasks undertaken by the programme. The programme has the following six types of activities:

Provision and rehabilitation of water and sanitation infrastructure and services in schools and health centres. The programme has developed design options for school latrines and has constructed and rehabilitated latrines, water, and handwashing facilities in schools and health centres. Latrines in schools aim to be child, gender, and disability friendly, including separate toilets for boys and girls, and to be consistent with the Ghana Education Service (GES) Minimum Guidelines on WASH in Schools.

Training activities on how to use and maintain water and sanitation facilities and services. This activity category includes training school teachers, students, health workers, and patients on proper use and maintenance of latrines, and supporting schools and health centres to develop or update their latrine operation and maintenance plans.

Education and behavioural change activities around sanitation, hygiene and safe water, particularly gender sanitation. This type of activity includes various activities aimed at increasing knowledge, changing behaviour, generating demand for sanitation facilities (a crucial component of the demand-driven sanitation CLTS approach), and creating long-term sustainability by generating leadership skills and supporting local monitoring of facilities:

CLTS activities in schools, health centres and community like Health and Hygiene Education Through Play and Sports (HHETPS) where children in schools and community members participate in games and activities to increase their knowledge, skills and practise of handwashing with soap. CLTS activities in the communities include activities such construction of tippy taps47 in groups,

Mentoring and coaching girls in school health clubs, children and youth ambassadors for WASH (CYAWASH) through the Youth as a Leader (YAL) module and other mentoring activities. These enable children and particularly girls to engage in the sanitation and hygiene activities as leaders and promoters. In the YAL module, children are supported to implement innovative child and youth led activities to ensure their district becomes open defecation free and ensure that people practise handwashing with soap.

Marketing and promotional activities like ante-natal sessions and child immunization sessions to promote hygiene in health centres. The health workers delivering antenatal and child immunisation services are trained on the promotion of hygiene practices and demonstration of appropriate handwashing techniques to mothers, relevant for the care of the new-born child and the infants.

Supporting the establishment of village savings and loans schemes for household heads to save towards improved sanitation, through training activities (especially for women), and supporting micro-finance institutions to establish financing schemes for householders, as well as conditional cash grants for the poorest people in the project districts.

Government capacity building activities are activities that enable the government to provide and scale-up WASH facilities and services, develop WASH policies that are gender and environmental responsive and build partnerships with other institutions to improve WASH services and facilities. The activities include:

47 The Tippy Tap is a simple device for hand washing with running water. A container of 5 litter with a small hole near the cap is filled with water and tipped with a stick and rope tied through a hole in the cap



Supporting Basis Sanitation Information System (BaSIS), a decentralised M&E sanitation system developed to aid in the implementation of CLTS at both sub-national and national levels.

Supporting the implementation of environmental management plans using the environmental management guidelines under this programme.

Supporting the establishment of strategic partnerships with institutions - school of health and hygiene, nursing training schools.

Supporting the development of a national school sanitation strategy for achieving universal access to sanitation in Ghanaian basic schools, including national guidelines for school WASH facilities.

Supporting communities to undertake the ODF verification and certification process.

Non-government capacity building activities include: training of non-government stakeholders to use the CLTS approach to promote hygiene in their communities, develop materials to promote WASH and contribute to existing campaigns and supporting establishment of private-public partnerships.

Dissemination and knowledge sharing activities across WASH stakeholders. The programme develops sector documentation and information through a research programme and promotes dialogue, including support for the learning alliance platform sessions on sanitation at National, regional and district level.

7.4.3 Outputs

Outputs are immediately delivered by the programme as a direct result of the activities undertaken. The outputs of the programme are:

Child, gender and disability friendly improved latrines constructed, and potable water systems and hand washing facilities provided/installed. These two outputs directly result from the construction and rehabilitation of latrines and hand washing facilities in schools and health centres. The construction and rehabilitation of water and sanitation facilities in schools and health centres are the responsibility of the CWSA. The CWSA should procure and supervise the construction of water and sanitation facilities in schools, and build capacity for operation and maintenance of the facilities constructed.

Training schools and health centres workers on how to use and maintain water and sanitation facilities and services, results directly in schools and health centres workers trained on the operation and maintenance of WASH facilities and services.

Increased knowledge of community members around sanitation, hygiene and safe water, and increased number of WASH promoters in the communities are expected to result from the several types of education and behavioural change activities around sanitation, hygiene and safe water.

Training of natural leaders to support the implementation of the CLTS approach at the community level and establishing natural leader networks.

Government staff is trained on technical skill such as implementing BaSIS functionality in districts. All training activities targeting government staff are expected to translate into trained government staff, implemented MIS systems and developed WASH guidance and policy documents.

Non-governmental organisations staff trained on how to use CLTS approach to promote hygiene in their communities is a direct output of training activities carried out with non-government stakeholders.



The dissemination and knowledge sharing activities translate into outputs such as the implementation of Knowledge Management Initiative, the attendance of CLTS Stocktaking Forum and the development of reports, newsletters and other dissemination materials.

7.4.4 Outcomes

Outcomes are the immediate to long-term results expected from the programme.

Immediate outcomes are the results expected to emerge by the end of the programme (June 2019):

The greater access to and use of gender and disability friendly WASH facilities in schools, health centre and households, as well as the increased proportion of people washing their hands with soap in schools, health centres and at home, are immediate project outcomes linked to both the availability of improved latrines, potable water systems and hand washing facilities and the increased knowledge of the importance of WASH behaviours. Greater access to and use of WASH facilities in schools, health centre and households is only possible if demand for these facilities and service exists. At the household level, demand generated also triggers households to construct or pay for the construction of HWWS facilities. Thus, the demand creation for sanitation through CLTS activities links directly to the availability of improved latrines, potable water systems and hand washing facilities. Moreover, greater access to and use of WASH facilities and improved hygiene practices across the community is not possible without trained schools and health centres on how to operate and maintain WASH facilities and services.

The increased knowledge, understanding and commitment of the community around sanitation, hygiene and safe water, including the WASH promoters in the communities, leads to change in individuals’ behaviour, resulting in an increase in the number of communities where open defecation is not a common a practice (Increased number of communities with ODF status).

More accountable and gender-responsive government departments (national, regional and local level) delivering WASH services is an immediate outcome of the programme due to many capacity building activities carried out with government officials.

Trained non-government organisations translate into more involvement of the private sector in WASH service delivery.

The programme has invested in creating knowledge (e.g. the development of guidance documents) and dissemination the knowledge created, both within UNICEF and to wider stakeholders. Thus, one of the programme outcomes is the increased understanding of WASH service implementation in Ghana.

Long term outcomes are expected to materialise following the completion of the Accelerated programme. These are:

Sustained access to gender and disability friendly WASH facilities in schools, health centre and households which results from ongoing maintenance of facilities constructed, as well as continued demand amongst households for construction of new facilities.

Sustained change in behaviours, including washing hands with soap and use of latrines, resulting from retained knowledge.

More people living ODF communities because of large number of communities with an ODF status.

Stakeholders are involved in WASH service delivery remaining accountable and responsive to the WASH needs of the community as they have been trained to be accountable and gender-responsive in delivering WASH services,



have access to adequate policies and systems, and are part of an enabling environment that produces and shares knowledge around WASH services and facilities.

7.4.5 Impacts

The ultimate impact of the programme is to improve the health and well-being of children, women and men in communities in the regions targeted. Specifically, by reducing the rate of under-five child mortality rates and percentage children who had diarrhoea, worm infestation, typhoid and other sanitation-related diseases.

7.4.7 Key assumptions/risks

• Successful realisation of the intended programme outputs, outcomes and impacts relies on certain assumptions holding true. Failure of these assumptions would pose risks to the programme’s success:

▪ Political will is maintained at all levels of government. National government commitment to achieving national ODF, through meeting funding commitments and political support, are fundamental to enabling upscaling of programme outcomes. Moreover, the regional and local support is highly important when trying to progress regional and local planning and sanitation infrastructure.

▪ Partners have sufficient capacity and accountability to deliver the activities. Consultants are contracted to support district-level government in their activities, particularly to help with monitoring activities. Further, there are risks related to behavioural changing activities and disruption of social norms. Field facilitators need to be fully trained and committed to the CLTS activities and social norm campaigns, and there is a risk that this is not the case.

▪ Government at all levels has the capacity and accountability to implement the programmes. Many government partners are part of the programme delivery, and they must be able to implement their components of the programme, within the programme timescales, and carry out ongoing management and accountability functions. This includes having the capacity to implement strategies and guidance documents developed through the programmes such as those related to mainstreaming gender equity and equality strategies.

▪ Beneficiaries are willing and able to engage in sustainable sanitation practices and have access to finance or the skills needed to construct WASH facilities. Without a willingness of householders to pay for sanitation, either directly or indirectly, achieving ODF is not possible. Familiarisation interviews noted that some communities are harder to engage than others (e.g. ethnic groups with strong social beliefs around defecation and sanitation overall).

▪ Students, teachers, health workers and households, particularly women, have time and intention to participate in all CLTS activities. Beneficiaries might not have the time nor the intention to participate in all the CLTS activities included in both programmes.

▪ The private sector is able to engage in demand generation. The private sector is fundamental to providing the supply components of sanitation infrastructure, as well as a crucial player in demand generation. However, there is a risk that they are not adequately engaged in generating demand for sanitation hardware and technical skills, potentially compromising ODF attainment; this could arise due to labour market shortages or lack of sufficient incentive (e.g. if the expected returns from participating in sanitation do not exceed the costs).

▪ The programme is aligned with other WASH programmes. There are numerous other WASH programmes in the initiative, some of which may be operational in the same regions. There is therefore an assumption that the programmes are well-coordinated, either covering separate geographic zones or carrying out complementary activities.

▪ Water supply can be improved using the programme’s recommended methods. Water supply access, particularly in the northern region, is a challenge in areas where the geological conditions are not optimal. The programmes



include activities that improve access to potable water, which is mainly through digging holes in the ground. However, this is not possible in Ghana’s Northern region.

The evaluation matrix was set out in the Inception Report. This maps the KEQs to evaluation indicators and data sources.

Table A.2.1 Evaluation matrix

TORQ number

Revised KEQ number

Key Evaluation Question (KEQ) Evaluation Indicators Internal documents

Secondary data/ literature

Interviews

Household survey/ observation

Focus groups

1. Effectiveness - In general

1.1.3 1 To what extent have the projects effectively addressed the specific needs of women and girls?

- Evidence of inclusion of gender equality objectives and consideration of the specific needs of women and girls in programme policy documents, in line with international best practice - Evidence of mainstreaming of programme gender policies in implementation - Evidence of the participation of women and girls in programme design and implementation - Availability of gender-disaggregated indicators, and evidence of equality of outcomes for men and women, boys and girls in programme MI - Evidence of equality of outcomes for men and women in survey results - Perceptions of women, girls and gender experts on the programme's effectiveness in addressing the needs of women and girls

1.1.4 and 3.1.1

2 In which cases were project targets not met, and what could have been done differently to enable them to have been reached? How should future programming be designed in order to overcome experienced challenges?

- Identification of challenges and lessons learnt from progress reports, qualitative interviews, and analysis of learnings from other evaluation questions - Benchmarking of programme against best practice for WASH programming - Perspectives of interviewees on challenges and opportunities for improvement of programme design

NA 3 How effectively has the programme responded to learnings and recommendations during delivery, and to what extent have key learnings been disseminated more widely?

- Evidence of incorporation of learnings and recommendations from progress reports into programme delivery - Evidence of dissemination of learnings to other UNICEF programmes or within Ghana

Annex 2: Evaluation matrix



TORQ number

Revised KEQ number



Interviews


Focus groups

1.1.2 4 What were (if any) the unintended effects (positive and negative) produced by the two projects?

- Perceptions on unintended effects

1.1.5 5 To what extent have health outcomes improved in the project areas?

- Perceptions of health professionals that the programme design and delivery is expected to result in positive health outcomes - Evidence from wider literature of the contribution of similar WASH activities to the expected health outcomes - Analysis of other potential contributing factors to any observed health outcomes

2. Effectiveness - WASH in rural communities and small towns

1.2.1, 1.3.1, and 1.5.1

6 What is the access to household sanitation in the targeted regions and districts? Are the facilities in use? Are there variations in access in rural communities versus small towns?

- Evidence of an increase in number of household sanitation facilities - Evidence of continued use of household sanitation facilities - Comparison of availability and use of household sanitation facilities between rural communities versus small towns - Stakeholder perceptions of effectiveness of implementation of sanitation activities

1.2.2 and 1.3.3

7 What do community members and school children know about the hygiene practices which the programme promoted? Are there variations in knowledge in rural communities versus small towns?

- Evidence of knowledge of hygiene practices among survey respondents - Evidence of knowledge of hygiene practices among focus group participants - Increase in the number and proportion of the population (in the targeted areas) practicing hand washing with soap at critical times - Comparison of knowledge of hygiene practices between rural communities versus small towns - Stakeholder perceptions of effectiveness of implementation of sanitation activities

1.2.3 8 To what extent do community members and school children have appropriate HWWS facilities? Are there variations in access in rural communities versus small towns?

- Evidence of an increase in HWWS at household level - Evidence of an increase in HWWS at schools - Comparison of HWWS levels between rural communities versus small towns - Stakeholder perceptions of effectiveness of implementation of HWWS activities

1.2.4, 1.3.4, 1nd 1.5.4

9 What is the access to safe water supply (including HWTS) in the targeted rural communities and small towns?

- Evidence of an increase in number of safe water facilities - Evidence of an increase in number and proportion of population using household water treatment and safe storage systems - Evidence of continued use of safe water facilities and water treatment behaviour - Comparison of access to safe water supply between rural communities versus small towns - Stakeholder perceptions of effectiveness of implementation of safe water supply activities



TORQ number

Revised KEQ number



Interviews


Focus groups

1.3.2 10 How is faecal sludge managed in the targeted small towns?

- Stakeholder perceptions of faecal sludge management - Alignment of reported/documented faecal sludge management procedures with UNICEF procedures - Documented evidence of faecal sludge management and indicators at the local level

3. Effectiveness - WASH in institutions (schools and health centres)

1.4.1 and 1.4.7

11 To what extent do school boys and girls have access to school sanitation facilities? What is the functionality status of WASH facilities in schools? Are there variations between rural communities and small towns?

- Student/teacher perceptions on access and barriers to access to school sanitation facilities - Students' reported use of school sanitation facilities - Evidence from MI of increased availability and use of school sanitation facilities

1.4.2 12 To what extent are the school sanitation facilities gender- and disability- friendly? Are there variations between rural communities and small towns?

- Alignment of programme design for sanitation facilities with international best practice for gender and disability mainstreaming in WASH programming (Inclusion of rails and ramps in latrines and by handwashing basins in design of school sanitation facilities, separate toilet facilities for both boys and girls which are conveniently sited not far from school and also provide privacy and security, changing room and sanitary pads for girls, separate squat hole for disabled pupils and supporting material to aid in using the toilet) - Achievement of target for number of schools that are gender- and disability-friendly - Students with disability and girls' reported use of school sanitation facilities - Availability of emergency menstrual kits with the School Based Health Coordinator, School Health Club Leaders or Class teachers - Evidence from MI of increased availability and use of school sanitation facilities by girls and children with disability



TORQ number

Revised KEQ number



Interviews


Focus groups

1.4.4 13 What proportion of schools have facility management plans in place?

- Proportion of schools with facility management plans in place - Perspectives on the effectiveness of support for facility management plan development and implementation - Perspectives on the ongoing implementation of facility management plans

1.4.5 14 What is the level of knowledge on menstrual hygiene management and hygiene behaviours?

- Survey and focus group evidence of knowledge on MHM (including how to calculate the next menstrual cycle in order to prepare for the next menstruation, materials to use to manage menstruation, how to manage menstrual cramps/pains, how to observe proper hygiene during menstruation, and the proper way of disposing soiled sanitary pads or sanitary materials)

1.4.6 and 1.4.7

15 To what extent are WASH facilities in health centres and CHPS compounds available? What is the functionality status of WASH facilities in health facilities?

- Evidence of increase in the number of WASH facilities in health centres and CHPS compounds - Perspectives of the ongoing functionality of WASH facilities in health centres and CHPS compounds

4. Effectiveness - District level WASH access

1.5.2 and 1.5.3

16 How many certified ODF communities are there within each of the targeted districts, and to what extent have the number and proportion of the population living in ODF communities increased?

- Achievement of target for population practicing ODF - Achievement of target for number of ODF communities - MLGRD interviewee perspectives on programme effectiveness and contribution to ODF

1.4.3 17 How does access to WASH facilities in schools in rural communities compare with schools in small towns? And what impact does it have on ODF status?

- Achievement of target for number of schools with adequate facilities for handwashing - Comparison of statistics on WASH facilities for rural versus small towns - Comparison of statistics on WASH facilities with achievement of ODF status - Stakeholder perceptions of contribution of WASH facilities in schools to ODF status

5. Effectiveness - Governance and knowledge management

1.6.1 18 What plans exist at the regional level for WASH implementation in the 5 regions?

- Evidence of increase in presence or quality of WASH implementation plans at regional levels - Perspectives of the effectiveness of support in developing/adopting WASH implementation plans

1.6.2 19 What systems for accountability exist at district levels?

- Evidence of increase in presence or quality of systems of accountability (DESSAP, BaSIS, review bodies, etc.) at district levels - Perspectives of the effectiveness of support in developing/adopting accountability systems

1.6.3 20 What monitoring systems exist for tracking WASH results at the regional and district level?

- Evidence of increase in presence or quality of monitoring systems at regional and district levels - Perspectives of the effectiveness of support in developing/adopting monitoring systems



TORQ number

Revised KEQ number



Interviews


Focus groups

1.7.1 21 Have the knowledge management components of the project improved understandings?

- Stakeholder perspectives on impact of knowledge management components - Documentary evidence of knowledge management outputs

1.6.4 22 Are appropriate systems for sanitation law enforcement established?

- Evidence from interviews and document review that district authorities have approved and gazetted bye-laws on environmental sanitation - Evidence from interviews that people in the municipal/district assembly area have knowledge of the bye-laws and how they (bye-laws) affect their sanitation behaviour

7. Efficiency

3.1.1 23 To what extent have the two projects met the intended targets in terms of service delivery in the project duration? In which cases were project targets not met, and what could have been done differently to enable them to have been reached?

- MI system evidence of achievement of delivery targets for all activities within the planned timeframes - Perspectives on effectiveness of process delivery and achievement of intended outputs

2.1.1 24 To what extent have the projects delivered the desired targets whilst maximizing the human and financial resources available efficiently?

- Comparison to costs incurred relative to budget forecasts - Benchmarking of programme unit costs and staff size to other WASH programmes in Ghana/the region - Perceptions of efficiency of programme management and governance

8. Relevance

3.2.1 25 Has research been appropriately designed to answer sector knowledge gaps?

- Stakeholder perspectives on relevance of research processes and outputs

NA 26 How relevant is the design of the programmes to national and international policies and strategies for WASH?

- Evidence from programme documents and interviews that national standards and targets are incorporated into programme design and achieved, e.g. AMCOW targets and SDGs

This evaluation reviewed key programme documentation and guidance documents – both to develop a better understanding of the project and as part of the internal document review. However, some programme documentation – such as the raw enhanced baseline data – was not available upon request, which limited the scope of the evaluation. This annex provides a list of all the key programme documentation reviewed:

Accelerated Programme baseline survey

Annex 3: Programme documentation reviewed



Accelerated baseline reports 2016-2019

Accelerated mid-term review

Accelerated programme proposal

UNICEF Ghana CLTS database

UNICEF Ghana WASH in Institution’s database

UNICEF WASH standard surveys – household, school, health facility

ODF verification and certification protocol

Menstrual hygiene management guidelines

Gender mainstreaming guidelines and toolkit for water, sanitation and hygiene

UNICEF community facilitation toolkit

Latrine Artisans Training Manual - for Basic Training

Quality Assurance for Toilet Construction - Implementation Guide

Provision of Technology Support to CLTS Communities - A Guide for District Facilitators

Minimum Guidelines for Household Toilets

MHM education booklet and flash cards

National Costed Strategy for WinS

Education sector midterm development plan

Ghana Poverty and Inequality analysis

CWSA Legislative Instrument

Period poverty impact on the economic empowerment of women

Mainstreaming MHM in schools through the play-based approach: lessons learned from Ghana

Education Sector Mid Term Development Plan (2018-2021)

The Education Strategic Plan 2018-2030

The table below maps the programme objectives to outcomes in the ToC and associated key KPM targets. It explains which programme targets were not achieved, achieved, and exceeded.

Annex 4: Programme target achievement



Programme objectives

Programme outcomes (ToC)

Key performance measurement targets Results based on programme targets

Increased, equitable access to suitable household and institutional sanitation facilities.

Greater access and use of gender and disability friendly facilities in schools, health centres and households

1. 500,000 additional people living in household that use improved sanitation

2. 80 ODF communities and smalls towns in 10 districts

3. 5 ODF districts 4. 10 districts with evidence-based ODF

plans. 5. 180 gender-sensitive, disability and

environmentally-friendly sanitation facilities constructed/rehabilitated in schools and health centres/CHPS compounds

6. 120 schools with sanitation facilities 7. One model for scaling up small town

sanitation developed 8. One strategy for scaling up school

sanitation developed 9. 180 water facilities provided in schools

and health centres/CHPS compounds 10. 50,000 household latrines constructed

1. 204,655 additional people living in household that use improved sanitation – NOT ACHIEVED

2. 411 rural communities and 34 smalls towns decelerated ODF (a total of 445 communities) –ACHIEVED

3. No ODF districts – NOT ACHIEVED 4. 10 districts with evidence-based

ODF plans – ACHIEVED 5. 189 gender-sensitive, disability and

environmentally-friendly sanitation facilities constructed/rehabilitated in schools and health centres/CHPS compounds. – ACHIEVED

6. 93 schools with gender-sensitive sanitation facilities constructed and 42 rehabilitated, 116 WASH facilities in health centres (58 sanitation facilities and 58 water points)– ACHIEVED

7. One model for scaling up small town sanitation developed– ACHIEVED

8. One model for scaling up small town sanitation developed – ACHIEVED

9. 162 water facilities provided in schools and health centres/CHPS compounds – PARTILLY ACHIEVED

10. 17,488 household latrines constructed -– NOT ACHIEVED

Adoption of good hygiene practices for improved health (including maternal, new-born and child health).

Increased proportion of people washing their hands with soap in schools, health centres and at home. Increased knowledge of community around sanitation hygiene and safe water. Increased number of communities with ODF status

1. 500,000 aware of safe hygiene practices 2. 500,000 reached with hygiene messages 3. 500,000 additional people practising

HWWS (including children and women) 4. 160 schools with hygiene promotion

programmes in place 5. 16,000 school-going girls receiving

MHM training/support within the school 6. 16,000 reached with messages on

menstrual hygiene management 7. 16,000 and 16,000 school girls and boys

reached with hygiene messages including MHM

8. 80 health institutions engaged in participatory hygiene promotion methodologies

9. Two reviewed curricular for health training institutions

10. Two partnerships established with health training institutions

1. 432,000 aware of safe hygiene practices – NOT ACHIEVED

2. 432,000 reached with hygiene messages – NOT ACHIEVED

3. 204,655 additional people practising HWWS – NOT ACHIEVED

4. 130 schools with hygiene promotion programmes in place – NOT ACHIEVED

5. 20,066 school-going girls receiving MHM training/support within the school –ACHIEVED

6. 20,066 reached with messages on menstrual hygiene management –ACHIEVED

7. 42890 school girls and boys reached with hygiene messages including MHM - ACHIEVED



11. 80 health centres with staff (number of males and number of females) trained for promoting hygiene

12. Two reviewed curricula for health training institutions

8. 84 health institutions engaged in participatory hygiene promotion methodologies –ACHIEVED

9. Two reviewed curricular for health training institutions - ACHIEVED

10. Partnership established with Nursing Training School in Tamale and the School of Hygiene also in Tamale –ACHIEVED

11. 60 Health centres with 1,150 staff - 772 Males and 378 Females trained– NOT ACHIEVED

12. Two reviewed curricula for health training institutions – ACHIEVED

Increased capacity at the community, district and regional levels to support sanitation service delivery and engage the private sector in sanitation market.

More accountable and gender responsive government departments deliver WASH services Private sector involved in the WASH service delivery Increased understanding of WASH service implementation in Ghana

1. 10 districts adopting and using accountability mechanisms (BaSIS…etc. to be agreed with IPs)

2. 10 districts implementing framework for PPP in sanitation.

3. 200 private sector businesses engaged in sanitation services (30% women managed businesses)

4. 10 districts that have accountability systems in place (Review body, coordinating structures etc.)

5. 10 districts implementing demand-responsive approaches (e.g. Funds set aside for sanitation)

6. 10 districts with functional information systems in place.

7. 10 districts with gender disaggregated data and evidence available in districts and region.

8. 36 coordination meetings held 9. 10 review meetings held 10. 10 districts with district-level framework

for private sector engagement in place 11. 200 sanitation-related private sector

business mobilized 12. 10 MoUs signed with participating

districts 13. 10 district level results-based plans for

sanitation 14. 20 staff (number of males and number

of females) with skills for budget and results monitoring

15. Database in place to capture sector-specific sex disaggregated data for WASH and health

16. 10 districts with Monitoring formats developed and used by district staff

17. Six lessons learning platforms strengthen

18. 10 districts equipped with skills for engaging private sector

1. 14 districts adopting and using accountability mechanisms – ACHIEVED

2. 14 districts implementing framework for PPP in sanitation – ACHIEVED

3. 240 private sector businesses engaged in sanitation services (30% women managed businesses) – ACHIEVED but no data on women managed businesses

4. 14 districts that have accountability systems in place – ACHIEVED

5. 14 districts implementing demand-responsive approaches (e.g. Funds set aside for sanitation) – ACHIEVED

6. 13 districts with functional information systems in place – ACHIEVED

7. All target districts have data on ODF communities and disaggregated information on populations– ACHIEVED

8. 113 coordination meetings held – ACHIEVED

9. 10 review meetings held – ACHIEVED

10. 14 districts with district-level framework for private sector engagement in place-– ACHIEVED

11. 240 sanitation-related private sector business mobilized – ACHIEVED

12. 10 MoUs signed with participating districts – ACHIEVED

13. 14 district level results-based plans for sanitation – ACHIEVED

14. 67 staff (number of males and number of females) with skills for budget and results monitoring – ACHIEVED



19. 10 districts with frameworks in place for regulating private sector involvement in sanitation

20. 10 districts with private sector actors (males and females) trained on sanitation promotion

21. 50,000 household latrines constructed

15. Database in place to capture sector-specific sex disaggregated data for WASH and health – ACHIEVED

16. 14 districts with Monitoring formats developed and used by district staff – ACHIEVED

17. Two lessons learning platforms strengthen – NOT ACHIEVED

18. 14 districts equipped with skills for engaging private sector – ACHIEVED

19. No district with frameworks in place for regulating private sector involvement in sanitation – NOT ACHIEVED

20. 14 districts with private sector actors (males and females) trained on sanitation promotion – ACHIEVED

21. 17,488 household latrines constructed - NOT ACHIEVED

Source: Ipsos MORI analysis of the ToC and KPM database

This annex contains details of the household survey and observation tools used for the endline survey.

The household survey questionnaire was set out in the Inception Report. It is based on the baseline questionnaire for the Accelerated programme for comparability between baseline and end-line. Attention has also been paid to the UNICEF Ghana WASH Standard Survey as well as the journal article by Khan et al (2017).48 Additional questions have been added to ensure the survey results will address the evaluation matrix requirements (particularly to ensure knowledge of WASH is reflected), and some questions were removed where these were not required by the evaluation matrix. The questionnaire was further refined following piloting during the enumerator training, which was delivered by Ipsos Ghana and FCG Sweden and attended by GAC.

Data from the endline household survey were also recoded into variables that are aligned with both the MDG and SDG WASH indicators. These are as follows:

Access to clean water – MDG/JMP indicators: Improved sources of water that are piped into the dwelling Other improved sources Unimproved sources (including surface water)

48 Khan SM, Bain RES, Lunze K, Unalan T, Beshanski-Pederson B, Slaymaker T, et al. (2017). Optimizing household survey methods to monitor the Sustainable Development Goals target 6.1 and 6.2 on drinking water, sanitation and hygiene: A mixed-methods field-test in Belize. PLoS OONE 12 (12): e0189089. https://doi.org/10.1371/journal.pone.0189089

Annex 5: Household survey



Access to clean water – SDG indicators: Basic: Improved sources of water that can be collected within a 30-minute round trip Limited: Improved sources of water that are more than a 30-minute round trip away. Unimproved Surface water

Sanitation SDG indicators:

At least basic: An improved sanitation facility that is not shared between households: Limited: An otherwise improved sanitation facility that is shared between two or more households Unimproved: An unimproved sanitation facility Open defecation

Survey questionnaire

UNICEF Ghana WASH

Household Survey Questionnaire 2019 “Good morning / afternoon / evening. My name is …… from Ipsos Ghana, the research organisation. We're conducting a survey about daily life in this area, and I would like to ask you some questions about the kinds of activities you do on a day-to-day basis. I may also ask you to show me some of the places you regularly go, which are not far away from your dwelling. I would be grateful if you could spare around 30 minutes of your time to answer the questions.

I would like to assure you that all the information we collect will be kept in the strictest confidence. It will not be possible to identify any particular individual in the results.”

Geo-coordinate of the Household Dwelling Lat:

Long:

H/No.

Name of the Household Head

Interviewer’s Name

Interview Date [automatic]

Time of Interview Starting Time: [automatic] Finish Time: [automatic]

Region Northern Region

Central Region

Upper East Region



Upper West Region

Volta Region

1. District/ Municipal NOTE: District/Municipal Codes:

( ) Bunkprugu Yunyoo

( ) Central Gonja

( ) Chereponi

( ) East Gonja

( ) East Mamprusi

( ) Gushegu

( ) Karaga

( ) Kpandai

( ) Nanumba North

( ) Nanumba South

( ) Saboba

( ) Tatale Sanguli

( ) West Mamprusi

( ) Yendi Municipal

( ) Assin South

( ) Kumbungu

( ) Mion

( ) Zabzugu

( ) Builsa North

( ) Garu Tempane

( ) Tempane

( ) Daffiama Bussie Issa

( ) Lawra

( ) Wa West

( ) Adaklu

( ) Keta

( ) Kpando

( ) Krachi East

( ) North Dayi



Program Type ( ) Accelerated

( ) Enhanced

2. Community/Zonal/Sub-District Council [Mark on areas provided]

Section A: Demographics

How many members are there in this household who are 18 years and above? Please include all family members who live in this household permanently and eat from the same pot as you. Do not include guests who have come to live temporarily or servants.

Can you tell me the names of all the members of the household who are of 18 years and above? Please start from the oldest member above 18 years.

INTERVIEWER LIST HOUSEHOLD MEMBERS WHO ARE 18 YEARS AND ABOVE.

Name Age Gender Is this person in the household now

1 Yes/No

2 Yes/No

3 Yes/No

3. Sex of the Respondent [1] Male

[2] Female

4. Age of the respondent [1] Less than 18 [CLOSE THE INTERVIEW]

[2] 18 - 29

[3] 30 -39

[4] 40 - 49

[5] 50 - 60

[6] Above 60

5. Are you the Head of this household?

[1] Yes [SKIP TO Q9]

[2] No

6. If No, how are you related to the household head?

[1] Spouse

[2] Parent

[3] Sibling



SOCIO-DEMOGRAPHIC CHARACTERISTICS OF HEAD OF HOUSEHOLD

(Respondent, if not head of household could answer on behalf of head of household)

[4] Child

[5] Other (Specify)…………………………

7. Age

[1] Less than 18

[2] 18 - 29

[3] 30 - 39

[4] 40 - 49

[5] 50 - 60

[6] Above 60

8. Sex

[1] Male

[2] Female

9. Marital Status [1] Single/Never married

[2] Informal/consensual

[3] Married

[4] Separated

[5] Divorced

[6] Widowed

10. Religion [1] Christian

[2] Muslim

[3] Traditional Religion

[4] No Religion

[5] Other (Specify)

11. Highest Educational Level

1 Primary 2 Middle 3 JSS/JHS 4 Secondary/O- or A-level 5 Vocational 6 Higher/Tertiary/ University 7 Don’t Know

8 No education



Section B: Size and age/sex distribution of the household

12. Occupation [1] Trader (Market/street/shop)

[2] Farmer

[3] Fisherman/Fishmonger

[4] Artisan (Carp. Mason, Plumb. Etc.)

[5] Processor

[6] Driver/Mechanic

[7] Dressmaker/Tailor

[8] Food vender/caterer

[9] Student

[10] Salaried worker

[11] Contract/casual workers

[12] Other (Specify)

13. Type of dwelling [1] Compound house

[2] Detached

[3] Semi-detached

[4] Flats

[5] Other (specify)…………………………

14. Type of main material of the outer walls of dwelling

SINGLE RESPONSE

Natural walls (1) No walls (2) Cane / Palm / Trunks (3) Dirt/earth/mud/mud bricks

Rudimentary walls (4) Bamboo with mud (5) Stone with mud (6) Plywood (7) Cardboard (8) Reused wood

Finished walls (9) Cement (10) Stone with lime / cement (11) Bricks



16. What is the household size?……………………………………..

(A household consists of a person or a group of persons, who live together in the same house or compound, share the same house-keeping arrangements and are catered for as one unit. Source: 2010 PHC in Ghana)

Q16b. Does your household have kids under 5 years?

17. Age/Sex distribution of the household members (The number, including the head of the household):

18. What is the average total household monthly income (in GHS)?

SINGLE RESPONSE, BE SURE TO PROBE THAT THIS IS IN NEW GHS, NOT THE OLD GHS

[1] <100

[2] 100 -300

[3] 301 - 500

[4] 501 - 1000

[5] 1001 - 2000

[6] 2000+

[7] Don’t Know

(12) Cement blocks (13) Wood planks (14) Slates/asbestos

15. What is the present tenancy arrangement of this household?

SINGLE RESPONSE

[1] Owner Occupied

[2] Renting

[3] Rent free

[4] Perching

[5] Other (Specify)……………………………………………

Sex/Age Under 5 years 5 - 18 years 19 - 60 years Above 60 years Total

Male

Female

Total



19. Is there someone with a physical impairment living in this household? (PROBE: Person with problem with upper and/or lower limbs or visually impaired. Please note that this can be an elderly person as well.)

[1] Yes

[2] No

SECTION C: Water

READ OUT: Now I will ask you about the water you use in your household

19. Which of the following are the main sources of drinking water and water for domestic use (e.g. cooking, washing, bathing, hand-washing) for members of your household?

SINGLE RESPONSE PER EACH SEASON

READ OUT What is your main source of water for drinking? (tick one)

What is your main source of water for domestic use? (tick one)

IMPROVED

Rainy Season

Dry Season

Rainy Season

Dry Season

01 Pipe-borne inside dwelling

02 Pipe-borne outside dwelling

03 Public tap/Standpipe

04 Tube well/Borehole/Pump

05 Protected dug well

06 Protected spring

07 Rainwater harvesting

08 Bottled water/Sachet water

UNIMPROVED

09 Unprotected dug well

10 Unprotected spring

11 Vendor provided water

12 Cart with small tank/drum



13 Bottled water (if the secondary source used by the household for cooking and personal hygiene is unimproved)

14 Tanker truck

15 Surface water - River/ stream, Dam, lake, pond, canal, irrigation channel

20. On average, how much time does your household spend collecting drinking water each day

(including travelling, queueing and returning)? RECORD IN MINUTES, CODE 999 FOR DON’T KNOW

a. …in the dry season? _ _ _ minutes

b. …in the rainy season? __ _ _ minutes

SECTION D: Sanitation

READ OUT: Now I would like to ask you about the sanitation facility you mostly use in your household 21. What type of toilet facility do household members primarily use? (select one)

READ OUT, SINGLE RESPONSE

Private (Exclusive use by the household)

Shared (Used by all Households in the compound)

Public (It is used by the public)

No facility/Bush/Field Skip to below

22. What kind of toilet facility do members of your household usually use? READ; OUT, SINGLE RESPONSE

IMPROVED

- Connection to a public sewer 1

- Connection to a septic tank (e.g. water closet with septic tank) 2

- Pour flush toilet to pit latrine or sewer (no water seal) 3

- Pour flush toilet to pit latrine or sewer (with water seal) 3

- Access to hygienic pit toilet (KVIP, VIP, basic clean pit latrine, Biofil digester/ micro flush toilet, urine diverting dry toilet)

4

UNIMPROVED

- Improved toilet facility shared by households 5



- Improved toilet facility opens to public 6

- Open pit toilet or pour flush into open pit or drain 7

- Bucket toilet 8

23. Which of the following features does your toilet have? (Tick all that apply). READ OUT, MULTIPLE RESPONSE

[1] Lighting

[2] Segregation (a private space where no one can see who is inside)

[3] Locks

24. When was this latrine constructed or installed? RECORD MONTH AND YEAR. CODE 999 FOR DON’T KNOW.

Select month

1 Select year

2 Don’t Know

3

25. [IF THERE IS A DISABLED PERSON IN THE HH, CODE 1 AT Q19] You said earlier that there is a person with a physical impairment in the household, can this person use the toilet?

(Is the toilet disability friendly: Access ramp, one cubicle provided for people with disability, No step at the entrance, Railings at the sides and back of the cubicle, Water at close distance)

[1] Yes [2] No

Sub-section: Households with Shared Toilet Only (Used by all Households in the compound): ASK IF CODE 2 AT Q21

26. How many households in total use this toilet facility? RECORD NUMBER, CODE 999 FOR DON’T KNOW

………………………………………………………………………

27. Would you be willing to pay to use a private toilet? [1] Yes

[2] No

Sub-section: Households who Use Public Toilet Only: ASK IF CODE 3 AT Q21

28. What is the main reason for not having a private household toilet?



DO NOT READ OUT, SINGLE RESPONSE

Landlord did not provide 1

No space 2

Too expensive 3

Landlord does not allow use by others 4.

Socio-cultural reason 5

Other: specify 6

Don’t know

29. How far do you walk to the public toilet to defecate? DO NOT READ OUT, SINGLE RESPONSE

Up to 50 metres 1 50 -100 metres 2 100-200 metres 3 200-500 metres 4 More than 500 meters 5 Don’t know 6

30. On the average, how many times do you use the public toilet facility in a day?


[1] Once a day

[2] Twice a day

[3] Thrice a day

[4] Other (specify)……………………………………………….

31. How much does your household spend in total a day using the public toilet facility? WRITE IN CEDIS, CODE 999 FOR DON’T KNOW

[1] No charge [2] Less than GHS1 [3] GHS1-GHS2 [4] GHS2.1-GHS5 [5] GHS5.1-GHS8 [6] More than GHS8.1

[7] Don’t know

32. Do all members of the household use the public toilet, [excluding children under five]? [1] Yes [2] No



33. Would you be willing to pay to use a private toilet? [1] Yes [2] No

34. Are there any times when you cannot use your main latrine? [1] Yes [2] No

ASK THOSE ANSWERING CODE 1 ABOVE

35. When are you unable to use your main latrine? Probe: after first response: Are there any other times that you are unable to use your latrine?

DO NOT READ OUT. MULTICODE.

36. Is there anyone in the household who does not use the latrine? DO NOT READ OUT. SINGLE CODE.

[1] Yes [2] No

ASK IF CODE 1 ABOVE

37. Why do they not use it? Probe: Are there any other reasons why they do not use the latrine? DO NOT READ OUT. MULTICODE EXCEPT CODE 7. OBSERVATION

At night 1 During the day 2 During the rainy season 3 When flooding occurs 4 When surge tide occurs 5 Other (specify) 6

Latrine is too far for them to reach it 1

Route to the latrine is too difficult for them to use 2

Access into the latrine is too difficult 3

Not enough space inside the latrine 4

Not enough support in the latrine 5

Not acceptable for men and women in the household to use the same latrine 6

Other (specify) 7

Don’t know 8



Sub-section: Households with Private Toilet and shared inside the compound Only: ASK IF

CODE 1 AND 2 AT Q21

READ OUT: I would now like to see the sanitation facility you mainly use in your household, Can you show it to me please?

38. Enumerator to observe toilet facility.

[1] Observed Not observed (give reasons) …………………………….

[2] Not in dwelling / plot / yard [3] No permission to see [4] Other reason…………………………………………….

39. If allowed to observe, which of the following did you notice?

MULTIPLE RESPONSE (Tick as many as applicable) [1] Used toilet papers scattered around [2] Presence of flies [3] Presence of bad odour [4] Faecal matter present on the slab [5] Urine [6] Other (specify)…………………………………………. [7] None of the above (i.e., the facility is clean)

40. Tick where appropriate?

• Evidence of use? • Does the toilet have male and female segregation features? • Are the doors lockable? • Do they provide adequate privacy? • Is the path to the toilets well-lit and safe? • Is the toilet facility disability-friendly? (Access ramp, one cubicle provided for people with disability,

No step at the entrance, Railings at the sides and back of the cubicle, Water at close distance) • How many toilet cubicles have you observed (for women/men/communal/functional?) WRITE IN

NUMBER

41. Maintenance/repair

Latrine appears in good working order (nothing broken, not full) 1

Latrine is not very well maintained (some things are broken) 2

Latrine is in poor condition (broken down, full) 3

Sub-section: Households who Use Open defecation (Free Range) Only: ASK IF CODE 4 AT Q21

42. If you do not use any facility, ‘open defecation/free range”, what is your main reason?



DO NOT READ OUT, SINGLE RESPONSE [1] Socio-cultural reasons: (Please explain) ……………………………

[2] No other option is available: (No toilet in the house and no public toilet)

[3] Household facility inadequate

[4] The public toilet is unkempt

[5] Simply, prefer open defecation/free range”

[6] Can’t afford public toilet

[7] Other (specify)……………………………………………………….

43. Would you be willing to pay to use a private toilet? [1] Yes [2] No

SECTION E: Sanitation and health of children

ASK IF Q17 HAS KIDS UNDER THE AGE OF 5: (Enumerators should administer these questions to households with children under five years only.)

44. Where do children under five (5) years defecate? DO NOT READ OUT, SINGLE RESPONSE

[1] Use the household private toilet (Skip to next section E)

[2] Use the shared toilet in the house (Skip to next section E)

[3] Use the public toilet (Skip to next section E)

[4] Chamber pot

[5] In the open

[6] Diapers

[7] Other (specify)…………………………………………………….

45. How do you dispose of the children’s excreta if they do not use the toilet facility? (Tick based on answer of respondent) DO NOT READ OUT, SINGLE RESPONSE

Put/rinsed into toilet or latrine 1

Put/rinsed into drain or ditch 2

Thrown into garbage 3

Buried 4



Left in the open 5

Other (specify)…………………… 6

46. How many times per month do children under 5 suffer from diarrhoea?’ (Diarrhoea: more than 3

loose stools within a 24-hour period) RECORD NUMBER OF TIMES PER MONTH, CODE 999 FOR DON’T KNOW …………………………………………………

46a. Do you have children in the household who go to school?

Yes 1 No 2

46b. As far as you know, do they have good latrines/toilets at school(s) the children go to?

READ OUT, SINGLE CODE Yes, for both boys and girls 1 Yes, but only for boys 2 Yes, but only for girls 3 No, they do not have good latrines 4 Don’t know 5

READ OUT: NOW I WANT TO ASK YOU ABOUT HYGIENE IN YOUR HOUSEHOLD

47. What do you use to wash your hands with? DO NOT READ OUT. SINGLE CODE.

Water and soap 1

Water and ash/soap substitute 2

Water only 3

Other (specify)…………………………… 6

48. Where do you usually wash your hands?

DO NOT READ OUT. SINGLE CODE.

49. Did you wash your hands with soap under running water yesterday? [1] Yes [2] No (Skip to next section)

Fixed facility (sink/tap) in the HH 1 Fixed facility (sink/tap) in the yard 2 Fixed facility (sink/tap) in a public place 3 Mobile facility (bucket/jug) in the HH 4 Mobile facility (bucket/jug) in the yard 5 Mobile facility (bucket/jug) in a public place 6 No handwashing place 7

SECTION F: Handwashing practices



50. If “Yes” to above, when? (Please, do not prompt them. Let them come out with their answers,

then you check their answers in the table)

DO NOT READ OUT, MULTIPLE RESPONSE

51. When do you wash your hands with soap under running

water? (Please, do not prompt them. Let them come out with their answers, then you check their answers in the table)

DO NOT READ OUT, MULTIPLE RESPONSE

Activity Adults

Before eating

After eating

After going to the toilet

After disposing stools of a child < 5 years

After cleaning a baby

Before food preparation

Before feeding a child

After returning from farm

After returning from social gathering (Church/Mosque, Funerals, etc.)

Other (specify) ………………………………

Activity Adults

Before eating

After eating

After going to the toilet

After disposing stools of a child < 5 years

After cleaning a baby

Before food preparation

Before feeding a child

After returning from farm

After returning from social gathering (Church/Mosque, Funerals, etc.)

Other (specify) ……………………………



52. What is the source of water used to wash your hands? (Please tick appropriately in the tick list below)

READ OUT, MULTIPLE RESPONSE

IMPROVED Tick if yes

01 Pipe-borne inside dwelling

02 Pipe-borne outside dwelling

03 Public tap/Standpipe

04 Tube well/Borehole/Pump

05 Protected dug well

06 Protected spring

07 Rainwater harvesting

08 Bottled water/Sachet water

UNIMPROVED

09 Unprotected dug well

10 Unprotected spring

11 Vendor provided water

12 Cart with small tank/drum

13 Bottled water (if the secondary source used by the household for cooking and personal hygiene is unimproved)

14 Tanker truck

15 Surface water - River/ stream, Dam, lake, pond, canal, irrigation channel

53. Now I will read out some statements. Please tell me for each one whether you strongly agree, tend to agree, neither agree nor disagree, tend to disagree or strongly disagree… READ OUT. SINGLE CODE

Strongly agree

Tend to agree

Neither agree nor disagree

Tend to disagree

Strongly disagree

Don’t know

Hands that look clean can have germs on them. 1 2 3 4 5 6



Washing hands with soap kills more germs than water alone 1 2 3 4 5 6 Washing hands with water prevents diseases, it’s not necessary to use soap as well. 1 2 3 4 5 6

OBSERVATION

READ OUT: Now I would like to see the area where you wash your hands, please can you show it to me?

54. Enumerators should verify the availability of hand-washing facilities at a designated point. (take running tap/pump as water) SINGLE RESPONSE

Only water 1

Water and soap/ash 2

Only soap/ash 3

Mud/sand and water 4

Only mud/sand 5

No water/soap/ash 6

Other Specify ____________ 7

55. Enumerators should verify how the hand-washing is practised.

Under running water (the use of cup of water, running tap/pump, etc.) 1

Not under running water (the use of basin of water etc.) 2

55aa. IF THERE IS NO SOAP/ASH AT THE HANDWASHING FACILITY, ASK: Do you have soap/ash in the household?

Soap was kept in the household but not at the handwashing facility 1

There is no soap in the household 2



SECTION G: MHM

ONLY ASK WOMEN IN THE HOUSEHOLD

READ OUT: NOW I WOULD LIKE TO ASK YOU ABOUT MENSTURAL HYGIENE

56. When did your last menstrual period start? DO NOT READ OUT, SINGLE RESPONSE

Within a week 1

Within two weeks 2

Within three weeks 3

A month ago 4

Currently pregnant / nursing an infant 5

Menopause/does not menstruate (skip to SECTION I) 6

Don’t know 7

Refuse 8

57. Did you have a private place to wash yourself when needed during your last menstrual period? READ OUT, SINGLE RESPONSE

Yes, in the toilet or bathroom 1

Yes, elsewhere 2

No 3

DK 88

58. Did you have access to materials when needed for managing your last menstrual period?

READ OUT, SINGLE RESPONSE

Yes, (disposable after each use) 1

Yes, (both disposable and re-usable) 2

Yes, (reusable) (SKIP to Q61) 3



59. In what way do you manage your menstrual cycle when it occurs? Use of: DO NOT READ OUT, SINGLE RESPONSE

Sanitary pad 1

Cotton wool 2

Old clothes 3

Toilet roll 4

Red flag (traditional pad) 5

Other (specify)………………………….

6

60. How do you usually dispose of materials after use? DO NOT READ OUT, SINGLE RESPONSE

In a special bin 1

In a bin with other household waste 2

In the latrine 3

Burning 4

Burying 5

In a field, bush, water body, beach or other open space 6

Other (specify)________ 7

DK 88

61. Due to your last menstruation, were there any social activities, school or work days that you did not attend?

Yes 1

No 2

No access to materials (SKIP to Q61) 4

No, do not use materials (SKIP to Q61) 5

DK (SKIP to Q61) 88



Don’t know 88

Refuse 99

SECTION I: Faecal sludge management (skip this section if code 4 No facility/Bush is selected at Q21)

READ OUT: Now I want to ask you how you dispose of sewage 62. Where does sewage from your toilet(s) go?


Directly to environment (piped to drain/field/yard) (Skip to SECTION J) 1

Leach pit (pit latrine, single holding tank, not sealed, no effluent pipe) 2

Septic tank (Two-chamber tanks with vent and effluent pipes) 3

Communal tanks 4

Treatment plant (Skip to SECTION J) 5

Other specify _______________

Don’t know (Skip to SECTION J) 88

63. Does the septic tank/leach pit/communal tank have a soakaway?

[1] Yes

[2] No

64. “Has your sanitation facility ever been emptied? [1] Yes

[2] No

65. How many times has your sanitation facility been emptied in the last 3 years? No. of times…………………. (if “none”, Go to next section) RECORD NUMBER OF TIMES, CODE 999 FOR DON’T KNOW

66. When was the last time your sanitation facility was emptied? How many months/years ago?

RECORD NUMBER OF MONTHS AGO, CODE 999 FOR DON’T KNOW

Months 1



67. How was the toilet facility emptied? Was it emptied by members of your household or by a public or

private service provider? READ OUT, SINGLE CODE

Mechanically (cesspool truck with pump or tank on truck with portable pump) 1

Manually (use of buckets, shovels etc.) 2

Other 3

DK 88

68. Who emptied the septic tank? READ OUT, SINGLE CODE

Self or someone in the household 1

Private cesspool truck operators and individuals 2

District/Municipal/Metropolitan Assembly 3

Institutional cesspool truck providers 4

Other service providers 5

69. Which of the following safety equipment did you/they use while emptying the septic tank? (Tick all that apply) READ OUT

Masks 1

Gloves 2

Gum Boots 3

Safety Uniform 4

No safety gadgets 5

70. How much did it cost to empty your septic tank? In Ghana Cedis....... WRITE IN COST, CODE 999 FOR DON’T KNOW

71. Were you satisfied with the service? [1] Yes

[2] No



72. Where does sludge get disposed? READ OUT

In open space near house 1

In covered pit near house 2

Into river/stream/drain 3

Transported away from immediate surroundings 4

Don’t know 5

73. What discourages you from regularly emptying your septic tank? (multiple answers possible) READ OUT

No service available 1

Unaware this is required 2

Tank is not full 3

Can’t afford 4

Others (specify) 5

Section J: Gender tasks

READ OUT: Now I want to ask you about some of the chores in the household

74. Tasks performed by gender READ OUT, MULTICODE

Activities at household level Men Women Boys Girls

Water

Who fetches water for family consumption?

Who stores and manages water?

Sanitation

Who cleans the toilet at home?

(Do not ask if code 4 No facility/Bush is selected at Q21)



Section L: WASH intervention

75. Has there been a Water, Sanitation and Hygiene project in your community in the past few years? READ OUT

Yes – on-going 1

Yes, 1 year ago 2

Yes, two years ago 3

Yes, three years ago, 4

Yes, over three years ago 5

Don’t know 6 (Skip Q78)

76. What did the project do? OPEN ENDED, RECORD

Don’t Know

77. Who was it that did the project? WRITE IN ORGANISATION

Don’t Know

78. READ OUT: I have now asked you all the questions I wanted to ask you. Do you have any questions for me?

Thank and close

Visual references for observation

ASK ALL Please show me the latrine you use most often. a. Thinking about the latrine you use most often, what type of latrine is it?

IF THE RESPONDENT DOES NOT KNOW INITIALLY, PLEASE USE THE SUPPORTING INFORMATION AND PHOTO SHEET (AND OBSERVATION IF NECESSARY) TO IDENTIFY THE LATRINE TYPE THEY USE MOST OFTEN. SINGLE CODE



1. Water seal pit latrine

Latrine with slab, pit and superstructure made up of conventional / local material such as cement, sand, brick chips, Mild Steel (MS) Wire, Corrugated Iron (CI) sheet or bricks, plastic or ceramics pan, bamboo, etc. The concrete slab is placed over a pit lined with a Reinforced Cement Concrete or bamboo ring, but it can sometimes be without a lining. The pit is covered with a concrete slab with pan and water seal trap for confining and disconnecting human contact to the human faeces. The water seal trap prevents odours and flies entering into the pit.



2. Ventilated improved pit latrine (VIP)

1.

A VIP latrine differs from a traditional pit latrine by having a vent pipe that is covered with a fly screen/cap. The pit is covered with concrete slab with a drop hole and drop hole cover but without pan and water seal trap. Wind blowing across the top of the vent pipe creates a flow of air which draws out odours from the pit. As a result, fresh air is drawn into the pit through the drop hole and the superstructure is kept free of smells.

3. Pit latrine but water seal is broken OR there is no Water Seal

This is the same as the first type of latrine (1) but there is no water seal trap or the water seal is broken. Latrine with slab made up of conventional / local material such as cement, sand, brick chip, MS Wire, plastic or ceramics pan, etc. The concrete slab is placed over a pit lined with a Reinforced Cement Concrete or bamboo ring, but it can be without a lining. The pit is covered with a concrete slab but it is without a water seal trap or the water seal trap is broken. The absence of a water seal trap, or the fact that it is broken, means that the faeces is not disconnected from human contact and the latrine does not prevent odours or flies from coming back up the pipe.

4. Pit latrine with slab

The latrine consists of a pit covered with a slab which is made with materials such as wooden planks, bamboo matts or other similar materials. The material is strong enough to prevent collapsing and provides a safe segregation between the faeces and the person. There is no water seal trap, and the faeces falls straight into the pit below.



5. Pit latrine without slab

The latrine consists of a pit but is without a slab and therefore does not provide a safe segregation between the faeces and the person.

6. Offset Pit Latrine

Latrine with slab, pit, water seal trap and superstructure made up of conventional / local material such as cement, sand, brick chips, MS Wire, plastic or ceramics pan, Syphon/Trap seal, bamboo, etc. The key difference with the Offset Pit Latrine is that the pit is dug beside the elevated superstructure/ latrine chamber, and the faeces is transported via a pipe/tube into the pit next to the latrine chamber and thus disconnects the user from the faeces. The pit is covered with a Reinforced Cement Concrete cover/wooden plank/ bamboo mat and air vent pipe. The water seal trap prevents odours and flies entering into the pit.

7. Open pit/Hanging latrine

A defecation platform is constructed over a pond, lake, river, canal, ditch or other water source. The faeces falls straight into the water source below.



8. Use bucket and empty into latrine

Buckets are used for defecation and the excreta is later thrown into a pit dug away from the household.

9. Open defecation / no toilet facility (bush / field as a latrine)

No latrine is used – respondent defecates in field, bush or other similar area.

10. Other Code this if the latrine of the

household is not covered in the list above, and describe the latrine in as much detail as possible.



ASK ALL Please show me the latrine you use most often. d. What changes or adaptations, if any, have been made to your main latrine? INTERVIEWER NOTE: WE NEED TO RECORD INFORMATION ABOUT ANY CHANGES OR ADAPTATIONS THAT WERE MADE TO THE LATRINE SINCE IT WAS ORIGINALLY BUILT /INSTALLED. 1. Rope guide A rope is hung over the

latrine to support a person (especially with disabilities) in squatting and using of latrine.

2. Ramp Sloping path for the disabled and the weak to go into the latrine

3. Grab bars / handrails / rope

Bars or hand rails are installed over the latrine to support a person (especially with disabilities) in finding, squatting and using the latrine. This can also include grab bars or a rope spread horizontally towards the latrine to help someone who is blind to reach the latrine.



4. Seat English/India seat installed instead of the local arrangements.

5. More space inside a built latrine

More space has been built / created inside the latrine to make it easier to move around for those using it, including for those disabled.

6. Other (specify) Code this if the improvement is not mentioned in the list above, and specify by writing in what improvement has been made to the latrine with as much detail as possible.

7. No adaptations visible Code this is there are no visible adaptations made to the latrine to make it easier to use, including for those who are disabled.

14

ASK ALL Please show me the latrine you use most often. h. Maintenance/repair 1. Latrine appears in good working order (nothing broken, not full); Seat, Slab and the privacy walls are all are not broken

2. Latrine is not very well maintained; Any one of seat, slab or the walls are broken but still latrine is useable

2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12.

Broken seat but useable Broken slab but useable Broken walls but useable

14

3. Latrine is in poor condition (broken down, full); Latrine is not useable due to

broken seat, broken slab or broken walls 13. 14. 15. 16. 17. 18. 19. 20. 21. 22. 23.

Not useable due to broken seat Not useable due to broken slab Not useable due to broken walls

This annex provides more detail on the context for the evaluation, including the international and national context.

A.6.1 International context

The Millennium Development Goals and WASH

The Enhanced and Accelerated programmes commenced while the Millennium Development Goals (MDGs), were the overarching framework for international development cooperation. The MDGs, which established following the Millennium Summit of the United Nations in 2000, did not create an explicit goal for WASH, although it was included in as part of other targets, most notably target 7C; to halve, by 2015, the proportion of the population without sustainable access to safe drinking water and basic sanitation. While progress was made under the MDGs, notably to halve the proportion of people without sustainable access to safe drinking water, the target for sanitation was not met. Furthermore, critique49 has been levied at the MDGs that they

49 See for example the Water Supply and Sanitation Collaborative Council (WSSCC) “Post-2015 WASH Targets and Indicators”, available at https://www.wsscc.org/2015/02/20/post-2015-wash-targets-indicators-questions-answers/.

Annex 6: Programme context

https://www.wsscc.org/2015/02/20/post-2015-wash-targets-indicators-questions-answers/

14

were limited in scope and ambition; even if both targets for safe drinking water and sanitation had been reached, they would have left large numbers of people unserved. Further, the MDGs did not include hygiene or access to essential services outside households. Finally, the MDGs did not fully recognise the water sector’s central role in human rights, poverty reduction, inequality elimination, peace and justice, and the environment, and they did not account for inequality, accessibility, water safety, safe management of sanitation nor the sustainability of services.

The international community recognised that WASH needed to be given a greater focus in the post-2015 agenda, exemplified by the United Nations General Assembly Resolution 64/292, which on 28th July 2010 explicitly recognised the human right to water and sanitation.

The Sustainable Development Goals and WASH

As a result of the extensive and rigorous consultation process as part of designing the Sustainable Development Goals (SDGs), established by Resolution 70/1 of the United Nations General Assembly in 2015, WASH was given its own goal, SDG 6. There was broad consensus among experts and organisations working in the fields of WASH and human rights that the overall vision is universal access to safe drinking water, sanitation and hygiene, seeking to "Ensure availability and sustainable management of water and sanitation for all”50. SGD 6 has eight targets to be achieved by at least 2030, these are:

Achieve universal and equitable access to safe and affordable drinking water for all.

Achieve access to adequate and equitable sanitation and hygiene for all and end open defecation, paying special attention to the needs of women and girls and those in vulnerable situations.

Improve water quality by reducing pollution, eliminating dumping and minimizing release of hazardous chemicals and materials, halving the proportion of untreated wastewater and substantially increasing recycling and safe reuse globally.

Substantially increase water-use efficiency across all sectors and ensure sustainable withdrawals and supply of freshwater to address water scarcity and substantially reduce the number of people suffering from water scarcity.

Implement integrated water resources management at all levels, including through transboundary cooperation as appropriate.

Expand international cooperation and capacity-building support to developing countries in water- and sanitation-related activities and programmes, including water harvesting, desalination, water efficiency, wastewater treatment, recycling and reuse technologies.

50 UNDP” Goal 6: Clean water and sanitation", available at https://www.undp.org/content/undp/en/home/sustainable-development-goals/goal-6-clean-water-and-sanitation.html.



14

Support and strengthen the participation of local communities in improving water and sanitation management

Protect and restore water-related ecosystems, including mountains, forests, wetlands, rivers, aquifers and lakes (by 2020)51.

Review of SDG 6 at the 2018 High-Level Political Forum

At the annual High-Level Political Forum (HLPF) in July 2018, over two thousand representatives from governments, UN system and other organizations, civil society, NGOs and the private sector, took stock of progress on the SDGs and reviewed SDG 6 in depth.

The SDG 6 Synthesis Report on Water and Sanitation52 produced by UN-Water presents a review of the latest SDG 6 indicator data. This report has produced a baseline from which to measure future progress and has identified gaps in knowledge, capacity and resource availability. The baseline data illustrates that at current progress SDG 6 is not on track to be achieved by 2030. The progress of SDG 6 from the Sustainable Development Goals Report 2018 is as follows53:

In 2015, 29 per cent of the global population lacked safely managed drinking water supplies, and 61 per cent were without safely managed sanitation services. In 2015, 892 million people continued to practise open defecation.

In 2015, only 27 per cent of the population in LDCs had basic handwashing facilities.

Preliminary estimates from household data of 79 mostly high- and high-middle-income countries (excluding much of Africa and Asia) suggest that 59 per cent of all domestic wastewater is safely treated.

Since 2000, 1.4 billion people have gained access to basic sanitation, such as flush toilets or latrine with a slab which are not shared with other households. However, in 2015, 2.3 billion people still lacked a basic sanitation service. The data reveal pronounced disparities, with the poorest and those living in rural areas least likely to use even a basic service54.

“Safely managed” sanitation services represent a higher service level that takes into account the final disposal of excreta. In 2015, 2.9 billion people used a “safely managed” sanitation service, i.e. a basic facility where excreta are disposed in situ or treated off-site. A further 2 billion people used a “basic”

51 UNDP” Goal 6: Clean water and sanitation", available at https://www.undp.org/content/undp/en/home/sustainable-development-goals/goal-6-clean-water-and-sanitation.html. 52 More information about the report is available at http://www.unwater.org/publication_categories/sdg6-synthesis-report-2018-on-water-and-sanitation/. 53 SGD Knowledge Platform” Sustainable Development Goal 6”, available at https://sustainabledevelopment.un.org/sdg6 54 WHO/UNICEF JMP Progress on Drinking Water, Sanitation and Hygiene: 2017 Update and SDG baseline.



http://www.unwater.org/publication_categories/sdg6-synthesis-report-2018-on-water-and-sanitation/

http://www.unwater.org/publication_categories/sdg6-synthesis-report-2018-on-water-and-sanitation/

https://sustainabledevelopment.un.org/sdg6

14

service, i.e. an improved facility that is not shared with other households. The 600 million who shared improved sanitation facilities with other households count as a “limited’ service55.

Challenges for realisation of the SDGs

One of the key features of the SDGs is their interdependence, which necessitates integrated action in order to ensure that all SDGs advance together. However, the background notes of the review of SDG 6 from the HLPF identified a number of challenges which need to be addressed for the successful implementation of SDG 656:

Political engagement: Achieving SDG 6 will require profound evolution of actions among policymakers and decision makers.

Data gap: More and better standardised data are required for national, regional and global monitoring, shared in an accessible way.

Climate change: Climate change effects will need to be monitored and managed, primarily in the areas of water availability, water quality and frequency of extreme weather events (e.g. floods and droughts).

The financing gap: Funds need to be made available to meet the yet unmet needs, particularly among the poor and vulnerable who are unable to access services.

One of the key pieces of feedback of the reporting on SDG 6 has been the importance of sharing more lessons learned and best practices57. This includes the sharing of monitoring data (e.g. data collection, reporting, etc.) related to implementation of SDG 6, which could significantly assist organisations working towards realising SDG 6 by 2030.

A.6.2 WASH in Ghana

Background

Before the early 1990s, the then Ghana Water and Sewerage Corporation (GWSC) was responsible for both urban and rural water supply and sewerage services. However, due to an urban bias in service delivery, a new department, the Community Water and Sanitation Division (CWSD) was created within the GWSC to focus on WASH service delivery in rural communities and small towns.

55 Ibid HLPF “2018 HLPF Review of SDG implementation: SDG 6”, available at https://sustainabledevelopment.un.org/content/documents/195716.29_Formatted_2018_background_notes_SDG_6.pdf. 57 UN-Water” Executive Summary for UN-Water SDG 6 Public Dialogue Report”, available at http://www.unwater.org/publications/sdg-6-public-dialogue-report-executive-summary/

https://sustainabledevelopment.un.org/content/documents/195716.29_Formatted_2018_background_notes_SDG_6.pdf

http://www.unwater.org/publications/sdg-6-public-dialogue-report-executive-summary/

http://www.unwater.org/publications/sdg-6-public-dialogue-report-executive-summary/

14

Wider changes in government in the 1990s have impacted the institutional structure of the WASH sector. After the new constitution was adopted in 1992, the Government of Ghana undertook a series of general reforms, including enacting five major laws that affected the WASH sector, including:

The Local Government Act 462 of 1993, which formalised decentralization by establishing District assemblies as the highest political, planning, and decision-making body for the total socio-economic development of local areas (District Assembly areas-including metropolitan and municipal assemblies).

Under the Statutory Corporation Act 461 of 1993, GWSC was transformed into a public company, the Ghana Water Company Limited (GWCL), responsible for urban water supply by managing 81 water supply systems country-wide. Responsibility for sewer systems was transferred to District Assemblies58.

The Water Resources Commission Act 552 of 1996 created the Water Resources Commission (WRC) to

be responsible for integrated water resource management including permits for water abstractions. The Public Utilities and Regulatory Commission (PURC) Act 538 of 1997, made the PURC responsible

for the regulation of the electricity sector and urban water supply; including the review of requests for tariff adjustments, the monitoring of service quality and protection of consumers.

The Community Water and Sanitation Agency (CWSA) Act 564 of 1998 transformed the CWSD into an

autonomous agency. It made the CWSA responsible for supporting District Assemblies and local communities to plan, implement, manage, monitor and evaluate their water supply, sanitation and improved hygiene services in small towns and rural communities throughout the country. The CWSA primarily facilitates and coordinates the implementation of the National Community Water and Sanitation Programme (NCWSP). Under the NCWSP, three main objectives; safe water supply, hygiene promotion and improved sanitation are prioritized as the desired health outcomes59. The CWSA, managers of NCWSP, has a team of experts at the national and the regional levels.

Policies

The National Water Policy (2007), Environmental Sanitation Policy (revised 2010) and the Water Sector Strategic Development Plan (WSSDP) 2012-2025 and the National Environmental Sanitation Strategy and Action Plan (NESSAP) (2011), are the key policy and strategic documents that guide the small towns and rural communities WASH sector in Ghana. Other relevant strategic government documents and plans are outlined in the table below60.

Table 1: Key Relevant WASH documents

Water Sanitation and hygiene

National Water Policy (2007) Environmental Health and Sanitation Policy (2010)

National Integrated Water Resources Management Plan (IWRM) (2012)

Behaviour Change Communication Strategy (2009)

58 United Nations (2004) “Freshwater Country Profile: Ghana”; p.2 59 Community Water and sanitation Agency-CWSA (Aug 2004; “Strategic Investment Plan 2005-2015” p.15 • 60 The Government of Ghana, Ministry of Sanitation and water Resources; The 2017 Ghana Water, sanitation and Hygiene Sector Performance Report 2018

14

The National Riparian Buffer Zone Policy (2012)

The National Environmental Sanitation Strategy and Action Plan (NESSAP) (2011)

The National Community Water and Sanitation Strategy (2014)

The MDG Acceleration Framework (2011)

The Project Implementation Manual-NCWSP (2014)

The Rural Sanitation Model Strategy (2012)

The District Operational Manual-NCWSP (2014)

The Strategic Environmental Sanitation Investment Plan (SESSIP) (2012)

Water Sector Strategic Development Plan (WSSDP) (2014)

WASH in Schools National Minimum Standards (2016)

Drinking Water Quality Framework for Ghana (2015)

National Costed Strategy for WASH in Schools (2017)

In addition, the UNICEF Gender Action Plan 2018-2021 (GAP) highlights strategic gender priorities for UNICEF and is one of the key gender-related policy documents that will guide the evaluation process, alongside the GAC policy on gender equality and the Ghana National Gender Policy 2015.

The UNICEF GAP outlines UNICEF’s framework for empowering and improving the wellbeing of adolescent girls in development and humanitarian contexts and for achieving gender equality results across all areas of its programmatic work. It identifies eight result areas across two themes which constitute strategic gender priorities:

(1) Gender equality in life outcomes for girls and boys from 0-18.

(2) Gender equality in care and support for women and children.

Result no.8 – Gender responsive water, sanitation and hygiene systems – is of direct relevance to the UNICEF-GoG WASH programmes. This acknowledges that women and girls are disproportionately affected by lack of sanitation and hygiene services at home and in public settings, yet are also under-represented in the design and management of WASH systems and in the WASH sector more broadly. It identifies the adoption of market-based approaches as an opportunity to engage women and empower women economically.

Other relevant result areas include Equality in education for girls and boys which addresses gendered barriers to educational achievement, including lack of appropriate hygiene facilities, Quality and dignified maternal care which addresses a lack of water and sanitation facilities for maternal women and girls, Positive gender socialization for girls and boys, and gender equality in health systems and workforce.

The GAC also prioritizes five interlinked results to improve the specific wellbeing of adolescent girls. These include facilitating dignified menstrual health and hygiene, advancing secondary education, and promoting improved nutrition for girls.

To achieve these results and make UNICEF a more gender responsive organisation, the GAC outlines the importance of strengthening accountability mechanisms and improving use of data – priorities which have informed our evaluation criteria. These include a focus on desegregating data by sex, developing indicators

14

that measure gender equality, conducting gender analyses, and a focus on organisational and programmatic learning in relation to gender.

Institutions and stakeholders

WASH service delivery in small towns and rural communities is decentralized in Ghana, with national level institutions providing policy and monitoring frameworks, while local government and communities plan, implement, coordinate, monitor and evaluate services.61 The water supply, sanitation and hygiene (WASH) sub-sector involves several government institutions, departments and agencies, development partners, civil society organisations, private sector and the citizenry (women, men and children) at various levels of administration (national, regional and local) in the planning, coordination, management, implementation and monitoring and evaluation.

Since 2017, the newly created Ministry of Sanitation and Water Resources (MSWR) is the lead ministry responsible for the formulation of policies, strategies and plans for the WASH sector. These have been done in close collaboration with sector stakeholders such as other ministries and agencies, donors, civil society groups and the private sector. The MSWR is in dialogue with sector stakeholders to establish a National Sanitation Authority with a supporting fund to accelerate and regulate the implementation of sanitation policies of government. At the MSWR; there are two technical directorates: 1) Water Directorate (focuses on water and related issues, and 2) Environmental Health and Sanitation Directorate (focuses on hygiene and sanitation issues-until 2017, this Directorate was within the ministry of Local Government and Rural Development.

It is also important to note that there used to be WASH Sector Working Group (made of Government representatives from key related ministries, departments and agencies, Development Partners, Civil Society Groups (represented by CONIWAS and WATSAN Journalists Network) and the private sector.62

Moreover, the CWSA has started engaging sector stakeholders to reform the NCWSP to enable them become “leading public sector small towns and rural water services delivery organisation by transforming the Agency into a public utility organisation63. The year 2019 has been set as the end period for CWSA to facilitate small towns and rural communities WASH services. During this transitional period the Agency is managing 89 piped water systems country wide with about 624 professional staff. The future strategic focus of CWSA will be known and agreed after 2019.

Table 2 outlines the roles and responsibilities of key stakeholders involved in the WASH service delivery in Ghana.

Table 2: Key sector stakeholders

Institution Roles and responsibilities

61 The national Community water and Sanitation Programme (NCWSP) 62 Unfortunately, this sector coordination and engagement platform has been inactive for the last two years.

15

Ministry of Finance Provision of finance through Government annual budgets and other sources of funding to the sector

Ministry of Sanitation and Water Resources

In collaboration with sector stakeholders formulate sector policies, programmes and plans

Ministry of Local Government Formulate and coordinate policies and plans of metropolitan, municipal and District Assembles

National Development Planning Commission

Formulate and coordinate spatial and economic plans of Ghana

Ghana Standard Authority Sets national standards including that of water supply

Public Utility and Regulatory Commission

Regulation of economic and quality of service for urban water supply as well as electricity in the country

Food and Drugs Authority Regulation and License of bottle and “sachet” water

Environmental Protection Agency Environmental Management, including sanitation and pollution of all forms

Water Resources Commission Regulation and management of Ghana’s water resources (both surface and underground)

Ghana Water Company Limited Responsible for water supply in urban areas through management of 81 systems

Community Water and Sanitation Agency

Facilitates, through the Metropolitan, Municipal and District Assemblies, the provision of safe drinking water and related sanitation services to rural communities and small towns in Ghana.

Ghana Health Services/ Ministry of Health

Prevention, promotion and management of health services and WASH in health facilities

Ministry of Education/Ghana Education Services School Health Program Unit

In collaboration with stakeholder establish standards and guidelines for WASH services in schools

Civil society Organizations e.g. CONIWAS64, religious groups and network of Journalists in WASH

Advocacy and engagement in sector dialogues in planning, research, monitoring and implementation

Development Partners Support in sector funding and contribute in sector dialogues including policy, strategy and plans formulation, implementation monitoring and evaluation

Partner Organisation (contractors, consultants)

Provide services to support Assemblies/communities to have access to and use of sustainable water supply, sanitation and hygiene services. This includes capacity building, knowledge management, construction of facilities, supply of goods and services in general.

Assemblies and beneficiary communities

Are at the centre of WASH planning, management, implementation, and monitoring and evaluation.

64 CONIWAS is the Coalition of NGOs in Water and Sanitation sub-sector. It has a secretariat and is usually represented in sector dialogues.

15

Apart from the above-mentioned stakeholders involved in the WASH delivery in Ghana, there are a number of partners from bi-lateral, multi-lateral, NGOs (including faith-based groups) and private sector contributing to both technical and financial resources for the delivery of WASH services in Ghana. At the bi-lateral level, the Governments of Canada, Germany, Denmark, Australia, Japan, Netherlands, United Kingdom, Spain, France, and USA (through USAID) have made substantial contributions to the sector. Also at the multi-lateral level, the World Bank, UNICEF, AfDB, and the EU were the key supporters.

Some civil society groups such as WaterAid, IRC, Plan International Ghana, World Vision International, Safe Water Network, Global Communities and faith-based groups (the Catholic Church, Church of Christ, Seventh Day Adventist, Presbyterian Church, and some Islamic groups) have made significant contributions to accelerating small towns and rural communities’ access to safe water supply, improved sanitation and hygiene practices.

However, many donors and CSOs have moved out of the sector since Ghana achieved lower middle-income country status in 2012. Currently, the active donors in the sub-sector include the World Bank, UNICEF, Canada, the Netherlands, USAID (through Global Communities), the African Development Bank, WaterAid, World Vision International, Plan Ghana, Catholic Relief services, Church of Christ and the Presbyterian Church.

Other WASH programmes are currently active in Ghana and potentially in the regions where the Enhanced and Accelerated regions are operating. However, the evaluation team has not yet received the list of WASH programmes currently running in Ghana – which UNICEF Ghana will send – thus we can determine to what extent any of these are active in the same communities as the Enhanced and Accelerated programmes. Known WASH programmes operating in Ghana currently include: Five District Water Supply Scheme (STRABAG), Sanitation Services delivery Implemented by Water and Sanitation for the Urban Poor with funding from USAID, Ghana-Spain Debt Swap Development Programme (DSDP) etc. Annex 6 provides the list of WASH programmes currently running in Ghana that we have found through our desk research.

Progress in accessibility and use of sustainable WASH services

The government has set targets of achieving nationwide open defecation free (ODF) status by 2020 and universal sanitation and water access supported by good hygiene practices by 2025.65 Substantial progress has been made in accelerating small towns and rural communities’ access to and use of sustainable drinking water supply, but not basic sanitation services. As at the end of 201766, the national coverage rate for basic drinking water, sanitation and hygiene services in small towns and rural communities were 79%, 21% and 48% respectively. The coverage rates for the five programme regions were as follows; Northern Region 12%, Upper east Region 8%, Upper West Region 15%, Volta Region 14% and Central Region 19%. The coverage rates for the entire country is depicted below. Access to and use of improved sanitation services and improved hygiene practices for the small towns and rural communities where majority of the poor live is rather low country-wide. It is therefore critically important to increase access to basic sanitation at a national scale.

65 From the National Water Policy/Water Sector Strategic Development Plan and NESSAP 66 Ghana Statistical Services 2017/18; MICS 2017/18 Report

15

Figure 1: National and Regional Accessibility to, and use of, Basic WASH Services (2017)

Source: Ghana Statistical Service: MICS 2017/18 Almost two of every five children in Ghana (44%) attend schools without toilets, and three out of every five (62%) attend schools without water supply67. There are inequities in access across regions, with children in the Western Region about half as likely to have access to school toilets (30%) as those in the Upper East Region (57%) and about 50% per cent less likely to have access to water than those in the Central Region (45% compared to 67% respectively). Even though data are not available on school facilities for handwashing and menstrual hygiene management, the low access rates of water supply and latrines show that there are also likely to be low rates for both, thus negatively impacting adolescent girls’ school attendance due to non-availability of facilities.

Although there are no reliable data on WASH facilities in health care, limited studies68 indicate that lack of continuous water supply, sanitation and handwashing facilities adversely affect quality of services.

67 UNICEF Ghana; Country Programme 2018-2022. Draft Programme Strategy Note: WASH p.3 68 UNICEF Ghana, Country Programme 2018-2022, Draft Programme strategy note: WASH

0

20

40

60

80

100

120Co

vera

ge in

Per

cent

ages

National and regional accessibility

Basic Drinking Water Basic sanitation Basic Hygiene

15

Key challenges in the WASH Sector in Ghana

Many of the main challenges facing the Ghana WASH sector are analogous to the challenges facing the WASH sector globally, including inadequate political engagement and coordination, sometimes poor quality of monitoring data, and the financing gap and climate change.

An explanation for the low results in sanitation coverage in Ghana is that putting sanitation facilities in place has not kept up with rapid urbanisation, prompted by Ghana’s rapid economic growth in recent years. The majority of households without access to safe water and sanitation lack the upfront funds needed to invest in their own solutions. Consequently, those living in poverty often pay up to ten times more per litre for water service from private vendors than their middle-class counterparts connected to piped water services. These water costs can be reduced through investments in improved household water assets such as connections, rainwater harvesting equipment, wells, and latrines. Unfortunately, this requires up-front investments that, without access to financing, are unrealistic for most people living in poverty. A key factor is that the major share of investments for the sector in Ghana historically has gone to the water sector, while sanitation has received little political prioritisation.

The coordination between public bodies responsible for WASH and the many NGOs working in the sector in Ghana also could be improved in some cases. Despite efforts towards addressing and adapting to WASH challenges, including in response to climate change, there are still gaps to be addressed mainly because many programmes and projects lack proper coordination at the assembly level. Various initiatives, such as the Sanitation Challenge launched on World Toilet Day in 2015, has tried to improve coordination between NGOs and the Ghanaian Ministry responsible for sanitation, and aiming to stimulate local politicians to prioritise sanitation.

Inadequate monitoring is another major challenge for policy makers, particularly for WASH in Schools and health care facilities which remains largely unmonitored at the global level. In Ghana this there is particularly need for reliable data on the functionality and service levels of existing WASH facilities in schools and health care facilities. Consequently, this may impact on the perceived importance of these among policymakers.

A.6.3 Enhanced and Accelerated WASH programmes

• The scope of this evaluation includes two UNICEF Ghana WASH programmes: the Enhanced and Accelerated Sanitation Programmes.

Programme rationale

The GoG-UNICEF WASH Programme is the over-arching programme through which the Government of Ghana and UNICEF Ghana collaborate on WASH. The GoG-UNICEF WASH Programme was established to promote equitable and sustainable access to water and sanitation services, with a particular focus on improving WASH outcomes and practices in small towns and rural communities, schools, and health facilities. UNICEF Ghana also actively participates in sector dialogues, undertakes sector thematic research, and sector investments through government institutions (including District Assemblies), NGOs and the private sector. The programme’s

15

overarching objectives are to increase the use of safe water supply, to improve sanitation facilities, and promote hygiene-related behaviour change in Ghana.

To achieve these broad objectives, UNICEF Ghana, with financial support from Global Affairs Canada, implemented two distinct sub-projects: The Enhanced WASH services in Schools and Communities in Ghana Programme (the Enhanced programme) and the Accelerated Sanitation in Northern Ghana Programme (the Accelerated programme).

Several studies and activities informed the decision to implement this large-scale community-led total sanitation (CLTS)-based programme in Ghana’s most deprived regions. These included a situational analysis conducted as part of the Ghana United Nations Development Assistance Framework – an action plan for development in Ghana signed in 2011, and a UNICEF-led June 2009 evaluation of previous CLTS programmes in Ghana. The latter found that, with sufficient national-level support, the CLTS approach could provide both an effective and efficient solution to the key sanitation challenges that were persisting in rural areas of the country (see section 2.3.2 for more information on CLTS). Both programmes follow similar approaches but have different geographical targets, and therefore the CLTS approach has been adapted for application in small towns for the Accelerated programme.

The Enhanced Programme

The Enhanced programme, which started in March 2012, was originally planned to run for three years. In June 2015, however, following the success of the initial grant and the potential for its achievements to be sustainably extended to a further 570,000 people (initially 430,000 people). UNICEF Ghana received additional funding to support an extension of activities until December 2018. The programme was implemented in the five most deprived regions of Ghana: Northern, Upper East, Upper West, Volta, and Central regions, where it sought to achieve long-lasting improvements in the health and wellbeing of children in schools, and women and men in communities and small towns, by reducing inequitable access to WASH services and changing behaviours.

The Enhanced extension proposal identifies three specific objectives:

▪ improve access to, and use of, water, sanitation and hygiene services;

▪ improve water, sanitation and hygiene practices and behaviours;

▪ strengthen the enabling environment and enhance national systems and capacity.

Programme design and delivery

The design of the Enhanced and Accelerated programmes consists of both demand and supply-side interventions, which target five key groups: households, schools, health facilities, government and non-governmental stakeholders, and the private sector. Programme activities can be categorised into the following: the provision and rehabilitation of sanitation and hygiene infrastructure in schools and health centres; behaviour change activities around sanitation, hygiene and clean water at the community level (including schools and health facilities); building the capacity of government and non-government stakeholders; and engaging the private sector in the sanitation market. (For a more detailed overview of activities, please see section 2.4.2.)

15

An important component in the design of the programme activities is use of the CLTS approach, particularly in the demand-side driven activities. Government departments at the district level are trained to coordinate CLTS activities, while civil society organisations are implementing the CLTS activities in the community, household and schools. Additionally, the gender, human rights and equity considerations are considered by UNICEF to be an important feature in the design and delivery of the programmes. The sections below explain more on the CLTS approach and the gender, human rights and equity considerations in the programme design and delivery.

Community-led total sanitation (CLTS)

Since 2014, the GoG-UNICEF WASH Programme has adopted the Community-led Total Sanitation approach to create demand for latrine ownership and use, among other things, in order to address open defecation within communities. The CLTS approach is one of the five pillars of the Rural Sanitation Model and Implementation Strategy (RSMS). CLTS involves triggering communities’ appetites for collective change and empowering communities to take their own actions to become open defecation free (ODF) by developing context-specific local solutions to open defecation. Under a CLTS approach, the target for change is the community, rather than the household. The CLTS approach has been implemented in communities, facilitated by government stakeholders working at the district level with implementing partners.

The Enhanced programme delivered a range of activities to support districts to facilitate CLTS in their communities, including establishing a network of Natural Leaders responsible for scaling up CLTS to neighbouring communities, establishing a CLTS league table, and undertaking ODF verification and certification, among others.

For the Accelerated programme, the CLTS approach has been adapted for implementation in small towns (which have a population range of 2,000 to 30,000 people, often lack the clearly defined community governance structures found in rural communities and manage their own water and sanitation systems), in order to trigger 80 small towns to end open defecation.69 The Accelerated programme is also working to build the capacity of Metropolitan Municipal and District Assemblies to facilitate CLTS implementation in these small towns.

As well as supporting districts to implement and facilitate CLTS activities within communities and small towns, the GoG-UNICEF WASH programme supports communities to become ODF by providing sustainable sanitation financing support and constructing latrines in health facilities and schools.

In addition to CLTS, the GoG-UNICEF WASH programme also undertook further behaviour change activities around handwashing and clean water, particularly in schools. The Health and Hygiene Education Through Play and Sport approach, for example, engages children in edutainment activities around handwashing with soap. Such behaviour change activities were in addition to the construction and rehabilitation of improved handwashing facilities and the provision of training on maintenance and proper use.

69 Accelerated Programme proposal

15

The two programmes also focused on the supply side by building the capacities of stakeholders in sanitation and hygiene service delivery at the national, regional, district and community levels. For instance, a business approach to the delivery of sanitation facilities was adopted, which involved training local artisans on good business practices and delivering sanitation marketing interventions.

Gender, human rights, and equity considerations

Ghana’s Human Development Index (HDI) for 2017 is 0.592 below the average of 0.645 for countries in the medium human development group, and Ghana is ranked 140 out of 189 countries globally. However, when the value is discounted for inequality, the HDI falls to 0.420. For gender inequality, Ghana was ranked 160 countries in the 2017 index on UN Gender Inequality Index (GII)70 with a GII value of 0.538. Gender inequality is evident in numerous statistics; 12.7 percent of parliamentary seats are held by women, and 54.6 percent of adult women have reached at least a secondary level of education compared to 70.4 percent of their male counterparts. For every 100,000 live births, 319 women die from pregnancy related causes; and the adolescent birth rate is 66.6 births per 1,000 women of ages 15-19. Female participation in the labour market is 74.8 percent compared to 79.2 for men.

There are significant WASH-related gender inequities in Ghana. With women and girls doing the most pressing house chores: cooking, cleaning and fetching water, as well as looking after the sick, old and children, the quality and availability of water and sanitation services is highly gender sensitive and impactful. The multiple roles women play often subject them to crushing workloads, leaving them with no time for other activities; further, they are often excluded from decision-making in the development of water and sanitation resources of their own communities.

The UN Education Gender Inequality Index for Ghana for 2017 is 34.9%, this inequality is due to a myriad of reasons, but as there is absence of data to assess the contribution and impact of lack of suitable WASH facilities. In the Report, “What works in Girls Education in Ghana 2012” it is noted that that much qualitative research confirms that menstruation (cramps, embarrassment etc) is given as a reason for absenteeism. Lack of suitable facilities for, and information on, menstrual hygiene management, for instance, leads to considerable female absenteeism in schools71.

Prior to the GoG-UNICEF WASH programme, WASH interventions rarely responded to men’s, women’s, boys’, and girls’ differential needs and gender mainstreaming was largely absent from WASH programming, according

70 The 2010 HDR introduced the GII, which reflects gender-based inequalities in three dimensions – reproductive health, empowerment, and economic activity. Reproductive health is measured by maternal mortality and adolescent birth rates; empowerment is measured by the share of parliamentary seats held by women and attainment in secondary and higher education by each gender; and economic activity is measured by the labour market participation rate for women and men. The GII can be interpreted as the loss in human development due to inequality between female and male achievements in the three GII dimensions.

71 Thériault, F., Maheu-Giroux, M., Blouin, B., Casapía, M. and Gyorkos, T. Effects of a Post-Deworming Health Hygiene Education

Intervention on Absenteeism in School-Age Children of the Peruvian Amazon. PLoS Negl Trop Dis. 8(8): e3007, 2014.

15

to the Accelerated proposal. The design of the Enhanced and Accelerated programmes therefore aims to address this by promoting gender mainstreaming in the WASH sector and embedding gender equity considerations in its key target outcomes.

The original immediate outcome set out in the Enhanced project log frame is ‘improved access to gender responsive, child and disability-friendly WASH services for children and youth in schools and communities in the five most deprived regions of Ghana’. The objectives, outcomes, and outputs of both projects seek to address the health inequities faced by women, children and new born by improving the availability of gender- and disability-friendly WASH facilities in schools and health centres, improving awareness of menstrual hygiene management, and strengthening gender responsive WASH programming through gender mainstreaming and by integrating gender issues into operational instruments at the district level.

Stakeholders

A range of stakeholders have been involved in the implementation of the Enhanced and Accelerated programmes. Some have been involved in both the Enhanced and Accelerated programmes, while others’ involvement has been exclusive to one programme only.

UNICEF Ghana: UNICEF Ghana is responsible for the overall design and implementation of the Enhanced and Accelerated programmes. In particular, UNICEF Ghana plays a key role in the management, coordination, and oversight of the programme, coordinating a diverse range of stakeholders including government partners responsible for implementing specific activities, technical experts, and maintaining partnerships with other organizations that are active in the WASH sector. UNICEF’s Chief of Wash programme has responsibility for day-to-day management of the programme, and receives technical guidance from various other UNICEF teams and specialists, including UNICEF’s sectoral programmes of Education; of Advocacy, Communication, Monitoring and Analysis; and Health and Nutrition; among others. UNICEF Ghana’s programmatic work in this area is implemented in line with both national and international strategies and agreements, embedding its work within the Ghana UN Development Assistance Framework 2012-16 and the Gender Mainstreaming Guidelines and Toolkit for Water, Sanitation and Hygiene Sector of Ghana, for instance.

Global Affairs Canada (GAC): Global Affairs Canada has been providing financial support to the GoG-UNICEF WASH programme since March 2012. GAC originally offered a 9m CAD grant to support the Enhanced WASH project until 30th June 2015, which was followed by a second round of funding to support an extension of the Enhanced project until December 2018, which brought the total grant funding to 17,999,605 CAD. In March 2015, GAC agreed to fund an additional project component focussing solely on Northern Ghana, the Accelerated programme. The value set out in the original proposal was 19,892,000 CAD, but the total funding received to date has not yet been confirmed by the evaluation team. GAC has recently developed and adopted its Feminist International Assistance Policy which commits 15 percent of its bilateral development assistance to initiatives that are advancing gender equality. In line with this framework, all programmes funded by GAC should consider the extent to which they are gender transformative and gender responsive. This evaluation will seek to understand to what extent the Enhanced and Accelerated Programmes have been aligned with this framework.

15

Government stakeholders

Environmental Health and Sanitation Directorate (EHSD): The EHSD, part of the Ministry of Sanitation and Water Resources, is the lead governmental agency responsible for sanitation policy development and sanitation programming delivery in Ghana. The EHSD has had significant involvement in the GoG-UNICEF WASH programme, including carrying out capacity development activities to improve the delivery of sanitation interventions, and it has also played a large role in planning, monitoring, and evaluating the programme. EHSD has staff at the national, regional and district levels. EHSD staff have also championed CLTS implementation within communities at the district level, working closely with other stakeholders, and national and regional officers of the EHSD to provide additional technical support and supervision for CLTS implementation.

Metropolitan, Municipal and District Assemblies (MMDAs): MMDAs are decentralized local government authorities responsible for planning, coordination, implementation and monitoring and evaluation of all socio-economic development efforts. Under the two programmes they are responsible programme planning, coordination implementation and monitoring and evaluation at the community level and for district level in both programmes. In the Accelerated Programme, they are involved in facilitating CLTS in small towns.

Office of the Head of Local Government Service (OHLGS). The local Government Service (LGS) which includes the Office of the Head of Local Government Service (OHLGS), MMDAs and the Regional Co-ordinating Councils RCCs, has as its governing body a LGS Council which oversees governance issues and ensures effective functioning of the LGS. The OHLGS has been involved in the design and review of the MMDA capacity assessment process in the Accelerated programme.

Department of Community Development: The Department for Community Development (DCD) exist to facilitate the mobilization and use of available human and material resources to improve upon the living standards of deprived rural and urban communities within an effectively decentralized system administration through Adult education and Extension Services.72 The Department for Community Development (DCD) is involved in both the Enhanced and Accelerated components, supporting community mobilization efforts and community development initiatives within the MMDAs. Specifically, the DCD has leads the social norms programming and Communication for Development (C4D) aspects of the project to encourage behaviour change, by providing both capacity building support for social norms programming and technical implementation support.

In the Accelerated programme, the DCD is also taking the lead at the national level in coordinating the capacity-building components of the programme, which also includes developing and maintaining partnerships with health training institutions. The DCD is also working with the Environmental Health staff at the district level to carry out community mobilization activities and develop CLTS processes within communities.

Community Water and Sanitation Agency (CWSA): The CWSA is the lead government agency responsible for the facilitation of safe water supply and improved sanitation and hygiene practices in small towns and rural communities, working through Metropolitan, Municipal, and District Assemblies. They are also the lead agency

72 http://dcd.gov.gh/

http://dcd.gov.gh/

15

for promoting handwashing with soap in Ghana. In both the Enhanced and Accelerated components, the CWSA is responsible for procurement of, and providing supervision for the construction of, WASH facilities in schools and health facilities. They also play a key role in strengthening the capacities of beneficiary institutions to operate and maintain their facilities over time.

Ghana Health Service: The Ghana Health Service and the Ministry of Health are the main Government stakeholders with responsibility for both preventive and curative health in the country. They are responsible for identifying health facilities to benefit from both projects. The GHS is also responsible for ensuring the senior management of health facilities are promoting good hygiene.

School Health Education Programme (SHEP): SHEP is responsible for the delivery of the WASH in schools (WinS) components of both projects. SHEP has facilitated capacity building for WinS delivery and has undertaken planning, monitoring and reporting of WinS interventions, as well as playing a key role in developing the WinS costed strategy and guidelines for WinS implementation in Ghana.

Environmental Protection Agency (EPA): The EPA led the Strategic Environmental Assessment process in developing Ghana’s Rural Sanitation Model and Implementing Strategy and provides oversight of the implementation of the Environmental and Social Management Framework.

At the regional level, the Regional Environmental Health Officers coordinate sanitation service delivery through the regional Inter-Agency Coordinating Committee on Sanitation (RICCS). Similar structure, called District Inter-Agency Coordinating Committee on Sanitation (DICCS) is formed at the district level.

Academic stakeholders:

• Kwame Nkrumah University of Science and Technology (KNUST): KNUST was one of the Sanitation Knowledge Management Initiative partners with responsibility for conducting research on the effectiveness of sanitation and hygiene interventions (including around cost-effectiveness, for instance). KNUST is also responsible for research and the design of fit-for-purpose and cost-effective sanitation facilities and has also been involved in the technical aspects of the design of sanitation marketing activities.

Civil society stakeholders:

Right to Play: Right to Play are a key civil society partner in both components of the programme. Right to Play are responsible for implementing hygiene promotion activities in schools based on the hygiene education through play and sport (HHETPS) approach and have worked with an education consultant to mainstream HHETPS into school and teacher training curricula. RTP also leads on the implementation of the Menstrual Hygiene Management component of the WinS programme and provides support to Ghana Education Service officers to build their capacity to engage children through play.

IRC: IRC is an international think-and-do-tank building strong WASH (water, sanitation and hygiene) systems.73 It is the main coordinating institution for the Sanitation Knowledge Management Initiative.

73 https://www.ircwash.org/

https://www.ircwash.org/

16

District Resource Persons: A network of civil society employees, known as District Resource Persons, was established to provide support to MMDAs at the district level.

National level civil society stakeholders:

Green Advocacy: Green Advocacy has responsibility for developing the Environmental and Social Management Framework for the Enhanced project and building capacity for its implementation

Curious Minds: Curious Minds supports the WinS component by providing media-based hygiene-promotion activities to school children, also working with Children and Youth Ambassadors in the Enhanced project to reach a wider audience.

Coalition of NGOs in Water and Sanitation: CONIWAS have been involved in both components of the project, supporting MMDAs on budget monitoring for WASH interventions and leading on advocacy and citizen engagement activities to advocate for increased government funding for sanitation.

RuffinLit: RuffinLit is market-based organization which is supporting the implementation of the sanitation marketing activities in both projects.

TREND: TREND is an institutional partnership that promotes knowledge management services within the WASH sector in Ghana. TREND is supporting sanitation marketing activities and are also responsible for the development of training and learning materials under the Sanitation Knowledge Management Initiative. They are also one of the civil society organisations that have provided human resources to the DRP network.

EDSAM, Afram Plains Development Organisation, Hope for Future Generations, IDC, EDS, CDA, New Energy: These civil society organisations provide districts with human resources for the District Resource Persons network, establish and support Natural Leader networks, assist with overall project monitoring and local capacity building, and are responsible for scaling up CLTS in project districts.

Private sector stakeholders:

Business development partners (BDPs): BDPs are private sector partners and NGOs working with the Accelerated programme districts to rollout the SanMark component of the programme at the district and community level. BDPs have also trained artisans in business management.

Technical service providers: These are businesses and NGOs who have assisted in training community technical volunteers and artisans in technical skills.

This annex contains the topic guides used for stakeholder interviews. These were included in the Inception Report.

Annex 7: Interviews

16

Guide for interviewing national government representatives

Introductory questions

1. Could you briefly tell me about yourself? What is your role in your organisation? What is [RESPONDENT’S ORGANISATION] in charge of? Confirm geographic remit of organisation.

2. What is your involvement in UNICEF supported WASH programmes? For how long have you worked in this role/been involved in these programmes? What other key staff/departments are involved in the programmes?

3. In your experience, what are/were the major issues around WASH at the national level? How are the sanitation conditions in schools and health centres in rural communities/small

towns/ regions? How have these changed over time? How does access to and use of WASH facilities differ by gender? Disability? Any other

marginalised groups? Apart from UNICEF supported WASH programmes, are there any other initiatives that are

active at the national level? What is the size/scale of the initiative, and what are their key activities?

4. What are the key WASH strategies and policies at the national level? How are these strategies being implemented?

5. How familiar are you with UNICEF supported WASH programmes? What is your understanding of the programmes’ objectives?

In your opinion, how well aligned are the activities and objectives of the programme with your organisation’s strategies/policies?

What is your opinion on the Community-led total sanitation (CLTS) approach74 used in these programmes? Do you believe the CLTS approach has had the desired effects?

Are there any changes that you would recommend in order to have future UNICEF WASH programmes more aligned with your organisation’s strategies/policies? With the needs of its intended beneficiaries? Probes:

o Are you aware of the Rural Sanitation Model Strategy (RSMS) and its pillar on demand creation for sanitation and hygiene facilities and services using the CLTS approach?

In your opinion, do the programmes adequately consider the specific needs of women and girls in Ghana?

6. To what extent do you think the programme has been effective in achieving these objectives? Probe: Could you please highlight one or two hygiene or sanitation aspects that you believe

have improved at national level?

Programme implementation

7. If not covered by Question 2 above: Please give me a description of programme activities that your organisation has been involved in?

Probes for specific government agencies: o Environmental Health and Sanitation Directorate: Were you involved in the capacity

development activities to improve the delivery, planning, monitoring and reporting

74 CLTS approach: CLTS involves triggering communities’ appetites for collective change and empowering communities to take their own actions to become open defecation free (ODF) by developing context-specific local solutions to open defecation. Under a CLTS approach, the target for change is the community, rather than the household. The CLTS approach has been implemented in schools, health facilities, and communities, facilitated by government stakeholders working at the district level with implementing partners.

16

of sanitation interventions (implementation within communities, technical support and supervision of CLTS implementation by national and regional officers)? Can you explain to us a bit more about what this activity involved?

o Department for Community Development: Were you involved in supporting community mobilization efforts and community development initiatives within the MMDAs? Can you explain to us a bit more about what this activity involved?

o Community Water and Sanitation Agency: Were you involved in the procurement of and supervision for the construction of WASH facilities in schools and health facilities? Can you explain to us a bit more about what this activity involved?

8. How, if it all, was your organisation involved in the design of the programmes? Did UNICEF consult your organisation during the design-phase? What about the design of the activities that you just mentioned? In your opinion, how effective was UNICEF’s role in the programme design? Specifically, to what extent do you think that UNICEF had a coordinated strategy in dealing with different ministries and MMDAs? Could something have been done differently?

Did the programme have sufficient financial resources and human capacity?

9. In your opinion, have stakeholders been effective in facilitating the programmes’ CLTS activities? If yes, why?

How effective was the involvement of UNICEF staff in CLTS activities? How effective was the involvement of regional and district authorities in CLTS activities? How effective was the involvement of civil society and private organisations in CLTS

activities? 10. Have you or your colleagues received any training/support (including logistical support) through the

programmes? (Probe: What type of training/support was provided? Probe on the following training/support provided by the programmes: overall management of WASH sector, support on building strategic relations with key WASH stakeholders (e.g. health centres), support to establish coordination mechanisms at the local/regional and national level, WASH sector monitoring, evaluations, database and reporting systems training, training on practical knowledge and skills to facilitate participatory hygiene at the local level, logistical support.

What were the results of the support for you/your organisation? Probe: any change in leadership, coordination and facilitation skills to deliver WASH services in Ghana.

How satisfied were you/your organisation with the training and support provided? In your experience, what worked well or less well? Could trainings have been carried out

differently? Are there other trainings/support that you believe you or your organisation would have

benefited from?

Outcomes

11. The programmes aimed to improve water access, and sanitation and hygiene practices. Is there any evidence that these aims have been achieved? How did such results arise?

What about health outcomes? How? Please provide one example. If not: On what timescale would you expect health outcomes to improve?

16

12. What accountability75 mechanisms exist for delivering WASH services at the national level? What mechanisms exist for gender-responsive delivery of WASH services? Have these been introduced or changed at all in recent years? If so, why? What were the key enablers? What are the key barriers?

Do you have any examples of how the department that you work for has contributed to the effective delivery of WASH services?

How effective were UNICEF supported WASH programmes in improving accountability/responsiveness?

13. How do you/your organisation share knowledge related to delivery of WASH facilities and services with other stakeholders? Has knowledge sharing changed at all in recent years? If so, why? What were the key enablers? What are the key barriers?

How effective were UNICEF supported WASH programmes in contributing to knowledge sharing among organisations involved in the delivery of WASH facilities and services?

Which information generated by the programmes did you find most relevant to your day to day activities (e.g. Household sanitation financing guidelines developed with toolkits for the implementation of a district sanitation funds, the Environmental and Social Management Framework (2017)?

Did you participate in the Sanitation Knowledge Management Initiative and/or the CLTS Stocktaking Forum?

Was there any other forum or platform established or used to disseminate results from the field? And how useful was this platform?

What are your views on the Sanitation Knowledge Management Initiative? And the CLTS Stocktaking Forum?

14. Please tell us about access to and use of sanitation facilities in the targeted regions and nationally. For example, available latrines.

Is there any evidence that the access to sanitation facilities in the targeted communities, schools and health centres has improved?

What about people’s behaviours when using the available sanitation facilities? In your view, what was the contribution of the programmes to this improvement (if any)? How effective were the programme activities76 in contributing to better access and use of

sanitation facilities? What other factors might have contributed to the improvement or lack of improvement in

accessing and using sanitation facilities? 15. Please tell us about access to potable water supply nationally and particularly in the targeted regions.

Is there any evidence that the access to potable water in the targeted communities, schools and health centres has improved? What about overall at the national level?

In your view, what was the contribution of the programmes to this improvement (if any)? How involved is the community in the implementation of safe water supply activities? Do

people know how to build/maintain water infrastructure?

75 Accountability includes: responsibility, answerability and enforceability. 1) Responsibility: clear definition of roles and responsibilities for service delivery and enabling coordination and cooperation between the different actors, 2) Answerability relates to informing, consulting and including stakeholders at all stages of service delivery and 3) Enforceability means monitoring performance, supporting compliance and establishing mechanisms for the use of corrective and remedial action where necessary. 76 Government capacity building activities to refer to: 1) Supporting Basis Sanitation Information System (BaSIS), a decentralised M&E sanitation system developed to aid in the implementation of CLTS at both sub-national and national levels, 2) Supporting the implementation of environmental management plans using the environmental management guidelines under this programme, 3) Supporting the establishment of strategic partnerships with institutions - school of health and hygiene, nursing training schools, 4) Supporting the development of a national school sanitation strategy for achieving universal access to sanitation in Ghanaian basic schools, including national guidelines for school WASH facilities and 5) Supporting communities to undertake the ODF verification and certification process.

16

Have campaigns promoting hygiene, household water treatment and storage helped improved water access?

What other factors might have contributed to the improvement or lack of improvement in access to safe water?

16. Is there any evidence that hygiene practices have improved, particularly handwashing with soap? What evidence is there that community members are aware of why and how hand-washing

is important for good personal hygiene? To what extent do community members understand the importance of handwashing with

soap (HWWS)? In your view, what was the contribution of the programmes to this improvement (if any)? What other factors might have contributed to the improvement or lack of improvement in

hygiene practices? •

17. How would you describe the current status of faecal sludge management systems in Ghana? Are there any key differences between small towns/rural communities/regions?

How has this changed in recent years? How have the programmes contributed to this? Can you share any policy documents related to faecal sludge management with us? What are the challenges to faecal sludge management?

Programme results and challenges encountered

18. What were (if any) the unintended effects (positive and negative) produced by the two programmes? 19. Are there any contextual or other factors that have impeded/enabled the programme to achieve its

objectives? Probe on: In your opinion, is obtaining the national ODF status a priority for government departments? Did you/your department have the capacity to implement programme activities? In your opinion, did project partners have sufficient capacity and accountability to deliver

programme activities? Were beneficiaries willing to participate in CLTS activities? Was the private sector engaged in demand generation activities? If yes, how did

government/UNICEF engage private businesses? 20. Which main barriers were faced during programme implementation? Probe: geological conditions in

some parts of the country, cultural barriers, implementing organisations’ capacities, programme design.

Are there any specific barriers faced by women/girls or people with disabilities in accessing WASH services? What is being done about this? What more could be done?

Were there any barriers/challenges to programme implementation due to the design of programme activities?

How did the programme respond to these challenges? Were any issues addressed? 21. If the programme were to be implemented again, what would you like to be done differently?

How could programme design have been improved?

Final remarks

22. Are there any key documents you would like to share with us that will help us with this evaluation? Probe: Documents such as strategic policy documents, internal monitoring documentation, statistical data (e.g. MICS)

23. Is there anyone else you would recommend we speak to apart from the government institutions, civil society, private sector organisations, UNICEF staff, schools’ administrators and health centres representatives? Would you be able to share their contact details or introduce us?

16

24. Is there anything else you would like to share with us before concluding?

16

Guide for interviewing regional and district level (MMDA) representatives



2. What is your involvement in the UNICEF-supported WASH programmes? For how long have you worked in this role/been involved in these programmes? What other key staff/departments are involved in the programmes?

3. In your experience, what are/were the major issues around WASH in your region/district? How are the sanitation conditions in schools and health centres in your rural

community/small town/ region? How have these changed over time? How does access to and use of WASH facilities differ by gender? Disability? Any other

marginalised groups? Are there any other initiatives that are active at the national level? What is the size/scale of

the initiative, and what are their key activities? 4. What are the key WASH strategies and policies at the regional and district levels? How are these

strategies being implemented? 5. How familiar are you with UNICEF supported WASH programmes? What is your understanding of

the programmes’ objectives? In your opinion, how well aligned are the activities and objectives of the programme with

your organisation’s strategies/policies? What is your opinion on the Community-led total sanitation (CLTS) approach77 used in these

programmes? Do you believe the CLTS approach has had the desired effects? Are there any changes that you would recommend in order to have future UNICEF WASH

programmes more aligned with your organisation’s strategies/policies? With the needs of its intended beneficiaries? Probes:

o Are you aware of the Rural Sanitation Model Strategy (RSMS) and its pillar on demand creation for sanitation and hygiene facilities and services using the CLTS approach?


6. To what extent do you think the programme has been effective in achieving these objectives? Probe: Could you please highlight one or two hygiene or sanitation aspects that you believe

have improved at national level?


7. If not covered by Question 2 above: Please give me a description of the programme activities that your organisation was involved in?

Probes for specific government agencies:


16

o Community Water and Sanitation Agency regional offices: Were you involved in the procurement of and supervision for the construction of WASH facilities in schools and health facilities? Can you explain to us a bit more about what this activity involved?

o Metropolitan, Municipal, and District Assemblies: Were you involved in the coordination implementation, or/and monitoring and evaluation of the programmes? Were you involved in facilitating CLTS in small towns?

8. How, if it all, was your organisation involved in the design of the programmes? Did UNICEF consult your organisation during the design phase? What about the design of the activities that you just mentioned? In your opinion, how effective was UNICEF’s role in the programme design? Could something have been done differently? Specifically, do you think that UNICEF has had a coordinated strategy in dealing with different ministries and MMDAs?

Were UNICEF staff adequately skilled to fulfil the programme activities? Did the programme have sufficient financial resources and human capacity?

9. Have you or your colleagues received any training/support through the programmes? (Probe: What type of training/support were provided? Probe on the following training/support provided by the programmes: overall management of WASH sector, support on building strategic relations with key WASH stakeholders (e.g. health centres), support to establish coordination mechanism at local and regional level, WASH sector monitoring, evaluations, database and reporting systems training.


How satisfied were you/your organisation with trainings and support provided? In your experience, what worked well or less well? Could trainings have been carried out

differently? Are there other trainings/support that you believe you or your organisation would have

benefited from? 10. Does your region/district have a concrete action plan to implement WASH (particularly around

sanitation and hygiene)? Does your plan consider gender-specific WASH needs? Did your organisation receive support in developing/adopting gender-sensitive WASH

implementation plans? How satisfied were you/your organisation with the support received?

11. What type of accountability systems exits in your organisation (e.g. DESSAP, BaSIS)? Do you/your organisation know how to use these systems? How do you use DESSAP

(sanitation and management tool)? Do you collect/monitor information using BaSIS? Do you feel you have received adequate support for developing/adopting accountability

systems? Did you have concrete targets to reach? Did you manage to reach these? And if not, why

not?

Outcomes

12. The programmes aimed to improve water access and sanitation and hygiene practices. Is there any evidence that these aims have been achieved? How did this result arise?

What about health outcomes? How? Please provide one example. 13. What accountability mechanisms exist for delivering WASH services at the regional and district levels?

What mechanisms exist for gender-responsive delivery of WASH services? Have these been introduced or changed at all in recent years? If so, why? What were the key enablers? What are the key barriers?

16

Do you have any examples of how the department that you work for has contributed to the effective delivery of WASH services?


14. How do you/your organisation share knowledge related to delivery of WASH facilities and services with other stakeholders? Has knowledge sharing changed at all in recent years? If so, why? What were the key enablers? What are the key barriers?

How effective were UNICEF supported WASH programmes in contributing to knowledge sharing among organisations involved in the delivery of WASH facilities and services?

Which information generated by the programme did you find most relevant to your day to day activities (e.g. Frameworks to regulate private sector involvement in sanitation?)



15. Please tell us about access to and use of sanitation facilities in the targeted regions and districts. For example, available latrines.

Is there any evidence that access to sanitation facilities in the targeted communities, schools and health centres has improved?

What about people’s behaviour when using the available sanitation facilities? In your view, what was the contribution of the programmes to this improvement (if any)? How effective were the programme activities78 in contributing to better access and use of


accessing and using sanitation facilities? 16. Please tell us about access to potable water supply in your region/district?

Is there any evidence that access to potable water in the targeted communities, schools and health centres has improved? What about overall at the national level?


people know how to build/maintain water infrastructure? Have campaigns promoting hygiene, household water treatment and storage helped to

improve water access? What other factors might have contributed to the improvement or lack of improvement in

access to safe water? 17. Is there any evidence that hygiene practices have improved, particularly handwashing with soap?

What evidence is there that community members are aware of why and how hand-washing is important for good personal hygiene?

To what extent do community members understand the importance of handwashing with soap (HWWS)?

In your view, what was the contribution of the programmes to this improvement (if any)?

78 Government capacity building activities to refer to: 1) Supporting Basis Sanitation Information System (BaSIS), a decentralised M&E sanitation system developed to aid in the implementation of CLTS at both sub-national and national levels, 2) Supporting the implementation of environmental management plans using the environmental management guidelines under this programme, 3) Supporting the establishment of strategic partnerships with institutions - school of health and hygiene, nursing training schools, 4) Supporting the development of a national school sanitation strategy for achieving universal access to sanitation in Ghanaian basic schools, including national guidelines for school WASH facilities and 5) Supporting communities to undertake the ODF verification and certification process.

16

What other factors might have contributed to the improvement or lack of improvement in hygiene practices?

18. How would you describe the faecal sludge management system in your rural community/small town/region? Are there any key differences between small towns, rural community in your district/region?


19. Have rural community/small town in your district/region achieved ODF status? If not, what else do you think needs to be done to obtain it?

Which steps have they carried out so far to obtain ODF status? How has UNICEF Ghana supported this?

What are the main challenges to obtaining ODF status?


20. In your view, what are main outputs and achievements of the programmes (if any)? 21. Is there any evidence that the programmes have resulted in improved health outcomes?

If not: On what timescale would you expect health outcomes to improve? 22. What (if any) were the unintended effects (positive and negative) produced by the two programmes? 23. Are there any contextual or other factors have impeded/enabled the programme to achieve its

objectives? Probe on: In your opinion, is obtaining national ODF status a priority for government departments? Did you/your department have the capacity to implement programme activities? In your opinion, did project partners have sufficient capacity and accountability to deliver

programme activities? Were beneficiaries willing to participate in CLTS activities? Was the private sector engaged in demand generation activities? If yes, how did

government/UNICEF engage private businesses? 24. Which main barriers were faced during programme implementation? Probe: geological conditions in

some parts of the country, cultural barriers, implementing organisations’ capacity, programme design. Are there any specific barriers faced by women/girls or people with disability in accessing

WASH services? What is being done about this? What more could be done? Were any barriers/challenges in programme implementation due to the design of

programme activities? How did the programme respond to these challenges? Were any issues addressed?

25. If the programme were to be implemented again, what would you like to be done differently? How could programme design have been improved?

Final remarks

26. Are there any key documents you would like to share with us that will help us with this evaluation? Probe: Documents such as regional or district level WASH plans and progress reports.



17

Guide for interviewing civil society organisations


1. Could you briefly tell me about yourself? What is your role in your organisation? What does [RESPONDENT’S ORGANISATION] do? Confirm geographic remit of organisation.

2. What is your/ your organisation’s involvement in UNICEF supported WASH programmes? For how long have you worked in this role/been involved in these programmes?

3. In your experience, what are/were the major issues around WASH (in particular around sanitation and hygiene) in your region/district?

How are the sanitation conditions in schools and health centres in your rural community/small town/ region? How have these changed over time?

How does access to and use of WASH facilities differ by gender? Or by disability? Any other marginalised groups?

Are there any other initiatives that are active at the national, regional, or district levels in which your organisation operates? What is the size/scale of the initiative, and what are their key activities? Has your organisation been involved in these initiatives? If so, are these programmes/activities and the UNICEF programmes/activities well-coordinated (covering separate geographic zones or carrying out complimentary activities)?

4. What are the key WASH strategies and policies (in particular relating to sanitation and hygiene) at the regional and district levels? How do these strategies impact on your organisation’s activities?

5. How familiar are you with UNICEF supported WASH programmes? What is your understanding of the programmes’ objectives?

In your opinion, how well aligned are the activities and objectives of the programme with your organisation’s strategies/policies/objectives?

What is your opinion of the Community-led total sanitation (CLTS) approach79 used in these programmes? Do you believe the CLTS approach has had the desired effects?

Are there any changes that you would recommend in order to have future UNICEF WASH programmes more aligned with your organisation’s strategies/policies? With the needs of its intended beneficiaries?

In your opinion, do the programmes, and the activities you are involved in adequately consider the specific needs of women and girls in Ghana?

6. To what extent do you think the programme has been effective in achieving these objectives? • Probe: Could you please highlight one or two hygiene or sanitation aspects that you believe have improved at the national/regional/district/community level? •


7. If not covered by Question 2 above: Please give me a description of programme activities that your organisation has been involved in?

Probes for specific civil society organisations:

Right to play: Were you involved in implementing CLTS and/or hygiene promotion activities in schools? Can you tell us a bit more about what these activities involved?


17

Curious minds: Were you involved in delivering hygiene promotion activities in school/ to school children? Can you tell us a bit more about what these activities involved?

RuffinLit: Were you involved in implementing sanitation marketing activities? Can you tell us a bit more about your involvement in these activities?

TREND: Were you involved in implementing sanitation marketing activities? Can you tell us a bit more about your involvement in these activities? Were you involved in the monitoring and evaluation of programme activities, or in knowledge management more broadly? How?

IRC: Have you been involved in the monitoring and evaluation of programme activities, or in knowledge management more broadly? How?

8. How, if it all, was your organisation involved in the design of the programmes? Did UNICEF consult your organisation during the design-phase? What about the design of the activities that you just mentioned? In your opinion, how effective was UNICEF’s role in programme design? Specifically, do you think that UNICEF has had a coordinated strategy in overseeing the involvement of different civil society and private sector organisations? Were the activities designed appropriately for the specific context? Could something have been done differently?

Did the programme have sufficient financial resources and human capacity? 9. In your opinion, have the programme’s CLTS and hygiene promotion activities been facilitated and

implemented effectively? How were UNICEF staff involved in facilitating CLTS and hygiene promotion activities? How

effective was their involvement? If the interviewee is involved in delivering CLTS/hygiene promotion activities: Did you receive adequate training and/or support?

How effective were [other] civil society organisations in facilitating and implementing CLTS activities? Was civil society involvement well-coordinated?

How were regional and district authorities involved in facilitating CLTS activities? How effective was their involvement? If the interviewee is involved in delivering CLTS/hygiene promotion activities: Did you receive adequate training and/or support?

How were private sector organisations involved in implementing or facilitating CLTS activities? How effective was their involvement?

10. In your opinion, have the programme’s sanitation marketing (SanMark) activities been facilitated and implemented effectively?

How were UNICEF staff involved in facilitating sanitation marketing activities? How effective was their involvement? If the interviewee is involved in delivering SanMark activities: Did you receive adequate training and/or support?

How effective were [other] civil society organisations in facilitating and implementing sanitation marketing activities? Was there effective coordination of civil society involvement?

How were regional and district authorities involved in facilitating sanitation marketing activities? How effective was their involvement? If the interviewee is involved in delivering SanMark activities: Did you receive adequate training and/or support?

How were private sector organisations, particularly Business Development Partners, involved in implementing sanitation marketing activities? How effective was their involvement?

11. If not covered by the questions above: Have you or your colleagues received any training/support through the programmes? (Probe: What type of training/support were provided? Probe on the following training/support provided by the programmes, support on building strategic relations with key WASH stakeholders (e.g. health centres), WASH sector monitoring, evaluations, database and reporting systems training, training on how to use the CLTS approach to promote good hygiene in communities.


How satisfied were you/your organisation with trainings and support provided?

17

In your experience, what worked well or less well? Could trainings have been carried out differently?

Are there other trainings/support that you believe you or your organisation would have benefited from?

Outcomes

13. The programmes aimed to improve water access and sanitation and hygiene practices. Is there any evidence that this aim has been achieved? How did this result arise?

What about health outcomes? How? Please provide one example. 14. What accountability mechanisms exist for delivering WASH services at the national, regional, or

district levels? What mechanisms exist for gender-responsive delivery of WASH services? Have these been introduced or changed at all in recent years? If so, why? What were the key enablers? What are the key barriers?

Do you have any examples of how the civil society organisation that you work for has contributed to the effective delivery of WASH services?


15. Has there been an increase in private sector involvement in WASH service delivery? How has the programme contributed to this? Has your organisation contributed to this? If so, how?

16. How do you/your organisation share knowledge related to the delivery of WASH facilities and services with other stakeholders? Has knowledge sharing changed at all in recent years? If so, why? What were the key enablers? What are the key barriers?

How effective were UNICEF supported WASH programmes in contributing to knowledge sharing among organisations involved in the implementation of WASH activities?

Which information generated by the programme did you find most relevant to your day to day activities (e.g. Gender mainstreaming guidelines for the WASH sector, the Environmental and Social Management Framework (2017)?

If not covered by question 7 above: Did you participate in the Sanitation Knowledge Management Initiative and/or the CLTS Stocktaking Forum?



Is there any evidence that the access to sanitation facilities in the targeted communities, schools and health centres has improved?

What about people’s behaviours when using the available sanitation facilities? In your view, what was the contribution of the programmes to this improvement (if any)? How effective were the programme activities80 in contributing to better access and use of


accessing and using sanitation facilities?


17

18. Please tell us about access to potable water supply nationally and particularly in the targeted regions. Is there any evidence that the access to potable water in the targeted communities, schools

and health centres has improved? What about overall at the national level? In your view, what was the contribution of the programmes to this improvement (if any)? How involved is the community in the implementation of safe water supply activities? Do

people know how to build/maintain water infrastructure? Have campaigns promoting hygiene, household water treatment and storage delivered

helped improved water access? What other factors might have contributed to the improvement or lack of improvement in



To what extent do community members understand the importance of handwashing with soap (HWWS)?

In your view, what was the contribution of the programmes to this improvement (if any)? What other factors might have contributed to the improvement or lack of improvement in

hygiene practices? 20. How would you describe current status of faecal sludge management systems in Ghana? Are there

any key differences between small towns/rural communities/regions? How has this changed in recent years? How have the programmes contributed to this? Can you share any policy documents related to faecal sludge management with us? What are the challenges to faecal sludge management?


21. Which contextual factors have impeded/enabled the programme to achieve its objectives? a. Did you feel you had the capacity to implement programme activities? Would you have

valued additional support from the government or UNICEF? b. Were beneficiaries (e.g. school children) willing to participate in CLTS activities? c. In your view, were the programmes aligned with other programmes?

22. In your opinion, which were the main challenges encounter when implementing programme activities?

23. If the programme were to be implemented again, what would you like to be done differently?

Final remarks


25. Is there anyone else you think we should speak to apart from the government institutions, civil society, private sector organisations, UNICEF staff, schools’ administrators and health centre representatives? Would you be able to share their contact details or introduce us?


17

Guide for interviewing private sector organisations


1. Could you briefly tell me about yourself? What is your role in your organisation? Is [RESPONDENT’S ORGANISATION] involved in the water, sanitation, and hygiene market? If so, how?

2. What is your involvement in UNICEF supported WASH programmes? For how long have you/your organisation been involved in these programmes?

3. In your experience, what are/were the major issues around WASH at the national, regional and district levels? How are the sanitation conditions in schools and health centres in rural communities/small

towns/ regions? How have these changed over time? How does access to and use of WASH facilities differ by gender? By disability? Any other

marginalised groups? Are there any other initiatives that are active at the national, regional, or district levels that

have sought to engage the private sector in demand generation activities? What is the size/scale of the initiative, and what are their key activities? Has your organisation been involved in these initiatives? If so, are these programmes/activities and the UNICEF programmes/activities well-coordinated (covering separate geographic zones or carrying out complimentary activities)?

4. What are the key WASH strategies and policies at the national, regional and district levels? How are these strategies being implemented? How do these strategies impact on your organisation’s activities?

5. How familiar are you with UNICEF supported WASH programmes? What is your understanding of the programmes’ objectives? How familiar are you with the Community-Led Total Sanitation (CLTS) approach used in

these programmes? What is your opinion of the CLTS approach81? Do you believe the CLTS approach has had the desired effects?

How familiar are you with the sanitation marketing activities implemented by this programme? What is your opinion of the sanitation marketing activities? Do you believe the sanitation marketing activities implemented under this programme have had their desired effects?

Are there any changes that you would recommend in order to have future UNICEF WASH programmes more aligned with your organisation’s strategies/policies? With the needs of its intended beneficiaries?


6. To what extent do you think the programme has been effective in achieving these objectives? Probe: Could you please highlight one or two hygiene or sanitation aspects that you believe have improved at national, regional, or district level?


17


7. If not covered by Question 2 above: Please give me a description of the programme activities that your organisation has been involved in, if any? Has your organisation been involved in implementing CLTS activities? Has your organisation been involved in implementing sanitation marketing activities? Specific probes for businesses that identify as Business Development Partners or Technical

Service Providers: o Business Development Partners: Has your organisation been involved in

implementing the sanitation marketing (SanMark) component of the UNICEF WASH programme? If so, how? Has your organisation been involved in training artisans in business management? If so, what did this involve?

o Technical service providers: Has your organisation been involved in training community technical volunteers and artisans in technical skills? If so, what did this involve?

8. How, if it all, was your organisation involved in the design of the programmes? Did UNICEF consult your organisation during the design-phase? What about the design of the activities we just mentioned? In your opinion, how effective was UNICEF’s role in the programme design? Specifically, do you think that UNICEF has had a coordinated approach in overseeing the development of different private sector and civil society organisations? Could something have been done differently?

Were UNICEF staff adequately skilled to fulfil these activities? Did the programme have sufficient financial resources and human capacity to fulfil these

activities? 9. If the interviewee/ interviewee’s organisation is involved in implementing CLTS activities: In your

opinion, have other stakeholders been effective in facilitating and implementing the programmes’ CLTS activities? If yes, why? How effective was the involvement of UNICEF staff in facilitating CLTS activities? Did you

receive adequate training and/or support? How effective was the involvement of regional and district authorities in facilitating CLTS

activities? Did you receive adequate training and/or support? How effective was the involvement of civil society organisations in CLTS activities? How effective was the involvement of other private sector organisations in CLTS activities?

10. If the interviewee/ interviewee’s organisation is involved in implementing sanitation marketing activities: In your opinion, have the programme’s sanitation marketing activities been facilitated and implemented effectively? How effective was the involvement of UNICEF staff in facilitating CLTS activities? Did you

receive adequate training and/or support? How effective was the involvement of regional and district authorities in facilitating CLTS

activities? Did you receive adequate training and/or support? How effective was the involvement of civil society organisations in CLTS activities? How effective was the involvement of other private sector organisations in CLTS activities?

11. Have you or your colleagues received any training/support through the programmes? (Probe: What type of training/support were provided? Probe on the following training/support provided by the programmes: support on building strategic relations with key WASH stakeholders (e.g. health centres), support to establish WASH sector monitoring, evaluations, database and reporting systems training, training on how to use the CLTS approach to promote hygiene in communities, training on working with local artisans and community support volunteers. What were the results of the support for you/your organisation? Probe: any change in

leadership, coordination and facilitation skills to deliver WASH services in Ghana. How satisfied were you/your organisation with trainings and support provided?

17

In your experience, what worked well or less well? Could trainings have been carried out differently?

Are there other trainings/support that you believe you or your organisation would have benefited from?

Outcomes

12. The programmes aimed to improve water access, and sanitation and hygiene practices. Is there any evidence that this aim has been achieved? How did this result arise? What about health outcomes? How? Please provide one example.

13. What accountability mechanisms exist for delivering WASH services at the national level? What mechanisms exist for gender-responsive delivery of WASH services? Have these been introduced or changed at all in recent years? If so, why? What were the key enablers? What are the key barriers? Do you have any examples of how your organisation has contributed to the effective

delivery of WASH services? How effective were UNICEF supported WASH programmes in improving

accountability/responsiveness? 14. How did the programme seek to engage your organisation/the private sector generally in demand

generation activities? 15. How do you/your organisation share knowledge related to the delivery of WASH facilities and

services with other stakeholders? Has knowledge sharing changed at all in recent years? If so, why? What were the key enablers? What are the key barriers? How effective were UNICEF supported WASH programmes in contributing to knowledge

sharing among organisations, particularly private sector organisations, involved in the delivery of WASH facilities and services?

Which information generated by the programme did you find most relevant to your day to day activities (e.g. Household sanitation financing guidelines developed with toolkits for the implementation of district sanitation funds, the Environmental and Social Management Framework (2017)?



16. Please tell us about access to and use of sanitation facilities in the targeted regions and nationally. For example, available latrines. Is there any evidence that the access to sanitation facilities in the targeted communities,

schools and health centres has improved? What about people’s behaviour when using the available sanitation facilities? In your view, what was the contribution of the programmes to this improvement (if any)? In your view, what was the contribution of your organisation/ the private sector more

generally to this improvement (if any?) Please provide one example. How effective were the programme activities82 in contributing to better access and use of

sanitation facilities?


17

What other factors might have contributed to the improvement or lack of improvement in accessing and using sanitation facilities?

17. Please tell us about access to potable water supply nationally and particularly in the targeted regions. Is there any evidence that the access to potable water in the targeted communities, schools

and health centres has improved? What about overall at the national level? In your view, what was the contribution of the programmes to this improvement (if any)? In your view, what was the contribution of your organisation/the private sector more

generally to this improvement (if any?) Please provide one example. How involved is the community in the implementation of safe water supply activities? Do

people know how to build/maintain water infrastructure? Have campaigns promoting hygiene, household water treatment and storage helped

improve water access? What other factors might have contributed to the improvement or lack of improvement in



To extent do community members understand the importance of handwashing with soap (HWWS)?

In your view, what was the contribution of the programmes to this improvement (if any)? In your view, what was the contribution of your organisation/the private sector more

generally to this improvement (if any?) Please provide one example. What other factors might have contributed to the improvement or lack of improvement in

hygiene practices? 19. Has demand for private sector WASH facilities and infrastructure increased?

In your view, what was the contribution of the programme to this increase, if any?


20. Which contextual factors have impeded/enabled the programme to achieve its objectives? Did you feel you had the capacity to implement programme activities? Would you have

valued additional support from government UNICEF? Were beneficiaries (e.g. school children) willing to participate in CLTS activities? In your view, were the programmes aligned with other programmes?

21. In your opinion, which were the main challenges encountered when implementing programme activities?

22. If the programme were to be implemented again, what would you like to be done differently?

Final remarks


24. Is there anyone else you think we should speak to apart from the government institutions, civil society, private sector organisations, UNICEF staff, schools’ administrators and health centre representatives? Would you be able to share their contact details or introduce us?


17

Guide for interviewing UNICEF staff involved directly in programme delivery


1. Could you briefly tell me about yourself? What is your role in UNICEF Ghana? In which department do you work? What is [RESPONDENT’S DEPARTMENT] in charge of?

2. What is your involvement in UNICEF supported WASH programmes? For how long have you worked in this role/been involved in these programmes? What other key staff/departments are involved in the programmes?

3. In your experience, what are/were the major issues around WASH at the national, regional, and district levels, particularly in the regions where the two programmes are being implemented? How are the sanitation conditions in schools and health centres in rural communities/small

towns/ regions? How have these changed over time? How does access to and use of WASH facilities differ by gender? Disability? Any other

marginalised groups? Are there any other initiatives that are active at the national level? What is the size/scale of

the initiative, and what are their key activities? If so, are these programmes/activities and the UNICEF programmes/activities well-coordinated (covering separate geographic zones or carrying out complimentary activities)?

4. What are the key WASH strategies and policies at the national level? How are these strategies being implemented?

5. How familiar are you with UNICEF supported WASH programmes? What is your understanding of the programmes’ objectives? In your opinion, how well aligned are the activities and objectives of the programme with

your organisation’s strategies/policies? What is your opinion on the Community-led total sanitation (CLTS) approach83 used in these

programmes? Do you believe the CLTS approach have had the desired effects? In your opinion, do the programmes adequately consider the specific needs of women and

girls in Ghana? 6. To what extent do you think the programme has been effective in achieving these objectives?

• Probe: Could you please highlight one or two hygiene or sanitation aspects that you believe have improved at national, regional, or district level? •


7. What role did you/your department have in designing the programmes? In your opinion, how effective was UNICEF’s role in the programme design? Were there any challenges you encountered during programme design? Could something have been done differently? Were UNICEF staff adequately skilled to fulfil the programme activities? Did the programme have sufficient financial resources and human capacity?

8. In your opinion, have stakeholders been effective in facilitating the programmes’ CLTS activities? Is yes, why? How effective was the involvement of UNICEF staff in CLTS activities? How effective was the involvement of regional and district authorities in CLTS activities?


17

How effective was the involvement of civil society and private organisations in CLTS activities?

9. Has the programme provided sufficient training and support to regional and district authorities, civil society organisations, and private sector organisations? What type of training and support was provided? (

10. How did UNICEF manage and coordinate the involvement of government, NGO, and private sector partners and implementers? Was this process effective? What challenges were encountered? How could governance and coordination be improved in order to improve the sustainability of the programmes?

11. What type of accountability systems exits within UNICEF? Do you/others in your organisation know how to use these systems? Did you have concrete targets to reach? Did you manage to reach these? And if not, why

not?

Outcomes

12. The programmes aimed to improve water access and sanitation and hygiene practices. Is there any evidence that this aim has been achieved? How did this result arise?

What about health outcomes? How? Please provide one example. If not: On what timescale would you expect health outcomes to improve?

13. What accountability mechanisms exist for delivering WASH services at the national level? What accountability mechanisms exist within UNICEF? What mechanisms exist for gender-responsive delivery of WASH services? Have these been introduced or changed at all in recent years? If so, why? What were the key enablers? What are the key barriers?

Do you have any examples of how UNICEF Ghana has contributed to the effective delivery of WASH services?


To what extent can monitoring and evaluation indicators be desegregated by gender? 14. How do the implementing partners share knowledge related to delivery of WASH facilities and

services with other stakeholders? Has knowledge sharing changed at all in recent years? If so, why? What were the key enablers? What are the key barriers? How have the UNICEF WASH programmes contributed to improvements in governance and coordination around WASH in Ghana?

How effective were UNICEF supported WASH programmes in contributing to knowledge sharing among organisations involved in the delivery of WASH facilities and services? What more could do they have done to improve knowledge sharing?


15. How has UNICEF Ghana shared lessons learnt with other UNICEF programmes or other WASH programmes within Ghana? How has UNICEF Ghana identified and acted on learning and recommendations from other similar projects carried out by development partners? To what extent have the programmes been adapted in response to ongoing learning?


Is there any evidence that the access to sanitation facilities in the targeted communities, schools and health centres has improved? What about access to sanitation facilities among women, youth, and other marginalised groups?

Is there any evidence that people’s behaviour when using the available sanitation facilities has changed? If so, how?

In your view, what was the contribution of the programmes to this improvement (if any)?

18

How effective were the programme activities84 in contributing to better access and use of sanitation facilities?

What other factors might have contributed to the improvement or lack of improvement in accessing and using sanitation facilities?

17. Please tell us about access to potable water supply nationally and particularly in the targeted regions and districts

Is there any evidence that the access to potable water in the targeted communities, schools and health centres has improved? What about overall at the national level?


people know how to build/maintain water infrastructure? How has this changed throughout the duration of the programmes?

Have campaigns promoting hygiene, household water treatment and storage delivered helped improved water access?

What other factors might have contributed to the improvement or lack of improvement in access to safe water?

18. Is there any evidence that hygiene practices have improved, particularly handwashing with soap? What evidence is there that community members are aware of why and how hand-washing

is important for good personal hygiene? To what extent do community members understand the importance of handwashing with

soap (HWWS)? In your view, what was the contribution of the programmes to this improvement (if any)? What other factors might have contributed to the improvement or lack of improvement in

hygiene practices? •

19. How would you describe current status of faecal sludge management systems in Ghana? Are there any key differences between small towns/rural communities/ regions?



20. What were (if any) the unintended effects (positive and negative) produced by the two programmes? 21. Are there any contextual or other factors have impeded/enabled the programme to achieve its

objectives? Probe on: In your opinion, is obtaining national ODF status a priority for government departments? Did the relevant government departments have the capacity to implement programme

activities? In your opinion, did the implementing partners have sufficient capacity and accountability to

deliver programme activities? Were beneficiaries willing to participate in CLTS activities?


18

Was the private sector engaged in demand generation activities? How? If yes, how did UNICEF/government departments engage private businesses? What challenges were faced in engaging the private sector in demand generation activities?

22. Which main barriers were faced during programme implementation? Probe: geological conditions in some parts of the country, cultural barriers, implementing organisations’ capacity, government capacity, programme design.

Are there any specific barriers faced by women/girls or people with disability in accessing WASH services? What is being done about this? What more could be done?

Were any barriers/challenges in programme implementation due to the design of programme activities?

How did the programme respond to these challenges? Were any issues addressed? 23. If the programme were to be implemented again, what would you like to be done differently?

How could programme design have been improved? To what extent were women and girls involved in programme design? To what extent were women and girls involved in programme implementation? Did this

improve across the duration of the programme? If so, how did UNICEF contribute to this improvement?

24. For gender experts/ specialists within UNICEF: How effective have the programmes been in addressing the specific needs of women and girls?

Final remarks




18

Guide for interviewing health centres representatives



2. In your experience, what are/were the major issues around WASH in health centres and CHPS compounds? Probe: which major issues are most prominent in your health centre?

3. How familiar are you with the work being carried out by the government and UNICEF to improve WASH services and facilities in health centres and CHPS compounds?

4. How familiar are you with UNICEF supported WASH programmes? What is your understanding of the programmes’ objectives? •

Programme activities targeted health centres

5. Have water and sanitation infrastructure been constructed and/or rehabilitated at any point across the last seven years at your health centre? If yes, could you explain which facilities were constructed? Probe: How many latrines and

hand washing basins are available in your health centre? Are the latrines that have been constructed gender-sensitive and disability-friendly?

6. Does your health centre have a well-defined latrine facility management plan in place? Does this plan consider gender-specific WASH needs? Did your health centre receive support in developing/adopting its latrine facility

management plan? How satisfied were you/your organisation with the support received?

7. Have you or your colleagues received training/support to ensure the good state and functionality of WASH facilities and services in your health centre? How are you/your colleagues using the skills acquired to ensure functionality and sustainable delivery of services?

8. Which marketing and promotional activities were disseminated in your health centres? Probe: Did these marketing and promotional activities include ante-natal sessions and child immunization sessions to promote hygiene practices? Have mothers and children been targeted by hygiene communication messages? Was your health centre part of the open defecation campaign? Were health workers involved in the social norms campaign targeting issues such as

Newborn Care, Community Management of Acute Malnutrition, Infant and Young Child Feeding and Integrated Management of Childhood Illnesses?

Outcomes.

9. Please tell us about access to and use of sanitation facilities in your health centre. For example, available latrines. Do you consider that the access and use of sanitation facilities in your health centre has

improved? If so, can you describe what triggered these improvements? What about people’s behaviours when using the available sanitation facilities in your health

centre? In your view, what was the contribution of the programmes to this improvement (if any)? What other factors might have contributed to the improvement or lack of improvement in

accessing and using sanitation facilities? 10. Please tell us about access to potable water supply in your health centre?

18

Is there any evidence that access to potable water in your health centre has improved? If so, can you describe what triggered these improvements?

How involved is the community in the implementation of safe water supply activities? Do people know how to use water infrastructure in health centres?

What other factors might have contributed to the improvement or lack of improvement in access to safe water in your health centre?

11. Is there any evidence that hygiene practices have improved, particularly handwashing with soap? Are health workers, mothers and other caregivers aware of handwashing practices? Has your health centre managed to reduce its neonatal death rate? To what extent do community members that come to your health centre understand the

importance of handwashing with soap (HWWS)? What evidence is there that community members are aware of why and how hand-washing

is important for a good personal hygiene? In your view, what was the contribution of the programmes to this improvement (if any)? What other factors might have contributed to the improvement or lack of improvement in

hygiene practices in your health centre and in the community? 12. Who is responsible for the operation and maintenance (O&M) of WASH facilities in your health

centre? Have there been any changes in the way WASH facilities and services are run in your health

centre? Are there any facilities that currently need repairing?


13. Is there any evidence that the programmes have resulted in improved health outcomes? If not: On what timescale would you expect health outcomes to improve?

14. Is there any evidence of greater access to and use of gender and disability friendly WASH facilities in your health centre? Is there any evidence of more visitors to your health centre washing their hands with soap?

15. In your opinion, have hygiene practices across the community been improved? Is there any evidence of this improvement?

16. What were (if any) the unintended effects (positive and negative) produced by the two programmes?

17. Are there any contextual or other factors that have impeded/enabled the programme to achieve its objectives? Probe on: Did you/your health centre have the capacity to implement the needed facility management

plan? 18. Did you/your team face any barriers when implementing the facility management plan? 19. Did you/your team face any barriers when promoting hygiene practices in your health centre? 20. Would you need additional support to continue to implement gender and disability friendly WASH

facilities and services in your health centres?

Final remarks

21. Anything else you would like to share with us before concluding?

18

Guide for interviewing school administrators



2. In your experience, what are/were the major issues around WASH in schools? Probe: which major issues are most prominent in your school?

3. How familiar are you with the work being carried out by the government and UNICEF to improve WASH services and facilities in schools?

4. How familiar are you with UNICEF supported WASH programmes? What is your understanding of the programmes’ objectives? •

Programme activities targeting schools

5. Have water and sanitation infrastructure been constructed and/or rehabilitated at any point across the last seven years at your school? If yes, could you explain which facilities were constructed? Probe: How many latrines and

hand washing basins are available in your school? Are latrines constructed gender-sensitive and disability friendly?

6. Does your school have a well-defined latrine facility management plan in place? Does your plan consider gender-specific WASH needs? Did your school receive support in developing/adopting latrine facility management plans? How satisfied were you/your organisation with the support received?

7. Does your school have a Menstrual Hygiene Management (MHM) plan? Does your plan include the provision of girl-friendly latrines that are specially designed to

help girls practice good menstrual hygiene? Does your plan include the provision of sanitation pads and other materials to allow girls to

practice good menstrual hygiene? Does your plan include providing facilities for sanitation pads and other materials to be

disposed of? 8. Have you or your colleagues received training/support to ensure the good state and functionality of

WASH facilities and services in your school? Are you/your colleagues aware of the Menstrual Hygiene Management (MHM) plan? Do you/your colleagues explain the available sanitation services and facilities to girls so they

know how to use these? How are you/your colleagues using the skills acquired to ensure functionality and

sustainable delivery of services? 9. Which CLTS activities were implemented in your school?

What about activities using the Play and Sports (HHETPS) approach? What about activities using the WASHSPLASH approach? Have school health clubs been established in your school?

10. Which marketing and promotional activities were disseminated in your school. Have children been targeted by hygiene communication messages? Was your school part of the open defecation campaign?

Outcomes.

11. Please tell us about access to and use of sanitation facilities in your school. For example, available latrines.

18

Do you consider that the access and use of sanitation facilities in your school has improved? If so, can you describe what triggered these improvements?

What about children’s behaviours when using the available sanitation facilities in your school?

In your view, what was the contribution of the programmes to this improvement (if any)? What other factors might have contributed to the improvement or lack of improvement in

accessing and using sanitation facilities? 12. Who is responsible for the operation and maintenance (O&M) of WASH facilities in your school?

Probe: are pupils involved in the O&M of WASH facilities? Are there any facilities that currently need repairing?

13. Is there any evidence that hygiene practices have improved, particularly handwashing with soap? Are children aware of handwashing practices? If school health clubs exist, which hygiene practices were promoted through these? How

many children were involved in these school health clubs? Are there any youth ambassadors for WASH in your school? What evidence is there that community members are aware of why and how hand-washing

is important for good personal hygiene? In your view, what was the contribution of the programmes to this improvement (if any)? What other factors might have contributed to the improvement or lack of improvement in

hygiene practices in your school and in the community? 14. Please tell us if children at your schools have contributed to the promotion of open defecation

areas? If yes, how? Which promotional activities carried out by children in your school were most effective?


15. Is there any evidence that the programmes have resulted in improved health outcomes amongst children in your school? If not: On what timescale would you expect health outcomes to improve?

16. Is there any evidence of greater access to and use of gender and disability friendly WASH facilities in your school?

17. Have girls’ school attendance rates increase? Do you believe having a menstrual hygiene management system in place contributed to

this increase? Do school girls come to school when they have their periods? Do you think that girls feel comfortable in school when they have their periods?

18. In your opinion, have hygiene practices across the community improved? 19. Is there any evidence of children in your school washing their hands with soap?

Is there any evidence of this improvement? 20. What were (if any) the unintended effects (positive and negative) produced by the two

programmes? 21. Are there any contextual or other factors that have impeded/enabled the programme to achieve its

objectives? Probe on: Did you/your school have the capacity to implement the needed facility management plan?

22. Did you/your team face any barriers when implementing the facility management plan? 23. Did you/your team face any barriers when promoting hygiene practices in your school? 24. Would you need additional support to continue to implement gender and disability friendly WASH

facilities and services in your school?

18

Final remarks

25. Anything else you would like to share with us before concluding?

Annex 8: Focus Group Guide This annex contains the topic guide used for focus groups in schools. It was contained in the Inception Report.

Purpose of focus groups in schools

Focus groups with students and teachers are being conducted as part of Ipsos’ evaluation of UNICEF’s Accelerated and Enhanced WASH Programmes in Ghana. The purpose of these focus groups is to gather insights on:

1. how effectively the school-related activities were implemented;

2. whether WASH facilities are available and being used by the teachers and children;

3. how WASH facilities are operated and maintained;

4. how children and others are promoting hygiene in their schools and communities;

5. whether the programme design and implementation met the needs of children, and particularly girls and children with disability; and any challenges faced.

There will be a total of 12 focus groups in six communities. These will be carried out in two gender segregated groups, consisting of six to ten school boys or school girls and a minimum of two teachers per group. There are some questions targeted to teachers, but their main role is to support the smooth running of the focus groups by building a more comfortable environment for the girls and boys participating in the focus groups.

The focus groups are structured in four sections:

1. Introduction: Trained interviewers/focus group mobilizers will provide an overview of the project and why the focus groups are taking place in the selected schools.

2. Warm-up: questions to get boys and girls taking about their overall experience in school and its available facilities.

3. Theme questions: 1) Facilities available and access to these facilities, 2) Menstrual hygiene management in schools, 3) hygiene in schools and hygiene awareness among pupils, 4) ODF status and 5) Knowledge and attitudes on UNICEF’s WASH programmes as well as views on how future WASH facilities and services should be implemented.

4. Final remarks.

18

Note to interviewers: The most important questions are in bold. This document is only intended as a guide and questions should be selected based on the girls and boys WASH knowledge and their involvement in programme activities.

The focus groups will be last around 2 to 2.5 hours.

Practicalities

The interviewers should ensure they have the following materials and equipment to carry out the focus groups:

Consent forms; Letter of study endorsement from UNICEF; Letter of authorisation from the Ghana Health Service; A camera or audio recorder to record the focus groups; Flipcharts/Pens/Pencils/Sticky-notes etc; Laptop to show the promotional videos developed under the programmes.

A: Introduction

Start by welcoming the students and teachers and thanking them for helping. Introduce yourself and the project:

• Since 2012, UNICEF Ghana has been building latrines and working with students and teachers on activities related to water, sanitation, and hygiene in the five most deprived regions of Ghana: the Northern, Upper East, Upper West, Volta, and Central regions. UNICEF Ghana has commissioned Ipsos to conduct a study on this programme.

• We are studying how students and adults are using toilets and hand washing spaces in schools. This means we are talking to students and teachers to find out more about which toilets and facilities are available to them and how they use these facilities. We are also talking to students about what they’ve learned about good hygiene practices.

• Today we will ask you about: your experience using the available toilet facilities in your school; your knowledge about how to use the available school facilities; and the activities you have been involved in to promote better handwashing and use of toilets in your school and at home.

• We would like to find out more about how girls and boys use schools’ facilities and their knowledge of how to use them. That is why we have separate session today in this school for girls and boys. In each session, we have 6 to 10 students and a minimum of two teachers.

Important remarks before starting with the questions:

• Be sensitive to the considerable diversity and inequalities in children’s lives, including differences based on gender, sexuality, age, ethnicity/cast, socio-economic status, language, disability, confidence levels and context.

• Discuss ethics: consent, recording, confidentiality, explain that girls and boys that do not have to talk about any issues if he or she finds difficult or upsetting.

18

B: Warm-up questions

Start asking the group about the overall experience in school and the sanitation and hygiene facilities available.

Let’s begin by getting to know you and your school. At your school, what ages are the students? And what school years are you in?

a. Where do you live? Do you live far from your school?

b. How do you get to school?

c. Are there other schools in your community? If yes, why do you come to this school?

d. How do you feel about the school you attend?

What do you think of your school and its sanitation and hygiene facilities? Are there things you really like about your school sanitation and hygiene facilities (e.g. are toilets clean? Is soap or ash available?)

e. Have sanitation and hygiene facilities in your school improved in the last years? If so, how and when?

f. Where are the sanitation and hygiene facilities? Are any of these facilities new?

g. How are these facilities maintained? And how often?

Explain what we mean by maintain: cleanness of the facilities, availability of cleaning tools, equipment and materials for operation and maintenance activities, periodic emptying of septic tanks, checking of soak and drain fields (not applicable where toilets are connected to sewerage networks) etc.

h. Who is responsible for the operation and maintenance (O&M) of WASH facilities in your school? Probe: are pupils involved in the O&M of WASH facilities?

i. Are there any problems with sanitation and hygiene facilities that needed repairing or replacing?

j. Where do students normally go when you need to defecate in school?

k. Teachers (After students have responded): What do you think about available schools’ facilities (also, are these gender-segregated)? Probe: What do you think of the design, location and convenience of WASH facilities in your school?

Is everyone at the school able to use the toilets and hand washing facilities in the same way? Prompt by asking: what about children with disability? Is there anyone else at the school who is not able to use the toilets and hand washing facilities in the same way?

Has anyone visited your school to talk to students (and teachers) about sanitation and hygiene?

l. What did they talk about? What did they do? What did you learn? Probe:

18

i. Were you involved in any sports activities that made you more aware of good hygiene practices?

ii. Were you involved in any other games or activities that made you more aware of good hygiene practices?

m. Teachers (After students have responded): Were you aware of the Enhanced/Accelerated Programme? Were school staff and students consulted in the design of the programme? How were staff and students involved in the programme delivery?

How do you feel about the community you live in? Have you seen any improvements in the recent years?

n. Do you have access to safe and adequate drinking water in your community?

C: Theme 1 – Facilities available and access to these facilities

Do you use the schools’ toilets? Are these only for boys/girls? How many times a day do you use them? Do you need to queue to use them?

o. Can you tell me what the toilets in your school look like? Show pictures of latrines.

p. Probe: Do the toilets in your school have:

i. Ramps?

ii. Rails that students can hold onto?

iii. Separate squat holes that students who have walking difficulties can use?

Are there separate toilet facilities for girls and boys in your school?

q. Are they situated in a convenient location that is close enough to the school?

r. Do they provide enough privacy and security for you to feel comfortable using them? Pay attention to differences in responses between girls and boys.

Do you feel comfortable using the toilets in your schools? Why/why not? Pay attention to responses of different types of students.

Are the latrines clean? Who cleans the latrines? Note responses received during warm-up questions.

Is toilet paper available in your school latrines? If not, what do you use to clean yourself after using the latrine?

Do you know how to use the available latrines in your schools? Do you use them or do you prefer going somewhere else? Do you know how to squat and defecate into the hole without any faecal element left on the edges of the squat hole?

s. Probes for girls only:

19

i. Do you feel comfortable using the available facilities when you have your menstruation?

ii. Do you have a bin available in the female latrines?

iii. Are there changing rooms available for you to use and are sanitary pads provided?

Are there any materials near the toilets that show you how to use them correctly?

Is there anything you don’t like about the available facilities in your schools? (e.g. dirty latrines, not enough latrines, lack of gender segregated latrines, lack of doors/locks, too far away)

t. How can the situation be improved?

Is there anything that prevents you from using the latrine? Probe on: availability of latrine, security of latrine, access, attitudes, knowledge.

Teachers: Could you explain how the facilities are maintained?

u. How frequently are the latrines cleaned? By whom?

v. How could children contribute more to the maintenance of the school’s facilities?

w. Is there a facility management plan available in your school? When was this put in place? What does the plan include? How effective is this?

D: Theme 2 – Menstruation Hygiene Management

Are there private latrines available only for girls?

x. If not, how would you describe the latrines?

Do you feel comfortable using the latrines at school during your menstrual period?

Did you have access to the materials you needed for managing your last menstrual period?

y. Which materials do you tend to use when you have you period? Where do you get them?

z. Did you have access to materials when needed for managing your last menstrual period?

aa. Are latrines/ toilets equipped with sanitary products? Is there any place else at school where you can get sanitary products if you need them? If yes, which ones?

bb. What do you do with menstrual materials after you’ve used them? Prompt: How do you usually dispose of menstrual materials after use?

Have you ever missed school because of your menstrual period? If so, why?

cc. During your menstrual period, did you feel like going to school?

19

dd. Why not? What can be improved in school for you to feel comfortable going to school?

How do you manage your menstruation?

ee. Do you know when will you have your period next? How do you calculate it?

Do you feel discomfort and/or pain when you have your period?

ff. How do you manage your menstrual cramps/pains?

E: Theme 3 – Hygiene and hygiene awareness

Where can you wash your hands in your school?

gg. Can girls/boys with reduced mobility wash their hands in the handwashing basins? Are handwashing basins accessible through ramps? Are rails provided for pupils to hold on to?

Do you normally wash your hands in school?

How do you wash your hands? Prompt by asking if they use soap/ash.

Does the hand-washing space in your school look like any of the following pictures? Show pictures of hand-washing spaces and ask to visit the hand-washing spaces at the end of the session.

What do you use to wash your hands at school? Prompt: Do you always find soap/ash when you wash your hands in school?

a. Teachers (After students have responded): How many hand-washing facilities are available and how often are these used by pupils? What is the water source for handwashing?

Do you know why you need to wash your hands? If yes: why is it important?

Is there anything that prevents you from washing your hands with soap/ash? Probe on: availability of water, availability of soap/ash, access, attitudes, knowledge. Ask if they could demonstrate how they wash their hands at the end of the session.

Does your school have a school health club? Are any of you members of this school health club?

hh. If yes, what do you do in the school health club/what does the school health club do?

ii. How do you feel about having school health clubs, do you see any benefits from them?

jj. What do you like/dislike about being part of the school health club?

kk. What have you learnt through your membership of the school health club?

ll. Teachers: How many boys and girls in your school are members of the school health club?

19

Does your school have a Children and Youth Ambassadors for WASH (CYAWASH) programme? What do the ambassadors do? Are any of you members of this programme (WASH ambassadors/advocates)?

mm. If yes, what actions have you been encouraged to take in your role as a WASH ambassador/advocate? What actions have you taken? Do you think these actions have been effective? What do you like/dislike about being a CYAWASH ambassador?

Have you received information recently on how to wash your hands or how to use toilets/latrines? Who provided the information/how did you find out? If yes, do you remember if they showed you a video, leaflet etc? What do you think about the materials they showed you?

Have you seen any of the following information before? Show three/four leaflets and if possible video (The epidemic, Sharp kid episode 1)

nn. If video is shown, ask they feel about it? Prompt on comparing their community to ones in the video.

F: Theme 4 – ODF status

Where do you defecate when you are not at school? Do you have a latrine in your home, or do you use a communal latrine, or do you defecate outdoors?

oo. How would you describe the state of the latrine?

pp. Who in your family uses the latrines? If some do not, ask where they defecate.

Do you teach your parents, family and community members about where they should defecate? If yes, how do you teach them?

Do you wash your hands when you are at home? When do you wash them (e.g. before dinner?)?

qq. Where do you wash your hands at home?

19

rr. Do your parents and siblings wash their hands? If not, why not?

Do you teach your parents and family about why and how they need to wash their hands? If yes, how do you teach them?

G: Theme 5 – Knowledge and attitudes on UNICEF’s WASH programmes

Are you aware of any organisations that have been running WASH programmes in your school? If yes, which organisations? What were their roles in improving access to WASH? If no, explain the role of UNICEF and its programmes.

What do you think about these programmes? Have they improved your access to WASH facilities (including latrines, wash basins, clean water)? Have the programmes changed the way you use WASH facilities?

ss. What facilities did your school have before the programmes started? How does this compare to the current situation?

tt. How have these changes affected you? Probe: Attendance at school, knowledge about good personal hygiene, health.

What more could the programme do to ensure all students in your school can access and properly use latrines and hand basins?

What more could be done in your community (outside of school) to ensure that all people can access improved WASH facilities and all people practice appropriate handwashing and defecation?

Teachers: Has your school’s administration supported the programmes? How? If no, why not?

Teachers: How could do programmes have been improved?

Teachers: Are there any contextual factors that the programme has not taken into consideration that have impacted on its effectiveness?

In your view, how should WASH facilities and services be implemented in the future? Probe: Any thoughts on how the WASH facilities should be designed to ensure better access and use? Teachers: Any thoughts on how the WASH facilities and services could be designed to ensure better maintenance and operationalisation?

Final remarks

Carry out the two observation activities: 1) observe hand washing spaces in schools and 2) how girls and boys (around 4 pupils) wash their hands at the available hand washing spaces.

Please sum up the main points of the discussion, compliment and thank the students and teachers.

Ask attendees if they would like to add anything else or if they have any questions.

19

Annex 9: Ethical considerations The evaluation was conducted in line with the UNEG Norms and Standards (2016); the UNEG Ethical Guidelines for Evaluation (2008); the UNICEF Procedure for Ethical Standards in Research, Evaluation, Data Collection and Analysis (2015); and UNICEF’s Ethical Research Involving Children guidance (2013). Ipsos is a member of the World Association for Social, Opinion and Market Research (ESOMAR). Ipsos Ghana has a Government research permit that allows Ipsos to conduct research in all the districts in Ghana. Ipsos obtained ethical approval to conduct this specific study from Ghana Health Service Ethics Review Committee on 9th July 2019.

All Ipsos staff are trained on the ESOMAR code of ethics and sign forms committing to adhere to the code. In adherence with these codes and the policies of the Ethics Review Committee, participants were treated with respect for dignity and diversity and in line with do-no-harm principles. Further, research tools, sampling protocols, data collection protocols, and data processing and storage protocols are also designed to respect participants’ rights, including rights to self-determination regarding their participation and use of their data, and rights to confidentiality. Further, the data collection team were trained on research ethics and appropriate treatment of study participants and were required to comply with these policies and procedures.

Participation in this study was voluntary and based on informed consent. Only willing participants were interviewed in an appropriate language after administration of an informed consent form. The above-named codes require that sufficient information is given to the participant (and for participants under 18, their parent/guardian) for them to reach an adequately considered decision about giving such consent. This was done using the information contained in the consent form, which was translated into the appropriate local languages. Participants who are below 18 years of age are considered minors and therefore parental/guardian consent was sought in the first instance, before the consent of the minor was sought and before the focus groups took place; both minors and their parents/guardians signed consent forms. This is in accordance to the ESOMAR Research Codes and Guidelines for Interviewing Children and Young People and UNICEF procedures for Ethical Research Involving Children.

Further, during data collection, the team sought permission from the district assembly/municipal authorities and from school headteachers within the sampled area. This allowed the team to carry out household interviews within the community and in schools respectively.

Participants were assured that their responses would be kept confidentially and that no personal-identifying information will be included in the report. All personally identifying information collected is being kept in the separate file and will be deleted at the end of this evaluation. Findings have been anonymised.

19

Safeguarding of minors was also ensured through implementation of child protection protocols which adhere to the Children’s Act (Act 50) of 1998 (Part 1, section 17), and the research teams received training on this. Focus groups with students were conducted in an open space where the participants and research teams would be seen but not necessarily heard by others. Safeguarding protocols required that any assurance of confidentiality also involves explicit mention of the limits to this, particularly on the responsibility to be prepared to act sensitively to safety concerns of children. A reporting protocol and process for referring concerns to the Department of Social Welfare was in place. Research teams were also trained to handle distress or discomfort of participants, including giving all participants the option to discontinue participation in the research at any time.

Reference was also made throughout this evaluation to the Revised Evaluation Policy of UNICEF (2018), UNICEF-adapted UNEG Evaluation Reports Standards (2017), the Global Evaluation Reports Oversight System Handbook (2017), and the Market Research Society Code of Conduct. The evaluation upheld the following key principles:

▪ Independence: The evaluation is free from bias.

▪ Impartiality: The external evaluators have given a balanced view of both strengths and weaknesses observed.

▪ Credibility: The evaluation is based upon reliable data and observations.

▪ Conflicts of interest: The external evaluators avoided any conflicts of interest that threatened the credibility of the evaluation.

▪ Accountability: External evaluators recognise their responsibilities and have accurately represented themselves and their data, and exercised prudence throughout the evaluation.

Annex 10: Programme expenditure The table below presents the expenditure data for the Enhanced programme, provided in January 2020.

Activities and sub-activities Budget assigned to activities and sub-activities

1. Water and Sanitation services $ 5,059,523.97

1.1. WASH Infrastructure in Schools $ 2,671,162.95

1.2 WASH Infrastructure in Health Facilities $ 1,183,505.28

1.3 Household Sanitation Facilities in communities $ 1,204,855.74

2. Behavioural change activities $ 3,138,105.84

2.1 CLTS Implementation $ 2,593,362.10

19

2.2 Handwashing $ 204,419.92

2.3 Schools Behaviour change $ 340,323.82

3. Enabling environment $ 4,602,904.84

3.1 Capacity building and training $ 2,028,907.56

3.2 Support and establishment of ME systems and practices $ 116,166.36

3.3 Research, knowledge building and management, dissemination and coordination

$ 1,446,792.63

3.4 Engagement with private sector $ 670,285.88

3.5 Improving financing options $ 262,476.89

3.6 Gender mainstreaming activities community level $ 78,275.52

4. Cross sectoral (Operation, admin, C4D, gender, HACT) $ 1,143,266.82

Total budget allocated to Enhanced programme activities $ 13,943,801.47

Source: UNICEF in Ghana (2020)

The evaluation team comprises staff from Ipsos MORI, FCG Consult, Ipsos Ghana, and Ipsos East Africa. Biographies of core team members are as follows.

Jessica Bruce – Project Director Jessica is Director of Ipsos’ Sustainable Development Research Centre in London, where she works across evaluations and primary research related to the SDG agenda. She is leading design of an evaluation training programme for the British Council, as well as evaluations of various development programmes, including DFID’s Economic Statistics Programme, DFID’s Aid Match II programme, the World Bank’s Transformative Carbon Asset Facility, and Earthwatch’s environmental sustainability training programme. She has also recently been a senior evaluator for Ipsos’ evaluations of Plan International UK’s South Asia WASH Results Programme, UNESCO’s Global Education Monitoring Report, and the World Bank’s Partnership for Market Readiness. Her research experience includes a study on civil society and individual giving in four African countries for the Charities Aid Foundation and a study on policymakers’ attitudes to gender equality in Kenya, Senegal and India for Women Deliver. She recently edited Ipsos’ publication ‘Understanding Society: A Woman’s World’. Jessica’s previous clients include the UK’s Department for Culture, Media and Sport, the Private Infrastructure Development Group, the African Development Bank, the Caribbean Development Bank, the GSM Association, and national and municipal governments. She also worked for two years carrying out grassroots development projects in

Annex 11: Evaluation team

19

rural Benin as a Peace Corps Volunteer. She holds an MSc in Economics from the University of Nottingham and a BSc in Economics and International Political Economy from the University of Puget Sound. Yaw Sarkodie – Team Leader Yaw is a WASH expert with FCG Sweden. He has more than 22 years working experiences in multi-disciplinary environment, information and knowledge management, poverty reduction programs, WASH planning and implementation, strategic planning and management, human resources management, project management, rural development and stakeholder participation, training and capacity building, monitoring evaluation and reporting. He has proven ability to constitute and lead teams composed of geographically diverse experts originating in various organizations and has rich experiences in policy and institutional development, project design, preparation appraisal in both Ghana and Tanzania. He has worked in the WASH sector in Ghana, Burundi, South Sudan, Niger and Sierra Leone and was also a member of the Technical Working Group on Sanitation for the Post 2015 development agenda set up by the Joint Monitoring Program (JMP); and currently a member of the Global Expert Consultative Group on Global Monitoring of WASH Affordability-an expert team formed by JMP to recommend monitoring indicators for affordability. Sarkodie was Ghana's focal point for the preparation and submission of the Pan African Water and Sanitation Monitoring and Evaluation Report. He has been actively involved in sustainable WASH services delivery in Ghana, as well as contributing to global discourse on WaSH issues. He also has extensive experience in coordination, planning and programming. Victoria Hildenwall – Quality Assurance Expert Victoria Hildenwall replaced Henning Goransson as Quality Assurance Expert in September 2019. Currently Head of the M&E Unit of FCG Swedish Development, Viktoria has the overall responsibility to oversee and assure quality of all services delivered by the Unit (including both processes and products). Viktoria has worked extensively on evaluation quality over the years. As an in-house evaluator at the Swedish Agency for Development Evaluation (SADEV), she conducted a meta-evaluation of UNDP´s decentralised evaluations (that is, evaluations commissioned and implemented by the organisation´s country offices) to assess their overall quality in terms of methodological rigour (validity of findings and credibility of conclusions drawn). As an evaluator, her work has spanned over programme analysis (identification of Theory of Change) and evaluation design (methodology); qualitative and quantitative data collection and analysis; development of assessment frameworks with criteria and indicators; different approaches to Results-Based Management, organisational assessments and methods for assessing multilateral development cooperation. Adriana Rodriguez – Project Manager Adriana is a Consultant in Ipsos MORI’s Policy and Evaluation Unit in London. Adriana is responsible for delivering a range of evaluation studies for public sector clients including national government departments, Directorate-Generals of the European Union and international organisations. She is an experienced evaluator, applying mixed qualitative and quantitative methods to the evaluation of complex policies and interventions. She holds a Bachelor’s degree in International Relations with Economics from Westminster University, as well as a MSc in European Public Policy from University College London. During her studies, she conducted two major research projects called: A comparative study of the European Development Policy in Angola and Ethiopia and To what extent is European Development Policy effective and coherent? Gerald Njoroge – Fieldwork Quality Assurance and Data Analysis Specialist Gerald is a Research Executive in Ipsos’ Public Affairs team in Kenya. He has implemented and managed multiple multi-country projects across Sub Sahara Africa, including for UN agencies, USAID development partners among others. He has conducted several WASH studies, including for a gender empowerment WASH

19

programme in Rwanda on behalf of Global Environmental and Technology Foundation, the Kenya RAPID WASH programme water management system assessment, and a baseline study for a WASH programme in Somalia with displaced communities commissioned by the Danish Refugee Council. Gerald has experience working within the operations unit as a Field Interviewer, Data Service Team leader and as an Operations Project Coordinator in running multi country studies across 15 Markets in Sub Sahara Africa. Gerald is currently pursuing a Master of Public Administration degree at the University of Nairobi and holds a Bachelor of Arts Degree in Political Science with a bias in International Relations and Social Administration from the Makerere University. William Mensah – Fieldwork Manager William is the Service Line Lead for Ipsos Ghana. He has over 30 years’ experience in market research in the West African sub-region. He has coordinated several quantitative and qualitative studies on a large scale in Ghana, Cote d’Ivoire, Sierra Leone, Senegal, Liberia, Republic of Togo, Republic of Benin, Burkina Faso, Niger, Nigeria, Uganda and DRC, playing the role of Project Manager. He has worked for both multinational, international and local clients such as; Nestle, Unilever, Cadbury, BAT, Standard Chartered Bank, Guaranty Trust Bank, International Olympic Committee, GlobeScan, CEPA, PURC, Ministry of Roads & Transport, Netmark, USAID, World Bank, Community Water and Sanitation Agency (CWSA), Ministry of Finance, The Independent Evaluation Committee of the World Bank, Intermedia Institute, ABT Associates, World Cocoa Foundation etc. His key competencies are proposal and report writing and fieldwork coordination which includes training and briefing and quality control. Judith Ofori – Sociologist & Community Development Expert Judith is a Gender Expert/Community Development Socio-Economist with FCG Sweden with 30 years of experience. She is a champion of community development, an advocate for gender equality and the empowerment of women, marginalised communities and vulnerable groups. She has, over the past 25 years, worked with governments, NGOs, multilateral/bilateral institutions and donor entities including the African Development Bank (AfDB) (in fifteen African countries), UN Agencies, the African Union (Ghana, Liberia, Nigeria, Ethiopia), European Union (Germany, Holland, Mexico, Nepal, Peru, Russia and Vietnam), DFID (Ghana, Nigeria, Sierra Leone), SIDA and the World Bank (Ghana). For the past twenty (20) years; she’s had ‘hands-on’ practical experience in project planning and management; working in rural and community development for International Donor Agencies; public, and voluntary sector organisations in the UK, Europe and over twenty (20) African countries including Ghana. Andrew Cleary – Statistics and M&E Specialist Andrew is Head of the International Research Methods Centre at Ipsos MORI in London with overall responsibility for the design and delivery of methodological aspects of Ipsos MORI’s multi-country surveys, particularly on sampling, weighting and data collection methodologies He focuses on project management and effective delivery of large-scale survey projects, such as on his work to develop and enhance Ipsos’ face-to-face data collection package (iField) for random probability surveys. Andrew has contributed to an array of EU- and UK-based projects, where his areas of expertise include sampling and weighting, including on random probability and quota sample designs and sampling hard-to-reach groups, business and general public surveys, all data collection modes, longitudinal surveys, and methods to reduce non-response. Virginia Nkwanzi – Statistics and M&E Specialist Virginia leads Ipsos’ Public Affairs practice for Sub-Saharan Africa and is based in Ipsos Uganda. She is a Social and Marketing Researcher with over 16 years’ experience. Her expertise covers socio economic research, Monitoring and Evaluation, product development, gender mainstreaming, financial services, trends research,

19

health care research and many others. She has conducted research for different organizations both local and international; survey design, fieldwork execution, survey tools design, data analysis and reporting and dissemination for both qualitative and quantitative.

Article I. TERMS OF REFERENCE WASH Summative Evaluation:

Impact of Enhanced and Accelerated Sanitation Programmes - International Evaluation Firm -

Title WASH Evaluation: To Determine Achievement and Effectiveness of Enhanced and Accelerated programmes

Purpose • Accountability: to provide rigorous evidence on the effectiveness of WASH programming from the Accelerated and Enhanced grants after six years of implementation to both the donors (vertical accountability) and the population groups affected by the programs in question (horizontal accountability).

• Organizational Learning: this evaluation is expected to inform future WASH programme design not only within UNICEF Ghana, but also among other in-country stakeholders with a vested interest in WASH. By identifying which of the Programme’s envisaged objectives (included in the Management Performance Framework) were not achieved, the evaluation is expected to provide the Programme managers with viable corrective and scalable strategies to overcome challenges

• Targeted Learning: To inform Programme Managers prioritization of future activities and strategies geared towards the implementation of a more gender-responsive and gender-

transformative WASH programming in the future Location Ghana Duration 9 months from signing contract Start Date December 2018 Reporting to WASH specialist

1. Evaluation Object The GoG-UNICEF WASH Programme was established in 2012 (with a new programme initiated in 2018) within the framework of the Ghana shared growth and Development Agenda (GSGDA), and the major priorities of the Water and Sanitation Sector in Ghana. The programme is, therefore, aligned with the targets of the National Environmental Sanitation Strategic Action Plan (NESSAP) and the Water Sector Strategic Development Plan (SSDP), and draws from the National Water Policy, and the Environmental Health and Sanitation Policy. The Programme is also based on the Situation Analysis carried out in the formulation of the United Nations Development Assistance Framework (UNDAF) in Ghana, as well as global initiatives such as the Sanitation and Water for All (SWA) partnership, to which Ghana is a partner. To this end, the programme focuses on equitable and sustainable access to water and sanitation services. In this regard, the priority emphasis of the programme is increasing the use of improved, safe sanitation facilities, primarily through the Community-Led Total Sanitation (CLTS) approach, and the promotion of improved hygiene behaviour. To this effect, with financial assistance from Global Affairs Canada, UNICEF Ghana has been implementing the

Annex 12: Terms of Reference

20

Enhanced and Accelerated sanitation programmes in rural communities, schools and health facilities since 2012. The Enhanced Water, Sanitation and Hygiene (WASH) Services in Schools and Communities in Ghana project was initiated in March2012 and ends in December 2018. The project focuses on sanitation, hygiene and water as it relates to the development of children and youth in Ghana. The project is based in the five most deprived regions of Ghana, the Northern, Upper East, Upper West, Volta and Central Regions with

the aim of reducing inequities in access to improved water, sanitation and hygiene (WASH) facilities and associated behaviours. In the Northern Region, the development situation is generally even more challenging than that of Ghana broadly. Here a different program is being implemented: the Accelerated Sanitation in Northern Ghana Programme. This second program fits within the framework of the GoG-UNICEF WASH programme with a focus on scaling up sanitation in small towns and districts towards achieving health outcomes. The ultimate outcome of the Accelerated Sanitation in Northern Ghana Programme is to observe improved health and well-being of boys, girls, women and men in the Northern Region of Ghana. The project is scheduled to be conclude by June 2019. The two programmes (The Enhanced and the Accelerated programmes) have similar approaches and methodologies to improve sanitation in Ghana. The only difference is that Accelerated focus on only Northern Region while Enhanced is covering five (5) regions in Ghana. However, both grants are focusing on nation-wide engagement to improve hygiene behaviour of target populations in communities, schools and health facilities. Enhanced programme (covering five regions) is facing its closing and UNICEF is planning to do the endline evaluation to know the achieved progress compare to targets during the entire programme duration. Accelerated programme will be closed in June 2019, and to the two evaluations will be combined into one single process with separate reports (and separate evaluation management responses) and a combined summary report. This combined evaluation will provide the data analysis and technical supports for the future WASH programme planning in the longer term. UNICEF proceeded baseline surveys for Accelerated grants to find out baseline information of key small towns and district WASH indicators on access to water and sanitation facilities in the 80 small towns and 13 districts, as a basis for further streaming of the project focus and strategies. And this served as a reference for the project mid-line surveys. The programmatic approaches adopted across the two programmes are outlined in section 4. Collectively the two projects operate across 27 districts in 5 regions (a detailed list if provided in Annex A), reaching a population of over 750,000 people, living in over 1,500 rural communities and 105 small towns. Over 100,000 school children have been targeted across 320 schools, whilst over 80 Health Facilities have been targeted. WHO-UNICEF Joint Monitoring Programme (JMP), Multi-Indicators Cluster Surveys (MICS) and the Demographic and Health Survey (DHS), and the National Population and Housing Survey will also provide useful references and trends in access to and use of water and sanitation services, which provide guidance to conduct WASH end-line evaluation. Project monitoring data will also be used to identify samples and to assess the achievement of intended targets.

2. Evaluation Purpose The main purposes of this WASH Evaluation are:

• Accountability: to provide rigorous evidence on the effectiveness of WASH programming from the Accelerated and Enhanced grants after six years of implementation to both the donors (vertical accountability) and the population groups affected by the programs in question (horizontal accountability).

• Organizational Learning: this evaluation is expected to inform future WASH programme design not only within UNICEF Ghana, but also among other in-country stakeholders with a vested interest in WASH. By identifying which of the Programme’s envisaged objectives (included in the Management Performance Framework) were not achieved, the evaluation is expected to provide

20

the Programme managers with viable corrective and scalable strategies to overcome challenges • Targeted Learning: To inform Programme Managers prioritization of future activities and

strategies geared towards the implementation of a more gender-responsive and gender- transformative WASH programming in the future

20

Article II. 2.1 Primary Audience, Utilization and Dissemination The Government of Ghana, specifically the Minister of Sanitation and Water Resources (MSWR) and its agencies (notably the Environmental Health and Sanitation Directorate (EHSD), Ministry of Finance (MoF), the Office of the Head of Local Government Service, and the National Development and Planning Commission (NDPC), alongside participating regions and Metropolitan, Municipal and District Assemblies (MMDAs), are the primary audience of the impact evaluation. The evidence generated from this evaluation will help inform the MSWR on the impacts of providing continued support to this vulnerable target population, and help understand the differential impacts of the programme across the country to help draw further political support to expand the WASH programme to a national level.

Evaluation Users Evaluation Uses Government of Ghana (predominantly Ministry of Sanitation and Water Resources, along with other relevant Ministries, Departments and Agencies)

To inform future policy and programming decisions, and contribute to the evidence base for future strategy and policy development

UNICEF Ghana Country Office To inform future programme design and implementation of WASH programming, including a more systematic mainstreaming of gender into WASH programmes

Global Affairs Canada To inform future engagements with UNICEF and other partners in the WASH sector, by identifying strategies that would allow to address the implementation and coordination challenges observed in the field.

Project implementing partners (notably participating Metropolitan, Municipal and District Assemblies, Regions and NGOs)

To inform the identification of future activities and strategies to adopt within the scope of their respective WASH programmes

WASH Sector Actors in Ghana ,such as the Coalition of NGOs in Water and Sanitation (CONIWAS), World Bank, Embassy of the Kingdom of the Netherland, and USAID

To increase knowledge on effective WASH strategies that could either be implemented or funded in the future

The work will add to the body of evaluation on WASH programming for the overall WASH sector, both within and outside of Ghana.

3. Evaluation Objectives The overarching objective of this evaluation is to answer whether the programme is improving sanitation and hygiene for community people and school children, if the projects met their targets and what could have been done differently to meet targets in the future. In doing so, the evaluation will allow the identification of lessons learned and will yield a series of strategic and operational recommendations on future WASH programme design within and outside of UNICEF. Further details are found in the Key Evaluation Questions section below.

4. Evaluation Scope Thematic scope: the evaluation will focus on the following Programme areas:

• Rural Sanitation – Community Led Total Sanitation (CLTS), Sanitation Marketing, Sanitation Financing, Social Norms Campaign (for Accelerated Sanitation this is also implemented in small towns)

• Hygiene – mostly aligned with implementation of CLTS at community level, the social norms campaign also has components of hygiene promotion; some limited focus on Household Water Treatment and Safe Storage (HWTS) (as a component of water safety planning) is included

• WASH in Schools – construction of WASH infrastructure, training on operation and maintenance, and implementation school hygiene promotion (including menstrual hygiene management)

• WASH in Health Facilities – construction of WASH infrastructure with operation and maintenance training

20

• WASH Enabling Environment – capacity support to participating regions and districts, support on local and national level information systems, law enforcement development, environmental assessments; national level policy and strategy development; engagement of WASH civil society in social accountability; gender mainstreaming in programming

• WASH Knowledge Management – support to sector learning events, documentation of programme components, and completion of research on WASH interventions

Geographical scope: The assignment will involve desk review of secondary data as well as field data collection in the five most deprived regions of Ghana, the Northern, Upper East, Upper West, Volta and Central Regions. Where relevant, national level engagement should also be included. Chronological scope: the evaluation will cover the Programme implementation between March 2012 and June 2019.

5. Evaluation Context The evaluation is funded by the project budgets and was agreed as an end of project activity with the project donor. Mid-term reviews have been completed for both projects, for Enhanced in 2015 and Accelerated during 2018.

6. Evaluation criteria This evaluation will be guided by the following criteria: effectiveness, efficiency and relevance.

7. Evaluation Questions Core research questions are detailed as follows:

1. Effectiveness 1.1. Overarching Questions

1.1.1. To what extent have the two projects met the intended targets in terms of service delivery in the project duration?

1.1.2. What were (if any) the unintended effects (positive and negative) produced by the two projects?

1.1.3. To what extent have the projects effectively addressed the specific needs of women and girls? 1.1.4.How should future programming be designed in order to overcome experienced challenges? 1.1.5.To what extent have health outcomes improved in the project areas?

1.2. WASH in rural communities 1.2.1. What is the access to household sanitation in the targeted regions and districts? 1.2.2. To what extent do community members and school children know about the hygiene

practices which the project tried to promote? 1.2.3. To what extent do community members have appropriate HWWS facilities? 1.2.4. What is the access to safe water supply (including HWTS) in the targeted rural communities?

1.3. WASH in small towns 1.3.1. What is the access to household sanitation in the targeted small towns and districts and are

the facilities in use? 1.3.2. How is faecal sludge managed in the targeted towns? 1.3.3. What do community members and school children know about hygiene and do they have

appropriate HWWS facilities? 1.3.4. What is the access to safe water supply (including HWTS) in the small towns?

1.4. WASH in institutions (Schools and health facilities) 1.4.1. To what extent do school boys and girls have access to the school sanitation facilities?

1.4.2.To what extent are the school sanitation facilities gender- and disability- friendly? 1.4.3. How does access to WASH facilities in school in rural communities compare with schools in

small towns? And what impact does it have on ODF status?

20

1.4.4. What proportion of schools have facility management plans in place? 1.4.5. What is the level of knowledge on menstrual hygiene management and hygiene behaviours?

1.4.6.Are there WASH facilities in health centres and CHPS compounds? 1.4.7.What is the functionality status of WASH facilities in institutions (schools and health facilities)?

1.5. District level WASH access 1.5.1. What is the number and % population using an improved source of household sanitation in

the targeted districts? 1.5.2. How many certified ODF communities are there within each one of the targeted districts?

1.5.3.What is the number and proportion of population living in ODF communities? 1.5.4. What is the number of people (in the targeted areas) using household water treatment and

safe storage systems? 1.5.5. What is the number and % population (in the targeted areas) practicing hand washing with

soap at critical times? 1.6. Governance / Enabling environment

1.6.1. What plans exist at the regional level for WASH implementation in the 5 regions? 1.6.2.What systems for accountability exist at district levels? 1.6.3.What monitoring systems exist for tracking WASH results at the regional and district level? 1.6.4.Are appropriate systems for sanitation law enforcement established?

1.7. Knowledge Management 1.7.1. Have the knowledge management components of the project improved understandings?

2. Efficiency 2.1. Overarching Questions

2.1.1. To what extent have the projects delivered the desired targets maximizing the human and financial resources available efficiently?

3. Relevance 3.1. Overarching Questions

3.1.1. In which cases were project targets not met, and what could have been done differently to enable them to have been reached?

3.2. Knowledge Management 3.2.1. Has research been appropriately designed to answer sector knowledge gaps?

In answering all these questions, the external evaluators will adhere to the UN Norms and Standards for Evaluation, as well as to the UNICEF’s revised Evaluation Policy, UNICEF Procedure for Ethical Standards in Research, Evaluations and Data Collection and Analysis and UNICEF’s Evaluation Reporting Standards as well as the Guidelines on the Integration of Gender Equality and Human Rights into Evaluation.

8. Methodology The evaluation shall be completed via a mixed methods approach. The programme will draw representative samples from the project target populations stated above. A significant amount of information is available in secondary data sources which should be reviewed, this includes monitoring data, research reports, consultant reports amongst others (see below). Key informant interviews should be completed nationally and in targeted regions. A survey of targeted households, schools, and health centres should be completed using a random sample approach, with disaggregation between households in rural communities and small towns. The survey sampling should also form the basis for key informant interviews with stakeholders in MMDAs that fall within the sample and Focus Group Discussions in communities/schools/health facilities. The programmes shall be evaluated over the full period i.e. Enhanced from 2012-2018, and Accelerated from 2015-2019. The assignment will involve desk review of secondary data as well as field data collection in the five most deprived regions of Ghana, the Northern, Upper East, Upper West, Volta and Central Regions. It is expected that the data collection will employ various approaches including observation, household surveys,

20

focus group discussion, structured and semi-structured interviews. Secondary data will also be analysed from national data sources and existing monitoring systems such as District Health Information Management System (DHIMS), Education Management Information System (EMIS) and the Basic Sanitation Information Management System (BaSIS). In addition, project monitoring systems, project implementation reports, and research reports should also be assessed. Specifically, it is expected that the methodology for data collection will include the following components:

• Desk review/secondary data: This task will identify existing data on the key indicators identified at national and MMDA levels. Data on indicators related to the Performance Management Framework will have to be sourced through desk reviews of impact-related documents. This task will also provide scoping guidance in the design of the key questions for the field assessment phase. This task will entail the following:

o A comprehensive review of documents such as project proposal, baseline report, interim reports, mid-term reviews, UNICEF-CLTS Database, UNICEF-WASH in Institution Database, trip reports and related documents, country water and sanitation policies and strategies, national norms and directives on water and sanitation and others useful documents will have to be reviewed (full extent to be determined in the inception phase).

o Other information can be sourced from participating regions and MMDAs, in consultation with focal agencies such as School Health Education Program (SHEP Coordinator) / Ghana Education Services (GES), Waste Management Division, Regional Environmental Health Office (REHO) / Municipal Environmental Health Office (MEHO), etc.

• Quantitative data collection: It is expected to collect data from all relevant sources for identified rural communities, small towns, schools and health facilities. Field observation will have to be carried out in terms of validating infrastructure constructed, facilities for hand-washing and menstrual hygiene. This will also include a household level survey using the standard WASH sector questionnaire.

• Qualitative data collection: The exercise will include semi-structured key informant interviews based on appropriately designed questionnaires, gender-segregated focus group discussions (mainly on qualitative data/contextual information), and structured observations particularly for the hygiene related indicators. These shall be collected to align with communities/schools/health facilities visited as part of the survey. The information should capture the gender dimensions of the situation, as well as other specific vulnerabilities within the society

• Sampling: It is expected to design a sampling strategy for the data collection and include it in the proposal for review and discussion with UNICEF. The sample will have to respond to the needs of the survey and be representative of the overall project population (as stated above). It will have to allow for significant comparisons in terms of access to water and sanitation between the rural communities, small towns, schools and health facilities, gender differences/disparities in accessibility. Rural communities, small towns, schools and health facilities shall be selected at random from those targeted by the programme. Subsequently, households and beneficiaries within towns/rural communities/schools/health facilities should also be selected at random.

In the submitted proposal, the consultancy should provide details on how data will be analysed. However, it is anticipated that quantitative data be analysed using a programme such as SPSS or Stata, with results presented in a way which could be easily comparable to baseline surveys. Qualitative data should be coded on specific themes to enable key trends and themes to be identified. The evaluation design will have some limitations in terms of degree of accuracy of household surveys, and also that some interviewed stakeholders may not have been engaged in the projects very long, partially due to staff turnover within a range of stakeholders. Limitations should be clearly stated in the proposal document.

9. Work Plan An evaluation reference group will be formed consisting of the WASH and M&E teams of the UNICEF Ghana country office, the WCARO Regional Evaluation Adviser, and representatives of MSWR and Global Affairs Canada.

20

9.1 Roles and Responsibilities The assignment will require extensive collaboration with the Ministry of Sanitation and Water Resources (MSWR) via the Environmental Health and Sanitation Directorate (EHSD), Community Water and Sanitation Agency (CWSA), Ghana Statistical Service, the Ghana Education Service – School Health Education Programme (GES-SHEP), Ghana Health Service (GHS) as well as key regional level Ministries Department and Agencies (MDAs) in the five (5) regions, and the focus District Assemblies. The evaluation team will report to the UNICEF WASH Specialist and a designated official within the Ministry of Sanitation and Water Resources.

9.2 Expected Deliverables and Timeframe It is expected that the assignment will be completed in nine (9) months including field work and reporting. The selected institution would be required to submit a detailed activity plan upon signing the contract, considering the indicated time frame. Any review of the time frame would be discussed after logistical requirements have been assessed, and in discussion with the WASH Specialist, UNICEF Accra. It is expected that the output of the evaluation will include the following:

• Inception report (max 20 pages excluding annexes and title/cover page) – Detailing output of literature review phase, and confirming the study context, approach and methodology for the field data collection component, quality assurance mechanisms and analysis, plus draft data collection tools included in the annexes

• Monthly progress reports (max 5 pages) – Providing updates of the progress of the assignment, including major processes and the achievement of key output. The formal of the progress reports will be agreed upon during the inception phase

• Debriefing on preliminary findings(5-page report or short PPT presentation) • Draft evaluation report (max 40 pages excluding annexes and title/cover page) (separate for

Enhanced and Accelerated projects) – This will present the draft results of the study including tabulated presentation of the end-line statistics of the indicators identified. The report will also present graphical presentation of the comparison of the baseline data, providing contextual information and basis (institutional weakness, social beliefs, norms, practices, socio-economic / socio-political factors etc.) for major observable differences and anomalies, patterns and trends. The report will also outline the study context, approach and methodology. The format of the report will be agreed upon during the inception phase

• Final evaluation report (max 40 pages excluding annexes and title/cover page) (separate for Enhanced and Accelerated projects) – The report will present the finalized output as defined under the Draft Baseline Report

• Final Combined Summary Report (max 10 pages) – summary document of key findings from both projects

• Key findings briefing document (2 pages) – summary of key points for advocacy purposes

Article III. Summary of Tasks and Deliverables Time Frame Activity Output December 2018 Desk review and methodology design Inception report including a

detailed description of the methodology for data collection and analysis. Data collection instruments must be included as annex of inception report.

January 2019 Training of data collection teams and supervisors Completion of all interviews at national and regional levels

Monthly progress reports

20

Pre-testing and finalization of data collection

instruments Field data collection and data analysis for Enhanced project MMDAs

February 2019

Data analysis and draft reporting for Enhanced project evaluation

Draft Enhanced evaluation report

February 2019

Debriefing on preliminary findings, conclusions and recommendations

5-page report or short presentation (PPT)

March 2019 Finalisation of Enhanced Project Evaluation Report Final Enhanced evaluation report

April 2019 Field data collection and data analysis for Accelerated project MMDAs

Monthly progress reports

May 2019 Data analysis and draft reporting for Accelerated project evaluation

Draft Accelerated evaluation report

May 2019 Debriefing on preliminary findings, conclusions and recommendations

5-page report or short presentation (PPT)

June 2019 Finalisation of Accelerated Project Evaluation Report Final combined summary report 2-page briefing note

Final Accelerated Evaluation Report Final Summary Report Final Briefing Note

10. Expected background and experience The selected institution should have a minimum of 10 years’ experience in the conduct of baseline surveys, statistical surveys / assessments and programme / project monitoring and evaluation in the WASH sector. Only joint applications of an international institution in partnership with a Ghanaian institution will be considered. The institution would be expected, as a minimum, to field the following key personnel: Team leader

• Minimum of a university degree in water, sanitation and hygiene related fields, public health, sociology, economics, international development, or social work

• An advanced qualification in area related to statistics, or project management will be an advantage • Over 10 years’ experience in the development sector, with a focus on at least two of the following

specialities within the WASH sector – project development and design, equity analysis, project monitoring and evaluation / compliance audit, governance, institutional development or sector assessments

• Strong track record in relating and networking and analytical skills • Ability to operate computer Microsoft office programs (MS Word, Excel, Power Point and Access)

is essential • Fluency in writing, reading and speaking English

Statistics and Monitoring and Evaluation Specialists

• Minimum of a university degree in statistics, water, sanitation and hygiene related fields, public health, sociology, economics, international development, or social work.

• And advanced qualification in statistics • Over 7 years’ experience in national level statistical, socio-economic / public health related

surveys, preferably in the WASH sector • Strong track record in relating and networking and analytical skills • Ability to operate computer Microsoft office programs (MS Word, Excel, Power Point and Access),


20

Sociologist / Community Development Specialist

• Minimum of a university degree in sociology, water, sanitation and hygiene related fields, public health, sociology, economics, international development, or social work.

• And advanced qualification in an area related to sociology / community development • Over 5 years’ experience in national level statistical, socio-economic / public health related surveys

and in the WASH sector • Strong track record in relating and networking and analytical skills • Ability to operate computer Microsoft office programs (MS Word, Excel, Power Point and Access),


Collectively the project team should have experience in the following areas of WASH programming:

• Community Led Total Sanitation • Sanitation Marketing • Sanitation Financing • Faecal Sludge Management • Social Norms Programming (or similar behaviour change approaches) • WASH in Schools • WASH in Health Facilities • WASH Information Systems • WASH Governance and Regulation • WASH Knowledge Management

11. Proposed Payment Terms Deliverable 1 Submission of Final Inception Report (Dec 2018) (25%) Deliverable 2 Submission of Final Enhanced Project Evaluation Report (March 2019) (30%) Deliverable 3 Submission of Combined Summary Report, Accelerated Evaluation Report and 2-page

Briefing Note (June 2019) (45%)

12. Structure of Technical Proposal The technical proposal shall include the elements listed below, it should not exceed 20 pages in length (excluding annexes and cover/title page):

• Appreciation of the assignment (the team is strongly encouraged not to present a cut and paste from the Terms of References of this evaluation);

• Draft methodology (including sampling strategy); • Experience profile of institutions forming the consortium; • Experience profile with key project team staff (with CVs included as an annex); • Experience profile of team members against core areas of WASH expertise • Summary of past relevant experience (with details in annex) • A copy of a recent evaluation report of which the Team Leader has been a primary author will need

to be submitted as part of the application Financial proposal should be submitted separately.

13. Structure of Financial Proposal The financial proposal must be submitted separately from the Technical Proposal. There is no length limit to the financial proposal. It should contain the following:

• Time commitments for the Principal Investigator, and other proposed team members

10

• Estimated costs of fieldwork trainings and workshop meetings and any other foreseen travel • Estimated costs associated with data collection (including surveys)

Annex A – List of Target Regions and Districts Central Region (All Enhanced)

• AOB • Assin South

Northern Region • Kpandai (Accelerated and

Enhanced) • Kumpungu (Enhanced) • Mion (Enhanced) • Tatale Sanguli (Accelerated and Enhanced) • Zabzugu (Enhanced) • Bunkpuru Yunyoo (Accelerated) • Karaga (Accelerated) • Yendi (Accelerated) • Nanumba North (Accelerated) • Gushegu (Accelerated) • Nanumba South (Accelerated) • Saboba (Accelerated) • Chereponi (Accelerated) • West Mamprusi (Accelerated) • East Mamprusi (Accelerated) • East Gonja (Accelerated) • Tamale Metro (Accelerated)

Upper East (All Enhanced) • Builsa • Builsa South • Garu

Upper West (All Enhanced) • Lambussie-Karni, • Sissala West, • Wa West

Volta Region (All Enhanced) • Keta Municipal • Kpando Municipal • North Dayi

10

For more information 3 Thomas More Square London E1W 1YW

t: +44 (0)20 3059 5000

www.ipsos-mori.com http://twitter.com/IpsosMORI

About Ipsos MORI’s Social Research Institute The Social Research Institute works closely with national governments, local public services and the not-for-profit sector. Its c.200 research staff focus on public service and policy issues. Each has expertise in a particular part of the public sector, ensuring we have a detailed understanding of specific sectors and policy challenges. This, combined with our methods and communications expertise, helps ensure that our research makes a difference for decision makers and communities.

Jessica Bruce Director [email protected] Viktoria Hildenwall Quality Assurance Director [email protected] Yaw Asante Sarkodie Team Leader [email protected] Judith Ofori Gender Expert [email protected] Adriana Rodriguez Project Manager [email protected] Robert Wragg Research Executive [email protected]

mailto:[email protected]




