Embed Size (px)
Citation preview
ODA BULTUM WOREDA, ETHIOPIA SEPTEMBER, 2013 BEATRIZ PÉREZ BERNABÉ AND LINDSEY PEXTON
2
ACKNOWLEDGEMENTS
GOAL and the Coverage Monitoring Network (CMN) would like to express our great appreciation
to all those who made possible the realization of this coverage assessment of the nutrition
program in the Woreda (district) of Oda Bultum. In the first place, to ECHO and USAID for
funding the CMN project which has contributed to creating capacity in the country and directly
supporting this investigation.
Thanks to all the participants of the SQUEAC training, including both MoH and GOAL staff for
their effort, high level of engagement and for the quality of their work. Grateful thanks are
extended to all at the GOAL office in Oda Bultum, for handling all the necessary preparations
for the start-up and during the investigation – in particular to Roza Dagne, Senior CMAM
Programme Officer, for his organizational support and involvement throughout the study. The
inputs in planning and carrying out the assessment given by Hailu Sitotaw, Senior Survey and
Assessment Coordinator and Zeine Muzeiyn, Nutrition Programme Coordinator, have been
greatly appreciated.
Finally, the team address their most sincere gratefulness to the staff of the health facilities
visited as well as the families and various community members for their hospitality, time and
cooperation. Very special thanks to the mothers and children who took part in the investigation.
ACRONYMS
BBQ
CDA
CMAM
ECHO
HEW
HC
LQAS
MAM
MoH
MUAC
RUSF
RUTF
SAM
SC
SQUEAC
USAID
WHZ
Barriers, Boosters and Questions
Community Development Army
Community-based Management of Acute Malnutrition
European Office for Coordination of Humanitarian Affaires
Health Extension Worker
Health Centre
Lot Quality Assurance Sampling
Moderate Acute Malnutrition
Ministry of Health
Mid-Upper Arm Circumference
Ready-to-Use Supplementary Food
Ready-to-Use Therapeutic Food
Severe Acute Malnutrition
Stabilisation Centre
Semi-Quantitative Evaluation of Access and Coverage
United States Agency for International Development
Weigh-for-Height Z-score
3
EXECUTIVE SUMMARY
Oda Bultum woreda is one of the nineteen woredas in West Hararghe Zone, Oromiya Regional
State. It is located in the eastern part of the country, 362km from Addis Ababa and 37km from
Chiro [Asebe Tefere]. The woreda consists of 37 rural kebeles with an estimated population of
181,732, of whom 29,804 are children under five years1. The population is predominantly
ethnically Oromo and Muslim by religion, with an average family size of 5.0. Bedessa town
serves as the main administrative center of Oda Bultum woreda.
The CMAM program in Oda Bultum began in August 2008 under ownership of the Ministry of
Health (MoH), with technical and logistical support provided by GOAL. There are currently 37
functioning health posts running Outpatient Therapeutic Programmes (OTPs) and 5 health
centres (operating both OTPs and Stabilisation Centres).
A coverage assessment based on the SQUEAC (Semi-Quantitative Evaluation of Access and
Coverage) methodology took place in September 2013 to assess GOAL´s CMAM project and to
build the capacity of MoH and GOAL´s staff in undertaking coverage assessments.
Main barriers identified and recommendations to improve coverage are described in the table
below.
Barriers Recommendations
Insufficient specific information about the program
Increase efforts to sensitize the communities about
the program, clarifying specific issues such as
admission criteria (particularly marasmus), OTP days
and that the treatment is free.
Lack of screening in the community
Strengthen existing outreach activities by increasing the number of CDA and revising the activity strategies
Wrong admission criteria
Promote adequate application of admission and discharge criteria
Increase capacities of CDA and HEW - Train CDA screening with MUAC to improve
case detection - Refreshment training for HEW on admission
and discharge criteria
Lack of training of program staff/volunteers
Weak referral system
Strengthen the referral system by developing a mechanism to monitor cases referred at community level, from OTP to SC (and vice versa) as well as for discharged children
1 CSA 2012
4
CONTENTS
ACKNOWLEDGEMENTS ........................................................................................................................................ 2
ACRONYMS .......................................................................................................................................................... 2
EXECUTIVE SUMMARY ......................................................................................................................................... 3
CONTENTS ........................................................................................................................................................... 4
1. INTRODUCTION ............................................................................................................................................ 5
1.1 CONTEXT ........................................................................................................................................................... 5
1.2 CMAM PROGRAM IN ODA BULTUM WOREDA ................................................................................................. 6
2. OBJECTIVES .................................................................................................................................................. 7
2.1 GENERAL OBJECTIVE ......................................................................................................................................... 7
2.2 SPECIFIC OBJECTIVES ......................................................................................................................................... 7
3. METHODOLOGY............................................................................................................................................ 8
3.1 GENERAL APPROACH ........................................................................................................................................ 8
3.2 STAGES .............................................................................................................................................................. 8
3.3 ORGANISATION OF THE STUDY ....................................................................................................................... 12
4. RESULTS ..................................................................................................................................................... 14
4.1 STAGE 1: IDENTIFICATION OF AREAS OF LOW AND HIGH COVERAGE AND BARRIERS TO ACCESS ................ 14
4.2 STAGE 2: VERIFICATION OF HIGH AND LOW COVERAGE AREAS HYPOTHESIS – SMALL AREA SURVEY ......... 21
5. DISCUSSION................................................................................................................................................ 26
6. RECOMMENDATIONS ................................................................................................................................. 27
ANNEX 1: EVALUATION TEAM ............................................................................................................................ 28
ANNEX 2: CHRONOGRAME ................................................................................................................................. 29
ANNEX 3: DATA COLLECTION FORM.................................................................................................................... 30
ANNEX 4: QUESTIONNAIRE FOR NON-COVERED CASES ....................................................................................... 31
ANNEX 5: BARRIERS – SOURCES & METHODS ...................................................................................................... 32
ANNEX 6: BOOSTERS – SOURCES & METHODS .................................................................................................... 33
5
1. INTRODUCTION
1.1 CONTEXT
Oda Bultum woreda is one of nineteen woredas in West Hararghe Zone, Oromiya Regional State.
It is located in the eastern part of the country, 362km from Addis Ababa and 37km from Chiro
[Asebe Tefere]. The woreda consists of 37 rural kebeles with an estimated population of
181,732, of whom 29,804 are children under five years2. The population is predominantly
ethnically Oromo and Muslim by religion, with an average family size of 5.0. Bedessa town
serves as the main administrative center of Oda Bultum woreda.
Livelihoods in the woreda mainly centre on rain fed
agriculture, with mixed farming constituting 90% and
agro-pastoralism estimated at 10%. Maize, sorghum, teff,
wheat and barley are the major food crops while chat,
coffee and pepper are the most important cash crops.
The woreda has faced consecutive crop failure and/or
below normal production over the past years, mainly due
to the failure and/or poor performance of both kremt
and belg rains. As a result of chronic food insecurity, Oda
Bultum has been included in the Productive Safety Net
Program (PSNP) since 2005. Currently a total of 20,501
beneficiaries across the woreda are targeted under
either the public work scheme (16,910) or direct support (3,591). Cash payments are given for a
period of six months, amounting to 85.00 Ethiopian birr/person/month or 17.00
birr/person/day. Those targeted in the public works programme are expected to work five days
per month.
Oda Bultum woreda is characterized as chronically food insecure, but according to the last
nutrition survey conducted in the woreda, although the nutritional status of under 5 population
has not improved significantly as compared with the year before, the malnutrition rate has
significantly decreased compared with the base line survey (2009 and 2011) meaning that the
current malnutrition rate can be considered normal. Based on national cut-offs, the prevalence
of GAM (MUAC <120 mm and/or oedema) is 1.5% and SAM (MUAC <110 mm and/or oedema) 0.2%.
The rate of GAM based on international standards (MUAC < 125 mm) goes up to 7.3%3.
2 CSA 2012
3 Report on nutrition and retrospective mortality survey conducted in Oda Bultum Woreda, West Hararghe, Zone of Oromiya
Region, conducted in March 2013, USAID and GOAL
6
1.2 CMAM PROGRAM IN ODA BULTUM WOREDA
GOAL has a long history of emergency response in Ethiopia, and since 2005 has been
USAID/OFDA’s emergency nutrition response partner. GOAL Ethiopia’s operational mandate in
nutrition is to support the provision of a package of nutrition services which is targeted at the
poorest of the poor and most vulnerable and also to strengthen the capacity of existing MoH
structures in order to ensure sustainable services for local communities.
The CMAM approach aims to build capacities within the local health services and the community
to prevent and treat malnutrition. GOAL works in partnership with the local health bureaus to
support them in developing their capacity to respond to nutrition emergencies. This approach
aims to enable various program goals to be sustained and promote an acceptable exit when the
situation has stabilized given contextual constrains. The capacity building of MoH staff and the
training of health extension workers and Community Development Army emphasizes Nutrition
Education. The MoH’s policy is scaling-up delivery of essential health services to local
communities. Their focus is on training of the health care worker and this initiative is supported
by UNICEF.
The CMAM program in Oda Bultum began in August 2008 under ownership of the Ministry of
Health (MoH) with technical and logistical support provided by GOAL. There are currently 37
functioning health posts running Outpatient Therapeutic Programmes (OTPs) and 5 health
centres (operating both OTPs and Stabilisation Centres).
Despite the programme being operational for 5 years, this is the first time that a coverage
assessment has been undertaken. The basis for the assessment was therefore the desire to gain
a better understanding of programme access and coverage in order to drive further
improvements. Developing capacity to undertake coverage assessments, particularly among
GOAL’s permanent Survey and Assessment Team was also paramount. Woreda and Zonal
representatives were included in the exercise to create awareness of coverage monitoring, as
well as to build capacity in coverage assessment tools and techniques. As such, 36 team
members were drawn from GOAL’s Survey and Assessment Team, GOAL’s district level CMAM
staff and woreda/zonal representatives. The process was coordinated by a Regional Adviser
from the Coverage Monitoring Network (CMN), together with a Nutrition Adviser from GOAL.
7
2. OBJECTIVES
2.1 GENERAL OBJECTIVE
To assess the coverage of GOAL´s CMAM program and to understand the barriers to accessing
health care in the areas of intervention within the Woreda of Oda Bultum for children aged
between 6 to 59 months, based on the Semi-Quantitative Evaluation of Access and Coverage
(SQUEAC) methodology.
2.2 SPECIFIC OBJECTIVES
To develop capacity of GOAL (GOAL Ethiopia Survey and Assessment team as well as those
directly involved in the program) and Ministry of Health staff to undertake CMAM program
coverage assessments using SQUEAC methodology.
Assess the global estimation of coverage in the target areas of the program.
Identify high and low coverage areas within the intervention area.
Identify barriers to access to treatment of severe acute malnutrition based on
information collected from mothers/caretakers of children with severe acute
malnutrition identified during the investigation and who are not enrolled in the program.
Make recommendations based on the results of the evaluation to improve the access to
treatment of severe acute malnutrition and increase the level of coverage in the program
intervention area.
Write a report presenting the results of the evaluation and taking into account the
differences identified.
8
3. METHODOLOGY
3.1 GENERAL APPROACH
The coverage assessment tool, Semi-Quantitative Evaluation of Access and Coverage (SQUEAC)4,
was developed by Valid International, FANTA, Brixton Health, Concern Worldwide, ACF and
World Vision in order to provide an efficient and accurate method of identifying barriers to
service access and to estimate the coverage of nutrition programs. This is a relatively time
efficient method and gathers large amounts of relevant information; promotes the collection,
use and analysis of data; and provides information on program activities and possible reforms.
The need for human, financial and logistical resources is relatively small. Furthermore, it is
easily reproducible and ensures program monitoring at low cost.
SQUEAC is an interactive, informal and intelligent investigation that collects a large amount of
data from different sources (i.e. using routine data as well as additional data collected in the
field), using a wide variety of methods and providing the means to organise the data. It is a
semi-quantitative assessment as it combines both quantitative and qualitative data.
The analysis of these data is guided by the two fundamental principles of exhaustiveness (of
information up to the point of saturation) and triangulation (information is collected from
different sources using alternative methods, crossing checking data until findings become
redundant before being validated). By focusing on the collection and intelligent analysis of data
during the field phase, the investigation sheds light on the operation of the service whilst
simultaneously providing an educated guess on coverage which allows for a smaller sample size
to be used in the final stage.
3.2 STAGES
SQUEAC allows for the regular monitoring of programs at low cost, helps identify areas of high
or low coverage and provides explanations for such situations. All of this information allows the
planning for specific and concrete actions in order to improve the coverage of programs.
The SQUEAC methodology consists of three main stages:
4 Myatt, Mark et al. 2012. Semi-Quantitative Evaluation of Access and Coverage (SQUEAC)/Simplified Lot Quality Assurance
Sampling Evaluation of Access and Coverage (SLEAC) Technical Reference. Washington, DC: FHI 360/FANTA.
9
3.2.1 STAGE 1: Identification of high and low coverage areas and barriers to access
This stage is based on the analysis of both quantitative data and qualitative information (already
available and collected during the investigation) in order to understand the various factors
influencing coverage, some of which have a positive effect and some a negative effect on
coverage. The SQUEAC approach helps to identify and understand these relevant factors and
their effects. The evaluation of these factors helps to develop a trend in the coverage rates
prior to conducting a field investigation in well-defined areas.
Analysis of quantitative program data: routine data (monthly reports) and records of
individual monitoring (register book and individual cards). The analysis of routine data is
used to assess the overall quality of services, to identify trends in admissions and
performance, and to determine if the program meets needs. This stage also helps to
identify potential problems related to the identification and admission of beneficiaries as
well as problems related to their treatment. Information such as MUAC measurements at
admission and numbers of defaulters can be used to assess early detection, recruitment
and effective communication channels. It also provides information on differences in raw
performance between different health facilities.
Collection and analysis of qualitative data through meetings in the community and
health facilities with those involved directly or indirectly in the program5. This phase of
the investigation is twofold: it serves to better inform and explain the results of the
analysis of routine data and it also helps to understand the knowledge, opinions and
experiences of all people concerned as well as to identify potential barriers to access.
Interview guides were used to orientate the process of obtaining information on coverage.
These interview guides were developed based on guides already used in other SQUEAC
investigations but also adapted to the context and modified/upgraded by the investigation
team.
The following methods to gather information were used:
- Focus Group Discussions
- Semi-structured interviews
These focus group discussions and semi-structured interviews were conducted with the following
sources of information:
- Health Extension Workers (HEW) and Stabilisation Centre (SC) nurses
- Community Development Army (CDA)
- Community leaders
5 We took advantage of these meetings in the community and health facilities to identify the local terminology used
to describe acute malnutrition (Oromifa) and the key informants in the community. This preliminary research is essential to facilitate the active and adaptive case-finding methodology that is used in stages 2 and 3.
10
- Religious leaders
- Community men
- Community women
- Mothers/caretakers of SAM children within the program
18 villages spread across the different kebeles in Oda Bultum were visited for the collection of
qualitative information. Inaccessibility due to the rainy season unfortunately limited the
selection of villages and this may be a biasing factor as these villages may also tend to have
lower coverage due to this seasonal inaccessibility.
The different people encountered and the various methods used allowed the investigation team
to collect information about the barriers and boosters to coverage of the CMAM program. The
data gathered was recorded on a daily basis using a tool called BBQ (Barriers, Boosters and
Questions). This tool not only allows for the organizing of information on a day to day basis, to
continue with the research of qualitative information in an interactive and directed manner, but
also ensures the triangulation of information. To guarantee the exhaustiveness of the process,
the research of information continued until saturation - until the same findings were obtained
from different sources, using different methods.
Altogether, the findings from the quantitative analysis and the conclusions from the
investigation team´s discussions were included in the BBQ with qualitative data collected in the
field to triangulate the set of all knowledge around barriers and boosters to coverage in Oda
Bultum Woreda.
Identification of potentially high and low coverage areas and formulation of a
hypothesis on coverage based on the evaluation of positive and negative factors.
Depending on the barriers and boosters found, the hypotheses on “high” or “low”
coverage areas are developed: the hypothesis about heterogeneity of coverage are based
on the identification of areas of good and “less good” coverage. Then, small-area surveys
are conducted to confirm or refute these hypotheses.
3.2.2 STAGE 2: High and low coverage areas hypothesis testing through small-area surveys
The objective of the second stage of the investigation is to confirm or reject, through small-
area surveys, the assumptions on areas of low or high coverage as well as the barriers to access
as identified in the previous stages of analysis. The small geographical survey method was used
to test the assumption of homogeneity/heterogeneity of coverage.
In this case, 12 villages were selected (6 villages in the area with potentially high or satisfactory
coverage; and 6 villages in the area with low or unsatisfactory coverage) to test the hypothesis
of homogeneity/heterogeneity of coverage. The villages were selected according to the criteria
identified to be the most relevant, according to the information triangulated up to that point in
the survey. The sample of small-area surveys was not calculated in advance; but rather was
based on the number of SAM cases found.
11
SAM Cases were searched for using the active and adaptive case-finding method (i.e going
from house to house based on key informants´ information to find all severely malnourished
children in the village). The case definition used was: "all children aged 6-59 months with the
following characteristics: MUAC <110 mm and/or presence of bilateral oedema, or who were
currently in the CMAM program for the treatment of SAM".
Analysis of the results was done using LQAS (Lot Quality Assurance Sampling) in order to obtain a
classification of coverage compared to the threshold value set at 30%. The decision rule was
calculated using the following formula:
100
dnd
n: number of cases found p: standard coverage defined for the area
The number of cases found and the number of cases covered was examined (see annex 3 for
form to gather the data in the field) based on the following criteria:
- If the number of cases covered was higher than the threshold value (d), then coverage
was classified as satisfactory (coverage meets or exceeded the standard).
- If the number of cases covered was lower than the threshold value (d), then coverage
was classified as unsatisfactory (coverage did not meet, neither exceeded the standard).
Throughout the small-area survey, a questionnaire (annex 4) was distributed to mothers or other
caretakers of all non-covered SAM cases detected in order to further understand the reasons
that these children had not received treatment – as this allows for the identification of barriers
to access. All “non-covered” children found during the study were referred to the appropriate
health services for treatment.
The information obtained through the questionnaires of the non-covered cases in the small-area
survey was added to the BBQ in order to triangulate information regarding barriers to coverage
in Oda Bultum Woreda.
The software XMind is a powerful tool capable of displaying findings in a visual and orderly
manner. It was used in Oda Bultum to develop two different Mind Maps summarizing the barriers
and boosters identified during the first two stages of the investigation, as well as the different
sources of information and methods utilized. Also, conceptual schemas for different barriers and
boosters identified were developed by the team in order to better understand the cause and
effect relationships between the various factors influencing coverage.
12
3.2.3 STAGE 3: Estimation of global coverage
Stage 3 was not conducted in this investigation due to the very low prevalence of SAM6 in the
area assessed and thus, an estimate of the overall coverage in Oda Bultum Woreda could not be
established.
The prevalence of severe acute malnutrition is indirectly related to the number of villages that
need to be assessed during a large area survey. This survey is necessary in order to build a
likelihood curve to determine the overall level of programme coverage. In this case, the amount
of villages required (n) to reach the minimum sample size of children (N) was not manageable
and thus stage 3 was infeasible.
100
*100
months 59 - 6between
population
*nceSAMprevale
population
village
average
Nn
However, for training purposes, a case study based on the actual findings of the present
investigation was conducted with the team in order for them to understand the different steps
to undertake during a coverage assessment and to be confident with all of the methodology
involved within a SQUEAC assessment.
3.3 ORGANISATION OF THE STUDY
3.3.1 Technical support from CMN project
GOAL and MoH received technical support from the “Coverage Monitoring Network” (CMN)
project. The CMN project is a joint initiative involving several organizations: ACF, Save the
Children, International Medical Corps, Concern Worldwide, Helen Keller International and Valid
International. The project aims to provide technical support and tools to CMAM programs in
order to help them assess their impact and to share and capitalize on lessons learnt with regards
to factors influencing their performance.
As part of this assessment, the support from the CMN project involved different phases. In a
first phase, technical support was provided remotely through exchanges between the team of
experts from CMN, José Luis Álvarez Morán (CMN Coordinator) and Beatriz Pérez Bernabé (RECO)
and GOAL staff, Hatty Barthorp (Global Nutrition Advisor), Zeine Muzeiyn (Nutrition programme
coordinator) and Hailu Sitotaw (Senior Survey and Assessment Coordinator) and Lindsey Pexton
6 Prevalence of SAM (MUAC < 110mm): 0.2% (95% IC 0.0-1.4) Report on nutrition and retrospective mortality survey conducted in
Oda Bultum Woreda, West Hararghe Zone of Oromiya Region, March 2013; 0.5% according to program admissions and screening data during the period of the year when the assessment was carried out ( end of hunger gap)
13
(Nutrition Advisor), for the planning and preparation of the evaluation. For technical support in
the field, Beatriz Pérez Bernabé was deployed to Oda Bultum to train the nutrition team in
SQUEAC methodology and to carry out the coverage assessment in the area of intervention.
3.3.2 Training and investigation
An investigation team composed of the national survey and assessment team, 5 members of
GOAL Ethiopia CMAM program and 3 representatives of the MoH (Oda Bultum Woreda and West
Hararghe Zone) were trained in SQUEAC methodology in order to be able to undertake future
coverage assessments. The coordination of the present evaluation was jointly done by the CMN
expert and an international Nutrition Advisor employed by GOAL.
The coverage assessment took place from August 31th to September 10th 2013 (chronogram
annex 2). Two days of introductory theoretical sessions concentrated on the importance of
assessing coverage and the basics of SQUEAC methodology, after which the investigation began
in earnest. The training process was then run concurrently with the investigation - in-classroom
sessions for each key stage of the study was alternated with guided practical implementation in
the field, all framed with iteractive briefing and debriefing sessions.
One additional day was added to the initial planning for the collection of quantitative program
data in health facilities needed for stage 1. This would ideally have been conducted in advance.
Photo 1. Groupwork.
(Oda Bultum Woreda, Ethiopia, September 2013)
14
4. RESULTS
According to the methodology explained above we present here the main results emerging from
our investigation:
4.1 STAGE 1: IDENTIFICATION OF AREAS OF LOW AND HIGH
COVERAGE AND BARRIERS TO ACCESS
The objective of this stage was to identify areas of high and low coverage and to have an initial
understanding regarding the reasons for poor access to treatment, using the program´s existing
quantitative data, together with qualitative information collected from the various
stakeholders.
Although first admissions were registered in August 2008, the analysis of the quantitative
information was carried out based on data corresponding to the last year of program activities
(July 2012 – July 2013). Routine program data was easily available and mainly extracted from
monthly reports. Individual monitoring data was collected by the team at the health facilities
during one working day at the beginning of the investigation. Inaccessibility due to the rainy
season and the absence of HEWs at certain health facilities as a result of a concurrent training
limited the collection of the full set of data. Information was collected from registration books
and outpatient record cards from 23 out of the total of 37 in the woreda.
4.1.1 Analysis of quantitative programme data – routine data: monthly statistical reports
A. Admissions: trends over time and capacity to meet needs
A seasonal calendar for the various seasonal events (child morbidity, climatic and agricultural
activities) was developed by the team and compared to the curve of admissions during the
period July 2012- July 2013 (227 children) to assess the capacity of the program to meet needs
(Figure 1).
According to the results of previous nutrition surveys conducted in the area, malnutrition rates
tend to increase between May and September (hunger gap period). The expected rise in cases
was clearly reflected in admissions trends: admissions experienced a gradual increase starting in
April, and peaking in July. When compared to the Malaria calendar, the increase also matched
the higher prevalence rates during the rainy season (from July to September). This suggests that
the CMAM program is responding to the seasonal increase in SAM cases each year.
15
Figure 1. SAM admissions in OTP and SC and seasonal calendar (Oda Bultum Woreda,
Ethiopia, September 2013)
B. Admissions in OTP/SC and admission criteria
From the total of 227 admissions, 88% correspond to SAM children admitted in OTP and 12% to
SAM children presenting complications and being treated at the SC. The percentage of cases
treated at SC level is unfavourably high. In programs with strong community outreach and early
case finding we would expect to see a lower rate of around 5%.
The analysis of the admission criteria for all the severely acute malnourished children in the
program shows that an extremely high proportion, 60%, have been admitted under oedema
SEASONAL CALENDAR Jul-12 Aug-12 Sep-12 Oct-12 Nov-12 Dec-12 Jan-13 Feb-13 Mar-13 Apr-13 May-13 Jun-13 Jul-13
SEASONS
CHILD MORBIDITY
AGRICULTURAL ACTIVITIES
Rains
Malaria
Land preparation, sowing & weeding
LOCAL EVENTSRamadan
Harvest
OTHER ACTIVITIES
Pety trade
HUNGER GAP
0
50
100
150
200
250N
um
be
r o
f ca
ses
Admissions over time (OTP & SC)
16
criteria (Kwashiorkor cases) and 40% corresponds to Marasmus cases identified only by MUAC
(<110 mm) – Although WHZ is considered as an independent criterion of admission according to
the national protocol (only at HC level, not in HP) no child was found to be admitted under WHZ
criterion.
C. Performance indicators
The performance indicators for OTP were calculated both based on the data from the monthly
statistical reports and from the data coming from the outpatient record cards. Although results
were the same and very satisfactory compared to SPHERE standard values, it remains uncertain
how often hidden deaths may have occurred as defaulter tracing records were not available.
GOAL
Programme SPHERE
Standards
Cured rate 98.3% >75%
Defaulter 0.4% <15%
Death 0.0% <10%
The proportion of children who did not respond to the treatment was only 0.1% and 1.1% were
transferred to SC.
E. Source of referral
Roughly equal numbers of cases were admitted into the program either as self-referrals (32%) or
as referrals by CDAs (33%). Unfortunately, 22% of admissions did not have the referral
mechanism registered on their outpatient record card. Referrals by HEWs were noted for 12%
cases, with only 1% from the mass campaigns.
Figure 2. Source of referral of SAM admissions (Oda Bultum Woreda, Ethiopia, September
2013)
1%
33%
0%12%22%
32%
0%
Campaign
CDA
Neighbour
Health extension worker
Information not available
Self-refered
Village leader
17
4.1.2 Analysis of quantitative program data – individual monitoring records: register books
and outpatient record cards
F. MUAC at admission
In Ethiopia, the cut-off point for admission of SAM children is MUAC <110 mm.
The analysis of MUAC at admission of those children admitted based on MUAC <110 mm (40% of
the total) shows that the majority of cases were admitted with a MUAC quite far from the
admission criteria, with a low median value of MUAC of 105 mm (figure 6). This distribution
suggests poor performance in the ability to identify marasmus cases in a timely manner. The
number of critical cases (MUAC <= 90mm), 16, is of particular concern in this regard. Most cases
are reached and admitted to the program late in the process of the disease which has a negative
impact on the chance of recovery and length of stay.
Figure 3. Distribution of MUAC at admission for SAM cases with MUAC <110 mm (Oda Bultum
Woreda, Ethiopia, September 2013)
F. Distance from village of origin to OTP
The distance (time to travel) between the village of origin of SAM children admitted in OTP was
analysed for 77.92% of total admissions in OTP. The villages were grouped into six categories
according to the time to travel (in minutes) to the HP/HCI. Figure 4 shows, from left to right the
closest to farthest: from 0 to 15 minutes; 16-30; 31-45; 46-60; 61-90, 91-120 and greater than 2
hours.
18
The analysis shows that indeed distance may influence the number of admissions: the number of
admissions dramatically decreases when the time to travel to the OTP is over 1 hour. However,
this could be due to a low number of villages located at such a distance as the geographical
spread of health facilities in Oda Bultum is generally good. Distance as a factor therefore
required further investigation.
Figure 4. Distribution of admissions according to distance (time to travel) to OTP (Oda
Bultum Woreda, Ethiopia, September 2013)
4.1.3 Analysis of qualitative data
The qualitative data was collected in 18 villages spread throughout the intervention area,
except for those non-accessible due to the rainy season. The methods and sources of
information used were those described in the methodology section (chapter 3) and findings were
triangulated using the BBQ on a daily basis.
Table 1 shows the list of the main barriers to coverage identified through the completion of
qualitative work in the field and the subsequent triangulation and analysis of information.
BARRIERS
Lack of specific information about the program
Although there is a good awareness about the existence of the program, there is a lot of specific information about it that often prevents the community from seeking treatment such as not knowing when the OTP days are, that the service is free or that Marasmus can be also treated within the program.
0
50
100
150
200
250
300
350
0-15 16-30 31-45 46-60 61-90 91-120 > 2 hours
Nu
mb
er
of
MA
S ca
ses
Time to travel (mins.) from village to OTP
19
Lack of screening in the community
There are insufficient CDAs and in some cases, where they are identified, they are not active. Crucially, CDAs are not allowed to use MUAC tapes to screen children and therefore identification of SAM cases is done using visual inspection only. “Proper” screening in the community is usually reduced to mass campaigns and mainly done by HEWs. Due to their various other responsibilities and the distance to some communities, screening is not conducted routinely outside of the mass campaigns.
Wrong admission criteria
From the various interviews in the community it was confirmed that most of the time CDAs only identify and refer to the OTP the oedematous and the most severely wasted children (those that can be easily identified without measuring). This is linked to the limitations CDAs have in the execution of their responsibilities as noted above.
Distance Distance to the health facility was reported by most of the members of the community as well as the HEWs as an obstacle to access and finalizing treatment.
Service not available
Either due to HEW meetings or mass campaigns taking place in the community, at times the OTP remains non-functional/closed even during OTP days. It was also found that some HPs are temporarily closed because of the poor condition of the building and others are still under construction.
Lack of staff at HP/HC level
An insufficient number of HEWs and nurses in some health facilities (sometimes only 1 HEW per HP) leads to work overload and non-appropriation of the CMAM program; minimizing time spent with caregivers and the level of information provided during weekly visits.
Lack of training of CDA /program staff
There is a complete lack of formal CMAM training of the CDAs since the creation of this cadre in January 2013. CDAs that are active for CMAM perform this role based on their pre-existing knowledge of malnutrition and of the program. HEWs also mentioned the need for refreshment training on CMAM for themselves.
Weak referral system
CDAs do not use MUAC tapes for screening children nor referral slips to follow those children referred to the OTP/SC. Together with the lack of feedback from HEWs to CDAs, this contributes to a weak level of case monitoring.
Table 1. Barriers to coverage emerged from the qualitative research (Oda Bultum Woreda,
Ethiopia, September 2013)
20
Other factors associated to the non-attendance or defaulting were mentioned during the
qualitative research but were not so significant/relevant in the context of Oda Bultum. Those
directly related to the community were: inaccessibility to the health facility during the rainy
season; the caregiver being busy due to household activities and/or other family
responsibilities; seasonal population movement due to cattle herding or agricultural
requirements and lack of money. Sharing RUTF was reported only once by the community and
was not considered a major problem by service providers. Regarding barriers related to service
delivery, rejection of healthy children, long waiting times and poor conditions in health
facilities were reported. Lack of absentee and defaulter tracing (although very few) was
recognized as a negative factor to coverage, but more significantly, poor follow up of
discharged cases leads to many cases of relapse from MAM to SAM. This was also linked to the
interruption of TSFP services and stock breakouts of CSB. Finally, insufficient supervision by the
woreda was reported from the staff working in some health facilities.
On the positive side, the program seems to be well known and the entire community, including
the mothers of the children enrolled in the programme, have a very positive perception/opinion
of it as well as the efficacy of the treatment itself. In fact, mothers of children discharged often
act as informal CDAs, referring possible cases by encouraging mothers to go to the HC/HP which
reflects a very positive peer-to-peer influence. Family support and the involvement of key
community figures such as village and religious leaders are also constructive mechanisms that
contribute to coverage: leaders often use community gatherings to provide sensitization
messages regarding both malnutrition and the program itself, when possible they refer cases
and generally maintain regular communication with HEWs.
Awareness regarding the causes and signs of malnutrition in the community was confirmed
through the data collection and the OTP was repeatedly stated as the first option considered for
treatment as the child´s health was viewed as a priority. But health-seeking behaviour may not
be considered optimal – many physical signs of malnutrition were seen in some children and
amulets (such as bracelets with leaves inside) from traditional medicine were found during
household visits.
The interface at the health facilities was highly valued by the beneficiaries, with HEWs often
praised for being friendly and welcoming. Routine screening at the HP is done for all children
that arrive for consultation and campaigns are used to sensitize communities regarding
malnutrition. The program profits from these situations to improve coverage. Finally,
coordination and support from GOAL was much appreciated by HEWs.
4.1.4 High and low coverage zones
Given the different positive and negative aspects influencing access to treatment and the
burden carried by the HEWs in this program, distance from home villages to the OTPs appeared
to be a factor influencing coverage.
21
It was thus decided to test the following hypothesis regarding the potential areas of high and
low coverage:
- Coverage is probably satisfactory in areas where distance from the village to the OTP is
low (less than 1 hour).
- Coverage is probably unsatisfactory in areas where distance from the village to the OTP is
high (more than 1 hour).
4.2 STAGE 2: VERIFICATION OF HIGH AND LOW COVERAGE AREAS
HYPOTHESIS – SMALL AREA SURVEY
To test the hypotheses of high and low coverage areas, 12 villages (six in the area of potentially
satisfactory coverage and six in the area of potentially unsatisfactory coverage) from different
kebeles were selected on the basis of the criteria identified: the distance to the OTP.
Village (Kebele) Distance
Satisfactory coverage area
Ahmed Mohamed Burka (Sefera) Kurikura Weba (Harereti) Elelie (Dida Dalo)
Cheruye (Besoso) Chira (Dida Dalo) Dida Oda (Gebida)
+
Unsatisfactory coverage area
Dadhi (Guba Gutu) Weketa (Oda Roba) Husen (Guba Gutu)
Keradi (Bate) Guda Burka (Ido Beriso) Berkele (Suri)
-
Table 2. Villages in potentially satisfactory and unsatisfactory coverage areas according to
the selected criteria (Oda Bultum Woreda, Ethiopia, September 2013)
Results from the active and adaptive case-finding are presented in table 3 and the analysis of
the results in table 4:
Satisfactory coverage area
Total number of SAM cases found 7
Covered SAM cases 2
Non-covered SAM cases 5
Recovering cases 4
Unsatisfactory coverage area
Total number of SAM cases found 7
Covered SAM cases 2
Non-covered SAM cases 5
Recovering cases 2
22
Table 3. Results from active and adaptive case-finding ‐ small-area survey (Oda Bultum
Woreda, Ethiopia, September 2013)
Calculation of decision rule/results Deductions
Satisfactory coverage area
Target coverage 30%
Number of covered cases (2) = decision rule (2)
Point coverage ≈ 30%
Satisfactory coverage hypothesis NON CONFIRMED
n 7
Decision rule (d) = n * (30/100)
d = 7 * 0.30
d = 2.1
d = 2
Covered SAM cases 2
Unsatisfactory coverage area
Target coverage 30%
Number of covered cases (2) = decision rule (2)
Point coverage ≈ 30%
Unsatisfactory coverage hypothesis NON CONFIRMED
n 7
Decision rule (d) = n * (30/100)
d = 7 * 0.30
d = 2.1
d = 2
Covered SAM cases 2
Table 4. Analysis of survey results of the small-area survey – Classification of coverage (Oda
Bultum Woreda, Ethiopia, September 2013)
The hypothesis of heterogeneity was therefore not confirmed suggesting that distance does not
influence in the spatial distribution of coverage. In fact, contrary to the initial hypothesis, none
of the mothers of those SAM cases found not to be covered mentioned distance as the reason for
their child not being in the program.
The reasons that emerged from the analysis actually related mostly to problems with a specific
health facility and to the performance/communication misunderstandings with the HEW
assigned. Such communication misunderstandings included:
- Rejection at the OTP site: three cases - two of them because the child had no oedema
- The child was previously identified as healthy by a HEW: two cases
- The mother waited for the HEW to come to the village for her child to be admitted: one
case
- The mother thought the child could not be in the programme after being discharged as
cured: one case
- OTP closed: one case
- Negligence of the HEW: one case
Only one mother was not aware that her child was malnourished.
23
0 2 4
Rejection
The child was previously told to be healthy by HEW
Lack of Awareness about Malnutrition
OTP closed
Waiting for the HEW to come to the village to be admitted
The mother thinks the child cannot be in the programme after beingdischarged as cured
Negligence
Figure 5. Reasons of the non-covered cases found in the small-area survey (Oda Bultum
Woreda, Ethiopia, September 2013)
Results from this stage suggest that coverage is quite homogeneous throughout the area of
intervention and that motives for defaulting and/or non-attendance are frequently linked to
service delivery.
The information obtained throughout this stage was added to the previous findings and
conclusions from quantitative and qualitative data. The table 6 below show the barriers and
boosters identified along the first two stages of the investigation – with those considered as the
main negative factors to coverage in bold. The MindMaps of annexes 5 and 6 show respectively
the sources of information and methods used to identify each of them.
24
Table 6. Barriers and boosters to coverage in Oda Bultum Woreda (Oda Bultum Woreda,
Ethiopia, September 2013)
Positive factors
VALUE Negative factors
Community
Awareness regarding malnutrition Insufficient specific information about the
program
Awareness regarding the existence of the program
Lack of screening in the community
Good health-seeking behaviour Distance
Positive perception of the program Seasonal barriers (rains)
Involvement of key community figures Caregiver busy
Family support Population movement
Positive peer-to-peer influence Lack of money
Sharing PPN
Service delivery
Lack of defaulters Wrong admission criteria
Interface at health facility Weak referral system
Sensitization at HP Lack of training of program staff/volunteers
Screening/sensitization done “out of the program”
Rejection
Lack of staff at HP/HC level
Service not available
Long waiting times
Absents and defaulters tracing
Conditions in health facility
Relapse
Coordination/collaboration
Coordination and support from GOAL
25
Figure 6 below shows one of the conceptual schemas developed by the team: how the
involvement of key community figures relates to other positive factors of the program and
ultimately has an impact on coverage.
Figure 6. Concept map (boosters) (Oda Bultum Woreda, Ethiopia, September 2013)
The involvement of key community figures in efforts to sensitize on malnutrition is highly
beneficial in terms of raising levels of awareness and appreciation of the significance of
malnutrition. Their engagement also has a bearing on community acceptance of the program
and these two factors help bolster treatment seeking behaviour. When community acceptance
of the programme is translated into support from family members completion of the treatment
(lack of defaulters) can be more easily achieved.
26
5. DISCUSSION
Although, due to the low prevalence of SAM in the area of intervention, an overall estimation of
coverage was impossible, findings from the SQUEAC assessment suggest a low coverage for
GOAL´s program in Oda Bultum Woreda. The coverage assessment did however reveal a number
of addressable barriers, as well as important boosters to program access coverage.
The program is well known and much appreciated by the community. However, they often lack
specific information on how the program runs or there are sometimes misunderstandings
surrounding the messages disseminated by HEWs, which prevent caregivers from seeking
treatment in an active manner. This leaves many malnourished children “uncovered” and
untreated by the programme.
The current status of CDAs and their limited involvement in the CMAM program has been
determined to be directly linked to unsatisfactory program coverage: not only the limited
quantity of CDAs in some areas but most importantly the low level of activities they perform.
Their inability to screen children using MUAC tapes means that the less severe marasmic
children are not identified and referred for treatment. On the other hand, the referral of
children is done in a completely informal way, which makes follow up and monitoring of these
children difficult, if not impossible. CDAs require urgent training and a revision of their tasks
needs to be conducted in order to contribute to increased coverage, improved treatment and
prevention of malnutrition. Also, refreshment training for HEWs is required with a focus on
admission and discharge criteria - from different stages of the investigation it has emerged that
both CDAs and HEWs are predominantly identifying oedematous children, with marasmic cases
remaining unidentified or even rejected at the health post.
Currently HEWs have a double level of responsibility, at HP and also screening in the
community. This high workload prevents them from providing the necessary attention to the
caregivers during their weekly visits - and sometimes even closing the HP during OTP days - and
also from routinely screening in the community (which is usually reduced to mass campaigns) as
desired. Community level screening is usually reserve for mass campaigns. CDAs and HEWs need
to work closer and re-define their tasks in order to ensure quality service delivery and
ultimately, to improve coverage.
Distance has proved not to be a major barrier to coverage. On the contrary, caregivers have
shown a very positive attitude and awareness with regards to malnutrition that encourages them
to overcome this obstacle in those cases where the OTP is further from the village. Other key
boosters to coverage that have become apparent during the investigation include awareness of
malnutrition, involvement of key community leaders and passive case finding at HP/HC level.
27
6. RECOMMENDATIONS
In line with what the study has found, recommendations and activities to improve coverage are:
Barrier Key recommendation Actions
Insufficient specific information about the program
Clarify understanding of the program functioning in the community
Increase sensitization about the program
Lack of screening in the community
Strengthen existing outreach activities
Revise CDA activities/strategies
Increase the number of CDAs
Wrong admission criteria Promote adequate application of admission and discharge criteria
Train CDA to screen with MUAC to improve case detection
Refreshment training for HEWs on admission and discharge criteria
Lack of training of program staff/volunteers
Increase capacities of CDA and HEW
Weak referral system Strengthen the referral system
Develop a mechanism to monitor cases referred at community level, from OTP to SC (and vice versa) and for discharged children
28
ANNEX 1: EVALUATION TEAM
Investigation team
BEATRIZ PÉREZ BERNABÉ, CMN´s Regional Coverage Advisor HKI
LINDSEY PEXTON, GOAL – Nutrition Advisor
HAILU SITOTAW, GOAL Ethiopia - Senior Survey and Assessment Coordinator
SEIFU SISAY, GOAL Ethiopia - Senior Survey & Assessment Officer
AFERA ASMEROM, GOAL Ethiopia - Senior Survey & Assessment Officer
SHIFERAW TADESSE, GOAL Ethiopia - Survey & Assessment Info. Officer
ABRAHAM LELANGO, GOAL Ethiopia - Survey & Assessment Info. Officer
ASSEN SEID, GOAL Ethiopia - Survey & Assessment Info. Officer
TIGABU HAILU, GOAL Ethiopia - Survey & Assessment Info. Officer
TEMIR MOHAMMOD, GOAL Ethiopia - Ass. Survey & Ass. Info. Officer
MEAZA MITIKU, GOAL Ethiopia - Ass. Survey & Ass. Info. Officer
TADELECH GEBITA, GOAL Ethiopia - Ass. Survey & Ass. Info. Officer
BIRTUKAN AYALEW, GOAL Ethiopia - Ass. Survey & Ass. Info. Officer
YEBERGUAL MEKONEN, GOAL Ethiopia - Ass. Survey & Ass. Info. Officer
ZENEBU GEBRESILASE, GOAL Ethiopia - Ass. Survey & Ass. Info. Officer
CHUCHU TADESSE, GOAL Ethiopia - Ass. Survey & Ass. Info. Officer
TURENESH LEGESSE, GOAL Ethiopia - Ass. Survey & Ass. Info. Officer
DAYAN TAYE, GOAL Ethiopia - Ass. Survey & Ass. Info. Officer
MASERESHA BOGALE, GOAL Ethiopia - Ass. Survey & Ass. Info. Officer
SEID MOHAMMED, GOAL Ethiopia - Ass. Survey & Ass. Info. Officer
YIMER MOHAMMED, GOAL Ethiopia - Ass. Survey & Ass. Info. Officer
SHEWANGIZAW TESHOME, GOAL Ethiopia - Ass. Survey & Ass. Info. Officer
BESFAT ABERA, GOAL Ethiopia - Ass. Survey & Ass. Info. Officer
ABIY ALEMU, GOAL Ethiopia - Ass. Survey & Ass. Info. Officer
DEMIS TEKLU, GOAL Ethiopia - Ass. Survey & Ass. Info. Officer
YOHANNES GIRMA, GOAL Ethiopia - Ass. Survey & Ass. Info. Officer
ALEMAYEHU GEZAHEGN, GOAL Ethiopia - Ass. Survey & Ass. Info. Officer
HENOK LEGESSE, GOAL Ethiopia - Ass. Survey & Ass. Info. Officer
MEKDES GEBREYSUS, GOAL Ethiopia - Data Entry Officer
ZINASH BOCHA, GOAL Ethiopia - Senior CMAM Programme Officer
ROZA DAGNE, GOAL Ethiopia - Senior CMAM Programme Officer
TENADAM AMEDIN, GOAL Ethiopia - CMAM Nurse
TIGIST BIRATU, GOAL Ethiopia - CMAM Nurse
ALIYE ABDUREHIMAN, MoH-West Hararghe Zone
MOHAMMED SEID, MoH-West Hararghe Zone
TEFERA GIRMA, MoH-Oda Bultum Woreda - CMAM Focal Person
AMIRA MOHAMMED, MoH-Oda Bultum Woreda, CMAM Focal Person
29
ANNEX 2: CHRONOGRAME
Date Activities
Thursday 29.08 Arrival of the CMN´s RECO to Addis
Friday 30.08 Trip Addis-Oda Bultum
Saturday 31.08 – Sunday 01.09
SQUEAC Orientation: - Introduction to the coverage assessments and to the SQUEAC methodology - Stage 1:
- Literature and quantitative data review and seasonal calendar - Qualitative research methods and preparation of field work: work
and review of questionnaires and selection of villages
Monday 02.09 Quantitative data collection in health facilities
Tuesday 03.09 – Thursday 05.09
Qualitative data collection: interviews in the community and health structures – Daily restitution of findings (BBQ) Collection of additional information needed for stages 2 and 3
Friday 06.09 - Saturday 07.09
Quantitative and qualitative information summary and analysis – Concept maps and learning about XMind software Preparation of stage 2: Study of spatial distribution of coverage and selection of villages Training on the Active and Adaptive case-finding method
Sunday 08.09 – Monday 09.09
Small-area survey
Tuesday 10.09
Synthesis and analysis of stage 2 information Conclusions and recommendations Case study on stage 3:
- Construction of the Prior - Learning about the BayesSQUEAC calculator software - Sample calculation and preparation of the wide-area survey
Wednesday 11.09 Trip back Bedessa-Addis Ababa
Thursday 12.09 Debriefing of preliminary results at GOAL Ethiopia head office in Addis Ababa
Friday 13.06 Trip Addis Ababa-Madrid of the CMN´s RECO
30
ANNEX 3: DATA COLLECTION FORM
SQUEAC: Data collection form Woreda: ________________________________ Kebele: __________________________________________
Village: ______________________________ Team: _________________ _____ Date: _____________________________
Child´s name and surname Age
(months) MUAC
Oedema
(+, ++, +++) SAM case
Recovering
child Verification: PPN/Ration card SAM case
Covered
(in the
program)
SAM case
Non-covered
(not in the
program)
☐ PPN ☐Ration card
☐ PPN ☐Ration card
☐ PPN ☐Ration card
☐ PPN ☐Ration card
☐ PPN ☐Ration card
☐ PPN ☐Ration card
TOTAL
31
ANNEX 4: QUESTIONNAIRE FOR NON-COVERED CASES
Questionnaire for caregivers of SAM cases NOT in the program (NON-COVERED cases)
Woreda: _________________________ Kebele: _________________________________
Village: _________________________ Name and surname of the child: __________________________
1. DO YOU THINK THAT THIS CHILD IS MALNOURISHED?
YES NO STOP!
2. DO YOU KNOW A PROGRAM/PLACE THAT CAN TREAT MALNOURISHED CHILDREN?
I YES NO STOP!
3. WHAT IS THE NAME/WHERE IS THIS PROGRAM? __________________________________
WHER
4. WHY THIS CHILD IS NOT IN THE PROGRAM?
1. Too far What distance do you have to walk? ___________ How many hours? ______________
2. No time/too busy to attend de program Which activity keeps the caregiver busy? _____________
3. The caregiver is sick
4. The caregiver cannot travel with more than one child
5. The caregiver is ashamed to attend the program
6. Security problems
7. No other person in the family can take care of the other children
8. The amount of Plumpy Nut given is not enough
9. The child has previously been rejected When? (approximate period) ________________
10. Other people´s child has been rejected
11. The husband has refused
12. The caregiver though the child needed to be intern in the hospital
13. The caregiver does not believes that the program can help the child (prefers traditional healers, etc.)
14. Other reasons: _____________________________________________________________
5. HAS THE CHILD ALREADY BEEN IN A PROGRAM FOR THE TREATMENT OF MALNUTRITION?
NO STOP! YES If yes, why the child is not enrolled currently?
Abandon, when? ____________ Why? ________________________________
Cured and discharged When? ____________________________________
Discharged because there no cured When? ___________________________
Others: ________________________________________________________ (Thank the caregiver)
32
ANNEX 5: BARRIERS – SOURCES & METHODS
33
ANNEX 6: BOOSTERS – SOURCES & METHODS
