88
1 . Formative Evaluation Report for The Project entitled “Accelerating efforts to reduce maternal, neonatal and child mortality in the Northern and Upper East regions of Ghana” Evaluators: Timothee GANDAHO, MD, PhD, Samuel BOSOMPRAH, MSc, PhD, September, 2015

Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

Embed Size (px)

Citation preview

Page 1: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

1

.

Formative Evaluation Report

for The Project entitled “Accelerating efforts to reduce

maternal, neonatal and child mortality in the Northern and

Upper East regions of Ghana”

Evaluators:

Timothee GANDAHO, MD, PhD,

Samuel BOSOMPRAH, MSc, PhD,

September, 2015

Page 2: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

2

Table of Contents

Acknowledgements ................................................................................................................... 5

Executive Summary .................................................................................................................. 6 Background ....................................................................................................................................... 6 Key Findings ...................................................................................................................................... 7 Conclusion ......................................................................................................................................... 9 Recommendations .............................................................................................................................. 9

1. Introduction ........................................................................................................................ 13 1.1 Background ................................................................................................................................ 13 1.2 Specific objectives of the formative evaluation: .......................................................................... 14

2. Scope of the Evaluation and Evaluation Questions ............................................................. 15 2.1 Scope of the formative evaluation ............................................................................................... 15 2.2 Evaluation questions: ................................................................................................................. 15

3. Evaluation Methods ............................................................................................................ 20 3.1 Ethical considerations ................................................................................................................ 20 3.2 Study design ............................................................................................................................... 20 3.3 Sample size consideration and sampling ..................................................................................... 21 3.4 Field work/Data collection .......................................................................................................... 22 3.5 Data Analysis .............................................................................................................................. 24 3.6 Data archiving ............................................................................................................................ 24 3.7 Quality Assurance ...................................................................................................................... 24

4. Evaluation findings ............................................................................................................. 26

5. Lessons learned ................................................................................................................... 61

6. Conclusions and Recommendations .................................................................................... 61 6.1 Conclusions ................................................................................................................................ 61 6.2 Recommendations ...................................................................................................................... 62 References........................................................................................................................................ 65

Appendices .............................................................................................................................. 66 AP1. Informed Consent Form .......................................................................................................... 66 AP2. Evaluation tools (In-depth interview guides, Short questionnaires, FGD guide) ...................... 67 AP3. Evaluation Framework ........................................................................................................... 81 AP4. List of National Decision-Makers/Stakeholders Interviewed ................................................... 85

AP5a. Evaluators’ work schedule. ................................................................................................. 87 AP5b. Percentage change in institutional neonatal deaths by districts in the two regions................. 87

Page 3: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

3

Acronyms and key definitions

Acronyms

ANC Antenatal Care

AP Appendix

BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity Based Agents

CHN Community Health Nurse

CHO Community Health Officer

CHPS Community Health Planning and Services

C4D Communication for Development

DA District Assembly

DHIMS Health Information Management System

DHMT District Health Management Team

EC European Commission

EMBRACE (model) European Model for Bioinformatics Research and Community Education

EmONC Emergency Obstetric and Neonatal Care

FGD Focus Group Discussion

GHS Ghana Health Service

HBPNC Home-Based Postnatal Care

HIV Human Immunodeficiency Virus

HRBA Human Rights-Based Approach

HSS Health Systems strengthening

IMNCI Integrated Management of Neonatal and Childhood Illnesses

JHPIEGO Johns Hopkins Program for International Education in Gynecology

and Obstetrics

JICA Japan International Cooperation Agency

KOICA Korea International Cooperation Agency

KMC Kangaroo Mother Care

LBW Low Birth Weight

MAF MDG5 Acceleration Framework and Action Plan

MDGs Millennium Development Goals

M&E Monitoring and Evaluation

MNCH Maternal Newborn and Child Health

MoH Ministry of Health

NGO Non-Governmental Organization

NBC Newborn care

NCC Newborn care corner

NCU Neonatal Care Unit

NMR Neonatal Mortality Rate

NR Northern Region

OECD/DAC Development Assistance Committee of the Economic Cooperation and

Development

PATH An International Health Organization

PNC Post Natal Care

PPME Public Private M E

QI Quality Improvement

QA Quality Assurance

Page 4: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

4

UER Upper East Region

UGMS University of Ghana Medical School

UNICEF United Nations Children’s Fund

URC University Research Co, LLC

WHO World Health Organization

UN United Nations

UNFPA United Nations Population Funds

USD United States Dollar

USAID United States Agency for International Development

Key definitions Low birth weight Weight of less than 2,500g, irrespective of gestational age

Newborn death Death within 28 days of birth of any live-born baby regardless of

weight or gestational age

Preterm birth A baby after born less than 37 completed weeks of gestation

Still birth A baby born with no signs of life at, or after 28 weeks' gestation (WHO). -

http://www.who.int/maternal_child_adolescent/epidemiology/stillbirth/en/

Page 5: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

5

Acknowledgements

The National Newborn Subcommittee members and UNICEF Ghana staff contributed to the

improvement of the evaluation tools, framework, preliminary results and draft report with their

constructive suggestions and comments; the evaluators wish to express their appreciation to them.

The evaluators would particularly like to acknowledge the technical and financial assistance received

from UNICEF for this evaluation.

We would also like to express our sincere gratitude to the experts and national key informants from

MoH, GHS, UNFPA, UNICEF Accra /Tamale, WHO, European Commission, USAID, JICA, the

Embassy of Japan, USAID/HSS Project, USAID/JHPIEGO project, PATH, the Pediatric Society of

Ghana, the Society of Obstetricians and Gynecologists of Ghana, the Ghana Registered Midwife

Association, the Teaching Hospitals, the School of Public Health, Project Fives Alive, the University

of Ghana Medical School (UGMS) and the Coalition of NGOs on Health; who provided valuable

information used for this evaluation.

The contributions of other United Nations agencies and development partners who were key

informants in this evaluation process are also greatly appreciated.

Special thanks to all national, regional and district Leaders or Directors or in-Charge and newborn

care focal persons of Ghana Health Service HQ/NR/UER who were interviewed at national, regional

and district levels. The information they have provided was key to this evaluation.

The evaluators are very grateful to the highly motivated health providers, especially the community

health officers, the community health nurses and midwives of the Northern region and Upper East

region, and to all the mothers who participated with enthusiasm in the focus group discussion sessions

as beneficiaries of newborn care. They have generated critical information for this formative

evaluation for the newborn care project in the Northern and Upper East regions.

Furthermore, we would like to commend the technical support and the valuable and relevant

comments received at various stages of this evaluation from UNICEF Staff – Dr. Hari Krishna

Banskota, Dr. Victor Ngongalah, Dr. Imran Ravji, Mrs. Felicia Mahama, Mrs. Anna Maria Levi, and

Mr. Clemens Gros – and from GHS Staff – Dr. Isabella Sagoe-Moses and Dr. Cynthia Bannerman.

We also extend our appreciation to all who contributed directly or indirectly to this evaluation.

Page 6: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

6

Executive Summary

Background The death of infants in the first 28 days of life is increasingly becoming a global health concern. This

is especially relevant as it undermines the achievement of the millennium development goal: to reduce

under-five mortality. Many countries in the developing world have, over the years, implemented a

series of interventions to reduce the burden of under-five mortality. In line with the National Child

Health Policy, UNICEF with funding support from the Government of Japan, has been providing

technical and financial assistance to GHS at the National level and in a selection of fourteen districts

of the Northern and Upper East regions of Ghana since October 2011 in order to implement the project

entitled “Accelerating efforts to reduce maternal, neonatal and child mortality in the Northern

and Upper East regions of Ghana”. This project was the subject of this evaluation. The following

are the key findings of the evaluation and recommendations for possible policy action.

The Specific objectives of this formative evaluation are:

1. To ascertain the effectiveness of the package of evidence-based maternal, neonatal and child

survival interventions with a focus on community-based (Home-based Postnatal) Care

(HBPNC) and facility-based integrated Management of Newborn and Childhood Illnesses

approaches in the selected districts of the Northern and Upper East regions;

2. To ascertain the project’s contribution to capacity building that is, developing training

resources and facility structures to respond to the high levels of newborn morbidity and

mortality in the selected districts of the Northern and Upper East regions;

3. To ascertain the effectiveness of the evidence-based advocacy by the project on the national

policy environment on issues related to newborn survival;

4. To draw lessons on the implementation capacity for the national expansion of the essential

newborn care model through home-based early postnatal care.

The scope of the formative evaluation covered the areas of implementation of the project in the two

regions (Northern and Upper East Region) of Ghana and at the national level. The evaluation covered

the two phases of the project from September 2011 to December 2014. The evaluation focused on,

and included the following: final beneficiaries, service providers, actors at the sub-national

decision-making level (district and regional health authorities), actors at the national decision-

making level, national professional societies and academia.

The formative evaluation attempted to provide answers to a number of questions to meet the

Development Assistance Committee of the Economic Cooperation and Development (OECD/DAC)

evaluation criteria as it pertains to relevance, effectiveness, efficiency, and sustainability as well as

UNICEF’s Coherence and Human Rights-Based Approach (HRBA) to Programming and Equity

for all target groups.

The formative evaluation employed a mixed method design consisting of qualitative and

quantitative components. The qualitative component consisted of in-depth interviews with 12 key

national decision-makers, donor partners, 16 sub-national health authorities, 12 service providers and

2 focus group discussion (FGD) sessions with mothers drawn from the communities in the two project

districts. The quantitative component involved abstraction of neonatal health indicators from the

District Health Information Management System (DHIMS 2) and other relevant data sources based

on indicators developed from the evaluation questions as well as project-specifics to assess the

project’s success and effectiveness.

Page 7: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

7

Key Findings

The main findings are summarized below according to specific evaluation objectives:

Objective 1: To ascertain the effectiveness of the package of evidence-based maternal, neonatal

and child survival interventions with a focus on community-based (Home-based Postnatal) Care

(HBPNC) and facility-based integrated Management of Newborn and Childhood Illnesses approaches

in the selected districts of the Northern and Upper East regions;

The established level-two newborn care units in the district hospitals have contributed, to some

extent, to an improved neonatal survival through an improved management of sick newborn

babies. For example, institutional neonatal deaths per 1,000 live births reduced by about 51% in

the Northern region and about 43% in the Upper East region over the project period (from 2012

to 2014). But challenges still remain with cases from the communities arriving late for

management. However, it should be noted that the project was implemented in well identified

districts and not in the entire region that contains a lot more districts; thus the findings

disaggregated and analyzed by project districts show mixed results.

During FGD, the beneficiaries expressed satisfaction with the home-based care for their babies

received from the CHOs/CHNs. They also indicated having received basic counseling on health

and wellbeing of mothers and babies and that the regular interaction with the CHOs/CHNs and

community volunteers provided the opportunity to share their experience and bring up their

challenges relating to the provision of care for their newborn babies and managing of their own

health.

Perceptions and reported statements from beneficiaries indicate that the health and wellbeing of

babies have improved substantially to their satisfaction. They reported positive behavioral change

and less diseases due to the newborn care intervention, especially education by nurses and

volunteers during home visits and supportive communication activities.

A review of the project proposal indicated that due considerations were given to changes in the

burden of neonatal deaths in the selection of the project regions. It also indicated that the regions

were chosen to consolidate the gains recorded following a series of interventions in the past.

A review of the Medium Term Health Expenditure Plan for 2014-2017 and the Ghana Shared

Growth and Development Agenda for 2014-2017 showed that many activities therein can directly

and indirectly impact the lives of the newborn. The National Newborn Strategy provides a more

focused framework on newborn survival, which can be operationalized through annual plans and

budgets with the support of health partners.

Objective 2: To ascertain the project’s contribution to capacity building that is, developing training

resources and facility structures to respond to the high levels of newborn morbidity and mortality in

the selected districts of the Northern and Upper East regions;

The project supported capacity building workshops for all district directors of health services and

district public health nurses in the project districts on essential newborn care. They are now able

to plan for newborn care activities. A review of the districts’ annual plans showed that they have

all featured newborn care activities. Key informants reported that the project trained sub-national

personnel as trainers who in turn trained sub-district and community health service providers on

newborn care.

Page 8: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

8

A desk review of project documents showed that a total of 4,151 health workers (369 Doctors,

midwives, medical assistants, nurses, 816 CHOs and CHNs) plus 1748 community volunteers and

1218 Red Cross mother-to-mother support groups with 18 supervisors and facilitators) were

trained on essential newborn care. However, while service providers reported satisfaction with the

training content, the capacity created might diminish over time especially under high attrition

situations. Out of $1,812,187, 45% ($808,356) were spent on training and capacity building and

35% ($617,271) spent for community and facility service delivery and demand generation. The

cost of training a health professional in newborn care was $194.73 on average per trainee.

Key informants, reported that the application of skills and knowledge acquired during the

newborn care trainings reinforced with supervision, monitoring and mentorship have helped to

improve the quality of service delivery by the trained providers.

Some of the beneficiaries reported to have been educated on essential newborn care through

community durbars and information from nurses and volunteers who spent time with them during

home visits.

Objective 3: To ascertain the effectiveness of the project’s evidence-based advocacy for issues

related to newborn survival and national policy environment.

The aim of the project to improve neonatal survival is very much aligned with the national child

health policy. A desk review of relevant policy documents showed that the objectives of the project

were aligned with the Government of Ghana’s Child Health Policy. The project is very much

placed within UNICEF’s global mandate to improve children’s health. The project is operationally

aligned with UNICEF’s significant presence in northern Ghana including a field office in Tamale

with technical and operational staff who provide close technical and monitoring support during

the project. The project is also very well aligned with the EMBRACE model articulated in the

Government of Japan’s Global Health Policy both conceptually as well as operationally.

The project supported the processes leading up to the development and launch of the National

Newborn Strategy and 2014-2018 Action Plan. Specifically, the project supported MoH/GHS to

organize 3 national level stakeholder meetings on newborn health. It also supported the

development of bottleneck analysis tools and decentralized monitoring and planning on newborn

health. As reported by key informants, the launch was very successful, and the advocacy and

communication around it contributed to place newborn health on the national agenda.

The Ministry of health and partners, recommended a newborn strategy and Ghana Health Service

was tasked to have it developed with newborn indicators for performance monitoring. The

national newborn strategy was launched in July 2014. The advocacy and communication around

it was such that it was attended by parliamentarians, civil society, embassy representatives, some

key media personnel and representatives from other sectors. In order to implement the National

Newborn Care Strategy, all 10 regions now have a newborn health focal person.

Objective 4: To draw lessons on the implementation capacity for national scale-up of the essential

newborn care model through home-based early postnatal care.

The Newborn Sub-Committee coordinates newborn care activities of all the partners involved in

implementing newborn interventions. Each region and district has a focal person for newborn

care. The training modules have been adopted and are being rollout to other regions and home

visits are being integrated into the CHPS structure.

Page 9: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

9

The DHMT/DA is supportive to newborn care but is yet to demonstrate ownership to

consolidate the achievements and the expansion of the newborn health interventions within

available or mobilized resources for the district. Its resources are not sufficient to be able to self-

support the needs for a long-term sustained newborn care intervention. Challenges however

remain. They pertain to the adequate mobilization of resources for such ownership and the

remuneration of community-based volunteers, the building of capacity for new staffs with more

training sessions, more NCU equipment and resolving problems pertaining to fuel and

motorbikes supply for home visits.

Government resources to the sector are limited and remain basic for the regions where the health

services depend on internally generated funds – mainly from the national health insurance

scheme. Unfortunately, the delay in payment by the insurance scheme is further threatening

service provision.

Service providers admit that despite the success of the project, there exist some bottlenecks,

which need to be addressed before scaling-up the essential newborn care model through home-

based early postnatal care. These include lack of motorbikes, insufficient personnel due to staff

attrition or trained nurses going back to school for further training, lack of means of

transportation for supportive supervision or performance monitoring, and volunteer fatigue due

to the absence of incentives. Conclusion

The home-based postnatal newborn care and neonatal intensive care models have been effective in

contributing to improved newborn survival in the two project regions of Upper East and Northern

regions of Ghana, to the extent possible given the scope and reach of the intervention. The enhanced

capacity of NCUs, with essential newborn care equipment as well as the enhanced capacity of health

personnel in terms of skills acquired for management of sick and preterm babies, have been important

enabling factors for saving the lives of many babies in the project districts. The evidence-based

advocacy efforts at all levels contributed significantly to making newborn issues a national priority,

especially culminating in the development and launch of a National Newborn Strategy and Action

Plan.

Recommendations

The challenge for improving newborn health lies in ending preventable newborn deaths and securing

Ghana’s future. Success will be measured in terms of lives saved and lives improved. Success will

depend on meeting the needs of women and their babies throughout the continuum of care and

committing to the following action items:

National Level:

1. The Government should commit enough resources to operationalize the National Newborn

Strategy and Action Plan. The Newborn strategy could be used as a framework for donor

support. Donor assistance should be mapped onto strategy priorities and donor projects have

to be coordinated in order to achieve strategy objectives. A system should be put in place

for the effective monitoring and assessment of achievements and resource management tools

should be setup so as to ensure accountability. A national budget line for newborn activities

needs to be envisioned. An advocacy group may also be put in place to ensure continued

resource mobilization for Newborn Strategy implementation.

Page 10: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

10

2. The MoH/GHS should ensure that every district hospital has a Newborn Care Unit for

secondary (level 2) care. All regional and teaching hospitals should have Newborn Intensive

care Units (NICU) for tertiary (level 3) care. Health Centers and Polyclinics where delivery

is conducted should be provided with Basic Emergency Obstetric and Newborn Care

(BEmONC) including Newborn Care Corners (NCC).

3. The GHS/MoH should establish resource centres in Regional and Teaching Hospitals along

with NCU, to the extent possible using existing structures, to facilitate on-the-job training

for newborn care. Staff from the resource centres should deliver a transferable skills program

through mentorship and periodic specialists’ visits to lower level facilities.

4. The GHS should scale-up the home-based postnatal newborn care model to all districts in

the regions and to other regions. The evaluators do not anticipate any delay if national and

sub-national decision-makers commit to this course. The materials have already been

developed and lessons learned can speed up a nationwide scale-up. GHS could take

advantage of the fact that in all ten regions, there is ongoing newborn activities in some of

the districts supported by various donors such as UNICEF (2 regions), USAID HSS (5

regions), USAID JHPIEGO (4 regions), PATH (4 regions), JICA (1 region) and KOICA (1

region). This will require coordination and harmonization on a minimum package of

effective newborn care for the reduction of newborn mortality.

5. The MoH should review the curriculum of the Midwifery and Community Health Training

Schools to include issues on newborn care or update and strengthen any existing ones such

as the training programs developed in collaboration with UNICEF and other development

partners using the newborn care training modules, and which have now been accepted as

national documents intended to be rolled out to other parts of the country. The MoH and

GHS should collaborate to formulate and approve a detailed implementation plan and budget

for the integration of the newborn care training package into the pre-service, postgraduate

and continuing education systems. The in-service training should also be reinforced for

those already in the field.

Sub-national (region and district) level:

6. The District Directors should collaborate with the District Assemblies (DA) to ensure that

newborn care issues become a standard agenda on district quarterly review meetings. This

implies advocacy work using neonatal mortality data from the district statistics so as to

inform the DA on the urgency of mobilizing funds to address newborn care issues as a

priority in the district. The DA should have a local budget line for newborn care as a

sustainable financing solution for both maternal and newborn care services within the

district. This will help the district address a number of challenges related to newborn care

activities such as incentive and motivation for volunteers and CHO/CHN, fuel supply and

maintenance for motorbikes used for home visits, and the supply of bicycles for volunteers.

The financial contribution of the district to newborn care activities will encourage

MoH/GHS to be supplemented with the recruitment of additional human resources which

are currently in short supply (nurses, midwives and pediatrician) and provide the needed

equipment such as new motorbikes for facilities and materials for NCU.

Page 11: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

11

7. Regions and districts could set up a community performance-based financing with

performance reward approach where community volunteers, providers and sub-district

teams will be given money for anticipated performance. An agreement will need to be

established with each district and group of providers to implement newborn care activities

with well-established results and coverage for which funds and financial incentives will be

provided based on performance in terms of percentage of expected results achieved. This

will be an option for better coverage and achievements with newborn care.

8. Institutionalization of perinatal death audit and newborn death audit would be fundamental

to ensure increased attention to newborn care and the causes and circumstances of neonatal

death in order to address them more effectively and reduce neonatal mortality. It will also

help avoid neonatal deaths due to poor performance or mistakes or inappropriate action of

the providers. To prevent those unnecessary neonatal deaths, Regional Health Management

Teams, Hospitals and the MoH/GHS should integrate newborn care indicators to the existing

M&E system to monitor performance, progress, facility neonatal deaths and achievements

in newborn care by providers and volunteers. This could be reinforced with provision of

newborn care registers for hospital, facilities and community visits. Regions and districts

health managers should be encouraged to use effectively these newborn indicators and

newborn death audit results in planning and implementation of health service decisions as

well as in assessing staff performance.

9. Regional and district leadership should be strengthened to drive the newborn agenda and

provide support for its implementation. The district directors should be tasked to develop a

comprehensive plan with costing for capacity building and refresher training schedules for

staff involved in newborn care. They must maintain a register of staff and track staff

movement in order to manage any capacity gap arising so as to reduce staff attrition. Further

steps should be taken to provide the necessary conditions to retain trained service providers

in the deprived communities. Part of the available resources should be used for incentive

and motivation of volunteers and CHO/CHN to do more home visits. Regional and district

leadership should reinforce the home-based postnatal care as part of routine activities and

demand accountability from the CHOs/CHNs by periodically assessing their home visit

registers. The number of newborns visited at home at day 3 and day 7 within 0-7 days

following birth should be included in the performance appraisal of the CHOs and CHNs.

Cross Sectoral Support:

10. Quality Assurance (QA), Quality Improvement (QI) and access to quality newborn care

services are important instruments that need to be deployed to attain the MDGs. MoH/GHS

is already putting in place a QA/QI system for health service delivery. This should be

extended to newborn care services at all levels, including at the facility levels with providers

and at the community level with volunteers. There should be an external and internal

newborn care quality audit system. The proposed system will increase competition and

motivate staff to better perform. It should also provide the needed supportive supervision

and mentorship to improve the quality of the newborn care services offered by the health

providers.

Page 12: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

12

11. National and subnational levels should support effective documentation, communication

and advocacy activities for newborn care. They should develop a communication strategy

for newborn care with a costed implementation plan. Further steps should be taken to

document newborn stories in the field (what is happening? what is new?) then record them

and use media to disseminate them. With support from local community members they

should document bad perceptions towards newborns especially neonates, negative

sociocultural practices and address them with behavioral communication, education of

mothers, husbands, in-laws and families during home visits and social/community

mobilization. They should also intensify health education involving community members,

opinion leaders, traditional and religious leaders to recognize the importance of care

requirements for newborns and mothers in order to improve survival rates. They should

consolidate the gains in C4D activities on newborn care using community volunteers in

order the generate demand. This will require the involvement of the District Assemblies to

motivate the volunteers actively involved in C4D to ensure that they continue the home-

based newborn care activities.

12. National and subnational levels should support secondary data analysis to identify barriers

to newborn care and address them and use operational research results to put more evidence

on the table in order to support the mobilization of funds and advocacy for newborn care.

Steps should be taken to address the issue of gender and ensure greater male involvement.

A human rights-based approach and equity should be part of the sub-national

implementation of newborn care activities. A system that will contribute to the sustainability

of newborn care activities and use quality improvement method at facility level with rewards

to regions, districts and selected providers that are improving newborn care should be

implemented.

Page 13: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

13

1. Introduction

1.1 Background

The days and weeks following childbirth – the postnatal period – is a critical phase in the lives of

mothers and newborn babies. Major changes occur during this period, which determine the wellbeing

of mothers and newborns. Yet, this is the most neglected time with regard to the provision of quality

services. Lack of appropriate care during this period could result in significant ill health and even

death. Most infant deaths occur during this time. The number of child deaths worldwide has declined

markedly in recent decades, largely through interventions to lower mortality after the first month of

life. The mortality rate among children under five years of age has fallen globally by 47% (from 90

deaths per 1000 live births in 1990 to 48 deaths per 1000 live births in 2012), but the neonatal

mortality rate (NMR) decreased only by 37% (from 33 deaths per 1 000 live births to 21 deaths per

1000 live births) over the same period and represented, in 2012, 44% of the total under five mortality

[1]. The global annual average rate of reduction in NMR since 1990 has been 2.0%, lower than that

of maternal mortality (2.6%) and under-five-year old mortality (2.9%) [2].

In Ghana, around 38 per cent of under-five deaths and 60 per cent of infant deaths occur during the

newborn period. According to the 2011 Multiple Indicator Cluster Survey, the U5MR was estimated

at 82 deaths per 1000 live births – that means 82,000 children die before reaching 5 years. Out of

these, 32, 000 die in the newborn period resulting in a neonatal mortality rate of 32 neonatal deaths

per 1000 live births [3]. An Emergency Obstetric and Neonatal Care (EmONC) assessment conducted

by the Ghana Health Service (GHS) in 2010 reported birth asphyxia as the major cause of intra-partum

neonatal death (41 per cent) at the facility level [4]. This is different than global causes where

prematurity is a major cause. In Ghana, however, birth asphyxia as major cause suggests that there is

an issue of quality of care at the facility level. It is thus critical to respond to these major causes of

neonatal deaths in order to accelerate neonatal mortality reduction.

The global response to end preventable newborn deaths led to the launch of a Global Newborn Action

Plan in June 2014 [5]. Targets have been set to reduce mortality rates and WHO and UNICEF lead

this work. This plan is bold and calls for a global Neonatal Mortality rate of 7 per 1000 live births by

2035. In Ghana, a number of policy responses were initiated including development of a National

Child Health Policy (2007-2015), which provides the framework to improve child survival along the

continuum of care for mother and child and MDG5 Acceleration Framework and Action Plan (MAF)

2011, which identified and prioritized three key areas of intervention: family planning, skilled

delivery, and EmONC for saving the lives of mothers and babies [6]. The Health Sector Medium-

Term Development Plan, 2014-2017 has also outlined improvement of access and quality of maternal

and newborn care as one of the critical interventions.

In line with the National Child Health Policy, UNICEF with funding support from the Government

of Japan has been providing technical and financial assistance to Ghana Health Service at the National

level and in a select fourteen districts of Northern and Upper East regions since October 2011 to

implement the project entitled “Accelerating efforts to reduce maternal, neonatal and child

mortality in the Northern and Upper East regions of Ghana”. The project was implemented in

Page 14: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

14

two phases. Phase 1 covered the period September 2011 to December 2013 in 11 districts in the

Northern and Upper East regions whereas phase 2 started in January 2013 and ended in December

2014 in the same focus districts as phase 1, but with the addition of three new districts in the Upper

East region resulting in a total of fourteen project districts. The key components of the project

included:

Minimum of two home visits (to mother and newborn) within the first 7 days after delivery

by appropriately trained community health workers;

Integrated Management of Neonatal and Childhood Illnesses (IMNCI) at facility and

community levels;

Scale up of Basic Emergency Obstetric and Newborn Care (BEmONC) at Community Health

Planning and Services (CHPS) and Health Centres;

Developing capacities on life-saving skills for midwives;

Promotion of key household and community practices related to delivery and newborn care;

Leveraging existing resources and initiatives in the project area;

Using a systems strengthening approach to enable sustainability beyond project period

To provide evidence on the effectiveness of this project for possible scale-up, an independent

formative evaluation was commissioned with funding from UNICEF Ghana. The aim of this

formative evaluation is to assess key components of the project.

1.2 Specific objectives of the formative evaluation:

1. To ascertain the effectiveness of the package of evidence-based maternal, neonatal and child

survival interventions with a focus on community-based care (Home-based Postnatal Care

(HBPNC)) and facility-based integrated Management of Newborn and Childhood Illnesses

approaches in the selected districts of the Northern and Upper East regions;

2. To ascertain the project’s contribution for capacity building, developing training resources

and facility structures to respond to the high levels of newborn morbidity and mortality in

selected districts of the Northern and Upper East regions;

3. To ascertain the effectiveness of the project’s evidence-based advocacy for issues related to

newborn survival and national policy environment,

4. To draw lessons on the implementation capacity for national scale-up of the essential

newborn care model through home-based early postnatal care.

Page 15: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

15

2. Scope of the Evaluation and Evaluation Questions

It is expected that the project partners– Ministry of Health (MoH), GHS, UNICEF and JICA – will

use the findings of the evaluation in their different capacities and functions, to develop future plans

and interventions and to inform policies and strategies.

2.1 Scope of the formative evaluation

The scope of the formative evaluation covered the areas of implementation of the project in the two

regions (Northern and Upper East Region) of Ghana. This evaluation also expanded its scope to the

national level, to ascertain its sphere of influence on the overall maternal newborn and child health

(MNCH) programming in Ghana. The evaluation covered the overall Government of Japan, UNICEF

and Government of Ghana’s partnership on the newborn health programming from the two phases of

the project from September 2011 to December 2014.

The evaluation focused on, and included the following beneficiaries and stakeholders in the process:

Final beneficiaries: newborn babies, mothers and caregivers;

Service providers: healthcare professionals whose capacity had been built (including doctors,

midwives, community health nurses and sub-district health professionals);

Sub-national decision-making level: District and Regional health authorities;

National decision-making level: national authorities and key stakeholders (MoH, GHS,

Development Partners – JICA, USAID, EC, PATH, JHPIEGO, UN System- UNICEF, WHO,

UNFPA, National Newborn Care committee);

National Professional Societies and Academia: Paediatric Society of Ghana, Society of

Obstetricians and Gynaecologists of Ghana, Ghana Registered Midwife Association, Teaching

Hospitals, School of Public Health.

2.2 Evaluation questions:

The formative evaluation attempted to provide answers to a number of questions to meet

Development Assistance Committee of the Economic Cooperation and Development (OECD/DAC)

evaluation criteria of relevance, effectiveness, efficiency, and sustainability including UNICEF’s

Coherence and Human Rights-Based Approach (HRBA) to Programming and Equity for all target

groups. Following an inception meeting with the National Newborn Sub-Committee and other

stakeholders at the national level, the evaluation questions were reviewed using the Terms of

Reference as the basis (See Table 2.1).

Page 16: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

16

Table 2.1: Evaluation criteria, targets and questions

Criteria Questions/Target groups

Relevance National decision-making level: Is the intervention relevant in terms of alignment of project objectives with national

strategy and stakeholder’s priority and needs?

Effectiveness:

National decision-making level: To what extent has the project contributed to the policy direction of the National

Newborn Strategy and Action Plan in terms of advocating for and facilitating to bring

newborn health onto the national agenda of MNCH Programming?

Sub-national decision-making level: To what extent has the project contributed to strengthen capacity of regional health

management teams and district health management teams for planning, informed

decision making and prioritization of the newborn health as per the National Child

Health Policy (2007-2015) and other national guidelines and protocols?

Service providers’ level: To what extent are the established level-two (without ventilator and incubators)

newborn care units in six District Hospitals perceived to have improved the

management and survival of sick newborn babies? Which are the

enabling/constraining factors that facilitated/hindered the management of sick

newborn babies in District Hospitals?

To what extent has there been an improvement in quality of care during post-natal

care in the health facilities targeted by the project?

To what extent has the training and mentorship component of the project responded

to capacity building needs of the different levels of service providers?

Final beneficiaries’ level: To what extent do beneficiaries report to have been reached by project

communication and social mobilization interventions, like community durbars,

mother support groups, community-based agents (CBA) and Red Cross mothers (in

the Upper East Region)?

To what extent do beneficiaries report an improvement in their newborn care and

health seeking practices (ANC, PNC, well baby clinic) as a consequence of improved

counseling by Community Health Officer (CHO)/Community Health Nurse (CHN),

CBA, Red Cross mothers and mother support groups?

To what extent has the intervention contributed to improve health and wellbeing of

newborn babies?

Efficiency:

National level: Were the allocated resources used efficiently to achieve the project objectives? Are

the available resources adequate to meet project needs?

Sustainability:

National level: Have policy makers at MoH/GHS demonstrated ownership over the different

interventions related to newborn survival?

Has the Government of Ghana prioritized the health and wellbeing of newborn

babies in the government’s policy documents (Ghana Shared Growth and

Page 17: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

17

Development Agenda for 2014-2017, Medium Term Health Expenditure Plan for

2014-2017) and allocation of resources (budget line on approved Ministry of Health

budget) for newborn health?

Sub-national decision-making level: Have District Health Management Team (DHMT) and District Assembly (DA)

demonstrated ownership and capacity for resource mobilization to be able to self-

support and consolidate the achievements and the expansion of newborn health

interventions?

Service providers’ level: Can the commitment and motivation of CHO/CHN and community volunteers that

were enhanced through the project last for a continued provision of home-based

services to mothers and their newborn babies? What are the bottlenecks and barriers

for home-based postnatal care within the framework of continuum of MNCH care?

Final beneficiaries’ level:

Can the behavioral changes among beneficiaries on essential newborn care be

sustained?

Coherence: National Has the project facilitated synergies and avoided duplications with interventions and

strategies promoted by other UN agencies and development partners (JICA, USAID,

EC, PATH and others) within the National Child Health Policy 2007-2015 and MAF?

Has the project given due importance to donor’s (Government of Japan) visibility in

line with UNICEF’s donor visibility guidelines;

Human rights-

based

approach

(HRBA):

National Has the project incorporated the HRBA to programming?

Has the project considered the equity approach (i.e. focus on most deprived areas,

areas with high prevalence of critical newborn and under-five mortality, low income

families) as well as facilitated the reduction of access barriers to MNCH services by

the target group?

Criteria Questions/Target groups

Relevance National decision-making level: Is the intervention relevant in terms of alignment of project objectives with national

strategy and stakeholder’s priority and needs?

Page 18: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

18

Effectiveness:

National decision-making level: To what extent has the project contributed to the policy direction of the National

Newborn Strategy and Action Plan in terms of advocating for and facilitating to bring

newborn health onto the national agenda of MNCH Programming?

Sub-national decision-making level: To what extent has the project contributed to strengthen capacity of regional health

management teams and district health management teams for planning, informed

decision making and prioritization of the newborn health as per the National Child

Health Policy (2007-2015) and other national guidelines and protocols?

Service providers’ level: To what extent are the established level-two (without ventilator and incubators)

newborn care units in six District Hospitals perceived to have improved the

management and survival of sick newborn babies? Which are the

enabling/constraining factors that facilitated/hindered the management of sick

newborn babies in District Hospitals?

To what extent has there been an improvement in quality of care during post-natal

care in the health facilities targeted by the project?

To what extent has the training and mentorship component of the project responded

to capacity building needs of the different levels of service providers?

Final beneficiaries’ level: To what extent do beneficiaries report to have been reached by project

communication and social mobilization interventions, like community durbars,

mother support groups, community-based agents (CBA) and Red Cross mothers (in

Upper East Region)?

To what extent do beneficiaries report an improvement in their newborn care and

health seeking practices (ANC, PNC, well baby clinic) as a consequence of improved

counseling by Community Health Officer (CHO)/Community Health Nurse (CHN),

CBA, Red Cross mothers and mother support groups?

To what extent has the intervention contributed to improve health and wellbeing of

newborn babies?

Efficiency:

National level: Were the allocated resources used efficiently to achieve the project objectives? Are

the available resources adequate to meet project needs?

Sustainability:

National level: Have policy makers at MoH/GHS demonstrated ownership over the different

interventions related to newborn survival?

Has the Government of Ghana prioritized the health and wellbeing of newborn

babies in the government’s policy documents (Ghana Shared Growth and

Development Agenda for 2014-2017, Medium Term Health Expenditure Plan for

2014-2017) and allocation of resources (budget line on approved Ministry of Health

budget) for newborn health?

Sub-national decision-making level: Have District Health Management Team (DHMT) and District Assembly (DA)

demonstrated ownership and capacity for resource mobilization to be able to self-

support and consolidate the achievements and the expansion of newborn health

interventions?

Page 19: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

19

Service providers’ level: Can the commitment and motivation of CHO/CHN and community volunteers that

were enhanced through the project last for a continued provision of home-based

services to mothers and their newborn babies? What are the bottlenecks and barriers

for home-based postnatal care within the framework of continuum of MNCH care?

Final beneficiaries’ level:

Can the behavioral changes among beneficiaries on essential newborn care be

sustained?

Coherence: National Has the project facilitated synergies and avoided duplications with interventions and

strategies promoted by other UN agencies and development partners (JICA, USAID,

EC, PATH and others) within the National Child Health Policy 2007-2015 and MAF?

Has the project given due importance to donor’s (Government of Japan) visibility in

line with UNICEF’s donor visibility guidelines;

Human rights-

based

approach

(HRBA):

National Has the project incorporated the HRBA to programming?

Has the project considered the equity approach (i.e. focus on most deprived areas,

areas with high prevalence of critical newborn and under-5 mortality, low income

families) as well as facilitated the reduction of access barriers to MNCH services by

the target group?

Page 20: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

20

3. Evaluation Methods

3.1 Ethical considerations

Individual consent was obtained before interviews or focus group discussion (FGD) were conducted.

Each participant was told the purpose and use of the information being collected by the evaluator.

Questions were posed to collect their opinions and views about the activities of the newborn projects

and their possible impact on newborn health indicators. Before the start of the interview or FGD,

participants were told that their participation was voluntary and that they were free to withdraw from

the interview or FGD at any time. They were also told that the information provided would remain

confidential and used anonymously. Participant who gave their consent were given an informed

consent form for signature (See Appendix AP1). Interviews and FGD were conducted in a private and

isolated place to ensure confidentiality and provide a comfortable environment. Evaluators did not

seek personal information, or opinions believed to be controversial. No risk is expected for

participants since the main aim of the evaluation is to improve newborn health (See Appendix AP2a0).

3.2 Study design

The formative evaluation employed a mixed method design consisting of qualitative and quantitative

components. The qualitative component consisted of in-depth interviews with key national decision-

makers, donor partners, sub-national authorities and service providers, and focus group discussions

(FGD) with mothers or caregivers drawn from the communities in one project district. Interview

guides (See Appendices AP2a1, AP2b1, AP2c1) and the FGD guide (See Appendix AP2d) were

developed for the in-depth interviews and FGDs respectively, based on the evaluation questions along

the OECD/DAC evaluation criteria for relevance, effectiveness, efficiency, and sustainability. It

also considered the two additional criteria of interest to UNICEF namely: Coherence and Human

Right-Based Approach to Programming and Equity.

The interview guides were translated into semi-structured questionnaires (See Appendices AP2a2,

AP2b2, AP2c2) with Likert scale responses (i.e. 1=fully disagree; 2=disagree; 3=no opinion; 4=agree;

5=fully agree) to quantify the degree of agreement, or disagreement to a set of statements or

declarations drawn from the interview guides. Respondents, were however, not prompted for the

reasons of their disagreement or their lack of opinion. The questionnaire was administered to as many

stakeholders and service providers as possible including those who took part in the in-depth

interviews. These questions were first pre-tested for their validity and their ability to elicit the right

kind of responses, they were then reviewed (as necessary) prior to being finalized for the main

fieldwork.

The quantitative component involved the abstraction of neonatal health indicators from the District

Health Information Management System (DHIMS (2) and other relevant data sources based on

indicators developed from the evaluation questions as well as project-specifics to assess the project’s

success and effectiveness. Service output indicators were extracted for the region as well as the project

districts. Due to the rarity of certain events, impact indicators such as neonatal deaths per 1,000 live

births were extracted as a regional level indicator for analysis.

Page 21: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

21

3.3 Sample size consideration and sampling

For the quantitative component, all 14 project districts were included in the extraction of key neonatal

health indicators.

For the qualitative component, in-depth information were collected from key target groups. Since this

was a qualitative research with recorded interviews, the evaluation team planned to carry total number

of interviews, for the entire evaluation exercise, of less than 45 for the two regions including FGD

sessions and at the national level but with adequate representation of all levels of the target groups.

At the national level, 12 key informants (See Appendix AP4) were contacted for in-depth interviews

in consultation with UNICEF Team. For the two regions, sampling for the in-depth interviews and

FGD sessions happened in two steps. First, the 7 districts in each project region were stratified

according to agreed criteria with stakeholders (Urban with hospital where there is a Neonatal Care

Unit, Rural with hospital, and Rural without hospital). One district was randomly selected from each

of this stratum giving a total of 3 districts for the interviews or FGD. If a stratum had only one district,

that district was included by default. The results of the selection were Kpandai, Savelugu-Nanton and

Tolon-Kumbungu in the Northern region, and Bolga Municipal, Kasena-Nankana West and Talensi-

Nabdam in the Upper East region. However, after a meeting between the evaluation team and the

regional director in the Northern Region, it was agreed to replace Kpandai with Bole since Bole and

Kpandai shared similarities in their performance for newborn care, but Bole had more frequent and

active home-based visits and social mobilizations compared to other selected sites. Having Bole

would cover that aspect of home visits as a key component of newborn care. Talensi-Nabdam was

also replaced with Bawku West in consultation with the regional director of the Upper East Region,

because the evaluation team thought that it was better to have representation of the districts that

experienced the two phases of the project.

Afterwards, 2 facilities (1 Health centre and 1 CHPS compound) were selected in consultation with

the District Director based on the fact that the facility in-charge (Midwife or CHO) had been in post

for the duration of the project. The midwife, CHN or CHO in each selected facility was contacted for

an in-depth interview. For the final beneficiaries in Bolga and in Savelugu districts, the evaluation

team, in consultation with the regional director, agreed to draw 8 to 12 mothers of newborn babies

from the communities with the assistance of the district director and community health officers. The

CHO explained the aim of the FGD to them, and upon obtaining their consent, they were asked to

report to the sub-district health centre. At the health centre, the evaluation team sought their consent

to participate in a FGD session. Drawing the mothers from the project communities was preferred to

the proposed approach of selecting PNC registrants from a NCU facility due to the possibility of

sickness of the mothers’ babies which may result in a state of distress, thus corroding their ability to

participate in a FGD. The regional directors in the project regions and the district directors in the

selected districts were also interviewed. These amount to a total of 28 in-depth interviews and two

FGDs carried out (Table 3.1).

Page 22: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

22

Table 3.1: Sample size for the in-depth interviews and focus group discussions among

sub-national, service providers, and final beneficiary target groups

Region Selected districts

Number of In-depth interviews Number

of FGDs

Total

Regional

Director/

NBC

Coordinator

District

Director/

NBC

Coordinator

Midwife

/CHO/

CHN

Northern Bole 2 2 2 6

Northern Savelugu-Nanton 2 2 1 5

Northern Tolon-Kumbungu 2 2 4

Upper East Bolga Municipal 2 2 2 1 7

Upper East Kasena Nankana West 2 2 4

Upper East Bawku West 2 2 4

Total 4 12 12 2 30

3.4 Field work/Data collection

For the quantitative component of the evaluation, key neonatal health indicators at the district level –

for all the 14 project districts – and at the regional level were extracted from DHIMS (2) over the

period spanning from 2011 to 2014 and submitted to consultants through a formal request by UNICEF

Ghana to the Director-General of the Ghana Health Service. Two NCU facilities (one in each project

region) were visited to assess their capacity for newborn care particularly focusing on referral-in and

referral-out indicators.

For the qualitative data, the evaluation team had in-depth interviews with 12 key national decision

makers and health partners. Following this, the evaluation team made a two-week field visit to six

districts in the two project regions and had in-depth interviews with sub-national authorities and

service providers as well as FGD with women beneficiaries. Table 3.2 in the appendix AP5a shows

the work schedule of the consultants.

Page 23: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

23

Figure 3.1: Maps of the Northern and Upper East regions with the

14 districts of newborn care intervention and 6 selected districts for interviews.

DISTRICTS OF THE NORTHERN REGION

Districts are colored for emphasis

DISTRICTS OF THE UPPER EAST REGION

Districts are colored for emphasis

Page 24: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

24

3.5 Data Analysis

For the quantitative component, indicators were defined for each evaluation question (where

appropriate). The percentage change in key newborn health indicators over the period spanning from

2012 to 2014 was estimated to ascertain the effect of the project in improving newborn survival.

Where the rate of change in the indicator in the project districts was faster than that in the entire region

over the same period, it was suggestive of the intervention being effective. Also, examining coverage

trends was essential for assessing project progress. Information on trends required at least two

separate and comparable measurements at two points in time. A measure of progress – the coverage

gap – defined as how much coverage would need to increase from the 2013 level to reach universal

coverage was estimated to examine coverage trends. The change from 2013 to 2014 was then

expressed as a percentage of this gap.

For the qualitative data, recorded interviews were transcribed verbatim. Data were analyzed manually

using two analytic approaches, namely: (1) Thematic analysis – looking for themes and patterns

among data (verification); and (2) Narrative analysis – in order to identify narratives or cases, and

explore how they differed between groups. No qualitative data analysis software was used for this

evaluation. The ideas, views, opinions and quotations from the transcribed (verbatim) data summaries

and the notes, were used to illustrate the reports by evaluation criteria. For the focus group discussion,

a Matrix for assessing the level of consensus in the focus group was used (Table 3.2). Data from the

semi-structured questionnaires were summarized using proportion of respondents with degree of

agreement or disagreement to a set of statements or declarations.

Table 3.2: Matrix for assessing level of consensus in focus group discussion

Focus

Group

Question

Member 1 Member 2 Member 3 Member 4 Member 5

1

2

3 The following notations were entered in the cells:

A = Indicated agreement (i.e., verbal or nonverbal) D = Indicated dissent (i.e., verbal or nonverbal)

SE = Provided significant statement or example suggesting agreement

SD = Provided significant statement or example suggesting dissent

NR = Did not indicate agreement or dissent (i.e., nonresponse)

3.6 Data archiving

All data collected during this evaluation exercise including recorded interviews (in MP4 format) and

transcribed data (in Word format and electronic version) were submitted to UNICEF Ghana Office

for archiving.

3.7 Quality Assurance

Researchers took appropriate and necessary measures to ensure the quality of the data collected from

the key informant interviews by minimizing ambiguity when presenting the questions to the

Page 25: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

25

interviewees. As in many health information systems, the researchers recognized the limitations of

the quantitative data extracted from the DHIMS (2) and tried – within the limited time for this exercise

– to validate suspected figures with facility records.

Researchers also recognized the limitations of in-depth interviews and FGDs in terms of the small

sample size or the limited number of participants interviewed. The information collected was

therefore analyzed and interpreted within their contextual thematic scheme and the individual

opinions and views expressed.

3.8 Challenges and limitations

As mentioned in section 3.7, extracting quantitative data from the DHIMS may have posed some

limitations relating to the completeness and accuracy of the data. Researchers used other sources

whenever possible to validate suspected figures.

Qualitative data have their own limitations in terms of sample size and generalization to the entire

population under study. For this evaluation, opinions and views collected during in-depth interviews

and FGDs with limited number of participants were therefore analyzed and interpreted within that

context of individual ideas and appreciations.

In the absence of control areas, it is difficult to attribute observed changes and achievements entirely

to implementation of the newborn care project which did not cover all the districts in each region.

However, examining changes from one period to another and trends over a period of time is bound to

assess the contribution of the NBC project when quantitative data is available.

The replacement of Kpandai district with Bole district which had very active home visits and social

mobilization may be seen as bias. This choice by the evaluation team and regional authorities offers

the advantage to better assess potential contribution of social mobilization to improving newborn

health in addition to home based care.

Page 26: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

26

4. Evaluation findings

This chapter presents the results of the evaluation exercise organized by four evaluation objectives

taking into account the evaluation criteria. For each of the four evaluation objectives and the relevant

evaluation criterion the evaluation finding was quoted and the available evidence was provided as an

explanation. The available evidence was gathered from the desk review, DHIMS (2)/facility records,

semi-structured questionnaires, in-depth interviews and the focused group discussions.

Objective 1: To ascertain the effectiveness of the package of evidence-based maternal, neonatal

and child survival interventions with a focus on community-based (Home-based Postnatal) Care

(HBPNC) and facility-based integrated Management of Newborn and Childhood Illnesses approaches

in the selected districts of the Northern and Upper East regions;

The established level-two newborn care units in District Hospitals have contributed, to some

extent, to an improved neonatal survival through an improved management of sick newborn

babies. However, challenges such as cases from the communities arriving late for management

still remain.

For example, institutional neonatal deaths per 1,000 live births reduced by about 51% (6.9‰ to 3.4‰)

in the Northern region and by about 43% (5.8‰ to 3.3‰) in the Upper East region over the project

period spanning from 2012 to 2014 (Figure 4.4). However, it should be noted that the project was

implemented in well identified districts and not in the entire region which contains several more

districts. Disaggregated and analyzed by project district, these findings show mixed results and are

presented in table 4.1 in the appendix AP5a.

The evidence from one neonatal care unit suggests improved management and survival of sick

newborns. For example, the NCU at Bolgatanga Regional Hospital in the Upper East recorded

downward trends in neonatal deaths per total admission since the start of the NCU in January 2014

(Figure 4.4a). It is possible that the skills acquired during the training are being applied in managing

sick babies referred to the NCU. However, there appeared to be stagnation in the Savelugu District

Hospital NCU in the Northern region (Figure 4.4a). A well-conducted death audits would bring out

the reasons for such a stagnation.

Page 27: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

27

Figure 4.4: Percentage change in institutional neonatal deaths per 1,000 live births over the

2012-2014 period in the project regions.

There was a general view among service providers (27 out of 32) that the established level-two

newborn care units in Hospitals improved the management and survival of sick newborn babies

(Figure 4.4b). However, three of them disagreed and two had no opinion probably because of the

challenges.

Page 28: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

28

The major element of success with NCU is the supply of equipment for saving the lives of preterm

babies, low birth weight babies, and babies with hypothermia or asphyxia. This equipment included

the resuscitation machine, the incubator, the baby warmer and the phototherapy machine. Other

important elements are the Kangaroo mother care, the body temperature controller, the Random blood

sugar test, the oxygen concentrator, and the fluid therapy.

The midwives in two NCUs assessed during the field visit mentioned some factors as having

contributed greatly to the reduction in neonatal deaths in the facilities. However, the statistics from

the facilities were not available to the evaluators to illustrate the impact of the equipment in the NCU

on the survival of premature newborns as stated by the midwives.

In spite of these achievements there remains serious challenges at the NCU in terms of staff attrition

– a number of staff trained in newborn care have left following their posting elsewhere or to further

their studies. For example, in the Upper East region out of 6 staff trained who started the NCU, only

3 were still working in the NCU during this evaluation. The other challenge is the lack of oxygen

cylinders. UNICEF brought more oxygen concentrators to the NCU. Participants also mentioned the

breakdown of the incubators, which they have no local capacity to repair. The administration was,

however, in contact with Accra which deployed a technician to repair the malfunctioning device.

Strategies to educate mothers to leave their babies in the NCU is also a challenge. This is because

mothers do not like to be separated from their babies at the hospital. This highlights the need to have

room in the facilities for mothers to stay. Sometimes, staff run out of key medicines for the newborn

and have to prescribe them. The Northern Region also faces some challenges as stated in the quote

below:

The beneficiaries expressed satisfaction with the home-based care their babies received from

the CHOs/CHNs. They also indicated that they received basic counseling on health and

wellbeing of mothers and babies and that the regular interaction with the CHOs/CHNs and

community volunteers provided the opportunity to share their experience and bring up their

challenges on providing care to their newborn babies, and taking care of their own health.

The results suggest that through sustained counseling and communication for development activities,

mothers were informed on the importance of kangaroo mother care to ensure the survival of low birth

weight babies.

“Another challenge is the space available for NCU which is too small. Normally when

babies are referred to us from home, we are not supposed to put them together with those

babies born in the hospital, but with limited space we have no choice than to put them

together. We also need a pediatrician for our NCU so that the cases we now refer to Tamale

Teaching Hospital can stay with us. Most of our client mothers do not like to be referred.”

(Provider of NCU in the Northern region)

Page 29: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

29

FGD mothers in the Northern region told us that they now report diarrhea cases, vomiting and fever

to the health facility and found that the treatments at the facility were effective to stop sickness in the

child. They emphasized that when they applied what they have learnt, they found that babies were no

longer falling sick and both mother and baby were in good health. This is in line with improvements

in newborn care and health seeking practices reported by the beneficiary mothers. The quote below

from a mother in the Upper East is reinforcing the observed improvements.

Table 4.3:Trends in uptake of kangaroo mother care, Half Year (HY) 2011 to

2014, Upper East region

Indicators 2011

HY

2012

HY

2013

HY

2014

HY

% change

(2014 vs 2013)

No. LBW Babies 912 963 1,138 1,297

% LBW 6.7 8.8 10.3 8

No. on KMC 99 175 249 536

% of LBW on KMC 10.9 18.2 21.9 41.3 88.9

Source: regional statistics data

“From the pictures we learnt that after using toilet we should wash our hands before

touching the baby and also before breastfeeding the baby. Putting all these into practice

helped us to keep the child away from sickness. Before many pregnant women were

delivering at home. These days more women are delivering in health facilities and we no

more see mothers and babies dying when we give birth there.”

(FGD participant, Upper East region)

Page 30: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

30

Photo 4: A Focus Group Discussion session in Upper East Region, 17th January 2015

Perceptions and reported statements from beneficiaries are that the health and wellbeing of

babies have improved substantially to their satisfaction. They reported positive behavioral

changes and less diseases due to the newborn care intervention – especially education by the

nurses and volunteers during home visits and supportive communication activities.

CHO and CHNs focused on improving the quality of newborn care provided during the postnatal

home visit. The long-term expectation is that every newborn will receive home-based care by a trained

health worker. In the Northern region, the percentage of babies visited at home by trained health

workers increased from about 30% in 2013 to 37% in 2014, representing a 10.5% gap closed to

achieve universal coverage of all newborn with home-based care (Table 4.4). The Saboba district was

the highest performer (53.8% of gap closed) with Yendi being the worst performer, which recorded a

reduction in home-based visits. In the Upper East region, the percentage of babies visited at home by

trained health workers increased from about 39% in 2013 to 58% in 2014, representing a 30% of gap

closed to achieve universal coverage of all newborn with home-based care (Table 4.4). Bawku West

was the highest performer with 45% of gap closed.

Page 31: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

31

The strategy of using community volunteers, who are very familiar with the communities and persons

within the communities, as agents to identify households where pregnant women are, and where

recent deliveries have occurred and inform and link-up with the CHOs/CHNs for scheduled visits was

an enabling factor for the home-based postnatal newborn care.

Table 4.4: Percentage of newborn babies who were visited at home by trained health worker

Region Project Districts

Expected target

neonates (i.e. 80%

of expected

deliveries)

Number of

babies visited

by health

worker

% of babies visited

% of gap

closed =[B-

A]*100/[100-

A]

2013 2014 2013 2014 2013

(A)

2014

(B)

Nort

her

n

Bole 2147 2210 805 866 37.5 39.2 2.7

Gushiegu 3879 3992 1681 1856 43.3 46.5 5.6

Kpandai 3794 3904 438 778 11.5 19.9 9.5

Saboba 2291 2357 520 1516 22.7 64.3 53.8

Savelugu-Nanton 4856 4997 1256 1795 25.9 35.9 13.6

Tolon-Kumbungu 3699 3699 410 654 11.1 17.7 7.4

Yendi 4123 4243 2231 1928 54.1 45.4 -18.9

Total All seven districts 24790 25402 7341 9393 29.6 37.0 10.5

Upper

Eas

t

Bawku Municipal 3193 3231 1204 1067 37.7 33.0 -7.5

Bolga Municipal 4363 4415 1657 2005 38.0 45.4 12.0

Kasena Nankana East 3646 3690 888 1102 24.4 29.9 7.3

Bawku West 3119 3156 2003 2533 64.2 80.3 44.8

Garu Tempane 4312 4363 3940 4064 91.4 93.1 20.4

Kasena Nankana West § 2344 2372 - 1379 - 58.1 -

Nabdam § 1136 1284 - 832 - 64.8 -

Talensi § 2679 2711 - 1511 - 55.7 -

Total All seven districts 24791 25223 9692 14493 39.1 57.5 30.2

Source: Home-based Postnatal Newborn Care Report, 2013-2014

§ Started reporting January 2014

Focus group discussions with beneficiaries who were mothers selected from rural communities of the

Northern and Upper East regions reported significant positive behavioral change towards newborn

care among the population as a consequence of home-visits and education by nurses and volunteers,

and supportive communication activities of the intervention. Most of them reported that the days of

children with many diseases are now behind them. They expressed hope in the future of their babies

and wished that the home visits would continue. The following are illustrative statements by mothers

during the FGDs:

Page 32: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

32

Participants in the Northern Region also reported that the health and wellbeing of their babies had

improved and their children were no longer as sick as they were in the past, thus affording them time

to go to the market to sell their wares and do business, or perform farming activities to support their

families instead of worrying, or having sleepless nights when their babies are sick. This expressed

happiness was shared by many of the mothers during the FGD sessions. One mother reported that

when she gave birth to a low birth weight baby, the nurse visited her at home and referred her to the

NCU where she was taught Kangaroo mother care, exclusive breastfeeding and other general newborn

care practices. Today the baby has put on weight and is doing well. The quote below reinforces the

perceived improvements in the health and wellbeing of the babies.

Beneficiaries also reported significant improvement in mothers’ health due to the intervention, which

sensitized and mobilized pregnant women to go to health facilities for antenatal care and facility

deliveries. FGD participants noted that the newborn care project is of great psychological helps to the

mothers. Since it reduces the frequency and severity of sickness in babies and children, it alleviates

the psychological and physical burden on mothers who no longer have to worry or spend sleepless

nights with their sick children as was the case in the past. These days, the babies are healthier and

mothers can stop worrying and carry out their daily economic activities. As reported by a mother

during the FGD:

“There has been major improvement in the lives of our babies. Our children are no

longer falling sick as it used to be. They are no longer dying or getting sick or having

skin diseases or eye diseases. We now go to the clinic to give birth.”

(FGD participant Upper East region)

“There have been changes. Hygiene practice is really helping since our children

are no longer having diarrhea and cholera as in the past. We were taught first

aid for fever in child. Putting this into practice has reduced convulsions in

children when they have fever. Now it is far better than what it used to be.

Children are no longer dying as we used to see. Immunization is also helping to

have fewer deaths”.

(FGD participant Upper East region)

“It is better now than before when many pregnant women were delivering at home.

These days more women are delivering in health facilities and we no more see mothers

dying when giving birth, even breach delivery is taking place in facility without

complication and both mother and babies are being saved”.

(FGD participant Upper East region)

Page 33: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

33

The project considered the equity approach to programming to some extent because project

interventions were targeted at poor and disadvantaged regions with relatively high maternal

and newborn mortality.

A review of the project proposal indicated that due considerations were given to changes in the burden

of neonatal deaths during the selection of the project regions. It also indicated that the regions were

chosen to consolidate the gains recorded following a series of interventions in the past

However, if the newborn care project in the Northern and Upper East regions has to ensure full equity,

it would necessitate heavy infrastructure, human resource distribution, transport and equipment

investments. For optimal results in a resource-constrained country, it is critical to prioritize activities

and focus on the worse performing areas in order to accelerate the bridging of the inequality gap.

A national key informant attested the following on equity considerations in programming.

The national newborn strategy provides the framework for priority newborn activities, which

can be operationalized through annual plans and budgets with the support of health partners.

A review of the Medium Term Health Expenditure Plan for 2014-2017 and the Ghana Shared Growth

and Development Agenda for 2014-2017 showed that many activities can directly and indirectly

impact the lives of the newborns. The National Newborn Strategy provides a more focused framework

on newborn survival, which can be operationalized through annual plans and budgets with the support

of health partners.

Eleven (11) out of 14 national decision-makers were of the opinion that newborn issues have been

given priority attention in the national and sector policy documents (Figure 4.10) but three (3) of them

gave no opinion or disagreed.

“Sometimes we need to concentrate our efforts into just very few places. For example,

in just Greater Accra or Central region where facilities are better, but need equitable

distribution of equipment and human resources. Then we look at the data that we have,

if many neonatal deaths are occurring in the Central region then it becomes our priority.

This means that we target problem areas where the burden is, concentrate our efforts in

those areas where help is needed and build up capacity without neglecting the other

areas”.

(National key informant)

Page 34: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

34

Objective 2: To ascertain the project’s contribution to the capacity building, developing training

resources and facility structures to respond to the high levels of newborn morbidity and mortality in

selected districts of the Northern and Upper East regions;

The project supported capacity building workshops for all district directors of health services

and public health nurses in the project districts on essential newborn care. They are now able

to plan for newborn care activities. A review of the districts’ annual plans showed that they have

all featured newborn care activities. Key informants reported that the project trained sub-

national personnel as trainers who in turn trained sub-district and community health service

providers on newborn care.

The project succeeded in putting government in the driving seat for newborn care and contributed to

build the capacity of providers to deliver quality service.

All the district directors of health services and district public health nurses in the project districts

reported to have been introduced to, or fully trained in newborn care. 26 out of 27 sub-national

decision-makers agreed or fully agreed that the project actually enhanced their capacity to plan and

prioritize newborn health services (Figure 4.3). One of them, however, had no opinion about this

statement.

Page 35: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

35

The project succeeded in creating a national and sub-national training capacity, which could be used

for in-school basic training of nurses and midwives. The development of the community-training

module for community-based agents, volunteers and nurses was fast-tracked as a result of the project

being implemented in the Northern and Upper East regions as mentioned by one of the national key

informant below:

“Coming back to development of tools, we had started long ago working on the

community module to train community-based workers, volunteers as well as

community health officers but it was left there in a draft form. When the NBC project

started the demand was high so that we had to go back and pick that draft document

and get funding to refine the draft and finalize it for use in training community workers

and volunteers of the project areas. Now it is printed and in use elsewhere and will be

used nationwide to implement the newborn strategy”.

(National key informant)

Page 36: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

36

Photo 2: Some training materials on essential newborn care

Key informants in the Upper East who underwent the training strongly stated that it had built their

capacity for service provision. They also indicated that the bottleneck analysis tool helped identify

the root causes of neonatal deaths as a first step in the planning process. They reported that in the past

when they did not meet their targets, they just assumed that it was due to in- or out-migration or

famine, and never questioned these assumptions or attempted to understand the root causes.

Another key informant in the Northern region reported that with the bottleneck analysis tool, the

bottlenecks and root causes were identified and the team developed a plan with a corresponding

budget to address the identified bottlenecks in newborn care. The plan was regional and every district

had its micro-plan. This plan or micro-plan are also tools to engage other funders in order to help the

district or region to address the bottlenecks.

Page 37: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

37

The project supported capacity building workshops for frontline health workers on essential

newborn care. A total of 4,151 health workers plus community volunteers were trained on

essential newborn care. However, while service providers reported satisfaction with the training

content, the capacity created might diminish over time especially under high attrition situations.

The capacity of a total of 4,151 health workers including Doctors, Midwives, CHOs/CHNs as well as

volunteers and mother support group members was enhanced through training in life-saving skills

and essential newborn care (Table 4.2). They were imparted with the requisite knowledge and skills

to provide counseling, preventive and curative interventions, including referral to higher levels of

care towards accelerated reduction of neonatal deaths.

Table 4.2: Participants in facility and home-based postnatal care trainings during project

implementation from September 2011 to December 2014 (Phase 1 and Phase 2)

Types of

training

Northern Region Upper East Region

Total

Type of

participants

Project

phases

Phase 1 Phase 2 Phase 1 Phase 2

Facility

based

newborn

care

206 57 81 25 369 Doctors, midwives,

medical assistants,

nurses

Home-based

postnatal

care

246 180 98 292 816 CHOs and CHNs

plus enrolled nurses

700 0 598 450 1,748 Community

volunteers

0 0 1,200 0 1,200 Red Cross mother-

to-mother support

groups

0 0 18 0 18 Red Cross

supervisors, mother

to mother support

group facilitators

Total 1,152 237 1,995 767 4,151

Source: UNICEF Ghana-Government of Japan newborn project final report, 2013 and March 2015

27 out of 35 (77%) service providers whose views were sought during the field visit to the project

regions agreed or fully agreed that the training and mentorship component of the project responded

“The program has assisted us very well. We benefited from a number of trainings from

the management level to lower level. Through the training program, we were able to use

the bottleneck analysis to identify our challenges, their root causes and find way-out for

controlling the challenges, we did not say we have a problem, but we have a challenge

meaning that we can find ways to solve them through the system and do better”.

(Key informant in the Northern region)

Page 38: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

38

to their capacity building needs (Figure 4.6). Six of them gave no opinion because they were not

selected for the formal training workshop. The two who disagreed, wanted more topics to be covered

by the training session.

The trainees are now ready with the requisite knowledge and skills to run newborn care units at

hospitals and provide home-based services to mothers and their newborns. The providers were very

happy to have undergone these trainings in newborn care. The statements below from providers are

an illustration.

Some providers in health centers and CHPS compounds who were interviewed indicated that there

were many things which they were not familiar with but have since been educated on during the

trainings on newborn care. These included how to:

“The training in newborn care was useful and helpful for my work. I go back to the

guidelines and protocols to manage each case and problem very well and every day. Before

the training, I knew little about newborn care and managing their sickness was difficult for

me and I use to refer them. The training has added values to my skills and knowledge and

now I am more confident to handle any case or complication”.

(Provider in the Upper East region)

“As a midwife I did not know much about newborn care and the training has equipped me

and given me confidence to do my work in newborn care. Now I have more experience

based on the guidelines and the counseling card. I feel confident to educate mothers even

though I do not understand very well the local language. I use pictures and they understand.

I have trained all my staff and they know how to manage newborn care and do it even if I

am not there”.

(Provider in the Northern region)

Page 39: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

39

conduct home-based follow up or monitoring visits;

talk to mothers during home visits and sometimes husband, mother- and father-in-laws;

take care of the baby or examine or assess the baby and appreciate individual and

environmental hygiene;

keep the baby warm after delivery (skin to skin approach, kangaroo mother care etc.);

position the baby at breast;

care for the newborn and identify danger signs;

perform resuscitation; and

take care of the cord.

.

It would have been good to have the cost analysis of the investment in capacity building but a

robust cost-efficiency analysis is beyond the scope of this evaluation, so researchers were unable

to determine if the allocated resources were used efficiently to achieve the project objectives.

A total of 4,151 health professionals and community-based agents were trained at the cost of eight

hundred and eight thousand, three hundred fifty six (USD 808,356) US Dollars (Table 4.5) meaning

that on average, the cost of training a health professional in newborn care was approximately USD

194.73. Out of USD 1,812,187, 45% (USD 808,356) was spent on training and capacity building and

35% (USD 617,271) was spent on community and facility service delivery and demand generation.

The indirect cost of the project is USD 109,055 (6%).

Table 4.5: Financial Resource utilisation

Item description Phase 1 Phase 2 Amount in

USD

1. Enhanced facility and community capacity

including development of resource materials

346,733.00

461,623.00 808,356.00

2. Community and facility service delivery

and demand generation

394,706.00

222,565.00 617,271.00

3. Strengthened monitoring and evaluation

45,000.00

79,280.00 124,280.00

4. Technical assistance

5,238.00

142,233.00 147,471.00

5. Communications and visibility

0

5,754.00 5,754.00

6.Cross sectoral project support

39,908.00

69,147.00 109,055.00

Total expenditure for Programme

831,585.00 980,602.00 1,812,187.00

Total programmable amount 831,794.00 981,330.00 1,813,124.00

Programmable balance

209.00

728.00

937.00

Source: Programme financial utilization report

phase 1 and phase 2 received from UNICEF

Page 40: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

40

There has been a reported improvement in quality of post-natal care by providers

Thirty-two (32) out of 35 (91%) service providers in selected facilities in the project districts agreed

or fully agreed that there has been an improvement in quality of care during post-natal period in the

health facilities targeted by the project (Figure 4.5). One provider disagreed with this statement and

two of them gave no opinion.

In terms of postnatal newborn care at the facility, the application of skills and knowledge acquired

during the newborn care trainings have helped improve the quality of postnatal care. Also,

supervision, monitoring and mentorship have helped improve the quality of service delivery by the

trained providers as it pertains to newborn care. Following are some quotes from service providers in

support of such quality improvement:

“There has been an improvement in quality of newborn care at the facility. Today due

to the training, we no longer fear or panic when there is a case of sick newborn. We

use the guidelines to manage the case. Our community encourages facility delivery

because of the quality of services provided. We have recorded less neonatal deaths for

the past 12 months”.

(Service provider in the Upper East region)

“Our facility report shows reduction in newborn deaths. Our admission report shows a

big increase in facility deliveries and more surviving babies than before. When we

identify any problem in the newborn that endanger the life of the baby we manage the

case using guidelines or we refer it to NCU. Newborn survival has improved a lot”.

(Service provider in the Northern region)

Page 41: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

41

These statements from providers need to be taken as opinions and views to be crosschecked or

confirmed with existing statistics. In fact, to confirm their statements, providers indicated that prior

to this project, mothers hardly came back to the facility with the baby during the postnatal period.

It is only since this project started, that mothers are using more and more facilities for postnatal care.

The provider said, “We have referred few newborns to NCU and all of them came back well and

healthy”. One provider from Upper East gave her own example of improvement in quality of delivery

saying:

Another provider from the Northern region gave her example of a low birth weight baby weighing

1.5 kg. From the knowledge acquired during the training, she referred the baby to the NCU and the

mother was taught Kangaroo mother care techniques. The baby consequently put on weight and is

now back to healthy levels after discharge from the NCU.”

Some of the beneficiaries reported to have been educated on essential newborn care especially

through community durbars by community volunteers. But many of the mothers reported to

have obtained education and information from nurses and volunteers who spend time with them

during home visits.

During the FGDs mothers reported to have been reached by project communication and social

mobilization interventions like community durbars through volunteers and other community-based

agents. Many of them noted the friendly and persistent attitude of the nurses and volunteers to educate

them about home-based newborn care, and disease prevention. They expressed willingness to obtain

more education and information and have nurses and volunteers spend more time with them during

home visits. The majority of them who took part in the focus group discussions were able to recall

the main messages regarding breast-feeding, hand washing, diarrhea management, danger signs in

children and pregnant women. They confirmed that the information they received was very useful in

their daily life and that they tried to put into practice the advice received.

“After the training, I received a case of twin delivery and the mother was bleeding.

Since I was taught how to stop bleeding, I gave her oxytocin I.V and was able to stop

the bleeding. If it was before the training, I would have just referred the woman to the

district hospital.”

(Service provider in the Upper East)

“I always participate in durbars. They teach us general care of the child, general danger

signs for newborn, hygiene, what to do during pregnancy, importance of facility delivery and

postnatal care, hand washing before eating and after using toilet for both mothers and

children, exclusive breastfeeding up to 6 months, baby feeding after 6 months, bathing and

clothing the baby, Kangaroo mother care. After the durbar they give us drinks and we like

that”.

(FGD participant, Upper East region)

Page 42: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

42

In order to make it easy for mothers to put education knowledge into practice their mother in-law and

husband were also taught newborn care good practices. The following is an illustration from a FGD

with beneficiary mothers.

Visual pictures and flip charts were easier for mothers to understand and retain the message being

transmitted. They found pictures and flipcharts helpful. However, some mothers clearly preferred the

case scenarios used in communicating the messages. In one case, the mother did not follow the

education on newborn care and her baby ended up dying, and in another, a mother put into practice

what she was thought and her baby survived. This was reported by one of the FGD mothers

“Education during home visits helped us to learn many things. It was important that when

nurses and volunteers came for home visits they insisted and met my mother-in-law and my

husband and educated them also on danger signs for the baby, care of the cord, exclusive

breastfeeding up to 6 months, how to position the baby at breast, when to immunize the baby

and that both mothers and baby should sleep under mosquito net. This has helped me to practice

and they support me and remind me what to do. My husband and in-laws are helping me to

keep the household environment clean”.

(FGD participant, Northern region)

“It was easier for us to have pictures and visual material that show how to practice and

compare sick babies and healthy babies when newborn care is provided. Our child

welfare book has at the back of its cover the danger signs for the baby we found it helpful

and we carry it with us”.

(FGD participant, Upper East region)

Page 43: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

43

Photo 3: A Focus Group Discussion session in the Northern Region, 22nd January 2015

Objective 3: To ascertain the effectiveness of the project’s evidence-based advocacy for issues

related to newborn survival and national policy environment,

The aim of the project to improve neonatal survival is very much aligned with the national child

health policy as well as UNICEF’s global mandate to improve child survival.

A desk review of relevant policy documents showed that the objectives of the project were aligned

with the Government of Ghana’s Child Health Policy. The project is very much placed within

UNICEF’s global mandate to improve children’s health. The project is operationally aligned with

UNICEF’s significant presence in northern Ghana including a field office in Tamale with technical

and operational staff who provided close technical and monitoring support during the project. The

project is also very well aligned with the EMBRACE model articulated in the Government of Japan’s

Global Health Policy both conceptually as well as operationally. First, the project in UER and NR

aims to create linkages between facilities and community-based services through various actors at

both community and facility levels. Second, the project is well positioned within the overall

Page 44: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

44

continuum of care for maternal, newborn and child health and leverages UNICEF’s investments for

other parts of the continuum of care which support healthy childhood such as immunizations and

prevention and treatment of the major child-killers such as malaria, diarrhea and pneumonia. .

On seeking the degree of agreement or disagreement of national stakeholders and health partners on

this issue, 12 out of 14 agreed or fully agreed that the intervention is relevant in terms of alignment

of project objectives with the national strategy and the stakeholders’ priorities and needs, and also in

terms of advocating for, and facilitating the introduction of newborn health into the national agenda

of the MNCH Programming (Figure 4.1). However, one stakeholder had no opinion and another fully

disagreed.

.

The project supported the processes leading up to the development and launch of the National

Newborn Strategy and Action Plan. Specifically, the project supported MoH/GHS to organise 3

national level stakeholder meetings on newborn health. It also supported the development of a

bottleneck analysis tool and decentralized the planning and monitoring of newborn health. As

reported by key informants, the launch was very successful, and the advocacy and

communication around it contributed to place newborn health on the national agenda

A desk-review of the project document showed that the project supported (financial and technical)

the MoH/GHS to organize workshops in Accra on newborn health issues. The aim of the meetings

was to have a common understanding of the package of maternal and neonatal health interventions

outlined in the current Child Health Policy and Strategy. The meeting took stock of the

implementation status with a focus on neonatal health. The project has also supported the

Page 45: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

45

development of the perinatal death audit tool. It has also advocated for the Ghana Pediatric Society

to bring newborn care issues into the national agenda. In order to implement the National Newborn

Care Strategy, all 10 regions are now provided with a newborn health focal person.

On seeking the opinion from national decision-makers and health partners, 11 out of 14 agreed or

fully agreed that the project has contributed to the policy direction of the National Newborn Strategy

and Action Plan (Figure 4.2). However three national key informants had no opinion about this

statement.

National Key Informant interviews substantiate the remarkable efforts undertaken by GHS with

support from UNICEF and funding from the Government of Japan, in the use of evidence-based

interventions on newborn and child healthcare in the Northern and Upper East regions. GHS have

built the capacities of primary, secondary and tertiary level healthcare providers in implementing

newborn care activities. Testimonies from stakeholders, providers and beneficiaries were packaged

and videotaped by the communication section of the MOH and used effectively to advocate for

newborn health through media, and during important fora to bring newborn health high on the national

agenda. The launching ceremony was attended by high profile stakeholders, donors and a pool of

national media. There is, however, still room for additional visibility for the newborn health program

in order to mobilize necessary resources to scale it up to a nationwide level and have it on a regional

African agenda. The following are quotes from national key informants in support to the impact of

the advocacy generated by the project.

Page 46: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

46

“I think that we have moved far, there is still room for improvement but we have come very

far because 4 or 5 years ago we did not have much to talk about newborn. Now we have a

newborn strategy. It was actually the Ministry of health and partners who recommended a

newborn strategy and Ghana Health Service was tasked to have it developed. We now have

the newborn indicators for performance monitoring. Now when we attend Ghana Health

Service meetings of directors, Ministry of health, everybody is talking about newborn”.

(National key informant)

“The national newborn strategy was launched in July last year and that launch was a very

big one. The advocacy and communication around it, was such that it went very far. We had

parliamentarians, civil society, embassy representatives, some key media personnel and

representatives from other sectors. Newborn became the talk of the town, Last year for

example almost all the professional association groups, we call them medical

superintendents, adopted a newborn theme for their annual general meeting. It was a very

busy year for us; everybody wants us to come and speak on newborn at their general annual

meeting”.

(National key informant)

Page 47: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

47

Photo 1: Evidence of Advocacy for newborn during the launch of the newborn strategy by Minister

of Health

Furthermore, the Newborn Care (NBC) project – through its advocacy efforts – influenced the

implementation of the newborn care strategy and the insertion of NBC indicators into the DHIMS2.

Training packages, guidelines and protocols were developed to help implement the NBC strategy.

There was an advocacy effort to keep newborn care upfront on the agenda of a number of health

summits and various in-country regional and national meetings. One national key informant clearly

stated, in the quote below, the important role UNICEF played in the advocacy campaign:

“I think in 2012, there was enlightened awareness and advocacy for newborn because we

have not been able to significantly reduce newborn mortality. There was awareness raising

and advocacy throughout the country. A national conference was held in 2012. In all the

regions various stakeholders, regional directors and public health workers met and

developed plans to implement newborn care strategy. Newborn care was on the agenda of

a various meetings, WHO and UNICEF regional meetings and meeting in West Africa.

Here UNICEF is doing a lot to try to keep newborn as part of the major health activities

in the country”.

(National key informant)

Page 48: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

48

Another key informant reported that there were lots of broad consultations before the project took off

and also while the project was running. Following broad consultations, the government was at the

table with other stakeholders to discuss issues of newborn care. The project succeeded in putting the

government in the driving seat for newborn care. The advocacy efforts of UNICEF brought together

many national stakeholders to participate in a meeting in Senegal where participating countries

reviewed the draft global newborn action plan as illustrated in the quote below:

Going down to the project districts, metallic signboards were erected at facilities and

Government of Japan stickers affixed on equipment and supplies as well as doorways to NCU

(See Photo 5).

Opinions sought from key national decision-makers showed 11 out of 14 agreed or fully agreed that

the project gave due importance on the government of Japan’s visibility (Figure 4.14) but three (3) of

them gave no opinion. They indicated that they have seen signboards of the Government of Japan at

NCU.

“I just remember a meeting we attended in Senegal concerning the newborn where

UNICEF came back to the Ghana newborn working group. From there everything was

about reviewing a global action plan for newborn which other countries were to adopt.

Working together stakeholders formed working groups of all players in the newborn and

child health space and I would say UNICEF played a very significant role in making it

happen”.

(National key informant)

Page 49: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

49

Photo 5: Evidence of visibility of Government of Japan’s contribution

Objective 4: To draw lessons on the implementation capacity for national scale-up of the essential

newborn care model through home-based early postnatal care.

The Newborn Sub-Committee coordinates newborn care activities of all partners involved in

implementing newborn interventions. Each region and district has a focal person for newborn

care. The training modules have been adopted and are being rollout to other regions and the

home visits are being integrated into the CHPS structure.

Page 50: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

50

A national newborn care coordinator was appointed at GHS. In each of the two regions where the

newborn care project is implemented, a regional newborn care coordinator and a district newborn

care coordinator were also appointed for each district. This arrangement enhanced linkages between

national, regional and local level professionals and facilitated knowledge and experience sharing

about the project. At the national level there is a Newborn Sub-Committee chaired by the National

Child health coordinator of GHS, who ensures coordination of the newborn care activities across all

partners involved in implementing the newborn strategy. The development and launch of the national

strategy for newborn care was another indication of ownership and leadership in promoting newborn

care. The GHS led in the development of the newborn care training modules in collaboration with

UNICEF and other development partners. These are national documents intended for rollout to other

parts of the country

Eleven (11) out of 14 national decision-makers agreed or fully agreed that Policy makers at MoH/GHS

demonstrated ownership over the different interventions related to newborn survival (Figure 4.9).

However three (3) of them gave no opinion.

Key national decision-makers expressed various opinions demonstrating their commitment to and

ownership of newborn survival interventions. As part of the advocacy for NBC, a scorecard for

reproductive maternal and newborn health was developed as requested by the African Heads of State

with a few newborn and maternal health indicators for monitoring purposes and more commitment to

newborn health. This also came out of one of the national key informants expressed in the quote

below.

Page 51: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

51

Newborn health, and especially neonatal mortality should be considered in its proper perspective.

That is, as something which is beyond the health sector because other factors such as roads and

transport, finance, local government budgeting and commitment also affect it. Yet every time, as is

the case with most health issues, the health sector alone bears the entire burden. It is necessary for

GHS to take it up and make it a multi-sectorial issue as is now the case for HIV/AIDS. In support to

such an idea, a key national informant had this to say:

The views of key informants were that it was important to move things to a stage where newborn care

is not taken as something new or exceptional (a UNICEF or Japan’s initiative), but as a routine

standard of care. It is crucial for the government to ensure that all the health training schools or

institutions which are currently producing nurses, midwives, doctors and community health nurses,

future providers receive basic training in newborn care. When these professionals come out of school,

they should have the basic knowledge and skills needed to sustain the newborn care program. This

could ensure ownership and sustainability. On the issue of ownership by the government, a national

key informant said:

“We have developed a scorecard for reproductive maternal and newborn health,

which was a requirement from 49 Heads of State of Africa. In that scorecard, we

have a few newborn and maternal health indicators and it is our hope that our

President and Ministers will own this scorecard, as we heard it is happening in

other countries. Because as they look at it, it becomes clear to them the

bottlenecks and hopefully that will lead to more commitment to newborn care”.

(National key informant)

“I had the opportunity of making a presentation to Members of Parliament and I

showed the other sector relevant issues for newborn care, even the negative cultural

practices causing neonatal and child deaths. For the first time many of them

acknowledged that they had never thought about that before and promised to

contribute in resolving the issues”.

(National key informant)

“As it is the case for newborn care, if we were to have behavioral change we need a

sustained effort at communication and at advocacy and that has been one area that

government has committed very little funds to and whatever we get from partners is

not enough. Newborn care has a lot to do with practices at home and community

practices, the traditional beliefs and practices that need to be changed and government

seems to be ready to finance it.”

(National key informant)

Page 52: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

52

The DHMT/DA is supportive to newborn care but is yet to demonstrate ownership to

consolidate the achievements and the expansion of the newborn health interventions within

available or mobilized resources for the district. Its resources are not sufficient to be able to self-

support the needs for a long-term sustained newborn care intervention. Challenges remain for

adequate resource mobilization for such ownership.

Most (17 out of 27) of the sub-national authorities interviewed agreed or fully agreed that the

DHMT/DAs demonstrated ownership to consolidate the achievements and the expansion of the

newborn health interventions (Table 4.11) They however, indicated that the DHMT currently does not

have the financial capacity for scale-up if funding wanes. There is also the need to design a

performance-based incentive package for community-based volunteers for the sustainability of the

home-based care. Ten (10) of them gave no opinion or disagreed.

Key informant in the Upper East region reported that in terms of demonstrating ownership, the

District Assembly (DA) is doing many things with their own limited resources to improve newborn

care. For instance, they are constructing CHPS compound, supporting the training of some senior

staff members to expand access and coverage of health interventions. This is what appears in the

quote below from a key informant:

“District Assembly is expected to mobilize internally generated funds to support health

activities. But the people are so poor; they cannot generate anything, Market resources’ are

so negligible and therefore a really difficult situation for them. But they all show commitment,

because when you look in their development plan, health is one of their major concerns in

terms of building infrastructure. Some Assemblies are supporting training for medical Doctors

for maternal and child health including the newborn”.

(Key informant in the Upper East region)

Page 53: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

53

In the Northern region, especially in Savelugu, the maternity ward was newly built in 2013 by support

of the Government of Ghana and the project provided NCU equipment. The Tamale municipal

assembly is financing some renovations for its health center.

Some key informants strongly believe that even if UNCEF is no longer financing newborn care

activities in the Upper East region, the activities will continue because the skills are there and people

will continue to work, the only problem is the staff movement and the need to build capacity of new

staffs with more training sessions and more NCU equipment. One of them said, “We just need to

move to routine services that will require limited resources to ensure community-based newborn care

services by volunteers with bicycles”. For sustainability one key informant has proposed:

In Northern region the proposed solution by key informants was to build extra capacity in those areas

where providers have not been able to reach. This would mean training hundred volunteers or more,

and possibly bearing the cost of an extra two hundred in anticipation of ownership and sustainability.

This is indicated in the quote below:

However, there are challenges in terms of available resources to finance home visits, maintenance of

the NCU and staff retention. There is an urgent need for resolving problems that prevent adequate

functioning of the NCU (oxygen cylinder, baby warmer, fluids etc.), and effective regular home visits

in some CHPS zones or health centers (motorbike and its maintenance). Staff shortage and attrition

is also a challenge to ensure smooth running of the facilities.

Government resources to the sector are limited and basically for the regions, where the health services

depend on the internally generated funds mainly from the national health insurance scheme.

Unfortunately, the delay in payment by the insurance scheme is further threatening service provision.

In some instances it was difficult to finance home visits due to the lack of fuel for the motorbikes.

One key informant presented the situation as follows:

“Services would be a routine that will not require extra resources. It will be good to have

civil society and goodwill person support the program and provide some bicycles or fuel

just for the volunteers to move within the community and see the newborn and the mother.

GHS will possibility give soaps to volunteers as motivation. This is what we wish could be

done for sustainability, but we are not there yet”.

(Key informant in the Upper East region)

“Once all staffs are trained and we use the local engineer who is helping us very well, I

think we will be able to own this newborn care program and we think we can do

everything possible to sustain such project”.

(Key informant in the Northern region)

Page 54: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

54

Hence, the main challenge is the availability of motorbikes to visit mothers in the community. For

some time now, most of the motorbikes supplied to CHO/CHN for outreach and home visits are not

in working condition. These motorbikes were supplied by the government and UNICEF some time

ago and they are now broken down. It is expected that UNICEF or other donors will step forward to

improve the situation.

Service providers did admit that despite the success of the project there exist some bottlenecks, which

need to be addressed. These include: infrequent home visits by nurses due to lack of motorbikes or

breakdown of their own motorbike which they use as a substitute, insufficient personnel due to staff

attrition or trained nurses going back to school for further training, lack of means of transportation

for supportive supervision or performance monitoring, and volunteer fatigue due to the absence of

incentives.

This is what participants in one of the FGDs suggested addressing these bottlenecks:

The enhanced commitment and motivation of CHOs can last for a continued provision of home-

based services if the high morale and enthusiasm of applying new skills and supportive

supervision can be maintained, and if the challenges around fuel and motorbike supply are

resolved.

When opinions of service providers in selected health centres and CHPS compounds whose staff were

trained in newborn care were sought, 33 out of 35 agreed or fully agreed that they remain committed

to the home-based services to newborns (Figure 4.12).

“From the government, we expect staff to visit at least seven homes in the day to be able to see

all newborns and their mothers. But when the fuel is not there, or the motorbike broke down,

we are not able to do this. Some of us have been using our own motorbikes for home visiting,

but when the fuel is not coming regularly then it becomes another issue”.

(Key informant in the Upper East region)

“Provide support to our facility to facilitate referral to hospital, which is far away.

Expand the facility for more space and recruit more nurses and volunteers for home

visits. Provide motorbikes for nurses to conduct more home visits. Give incentives to

volunteers (rain coats and robber boots) to allow them to do home visits during rainy

season and visit the far to reach areas.”

(FGD participant, Upper East region)

Page 55: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

55

A review of some district annual reports, for example Bole District, showed that trained staff attrition

has been high especially among the CHOs/CHNs (Table 4.5) who are the key drivers of the home-

based newborn care. These staff that had training in newborn care have either been reposted to other

districts or gone on study leave. Until the district managers commit to institutionalize periodic training

in newborn care and pre-service is strengthened for all CHNs and CHOs (new and old) this situation

has a potential threat to the sustainability of the home-based services.

Table 4.5: Number of trained Health Professionals at post, Bole district

Category of Staff # Trained Attrition % attrition

Nurse Manager 1 0 0.0

Medical Assistants 5 1 20.0

Midwives 7 2 28.6

Staff Nurses 1 0 0.0

CHOs/CHNs 48 25 52.1

Enrolled Nurses 10 0 0.0

Total 72 28 38.9

Source: Bole district annual report, 2014

Page 56: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

56

The good news is that those currently posted exhibited high morale and the enthusiasm with regards

to the application of new skills, supportive supervision and better relationships with mothers who

showed gratitude for saving their babies. However in the long run, motivation might deteriorate

especially if basic needs such as the supply of basic newborn care equipment, the renovation of the

facilities to allow more space for service delivery, and the introduction of financial and nonfinancial

motivation schemes (especially for midwives, CHOs/CHNs) are not provided.

Providers demonstrated high motivation and commitment to conduct home visits to mothers. In the

Upper East region, they reported that there was at times no fuel for home visits or there was a delay

in obtaining it. The motorbikes provided by the Ghana Health Service were no longer functional and

had not been replaced. This posed a challenge for regular home visits. In addition some providers face

language barriers while communicating with mothers and had to use unpaid volunteers as interpreters.

They also had no raincoats, an item which could facilitate home visits during the rainy season. The

available space for providing facility care was very small and the same room was used for delivery,

counseling and hospitalization of mothers who had just delivered babies. All these provided highly

demotivating conditions for the providers to carry out their work and home visits. The good news is

that when asked, they indicated that they remained motivated for their work. The quote below is an

evidence of such motivation and commitment.

Providers also reported that the type of bag provided by UNICEF is difficult to handle and manipulate

in the field. A bag that could be hung on the back would be more comfortable. Some major equipment

for newborn care such as oxygen cylinders were lacking everywhere. Some innovation might be

needed to ensure that the facility delivery trend is maintained.

The behavioral changes among beneficiaries on essential newborn care can be sustained if the

newborn care projects approach to influence beneficiary behavior through home-based

education by nurses and volunteers and communication for development activities are

maintained and sustained.

The newborn care project’s approach to influence beneficiary behavior through home-based

education by nurses and volunteers proved to be effective in delivering correct information

concerning when and where to go when a newborn or a child is sick or has danger sign. The mothers

in the FGD indicated that they have found the information on caring for their babies very useful and

that they will continue to practice it. Therefore, efforts in sustaining the gains in communication for

development activities are critical for the mothers to maintain the practice.

“We have only one motorbike for the facility to conduct outreach for immunization and home

visits. I do not have a personal motorbike. I take taxi for home visits and taxi people charge me

a lot. But, I like the work and try to do home visits whenever possible. Because of high staff

attrition after their training in newborn care, we have no midwife and even though I am a nurse,

I do the deliveries myself, antenatal and postnatal care, and home visits”.

(Provider in the Northern region)

Page 57: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

57

The FGD with beneficiaries revealed that the right information might not reach the right person.

Culturally, the family structure foresees a strong role for mothers-in-law as main family decision

makers on issues pertaining to child bearing, feeding and treatment. However, mothers-in-law are

usually not direct targets of communication and awareness intervention. The findings of the FGDs

suggested that direct targeting of this group would be beneficial for newborn care.

In the Northern region, beneficiaries reported to be very happy about the work of nurses and

volunteers and it should continue. They acknowledge that some of the negative behavior in the past

has changed. Mothers reported during FGDs that today, hand-washing for the kids before eating and

after using toilet is in practice and personal and environment hygiene practices are duly observed.

They also reported that in the past, there were many cord infections due to cultural practices such as

putting various products (local herb or traditional medicine) on the cord. This behaviour has changed

with the teaching mothers, in-laws and husbands how to care for the cord. Mothers reported to be

happy with the changes and expressed their wish for the program to continue.

It should be noted that as reported by one beneficiary, some mothers are not available to meet with

nurses and volunteers when they visit due to their economic activities or other obligations that enable

them to generate money to pay school fees for their children. These women see the visits from the

nurses and volunteers as a burden, especially when no prior appointment has been arranged.

Thankfully this is a rare occurrence. One beneficiary said:

In the Upper East, beneficiaries reported that their babies are healthier and that they now practice

exclusive breastfeeding, maintain a clean environment and observe sound hygiene practices. They

reported cultural practices were not easy to change due to their mothers-in-law who needed to approve

the new practices that were taught before they could be implemented. Thanks to their persistent visits,

volunteers and nurses were able to convince mothers-in-law on the mothers’ behalves to allow the

implementation of these new practices. This is illustrated in the quote below.

“Now the community better understands the care of the cord. Just to clean it and do not

put anything on the cord. Taking care of the newborn is a collaborative responsibility which

brings husband and mother in-laws to be supportive to the wife”.

(FGD participant, Upper East region)

“This behaviour has changed with the teaching of how to care for the cord to mothers,

in-laws and husband. Mothers reported to be happy with the changes and would like

the program to continue”.

(FGD participant, Northern region)

Page 58: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

58

Beneficiaries in the Upper East region reported behavioral changes that can be sustained. They believe

the project is helping them have healthy babies. Thanks to the project, children are better and mothers

now have time to conduct the economic activities required to support the family instead of worrying

when their babies are sick. According to them, the community now better understands how to care

for the cord. Education on the use of mosquito net has also helped reduce malaria cases.

The project facilitated synergies and avoided duplications of interventions and strategies

promoted by other UN agencies and development partners through partner forums, national

newborn sub-committee and MAF implementing committee meetings.

In countries where there are multiple partners it is important to create such fora to avoid the

duplication of projects in one district and to leverage others’ resources for optimum results or

synergies.

In-depth interview with UN agencies, development partners and national decision-makers revealed

that partners do have a forum where they meet quarterly to share information on on-going and planned

projects. The National Newborn Sub-committee also provides the platform for health partners to

discuss and share further information on newborn activity. There is also a MAF implementation

committee that meets quarterly to discuss progress and the current commitment of partners.

When opinions were sought, 5 out of 7 national decision-makers and partners agreed or fully agreed

that the project facilitated synergies and avoided duplications (Figure 4.13) but one of them disagreed

and another gave no opinion. The questionnaire, however, did not ask for reason for such

disagreement.

“When the visitors came they met my mother in law and educated her on exclusive

breast feeding, care of the cord, danger signs, child welfare and the importance of

immunization, eye diseases and skin diseases and that both mothers and baby should

sleep under mosquito net. They understood and complied with it and we can see the

changes”.

(FGD participant, Upper East region)

Page 59: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

59

UNICEF, with funding from the Japanese government, started the NBC project in the Northern and

Upper East regions. Today, JICA is implementing the NBC in the Upper West. The USAID-funded

University Research Corporation (URC) for Systems for Health Project is to be implemented in 5

other regions namely: the Volta, Northern, Central, Western and Greater Accra regions. PATH is also

implemented in the following 4 regions: Brong Ahafo, Ashanti, Eastern and Volta regions. This

coordination has helped ensure synergy and avoided duplication.

Ghana Health Service now has a newborn care secretariat to coordinate the activities of its various

health partners’. The secretariat will streamline the partners’ requests and make sure that there is

minimal duplication in the same area for NBC. However, partners sometimes have their own plan

which governs their funding allocations – thus forcing beneficiaries to either take it or leave it. It is

very difficult to decline a partner’s money. This is a challenge and the secretariat often needs to further

discuss issues and come to an agreement with the partner in question. One way of handling such

situation has been proposed by a key informant.

The project incorporated the HRBA to programming in accordance with UNICEF’s global

mission. Children’s rights were paid attention to in the project’s design – especially the right of

newborns to survive and have a good quality of life.

The general view of the national key informants was that the project considered in its design “the

Convention on the Rights of the Child” which forms part of UNICEF’s global mission.

Nonetheless, the rights of the child should be reflective of the child’s life beyond the mere

considerations implemented during the project’s design. Key informants reported that newborns’

rights go as far as giving maternity leave to working mothers in order to afford her time to breastfeed

the baby, and offering her a space at work so she can continue breastfeeding the baby. Below is a

quote:

:

“We need a good coordinating secretariat, because just like HIV coordination bodies. In

some countries, you have partners from different institutions treating HIV patients

differently, but in Ghana that does not happen. With the NACP and the Ghana Aids

Commission you cannot just come in and start anything anyhow. Ghana Health Service

should have a strong coordination council for newborn health”.

(National key informant)

“The newborn right goes beyond the newborn. It is a big issue especially with working mothers

who are breastfeeding and the duration of maternity leave. The work places should have space

that could allow mothers to breastfeed their babies on demand. It is a very big human right for

the baby because we are denying the baby to receive mother’s care”.

(National key informant)

Page 60: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

60

The use of community structures such as community-based volunteers as well as the home visits

by CHOs/CHNs has helped to bring MNCH services to the doorstep of mothers and babies.

Twelve (12) out of 14 national decision-makers/partners expressed the view that the project, by

training staff on newborn care and equipping all health centres and district hospitals with newborn

care equipment has removed geographical barriers that limited access to newborn care. The home-

based care and the community mobilization components has also been recognized to facilitate the

reduction of access barriers to newborn services at the community level (Figure 4.17). Two

participants, however, expressed no opinions.

A national key informant has a solution, we quote:

“Because of bad roads, long distance, difficult to reach areas and insufficient human resources,

ensuring access to MNCH across the country is a big challenge. Possible solution is that District

Assembly or partners could sponsor training of CHO/CHN or midwives so that they will remain

within the district after training for 5 years under an agreement to provide health services and

conduct home visits to mothers. Donors could also help construct waiting homes or rooms for high

risk pregnancies so that during the last month the woman can stay there, close to a health facility

for delivery”.

(National key informant)

.

Page 61: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

61

5. Lessons learned

From this evaluation we can derive the following lessons that could be used in future design,

implementation of the next phase of the project and other relevant interventions. These lessons will

help to be more effective in implementing other newborn projects.

1. Using the community structures (CHOs/CHNs, community volunteers, mother support groups)

improved timely home-based care and helped foster provider-community partnership.

2. Immediate follow up has helped to avert the newborns from negative cultural practices (such as

applying herbs to the cord which has potential for infection)

3. Fathers’ involvement in the home-based visits had contributed to improve acceptance of

practices on care of the newborn.

4. Periodic training of Midwives and CHOs improved their knowledge and skills and gave them

confidence to deliver quality care.

5. The establishment of the newborn care units (NCU) created demand for newborn care and saved

the lives of many preterm and low birth-weight babies.

6. Conclusions and Recommendations

6.1 Conclusions

The home-based postnatal newborn care and neonatal intensive care models have been effective in

contributing to improved newborn survival in the two project regions of Upper East and Northern

regions of Ghana, to the extent possible given the scope and reach of the intervention. The enhanced

capacity of NCUs, with essential newborn care equipment as well as the enhanced capacity of health

personnel in terms of skills acquired for management of sick and preterm babies, have been important

enabling factors for saving the lives of many babies in the project districts. The evidence-based

advocacy efforts at all levels have contributed significantly to making newborn issues a national

priority especially culminating into the development and launch of a National Newborn Strategy and

Action Plan.

Page 62: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

62

6.2 Recommendations

The challenge for improving newborn health lies in ending preventable newborn deaths and securing

Ghana’s future. Success will be measured in terms of lives saved and lives improved. Success will

depend on meeting the needs of women and their babies throughout the continuum of care and

committing to the following action items:

National Level:

1. The Government should commit enough resources to operationalize the National Newborn

Strategy and Action Plan. The Newborn strategy could be used as a framework for donor

support. Donor assistance should be mapped onto strategy priorities and Donor projects have

to be coordinated to achieve strategy objectives. A system should be in place for effective

monitoring and assessment of achievements and resource management tools should be in

place to ensure accountability. A national budget line for newborn activities needs be

envisioned. An advocacy group may be put in place to ensure continued resource mobilization

for Newborn Strategy implementation.

2. The MoH/GHS should ensure that every district hospital has Newborn Care Unit for

secondary level 2 care. All regional and teaching hospitals should have Newborn Intensive

care Units (NICU) for tertiary level 3 care. Health Centers and Polyclinics where delivery is

conducted should have provision of Basic Emergency Obstetric and Newborn Care

(BEmONC) including Newborn Care Corner (NCC).

3. The GHS/MoH should establish resource centres in Regional and Teaching Hospitals along

with NICU, to the extent possible using existing structures, to facilitate on job training on

newborn care. Staff from the resource centres should deliver a transferable skills program

through mentorship and periodic specialists’ visits to lower level facilities.

4. The GHS should scale-up the home-based postnatal newborn care model to all districts in the

regions and to other regions. The evaluators do not anticipate any delay if national and sub-

national decision-makers commit to this course. The materials have already been developed

and lessons learned can speed up nationwide scale-up. GHS could take advantage of the fact

that in all ten regions there is ongoing newborn activities in some of the districts supported by

various donors such as UNICEF (2 regions), USAID HSS (5 regions), USAID JHPIEGO (4

regions), PATH (4 regions), JICA (1 regions) and KOICA (1 region). This will require

coordination and harmonization on a minimum package of effective newborn care

interventions for the reduction of newborn mortality.

5. The MoH should review curriculum of the Midwifery and Community Health Training

Schools to include issues on newborn care or update and strengthen any existing such training

program using the newborn care training modules, which were developed in collaboration

with UNICEF and other development partners and which have now been accepted as national

documents intended for nationwide roll-out. The MoH and GHS should collaborate to

formulate and approve detailed implementation plan and budget for integration of the newborn

care training package into the pre-service, postgraduate and continuous education systems.

The in-service training should also be reinforced for those already in the field.

Page 63: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

63

Sub-national (region and district) level:

6. The District Directors should collaborate with the District Assemblies (DA) to ensure that

newborn care issues become a standard agenda on district quarterly review meetings. This

implies advocacy work using neonatal mortality data from the district statistics as to inform

the DA on the urgency of mobilizing funds to address newborn care issues as a priority in the

district. The DA should have a local budget line for newborn care as a sustainable financing

solution for both maternal and newborn care services within the district. This will help the

district address a number of challenges related to newborn care activities such as incentive

and motivation for volunteers and CHO/CHN, fuel and maintenance for motorbikes used for

home visits, bicycles for volunteers. The financial contribution of the district to newborn care

activities will encourage MoH/GHS to supplement with recruitment of additional Human

resources in shortage (nurses, midwives and pediatrician) and provide needed equipment such

as new motorbikes for facilities and materials for NCU.

7. Regions and districts could have in place a community performance-based financing scheme

with performance reward approach where community volunteers, providers and sub-district

team will be given money for anticipated performance. An agreement will be established with

each district and group of providers to implement newborn care activities with well-

established results and coverage for which funds will be provided and financial incentives

provided based on performance and percentage of expected results achieved. This will be an

option for better coverage and achievements with newborn care.

8. Institutionalization of perinatal death audit and newborn death audit would be fundamental to

ensure increased attention to newborn care and the causes and circumstances of neonatal death

in order to address them more effectively and reduce neonatal mortality. It will also help avoid

neonatal deaths due to poor performance or mistakes or inappropriate action of the providers.

To prevent those unnecessary neonatal deaths, Regional Health Management Teams, Hospitals

and the MoH/GHS should integrate newborn care indicators to the existing M&E system to

monitor performance, progress, facility neonatal deaths and achievements in newborn care by

providers and volunteers. This could be reinforced with provision of newborn care registers

for hospital, facilities and community visits. Regions and district health managers should be

encouraged to use effectively these newborn indicators and newborn death audit results in

planning and implementation of health service decisions as well as in assessing staff

performance.

9. Regional and district leadership should be strengthened to drive newborn agenda and provide

support for implementation. The District Directors should be tasked to develop a

comprehensive plan with costing for capacity building and refresher training schedule for staff

involved in newborn care. They must maintain register of staff and track staff movement to

manage any capacity gap arising in order to reduce staff attrition. Further steps should be taken

to provide the necessary conditions to retain trained service providers at the deprived

communities. Part of the available resources should be used for incentive and motivation of

volunteers and CHO/CHN to do more home visits. Regional and District Leadership should

reinforce the home-based postnatal care as part of routine activity and demand accountability

Page 64: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

64

from the CHOs/ CHNs by periodically assessing their home visit register. Number of

newborns visited at home at day 3 and day 7 within 0-7 days following birth should be

included in the performance appraisal of the CHOs and CHNs.

Cross Sectoral Support:

10. Quality Assurance (QA), Quality Improvement (QI) and access to quality newborn care

services is important for attainment of the MDGs. MoH/GHS is already putting in place

QA/QI system for health service delivery. This should be extended to newborn care services

at all levels including facility level with providers and community level with volunteers. There

should be an external and internal newborn care quality audit system. Proposed system will

improve evidence-based managerial decision-making at facility and local levels and will

increase the competition and staff motivation to better perform. It should provide needed

supportive supervision and mentorship to improve quality of the newborn care services

offered by the health providers.

11. National and subnational levels should support effective documentation, communication and

advocacy activities for newborn care. They should develop a communication strategy for

newborn care and its implementation plan with costing. Further steps should be taken to

document newborn stories in the field (what is happening? what is new?) then record them

and use media to disseminate them. With support from local community members they should

document bad perceptions towards newborns especially neonates, negative sociocultural

practices and address them with behavioral communication, education of mothers, husbands,

in-laws and families during home visits and social/community mobilization. They should

intensify health education involving community members, opinion leaders, traditional and

religious leaders to recognize the importance of the care of the newborn and the mothers for

their greater survival. They should consolidate the gains in C4D activities on newborn care

using community volunteers. This will require the involvement of the District Assemblies for

a small motivation of the volunteers actively involved in C4D to ensure that they could

continue the home-based newborn care activities with mothers, community participation and

demand generation for newborn care.

12. National and subnational levels should support secondary data analysis to identify barriers to

newborn care and address them and use operation research results for more evidence on the

table to support funds mobilization and advocacy for newborn care. Steps should be taken to

address the issue of gender and ensure greater male involvement. Human rights based

approach and equity should be part of the sub-national implementation of newborn care

activities. They should put in place a system that will contribute to sustainability of newborn

care activities and use quality improvement method at facility level with rewards to regions,

districts and selected providers that are improving in newborn care.

Page 65: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

65

References

1. UN Inter-agency Group for Child Mortality Estimation I: Levels and trends in child mortality:

Report 2013. New York: UNICEF, 2013. (http://www.childinfo.org/%1Fles/Child_Mortality_Report_2013.pdf, accessed 12 June 2014).

2. Darmstadt GL, Kinney MV, Chopra M, et al. for The Lancet Every Newborn Study Group,. : Who has been caring for the baby? . Lancet 2014, published online 19 May 2014.

http://dx.doi.org/10.1016/S0140-6736(14)60458-X. 3. Ghana Statistical Service"GSS", 2011: Ghana Multiple Indicator Cluster Survey with an

Enhanced Malaria Module and Biomarker Final Report. Accra, Ghana, 2011.

4. Ministry of Health/Ghana Health Service GoG: National Assessment for Emergency Obstetric and Newborn Care. . Accra, Ghana 2011.

5. UNICEF, WHO, 2014 : Every Newborn: An Action Plan To End Preventable Deaths; 30 JUNE 2014. Johannesburg, South Africa.

6. Ministry of Health, Government of Ghana, United Nations Country Team in the Republic of Ghana,

2011: Ghana MDG Acceleration Framework And Country Action Plan Maternal Health. Jul 1, 2011

Page 66: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

66

Appendices

AP1. Informed Consent Form

Formative evaluation of the project entitled “Accelerating efforts to reduce

maternal, neonatal and child mortality in the Northern and Upper East regions”

INFORMED CONSENT FORM

Thank you for agreeing to participate in this Evaluation.

The purpose of this formative evaluation is to understand whether the intended objectives of the

newborn care project implemented in Northern and Upper East regions of Ghana have been achieved.

Specifically, the evaluation will determine to what extent the intervention has been able to meet its

objective to create capacity, tools and structures to respond to the high levels of newborn morbidity

and mortality in the two-targeted regions.

The project partners– MoH and GHS, UNICEF, and JICA – will use the findings of the evaluation in

their different capacities and functions, to develop future plans and interventions and to inform

policies and strategies.

The methods that will be used to meet this purpose include face-to-face semi-structured interview and

focus group discussions with selected key informants.

You are encouraged to ask questions or raise concerns at any time about the nature of the evaluation

or the methods to be used.

Please contact me at any time at the e-mail address ([email protected]) or telephone number (+233-

244-280-495).

Our discussion will be audio taped to help me accurately capture your insights in your own words.

The tapes will only be heard by me for the purpose of this evaluation. If you feel uncomfortable with

the recorder, you may ask that it be turned off at any time.

You also have the right to withdraw from the evaluation at any time. In the event you choose to

withdraw from the evaluation all information you provide (including tapes) will be destroyed and

omitted from the final paper.

Insights gathered by you and other participants will be used in writing an evaluation report, which

will be disseminated. Though direct quotes from you may be used in the paper, your name and other

identifying information will be kept anonymous.

By signing this consent form I certify that I ___________________________ was informed and has

agree to participate in this evaluation (Print full name here).

____________________________ ______________

(Signature) (Date)

Page 67: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

67

AP2. Evaluation tools (In-depth interview guides, Short questionnaires, FGD guide)

AP2a0: General Background Data

1. Introduction to the objectives of the evaluation

To ascertain the effectiveness of the package of evidence-based maternal, neonatal and

child survival interventions with a focus on community-based (Home based Postnatal Care)

and facility based (IMNCH) approaches in the selected districts of the Northern and Upper

East Region;

To ascertain the project’s contribution for the capacity building, developing training

resources and facility structures to respond to the high levels of newborn morbidity and

mortality in selected districts of the Northern and Upper East Region;

To ascertain the effectiveness of the evidence-based advocacy of the project on the national

policy environment on the issues related to newborn survival;

To draw lessons on the implementation capacity for national scale-up of the essential

newborn care model through home-based early postnatal care;

2. General information

a. First Name and Surname: ________________________________________

b. Title and post/position: ________________________________________

c. Institution /Organization _+_____________________________________

d. Government _____Civil society____ Donor ____ Partner _____Other _____

e. Sex F _______ M _________

f. Age_______________________________________

g. Place of interview: ____________________________________________

h. Date of interview: (DD_______ MM _________ YYYY___2015____

Page 68: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

68

i. Time of interview: Beginning________________ End _____________

j. Interview order number____________________________________

AP2a1: Interview Guide for Key National Decision-Makers/Stakeholders

1. Introduction to the objectives of the evaluation

To ascertain the effectiveness of the package of evidence-based maternal, neonatal and child

survival interventions with a focus on community-based (Home based Postnatal Care) and

facility based (IMNCH) approaches in the selected districts of the Northern and Upper East

Region;

To ascertain the project’s contribution for the capacity building, developing training

resources and facility structures to respond to the high levels of newborn morbidity and

mortality in selected districts of the Northern and Upper East Region;

To ascertain the effectiveness of the evidence-based advocacy of the project on the national

policy environment on the issues related to newborn survival;

To draw lessons on the implementation capacity for national scale-up of the essential

newborn care model through home-based early postnatal care;

2. A brief introduction to the semi-structured interview

Be assured that this interview is confidential and what you say will be used only for the

purposes of this evaluation but anonymously;

Please do not hesitate if you wish withdraw at any time

3. General information (See separate page)

4. Interview order number_________________________________

5. Introduction to the topic under review:

We are here to evaluate the newborn care project of GHS / UNICEF / Government of Japan

in Northern and Upper East regions of Ghana

6. Permission to proceed and to record the discussion

7. Questions:

Page 69: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

69

Relevance:

R1: Is the intervention relevant in terms of alignment of project objectives with national strategy and

stakeholder’s priority and needs?

Effectiveness:

E1: To what extent has the project contributed to the policy direction of the National Newborn

Strategy and Action Plan?

Efficiency:

Eff1: Were the allocated resources used efficiently to achieve the project objectives? Are the available

resources adequate to meet project needs?

Sustainability:

S1: To what extent does policy makers at MoH/GHS demonstrated ownership over the different

interventions related to newborn survival?

S2: To what extent has the Government of Ghana prioritized the health and wellbeing of newborn

babies in the government’s policy documents (Ghana Shared Growth and Development Agenda for

2014-2017, Medium Term Health Expenditure Plan for 2014-2017) and allocation of resources

(budget line on approved Ministry of Health budget) for newborn health?

Coherence: C1: To what extent does the project facilitated synergies and avoided duplications with interventions

and strategies promoted by other UN agencies and development partners (JICA, USAID, EC, PATH

and others) within the National Child Health Policy 2007-2015 and MDG5 Acceleration Framework

(MAF)?

C2: To what extent has the project given due importance on donor’s (Government of Japan) visibility

in line with UNICEF’s donor visibility guidelines;

Human right based approach (HRBA):

H1: To what extent does the project incorporated the Human right-based approach to programming?

H2: To what extent does the project consider the equity approach (i.e. focus on most deprived areas,

areas with high prevalence of critical newborn and under-5 mortality, low income families) and

facilitate the reduction of access barriers to MNCH services by the target group?

8. Do you have any suggestion for improvement of newborn health care?

9. Ask if s(he) would like to add further comments.

10. Bring the meeting to a close by summarizing the main points.

11. Thank the key informant

12. Write the time

Page 70: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

70

AP2a2: Short Questionnaire for Key National Decision-Makers / Stakeholders

Introduction: We are conducting a formative evaluation of the above-mentioned project. We would

appreciate your contribution to this evaluation.

PART A: Please indicate to which group you belong:

/_ / Ministry of Health / Government of Ghana

/_ / Ghana Health Service

/_ / UNICEF or Other partner involved in project funding or implementation

/_ / Donor Partner

/_ / International Organization

/_ / NGO or Civil society

/_ / University or Academia

/_ / UNICEF staff

/_ / Other (please specify) ______________________________

PART B: Please indicate your degree of agreement or disagreement with each of the following

statements on a scale from 1 to 5

5 : Fully agree

4 : Agree

3 : No opinion

2 : Disagree

1 : Fully disagree

1) R1: The intervention was relevant in terms of alignment of project objectives with national strategy

and stakeholder’s priority and needs.

1 2 3 4 5

2) E1: The GHS/UNICEF/Government of Japan project contributed to the policy direction for the

national newborn health strategy and action plan

1 2 3 4 5

3) Eff1a: The allocated resources were used efficiently to achieve the project objectives.

1 2 3 4 5

4) Eff1b: The available resources were adequate to meet project needs.

1 2 3 4 5

Page 71: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

71

5) S1: The policy makers at MoH/GHS demonstrated ownership over the different interventions

related to newborn survival.

1 2 3 4 5

6) S2: The Government of Ghana prioritizes the health and wellbeing of newborn babies in the

government’s policy documents (Ghana Shared Growth and Development Agenda for 2014-2017,

Medium Term Health Expenditure Plan for 2014-2017) and allocation of resources (budget line on

approved Ministry of Health budget) for newborn health.

1 2 3 4 5

7) C1: The GHS/ UNICEF/Government of Japan project facilitated synergies and avoided

duplications with interventions and strategies promoted by other UN agencies and development

partners (JICA, USAID, EC, PATH and others) within the National Child Health Policy 2007-2015

and MDG5 Acceleration Framework (MAF).

1 2 3 4 5

8) C2: The GHS/UNICEF/Government of Japan project gave due importance on donor’s

(Government of Japan) visibility in line with UNICEF’s donor visibility guidelines.

1 2 3 4 5

9) H2b: The project facilitated the reduction of access barriers to MNCH services by the final

beneficiaries.

-

1 2 3 4 5

PART C: Please answer the following questions with your own opinion.

12) What are the two major contributions of the newborn care project by UNICEF to Ghana health

agenda?

a)__________________________________________________________

b) _________________________________________________________

13) In future how do you see the newborn project evolve? What does it take to get there?

_______________________________________________________________

_______________________________________________________________

_______________________________________________________________

Page 72: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

72

AP2b1: Interview Guide for Key Sub-National Decision-Makers

13. Introduction to the objectives of the evaluation

To ascertain the effectiveness of the package of evidence-based maternal, neonatal and child

survival interventions with a focus on community-based (Home based Postnatal Care) and

facility based (IMNCH) approaches in the selected districts of the Northern and Upper East

Region;

To ascertain the project’s contribution for the capacity building, developing training

resources and facility structures to respond to the high levels of newborn morbidity and

mortality in selected districts of the Northern and Upper East Region;

To ascertain the effectiveness of the evidence-based advocacy of the project on the national

policy environment on the issues related to newborn survival;

To draw lessons on the implementation capacity for national scale-up of the essential

newborn care model through home-based early postnatal care;

14. A brief introduction to the semi-structured interview

Be assured that this interview is confidential and what you say will be used only for the

purposes of this evaluation but anonymously;

Please do not hesitate if you wish withdraw at any time

15. General information (See separate page)

16. Interview order number_________________________________

17. Introduction to the topic under review:

We are here to evaluate the newborn care project of GHS / UNICEF / Government of Japan

in Northern and Upper East regions of Ghana

18. Permission to proceed and to record the discussion

19. Questions:

Effectiveness

E2: To what extent has the project contributed to strengthen capacity of regional health management

teams and district health management teams for planning, informed decision making and

prioritization of the newborn health as per the National Child Health Policy (2007-2015) and other

national guidelines and protocols?

Sustainability

S1: To what extent do RCC/GHS, District Assembly/DHMT demonstrate ownership and capacity

for resource mobilization to be able to self-support and consolidate the achievements and the

expansion of newborn health interventions?

Page 73: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

73

20. Do you have any suggestion for improvement of newborn health care?

21. Would you like to add further comments?

22. Bring the meeting to a close by summarizing the main points.

23. Thank the key informant

24. Write the time

AP2b2: Short Questionnaire for Key Sub-National Decision-Makers

Introduction: We are conducting a formative evaluation of the above mentioned project. We would

appreciate your contribution to this evaluation.

PART A: Please indicate to which group you belong to

/_ / Ghana Health Service Regional level

/_ / Ghana Health Service District level

/_ / UNICEF Other partner involved in project implementation

/_ / International organization

/_ / NGO or Civil society

/_ / UNICEF staff

/_ / Other (please specify) ______________________________

PART B: Please indicate your degree of agreement or disagreement with each of the following

statements on a scale from 1 to 5

5 : Fully agree

4 : Agree

3 : No opinion

2 : Disagree

1 : Fully disagree

1) E2: The GHS/UNICEF/Government of Japan project has contributed to strengthen capacity of

regional health management teams and district health management teams for planning, informed

decision making and prioritization of the newborn health as per the National Child Health Policy

(2007-2015) and other national guidelines and protocols.

1 2 3 4 5

2) S1: The DHMT and District Assembly demonstrated ownership and capacity for resource

mobilization to be able to self-support and consolidate the achievements and the expansion of

newborn health interventions.

1 2 3 4 5

Page 74: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

74

PART C: Please answer the following questions with your own opinion.

3) What are the two major contributions of the newborn care project by UNICEF to Ghana health

agenda?

a)__________________________________________________________

b) _________________________________________________________

4) In future how do you see the newborn project evolve? What does it take to get there?

_______________________________________________________________

_______________________________________________________________

_______________________________________________________________

AP 2c1: Interview Guide For Service Providers

(Obstetricians, Pediatricians, Neonatologists, Nurses, Midwives, Patronage Nurses, GPs)

25. Introduction to the objectives of the evaluation

To ascertain the effectiveness of the package of evidence-based maternal, neonatal and child

survival interventions with a focus on community-based (Home based Postnatal Care) and

facility based (IMNCH) approaches in the selected districts of the Northern and Upper East

Region;

To ascertain the project’s contribution for the capacity building, developing training

resources and facility structures to respond to the high levels of newborn morbidity and

mortality in selected districts of the Northern and Upper East Region;

To ascertain the effectiveness of the evidence-based advocacy of the project on the national

policy environment on the issues related to newborn survival;

To draw lessons on the implementation capacity for national scale-up of the essential

newborn care model through home-based early postnatal care;

Page 75: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

75

26. A brief introduction to the semi-structured interview

Be assured that this interview is confidential and what you say will be used only for the

purposes of this evaluation but anonymously;

Please do not hesitate if you wish withdraw at any time

27. General information (See separate page)

28. Interview order number_________________________________

29. Introduction to the topic under review:

We are here to evaluate the newborn care project of UNICEF / Government of Japan in

Northern and Upper East regions of Ghana

30. Permission to proceed and to record the discussion

31. Questions:

Which training package did you attend?

Effectiveness

32. Are training contents (including protocols and guidelines) suitable for the Ghanaian

Northern and Upper East regions newborn care delivery system? Why?

33. Was this training pertinent to your current daily work? Why?

34. Before attending the training, did you feel the need to upgrade your knowledge and skills?

Why? In which field/s?

35. Do patients appreciate the improvement in newborn care in your health facility? Why do

you say this?

36. Since you started applying the acquired skills, is there any noticeable improvement in

newborn care for the mothers who deliver in your health facility? Why do you say this?

37. Do you feel that the training enabled you to fully apply, in your daily practice, what you have

learnt? Why?

38. How often do you apply the acquired skills and knowledge into work practice?

39. Were you reluctant to accept new practices/procedures (reluctant to change)? Which ones?

Why?

40. Did the acquired knowledge and skills affect (could be both, positively and negatively) your

self-confidence and the value you put on your daily work? Why?

41. What is the significance, if any, of providing newborn care?

Page 76: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

76

42. Is there a facilitative supervision and monitoring system in place? Is this system able to

support you to apply acquired skills, and reliable information and data for decision makers?

Why? Please describe. What is your involvement in the monitoring process?

43. To what extent has there been an improvement in quality of care during delivery and post-

natal care in the health facilities targeted by the project?

44. To what extent was the training and mentorship component of the project responding to

capacity building needs of the different levels of service providers?

45. Is the training package relevant to your needs for better performance?

46. Can you describe the visit by a supervisor for newborn care practice, when, duration and

process of the supervision?

47. What kinds of topics were covered during the training sessions? What happened after the

training?

Sustainability

48. At the work place, are there some conditions that prevent you to correctly practice your skills?

(i.e. non-confident in skills despite training, shortage/lack of basic equipment/amenities,

drugs, time constraints, referral etc.). Please, describe.

49. Are you receiving any incentive/did you expect to be incentivized/awarded for delivering

quality MNCH services? Please, describe.

50. Do you have any suggestion for improvement of newborn health care?

51. Would you like to add further comments?

52. Bring the meeting to a close by summarizing the main points.

53. Thank the key informant

54. Write the time

Page 77: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

77

AP 2c2: Short Questionnaire For Service Providers

Introduction: We are conducting a formative evaluation of the above-mentioned project. We would

appreciate your contribution to this evaluation.

PART A: Please indicate to which group you belong

/_ / Ghana Health Services Regional level

/_ / Ghana Health Services District level

/_ / Health facility (Health centre/CHPS)

/_ / UNICEF or Other partner involved in project implementation

/_ / International organization

/_ / NGO or Civil society

/_ / Other (please specify) ______________________________

PART B: Please indicate your degree of agreement or disagreement with each of the following

statements on a scale from 1 to 5

5 : Fully agree

4 : Agree

3 : No opinion

2 : Disagree

1 : Fully disagree

1) E3a: The established level-two (without ventilator and incubators) newborn care units in six

District Hospitals have improved the management and survival of sick newborn babies.

1 2 3 4 5

2) E4: There has been an improvement in quality of care during delivery and post-natal care in the

health facilities targeted by the project.

1 2 3 4 5

3) E5: The training and mentorship component of the project responded to capacity building needs

of the different levels of service providers.

1 2 3 4 5

4) S4a: The enhanced commitment and motivation of CHO/CHN and community volunteers (CBAs)

will last, for a continued provision of home-based services to mothers and their newborn babies.

1 2 3 4 5

Page 78: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

78

PART C: Please answer the following questions with your own opinion.

5) What are the two major contributions of the newborn care project by UNICEF to Ghana health

agenda?

a)__________________________________________________________

b) _________________________________________________________

6) In future how do you see the newborn project evolve? What does it take to get there?

_______________________________________________________________

_______________________________________________________________

_______________________________________________________________

AP2d: FGD Guide For Beneficiaries

1. Introduction to the objectives of the research

2. A brief introduction to the rules of focus groups

a. Everything said and done is confidential and will not be used outside the room except for the

purposes of this evaluation;

b. Every statement is right;

c. Please do not hesitate to disagree with someone else;

d. Please do not all talk at once

3. Ask people to describe who they are and say few words about themselves

4. Introduce the topic under review - We are here to evaluate the Newborn care project and its home

based visits and get your opinions and appreciations

5. Ask for permission to proceed and to record the discussion

6. Ask questions

Effectiveness

Is the communication package relevant to your demands and needs?

Can you describe the visit by a nurse or a health agent, when, duration and process of the visit and

topics of counseling if you received it?

Page 79: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

79

What kind of topics did nurse or community agent covers during communication on children’s

care: provide information? or /and show examples or/and give you an example for practice?

What is the primary form of communication that makes you to understand the message?

a. Flip chart on key child feeding, caring and health seeking practices? b. mother

card?

c. posters?

d .examples?

e. practice?

f. Other, please specify?

How often do you apply acquired skills and knowledge into practice?

Did you/ or other family members receive adequate information on newborn or child care?

And who provided this information?

How long it takes the counseling on newborn or child care? Do you think that duration of

the counseling and the content is sufficient?

Did you receive any information or sensitization project communication or social

mobilization interventions, like community durbars, mother support groups, community

based agents and Red Cross mothers (in Upper East Region)?

Did you see an improvement in your newborn care and health seeking practices (ANC,

PNC, well baby clinic) as a consequence of improved counseling by CHO/CHN, CBA,

Red Cross mothers and mother support groups?

How much have the intervention contributed to improve health and wellbeing of newborn

babies and their mothers and in terms of changing health seeking behavior?

What kind of information do you want or need to receive as a mother/caregiver on

newborn or child care?

What are the ways you would like to receive that information?

What was the content and form of information you have received so far on newborn or

child care?

Was the content of materials easy to understand and practical?

Sustainability

What changes did you noticed in people’s behavior for essential newborn care? How do

you see these changes continue in future? What do you see as bottlenecks or barriers that

hinder the capacity of mothers and caregivers to access and use quality newborn care

services for them and their babies?

To what extent is the commitment and motivation of CHO/CHN and community

volunteers (CBAs) that was enhanced through the project perceived to last, for a continued

provision of home-based services to mothers and their newborn babies?

Page 80: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

80

7. Do you have any suggestion for improvement in newborn care?

8. Ask if they would like to add further comments.

9. Bring the meeting to a close by summarizing the main points.

10. Thank you!

Page 81: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

81

AP3. Evaluation Framework

Criteria Questions/Levels Indicators (where

appropriate)

Sources of data Methods

Relevance:

R1 R1: National decision-making

Is the intervention relevant in terms

of alignment of project objectives

with national strategy and

stakeholder’s priority and needs and

also in terms of advocating for and

facilitating to bring the newborn

health into the national agenda of

MNCH Programming?

Degree of congruence

between project strategy

and stakeholders priority

and needs

Documents: Government

of Ghana Newborn care

policy; National Child

Health Policy, Project

documents; UNICEF's

newborn care strategy

documents; Government of

Japan's MNCH strategy

documents

Key Informants:

GHS Family Health

Division; MoH/GHS

PPME, Donor Partners,

(See Appendix AP4)

Desk review

Semi-

Structured

Interviews/In-

depth

interviews

Effectiveness

E1 E1: National decision-making

To what extent has the project

contributed to the policy direction

of the National Newborn Strategy

and Action Plan in terms of

advocating for and facilitating to

bring the newborn health into the

national agenda of MNCH

Programming?

Advocacy campaign;

National newborn strategy

and Action Plan

developed and launched

MoH policy documents;

Project monitoring reports

Key Informants:

GHS, Family Health

Division; MoH/GHS

PPME (See Appendix

AP4)

Desk review

Semi-

Structured

Interviews/In-

depth

interviews

E2 E2: Sub-national decision-making

To what extent has the project

contributed to strengthen capacity

of regional health management

teams and district health

management teams for planning,

informed decision making and

prioritization of the newborn health

as per the National Child Health

Policy (2007-2015) and other

national guidelines and protocols?

Number of

Regional/district health

management teams

trained in newborn care;

Proportion of district

action plans that have

newborn care activity;

Documents:

Regional/District Health

Action Plans

Key Informants:

Regional/District Directors

of Health Services

Desk review

Semi-

Structured

Interviews/In-

depth

interviews

E3 E3: Service providers

(a) To what extent is the established

level-two (without ventilator and

incubators) newborn care units in

six District Hospitals improved the

management and survival of sick

newborn babies?

(b) Which are the enabling/

constraining factors that facilitated/

hindered the management of sick

newborn babies in District

Hospitals?

Neonatal deaths per 1,000

live births;

Neonatal deaths per total

admission into NCU.

% of kangaroo mother

care; % of sepsis

management

DHIMS; District annual

and activity report;

Facility data from two

NCUs

Key Informants:

Services Providers

Extraction of

data from

district annual

report

Semi-

Structured

Interviews/In-

depth

interviews

Page 82: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

82

E4 E4: Service providers To what extent has there been an

improvement in quality of care

during post-natal care in the health

facilities targeted by the project?

DHIMS (2)

Key Informants:

Services Providers

Extraction of

data from

DHIMS

Semi-

Structured

Interviews/In-

depth

interviews

E5

E5: Service providers:

To what extent has the training and

mentorship component of the

project responded to capacity

building needs of the different

levels of service providers?

Number of health

professionals (Nurse,

Midwife, CHN/CHO)

trained in life-saving

skills and essential

newborn care

Documents

Project monitoring reports;

Training modules and

materials;

Key Informants

Health professionals

trained in newborn care

Desk Review

Semi-

Structured

Interviews/In-

depth

interviews

E6 E6: Final beneficiaries’ level:

To what extent do beneficiaries

report to have been reached by

project communication and social

mobilization interventions, like

community durbars, mother support

groups, community based agents

and Red Cross mothers (in Upper

East Region)?

Number of visits, social

mobilisation events,

durbars

Document: Project

monitoring report, District

annual and activity report

Beneficiaries

Mothers in project

communities

Desk review

Data extraction

from Project

monitoring

report

FGD

E7 E7: Final beneficiaries’ level:

To what extent do beneficiaries

report an improvement in their

newborn care and health seeking

practices (ANC, PNC, well baby

clinic) as a consequence of

improved counseling by

CHO/CHN, CBA, Red Cross

mothers and mother support

groups?

Percentage of low birth

weight babies on

kangaroo mother care;

Percentage of

mother/infant pairs

exclusively breastfeeding

at discharge.

Project review report;

DHIMS (2)

Beneficiaries

Mothers in project

communities

Extraction of

data

FGD

E8 E8: Final beneficiaries

To what extent do the intervention

contributed to improve health and

wellbeing of newborn babies?

Percentage of babies

visited at home by trained

health worker

DHIMS (2)/ District

annual and activity report

Beneficiaries: Mothers in

project communities

Extraction from

DHIMS/

District record

FGD

Page 83: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

83

Efficiency:

Eff1 Eff1: National

(a)Were the allocated resources used

efficiently to achieve the project

objectives?

(b) Are the available resources

adequate to meet project needs?

Some measure of

productivity: Service

output per estimated cost

of the intervention

package (=Service

output/total cost)

Project monitoring report;

District annual and activity

report; DHIMS (2)

Key Informants:

Donor Partners

Extraction of

Cost of project

from Project

monitoring

report; and

service output

from

DHIMS (2).

Semi-structured

interviews/In-

depth interview

Sustainability:

S1 S1: National

Have policy makers at MoH/GHS

demonstrated ownership over the

different interventions related to

newborn survival?

Evidence of dedicated

officer for newborn care

at all levels

Documents: National

Policy documents on

newborn care.

Key informants: DG,

GHS Family health

division

Desk review

Semi structured

interview/In-

depth interview

S2 S2: National

Has the Government of Ghana

prioritized the health and wellbeing

of newborn babies in the

government’s policy documents

(Ghana Shared Growth and

Development Agenda for 2014-

2017, Medium Term Health

Expenditure Plan for 2014-2017)

and allocation of resources (budget

line on approved Ministry of Health

budget) for newborn health?

Evidence of newborn

issues in government

policy documents. Budget

line for newborn in

approved MoH budget.

Amount allocated as a

percentage for the total

budget for newborn care

(GHS)

National Policy documents

on newborn care and

MoH/GHS budget

Key informants: DG,

MoH-PPME, GHS Family

health division

Desk review

Semi structured

interview/In-

depth interview

S3 S3: Sub-national decision-making

Have DHMT and District Assembly

demonstrated ownership and

capacity for resource mobilization

to be able to self-support and

consolidate the achievements and

the expansion of newborn health

interventions?

Evidence of dedicated

officer for newborn care;

Proportion of MMDA

budget dedicated to

MNCH activity.

Documents:

Project activity reports.

Project monitoring

database. Newborn care

budget forecasts. Sub-

national newborn care

budgets.

Key Informants:

Regional/District Health

Managers; District

Assembly

Desk review

Semi-

Structured

Interviews/ In-

depth interview

S4 S4: Service providers

(a) Can the commitment and

motivation of CHO/CHN and

community volunteers (CBAs) that

was enhanced through the project

last, for a continued provision of

home-based services to mothers and

their newborn babies?

(b) What are the bottlenecks and

barriers for the home-based

postnatal care within the framework

of continuum of MNCH care?

Proportion of trained

health professionals (HP)

at post;

Documents

Districts annual reports;

Key Informants: Trained

HPs

Extraction from

district annual

reports

Semi-

Structured

Interviews/ In-

depth interview

Page 84: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

84

S5 S5: Final beneficiaries

Can the behavioral changes among

beneficiaries on essential newborn

care be sustained?

Key informant: Mothers

in project communities

FGD

Coherence:

C1 C1: National

Has the project facilitated synergies

and avoided duplications with

interventions and strategies

promoted by other UN agencies and

development partners (JICA,

USAID, EC, PATH and others)

within the National Child Health

Policy 2007-2015 and MDG5

Acceleration Framework (MAF)?

Evidence of an

established system

supporting coordination

among UN agencies and

development partners

Key Informants:

DG, GHS family health

division, Donor Partners,

UNICEF Project Staff

(See Appendix AP4)

Semi-

Structured

Interviews/ In-

depth

interviews

C2 C2: National

Has the project given due

importance on donor’s (Government

of Japan) visibility in line with

UNICEF’s donor visibility

guidelines.

Evidence of donor’s

visibility in the project

districts

Project districts/NCU

Key Informants:

DG, GHS family health

division, Donor Partners,

UNICEF Project Staff (See

Appendix AP4)

Take photos of

signboards,

equipment, and

supplies with

Government of

Japan stickers

affixed on.

Semi-

Structured

Interviews/ In-

depth

interviews

Human right

based

approach

(HRBA):

H1 H1: National

Has the project incorporated the

HRBA to programming?

Program documents spells

out HRBA elements Documents

The conversion of the right

of the child (i.e. the

standard),

Planning and general

project documents.

Desk review

(review project

document

against the

standard)

H2 H2: National

(a) Has the project considered the

equity approach (i.e. focus on most

deprived areas, areas with high

prevalence of critical newborn and

under-5 mortality, low income

families)

(b) Has the project facilitated the

reduction of access barriers to

MNCH services by final

beneficiaries?

Justification/Criteria for

selecting project regions.

Home-based newborn

postnatal services – PNC

for newborn (48 hours)

Project document

Key Informants:

DG, GHS family health

division, Donor Partners,

UNICEF Project Staff (See

Appendix AP4)

Desk review

Semi-structured

interviews/ In-

depth

interviews

Page 85: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

85

AP4. List of National Decision-Makers/Stakeholders Interviewed No. Participants

Name

Designation Organization Email Address Contact Remarks Area of

Discussion

5. Dr. Isabella

Sagoe-Moses

GHS National

Child Health

Coordinator,

GHS

[email protected] 024 464

6065

Interview

and

Inception

Meeting

Overall project

6. Dr. Patrick

Aboagye

Director Family

Health

Division,

GHS

[email protected] 024 328

3327

Interview

and

Inception

Meeting

Overall project/

MAF

7. Gloria

Quansah-

Asare

Deputy

Director

General

Ghana Health

Service

[email protected] 024 373

3541

Interview

and

Inception

Meeting

Overall project

8. Dr. Agongo

PPME

Director, PPME Programme

Planning,

Monitoring

and

Evaluation,

GHS

[email protected] 024 429

3835

Interview Policy Planning

Issues

9. Dr, George

Amofah

Retired, Deputy

Director

General, GHS

[email protected] 024 432

2843

Interview National

newborn

strategy

development/

general PH

issues

10. Dr. Odame PPME Ministry of

Health

[email protected] 020 886

8792

Inception

Meeting

Interview

on NNS and

MAF

11. Dr. Lorna

Renner

Pediatrician Paedieatric

Society of

Ghana

[email protected] 020 824

3945

Bigger

picture chat

12. Dr Linda

Vanotoo

Grater Accra

Regional

Director

Ghana Health

Service

[email protected] Inception

Meeting

Interview

Perinatal audit

13. Sodzi Sodzi

Tettey

Project Five

Alive

Project

Director

[email protected] 020 630

1109

Inception

Meeting

Interview

Quality of Care

14. Dr. Cynthia

Bannerman/

Deputy

Director

Institutional

Care

Directorate,

Ghana Health

Service

[email protected] 0302662014 Inception

Meeting

Interview

15. Christina

Akuffo

Nurse Quality A GHS [email protected] 023 306

6615

057 842

1237

Quality

Assurance

16. Itsuko

Shirotani

JICA [email protected] 0244871042 Inception

Meeting

Interview

Donor

perspective

17. Salamatu

Futa

USAID [email protected] 0244247903 Inception

Meeting

Interview

Donor

perspective

Page 86: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

86

18. Vandana USAID Inception

Meeting

Interview

Donor

perspective

19. Dr. Divine

Atpura and

Alex Nazar

USAID System

for Health

[email protected] 0244760

799

Inception

Meeting

Interview

Health System

20. Chantelle

Allen/Team

Joyce

ablorrdeppey

Jhpiego [email protected] 0545410970 Inception

Meeting

Interview

Training/quality

of care

21. Patience

Cofie

PATH [email protected] 0242681272 Inception

Meeting

Interview

22. Jannet Mortoo EC Programme

Officer EC

Bigger

picture

chat- MAF

MAF support

from EC

23. Esi Amoaful GHS Director

Nutrition

Division

Inception

Meeting

Interview

24. Dr Robert

Mensah

UNFPA Bigger

picture

chat- MAF

UNICEF Cross Sectoral/Accra

21 David WASH Wash in

Health

Facility

22 Lilian and

Gloria

Nutrition Chat Breast

Feeding –

Discussion

23 Anna Maria/

Clemens

M&E Anna is

team of

evaluation

24 Philomena/

Peter

LEAP 1000 UNICEF LEAP 100

Chat

25 Emelia Allen Child

Protection

Birth

Registration

26 Surangani/

Charity

C4D Chat on

C4D

27 Monica and

Evelyn

Baddoo

Communication Visibility

UNICEF/Tamale

28 Ms. Felicia

Mahama, Dr.

Imran

H&N Interview

Page 87: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

87

AP5a. Evaluators’ work schedule.

Table 3.2 Work schedule

Date Time Activity Venue

2 January 2015 9am – 12 noon Arrival of International Consultant Accra

5th January 11am-12.15pm Internal Meeting with UNICEF Team UNICEF House

6th January 11am- 1pm Inception meeting with National

Newborn Sub-Committee

UNICEF House

7th – 9th January 8am-4pm each day Bilateral meetings with Key National

decision makers and health partners for

in-depth interviews

Accra

13th – 14th January Travel to Tamale (Northern region) by

road with a sleep over in Kumasi.

Introductory meeting with UNICEF

Tamale Team.

Meeting with Regional Director, GHS,

Tamale

Travel to Bolga (Upper East Region)

Tamale

Bolga

14th to 19th January am/pm Field visit Upper East Region

19th January Afternoon/evening Return to Tamale

20th – 28th January am/pm Field visit Northern Region

28th January Morning Return to Accra

28th – 4th February Data analysis, report writing and

presentation

Meetings with partners

Submission of draft Evaluation

report

UNICEF House

5th February National Debriefing Meeting UNICEF House

7th February Departure from Accra (International

Consultant)

AP5b. Percentage change in institutional neonatal deaths by districts in the two regions

Table 4.1: Percentage change in institutional neonatal deaths per 1,000 live

births over the period 2012-2014 in the project districts

Region/Districts 2012 2014 % change

Northern 6.9 3.4 -50.7

Bole 7.6 6.7 -11.8

Gushiegu 1.8 0.59 -67.2

Kpandai 3.1 1.8 -41.9

Saboba 0.7 1 42.9

Savelugu-Nanton 1.4 0 -100.0

Tolon 1.5 0 -100.0

Page 88: Formative Evaluation Report for - UNICEF · PDF file3.3 Sample size consideration and ... BEmONC Basic Emergency Obstetric and Newborn CarCBACommunity ... MAF MDG5 Acceleration Framework

88

AP6. Terms of Reference (Attached)

Kumbungu 1.3 0 -100.0

Yendi 2.2 0 -100.0

Upper East 5.8 3.3 -43.1

Bawku 12 4.1 -65.8

Bawku West 4.7 0.83 -82.3

Bolgatanga 6.9 5.3 -23.2

Kasena-Nankana 17.2 13.4 -22.1

Kasena-Nankana West 1.2 0 -100.0

Garu-Tempane 0 0.49

Talensi 0 0

Nabdam 1.1 0 -100.0

Source: Ghana Health Service DHIMS(2) as @ 19th Jan 2014