Assessing the Impact of Using PBI to Improve the Quality of Obstetric

and Newborn Care in Malawi

PI: Manuela De Allegri

Co-PI: Adamson Muula

Scientific coordinator: Stephan Brenner

The teamAdamson Muula

Don Mathanga

Jobiba Chinkhumba

Jacob Mazalale

Christabel Kambala

Till Bärnighausen

Tom Bossert

Manuela De Allegri

Stephan Brenner

Albrecht Jahn

Paul Robyn

Malabika Sarker

Julia Lohmann

Bjarne Robberstad

Gaute Torsvik

Outline of the day

• Study objectives and structures - Manuela

• Baseline data collection - Adamson

• Household survey: service utilization - Jacob

• Household survey: complications – Jobiba (Manuela)

• Health facility tools: clinical quality of care - Stephan

• Exit interviews: perceived quality of care - Christabel

• Providers’ tool: basic results - Julia

Study objectives & overall study design

Manuela De Allegri

Our research question

What is the impact of a combination of supply-side and demand-side interventions

on quality of maternal care services?

At what cost are the benefits of the RBF intervention produced?

Focus: impact evaluation, not operational/process evaluation

Impact evaluation

Focus is on the counterfactual:

What would have happened to quality of maternal care services in the absence of RBF?

Reflected in study design: mixed methods controlled before & after design

Mixed methods in practice

QUAN: What changes (if any) on specific measurable utilization and quality indicators does RBF produce?

At what costs for society are these changes produced?

QUAL: How and why does RBF produce these changes?

How do the various stakeholders experience the intervention?

The quantitative design

Untreated control with pre-test and multiple post-test measurements (i.e.controlled before and after study)

Collect information on all indicators at baseline (i.e. before RBF is implemented), at mid-term, and at end point in all intervention facilities and in all facilities initially surveyed by the implementation team, but later not included in the intervention

Prospective design

The analytical approach

t=0 t=1

0

0,5

1

1,5

2

2,5

3

3,5

4

1,5

2,5

3,5

1

1,5

2

Difference-in-Difference with parallel time-trend#

of h

os

pita

l vis

its

Scope, focus, outcome

Scope: ANC, delivery, early PNC, and neonatal care

Focus: continuity of care around EmONC

Outcome: utilisation and quality indicators

Our conceptual model

vPROCESS OUTCOME

Provision of care

Experience of care

INPUT

MIXED METHODS APPROACH

INPUT PROCESS OUTCOME

Infrastructuralassessment

Providers structured interviews

Provider-patient direct observations

Exit interviews

Assessment of clinical records

Assessment of clinical records

Community-based survey

Providers structured interviews

Experience of care

FGD & in-depth interviews with stakeholders

Provision of care

ECONOMIC EVALUATIONPhD project of

Jobiba Chinkhumba under supervision of

Professor Bjarne Robberstad

Conceptual questionsRBF will be compared to current MCH funding

scheme (comparator):

Costs

DALYs averted

Conceptual questions

What are the costs of a RBF scale up likely to be?

Costs

DALYs averted

Conceptual questions

How much is RBF likely to improve maternal and child health?

Costs

DALYs averted

Conceptual questionsAre the incremental costs likely to be «value for money» compared to best available opportunities?(best opportunities illustrated by the cost-effectiveness threshold)

Costs

DALYs averted

Cost-effectiveness threshold

RBF not «cost-effective»

RBF «cost-effective»

How will we calculate effectiveness?

• Adverse events from the household survey (preferred method)

• Systematic review and meta-analysis of existing data (common back-up approach)

• Base the DALY estimates on either or both these sources

• Final decision after follow up data has been analysed

Principle: To apply the best available evidence

Decision model• Decision tree

– Current standard of care vs RBF– Branches for all major potential adverse outcomes

• Probabilities of adverse outcomes– Clinical trial and/or metaanalysis

• Costs– Facility and patient surveys

• Consequences of AO and DALY estimates– Standard BOD 2010 assumptions – http://www.healthmetricsandevaluation.org/gbd

Baseline data collection

Adamson Muula

Field work logistics

• Data collection completed between March & May 2013

• Field coordinator: Judith Daire

• Field supervisors: Jacob, Christabel, Jobiba

• Supported by: Julia and Stephan

• 30 interviewers recruited & trained at CoM

Great achievement ...

• 32 health facility assessments

• 79 providers‘ interviews

• 825 exit interviews: 390 ANC; 203 delivery; 232 PNC

• 337 direct observations: 236 ANC, 76 non-complicated deliveries, 8 complicated deliveries

• 1891 women (having delivered in prior 12 months) reached through community-survey

... in spite of some challenges

• Complex organization of field work procedures due to facility-based & community-based data collection

• Complex field logistics due to timing of service provision at surveyed facilities

• Non-welcoming attitude at control facilities

• Difficulties with programming digital data collection – resulting in long and tedious data cleaning process

THANK YOU