Measuring the Impact: Cross-Country Quantitative Analysis

Previous Studies

Background

• Debate regarding the capacity of developing countries, particularly low-income countries, to effectively spend significant new resources for health

Measuring absorptive capacity

• Ratio of cumulative disbursements in each quarter over total grant commitment for Phase One.

Data and Methods

• Time series and cross sectional data on disbursement, grants characteristics and country characteristics

• Various estimation methods

Major Findings

• Low-income countries, were more likely to have a higher rate of disbursement than high income countries holding everything else constant.

• For a given level of income, countries with less-developed health systems were more likely to have a higher rate of grant disbursement than nations with more-developed health systems.

Data and Methods

• Time series and cross sectional data from 1995 to 2004 on DTP3 coverage, country’s characteristics

• Various estimation methods

Results• In countries with DTP3 coverage of 65%

or less at baseline, ISS spending per surviving child had a significant positive effect on DTP3 coverage.

• This effect was not present in countries with DTP3 coverage of 65–80% or 80% or more at baseline.

Evaluating the Impact of the Global Fund on Fighting against Tuberculosis

IHEA 6th World Conference July 9, 2007Session: Evaluating Global Health Initiatives

Presenter: Chunling Lu

Authors: Chunling Lu, PhD, Senior Research Associate Catherine Michaud, PhD, Senior Research Scientist

Andrew Stokes, Research AssistantChristopher J.L Murray, M.D, DPhil

Institute Director, Institute for Health Metrics and Evaluation and Professor of Global Health, University of Washington

Effects of GFATM on Malaria

IHEA 6th World Conference July 9, 2007Session: Evaluating Global Health Initiatives

Presenter: Chunling Lu

Authors: Catherine Michaud, PhD, Senior Research ScientistFelix Masiye, PhD, Research Fellow

and

Christopher J.L Murray, M.D, DPhilInstitute Director, Institute for Health Metrics and Evaluation and Professor of

Global Health, University of Washington

Measuring the Performance of Global Fund and PEPFAR Monies on the Expansion of ARV

Treatment

IHEA 6th World Conference July 9, 2007Session: Evaluating Global Health Initiatives

Presenter: Christopher Murray

Authors: Chunling Lu, PhD, Senior Research AssociateCatherine Michaud, PhD, Senior Research Scientist

Felix Masiye, PhD, Research Fellow

and

Christopher J.L Murray, M.D, DPhilInstitute Director, Institute for Health Metrics and Evaluation and Professor of Global

Health, University of Washington

Positive Synergies between GHIs and HSS

1.Impact of GHIs on HSS?2. Impact of HSS on GHIs?

Positive Synergies between GHIs and HSS

1.Impact of GHIs on HSS?

Measuring the impact of GHIs

• Disbursements of GHIs to recipient countries

Global Resources for Health- by IHME

Global Resources for Health- by IHME

• A comprehensive data on all types of resources flows for health is lacking.

• A subset of global players

10.6

12.9

15.4

17.7

20.1

10.59.69.7

7.57.47.3

6.35.55.45.24.9

4.3

0

5

10

15

20

25

1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006

USD

Bill

ions

(Con

stan

t 200

6)

Total IAFH, 1990 - 2006

Percentage Increase from 1990 - 2006: 370%Average Growth Rate per Year: 10.5%

Selecting GHIs

• Over 80 GHIs • Largest GHIs: GFATM, GAVI, and

PEPFAR• Time-series cross-sectional disbursements

data available

Evaluation Matrix of the Impact of GHIs on Health Systems

Outputs of a health system

GHIs investments

Process of a health system

Inputs of a health system

Outcomes of a health system

Input MeasuresMoney value of all resources used by thehealth system• Government/Public health spending• Private health spending (households out-

of-pocket health spending)• External sources (i.e. GHIs)

Process Measures

• Human resources• Infrastructure and equipment• Drug supply• Operational measures• Program activities

Output Measures

• Coverage/Effective Coverage- target diseases (HIV, Malaria, TB)- medical interventions for non-target diseases- basic medical services (i.e. immunization)

Outcomes Measures

GHIs investments

Government spending on health

Coverage of skilled-attendance at delivery

Coverage of immunization

Child mortality

Cross-country quantitative study- Four projects in eight months

Impact on government health spending – substitution effect?

• Government spending on health indicates the commitment of a government to public health.

• Expanding government spending on health is critical to maintaining financing sustainability in developing countries.

Impact on interventions of non-target diseases or programs

• Immunization• Skilled-attendance at delivery

Impact on population health outcomes

• Child mortality

Time-series and Cross-sectional Data

• Disbursements of GFATM, GAVI, and PEPFAR

• Government Health Spending- NHA data by WHO- OECD CRS on external sources

• Child mortality- IHME

• Country characteristics variables

Data sources: WHO 2008

01020304050607080

Government Health Spending Spending of MoH Social Security Funds

% of imputed % of well-documented % of missing

Figure 1 Public Spending from 1996-2005 across 127 Countries

Missing Data

Dealing with missing data and low-quality data

• Developing estimation methods• Providing uncertainty level for estimates• Test the sensitivity of findings to estimates

derived from various methods

Positive Synergies between GHIs and HSS

2. Impact of HSS on GHIs?

Challenges

• Outcome variables- How to differentiate the health outcomes by GHIs from that by health system

• Measurements of the strength of a health system

Limitations of Cross-Country Quantitative Analysis

1. Data availability the statistical power2. Robustness of findings3. Not able to provide evidence on detailed

practice in specific countries