Upload
others
View
4
Download
0
Embed Size (px)
Citation preview
Document of
The World Bank
Report No: ICR00003781
IMPLEMENTATION COMPLETION AND RESULTS REPORT
ON A
CREDIT
IN THE AMOUNT OF SDR 38.3 MILLION
(USD60.0 MILLION EQUIVALENT)
TO THE
SOCIALIST REPUBLIC OF VIETNAM
FOR A
NORTHERN UPLAND HEALTH SUPPORT PROJECT
August 24, 2016
Health, Nutrition and Population Global Practice
East Asia and Pacific Region
CURRENCY EQUIVALENTS
(Exchange Rate Effective February 29, 2016)
Currency Unit = Vietnamese Dong (VND)
VND 22,174.64 = USD 1.00
FISCAL YEAR
January 1 – December 31
ABBREVIATIONS AND ACRONYMS
ADB Asian Development Bank
CNHSP Central North Health Support Project
CPMU Central Project Management Unit
CPS Country Partnership Strategy
DO Development Objective
EC European Community
FA Financing Agreement
FHCIC Free Health Care Insurance for Children under six years
HCFP Health Care Funds for the Poor
HCWM Health Care Waste Management
HEMA Health Care Support to the Poor of the Northern Upland and Central
Highlands
HI Health Insurance
HIC Health Insurance Card
HIS Health Information System
HMIS Health Management and Information System
ICR Implementation Completion and Results Report
IEC Information, Education, and Communication
IMR Infant Mortality Rate
IOI Intermediate Outcome Indicator
IPF Investment Project Financing
ISR Implementation Status and Results Report
KPI Key Performance Indicator
MHI Millennium Development Goal’s Health Indicator
MHSP Mekong Regional Health Support Project
MDG Millennium Development Goal
M&E Monitoring and Evaluation
MMR Maternal Mortality Rate
MoH Ministry of Health
MTR Midterm Review
NMR Neonatal Mortality Rate
NORRED North East Red River Delta Region Health System Support Project
NUP Northern Upland Health Support Project
PAD Project Appraisal Document
PDO Project Development Objective
PPMU Provincial Project Management Unit
QER Quality Enhancement Review
RF Results Framework
SBA Skilled Birth Attendant
SHI Social Health Insurance
SIL Specific Investment Loan
TTL Task Team Leader
UHC Universal Health Coverage
VHI Voluntary Health Insurance
VHLSS Vietnam Household Living Standard Survey
Senior Global Practice Director: Timothy G. Evans
Practice Manager: Toomas Palu
Project Team Leader: Anh Thuy Nguyen
ICR Team Leader: Andre Medici
ICR Primary Author Andre Medici
VIETNAM
Northern Upland Health Support Project
TABLE OF CONTENTS
A. Basic Information…………………………………………………………………....i
B. Key Dates .................................................................................................................... i
C. Ratings Summary ........................................................................................................ i
D. Sector and Theme Codes ........................................................................................... ii
E. Bank Staff ................................................................................................................... ii
F. Results Framework Analysis ...................................................................................... ii
G. Ratings of Project Performance in ISRs ................................................................... ix
H. Restructuring .............................................................................................................. x
I. Disbursement Profile .................................................................................................. xi
1. Project Context, Development Objectives and Design ............................................... 1
2. Key Factors Affecting Implementation and Outcomes .............................................. 6
3. Assessment of Outcomes .......................................................................................... 15
4. Assessment of Risk to Development Outcome ......................................................... 23
5. Assessment of Bank and Borrower Performance ...... Error! Bookmark not defined.
6. Lessons Learned ..................................................................................................... 277
7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners .......... 28
Annex 1. Project Costs and Financing .......................................................................... 30
Annex 2. Project Outputs (According PAD -Annex 4) ................................................ 31
Annex 2A. Project Output Map……………………………………………………….43
Annex 3. Economic Analysis: Outputs Efficiency, Benefits and Equity Impacts…….48
Annex 3A Estimated Unitary Costs of the Project Outputs ……………………..……59
Annex 4. Bank Lending and Implementation Support/Supervision Processes………..60
Annex 5. Results Framework: Analysis of the PDO Achievement…..………...……..62
Annex 5A. Rating of the Indicators According Achievement………………………...66
Annex 6. Borrowers ICR………………………………………………………….…...71
Annex 6A. Project Risk management (Borrowers View)……………………………..86
Annex 6B. Project Results Framework (Borrowers View)……………………………89
Annex 6C. Achievements by components (Borrowers View)……………………...…92
Annex 7. Summary of the Project Aide-Memoires……………………………………98
Annex 8. List of Supporting Documents……………………………………….…….100
MAP…………………………………………………………………………….…….102
i
A. Basic Information
Country: Vietnam Project Name: Northern Upland
Health Support Project
Project ID: P082672 L/C/TF Number(s): IDA-43980
ICR Date: 08/31/2016 ICR Type: Core ICR
Lending Instrument: SIL Borrower:
SOCIALIST
REPUBLIC OF
VIETNAM
Original Total
Commitment: XDR 38.30M Disbursed Amount: XDR 37.31M
Revised Amount: XDR 38.30M
Environmental Category: B
Implementing Agencies:
Ministry of Health, Vietnam Central Project Management Unit (CPMU)
The Seven Northern Upland Provinces (Cao Bang, Bac Kan, Lao Cai, Ha Giang, Son La, Dien
Bien and Lai Chau) and their respective Provincial Project Management Units (PPMU)
Co-financiers and Other External Partners: No
B. Key Dates
Process Date Process Original Date Revised / Actual
Date(s)
Concept Review: 05/31/2006 Effectiveness: 10/08/2008 10/08/2008
Appraisal: 11/15/2007 Restructuring(s): — 08/29/2014
Approval: 03/13/2008 Mid-term Review: 07/16/2012 07/16/2012
Closing: 08/31/2014 02/29/2016
C. Ratings Summary
C.1 Performance Rating by ICR
Outcomes: Satisfactory
Risk to Development Outcome: Moderate
Bank Performance: Satisfactory
Borrower Performance: Satisfactory
C.2 Detailed Ratings of Bank and Borrower Performance (by ICR)
Bank Ratings Borrower Ratings
Quality at Entry: Moderately Satisfactory Central Government -
CPMU: Satisfactory
Quality of Supervision: Satisfactory Regional Governments
PPMU: Moderately Satisfactory
ii
Overall Bank
Performance: Satisfactory
Overall Borrower
Performance: Satisfactory
C.3 Quality at Entry and Implementation Performance Indicators
Implementation
Performance Indicators
QAG Assessments
(if any) Rating
Potential Problem Project
at any time (Yes/No): No
Quality at Entry
(QEA): None
Problem Project at any
time (Yes/No): No
Quality of
Supervision (QSA): None
DO rating before
Closing/Inactive status: Satisfactory
D. Sector and Theme Codes
Original Actual
Sector Code (as % of total Bank financing)
Central government administration 8 8
Compulsory health finance 15 15
Health 71 71
Sub-national government administration 6 6
Theme Code (as % of total Bank financing)
Health system performance 100 100
E. Bank Staff
Positions At ICR At Approval
Vice President: Victoria Kwakwa James W. Adams
Country Director: Achim Fock (Acting) Ajay Chibber
Practice
Manager/Manager: Toomas Palu Fadia M. Saadh
Project Team Leader: Anh Thuy Nguyen Maryam Salim
ICR Team Leader: Andre Medici —
ICR Primary Author: Andre Medici —
F. Results Framework Analysis
Project Development Objectives (from Project Financial Agreement)
The objective of the Project is to increase the utilization of district health services
especially among the poor and ethnic minorities population of the Northern Upland
iii
Provinces through improving the quality of district-level hospitals and reducing financial
constraints to access to health services.
Revised Project Development Objectives (as approved by original approving authority)
The PDO was not revised. The Project originally had four Key Performance Indicators
(KPIs) and 10 Intermediate Outcome Indictors (IOI) in the Project Appraisal Document
(PAD). However, most of baselines and targets of the project indicators were set during
the first year of implementation according to the result of a Baseline Survey conducted in
June 2009 and published in July 2009. The four KPIs were retained and the number of IOIs
increased from 10 to 13. No other changes were made to the indicators. All KPIs and most
of the IOIs’ targets were achieved by December 31, 2015 (before the project Closing Date
of February 29, 2016).
(a) PDO Indicator(s)
Indicator1 Baseline Value
Original Target
Values (from
approval
documents)
Formally
Revised
Target
Values
Actual Value
Achieved at
Completion or
Target Years
Indicator 1: Utilization rates of in-patient services in District Hospitals among Decision 139
beneficiaries
Value
(Quantitative or
Qualitative)
0.027 0.033 — 0.096
Date achieved 06/30/2009 08/31/2014 12/31/2015
Comments
(including %
achievement)
Target surpassed (191%).
Indicator 2: Utilization rates of out-patient health services in district hospitals by Decision
139 beneficiaries
Value
(Quantitative or
Qualitative)
0.067 0.075 — 0.247
Date achieved 06/30/2009 08/31/2014 12/31/2015
Comments
(including %
achievement)
Target surpassed (229%).
Indicator 3: Percentage of households who experience catastrophic healthcare expenditures in
the year prior to the survey.
Value
(Quantitative or
Qualitative)
14.27% 13.23% — 2.00%
Date achieved 06/30/2008 08/31/2014 08/31/2014
Comments Target surpassed (561%). Not measured during the 18-month extension period.
1 Percentage of achievement in this table was calculated as a simple percentage increase of the last actual
on the proposed target.
iv
(including %
achievement)
Indicator 4:
Proportion of district hospitals that provide full set of health services according
to the national norms (Decision 23/205/QB- BYT) adjusted to the Northern
Uplands.
Value
(Quantitative or
Qualitative)
39.1% 70.0% — 80.4%
Date achieved 06/30/2008 08/31/2014 12/31/2015
Comments
(including %
achievement)
Target surpassed (15%). For operational reasons, after the MTR, this indicator
was adjusted to be read as “average percentage of health services covered by the
district hospitals”.
(b) Intermediate Outcome Indicator(s)
Indicator Baseline Value
Original Target
Values (from
approval
documents)
Formally
Revised
Target Values
Actual Value
Achieved at
Completion or
Target Years
Indicator 1: Percent of patients satisfied with the health services;
Value
(Quantitative
or Qualitative)
8.5% 10.2% — 84.4%
Date achieved 07/31/2009 08/31/2014 08/31/2014
Comments
(including %
achievement)
Target surpassed (727%). Not measured during the 18-month extension period.
Indicator 2: Adherence of treatment protocols for selected conditions in impatient settings.
Value
(Quantitative
or Qualitative)
— — — —
Date achieved 06/30/2009 08/31/2014 08/31/2014
Comments
(including %
achievement)
Measured by sub-indicators 2.1 to 2.9 (all surpassed). Not measured during the
18-month extension period.
Indicator 2.1: Percent of health workers with knowledge of diagnosing and treating Level
A/B/C dehydrated diarrhea
Value
(Quantitative
or Qualitative)
9.7% 14.0% — 95.2%
Date achieved 06/30/2009 08/31/2014 08/31/2014
Comments
(including %
achievement)
Target surpassed (580%). Not measured during the 18-month extension period.
Indicator 2.2: Percent of health workers with knowledge of diagnosing and treating severe
pneumonia
Value
(Quantitative 13.2% 18.5% — 86.9%
v
Indicator Baseline Value
Original Target
Values (from
approval
documents)
Formally
Revised
Target Values
Actual Value
Achieved at
Completion or
Target Years
or Qualitative)
Date achieved 06/30/2009 08/31/2014 08/31/2014
Comments
(including %
achievement)
Target surpassed (370%). Not measured during the 18-month extension period.
Indicator 2.3: Percent of health workers with knowledge of diagnosing and treating poisoning
Value
(Quantitative
or Qualitative)
26.8% 37.5% — 83.0%
Date achieved 06/30/2009 08/31/2014 08/31/2014
Comments
(including %
achievement)
Target surpassed (121%). Not measured during the 18-month extension period.
Indicator 2.4: Percent of Reasonable diagnoses of severe pneumonia
Value
(Quantitative
or Qualitative)
45.5% 63.7% — 71.1%
Date achieved 06/30/2009 08/31/2014 08/31/2014
Comments
(incl. %
achievement)
Target surpassed (12%). Not measured during the 18-month extension period.
Indicator 2.5: Percent of Clinical health workers’ reasonable diagnosis of general pneumonia.
Value
(Quantitative
or Qualitative)
19.6% 27.4% — 57.0%
Date achieved 06/30/2009 08/31/2014 08/31/2014
Comments
(including %
achievement)
Target surpassed (108%). Not measured during the 18-month extension period.
Indicator 2.6: Percent of Reasonable diagnoses of Level A dehydrated diarrhea
Value
(Quantitative
or Qualitative)
37.2% 52.1% — 78.9%
Date achieved 06/30/2009 08/31/2014 08/31/2014
Comments
(including %
achievement)
Target surpassed (51%). Not measured during the 18-month extension period.
Indicator 2.7: Percent of Reasonable diagnoses of Level B dehydrated diarrhea.
Value
(Quantitative
or Qualitative)
48.9% 68.5% — 85.1%
Date achieved 06/30/2009 08/31/2014 08/31/2014
Comments
(including % Target surpassed (24%). Not measured during the 18-month extension period.
vi
Indicator Baseline Value
Original Target
Values (from
approval
documents)
Formally
Revised
Target Values
Actual Value
Achieved at
Completion or
Target Years
achievement)
Indicator 2.8: Percent of Reasonable diagnosis of Level C dehydrated diarrhea
Value
(Quantitative
or Qualitative)
41.2% 57.7% — 80.0%
Date achieved 06/30/2009 08/31/2014 08/31/2014
Comments
(including %
achievement)
Target surpassed (39%). Not measured during the 18-month extension period.
Indicator 2.9: Percent of Reasonable diagnoses of poisoning.
Value
(Quantitative
or Qualitative)
61.1% 85.5% — 86.2%
Date achieved 06/30/2009 08/31/2014 08/31/2014
Comments
(including %
achievement)
Target achieved. Not measured during the 18-month extension period.
Indicator 3: Percent of eligible district health staff who have successfully completed training
provided by the project
Value
(Quantitative
or Qualitative)
— — — —
Date achieved 06/30/2008 08/31/2014 12/31/2015
Comments
(including %
achievement)
Measured by sub-indicators 3.1 to 3.3 (all surpassed).
Indicator 3.1: Percent of doctors and assistant doctors and pharmacists at district hospitals
trained by the project
Value
(Quantitative
or Qualitative)
0% 80% — 189%
Date achieved 06/30/2008 08/31/2014 12/31/2015
Comments
(including %
achievement)
Target surpassed (136%).
Indicator 3.2: Percent of health staffs with completed short-term training courses compared to
the plan.
Value
(Quantitative
or Qualitative)
0% 80% — 357%
Date achieved 06/30/2008 08/31/2014 06/30/2014
Comments
(including %
achievement)
Target surpassed (336%). Not measured during the 18-month extension period.
vii
Indicator Baseline Value
Original Target
Values (from
approval
documents)
Formally
Revised
Target Values
Actual Value
Achieved at
Completion or
Target Years
Indicator 3.3: Percent of health staffs completed long-term training courses compared to the
plan
Value
(Quantitative
or Qualitative)
0% 80% — 88%
Date achieved 06/30/2008 08/31/2014 06/30/2014
Comments
(including %
achievement)
Target surpassed (10%). Not measured during the 18-month extension period.
Indicator 4: Percent of eligible district hospitals with acceptable operations and maintenance
plans and budget for facility and equipment maintenance
Value
(Quantitative
or Qualitative)
— — — —
Date achieved 06/30/2008 08/31/2014 12/31/2015
Comments
(including %
achievement)
Measured by sub-indicators 4.1 and 4.2 (both surpassed)
Indicator 4.1: Percent of district hospitals having schedule and budget for maintenance of
infrastructure
Value
(Quantitative
or Qualitative)
49.2% 40.0% — 79.7%
Date achieved 06/30/2008 08/31/2014 12/31/2015
Comments
(including %
achievement)
This indicator had a target value of 40% in the PAD. The baseline was measured
after the project approval. The target value was not revised during
implementation to be compatible with the baseline. Consequently, achievement
for this indicator has been calculated over the baseline value of 49.2% instead of
its target value of 40%. Baseline value surpassed (62%)
Indicator 4.2: Percent of district hospitals having schedule and budget for maintenance of
equipment.
Value
(Quantitative
or Qualitative)
77.1% 40.0% — 89.1%
Date achieved 06/30/2008 08/31/2014 06/30/2014
Comments
(including %
achievement)
This indicator had a target value of 40% in the PAD. The baseline was measured
after the project approval. The target value was not revised during
implementation to be compatible with the baseline. Consequently, achievement
for this indicator has been calculated over the baseline value of 77.1% instead of
its target value of 40%. Baseline value surpassed (16%).
Indicator 5: Number of Health facilities constructed renovated, and/or equipped.
Value
(quantitative
or Qualitative)
0 61 — 64
viii
Indicator Baseline Value
Original Target
Values (from
approval
documents)
Formally
Revised
Target Values
Actual Value
Achieved at
Completion or
Target Years
Date achieved 06/30/2009 08/31/2014 12/31/2015
Comments
(including %
achievement)
Target achieved. This core indicator was added during project implementation.
Indicator 6: Percent of recently discharged patients satisfied with health services.
Value
(Quantitative
or Qualitative)
8.5% 10.2% — 84.4%
Date achieved 06/30/2009 08/31/2014 06/30/2014
Comments
(including %
achievement)
Target Surpassed (727%). This indicator was added during project
implementation.
Indicator 7: Number of People with access to a basic package of health, nutrition &
reproductive health services.
Value
(Quantitative
or Qualitative)
20% 70% — —
Date achieved 06/30/2009 08/31/2014 12/31/2015
Comments
(including %
achievement)
This indicator was added during project implementation. It cannot be measured
because the baseline and the target (expressed in percentage) were not converted
in the number of beneficiaries with access to the basic package of health,
nutrition & reproductive health services during the project life. Nevertheless, the
number of beneficiaries reached was 270,254 by December 2015.
Indicator 8: Percentage of Decision 139 beneficiaries who have received Health Insurance
Cards
Value
(Quantitative
or Qualitative)
82.1% 70% — 95.2
Date achieved 06/30/2009 08/31/2014 12/31/2015
Comments
(including %
achievement)
Target surpassed (16%). The target of this indicator was established before the
baseline survey. For this reason, the baseline value in the PAD was higher than
the target value. The project team did not revised the target during project
implementation. Consequently, the achievement of this indicator has been
calculated over the baseline instead the target.
Indicator 9: Percentage of 139 Beneficiaries with cards who can correctly identify at least
three benefits covered under the HCFP program
Value
(Quantitative
or Qualitative)
14.8% 75% — 57.4%
Date achieved 06/30/2009 08/31/2014 08/31/2014
Comments
(including %
achievement)
The target was not achieved. Not measured during 18-month project extension
period. However, the following alternate indicator was measured during
implementation “% the poor knowing at least 1 right of health insurance card
holders”. The achievement of this indicator was 95% by December 2015.
ix
Indicator Baseline Value
Original Target
Values (from
approval
documents)
Formally
Revised
Target Values
Actual Value
Achieved at
Completion or
Target Years
Indicator 10: Percentage of households who identify financial barriers as a main cause for not
seeking health care.
Value
(Quantitative
or Qualitative)
2.0% 1.8% — 1.2%
Date achieved 06/30/2009 08/31/2014 8/31/2014
Comments
(including %
achievement)
The target was surpassed (50%). Not measured during the project extension.
Indicator 11: Project management units at central level and provincial level established; bank
accounts opened, staff and consultant recruited, system established
Value
(Quantitative
or Qualitative)
— — — Achieved
Date achieved 06/30/2008 06/30/2009
Comments
(including %
achievement)
CPMU/PPMUs management, staff and consultants fully board by June 2009.
Project operational and financial systems have been maintained throughout the
project life.
Indicator 12: Project management units prepare adequate plans, meet annual implementation
targets and provide timely financial and activity report.
Value
(Quantitative
or Qualitative)
— — — Achieved
Date achieved 06/30/2009 12/31/2015
Comments
(including %
achievement)
The project's annual working plans, progress reports, interim financial reports and
audits reports were submitted in time with acceptable quality, except for some
reports on project result indicators toward the last year of project implementation.
Indicator 13: Availability of data for project monitoring and evaluation
Value
(Quantitative
or Qualitative)
— — — Achieved
Date achieved 02/15/2016
Comments
(including %
achievement)
The project collected data from project related surveys (2009, 2012, and 2014)
and from administrative records by the CPMU. Final project evaluation was
conducted and completed in mid-February 2016.
G. Ratings of Project Performance in ISRs
No. Date ISR
Archived DO IP
Actual
Disbursements
(USD millions)
1 11/21/2008 Satisfactory Satisfactory 0.00
2 01/22/2010 Moderately Satisfactory Moderately Satisfactory 3.03
3 02/21/2011 Moderately Satisfactory Moderately Satisfactory 9.47
x
4 03/26/2012 Moderately Satisfactory Moderately Satisfactory 18.64
5 04/24/2013 Satisfactory Moderately Satisfactory 28.87
6 08/26/2013 Satisfactory Moderately Satisfactory 35.97
7 05/05/2014 Moderately Satisfactory Moderately Satisfactory 49.80
8 11/24/2014 Moderately Satisfactory Moderately Satisfactory 53.51
9 05/14/2015 Moderately Satisfactory Moderately Satisfactory 55.96
10 07/29/2015 Satisfactory Moderately Satisfactory 55.96
11 02/25/2016 Satisfactory Moderately Satisfactory 57.04
H. Restructuring (if any)
Restructuring
Date(s)
Board
Approved
PDO
Change
ISR Ratings at
Restructuring Amount Disbursed
at Restructuring in
USD, millions
Reason for Restructuring & Key
Changes Made DO IP
08/29/2014 No MS MS
52.1
Level II restructuring: extension of the
closing date from August 31, 2014 to
February 29, 2016. The project
achieved its KPIs’ targets on time and
generated savings. The restructuring did
not change the PDOs and KPIs/IOIs,
but redirected the remaining funds to:
(a) Maximize efficiency and enhance
the sustainability of the Project
investments in the seven provinces, and
(b) Contribute to the achievement of
Vietnam health MDGs in related areas,
especially toward reducing child
mortality and improving maternal
health. Many of the IOIs (especially
those which required a new client
survey) were not monitored because
Government decisions to do not
conduct a project survey after the 18-
month extension period. The end of
project survey was conducted in May-
June 2014 and published in October
2014.
xi
I. Disbursement Profile
1
1. Project Context, Development Objectives, and Design
1.1 Context at Appraisal
1. In the 20 years prior to 2007, Vietnam had one of the highest rates of economic growth
and poverty reduction in the world, despite the persistence of higher inequality. By appraisal
(2007), the country had accumulated a remarkable development performance because of important
economic, social, and political reforms delivered since 1986. These reforms contributed to an
average annual growth rate of 7.4 percent per year between 1990 and 2008, coupled with a fast
reduction in poverty from 58.0 percent to 14.5 percent in the same period, based on the national
poverty line, and a fall in the proportion of people living under a dollar a day from 63.0 percent to
21.5 percent between 1993 and 2006.
2. Despite the fact that an important dimension of the nature of the Vietnamese
economic growth had been its inclusiveness, inequalities and disparities across regions
remained a challenge. In 1993, poverty in rural areas was 2.6 times higher than in urban areas.
By 2008, it was 5.7 times greater. Poverty was concentrated in the Northeast, North, Central Coast,
and Central Highlands—with approximately one-third of the population in those regions living
below the poverty line (mostly associated with ethnic minorities) by the time of project appraisal.
3. In 2007, most Vietnam’s health-related Millennium Development Goal (MDG)
outcome indicators were on track. The under-five mortality rate decreased by 65 percent
between 1990 and 2005 from 53 to 19 per 1,000 births. The maternal mortality ratio fell by two-
thirds, from 250 deaths in 1990 to 85 deaths per 100,000 births in 2007. Cases of Malaria were
significantly reduced, accounting for only 35 deaths in 2007. Tuberculosis programs, since 1997,
had reached and exceeded the global targets for control, detecting 70 percent of new smear-positive
pulmonary tuberculosis cases, curing 85 percent of these detected cases, and leading to a 44 percent
decrease of the incidence rates over the period 1997–2004. No progress was registered in the
reduction of HIV-AIDS cases and the incidence went up from 0.34 percent in 2001 to 0.44 percent
in 2005 among the general population.
4. Despite progress in health-related MDGs, the burden of disease in Vietnam, at project
appraisal, was already concentrated in non-communicable diseases, injuries, and accidents,
accounting for almost three-quarters of reported deaths nationwide. In 2008, hypertension
prevalence had almost doubled in less than 20 years and close to one-third of adults had high blood
cholesterol. Overweight, obesity, and physical inactivity prevalence increased because of changes
in nutrition and the labor market structure and smoking rates for males remained at 56 percent in
the 10 years before the Project appraisal.
5. Improvements in health system protection. Since 1992, Vietnam’s Government
intended to achieve universal health coverage (UHC) by expanding health insurance (HI)
mechanisms. Despite this intention only 49 percent of the Vietnamese population was covered by
HI at the Project appraisal. The main impacts of the expansion of HI in the 15 years before 2007
were increased utilization of health facilities, reduction of health out-of-pocket payment, and
trimming of catastrophic spending risks for families.
2
6. In 2007, Vietnam offered four types of HI for its citizens: (a) Social Health Insurance
(SHI), for those employed in the formal labor markets, retirees, and disabled and meritorious
people, representing only 9 percent of the population; (b) Health Care Funds for the Poor (HCFP),
directed to the poor ethnic minorities in mountainous areas (such as the Northern Upland
Provinces) and inhabitants in disadvantaged communities, enrolling 18 percent of the country’s
population2; (c) Free Health Care Insurance for Children under six years (FHCIC) addressing 11
percent of the population, and; (d) Voluntary Health Insurance (VHI), covering self-employed and
informal sector workers, dependents of the SHI members, and students and children over six years,
covering 11 percent of the entire population. The SHI was financed by payroll contributions while
the HCFP and FHCIC were financed by fiscal revenues. The VHI was financed by premiums
calculated according to the insureds’ ability to pay.
7. Together with the health reforms from the early 1990s to late 2000s, Vietnam
introduced changes in the provision of health services. The most important was the
liberalization of the health care and pharmaceuticals markets. This was accompanied by the
introduction of user fees at public health facilities and the transfer of health workers’ salary
payments from local communities to the central Government. Some of these changes did not
improve the health care delivery process, particularly in the disadvantage regions, given the poor
access and quality of health services in rural areas and district hospitals. Health services were
delivered by public and private hospitals, with the latter mostly providing specialized care in urban
areas, while primary and basic care were provided by public hospitals and community health
centers. A large share of pharmaceuticals was purchased directly by the population from private
vendors with traditional medicine playing a major role, as it is recognized by the Government as
part of the health system and offered by public and private providers.
8. Despite progress resulting from health reforms, coverage and health outcomes were
not uniform within the country. Poverty reduction, HI coverage, and health improvement
remained uneven, with some segments of the population lagging behind the national average and
high disparities in health indicators among regions. For example, in the Northern Upland
Provinces,3 the infant mortality rate (IMR) was 60 per 1,000 live births compared to the national
average of 18 per 1,000 live births. While the majority of births were attended by skilled health
workers, less than 20 percent of births by ethnic minority women were attended by qualified
personnel. In the Northern Upland provinces, the role of district hospitals was compromised by
the lack of adequate and qualified human resources and poor conditions of the physical
infrastructure. The poor and ethnic minorities did not use these hospitals because of poor quality
of the services and difficulty of access. Consequently, these groups were particularly vulnerable
to receiving suboptimal health care and achieving poor health outcomes, particularly mothers and
children.
9. Rationale for World Bank assistance. The Project design was based on (a) best practices
from World Bank health projects and other initiatives in countries similar to Vietnam and (b) the
government health policy to improve equity on achievement of health outcomes at the regional
2 These two insurances—SHI and HCFP—are referred to as Compulsory HI Schemes. 3 The extremely disadvantaged population of the Northern Upland Provinces led the Government to issue the
Decision 139 of 2002 to cover the following beneficiaries: poor, ethnic minorities living in the mountainous
provinces and population living in Government-defined difficult communes. The project initially limited the support
to the poor, but then expanded to the ethnic minorities following the Decision 139.
3
level. Both recommended interventions on the supply side (improving infrastructure, training
human resources, improving management, and maintenance of health infrastructure) and demand
side (providing means to the poor and ethnic minorities to gain access to quality health services).
Therefore, this Project focused on district hospitals and complemented another health project in
the NUP areas approved in June 2008 (Health Care Support to the Poor of the Northern Upland
and Central Highlands - P110251, EC Grant No. TF-091328-VN) and which focused on primary
care service delivery at the commune and village levels. The Project was consistent with the second
pillar of the Country Partnership Strategy (CPS) for Vietnam (Report No. 38236-VN) issued in
January 3, 2007, especially in the area of strengthening social inclusion, assuring economic growth
with social equity by improving social services to the poor and marginalized groups.
1.2 Original Project Development Objectives (PDO) and Key Indicators
10. According the Financing Agreement (FA), the objective of the project was to increase
the utilization of district health services, especially among the poor and ethnic minorities
population of the Northern Upland Provinces through improving the quality of district-level
hospitals and reducing financial constraints to access to health services. The PDO in the
project appraisal document (PAD) had slightly different phrasing, but its main essence was
consistent with the FA. The PDO in the PAD was as follows: Increase utilization of district health
services especially among Decision 139 beneficiaries4 in the Northern Upland Provinces through
(a) strengthening of district hospitals and (b) reducing financial constraints to accessing health
services. For the purpose of this Implementation Completion and Results Report (ICR), this PDO
is split into three parts, all especially aimed at the poor and ethnic minorities’ population of the
NUP: (a) increase utilization of district hospitals services5; (b) improve the quality of district-level
hospitals; and (c) reduce financial constraints to access health services.
11. The four PDO-level indicators (Key Performance Indicators [KPIs]) were defined in
the results framework (RF) of the PAD. The KPIs are the following: (a) utilization rates of
inpatient services in district hospitals among Decision 139 beneficiaries; (b) utilization rates of
outpatient services in district hospitals among Decision 139 beneficiaries; (c) percent of
households which experience catastrophic health care expenditures in the year prior to the survey;
and (d) proportion of district hospitals that provide full set of health services according to the
national norms (Decision 23/2005/QD-BYT) adjusted for the Northern Upland.6 As baseline data
were not available for some indicators at appraisal stage (KPI #4 and Intermediate Outcome
Indicators [IOIs] #1, #2, and #6), the targets were nevertheless set based on the team’s best guess
estimates of the expected percentage of increase/decrease during the project life. Baselines known
at appraisal were adjusted to the outcome of the Baseline Survey of 2009 (KPIs#1, #2, and #3 and
IOIs #3, #4, #5, and #7). However, these changes were not formally recorded through restructuring.
4 The Decision 139 created the HCFP in 2002 to increase access to health care and reduce the financial burden of
health expenditure faced by the poor and ethnic minorities. 5 The first part of the PDO is the overarching objective of the Project. The 2nd and 3rd parts are the means to achieve
this overall objective throughout supply and demand side interventions. 6 According to annex 3 (Project Results Framework) of the PAD, page 32. This indicator was kept but was measured
as the percent of the health services that district hospitals can provide in comparison with the national norms
(Decision 23/2005/QD-BYT).
4
1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and
reasons/justification
12. The PDO was not revised during project implementation. However, the original
baselines and targets were revised for KPIs and IOIs during project implementation (without a
formal restructuring) to reflect the outcome of the baseline survey conducted in the first semester
of 2009 and published in July 2009. Baseline values of few indicators were kept as in the original
PAD and, at the end of the project, appeared to be modest if compared with the values achieved
after implementation. See section 2.3 on monitoring and evaluation (M&E) for details.
1.4 Main Beneficiaries
13. The main project beneficiaries were meant to be the poor and ethnic minorities of the
Seven Northern Upland Provinces: Bac Kan, Cao Bang, Son La, Ha Giang, Lao Cai, Dien
Bien, and Lai Chau. These populations were to benefit from project investments by increased
access to quality health services provided by the district-level hospitals and by receiving subsidies
(meals and transportation) to improve their regular access to the district hospitals. Health
professionals and managers of the district hospitals in the NUP provinces were to benefit from
training, improved infrastructure, and additional resources from fees received from the HCFP to
increase financial sustainability of these hospitals.
1.5 Original Components
14. The project consisted of three components aimed at increasing the utilization of district
health services in the NUP provinces (supply-side interventions) and reducing the financial
constraints to accessing health services for the poor and ethnic minorities (demand-side
interventions).
Component 1: Strengthening District-level Health Services (cost estimated at USD42.9
million).
15. This component aimed to improve the quality of district hospitals through the following
three subcomponents: (a) human resources development, to strengthen the health workforce
capacity and increase staff retention, expertise, and technical capacity of district hospitals through
the provision of long-term training for doctors and specialists at level 17; (b) improving the quality
of district hospitals, through the provision of basic medical equipment and selective facility repair
and refurbishment; and (c) improving hospital management, to support the investment in district
hospitals by creating a management environment that will sustain project outcomes measured by
quality of improvement, infrastructure maintenance, and human resources management. This
component involves training district hospital management staff in basic principles of management
of health care institutions, developing hospital maintenance plans as an integral part of the
management improvement program, and creating a management excellence award program for
district hospitals that meet good management practices.
7 Level-1 specialists are doctors with the following specializations: internal medicine, obstetrics, intensive care,
traditional medicine, and imaging.
5
Component 2: Increasing Financial Access to Healthcare Services for Decision 139
Beneficiaries (cost estimated at USD10.0 million)
16. This component aimed to address the demand-side constraints of access to health services
by piloting mechanisms to further reduce financial barriers for the Decision 139 beneficiaries in
the project provinces. The component had three subcomponents. They are: (a) support for direct
catastrophic and nonmedical expenditures of health care for Decision 139 beneficiaries, aimed to
get information on the distribution and pattern of catastrophic expenditures and to support
nonmedical expenditures such as primarily transportation and food for the beneficiaries, which
had a higher proportion (62 percent) of total inpatient out-of-pocket costs in the NUP areas; (b)
strengthening capacity for HCFP, through institutional capacity-building activities to increase the
financial capacity of the district hospitals to enroll beneficiaries and recover the medical expenses
incurred by them. It involves the identification of beneficiaries according to poverty and ethnicity
criteria and issuing and distributing the beneficiaries’ cards among them; and (c) strengthening
local access to health services through promoting health seeking behavior, to increase the
knowledge and understanding of the rights, entitlements, and benefits covered by the HCFP among
the beneficiary population through information, education and communication (IEC) campaigns.
Component 3: Monitoring, Evaluation, and Project Management (Cost estimated at
USD13.1 million, of which USD6.0 million from Government counterpart)
17. This component supported the set up and management of the Central Project Management
Unit (CPMU) and Provincial Project Management Units (PPMUs) through (a) consulting services
to cover technical issues as well as procurement, financial management, and disbursement; (b)
training of project management staff; (c) provision of necessary office equipment; (d) financing of
incremental operating costs; and (d) M&E activities, including baseline data collection, indicator
updates, midterm review (MTR), end-of-project completion report and audits. This component
also supported an initiative to streamline the Ministry of Health (MoH) internal procurement
review and approval process, which was considered a critical initiative in the broader public
procurement reform efforts in the sector and the country.
1.6 Revised Components
18. The project restructuring of August 29, 2014, proposed an 18-month extension of the
project closing date and marginally reallocated the project funds among the components.
With 82 percent of the project funds disbursed, an end project evaluation was conducted from May
to June 2014 in the NUP targeted provinces, revealing that all KPIs and most of the IOIs had
already been achieved by the project before restructuring, saving 18 percent of the project funds.
The restructuring did not propose any review of the project components.
19. The Government proposed to use the remaining funds to continue activities under the
project components. No other significant changes were made to the project. The 18-month
extension was proposed to allow adequate time to: (a) improve the sustainability of the project
outcomes by continuing and adding training courses on specialized techniques and skills such as
the use and maintenance of equipment; (b) ensure that the financing and management of the HCFP
had been transferred to the provincial authorities and finance the entitled activities properly; (c)
improve the efficiency of the project throughout, providing technical assistance in health service
6
delivery, especially related to outpatient services which are essential to improve maternal and child
health and to the achievement of the MDGs in the Project areas until December 2015; (d) ensure
proper use of the equipment; and (e) provide the technical support for the development of policies
to strengthen the district-level health services and support the achievement of the MDGs.
2. Key Factors Affecting Implementation and Outcomes
2.1 Project Preparation, Design, and Quality at Entry
20. Soundness of the background analysis. The project team accessed extensive and relevant
background information about the health conditions, government initiatives, and constraints in the
national context and in the project intervention areas. The project benefited from lessons learned
in designing and implementing other health projects in Vietnam, such as the Mekong Regional
Health Support Project (MHSP) and the National Health Support Project. The project was prepared
as a specific investment loan (SIL) inspired by the design of a similar approved project —the
MHSP—which provided the project team important lessons to guide the NUP project preparation,
which was the first of four similar regional programs 8 . The main lessons learned that were
incorporated in the project design were: (a) the need to focus on the demand-side interventions and
underpin quality health coverage to increase utilization of health facilities; (b) the relevance of
addressing regional variations to adapt investments to local conditions; (c) the need to improve
project implementation skills at the local levels through technical support by the central level and
by exchange of experiences among the PPMUs; and (d) the need to build capacity (by training
staff) on World Bank fiduciary procedures and guidelines for the local implementation units, to
avoid implementation delays, particularly with regard to civil works and procurement of
equipment.
21. Assessment of the project design. The PDO, the balance of the activities among the
project components, and project implementation arrangements were formulated realistically and
in line with project complexity. The project design addressed many aspects of the expected
implementation challenges associated with one central and seven local project implementation
units, difficulties in accessing mountainous areas, scattered distribution of the beneficiary
population, and the social (ethnic minority) issues and environmental (associated mostly with the
hospital waste management issues) safeguards.
8 The MHSP (P079663) closed in June 2012 and was the first of a generation of universal coverage projects in
Vietnam that aimed at expanding coverage for the poorest population. The project indicators were achieved and
even exceeded and the project brought huge benefits to the poor, because the enrollment of the poor became a
national policy based on this experience. However, some shortcomings were observed with regard to efficiency,
such as the potential supplier-induced demand in a fee-for-service environment, leading the World Bank to open a
dialogue with the client to implement policy interventions with the Vietnam Social Security Administration to focus
on provider payment reforms. Beside the Mekong and the NUP Project, there are two more similar projects—the
Central North Health Support Project (CNHSP) and North East Red River Delta Region Health System Support
Project (NORRED)—both still in implementation. All projects provided support to the health sector in provinces
that were geographically difficult to access, economically depressed, and had interventions in both demand and
supply sides. Despite that, each project had some different features. The MHSP is for provincial hospitals, while the
NUP focuses on district-level hospitals and the CNHSP deals with district hospitals and community health centers.
The NORRED Project was designed for both provincial and district hospitals with some distinctive interventions.
7
22. Other complementary World Bank health projects and international partner
activities. The PDO did not overlap with other institutional partner support and projects, but
complemented activities supported by another World Bank project (Health Care Support to the
Poor of the Northern Upland and Central Highlands [HEMA] - P110251), which focused on
primary care in three (Son la, Lai Chau, and Dien Bien) of the seven provinces addressed by the
project. HEMA (approved in June 2008) was designed to improve access to primary health care
by the poor at community health centers (not district hospitals). HEMA was trust-funded by the
European Commission and managed by the World Bank. During project preparation, the Asian
Development Bank (ADB) was also financing activities related to the promotion and prevention
in three of the project provinces, complementing other health interventions (such as immunization,
promotion, and preventive interventions) in these provinces. All these activities enhanced the
perspective of an integral health care approach as promoted by the World Bank’s health strategy
in Vietnam and supported the idea to focus the project design on district hospitals.
23. Adequacy of Government’s commitment. The MoH authorities were completely
involved in the project preparation and in development of its design, which followed the MHSP
model. They expressed their full commitment to guarantee adequate budget for communications,
supervision, monitoring, evaluation, and travel during implementation. The provincial health
authorities and district hospital managements were also committed to participate in the project
development. Field visits were organized during preparation to establish links and networks with
the local government health and hospital’s authorities.
24. Project preparation timeline. The project was prepared in 22 months (from concept
review in May 2006 to approval in March 2008). This is considerably longer than the average for
Health Nutrition and Population Global Practice projects of 18 months, but still under the World
Bank’s benchmark for Investment Project Financing of 24 months. The FA was signed four months
after approval (July 10, 2008) and declared effective on October 8, 2008. The project was to be
implemented in six years in view of the institutional implementation conditions and challenges at
the Northern Upland Provinces.
25. Assessment of risks. The overall risk for the project was rated Substantial and mitigation
measures were appropriately described in the PAD. The major risk of not achieving the PDO was
a possible uneven implementation progress across project components. It could prevent the
positive effects of the coordination between the demand side incentives and the supply side
investments on improving health access to the poor during the project implementation. Other risks
were associated with weak procurement capacity at the provincial level and at district hospitals
and the difficulty in retaining trained medical staff at the district hospitals. The CPMU issued an
operational manual that was approved before project effectiveness. During implementation, as
indicated in the project missions, project risks were well identified and managed. Some measures
to mitigate the project risks during implementation are described in annex 6A of this ICR.
26. Quality Enhancement Review (QER) and Decision Review Meeting. The project
underwent a QER in May 2007; the decision review meeting was held in September 2007. During
both meetings, the element of the project that was mostly appreciated by the reviewers was its
good rationale and direction and a solid results framework even though baselines and targets were
to be confirmed at a later stage through the baseline survey. Some of the issues and
recommendations raised during these meetings not only revealed that it was crucial to confirm
8
quality at entry, but were eventually found to be critical during implementation. The main
recommendations to the team were to (a) provide more details on technical aspects of the project
interventions (such as incentives for human resources retention, capacity building at district
hospitals, governance of district hospitals, health promotion, implementation and financing of the
project pilots)9; (b) focus the economic analysis on direct benefits for the target population; (c)
improve the section on lessons learned; (d) improve the description of institutional arrangements
for data collection related to project M&E; and (e) reassess project risk ratings, which appeared to
be overestimated. The PAD addresses all the recommendations, except point ‘d’ on institutional
arrangements for data collection for project M&E, which was not entirely addressed.
2.2 Implementation
27. Project implementation timeline. The project was to be implemented in six years (one
year more than the usual five-year implementation period for SIL projects in the health sector),
because of the difficult institutional implementation conditions in the Project areas. The original
Project closing date of August 31, 2014, was extended by 18 months to February 29, 2016.10 Even
when the PDOs were substantially achieved, project extension was justified by the Government as
a way to use the project savings to ensure sustainability of project interventions and to contribute
to the achievement of the health MDGs in the NUP areas.
28. The July 2012 MTR confirmed that the PDO remained relevant with a satisfactory
performance. A MTR survey was conducted by a local research institute on behalf of the CPMU
from November 2011 to May 2012. Substantial progress was registered in the achievement of the
KPI and IOI targets. By the end of 2012, inpatient utilization of the Decision 139 beneficiaries
increased more than two times and outpatient utilization increased 28 percent compared with the
2009 baseline. According to the survey results, the percentage of poor households suffering
catastrophic health expenditures was reduced from 22.1 percent to 13.3 percent (almost a 9 percent
reduction). The proposed targets in these areas expected about 10 percent to 15 percent
improvement by project closing. Therefore, the project far exceeded these targets, which were
established quite modestly from a low base. The MTR survey also revealed that the district
hospitals in the Project areas were able to deliver 10 percent more services, approaching the
expected number of services to be provided by district hospitals, according the MoH’s regulations.
9 These pilots of ‘incentive packages’ would consist of a combination of different types of incentives (monetary and
non-monetary), which would be designed according to health labor market studies. It would include differentials
payments according to hardship areas, subsidies for continuing education or education costs for the health workers’
children, housing allowances schemes, fast tracked promotion through pay grades for health workers in remote
areas, increased recruitment of students from the project provinces to medical schools, and so on. Paragraph 25 of
the PAD said “Evaluation of the effectiveness of the pilots will be an important part of the project’s overall
effectiveness”. However, this activity was not implemented by the project because of its weak political feasibility in
the context of health human resources policies in the country. 10 The main reasons for project extension were (a) improving the sustainability of the project results by continuing
and adding supplemental training courses on specialized techniques and skills, use and maintenance of equipment,
and other training activities; (b) ensuring that the financing and management of the HCFP have been transferred to
the provincial authorities and are running; (c) improving the project efficiency by strengthening the technical
assistance in health service delivery with particular emphasis on delivery and outreach of essential outpatient
services to maternal and child health care related to the achievement of the MDGs; and (d) distilling lessons learned
and disseminating them.
9
29. At the MTR, delays in procurement and project disbursement led to classify the
overall project implementation as Moderately Satisfactory. In 2012, only one-third of the
credit funds were disbursed. The reasons attributed to this poor performance were: (a) the issuance
of Government Resolution 11 of 2011 limiting some capital investments (vehicles/office
equipment) even if externally funded; (b) inability to translate provincial proposals for non-
training recruitment/retrenchment options into pilot activities; (c) lack of a plan for the utilization
of the technical assistance subcomponent under Component 2 to analyze the barriers to access by
the Decision 139’s beneficiaries; (d) no decisions taken at the time of the MTR on investments
related to the disposal of solid waste treatment at the district hospitals11; (e) delay in the update of
the NUP definition of beneficiary based on the revision of Decision 139 and inclusion of “HEMA”
districts; and (f) slow development of a strategy for IEC activities. These issues were appropriately
addressed after the MTR, resulting in a small reallocation of the project proceeds to Component
1. As a result, the project improved its performance and procurement was classified as satisfactory
in the last project Implementation Status and Results Report (ISR).
30. All four KPIs’ final targets were achieved and surpassed even before the original
project closing date of August 2014. However, the development objective (DO) was rated
Moderately Satisfactory at the time of the original closing date because of internal evaluations of
the World Bank team, which assessed slow progress in the RF, given a reduction of the achieved
values of some indicators from 2012 to 201412. However, by August 2014, the project showed
achievement toward the KPIs and a significant number of IOIs compared with the targets,
including: (a) knowledge of the health workers in diagnosing and treating common illnesses; (b)
knowledge of the benefits of HI by the beneficiaries; and (c) availability of facility and equipment
maintenance plans.
31. The final ISR, of February 25, 2016, rated the DO Satisfactory and the
implementation progress Moderately Satisfactory. All major planned activities were completed
by the closing date. Increased utilization of the district-level health services in the NUP region by
the beneficiaries was noted. The four KPIs used to measure the achievement of the PDO in the
seven provinces were fully achieved, as follows: (a) average inpatient visits per capita in the district
hospitals increased 3.9 times over the baseline and was 2.9 times greater than the end project target
for 2014; (b) average outpatient visits per capita increased 3.7 times over the baseline and was 28
percent greater than the end of project target for 2014; (c) percentage of households experiencing
catastrophic health expenditures was reduced from 14.7 percent in 2008 (baseline) to 2.0 percent
(2014), overachieving the target of 13.2 percent for 2014; and (d) average percentage of health
package procedures covered by the district hospitals according the national norms increased from
39 percent (2008) to 80.4 percent (2015), surpassing the target of 70 percent (2014).
11 In fact, the MoH/CPMU requested to buy the incinerators for solid waste. However, due to the World Bank’s new
regulation not allowing using IDA credit for burning technology, the activities were delayed and then cancelled at
the World Bank’s request. 12 See the Project ISR Sequence 7, issued in May 2014. According to this ISR, “the outpatient utilization increased
to 0.082 from 0.067 visits per capita (a 22 percent increase against the end-of-project target of 10 percent increase);
this was a slight (5 percent) decrease from 2012 of 0.086. In August 31, 2014, the district hospitals supported by the
project could provide 71 percent of the services required according to the national standards, compared with 52
percent at the baseline.”
10
32. Project contribution was crucial in supporting the PDO progress toward the MDGs
on maternal and child mortality. During project implementation, all health workers and
traditional birth attendants of the seven provinces were trained and received technical advice and
monitoring from the MoH’s Department of Maternal and Child Care. All the provinces and district
hospitals were provided with training materials and equipment to improve the performance of
neonatal care units. The hospitals also received clean delivery kits to help improve hospitals’
performance and home-based birth delivery. By December 2015, the Project undertook an
evaluation of maternal and neonatal mortality in the Northern Upland provinces, which indicated
that the Project contributed to the improvement of maternal and child health. The IMR, neonatal
mortality rate (NMR), and MMR decreased from 31.1 per 1,000, 11.2 per 1,000, and 178 per
100,000 in 2007–2008 to 29.4, 10.8, and 106 in 2013–2014, respectively. Data for 2015 was not
processed at the time of this ICR.
33. CPMU commitment and capacity. The CPMU was established before project
effectiveness and was always committed to improve the project performance and achieve its goals.
The CPMU built strong and systematic capacity and was essential in establishing the operational
framework of PPMUs, necessary to implement the project in each of the seven Northern Upland
Provinces. The CPMU was in charge of launching and operating project management systems,
including human resources, fiduciary functions, offices, and equipment throughout the
implementation of the project. During project implementation, the CPMU and the PPMUs
prepared the project’s annual plans, monitored annual implementation targets, and provided timely
interim financial and activity reports and audits. Exception should be made to some reports on
project result indicators toward the last year of project implementation because of some staff
shortness from both CPMU and PPMUs.
2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization
34. Design. Despite the extended time to prepare the project, the Government and the World
Bank agreed that the baseline survey and the completion of the RF would be completed during the
beginning of Project implementation as part of the Project’s Component 3, activity (v)13. Some
baselines and targets for the KPIs and IOIs set in the PAD (annex 3) were to be confirmed or
changed after the 2009 Baseline Survey. The poor quality of the health information system (HIS)
in the country, provinces, and district hospitals at the time of project preparation established
barriers to confirm upfront KPI and IOI baselines and therefore targets. For this reason, the
Government and the World Bank team opted for a baseline survey to be launched after project
effectiveness. The RF would be monitored and evaluated by comparing data from the baseline
survey with the results of a midterm evaluation survey and a project-end survey. The baseline
survey was part of the project design, as indicated in the PAD.
35. The four KPIs were linked to the three parts of the project PDO14 - KPI #1 (increase
utilization rates of inpatient services in district hospitals among Decision 139 beneficiaries) and
KPI #2 (increase utilization rates of outpatient services in district hospitals among Decision 139
beneficiaries)15 were linked with the PDO part 1, increase utilization of district health services;
13 See page 5 of the PAD. 14 See paragraph 10 of this ICR. 15 The baseline inpatient and outpatient rates of district hospitals, apparently low, have to be seen as part of a global
inpatient and outpatient rates that include other health services, such as community health centers and community
11
KPI #4 (proportion of district hospitals that provide full set of health services according to the
national norms) was linked with the PDO part 2, improving the quality of district-level hospitals,
and KPI #3 was linked to PDO part 3, reduce financial constraints to access health services16.
36. The 13 IOIs were fully aligned with the project components. Component 1 (Strengthening
District-level Health Services) was evaluated by six IOIs, addressing the following dimensions:
patient satisfaction (IOI #1), adherence to treatment protocols (IOI #2), human resources training
(IOI #3); maintenance plans and budget for facility and equipment maintenance (IOI #4); health
facilities constructed, renovated, and equipped (IOI #5)17; recent discharged patients (IOI #6).
Component 2 (Increasing Financial Access to Healthcare Services for Decision 139 Beneficiaries)
was monitored by four IOIs addressing: people with access to a basic package of health, nutrition,
and reproductive health services (IOI #7)18; percentage of Decision 139 beneficiaries who had
received health insurance cards (HICs) (IOI #8); percentage of Decision 139 beneficiaries with
cards, who can correctly identify at least three benefits covered under the HCFP program (IOI #9)
and households identified with financial barriers for not seeking health care (IOI #10). Component
3 (Monitoring, Evaluation, and Project Management) was monitored by the following tasks:
project management units at the central and provincial level established and maintained, bank
accounts opened, staff and consultants recruited and systems established (IOI #11); availability of
data for project M&E in years three and six (IOI #12); and project management units provided
with adequate plans, annual implementation targets met, and timely financial and activity reporting
(IOI #13).
37. Implementation. During the Baseline Survey in 2009, the Government and the World
Bank team considered that, given operational reasons, KPI #4 was adjusted to measure “the
average percentage of health services provided by the district hospitals.” Baseline values were
available for all KPIs after project approval. The baselines for KPI #119 and #2 were calculated as
of June 2009, while baseline values for KPI #3 (from VHLSS) and #4 (from hospital administrative
records) were calculated as of June 2008. The 2009 Baseline Survey conducted during the first
semester of 2009 provided baseline values for the NUP and HEMA Projects.
38. Baseline values for some IOI’s received special treatment during project
implementation. IOIs #1, #2, and #5 to #10 baselines were calculated as of June 2009 while
baselines for IOI # 3 and #4 were calculated as of June 2008. No baselines were established to the
three IOIs related to Component 3 (IOIs #11, #12, and #13) because they were process indicators.
Given their complexity, IOIs #2, #3, and #4 were split into several sub-indicators to be
health stations (for outpatients) and provincial hospitals, national hospitals, and private hospitals for inpatient rates.
According to the Vietnamese Household Living Standard Surveys (VHLSS) 2014, in the Northern Upland
Provinces, the number of total inpatient visits per inhabitant was 0.116 (0.063 in district hospitals) and the number
of outpatients visits was 0.462 per inhabitant (0.074 in district hospitals). 16 Baselines and follow up data for this indicator was obtained from household surveys (VHLSS). 17 IOIs #5 and #6 did not appear in the PAD and were added during project implementation. 18 IOI #7 did not appear in the PAD and was added during project implementation. However, it cannot be measured
because the baseline and target (expressed in percentage) were not converted by the number of beneficiaries as
referred to in the title of the indicator. This indicator is therefore not a measure for the purpose of this ICR.
However, by December 2015, the number of beneficiaries reached was 270,300. 19 Baseline for KPIs #1 and #3 and for IOI #9 were initially established in annex 3 of the PAD as based in the
VHLSS 2004. However, the baseline of these indicators was revised to June 30, 2009 (KPI #1 and IOI #9, using the
baseline survey) and to June 2008 (KPI #3, using the VHLSS 2008), respectively.
12
appropriately measured. IOI #2 (adherence to treatment protocols) were split into nine sub-
indicators of three relevant health conditions affecting population in the NUP areas—diarrhea,
pneumonia, and poisoning20. A similar process was used to measure IOI #3 (percent of eligible
district health staff who have successfully completed training provided by the project), measured
by three sub-indicators21, and IOI #4 (proportion of district hospitals with acceptable operations
and maintenance plans and budget for facility and equipment maintenance) measured by two sub-
indicators22. All baselines for the IOI #2 to IOI #4 sub-indicators were calculated as of June 2009.
39. Supplemental M&E indicators. To improve the monitoring of project achievements, the
CPMU created and monitored additional indicators that were not part of the PAD and legal
document (Supplemental Letter #2), but were reported in project ISRs. These indicators were
recently discharged patients (IOI #6); number of health facilities constructed, renovated, and/or
equipped (IOI #5), and; number of inpatient beneficiaries with access to a basic package of health,
nutrition, or reproductive health services (IOI #7). The latter was crucial to support the
achievement of the health MDGs in the NUP areas upon project closing. However, no baseline
was established for these indicators.
40. MDG’s health indicators (MHIs) monitored after project extension and measured by
the Government. The level II restructuring of August 2014 to extend the project by 18 months
did not use that opportunity to formally revise the RF and establish the baselines and targets from
the 2009 Baseline Survey. Considering the need to strengthen the capacity to improve health
MDGs by the district hospitals in the NUP areas, some additional indicators (not followed by the
World Bank) were followed by the CPMU and the PPMUs to compare positive variations between
June 2009 and December 2015. These indicators were the following: percentage of district
hospitals providing caesarean section surgeries (MHI #1); percentage of district hospitals
providing blood transfusion (MHI #2); percentage of district hospitals having continuous positive
airway pressure systems (MHI #3); percentage of district hospitals having light for jaundice
phototherapy treatment (MHI #4); percentage of district hospitals having oxygen breathing
systems (MHI #5); and percentage of district hospitals having newborn resuscitator (MHI #6).
Several other indicators were part of this M&E block, but without baselines.
41. Final targets for the project RF’s indicators. All KPIs and IOIs (except IOIs #11 to #13,
which required yearly monitoring) considered August 31, 2014 as the final target date. By
agreement between the Government and the World Bank, the CPMU did not extend the final target
indictors as part of the project restructuring of August 2014, but as mentioned before, the project
missed the opportunity to formally revise the 2009 baselines and update targets to 2015. Even
keeping the original date for the final targets, some of the project indicators were measured by
20The description of the 9 sub-indicators could be found in Annex 5. According the provincial health authorities,
these three conditions represented the majority of the demand for health care in district hospitals. 21 The sub-indicators are (a) percentage of doctors and assistant doctors at district hospitals trained by the project;
(b) percentage of health staffs who completed short-term training courses compared to the plan and; (c) percentage
of health staffs who completed long-term training courses compared to the plan. 22 The sub-indicators are (a) proportion of district hospitals with acceptable operations and maintenance plans and
budget for facility maintenance and (b) proportion of district hospitals with acceptable operations and maintenance
plans and budget for equipment maintenance.
13
December 31, 2015 by the independent evaluation of the project, conducted by the CPMU during
the project extension period. These indicators are KPIs #1, #2, and #4; IOI #3, #4, #5, and #8.
42. Intermediate targets and M&E processes. Given the hard conditions to obtain health
information associated with the NUP areas and district hospitals, the project established its own
administrative records at the PPMUs and associated its evaluations to midterm and final surveys
with the same methodology used in the 2009 Baseline Survey. The current Government’s surveys
are improving, but they still could not be used as the main source of information for the project’s
M&E processes because of lack of reliability and availability of the data, especially for project
areas where the sample surveys are not representative. During the project life, three surveys
associated to the M&E process were done: for the baseline (first semester of 2009); for the MTR
(November 2011–May 2012), and at the project original closing date (before the extension, May–
June 2014). Intermediate targets are measured only once (May 2012) and final targets in June 2014.
District hospitals’ administrative records and eventual regional surveys provided additional
information that was recorded in the project’s ISRs in the interim.
43. Utilization of M&E data. Given that district hospitals’ HIS had weak capacity in the first
years of the project implementation, the utilization of M&E data by the CPMU and PPMUs was
limited during part of the project life. Some difficulties remained in the use of definitions and in
the calculation of some project indicators by the PPMUs and therefore building the aggregated
data for some indicators was delayed. As a result, many data and information were not recorded
timely and accurately, leading to delays in the submission of M&E reports. The MTR evaluation
was useful to refocus the project, in the needed areas, with more support, such as providing
technical assistance of qualified consultants to support the provinces to improve data reporting and
accurate calculation of the project indicators. At the project end, the RF was reported regularly by
the consultants, creating the database for the final project evaluation.
2.4 Safeguard and Fiduciary Compliance
44. There were two types of safeguard policies triggered by the project: social and
environmental. The World Bank’s safeguard policy on Indigenous People (OP/BP 4.10) was
triggered given that the majority of project beneficiaries were ethnic minority groups. Since
its design, the project was expected to have positive impacts for these beneficiaries, which
represented 82 percent of the NUP population. As a result, the project increased demand and
utilization of health services by the poor and ethnic groups, with positive impacts on reducing the
financial burden on health care for beneficiary households through increasing coverage and
strengthening the implementation of the HCFP. The project execution was considered satisfactory
on implementing the indigenous people policy, given that ethnic minorities were part of the project
primary target population.
45. The project hospitals achieved remarkable improvements in the waste management
process compared with the baseline (2007), when most of them did not comply with health
care waste management regulations. The project design included the development of a health
care waste management (HCWM) plan to provide district hospitals with proper disposal of the
medical wastes. It included the preparation of specific HCWM plans for each district hospital
under the project; training and use of IEC materials on HCWM for district hospital staff;
procurement of HCWM equipment and supplies; and; the introduction of wastewater treatment
14
facilities at district hospitals. The implementation of the HCWM plans experienced some delays
during the project life and was classified in the last ISR as moderately satisfactory, because of
difficulties encountered in the procurement of equipment and corresponding staff training and
changes in the World Bank’s policy on the procurement of some HCWM that were included in the
original procurement plans, such as incinerators.
46. By the end of the project, most of the district hospitals had a well-prepared HCWM
plan and monitoring program. Out of the 65 project hospitals, 59 have trained their HCWM
officers. All project hospitals fully complied with the regulations on waste segregation and
collection. Regarding waste storage, 45 percent of project hospitals fully complied and 51 percent
of project hospitals partly complied with the related regulation. By project closing, around 60
percent of project hospitals were treating their hazardous waste and 38 percent of project hospitals
were treating wastewater in a proper manner. Because none of the hospitals had relevant
experience in waste management processes at project onset, the project was quite successful in
that regard and in the CPMU’s ability to make sure that all district hospitals would continue
implementing HCWM plans after project closing.
47. The project provided good solutions and implementation of waste management
processes at district hospital level. Basic training on regulations related to health care waste
management and nosocomial infection control were provided for the district hospital managers
and the staff. The CPMU and PPMUs received guidance for planning and implementing measures
for health care waste management. Consumables and equipment for health care waste management
were procured by the PPMUs and distributed to the project district hospitals. The availability and
proper use of waste containers, transportation, and cooling devices resulted in significant
improvements in health care waste separation, collection, storage, and final disposal in district
hospitals.
48. Financial management. The financial management and counterpart’s fund commitment
performance moved from satisfactory, in the first years of project implementation to moderately
satisfactory in the last three years of implementation because of some delays in the audit processes.
The CPMU and PPMUs were appropriately staffed during the great part of project implementation.
Financial reports were delivered with satisfactory quality and audit reports were provided, but with
some delays. Despite the fact that project implementation went faster than project disbursement,
the 18-month project extension lead to the use of 97 percent of the loan funds (USD58.4 million).
About USD1 million equivalent of the project funds were unused and returned to the World Bank.
49. Procurement. During the major part of the project life, procurement performance was
rated as moderately satisfactory because of some delays in the delivery of procurement packages,
but it was improved during project implementation. The Government and the World Bank were
proactive in solving many of the problems and in reviewing and adjusting procurement plans. Most
of the packages were implemented on schedule or slightly behind schedule. Post review of
procurement found no major deviation or non-compliance. At closing, procurement plans were
respected with no complaints and progress was rated as satisfactory.
15
2.5 Post-completion Operation/Next Phase
50. Project sustainability is likely, by the commitment of provincial government’s
budgets, to finance transport and meals subsidies for the poor/ethnic minorities to access
health care services at district hospitals, maintain the equipment, and retain skilled human
resources in district hospitals. The MoH issued national norms to create Provincial Health Funds,
establishing incentives to keep health care providers in the mountainous provinces. MoH Decision
38, issued in 2012, allows the use of district hospital revenues to maintain and buy new equipment.
In addition, Decision 14, also issued in 2012 by the Prime Minister, allows the Provinces to use
their fiscal revenues to support transportation and meals to the poor and near poor for their visit to
district hospitals. Both government’s decisions were taken as the positive outcomes of the NUP
project23. Four of the seven provinces have established provincial HCFP to finance meals and
transportation for the poor and ethnic minorities’ medical visits. The provinces which have not yet
established HCFP - Dien Bien, Bac Kan, and Cao Bang - are in the process of doing so with support
from the MoH. However, during the project extension phase, between 2014 and 2015, the number
of district hospitals with budgets to maintain infrastructure and equipment had a slight reduction
by 20 percent and 10 percent, respectively.
3. Assessment of Outcomes
3.1 Relevance of Objectives, Design, and Implementation
Rating: Substantial
51. Relevance of Objectives: High. The project was fully aligned with Vietnam’s CPS 2007–
2011 (Report 38236-VN) and with Vietnam’s CPS 2012–2016 (Report 85986-VN) in terms of
priorities for the health sector and also with the country’s future health strategies. The project was
essential to support the country’s health sector along with its transition from low- (IDA) to middle-
income country (IBRD), improving quality coverage for essential district hospital services to the
poor and ethnic minorities in the Northern Upland Provinces, especially to mothers and children.
The project was also a priority of the MoH and provincial government by its relevance to reduce
health coverage inequities and strengthen district hospitals’ performance, contributing to attending
the health needs of the mountainous areas under the Vietnamese Health Reform priorities. The
project was consistent with the country’s Health Sector Development Plan (2011–2015) and the
Health Sector Strategy, with a vision to 2020. The PDO remains relevant now and in the coming
years. In 2016, the Vietnamese Government and the World Bank launched the Vietnam 2035
Agenda, which is completely in line with the project objectives, especially the objective of
increasing quality health services access to the poor24.
23 Center for Environmental and Health Studies. 2014. Report on the Final Evaluation of the Northern Uplands
Health Support Project, Hanoi, July 2014. 24 The World Bank and the Ministry of Planning and Investment of Vietnam. 2016. Vietnam 2035: Toward
Prosperity, Creativity, Equity and Democracy, Hanoi, February 2016. According this document ‘the major policy
challenge facing Vietnam’s health system over the next 20 years will be to achieve universal health coverage that is,
to ensure that everyone has access to high quality services without suffering financial hardship. The objective of
universal health coverage is closely linked to Vietnam’s overall equity agenda, both in ensuring access to services to
promote social inclusion and in reducing poverty due to out-of-pocket payments for health care’.
16
52. Relevance of Design: Substantial. The project design drew from projects in other poor
regions of the country (such as MHSP, CNHSP, and NORRED) aimed at ensuring equity in the
coverage and access to health services for the poor and ethnic minorities. The project design was
partially based from the MHSP design. It also was influential in developing the design of the
CNHSP and NORRED projects—both still under implementation. The project design
appropriately addressed both demand and supply sides to improve coverage and access to health
services provided by district hospitals in the mountainous areas.
3.2 Achievement of Project Development Objectives
Rating: High
53. The overall efficacy rating is the result of the assessment of the achievement of the
indicators associated with the three parts of the PDO, which are rated High (part 1), High (part
2), and Substantial (part 3) (See annex 5, Analysis of the PDO Achievement).
54. Achievement of the project PDO part 1: High. The PDO part 1, related to the supply-
side interventions, was increased utilization of district health services (in the project case, district
hospitals). The utilization level of district hospitals could be measured majorly by two indicators:
inpatient utilization rates (KPI #1) and outpatient utilization rates (KPI #2). The targets for these
two KPIs were significantly surpassed as the project created effective access to district hospitals
services for the poor and ethnic minorities. Targets were exceeded by a set of health services that
were broader than those originally defined as the project target.
55. Achievement of the project PDO part 2: High. The PDO part 2, related also to the supply
side, was improving the quality of district-level hospitals. This part of the PDO was measured by
the proportion of district hospitals that provide the full set of health services according the national
norms (KPI #4), which was surpassed, and seven IOIs (#1 to #7, six surpassed and one achieved).
These IOIs addressed quality aspects of health care at district hospitals, such as client satisfaction,
adherence of treatment protocols for select health conditions, and interventions for inpatients;
training of district hospitals staff on maintenance schedule, and budget allocation for hospital
infrastructure and equipment; construction/renovation and equipment of health facilities; and
patients discharged.
56. Achievement of the project PDO part 3: Substantial. The PDO part 3, related to the
demand side, was reducing financial constraints to access health services. This part of the PDO is
measured by the percentage of households experiencing catastrophic health care expenditures in
the year before the survey (KPI #3), which was surpassed, and three IOIs: #8 and #10 (surpassed)
and #9 (not achieved)25. These IOIs addressed aspects such as the possession of HICs by the
beneficiaries, their knowledge about how to use/access the HI benefits and the percentage of
households identifying financial barriers as the main cause for not seeking health care.
Achievement of this part of the PDO is considered substantial according the methodology used to
rate the PDO’s efficacy. IOI #9 (percentage of Decision 139’s beneficiaries with cards who can
correctly identify at least three benefits covered under the HCFP program) was not achieved
25 The CPMU manifested to the World Bank team, after the MTR that the achievement of the IOI #9 should be
ambitious, but did not suggest to change the indicator or its targets during the project implementation.
17
according to the criteria used to evaluate PDOs26. One possible reason, among others, for this
failure may be the need for more appropriate IEC activities to increase the understanding of the
HIC benefits to the ethnic minorities.
Table 1. Achievement of Project Indicators
Target Achievement Status KPI IOI
IOI
Sub-
indicators
Total
Target surpassed 4 9 12 25
Target achieved or substantially achieved (>=85
percent met) 0 2 2 4
Target partially achieved (65%–84% met) 0 0 0 0
Target not achieved (<65% met) 0 1 0 1
Not considered 0 1 0 1
Total 4 13 14 31
% surpassed and achieved/substantially achieved 100 80 100 94
Source: ICR Datasheet, pages iii–ix. This table does not include progresses in the MHI because they are not part of
the PAD or the legal agreement and neither had established targets.
57. An overall analysis of the 31 project indicators shows that 94 percent of the project
indicators were surpassed, achieved, or substantially achieved by the end of project
implementation. As can be seen in table 1, the project had 4 KPI’s, 13 IOI’s (three of them split
into sub-indicators) and 14 sub-indicators used to measure three IOI’s (#2, #3, and #4). Table 1
shows the level of achievements of project indicators. Part of this good performance could be
explained by modest targets attributed to the KPIs and IOIs at the design stage of the project RF.
The lack of information to foresee how the project investments should affect the district hospital
performance made it difficult for the Government and the World Bank to increase the project risk
by using challenging targets for the project indicators. During the project restructuring of 201427,
both— Government and the World Bank—should have formally agreed on setting more ambitious
targets for some of the KPIs and IOIs for the following 18-month extension, based on the
accomplishments verified by the 2014 end-of-project survey. However, this was not done because
many original project activities were not continued during the extended period and were replaced
by new training activities and the provision of equipment/consumables for maternal and child
health care for achieving the MDGs.
58. The project showed excellent performance on the achievement of the RF targets.
There are two reasons for this: first, the uncertainty about the results of the project investments
during the project design and beginning of implementation led the Government and the project
team to be cautious about the KPIs and IOIs targets. Second, as it happens in other international
contexts (see the efficiency analysis section of this ICR), projects that drive the investments to
26 See annex 5, paragraphs 2 and 3 and annex 5A. 27 The project had problems that should lead to require an earlier project restructuring, such as adjustments in the
indicators baselines and others. However, the decision process to do a project restructuring in Vietnam is complex. It
requires six months of anticipation and needs to be approved by the country’s prime minister. The MoH did not
considered adjustment in the project indicators a relevant reason to require a project restructuring. Only subjects
considered more relevant, such as a project extension, should be reasons to lead the sectoral ministers to submit to
the prime minister a project restructuring in Vietnam.
18
simultaneously solve supply- and demand-side constraints are proven to be more efficient in
achieving good performance in outputs and outcomes.
59. Assessment of achievement of the project activities and outputs. Another way to verify
the PDO’s achievements is evaluating the completion of the project outputs by activity,
subcomponent, and component (see annex 2). According to the Project Output Map (annex 2A),
the project was structured in three components, six subcomponents, 19 activities, and 29 outputs.
Two outputs were not in the PAD, but were added during project implementation (long-term
training for pharmacists and short-term training for health staff on MDG-related activities). The
other 27 outputs could be split into two groups: 10 with quantitative results and 17 with qualitative
results. For the first group, the same rating categories could be used to classify the KPIs and IOIs
in the PDO analysis (surpassed, achieved, substantially achieved, partially achieved, and not
achieved)28. Assessment for the second group is based on the ISR records. This exercise allowed
the ICR team to confirm that 9 out of 10 outputs in the first group were surpassed and one was
substantially achieved. These outputs are related to long-term and short-term training activities,
techniques and skills transfers, medical equipment provision and facilities
construction/improvement, management training, and hospital maintenance plans. In the second
group of 17 outputs, 9 were achieved, 7 were partially achieved, and 1 not achieved and cancelled
(the issuance of a management excellence award program), because the MoH created a systematic
process to award health services and personnel with excellent performance which was not related
to the NUP. This system has not been followed up by the CPMU and the project. The achievements
(full or partial) are related to support capacity improvement of the HCFP, implementation of
studies on the benefits’ incidence of the program, provision of office equipment to PPMUs, and
others.
60. Substantial progress has been verified in KPIs and IOIs during the 18-month project
extension. Table 2 shows the achievements during the project extension period: three KPIs, two
IOIs, and a few MHI indicators were achieved. At project closing, three of the four KPIs were
measured and were found to have shown improvements. Short- and long-term training for health
staff at the district hospitals level have also improved with increases from 20 percent to 115 percent.
Some problems were identified with regard to the health budgets and the maintenance and
operations of district hospitals’ infrastructure and equipment, which could have indicated
commitment toward sustainability of project interventions. Substantial progress could be
registered in the health MDG indicators, especially in the reduction of maternal mortality, given
that it was one of the main reasons used by the Government to request the 18-month extension of
the closing date.
Table 2. Indicators Achievements during the Project Extension Period
Indicator Baseline
(2009)
End
Project
Survey
(2014)
Extension
Phase
(2015)
Percent
Increase
During the
Project
extension
period
(2014–2015)
KPIs
28 See description of the indicators rating in paragraph 2 of annex 5.
19
Indicator Baseline
(2009)
End
Project
Survey
(2014)
Extension
Phase
(2015)
Percent
Increase
During the
Project
extension
period
(2014–2015)
#1 - Increase utilization rates of inpatient services in
district hospitals among Decision 139 beneficiaries (per
100 inhabitant)
0.025 0.085 0.096 13
#2 - Increase utilization rates of outpatient services in
district hospitals among Decision 139 beneficiaries (per
100 inhabitant)
0.067 0.206 0.247 20
#4 - Proportion of district hospitals that provide full set
of health services according to the national norms (%)
39.1 71,4 80,4 13
IOIs
#2.1 - Percentage of health staffs who completed short-
term training courses compared to the plan
0 357.3 427.3 20
#2.2 - Percentage of health staffs who completed long-
term training courses (specialty level-1 doctors)
compared to the plan
0 88.0 189.2 115
#3.3 - Total number of health staffs trained in short-term
and long-term courses
0 8,929 11,868 33
#4.1 - Percentage of district hospitals with acceptable
operations and maintenance plans and budget for facility
maintenance
49.2 99.3 85.9 −13
#4.2 - Percentage of district hospitals with acceptable
operations and maintenance plans and budget for
equipment maintenance
77.1 99.2 93.8 −5
MHIs
#1 - Percentage of district hospitals providing caesarean
section surgeries 74.6 81.2 92.1 13
#2 - Percentage of district hospitals providing blood
transfusion
56.7 67.2 82.8 23
#3 - Percentage of district hospitals having CPAP
devices 25.0
75.0 79.7 6
#4 - Percentage of district hospitals having light for
jaundice phototherapy treatment 21.4 82.1 87.5
7
#5 - Percentage of district hospitals having oxygen
breathing system
42.9
75.0
78.1
4
#6 - Percentage of district hospitals having newborn
resuscitator
46.4 92.9 92.2 −1
Note: CPAP = Continuous Positive Airway Pressure.
3.3 Efficiency
Rating: Substantial
61. A quantitative cost-benefit analysis was not included in the PAD because there was
no empirical basis for estimating the project’s impact on health outcomes at that time.
However, project investments improved the coverage and quality of health services,
especially among the poor and ethnic minorities population of the NUP, increasing the
efficiency in both the supply and demand side of health care. Annex 3 presents the project’s
economic analysis based on (a) rationale of the government investment in the region; (b) the
20
efficiency on achieving quality access for district hospitals by the NUP population (project
outputs); (c) the project contribution to improve health benefits (project outcomes) and; (d) the
impact of the project interventions in the equity on health access and health spending in benefit of
the poor and ethnic minorities. On the supply side, the project effectiveness was associated with
improvements in the outputs of district-level health services by providing medical equipment and
better trained personnel. On the demand side, effectiveness was associated with increases in the
utilization of cost-effective health care by poor and ethnic minorities, whose previous levels of
health care utilization were considerably lower than those of other social groups in the country.
62. The Government’s choice to invest in this region was based on the unfavorable
socioeconomic conditions and the difficult access and generally poor quality of health
services, especially at district health hospitals. The Northern Upland is the region with the
highest concentration of ethnic minorities, which is also part of the Decision 139’s beneficiaries
targeted by the project. The central Government sought to improve the health status of the
population and reduce the health gap between the NUP provinces and the rest of the Vietnamese
population. The project achieved this by improving the efficiency on delivering health outputs,
reducing maternal mortality in higher proportion than the country’s average, and reducing the
equity gap in out-of-pocket health spending between the poorest and the richest income quintiles.
63. The main conclusions of the economic analysis are the following: (a) the project was
efficient in delivering its outputs, reducing costs for training, equipment installation, and civil
works. This led to the savings being used, during the project extension period, for new activities
related to the MDGs in the project area and (b) the project interventions and subsidies to the poor
and ethnic minorities were efficient by contributing to reducing maternal, neonatal, and infant
mortality and by improving the equity on assessing health care and reducing health spending to
poor and ethnic minorities.
64. The project has significantly contributed to the improvement of equitable access of
health care in the NUP areas. Project benefits resulted in health improvements of the poor and
ethnic minorities as they represent a high share of the Northern Upland populations in addition to
the project demand-side subsidies targeted to the poor and ethnic minorities.
Allocative efficiency
65. Improving access to mother and child health interventions at the community-level
hospitals is recognized by literature as a highly cost-effective investment. Most of the health
interventions provided by district hospitals are focused on reducing maternal, newborn, and child
morbidity and mortality. A recent Diseases Control Priorities Program Third Edition publication29
shows high economic return rates for interventions aimed at increasing coverage of services where
good evidence exists for demand-side interventions to motivate service uptake. Following this
evidence, the project interventions contributed to the reduction of infant and maternal mortality
29 Black, Robert E., Ramanan Laxminarayan, Marleen Temmerman, and Neff Walker, Editors. 2016., Reproductive,
Maternal, Newborn, and Child Health, Disease Control Priorities, Third Edition (Volume 2), Ed. Washington (DC):
The International Bank for Reconstruction and Development/World Bank; Apr 5, 2016. ISBN-13: 978-1-4648-
0348-2ISBN-13: 978-1-4648-0368-0.
21
rates in the NUP regions, which were reduced from 31.1 to 29.4 per 1,000 births alive and from
178 to 106 per 100,000 births alive between 2008 and 2014, respectively.30
66. Improving health infrastructure and skilled professional attendance at district
hospitals were identified as the main factors to remove barriers to health service availability
for women and children in the poor mountainous areas of the Northern Upland provinces.
Therefore, closing the gap in health care access required targeting resources on the identified
barriers and on ethnic minority’s populations. The return on infrastructure investments has clear
long-term benefits, which facilitate improvements in the provision of quality services. In addition
to infrastructure investments, the project funded some important and highly cost-effective demand-
side interventions under Component 2.
Efficiency of project preparation and implementation
67. No major delays were registered during project preparation. The project was prepared
in 22 months, which is slightly longer than the average for health projects in the World Bank, by
4 months, and longer than the average time for IPF preparation of 18 months for all sectors.
However, considering that no previous experience and information were available to inform the
project design at the concept stage, several documents were elaborated or assessed during project
preparation, which required substantial time from the project team and country staff31.
68. The PDO was achieved within the originally planned period. The project’s
implementing efficiency is associated with the fact that, by July 2014, the project had substantially
accomplished the KPIs and IOIs using only 82 percent of the credit amount. Given the need to
improve MDGs’ outcomes and consolidate the project beneficiary gains, the Government
proposed an 18-month extension to use the remaining funds. By its closing in February 2016, the
94% of the project indicators (KPIs and IOIs) were achieved and surpassed and 97 percent of the
credit proceeds had been disbursed.
69. The efficiency associated with the project closing date extension period could be
considered moderate. The investments in training and equipment to improve mother and child
care (see table 2) were appropriate and contributed to further achieve project outcomes. However,
the provincial budget’s consolidation to guarantee the maintenance of health infrastructure was
30 The impact on maternal mortality reduction is directly associated with the quality of birth attendance provided by
better access and quality birth delivered at the district hospitals. The impact on infant mortality appears to be more
modest, because it should be associated to other non-hospital and nutritional interventions at the community levels. 31 The main reports produced to feed the project preparation are: Health status in the seven provinces of Northern
Upland; Health care service use and accessibility status in the seven provinces of Northern Upland: Son La, Dien
Bien, Lai Chau, Cao Bang, Bac Can; Health Human Resource Analysis in the seven provinces of Northern Upland;
Health care for the Poor: Identification of the needs and proposal of investment for capacity building and
management capacity development; Assessment of the HCFP in the seven provinces of Northern Upland; Health
care for the poor: management according to the Decision 139 in seven provinces of Northern Upland (most difficult
provinces); Health System Assessment for seven provinces of Northern Upland; Inventory of medical equipment in
hospitals of seven provinces of Northern Upland; Output indicators after analysis; Socioeconomic, Demographic,
Cultural Geographic and Health Status indicators: Morbidity, Mortality, Crude Death Rate/IMR, Under 5-child
nutrition status; List of medical staff to be trained and; Cost estimate and Cost Table for training component.
22
slightly reduced from 99.3 percent in 2014 to 79.7 percent in 2015, indicating that efforts may be
needed to ensure proper maintenance of project investments32.
Fiscal impact and sustainability
70. The fiscal impact of the project was considerable at the provincial and district
hospitals’ level. The project had significant fiscal impact in the additional recurrent expenditures
needed to maintain project investments in infrastructure and equipment of district hospitals,
enabling these hospitals to receive funds from the HCFP. For example, from 2008 to 2013, the
average revenue of the 28 district hospitals supported by the project increased by 336 percent.33
On the other hand, the MoH supported the creation of the Provincial Health Funds, establishing
the financial basis to support the recurrent costs of district hospitals in the future, guaranteeing the
sustainability of project investments beyond the project life with a significant fiscal impact.
Vietnam’s rapid economic growth, expected over the coming years, will ensure that provincial
health budgets and district hospitals sustainability are likely to continue to grow at a high rate.
3.4 Justification of Overall Outcome Rating
71. The overall outcome rating is Satisfactory. This rating takes into account the project
remained relevant throughout implementation and beyond and the activities financed by the project
being efficient in providing supply- and demand-side interventions to improve quality access to
health care at the district hospitals. In addition, the overall achievement against the PDOs is
considered substantial.
Table 3. Summary of Outcome Ratings
Project Outcome Ratings
Relevance Substantial
Efficacy High
Efficiency Substantial
Overall Outcome Rating Satisfactory
3.5 Overarching Themes, Other Outcomes and Impact
(a) Poverty Impacts, Gender Aspects, and Social Development
72. The project specifically targeted the poorest mountainous regions, home of ethnic
minorities and isolated communities, and focused on the most vulnerable—women and children,
achieving excellent outcomes in the reduction of maternal mortality and improving the coverage
of HI and utilization of district hospitals by poor and ethnic minorities. The equity dimension was
important in conceptualizing and implementing the project, which contributed to social inclusion
32 The CPMU considered that district hospitals did have plans and minimum budgets for infrastructure and medical
equipment maintenance and repairing at the project end. These types of plans and budgets are submitted yearly by
the provincial Departments of Health to the MoH. However, in 2015 (the last year of the project implementation),
the buildings, and equipment supported by project were still in good condition. Because of that, part of these budgets
was not spent because it was not necessary. 33 Center for Environment and Health Studies. 2016. Report on the Final Evaluation of the Northern Uplands Health
Support Project in the Extension Phase, Hanoi, February 2016.
23
and promoted a solid basis to increase access to better quality health services by providing
economic subsidies, such as transportation costs and meals to encourage medical visits by poor
and ethnic minorities. More information on the equity impact of the project and the district
hospitals access improvement for the poor and ethnic minorities is addressed in Annex 3.
73. Strengthening local levels’ facilities and staff based on social consultations. The project
showed progress in building capacity at the Provincial Health Services Management in the NUP,
benefiting government representatives and different categories of health professionals. The project
investments were defined according to the findings of extensive consultations over a period of
eight weeks in 2007 by the Government team, involving site visits to 10 district hospitals, focus
group discussions with hospital staff at provincial and district levels, 20 in-depth interviews with
health workers, and 40 direct interviews with patients at the district hospitals. According to the
end line project evaluation results, realized by the end of 2015, more than 90 percent of the
interviewed district hospital staff said that they were entirely satisfied with the training received
and its appropriateness to job demands.
74. Environmental contribution of the project. The project improved the awareness of
environmental issues at district hospitals by training personnel on appropriated treatment of
medical and hazardous hospital waste, and contributed to improving the environmental quality and
reducing the risk of hospital infection for the community.
3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops
75. At the project original closing date and at the end of the extension phase, the CPMU
contracted a consulting firm to conduct beneficiary surveys and collect administrative data
comparable with the 2009 Baseline Survey and the 2012 MTR survey34. The main findings of
the beneficiary surveys highlighted that the support and interventions of the project had a strong
influence on changing health care services seeking behavior of the poor/ethnic minorities. The
beneficiaries perceived a continuous improvement of service delivery capacity of district hospitals,
which attracted more people to use the district hospitals’ health facilities. In addition, the project
financing support policy to the poor increased the opportunity to use inpatient services at the
district hospitals by the poor/ethnic minority patients. Moreover, the coverage of HIC increased
and the people’s perception about the district hospitals improved, contributing to create positive
changes in the poor/ethnic minorities’ behavior of seeking health care services.
4. Assessment of Risk to Development Outcome
Rating: Moderate
76. The project strengthened health policies and institutions at the district level,
improved health techniques and technology, provided new managerial tools, and created
financial sustainability of district hospitals in the NUP areas. District hospitals that participated
in the project strengthened quality of the services, developed qualified health workforce and will
34 The assessment tools of these surveys included questionnaires for health staffs, households, and in/outpatients at
district hospitals, in-depth interviews with leaders/staffs at CPMU/PPMUs, leaders at provincial health departments
and social insurance units, leaders/staffs at provincial/district general hospitals, heads of commune health centers,
trainers and trainees of the project, and focus group discussions with patients who completed their treatment at
district general hospitals in the last three months.
24
continue to satisfy the local people’s needs based on sustainable mechanisms for service provision
at the district and commune levels. Supported health care policies for the poor and ethnic minority
groups are based on better management capacity of the resources received from HCFP. The project
contributed significantly to the implementation of policies on strengthening human health
resources, especially for health care at the district level. The district hospitals developed plans to
maintain and use the health staffs recruited from the local human resources after training and plan
to maintain a regular training system to strengthen the capacity of health staff. On the financial
side, all project provinces had specific commitments to provide funds for the maintenance of
equipment and upgrading and repairing of facilities after the project phases out. Besides the
commitments and specific plans, the localities still face a limited budget, creating, despite the
government provisions and regulations to assure the future sustainability of the project investments,
some uncertainty regarding the adequacy of the resources to guarantee the continuation of project
outcomes after closing.
5. Assessment of Bank and Borrower Performance
5.1 Bank Performance
(a) Bank Performance in Ensuring Quality at Entry
Rating: Moderately Satisfactory
77. The project preparation time (22 months) was a bit longer than the average for Health
Nutrition and Population Global Practice projects, but still under the World Bank’s
benchmark for IPFs. The signing of the financial agreement happened 4 months after Board
approval (July 10, 2008) and the project effectiveness took 3 additional months (October 8, 2008),
totaling 29 months between the issuance of the project concept note and the effectiveness. The
project was proposed to be implemented in six years given the institutional implementation
conditions at the NUP, but despite achievement of the PDO within its planned time frame, the total
implementation time was about eight years after project extension. The World Bank team
supported the preparation and transferred to the government team the appropriate technical skills
to prepare the project and used the lessons learned for previous World Bank projects in the country.
The project design was totally consistent with the CPS Government Strategy, described in
paragraph 9 of this ICR. Given the experience in previous projects, the Government and the World
Bank, during preparation, concentrated their efforts on preparing the institutional arrangements to
implement the PDOs and mitigate the main challenges that could prevent the achievements. The
project Operational Manual was prepared and approved by the World Bank before effectiveness,
which contributed to guide the project implementation at the CPMU and the PPMUs.
78. Despite the good and focused design, the project could have improved efficiency in its
implementation if the baseline survey had been carried out during the project preparation
period. The project RF was well designed. However, the baseline survey was designed during
implementation as part of the project Component 3, as can be read in the PAD35. During the
project’s QER, no recommendation was provided to change this implementation arrangement, but
the project could have sped up its implementation if the team pushed to set up the baseline survey
35 in the paragraph 97(page 50) of the PAD is written the following: “This component will support the set up and
management of the Central Project Management Unit (CPMU) and PPMUs through: ….(v) M&E activities,
including the baseline data collection, indicator updates, MTRs, end of project completion report, and audits”.
25
as part of the project preparation instead of completing it in the first semester of 2009 when the
project had been under way for almost one year of implementation. If the baseline survey had been
carried out during project preparation, the World Bank team would have had more room to assist
the Government in establishing a sound and rigorous project M&E system.
(b) Quality of Supervision
Rating: Satisfactory
79. The task team conducted 11 implementation support visits during the project life.
Financial management and safeguards missions were performed adequately. In addition to formal
missions, the project benefited from having during most of the implementation a field-based task
team leader (TTL) who was in constant communication with the CPMU36. The TTL and the team,
including the fiduciary technical support were readily available to solve issues and explore the best
solutions to address technical and administrative problems, and the borrowers’ report (annex 6)
recognizes the capacity of the World Bank team in addressing promptly the demands from the
client. Records of ISRs and aide memoires were kept adequately and the project documents were
easily made available. The team composition, staff resources, and budget to supervision missions
were well used during the project implementation. The quality of the aide memoires was adequate
for supervision purposes. Annex 7 summarizes the project aide memoires.
80. The ISRs reported progress on most of the indicators, but a complete view of these
indicators was only available after the MTR survey. The team was aware of the outdated
baseline and targets during project implementation, but the RF depended on administrative records
and surveys that sometimes were not available or delayed. The project team made many efforts to
improve data availability of the RF, and many district hospitals implemented IT systems to
improve their administrative records as part of the tasks proposed by the project. However, the
response was sometimes slow because it depended on the local HIS implemented by the PPMUs
at the provincial level and their support to the district hospitals.
81. Environmental and social safeguards were well monitored resulting in considerable
improvements in the waste management systems at the district hospitals and a change of
perception of the quality of these hospitals by poor and ethnic minorities. The World Bank
team’s performance was crucial to guarantee the procurement for the hospital waste management
equipment and to help the country design adequate solutions for district hospitals.
(c) Justification of Rating for Overall Bank Performance
Rating: Satisfactory
81. The project was implemented in a very proactive manner by the Bank team, in terms of
technical assistance, good project reports and follow up of the project implementation results.
The client was satisfied with the Bank assistance during implementation and the Bank team try to
move the project positive agenda in the best way, recognizing the limitations and constraints of
the government health policy in the country. Given all features described in paragraphs 78 to 80,
36 Along the implementation, the project had four TTLs, but from 2010 to 2016, local TTL’s were in charge of
implementation.
26
combined with project results in terms of relevance, efficacy and efficiency, the overall Bank
performance could be rated as satisfactory.
5.2 Borrower Performance
(a) Government Performance
CPMU Performance: Satisfactory
82. The CPMU was established with sufficient human resources (including the national
consultants) and good capacity to manage and implement the activities and to use the
financial resources. As expressed in paragraph 23, the MoH authorities were totally committed
to project preparation and to development of the project design. After the project be approved, they
worked to guarantee adequate budget for communications, supervision, M&E, and travel during
implementation. The provincial health authorities and district hospital managements were also
committed to participate in the project development. Field visits were organized during preparation
to establish links and networks with the local government health and hospital’s authorities. The
CPMU has promoted an effective coordination with the functional departments of the MoH,
accelerating the processes for evaluation and approval of the project needs in the higher
government levels. The handbook for guiding the implementation of the project was compiled and
issued by the decision of the Minister of Health right at the beginning of the project, contributing
to speeding up its implementation. The Government established a project steering committee that
works satisfactorily with regular meetings to solve issues during the project implementation.
83. The CPMU was able to promote the effort, commitment, and responsibility of the
provincial leaders to increase the efficiency of the project on the economic and social
development of the provinces. The CPMU also provided capacity building for the PPMUs’ staff
to improve project management, including financial, accounting, and procurement issues; assets
management; civil works; training on HCFP assessment; and M&E activities. The CPMU
performance was essential to implement the project on time and exceed the targets. The CPMU
assistance to the provinces and district hospitals was highly effective, providing significant support
to the health management of the NUP beneficiaries.
(b) Implementing Agency or Agencies Performance
Implementing Agency Performance Rating (PPMUs): Moderately Satisfactory
84. In the provinces with high attention and strong support from the local governments,
the PPMUs performed the project more efficiently than in others. The progress on the project
implementation should be considered uneven among the PPMUs. Given this situation, some delays
in auditing and reporting happened with differences in performance of health indicators among
them. Additionally, many PPMUs did not provide stable human resources that could guarantee
constant improvements in the project management and results M&E, especially close to the end of
the project when staff departure was observed in some PPMUs.
(c) Justification of Rating for Overall Borrower Performance: Satisfactory
85. The overall borrower performance could be classified as satisfactory, given that
despite the weakness of part of the PPMUs on M&E, the CPMU provided all elements to fix these
problems by contracting local consultants and provide technical assistance to produce data and
27
good reports that fed the final project evaluations. On the other hand, the CPMU direction
increased the managerial autonomy of the district hospitals and assist the provincial health
departments on improve the general management of the local health systems in the project areas
6. Lessons Learned
86. Investments in hospital infrastructure, equipment, improved skills of health staff, and
incentives to retain qualified human resources at grassroots levels (provinces, districts, and
communities) are important factors to achieve basic health goals (such as the MDGs) and
improve access to health services for achieving universal health coverage. The project assisted
localities to develop their health plans, to implement preparation steps for maintaining the
activities after the project ended, and to continue the issuance of policies and plans on health
support for the northern mountainous provinces. Two aspects could be highlighted: (a) the health
workforce development policy and (b) the retention and training of teams of village midwives to
serve remote areas where home birth delivery is necessary.
87. The project supported progressive improvements in the financing mechanisms for
provincial health services, but financial sustainability at the district hospitals in the NUP
areas remains a challenge. National and provincial hospitals have more ability to increase their
revenues and sustain and balance their budgets than district hospitals, which face lower levels of
funding, contributing to increasing the risk of poor quality services. However, in Vietnam, HI
payments and user fees are the dominant source of finance for public (non-district) hospitals. The
Government progressively is introducing mechanisms to strengthen the sustainability of the district
health services, including the use of government bonds, which, according to a MoH study, reached
91 percent of district hospitals by 2011. However, the Government needs to set up mechanisms
that could allow these hospitals to have adequate financing as well as higher autonomy and less
dependency on the provincial revenues. Part of this effort should be to increase and adjust the price
of the health services provided by the district hospitals (and paid by the HI) according to their real
costs, especially considering that the district hospitals received lower fees than those paid to
regional and national public hospitals.
88. Balanced interventions and health investments in both the supply and demand sides
enable district hospitals to improve coverage, access to health services, and the range of
services offered in deprived areas. Although district hospitals in Vietnam have the ability to
provide about 75 percent–85 percent of their assigned service list, these hospitals in the Northern
Upland provinces at the beginning of the project only provided 35 percent of the list because of
lack of medical equipment, lack of health professionals, particularly specialists, and the weakness
of provider payment mechanism and service price list, which do not support or encourage district
hospitals to provide services. The project increased the range of services from 35 percent to 80
percent addressing investments in medical equipment and human resources, but failed in
addressing innovative provider payment mechanisms to increase incentives for adequate health
human resources retention.
89. Support for transport and meals are efficient mechanisms to increase the utilization
of the health services as these were the key barriers for the poor and ethnic minority
population. Once understanding their benefits and availability of health services, the poorest
populations living in distant areas need the means to travel and sustain themselves when seeking
28
health care at district hospitals. Further, the support policy must be monitored closely to avoid
abuse or overuse by both the service providers (health facilities) and users (patients and their
family) such as unnecessary longer stay for inpatients, unnecessary hospitalization of the patients
who can be treated as outpatients, and so on.
90. Despite the investments and incentives implemented by the project, district hospitals
still have difficulty to retain staff. They have less capacity to generate incentives as well as
reward health staff and improve their salaries. Thus, the provider payment mechanism should be
changed and revised to encourage hospitals to produce better services. The project was able to
solve partially problems of staff, benefiting from rotation of health workers from provincial
hospitals and supplemental salaries to skilled doctors and health specialists working at the district
hospitals. Some district hospitals in Vietnam are using capitation as an incentive to retain staff and
pay for performance, but this kind of arrangement was not used in the district hospitals of the NUP
provinces.
91. The institutional capacity of the PPMUs and CPMU plays an important role in
implementing the project. Staff in implementation units should be well trained on all aspects of
project management and work closely with the technical support from the central project team.
Staff turnover should be limited. During original project implementation time, the quality of
project implementation by the PPMUs was granted, but during the extension period, the transition
of the project staff to the Health Provincial Services was not completed and some of the
administrative functions, such as M&E, were missed. Although CPMU had only 20 staff (less than
many other projects of the same scale), project activities were well implemented, the schedule and
work plan were always on track, and the targets/results were surpassed compared to the planned
outputs.
92. M&E systems and plans should be built ahead of the project effectiveness with
standard forms for collecting data at the implementing sites. This will allow future projects to
have good data and reference sources from the start, improve monitoring of projects achievements
and get the accurate information to set more realistic baselines for some project indicators.
93. Strengthened coordination to assure financial sustainability is key to guarantee the
project continuity at the provincial and local levels. The PPMUs should closely follow up with
Provincial People’s Committees to approve annual financial plans at the earliest, so activities can
be implemented at the beginning of each year.
7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners
94. The borrower issued a report, annexed to this ICR in full, where the main comments
and issues raised are summarized as follows:
(a) The project has successfully achieved its overall development objective. The
utilization of district health services (by the poor and vulnerable population defined
accordingly with the Decision 139) has sharply increased after IEC campaigns,
improving the effectiveness of the Government’s priority to UHC, both by improving
the geographical accessibility to quality health services at district hospitals and by
29
reducing the financial burden in accessing health services for the poor and ethnic
minorities.
(b) The project has been successful on implementing the health care policies for the
poor, increasing their access to quality health services. The number of poor people
receiving financial assistance (meals and travel costs) to visit district hospitals from
NUP has increased yearly, along with the significant increase in the rate of use of
district health services among the general population, especially vulnerable groups
such as poor and ethnic minorities. The average number of inpatient and outpatient
visits per capita per year of ‘HN’ (Ho Ngheo or Poor Household in English) in the
district hospitals increased 390 percent (from 0.0247 to 0.096) and 369 percent (from
0.067 to 0.247) from 2009 to the end of 2014/2015, respectively, exceeding the
project’s expectations. The awareness and confidence of the vulnerable population on
health care services has improved. The percentage of households who experienced
catastrophic health care expenditures at the project end was substantially lower than
the baseline.
(c) The project was designed and implemented based on the needs and
recommendations of the local authorities and communities. The MoH
accumulated experience in implementing projects with similar components and
activities. Therefore, the implementation of the project was quite favorable and the
risk management process was well conducted and did not menace significantly the
project performance.
(d) The project implementation was provided with sufficient human resources
(including national consultants) and good capacity to manage and implement the
activities and to use the financial resources. The CPMU provided capacity building
for the PPMUs’ staff on project management, including financial, procurement,
accounting, assets management, civil works, training, and M&E activities.
(e) The success in implementation of the project was also the consequence of good
management and positive support from the World Bank team. Close monitoring
of the project activities, timely issuance of no objections for procurement and work
plan, suggestions for important solutions and recommendation for speeding up
implementation progress, and participation in technical missions were some of the
valuable contributions of the World Bank team to the success of the project.
(f) For the improvement of future projects, the M&E system and M&E plan should
be built ahead of effectiveness with standard forms for collecting data from the
PPMUs and implementing sites. This will allow future projects to have good data
and reference sources from the start and improve monitoring on the projects’
achievements. The PPMUs should closely follow up with Provincial People’s
Committees to approve annual financial plans at the earliest, so activities can be
implemented at the beginning of each year.
30
Annex 1. Project Costs and Financing
(a) Project Cost by Component (in USD, Million equivalent)
Components Appraisal Estimate
(USD Millions)
Actual/Latest
Estimate (USD
Millions)
Percentage of
Appraisal
1. Strengthening District-level
Health Services 42.90 40.40 94.0
2. Increasing Financial Access to
Health Care Services for
Decision 139 Beneficiaries
10.00 10.03 101.0
3. Monitoring, Evaluation and
Project Management 13.10 9.71 61.1
Total Baseline Cost 66.00 60.14 88.5
Total Financing 66.00 60.14 91.1
Note: * Exchange rate loss is USD 1.59 million. Total IDA credit is USD58.41 million at project closing.
(b) Financing
Source of Funds
Appraisal
Estimate
(USD Millions)
Actual/Latest
Estimate
(USD millions)
Percentage of
Appraisal
Borrower 6.00 1.73* 28.8
International Development Association 60.00 58.41 97.3
Total Financing 66.00 60.14 91.1
Note: *Without in kind contribution.
31
Annex 2. Project Outputs
Summary
1. The NUP project implementation presented a relatively good performance. The project had
3 components, 6 subcomponents, 19 identified activities, and 29 identified outputs. Annex 2A
provides the project output map before the extension period and during the 18-month extension
phase, identifying project achievements compared with planned or expected outputs. Of the 29
identified outputs, 8 outputs were surpassed, 9 were achieved, 9 were partially achieved, 1 was not
achieved, and 2 were added during project implementation as supplemental outputs37.
2. Project outputs of Components 1 and 2 are related to the PDOs, while outputs of
Component 3 are related with project administrative tasks. Giving that, this annex analyzes the
achievements related to Components 1 and 2. Outputs for Component 3 are found in the project
output map (Annex 2A).
A: Achievements during Original Project Implementation Period (October 2008–August
2014)
Component 1: Strengthening District-level Health Services
3. Component 1 was organized in three subcomponents (a) human resources
development; (b) improving quality of the district hospitals and; (c) improving hospital
management. The first subcomponent tackled training activities for health workers; the second
included activities related to basic medical equipment minor repairs and upgrading in
infrastructure and the last included subjects related to training of district hospitals management
staff, developing hospital maintenance plans, and a management excellence award program. This
component has 14 outputs: 8 were surpassed, 2 were achieved, 2 are supplemental outputs, 1 was
partially achieved, and 1 was not achieved.
(a) Subcomponent 1: Human resources development
4. General conditions. This subcomponent has five activities: (a) long-term training for
health workers; (b) short-term training for health workers; (c) technical handover/skills transfer;
(c) innovative incentive schemes to retain health workers (Phase 1) and; (d) innovative incentive
schemes to retain health workers (Phase 2). All activities were implemented by 10 outputs: 5 were
surpassed, 2 achieved, 1 partially achieved, and 2 are supplemental outputs (without targets
defined upfront).
5. The project financed long- and short-term trainings, covering the costs of
examination and tuition fees, living allowances and per diem, training materials, and
transportation. The training activities were based on plans prepared by district hospitals and the
PPMUs, which were revised and approved by the CPMU and the World Bank. As can be seen, the
37 The achievement of the supplemental outputs is difficult to measure given that no baselines were established at
the onset.
32
number of trained staffs exceeded the original training plans. This subcomponent had 8 outputs: 5
were surpassed, 1 was achieved, and 2 were supplemental outputs (without defined targets upfront).
6. Long-term training: level-1 specialists. Table 2.1 shows that under the project funds, 377
level-1 specialist doctors were supported: 246 at the district level and 131 at the provincial level.
This represents an increase of 206 percent of the original project plans. Out of those 377 level-1
doctors, 309 graduated before December 2013. The participation of doctors from ethnic minorities
was 48 percent. Around 97 percent of the trained doctors returned to work at the original district
hospitals, representing a significant percentage of retention of qualified doctors.
Table 2.1. Number of Level-1 Doctors Trained in Project Provinces According to Plan
Project
Provinces
Level-1
Doctors
as
Planned
Level-1 Doctors Graduated Level-1 Doctors who Returned
to Work at Localities Number
Supported
by Project
Number of
Person
Years under
Project
Support
Total Female
(%)
Ethnic
Minority
(%)
Total Female
(%)
Ethnic
Minority
(%)
Cao Bang 23 56 58.9 92.9 54 59.3 92.6 56 (243%) 102 (222%)
Bac Kan 31 36 50.0 100.0 34 50.0 100.0 50 (161%) 96 (155%)
Ha Giang 38 89 31.5 14.6 86 31.4 14.0 96 (253%) 176 (232%)
Lao Cai 22 38 39.5 18.4 35 37.1 28.6 51 (232%) 97 (220%)
Lai Chau 22 17 35.3 47.1 17 35.3 47.1 19 (86%) 37 (84%)
Dien Bien 19 47 34.0 42.6 47 34.0 42.6 50 (263%) 95 (250%)
Son La 28 26 34.6 38.5 26 34.6 38.5 55 (196%) 102 (182%)
Total 183 309 40.5 47.2 299 40.1 48.2 377 (206%) 705 (193%)
District level 104 — — — — — — 246 (236%) 457 (220%)
Province level 79 — — — — — — 131 (166%) 248 (157%) Source: Center for Environment and Health Studies (2014) Final Report: End-line evaluation of the Northern Uplands Health Support Project, Hanoi, 2014.
7. Long-term training: assistant doctors to become medical doctors. The project
supported 1,467 assistant doctors attending the four-year additional training, exceeding 98 percent
of the agreed target on the original project plans. Of this number, 54 percent were ethnic minorities.
Additionally, the project trained 98 assistant pharmacists, attending the four-year additional
training to become pharmacists (32 percent ethnic minorities). This additional outcome was not
originally planned under the project.
8. Short-term trainings: curative care. The project sponsored 3,187 doctors to receive
short-term training in several specialties (table 2.2), compared with 1,041 originally planned by
district hospitals in agreement with the PPMUs (306 percent increase). The CPMU surveyed the
opinions of 226 health staffs who participated in short-term trainings. According to the survey, the
participants highly appreciated the training methods and contents of the courses. Approximately
91 percent and 93 percent of respondents were satisfied with the teaching methods and contents of
the courses and 75 percent reported that the training durations were appropriate.
Table 2.2. Number of Doctors Receiving Short-term Trainings according to the Type of Trainings
Course Plan Total Percentage of Plan
Anesthesia 92 135 147
Lab test 95 149 156
Rehabilitation 105 121 115
Surgery 101 143 142
Pediatrics 103 214 208
Internal medicine 111 194 175
33
Course Plan Total Percentage of Plan
Image diagnosis 165 404 244
Obstetrics 102 191 187
Communicable diseases 80 165 206
X-ray 87 103 118
Pediatrics emergency n.a. 681 —
Obstetrics emergency n.a. 687 —
Total 1,041 3,187 306 Source: Center for Environment and Health Studies (2014) Final Report: End-line evaluation of the Northern Uplands Health Support Project,
Hanoi, 2014.
9. Short-term trainings: preventive care. The project trained 335 staffs (156 at the
provincial level and 179 at the district level) focusing on the planning, implementation, evaluation,
and monitoring of preventive medicine activities at the primary level. This number exceeded the
originally planned target of the PPMUs, by 57 percent. The trainings included the preventive and
counseling aspects of several diseases including HIV, tuberculosis, and malaria, which are targeted
at the MDGs.
10. Other short-term trainings: hospital management, health systems management,
information systems, and maintenance of medical equipment. Table 2.3 shows that the number
of trained staffs on these specific short-term trainings exceed the original plans agreed with the
district hospitals and the PPMUs. The number of personnel trained on medical waste management
shows that a huge part of the district hospital staff was involved in the accomplishment of the
project’s environment safeguard, given that almost 3,000 staffs were trained in this specific subject.
Table 2.3. Number of Hospital Staffs Trained and
Comparison between the Planed and Achieved
Field of training Number of Staff
Training Planed
Staff
Trained
in 2014
Percentage Plan Achievement
Hospital management 303 477 157 Surpassed
Medical waste management 215 2,922 1,300 Surpassed
Health Management and Information
Systems (HMIS) 358 397 11
Surpassed
Maintenance of equipment 134 169 126 Surpassed
Source: Center for Environment and Health Studies (2014) Final Report: End-line evaluation of the Northern Uplands
Health Support Project, Hanoi, 2014.
11. Incentives to retain human resources in district hospitals: The PAD included, as one
of the project activities, the use of incentive schemes to attract and retain the newly trained health
workers at district hospitals because of a severe shortage of health staff in the NUP areas. To
address this issue, the CPMU and the PPMUs discussed and proposed solutions in workshops that
were held to seek sustainable solutions to maintain adequate health workforce for disadvantaged
areas in the NUP. The workshops discussed national and international experiences on enrollment
and training of health staff. However, the solutions and recommendations focused mainly on
training contents and less on financial support, sustainability, and budget feasibility of the
proposed incentives. To tackle the staff shortage in the NUP areas, during the health workers’
training period, the World Bank agreed that the provinces could hire retired doctors to work at the
district hospitals and health centers. Some provinces, including Cao Bang, Lao Cai, and Bac Kan
had used this hiring mechanism to partially overcome staff shortages with positive results.
34
12. District hospitals gained expertise on many specialties such as surgery, internal
medicine, obstetrics, pediatrics, intensive care, traditional medicine, nursing, and anesthesia,
focusing on emergency techniques and surgery. The project financed the technical support to
the district hospitals’ doctors by assigning provincial doctors to guide the former on every step of
these specific techniques and helping them practice these techniques themselves. In 2014, over
670 provincial doctors participated in the transfer of these techniques to district hospitals
exceeding 167 percent of the PPMU project’s plans. This activity was planned as a result of the
project MTR in 2012.
(b) Subcomponent 2: Improving quality of district hospitals
General conditions. When compared with the original plans, two outputs of this subcomponent
were surpassed.
13. Equipment for district hospitals was procured in two phases (as recommended in the
PAD): Phase 1 in 2011–2012, and Phase 2, in 2013–2014. Table 2.4 shows the number of
equipment acquired in the two phases in the seven provinces, which reached 4,400 units (109
percent of the original procurement plans). Phase 1 included ambulances to address the transfer of
emergency patients in remote districts and four groups of medical equipment: laboratory
equipment, treatment equipment, monitoring devices and ventilators, and infection control team
equipment. Phase 2 included equipment of high-value technology and trained staffs capable of
using this equipment. The project combined investment and infrastructure facilities and human
resource staffing and training in a synchronous manner to avoid under use and depreciation of the
equipment.
Table 2.4. Number of Project-supported Equipment for the District Hospitals at Seven NUP (2012–2014)
Equipment Son
La
Ha
Giang
Bac
Kan
Cao
Bang
Dien
Bien
Lai
Chau
Lao
Cai Total Plan
Percentage
Obtained
Lab test devices 181 131 128 127 83 74 98 822 664 123.8
Intensive care equipment 117 145 126 195 93 78 119 873 1255 69.6
Monitoring equipment 21 24 28 23 14 11 21 142 157 90.4
Ventilators 22 19 14 25 14 4 14 112 217 51.6
Infection control equipment 29 28 22 34 14 10 23 160 173 92.5
Incinerators 0 0 0 0 0 0 0 0 37 0.0
Ambulance care 8 5 5 14 6 6 8 52 52 100.0
Ultrasound machine 24 14 16 16 11 11 17 109 77 141.6
Endoscopic machine — — — — — — — — 37 —
Emergency care devices 59 69 43 75 37 36 47 366 390 93.8
Internal medicine 111 40 59 59 25 18 35 347 209 166.0
Surgical room equipment 85 46 55 65 40 36 55 382 225 169.8
Surgical equipment 158 82 94 120 62 72 67 655 548 119.5
Examining equipment, 69 55 53 68 37 35 48 365 296 123.3
X-ray machine 18 18 16 12 14 9 15 102 78 130.8
Pediatric/ obstetrics surgery 19 7 3 31 25 68 21 174 0 —
Intensive care in obstetrics 15 11 3 31 20 45 35 160 0 —
Total 936 694 665 895 495 513 623 4,821 4,415 109.0 Source: Center for Environment and Health Studies (2014) Final Report: End-line evaluation of the Northern Uplands Health Support Project,
Hanoi, 2014.
14. However, a small portion of the equipment provision was delayed. Some of the
equipment supplied by the project faced difficulties in the import procedures and by 2014, few
35
bidding processes were ongoing. The delivery of part of the acquired equipment to hospitals in
remote areas was challenging at times because of difficult weather conditions.
15. The equipment installed by the project achieved high level of utilization. The 2014
project results survey shows that 95 percent of the equipment was delivered and installed into the
district hospitals and was frequently used for diagnosis and treatment. Only 1.4 percent of the
installed equipment was broken and 3.9 percent was unused because of lack of technical skills or
utility/infrastructure associated problems. Table 2.5 shows that in January 2014, the installed
equipment achieved a high level of monthly utilization in the respective district hospitals.
Table 2.5. Number of District Hospitals’ Patients using Equipment in the NUP Areas (January 2014)
Group of Equipment Number of Cases using the Equipment/Month
Surgical room 1,116
Obstetric monitoring 159
Cardiograph 49
Semiautomatic biochemical analyzer 1,981
Automatic hematological analyzer 600
Ultrasound 418
X-ray (high voltage) 466
Source: Center for Environment and Health Studies (2014) Final Report: End-line evaluation of the Northern
Uplands Health Support Project, Hanoi, 2014.
16. The project implemented the upgrading or new construction of 18 district hospitals
compared to 10 that were originally planned. All civil works were carried out in 6 provinces
between 2012 and 2013. The only province where civil works were delayed was Lao Cai, where
two hospitals were expected to have minor repairs. These hospitals were supposed to be upgraded
further using public resources.
(c) Subcomponent 3: Improving hospital management
17. General conditions. This subcomponent had three outputs: one was achieved, one partially
achieved, and one not achieved.
18. Many hospital managers and provincial authorities were trained in hospital
management programs by recognized institutions. The project has signed an agreement with
training units such as the School of Public Health, Bach Mai Hospital, and Hanoi Medical
University to organize management training for 477 hospital leaders and heads of departments in
provincial and district hospitals. The training contents were focused on human resources
management, health financial management, health-financing analysis, procurement and bidding
processes, and quality control of hospital performance. Considering that the planned goal was the
training of 303 hospital managers, the original target was exceeded by 57 percent. The trainees
evaluated the quality of the courses very positively. Survey results from in-depth interviews with
some hospital leaders showed that the management skills that were learned were effectively used
in the preparation of the district hospitals annual plans, plans for human resource development,
and other activities of the hospital. Overall, the project achieved its goals and the project funds
were used effectively.
36
19. Most of the district hospitals under the project developed and implemented
maintenance plans for facilities and equipment and increased the budgets to implement these
plans in 2013. The results of the 2014 survey revealed that, when compared with the early project
implementation years, the proportion of district hospitals having annual plans for maintenance
increased, but with some shortcomings by the end of the project. In December 2013, 99 percent of
the hospitals developed maintenance plans. However, at the end of 2014, only 80 percent of the
district hospitals kept these plans active. The executed budget for facilities and equipment
maintenance at the district hospitals achieved 87 percent of the total estimated costs in 2013,
compared to 57 percent in 2008.
20. Excellence award for human resources. The project planned to implement an excellence
award for human resources, but this initiative was the only activity that was not implemented, as
a project activity, during the project life. The CPMU did not collect information on this activity.
However, annually, the provincial health departments follow health units and individuals who
performed well. They are rewarded and apprised according to the Emulation and Reward
Regulation.
Component 2: Increasing Financial Access to Healthcare Services for Decision 139
Beneficiaries
21. Component 2 was organized around three subcomponents: (a) support for direct
catastrophic and nonmedical expenditures of health care for Decision 139 beneficiaries; (b)
strengthening capacity for HCFP; and (c) strengthening local access to health services
through promoting health seeking behavior. The first subcomponent addressed the support for
nonmedical expenditures to poor and ethnic minorities’ beneficiaries. The second supported
institutional capacity building for district hospitals to identify target beneficiaries; to provide them
HICs and to institute mechanisms to receive medical expenses payments. The third subcomponent
developed IEC campaigns to increase awareness and knowledge from ethnic minorities about their
rights, entitlements, and benefits from HI. This component had four outputs and, at the project end,
three were achieved and one was partially achieved
(a) Subcomponent 1: Support for direct catastrophic and nonmedical expenditures of health care
for Decision 139 beneficiaries.
22. The number of project assistance recipients and the value of the monetary allowances
increased significantly during the project. In 2009, the project was committed to support
730,183 (only the poor) beneficiaries (or 25 percent of the population in the seven provinces), but
in 2013 this number jumped to 3,185,341 beneficiaries, given the inclusion of the ethnic minorities
(82 percent of the population in the same provinces). This support covered recurrent costs for travel
and meals for the population seeking health services at the district hospitals. From June 2009 to
August 2011, the project covered these expenses for the population under the poverty threshold.
Since September 2011, the project coverage was expanded for all ethnic minorities (including
those above the poverty threshold) because they are quasi-poor and represented the majority of the
NUP population. Until May 2012, the value of the allowance was VND 15,000 per day for
inpatient meals. After May 2012, the allowance value for inpatient meals increased to VND 25,000
per day. The allowance for travel varied according to the distance (VND 60,000 under 100 km and
37
VND 100,000 for 100 km or more). Table 2.6 shows the expansion of the coverage of the
nonmedical expenditures for poor and ethnic minorities from 2009 to 2013 in the seven NUP areas.
Table 2.6. Number of Targeted Beneficiaries of NUP, 2009–2013
Province
Number
of
Districts
with
NUP
Support
Persons in
2010 (below
the poverty
threshold)
Number of
Persons in
2011 (below
the poverty
threshold)
Number of
Persons since
August 2011
(poor + ethnic
minorities of
communes 30a)
Number of
Persons since
June 2012
(added from
HEMA Project)
Number
of
Persons
in 2013
Dien Bien 9 29,399 50,008 83,426 466,000 411,405
Lai Chau 7 50,279 78,916 133,933 300,000 263,390
Son La 9 157,653 103,054 339,126 700,000 700,000
Cao Bang 13 136,497 204,305 427,453 427,453 425,342
Bac Kan 8 73,748 87,949 282,154 282,154 263,390
Ha Giang 10 155,522 318,243 607,181 607,181 647,194
Lao Cai 8 127,085 313,736 424,145 424,145 471,082
Total 64 730,183 1,156,211 2,297,418 3,206,933 3,185,341 Source: Center for Environment and Health Studies (2014) Final Report: End-line evaluation of the Northern Uplands Health Support Project,
Hanoi, 2014.
23. The number of inpatients’ coverage increased substantially, after the implementation
of the policy, extending the allowance for all ethnic minorities. According to table 2.7, in 2009
only 1,054 district hospitals’ inpatients were supported by the project, compared to 244,801 in
2013. The accumulated number of inpatients who benefited by the project from 2009 to 2013
reached 544,000, expending 86 percent of the resources planned for this activity in the project.
The household survey results at the project end line (2014) showed that 77 percent of the
respondents reported that they were supported on travel cost for inpatient treatment at the district
hospitals and 78 percent of those said that the NUP project supported meal costs for the poor/ethnic
minority people during their treatment at the district hospitals. The project contributed significantly
to promote the increased access to district hospitals for the poor and ethnic minorities in the seven
provinces.
Table 2.7. Total District Hospitals’ Inpatients Supported by Provinces, 2009–2014
Provinces 2009 2010 2011 2012 2013 First Three Months
of 2014
Dien Bien 411 1,779 3,814 16,889 30,058 7,524
Lai Chau — 1,264 4,205 13,605 18,132 4,716
Son La — 875 2,617 24,856 44,195 10,106
Cao Bang 186 4,853 18,129 44,248 45,044 10,533
Bac Can — 3,654 12,769 26,975 27,756 7,088
Ha Giang 30 3,457 17,727 38,781 39,702 11,231
Lao Cai 427 3,508 20,252 35,345 39,914 8,020
Total 1,054 19,390 79,469 200,709 244,801 59,218
Source: Center for Environment and Health Studies (2014) Final Report: End-line evaluation of the Northern Uplands
Health Support Project, Hanoi, 2014.
38
(b) Subcomponent 2: Strengthening capacity for HCFP
24. Given improvements and gains of scale and efficiency in the management of this
subcomponent, 100 percent of the objective was achieved with a substantial cost reduction.
Since May 2012, the management functions of the HCFP were extended to the HEMA Project
provinces, increasing the total number of HCFP beneficiary districts up to 64 districts compared
to the initial 50 districts. Two out of five of the HEMA Project provinces (Dien Bien and Lai Chau)
upgraded to manage HCFP were part of the NUP project. Therefore, the planned budget of this
subcomponent was reduced by 71 percent compared to the plan in the project document, (from
USD 872,282 to USD 255,925). On the other hand, simplifications to transfer the HCFP funds to
provinces with poor and ethnic minorities reduced the cost of the procedures that were expected
to be created at the provincial HCFP management units.
25. In March 2014, all activities of this subcomponent were completed and high efficiency
in strengthening the management capacity of the HCFP was achieved. These activities
(including training courses for the fund management unit, support of equipment, and monitoring
of the fund) were completed, disbursing USD 191,652.59 (75 percent of the planned budget after
adjustment).
(c) Subcomponent 3: Strengthening local access to health services through promoting health
seeking behavior
26. IEC-related activities, such as assessment needs, staff training, and provision of IEC
equipment paved the way to raise awareness and changes in behavior, thus promoting higher
utilization of health care services for the poor and ethnic minority people. From 2010 to 2014,
the project implemented several IEC activities such as (a) conducting an IEC needs assessment in
the project provinces as a basis to develop the IEC strategy and prepare IEC materials for
community and training for educators; (b) training 411 district and 9,070 commune staffs in the
seven provinces, for IEC activities; (c) providing essential IEC equipment, such as digital video
cameras, non-linear editing systems, loudspeakers, cameras, recorders, portable speakers, image
editing kits, and the so on to the provincial IEC centers established by the project; and (d) printing
and distributing materials to the provincial centers and district rooms to conduct IEC activities for
the community, according to the plan that was proposed by the PPMUs and approved by the CPMU.
27. The local health staffs and community reported that the IEC activities of the project
substantially reached out to the poor and ethnic minority people. According to the final report
of the CPMU, in 2013, 880 out of 899 IEC project communes in the provinces had IEC activities
implemented, reaching 98 percent of the original plan. The total number of local people benefiting
from the IEC activities of the project was 25,744.
28. The effectiveness of the IEC activities of the project was highly appreciated based on
the results of the household survey at the end line as compared to the end-line target of the
project. The end-line evaluation results show that most NUP inhabitants who had HICs could
name at least one right of the HICs (IOI #9). The rate of respondents who could name at least three
rights of the HICs increased significantly from 14.8 percent to 57.4 percent.
39
29. The implementation of the IEC subcomponent led to savings in the project. As of
March 31, 2014, disbursement of the subcomponent reached USD1.13 million (86 percent of
adjusted plan). However, by March 31, 2014, IEC activities continued to be implemented until
December 31, 2014, for an amount of USD 186,625.
B: Project Achievements during the 18-Month Project Extension Period (August 2014–
February 2016)
Component 1: Strengthening District-level Health Services
(a) Subcomponent 1: Human resources development
30. Long-term training: level-1 specialists. During the project extension, 57 level-1
specialists graduated before December 2015, bringing the total number of graduated doctors to
367. Ten additional level-1 specialists were expected to finish graduation after project closing.
Thus, by the end of the extension phase, there was no change in the number of level-1 specialists
supported by the project and an addition of 57 new graduated doctors. At the late extension phase,
nearly 97 percent of the trained level-1 specialists, who graduated to doctors, returned to their
former workplace: 57 percent of the level-1 trained doctors are ethnic minorities and 53 percent
are female. This reflects increased sustainability of the human resources supply in the provinces.
31. Long-term training: assistant doctors to become medical doctors. During the project’s
extended phase, 324 additional assistant doctors graduated as medical doctors using the project
funds, totaling 1,058 assistant doctors who transformed to medical doctors along the project. In
total, it represents 160 percent of the original plans’ targets. Additionally 40 pharmacists graduated
in the extension phase (see table 2.8).
Table 2.8. Number of Four-year Additional Doctors and Pharmacists who Graduated by the Evaluation Time
Health Professionals Trained Cao
Bang
Bac
Kan
Ha
Giang
Lao
Cai
Lai
Chau
Dien
Bien
Son
La Total
Support for four-year added doctors
Total doctors graduated (#) 131 53 213 145 142 192 182 1,058
Doctors graduated in regular project time (#) 104 42 130 106 91 142 119 734
Doctors graduated in project extension phase (#) 27 11 83 39 51 50 63 324
Doctors returned to work at their units (#) 122 44 208 96 136 193 179 978
Plan achievement (%) 147 144 192 179 128 179 148 160
Rate of ethnic minority doctors (%) 99 100 36 36 15 34 44 46
Rate of female doctors (%) 99 50 32 41 43 29 40 45
Support for four-year additional pharmacist training
Total pharmacists graduated (#) 13 7 15 23 13 16 11 98
Pharmacists graduated in regular project time (#) 6 6 9 10 6 13 8 58
Pharmacists graduated in project extension phase (#) 7 1 6 13 7 3 3 40
Rate of ethnic minority pharmacists (%) 92.3 71.4 20 26.1 7.7 6.0 18.2 30.6
Rate of female pharmacists (%) 86.4 71.4 60 82.6 69.2 50 72.7 70.4 Source: Center for Environment and Health Studies (2016) Final Evaluation Of The Northern Uplands Health Support Project In The Extension
Phase, Hanoi, 2016.
32. Short-term training: curative care. The extended phase contributed to complete the
plan’s achievement for some important short-term trainings (emergency care and x-ray) and to
exceed the plan in other short-term trainings that were already accomplished in the regular project
40
time, adjusted according to the needs from district hospitals (anesthesia, and diagnose imaging).
During the extension phase, additional training was provided in areas that were not present in the
project’s regular time (ear, nose, throat and dental care, and nursing managing). The extension
phase added 18 percent additional short-term trainings in this specific activity. Therefore, at project
closing, the short-term trainings completed was 2.6 times the original plans’ goals, as seen in table
2.9.
Table 2.9. Number and Rate of Doctors Attending Short-term Training in the Extension Phase
Course Plan
Regular
Project
Time
Extension
Phase
Percentage
Added in
Extension
Phase
Total in
Project
Life
Percentage
Plan Increase
Anesthesia 117 135 22 19 157 134
Teasing 95 149 0 — 149 156
Emergency care 140 121 31 22 152 109
External medicine 101 143 0 — 143 142
Pediatrics 103 214 0 — 214 208
Internal medicine 111 194 0 — 194 175
Diagnostic imaging 200 404 32 16 436 218
Obstetrics 102 191 0 — 191 187
Communicable diseases 80 165 0 — 165 206
X-ray 107 103 18 17 121 113
Ear, nose, and throat, and dental 108 0 70 65 70 65
Nursing management 70 0 63 90 63 90
Pediatric emergency n.a. 681 0 — 681 —
Obstetric emergency n.a. 687 0 — 687 —
Total 1,334 3,187 236 18 3,423 257 Source: Center for Environment and Health Studies (2016) Final Evaluation Of The Northern Uplands Health Support Project In The Extension
Phase, Hanoi, 2016.
33. Additional short-term training on MDG related activities in the project extension.
Along the regular project time, training for obstetric and pediatric emergencies was important to
prepare health staff to tackle maternal and neonatal mortality for ethnic minorities in the NUP
mountainous areas. During the project extension, the CPMU organized additional training to
certify health staff on appropriate interventions to increase the accessibility and utilization of
maternal and child health care services in villages, community health centers, and district hospitals.
The CPMU worked with the related stakeholders (Maternal and Child Health Department,
Provincial Departments of Health, and so on) to establish training and certification for skilled birth
attendants (SBA) (directed to general doctors and other health workers) and for specialists to work
on emergency in gynecological and obstetric care. Specific training was delivered to midwives at
the village level. As result, between August 2014 and December 2015, the following professionals
were trained and certified: (a) 794 health staffs working on obstetric and gynecologic services were
certified as SBAs; (b) 307 general doctors working on obstetric and gynecologic services at the
district/commune levels were certified as SBAs; (c) 110 medical doctors were trained on
emergency care for obstetric and gynecologic services at district level; (d) 252 village midwives
graduated; and (e) 79 clinical staff at neonatal units of the district hospitals, and other minor
trainings were provided according to the needs.
41
34. Short-term training: preventive care. Given that the training in this area was achieved
and exceeded during the project’s original phase, no additional training was provided during the
extension phase.
35. Short-term training: hospital management, medical waste management, HMIS, and
maintenance of medical equipment. During the regular project time, the goals for these four
kinds of short-term trainings were already achieved and surpassed. In the project extension time
(table 2.10), these short-term training courses were also delivered with 75 percent more staff over
the original plans (761 staff) trained. At the end of the project, the number of professionals trained
in hospital management was 2.5 times more than what was originally planned, focusing on areas
such as human resources management, health financial management, health financial analysis,
procurement management, bidding, and hospital quality management. In the area of medical waste
management, no additional training was delivered in the project extension phase, but it already
achieved 13.6 more times of staff training than planned. Regarding HMIS, the extension time
added 125 percent of trained staff. Regarding the maintenance of equipment, the achievement was
28 percent of additional trained staff. In summary, the extension phase provided additional trained
staff to the already achieved goals. The overall achievement of short-term training for health staff
was almost five times higher than planned.
Table 2.10. Health Staffs Attending Hospital Management, Health Systems Management, Information
Systems, and Maintenance of Medical Equipment Short-term Training in the Extension Phase
Course Plan
Regular
Project
Time
Extension
Phase
Percentage
Added in
Extension
Phase
Total in Project Life Percentage
Plan Increase
Hospital management 303 477 276 91 753 249
Medical waste management 215 2,922 0 — 2,922 1,359
HMIS 358 397 448 125 845 213
Maintenance of equipment 134 169 37 28 206 154
Total 1,010 3,965 761 75 4,726 468 Source: Center for Environment and Health Studies (2016) Final Evaluation Of The Northern Uplands Health Support Project In The Extension
Phase, Hanoi, 2016.
(b) Subcomponent 2: Improving quality of district hospitals
36. Given that this activity was accomplished in the regular project time, few new
equipment was provided during the extension period to the district hospitals. Only new
equipment related to improving the quality of care for mother and children undergoing surgeries
and intensive care were acquired during that period. Equipment included the model of delivery
attendance instruction and neonatal resuscitation and accompanied appendixes; clean birth
delivery packages; instrument bags for village midwives; equipment for infant weight and length
measurement and kits for newborn resuscitation (including vacuum and suction pipe, metals
collection box, heating lamp, mask, and oxygen ventilation). Therefore, during the extension
period, the project’s new purchases were mainly focused on equipment for neonatal units and
village health teams, including instruments and consumable supplies for neonatal units. In total,
the project, with the extension phase, purchased 5,065 medical equipment; 14.7 percent more than
what was originally planned.
42
37. By the end of 2015, all equipment acquired by the project achieved high levels of
utilization in the district hospitals. Figure 2.1 shows the levels of utilization of almost 100
percent in three of the seven provinces. The only province where the level of utilization was lower
than 90 percent was Cao Bang, which was expected due to the small demand for hospital inpatient
facilities when compared with other project district hospitals. No new constructions and civil
works in the district hospitals were financed by the project during the extension phase.
Figure 2.1. Percentage of the Equipment Acquired by the Project that are in Use in the District Hospitals of
the Seven NUP Provinces.
Source: Center For Environment And Health Studies (2016) Final Evaluation Of The Northern Uplands Health Support Project In The
Extension Phase, Hanoi, 2016.
(c) Subcomponent 3: Improving hospital management
38. Short-term trainings on hospital management and maintenance plans for facilities
and equipment were kept as project priorities during the project extension phase for
sustainability purposes. However, the district hospitals and health provincial authorities at the
provincial level are still struggling with the perspectives of increasing budgets to build long-term
sustainability of project interventions. One of the lessons learned in this area is that policy
development is a process that requires a lot of time and involvement of ministries and provincial
people's committees in Vietnam. In this case, the project could be considered successful, given
that between 2014 and 2015 (a) the percentage of district hospitals with plans to maintain physical
infrastructure increased from 80 percent to 86 percent and the budget for these activities increased
by 11 percent and (b) the percentage of district hospitals with plans to maintain medical equipment
increased from 91 percent to 94 percent and the budget for these activities increased by 47 percent.
Component 2: Increasing Financial Access to Healthcare Services for Decision 139
Beneficiaries
39. Three out of the four outputs of this component were achieved and one partially
achieved before the extension period. However, no additional information about the indicators
of this component has been provided during the extension phase, making it difficult to know if the
achievements remain sustained. There are indirect evidences (such as budget allocations)
confirming that the support to health care nonmedical expenditures for the poor, by providing
meals and transportation subsidies in the seven provinces, worked during the project extension
phase. However, IEC activities were not carried out during the extended period (only until
December 2015).
93.799.6
93.887.9
100 99.7 97.3 95.1
0
20
40
60
80
100
Son La Ha Giang Bac Kan Cao Bang Dien Bien Lai Chau Lao Cai Total
43
Annex 2A. Project Output Map
Subcomponent Related Activities Planned Outputs
Output Achieved
Before Extension
(October 2008–
August 2014)
Achievement as
Percentage of the
Planned Output
before Extension
Output Achieved
during Project
Extension Period
(February 2016)
Additional
Achievement as
Percentage of the
Planned Output
Total Percentage of
Achievement of the
Planned Output
during the Project
Life
Component 1: Strengthening District-level Health Services (USD42.9 million)
(a) Human
resources development
Long-term training
activities for health workers
Train 183 level-1
specialists
309 level-1
specialists trained 169
58 level-1 specialists
finished training*1 32 201 (Surpassed)
741 assistant doctors to become medical
doctors
734 assistant doctors became medical
doctors
99 324 additional assistant doctors became medical
doctors*2
44 143 (Surpassed)
No planned outputs for assistant
pharmacists
58 assistant pharmacists became
pharmacists —
40 additional assistant pharmacists became
pharmacists —
Complementary
Output
Short -term training activities for health
workers.
1,334 medical doctors to receive
short-term training in
medical techniques on curative care
3187 medical doctors received short term
training in medical
techniques on curative care
238
236 additional medical doctors received short-
term training in medical
techniques on curative care
18 256 (Surpassed)
No planned
additional short-term
training on MDG-related activities
—
—
1542 health staffs
received short-term
training on MDG-related activities*3
— Complementary
Output
213 health staffs to
receive short-term training in preventive
care
336 health staffs
received short-term training in preventive
care
157 — — 157 (Surpassed)
1,010 health staffs to receive short-term
training on hospital
management, HMIS, and maintenance of
medical equipment
3,965 health staffs received short-term
training on hospital
management, HMIS, and maintenance of
medical equipment
393
761 additional health staffs received short-
term training on
hospital management, HMIS, and maintenance
of medical equipment
75 468 (Surpassed)
Techniques/skills
transfer
Promote knowledge
sharing between provincial and
district level
facilities (402 times of technique
transfers at the
district hospitals)
Over 670 times of
technique transfers were implemented in
the district hospitals
at regular project time
167
The project gave
priority to transfer techniques/services for
emergency of obstetrics,
pediatrics, that district hospitals have not had
ability to do as required
by the MoH.
—
167 (Surpassed)
Transfers of techniques were
achieved establishing
sustainable mechanisms to
develop capacity in
service provision, especially at district
and commune levels
Innovative incentive schemes to retain
health workers
Study on the characteristics of
health workers
In 2009, the CPMU conducted a study on
‘Health Human
— — — Achieved
44
Subcomponent Related Activities Planned Outputs
Output Achieved
Before Extension
(October 2008–
August 2014)
Achievement as
Percentage of the
Planned Output
before Extension
Output Achieved
during Project
Extension Period
(February 2016)
Additional
Achievement as
Percentage of the
Planned Output
Total Percentage of
Achievement of the
Planned Output
during the Project
Life
Phase I - Rapid labor
market assessment
currently working in
the NUP and job
characteristics in the NUP provinces
(salaries and
opportunities)
Resource Analysis in
7 Provinces of
Northern Upland’. The study was used
to plan human
resources health needs in the NUP and
establish training
goals and incentives
Innovative incentive
schemes to retain
health workers
Phase II - Developing
and implementing innovative incentive
schemes
Develop potential
incentive
schemes and study their likely impact on
recruitment and
retention
Backup arrangement
for health staffs when
being trained;
Hiring doctors who
are retired to work for the
hospitals/district
health centers.
— — —
This output was
partially achieved,
The provinces need to continue training
and developing
mechanisms and policies to support
retention of health
staffs at district level.
(b) Improving quality of
district hospitals
Basic medical equipment
Acquire and install 4,415 medical
equipment
4,821 medical equipment were
acquired and installed
109 244 additional medical equipment were
acquired and installed
6 115 (Surpassed)
Minor repairs and
upgrading
10 district hospitals
to have new
construction and
upgrades
18 district hospitals
had new construction
and upgrades 180 — — 180 (Surpassed)
(c) Imp1roving
hospital
management
Training of district
hospital management
staff
303 health staffs to
attend hospital
management training
477 health staffs had
attended hospital
management training 157
276 additional health
staffs had attended
hospital management training
91 248 (Surpassed)
Developing hospital
maintenance plans
All district hospitals
to have infrastructure
and equipment maintenance plans
80% of the district
hospitals had
maintenance plans in December 2014
80
86% of the district
hospitals had
maintenance plans in December 2016
86 86 (substantially
achieved)
Management
excellence award program
The manager
excellence award program does not
appear in the
program execution documents*4
— — — — Not achieved
Component 2: Increasing Financial Access to Healthcare Services for Decision 139 Beneficiaries (USD10.0 million)
(a) Support for
direct
catastrophic and
nonmedical
Study of distribution
and patterns of catastrophic
expenditures in the
first year
Develop a survey in
the first project year to establish a
baseline for the KPI
The survey was done
in the first year of project execution and
the baseline for
Achieved
Two other similar
surveys were developed in the MTR and in the
original project closing
to follow up the project
Achieved Achieved*5
45
Subcomponent Related Activities Planned Outputs
Output Achieved
Before Extension
(October 2008–
August 2014)
Achievement as
Percentage of the
Planned Output
before Extension
Output Achieved
during Project
Extension Period
(February 2016)
Additional
Achievement as
Percentage of the
Planned Output
Total Percentage of
Achievement of the
Planned Output
during the Project
Life
expenditures of
health care for
Decision 139 beneficiaries
and IOI related to the
project Component 2
Component 2 KPI
and IOI set
Component 2 KPI- and
IOI-related targets.
730,183 beneficiaries (25% of the
population in the
seven provinces) were targeted to be
supported according
to the 2009 survey
3,185,341 beneficiaries (82% of
the seven provinces
population) were supported in
December 2013
according to the end project survey
Achieved — — Achieved*6
(b) Strengthening capacity for
HCFP
The HCFP capacity is measured by their
capacity, at the provincial level, to provide the
support for transportation and food allowance.
All activities were
completed with a
disbursement rate of 75% of the budget
schedule after the
adjustment. Achieved
At the time of the
extension-phase project
evaluation, three provinces (Dien Bien,
Bac Kan and Cao Bang)
have not yet established the health care for the
poor fund because they
could not balance the funds for this activity.
— Partially achieved
(c) Strengthening
local access to
health services
through promoting
health seeking
behavior
IEC activities, such as
(a) searching for
assessment needs, (b)
training staff in IEC
and, (c) providing IEC equipment
899 communities in
the seven provinces
to receive IEC
activities
880 communities in
the seven provinces
received IEC
activities addressing
25,744 local people in December 2013.
98
No IEC activities were
identified as performed
in the project extension
phase. — Achieved*7
Component 3: Monitoring, Evaluation, and Project Management (USD13.1 million)
No subcomponent
has been identified
Strengthen procurement, financial
management, and disbursement
The CPMU
contributed to increase local
capacity (PPMUs) for
fiduciary procedures. Some delays in
procurement of
medical equipment
were identified. At
the end of the first
phase, disbursement was 88% of project
loan.
Partially achieved
The project
management was effective, focused on
promoting the project
results, and enhanced the sustainability. The
procedures and
processes of project
implementation were
relatively clear and no
significant gaps were identified.
— Partially achieved*8
Training of project management staff The CPMU, with
World Bank support, provided training for
the project
Achieved
In the extension phase
the PPMUs reduced staff and increased
rotation creating some
Partially achieved Partially achieved
46
Subcomponent Related Activities Planned Outputs
Output Achieved
Before Extension
(October 2008–
August 2014)
Achievement as
Percentage of the
Planned Output
before Extension
Output Achieved
during Project
Extension Period
(February 2016)
Additional
Achievement as
Percentage of the
Planned Output
Total Percentage of
Achievement of the
Planned Output
during the Project
Life
management staff at
the PPMUs
gaps on trained staff,
especially for M&E
purposes
Provision of necessary office equipment Office equipment
was provided to the
CPMU. However, some PPMUs were
difficult to be fairly
equipped.
Achieved
Office equipment was
provided to the CPMU.
However, some PPMUs had difficult to be fairly
equipped.
Achieved Achieved
Financing of incremental operating costs The project activities were financed
properly. Achieved
Some PPMUs had constraints to finance
operational costs during
the extension phase.
Partially achieved Partially achieved
M&E activities Baseline data
collection
As scheduled, this
was completed in the
first semester of 2009
Achieved — — Achieved
Indicators’ update Indicators’ update was done based on
surveys (2009, 2012,
and 2014) and project administrative
records
Achieved
Some indicators were not updated after the
project extension Partially achieved Partially achieved
MTR
MTR was done in 2012
Achieved — — Achieved
End-of-project
completion reports
The first end-of-
project completion report was done in
July 2014 (regular
project time)
Achieved
The second end of
project completion report was done for the
extension phase in
February 2016 (project extension phase)
Achieved Achieved
Audits Project financial
audits were done for
procurement and financial
management with
some delays*9
Partially achieved — — Substantially
achieved
Source: Center for Environment and Health Studies (2014) FINAL REPORT: End-line evaluation of the Northern Uplands Health Support Project, Hanoi, 2014, and Center for Environment and Health
Studies (2016) FINAL EVALUATION OF THE NORTHERN UPLANDS HEALTH SUPPORT PROJECT IN THE EXTENSION PHASE, Hanoi, 2016.
(*1) After the project closing in February 2016, there were 10 remaining Level 1 medical doctors expecting to be graduated.
(*2) After the project closing in February 2016 there were 409 remaining assistant doctors having training to be graduated medical doctors. So, the total assistant doctors supported by the project were
1467. (*3) Training was offered at provincial, district and village level according the specialty.
(*4) The Government had their own long standing system and mechanism for awarding well-performing hospitals. They did it every year as a routine activity. However this did not appear as a project
related activity and the results of this award mechanism were not reported to the Bank in the project documents.
47
(*5) The last survey was related with the original project closing. During the extension project phase a new survey was not planned given that the extension was not planned to follow-up the Project Component 2.
(*6) This achievement is related to the project regular time. The increase of the number of beneficiaries entitled to receive benefits is related to the inclusion of all ethnic minorities as project
beneficiaries by May 2011. In the first three months of 2014 the district hospitals under the project provided subsidies to 59,182 beneficiaries. From 2009 to 2013, the number of ethnic minorities’ inpatients receiving subsidies increased from 1054 to 244,181 totalizing more than 545 thousand inpatient along this period.
(*7) Despite the 98% of the target achievement compared to the plan in the regular project time, the Project Evaluation Report of the Extension phase recommend strengthening of the IEC activities
under the project. (*8) The implementation progress of some training courses and provision of medical equipment were slightly slow according the project registers.
(*9) The delays in the project financial audits were registered in the beginning of the project. In the last 3 years of execution and during the project extension, the audit reports were submitted on time.
48
Annex 3. Economic Analysis: Output Efficiency, Benefits, and Equity Impacts
Introduction
1. This annex addresses the project’s efficiency by analyzing (a) the rationale of the
Government interventions in the project and in the Northern Upland Provinces; (b) the
efficiency in achieving quality access for the district hospitals by the NUP population (project
outputs); (c) the project contribution to improve health benefits (project outcomes); and (d)
the impact of the project interventions on the equity on health access and health spending in
benefit of the poor and ethnic minorities.38 According to the project economic analysis (annex
9 of the PAD), the expected benefits were associated with health outcome improvements for the
NUP areas’ population and to narrow the health spending gap between the poor and the average
population. The benefits would be achieved by the following main interventions: (a) improve
efficiency on the health services delivery by increasing supply of skilled human resources and
refurbishing and equipping the district hospitals; (b) improve efficiency for the poor on assessing
health services by removing financial barriers to increase health care services utilization; and (c)
reduce the risk of impoverishment for the NUP poor and ethnic minorities populations.
Rationale of the Government Interventions
2. As stated in the PAD, the Northern Upland Provinces constitute the most
disadvantaged region in Vietnam, with an unusually high concentration of poor and ethnic
minorities living in sparsely populated, mountainous localities under difficult circumstances.
The Government’s choice to invest in this region was based on the unfavorable socioeconomic
conditions and the difficult access and generally poor quality of the health services, especially at
the district health hospitals. The Government seeks to improve the health status of the population
and reduce the health gap between the NUP and the rest of the Vietnamese population. The project
achieved this by improving the efficiency on delivering health outputs, reducing maternal mortality
in higher proportion than the country’s average, and reducing the equity gap in out-of-pocket
health spending between the poorest income quintile and the richest income quintile.
Efficiency on Achieving the Project Outputs
3. The project surpassed all relevant planned output targets increasing efficiency by
reducing unitary costs. Annex 3A shows the project’s planned and achieved output targets and
38 According the PAD, a quantitative economic analysis based on costs was not feasible for the project because there
was no empirical basis for estimating the project’s health outcome costs. Neither an economic rate of return nor a
net present value of the benefits of the project was calculated/forecasted upfront during the project appraisal. The
PAD highlighted the difficulty to estimate costs of the proposed interventions. Given the special conditions to
implement project activities in the NUP areas, the economic costs may differ significantly from their financial costs
or budget expenditures. For example, under Component 1, the opportunity cost of the personnel sent for training is a
significant economic cost of the project that may not be completely reflected in the project budget. It has been the
experience of other projects (for example, the ADB-supported Health Care in the Central Highlands Project) that
sending large numbers of personnel for training from facilities that are already understaffed imposed a serious
budgetary burden on the public health system. Another example is associated with the reimbursement of the travel
and food costs of poor hospital inpatients, which is an income transfer, not an economic cost. Economic costs in this
case would be limited to the cost of administering the transfers and the cost of any additional health care utilization
that might result from this support.
49
the correspondent budget allocations. These outputs, related to training human resources and
equipping, repairing, and upgrading infrastructure in the district hospitals, had targets that were
surpassed during the project execution with simultaneous reductions in the costs, thus increasing
savings that allowed the project to execute additional activities during the extension phase. For
example, the implicit unitary cost for the level-1 specialists and the transformation of assistant
doctors to medical doctors were, at the end of the project, 50 percent and 37 percent lower than
the original plans, while the number of individuals trained as level-1 specialists and medical
doctors were 101 percent and 50 percent higher than planned.
Analysis of the Contribution to the Project Outcomes
a) Benefits associated with reducing mortality
4. The analysis of the project interventions and PDO achievements (annexes 2 and 5 of
this ICR) evidenced improvements in the access and utilization of health services in the NUP
areas. This contributed to the reduction of maternal and child mortality rates, which are two major
problems in the burden of diseases of the NUP areas. The project interventions were intended to
improve the quality and efficiency in delivering prenatal care, birth delivery, and childcare in the
first 12 months of life. So, the reduction of maternal and child mortality rates along the project
implementation should be relevant benefits reflecting improvements in the health status of the
NUP areas’ population.39
5. Current literature evidences that the project used the right interventions to reduce
maternal and child mortality rates. Most maternal and newborn deaths can be prevented using
existing, proven, cost-effective interventions, such as clean delivery packages, composed of
antibiotics, sterile blades for cutting umbilical cords, drugs that prevent and treat postpartum
hemorrhage, resuscitation, immediate and exclusive breastfeeding, and education and
communication to the mother to keep the newborn warm with skin-to-skin contact and
breastfeeding.40 Increasing access to mothers to deliver their babies at first-level facilities and/or
by SBAs was the focus of the project, providing an opportunity to expand quality services around
the time of birth. All these initiatives were used by the project interventions, providing a solid
ground to start a process to reduce maternal, neonatal, and infant mortality in the northern
mountainous provinces in Vietnam.
6. As can be seen, between 2007 and 2014, the maternal mortality reduction was
remarkable in the NUP areas when compared with the average reduction verified in the
country, but it is still very high in the region compared with the national benchmark and
international standards. During the project regular time implementation, MMR in the NUP areas
reduced 40 percent compared with 7 percent in the whole country. However, MMR in the seven
NUP provinces is still nearly two times higher than the country average, as seen in table 3.1.
39 However, given that both—maternal and infant mortality—are affected by many factors, such as water and
sanitation conditions, nutrition patterns, and others, it is difficult to attribute the results (for good or for bad) only to
the health interventions. 40 Bill and Melinda Gates Foundation. 2016. Maternal, Newborn and Child Health Strategy Overview.
http://www.gatesfoundation.org/What-We-Do/Global-Development/Maternal-Newborn-and-Child-Health
50
Table 3.1. Maternal, Neonatal, and IMR Vietnam and the NUP: 2007–2014
Years MMR
(per 100,000 born alive)
NMR
(per 1,000 born alive)
IMR
(per 1,000 born alive)
Vietnam NUP Vietnam NUP Vietnam NUP
2007 58 178 13.0 11.2 21 31.1
2014 54 106 12.0 10.8 18 29.4
Percent of
reduction
−6.9 −40.4 −7.7 −3.6 −14.3 −5.4
Source: Vietnam: World Bank Data (http://data.worldbank.org/indicator/SP.DYN.IMRT.IN?locations=VN;
http://data.worldbank.org/indicator/SH.STA.MMRT?locations=VN;
http://data.worldbank.org/indicator/SH.DYN.NMRT?locations=VN). Government of Vietnam: NUP Project
Surveys 2009 and 2014.
7. The project opened the floor to continue achieving progress on reducing MMR in the
region and the relevance to keep it sustainable along the time. A study on maternal and neonatal
mortality financed by the CPMU as part of the project closing evaluation activities found that in
2014, the still high MMR in the NUP areas is related to: (a) the high proportion of home birth
deliveries, representing 38 percent of the total birth deliveries in the NUP41; (b) delays in detection
and decision to seek care by women and their families; (c) delays in reaching care, mainly due to
difficult geographic conditions and lack of transportation means; and (d) delays in receiving health
care and appropriate treatment at the health facilities, mainly due to shortcomings in compliance
with the process of care and monitoring of pregnant mothers, especially during and after delivery.
The reduction registered in the MMR between 2007 and 2014 was not uniform among the seven
provinces and in one of them (Bac Kan) the MMR increased during the project implementation
(figure 3.1).
Figure 3.1. MMR in Six Northern Upland Provinces
Source: Ministry of Health of Vietnam: Survey on Maternal and Neonatal Mortality in the Northern Upland Provinces: 2007-2008 and
2013-2014.
8. Though there is no information related to MMR in the 2015 report, the project
extension phase provided training for midwives to reduce maternal mortality at home, thus
providing safer birth delivery. The administrative registration of the NUP health services showed
41 According to the study, the risk of maternal mortality in a home delivery is 3.7 times higher than when deliveries
are performed at health facilities.
409
215
167143 133
46
178
83
150
104 95
128
75
106
0
50
100
150
200
250
300
350
400
450
Dien Bien Lai Chau Son La Cao Bang Lao Cai Bac Kan Total
2007-2008
2013-2014
51
that between 2014 and 2015, the proportion of mothers giving birth at home supported by village
midwives increased from 6.4 percent to 16.5 percent and the proportion of clean delivery packages
provided to the mothers giving birth at home increased from 34 percent to 43.1 percent in the same
period respectively. The continued training of village midwives and policies aimed at maintaining
a network of village midwives show that the possibilities to increase the project benefits in the
extension phase were very high, given that these solutions are crucial for safe motherhood in the
mountainous areas.
9. The reduction of neonatal and infant mortality in the NUP areas was lesser than the
country average. Table 3.1 shows that IMR and NMR reduced by 5.4 percent and 3.6 percent in
the NUP between 2007 and 2014, compared with 14.3 percent and 7.7 percent in the national
average, respectively. These results were also influenced by the high proportion of home birth
deliveries in the NUP areas. Neonatal mortality among newborns delivered at home was 14.2 per
1,000 live births, almost twice as much as in health facilities, according to the end-of-project
survey.
10. Figures 3.2 and 3.3 show that the NMR and IMR increased in some of the provinces
along the project execution period, despite the decreasing average. In fact, in the case of NMR,
only Bac Kan and Son La presented decreases in NMRs and IMRs. In the case of IMRs, decreases
were verified in Ha Giang, Cao Bang, Son La, and Bac Kan. Given that most of the data is from
administrative registrations, maybe the reporting system in some provinces, such as Lai Chau and
Dien Bien, had improved sharply during (and because) the project implementation, increasing the
confidence in the administrative records of NMR and IMR in 2014, compared with 2008, when
the mortality data were not captured well.
Figure 3.2. NMR in Six Northern Uplands Provinces
Source: Ministry of Health of Vietnam: Survey on Maternal and Neonatal Mortality in the Northern Upland Provinces: 2007-2008 and
2013-2014.
11.5
13.8
12.3
9.1
7.4
11.711.2
17.1
14
12.812.1
5.6 5.9
10.8
0
4
8
12
16
20
Lai Chau Lao Cai Cao Bang Dien Bien Bac Kan Son La Total
2007-2008
2013-2014
52
Figure 3.3. IMR in Six Northern Uplands Provinces
Source: Ministry of Health of Vietnam: Health Statistical Yearbooks 2008 and 2014.
11. The disadvantaged situation of the MDGs in the NUP areas, revealed during the
project implementation by the improvement of the medical records, lead the Government to
focus the project extension on reducing the MMR and IMR, striving to achieve the MDGs on
health, by providing clean birth delivery packages, supporting basic tools for newborn care,
equipment for neonatal units at the district hospitals, and related training. Despite the outputs
related to these tasks being positively recorded (see annex 2), their impact on the reduction of IMR
and NMR after the project extension was unknown at the time this ICR was prepared.
b) Benefits associated to improved supply and quality of health services
12. The project interventions (health staff training, management improvement, and
investment in infrastructure, equipment, and maintenance) increased the availability and
quality of the essential health services delivery in the district hospitals benefiting the
population in the NUP areas. The number of inpatient and outpatient visits per capita delivered
in the district hospitals increased by 290 percent and 270 percent, between 2009 and 2015,
respectively, according to the project administrative records.
13. However, more important than the number and coverage of the services delivered is
the quality of these services. Table 3.2 shows indicators associated with the benefits provided by
the project in improving the quality and capacity to deliver health services to mother and children
by district hospitals. The number of health services that district hospitals can according to the
national norms increased by 105%. Hospitals that have equipment needed for quality procedures,
such as positive airway pressure machines, phototherapy for newborn machines, oxygen breathing
systems and newborn resuscitators increased remarkably. High improvements were achieved by
following the national quality norms, enhancing diagnosis capacity, and having skilled personnel
and equipment to implement appropriate techniques for regular and emergency care services.
40 40
33 33
28
2321
31.1
34.2
24.3
34.4
42.6
23.2
29.2
17.9
29.4
0
10
20
30
40
50
Ha Giang Cao Bang Dien Bien Lai Chau Son La Lao Cai Bac Kan Total
2008
2014
53
Table 3.2. Quality Improvements of the Services Delivered by the District Hospitals in the NUP Areas:
Project Baseline (2009), Regular Closing Date (2014), and Extended Closing Date (2015)
Health Services Quality Indicator
Project
Baseline
(%)
Project
Regular
Closing
Date
(%)
Project
Extension
Closing Date
(%)
Quality
Improvements
at Extension
Closing Date
(%)
Percentage of health services that district
hospital can provide according to the national
norms
39 71 80 105
Percentage of medical records with appropriate
diagnosis of acute respiratory infections 18 39 62 244
Percentage of medical records with appropriate
diagnosis of acute diarrhea 41 69 82 200
Percentage of medical records with appropriate
diagnosis of acute poisoning 61 86 89 46
Percentage of district hospitals equipped with
skilled staff to implement appropriate pediatric
techniques
69 75 85 23
Percentage of district hospitals equipped with
skilled staff to implement appropriate obstetric
techniques
65 84 88 35
Percentage of district hospitals having capacity
for caesarean section. 86 89 92 7
Percentage of district hospitals having capacity
for blood transfusion 64 75 83 137
Percentage of district hospitals having breathing
machines with continuous positive airway
pressure
25 75 80 220
Percentage of district hospitals having light for
jaundice phototherapy treatment 21 82 88 219
Percentage of district hospitals having oxygen
breathing systems 43 75 78 81
Percentage of district hospitals having newborn
resuscitators 46 93 92 100
Source: Center for Environment and Health Studies (2016), Final Evaluation of the Northern Uplands Health Support Project in The Extension
Phase, Hanoi, 2016.
c) Benefits associated with increased HI coverage and health services demand to prevent
catastrophic expenditures for the poor and ethnic minorities
14. The project increased not only the coverage, but also the demand for health services
utilization by providing transportation and meals subsidies to the poor and ethnic minorities.
Because of adjustments, the number of project beneficiaries increased significantly. From 2009 to
2011, only the poor were covered by the project, but given that the majority of the ethnic minorities
were quasi-poor, they were included as beneficiaries of the subsidies since 2012. The number of
project recipients, were 730,183 persons in 2009 (25 percent of the NUP population) when
compared with 3,185,341 in 2013 (82 percent of the NUP population). Since 2011, the project
implemented a communication campaign (IEC) to increase the beneficiaries’ awareness about HI
rights and the processes to achieve their subsidies. The number of beneficiaries knowing at least
three HI rights increased substantially.
54
15. The number of beneficiaries receiving direct monetary subsidies for transportation,
meals, and health expenditures, increased several times along the project implementation, as
can be seen in figure 3.5.42 They were only 1,000 in 2009 but increased to almost 245,000 in 2013.
From the project start in 2009 to January 2014, the number of poor/ethnic minority inpatient who
received support to use medical services from the project totaled 545,423, with registered
expenditures of USD7,398,693.69 (around USD13.56 per inpatient beneficiary). As of January
2014, 86 percent of the budget for this activity has been spent, but the sustainability of these
subsidies was granted during the project extension and hopefully beyond, according the CPMU
information.
Figure 3.4. Number of Poor/ethnic Minorities Benefited by Inpatient Heath Subsidies (transportation, meals,
health expenditures) by the NUP Project 2009–2013
16. The subsidies paid to increase coverage and access to health services were crucial to
reduce out-of-pocket health-related spending, prevent catastrophic expenditures, and
increase the coverage of HI in the NUP areas. According to the CPMU sponsored surveys,
between 2009 and 2014, the percentage of households which experienced catastrophic health care
expenditures in the NUP had a remarkable reduction: from 15 percent to 2 percent, respectively.
The percentage of population living in these areas with HICs increased from 82 percent to 95
percent between June 2009 to December 2015, mostly among the poor and ethnic minorities
populations.
Analysis of the Project Impact on Equity Pro-poor and Ethnic Minorities
(a) Methodological considerations
17. The equity analysis of the project interventions will consider the impact of the project,
mostly from 2012 to 2014. During 2009 and 2011, most of the efforts of the NUP Project were
concentrated on planning and implementing the process to provide training for health staff,
42 The project’s 2009 Baseline Survey showed that, of the inpatients in the NUP areas who visited the district
hospitals in 2008 to assess health care, 96 percent paid for food, 88 percent paid for transportation, and 5 percent
paid for other costs, such as medication, exams, tests, and so on. The inpatient average daily cost paid by the
families was VND 122,224 (USD 7.46 at the 2008 exchange rate) and the average stance was 4.4 days, representing
an average cost of USD 32.80 per inpatient. To finance these costs, 33 percent of the families got loans, 22 percent
asked for family support, and 5 percent sold assets or means of production.
1054 1939079469
200709
244801
2009 2010 2011 2012 2013
Number of Beneficiaries
55
procurement for investments in infrastructure and equipment of the district hospitals, and the
institutional arrangements to finance incentives to the poor and ethnic minorities to increase the
access of health services and utilization of district hospitals. From 2012 to 2015, the project was
able to measure the impact of these investments for improving the equity on accessing the project
benefits, especially to the poor and ethnic minorities.
18. This equity analysis will consider the progress on equity indicators related to (a)
affiliation to HI; (b) household income spent on health; and (c) household income spent on
inpatient visits. It is based on the data collected through the VHLSS of the respective years. Equity
impact will be calculated on variables such as gender and age of the beneficiaries, ethnicity,
residency status (rural or urban), and income (poorest, near poor, middle, near richest, and richest
income quintiles).
19. Main hypotheses. The main hypotheses used in this analysis are the following: (a) the
project improved the equity on accessing inpatient visits, disproportionally benefiting women and
children over five years; (b) the project reduced the relative spending of the poor and ethnic
minorities to access inpatient visits; (c) the project increased the number of inpatient visits of the
poor in the district hospitals, increasing the equity of the access to hospital services43; (d) the
project increased the proportion of the poor and ethnic minorities having HI; and (e) the project
reduced the participation of health spending in the poor families’ out-of-pocket expenses,
increasing the equity of the health spending.
(b) Equity impact on inpatient medical visits for women, children, and rural population
20. Given that the project was focused on achieving the health MDGs, it was expected
that the proportion of inpatient visits related to mother and children would increase faster
than for other groups, increasing the gender and age equity on the access to the district
hospitals. The equity impact is measured by the equity ratio.44 As shown in table 3.3, between
2010 and 2014, women inpatient visits increased 32 percent as a proportion of men inpatient visits,
and children inpatient visits increased 181 percent as a proportion of total inpatient visits at the
district hospitals. Rural population inpatient visits also increased 19 percent as a proportion of
urban medical visits during the same period. So equity in accessing inpatient services at district
hospitals was improved for women (because women at a reproductive age always need more
services than men because of reproductive health issues), for children until they are five years old
(where the associated mortality risks are higher), and for the rural population, who were previously
underserved by the health system.
43 Considering the nature of the Project investments, the impact on health services utilization in equity is more
sensitive to the inpatient than to outpatient visits to the district hospitals. The Project influenced the Provincial
Departments of Health to functionalize the health referral process, according the levels of complexity in the health
care provision, inducing the population to do not use district hospitals for unnecessary outpatient services for that
level of complexity. So, many outpatient visits that in the past went to district hospitals (especially among the center
village populations) were redirect to Community Health Centers. So this analysis is pertinent only for inpatient
services. 44 The equity ratio for inpatient visits is defined as the proportion of the visits per capita in the category where the
number of visits per capita is expected to increase disproportionally as a proportion of the complement (for example,
women compared to men) or the total visits per capita (poor compared to the total population). If this ratio has
positive variations, the equity impact was achieved, but if the ratio has negative variation, the equity was reduced.
56
Table 3.3. Equity Gap on Annual Per Capita Inpatient Visits at the District Hospitals
NUP Provinces: 2010–2014
Annual Per Capita inpatient Visits:
Equity Ratios for Different
Population Groups
2010 2012 2014
Increase
2010–2012
(%)
Increase
2012–2014
(%)
Increase
(2010-2014)
(%)
Gender Equity Ratio
Women visits/men visits 1.14 1.46 1.50 28 3 32
Child Visits Equity Ratio
0–5 years old visits/average visits 1.00 1.13 2.81 13 149 181
Place of Residency Equity Ratio
Rural visits/urban visits 0.94 0.78 1.12 −17 44 19 Source: VHLSS, 2010, 2012, and 2014.
(c)Equity on the impatient visits to the district hospitals benefiting the poor and near poor
21. The project did not collect data for the near poor, because its focus was the poor and
ethnic minorities. However, the information of VHLSS use the classification of poor and near
poor in two ways: (a) The global poverty line (the World Bank criteria); and (b) by income quintile,
which means that the poorest should be considered the first quintile and the near poorest are the
second quintile. Table 3.4 shows the impact of the project in the inpatient visits to the district
hospitals by poverty status and income quintiles.
Table 3.4. Inpatient Visits to the District Hospitals According Poverty Status and Income Quintiles
Poverty Status and
Income Quintiles
Inpatient Visits to the District Hospitals per inhabitant
2012 2014 Increase (2012-2014)
(%)
Poverty Status (according World Bank Global Poverty Line)
Poor 0.037 0.054 45.9
Near-Poor 0.088 0.070 -20.5
Non-Poor 0.056 0.075 33.9
Income Quintiles
Poorest 0.028 0.044 57.1
Near-Poorest 0.034 0.061 79.4
Middle 0.048 0.063 31.3
Near-Richest 0.083 0.093 12.0
Richest 0.047 0.053 12.8
Average 0.048 0.063 31.2 Source: VHLSS, 2012 and 2014.
22. Table 3.4 shows that the inpatient visits in the district hospitals, between 2012 and
2014, had a higher increase for the poor than for the average population and the non-poor.
This is a strong evidence of the positive impact of the Project in the equity of the health services
utilization at the district hospitals level. The inpatient visits per inhabitant increased in average
31% compared with 57% in the poorest quintile and 46% among the poor. Regarding the near poor,
the income quintile approach shows an increase of 79% in the near poorest quintile. However,
using the global poverty line approach, the data shows a reduction of 20% of the number of
57
inpatient visits between 2012 and 2014, which could be associated with methodological
differences in the way that the information associated with each year was captured45.
(d) Equity on the relative spending of the poor and ethnic minorities to access inpatient visits
23. Another way to verify the equity impact is the proportion of the out-of-pocket
expenditures of mothers and children and the poor and ethnic minorities on inpatient visits
to the district hospitals compared to other groups, along the project implementation. The
VHLSS data shows that the spending per inpatient visit at the district hospitals as a share of project
target groups was relatively reduced. As can be seen in table 3.5, the equity ratio for the inpatient
visit spending improved for women compared to men; for children compared to the average
population; for ethnic minorities compared to the Kin/Hoa ethnicity; for rural populations
compared to urban, and especially for the poorest quintile compared with the average population.
The relative reduction of the out-of-pocket spending with inpatient visits verified for the poor was
supported by the project subsidies for meals, transportation, and medical expenses, the last covered
by the HI, during the project implementation.
Table 3.5. Equity Gap in Out-of-Pocket Spending for Inpatient Visits to District Hospitals NUP Provinces:
2010–2014
Out-of-Pocket Spending per
Inpatient Visit: Equity Ratios for
Different Population Groups
2010 2012 2014
Increase
2010–2012
(%)
Increase
2012–2014
(%)
Increase
(2010–2014)
(%)
Gender Equity Ratio
Women/men spending 1.01 0.89 0.91 −12 2 −10
Child Visits Equity Ratio
0-5 years old/average spending 0.72 0.79 0.53 10 -33 −26
Ethnic Minorities Equity Ratio
Ethnic minorities/Kin-Hoa spending 1.19 0.56 1.13 −53 102 −5
Place of Residency Equity Ratio
Rural/urban spending 1.34 1.89 1.11 41 −41 −17
Income Equity Ratio
Poorest quintile/average spending 1.04 0.69 0.60 −34 −13 −42 Source: VHLSS, 2010, 2012, and 2014.
(e) Equity impact of the project in increasing HI coverage
24. The VHLSS data also shows relevant impacts in the coverage of HI during the project
implementation time, especially for the poor. The total NUP population without HI, decreased
slightly between 2010 and 2014 (from 7.4 percent to 7.2 percent, respectively) and the HI coverage
for the poor improved. The percentage of the poorest economic quintile affiliated to the HI
increased from 98.6 percent to 99.2 percent between 2010 and 2014 and for the near-poor (second
poorest quintile) the HI coverage increased from 93.5 percent to 98.9 percent, according the
VHLSS data. As part of the affiliation to the HI mechanisms for the poor, the project spent USD
8.5 million in subsidies for the poor and ethnic minorities, contributing to pay for transportation
45 The data for 2012 used the international poverty line (in purchasing parity power –PPP) of and income of USD
1.25 a day. However, this line was upgraded to US$ 1.90 in the 2014 data analysis, creating difficulties to compare
both years using the World Bank global poverty line criteria.
58
and meals for 596,700 poor and ethnic minorities’ inhabitants of the NUP areas, with an average
expenditure of USD 14.27 per medical visit.
(f) Equity impact on reducing the share of out-of-pocket expense in health for the poor and
ethnic minorities
25. During project implementation, the poorest quintile reduced the health spending as
a share of the out-of-pocket expense in the NUP areas. From 2010 to 2014, the share of out-of-
pocket health spending of the poorest quintile was reduced slightly from 6.7 percent to 6.5 percent,
while the average out-of-pocket family spending on health increased from 7.9 percent to 8.3
percent in the NUP areas. Probably health subsidies to the poor and ethnic minorities for visiting
district hospitals and improvements in the health care assistance at the villages, sustained partially
by the project, had positively affected the family budgets, contributing to avoiding the risk of
catastrophic health expenditures for these populations. That is one of the reasons why the
proportion of the NUP families with catastrophic health expenditures reduced from 10.4 percent
to 2.0 percent during the project execution, as is demonstrated by the project KPI #3.
Final Considerations
26. The present economic analysis intends to demonstrate the following: (a) the project
was efficient in delivering its outputs, reducing unitary costs for training, equipment installation,
and civil works during implementation, compared with the original implicit costs. This allowed,
savings from the original implementation time to be used during the project extension in new
activities related to improving the MDGs in the project area and (b) efficient project interventions
and subsidies to the poor and ethnic minorities contributed to reducing maternal, neonatal, and
infant mortality and improved the equity in assessing health care and reducing health spending for
the poor and ethnic minorities.
59
Annex 3A. Estimated Unitary Costs of the Project Outputs According to the Original Plan (2009) and Closing Implementation
(2016)
Component and Output
Project Outputs Budget (USD, thousands) Estimated Unitary Cost (USD)
Planned Implemented Variation
(%) Planned Implemented
Variation
(%) Planned Implemented
Variation
(%)
Trained Level-1 specialists 183 367 101 691.7 687.8 −1 3,779.78 1,874.11 −50
Assistant doctors trained as
doctors*1 741 1,156 56 6,236.0 6,085.6 −2 8,415.65 5,264.36 −37
Short-term-doctors trained in
curative care techniques*2 1,334 4,965 272 2,409.3 1842.3 −24 1,806.07 371.00 −79
Short-term-doctors trained in
preventive care 213 336 58 211.0 200.1 −5 990.61 595.53 −40
Skills techniques transfer to
district hospitals*3 402 670 67 365.6 296.9 −19 909.45 443.13 −51
Basic medical equipment
(number of units installed) 4415 5165 17 26,698.5 25,648.5 −4 6,047.23 4,965.83 −18
Minor repairs and upgrade of
district hospitals (number) 10 18 80 3,465.9 3,465.9 — 346,590.00 192,550.00 −44
Training of district hospital
management staff 303 753 248 729.5 493.4 −32 2,407.59 655.25 −73
Note: *1 Includes 98 trained pharmaceutical assistants transformed to pharmacists which was not planned at the beginning of the project. *2 Includes 1,542 health staffs completing short-term training on MDG-related activities during the extension phase. *3 Times of skill transfers. The budget includes expenses to support mobilized staff.
60
Annex 4. Bank Lending and Implementation Support/Supervision Processes
(a) Task Team members
Names Title Unit
Lending
Bukhuti Shengelia Task Team Leader (left the World Bank)
Mai Thi Nguyen Team Member GED02
Nga Quynh Nguyen Senior Program Assistant EAPDE
Maryam Salim Team Leader MDI
Hoi Chan Nguyen Country Counselor (retired)
Samuel S. Lieberman Task Team Leader (retired)
Kelichi Ohiri Health Specialist/Team Member (already left the
World Bank
Lingzhi Xu Senior Operation Officer GHN03
Marko Vujicic Economist (left the World Bank)
Lan Thi Thu Nguyen Safeguards Specialist GEN2B
Hung Viet Le Financial Management Specialist EAPCO
Supervision/ICR
Mai Thi Nguyen Team Member GED02
Kari L. Hurt Team Leader GHN06
Anh Thuy Nguyen Team Leader GHN02
Bukhuti Shengelia Task Team Leader (left the World Bank)
Andre C. Medici ICR Author GHN04
Hoang Xuan Nguyen Procurement Specialist GGO08
Mai Thi Phuong Tran Senior Financial Management GGO20
Sang Minh Le Environment Safeguards Specialist GHN02
Giang Tam Nguyen Social Safeguards Specialist GSU02
Nga Thi Anh Hoang Program Assistant EACVF
Nghi Quy Nguyen Social Development Specialist GSU02
Trang Phuong Thi Nguyen Safeguards Specialist EASVS
Maryam Salim Team Leader MDI
Nguyen Hoang Nguyen Procurement Specialist GGODR
Quynh Xuan Thi Phan Financial Management Specialist GEFPO
Maya Razat Program Assistant GSP
Minh Thi Hoang Trinh Program Assistant AFCNG
Nga Quynh Nguyen Senior Program Assistant EAPDE
Duong Minh Duc Public Health Consultant
61
(b) Staff Time and Cost
Stage of Project Cycle
Staff Time and Cost (Bank Budget Only)
No. of Staff Weeks US$, thousands (including
travel and consultant costs)
Lending
FY06 19.89 96.00
FY07 31.35 246.50
FY08 34.80 145.50
Total: 86.04 488.00
Supervision/ICR
FY09 24.50 83.70
FY10 26.50 96.80
FY11 17.00 75.00
FY12 14.00 53.50
FY13 22.00 70.20
FY14 17.30 49.50
FY15 14.40 42.00
FY16 26.80 100.50
Total: 162.50 571.00
62
Annex 5. Results Framework - Analysis of the PDO Achievement
Introduction
1. The PDO has three parts: (a) PDO 1 - increase the utilization of district hospital services;
(ii) PDO 2 - improve the quality of district-level hospitals; and (iii) PDO 3 - reduce financial
constraints to access health services. The objective of this annex is to attribute the KPIs and IOIs
to the three parts of the PDO and to rate the PDO (and its parts) according of the achievement of
the corresponding indicators.46
2. The methodology to define the indicators’ ratings is the following: For quantitative
indicators, the achievement at the end of the project is compared with its end target. If the result is
above 105 percent the indicator was surpassed; if it is between 95 percent and 104 percent, it was
achieved. If it is between 85 percent and 94 percent, it was substantially achieved. If it ranges
between 65 percent and 84 percent, it was partially achieved, and if it is lower than 65 percent, it
was not achieved. For qualitative indicators, the classification is only achieved (if the qualitative
target was accomplished) and not achieved (if it was not accomplished). The IOI #7, with no
reliable information about baselines and targets of values of achievement will not be considered
as part of the PDO rating.
3. The criteria used to calculate the indicator achievement is the following: (a) if the baseline
is not zero, it is calculated on the difference between what was intended (baseline) and actual47
and divided by the difference between the target and the baseline; (b) if the baseline is zero it is
calculated on the coefficient between the actual and the target; and (c) if the target is lower than
the baseline, it is calculated on the coefficient between the actual and the baseline.
4. Annex 5A presents a table calculating the rating of each indicator according to
achievements recorded during project implementation. This table has the following columns: (a)
original indicators (according to the PAD); (b) indicators added during project implementation;
(c) value and date of the indicator baseline; (d) value and date of the indicator target; (e) value and
date of the indicator at the project’s original closing date of August 31, 2014; (f) percentage of
target achieved at the project’s original closing date; (g) value of the indicator at the project’s
revised closing date of February 29, 2016; (h) percentage of target achieved at the project’s revised
closing date; and (i) indicator achievement rate according to the methodology presented in
paragraph 2.
5. The rating of the PDO is attributed to the proportion of the indicators’ values that have
been surpassed, achieved, or substantially achieved as a share of the total project indicators. It is
high, when more than 95 percent of the indicators’ target values have been surpassed, achieved, or
substantially achieved; substantial, from 75 percent to 94 percent of achievement; modest, from
50 percent to 74 percent, and negligible when less than 50 percent of the indicators have met their
target values.
46 PDOs and IOIs will have the same weight to classify the PDO’s achievement. 47 A= (Ia-Ib)/(It-Ib), where A is achievement, Ia is indicator’s actual; Ib is indicator’s baseline, and It is indicator’s
target.
63
6. This annex has three tables. Table 5.1 shows the KPIs and IOIs distributed according to
the three parts of the PDO. Table 5.2 summarizes the results found in annex 5A, and table 5.3
summarizes the PDOs’ rating to measure the project efficacy.
Project KPIs and IOIs related to the PDO
7. Number of indicators. According to the PAD, the project had 4 KPIs and 10 IOIs. All
KPIs and IOIs #1 to #7 were related to the PDO. The other IOIs (#8 to #10) were designed to
measure project management performance. Some IOIs were complex to be measured through just
one indicator. Then, during the project implementation, the Government and the World Bank
agreed on creating subindicators to measure these complex and multidimensional IOIs.
Accordingly, IOIs #2, #3, and #4 were measured by nine, three, and two subindicators, respectively.
Additionally, other IOIs were included during the project execution without a formal project
restructuring. Other administrative indicators were also included using similar processes. The last
project ISR,48 issued in February 2016, lists a total of 25 KPIs (4) and IOIs (21). Table 5.1 shows
the PDO parts 1, 2, and 3 and the corresponding KPIs and IOIs. PDO part 1 was measured by 2
indicators, PDO part 2 by 19 indicators and subindicators, and PDO part 3 by 4 indicators.
Table 5.1. Distribution of the Project Indicators (KPIs and IOIs) according to the PDO parts.
48 The project documentation does not reflect when all these indicators and the corresponding baselines were set and
included. They are not in the PAD and start to appear only in ISR #6 (issued in August 2013). From 2008 (project
starting) to 2013, the five project ISRs do not have clear information on the project M&E and RF.
PDO’S PARTS KPI (*) IOI * IOI Sub Indicators **
PDO #1:
Increase
utilization of
district health
services.
(2 indicators)
KPI #1: Increase
utilization rates of
inpatient services in
district hospitals among
Decision 139 beneficiaries
KPI #2: Increase
utilization rates of
outpatient services in
district hospitals among
Decision 139 beneficiaries
PDO #2:
Improve the
quality of
district-level
hospitals
(19 indicators)
KPI #4: Proportion of
district hospitals that
provide full set of health
services according to the
national norms (Decision
23/205/QB- BYT)
adjusted to the Northern
Uplands
IOI #1: Percentage of patients
satisfied with the health services
IOI #2: Adherence of treatment
protocols for selected conditions
in inpatient settings (based on
six subindicators)
IOI #2.1: Percentage of health workers
with knowledge of diagnosing and
treating Level A/B/C dehydrated
diarrhea
IOI #2.2: Percentage of health workers
with knowledge of diagnosing and
treating severe pneumonia
IOI #2.3: Percentage of health workers
with knowledge of diagnosing and
treating poisoning
IOI #2.4: Percentage of reasonable
diagnoses of severe pneumonia
IOI #2.5: Percentage of clinical health
workers’ reasonable diagnosis of general
pneumonia
64
Note: * According to page 34–36 of the PAD.
** According to project ISRs Sequence #6 to #11 (last).
8. Indicators’ baseline. Despite the fact that annex 3 of the PAD presents the project RF with
the project indicators (KPIs and IOIs), the RF was incomplete and most of the baselines and targets
were revised in 2008 (VHLSS) and 2009 (Baseline Survey). Different dates for project baselines
could be found. Some indicators incorporated during project implementation do not have baselines.
Some indicators should report progress twice during the project life (Year 3 and Year 6). This is
the case for KPIs #1, #2, and #3 and IOIs # 3, #5, and #6. The survey developed for the MTR
IOI #2.6: Percentage of reasonable
diagnoses of Level A dehydrated
diarrhea
IOI #2.7: Percentage of reasonable
diagnoses of Level B dehydrated
diarrhea
IOI #2.8: Percentage of reasonable
diagnosis of Level C dehydrated
diarrhea
IOI #2.9: Percentage of reasonable
diagnoses of poisoning
IOI #3: Percentage of eligible
district health staffs who have
successfully completed training
provided by the project (based
on three subindicators).
IOI #3.1: Percentage of doctors and
assistant doctors at district hospitals
trained by the project
IOI #3.2: Percentage of health staffs
with completed short-term training
courses compared to the plan
IOI # 3.3: Percentage of health staffs
completed long-term training courses
compared to the plan
IOI #4: Percentage of eligible
district hospitals with acceptable
operation and maintenance plans
and budgets for facility and
equipment maintenance (based
on two subindicators).
IOI #4.1: Percentage of district hospitals
having schedule and budget for
maintenance of infrastructure
IOI #4.2: Percentage of district hospitals
having schedule and budget for
maintenance of equipment
IOI #5: Number of health
facilities constructed, renovated,
and/or equipped
IOI #6: Percentage of recently
discharged patients satisfied with
health services
IOI #7: Number of people with
access to a basic package of
health, nutrition, and
reproductive health services
PDO #3:
Reduce
financial
constraints to
access health
services.
(4 indicators)
KPI #3: Percentage of
households who
experienced catastrophic
health care expenditures
in the year prior to the
survey
IOI #8: Percentage of Decision
139 beneficiaries who have
received HICs
IOI #9: Percentage of 139
beneficiaries with cards who can
correctly identify at least three
benefits covered under the
HCFP program
IOI #10: Percentage of
households who identify
financial barriers as a main cause
for not seeking health care
65
captured the results of these indicators at Year 3 of project implementation. All other indicators
had to be measured at project end, according to the PAD, but some were followed by the CPMU
using the project administrative registries.
9. Results of the PDO’s achievement evaluation: Substantial. Project efficacy could be
considered substantial according to the ratings obtained in each one of the parts of the methodology.
Efficacy of PDO 1 is High, given that all KPIs surpassed their target values. Efficacy of PDO 2 is
considered High, given that the percentage of indicators surpassed or achieved the targets is 100
percent. Efficacy of PDO 3 is considered Substantial, given that 75 percent of the indicators
surpassed their target values.
Table 5.2. Summary Table of Indicator’s Achievement
Rating Categories KPI’s IOIs Total
PDO Part 1 - Increase utilization of district health services - High
Surpassed (>105%) 2 — 2
Achieved (95%–105%) — — —
Substantially achieved (85%–104%) — — —
Partially achieved (65%–84%) — — —
Not achieved (<65%) — — —
Not considered — — —
Total PDO Part 1 2 — 2
PDO Part 2 - Improve the quality of district-level hospitals - High
Surpassed (>105%) 1 13 14
Achieved (95%–105%) — 4 4
Substantially achieved (85%–104%) — — —
Partially achieved (65%–84%) — — —
Not achieved (<65%) — — —
Not considered — — —
Total PDO Part 2 1 17 18
PDO Part 3 - Reduce financial constraints to access to health services - Substantial
Surpassed (>105%) 1 2 3
Achieved (95%–105%) — — —
Substantially achieved (85%–104%) — — —
Partially achieved (65%–84%) — — —
Not achieved (<65%) — 1 1
Not considered — — —
Total PDO Part 3 1 3 4
Grand Total 4 20 24
Table 5.3. Summarized Rating for Project Efficacy
PDOs Parts Efficacy Rates Based on the Achievement of the Indicators
PDO Part 1 High
PDO Part 2 High
PDO Part 3 Modest
Overall Rating Substantial
66
Annex 5A. Status of the Indicators According to Achievement
Original Indicators
(According to the
PAD)
Indicators Added
during Project
Implementation
Value and
Date of the
Indicator
Baseline
Value and
Date of the
Indicator
Target
Value and
Date of the
Indicator at
Project
Original
Closing Date
(Aug 31, 2014)
Ratio of
Achievement
at Project
Original
Closing Date
Value of the
Indicator at
the End of
Project
Extension
Period
(February
29, 2016)
Ratio of
Achievement
at the End of
the
Project
Extension
Period (*)
Rating of
the
Indicator
Key Performance Indicators (KPIs)
KPI #1: Increase
utilization rates of
inpatient services in
district hospitals among
Decision 139
beneficiaries
— 0.027
(June 2009)
0.033
(August
2014)
0.081
(December
2013)
9.00
0.096
(December
2015)
11.50 Surpassed
KPI #2: Increase
utilization rates of
outpatient health
services in district
hospitals among
Decision 139
beneficiaries
—
0.067
(December
2009)
0.075
(August
2014)
0.082
(December
2013)
1.87
0.247
(December
2015)
22.50 Surpassed
KPI #3: Percentage of
households who
experienced catastrophic
healthcare expenditures
in the year prior to the
survey
— 14.27%
(June 2008)
13.23%
(August
2014
2.0%
(August 2014) 11.80 Not measured — Surpassed
KPI #4: Proportion of
district hospitals that
provide full set of health
services according to the
national norms
(Decision 23/205/QB-
BYT) adjusted to the
Northern Uplands
— 39.1%
(June 2008)
70%
(August
2014)
71.4%
(October 2014) 1.05
80.4%
(December
2015)
1.34 Surpassed
Intermediate Outcome Indicators (IOIs)
IOI #1: Percentage of
patients satisfied with
the health services
— 8.5%
(July 2009)
10.2%
(August
2014)
84.4%
(October 2014) 44.71 Not measured — Surpassed
67
Original Indicators
(According to the
PAD)
Indicators Added
during Project
Implementation
Value and
Date of the
Indicator
Baseline
Value and
Date of the
Indicator
Target
Value and
Date of the
Indicator at
Project
Original
Closing Date
(Aug 31, 2014)
Ratio of
Achievement
at Project
Original
Closing Date
Value of the
Indicator at
the End of
Project
Extension
Period
(February
29, 2016)
Ratio of
Achievement
at the End of
the
Project
Extension
Period (*)
Rating of
the
Indicator
IOI #2: Adherence of
treatment protocols for
selected conditions in
inpatient settings (based
on six subindicators)
IOI #2.1: Percentage of
health workers with
knowledge of diagnosing
and treating Level
A/B/C dehydrated
diarrhea
9.7
(June 2009)
14.0%
(August
2014)
95.2%
(October 2014) 19.88 Not measured — Surpassed
IOI #2.2: Percentage of
health workers with
knowledge of diagnosing
and treating severe
pneumonia
13.2%
(June 2009)
18.5%
(August
2014)
86.9%
(October 2014) 13.91 Not measured — Surpassed
IOI #2.3: Percentage of
health workers with
knowledge of diagnosing
and treating poisoning
26.8%
(June 2012)
37.5%
(August
2014)
83.0
(October 2014) 5.25 Not measured — Surpassed
IOI #2.4: Percentage of
reasonable diagnoses of
severe pneumonia
45.5%
(June 2009)
63.7%
(August
2014)
71.1%
(October 2014) 1.41 Not measured — Surpassed
IOI #2.5: Percentage of
clinical health workers’
reasonable diagnosis of
general pneumonia
19.60%
(June 2009)
27.44%
(August
2014)
57.00%
(October 2014) 4.77 Not measured — Surpassed
IOI #2.6: Percentage of
reasonable diagnoses of
Level A dehydrated
diarrhea
37.2%
(June 2009)
52.1%
(August
2014)
78.9%
(October 2014) 2.80 Not measured — Surpassed
IOI #2.7: Percentage of
reasonable diagnoses of
Level B dehydrated
diarrhea
48.9%
(June 2009)
68.5%
(August
2014)
85.1%
(October 2014) 1.85 Not measured — Surpassed
IOI #2.8: Percentage of
reasonable diagnosis of 41.2%
(June 2009)
57.7%
(August
2014)
80.0%
(October 2014) 2.35% Not measured — Surpassed
68
Original Indicators
(According to the
PAD)
Indicators Added
during Project
Implementation
Value and
Date of the
Indicator
Baseline
Value and
Date of the
Indicator
Target
Value and
Date of the
Indicator at
Project
Original
Closing Date
(Aug 31, 2014)
Ratio of
Achievement
at Project
Original
Closing Date
Value of the
Indicator at
the End of
Project
Extension
Period
(February
29, 2016)
Ratio of
Achievement
at the End of
the
Project
Extension
Period (*)
Rating of
the
Indicator
Level C dehydrated
diarrhea
IOI #2.9: Percentage of
reasonable diagnoses of
poisoning
61,1%
(June 2009)
85.5%
(August
2014)
86.2%
(October 2014) 1.03 Not measured — Achieved
IOI #3: Percentage of
eligible district health
staff who have
successfully completed
training provided by the
project
IOI #3.1: Percentage of
doctors and assistant
doctors and pharmacists
at district hospitals
trained by the project
0
(June 2008)
80
(August
2014)
102
(December
2013)
1.27
189
(December
2015)
2.36 Surpassed
IOI #3.2: Percentage of
health staffs completed
short-term training
courses compared to the
plan *1
0
(June 2008)
80
(August
2014)
357
(June 2014) 4.46 — — Surpassed
IOI # 3.3: Percentage of
health staffs completed
long-term training
courses compared to the
plan *1
0
(June 2008)
80
(August
2014)
88
(June 2014) 1.10 — — Surpassed
IOI #4: Percentage of
eligible district hospitals
with acceptable
operations and
maintenance plans and
budget for facility and
equipment maintenance
IOI #4.1: Percentage of
district hospitals having
schedule and budget for
maintenance of
infrastructure *2
49.2
(June 2009)
40.0
(August
2014)
99.3
(October 2014) 2.02
79.7
(December
2015)
1.62 Achieved
IOI #4.2: Percentage of
district hospitals having
schedule and budget for
maintenance of
equipment. *2
77.1
(June 2009)
40
(August
2014)
99.2
(October 2014) 1.29
89.1
(December
2015)
1.16 Achieved
IOI #5: Number of
health facilities
constructed, renovated,
and/or equipped
0
(June 2008)
61
(August
2014)
63
( August 2014) 1.03
64
(December
2015)
1.05 Achieved
69
Original Indicators
(According to the
PAD)
Indicators Added
during Project
Implementation
Value and
Date of the
Indicator
Baseline
Value and
Date of the
Indicator
Target
Value and
Date of the
Indicator at
Project
Original
Closing Date
(Aug 31, 2014)
Ratio of
Achievement
at Project
Original
Closing Date
Value of the
Indicator at
the End of
Project
Extension
Period
(February
29, 2016)
Ratio of
Achievement
at the End of
the
Project
Extension
Period (*)
Rating of
the
Indicator
IOI #6: Percentage of
recently discharged
patients satisfied with
health services
8.5
(June 2009)
10.2
(August
2014)
84.4
(August 2014) 44.65 — — Surpassed
IOI #7: Number of
people with access to a
basic package of health,
nutrition, and
reproductive health
services *4
20%
(June 2009)
70%
(August
2014)
244,801
(August 2014) —
270,274
(December
2015)
— Not
considered
IOI #8: Percentage of
Decision 139
beneficiaries who have
received HICs *5.
— 82.1 (June
2009)
70
(June 2013)
94.3
(June 2012) 1.15
95.2
(December
2015)
1.16 Surpassed
IOI #9: Percentage of
139 beneficiaries with
cards who can correctly
identify at least three
benefits covered under
the HCFP program. *6
— 14.8
(June 2009)
75.0
(August
2014)
57.4
(August 2014)
0.57
— —
.Not
Achieved
IOI #10: Percentage of
households who identify
financial barriers as a
main cause for not
seeking health care.
— 2.0
(June 2009)
1.8
(August
2014)
1.2
(August 2014) 4.03 — — Surpassed
Note: *1 This indicator was not listed in the PAD. The information was obtained from the Center for Environment and Health Studies (2014) Final Report: End-line evaluation
of the Northern Uplands Health Support Project, Hanoi, 2014, and Center for Environment and Health Studies (2016) Final Evaluation of the Northern Uplands Health
Support Project in the Extension Phase, Hanoi, 2016. *2 The target of this indicator was established before the baseline survey. For this reason, the baseline value in the PAD was higher than the target value. However, the project
did not revise the target during project implementation. For this reason, the achievement for this indicator had been calculated over the baseline instead of the target. Even
using these criteria, the actual value for this indicator was twice the baseline value by August 2014, but was only 62 percent over the baseline by the end of the project
extension period, which could bring some doubts on the sustainability of infrastructure maintenance. Despite this issue, the ICR team considered the target achieved. *3 According to the PAD, this indicator’s provisory baseline was 20.14 percent in 2014. An actualization of this indicator was expected to update the baseline after start the
project implementation.
70
*4 This indicator cannot be measured because the baseline and the target were not converted in the number of beneficiaries with access to the basic package of health,
nutrition, and reproductive health services during the project life. *5 The target of this indicator was established before the baseline survey and was based according to the preparation team best guess estimate. However, the project did not
revise the target during project implementation. For this reason, the achievement for this indicator had been calculated against the baseline instead of the target. Despite this
issue, the ICR team considered the target achieved. *6 The Government additionally followed the indicator ‘Percentage of Decision 139 Beneficiaries with cards who can correctly identify at least three benefits covered under
the HCFP program’. The achievement of this indicator was 95 percent in December 2015.
71
Annex 6: Summary of Borrower's ICR
Northern Uplands Health Support Project
1. Context
1. The World Bank supported the Vietnamese Government with a loan to implement the
Northern Uplands Health Support Project (hereafter called the NUP). The project was
implemented in eight years (2008–2015) in the seven northern provinces of Cao Bang, Bac
Kan, Lao Cai, Ha Giang, Son La, Dien Bien, and Lai Chau.
2. The general objective of the project was to improve the health status of the poor ethnical
minorities in these provinces, which required strengthening the capacity of the health care
system, providing better quality health services, and improving access. The legal background
of the project is composed of:
(a) the general policies and decisions of the party and the state during the period
2001–201049 to promote health care and protect people’s health;
(b) the master plan to develop Vietnam’s health care system up to 2010 and vision to
2020;50 and
(c) the creation of the HCFP in 2002 (known as Decision 139) to increase access to
health care and reduce the financial burden of health expenditure faced by the poor
and ethnic minorities.
3. To achieve this general objective, the Project supported (a) the upgrade of district
hospitals in these seven provinces, by training health staff, developing human resources for
health care, renovating these hospitals, providing medical equipment to achieve better health
care services, creating mechanisms and skills to repair infrastructure and medical devices; and
(b) the increase of health care services’ access for the poor and the ethnical minorities by
providing economic subsidies for transportation and meals and ensuring equity in protecting,
caring, and improving people’s health.
4. The seven NUP provinces constitute the most disadvantaged regions in Vietnam, with
high concentrations of poor and ethnic minorities living in sparsely populated, mountainous
localities, under difficult circumstances. As a result of these unfavorable socioeconomic
conditions, and despite somewhat higher public health expenditure per capita than in other
regions, health services in these provinces are difficult to access and generally poor in quality,
while the health status of the population is significantly worse than the rest of the Vietnamese
population.
49 Declared at Decision No. 35/2001/QD/TTg (dated 03/19/2001) and the Resolution No. 46-NQ/TW (dated
02/19/2005). 50 Decision No. 153/2006/QD-TTg of the prime minister.
72
5. The health system in these mountainous provinces was weak with regard to the quantity
of qualified human resources, infrastructure, medical equipment, and health financing. The
investments provided by the project for these provinces significantly improved the health
supply and the health status of local people, contributing toward the achievement of the MDG
targets in health, in accordance with the party and the state policies. It also contributed to
developing Vietnam’s health sector toward fairness and efficiency.
6. After seven years of project implementation, the health indicators of the Northern
Upland Provinces have improved significantly. The inpatient services’ utilization of the poor
ethnical minorities (beneficiaries under Decision 139) increased almost four times between
2009 and 2015. The outpatient services’ utilization rates increased two times in the same period
and the proportion of households with catastrophic health expenditures reduced from 10
percent to 2 percent, between 2009 and 2014.
7. Health human resources (weak and inadequate in the past) have been strengthened. All
health staff in district hospitals (111,800 workers) were trained. Health facilities were repaired
and upgraded in parallel with investments in infrastructure. Equipment provided to district
hospitals have been upgraded and used effectively.
8. The project design draws on best practice examples and lessons learned from other
international and Vietnam health investment projects. The project supported interventions on
both the ‘supply’ and ‘demand’ sides. On the supply side, the project supported district
hospitals by providing training and developing health human resources and repairing and
upgrading district hospitals’ infrastructure and equipment. On the demand side, the
contribution was facilitated, by providing economic subsidies to the poor, access to good
quality services, and thereby, increasing the probability of success and efficiency of
investments to improve the population’s health.
9. The project design is well articulated with other donor-supported activities in the region
and therefore does not cause any duplication. It only complements other ongoing initiatives.
There are some projects also implemented in the NUP, such as Project 225, financed by the
Government, to upgrade the provincial district hospitals. Between 2005 and 2007, Project 225
has invested around US$10 million (VND 169 billion), which only met 20 percent of the total
health investment needs for the district level in these seven provinces. HEMA (sponsored by
the European Commission) supported health care investments for three of the seven NUP
provinces (Dien Bien, Son La, and Lai Chau). The HEMA Project was focused on the
community health services. The Global Alliance for Vaccination and Immunization Project
(2007–2010) supported training for village health workers in 10 provinces, including 4 of the
Northern Upland provinces (Ha Giang, Cao Bang, Bac Kan, and Dien Bien). The ADB
financed a project for development of provincial preventive medicine system (by providing
equipment and training for laboratories). This project was also financed by several bilateral
donors and nongovernmental organizations with limited funds.
10. The PDO risks were well managed and limited at the lowest level with appropriate
measures from the project. To avoid the abuse of inpatient services (admitted with mild cases)
at district hospitals in the project areas, the project provided trainings for the officers on topics
73
such as management, operation of the fund for the poor, and verifying the admission cases.
The audits (in collaboration with HI officers) of using funds for the poor were regularly
conducted, to ensure that the admission cases were appropriate, inpatient cases received
treatments, and inpatient’s medical records were maintained. The monitoring activities, which
focus on support for the poor, from the CPMU and the PPMUs to the hospitals had been made
periodically. The results of the supervisions and audits showed that management of the HCFP
was done in accordance with the regulations of the Government and the project.
11. The risk of the non-poor group also benefiting from the interventions of the project,
leading to rising inequality, was well managed by different interventions. The project
developed a good management system to control these risks by coordinating well with Vietnam
Social Insurance in the provinces, for review and synthesis of the project beneficiaries annually.
The list of the poor and ethnic minorities who were qualified for support from the NUP was
provided to district hospitals. To receive the NUP support, the patients should present the
insurance card with the code ‘HN’ or the certification of poor from the communal people’s
committee
12. Many officers/staff were sent for trainings causing short-term shortage of health
workers and affecting the availability and quality of services. However, various activities were
implemented to alleviate these difficulties. The implementation of training courses was spread
throughout the project duration. District hospitals had plans for replacing the staff who
attended trainings. The leaders, physicians, and assistant doctors working at ancillary
departments (management board, departments of planning, financial, or infection control) at
the hospital also shared the responsibility of treating and caring for patients. Medical staffs are
required to work in night shifts more frequently. The project allowed district hospitals to sign
contracts/hire retired medical doctors or temporarily transfer staff among hospitals to help each
other when their staff were attending the training. The project risk management is described in
annex 6A.
2. Achievement of the PDOs
13. The project has successfully achieved its overall DO. The utilization of district health
services (by the poor and vulnerable population defined accordingly with Decision 139) has
increased sharply after IEC campaigns, improving the effectiveness of the Government’s
priority to UHC, both by improving the geographical accessibility of quality basic health
services at district hospitals and by reducing the financial burden in accessing health services
for the poor and ethnical minorities.
14. The project has been successful in implementing the health care policies for the poor
and increasing their access to quality health services. The number of the poor receiving
financial assistance (meals and travel costs) to visit district hospitals from the NUP has been
increased yearly, along with a significant increase in the rate of using district health services
among the general population, especially among vulnerable groups such as the poor and ethnic
minorities. The average number of inpatient and outpatient visits per capita per year of ‘HN’
(Ho Ngheo or Poor Household in English) in district hospitals, increased 390 percent (from
0.0247 to 0.096) and 369 percent (from 0.067 to 0.247) from 2009 to the end of 2014,
74
respectively, exceeding the project’s expectations. The awareness and confidence of the
population vulnerable to health care services has been improved. The percentage of households
who experienced catastrophic health care expenditures in the year before the survey reduced
81 percent in comparison to the baseline. Detailed data is presented in annex 6B.
15. The activities related to the project components51 have remarkably improved the supply
and quality of the health services offered by district hospitals in the seven provinces. The
percentage of health techniques (based on the national list of techniques/protocols defined by the
MoH) in the seven provinces’ district hospitals increased from 39.1 percent at the baseline (2009)
to 80.4 percent at the end of 2015. The percentage of patients who were satisfied with health care
services increased from 8.5 percent at the baseline to 84.4 percent at the end line. Around 90
percent of patients were satisfied with the qualification of medical staff, facility infrastructure,
medical equipment, and drugs used for treatment (annex B).
16. District hospitals could provide more complex clinical techniques, especially
emergency surgeries and various endoscopic surgeries. The capacity of obstetric and newborn
care in district hospitals has improved significantly, with more than 75 percent of the district
hospitals providing cesarean section and blood transfusion services. The strengthened hospital
capacity and improvement of staff quality, has as a consequence, reduced the transferring of
patients to a higher-level hospital, and average length of inpatient stay by a half of day, in
comparison to the baseline (6.6 days down to 6.1 days).
17. At the end of the project, the number of health staffs with higher qualifications (general
medical doctors and medical doctors-level-1 specialists) was doubled. The knowledge and
skills of health workers to manage common health episodes were significantly raised (annexes
6B and 6C). Besides district hospitals’ infrastructure and equipment were upgraded by the
project, thereby improving district hospitals’ efficiency by absorbing a higher number of
medium to complex cases and by reducing referral to provincial hospitals. These positive
effects in the district hospitals’ performance are widely recognized and have long-term effects,
increasing the sustainability of the provincial health systems. However, the hospitals still lack
specialized doctors (in surgery, trauma, and in specialized departments such as eyes and dental)
which requires appropriate measures to attract qualified human resources to fulfill this need at
district hospitals.
3. Analysis of the Results Framework (baselines, targets, results)
18. There were no major changes in the Project Development Indicators (PDI’s) and
Intermediate Indicators (II’s) with regard to concept or calculation formula during the project
execution. The only exception is the indicator of proportion of district hospitals that provide
full set of health services according to the national norms (Decision 23/205/QD-BYT), which
was adjusted for better measuring the improvements of the district hospitals’ capacity. Some
51 Such as strengthening capacity in provision and quality of health care service at district level; comprehensive
interventions of the NUP in supporting human resources; investment in upgrading infrastructure; procurement,
purchasing, and installment of medical equipment; and improvements in hospital management have remarkably
improved the health services (quality and quantity, diversify the types of services) in the seven provinces.
75
key maternal and child health indicators were presented to reflect the impacts toward the
achievement of MDGs targets at the project provinces.
19. The evaluation results showed that 9 out of 10 KPIs were achieved and even exceeded
several times the objectives set in the project document. One indicator where the goal was not
achieved is the percentage of ‘HN’ beneficiaries knowing at least three beneficiaries’ rights
guaranteed by the HI system. This indicator reached 57.4 percent of the expected target at the
end of the project. The results also indicated that the trainings and equipment support improved
the district hospitals’ capacity of providing obstetric/pediatric emergency services,
contributing to improving the MDG performance in the region. In the extension period, in
collaboration with Mother and Child Health centers, PPMUs and CPMU focused on the
trainings that aimed at improving the capacity for newborn care, maternal health care,
traditional birth attendants, and providing clean delivery kits for the traditional birth attendants.
The results of the survey in 2015 showed that the maternal mortality ratio, adjusted in seven
provinces, is 98 per 100,000 live births, significantly reduced in comparison to the 2008 survey
data. The IMR was 10.6 percent, similar to the 2008 survey results in 2015. The MMR and
IMR in the seven project provinces are still higher than those of the whole country, which set
out the needs to continue communications on safe motherhood to the local people, especially
ethnic minority groups, develop the traditional birth attendants’ networks in remote villages,
and strengthen the management of pregnancy and antenatal care at the commune health centers,
to achieve the MDGs of the country. The project RF is presented in annex B.
4. Achievements by Components
20. The performed activities and interventions are the same as defined in the project
document. The outputs of the components have surpassed the targets that were set. See detailed
information in annex 6C.
4.1 Strengthening District-level Health Services (disbursement rate=90 percent (USD
38,584,300/USD 42,880,362)
21. The project has supported long-term and short-term trainings for health staff in the
NUP in many fields. The training programs fulfilled actual needs of the locality and were
implemented in accordance with guidelines, policies, and strategies of the MoH. The training
activities improved staff skills and reduced health workforce shortages, providing
professionals who were better prepared. It also contributed to reduce the unbalanced health
skills in district hospitals and provide a stable workforce to attend their needs in the long term
in the seven provinces. The project has supported the development of 377 level-1 doctors,
achieving 266 percent of the original target, and 56 percent are ethnic minority doctors. About
1,500 assistant doctors (46 percent with ethnical minority background) were trained and
converted to principal doctors. The project has also trained 98 intermediate pharmacists.
22. Overall, 95 percent of the staff who graduated through the project training activities
returned to work at the host hospitals. Since 2011, the PPMUs decided to enroll assistant
doctors from commune health stations as part of the long-term training activities and to include
76
important new specialties in the level-1 training, such as image diagnoses, tuberculosis
treatment, clinical pathology, anesthesiology, and others.
23. The short-term training courses on specialized contents have improved knowledge and
practical skills on disease treatment to address the shortage of professional capacity and
strengthen the decentralized techniques in the district hospital. Besides the clinical areas,
trainings on preventive medicine and health management and related areas (hospital
management, health information systems, medical equipment repair, and medical waste
management) were also provided.
24. To improve maternal and child health status and support the achievement of the health
MDGs by 2015, relevant training for nurses and emergency care for maternal and newborn
babies were also added in the training programs. The project also conducted short-term training
courses on safe motherhood, focusing on maternal and newborn health care.
25. The implementation of non-training human resources activities such as ‘technology
transfer; (in collaboration with the Government 1816 program)52 and ‘rotation of doctors’
(temporary placement of a doctor from a higher-level facility at a district hospital) had partially
alleviated the shortage of doctors and other staff in the district hospitals, providing capacitation
on specific techniques while emphasizing more practical skills. Detailed training results are in
tables 6.1–6.6 of annex 6C.
26. The project also performed activities to repair and upgrade 18 district hospitals (100
percent working plan, completed in 2010–2012) providing the installation and effective use of
the medical equipment supported by the NUP (table 3.7, annex 6C).
27. The project provided 52 ambulances and 5,065 essential medical equipment for 64
district hospitals (average of 80 devices per hospital achieving 102 percent of the working
plan). It included valuable equipment such as high-tech x-ray machines (102), endoscopic
systems (37), ultrasound systems (109), ventilators (112), and monitor tracking devices (142)
(table 3.8, annex 6C).
28. The equipment provided is strongly based on the hospitals’ demands, possible because
of carefully reviewing and assessing the needs before starting the purchasing process. The
equipment that was received immediately had a positive effect on providing health care for the
people and offering favorable conditions for the health staffs to practice the knowledge and
skills that they had gained from the long-term and short-term training programs.
29. The maintenance of this equipment had complied with the requirements. Health staff
were fully trained and instructed to adequately use and explore the equipment. The specific
teams responsible for maintaining and repairing the medical equipment were established in
each of the seven provinces.
52 1816 Program is a program of the Government of Vietnam on ‘Sending the professional staff from higher-
level hospitals to support lower-level hospitals professionally to improve the quality of the health care services’.
77
4.2 Increasing Financial Access to Healthcare Services for Decision 139 Beneficiaries
(disbursement rate=100.5 percent (USD 10,048,816/USD 10,000,000)
30. To increase the use of health services for the poor and minorities, the project financed
travel expenses, meals, and direct costs of the beneficiaries who were not supported by HI or
other health care funds. This support helped the poor/minorities access and use quality basic
health care services, ensure fairness in the health financial protection, and improve the health
of the beneficiary population. It sustained the effective implementation of the provincial health
care policy for the poor/minority, defined in Decision 139. From 2009 to 2014, around 65,000
inpatients’ medical visits received support. (Table 6.9 annex 6C). The percentage of poverty
households which experienced catastrophic health care expenditures (according to the World
Health Organization criteria) reduced from 10.4 percent to 2.0 percent between 2009 and 2014
surpassing the target of 9.4 percent. (Annex 6B).
31. The final project survey showed that the rate of households which experienced
expenditures for health care services decreased five times and three times compared to the
baseline and MTR, respectively. In 2015, the HCFP (supported by Decision No. 14/2012 / QD-
TTg) was operated in four provinces (Lao Cai, Ha Giang, Son La, and Lai Chau). Total 269,400
inpatient visits were supported in four provinces in the 18 months of the project extension
period (table 6.10, annex 6C). Three other provinces—Cao Bang, Bac Kan, and Dien Bien—
had not implemented this decision as the provincial budgets were not adequate or allocated to
continue these supports. During the extension period, the number of inpatients in these three
provinces was slightly reduced in comparison to the previous period. Without support on
transport and food allowances, poor households tended not to go to the hospitals as they could
not afford travel and food during their hospitalization period. This suggested that the financial
barrier is still one of the main reasons that limit access to and use of health care services to
households, especially poor households, so the NUP’s support for the poor in accessing health
services is very important.
32. The project also promoted health-seeking behavior for the poor by developing IEC
activities to increase the target population’s knowledge about their benefits under the HI
scheme and the additional support provided by the project. As part of the activities related to
this area, IEC material was distributed to the households, IEC messages were announced in the
community and newspapers, movies, videos, and comedies were produced and broadcasted on
mass media (TV, speakers, and radio) and posters were displayed in the hospital
departments/rooms. The project provided some essential communication equipment to be used
in the provincial centers for the IEC activities. Nearly 5,000 meetings and campaigns were
conducted at crowded places and the IEC teams visited around 8,000 households.
33. The IEC activities increased access and utilization of health care services for the
poor/ethnic minority groups in the district hospitals. The final project evaluation showed that
94.7 percent of the poor/ethnic minority people could tell at least one right of HI cardholders,
94.3 percent of the poor/ethnic minority people in the project areas had HICs and the number
of inpatients at district hospitals sharply increased.
78
34. To strengthen capacity for the HCFP, the project provided training for the health
provincial teams and basic equipment (computers, printers, and photocopiers) to district
hospitals. The PPMU implemented systematic supervisions in the HCFP and guaranteed that
the management of the fund activities was in compliance with the provided guidance.
4.3 Monitoring, Evaluation, and Project Management (disbursement rate=85.3 percent
(USD 6,075,917/ USD 6,744,638)
35. The project maintained an adequate management structure and implementation
arrangements. The Project Steering Committee at the MoH was established to directly lead
project management. The CPMU and PPMUs were the key players in implementing and
managing the project. Monthly meetings with the PPMUs gave district hospital leaders the
chance to express their needs, comments, and suggestions for more appropriate and informed
implementation. The coordination between the MoH, CPMU, and PPMUs were systematic and
effective.
36. The project was implemented faster than previously scheduled, in the original plan.
The project outputs have reached and exceeded the outputs agreed in the project document.
After the project closing, the rate of disbursement was estimated at 96 percent. The goals and
targets are completed and beyond the schedule, at a cost lower than originally expected,
suggesting that the project was very cost-effective as well. The MoH authorities are satisfied
with the project achievement results.
5. Restructuring: Project Extension for 18 Months
37. After five years of execution, the project was implemented on schedule and the outputs
were achieved and exceeded the original plan, with savings in the original budgets (mainly
through procurement and tendering activities). However, there were a number of new
activities—non-planned in the original project design—that could be implemented with the
saved funds to promote efficiency and enhance the future sustainability of the project. The
main arguments to explain the rationale for the project extension are
• although the district has invested in training and upgrading of facilities and
equipment, the needs for medical support at the seven NUP provinces were still
huge because the district and commune levels were still facing many difficulties;
• the project investment has significantly helped the quality of services, increased
the number and type of health services provided, and increased accessibility and
the use of health services by the population at the district level. However, the NUP
team identified additional needs to be implemented in the short term (12–18
months) to guarantee the project’s long-term efficiency of investments and
maximize the effective use of the upgraded health facilities and the acquired
equipment; and
• the health indicators in general, especially those related to the health MDGs in the
NUP are the lowest compared to other regions in the country. The prime minister
79
issued Resolution No. 05/NQ-CP, dated 01/13/2014, on accelerating the
implementation of the health MDGs. Therefore, the support to accomplish this
resolution in the NUP areas (especially the goal of reducing child mortality rate
and MMR) was very urgent, especially considering the budget cuts that occurred
during this period.
38. Given these arguments, the DO and the scope of the interventions for the extension
phase did not change in comparison with the original stated in the project document. In addition,
the extension phase of the project looked for ways to
• maximize efficiency and enhance the sustainability of the project investments in
the seven provinces; and
• contribute to the achievement of Vietnam’s health MDGs in the related areas,
especially by reducing child mortality and improving maternal health.
39. During the extension phase, the project has achieved the objectives and implemented
the planned activities such as training on maternal and childcare and procuring equipment for
newborn health units and consumables to ensure safe birth delivery. The project management
was effective, focused on achieving the project results and promoting the sustainability of the
investment.
40. Special attention was dedicated to improve inpatient and outpatient services utilization
and capacity to implement clinical techniques at the district hospitals. The project has
contributed significantly to reach the MDGs (especially in reducing maternal and child
mortality) through capacity-building activities on obstetric and neonatal emergency at the
health facilities, particularly at district hospitals. The project contributed to the development
of village midwives’ teams and to increase the percentage of women giving birth at home, with
SBAs’ assistance, by providing clean delivery packages for pregnant women. During 2008–
2014, the maternal and child mortality rates declined remarkably, as can be seen in the annex
6B: Project Results Framework, MDG indicators.
6. Beneficiaries
41. The main project beneficiaries, as stated in the project document, were the poor and
ethnic minorities in the Northern Upland Provinces. These population groups were supported
by reducing their financial barriers to access and make better use of quality health services.
Project investments focused on the district level, an appropriate level at which the poor and
ethnic minorities had the ability to access their health care needs.
42. Other project beneficiaries were the health workers of the district hospitals in the seven
NUP areas who participated in trainings in many fields, such as (a) training level-1 doctors for
treatment; (b) doctors with four-year added training; and (b) short-term training courses for
clinical, preventive medicine, and health management. Since 2011, the project extended the
training support to pharmacists and assistant doctors at the community health centers.
80
43. Last, but not the least, the project benefited the health management agencies at
provincial and district levels (Department of Health, Division/District Health Centre) by
improving their managerial capacity. The district hospitals’ management teams were also
benefited by trainings that ensured better organization and means to provide health care for the
poor and ethnic minorities.
7. Evaluation of Other Project Aspects (risks, safeguards, fiduciary)
44. The project was designed based on the needs and recommendations of the local
authorities and communities. There were several rounds of referendums, for finalizing project
objectives and activities, with different local stakeholders (the provincial people committees,
department of health, district hospitals, and specialists), using different methods, including
participatory rapid assessment, consultation workshops, and direct consultations. The MoH
accumulated experience in implementing projects with similar components and activities.
Therefore, the implementation of the Project was quite favorable. The risk management
process was well conducted and did not significantly impede the project’s performance.
45. Environment safeguards. The project provided good solutions and implementation of
waste management processes at district hospitals. Basic training on regulations related to
HCWM and nosocomial infection control was provided for the district hospital managers and
staff. The CPMU and PPMUs received guidance for planning and implementing measures for
HCWM. Consumables and equipment for HCWM 53 were procured by the PPMUs and
distributed to the project district hospitals. The project hospitals achieved remarkable
improvements in the waste management process compared to the baseline (2007), when most
of them did not comply with the HCWM regulations. By 2013, a well-prepared HCWM plan
and monitoring program started to be implemented and all district hospitals had strengthened
institutional arrangements for that. The availability and proper use of waste containers,
transportation, and cooling devices resulted in significant improvements in health care waste
separation, collection, storage, and final disposal in district hospitals.
46. Social safeguards. The project beneficiaries included the poor and ethnic minorities
living in disadvantaged areas. Given the typical geographical condition of the NUP provinces,
the proportion of ethnic minorities is quite high. Data reported from PPMUs and surveys are
always disaggregated by Kinh (the majority of the population) and ethnical minorities to
indicate that the support from the NUP is considerable to ethnical minorities and the poor.
Most of the ethnical minority health staff received long- and short-term trainings, which
contributed to the increased ratio of ethnical minority doctors in the seven project provinces.
The percentage of beneficiaries who are ethnical minority is higher than 75 percent. In some
provinces, ethnical minority staff account for a higher percentage of total participants (such as
91 percent in Cao Bang). The project tailored IEC campaigns with the linguistic and cultural
characteristics of ethnic minorities. The IEC materials were translated into the local languages
and, to the extent possible, verbal and graphical means of information transfer were used. The
key success was the development and implementation of a strategy that reaches the diverse
53 It includes 23,000 kg of color-coded plastic bags, 25,000 sharp boxes, together with fixation frames, more
than 500 waste containers, 12 waste on-site transportation devices, and 64 cooling devices.
81
and dispersed beneficiaries in the Northern Upland Provinces, given the ethnical diversity of
the region and the difficult geographical terrain.
47. Fiduciary aspects. The project did not register major fiduciary problems (regarding
procurement of civil works or medical equipment acquisition). The counterpart funds from the
MoH and provincial budgets were allocated/provided on time and met the demand for timely
implementation of the management activities of the project.
8. Bank and Borrower Performance
48. The project implementation had sufficient human resources (including national
consultants) and good capacity to manage and implement the activities and use the financial
resources. The CPMU provided capacity building for PPMUs staff on project management,
including financial, procurement, accounting, assets management, civil works, training, HCFP,
and M&E activities. In addition, the project had international experts in the related fields. The
project was implemented on time and exceeded the set targets. Project interventions were
highly effective, providing significant benefits to health care of the NUP beneficiaries.
49. Project management activities were carried out with high demand for quality
management. Due to the design (simultaneous interventions on both the ’supply’ and demand’
sides) the project interventions were completed and exceeded the targets.
50. The basic elements for managing the project (Project Manual, job description for each
position, and financial management software) were available right from the start of the
implementation. The accounting software (installed at the central and provincial levels) met
the regulations of the Ministry of Finance and the project requirements. The financial
management system provided accurate and timely information on whether the credit proceeds
were used for the intended purposes. Cash accounting were conducted monthly. Financial
statements were prepared quarterly by the PPMUs and sent to the CPMU for consolidation and
submission to the World Bank. The procurement processes and procedures strictly followed
the requirements and regulations laid down by the Vietnamese Government and the World
Bank.
51. The involvement of stakeholders (CPMU, related departments of the MoH, PPMUs,
Department of Health, provincial/district hospitals) in the planning process indicated that the
plans were carefully considered before approval. The project plan and adjustments/revisions
were made based on actual demands and need of localities, in accordance with the project
objectives so as not to impede the progress of the project. The procedures and processes of
approval were agreed by both the MoH and World Bank to harmonize both institutional
perspectives. Therefore, in general, the progress on implementing the activities satisfied the
request of the World Bank and the MoH.
52. The operation implementation was sequenced as follows: staff training, hospital repair,
upgrade and acquisition of new medical equipment or supply. Communication activities were
carried out simultaneously to encourage people to seek medical care and treatment at the health
82
facility. The project activities were carried out in accordance with the approved working and
financial plans.
53. The M&E process was implemented according to the project M&E framework. The
baseline survey provided the initial and final targets of the project indicators. The MTR and
final project evaluation reviewed all the targets. The progress monitoring missions collected
outcomes at the localities with the appropriate tools. The data on indicators and performance
coming from provinces was aggregated, analyzed, and used to adjust project activities as
needed (some adjustments to increase the quotas and additional medical staff training,
enhanced communication activities to change behavior, and others were decided and
implemented in the middle of the project cycle).
54. The CPMU also provided training on monitoring, evaluation, and reporting and data
quality assurance for the PPMUs staff.
55. The success in implementation of the project is also the consequence of good
management and positive support from the World Bank team. Closed monitoring of project
activities, timely issuance of ‘no objections’ for procurement and work plan, suggestions for
important solutions and recommendation for speeding up the progress of the project, and
participation in technical missions were some of the valuable contributions of the World Bank
team to the success of the project.
56. For the improvement of future projects, the M&E system and M&E plan should be
built at the beginning, with the standard forms for collecting data from the PPMUs and
implementing sites. This will allow future projects to have good data and reference sources
from the beginning, improving the follow-up of the projects’ achievements. The PPMUs
should closely follow up with provincial people’s committees for approval of annual financial
plans as soon as possible, so the activities can be implemented at the beginning of each year.
9. Arrangements for Sustainability of Results
57. The NUP project investment presents a high level of sustainability and has contributed
to strengthening the NUP health systems. The project interventions were in line with the
policies and priority of the health sector and consistent with local needs. The seven provinces
are committed to continue to perform efficiently and maintain the sustainability of the project.
Many interventions were recognized to guarantee long-term sustainability to provide and
promote access of health services to the poor and ethnic minorities, such as the following:
• Training/developing human resources. The doctors (who received support from
NUP) will receive funds (from the provincial, host hospitals) to further continue
long-term trainings to become specialized doctors in surgery, traumatic, and
subordinate departments. This will certainly help hospitals to use more techniques
at district hospitals. For the policies’ implementation to strengthen health
workforce, especially for health care at the district level, the localities recruited
local staff and organized plans to maintain regular trainings to strengthen the
capacity of health staff.
83
• Quality of care. The model of technology transference among different health
complexity levels has paved a sustainable mechanism to develop capacity in the
provision of health care services and to constantly improve the quality of the
health care services at all levels, especially at the district level.
• Upgraded medical equipment. All provinces participating in the project
provinces have specific commitments to provide funds to maintain equipment and
upgrade and repair infrastructure, after the project closing. The infrastructure and
equipment will continue to be used and will ensure the quality in providing health
care services for the local people at district hospitals.
• Funding access to health for the poor. The project’s high sustainability was also
demonstrated through the health care policies for the poor/ethnic minorities
currently being applied. Infrastructure, equipment, and human resources training
supported by the project have worked well in providing quality health care
services to beneficiaries. Many provinces have mobilized funds to support the
local people to access and use health care services at district hospitals. Four of the
seven provinces have established provincial HCFPs. In the remaining three
provinces, where the Decision 1454 is not implemented, the Government allocated
partial funds (of transportation and meals) for the ethnical minorities and the poor
who need to be served by district hospitals.
• Other aspects. The IEC materials developed by the project and the increased
management capacity of the HCFP have been essential to support the provinces to
implement the program effectively and transparently. These are considered as
sustainable contributions for health care activities for the poor.
10. Lesson learned
58. The NUP project is designed with specific activities after careful consultations with
local stakeholders in the seven provinces. Right after coming in to effect, the project activities
are implemented, without any delays. Some projects are designed in the form of a project
framework; the project started with need assessments and specific activities are designed after
that, consequently, the project is implemented slowly.
59. The project is built bottom-up, based on local needs, and referred to the criteria and
standards issued by the MoH and Ministry of Finance (list of medical equipment, contents, and
training curricula). The NUP project does not develop its own cost norm or project
implementation guidance, but applies the existing regulations, so that project implementation
is fast, convenient, and highly sustainable.
60. Interventions of the NUP project are designed on both the supply and demand sides,
including supporting the provision of services (trainings, hospital renovations, providing
54 Decision 14, issued by the prime minister in 2012 on the revisions to Decision 139, issued in 2008 for the
health care for the poor.
84
equipment) and using services. This design brings high efficiency: the improvement in the
quality of health service, interacting with the support for the poor, and increase in the access
of the poor to health services.
61. The NUP’s support focused on northern mountainous region, with comprehensive
investments for district hospitals, and created equal development for the health sector in the
northern mountainous region. Therefore, the project implementation is far more favorable in
comparison with the projects where targets are scattered in different regions across the country.
62. The NUP project focused on the district level, at the grassroots level of health of
Vietnam (from the district level and below). This is one of the few investments in health that
support the district level in Vietnam. The project targeted the districts of the most difficult
areas in Vietnam, focused on primary health care, maternal, and child health care, which are
very important factors that contribute to pursuing the MDGs and the UHC in Vietnam today
and up to 2035. The effectiveness of these investments is significantly observed. The project
interventions (the support for long-term training, the IEC activities on health and support for
the poor, and so on) should be continued, using regional and local funds to ensure that the
project achievements will be strengthened and sustained.
63. The project assisted localities to develop their health plans and implement preparation
steps for maintaining the activities after the project ended and to continue the issuance of
policies and plans on health support for the northern mountainous provinces. Particularly, two
aspects could be highlighted: (a) the health workforce development policy and (b) the retention
and training of a village midwives team to serve remote areas where home birth delivery is
necessary.
64. A number of activities, for example, procurement of equipment or training of staff,
although the work plan has been approved a year before, are still reconfirmed with local needs
(quantities and types of training/equipment) before deployment. This ensures that the project
investments are appropriate and meet the real needs from district hospitals.
65. The capacity of PPMUs plays an important role in implementing the project. The staff
of the PPMUs should be stable, be well trained in project management, and work closely with
the technical support and regular supervisions from the central project. Although the CPMU
has only 20 staff (much less than many other projects of the same scale), the project activities
were well implemented, the schedule and work plan were always on track, and the
targets/results were surpassed in comparison to those assigned at the start of the project.
66. The support for the poor to access health care services is very well implemented in the
seven NUP provinces. Experience in implementing and critical results gained from the NUP,
is evidenced, and driven to replicate this policy nationally, through Decision No. 14 of the
Government. The policy of providing HICs to the poor should be continued to achieve
universal HI.
67. The very close collaboration and helpful support from the World Bank office in Hanoi,
by quickly exchanging information and solving the difficulties, which was the responsibility
85
of the TTL and relevant World Bank staff greatly contributed to the achievements and success
of the NUP.
86
Annex 6A. Project Risk Management (Borrower’s View)
Risks Level of
Risks Measures to Overcome
The non-poor group could
benefit from the intervention
of the project, leading to
rising inequality
Medium The project developed a good management system to
control these risks by coordinating well with Vietnam
Social Insurance in the provinces for review and
synthesis of the project beneficiaries annually. The list of
the poor and ethnic minorities who were qualified for
support from the NUP was provided to the district
hospitals. To receive the NUP support, the patients
should present the insurance card with the code ‘HN’ or
the certification of poor from the communal people’s
committee
The project implementation in
the district hospitals may not
be synchronized with the
commune health centers,
which could misuse the funds
of the HI system.
High Because the project investment and health care support
for poor people was focused at the district level, a large
part of the poor, instead of seeking health checkups at the
commune health center, tended to go to the district
hospital to access and use the support of the project,
compromising the use of the HI fund at the commune
level due to the impact from the abovementioned patient
flow. The project had controlled this risk well by
supporting only inpatients and setting the cap for the
length of stay (10 days) to avoid overuse of the support.
Uneven implementation
progress across project
components undermines the
integrated health systems
development approach
Potential mismatch between
the timing of demand-side
interventions that are expected
to boost demand for services
and investments
The project developed a yearly work plan to ensure that
the activities were implemented in an appropriate
sequence. Closely monitoring the implementation
progress to ensure coordination between components.
The completion of civil works in the first two years has
created a favorable condition for installing the medical
equipment. The trainings for health staff on using and
maintaining the different types of equipment were
conducted before and immediately after the equipment
was provided. A four-year training to upgrade assistants
to become medical doctors for commune health stations,
was implemented in 2011, when many doctors from the
district level graduated and covered for the commune
heath center. The IEC support for the poor and training
for district hospitals to manage funds for the poor were
implemented in parallel with others.
Staff not qualified for
entrance exams for level-1
specialists
Medium The project supported the staff preparation for entrance
exams.
The staff did not return to the
district hospitals to work after
graduation
Medium Recruitment of staff training was done in accordance with
local needs and selection criteria of the project. Staff
signed a terms of responsibility to return to their origin
district hospitals’ workplaces before going for training.
The certificate was issued only after staff completed the
training and returned to work in the district hospital.
Too many officers/staffs were
sent for training, affecting the
health services delivery at
health facilities
Medium The district hospital managed to schedule plans before
sending staff for training. Training courses were
organized to be spread throughout the project duration,
thus avoiding problems that could compromise the
service delivery process at the district hospitals.
87
Risks Level of
Risks Measures to Overcome
Otherwise, the project supported temporary staff transfers
among hospitals to help each other when their staff was
attending the training.
Delays in hospital civil works
could compromise the
investment plans
Medium Civil works investments were selected in district
hospitals where the area for construction is ready and
where PPMU and CPMU capacity to develop the
fiduciary tasks exist.
Project delays due to
specialized procurement,
tendering, and construction
tasks
High Procurement tasks followed the existing bidding laws and
instructions. The MoH conducted administrative reforms
in procurement, creating favorable conditions for faster
implementation. Project staff, and especially the team
working with the World Bank are trained and
accumulated experience in procurement before the
project started.
Medical equipment provided
to the district hospital by the
project are not used
effectively
Medium During the implementation, the team identified
investment needs to match the acquired equipment with
the social and epidemiological needs and organized
training and human resource development to use
equipment from the first year of the project. The team
also trained staff to carry out minor repairs, creating
favorable conditions for the installation and efficient
operation of the equipment provided.
Equipment is inadequately
maintained throughout the life
of the project
The civil works investments
in facility repairs and
refurbishments were not
maintained, leading to rapid
degradation of the facilities
Medium The project provides trainings for the hospital leaders on
hospital management, including the section of managing
the equipment. The hospital had plans and allocated
funds for maintaining/repairing equipment and facilities
based on the actual hospital needs. These tasks are
gradually improved because the hospitals are aware of the
important roles of regularly maintaining the equipment.
Unrecognized barriers
continue to limit access for the
poor, despite improved
financial access
Medium— The project regularly collected data on inpatient visits
who received support from the NUP. Data showed that
the number of beneficiaries gradually increased. The rate
of occupied hospital beds has increased to average 6%
per year. Another activity is the promotion of health-
seeking behavior through information dissemination and
other outreach activities regarding benefits and
entitlements under the HCFP. The improvement of
quality of the services also was a factor to attract the poor
who went to the hospitals for treatments
Coordinating capacity
strengthening for HCFP
management that builds on
existing systems and does not
create a duplicate
administrative system
supported by the project
Low The project has guidance for PPMUs and district
hospitals, which clearly states the roles and
responsibilities of the staff administering HCFP. The
management of HCFP has been strengthened during the
project life, through various support, including training
courses for the fund management unit, and support of
equipment to help fund management at the hospitals more
conveniently and rapidly. The examination and
monitoring of the health care for the poor has been
carried out in a systematic way, to ensure that the
88
Risks Level of
Risks Measures to Overcome
implementation is in accordance with the project’s
regulations.
The beneficiaries, not being
adequately informed of their
benefits, do not make
adequate use of their HICs
Low The project improved the quality of medical services and
conducted various types of behavior change
communication interventions to include messages geared
toward educating beneficiaries on health education, the
scheme and their benefits (support indirect costs in
addition to support from the state through the HCFP) to
poor families who went for the services.
89
Annex 6B. Project Results Framework
No. Outcome Indicators
Baseline
Survey
(2009)
MTR
(2012)
Final
Survey
(2014)
Extension
Phase
(2015)
Disparity
2009–
2014/2015
Targeted
Year 6
Indicator 1:
(original)
Utilization rates of inpatient
services in district hospitals
among Decision 139
beneficiaries (according to
household survey)
0.063 0.054 0.074 n.a. 117%
10%
increase
(supplement) Utilization rates of inpatient
services in district hospitals
among Decision 139
beneficiaries (according to
reports of district hospitals)
0.0247 0.049 0.085 0.096 389%
10%
increase
Indicator 2:
(original)
Utilization rates of outpatient
services in district hospitals
among Decision 139
beneficiaries (according to
household survey)
0.016 0.071 0.032 n.a. 200%
>15%
increase
(supplement) Utilization rates of outpatient
services in district hospitals
among Decision 139
beneficiaries (according to
reports of district hospitals)
0.067 0.162 0.206 0.247 369%
>15%
increase
Indicator 3: Percentage of households
which experienced
catastrophic health care
expenditures in the year prior
to the survey
10.4 7.1 2.0 n.a.
−81%
> 10%
decrease
Indicator 4: Percentage of health services
according to the national
norms that district hospitals
can implement
39.1 47.5 71.4 80.4 206%
70%
Result Indicators for Each Component
Component 1: Strengthening District-level Hospitals
Indicator 5: Percentage of patients
satisfied with health care
services at district hospitals
8.5 48.8 84.4 n.a. 993%
Increased
by 20%
Indicator 6: Adherence to treatment
protocols in treatment of
three common diseases
(MOH)
— — — — —
Increased
by 40%
Percentage of health staffs
with correct knowledge of
diagnosis and treatment for
ARIs
0.4 41.6 83.6 n.a. 20,900% —
Percentage of health staffs
with correct knowledge of
diagnosis and treatment for
severe pneumonia
13.2 58.1 86.9 n.a. 658% —
Percentage of health staffs
with correct knowledge of 9.7 65.5 95.2 n.a.
981%
—
90
No. Outcome Indicators
Baseline
Survey
(2009)
MTR
(2012)
Final
Survey
(2014)
Extension
Phase
(2015)
Disparity
2009–
2014/2015
Targeted
Year 6
diagnosis and treatment for
diarrhea level A/B/C
Percentage of health staffs
with correct knowledge of
diagnosis and treatment for
poisoning
26.8 53.8 83.0 n.a. 310% —
Percentage of ARIs medical
records with appropriate
diagnosis
18.5 38.6 62.2 n.a. 336% —
Percentage of severe
pneumonia medical records
with appropriate diagnosis
45.5 54.5 71.1 n.a. 156% —
Percentage of pneumonia
medical records with
appropriate diagnosis
19.6 20.9 57.0 n.a. 291% —
Percentage of diarrhea level
A/B/C medical records
appropriate diagnosis
41.1 69.4 81.6 n.a. 199% —
Percentage of poisoning
medical records appropriate
diagnosis
61.1 86.2 88.8 n.a. 145% —
Indicator 7: Percentage of district health
staffs trained by the project — — — — 145% 80%
Percentage of health staffs
completed short-term
training courses compared to
the plan
0.0 102.0 357.3 427.3 — —
Percentage of health staffs
completed long-term training
courses (specialty level-1s
doctors) compared to the
plan
0.0 54.9 88.0 189.2 — —
Total number of health staffs
trained in short-term and
long-term courses
0 2,664 8,929 11,868 — —
Indicator 8: Proportion of district
hospitals with acceptable
operations and maintenance
plans and budget for facility
and equipment maintenance
— — — — — 40%
Proportion of district
hospitals with acceptable
operations and maintenance
plans and budget for facility
49.2 30.8 99.3 85.9 175% —
Proportion of district
hospitals with acceptable
operations and maintenance
plans and budget for
equipment maintenance
77.1 61.5 99.2 93.8 122% —
Component 2: Increasing Financial Access to Healthcare Services for Decision 139 Beneficiaries
91
No. Outcome Indicators
Baseline
Survey
(2009)
MTR
(2012)
Final
Survey
(2014)
Extension
Phase
(2015)
Disparity
2009–
2014/2015
Targeted
Year 6
Indicator 9: Percentage of Decision 139
beneficiaries provided with
HI cards
94.3 86.8 94.3 95.2 101% 70%
Indicator 10 Percentage of 139
beneficiaries with cards, who
can correctly identify at least
3 rights of the health
insurance card holders
14.8 19.8 57.4 n.a. 388% 75%
Indicator 10
(supplemental)
Percentage of 139
beneficiaries with cards, who
can correctly identify at least
one right of the health
insurance card holders
74.0 98.1 94.7 n.a. 128% —
Indicator 11: Percentage of households
who identify financial
barriers as a main cause for
not seeking health care
2.0 0.9 1.2 n.a. −40%
>10%
decrease
MDG indicators 2009 2014 2015 (%)
1 Percentage of district hospitals providing
caesarean section 74.6 82.1 92,2 124
2 Percentage of district hospitals providing blood
transfusion 56.7 67.2 82,8 146
3 Percentage of district hospitals having neonatal
units meeting standards — — 49,9 —
4 Capacity of district hospitals in obstetric and
pediatric emergency care — — — —
5 Having Continuous Positive Airway Pressure
devices 25.0 75.0 79.7 319
6 Having light for jaundice phototherapy treatment 21.4 82.1 87.5 409
7 Having oxygen breathing system 42.9 75.0 78,1 182
8 Having newborn resuscitator 46.4 92.9 92.2 199
MDG indicators 2007–2008 2013–2014 (%) 8 IMR 31.1 29.4 −5
9 NMR 11.2 10.8 −4
11 MMR 178 106 −40
92
Annex 6C. Achievements by Components
Table 6.1. Status of Health Human Resources at District Hospitals in Seven Project Provinces over the
Evaluation Periods
Health Human Resources Baseline
(2008)
Final
(2013)
Extension
Phase
(2015)
Disparity
Baseline –
Extension
Phase
Number of Health
Staffs who Returned
to Work after Being
Trained under
Support of the Project,
till 2015
Number of health staffs 599 1,097 1,286 687 (2.2 times) 1,329
Master, MD 1 11 17 16 0
Level-2 doctors 0 9 11 11 0
Level-1 doctors 148 325 396 248 (2.7 times) 351
General doctor 450 752 818 368 (1.8 times) 978
Postgraduate or above 27 87 116 89 (4.3 times) 33
Pharmacist (college,
secondary school) 207 486 554 347 0
Midwife 355 525 612 257 0
Nurse 1,124 2,059 2,238 1,114 0
Number of district
hospitals 61 67 68 — 68
Source: Reports of district hospitals.
Table 6.2. Number of Level-1 Doctors Trained in Project Provinces
Province Plan
Level-1 Doctors who Graduated
Level-1 doctors who
Continue to Work at
Localities
Total of
Level-1
Doctors
Supported
by the
Project
Total of
Person-
Years55 Late
2013
Late
2015
%
Female
%
Ethnic
Minority
Total %
Female
%
Ethnic
Minority
Cao
Bang 23 56 56 57 91 56 57 89 56 (243%)
112
(243%)
Bac Kan 31 36 50 50 98 44 57 95 50 (161%) 100
(161%)
Ha
Giang 38 89 98 68 59 95 69 57 98 (258%)
196
(258%)
Lao cai 22 38 50 38 20 49 39 24 51 (232%) 101(230%)
Lai Chau 22 17 18 28 50 18 28 50 19 (86%) 37 (84%)
Dien
Bien 19 47 50 50 44 50 50 44 50 (263%)
100
(263%)
Son La 28 26 53 42 21 53 42 21 53 (189%) 106
(139%)
Total 183 309 367 53 57 351 55 56 377(206%)
752
(205%)
93
Table 6.3. Number of Four-year Added Doctor and Pharmacist Graduated by the Evaluation Time
Cao
Bang
Bac
Kan
Ha
Giang
Lao
Cai
Lai
Chau
Dien
Bien
Son
La
Total
Support for four-year added doctor training
Total 185 82 302 208 182 250 258 1467
Number of doctors who graduated by late
2014
104 42 130 106 91 142 119 734
Person years supported by late 2014 505 191 788 544 577 714 690 4009
Converted to number of persons with full
four-year added training
126 48 197 136 144 178 173 1002
Number of doctors who graduated by late
2015
131 53 213 145 142 192 182 1058
Number of doctors who returned to work at
their units
122 44 208 96 136 193 179 978
Person years supported by late 2015 586 231 960 646 668 822 829 4742
Converted to number of persons with full
four-year added training
147 58 240 162 167 206 207 1186
Plan achieved (%) 147 144 192 179 128 179 148 160
Rate of ethnic minority doctors 99.5 100 36.1 36.1 15.4 33.6 44.2 46.1
Rate of female doctors 98.9 50 32.1 40.9 43.4 29.2 39.9 45.1
Support for four-year added pharmacist training
Total 13 7 15 23 13 16 11 98
Rate of ethnic minority pharmacists (%) 93 86 31 17 7 6 18 31
Person years supported by late 2014 31 11 28 57 29 29 19 204
Converted to number of persons with full
four-year added training 7.8 2.8 7.0 14.3 7.3 7.3 4.8 51.0
Graduated by 2015 7 1 6 13 7 3 3 40
Person years supported by late 2015 44 18 41 82 42 45 30 302
Converted to number of persons with full
four-year added training 11 4.5 10.25 20.5 10.5 11.25 7.5 75.5
Rate of ethnic minority pharmacists 92.3 71.4 20 26.1 7.7 6.0 18.2 30.6
Rate of female pharmacists 86.4 71.4 60 82.6 69.2 50 72.7 70.4
Table 6.4. Number and Rate of Clinical Staff Attending Short-term Training Courses on Health
Examination and Treatment in the Extension Phase
Course Plan Total Percentage versus Project
Document
Anesthesia 117 157 134
Testing 95 149 156
Emergency care 140 152 109
External medicine 101 143 142
Pediatrics 103 214 208
Internal medicine 111 194 175
Diagnostic imaging 200 436 218
Obstetrics 102 191 187
Communicable diseases 80 165 206
X-ray 107 121 113
Ear, nose, and throat, and dental 108 70 65
Nursing management 70 63 90
Pediatric emergency n.a. 681 —
94
Course Plan Total Percentage versus Project
Document
Obstetric emergency n.a. 687 —
Total 1.334 3.423 257
Table 6.5. Number and Percentage of Doctors/Assistant Doctors Participating in Short-term Training
Courses on Mother and Child Health Care in the Extension Phase
Training Course
Planned Quantity
(Number of
Trainees per
Course)
Actual
Quantity
(Number of
Trainees per
Course)
%
Training to standardize the health workers who work
on obstetrics/gynecology be certified as SBA 35 22 80
Training to standardize the health workers who work
on obstetrics/gynecology be certified as SBA 600/30 courses 794/30 courses 113
Training to standardize the general doctors who work
on obstetrics/gynecology at district/commune be
certified as SBA
420/21 courses 307/12 courses 118
Emergency care for obstetrics//gynecology medical
doctors at district level 154/7 courses 110/5 courses 69
Village midwives 240/12 courses 252/12 courses 105
Training for clinical staff at neonatal units - district
hospitals
70–100/7–10
courses 79/6 94
Training for service providers on Integrated
Management of Childhood Illness 350/14 courses 140/6 courses 40
Table 6.6. Number of Doctors Trained on Management, Comparison between the Planned and the Actual
Achieved
Type of Training Planned
Quantity
Actual
Quantity % Achieved
Hospital management 603 753 125 Exceeded
Medical waste management 215 2,922 1,300 Exceeded
Health system management and HMIS 806 845 105 Exceeded
Maintenance of medical equipment 171 206 120 Exceeded
Effectively using the medical equipment provided by
the project — — — —
Effectively using laboratory equipment 35 35 100 Achieved
Exploring and using equipment for emergency care,
fluid vacuum, aerosol 7 7 100 Achieved
Effectively using disinfection equipment 1 31/1 100 Achieved
95
Table 6.7. Upgrading of 18 District Hospitals
No. Name of Facilities, Location Start
Date
Total Investment
(VND)
Completed
Date
Settlement
Date
1 Renovate, minor repair of Pac
Nam district hospital September
2010
884,641,819 January 2011
January 2012
2
Renovate, minor repair of
Ngan Son district hospital 780,595,945
3
Renovate technical house of
Bao Lam general district
hospital
March
2010
694,900,000
June 2010
June 2012
4
Renovate technical house and
toilet - Tra Linh general district
hospital
859,000,000
5
Inpatient house for external
and internal department -
Quang Uyen general district
hospital
3,750,000,000 March 2010–
March 2011
6
Inpatient house for external,
internal, and traditional
medicine department - Hoa An
general district hospital
July 2011 10,354,000,000 July 2012 July 2013
7 Renovate Muong Cha district
hospital
August
2009 2,480,318,000 June 2012 June 2012
8 Renovate Dien Bien Dong
district hospital
December
2010 11,469,000,000
December
2011
December
2012
9
Meo Vac general district
hospital: technical house and
supporting facilities
April
2010
4,531,446,000
April 2011 April 2012 10
Quan Ba general district
hospital: technical house,
toilet, and water tank
3,602,837,534
11
Hoang Su Phi general hospital:
outpatient examination house
and administrative
5,020,981,190
12 Minor repair of Phong Tho
district hospital
November
2009
163,940,587
December
2009
December
2010 13
Minor repair of Than Uyen
district hospital 617,135,748
14 Minor repair of Tan Uyen
general district hospital 163,194,151
15 Build high-tech house for Than
Uyen District Health Centers May 2011 8,525,000,000 May 2012 May 2013
16
Repair and upgrade building
for technical, pharmacy
department, nutrition
department of Moc Chau
general district hospital
November
2011 12,937,587,000
November
2012
November
2013
17
Renovate consultation
department and emergency
resuscitation department of
Mai Son general district
hospital
December
2011 907,320,000
September
2012
September
2012
96
No. Name of Facilities, Location Start
Date
Total Investment
(VND)
Completed
Date
Settlement
Date
18
Renovate technical house of
Muong La general district
hospital
December
2012 1,249,290,000
December
2012
September
2012
Total 68,991,187,974
Table 6.8. Bids Procured by CPMU
No Code Name of Bids Quantity Total Cost
(US$) Notes
1 ICB 01 Equipment and technology (6 types) 278 1,839,775.25
2 ICB 02 Equipment for intensive care (8 types) 873 1,019,763.17
3 ICB 03 Monitor (3 kinds) 142 845,600.00
4 ICB 04 Ventilator (2 kinds) 112 1,281,178.00
5 ICB 05 Sterilization device (3 kinds) 160 1,449,028.38
6 ICB 06 Furnaces for medical waste treatment 39 Canceled
7 ICB 07 Ambulance (52 pieces) 52 3,071,809.16
8 ICB 08 Working cars (8 pieces) 8 908,794.77
9 ICB 09 Communication equipment (11 kinds) 140 524,408.00
10 ICB 10 Ultrasound (2 types) 109 1,471,900.00
11 ICB 11 Endoscopic systems (2 types) 77 Canceled
12 ICB 12 Cardiopulmonary resuscitation (2 types) 366 386,805.00
13 ICB 13 Medical equipment (4 categories) 337 956,065.00
14 ICB 14 Operating room equipment (6 types) 396 4,257,544.49
15 ICB 15 Surgical instruments (7 types) 655 532,867.05
16 ICB 16 Special equipment (6 types) 365 2,198,896.00
17 ICB 17 X-ray (2 types) 102 1,309,275.00
18 ICB 18 Laboratory equipment (9 categories) 565 1,095,884.64
19 ICB 19 Basic equipment (19 kinds) 115 467,885.41
20 ICB 20 Monitor device (5 categories) 94 764,210.00
21 NCB 01 Ambulance (7 pieces) 7 393,180.20
22 ICB 21 Pediatric surgical equipment and
gynecology (6 types) 174 589,105.60
23 ICB 22 Emergency resuscitation equipment and
obstetrics (7 types) 160 529,963.96
24 ICB
01/2015
Model for trainings on obstetric care and
clean package for delivery, tool bag for
traditional birth attendants s
1.276.708,92
Total: 25,893,939.08
97
Table 6.9. Total Times of Inpatients Supported by NUP, by Provinces, 2009–2014
Province 2009 2010 2011 2012 2013 QI/2014
Dien Bien 411 1,779 3,814 16,889 30,058 7,524
Lai Chau — 1,264 4,205 13,605 18,132 4,716
Son La — 875 2,617 24,856 44,195 10,106
Cao Bang 186 4,853 18,129 44,248 45,044 10,533
Bac Can — 3,654 12,769 26,975 27,756 7,088
Ha Giang 30 3,457 17,727 38,781 39,702 11,231
Lao Cai 427 3,508 20,252 35,345 39,914 8,020
Total 1,054 19,390 79,469 200,709 244,801 59,218
Table 6.10. Number of Poor/ethnic Minority People being supported from the HCFP in the Project
Provinces - Extension phase
No. Contents Son La Ha Giang Lai Chau Lao Cai Total
1 Total times
poor/ethnic
minority
people
were
supported
52,597 82,676 46,147 87,985 269,405
Supported
travelling
expenses
51,174 82,676 305 57,354 191,509
Supported
meal
expenses
52,597 82,676 36,907 87,985 260,165
Supported
direct
expenses
for health
care
3,896 — 8,935 45,683 58,514
2 Total
expenses
supported
by the
HCFP
(VND)
16,175,884,199 33,397,067,456 30,185,983,270 19,128,174,711 98,887,109,636
Supported
travelling
expenses
5,007,080,521 — 1,138,446,108 2,213,686,966 8,359,213,595
Supported
meal
expenses
10,843,276,112 — 4,998,535,291 16,203,882,250 32,045,693,653
Supported
direct
expenses
for health
care
325,527,566 — 24,049,001,871 710,605,495 25,085,134,932
98
Annex 7. NUP (P082672) Project Supervision Missions Datasheet
Year
Number of
Missions, Dates, and
TTLs
Provinces
Visited During
the Missions
Main Issues Raised and Solved during the Missions
2009 2 (May and
November)
TTL: Bakhuti
Shengelia
2 provinces
Ha Giang, Dien
Bien
May: Review project implementation progress by components; discuss the baseline survey; discuss on how to
accelerate implementation of work plan; review arrangements for the implementation of ‘Health Care for the Poor’
component at the district level; and finalize training arrangements.
June: Review the implementation of 2009 plans; discuss 2010 action plans, related procurement plans, and the
measures to improve the effectiveness of implementation; assess the participatory planning process under the HEMA
Project and review M&E arrangements; conduct post review of procurement activities in 2009 and the financial
management practices; and conduct an implementation assessment in Dien Bien province.
2010 2 (June and
December)
TTL: Bakhuti
Shengelia
2 provinces: Lai
Chau, Son La
(for financial
management
review)
June: Review the implementation progress of 2010 annual work plans; review implementation of the
recommendations of November 2009 supervision mission; review procurement and financial management processes
at the CPMU and PPMUs; update implementation and procurement plans as needed; and assess developments in the
health policy field that may have implications for the projects.
December: Review the implementation progress of 2010 annual work plans; review implementation of the
recommendations of November 2009 supervision mission; review procurement and financial management processes
at the CPMU and PPMUs; update implementation and procurement plans as needed; and assess developments in the
health policy field that may have implications for the projects.
2011 2 (May and October)
TTLs: Bakhuti
Shengelia and Kari
Hurt
Note: An additional
safeguard review
mission in December
2011
4 provinces
Bac Can, Cao
Bang, Lai Chau,
Lao Cai
May: Review the general assessment of the project progress by component and activities; review procurement,
financial management, and environmental safeguards; and review social safeguards.
October: Discuss the progress toward achieving the DOs of increased utilization particularly by the poor in the project
provinces and strengthen district hospitals; review the implementation progress of 2011 annual work plans and look at
the priorities for the 2012 annual work plans; document the progress against the recommendations of December 2010
supervision mission; assess procurement and financial management processes at the CPMU and PPMUs; update
implementation and procurement plans as needed; and support progress of the current project implementation issues,
particularly concerning the HCWM plan and investments in support of the hospital plans and the implementation of
pilots for retention of rural health professionals in the project provinces.
2012 1 (July - MTR
mission)
TTL: Kari Hurt
3 provinces
Ha Giang, Son
La, and Dien
Bien (for
financial
management
only)
Review the general assessment of the project progress by component and activities; review procurement, financial
management and environmental safeguards, review social safeguards. Evaluation of the Project Results Matrix as part
of the MTR.
99
Year
Number of
Missions, Dates, and
TTLs
Provinces
Visited During
the Missions
Main Issues Raised and Solved during the Missions
2013 2 (May and
December)
TTL: Thuy Anh
Nguyen *December mission
for financial
management review
only.
4 provinces
Son La (May),
Son La, Lai
Chau, Dien Bien
(December for
financial
management
review)
May: Review the general assessment of the project progress by component and activities; time remaining for
disbursement, M&E, procurement medical equipment, financial management, and environmental safeguards; and
review social safeguards.
December: Update the World Bank’s understanding on the financial management arrangements for the project since
last financial management supervision missions (Son La in November 2012, Lai Chau in May 2013, and Dien Bien in
April 2012) which covers all areas of financial management, including planning and budgeting, disbursement, funds
flows, accounting system and software, reporting, and auditing. The mission will also follow up on all outstanding
issues raised during the previous mission and in the 2012 audited financial statements.
2014 2 (January and
September**)
3 provinces:
Dien Bien
(April),
Cao Bang, Bac
Can (September
2013)
January: Review the general assessment of the project progress by component and activities; time remaining for
disbursement, M&E, procurement medical equipment, financial management, and environmental safeguards; and
review social safeguards.
September: interim mission. No official announcement was sent.
2015 2 (May and
December**)
No site visit May: Key issues for ministry and project management attention, progress and key issues by component and
subcomponent; M&E; financial management and disbursement arrangements; procurement, safeguards
December: Review the implementation progress of the project by components and actions agreed and concluded in
the aide memoires from mission in May, 2015, to discuss and agree the final actions for project closing in February;
to discuss the preparation for the final implementation support mission combined with the ICR mission that is planned
in late February/early March 2016.
2016 1 (February) Lai Chau; Lao
Cai
Closing Mission - Discussion of the ICR assessment
Notes:
*There are also procurement post review missions, at least one every year.
** No aide memoires for September 2014 mission because of the TTL’s sudden sickness and December 2015 mission as it may be combined with the ICR
mission which is two months later.
100
Annex 8. List of Supporting Documents
1. Bill and Melinda Gates Foundation.2016. Maternal, Newborn and Child Health Strategy
Overview, in http://www.gatesfoundation.org/What-We-Do/Global-
Development/Maternal-Newborn-and-Child-Health.
2. Ekman, B; L. Thanh, H. A. Duc, and H. Axelson. 2008. “Health insurance reform in
Vietnam: A Review of Recent Developments and Future Challenges.” Health Policy and
Planning 23:252–263, doi:10.1093/heapol/czn009.
3. Center for Environmental and Health Studies. 2014. Report on the Final Evaluation of
the Northern Uplands Health Support Project, Hanoi, July 2014.
4. Center for Environmental and Health Studies. 2016. Report on the Final Evaluation of
the Northern Uplands Health Support Project in the Extension Phase, Hanoi, February
2016.
5. Fosberg, L.T,.2011. The Political Economy of Health Care Reform in Vietnam, Oxford –
Princeton Global Leaders Fellow, Woodrow Wilson School of Public and International
Affairs, Princeton University, 2011.
6. World Bank. 2011. Country Partnership Strategy for the Republic of Vietnam, IBRD,
IDA, Report No. 62500-VN.
7. World Bank. 2006. Mekong Regional Health Support Project. Project Appraisal
Document. Washington DC, February 9, 2006.
8. World Bank. 2012. Mekong Regional Health Support Project Implementation
Completion Report Results, Washington DC, December 2012.
9. World Bank. 2008. Northern Upland Regional Health Support Project, Project Appraisal
Document, The World Bank, Washington DC, February 2008.
10. Government of Vietnam, Ministry of Health. Annual Health Statistics Year Book, Hanoi,
Series from 2007–2014.
11. Government of Vietnam. 2012. Ministry of Health. Master Plan on Universal Health
Insurance Coverage, draft, Hanoi, June, 2012;
12. Government of Vietnam, Ministry of Health. 2015. Success Factors for Women’s and
Children’s Health, Ed. WHO, Geneva, 2015
13. Lieberman, S. and Wagstaff, A. 2009. Health Financing and Delivery on Vietnam –
Looking forward. Health, Nutrition and Population Series, The World Bank, Washington
DC, 2009.
14. Tiberti, L. 2000. Health Sector Reform in Transition Economies: The Case of Vietnam,
University of Florence, Florence, 2000.
101
15. Toan, Tran Khanh .2012. Antenatal and Delivery Care Utilization in Urban and Rural
Contexts of Vietnam: A study in two health and demographic surveillance sites, Doctoral
thesis at the Nordic School of Public Health NHV, Gothenburg, Sweden, 2012.
16. World Health Organization .1999. Mother-Baby Package Costing Spreadsheet, Geneva,
Switzerland, 2012.
REPORTS PRODUCED BY THE GOVERNMENT TO SUBSIDIZE THE PROJECT
PREPARATION
1. Health status in the 7 provinces of Northern Upland
2. Health care service use and accessibility status in the 7 provinces of northern upland: Cao
Bang, Bac Kan, Lao Cai, Ha Giang, Son La, Dien Bien and Lai Chau)
3. Health Human Resource Analysis in the 7 provinces of northern upland - Health care for
the Poor: Identification of the needs and proposal of investment for capacity building and
management capacity development
4. Assessment of the Healthcare Fund for the Poor in the 7 provinces of northern upland -
Health care for the poor: management according to the Decision 139 in 7 provinces of
northern upland (most difficult provinces)
5. Health System Assessment for 7 provinces of northern upland - Inventory of medical
equipment in hospitals of 7 provinces of northern upland
6. Output indicators after analysis - Socioeconomic, Demographic, Cultural Geographic and
Health Status indicators: Morbidity, Mortality, CDR/IMR, Under 5-child nutrition status
7. List of medical staff to be trained and cost estimate and cost table for training component
102
Map
Source: World Bank Maps