Upload
others
View
2
Download
0
Embed Size (px)
Citation preview
1. BASIC INFORMATION
a. Basic project data
Project title: AGRICULTURAL SECTOR REHABILITATION PROJECT (ASREP)
Project code: P-SL-AA0-
007
Instrument number(s): 2100150009295 (loan); 21000155004167 (grant)
Project type: Lending Sector: Agriculture & Agro-industry
Country: SIERRA LEONE
Environmental categorization (1-3):
Processing Milestones Key Events Disbursement and Closing date
Financing source: ADF Loan
UA10,000,000
Financing source: ADF Loan
UA10,000,000
Financing source: ADF Loan
UA10,000,000
Date approved: 2 February 2005 Cancelled amount: USD 285,456.30
Original disbursement deadline: 31
December 2011
Date signed: 4 April 2005 Supplementary financing: NIL Original closing date: 31 March
2012
Date of entry into force: 2
August 2005
Restructuring: African Food Crisis
Response – 24 July, 2008. Amount: UA 833,105 (USD 1, 291, 154)
Revised disbursement deadline: 31
December, 2013
Date effective for 1st
disbursement: 26 February 2006
Extensions (specify dates): 1st on 24 June
2011. From 31 December 2011 to 31
December 2012; 2nd on 14 September
2012. From 31 December 2012 to 31
December 2013
Revised closing date: 31 March,
2014
Date of actual 1st: 9 November
2006
Financing source: ADF grant
UA2,000,000
Financing source: ADF grant
UA2,000,000
Financing source: ADF grant
UA2,000,000
Date approved: 2 February 2005 Cancelled amount: USD 34,553.76 Original disbursement deadline: 31
December 2011
Date signed: 4 April 2005 Supplementary financing: NIL Original closing date: 31 March
2012
Date of entry into force: 2
August 2005
Restructuring: African Food Crisis
Response – 24 July, 2008. Amount: UA 833,105 (USD 1,291,154)
Revised disbursement deadline: 31
December 2013
Date effective for 1st
disbursement: 26 February 2006
Extensions (specify dates): 1st on 24 June
2011. From 31 December 2011 to 31
December 2012; 2nd on 14 September
2012. From 31 December 2012 to 31
December 2013
Revised closing date: 31 March
2014
Date of actual 1st dis 9
November, 2006.
b. Financing sources
Financing source/ instrument Approved
amount (UA):
Disbursed amount
(UA):
Percentage disbursed
(%):
ADF Loan 10,000,000 9,598,679.81 95.99
PCR EVALUATION NOTE FOR PUBLIC SECTOR OPERATIONS
ADF Grant 2,000,000.00
1,516,474.97 75.82
Government: 1,600,000.00 422,400.00 26.40
Other (ex. Co-financiers):
TOTAL: 14,100,000.00 11,537,554.78 83.90
Co-financiers and other external partners: None
Execution and implementation agencies: Ministry of Agriculture, Forestry and Food Security (MAFFS)
c. Responsible Bank staff
Position At approval At completion
Regional Director Frank Joseph Marie PERRAULT
Sector Director Chiji Chinedum OJUKWU
Sector Manager Dougou KEITA
Task Manager Yero BALDEH
Alternate Task Manager
PCR Team Leader
PCR Team Members
d. Report data
PCR Date: 25 June 2014
PCR Mission Date: From: 2 June 2014 To: 11 June 2014
PCR-EN Date:
Evaluator/consultant: Tesfaye Teklu Task Manager: Foday Turay
2. PROJECT DESCRIPTION
The African Development Bank approved financing the Agricultural Sector Rehabilitation Project
(ASREP) in February 2005. The Government of Sierra Leone (SoL) and the Bank signed the loan and
grant agreements in April 2005, and committed to implement it over a period of six years. The purpose
of the project was to rehabilitate the agriculture sector to enable farmers to restart farming, and increase
agricultural production and income in targeted five districts of the country. ASREP started operation in
2006 and closed in 2013, after a two-year extension.
a. Rationale and expected impacts:
The large but low performing pre-war agriculture sector worsened during the decade-long civil war as
consequences of widespread population displacement, abandonment of farms, degradation of agriculture
support institutions such as agriculture research and extension, and deterioration of rural infrastructure.
The period was marked by continuous decline in agriculture production and income, and deterioration in
living conditions (decline in food availability and food security, and rising poverty).
Major effort was necessary to assist Sierra Leone to jump start the agriculture sector, i.e. to enable
farmers to engage in farming through rehabilitating farms, rehabilitating agriculture infrastructure and
institutions, producing and disseminating improved seeds and planting material, and providing training
on improved agricultural practices. It was in this context that the Government of Sierra Leone (GoS)
sought donor assistance including the African Development Bank to support the financing of the
agriculture sector rehabilitation program (ASREP). The objective of the AfDB ASREP was to increases
agricultural production and incomes of small farmers in five districts through support to agricultural
production, capacity building of MAFFS and rural communities, and strengthening project management
and coordination.
The expected impacts of the project were improved agricultural productivity and production, increased
farm income, enhanced food security, and reduced poverty. The intended beneficiaries were primarily
small farmers but also rural households at large in the ASREC districts.
b. Objectives/Expected Outcomes:
The specific objective of ASREP was to increase agricultural production and improve farm income.
Attainment of this objective would contribute to the sector goal of enhancing food security and poverty
reduction.
Expected outcomes would include: (1) farmers with access to farm assets (e.g. farm tools and
implements); (2) area of land reclaimed and cultivated at farm level; (3) farmers with access to improved
inputs (i.e., improved seeds/planting materials); (3) farmers trained through farmer field school (FFS)
and/or contacted extension agents; (4) increased cropping intensity particularly among IVS farms; (5)
improved crop yield; and (6) improved farm level output.
The intended beneficiaries were small farmers in ASREC districts.
c. Outputs and intended beneficiaries:
The following were the key outputs expected to contribute to the expected outcomes by component:
Component 1: Agriculture production support
Lowland farms rehabilitated mainly in Inland Valley Swamps (quantity of farm tools and
equipment supplied, water control structure established, and quantity of improved rice seed
disseminated)
Tree crop plantation rehabilitated (quantity of farm tools and equipment supplied, quantity of
improved planting material disseminated)
Improved seed and planting materials produced (foundation seeds produced at research stations
(rice, non-rice annual crops and planting material for tree crops), seeds and improved planting
material multiplied and disseminated)
Environmental and social monitoring programme implemented
Component 2: Capacity building of MAFFS and rural communities
MAFFS field offices and facilities rehabilitated/reconstructed (civil works, vehicles, office
equipment, and supplies)
Rice Research Stations (RRS) and Institute of Agricultural Research (IAR) stations
rehabilitated/reconstructed (civil works, vehicles, office equipment, and supplies)
Community-based extension services promoted (i.e. Farmer field schools, Farmer Facilitators,
farmers trained)
Rural roads rehabilitated
Component 3: Project management and coordination (project implementation unit strengthened through
technical, institutional and logistic support).
d. Principal activities/Components:
The project was designed to achieve its expected outputs, outcomes and project objectives through the
following components and associated activities:
Component 1: Agriculture production support
Rehabilitate lowland farms (Inland Valley Swamps, IVS)
Rehabilitate tree crop plantation
Produce quality foundation seed and planting material, multiply and disseminate
Design and implement environmental and social monitoring programme
Component 2: Capacity building of MAFFS and rural communities
Rehabilitate and reconstruct MAFFS field offices and facilities (civil works, vehicles, office
equipment, and supplies)
Rehabilitate and reconstruct RRS and IAR stations (civil works, vehicles, office equipment, and
supplies)
Build capacity of extension services (i.e. establish farmer field schools, and train farmers, farmer
facilitators, and extension agents)
Rehabilitate rural roads
Support to health services (i.e. awareness promotion of HIV/AIDs and malaria prevention)
Component 3: Project management and coordination
In short, the main project activities included: a) rehabilitation of lowlands farms mainly farms in Inland
Valley Swamps; (2) rehabilitation of tree crop plantation; 3) seed and planting materials production and
dissemination; (4) capacity building of research and extension services (establishment of farmers
training schools); (5) rehabilitation of Ministry of Agriculture, Forestry and Food Security (MAFFS)
offices, research and extension infrastructure; and (6) rehabilitation of feeder roads to increase farmers
access to markets.
III. PROJECT PERFORMANCE ASSESSMENT
RELEVANCE
a. Relevance of the project development objective:
ASREP sought to get agricultural growth started through rehabilitation of farms to increase agricultural
production and income in five districts of Sierra Leone. This objective was consistent with the urgency
and priorities that the Government of Sierra Leone accorded to the rehabilitation of the agriculture sector
on a path to long-term sustainable growth in agricultural production to achieve national food security
and poverty reduction.
The project objective was also in accord with the poverty reduction strategy adopted at the time of the
project preparation and appraisal, i.e. Interim Poverty Reduction Strategy Papers of 2001–2003 (I-PRSP I) and 2003–2005 (I-PRSP II), in which agriculture was the core sector for poverty reduction.
The ASREP objective remained aligned with the country’s main policy documents from appraisal to completion. The 2009 Sierra Leone CAADP, for example, included ASRAP’s interventions under its agriculture infrastructure development sub-program. The ASREP objective was also consistent with the
Government’s Agenda for Change (2008 - 2012) and Agenda for Prosperity (2013 – 2018).
The project was consistent with the Bank Group strategy for Sierra Leone as indicated in its country
strategy paper for 2002 – 2004 in which it addressed post-conflict recovery program. The project was also in line with the Bank’s vision and plan for poverty reduction through agriculture and rural development.
The PCREN rates the objective of ASREP highly satisfactory (4) in agreement with the PCR; it was
aligned with Sierra Leone rehabilitation and development policies and strategies, the Bank’s sector strategies and the beneficiary needs throughout of the project cycle.
b. Relevance of project design (from approval to completion):
The appraisal report establishes the rationale and the logic of the Agricultural Sector Rehabilitation
Project (ASREP). In the aftermath of the decade-long civil war, the Government of Sierra Leone sought
donor assistance including the African Development Bank to support the financing of the agriculture
sector rehabilitation program to jump start agricultural growth. In response, the Bank requested FAO to
prepare the ASREP project and then sent its mission to Sierra Leone in 2004 for appraisal. The design of
ASREP was informed by the findings of the 2003 agricultural sector review and consultations with
principal stakeholders including project beneficiaries and development partners.
The thrust of ASREP was to achieve its objective of increases in agricultural production and incomes of
small farmers in five districts through (1) support to agricultural production, (2) capacity building of
MAFFS and rural communities, and (3) strengthening project management and coordination. The logical
framework in the appraisal reports shows the internal logic of the project where the activities in the three
components (section 2d) generate a range of outputs (2c) that contribute to the realization of the
project’s objective (increase in agricultural production and income). Missing in the results chain are the intermediate outcomes that connect the output stream to the project objective. The framework specifies
key assumptions and risks, and some mitigation measures.
However, there were deficiencies in quality-at-entry and implementation arrangements as evident in the
project’s prolonged start up and suspension. Some adjustments were necessary during the project implementation that involved cancellation of some activities, reprioritization of remaining activities, and
revision of planned targets.
Overall, the PCREN rates design relevance 3 (satisfactory) in line with the PCR rating, i.e. the design
was relevant at appraisal but adjustments were required during the implementation phase, which were
done satisfactorily.
EFFECTIVENESS
c. Effectiveness in delivering outputs:
The project achieved the following key outputs by component with ratings of their performance (relative
to targets set at appraisal):
Component 1: Agriculture production support
Lowland farms (Inland Valley Swamps) developed, 3529 ha (70.6%)
Tree crops rehabilitated, 7300 ha (91.2%)
Lowland rice seed produced, 44 Mt (126%)
Seed and planting materials, other crops (not reported)
Component 2: Capacity building of MAFFS and rural communities
MFFS field offices rehabilitate, 93 (107%)
RRS facilities rehabilitated for production of seed rice, 3 (100%)
IAR stations rehabilitated for production of seed for other food crops, 3 (100%)
Farmer field schools (FFS) established for training of farmers and extension agents, 325 (33%)
District facilitator trained, 12 (100%)
Farmer Facilitators (not reported)
Farmers trained on improved crop production practices (no specific figure on farmers trained)
MAFFS district staff trained in different technologies, 100%
Scientists trained in different technologies, 58%
Communities sensitized in HIV/AIDs and malarial prevention
Market sheds constructed, 2 (50%)
Feeder roads rehabilitated, 138 km (34%)
Bridges/water crossing points established, 69 (150%)
Component 3: project management and coordination
Overall assessment: The expected outputs, as listed in section 2C, constitute arrays of results: farms
rehabilitated (IVS and plantation farms), agricultural institutions rehabilitated and reconstructed,
improved seeds and planting material produced, farmer-based organizations (FBOs) established, out-
grower schemes developed for multiplication and onward sale of certified seed to farmers, farmer field
schools (FFS) established, farmers and facilitators trained through FFS, and rural roads rehabilitated.
Most of the expected outputs are reported. ASREC had achieved most of its revised output targets. The
notable under performers were the establishment of FFS schools and rehabilitation of rural roads. There
are missing data on some expected outputs, for example, non-rice crop seeds and planting materials
produced and disseminated, numbers of farmers and facilitators trained through FFS, FBOs formed, and
out-grower schemes developed.
Missing in the PCR is a summary table showing the outputs by year, which enables to characterize the
temporal dimension of the outputs. Note that by the time of the medium-term mission (three years into
the implementation phase of the project), the key planned outputs were not implemented – rehabilitation of farms, seed production, establishment of out-grower scheme.
Extra: The output performance of ASREP was boosted positively by Bank’s Africa Food Crisis Response (AFCR). Targeted farmers in ASREP districts received free improved seed, fertiliser, farm
tools and power tillers, rice mills, threshers and cutters. The Department of Operation Feed the Nation
(OFTN) within MAFFS managed the selection of beneficiaries and distribution process.
In short, the project was slow to start and took longer than the planned timeframe but largely achieved its
planned outputs at project closure. PCREN rates the project’s outcome performance satisfactory (3) in line with the PCR.
d. Effectiveness in delivering outcomes:
As shown in section 2(b), the expected outcomes of ASREP would include: (1) farmers with access to
improved seeds/planting materials; (2) farmers trained through FFS -farmers field school; (3) farmers
with extension contact -through staff-led or farmer-led; (4) farmers participating in FBOs (farmer-based
organizations); (5) improved crop yield; and (6) improved farm level output.
Some of these outcomes were reported in different sections of the PCR. The following are reported
under “Beneficiaries” in section II B (5) as well as under “output” in section II B (3):
13600 farm families involved in the production of food crops and those involved in the production of crop seeds and other planting materials, 85%
3400 farm families involved in tree crops (especially cacao, coffee, and oil palm) farming systems and production, 86% (female 38%)
15,470 farm families that directly benefited from training, receipt of farm tools and equipment, extension services facilitated by the project, 85%
1500 farmers trained in good agricultural practices through Farmer Field School (FFS), 75% (female 35%)
3500 farmers received training on HIV/AIDs and malaria prevention, 59%
Increase in farm level productivity by crop in the target districts. The target was either met or exceeded in only two cases, rice (probably due to the NERICA variety) and groundnut.
These measures of “extent of participation” of beneficiaries indicate sizable numbers of farmers benefited through ASREN in the project districts. Note, however, information was lacking on level of
adoption of improved technology at farm level (e.g. adoption of improved rice variety, improved tree
crop variety), and incremental production and farm income among project beneficiaries. The PCR
reports productivity growth by crop in the target districts, presumably drawing from MFFS production
assessment statistics. But, because the M&E system was not developed and no project-specific outcome
variables were monitored, it is hard to relate the district-level performance to outcome performance at
project level.
The PCREN recognizes the project had reached sizable numbers of rural households in providing
support for rehabilitation of their farms. And there were signs of growth in agricultural production and
productivity. Hence, the PCREN rates the project’s outcome performance marginally satisfactory (3), a slight downgrade to the satisfactory rating (3) in the PCR
e. Project development outcome:
Essentially, the direct measurement of achieving the project’s development objective is estimating the increase in household incomes and reduced household food insecurity of the project’s beneficiaries. If
there are no direct measures of these impact indicators, the performance in output and outcome (i.e. the
intermediate indicators between outputs and final impact outcomes) can be used to gauge progress
towards achieving the development outcomes (i.e. the final outcomes).
Seemingly along the direct measurement approach, the PCR reports the following evidence of the
development effectiveness of ASREC:
Enhanced food security (through increase in agricultural production), 97.9%
Reduce poverty (growth in GDP per capita), 110% (GDP per capita)
Population in poverty decreased 66.4% at appraisal to 52.9% in 2013.
The PCR concludes that “all the project activities were carried out resulting in the achievement of most of the planned outputs and significant progress with the planned outcomes (enhanced food security and
reduction in poverty”. The PCR conclusion is notwithstanding its own admission in the section under cost-benefit analysis of the “weakness in the project M&E and the absence of studies to ascertain achievements in outputs and outcomes”.
The PCREN cannot comment on these results since no documents were available to assess the source of
data, survey methodology, and method of measuring the indicators. Even harder is to confirm the
attribution of the outcome performance to the project, which the PCR suggests, in absence of counter
factual evidence. There are several factors that contribute to increase in household income and food
security. Some of these factors are due to the project. But there are others that hinder or boost the
translation along the results chain. There is no evidence that the analysis accounted for the other factors
in counter factual framework. It is thus hard to give credence to the attribution to the project.
The alternative in absence of clear-cut evidence of the ASREC impact on enhanced farm income, food
security and poverty reduction is to follow the rating methodology recommended in the Staff Guidance
Note on project completion reporting and rating. The PCREN rates the development objective of the
project marginally satisfactory (3) since it rates output performance satisfactory (3) and outcome
performance marginally satisfactory (3). This rating is a slight downgrading of the satisfactory rating in
the PCR.
f. Beneficiaries:
Using evidence, the evaluator should provide an assessment of the relevance of the total number of
beneficiaries by categories and disaggregated by sex.
3400 farm families involved in tree crops (especially cacao, coffee, and oil palm) farming systems and production, (female 38%)
1500 farmers that Farmer Field School (FFS) type of training and extension of good practices, female 35%
A simple way to show the extent of female participation is to compare % women farmers to total farm
households in the catchment area to the ratio of female to total beneficiaries in the project. Information is
lacking on women farmers to total in the project areas. The table in II B (5) presents a scantly evidence
of female participation. Notwithstanding the expressed commitment to “gender balancing” in the appraisal report, there is little to show in the performance of the project.
g. Unanticipated additional outcomes (positive or negative, not taken into consideration in the
project logical framework):
Following the improvement in road and market access by the project, financial institutions and banks have started servicing some of the intervention communities.
The revenue generating capacity of the project districts through, for example, levies on use of market facilities and warehouses, had enhanced.
EFFICIENCY
h. Timeliness:
The project became effective for disbursement on 2 February 2006. Project implementation ended on 31
December 2013, meaning that it took 7 years and 10 months (~8 years) from when it became
disbursement effective to when implementation ended. The ratio of planned and actual implementation
time is 0.63.
PCR rates unsatisfactory (2) and PCREN concurs.
i. Resource use efficiency:
As per the PCR, the median physical implementation of RLF outputs financed by all financiers was
100%, and the commitment rate was 83.90%. Hence, the ratio of the median percentage physical
implementation of the project outputs and commitment rate is 1.19. As per the PCR guidance, this ratio
translates to highly satisfactory (4) rating for resource use efficiency.
The PCR concludes that the project delivered all or more outputs than expected within the available
budget. But recall the issues raised in section 3 (C) above about the reporting on output performance:
fragmentary, incomplete (not sure the key outputs were identified and reported), unbalanced across the
project’s components, and output execution rates were not shown for some of the reported outputs. The PCREN downgrades the rating to 3, satisfactory.
j. Cost-benefit analysis:
Unable to Rate: The PCREN seeks more information on key parameters and assumptions built in the
computations of the economic rates of return at appraisal and completion. The summary discussion in
the appraisal report as well as the PCR is not sufficient for the PCREN to validate the ratio of ERR at
completion and anticipated ERR at appraisal.
k. Implementation progress:
The PCR considered two IP rating criteria: time overran, weakness in the project M&E, and unmet
counterpart funding. The PCREN considered additional IP criteria and the evidence shows the
following:
Compliance with project covenants
Compliance with general and specific conditions set in the legal documents for loan effectiveness: Loan and grant conditions were fully complied.
Compliance with environmental and social safeguards: Category 2 project --low environmental risk. ESMP was developed and implemented.
Audit compliance: Among the factors that triggered suspension of the project in mid-2009 was the failure to submit the 2008 audit report. The problem of late submission continued in 2009 and
2010.
Project systems and procedures
Procurement: The 2014 PCR mission reports procurement processes complied with the Bank’s procurement rules and procedures. The performance of suppliers, contractors and consultants was
satisfactory, on average. But there were delays in execution within contract duration.
M&E performance: Setting-up of a project monitoring and evaluation commenced during the time of the MTR (2010). Baseline survey was not carried out. The overall performance of the
project’s M&E was insufficient in terms of informing management on the effectiveness of the project achieving its planned results.
Financial management: The 2014 PCR mission concluded “the FM system up to PCR operates in a manner that assures timely financial reporting in line with the Bank’s fiduciary requirements for the project”.
Project execution and financing
Project management: The performance was weakened particularly in the early years of the project implementation because of problems with the AfDB/IFAD Joint Project Implementation
arrangement and its disbanding later without adequate staffing of key positions for the ASREP
Project Coordinating Unit (PCU). The project was fully staffed in 2011, two years before closing.
Disbursement: At appraisal, the ADF loan was to finance UA 10 million representing 71 % and ADF grant to finance UA 2 million representing 14% of the total project cost. GOSL
contribution was to amount to UA 1.6 million equivalent to11 % of the project cost. The
contribution of the beneficiaries at appraisal was estimated at UA 0.5 million or 4 % of the total
project cost.
The Bank’s disbursement rate at the time of mid-term review (2010) was only 17%. GoSL contribution was 11.5% and contributions from the beneficiaries was 3.5 %. Disbursement was
suspended in 2009 and continued until 2010.
By the closure of the project, disbursement was 95.99% for ADF loan and 75.82% for ADF
grant.
Budget commitment rate: %
Counter part funding: At completion, the GOSL disbursed 26.4% as counterpart funds contribution to the project.
No co-finance funding
Overall assessment: Progress in implementation was slow in the early years averaging 10% progress in
terms of achieving the planned outputs in the first three years. As stated in the MTR 2010 report, limited
capacity of the major Government agencies which were responsible for implementation of major
activities, coupled with weak PCU leadership and the unnecessarily excessive control of day to day
project activities by the Executing Agency were the major causes of inefficiency in project
implementation. The disbanding of the AfDB/IFAD joint PCU and the lack of a dedicated operational
staff at the country office (SLFO) exacerbated the situation.
The project implementation apparently accelerated in the after 2010/11 following the lifting of the
suspension, as evident, for example, from the substantial increase in ADF loan disbursement.
Th PCREN rates the overall project implementation performance unsatisfactory (2), downgrading the
satisfactory rating in the PCR.
SUSTAINABILITY
The marginally satisfactory rating of DO in section 3C suggests that the project is reasonably on track to
achieve its development objective. Whether the benefits would continue after completion is the topic of
assessing sustainability. There are financial, institutional, technical and environmental risks that could
limit the sustenance of continued flow of benefits.
l. Financial sustainability:
The main objective of ASREP was to enable smallholders to start farming through rehabilitating their
farms, supplying them with improved farm inputs, and training them in improved agricultural practices.
The sustenance of the production and income gains that emerge from the rehabilitated farms would
depend on future profitability and ability to self-finance investment in farming out of the improved farm
incomes.
But still to sustain productivity and income growth at farm level, MAFFS and its collaborating partners
need to continue development of technologies suitable to lowland and upland farming systems and
deliver adequate FFS-based extension systems. In addition, these government institutions had to finance
through budgetary allocations the maintenance of the agricultural infrastructure such as roads and
rehabilitated research buildings and living quarters that were mainstreamed into respective parent
institutions. The PCR raises doubts about the capacity of the government to continue financing these
activities. The shortfall in the counterpart funding reinforces these doubts although the reason for it is
not fully known. The threat to sustainability of ASREC benefits due to insufficient public financing was
unmitigated risk at the closure of the project.
ASREP transferred some rehabilitated infrastructure to the communities such as water controlled
structures in the Inland Valley Swamps. It was not clear at the closure of the project about the modalities
for cost recovery or revenue generation for the sustenance of the beneficiary-owned assets.
In agreement with the PCR rating, the PCREN rates financial sustainability unsatisfactory (2) since there
were no explicit exit strategies at project completion especially for community-owned assets. Reliance
on public financing may not be sufficient to ensure continued flow of benefits associated with the project
after completion.
m. Institutional sustainability and strengthening of capacities:
Capacity building was the core component of ASREC in the project districts, i.e. formation of farmer-
based organizations, development of out-grower scheme, establishment of farmer field schools (FFS),
training of farmers and facilitators through FFS, and delivery of new farm knowledge and skills
extension through farmer-led or staff-led community-based extension system. Perf romance in
achievements of these outputs was satisfactory during the project’s life cycle.
The administrative structure in the country was devolving into decentralized administrative framework
at the time of the project implementation, which is positive to align development interventions with the
needs of the farming population. In defence of placing responsibility for feeder roads with local
government, for example, the MTR argues that district administrators with knowledge of local
conditions and contractors can assess costs more accurately, and offer close supervision for completion
of contracts in a timely and satisfactory standard.
The PCREN recognizes the project’s significant contribution to institutional sustainability and capacity strengthening. However, there was uncertainty in sustaining these gains because the threat for their
sustainability, and mitigation strategies were not in place at project exit. The PCREN rates
sustainability of institutions and capacities unsatisfactory (2), in line with the PCR rating.
n. Ownership and sustainability of partnerships:
There are three notable levels of ownership; ownership of private farms by the primary beneficiaries of
the project, community owned infrastructure, and public goods managed by central or local governments
(e.g. roads, MAFFS facilities).
The principal contribution of ASREP was enhancing the economic value of privately owned farmers
through agricultural rehabilitation interventions – provision of improved inputs and extension services, and building rural infrastructure. In addition, rural infrastructure such as roads and rehabilitated research
buildings and extension staff quarters had already been mainstreamed into respective parent institutions
or handed over to the communities. The ASREP has thus positively contributed to the enhancement of
ownership of assets at all the three levels.
The PCREN rating of 3 (satisfactory) in line with the PCR.
o. Environmental and social sustainability:
The project was classified as category 2, implying that the activities had minor adverse environmental
impacts and those foreseeable could be mitigated during project implementation. The project supported
activities that promote environmental protection such tree crop plantations and rehabilitation of water
control and management systems in the lowlands. The negative impacts would be extensive clearing of
land and resultant soil erosion, and improper maintenance of drains clogged with weeds that would
cause siltation and provide breeding ground for disease vectors. The ASREP developed and
implemented Environmental and Social Management Plan (ESMP) to mitigate adverse environmental
consequences. However, the evidence on effectiveness of ESMP is scantly.
The ASREP also had components that promoted social benefits such as improved health (e.g. reduction
in HIV prevalence rate and reduction in malaria related deaths).
In short, the ASREP was a low environmental risk, developed and implemented ESMP, contributed to
improved health (HIV/AIDs and malaria control) but lacked strategies for sustainability. PCREN rates
environmental sustainability satisfactory (3) in line with the PCR.
4. PERFORMANCE OF STAKEHOLDERS
a. Bank performance:
The Bank was responsive to the Borrower (Government of Sierra Leone) request for ADF financing to
rehabilitate the agriculture sector in the wake of the ten-year civil war. The project appraisal was
informed by the findings of the 2003 agricultural sector review and involved consultations with principal
stakeholders including prospective project beneficiaries and development partners. However, the quality
at entry was not satisfactory as evident in prolonged delays in start-up with no significant progress for
three years (2006-2009) followed by project suspension (2009-2010). The project’s implementation accelerated only towards the end of the extended timeframe.
The Bank enforced fiduciary requirements and safe guards. The procurement processes complied with
the Bank’s procurement rules and procedures. Performance was satisfactory on average. Corresponding with the time path of the procurement activities, ADF disbursement was slow in the initial years but
attained more than 80% at project closure. The Bank supported development of the project’s M&E system but was weak in establishing baseline data and tracking the project’s outputs and outcomes.
The Bank facilitated the implementation of the project through supervision missions. The most notable
supervision mission was the mid-term review (MTR). The recommendations in the MTR involved
cancellation of some planned activities and readjustment of targets for some outputs, and called for shift
in emphasis towards the agriculture support component. The frequency of the Bank’s supervision missions increased after 2010 and responded better in addressing emerging problems. The PCREN
cannot confirm the Bank undertook quality supervision including adequate involvement of required
skills mix.
Overall, the performance of the Bank in quality-at-entry and implementation effectiveness was
unsatisfactory (2), a downgrading of the satisfactory rating in the PCR.
b. Borrower performance
The Borrower initiated and participated in the formulation of the project. It signed on to the financing
agreements with the Bank and committed to fulfill conditions including provision of counterpart funds.
However, the quality of entry was not satisfactory as evident in the prolonged start-up of the project’s implementation. Weakness in project management was one of the contributing factors for the poor start
of the project to which excessive control of day to day activities of the project by the executing agency
was one of several factors.
The Borrower ensured project’s compliance with environmental safeguards. Financial and audit reporting was weakened in initial years because of problems with the AfDB/IFAD joint implementation
arrangement, which precipitated suspension of ADF disbursement between July 2009 and September
2010. Audit reports were submitted but late in some years such as 2008, 2009 and 2010. The
performance of the project’s M&E was weak in terms of informing management on effectiveness of the project achieving its planned results. The Borrower was not fully in compliance with meeting its
commitment to counterpart funding. There was no adequate exist strategy for financial sustainability
after the project closure.
Overall, the performance of the Borrower in design, readiness and implementation was unsatisfactory (2)
in line with the PCR.
c. Performance of other stakeholders
Among the key other stakeholders were the service providers (i.e. suppliers, contractors and
consultants). The performance of these providers was satisfactory on average in terms of quality of work
performed, timeliness of execution of contracts, and quality of management and financial soundness.
These were few instances of contract termination due to non-performance or inability to rectify
identified defects.
The PCR rates the performance of other stakeholders satisfactory (3) in line with the PCR.
5. SUMMARY OF OVERALL PROJECT PERFORMANCE
a. Overall assessment:
Overall, the performance of the project was marginally satisfactory (3-). Relevance in objective and
design was satisfactory. Progress towards achieving the project’s development objective was on track but not certain. Proxy indicators of operational efficiency such as timeliness and implementation
effectiveness suggest some scope for improvement. The project’s performance on sustainability was mixed; unsatisfactory in financial, institutional and environmental sustainability, and satisfactory in
strengthening ownership.
b. Design, implementation and utilization of the M&E (appreciation of the evaluator):
As per the APR, it was anticipated that the project would conduct a basic socio-economic survey within
6 months from 1st disbursement date to determine the socio-economic baseline case of the beneficiaries
and define the performance indicators to be followed. These indicators would be used in monitoring,
supervision and mid-term review. A continuous monitoring of the project effects and impacts on the
target groups would therefore be carried out through the constitution of representative samples of the
beneficiaries in the project area. During the mid-term review as well as project completion, two other
surveys would be conducted in the 3rd and last project years to assess the project performance and
impact.
In practice, none of these desirable M&E activities were implemented. The overall performance of the
project’s M&E was insufficient in terms of design completeness, tracking progress on performance along the results chain, and informing management on the effectiveness of the project achieving its
planned results.
The PCR rates the overall M&E performance unsatisfactory (2).
6. EVALUATION OF KEY LESSONS LEARNED AND RECOMMENDATIONS
a. Lessons learned:
Short supply of quality seed and planting material (rice, non-rice annual crops, and tree crops) was considered the key constraints that needed be addressed to make full use of the rehabilitated
farms.
Project management appears to be focused on the physical rehabilitation and equipping of the stations, rather than the objective of making agricultural inputs available to farmers.
Recommendations:
Ensure seed and planting material production activities focus on the goal of making seed available to farmers on a sustainable basis, rather than simply physical rehabilitation of the RRS
and IAR.
Ensure that multiplication of tree crop planting material is undertaken as a private sector activity either through individual farmers or FBOs will contribute to sustainability.
Make sure that ‘handout’ programmes intended to enable farmers to restart farming are linked to long term development.
Work closely with district authorities in the civil works and rehabilitation and rural feeder roads programmes to ensure value for money and timely completion.
Poor management performance needs to be addressed by inculcating results based culture through an incentive system (MTR 2010)
7. COMMENTS ON PCR QUALITY AND TIMELINESS
The PCR was complete in coverage of the dimensions of evaluation (relevance, efficacy in output and outcome,
efficiency, sustainability, Borrower and Bank performance). Access to data/evidence was limited. Quality of
analysis in terms of rigor and objectivity was poor. There were notable inconsistencies between rating and
explanation.
Overall, the performance of the PCR was 2 (unsatisfactory).
8. SUMMARY OF THE EVALUATION
Criteria PCR PCREN Reason for disagreement/ Comments
RELEVANCE
Relevance of project development objective 4 4
Relevance of project design 3 3 The text in the PCR is not demonstrating the design of the project, i.e. rationale for its
choice, its internal logic, the extent the
design was informed, assessment of its
realism, and adjustment in scope over the
whole project life cycle, and relevance ex
post. The rating of 4 (i.e. the design was
solid and remained appropriate without
requiring adjustment throughout the
project’s life cycle) was not demonstrated. As shown in 3C, there were design
deficiencies and adjustments were required.
The PCREN rates design relevance
satisfactory (3).
EFFECTIVENESS
Development objective (DO) 3 3
EFFICIENCY
Timeliness 2 2
Resource use efficiency 4 3
Cost-benefit analysis 3 UTR
Implementation progress (IP) 3 2 The PCREN rates IP effectiveness unsatisfactory because of weak performance
in compliance with project covenants (e.g.
delays in audit reports leading to
disbursement suspension), project system
and procedures (delayed procurements and
weak project M&E in design and
implementation) and project execution and
financing (such as disbursement flows from
the Bank and unmet counterpart funding
from GoSL).
SUSTAINABILITY
Financial sustainability 3 2 Uncertainty about sustainability because of undeveloped strategies
Institutional sustainability and strengthening
of capacities
2 2 Uncertainty about sustainability because of
undeveloped strategies
Ownership and sustainability of partnership 3 3 ASREP enhanced the economic value of private ownership of farming
Environmental and social sustainability 3 2 ASREP was low environmental risk, developed and implemented ESMP,
contributed to improved health (HIV/AIDs
and malaria control), but lacked strategies
including funding for sustainability.
OVERALL PROJECT COMPLETION
RATING
Satisfactory on average
Bank performance: 3 2 Sound design but weak in supervision and
implementation particularly in the period
between 2005 and 2010.
Borrower performance: 2 2
Performance of other shareholders: 3 3
Overall PCR quality: 2 Fair: low-objectivity, consistency, and rigor of analysis in identifying influential factors,
and drawing lessons learned.
9. PRIORITY FOR FUTURE EVALUATIVE WORK: PROJECT FOR
PERFORMANCE EVALUTION REPORT, IMPACT EVALUTION,
COUNTRY/SECTOR REVIEWS OR THEMATIC EVALUATION STUDIES:
- Project is part of a series and suitable for cluster evaluation
- Project is a success story
- High priority for impact evaluation
- Performance evaluation is required to sector/country review
- High priority for thematic or special evaluation studies (Country)
- PPER is required because of incomplete validation rating
Major areas of focus for future evaluation work:
a) Performance evaluation is required for sector/ country review
b) Cluster evaluation (institutional support)
c) Sector evaluation (budgetary support or public finance management reforms)
Follow up action by IDEV: Identify same cluster or sector operations; organize appropriate work or consultation mission to
facilitate a), b) and/or c).
Division Manager clearance Director signing off
Data source for validation:
Task Manager/ Responsible bank staff interviewed/contacted (in person, by telephone or email) Documents/ Database reports
Attachment:
PCR evaluation note validation sheet of performance ratings List of references
Appendix 1
PROJECT COMPLETION REPORT EVALUATION NOTE Validation of PCR performance ratings
PCR rating scale:
Score Description 4 Very Good – Fully achieved with no shortcomings 3 Good – Mostly achieved despite a few shortcomings 2 Fair – Partially achieved. Shortcomings and achievements are roughly balanced 1 Poor – very limited achievement with extensive shortcomings
UTS Unable to score/rate NA Non Applicable
Criteria Sub-criteria
PCR
work
score
IDEV
review Reasons for deviation/comments
RELEVANCE Relevance of the
project development
objective (DO) during
implementation
Relevance of project
design (from approval
to completion)
OVERALL RELEVANCE SCORE
EFFECTIVENES
S* Effectiveness in delivering outcomes
Outcome1
Outcome2
Effectiveness in delivering output
Output1
Output2
Development objective (DO)
Development
objective rating
Beneficiaries
Beneficiary1
Criteria Sub-criteria
PCR
work
score
IDEV
review Reasons for deviation/comments
Beneficiary2
Unanticipated outcomes (positive or negative not considered in the project logical
framework) and their level of impact on the project (high, moderate, low)
Institutional
development
Gender
Environment &
climate change
Poverty reduction
Private sector
development
Regional integration
Other (specify)
EFFECTIVENESS OVERALL SCORE
EFFICIENCY Timeliness (based on
the initial closing
date)
Resource used
efficiency
Cost-benefit analysis
Implementation
progress (from the
IPR)
Other (specify)
OVERALL EFFICIENCY SCORE
SUSTAINABILIT
Y
Financial
sustainability
Institutional
sustainability and
strengthening of
capacities
Ownership and
sustainability of
partnerships
Environmental and
social sustainability
*The rating of the effectiveness component is obtained from the development objective (DO) rating in the latest
IPR of the project (see Guidance Note on the IPR).
The ratings for outputs and outcomes are determined based on the project’s progress towards realizing its targets,
Criteria Sub-criteria
PCR
work
score
IDEV
review Reasons for deviation/comments
and the overall development objective of the project (DO) is obtained by combining the ratings obtained for
outputs and outcomes following the method defined in the IPR Guidance Note. The following method is applied:
Highly satisfactory (4), Satisfactory (3), Unsatisfactory (2) and Highly unsatisfactory (1).
Criteria Sub-criteria
PCR
Work
score
IDEV
review
Reasons for deviation/comments
BANK
PERFORMANCE
Proactive identification and
resolution of problems at
different stage of the project
cycle
Use of previous lessons
learned from previous
operations during design and
implementation
Promotion of stakeholder
participation to strengthen
ownership
Enforcement of safeguard and
fiduciary requirements
Design and implementation of
Monitoring & Evaluation
system
Quality of Bank supervision
(mix of skills in supervisory
teams, etc)
Timeliness of responses to
requests
OVERALL BANK PERFORMANCE SCORE
BORROWER
PERFORMANCE
Quality of preparation and
implementation
Compliance with covenants,
agreements and safeguards
Provision of timely counterpart
funding
Responsiveness to supervision
recommendations
Measures taken to establish
basis for project sustainability
Timeliness of preparing
requests
OVERALL BORROWER PERFORMANCE
SCORE
PERFORMANCE
OF OTHER
STAKEHOLDERS
Timeliness of disbursements
by co-financiers
Functioning of collaborative
agreements
Quality of policy dialogue
with co-financiers (for PBOs
only)
Quality of work by service
providers
Responsiveness to client
demands
OVERALL PERFORMANCE OF OTHER
STAKEHOLDERS
The overall rating is given: Very Good, Good, Fair and Poor.
(i) Very Good (HS): 4 (ii) Good (H): 3 (iii) Fair (US): 2
(iv) Poor (HUS): 1
DESIGN, IMPLEMENTAION AND UTILIZATION OF MONITIRING AND EVALUATION
(M&E)
Criteria Sub-criteria IDEV
Score Comments
M&E DESIGN M&E system is in place, clear,
appropriate and realistic
Monitoring indicators and
monitoring plan were duly
approved
Existence of disaggregated
gender indicator
Baseline data were available or
collected during the design
Other, specify
OVERALL M&E DESIGN SCORE
M&E
IMPLEMENT
A-TION
The M&E function is
adequately equipped and staffed
OVERALL M&E IMPLEMENTATION
SCORE
M&E
UTILIZATION
The borrower used the tracking
information for decision
OVERALL M&E UTILIZATION SCORE
OVERALL M&E PERFORMANCE SCORE
PCR QUALITY EVALUATION
Criteria PCR-EVN
(1-4) Comments
QUALITY OF PCR
1. Extent of quality and completeness of the PCR
evidence and analysis to substantiate the ratings of
the various sections
2 The PCR description of the project was not
logically structure, concise and clear. It is rated
fair (2) in quality of evidence and rigor of analysis
2. Extent of objectivity of PCR assessment score 2 Limited objectivity.
3. Extent of internal consistency of PCR assessment
ratings; inaccuracies; inconsistencies; (in various
sections; between text and ratings; consistency of
overall rating with individual component ratings)
2 Often notable divergence between ratings and text.
For example, the discussion about the design
relevance in section 3(b) was not about the
rationale for the project’s choice, its internal logic, test for realism and adjustments during the lifetime
of the project.
4. Extent of identification and assessment of key
factors (internal and exogenous) and unintended
effects (positive or negative) affecting design and
implementation
1 No cause-effect frame of analysis to identify
influential factors affecting performance or result
5. Adequacy of treatment of safeguards, fiduciary
issues, and alignment and harmonization
3 Good
6. Extent of soundness of data generating and
analysis process (including rates of returns) in
support of PCR assessment
1 Poor because of uncertainty in source and
methodology of data generation, and weak data
organization and summary.
7. Overall adequacy of the accessible evidence (from
PCR including annexure and other data provided)
2 Limited (fair) access to evidence to enable the
PCREN to validate the PCR ratings. No technical
note annexed.
8. Extent to which lessons learned (and
recommendations) are clear and based on the PCR
assessment (evidence & analysis)
1 Too many lessons learned. Hard to map most of
the lessons to the project’s experience. Quality of lessons learned reflects point 4 (rigor of analysis)
and point 6 (quality of data generated).
9. Extent of overall clarity and completeness of the
PCR
2
Other (specify)
PCR QUALITY SCORE 2 Fair: low-objectivity, consistency, and rigor of
analysis in identifying influential factors and
drawing lessons learned.
PCR compliance with guidelines (PCR/OM; IDEV)
1. PCR Timeliness (On time = 4; Late= 1) 4
2. Extent of participation of borrower, Co-financiers
& field offices in PCR preparation
3
3. Other aspect(s) (specify)
PCR COMPLIANCE SCORE 3.5
*** rated as Very Good (4), or Good (3), or Fair (2), or Poor (1)
References
African Development Bank Group. 2014. Republic of Sierra Leone: Agriculture Sector Rehabilitation
Project (ASREP). Project Completion Report.
African Development Bank Group. 2014. Republic of Sierra Leone: Agriculture Sector Rehabilitation
Project (ASREP). Project Completion Report Preparation Mission, 02-11 June. Aide Memoire.
African Development Bank Group. 2013. Republic of Sierra Leone: Agriculture Sector Rehabilitation
Project (ASREP). Supervision Mission, 02-12 April. Aide Memoire.
African Development Bank Group. 2012. Republic of Sierra Leone: Agriculture Sector Rehabilitation
Project (ASREP). Supervision Mission, 18-26 January. Aide Memoire.
African Development Bank Group. 2012. Staff guidance on project completion reporting and rating.
Quality Assurance Department (ORQR).
African Development Bank Group. 2011. Republic of Sierra Leone: Agriculture Sector Rehabilitation
Project (ASREP) and NERICA Rice Dissemination Project. Supervision Mission, 18-30 July. Aide
Memoire.
African Development Bank Group. 2011. Staff guidance on implementation progress and results (IPR) for
public sector operations. Quality Assurance Department (ORQR).
African Development Bank Group. 2010. Republic of Sierra Leone: Agriculture Sector Rehabilitation
Project (ASREP). Mid-term Review.
African Development Bank Group. 2009. Republic of Sierra Leone: Agriculture Sector Rehabilitation
Project (ASREP). Mid-term Review Mission, 27 August-12 September 2009. Aide Memoire.
African Development Bank Group. 2004. Republic of Sierra Leone: Agriculture Sector Rehabilitation
Project (ASREP). Appraisal Report.