Upload
others
View
0
Download
0
Embed Size (px)
Citation preview
1. BASIC INFORMATION
a. Basic project data
Project title: Gender Responsive Skills and Community Development Project (GRSCDP) Project code: P-GH- IDO 003 Instrument number(s): Loan and Grant Project type: Investment Sector: Social Country: Ghana Environmental categorization (1-3) : 3
Processing Milestones Key Events Disbursement and Closing date
Date approved: 19/12/07 Cancelled amount: Original disbursement deadline: 31/12/12
Date signed: 14/05/08 Supplementary financing: Original closing date: 31/12/12 Date of entry into force : 20/01/09
Restructuring: Revised disbursement deadline: 30/09/15
Date effective for 1st disbursement:20/01/09
Extensions (specify dates): 31/12/13 and 30/09/15
Revised closing date: 30/09/15
Date of actual 1st : 13/10/09
b. Financing sources NOTE: The PCR reports financing in a very confusing manner mixing loan and expenditure in the funding part of the table. The correct figures seem to be as follows
b. Financing sources
Financing source/ instrument (MUA)
Approved amount
(MUA) : Disbursed amount
(MUA) : Percentage disbursed
(%):
Loan: 5.95 2.09 35.13
Grant: 2.36 1.5 63.56 Government: 1.09 0.5 45.87
Other (ex. Co-financiers):
TOTAL : 9.4 4.09 43.5
Co-financiers and other external partners:
Execution and implementation agencies: Ministry of Women and Children’s Affairs later renamed Ministry of Gender ,Children and Social Protection (MoGCSP) c. Responsible Bank staff
Position At approval At completion
Regional Director J. Litse Abdellatif Bernoussi
Sector Director Tom Hurley Sunita Pitamber
Sector Manager Sibry Tapsoba Justin Murara
Task Manager Yeshi Dejene Mulle Chikoko
Alternate Task Manager Efua Amissah-Arthur PCR Team Leader Ms. Mulle Chikoko
PCR Team Members Ms Efua Amissah-Arthur; Mr
PCR EVALUATION NOTE FOR PUBLIC SECTOR OPERATIONS
Philip Doghle; Mr Daniel Osei-Boakye; Procurement Officer; Mrs Joana Opare.
d. Report data
PCR Date : SEPTEMBER 2016 PCR Mission Date: From: 5/09/16 To: 16/09/16 PCR-EN Date: 5 July 2018
Evaluator/consultant : Stephen CUNNINGHAM Peer Reviewer: Judith OFORI
2. PROJECT DESCRIPTION Summary from Appraisal Report including addendum/corrigendum or loan agreement, and taking into account any modification
that occurred during the implementation phase.
a. Rationale and expected impacts: Provide a brief and precise description on the project/programme rationale (concerns/questions raised), expected impacts and the
intended beneficiaries (directly or indirectly impacted by the project/programme). Highlight any change that occurred during the
execution phase.
The Project aims to reduce gender-based inequalities as an effective means to reduce poverty within the
context of GPRS II and is based on the need to increase women’s access to and control over productive
assets and improved access to social service and protection. The Project aligns with the
third pillar of GPRS II: ‘vigorous human resource development’ which seeks to promote skills and
entrepreneurial development.
The Project is also aligned with the Bank’s Country Strategic Paper (2005-2009) under the second pillar
“support pro-poor and pro-gender-equity policies”. The Project will contribute to the MDGs related
to reducing extreme poverty and hunger and gender equality.
The beneficiaries are intended to be the Ghanaian population as a whole but women in particular.
The main impact will be to contribute to gender equitable socio-economic development by reducing the
rate of women living in extreme poverty by 7.5% by end of project implementation and by 10 % by
2015 as measured by National statistics and GPRS
Related to this will be an improvement by 2012 in national capacities for enhanced gender
mainstreaming as measured by:
Percentage of MDAs budget allocated for gender related activities increased by 20% from the
current level
Number of gender sensitive projects increased at district level by 30% from current level
All Bank-financed projects are gender-sensitive projects.
And an enhancement of women’s marketable skills through greater access to quality training & an
improvement in women’s productivity and self-sufficiency through greater access to services in micro
and small entrepreneurship, measured by:
80% of skills trainees gainfully employed or self-employed
Percentage of women having access to Micro Finance Institutions (MFIs) increased by 25%
from current rate
Volume of average loans taken by women micro and small entrepreneurs increased by 25%
from current rate
b. Objectives/Expected Outcomes: Provide a clear and concise description of the project objectives, expected outcomes, and intended beneficiaries. In so doing,
highlight any revision/amendment.
The project goal is to promote gender equitable socio-economic development. The objectives are to: (i)
improve national capacities for enhanced gender mainstreaming and (ii) improve access to quality skills
training for gainful employment and entrepreneurial development of women.
The expected OUTCOMES are:
Longer-Term Outcomes: The living conditions of disadvantaged groups, women in particular have
improved
Medium-Term Outcomes:
1.1 Human resources base and professional competency for increasing gender mainstreaming into
development management strengthened
2.1 Women’s marketable skills enhanced through greater access to quality training
2.2 Women’s productivity and self-sufficiency improved through greater access to services in micro and
small entrepreneurship
c. Outputs and intended beneficiaries: Provide a clear and concise description the expected outputs and intended beneficiaries. In so doing, highlight any
revision/amendment.
The expected outputs from the Logical Framework are:
1.1 Capacity built within MOWAC, selected line ministries, agencies, and district level administrative
structures through scholarships, training programs and workshops, revision of gender training
curriculum, gender analysis of key planning frameworks, and improvement of system of gender
disaggregated data collection and analysis
NB There is no number 2 in the results matrix; the numbering here is as per the matrix
3.1 Curriculum improved and more responsive to market needs
3.2 Training centers rehabilitated and equipped
3.3 Teachers trained
3.4 Enrolment increased and girls encouraged to study non-traditionally female trades
4.1 Access to internet and ICT services increased
4.2 Capacity for service delivery to women in micro and small entrepreneurship improved
5.1 Efficient management of project activities
d. Principal activities/Components: Provide a clear and concise description of the principal activities/components. In so doing, highlight any revision/amendment.
Annex III of the Appraisal Report includes a Project Implementation Schedule that lists the main
activities by Component as follows:
Component 1 Institutional Strengthening of Enhancing Gender Mainstreaming
1. Assessment of Training Needs and Existing Curricula and Revision of Curricula and field testing
and training of trainers
2. Training of professionals
3. Training of Decision Makers
4. Staff selection for long term and training
5. Study on Gender and Culture & Validation Workshop
Component II Support To Skills Training and Entrepreneurial Development
1. Study on Vocational Skills conducted, field tested
2. Teachers in new curricula trained
3. Construction and Upgrading of Centers
4. A committee for scholarship selection and award established
5. Scholarship to girls awarded
6. Study on training needs of women entrepreneurs
7. Study on women’s access to micro-credit and the capacity of MFI conducted and validated
8. Training of staff of MFI, BDS, GHAMFIN, NBSSI, DCD extension workers
9. Data base of service providers operational
Component III: Project management and coordination
1. Midterm-Review
2. PCR
3. PROJECT PERFORMANCE ASSESSMENT
IMPORTANT NOTE: It was not possible to undertake an independent evaluation of this
project because the only documents available to the evaluation were the project appraisal and
the PCR. The following sections simply review the PCR. The PCREN has not been able to
provide its own scores, only to suggest corrections to the PCR scores in a couple of places.
RELEVANCE
a. Relevance of the project development objective: Evaluation of the relevance ex-ante and ex-post (including during the implementation phase). The relevance of the project
objective (during the evaluation ex-ante and the post-evaluation) in terms of alignment with country’s development priorities
and strategies, the beneficiary needs (including any changes that may have occurred during the implementation), applicable
Bank sector strategies, the Bank country/ regional strategy, and general strategic priorities of the Bank. This criterion equally
assesses the extent to which the project’s development objective was clearly stated and focused on outcomes and the realism of
the intended outcomes in the project setting.
The discussion presented by the PCR is consistent with a rating of 4.
b. Relevance of project design (from approval to completion): The evaluator should provide an assessment of the relevance of the project design regardless of the one provided in the PCR.
The evaluator will also comment on the PCR conclusion for this section, and will provide an evaluation of the relevance of the
project design. The latter assesses the soundness and the timing of eventual adjustments, or technical solutions to ensure the
achievement of the intended results (outcomes and outputs), the adequacy of the risk assessment, environmental and social
protection measures, as well as the implementation arrangements. For Programme Based Operations (PBO), an assessment will
be made on the relevance of the prior actions, the policy dialogue and the extent to which the operation could have been more
pro-poor in its design.
The points made about the project design are consistent with a rating of 3, although the number of
potential beneficiaries (section 4.2 of the Appraisal report) seems to be very optimistic.
EFFECTIVENESS
c. Effectiveness in delivering outputs: Evaluation of the extent to which the project achieved its stated results (obtained from the logical framework) based on the last
Implementation Progress and Results Report (IPR) and by considering accurate reporting of direct or indirect evidence on
intended and unanticipated outputs. In the absence of sufficient data (as direct evidence), indirect evidence (such as project
outcomes and other pertinent processes/elements of the causal chain) should be used particularly in the evaluation of the extent
to which the project is expected to achieve its stated results/ objectives. The absence of sufficient data to assess the effectiveness
should be indicated (and clearly detailed in the PCR quality evaluation section). The PCR score should equally be indicated in
this section.
The report on the outputs is clearly evidence-based and fully justifies the rating of 2.
d. Effectiveness in delivering outcomes: Evaluation of the extent to which the project achieved its intended set of outcomes (including for Program Based Operations
(PBOs) where complementary measures are necessary for their implementation, namely public awareness, policy dialogue and
institutional arrangements for instance). The evaluator should make an assessment based on the results of the last project
Implementation Progress and Results (IPR). The evaluator shall indicate the degree to which project outcomes (intended and
unanticipated) as well as reasons for any eventual gap were discussed in the PCR.
The assessment presented in the PCR is evidence-based insofar as evidence is available. In many
cases, such evidence is lacking suggesting that the M&E system was ineffective. The rating of 2 is
justified by the discussion in the PCR.
e. Project development outcome:
The ratings derived for outcomes and output are combined to assess the progress the project has made towards realizing its
development objectives, based on the rating methodology recommended in the Staff Guidance Note on project completion
reporting and rating (see IPR Guidance Note for further instruction on development objective rating).
The PCR has a rating of 3 for the PDO. However, given that both outcomes and outputs were rated to
2, the PDO rating should also be 2.
f. Beneficiaries: Using evidence, the evaluator should provide an assessment of the relevance of the total number of beneficiaries by categories
and disaggregated by sex.
The beneficiaries discussed in the PCR represents only a small subset of those mentioned in the
Appraisal Report (section 4.2) although as noted above the estimate of potential beneficiaries in that
Report seems to be very optimistic.
g. Unanticipated additional outcomes (positive or negative, not taken into consideration in the project
logical framework): This includes gender, climate change, as well as social and socio-economic- related issues. Provide an assessment of the extent to
which intended or unanticipated additional and important outcomes have been taken into consideration by the PCR. The
assessment should also look at the manner the PCR accounted for these outcomes.
The PCR identifies only that some numbers trained were higher than expected
EFFICIENCY
h. Timeliness: The timeliness of project implementation is based on a comparison between the planned and actual period of implementation
from the date of effectiveness for first disbursement. For Programme Based Operations (PBOs), the timely release of the
tranche(s) are assessed through this same criterion.
The PCR says here that the end date is Dec 2013 but there was an extension in the disbursement
deadline to 30 Sept 2015 that presumably also affected the closing date. If so, instead of 4 years the
project actually ran for almost 7 years rather than the 5 claimed in this section by the PCR. There is no
rating in this section of the PCR but the summary table includes a 2 rating for timeliness which seems
to be the maximum that could be given.
i. Resource use efficiency: Provide and assessment of physical implementation (based on outputs delivered) against resources used (based on cumulative
commitments) at completion for all contributors to the project (the Bank, Government, and others). This criterion would
normally not apply to PBOs, as there is often no direct link between the outputs and the amount of contribution (in which case
the rater would indicate N/A).
A substantial amount of the special fund was not justified and disbursement rate was extremely low.
The project could only use 43% of financial resources in 7 years of implementing a 4 year program.
With 56.5% of undisbursed funds, special account unjustified at PCR, Fraud investigations, the
Review recommends Highly unsatisfactory (1) for Resource use efficiency.
j. Cost-benefit analysis: Provide an assessment of the timeliness of the development outputs, and the extent to which costs of the costs have been effective
and have been provided in the most efficient manner. The PCR rating should be discussed. The evaluator should verify whether
the benefits of the project (achieved or expected) exceed its actual costs. To achieve this, evidences will mainly be based on a
comparison between Economic Rates of Return (ERR) calculated at appraisal, the mid-term review and completion. When
commenting PCR ratings, the degree of utilization of valid sources for evidence justifying the rating assigned should be taken
into consideration. The evaluator should ensure of the validity of assumptions and that the same model was used for the
calculation of others ERRs. For PBOs for which this calculation model does not apply, an assessment could be done with regards
to the contribution of policy reforms to economic growth. In the absence of sufficient evidence, an appropriate rating should be
assigned.
N/A
k. Implementation progress: The assessment of the Implementation Progress (IP) on the PCR is derived from the updated IPR and takes into account the all
applicable IP criteria assessed under the three categories : i) Compliance with covenants (project covenants, environmental and
social safeguards and audit compliance), ii) project systems and procedures (procurement, financial management and
monitoring and evaluation), and iii) project execution and financing (disbursement, budget commitments, counterpart funding
and co-financing).
Progress is reported to be good on component 1 only. The PCR raises some very serious issues
concerning Project Management and the 2 rating seems appropriate
SUSTAINABILITY
l. Financial sustainability: Provide an assessment of the extent to which funding mechanisms and modalities (eg. Tariffs, user fees, maintenance fees,
budgetary allocations, other stakeholder contributions, aid flows, etc.) have been put in place to ensure the continued flow of
benefits after completion, with particular emphasis on financial sustainability. For PBOs, the assessment should focus on
financial sustainability of reforms, as well as the Bank’s policy dialogue to promote financial sustainability of the reforms.
This aspect seems to be a disaster – not only are there no apparent plans for post-project financial
sustainability, it seems that there is not even a strategy to deal with scholarships that overhang the
project closure date. This is completely unacceptable and the Evaluation strongly agrees with the 1
rating given by the PCR (and hopes that a successful solution was later found).
m. Institutional sustainability and strengthening of capacities: Provide an assessment of the extent to which the project has contributed to the strengthening of institutional capacities –
including for instance through the use of country systems – that will continue to facilitate the continued flow of benefits
associated with the project. An appreciation should be made with regards to whether or not improved governance practices or
improved skills, procedures, incentives, structures, or institutional mechanisms came into effect as a result of the operation. For
PBOs, this should include an assessment on the contributions made to building the capacity to lead and manage the policy
reform process; the extent to which the political economy of decision making was conducive to reform; the Government’s
commitment to reform; and how the design reinforced national ownership.
Given what is said in the PCR the rating of 2 may seem a little on the harsh side. However, it is true
that there is no evidence to demonstrate the impact of the activities and outputs so the 2 rating does
seem to be appropriate.
n. Ownership and sustainability of partnerships: Provide an assessment of whether the project has effectively involved relevant stakeholders, promoted a sense of ownership
amongst the beneficiaries (both men and women) and put in place effective partnerships with relevant stakeholders (eg. local
authorities, civil society organizations, private sector, donors) as required for the continued maintenance of the project outputs.
For PBOs, the assessment should measure the extent to which the Government’s capacity to conduct consultations during policy
dialogue and the extent to which the Bank supported the Government in deepening the consultation processes.
From the PCR it appears that ownership began to be created but was lost with changes in personnel.
The 2 rating seems justified
o. Environmental and social sustainability: Provide an assessment of the objectivity of the PCR rating on the project’s implementation of environmental and social
mitigation/enhancement measures with regard to the Environmental and Social Management Plan (ESMP), the capacity of
country institutions and systems, as well as the availability of funding to ensure the environmental and social sustainability of
the operation. This criterion would normally only apply to Environmental Category I and II projects.
The PCR reports a 4 rating for this category. However, the project was ranked a Category III and this
criterion would not normally apply. Even if it does, this Review recommends a satisfactory (3) rating.
The Environmental aspects with ESMP may have been well covered, but social sustainability is not
guaranteed. The participation, ownership, absence of sustainability plans to ensure continuous social
engagement is not guaranteed.
4. PERFORMANCE OF STAKEHOLDERS
a. Bank performance: (Preparation/approval, ensure of Quality at Entry (QAE) : quality of the supervision, completion) : Provide observations on the
objectivity of the PCR ratings and feedback provided by the Borrower, and if necessary, re-assess the Bank’s performance
throughout the project cycle (design, implementation, completion) by focusing on evidence from the PCR in relation to 7 criteria
defined in the PCR Guidance Note.
The PCR is critical of Bank performance because (a) a fraud investigation took too long to complete
and (b) there were frequent changes in task manager. Both factors negatively affected project
performance and the 2 rating for this section seems justified.
b. Borrower performance: Provide observations on the objectivity of the PCR ratings, and if necessary, re-assess the Borrower’s performance throughout
the project cycle (design, implementation, completion) by focusing on evidence from the PCR in relation to questions defined in
the PCR Guidance Note.
Project management seems to have been very poor with political interference regarding the project
manager. The 1 rating seems justified. The list of other failings include, Lack of counterpart funding,
M&E system developed but not fully utilised. Outstanding audit reports among others.
c. Performance of other stakeholders: Provide observations on the objectivity of the PCR ratings, and if necessary, re-assess the other shareholders’ performance
throughout the project cycle (design, implementation, completion) by focusing on evidence from the PCR in relation to relevant
questions specific to each stakeholder (co-financiers, NGO, contractors and service providers).
The explanations put forward by the PCR are consistent with a 2 rating
5. SUMMARY OF OVERALL PROJECT PERFORMANCE
a. Overall assessment: Provide a summary of the project/programme’s overall performance based on the PCR 4 key components (Relevance,
Effectiveness, Efficiency and Sustainability). Any difference with the PCR and the reasons that have resulted in them should
be mentioned. For cases with insufficient evidence (from the PCR and other documents) available, the evaluator should
assign a partly satisfactory rating (to be revised) until a PPAR is complete.
From the PCR it seems clear that the project’s performance was poor. The PCR has an average score
of 2.5 but this should be lower because (a) the PDO should be 2 not 3 and (b) environmental
sustainability should be removed. Making these two changes would reduce the average score to 2.2
which seems to be a clearer reflection of the project quality. The Review recommends an
unsatisfactory (2) rating
b. Design, implementation and utilization of the M&E (appreciation of the evaluator): Provide an assessment of planned and actual cost of the design, implementation and utilization of the M&E system. Design : To
which extent the project M&E system was explicit, adequate and realistic to generate and analyse relevant data ;
Implementation : To which extent relevant data was collected – Elements of M&E implementation and effectiveness in the
PCR ; Utilization : degree of utilization of data generated for decision-making and resource allocation – elements of M&E
utilization in the PCR.
No information on which to judge
6. EVALUATION OF KEY LESSONS LEARNED AND RECOMMENDATIONS
a. Lessons learned: Provide a brief description of any agreement/disagreement with all or part of the lessons learned from the PCR after analysis of the
project performance with regards to each of the key components of the evaluation (Relevance, Effectiveness, Efficiency, and
Sustainability). List the PCR main new and/or reformulated pertinent (and generic) lessons learned for each of these components here. It
is recommended that no more than five lessons learned are discussed. Key questions and targeted audience must also be specified for
each lesson learned.
The PCR has the following lessons. The Evaluation’s opinion of each lesson is given.
1. Project/political leadership NOT VALIDATED – MOVED TO RECOMMENDATIONS
2. Effective Stakeholder collaboration VAIDATED
3. Transition, especially from one political/technical leadership to another NOT VALIDATED –
REPEAT OF 1
4. Capacity enhancement of Project teams in Bank’s procurement and administrative processes –
VALIDATED
5. Sustainability plan/exiting strategy NOT VALIDATED – MOVED TO RECOMMENDATIONS
6. Handing overs -NOT VALIDATED – TOO PROJECT SPECIFIC
7. Commitment to long-term scholarships - NOT VALIDATED – COMBINED WITH 5 AND
MOVED TO RECOMMENDATIONS
8. Ownership and Capacity building – VALIDATED
9. Communication with stakeholders – NOT VALIDATED, TOO CLOSE TO 2
10. Quality at Entry – NOT VALIDATED, IT IS A RECOMMENDATION BUT IS TOO PROJECT
SPECIFIC IN THIS CASE AND SHOULD ALREADY BE PART OF GOOD PRACTICE
OTHERWISE
11. Investigations into fraud allegations – NOT VALIDATED, REFORMULATED AS A
RECOMMENDATION
12. Government Counterpart Funds – NOT VALIDATED, REFORMULATED AS A
RECOMMENDATION
b. Recommendations: Provide a brief description of any agreement/ disagreement with all or part of the recommendations from the PCR. List the PCR main
new and/or reformulated recommendations (requiring more actions by the Borrower and/or the Bank) here.
1. Scholarships – NOT VALIDATED, TOO PROJECT SPECIFIC
2. Women access to MFIs - NOT VALIDATED, TOO PROJECT SPECIFIC
3. Follow-up - NOT VALIDATED, TOO PROJECT SPECIFIC
The Evaluation suggests instead the following 3 generic recommendations that emerge from the PCR:
1. The role of political leaders, Executing Ministry’s administration, Steering committee and
Political actors in Project Management should be clearly defined in order to ensure effective
Project management. It is necessary to orient new leadership on the Project including the
policies and procedures.
2. Where a project commits to activities such as scholarships that go beyond its closure date, a
clear exit strategy must be put in place at appraisal or developed during project implementation
so that after project closure, beneficiaries are not adversely affected.
3. Where a Project is subject to serious fraud allegations, the Bank’s investigation service
(IACD) should attempt to ensure that project activities can continue with minimum disruption
whilst investigations are conducted.
7. COMMENTS ON PCR QUALITY AND TIMELINESS The overall PCR rating is based on all or part of the criteria presented in the annexe and other: The quality of the PCR is rated
as highly satisfactory (4), satisfactory (3), unsatisfactory (2), and highly unsatisfactory (1). Le timeliness of the PCR is rated as
on time (4) or late (1). The participation of the Borrower, co-financier, and the bank’s external office(s) are rated as follows:
Very Good (4), Good (3), Fair (2), Poor (1).
The PCR team seems to have been objective and to have based its scores on the evidence but the
Evaluation has no information on how complete this evidence is. The score for the Project
Development Objective is incorrectly calculated and a score for environmental sustainability is
included when not required given project category. Correcting for these two things lowers the average
score for the Project from 2.5 to 2.2 which seems to be a more accurate representation of the Project’s
performance, given what is presented in the PCR.
No accessible evidence was provided either from the PCR including annexure or other data making it
impossible to undertake an independent evaluation. The PCR is clearly written and presented but in
the absence of any other evidence one cannot tell how complete it is.
The lessons and recommendations tend to be mixed up with many of the former in fact belonging to
the latter category. They are often far too project-specific to be very useful for informing future
projects
Given the above, the PCR is rated unsatisfactory (2.33).
It is not entirely clear but the PCR seems to have been completed one year after the end of the project.
8. SUMMARY OF THE EVALUATION This is a summary of both the PCR and IDEV ratings with justification for deviations/comments. Appropriate section of the
PCR Evaluation should be indicated in the last column in order to avoid detailed comments. The evaluator must provide a
reasonable explanation for each criterion the PCR rating is not validated by IDEV. Consequently, the overall rating of the
project could be “equally satisfactory”.
Criteria PCR PCREN Reason for disagreement/ Comments
RELEVANCE
Relevance of project development objective 4 4
Relevance of project design 3 3
EFFECTIVENESS
Development objective (DO) 3 2 The PCR calculation is incorrect given the
ratings for outcomes and outputs
EFFICIENCY
Timeliness 2 2
Resource use efficiency 2 1 The project could only use 43% of financial
resources in 7 years of implementing a 4
year program.
With 56.5% of undisbursed funds, special
account unjustified at PCR, Fraud
investigations
Cost-benefit analysis na na
Implementation progress (IP) 2 2
SUSTAINABILITY
Financial sustainability 1 1
Institutional sustainability and strengthening of
capacities
2 2
Environmental and social sustainability 4 3 Category should probably not be included.
Social factors not covered
OVERALL PROJECT COMPLETION
RATING
2
Bank performance: 2 2
Borrower performance: 1 1
Performance of other shareholders: 2 2
Overall PCR quality:
9. PRIORITY FOR FUTURE EVALUATIVE WORK: PROJECT FOR
PERFORMANCE EVALUTION REPORT, IMPACT EVALUTION,
COUNTRY/SECTOR REVIEWS OR THEMATIC EVALUATION STUDIES:
- Project is part of a series and suitable for cluster evaluation
- Project is a success story
- High priority for impact evaluation
- Performance evaluation is required to sector/country review
- High priority for thematic or special evaluation studies (Country)
- PPER is required because of incomplete validation rating
Major areas of focus for future evaluation work:
a) Performance evaluation is required for sector/ country review
b) Cluster evaluation (institutional support)
c) Sector evaluation (budgetary support or public finance management reforms)
Follow up action by IDEV: Identify same cluster or sector operations; organize appropriate work or consultation mission to
facilitate a), b) and/or c).
Division Manager clearance Director signing off
Data source for validation:
Task Manager/ Responsible bank staff interviewed/contacted (in person, by telephone or
email)
Documents/ Database reports
Attachment:
PCR evaluation note validation sheet of performance ratings
List of references
Appendice 1
PROJECT COMPLETION REPORT EVALUATION NOTE Validation of PCR performance ratings
PCR rating scale:
Score Description 4 Very Good – Fully achieved with no shortcomings 3 Good – Mostly achieved despite a few shortcomings 2 Fair – Partially achieved. Shortcomings and achievements are roughly balanced 1 Poor – very limited achievement with extensive shortcomings
UTS Unable to score/rate NA Non Applicable
Criteria Sub-criteria PCR
work
score
IDEV
review Reasons for deviation/comments
RELEVANCE Relevance of the project
development objective
(DO) during
implementation
4 4
Relevance of project
design (from approval to
completion)
3 3
OVERALL RELEVANCE SCORE
EFFECTIVENESS* Effectiveness in delivering outcomes
Outcome1
Outcome2
Effectiveness in delivering output
Output1
Output2
Development objective (DO)
Development objective
rating
3 2 The PCR calculation is incorrect given the ratings for
outcomes and outputs
Beneficiaries
Beneficiary1
Criteria Sub-criteria PCR
work
score
IDEV
review Reasons for deviation/comments
Beneficiary2
Unanticipated outcomes (positive or negative not considered in the project logical
framework) and their level of impact on the project (high, moderate, low)
Institutional
development
Gender
Environment & climate
change
Poverty reduction
Private sector
development
Regional integration
Other (specify)
EFFECTIVENESS OVERALL SCORE
EFFICIENCY Timeliness (based on the
initial closing date) 2 2
Resource used
efficiency
2 1 The project could only use 43% of financial
resources in 7 years of implementing a 4 year
program.
With 56.5% of undisbursed funds, special account
unjustified at PCR, Fraud investigations
Cost-benefit analysis na Na
Implementation progress
(from the IPR) 2 2
Other (specify)
OVERALL EFFICIENCY SCORE
SUSTAINABILITY Financial sustainability
1 1
Institutional
sustainability and
strengthening of
capacities
2 2
Ownership and
sustainability of
partnerships
2 2
Environmental and
social sustainability 4 3 Category should probably not be included. Social
factors not covered
*The rating of the effectiveness component is obtained from the development objective (DO) rating in the latest IPR of the
project (see Guidance Note on the IPR).
The ratings for outputs and outcomes are determined based on the project’s progress towards realizing its targets, and the
overall development objective of the project (DO) is obtained by combining the ratings obtained for outputs and outcomes
following the method defined in the IPR Guidance Note. The following method is applied: Highly satisfactory (4),
Criteria Sub-criteria PCR
work
score
IDEV
review Reasons for deviation/comments
Satisfactory (3), Unsatisfactory (2) and Highly unsatisfactory (1).
Criteria Sub-criteria PCR
Work
score
IDEV
review Reasons for deviation/comments
BANK
PERFORMANCE Proactive identification and
resolution of problems at different
stage of the project cycle
Use of previous lessons learned
from previous operations during
design and implementation
Promotion of stakeholder
participation to strengthen
ownership
Enforcement of safeguard and
fiduciary requirements
Design and implementation of
Monitoring & Evaluation system
Quality of Bank supervision (mix
of skills in supervisory teams, etc)
Timeliness of responses to
requests
OVERALL BANK PERFORMANCE SCORE 2 2
BORROWER
PERFORMANCE
Quality of preparation and
implementation
Compliance with covenants,
agreements and safeguards
Provision of timely counterpart
funding
Responsiveness to supervision
recommendations
Measures taken to establish basis
for project sustainability
Timeliness of preparing requests
OVERALL BORROWER PERFORMANCE SCORE 1 1
PERFORMANCE
OF OTHER
STAKEHOLDERS
Timeliness of disbursements by
co-financiers
Functioning of collaborative
agreements
Quality of policy dialogue with
co-financiers (for PBOs only)
Quality of work by service
providers
Responsiveness to client demands
OVERALL PERFORMANCE OF OTHER
STAKEHOLDERS
2 2
The overall rating is given: Very Good, Good, Fair and Poor.
(i) Very Good (HS) : 4
(ii) Good (H) : 3
(iii) Fair (US) : 2
(iv) Poor (HUS): 1
DESIGN, IMPLEMENTAION AND UTILIZATION OF MONITIRING AND
EVALUATION (M&E)
Criteria Sub-criteria IDEV
Score Comments
M&E DESIGN M&E system is in place, clear,
appropriate and realistic UTS
Monitoring indicators and
monitoring plan were duly
approved
UTS
Existence of disaggregated gender
indicator UTS
Baseline data were available or
collected during the design UTS
Other, specify
OVERALL M&E DESIGN SCORE
M&E
IMPLEMENTA-
TION
The M&E function is adequately
equipped and staffed UTS
OVERALL M&E IMPLEMENTATION SCORE
M&E
UTILIZATION
The borrower used the tracking
information for decision UTS
OVERALL M&E UTILIZATION SCORE
OVERALL M&E PERFORMANCE SCORE UTS
PCR QUALITY EVALUATION
Criteria PCR-EVN
(1-4) Comments
QUALITY OF PCR
1. Extent of quality and completeness of the PCR
evidence and analysis to substantiate the ratings of
the various sections
2 The PCR team seems to have based its scores on
the evidence but the Evaluation has no information
on how complete this is
2. Extent of objectivity of PCR assessment score 3 Seems objective
3. Extent of internal consistency of PCR assessment
ratings; inaccuracies; inconsistencies; (in various
sections; between text and ratings; consistency of
overall rating with individual component ratings)
3 Seems consistent except for calculation of PDO
and question of whether to score environmental
sustainability
4. Extent of identification and assessment of key
factors (internal and exogenous) and unintended
effects (positive or negative) affecting design and
implementation
- No information
5. Adequacy of treatment of safeguards, fiduciary
issues, and alignment and harmonization
- No information
6. Extent of soundness of data generating and
analysis process (including rates of returns) in
support of PCR assessment
- N/A
7. Overall adequacy of the accessible evidence (from
PCR including annexure and other data provided)
1 No documents provided
8. Extent to which lessons learned (and
recommendations) are clear and based on the PCR
assessment (evidence & analysis)
2 The Review recommends 2 (unsatisfactory). The
PCR, used findings, lesson, recommendations
interchangeably.
9. Extent of overall clarity and completeness of the
PCR
3 The Review recommends 3 (satisfactory); the PCR
was complete, the numerous outputs and outcomes
tables were fully completed. All the required
narratives were given and also most if not all the
ratings completed with analysis.
Other (specify)
PCR QUALITY SCORE 2.33
PCR compliance with guidelines (PCR/OM ; IDEV)
1. PCR Timeliness (On time = 4; Late= 1) 1 Seems to have been done one year after the
completion date
2. Extent of participation of borrower, Co-financiers
& field offices in PCR preparation
UTS
3. Other aspect(s) (specify)
PCR COMPLIANCE SCORE
*** rated as Very Good (4), or Good (3), or Fair (2), or Poor (1)
References