/z2 Z2/

PORTFOLIO MANAGEMENT:

NEXT STEPS

A PROGRAM OF ACTIONS

Opertous Polkey Deprmet~JuIy 221,1993

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

THE WORLD BANKWashington, D.C. 20433

U.S.A.

LEWIS T. PRESTONPrMdent June 15, 1993

MEMORANDUM TO THE EXECUTIVE DIRECTORS

Subject: Poroio~ Manajement: Next Sts-A Proiranhof AconS

In my concluding statement covering the constructive Board seminar of May 4, 1993, on the draftmanagement proposal for implementing the recomm ations of tie Portfolio Management Task Force(PMTF) Report, ¡ reiterated that poverty reductíon is the prime objective of the Bank. This objectivecan be attalned only by restoring or accelerating economic growth and by focusing on policies andprograms tbat specifically benefit the poor. To achieve ibose goals, we are totally committed to obtainingbetter results on the ground and to ensuring a greater development impact of our operations in ourborrower countries. I want to emphasize that bringing about the institutional changes required forimplementing the PMTF recommendations will require sustamed leadership from managemenx t atlleveis, and strong and continuous support from the Board in attaching as much importance to lendingresults as to lending volumes.

The attached document, Potfolso Management: Nexa Steps-A Program of Actons, is organizedaround the same major implementation themes as the PMTF Report and reflects our recent discussions.It describes the steps that are planned for strengthening portfolio performance management, as well asthose already under way within the various Vice Presideatial Units. lt also spells out the respor1sibilitiesand roles of borrowers, the Board, and Bank management and staff, and the timing of each of the actíons.A table summarizes dic internal processes the Bark uses to oversee portfblio performance; and a detailedmatrix presents in tabular form cach of the implementation themes, with the associated actions,responsibilities, and time fames.

A number of task forces and special studies have been launched to assess how to respond to someof the PMTF recommendations. We will share the findings of those studies with the Board as theybecome available. The Board will also be ragulsr¡y informed of the progress in implementing the "NextSteps" ction program through the discussions on County Assistance Strmtegies and through the AnnualReport on the Portfolio Performance of March 1994. Finally, we will present to the Board an assessmentof the overall implementation performance of the uNext Steps' action program within the next 12 months(lune 1994).

. .~~~~~~~~

PORTFOL10 MANAOEMENT: NEXT STEPS-A PROGRAM OF ACr1ONS

Table of Contents

INTRODUCTION ................................................. 1

A. INTRODUCING THE CONCEPT OF COUNTRY PORTFOLIO PERFORMANCEMANAGEMENT LINKED TO THE BANK'S CORE BUSINESS PROCESSES ...... 4

Country Portfoíio Performance Reviews .......................... 4Portfolio Performance Management in Country Assistance Strategies .... 5.... Effect of Portfolio Performance Management on the Volume and Composition

of Lending ....................................... 5Annual Portfolio Perfornce Reporting .......................... 6

B. PROVIDING FOR MORE ACTIVE PROJECTIPORTFOLIO RESTRUCTURING .... 7

Country Portfolio Resucturing ................................ 7Project Restructuring ...................................... 8

C. IMPROVING THE QUALITY OF PROJECTS ENTERING THE PORTFOLIO ...... 8

Country Commitment, Broad-Based Participation in Project Preparation, andChanges in Stakeholder Comimitment ....................... 9

Rigorous Analysis of Project Risks/Se1 sitivities ..................... '1Implementation Emphasis dur¡ng Design and Appraisal ................ 11Implementation Plans ..................................... 12Cofinancing ........................................... 12Economie and Sector Work ................................. 12Loan Documents ........................................ 13Covenant Database ...................................... 13Bank Use of Financial Covenans ............................. 13

D. DEFINING THE BANK'S ROLE IN, AND IMIPROVING ITS PRACTICE OF,PROJECT PERFORMANCE MANAGEMENT .13

Bank's Role in Project Performnce Management .13Performance Monitoring and Portfolio Management Information Systems. .... 14Decisiveness in Portfolio Performance Management .16Procurement Issues .17

E. ENHANCING OED'S ROLE AS AN INSTRUMENT OF INDEPENDENTACCOUNTABILITY, AND GIVING GREATER EMPHASIS TO EX POSTEVALUATION .17

- li -

F. CRPATING AN INTERNAL ENVIRONMENT SUPPORTIVE OF BETrERPGiRTFOLIO MANAGEMENT ... 18

Skill Needs and Reruitment ..................................... 18Skill Mix ............................................ 18Recruitment ........................................... 18Young Professionals ..................................... 18

Training .... 19Management and Leadership Tranng ........................... 19Bank Skills ........................................... 19Professional Skllls ....................................... 19

Rewards and Incenftives ........................................ 19Operational Managers .................................... 19Operational Staff ........................................ 20Promotion Criteria ...................................... 20Staff Continuity ........................................ 20

G. GIVING ATIENTION TO GENERIC FACTORS AFFECTING PORTFOLIOPERFORMANCE ............................................ 20

Businms P ices and Portfolio Management ...................... 20Ressess, Simplify the OD system ......... .................... 21Field Offices .......................................... 21Technical Assistance . .................................... 22Inspection Function . ..................................... 22Budgetary Implications ..................................... 22

CONCLUSION ......................................... 23

PORTFOLIO MANAGEMENT: NEXT STEPS-A PROGRAM OF ACTIONS

INTRODUClION

1. The Portfolio Managenlent Task Force (PMTF) was convened a year ago to review and makerecommendations for improving the Bank's portfolio management functions. Analyzing the causes of thedeclining performance in the Bank's portfolio of operations under implementation over the past decade,the PMTF Report' was an important milestono in the Bank's self-evaluation efforts. The principal thrustof the Report's recommendations was to make the country operations portfolio the basic unit ofmanagerial accountabiity within the Bank and to rebalance internal processes, priorities, and incentivesto ensure continuous and adequate emphasis on the management of operations under implementation. TheReport also emphasized that bringing about this change will require sustained leadership from all leveisof management and strong support from the Board in attaching as much importance to lending results asto lending volumes. Discussions with the Executive Directors and consultations with Bank staff andmanagement indicate broad and strong institutional support for the Report's recommendations and clearlyunderline the urgency fer tieir implementation.

2. Taking as its point of departure the Report's recommendations and the follow-up discussions withthe Board and staff, ihis paper presents a dotailed program of actions to revitalize the way the Bankassists its Borrowers. Ibis action program must be considered within the Bank's overarching objectiveof poverty reduction, which can be achieved only by restoring or accelerating economic growti and byfocusinç on policies and programs that specífically benefit the poor.2 It is therefore critical that thelinkage bttween the Bank's central focus on poverty reduction and the proposed heightened emphasis onimplementation be fully recognized. While commitment leveis and thd size of resource transfers cannotbe ignored, the ultimate test of the Bank's operations is the actual performance of Bank-flnancedprograms, including their long-term development impact On our clients and particularly on the poorestsegments of their populations. Indeed, many of the key issues and concerns identified in the devolopmentof the proposed action program involve instruments that are essential to improving the poverty impactof the Bank's portfolio: for example, addressing effectively the issues of partcipadon, project ownership,NGO involvement, and improved performance monitoring indicators.

3. As Mr. Preston emphasized in his cover memorandum to the Report, "the performance of theprojects and programs we finance, and their ultimate development impact, is a fundamental measure ofthe Bank's ability to assist our members effectively." Given this objective, reversing the decline in thequality of the portfolio must have the highest priority in the Borrowers' and the Bank's agendas.Borrowers, the Board, and Bank management and staff must sustain their attention to improvingproject/portfolio management, building upon dhe frank examination of portfolio quality that marked theReport. Tbis focus places the Bank at the cutting edgo of the aid effectiveness debate. Ile Bank's abilityto work with Borrowers in improving the quality of their portfolios will be central to sustaining theBank's role in its development efforts across the regions.

4. All the regions have initiated follow-up to die Report. Indeed, the regions provided many of the"best practices" on which die Report drew in making its recommendations. Management is committed

1. Effective Implementation. Key to Development pact (R92-125), November 3, 1992; hereafterreferred to as the Report.

2. See World Devwe¡pment Report 1990 on poverty.

-2-

to making best practice standard Bank practice. Improving the performance management of the Bank'sportfolio will be an evolutionary and complex process involving stakeholders inside and outside the Bank.Wbat is presented bero la not intended to be a rigid implementation plan. Rather, it is a set of actions-some mandatory, some discretionary-based on new and existing practices, supported by more efficientand client-oriented business practices and processes, and intended to progressively reorient the Bankculture toward more effective and consistent emphasis on the sustainable development lnipact of Bank-financed operations. Because cach Borrower embodies a unique set of circumstances and challenges, theapproach to portfollo performance must be botb tailored teo flt individual country circunstances andflexible enough to adapt over time. We must remember, however, that while new institutional behaviorand improved Bank proceases can contribute to better portfolio performance, it is the actions of ourBorrowers that will ultimately have the greatest impact on project quallty. For this reason, strongerBorrower commitment to, participation in, and ownership of Bank-financed operations are essential tosuccess. Therefore, the most important change proposed in tbis paper is placing portfolioperformance-the managoment of the operations under implementation-at the center of country assistancestrategies. This single fundamental change should allow the Bank and its shareholders to ensure thatthe processes, incentives, and priority changes proposed are fully responsive to t:e Bank's support forits Borrowers' objectives ií poverty reduction.

S. Instruments to monitor the implemonution of the "Next Steps" program of actions will be put inplace at the project, country, regional, and institutional levels. The flow chart below outlines the internalprocesses the Bank uses to oversee portfolio performance.

* At the project level, accountability for performance management will rest with the countryand divisional units. The existing system of supervision is being strengdened with additionalresources and more systematic reviews of project performance by regional management,particularly for those projects identifled as "facing problems." OED, tdrough thePerformance Audit Reports and impact evaluation studies, will continue to provide valuablelessons of experience.

* At the country Lovel, two relatively new instruments will play a major rolo in theimplementation of dioe Next Steps' action program: Country Portfolio Performance Reviews(CPPRs) and Country Assistance Strategy (CAS) discussíons. CPPRs wil provide anopportunity for broader and more systematic interaction with Borrowers-and particularlywith core agencies in government on portfolio implementation (see paras. 8-9). CAS Boarddiscussions wlll provide an opportunity for Executivo Directors to give feedback to staff andmanagement on key implementation issues (see para. 10).

* At tie institutional level, the Annual Report on Porifolio Performance (ARPP) will informExecutive Directors of progress achieved and will develop action plans to remedy pendinggeneric issues (seo para. 12). Finally, as part of the semiannual Board briefings on regionallending programa, RVPs will present the implementation status of their portfolios, includingspecific regional action programs.

Tie flow chart also sunmarizes the various feedback mechanisms for the Bank's portfolio managementprocesses, lncluding evaluation led by OED. The strengthening of all these internal processes will besupplemented by intensified efforts to increase Borrower ownership of and commitment to Bank-flnancedprojects (seo paras. 18-21).

porMolao Management Review Proceso

AT PROJECT LEVEL AT COUITRY LEVEL AT BOARO

AGGREGATE BY COUNTRYíSECTO.R * AnfuOl Repon of P e nfol p_nf_m* Prolect Supeovisbn Report .(ounlryand U~ smn).A~~~~~~~~~~~~~~~~~~~~~~~~~RW ed by Board in JanualyCompleted by Task Manager

IJ Revlewed by Divislon Chlef AGGREGATE * Country Portfollo- Sen~iannual reviews by a CO NIWf Pethnce Rev~w o Anued r~Deparment aM Reglon CP R) _- lUdtenr reviews as appropriate _ En_)vI- Restructullfg as approprlate __*Eiomn

-ricorpotate .orrower co tcsns ~ J g0veWrn t P E-== M~WE a Rrouitcng_as approprlate I

o ~ ~ ~~~~~~wk A:;o9C~ aif

Producd for Country he of pr* ot oflefs=~~~~~~~~~~~~~~S | = / -oarrei | Bbto

(ICRS)~~~~~ DIIWT TOTR 80R *#R

^ O r.orate mop ower col ento | DGO Evadon Motu_ I * ~~~~~~~~~~~~~~OE PrXtmtpdacbs DEYEtOPMEN

_ NPUTI~4T0 *OEDCOUfltVYAS8IStSI OED Secor R*v .* New ocns[IPU1 INTO gED Pro¢ess Reviw s _-New ESW

. _ ~~~~~~~~~~~~~~~~~~~Evaluadon Resuns$* OED Prolect Pertormaroe Audit Reports,

Prodcd fbr about half of porNdO110

• OED Profet Impac Evaluatin Studbs |

N9PUIIN1,10 I *OED CountryAses~* Roews - ATRIBUIE

sadFS1 12

- 4-

6. After outlining recent trends ín the Bank's portfolio and discussing the key factors that contributedto the decline, the Report developel recommendations i seveo major areas to reverse the deterioratíon:

A. Iutroducing the concept of country portfolio performance management linked to the Bank'score business processes;

B. Providing for more active projectfpotfolio ostructwring;

C. Improving the quality of projects enteríng the portfolio;

D. Defining the Bank's role in, and improving its practice of, project performance management;

E. Enbancing OED's role as an instrument of independett accountability, and giving greateremphasis to ex post evaluation;

F. Crating an internal environment supportive of beter portfolio performance management; and

G. Giving attention to generic factors affecting portfiolio performance.

In each of these major areas, this paper reviews the steps that are planned for strengthening portfoliomanagement, as well as those that are already under way within the various vice presidential units,spelling out the roles and responsibilities of the Borrowers, the Board, and Bank management and staff,and the expected timing of the various actions. Annex A presents these actions in tabular forn.

A. INMRODUCING THE CONCEPT OF COUNTRY PORTFOLIO PERFORMANCEMANAGEMENT LINKED TO TRE BANKiS CORE BUSINESS PROCESSES

7. 'he effective implementtion of Bank-fiaced operations is vital to the growth prospects of ourBorrowers and, their ability to achieve sus le poverty reduction. Compared with annual new lendingof about US$22 billion, outstanding commitments on projects and programs under implementation totalabout US$140 billion, representing some US$370 bilion in project and program costs. Moreover, thetotality of operations in a country better representa the Bank's development assistance efforts than canany single project, or a single year's lending program. 'Terefore, the country portfolio is the appropriateunit for evaluating the development impact of the Bank`s operations in a country, and particularly theimpact on poverty reduction, sustainable development, and private sector development. Increasíng theeffectiveness of our operations requires an immediate action program over the whole spectrum of theproject cycle, from economic and sector work (ESW) and assistance in dealing with implementationproblems, to improved mechanisms for measuring performance. To make these improvements part ofan integrated assistance program, and to ensure that operations under implementation continue to reflectthe Borrower's priorities, all regions will be required to conduct regular CPPRs and to reflect the findingsof the CPPRs in their CASs.

8. Couey.+ Potfolio Perfomance Revews. Bank staff have long used Country ImplementationReviews (CIRs) to address, with their country counterparts, any implementation problems. Traditionally,CRs focused on project lmplementation issues and almed to reach agreeiIent on remedial actionprograms. The most effective CIRs gradually evolved aito discussions of generic issues affecting

portfolio performance-for example, the availabiity of counterpart funding, civil service salaries, theoverall macroeconomic performance, and the external economic environment. In the LAC and AfricaRegions, some CIRs expanded into discussions of the likely impact of portfolio performance on theBank's assistance strategy, and therefore~ were precursora of the CPPR.

9. Bullding on initiatives that are now under way in the regions, the Dank has decided to immediatelysystematize its approach to the CPPR. The purpose of the CPPR wiIl be to

* assess the Borrower's continuing ownership of and commitment to the existing country portfolio;* review progress in key programs and projects under implementation;* identify solutions to problems in individual Bankc-supported programs and projects;* identify generic issues;* establish clear links between the countrg asdstance strategy and implementation

performance-whether through new lending, the provision of implementation assistance, ESW,and/or donor coordination; and

* develop with the Borrower specific time-bound remedia] action programs.

The format and content of CPPRs should remam flexible to allow country deparments to tailor them tospecific country conditions. The frequency of CPPRs will be determined on the basis of the size,composition, and status of the specific country portfolio: each large country portfolio will be discusedannually with Borrowers; smaller portfolios wil be reviewed regularly, but less frequently. On the basisof the CPPR discuasions, the Borrower and the Bank should agree on an action plan that clearly statesthe roles and responsibilities of, and the level of resources to be committed by, the various partiesinvolved (Borrower, lmplementing agencies, Bank staff, and other financiers and stakeholders). Theaction plan will then be used as an input to the Bank's core business planning and budgeting processes.lhe Operations Policy Departnent (OPR) will identify and disseminate best practice examples of CPPRs.A first core of best practice CPPRs has been identified (Phhiippines, Morocco) and is being disseminated.Reviewing experience with CPPRs will allow the Bank to identify any broader sectoral and country issuesas input into the ARPP. OPR will review experience with the CPPRs over the next 12 months and wil;prepare a report .or management consideration by June 1994. 'he report wil be forwarded to the Boardthereafter.

10. Poitfolio Pe>fo*manne Maagemet in Couy Asie Strogies. CASs for alí activeBorrowers are submitted to the Board on an agreed annual schedule. OD 2.11, Cow¿try AssistanceStrategies, issued in November 1992, called for explicit treaument of portfolio management issues,including the rationale for the review cydle of individual CPPRs. Country department directors areresponsible for ensuring that portfolio performance management experiences are fully reflected in CASs.CASs have already begun to address portfolio implementatian issues more consistently. On the basis ofthe feedback given by the Executive Directors, OPR will prepare a status report and makerecommendations to improve the CAS presentations fi&er. This repon will be first discussed by theStecring Committee iu late July 1993 and will be pred to the Board in August. In parallel, OPR willbe disseminating best practice examples (see Box 1).

11. Effect of Portfolio Perfúoe Managemnt on the Volume and ComposWon of Lendig. Thesize and composition of Bank lending will reflect heightened attention to portfolio performance. Whileaddressing Borrowers' resource requirements remains a critical objective of the Bank group, it is not inthe best interest of either the borrowing countries or of the Bank to increase commitmnents to sectors orentities where project performance is inadequate. In such countries/sectors, priority must be given to

-6-

remnedial measures to ensure the effective use of committed funds or to reorient lending to differentsectors or Borrowers within the country. If new commitments could undermine the successful completionof ongoing projects by straining implementation capacity or drawing off counterpart funds, the Bank andthe Borrower should agree to reduce the level of new comrnitments to concentrate scarce countryimplementation capacity and counterpart funding resources on the speedy completion and disbursementof existing operations. Conversely, when a government has demonstrated its willingness and ability totake serious remedial actions to improve its portfolio performance, the Bank should make newcommitments available so that the government is not handicapped by lack of resources. ExplicitWy linking

Box 1: Format and Content of the Tunisla CAS

5t+-b*,0,"t,"..U." ",",...

1%~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~. -; ..... ......... .. 8 . .

1~~~~~~~~~~~~~~~~A

portfolio perforance and country lending program will also requi re careful attention to he lesonsof past experience in the design of lending progran*c s. TIe resulting improvements in project perfomianceand implementation capacity will allow subsequent ínoreases in comidtments, facilitating the achíevementof the Bank's poverty reduction objectives and ensuring a far gre development impact of Bank-ftnanced operations.

12. Annul Po4foUio Perfonmance Repo~n. The Anmxul Report of Implementatíon and Supervision(ARIS) was replaced this year by the Annual Report on Portfolio Perforrance.3 The 1992 ARPP usedregional summres of country implementation reviews to provide the first countrybased overview of thestate of the Banl's portfiolio. Fuwm ARPPs will build on e suggestions made by Executive Directorsduring the review of the fidt ARPP to f er streoen this country focus. They will also more fullyaddress the linkages between OED evaluation and supernion rreults and weil incorporate any revisionsin performance monitoring indicators that emerge n*om the ongoing OEDIOPR work on rating

3. ARPP (R93-20, IDAIR93-23), FebFr ary 16, 1993.

3. ARP .R3-0 IDI9-3,Fbur 6 93

.7 -

methodology (see para. 34). Finally, they wil contan a sectoral chapter, based on sector reviews carriedout by tbs Central Vice Presidency Units (CVPUs), that will focus on key implementation issues. OEDwill carry out an independent review of the ARPP procesa in accordance with guidelines to be presentedto th Joint Audit Conittee o the first quarter of FY94. In Juno 1993, OPR, which coerdinatespreparation of the ARPP and has developed with OED the strucure and information requirements for theregional submissiona to the 1993 ARPP, will issae instructons to the regions. By late November 1993,regional and CVPU management will provide summaries, based whenever possible on CPPRs, and sectorportfolio reviews. The next ARPP will be presented to the Board for discussion in March 1994, andregional management will answer Executive Directors' questiona on ceountry portfolio performance.

13. Besides the ARPP, ¡be responsible CVPUs will present to tie Board two other annualreports-Poverty (in April) and Environment (in September)-which will, inter alia, discussimplementation issues. Finally, the three CVPUs will produce sector reviews focusing on best practiceand lessons of experience, in accordance with the agreed Board schedule.

B. PROVIDING FOR MORE ACTIVE PROJECTIPORTFOUO RESTRUCTURING

14. Cowatry PonfoUio Restuc . At times projects in a country portfolio face conionimpleinentation problems. A number of factors can cause such a situation-a change in government,shifts in priorities, a sudden large price variation in key exports, overly optimistic assumptions aboutinstitutional capacity, or a contraction ín counterpart funding. Experience indicates that, when problemsaffect a large portion of projects, it is appropriate to restructre the entire portfolio. The alternative ofleuing task managers and individual govermment agencies address problem projects on a casa-by-casebasis across the entire portfolio has proven costly and ultimately unsuccessfil. Therefore, when a largenumber of projects in a country program face similar issues, the Bank will use portfolio restructuring toreassess with governmrnts the scopo and objecdves of their country programs. Restructuring ís a lengthyprocess, because it serves to bridge differences of views among central ministries, relevant localauthorities or implementing agencies, and the Bank and its cofinanciers. Nevertheless, such in-countrydebate is essential for achieving consensus and often is tuatrumental in resolving long-standing differencesof views that may have interfered with project implementation and portfolio performance. It is importantto hi,-Ihlight that when a portfolio restructuring is undertaken, the Bank and the government have alwaysreached a basic agreement on the rationale for the restructuring and on the sometimes difficult decisionsto reduce substantially Bank financing of, or even cancel, entire projects. Borrowers have come torecognizs that it was not in their best interest to pursue projects or programs tiat no longer reflected keygovernment priorities or that did not appear likely to produce the benefita initially anticipated. It isprecisely on the basis of these kinds of conclusions that governments can build stronger and moreresponsive portfolio implementation records.

15. As a reflecion of the increased attention to portfolio managemnent in the policy dialogue which hasdeveloped in recent years, a number of country portfolios were restructured in FY92: Brazil, India,Nigeria, and Turkey. These efforts followed extensive restructuring reviews with Borrowers, examinedproject objectives and implementation performance against die current country assistance strategy, andsought to realign portfolios with evolving couuty priorities. Projects or project ceomponents werecanceled after efforts to remedy underlying problems had proved unsuccessful. The restructuring of theBrazil portfolio ilustrates this type of approach (Box 2).

-8-

Box 2: rl: Portfolio Restructurng

~* til hi0n 04 fha B wih 81t Ub and Ati11# oi,iin kYn fEt*: L*$i. b;Uop 0<fa:oU thi p*I4 254<0t30 bcn) wex o¡cdncd~~~~~~~~~

0aj ---` ni.ías .. . lha ..... tviwn n~faBn sdsi¡iíotvwo h cfo (whiohfot

16-rjc ThiA 7uty4aa g. ~ f Iiut@ideprtmn directors fh andJ reginaliy Ipresidenttso willw now

thsa.e 2t.Gy. monitoroteperform~anepi#y ofh Wividuatl problem promjvets tuoa prvnt havingalrgepartItof * tlw por o~~n i a~ aaotfolio indsrs.mi rcs a egun. F1or p~sex h ample, in PY93, LAC unera o aa hrceviewI~of~. Argenina problem* pojcs As aonia tslt, thre projects wee canfelei~dty (¡ iba mlnfcat Housing and

Ba4AiN Df) two¡n~b prjectsl were U restructured (Water Supplyi pO SEGA) and imortant reallocBaztion oloan poed wee tae t~o snvromentalan ~otwher Bo~r4t ass*oc. w qiated with the> Yiacyrea aydot1oii tPrjc.B Jly 1~t, 199,o*hi one prjetÓs$gci of LeAoete Argnnaotlo~ isk exec ed to avea ratin stdatu

16 . Simiar ctivitestn¡rare Cow entr udeprtakent inalregtons.vidnregional1992ethe sid egins havenosystrlem 7oeatical ons,tor they pefranre woflindia ODrobestrctring53 thes. toaevn caingcallargon pagrteofithe poretfolio oudisreac. hi pover s haS. bilgbn.i For3 Thmpe, ARPP3,A wilprvderto an rnalyiso

BANADiE)two projectser restrucrig.Wtu mroed (Wte upplly atetr SGand aore vioiporat rtealoctionat porfoiloaes n prthcee ontee tof suenvirionmenand aCPPs thervlm work prsojcate wiceltion sheouldt idrorelsectric

Pr9,wet n the JuIy 1, 1993, OOIk one proje of the Areeted a to rtfoo is cml e. pOter tha tingtau

of7. Thmla aepot'vts aromendto 0W stelngudtaken tohe ruagityof Spraions atunty 192 is reing iona aved

1r6etrcts readis foro aoudtey arepartPn directorsing a3 oerse Lofae prcvnts a ioln anrwsystemaoearnmyonitcl reviewan o fvety tba vualso lomF pr3. eth to proeve de havicg anarans partc lnaEtwd projects hveratro regta. red (amter o Su itapity at entry,aoc mor igportantring uocatiosioiso lo trontets w of madeton andhonm p aetid the vosign of aroat nathonae Yaopydecreatise aSuchPajctiv en Bye beuly mofoe re troCtUs so thea rges cal is expected tthaue bei cmale.

17. iem eporoacts evonien atow te f e rengten ine aqireai of eractiv n a stmmatrh iszregiong beoave

projectu redioeratiorsa and Chec dred g orn s staccoding 53 oters Lof n tde Csc laribe d iniOD 10

wpsthrgvanm entscauldreiwachoatvitr US$at sislonI promote cros-egioARPl provider ofan pnactsies.oRegonaled prnajectestuts ringaso Wthke anmber ofvetitds úmr quality at entry,admr iün attntuion tosortolgth bssuns inteare rmogiga contexetod opervitionsan loRs the volumn of nrow toperaftionhodderase aSuerFct5vities are beunk nofitúere brucVPrín so exetet bto prhies cmpbetdse. iae hogotteBi

17s ew aeporoachs revommedtonutof sxtriengten theo qareas of opti ont ar entmmryise reevogw.ceae

4.attnn Regonale lroet san c rojet (Ratse 3 or 4 bycrd to he RVsmereetaboligdscrbed j erao toD 13.05,

Project Srqder adnron.

uptra te.ia reiwa vt htas rmtsassrgoa rnfro etpatcs

t~~Prjc S. riso

-9-

18. Cowatry Cond*ment, Br~ad-Based Padfp n in Prjet Phpon, and Changes inS*akeholder Comm_nent. Clearly, the appropriate scope, degree, and kind of ownership andcommitment vary with the type and objectives of an operation. The factors that promote ownership, aswell as the factors that make it wane despite clear commitment at the inception, need to be considered;particular attention needs to be paid to the appropriate balance between the roles of the Borrower andBank staff and to the affect of changes in policios and key personnel. Either can alter the effectiveownership of programs and projects under imploementation. lt should be clearly understood, however,that the primary responsibility for fostering broad participation and commitment rests with the Borrower,not the Bank. One basic test of "ownership" is the Borrower's leadership of project preparation. Forsome investment projects, the appropriate concept of ownership is as straightforward as a clearcommitment of the relevant ministry, which is demonstrated by the quality and determination of staffassigned for project preparation and follow-up. At the other end of the spectrum-for complexadjustment operations-broad political support for difficult undertakings is essential. Ownership ls anarea in which the Bank's understanding is still evolving; however, the findings of a recent OED study 5

underline that the main operational implications for gauging Borrower ownership and commitment areas follows:

• assessing, whenever possible during preparatory ESW and the early stages of project design, theinstitutional capacity and role of various constituencies (beneficiaries, NGOs, etc.), including theproject's likely impact on those constituencles;

- developing a systematic treatnent of the implications of ownership and identifying, as early aspossible ín the project cycle, the steps for facilitating the building and sustainability of ownership;

* reflecting the key country antecedents of ownership in assessing the possible impact of a projector program design;

* assessing the extent to which government agencies and key interest groups understand thenecessary policy changes; and

• clarifying institutional development benefits to provide incentives for the agencies to back upproject/program objectives.

To succeed, the design of an operation should be based on a sound understanding of the problems, needs,and interests of the people who are meant to benefit from it. Recogonizing this, the Bank isexperimenting with new approaches in participatory development and identifying practices that can beapplied widely in its operations. The numerous initiatives now under way within the Bank reflect theseemerging operational standards and variations in the level of ownership appropriate for different kindsof programs and projects. Such actions includejoint sector work with government agencies; beneficiaryassessments and stakeholder workshops during preparation and appraisal; greater involvement of NGOsin the design, preparation and implementation of projects; and broader and more substantive Borrowerparticipation and leadersuip throughout the project cycle, with increased focus on implementationplanning, project launch workshops, joint implementation reviews of country portfolio, and joint

5. SY¡ntural and Sectoral Adjustent Operatdons-Second OED Revew, June 30, 1992, Chapter 10.Th'i study covered adjustment operations; however, the findinga are also relevant to investentprojects.

- 10-

discussions of country strategy, including pipeline development. Broadening ese efforts ad fostesinggreater government involvement in the portfolio has a high priority in the Bank's work program. Thisis consistent with the lessons leasned from the Narmada project.W More specifically, EXT (andsubsequently OPR) and the Participatory Development Learning Group have been analyzing andmonitoring participation initiatives (this work has largely bcen financed by SIDA). Borrowers, NGOs,and the donor community, many of whom have significandy greater experience than the Bank in workingwith organizations outside of government, are being consulted. Work in this arca includes efforts to tapthe potential contributions of beneflciaries, and particularly the poor, to Bank work. For example, amethodology for conducting participatory poverty assessment has recently been developed for staffguidance. Speaking of the participation of the poor in the design and implementation of projccts directlyaffecting them, Mr. Preston emphasized that this should become the norm for our operations in the yearsto come. Dissemination of lessons learned and best practices on participation will be facilitated throughthe OPR's publication of a Participation Handbook in April 1994. This effort is being pursued incooperation with GTZ, an agency with considerable interest and experience in this area. In addition, a"Participation Fund" is being considered to provide task managers with budget resources to inmrase thelevel of stakeholder participation in project development and design.

19. OD 9.00, Processing of Investment Lending, will be modified to require Bank staff to assess, atvarious stages during project preparation, the country commitment to and local support for proposedoperations.7 Bank staff will also be required to cdarify in the project documents the degree of ownershipachieved and the roles played by various constituencies during project preparation and implementation.A revised OD 9.00 will be circulated to staff by September 1993. Strengthening training programs topromote more systematic use of participatory approaches will be a priority objective of the trainingdivision (PMDTR) in Spring 1994.

20. Two issues important for facilitating participationlownership are information disclosure andNGO/Bank relations. The availability of adequate and timely information on proposed projects isessential if interested groups outside governments are to be able to commert on the content or impact ofBank-supported projects. Ensuring that the Banl's policy on disclosure facilitates a more fluid and openinteraction on Bank activities wil have a major impact on the ownership issue. In addition, attention isbeing paid to strengthcning the dissemination of information about the Bank's environment work,specifically GEF projects and environmental assessments. Proposals covering these issues are sebeduledfor Board consideration in July 1993.

21. In the 10 years that the NGO-Bank Committee has been operating, there has been a remarkableinorease in Bank/NGO cooperation at the project level. The Committee has also reviewed broader issuesof Bank policy: for example, how to promote participation and accouatbility, and how to orient aidmore towards meeting the needs of the poor. Looking to die future, the Committee expects to continueto work on policy issues and to maintain interest in and support for the Bank's work on participation.This and other channels will facilitate an expanded role for NGOs in development.

6. Lessons learned from Narmada (Sec M93-516), May 24, 1993

7. OD8.60, AdjustmentLendng Pol&y, addresses the importance of borrower ownership in adjustmentlending (ses paras. 37-40).

- 11 -

22. During the preparation of the PMTF Report, a constructive senunar was beld with a group ofexperienced govermnent officials familiar with Bank practices and processes. Much of the discussionfocused on the negotiation process; and the participanta' powerful and critical observations about Bankpractices affected many of the recommendations of the Report. Looking more broadly at the problemsfacing the Bank and its Borrowera, it would appear that this model of interaction could be useful inimproving Bank work during the preparation and implementation stges. With this in mlnd, the regionsand OPR will work together over the next year to hold seminars on Bank procedures to identify howprocesses can be improved from the Borrower's perspective. It is to be hoped tbat the ideas of peoplewho have had experience with the Bank, both inside and outside of borrowing governments, can enrichthe menu of proposed actions and help build greater consensus behind creating the "implementationculture" that is now an important objective of the Bank.

23. Rigorous AnalysLi of Project Rfsks/Snsé~vWes. Recognizing that uncertainties surround appraisalprojections, existing Bank guidelines8 cali for appraisals to provide the "expected" outcome-that is, themean outcomo across the set of possible outcomes. However, the statistically "expected" outcome is alsosubject to uncertainties and risks-something that few appraisals take into account. Therefore, manyappraisals convey a greater degree of cortainty than la warrned and frequently end up being overlyoptimistic. The amounts of time allotted for project completíon often are exceeded when estimates ofmstitutional improvements from past operations are overstated or when everyone, including the Borrower,believes that decisions will be made faster than they usually are.

24. Experience also shows that macrooconomic factors, the external environment, and the institutionalframework are critical to success. Even though sucb issues usually cannot be addressed by an individualproject, they must be consistently taken into account in project design and appraisal. In evaluatingprojects and programa, lead economists and project advisers will ensure that appraisals are makingrealistic assessments of the likely macroeconomic and institutional development and of the sensitivity oftie project outcome to variations in the assumed macroeconomic and institutional parameters.Considerable work has been done already on the issues in project economic appraisal and risk andsensitivity analysis. Tibis work, referred to in the Report,9 is dio starting point for developingquantitative appraisal methodologies for the more diverse and complex projects or project componentsin our lending program and for the social and environment sectors, which are growing in importance.OPR will coordinate this effort with the CVPUs and produce new guidelines on riskfsensitivity analysisby December 1993. The CVPUs wil develop specific saff guidance on risklsensitivity analyses invarious sectors in FY94 and, jointly with PMDTR, will offer a training workshop on risklsensitivityanalysis in Spring 1994.

25. Implmenton Emphas¡s dung Degan and Appmisol. A basic step to improving portfolioperformance is strengtiening the analysis of the institutional capacities of implementing agencies. Anumber of regional initiatives are under way to make project design conmmensurate with implementationcapacity. For example, the MNA region ls grounding project and program diagnostics in ESW,clarifying project objectives eariier in the project cycle, and giving attention to die respective roles of theBank and the Borrower. South Asia is expanding standard project analysis to cover the risks of weak

8. OMS 2.21, Econom¿c Analysis of Projects.

9. PMTP Report, Annex A, p.6. Economic Analysis of Proiects, June 1992.

- 12-

implementation capaclty, chagg Borrower and slakebolder commitment, and interest group oppositionto proposed project reforma. In Africa, the focus la on simpler design, adjusted to respond to theprogressive capaclty bui1ding of lmplomenltng agencies; early agreement with Borrowers on policies,embodied in formal Letters of Sector Policy; and the Identification of key project monitoring indicatorsto gauge progresa during imple tation. MeanwhUe, asl reglons have instructed their staff to addressmore systaically the generlc imetaton issies ideutitied by the Senior Operations Advisers. Thiseffort provides new operatona wlth feedback about the lessona learned from ongoing and completedprojects to ensure at ths tes of the past are not repeated. In addition, through online services, allstaff nww haye electronic accesa to reglonal lending operation databasea and to the OED database, twosources of information about 1he implementation perfornance of ongolg and completed project.Finally, expanded participation by NGOs has emerged as an important tool in facilitating implementation,particularly in die social soctora, where NGOs have broad experience in addressing implementationconstraints and in mobilizing community support for development actities.

26. Implen*on Plns. Afl projects and programa require sound diagnostics, ownership, realisticappraisalnsad risk analysis, and well-focused monitoring systems. But they differ in how firm1yimplcmenttion plans need to be constmcted ex ante. For largo projects such as dama, power stations,and major inftastructure facilities, a comprehensive Implemenaon plan must be prepared in advance lestearlier phases of tie investment ha wasted. However, for projecta tiat are made up of components thatcan exist Independently, as la most social sector programs/projects, experience has demonstrated thatdetailad ex ante implementtion planning beyond the itial years may not be desirable, given the needto adapt service delivery to evolving local dema~d conditona. A differentiatcd approach to ex antedocumentation is thus warranted to correspond to different project characteristics. For large, "lumpy"infrastructre projecta, loan documenta wlll continuo to include a clear-cut project description and adeulled implementation plan that has been disu and agreed with the Borrower. For maintenanceand rehabiliton projects, and for many projecs in te social sectors and some other arcas, loandocuments should include general tents diat describe project content and clearly set out projectobetives aud responsibilities, íncluding well-defined eligibility criteria and procedures, plus a time-phased implementation plan for the initha years. Carefal monitoring during the initial years wil ensurethat the implementtion plan for subsequent years la based on actual experience. For such projects, aformal midterm review will normally be required. Division chiefi and task managers will be responsiblefor ensuring diat appropriate implementation plans (including procurement) are prepared by the Borrowerduring project preparation and appraised by qualesd Bankt.

27. Coflwsig. The Report underlined the complexity that multiple cofiwanciers add to projects, withpossible adverse consequences for implementation performance. At the sme time, cofinancing providesimportant benefits to Bank Borrowers and developmont partiers. Cofinancing has grown from US$8billion ín FY88 to US$13 billion in FY92, and involves a wide variety of cofinancing partiers: bilateraldonora, multilateral financial institions, expon credit agencies, and private market sources. It isessential that the Bank seek ways to improve thd efflciency and effctiveness of cofinancing. To this end,in lanuary 1993, Cofinancing and Financial Advisory Services (CFS) established a task force comprisingrepresentatives from the regions, die Legal Department (LEG), and other Bank units, to examine theBank's current approach to cofinancing and to inake recommendations en how it can be improved. Thetask force ls consulting widely with Bank task managers and with major cofinanciers. The report oncofilancing will be conipleted by December 1993, and ita findings will be shared with the Board.

28. Ecoanuc awd Sedor Work. If projects are to have a development impact, their design must bebased on thte identification of issues affecting development aud die effective diagnosis of the causes of the

- 13 -

problem. This means tbat operations must be rootet In sound diagnostie ESW, undertalen with the fuilparticipadon of Borrowers and targeted at Identifying constraints that can be relaxed through publicpolicies andlor lnvestments. Th. CVPUs are preparing a review of the Bank's ESW to see how well Itdiagnoses development problems, takes account of implemenaton constraints ii its policy and investmentprescriptions, and provides a basis for projectlprogram identification within the agreed CAS framework.OPR will report on the findings of tho ESW study by December 1994 and will share these findings withthe Board. The CVPUs will then take the lead ¡i disemnating best practices.

29. Loa DocwuenUt. Loan covenantt are an essential element in the relationsbip between Borrowersand the Bank. In administering ita loan portfolio, the Bank has always emphasized the importance ofcompliance with loan covenants. To enhance compliance, Bank staff should ensure during appraisal thatBorrowers understand the required acope and coverage of implementation arrangements, and shouldmonitor agreements closely during implementation. Reflecting this, the elements of implementation plansthat are considered essen~ial for achieving tbe proJact's objectives should be incorporated into LoanAgreements lf they can be reallcally cast as legal obligations. Other features of implementation plansshould be incorporated into lettera of ¡nUt (not made covenans). Such leltera would not be legallybinding but would constitute a convenient performance monítoring system that could be modified byagreement between the parties as the project progressed. By S stb 1993, OPR will issue newoperationa guidelines on projecl preparation and appraisal to make mandatory the preparation ofimplementation plans and their inclusion in loan documents in an aoprip form. LEG will work withPMDTR to develop a specific training progra to fimiiarize task managers with these concerns, withthe strucwte of the Bank's legal documents, and, in general, with the legal framework of Bank lendingoperations. This training should be available by Juna 1994.

30. Cowna# Dde *. Suppordng the broader work on covea recomended in the Report, LEOhas initiated stepa to create a Bankwlde covenant databas.. Tie objective will be to give regional staffand operauional lawyers access to accurate and curnt information about the obligations containad in legaldocuments and covenants effectlveness. This database la expectad to be In operation by Dacember 1993.The regions will be responsible for providing evaluation and outcome inffrmation derived from projectperformance management actvities.

31. ~ank Use of Pwi Coves. The Report called for an evaluation of the Bank use of financialcovenants, with a view to using the findinga in revisad instuctions to staff and in training. To this end,OPR is currently reviewing the financial monitoring requiraments, including financial covenants, mostoften used in Bank operations. Ibe review will be completed by December 1993, and OD 10.60,Accouwlng, F l Rporng, andAung, will be reid accordcly by April 1994. To familarizestaff and Borrowers with the Bank's fiduclay requ~ s, the Acout Department (ACI) willprepare a handbook on financial accounability by December 1993. PMDTR will then revise the trainingmodules on financial analysis and project accounting by April 1994.

D. DEliNING TBE BANK'S ROLE IN, AND JMROVING m PRACnCCE OF, PROJEICTPERFORMANCE MANAGEMENT

32. Ba~'s Role i Pwjet Perfonuance Maaageeat. Complementing e increased emphasis onportfolio performance managemant in counry istance stratgies, country deparlmenls have sttard to

- 14 -

enhance their ability to facilitato on-the-ground results by adoptíng more efflcient and dllent-orientedbusiness practices and addressing key accountability issues. Across the regions, various experiments areunder way to involve Borrowers more effectively in upstream sector work and project preparation.Workshops are being used to bring clients up to date on sectoral approaches and to involve them, jointlywith Bank staff, in designing strategies and Bank assistance programs. The respective roles of theBorrower, implementing agencies, NGOs, and the donor community are discussed ear1y in the projectcycle to pavo the way for project preparation and implementation. Project launch workshops are nowused routinely, and regional training programs on Bank procedures (procurement and disbursement) arehelping Borrowors' administrative staffs to discharge their responsibility more efficiently. In Africa, theAF6 Country Department has broadened the use of 'participatory assessments" in new projects. Thisnew proceduro is almed at clarifying the views and roles of various stakeholders as an input to decisionsregarding the design, implementation, and evaluation of operations and the policies that will affect them.OPR will monitor the effectiveness and cost implications of tiis initiative to assess whether this pilotexercise should be expanded. The Economic Development Institute (EDI) is also refining its role inimproving project management by exploring diree main areas:

* With respect to the overail policy environment, a series of workshops aimed at policymakers andsenior project managers would specifícally link improved project performance to the macro andsectoral policy frameworka.

3 In die areas of project design and planning, EDI would disseminate a number of proventechniques focusing on promoting consensual and team-building methods for setting project goalsand objectives.

• A training program for project managers focusing on implementation issues could cover, interalia: people and financia1 management; planning and implementation tools such as Critical PathAnalysis; legal aspects, disbursement, progress reporting, and repayments; procurement policyand practices; project funding, including government budgetary transfers; and monitoring andcontrois.

Al! these initiatives are sfilí in an early stage of development and wil be thoroughly tested by the regions,with input from Borrowers, and monitored by OPR before broader application is considered.

33. Perfonwe MoaftorIag and Porfolio Management informoln Systems. The present ratingsystem converts judgmens on a large number of factora into numerical ratings. Follow-up analysis tothe 1992 ARPP conducted by OED and OPR has indicated that this creates three problems:

* lhe present system has focused too heavily on the mechanics of implementation-iLe., thephysical and financia1 aspects-with too little attention to monitoring progress towards theproject's development impact objectives.

* Since the system inevitably depends on die judgment of individuals, it lacks consistency acrossunits.

* At the early stages of implementation, the expectation of project performance is marked byconsistent optimism.

- 15 -

It ís tíniely to consider modifying the perfornance monitoring system and indicators, focusing on theissue of development impact and based on agreed and more realistic statements of project objectives.

34. Any performance rating system must include the mechanics of implementation-disbursements,contracting, the availability of counterpart funds, progress in construction, and so forth. However, itmust a1so adequately monitor progress toward the project's development objectives, including its impacton sectoral policies and institutional development. To this end, every project must include a lucidstatement of objectives and specific benchmarks by which to judge progress in achieving these objectives.This will strengthen the basis for assessing the progress of projects under implementation and forevaluating them on completion. Individual project objectives must reflect the sectoral and countrystrategic objectives laid out in the CAS, although they normally will address only a subset of the moregeneral objectives. Exercising more selectivity as to which project components are clearly essential tothe project's core objectives will promote more realistic project designs. Since long-term developmentimpact and project sustainablity can be fully ascertained only after project implementation, an interimperformance feedback is necessary. ibe implementation plan should include explicit benchmarks linkingrequired actions with the stated development impact objectives of the project. Such benchmarks couldbe modified, however, during project implementation to reflect changes in country or projectcircumstances. This will enable Bank staff to track the project's progress toward long-term developmentobjectives. For most projects, progress on physical implementation is an important source of indicators,especially for costs. But other variables also are important, especially for tracking the evolution of likelybenefits, which tend to be heavily influenced by changes in narket conditions in sorne projecu and bysocial and cultural factors in others. OED has been playing a lead role in developing an approach toselecting project-specific factors for monitoring the evolution of project costs and benefits duringimplementation, and of benchmarks to monitor progresa towards development impact objectives. Workon a revised rating methodology is now being carried out by OED and OPR, and a report formanagement review will be issued by December 1993.

35. While specific project objectives and targets allow for differences in country circumstances, someperformance indicators in a given subsector-such as the rate of progress in making textbooks available,the increase in female enrollment in education projects, the rate of reduction in water losses in watersupply projects, or the rate of increased self-financing in power projects-tend to be broadly the sameacross countries. The CVPU sector departnents, which are responsible for monitoring the differentsubsectors' performances, will identify sector-specific indicators that are most relevant to project designand monitoring. In addition, DEC will develop a set of macroeconomic indicators that are most relevantto monitoring sectoral performance. This work will draw on existing best practices, OED snd CVPUwork on indicators, and, as appropriate, the expertise of Borrowers and other experienced groups. ByMarch 1994 the CVPUs and DEC will develop an agreed menu of indicators across the major subsectorsin which the Bank operates. Thereafter, regional management will be required to ensure that project- orsector-specific indicators and development impact benchmarks are clearly identified in Staff AppraisalReports; project performance will then be monitored against such benchmarks.10 As part of the nextARPP, CVPUs will assess the progress made and problems encountered in sector performance rating andwill analyze any rating inconsistencies that may occur between regions or countries.

36. To improve the tracking of portfolio management data, OPR established a Portfolio ManagementInformation Task Force (PMITF) in March 1993. With the regions, OED, and other vice presidential

10. Lessons learned from Narmada (See M93-516), May 24, 1993.

-16 -

units, the PMITP is assessing the Banl's portfolio ioation data needs, including filing and retrievalpractices, in order to ident¡fy any gaps in, or limitations of, the existing Management Information System.The work will be completed by September 1993.

37. Midterm reviews provide an opportunity to discuas with Borrowers, in greater depth than is possiblein normal supervision misslos, progress toward achieving project/program objectives and the need forany remedial action. The regiona are already conducting midterm revlews for new projects, and they areexpanding the practico. In line with the recommendations of the Report, midterm reviews will not bemandatory becauso a flexible approach to establishing a framework for midcourse corrections seemsdesirable. OPR and the regions wil conduct an ex post atslysis of midterm reviews by September 1994.On the basis of their findings, the formulation of a Bankwido approach will be considered.

38. DecWvewss in Por/olio Pe.fonnwe Manoemewt. Recognizing dtat the advice provided toBorrowers through project and program supervision is one of the most impora forms of developmentassistance that the Bank provides, all regions have incroased the resources they are devoting to projectsupervision. Aross the Bank, the number of supervision staff-weeks per project rose by 9 percent inFY91 and 16 percent in FY92. In ECA, the intensive supervision effort is responding to die needs ofan instittionally demanding reform program in a politically and economically unstable environment; itsims to set the stage right la new or reactivated countries in terms of procurement, disbursement,accounting, and auditing procedures. TIe Africa Rogion also shsrply intnsified its portfolio managementefort in FY92. The regions are also tking steps to enhance the preparation of supervision missions,before their departure, dtrough systematic review of the project's ful history, review of the performanceof other projects in the same or related sectora, and examination of generic issues identified in the courseof economic and sector work or CPPRs. Ihe most comnmon managerial practices that are emerging acrossthe Bank are as follows:

• The RVPs and county departmets meet regularly (typically twice a year) to review the statusof each country portfolio.

• RVP offices monitor a series of implemtion variables that give regular feedback to thecountry management teams on undisbursed loan balances, disbursoment lags, audit compliancestatus, projeçts widi effeftiveness delays greater tIan six mondis, projects rated 3 or 4,investment projects older than 8 years, projects wihi closing dates extended for more than oneyear, and completion repor status. RVPs are also moeting selectively with returning supervisionmissions.

* Country department directors are actively leading discussions witd high-level goverrmentrepresentatives and cofinanciers on country strategy and portfolio implementation reviews. Allcountry departmsnts have instituted regular monitoring of the implementation progress of problemprojects, and division chiefs review all 3- and 4-rated projects prior to the departure ofsupervision itisions.

* Country department directors are using country tesms and peer reviewers to ensure that country,sector, and project issues are more fully dealt with by drawing upon regional experience andknowledge from other parts of the Bank.

- 17-

In addition, from July 1993 a structured review mechanism will be establisbed for projects that have beenin problem status for over 12 months. Specificaliy, for such projects, departmental managers will berequired to agree with the Borrower on a specific action plan that will specify the basis on which supportfor the project can be sustained. If this action plan cnnot be implemented within the following 12months, the project involved shouid be canceled or restructured. The impact of heightened attention toportfolio performance management will be reviewed by the Board during the annual ARPP exercise and,for individual countries, during the CAS discussions.

39. Pru¿wmt Issues. To save staff and Borrower time during implemention, OPR has issuedinstructions concerning the mandatory use of standard bidding and contract documents for all internationalcompetitive bidding (CB) procurement. This was a cental recommendation of the Report. LEG hasprepared standard language to be introduced into the procurement scbedules of Loan and CreditAgreements for this purpose. Accordingly, alí Loan and Credit Agreements negotiated after May 1,1993, include provisions that make Borrowers' use of the Bank's standard bidding documents mandatoryfor ICB procurement, witd such modifications as the Bank may accept to meet the requirements of theproject. These provisions will be incorporated in the procurement guidelines when they are revisedduring the course of FY94. Bank-issued standard bidding documents are already available to cover theprocurement of most goods, works, and consultant services. For procurement in other specificareas-such as turnkey contracts, supply-and-crect contract, textbooks, and pharmaceutical and small-scale operations-OPR has initiated or supported work. Where no such standard Bank-issued biddingdocuments exist, other internationally recognized standard forms agreed with the Bank will be used. Theregions, assisted by the Policy and Procurement Coordination Unit of OPR and the Legal Department,will contdnue to help Borrowers develop more effective standard local competitive bidding (LCB)documents. EDI will also continuo to strengthen Borrowers' understanding of the Bank's procurementguidelines. OPR has begun a comprehensive review of procurement guidelines to address issues in socialsector projects, projects in which there is substantial private sector participation, and adjustmentoperaions; the review is scheduled for completion in June 1994.

40. As of April 1, 1993, an Advisory Procurement Review Committee (APRC) is in place to ensureBanhvide consistency on decisions for the award of large contracts (above US$25 million for goods andworks, and US$10 million for consultant services). For such contacta, staff havo been instructed tosubmit award recommendations for review by APRC. OPR and the Legal Department are reviewing theReports recommendation for third-party verification to assess its feasibiiity, identify qualified outsideagencies, and outline an implementation plan. The review will be completed by December 1993 andshared with the Board.

E. ENHANCING OED'S ROLE AS AN INSTRUMENT OF INDEPENDENTACCOUNTABILITY, AND GPVING GREAT EMP ~ASIS TO EX POST EVALUATION

41. The Board of Directors has approved OED's recommendations on in the work program it presentedto the Joint Audit Committee. TIe one recommendation of the Report that falls within the mandate ofBank management relates to replacing the Project Completion Report (PCR) by an ImplementationCompletion Report (ICR). The ICR will contain a retrospective summary of implementation results andexpected outcome, with brief explanations of major deviations from agreed objectives. A particular thrustof die new ICR format 1s to reassess Borrower comnitment to operational objectives; appraise theBorrower's plan for transition tuoperadons, including the perfornance targets to be reached at various

.18-

stages of the operational period; analyze the risks atached to the sustainability of the operation; andreevaluate expected project benefits. The econonic rate of return will be recalculated, as appropriate,with çlear indications of the basis for the input-output data used and the rationale for projected benefits.Social and environmental costs and benefits will also be covered to the extent feasible. Borrowers willbe responsible for providing a project operation plan before the ICR mission, and for sending commentsto the Bank on the ICR report. In collaboration with OED, OPR is preparing guidelines on the ICR,which it will issue to staff by the end of July 1993. Country departments will send ICRs directly to theBoard, with a copy to QED. OED will subsequently prepare DGO evaluative notes for the Board and,in due course, will audit a representative sample of projects with inereased emphasis on sustainability.For some projects, impact evaluations will be undertaken after about five years, to provide more robustassessments of development impact. In addition, OED will, from FY94 onwards, enhance its evaluationwork through impact studies and country assistance reviews.

F. CREATING AN INTERNAL ENVIRONMENT SUIPPORTIVE OF BhTER PORTFOUOMANAG1MENT

42. The key action areas for implementing the Report's recommendations include skill needs andrecruitrnent; training; and rewards and incentives, as represented by performance management andpromotion criteria. Work to revise many of the Bank's current policies and practices in these areas wasinitiated as a result of last year's Personnel Function Study. The actions coming out of that study formthe backbone of the planned follow-up to the Report. Many of the enabling changes (for example, thenew performance management process for managers) have already been put in place. Attitude Surveys,such as the one recently distributed to the staff, will also serve as benchmarks to firther assess whetherthe "Next Steps" action program is inducing changes in behavior.

S.4 Needs and Recmintent

43. SkI Mix. Personnel and Administration (PAA) has receny put in place a revised staff planningmechanism to help identify the Banl.s skill needs and to recmit new staff with the needed skills. SectorStaffing Groups, led by senior staff from the CVPUs, have been established to help analyze and resolvekey personnel issues facing their particular sectors. They have identified shortages in skill areas inindividual sectors throughout the Bank. Skill gaps are concentrated mainly in the areas of procurement,private sector development, environment, the financial sector, the social sciences, population and humanresources, and public sector management. The sklll needs wlll be reflected in the FY94 recruitmentprogram. PAA will review initial experience with the Sector Staffing Groups in the course of FY94.

44. Recrument. Most recruitment in FY94 will be directed to areas of shortage identified by the SectorStaffing Groups. CVPUs will play a key role in this effort by assisting PAA in identifying qualifiedcandidates. Financial and Private Sector Development (FPD), for example, is actively seekingrestructuring and regulatory specialists. Of particular relevance to the Report follow-up, the RecruitmentDivision has intensified its efforts to identify larger numbers of qualified candidates with "hands-on"project implementation experience in developing countries.

45. Yowa¿ PFofessionuJs. The criteria for the Young Professional (YP) Program have been changed torecognize work experience as well as academic excellence and to include candidates with technical andsocial science backgrounds, such as antbropologists and sociologists. This year, the program stressed

- 19 -

the value of work e>perience in a developing country and specified an interest in the following skill areas:agronomy (e:tension services, irrigation), civil engineering, environment (solid waste management,industrial pollution, water and sanítadon), and public health. The response was encouraging: the numberof applications to the program increased significant¡y, and an increased number of YPs had a professionalbackground corresponding to the identified skill needs. In March 1993, offers of employment in the YPProgram were mado to 49 candidates, lncluding 9 witb technical backgrounds and 7 with a combinationof technical skills and additional education in finance or economics. Candidates with Master's degreeseducation have on average 4.3 years of work experlence, and those with PhDs haye on average 2.0 years.

Trg

46. Training programs are being introduced or revised by the training division as a result of the Report'srecommendations on management and leadership, oi Bank skills, and on professional skills.

47. Managemewnt ad Leadership Tmiaing. Changes in tie execuive development program, designedto give greater emphasis to quality management and implementation effectiveness, will be implementedin June 1994. Priority for placement in the program wil be given to new managers. Effectiveinimediately, this program is also being offered to candidates cleared for positions as division chief. Thetraining module for task managers will be oriented toward i_plemontation effectivenesa and will be pilot-tested In the first quarter of FY94. A new module on advanced negotiation skills is being designed byPMDTR and will be introduced in September 1993.

48. Bank Ski. The orientation program for new staff, which provides an overview of countryoperations, financial policies, operations evaluation, and so forth, is being strengthened. From FY94,it will be required for alt new staff prior to confirmation. A new three-week orientation program forincoming YPs, which also emphasizes effective portfolio management, has been introduced this year.An integrated curriculum dealing with Bank skills related to the entire operational process-economic andsector work, project design and appraisal, and implomentation-will be introduced for alt operational ataffin June 1994. A series of workshops on innovative approaches and best practices in projectimplementation for experienced taff will also be introduced in FY94. The seminar on lending operationswas reintduced in November 1992. Training faciities for field staff are being strengthened, both inthe field (primarily through self-paced learning) and at headquarters (for visiting field staff).

49. Professionol Sldlls. Additional training programa to enhance professional skills are being designedby the training division in conjunction with the Staffing Oroups and technical staff from the regions,CTR, LEG, and the CVPUs. These programa will complement the core program on Bank slklls and willcover participatory devolopment, risk/senitivity analysis, project negotiations, procurement guidelines,disbursement guidelines, contact administration, and financial analysis. Tbey will be offered beginningin FY94.

Rewanls and Incentives

50. Opertonaa Mwangers. A new performance management process, using a combination of reportson performance results, compotencies, and development initiatives, has been introduced this year for theperformance evaluation of managers. Training in die new system began in December 1992; by mid-March, all managers had completed dio training program. The new system of management competenciesaddresses the Report's concerns by focusing managerial attention on key aspects of portfolio management,

-20-

lncluding maintaining a strong client focus, building an effective team, and ensuring effective portfoliomanagement. It ls crítica for the Bank's success in sustaining the emphasis on implementation culturethat these competencies be consístently consídered in perfornance reviews. PAA will review theefficiency of the new system In the course of FY94.

51. Opeiona¡ Sff. A similar performance management process, based on a combination ofcompetencies and outputs, is being desig~ed for operational staff and will be implemented in FY94. Thisnew process wil take lito account issues raised by the Report by focusing the attention of rmanagers onstaff performance in ensuring effective project implementation, monitoring, and assistance; promotingclient/beneficiary participation and commitment; and ensuring conditions for long-term projectdevelopment impact and sustainability. It la hardly possible to overemphasize the importance of theseincentives to reinforce the change In culture toward implementation. Use in conjunction with revisedwork program priorities, appropriate reso':rce allocations, and a consistent message from managers, thenew performance allocation criteria should have a significant impact on implementation. The new processwill be announced to staff at the beginning of FY94 and will frst be used to assess staff performanceduring the FY94 performance appralsal cycle (February 1993 tirough June 1994).

52. Promotion Citeria. The promotion criteria used by thi panels to clear staff to Level 25 and aboveare currently being revised to reflect the Report recorúnendations. The revised criteria, which giveadditional weight to portfolio management work, will be used for the FY94 annual clearance cycle. Inaddition, promotion criteria for promotions up to Level 24 have been revised as part of the OccupationalStreams Project. The revised promotion criteria preeny cover the fiancial analyst, country officer,operations officer, technical specialist, and economist streans and give appropriate weight to portfoliomanagement work. The new criteria will be applied in FY94.

53. StaffCon~. Personnel ¡s also monitoring and influencing staffing continuity through the newreassignment mechanism-the Interna! Staffing Proceas. This process allows Peannel to identdfy casesin which staffing continuity, either for an individual or a work group, is essential for sustaining theimplementation culture. The new process was introduced at the end of 1992.

G. GIVING ANTION TO GENERIC FACTORS AFFECTING PORTFlLIOPERFORMANCE

54. Buss Pracices and PortJlo ManageeWnt. A number of actions now being undertaken areintended to simplify and enhance the quality of review processes and to promote increased delegation ofsome responsibilities to country department directors and resident missions.

55. All regions are engaged in systematic efforts to improve their efficiency, an area in which progresshas been mixed over the past few years. Under the present project processing review system, there areclear incentives tu anticipate and attempt to address the comments of many reviews. Ths Bank has tendedto equate consensus with project quadity, and this series of reviews has produced a pattern of addedcomponents and requirements, alí contributing to project complexity. The costs and benefits of suchreviews have not been systematically considered; however, it ls now clear that this situation cannot besustained in an era of growing budget presaures. Several proposals to address this problem have beenidentified; suggestions include moving toward a more icued regional review of project proposals;reorienting departmental review processs toward a moro in-depth and condensed review of project

- 21 -

quality; conducting departmental reviews at an earlier stage of tle project cycle to more aggressivelydetermine whether project preparation 18 proceeding as intnded and should continuo; and being moreconsistently selectivn in addressing issues. OPR and the Organization Planning Staff (ORG) will followthese initiativos and assess their impact on efficiency for evental broader applications. In addition, ORGhas begun a study on the efflciency and effectiveness of the review process. 'be study is will becompleted by November 1993.

56. To complement these regional initiatives, in February 1993 the Human Resources Development andOperations Policy (HRO) vice presidency established a committee on Business Innovation andSimplification (BIAS). Tnis committee, drawn from across the Bank, has two mnajor objectives: to guideOPR In identifying and analyzing areas of present practices where reform/simplification appears timely;and to vet and, when appropriate, cofinance speciflc regional proposals for busines~ practice simplificationand innovation. It ls olear that among ongoing regional efforts there are many examples of improvedpractices that could bave significant bonefits if the results were disseminated and made sustainable. Underthe direction of the committee, OPR has taken the lead role in assessing specific initiatives andrecommending where th8se various efforts can and should be disseminated and integrated into Bankpractico. The committee also provides a framework witbin which regions/departmonts/divisions can testspecifkc ideas. 'be intention is to provide a coordination mechanism, institutional support, and somefunding to supplement regional efforts and thus facilitate the process of identifying ideas with potentialfor Bankwide application. In addition, BIAS provides a window on innovative business practices beingconducted outsido the Bank. Multilateral development banks and many bilateral agencies have concernssimilar to the Bank's; moreover, many business entities are seeking innovativo ways to retain theircompetitive positions in a rapidly changing enviroment. Whie the Bank remain~ unique in many ways,the growing Importance of flexibility and responsiveness is a central theme in today's businessenvironment. The Board will be informed of the status of the various initiatives by December 1993.

57. Reassess, Spj the OD system. OPR has also introduced a more user-friendly system todocument and disemathe Bank's operational policies and procedures. With three output categories,the new system is intended to ensure greater consistency and clarity in the use of existing BankOperational Policies acrosa the operations complex. First, "Operational Policies" are short, focusedstatements of policy, prepared in tandem with policy papers and presented simultaneously for the Board'sconsidoration, in line with the reconumendation of the Ad Hoc Committee on Board Procedures. Second,Bank Procedures" statements spell out dhe common set of procedures or documentation that providesa framework for ensuring consistency and quality across the regions. Third, "Guidelines and BestPractices" contain explanatory and advisory materia1s that help Bank staff carry out Bank policies. TheJAC has been briefed on tIis new system, which is now operational. OPR will conduct a survey of theefficiency of the new system by the end of FY94 and will share the results with the Board. Reflectingthese results, all existing Opertonal Directives will be converted to this system by the end of FY95.

58. dO.ces. ORG has started a review of the role of fheld offices in portfolio management. Thereview will address each of the specific recommendations of the Report. lt will be a comparative analysisof field office experience in regard to implementation activities such as facilitating borrower commitmentduring implementation; approvals for routine procurement actions and use of loan/credit proceeds;advising on proposed modifications of implementation plans and schedules; and deepening assessmentsof executing agency ownership and capabilities. Tbe review will also consider how these conclusionsrelate to the scope for making greater use of third party verification and cortification, for example, ofstatement-ofenditure claims and related disbursoment documentation. While the types ofresponsibilities of field offlces vary with country conditions, the size and scope of the lending program,

-22 -

advisory services, and cost, the Bank needa to talco into account tie cost-effectiveness of differentfunotiona delegating different functiona to offise of different sizes. Against this background, regionalvice presidents will be In a beter position to decide he appropriate role of field offices in individualcountries. lbe review will be ready for management considoration by September 1993 and will then beforwarded to the Board.

59. Teduid Assls. ORG ls also reviowing the implementation of the recommendations of theFY91 Technical Assistance Task Force. Bu¡iding ownership and comnmitment through techuical assistancela particularly problematc becauae of the Inherent tenalon between providing implementation assistanceand crowding out Borrower effors anud ownership. lhe ORO review is assessing progress on themanagement of technlcal assistnce; processes affecting the quality of tochnlcal assistance provided toBorrowers; Borrower and agency participatíon, as reflected l the design of new projects and theamendment of existing ones; and institudon building and public adminsitration, with respect to thedevelopment of staff exportise. The results of the review ate due la September 1993. A second area ofwork is with the Institutional Developmaent Fund (IDF). By early June about $13 million in IDF fundshad becen committed to speciflc work on capacity building across all six regions. While start-up has becenslow, this new grant facility should enable staff to faciitate institution building. A budget of $25 millionwill be available in FY94. An anual report on IDF wil be prepared and made available to the Boardin September. la a third arca, OED and OPR havo becen working witd the UNDP on an issues-focusedseminar on technical assistance, to be preseted under the auspices of the Development AssistanceCommlttec (DAC) in January 1994. Tbis sominar will assess key problems facing multilateral andbilatera donors and will work toward producing a program of actions directed at addressing these issues;It will also offer an oppornlty for developing a dialogue among donors on these issues. Finally, OPRwill continue to produce and make avallable to the Board an annual report on techical assistance thatocuses on the problema facing this type of Bank asistance.

60. Inspecion Funclion. Responding to die suggeation hiat die Bank consider establishing anindependent operations inspection unit, a review was commissioned to talce account of experience withinspection functions in selected member countries and consider whetier a new inspection funetion wasneeded to augment the Bank's existing supervision, audit, and evaluation functions. A related butseparate issue analyzed was whether die Bank has ready access, when necessary, to a reliable source ofindependentjudgment about apeciflc operaions that may be facing severo implementation problems. Thereview concluded that the interests of the Bank would be best served by the establishment of anindependent Inspection Panel. The draft report was circulated to Executive Directors in June 1993 fordiscussion at an informal seminar in July. It will then be revised l 1ight of the discussion andsubsequently submitted to eie Board for approval.

61. Budgetary Implc ns. The overall budgetary implications of the Report's recommendations needto be assessed over time as the impact of the many activities discussed aboye beconie clear. Unitsconcerned with portfolio management issues and evaluation were asked to include in their FY94-96business plan submissions an initial program of actions, witd associated resource needs, to implement theReport`s recommendations. The analysis of these plans was reflected in ¡he FY94 Proposed Budget, tobe discussed by the Board in June 1993. The prelimlnary es~iate was for an inerease of about 12percent in portfolio management activities, from about US$152 million in FY93 to about US$170 millionin PY94. lhe inorease reflects primarily (a) plans to increase the average intensity per projectsupervised during the year; (b) expansion and strengthening of ficld offices; and (c) expansion of portfolioreviews and associated tasks. The total increase in resources required by the broader program will bemet partly by redeployment and partly from the micase noted above. The precise cost of a number of

- 23 -

initiat}ves mentioned in this paper wlll emerge over time. For example, the costs of the new CPPRs andthe costs of increased emphasis on ownership/participation will become clear only with experience. ThePlanning and Budgeting Department (PBD) and OPR will monitor these costs closely over the next year,and their flndings will be retlected in next year's budget.

CONCLUSION

62. This paper sets out in detail actions that wil provide a basis for major improvements in the Bank'simplementation perfo.mance. Within the context of the Bank's focus on poverty reduction, theseimprovements will be critical to achieving the on-the-ground impact that marks successful and sustainabledevelopment efforts. The success of this effort will require the full and continuous commitment of Bankstaff, management, the Board and, most importantly, our Borrowers. As this program has beendeveloped, internal meetings and discussions have emphasized and explained the priority of this programand the critical importance given to it by the management and Board. Sustained leadership from Bankmanagement and continuous support from the Board should ensure that clear and consistent messages aregiven on the importance of the implementation culture. Regions and country departments wilí, in turn,be responsible for ensuring that this level of priority and emphasis is sustained in the interaction with ourBorrowers. However, the Bank is only one actor. There is little doubt that our Borrowers will benefitsignificantly and the development effectiveness of this plan of action will be greatly enhanced if otherbilateral and multilateral agencies conduct a similar self-examination of their portfolios with follow-upaction plans. Progress will be reviewed in a Board briefing on the implementation experience of theuNext Steps" action program in June 1994. With a strong Board endorsement, this program willcontribute to major improvements in the Bank portfolio.

IMI m~~I

¡ F II

t ,~ 1...!I

:,g«LZU *} 4

.-:« 1Sl~~~ Jo I~. o

I- sv.$ .:mn. , saLo ,S

........... %-':> -12~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~7.

$ < ,F ;,, :i~~~~~~~~~~~~~~..... ..'... ....'. ..'

-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -

ME.~~~~~~~~~~~~~~~~~~~~~~~~~~~4

* ~ ~ ~ óm ssw°» *4t0plx

AAa<s~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

|O nAJo1

A. PROVIDING FOR MORE ACTIVE PROJECT/PORTFOLIO RIESRUCTURING

k,gN~~~~~~ Z,* * sa9

-Wiz, - Z.-~~~~ASmd I C9~C

.~~ ~~~ ~~~ ...4: " .. .... ..

.W*.*.ngplans * ~~~~~~~~~~~~~~~~~Co~ hd~oWgms *Disems Approve a£ As apprpite |;; ̂.vi.b:h approprat

. *~~~~~~~~~~~~~ Implemant pl*b po.pá., e *~m*totm, :. th icanirucn ot

proj«u~~~~~~~~~~~~~~~pkg

e' Et-tup r......... *g Ralmanas * Countr ertiacl t * Dissl ma * Approvcmn * As apy ¡ ARPP* Durn -~

Ieoding revew* Onacottry

"a , , , ,g CAS

ci. . . :d>*uss,ions

, . i , . , . , --~~1

C. IMROVING TB QUALITY OF PROJC'TS IMNTRCINOG T£ PORTPOLIO *>

Ob3ete : 2. are< ei.hb s

E, br)>

I_ - : e -...':!e.'!!:'-: ' -^ ... -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~... ..

"5S... .".i ;, *... .. ...... p

je.

'1~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~,zflo IZ11%6IOII~~f . . WL0UbOaI * $us***jzSmcX s

> v8;Tv ak. ResoibW Dow .er;;::2: < :.:

Objetive #: Evaluate Bank use of fmancdal covenauts

w of# 4 so'* nig _. sO- I . .. N;C_ .I ,D

*w' n **,oia 'ea~g ",,: res.,,, , "n,

' ib 10,60. 'A.,o;. .ACTIOPR . .. .

'ovtmm * Pk*'U * S Ap, -'.

i- D. DFPINING TIIE BANK'S ROLE IN, AN! IMPROVING ITS PRACTICE OF, PROJECT PERFORMANCE MANAGEMENT

Oaj sv a1 Or the Bh's prope role aud pay ipetial aisn tAart-up

OltoIectfr. I2: Devebp nw ef>mae moio systm suid impro e the porifáio msusemznt ifrmatioo system

! m E , 0~~~~~~~~~~ ~ 'fCoMle *f ''o~ ''' XDc, b19

Sepenbe 1993rCon~ . s * Sptmber 1994

O~iecl,, 13: Increme Bamk's deciveneuss lo porifoo performance maaagemeot

* Y de , ~iy aud * lnforwued ghr ' ''-C Ongo';g. 2: :g.-.- .. ope,o .. .;..e. ARPP

4 .otianlsm * Couty4epai¶snents *Discus ¡rneia Intormed el~Ob Gen~alz íuactiou plenaARPP' FY9

rmpeuidaa.o .

qIzu~~oj4~~ * ~~ * . .

W-000~~~~~~~~~~.~

* L ENHANCING OBD'S ROLE AS AN ¡YSTRUMENT OF INDEPENDENT ACCONTABILUTY, AM) GIVING GRKTR EMHASIS TU EX POEVALUATION

O0c B d Bank foiowup beyad disbunezet cmple to mam deveopint impac ad replace tbe PCR wld ao 1e ComplelbRep~ «aC

'4. '. .4 ... ~~~~~~~~~~~~~~~~~~~~~~~~~~~-.l ..... ...

1'.

P~4~~~~~~~~~~~44

E4 2S .- X/

~4'. vw~rrw 4.....

~~~~~~1 1

1 e 0 f :- 0e e .: ' 2

«Ñ.:9*. '.<

a ".. 4

1~ ' %Ji5k¡# :X ...ij. jiUjj~~nwooh

1 ~ ~ ~ ~ Yn

.v.~~~~~~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ .' - ' d.. .

!,~~~~ ,t

1i 1Wi i1 i. 7 i°t: W X

">llaZW~~~~~~~~~~~~~~~~

.. .. ...... .... .... ~ ~ ~ ~ ~~ .... ~ .

...... ~ ~ ~ ~ ~ ~ ~ ~ :9. ...

110,2 ~ ~ 4

«,,.,.<>/.~~ ~ ~~ ,. a iM.,..».

__ mpwp l~~A

«<~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ < .

1@~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ M W. orlovai o.

...~~~~~~~~~~~~~~~~~~~~.....

.4;. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~.~.....

MI,~~~~~~~~a~mua¿OljLO ~RUW EIV UID

'2.5~~~~~~~~~~~~~u14 dicSN Lnp zEiemetto oZA{f Dze tC

Obje~ ct le,pes h dicea of te ,nmendations of tue TécbIaI Ausstanrce Task Forc

lí

Obe * #4:0 Asses fe~ of then OldplemeDaeio0 of th e to of ~m o» pioce of sk Forei

;;a ^ ' 'h-i-* me',d * Anaua: '

,Ysj Pneo1A*,otoT . Y* O?OB ,afMá .zct * han 1-94

Objede #5: Ases f.albibkj of as iud.pcadeut Iuspecti.n office to review llanl actur>ns or problea projortFi of controverslilaute ad reco_md oilo

~~~~> J QiRIn . ........p... .. .01 .. :.í c Siphtu.:j pWtedM i _ Utif iy . . * D; . -l * IuI 1993

.* provo * Ausgust 1993

Objecive #6: Assess budgetary iníplhcations of improved portfolio manangement.* nocorporate ¡a FY94 budgot * PBt/afl Bank urnits * Approve FY94 * ¡une 1993specific resouxco nocds and budgctplan of actions of al units

* During annual budgi * PBDI&a Bank.Uni" * Approve annual * An~ay in Jueit parat~ou. :iv~o* o ng : .. :. budgels

' ',ufe years

O|ce 7: P=ovW s1~ kadersip for aud monitor the implmen¡tatiou of "Next Steps. e

Ic.,~p l. ,,,; ,, P,EXC a, r,gonl * Provi cOist.nt * Conti ly

* Revicw * ¡hileun 1994