48391 Public Disclosure Authorized - World Bank · The five sections are: i) Introduction - establishing the context, i.e., why was it decided to use an MfDR approach? Where did the

48391P

ublic

Dis

clos

ure

Aut

horiz

edP

ublic

Dis

clos

ure

Aut

horiz

edP

ublic

Dis

clos

ure

Aut

horiz

edP

ublic

Dis

clos

ure

Aut

horiz

edP

ublic

Dis

clos

ure

Aut

horiz

edP

ublic

Dis

clos

ure

Aut

horiz

edP

ublic

Dis

clos

ure

Aut

horiz

edP

ublic

Dis

clos

ure

Aut

horiz

ed

Sourcebook: Second Edition

Table of Contents

Acknowledgments 1 I. Introduction 5

Progress Has Been Dramatic 5 Moving From the First to the Second Edition Sourcebook: Three Target Audiences 6 Development of the Second Edition of the Sourcebook 7 Criteria for Case Selection 8 What is Managing for Development Results? 9 Key MfDR Terms 11 MfDR in Action 13 Promoting Country to Country Learning & Sharing Best Practice 14

II. Emerging Good Practice in Managing for Development Results for Political Decisionmakers and Senior Management 17

China: Towards Results-Based Strategic Planning 21 Vietnam: Design/Implementation of Results-Based Planning and Monitoring 35 Haiti: Lessons from the Interim Cooperation Framework (ICF) from 2004 - 2006 & the Extended and Revised ICF from 2006 Onwards 47 Peru: Alerta Miraflores: Managing to Increase Citizen Safety – Lima, Peru 59 Mozambique: Poverty Observatory and the Program Aid Partnership (PAP): Policy Dialogue, Civil Society Consultation, and Mutual Accountability of Donors and Government. 69

Emerging Good Practice in Managing for Development Results

III. Emerging Good Practice in Managing for Development Results for Technical Practitioners 79

Chile: External ISO Standards Certificate of Management Improvement Programs in the Public Sector 83 Malaysia: Integrated Results-Based Management – the Malaysia Experience 95 Rwanda: Performance-Based Financing in Health 105 Burkina Faso: Performance-Based Management: Case Study on the Monitoring and Evaluation Unit of PNGT2 117 Benin: Experience with Results Based Management: The Case of the Ministry of the Environment, Habitat, and Urbanism 131

IV. Emerging Good Practice in Managing for Development Results in Civil Society and the Private Sector 143

Mongolia: The Role of Civil Society in MfDR 147 Ecuador: U.S. Agency for International Development (USAID)/Geneva Global Initiative: “The Time is Now, Strategically Mobilizing Anti-Trafficking Organizations in Ecuador” 155 Tanzania and Zambia: In-Country Coordination to Improve Diflucan® Access 167 Indonesia: Aceh and Nias – Results Achieved using Community Reconstruction following the Tsunami Disaster in December 2004 179

V. Conclusions 191


1

Acknowledgments This Second Edition of the Sourcebook on Emerging Good Practice on Managing for Development Results is the product many months of dynamic collaboration and discussion. The Sourcebook Team (Elizabeth Ashbourne – Senior Operations Officer, Team Leader; Jeffrey Balkind and Hannah Cooper – Consultants) based in the World Bank, would like to thank its dedicated partners, in particular the case authors: Xu Lin (Director-General, National Development and Reform Commission, People’s Republic of China), Mei Wang (Senior Economist, World Bank, Beijing), Carolyn Turk and Rob Swinkels (Senior Poverty Economists, World Bank, Hanoi), Quang Hong Doan (Senior Economist, World Bank, Hanoi) Lisa Campeau (consultant, World Bank), Jeff Franco and Ignacio Soto (Voxiva Inc.), Momade Saide and Virgulino Nhate (Ministry of Planning and Development, Mozambique), Cristian Venegas Díaz, (Coordinador Técnico PRODEV Departamento de Gestión Dirección de Presupuestos, Government of Chile) and Heidi Berner Herrera (Jefa Division Control de Gestion, Ministerio De Hacienda, Government of Chile), Koshy Thomas (Deputy Undersecretary, Tax, Division, Ministry of Finance, Malaysia), Louis Rusa (National PBF Coordinator-Ministry of Health), Gyuri Fritsche, (Health Care Financing Specialist-Management Sciences for Health, Government of Rwanda), Claude Bationo – Begnadehi (Chief, Monitoring and Evaluation Unit for the Community-Based Rural Development Project – Burkina Faso), Pascal Z. Yaha (Director of Project Monitoring, Ministry of Economy & Finance – Benin), Jargalsaikhan Dambadarjaa (Open Society Forum – Mongolia), Timothy


2

Ogden (Geneva Global, Inc.), Daniel Aukerman (Vice President, International Programs, Interchurch Medical Assistance (IMA) Mavere Tukai, (East Africa Regional Pharmaceutical Manager, Tanzania, (IMA), Heather Lauver, (Assistant Director, Pfizer Inc. International Philanthropy Program). Contact information for the case authors can be found at www.mfdr.org/Sourcebook. The Sourcebook Team would also like to thank the Sourcebook Review Panel for its unfailing commitment:

Arne Paulson (IADB) Charles Clift (OECD/DAC – DfID) Gerardo Martinez-Freyssinier (IADB) Heather Lauver (Pfizer) Jargal Dambadarjaa (Open Society Forum – Mongolia) Jeff Franco (Voxiva Inc.) Jeffrey Gilbert (World Bank) Martine Stabile (IADB) Max Pulgar-Vidal (IADB) Michelle Gyles – McDonough (UNDP) Momade Saide (Mozambique) Nicole Robillard (Canadian International Development Agency) Patricia Rogers (World Bank) Per Bastoe (Asian Development Bank) Roberto Garcia-Lopez (IADB) Ryan Androsoff (World Bank) Timothy Ogden (Geneva Global, Inc.) Elizabeth Ashbourne (World Bank) – Review Panel Chair

And in particular the Inter-American Development Bank for hosting, coordinating, and participating in the Review Panel meeting.


3

The Sourcebook Team gratefully acknowledges the generous participation and advice of many colleagues in particular: Pedro Arizti, Isaac Matzner, Sheila Daunt, Patricia Rogers, Jayme Mendes Dos Santos (World Bank); Robert T. Schreiber & Caroline Hope (consultants); Benoit-Pierre Laramee, (CIDA); Pamela Johnson (Voxiva Inc.); Rachel Seligson Carter (Pfizer); Philip Bastiaenen (Ministry of Foreign Affairs, Netherlands); Herman Snelder (consultant, OECD). In particular, the Sourcebook Team would like to thank Elizabeth Ashbourne and Susan Stout for their guidance, coordination, and outstanding advice, without which the Sourcebook would not have been possible.


4


5

Introduction

Progress Has Been Dramatic

At the High-Level Forum on Aid Effectiveness (held in Paris in March 2005), partner countries, multilateral and bilateral donors endorsed the Paris Declaration, committing countries and donor agencies to aid effectiveness, including strengthening how they manage for development results. Specifically, they adopted a set of actions (such as stronger leadership and improved statistical capacity) that have helped them to better manage the implementation of aid programs, focusing on the desired results and uses of information to improve decision-making -- most importantly, focusing on the actual outcome and impact of programs, not the inputs used or physical outputs produced. Results management that is focused at the outcome level implies that policymakers and practitioners in partner countries and donors need to evaluate, for example, whether improvements are taking place in the quality of learning in schools and drop-out rates are declining, not only the how many classrooms are built or the volume of textbooks distributed. They need to assess if roads lead to employment and markets through economic regeneration, not just how many kilometers of highways were paved in a given project; and they need to be able to adapt their policies and programs when evidence suggests that results are not being achieved. The Sourcebook’s First Edition, which was issued in March 2006 as a product of the OECD-DAC Joint Venture on Managing for Results, provided emerging cases of this approach. The First Edition was structured around a set of core principles that were

I

Introduction

6

adopted at the Second Roundtable on Managing for Development Results (held in Marrakech, Morocco, February 2004). Some twenty cases illustrated how countries and donor agencies were managing for results. Cases were organized in three groups: national level, sector level, and specific project/program level.

Moving From the First to the Second Edition Sourcebook: Three Target Audiences

Managing for Development Results (MfDR) requires the participation and commitment of all development partners and practitioners: from committed political leadership, to a strong public sector, to an empowered civil society. This Second Edition of the Sourcebook is therefore structured to address three target audiences: 1) political decision-makers and management, 2) technical and institutional practitioners, and 3) civil society and the private sector. The new Sourcebook allows readers to observe MfDR in action. It presents 14 new cases that were selected from OECD/DAC Regional Mutual Learning Initiatives (MLIs) – regional workshops where countries discussed their experiences with managing for development results, as well as other sources. Each case is considered to be an outstanding or innovative case of MfDR in practice. Not all cases present complete pieces of work – in fact, the emphasis is on showing effective and successful processes. The cases show that all three constituencies need to be fully engaged if MfDR is to evolve from words and protocols to concrete actions and from blueprint plans to implementation reality. Each case is divided into five sections, allowing readers to access areas of specific interest and to compare approaches across cases. The five sections are: i) Introduction - establishing the context, i.e., why was it decided to use an MfDR approach? Where did the impetus come from? ii) Application - What does MfDR look like in action? How are policy objectives translated into a results


7

framework? Once you have buy-in, how do you move forward? iii) Problem solving - issues of note that arose during implementation, and mitigation strategies iv) Results - What did using an MfDR approach achieve? Can the process be replicated? What tools can readers use to implement a similar MfDR framework in their countries? v) Conclusion - Where will the project/program go from here?

Development of the Second Edition of the Sourcebook

The OECD/DAC Joint Venture on Managing for Development Results is a platform for sharing emerging good practices of donors and partner countries and for promoting the mutual learning in the area of MfDR. In the past year, the Joint Venture included in its work program the MfDR Mutual Learning Initiatives (MLIs). The MLIs were a series of four regional workshop organized in Singapore, Burkina Faso, Uganda, and Uruguay to collect and disseminate practical lessons of country experience with MfDR. During 2006, these MLIs brought together officials from over 40 partner countries that have been applying the concepts and principles of MfDR in their development planning and implementation systems. The majority of the cases in the Sourcebook’s Second Edition are developed from presentations at the MLIs. Each MLI nominated several cases for consideration. Subsequently, some 25 cases were reviewed by the Sourcebook Review Panel (comprised of representatives of partner countries, donors, civil society, and the private sector). All cases that were accepted into the Sourcebook were posted on the MfDR website; cases that were not accepted are also being posted in the website in a separate section (www.mfdr.org/Sourcebook), and will be considered for possible inclusion in later print editions of the Sourcebook. Effective MfDR needs to involve actors beyond the narrowly defined public sector agencies – those in civil society, the private

Introduction

8

sector, academic institutes, and training institutions. Many of these actors are located in countries and in regional agencies and this is where the new Sourcebook seeks to advance the knowledge of current practice. In addition to cases that stemmed from the MLI programs, cases were sought from a variety of organizations that use MfDR in their development work. These cases help to broaden the scope of the discussion by adding important insights into how the private sector manages for results and how civil society views the emerging issues. This broader approach demonstrates the importance of information sharing among a variety of groups and organizations, and highlights the lessons learned from implementing MfDR that transcends the usual institutional actors.

Criteria for Case Selection

The Review Panel used the following criteria for inclusion in the print edition of the Second Edition Sourcebook:

Strength and content: Is this a good example of MfDR? Does it contribute to current knowledge? Does it introduce new ideas and demonstrate any new approaches?

Credibility and prospects: Are there realistic prospects for further implementation of the results framework (at the national, sectoral, or program/project level)? Is there evidence that the approach is replicable elsewhere?

Layering: Does the case build on or complement cases presented in the First Edition of the Sourcebook? Does it offer regional or contextual variety?

Format and target audience: Does it respond to the needs of one of the target audiences?

Problem-solving: Does the case show good evidence of identifying problems and presenting solutions?


9

What is Managing for Development Results?

Managing for Development Results (MfDR) is a management strategy that focuses on using information on results to improve decision-making. MfDR involves using practical tools for strategic planning, risk management, progress monitoring, and outcome evaluation. Today’s results agenda has its roots in the Millennium Development Goals. When the international community agreed to focus on addressing seven specific aspects of poverty, the inevitable question arose: How will we know we have succeeded? We are now half way to the deadline set for achieving the Millennium Development Goals. Are we making progress? Are countries and donors getting results? Can foreign aid deliver? Are governments responding to their citizens’ priorities and demonstrating how they are spending their tax dollars? The need for greater effectiveness and accountability in managing for development results is changing the way partner countries and development agencies work together. Citizens in both donor and partner countries are demanding better performance of development interventions. The concept of MfDR draws on the theories of development change and change management. At its core are notions of:

Goal-oriented: setting clear goals and results provides targets for change, and opportunities to assess whether change has occurred. Causality (or “results chain”): various inputs and activities leading logically to outputs, outcomes, and impact. Continuous improvement: periodically measuring results provides the basis for adjustment (tactical and strategic shift) to keep programs on track and maximize their outcomes.

Source: Monitoring & Evaluation Theme Paper: Third International Roundtable on MfDR, Vietnam, 2007.

Introduction

10

MfDR can be viewed through three main lenses: 1) the country/macro level; 2) the specific sector or project/program (micro) level; and 3) as a management challenge at the organizational level.

Country-level results management requires sound planning systems, strategic assistance plans and effective budgeting that reinforces the execution of agreed plans. Managing for Development Results at the project/program level starts with a shift at the concept phase. Rather than focusing on amounts of inputs, project or program managers start by thinking about the ultimate goals and objectives of the development project/program, and identify the means that will take them there. Focusing and reporting on just inputs and outputs (as is still used in many places) is a limiting approach. Thus, practitioners need to establish baselines (quantitative descriptions of the status quo), articulate clear objectives (why undertake the project/program? what does it hope to achieve?), and spell out monitorable indicators (practical ways to measure progress and performance). The project/program is then managed within a structure that is flexible enough to allow for changes and adjustments as the project/program evolves.

At the overall management organizational level, the MfDR approach represents a fundamental shift at the policy level, especially in the nature of thinking, acting, and managing within the public sector about what needs to be done to gain the benefits. This in turn has implications for many other aspects of management, for example, on existing accountability and reward mechanisms, as well as the manner in which government relates to its citizens. The macro components of MfDR involve strong leadership by a country’s political leadership and senior management that act as a catalyst for culture change. They create incentives that leads to a change in organizational and cultural behavior.


11

Managing for Development Results also looks at issues concerning information supply and demand. For instance, how much statistical information/data does a country, organization, or project have available? Are decision-makers using information on the project for decision-making? Is the information from the project /program/department/ fed back into decision about project /programs/departments? Is there an interest in the information? How is it being used?

Key MfDR Terms

Additional information on key MfDR concepts and principles is provided in the first chapter of the Sourcebook’s First Edition. To arrive at a common understanding, the following are definitions of the key terms that are used in the discussion of MfDR and are found throughout the cases. Some terms have similar meanings, while others represent shades of difference or nuance. For example: Managing for Development Results, Results-Based Management, outcome-oriented measurement, and development impact measurement -- to some practitioners these four terms are synonyms, but to others they represent a difference in approach.

Inputs: The financial, human, and material resources used for the development intervention (the number and dollars volume of loans, the number of staff working in a ministry on a particular program or project, the number of textbooks delivered to schools).

Outputs: the physical outputs produced by the development intervention (the kilometers of all- weather paved roads built, the number of power plants constructed, the number of health clinics built and quantities of medicines distributed).

Effects: Intended or unintended change due directly or indirectly to the development intervention, (did extending micro-credit to start-up entrepreneurs lead to growth of firms over the short term, and also over the long term?).

Introduction

12

Outcomes: The likely or achieved short-term and medium-term effects of an intervention’s outputs, (is the person’s quality of life improved? Is there a reduction in poverty as a result of the intervention, is there a reduction in carbon emissions?).

Development impact: The broader outcome of the program over a longer term. Are people’s lives improved? Is public health improved as a result of the intervention (e.g. the availability of drugs to combat HIV/AIDS; is asthma in a country (or in a particular region/city) reduced as result of the reduction in carbon emissions?).

Logical framework: Management tool used to improve the design of interventions, most often at the project level. It involves identifying strategic elements (inputs, outputs, outcomes, impact) and their causal relationships, indicators, and the assumptions or risks that may influence success and failure. It thus facilitates planning, execution and evaluation of a development intervention.

Results: The outcomes and impact of actively managing for results over a wide range of development interventions.

Results chain: an intervention that stipulates the necessary sequence to achieve desired objectives beginning with inputs, moving through activities and outputs, and culminating in outcomes, impacts, and regular feedback to the responsible officials and civil society stakeholders.

Results framework: The program logic that explains how the development objective is to be achieved, including causal relationships and underlying assumptions.

Results-Based Management: A management strategy focusing on performance achievements by use of key performance indicators and design/implementation of effective monitoring and evaluation systems.


13

MfDR in Action

The gains made in recent years in managing for results more effectively show that the process is diverse, adaptive, creative, and inclusive. MfDR as an approach to development management is evolving rapidly, as practitioners learn by doing. MfDR comprises both what countries are doing to manage toward outcomes, and what development agencies are doing to measure and monitor whether the resources that they allocate to poverty reduction and economic growth are making a difference. At the recipient country level, public sector institutions (ministries, line agencies, etc) and development agencies are using MfDR strategies to plan and measure the effectiveness of their organizations. The theories of development change and management change reinforce each other in this process. Development practitioners are applying MfDR at many levels and in many contexts: for example, managers and staff in national public sector ministries, international development institutions, and bilateral donors are all experimenting with various approaches, while adding to both conceptual and practical knowledge about MfDR. Development agency and public sector managers are the generators of knowledge about the implications of MfDR and its practical application within their areas of work. These experiences are documented in the Sourcebook examples. Experience with the first edition Sourcebook shows that it is useful to look at emerging practices built around five themes: Leadership and Accountability; Evaluation and Monitoring; Mutual Accountability and Partnerships; Planning and Budgeting; and Statistics. These themes constitute the core elements of building country capacity to manage for results.

Leadership: The China, Vietnam, Mozambique, and Lima Alerta Miraflores Peru cases demonstrate where effective leadership has been shown to have made the key difference.

Introduction

14

Evaluation and Monitoring: The Malaysia, Haiti, and Benin cases show the need to strengthen the linkages between the Poverty Reduction Strategy (PRS) papers, budget formulation/implementation, and results assessment systems.

Mutual Accountability: The Mozambique, Mongolia, Chile, and Indonesia tsunami cases show that to achieve effective public sector improvement, it is necessary to develop group accountability and not just individual accountability, by providing incentives to reward efficiency and subjecting systems to external scrutiny. It is also important to strengthen the dialogue with civil society.

Planning and Budgeting: The China, Vietnam, Malaysia, and Benin cases show how important it is to have budgeting systems that reinforce the national and subnational planning systems and that guide resources to where the greatest needs are.

Statistics: The Ecuador, Burkina Faso, and Rwanda cases show the urgent need to strengthen country capacity to improve data in all central agencies (Ministries of Finance, National Planning Commissions) on performance and outcomes ‘across the board.’ One cannot assess results if one cannot measure them -- this holds true in all cases.

Promoting Country to Country Learning & Sharing Best Practice

One of the main takeaways from the Third International Roundtable on Managing for Development Results, was the tremendous thirst of sharing best practice and common experiences. Countries, organizations, development partners, are learning MfDR techniques and best practice as they go. The ability to provide a sense of difference experiences while implementing MfDR is found to be extremely valuable. These cases offer examples of development projects and programs that are implementing MfDR tools and techniques. They do not represent a


15

how – to guide, but rather help to bring awareness to the broad reach and applicability of managing for development results. Additional background materials for each case can be found on the Sourcebook website: www. mfdr.com/Sourcebook/html.

"A strategy without metrics is just a wish, and metrics that are not aligned with strategy

are a waste of time."

- Emory Powell, Texas Instruments

Introduction

16


17


for Political Decision Makers and Senior Management

Political decision-makers and senior management have much to gain from MfDR. First, these groups set the agenda for public sector management; they provide the impetus for cultural change and for decision-making based on results. The Asian Development Bank explains: “the role of leadership is essential not only to set the direction for and consistently contribute to clarifying the core functions and objectives of the organization but also to model the behavior and attitude to support results orientation”1. Second, MfDR offers direct benefits to political decision-makers as a “means of demonstrating how they are addressing the needs and concerns of their citizens. In this way, it can represent a powerful tool to increase the credibility of political institutions with corresponding political gains in some jurisdictions.”2 MfDR can assist government leaders in conceptualizing and communicating both what they are going to do and how they are going to do it. Effective reporting holds leaders to account for what they achieve, supports good government, and provides a process through which they can ensure they are still on track and alter things if necessary.3 At the Third International Roundtable on Managing for Development Results, the theme paper on Leadership and 1 Asian Development Bank. An Introduction to Results Management. 2006. 2 Perrin, Bruce. Moving from Outputs to Outcomes: Practical Advice from Governments Around the World. Based upon the Roundtable “Moving from Outputs to Outcomes: Implications for Public Sector Management” December 15–16, 2004. Sponsored by The World Bank and the IBM Center for The Business of Government. 3 Third International Roundtable on Managing for Development Results. Theme Paper: Leadership & Accountability: Creating a Culture of Results. 2007.

II

Political Decision Makers & Senior Management

18

Accountability stressed: “National leaders face some key challenges in taking an MfDR agenda forward. Successful application is based on an ability to create a leadership and management culture that is focused on results and not just on compliance, or on implementing new administrative and operational systems. Viewed simplistically, the “Public Sector” is an administrative bureaucratic culture that emphasizes the management and measurement of inputs, activities, and outputs. A results-oriented culture, however, focuses on the achievement, monitoring, and reporting of outcomes. The former requires public managers and leaders to be familiar with and apply appropriate laws, hierarchies, regulations, and procedures; the latter requires managers and leaders to diagnose problems, design solutions, take risks, and develop adaptive implementation approaches. Successful leadership involves an appropriate combination of new and old ways of operating to ensure that effective administrative accountability is combined with results-focused innovation and mobilization of stakeholders and resources.” This section provides examples of emerging good practices of political decision-makers and senior management in managing for development results. The cases highlight:

Why MfDR is important for political decision makers and senior management

How results feed back into the decision-making process

How general policy objectives and strategies translate into a results framework at the outcome level

The cases illustrate real-world examples of leadership – where political decision makers and senior management demonstrated a strong commitment to MfDR that led to a change in how their countries, organizations, and institutions do business.


19

For example, China is applying monitoring and evaluation tools in the evaluation of their 11th National Five Year Plan for Economic and Social Development. By using these techniques, Chinese leaders are showing a new commitment and interest in tracking and reporting on the results of their plans, paving the way for accountability and transparency. Similarly, Vietnam’s Socio-economic Development Plan, which has been adopted at the highest levels of national government and is now being implemented in the poorer provinces and regions outside of the main cities, uses monitoring and evaluation to document and demonstrate results. Major pieces of analytic work commissioned under the development plan, such as improving Vietnam’s trade regime for entry into the World Trade Organization (WTO), played a key role in tapping the energy and drive of the private sector with the rigor of strong central planning. Political commitment to managing for development results is found beyond emerging economic powerhouses. Despite extremely difficult development challenges, Haiti, through strong domestic and international political will, has developed a robust results framework for the country’s Interim Cooperation Framework. The commitment of political decision-makers to results is not limited to the national sector. In Miraflores, a municipality outside Lima, Peru, the Mayor uses Results technology to make important decisions about citizen safety. Political decision-makers and senior management are also managing for results through their engagement with civil society. In Mozambique, political decision-makers recognized that dialogue with civil society on development results needed strengthening. A Poverty Observatory was developed, coordinating the process of monitoring and evaluation of all poverty reduction activities and allowing for consultation with civil society in the national development processes.

Political Decision Makers & Senior Management

20

Senior management and political decision makers are important, if not crucial catalysts in by expressing their demand for information on results. The cases in this section demonstrate that by demanding information on results, political and senior officials encourage better documentation, reporting, and dissemination of information on results that matter to citizens, thus unleashing a virtuous circle of accountability and transparency. The subsequent chapters shows that in addition, successful implementation of MfDR also requires commitment and involvement of technical practitioners, the civil society, and the private sector.


21

Chart 1. Po verty Headco unt Ind ex

0

5

10

15

20

25

30

35

1990 2000 2003 2006

China: Towards Results-Based Strategic Planning Mei Wang, (Senior Economist, World Bank); Xu Lin, (Director-General,

National Development and Reform Commission, People’s Republic of China)

China has achieved spectacular growth and social progress (the poverty headcount index declined from 32 percent to eight percent between 1990 and 2006).

However, leaders face a set of new developmental challenges after decades of GDP-centered economic growth, including a steady increase in income inequality.

At all levels, policy makers realize that it is imperative to introduce participation, transparency and accountability into government work. This ensures the successful implementation of the new results-based approach to planning and the delivery of sustainable and more equitable development.

The Eleventh Five-Year Plan contains comprehensive output and outcome indicators, which are monitored and evaluated closely. The population is well informed through the media and public consultations, which is an unprecedented development in China.

Introduction:

China’s economic achievement in the last quarter of a century (1978-2006) is well-known. China’s GDP has been growing at about ten percent per year and its per capita income has increased six-fold since 1978. About 500 million people have been lifted out of poverty at the dollar-a-day expenditure level (WDI, 2006). The poverty headcount index (the percentage of population living in households with income per person below the poverty line) decreased from 32 percent to eight percent between 1990 and 2006 (Chart 1).

Source: World Bank, 2006

China

22

However, as the Chinese government is well aware, China is now facing a set of new developmental challenges after decades of GDP-centered economic growth. The country’s income inequality has been increasing steadily, the Gini Index is now above 40 (Chart 2),1 and the demand for natural resources has also caused increasing concern over the sustainability of China’s economic development. In 2004, China’s GDP accounted for only four percent of global GDP, but its share of global energy usage was three times larger, and it accounted for 28 percent of the world’s rolled steel consumption and 50 percent of its cement use (Stiglitz, 2005).

In late 2003, to address this new and more difficult set of challenges, the Chinese government decided to adopt a new development strategy: “the scientific concept of development.” This plan calls for a comprehensive, coordinated and sustainable people-centered strategy to promote overall harmonious development of the economy, society and its people. Under this new concept, the country will focus on achieving “five balances:” balances between rural and urban development, inland and coastal development, social and economic development, nature and man

1 The upper line is without adjustments for spatial cost of living differences and the lower line is with adjustments.

Chart 2. Gini Index45.3

41.1

20

30

40

50

1980 1985 1990 1995 2000 2005

Source: Ravallion and Chen (2004) World Bank, from NBS 2003 Rural/Urban Surveys Household Surveys


23

development, and domestic and international development. In short, the country seeks to grow in a more sustainable and equitable way. This new approach, which is more complex than the previous GDP-centered strategies, relies much more on the efficiency and effectiveness of the government. National leaders realized that it is imperative to introduce participation, transparency and accountability into government work to ensure the successful implementation of the new development concept and the delivery of sustainable and more equitable development. In March 2004, for example, Premier Wen Jiabao highlighted that the country must “speed up the formation and improvement of the systems for making collective decisions on major issues, for soliciting opinions from experts, for keeping the public informed and for holding public hearings, and for accountability in policy-making.”2 In addition, the 2003 SARs crisis heightened the demand for information transparency and accountability within the government. Two high-ranking officials were held accountable and fired for hiding jurisdiction information and their inability to keep the pandemic under control. Premier Wen Jiabao has since repeatedly called on government departments at various levels to establish accountability systems and undertake monitoring and evaluation (M&E) activities (Wen, 2004, 2005, 2006). In 2005 for example, Premier Wen pledged to “increase the transparency of government work and boost popular confidence in government…” and “…establish a scientific system for evaluating government performance and a system for comprehensively evaluating economic and social development.”3

2 “Annual Report on the Work of the Government” presented to the National People’s Congress, March 2004, 3 Ibid, Report of March 2005.

China

24

Application:

The National Five-Year Plan for Economic and Social Development (FYP) is a key strategy document of the Chinese government, which sets the development priorities of the country in the five-year period covered. Since the establishment of the Peoples’ Republic in 1949, China has formulated and implemented ten National FYPs and is now implementing the Eleventh FYP (2006-2010). Mid-Term Review of the Tenth Five-Year Plan Until 2003, China did not monitor or evaluate activities conducted as part of the Five-Year Plans. The Strategic Planning Department (SPD) of the National Development and Reform Commission (NDRC), the agency tasked with preparing the Plans, carried out a mid-term review for the Tenth FYP (2001-2005). The review was a comprehensive exercise, involving every province and Ministry, and opinions from 100 experts and 26 staff of the SPD were collected using a questionnaire. The report reviewed the implementation of the Tenth FYP in nine categories, such as macro-regulatory targets and environment issues, and identified five major problems, including China’s unhealthy pattern of economic growth.. These findings initiated discussions on China’s need to employ a new development approach and contributed to the new people-centered development concept. While the review was informal and experimental in scope, it represented a significant change in the mindset of top Chinese policy makers. For the first time in history, Chinese leaders examined and publicized both successful and unsuccessful aspects of the FYP and incorporated the results and recommendations of the review in subsequent work. Monitoring and Evaluation of the Eleventh Five-Year Plan In light of the new developmental approach adopted by the government, the Eleventh FYP was prepared with a more


25

“scientific concept of development” and “five balances” as the guiding principles, to ensure that the economic growth is consistent with environment and natural resource constraints. Also, to spread the benefits of growth more broadly in order to create a harmonious society. The Plan was prepared with participation from various levels of government representatives, National People’s Congress (NPC) deputies, scientific and economic experts as well as ordinary Chinese citizens. Eighty individuals were given a reward for their suggestions, including a 12 year-old migrant child who asked for compensation for the unfair treatment she received at school. Some 34 revisions were made to the FYP according to NPC representatives’ suggestions, and high quality ministry and provincial-level inputs were also important additions. The comprehensive nature of the Eleventh FYP and the society’s involvement in developing the strategy has compelled the NDRC to undertake serious and formalized monitoring and evaluation of the activities outlined in the Plan. Based on the mid-term review of the Tenth FYP, the NDRC decided to develop an M&E framework for the Eleventh FYP and to establish an M&E methodology and institutionalize arrangements for future Plans. To benefit from international experience in this area, NDRC commissioned a World Bank Institutional Development Fund project to support the effort.4 There are two components to the new M&E framework: First, an indicator matrix was introduced for the first time in the development of the Eleventh FYP. It contains 22 indicators with baseline information and targets. The introduction of this type of matrix was a significant improvement; in previous Plans the indicators were embedded in the text without baseline information. Among the 22 indicators, eight are defined as obligatory (such as reduction of energy consumption per unit GDP, total cultivated

4 See “China: Monitoring and Evaluation of the Development Progress,” June 2004. TF No. 053678.

China

26

land maintained, and reduction of emission volume of major pollutants), which the government is openly committed to meet. The government has agreed to try to create a favorable environment to help achieve the anticipated benchmarks of the other 14 indicators, however reaching the identified targets will rely primarily on market forces. The 22 indicators are grouped into four categories: economic growth; economic structure; population, resources and environment; and public services and life quality. Six indicators target economic growth and structure and 16 indicators focus on environment, resources and social development. Policymakers have committed to carry out a mid-course evaluation of the Plan, to be reviewed by the NPC Standing Committee. Second, the NDRC has started to build an overall M&E Framework for the Eleventh Plan, which is consistent with the 22 indicator matrix, but broader in scope with different input and outcome indicator matrixes. The framework will be used in the mid-course evaluation of the Eleventh Plan in mid-2008. The development of the M&E Framework followed an internationally recognized “Ten Steps” model (Jusek and Rist, 2004) and was jointly created by a team of international and domestic experts. Among other activities, the team reviewed the international experiences of the OECD and various developing countries and convened consultative meetings with both international and domestic experts. The model is based on a sequence of ten steps or activities that are considered critical for establishing a results-based M&E framework. For example, the readiness assessment, articulated by the first step of the model, revealed that a favorable political environment and a well-positioned champion, the SPD of the NDRC, are factors that positively impact the Chinese M&E effort. However, China’s current deficient statistical system, weak M&E knowledge and capacity, and disconnect between planning and budgeting constitute bottlenecks. The team’s analysis recommended that China should not to rush into development of a technically


27

comprehensive M&E system. A basic, workable M&E framework, which can be strengthened over time as the constraining factors are improved, should be the priority.

Table 1. Illustration of the Output/Outcome Indicators of China’s 11th FYP M&E Framework

Five Balances

Goals Output / outcome

indicators

2005 Baseline

2010 Targets /

benchmarks

Data Source

Regularity of Data

Collection Urbanization ratio

43% 47% Pop. Census; 1% Pop. Sample Survey

Every 5 yrs. Promoting urbanization

No. of migrant workers converted into resid.

12,600,000 45,000,000 Rural hhd survey and agri. census

RHS: yearly; AC: every 10 yrs.

Urban/rural income ratio

3.22 Tbd Rural and urban hhd survey

Yearly

Urban and Rural Balance

Reducing urban/rural income gap Growth rate

of rural per capita

6.2 Tbd Rural hhd survey

Yearly

Source: A Monitoring and Evaluation Framework for China's Five-Year Plans. IDF Project Report (draft), October 2006

Table 1 presents an example of the output/outcome indicator matrix. A set of goals was selected for each of the “Five Balances,” and in turn, a set of indicators was established for each of the goals. Nearly 70 candidate indicators were selected, and the final indicators will be chosen based on factors such as technical suitability, availability of statistical information, and responsible agencies. Relevant ministry-level targets/benchmarks and consultation with experts will influence the selection process for the 2010 targets/benchmarks.

China

28

Table 2. Illustration of the Input Indicators of China’s 11th FYP M&E Framework

Input Indicators Five Balances

Goals Gov’t policies and regulations

Gov’t Expenditures

Allocation of human resources

2005 Baseline

2006 to 2010

Promoting urbanization

e.g., permitting enrollment of migrant workers’ children in urban schools

e.g., central gov’t’s expenditure composition and distribution

Personnel quota

Tbd Tbd Urban and Rural Balance

Reducing urban/rural income gap

e.g., abolishing compulsory education fee in rural areas

e.g., central gov’t’s transfer payment to cover the compulsory education cost in rural areas

Tbd Tbd

Source: A Monitoring and Evaluation Framework for China's Five-Year Plans. IDF Project Report (draft), October 2006

Table 2 presents an illustration of the input indicator matrix. For urban and rural balance, for example, two goals were identified and three sets of input indicators were chosen for each goal (government policies and regulations, government expenditures and human resource allocations). This new matrix will assist leaders to better understand the logic behind FYP successes and failures. It also aims to increase the government’s awareness of the important relationship between planning and budgeting. The information for the 2006-2010 FYP will be collected using the data from the 2005 baseline, which the government is currently in the process of identifying.


29

Problem Solving: Measures Introduced to Ensure Implementation

The transition outlined by the Eleventh FYP from GDP-centered growth to a more balanced development approach is a daunting task for China. A major challenge for implementation is how to ensure that different public agencies, especially local governments, will act accordingly. To address the challenge, the central government is focusing on the obligatory indicators and has introduced a series of measures to provide incentives for implementation.5 First, the State Council has assigned responsibility to government agencies and departments for each of the eight indicators. For example, the NDRC is responsible for the overall reduction of the energy consumption per unit of GDP and the State Environment Protection Administration is in charge of the overall reduction of total major pollutants emission volume. Table 3. Illustration of Provincial Assignment: Reduction of Energy

Consumption per Unit GDP 2005 Baseline 2010 Target Region Tons of coal / 1,000 Yuan

Tons of coal / 1,000 Yuan

% of Reduction

Nationwide 12.2 9.8 20% Beijing 8 6.4 20% Jilin 16.5 11.6 30% Fujian 9.4 7.9 16% Gansu 22.6 18.1 20% Source: See Footnote 3

Second, the targets for the reduction of energy consumption per unit of GDP and reduction of total major pollutants emission volume have been divided and assigned to all 27 provinces and

5 Notice on Implementation of Major Objectives of The Eleventh Five-Year Plan for National Economic and Social Development of the People’s Republic of China and Task Assignments by State Council. Document 2006. No. 29.

China

30

four provincial-level municipalities (see Table 3 and 4 for illustration)6 based on criteria such as level of economic development, industry structure, and total energy consumption. These targets have been further divided and assigned to lower levels of government and the targets help track expenditures of priority enterprises.

Table 4. Illustration of Provincial Assignment: Reduction of COD and SO2 Emission Volume

2005 Emission Volume

2010 Target % of Reduction Region

COD SO2 COD SO2 COD SO2 Nationwide 14142 25494 12639 22467 10.6% 11.9% Beijing 116 191 99 152 14.7% 20.4% Jilin 407 382 365 364 10.3% 4.7% Fujian 394 461 375 424 4.8% 8.0% Gansu 182 563 168 563 7.7% 0.0% Source: See Footnote 3

Third, a public reporting system has been established for the proportion of cultivated land, reduction of energy consumption per unit GDP, and reduction of total major pollutants emission volume—the three most important obligatory indicators for the government. The monitoring information on these indicators, nationwide and for each province will be reported annually. Fourth, the central government asked the Ministry of Personnel to incorporate all the obligatory indicators into the performance evaluation system for provinces and ministries and required that the chief officials of all the provinces use the three most important obligatory indicators.

6 The assignment for the total cultivated land is under discussion among the relevant ministries and respective provinces.


31

Results:

The introduction of the output/outcome indicators with targets and implementation measures adopted by the central government has helped to strengthen the accountability and improve the transparency of the government’s work. Already, the monitoring information on the energy consumption per unit GDP and the total COD and SO2 emission volume during the first six months of the Eleventh FYP has been made public: none of the levels have decreased as outlined in the Plan, instead amounts have increased due to economic growth. Such a level of transparency on Plan implementation is historically unprecedented in China. The publicity has stimulated open discussions among policy makers, experts and the media on causes of the initial failures and measures required to move forward. The provincial governments, guided by signals and incentives from the central government, have also started to take concrete, comprehensive measures to ensure implementation. Box 1 presents a small set of the overall measures introduced by Zhejiang Province on energy saving and environment protection. The province has also developed a new set of performance criteria (in a public opinion poll) for evaluating high-ranking officials, which includes questions on environment, energy and support to the poor.

China

32

Box 1. Zhejiang Province: Example Measures Taken to Ensure Progress on Energy Saving and Environment Protection

One hundred enterprises were chosen to pilot new indicators for clean production, energy and raw material saving, and comprehensive utilization of solid waste. A statistical information system was established to monitor the usage of energy, raw material, water and the omission of major pollutants by these 100 enterprises. By 2007, the reuse rate of water by these enterprises should reach 65% and solid waste 90%. The system will monitor the companies’ efforts to transform the existing industrial parks through ecological transformation such as comprehensive usage of resources, joint land use, concentrated waste treatment and sharing of electricity. By 2007, ten demonstrating ecological industrial parks should be established. In those parks, more than 70% of the enterprises should pass the cleaning production exam or be accepted by the ISO14000 environment management system. The wastewater treatment rate should reach 80%. Companies will also be rated on their ability to establish a group of green architectures and communities. By 2007, 50% of new commercial buildings should reach the standard for energy saving, such as using energy-saving walls, windows, heating and air conditioning systems and utilizing solar energies. The green communities should set up facilities to help comprehensively utilize treated wastewater and rain and river water and to recycle the garbage. In addition, the heavy energy- and water-consumption enterprises will be closely monitored and technical transformation will be expected. Every quarter the responsible government agencies will report progress to the provincial officials in their respective areas.

Conclusion:

China is preparing to meet the developmental challenges resulting from its three-decade long spectacular economic growth by moving towards results-based strategic planning. With the overall goal of seeking a balanced and sustainable development approach, for the first time in history, China has introduced obligatory outcome indicators with targets in its new Eleventh Five-Year Plan and


33

adopted systematic and transparent institutional measures to implement the Plan. These efforts have brought an unprecedented level of transparency to China’s FYP implementation, which is resulting in institutional and policy transformation for the country. China is also working to create its first overall FYP M&E framework and will use the framework to conduct a mid-course evaluation of the Eleventh Plan. The M&E framework, complete with input and outcome indicators, will allow both monitoring of the Plan’s implementation progress and analysis of the potential causes of success and/or failure. International experience suggests that results-based M&E is a powerful tool for ensuring the effectiveness of a country’s development policies and the delivery of tangible results by the government. However, the development and implementation of the framework are difficult and lengthy processes. China’s recent national M&E efforts illustrates how a large country with little M&E experience can establish a results-based strategic planning system based on greater transparency and accountability.

China

34


35

Vietnam: Design/Implementation of Results-Based Planning and Monitoring1

Carolyn Turk and Rob Swinkels (Senior Poverty Economists, Word Bank, Hanoi) and Quang Hong Doan (Senior Economist, World Bank, Hanoi)

Vietnam’s economic and social progress stands out among East Asian countries. The poverty rate has fallen from 58 percent in 1993 to less than 20 percent in 2004.

The country is going through a far-reaching transformation from an inward-planned economy to one that is globalized, market-based and comprehensive.

The transition from a central command approach towards a more market-based economic development plan prompted a growing realization within Government of Vietnam of the need to shift from planning for production targets towards planning for achieving social outcomes

The debate in the Government and National Assembly has shifted from a single focus on the ‘pace ’ of growth to a broader debate on the ‘quality’ of growth. This change reflects the idea that achieving economic growth and developing infrastructure are not ends in themselves, but should be translated into better lives for the people.

the critical next step is to monitor progress towards desired outcomes, using a clear and effective national M&E system.

Introduction:

Since 2001, national planning processes have undergone considerable reform. Traditionally, draft plans were classified, with discussions during the drafting process largely internal to government. Only data from government reporting systems were used to underpin strategic direction or to report on past progress. The past five years have seen a shift to a new approach to planning that is based on consultations with a wide range of stakeholders

1 This case builds on “Vietnam: A Comprehensive Strategy for Growth and Poverty Reduction” MfDR Sourcebook First Edition, page 44.

Vietnam

36

and makes use of independent survey data on poverty and other social outcomes.2 This new approach has helped the government develop a clear focus on results in medium-term development planning and monitoring. This transformation did not occur overnight. It resulted from an extended period of government, donor, and non-profit organization (NGO) collaboration (most of the donor management has been in-country). The process involved incremental steps forward, supported by responsive, flexible donor resources, rather than a one-off package of radical reforms. There are two main dimensions to the reforms discussed in this case: the introduction of evidence-based planning processes, using consultations and high quality data, and the introduction of globally accepted principles of M&E.

Application:

Vietnam traditionally relies on five-year Socioeconomic Development Plans (SEDP) to elaborate its medium-term development strategies, set out development targets, and identify policies to achieve these targets. Until the production of the most recent SEDP 2006-2010, the drafting of the Plan was characterized by limited participation from stakeholders outside government. The planning process itself was confined to Government officials and draft documents were kept confidential. Plans contained too many targets for investment and the production of industrial goods and agricultural commodities, with little emphasis on outcomes. The turning point in moving to an MfDR approach involved substantial planning reform, specifically the preparation and implementation of the Comprehensive Poverty Reduction and Growth Strategy (CPRGS). The Government’s intention to prepare a comprehensive strategy to reduce poverty was first announced in December 1999 at the Consultative Group (CG) meeting with donors. Donors, encouraged by more institutionalized initiatives to

2 See the evolution of results-oriented planning/monitoring in the Vietnam Development Report 2007.


37

introduce Poverty Reduction Strategy Papers (PRSPs), embarked on an extended period of support to reform Vietnam’s planning processes. Vietnam’s first Poverty Reduction Strategy Paper (PRSP) was drafted in Vietnamese by local officials and consultants, but the process included many interactions with donors and NGOs. This collaboration prompted several innovations in the approach to planning such as a consultative process aimed at introducing a results-focus to the strategy. As part of the CPRGS process, the Government embarked on an intensive round of consultations with poor communities that provided feedback on the draft plan – it was the first time that such an exercise had been part of a government planning process. The Government also requested support in strengthening its ability to meet the Millennium Development Goals (MDGs). Collaborative work in a number of thematic areas provided the analytical basis for localizing the MDGs into a set of Vietnam Development Goals (VDGs). These goals and corresponding targets formed the backbone of the CPRGS and its monitoring framework, ensuring a focus on outcomes and serving as a reference point for planning guidelines. The CPRGS represented the first serious effort to move from input-focused, operational planning towards result-based plans formulated in a participatory process. The CPRGS was piloted for implementation in 24 provinces in Vietnam. The roll-out process received strong support by donors and NGOs. Feedback from provincial officials provided strong supporting evidence of the relevance, feasibility and superiority of the new planning approach. The CPRGS roll-out led to changes in regulations linked to the new SEDP 2006-2010. These include the Prime Minister’s Directive 33, instructing ministries and provinces on how to prepare the SEDP 2006-2010, and subsequent guidelines issued by the Ministry of Planning and Investment that provided for widespread consultations with people outside government and focused on achieving social outcomes as expressed in the VDGs. Additional impetus towards a results-based approach has come from public administration reforms,

Vietnam

38

increased participation of persons outside the traditional structures in policy-making processes, the revised budget law (2002) requiring greater transparency of budget information, and use of experimental Medium Term Expenditure Frameworks (MTEFs). The SEDP was approved by the National Assembly in June 2006. Introducing a focus on results in the SEDP 2006-2010 included: Strengthening the evidence base - While the previous SEDPs were strictly confidential, participatory consultations on the draft SEDP 2006-2010 brought in stakeholders from inside and outside the Government, both at the national and sub-national levels. Consultations included workshops with officials from local Government agencies, the domestic private sector, local associations, international NGOs, local NGOs, development partners, and researchers. The views of women, communities, and grassroots officials were solicited through intensive research at the community level. At the request of the MPI, local and international NGOs facilitated some of the consultations. The MPI collaborated with these partners to develop a common consultation framework based on the draft SEDP and NGO experience in poor communities. Feedback from poor communities in participatory research exercises in 17 sites across the country revealed that the consultations made a real difference in the content and relevance of the Plan. Recognizing the weaknesses of internal reporting processes, planners concluded that they needed independent, reliable data on poverty and other social outcomes for planning. The SEDP 2006-2010 includes diagnostic work on key challenges. In addition to the traditional and sometimes unreliable data on poverty and other social outcomes collected through reporting mechanisms of line ministries, planners have used high-quality survey data produced by the General Statistics Office (GSO) - the product of ten years of collaborative work between donors and GSO to put in place high quality livings standard surveys - to describe the challenges of


39

reducing poverty. Research institutes also produced studies using GSO data at the request of MPI. Focusing on results - the transition from a central command approach towards a more market-based economic development plan prompted a growing realization within government of the need to shift from planning for production targets towards planning for achieving social outcomes. This transition resulted partly from the work on the VDGs; the guidelines for preparing the SEDP required the VDGs to be central to the SEDP so as to ensure a strong results-orientation. Consequently, the SEDP 2006-2010 is much more balanced between economic growth and social development objectives. Some initiative for innovation came from line ministries. For example, when the Ministry of Agriculture and Rural Development (MARD) started preparing its new five-year sector plan in 2005, it drew upon its previous experience with the CPRGS process. Under the leadership of the MARD Minister, donors collaborated intensively with MARD staff and produced log frames and results-based monitoring frameworks for their five-year plan. The results were presented at seminars that generated strong interest from other ministries. Later in 2005, MPI linked the draft SEDP 2006-2010 to a results-based monitoring framework and requested help and training from donors for this task. Donors responded jointly, providing considerable input into this process.

Problem Solving: Challenges and Solutions

Earlier Government Plans consisted of long lists of general targets, with little structure showing how overall goals translated into groups of specific objectives. While many indicators were used, they were often not linked to any of the plan objectives. As a solution, the new monitoring framework breaks with this tradition and links the main overall goals to a set of clear and specific objectives. Each of these objectives is linked to input/activity,

Vietnam

40

output and outcome indicators. As such, it presents a ‘results chain’ (Table 1), providing a solid basis for monitoring the results of the SEDP 2006-2010. The monitoring framework covers four development pillars: economic growth, social progress, the environment, and governance. Table 1: Example of a results chain for one of the SEDP objectives

Changes under the control of government Changes outside the control of government

Objective Activity/ input Output Outcome/ Impact 2.2.2 Ensuring access to health care services for all

Universalise coverage of health insurance

Proportion of people with health insurance; proportion of the poor and near poor with free health insurance

Proportion of people reporting satisfaction with service obtained with health insurance, by ethnicity and quintile

Source: Monitoring framework of the SEDP 2006-2010

Lack of incentives and capacity were two of the most important challenges in applying the new approach to SEDP planning process. Planners in line departments and provinces were unsure about how far they could go in re-orienting their plans without jeopardizing the prospects of plan approval, and they were concerned that submission of plans that were radically different from earlier formats might result in less funding. In a system in which planners are accustomed to following planning instructions issued by higher-level authorities, unambiguous, central-level commitment to reform planning processes and incentive structures is extremely important. To emphasize the determination of the Government to adopt a result-based approach in the SEDP 2006-2010, the Prime Minister issued the Directive 33, and the MPI Minister published the follow-up Circular 7681. These ordinances explicitly required planners to focus on the results orientation of the CPRGS by incorporating the VDGs as central components of the SEDP. In developing the monitoring framework, the MPI used internationally accepted M&E principles and concepts to produce a results framework that goes beyond the VDGs.


41

Adopting a stronger outcome focus demanded skills not used in the past input-oriented plans. Part of the solution was to intensify the training programs offered to officials in result-based participatory planning, design of M&E systems, and reporting. The CPRGS involved several training courses for MPI staff, other line ministries and staff in the provinces participating in the roll-out process (supplemented by training sessions outside the capital on results-based M&E for planners at various levels). Participants were trained on how to improve the structure of plans, define clear objectives, and link these objectives to indicators. A planning manual was developed to provide basic guidance on the application of key principles in the results-based approach to planning. Collaboration with actors outside government was important in introducing participatory forms of training and in transferring new skills to government officers. Intensive collaboration with NGOs provided specific expertise in arranging community consultations. The MPI worked alongside NGOs in designing the research framework for the consultations, implementing the fieldwork, and analyzing the findings.

Results:

Changes in mindset - In recent years, the debate in the Government and National Assembly has moved away from a single focus on the ‘quantity’ of growth to a broader debate on the ‘quality’ of growth. This change reflects the idea that achieving economic growth and developing infrastructure are not ends in themselves, but should be translated into better lives for the people. Changes in content of the SEDP – The evidence used to underpin the strategic direction of the SEDP has delivered important improvements in its scope and content. The consultations played an important role in shaping the final SEDP 2006-2010. The influence on the final document ranges from the expansion of some sections (gender and children) to structural

Vietnam

42

shifts in emphasis (mainstreaming of poverty issues beyond targeted programs), to language modifications (especially regarding activities previously described as ‘social evils’). OXFAM acknowledged the role of community consultations in improving the quality of the SEDP (World Bank, 2006). The use of high quality quantitative and qualitative data in the analysis has encouraged the SEDP to recognize a number of marginalized groups that have not benefited from the country’s economic growth. Earlier analysis had been limited to official data and was less effective at describing the deprivations of groups such as ethnic minorities, people living with disabilities and rural-urban migrants who face difficulties in accessing social and administrative services on grounds of their registration status. As a result, there is a consistent approach throughout the SEDP to reduce poverty by including the poorest in economic growth plans and reaching disadvantaged groups through the mainstream delivery of public services. Changes in approaches to M&E - The result-based M&E framework is structured around the four pillars of the SEDP and has notable strengths. First, the framework includes a substantial pillar to measure progress in the promotion of good governance. This development represents important progress, since previous attempts to systematically measure the results from governance reforms were weak. Though some governance indicators were included in the earlier framework such as monitoring of the promotion of grassroots democracy, progress was typically measured through self-reporting by communes and therefore lacked independent verification. The governance pillar commits the Government to measuring progress in improving transparency and participation, strengthening decentralization, and managing public resources. The MPI showed strong political commitment to the development of a monitoring framework for the SEDP 2006-2010 that applies sound principles of results-based M&E. Much of the information will must be provided or verified through independent


43

surveys, a shift from the previous reliance on administrative systems for measuring and reporting results. Donors supporting the Poverty Reduction Support Credit (PRSC) V (the European Community, Australia, Denmark, the Netherlands, and Ireland) have all stressed the importance of a harmonized, well-coordinated approach. This joint approach has helped to sharpen the focus on M&E, as measuring the outcomes and impact of development programs is a priority of the donors and Government alike. The PRSC V results framework consists of clear Program Development Objectives (PDO) and comprehensive outcome indicators. Policy measures cover the following: (i) Completing the transition to a market economy through actions to further trade integration, strengthen the equity/capital base of State Owned Enterprises, accelerate banking reform, and improve the investment climate; (ii) ensure social inclusion and sustainable development through actions to improve the financing of healthcare for the poor, quality of education for disadvantaged groups, and protection of the environment; and (iii) build modern governance by adopting a planning process focused on development outcomes, strengthening the transparency and management of public finances, simplifying procedures for obtaining public services, and fighting corruption. More than fifty indicators are used to track actual progress. Progress is satisfactory or highly satisfactory in all areas except for state enterprise reform, which too shows some improvement.

Conclusion:

1. Though the SEDP has earned praise for its open process, improved content and emphasis on desired outcomes, this has not been a swift transformation. Mainstream national planning processes have been evolving since 2000. In addition to political will from high levels of government, it has taken the interim step of a “parallel” plan and sustained, flexible donor

Vietnam

44

support managed from in-country to shift slowly from a closed planning process geared to describing inputs-production targets to a broadly consulted plan that focuses on outcomes. Challenges remain at several levels, including improving the linkages between Plans and budgets by providing more detailed costing and a better match between objectives and inputs. This requires effective cooperation between the MPI, the Ministry of Finance (MoF) and other Ministries and agencies responsible for SEDP implementation. Planning reforms need to cover provincial and sub-national levels, and include institutional changes, in order to strengthen spatial planning.

2. It is critical to continue efforts to improve accountability. Corruption represents one of the most significant risks to growth and sustained poverty reduction over the Plan period. The Government is fully aware of the threat that corruption poses to the achievement of Plan outcomes and is strongly committed to fight against it. In 2006, the Government passed a law on anti-corruption (which accords a role to the media and civil society in ensuring the integrity of government officials) and established a high level steering committee to oversee the implementation of the anti-corruption strategy. The Government plans to ratify the international convention on anti-corruption in early 2007. Three high profile cases of large-scale corruption have been investigated and have brought perpetrators to justice. However, increased decentralization in the allocation and usage of public resources is likely associated with a potential rise in corruption unless the decentralization is accompanied by further improvement in planning and managing public resources, including improved selection and evaluation of public investment and an effective results-based M&E system.

3. Results-based monitoring is a new concept in Vietnam. The use of outcomes for monitoring progress can lead to confusion as government officials do not always understand why they need to measure things that are outside government control.


45

Also, the use of independent household or enterprise surveys for measuring outcomes is a new phenomenon for many officials. It is therefore important that government officials write guidelines for the application of the results-based monitoring framework across ministries and provinces. These should include clear explanations of the monitoring terminology and concepts that are being applied, the definition of each indicator, its data source, and collection responsibilities and frequency. The drafting process requires close collaboration between the MPI and the Statistics Office.

Vietnam

46


47

Haiti: Lessons from the Interim Cooperation Framework (ICF) from 2004 -2006 & the Extended

and Revised ICF from 2006 Onwards Lisa Campeau, (Consultant, Licus/Fragile States Unit, World Bank)

For many years, Haiti has experienced protracted instability and slow economic growth, but the situation is turning around under stronger leadership and a coordinated support program from external donors.

Managing for development results in this situation especially requires committed leadership, broad consultations with stakeholders, public dialogue around results, and strong capacity building initiatives.

Clear evidence of early results is imperative for an MfDR approach to gain momentum and reverse the marked income inequality and violence throughout the country.

The Haiti Interim Cooperation Framework is a good example of a results-based program used to draft a comprehensive plan for aid-management/coordination in a challenging security environment.

Introduction:

Haiti’s deteriorating political and economic situation called for an effective and coordinated approach to reconstruction and development, hence the birth of the county’s results management process, known as the Interim Cooperation Framework (ICF). The sequence of events leading to the results program includes the following steps: United Nations (UN) Security Council Resolution 1529 mandated a rapid assessment to evaluate Haiti’s transitional needs. As Haiti was plagued by prolonged political conflicts and natural disasters, in February 2004, the UN Security Council Resolution 1529 authorized a temporary and immediate deployment of a multinational force to evaluate humanitarian and

Haiti

48

security needs as well as elaborate the UN’s humanitarian action plan for the country. Donors supported the transitional government in launching the ICF, provided certain preconditions were met. Building on UN Security Council Resolution 1529, a joint needs assessment was proposed at an informal donor’s meeting in Washington, DC in March 2004. The objective of the needs assessment was to elaborate a national reconstruction and development program, covering both short and medium-term priorities over a two-year transitional period. At this meeting, the Haitian government and international organizations also agreed that various preconditions would be needed to develop and implement a results-based framework, including security, a stable political environment and sufficient government capacity to carry out the exercise. UN Security Council Resolution 1542 authorized the UN Stabilization Mission in Haiti (MINUSTAH) and suggested the formation of a long-term development strategy. In conjunction with a multidimensional stabilization mission, in April of 2004, the newly appointed transitional government in Haiti launched the ICF. UN Security Council Resolution 1542 authorized a security force primarily deployed to cover the election process, reform the Haitian police and implement a Disarmament, Demobilization and Reintegration (DDR) program. The UN resolution also urged the international community to assist the transitional government in designing a long-term development strategy that would stabilize the country and combat poverty. Government presented the ICF, which covered the two-year transitional period. In July 2004, the transitional government presented the ICF at a donor’s conference in Washington, DC.

Application:

The ICF was inspired by similar exercises and lessons from East Timor, Afghanistan and Iraq and modeled after Liberia’s Joint


49

Needs Assessment and the draft concept note from the Sudanese Joint Assessment Mission. The ICF development process resulted in not only the establishment of key reform targets for the reconstruction and rehabilitation of Haiti, but also strong national and international support for the program as a whole. These results were achieved using the following methods: Thematic groups or sector tables were created for ICF preparation. Stakeholders were divided initially into thematic groups and later converted into sector tables. Each thematic sector table was led by a government representative and included the participation of multilateral and bilateral donors and members of international agencies.

The ICF results-focused matrix was developed. Detailed results matrices were prepared for each thematic group and combined into an overall transitional results matrix. The document summarized specific development targets, estimated costs and outlined the implementation of priority interventions over the transition period. The ICF timeframe also included a six-month overlap following the transitional government’s term to make financial resources available to the newly elected government.

Multi-stakeholder consultation and information dissemination workshops were implemented. Non-governmental organizations (NGOs) and civil society participated in regional workshops and a National Consultation Workshop for ICF preparation. Recommendations were also discussed in the capital and in three other communes throughout Haiti during several workshops organized by the government that included political parties, civil society and local authorities.

Committed leadership by the Prime Minister through chairmanship of A Joint Committee for the Implementation and Monitoring of the ICF with

Haiti

50

participation from other representatives from the government, national stakeholders and the international community.

A Unit for Implementation and Follow-up served as a secretariat for the Joint Committee, providing operational support and overseeing the collection and sharing of information on implementation progress.

Civil society, the private sector, and a national think tank were involved in a consultative committee for oversight and monitoring. The committee’s role was to ensure that programs and projects remained on track and to hold both donors and government accountable to the agreed timetable for results. The private sector was also represented in the committee to monitor implementation of economic governance reforms.

In addition to these various coordination mechanisms, government and donors maintained dialogue through regular donor coordination meetings in Port-au-Prince and abroad.

Problem Solving:

Implementing the 2004-2006 ICF resulted in the following issues, mitigation strategies and lessons learned, some of which are being applied to the revised ICF of 2006/07:

1. Lessons on Institutional Capacity, Project Management and Results Coordination/Reporting

Inadequate quality and quantity of human resources affected ICF implementation. A scarcity of skilled labor and lack of adequate incentives to attract, maintain and motivate civil servants has been a major impediment to public sector efficiency and effective ICF implementation. Medium-term reforms and a longer-term systemic restructuring of the civil service are required to increase the efficiency of scarce human resources.


51

Country offices are necessary for decentralized donor decision-making. Some external partners have strengthened their presence in the country to better participate in day-to-day discussions and monitoring. However, building institutional capacity continues to be a challenge requiring significant financial and technical assistance from both the government and donors.

Institutional capacity building and technical assistance are still required for effective ICF implementation. Ensuring the effective implementation of a results-based plan requires not only a detailed strategy to create the overall framework, but also a commitment to the new system. A capacity building action plan has been drafted, in close collaboration with the Prime Minister’s office, and is currently under discussion. Additionally, although oversight mechanisms exist, they need technical reinforcement.

Multiple Project Implementation Units (PIU) affected capacity building and created overlap. To facilitate project implementation and ensure fiduciary accountability, donors established project implementation units (PIUs), often resulting in parallel project implementation structures. Such structures ultimately hindered institution building, as the government was neither given the means to improve national execution capacities, nor had the experience to coordinate external assistance. While parallel PIUs are still used in most externally financed projects, external partners are taking action to consolidate them and rely on single PIUs to coordinate activities financed by multiple partners. Donors are also looking at creating joint PIUs and additional mechanisms to strengthen capacity within ministries.

Coordination processes and establishment of permanent focal points are needed. Many coordinators

Haiti

52

of oversight and monitoring committees, focal points in the ICF preparatory phase, were not re-appointed, slowing ICF implementation. The production of checklists, a method frequently used to integrate cross-cutting themes, is not enough to ensure that conflict sensitivity is embedded in a results framework and considered in program implementation.

2. Lessons on Data Quality of Output/Outcome

Targets, Monitoring and Reporting

ICF reporting was affected by weak monitoring and evaluation. Although the 2004-2006 ICF included a coordinated process for country-level implementation, monitoring and evaluation processes are still at an early stage. More regular reporting on ICF implementation is needed, with better information on project disbursements relative to agreed targets.

Lack of available data impacted the development of ICF outcome targets and monitoring of actual results. Haiti’s prolonged conflict affected the availability of relevant data required for baseline indicators. The lack of a coherent policy on the use of data for baseline indicators has impacted monitoring and evaluation of ICF implementation. Some data discrepancies may be resolved with the third Demographic Health Survey in 2006. Additionally, the government plans to develop a statistical capacity enhancement strategy.

3. Lessons on Linking Security, Implementation Speed

and Quick Wins

Insufficient linkage existed between ICF preparation and MINUSTAH deployment. MINUSTAH was deployed in May 2004 after thematic groups had drafted the ICF. Although thematic groups included sectors such


53

as Police, DDR, elections, justice and prisons in the framework development process, greater attention to the overall connection with peacekeeping or peace-building issues, with clear development interventions included in the ICF, would have been beneficial. Linkages were ultimately made between the UN peacekeeping mission, MINUSTAH, and the ICF, although only towards the end of the thematic teams’ work.

Immediate implementation results and quick wins are imperative. In Haiti, following the completion of the results framework, it took six to eight months for the establishment of implementation mechanisms, which functioned at various degrees of effectiveness. Attention should have been given to outlining projects or programs where funding could be immediately available for disbursement and where immediate results could be seen within the first quarter.

4. Lessons on Transparency, Accountability,

Communication and Information Dissemination

Greater attention to communication and information dissemination is needed. While there were clear attempts to make the ICF process participatory and consultative, civil society actors felt that local NGOs and other national stakeholders, such as the press, were not adequately involved in ICF thematic groups and post-ICF planning and implementation. Additionally, a communications strategy may have helped to raise awareness about the ICF process and manage expectations with regards to implementation results.

Disseminating budget and procurement information for greater transparency and accountability is necessary. Since 2004, to ensure transparency and stakeholder access to information, the government has

Haiti

54

disseminated information on budget allocations and execution and procurement contracts through the Ministry of Finance’s website.

5. Lessons on Differentiating between Development

Assistance Needs, Budgets and Reporting

Clear distinction between humanitarian needs and development assistance is required. Although humanitarian needs were not specifically budgeted for under the ICF, it was difficult for assessment teams to distinguish between humanitarian and development needs during the assessment. In addition, as Haiti was severely affected by floods immediately following the donor conference, a Flash Appeal was launched in September 2004, diverting attention and focus from the ICF. Eventually an agreement was reached that the Flash Appeal would cover humanitarian assistance, whereas the ICF would cover transitional and medium-term development needs.

Results:

1. ICF Results Some results have been achieved, although increased disbursements are desirable. In addition to creating the conditions for free and fair elections (held in February and April 2006), improving the population’s access to basic services, rehabilitating the deteriorating economy (after years of negative growth), and becoming eligible for debt relief, many visible results can also be cited from Haiti’s ICF process. Results achieved cover police reform, the development of an Interim Poverty Reduction Strategy Paper (PRSP), increased transparency in the use of public resources, provision of electricity, reconstruction of primary and secondary roads across the country, distribution of seeds and tools, improved


55

access to basic services, distribution of school materials, and rehabilitation of schools.

Newly elected government officials are extending and revising the transitional ICF for continued donor coordination and aid effectiveness. Building on the results achieved under the 2004-2006 ICF initiated by the previous transitional government, the newly elected President Préval and his administration presented an extended and revised one-year ICF at a donor conference held in Port-au-Prince in July 2006. The program received strong endorsement from the donor community and representatives from the Haitian society, including the Parliament. The extended ICF focuses on infrastructure, security, governance, primary education, electricity and basic services.

The ICF framework has improved government engagement with donors and national stakeholders. Haiti has come a long way, considering the lack of coordination mechanisms and poor linkages between government, donors and national stakeholders in the past. There is strong government will and support among some donors for better coordination and prioritization of resources. More specifically, the government would like to improve the impact of pledged funds under the original and extended ICFs, although there is no consensus on how to move forward on this issue. The government has also taken steps to move toward results-based budgets in a number of ministries in accordance with an Organic Law on Budget Preparation and Execution adopted in 2004.

ICF lessons also apply to Poverty Reduction Strategy Papers (PRSPs). The ICF served as a bridge to an Interim Poverty Reduction Strategy Paper (IPRSP). Using a similar process and lessons gained from the first ICF, the IPRSP describes the government’s consultation plan for

Haiti

56

the preparation of a full PRSP in 2006/07, with a calendar for national and local consultations.

2. ICF Replication

Although the creation of the ICF was an effective country-driven process, which resulted in a comprehensive development plan, opportunities for improvement when replicating a similar process might involve i) more coordination between emergency and development assistance to avoid redundancies; ii) enhanced analysis of existing institutional capacity and clear plans for capacity enhancement; iii) improved integration of cross-cutting issues in development of targets and monitoring of results; iv) development of more detailed criteria for stakeholders; and v) joint pooling of funds or channeling of resources through a single trust fund or unique pass-through mechanism to ensure aid effectiveness, transparency, and clear reporting/tracking of disbursement.

Conclusion:

With the ICF revision and wide approval by various stakeholders, next steps include:

1. Delivering Quick Wins for Successful Implementation of Longer-term Development Results

Widespread criminal and political violence continues to impact the implementation of the ICF. Despite the deployment of the MINUSTAH with the explicit mandate to restore security, violent crime proliferated during the transitional period. While MINUSTAH’s mandate has been extended until March 2007, creating and maintaining a secure and stable environment will be a complex medium-term goal. This objective will require significant institutional and financial resources for disarmament and


57

reintegration, police reform/restructuring, and a reform of institutions of justice.

“Quick Wins” to be implemented with donor support. Since his election, while building credibility to secure external funding for Haiti, President Préval has worked relentlessly to appease the country. The new administration has promised “quick wins” in the area of social development and employment in an attempt to bring about greater stability.

Inclusive country-wide dialogue with all political and social forces is planned. Critical to political stability, the government’s plan to implement a consultative process for the preparation of the PRSP might provide opportunities for a broad dialogue. The nature of a multi-party government and the participation of former opposition party leaders in the President’s trips abroad also help build political diversity and dialogue.

2. Enhanced Results Planning and Implementation

Processes for Improved Service Delivery

Revised ICF processes and outputs still need to be determined by the newly elected government As a result of the election of the new government, revision of ICF matrices and coordination structures is on stand-by. Building on lessons from the transitional ICF, the new government has a unique opportunity to improve the results-based process in Haiti. For the revised ICF, the Haitian government has specifically emphasized the importance of aid harmonization, according to the principles of the 2005 Paris Declaration, and in particular the alignment of international aid with the country’s priorities.

Haiti

58

Monitoring disbursements continues to be a challenge. With the absence of a joint mechanism for managing funds, there has been a lack of accurate, timely and coordinated reporting on disbursement results. Composite figures presented at donor conferences have been unverifiable and appeared to be guesswork at best. As a result, there is no consensus on funds actually spent against the ICF. However, based on actual visible results, all agree that there has been below-optimal use of resources in transitional ICF implementation. A special unit has been established in the Primature (Coordination Cell) to track disbursements related to the ICF. The government has also allocated resources to reinforce their own capacity in this regard.


59

Peru: Alerta Miraflores: Managing to Increase Citizen Safety – Lima, Peru

Jeffrey Franco and Ignacio Soto, (Voxiva Inc.)

This case demonstrates strong use of evidence-based decision-making and management for results; It also shows leadership and political commitment at the local government level with a results orientation; This is a good example of citizen engagement in monitoring results.

Introduction:

Miraflores, one of 42 municipalities of Lima, Peru, is home to approximately 100,000 residents and is a bustling commercial, residential and tourism center. Miraflores is well known for innovative ideas in city management, including the introduction of a municipal police force – the Seranazgo – to complement the Peruvian National Police. Nonetheless, in the 1990s, Miraflores experienced escalating crime (e.g., robbery, assault, and drug possession) that was having a negative impact on the quality of life and threatened to hinder tourism and economic activities. Indeed, over 70 percent of citizens in the Lima metropolitan area ranked crime as the city’s biggest problem. By the 2002 election, crime had become by far the top concern for the citizens of Miraflores. The newly elected Mayor made crime reduction and citizen security his top priority. From his previous experience in Miraflores and as a former advisor in metropolitan Lima, the Mayor knew that good and timely information was a key factor to truly manage citizen security1 and to deliver on his promise. He wanted to replace the existing inefficient paper-based system with a new management system that would allow him and his municipal employees to focus on crime reduction, respond quickly to reported crime, and engage the citizens in security.

1 Former New York Commissioner of Police William Bratton also stressed this point in a report for the Metropolitan Lima Municipality.

Peru

60

Application:

According to municipal law, the Mayor chairs a Citizen Security Committee and is responsible for the security of Miraflores citizens. As his first official action after election, the Mayor visited the heads of the two stations of the National Police located in Miraflores to emphasize the importance of working as a team and leveraging reliable information to fight crime. The municipality also began collaborating with Voxiva, a Lima-based private company that is a leading global provider of practical information solutions to enhance safety, strengthen health care systems, and improve government service delivery.2 Together, they designed Alerta Miraflores, a model approach to managing crime that uses Voxiva’s web and phone-based technology to:

Give citizens a way to report incidents to local security officials to record and take action;

Capture data electronically and display it on reports and maps to let public safety officials pinpoint the areas from which citizens are calling, define priorities and dispatch the closest officers;

Allow municipal officials to manage citizen security pro-actively, respond more rapidly and analyze results; and

Create greater transparency and accountability for results, thereby encouraging greater citizen participation.

The Alerta Miraflores central phone number is displayed throughout the city. When residents dial in, they are connected to an automated incident reporting system. They can talk to an operator or record their message. When a caller reports an emergency, officials confirm incident details and immediately dispatch a police unit or an ambulance. In non-emergency cases, reports are logged, and officials initiate action and call back the 2 For example, Voxiva worked with the Peruvian Navy to implement the company’s HealthNet application to support disease surveillance in some of the most remote areas of the Amazon basin. The solution has been expanded by the Navy to all its facilities and has been introduced in five additional Latin American countries.


61

citizen in a timely manner. All reports are instantly captured in a central database in real time that is monitored 24 hours a day by security officials at a command center. The database includes reports registered by citizens (through Interactive Voice Response interface) and by municipal police operators (through Web interface) who receive the information through other channels such as radio UHF communications and from municipal police officers in the field. Currently, all crime reports in Miraflores are logged into the CitizenNet (an internet and interactive voice response based system). The police’s use of the information has reduced crime in several ways. In urgent cases, they use the calls to dispatch officers to the scene and to intervene appropriately. Police employ the system to help triage non-emergency calls in order to allocate resources more efficiently. And the system is used to identify “hot spots” and trends to plan more strategic approaches to problem areas. The system also records what actions police or other officials have taken (and whether the case is in process or resolved). This information is recorded in the central database, which is viewable by any authorized user and greatly increases transparency and accountability. A historical record of all reported cases is continually maintained and available for review. The data is also reviewed by the Committee of Citizen Security, the Mayor himself, and information analysis experts to search for trends and to implement strategic initiatives based on the results. From a technology perspective, Alerta Miraflores supports integrated communication across channels (phone, web, text messaging, and e-mail). Citizens can leave voice messages that are stored in the centralized database in a voice-clip format, and authorities can respond by phone using text messages for internal alerts.

Peru

62

Voxiva also worked with the municipality to establish a Command Center that rapidly became the hub of the Miraflores “managing for development results” activities. The Command Center functions as a Citizen Security Data Center and a strategic situation room.

The Miraflores Public Safety manager regularly generates a comprehensive report of charts and graphs showing crime statistics by type, location, time of the day, day of the week, and month of the year. This information is accessible in the Command Center and has been used to re-allocate resources to locations and times of greatest need and to justify additional resources.

Every week, the Mayor brings together the Citizen Security Committee in the Command Center and leads a review with all of the authorities concerned. Alerta Miraflores provides the Committee with a rich data set for analysis, decision-making, and monitoring of trends and results. The Mayor leads all of the concerned officials in an evaluation of those results; they make decisions about the strategy, coordinate actions of the National and the Municipal Police forces, and adopt corrective measures.

The Command Center is a powerful community relations tool. Press briefings are held at the center, and it allows the municipality to communicate more effectively with the community.

Finally, the Command Center creates focus and visibility for data about results. It has established a sense of ownership among the community for the technology solution and, more importantly, ownership of the process of achieving results.

The introduction of Alerta Miraflores has enjoyed high-level political support. Using this system, the Mayor can and regularly and personally monitor citizen reports. He sees it as his responsibility to make sure that the citizens are properly attended


63

to and that his protocol of an “immediate response to every report” is fulfilled. The effectiveness of this monitoring has been remarkable for discipline and morale.

Problem Solving:

Alerta Miraflores has proven to be a simple and effective problem-solving tool, supporting the efforts of an important municipality to address a problem of great concern to its citizens. Two specific additional areas deserve mention. Citizen Participation: It was clear from the beginning that a high degree of citizen participation was THE critical success factor for this project. Citizen security was the key problem perceived by the people and the solution depended on citizen participation alongside the Serenazgo, National Police and other actions3. The Alerta phone number was widely published across the city. Citizens were encouraged to report suspicious activity. However, a lack of financing limited a campaign to promote system usage and reporting. As a result, the municipality relied on word of mouth, press coverage, and exposure at local and international events. Emphasis was placed on promoting crime reduction as a consequence of citizen participation. The concept of continued citizen participation to reduce crime was very effective in gaining acceptance for the system. Technological Innovation: As the technology partner, Voxiva introduced a series of technological innovations to address specific problems. Important innovations have also come from the Citizen Security Committee. A director of one of the National Police Stations proposed an innovative

3 Other ways of fostering such participation as filling complaints and receiving individual information were implemented in an additional module, and then a third one, providing a way for citizens to monitor municipal services was developed.

Peru

64

approach to address security concerns on the part of local businesses. This led to a partnership between Voxiva and Visanet, the Visa Cards processor, to allow point of sale devices to send silent emergency alarms to Alerta (POS SOS). An agreement has been reached between the municipality, Visanet and Voxiva to insure immediate police action and a responsible use of the alarm by business operators. Less than 3 months after Alerta was installed, the National Police, along with Miraflores and San Isidro4 Serenazgo forces, used reports from a POS SOS plus additional citizen reports in the Alerta system to capture two murderers. Voxiva and Visanet were awarded the 2006 Business Creativity Award in Peru for the use of the POS SOS to improve citizen security.

Results:

Increased Citizen Involvement: The metric introduced from the beginning was crime reduction as a consequence of citizen participation. In total, 133,000 reports have been registered in

4 A neighbor district that is also one of Voxiva’s customers


65

Alerta Miraflores since 2003 with an increase each year. Forty-seven percent of reports to the system are citizens reporting suspicious activities in their neighborhoods. The GIS map on the previous page displays citizen reports from June 2003 to June 2006. The distribution of reports shows that citizens in every zone of Miraflores have participated in the citizen security system by reporting suspicious activity in their neighborhoods. Achieving Results for Citizens: By improving its ability to rapidly respond to reported incidents; providing timely feedback to citizens; and capturing detailed crime information, the municipality is better able to prevent crime and increase citizen security. The graph at the right was generated by the Alerta system and demonstrates a drop in robberies over the past three years. In fact, Alerta Miraflores has reported a 68% drop in robberies since 2003. In part, the improvements have enhanced tactical effectiveness: several of the reports have led to the immediate capture of thieves. In addition, the increasing number of citizen calls has fostered preventive measures. Other results have had an impact on quality of life. For example, noise disturbance calls that were categorized by type (see Table 1 on the following page) and localized using GIS mapping led the Miraflores’ Municipal Council to implement a noise reduction ordinance in 2004. Some results have been more strategic, allowing the municipality to do better planning and resource allocation. Altogether, municipal officials estimate that there has been a 30 percent reduction in assaults and a significant reduction in overall crime.

Municipal officers analyze data from Alerta Miraflores to detect daily, weekly, monthly and yearly trends.

Peru

66

Table 1 Sum of Total Number of Reports by Type Year

Type 2003 2004 2005 2006 Total

General Assault/Robbery 2,019 5,857 4,131 2,221 14,228 Accident 1,273 3,842 3,467 2,599 11,181 Suspicious Activity 1,480 9,035 13,533 14,667 38,715 Noise / Neighbor disturbance 4,734 13,944 16,530 11,804 47,012 Other 8,160 13,785 21945 Total General 9,506 32,678 45,821 45,076 133,081 Source: Alerta Miraflores

High Report Reliability: Alerta Miraflores receives a very low percentage of false reports. Since its launch in 2003, 98% of the reports logged by the system were confirmed incidents like noise and accidents, or suspicious or criminal activity (robbery, assault, drug possession, prostitution, and delinquency). The high report reliability ensures that the municipal staff dedicates its time to responding to legitimate cases. Transparency and Citizen Inclusion: Every report is kept in the database, providing both citizens and the municipality with a track record and fostering accountability. The transparency of the system enables the Mayor and his staff to call citizens to acknowledge their valuable reports or apologize in case of mistakes. The system is straightforward and user-friendly and phone calls to report crimes and other incidences have been logged by all citizens, including senior citizens and children. There is a sense of renewed trust and collaboration between the citizens of Miraflores and the police force. Security Management: The introduction of Alerta has served as a catalyst for enhanced management of citizen security. Coordination among all those involved is no small achievement: two different police organizations and other auxiliary forces like firemen are working as a unified team with a single focus on


67

improving the security of citizens and reducing crime. Any Latin American city council would be proud of this accomplishment. As a national police official noted, “Now we know our enemy better.” This type of information permits intelligence actions against gangs and, in general, more sophisticated and informed police work. Replicability: The success of the Miraflores Alerta system has gained attention as a model for other municipalities within Peru and beyond. To date, eight municipalities in Lima have contracted the Voxiva Citizen Security Solution. Two more municipalities are already operating under the system in Argentina. One of the Argentina citizen security solutions is under the umbrella of a partnership agreement between Voxiva and Microsoft to deploy the Voxiva Product CitizenNet across Latin America. CitizenNet is not only technologically innovative; it is also cost effective. While Miraflores is a relatively affluent municipality, other poorer communities such as Villa María del Triunfo have also successfully budgeted for and are implementing similar systems.

Conclusion:

Alerta Miraflores has been instrumental in improving the process by which the Municipality of Miraflores is confronting crime and solving problems facing its citizens. It has truly enhanced the organizational capacity of the municipality to reduce criminal activity within its boundaries. There are a number of conclusions that can be reached:

Citizen engagement has been a critical factor in achieving important development results.

Collaboration across organizations and agencies through the Citizen Security Committee and a public-private partnership between the municipality and Voxiva were important success factors.

Timely information provided by citizens has stimulated enhanced management of the Miraflores security forces, in

Peru

68

both tactics and strategy. It has also assisted officials to bring in increased financial resources and equipment.

Timely, transparent data are useful to guide responses and also to provide the municipality with a rich data set for analysis, decision-making, and monitoring of results.

Alerta Miraflores has proven its value: the combination of effective leadership, sufficient resources, and innovative technology has engaged citizens and is continuing to contribute to a significant reduction of crime in Miraflores. Miraflores has been very willing to share its experience with other communities in Perú and Latin America and has helped to found a network, “Digital Cities/ Safe Communities,” established by Voxiva and Microsoft to promote citizen security and services. The municipality is working with support from Voxiva and Microsoft to improve its analytical capabilities, adopt more powerful analytical tools and make better predictive use of its current and historical data. For example, Voxiva participated in the Windows Vista and Office 2007 platform launch with Alerta Miraflores. The new Microsoft Excel is a powerful analytical tool that municipalities can use for improving the quality of their reports and analytical skills. Miraflores continues to add other citizen services to Voxiva’s suite of applications as additional modules to provide broader support for municipal management. For example, a Citizen Participation Module receives complaints, suggestions, and queries from citizens on any issue outside of crime reporting. Officers must return all calls in a pre-established time period. Another module, the Municipal Services Module, provides citizens with their current tax account information by entering their Tax or ID number. New modules continue to be developed to expand citizen access to municipal services and support the expansion of e-government within Miraflores.


69

Mozambique: Poverty Observatory and the Program Aid Partnership (PAP): Policy Dialogue,

Civil Society Consultation, and Mutual Accountability of Donors and Government.

Momade Saide and Virgulino Nhate (Ministry of Planning & Development, Mozambique)

Mozambique’s Poverty Observatory demonstrates that clear measuring of results can be obtained.

The Poverty Observatory stimulates policy dialogue, civil society consultation and alignment of donors and the Government of Mozambique (GoM) around monitoring and evaluation (M&E).

The Poverty Observatory not only constitutes the pillars for participation, but also strengthens the country’s territorial axis in implementing its Poverty Reduction Strategy M &E system, as a tool to measure results and policy dialogue.

All actors recognize that their participation is essential for increasing the efficiency and effectiveness of public policies oriented toward poverty reduction and for fostering national ownership of the poverty reduction strategy by democratizing the decision-making process. It provides the beneficiaries with a voice in public policy making.

The Mozambique Poverty Observatory has demonstrated that country-level participatory M&E systems can be a success; and (ii) the goal of mutual accountability between donors and government is possible.

Introduction:

The Action Plan for the Reduction of Absolute Poverty (PRS), or “Plano de Acção para a Redução da Pobreza Absoluta (PARPA),” for 2006-09, Mozambique’s second PRS, outlines a strategy to achieve the country’s development goals in line with the Millennium Development Goals (MDGs) and with the broad objectives of Mozambique’s Agenda 2025. PARPA II is intended to reduce the incidence of poverty from 54 percent to 45 percent

Mozambique

70

by 2009. It updates the PARPA I for 2001-05. Revised medium-term strategies in some sectors, including the Education Sector Strategic Program for 2005-09, the National HIV/AIDS Strategy for 2005-09, and the Strategic Plan for the Health Sector for 2005-10, all fed into the PARPA II and the Five-Year Government Program for 2005-09. The Government of Mozambique (GoM) has applied this concept in its relationships with both civil society and donors and has thereby been able to be more effective and efficient in formulating and implementing its national policies, specifically in linking the country’s macro-framework with the sectoral policies. The GoM has linked planning and monitoring with the budget and created effective mechanisms for participatory poverty monitoring and aid coordination. This is a substantial step forward in managing for results compared to the old system, in which there was a gap between macro and sectoral policies, no transparent links with the budgets, insufficient civil society participation, and a lack of coordination and harmonization with and among donors. In the sequence of building an inclusive PARPA M&E system and after studying the experience of other countries, some civil society organizations requested the constitution of a Poverty Observatory (PO) with the objective of facilitating the interaction between the civil society and the Government in the decision-making process regarding poverty reduction activities. The GoM responded positively to the request and, in 2003, created a National Poverty Observatory as a consultative forum for discussion of poverty reduction issues, which includes Government representatives and civil society and international partners. Thus, while the initial impetus of developing a PO came from civil society, the Government actually decided to create this mechanism. This decision has led to a fruitful dialogue and a set of harmonized actions since 2003.


71

Application:

The Poverty Observatory is organized around the following principles: (i) mutual transparency, (ii) reciprocal accountability, and (iii) democratic dialogue. It is important to emphasize that Poverty Observatory deliberations are taken into account in policy-making and planning at provincial and national levels. This is crucial in a country that is geographically large like Mozambique. The objective of the Poverty Observatory is to accompany and coordinate the process of monitoring and evaluation of all activities intended for poverty reduction based on national and provincial strategic plans, including reports coming from civil society. These forums meet at least twice a year during the planning cycle. The selection of the civil society representatives is carried out by the civil society itself in coordination with the Government, and the following institutions are consulted: religious institutions, workers’ unions, private sector associations (trade association, industry association); NGO networks, scientific associations, and universities. In order to guarantee the effective participation of all citizens from different country levels in the M&E system, in 2004, the provincial governments promoted the establishment of the Provincial Poverty Observatory (PPO). This exercise was a considerable achievement in participatory poverty monitoring. The Poverty Observatory, as an ongoing mechanism for participatory policy dialogue, is continuing to gain strength. For example, significant development information, in both Portuguese and English, is increasingly made available on the Poverty Observatory website (www.op.gov.mz). In order to obtain clearer measures of results, two key components were incorporated into the PARPA M&E system: (i) the strengthening of the linkages among the planning, budgeting and monitoring instruments, and (ii) a better alignment with the planning cycle at all levels of the Government and the international

Mozambique

72

community. To achieve this goal, a Program Aid Partnership (PAP) was established in April 2004 between the Government and the group of budget support donors through a Memorandum of Understanding (MoU), which set out the principles, terms and operations for the partnership. There are currently 18 members, and the group is known as the G-18 or PAPs. The MoU clarifies the Government’s performance and reporting commitments, as well as those of the PAPs. While the GoM is accountable based on the terms of its Performance Assessment Framework (PAF), the PAPs have also agreed to a number of specific commitments. These obligations are an effort to effectively implement the concerns of the Paris Declaration at the country level and to provide a well-structured mechanism of dialogue and alignment between partners on poverty reduction. The overall objective of the Program Aid Partnership is to contribute to poverty reduction in all its dimensions by supporting the evolution and implementation of Plano da Acção para a Redução da Pobreza Absoluta (PARPA). The PAP does this in the following ways:

It builds a partnership based on frank and open dialogue on the content and progress of Mozambique’s poverty reduction strategy as set out in the PARPA and operationalized through the Medium Term Fiscal Framework (Cenário Fiscal de Médio Prazo - CFMP), the Economic and Social Plan (Plano Económico e Social - PES), which includes priority indicators and targets as defined in the PAF, and the State Budget (Orçamento do Estado - OE).

It provides financing to the public sector for poverty reduction that is clearly and transparently linked to performance. It improves aid effectiveness and country ownership of the development process; reduces transaction costs; allows allocative efficiency in public spending and predictability of aid flows; increases the effectiveness of the state and public


73

administration; improves monitoring and evaluation (M&E); and strengthens domestic accountability.

Monitoring and dialogue processes follow an annual review process aligned with the Government of Mozambique’s planning, budgeting and monitoring cycle. Monitoring and dialogue processes use several types of documents:

Planning documents including the PARPA, PES, CFMP and OE;

Monitoring documents including the Review of PES (mid-year), Budget Execution Reports, the General State Account and annual audit reports (including the Administrative Tribunal Report), a Value for Money audit, and a report on PFM assessment.

There are two joint GoM-PAP reviews on program aid: the annual review (following the production of the review of the PES) and a mid-year review (prior to submission of the PES and OE to the Parliament). The annual review is focused on coming to a joint view on performance, which serves as the basis for commitments. The mid-year review focuses the dialogue on forward planning and budgeting and agreement on the PAF. The joint view on performance may include divergent opinions if acceptable to all signatories. In the exceptional case in which a joint view cannot be reached, following increasingly higher-level dialogue, views will be reported separately. In addition, PAPs have elaborated terms of reference, which define the organization structure and responsibilities at different levels in order to effectively support the Government’s poverty reduction strategy. As well as evaluating the Government’s performance through a focus on the PAF, the Joint Review of the Budget Support Program also involves an independent review of the performance of the PAPs as measured against the indicators in the PAF matrix.

Mozambique

74

Problem Solving: Tackling Issues at the Policy Level

The Poverty Observatory facilitates a regular dialogue between the GoM and stakeholders on development objectives and poverty issues. For example, in 2004, civil society organizations decided to elaborate an Annual Poverty Report (RAP) and proceeded to present it for discussion at the National Poverty Observatory. The RAP had four primary objectives: to identify the evolution of the people’s perception of poverty; to evaluate the impact of the PARPA closer to the communities; to verify the growth and distribution of national wealth; and to establish an open dialogue about the causes of poverty in Mozambique and identify ways to overcome it. Regarding the question of ownership and the Government’s leadership in the preparation of PARPA II, 22 working groups were created and included the participation of the Government, civil society and international partners. Such groups worked throughout the year to produce sector reports for the PARPA and inputs for the joint and mid-year reviews. Two POs were held at the national level to present and discuss preliminary versions of the document. In order to capture the viewpoint of the provinces in relation to PARPA II, nine PPOs were organised. PARPA II and its strategic M&E indicators were finalized and agreed upon in June 2006. This process turned out to be impressively consultative. The use of the PO in the process or PARPA review as a channel to create a healthy environment for policy dialogue once more demonstrated the importance of this type of forum. The discussion on the design of the most robust M&E indicators for monitoring the effectiveness of development assistance has entered a new era with the new PARPA. The PARPA II has a strategic matrix, and the PAF indicators used by the donors to assess GoM performance is a subset of this strategic matrix and


75

does not create any imposition on the Government by donors.1 This represents a fundamental step forward in terms of priority setting, monitoring of results, and alignment of donors to the Government’s policy choices. Performance against core MoU commitments has become stronger than performance against broader aid effectiveness commitments. The improvements are particularly accentuated in the areas of alignment and harmonization, although PAPs are urged to improve in-year predictability of disbursements.

Results:

To date, besides the creation of the first national level poverty observatory, nine Provincial Poverty Observatories (PPOs) have also been created. The Government’s target is to increase this number to eleven in 2007 and to stay at that level for a few years to consolidate the experience. The increase in the number of PPOs is an output indicator. However, in the critical area of governance and public sector reform, the PAF includes specific outcome indicators that measure the building of a unified personnel system, the efficiency of the courts, the expansion of the decentralization outreach (increasing the number of participating municipalities), and the extension of the “proportional targets” for direct execution of the budget through the e-SISTAFE (electronic budget management system). In the area of procurement, outcome indicators include targets to make the Government’s goods and services acquisition system more transparent and efficient. From these achievements, it is clear that a country-level participatory M&E system is indeed emerging. The Poverty Observatory not only constitutes the pillars for participation, but

1 In order to facilitate the achievement of the targets of the PARPA, the Government has challenged the donors to implement the Paris Declaration by decentralizing authority to the local missions and by respecting the defined targets: 85 percent of external aid must be included in the general budget (no more than 15 percent off-budget); and 70percent of aid must be available on time.

Mozambique

76

also strengthens the territorial axis in the implementation of the M&E system of the PARPA as a tool to measure results and policy dialogue. All actors recognize that their participation is essential for increasing the efficiency and effectiveness of public policies oriented toward poverty reduction and for fostering national ownership of the poverty reduction strategy by democratizing the decision-making process. The need for participation in planning and in monitoring and evaluation increases proportionately as one moves down the administrative-political hierarchy. Since the problems arise at the local level, the effectiveness of political intervention needs to be recognized and monitored jointly with the beneficiary population (civil society), international partners and the Government. Currently, donor financing that relies on country systems and the coordination between international partners and the Government is becoming more widely applied in Mozambique. The system of direct support to the State Budget through PAPs has improved the quality and ownership of foreign aid according to the Paris Declaration principles. In Mozambique, budget support accounts for 30 percent of total development assistance. In 2005-2006, PAP members have provided approximately thirty percent of their assistance through the regular budget, thus relying on country systems. The PAF monitoring process relies mainly on line ministries and decentralized government-level M&E systems to track progress on the implementation of PARPA. Efforts are being made to strengthen these systems.

Conclusion:

The Government believes that a frank and open dialogue between all the actors engaged in the fight against poverty is crucial to achieve satisfactory development results and strengthen the country’s young democracy. The PO as a consultative forum plays this role at all levels and gives an opportunity to civil society stakeholders to influence the course of policy implementation; in


77

other words, the PO gives the beneficiaries a voice in public policy. Similarly, the Government and the international partners must use these occasions to learn from and revise their performance regarding anti-poverty activities. Providing financing to the public sector for poverty reduction that is clearly and transparently linked to performance measures is a crucial part of improving aid effectiveness and country ownership of the development process. The key lesson is that a country-level participatory M&E system can emerge successfully. The Government believes that Budget Support is the best aid modality to sustain a wide reaching poverty reduction plan in a democratic, economically stable country like Mozambique. In the long term, Mozambique wishes to reduce aid dependency and increase internal resources through sustained, broad-based economic growth and improved tax collection. It is recognized that for some time to come, the country will need aid to support its poverty reduction strategy. It is therefore important that the donor financing mechanisms rely on the country systems, rather than undermining them, in order to ensure maximum effectiveness.Accuracy of the data at district level and the need for local policymakers to understand the dynamics of poverty are some issues that still need to be tackled.

Mozambique

78


79

Emerging Good Practice of Managing for Development

Results for Technical Practitioners

Managing for Development Results requires the involvement and commitment of technical practitioners. “Strong support from top political or administrative levels is essential to provide legitimacy and priority… but unless there is also support through the system, particularly at the middle management level – an outcome focus runs the risk of becoming a mere administrative exercise rather than representing an actual change in thinking or managerial mindset… In particular, an outcome – oriented approach is more likely to be relevant and to be perceived as useful when there is sufficient flexibility such that program areas can develop an approach that is meaningful for their own context and when all staff are actively involved in the development of the process.”1 Managing for Development Results includes a shift in organizational culture and an orientation towards results, as well as an interest in using results for decision making by both politicians and civil society. Implementing Managing for Development Results also requires techniques and tools. This section intends to provide MfDR practitioners with emerging good practices in terms of tools, techniques and methodologies in MfDR. Specifically,

1 Perrin, Bruce. Moving from Outputs to Outcomes: Practical Advice from Governments Around the World. Based upon the Roundtable “Moving from Outputs to Outcomes: Implications for Public Sector Management” December 15–16, 2004. Sponsored by The World Bank and the IBM Center for The Business of Government.

III

Technical Practitioners

80

Results in Action: Moving from policy objectives to programmes

How to MfDR in a sector (health, environment, education) or in a central ministry (finance, budget, planning)

Reaching beyond the capital city (how to move MfDR from national level to the regions/provinces)

What are some practical tools, methods, approaches for MfDR?

The cases in this section rely on references to specific tools and direct readers to those tools. Readers should be able to draw on these cases and apply them in their specific technical scenarios. For instance, in Chile, the government seeks to support and strengthen its Management Improvement Program by evaluating the impact on public institutions of implementing independent external certification of the program’s compliance with the International Organization for Standardization (ISO). Malaysia’s Integrated Results based management (RBM) is at the forefront of modern public sector administration under which strong incentives – both positive (by way of financial rewards for good performance) and negative (by way of penalities and sanctions) help drive public sector efficiency. Clear M&E systems lie at the heart of Malaysia’s sophisticated public administration tracking system. The Rwanda performance-based contracting in health is aimed at improving overall health insurance. It is an excellent example of the same principles, but now applied at the sector level. Preliminary evidence shows the PBC in Rwanda has made significant improvements to health service delivery in key districts in and around Kigali.


81

In Burkina Faso, the Government’s Community-Based Rural Development Project – Phase 2 (PNGT2) demonstrates efforts made in implementing monitoring and evaluation systems at the local level. This program has national coverage and highlights a strong willingness to place monitoring and evaluation of development programs in the hands of those who benefit from the program’s activities. The outcomes achieved from the program are rolled up from the community level, making an impact on national policies and decision-making processes. Responding to an increase in environmental and political pressures, technical practitioners implemented results based management practices at Benin’s Ministry of Habitat, Environment, and Urbanism (MEHU). Due to this shift, the MEHU clearly articulates funding needs and priorities to the central government, has been rewarded with a budget increase, and better responds to the needs of the constituents its serves. Technical practitioners do not operate in isolation. Their task is to implement the policies that are adopted by their leadership. Thus, the visionary approach endorsed by political leaders and senior management can only be effective, if the technical systems underpinning these policies are designed and implemented soundly. This requires good tracking of information and a willingness to make mid-course correction or adjustments where necessary. The Malaysia RBM is a good example of how mid-course adjustments are indeed made, as the budget information is correlated closely with the expenditure tracking system. The same applies in the China and Vietnam cases.

Technical Practitioners

82


83

Chile: External ISO Standards Certificate of Management Improvement Programs in the

Public Sector Cristian Venegas Díaz, (Coordinador Técnico PRODEV Departamento de Gestión Dirección de Presupuestos, Government of Chile) and Heidi Berner

Herrera (Jefa Division Control de Gestion, Ministerio De Hacienda, Government of Chile)

To improve the management of Chile’s public institutions, ISO

standards certification was applied to the government’s Management Improvement Programs.

The main benefits of the certification process include improving the use of management information in the decision making process; changing to a culture of continuous improvement; and making the system more transparent to the society, showing the government’s willingness to be evaluated by an international standard.

In addition, the project will have a positive social impact improving management of Chile’s public institutions in areas that will favorably impact, directly or indirectly, the Chilean people as users of public services.

There will also be greater transparency: the ‘Payment for Performance’ across the public sector is a remarkable initiative that shows the government’s willingness to be judged by an international standard, not just one of its own making.

Introduction: Steps that led to the Public Management Improvement Program (PMG)

Chile’s initiatives to improve public management cover such important areas as finance, human resources, internal controls, e-governance and decentralization. To ensure the efficient allocation and use of public resources, the Government adopted the concept of result-based budgeting, in developing institutions for management to measure performance and to evaluate budget related outcomes. These initiatives started in the 1990s with the design and implementation of specific instruments -- performance

Chile

84

indicators and program evaluations -- that gave the baseline, starting in 2001, to implement a results-based management control system administered by the National Budget Office (DIPRES) of the Ministry of Finance (MoF). The purpose of the management control system is to allocate public resources to the various programs, projects and institutions, more efficiently, thus improving institutional management. This system has provided a model for evaluating and monitoring the managerial budget process, by incorporating the concepts of results based budgeting, that includes the following instruments: strategic definitions, performance indicators, program and institutional evaluations, comprehensive management reports, and management improvement programs. The instruments developed, besides all being integrated to the budget process, generate a synergy derived from its conceptual design elements and the processes that control its implementation. One of the instruments of the Chilean management control system is the Management Improvement Program (PMG), launched in 1998, with the enactment of Law No# 19,5531. This law required that each institution commit yearly to achieve management objectives whose accomplishments are tied to a monetary incentive for all the employees on the institution’s payroll. The aforementioned law establishes that upon reaching management objectives committed to in an annual PMG, those public servants would receive a salary increase in the next year. This increase would be 5% if the institution where they work reached 90% or better of its committed annual goals. The increase would be 2.5% if the institution accomplished a 75% or better and less than 90%2 and there is no increase if the institution does not accomplish at least 75% of its committed goals.

1 Law 19,553 of February 1998 and Decree 475 of May 1998 2 Law 19,882 of 2003 modified the established percentage in Law 19553 of 3% and 1.5%


85

Application: Description of the Management Improvement Program PMG

The Management Improvement Program PMG has three periods of development clearly defined, with the accomplishment of the following management objectives and commitments included:

1. The achievement of goals established on the basis of indicators (1998 – 2000).

2. The achievement of management systems’ objectives established in the PMG Basic Framework Program (2001 – 2004).

3. The achievement of management systems’ objectives established in the PMG Advance Framework Program, which includes the requirements of the external certification model, such as ISO 9001:2000 standard (2005 until now).

1998 – 2000: During the program’s first year, management’s committed objectives were expressed in indicators. A large percentage of these referred to routine institutional activities, centered on processes and with fairly achievable targets or goals. As a result, unequal efforts were rewarded equally, and for the vast majority of the institutions, achieving PMG objectives did not require management efforts that were pertinent or relevant, which is an undesirable situation for a program that links monetary incentives to management improvements. The uneven results were explained, among other reasons, by the unequal level of management capacity in the various institutions and insufficient technical work in the verification of commitment achievements. 2001 – 2004: Modifications were incorporated in the formulation and application of the PMGs from 2001 on, with the objective of contributing to the development of strategic areas of public management. The elaboration of the Framework Program and other processes used yearly in the development of the PMGs

Chile

86

includes a set of principles or requirements for this process. These principles are that the management systems incorporated in the framework program must be pertinent, simple, comprehensible, demanding/comparable, participatory, flexible and verifiable. The PMGs Framework Program has 5 areas for management improvement, common to all public sector institutions (human resources, customer service, management planning and control, financial management and gender). Each area consists of systems that derive from policies formulated for modernizing public administration. These eleven systems correspond to the following: Areas Systems

Training Performance evaluation Human Resources Hygiene, security and improved work environment Integral customer service system Customer Service E-governance Internal audits Integrated territorial management Management Planning and

Control Planning and Management Control Government procurement Financial Management Financial management – countable

Gender Focus Gender focus For each management system included in the Framework Program, different stages were identified to achieve the system’s objective (4 or 6 defined development stages). Each stage is specifically defined with its contents and requirements in such a way that, upon completion, it advances the development of each system according to its characteristics and basic requirements,3 conforming therefore to an accreditation model. The PMG of each public institution consists in the identification of the development stage proposed to

3 See document: Medios de Verificación PMG 2006 at www.dipres.gob.cl Management control system/Management Improvement Program/Formulation 2006


87

be reached for each system in the Framework Program each year. The committed stage constitutes their “management objectives”. The evaluation of achievements under the committed stage is done by a network of experts from public sector institutions having responsibility in each of the systems. 2005 - After four years of application of the PMGs Basic Framework Program, the results obtained and the opinion of the great majority of the actors involved, showed that this model of implementation of the PMG was an instrument that helped to improve public management. Therefore, a new step was taken in the development of the PMG’s management systems, considering that in 2004 an important number of Institutions had reached the higher stages defined in the Basic Framework Program. This new step incorporates the certification of PMG´s management systems, applying an external standard that would not only promote continued excellence, but would also make program achievements easily recognizable in the society at large. Thus, the Advanced Framework Program incorporates the objectives of the Basic Framework Program and the requirements of standard ISO 9001:2000, including new system stages, gradually complying to an external certification model such as ISO 9001:2000 standard. The complete process of ISO 9001:2000 certification of the PMG is estimated to cover up to 105 public institutions involving, approximately, 45,000 public sector employees. ISO 9001:2000 certification is a process in which accredited certification entities evaluate the processes that an organization should have in order to certify their management systems. The ISO standard establishes the minimum requirements that must be achieve in order for organizations to be certified as having quality management systems, with an emphasis on continual improvements in the operation of these systems. The standard is applicable to 9 of the 11 systems that make up the PMG Framework Program. Applying this standard will make these systems more rigorous and enable certification.

Chile

88

In the formulation of the 2006 PMGs, the systems incorporated in the Advanced Framework Program are training, performance evaluation, hygiene, security and improved workplace environment, integral customer service system, internal audits and planning/management control. For the 2007 process, the PMG will include the government procurement system. To strengthen the implementation of the Advance Framework Program, the Inter-American Development Bank (IDB) and the Chilean Government agreed on a technical cooperation under the PRODEV framework.4 The technical cooperation objective is to support and strengthen in the Chilean management control system -- the Management Improvement Program (PMG). This technical cooperation has various actors that participate and collaborate at different times in the formulation and application of the PMG. The actors are: i) the IDB, which conveys the non-reimbursable technical cooperation, ii) DIPRES, which executes the technical cooperation, iii) the National Standardization Institute, which does training for professionals and technical staff that work in the public institutes, iv) General Comptroller of the Republic, which audits the financial statements of PRODEV, and v) Public Institutes, which act as co-executors of the technical cooperation that provides training and certification from the project. The technical cooperation consists of three components. Component 1 – Impact Evaluation of the Management Improvement Program. Component 2 – Support for public institutions that will obtain ISO 9001:2000 certification for their management systems. Component 3 – Development of technical support elements and strengthening of the entity responsible for the coordination and the technical and operational development of the PMG.

4 Program to implement the external pillar of the MTAP for development effectiveness (PRODEV), document GN-2346-5, IDB website.


89

The impact evaluation of the PMG component, to be conducted by evaluation experts, includes assessing outputs, midterm results, and outcomes achieved by the PMG. Recommendations are expected on improving the design and management of the PMG. In support of the public institutions that will obtain ISO 9001:2000 certification of their management systems, two phases will be developed: (i) preparation, and (ii) certification. Training of the participating public institutions is needed. The certification process includes audits to be carried out by accredited external entities. The final component includes development of technical support elements and strengthening of the entity in charge of coordinating and supervising the technical/operational development of the PMG. So this component will provide training to this unit on the application of ISO 9001:2000 standards applied to the PMG systems. Also, this component will assist Dipres to prepare, update and distribute reference manuals to be used by the public institutions carrying out the implementation and certification process. These manuals document all of the requirements for the ISO 9001:2000 certification, identifying the technical requirements associated with the various PMG systems, and guidelines regarding any exceptions.

Problem Solving:

During the process of certification of the PMG´s management systems of the public institutes carried out the years 2005 – 2006, the following problems were detected: i) difficulties in understanding the language of the norm, complex and technical; ii) slow appropriation of the quality management system inside the institutions; iii) concern on the part of the management teams responsible for the systems covered by the internal audits , iv) lack of communication and socialization processes inside the institutions; v) tendency to develop excessively complex procedures, vi) tendency to improve the processes of management

Chile

90

at the same time that they are being implemented in the framework of the 9001:2000 standards; and vii) difficulty to incorporate all the personnel of the institution in the process of certification. The difficulties found were addressed through: i) training public sector employees in ISO standards associated to the PMG before initiating the process of certification, as well as during the process; ii) implementing a policy of communication and socialization of the process of certification in the institution; iii) involving a greater number of employees in the process of certification; and iv) drawing on the support of external consultants to simplify the process.

Results:

With the development of PRODEV, the following results are expected to be achieved: i) Certification under ISO 9001:2000 regulation of 58 public institutions, where each one will have at least 4 out of 11 PMG systems certified; ii) Impact Evaluation of PMGs during 2001 – 2005, where recommendations are expected to improve design and management aspects of the PMGs; iii) 698 government representatives trained in ISO regulations; and iv) 9 Reference Manuals compiled. The project is expected to be executed within 26 months starting from the eligibility date of this agreement. The agreement was signed May 8, 2006. The MoF’s National Budget Office (DIPRES), through its Management Control Division, is responsible for project execution. DIPRES is in charge of the technical and operational aspects of the project and coordinates all of the activities required during the stages of formulation, execution, and evaluation of the project. The structure, operation, and administration of the project has been done with the institutional framework created by the government for the PMGs. For this operation, the DIPRES Management Control Division will serve as the coordinating unit, through its Management


91

Department, which is composed of nine professionals with experience in design and implementation processes of public management instruments. The department is supported by other parts of the MoF for administrative and accounting aspects of the project. The Division will be strengthened by hiring certified professionals, ensuring compliance under ISO standards. The agreement between IDB and Chile was signed in May 2006. Results accomplished to date (table below) show that the most advanced item relates to the training of DIPRES employees (15 trained). Another important result is the number of certified systems to date (46 systems). With regard to reference manuals, the Manual for Government Procurement was prepared during 2006, since it is the new system that has entered the Advanced Program Framework in 2007. The rest of the manuals will be updated during 2007. All public sector employees in the Project will undergo training in 2007. Additionally, during 2006 technical support was given to public institutions through: i) the development of workshops to analyze the link between technical requirements of the PMG systems and ISO standards requirements, ii) the elaboration of terms of reference to be used as a framework to hire external consultant for the ISO 9001:2000 certification , iii) the registration of National Institute of Normalization Consultants, iv) the assignment of resources to finance the 75% of the Certification’s cost, with the remaining 25% financed by each public institution.

Chile

92

Targets achieved by December 2006

Item Total Project Commitments

Total Programmed for 2006

Total completed December

2006

% completed

of 2006 program

% of Project Commitments

completed

Impact Evaluation of PMG5

1 -6 - - -

No. Certified Services A 58 22 22 100% 38%

No. Certified Services B 116 44 467 105% 40%

No. Trained Public Employees

698 336 08 0% 0%

No. DIPRES employees trained – Expert Network

10 10 15 150% 150%

Reference Manuals 9 7 19 14% 11%

The key aspects of the certification are: i) The close involvement of the authorities, and ii) the use of external specialists who can help to implement the quality management system inside public institution.

Conclusion: Benefits and Risks of the Project

The main benefits of the certification process of PMG’s system are: i) improving the processes and procedures in the institution; ii) improving the use of management information in the decision

5 The Impact Evaluation of the PMG will begin with the hiring process of consultants during the first trimester of 2007. 6 The Impact evaluation will be developed during 2007. 7 21 Institutions certified 2 systems each and one Institution certified 4 Systems. 8 10 training courses will be held between January- May 2007. 297 public employees will be trained; reminder will undergo training during the second half of 2007. 9 In December 2006 the reference manual was sent to institutions. Updated version of the existing 6 reference manuals will be available to institutions in early 2007: Manuals cover Training, Hygiene/Security and improved workplace environment, Performance Evaluation, Internal Audit, Integral customer service, Planning and Management Control.


93

making process; iii) changing to a culture of continuous improvement; and iv) making the PMG system more transparent to the society, showing the government’s willingness to be evaluated by an international standard. In addition, public institutions will be modernized because: i) a "quality" concept is strongly installed in the provision of good and services; ii) the focus is on the "client" needs, iii) improvisation is reduced, and the management is "professionalized"; and iv) the concept of quality management is introduced throughout the institution. The project will have a positive social impact improving management of Chile’s public institutions in areas that will favorably impact, directly or indirectly, the Chilean people as users of public services. There will also be greater transparency: i) the ‘Payment for Performance’ across the public sector is a remarkable initiative that shows the government’s willingness to be judged by an international standard, not just one of its own making. ii) The External ISO Standards Certificate of Management Improvement Program in the public sector will not be affected by changes in government, because this is part of an agreement between the Congress and DIPRES. Critical risks considered in the certification process are: i) not having full support and leadership of the Director of the Institution, ii) not knowing and understanding the methodology of the ISO standards, iii) not aligning the processes with the demands of the ISO standards, and vi) not linking the institutional strategic objectives with the demand to implement the Quality Management System under the ISO 9001:2000 standard.

Chile

94


95

Malaysia: Integrated Results-Based Management – the Malaysia Experience

Koshy Thomas, (Deputy Undersecretary, Tax Division, Ministry of Finance)

“The Malaysia Modified Budgeting System (MBS) provides greater managerial flexibility and accountability…it performs on par with international best practice and is being used in some developing countries as a model.” (World Bank report (1999) on Public Expenditure Management in Malaysia: How Malaysia Performs.

The Integrated Results Based Management system incorporates a long-term macro-planning framework through effective horizontal and vertical integration for national development and capacity-building.

Integrated Performance Management Framework, part of Results Based Budgeting, provides baseline data that allows measurement of comparable progress and results at predetermined intervals. Performance data are explicitly focused on measuring performance progress areas such as key result areas (KRAs), goals, objectives, outcomes, outputs, and activities. Such performance data are monitored against predetermined targets.

A well-structured national level strategic plan lays the foundation for focused sector and program level plans and allows ministries and departments to establish linkages to higher-level key result areas. The targets under the Millennium Development Goals (MDG) typically form part of such a macro strategic planning framework.

The system is critical to meeting Malaysia’s national needs and the challenges of globalization and regional competition.

Introduction:

Malaysia’s economy functions very well in an extremely competitive East Asian region, driven by a solid partnership between efficient public sector management and private sector entrepreneurial spirit. Results Based Management (RBM) is a contemporary management approach that helps enable successful economies. It focuses on the appropriate and timely achievement

Malaysia

96

of relevant goals and objectives through strategic planning, systematic implementation and resource usage, performance monitoring, measurement and reporting as well as systematic use of performance information to improve policy decision making and program performance at all levels. It emphasises the importance of achieving results through systematic goals and objectives and clearly states how results should be attained. The RBM approach has been used in many countries as a strategic performance planning tool – it was first introduced in Malaysia in 1990 and is now being adopted in several countries such as India (Kerala), the Philippines, Namibia, Mauritius, Bhutan, Nepal, Bangladesh, Pakistan, Botswana, Vietnam, and South Africa, to name a few. The early years of RBM stem from the Management by Objectives (MBO) approach and the Program Performance Budgeting System (PPBS) developed in the 1960s, which were early attempts to focus on results and objective achievement. However, these systems lacked detailed processes for implementation. In the 1970s, the Logical Framework (LF) approach was introduced in an effort to better track the process. LF was used extensively in many countries and organizations in the 1980s-90s in various forms as a management and planning tool. LF later evolved into the rudimentary versions of RBM, which has been successfully used by a number of countries to drive more focused planning and implementation of public sector programs.

Application:

The Integrated RBM (IRBM) System: a Focus on Outcomes. Despite these early developments, integrated approaches to render the RBM system a comprehensive, dynamic and practical performance management system for government-wide implementation were limited. Early versions of the RBM were focused either on the budgeting system or on the personnel performance system, and there was minimal or no integration between the two. There was also minimal integration between the


97

development (public sector investment programs) and operating budgets in these countries. In addition, these systems did not reflect the fact that most developing countries have limited human resources and technological capacity. As a result, an RBM system developed in the late 1990s based on the performance framework was first introduced in the Malaysian public sector in 1990 under the Modified Budgeting System (MBS)1. However, the original performance framework did not integrate the operating and development budgets nor the personnel performance system, and it created only limited linkages between budget performance, resource usage, and policy implementation. These gaps were identified as fundamental missing links in this version of the RBM system2. Based on the lessons learned since 1990 and the identified shortcomings of the original RBM system, a revised Integrated RBM (IRBM) system3 was developed in 1999, using an Integrated Performance Management Framework (IPMF). Unlike typical RBM systems, the IRBM system takes into account and integrates critical performance components such as the Results-Based Budgeting (RBB) system and the Personnel Performance System (PPS), both of which utilize the IPMF. The IRBM system requires top management within the Ministry and Departments to be actively involved in strategic performance planning, consultation efforts and consensus-building with lower management levels. This system essentially focuses on analysis of client needs and problems and on results at the various stages of program implementation, such as efficient resources utilisation (inputs), activities completion, outputs completion, and outcome/impact achievement.

1 Rasappan, Arunaselam (1994), “How to measure success”, Khidmat, Kuala Lumpur, October, p. 27-29. 2 Rasappan, Arunaselam. (1999), “Budget Reform - Malaysia”, Paper presented at World Bank Budgeting & Performance Management Workshop, Washington DC, July 15-16. 3 Ibid.

Malaysia

98

The cornerstone of the Integrated Results Based Management system is its strategic use of the Program and Activities approach within a long-term macro-planning framework. It focuses on systematic and structured performance measurement and involves linkages with policy-making, resources management, program performance improvement, and other crucial success factors in performance management. The IRBM system consists of five key components - two primary and three complementary or support components. The primary components are the Results-Based Budgeting System (RBB) and the Results Based Personnel Performance System (PPS). There are three support components:

The Results-Based Monitoring and Evaluation (M&E) System is used for systematic and focused program planning, performance monitoring, performance evaluation, performance reporting and information utilization for program improvements and policy decision-making. This factor helps to ensure systematic and structured performance planning, management, and measurement under the RBM and helps to forge tighter linkages between resource use and policy implementation.

The Management Information System (MIS) is used to provide the basis for an effective decision support system at different levels of an organisation.

An Enabling E-Government (EG) System The primary components under the IRBM provide the necessary framework for planning, implementing, monitoring and reporting on organisational performance, with systematic links to personnel performance. The M&E, MIS and EG support components provide the dynamic dimension to the entire performance framework. The system brings to life static information by establishing relationships between cause and effect, which is


99

important for resource allocation decisions by the Central Budget Office.4 Results-based Budgeting (RBB) is a strategic management tool designed to improve resource management and public sector accountability. The core of the RBB system is the IPMF, and it focuses primarily on performance measurement and linkages with policy-making and resource management. It targets the results of programs and activities undertaken by government agencies using public monies. The RBB results are classified under various performance components: inputs, processes, outputs, outcomes, and impact. Though traditionally there is overwhelming emphasis on outputs and outcomes measurement, RBB also focuses on purposive input application and activity completion, recognizing the close relationship between input use and output performance. Various dimensions of output performance also affect the desired outcome of the program or activity. Therefore, the RBB measures results achieved at almost every stage of the project from input application, activity completion, outputs delivery, and impact achievement, using a unique Results Ladder approach5. The RBB drives the RBM system and is implemented using the Modified Budgeting System (MBS). The Performance Agreement for the IPMF is normally prepared using a unique program logic tool called the ProLLTM as part of the budgetary process6. The RBB system’s strong focus on results is evident in the approach

4 The Malaysian version of the IRBM has been implemented over the past few years with technical and capacity-building support from the Center for Development & Research in Evaluation (CEDRE) Malaysia. CeDRE Malaysia is a development and research center in Malaysia, conducting M&E and budget analysis. The center also serves as technical advisor to several governments in Asia and Africa on the IRBM system.conducting M&E and budget analysis. The center also serves as technical advisor to several governments in Asia and Africa on the IRBM system. 5 Rasappan, Arunaselam (2000), “Moving to Performance-Based Management”, Asia Development Forum Workshop: “Public Expenditure Management: New Processes and Technologies: From Rules To Results: Singapore: June 7-8 6 ProLL is the Program Logic & Linkages Model that was developed for public sector program planning and evaluation by Arunaselam Rasappan, the trainer/advisor on MBS and evaluation to the Government of Malaysia since 1991.

Malaysia

100

and strategic components used in the IPMF planning process. The RBB requires considerable strategic inputs and needs assessments before goals and objectives are set for each program and project. A good budgeting system must not only have strong linkages between resource use, outputs achievement and ensuing program results but should be closely linked with policy implementation. Who drives the system? The Treasury drives the RBM public sector reform and the performance agenda using Integrated Performance Management Framework as part of the RBB system. This allows the planning framework to be integrated with the budgeting process, so that managers can be held accountable for the resources provided to them. Since the planned programs are driven through the IPMF, human capital plays a pivotal role in organisational and personnel performance. These dynamics are captured in an integrated monitoring system that forms the basis of a comprehensive MIS. Although the focus of performance measurement under the IRBM is on results (outputs, outcomes, and impact) and its various dimensions, there is also equal emphasis on the efficient use and management of inputs and work processes.

Problem Solving: the Integrated Performance Management Framework (IPMF)

The IPMF is mandated as the strategic planning framework under IRBM. Therefore, all ministries and departments are required to prepare their strategic plan for resource allocation using the IPMF as part of the RBB system. The IPMF has been designed in sufficient detail to capture specific information at various levels, as well as to establish strategic linkages between one level and the next. The information focuses on results at different levels that are important to relevant stakeholders. The comprehensive nature of the IPMF also focuses on long-term goals that are broken down into shorter-term budget-linked objectives. RBM strongly subscribes to the systems theory that allows not only vertical


101

integration, but also horizontal integration at the program level. Programs are normally defined at the national and sectoral levels, and the eighteen components in the standard Malaysian IPMF provide sufficient baseline details that can be used by management for program planning and target setting. The IPMF also establishes the necessary linkages between sectors and creates inter-agency coordination. As a result of the integrated nature of the IPMF, it has become the primary performance monitoring and reporting tool.

Results:

IPMF baseline data allow measurement of comparable progress and results at predetermined intervals, which are dependent on the nature of the program that is being measured. Baseline data can be classified into profile data and performance data. Profile data provide critical information about the entity measured and is not typically prone to major changes. Examples of profile data include general information about the nature of an entity as well as client, stakeholder, staff, and budget profiles. Profile data, such as budgetary allocation and personnel, may change in the course of the year and therefore need to be periodically monitored and reported. Performance data are explicitly focused on measuring performance progress areas such as key result areas (KRAs), goals, objectives, outcomes, outputs, and activities. Such performance data are monitored against predetermined targets. A well-structured national level strategic plan lays the foundation for focused sector and program level plans and allows ministries and departments to establish linkages to higher-level key result areas. For example, the targets under the Millennium Development Goals (MDG) typically form part of such a macro strategic planning framework. Personnel Performance System, MIS and M&E need to be inter-linked. Clear lines of accountability need to be established within the IPMF so that linkages can be created between organizational performance and personnel performance. An

Malaysia

102

individual’s work program is linked to the IPMF through the group work program resulting in more focused and effective Human Resource Development (HRD) and Human Resource Management (HRM) programs. The Results-based Personnel Performance System provides better grounding for planning and implementation of HRD/HRM. The M&E and MIS components must drive the linkages within the IPMF in order to produce timely, accurate, and reliable information for both program improvement and policy decision-making. Program monitoring tracks key performance indicators (KPIs), both operational and results-based, at different program levels, so that timely and appropriate steps can be taken to keep a program on track and to ensure that its objectives or goals are met in the most effective manner. The KPIs generate reports for specific stakeholders and user groups including the Budget Office, Cabinet, Permanent Secretaries, Department Heads, and Donor and Aid Agencies.7 The information can be aggregated into an early warning system or information dashboard for management, providing useful information on shortfalls in resource utilisation, over-expenditure, variances in achievement, and the identification of weaknesses so that remedial actions can be taken. This process enables timely decisions to be taken on funding and apportionment of resources, human resource deployment, policy adjustment, and program improvement.

Conclusion:

It is critical that public sector program planning and implementation is carried out in a systematic and integrated manner. Given the increasing resource constraints faced by governments and the increasing demands from the public for better quality and more responsive services, public sector planners must be more results-oriented in program planning and delivery so

7 Reports are generated on a weekly, monthly, quarterly, or annual basis with frequency (and depth of reports) determined by the level of managers.


103

that the best ‘value for money’ can be realized. The IRBM system, with its integrated approach to performance improvement, is a practical and proven solution to systematic program planning, implementation, monitoring, evaluation, and improved policy decision-making in the public sector. And Malaysia continues to improve the IRBM system to meet the country’s needs. For example, in 2004, the introduction of a unique computerized application solution, the PPMSTM8, increased the system’s efficiency.

8 PPMS software was developed by CeDRE for the IRBM system -- see www.cedre.org.my/ppms

Malaysia

104


105

Rwanda: Performance-Based Financing in Health Louis Rusa, (National PBF Coordinator-Ministry of Health Rwanda), and Gyuri Fritsche, (Health Care Financing Specialist-Management Sciences for

Health)

Performance based financing in Rwanda can serve as a guide for other countries that want to apply similar schemes. Essential elements for success were the ‘roll-out plan’ and strong and consistent leadership at the highest level in the Ministry of Health.

Introduction:

Rwanda is one of the poorest countries in the world and has a typical epidemiological profile for sub-Saharan Africa. The average Rwandan lives on less than US$0.70 per day. Per capita annual health spending averages about US$14, with donors funding over 40%, government about one-third, and beneficiaries contributing roughly one-quarter. 1 Performance Based Financing (PBF) is an approach to health financing that shifts attention from inputs to outputs, and eventually outcomes, in health services. Whilst inputs are necessary to finance health services, a predominant focus on inputs has failed to deliver the results that are necessary, if the country is to achieve its Millennium Development Goals. The key premise in Output-based aid is that it “seeks to address weaknesses by delegating service delivery to a third party under contracts that link payment to the outputs or results delivered. It thus has the potential to improve incentives and accountability, while also expanding opportunities for mobilizing private financing. The focus shifts not only from inputs to outputs, but also toward the Holy Grail of development outcomes.”2 1 2003 National Health Accounts 2 “Contracting for public services; Output-based aid and its applications”, WB/IMF 2001, page 91

Rwanda

106

Rwanda has scaled-up community health insurance, from 7 to 51 percent of the population between 2003 and 2006, leading to increased access to health services. One of the key challenges facing policy makers is how to ensure sufficient supply of services of reasonable quality to meet this increased demand. Universal characteristics of health services in poor countries are inefficiency, wasteful use of resources, low quality of services, and an unmotivated workforce. Financing instruments available to donors and Governments seem unable to address these core causes in an efficient manner; proof of which is that from sizeable resources made available to the health sector in a country such as Rwanda, only a small portion reaches the health centers.3 Performance based financing, or ‘pay-4-performance’ or ‘output based aid’ as it is generally referred to, consists of a family of various methods and approaches that all aim, through differing levels of intervention, at linking incentives to performance.4 For instance, in a health program in Haiti, MSH converted 10% of the historical budget of NGO grantees into a performance bonus, if they would reach certain predetermined performance targets;5 in Afghanistan, Performance Based Partnership agreements, written between the Ministry of Health and Non-Governmental Organizations, covered 10 provinces and NGOs could receive up to 11% of the contract amount if they performed exceptionally well. Contracting and performance based financing are linked. Contracting is the ‘tool’ which is used to operationalize

3 “Paying for health in two Rwandan provinces: financial flows and flaws”, Andreas Kalk, Jean Kagubare Mayindo et al, Tropical Medicine and International Health, Vol 10, No 9 pp 872-878, September 2005. 4 “Can ‘Pay for Performance’Increase Utilization by the Poor and Improve the Quality of Health Services?” Discussion paper for the first meeting of the Working Group for Performance Based Incentives, Center for Global Development. Rena Eichler, http://www.cgdev.org/doc/ghprn/PBI%20Background%20Paper.pdf 5 “Promoting preventive health care, Paying for performance in Haiti”, Rena Eichler, Paul Auxila and John Pollock, in “Contracting for public services, Output-based aid and its applications’ page 65


107

performance-based financing; the literature on performance based financing is therefore closely linked to the literature on contracting, ‘contract theory’ or ‘incentive theory.’ Performance Based Financing started in Rwanda as early as 2001. Several factors led to this -- NGOs working in Rwanda at the time felt that, although they paid health workers a ‘bonus’ salary supplement, the outputs of the health services were stagnating and in some cases even deteriorating. Another reason was that experiences from other contexts, such as an innovative pilot scheme contracting health services in Cambodia, were replicated. 6 Various organizations, including NGOs, that had been working in the pilot schemes found new and innovative ways to increase performance of health services. This experience was applied to Rwanda. In the South-west of Rwanda, the NGO Memisa/Cordaid started a PBF scheme in Cyangugu province in 2001,7 and the NGO HealthNet International (HNI) started a similar scheme in the Southern province of Butare in 2002.8 In 2005, the Belgian Technical Cooperation (BTC) also started a PBF scheme. The ‘Cyangugu model’ and the ‘Butare model’, although with a different set-up, proved successful to the extent that the Ministry of Health, when both NGOs had funding problems, decided to fund the purchase of the outputs (2004-2006). The Ministry of Health decided to roll out performance based financing in all health facilities in Rwanda.9 A review comparing both schemes to a

6 For a description of the Cambodia contracting experience see “Buying results: Contracting for health services in developing countries” Benjamin Loevinsohn, April Harding, Lancet 2005; 366: 676-81 and “Achieving the twin objectives of efficiency and equity: contracting health services in Cambodia” ERD Policy Brief Series no 6. Asian Development Bank, March 2002. Blushan I, S. Keller S. and B. Schwartz “Improving Government Health Services through Contract Management: a Case from Cambodia” Rob Soeters R and F. Griffiths, Health Policy and Planning, 18 (1): 74-83, 2003. 7 ‘Performance-based financing and changing the district health system: experience from Rwanda’, Rob Soeters et al, Bulletin of the WHO, November 2006, 8 (11) 8 ‘Reviewing institutions of rural health centers: the Performance Initiative in Butare, Rwanda’, Bruno Meesen et al, TMIH, Volume 11, No 8, pp 1303-1317, August 2006 9 Health Sector Policy’ Government of Rwanda, February 2005

Rwanda

108

traditional (input based) financing scheme found the output-based scheme to be superior.10 A recent paper provides an excellent description of the Rwandan PBF model.11

Application:

The Ministry of Health’s strong political will and support moved Performance-based Financing into the implementation phase. The Ministry saw output-based financing as a way to enhance quality, as a method to counteract some of the negative effects of the obligatory pre-payment schemes on provider behavior, and as a way to motivate the underpaid health workforce. Due to the enormous pressure for results, time was simply lacking to embark on long and thorough planning cycles with clear results frameworks. In fact, the overarching objective was crystal clear: to implement PBF, as soon as possible, at all levels in the Rwandan Health System; at the community level, at the health center level and at the district hospital level. A rigorous intervention study, or ‘roll-out plan’ provided a road-map, including timelines, for implementation.12

10 “Comparison of two output based schemes in Butare and Cyangugu provinces with two control provinces in Rwanda” Global Partnership on Output- Based Aid (GPOBA), the World Bank, Robert Soeters, Laurent Musango, Bruno Meesen, September 2005. 11 ‘Rwanda, Performance Based Financing for Health’, Center for Global Development Working Group on Performance- based Incentives. Miriam Schneidman, Louis Rusa, October 20, 2006. 12 “Proposed Evaluation and Roll-Out plan, Performance Based Contracting Rwanda.” Version 12 Dec. 2005, WB and Ministry of Health.


109

Figure 1: the three Phases; Phase 0 the existing PBF schemes,

Phase 1, the districts in which PBF will be introduced, and Phase 2, the seven control districts.

USAID, the BTC, the WB, HNI and Cordaid provided technical assistance to the Ministry to implement PBF. A collaborative approach, involving all actors was initiated. The following milestones in the implementation phase were followed:

1. The design, during 2005, of a national roll-out plan, that would allow for a thorough evaluation of the impact of PBF on health services in Rwanda;

2. A period of 1.5 months of review and comparison of the existing pilot PBF pilot schemes;

3. Two national workshops in January 2006, one, a two-day workshop in the capital and the second, a one-week retreat with all key partners with as purpose to discuss results, teambuilding and to plan for next steps;

Rwanda

110

4. A national three day workshop mid-February 2006 in Kigali, in which all key stakeholders participated, to design a common PBF model. Seeking common ground and consensus building between partners with different models was the ultimate aim; consensus was built using techniques such as Delphi13 and the Six Thinking Hats™.14 Design of a national PBF model for health centers with a choice for a separation of functions, an ‘internal market’ or ‘quasi market’ design and decentralized governance through district level steering committees. The national model is a mix between the two oldest PBF models; predominantly based on the ‘Butare model’, but with some crucial elements of the ‘Cyangugu model’;

5. The design, shortly after the February workshop, of contract models that would be used by the Ministry and the local Government;

6. Division of the 23 Phase 0 and Phase 1 Districts in areas where partners would provide TA to the Ministry for the roll-out plan;

7. A ‘phase of uncertainty and transition’: between end-February and mid-May 2006, in which partners had to change their existing PBF models and make operational plans for the new PBF districts.

8. Between May-July 2006: roll out of the new national PBF model for Health Centers in 23 districts.

9. Mid-May to mid-July 2006: a technical working group drafted a national PBF model for District Hospitals, field testing, modification and adaptation of the new DH PBF model during a two-day workshop in mid July.

13 From ‘Prioritization Process Using Delphi Technique’ White paper by Alan Cline, Carolla Development http://www.carolla.com/wp-delph.htm 14 ‘Six Thinking Hats’ Edward De Bono, First Back Bay paperback edition, revised and updated, 1999.


111

10. Roll out of the national DH PBF model mid-July onwards: as of December 2006: new model introduced in 23 District Hospitals.

Problem Solving:

Problems that arose during the design and implementation phases were:

1. Diverging views on different models and the best model: existing Health Center PBF models had proponents and opponents, varying designs were rooted in personal preferences rather than in, admittedly, the scarce or even absent evidence base. Path-dependency seemed to determine preferences rather than sound logic or reference to the published peer-reviewed literature. Mitigation Strategies: attempt consensus building techniques, emphasize common points rather than differences; aiming at a common District Hospital PBF model to work towards standardization of all models in the near future;

2. Coordination is a challenge in a hectic implementation phase: whereas in the second quarter of 2006, 12 technical working group meetings had been held, only two meetings were held in the third quarter due to the intense implementation phase. Mitigation Strategies: attempt to keep regular, well-documented, meetings. Shift attention to areas of common interest such as Monitoring and Evaluation, analysis of results and creating a national PBF database.

3. Technical assistance to the district health authorities and to the central PBF department: understaffing at all MOH departments due to recent administrative reforms; only 3 technical staff at the central PBF department responsible for maintaining the payment function, in addition to the policy setting, oversight, coordination and regulatory roles and functions. No staff at the district health departments

Rwanda

112

with ‘verification/control’ in their job descriptions. Mitigation Strategies: work through an extended team approach: use staff from all agencies involved in PBF for filling gaps. Training of a group of master trainers, from all concerned agencies, in knowledge on modern facilitation techniques and the new national PBF model. Creation of a national PBF database, using the recently (June 2006) developed 3G network which has a 90% coverage in Rwanda. Finding additional funds for paying for the additional human and other resources necessary for the ‘verification/control’ function.

4. PBF roll-out parallel to the introduction of far-reaching administrative reforms: Parallel administrative reforms, leading to health facilities being made responsible to the Mayor’s office instead of the central MOH, decentralization of staff management and the creation of a new ‘district health office’ with a political rather than a medical technical function lead to role confusion and conflict. Mitigation Strategies: design of the new national PBF model for health centers take into account these administrative reforms; separation of functions: separating the audit function (new district health office) from the quality regulatory function (Hospital). Start Virtual Leadership Development and team building exercises between district health and hospital teams.

Results:

Essential elements for success were the ‘roll-out plan’ and strong and consistent leadership at the top level of the Ministry of Health. Performance improvements that have been documented in Rwanda after the introduction of performance incentives for primary health care and HIV/AIDS service products have been impressive. For instance:


113

For the ‘Cyangugu model’: results on primary health care indicators include increasing the number of new users from 0.31 consultations/capita/year in 2002 to 0.75 consultations/capita/yr in 2005; increasing the number of institutional deliveries from 27% in 2002 to 40% in 2005; increasing the percentage of fully vaccinated children from 70% in 2002 to 77% in 2005 and increasing the contraceptive prevalence rate from 0.44% in 2002 to 7% in 2005.

For the ‘Butare model’: Results include – from two different districts- an increase of new consultations from 0.36 and 0.47 in 2001 to 0.65 and 0.85 consultations/inhabitant/yr respectively, institutional deliveries from 7% and 13% in 2001 to 24% and 39% in 2005 respectively, and increasing the percentage of women with between 2 and 5 tetanus vaccinations from 32% and 47% in 2001 to 51% and 78% in 2005 respectively

Such performance improvements have not come by themselves. Frequently one refers to a so-called ‘black box’ of managerial entrepreneurship, which, guided by an ‘invisible hand’ of financial incentives, would lead to the right managerial decisions and devise the right managerial interventions to maximize benefits. However, this has not led to a ‘laissez faire’ approach in the successful PBF pilot programs in Rwanda. In Rwanda, supporting agencies assisted the service providers in devising successful strategies to boost health center productivity, either through in-built program components such as the ‘business plan’ or through facilitating regular exchange between service providers where providers could learn from each other and emulate successful interventions. A nationwide roll-out could be replicated, as was shown by the introduction of PBF in the Democratic Republic of Congo and Burundi. Essential tools used were: (i) study visits; (ii) consulting relevant literature; and (iii) mounting of a national workshop, or a

Rwanda

114

series of workshops at the national level, facilitated for decision-makers (it brought buy-in from politicians and technicians alike).

Conclusion:

The national roll-out of PBF in Rwanda has been impressive. This is an ongoing process of implementing what is known to work, examine results, and adapt the system accordingly. The decentralization in the near future of the PBF budgets to the districts will introduce a new element that could be vital in the successful nationwide introduction of PBF. ‘Managerial entrepreneurship’ cannot be managed centrally. Important next steps for the national roll-out include the introduction of a national PBF database that will assist the district steering committees and the national level in managing the national PBF model.15 The basic idea is to have the district health department and district hospital staffs do data entry through internet, in one central database managed by a professional service provider. This technical solution has been made possible by the introduction of third generation technology (GPRS) by one mobile phone provider. An excellent coverage of about 90% of Rwanda by the mobile phone provider, allows internet access at a slow but reasonable speed with a fairly stable connection. This database will allow users to input data (the results of the quantity verification by the USF, the results of the quality supervision by the district hospital), and to print their quarterly invoices for discussion in the quarterly PBF steering committee meetings. Furthermore, the database will have the possibility for district staff to analyze their own PBF results by an in-built pivot chart function.

15 http://www.pbfrwanda.org.rw, expected to be operational by the end of December 2006.


115

In addition to the above features, which will assist the district in managing PBF results and decision making around payments, and the national level in compiling and communicating results, the following modules will be added:

A Geographic Information System interface, which will allow presenting and analyzing data using maps, for use at the national level.

A module which will allow users at the national level to model indicators based on a given budget, using baselines from the previous year. Such a module can be invaluable for district level users also, when the PBF budget will be decentralized, which is planned to happen in the near future. District level users could then decide to use different unit values for indicators, depending on local decisions, rather than, which is the case currently, centralized decisions.

A module that will enable users to automate the writing of contracts, and their amendments, using data stored in the database to model future growth.

Sustainability of the new Rwandan PBF system has so far been good; first, it is embedded in the new national health policy, second, political support is strong, third, the government purchases , financed from the budget, a basic package of health indicators nationwide, fourth, various donors (USAID, WB/MAP, GF, national malaria control program, national tuberculosis control program, etc.) are purchasing performance indicators specific to HIV/AIDS, malaria and TB, using the same administrative model. The Government as a show of commitment, has recently increased considerably the budget allocated to purchasing basic health indicators.

Rwanda

116


117

Burkina Faso: Performance-Based Management: Case Study on the Monitoring and Evaluation Unit

of PNGT2 Bationo Begnadehi Claude, Socio-Economist, (Chief, Monitoring and Evaluation

Unit for the Community-Based Rural Development Project - PNGT2)

The Government of Burkina Faso’s Community-Based Rural Development Project – Phase 2 (PNGT2) demonstrates monitoring and evaluation (M&E) systems that have been developed at the local level. The program has national coverage and highlights a strong willingness to place M&E of development programs in the hands of those who benefit from the program’s activities. The outcomes achieved from the program are rolled up from the community level, making an impact on national policies and decision-making processes. It is thus a good example of grassroots citizen input leading to national initiatives to strengthen MfDR.

Introduction:

The Government of Burkina Faso’s first program on Community-Based Rural Development achieved important outcomes. As a result, an agreement was reached between the Government and its technical and financial partners1 on financing a second phase, PNGT2, which officially began on February 19, 2002. This second Community-Based Rural Development Project supports the National Program for Decentralized Rural Development (PNDRD). Its primary aim is to build the people’s capacity so that they can manage their own sustainable, equitable, and productive development and to facilitate implementation by rural communes. It is part of a comprehensive poverty reduction strategy developed by the Burkinabé Government. 1 Technical and financial partners include the World Bank (IDA), the International Fund for Agricultural Development (IFAD), the Global Environment Fund (GEF), the United Nations Development Program (UNDP), and the Danish cooperation authorities (DANIDA).

Burkina Faso

118

The overall aim of PNGT2 is to rapidly reduce rural poverty and to promote sustainable human development. To achieve this goal, the program has three specific objectives:

Build the organizational and management capacities of villages and groups of villages;

Improve the living conditions of rural populations through productive investments and socioeconomic infrastructures in the rural areas of Burkina Faso; and

Preserve and restore the natural resources of local territories (water, soil, vegetation, fauna).

The intervention strategy of PNGT2 is “learning by doing”. The program will be implemented over 15 years in three phases each lasting five years. PNGT2 covers all 45 provinces of Burkina Faso using two different intervention modes (direct and indirect). The direct intervention mode is used in 26 provinces, while the other 19 provinces are concerned with the concerted intervention mode. In the latter case, PNGT2 intervenes by making local investment funds available to other large-scale projects and programs already in place in the field. PNGT2 has made comprehensive use of the participatory monitoring tools designed for beneficiary communities to monitor and evaluate the implementation of project activities. This is consistent with the context of PNGT2, which is classified as a “decentralized and participatory development (DPD)” project. In Burkina Faso, one of the main tenets of the Decentralized Rural Development Policy Letter (LPDRD)2, adopted in July 2002, includes the complete accountability of village-level communities in the conduct of the development actions carried out at their level. The use of participatory monitoring tools contributes to building the capacities of these stakeholders and enhances their ability to

2 LPDRD: the reference document as regards decentralized and participatory rural development.


119

plan, monitor, and evaluate the activities. The advent of performance-based management (PBM) should be credited to the CIDA, the World Bank, and a number of other bilateral and multilateral partners. The Government of Burkina Faso adopted the approach after recognizing its relevance for the monitoring and evaluation of development policies, projects, and programs. The desire to harmonize management tools and methods to facilitate the aggregation of data and, in particular, the obligation to report on the management of official development assistance, guided the full acceptance of the principles of PBM by the Government of Burkina Faso.

Application:

PNGT2 developed participatory management and evaluation (PME) tools and made them available to village-level communities (beneficiaries). These PME tools are an integral part of the Monitoring and Evaluation Mechanism of PNGT2, which has two fully complementary subcomponents:

1. Monitoring of performance (execution); and

2. Monitoring of three impact areas: • Socioeconomic impact, that is, impact on living

conditions, incomes, and poverty in rural areas; • Institutional impact, or impact on decentralized

institutional capacities; and • Environmental or ecological impact.

Monitoring and evaluation in the PNGT2 context is designed to provide information to facilitate decision-making at three levels:

1. At the level of the state and technical and financial partners, in order to: • Ensure proper technical and financial execution; • Monitor developments in relation to the objectives set;

and • Provide information to external monitoring missions.

Burkina Faso

120

2. At the level of the program management structure, in order to: • Monitor implementation performance; and • Track the results produced by interventions.

3. At the level of the beneficiary communities, in order to: • Engage in the planning, monitoring, and evaluation of

the results of their achievements. The monitoring of PNGT2 execution makes it possible to periodically (each quarter) take stock of:

Progress of the physical and financial execution of the project;

The extent to which the physical and financial outcomes have been achieved; and

Any difficulties/problems encountered, corrective measures proposed, and adjustments made.

The impact monitoring is aimed at assessing the extent to which each of the various PNGT2 activities has achieved its development objectives (in terms of the primary impact indicators selected). From an institutional standpoint, the monitoring of program execution is carried out at two complementary and interdependent levels: central and decentralized.

The central level includes the Monitoring and Evaluation Unit, staffed by four senior personnel and three support staff. It is responsible for periodically centralizing all the data on PNGT2 execution and producing program progress reports. In addition, it contributes to the organization and conduct of the quarterly stock-taking and activity programming workshops. Finally, it works to build the capacities of the stakeholders involved in the implementation of the PNGT2 monitoring and evaluation system.


121

The decentralized level consists of the province and village levels. • Each of the 18 provincial coordination offices includes

one senior staff member responsible for monitoring and evaluation. The collection and forwarding of village-level data to the province level is performed by service providers previously trained under PNGT2 as part of village capacity building to master participatory monitoring tools (cascading training).

• At the village level, a participatory monitoring and evaluation mechanism is provided. Image-based tools have been made available to communities to enable them to plan, monitor, and evaluate the development activities in their territories. Each village-level commission (CVGT) has named three of its members to receive training in the use of these tools.

A monitoring and evaluation database (BDSE), initially developed in MS Access and subsequently in SQL-Server, should make it possible to capitalize on all the information on PNGT2 progress.

Three impact areas are monitored under PNGT2. These areas are: (i) socioeconomic; (ii) institutional; and (iii) environmental. Three senior personnel are responsible at the central level for each of the impact areas. Three national monitoring committees made up of all the partners involved in or concerned with the impacts have been established, one for each of the three areas mentioned, to participate in the validation of the various outcomes identified. The Permanent Secretariat of the National Framework for Concertation of Rural Development Partners (SP/CNCPDR) is expected to assume ownership of all the mechanisms developed by PNGT2 as well as their results so as to ensure their sustainability at the national level.

Burkina Faso

122

The overall approach for each of the three areas first involves preparing methodological and technical documentation. Then, these methodological and technical documents are used to conduct baseline studies, the results of which will serve as benchmarks for future assessments. To do so, a minimum battery of indicators has been defined and will be periodically updated and analyzed in order to determine the program’s impact in the area concerned.

Problem Solving:

The main lessons are:

The target population showed a genuinely keen interest in using the participatory monitoring tools for monitoring and controlling their activities;

The use of figures and images to symbolize the different periods of the year and to distinguish between programmed activities enabled the target populations to rapidly assume ownership of the contents of the participatory management tools, which also facilitated their use;

The PME tools are now an integral part of daily life in the villages, which have had the opportunity to experiment with them in action and "learn by doing";

The training and use of service providers as intermediaries between the village level and province level made it possible for PNGT2 to provide hands-on assistance to all the villages concerned;

The quality of the data collected is guaranteed when the people are empowered;

The low literacy level of the people delays mastery of the PME tools in some villages;

Establishing and parameterizing a database for monitoring decentralized and participatory development (DPD) projects has proven to be difficult and complex;


123

The cost of implementing the mechanism is high and includes stakeholder capacity building, hands-on monitoring, purchase of compatible computer hardware and equipment, etc.

Easing the various constraints requires substantial resources, in particular for:

Providing literacy training to the stakeholders concerned so as to help them assume ownership of the participatory monitoring tools;

Evaluating and providing each stakeholder with the financial resources and equipment essential to operationalize the mechanism;

Making financial resources available to stakeholders at the central and provincial levels to guarantee quality control of the information gathered and thereby ensure the credibility of the system as a whole; and

Identifying and setting the parameters for a simple and user-friendly database for storing and using the information on program execution.

Furthermore, the two components of the PNGT2 monitoring and evaluation mechanism were implemented in accordance with the monitoring and evaluation procedures manual. However, during program execution it was determined that one essential link, namely intermediate results, was missing. Performance monitoring is predicated on quantitative and qualitative aspects of project execution, and the fact that impact monitoring is national makes it impossible to single out the effects imputable strictly to PNGT2 activities. Accordingly, it was necessary to introduce a supplementary mechanism to identify these effects. The Monitoring and Evaluation Unit therefore developed a methodology for

Burkina Faso

124

monitoring these effects, the implementation of which targets the following objectives:

Assess the extent to which the achievements financed by PNGT2 contribute to realizing the program’s three specific objectives;

Generate the information necessary for assessing the PNGT2 contribution to poverty reduction in rural areas and to the promotion of sustainable development.

The first study on intermediate results was carried out in the first half of 2006.

Results:

In terms of results or gains, after five years of operation, the monitoring and evaluation mechanism of PNGT2 has made it possible on a regular basis to update the table of key PNGT2 performance indicators as well as to identify program impacts. The data of the main performance indicators from the start of the project through September 30, 2006 may be summarized as follows:

Training of over 16,000 village representatives (about seven percent of whom were women) in mastering and using the participatory management tools;

Effective coverage of 3,013 villages;

Support for the establishment of 2,961 CVGTs;

Support for the preparation of 2,986 Territory Management Plans (PGTs);

5,454,952 person/hours of training provided to the target villages on various topics (literacy training, participatory


125

monitoring and evaluation, procurement, financial management, agricultural production techniques, etc.);

Signing of 9,234 cofinancing agreements with CVGTs through the village-level window;

Effective transfer of CFAF 21,552,778,578 to the accounts of CVGTs for carrying out 15,197 microprojects through the village-level window, microprojects whose supervision is provided entirely by the beneficiary communities;

Effective signing of 79 agreements with Provincial Technical Coordination Committees (CCTPs);

Effective transfer of CFAF 2,961,466,878 for the financing of 299 structural microprojects; and

Established socioeconomic infrastructures deemed to be economically viable and technically valuable in the following proportions: • 100 percent for health infrastructures; 87.5 percent for

schools; 98.6 percent for nonformal education infrastructures; 91.8 percent for water supply works; 93.4 percent for storage infrastructures; and 98.7 percent for the vaccination centers;

• 83 percent of the improved bottomlands and 64.91 percent of the market garden perimeters are in use; and

• 97.5 percent of producers state that they have adopted water and soil conservation and soil defense and restoration (CES/DRS) techniques.

In respect to the three impact areas of PNGT2, progress as of March 31, 2006 may be summarized as follows:

Impact on rural living conditions, incomes, and poverty according to the results of the baseline study conducted in 2004:

Burkina Faso

126

• The per capita annual income in rural areas was evaluated at CFAF 53,762;

• The rural poverty line was evaluated at CFAF 41,884 - with this threshold, 41.58 percent of rural residents were classified as poor;

• 55 percent of households in rural areas had access to potable water; and

• Women had to travel an average of 1.8 kilometers to collect water.

Environmental impact according to the results of the baseline studies and evaluations: • The existence of three types of classified areas was

observed: (i) those characterized by considerable human encroachment (agricultural activities) (for example, in a single decade (1992-2002) about 30 percent of the area of the Dida classified forest became farmland); (ii) those where developments have been positive (regeneration) (for example, in 1992, 15 percent of the Maro classified forest was taken up by agricultural activities, as compared to four percent in 2002, due to interventions by PNGT2); and (iii) those where the situation has remained unchanged thanks to the actions of other parties, as in the case of the following forests: Nazinga Ranch, KABORE Tambi National Park, etc.

• Between 1992 and 2002, ten percent (1,444,316.3 hectares) of natural formations (savannah, steppes, forests) became farmland.

• There has been a gradual decrease in the areas burned annually by brush fires (5,818,618.75 hectares for crop season 2001/2002, 5,520,037.5 hectares in 2002/2003, and 4,601,668.75 hectares for 2003/2004).

• There has been a decrease in the volume of water drained from the main watersheds between 2003 and 2004. The reductions were as follows: 1,511.8 million cubic meters for the Comoé watershed, 2,799.8 million cubic meters for


127

the Mouhoun watershed, 690.5 million cubic meters for the Nakambé watershed, and 1,473.4 million cubic meters for the Niger watershed.

• An improvement in chlorophyllous activity in the Sahel and a decline in the level for the Central Plateau and the South Central region was observed.

• The situation as regards the area of bare ground remains unchanged.

• There has been an increase in the surface area of bodies of water from 78,311.3 hectares in 1992 to 104,695.54 hectares in 2002.

According to the results of the 2004 baseline study for institutional impact: • About 34 percent of national territory is structured around

CVGTs. Nearly 53 percent of the CVGTs were functional, based on the following criteria: (i) holding of general meetings; (ii) participation in the general meetings; (iii) collection of statutory contributions; (iv) execution of activities under the annual investment program; and (v) holding of office meetings.

• 52 percent of the Provincial Technical Committees (CCTPs) were functional, based on the following criteria: (i) holding of regular meetings; (ii) participation in sessions; (iii) member contributions to CCTP financing; and (iv) execution of the decisions taken by the CCTPs.

• The impact assessments that would have made it possible to measure the changes among these institutions have not been conducted due to institutional change at the village level.

• The national framework for concertation of rural development partners (CNCPDR) has been created and is operational.

According to the results of intermediary study No. 1 conducted in 2006 to assess the effects of PNGT2:

Burkina Faso

128

• Almost all of the socioeconomic and educational infrastructures established have management committees.

• The people’s access to basic social services has improved as a result of infrastructure efforts in the areas of basic education, health, and potable water.

• Moreover, productive investments (bottomlands, market gardening areas or perimeters, etc.) have enabled communities to diversify their income sources.

Furthermore, the results obtained in the impact areas are disseminated among policy makers as a tool to aid in decision-making. In terms of information use, the following actions are noteworthy: (i) the periodic forwarding of PNGT2 reports to the Directorate of Research and Planning of the ministry responsible for agriculture for purposes of consolidations to assess the progress of the Public Investment Program (PIP) from a sectoral standpoint (agriculture, water supply, and fisheries resources); (ii) use of data from PNGT2 and many other programs by the staff of the Office of the Prime Minister for purposes of preparing program statements and the state of nation delivered by the Prime Minister annually to the Burkinabé Parliament; and (iii) forwarding of the reports on performance monitoring to the technical and financial partners, but also to the technical ministries that have an interest in program results (Ministry of Environment and Living Conditions, Ministry of Economy and Development, Ministry of Finance and Budget, etc). At the program execution level, the various results obtained periodically are forwarded to the national administration through the Research and Planning Directorates of the ministerial departments, as well as to the technical and financial partners.

Conclusion:

The PNGT2 monitoring mechanism makes it possible to report on program performance. It is well suited to the participatory intervention mode because it involves and empowers stakeholders from the village level to the central level. The provincial databases


129

are provided with data and updated regularly. In light of the results obtained by PNGT2, the Government of Burkina Faso has expressed its intention to extend the second phase to the greatest possible extent to the entire national territory. Next steps include: 1) a second study on the monitoring and evaluation mechanism conducted in the first half of 2007; 2) concerning institutional impact, the current review of the methodological document to take into account changes relating to progress within the decentralization process as well as the social capital component; 3) completion in 2007 of a summary paper on the environmental impact in order to superimpose the various indicators that were monitored and thereby characterize the situation as regards brush fires, land usage, and plant cover in Burkina Faso; and 4) continued dissemination of the various results obtained in connection with the implementation of the monitoring and evaluation mechanism.

Burkina Faso

130


131

Benin: Experience with Results Based Management -The Case of the Ministry of the

Environment, Habitat, and Urbanism Mr. Pascal Z. Yaha (Directeur du Suivi des Projets Ministère du

Développement de l’Economie et des Finances)

Benin, country facing severe resource constraints, has nevertheless been able to use Results Based Management (RBM) for several years, specifically in the Ministry of Environment, Habitat, and Urbanism (MEHU). MEHU works closely with the Ministry of Finance and reports to it on budget execution and progress. RBM was adopted with the goal of allowing Benin’s various ministries to become directly involved in the preparation, implementation, execution, and monitoring and evaluation of program budgets in a framework focused on results. The MEHU faced obstacles, was able to overcome them, and has demonstrated strong progress in developing its results tracking systems in towns and villages.

Introduction:

Benin adopted the Performance-Based Management (PBM) reform in 1999 following a government seminar. The primary objective of the reform is to initiate a process of gradual transition from resource-based budget management to budget management focused on objectives. This transition will be achieved through (i) increased accountability of spending in ministries in the preparation, execution, and monitoring and evaluation of their Program Budgets (PBs) in the context of performance-based management; and (ii) the strategic, efficient, and equitable use of all public resources with a view to promoting growth and improving the living conditions of the people, and thereby reducing poverty in Benin. To enhance the likelihood of successful reform, a pilot phase was conducted involving the Ministry of National Education (MEN), the Ministry of Public Works and Transportation (MTPT),

Benin

132

the Ministry of Public Health (MSP), the Ministry of Rural Development (MDR), and the Ministry of Environment, Housing, and Urban Planning (MEHU). The MEHU voluntarily embraced this reform at the time of the government seminar for a number of reasons. Ever since its creation in July 1991, despite the efforts put forward by the government, the MEHU had experienced serious problems with advancing its various priorities. For instance, more than 86 percent of Benin’s arable land was degraded. Studies in air quality revealed that the cost of pollution in Cotonou had reached 1.2 percent of Benin’s GNP. Urban areas tended to see a greater increase in poverty, and in Benin more than 40 percent of the population was living in urban areas (which is projected to rise to 50 percent by 2010). Furthermore, there was a need for 6,000 housing units per year in the five largest cities in Benin. Benin also experienced a significant production of solid waste (342,000 metric tons in 1998, 60 percent from Cotonou alone). Finally, the MEHU felt it had a poor capacity for budget preparation to address these growing problems.

Application:

Implementation of budget reform began in 1999 with the preparation of the Program Budget (PB). This budget tool makes it possible to establish logical links between the sectoral objectives, actions, and program outcomes that need to be defined. Performance indicators play a key role in this approach, both as a monitoring and evaluation (M&E) tool and as information to be taken into account for resource allocation purposes. The first experience with the preparation of the PB considered the Effectiveness Outcomes and Performance Indicators as tools for the preparation and monitoring and evaluation of the budget. The strengths of this initial experience were (i) the MEHU’s ability to define a program structure that reflects the connections between its


133

activities and the specific objectives defined; and (ii) the presentation for each program of a logical framework interlinking the objectives, projects, activities, and indicators. The weaknesses included (i) the problem of ensuring consistency between existing policies and strategies, objectives, and resources for implementation; and (ii) the unreliability of the program expenditure framework (which spanned a one year time frame) setting projections for technical and financial activities.

Problem Solving:

To address these weaknesses, in 2002-2003, the PB was used to develop a Five-Year Strategic Plan (2002-2006), serving as a frame of reference for (i) enhancing the reliability of the forthcoming PB preparation exercises; (ii) providing MEHU with a solid basis for short, medium and long-term actions so as to better steer and enhance the effectiveness of government intervention on the ground; and (iii) consolidating the existing sub-sectoral policies and harmonizing the sectoral strategy with the National Long-Term Perspective Studies (NLTPS-Benin 2025), the Government Action Program (PAG), and the Poverty Reduction Strategy Paper (PRSP). Consequently, the policy objectives are broken down into actions in the PBs, which are now prepared every three years on the basis of the Strategic Plan and contain a program of actions used as the basis for selecting the activities to be executed. At the same time, a study was launched on the organizational and institutional audit of the MEHU to address all the reforms initiated in the country (budget reform, administrative reforms, and decentralization/deconcentration). The findings of this study were taken into account in the preparation of the PB. The PB preparation process was strengthened by the introduction of the Medium-Term Expenditure Framework (MTEF) used as budgetary support for the PRSP, which first occurred in the PB for 2002-2004. It serves as the frame of reference for the programming

Benin

134

of budget expenditure. Subsequently, the budget allowance has been made available to the MEHU, which serves as the payment authorizing entity under the auspices of a Delegate of the Financial Comptroller in the Ministry of Finance. In addition, a number of associated measures were taken, not solely to strengthen the PB preparation exercise, but also to strengthen all the mechanisms put in place in the context of the budget reform. To make it operational, the annual tranche for the PB is broken down into an Annual Work Plan (PTA), an Appropriation Consumption Plan (PCC), and a Procurement Plan (PPM) (see Annex). To meet the requirements of the budget reform, the MEHU strengthened its programming and M&E system by the creation of (i) special units, namely Monitoring and Evaluation Units (CSEs), Procurement Units (CPMs), and Communication Units (CCOMs), staffed exclusively by qualified professionals and provided with the necessary material and financial resources; (ii) sub-units within the technical structures, devoted to programming and M&E and headed by focal points recruited or designated for the purpose; and (iii) the naming of one Supervisor per program as part of the Council of the Office of the Minister (see organization chart). The tools necessary for the system to function are prepared on the basis of the PB, and primarily include:

The Annual Work Plan (PTA) or Annual Operationalization Plan, a technical reference paper for implementation and monitoring and evaluation, which includes the activities and tasks, their timetable and weighting, the parties responsible, and the modes of execution;

The projected Appropriation Consumption Plan (PCC) outlining the monthly disbursements to be made for each project or action;


135

The Procurement Plan (PPM) setting forth the planning for all contracts to be awarded in connection with executing the various projects listed for the year;

The mission letters sent by the Minister to the heads of all structures (whether or not they manage appropriations) and to all the structure supervisors (members of the Minister’s Office) to identify the mission assigned for the year and the criteria for assessing its implementation;

The monthly and quarterly PTA execution report statements;

The assessment letters, from the Minister to each structure head, in which a qualitative and quantitative assessment of the missions assigned to that head for the year is provided (see annex); and

The mid-year report on monitoring and evaluation of the PB examining the level of physical and financial execution of the various programs and the performance achieved during a half-year period.

Furthermore, additional measures were taken through the implementation of a Capacity Building Program financed with budgetary support, which focused on (i) personnel recruitment and redeployment; (ii) equipment; (iii) the organization of various training modules; (iv) the conduct of targeted studies and surveys (for example, a budget management procedures manual and establishment of a statistical data collection, analysis, treatment, and publication system); and (v) computerization/networking/training (for example, there was a 537 percent increase in the installed base of computers and the Intranet network, with an increase from 30 microcomputers in 1999 to 161 in 2003(71 are connected to the Intranet network that has been functional since 2001)). To ensure efficient management of the system, the following concerted action frameworks have been established:

Benin

136

The Council of the Office of the Minister, which meets every Monday and conducts a monthly review of the MEHU’s PTA;

Quarterly reviews of the execution of the PTA chaired by the Minister in person;

Management Committees, which meet every Thursday within each structure (weekly review of the PTA);

The Board of Directors, which meets every two weeks;

Monthly meetings of superintendents; and

Quarterly meetings of the Investment Programming and Monitoring Committee (CPSI) of the MEHU.

All of these concerted action frameworks are subject to reports that identify tasks and instructions as well as deadlines for improving the performance achieved or modifying the approach. The DIVI and the Chamber of Accounts also conduct audits of the mid-year and annual PB execution reports and make recommendations that must be implemented. The implementation of the MEHU M&E system presented is possible because of the institutional framework for monitoring and evaluation described above, which is composed of the Monitoring and Evaluation Units and Sub-units and the procurement unit under the attentive supervision of the Office of the Minister.

Results achieved over a five-year period (2000-2004):

Administration

The budget tripled between 1999 and 2004 to over CFAF 29 billion in 2004;

MEHU’s Priority Investment Plan (PIP) for 2004 represents 14.51 percent of the national PIP (as compared to 3.9 percent in 1999);

A sectoral guide or “KIT” on partnership with the communes has been prepared, which clearly indicates and outlines the


137

types of partnerships that should exist between the communes and the MEHU;

The update of the Strategic Plan 2002-2006 has taken place to reflect ongoing changes (decentralization, deconcentration, budget reforms, and PRSC);

There has been a 537 percent increase in the installed base of computers and the Intranet network, with an increase from 30 microcomputers in 1999 to 161 in 2003, (71 are connected to the Intranet network that has been functional since 2001);

Strategic management and the adoption of management by objectives at all levels of Ministry structures have been introduced;

The services rendered to the people have been improved through the creation and operationalization of a users’ bureau;

Productivity (PIP executed per staff member) improved from CFAF 43.6 million in 1999 to CFAF 70.5 million in 2003; and

MEHU’s FY 2003 Budget was more than 95 percent executed as of December 31, 2003.

Environment/Territorial Development

Framework law translated into seven national languages (Fon, Baatonu, Adja, Yoruba, Ditamari, Dendi, and Peulh);

Environmental education in primary and secondary schools and experimentation with an instructional package in six pilot secondary schools since October 2002;

Environmental impact assessments (EIAs) for all new projects (78 cases assessed and 61 impact assessments validated with certificates of environmental conformity from 1997 to 2002);

Audits of certain industrial sub-sectors and/or facilities (over 25 factories audited from 1997 to 2002);

Benin

138

Construction of two sub-units for the waste sludge treatment plant at Parakou;

Creation, installation, and outfitting of environmental units in the ministries, with the basic mission of ensuring the inclusion of environmental dimensions in development policies, plans, programs, and projects (units in place in five ministries [Transport, Energy; Finance, Commerce, Security; Planning]);

Validation of 77 environmental impact assessments;

Monitoring of ten environmental management plans;

Monitoring and auditing of ten industrial units; and

Training of 172 persons in environmental assessment. Environmental Police:

12 new officers sworn in on August 26, 2002;

2,664 inspections in 2001 (222 inspections per month);

622 inspections from January to June 2002 (103 inspections per month);

Organization of Monthly Environmental Health Days (JMESs) beginning in March 2000; and

Adoption of the Strategy to Combat Atmospheric Pollution in August 2000.

Regulation:

Decrees & Interministerial Orders passed that established air quality standards in the Republic of Benin; regulated the import, marketing, and distribution of used materials and capital goods; established the accepted percentage of motor oil in the fuel of two-wheeled vehicles (six percent); regulated the import of substances that deplete the ozone layer and devices and equipment that use such substances;


139

Training of 292 automobile and motorcycle mechanics in the departments of Benin;

Inspection of 4,658 motor vehicles in 44 sessions from June 2003 to December 2003;

Decline in the percentage of four-wheeled vehicles with higher polluting levels from 62 percent to 37 percent (72 percent to 43 percent for two-wheeled vehicles) from 2002 to 2003;

Construction and observation of 87 second order geodesic markers to enhance the density of the network of first order markers;

Implementation of the Green Corridor Project in the ten Urban Districts (CUs) of Benin;

Reforestation efforts resulting in one million plantings from 1998 to 2001; and

Municipal Environmental Action Plan (PMAE) implemented in 24 secondary cities from 1997 to 2001 (including 19 cities starting in 1999) together with 1,326 micro-projects (217 of which are priority projects implemented to benefit grassroots populations).

Urban Planning/Sanitation

Roads and sanitation: • Roads and sanitation works have been completed in 29

cities: Kouandé, Bassila, Ouaké, Banikoara, Djougou, Kandi, Savè, Savalou, Glazoué, Aplahoué, Azovè, Comé, Dogbo, Grand-Popo, Bopa, Pobè, Avrankou, Adjohoun, Kétou, Sakété, Dassa-Zoumè, Sakété, Ouidah, Allada, Cotonou, Dassa-Zoumè, Abomey, Parakou, and Porto-Novo. These works concern the paving and drainage of about 80.404 km of roadways, the construction of 83.267 km of gutters and 19.28 km of collector sewers, drainage and resurfacing of 5.623 km of roadways, and the paving of 10.672 km (Parakou).

Benin

140

Real estate promotion: • Improvement of peri- urban land at Abomey

Calavi (50 hectares) and Agblangandan on the former firing range (105 hectares) for the construction of low-cost housing (with water and electricity connections, telephone, and primary and secondary roads) for a total cost of about CFAF six billion;

• Two real estate promoters selected by competitive bidding engaged in housing programs in Abomey Calavi: (i) the BETSAEEL BUILDING Group (GBB) S.A., SPI1 and SPI 2 (1,100 housing units planned; 169 units started and 28 completed); and (ii) the BENIN KASHER Group (BK) S.A.R.L., SPI3 (600 housing units planned; 75 units started and 4 completed);

• 2,000 housing units planned in Agblangandan: four separate lots awarded to two real estate promoters through a competitive bid (3,803 housing units committed in total); and

• Completion of studies on the project to build the administrative tower in Cotonou and the headquarters of various institutions (National Assembly, High Audiovisual and Communications Authority, Economic and Social Council, Constitutional Court, High Chancellery, MCRI-SCBE and High Court of Justice) as part of the Special Program for Rehabilitation of the City of Porto-Novo.

Stakeholders

The Minister of the Environment, Habitat and Urbanism provides supervision, acts as the Chair of Budget Arbitration in the Ministry, and Chair of quarterly monitoring meetings;

The Members of the Office of the Minister (Chief of Staff, Deputy Chief of Staff, technical advisors), designated as program supervisors under the PB, are responsible for approval, facilitation, and monitoring;


141

The Secretary-General of the MEHU acts as the Coordinator of the Performance-Based Management Reform and prepares and implements the Procurement Plan (PPM);

The Director of Programming and Forecasting (DPP) focuses on programming and M&E, coordination of the preparation of the PB and the PTA, and preparation of the PB Performance Reports;

The Members of the Monitoring and Evaluation Unit (directly reporting to the DPP and headed by a team of three qualified specialists) monitor the PTA execution and are responsible for field visits to inspect physical achievements;

The Chief of the Statistical Office contributes to preparing the performance indicators and measuring them by means of targeted socioeconomic surveys;

The Central Directors and Deconcentrated Directors produce their respective program budgets, PTAs, and PPMs with assistance from their Focal Points (economic planners or statisticians), who are correspondents with the DPP and the Monitoring and Evaluation Unit;

The Development Project and Program Coordinators, or Technical Directors, report monthly to the DPP on the progress of their respective projects using monitoring sheets;

The Director of Internal Inspection and Audit (DIVI) conducts an internal audit of the PB performance report of the MEHU as prepared by the DPP;

The Auditors of the Supreme Court’s Chamber of Accounts receive the Performance Report and perform an external audit thereof for final approval or possible corrections; and

The Chief of the Ministry’s Communications Unit publicizes the achievements of the MEHU and conducts opinion surveys used to improve performance.

Benin

142

Conclusion:

This program has obtained results gradually, and its success is attributable to unfailing support at the highest level of the MEHU (the Minister himself). To ensure the sustainability of the system, it is important to avoid inconsistency, which might jeopardize the participation of the main implementation stakeholders. The Secretary-General of the Ministry, generally serving a five-year term of service, should be given sufficient accountability to serve as the operational linchpin in the system so as to avoid any damage in the event of a change in Minister. The implementation of the budget reform enabled the MEHU to build its capacities by permitting it to transition gradually from a resource-based budget to a performance-based budget. The M&E system made an enormous contribution to MEHU’s success in this program, as it is a powerful tool for assessing program and decision-making performance in regards to the operationalization of ministry structures. The development of such a Program Budget system is an essential precondition for success in performance-based management.


143

Emerging Good Practice of Managing for Development

Results in Civil Society & the Private Sector

Civil society’s involvement in MfDR encourages transparency in the public sector by stimulating a demand for public sector results from civil society who often at least claim to embody citizen voice. NGOs, CBO, and the private sector are all partners in international development and have a strong role to play in Aid Effectiveness and MfDR. Civil society stimulates and triggers civic engagement, and can catalyze a culture of results within the country’s populations at large. They also represent development partners using MfDR in their work on development projects and programs. Interestingly, managing for development results finds its origins in the private sector: “While MfDR is a new concept in the development community, the core concept is well established and has been applied in many public and private organizations in the last two decades. These other organizations have used terms such as ‘results-based management’ ‘performance management’, and ‘managing for outcomes’. The core of these diverse concepts is basically the same… Results management has its roots in business management theories, applied social research, program evaluation, and expenditure management. The approach, initially applied in private sector organizations, moved quickly to the public sector as part of reform efforts in the 1980s and 1990s.”1

Often, however, the civil society group is not actively considered in discussions on MfDR. They tend to be limited to the public section. This section highlights some of the ongoing work of civil

1 Asian Development Bank. Introduction to Results Management: Principles, Implications, and Applications. ADB: 2006,

IV

Civil Society & the Private Sector

144

society and the private sector in MfDR. It provides examples of where civil society and the private sector are actively engaged in creating culture of managing for development results and/or are implementing MfDR techniques through their work in international development. Specifically, the cases demonstrate that:

Civil society & the private sector are important partners in international development and have a strong role to play in Aid Effectiveness and MfDR

Furthermore, “an outcome approach [results approach] requires a strategic focus central to the raison d’être of government and directly connected to something that matters to the citizenry, such as a focus on poverty reduction or democracy.”2 At the Third International Roundtable on Managing for Development Results in Hanoi, Vietnam, Kumi Naidoo, Secretary General of CIVICUS, stated: “Successful management of aid resources for development results requires that civil society not be merely a mobilizer of local groups and individuals, nor even a “watchdog” on outcomes, but that it also play an integral part in decision making at all levels. Civil society must go beyond acting as a catalyst of change at the local level to include pressing for change in power structures to enable marginalized groups to play a greater role in influencing decisions that affect their lives.” In Mongolia, for example, the Open Society Forum created a “National Development Dialogue” where experts discuss and contribute to the evaluation of national economic policies.

2 Perrin, Bruce. Moving from Outputs to Outcomes: Practical Advice from Governments Around the World. Based upon the Roundtable “Moving from Outputs to Outcomes: Implications for Public Sector Management” December 15–16, 2004. Sponsored by The World Bank and the IBM Center for The Business of Government.


145

Mongolians of different age groups also used the National Development Dialogue to share scenarios and spark debate on Mongolia’s future. Local Ecuadorian civil society, in collaboration with Geneva Global and USAID, used a simplified MfDR framework, in their efforts to reduce human trafficking. And two years since the deadly tsunami, communities are using MfDR in their based reconstruction projects. These cases demonstrate the expansion of MfDR beyond the public sector, and how civil society and the private sector are playing a key role in building a MfDR culture. Furthermore, companies like Pfizer are using results based management to improve the free distribution of Diflucan in Tanzania and Zambia. By using a results approach, Pfizer and its partner, Interchurch Medical Assistance, have seen an increase in reporting on the use of Diflucan; improved country wide product deliver; and expanded product access. The Aceh & Nias case of community reconstruction after the Tsunami demonstrates how vital the role of civil society can be, especially when government is overwhelmed by the magnitude of such a disaster. The NGOs played a crucial role in delivering outputs that made a difference in the daily lives of people struggling to rebuild their homes and find an economic livelihood again.

Civil Society & the Private Sector

146


147

Mongolia: The Role of Civil Society in MfDR Jargalsaikhan Dambadarjaa (Open Society Forum - Mongolia)

As a result of frequent organizational structure changes, short-term thinking, little accountability on the part of the public leaders, and the absence of performance measuring systems, Mongolia has faced many challenges in its transition from a centrally planned economy to a market economy. Initiatives such as the National Development Dialogue, the Social Auditing Pilot Model for Participatory Assessment and Monitoring, and the World Bank’s Public Expenditure Tracking Survey, the Open Society Forum (OSF) is helping to build a successful national dialogue and shared vision for Mongolia’s future. The active role of civil society is a key factor in the success of these initiatives. This requires informed citizen participation in policy formulation, implementation and monitoring processes of public management.

Introduction:

Over the past 15 years, Mongolia has experienced a transition from a centrally planned economy to a market economy. The country has faced challenges in managing for results because of rapidly changing national development strategies, minimal accountability of public leaders, and unpredictable and frequent changes in public management mechanisms such as organizational structure, legal process and administrative environment. The chaotic organizational culture creates uncertainty and short-term thinking on the part of the private sector, which in turn limits strategic thinking and major foreign and domestic investment. Poor performance in the public management system is also caused by the virtual absence of any type of performance measuring system. The fact that public officers receive low salaries reduces incentives for efficiency and increases corruption. When Mongolian officials do report on the progress of government

Mongolia

148

projects, the amount of money spent is the primary measurement. This type of evaluation reflects a digression to the use of the monitoring methods of a centrally planned economy where the primary government indicator of performance is the number of goods and services produced. Despite these challenges, Mongolia’s civil society has been actively engaged in public management reform and maintains that the country could create a more successful transition if the leaders and citizens efficiently joined their efforts to achieve development results.

Application:

A significant number of non-governmental organizations (NGOs) are involved in addressing the issues described above. One the most well-known organizations, the Open Society Forum (OSF), aims to support the active participation of informed citizens in forming public policies as well as monitoring and implementing the progress of these programs. The OSF has developed a number of programs in the past, and the following sections demonstrate the organization’s commitment to this issue. National Economic Development Strategy Planning The prerequisite to a successful strategy for economic development is the commitment by private sector and civil society stakeholders to outline a shared vision of the country’s future. The OSF established the “National Development Dialogue,” a forum for experts to regularly discuss national issues and contribute to the identification and evaluation of critical economic policies. One of the programs under this new initiative was a series of lectures and discussions led by economists and targeted to Mongolian policy makers and others in the policy community. In 2005, the OSF co-hosted two separate events in Mongolia featuring two Nobel Prize winner economists, Robert Mundell and Professor Joseph Stiglitz.


149

The results of another program, a public discussion of future national development scenarios, will be published as a journal. Mongolians of different age groups and experiences created the scenarios as articles, to be published in magazine format worldwide in the year 2025. The contents of the journal are simple and presented in an entertaining manner. In parallel, the OSF hosted a web page for these scenarios, which initiated further online discussions and debates. The third program under the initiative was the development of a web portal, which has since transformed into one of the main tools to directly reach the public on this issue. On this website (see: www.opensocietyforum.mn ), the OSF posts articles on policy issues written by leading economists and enables the web-audience to post their own comments.

Problem Solving:

Public Management Performance Monitoring In Mongolia, the initial stage of the integration of performance quality measurement into public management is characterized by a shift from outputs to outcomes. Civil society’s first priority was to gather all available public sector management information and make it accessible to society, in other words, create public governance transparency. This information includes key economic issues, projects, influences on and processes of decision making, names of people involved, size of budgets and final project outcomes. The OSF supports this initiative to provide broad public access to information resources about policies, laws and regulations. It has created a physical and virtual space for quality research and analysis of policy and governance in the economic and social development fields. As a result of many years of advocacy by the OSF and other civil society organizations, the Mongolian Parliament adopted the

Mongolia

150

“Public Service Broadcasting (PSB)” law in January 2005, and the former state-owned National Radio and TV service was converted into a public broadcasting entity. Civil society’s second priority has been to enforce accountability of local and central government officials. To address this issue, the OSF initiated and developed a coalition of organizations to promote open and responsible budgeting processes. The entire budget framework statement document for 2006 was published for the first time on the OSF website. In addition, the OSF organized the first public debate on the government budget with the Minister of Finance, civil society, donor institutions and local province representatives. The OSF also co-convened monthly meetings on the harmonization of donor activities focused on small and medium size enterprises (SMEs) and the development of micro-credit. Related officials were required to make progress reports in terms of outcomes rather than outputs. Finally, a memorandum of understanding between the Ministry of Finance, the Parliament Standing Committee and the OSF on regular releases of detailed budget information, cooperation in the area of public discussions, and analysis of budget documents is planned. Individual Project Monitoring Secondary Education Budget Monitoring The OSF, in collaboration with the World Bank, analyzed the educational sector’s budget spending in Mongolia, focusing on the secondary education sub-sector. The Public Expenditure Tracking Survey (PETS) method developed by the World Bank was applied to assess transparency of the budgeting process (e.g., public access to budget information). The goal of the exercise was to identify misuse of educational funds, which causes delays and hampers effective delivery of education services in urban and rural schools. Budget data was collected from local education authorities, 118 randomly selected schools (representing a national pool of over 700 schools), and the Ministry of Education. Extensive and in-


151

depth analytical reports were prepared around themes central to education budget programming such as education budget laws and regulations, funds allocation (salary and non-salary expenditure by schools), and budgeting processes (budget submission, approval and disbursement processes). The report will be used as a basis for public discussions on transparency in the educational finance system, which is necessary to create public awareness and participatory opportunities. The database compiled under this initiative was the first collection of school budget information on a national level, which will be available to the public online. The final report was released in the fall of 2006 in English and Mongolian. The results of this research are expected to have a visible impact on the education sector’s decision-making, especially in the area of participatory budget spending at school levels as well as some impact on resource allocation decisions at the central level. For example, the study shows that 80 percent of the school system’s fixed costs goes to heating buildings, and 70 percent of the variable costs per student goes to teacher salaries. This study also helped the OSF to learn directly about the application of international-level research tools and gain skills in implementing budget tracking studies. This experience will allow the OSF to apply the same skills in research of other social sectors (e.g. health) and implement activities aimed at monitoring the allocation and expenditure of public funds. Social Service Quality Monitoring Project The OSF initiated the Social Auditing Pilot Model for Participatory Assessment and the Monitoring of the Quality of Social Services by collecting relevant data. The key challenge in the assessment of social services is the lack of good analytical studies and systemic evaluations. Since the public has limited information and knowledge on social service performance, there is not a clear demand for improvement. In order to present an evaluation of the existing situation and offer alternative policy options, the OSF

Mongolia

152

commissioned a study in one of the Ulaanbaatar city districts in Mongolia. Songino-Khairkhan (SK) is a major in-migration area of Ulaanbaatar. In this district, apartment houses and “gers”(nomads’ houses) are predominantly populated by migrants from rural areas. The OSF research covered issues related to living conditions, access to education, health and administrative services in SK. Important problems included confusing census data from various sources, lack of registration of migrants, and limited access to safe drinking water, public schools, and police and health services.

Results:

Civil society is an integral player in public management performance monitoring.

Information monitoring should be disseminated to all levels of society, and community MfDR practitioners should make the data available to their districts.

Society’s capacity to create broad dialogue on public management outcomes is essential to success.

Cooperation with government institutions is also important to ensure accurate monitoring.

Multilateral and bilateral institutions have rich experiences in implementing MfDR systems.

Monitoring of the public performance is possible only when the government is fully committed.

Incentives for MfDR must be created by civil society.

Monitoring of public management performance is a large, expensive and long-term job.


153

Conclusion:

The OSF is interested in establishing a local community of practitioners, including individual researchers, policy makers, and academic, public organization and NGO representatives, to focus on achieving nation-wide development results. The purpose of this community of practice (CoP) will be to more effectively involve communities in project implementation and monitoring. In the development of this program, the OSF must recognize that it is critical for every individual stakeholder to contribute to the assessment of public management performance at local, regional and national levels, and that the integration of performance measures into public budgeting and management should be done on a step-by-step basis. At the beginning, the initiative will focus on specific development program issues, and the local community will be a "project and program monitoring body" (PPMB). The first immediate action will be to train NGOs on result-based management and monitoring concepts. The training will focus on community participation in project implementation and project monitoring. The CoP will have a Mongolian and English language website that will be updated daily with NGO information and comments. It will focus on project outcomes through performance measurement and related programs. The website, a visible interface, will display well-structured information with a creative and efficient database to avoid the risk of becoming a chat-room. Participants would be the actual beneficiaries of international and bilateral projects and local civil societies. (I don’t understand this sentence) Finally this community will be responsible for incorporating performance indicators in project creation; examining how aid resources are received, included in the budget and used; and identifying the person in charge of measuring the end results.

Mongolia

154


155

Ecuador: U.S. Agency for International Development (USAID)/Geneva Global Initiative: “The Time is Now, Strategically Mobilizing Anti-

Trafficking Organizations in Ecuador” Timothy Ogden (Geneva Global Inc.)

USAID and Geneva Global (contractor) recognized that in order to invigorate anti-trafficking efforts by local civil society in Ecuador, the local organizations had to create their own goals and objectives. To assess its projects, including the development of anti-trafficking programs, Geneva Global requires non-governmental organizations (NGOs) in the recipient countries to use a simplified results-based framework -- “Need-Program-Results” (NPR), to ensure that the project managers have identified the local needs appropriately, defined expected results, and designed clear means for measuring effective progress. Good project management using this results approach played a vital role in enabling the anti-trafficking program implementers to achieve higher-than-expected results in Ecuador.

Introduction:

Trafficking in persons is a criminal activity that occurs throughout the world. The United Nations Office on Drugs and Crime (UNODC) has found that more than 130 countries are sources or recipients of trafficked persons. Trafficking in persons has been a particular problem in Ecuador: the country serves as a transit point for international trafficking rings and a source of internationally and domestically trafficked victims. In 2003, the ILO estimated that 5,000 children in Ecuador were forced into prostitution. However, as trafficking is a criminal activity, its scope is difficult to assess. Measuring the trafficking problem in Ecuador was especially complicated for several reasons. First, there was no common terminology to describe the issue. Additionally, in rural and indigenous areas, traffickers exploited the cultural practice of placing children with distant relatives to learn skills or gain access

Ecuador

156

to paid work by preying on these children away from the watchful eyes of their parents. Finally, while Ecuador was one of the first countries to subscribe to an international agreement addressing the fight against human trafficking, “The Protocol to Prevent, to Repress and to Punish Human Trafficking, Particularly of Women and Children” (known as the Protocol of Palermo), the country did not have an appropriate domestic legal framework to criminalize and prosecute trafficking in persons. As a result, in 2005, Ecuador was classified as a Tier 3 country defined as “not fully complying with the minimum standards for the elimination of trafficking and not making significant efforts to do so.” In 2005, Geneva Global Inc. and the U.S. Agency for International Development (USAID) entered into a joint venture to address the issue of trafficking in persons across Ecuador. Geneva Global is a professional services firm that provides independent research, advice and grant management services to private and technical donors investing in the world’s developing nations. To assess all projects it researches, Geneva Global requires nongovernmental organizations (NGOs) to use a simplified Managing for Development Results (MfDR) framework to ensure that projects have appropriately identified local needs, defined expected results and designed means for measuring progress. Geneva Global calls this approach “Need – Program – Results” (NPR). Simply, an NGO must outline the underlying needs in a community, articulate a program that clearly is targeted to meet those needs, and define the measurable results expected to address the need and its root causes. Geneva Global applied the NPR framework to the joint anti-trafficking initiative in Ecuador.

Application:

Geneva Global and USAID recognized that in order to invigorate anti-trafficking efforts by local civil society in Ecuador, the local civil society organizations had to create their own goals and objectives. Therefore, these two organizations convened a series of


157

meetings and conferences for interested private organizations and government institutions. Nearly 30 NGOs operating in Ecuador and dealing with various aspects of child labor, child and family issues, education and trafficking attended the meetings. During these meetings and conferences, local NGOs were trained on NPR framework basics. The forums also strived to form a general consensus on the scope and definition of trafficking in Ecuador. This process helped ensure agreement among NGOs on program priorities and demonstrate the relation among various proposed projects within the anti-trafficking initiative. Based on these meetings, individual NGOs were able to decide whether they were interested in participating in the initiative and, if so, learn how to propose a project. A diverse portfolio of 11 projects from 11 implementers was developed and selected for funding by USAID and Geneva Global. The projects covered a broad geographical base with interventions divided into three focus areas: prevention, rescue and rehabilitation, and legislation and legal sponsorship. The main focus was on prevention, especially education about trafficking. This priority was established because the local organizations agreed that the Ecuadorian society had little knowledge of trafficking. Long-term success required helping society understand the problem of trafficking, its scope in Ecuador, and the price that society as a whole was paying because of rampant trafficking in persons, especially minors. Below are the agreed upon objectives (in aggregate for all 11 projects) across the three focus areas: Prevention Human trafficking prevention efforts target people who are in danger of falling victim to traffickers, as well as people who are in the best position to help potential trafficking victims. Interventions include prevention workshops, training seminars and collaborative work between organizations. The following benchmarks were

Ecuador

158

agreed upon to gauge the progress of human trafficking prevention projects: Objective:

40,000 to 60,000 potential victims will be educated directly via schools, communities and workshops and indirectly through media campaigns about the risk of falling prey to trafficking. This dissemination of information on how to identify and avoid deception will significantly reduce the number of potential victims being lured into dangerous situations.

Objective:

10,000 to 14,000 community members, teachers and leaders will be educated about the danger of trafficking. All of these individuals will then be mobilized to educate others and prevent girls and young women from falling pray to traffickers.

Objective:

1,500 to 2,000 victims of trafficking, at-risk individuals and providers for these individuals will receive job and skills training and economic aid to supply them with options for earning money legitimately so that they are less vulnerable to trafficking.

Rescue and Rehabilitation Victims of human trafficking require significant psychological, medical and legal assistance to overcome trauma. Additionally, cultural stigma can hinder their reintegration into society. One of the priorities of the USAID and Geneva Global initiative is to ensure that victims are carefully nurtured and provided with consistent psychological care.


159

Objective:

1,000 to 1,500 victimized children and young women will receive personalized psychological and medical care to recover from sustained abuse. These services, in addition to critical skills training, will enable these children and women to re-enter society.

Legislation and Legal Sponsorship When this initiative was launched, Ecuador did not have the infrastructure to control and stifle the trafficking industry. Ecuadorian NGOs funded through the program independently decided that a key objective of the initiative would be to advocate for acceptance of anti-trafficking legislation by the Ecuadorian legislature. Additionally, they planned ways to work alongside police officials and lawyers so anti-trafficking laws could be fully implemented and enforced. Legislation Objective:

150 to 200 officials and lawyers will receive training in effectively combating trafficking through legislation. In the long run, these interventions will lead to legal, political and cultural improvements in Ecuador’s approach to trafficking.

There will be advocacy efforts to adopt laws within the Ecuadorian legislature to enforce international anti-trafficking conventions that Ecuador already supports on a global level.

Legal Sponsorship Objective:

250 to 300 female victims of human trafficking will receive direct legal assistance that will lead to the mobilization of the justice system and increased prosecutions.

Problem Solving:

Trafficking is a criminal activity and there is limited knowledge of the scope of the overall problem; therefore, one of the major difficulties in developing anti-trafficking initiatives was identifying

Ecuador

160

useable benchmarks and measurable outcomes. As a result of the general lack of understanding about the issue as well as schemes to traffic children without the knowledge of their parents, education became a priority for many of the local NGOs. Educating the general public about the nature and structure of trafficking in Ecuador was not only an effective strategy to address the problem; the results of education efforts became central to generating future surveys and measurements that could provide reliable data on the success of the initiatives. In order to capture this data, most projects focused measurements on outputs that could be reasonably expected. For example, rather than trying to measure the direct impact of education on trafficking, initiatives focused on the quality of the education individuals received. One measurement was a test given to all education seminar attendees after the sessions to assess their knowledge and understanding of the issue. These types of measurements were designed to give the evaluators a true sense of attendee learning and understanding, as opposed to providing meaningless data, such as numbers of individuals attending workshops or classroom sessions.

Results:

Prevention Results:

71,966 people were educated about the risks of human trafficking through various mass media campaigns. The original goal was exceeded by 120%.

251,961 community members, including teachers and leaders, learned about the dangers of human trafficking and how to prevent it in their communities. This goal was exceeded by 1,799%.

1,632 trafficking victims and their providers received skills training and orientation in order to earn money legitimately. This goal was exceeded by 82%.


161

An unexpected outcome of this initiative was the media coverage it received. While trafficking has not been commonly discussed in Ecuador, the issue is slowly gaining general attention.

Rescue Results:

2,521 victimized children and young women received personalized psychological and medical care to recover from sustained abuse. They also received orientation and support to develop skills in order to become less vulnerable to human trafficking. This goal was exceeded by 168%.

Legislation and Legal Sponsorship Results:

In May 2005, thanks to the work of civil society groups, including some of the organizations implementing this initiative, the Congress of Ecuador approved the “Ley Reformatoria al Código Penal” for sexual exploitation crimes against minors. This law created criminal proceedings and sentencing guidelines for different types of human trafficking offenses (prostitution of minors, sexual exploitation, child pornography, sexual tourism etc.), and was signed by the President and published in the Official Registration in June 2005. This is landmark legislation in the battle against trafficking in persons.

2,245 social leaders received training on the “Ley Reformatoria al Código Penal” and the “Código de la Niñez y Adolescencia.” They also received training on the protocol for reporting trafficking incidents.

131 victims of human trafficking, primarily women and children, received legal support from the lawyers of participating organizations.

Ecuador

162

Through this alliance, an effective legal sponsorship has been established that positions participating implementers as pioneers in sponsoring cases for victims of human trafficking, particularly sexual exploitation cases, in the Ecuadorian courts. To date, seven cases have been successfully prosecuted by the courts and have become precedents within Ecuadorian jurisprudence.

Conclusion:

Strategic Alliances USAID and Geneva Global encouraged and nurtured strategic alliances among implementers and other institutions involved in anti-trafficking endeavors. One of the major focuses of the Geneva Global field staff has been to encourage and facilitate opportunities for information and resource sharing. In addition, implementers have shown creativity in finding new and effective ways to work together both with other partners in this initiative and various organizations involved in trafficking prevention efforts outside the scope of this program. The implementers adopted a variety of methods, including the training of journalists and other members of the media, to maximize exposure and raise awareness of the issues involved in human trafficking. Relationships Relationships built between implementers and local institutions enabled the development of better practices and improved intervention performance. A strong local network has been created and developed through this initiative, however a national network does not currently exist. While there are differences in conceptual and methodological focuses within the current network, the importance of increasing the scope and influence of this network is crucial for a wider impact. Establishing strategic alliances is a model that should be replicated, and pooling resources and sharing information are strategies needed to achieve success in areas where


163

projects have been started and to bring this model to more parts of Ecuador and other countries. Legislation and Legal Challenges The prosecution process has not been easy in spite of the approval of the new law. Cases of human trafficking are not isolated individual cases, but are strongly linked to criminal organizations, and victims, their families, and civil society organizations are fearful to face these organized crime rings in court. In addition, citizens and many law enforcement officials still do not have appropriate knowledge of the law and its application. The lack of clarification has left a number of loopholes that can be exploited and allows those convicted to be released without punishment. Other challenges within the judicial system remain and additional steps will need to be taken before those responsible for trafficking will experience the full authority of the law. Combating human trafficking is clearly linked to the political and operative levels of government. The participation of local governments, as supporters or leaders in the development of several initiatives, has and will continue to enable a focus on human trafficking. Although it is not yet possible to speak about a clear and defined political will, it is evident that certain municipal administrations have shown a basic knowledge of the issue when they have participated in projects to fight human trafficking. It is important that local governments develop public policies addressed to vulnerable groups, coordinate the work of local institutions and develop clearer definitions of specific laws and regulations. Changing Attitudes The issue of human trafficking has been considered a cultural norm not publicized in the media and largely ignored by Ecuadorian society. It is clear from the results of the projects in this initiative that the alliance of implementers has had an impact on the prevailing societal attitudes, causing many to re-evaluate what is permissible within the culture. However, the fear of reprisals has

Ecuador

164

also hindered the ability to talk openly and discuss trafficking issues. The strength and influence of the trafficking network is far-reaching, which has meant that overcoming these often unspoken cultural norms has proved to be a great challenge for the implementers. This is long-term work that needs continued investment to maintain the momentum that has been established. Implementation of new laws and discussions in the national media about trafficking issues are steps in the right direction. Considerable time will be needed before the full impact of changes in cultural perspectives can be felt. Best Practices There is a need for specialized centers to assist victims as well as a need for a common methodology for appropriate interventions. The gaps in cultural understanding of the true nature of trafficking, coupled with a lack of commonly accepted best practices, has meant that cases are often taken out of context and interventions are not as effective as they could be. However, different implementers have pioneered a number of resources that will be useful for future initiatives. Implementers also have learned to improve their program management and results reporting procedures. Planning for the Future Taking into account the current conditions in Ecuador and the knowledge gained from this first-time initiative, the time is ripe for a more strategic, nationally coordinated program. It is clear from the results of this project that a large number of these implementers have made huge steps with relatively small resources. Strategic, nationwide investment by the national government, as well as other funding organizations, is needed for implementers to continue to move forward with this work. Conclusion The actual number of lives impacted by this initiative greatly exceeds the expected number. The individual project reports show


165

the ability of the implementers to respond to the demand within the communities in which they work. Good project management has played a vital role in enabling implementers to achieve higher-than-expected results. The long-term impact of the projects is difficult to measure as it is hard to evaluate downstream impacts. We do know that many professionals, community leaders and families have come to a better understanding of human trafficking and its impact in their communities.

Ecuador

166


167

Tanzania and Zambia: In-Country Coordination to Improve Diflucan® Access

Daniel Aukerman, (V.P. International Program Development and Technical Support, I.M.A)., Mavere Tukai, (East Africa Regional Pharmaceutical Manager, Tanzania, I.M.A.), and Heather Lauver, (Assistant Director,

Pfizer International Philanthropy Program).

The Diflucan® Partnership Program and Interchurch Medical Assistance, Inc. (DPP/I.M.A) increased in-country capacity to distribute, deliver and strengthen the drug management systems in Tanzania and Zambia. This program expanded coverage of drug availability and advanced the quality of care in two countries and can be replicated elsewhere. By comparing cross-country experience, in this case in health, the MfDR process gets reinforced and can evolve into a stronger program over time. The role of the private sector (as donor) is especially valuable in delivering effective medicine to patients and providing M&E systems need to track delivery, usage and non-leakages.

Introduction:

The Diflucan® Partnership Program (DPP) is a Pfizer Inc. initiative to make its antifungal medicine, Diflucan® (fluconazole), available free of charge for the treatment of Cryptococcal Meningitis (CM) and Esophageal Candidiasis (EC), two opportunistic infections associated with HIV/AIDS. Worldwide, as of summer 2006, the DPP had treated over 200,000 persons, launched programming in forty-six countries, and delivered five million tablets to help HIV patients live more comfortably. Given the magnitude of the global project, DPP programming has primarily called on health ministries and in-country applicants to order, receive, distribute and monitor the Diflucan®. Drug distribution and monitoring once Diflucan® arrives at port depend upon the in-country systems and thus limits

Tanzania & Zambia

168

the potential reach of the DPP. Outcomes such as increased utilization reporting, improved country-wide product delivery, and expanded product access, which is systemic and extends geographically, have been a challenge. At the program design stage, the DPP planned implementation consistent with the first two Managing for Development Results (MfDR) principles: 1) focus the dialogue on results and 2) align programming, monitoring and evaluation with results. The DPP addresses the United Nations (UN) Millennium Development Goal Six to combat HIV/AIDS and Goal Eight to work through global partnerships for development. Focus the dialogue on results: The DPP set out both a quantitative objective (to expand coverage of drug availability to additional hospitals) as well as a qualitative goal (to advance the quality of care). CM is dire if left untreated, and as relapse rates are as high as 50-60 percent,; maintenance therapy, such as Diflucan®, is required to prevent recurrence. EC contributes to malnutrition by causing severe pain and difficulty swallowing when eating, thereby contributing to advancement of HIV/AIDS and early death. In Tanzania, the DPP placed targets on institution enrollment and enrolled 60 people over two years. Interchurch Medical Assistance, Inc. (I.M.A.) trained staff on drug inventory management and proper prescribing to improve the quality of care of the HIV/AIDS patients. The DPP seeks to assure that Diflucan® is used for serious HIV-related conditions. The second goal of the DPP is to strengthen drug management of national healthcare systems through trainings and supportive supervision. Specifically, the DPP endeavors to enhance the capacity of healthcare personnel to manage opportunistic infections (OIs) and to practice good program implementation (i.e., self-monitoring, reporting and rational drug management). The DPP also contributes directly to the following national objective: “strengthening the healthcare structure of Tanzania through expansion of healthcare personnel, facilities and equipment and


169

comprehensive training in the care and treatment of People Living with HIV/AIDS (PLWHA).”1 Align programming, monitoring and evaluation with results: Scale-up of the enrollment and preparation of new sites to receive products has been the first priority of the DPP/I.M.A.. Although the number of institutions is quantifiable, qualitatively, the metrics of proper prescribing and product management are more subjective. I.M.A. organized teams to go out in the field in Tanzania to do training. Open and frequent communication with the institutions has been necessary to track program progress. The country coordinators assumed the responsibility for monitoring the flow of the product within their catchment areas and reporting any diversions. Regarding the secondary goal of strengthening drug management, the DPP/I.M.A. supplies:

Capability of DPP partners and institutions to address supply chain system needs;

Capacity to monitor and make recommendations to improve pharmaceutical practices which, although in the context of a specific program, have broader applicability; and

Ability to deliver quality of care and treatment for PLWHA who suffer from OIs (the DPP/I.M.A. measures this target by tracking the percentage of recipient institutions reporting).

Application:

The I.M.A. Implementation Approach: The I.M.A approach to implementation focuses on partnership, in particular with Christian Health Association (CHA) missions abroad. The DPP/I.M.A. initiative has provided the technical and operational inputs for pharmaceutical support to ensure availability of Diflucan®. The 1 Tanzania National Guideline for HIV/AIDS Care and Treatment. Tanzanian Ministry of Health: The National HIV/AIDS Care and Treatment Plan 2003-2008.

Tanzania & Zambia

170

backstopping work by the Regional Pharmaceutical Manager has included information dissemination, preparation of applicants, and assistance in completion of the application forms. I.M.A. has also advised on logistical approaches such as whether hospitals should organize a pooled procurement system or a single uncoordinated procurement, or determining the requirements for instituting a pull system instead of a push system for product reordering. The DPP/I.M.A. needed to adjust the implementation efforts to accommodate the differences between the drug distribution systems in the two countries. In Zambia, the Christian Health Association Zambia (CHAZ) has procured and distributed Diflucan® using its own supply system, and all Diflucan® received is stored in CHAZ warehouses and distributed to hospitals. In Tanzania, Diflucan® is distributed by the Medical Stores Department (MSD), the national government-run supply system. The Tanzanian national procurement system lacks some of the flexibility present in Zambia. Monitoring and Reporting is Integral in the Systems of Both Countries: Tanzania: In Tanzania, the National AIDS Control Program (NACP) coordinates all HIV/AIDS activities in the country, including DPP implementation. In order for the DPP/I.M.A. to effectively promote a results framework, the work had to be done in conjunction with the NACP using the following mechanisms:

1. Treatment Facility Applications: Institutions submit evidence of need to the NACP, who approves Diflucan® recipients.

2. Treatment Facility Reports: Hospitals are required to submit reports to the NACP on a quarterly basis. Based on these reports, the NACP (working with MSD and I.M.A) decides the quantity of Diflucan® to supply regularly to the points of care from central stocks.


171

3. National DPP Program Progress Report: For better performance and more frequent progress monitoring, the DPP has requested that the NACP to compile reports and requests every six months. This report, together with a duly filled shipping advice, is sent to the DPP program manager at Axios for initial review. After initial review, Axios sends its recommendations to senior Pfizer and I.M.A. reviewers for refinement and approval.

4. Multi-stage Product Delivery Reports: a) The NACP affirms receipt of the Airway Bill and communicates to MSD; b) MSD notifies the NACP of product location after it clears the consignment with a tax exemption and places the product in its storage depots countrywide; and c) hospitals and local MSD depots notify the NACP when the institutions pick up their designated allotments from the depot.

The NACP reporting framework has some distinct advantages. The system is centralized and gives an overall country perspective if all inputs are timely. Moreover, the stakeholders, i.e. the institutions, DPP partners, NACP and MSD, all work together to achieve the same objective. The centralized nature of this program makes for easy linkages between the DPP and other programs such as antiretroviral therapy (ART) roll-out and treatment of opportunistic infections (OIs). DPP/I.M.A. activities in Tanzania have reinforced the four steps in the process described above:

1. On-site training of over five hundred hospital staff in DPP management has helped double the countrywide number of DPP trained personnel over an eighteen month period.

2. On-site mentoring and ongoing availability has improved the number and the quality of the treatment facility

Tanzania & Zambia

172

reports. On-site support has also allowed for the redistribution of Diflucan® from slow consuming facilities; for example, in one year, over 40,000 tablets in facilities and at risk for expiration were redistributed.

3. I.M.A. personnel have facilitated the preparation for the national DPP reports by consolidating reports from faith-based facilities, participating in the report writing and arranging for monitoring by the national OIs steering committee that oversees the project.

4. I.M.A. has organized meetings to bring together the various supply system stakeholders that form the national ART logistics system to discuss and propose ways to improve.

Zambia: In Zambia, CHAZ has established a supply system for essential commodities supported by various partners such as the Global Fund for AIDS, TB and Malaria (GFATM). As a means of bolstering this system and assuring Diflucan® access for mission hospitals disassociated from the existing public system, the DPP/I.M.A. worked with CHAZ to develop a simple, straight-forward supply mechanism, guided by the hospital consumption reports, to form a basis for acquisition of new supplies from CHAZ. Diflucan® has been delivered to a) public hospitals through the government’s own Central Medical Store of Zambia, and b) affiliates of CHAZ for all hospitals that are under the CHAZ umbrella. However, the two DPP/I.M.A. programs are not linked. Given this division of responsibility and the Ministry of Health’s (MOH’s) desire to administer its own system, the coordination efforts of the DPP/I.M.A. have focused exclusively on CHAZ institutions. The four basic monitoring stages in the results framework are similar to Tanzania’s approach; however, a few differences in the Zambian program are noticeable:


173

1. Treatment Facility Applications: New applications go directly to CHAZ for approval.

2. Treatment Facility Reports: Monthly progress reports are submitted to CHAZ for review.

3. CHAZ DPP/I.M.A. Program Progress Report: Instead of integrating its information into an overarching national report, updates and restocking requests are sent directly to Axios.

4. Multi-stage Product Delivery Reports: Communication regarding product location in-country occurs strictly between CHAZ and its member hospitals.

CHAZ, having independent decision-making authority, can move forward with implementation at its own pace without waiting for the government’s report to be published. CHAZ also has flexibility and a quick response time. The ability to determine program policies, such as a “no report no product” rule for facilities, establishes accountability. To ensure the product delivery is smooth, timely and efficient, the DPP/I.M.A. and CHAZ have devised a centralized procurement model through which hospitals receive training, technical support and access to Diflucan®.

Problem Solving:

Tanzania: The multiple players within the Tanzanian system are presented with ongoing challenges. Delays have resulted because facilities cannot go directly to the Diflucan® distributors for resupply. Instead, the NACP must establish the quantities to be dispersed to the facilities. To mitigate this logistical challenge, the DPP/I.M.A. has proposed a new plan whereby hospitals submit reports to MSD instead of the NACP or I.M.A. Given that the NACP is overburdened by national HIV/AIDS demands, it lacks the staff to process Diflucan® reports originating

Tanzania & Zambia

174

from the non-MOH institutions. Recognizing these challenges, the DPP/I.M.A approached the NACP to offer assistance in the implementation of the national program. As a result, the DPP/I.M.A. has been tasked with coordinating all of the DPP implementation activities for mission hospitals. The activity of the I.M.A. country coordinator is officially supervised by the NACP, which also receives the coordinator’s quarterly reports. The DPP/I.M.A. has instituted monthly reports from the mission hospitals, an approach that is more effective because even hospitals that fail to report for one month have the ability to submit a report the next month, which ultimately will be consolidated in the quarterly report. I.M.A.’s role is to provide support in advising hospitals on ordering procedures since the reorder process is coupled with previous consumption reporting. Currently, hospitals simply submit their requirements to MSD. This process is perceived as having a negative impact on the project because hospitals receive Diflucan® whether or not they provide reports. I.M.A has advised hospitals and MSD on the reordering process. Progress was evident when the NACP, I.M.A and MSD met to lay down the “standard of practice” for Diflucan® supply management in July 2006. Zambia: Unlike Tanzania, DPP expansion in Zambia has succeeded more readily because CHAZ has program ownership and serves as the sole coordinator of the DPP for the facilities under its management. Pharmacy management at the facility level is often carried out by staff without formal training and is an obstacle faced by the Zambian program. Thus the DPP/I.M.A. has focused on improving standard good pharmacy practice. Detailed and vigilant monitoring of product arrival and utilization has been stressed. In addition, inventory management training has been integrated into ART training given at CHAZ facilities.


175

Results: Table 1: Summary of ten month achievements under I.M.A

supported the DPP in Tanzania and Zambia

In both programs, the principle target was to increase the number of facilities receiving Diflucan®. Fifty-four institutions currently are receiving medicine due to the program’s efforts. The secondary goal was to strengthen the drug management system. In Zambia, the DPP/I.M.A successfully reached 100 percent reporting for CHAZ facilities as a result of the clear “no report no Diflucan®” policy. CHAZ is also responsible for warehouse and distribution inventories, giving it control over the supply for non-compliant facilities. In Tanzania, inadequately trained staff, poor infrastructure and negligence have contributed to poor reporting. Even with these challenges, I.M.A. has successfully managed to keep updated records (updated reports from 28 out of 32 hospitals (87.5 percent) have been obtained). Contributing to the difficulties in reaching 100 percent reporting is the fact that the faith-based network does not stock or distribute independently. Diflucan® supply management in Tanzania needs a thorough review to establish a user-friendly supply and reporting system to encourage collective participation in improving Diflucan® access. To address this issue, I.M.A began writing standard operating procedures for hospitals and MSD product distribution in Tanzania.

Tanzania Zambia Total

Number of hospitals enrolled 33 28 61

Number of hospitals receiving Diflucan® 30 24 54

Number of hospital staffs trained 558 28 586

Number of OC treated 1,602 1534 3,136

Number of CM cases treated 248 454 702

Tanzania & Zambia

176

Conclusion:

Over three thousand patients have benefited from the DPP in Zambia and Tanzania alone. Working with different country dynamics and different approaches resulted in several lessons: 1. At all program phases, focus the results dialogue on integration with national systems, development agencies and other stakeholders. Partnership and dialogues are key factors in improving development projects. Working with the national system may be difficult at the start, but it has more impact and longer-term effects. This approach has shown promise as the DPP reporting and request process is more efficient and training is part of the national curriculum. 2. Align actual programming, monitoring and evaluation activities with expected results. Both in Zambia and Tanzania, the DPP/I.M.A. quickly learned that human capacity was the biggest challenge from the national coordinating level down to the facility level. Each step of DPP implementation has faced shortages or a lack of qualified staff for good supply management and clinical care for PLWHA. Given the critical connection between human resources and achieving results, the DPP’s programming targeted human capacity development. The intensity and frequency of supportive supervision has been effective in bolstering the skills of staff working with mission hospitals, often in remote and underprivileged areas. 3. Keep the results reporting system simple and user-friendly. In both countries, management through an ongoing assistance model achieved positive outputs beyond those expected. Although traveling from site to site involved increased effort from DPP staff, these visits permitted the training of a larger number of hospital staff and witnessing of hands-on job demonstrations. Working with the pharmacy technicians at the institutions


177

permitted direct feedback on how to keep the reporting system user-friendly. 4. Manage for, not by, results through arranging resources to achieve outcomes. In-country dynamics (leadership, management, set-up, teams and partners) play a key role in development and program implementation. A persistent focus on results, expanded coverage and improved quality allowed for work in Tanzania to maintain a long-term vision even though the short-term results were slow to appear. The DPP/I.M.A. recognized the need to shift efforts toward greater involvement with the MOH. However, efficiency in Zambia was increased by having the operational monitoring activities under one unit of the partnership. 5. Use results information for management learning, decision-making, reporting and accountability. Examination of the achievement of suboptimal results will give management the opportunity to improve the overall system. Key factors such as workforce shortages, insufficient capacity and inefficient processes are correctable. When program management demonstrates the ability to make work easier for institutional and national partners, the program gains credibility and partners sense that all are working together for a common goal.

Tanzania & Zambia

178


179

Aceh and Nias, Indonesia: Results Achieved using Community Reconstruction following the

Tsunami-Disaster in December 2004 Lisa Campeau, (Licus/Fragile States Unit, World Bank)

This case analyzes the reconstruction of the Indonesian provinces of Aceh and Nias following the devastating tsunami of December 2004 and subsequent earthquake in Nias in March 2005. The case focuses on community involvement in prioritization and sequencing of needs, planning and project implementation, and monitoring and evaluation of progress. Lessons can be drawn from the challenges of implementing Community Driven Development (CDD) programs in Aceh and Nias including the need to i) integrate priority-setting and monitoring into broader government programs and ii) improve reporting and tracking of outcomes.

Introduction:

On December 26, 2004, a Sumatran-Andaman earthquake triggered several massive tsunamis, devastating coastal communities in Indonesia, Sri Lanka, Thailand, India and the Maldives. One of the worst natural disasters in 100 years, the tsunami destroyed most of the existing infrastructure in Aceh and Nias, including houses, boats, schools, health clinics, power lines, roads and ports.

The disaster required large-scale community reconstruction. Damage assessments of Aceh and Nias show that destruction amounted to approximately 100 percent of GDP, with 167,000 dead or missing and an estimated 500,000 displaced from their homes. Three months later, the Nias province was hit by a major earthquake, killing 900 people, displacing 13,500 families and impacting approximately ten percent of the North Sumatran province’s economy. Aceh and Nias were poor prior to the disasters, and recovery was further complicated in Aceh by conflict. With such destruction

Indonesia

180

(approximately 6,400 villages in Aceh and Nias were affected), community prioritization of needs and self-sufficiency became critical.

Community institutional infrastructure was already in place with broad coverage. Prior to the tsunami, donors and non-governmental organizations (NGOs) were contributing to several Community Driven Development1 (CDD) programs. Therefore, an institutional infrastructure was already functioning, allowing for immediate mobilization for disaster relief and continued involvement in further reconstruction and rehabilitation activities thereafter. Two large-scale community projects had a network of well-trained community facilitators and professionals to help tsunami-affected communities plan their recovery responses: the Kecamatan Development Project (KDP) and the Urban Poverty Project (UPP). Rather than create new institutions and processes, these projects were scaled-up and expanded to respond to urgent recovery needs.

Reviews demonstrate that CDD is less costly and more efficient. Service delivery through community development costs on average 56 percent less than the same services provided by ministry and government contracting. CDD is geared to small-scale, local infrastructure and household needs. It is also well suited in conflict areas (e.g., Aceh) in dispute resolution.

Application:

Communities assisted in preliminary damage assessment and in planning priority reconstruction programs. Whereas already weak local government was further overwhelmed by

1 The following definition of Community-Driven Development is taken from the World Bank’s Africa Toolkit and the World Bank’s ESSD website. CDD “is the process by which community groups assume control and authority over decisions and resources in development projects which affect their lives. This means reversing control and accountability from central authorities to community organizations in the initiation, planning, implementation, operation, maintenance and evaluation of development projects with agencies playing a supportive role. Community-driven development is most appropriately used for the provision of local goods, basic services and the management of natural resources.”


181

the disaster’s size, community leaders had the knowledge and capacity to immediately assist in information gathering and dissemination. The KDP and UPP facilitators and local villagers helped communities prepare maps showing the extent of damage in each village and plan priority rehabilitation programs.

Two years after the tsunami, CDD programs were also used for stock-taking. CDD has helped to re-align overall reconstruction programming to focus on actual outcomes or results achieved and apply results information to learning and decision-making. To provide an accurate overview of the current status of village reconstruction throughout Aceh and Nias, KDP is currently undertaking both a social and an infrastructure assessment:

1) Local facilitators gather data. Although limited in skills and training, local facilitators from KDP are used to gather data. They are ultimately more effective than external assessment enumerators because of their ability to involve relevant village resource persons, their sensitivity to local circumstances, and their knowledge of potential conflict triggers.

2) Results assessments impact future government allocations. Information gathered by communities on actual outcomes will be used to influence overall national and provincial government reconstruction spending and allocations in various sectors from FY07 onwards.

3) Data made available for broad usage. Assessment data will be made available to government agencies, donors, NGOs, researchers and others and will serve as a resource for development practitioners, academics and government officials, complementing existing databases at the village level.

Indonesia

182

4) Two types of assessments planned. i) a social assessment to provide a “snap shot” of each village; and ii) an infrastructure assessment.

Problem Solving:

Lessons on Implementation Delivery Speed of Results and Quality of Results

1) Focusing on long-term capacity-building versus ensuring delivery speed. Studies show that bringing funds “on-budget”2 and supporting their allocation through regular Ministry of Finance or Planning structures significantly strengthens capacity and sustainability of programs by implementing them through the budget process. Unfortunately, in this case, using “on budget” systems significantly delayed project disbursements - the government budget for reconstruction projects was only approved at the end of June 2005. Although usually considered “speedier,” however, some “off-budget” bilateral donors and NGOs experienced delays due to their own time-consuming procedures.

2) Using existing projects versus implementing new projects. Although use of “on-budget” processes may have resulted in a lengthier disbursement process for planned CDD projects, objectives were still achieved in a timely manner by building on existing projects rather than preparing and implementing new projects.

3) Ensuring rapid reconstruction versus using lengthier community-driven priority-setting. CDD projects are not a “quick fix,” but rather a long-term investment in an implementing agency’s time and human resources. In circumstances of great stress, when basic needs are the

2 Donor funds channeled through the government budget are defined as “on-budget.” Funds channeled directly from donors to programs or projects are defined as “off-budget.”


183

priority, communities and vulnerable individuals lose enthusiasm for long-term participatory projects in favor of quick action or hand-outs. Some basic needs were met temporarily by the relief effort; however, use of CDD for other community requirements took more time.

4) Using CDD procurement methods versus government procurement and contracting. Although using a CDD participatory process may seem lengthy, evaluating reconstruction projects over their lifespan, from start to finish, has shown that CDD is arguably faster than other methods. Community-implemented projects can skip complicated procurement processes by using local or village-level labor and by applying “shopping” procedures over employing international procurement or national “bidding” processes. The use of simple procurement procedures has been crucial to ensuring speedy project delivery, and the CDD approach has proven to take less time than government contracts.

5) Ensuring speedy reconstruction versus gradual recovery using capacity building and employment generation. In Aceh and Nias, many villages emphasized capacity building and employment generation for local people and institutions rather than using outside labor to achieve infrastructure development. This required training local people in carpentry and other skills. Although slowing the pace of physical community reconstruction results, such a process not only benefited villagers through employment generation, but also ensured ownership of infrastructure, providing greater incentive among villagers to maintain those facilities.

6) Meeting community priorities versus selecting projects based on capacity to deliver quality results. Although successful in quickly mobilizing staff, many agencies in Aceh could have achieved quality results more

Indonesia

184

rapidly had they prioritized community requests according to their core strengths. The KDP supports an open menu, listing projects covering various sectors, and an NGO has the opportunity to select their community project based on the organization’s strengths as opposed to choosing projects based on the preferences of the community.

Lessons on Using CDD for Disaster Reconstruction

1) Employing community participation versus implementing community requests and/or managing their expectations. While the CDD program in Aceh and Nias has arguably the broadest participation of communities in any recovery effort following either a disaster or conflict, use of this method for broad small-scale reconstruction has its drawbacks. Consultation, although beneficial for “ownership” purposes, can also create expectations for receiving assistance in the future. After involving communities early in the consultation and priority-setting processes, further information dissemination on actual reconstruction plans is required, not only to manage expectations, but also for transparency purposes

2) Implementing community priority-setting versus applying centrally developed planning processes. Too often coordination exercises have been concentrated in Banda Aceh rather than in the communities. Although community-level data collection, analysis and dissemination for policy and programming often can be less than perfect, some data collection may have been more accurate had it been based on the community or sub-district. Overall data collection could be improved through continual feedback that involves consolidating community and central data and later disseminating it into the community.


185

3) Using CDD community prioritization and implementation versus employing broader government planning and implementation. There is insufficient coordination between CDD planning and priority-setting, known as “village visioning,” and the establishment of broader government priorities for reconstruction and rehabilitation. Discussions are underway to use KDP facilitators and CDD community prioritization methods for broader government planning. However, the transition from centrally planned, designed and managed projects to CDD and local government plans and management platforms will take time.

4) Accepting village-level monitoring and evaluation versus tracking and reporting on overall CDD investments across all communities. The KDP uses village-level monitoring by posting pertinent project implementation information (funds received, project costs, village level contributions, etc.). However, developing monitoring and evaluation systems to track all investments across such a large number of communities has proven difficult. Monitoring and evaluation is critical to improving results, and ensures on-going corrections, design simplifications and improvements.

5) Accepting private investments versus only allowing reconstruction of public assets. CDD Programs are intended to assist in the reconstruction and rehabilitation of public assets. Considering the magnitude of personal losses in the case of the tsunami, the use of CDD grants for the recapitalization of lost private and personal assets was accepted (in place of traditional insurance). CDD and community grants have been proven to be effective for recapitalization by targeting vulnerable groups; however, they have not been as well suited for micro-finance projects.

Indonesia

186

Lessons on Coordination, Monitoring and Evaluation and Reporting Processes

1) Focusing on strategic planning for targeted results

versus solely employing simple information-sharing processes. While coordination among the various agencies was adequate in the short-term, existing structures were not suitable for long-term reconstruction. Recovery coordination was little more than an information-sharing exercise, as opposed to a process involving overall strategic planning, apportionment of responsibility and sequencing of priority actions. No mechanisms or criteria were used to determine agency roles on specific issues and policy consultations. While avoiding duplication, such a process left critical problems and gaps unaddressed.

2) Using labor-intensive approaches versus applying complicated outcome reporting systems. With over 300 players and 1,500 projects throughout Aceh and Nias, reporting on outcomes became particularly challenging. To date, there is no unified comprehensive system for collecting, monitoring and reporting on outcomes. A complex internet-based reporting system was created requiring data inputs on outputs or outcomes, however many found the system difficult to understand and ceased to update their information. Stakeholders learned the importance of monitoring outcomes and engaging development partners, but also recognized that a simple reporting system is imperative.

Results:

Short-term: CDD programs represent 20 percent of the overall donor community reconstruction portfolio (US$5 billion). CDD programs are the third largest in funding receipts after roads and bridges and governance and


187

administration projects (KDP is primarily a governance project that builds village infrastructure). Successful CDD reconstruction and rehabilitation results have led to additional increased funding for FY07.

Short-term: CDD projects disbursed funding ahead of schedule. By December 2006, 18 months ahead of schedule, KDP had disbursed all project funds totaling US$65 million. A new financing window is being developed for additional community grants. Disbursements ($1.5 billion) under the community development program are impressive, considering that only 30 percent of funds have been disbursed from global reconstruction funds.

Long-term: CDD programs yielded more outputs than any other program. In August 2006, reconstruction outputs (funded by the Multi-Donor Trust Fund) included roads, bridges, schools, houses, water/sanitation, street lights, health clinics and waste disposal systems.

Long-term: CDD programs also contributed to longer-term outcomes. International auditors found that corruption rates were less than three percent among CDD programs in villages and urban areas - much lower than most other government projects that have been audited.

Replication: CDD may be replicable when there is long-term commitment, as opposed to a reliance on quick fixes to provide instant accountability. CDD programs may be less effective if not installed prior to a disaster, as they are human resource-intensive and require tremendous technical assistance. CDD requires a more active donor presence than standard projects to establish anti-corruption and free-choice precedents in the field, which government-hired consultants rarely have the autonomy to do, especially in transition, crisis and post-conflict countries.

Indonesia

188

Conclusion:

Conduct a community-based assessment and feed data into a broader outcome reporting process. Good data, information and communication are critical to reconstruction planning, policy-making and successful implementation outcomes. Despite the global attention to Aceh’s reconstruction, there is no unified comprehensive system for collecting and disseminating key data. However, the CDD stock-taking exercise should yield valuable data on outcomes achieved in Aceh and Nias.

Apply CDD lessons and use participatory processes for broader government results in planning, implementation, monitoring and reporting on outcomes. Clear processes, responsibilities and structures need to be established for effective overall results coordination and implementation among various stakeholder groups. This requires cooperation between provincial and district governments, community leaders, implementing agencies and donors.

Expand CDD programs to include conflict-affected persons and returnees. Aceh’s history of conflict has led to high levels of mistrust among communities, government, and the Free Aceh Movement. CDD programs can play an important neutral role in contributing to stability and peace by supporting the reintegration of former combatants into local structures and activities. Involving broader conflict-affected areas will require training of CDD facilitators and village heads in conflict resolution techniques.

Move from areas only affected by the tsunami to all sub-districts in Aceh and Nias. Rather than focusing solely on tsunami-affected areas, the next step will be to ensure broader coverage of social services across Aceh and Nias and transition from rehabilitation and reconstruction to longer-term sustainable development. Now that the initial emergency has passed, greater coordination is needed in formulating and


189

adopting a participatory, results-focused, medium-term development plan. This plan should focus on lessons gained from community participatory planning and monitoring and provide for cooperation between the community and its stakeholders. CDD programs continue to play a role, not only to promote community-driven investment choices, but also to ensure governance and accountability to citizens.

Indonesia

190


191

Conclusions

Building on the First Edition, the Second Edition Sourcebook focuses on experience with MfDR in action. This edition of the Sourcebook seeks to help readers identify cases geared towards these three target audiences. Readers are able to identify themselves with one (or more) of the audiences and to select cases that are most relevant to their interests and expertise. Furthermore, cases followed the same five part format: Background, Application, Problem –Solving, Results and Outcomes, and Conclusions, which allowed readers to cross-reference the various approaches throughout the book. Another unique component of the Sourcebook was that countries self-selected their own examples of best practices. The cases were presented in four regional Mutual Learning Initiatives and then evaluated by an independent Review Panel comprised of donors, country partners, civil society, and the private sector. Many of the cases in the Second Edition were also presented by countries, both in the Results Marketplace of the Third International Roundtable on Managing for Development Results in Hanoi, Vietnam, (www.mfdr.org/RT3) and as examples of cutting edge MfDR approaches in individual sessions. What are the key lessons from the Sourcebook? How replicable are the cases among countries of different income levels, and different institutional capacities, and with differing historical precedent in engaging in the rapidly evolving field of MfDR? It is important to note, that there are many countries that can show progress in MfDR that are not mentioned here – some are discussed in the First Sourcebook, such as Brazil, Uganda, Tanzania, and Nepal,

V

Conclusions

192

where these countries and their line agencies are improving their macro planning and results measurement systems. Others have interesting new points to demonstrate, such as work in the Rwanda health sector in to improve the incentives for performance-based health insurance and individual contracting. Progress is also evident in Benin, where results management techniques are improving their urban land management capacity, or in Ecuador, where they are reducing the propensity of human trafficking cases. Others have had to grapple with emergency recovery programs, such as Haiti Stabilization or Indonesia/Tsunami Reconstruction and they are understandably focused on the short-term, but they also know that in the long term their basic ability to deliver results is at stake. Review of the cases presented here suggests the following lessons:

Size does not matter. There are successful experiences with adopting the principles in both small and large countries. Very large countries like China and India are building effective results management programs by building up local government and pursuing decentralization systems.

Income level can affect results. Countries with more resources are more able to mount surveys and develop institutions that can design, implement, and monitor program effectiveness. Also, they tend to have a sufficient number of agencies, both in government and in the NGO sector which can monitor progress and call for reform. Conversely countries that are strapped for resources are constantly fire-fighting, and they find it difficult to devote the time that effective MfDR needs. Results management can be an important element of the efforts fragile states make to build effective systems of governance. Yet the attempt at time saving can prove very costly in the end. Hastily thought-out results frameworks can end up trapping officials in a futile exercise trying to implement something that in the end gets thrown by the wayside.


193

Good data are needed in all countries. While gathering data may be more challenging in geographically large countries where census data and surveys are costly to organize and is particularly challenging in low income countries, this task needs to be done. Support from partners for development of systems and data is crucial. Vietnam - localizing the MDGs, the Haiti Recovery case, and Peru - dealing with local crime, are all examples of how potent improvements in information systems can be. Without statistical capacity, including developing the technological systems, it is impossible to effectively manage for results.1

Participation by and regular interaction with the public is key for ensuring accountability for results. The Mozambique Poverty Observatory and Mongolia Open Society Forum are good examples. If government leaves out civil society, the effectiveness of MfDR will be undermined. Strength in MfDR comes from joint action, not separate steps taken in isolation.

Capacity to manage for results exists in all countries, but it needs to be unleashed. Capacity development for MfDR (as well as general substantive national capacity development) can unleash and help motivate existing human resources and improve accountability.

Change needs incentives. In general, both individuals and organizations are motivated when they see evidence of results. MfDR requires a cultural shift in behavior. Formal and informal signals are essential for making organizational change a reality. Igniting the desire for change takes strong and committed leadership.

1 At the meetings of Governors to the World Bank and International Monetary Fund held in Washington on April 14-15, 2007, countries endorsed a plan for accelerating the strengthening of data collection capacities in partner countries. This was one of the actions suggested by the Marrakech Roundtable (2004) and reaffirmed in Hanoi.

Conclusions

194

Income distribution and inequalities in access to infrastructure and social services will undermine the delivery of effective results management in all countries.

Discussion of these issues will continue to shape the work among country partners and the donor community as they prepare for the 2008 Ghana High Level Forum on Aid Effectiveness. Participants in the Hanoi Roundtable expressed their strong desire for increased country-to-country learning in tackling these issues and others that may arise in the coming year. As one forum for such learning, preparation of an annual MfDR Sourcebook is being considered, as users indicate that the Sourcebook is a valuable tool that provides an opportunity to showcase examples of best practice on MfDR in a variety of settings, and to table pending issues. The Roundtable and the OECD/DAC Joint Venture on MfDR proposed several possibilities for the next edition of the Sourcebook that will build on this demand for increasingly specialized cross-country and cross-partner learning exercises and events.

Documents

48391 Public Disclosure Authorized - World Bank · The five sections are: i) Introduction - establishing the context, i.e., why was it decided to use an MfDR approach? Where did the