Assessment of Local Governance and Development Performance in Indonesia:

Current Models, Challenges and Future Perspectives1

Dr. Astia Dendi2

Abstract

This paper explores the current approach and tools of performance measurement implemented by the government of Indonesia in the decentralized governance context. Following the 1997’s financial crisis, Indonesia embarked on implementing the so-called “Big Bang” decentralisation policy which actually began in 2001. It has led to the devolution of authorities to provincial and local governments. Thus, these sub-national governments have more opportunities and responsibilities to develop their regions according to local needs and people’s aspiration involving the private sector and civil society organizations. Within the framework of national decentralization and an increasingly competitive global context, sub-national governments have made tremendous efforts toward the provision of public services, to achieve sustainable economic growth and reduce regional disparities and poverty. However, the implementation of decentralisation policy has not always been a smooth process nor has it effectively generated all desirable changes and impacts. The current shortcomings call for a better policy, practical tools and systemic capacity building for performance management. A long lasting debate at policy and implementation levels has been on efficiency, applicability and comparability of the different approaches and tools for local governments’ performance assessment introduced by the central government agencies including by the Ministry of Finance, the Ministry of Home Affairs and the Ministry of Administrative Reform as well as the Ministry of National Development Planning (BAPPENAS). It can be concluded that the current models of performance assessment in Indonesia overwhelm the local governments rather than stimulate and capacitate them to enhance their learning process, strengthen their capacity for policy formulation and strategy alignments in addressing local/ regional issues and emerging global challenges. Moreover, it turned out that the central government-driven models of performance assessment were notably rich in their number of indicators but rather poor in term of attainability or accessibility of verifiable data in cost-effective ways. We suggest a nested model of performance assessment that

1 A paper submitted to International Association of Schools and Institutes of Administrations (IASIA)

for a presentation in the coming congress entitled “Public Sector Strategy for Overcoming Growing Global Inequality, to be held in Bali, Indonesia from 12-17 July 2010.

2 Dr. Astia Dendi, Senior Advisor of Decentralisation as Contribution to Good Governance (GTZ DeCGG). Kementerian Dalam Negeri Republik Indonesia, Jl. Meredeka Utara No.7 Jakarta. Telephone +62 21 351 1584; Fax +62 21 386 8167; Email: Astia.Dendi@gtz.de. The perspectives presented in this paper do not represent official views of Deutsche Gesellschaft fuer Technische Zusammenarbeit (GTZ) GmbH.

2

integrates three spheres of performance evaluation namely financial/budget performance, governance performance and development performance. In addition, the paper discusses lessons learned and suggests process and steering structures toward effective implementation of the nested model of local governance/ development performance evaluation. Keywords: Good governance, Indonesia, local governance, performance

assessment, financial performance, governance performance, development performance.

1. Introduction 1.1 Background

Following the financial crisis, Indonesia since 1999 embarked on implementing a decentralisation policy that was called “Big Bang” by the World Bank due to its radical shift in authority from national to subnational government. Implementation actually began in 2001 and has led to the devolution of authorities to provincial and local governments. Indonesia is a unitary state that presently consists of 524 autonomous regions (33 provinces, 398 districts, and 93 municipalities (MoHA, 2009). The sub-national governments have more opportunities and responsibilities to develop their regions according to local needs and people’s aspirations, involving the private sector and civil society organizations. Within the framework of decentralization and an increasingly competitive global context, central and sub-national governments have made tremendous efforts towards improved provision of public services, achievement of sustainable economic growth and reduction of regional disparities and poverty. However, the implementation of the decentralisation policy has not always been a smooth process nor did it effectively generate all desirable changes and impacts. Some challenges remain whichamong others include the continuation of bureaucratic and governance reform along with fiscal decentralization and enhancement of cross-sectoral and vertical policy coherence. Meanwhile, the National Mid-Term Development Plan (RPJMN) of 2010-2014 has put strong focus on the need to establish better inter-sector coordination and national-regional/local interaction to pursue economic development and people’s welfare improvement (Agenda I). Along with that, the RPJMN 2010-2014 also emphasizes the improvement of governance (bureaucratic and governance reform) which also calls for a more sytematic and holistic implementation of performance management including planning, performance-based budgeting and performance measurement. Policy makers and some prominent civil society organizations in Indonesia have increasingly considered performance measurement as a management instrument to enhance the quality of services produced by public sector agencies. Furthermore, it is believed that an appropriate system of performance management would promote organizational learning and strengthen customer-orientation among public agencies. One of the

3

fundamental challenges, however, has been the establishment of a coherent regulatory framework, followed by effective collaborative efforts to improve the performance of public agencies at all levels.

1.2 Scope and Structure of the Paper

This paper explores the current approach and tools of performance measurement implemented by the government of Indonesia in the decentralized governance context. We analyze and discuss the underlying concepts and legal frameworks of performance measurement applied in Indonesia. Prospects and challenges facing the implementation of outcome-focused performance measurement based on day-to-day observations during five recent years as well as local stakeholders’ perspectives are discussed. The paper concludes by addressing some outstanding issues and action-oriented recommendations to make the outcome-focused performance measurement work better. 2. Performance Measurement System in the Public Sector: A

Theoretical Perspective 2.1 Means-Ends Structure and Performance Measurement Governments in developing countries, including the Indonesian government, have increasingly drawn attention to the application of performance measurement along with the notion of bureaucratic reform and good governance practices. In the public sector context, performance measurement is considered as a connector between information and management decision-making for the benefits of the public. By using valid and reliable instruments, performance measurement will provide elected officials with sound information and profound understanding on policy outcomes and constraints and, thus, will drive adjustment or alignment of policy or programme to create greater vertical and horizontal coherence through evidence-based policy making and programme planning. A reliable performance measurement will, therefore, enhance the efficiency of resource use. Furthermore, it will enhance effectiveness (achievement of the planned policy or programme outcomes), transparancy as well as vertical and horizontal accountability. However, stakeholders lack a common understanding or even a “persistent” debate on the meaning and design of reliable and cost effective instruments to measure public sector performance. Indeed, the framework of performance measurement can be traced back to the Logical Framework concept (means-ends structure) introduced by USAID in the late 1960s (Dale, 2003). The logical framework helps development planners to structure project strategy along logical effect-chains (linear means-ends relationship) from transforming inputs into outputs toward specified goals. In addition, the logical framework addresses another dimension of development strategy namely indicators and achievement

4

targets (performance measures) which generally consist of output, purpose and goal to be monitored and evaluated. In the context of regional development, for example, Dale (2003) introduces an extended means-ends structure in a logical framework for planning and monitoring [or performance measurement]. The model follows the logical cause-effect chains in line with the “convention” of development science. Dale’s notion of logical framework has the following means-ends structure: “Activities (with input indicators)–Outputs–Immediate Objectives–Effect Objectives–

Development Objectives” Dale’s means-ends structure is clearly an objective-centered model. He maintains the argument that any development effort should ultimately bring about improvement of the quality of people’s life. Therefore, development programmes or services should focus on creating benefits for specified beneficiaries (target groups) as well as impacts for other people (broader target population). In this perspective, Dale (2003) furthermore argues that the output, namely the products or services generated through utilization of inputs in implementation tasks (activities), tell nothing about benefits for the people; therefore development should go beyond outputs and the means-ends structure should show specific and unambigious immediate changes toward desirable benefits in terms of improvement in quality of life of people. Furthermore, it is noticable that Dale (2003) differentiates objectives into three hierarchies (means-ends structure) namely the immediate objective, effect objective, and development objective. The notion of immediate objective refers to immediately intended results of the outputs for specified beneficaries. The notion of effect objective refers to direct and unambigious intended improvements for intended beneficaries as the direct effect of immediate objective achievement. By contrast to these objectives, the term development objective refers to the overall intended impacts of any planned development intervention or service; this objective should clearly express benefits in terms of some improvement in the quality of life of people. In contrast to Dale’s result chains (means-ends structure), GTZ introduced a revised result chains model recently (Sckeyde and Wagner, Eds., 2008: page 8) as follows:

Inputs – Activities – Outputs – Objectives –Indirect Results

GTZ defines outputs as short term results of the activities in terms of equipment, materials or services and these are available for use by other actors (Sckeyde and Wagner, Eds., 2008: page 7). The notion of ‘use of outputs’ expresses the change process that intermediaries and target groups undergo in order to achieve specified objectives. In this perspective, the objective is defined as the direct results among intermediaries and target groups with clear causal link to activities and thus can be quantitatively attributed to an individual measure (Sckeyde and Wagner, Eds., 2008: page 7). The notion of ‘indirect results’, however, can no longer be causally and quantitatively attributed to an individual measure; these results depend on

5

inputs from many other actors whose share in the overall change may be plausibly demonstrated but can not necessarily be isolated or quantified (Sckeyde and Wagner, Eds., 2008: page 7). Based on these two models, Dale (2003) and GTZ (Sckeyde and Wagner, Eds., 2008) and the given examples, some common perspectives can be identified. The first is, that the ‘output’ is produced within the service provider’s system while the further change process and results/ objectives occur in the ‘client system’ (intermediary and target population). Secondly, Dale’s notion of ‘immediate objective’ appears closely compatible with GTZ’s notion of ‘use of output’. Thirdly, both models share the principle that the means-ends structure should clearly and unambiguously specify intermediate changes/ improvements that clarify how outputs will make significant effects toward realization of intended benefits for the specified target population. The term ‘specific and unambiguous’ changes in both models call for appropriate selection and formulation of clear and measurable indicators of the intended changes/benefits. The notion of ‘measurable indicators’, however, should not be interpreted as merely quantitative indicators but also include qualitative indicators or a combination of both. In contrast to these two models, some practitioners develop other models although theyshare the convention of “if-then” logic in the academic literature. Schmidt (2009:32), for example, introduces the following means-ends structure: “Input – Outcome – Purpose – Goal”. Schmidt’s notion of outcome refers to specific results that a project team must deliver by managing inputs, while the purpose refers to the anticipated impact of doing the project (change expected from producing outcomes). Furthermore, Schmidt (2009:32) defines Goal as the high level, big-picture strategic or program objective to which the project contributes. Furthermore we explore how scholars and practitioners define performance measurement. We recognized that there exist different perspectives, meanings, and scope of what the performance measurement is about (Mardiasmo, 2002; Hatry, 2006; Jantz, 2008). Hatry (2006: page 3) defines performance measurement as regular measurement of the results (outcomes) and effeciency of services or programs. Furthermore, the Hatry’s notion of service effeciency differ from the widely adopted concept of output-based effeciency (input-to-output ratio). In contrast, Hatry suggests outcome-based effeciency indicator concept. For years, scholars share and maintain arguments that the input-to-output ratio as indicator of effeciency is risky for a simple reason that, the effeciency can be increased by reducing the quality of output. Instead of using “cost per client served”, for example, it would be more accurate to use “cost per client whose condition improved after services” as efficiency indicator (Hatry, 2006). The first ratio is an output-based while the second ratio is an outcome-based efficiency indicator. Furthermore, the emphasis on regular measurement of progress toward specified outcomes, Hatry argues, is a vital elements of customer-oriented process at managing-for-results to ensure maximum benefits and minimum negative consequences for customers of services or programs. Furthermore,

6

Hatry (2006) argues that beside regular tracking for budget purposes, managers in public sector need more frequent outcome information to assess the success of their programm activities, identify significant problems to achieve the specified outcomes as well as to motivate personnel to strive for continues service improvement. However, performance measurement has another vital goal, namely to assess and ensure equity. Hatry (2006) maintains argument that a well-designed measurement system will enable agency managers to assess the fairness of services or programs and adjust them appropriately. More recently, GTZ Sector Network “Governance Asia” (2010: 7-8) defines performance measurement

“... as measurement on recurring basis of the outcomes or results and effectiveness of services or programs. The operative term in this instance is recurring. Consistent measurement or advancement toward specific outcomes is a critical part of all management efforts to impove results. An important aspect of performance measurement in local government is its customer orientation. Local government focuses on increasing benefits and decreasing negative consequences. This can refer to citizens who receive direct services or individuals or business affected by policy decisions or service delivery...”

From a system perspective, Mardiasmo (2002) defines performance measurement as an assessment tool which assesses strategy implementation through financial and non-financial measures. In addition, Mardiasmo (2002) emphasizes that budget is one of the financial measures used to assess strategy implementation; it is a primary instrument of many function of decision which is used as a tool to achieve organization goals. Jantz (2008) defines performance measurement as regular collection, recording and evaluation of performance data. With reference to Hood (2007)3 he reveals two different performance measurement systems:

1. Target system that measure current performance of a period (using previously defined performance metrics);

2. Rankings that measure current or past performance in relation to other comparable entities (often known as benchmarking). The objective here is to inform customers about entity’s performance or to provide political decision-makers with starting points for increasing performance.

Scholars and practitioners recognize that performance measurement is a management tool to maximize success. In line with this perspective, the Joint

3 Christopher Hood. 2007. Public service management by numbers: Why Does it Vary?

Where Has it Come From? What are the Gaps and the Puzzles? In Public Money and Management, April 2007 Vol 27 (2) 95-102.

7

National Performance Management Advisory Comission of United States of America and Canada (NPMAC, 2009) defines performance management as4,

“...is an on going, systematic approach to improving results through evidence-based decision making, continous organizational learning, and a focus on accountability for performance...”

Along with this definition, the Commission elaborates that performance management is integrated into all aspects of an organizations’s management and policy-making processess and transforms an organization’s practices so that they are focused on achieving improved results for the public. Performance measurement is part of a broader system of performance management. In public institution context, performance management consists of systematic recording and tracking of performance of public orgnanisations in order to promote a continous improvement process. Furthermore, we uncovered from academic and practitioners literature that performance measurement and reporting are critical elements of performance management; however, they are insufficient to promote organization’s learning and improved outcomes for public (Thiel and Leeuw, 2002; Dale, 2003; NPMAC, 2009). Thus, the notion of performance management, goes beyond measurement and reporting of organization’s performance; it attempts to systematically use performance data as well as other tools to promote continous learning for improvement, and strengthen organization’s focus on results (Sckeyde and Wagner, Eds., 2008; NPMAC, 2009; Kompas, 20105) 2.2 Types of Indicator in Performance Measurement System Boyne (2002) reports that scholars and practitioners generally draw performance indicators upon two corresponding but not entirely consistent models of organizational performance that contain a sequence of service production process. The first model links economy6, efficiency and effectiveness (3Es model) as key dimensions to measure organization performance (Boyne, 2002: page 17). The second model (Boyne 2002: page 17) is a linear structure of inputs-outputs-outcomes (IOO model). The latter has been discussed under Section 2.1 taking the Dale’s and GTZ models as examples.

4 National Performance Management Advisory Commission. 2009. A Performance

Management Framework for State and Local Government: From Measurement to Management and Improving, www.gfoa.org/.../PMCommissionFrameworkPUBLIC-REVIEW-DRAFT.pdf

5 Kompas News, 6th of May 2010 page 4: “Komisi II Pertanyakan Pemda”. 6 The term ‘economy’ (Boyne, 2002) is frequently equated with the level of spending on a

service, but it is more accurately defines as the cost of procuring specific service inputs of given quality.

8

In the 3Es model, the term ‘economy’ (Boyne, 2002) is frequently equated with the level of spending on a service, but it is more accurately defined as the cost of procuring specific service inputs of given quality (for example premises, staff, and equipment). However, this notion of economy is not without questions and it involves political issues. While the notion of economy leads authorities to seek to minimize the price paid for inputs or production factors, there is no common perception in literature whether squeezing the wages for labours or civil servant is good or bad performance. Moreover, Boyne (2002) maintains the argument that high or low spending in it reveals nothing about service standards, or the success or failure of local authorities. In contrast to economy, economists define efficiency in term of ‘technical effeciency’ and ‘allocative efficiency’ (Boyne, 2002). The first refers to cost per unit of output, while the later refers to the responsiveness of services to public preferences. However, the 3Es model ususally adopts the notion of ‘technical effeciency’ (Boyne, 2002). The concept of effectiveness refers to the achievement of the formal objectives of policy or services (Boyne, 2002; Dale, 2003; Sckeyde and Wagner, Eds., 2008). Recent academic and practitioners literature (Boyne, 2002; Dale, 2003; Sckeyde and Wagner, Eds., 2008; Dendi et al., 2010), however, informs that public organizations increasingly adopt a model with broader criteria or organizational performance. This model corresponds to the means-ends structure discussed in Section 2.1. However, the fundamental structure of the model involves “IOO” (Input-Output-Outcomes) indicators. As well, the “IOO” model embraces all elements of the 3Es model (inputs include expenditure, efficiency is the ratio of outputs to inputs, and outcomes include formal effectiveness). More over, the “IOO” model (Boyne, 2002) or its corresponding extended models (Dale, 2003; Sckeyde and Wagner, Eds., 2008; Dendi et al., 2010) not only make the “implicit” from the 3Es model to “explicit”7, but also emphasis to adopt both quantitative and qualitative dimensions of performance along with equity or fairness criteria. The equity or fairness of service provision can be assessed, for examples, by the distribution of outputs and income by gender, age, race, and geographical areas (Boyne, 2002). Performance measurement system in Indonesia adopted an extended “IOO” model as stipulated among others in Law 25 of 2004 regarding National Development Planning System, Government Regulation (GR) 39 of 2006 on Supervision and Evaluation of Development Plan Implementation, Decision of the Head of State Administration Institute (LAN) 239/IX/6/8/2003 on guidelines of performance reporting for governmental agencies, and Government Regulation 6 of 2008 on Guidelines of Evaluation of Local Governance

7 Readers interested to further explore conceptual advantages and disadvantages of the

“IOO” model can refer to Boyne (2002: page 18).

9

Implementation. The model involves a set of performance indicators namely Input, Output, Outcome, Benefit, and Impact. From academic and practitioners literature we identified that different performance measurement system used different sets of indicators reflecting different scope and focus of performance measures. OECD (2009: page 32) recognizes four categories of performance indicators, namely:

• Input measures: reveal what resources (e.g. people, money, and time) are used in what amounts to produce and deliver goods and services;

• Process measures: reveal the way in which activities are undertaken by a programme or project with the resources described;

• Output measures: capture the goods and services activities produced (e.g. number of Small and Medium Enterprises served, kilometres of road built);

• Outcome measures: capture the dimension that is expected to change as a result of an intervention (policy, programme, or project) and the outputs produced.

GTZ Sector Network “Governance Asia” (2010) recognizes three categories of performance incdicators consistent with the GTZ means-ends structure (Sckeyde and Wagner, Eds., 2008) but not explicitly consistent with the OECD’s notion of performance indicators, particularly in terms of process indicators:

• Ouput-oriented indicators: these indicators focus on the direct output of administrative actions Moreover, the primary relation of output indicators is with the input. Thus, output indicators always refer to efficiency and do not capture the notion of effectiveness;

• Outcome-oriented indicators: These indicators attempt to establish a relation between the effects of political decision making and subsequent activities. The outcome may include intended as well as non-intended effects and, therefore, outcome-oriented indicators refer to effectiveness;

• Process-oriented indicators: these indicators attempt to capture quality in a wider sense.

Furthermore, we observed and found in literature (Boyne, 2002; LGSP, 2009; Dendi et al., 2009) that In practice, however, the “IOO” model remains vulnerable to inappropriate orientation on quantitative input and output measures of performance. We further discuss this issue in the sections to come.

10

3. Performance Measurement System in Decentralized Indonesia Performance measurement in Public sector has multiple dimensions. In Indonesia there exists numerous public sector performance measurement instruments institutionalized or enacted through Government Regulation (GR). Table 1 presents a comparative summary of performance reporting and measurement instruments currently applied in Indonesia. We focuse on four performance measurement and evalution instruments. The first is LAKIP (Annual Performance Accountability Report of Governmental Institutions). It began effective in 1999 according to Presidential Instruction Number 7 of 1999. It was then consolidated through the operational guideline issued by the State Administration Agency (Decision 239/IX/6/8/2003). It was the first attempt to establishing a performance accountability system in Indonesia (Ind. SAKIP-Sistem Akuntabilitas Kinerja). It relies on self-assesment approach in performance measurement and reporting to ensure that individual governmental agency comply with the established operating procedures and standards (process indicators) and attempts to strengthen result-orientation (output and outcome indicators) among governmental agencies in any development intervention or service. While the process indicators refer to the norms, standards and procedures applied in the Indonesian administration system, the results indicators refer to performance indicators set up by government agencies in their strategic plan and action plan documents. Conceptionally these strategic and action plans must be coherence with the national mid-term development plan. The second instrument is the Evaluation of Implementation of Regional/ Local Governance (EPPD- Evaluasi Penyelenggaraan Pemerintahan Daerah). The EPPD pursues a number of objectives, namely: (i) assess the performance of sub-national governance in an attempt to increase governance performance according to the principle of good governance; (ii) assess regional/ local capacity in achieving the specified objectives of regional autonomy, namely improvement of people’s welfare, improvement of public service quality and regional competitiveness among others; and (iii) monitor the progress of establishment of soft and hard infrastructure for governance (institutional, infrastructure and human resources) in the newly established autonomous region. To pursue these objectives, the EPPD adopted three different performance measurement modules. The first module, namely the evaluation of sub-national (regional/ local) governance performance (EKPPD), refers to the first objective of performance assessment (EPPD). The second module, namely evaluation of capacity to execute regional autonomy performance (EKPOD), refers to the second objective of performance assessment. The EKPOD, however, will be executed only in regions graded as poor performance for three consecutive years. The third module, namely evaluation of the newly established autonomous region, refers to the third objective of performance assessment. In this paper, however, we discuss mainly the first module namely the EKPPD.

11

Furthermore, Government Regulation (GR) 6 of 2008 defines “the performance of implementation of local governance” as results of implementation of local governance functions measured through input, process, output, results and benefit/ impact indicators. In line with this definition, EKPPD is defined as a systematic process of collection and analysis of governance performance data using performance measurement system. The EKPPD sets two arena of performance assessment, namely the elected policy makers (policy making level) and implementing agencies at povincial as well as at district/ municipal levels. Furthermore, the performance assessment for policy makers involves thirteen (13) dimensions with 63 governance-related key performance indicators in total for provincial level, and 74 indicators for district/ municipal level (more detail in Table 2). These dimensions, among others, include public order; coherency between regional and national policies; effectiveness of relationship between regional/ local government and the regional/ local legislative council; effectiveness of decision making process by the provincial/ local legislative council; effectiveness of decision making proces by the Head of Region ; Compliance to the regulations and standards; transparency; and innovation in governance. Meanwhile, the performance assessment for implementing agencies involves nine (9) dimensions with a total of 151 and 174 key performance indicators for provincial and district/ municipal level respectively. The third instrument is EPRPD (Evaluation of Sub-National Development Plan Implementation). It includes assessment of inputs-outputs relation and realization as well as achievement of outcomes (including benefits/ impacts) of development programs or services using the regional (province) and local (district/ municipal) budgets. Indeed, article 51 of GR 8 of 2008 necessitates elaboration of Ministrial Regulation for operational guidelines of EPRPD. However, that regulation has not yet been prepared. The leading agency for EPRPD at national level is the Ministry of Home Affairs. At provincial level, the Head of Provincial Development Planning Board is in charge of conducting the EPRPD. Similarly, the Head of Local Development Planning Board is in charge of conducting the EPRPD at district/ municipal level. The fourth instrument refers to the supervision and evaluation of national development plan implementation (Ind. PEPRP) according to Law 25 of 2004 and GR 39 of 2006. It focuses on programs financed through national budget including programs financed through deconstration and co-administration schemes. The leading agencies of PEPRP at anational level is the State Ministry of National Development Planning (BAPPENAS). It involves quarterly progress monitoring, evaluation of the implementation of annual plan of Ministries/Non-Ministerial State Institutions, and evaluation of the implementation of mid-term (five years) development plans of Ministries/ Non-Ministerial State Institutions. Both progress monitoring and evaluation of annual plans were done quarterly, while the evaluation of mid-term development plans must be conducted at least once during the five-year implementation period, latest 1 year before the end of period. The mid-term development plan evaluation focuses on measuring outputs, outcomes,

12

macro-economic framework condition/ impacts as specified in the mid-term dvelopment plan documents. Table 1: Overview of Instruments of Performance Measurement for Central

and Sub-National Governments

Indicators

Instrument and Arena of Asssessment

Leading Agency and Particular Legal Base

Inpu

t

Pro

cess

Out

put

Out

com

e

Ben

efit/

Im

pact

Remarks

LAKIP All central level and sub-national level governmental institutions.

Ministry of Civil Service and Bureaucratic Reform; Presidential Instruction Number 7 of 1999 and the operational guideline from State Administration Agency (Decision 239/IX/6/8/2003);

yes yes yes <yes <yes < yes denotes that although conceptionally the indicator must be reported, we lack the evidence that these indicators were appropriately reported.

EKPPD (Module 1 of EPPD) Policy makers (Head of ) Provincial Governments; District/ Municipal Governments; Provincial/ Local Parliaments (DPRD); and Implementing Agencies at Provincial and District Levels (SKPD).

Ministry of Home Affairs (MoHA) Law 32 of 2004 on Regional/ Local Governance; GR 3 of 2007 on sub-national Government report to Central Government and information to provincial/ local parliament); GR 38 of 2007 on Functional Assignments; GR 41 of 2007 on Organization of Local Government; GR 6 of 2008 on Evaluation of Local Governance Implementation; Regulation of MoHA 73 of 2009;

yes yes yes yes yes

13

Indicators

Instrument and Arena of Asssessment

Leading Agency and Particular Legal Base

Inpu

t

Pro

cess

Out

put

Out

com

e

Ben

efit/

Im

pact

Remarks

EPRPD (Evaluation of Sub-National Development Plans Implementation): Provincial & Local Governments and Implementing Agencies (SKPD)8.

Ministry of Home Affairs (MoHA, Governor, Regent/ Mayor); Law 32 of 2004 (Article 154); GR 8 of 2008 on Steps, Procedures of Formulation, Supervision and Evaluation of Sub-National Development Plans;

yes yes yes yes ?

• Evaluation of National Development Plans Implement-ation:

• Progress

Monitoring of Implementation of Development Plans;

• Evaluation of

Implementation of Annual Plan of Ministries/ Non-Ministrial State Institutions

• Evaluation of

Strategic Plan

Ministry of National Development Planning (BAPPENAS) Law 25 of 2004 on National Development Planning System; GR 39 of 2006; [*Ref. Article 4 of Para 4 under Chapter two of GR 39 of 2006; **Ref. Article 3 of Para 13 under Chapter two of GR 39 of 2006 reg evaluation of the Annual Plan of Ministry/ Institution & Annual Plan of Government]; [*** Article 5 of Para

Yes* Yes* Yes

Yes* Yes* Yes

Yes* Yes* Yes

No Yes** Yes***

No No Yes***

8 This evaluation is primarily based on quarterly report of respective agencies to the Head of

BAPPEDA (Development Planning Board).

14

Indicators

Instrument and Arena of Asssessment

Leading Agency and Particular Legal Base

Inpu

t

Pro

cess

Out

put

Out

com

e

Ben

efit/

Im

pact

Remarks

of Ministries/ Non-Ministrial State Institutions & National Mid-Term Development Plan

15 under Chapter three as well as in the Elucidation of GR 39/ 2006 reg. Evaluation of Strategic Plan of Ministries/ Non-Ministrial State Institutions & National Mid-Term Development Plan

Table 1 shows that these existing performance measurement instruments have some shared objectives and measure the same dimensions of performance (economy, effeciency and effectiveness) and the same level of indicator along results chain (input-process-output-oucome-benefit/ impact). In addition, we uncovered that these instruments put too strong focus on quantitative measures of performance (particularly input and outputs) and pay less attention to the qualitative dimension of performance. In addition, all instruments overlook the different context that influence [local] government’s performance; this may lead to punishing inappropriately poor regions working under difficult conditions or rewarding inappropriately better-off regions working under favorable context. We further discuss these issues and other problems under Section 4. Table 2: Arena and number of performance indicators at provincial,

municipal and district levels

Province Municipal District Policy level

Implement- ation level

Policy Level

Implement-ation level

Policy Making Level

Implement-ation level

General Administration

28 28 28

Obligatory Functions

107 120 120

Discretional Functions

16 26 26

Total 63 151 74 174 72 174

15

The number of performance indicators involved for the assessment of local governance performance require robust and plausible baseline data and milestones to measure the progress. In addition it will require tremendous efforts and strong commitment to collect evidence and report the progress of achievements. Sub-national governments, however, have not only to do self-assessment and deliver report according to the EKPPD framework but also for other performance evaluation schemes as presented in Table 1. Overall, sub-national government has to deliver more than 50 different kinds of petformance and accountability report annually. Table 3 highlights selected dimensions and example of performance indicators for policy makers and implementing agencies at provincial level only according to EKPPD framework. Tabel 3: Some dimensions and key performance indicators (KPIs) for local

governance performance (EKPD) at Provincial level Arena Performance Dimension Examples Key Performance Indicators

Policy Makers (63 KPIs)

Public Order (10 KPIs) • Availability of Provincial Regulation on Spatial Plan (Yes/ No);

• Proportion (%) of District/ Municipal in the province having Regulations on building permit;

• Frequency of demonstration against Provincial/ District/ Municipal Regulations;

Coherency between regional and national policies (13 KPIs)

• Proportion (%) of regional development priorities in coherence with national development priorities;

• Obligatory functions (%) implemented by Provincial/ District/ Municipal Government according to GR 38 of 2008 (26 obligatory functions in total);

• Ratio of expenditure for health and education to total expenditure (%);

• Proportion of district/ municipalities of the province establishing regulations on public service (%);

• Coherency between provincial/ local governance structure with GR 41 of 2008;

Intensity and

effectiveness of public consultation (2 KPIs)

• Number of public consultation conduceted; • Existence of information media thatis

reinforced by Governor’s regulation and accessible by public;

Innovation in local

governance (10 KPIs) • Ratio of budget allocated for innovative

governance to total expenditure (%); • Economic growth (GRDP growth); • Establishment of E-procurement (Yes/ No);

Implementing Technical Policies of • National programs implemented by

16

Agency: General Adminstration Aspect

governance (2 KPIs) implementing agencies at regional/ local level (%);

• Existence of Standard Operating Procedures (Yes/ No);

Institutional Structure (3

KPIs) • Aggreement of organizational structure, tasks

and functions assignment with GR 41 of 2007; • Number of functional positions without

occupant;

Regional/ Local Development Planning (4 KPIs)

• Existence of planning documents (numbers); • Number of programms in regional development

plan documents accomodated in the implementing agency’s development plan documents;

Local Financial Management (5 KPIs)

• Ratio of implementing agency’s budget to total expenditure of region;

• Ratio of capital expenditure to total expenditure of the implementing agency;

Implementing Agency: example of Obligatory functions

Health Sector (11 KPI) • Live expectancy; • Number of villages (%) included in the

Universal Child Immunization Service; • Number (%) of undernourished children under

five years old getting appropriate medical treatments;

• Number of identifiable dengue fever cases treated appropriately;

Overall, the EKPPD for policy making and imlementation levels captured mainly the process and output-oriented indicators. However, for the case of obligatory functions (at implementation level) a few number of outcome-oriented and impact indicators were included, such as life expectancy, regional economic growth, increased productivity of food crops, sectoral contribution to GDRP, and reduced unemployment rate.

4. Prospects and Problems in the Implementation of Performance

Measurement 4.1 Prospects The Indonesian government has apparently made tremendous efforts and innovation to establishing performance measurement system as a management tool to improve results and its public accountability. Conceptionally the Indonesian performance measurement system adopts an extended “IOO” (input-output-outcome) model in which the outcome is further elaborated into benefit and impact. Adoption of this model could stimulate change toward estabalishing more efficient and responsive goverment and strengthening focus on results along the principles of good governance. We observed that implementation of performance measurement, reporting and evaluation has considerably induced multiple stakeholders’ debates and dialogues which, in turn, could help improve the quality of governance and, thus, the quality of democratic and economic outcomes at all levels.

17

However, implementation of performance measurement in decentralized context in Indonesia apparently faced a number of fundamental chalenges. We discuss some of these challenges in the following section. 4.2 Issues and Problems 4.2.1 General Issues and Problems We recognized that numerous existing sectoral laws and regulations could provide plausible legitimicacy for the central and sub-national level governments to plan, allocate resources and implement performance measurement. These laws and regulations also provided doorway for civil society organizations to participate in performance measurement and evaluation. However, these sectoral regulatory frameworks brought about not only legitimacy and greater potential toward creating better governance outcomes but also notable conflicting orientation, overlapping authorities and missmatching criteria and indicators of performance (e.g. GR 3 of 2007 and GR 6 of 2008). Furthermore, inter-ministrial platform to coordinate and sincronize regulatory frameworks and instruments of performance measurement has not been effective due to lack of functional clarity and collective commitment among others. Meanwhile, majority of eligible civil society organizations lack the incentive, secure financial resource and technical capacity to establish and mobilize multi-stakeholders’ processes to make performance measurement works for enhancing evidence-based policy making and strengthens outcome-orientation among public agencies at all levels. However, indepth analysis on regulatory framework and its consequences to effectiveness of perfromance measurement is beyond the scope of this paper. Some previous practioners’ and our own observations added evidence to support the multiple principals and multiple tasks (goals) trade-off hypothesis (Propper and Wilson, 2003). According to this hypothesis, a public sector agency inevitably serves multiple stakeholders ranging from self-interested individuals, the service users, tax payers, politicians, private sector organizations and civil society organizations at different levels among others. This feature, in one way or another, determines public sector organizations to pursue multiple ends (multiple goals) which are not necessarily coherence each other. These multiple and incoherence goals created difficulty in selecting appropriate performance indicators and in avoiding the “3Es trade-off” (Economy, Effeciency and Effectiveness). For instance, to pursue the goal of improved quality and scope of health service for the poor people, local governments do have difficulty to match both effeciency and effectiveness criteria. This is, of course, not to say that a public agency has unchallenged legitimacy to deliver service for public at any cost. Elected policy makers face increasing challenge to provide services in more accountable, efficient and cost effective ways.

18

Furthermore, contextual diversity and inter-regional disparity in Indonesia reasonably mobilized local governments and other stakeholders including politicians to challenge the application of uniform criteria and indicators of local governance performance. This debate calls for improvement of methodology including selection of appropriate criteria and key performance indicators along with appropriate weighting of performance scores. These call for more systematic and coherence performance measurement system, improvement of indicators selection process along with capacity building efforts, particularly at regional and local levels.

4.2 Issues and Problems at Regional/ Local Level The problems inherent in the regulatory framework of performance measurement as discussed in Section 4.1 have far reaching consequences to sub-national level governments and other stakeholders. The response of sub-national government and legislative councils toward the central government-led performance measurement varied considerably. A few number of sub-national governments, e.g. Solok district of West Sumatera province, Sragen district of Central Java, Tarakan City in Kalimantan, Jembrana district in Bali as well as a number of regions in East Java and Eastern Indonesia (e.g. Mataram City and Bima City) were widely recognized for their “best practices” innovation in local governance and performance management among others. However, in turned out that outcome-oriented services and performance measurement had not been fully adopted as governance perspective and instrument by the majority of sub-national governments. We observed that support structure to coordinate and implement outcome-focus performance measurement was either not in place or ineffective. Moreover, many performance reports maintained the “tradition” of reporting financial inputs and public expenditures to produce physical outputs and services. Furthermore, sub-national governments in most regions lacked reliable and consistent baseline data and benchmarks for the formulation of realistic performance targets as well as for milestones of performance measurement. Although the central and sub-national governments have made substantial efforts to improve availability and quality of data, following problems remain pervasive in sectoral and regional planning documents:

• Lack of horizontal and vertical coherence among planning documents (including their objectives framework and performance indicators);

• Vague link between outputs and outcomes; • Outputs and outcome indicators were not clearly defined; • End performance targets were defined but no sufficient baseline data

were given as milestones for performance measurement. Furthermore, it was well recognized that capacity of human resources to adopt the Input-Output-Outcome (IOO) model in planning and performance

19

measurement varied considerably among regions. This limiting factor calls for a more comprehensive capacity building approach including individual, organizational and system levels. 5. Lessons Learned and Ways Forward The Indonesian government has made tremendous efforts to establishing performance measurement system as a management tool to improve results and its public accountability. Conceptionally the Indonesian performance measurement system adopts an extended “IOO” (input-output-outcome) model in which the outcome is further elaborated into benefit and impact. The model shows the interest of Indonesian government to strengthen focus on results and, thus, strengthen its client orientation (benefits/ impacts for citizens, individuals and business). However, implementation of the adopted model in Indonesia has not brought sufficient incentive for public agencies at all levels of administration to perform better. Major problems and obstacles to implement performance measurement system in Indonesia are discussed in the previous section. In short, we uncovered that inherent institutional complexity along with the overwhelming data needed for self-assessment and reporting limit its actionability and effectiveness. The established system is just not doable! Furthermore, we observed that the currently applied performance measurement instruments maintain the “tradition” to focus on quantitative measures and overlook the value of qualitative dimension of performance. These quantitative measures tell us very little about why a public agency performs or does not perform well. In the absence of coherent objectives frameworks, effective support structures, clearly defined performance indicators and competent human resources, the central and sub-national governments do have difficulties in implementing the concept of outcome-oriented performance measurement as well as the concept of performance-based budgeting. These problems, however, were not fully indpendent from conflicting political orientation along with the multiple principals trade-off as discussed in Section 4.1. One among several critical challenges is, thus, to establish a performance measurement system that meets the citeria of good governance, coherent legal and institutional framework that clarify interlinkages between planning, performance measurement and performance-based budgeting. Understanding of this issue would facilitate the development of a nested model and workable concepts as well as self-dynamic (more client orientation-driven than regulation-driven) performance measurement system. It is necessary to enhance evidence-based and accountable decision-making and to strengthen the focus of any intervention on results (outcomes). Toward these objectives the following perspectives and measures are forwarded:

20

The State Ministry of National Development Planning should take a leading role in exploring best practices, facilitating multiple stakeholders’ dialogues toward establing coherent conceptual, legal and institutional frameworks for performance measurement. This is in line with the perspective in the Indonesian Mid-Term Development Plan of 2010-2014.

• In our view, impact monitoring and evaluation play a critical role in complementing the recurring performance measurement. It attempts to capture longer-term and aggregated outcomes, namely ends outcomes in terms of benefits/impacts for the public. These ends outcomes, of course, are the results (effects of interactions) of multiple-actors’ efforts. In economic terms, these ends outcomes are indeed the sum of marginal outcomes among multiple actors in governance and development. In contrast to impact monitoring and evaluation, performance measurement should maintain focus on the quality of process and the quantitative and qualitative dimensions of immediate outcomes.

• Furthermore, the impact monitoring and evaluation attempts to capture not only the formalistic performance indicators (set up by service providers) but also external stakeholder’s perspective (external indicators). The notion of external indicator measures the satisfaction of the public on responsiveness, effectiveness and accountability of the public service agencies.

• Along with the above efforts, Civil Society Organizations should use the opportunities emanating from the decentralization policy and regional autonomy to play an active role to make performance measurement work for improving public services as well as for improving democratic and economic outcomes. This would need a comprehensive capacity building covering individual, organization and system levels.

21

Acknowledgements The author gratefully acknowledges Dr. Manfred Poppe, Senior Component Team Leader of Decentralisation as Contribution to Good Governance, for his valuable comments for the revision of this paper. I thank my colleagues from the Decentralisation as Contribution to Good Governance (DeCGG) Project, particularly M.P. Dwi Widiastuti, Mr. Mario Vieira, Mr. Arifin Bhakti for sharing their observations and empirical experience from sub-national level. I also thank Meithya Rose Prasetya and Jevelina Punuh for their assistance in literature review and data collection.

References Anonymous. 2009. Hasil penilaian kinerja pelayanan publik kabupaten/ kota

provinsi Nusa Tenggara Barat 2009. Pemerintah Provinsi Nusa Tanggara Barat dan GTZ-Good Local Governance (GLG). Unpublished.

Anonymous. 2009. Laporan Hasil Evaluasi Pemeringkatan Kinerja Penyelenggaraan Pemerintahan Daerah Propinsi, Kabupaten, dan Kota Berdasarkan LPPD Tahun 2007 Tingkat Nasional. Tim Independen Universitas Gadjah Mada dan Kementerian Dalam Negeri Republik Indonesia.

Anonymous. 2010. Laporan akuntabilitas kinerja instansi pemerintah (LAKIP) tahun 2009. Pemerintah Kabupaten Sumba Barat Daya.

Anonymous. 2010. Technical paper on performance measurement for local governance. GTZ Sector Network “Governance Asia”, Working Group “Decentralization and Local Governance.

Boyne, G.A. 2002. Concepts and indicators of local authority performance: An evaluation of the statutory frameworks in England and Wales. Public Money and Management April-June 2002: pp 17-24. Great Britain.

Boyne, G.A., K.J. Meier, L.J. O′Toole Jr, and R.M. Walker. 2006. Public service performance: Perspectives on measurement and management. Cambridge University Press. New York. 319 pp.

Bryson, J.M. 2004. Strategic planning for public and non-profit organizations, 3rd Edition: A guide to strengthening and sustaining organizational achievement. Jossey-Bass, A Wiley Imprint. San Francisco. 430 pp.

Carol, P., D. Wilson. 2003. The use and usefulness of performance measures in the public sector. CMPO Working Paper Series No. 03/073. www.bristol.ac.uk/cmpo/publications/papers/2003/wp73.pdf

Dale, R. 2003. The Logical Framework: An easy escape, a straitjacket, or a useful planning tool? Development in Practice (Vol. 13, Number 1: pp 57-70). Oxfam GB Carfax Publishing.

22

Dendi, A., H.J. Heile, S. Makambombu. 2009. Improving the livelihoods of the poor: Challenges and lessons from East Nusa Tenggara. In: Budy P. Resosudarmo and Frank Jotzo, Eds. (2009): Working with nature against poverty; Development, resources and the environment in Eastern Indonesia. ISEAS Publishing. Singapore. pp 305-320.

Dendi, A., A. Zaini, M. Afifi, R.Saleh H. 2010. Perencanaan strategik partisipatif pengembangan ekonomi lokal dalam bingkai ekonomi kerakyatan. Pusat Penelitian Kependudukan dan Pembangunan Universitas Mataram. Forthcoming publication.

Diamond, J. 2005. Establishing a performance management framework for government. IMF Working Paper.

Dwiyanto, A., Partini, Ratminto, B. Wicaksono, W. Tamtiari, B. Kusumasari, M. Nuh. 2006. Reformasi birokrasi publik di Indonesia. Gadjah Mada University Press. Yogyakarta. 274 pp.

Francesco, M.D. 2001. Process not outcomes in New Public Management? Policy coherence in Australian Government. The Drawing Board: An Australian Review of Public Affairs (Volume 1, Number 3, March 2001: 103-116. Sydney.

Hatry, H.P. 2006. Performance measurment: Getting results. 2nd Edition. The Urban Institute Press. Washington, D.C. 326 pp.

Kelly, J.M., and W.C. Rivenbark. 2003. Performance budgeting for state and local government. M.E. Sharpe. Armonk, New York; London, England. 271 pp.

LGSP, 2009. Tantangan dalam penyelenggaraan evaluasi kinerja berbasis hasil (outcome-based) untuk Pemerintah Daerah di Indonesia. Local Governance Support Program (LGSP), USAID. www.lgsp.or.id.

Mahsun, M. 2006. Pengukuran kinerja sektor publik. Edisi I. BPFE-Yogyakarta. Yogyakarta. 249 pp.

McNamara, C. 2006. Field Guide to Consulting and Organizational Development: A collaborative approach and systems approach to performance, change and learning. Authenticity Consulting, LLC. Minneapolis, MN USA. 499 pp.

MoHA. 2009. Laporan hasil evaluasi pemeringkatan kinerja penyelenggaraan pemerintahan propinsi, kabupaten dan kota berdasarkan LPPD Tahun 2007 tingkat nasional. Kementerian Dalam Negeri Republik Indonesia (MoHA-Ministry of Home Affairs of Republic of Indonesia).

OECD. 2009. Governing regioanal development policy: The use of performance indicators. OECD publishing. www.sourceoecd.org/governance/9789264056282.

Schmidt, T. 2009. Strategic project management made simple: Practical tools

for leaders and teams. John Wiley & Sons, Inc. Hoboken, New Jersey. 251 pp.