Module 5.3Measuring the performance of PFM systems

Module map

Measuring performance: What?

• The objectives of PFM-(cf. module 1.1):• The 3 specific objectives

Aggregate fiscal discipline Allocation of resources in conformity with policy objectives Operational efficiency

• PFM “traditional” values: Regularity, timely, fair procedures

• The role of Parliament in democratic society, accountability, transparency

Measuring PFM system performance: tools?

4

PER CFAA CPAR ROSC EC Audits FRA OECD/DAC NPA PEFA PMFReview of public expenditure policies and budgetary outcomes X

Review of political incentives X XHigh Level overview of PFM performance X XComprehensive review of PFM functions XIdentification of PFM strengths and weaknesses

X X X X X X X X

Compliance Testing X X XIn-depth analysis of capacity factors X X X X

Recommendations for reform X X X X XAssess risk to public funds X XTrack Progress over time X X

Measuring PFM performance: PEFA

5

PEFA Strengthened approach

1.Encouraging a country-led reform program defined within a Government Reform Strategy and Action Plan

2.Ensuring a harmonised and donor aligned program of support for the country reform program

3.Adopting a common assessment framework applied to a common pool of PFM information for measuring and monitoring PFM reform progress overtime

PEFA Initiative launched

Paris Declara-tion

Mainly IMF interest in

PFM systems

More donors

PEFA Framework

1983

Interest e.g. WB, regional

banks

1998 2001 March 2005

June 2005

•Strengthened approach developed by 8 donors and shared by many others

•Need for result orientation in development of PFM systems and harmonization of PFM analytical work

•Many donors show interest in PFM because of increased preference for budget support as aid modality

Measuring PFM performance: PEFA

• Focus on high level performance indicators applied to central government level

• Widely applicable: Relevant to countries at all levels of development

• Assessment must be evidence based• Capable of calibration to capture progress over

time

7

The PEFA Design Principles

• Comprehensive: cover all aspects of the PFM cycle• Use data that can be collected cost effectively• Define the achievement based on the 3 objectives• Incorporate a measure of the impact of Donor

practices on PFM performance

8

The PEFA Design Principles

9

PEFA Model

10

The Central Government Budget

PEFA Scope

PEFA

• 6 critical areas of a PFM system are distinguished

• On which performance is measured with a standard set of

• 28 government performance indicators

• 3 donor indicators, reflecting donor practices influencing the government’s PFM systems

• Most indicators are composed of various dimensions: in total 72 dimensions

• Calibration and Scoring

• Specific calibration of scores using a four point ordinal scale (A, B, C and D)

• Intermediate scores (B+, C+, D+) for multi-dimensional indicators, where dimensions score differently

• Arrow ▲ can indicate an improvement not reflected in change of indicator score

• Two scoring methods:

• Method M1 ‘weakest link among dimensions’

• Method M2 ‘average of dimensions’

PEFA Scorings

Budget formulation and preparation

13

Indicators PI-11:Orderliness and Participation in the annual Budget Process

Scores the existence and adherence to a fixed budget calendar, the utilisation of clear and comprehensive budget circulars whose preparation involves political inputs

Budget formulation and preparation

14

Indicators PI-12: Multi-year perspective in fiscal planning, expenditure and budgeting

Scores multi-year forecasts, links to the annual budgets; existence of fully costed sector strategies, the frequency and scope of debt sustainability analysis carried out and the linkages between budgets and forward expenditure estimates

Predictability and Control

15

Indicators PI-16: Predictability in the availability of funds for commitment of expenditures: scores the extent to which cash flows are forecast and monitored, ceilings for expenditure commitment are reliably communicated to spending ministries and the frequency and adjustments to budget allocations

Indicators PI-17: Recording and Management of Cash Balances: scores the regularity, comprehensiveness and timeliness of reporting on domestic and foreign debt, and on cash balances; and the controls instituted for contracting loans and guarantees

Predictability and Control

16

Indicators PI-18: Effectiveness of Payroll controls: scores the degree of integration between the personnel database and the payroll, the timeliness of updates to the personnel records and payroll, the effectiveness of internal controls to make changes to personnel and payroll records and the utilization of payroll audits to guard against ghosts and double dippers

Indicators PI-19: Competition, Value for Money and Controls in Procurement: scores the degree of utilization of the open tender method, the justification of the use of less competitive methods and the existence and operation of a procurement complaints mechanism

Predictability and Control

17

Indicators PI-20: Effectiveness of Internal Controls for non-salary Expenditures: scores the effectiveness of a commitment control mechanism and the comprehensiveness, understanding of other expenditure control procedures and the degree of compliance to the expenditure rules

Indicators PI-21: Effectiveness of Internal Audit: scores the coverage of internal audit and the degree to which system audits are carried out; adherence to a fixed schedule of reporting and the extend to follow up and management response to findings.

Accounting, Recording and Reporting

Indicators PI-22: Timeliness and Regularity of Account Reconciliation: scores the regularity and timeliness of accounts reconciliation and the clearance of suspense accounts and advances

Indicators PI-24: Quality and timeliness of in-year budget reports: scores the scope and consistency of reporting format with the budget of the in-year budget reports; the regularity and timeliness of the issuance of in-year budget reports and on the accuracy of the data

Indicators PI-25: Quality and timeliness of Annual financial statements: scores the completeness of financial information vis-à-vis revenue, expenditure, financial assets and liabilities; the timeliness of the submission of final accounts and the accounting standards adopted

External Audit and Legislative Oversight

19

Indicators PI-26: Scope, Nature and Follow-up of External Audit : scores the audit coverage, audit standards and range of audits performed; the timeliness of audit submittals to the legislature and the extent of audit follow up

Indicators PI-27: Legislative Scrutiny of the Annual Budget Law: scores the scope of legislative scrutiny beyond details of revenue and expenditure; the establishment and respect of well defined procedures; the adequacy of the time allotted for the legislature to review the budget and the adherence to strict rules on the extent and nature of amendments to the budget

Indicators PI-28: Legislative Scrutiny of External Audit Reports: scores the timeliness of examination of audit reports, the extent of hearings on key audit findings and the the issuance of legislative follow up actions and implementation by the executive

The PEFA Iceberg – how comprehensive?

20

PEFA Assessment

Fixed asset register

Supply chain management

Financial administrative network

Capacity

ProcurementPolitical context

Market

Quality of expenditure management

Engaged civil society

The weakest link?

21

the strength of a foundation is determined by it weakest footing.

The performance of a PFM system is determined by the strength of its weakest PFM activity.

As a general rule PFM scores, like the strengths of foundation footings, cannot be averaged

Movements in scorings may vary only slowly over time

The weakest link (M1 method) will contributes further to slow changes in indicator scorings

Significant improvements may not be reflected in a score change

‘Cherry picking’ of performance indicators without substantial improvements

PEFA in practice

22

Budget Support and PEFA

23PFM Progress over time

Measurements in progress over time using PEFA

Requirements for demonstrated progress to facilitate GBS disbursements

Agreed reviewable milestones

Key messages

• The PEFA Framework measures the performance of PFM systems in achieving their objectives

• PEFA generally focuses on the “basics” of PFM systems

• PEFA can be used to monitor progress in strengthening systems

• PEFA does not deal with policy issues