KPI monitoring in Ukraine

Viktor Maziarchuk

Financial and economical analysis office at the VRU

Head

Structure of presentation

About us

Types of products

KPI

Founders

Timeline

2015.10 Start

2015.11 Letter of Intent

2016.02 Kick off event

2016.08 New team

2016.12 Mid term review

Part of VRU

2017.04 New structure

2018.05 New phase

Objectives

Enhances the effectiveness and quality of economic and financial legislation

being produced by the VRU

Increase the utilisation of financial information available

in Ukraine for effective oversight

and scrutiny

Beneficiary

Committee on

Economic Policy

Committee on

Industrial Policy and

Entrepreneurship

Budget Committee Committee on

Taxation and Customs

Policy

Committee on

Financial Policy and

Banking

Committee on Social

Policy, Employment

and Pension Provision

Committee on Fuel

and Energy Complex,

Nuclear Policy and

Nuclear Safety

Committee on State

Building, Regional

Policy and Local Self-

Government

StructureHead of Office

Budget cluster

1) Senior economist

2) Analyst

3) Analyst

Financial issues

1) Senior economist

2) Analyst

Economic issues

1) Senior economist

2) Analyst

Cross-Cutting Issues

1) Senior economist

2) Analyst

Communication manager

Legal expert Disainer

I) Committee on State

Building, Regional Policy and

Local Self-Government

II) Committee on Economic

Policy

III) Committee on Industrial

Policy and Entrepreneurship

I) Budget Committee I) Committee on Taxation

and Customs Policy

II) Committee on Financial

Policy and Banking

I) Committee on Fuel and

Energy Complex, Nuclear

Policy and Nuclear Safety

II) Committee on Social

Policy, Employment and

Pension Provision

Structure of presentation

About us

Products

KPI

Products & Services

Reactive (by request)

Proactive (working

plan)

FEAO products

I) Products:Quarterly budget review Budget best practices

II) Financial/economic synthesis (1 page):InfographicsAccounting chamber 1 page reportReports of State Agency reviews

III) OtherAnswers of State agency - Budget revenue and expenditureMethodology of financial and economic analysis of draft Laws

I) Products:Legal researchSocio-economic researchCost benefit analysisFinancial analysis

Products & Services

Written Oral

65 requests 23 MP – 13 Committees

129 products

Products

InfographicsProducts Accounting Chamber reports

Products

Series budget infographics

1. General budget process2. Budget preparing3. VRU adopt State budget4. Budget execution

Communication Tools

Office space in the VRU

Materials available at:

i) FEAO website - 1416 people/month

ii) FB page - 1306 people

iii) monthly newsletter – 1380 people

Contributing to the parliamentary TV channel

Following the Committees’ agenda and participation in the Committees’ meetings

Networking with MPs’ service

Delivering advise to the Committees’ staff

Collaboration with media outlets and NGO

Structure of presentation

About us

Products

KPI

Law base pillars

KP

I

Budget code

MoF orders

State authority and MoF orders

Law bases

Budget codeArticle 1. budget program passport - a document that defines the objective, tasks, and areas of use of budget funds, state authorities, effective indicators (KPI), and other characteristics of budget program under the budget allocation established by the Law on the State Budget of Ukraine (decision on a local budget);

Article 20.5. The effective indicators of a budget program are used to assess the efficiency of the budgetary program and include quantitative and qualitative parameters defining the outcome of the budget program, its progress, the degree of achieving its objective and performing its tasks. Such indicators shall be supported by the official state statistic, financial and other reporting, accounting information, and statistical and internal (managerial) reports. The list of effective indicators for each budget program is developed by key spending units according to the regulation by the central executive body, which ensures formulation of the State budget policy.

Law bases

Budget code

Article 6. The participants of budget process assess the budget programs performance throughout all the stages of budget process, which implies monitoring, analyzing, and exercising control over the use of budget funds in an efficient manner and for the intended purpose. The budget programs performance is assessed based on the analysis of budget program effective indicators, as well as other information contained in budget requests, breakdowns of costs, budget program passports, and reports on meeting breakdowns of costs and meeting the requirements of budget program passports. Organizational and methodological foundations of assessment of budget programs performance are defined by the central executive body, which ensures formulation of the State budget policy.

The results of assessment of budget programs performance give the grounds to make decisions on duly introducing changes to budget allocations of the current budget period, and corresponding proposals to the draft budget for the planned budget period and to the budget projections for the two budget periods following the planned one, including suspension of the respective budget programs.

Law bases

Budget code

Article 22. 5. A key spending unit:

6) develop and approve passports of budget programs and generate reports on their execution, analyze the performance indicators of budget programs (if Performance Program Budgeting method is used in the budget process), ensuring in the process the following:

Law bases

MoF order: KPI at PBP

The KPI quantity at the PBPcan be defined by stateauthority according to PBPspecific.

MoF order: Budget request

The KPI quantity should not beexcided more that 4 per one foreach KPI groups, in general.

Comment:

16 KPI max for a budget program =4 KPI for each of 4 KPI groups

Law bases

MoF special order: KPI at PBP

KPI criteria:

1. Realistic - determined taking into account the indicators of the current situations and existing needs of the economy and society, the indicators determined by the forward-looking documents on economic and social development and the possibilities of the resource of the state budget.

2. Relevant – consistent with the program and strategic documents, the strategicobjectives of the key managers, and the indicators of their performance in themedium term, characterize the most important aspects of the expected result

3. Social significance - highlight the effect that the economy and society receive through the activities, provide an opportunity to assess the impact on economic and social development, the efficiency of the provision of public services.

Law bases

MoF special order: KPI at PBP

KPI groups:

1. Expense

2. Product

3. Effectiveness

4. Quality

The KPI developed based on strategic goals of state authority in a midterm period.

KPI monitoring

Why we focus on KPI – hypothesis:

1) no linkage between expenditure and outcomes (KPI)

2) show “real picture”:• nobody know what is going on• collect and publish information• produce and promote analytics

3) instrument for MP and civil society

4) change focus of from budget “consumption” to outcomes

As a result budget expenditure system more efficiency.

Steps

1. Collect PBP reports from web-sites (less than 40%)

2. Send request to state authorities that not publish reports

3. Form template

4. Make data available to calculation (from pdf to excel)

5. Data classification

6. Analysis

Data classification

Data classification

Budget system

State budget

125 state authorities

481 passport of budget programs – PBP:

437 adopted

44 or 9% not adopted (2018-06-15)

10,742 KPI at the PBP

average 22 KPI per PBP

max 153 KPI per PBP Ministry of Defense

Local budgets

9618 local authorities

… PBP

… KPI

Key findings

General information for PBP 2016

85 from 458 budget programs not required MoF recommendation (19% of all program) for quantity of KPI

2,718 from 10,742 KPI in this budget programs (25% of all KPI)

32 KPI on average in this budget programs

Key findings

Government efficiency 2016

32% of KPI achieved

51% of KPI achieved

+/- 10% deviation

12% of KPI non achieved or

doubled:

8% of KPI not started

4% of KPI over

0

8

1 1 1 2 2 23

57

12

32

7

42 1 1 1 1 0 0 0 0

4

0

5

10

15

20

25

30

35

<-1

00

-10

0>-

10

0-9

0-8

0-7

0-6

0-5

0-4

0-3

0-2

0-1

0 01

02

03

04

05

06

07

08

09

0<1

00

10

0>1

00

Shar

e, %

Level of deviation plan vs fact, %

Key findings

Distribution last figure

9% of data could be not

reliable/ manipulated

-5

0

5

10

15

20

25

30

35

40

1 2 3 4 5 6 7 8 9

Shar

e, %

Figures

2016 KPI fact

Normal

Deviation

Key findings

Bernulli distribution and bid data (two last numbers)

Non normal distribution

Shift figures:

0 - 9

0,00,51,01,52,02,53,03,54,04,55,05,56,06,57,07,58,0

0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95

Shar

e, %

Two last figures KPI distrubution

% total

Probability, %

Key findings

Government efficiency 2016

0

200

400

600

800

1000

1200

1400

Level of deviation plan vs fact, %

Expendutire Product Efficiency Quality

Key findings

KPI groups

KPI expendutire; 2947; 28%

KPI product; 3856; 36%

KPI effichiency ; 2277; 21%

KPI quality; 1650; 15%

Key findings

Ministry of interior KPI distribution, 2012-2015

4 year

8 budget programs

1001 KPI

250 KPI per year

30 KPI on goal

7 KPI per year oriented on goal

Goal3%

Process50%

Environment46%

Non indentify1%

Key findings

Ministry of interior KPI distribution, 2012-2016

Normal distribution

2012 – 2016 MoI = const

0

5

10

15

20

25

30

35

<-1

00

-10

0

>-1

00

-90

-80

-70

-60

-50

-40

-30

-20

-10 0

10

20

30

40

50

60

70

80

90

<10

0

10

0

>10

0

Shar

e, %

Deviation fact vs plan, %

2016

aver 2012-2016

Key findings

Ministry of interior KPI distribution, 2012-2016

Normal distribution

2012 – 2016 non constant

Shift to over-fulfillment

0

5

10

15

20

25

30

35

40

45

-60 -50 -40 -30 -20 -10 0 10 20 30 40 50 60

Shar

e %

Deviation plan vs fact КРІ, % (-/+ 60%)

2012

2013

2014

2015

2016

Key findings

Ministry of interior KPI distribution, 2012-2016

KPI split to:

1. money oriented

2. not oriented for money

KPI money oriented shifted to

over-fulfillment side

KPI not oriented for money

shifted to under -fulfillment

0

5

10

15

20

25

30

35

<-1

00

-10

0

>-1

00

-90

-80

-70

-60

-50

-40

-30

-20

-10 0

10

20

30

40

50

60

70

80

90

<10

0

10

0

>10

0

Shar

e, %

Level of deviation Government budget program KPI , %

KPI not money

KPI money

Key findings

Ministry of interior KPI, 2012-2016

Important notes

FEAO prepare information notes for 29 Ministries (over 80% of budget programs)

1. General information about Ministry1. strategic goals

2. Tasks

3. Funding 2016 - 2020

2. Budget program information1. strategic goals

2. Tasks

3. Funding for 5 years

4. Select main KPI and present its fulfillments

5. Present information to MP

3. Present notes to MP at the budget discussion in Parliament Budget Committee

Important notes

Questions?