1

Government Wide Monitoring Government Wide Monitoring and Evaluation Systemand Evaluation System

Ronette Engela The PresidencyConrad Barberton National TreasuryAkiiki Kahimbaara StatsSaMohanuoa Mabidilala DPLGHenk Serfontein DPSA

2

Content of presentationContent of presentationOverview of the GWM&E framework Programme Performance Information Evaluations Census and Surveys - National Statistical System Derived and Transversal Information Systems

DPLG Local Government SystemDPSA Systems

3

In 2005 Cabinet approved In 2005 Cabinet approved an an implementation planimplementation plan

to develop to develop a monitoring and evaluation a monitoring and evaluation systemsystem

for use across governmentfor use across government

To encompass To encompass Monitoring implementation, effectiveness,

validationEvaluation impact and process evaluationEarly warning proactively identify blockagesVerification validates integrity of dataData collection using existing capacitiesAnalysis research driven assessments Reporting appropriate & customised

to target groups

The centre of government

needs to provide

clear policies and frameworks

MandatePresidency

4

Composite system that draws its data from contributory systems Emerging system built up over time with consistent and sustained participation by all stakeholders

Policy and Standards

Reporting and Databases

Capacity Building

Lead Agency DPSAPresidency SAMDI

Partner Agencies OPSC

Stats SA

National Treasury

Presidency

Stats SA

DPLGNational Treasury

DPSA

Presidency

Stats SA

DPLGDPSA

Phased implementation through 3x work streams

5

Are we still on track?Are we still on track?Missed a number of deadlinesInformation needed to inform next phase of development not in place

Audit of reporting requirements and M&E systems in gov late P&P shifted to Policy and Standards

Independent development of systemsNearly all provinces

Lack of own capacity - many consultants in sectorsMisconception re ‘System: IT’ vs‘Framework’

Reporting labyrinthReporting labyrinthplethora of dataplethora of data

For example:For example:Provincial dept Provincial dept reports to:reports to:•Own dept and exec authority•National concurrent dept•Premier Office•National Treasury•Presidency

6

Entry pointsEntry points

Executive reportingEvidence based decision making for:

Resource allocationPolicy refinement

Extensive executive interestSupport government implementation focus

RefineRefine

7

GWM&E System: data terrainsGWM&E System: data terrains

Census and Survey Information

Registers and Admin data

Evaluations

Programme Performance Information

8

Census and Survey Information

Registers and Admin data

Evaluations

Programme Performance Information

GWM&E System: policy platform

Policy FrameworkPolicy Framework

9

GWM&E Framework: Programme Performance Information

PPIE

C&SDept Dept Dept Dept

Derived information system DPLGDWAF

Transversal systems PERSAL BAS

10

GWM&E System: GWM&E System: Executive reports

Census and Survey Information

Registers & Admin data

Evaluations

PPI

Dept Dept

Derived information system

Transversal systems

Dept

PoAPoA

NationalIndicators

11

10 categories of 10 categories of Development IndicatorsDevelopment Indicators

Economic growth & transformationEmploymentPoverty & inequalityHousehold & community assetsHealth

EducationSocial CohesionSafety & SecurityInternational relationsGood governance

12

Economic growth & transformation

GDP, GDP per capitaFDICFCGovernment debtBudget deficit

CPI, CPIXBonds point spreadR&DExportsBEEBlack managers

13

Employment Poverty and Inequality

EmploymentUnemploymentExpanded Public Works

Per capita incomeInequality (Gini Theil)Poverty headcountPoverty gap analysisSocial assistance grantsLife expectancyLiving Standards Measures

14

Household & Community Assets

DwellingsWatersSanitationElectricityLand restitutionLand redistribution

15

Employment Poverty and Inequality

EmploymentUnemploymentExpanded Public Works

Per capita incomeInequality (Gini Theil)Poverty headcountPoverty gap analysisSocial assistance grantsLife expectancyLiving Standards Measures

16

Health Education

HEALTHInfant and child mortality rate Severe malnutrition under five yearsImmunisation coverage Maternal mortality ratio HIV prevalenceTuberculosis (TB) Malaria

EDUCATIONEducator - learner ratioEnrolmentMatric pass rateMatriculants with mathematics higher grade passesAdult literacyGraduating science, engineering and technical (SET)

17

Social cohesion

Strength of civil societyVoter participationVoters per provinceWomen who are members of legislative bodiesConfident of a happy future for all races

Country going in the right direction Identity in the form of class Pride in being South African Public opinion on race relations

18

Safety & Security International relations

SAFETY & SECURITYBroad categories of crimes Contact crime Property crime Detection rate Charges referred to court Conviction rateInmatesRoad accidents and fatalities

INTERNATIONAL RELATIONSPeace operations Democratically elected governments in Africa GDP growth in Africa Tourism Mission operations Diplomats trained

19

Good governance

GOOD GOVERNANCETax returns Audits Corruption perceptions Budget transparencyPublic opinion on delivery of basic services Red tape, business Biodiversity

20

Standard format

IndicatorCategory linked to 10 Year ReviewGoalTrend analysisTableGraphDefinitionData source(Data note)

21

Indicators future

Publication - annuallyWill add some indicatorsMinimum requirement for all deptsReplicated in all spheres (as applicable)Develop data for

Community assetsHealthEducationSafety & Security on district level

22

GWM&E System: GWM&E System: Executive reports

Census and Survey Information

Registers & Admin data

Evaluations

PPI

Dept Dept

Derived information system

Transversal systems

Dept

PoAPoA

NationalIndicators

23

Census and Survey Information

Evaluations

PPI

Dept Dept

Derived information system

Transversal systems

Dept

Area of Responsibility

24

EvaluationEvaluationNeed for detailed policy evaluations on a number of different levels:

Departmental led reviews and evaluation of policies and programmesSectoral reviewsBroad, cross cutting reviews led by centre of government

Monitoring and evaluation have different purposes

25

Programme Performance Programme Performance InformationInformation

Kuben Naidoo National Treasury

26

Constitution S.215 S.216Budget & Exp Management

PFMA & MFMA

Ensure information on inputs, outputs and outcomes …

MandateTreasury

…underpins planning, budgeting, implementation management and accountability reporting…

…to promote transparency andexpenditure control.

27

Programme Performance Information

E

C&SDept

Derived information system

Area of Responsibility

Transversal systemsLine manager

Dept exec

Programme manager

Exec Authority

Dept

28

Framework for Managing Performance Information

Key Performance

Concepts

IMPACTS

OUTCOMES

OUTPUTS

INPUTS

ACTIVITIES

The developmental results of achieving specific outcomes

The medium-term results for specific beneficiaries that are the consequence

of achieving specific outputs

The final products, or goods and services produced for delivery

The processes or actions that use a range of inputs to produce the desired

outputs and ultimately outcomes

The resources that contribute to the production and delivery of

outputs

What we use to do the work?

What we do?

What we produce or deliver?

What we wish to achieve?

What we aim to change?

Plan, budget, implement and

monitor

Manage towards achieving these

results

IMPACTS

OUTCOMES

OUTPUTS

INPUTS

ACTIVITIES

The developmental results of achieving specific outcomes

The medium-term results for specific beneficiaries that are the consequence

of achieving specific outputs

The final products, or goods and services produced for delivery

The processes or actions that use a range of inputs to produce the desired

outputs and ultimately outcomes

The resources that contribute to the production and delivery of

outputs

What we use to do the work?

What we do?

What we produce or deliver?

What we wish to achieve?

What we aim to change?

Plan, budget, implement and

monitor

Manage towards achieving these

results

29

Framework for Managing Performance InformationPerformance

Indicators

IMPACTS

OUTCOMES

OUTPUTS

INPUTS

ACTIVITIES

RELATIONSHIP INDICATORS

OPINION-BASED INDICATORS

Types of direct indicators

QuantityQuality

Cost / Price Timeliness

Start and end timesDistributionAdequacy

Accessibility

DIRECT INDICATORS

Economy

Efficiency

Effectiveness

Data gathered through surveys

Equity

Indicators

Indicators

Indicators

Indicators

IndicatorsData gathered mainly by management information

systems

Calculated using a combination of direct indicators and other

dataIMPACTS

OUTCOMES

OUTPUTS

INPUTS

ACTIVITIES

RELATIONSHIP INDICATORS

OPINION-BASED INDICATORS

Types of direct indicators

QuantityQuality

Cost / Price Timeliness

Start and end timesDistributionAdequacy

Accessibility

DIRECT INDICATORS

Economy

Efficiency

Effectiveness

Data gathered through surveys

Equity

Indicators

Indicators

Indicators

Indicators

IndicatorsData gathered mainly by management information

systems

Calculated using a combination of direct indicators and other

data

30

Link to individual performance Link to individual performance agreements of line managers and HOD agreements of line managers and HOD

Ensure Programme Performance Information is appropriately used for planning, budgeting and managementplanning, budgeting and management purposes •set performance standards and targets priorto the start of each year•review performance and take management action•evaluate performance at the end of a service delivery period

PPI manual per sector - extensive consultation to determine needs of

different usersNext steps

Treasury (2)

31

National Statistical SystemNational Statistical System

Akiiki Kahimbaara StatsSA

32

Evaluations

PPI

Dept Dept

Derived information system

Transversal systems

Dept

Area of Responsibility

Census and Survey Information

Registers and Admin data

33

The Statistics Act (No. 6 of 1999) The Statistics Act (No. 6 of 1999) 2002 January Cabinet 2002 January Cabinet LegkotlaLegkotla

State of the Nation Addresses State of the Nation Addresses 2004 and 20052004 and 2005

•Section 14 subsection 6 clauses (a), (b) and (c) where the Statistician-General may advise an organ of state on the application of quality criteria and standards

•Section 14 subsection 7 clauses (a) and (b) grants the Statistician-General power to designate statistics produced by other organs of state as official statistics

•Section 14 subsection 8 clauses (a) and (b) authorisesthe Statistician-General to comment on the qualityof national statistics produced by another organ of state; and to publish such other department’s statistics

MandateStatsSA

34

Demand for information[user groups or

indicator categories]

Stats SAOther producers

[departments, CSOs, etc]

• Insufficient supply of quality information• Uncertain quality?Poor comparabilityIsolated producersNo shared standards• Insufficient statistical skills

Official statistics

Unknown quality

Problem

35

Demand for information[user groups or

Indicator categories]

Stats SAOther producers

[departments, CSOs, etc]

Regulatory environment• Statistics Law• Governance structure• Quality standards• Advocacy programme• Code of conduct

• Adequate information• Reliable information

[quality, sustainable]• Sufficient skills

Coordination

Feedback

Proposed solution

Feed

back

36

Proposed new approachProposed new approach

Use official statistics to ensure quality

• SASQAF [quality framework]•Framework of international best practice•UN Principles of Official Statistics

Maintain the decentralised system of statistical production

Transform all national statistics into official statistics

SASQAFDraft available

37

Transform existing departmental Transform existing departmental data (registers & surveys) data (registers & surveys) into sustainable sources into sustainable sources

of official statisticsof official statistics

•More use of administrative data than of surveys - sustainability and cost

• Agreements and collaboration between Stats SA and individual dept

• Joint Working Party between Stats SAand each of the departments to improve quality

ProgrammeNSS

38

Access registers or datasets and map Access registers or datasets and map them against indicators in the compendiumthem against indicators in the compendium

•Identify gaps – where the compendium of indicators should be coming fromthe department• Assess quality of registers or datasets for usability• Suggest improvements

Working with individual departmentsNext steps

StatsSA

39Build statistical capacity in departmentsBuild statistical capacity in departments

•Encourage departments to establishstatistical capacity – as part of existing M&E units or as components on their own•Allocate sufficient resources: part of MTEF budgeting for sustainability

•Implement statistical training programmeswith support from Stats SA

Audit departments for statistical capacityNext steps

StatsSA (2)

40

ConcernsConcerns

Inadequate appreciation of the practice of managing for results

• Measurement of performance and “objective” info for planning and decision-making not a priority

• Current practice of existing M&E units without measurement

Circumlocutionary behaviour of public officers (multiple formalities)

• Waiting for approval (all the way to the minister!): inadequate delegation of authority

• Lack of institutional memory in government departments [change the head: start afresh]

•Operational silos – protecting one’s turf•Cover-ups: fear of exposure

41

Derived system:Derived system:DPLG local government DPLG local government

Mohanuoa Mabidilala DPLG

42

GWM&E Framework: DPLG Derived Information

PPIE

C&SDept Dept Dept Dept

Derived information system

Transversal systems

43

Constitution Chapter 3 & 7 Constitution Chapter 3 & 7 Municipal Systems ActMunicipal Systems Act

Municipal StructureMunicipal Structure’’s Acts Act

Function of DPLGFunction of DPLG•develop national policies and legislation with regards to Provinces and Local Govt, and to monitor the implementation•to support Provinces and Local Govt in fulfilling their constitutional and legal mandate

MandateDPLG

44

DPLG Monitoring, ReportingDPLG Monitoring, Reporting& Evaluation& Evaluation

ResponsibilityDPLG

•Developing and implementing an integrated monitoring and evaluation system (dplg)

•Providing leadership and support to Local Govt for the successful implementation of the GWM&E framework

•The development of an MR&E capacity in the provinceand local government to monitor and reportimplementation of the 5 Year Local GovernmentStrategic Agenda (2006 – 2011)

45

ResponsibilityDPLG 2

5 Local Govt 5 Local Govt KPAsKPAs•Municipal Transformation and Organisational Development

•Basic Service Delivery•Local Economic Development•Financial Viability and Management•Public Participation and Community Involvement

DPLG will develop and implement a system for assessing local govt

service delivery

46

Collaboration with key Collaboration with key departmentsdepartments

ResponsibilityDPLG

•The Presidency•National Treasury•Dept of Public Service and Administration•Dept of Water and Forestry•Dept of Mineral and Energy•Dept of Housing•Statistics South Africa•South African Institute for ManagementDevelopment (MR&E capacity building)

47

Varying degrees: Varying degrees: ofof understanding understanding and and

ofof M&E capacity and readinessM&E capacity and readiness

•Lack of integrated approach to local governmentM&E across 3 spheres

•Lack of appropriate M&E reporting structures tomonitor local government service delivery

•Capacity and capability to monitor local govtservice delivery

DPLG assessed readiness of provincial depts of local govt and Office of Premiers to do

Local govt M&EProblem

48

Standardise approach to Monitoring, Standardise approach to Monitoring, Reporting and EvaluatingReporting and Evaluating

Develop and implement MR&E policy Capacity building strategyInstitutionalise MR&E systems across the three

spheres of govt by using existing structures withinIGR Act as M&E forums

DPLG did assessment of business processes that inform MR&E

system Proposed Strategy

49

Types of local government Indicators

43%

36%

21%

9%

2%

20%

0%

45%

27%

8%

0%5%

10%15%20%25%30%35%40%45%50%

Input Activities Output Outcome Impact

Indicator category

Perc

enta

ge

Initially 500

Collated allCollated allindicators indicators

on local on local govtgovtPhase 1Phase 1

Now ±150

50

Shared understanding of Local government indicator Shared understanding of Local government indicator development across all three spheres Phase 2development across all three spheres Phase 2

GWM&E System

National

Provincial

District

Local

GPOA

PGDS

IDP

IDPIndicators

51

Achievements to dateAchievements to date Done

Readiness assessment of provincial dept of local governmentDraft Local Government MR&E framework, aligned to GWM&EMR&E business process mapping of Local government indicatorsNational Local Government MR&E forum establishedCollaboration with SAMDI and NT TAU for capacity building

52

Next stepsNext stepsPlans

Finalise Local Government M&E policy frameworkDevelop a compendium of indicators for Local GovernmentDesign and implement Local govt MR&E system that extracts information with multiple sector departments that deal with Local Government –based on impact model Capacity building for provinces and municipalities

53

Derived system:Derived system:DPSADPSA

Henk Serfontein DPSA

54

Public Service ActPublic Service Act•Responsible for public service transformation •Custodian of public management frameworks •Performance and knowledge management •Service delivery improvement

CoCo--Chair of the Governance Chair of the Governance and Administration Clusterand Administration Cluster

CoCo--Chair the GWM&E Task Chair the GWM&E Task Team Team

Aim Increase public

service effectiveness and improve governance

MandateDPSA

55

GWM&E System: Executive reportsGWM&E System: Executive reports

Census and Survey Information

Evaluations

PPI

Dept Dept

Derived information system

Transversal systems

Dept

PoAPoA

NationalIndicators

Area of Responsibility

PM WatchPM WatchCabinet Lekgotla Cabinet Lekgotla ReportReport

56

Lead agency on databases Lead agency on databases and Reporting Work streamand Reporting Work stream

ResponsibilityDPSA

Two major initiatives currently in developmentTwo major initiatives currently in development

Public Management Watch Public Management Watch Extracts HR and Budget expenditure data Identifies vulnerable departments

HR HR utilisationutilisation Report Report Derived from departmental annual reports Focuses on how well HR is used

57

• Will draw from various GWM&E sources •To provide useful, practical updates•Executive level information •High strategic overview.

New development

Creation of Creation of Year end Report Year end Report bibi-- annual Cabinet annual Cabinet maKgotlamaKgotla

58

Linking the performance management and measurement systems needs dedicated attention

•DPSA M&E capacity overstretched (APRM etc.) •Building public participation systems a key issue •Must enhance GWM&E stakeholder relations •Overcoming formalistic compliance by sectors and provinces.

Task: Improving public management outside financial and

expenditure issues Challenges

DPSA

59

•• Priority streamlining reporting burden Priority streamlining reporting burden •• Compilation of reporting requirement Master list Compilation of reporting requirement Master list •• Undertake consultations with systems usersUndertake consultations with systems users•• Focus on improving understanding of their needs Focus on improving understanding of their needs •• Develop database architectureDevelop database architecture

Priority: Work stream 2 objectives

Data bases and reporting

Next stepsDPSA

60

ConcernsConcernsNeed to incorporate Public Opinions/ Imbizo processesPublic Entities and Constitutional Bodies

Conceptual clarity - IGRKey role players involvedConvergence around indicators

Data quality will improve with public attention and utilisation

Steady does it

Myth: Myth: ““at the press of a at the press of a buttonbutton””Systems to alert about Systems to alert about problem, need political problem, need political will to deal with itwill to deal with it

Conclusion

61

KeKe yaya lebogalebogaKeKe a a lebohaleboha

KeKe a a lebogalebogaNgiyabongaNgiyabonga

NdiyabulelaNdiyabulelaNgiyathokozaNgiyathokoza

NgiyabongaNgiyabongaInkomuInkomu

NdiNdi khoukhou livhuhalivhuhaDankieDankie

Thank youThank you