Document of The World Bankdocuments.worldbank.org/curated/en/...PUBLIC-disclosed-11-6-15.pdf · Document of The World Bank ... IPSAS International Public Sector Accounting Standards

Document of The World Bank

Report No: ICR00003373

IMPLEMENTATION COMPLETION AND RESULTS REPORT (IDA-41090 IDA-48640)

ON A

CREDIT IN THE AMOUNT OF SDR 72.20 MILLION

(US$104.64 MILLION EQUIVALENT)

TO THE

ISLAMIC REPUBLIC OF PAKISTAN

FOR THE

SECOND IMPROVEMENT TO FINANCIAL REPORTING AND AUDITING PROJECT

June 15, 2015

GGODR Governance Global Practice South Asia

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

i

CURRENCY EQUIVALENTS

(Exchange Rate Effective May 2015)

Currency Unit = Pakistani Rupees (PKR) PKR101.94 = US$l

FISCAL YEAR July 1 - June 30

Abbreviations and Acronyms

AATI Audit and Accounts Training Institute ACCA Association of Chartered Certified Accountants ACDA Audit-command-language Certified Data Analyst ADB Asian Development Bank AG Accountant General AGP Auditor General of Pakistan AGPR Accountant General Pakistan Revenue AMIS Audit Management Information System BER Budget Execution Report CAATS Computer-assisted audit techniques CFAO Chief Finance and Accounts Officer CGA Controller General of Accounts CISA Centre for Information System Audit DAC Departmental Accounts Committee DAGP Department of the Auditor General of Pakistan DAO District Accounts Office DCS Direct Credit Scheme DDO Drawing and Disbursement Officer DfID Department for International Development (UK) DG Director General EAD Economic Affairs Department EC European Commission EMP Environmental management plan FAM Financial Audit Manual FABS Financial Accounting and Budgeting System FBR Federal Board of Revenue FD Finance Department GDP Gross Domestic Product GGODR Governance Global Practice GFS Government Financial Statistics GoP Government of Pakistan ICR Implementation Completion and Results Report ISR Implementation Support and Results Report IDA International Development Association IFMIS Integrated Financial Management Information System IMF International Monetary Fund INTOSAI International Organization of Supreme Audit Institutions IPSAS International Public Sector Accounting Standards IT Information technology

ii

ITP Intensive Training Program KP Khyber Pakhtunkhwa LGO Local Government Ordinance M&E Monitoring and evaluation MBA Masters of Business Administration MoF Ministry of Finance MTBF Medium-term budget framework MTFF Medium-term fiscal framework NAM New Accounting Model NBP National Bank of Pakistan OAG Office of the Auditor General OAGP Office of the Auditor General of Pakistan OBI Open Budget Index OCGA Office of the Controller General of Accounts PAAS Pakistan Audit and Accounts Service PAC Public Accounts Committee PAO Principal Accounting Officer PDO Project development objective PEFA Public Expenditure and Financial Accountability PER Public Expenditure Review PFM Public financial management PFO Pakistan Fiscal Operation Report PI Performance Indicator PIFRA Project to Improve Financial Reporting and Auditing PPRA Public Procurement Regulatory Authority PRSP Poverty Reduction Strategy Paper PSDP Public Sector Development Program QAG Quality Assurance Group SAP CC SAP Competency Center SBP State Bank of Pakistan SARFM South Asia Regional Financial Management TMAs Tehsil/Town Municipal Administrations UPFO Updated Pakistan Fiscal Operations

Vice President: Annette Dixon Country Director: Rachid Benmessaoud Senior Global Practice Director: Mario Marcel Cullell Practice Director: Samia Msadek Practice Manager/Sector Manager: Fily Sissoko Project Team Leader: Paul Welton ICR Team Leader: Furqan Ahmad Saleem

iii

Contents Abbreviations and Acronyms ...................................................................................................................................i Contents .................................................................................................................................................................. iii Data Sheet ................................................................................................................................................................iv ICR Abstract .......................................................................................................................................................... xv 1. Project Context, Development Objectives, and Design ..................................................................................... 1

1.1 Economic and Financial Reporting Context at Appraisal .............................................................................. 2 1.2 Project Development Objectives, Key Indicators (as approved), and Beneficiaries ...................................... 2 1.3 Original and Revised Components (as approved) .......................................................................................... 3 1.5 Other significant changes ............................................................................................................................... 5

2. Key Factors Affecting Implementation and Outcomes ..................................................................................... 5 2.1 Project Preparation, Design, and Quality at Entry .......................................................................................... 6 2.2 Implementation .............................................................................................................................................. 7 2.3 M&E Design, Implementation, and Utilization............................................................................................ 10 2.4 Environmental Safeguard and Fiduciary Compliance .................................................................................. 10 2.5 Post-Completion Operation and Next Phase ................................................................................................ 11

3. Assessment of Outcomes .................................................................................................................................... 12 3.1 Relevance of Objectives, Design, and Implementation ................................................................................ 13 3.2 Achievement of Project Development Objectives ‐ Efficacy ....................................................................... 13 3.3 Efficiency ..................................................................................................................................................... 14 3.4 Justification of Overall Outcome Rating ...................................................................................................... 15 3.5 Overarching Themes, Other Outcomes, and Impacts ................................................................................... 16 3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops ............................................. 16 Not applicable .................................................................................................................................................... 16

4. Assessment of Risk to Development Outcome ................................................................................................. 16 5. Assessment of World Bank and Borrower Performance ................................................................................ 16

5.1 Bank performance in ensuring quality at entry ............................................................................................ 16 5.2 Bank performance in quality of supervision ................................................................................................ 17 5.3 Justification of Rating for Overall Bank Performance ................................................................................. 18 5.4 Government Performance ............................................................................................................................ 18 5.5 Implementing Agency or Agencies Performance ......................................................................................... 18 5.6 Justification of rating for overall borrower performance ............................................................................ 19

6. Lessons Learned ................................................................................................................................................. 19 6.1 Design ......................................................................................................................................................... 19 6.2 Implementation ........................................................................................................................................... 20

7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners ............................................... 20 7.1 Borrower/implementing agencies ................................................................................................................. 20 7.2 Co-financiers, other partners and stakeholders............................................................................................. 20

Annex 1. Project Costs and Financing ................................................................................................................. 21 Annex 2. Outputs by Component ......................................................................................................................... 22 Annex 3. Economic and Financial Analysis ......................................................................................................... 25 Annex 4. Bank Lending and Implementation Support/Supervision Processes ................................................. 26 Annex 5. Beneficiary Survey Results .................................................................................................................... 27 Annex 6. Stakeholder Workshop Report and Results ........................................................................................ 28 Annex 7. Summary of Borrower's ICR and/or Comments on ICR ................................................................... 29 Annex 8. Comments of Co-financiers and Other Partners/Stakeholders ......................................................... 45 Annex 9. List of Supporting Documents .............................................................................................................. 46 Annex 10. Assessment of Achievements by Objective ......................................................................................... 47 Annex 11. Case Studies on Successful PFM Changes under PIFRA II ............................................................. 54 Annex 12. PIFRA II Contribution to Poverty Reducing Outcomes .................................................................. 57 Map ......................................................................................................................................................................... 59

iv

Data Sheet

A. Basic Information

Country: Pakistan Project Name: Second Improvement to Financial Reporting and Auditing Project

Project ID: P076872 L/C/TF Number(s): IDA-41090,IDA-48640 ICR Date: 12/18/2014 ICR Type: Core ICR

Lending Instrument: SIL Borrower: ISLAMIC REPUBLIC OF PAKISTAN

Original Total Commitment:

SDR 72.20M Amount Cancelled: SDR 7.10M

Revised Amount: SDR 65.10M Disbursed Amount: SDR 64.02M Environmental Category: B Implementing Agencies: Auditor General of Pakistan Co-financiers and Other External Partners: None B. Key Dates

Process Date Process Original Date Revised / Actual Date(s)

Concept Review: 07/10/2002 Effectiveness: 11/08/2005 11/08/2005

Appraisal: 12/20/2004 Restructuring(s): 1/2011; 5/ 2013; 12/2014

Approval: 09/06/2005 Mid-term Review: 01/31/2008 02/29/2008 Closing: 12/31/2010 12/31/2014 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Satisfactory Risk to Development Outcome: Moderate Bank Performance: Satisfactory Borrower Performance: Satisfactory

C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Quality at Entry: Satisfactory Government: Satisfactory

Quality of Supervision: Satisfactory Implementing Agency/Agencies:

Satisfactory

Overall Bank Performance:

Satisfactory Overall Borrower Performance:

Satisfactory

C.3 Quality at Entry and Implementation Performance IndicatorsImplementation Performance

Indicators QAG Assessments (if any)

Rating

Potential Problem Project at Yes Quality at Entry: Satisfactory

v

any time (Yes/No): Problem Project at any time (Yes/No):

No Quality of Supervision: Satisfactory

DO rating before Closing/Inactive status:

Satisfactory

D. Sector and Theme Codes Original Actual Sector Code (as % of total Bank financing) Banking 2 2 Central government administration 45 45 Sub-national government administration 53 53

Theme Code (as % of total Bank financing) Administrative and civil service reform 25 25 International financial standards and systems 25 25 Public expenditure, financial management and procurement 50 50

E. Bank Staff Positions At ICR At Approval Vice President: Annette Dixon Praful C. Patel Country Director: Rachid Benmessaoud John W. Wall Senior Global Practice Director: Mario Marcel Cullell Practice Director Samia Msadek

Practice Manager/Sector Manager: Fily Sissoko Robert J. Saum

Project Team Leader: Paul Welton Anthony Graeme/ Ismaila B. Ceesay

ICR Team Leader: Furqan Ahmad Saleem ICR Primary Authors: Sati Achath; Qurat ul Ain Hadi F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document)

PIFRA II development objectives were to (a) build capacity to improve accuracy, comprehensiveness, reliability, and timeliness of financial and fiscal reporting at all levels of government; (b) improve public financial management, accountability, and transparency; (c) enhance the capacity of public sector managers to use credible financial information for better and informed decision-making; and (d) facilitate oversight of the use of public monies, and increase the national and international credibility of government’s financial statements and assurance processes. Revised Project Development Objectives (as approved by original approving authority) The project development objectives (PDOs) were not revised. However, there were improvements made in the results framework, refining the outcomes which were better aligned to the PDOs, including new milestones for closer monitoring of implementation progress. Additional financing

vi

continued with these refined indicators and with enhanced targets as compared to those originally approved justifying the additional investment of time and resources for systems upgrade and improvements. (a) PDO Indicators

Indicator1 Baseline value Original target values

(from approval documents 2005)

Formally revised target values

(MTR/Additional Financing 2010)

Actual value achieved at completion or target years

Indicator 1: Effective and transparent fiscal administration (MTBF in place and FABS supported)

Value quantitative (or qualitative)

Pilot Medium-Term Budget Framework (MTBF) implementation underway for 5 federal ministries

Federal budgets prepared under MTBF principles

Financial Accounting and Budgeting System (FABS) supports MTBF (Addl. Fin)

- Federal budgets are prepared under MTBF principles - FABS supports MTBF

Date achieved June 2005 2011 2014 December 2014

Comments (including % achievement)

Achieved. The MTBF configuration in SAP has been completed. This helps medium term forecasts (forward estimates) to be prepared on the basis of actual expenditures data generated on real time basis through Integrated Financial Management Information System (IFMIS). The Medium-term Fiscal Framework (MTFF) is presented to the cabinet for decision-making on the resource allocation and even policy making. The MTBF process commences with the preparation of Medium Term Fiscal Framework (MTFF). This is prepared on the basis of actual expenditure data generated from FABS. On the basis of MTFF fiscal space for new polices is prepared and budget strategy is presented to the Cabinet for approval of ceilings for individual ministries. These ceilings are communicated to the line ministries through a letter from MoF. Line ministries prepare medium term budget and link it with outputs and targets. The system is configured to enable departments prepare MTBF. This is once approved by the cabinet, disseminated to public and uploaded on MoF website. This document is commonly called a Green Book. Quarterly budget execution reports are prepared. Financial information is generated through the system, whereas non -financial information on achievement of targets against allocated budget is collected manually.

Indicator 2: Effective and transparent financial reporting, control, and audit (timely, reliable financial statements submission and timely audit)


Accounts submitted to Office of the Auditor General (OAG) 3-4 months after due date. OAG accounts 15 months or more after year-end. Quality poor.

Audited accounts presented to the legislature within 12 months of the fiscal year end.

OAG certification and audit report within 6 months

Certification and audit is completed within 6 months of the fiscal year-end and two additional months are spent for rigorous quality assurance process and submission to the legislature.



Achieved. Audit of 153 financial statements /appropriation accounts are prepared within 6 months for the last consecutive 5 years. These reports undergo detailed quality reviews in accordance with Quality Management Framework in two months. The target is partly achieved due to timely submission of unaudited financial statements to OAG due to FABS and improved business processes, and partly due to risk-based audit approach which is now adopted by OAG for the purpose of financial audit. This is a major achievement to prevent misuse of resources through timely and effective audit as a deterrence and also to expedite financial accountability on all issues identified.

Indicator 3: Effectively tracked and managed poverty reducing spending (timely reliable PRSP reports)


All Poverty Reduction Strategy Paper (PRSP) reports prepared manually with no data assurance

-Reliable Monthly Budget Execution Reports available within four weeks of the end of each month -Accurate, timely and comprehensive financial statements used for

System generated PRSP reports; PETS established

- Reliable monthly budget execution/PRSP reports are generated from the system; PETS established; Accurate, timely and comprehensive financial statements are used for decision-making by PAOs and MoF

1 For ease of comparison, the table uses indicators as refined during implementation.

vii






decision-making by Principal Accounting Officers (PAOs) in line departments and MoF



Achieved. FABS was simultaneously implemented at all three levels of government with a consistently applied chart of accounts and forms and procedures in all parts of the country to capture pro-poor expenditures for education, health, water and sanitation etc. FABS is capable to generate reliable reports that track actual expenditure from federal to provinces and to districts and real-time reports facilitate the compilation of a consolidated PRSP report on a quarterly basis. PRSP reports are publicly available on MoF website. Staff at PRSP Secretariat have been trained and are able to utilize the system to generate reports. Despite radical changes due to local government reforms in 2001 to post 18th amendment reforms since 2010, the system continued to be effective in tracking pro-poor expenditures incurred at grass root level. Furthermore, FABS data also facilitates Public Expenditure Review (PER) and various studies/analysis including PETS and Expenditure and Quality of Service Delivery Survey to help in identifying sources of inefficiencies and bottlenecks in the flow of funds and to recommend remedial actions. Continuous efforts are underway to make system even better and more user friendly for reporting. BERs are used by executives to make expenditure decisions on the basis available budget. Annual financial statements are also used by PAOs and MOF, for example, as a historical base for budget preparation etc. The comparative data by districts, entity or other such classification could be a huge support to a performance management framework, but such analysis and use of data can only be enhanced by progressively providing further support to the executives and policy makers.

Indicator 4: Effective corruption and inefficiency reducing measures related to accounts and budget control applied (system monitoring of use of functionality, bills, and issuance of checks etc.)


Manual issuance of checks and no systematic review of workflow practices

-Proper recording of accounting entries according to New Accounting Model (NAM) to enable federal unidentified expenditures reduced to no more than 0.5 percent of total actual expenditures - Full real-time quarterly reconciliation of expenditures and revenues at federal and provincial levels

System workflow reporting fully under the Office of the Controller General of Accounts (OCGA) and regular OAG audit.

System reporting fully under OCGA and regular OAG audit. Unidentified expenditures reduced to no more than 0.5 percent and quarterly reconciliations are near real-time.



Achieved. Issuance of manual checks has been abandoned through regular system reporting and support available to accounts offices. This has significantly improved the fiscal discipline over the payments. This is appropriately indicated as achieved. In addition, the use of Computer Assisted Audit Techniques (CAATS) has enabled auditors to perform audit of transaction data in an efficient manner, which has led to enhanced accountability on budget execution.

Indicator 5: Establish stable FABS operations and maintenance support in Government of Pakistan (GoP)


- FABS system tested in Abbottabad site and replication started

- Establish stable FABS operations and maintenance support in GoP (implement a data warehouse, and an information security and disaster recovery plan). - Acceptance of, and direct management and control of, handed over PIFRA accounting sites – (intermediate results indicator target)

All FABS sites on full work flow (e.g., Khyber Pakhtunkhwa (KP) and Balochistan); FABS support unit established under OCGA for ongoing maintenance and support.

FABS has been fully established and is operational. More than 500 offices and 6,200 users are using FABS site implementation complete where law and order situation has allowed.


viii







Achieved. FABS site implementation complete where law and order situation has allowed. FABS is fully transitioned and support is provided by OCGA. SAP upgrade to ECC06 has been complete together with a server upgrade, and information security procedures are in place and implemented. Civil work on disaster recovery site has been completed, which is yet to be operationalized. Finally, 64 mainstream technical positions have been authorized and budgeted by the Finance Ministry and against these positions 47 persons are performing their duties to ensure stable FABS operations and maintenance support.

Indicator 6: Adequate numbers of trained staff in MoF and Controller General of Accounts (CGA) to manage and apply system to fiscal and financial management (trained MoF, CGA, and line ministry Chief Finance and Accounts Officers (CFAOs))


Very few staff trained in systems operation or new work practices.

Professional training: 20 senior, including operation, staff, 20 CFAOs

Training of 1,045 senior officers and 58 CFAOs.

Date achieved June 2005 2013 December 2014


Achieved (Target exceeded). Training programs continuously evolved in the light of feedback and assessment provided by the trainees and trainee organizations. Over 7,500 staff, representing the OCGA, OAGP, ministries, divisions, and departments at the provincial and federal level, were trained on usage of system. It included senior officers of ministries, divisions, department (i.e., finance managers as termed in the Planning Commission 1) CFAOs; other officers of executive departments; and AGs/District Accounts Offices (DAOs). In addition, tens of thousands of Drawing and Disbursing Officers (DDOs) were trained on the new chart of accounts to enable appropriate classification of expenditures as a building block of the financial data.

(b) Intermediate Outcome Indicators

Indicator Baseline value Original target values (from approval documents 2005)

Formally revised target values (MTR/Additional

Financing 2010)

Actual value achieved at completion or target

years

Indicator 1: Effective reconciliation of accounts for fiscal reporting and planning 1.1 Discrepancies between monetary survey credit and government fiscal deficit (for 2004/05 equals 1.2% of Gross Domestic Product (GDP))


Estimated statistical discrepancy 2004/5 -1.2% GDP

Proper recording of accounting entries according to NAM to enable federal unidentified expenditures reduced to no more than 0.5 percent of total actual expenditures

Full reconciliation 0.5% of GDP in July- Sept 2014

Date achieved June 2005 2012 2013 December 2014 Comments (including % achievement)

Achieved. The statistical discrepancies previously termed as, unidentified expenditure are defined as a discrepancy between overall fiscal deficit estimates from the revenue and expenditure accounts of government and the State Bank of Pakistan’s estimate of deficit financing. Statistical discrepancy is presented in the Pakistan Fiscal Operations (PFO) tables on the Ministry of Finance (MoF) website. A large value for this entry signifies the weakness in the accounting system and a need for action to address and rectify the problem. Various reforms under PIFRA have significantly contributed to reducing statistical discrepancies. These reforms include, eliminating redundant processes, timely recording of expenditure in appropriate accounts and effective reconciliation of accounts. As a result of these reforms, Statistical Discrepancy has significantly reduced to 0.5% of GDP during first quarter of fiscal year 2014-15 as compared to -1.2% of GDP at commencement of PIFRA II in May 2005. Data to calculate this indicator is collected and collated by the Ministry of Finance (MoF). Quarter-wise variations occur because the Economic Affairs Division (EAD) and self-accounting entities do not report their expenditures on time. However, full reconciliation is not technically possible due to delay in recording of expenditure from the entities which are still not captured in FABS on real time basis, such as departmentalized accounts (WAPDA, NHA) and reconciliation delays with EAD.

1.2 Issue of reliable, timely BERs to PAOs Value quantitative (or qualitative)

No BERS—civil accounts only

100% and full commitment records

BERs are available at every level of government through the system.

ix



Financing 2010)


years Commitment accounting functionality exists. MoF issued notification for 4 ministries to start compliance.

Date achieved June 2005 2013 December 2014 Comments (including % achievement)

Partially achieved. SAP link has already been given to departments, and they have access in the system to generate BERs when required by PAOs. AGs also send hard copies to them for reconciliation. System utilization for issuance of BERs has increased more and more as PAOs become aware of and are trained in using SAP. Commitment functionality has been fully developed by PIFRA through implementation of FABS. However, budgetary commitments are recorded with a lag at the same time when bill is recorded. MOF has issued notification for four ministries which will push ministries and departments to record commitments on a timely basis and will link the release of funds with in-time recording of commitments.

Indicator 2: Accurate, comprehensive, and timely reports, based on international standards of accounting and internal control.

2.1 Application of International Public Sector Accounting Standards (IPSAS) Value quantitative (or qualitative)

IPSAS No Change Financial statements did not comply with IPSAS.

District, provincial and federal governments are using NAM and common account classifications that can be readily aggregated and disaggregated by automated systems

Progress on implementing other aspects of accruals —part of long-term roadmap

-The budget formulation and execution is based on administrative, economic and sub-functional classification, using Government Financial Statistics (GFS)/COFOG standards Public Expenditure and Financial Accountability (PEFA) Performance Indicator (PI) -5 ‘A’ rating) -Financial statements at federal and provincial level now largely comply with IPSAS cash-basis systems.


Largely achieved. Financial Statements are being prepared under cash basis of accounting and are compliant with the format given by the cash basis IPSAS. However it lacks certain disclosures, such as Third Party Payment Disclosure, commitment accounting, undrawn borrowing facilities and undrawn external assistance disclosures. The system has been recently developed (October 2014) to capture Third party disclosures, thus automatically tagging the 3rd party payments at data capturing stage and disclosing it in Monthly Civil Accounts and Annual Financial Statements as separate item together with other missing disclosures. In addition, CGA has approved the accrual road map and have plans to share it with MoF and other stakeholders.

2.2 Implementation of commitment accounting Value quantitative (or qualitative)

No commitments recording Implementation of commitment accounts for all major projects, including those paid through assignment accounts.

Commitment functionality has been fully developed by PIFRA through implementation of FABS. However, commitments are recorded with considerable time lag therefore benefits are not fully reaped from the available functionality in the system.

Date achieved June 2005 2013 December 2014 Comments (incl. % achievement)

Partially achieved. MoF has issued notification for four ministries that will push ministries and departments to record commitments on a timely basis and link the release of funds with in-time recording of commitments.

x



Financing 2010)


years

Indicator 3: System functionality tracking for improved efficiency and reduced discretion 3.1 Bills passed within 3 days (%) Value quantitative or (Qualitative)

No bill tracking 80% Aggregate figure for August 2014 stands at 48% passed within 3 days.

Date achieved June 2005 2013 December 2014


Partially achieved. Province-wise breakdown: Islamabad, 46%; Punjab, 41%; KP, 51%; Sindh, 39%; and Balochistan, 83%. Delays owe to the fact that bills are submitted manually and that often documentation received from line departments and ministries is incomplete. Sometimes, delays occur in the accounts offices due to unknown reasons. Therefore, to build pressure of the citizens as well as the officers, a SMS-based query and Dashboard for Bill Tracking reports are also developed. It is also noted that in areas where there is strong supervision by Deputy Accountant General (AG), AG, and Additional AG, bills are being processed in a timely manner (e.g., Balochistan).

3.2 All checks issued by system/direct vendor payment


Manual issuance of checks

Direct payments to 70% of vendors

All checks are issued through system and suppliers’ checks are made payable direct to the vendors’ bank account.


Achieved (target exceeded)

Indicator 4: Achieve comprehensive data entry to FABS 4.1 Comprehensive data entry coverage (% expenditure and receipts) Value quantitative (or qualitative)

No coverage of projects, self-accounting entities, foreign-financed transactions, and MoF transactions incomplete or non-timely

100% Data is collected by the AG on a regular basis. However, time lags may occur due to delay in ex post transactions from departmentalized accounts and self-accounting entities.


Partially achieved. System is capable of capturing all transactions, but at times information received beyond the deadline of 10th of every month from self-accounting entities causing delays in the preparation of the whole of government accounts.

4.2 Interfaces with SBP/NBP, FBR and EAD established


No interfaces with key revenue and financing systems

All interfaces in place.

Interfaces have been developed for the State Bank of Pakistan (SBP)/ National Bank of Pakistan (NBP), Federal Board of Revenue (FBR) and the Planning Commission (PC)

Date achieved June 2005 2013 2014


Partially achieved. Interface with the PC is complete and data on Public Sector Development Program (PSDP) has been populated. Interfaces with EAD and Federal Board of Revenue (FBR) though completed but awaited activation until the end of the project life. The information is generated in EAD’s and FBR’s respective systems and is then entered in to the FABS on a monthly basis. This slightly delays compilation of the monthly accounts. Nevertheless, this delay does not have a profound effect on the timeliness of the monthly, quarterly, or annual accounts.

Indicator 5: Effective and timely audit of government accounts using international standards of auditing 5.1 Application of modern audit practices

xi



Financing 2010)


years Value quantitative (or qualitative)

Manual transaction and sample-based audit. No staff trained in Financial Audit Manual (FAM) or CAATS. Few with professional degrees

Relevant Auditor General’s staff trained in certification, financial audits, and Audit Management Information System (AMIS)

Second cycle: Quality control framework effectively operational and used for the FY10-11 audit. Information system audit on-job- training for FAO staff. Quality management facility, information system audit, and AMIS incorporated into regularity audit practices.

Quality control framework operational since 2010-11, and a large number of auditors have been trained.


Achieved. Quarterly control framework operational since 2010-11. Information system audit established and intense training carried-out. Updated status of training: Over 40 performance audit specialists trained (including master trainers). 4,946 auditors, FAM training 2,559 auditors, training in CAATS 197 auditors, Local training for degrees/certification courses for OAGP 907 auditors, training in performance/information system, public-private partnership, forensic auditing, etc. The application of AMIS is pending (see indicator 5.3 below). Participants were tested for the certification exams such as, PIPFA, and CISA etc. PIPFA program started in 2005 under PIFRA. In order to align the training with the reform, training of Subordinate Account Service (SAS) was replaced with Pakistan Institute of Public Finance Accountants (PIPFA) qualification. An impact assessment was conducted in 2013 showing the average success rate of 72%. The curriculum was also revised in 2013 based on an impact assessment the training. In addition, a number of performance indicators were used to ensure that training/education bearing results. For instance, Bill Tracking System is also used to monitor the performance of officers identifying the delay in bill processing by officer. DDO wise Budget execution report serves an effective monitoring and performance measurement tool by the Managers. Further, historical data in SAP provides basis for MTBF, thus significantly contribute in budgetary decision making such as sectoral allocation and etc.

5.2 Performance audit of policy-linked programs (such as poverty reduction)


Few policy-oriented audits.Relevant Auditor General’s staff trained in performance audits

-Development of Performance Audit Manual. Revision and development of guidelines. -Training of 30 Pakistan Audit and Accounts Service (PAAS) officers on performance audit and supervision of 3 audits by the consultant.

Trained 30 PAAS officers who have conducted three performance audits to ensure on-the-job training.



Achieved (target exceeded). Performance audits were conducted on the following programs: (a) Solid Waste Management Program, (b) Benazir Income Support Program, and (c) Procurement of Wheat by Government of Sindh. Performance audits are now a part of annual audit plan. During last three years, 19 performance audit reports/special studies have been presented to the legislature. The Performance Audit Wing of the DAGP is being strengthened with additional resources. Finally, the Intensive Training Program (ITP) courses are restructured in line with International Organization of Supreme Audit Institutions (INTOSAI) guidelines.

5.3 Implementation of AMIS Value quantitative (or qualitative)

No formal AMIS Acquisition of software, preliminary pilot and roll-out to key sites

AMIS acquisition could not be completed.


Not achieved. While AMIS can further improve the audit efficiency, PIFRA II achieved the overall objective of transparent and credible financial statements through FABS, risk-based financial audit and CAATs. Many attempts were made to procure AMIS during the project but failed for multiple reasons. Primarily there was lack of

xii



Financing 2010)


years understanding on difference between the modified versus off-the-shelf software, which eventually led to over-specified tender documents restraining the competition. Also, there were internal deliberations, bureaucratic and procedural delays that hindered the procurement of the AMIS in PIFRA II. Two attempts to procure AMIS were made during the Additional Financing period, which failed due to inadequate market response spurred by repetitive bidding. On the last occasion, a single bid was received for an off-the-shelf AMIS software bid invitation. Bid evaluation report shared with the Technical Committee on Audit as well as with the World Bank Team. The bid was found to be non-responsive.

5.4 Training of PAAS staff to replace consultants Value quantitative (or qualitative)

During transaction roll-out, a heavy reliance on consultants.

Acceptance of, and direct management and control of, handed over PIFRA accounting sites

Staff to replace consultants. Only audit experts and few supervisors to be retained until project end. Additional 25 PAAS officers trained on Financial Audi Manual, performance and information system audit.

- During transaction roll-out, a heavy reliance on consultants. - PAAS officers trained onFAM, Performance and information system audit. - Phased reduction of consultants; only audit experts and few supervisors have been retained - Field Audit Offices have carried out two IS Audits of public sector organizations, the reports are submitted to the legislature.



Achieved (target exceeded): More than 260 officers have been trained from OAGP and OCGA: (a) 40 foreign degree-level trained; (b) 78 IFCE certified; (c) 20 MBA, CISA, ACCA, ACDA; (d) 46 SAP Certified/Academy; (e) 66 training in special sectors (performance auditing, forensic auditing, environmental auditing, information system auditing, energy sector auditing etc.); (f) 6 ACL master trainers; and (g) 8 FAM master trainers. The DAGP has adopted INTOSAI guidelines, developed a handbook, and has revised the training approach and modules to replace an outdated curriculum. CISA has developed training modules specific to professional needs of senior audit management and field auditors.

Indicator 6: Management skills transfer 6.1 Transfer of Monitoring and Evaluation (M&E) and change management skills to OCGA and OAGP Value quantitative (or qualitative)

Limited skills transfer in transactions roll-out phase of PIFRA II

Communication Management Plan progress report

-Joint work with MoF, OCGA, OAGP, and PIFRA team -Full takeover of M&E and change management functions by GoP

Regular progress reports on communication were produced. Mainstreaming of M&E and change management functions completed.



Achieved. Management guidelines on each process/activity of system published for all officers/staff levels in OCGA and OAGP. M&E, FABS and Change Management Wings share M&E and change management techniques and methodologies with OCGA and OAGP from time to time and are always available for any further assistance when required by stakeholders. M&E team also visited AG offices and DAOs for knowledge transfer.

G. Ratings of Project Performance in ISRs

No. Date ISR archived

Development objective Implementation progress Actual disbursements

(US$ millions) 1 02/28/2006 Satisfactory Satisfactory 9.98 2 09/12/2006 Satisfactory Satisfactory 14.01 3 02/01/2007 Satisfactory Satisfactory 19.58 4 08/30/2007 Satisfactory Satisfactory 19.58 5 03/11/2008 Satisfactory Satisfactory 28.47

xiii

6 09/09/2008 Satisfactory Satisfactory 36.57 7 02/15/2009 Satisfactory Satisfactory 36.86 8 07/13/2009 Satisfactory Satisfactory 44.97 9 06/04/2010 Satisfactory Satisfactory 61.37 10 04/16/2011 Satisfactory Moderately Satisfactory 66.92 11 03/11/2012 Satisfactory Moderately Satisfactory 75.43 12 11/30/2012 Satisfactory Moderately Satisfactory 80.59 13 04/13/2013 Satisfactory Moderately Satisfactory 84.48 14 11/23/2013 Satisfactory Moderately Satisfactory 92.87 15 04/20/2014 Satisfactory Moderately Satisfactory 97.06 16 11/21/2014 Satisfactory Moderately Satisfactory 97.63

H. Restructuring (if any)

A Level-1 restructuring was done in January 2011 to provide an additional credit equivalent of US$24.5 million and to extend the original closing date of the project from June 30, 2011, to June 30, 2013. The main objectives of the Additional Financing were to (a) provide essential interim support for the continued operation and maintenance of the FABS; and the consolidation of the audit function, including the full development of AMIS; (b) address some continuing issues that were impeding the full realization of fiscal and financial management outcomes from the IT-based financial management and auditing systems developed under PIFRA I & II; and (c) build capacity of the MoF, OCGA, and OAGP to apply the automated information systems to key areas of PFM reform and further integrate the processes of fiscal policy, budget preparation, and financial management. Restructuring included revising component activities as well as a refinement of the institutional implementation arrangements. In addition, the Additional Financing invoked the country financing parameters for Pakistan by allowing for 100 percent financing of project activities during the extended life of the project. There was a realignment of the project components and the institutional arrangements to better address implementation necessities as well as strengthen stakeholder coordination (see section 1.4). Establishing two separate and distinct component steering committees – one responsible for audit and the other responsible for FABS – strengthened the institutional structure of the original PIFRA II. These two committees were responsible for overseeing the project components’ implementation within their agreed policy framework, reviewing key implementation strategies, providing strategic high-level directions, and reviewing overall implementation progress.

A Level-2 restructuring was done to extend the closing date of the project from June 30, 2013, to December 31, 2014. The extension was required to provide the extra time required to carry out a SAP software upgrade for the existing FABS financial information system.

Third restructuring in December 2014 took place just before project closing; the World Bank cancelled the unutilized amount of US$10.3 million equivalent from the project and reallocated it to the Pakistan Program.

xiv

I. Disbursement Profile

xv

Islamic Republic of Pakistan Second Improvement to Financial Reporting and Auditing

Implementation Completion and Results Report (ICR)

ICR Abstract

The Second Improvement to Financial Reporting and Auditing (PIFRA II) satisfactorily achieved all of its four PDOs. The project has remarkably improved the public financial management (PFM) landscape of Pakistan by bringing basic financial discipline at all levels of government with more than 500 government entities and around 7,000 line managers benefiting from the financial accounting and budgeting system. This has facilitated processing and recording millions of government transactions in a single database and generating reconciled information on a virtually real-time basis for tracking poverty expenditures, timely audits, and informed decision making. The audit cycle has been reduced from 33 to 8 months for submission to the legislature. Training and professionalization of thousands of staff in accounting, auditing, and information technology has ensured the sustainability of these reforms. PIFRA II achievements extend even wider in the context of the political economy and capacity challenges. However, there continues to be a need for better use of system functionalities and use of information for decision-making purposes over time as a contribution to effective public resource management.

Arguably, the choices made at the design stage of PIFRA II were responding to the reform-space at the time, but there is a need for a greater focus on the clients or users of the new systems (reform-subjects). For instance, audit reforms should focus more on the role of Departmental Accounts Committees (DACs) and Public Accounts Committees (PACs). Similarly, accounting and financial reporting reforms should focus more on how Ministry/Departments of Finance, Planning institutions, Principal Accounting Officers (PAOs), Public Procurement Regulatory Authorities (PPRAs), and budget-holders will benefit from various system functionalities and use information for decision-making. While effective communication and capacity building are key elements of PFM reforms, the incentives and mechanisms for coalition building among institutions for joint ownership and sustainability of change are equally critical. Additionally, linking PFM reforms to strategic goals of effective PFM can guide the reforms in the direction of results, and at the same time, provide flexibility to prioritize the specific reform interventions to lift specific impediments to achieving these goals.

1

1. Project Context, Development Objectives, and Design

The development hypothesis underlying both the first and second Project to Improve Financial Reporting and Auditing (PIFRA I and II) was that financial control, reporting, and audit systems used by the Government of Pakistan (GoP) would make a substantial contribution to sound PFM, which in turn impacts on better governance and therefore improves the efficiency and effectiveness of public expenditure and increases the potential for economic growth and poverty reduction.

Figure 1:

PIFRA I’s key achievements were (a) the separation of accounting and auditing functions by creating the Office of the Controller General of Accounts (OCGA), through the enactment of two Ordinances; (b) a road map was developed for better public sector accounting and financial reporting systems through the adoption of improved accounting principles and standards by the GoP using the New Accounting Model (NAM); (c) design and implementation of a new budget classification system and chart of accounts (CoA) conforming to international standards; and (d) the design and implementation of a system for automation of budgeting, accounting and reporting functions of the federal and provincial governments and implementation of this system at 30 sites. The project also strengthened the capacity of the Office of Auditor General (OAG) through the introduction of enhanced standards, systems, and better trained staff. The testing of the new Financial Audit Manual included use of computer aided audit techniques (CAATS) software Audit-Command Language for acquisition of data from the SAP system in the Abbottabad District. The project also financed foreign degrees and nearly fifty officers were sent abroad to attend various short-term courses on Debt Management, Government Regulations, Macroeconomic Policy and Management, Foreign Exchange Management, Budgeting and Financial Management in the Public Sector. PIFRA II supported widespread replication and roll-out of the systems developed in PIFRA I and accommodated the different tiers of government by working with other GOP reform efforts such as introducing devolved local government, introducing a medium-term budget framework (MTBF), and assisting in deepening the changes in government financial management. These parallel efforts, which were seen as complementary, were coordinated by the Ministry of Finance (MoF) and supported by the European Commission (EC), Asian Development Bank (ADB), UK Department for International Development (DfID), and International Monetary Fund (IMF), respectively. PIFRA II Additional Financing supported the systems and applications upgrades, capacity building aimed at sustainability of operation of these systems, and enhancing the functionality such as MTBF.

2

These additional cost and time investment led to the achievement of new and enhanced results’ targets under the Additional Financing. For instance, the original target for audit completion was enhanced from 12 months to 6 months and FABS support to MTBF was introduced.

This section of the ICR provides a synopsis of overall context, development objectives and project design. An overview of the project costs and financing is given in the Annex 1, whereas outputs achieved by components is given in the Annex 2. List of supporting documents on the file is given in Annex 9 and Annex 11 provides some analysis of successful improvement examples in system functionality and use, at times beyond the scope of PIFRA II.

1.1 Economic and Financial Reporting Context at Appraisal

At the time of project preparation in 2005, Pakistan’s economic growth had accelerated from an average of 3.3 percent during 1997-2002 to 6.4 percent in FY2003/04 and 8.4 percent in FY2004/05. Public debt fell to 61 percent of Gross Domestic Product (GDP) in this same time period from almost 90 percent in FY2000/01. Improved fiscal performance and growing fiscal space resulting from savings on interest expenditures, generous external support, and improved revenue administration have enabled the Government to exceed targets for education spending.

The Government had recognized that weak governance was at the heart of Pakistan’s poverty and development challenges and that PFM reform was essential. Actions had been taken to address critical issues. Government accounting, for example, had been separated from government auditing by the creation of an Office of the Controller General of Accounts (OCGA) and PIFRA I completed significant improvements in accounting and auditing framework and piloting of IFMIS. 1.2 Project Development Objectives, Key Indicators (as approved), and Beneficiaries

PIFRA II’s project development objectives (PDOs) included the following: (a) build capacity to improve accuracy, comprehensiveness, reliability, and timeliness of financial and fiscal reporting at all levels of government; (b) improve PFM, accountability, and transparency; (c) enhance the capacity of public sector managers to use credible financial information for better and informed decision-making; and (d) facilitate oversight of the use of public monies and increase the national and international credibility of government financial statements and assurance processes. Key indicators were:

Implementation and operation of Information Technology (IT)-based budgeting and accounting systems, for example, number of sites operational and establishment of a SAP competency center;

Extent of reconciliation of accounts achieved and progress toward introducing interface with State Bank of Pakistan (SBP), National Bank of Pakistan (NBP), and Federal Board of Revenue (FBR);

Adoption of the new accounting model and reduced level of data entry errors; Improvement to scope, timeliness, and quality of reporting delivered to end-users; Extent to which entity accounts are audited using modern audit practices, and savings are

identified by the audit; and Creation and maintenance of a pool of professionally qualified government accountants and

auditors capable of developing and sustaining further initiatives.

Revised objectives (as approved by original approving authority) and Key Indicators, and Reasons/Justifications:

The PDOs were not revised. However, there were improvements made in the results framework, refining the outcomes which were better aligned to the PDO, including new milestones for closer

3

monitoring of implementation progress. Additional financing continued with these refined indicators and enhanced (and some new) targets as compared to those originally approved justifying additional investment of time and resources to upgrade system applications and hardware.

Primary beneficiaries of project: Office of the Auditor General of Pakistan (OAGP) in terms of capacity building. OAGP staff

benefited by receiving upgraded training facilities and programs of internationally accepted professional standards.

Office of the Controller General of Accounts. Strengthening of OCGA administrative and management powers through capacity building of CGA staff; and recruitment of senior staff with qualifications and experience.

Ministry of Finance at the federal level and Finance Departments at provincial level. Processes were improved to enhance efficiency for budget preparation and compilation.

Secondary beneficiaries of the project: Ministries, line departments, and agencies at federal and provincial level and districts across the

country received support in budget preparation and execution. Civil servants, pensioners, and government contractors and suppliers who benefit from

improvements in PFM through enhanced efficiency and transparency in the payment processes. Citizens, elected representatives, civil society, and development partners who can access the

timely budget execution information made public on the MoF website.

1.3 Original and Revised Components (as approved)

The project originally consisted of four components. At the time of the Level 1 restructuring in January 2011, component activities were revised. Instead of four components under PIFRA I, the Additional Financing phase comprised three main components aligned with the PIFRA II structure: Component 1, Financial Accounting and Budgeting System—MoF/CGA; Component 2, Audit; and Component 3, Project Management. Component 1 under Additional Financing integrated the original Components 1 and 2 into a holistic component since the MoF/CGA had taken over FABS operations and maintenance during the Additional Financing period. The revised Component 2 provided additional resources to further strengthen the external audit function through the establishment of AMIS initiated under PIFRA II and also allowed the OAGP to fully transition to systems and performance audit practices. The table 1 below summarizes the original and revised components:

Table 1: Original and Revised Components Original Components: Revised Components: Component 1: Financial Accounting and Budgeting System (FABS) (US$48.74 million) Component 1 comprised three phases.

(a) Phase 1 included extending FABS designed under PIFRA I to the balance of district accounts offices (DAOs), to the headquarters of some departments currently responsible for their own accounting, and to key sites in Azad Jammu and Kashmir.

(b) Phase 2 included further extension of PIFRA II systems to 140 central line ministries and provincial line departments, and 105 district government finance offices, plus the design and pilot implementation in 15 tehsils (smaller local governments) of a simpler financial management system in the immediate term, with the option to extend the application to a total of all 105 tehsil municipal administrations at the district headquarters around the country.

Component 1: FABS (US$19.40 million) This revised Component 1 consisted of four subcomponents:

(a) Subcomponent 1, Continued Operation and Maintenance of FABS. The emphasis was on (i) deepening reforms by ensuring that core systems functionalities (i.e., budgeting, accounting, payroll, general provident fund, and pensions) available in the system were implemented fully and uniformly at all levels of government; (ii) improving the comprehensiveness, accuracy, and timeliness of data being entered into the system and of the reports and financial statements produced by the system, and ensuring reports and statements were readily available for financial managers at the federal, provincial, and district levels; and (iii) ensuring that transaction processing was implemented as intended and in a

4

(c) Phase 3, implemented in parallel with Phases 1 and 2, provided for the following: (i) comprehensive end-user training program, training of finance managers, and building accounting capacity through training on financial management, the new accounting model, and the new chart of accounts; (ii) design of an interface between PIFRA systems and the NBP and SBP, FBR (then the Central Board of Revenue), and the Planning Commission, which allows for timely reconciliation of accounts and accommodates MTBF provisions in system configuration; (iii) development of overall systems and data architecture and capacity to ensure compatible information flows and summarization among national, provincial, district, and tehsil governments, and between PIFRA and non-PIFRA systems for whole-of-government reporting; (iv) upgrading of DAOs and other infrastructure as may be necessary; (v) determining the feasibility, benefits, and costs of extending FABS to the Ministry of Defense and Railways; and (vi) establishment of data warehousing capability and system.

manner that optimized systems use, eliminated redundant business processes, and followed standard accounting and data management and control practices.

(b) Subcomponent 2, CGA Capacity Development and Transition Plan. SAP competency center units servicing sites in Islamabad HQ and the provinces were streamlined and transferred to the CGA.

(c) Subcomponent 3, Fiscal Reporting on General Government. Consolidation of federal and provincial accounts in a general PFO Report was planned to achieve by end-FY2010/11. The CGA and AG Pakistan Revenues was responsible for performing all of the technical work to produce the fiscal reports.

(d) Subcomponent 4, FABS Support to MTBF Implementation. Integration of the MTBF on the SAP platform, which had been delayed pending acquisition of the SAP upgrade, was planned to provide greater assurance of data integrity and efficiency during budget preparation and budget execution.

Component 2: Capacity Building and Upgrading of the Office of the Auditor General of Pakistan (US$l9.345 million) Component 2 comprised four sub-components.

(a) Subcomponent 1. Reorganize and support technical assistance for audit management and development activities to implement new methodologies and increase office space and equipment to meet the needs of expanded professional audit functions directed at improving the central capacity to report publicly on the full regularity and performance audit mandate, including the adequate audit of revenue and the certification of annual accounts of corporate entities and national, provincial, and district levels of government;

(b) Subcomponent 2. Physical upgrading and IT support for field audit, including working conditions and equipment upgrading, thus enabling efficient implementation of audit methods designed during PIFRA I;

(c) Subcomponent 3. Implementation of an integrated audit management information system (AMIS) to improve audit control, reporting, and follow-up through better communications and planning between Islamabad and decentralized audit directorates; and

(d) Subcomponent 4. Upgraded training facilities and programs to develop staff to internationally accepted professional standards. (Deficiencies identified by a training needs analysis conducted during PIFRA I were corrected through the implementation of a comprehensive training program conducted in upgraded training facilities.)

Component 2: Audit (US$4.67 million) The Quality Management Framework had been finalized and required support of consultants for timely implementation. Completion of these initiatives enabled the OAGP to adopt standards of the Information Systems Audit Control Association. Adequate training of officers of the Pakistan Audit and Accounts Service (PAAS) to sustainably conduct all activities of the audit without the support of consultants was another critical activity. There was also need for structural reform in audit planning, execution, and reporting to allow the OAGP to achieve improved ratings under the Public Expenditure and Financial Accountability (PEFA) framework.

Component 3: Capacity Building and Upgrading of the Offices of the Controller General of Accounts (US$18.47 million) Component 3 comprised two subcomponents.

(a) Subcomponent 1, Strengthening government financial management policy and capacity: (i) strengthening CGA administrative and management powers through capacity building of CGA staff and recruitment of senior staff; (ii) building skilled internal audit complement via training and recruitment, and developing a comprehensive set of internal

5

controls, internal audit work programs, and reporting regime; (iii) revising/developing government accounting, budgeting, financial, and internal control manuals; and (iv) building IT capacity through recruitment of technical professionals and providing technical training opportunities for the staff responsible for the systems.

(b) Subcomponent 2, Strengthening of the OCGA facilities and systems to support the IT-based environment: (i) renovating OCGA headquarters to adequate standards to support a computerized environment; (ii) constructing and renovating OCGA in Balochistan and Sindh to adequate standards and to support a computerized environment; (iii) converting Balochistan Treasury Offices to DAOs; (iv) improving internal controls specified by the OCGA in consultation with the OAG and MoF Finance Departments for use by chief financial officers, and piloting implementation in 10 sites; and (v) improving records management to prepare financial reports to assist accountability and management.

Component 4: Project Management (US$6.45 million) The framework for project management comprised the following arrangements:

(a) Steering Committee, chaired by the Auditor General, was established with general powers to oversee project policies, implementation, and responsibility for reviewing proposals for capacity development and for post-completion activities.

(b) Project Directorate was empowered to ensure that project implementation ran smoothly and coordinated (between components, between provinces, etc.) and not held up by administrative bottlenecks.

Component 3: Project Management (US$0.43 million) The capacity of the project management team had been strengthened to effectively perform the monitoring and evaluation (M&E) functions and reporting of monitoring indicators to the various technical and steering committees.

1.5 Other significant changes

The project was restructured three times during implementation:

First restructuring. Level I restructuring was done in January 2011 to provide Additional Financing of US$24.5 million and also to extend the closing date of the original project from June 30, 2011, to June 30, 2013. The Additional Financing was aimed at helping finance activities associated with deepening the reforms that had been successfully established under PIFRA II and also allowed for a phased transition to substantial incorporation of the system under GOP line agencies.

Second restructuring. Level-2 restructuring of the project was done in May 2013 to extend the closing date of the project from June 30, 2013, to December 31, 2014. The extension was required to provide the extra time for SAP software upgrade for the existing financial information system.

Third restructuring. The third restructuring was done in December 2014 before the project closing; the World Bank cancelled an unutilized amount of US$10.3 million from the project and reallocated it to the Pakistan Program. The reallocation was done to avoid losing the country’s unutilized International Development Association (IDA) funds for this operation.

2. Key Factors Affecting Implementation and Outcomes

The World Bank has been providing support to the Government of Pakistan for improving financial accounting and reporting since late 1990s. There was a clear rationale and analytical foundation for the Bank’s continuous intervention in the PFM reforms through PIFRA II. This section of the ICR gives an account of key project design and implementation issues.

6

2.1 Project Preparation, Design, and Quality at Entry

Lessons learned during PIFRA I. The background analysis in the Project Appraisal Document included key lessons from PIFRA I and an outline for key challenges for reform continuity. Some of the key lessons from PIFRA I, which influenced the design of PIFRA II, are:

Stakeholder participation and strong ownership is critical to project success. The project’s design should provide for early extensive interactions with a wide range of government counterparts at federal, provincial, and lower levels of government. Also, finance and line agencies in addition to core accounting and auditing staff across government should be included in this participatory process to acquire and maintain solid support and engagement in advancing the reform agenda and monitoring the project’s implementation.

Strong project management is essential for smooth implementation. Having personnel in the project directorate with a project-oriented mindset makes a significant difference to the quality and timeliness of project outputs. Measures to limit the turnover of such staff, as defined above, would make the directorate more effective.

Experienced management of large-scale IT systems implementation is critical. There is a critical need to have a government IT manager in place for the continued success of PIFRA I and II. A general manager and Management Information Systems manager had in fact been appointed along with the project staffing; the SAP competency center had commenced with the initial appointment of 17 staff.

Analytic underpinning. As part of project preparation, sector background was studied, main sector issues were analyzed in depth, and government strategies to deal with these issues were also considered. The project design took into account the sector issues as well as government strategies.

Design features. Given the above background knowledge, the project design was kept straight forward, realistic, and well structured, even though it was complex because of the countrywide scope of the project. Being a follow-on project, PIFRA II adopted the basic design envisioned by PIFRA I but extended it to local-level government. The project’s approach was appropriate, both in its IT aspect and its organizational arrangements. Likewise, the results framework was clear, relevant, and monitorable. The design focused on the districts where capacity was already sufficient to permit introducing the new accounting and financial reporting systems (namely, the main cities, where the bulk of government expenditure occurs) and on the close and continuous interaction with other donors to avoid gaps and duplication.

Risk-mitigation. Several potential risks and mitigation measures were spelled out in the Project Appraisal Document. Risks assessed as High were (a) government fails to maintain commitment to fiscal and budgetary reforms; (b) resistance to financial transparency cannot be overcome; (c) Auditor General of Pakistan (AGP) and CGA discharge of responsibility for recruitment, staff deployment, and development of career path does not become effective and clear; (d) the size of the project overwhelms implementation capacity; and (e) adequate and qualified accounting, auditing, and IT staff are not hired, trained, and equipped with effective tools and are not retained to maintain and operate the system. The risks pertaining to government’s commitment and transparency that materialized during implementation were managed appropriately through IMF and the Bank’s poverty reduction support credit and development policy lending. However, risks related to discharge of responsibility between AGP and CGA over staffing and retention of trained staff remained key challenges throughout the life of the project. Procurement risks were properly identified, and mitigation mechanisms were designed and incorporated into the project design, including supervision missions every six months and an increased size of the sample in ex post reviews.

7

Design shortcomings. Some of the design shortcomings include the following:

The PDOs were both complex and ambitious. There were four key PDOs and six PDO indicators to begin with. Achievement of these objectives in some cases required policy shifts putting additional pressure on the implementing agencies. In retrospect, the additional financing was a missed opportunity to sharpen and refocus the PDO and results framework.

The focus of the project remained more toward improving financial accounting, auditing, and controls. The element of fiscal management reform was not fully envisaged or captured at the time of project design. This resulted in lack of interest and ownership for the project by MoF.

While PDO focused on “decision-making based on information”, the coverage among results indicators to adequately measure this PDO aspect of actual change in the decision-making behavior was insufficient.

The project was housed in OAGP, whereas project activities were largely driven by the OCGA. The project was attached to a single institution, which functionally lacked ownership or commitment to the objectives of other components.

The change management activities lacked interactive approach to support beneficiaries and clients with new procedures, systems, and reports. There was more emphasis on communicating PIFRA II components, activities, and benefits (for awareness-raising) than engaging beneficiaries in how it could help them in performing their respective roles more effectively. The change management approach did not envisage separate processes using audience-oriented vocabulary for each set of stakeholders such as MoF Finance Departments, public and departmental accounts committees, Planning Commission, PAOs, PPRAs, and budget-holders and building coalitions with these institutions and stakeholders for a joint ownership of PFM reforms. Finally, there was no earmarked funding allocated to the change management activities. These were combined with other project management activities and financed by the counterpart funding. Availability of counterpart funding proved to be a significant bottleneck directly affecting the implementation of change management activities.

2.2 Implementation

Overall implementation of the project was sound. Missions were conducted on a regular basis and there was consistent coordination and follow-up with the Government. The project benefited from additional financing and the closing date extension (see subsection 1.5). The midterm review was conducted as per the legal covenant in February 2008 and assessed the progress to date on all project components, implementation issues, and actions to be taken to ensure successful project completion. The project was successful in achieving its development objectives (Section 3). This was made possible by rolling out a well-parameterized, commercially available, off-the-shelf management information system (SAP) through a sustained implementation process with very a high-level of government commitment. PIFRA has been a huge PFM reform program and it had its allies and also there were pockets of resistance to change. This is the very nature of difficult reforms with political economy considerations. That said, the achievements of the project clearly demonstrate that project allies (through coalitions built) were able to limit the impact of resistance.

Managing the resistance to change. PIFRA II achievements have not been without challenges. The entrenched political economy and capacity challenges often appeared as turf issues (protection of patronage held by the accounts offices as internal control agents) or service issues (given that the accountants and auditors belong to different cadres). Pilot sites and initial replications met with huge resistance due to fear of losing one’s job, authority, and/or patronage. Similarly, the project implementation was affected by long delay in establishing district account offices in Baluchistan. - Nevertheless, the resistance in accounts and audit offices was handled through a long-drawn process

8

of change management, communication, and capacity building. Traces of that resistance are still evident in the accounts offices where it has lost authority of the budget re-appropriation recording function (now directly managed by the line ministries in the system) but must still do the thankless job of accounting transactions for civil works. Recent jurisdictional differences between OAGP and OCGA over the posting transfers of staff also reflect the political economy aspects of PFM.

The following factors have contributed in reducing the impact of the political economy challenges and sustained the pace of reforms during implementation.

(a) Sequencing. PFM reforms were sequenced to front-load the neutral interventions (without a high political economy impact), which included, for instance, automating existing accounting processes with a refined chart of accounts, capacity building, and infrastructure improvement for equipment. Significant reforms of the business processes were progressively introduced at a later stage.

(b) Capacity building, PIFRA II effectively built capacity of key resources such as SAP Competency Center (CC) and many PAAS staff, audit and accounts officers, and individual consultants are now absorbed by OAGP and OCGA. With adequate capacity and a dependency to perform government functions such as payroll, the system has a high certainty of sustainability.

(c) Incentives. Incentives such as project allowances were selectively used to co-opt people into the reforms process and make them allies to support the implementation or to at least stop creating hurdles.

However, change management strategy focused more on the primary system users such as AGPR, AGs, and DAOs, audit officers, and to some degree MoF Finance Departments and did not effectively focus on stakeholders such as PACs, PAOs, and DDOs. Citizens, researchers, and media were the least talked about. Secondly, the change management and communication activities were supply focused and not sufficiently interactive. This has led to sub-optimal reform ownership beyond the PIFRA II project implementation units or SAP CCs. Effective change management drove the SAP roll-out to more than 500 sites, but modest political economy challenges are still questionable with regard to new procedures and changes.2 Finally, the full-time designated officers at headquarters and regional offices did not support the communication and change management activities. Due to staffing and counterpart-funding constraints, change management activities were not effectively designed and implemented.

Delay in releasing counterpart funding. One of the major issues that impacted performance was unpredictability in the receipt of counterpart funds. In FY2010-11 this was a recurring pervasive issue on project performance, which suffered directly. Initially the overall decline in the economic health of the country led to an across-the-board cut applied to the development portfolio. For PIFRA II in particular, delay in finalization of the SAP upgrade contract, related requirement for approval of revised PC1, and then uncertainty about the extension of the project’s closing date by the World Bank all directly impacted the government decisions regarding allocation of budget and the release of budgeted funds. Counterpart funds were to cover salaries of staff and operational expenses. Since the project was directly entered into the national budgeting and accounting system, full allocation and release of budget was also required for IDA funds since budget checks are an integral control in the

2 For example, DAOs are not happy about losing the budget entry/re-appropriations function that is now carried out by MoF Finance Departments. Similarly, DAOs consider endorsement and accounting of assignment account or works payments as an additional burden and responsibility without any power to scrutinize and control expenditures.

9

system. Due to paucity of the counterpart funds, several expenses during the course of the project life were funded from IDA resources rather than the government contribution as originally planned.

Varying project management styles during the life of the project. While the Government assigned relatively senior civil service officers to PIFRA II compared to other development projects, there remained a high turnover (as an issue across-the-board). For instance, from inception in 2005, there were nine project directors. This lack of continuity and frequent changes in leadership delayed some of the project activities such as operationalization of the Disaster Recovery Center.

Capacity constraints at the OAGP level. The OAGP lacked the technical capacity to make independent judgments on complex technical issues related to the AMIS purchase. These issues included decisions about modified versus off-the-shelf software and over-specified tender documents that restrained competition. Also, there were internal deliberations and bureaucratic and procedural delays that hindered procurement of AMIS in PIFRA II. Two attempts to procure AMIS were made during the Additional Financing period, however both failed due to inadequate market response spurred by repetitive bidding.

Insufficient incentive for the collaborative tasks. There was a lack of a formalized mechanism to promote a collaborative work environment among stakeholders. Some outcome indicators went unaccomplished; the interface between the Economic Affairs Department and Federal Board of Revenue, though completed, awaited activation due to lack of a coordination mechanism between the project directorate and counterparts.

Lack of physical presence in the stakeholder location. Unlike SAP and the audit competency center in the OCGA and OAGP, the project team was not stationed in the MoF. Since the project staff were not accessible within the Ministry, responsiveness to the queries remained an issue. Vested interests, which viewed the implementation of the new system as a threat to their position, were able to build a successful opposition to the new system and cause significant delays in its implementation.

Extension of Financial Accounting and Budgeting System (FABS) to Tehsil/Town Municipal Administrations (TMAs) was one of the most ambitious deliverables of PIFRA, which were partially achieved due to several legislative amendments in 2010 onwards. PIFRA Team carried out research on the basis of four provincial Local Government Ordinances issued in 2001 and their subsequent amendments for this fourth tier of the government. The aim was to have a uniform solution conforming to the national accounting standards and procedures. A mini-SAP Solution was developed in-house for meeting the financial reporting needs of TMAs. A TMA specific new function cum object based CoA for TMAs, in line with the logic and structure of National CoA, synchronizing the TMA’s core functions and reporting requirements was also developed. For certain large TMAs selected across the country, LAN/WAN surveys were done, vendor for installation of LAN connectivity was selected and hardware was also procured. The home grown solution named TMA. Functionality was piloted at two TMAs – Chiniot and Rawal Town. The solution was being rolled out to TMAs in KP where two TMAs were brought on the system, a master trainers’ team was developed, CoA was adopted across the province for budget preparation and WAN connectivity was provided to eleven TMAs. During the year 2010, all provincial governments came up with their own draft Local Government Ordinances (LGOs). By the year 2010 two provinces – Balochistan and Sindh have promulgated these Ordinances while in Punjab and KP these legislations were approved by the respective bodies during 2012-13. Unlike LGO 2001, the four provincial LGOs do not provide a common and universal responsibility matrix and reporting line for financial management which was a limitation to implement further the TMA functionality developed by PIFRA. To continue with extension of FABS to TMAs, PIFRA had to develop four different models for the four provinces which was not possible keeping in view the remaining life period of the project

10

as well as some other critical activities before the project closure. Keeping in view the above, PIFRA in consultation with CGA decided to discontinue the efforts on this project deliverable.

2.3 M&E Design, Implementation, and Utilization

The M&E design provided for outcome indicators and results indicators for each component to assess progress in meeting the project implementation targets and objectives. OAGP had adequate methods for collecting the data for these indicators. At the time of Additional Financing in January 2011, original indicators were refined to ensure transfer of management skills and ownership to the primary stakeholders.

During implementation, data was gathered by the Management Information Systems wing and the M&E cell from monthly progress reports submitted by provincial sub-directorates and the respective implementing agencies. The implementation data, including achievement of milestones and any slippages occurring plus an analysis of the implications of the current status for future achievement of benchmarks, were reported quarterly to the Steering Committee and the World Bank. The Steering Committee was mandated to oversee implementation within its policy framework and post-completion activities. Implementation arrangements had requisite built-in flexibility, including re-prioritization of implementation in select provinces that were in more advanced stages of readiness.

The M&E team carried out qualitative surveys at regular intervals. These surveys were conducted to identify implementation issues and address them appropriately. Likewise, field visits to the various DAOs identified difficulties encountered in the printing of the monthly pay-slips for employees due to paper shortage. Therefore, the FABS team disseminated pay-slips to employees via email. In addition, data collected was used for decisions with regard to procurement activities and allocation of supplies to the Accountant Generals and DAOs.

2.4 Environmental Safeguard and Fiduciary Compliance

PIFRA II was assessed as a category C project during preparation, and therefore environmental assessment was not required. However, it was later discovered that the project would need to undertake some construction activities that resulted in a change to category B. Accordingly, an environmental assessment was undertaken and an environmental management plan (EMP) was prepared. However, construction activities had already started, and this constituted non-compliance with the environmental safeguards.

Consequently, the project started implementing the environmental management plan at the construction sites where the supervision engineers and construction contractors lacked the experience with EMP compliance, particularly worksite occupational health and safety protocols. The Bank safeguard team provided supervision support for the entire safeguard compliance. They held periodic meetings with the project personnel to address key issues and also carried out field visits to determine the compliance level at the construction sites. The situation gradually improved but, despite the Bank’s intervention, complete compliance was never achieved. Another challenge was lack of appropriate clauses and scope of work pertaining to environmental management in the contract of both the supervision engineers and contractors.

The project complied with fiduciary covenants during implementation. Internal control arrangements were in place, and adequate financial management system and records were maintained. In addition, procurement oversight during supervision was satisfactory. Fiduciary issues and problems were identified early on and remedial action were taken. In PIFRA II, off-the-shelf procurement of IT systems, servers, civil works, and SAP upgrade were among major activities that took a lot of time after the project launch. Advance procurement actions could have been very useful to avoid wastage

11

of considerable time at the project launch. Similarly, criteria for acceptance of assets and the policies and procedures for inventory management was developed after substantial procurements had already been made. Preparing and verifying the complete asset register was therefore a considerable effort.

2.5 Post-Completion Operation and Next Phase

Detailed sustainability strategies were prepared in 2012 for FABS and the auditing components and agreed on with the World Bank. Transition plans were subsequently finalized. These transition arrangements for continuing future operations appear to be adequate as demonstrated by the following measures adopted by the Government.

(a) Transitional arrangements

Institutional arrangements. Appropriate institutional arrangements are in place for the continuity of the reform process. A SAP Competency Center has been set up under OCGA and provides operational and maintenance services along with the end-user support.

M&E arrangements. M&E arrangements are institutionalized in OAGP and OCGA. M&E records pertaining to the project are transferred to the relevant parts of the government following mainstreaming of PIFRA II. The quality assurance cell in OCGA ensures that performance continues to be assessed and improved. A multi-tiered M&E process exists in the OAGP, which includes an Internal Compliance Unit in each Field Audit Office and two Quality Assurance/Control Committees performing under DG Audit Policy to review audit planning, execution, and reporting phases.

Staffing arrangements. MoF has approved the creation of positions for the key consultants, 64 in OCGA and 17 in OAGP. Currently, a temporary arrangement has been allowed by the MoF to extend the existing contracts for another year on existing terms and conditions. These positions will be filled by new hiring on a competitive basis on terms and conditions (including salary packages) as approved by the Finance Division (Regulation Wing). The OCGA and OAGP will be implementing these new arrangements for the sustainability of the reforms. A central maintenance team at CGA Islamabad comprising 10-15 professionals will be providing high-level technical maintenance support to the system. In addition, five director-level positions are initially planned at each of the provincial headquarters and AGPR Islamabad to provide maintenance support at these locations. Each office (provincial as well as federal) is supported by 4-5 SAP-qualified and -experienced technical experts such as system and network administrator, database administrator, basis expert, and Financial Information (FI) experts.

Training arrangements. Adequate arrangements are in place for the sustainable delivery of training courses to support the activities of both OCGA and OAGP. Under the transition plan approved by the DAGP, training activities pertaining to FABS and audit components have been transferred to Audit and Accounts Training Institutes (AATIs). Appropriate technical resources are available to continue the operations after project closure. To provide proper training, additional space, equipment, facilities, and faculty are made available. A new Audit and Accounts Training Institute (AATI) building has also been built at Lahore.

Policy arrangements. Reforms are now deeply embedded in the government’s machinery and are fully institutionalized. Among the examples is the redesign of the syllabus of the Federal Public Service Commission (Final Passing-out Exam of the Pakistan Audit and Account Services) to include the new accounting model and chart of accounts and the Financial Audit Manual (FAM). However, the actual transition did face some challenges. When FABS was handed over by PIFRA II to OCGA, physical infrastructure at certain sites was in disrepair. Presently, the OCGA is processing procurement of various such items under PPRA rules, which have lesser flexibility compared to what

12

was available for procurement under PIFRA II.3 Similarly, certain pending activities from within the project scope and system improvements are continuing.

(b) Follow-on operation

Building on the past reforms, the Government of Pakistan has expressed interest in follow-up PFM engagement to promote efficiency and effectiveness of service delivery expenditures by piloting an approach to strengthen implementation of PFM reforms in sector ministries, provinces, districts, and service units. It is envisaged that the objective will be accomplished through reforms in the area of public investment management, MTBF/integrated service delivery reporting, service monitoring, and effective accountability by strengthening DAC/PAC and other related entities. To address issues related to improving service delivery and institutional governance, it would be necessary to increase the reforms scope to include a second generation of PFM reforms for improved economic governance.

3. Assessment of Outcomes

Investment in PFM reforms by the Government of Pakistan has achieved concrete results. The Government and development partners repeated a PEFA assessment for the federal as well as provinces. These assessments confirm certain PFM improvements as well as pending shortcomings in recent years. Substantial progress on transparency through public availability of information is evident from the good ratings for related indicators in PEFA assessments and the improved score of 58 from 38 in the 2012 Open Budget Survey for Pakistan.4 Extensive budget documentation is prepared and published. Annual financial statements and audit report are laid before parliament within eight months of the end of the fiscal year. In-year reports are uploaded on websites of the finance departments. Figure 3.1 shows significant improvement in PEFA performance indicators (PI) average scores from 2009 to 2012 in almost all areas. The one exception in “accounting, recording, and reporting” can be attributed to non-rating (0 score) of PI-23 on “availability of resource information at service delivery units” due to the impact of the 18th Constitutional Amendment and pending issues with IPSAS cash-basis compliance.

Figure 2:

3 Novation for on-site support of M/s Siemens through contract modification with SAP Malaysia was not finalized under PIFRA and is now being pursued afresh with SAP Malyasia at the OCGA. Consultant transition is temporary for one year while new terms of engagement are being worked out. The challenge of new product development is being met adequately as are post-transition implementation of the OM Module and Debt Management solution in KP, dashboard of real-time BER for DDOs, PSDP dashboard for KP, and debt management solution for Sindh. 4 International Budget Partnership Full Report 2012 http://internationalbudget.org/publications/obi-full-report-2012-english/

13

3.1 Relevance of Objectives, Design, and Implementation

Relevance of objectives: Rating: High

The objectives of the project remain highly relevant. They reflect the paramount importance of a strong governance and accountability. The objectives and the design are consistent with both the Country Assistance Strategy at approval and the Country Partnership Strategy 2010-2013 – mainly by improving governance and accountability. They are also timely and appropriate to the needs of the country. The objectives are fully aligned with the Government’s plan to continue pursuing a comprehensive strategy to improve public accounting and auditing. The key reform priorities are identified in the 2003 Country Financial Accountability Assessment.

Relevance of design: Rating: Substantial

The project’s core design and implementation arrangements remain substantially relevant. The project’s implementation process was also effective and efficient for achieving the project objectives. Considering these factors, the relevance of design, and implementation is rated Substantial.

3.2 Achievement of Project Development Objectives ‐ Efficacy

The PIFRA II was successful in achieving its overall objectives. A Substantial rating is based on the fact that all originally approved targets for the PDO as well as the intermediate outcome indicators were either achieved or exceeded. Enhanced targets for the PDO indicators approved for as part of the Additional Financing were also achieved. However, some of the new intermediate outcome indicators introduced during the implementation, for better progress monitoring, were partially achieved (except acquisition of AMIS, which was not completed) and the government is working to complete these pending activities. While achievement of these intermediate outcomes could have further strengthened the overall achievement of the project, they were not critical for achieving the PDOs. For example, the AMIS could not be implemented; however, the audit quality and timeliness improved due to the application of risk-based audit approach, Computer Assisted Audit Techniques, and improved quality assurance procedures. Similarly, the completion of disaster recovery center was one of many complementing actions on information systems’ security which range from the routine information security procedures and back-ups to the information systems audits completed.

In summary, PIFRA II has remarkably improved the PFM landscape of Pakistan, bringing basic financial discipline to 500 government entities and nearly 7,000 line managers. This has facilitated processing and recording of millions of government transactions in a single database and generating and reconciling information on virtually a real-time basis for tracking poverty expenditures, timely audits, and informed decision-making. Audit cycle has been reduced from 33 to eight months for submission to the legislature. Training and professionalization of thousands of staff in accounting, auditing, and IT has ensured the sustainability of these reforms. PIFRA II achievements are broader in the context of entrenched political economy and capacity challenges.

Basic payroll, pensions, payments, and accounting functionalities on a country-wide roll-out included Gilgit Baltistan, railways, and externally financed projects. Budget execution reports are periodically posted on websites. Chart of accounts and business processes are based on solid data-architecture ensuring the ease of system understanding and effective use by the spending units. Thousands of workers, receiving both classroom and on-the-job training, resulted in 264 personnel earning professional certifications. Significant work has been done on the interfacing with SBP/NBP, EAD, and FBR. Recent innovations for bill tracking, salary slips by emails, and direct credit of pensions

14

are commendable. This has been an enormous achievement to overcome in a culture of secrecy and lack of public information.

The PDO was clearly defined and the project activities were closely linked with the achievement of these objectives. While the PDO has four stated objectives, there is a need to unpack these compound objectives for the purpose of this review as below: 1. Build capacity to improve accuracy of financial and fiscal reporting at all levels of Government 2. Build capacity to improve comprehensiveness of financial and fiscal reporting at all levels of

Government 3. Build capacity to improve timeliness of financial and fiscal reporting at all levels of Government 4. Build capacity to improve reliability of financial and fiscal reporting at all levels of Government 5. Improve Public Financial Management 6. Improve accountability 7. Improve transparency 8. Enhance the capacity of public sector managers to use credible financial information for better

and informed decision making 9. Facilitate oversight of the use of public monies 10. Increase the national and international credibility of Government financial statements and

assurance processes.

While the project results framework with PDO indicators and intermediary outcome indicators provide sufficient coverage to measure achievement of the above PDOs, a rearranged presentation of project’s major outcomes and achievements against each of the unpacked objectives is given in the table given in Annex 10. The table indicates that out of 10 project objectives, achievement of four objectives is rated as High and achievement of other six is rated as Substantial.

3.3 Efficiency

A comparative study of IFMIS among countries shows that Pakistan’s investment per user on IFMIS is in line with its comparators with equivalent or better utilization of features.5 Another study shows that IFMIS implementation time (2002-2009)6 under PIFRA I and PIFRA II is comparable to the average period to implement an IFMIS (7.9 years) and that 80 percent of the Bank projects financing the IFMIS were extended7. On the benefits side, examples shown in Section 3.2 demonstrate that operational and administrative efficiencies were derived as a result of design choices made and implementation approaches adopted under the project. The project has improved the financial and fiscal accountability in Pakistan through the provision of comprehensive, reliable, timely, and accurate data for informed decision-making.

A credible economic benefit calculation leading to a net present value (NPV) and economic rate of return (ERR) is not possible for such broad-based PFM reforms as it is difficult to claim attribution to all downstream benefits while there were many external factors potentially affecting accrual of those benefits. Nevertheless, it is plausible to state that PIFRA II has made a strong contribution to the following benefits and the impacts included in Annex 12:

5 Ali Hashim, A Handbook on Financial Management Information Systems for Government – A Practitioners Guide for Setting Reform Priorities, Systems Design and Implementation (World Bank, 2014) 6 IFMIS implementation was mainly completed from 2002 to 2009. This excludes the initial period up till 2002 spent on policy and institutional reforms and post 2009 period especially the Additional Financing which supported the systems and applications upgrades, capacity building aimed at sustainability of operation of these systems, and enhancing the functionality such as MTBF. 7 Financial management information systems : 25 years of World Bank experience on what works and what doesn't - Dener,Cem; Watkins,Joanna Alexandra; Dorotinsky,William Leslie (World Bank, Apr 2011)

15

(a) Enhanced fiscal discipline and expedient tracking of poverty expenditures saved billions of dollars. Salary related scams (especially ghost workers) were common in Pakistan, which are not there anymore after the new systems have been implemented with enhanced requirements of the human resource records and direct payment to the recipient’s bank accounts.

(b) Audit cycle was reduced from 33 to eight months for submission to the legislature after the fiscal year-end.

(c) Audit has become an effective deterrent to fraud and corruptions due to increased timeliness and use of Computer Assisted Audit Techniques.

(d) US$1.63 billion recovered due to audit reforms since 2005. (e) The strengthened use of single treasury account brought enormous interest savings by

improved ways and means position. (f) Automation created a savings on wage bill for accountants and auditors in the country with no

increase in the total number of government accountants and auditors during the last decade while government budget and expenditure increased three times and the number of transactions doubled. Online real-time data availability has also reduced the staff time and effort to collect and aggregate data from district offices.

(g) Enhanced resilience was evidenced by business continuity during crisis and salaries of public servants continued without disruption.

(h) Direct credit schemes for salary and pension payments helped to eliminate queues outside NBP in extreme weather and to identify ghost pensioners.

(i) PIFRA II has led to a significant use of country FM systems for externally financed projects with 16 Bank-financed projects and one United States Government project on FABS. The use of country FM systems eliminates the need for a new system, additional staff, implementation of a project-specific accounting software, and audit by private firms. On one hand, this saves cost; on the other hand, it leads to the availability of real-time project information in the government system. Finally, enhanced fiduciary reliance on country PFM systems has enabled the development partners to significantly increase the use of budget support and results-based financing mechanisms to support government-own programs instead of ring-fenced investment projects. Auditor General’s auditing of externally financed projects saved considerable fees otherwise paid to the private audit firms.

(j) E-transmittal of salary slips saved millions of dollars on paper, printing, and postage. (k) Utmost reliance on SAP information by MoF to make expenditure authorization decisions in

the capacity of the financial adviser for various ministries. (l) Increased assurance due to high quality in-year and year-end government reporting has

enabled the development partners to enhance the use of budget support and results-based funding modalities in Pakistan leading to enhanced aid-effectiveness.

The broad-based positive impact of the project given above benefits is assessed as Substantial.

3.4 Justification of Overall Outcome Rating

The overall project outcome rating is satisfactory. The justification for ratings of relevance, results achievement/efficacy, and efficiency are elaborated in subsections 3.1, 3.2, and 3.3, respectively. All ratings were either Substantial (S) or higher, with a High (H) rating for relevance of objectives and the achievement of four of the ten objectives aforementioned in subsection 3.2.

Table 3. Calculation of the Overall Project Outcome Rating Relevance of Efficacy: Objective Efficiency Overall

Outcome Objectives Design 1 2 3 4 5 6 7 8 9 10 High Substantial S S H S S H H S H S Substantial Satisfactory

16

3.5 Overarching Themes, Other Outcomes, and Impacts

PIFRA II led way to several positive impacts in terms of better tracking PRSP expenditures, better allocations, reducing corruption, grass-root level community engagement etc. These impacts are captured in Annex 12.

3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops

Not applicable 4. Assessment of Risk to Development Outcome

Rating: Moderate The Government has demonstrated its strong commitment to sustain the achievements of PIFRA II. A detailed sustainability (exit) strategy was prepared in 2012 for FABS and the auditing components and agreed on with the World Bank. Transition plans were subsequently finalized.

The well-established FABS operations have successfully transitioned to the institutions. To continue with system maintenance and smooth operations, the retention of system experts is critical. These technical experts are essential management of computer servers, SAP-based programming, IT system audits, quality assurance, and management of wide/local area networks. The MoF has also approved the creation of the post for key consultants.

FABS has also developed the complete functionality to record commitments. The functionality requires creating a purchase order before the recording of commitment; payment is allowed only once the purchase order is created and commitment is recorded. But this control is often not implemented in accordance with the procedure.

The project could have better utilized the change management strategy. The Change Management Unit carried out activities as a part of a communication strategy highlighting project deliverables and results. The strategy focused more on primary stakeholders within the AGP and CGA organizations and on awareness of reforms instead of strengthening the use of improved systems.

While information and system security procedures are in place, the Disaster Recovery Center was not functional as planned by the end of the project. The CGA will be able to complete the ongoing work on the Disaster Recovery Center in a few months.

5. Assessment of World Bank and Borrower Performance

5.1 Bank performance in ensuring quality at entry

Rating: Satisfactory

During preparation and appraisal, the Bank took into account the adequacy of project design and all major relevant technical and institutional, including procurement and financial management. In addition, major risk factors and lessons learned from earlier projects in the governance sector were considered and incorporated into the project design. Project preparation was carried out with an adequate number of specialists who provided the technical skills mix necessary to address sector concerns and a good project design. The Bank provided adequate human and financial resources to ensure quality preparation and appraisal work (details in Annex 4). The project was consistent with the Country Assistance Strategy and government priorities in the sector at the time. The Bank had a consistently good working relationship with the borrower during preparation and appraisal. Hence, with some minor shortcomings in project design, as mentioned in Section 2.1, Bank performance at preparation is rated Satisfactory.

17

5.2 Bank performance in quality of supervision


A Quality Assessment of Lending Portfolio was carried out in September 2008 by a Quality Assurance Group (QAG) panel of reviewers. The panel focused on development effectiveness, and supervision inputs and processes as Highly Satisfactory; the quality of Bank supervision, and candor and realism of Implementation Status and Results (ISR) reports as Satisfactory; and the quality of design, quality of implementation and the fiduciary/safeguard aspects as Moderately Satisfactory. The QAG panel offered the following notes:

Implementation of PIFRA II by the government has been distinctly better than that of PIFRA I (difficulties of which were, however, largely outside the control of the Bank team). Good capacity in the implementing agencies, strong overall support by the government (after initial hesitancies by certain stakeholders, mainly for turf reasons), and attention to emerging issues were strong points.

The focus on development effectiveness and supervision inputs and processes are the signal strengths of PIFRA II. PIFRA II is a rare case of close oversight and supervision in real time, over the entire life of the project, by competent and experienced professionals. Moreover, the nature of the supervisory relationship was (and remains) unusually collegial. Despite the turnover of project management, the Bank team kept itself appraised of every significant development and intervened promptly to help resolve emerging issues.

The over-detailed results framework and the generic ex ante arrangements for monitoring did not in practice prove a problem because of the task team location in the field, close interaction with project management, and experience and expertise of the main consultants.

Overall, procurement oversight during supervision has been satisfactory. Procurement issues and problems have been identified early on and remedial action attempted.

The seamless interaction with the PIFRA I team, the intensive engagement of field-based staff, and the continuity of participation by experienced and committed consultants has ensured high-quality results in an otherwise challenging environment.

The same rigor continued after September 2008 and the Bank’s performance during the implementation of the project was satisfactory. Sufficient budget and staff resources were allocated, and the project was adequately and intensively supervised, and closely monitored (details in Annex 4). The task team prepared aide-memoires regularly, prepared and alerted the government and the PIFRA Directorate to problems with project execution, and facilitated remedies in a timely manner in conformity with Bank procedures. The ISRs realistically rated the performance of the project both in terms of achievement of development objectives and project implementation.

The Bank’s task team played a crucial coordinating and proactive role in suggesting and promoting solutions, in a mode of partnership with other donors. The team focused on relevant and clear performance indicators. The task team also monitored safeguard and fiduciary compliances. They developed continuous dialogue and communication with project staff and were successful in preventing potential problems from emerging and gathering information on weaknesses in internal financial controls. For example on the FM side, Bank staff were proactive in supervision efforts, going beyond the review of FMRs and audit reports. On procurement side, the Bank remained responsive to the capacity needs of PIFRA II and assisted them in building and retaining a procurement support structure that best suited the implementation needs.

18

5.3 Justification of Rating for Overall Bank Performance


With a Satisfactory rating for quality at entry as well as for the quality of supervision, overall Bank performance is rated as Satisfactory in accordance with Impact Evaluation Group harmonized rating criteria.

5.4 Government Performance


As mentioned in Section 2.1, the Government had shown its commitment to the objectives of the project at the time of project preparation. During implementation, there was a high level of ownership and commitment to the project at all levels of executive management in OCGA; however, other key stakeholders, including the Planning Commission, Economic Affairs Division, FBR, and the MoF ownership level for the project varied due to lack of communication channels and project-funds managed by them. Senior government officials worked closely with the Bank’s project team on a continual basis, and cooperated fully with the task team. Appropriate levels of review and approval were usually in place. Financial accountability and follow-up was observed. Expenditures were duly authorized before they were incurred. And documentation was maintained properly for periodic review. By and large, borrower commitment was good at federal and provincial level with the exception of Balochistan province, which had been affected by factors beyond the jurisdiction of the project. While there were some factors that impacted performance including the unpredictability in the receipt of counterpart funds and turnover of senior managers as mentioned in Section 2.2, there impact was marginal. Considering all the aspects, overall government performance is rated Satisfactory.

5.5 Implementing Agency or Agencies Performance Rating: Satisfactory General execution and implementation. In the beginning, performance of the OAGP was Satisfactory. Key milestones such as risk-based auditing and geographic dispersion of FABS were achieved all over the country. These achievements greatly contributed toward the project’s success. However, OAGP performance deteriorated toward the end of the project, but the project activities had achieved a momentum of their own. As mentioned earlier, the project had nine project directors in nine years, with four project directors in the last two years but continuity of the rest of the project management team ensured continuity of policies and activities’ momentum. In more recent years, the project did suffer somewhat due to political economy issues, particularly because of jurisdictional differences between OCGA and OAGP over HR issues, but again the negative impact was marginal given the enormous project achievements. Financial management. While the risk of using funds for unintended purposes remained Moderate, the FM performance of the project declined in the last two years of the project from Satisfactory to the closing rating of Moderately Satisfactory. To safeguard the assets procured under the project, a Policies and Procedures Manual for Procurement and Inventory Management of Assets was issued. Acceptable audit reports for the project were received on a timely basis. Since the project was based in the country’s supreme audit institution, private sector auditors were engaged. However, delays were often observed in receipt of quarterly financial reports.

19

Procurement. Procurement of all works, goods and technical services under the project followed the procurement guidelines, Procurement under IBRD Loans and IDA Credits. A fully dedicated Director General (DG) Procurement, supported by a Director Procurement and consultant, formed the procurement unit for PIFRA II. However, the primary support was drawn from the consulting staff and that of an associated engineering cadre. While PIFRA II recognized and acted on need to strengthen procurement support system, less attention was paid on having a commensurate contract administration capability. This entailed some delays in implementation of goods and works contracts. Overall performance of OAGP was Satisfactory during implementation. 5.6 Justification of rating for overall borrower performance Rating: Satisfactory With a Satisfactory rating for government performance and a Satisfactory rating for implementing agencies’ performance, overall borrower performance is also rated as Satisfactory in accordance with Impact Evaluation Group’s harmonized rating criteria. 6. Lessons Learned The sections looks at lessons learned from the project design, implementation, safeguards, and fiduciary aspects. 6.1 Design

Since the implementation rationale for PIFRA projects was to achieve proper and better management of public resources, it is important to frame such projects as public resource management projects rather than as accounting projects.

In any PFM reforms, the MoF has the central role, but PIFRA II was implemented by OAGP. This may have been the right approach at the time of project design, but it created issues and rifts during project implementation. For future PFM programs, the federal and provincial ministries of finance should be given the leadership role without compromising the presence of OCGA and OAGP. In addition, the World Bank should not remain indifferent to the possibility of institutional frictions within the government during the design and implementation of reforms. A lot more effort needs to be put in for identifying the potential pockets of resistance to these reforms, and dialogue initiated with all parties to best manage that resistance upfront.

In future projects, significant attention should be given to communication and change management components. The project remained focused on processes and systems development; only toward the end was some attention given to service delivery to the clients. The benefits of PIFRA II are not accruing fully to ministries, departments, and autonomous institutions as yet. In its design, the PIFRA II did not adequately address the low FM capacity at line ministries. Similarly, the audit component design did not have focus on the role of the Public Account Committee (PAC), which is the premier accountability forum. The PAC involvement would have enabled alignment of the legislature expectations with project outputs. In future projects, the focus needs to be on clients and beneficiaries that will also improve political ownership of reforms. An effective change management component should support the institutional collaboration and intended behavior change. PIFRA II relied on civil servants for the communication and change management activities and the performance was

20

less than satisfactory, nevertheless, there were some successful examples of change. Annex 11 captures a few such case studies to analyze the demand-driven change process for systems improvement and use of information for decision making.

When multiple donors are interested in project success, joint implementation arrangements should exist from the beginning to increase momentum toward effective implementation.

Reliance on counterpart funds for critical activities such as change management can affect the implementation.

6.2 Implementation

Stakeholders should be the primary drivers of project objectives. Stakeholder participation and strong ownership are critical to project success. Investments on PIFRA-type projects can be capitalized optimally by having a robust technical support structure to support the user community. In addition, PIFRA’s support organization has to work as a service organization within the government structure; therefore, it is important to strengthen the service delivery mindset. Moreover, this organization should be governed through key stakeholders such as the Ministry of Education, Ministry of Health, and provincial governments etc.

Recognizing the importance of technical support for the sustainability and success of a project as such as PIFRA II, the Government should create a career path for technical staff and recognize them as an integral part of civil service. In addition, motivational strategy should be defined particularly in the situation where multiple ministries, agencies and departments are involved in the reform process.

7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners 7.1 Borrower/implementing agencies

Elaborate comments on ICR, received from the Economic Affairs Division, Finance Division, Office of the Auditor General and Office of the Controller General of Accounts, are fully acknowledged. These comments confirmed the ICR conclusions and ratings. AGP confirmed the ratings assigned to all outcome indicators and provided revised numbers of trainings conducted under PIFRA II, which were accordingly incorporated in the final ICR. CGA confirmed ratings assigned to all indicators except the one related to SAP support to MTBF and suggested that the target was partially achieved. The ICR team retained the rating assigned as the complete rollout of MTBF to all ministries and related reforms were not within the scope of PIFRA II and should be taken up as part of the next phase of PFM reforms. The ICR team also recognizes the fact that EAD and FD are eager to delve into the human resource and governance issues building on the earlier PFM systems reforms and enhance the use of system for decision making and improving service delivery. PIFRA II has provided a solid foundation for the next generation of these reforms. 7.2 Co-financiers, other partners and stakeholders

Not applicable

21

Annex 1. Project Costs and Financing Table A1.1: Project Cost by Component

Project Component/Category (IDA-4109) (IDA-4864) Total

US$ million (equiv) US$ million (equiv) US$ million (equiv) FABS Technical assistance / Consulting services 12.251 4.013 16.264 Training and Development 3.217 0.486 3.703 Civil works 8.090 0.754 8.844 Equipment 29.837 5.531 35.368 Vehicles - 0.001 0.001 Capacity Building and Upgrading (OAGP)

Technical assistance / Consulting services 1.995 1.693 3.688 Training and Development 1.466 0.451 1.917 Civil works 5.151 0.039 5.19 Equipment 1.682 0.514 2.196 Vehicles 0.644 - 0.644 Capacity Building and Upgrading (OCGA)

Technical assistance / Consulting services 1.041 2.092 3.133 Training and Development 2.565 0.830 3.395 Civil works 7.928 - 7.928 Equipment 0.040 0.113 0.153 Vehicles 0.071 - 0.071 Project Management Technical assistance / Consulting services 0.675 0.868 1.543 Training and Development 1.431 - 1.431 Civil works - - - Equipment 0.471 0.052 0.523 Vehicles 0.209 - 0.209 Total 78.764 17.437 96.201 Table A1.2: Financing

Source of Funds Appraisal estimate

(US$ millions) Actual/Latest estimate

(US$ millions) Percentage of

appraisal Borrower 18.137 18.137 100.00 IDA 108.504 96.201 88.66

Total 126.641 114.338 90.285

22

Annex 2. Outputs by Component

Intermediate outcome indicators Achievements to date Comments/Actions taken

Component 1: FABS-MoF/CGA Government of Pakistan has purchased SAP Enterprise Resource Planning (ERP) software and is using it for its business process automation of the following. 01 Commitments and payments Appendix 02 Receipts Appendix 03 General Ledger Appendix 04 Budgeting Appendix 05 Aggregation and reporting Appendix 06 Fixed assets Appendix 07 Banking Appendix 08 Cash forecasting Appendix 09 Development projects Appendix 10 Payroll and General Provident Fund Appendix 11 Pensions Appendix 12 Staff expenses Appendix 13 Communication and validation Appendix

1. Effective reconciliation of accounts for fiscal reporting and planning

1.1 Discrepancies between monetary survey credit and government fiscal deficit (for 2004/05 equals 1.2% of GDP).

Various reforms under PIFRA II have significantly contributed to reducing statistical discrepancies. These reforms include eliminating redundant processes, timely recording of expenditure in appropriate accounts, and effective reconciliation of accounts.

Statistical discrepancy varies between quarters. Data to calculate this indicator is collected and collated by the MoF. Quarter-wise variations occur because EAD and self-accounting entities do not report their expenditures on time.

1.2 Issue of reliable, timely BERS to PAOs

BERs are available at every level of government through the system. SAP link has already been given to departments, and they have access in the system to generate BER when required by PAOs. AGs also send hard copies to them for reconciliation. System utilization for issuance of BERs has increased more and more as PAOs become aware of and are trained in using SAP. Commitment functionality has been fully developed by PIFRA II through implementation of FABS. However, commitments are not being recorded in a timely manner and in true spirit. Commitments are recorded at the same time when bill is recorded.

MoF has taken initiatives which will push ministries and departments to record commitments on a timely basis and which will link the release of funds with in-time recording of commitments.

2. Accurate, comprehensive, and timely reports, based on international standards of accounting and internal control.

2.1 Application of IPSAS

System has been developed to capture and report third-party payments. Report by consultant on accrual accounting was sent to OCGA for necessary action and will be finalized soon.

2.2 Implementation of commitment accounting

Commitment functionality has been fully developed by PIFRA II through implementation of FABS. However, commitments are not being recorded in a timely manner and in true spirit. OCGA monitors the implementation of the commitment accounting.

MoF has taken initiative which will push ministries and departments to record commitments on a timely basis and which will link the release of funds with in-time recording of commitments.

3. System functionality tracking for improved efficiency and reduced discretion

3.1 Bills passed within 3 days (%)

Aggregate figure for August 2014 stands at 48% passed within 3 days. Province-wise breakdown below: Islamabad, 46% Punjab, 41% KP, 51% Sindh, 39% Balochistan, 83%

The fact that bills are submitted manually adds delays in processing time. Often incomplete documentation is received from line departments and ministries, which further delays the process. Bills processed in the last quarter of the financial year are almost equal to what is processed throughout the year. High volume

23

and manual submission of bills has skewed the workload and affects the overall average. In areas where there is strong supervision by DAG, AG, and Additional AG, bills are being processed in a timely manner (e.g., Balochistan).

3.2 All checks issued by system/direct vendor payment

All checks are issued through system and suppliers’ checks are made payable direct to the vendors’ bank account.

4. Achieve comprehensive data entry to FABS

4.1 Comprehensive data entry coverage (% expenditure and receipts)

Data is collected by the AG on a regular basis. However, time lags may occur due to delay in ex post transactions from departmentalized accounts and self-accounting entities. (Information is received beyond the deadline of 10th of every month.) System is capable of capturing all transactions.

4.2 Interfaces with SBP/NBP, FBR and EAD established

Interfaces have been developed for SBP and NBP PIFRA II team has also developed a robust interface for the Planning Commission and data on PSDP has been populated.

Interface with EAD and FBR has also been developed, however, no progress has been made to date.

Component 2: Audit

5. Effective and timely audit of government accounts using international standards of auditing

5.1 Application of modern audit practices

Quarterly control framework operational since 2010-11. Information system audit established and intense training carried out. Updated status of training: Over 40 performance audit specialists trained (including master trainers). FAM training: 4,946 auditors trained against the project-end target of 4,744 (94% completed). Training in CAATS: 2,559 officers/officials trained against project-end target of 2,618 (completed 92%). Local training for degrees/ certification courses for OAGP: 197 out of the project-end targets of 204 (93.6% completed). Training in special sectors: Training in performance, information system, public-private partnership, forensic auditing. 907 officers/officials of DAGP trained against the project-end target of 1,023.

5.2 Performance audit of policy-linked programs (such as poverty reduction)

Trained 30 PAAS officers who have conducted 3 performance audits to ensure on-the-job training. Performance audits were conducted on the following projects: - Solid Waste Management Program - Benazir Income Support Program - Procurement of Wheat by Government of Sindh. Performance audits are now a part of annual

24

audit plan; during last 3 years, 19 performance audit reports/special studies have been presented to the legislature. The Performance Audit Wing of the DAGP is being strengthened and refurbished by providing additional resources. The ITP courses are restructured in line with INTOSAI guidelines.

5.3 Implementation of AMIS

A single bid was received for off-the-shelf AMIS software bid invitation.

Bid evaluation report shared with the Technical Committee on Audit as well as with the World Bank team. This process concluded recently and the bid was found to be non-responsive.

.

5.4 Training of PAAS staff to replace consultants

Status of PAAS officers trained:

264 officers have been trained from OAGP and OCGA: 40 foreign degree-level trained. 78 IFCE certified. 20 MBA, CISA, ACCA, ACDA 46 SAP Certified/Academy 66 Training in special sectors (performance

auditing, forensic auditing, environmental auditing, information system auditing, energy sector auditing).

6 ACL master trainers 8 FAM master trainers DAGP has adopted INTOSAI guidelines, developed a handbook, and has revised the training approach and modules to replace an outdated curriculum. CISA has developed training modules specific to professional needs of senior audit management and field auditors.

Component 3: Project Management

6. Management skills transfer

6.1 Transfer of M&E and change management skills to OCGA and OAGP.

Mainstreaming of M&E and change management functions is on target. Management guidelines’ on each process/activity of system published for all officer/staff levels in OCGA and OAGP. M&E, FABS, and CM Wings share the M&E and CM techniques and methodologies with OCGA, OAGP, and MoF from time to time and are always available for any further assistance when required by stakeholders. M&E team also visited AG offices and DAOs for knowledge transfer.

Vigilance will be required to ensure regular monitoring is undertaken post PIFRA.

25

Annex 3. Economic and Financial Analysis Not applicable

26

Annex 4. Bank Lending and Implementation Support/Supervision Processes Table A4.1: Task Team members

Names Title Unit

Supervision/ICR Abid Khan Program Assistant SACPK Altaf Ahmad Senior Program Assistant LEGCF Ali Hashim IFMIS Consultant SARFM Aman Trana Senior Financial Management Specialist SARFM Amir Munir Senior IT Officer, Program Manager ITSCR Anwar Ali Bhatti Financial Analyst SACPK Anthony Graeme Lee Senior Municipal Finance Specialist SASDU Asif Ali Senior Procurement Specialist GGODR Furqan Ahmad Saleem Senior Financial Management Specialist GGODR Gloria Avershima Kwembe Program Assistant GGODR Hanid Mukhtar Senior Economist GMFDR Hasan Saqib Senior Financial Management Specialist SARFM Ismaila B. Ceesay Lead Financial Management Specialist GGODR Javaid Afzal Senior Environmental Specialist GENDR Mirza Bisharat Baig Senior Resource Management Officer BPSGR Paul Welton Lead Financial Management Specialist/Task Team Leader GGODR Pazhayannur K. Subramanian Lead Financial Management Specialist GGODR Rehan Hyder Senior Procurement Specialist GGODR Riaz Mahmood Financial Management Analyst GGODR Saeeda Sabah Rashid Senior Public Sector Specialist GGODR Sajid Maqsood IT Officer ITSCR Shahzad Sharjeel Senior Communications Officer SAREC Syed Waseem Abbas Kazmi Senior Financial Management Specialist GGODR

Table A4.2: Staff Time and Cost

Stage of project cycle Staff time and cost (Bank Budget only)

Number of staff weeks US$ thousands (including travel and

consultant costs) Lending

FY05 23.4 459.10 FY06 12.0 89.17

Subtotal 35.4 548.27 Supervision/ICR

FY06 8.8 101.56 FY07 18.2 156.48 FY08 16.3 253.14 FY09 4.8 71.36 FY10 6.0 118.93 FY11 8.6 176.65 FY12 6.8 142.37 FY13 8.0 136.15 FY14 10.2 112.03 FY15 32.3 364.35

Subtotal: 120.0 1,633.02

27

Annex 5. Beneficiary Survey Results NA

28

Annex 6. Stakeholder Workshop Report and Results Not applicable

29

Annex 7. Summary of Borrower's ICR and/or Comments on ICR

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

Annex 8. Comments of Co-financiers and Other Partners/Stakeholders Not applicable

46

Annex 9. List of Supporting Documents

Project Implementation Plan Project Appraisal Document for Pakistan: Second Improvement to Financial Reporting

and Auditing (PIFRA II) dated August 2005 (Report No: 33121-PAK)

Aide Memoires, Back-to-Office Reports, and Implementation Status Reports.

Project Progress Reports.

Borrower's Evaluation Report dated ----------2015 *including electronic files

47

Annex 10. Assessment of Achievements by Objective

Objective 1: Build capacity to improve accuracy of financial and fiscal reporting at all levels of Government (Rating: Substantial) PDO Outcome Baseline Target Actual Comments 1. Effective reconciliation of accounts for fiscal reporting and planning (discrepancies between monetary survey credit and government fiscal deficit)

Estimated statistical discrepancy 2004/5 -1.2% GDP

Proper recording of accounting entries according to NAM to enable federal unidentified expenditures reduced to no more than 0.5 percent of total actual expenditures

0.5% of GDP in July- Sept 2014

Achieved. - Various reforms under PIFRA II have significantly contributed to reducing statistical discrepancies. These reforms include, eliminating redundant processes, timely recording of expenditure in appropriate account codes, computerized aggregation of accounts for several jurisdictions, and effective reconciliation of accounts.

2. Effective corruption and inefficiency reducing measures related to accounts and budget control applied (system monitoring of use of functionality, bills, and issuance of checks etc.)

Manual issuance of checks and no systematic review of workflow practices

-Proper recording of accounting entries according to NAM to enable federal unidentified expenditures reduced to no more than 0.5 percent of total actual expenditures - Full real-time quarterly reconciliation

System reporting fully under OCGA and regular OAG audit. Unidentified expenditures reduced to no more than 0.5 percent and quarterly reconciliations are near real-time.

Achieved. - Issuance of manual checks has been abandoned through regular system reporting and support available to accounts offices. - This has significantly improved the fiscal discipline over the payments. - Use of Computer Assisted Audit Techniques (CAATS) has enabled auditors to perform audit of transaction data in an efficient manner, which has led to enhanced accountability on budget execution.

Objective 2: Build capacity to improve comprehensiveness of financial and fiscal reporting at all levels of Government (Rating: Substantial) PDO Outcome Baseline Target Actual Comments 3. All checks issued by system/direct vendor payment

Manual issuance of checks

Direct payments to 70% of vendors

100% checks are issued through system and suppliers’ checks are made payable direct to the vendors’ bank account

Achieved. - Comprehensive transactions data for federal, provincial, and district levels of government is captured by FABS on a near real time basis. - Five dimensional comprehensive chart of accounts is used to capture information by entity, function, and economic classification. - Cash balance instantaneously known compared to when commercial banks held accounts.

4. Achieve comprehensive data entry to FABS

No coverage of projects, self-accounting entities, foreign-financed transactions, and MoF transactions incomplete or non-timely

100% Data is collected by the AG on a regular basis from the departmentalized accounts and self-accounting entities.

Achieved. - 16 Bank-financed and one United States Government financed projects are using FABs, so real time data for those projects is available in the system - For other projects and self-accounting entities, the system is capable of capturing all transactions, but at times information received beyond the deadline of 10th of every month causing a few days’ delays in the preparation of the whole of government accounts.

5. Application of International

Financial statements did

District, provincial and federal

Annual financial statements are

Achieved. -The budget formulation and execution is

48

Public Sector Accounting Standards

not comply with IPSAS

governments are using NAM and common account classifications that can be readily aggregated and disaggregated by automated systems

largely compliant with IPSAS

based on administrative, economic and sub-functional classification, using Government Financial Statistics (GFS)/COFOG standards Public Expenditure and Financial Accountability (PEFA) Performance Indicator (PI) -5 ‘A’ rating) -Financial statements at federal and provincial level now largely comply with IPSAS cash-basis systems. - Financial Statements are being prepared under cash basis of accounting and are compliant with the format given by the cash basis IPSAS. However it lacks certain disclosures, such as Third Party Payment Disclosure, commitment accounting, undrawn borrowing facilities and undrawn external assistance disclosures. The system has been recently developed (October 2014) to capture Third party disclosures, thus automatically tagging the 3rd party payments at data capturing stage and disclosing it in Monthly Civil Accounts and Annual Financial Statements as separate item together with other missing disclosures. - In addition, CGA has approved the accrual road map and have plans to share it with MoF and other stakeholders.

Objective 3: Build capacity to improve timeliness of financial and fiscal reporting at all levels of Government (Rating: High) PDO Outcome Baseline Target Actual Comments 6. Issue of reliable, timely BERs to PAOs.


100% BERs issued BERs are available at every level of government through the system

Achieved. - SAP link has already been given to departments, and they have access in the system to generate BERs when required by PAOs. - AGs also send hard copies to them for reconciliation. - System utilization for issuance of BERs has increased more and more as PAOs become aware of and are trained in using SAP.

7. Effective and transparent financial reporting, control, and audit (timely, reliable financial statements submission and timely audit)

Accounts submitted to Office of the Auditor General (OAG) 3-4 months after due date. OAG accounts 15 months or more after year-end. Quality poor.

OAG certification and audit report within 6 months

Certification and audit is completed within 6 months of the fiscal year-end and two additional months are spent for rigorous quality assurance process and submission to the legislature.

Achieved. - Audit of 153 financial statements /appropriation accounts are prepared within 6 months for the last consecutive 5 years (previously 24-28 months). - Internal controls have strengthened due to improved transparency and by enabling stakeholders to generate timely reports through the system on a real-time basis. And FABS enabled MTBF budget preparation and expenditure tracking in the system. - These reports undergo detailed quality reviews in accordance with Quality Management Framework in two months. - The target is achieved due to timely submission of unaudited financial statements to OAG due to FABS and improved business processes, and then due to risk-based audit approach which is now adopted by OAG for the purpose of financial audit. - This is a major achievement to prevent

49

misuse of resources through timely and effective audit as a deterrence and also to expedite financial accountability on all issues identified.

Objective 4: Build capacity to improve reliability of financial and fiscal reporting at all levels of Government (Rating: Substantial) PDO Outcome Baseline Target Actual Comments 8. Effective and timely audit of government accounts using international standards of auditing to ensure reliability of financial and fiscal reporting at all levels of government


- Relevant Auditor General’s staff trained in certification, financial audits, and Audit Management Information System (AMIS) - Quality control framework effectively operational and used for the FY10-11 audit. Information system audit on-job- training for FAO staff. Quality management facility, information system audit

Quality control framework operational since 2010-11, and a large number of auditors have been trained

Achieved - Audit quarterly control framework operational since 2010-11. Information system audit established and intense training carried-out. Updated status of training: Over 40 performance audit specialists

trained (including master trainers). 4,946 auditors, FAM training 2,559 auditors, training in CAATS 197 auditors, Local training for

degrees/certification courses for OAGP 907 auditors, training in

performance/information system, public-private partnership, forensic auditing, etc.

Objective 5: Improve Public Financial Management (Rating: Substantial) PDO Outcome Baseline Target Actual Comments 9. Establish stable FABS operations and maintenance support in GoP

- FABS system tested in Abbotabad site and replication started at 30 sites

- All FABS sites on full work flow (e.g., Khyber Pakhtunkhwa (KP) and Balochistan); FABS support unit established under OCGA for ongoing maintenance and support - Acceptance of, and direct management and control of, handed over PIFRA accounting sites

FABS has been fully established and is operational. More than 500 offices and 6,200 users are using FABS site implementation complete where law and order situation has allowed.

Achieved. - FABS site implementation complete where law and order situation has allowed. - FABS is fully transitioned and support is provided by OCGA. - SAP upgrade to ECC06 has been complete together with a server upgrade, and information security procedures are in place and implemented. - Finally, 64 mainstream technical positions have been authorized and budgeted by the Finance Ministry and against these positions 47 persons are performing their duties to ensure stable FABS operations and maintenance support.

10. Effective and transparent fiscal administration (MTBF in place and FABS supported)

Pilot Medium-Term Budget Framework (MTBF) implementation underway for 5 federal ministries

- Federal budgets prepared under MTBF principles - Financial Accounting and Budgeting System (FABS) supports MTBF

- Federal budgets are prepared under MTBF principles - FABS supports MTBF

Achieved. - The MTBF configuration in SAP has been completed. - This helps medium term forecasts (forward estimates) to be prepared on the basis of actual expenditures data generated on real time basis through IFMIS. - The MTFF is presented to the cabinet for decision-making on the resource allocation and even policy making.

11. Training of PAAS staff to replace consultants

During transaction roll-out, a heavy reliance on consultants.

- Acceptance of, and direct management and control of, handed over PIFRA accounting sites - Staff to replace consultants. Only audit experts and few supervisors to be

- During transaction roll-out, a heavy reliance on consultants. PAAS playing increasing role but still dependent on

Achieved. - More than 260 officers have been trained from OAGP and OCGA: (a) 40 foreign degree-level trained; (b) 78 IFCE certified; (c) 20 MBA, CISA, ACCA, ACDA; (d) 46 SAP Certified/Academy; (e) 66 training in special sectors (performance auditing, forensic auditing, environmental auditing, information system auditing, energy sector

50

retained until project end. Additional 25 PAAS officers trained on Financial Audi Manual, performance and information system audit.

consultants. - Phased reduction of consultants, reducing total number. PAAS officers trained on FAM, Performance and information system audit.

auditing etc.); (f) 6 ACL master trainers; and (g) 8 FAM master trainers. - The DAGP has adopted INTOSAI guidelines, developed a handbook, and has revised the training approach and modules to replace an outdated curriculum. CISA has developed training modules specific to professional needs of senior audit management and field auditors. - Only audit experts and few supervisors had to be retained since PAAS officers trained on FAM, performance and information system audit.

12. Management skills transfer

Limited skills transfer in transactions roll-out phase of PIFRA II

- Communication Management Plan progress report- - Joint work with MoF, OCGA, OAGP, and PIFRA team -Full takeover of M&E and change management functions by GoP

Regular progress reports on communication were produced. Mainstreaming of M&E and change management functions completed

Achieved. - Management guidelines on each process/activity of system published for all officers/staff levels in OCGA and OAGP. M&E, FABS and Change Management Wings share M&E and change management techniques and methodologies with OCGA and OAGP from time to time and are always available for any further assistance when required by stakeholders. M&E team also visited AG offices and DAOs for knowledge transfer.

Objective 6: Improve accountability (Rating: High) PDO Outcome Baseline Target Actual Comments 13. Effective and timely audit of government accounts using international standards of auditing to ensure reliability of financial and fiscal reporting at all levels of government


- Relevant Auditor General’s staff trained in certification, financial audits, and Audit Management Information System (AMIS) - Second cycle: Quality control framework effectively operational and used for the FY10-11 audit. Information system audit on-job- training for FAO staff. Quality management facility, information system audit

- Quality control framework operational since 2010-11, and a large number of auditors have been trained - Audit quarterly control framework operational since 2010-11. Information system audit established and intense training carried-out.

Achieved - Updated status of training: Over 40 performance audit specialists




- Timely accountability through enhanced number of DAC meetings - Rigorous legislative oversight through PAC proceedings - Significant increase in recoveries based on audit observations - Improved IFMIS and use of CAATs reduced the chances of salary frauds. Ghost employees are eliminated due to a requirement of HR details and direct payment to recipients’ bank accounts

14. Performance audit of policy-linked programs (such as poverty reduction)

Few policy-oriented audits.

- Relevant Auditor General’s staff trained in performance audits

-Development of Performance Audit Manual. Revision and development of guidelines - Trained 30 PAAS officers who have conducted three performance audits with on-

Achieved. - Performance audits were conducted on the following programs: (a) Solid Waste Management Program, (b) Benazir Income Support Program, and (c) Procurement of Wheat by GoSindh. - Performance audits are now in annual audit plan. - During last three years, 19 performance audit reports/special studies have been presented to the legislature. - The Performance Audit Wing of the DAGP

51

the-job training. is being strengthened with additional resources. - Intensive Training Program courses are restructured in line with INTOSAI guidelines.

Objective 7: Improve transparency (Rating: High) PDO Outcome Baseline Target Actual Comments 15. System functionality tracking for improved efficiency and reduced discretion (Bills passed within 3 days (%))

No bill tracking

80%

Aggregate figure for August 2014 stands at 48% passed within 3 days.

Substantially achieved. - Province-wise breakdown: Islamabad, 46%; Punjab, 41%; KP, 51%; Sindh, 39%; and Balochistan, 83%. - Delays owe to the fact that bills are submitted manually and that often documentation received from line departments and ministries is incomplete. - To build pressure of the citizens as well as the officers, a SMS-based query and Dashboard for Bill Tracking reports are also developed.

16. Issue of reliable, timely BERs to PAOs.


100% BERs issued BERs are available at every level of government through the system

Achieved. - Intra-year fiscal reports are presented as PFO reports on the MoF website on a quarterly basis. The PIFRA II team has also developed UPFO tables to report fiscal operations consistent with the Government Financial Statistics Manual 2001 presentation and fully reconciled with PFO. - These quarterly fiscal reports have made possible the timely macro-economic review and analysis by policymakers. - The budget transparency and effective fiscal administration will further improve, and sustenance of the reforms transparency has been improved in recent years as noted in various assessments (PEFA and Open Budget Index). Now the budget documents, which include annual budget statements, statements of appropriation, details of demand and appropriations, MTBF, the budget in brief and budget speech, and others, are available to the general public through the MoF website. - SAP link has already been given to departments, and they have access in the system to generate BERs when required by PAOs. - AGs also send hard copies to them for reconciliation. - System utilization for issuance of BERs has increased more and more as PAOs become aware of and are trained in using SAP. - Timely issuance of pay slips by email

Objective 8: Enhance the capacity of public sector managers to use credible financial information for better and informed decision making (Rating: Substantial) PDO Outcome Baseline Target Actual Comments 17. Adequate numbers of trained staff in MoF and Controller

Very few staff trained in systems operation or new work

Professional training: 20 senior, including operation, staff, 20 CFAOs

Training of 1,045 senior officers and 58 CFAOs.

Achieved (Target Exceeded) - Training programs continuously evolved in the light of feedback and assessment provided by the trainees and trainee organizations.

52

General of Accounts (CGA) to manage and apply system to fiscal and financial management (trained MoF, CGA, and line ministry CFAOs).

practices.

- Over 7,500 staff, representing the OCGA, OAGP, ministries, divisions, and departments at the provincial and federal level, were trained on usage of system. It included senior officers of ministries, divisions, department (i.e., finance managers as termed in the PC 1 - CFAOs; other officers of executive departments; and AGs/District Accounts Offices (DAOs). - In addition, tens of thousands of Drawing and Disbursing Officers (DDOs) were trained on the new chart of accounts to enable appropriate classification of expenditures as a building block of the financial data.

18. Effectively tracked and managed poverty reducing spending (timely reliable PRSP reports)

All Poverty Reduction Strategy Paper (PRSP) reports prepared manually with no data assurance

-Reliable Monthly Budget Execution Reports available within four weeks of the end of each month -Accurate, timely and comprehensive financial statements used for decision-making by Principal Accounting Officers (PAOs) in line departments and MoF - System generated PRSP reports; PETS established

- Reliable monthly budget execution/PRSP reports are generated from the system; -PETS established; -Accurate, timely and comprehensive financial statements are used for decision-making by PAOs and MoF

Achieved. - FABS was simultaneously implemented at all the three levels of government with consistently applied chart of accounts and forms and procedures in all parts of the country to capture pro-poor expenditures for education, health, water and sanitation etc. - FABS is capable reliably track expenditures of federal, provincial and districts tiers and consolidated PRSP reports are quarterly produced - PRSP reports are publicly available on website. - Staff at PRSP Secretariat have been trained and are able to utilize the system to generate reports. - Despite radical changes due to local government reforms in 2001 to post 18th amendment reforms since 2010, the system is effective in tracking pro-poor expenditures incurred at grass root level. - FABS data facilitates Public Expenditure Review (PER) and various studies/analysis including PETS and Expenditure and Quality of Service Delivery Survey to help in identifying sources of inefficiencies and bottlenecks in the flow of funds and to recommend remedial actions. - Continuous efforts are underway to make system even better and more user friendly for reporting. - BERs are used by executives to make expenditure decisions on the basis available budget. - Annual financial statements are also used by PAOs and MOF, for example, as a historical base for budget preparation etc. - MoF actively using FABS data for cash management and financial forecasting - Quarterly and annual financial statements of externally-financed projects submitted to donors - The comparative data by districts, entity or other such classification could be a huge support to a performance management framework, but such analysis and use of data can only be enhanced by progressively

53

providing further support to the executives and policy makers.

Objective 9: Facilitate oversight of the use of public monies (Rating: High) PDO Outcome Baseline Target Actual Comments 19. Effective and timely audit of government accounts using international standards of auditing (Application of modern audit practices)


- Relevant Auditor General’s staff trained in certification, financial audits - Quality control framework effectively operational and used for the FY10-11 audit. Information system audit on-job- training for FAO staff. - Quality management facility, information system audit incorporated into regularity audit practices.

Quality control framework operational since 2010-11, and a large number of auditors have been trained

Achieved. - Use of IFMIS, and timely and effective audit has prevented misuse of public monies through enhanced oversight - Quarterly control framework operational since 2010-11. Information system audit established and intense training carried-out. Update: Over 40 performance audit specialists




Objective 10: Increase the national and international credibility of Government financial statements and assurance processes (Rating: Substantial) PDO Outcome Baseline Target Actual Comments 20. Achieve comprehensive data entry to FABS (foreign-financed projects)

No coverage or incomplete information of projects, SAEs, foreign-financed and MoF transactions

100% - Data is collected by the AG on a regular basis for projects and SAEs not on FABS

Achieved. - 17 foreign financed projects directly on FABS with real time reporting - Increased assurance due to high quality in-year and year-end government reporting has enabled the development partners to enhance the use of budget support and results-based funding modalities in Pakistan

54

Annex 11. Case Studies on Successful PFM Changes under PIFRA II How did change happen? Analysis shows that demand-driven changes were triggered when an external influence met with internal capacity desire to change. After this triggering, the internal change agents had to reach out to other relevant institutions for building a coalition for change and push through a regulatory reform to enable change implementation. Finally, the implementation of change itself had its own dynamics with often a spread-out population benefiting or implementing that change. The case studies showed that while there is some reform space to promote such changes or improvements, these changes actually took years to get to some level of success given the lack of adequate incentives and suitable enablers to harness such changes.

Change aspect

Direct credit scheme (DCS) for pensioners

Use of country system for foreign-financed projects

SMS-based query PIFRA – A platform for DSS e-Transmittal of salary slips Pension system for Pakistan railways

Issues Changing behaviors Coalition to manage the politics of reforms

Dedicated implementation team

High-level support Dedicated implementation team

Setting up a system

Reform challenge

How to facilitate pensioners? How to enhance financial management system harmonization?

How to improve system functionality for improved efficiency and reduced discretion

How to use information in PIFRA to support decision-making?

Employees were not receiving the salary slips due to shortage of paper

How to facilitate pensioners in Pakistan railways?

How the reform initiated

A case was filed against Sindh Education Department in Supreme Court of Pakistan in 2006, when decomposed bodies of a retired Professor and his wife were found after 15 days. The couple died due to starvation because of the delay in the payment of pension and other dues.

The demand was generated following the commitment made under Paris Declaration in 2005, for the existence of adequate country financial management systems and their effective use for channeling aid.

The reform was initiated to improve the efficiency and accountability of AG offices. One of the project intermediate outcome indicators was to reduce the bill clearing time to 3 days. However for various reasons, that include processing modalities and vested interests, the bill clearing time takes more than 10 days. Therefore to build the pressure of the citizens as well as the officers, a SMS based query and Dashboard for Bill Tracking reports was planned.

On the direction of the Chief Minister, Punjab, a Public Affairs Unit (PAU) has been established in early 2014 to update Chief Minister regarding activities going on in the province. In July 2014, on the Chief Minister’s direction, PIFRA connectivity was established for the Chief Minister’s office (CMO). CM instructed to use PIFRA platform to develop a decision oriented reporting system to monitor development projects and ongoing activities.

Inability to print salary slips for employees was the common shortcoming noted in most of the DAO’s offices during the survey conducted by the M&E team of the project. The complaint was shared with FABS team and the DG FABS who was an experienced Government servant developed a solution with his team for the e-transmittal of the salary slips.

The Federal Railways Minister invited various Banks for a solution for pension disbursement. Project Team (Project Director and DG-FABS) overheard it and contacted Federal Minister Pakistan Railways in Oct. 2013 with a free offer to provide PIFRA’s DCS solution. Later an order was also made by the Federal Ombudsman’s office in April 2014 which expedited the implementation.

Timeframe Outputs Outcomes

The solution was launched in 2006/07 and matured in 2011. Output: Pensions are now directly credited in the pensioners’ bank accounts. Outcome: Improved service delivery and corruption reducing measures.

The process was initiated in 2007 and matured in 2010 Output: SAP terminals at the project implementing entities, with enhanced functionality specific to the project. Outcome: Improve accuracy, comprehensiveness and reliability of the financial statements.

The solution was conceptualized in January 2014. Its technical parameters were finalized by March 2014. Solution was tested in May 2014 and launched in June 2014.

It took less than six months to develop a functional reporting system.

Output: A decision oriented Reporting System was developed and link was provided to Chairman P&D, the Secretaries of the Departments, the Commissioners, DCOs and Finance Department. A dashboard for the Chief Minister

HR related all reports were restructured and a plan was developed before system upgrade to ECC6 in early 2014. Service was first launched in August 2014. Output: An e-transmittal solution for the salary slips, payroll register and bank advices. Outcome: Improved service delivery

PIFRA initially contacted PR in Oct. 2013. Solution was put in use in January 2014. Output: Pension automation and SAP installation in PR. Outcome: Improved service delivery and corruption reducing measures.

55

is also developed.

Outcome: It enhances the capacity of the public sector managers to use credible financial information for better and informed decision making.

Senior leader ~1

How did you manage the politics of the reform?

Citizen pressure and a Supreme Court directive created an opportunity for change. A pension task force was constituted with representation from Accountant General Pakistan Revenues, Controller General Accounts, Finance Division and SBP to prepare and implement Standard Operating Procedures. CGA/PIFRA and SBP were monitoring the progress of Account offices and Banks respectively.

Federal Government initiative. A working group was formed with representations from World Bank, MoF, EAD, SBP, CGA and PIFRA. Working group was jointly led by the Government of Pakistan and the World Bank.

As the vested group was not directly part of the implementation process, therefore minimal to no resistance was faced.

A working group was formed led by Special Secretary to the Chief Minister, which includes the stakeholders’ including PIFRA’s representative, Planning and Development (P&D) Department, Finance Department and the technology team in the Chief Minister’s Office. The approach and benefits for developing such Decision Support System was communicated to the working group members as well as the stakeholders in the Regular meetings.

It was a demand driven process and faced minimal to no resistance

Citizen pressure, high level commitment and a decision from Federal Ombudsman created an opportunity for change. Federal Ombudsman’s order facilitated the initiative taken by PIFRA,

Implementers 2-3

What elements made your reform successful?

Staff were protected. All stakeholders were on board. General public was the direct beneficiary of the reform.

Demand-driven. All stakeholders were on board. Regular working group meetings. Clear division of responsibilities and follow-up on action items. Commitment from MoF.

Demand-driven Easy access to information without any human interaction.

Demand was raised from the Chief Minister Punjab. Task was led by Chief Minister Office. Instructions issued from the Chief Minister for the use of Decision Oriented Reporting System.

Demand driven. The reform was the need of the time.

Demand-driven and personal interest from Federal Minister, Pakistan Railways. Active role by FABS team and availability of the ready solution already functioning in Federal/provincial ministries.

Which implementation challenges did you need to address?

Task force revised old and outdated pension rules and Procedures to facilitate direct credit process.

To curtail rent-seeking activities, following actions were taken by the PIFRA;

- conducted business ethics workshops. - Monthly monitoring reports are submitted to Deputy Accountant General to review the progress on pension cases.

To curtail rent-seeking activities, following actions were taken by the Government;

Revised accounting procedures for establishment and operation of the revolving fund accounts by CGA. To remove any anomalies Donor / PIFRA coordinated on regular basis to ensure that the new government classification system align with the requirements of the donor classification of funds disbursed. PIFRA developed standardized reporting template meeting

Finalization of the technical parameters was the only challenge that was faced during the development of the solution.

PIFRA earlier made multiple unsuccessful attempts to bring P&D and Finance Department (FD) at a point of agreement to develop a link between Financial and Project Data. Later, the Chief Minister Office took a lead, and after series of meetings with P&D and Finance Department a successful application was developed where general serial no. (ADP No.), were mapped with project codes in FD.

Initially Staff and account offices faced difficulty in gathering the email addresses of employees. However, the product soon picked the momentum and till now 65,000 applications are received to enroll in the e-transmittal facility. Low paid non gazetted and far off located employees do not use ICT. Program for SMS based alerts is under preparation to overcome this challenge.

The implementation challenges for the reform were minimal which resulted in less than six months’ time for the complete automation. The factors that contributed in minimizing the challenges were; Provision in the law due to earlier reforms in the Federal and Provincial Ministry. Experienced team. Commitment of the

56

Federal and Provincial Government of KP and Punjab showed their commitment to the reform process and made it mandatory for all the existing retirees to process their cases through DCS.

In four years the total number of pensioners benefited from the DCS is 120k, whereas total number of pensioners in the country is 1.2million.

Donors Requirements. In four years 18 projects, largely the World Bank’s projects are using the country system. The total number of donor funded projects is above 100.

leadership. Till the end of October approx.7000 pensioners were paid from the system, whereas total number of pensioners in the Pakistan Railways is approximately 150k. Slow implementation progress was due to resistance from the IT department in Pakistan Railways.

How did you collaborate across units within your agency or across multiple agencies?

Live demos to family Pensioners Associations and users. Regular meetings were held for the implementation of the SOPs.

Regular coordination meetings were held between Donors and PIFRA. Further frequent Working group meetings led by the Bank helped in addressing the implementation challenges, such as revision of accounting procedures etc.

Solution development required collaboration of FABS team with the service providers. Regular working group meetings were held with the service provider led by FABS team. FABS team developed system based bill data collection program, where system auto collects the bill status. Once data is collected it is sent to Cloud Based PTCL server by SAP through auto generated email, twice a day.

Chief Minister Office held regular meetings of the working group and disseminated the benefits among stakeholders for using the decision support system.

Collaboration with the CGA office was required at the time to launch the service. Presentation with the test email was made to the CGA.

PIFRA’s commitment was limited to Solution Development, pilot Implementation at Lahore, Punjab and technical support. PIFRA trained 20 staff members of Pakistan Railways twice who later led the automation process.

Beneficiaries 1-2

How was communication during the reform period?

Dissemination campaign was launched by PIFRA and AG through flyers. panaflex, banner and newspapers. Presentations were made to the Senior executives and PAOs.

Donors encourage use of country system by making it mandatory in the Development Credit Agreement. (such as Multi Donor Trust Funds and USAID projects)

Dissemination campaign launched by FABS and AGs through: jurisdiction specific single page flyers Mass email messages Mass SMSs to different stakeholders Ads in the news papers

It was communicated to the members of the Public Affairs Unit in the regular meetings.

Communication was made through Facebook walls. Live demos were made during training by FABS. Emails to different stakeholders who spread it through word of mouth.

In November 2014, the Pakistan Railways administration has advised all pensioners to open account in the nearest bank account to benefit from DCS.

What is the most valuable thing to come from the reform?

Pensioners’ satisfaction. Identification of ghost pensioners saving millions of rupees. On Government exchequer. Correct estimation of liabilities.

Improved international credibility of Government’s Financial Statement. Savings in the cost of designing, implementing, and monitoring project specific systems for every single project.

Drawing and Disbursing Officers and the individual Government employees whose bills are processed by Account offices can know the status of the bill through SMS query and response. More than 7000 SMS were received in 4 months’ time from Jun2014-Sept2014.

Smart Reports for the decision makers. Sensitization for fair and accurate reporting.

Beneficiary’s satisfaction. Saving to Government on account of Heavy Duty printers (Each costed PKR. 22 lac and 150 account offices each needing 1 to 2). Huge saving on account of stationary (printing of payroll registers and salary slip etc.) and printers’ repair and maintenance

Pensioners’ satisfaction. Saved PKR 10 million for the purchase of new system. Identification of ghost pensioners saving millions of rupees on Government exchequer.

57

Annex 12. PIFRA II Contribution to Poverty Reducing Outcomes

(a) Poverty Impacts

Timely and reliable Poverty Reduction Strategy Paper (PRSP) reporting, facilitating expenditure tracking of sectors, and improved audit of service delivery were incorporated as one of the project outcome indicators for PIFRA II Additional Financing. FABS supported the compilation of PRSP reports by providing reliable data. PRSP reporting also facilitated audits resulting in poverty reduction programs now being subject to performance audits. Improved reliability of data combined with assurance of effectiveness of spending is progressively enhancing the credibility of poverty reduction efforts. In addition, FABS data also facilitates Public Expenditure Reviews (PER). The PER objective is to analyze the overall budgetary, fiscal, and financial management system of the national/subnational government and to assess the adequacy and efficiency of expenditure for social services.

Strengthened PFM systems introduced under PIFRA II ensured that money would be used for the intended purposes in line with the public policy objective. Consequently, services would be delivered more efficiently and economically, which in turn should support both effective employment of national resources and faster progress toward the Millennium Development Goals. While there are many other factors at play, the results of PFM improvements will contribute to increasing access to social services and reducing poverty through growth-led as well as targeted programs. Furthermore, strengthened PFM institutions and systems would encourage transparency and accountability in budget execution. Accountability, in turn, reduces corruption and creates merit-based incentives for accelerated and effective delivery of services. In addition, transparent financial reporting, which leads to informed decision-making, economic viability, and credibility of the state, provides a conducive environment to attract direct and indirect private investment. Consequently, investment-led economic growth will take poverty reduction forward. (b) Gender aspects and social development

Because of the project, pensions are now directly credited to the pensioners’ bank accounts. This has resulted in improved service delivery and reducing corruption as well as providing a convenient facility for women who do not wish to go out to collect their pension amounts. The project was also instrumental in identifying ghost pensioners, which led to government savings of millions of rupees.

(c) Institutional Change/Strengthening

The project resulted in substantial institutional development impact in the following areas (more details are in subsection 3.2):

The single biggest institutional reform undertaken under PIFRA II was the separation of accounts and audits. This reform not only removed the obvious conflict of interest between accounts and their audits, but also established the institution of CGA to manage all accounting activity in Pakistan. The impact of this institutional reform in terms of strengthening the PFM system is significant.

As a result of PIFRA II, many government officials were able to receive local and foreign training. This resulted in enhanced skill sets and the potential of more acceptance for future reforms. The officials of the CGA and AGP are better informed of international good practice

58

approaches and would be willing to steer institutional changes within their departments. The quality of financial reporting and audit has also improved because of the personnel training.

A quality management framework for public sector audit was developed and implemented under PIFRA II. The framework prescribes quality controls to be applied at all audit stages. At the end of the audit process, an independent review is carried out improving the quality of the audit process and reports.

CISA was established within AGP to provide training and on-the-job support in the area of information systems audit. The CISA has trained more than 100 auditors in information system audits and will continue to support AGP in this emerging area.

PIFRA II has become the prime resource of trained professionals for PFM reforms in Pakistan. Almost all PFM projects in Pakistan today are being led by professionals who were first exposed to PFM reforms through PIFRA II. These professionals are engaged in PFM reforms at different levels and are aware of the centrality of PIFRA II.

(d) Other unintended outcomes and impacts (positive or negative)

Some other positive, yet unintended, outcomes listed below:

The introduction of school-based budgeting in Sindh province allowed allocation and availability of resources to primary service delivery units. FABS was the only possibility to work with the thousands of schools in the system.

Although a phased approach for roll-over of critical reforms was not envisaged at the project design stage, the program adopted a phased implementation approach, beginning with pilot sites, followed by roll-out and replication sites. The pilot and roll-out sites were identified and implemented during PIFRA I, which set the stage for PIFRA II and replication sites across the budget execution centers. Unlike many PFM reform programs aimed at achieving critical reforms at one go but too little effect, the phased approach in PIFRA II had contributed to the successful implementation of SAP.

There have been some jurisdiction differences between the AGP and CGA over discharge of responsibility for recruitment, staff deployment, and development of career path due to separation of audit and accounts functions, but overall these led to strengthening of the CGA office.

59

Map