HIGH OUTPUT QUALITY MONITORING VIA THE ECB QUALITY REPORTS ON STATISTICS

HIGH OUTPUT QUALITY MONITORING VIA THE ECB QUALITY REPORTS ON STATISTICS

Pedro Abad

Banco de España

European Conference on Quality in Official Statistics

Viena, 2-5/06/2014

STATISTICS DEPARTMENT 2

HIGH OUTPUT QUALITY MONITORING VIA THE ECB QUALITY REPORTS ON STATISTICS

1. Introduction. Main elements of the Statistics Quality Framework: towards high-quality output

2. Quality monitoring and reporting on financial

accounts

Quantitative quality indicators

3. Output enhancement from the ECB quality monitoring reports: the experience of Banco de España

4. Conclusions


1. MAIN ELEMENTS OF THE STATISTICAL QUALITY FRAMEWORK: TOWARDS HIGH-QUALITY OUTPUT

The ECB statistical system rests on three pillars:

the regulations setting obligations on economic agents

the guidelines given to NCBs detailing the datasets to be transmitted and other statistical requirements, and

the quality framework

The ESCB Public Commitment on European Statistics set the principles that govern the collection, compilation and dissemination of statistics by the ECB.

The ECB Statistics Quality Framework combines these principles and others of good governance with the quality assurance procedure developed by the ECB.

Quality reports are at the core of the system as they review main quality elements and draw some conclusions/recommendations in order to enhance statistics.



Quality is defined in relation to how well statistics are fit for purpose, and, therefore, meet users´ and other stakeholders´ expectations. This is a multidimensional concept.

Guiding principles from the Public Commitment on European Statistics by the ESCB

Institutional environment Statistical processes High output quality

1 Professional independence 7 Sound methodology 11 Relevance

2 Mandate for data collection 8 Appropriate statistical procedures

12 Accuracy and reliability (including stability)

3 Adequacy of resources 9 Minimisation of the reporting burden

13 Timeliness (including punctuality)

4 Commitment to quality 10 Cost-effectiveness14 Consistency and comparability

5 Statistical confidentiality 15 Accessibility and clarity

6 Impartiality and objectivity



Data Quality Framework: High Output Quality

Principles Quality assurance procedures Principles Quality assurance procedures

Relevance

Medium-term strategy / annual work programmes

Merit and cost exercises Seminars and consultations

with users

Consistency and comparability

Reporting schemes follow international accounting standards and best practices

Regular internal quality monitoring reports

Accuracy and reliability (including stability)

Reporting schemes follow international accounting standards and best practices

Completeness and plausibility checks

Revision analysis and consistency analysis between different datasets

Regular internal quality monitoring reports

Accessibility and clarity

Intensive use of up-to-date electronic system for disseminating data

Press releases, regular publications

Data warehouse, joint country-tables

International standards in the denomination of variables

On-line documentation

Timeliness (including punctuality)

Transmission calendars Release calendars Compliance with IMF SDDS

standards


2. QUALITY MONITORING AND REPORTING ON FINANCIAL ACCOUNTS – QUALITY REPORTS

The main two procedures implemented by the SQF to report on quality to the ECB senior

management are: The audit reports prepared by the Internal Auditors Committee (IAC) Quality reports prepared by the Statistics Committee (STC)

Quality reports prepared by the STC review main quality elements with the help of some

usually pre-defined indicators and draw some conclusions and/or recommendations in order to

enhance the quality of statistics

Several ECB statistical guidelines require the preparation of quality reports, mainly focused on

the compilation of euro aggregates: Balance of Payments and International Investment Position Statistics (biannual), Government Finance Statistics (internal), Monetary and Financial Statistics, Centralised Securities Database (internal) Financial Accounts.

All these reports are based on a common framework, although they are tailored to each

specific statistical domain and so their content is not fully homogenous.

The preparation of these reports by the ECB with the assistance of the expert groups can be

seen as the final step of the quality monitoring process, which also includes other internal

quality reports.


2. QUALITY MONITORING AND REPORTING ON FINANCIAL ACCOUNTS

ECB compiles euro area financial accounts on the basis of:

national financial accounts transmitted under Guideline ECB/2013/24, and

datasets already available in the ECB data warehouse from other ECB

guidelines. The most important are: banking statistics, securities statistics,

government finance, and external statistics.

ECB Guideline requires the transmission of:

stocks, transactions, other flows and counterpart information for selected items

information on major events and revisions

the preparation of an annual quality report including at least: coverage,

compliance with relevant definitions and revisions

The quality monitoring process carried out by the ECB with the assistance of the expert group on financial accounts also includes several internal quality reports.



Table 2. Coverage of quality monitoring reports on financial accounts

Periodicity OrientationProduction

Collection Compilation Dissemination

Internal

quality monitoring reports

Feedback country report Quarterly Compilers X

Regular transmission report (EA level, information by country)

Quarterly Compilers X X

Integration report (EA level) Quarterly Compilers/

internal users X

Methodological notes (EA level) Ad-hoc Compilers/

internal users X

Quality report(EA level, public) Annual Internal users/

public X X X



It is inspired by the basic principles of the SQF Public (since 2013) Coverage:

Institutional and legal framework Quality assurance procedures and user involvement Coverage Revision analysis Consistency and comparability Timeliness of national contributions Data accessibility Recent developments NCB-specific assessment, recommendations and non-

compliance cases (only internal version)Comprises a set of quantitative indicators at EA level

EURO AREA QUARTERLY FINANCIAL ACCOUNTS REPORT


2.1 QUANTITATIVE QUALITY INDICATORS

Revision analysis

Errors or revisions are defined as the difference between the current values of a variable (as transactions) and the values when they were released for the first time. The quality report presents four indicators to assess accuracy and reliability.

Positive revisions (R) as % of positive revisions in relation to total observations

Directional reliability (Q) as % of first release of data with the same sign as the last release

MACE [Mean Absolute Comparable Errors] which is the mean of absolute errors on transactions in relation to stocks, and

RMSRE [Root Mean Square Relative Errors] which is a relative measure of errors in relation to the variability series

Consistency analysis

Statistical discrepancies arise in the compilation process due to the integration of different data sources. The quality report assesses discrepancies in financial instruments (horizontal inconsistencies), and between the capital and the financial accounts (vertical inconsistencies) by means of the RMSE indicator.

RMSE [Root Mean Square Errors]


2.1 QUANTITATIVE QUALITY INDICATORS

Euro area quality reliability indicators

for households


3. OUTPUT ENHANCEMENT FROM THE ECB QUALITY REPORTS: THE EXPERIENCE OF THE BANCO DE ESPAÑA

The BdE contribution is fully compliant with the ECB data requirements

The BdE takes note of the conclusions and recommendations to enhance, as far as possible, the Spanish financial accounts and therefore the national contributions to the ECB

In recent years, changes have been introduced in several areas following quality monitoring reports: to reduce inconsistencies with securities issues statistics due to the

treatment of own securities used as collateral at the ECB to reduce inconsistencies with money and banking statistics due to

the treatment of factoring operations related to general government transactions

to harmonise the treatment of repos transactions across euro area countries

other discrepancies between data sources are regularly monitored Discrepancies between net lending/borrowing and net financial

transactions of non-financial corporations and households are monitored by the BdE and the INE


3. OUTPUT ENHANCEMENT FROM THE ECB QUALITY REPORTS: THE EXPERIENCE OF THE BANCO DE ESPAÑA

Spain. Discrepancies between net lending/borrowing and net financial

transactions


4. CONCLUSIONS

Monitoring and reporting on quality is an essential element of the Statistics Quality Framework.

Several ECB Guidelines require the preparation of annual quality reports, which to some extent are the final step of the quality monitoring process carried out by the ECB with the assistance of the expert groups.

By means of ECB quality reports the Statistics Committee can draw the attention of the highest decision-making bodies of the ECB to relevant quality problems on euro area statistics, and related national contributions, so that the ECB and national central banks can resolve underlying problems.

The quality report on financial accounts is an example of this strategy. This and other internal quality monitoring reports cover the compliance by countries with the ECB regulations and other quality aspects, in particular revisions and consistency with other statistics, which are examined on the basis of a set of quantitative indicators.

Finally, the paper provides some information about the changes introduced in the Spanish financial accounts following the conclusions and recommendations of the ECB quality reports.

STATISTICS DEPARTMENT

THANK YOU FOR YOUR ATTENTIONPedro Abad

Documents

HIGH OUTPUT QUALITY MONITORING VIA THE ECB QUALITY REPORTS ON STATISTICS