Snapshot Tool for Implementation of ACSS Code of Practice · format June-July 2015: training session for AMS and ASEANstats on the use of self-assessment tool July-October 2015: country

Snapshot Tool for Implementation of ACSS Code of Practice

Self-Assessment Measures

Presentation by ASEAN Secretariat

Seminar on "Strengthening national statistical systems to meet the challenges

of the UN SDGs"

Bangkok, Thailand, 13-14 December 2016

Outline of Presentation

o Background: ASEAN Community Statistical System (ACSS) Code of Practice (CoP)

o Snapshot Tool o Pilot Exercise o Result of the Pilot Exercise o Conclusions

2

Background

The ACSS Code of Practice (CoP) was adopted at the 2nd Session of the ASEAN Committee Statistical System (ACSS) Committee in Sep 2012

The ACSS CoP is consistent with the Fundamental

Principles of Official Statistics adopted by the UN Statistical Commission in 1994, and reflects the values stipulated in the ACSS Strategic Plan

The 2nd ACSS Committee agreed to implement the 8

principles in the CoP and to conduct self-assessments based on the indicators in the CoP

3

Background These principles are subsumed in 3 major dimensions:

A. Institutional Environment

1. Mandate for data collection 2. Professionalism and integrity 3. Confidentiality 4. Accountability 5. Statistical Cooperation and Coordination

B. Statistical Process 6. Cost Effectiveness 7. Reduced Respondent Burden

C. Statistical Output 8. Commitment to Quality (Relevance, Reliability, Timeliness, Comparability,

Accessibility)

Emphasised that Quality Monitoring Tool (Snapshot) should be in line with the ACSS CoP and assessment measures agreed by the ACSS Committee (for Section A, B and C)

4

The Snapshot Tool

The ‘Snapshot tool’ quality framework was developed by the European Statistical System to assess key aspects of the national statistical systems (institutional environment); as well as in different sectors, specific sector statistics and key indicators for these sectors (statistical output) by asking a series of clear questions with pre-defined answers using Microsoft Excel®.

5

The Snapshot Tool

How assessment is done The pre-defined answers automatically give an intuitive

colour code for good (green), average (orange) and poor (red) achievement for the issue concerned.

Not complying with quality requirements

Partially complying with quality requirements

Fully complying with quality requirements

The Snapshot Tool

How assessment is done This is complemented by an arrow indicating a positive,

unchanged or negative on-going or expected development.

This approach provides an intuitive and easy-to-interpret assessment of the national statistical system or of sector statistics, which is visible at first glance.

The Snapshot Tool

The tool consists of two modules: Module 1: assesses the functioning of the national

statistical system (NSS) as a whole, with a special focus on institutional, legal and organizational aspects, (to be

implemented to CoP Section A). Module 2: assesses the performance of the statistical

system in a specific sector (such as trade in goods, foreign

direct investment, trade in services,…). In this module, there is a special complement to assess the quality of specific sector indicators, (to be implemented to CoP Section B

and C).

8

The Exercise

Purpose of the exercise To test the questionnaire

o this exercise is to test the appropriateness of the self-assessment questionnaire on the implementation of the code of practice for use as an input to the quality report.

To prepare an ASEAN quality report o the ASEAN quality report is to collect in one single place all kind of

information relating to the quality of national statistics but also to present the quality of aggregated estimates at the ASEAN level.

The Exercise

Schedule of Pilot Implementation of the CoP Section A, B and C self assessment using the Snapshot Tool: January 2015: Workshop on Advancing Quality Monitoring April-May 2015: finalization of the self-assessment tool using the snapshot format June-July 2015: training session for AMS and ASEANstats on the use of self-assessment tool July-October 2015: country self-assessments November 2015: Reporting result on assessment to the Committee End of 2015: Regional exchange and discussion on the results of the self-assessments and on the revision of the tool

Jan-Oct 2016 : Pilot exercise for CoP Section B and C (discussed during the Working Group meeting)

10

Result: AMS’ Self-Assessment of Section A Using Snapshot Tool

All 10 National Statistical Office of the Member States participated in the pilot exercise (Thank you, AMSs!!!)

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

KP1 KP2 KP2 KP4 KP5

In general, very good application of the 5 Key Principles of the ACSS CoP at the regional level

Result: AMS’ Self-Assessment of Section A Using Snapshot Tool

DETAILED RESULTS PER KEY PRINCIPLE (EXAMPLE KP 1)

KEY PRINCIPLE 1: MANDATE FOR DATA COLLECTION

88%

12%

1.1 THE MANDATE OF THE NATIONAL STATISTICAL AUTHORITIES OF THE AMSS TO COLLECT INFORMATION FOR THE

PRODUCTION AND DISSEMINATION OF OFFICIAL STATISTICS IS STIPULATED IN NATIONAL LAWS AND/OR NATIONAL LEGAL

FRAMEWORKS.

100%

1.1.1 IS THE MANDATE TO COLLECT INFORMATION FOR THE PRODUCTION AND

DISSEMINATION OF OFFICIAL STATISTICS STIPULATED IN AMS’ NATIONAL LAWS AND/

OR NATIONAL LEGAL FRAMEWORKS?

100%

1.1.2 IS THE OBLIGATION TO FURNISH INFORMATION REQUISITIONED BY THE

NATIONAL STATISTICAL OFFICE STIPULATED IN THE AMS’ NATIONAL LAWS AND/ OR

NATIONAL LEGAL FRAMEWORKS?

100%

1.2 OFFICIAL STATISTICS PRODUCED BY NATIONAL STATISTICAL AUTHORITIES OF THE AMSS ARE PROVIDED TO THE

ASEANSTATS FOR THE COMPILATION OF ASEAN STATISTICS, UNDER THE COORDINATION OF THE NATIONAL

STATISTICAL OFFICES.

80%

20% 1.2.1 HAS THE NATIONAL STATISTICAL OFFICE APPOINTED A COUNTRY FOCAL POINT TO

COORDINATE THE PROVISION OF OFFICIAL STATISTICS TO ASEANSTATS?

100%

1.2.2 ARE ACSS STATISTICAL WORK PROGRAMMES AND ACTIVITIES INCORPORATED IN THE

AMS’ NATIONAL STATISTICAL PROGRAMMES AND ACTIVITIES, WHERE RELEVANT?

70%

30% 1.2.3 ARE THERE AGREED DATA TEMPLATE/ PROTOCOL AND DATA DISCLOSURE GUIDELINES/

POLICIES TO HELP THE NATIONAL STATISTICAL OFFICE COORDINATE THE COMPILATION AND

PROVISION OF OFFICIAL STATISTICS TO ASEANSTATS?

70%

30%

Result: SITS: Assessment Measures Sections B (Statistical Process) and C (Statistical Output)

Detailed questions

Key Principle 6: Cost Effectiveness and Adequacy of Resources 60 40 0 0

6.1 Procedures exist to optimise resources utilised in the production and dissemination of ASEAN statistics.

70 30 0 0

6.1.1 Are costing and budgeting procedures in place in AMS and ASEANstats to optimise resources utilised?

70 30

6.3 Productivity potential of information and communications technology is being optimised for data collection, production and dissemination.

50 50 0

6.3.1 Are sufficient resources allocated by AMS and ASEANstats to exploit the full potential of information and communications technology for data collection, production and dissemination?

50 50

6.4 Staff, financial, and technological resources are available both in magnitude and in quality to meet the current needs for ASEAN statistics.

60 40 0 0

6.4.1 Are adequate staff, financial, and technological resources available in AMS and ASEANstats to meet the current needs for ASEAN statistics?

60 40


Key Principle 8: Commitment to Quality 67 22 3 8

8.3 Timeliness - A release calendar is established for ASEAN statistics at both national and ASEAN levels, and the periodicity of statistics released takes into account user requirements as much as possible.

80 15 0 5

8.3.1 Is an advanced release calendar established by AMS for national statistics and ASEANstats for ASEAN statistics?

80 10 10

8.3.2 Do the periodicity and timeliness of statistics released by AMS for national statistics and ASEANstats for ASEAN statistics follow dissemination standards?

80 20

8.4 Comparability - ASEAN statistics are comparable, and detailed concordance as set by international.

55 25 0 20

8.4.1 Are ASEAN statistics consistent or reconcilable over a reasonable period of time?

40 20 40

8.4.2 Is detailed concordance available as set by international standards between AMS’ national classification systems and the corresponding ASEAN classification systems?

70 30


8.5 Accessibility - ASEAN statistics are produced and made readily available through dissemination mechanisms utilising modern information and communication technology, and where appropriate, traditional hard copy.

72 18 4 6

8.5.1 Are statistics disseminated by AMS and ASEANstats in forms that meet users’ needs, utilising appropriate information and communication technology?

70 30

8.5.2 Are unpublished statistics that can be disseminated made available by AMS and ASEANstats to users upon request?

50 40 10

8.5.3 Are contact points for AMS and ASEAN statistical releases clearly identified? 90 10

8.5.4 Are updated catalogues of publications, documents and other services to users maintained by AMS and ASEANstats?

70 10 10 10

8.5.5 Are prices of the statistical products and services, where appropriate, clearly disclosed by AMS?

80 10 10

Detailed questions ASEAN

Key Principle 8: Commitment to Quality

8.1 Relevance - ACSS is guided by the statistical requirements reflected in the ASEAN Community Blueprints and user-producer consultations.

8.1.1 Is ASEANstats able to identify major data users and conduct periodical user consultation/ feedback on the statistical requirements reflected in the ASEAN Community Blueprints?

↗

8.1.2 Does ASEANstats regularly update ACSS Committee on ASEAN regional policy issues and initiatives?

↗

8.1.3 Are the requirements prioritised by the ACSS Committee and appropriately incorporated in the ACSS annual work plans and multi-year action plans?

↗

8.1.4 Is user feedback sought on ASEAN statistics disseminated by ASEANstats? →





8.2 Reliability - Monitoring mechanisms are in place to regularly review and improve the quality of the statistical processes and products.

8.2.1 Are periodic reviews of statistical activities conducted by AMS and ASEANstats to improve quality of statistical processes and products?

↗

8.2.6 Is a regular and transparent schedule established by AMS and ASEANstats for data revisions?

↗

8.2.7 Are preliminary and / or revised data clearly identified by AMS and ASEANstats? →

8.2.8 Are revision studies undertaken by AMS and ASEANstats on a regular basis? →




8.5 Accessibility - ASEAN statistics are produced and made readily available through dissemination mechanisms utilising modern information and communication technology, and where appropriate, traditional hard copy.

8.5.1 Are statistics disseminated by AMS and ASEANstats in forms that meet users’ needs, utilising appropriate information and communication technology?

↗

8.5.2 Are unpublished statistics that can be disseminated made available by AMS and ASEANstats to users upon request?

→

8.5.3 Are contact points for AMS and ASEAN statistical releases clearly identified? →

8.5.4 Are updated catalogues of publications, documents and other services to users maintained by AMS and ASEANstats?

→

Conclusions

Findings from the exercise Snapshot tool is a useful tool, simple, user friendly Easy to understand results (in the future) Initial framework for peer review Observations to improve the self-assessment questionnaire Need to reformulate the questions in order to get useful and

appropriate comments Build as far as possible quantitative indicators Add quality indicators: i.e completeness (information provided by

AMS compared with relevant ASEAN guidelines: calculation of indicators

of completeness); reliability (validation checks at the national and

ASEANstat level), timeliness …

Participative process: in national as well as regional (some of the

questions need to be answered together by AMS and by ASEANstats).

THANK YOU

Documents

Snapshot Tool for Implementation of ACSS Code of Practice · format June-July 2015: training session for AMS and ASEANstats on the use of self-assessment tool July-October 2015: country