16.9.2014 Olli-Pekka Penttilä Director of Statistics / Co-ordinator Finnish Customs Nordic meeting 2014 Preparation for exchange of micro-data on intra-EU

16.9.2014Olli-Pekka Penttilä

Director of Statistics / Co-ordinator Finnish Customs

Nordic meeting 2014

Preparation for exchange of micro-data on intra-EU trade between Member States

SIMSTAT-Pilot ESSnet

216.9.2014 Nordic meeting

Outline of SIMSTAT: • Short background• Task of the ESSnet1 (Pilot)• Work plan• Organisation of the work• Next steps• Discussion

316.9.2014 Nordic meeting

Background and history (1)

• Intrastat began operating 1 Jan 1993• most burdensome area of business statistics• in addition, perceived burden (by PSI’s) is an issue• many initiatives to deliver reform• achievements not sufficient

Nordic meeting 416.9.2014

Background and history (2)

• lack of consensual solution• conflicting priorities of Member States• need to unblock the situation• new element, what can it be?• the analogy to administrative data• double collection of data is an opportunity• EU legislation cannot force MS to reduce burden


The proposal from 2012 – SIMSTAT1. Reduce data collection requirements on the intra-EU

imports (arrivals) side.2. Enable Member States to design the data collection

for the intra-EU imports according to their needs.3. Create an additional data source by making the

exchange of micro data for intra-EU exports (dispatches) mandatory.

4. Member States can chose whether to use the exchanged data. A country specific, gradual implementation of the SIMSTAT system would be possible.


The Benefits

• Potential for reduction in burden• Quality maintained through additional data source• Remove duplicate data collection• Developing generic solutions for microdata exchange

which can be used to support the wider ESS Vision


The Costs

• Low cost solutions were made in the past• The sharing of data is a new element, which will have

– set-up costs (funding from Eurostat: ESSnet, Grants)– maintenance costs– additional costs for Member States– additional burden on Intra-EU exporters


The Risks

• Will Member States realise this potential reduction?• Will Member States use this new data set?• will SIMSTAT deliver a burden reduction?• will the burden reduction be sufficient?• will the figures still be good enough quality and in time

for National Accounts• will there be consensus of the Council and of the

European Parliament (for example on FRIBS)?


Background clear, but what to do?• Test the technical feasibility (ESSnet1+2)

• Pursue methodological considerations (ESSnet1+2, WG Methodology)

• Measure burden reduction potential (WG Methodology & Quality, SIMSTAT PTF and SG, BSDG)

• Evaluate the impact on quality (ESSnet1+2, WG Methodology & Quality, SIMSTAT PTF, SG of ITGS, BSDG)

• Draft legal text (WG Methodology, SG of ITGS, BSDG)

• Seek consensus (SIMSTAT PTF, SG of ITGS, BSDG, ESSC)

-> Target timetable: trials analysis ready on Q4/2015? Final decisions during 2016?

10Nordic meeting

SIMSTAT-Pilot ESSnet1 timeline: 12 months, 27.9.2013 – 26.9.2014Main tasks of the ESSnet11. Resolving the open technical issues for the

practical micro-data exchange pilot2. Elaboration of validation rules, data-exchange

workflows, timetable et.c to be applied in the future ”production test”-environment

3. Planning of data compilation arrangements to be applied in the future ”production test” environment

4. Quality evaluation of the test results

16.9.2014

11Nordic meeting

Work plan:1. WP1: Producing the technical documentation and

instructions on technical issues (Krassimir Ivanov)2. WP2: Setting up the confidentiality arrangements

(Timo Koskimäki)3. WP3: Establishing rules for validation and credibility

checks, error correction and revision of data (Maria Serena Causo

4. WP4: Designing the production timetable for the trial tests (Rafael Mastrangelo)

5. WP5: Elaborating and implementing the data compilation guidelines (Olli-Pekka Penttilä)

6. WP6: Elaborating and implementing the quality evaluation of the test results (Søren Rich)

All work will be based on work already done in PTF and Quality/Methods groupsSolutions developed and adopted for testing, no “final” decisions to be made

16.9.2014


Organisation of the work of ESSnet1

– Participating MSs:• Austria, Bulgaria, Czech Republic, Denmark, Estonia,

Germany, Greece, Italy, Luxemburg, Malta, Poland, Portugal, Romania, Slovakia, Finland (Co-ordinator), United Kingdom, France

– One leading country for each work package– Most of the work to be done in-between the

meetings – EUROSTAT expected to participate actively

although it can not be reflected in the work plan


ESSnet1

Nordic meeting 14

4 meetings:• Kick-off meeting on 7th November in Luxembourg• 2nd meeting on 22. – 23. January 2014 in Lisbon• 3rd meeting on 7. – 8. May 2014 in Sofia• Final 4th meeting on 10. – 11. September 2014 in

Helsinki

16.9.2014

Nordic meeting 15

WP1 tasks: Producing the technical documentation and instructions on technical issues for the exchange of micro-data during the ESSnet and future production-test environment 1.1. Elaboration of operational rules to be followed for those variables which are currently not collected by the countries participating in the trial test. This shall also include the operational rules for the simulation of the partner ID along the principle that was agreed in the PTF. (Draft February, Final 3th meeting)

1.2. Definition and description of the dataset structure and detailed data exchange rules: “Doc Met 400 SIMSTAT” (data elements on 2nd meeting, Draft document accepted by PTF February, Final on ESSnet 3rd meeting)

16.9.2014

Nordic meeting 16

Follow up the WP1 by Co-ordinator:• DocMet400SIMSTAT: micro data file specifications

(mandatory, monthly). New mandatory data elements:– “ID of partner trader”, “Country of origin in dispatches”

• Other data file: other data file specifications (voluntary, monthly) will include?– missing value estimates, below threshold trade estimates– the most important missing PSIs and estimate for that– confidential CN8’s monthly– data related to specific goods and movements– Intra-EU trade data coming from customs declaration

16.9.2014

Nordic meeting 17

WP2 tasks: Setting up the confidentiality arrangements for the trial micro-data exchange2.1. proposal for the contractual arrangements for the ESSnet data exchange and for the future production tests using the data transmission network (final on 2nd meeting)

2.2. preparation of a draft contract template to be used in the context of the ESSnet and for the future production tests using the data transmission network (semi-final on 2nd meeting, final on April 2014)

2.3. proposal for the practical organization of the micro-data exchange in the context of the ESSnet and signatures of the data exchange contracts (on 3rd meeting)

16.9.2014

Nordic meeting 18

Follow up the WP2 by Co-ordinator:

• Agreements signed (in force until end of 2017) between ESSnet1 participating MSs:– Multilateral agreement (all MSs exept France)– Bilateral agreement with France– CIRCABC folder for signed agreements (also for Trials,

managed by Olli-Pekka Penttilä)

• Micro data exchanged between the 14 MSs on May-June 2014

• Plan how new MSs will sign the agreements

16.9.2014

Nordic meeting 19

WP3 tasks: Establishing rules for micro data validation and credibility checks, error correction and revision of data3.1. Proposal for the rules to applied in the future production tests as part of data transmission network (draft on 2nd meeting, final on 3rd meeting)

3.2. Proposal for the rules to be applied in the MS’s IT systems/data validation processes prior to the transmission of the data and after receiving the data (draft on 2nd meeting, final on 3rd meeting)

3.3. Proposal for a workflow on how to sending and receiving MS deal with errors that the receiving MS detects in the delivered data (draft on 3rd meeting, final on June PTF)

3.4. Testing of the feasibility of the proposed rules using the exchanged data as to reach a balance between error detection (data quality) and data validation/error correction burden for the MS’s (on 4th meeting)

16.9.2014

Nordic meeting 20

Follow up the WP3 by Co-ordinator:

• Rules for micro data validation and credibility checks and error correction done.

• Testing of the new rules will be done during trial tests.

16.9.2014

Nordic meeting 21

WP4 tasks: Designing the production timetable for the trial tests adjusted to the production timetables of the MS’s and the current legal framework. The timetable should accommodate the impact of late responses and data corrections4.1 Draft proposal for the production timetable with regard to the future “production test” context where data transmission network will be used. (final on 2nd meeting)

4.2.Proposal for the timing and practices of data extractions and data exchange in ESSnet context in order to simulate the future “production test” timetable (draft on February 2014, final on 3rd meeting)

4.3. Final proposal for the production timetable with regard to the future “production test” -context where data transmission network will be used including the impact of late response and data corrections (final on 4th meeting)

16.9.2014

Nordic meeting 22

Follow up the WP4 by Co-ordinator:• 4 delivery dates during the month will be used to

transfer micro data between the sending MS to the HUB for onward transfer to the receiving MS. The deadline for all MSs is t+30.

• The delivery of all monthly 2013(?) and 2014 data by the Member States to the HUB should be transferred either before the start of the trial or the data could be staggered during the trial test.

• The trial test expects participating Member States to transfer 2014 and 2015 Intrastat data according to the agreed timetable and file specification (DocMet400 SIMSTAT).

16.9.2014

Nordic meeting 23

WP5 tasks: Elaborating and implementing the data compilation guidelines5.1. Overview to national compilation practices (final on 2nd meeting)

5.2 Proposal for the common practices to be applied for the data compilation during the ESSnet. (final on 3rd meeting)

5.2 Proposal for the common practices to be applied for the data compilation in the future “production testing” environment. (final on 4th meeting)

16.9.2014

Nordic meeting 24

Follow up the WP5 by Co-ordinator:• National test databases done for sending and

receiving of micro data files• Merging received other MSs Intra-EU exports and

national Intra-EU imports data• Invoice value will be analyzed• Simulation of “ID of partner trader” if not collected

16.9.2014

Nordic meeting 25

WP6 tasks: Elaborating and implementing the quality evaluation of the test results6.1 Proposal for an analysis plan to evaluate the quality of the statistics to be produced during the ESSnet work. The plan should include both the production of short-term estimates and the production of detailed data. (draft on February-March 2014, final on 3rd meeting)

6.2 Compilation and quality evaluation of statistics using the data exchanged during the ESSnet. Each participating MS should compile and analyze the data concerning their own economy obtained from other Member States. (reports August 2014, final on 4th meeting)

6.3 Summary report on the results of the data analyses performed by all participating MS’s. (final on 4th meeting)

16.9.2014

Nordic meeting 26

Follow up the WP6 by Co-ordinator:• Analysis plan has over 10 Excel spreadsheets. Only

10 MSs had time to do analysis during the summer, first time is the most burdensome.

• Summary report, first remark: – micro data from France and Italy with collected “ID

of partner trader” has good quality. – other MSs has to simulate “ID of partner trader”,

only CZ and GB successfully. • Analysis plan for trial tests: process and product

16.9.2014


Next steps, the outline:• The SIMSTAT project will prepare a report

demonstrating the outcomes of the trial tests in 2015. • The report will be presented to the ESSC in May 2016

for taking a decision on the eventual implementation of SIMSTAT.

• FRIBS will provide the legal basis for SIMSTAT if the ESSC decides to introduce a mandatory micro-data exchange in the area of intra-EU trade statistics.


Next steps, detailed (1):• Connect to CCN network (ready 10/2014)• Grants 2014: IT, methodological issues (final 3/2015)• ESSnet2 Co-ordinated by France (proposal 13.6.2014):

– Work package 1: Testing of functioning and the interoperability of the IT infrastructure and tools (final 3/2015)

– Work package 2: Pilot micro-data exchange (4-9/2015)– Work package 3: assess the respect of validation and

compilation rules for exchanged data and provide an improvement strategy (final 11/2015)

– Work package 4: Analysis and documenting of trial results (final 12/2015)


Next steps, detailed (2):• SIMSTAT Trial test April - September 2015• FRIBS legislative drafting should be ready H1/2015

(final June 2015?)• SIMSTAT ESSnet2 analysis of trial results 10-12/2015

(final December 2015)• FRIBS legislative process 2016, FRIBS in force 2017

(or 2018?)• EU-Grants 2016 or/and 2017: IT, methodological issues ???• 7 MS missing Trials? Connecting to CCN and testing 2016?• Information to PSI’s (new data elements to be collected etc) on

H2/2016?• Total EU-testing (28 MSs) during 2017?

16.9.2014 Nordic meeting 30

Thank you!

• Discussion

Documents

16.9.2014 Olli-Pekka Penttilä Director of Statistics / Co-ordinator Finnish Customs Nordic meeting 2014 Preparation for exchange of micro-data on intra-EU