View
248
Download
3
Category
Tags:
Preview:
Citation preview
Eurostat
SDMX@Eurostat: achievements and challenges
Emanuele Baldacci, DirectorEuropean Commission - EurostatDirector Methodology; Corporate statistical and IT services
emanuele.baldacci@ec.europa.eu
29 September 2015 SDMX Global Conference
Eurostat
• Start-up phase: implementation
• Maturity phase: consolidation
• Moving to the future: modernization
2
Outline
Eurostat
3
Why SDMX?
• To improve the exchange of statistical data and metadata, which is complex, resource intensive and expensive
• To provide a standardisation layer replacing or reconciling specific approaches and solutions
• To take advantage of opportunities related to new technologies for machine to machine exchange
In the ESS:2007: Endorsement at ESS Committee
2008: UNSC recognition
2009-2014: SDMX inserted in legal acts for data-metadata exchange
Eurostat
4
SPECIFY NEEDS
DESIGN BUILD COLLECT PROCESS ANALYSE DISSEMINATE EVALUATE
Initial focus on data collection from countries
This helped the integration of statistical processes in the European Statistical System
Phase 1
Eurostat
5
SPECIFY NEEDS
DESIGN BUILD COLLECT PROCESS ANALYSE DISSEMINATE EVALUATE
Eurostat: Data collection by file format (2014)• SDMX-ML + EDI = 47.5% exchanged files.
SDMX-EDI declining, XML increasing
• CSV/Excel (mappable to SDMX)= 33.5%Examples of lowering the burden to data providers, with webforms and CSV/SDMX
Some issues:
• Business buy-in slow, as considerable initial investment needed
• Real advantages not clear enough for the business
• Heavy project management needed
Eurostat
6
SPECIFY NEEDS
DESIGN BUILD COLLECT PROCESS ANALYSE DISSEMINATE EVALUATE
A Hundred DSDs shared with ESS countries
• DSDs used for regular data exchange within the European Statistical System
• DSDs for global use with international org. (national accounts, BoP,…)
Eurostat
Reference Metadata: full compliance with SDMX ESMS
• Euro SDMX Metadata Structure
ESQRS
• ESS Standard for Quality Reports Structure
SIMS
• Single Integrated Metadata Structure
7
SDMX concepts and codes for creating Metadata Structure Definitions
ESS Metadata Standards covering all Eurostat domains on the web
SPECIFY NEEDS
DESIGN BUILD COLLECT PROCESS ANALYSE DISSEMINATE EVALUATE
Eurostat
8
Exposed using SDMX-RI
Web dissemination
Census Hub
• Census Hub opening: 8 December 2014• 32 countries connected (EU & EFTA)• 30 using the SDMX-RI• Until now >50 000 data requests
SPECIFY NEEDS
DESIGN BUILD COLLECT PROCESS ANALYSE DISSEMINATE EVALUATE
SDMX-RI infrastructure enabling buy-inand use of web services for exposing data
DataLoaded in
DDB
Eurostat
SDMX Dissemination Web Services
9
Directly from our dissemination chain:
• complete list of available datasets
• complete structure definition of a given dataset
• download of a subset or a full dataset, customizing requests for data
SPECIFY NEEDS
DESIGN BUILD COLLECT PROCESS ANALYSE DISSEMINATE EVALUATE
http://ec.europa.eu/eurostat/data/web-services
Eurostat
InputThrough-
put OutputProcess
1Process
2
10
Standard for describing data and metadata throughout the exchange, production and dissemination process
Metadata as the decisive abstraction layer to be carried through
Broadening the use of SDMX
Phase 2
SPECIFY NEEDS
DESIGN BUILD COLLECT PROCESS ANALYSE DISSEMINATE EVALUATE
Eurostat
Eurostat SDMX roadmap
• Lower the “entry threshold”
• More global data sharing(standard concepts/codes across domainsand across organisations)
• Metadata sharing for global use
• More sophisticated SDMX IT infrastructure
• Broadening SDMX use cases (e.g. SDMX-based validation)
• Achieve synergies with other standards (e.g. DDI, Linked Open data)
11
Eurostat
SDMX challenges
Main challenges for the years to come:
• Strengthening implementation
• Facilitating data consumption
• Supporting statistical process innovation (collection, analysis, dissemination)
• Broadening SDMX scope (e.g. content validation)
• Enhancing communication and training
12
Eurostat
“Innovation is about making mistakes and learning from them”
emanuele.baldacci@ec.europa.eu
13
Recommended