OECD Expert Group on OECD Expert Group on Statistical Data and Metadata ExchangeStatistical Data and Metadata Exchange

Geneva 10-11 May 2007Geneva 10-11 May 2007

OECD.Stat SDMX Web Service (II)OECD.Stat SDMX Web Service (II)Jens Dossé, OECDJens Dossé, OECD

OECD SDMX OECD SDMX web serviceweb service

Fully conforms to SDMX Fully conforms to SDMX standard standard v2v2

Currently two methods:Currently two methods: extract data in extract data in GenericDataGenericData message message extract extract Data Structure Data Structure DefinitionsDefinitions (Key Families), (Key Families), concepts and code listsconcepts and code lists

Method planned for the Method planned for the future: extract future: extract MetadataMetadata

Secured access: integrates Secured access: integrates authorizationauthorization to OECD.Stat to OECD.Stat

Still needs a few additional Still needs a few additional conventionsconventions bridging bridging SDMX-ML limitsSDMX-ML limits

Additional conventions I Additional conventions I

For time series data, independently of the presence of a FREQUENCY dimension, For time series data, independently of the presence of a FREQUENCY dimension, the requested time frequency must be specified using a the requested time frequency must be specified using a TIME_FORMATTIME_FORMAT attribute: attribute:

<query:And><query:Attribute name="TIME_FORMAT">P1M</query:Attribute><query:Time>

<query:StartTime>1960-01</query:StartTime><query:EndTime>2005-12</query:EndTime>

</query:Time></query:And>

Format values: Format values: P1YP1Y=annual, =annual, P6MP6M=semi-annual, =semi-annual, P3MP3M=quarterly and =quarterly and P1MP1M=monthly data =monthly data Time formats: Time formats: yyyyyyyy, , yyyy-mmyyyy-mm or or yyyy-mm-ddyyyy-mm-dd with digits y=year, m=month and d=day, e.g. '2005-09-01'. with digits y=year, m=month and d=day, e.g. '2005-09-01'. This attribute is then also present in the resulting generic data message at the Series level of time-series This attribute is then also present in the resulting generic data message at the Series level of time-series data.data.

Non-time-seriesNon-time-series data: omit all time specifications in the query message data: omit all time specifications in the query message

The response messages will use The response messages will use 99999999 as (obligatory) time periods as (obligatory) time periods

NEWNEW: If the : If the <query:StartTime><query:StartTime> or or <query:EndTime><query:EndTime> nodes are not present nodes are not present then all the available values are returnedthen all the available values are returned

Additional conventions II Additional conventions II

OECD.Stat data warehouse:OECD.Stat data warehouse:

EachEach dataset (multidimensional cube) has its own Data Structure dataset (multidimensional cube) has its own Data Structure Definition.Definition.

Datasets can share Datasets can share common dimensionscommon dimensions but which can have but which can have additional non-common codesadditional non-common codes

Several datasetsSeveral datasets within one message within one message

Query message can request data from several datasets Query message can request data from several datasets

NEWNEW: Different datasets can now be returned within one SDMX : Different datasets can now be returned within one SDMX generic message using “generic message using “MessageGroupMessageGroup”s”s

Additional conventions III Additional conventions III

For simplicity during the start-up of this generic web service, the returned For simplicity during the start-up of this generic web service, the returned message headers message headers onlyonly contain contain required informationrequired information

Maximum size of generic data message: Maximum size of generic data message: 5 MB5 MB

message is truncated if necessary message is truncated if necessary

Requests need to be split if this limit is exceeded Requests need to be split if this limit is exceeded

(Still) open challenges (Still) open challenges for SDMX web servicesfor SDMX web services

!!!: Big message sizes for any data format: need standard for !!!: Big message sizes for any data format: need standard for compressioncompression

!!!: Security: need standard for !!!: Security: need standard for authentication/encryptionauthentication/encryption

!!!: Need for!!!: Need for another message format: another message format: generic cross-sectionalgeneric cross-sectional format format remove the special status of the Time dimension (as not all statistical remove the special status of the Time dimension (as not all statistical data are time series)data are time series)

Registering SDMX web services in the SDMX Registering SDMX web services in the SDMX registryregistry

Multilaterally connecting organisations’ SDMX web services to build Multilaterally connecting organisations’ SDMX web services to build international international networksnetworks

OECD SDMX web service client exampleOECD SDMX web service client example

Simple web applicationSimple web application

Provides Provides example queryexample query messagesmessages

Source codeSource code freely available freely available

Secured access to all data in Secured access to all data in data warehouse (integrates data warehouse (integrates authenticationauthentication and and authorization to OECD.Stat)authorization to OECD.Stat)

Contacts/LinksContacts/Links

jens.dosse@oecd.orgjens.dosse@oecd.orgdouglas.paterson@oecd.orgdouglas.paterson@oecd.org

httphttp://stats.oecd.org/OECDStatWS_SDMX/SDMX.asmx://stats.oecd.org/OECDStatWS_SDMX/SDMX.asmx

httphttp://stats.oecd.org/OECDStatWS_SDMX/Test_SDMX.aspx://stats.oecd.org/OECDStatWS_SDMX/Test_SDMX.aspx