21
ILO Department of Statistics Edgardo Greising [email protected]

DEALING WITH METADATA(s) IN ILOSTAT

Embed Size (px)

Citation preview

Page 1: DEALING WITH METADATA(s) IN ILOSTAT

ILO Department of Statistics

Edgardo Greising

[email protected]

Page 2: DEALING WITH METADATA(s) IN ILOSTAT

ILO Department of Statistics

Edgardo Greising

[email protected]

Page 3: DEALING WITH METADATA(s) IN ILOSTAT

I. Introduction

II. ILOSTAT and GSBPM

III. Types of Metadata

IV. ILOSTAT Metadata

V. Summary

Page 4: DEALING WITH METADATA(s) IN ILOSTAT

ILOSTAT, new Statistical Information System

New data collection approach

New application

Standards based

Metadata driven

Page 5: DEALING WITH METADATA(s) IN ILOSTAT

ILOSTAT modules

Data collection

Data cleaning process

Data dissemination

Workflow control

Metadata

Page 6: DEALING WITH METADATA(s) IN ILOSTAT

GSBPM phases1. Specify needs

2. Design

3. Build

4. Collect

5. Process

6. Analyse

7. Disseminate

8. Archive

9. Evaluate

Overarching processes: Quality management

Metadata management

Page 7: DEALING WITH METADATA(s) IN ILOSTAT

ILOSTAT Compliance

GSBPM ILOSTAT

1. Specify needs Specification of requirements prior to collection launching

2. Design Design of Questionnaires, Dissemination tables, charts, DSD, etc

3. Build Questionnaire production, Tools development

4. Collect Data collection

5. Process Consistency check, Editing, Calculated indicators

6. Analyse Data analysis, Estimations and Projections

7. Disseminate Data dissemination (different channels)

8. Archive Data and metadata preservation

9. Evaluate Process evaluation (continuous)

Quality management Data and metadata quality evaluation (continuous)

Metadata management Metadata maintenance

Page 8: DEALING WITH METADATA(s) IN ILOSTAT

Metadata classification

Metacontent

Methodology

Quality

Metadata

Business

Technical

Process

Structural

Reference

Descriptive

(conceptual)

Ext. Resources

Page 9: DEALING WITH METADATA(s) IN ILOSTAT

Technical metadataSystem’s parameters Govern automatic updates and scheduled batch processes

Data Collection

Structural validation of data collection instruments

User Access Control Roles, credentials, access rights, etc.

Process Consistency rules

Formulas for calculated indicators

Dissemination website Context information: language, country, subject

Page 10: DEALING WITH METADATA(s) IN ILOSTAT

Process metadataILOSTAT’s workflow

Information used by: Country Specialists: to manage data collection activities

Supervisors: to have real time information about the amount and quality of the data compiled

Applications: to drive the data workflow

Page 11: DEALING WITH METADATA(s) IN ILOSTAT

Structural metadataBusiness Structural metadata

The “heart” of the metadata driven system. code lists for all the concepts in use

definition of all the artefacts used for

• data collection: questionnaires, DSD’s, etc.

• Dissemination: tables, charts, navigation menus, etc.

Stored in a single metadata repository Shared by all the modules

Single point of maintenance

Page 12: DEALING WITH METADATA(s) IN ILOSTAT

Statistical metacontentBusiness Reference metadata Descriptive metadata Statistical metacontent

Typical “data about data”.

Two classes of metacontent: Observation value status: sometimes called flag

Notes: controlled vocabulary, coded at collection time

Cleaning module Checking for mandatory and contradictory notes

Dissemination module Table metadata, flags and footnotes

Consolidation of footnotes at display time

Page 13: DEALING WITH METADATA(s) IN ILOSTAT

Unlabeled stuff Labeled stuff

The bean example is taken from: A Manager’s

Introduction to Adobe eXtensible Metadata Platform,

http://www.adobe.com/products/xmp/pdfs/whitepaper.pdf

Page 14: DEALING WITH METADATA(s) IN ILOSTAT

Statistical metacontentBusiness Reference metadata Descriptive metadata Statistical metacontent

Typical “data about data”.

Two classes of metacontent: Observation value status: sometimes called flag

Notes: controlled vocabulary, coded at collection time

Cleaning module Checking for mandatory and contradictory notes

Dissemination module Table metadata, flags and footnotes

Consolidation of footnotes at display time

Page 15: DEALING WITH METADATA(s) IN ILOSTAT

Methodological metadataBusiness Reference metadata Descriptive metadata Methodological metadata

Methodology used for primary data collection and processing

“Source & Methods” connected to the series by the “survey” key

collects and disseminates the information in DDI 2.x

Page 16: DEALING WITH METADATA(s) IN ILOSTAT

Quality metadataBusiness Reference metadata Descriptive metadata Quality metadata

Stored in the Workflow tables

Provides information about the quality of the data Ex.: After consistency checking data status can be “Error”,

“Ready for dissemination” or “Ready by allowance”.

Additional quality related information attached as comments

Page 17: DEALING WITH METADATA(s) IN ILOSTAT

External resourcesBusiness Reference metadata External resources

Artefacts and documents related to the studies Questionnaires

Methodological guidelines

Maps

Full study metadata in DDI and microdata files will be available from the ILO Central Microdata Repository

Page 18: DEALING WITH METADATA(s) IN ILOSTAT

GSBPM ILOSTAT DEFINES/PRODUCES USES

1. Specify needs Specification of requirements prior to collection launching Structural, Metacontent Process, Quality (previous), Methodology

2. Design Design of Questionnaires, Dissemination tables, charts, DSD, etc Technical, Structural, Metacontent Quality (previous)

3. Build Questionnaire production, Tools development Process Technical, Process, Structural

4. Collect Data collection Metacontent Technical, Process, Structural

5. Process Consistency check, Editing, Calculated indicators Metacontent, Quality Technical, Process, Structural, Metacontent

6. Analyse Data analysis, Estimations and Projections Metacontent, Quality

7. Disseminate Data dissemination (different channels) ALL

8. Archive Data and metadata preservation Ext. Resources ALL

9. Evaluate Process evaluation (continuous) Process, Quality Process, Quality

Quality management Data and metadata quality evaluation (continuous) Technical, Quality Reference

Metadata management Metadata maintenance ALL ALL

Page 19: DEALING WITH METADATA(s) IN ILOSTAT

E-mail:[email protected]

Skype:egreising

Twitter: egreising

LinkedIn: http://www.linkedin.com/in/egreising

Page 20: DEALING WITH METADATA(s) IN ILOSTAT

E-mail:[email protected]

Skype:egreising

Twitter: egreising

LinkedIn: http://www.linkedin.com/in/egreising

Page 21: DEALING WITH METADATA(s) IN ILOSTAT

Knowing how it goes.

E-mail

(country + user)

Qtable

(country + indic + survey)