DEALING WITH

METADATA . IN

ILOSTATILO Department of Statistics

Edgardo Greisinggreising@ilo.org

DEALING WITH

METADATA(s) IN ILOSTAT

ILO Department of StatisticsEdgardo Greising

greising@ilo.org

I. IntroductionII. ILOSTAT and

GSBPMIII. Types of Metadata IV. ILOSTAT MetadataV. Summary

Agenda

ILOSTAT, new Statistical Information SystemNew data collection approachNew applicationStandards basedMetadata driven

Introduction

ILOSTAT and GSBPMILOSTAT modules

Data collectionData cleaning processData disseminationWorkflow controlMetadata

ILOSTAT and GSBPMGSBPM phases

1. Specify needs2. Design3. Build4. Collect5. Process6. Analyse7. Disseminate8. Archive9. EvaluateOverarching processes:

Quality management Metadata management …

ILOSTAT and GSBPMILOSTAT Compliance

GSBPM ILOSTAT1. Specify needs Specification of requirements prior to collection launching2. Design Design of Questionnaires, Dissemination tables, charts, DSD, etc3. Build Questionnaire production, Tools development4. Collect Data collection5. Process Consistency check, Editing, Calculated indicators6. Analyse Data analysis, Estimations and Projections7. Disseminate Data dissemination (different channels)8. Archive Data and metadata preservation9. Evaluate Process evaluation (continuous)Quality management Data and metadata quality evaluation (continuous)Metadata management Metadata maintenance

Types of Metadata Metadata

classification

Metacontent

Methodology

Quality

Metadata

Business

Technical

Process

Structural

ReferenceDescriptive (conceptual)

Ext. Resources

ILOSTAT MetadataTechnical metadata

System’s parameters Govern automatic updates and scheduled batch

processesData Collection

Structural validation of data collection instruments User Access Control

Roles, credentials, access rights, etc.Process

Consistency rules Formulas for calculated indicators

Dissemination website Context information: language, country, subject

ILOSTAT MetadataProcess metadata

ILOSTAT’s workflow

Information used by: Country Specialists: to manage data collection activities Supervisors: to have real time information about the

amount and quality of the data compiled Applications: to drive the data workflow

ILOSTAT MetadataStructural metadataBusiness Structural metadata

The “heart” of the metadata driven system. code lists for all the concepts in use definition of all the artefacts used for • data collection: questionnaires, DSD’s, etc.• Dissemination: tables, charts, navigation menus, etc.

Stored in a single metadata repository Shared by all the modules Single point of maintenance

ILOSTAT MetadataStatistical metacontentBusiness Reference metadata Descriptive metadata Statistical metacontent

Typical “data about data”. Two classes of metacontent:

Observation value status: sometimes called flag Notes: controlled vocabulary, coded at collection time

Cleaning module Checking for mandatory and contradictory notes

Dissemination module Table metadata, flags and footnotes Consolidation of footnotes at display time

Unlabeled stuff

Why is this metadata relevant?

Labeled stuff

The bean example is taken from: A Manager’sIntroduction to Adobe eXtensible Metadata Platform,

http://www.adobe.com/products/xmp/pdfs/whitepaper.pdf

ILOSTAT MetadataStatistical metacontentBusiness Reference metadata Descriptive metadata Statistical metacontent

Typical “data about data”. Two classes of metacontent:

Observation value status: sometimes called flag Notes: controlled vocabulary, coded at collection time

Cleaning module Checking for mandatory and contradictory notes

Dissemination module Table metadata, flags and footnotes Consolidation of footnotes at display time

ILOSTAT MetadataMethodological metadataBusiness Reference metadata Descriptive metadata Methodological metadata

Methodology used for primary data collection and processing

“Source & Methods” connected to the series by the “survey” key collects and disseminates the information in DDI 2.x

ILOSTAT MetadataQuality metadataBusiness Reference metadata Descriptive metadata Quality metadata

Stored in the Workflow tablesProvides information about the quality of the data

Ex.: After consistency checking data status can be “Error”, “Ready for dissemination” or “Ready by allowance”.

Additional quality related information attached as comments

ILOSTAT MetadataExternal resourcesBusiness Reference metadata External resources

Artefacts and documents related to the studies Questionnaires Methodological guidelines Maps

Full study metadata in DDI and microdata files will be available from the ILO Central Microdata Repository

SummaryGSBPM ILOSTAT DEFINES/PRODUCES USES

1. Specify needs Specification of requirements prior to collection launching Structural, Metacontent Process, Quality (previous), Methodology2. Design Design of Questionnaires, Dissemination tables, charts, DSD, etc Technical, Structural, Metacontent Quality (previous)3. Build Questionnaire production, Tools development Process Technical, Process, Structural4. Collect Data collection Metacontent Technical, Process, Structural5. Process Consistency check, Editing, Calculated indicators Metacontent, Quality Technical, Process, Structural, Metacontent6. Analyse Data analysis, Estimations and Projections Metacontent, Quality7. Disseminate Data dissemination (different channels) ALL8. Archive Data and metadata preservation Ext. Resources ALL9. Evaluate Process evaluation (continuous) Process, Quality Process, QualityQuality management Data and metadata quality evaluation (continuous) Technical, Quality ReferenceMetadata management Metadata maintenance ALL ALL

Questions?E-mail:greising@ilo.org

Skype:egreising

Twitter: egreising

LinkedIn: http://www.linkedin.com/in/egreising

Thank you!E-mail:greising@ilo.org

Skype:egreising

Twitter: egreising

LinkedIn: http://www.linkedin.com/in/egreising

Knowing how it goes.

Application development

E-mail (country + user)

Qtable (country + indic + survey)