Metadata Framework for a Statistical Data Warehouse

Metadata Framework for a Statistical Data

Warehouse

Lars-Göran Lundell

Statistics Sweden

Cardiff 24 May 2012

Metadata and Data Warehouse

• Metadata is the DNA of the data warehouse, defining its elements and how they work together.

- Ralph Kimball

• Metadata plays a very active and important part in the data warehouse environment.

- Bill Inmon

Last workshop …

• General metadata definitions• Metadata for a Statistical Data Warehouse• Metadata standards• Metadata quality

• What’s next?• More detailed descriptions• Standards• Collection and usage• Storage• More

Metadata Framework for SDWH

• Overview and Conceptual Model• Terms, definitions and relations

• Basis for discussions• Priorities, relations

• Basis for more detailed work• Roadmap

• First version “ready”• Final version July 2013

Metadata categories

Active

Passive

ReferenceStructural

FormalisedFree-form

SDWH metadata requirements

• Active metadata• Assistance to end-users • Enables a metadata

driven architecture

• Formalised metadata• Must be easy to find,

compare and evaluate

• Structural metadata• Link between metadata

and data

Active

Passive

ReferenceStructural

Formalise

d

Free-form

Metadata subsets

SDWHMetadatarequirements

Metadata Structures

• Metadata layer – conceptual, all metadata• Metadata registry – logical, standardised storage• Metadata repository – physical storage

Quality

ProcessActive

Passive

ReferenceStructural

Formalised Free-form

A metadata item The metadata layerThe data store

Statistical

GSBPM, SDWH and Metadata

1SpecifyNeeds

2Design

3Build

4Collect

5Process

6Analyse

7Disseminate

8Archive

9Evaluate

Metadata

SDWH

• The SDWH needs metadata from the “early processes”• Specify needs, Design, Build

• “Early processes” need SDWH metadata• E.g., during the Design process

Metadata and the Data Warehouse Layers

Source Layer

Integration Layer

Interpretation and Data Analysis Layer

Data Access LayerM

etad

ata

Laye

r

Minimum requirements (?)

• Statistical metadata• Variable name, definition, reference time and source • Value domain (classification) mapped to the variable

• Process metadata• Load time

 • Technical metadata

• Physical location• Data type

 • Authentication metadata

• Access rights mapped to users and groups

Thank you!