Leaving a Metadata Trail

Chapter 14

Defining Warehouse Metadata

Data about warehouse data and processing Vital to the warehouse Used by everyone

MetadataMetadata

The Key to Understanding Warehouse Information Specifies data location Manage data Aids use of information Describes the data Documents the development process Provides a record of changes Records enhancements over time

Metadata UsersIT developers

ETT

Operational

Enduser

Metadatarepository

WarehouseEnd users

Types of Metadata End user: - Key to a good warehouse - Navigation aid - Information provider ETT: - Maps structure - Source and target information - Transformations Operational: - Load, management, scheduling processes - Performance

Developing a Metadata Strategy Define a strategy to ensure high-quality metadata

useful to users and developers. Primary strategy considerations: - Define goals and intended use. - Identify target users. - Choose tools and techniques. - Choose the metadata location. - Manage access to the metadata. - Manage access to the metadata. - Integrate metadata from multiple tools. - Manage change.

Defining Metadata Goals and Intended Usage Define clear goals. Identify requirements. Identify intended usage.

MetadataMetadata

Identifying Target Metadata Users Who are the metadata users? - Developers - End users What information do they need? How will they access the

metadata?

Choosing Metadata Tools and Techniques Tools - Data modeling - ETT - End-user query and analysis Database schema definitions COBOL copybooks Middleware tools

Choosing the Metadata Location

Usually the warehouse server Possibly on operational platforms Desktop tool with metalayer

Metadatarepository

Warehouse End users

ExternalsourcesOperationalDatasources

Managing the Metadata Managed by the metadata manager Maintained by the metadata architect Standards produced by the metadata

architectMetadatarepository

Warehouse End users

ExternalsourcesOperationalDatasources

Integrating Multiple Sets of Metadata Multiple tools may generate their

own metadata. There are many metalayer

integration issues. Metadata exchangeability is

desirable.

Managing Changes to Metadata Different types of metadata have

different rates of change. Consider metadata changes

resulting from refresh cycles.

Examining Types of Metadata ETT metadata End user metadata

ETTEnduser

Metadatarepository

Warehouse

ETT Metadata Business rules Source tables, fields, and key values Ownership Field conversions Encoding and reference table Name changes Key value changes Default values Logic to handle multiple sources Algorithms Time stamp

Extraction

Externalsources

Operational data sources

Staging file

Extraction Metadata Space and storage requirements Source location information Diverse source data Access information Security Contacts Program names Frequency details Failure procedures Validity checking information

Extraction

Externalsources

Operational data sources

Staging file

Transformation Metadata Duplication routines Exception handling Key restructuring Grain conversions Program names Frequency Summarization

TransformExternalsources

Operational data sources

Staging file

Transportation Metadata Method of transfer Frequency Validation procedures Failure procedures Deployment rules Contact information

Externalsources

OperationalData sources

ETT

TransportTransport

Transport

Staging file

Metadatarepository

Warehouse

End-User Metadata 739516 1816 666 17.62 Associate the metadata description Analogous to Oracle Data Dictionary views

Enduser

Metadatarepository

Warehouse

Example of End User Metadata

TableName

ColumnName

Data Meaning

Product Prodid 739516 Unique identifier for the product

ProductProduct

Valid_dateWare_loc

01/971816

Last refresh dateWarehouse location number

Product Ware_bin 666 Warehouse bin number

Product Weight 17.62 Packed shippingWeight in kilograms

More End-User Metadata Information

Location of fact and dimensions Availability Description of contents Algorithms for derived and summary data Owners of data and telephone number

Enduser

Metadatarepository

Warehouse

Historic Context of Data Supports change history Maintains the context of information

Operational Warehouse

Content

Structure

repositoryMetadata

94 95 96 97 98

Types of Context Simple: - Data structure - Naming conventions - Metrics Complex: - Product definitions - Markets - Pricing External: - Economic - Political

Warehouse

94 95 96 97 98

Additional Metadata Content and Considerations Summarization algorithms Relationships Stewardship Permissions Pattern analysis Reference tables

Metadata Management Tools Carleton Evolutionary Technologies Hewlett Packard Informatics Oracle Designer Platinum Technology Prism Solutions Sagent

Common Warehouse Metadata

Design and Administratioon

Common Warehouse Metadata Future

WarehouseBuilder

Oracle8iServer

Discover

ExpressServer

Commonmetadata

Summary

This lesson discussed the following topics: Definitions Integration Contents Storage Creation Selection Tools