77
IBM Tivoli Enterprise Console, Version 3.9 Warehouse Enablement Pack, Version 1.2.0 Implementation Guide For Tivoli Enterprise Data Warehouse, Version 1.1 SC32-1236-00

Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

  • Upload
    others

  • View
    8

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

IBM Tivoli Enterprise Console, Version 3.9 Warehouse Enablement Pack, Version 1.2.0

Implementation Guide

For Tivoli Enterprise Data Warehouse, Version 1.1

SC32-1236-00

Page 2: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

First Edition (August 2003)

This edition applies to version 1, release 1, of Tivoli Enterprise Data Warehouse and to all subsequent releases and modifications until otherwise indicated in new editions.

© Copyright International Business Machines Corporation 2003. All rights reserved.

US Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule First Contract with IBM Corp.

Tivoli Enterprise Console Warehouse Implementation Guide ii

Page 3: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

Contents

1 ABOUT THIS DOCUMENT.................................................................................................................... 2

1.1 Related documentation .....................................................................................................................................2 1.1.1 IBM Tivoli Enterprise Console library .......................................................................................................2 1.1.2 Tivoli Enterprise Data Warehouse library...................................................................................................3 1.1.3 IBM DB2, DB2 Data Warehouse Center, and DB2 Warehouse Manager library ......................................3

2 OVERVIEW ............................................................................................................................................ 5

2.1 Overview of Tivoli Enterprise Data Warehouse ............................................................................................5

2.2 Overview of Tivoli Enterprise Console warehouse pack ...............................................................................6

3 INSTALLING, CONFIGURING AND UNINSTALLING.......................................................................... 9

3.1 Prerequisites ......................................................................................................................................................9

3.2 Supported hardware and software ..................................................................................................................9

3.3 Limitations.........................................................................................................................................................9 3.3.1 Installation procedures ................................................................................................................................9 3.3.2 Clean and reset process .............................................................................................................................10 3.3.3 Process logs...............................................................................................................................................10 3.3.4 Component instance names must be unique..............................................................................................10

3.4 Data sources.....................................................................................................................................................11

3.5 Pre-installation procedures ............................................................................................................................11 3.5.1 ODBC database client ...............................................................................................................................11 3.5.2 ODBC drivers ...........................................................................................................................................11

3.6 Installation of the warehouse pack ................................................................................................................12

3.7 Post-installation procedures...........................................................................................................................12 3.7.1 Target and source database configuration.................................................................................................12

3.7.1.1 Setting up the CDW_TWH_CDW source and targets ..........................................................................12 3.7.1.2 Setting up the EC1 source and targets...................................................................................................12

3.7.2 Predicate and event class configuration ....................................................................................................13 3.7.3 Scheduling and executing processes .........................................................................................................13

3.7.3.1 Executing EC1_C05_Create_ETL_Objs_Process ................................................................................13 3.7.3.2 Scheduling and executing EC1_c10_ETL1_Process ............................................................................13

3.8 Uninstalling the warehouse pack ...................................................................................................................14

3.9 Defining and using availability predicates for detecting outages................................................................14 3.9.1 Defining availability outages ....................................................................................................................14 3.9.2 Detecting availability outages ...................................................................................................................15 3.9.3 Pre-configured rules.........................................................................................................................................15 3.9.4 Data requirements for availability outages................................................................................................15

3.9.4.1 Defining meta data by sending in events ..............................................................................................16 3.9.4.2 Defining meta data in the fact files .......................................................................................................16

Tivoli Enterprise Console Warehouse Implementation Guide iii

Page 4: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

3.9.5 Availability BAROC classes defined........................................................................................................17 3.9.5.1 Component type BAROC definitions ...................................................................................................17 3.9.5.2 Measurement group BAROC definitions..............................................................................................18 3.9.5.3 Component BAROC definitions ...........................................................................................................19 3.9.5.4 List_Of component and component type BAROC definitions..............................................................19 3.9.5.5 Outage events BAROC definitions .......................................................................................................21 3.9.5.6 Availability error BAROC definitions ..................................................................................................21

3.9.6 Defining outage parameters in prolog facts ..............................................................................................22 3.9.7 Analyze operational environment .............................................................................................................22

3.9.7.1 Identify systems that require outage detection ......................................................................................22 3.9.7.2 Identify measurement information ........................................................................................................23 3.9.7.3 Identify component information ...........................................................................................................24 3.9.7.4 Component type domain states .............................................................................................................25 3.9.7.5 Allowable state transitions ....................................................................................................................26 3.9.7.6 Detecting instances dynamically...........................................................................................................26

3.9.8 Implementation .........................................................................................................................................27 3.9.8.1 Outage related predicates ......................................................................................................................27 3.9.8.2 Parameter length constraints .................................................................................................................30 3.9.8.3 Customizing the predicate file with custom outage definitions ............................................................30

3.9.9 Operation...................................................................................................................................................34 3.9.9.1 Installation.............................................................................................................................................34 3.9.9.2 Configuration ........................................................................................................................................34 3.9.9.2.1 Changing default values in the warehouse.rls file.............................................................................34 3.9.9.2.2 Changing default values in the warehouse.pro file ...........................................................................35 3.9.9.2.2.1 HP Openview changes.....................................................................................................................35 3.9.9.2.2.2 Single event outage changes............................................................................................................35 3.9.9.3 Uninstalling...........................................................................................................................................35 3.9.9.4 Warehouse rules startup and operation .................................................................................................36 3.9.9.5 Warehouse ruleset details......................................................................................................................39

4 MAINTENANCE ................................................................................................................................... 41

4.1 Backing up and restoring ...............................................................................................................................41

4.2 Maintenance processes and commands.........................................................................................................41 4.2.1 Warehouse enablement pack maintenance processes................................................................................41 4.2.2 Tivoli Enterprise Console maintenance commands ..................................................................................41

4.3 Warehouse data prune control ......................................................................................................................42

5 EXTRACT TRANSFORM AND LOAD PROCESSES ......................................................................... 44

5.1 EC1_c05_Create_ETL_Objs_Process...........................................................................................................44

5.2 EC1_c10_ETL1_Process ................................................................................................................................45

5.3 EC1_c15_Clean_and_Reset_Process.............................................................................................................46

5.4 EC1_c20_Get_ExtCtl_Last_Date_Process....................................................................................................46

5.5 EC1_c25_Drop_ETL_Support_Process........................................................................................................47

6 TROUBLESHOOTING ......................................................................................................................... 48

Tivoli Enterprise Console Warehouse Implementation Guide iv

Page 5: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

6.1 Errors with the tec_t_last_etl_evt table ........................................................................................................48 6.1.1 No rows in table or no row with key_id value of ‘1’ ................................................................................48 6.1.2 No tec_t_last_etl_evt table........................................................................................................................48

6.1.2.1 Recovery ...............................................................................................................................................48

6.2 EC1_c05_Create_ETL_Objs_Process...........................................................................................................48

6.3 EC1_c10_ETL1_Process ................................................................................................................................49 6.3.1 EC1_c10_s010_Extract step .....................................................................................................................49 6.3.2 EC1_c10_s030_Load step.........................................................................................................................49

7 CENTRAL DATA WAREHOUSE INFORMATION .............................................................................. 51

7.1 Component configuration...............................................................................................................................54 7.1.1 Component type (table CompTyp)............................................................................................................54 7.1.2 Component (table Comp)..........................................................................................................................54 7.1.3 Component relationship type (table RelnTyp) ..........................................................................................55 7.1.4 Component relationship rule (table RelnRul) ...........................................................................................55 7.1.5 Component relationship (table CompReln)...............................................................................................55 7.1.6 Attribute type (table AttrTyp) ...................................................................................................................55 7.1.7 Attribute rule (table AttrRul) ....................................................................................................................55 7.1.8 Attribute domain (table AttrDom).............................................................................................................55 7.1.9 Component attribute (table CompAttr) .....................................................................................................55

7.2 Component measurement...............................................................................................................................55 7.2.1 Measurement group type (table MgrpTyp) ...............................................................................................56 7.2.2 Measurement group (table MGrp) ............................................................................................................56 7.2.3 Measurement group member (table MGrpMbr)........................................................................................56 7.2.4 Measurement unit (table Munit)................................................................................................................56 7.2.5 Time summary (table TmSum) .................................................................................................................57 7.2.6 Measurement source (table MSrc) ............................................................................................................57 7.2.7 Measurement type (table MsmtTyp) .........................................................................................................57 7.2.8 Component measurement rule (table MsmtRul) .......................................................................................57 7.2.9 Measurement (table Msmt) .......................................................................................................................58

7.3 Helper tables....................................................................................................................................................58

7.4 Exception tables ..............................................................................................................................................58

7.5 Incremental extraction ...................................................................................................................................58

8 DATA MART SCHEMA INFORMATION ............................................................................................. 59

APPENDIX A OPERATIONAL ENVIRONMENT ANALYSIS WORKSHEETS ...................................... 60

A.1 Component types.............................................................................................................................................60

A.2 Component type relationships .......................................................................................................................61

A.3 Allowable warehouse component states ........................................................................................................62

A.4 Measurement groups ......................................................................................................................................63

A.5 Measurement group states .............................................................................................................................64

Tivoli Enterprise Console Warehouse Implementation Guide v

Page 6: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

A.6 Components .....................................................................................................................................................65

A.7 Component relationships................................................................................................................................66

A.8 Domain states ..................................................................................................................................................67

A.9 Event criteria for condition detection ...........................................................................................................68

A.10 Domain state transitions.............................................................................................................................69

A.11 Event criteria for dynamic component instances .....................................................................................70

Tivoli Enterprise Console Warehouse Implementation Guide vi

Page 7: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

1 About this document This document describes the warehouse enablement pack, version 1.2 for IBM® Tivoli Enterprise Console®, Version 3.9. This warehouse enablement pack uses the product code of EC1. This code is used as part of the name for the scripts and processes for this warehouse enablement pack as well as to define the schema owner of any tables created in the central data warehouse database. This version of the warehouse enablement pack extracts availability data from the Tivoli® Enterprise Console database for outage reporting in the warehouse by IBM Tivoli Service Level Advisor (ITSLA). Availability data is defined as events that report a transition in the state of an object. An example of this would be a router changing from the available state to the unavailable state. This document covers the following topics:

• Installing and configuring the warehouse pack

• Implementing the availability BAROC classes, predicates, and rules in the Tivoli Enterprise Console rule base

• The data flow and data structures used by the warehouse pack

1.1 Related documentation You can access many Tivoli publications online using the Tivoli Information Center, which is available on the IBM Software Support Web site:

http://www.ibm.com/software/tivoli/library/

The following sets of documentation are available to help you understand, install, and manage this warehouse pack:

• Tivoli Enterprise Console

• Tivoli Enterprise Data Warehouse

• IBM DB2®, DB2 Data Warehouse Center, and DB2 Warehouse Manager

The following sections list and briefly describe these libraries.

1.1.1 IBM Tivoli Enterprise Console library The following Tivoli Enterprise Console documents are available on the Tivoli Enterprise Console documentation CD:

• IBM Tivoli Enterprise Console Adapters Guide, SC32-1242

Provides information about supported adapters, including how to install and configure these adapters.

• IBM Tivoli Enterprise Console Installation Guide, SC32-1233

Describes how to install, upgrade, and uninstall the IBM Tivoli Enterprise Console product.

• IBM Tivoli Enterprise Console Command and Task Reference, SC32-1232

Provides details about IBM Tivoli Enterprise Console commands, predefined tasks shipped in the task library, and the environment variables that are available to tasks that run against an event.

• IBM Tivoli Enterprise Console Rule Developer’s Guide, SC32-1234

Describes how to develop rules and integrate them for event correlation and automated event management.

• IBM Tivoli Enterprise Console Rule Set Reference, SC32-1282

Provides reference information about the IBM Tivoli Enterprise Console rule sets.

• IBM Tivoli Enterprise Console User’s Guide, SC32-1235

Tivoli Enterprise Console Warehouse Implementation Guide 2

Page 8: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

Provides an overview of the IBM Tivoli Enterprise Console product and describes how to configure and use the IBM Tivoli Enterprise Console product to manage events.

• IBM Tivoli Enterprise Console Release Notes, SC32-1238

Provides release-specific information that is not available until just before the product is sent to market.

1.1.2 Tivoli Enterprise Data Warehouse library The following Tivoli Enterprise Data Warehouse documents are available on the Tivoli Enterprise Data Warehouse Documentation CD:

• Tivoli Enterprise Data Warehouse Release Notes, GI11-0857

Provides late-breaking information about Tivoli Enterprise Data Warehouse and lists hardware requirements and software prerequisites.

• Installing and Configuring Tivoli Enterprise Data Warehouse, GC32-0744

Describes how Tivoli Enterprise Data Warehouse fits into your enterprise, explains how to plan for its deployment, and provides installation and configuration instructions. It provides an introduction to the built-in program for creating and running reports, and contains maintenance procedures and troubleshooting information.

• Enabling an Application for Tivoli Enterprise Data Warehouse, GC32-0745

Provides information about connecting an application to Tivoli Enterprise Data Warehouse. This book is for application programmers who use Tivoli Enterprise Data Warehouse to store and report on the data of their application, data warehousing experts who import Tivoli Enterprise Data Warehouse data into business intelligence applications, and customers who use their local data in the warehouse.

1.1.3 IBM DB2, DB2 Data Warehouse Center, and DB2 Warehouse Manager library

The DB2 library contains important information about the database and data warehousing technology provided by the IBM DB2 database, DB2 Data Warehouse Center, and DB2 Warehouse Manager. Refer to the DB2 database library for help in installing, configuring, administering, and troubleshooting the DB2 database, which is available on the IBM Web site:

http://www-3.ibm.com/software/data/db2/library/

After you install the DB2 database, its library is also available on your system.

The following DB2 documents are particularly relevant for people working with the Tivoli Enterprise Data Warehouse product:

• IBM DB2 Universal Database for Windows Quick Beginnings, GC09-2971

Guides you through the planning, installation, migration (if necessary), and setup of a partitioned database system using the IBM DB2 product on Microsoft Windows.

• IBM DB2 Universal Database for UNIX Quick Beginnings, GC09-2970

Guides you through the planning, installation, migration (if necessary), and setup of a partitioned database system using the IBM DB2 product on UNIX.

• IBM DB2 Universal Database Administration Guide: Implementation, SC09-2944

Covers the details of implementing your database design. Topics include creating and altering a database, database security, database recovery, and administration using the Control Center, a DB2 graphical user interface.

Tivoli Enterprise Console Warehouse Implementation Guide 3

Page 9: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

• IBM DB2 Universal Database Data Warehouse Center Administration Guide, SC26-9993

Provides information on how to build and maintain a data warehouse using the Data Warehouse Center.

• IBM DB2 Warehouse Manager Installation Guide, GC26-9998

Provides the information to install the following Warehouse Manager components: Information Catalog Manager, warehouse agents, and warehouse transformers.

• IBM DB2 Universal Database and DB2 Connect Installation and Configuration Supplement, GC09-2957

Provides advanced installation considerations and guides you through the planning, installation, migration (if necessary), and set up of a platform-specific DB2 client. Once the DB2 client is installed, you then configure communications for both the client and server, using the DB2 GUI tools or the Command Line Processor. This supplement also contains information on binding, setting up communications on the server, the DB2 GUI tools, DRDA(®) AS, distributed installation, the configuration of distributed requests, and accessing heterogeneous data sources.

• IBM DB2 Universal Database Message Reference Volume 1, GC09-2978 and IBM DB2 Universal Database Message Reference Volume 2, GC09-2979

Lists the messages and codes issued by the DB2 database, the Information Catalog Manager, and the Data Warehouse Center, and describes the actions you should take.

Tivoli Enterprise Console Warehouse Implementation Guide 4

Page 10: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

2 Overview The following sections provide an overview of Tivoli Enterprise Data Warehouse and the Tivoli Enterprise Console warehouse pack.

2.1 Overview of Tivoli Enterprise Data Warehouse Tivoli Enterprise Data Warehouse provides the infrastructure for the following:

• Extract, transform, and load (ETL) processes through the IBM DB2 Data Warehouse Center tool

• Schema generation of the central data warehouse

• Historical reporting

As shown in Fi , Tivoli Enterprise Data Warehouse consists of a centralized data store where historical data from many management applications can be stored, aggregated, and correlated.

gure 2.1

Figure 2.1 Tivoli Enterprise Data Warehouse overview

The central data warehouse uses a generic schema that is the same for all applications. As new components or new applications are added, more data is added to the database; however, no new tables or columns are added in the schema.

A data mart is a subset of a data warehouse that contains data tailored and optimized for the specific reporting needs of a department or team.

The central data warehouse ETL reads the data from the operational data stores of the application that collects it, verifies the data, makes the data conform to the schema, and places the data into the central data warehouse.

The data mart ETL extracts a subset of data from the central data warehouse, transforms it, and loads it into one or more star schemas, which can be included in data marts to answer specific business questions.

A program that provides these ETLs is called a warehouse enablement pack, referred to as a warehouse pack in the rest of this document.

Tivoli Enterprise Console Warehouse Implementation Guide 5

Page 11: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

The ETLs are typically scheduled to run periodically, usually during non-peak hours. If an ETL encounters data that it cannot correctly transform, it creates an entry in an exception table. Exception tables are described in Section 7.4, Exception tables.

2.2 Overview of Tivoli Enterprise Console warehouse pack The Tivoli Enterprise Console, Version 3.9 Warehouse Enablement Pack, Version 1.2.0 for Tivoli Enterprise Data Warehouse, Version 1.1, is an implementation of the IBM DB2 Data Warehouse Center. As such, it provides a Data Warehouse Subject Area named EC1_Tivoli_Enterprise_Console_v3.9_Subject_Area. The following processes are within this subject area:

• EC1_c05_Create_ETL_Objs_Process

This process creates the tec_t_last_etl_evt extraction control table and any other objects (stored procedures, etc.) needed by the ETL process in the Tivoli Enterprise Console event database. For a detailed description of this process refer to section 5.1 EC1_c05_Create_ETL_Objs_Process. gives an overview of this interaction.

Figure 2.2

Figure 2.2 EC1_c05_Create_ETL_Objs_Process Overview

EC1_c05_Create_ETL_Objs_Process

Tivoli Enterprise Console event

database

Central data

warehouse

• EC1_c10_ETL1_Process

This process extracts all AVAIL_Outage events and related data from the event database and loads it into the central data warehouse database. For a detailed description of this process refer to section 5.2 EC1_c10_ETL1_Process. provides an overview of this interaction. Figure 2.3

Figure 2.3 EC1_c10_ETL1_Process Overview

Tivoli Enterprise Console Server

Generated availability events

Tivoli Enterprise Console Event

Database

EC1_c10_ETL1_Process Availability components and measurement data

Central data

warehouse

Tivoli Enterprise Console Warehouse Implementation Guide 6

Page 12: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

• EC1_c15_Clean_and_Reset_Process

This process drops the staging tables created for this ETL, reinitializes the values in the Extract_Control table in the central data warehouse database for this ETL, and reinitializes the tec_t_last_etl_evt table in the TEC database. For a detailed description of this process refer to section 5.3 EC1_c15_Clean_and_Reset_Process. Fi provides an overview of this interaction. gure 2.4

Figure 2.4 EC1_c15_Clean_and_Reset_Process Figure 2.4 EC1_c15_Clean_and_Reset_Process

• EC1_c20_Get_ExtCtl_Last_Date_Process • EC1_c20_Get_ExtCtl_Last_Date_Process

This process retrieves the last examined date_reception value stored in the Extract_Control table in the central data warehouse database. For a detailed description of this process refer to section 5.4 EC1_c20_Get_ExtCtl_Last_Date_Process. provides an overview of this interaction.

This process retrieves the last examined date_reception value stored in the Extract_Control table in the central data warehouse database. For a detailed description of this process refer to section 5.4 EC1_c20_Get_ExtCtl_Last_Date_Process. provides an overview of this interaction. Figure 2.5Figure 2.5

Figure 2.5 EC1_c20_Get_ExtCtl_Last_Date_Process

Central data

warehouse

EC1_c15_Clean_and_Reset_Process Reinitializes Extract_Control and tec_t_last_etl_evt tables and drops Stage tables.

Tivoli Enterprise Console event

database

EC1_c20_Get_ExtCtl_Last_Date _Process

Retrieves the last date_reception value examined by the last ETL process run.

Process log output

Central data

warehouse

Tivoli Enterprise Console Warehouse Implementation Guide 7

Page 13: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

• EC1_c25_Drop_ETL_Support_Process

This process drops the tec_t_last_etl_evt extraction control table, and two stored procedures in Informix, in the Tivoli Enterprise Console database. For a detailed description of this process refer to section 5.5 EC1_c25_Drop_ETL_Support_Process. Figu provides an overview of this interaction. re 2.6

Figure 2.6 EC1_c25_Drop_ETL_Support_Process

Tivoli Enterprise console event

database

EC1_c25_Drop_ETL_Support_Process Drops tec_t_last_etl_evt extraction control table.

central data

warehouse

Tivoli Enterprise Console Warehouse Implementation Guide 8

Page 14: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

3 Installing, configuring and uninstalling This section describes the information about installing, configuring, and uninstalling the warehouse pack.

3.1 Prerequisites Before installing the Tivoli Enterprise Console warehouse pack, the following software must be installed:

• Tivoli Enterprise Console Version 3.9

• IBM DB2 Universal Database Enterprise Edition Version 7.2

• IBM DB2 Universal Database Enterprise Edition Version 7.2 fix pack 6

• Tivoli Enterprise Data Warehouse required e-fixes to IBM DB2 UDE v7 fix pack 6 (1.1-TDW-0002)

• Tivoli Enterprise Data Warehouse Version 1.1

• Tivoli Enterprise Data Warehouse 1.1 fix pack 2 (1.1-TDW-FP02)

You can obtain the Tivoli Enterprise Data Warehouse fix pack from the Tivoli Enterprise Data Warehouse Web site at the following address:

http://www.ibm.com/software/sysmgmt/products/support/TivoliDataWarehouse.html

Click the Downloads link in the Self help section.

3.2 Supported hardware and software The Tivoli Enterprise Console warehouse enablement pack, Version 1.2 supports Tivoli Enterprise Console Version 3.9. It supports DB2®, Informix®, Microsoft® SQL Server, Oracle, and Sybase as documented in the Tivoli Enterprise Console, Version 3.9 product announcement.

For information about the hardware and software requirements of Tivoli Enterprise Data Warehouse, see the Tivoli Enterprise Data Warehouse Release Notes.

3.3 Limitations The following sections describe limitations and workarounds.

3.3.1 Installation procedures The database user that installed the warehouse must also install the warehouse pack. The example below assumes that this user is db2. If you cannot install this warehouse pack using the same user ID that was used to install Tivoli Enterprise Data Warehouse core application, you must create a user temporary tablespace for use by the installation program. The user temporary tablespace that is created in each central data warehouse database and data mart database during the installation of Tivoli Enterprise Data Warehouse is accessible only to the user that performed the installation.

If you are installing the warehouse pack using the same database user name that was used to install Tivoli Enterprise Data Warehouse, or if your database user has access to another user’s temporary tablespace in the target databases, no additional action is required.

If you do not know the user name used to install Tivoli Enterprise Data Warehouse, you can determine whether the tablespace is accessible by attempting to declare a temporary table while connected to each database as the user that will install the warehouse pack. Use the following commands:

db2 "connect to TWH_CDW user installing_user using password" db2 "declare global temporary table t1 (c1 char(1)) with replace on commit preserve rows not logged"

db2 "disconnect TWH_CDW"

db2 "connect to TWH_MART user installing_user using password"

Tivoli Enterprise Console Warehouse Implementation Guide 9

Page 15: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

db2 "declare global temporary table t1 (c1 char(1)) with replace on commit preserve rows not logged"

db2 "disconnect TWH_CDW"

Where:

installing_user Identifies the database user that will install the warehouse pack.

password Specifies the password for the installing user.

If the declare command is successful, the specified database user can install the warehouse pack. No additional action is required.

If the declare command fails, run the following DB2 commands to create a new tablespace for the installation in both the central data warehouse database and data mart databases:

db2 "connect to TWH_CDW user installing_user using password" db2 "create user temporary tablespace usertmp2 managed by system using (' usertmp2)"

db2 "disconnect TWH_CDW"

db2 "connect to TWH_MART user installing_user using password" db2 "create user temporary tablespace usertmp3 managed by system using (' usertmp3')"

db2 "disconnect TWH_MART"

Where:

installing_user Identifies the database user that will install the warehouse pack.

password Specifies the password for the installing user.

3.3.2 Clean and reset process The EC1_c15_Clean_and_Reset_Process provided with this warehouse pack cannot remove the data from the central data warehouse database tables that has been extracted from the Tivoli Enterprise Console event database. This is because the default warehouse MsmtTyp table defines the measurement types used for the data and they have an MSrc_Cd of ‘MODEL1’ instead of EC1.

You might be able to manually remove test data or, if no other data needs to be saved in the Msmt table on your test system, you can change the prune measurement control value so that all data is pruned on the next run, and then reset the prune measurement control value back to the preferred value.

Data removal should only be attempted in a test installation. In a production installation, only the prune control process should be used to remove measurement data.

3.3.3 Process logs The output in the process logs is only provided in English. These logs are not translated.

3.3.4 Component instance names must be unique For this release of the Tivoli Enterprise Console warehouse enablement pack, version 1.2.0, component names must be unique per component type, that is no two component instances of the same type can share the same name

In that case, this version of the warehouse enablement pack does not fully implement an appropriate way to model hierarchical naming. Instead, it can be best used to represent a flat naming structure. For example, you may have the following naming relationships at your site:

Component types:

IP_HOST ADMIN_SERVER APP_SERVER WEB_APP SERVLET

Tivoli Enterprise Console Warehouse Implementation Guide 10

Page 16: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

And if you have the same servlets running in two different servers, you could represent them as the following components:

hosthameA.austin.ibm.com hostname Default Server Trade BuyStock

hosthameB.austin.ibm.com hostname2 Default Server2 Trade2 BuyStock

But due to this limitation, the two servlets are considered the same component instance, since their parent names in the relationship hierarchy are not considered. So, these relationships can not be fully represented using this version of the warehouse enablement pack. The EC1_c10_ETL1_Process will fail and log an error at the attempt to enter the second BuyStock component instance name into the warehouse.

A workaround for this problem is to specify a different name for each one of the servlets. So, if you had:

hosthameA.austin.ibm.com hostname Default Server Trade BuyStock

hosthameB.austin.ibm.com hostname2 Default Server2 Trade2 BuyStock2

this scenario is supported.

This limitation will be addressed in a subsequent maintenance release.

3.4 Data sources This version of the Tivoli Enterprise Console warehouse pack does not use the tables and triggers defined with the Tivoli Decision Support for Event Management Guide or the Tivoli Enterprise Console Warehouse Enablement Pack, version 1.1.0.

The source of Tivoli Event Console data for the Tivoli Enterprise Console warehouse enablement pack, version 1.2.0 is the event database. Specifically, the tec_t_evt_rep and tec_t_slots_evt tables of the event database are defined as sources of the warehouse pack. The tec_t_evt_rep table contains the base slots of the availability events and tec_t_slots_evt contains the extended slots of these events.

3.5 Pre-installation procedures The following sections describe pre-installation procedures.

3.5.1 ODBC database client If the Tivoli Enterprise Console event database and Tivoli Data Warehouse do not use the same database vendor type, you must install a database client for the Tivoli Enterprise Console Database on the Tivoli Data Warehouse machine. For example, if the Tivoli Enterprise Console database is implemented on a Sybase server, then the Sybase client must be installed on the Warehouse machine. This is so the ODBC connection to be set up can use that client to communicate with the Tivoli Enterprise Console database.

3.5.2 ODBC drivers See the getting started information in Installing and Configuring Tivoli Enterprise Data Warehouse to create a System DSN entry using the ODBC driver that is appropriate to the event database vendor:

DataWHSE 3.60 32-bit INFORMIX

DataWHSE 3.60 32-bit Oracle8

DataWHSE 3.60 32-bit SQL Server

DataWHSE 3.60 32-bit Sybase

IBM DB2 ODBC DRIVER

Tivoli Enterprise Console Warehouse Implementation Guide 11

Page 17: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

These drivers are installed with the installation of any version later than and including DB2 7.2. The driver used with Oracle 8 can also be used with Oracle 9. Specify TEC as the data source name.

3.6 Installation of the warehouse pack Install the warehouse pack as described in “Installing and Configuring Tivoli Enterprise Data Warehouse.” The installation media for the warehouse pack is located on the IBM Tivoli Enterprise Console support version 3.9 CD.

3.7 Post-installation procedures Section 3.7.1 describes the information about configuring the warehouse pack data sources and targets. After you install the warehouse pack, use the procedures described in the getting started section of Installing and Configuring Tivoli Enterprise Data Warehouse to use the Data Warehouse Center to perform the following configuration tasks for data sources and targets.

“ ” on Page 12 and “ ” on Page 12 describe post-installation procedures required by this warehouse pack. Setting up the CDW_TWH_CDW source and targets

3.7.1.1 Setting up the CDW_TWH_CDW source and targets

Setting up the EC1 source and targets

3.7.1.2 Setting up the EC1 source and targets

3.7.1 Target and source database configuration Make sure the control database is set to TWH_MD following the directions in “Installing and Configuring Tivoli Enterprise Data Warehouse” manual.

1. As described in the “Installing and Configuring Tivoli Enterprise Data Warehouse” manual, specify the properties for the warehouse target in CDW_TWH_CDW_Target. These properties are in the Database page.

• Do not change the value of the Data Source field. It must be TWH_CDW.

• In the User ID field, type the user ID used to access the central data warehouse database. The default value is db2admin.

• In the Password and Verify password fields, type the password used to access the central data warehouse database by the previously defined user.

2. Specify the same properties for the warehouse source in CDW_TWH_CDW_Source on the Data Source page.

As described in the “Installing and Configuring Tivoli Enterprise Data Warehouse” manual, specify the properties for the central data warehouse target in EC1_TWH_CDW_Target. These properties are in the Database page.

• Do not change the value of the Database Name field. It must be TWH_CDW.

• In the User ID field, type the user ID used to access the Tivoli Enterprise Data Warehouse central data warehouse database. The default value is db2admin.

• In the Password and Verify password fields, type the password used to access the central data warehouse database by the previously defined user.

Specify the following properties for the EC1 warehouse source in EC1_TEC_Source on the Data Source page.

• In the Data Source Name field, type the ODBC System DSN name used to access the Tivoli Enterprise Console database that was created following the directions in “ ” on Page 11. ODBC drivers

Tivoli Enterprise Console Warehouse Implementation Guide 12

Page 18: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

• In the User ID field, type the user ID used to access the Tivoli Enterprise Console database. This should be the same user ID that is set as the RIM RDBMS user when the Tivoli Enterprise Console command wgetrim tec is run.

• In the Password and Verify password fields, type the password used to access the database by the previously defined user.

3.7.2 Predicate and event class configuration Follow the steps described in “ ” on Page 14 to create new availability facts and initialize the Tivoli Enterprise Console event server with the new availability event class and rule definitions.

Defining and using availability predicates for detecting outages

3.7.3 Scheduling and executing processes The following two processes included with this warehouse pack need to be configured to run to prepare the ETL process to run.

3.7.3.1 Executing EC1_C05_Create_ETL_Objs_Process Run the EC1_C05_Create_ETL_Objs_Process once to create the required objects in the Tivoli Enterprise Console database. See the explanation in section 5.1, EC1_C05_Create_ETL_Objs_Process, for information on running the EC1_c05_Create_ETL_Objs_Process, which creates the required warehouse extract control table in the Tivoli Enterprise Console database. For an Informix database, this process also creates two required stored procedures, which are used by the EC1_c10_ETL1_Process.

3.7.3.2 Scheduling and executing EC1_c10_ETL1_Process Follow the steps for scheduling application ETL processes in “Installing and Configuring Tivoli Enterprise Data Warehouse” to schedule the EC1_c10_ETL1_Process to run on a regular basis as preferred.

Tivoli Enterprise Console Warehouse Implementation Guide 13

Page 19: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

3.8 Uninstalling the warehouse pack 1) Perform the following steps to remove the warehouse pack.

2) Run the EC1_c25_Drop_ETL_Support_Process as described in section 5.5, EC1_c25_Drop_ETL_Support_Process, to remove the tec_t_last_etl_evt table from the Tivoli Enterprise Console database.

3) Uninstall the warehouse pack as described in “Installing and Configuring Tivoli Enterprise Data Warehouse” manual.

When the warehouse pack is removed, all the tables defined with a schema of EC1 are removed but the data in the CDW tables remains and is still useable by other applications. The measurement data in the Msmt Central Data Warehouse table is removed over time by the prune control processes but the rest of the data such as the measurement groups, the component types, and the components remain in the warehouse.

3.9 Defining and using availability predicates for detecting outages The following section explains using availability predicates for detecting outages.

3.9.1 Defining availability outages This warehouse pack includes a new set of classes, rules, and fact files that are used for detecting outages in an enterprise. An outage is defined as occurring when a component changes from a Normal state to any other state. An example is when a node in a network ceases to function. That is, it changes to the Unavailable state. An outage event can depict any number of states, usually occurring sequentially, such as a network segment changing from Normal to Marginal and, finally, to Critical, when it fails. An event would be generated to show the state change from Normal to Marginal and another one for the state change from Marginal to Critical. Each event would indicate the start date and time of the state change, the state the component changes into, and the duration of the state change.

Outages are detected at the component level. A component represents any hardware or software entity active in the environment that can have its status sent to the Tivoli Enterprise Console Server by way of Tivoli Enterprise Console events. Components are instances of component types. For example, alpha.ibm.com is an instance of the component type Node. There can be many components defined for one component type. Components are often grouped to represent high-level systems that have a highly visible impact on company operations. Examples are order processing, accounting and payroll. Usually, the component types for the components within these systems have hierarchical relationships. For example, an order processing system can consist of a front-end Web server, a servlet engine, several servlets, and a backend database. This implicitly includes the nodes where the Web server and the servlet engine run. Assuming the Web server and the servlet engine run on the same node, the component type relationships would be as follows:

Node Web Server Servlet Engine Order Servlet Database

At a systems level, an outage in the order processing system could be caused by any component represented in the relationship; therefore, it is important to understand the relationship between components types.

Outages can be detected in single components or systems of components. The systems can be static or dynamic in nature. Static implies that the components in the system rarely change and that they always reside on the same machine. An example of this is a Line of Business (LOB). A dynamic system is one that is not well defined and changes frequently. An example of this is a network that has nodes that are routinely added or removed.

Tivoli Enterprise Console Warehouse Implementation Guide 14

Page 20: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

3.9.2 Detecting availability outages The Tivoli Enterprise Console server has several means for detecting outages for both static and dynamic systems:

• From a specially defined single event that contains outage start time and duration information

• From AVAIL_Outage events sent directly in to the event server

• From a series of events that are interpreted as changes between various outage states

• From Tivoli Enterprise Console missed heartbeat events

3.9.3 Pre-configured rules The warehouse pack is delivered with outage facts capable of detecting HP Openview outages for nodes, segments and networks. For nodes, the rules determine when a node is Up, Down or Marginal. Node instances are detected dynamically. For segments and networks, the rules determine when the segment or network becomes marginal or critical. Segments and networks are defined statically in the fact file, warehouse.pro. When any HP Openview outage is detected a class AVAIL_Outage event is generated for later ETL extract processing. As these rules are dependent on the OV events, the tecad_ov.baroc is a prereq for this capability and must be included in the rule base for this to work. There are also facts for detecting ITM engine and Heartbeat outages. The outages depend upon the new TEC_Heartbeat_missed event class. The beginning of an outage occurs when a TEC_Heartbeat_missed event is generated. This event is generated in two cases: • By the heartbeat.rls ruleset, when an element being monitored by heartbeat is detected as down • By the ebusiness.rls ruleset, when ITM outage events like: HeartBeat_Off, HearBeat_EndpointUnreachable are detected. For more details on these rules please see the Enterprise Console Rule Set Reference. Eventually, this event is closed by the heartbeat or ebusiness rulesets, which signals that the outage is over. A change rule is used to reanalyze the event after it is closed. When it is analyzed, the AVAIL_Outage event is generated. As these rules are dependent on the TEC_Heartbeat_missed event, the TEC3.9 tec.baroc as well as heartbeat.rls and ebusiness.rls are prereq for this capability and must be included in the rule base for this to work. Finally, you can generate an outage from a single event defined statically in the fact file. An event’s class and a set of the event’s attributes are defined that classify that type of an event as one that defines an outage. The event’s component, component type and outage state are defined with rules predicates. When an event of the defined class with matching values for the specified attributes is processed by the rules an AVAIL_Outage event is generated.

3.9.4 Data requirements for availability outages Availability meta data, the definition of component types, components, measurement groups and measurement states, can best be defined in the rule fact files. Some meta data can be defined by sending events of classes AVAIL_Relationship and AVAIL_Comp_Type_State which update rule fact files. A combination of the two methods may be used to define meta data but some predicates define rules processing functionality that cannot be accomplished by sending in availability events.

In order for the ITSLA reports to be generated properly the following requirements must be adhered to when defining outage parameters.

• Component types must be defined before any other event references it

• Components must be defined before any other event references it

• Component types must have defined measurement states in the Tivoli Enterprise Data Warehouse MsmtRul table

Tivoli Enterprise Console Warehouse Implementation Guide 15

Page 21: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

• Measurement groups must be defined for each set of measurement states component types can have. For example, if you have the following component types and allowed measurement states for them:

CompTypA Degrading, Unavailable

CompTypB Degrading, Unavailable

CompTypC Unavailable

Then you would need two measurement groups defined similar to the following:

MsrGp1 Degrading, Unavailable

MsrGp2 Unavailable

ITSLA reporting uses these definitions to classify the outage data.

• Measurement groups must have defined measurement states in the MGrpMbr table

• Prolog facts must be implemented for rules processing to interpret availability outages

Note: See , section 3.3.4 to understand the limitations of this warehouse enablement pack when defining component instance names.

Component instance names must be unique

3.9.4.1 Defining meta data by sending in events Only the AVAIL_Relationship and AVAIL_Comp_Type_State event classes dynamically store meta data in the fact files. This can be useful if all the outage events for a particular component will always be sent in using an AVAIL_Outage event. The fact files are not updated dynamically by other meta data events. The measurement groups meta data is not required for the rules to process AVAIL_Outage events, but it is required, though, that events defining the measurement group meta data be sent in to properly define the meta data in the warehouse tables for the ITSLA reporting to work. Unless the necessary measurement groups and their allowed measurement states are already defined, the classes that must be sent in are the AVAIL_Measurement_Group_Type and AVAIL_Measurement_Group_State_Type classes.

An AVAIL_Outage event fails rules’ processing if the component type, component instance and the allowable component type states are not defined in the fact files by one of the available methods. These requirements are checked as part of validating an AVAIL_Outage event. When the meta data is defined by sending in events and not in the fact files, only AVAIL_Outage events can be used to report outages on components because the necessary definitions to correlate a series of outage state transitions would not be defined.

Note: Classes of AVAIL_Comp_Type_Relationship and AVAIL_Comp_Relationship should only be used by the fact files to generate classes of this type. They should not be sent in as events because the fact files will not be updated with information sent in with these events. Only the AVAIL_Relationship events can be used to define component types and components when defining meta data by sending in events.

Event classes that dynamically update the fact files are:

• AVAIL_Comp_Type_State

• AVAIL_Relationship

Multiple events with the same information do not cause duplicate entries in the fact files nor in the warehouse tables.

3.9.4.2 Defining meta data in the fact files If the fact files are manually updated, then all meta data is defined in the fact files and the rules generate events of the appropriate classes when the event server is restarted. The generated events are validated by the rules and, if they pass validation, they are inserted into the event repository and are later extracted by the ETL process and used to update the warehouse tables. The ETL process filters out any duplicate entries so multiple restarts of the event server does not cause duplicate entries in the warehouse tables.

The list of event classes that can be generated by the fact files at TEC server start up are:

Tivoli Enterprise Console Warehouse Implementation Guide 16

Page 22: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

• AVAIL_Comp_Type_Relationship

• AVAIL_Comp_Type_State

• AVAIL_Measurement_Group_Type

• AVAIL_Measurement_Group_State_Type

• AVAIL_Comp_Relationship

• AVAIL_Error

Meta data defined dynamically through AVAIL_Relationship and AVAIL_Comp_Type_State events does not allow the same functional capabilities as when it is defined using the provided predicates in the fact files. The predicates allow for the definition of component type domain states, allowable state transitions and dynamic detection of new component instances. When any of these functionalities are desired, or when outages are determined by methods other than the sending of an AVAIL_Outage event, then the meta data relationships must be defined using the predicates and fact files. Only the definitions in the fact files can correlate outage information that spans multiple events or missed heartbeat events. The fact files can also process AVAIL_Outage events.

3.9.5 Availability BAROC classes defined Following are the new BAROC classes defined for use with this warehouse pack. Each class is described and length limits for the slot values are defined. The length limits are limits imposed by the size of the columns in the Tivoli Enterprise Data Warehouse where the values are inserted. See section 3.9.8.2 for a table detailing the allowable columns lengths. The values assigned to a slot are case sensitive.

When events are extracted by the warehouse pack they are grouped and processed in the following order:

• Component types • Component type allowed states • Components • Measurement groups • Measurement groups allowed states • Outage events

This order enables component types to be defined before components that reference them when all the new values are in the same set of events being extracted and not already in the warehouse tables. When a parent and child value are defined in the same set of events currently being extracted, it is up to you to guarantee that the parent value is defined in an event that precedes the child value that requires it.

NOTE: Even if your warehouse database is internationalized, the measurement state values MUST be entered in English as these values are only translated on the reports but are created in the database only in English.

3.9.5.1 Component type BAROC definitions

The AVAIL_Comp_Type_Relationship class is an internally generated event only. It is generated either by the data in the fact files when the TEC server is restarted or by the arrival and processing of an AVAIL_Relationship event. The new component type is defined in the child_comp_type slot. A description of what the component type represents is required. The parent_comp_type slot should already exist or should be defined by an event in the same set of extracted events as the child_comp_type slot but with an earlier date_reception so it is processed first.

Tivoli Enterprise Console Warehouse Implementation Guide 17

Page 23: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

The new component type stored in the child_comp_type slot can only contain 17 or fewer characters. This is true for the parent_comp_type slot value as well. The child_comp_type_descr slot can contain 120 characters or less. TEC_CLASS:

AVAIL_Comp_Type_Relationship ISA EVENT DEFINES { parent_comp_type: STRING; child_comp_type: STRING; child_comp_type_descr: STRING; }; END The AVAIL_Comp_Type_State class is used to define the measurement states a component type can change to. This class is caught by rules’ processing to dynamically update the fact files and then it is inserted into the event tables for later ETL extraction and processing into the warehouse. The possible states are in the warehouse MsmtTyp table where the MSrc_Cd column is ‘MODEL1’. See section 7.2.8 Measurement type (table MsmtTyp) for a list of the possible state transition values. The state slot must contain one of the state transition values from this table and the value must be in English to match the value in the MsmtTyp table. It is required to define the states that a component type can change to. When you send events of this class, you can only specify one component type and allowable state per event. If you want to define three possible transition states for one component type, you need to send three AVAIL_Comp_Type_State events to define the three allowable states. See “ ” on Page 51 for examples of this. Central data warehouse information

Central data warehouse information

The component type stored in the slot comp_type can contain 17 or fewer characters. The state slot will contain one of the possible state choices from the MsmtTyp table so the maximum length of the slot is not a an issue. TEC_CLASS: AVAIL_Comp_Type_State ISA EVENT DEFINES { comp_type: STRING; state: STRING; }; END

3.9.5.2 Measurement group BAROC definitions The AVAIL_Measurement_Group_Type class is used to define new measurement group names in the warehouse tables. A description of what the measurement group represents is required.

The new measurement group name stored in the slot code can only contain 6 or fewer characters. The description can contain 120 characters or less. TEC_CLASS: AVAIL_Measurement_Group_Type ISA EVENT DEFINES { code: STRING; descr: STRING; }; END The AVAIL_Measurement_Group_State_Type class is used to define, in the warehouse tables, the measurement states a measurement group can have. The possible states are in the warehouse MsmtTyp table where the MSrc_Cd column is ‘MODEL1’. See section 7.2.7, Measurement type (table MsmtTyp) for a list of the possible state transition values. The state slot must contain one of the state transition values from this table and the value must be in English to match the value in the MsmtTyp table. It is required to define the states a measurement group can change to. When you send events of this class you can only specify one measurement group and allowable state per event. If you want to define three possible transition states for one measurement group you would need to send in three AVAIL_Measurement_Group_State_Type events to define the three allowable states. See “ ” on Page 51 for examples of this.

Tivoli Enterprise Console Warehouse Implementation Guide 18

Page 24: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

The new measurement group name stored in the slot code can only contain 6 or fewer characters. The state slot will contain one of the possible state choices from the MsmtTyp table so that the maximum length of the slot is not a problem. TEC_CLASS: AVAIL_Measurement_Group_State_Type ISA EVENT DEFINES { code: STRING; state: STRING; }; END

3.9.5.3 Component BAROC definitions The AVAIL_Comp_Relationship class is an internally generated event only. It is generated either by the data in the fact files when the TEC server is restarted or by the arrival and processing of an AVAIL_Relationship event. The new component is defined in the child_comp slot. A description of the component in the child_comp_descr slot is not required but if you decide not to include a description enter the string value “NULL”. The parent_comp_type and child_comp_type slots should already exist or should be defined by events in the same set of extracted events as the child_comp but with an earlier date_reception so they are processed first.

The new component stored in the slot child_comp can contain 254 or fewer characters. This is true for the parent_comp slot value as well. The parent_comp_type and child_comp_type slots can be of length 17 or less. The child_comp_descr slot can contain 254 characters or less.

TEC_CLASS: AVAIL_Comp_Relationship ISA EVENT DEFINES { parent_comp_type: STRING; parent_comp: STRING; child_comp_type: STRING; child_comp: STRING; child_comp_descr: STRING; }; END

3.9.5.4 List_Of component and component type BAROC definitions The AVAIL_Relationship class is used to define a hierarchy of new component types and components. The number of items in each list must be the same. Send in multiple of these events to define multiple components for a component type. When the rule engine processes an event of this class it parses the list of values and generates events of the classes AVAIL_Comp_Type_Relationship and AVAIL_Comp_Relationship for the warehouse pack extract to process and create new component types and new components. You will need to send additional events of class AVAIL_Comp_Type_State to define the measurement states the component types can have. The AVAIL_Comp_Type_State events should arrive after the event of class AVAIL_Relationship so the component types exist before a state transition value is defined for them. You will also need to send additional events of classes AVAIL_Measurement_Group_Type and AVAIL_Measurement_Group_State_Type if the new component types allowable measurement states define a new, unique combination of measurement states. This class is caught by rules processing to update the fact files dynamically and then is inserted into the event tables for later ETL extraction and processing into the warehouse.

The class should be used to define your hierarchical relationships. It is assumed that the top-most component type has no parent. Do not use this class to define a random set of component types and components because the list implies a hierarchy of relationships. For example, assume we provide the following values in an event:

ct1, ct2

ct1_descr, ct2_descr

cp1, cp2

Tivoli Enterprise Console Warehouse Implementation Guide 19

Page 25: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

cp1_descr, cp2_descr

In this example ct1 has no parent component type, ct1_descr is the description, cp1 is a component instance of the ct1 component type, and cp1_descr is the description for component cp1. The parent component type of ct2 is ct1 and its description is in ct2_descr. A component instance definition of cp2 is defined for a component type ct2 and its description is cp2_descr. It has a parent component instance of cp1.

New component types are defined in the comp_types slot and should be entered in hierarchical order with the top-most parent component type defined first and each component type separated by commas. A description of each component type provided in the same order as the component types is required in the comp_type_descrs slot, each description separated by commas. The comp_instances slot contains the component instances that should be defined for each of the component types defined in the comp_types slot. Again, each comp_instance should be defined in the same order as the component types and separated by commas. A description of each component provided, in the same order as the components, is required in the comp_descrs slot, with each description separated by commas.

Each new component type stored in the comp_types slot can be 17 or fewer characters. Each description in the comp_type_descrs slot can contain 120 characters or less. Each new component stored in the comp_instances slot list can be 254 characters or less and each description in the comp_descrs slot can also contain 254 characters or less.

Tivoli Enterprise Console Warehouse Implementation Guide 20

Page 26: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

TEC_CLASS: AVAIL_Relationship ISA EVENT DEFINES { comp_types: LIST_OF STRING; comp_type_descrs: LIST_OF STRING; comp_instances: LIST_OF STRING; comp_descrs: LIST_OF STRING; }; END

3.9.5.5 Outage events BAROC definitions The AVAIL_Outage class is used to define outage events. An event of this type can be sent to the Tivoli Enterprise Console event server to be passed directly to the extract processing. Alternatively, some events cause an outage event to be generated when a state transition has occurred. The rules can catch events indicating a state change has occurred and then wait for an event that says the state has changed again. The rules correlate the two events and generate an AVAIL_Outage event after the second event has been received that indicates the state transition has occurred. The generated event will be given a reformatted date_reception value of the first event as the outage_start_time and the difference between the arrival time of the two events will be the outage_duration time. The rules can only generate AVAIL_Outage events when all the meta data and allowable state transitions are manually defined in the fact files.

The comp_type, comp_instance and the component type’s state must already be defined before an event of this type can be processed successfully by the rules. The comp_type and comp_instance values should already be defined. The component type stored in the comp_type slot can contain 17 or fewer characters. The comp_instance slot can contain 254 characters or less. The state_became slot must be one of the values in the warehouse MsmtTyp table where the MSrc_Cd column is ‘MODEL1’. See section 7.2.7, Measurement type (table MsmtTyp) for a list of the possible state transition values. The state_became slot must contain one of the state transition values from this table and the value must be in English to match the value in the MsmtTyp table. The outage_duration slot must be an integer and is the number of minutes the outage lasted. The outage_start_time is the time the state of the component changed initially and must be in GMT time with the following ISO format: yyyy-mm-dd.hh.mm.ss.nnnnnn

where

Trailing blanks can be included Leading zeroes can be omitted from the month, day, or hour Microseconds can be truncated or eliminated Hours are based on the 24-hour clock TEC_CLASS: AVAIL_Outage ISA EVENT DEFINES { outage_start_time: STRING; outage_duration: INTEGER; state_became: STRING; comp_type: STRING; comp_instance: STRING; }; END

3.9.5.6 Availability error BAROC definitions The AVAIL_Error class is used to send error messages to the event server when errors are found in the availability events received or generated. The message text is in the msg slot of the event. The rules check and generate error messages for the following errors:

Tivoli Enterprise Console Warehouse Implementation Guide 21

Page 27: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

• The max length of a value has been exceeded

• The outage_start_time slot has an improperly formatted date value

• A parent component type, component type state or parent component is not defined before an outage event using them arrives

• The length of each list specified in the AVAIL_Relationship event’s slots is not the same

TEC_CLASS: AVAIL_Error ISA EVENT DEFINES { severity: default = WARNING; }; END

3.9.6 Defining outage parameters in prolog facts Before an outage can be detected when defining Availability data in the prolog facts, you the user must define a set of parameters that establish the hierarchy of system components to be monitored for state transition outages. The parameters include:

• Component type definitions

• Component definitions

• Allowable measurement states for each component type

• Definition of measurement groups

• Definition of allowable states for each measurement group

These parameters are made available to the Tivoli Enterprise Console server using prolog facts loaded from a user-defined predicate file called warehouse.pro. The following sections describe the steps to use in order to define and effectively detect outages using fact files in the Tivoli Enterprise Console server.

3.9.7 Analyze operational environment The following steps are used to develop outage facts for the Tivoli Enterprise Console server. Worksheets are provided in Appendix A.

3.9.7.1 Identify systems that require outage detection First, all individual components or systems of components that require outage detection must be identified. Each component must belong to a defined component type. Multiple components can belong to a single component type. The component type is a string that must be unique among all component types and cannot contain spaces. An associated description is required for each component type. This description will be on the ITSLA reports, so they should be descriptive. A batch jobs example is shown below. The worksheet for this information can be found in Appendix A.1.

Component Type Description

MVS_ SYSTEM MVS System

PR1_BATCH_JOBS Batch Jobs

Table 3.1 Component Types

Tivoli Enterprise Console Warehouse Implementation Guide 22

Page 28: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

Next, the relationships between the component types (if any) must be specified. These relationships are parent/child associations. They represent the order of direct interaction between the component types. If a component type has no parent, use the “NULL” string value. The component type relationships for the batch jobs scenario are shown below. The worksheet for this information can be found in Appendix A.2.

Parent Child

NULL MVS_SYSTEM

MVS_SYSTEM PR1_BATCH_JOBS

Table 3.2 Component Type Relationships

3.9.7.2 Identify measurement information Each component type must have measurement states defined for it. The measurement states are: Available, Unavailable, Unreachable, Unmanaged, Degrading, Responding, Repairing, Closing or Unknown. Because the Tivoli Enterprise Console rules detect outages, the Available state is generally not used. A component is assumed Available if it is not in one of the other states. The worksheet for this information can be found in “

” on Page 62. The following states are defined for the batch jobs example: Allowable warehouse

component states Component Type Allowable States MVS _SYSTEM Degrading, Unavailable PR1_BATCH_JOBS Unavailable

Table 3.3 Allowable Warehouse Component States

Measurement group codes must be defined for each component type with a unique set of measurement states. The code name should be representative of the functionality of the collection of components. The description will be on the ITSLA reports, so it should clearly describe the functionality. The worksheet for this information can be found in Appendix A.4.

Measurement Group Code Description

BJ1 The batch jobs msmt for D, U states.

BJ2 The batch jobs msmt for U states.

Table 3.4 Measurement Groups

Tivoli Enterprise Console Warehouse Implementation Guide 23

Page 29: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

Each measurement group must have measurement states defined for it. The group states for the batch jobs example are shown below. The worksheet for this information can be found in Appendix A.5.

Measurement Group Code Measurement Group States

BJ1 Unavailable

BJ1 Degrading

BJ2 Unavailable

Table 3.5 Measurement Group States

3.9.7.3 Identify component information After the component types and measurement group parameters are specified, the components must be defined with an associated description. C within their respective component type. Refer to

, section 3.3.4 for a description of this limitation. A good practice is to include the name of the machine where the component runs and the type of component in the component name. If more than one component of the same type is running on the same machine, append a delineator such as 1 to the name (the component names for two Payroll batch jobs running on MVS system S033 could be S033_1_Payroll and S033_2_Payroll). For clarity, each component should be listed with its associated type. Components for the batch jobs scenario are shown below. The worksheet for this information can be found in Appendix A.6.

omponent instance names must be uniqueComponent instance names must be unique

Component Component Type Description

S033 MVS_SYSTEM MVS system S033

Payroll PR1_BATCH_JOBS Payroll job on S033

Parts PR1_BATCH_JOBS Parts job on S033

S081 MVS_SYSTEM MVS System S081

Parts2 PR1_BATCH_JOBS Parts job on S081

Table 3.6 Components

Next, the relationships between the components, if any , must be specified. These relationships are parent and child associations and must correspond to the defined component type relationships. The relationships represent the order of direct interaction between the components. If a component has no parent, use the “NULL” string value. The component relationships for the batch jobs scenario are shown below. The worksheet for this information can be found in “ ” on Page 66. Component relationships

Parent Child

NULL S033

S033 Payroll

S033 Parts

S081 Parts2

Table 3.7 Component Relationships

Tivoli Enterprise Console Warehouse Implementation Guide 24

Page 30: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

3.9.7.4 Component type domain states

Each component is located in its own logical domain. Nomenclature throughout the different component domains can vary widely. For instance, a node is generally talked about as being up or down to indicate whether it is functioning normally or not. On the other hand, a network is considered Normal, Critical or Marginal, depending on whether it is operating correctly, has failed, or is operating at reduced capacity, respectively. These are different nomenclatures that represent similar ideas. The warehouse pack supports eight states: Available, Unavailable, Unreachable, Unmanaged, Degrading, Responding, Repairing or Closing. These states do not map to component type domain states. To create outage information for a component, the domain states must be defined in terms of the domain nomenclature, then these states must be mapped to the warehouse pack states. For the batch jobs example, the domain states are defined below. The worksheet for this information can be found in “ ” on Page 67 Domain states

Component Type Domain State

MVS_SYSTEM up, marginal, down

PR1_BATCH_JOBS up, down

Table 3.8 Domain States

Because the Tivoli Enterprise Console server detects conditions of interest using events, each of the previous domain states needs to be specified in terms of the event criteria that can detect the condition. This includes specifying an event class name and a corresponding set of attribute values that indicate when the condition has occurred. The worksheet for this information can be found in “ ” on Page 68. This is presented in Table 3.9 for the batch jobs example, all others follow a similar pattern. Also, the class name attributes are examples:

Event criteria for condition detection

Tivoli Enterprise Console Warehouse Implementation Guide 25

Page 31: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

Component Type Domain State Event Class Attribute Condition

MVS_SYSTEM up TEC_Notice msg=SystemUp,

source=MVS_Monitor

MVS_SYSTEM down TEC_Notice msg=SystemDown

source= MVS_Monitor

MVS_SYSTEM marginal TEC_Notice msg=SystemMarginal

source= MVS_Monitor

PR1_BATCH_JOBS up TEC_Notice msg=JobUp,

source=BJ_Monitor

PR1_BATCH_JOBS down TEC_Notice msg=JobDown,

source=BJ_Monitor

Table 3.9 Event Criteria for Condition Detection

3.9.7.5 Allowable state transitions The condition of a component changes each time it makes a transition between domain states. This is considered an outage. When this occurs, the appropriate Tivoli Data Warehouse state is generated as a Tivoli Enterprise Console event. Only certain state transitions are allowed for a component. Some states are achieved by a direct transition from the normal state. This is considered the first position in a sequence of states. The state just prior to a direct transition to the normal state is considered the last position in a sequence. All other positions are considered intermediate. Component types that have a single state use the last position. The worksheet for this information can be found in “ ” on Page 69. The allowable domain state transitions and their associated resulting Tivoli Data Warehouse state for the batch jobs system is as follows:

Domain state transitions

Component Type Domain State 1 Domain State 2 Warehouse State Position

MVS_SYSTEM Marginal Down Degrading First

MVS_SYSTEM Marginal Up Degrading Last

MVS_SYSTEM Down Up Unavailable Last

MVS_SYSTEM Down Marginal Unavailable First

PR1_BATCH_JOBS Down Up Unavailable Last

Table 3.10 Domain State Transitions

3.9.7.6 Detecting instances dynamically Under certain circumstances, instances that require outage detection are not known when the rules are deployed. An example of this is when Nodes are added or deleted from a network dynamically. This is an example of a dynamic system as discussed in “ ” on Page 14. For this case, event criteria are used in conjunction with event attribute names that uniquely identify the component instance to dynamically detect component instances. When an event is received, the values of the attributes are extracted from the event. If no instances are found that have the key value based upon the extracted attributes, a component instance is added to the internal engine fact base along with the component description defined in the event criteria. See predicates

Defining availability outages

Tivoli Enterprise Console Warehouse Implementation Guide 26

Page 32: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

create_event_criteria and detect_dynamic_instance implementation examples in “ ” on Page 27, for examples of how this is implemented. Event criteria and predicates are explained in the “Tivoli Enterprise Console Rule Developer’s Guide” along with the concepts required to understand implementing rules and predicates in a Tivoli Enterprise Console rule base.

Implementation

3.9.8 Implementation

There are no dynamic instances in the batch jobs example; however, the following table is representative of the information that needs to be collected during the analysis phase. The worksheet for this information can be found in Appendix A.11.

Event Criteria Attributes(s) Component Type New Instance Component Description

Heartbeat_up (predefined event criteria)

fqhostname Heartbeat Component Monitored by Heartbeat

Table 3.11 Event Criteria for Dynamic Component Instances

In order to implement outage detection, you must write a predicate for each value to be defined. It can be added to the warehouse.pro file or added to a separate file. If a separate file is used, it must be consulted using the init_outage predicate located in the warehouse.pro file. The following section describes the predicates that are used to specify outage parameters:

3.9.8.1 Outage related predicates All parameters for each predicated should be filled in with values to properly define the outage data.

comp_type_relationship(parent_comp_type, child_comp_type, child_comp_type_descr)

where

parent_comp_type – the type of the parent component. “NULL” if there is no parent child_comp_type – the type of the child component child_comp_type_descr – description of the child component type

Use this predicate as many times as necessary to complete the parent/child sequence and define the child_comp_type. Parents must be defined before children. This predicate contains the information used to generate events of the AVAIL_Comp_Type_Relationship class discussed in “

” on Page 17 . Refer to “ ” on Page 22, for a description of what is needed and tables with sample data.

Component type BAROC definitions

Component type BAROC definitions

Identify systems that require outage detection

allowable_state(component_type, allowable_state)

where

component_type – a previously defined component type allowable_state – one of the eight allowable states for Tivoli Data Warehouse components

Use this predicate as many times as necessary to describe the allowable states for each component type. This predicate contains the information used to generate events of the AVAIL_Comp_Type_State class discussed in section “ ” on Page 17. Refer to “ ” on Page 23, for a description of what is needed and tables with sample data

Identify measurement information

Tivoli Enterprise Console Warehouse Implementation Guide 27

Page 33: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

measurement_group(component_group_code, component_group_code_description)

where

component_group_code – logical name for a group of components types with like measurement states

component_group_code_description – associated component type measurement group description

Use this predicate as many times as necessary to define measurement groups and their descriptions. This predicate contains the information used to generate events of the AVAIL_Measurement_Group_Type class .

measurement_group_state(component_group_code, allowable_state)

where

component_group_code – corresponds to a predefined component group code allowable_state – one of the eight allowable states that components in this measurement group can have

Use this predicate as many times as necessary to describe the allowable states for each measurement group. This predicate contains the information used to generate events of the AVAIL_Measurement_Group_State_Type class discussed in “ ” on Page 17. Refer to section “

” on Page 23, for a description of what is needed and tables with sample data. Measurement group BAROC definitions Identify

measurement information

comp_relationship(parent_comp, parent_comp_type, comp, comp_type, comp_descr)

where

parent_comp – the parent component for the component defined in the comp parameter

parent_comp_type – the type of the parent component. Must be a predefined type.

comp – a unique identifier for the component

comp_type – the type of the component. Must be a predefined type

comp_descr – the description of the component

Use this predicate as many times as necessary to complete the parent/child component sequence and define the comp_type. Parents must be defined before children. This predicate contains the information used to generate events of class AVAIL_Comp_Relationship discussed in “ ” on Page 19. Refer to “ ” on Page 24, for a description of what is needed and tables with sample data.

Component BAROC definitionsIdentify component information

outage_transition(comp_type, start_domain_state, next_domain_state, warehouse_state, position)

where

comp_type – a previously defined component type start_domain_state – the start state for the component type next_domain_state – the next state that results in an outage warehouse_state – the outage state achieved by the transition between the two states position – clarifies the first and last position from those in between

This predicate should be filled out with the values for your component types similar to the examples in Table 3.10, Domain state transitions. This defines the allowable transitions a component might go through before reaching the Normal, or Available, state again. Refer to “ ” on Page 26 for a description of what is needed and tables with sample data. This predicate is used to define rules processing that

Allowable state transitions

Tivoli Enterprise Console Warehouse Implementation Guide 28

Page 34: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

interprets availability data and does not have a comparable BAROC class. This information must be defined using predicates.

create_event_criteria(crit_name, event_class, fire_on_non_leaf_node, attribute_crit)

This predicate is documented in the IBM Tivoli Enterprise Console Rule Developer’s Guide. It is listed here for completeness.

outage_state(event_criteria_name, domain_state, component, component_type)

where

event_criteria_name – the name of the event criteria used to identify this state domain_state – name of the domain state component – component name to which this state applies component_type – the type of the component

This predicate is used to identify the various domain states that a component type can have and define them in the event criteria attributes. Refer to “ ” on Page 25 for a description of what is needed and tables with sample data. This predicate is used to define rules processing that interprets availability data and does not have a comparable BAROC class. This information must be defined using predicates.

Component type domain states

detect_dynamic_instance(event_criteria_name, key_attributes, comp_type, comp_descr)

where

event_criteria_name – name of the event criteria used to detect the dynamic instance key_attributes – name of attributes that delineate a unique instance comp_type – the type of the component comp_descr – the description of the dynamically created component instance.

An Availability rule compares every event to the criteria added by the detect_dynamic_instance predicate and, if a match is found, a new component instance is created. Refer to “ ” on Page 26, for a description of what is needed and tables with sample data. This predicate is used to define rules processing that interprets availability data and does not have a comparable BAROC class. This information must be defined using predicates.

Detecting instances dynamically

Tivoli Enterprise Console Warehouse Implementation Guide 29

Page 35: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

3.9.8.2 Parameter length constraints Certain parameters have constrained lengths. These constraints are shown in Table 3.12 below.

Parameter Required formatting

component_types 17 characters or less

component_types_descr 120 characters or less

component 254 characters or less

component_descr 254 characters or less

state 120 characters or less (predefined value from the MsmtTyp table)

measurement_code 6 characters or less

measurement_descr 120 characters or less

outage_start_time yyyy-mm-dd-hh.mm.ss.nnnnnn

state_became 120 characters or less

outage_duration Integer values only

Table 3.12 Required Formatting for Availability Event Slot Names

The outage_start_time should be in GMT time and should be specified using the ISO format as follows:

yyyy-mm-dd-hh.mm.ss.nnnnnn

where

• Trailing blanks can be included • Leading zeroes can be omitted from the month, day, or hour • Microseconds can be truncated or eliminated • Hours are based on the 24-hour clock

3.9.8.3 Customizing the predicate file with custom outage definitions The final step is to customize the predicate file, warehouse.pro. This file contains the facts that describe how to detect an outage for each component. The listing below shows the predicate file customizatons done for the batch jobs example. After customizing the predicate file it is necessary to recompile and reload the rule base and then stop and restart the event server to put the changes into effect.

First, add the call to initialize your custom facts after the line in the warehouse.pro file as seen in bold below. Our example below shows the call for ‘batch jobs’ after the bold line.

%-------------------------------------------------------------- % Initialize Outage parameters. %-------------------------------------------------------------- init_outage(_persistent_file):- % Turn startup mode on. begin_startup_mode(_persistent_file),

% <<<< Add all user-defined facts here <<<<

Tivoli Enterprise Console Warehouse Implementation Guide 30

Page 36: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

init_batch_jobs, % Single Event Outage Example init_single_event_outage, % Call initialization routine for each outage type. init_hpov, % Initialize heartbeat related outages init_heartbeat('Heartbeat'), % Turn off startup mode. end_startup_mode. Then add your custom definitions before the line in warehouse.pro that says:

% Outage related predicates <<< Do not edit below this line. <<< The following code is the custom definition for the facts for the batch jobs example.

init_batch_jobs:- %--------------------------------------------------------------------

% Example Single event type definition % % Argument List: Parent, Child_Comp, Descr, % % NOTE: This information is a combination of the data collected in % Worksheets A.1 & A.2 and instructions concerning gathering this % information can be found in Section 3.9.7.1, Identify systems % that require outage detection. %-------------------------------------------------------------------- comp_type_relationship('NULL', 'MVS_SYSTEM', 'MVS System'), comp_type_relationship('MVS_SYSTEM', 'PR1_BATCH_JOBS', 'Batch Jobs'), %-------------------------------------------------------------------- % Component Type State Declarations % % Argument List: Comp_Type, State % % NOTE: This information comes from the data collected in Worksheet A.3 % and instructions concerning gathering this information can be % found in Section 3.9.7.2, Identify measurement information. %-------------------------------------------------------------------- allowable_state('MVS_SYSTEM', 'Degrading'), allowable_state('MVS_SYSTEM', 'Unavailable'), allowable_state('PR1_BATCH_JOBS', 'Unavailable'), %-------------------------------------------------------------------- % Measurement Group Declarations % % Argument List: Code, Description % % NOTE: This information comes from the data collected in Worksheet A.4 % and instructions concerning gathering this information can be % found in Section 3.9.7.2, Identify measurement information. %-------------------------------------------------------------------- measurement_group('BJ1', 'The batch jobs msmt system for D, U states.'), measurement_group('BJ2', 'The batch jobs msmt system for U states.'), %-------------------------------------------------------------------- % Allowable Measurements for measurement group

Tivoli Enterprise Console Warehouse Implementation Guide 31

Page 37: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

% % Argument List: Code, State % % NOTE: This information comes from the data collected in Worksheet A.5 % and instructions concerning gathering this information can be % found in Section 3.9.7.2, Identify measurement information. %-------------------------------------------------------------------- measurement_group_state('BJ1', 'Unavailable'), measurement_group_state('BJ1', 'Degrading'), measurement_group_state('BJ2', 'Unavailable'),

%-------------------------------------------------------------------- % Define component definitions for the instances above. % They should be specified, in order, from parent to child nodes. % % Argument List: Parent_Comp, Parent_Type, Child_Comp, Child_Type, Description % % NOTE: This information is a combination of the data collected in % Worksheets A.6 & A.7 and instructions concerning gathering this % information can be found in Section 3.9.7.3, Identify component % information. %-------------------------------------------------------------------- comp_relationship('NULL', 'NULL', ‘S033', 'MVS_SYSTEM',‘NULL'), comp_relationship('S033', 'MVS_SYSTEM', ‘Payroll’, 'PR1_BATCH_JOBS', 'Payroll job on S033'), comp_relationship('S033', 'MVS_SYSTEM', ‘Parts’, 'PR1_BATCH_JOBS', 'Parts job on S033'), comp_relationship('NULL', 'NULL', ‘S081’, 'MVS_System', 'NULL'), comp_relationship('S081', 'MVS_System', ‘Parts2’, 'PR1_BATCH_JOBS', 'Parts Job on S081'),

%-------------------------------------------------------------------- % Outage Criteria % % Argument List: Type, state_1, state_2, warehouse_state, position % % NOTE: This information comes from the data collected in Worksheet A.10 % and instructions concerning gathering this information can be % found in Section 3.9.7.5, Allowable State Transitions. %-------------------------------------------------------------------- outage_transition('MVS_SYSTEM', marginal, down, 'Degrading', first), outage_transition('MVS_SYSTEM', marginal, up, 'Degrading', last), outage_transition('MVS_SYSTEM', down, up, 'Unavailable', last), outage_transition('MVS_SYSTEM', down, marginal, 'Unavailable', first), outage_transition('PR1_BATCH_JOBS', down, up, 'Unavailable', last),

...

Similar outage states for other components

...

%-------------------------------------------------------------------- % Define the event criteria. % % Argument List: Criteria_Name, Event_class, Fire_on_Leaf, Attr_Criteria % % NOTE: The Criteria_Name MUST be unique. % % NOTE: This information comes from the data collected in Worksheet A.9

Tivoli Enterprise Console Warehouse Implementation Guide 32

Page 38: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

% and instructions concerning gathering this information can be % found in Section 3.9.7.4, Component Type Domain States. %-------------------------------------------------------------------- create_event_criteria(custom_MVS_up, 'TEC_Notice', no, [[‘msg’, equals, 'MVSUp'], [‘source’, equals, 'MVS_Monitor']]), create_event_criteria(custom_MVS_down, 'TEC_Notice', no, [[‘msg’, equals, 'MVSDown'], [‘source’, equals, 'MVS_Monitor']]), create_event_criteria(custom_MVS_marginal, 'TEC_Notice', no, [[‘msg’, equals, 'MVSMarginal'], [‘source’, equals, 'MVS_Monitor']]), create_event_criteria(custom_job_up, 'TEC_Notice', no,

[[‘msg’, equals, 'JobUp'],

[‘source’, equals, 'JOB_Monitor']]),

create_event_criteria(custom_job_down, 'TEC_Notice', no,

[[‘msg’, equals, 'JobDown'],

[‘source’, equals, 'JOB_Monitor']]),

...

Similar event criteria for other component states

...

%-------------------------------------------------------------------- % Outage Criteria % Argument List: event_criteria, state, instance, instance_type % % NOTE: This information comes from the data collected in Worksheet A.8 % and instructions concerning gathering this information can be % found in Section 3.9.7.4, Component Type Domain States. %-------------------------------------------------------------------- outage_state(custom_MVS_up, up, 'S033', 'MVS_SYSTEM'), outage_state(custom_MVS_down, down, 'S033', 'MVS_SYSTEM'), outage_state(custom_MVS_marginal, marginal, 'S033', 'MVS_SYSTEM'), outage_state(custom_MVS_up, up, 'S081', 'MVS_SYSTEM'), outage_state(custom_MVS_down, down, 'S081', 'MVS_SYSTESM'), outage_state(custom_MVS_marginal, marginal, 'S081', 'MVS_SYSTEM'), outage_state(custom_job_up, up, 'Payroll', 'PR1_BATCH_JOBS'), outage_state(custom_job_down, down, 'Payroll', 'PR1_BATCH_JOBS'), outage_state(custom_job_up, up, 'Parts', 'PR1_BATCH_JOBS'),

outage_state(custom_job_down, down, 'Parts', 'PR1_BATCH_JOBS'),

outage_state(custom_job_up, up, 'Parts2', 'PR1_BATCH_JOBS'),

outage_state(custom_job_down, down, 'Parts2', 'PR1_BATCH_JOBS'),

...

Similar outage states for other components

...

Tivoli Enterprise Console Warehouse Implementation Guide 33

Page 39: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

3.9.9 Operation

3.9.9.1 Installation Installation of the outage predicates into a Tivoli Enterprise Console Server is achieved through a shell script located at:

$TWH_TOPDIR/apps/ec1/v39010/misc/rules

This shell script must be run from the Tivoli Enterprise Console server. If the central data warehouse server is not the same as the Tivoli Enterprise Console server, copy the rules directory, shown in the path above, and everything below it, to a directory on your Tivoli Enterprise Console server and run the installation steps from there.

Installation steps:

1. Source the Tivoli environment. 2. Change to the directory that contains the install script. This would be <location_of_rules_dir>/rules/. The

install file is named install.sh. 3. Run the install.sh script with the target rule base name as the only argument.

This script imports the warehouse.baroc, warehouse.rls and warehouse.pro files into the targeted rule base. The preconfigured rules are available after you install the warehouse package in <rulebase_dir>/TEC_TEMPLATES/warehouse.pro.

NOTE: The rule base needs to be recompiled and reloaded and the TEC Server needs to be stopped and restarted before the changes take effect.

3.9.9.2 Configuration Before the rules can properly process Availability events some default values must be reconfigured to values that are suitable for your site. The warehouse.rls file contains two default values that can be customized, the warehouse.pro file contains default values for network and segment values that must be reset, and any fact file customizations to be implemented at your site must be added.

NOTE: You will need to recompile the rule base after importing the fact file, rule and class definitions into the rule base. When you are ready to use the new classes, rules, and fact file predicates and definitions, load the newly compiled rule base changes into your rule base, than stop and restart the event server to activate the changes.

3.9.9.2.1 Changing default values in the warehouse.rls file Two configuration values can be set for the outage rules. The first is the location of a persistence file that stores all outage related parameters as the system runs. Each time an outage related parameter changes within the rule engine facts, the change is written to this file. Whenever the Tivoli Enterprise Console event server is restarted, this persistence file is read and the outage facts are restored to the same state they were in prior to the shutdown. To change the file name, modify the statement that calls the init_outage predicate as follows:

init_outage (filename), filename is the path and file name of the fact file you want to use.

The second configuration value is the outage fact cleanup frequency. As the server runs, outage facts are collected. It is possible that a start-of-outage event is received without a corresponding end-of-outage event. In this case, the start-of-outage event remains in memory indefinitely unless there is a mechanism for clearing the outage fact table. A timer is set in the avail_startup rule in the warehouse.rls file that performs this clean up task. The default value is 86 400 seconds (24 hours). Outage events that need to be correlated must have the second event arrive within the time specified for the cleanup frequency or the first event will have been purged. The ETL process can run while events are stored in memory that have not yet been correlated by the arrival of the

Tivoli Enterprise Console Warehouse Implementation Guide 34

Page 40: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

second event. The timer set in the rules is unrelated to the running of the ETLprocess and only when the outage event is generated and written to the event database will it be captured for ETL processing.

To change this value, modify the statement that sets the maintenance timer as follows:

set_timer (_tick, seconds, ‘wh_purge_outage’),

seconds is the length of time (in seconds) you want to elapse between purges.

To set both of these values to the desired value edit the <rulebase_dir>/TEC_RULES/warehouse.rls file

3.9.9.2.2 Changing default values in the warehouse.pro file The warehouse.pro file is pre-configured to catch HP Openview network, segment, and node outages, missed heartbeat events, and provides an example of single event outage detection. The missed heartbeat events will work without changes but the HP Openview default values for network and segment instances must be customized for your site and single events that define an outage must be defined if they are to be used.

3.9.9.2.2.1 HP Openview changes The HP Openview pre-configured segment and network instances need to be changed and augmented to include segments and networks you want to monitor for outages in your environment. After the install.sh script has been run edit file <rulebase_dir>/TEC_TEMPLATES/warehouse.pro and find the predicates ‘add_hpov_segment_instance’ and ‘add_hpov_network_instance’. Change the IP address masks there to the IP addresses of the networks and segments in your environment that are to be monitored. Repeat those lines for as many as you want to monitor.

3.9.9.2.2.2 Single event outage changes Single events can be defined as containing outage information by following the sample definition provided in file warehouse.pro in predicate ‘init_single_event_outage’. The predicates ‘comp_type_relationship’, ‘allowable_state’, ‘create_event_criteria’, ‘comp_relationship’ and ‘outage_state’ are used to define single event outages. The ‘comp_type_relationship’, ‘allowable_state’ and ‘comp_relationship’ predicates define the meta data for the single event outage. That is, they define the component types, their allowable states and the component instances. The ‘create_event_criteria’ predicate defines which event attributes will cause an AVAIL_Outage event to be generated based on the event’s class and slot values. The ‘outage_state’ predicate associates the event criteria with the component type and component instance, and defines which attributes of the event will be used as the ‘outage_start_time’ and ‘outage_duration’ in the AVAIL_Outage event to be generated, as well as the outage state that the AVAIL_Outage event will report in slot ‘state_became’. When an event arrives that matches the criteria defined for a single event outage the proper slots are copied to the AVAIL_Outage event for outage_start_time and outage_duration, the state_became slot is determined and set and the AVAIL_Outage event is generated. The example shown in the warehouse.pro file shows the date slot supplying the value for the outage_start_time and the repeat_count slot value for the outage_duration value.

3.9.9.3 Uninstalling

Uninstalling the outage predicates from a Tivoli Enterprise Console Server is done by a shell script located at:

$TWH_TOPDIR/apps/ec1/v39010/misc/rules

This shell script must be run from the Tivoli Enterprise Console server. If the central data warehouse server is not the same as the Tivoli Enterprise Console server, run this script from the directory on your Tivoli Enterprise Console server where you copied the $TWH_TOPDIR/apps/ec1/v39010/misc/rules to for the installation steps and run the uninstall.sh script from there.

Steps to uninstall:

1. Source the Tivoli environment

Tivoli Enterprise Console Warehouse Implementation Guide 35

Page 41: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

2. Change to the directory that contains the uninstall script. This would be <location_of_rules_dir>/rules/. The uninstall file is named uninstall.sh.

3. Run the uninstall.sh with the target rule base name as the only argument.

This script uninstalls the warehouse.baroc, warehouse.rls and warehouse.pro files from the rule base and target rule base.

NOTE: The rule base needs to be recompiled and reloaded and the TEC Server needs to be stopped and restarted before the changes take effect

3.9.9.4 Warehouse rules startup and operation The diagram in depicts the rule base operation of the outage rules. The arrows pointing to the right out of the diamonds are followed when the answer to the question in the diamond is ‘yes’, otherwise the down arrow is followed.

Figure 3.1

Tivoli Enterprise Console Warehouse Implementation Guide 36

Page 42: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

Figure 3.1 Operation of ITEC Warehouse Rules.

Tivoli Enterprise Console Warehouse Implementation Guide 37

Page 43: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

When the server starts, the TEC_Start rule triggers a call to the init_outage predicate. This predicate performs initialization for all outage related facts. This predicate calls all user-defined outage predicates. It also starts the fact clean up timer.

The initialization triggers generate events that contain the meta data needed by the ITSLA application for detecting outages. Because these events are generated internally, they are processed through the rules. A rule fires on these events and validates them. If the meta data events are valid, then they are sent to the event repository. If not, they are dropped and an AVAIL_Error event is generated that contains a description of the problem in its msg attribute.

As events are received from the monitored environment, the rules detect new instances and outages based upon the defined outage facts. Each time a new component instance is detected, an AVAIL_Component_Relationship event is generated. This event is validated and if valid, allowed to proceed to the event repository. If it is not valid, it is dropped. Each newly detected outage causes an AVAIL_Outage event to be generated. This event is processed through the rules and is validated. If it is valid, the status is changed to CLOSED and is stored in the event repository, otherwise, it is dropped and an AVAIL_Error is generated.

It is possible for a user to send an AVAIL_Outage event directly to the server. This is useful if an external monitor can detect an outage. In this case, the components and component types must already be defined. The component types and components can be defined statically using a fact file, or dynamically by sending an AVAIL_Relationship event to the server. This event has slots for the component type, component type description, component and component description. Each slot is a list and must contain the parent/child relationships. An example of sending this event is as follows:

wpostemsg comp_types ='[compTypeA, compTypeB, compTypeC, compTypeD]' \ comp_type_descrs=['compTypeADescr, compTypeBDescr, compTypeCDescr, compTypeDDescr']\ comp_instances ='[compA, compB, compC, compD]' \ comp_descrs =['compADescr, compBDescr, compCDescr, compDDescr'] \ AVAIL_Relationship EVENT

When the server receives this event, a new set of components types and their corresponding components are created. A set of AVAIL_Comp_Type_Relationship and AVAIL_Comp_Relationship events are generated. They are processed the same as those that are defined statically.

All of the availability events get created with a status of CLOSED. The only exception to this is the AVAIL_Error event.

Other rules include a timer rule that periodically cleans the outage fact table and a change rule that is used to reanalyze TEC_Heartbeat_missed events in order to allow standard rule processing when they are in the closed state. These rules are shown in . Figure 3.2

Tivoli Enterprise Console Warehouse Implementation Guide 38

Page 44: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

Figure 3.2 ITEC Warehouse Timer and Change Rules

3.9.9.5 Warehouse ruleset details

Specific warehouse.rls ruleset descriptions follow: rule: avail_startup

The avail_startup rule is a configuration rule that runs upon receipt of the TEC_Start event, which is sent during initialization of the event server. By customizing this rule, you can configure the behavior of the warehouse rules. rule: commit_outage_meta_data The commit_outage_meta_data rule runs upon receipt of any of the following events:

• AVAIL_Comp_Type_Relationship • AVAIL_Comp_Relationship • AVAIL_Comp_Type_State • AVAIL_Measurement_Group_Type • AVAIL_Measurement_Group_State_Type

Tivoli Enterprise Console Warehouse Implementation Guide 39

Page 45: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

These events are used to define metadata related to components being monitored for availability outage data. When one of these events is received, the rule validates the event, closes it, and then uses the commit_set predicate to prevent further processing by the rule engine. These events are later read by the warehouse ETL. rule: process_avail_relationship The process_avail_relationship rule runs upon receipt of the AVAIL_Relationship event, which is used to define monitored components. When this event is received, the rule processes it using the process_avail_relationship predicate, which results in the generation of one or more AVAIL_Comp_Relationship events. rule: commit_outage_occurence The commit_outage_occurrence rule runs upon receipt of the AVAIL_Outage event, which is used to indicate an outage. When one of these events is received, the rule validates the event, closes it, and then uses the commit_set predicate to prevent further processing by the rule engine. These events are later read by the warehouse ETL. rule: instance_detection_all_events The instance_detection_all_events rule runs upon receipt of any event except the internal TEC_Start, TEC_Stop, and TEC_Error events. When an event is received, the rule analyzes the event to see if it came from a component that has not previously been defined for availability outage monitoring. If so, the rule dynamically generates an AVAIL_Comp_Relationship event to add the new metadata to the warehouse. rule: outage_detection_all_events The outage_detection_all_events rule runs upon receipt of any event except the internal TEC_Start, TEC_Stop, and TEC_Error events. When an event is received, the rule analyzes the event to determine whether it represents an outage (including start time and duration) or is part of a sequence of events representing an outage. If so, the rule generates an AVAIL_Outage event. change_rule: heartbeat_restored_outage The heartbeat_restored_outage change rule runs when the status of a TEC_Heartbeat_missed event is changed to CLOSED. When this happens, the event is reanalyzed in order to detect possible outage information. timer_rule: purge_outage_data The purge_outage_data timer rule runs upon expiration of the time set by the avail_startup rule. When this timer expires, any expired outage data is retracted from the knowledge base.

.

Tivoli Enterprise Console Warehouse Implementation Guide 40

Page 46: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

4 Maintenance

4.1 Backing up and restoring Be sure to perform backup actions before purging any data from the event repository. It is your responsibility to ensure you have sufficient backups to restore as much event data as you want to be able to store in the central data warehouse. Additionally, you might be required to have these tables available, not only for disaster recovery of the central data warehouse, but also to perform migration of the event data to any future version of the Tivoli Enterprise Console warehouse enablement pack.

NOTE: You should take great care when restoring the Tivoli Enterprise Console event database. The extraction control table, tec_t_last_etl_evt, contains a direct reference to the last event examined by the extraction process. If you restore the event database using a backup that was created prior to the last extraction, the extraction control table will refer to an event that is no longer the last event examined. You must ensure that the data in this tec_t_last_etl_evt table does not get modified by your backup and or restore policies.

4.2 Maintenance processes and commands Two maintenance processes are provided with this warehouse pack. They are EC1_c15_Clean_and_Reset_Process and EC1_c25_Drop_ETL_Support_Process. Additionally, the Tivoli Enterprise Console, version 3.9 wtdbclear and wtdbclear.pl purge commands, have been modified to check the last event examined for extract processing before clearing rows from the Tivoli Enterprise Console event repository.

4.2.1 Warehouse enablement pack maintenance processes EC1_c15_Clean_and_Reset_Process

This process removes any event data that might exist in any staging table for this warehouse pack, sets the Extract Control starting timestamp (ExtCtl_Strt_DtTm) back to ‘1970-01-01-00.00.00.000000’, and resets the tec_t_last_etl_evt table to one row with values (1,0,0,0). For more details on this process, see section 5.3, EC1_c15_Clean_and_Reset_Process.

EC1_c25_Drop_ETL_Support_Process

This process removes the tec_t_last_etl_evt table and, in the case of Informix, two stored procedures. The EC1_c05_Create_ETL_Objs_Process of the Tivoli Enterprise Console Warehouse Enablement Pack, version 1.2.0, installed these objects. This process can be run as the first step of reinitializing the tec_t_last_etl_evt table or as the first step in uninstalling this warehouse pack. For more details on this process, see section 5.5, EC1_c25_Drop_ETL_Support_Process.

4.2.2 Tivoli Enterprise Console maintenance commands wtdbclear and wtdbclear.pl

The Tivoli Enterprise Console purge commands (see “Tivoli Enterprise Console Command and Task Reference” Manual) have been modified to handle some specific situations that might occur when clearing event data in an environment where Tivoli Enterprise Console is warehouse-enabled. These commands now have a new -w option to allow you to specify that you want to force the removal of event data from the event repository. The -w option can be specified to force the purging of event data, whether or not it has been extracted. If the warehouse pack has not been installed, you will see no change when attempting to clear event data using these commands.

Unless the -w force option is specified, the purge commands only delete events with a date_reception prior to the value contained in this table. When the table is still in the initialized state and the event key values are zero, the wtdbclear command will not run and instead will shows a warning message that the data about to be purged has not been extracted yet. The -w option allows you to clear the event repository anyway. If you want to avoid this

Tivoli Enterprise Console Warehouse Implementation Guide 41

Page 47: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

restriction before the first extraction, run the EC1_c05_Create_ETL_Objs_Process shortly before the first extraction is scheduled to run.

If you decide to force the purge, you are presented with a warning indicating that you might be losing event data, but the purge continues as directed. There are two situations where this might occur:

• You are trying to purge event data and the extract was never started or run.

• You are trying to purge event data that arrived after the most recent extraction.

If you specify a time prior to the value contained in the new table, you see no change. If you specify a time after the value contained in the new table, you will get a message indicating that the time has been adjusted and the purge continues. This ensures that you do not lose relevant event data, unless you specify the force option.

You might see any of the following messages when using the purge commands in the following situations:

• You specify a purge date that is later than the date_reception value in the extraction control table. The specified date reception value to use for purging events from the event repository is after the date reception value of the last event that was extracted by the ETL process. The date reception value for the purge has been adjusted to prevent the deletion of events that might not have been extracted and placed into the data warehouse.

• You specify the -w force option and a purge date that is later than the date_reception value in the extraction control table.

The specified date reception value to use for purging events from the event repository is after the date reception value of the last event that was extracted by the ETL process. Since the warehouse force option was specified, events that have not been placed into the data warehouse might be purged.

• You do not specify -w force option and the extraction has never been run. The data warehouse ETL process has never been run. Since the warehouse force option was not specified, no events can be deleted from the event repository.

• You specify -w force option and the extraction has never been run. The data warehouse ETL process has never been run. Since the warehouse force option was specified, events will be purged from the event repository that might not have been extracted and placed into the data warehouse.

4.3 Warehouse data prune control The Tivoli Enterprise Data Warehouse provides a prune control process to remove aged records from the Msmt table. The pruning process uses an entry in the Prune_Msmt_Control table, inserted by a warehouse pack when it was installed, to control how long measurement data is kept in the Msmt table for a specific MSrc_Cd and TmSum_Cd. Measurement data older than the PMsmtC_Age_In_Days for a MSrc_Cd and TmSum_Cd are removed when the prune control process runs. The prune control process finds the MsmtTyp_ID’s in the Msmt table with the required TmSum_Cd value and matches the corresponding MSrc_Cd value in the MsmtTyp table for the MsmtTyp_ID and then removes any rows whose Msmt_Strt_Dt value is older than the PMsmtC_Age_In_Days value.

This warehouse pack does not insert a row into the Prune_Msmt_Control table at install time because the MsmtTyp’s used to define the measurement data are shared MsmtTyp’s that other warehouse packs may also use. Because several warehouse packs could use the same MsmtTyp’s, MODEL1 in our case, prune control capability is not provided that might prune other warehouse packs’ data. This is a limitation of Tivoli Enterprise Data Warehouse version 1.1 which has been addressed in version 1.2.

To manually remove any Msmt data for all installed warehouse packs that use the shared ‘MODEL1’ MSrc_Cd from the MsmtTyp table and a TmSum_Cd of ‘P’ refer to the warehouse trigger, twg.Prune_Msmt_Trg, in Appendix C of the Enabling an Application for Tivoli Enterprise Data Warehouse for sample sql on how to do this.

Tivoli Enterprise Console Warehouse Implementation Guide 42

Page 48: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

If you want to regularly prune all Msmt data for all installed warehouse packs that use the shared ‘MODEL1’ MSrc_Cd and a TmSum_Cd of ‘P’ add a row to the Prune_Msmt_Control table using the following sql as an example.

INSERT INTO TWG.Prune_Msmt_Control

(MSrc_Cd, TmSum_Cd, PMsmtC_Age_In_Days)

values(‘MODEL1’, ‘P’,000010000);

Sample data in the Prune_Msmt_Control table looks like the following:

MSrc_Cd TmSum_Cd PMsmtC_Age_In_Days

EC0 H 300

MODEL1 H 10000

Table 4.1 Sample data in the Prune_Msmt_Control table

Although the PMsmtC_Age_In_Days column heading includes Age_In_Days, the column represents a date duration whose format is yyyymmdd. The values shown in the preceding table, 300 and 10000, represent 3 months and 1 year, respectively. Preceding zeros are not included in the date duration value.

For example, a value of 108 is interpreted as follows and represents 0 years, 1 month, and 8 days.

Years Months Days

0000 01 08

Tivoli Enterprise Console Warehouse Implementation Guide 43

Page 49: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

5 Extract transform and load processes Tivoli Enterprise Console warehouse enablement pack has the following ETL processes. You must ensure that a process is not run multiple times simultaneously. In particular, running more than one ETL process at a time results in the purge, update, insert, and query actions of one process interfering with the same actions of another.

The sources and targets specified for processes in the EC1 process task flow are generic source and target table names and do not necessarily indicate the actual tables used as the source or target when the process is run. This information is detailed in the following explanations of the functions and tables being updated as well as in the Erwin schema diagrams included in files in the $TWH_TOPDIR/apps/ec1/v39010/misc directory.

5.1 EC1_c05_Create_ETL_Objs_Process The EC1_c05_Create_ETL_Objs_Process creates the tec_t_last_etl_evt table, and two stored procedures for Informix, in the Tivoli Enterprise Console database and is generally only run once after the warehouse pack is installed. This table is used by the EC1_c10_ETL1_Process to determine the last event that was examined from the source database. The table has one row, which contains a key_id value of 1 and the three-value unique event key associated with the last extracted event. The unique event key fields are date_reception, server_hndl, and event_hndl. This process creates the table and initializes the key_id value to 1 and each of the event key column values to 0. When all the event key values are 0 this indicates that an extraction has not yet occurred.

This table should not be removed or altered in any way while the warehouse pack is installed. Any changes made to this table might cause the extract process to fail. The extract also uses the table to insert other, temporary rows for use during the extract process itself. These rows are removed when the extract process completes.

If you drop the Tivoli Enterprise Console event database and recreate it, you must run this process again to recreate the extraction control table. You can run EC1_c20_Get_ExtCtl_Last_Date_Process to see the date_reception value of the last event examined in the last extract process and use this to update the date_reception value in the tec_t_last_etl_evt table. You should put a one in the server_hndl and event_hndl columns if you update the date_reception value.

The log output for this process can be found in the $VWS_LOGGING directory in file EC1_c05_s010_Create_ETL_Objs.log.

NOTE For Sybase Users:

In order for the EC1_c05_Create_ETL_Objs_Process to work properly, the database configuration value ddl in tran must be set to true in the Tivoli Enterprise Console event repository. This must be done before the EC1_c05_Create_ETL_Objs_Process is run.

Here are the steps required to set the ddl in tran configuration value to true:

1. Logon to the master database as the sa user, where <password> is the password for the sa user and <server> is the database server name:

isql -U sa -P <password> -S <server> -D master

2. Issue the following commands, where <dbname> is the name of the Tivoli Enterprise Console event repository database:

1> sp_dboption <dbname>,"ddl in tran",true

2> go

3. While still at the isql command prompt, issue the following commands, where <dbname> is the name of the Tivoli Enterprise Console event repository database:

1> use <dbname>

2> go

1> checkpoint

Tivoli Enterprise Console Warehouse Implementation Guide 44

Page 50: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

2> go

1> quit

To reset the above setting after running the EC1_c05_Create_ETL_Objs_Process, repeat the above steps replacing the value true in step 2 with the value false.

5.2 EC1_c10_ETL1_Process The EC1_c10_ETL1_Process is designed to extract all AVAIL_* events, except the AVAIL_Error events, from the event database and place the event data into the central data warehouse. The EC1_c10_ETL1_Process obtains event data by extracting data from the tec_t_evt_rep and tec_t_slots_evt tables. These tables are part of the event database provided by Tivoli Enterprise Console 3.9. In section 7.1, sample event data is used to illustrate how this event data is stored in the central data warehouse tables.

This process is intended to run at regular intervals that you schedule. For instance, you can schedule this process to be run nightly using the scheduling functions of the DB2 Data Warehouse Center. In this way, the central data warehouse is updated with new data based on the events received during that day. The EC1_c10_ETL1_Process uses the extraction control table described in “ ” on Page 44 to ensure that it only extracts new events. This function of extracting only new data from a data source is called extract control. This warehouse pack implements extract control differently than what is described in the manual, “Enabling an Application for Tivoli Enterprise Data Warehouse”.

EC1_c05_Create_ETL_Objs_Process

This process has the following steps:

• EC1_c10_s010_Extract

This step extracts the ITSLA outage event data from the event database event repository tables and stores it in the warehouse in the EC1.Stage_AVAIL_events table. The process updates the Tivoli Enterprise Console tec_t_last_etl_evt table at the end of this step with the last examined date_reception so the table is ready for the next extract run. The Extract_Log and Extract_Control tables are updated with the date_reception of the first event examined in the extract run, the date_reception of the last event examined in the extract run, as well as these values converted into time-stamp format.

The log output for this step of the process, including any exception messages, can be found in the $VWS_LOGGING directory in file EC1_c10_s010_Extract.log.

• EC1_c10_s020_Transform

In this step, the extracted event data in the EC1.Stage_AVAIL_events table is transformed to conform to the requirements of the tables in the central data warehouse and is placed in staging tables ready for the load process to load the transformed data into the central data warehouse.

The log output for this step of the process, including any exception messages, can be found in the $VWS_LOGGING directory in file EC1_c10_s020_Transform.log.

• EC1_c10_s030_Load

This step loads the data from the staging tables populated during the transform process into the central data warehouse. This step checks that data is being loaded into the central data warehouse tables in the correct order and issues exception messages for events that could not be processed. No duplicate data is loaded into the warehouse database. Instead, each event is first checked to see if it replicates information in the central data warehouse or another event in the same extraction, and, if it does, the event is not loaded. One of the last functions of this script is to remove any exception messages stored in the EC1.Exception_Log that are more than one month old to keep this table from filling up the central data warehouse database.

The log output for this step of the process, including any exception messages, can be found in the $VWS_LOGGING directory in file EC1_c10_s030_Load.log.

If the EC1_c10_s010_Extract completes successfully, the following step EC1_c10_s020_Transform runs automatically. Likewise, the successful completion of EC1_c10_s020_Transform starts the EC1_c10_s030_Load step. Each step produces a log file. The log files are documented in the “Installing and Configuring Tivoli Enterprise Data Warehouse” manual. In the event of an error during any of the processing,

Tivoli Enterprise Console Warehouse Implementation Guide 45

Page 51: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

the appropriate log files should be reviewed for errors or warnings. After correcting the logged error, the failing step may need to be run again. Read the recovery information in the error message found in the output log to ascertain if the step should be rerun or not. For more details concerning errors and recovery from errors, see “ ” on Page 48. Troubleshooting

5.3 EC1_c15_Clean_and_Reset_Process This process drops the EC1 staging tables created by this ETL during extract processing, reinitializes the values in the central data warehouse database Extract_Control table starting time stamp (ExtCtl_Strt_DtTm) back to ‘1970-01-01-00.00.00.000000’ for this ETL, and reinitializes the tec_t_last_etl_evt table in the Tivoli Enterprise Console database to one row with values (1,0,0,0).

This process can be used to reset the EC1 schema and extract control values stored in both the warehouse pack and the Tivoli Enterprise Console tables to allow warehouse pack processing of events to start again. Any new component types, components, measurement groups and measurement values would not be removed from the warehouse database. For more details on this, see “ on Page 9. This process is not scheduled, but should be run when needed.

Limitations

The log output for this step of the process, including any exception messages, can be found in the $VWS_LOGGING directory in file EC1_c15_s010_Clean_and_Reset.log.

5.4 EC1_c20_Get_ExtCtl_Last_Date_Process This process retrieves the last examined date_reception value stored in the central data warehouse database Extract_Control table when the prior EC1_c10_ETL1_Process was run. The value is updated in the final steps of the extract program each time the EC1_c10_ETL1_Process is run. This process is not scheduled, but should be run when needed.

Run this process when you want to know the highest date_reception value examined in the last extract process run or need to restore the extract control data in the tec_t_last_etl_evt table and need to know the highest date_reception value examined in the Tivoli Enterprise Console event repository during the prior run of the EC1_c10_ETL1_Process. It is not a requirement to set this value in the tec_t_last_etl_evt table if this table has been dropped and recreated after the extract process has been running. This is because no duplicate values are loaded into the central data warehouse during extract processing but run time can be lessened by extracting only previously unseen events. The last examined date_reception value can be seen in the process log output in $VWS_LOGGING directory in file EC1_c20_s010_Get_ExtCtl_Last_Date.log. If you update the date_reception value in the tec_t_last_etl_evt table, you should also put a 1 in the server_hndl and event_hndl columns.

Following is an example of how the process log output looks when this process is run: ====== Began 2003.04.03 09:59:36.390 (BuildDate: Wed 11/13/2002 ) ====== ======================== = Source Datasource : twh_cdw = Source User Name : db2 = DB Vendor Returned : DB2/NT, 07.02.0004 = DB Vendor Name : IBM DB2 = Target Datasource : twh_cdw = Target User Name : db2 = DB Vendor Returned : DB2/NT, 07.02.0004 = DB Vendor Name : IBM DB2 = Src DS = Tgt DS : (twh_cdw) = (twh_cdw) Ignoring cross \ datasource directives for improved efficiency = Input File : C:/TWH_1.1/apps/ec1/v39010/etl/sql/EC1_c20_s010_Get_ExtCtl_Last_Date.db2 ======================== = Script file line # : 1 = Exec at Source DS : twh_cdw (IBM DB2) = SQL Statement : "select 'Extract Last Date Reception: ', \ ExtCtl_To_IntSeq from TWG.Extract_Control where ExtCtl_Source = 'event \

Tivoli Enterprise Console Warehouse Implementation Guide 46

Page 52: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

repository' and ExtCtl_Target = 'EC1.Stage_AVAIL_events'" = Elapsed Time : 00:00:00.203 = Rows Modified : None = Num Result Cols : 2 1 EXTCTL_TO_INTSEQ ----------------------------- ------------------- Extract Last Date Reception: 1049384596 ----------------------------- ------------------- = Rows Selected : 1 = Select Cursor : Closed = Successful Execution: No Errors ========================

5.5 EC1_c25_Drop_ETL_Support_Process This process drops the tec_t_last_etl_evt extraction control table, and two stored procedures for Informix, from the Tivoli Enterprise Console database. Running EC1_c05_Create_ETL_Objs_Process created the table, and stored procedures for Informix. This process is run to remove the tec_t_last_etl_evt extraction control table in the Tivoli Enterprise Console database and has two purposes. First, if you want to reinitialize the Tivoli Enterprise Console database extraction control table, tec_t_last_etl_evt, so the extract examines the entire Tivoli Enterprise Console event database again, you can run this process and then recreate the extraction control table by running EC1_c05_Create_ETL_Objs_Process. The second reason to run this script would be to start the removal of this warehouse pack from the Tivoli Enterprise Data Warehouse. See “

” on Page 14. Uninstalling the warehouse

pack

The log output for this step of the process, including any exception messages, can be found in the $VWS_LOGGING directory in file EC1_c25_Drop_ETL_Support.log.

Tivoli Enterprise Console Warehouse Implementation Guide 47

Page 53: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

6 Troubleshooting Each process, as well as each step in the ETL process, creates a detailed log file in the logging directory. The logging directory is specified in the $VWS_LOGGING environment variable. Each log file has a name in the form <step name>.log. Example: EC1_c05_s010_Create_ETL_Objs.log.

Each log file contains information including such things as the source and target databases, the driver being used, each SQL statement issued, and any resulting warning or error messages.

Many of the ETL steps also log errors to the EC1.Exception_Log table in the central data warehouse database. These table entries are then printed in the log file where they can be easily seen. Section 7.4, Exception tables, shows the layout of the EC1.Exception_Log table.

Refer to Chapter 9 Troubleshooting, in Installing and Configuring Tivoli Enterprise Data Warehouse for additional troubleshooting activities common to the Data Warehouse Center. Also, the Tivoli Enterprise Data Warehouse Release Notes contain a list of known problems and should be reviewed thoroughly.

6.1 Errors with the tec_t_last_etl_evt table The following are some possible problems and recovery methods when the tec_t_last_etl_evt table has been altered or removed.

6.1.1 No rows in table or no row with key_id value of ‘1’ If the key_id row with 1 is missing, the extract assumes it must extract all events again and run as if all zeros were found in the table for the event key columns. When the extract process completes, it properly sets the values in that row to the event key value of the last event examined and it sets key_id to 1. No recovery steps are required.

6.1.2 No tec_t_last_etl_evt table The extract process fails if the table is missing. The error is logged in the EC1_c10_s010_Extract.log file. It indicates that there is no such table and that the extract failed. This file can be found in the $VWS_LOGGING directory.

6.1.2.1 Recovery If the extract has never been run, schedule the EC1_c05_Create_ETL_Objs_Process to create this table and run the EC1_c10_ETL1_Process again from the first step, EC1_c10_s010_Extract.

If the extract has previously been run, you need to run the EC1_c05_Create_ETL_Objs_Process to create this table and then update the event key columns with the event key of the last event that was examined by the last extract. The Extract_Log or Extract_Control entries in the central data warehouse database can be helpful in determining the last event processed as it shows the date range of the beginning and ending events processed by the extract. You can also run EC1_c20_Get_ExtCtl_Last_Date_Process to retrieve the last date_reception value processed by the prior extract.

6.2 EC1_c05_Create_ETL_Objs_Process The following error message is seen when running EC1_c05_Create_ETL_Objs_Process for a Tivoli Enterprise Console Sybase database where ddl in tran was not set to true. See section 5.1 for more information on setting this value to successfully run this step against a Sybase database. = Source Datasource : TEC_whse_sybase_11.9.2 = Source User Name : tec = DB Vendor Returned : SQL Server, 11.50.0000 = DB Vendor Name : Sybase = Target Datasource : twh_cdw = Target User Name : db2 = DB Vendor Returned : DB2/NT, 07.02.0004 = DB Vendor Name : IBM DB2

Tivoli Enterprise Console Warehouse Implementation Guide 48

Page 54: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

= Input File : \ C:/TWH_1.1/apps/ec1/v39010/etl/sql/EC1_c05_s010_Create_ETL_Objs.sybase ======================== = Script file line # : 17 = Exec at Source DS : TEC_whse_sybase_11.9.2 (Sybase) = SQL Statement : "create table tec_t_last_etl_evt ( key_id int \ default 0 NOT NULL, date_reception int default 0 NOT NULL, server_hndl \ int default 0 NOT NULL, event_hndl int default 0 NOT NULL, CONSTRAINT \ tec_i_last_etl_evt PRIMARY KEY nonclustered (key_id) WITH FILLFACTOR=85 )" = : = CDW8087E : SQL_ERROR: 'ExecDirect' 2003.03.31 13:39:40.312 = CDW8087E (HY000) : (Err:2762) [MERANT][ODBC Sybase driver][SQL \ Server]The 'CREATE TABLE' command is not allowed within a multi-statement \ transaction in the 'TEC' database. = : = Elapsed Time : 00:00:00.406 = Failed Execution : Error ignored as requested ========================

6.3 EC1_c10_ETL1_Process If you find you need to rerun the Extract step after it has run to completion and updated the row in the tec_t_last_etl_evt table where the extract control information is stored, you should use SQL to see the value in column ExtCtl_From_IntSeq in the warehouse Extract_Control table and place this value in the date_reception column of the tec_t_last_etl_evt table where key_id = 1. You should also change the event_hndl column in this table to a one. This allows you to process the same set of events again.

6.3.1 EC1_c10_s010_Extract step The following is an example of an error that could be seen in the extract step of the ETL. The Warning Text label explains what went wrong. The Recovery label explains how to fix the problem found. The text after the Calculation label explains what was being done at the time of the error.The Value label shows the problem value, if applicable. The Allowable Low Value and Allowable High Value labels show what the low and high possible values for the Value label could be, if applicable. ***** WARNING *****

Warning Text: A row in the tec_t_last_etl_evt table with a value of 1 for the key_id column does not exist. The extract will run, reading all events from the event repository. This will NOT cause duplicates in the warehouse. Recovery: No manual recovery steps needed. The extract was successfully run. Calculation: retrieving the key_id field from tec_t_last_etl_evt Value: N/A Allowable Low Value: N/A Allowable High Value: N/A

6.3.2 EC1_c10_s030_Load step The following is an example of an error that could be seen in the load step of the ETL. The text after the Calculation label explains what was being done at the time of the error. The Error Text label explains exactly what went wrong. The For Insert CDW table label shows the Tivoli Data Warehouse table that was being updated. The Recovery label explains how to fix the problem found. The Value in Error label shows the problem slot value in the event and the Other event slot values labels show additional slot values for the event experiencing the error so that you can identify the event that caused the error message, fix it and resubmit it.

The error text shown here is formatted to put the label headers on a new line. This is not the case when the output is viewed in the process output log.

Tivoli Enterprise Console Warehouse Implementation Guide 49

Page 55: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

***** ERROR ***** Calculation: checking existence of parent_comp_type in the CDW CompTyp table before the RelnRul table is updated Error Text: The parent_comp_type, Bad_par_comp_type, of class AVAIL_Comp_Type_Relationship for component type, child_comp_bad_pa, is an undefined component type. No entry for this record can be entered into the RelnRul table. For Extract Step: EC1 c10 s030 Load For Insert into CDW Table:RelnRul Recovery: Verify that the parent_comp_type is correct and, if it is, send in an event of class AVAIL_Comp_Type_Relationship to define that component type and then send in this failed event again to define the parent/child conponent type relationship this event was trying to define. If the parent component type is not correct resend an AVAIL_Comp_Type_Relationship for this child component type and description with a corrected parent component type. This event will be deleted. Value in Error: Bad par comp type Other event slot values: child comp type descr: descr for child comp type1 Other event slot values: N/A

Tivoli Enterprise Console Warehouse Implementation Guide 50

Page 56: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

7 Central data warehouse information The following example shows the data flow associated with a sample of availability events that have been generated by the event server. The data flow shows the setup events sent to the Tivoli Enterprise Console event database and three specific examples of AVAIL_Outage events. The AVAIL_Outage events are examples of the information that is sent to the Tivoli Enterprise Console event server given the following scenarios.

Scenario 1: A customer has an application named Payroll that runs on an MVS system with a system ID of S033. At 11:00 a.m. (local time) on 31Dec2002 the payroll job abend occurs. The job is fixed and runs again at 1 p.m. In this case, the customer wants to record the time from 11 a.m. to 1 p.m. as down.

Scenario 2: A customer has another application named Parts running on MVS system S033. For an hour, between the time of 8:00 a.m. and 9:00 a.m. the system has a slow response time. In this case, the customer wants to record the time from 8:00 a.m. to 9:00 a.m. as degraded.

Scenario 3: A customer has another Parts application running on MVS system S081. The customer decides to use this Parts application during the degradation period of the Parts application in scenario #2. They revert back to using the main Parts system when, for some reason, this application becomes unavailable from 6:16 p.m. and 6:20 p.m. In this case, the customer wants to record the time from 6:16 p.m. to 6:20 p.m as unavailable.

AVAIL_Comp_Type_Relationship parent_comp_type: NULL (NOTE: This is the NULL string.)

child_comp_type: PR1_BATCH_JOBS child_comp_type_descr: Batch Jobs AVAIL_Comp_Type_Relationship

parent_comp_type: NULL (NOTE: This is the NULL string.) child_comp_type: MVS_SYSTEM child_comp_type_descr: MVS System AVAIL_Comp_Type_State comp_type: PR1_BATCH_JOBS state: Unavailable AVAIL_Comp_Type_State comp_type: PR1_BATCH_JOBS state: Degrading AVAIL_Comp_Type_State comp_type: PR1_BATCH_JOBS state: Available AVAIL_Comp_Type_State comp_type: MVS_SYSTEM state: Unavailable AVAIL_Comp_Type_State comp_type: MVS_SYSTEM state: Degrading AVAIL_Comp_Type_State comp_type: PR1_BATCH_JOBS state: Available AVAIL_Comp_Relationship parent_comp_type: NULL (NOTE: This is the NULL string.) parent_comp: NULL (NOTE: This is the NULL string.) child_comp_type: MVS_SYSTEM child_comp: S033 child_comp_descr: NULL (NOTE: This is the NULL string.) AVAIL_Comp_Relationship parent_comp_type: NULL (NOTE: This is the NULL string.) parent_comp: NULL (NOTE: This is the NULL string.) child_comp_type: MVS_SYSTEM child_comp: S081

Tivoli Enterprise Console Warehouse Implementation Guide 51

Page 57: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

child_comp_descr: NULL (NOTE: This is the NULL string.) AVAIL_Comp_Relationship parent_comp_type: MVS_SYSTEM parent_comp: S033 child_comp_type: PR1_BATCH_JOBS child_comp: Payroll child_comp_descr: Payroll job on S033 AVAIL_Comp_Relationship parent_comp_type: MVS_SYSTEM parent_comp: S033 child_comp_type:PR1_BATCH_JOBS child_comp: Parts child_comp_descr: Parts job on S033 AVAIL_Comp_Relationship parent_comp_type: MVS_SYSTEM parent_comp: S081 child_comp_type: PR1_BATCH_JOBS child_comp: Parts2 child_comp_descr: Parts job on S081 Note: The two parts applications are given unique component instance names because they are associated with the same component type, PR1_BATCH_JOBS. This is a restriction associated with this warehouse pack even though their naming hierarchy would otherwise uniquely identify them. For more information on this limitation refer to , section 3.3.4. Component instance names must be unique

AVAIL_Measurement_Group_Type code: PR1MVS descr: MVS Batch Jobs via TEC ETL AVAIL_Measurement_Group_State_Type code: PR1MVS state: Unavailable AVAIL_Measurement_Group_State_Type code: PR1MVS state: Available AVAIL_Measurement_Group_State_Type code: PR1MVS state: Degrading AVAIL_Outage outage_start_time: 2002-12-31-11.00.00 outage_duration: 120

state_became: Unavailable comp_type: PR1_BATCH_JOBS comp_instance: Payroll

AVAIL_Outage outage_start_time: 2002-12-31-08.00.00 outage_duration: 60

state_became: Degrading comp_type: PR1_BATCH_JOBS comp_instance: Parts

AVAIL_Outage outage_start_time: 2002-12-31-06.16.00 outage_duration: 4

state_became: Unavailable

Tivoli Enterprise Console Warehouse Implementation Guide 52

Page 58: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

comp_type: PR1_BATCH_JOBS comp_instance: Parts2

The next few sections describe how the data from the event flow is stored in the Central Data Warehouse after EC1_c10_ETL1_Process runs.

Tivoli Enterprise Console Warehouse Implementation Guide 53

Page 59: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

7.1 Component configuration Shaded columns in the following tables are translated. These columns are also marked with an asterisk (*) after the column name. For a description of these tables, please refer to “Enabling an Application for Tivoli Enterprise Data Warehouse.”

7.1.1 Component type (table CompTyp) CompTyp_Cd CHAR(17)

CompTyp_Parent_Cd CHAR(17)

CompTyp_Nm VARCHAR(120)

CompTyp_Strt_DtTm TIMESTAMP

CompTyp_End_DtTm TIMESTAMP

IP_HOST NULL IP Host 2002-01-19-11.36.54.000000

9999-01-01-00.00.00.000000

PR1_BATCH_JOBS NULL Batch Jobs 2002-01-19-11.36.54.000000

9999-01-01-00.00.00.000000

MVS_SYSTEM NULL MVS_System 2002-01-19-11.36.54.000000

9999-01-01-00.00.00.000000

7.1.2 Component (table Comp) Comp_ID INTEGER

CompTyp_Cd CHAR (17)

Centr_Cd CHAR(6)

Cust_ID INTEGER

Comp_Corr_ID INTEGER

Comp_Nm VARCHAR (254)

Comp_Corr_Val VARCHAR (254)

Comp_Strt_DtTm TIMESTAMP

Comp_End_DtTm TIMESTAMP

Comp_Ds VARCHAR (254)

1 MVS_SYSTEM

CDW 1 S033 2002-02-19-

11.36.54.0

00000

9999-01-01-00.00.00.000000

2 MVS_SYSTEM

CDW 1 S081 2002-01-19-11.36.54.000000

9999-01-01-00.00.00.000000

3 PR1_BATCH_JOBS

CDW 1 Payroll 2002-01-19-11.36.54.000000

9999-01-01-00.00.00.000000

Payroll job on S033

4 PR1_BATCH_JOBS

CDW 1 Parts 2002-01-19-11.36.54.000000

9999-01-01-00.00.00.000000

Parts job on S033

5 PR1_BATCH_JOBS

CDW 1 Parts2 2002-01-19-11.36.54.000000

9999-01-01-00.00.00.000000

Parts job on S081

Tivoli Enterprise Console Warehouse Implementation Guide 54

Page 60: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

7.1.3 Component relationship type (table RelnTyp) RelnTyp_Cd CHAR(6) RelnTyp_Nm * VARCHAR(120)

PCHILD Parent Child Relationship

* This column is translated.

7.1.4 Component relationship rule (table RelnRul) CompTyp_Source_Cd CHAR(17)

CompTyp_Target_Cd CHAR(17)

RelnTyp_Cd CHAR(6) RelnRul_Strt_DtTm TIMESTAMP

RelnRul_End_DtTm TIMESTAMP

MVS_SYSTEM PR1_BATCH_JOBS PCHILD 2002-01-19-11.36.54.000000

9999-01-01-00.00.00.000000

7.1.5 Component relationship (table CompReln) CompReln_ID INTEGER

Comp_Source_ID INTEGER

Comp_Target_ID INTEGER

RelnTyp_Cd CHAR(6)

CompReln_Strt_DtTm TIMESTAMP

CompReln_End_DtTm TIMESTAMP

1 1 3 PCHILD 2002-01-19-11.36.54.000000

9999-01-01-00.00.00.000000

2 1 4 PCHILD 2002-01-19-11.36.54.000000

9999-01-01-00.00.00.000000

3 2 5 PCHILD 2002-01-19-11.36.54.000000

9999-01-01-00.00.00.000000

7.1.6 Attribute type (table AttrTyp) Not Applicable

7.1.7 Attribute rule (table AttrRul) Not Applicable

7.1.8 Attribute domain (table AttrDom) Not Applicable

7.1.9 Component attribute (table CompAttr) Not Applicable

7.2 Component measurement The following sections describe the component measurement.

Tivoli Enterprise Console Warehouse Implementation Guide 55

Page 61: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

7.2.1 Measurement group type (table MgrpTyp) MgrpTyp_Cd CHAR (6) MgrpTyp_Nm * VARCHAR (120)

GROUP Aggregate Types or Group Functions

TRANS State Transition Group

* This column is translated.

7.2.2 Measurement group (table MGrp) MGrp_Cd CHAR (6) MgrpTyp_Cd CHAR (6) MGrp_Parent_Cd CHAR (6) MGrp_Nm * VARCHAR (120)

PR1MVS TRANS NULL MVS Batch Jobs via TEC ETL

TOT_E GROUP NULL Total Value Exist

* This column is translated.

7.2.3 Measurement group member (table MGrpMbr) MGrp_Cd CHAR (6) MgrpTyp_Cd CHAR (6) MsmtTyp_ID INTEGER

PR1MVS TRANS 1

PR1MVS TRANS 2

PR1MVS TRANS 5

TOT_E GROUP 1

TOT_E GROUP 2

TOT_E GROUP 3

TOT_E GROUP 4

TOT_E GROUP 5

TOT_E GROUP 6

TOT_E GROUP 7

TOT_E GROUP 8

TOT_E GROUP 9

7.2.4 Measurement unit (table Munit) Munit_Cd CHAR (6) MunitCat_Cd CHAR (6) Munit_Nm * VARCHAR (120)

Min TM Minutes

* This column is translated.

Tivoli Enterprise Console Warehouse Implementation Guide 56

Page 62: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

7.2.5 Time summary (table TmSum) A measurement can be summarized over this period. TmSum_Cd CHAR TmSum_Nm * VARCHAR (120)

P Point

* This column is translated.

7.2.6 Measurement source (table MSrc) MSrc_Cd CHAR (6) MSrc_Parent_Cd CHAR (6) MSrc_Nm * VARCHAR (120)

Tivoli NULL Tivoli Application

MODEL1 NULL Tivoli Common Data Model V1

EC1 Tivoli Tivoli Enterprise Console

* This column is translated.

7.2.7 Measurement type (table MsmtTyp) MsmtTyp_ID INTEGER

Munit_Cd CHAR (6) MSrc_Cd CHAR (6) MsmtTyp_Nm * VARCHAR (120)

MsmtTyp_Ds * VARCHAR (254)

1 Min MODEL1 Available The amount of time that the resource is available

2 Min MODEL1 Unavailable The amount of time that the resource is unavailable

3 Min MODEL1 Unreachable The amount of time that the resource is unreachable

4 Min MODEL1 Unmanaged The amount of time that the resource is unmanaged

5 Min MODEL1 Degrading The amount of time that the resource is degrading

6 Min MODEL1 Responding The amount of time that the resource is responding

7 Min MODEL1 Repairing The amount of time that the resource is repairing

8 Min MODEL1 Closing The amount of time that the resource is closing

9 Min MODEL1 Unknown The amount of time that the state of the resource is unknown

* This column is translated.

7.2.8 Component measurement rule (table MsmtRul) CompTyp_Cd CHAR (17) MsmtTyp_ID INTEGER

PR1_BATCH_JOBS 1

PR1_BATCH_JOBS 2

PR1_BATCH_JOBS 5

Tivoli Enterprise Console Warehouse Implementation Guide 57

Page 63: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

MVS_SERVER 1

MVS_SERVER 2

MVS_SERVER 5

IP_HOST 1

IP_HOST 2

IP_HOST 5

7.2.9 Measurement (table Msmt) Msmt_ID BIGINT

Comp_ID INTEGER

MsmtTyp_ID INTEGER

TmSum_Cd CHAR

Msmt_Strt_Dt DATE

Msmt_Strt_Tm TIME

Msmt_Min_Val FLOAT

Msmt_Max_Val FLOAT

Msmt_Avg_Val FLOAT

Msmt_Tot_Val FLOAT

Msmt_Smpl_Cnt INTEGER

Msmt_Err_Cnt INTEGER

1 3 2 P 2002-12-31

11:00 120

2 4 5 P 2002-12-31

8:00 60

3 5 2 P 2002-12-31

6:16 4

7.3 Helper tables Not Applicable

7.4 Exception tables One exception table, EC1.Exception_Log is provided. It is used to identify cases where data might fall outside of an allowable, expected range of values or cases where extract run-time problems were encountered and logged. The data written to the table should identify the table and the calculation performed in which the exception was found, the range of values expected, as well as the actual values found.

Error_DtTm TIMESTAMP

Process_Nm VARCHAR(120)

Step_Nm VARCHAR(120)

Table_Nm VARCHAR(120)

Calculation VARCHAR(120)

Result_Val_Low VARCHAR(120)

Result_Val_Low VARCHAR(254)

Actual_Value VARCHAR(254)

Error_Msg_Text VARCHAR(254)

Error_Recovery VARCHAR(254)

Reported_DtTm TIMESTAMP

Any data exception identified during the running of a step appears at the end of the log file produced by that step. It is not necessary for this table to be checked manually. Refer to section 6, Troubleshooting, for more information on using the data exception to troubleshoot problems.

7.5 Incremental extraction The event data extracted by the EC1_c10_ETL1_Process is controlled by the values contained in the tec_t_last_etl_evt table in the Tivoli Enterprise Console event database. This table contains the unique identifier for the last event that was examined. The EC1_c10_ETL1_Process uses the date_reception, server_hndl and event_hndl values stored in the tec_t_last_etl_evt table to extract only those events that were received after the event key stored in this table where key_id = 1. This keeps the extract process from having to reprocess events. If events are reprocessed, duplicate entries in the Tivoli Data Warehouse will not result because duplicates are checked before an entry is inserted into a central data warehouse table.

Tivoli Enterprise Console Warehouse Implementation Guide 58

Page 64: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

8 Data mart schema information Not Applicable

Tivoli Enterprise Console Warehouse Implementation Guide 59

Page 65: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

Appendix A Operational environment analysis worksheets These worksheets are provided as a means for gathering the information discussed in “

” on Page 22. Analyze operational

environment

A.1 Component types This worksheet is used to gather the information discussed in “ ” on Page 22.

Identify systems that require outage detection

Component Type Description

Tivoli Enterprise Console Warehouse Implementation Guide 60

Page 66: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

A.2 Component type relationships This worksheet is used to gather the information discussed in “ ” on Page 22 .

Identify systems that require outage detection

Parent Child

Tivoli Enterprise Console Warehouse Implementation Guide 61

Page 67: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

A.3 Allowable warehouse component states This worksheet is used to gather the information discussed in “ ” on Page 23. Identify measurement information

Component Type Allowable States

Tivoli Enterprise Console Warehouse Implementation Guide 62

Page 68: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

A.4 Measurement groups This worksheet is used to gather the information discussed in “ ” on Page 23. Identify measurement information

Measurement Group Code Description

Tivoli Enterprise Console Warehouse Implementation Guide 63

Page 69: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

A.5 Measurement group states This worksheet is used to gather the information discussed in “ ” on Page 23. Identify measurement information

Measurement Group Code Measurement Group States

Tivoli Enterprise Console Warehouse Implementation Guide 64

Page 70: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

A.6 Components This worksheet is used to gather the information discussed in Section “ ” on Page 23.

Identify measurement information

Component Component Type Description

Tivoli Enterprise Console Warehouse Implementation Guide 65

Page 71: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

A.7 Component relationships This worksheet is used to gather the information discussed in “ ” on Page 23. Identify measurement information

Parent Child

Tivoli Enterprise Console Warehouse Implementation Guide 66

Page 72: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

A.8 Domain states This worksheet is used to gather the information discussed in “ ” on Page 25. Component type domain states

Component Type Domain State

Tivoli Enterprise Console Warehouse Implementation Guide 67

Page 73: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

A.9 Event criteria for condition detection This worksheet is used to gather the information discussed in “ ” on Page 25. Component type domain states

Component Type Domain State Event Class Attribute Condition

Tivoli Enterprise Console Warehouse Implementation Guide 68

Page 74: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

A.10 Domain state transitions This worksheet is used to gather the information discussed in “ ” on Page 26. Allowable state transitions

Component Type Domain State 1 Domain State 2 Warehouse State Position

Tivoli Enterprise Console Warehouse Implementation Guide 69

Page 75: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

A.11 Event criteria for dynamic component instances This worksheet is used to gather the information discussed in “ ” on Page 26. Detecting instances dynamically

Event Criteria Attributes(s) Component Type Component Description

Tivoli Enterprise Console Warehouse Implementation Guide 70

Page 76: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

Notices

This information was developed for products and services offered in the U.S.A. IBM can not offer the products, services, or features discussed in this document in other countries. Consult your local IBM representative for information on the products and services currently available in your area. Any reference to an IBM product, program, or service is not intended to state or imply that only that IBM product, program, or service can be used. Any functionally equivalent product, program, or service that does not infringe any IBM intellectual property right can be used instead. However, it is the user's responsibility to evaluate and verify the operation of any non-IBM product, program, or service.

IBM can have patents or pending patent applications covering subject matter described in this document. The furnishing of this document does not give you any license to these patents. You can send license inquiries, in writing, to:

IBM Director of Licensing IBM Corporation North Castle Drive Armonk, NY 10504-1785 U.S.A. For license inquiries regarding double-byte (DBCS) information, contact the IBM Intellectual Property Department in your country or send inquiries, in writing, to:

IBM World Trade Asia Corporation Licensing 2-31 Roppongi 3-chome, Minato-ku Tokyo 106, Japan The following paragraph does not apply to the United Kingdom or any other country where such provisions are inconsistent with local law:

INTERNATIONAL BUSINESS MACHINES CORPORATION PROVIDES THIS PUBLICATION "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

Some states do not allow disclaimer of express or implied warranties in certain transactions, therefore, this statement might not apply to you.

This information could include technical inaccuracies or typographical errors. Changes are periodically made to the information herein; these changes will be incorporated in new editions of the publication. IBM can make improvements and/or changes in the product(s) and/or the program(s) described in this publication at any time without notice.

Any references in this information to non-IBM Web sites are provided for convenience only and do not in any manner serve as an endorsement of those Web sites. The materials at those Web sites are not part of the materials for this IBM product and use of those Web sites is at your own risk.

IBM can use or distribute any of the information you supply in any way it believes appropriate without incurring any obligation to you.

Licensees of this program who wish to have information about it for the purpose of enabling: (i) the exchange of information between independently created programs and other programs (including this one) and (ii) the mutual use of the information, which has been exchanged, should contact:

IBM Corporation 2Z4A/101 11400 Burnet Road Austin, TX 78758 U.S.A. Such information can be available, subject to appropriate terms and conditions, including in some cases payment of a fee. The licensed program described in this document and all licensed material available for it are provided by IBM under terms of the IBM Customer Agreement, IBM International Program License Agreement or any equivalent agreement between us.

Tivoli Enterprise Console Warehouse Implementation Guide 71

Page 77: Tivoli Enterprise Console, version 3.9.0 Warehouse …publib.boulder.ibm.com/tividd/td/tec/SC32-1236-00/en_US/... · 2004-06-10 · 1 About this document This document describes the

Any performance data contained herein was determined in a controlled environment. Therefore, the results obtained in other operating environments can vary significantly. Some measurements can have been made on development-level systems and there is no guarantee that these measurements will be the same on generally available systems. Furthermore, some measurement may have been estimated through extrapolation. Actual results can vary. Users of this document should verify the applicable data for their specific environment.

Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not tested those products and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products.

All statements regarding IBM's future direction or intent are subject to change or withdrawal without notice, and represent goals and objectives only.

This information contains examples of data and reports used in daily business operations. To illustrate them as completely as possible, the examples include the names of individuals, companies, brands, and products. All of these names are fictitious and any similarity to the names and addresses used by an actual business enterprise is entirely coincidental. This information contains sample application programs in source language, which illustrate programming techniques on various operating platforms. You can copy, modify, and distribute these sample programs in any form without payment to IBM, for the purposes of developing, using, marketing or distributing application programs conforming to the application programming interface for the operating platform for which the sample programs are written. These examples have not been thoroughly tested under all conditions. IBM, therefore, cannot guarantee or imply reliability, serviceability, or function of these programs. You can copy, modify, and distribute these sample programs in any form without payment to IBM for the purposes of developing, using, marketing, or distributing application programs conforming to IBM's application programming interfaces.

If you are viewing this information in softcopy form, the photographs and color illustrations might not appear.

Trademarks The following terms are trademarks of International Business Machines Corporation in the United States, other countries, or both:

IBM, the IBM logo, Tivoli, the Tivoli logo, DB2, NetView, Tivoli Enterprise, Tivoli Enterprise Console, TME, Informix.

Microsoft, Microsoft SQL Server, and Windows are trademarks of Microsoft Corporation in the United States, other countries, or both.

UNIX is a registered trademark of The Open Group in the United States and other countries.

Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both.

Other company, product, and service names can be trademarks or service marks of others.

SC32-1236-00

---------------------- End of Document ----------------------

Tivoli Enterprise Console Warehouse Implementation Guide 72