Www.bmc.com 1 BMC SOFTWARE Copyright Extending Perceive (Part 1) Perry Stupp Principal Software Consultant

www.bmc.com

1 BMC SOFTWARE Copyright

Extending Perceive Extending Perceive (Part 1)(Part 1)

Perry StuppPrincipal Software Consultant


Why extend Perceive?Why extend Perceive?

Main reasons:

1) To bring it up to par with Visualizer2) To add capabilities that Visualizer will never have.3) To prototype potential solutions for Perform.Next4) To empower customers to deliver the content that

they want, in the format they want it – now!


Why customers want ‘extended’ PerceiveWhy customers want ‘extended’ Perceive

The fully extended Perceive provides a powerful interface for Enterprise Performance Assurance complete with;

- Dynamic Web Publishing / Variable date range selection- Self-serve data export- Streamlined processing

- Fewer moving parts- The ability to nearly eliminate the NT requirement- Portal Potential - Advanced reporting / Analysis capabilities

- Statistics- Exceptions- Descriptive Text- Correlation with external / extended data sources


The basics – Minimal XML PrimerThe basics – Minimal XML Primer

All configuration files are in XML

Each element is defined by a tag. Ex. <tag>

The first tag is known as the root element.

Each tag must be terminated. Ex. <tag/> or <tag></tag>

Tags can be Hierarchical. Ex. <tag1><tag2/><tag3></tag3></tag1>

Tags can have attributes. Ex. <tag name=“value”/>

Tags can contain data. Ex. <tag>value</tag>

Text in data tags can be escaped. Ex. <![CDATA[…]]>

<!– This is a comment, it can span multiple lines … -->

For more information see:


The basicsThe basics

Perceive only knows about 3 data types;

Text Any string data

Series A set of values for any single metric

Multi-series - One or more sets of values. As ‘extenders’ we are limited to multiple instances of the same metric.- To get multiple metrics in the same graph, you can simply drag and drop them!


The basicsThe basics

The Data type for each metric is defined in the common.xml file located in …\jakarta-tomcat-4.0.3\webapps\qtv\WEB-INF

Metric Type Data-TypeText com.bmc.qtv.data.PMDataObject

Series com.bmc.qtv.data.PMSeries

Multi-series com.bmc.qtv.data.PMMultiSeries


The basics – More about common.xmlThe basics – More about common.xml

Configuration file that controls the list of available metrics, regardless of which data source(s) they are ultimately derived from.

Hierarchical structure starting with the root element <common>.

All metrics are defined by the <metric> tag.

Related metrics can also be group by the <metric> tag.



Grouping related metrics – Minimal metric descriptions for a hierarchical structure for the metric tree in the administrative “create/edit views” section.

<metric name="CPU" label="CPU"> <metric …/> <metric …/></metric>



Defining a metric – Full metric descriptions have the following attributes;

Attribute Description Req’d

Name Name of the metric Y

Label Not yet supported / exposed N

Description Not yet supported / exposed N

Data-type As per the previous page Y

Units Specifies Unit of metric – Also in vis-sql.xml N

Floor The lowest possible value N

Ceiling The highest possible value N



Some points of interest;

Attribute FactorName - Must be unique!Units - Only like series data can be combined into a

single graph – from the GUI!- Auto-scaling can be enforced in 7.1.00 by placing a space at the end of the string. Not needed in 7.1.01- Not required yet overrides vis-sql.xml units- Specifies Units for legends

Floor - Defaults to 0 if not specified

Ceiling - Will auto-scale if not selected- Will clip if specified- Will auto-scale if set to “unbounded” in 7.1.01.


The basics – vis-sql.xmlThe basics – vis-sql.xml

Describes the metrics derived from the Visualizer DB.

Hierarchical structure starting with the root element <vis>.

Includes specialized entries for <nodeDiscovery>, <intervalDiscovery> and <System Info> which discover and define available data within the database (topic for an advanced class)

For each metric there is the hierarchical structure;

<query …> <sql> SELECT … </sql></query>


The basics – vis-sql.xmlThe basics – vis-sql.xml

The Query tag contains the following attributes;Attribute Description Req’d

Name Name of the metric Ypath Hierarchical path specified in common.xml YOs Allows you to overload the same metric YDate Looks like this has been deprecated NTime Looks like this has been deprecated NValue Looks like this has been deprecated NUnits The Unit of measure YTopN Top N highest values NKey Looks like this has been deprecated

- Required for a multi-seriesY(M)


The basics – More about vis-sql.xmlThe basics – More about vis-sql.xml

Some points of interest;

Attribute Factor

Name - Must be unique!- Must match the end of the PATH

Path - Must be unique- Must be consistent with the hierarchical entries in the common.xml

OS - Maps to the entry in the CAXNODE table- All, UNIX, NT (Can be extended!)

Units - Only like series data can be combined into a single graph – from the GUI!- Auto-scaling can be enforced in 7.1.00 by placing a space at the end of the string. Not needed in 7.1.01- Required but unclear why (must be internal to code)- Best to keep this the same as common.xml

TopN - Can be any value – imagine that!- only applicable to Multi-Series



A few more things…

- Some results are affected

Ex. <operation name="DivideOperation" operand="1024"/>

- Network Packets is an anomaly

- Most of the per platform differences are on the Network, the other is memory.

- There are some hard-coded rules to address these!


Vis-sql.xml – SQL PrimerVis-sql.xml – SQL Primer

Just enough SQL to get at the data

SQL – Structured Query Language

Schema – Defines the tables, the fields in each table, and the relationships between fields and tables.

Database – A grouping of tables, views, stored procedures, user permissions etc. This nomenclature applies well to SQL Server, Access and any ODBC data. Oracle, however, calls this a Tablespace.



The Select Statement – Retrieves rows from the database

Select all fields for all records from a table

select * from tableA

Select specific fields from a tableselect field1, field2 from tableB

Select specific fields from more than one tableselect table1.field1, table2.field1 from tableA, tableB*

*Note – This is not a clean query for reasons beyond the current scope.



The Where Statement – Specifies a search condition to restrict the rows returned.

Select field1 from tableA and tableB but only for records where field3 of the two tables are equal.

select tableA.field1, tableB.field1 from tableA, tableBwhere tableA.field3 = tableB.field3



The Where Statement – Conditions. = Equals

< Less Than> Greater Than!> Not Greater, !< Not Less!= or <> Not Equalbetween X and Y or between ‘X’ and ‘Y’in (0, 1, 5, 7) or in (‘CA’, ‘MA’)

String Matcheslike ‘abc%’ - A string that starts with abc.like ‘abc_’ - A four character string starting with abc.

NULL FieldsIS [NOT] NULL



Order by – Sorting the selected outputEx. List all last names in the phonebook in descending alphabetical order.

select distinct(lastname) from phonebookorder by lastname desc

Group by – Aggregate records for computationWhen used with a GROUP BY clause, each aggregate function produces a single value for each group, rather than for the whole table.

Having - Specifies a search condition for a group or an aggregate.



Some useful functionsDistinct() – Remove duplicate entries from fields.

Ex. List all names within a phonebookselect distinct(lastname) from phonebook

Count() – A count of the selected records

Ex. How many Stupps are in the phonebookselect count(lastname) from phonebook where lastname like ‘Stupp’

Ex. How many unique names are in the phonebookselect count(distinct(lastname)) from phonebook



Some other functions

avg() min() max() sum() trim()


Vis-sql.xml – DB Specific syntaxVis-sql.xml – DB Specific syntax

Because there are some functions that are unique to a given database, 7.1.01 allows for a separate vis-sql.xml file per data type. For instance there is or can be a vis-sql-oracle.xml and vis-sql-sqlserver.xml etc…

Ex’s to_date() vs. convert() substr() vs. substring() || vs. +



A closer look at the SQL tag.

The SQL Tag is a data element with Text in it. It must contain valid SQL as per;

SELECT INT_START_DATE as D, INT_START_TIME AS T, METRIC AS VINT_START_DATE as D, INT_START_TIME AS T, METRIC AS V FROM CAXINTVL, CAXCPUS, CAXCPUD WHERE INTVL=CAXINTVL.INDEXX AND CAXCPUS.INDEXX=CPUS AND NODE_NAME='!NAME!'NODE_NAME='!NAME!' AND CAXINTVL.INTTYPE='M' AND CAXCPUD.CPU_UTIL >= 0 ANDAND CAXINTVL.INT_START_DATE BETWEEN '!START!' AND '!END!' CAXINTVL.INT_START_DATE BETWEEN '!START!' AND '!END!' GROUP BY CAXINTVL.INT_START_DATE, CAXINTVL.INT_START_TIME ORDER BY CAXINTVL.INT_START_DATE, CAXINTVL.INT_START_TIMEORDER BY CAXINTVL.INT_START_DATE, CAXINTVL.INT_START_TIME


The basics – The “other” Data sourceThe basics – The “other” Data source

udrmappings.xml

Describes the metrics derived from the UDR Data.

Hierarchical structure starting with the root element <mappings>.

Further subdivided by <os name=“value”>



Mapping’s specify the metric that is returned for a given graph (system info a special case that returns multiple metrics). The hierarchy follows the same structure as the common.xml file. It’s attributes are;

Attribute Description Req’d

path - Hierarchical path specified in common.xml Y

Metricgroup - Metric group from which the value is derived.- Maps to entries in the mgt file.

Y

Summarize - Should the retrieve values be summarizedInto one value. - Generally used in tables or to convert a multi-series metric into a series.

N

Special - Multiple Uses not fully documented yet- A Number indicates TopN, for use with MultiSeries values

N



Each mapping has at least one <Metric> tag. The name attribute is the name of the metric as defined by the appropriate .mt file. If the options key attribute is specified then this indicates that it will distinguish between various instances of the metric.

If a “key” metric is specified, there will be another metric tag indicating the metric that will provide the values.

The metric name must be an entry in the .mt file.

Just as with Investigate filters can be used to limit result sets

<filter-expression column="Processor Time" comparator="Top“ value="10"/>

Could be used to create “advanced” data sources.


Getting Started – Adding a metricGetting Started – Adding a metric

The easiest way to add a metric is to simply copy and paste and existing metric and alter the details. For instance, let’s add CPU Wait Time.

Update common.xml

1) Copy <metric name="CPU Utilization" label="CPU Utilization" data-type="com.bmc.qtv.data.PMSeries" units="% " floor="0" ceiling="100"/>

2) Change “CPU Utilization” to “CPU Wait Time”



Update udrmappings.xml

UDR Agent Details are in qtv\mgroupMap “System Statistics” to SysStat.mt from MetricTable.mgtLook for “CPU Wait Time” Metric in SysStat.mt file.

1) Copy from UNIX Section<mapping path="CPU/CPU Utilization" metricGroup="System Statistics" summarize="false">

<metric name="CPU Utilization"/> </mapping>


You may need to do this for each platform! Since they are all (possibly) in different metric groups.



Update vis-sql.xml

This usually requires some comfort with the schema. See the 3.7 documentation for details although even that may not be enough. You should be comfortable browsing the database with native tools.

In this case, we need the CAXCPU tables.



1) Copy

<query name="CPU Utilization“ path="CPU/CPU Utilization“ os="all“ units="%">

<sql> SELECT CAXINTVL.INT_START_DATE AS D, CAXINTVL.INT_START_TIME AS T,

AVG(CAXCPUD.CPU_UTIL) AS V FROM CAXINTVL, CAXCPUS, CAXCPUD WHERE CAXCPUD.INTVL=CAXINTVL.INDEXX AND CAXCPUS.INDEXX=CAXCPUD.CPUS AND CAXCPUS.NODE_NAME='!NAME!' AND CAXINTVL.INTTYPE='M' AND CAXCPUD.CPU_UTIL >= 0 AND CAXINTVL.INT_START_DATE BETWEEN '!START!' AND '!END!' GROUP BY CAXINTVL.INT_START_DATE, CAXINTVL.INT_START_TIME ORDER BY CAXINTVL.INT_START_DATE, CAXINTVL.INT_START_TIME </sql> </query>


3) Change each instance of CPU_UTIL to CPU_WAIT (there are 2)



NOTE! You must restart Perceive in order for these changes to take effect!

NOTE! You will re-start Perceive a lot when building extensions!

NOTE! It’s generally a good idea to test whether the file is valid by loading it in a browser before re-starting Perceive.


Getting Started – More AdvancedGetting Started – More Advanced

Getting a little more creative.

<query name="Top 10 Commands CPU Utilization - Chart" path="Commands/Top 10 Commands CPU Utilization - Chart" os="all" topN="10" units=“% "> <sql> SELECT CAXINTVL.INT_START_DATE AS D, CAXINTVL.INT_START_TIME AS T, CAXCOMMD.CPU_UTIL AS V, CAXCOMMS.COMMAND_NAME AS KEY1CAXCOMMS.COMMAND_NAME AS KEY1 FROM CAXINTVL, CAXCOMMS, CAXCOMMD WHERE CAXCOMMD.INTVL=CAXINTVL.INDEXX AND CAXCOMMS.INDEXX = CAXCOMMD.COMMS AND CAXCOMMS.NODE_NAME='!NAME!' AND CAXINTVL.INTTYPE='M' AND CAXINTVL.INT_START_DATE BETWEEN '!START!' AND '!END!' ORDER BY CAXINTVL.INT_START_DATE, CAXINTVL.INT_START_TIME </sql> </query>

Documents

Www.bmc.com 1 BMC SOFTWARE Copyright Extending Perceive (Part 1) Perry Stupp Principal Software Consultant