Upload
hung-nguyen
View
4
Download
0
Tags:
Embed Size (px)
Citation preview
November 2013
Oracle Enterprise Manager System Monitoring Plug-in for Oracle ZFS Storage ApplianceInstallation Guidev1.4.1
Oracle Enterprise Manager System Monitoring Plug-in for Oracle ZFS Storage Appliance, Installation Guide v1.4.1
Table of ContentsIntroduction.............................................................................................................................. 1About the Oracle ZFS Storage Appliance Plug-in................................................................. 2Software Requirements........................................................................................................... 2Prerequisites for Monitoring Oracle ZFS Storage Appliances.............................................. 3Configuring the Oracle ZFS Storage Appliance.................................................................... 3Configuring for Clustered Appliances.................................................................................... 4Configuring Database-to-Appliance Mappings ..................................................................... 4Plug-in Management in the Oracle Enterprise Manager Cloud Control 12c Environment..5Deploying the Plug-in............................................................................................................. 5Adding Instances for Monitoring............................................................................................ 7Adding Clustered Systems for Monitoring............................................................................. 8Viewing Metrics and Reports................................................................................................. 8Upgrading the Plug-in........................................................................................................... 10Un-deploying the Plug-in...................................................................................................... 10Metric and Collection Information....................................................................................... 11Metric Collections on Datasets............................................................................................. 17Metric Collections on Pools, Projects and Shares............................................................... 18Reports................................................................................................................................... 18Default Threshold Values..................................................................................................... 20Stateless Alerts...................................................................................................................... 24Oracle ZFS Storage Appliances Alerts................................................................................. 25Communicating Performance Issues to the Appliance Administrator.................................25Troubleshooting Monitoring Deployments........................................................................... 26Metric Collection Errors....................................................................................................... 29Oracle ZFS Storage Appliance Audit Logs......................................................................... 31Host Agent Logs .................................................................................................................. 32Additional Resources............................................................................................................ 33
Oracle Enterprise Manager System Monitoring Plug-in for Oracle ZFS Storage Appliance Installation Guide v1.4.1
Introduction
The Oracle ZFS Storage Appliance Plug-in extends Oracle Enterprise Manager Cloud Control to add support for monitoring Oracle ZFS Storage Appliances. This guide provides an overview of the plug-in and how to install and deploy it for monitoring Oracle ZFS Storage Appliances.
This guide is written for the Oracle Enterprise Manager Cloud Control administrator. As Cloud Control administrator, you will communicate closely with the Oracle ZFS Storage Appliance storage administrator to discuss performance and analysis of database deployments on the Oracle ZFS Storage Appliance.
If you need additional information about the operation of plug-ins for the Oracle Enterprise Manager Cloud Control product, see the Oracle Enterprise Manager Concepts Guide. You can download the document, as well as other related documentation, from the Oracle Web site (see the section at the end of this document).
Important: Oracle Enterprise Manager Cloud Control was previously called Oracle Enterprise Manager Grid Control (versions 10g and 11g). These older versions are no longer being supported, but users familiar with them should note that references to grid control are now replaced with cloud control. These legacy versions' documentation still reflects the old terminology.
Also note that the Sun ZFS Storage Appliance is now called the Oracle ZFS Storage Appliance, and similarly to the Oracle Enterprise Manager software, legacy Appliance Kit software versions and their documentation may still reflect the former name. References to Sun ZFS Storage Appliance, Sun ZFS Storage 7000, and Sun ZFS Backup Appliance all refer to the same family of Oracle ZFS Storage Appliances.
1
Oracle Enterprise Manager System Monitoring Plug-in for Oracle ZFS Storage Appliance Installation Guide, v1.4.1
About the Oracle ZFS Storage Appliance Plug-in
The Oracle ZFS Storage Appliance plug-in for Oracle Enterprise Manager Cloud Control provides first-class monitoring for all Oracle ZFS Storage Appliance models. The plug-in provides the following primary features:
Gathers and presents storage system, configuration, and performance information for Oracle ZFS Storage Appliances
Raises alerts for pre-selected configuration and monitoring data
Provides out-of-the-box reports that complement Oracle ZFS Storage Analytics reports
Ties together Oracle databases deployed on NFS shares from Oracle ZFS Storage Appliances
Supports monitoring by remote agents in the Cloud Control environment
When you configure the Oracle ZFS Storage Appliance to be monitored by Oracle Enterprise Manager Cloud Control, an analytics worksheet and datasets are enabled. The worksheet and datasets help bridge a Cloud Control administrator's view of the appliance to the storage administrator's more detailed view. This enables both administrators to be fully conversant in discussing performance or analysis of database deployments on the appliance.
Software Requirements
Deployment of the Oracle Enterprise Manager System Monitoring Plug-in for Oracle ZFS Storage Appliance requires that the following software versions already be installed:
Oracle Enterprise Manager Cloud Control 12c Release 3 (12.1.0.3)
Oracle ZFS Storage Appliance 2010 Q3.4.4 software or later
Note: Additional software requirements, such as the correct Java version in Cloud Control, are met through the required applications listed, provided the correct versions are installed. The plug-in can be installed on any operating system in which Enterprise Manager Cloud Control is running.
2
Oracle Enterprise Manager System Monitoring Plug-in for Oracle ZFS Storage Appliance Installation Guide, v1.4.1
Prerequisites for Monitoring Oracle ZFS Storage Appliances
Before you can deploy and monitor Oracle ZFS Storage Appliances, you must:
Install the Oracle Enterprise Manager Cloud Control environment. Be sure to pay attention to security sections in the Oracle Enterprise Manager Administrator's Guide.
Configure the Oracle ZFS Storage Appliances for monitoring using the workflow included with Oracle ZFS Storage Appliances, as identified in the next section.
Configuring the Oracle ZFS Storage Appliance
To set up each appliance for monitoring from Cloud Control, enter the Oracle ZFS Storage Appliance Browser User Interface (BUI) as root or another user with the proper role to create worksheets, enable datasets, and create users and user roles on the appliance.
1. Log in to the BUI and go to the Maintenance/Workflows tab.
2. Run the Configure for Oracle Enterprise Manager Monitoring workflow, entering and recording the password for the oracle_agent user. You can rerun the workflow later to reset the worksheet, re-enable data sets, or recreate the user. The following occurs when the workflow is run:
A restricted role user named oracle_user is created for use by the monitoring agent. The restricted access role associated with the oracle_agent user is also named oracle_agent. An analytics worksheet is automatically created to ensure smooth communication between the Cloud Control administrator and the storage administrator tasked with administering the target Oracle ZFS Storage Appliance.
3. Communicate the oracle_agent password to the Cloud Control Administrator along with the following information:
IP address for the network interface that you want Cloud Control to use to access management information (this interface must allow administration).
Full DNS name that is used to locate the system and corresponds to the IP address.
3
Oracle Enterprise Manager System Monitoring Plug-in for Oracle ZFS Storage Appliance Installation Guide, v1.4.1
Port that is used by the BUI (typically 215 unless your company has a requirement to change management ports).
Important: If you are going to deploy/upgrade to v1.4.x of the plugin, you must carry out the following additional steps:
1. Log in to the BUI and go to the Configuration/Users tab.
2. Click to edit the Oracle Agent User name field.
3. Uncheck Kiosk user if it is checked.
Configuring for Clustered Appliances
If the Oracle Enterprise Manager Cloud Control will monitor resources available on a clustered Oracle ZFS Storage Appliance, you must configure both cluster heads. The oracle_agent user and worksheet will automatically be created on the peer head, but datasets will not become active on the peer. If the head containing the monitored resources fails, Oracle Enterprise Manager Cloud Control monitors the resources on the head that takes over the exported resources from the failed head. If the configuration step was not run on this head, the datasets might not be active, resulting in Metric Collection errors with the message workflow parameters are invalid. (See Table 5: Common Metric Collection Errors .)
Note: When running the Configure for Oracle Enterprise Manager Monitoring workflow on the second head, the oracle_agent user does not have to be "Recreated." If you choose the option to Recreate the user, be sure to use the same password as the peer head set (the oracle_agent password will always be the same on both heads of the appliance).
Configuring Database-to-Appliance Mappings
In order for reports that include database-to-appliance mappings to work, the database and the host on which the database resides must be monitored by Oracle Enterprise Manager Cloud Control. For instructions on monitoring host storage and database instances, refer to the documentation provided with Oracle Enterprise Manager Cloud Control software (see the References section at the end of this guide ).
4
Oracle Enterprise Manager System Monitoring Plug-in for Oracle ZFS Storage Appliance Installation Guide, v1.4.1
Plug-in Management in the Oracle Enterprise Manager Cloud Control 12c Environment
This section describes how to deploy and use the Oracle Enterprise Manager System Monitoring Plug-in in the Oracle Enterprise Manager Cloud Control 12c environment. If you need to upgrade your Oracle Enterprise Manager Cloud Control 11g environment to Cloud Control 12c, please follow the instructions at:
http://www.oracle.com/technetwork/oem/grid-control/downloads/oem-upgrade-console-502238.html
The Oracle Enterprise Manager Cloud Control Upgrade Guide is located at:
http://download.oracle.com/docs/cd/E24628_01/upgrade.121/e22625.pdf
For further detailed instructions on the Oracle Enterprise Manager's Plug-in Manager and the plug-in deployment process, see section 18 "Plug-in Manager" of the Oracle Enterprise Manager Cloud Control 12c Administrator's Guide at:
http://docs.oracle.com/cd/E24628_01/doc.121/e24473/plugin_mngr.htm
Deploying the Plug-in
Deploying the Oracle Enterprise Manager System Monitoring Plug-in in the Oracle Enterprise Manager Cloud Control 12c environment is a two-step process. First, the plug-in is imported into the Oracle Enterprise Manager Cloud Control 12c environment using the emcli command. Then the Oracle Enterprise Manager console is used to deploy the plug-in by installing it on the Oracle Enterprise Manager's Management Service and Management Agent, and adding the Oracle ZFS Storage Appliance as a target. The emcli command can also be used to monitor the installation of the plug-in. These steps are described in more detail in the following instructions.
Note: The Plug-in must be deployed on Oracle Enterprise Manager's Management Service before being deployed on Management Agents. The Oracle ZFS Storage Appliances can only be added as targets for monitoring once the Plug-in is deployed on both the Management Service and the Management Agent.
To import the plug-in into the Oracle Enterprise Manager Cloud Control 12c environment, complete these steps:
1. Set up the software library path (if you have not already done this).5
Oracle Enterprise Manager System Monitoring Plug-in for Oracle ZFS Storage Appliance Installation Guide, v1.4.1
(a) From a terminal window on the server hosting Oracle Enterprise Manager 12c , create a new directory for the software library; for example: /home/oracle/swlib1.
(b) In the Oracle Enterprise Manager console, go to Enterprise>Provisioning and Patching> Software Library>Actions>Administration and click Add Button.
(c) In the popup window, enter swlib1 and /home/oracle/swlib1.
(d) Wait a minute or so. Then, on the Oracle Enterprise Manager host, verify that the following directory structure was created under /home/oracle/swlib1:
[oracle@aie-4200i swlib1]$ ls
0 11 14 17 2 22 25 28 30 33 36 39 41 44 47 5 52 55 58 60 63 66 69 71 9 1 12 15 18 20 23 26 29 31 34 37 4 42 45 48 50 53 56 59 61 64 67 7 72 10 13 16 19 21 24 27 3 32 35 38 40 43 46 49 51 54 57 6 62 65 68 70 8
2. Install the emcli kit.(a) Download emclikit.jar by following instructions available on the Enterprise
Manager Console: Setup>Command Line Interace.
(b) Copy the downloaded kit to a new folder; for example: /home/oracle/emcli.(c) Install the kit and set it up to work with your EM by entering:
java -jar emclikit.jar client -install_dir=/home/oracle/emcli
./emcli setup -dir='/home/oracle/emcli' -url='https://:port/em' -user=sysman
(d) Download the plug-in file (*.opar file) to a directory; for example: /home/oracle/12g_plugins
(e) Upload the plug-in using the emcli command as shown in the following example:
./emcli import_update -file= "/home/oracle/12g_plugins/12.1.0.1.0_oracle.sun.oss7_2000_0.opar" -omslocal
Note: On a Windows system, the emcli command does not require any quotes.To deploy the plug-in on the Oracle Enterprise Manager using the Oracle Enterprise Manager console, complete these steps:
6
Oracle Enterprise Manager System Monitoring Plug-in for Oracle ZFS Storage Appliance Installation Guide, v1.4.1
1. Go to Setup > Extensibility > Self Update. Click on Plugin. Make sure Oracle ZFS Storage Appliances is listed as Downloaded. Select it and click on Actions > Readme for the details of the plug-in.
2. Click on the Plug-in tab or Setup > Extensibility > Plugin. Expand the Servers, Storage and Network category.
3. Select the plug-in Oracle ZFS Storage Appliances from the table displayed in step 2 and select: Actions > Deploy On > Management Servers.
4. In the popup window, enter the Repository Password and click Continue.
5. Wait until the prerequisite checks are done. Click Next and then Deploy. An emcli command is displayed that can be run to monitor the status of the deployment and let you know when it has completed successfully. For example:
emcli get_plugin_deployment_status -plugin_id=oracle.sun.oss7
To install the Oracle Enterprise Manager System Monitoring Plug-in on a local Management Agent, complete the following steps:
1. Select Setup > Extensibility > Plugins.
2. Click and highlight the plug-in that you installed on the Management Server.
3. Select Deploy On > Management Agent.
4. Click Add.
5. In the Target Type drop-down list, select Agent and highlight the local agent displayed in the table, then click Select.
6. Click Continue.
7. Wait until the prerequisite checks are done. Click Next and then Deploy. An emcli command is displayed that can be run to monitor the status of the deployment and let you know when it has completed successfully. For example:
emcli get_plugin_deployment_status -plugin_id=oracle.sun.oss7
Note: It will take about 3 minutes to complete this process.
Adding Instances for Monitoring
To install the Oracle Enterprise Manager System Monitoring Plug-in on a local Management Agent, complete the following steps:
1. Select Setup -> Add Target -> Add Targets Manually.
7
Oracle Enterprise Manager System Monitoring Plug-in for Oracle ZFS Storage Appliance Installation Guide, v1.4.1
2. Select Add Targets Declaratively by Specifying Target Monitoring Properties .
3. In the Target Type drop-down list, select Oracle ZFS Storage Appliance.
4. For Monitoring Agent, click Search. In the window that is displayed, for Target Type drop-down list, select Agent and from the table below, click and highlight the agent you want to use for monitoring your target. Click Select.
5. Click Add Manually.
6. Add the Target details and click OK. Wait until you see the confirmation and click OK.
7. To find the target, click on Targets->All Targets.
8. Select the Oracle ZFS Storage Appliance added above to go to the appliance home page.
Adding Clustered Systems for Monitoring
Clustered systems require consideration by the Oracle Enterprise Manager Cloud Control administrator. Unless Cloud Control is being used as a general purpose Configuration Management Database (CMDB) or general purpose asset and resources tracking, only the head or heads containing active exported resources in use by other monitored software resources should be monitored by Oracle Enterprise Manager Cloud Control (even though the configuration workflow must be run on both cluster heads).
If a storage head fails and an alternate head takes over the resources, the new head owning the logical resources will become monitored (if the takeover head was properly configured). Within the metric reports, the status of the cluster is given. The cluster status is important, as shared resources in an active/active environment will become even more pressured and the actual event of a cluster failover can cause disruption in quality of service objectives.
Viewing Metrics and Reports
The two primary ways to gather information about monitored instances of an Oracle ZFS Storage Appliance are viewing metrics and accessing reports. These are each described in the following sections.
IMPORTANT: By default, all metrics that collect information from datasets on Oracle ZFS Storage Appliances are disabled upon installation. Likewise, certain metrics for pools, projects, and shares are disabled too. Further details on types of metrics as well as their
8
Oracle Enterprise Manager System Monitoring Plug-in for Oracle ZFS Storage Appliance Installation Guide, v1.4.1
settings and how to manage them are included in the section Metric and Collection Information later in this document.
Viewing Metrics
As a rule, more point in time information is available in raw metric information than in reports. Reports are focused on information that database administrators can use to analyze database performance and storage capacity.
To view the raw information collected from an Oracle ZFS Storage Appliance:
1. Log in to Enterprise Manager Cloud Control.
2. Go to Targets > All Targets and select the Oracle ZFS Storage Appliance as the target.
3. From the target's home page, select Oracle ZFS Storage Appliance > Monitoring > All Metrics.
4. View the categories and information collected from the last collection interval.
The raw metric information that you have access to can be found in Table 1: Metric and Collection Information.
Accessing Reports
To view reports for an Oracle ZFS Storage Appliance:
1. Log in to Enterprise Manager Cloud Control.
2. Go to Targets > All Targets and select the Oracle ZFS Storage Appliance as the target.
3. From the target's home page, select Target > Information Publisher Reports.
The information to which you have access is the same reports as those in the Enterprise Manager 10g/11g environment. Please refer to Table 2: Reports
Reports accessed using a target instance differ from reports accessed from the Reports tab within Oracle Enterprise Manager Cloud Control. All instance reports are available under the tabs Reports >Storage>Network Attached Storage. When accessed from the Reports tab, reports include configurable time spans for graph display where applicable.
Note: Administrators can clone and customize all system reports from the Reports tab.
9
Oracle Enterprise Manager System Monitoring Plug-in for Oracle ZFS Storage Appliance Installation Guide, v1.4.1
Upgrading the Plug-in
Ensure the following steps are performed on the Oracle ZFS Storage Appliance.
1. Log in to the BUI and go to the Configuration/Users tab.
2. Click to edit the Oracle Agent user field.
3. Uncheck Kiosk user if it is checked.
Note: If you are upgrading the Plug-in from v1.x to v1.4.x, you must ensure that you are on Oracle Enterprise Manager 12.1.0.3.0 and the following steps are performed.
After importing v1.4.x of the Plug-in and deploying it on an Oracle Enterprise Manager Management Service and Management Agent, you must restart Oracle Enterprise Manager's Management Service. The steps to do so are:
1. Log in to the host where Oracle Enterprise Manager's Management Service is residing.
2. Navigate to where Oracle Enterprise Manager's Management Service is installed (typically in a folder called Middleware).
3. Navigate to Middleware/oms/bin and run the following two commands:
emctl stop oms
emctl start oms
Un-deploying the Plug-in
Oracle Enterprise Manager System Monitoring Plug-in can be un-deployed from the agent and Oracle Enterprise Manager. By un-deploying the plug-in from the agent, all the associated targets on the Oracle ZFS Storage Appliance that are monitored by the agent are automatically removed.
To un-deploy the plug-in from the agent and Oracle Enterprise Manager:
1. Go to Setup > Extensibility > Plug-ins.
2. Under the Servers, Storage and Network category, select Oracle ZFS Storage Appliances.
3. Go to Undeploy From > Management Agent > Add.
4. Select the Agent from which the plug-in is to be un-deployed and click Continue.
10
Oracle Enterprise Manager System Monitoring Plug-in for Oracle ZFS Storage Appliance Installation Guide, v1.4.1
An emcli command that you can run from the command prompt to monitor the status of the undeployment activity is displayed.
5. To remove the plug-in from the management servers, repeat preceding steps 1-4 while selecting Management Servers in step 3.
To permanently delete the plug-in:
1. Go to Setup > Extensibility > Self Update > Plug-ins.
2. Under the Servers, Storage and Network category, select Oracle ZFS Storage Appliances.
3. Select Actions > Remove. This will remove the plug-in permanently from the system.
Metric and Collection Information
Oracle Enterprise Manager Cloud Control displays a direct mapping of information to datasets and information collected in the target Oracle ZFS Storage Appliances.
Note: Information collected by the System Attributes from the Workflow dataset indicates items that cannot be enabled/disabled by an administrator. This information is collected through scripts on each Oracle ZFS Storage Appliance.
TABLE 1: METRIC AND COLLECTION INFORMATION
METRIC NAME
(X=DISABLED BY DEFAULT)
COLUMN (*=KEY FIELD,
! = CONDITIONS ENABLED)
ORACLE ZFS STORAGE
APPLIANCE DATASET
POLLING
INTERVAL
(MINUTES)
UPLOAD FREQUENCY
TO EM REPOSITORY
Response Status N/A (request for host name) 5 Every polling cycle.
Monitoring Prerequisite Status! N/A 30 Upload when there is a
severity change on the
thresholds.Message
X Appliance Logical Status Share Identifier* System Attribute from
Workflow
20 Every 72nd polling cycle
(or once per day) or
whenever there is a
severity change on the
thresholds.
Available (GB)
Total Allocated (GB)
Data (GB)!
11
Oracle Enterprise Manager System Monitoring Plug-in for Oracle ZFS Storage Appliance Installation Guide, v1.4.1
METRIC NAME
(X=DISABLED BY DEFAULT)
COLUMN (*=KEY FIELD,
! = CONDITIONS ENABLED)
ORACLE ZFS STORAGE
APPLIANCE DATASET
POLLING
INTERVAL
(MINUTES)
UPLOAD FREQUENCY
TO EM REPOSITORY
Share Percentage Used (%)!
Quota (GB)
Reserved (GB)!
Unused Reserved (GB)!
SCSI LUN
X Appliance Project Status Project Identifier* System Attribute from
Workflow
20 Every 72nd polling cycle
(or once per day) or
whenever there is a
severity change on the
thresholds.
Available (GB)
Total Allocated (GB)
Project Data (GB)!
Project Percentage Used!
Quota (GB)
Reserved (GB)!
Unused Reserved (GB)!
Total Share Quota (GB)!
Project Over Allocated!
Configuration Service Status Service Name* System Attribute from
Workflow
60 Every 24th polling cycle
(or once per day) or
whenever there is a
severity change on the
thresholds.
Service Status!
X Appliance Capacity Totals Storage Pool* System Attribute from
Workflow
20 Every 72nd polling cycle
(or once per day) or
whenever there is a
severity change on the
thresholds.
Pool Owner
Pool Status
Data Profile
Available (GB)
12
Oracle Enterprise Manager System Monitoring Plug-in for Oracle ZFS Storage Appliance Installation Guide, v1.4.1
METRIC NAME
(X=DISABLED BY DEFAULT)
COLUMN (*=KEY FIELD,
! = CONDITIONS ENABLED)
ORACLE ZFS STORAGE
APPLIANCE DATASET
POLLING
INTERVAL
(MINUTES)
UPLOAD FREQUENCY
TO EM REPOSITORY
Total Used (GB)
Pool Used Percentage (%)!
Total Capacity (GB)
Data Disks
Spare Disks!
Log Disks
Cache Disks
Total Project Quota (GB)!
Quota Allocation (GB)!
Reservations (GB)!
Pool Over Allocated!
X Cache ARC Size (MB)! arc.size 30 Every polling cycle.
X Cache L2ARC Size (MB)! arc.l2_size 30 Every polling cycle.
X CPU Utilization %! cpu.utilization 30 Every polling cycle.
X Disk I/O Ops Operations per Second! io.ops 30 Every polling cycle.
X Disk I/O Ops Latency Operations per Second! io.ops[latency] 30 Every polling cycle.
X Disk I/O Bytes KB per Second! io.bytes 30 Every polling cycle.
X Network Device KB per Second! nic.kilobytes 30 Every polling cycle.
X NFSv2 Ops Operations per Second! nfs2.ops 30 Every polling cycle.
X NFSv3 Ops Operations per Second! nfs3.ops 30 Every polling cycle.
X NFSv3 Ops Latency Operations per Second! nfs3.ops[latency] 30 Every polling cycle.
X NFSv4 Ops Latency Operations per Second! nfs4.ops[latency] 30 Every polling cycle.
X NFSv4 Ops Operations per Second! nfs4.ops 30 Every polling cycle.
13
Oracle Enterprise Manager System Monitoring Plug-in for Oracle ZFS Storage Appliance Installation Guide, v1.4.1
METRIC NAME
(X=DISABLED BY DEFAULT)
COLUMN (*=KEY FIELD,
! = CONDITIONS ENABLED)
ORACLE ZFS STORAGE
APPLIANCE DATASET
POLLING
INTERVAL
(MINUTES)
UPLOAD FREQUENCY
TO EM REPOSITORY
X SMB Ops Operations per Second! smb.ops 30 Every polling cycle.
X SMB Ops Latency Operations per Second! smb.ops[latency] 30 Every polling cycle.
X iSCSI Ops Operations per Second! iscsi.ops 30 Every polling cycle.
X iSCSI Ops Latency Operations per Second! iscsi.ops[latency] 30 Every polling cycle.
Initiator And Target Groups Protocol* System Attribute from
Workflow
60 Every 24th polling cycle
(or once per day).
Group Name*
Address
Initiator Or Target
X Cache ARC Activity Activity* arc.accesses[hit/miss] 30 Every polling cycle.
Accesses per Second! arc.accesses[hit/miss]
X Cache L2ARC Activity Activity* arc.l2_accesses[hit/miss] 30 Every polling cycle.
Accesses per Second! arc.l2_accesses[hit/miss]
X Cache ARC Activity by Share Share* arc.accesses[share] 30 Every polling cycle.
Accesses Per Second! arc.accesses[share]
X Cache L2ARC Activity by
Share
Share* arc.l2_accesses[share] 30 Every polling cycle.
Accesses Per Second! arc.l2_accesses[share]
X NFS V3 Operations By Share NFS V3 Share* nfs3.ops[share] 30 Every polling cycle.
Operations Per Second! nfs3.ops[share]
X NFS V4 Operations By Share NFS V4 Share* nfs4.ops[share] 30 Every polling cycle.
Operations Per Second! nfs4.ops[share]
X Network I/O Direction Direction* nic.kilobytes[direction] 30 Every polling cycle.
Kilobytes Per Second! nic.kilobytes[direction]
X Network I/O by Device Device* nic.kilobytes[device] 30 Every polling cycle.
14
Oracle Enterprise Manager System Monitoring Plug-in for Oracle ZFS Storage Appliance Installation Guide, v1.4.1
METRIC NAME
(X=DISABLED BY DEFAULT)
COLUMN (*=KEY FIELD,
! = CONDITIONS ENABLED)
ORACLE ZFS STORAGE
APPLIANCE DATASET
POLLING
INTERVAL
(MINUTES)
UPLOAD FREQUENCY
TO EM REPOSITORY
Kilobytes Per Second! nic.kilobytes[device]
X Network Interface
Performance
Interface* net.kilobytes[interface] 30 Every polling cycle.
Kilobytes Per Second! net.kilobytes[interface]
Physical Component Status Cluster State! System Attributes from
Workflow
60 Every 24th polling cycle
(or once per day) or
whenever there is a
severity change on the
thresholds.
Cluster State Description
Cluster Peer State
Cluster Peer State Description
Appliance Information Reported Node Name System Attributes from
Workflow
60 Every 24th polling cycle
(or once per day). Product Model
Appliance Serial Number
Chassis Serial Number
Appliance Software Version
Install Time
Last Update Time
Power On Time
Up Time
Clustered
Cluster Peer
X Appliance Logical Information Share Identifier* System Attributes from
Workflow
60 Every 24th polling cycle
(or once per day). Share Type
Mount Point
Data Profile
Data Compression
Cache Usage
NFS
CIFS
15
Oracle Enterprise Manager System Monitoring Plug-in for Oracle ZFS Storage Appliance Installation Guide, v1.4.1
METRIC NAME
(X=DISABLED BY DEFAULT)
COLUMN (*=KEY FIELD,
! = CONDITIONS ENABLED)
ORACLE ZFS STORAGE
APPLIANCE DATASET
POLLING
INTERVAL
(MINUTES)
UPLOAD FREQUENCY
TO EM REPOSITORY
IP Network Interfaces Interface* System Attributes from
Workflow
60 Every 24th polling cycle
(or once per day).IP Address*
CIDR
IP Version
Management Enabled
IP Network Interface to Datalink
Relationship
Interface* System Attributes from
Workflow
60 Every 24th polling cycle
(or once per day).Datalink*
Jumbo Frames
IP Datalink to Device
Relationship
Datalink* System Attributes from
Workflow
60 Every 24th polling cycle
(or once per day).Link*
Class
IP Network Devices Network Device* System Attributes from
Workflow
60 Every 24th polling cycle
(or once per day) or
whenever there is a
severity change on the
thresholds.
Device Speed!
Device Up!
Alert Log Contents Problem ID* System Attributes from
Workflow
20 Every polling cycle.
Time Stamp
Alert Description!
Alert Log Problem Type!
Fault Log Contents Problem ID* System Attributes from
Workflow
20 Every polling cycle.
Time Stamp
Description!
Fault Log Problem Type!
Problem Log Contents UUID* System Attributes from
Workflow
20 Every polling cycle.
Description!
Problem Severity!
Problem Log Code!
Replication Activity Problem ID* 20 Every polling cycle.
16
Oracle Enterprise Manager System Monitoring Plug-in for Oracle ZFS Storage Appliance Installation Guide, v1.4.1
METRIC NAME
(X=DISABLED BY DEFAULT)
COLUMN (*=KEY FIELD,
! = CONDITIONS ENABLED)
ORACLE ZFS STORAGE
APPLIANCE DATASET
POLLING
INTERVAL
(MINUTES)
UPLOAD FREQUENCY
TO EM REPOSITORY
System Attributes from
Workflow
Time Stamp
Replication Description!
Replication Log Problem Type
Metric Collections on Datasets
By default, all Metrics that collects information from datasets on Oracle ZFS Storage Appliances are disabled. If you are interested in any particular dataset, please enable the Metric Collection for that dataset.
There are two methods to enable metric collection for datasets on Oracle Enterprise Manager.
Method 1:
1. Go to the Target Homepage.
2. Click on the Oracle ZFS Storage Appliance Menu > Monitoring > All Metrics.
3. Select the Metric that you wish to enable.
4. Click on the Modify button.
5. Click on the Enabled radio button for Data Collection and Click OK.
Method 2:
1. Go to the Target Homepage.
2. Click on the Oracle ZFS Storage Appliance Menu > Monitoring > Metric and Collections Setting.
3. Select All Metrics under the View drop-down menu.
4. Find the Metric that you wish to enable.
5. Select the Disabled link under the Collection Schedule column.
6. Click on the Enabled button under Data Collection and click Continue.
7. You will be brought back to the Metric and Collections Settings page with a notification box. Ensure that you click on OK.
17
Oracle Enterprise Manager System Monitoring Plug-in for Oracle ZFS Storage Appliance Installation Guide, v1.4.1
You can use the previous steps to disable a dataset collection at any time.
Note: Once you enable the metric collection for any dataset, you must ensure that the storage administrators also enable the same dataset on the Oracle ZFS Storage Appliances; otherwise you will start receiving metric collection errors.
Metric Collections on Pools, Projects and Shares
The following metrics that collect information about the pool, projects, and shares information are disabled by default:
Appliances Capacity Total
Appliances Project Status
Appliances Logical Status
Appliances Logical Configuration
Please follow the same steps shown previously for enabling dataset metrics if you would like to collect information on these metrics.
Note: Metrics and information presented on charts can take up to 60 minutes to populate as a result of polling intervals for relatively static information. Also, due to differing polling cycles between statistics, some information can populate prior to other information.
Reports
The following table lists available reports and the information they can provide on the Oracle ZFS Storage Appliances status.
18
Oracle Enterprise Manager System Monitoring Plug-in for Oracle ZFS Storage Appliance Installation Guide, v1.4.1
TABLE 2: REPORTS
REPORT DESCRIPTION PURPOSE
Monitored System Home Page Contains a series of graphs that give a high-level view
of system throughput and clients accessing the
system. It is important to note whether a system is Up
or Down. Up means the management interface is
available and simple properties can be queried that
do not require login. Down means the management
interface is not available. If a system is set up with
more than one interface that allows access to the
data path, it could be Down when the data path
interface is still accessible. The management
interface being flagged as Down is sufficient to
warrant attention to the system.
Provides at-a-glance status of the system.
Database to Appliance Mapping
Report -- available from the
Reports tab or from a
preselected Target Report's tab
Shows mapping of databases using NFS to access
data residing on an Oracle ZFS Storage Appliance.
Uses Detail Report links to view deeper information
on mappings and component activity. Detail Reports
provide information such as:
Database file mapping to exports residing
on appliances
Export data on which files reside
(including capacity consumption and
cache activity)
The appliance on which a share resides
The network interface and its activity
through which a database file is accessed
Provides a detailed view of how an Oracle
ZFS Storage Appliance is being accessed
using Oracle databases. Provides easy
navigation to drill into the configuration
and activity of Oracle ZFS Storage
Appliance components that relate to the
database deployment.
Storage Capacity Report --
available from the Reports tab
or from a preselected Target
Report's tab
Gives detailed information about how storage is used
on an individual Oracle ZFS Storage Appliance.
Shows how capacity is allocated on an
Oracle ZFS Storage Appliance.
Network Interface Information
Report -- available from the
Reports tab or from a
preselected Target Report's tab
Provides details of network configuration and
generalities about network activity over the past 24
hours.
Provides a detailed view of how ports on
an Oracle ZFS Storage Appliance are
configured. Shows aggregate activity of
the interfaces.
19
Oracle Enterprise Manager System Monitoring Plug-in for Oracle ZFS Storage Appliance Installation Guide, v1.4.1
REPORT DESCRIPTION PURPOSE
Federated Database to
Appliance Mapping Report --
available from Reports tab>
Storage>Network Attached
Storage group
Similar to the Database to Appliance Mapping Report,
but applicable to all monitored Oracle ZFS Storage
Appliances.
Provides a detailed view of how all
monitored Oracle ZFS Storage Appliances
are being accessed for information
through Oracle databases. Provides easy
navigation to drill into the configuration
and activity of individual Oracle ZFS
Storage Appliances and their components.
Federated Capacity and
Configuration Report -- available
from Reports tab>Storage>
Network Attached Storage group
Gives high-level information about all monitored
Oracle ZFS Storage Appliances.
Provides a side-by side-comparison of all
Oracle ZFS Storage Appliances (and
software installed on them) to help you
make decisions on where to allocate
capacity.
Default Threshold Values
You can set custom thresholds for some metrics within Oracle Enterprise Manager Cloud Control. The alerts received are contained within the product and are not set as Alerts and Thresholds on the Oracle ZFS Storage Appliance itself.
The following table shows the metrics that have thresholds set with their default values.
TABLE 3: METRICS AND DEFAULT THRESHOLD VALUES
METRIC/COLUMNS COMPARISON
OPERATOR
WARNING CRITICAL PURPOSE
Alert Log Contents/Alert Description Matches N/A N/A Issued when the Alert Description Matches (via
Regular Expression) the thresholds.
Alert Log Contents/Alert Log Problem
Type
Matches N/A N/A Issued when the Alert Log Problem Type Matches
(via Regular Expression) the thresholds.
Problem Type can be Minor/Major/Critical.
Appliances Capacity Totals/Pool Over
Allocated
= N/A YES Issued when the Total Project Quota (GB) crosses
the Pool's Total Capacity (GB).
Appliances Capacity Totals/Pool Used
Percentage
>= 80 90 Issued when the total capacity available on this
appliance crosses the threshold.
Appliances Capacity Totals/Quota
Allocations
>= N/A N/A Issued when the Quota Allocations crosses the
thresholds.
20
Oracle Enterprise Manager System Monitoring Plug-in for Oracle ZFS Storage Appliance Installation Guide, v1.4.1
METRIC/COLUMNS COMPARISON
OPERATOR
WARNING CRITICAL PURPOSE
Appliances Capacity
Totals/Reservations
>= N/A N/A Issued when the Reservation crosses the
thresholds.
Appliances Capacity Totals/Spare
Disks
= N/A N/A Issued when the Total Project Quota crosses the
thresholds.
Appliances Logical Status/Reserved = N/A N/A Issued when the Reserved value is equal to the
thresholds.
Appliances Logical Status/Share Data >= N/A N/A Issued when the Share Data crosses the
thresholds.
Appliances Logical Status/Share
Percentage Used
>= 80 90 Any share issues these warnings. A Cloud Control
administrator can make specific shares trigger on
other values.
Appliances Logical Status/Unused
Reserved
= N/A N/A Issued when the Project Data crosses the
thresholds.
Appliances Project Status/Project Over
Allocated
= N/A YES Issued when the Total Share Quota (GB) of any
project crosses the corresponding Project Quota
(GB).
Appliances Project Status/Project
Percentage Used
>= 80 90 Issued when the Project Percentage used crosses
the thresholds.
Appliances Project Status/Reserved = 0 N/A Issued when the Reserved value is equal to the
thresholds.
Appliances Project Status/Total Share
Quota
>= N/A N/A Issued when the Total Share Quota used crosses
the thresholds.
Appliances Project Status/Unused
Reserved
= N/A N/A Issued when Cache ARC (MB) crosses the
thresholds.
21
Oracle Enterprise Manager System Monitoring Plug-in for Oracle ZFS Storage Appliance Installation Guide, v1.4.1
METRIC/COLUMNS COMPARISON
OPERATOR
WARNING CRITICAL PURPOSE
Cache ARC Activity >= N/A N/A Issued when Cache ARC Activity (Accesses per
Second) crosses the thresholds.
Cache ARC Activity by Share >= N/A N/A Issued when Cache ARC Activity by Share
(Accesses per Second) crosses the thresholds.
Cache L2ARC >= N/A N/A Issued when Cache L2ARC (MB) crosses the
thresholds.
Cache L2ARC Activity >= N/A N/A Issued when Cache L2ARC Activity (Accesses per
Second) crosses the thresholds.
Cache L2ARC Activity by Share >= N/A N/A Issued when Cache L2ARC Activity by Share
(Accesses per Second) crosses the thresholds.
Configuration Service Status = N/A N/A This threshold is enabled but left off because a
service that is down does not necessarily indicate a
problem (the service might not be configured or
operating by plan).
CPU Utilization % >= 80.00% 90.00% Issued after three separate polling intervals. See
the CPU Utilization above this percentage. Overuse
of CPU can result in lower performance.
Disk I/O Bytes >= N/A N/A Issued when Disk I/O Bytes (KB/S) crosses the
thresholds.
Disk I/O Ops >= N/A N/A Issued when Disk I/O Ops (Operation per Second)
crosses the thresholds.
Disk I/O Ops Latency >= N/A N/A Issued when Disk I/O Ops Latency (Operation per
Second) crosses the thresholds.
Fault Log Contents/Description Matches N/A N/A Issued when the Fault Description Matches (via
Regular Expression) the thresholds.
Fault Log Contents/Fault Log Problem
Type
Matches N/A N/A Issued when the Fault Log Problem Type Matches
(via Regular Expression) the thresholds.
Problem Type can be Minor/Major/Critical.
IP Network Device/Device Speed
Oracle Enterprise Manager System Monitoring Plug-in for Oracle ZFS Storage Appliance Installation Guide, v1.4.1
METRIC/COLUMNS COMPARISON
OPERATOR
WARNING CRITICAL PURPOSE
IP Network Device/Device Up = N/A N/A This threshold is enabled but left off because a
device that is down does not necessarily indicate a
problem (the device might not be configured or
operating by plan).
iSCSI Ops >= N/A N/A Issued when iSCSI Ops (Operation per Second)
crosses the thresholds.
iSCSI Ops Latency >= N/A N/A Issued when iSCSI Ops Latency (Operation per
Second) crosses the thresholds.
Monitoring Prerequisite/Status = N/A 0 Issued when the credentials provided are incorrect.
Once this alert is triggered, all metric collection will
stop until this alert is cleared.
Network Device >= N/A N/A Issued when Network Device (KB/S) crosses the
thresholds.
Network I/O by Device >= N/A N/A Issued when Network I/O by Device (KB/S) crosses
the thresholds.
Network Interface Performance >= N/A N/A Issued when Network Interface Performance
(KB/S) crosses the thresholds.
NFSv2 Ops >= N/A N/A Issued when NFSv2 Ops (KB/S) crosses the
thresholds.
NFSv3 Ops >= N/A N/A Issued when NFSv3 Ops (KB/S) crosses the
thresholds.
NFSv3 Ops by Share >= N/A N/A Issued when NFSv3 Ops by Share Ops (KB/S)
crosses the thresholds.
NFSv3 Ops Latency >= N/A N/A Issued when NFSv3 Ops Latency (KB/S) crosses
the thresholds.
NFSv4 Ops >= N/A N/A Issued when NFSv4 Ops (KB/S) crosses the
thresholds.
NFSv4 Ops by Share >= N/A N/A Issued when NFSv4 Ops by Share Ops (KB/S)
crosses the thresholds.
NFSv4 Ops Latency >= N/A N/A Issued when NFSv4 Ops Latency (KB/S) crosses
the thresholds.
23
Oracle Enterprise Manager System Monitoring Plug-in for Oracle ZFS Storage Appliance Installation Guide, v1.4.1
METRIC/COLUMNS COMPARISON
OPERATOR
WARNING CRITICAL PURPOSE
Physical Components Status/Cluster
States
= AKCS_
OWNER
AKCS_
STRIPPED
State of the cluster. OWNER means this head is
the owner of the cluster and that resources are
likely in contention. STRIPPED means that the
head's resources are owned by its peer and must
have the takeover redone.
Problem Log Contents/Description Matches N/A N/A Issued when the Problem Description Matches (via
Regular Expression) the thresholds.
Problem Log Contents/Problem Log
Code
Matches N/A N/A Issued when the Problem Log Code Matches (via
Regular Expression) the thresholds.
Problem Log Contents/Problem
Severity
Matches N/A Major|
Critical
Issued when the Problem Severity Matches (via
Regular Expression) the thresholds.
Problem Type can be Minor/Major/Critical.
Response/Status = N/A Down Provides information to the Home Page report on
availability of the system's management path.
Replication Activity/Replication
Description
Matches N/A Replication
of.*
Issued when the Replication Description Matches
(via Regular Expression) the thresholds.
SMB Ops >= N/A N/A Issued when SMB Ops (Operation per Second)
crosses the thresholds.
Stateless Alerts
Most of the alerts that are set in the preceding table are stateful alerts. They will be automatically cleared once the underlying conditions are cleared. F or example, if a threshold of 20 is set for SMB Ops, an alert will be triggered once the value crosses 20, and cleared once the value drops below 20.
However, there are a few special cases where the alerts cannot be cleared automatically. In these scenarios, these alerts are defined as stateless and you must manually clear them. All alerts from the following metrics are stateless alerts:
Alert Log
Fault Log
24
Oracle Enterprise Manager System Monitoring Plug-in for Oracle ZFS Storage Appliance Installation Guide, v1.4.1
Problem Log
Replication
To clear stateless alerts, follow these steps:
1. Click on the Alert either from the Homepage or Incident Manager.
2. Click on the Clear button.
3. Click on Yes to permanently clear the alert.
Oracle ZFS Storage Appliances Alerts
If real-time monitoring for performance is required, the storage administrator should create appliance alerts and thresholds on the Oracle ZFS Storage Appliances user interfaces. This allows the storage administrator to view performance issues using real-time indicators, as opposed to the slower polling interval required by Oracle Enterprise Manager Cloud Control.
Communicating Performance Issues to the Appliance Administrator
The most common use of the Oracle Enterprise Manager Cloud Control Plug-in (besides simple capacity monitoring and high-level information collection) is analysis of application performance degradation relating to a database. Using the information within Oracle Enterprise Manager Cloud Control, you can provide to the Oracle ZFS Storage Appliance administrator information related to performance degradation for an application on the appliance.
An Oracle ZFS Storage Appliance administrator has the advantage of real-time appliance analytics that can be viewed in various time spans (including initial configuration), monitored by Oracle Enterprise Manager Cloud Control. Some datasets are available from the first power-on of the appliance.
When you present performance information to the storage administrator, you can start the analysis process by using the Oracle Enterprise Manager worksheet. This worksheet displays a down to the second view of the data compared to the less useful view of the data available to Oracle Enterprise Manager Cloud Control (often a 10- or 20-minute
25
Oracle Enterprise Manager System Monitoring Plug-in for Oracle ZFS Storage Appliance Installation Guide, v1.4.1
collection cycle averaged over the period, resulting in a much less dramatic performance curve).
Using the worksheet and/or other datasets available to the storage administrator, you can analyze the cause down to the hardware or software component level. In the event of a resource contention issue, you can study levels of client access to determine how and when individual clients are accessing a system, along with the resources they are accessing. These can include network interfaces, logical resources (such as shares), and more.
When initial communication is complete, the storage administrator should complete the analysis of the problem and propose a way to resolve a disruption in quality of service. If the problem is not likely associated with the Oracle ZFS Storage Appliance, the storage administrator can provide this information.
Troubleshooting Monitoring Deployments
A variety of reasons could cause a monitored instance in the Oracle Enterprise Manager Cloud Control to return information improperly. These issues are typically associated with misconfiguration, incorrect permissions, or changing the configuration within one of the products.
26
Oracle Enterprise Manager System Monitoring Plug-in for Oracle ZFS Storage Appliance Installation Guide, v1.4.1
TABLE 4: APPLIANCE PROBLEMS AND SOLUTIONS
PROBLEM SOLUTION
Appliance does not appear to be Up
after being added as an instance to
be monitored.
If the appliance previously reported as Up but is now indicating it is Down, this is an indication
that the management path is no longer available on the target Oracle ZFS Storage Appliance.
The data path might still be available on the appliance, but this critical problem must be
addressed.
If this is the first time an appliance was added and it has never reported as being Up, an error
could have been made when setting up the instance for monitoring. Check the following:
Is the designated DNS name or IP address correct in the asset's Monitoring
Configuration?
Is the port number correct? The Oracle ZFS Storage Appliance default port is 215.
Is the named asset accessible from the Agent system?
Note: The Up/Down indicator does not require authentication to the target Oracle ZFS Storage Appliance, so the user ID and password are not relevant here.
Checking the value of the Response Metric should show 0. This indicates that the Agent
properly invoked the metric collection program, but the metric collection program could not
connect to the target Oracle ZFS Storage Appliance given the parameters in the Monitoring
Configuration.
After determining 0 is shown in the field, ensure the appliance is available at the configured IP
address in the monitoring configuration. However, if the value of Response Metric is a 1, the
appliance is likely available. In this case, refresh the Cloud Control page reporting that the
appliance is down. It will likely return to "Up".
Appliance is Up, but all metrics are
not being collected.
Check the metric collection errors. (See Table 4: Common Metric Collection Errors.) The most
common problem is misconfiguration of the password for the oracle_agent user within the Oracle Enterprise Manager Cloud Control environment.
For most metric collection errors, a summary message and Java Stack Trace is generated
and viewable by clicking the Metric Collection Errors link under the Home Page reports, then
clicking through the text Exception. The initial summary message will contain information
about the problem.
27
Oracle Enterprise Manager System Monitoring Plug-in for Oracle ZFS Storage Appliance Installation Guide, v1.4.1
PROBLEM SOLUTION
Appliance is Up, but some metrics
are not being collected.
Determine what metrics are not being collected. Most likely, it is a group of metrics such as:
ARC Activity by Share
L2ARC Activity
L2ARC Activity by Share
NFS V3 Operations by Share
NFS V4 Operations by Share
There are usually three reasons for selective metrics not being collected:
A component does not exist on a system This is common with the L2ARC metrics. In
this case, you will not receive a metrics collection error; rather, the Last Upload time on the
All Metrics page will remain empty (or include a -).
The Configure for Oracle Enterprise Manager Monitoring workflow was not run on
the target Oracle ZFS Storage Appliance. This problem embodies in metrics that
are not collected by default on a target appliance, such as those listed above. In
this case, Metric Collection Errors will occur within the Oracle Enterprise Manager
Cloud Control until the target Oracle ZFS Storage Appliance is correctly
configured (See Table 4: Common Metric Collection Errors.)
A storage administrator either disabled or removed datasets on the target system.
Ask the storage administrator to re-enable the dataset name that is in error. See
Metric Information in Table 1: Metric and Collection Information.
The list of shares in the Appliance
Logical Configuration and the
Appliance Logical Status appear to
be out of sync.
The layout of filesystems and their values are collected at different intervals than the capacity
information for them. In the worst case, a metric could be as much as 60 minutes out of sync
with a storage appliance. If you have a need for real-time information, refer to the Oracle ZFS
Storage Appliance BUI.
A Report or Metric Detail is not
showing a change made on the
system.
Metrics and information about system values are collected in varying intervals. In the worst
case, a metric could be as much as 60 minutes out of sync with a storage appliance. If you
have a need for real-time information, refer to the Oracle ZFS Storage Appliance BUI.
Unexpected Workflow Errors in
running the Configuration Workflow
on the Oracle ZFS Storage
Appliance, such as aksh: uncaught
error exception (code EAKSH_
GENERIC_DATASET_
INVALID)
Be sure you are running the configuration script as a user with root access (or equivalent)
with respect to creating and manipulating datasets and users.
28
Oracle Enterprise Manager System Monitoring Plug-in for Oracle ZFS Storage Appliance Installation Guide, v1.4.1
PROBLEM SOLUTION
Database mapping reports do not
show database artifacts that are
mapped to NFS v3 or v4 shares on
an Oracle ZFS Storage Appliance.
The host, database, or both are not being monitored by Cloud Control. Install the proper
agents on the host to ensure that storage attached to the system is being monitored.
Refer to the Oracle Enterprise Manager Cloud Control documentation for information on
monitoring host and database assets. See the Oracle Enterprise Manager Cloud Control
documentation listed in the Additional References section at the end of this document .
Database mapping reports do not
show database artifacts that are
mapped to iSCSI or Fibre Channel
shares on an Oracle ZFS Storage
Appliance.
iSCSI and Fibre Channel mappings are not supported in this release.
Metric Collection Errors
Some common configuration problems manifest as Metric Collection Errors. In most cases, these error messages contain two parts a short string followed by a longer Java Stack Trace. The first string usually contains enough information to isolate the problem. The Java Stack Trace is used by the Support Center (if the problem is complex). Common Metric Collection problems are as follows:
TABLE 5: COMMON METRIC COLLECTION ERRORS
METRIC COLLECTION ERROR PROBLEM / SOLUTION
login denied Problem: The user ID/password combination on the target Oracle ZFS Storage Appliance is incorrect.
Possible causes:
The Monitoring Configuration in Oracle Enterprise Manager Cloud Control is
using the incorrect user ID/password. The user ID should be oracle_agent and the password should be the one provided by the storage administrator.
The target Oracle ZFS Storage Appliance does not have the proper user ID and
role.
Solution:
Change the monitoring configuration to use the proper user ID/password provided by the
storage administrator, or reconfigure the target Oracle ZFS Storage Appliance to use the
correct oracle_agent password.
29
Oracle Enterprise Manager System Monitoring Plug-in for Oracle ZFS Storage Appliance Installation Guide, v1.4.1
METRIC COLLECTION ERROR PROBLEM / SOLUTION
workflow parameters are invalid Problem: The parameters passed to the Oracle ZFS Storage Appliance are invalid.
Possible causes:
A dataset on the Oracle ZFS Storage Appliance is not enabled.
Resources from a monitored cluster head failed to the secondary head, for which
the storage administrator did not run the Configuration workflow.
An internal error occurred and the Agent is calling an incompatible workflow.
Solution:
Run the Configuration workflow on the target Oracle ZFS Storage Appliance and
recreate the worksheet.
Ask the storage administrator to ensure that all of the datasets in the created
worksheet are being collected properly.
(dataset name) is not enabled on the
filer. Please enable the dataset on the
filer or disable the collection of this
metric.
Problem: A dataset cannot be found on the Oracle ZFS Storage Appliance.
Possible causes:
A dataset on the Oracle ZFS Storage Appliance is not enabled.
Solution:
Ask the storage administrator to enable the dataset on the Oracle ZFS Storage
Appliance.
Disable the collection of this dataset metric with the following steps:
1. Navigate to Metric and Collection Settings of the target.
2. Change the Drop-down View option from All Metrics.
3. Look for the metric name you wish to disable.
4. Click on the Collection Interval (such as Every 30 minutes).
5. Click the Disable button to disable collection of this metric.
30
Oracle Enterprise Manager System Monitoring Plug-in for Oracle ZFS Storage Appliance Installation Guide, v1.4.1
METRIC COLLECTION ERROR PROBLEM / SOLUTION
cannot find specified workflow Problem: A workflow that collects and returns metrics to the Agent cannot be found on the Oracle ZFS Storage Appliance.
Possible causes:
The oracle_agent user (or user that you configured in the Monitoring Configuration) does not have workflow.*.* read authorization.
An Oracle ZFS Storage Appliance is being used without the workflow on it, either
because the version of the Oracle ZFS Storage Appliance software is incorrect,
or the workflow was removed by a storage administrator.
Solution:
1. Run the Unconfiguration workflow on the target Oracle ZFS Storage Appliance
and remove the oracle_agent user.2. Run the Configuration workflow on the target Oracle ZFS Storage Appliance and
recreate the oracle_agent user.3. Ask the storage administrator to ensure the Configure/Unconfigure workflows are
on the system.
4. Contact the Support Center if the Agent fails to start collecting the dataset again.
workflow failed Problem: The Agent is accessing an Oracle ZFS Storage Appliance, but the workflow is failing in some way.
Possible causes:
A problem in the metric collection logic is causing it to fail on the Oracle ZFS
Storage Appliance.
Solution:
Run the Configuration workflow on the Oracle ZFS Storage Appliance and recreate the
worksheet.
If you are still receiving this error after recreating the worksheet, contact the Support Center.
Oracle ZFS Storage Appliance Audit Logs
For detailed debug information, use the audit log on the target Oracle ZFS Storage Appliance to determine which actions occurred with the workflows. Creation and deletion of artifacts (or the failure to create or delete an artifact) are traced in the audit log.
Note: Debugging the reason that an Agent is not properly collecting statistics from an Oracle ZFS Storage Appliance can be more difficult and should only be done by service personnel. Contact the Oracle Support Center (see Additional Resources at the end of this document).
31
Oracle Enterprise Manager System Monitoring Plug-in for Oracle ZFS Storage Appliance Installation Guide, v1.4.1
Host Agent Logs
Most problems (other than version mismatches) occur during the collection of metrics or response information. Look for the following logs on the Agent host. The Agent host may differ from the management station location. In the following file locations, %AGENT_LOCATION% indicates the home directory of the Agent, typically similar to /export/home/oracle/OracleHomes/agent10g. An asterisk (*) indicates there are several files with additional extensions.
TABLE 6: HOST AGENT LOGS
Agent Log Location Description
%AGENT_HOME%/sysman/emd/oracle_zfs_plugin/logs/{target_ip}/sun_storage_7000_pipe.log*
Contains errors that occurred below the Oracle Enterprise
Manager Cloud Control framework during attempts to
communicate with the Oracle ZFS Storage Appliance. These files
will typically contain fine-grained details on connection problems
with an Oracle ZFS Storage Appliance or failures that occur while
retrieving information.
%AGENT_HOME%/sysman/log/emagent.trc* These logs also contain connection exceptions as well as any information on using the data returned from the system to
populate database tables.
%AGENT_HOME%/sysman/log/* Remaining logs will contain finer-grained details on various elements in the Cloud Control Framework and 90% of the issues
can be diagnosed with the above.
Note: When interacting with the Support Center or developers within your own help desk, include the sun_storage_7000_pipe.log* files and emagent.trc* files for reference.
32
Oracle Enterprise Manager System Monitoring Plug-in for Oracle ZFS Storage Appliance Installation Guide, v1.4.1
Additional Resources
Oracle Support Centerhttp://www.oracle.com/support
Patches and updates downloads from My Oracle Support (MOS)https://support.oracle.com/CSP/ui/flash.html#tab=PatchHomePage%28page=PatchHomePage&id=g1ws8maj%28%29%29
Oracle Unified Storage Systems Documentationhttp://www.oracle.com/technetwork/documentation/oracle-unified-ss-193371.html
Oracle Enterprise Manager Concepts Guidehttp://docs.oracle.com/cd/B10501_01/em.920/a96674/toc.htm
Management of Heterogeneous Data Centers Using Oracle Enterprise Manager Plug-ins, An Oracle White Paperhttp://www.oracle.com/technology/products/oem/pdf/dc_mgmt.pdf
Oracle Enterprise Manager Cloud Control 12c documentation, including Oracle Enterprise Manager Concepts and Oracle Enterprise Manager Administration Guidehttp://docs.oracle.com/cd/E24628_01/index.htm
Oracle Enterprise Manager Cloud Control Upgrade Guidehttp://download.oracle.com/docs/cd/E24628_01/upgrade.121/e22625.pdf
Oracle Enterprise Manager, Cloud Control Extensibility Programmers Reference,12c Release3 (12.1.0.3)http://docs.oracle.com/cd/E24628_01/doc.121/e25161/title.htm
33
Oracle Enterprise Manager System Monitoring Plug-in for Oracle ZFS Storage Appliance, Installation GuideNovember 2013, Version 1.4.1Oracle Application Integration Engineering
Oracle CorporationWorld Headquarters500 Oracle ParkwayRedwood Shores, CA 94065U.S.A.
Worldwide Inquiries:Phone: +1.650.506.7000Fax: +1.650.506.7200
oracle.com
Copyright 2013, Oracle and/or its affiliates. All rights reserved. This document is provided for information purposes only and the contents hereof are subject to change without notice. This document is not warranted to be error-free, nor subject to any other warranties or conditions, whether expressed orally or implied in law, including implied warranties and conditions of merchantability or fitness for a particular purpose. We specifically disclaim any liability with respect to this document and no contractual obligations are formed either directly or indirectly by this document. This document may not be reproduced or transmitted in any form or by any means, electronic or mechanical, for any purpose, without our prior written permission.
Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective owners.
Intel and Intel Xeon are trademarks or registered trademarks of Intel Corporation. All SPARC trademarks are used under license and are trademarks or registered trademarks of SPARC International, Inc. AMD, Opteron, the AMD logo, and the AMD Opteron logo are trademarks or registered trademarks of Advanced Micro Devices. UNIX is a registered trademark licensed through X/Open Company, Ltd. 0611
Configuring the Oracle ZFS Storage ApplianceConfiguring for Clustered AppliancesConfiguring Database-to-Appliance Mappings Deploying the Plug-inAdding Instances for MonitoringAdding Clustered Systems for MonitoringViewing Metrics and ReportsViewing MetricsAccessing Reports
Upgrading the Plug-inUn-deploying the Plug-inMetric Collections on DatasetsMetric Collections on Pools, Projects and SharesStateless AlertsOracle ZFS Storage Appliances AlertsMetric Collection ErrorsOracle ZFS Storage Appliance Audit LogsHost Agent Logs