Smart Connector Users Guide

SmartConnector™ User’s Guide

Topics Applicable to All ArcSight™ SmartConnectors

May 15, 2011

Topics Applicable to All ArcSight™ SmartConnectors

Copyright © 2001-2011 ArcSight, Inc. All rights reserved.

ArcSight, the ArcSight logo, ArcSight TRM, ArcSight NCM, ArcSight Enterprise Security Alliance, ArcSight Enterprise Security Alliance logo, ArcSight Interactive Discovery, ArcSight Pattern Discovery, ArcSight Logger, FlexConnector, SmartConnector, SmartStorage and CounterACT are trademarks of ArcSight, Inc. All other brands, products and company names used herein may be trademarks of their respective owners.

Follow this link to see a complete statement of ArcSight's copyrights, trademarks, and acknowledgements: http://www.arcsight.com/company/copyright/

The network information used in the examples in this document (including IP addresses and hostnames) is for illustration purposes only.

This document is ArcSight Confidential.

Revision History

Document template version: 2.1

ArcSight Customer Support

Date Description

05/15/2011 Added new CEF encrypted Syslog destination and expanded CEF chapter.

02/15/2011 Added features to CEF Syslog destination and updated installation screens and procedures, added information on deleting file connectors and improved readability of FAQ appendix.

11/30/2010 Added feedback from reorganization review, corrected and closed various open tickets.

02/24/2010 Added chapter 10, defining the new CEF destination.

09/30/2009 Complete restructure of the guide and the addition of the “Configuring SmartConnectors” chapter.

08/24/2009 Added Model Connector information, new FAQ entries, and updated configuration fields.

Phone 1-866-535-3285 (North America) +44 (0)870 141 7487 (EMEA)

E-mail [email protected]

Support Web Site http://www.arcsight.com/supportportal/

Protect 724 Community https://protect724.arcsight.com

http://www.arcsight.com/company/copyright/

http://www.google.com

mailto:[email protected]

http://www.arcsight.com/supportportal/

https://protect724.arcsight.com

Contents

About This Book ....................................................................................................................................... 7

Who Should Read This Book ............................................................................................. 7

Related Documentation .................................................................................................... 8

ArcSight Customer Support .............................................................................................. 9

Chapter 1: Introduction to ArcSight Products ................................................................... 11

Arcsight Components ..................................................................................................... 12

ArcSight ESM .......................................................................................................... 12

ESM Manager ................................................................................................... 12

ESM Database .................................................................................................. 12

ESM Console .................................................................................................... 12

ArcSight Web ................................................................................................... 12

ArcSight SmartConnectors ....................................................................................... 13

Supported Data Sources .................................................................................... 13

Event Severity .................................................................................................. 14

ArcSight FlexConnectors .......................................................................................... 14

Arcsight Connector Appliance .................................................................................... 15

ArcSight Logger ...................................................................................................... 15

ArcSight Network Synergy Platforms (NSP) ................................................................. 15

Chapter 2: SmartConnector Overview ............................................................................... 17

Features ...................................................................................................................... 18

Data Collection Methods ................................................................................................. 20

Mapping to Vendor Events .............................................................................................. 20

Filter and Aggregate Events ............................................................................................ 20

SmartConnector Types ................................................................................................... 21

File Connectors ....................................................................................................... 22

Deleting Log Files After Processing ...................................................................... 22

Database Connectors ............................................................................................... 23

Scanner Connectors ................................................................................................ 24

API Connectors ....................................................................................................... 25

SNMP Connectors .................................................................................................... 25

Microsoft Windows Event Log Connectors ................................................................... 26

Syslog Connectors ................................................................................................... 27

ArcSight Confidential SmartConnector User’s Guide 3

FlexConnectors ....................................................................................................... 27

Model Import Connectors ......................................................................................... 28

Other Connectors .................................................................................................... 28

Chapter 3: Planning for Deployment ................................................................................. 29

Overview ..................................................................................................................... 29

Supported Platforms ...................................................................................................... 30

Deployment Scenarios ................................................................................................... 30

Deployment Scenario One ........................................................................................ 30

Deployment Scenario Two ........................................................................................ 31

Deployment Scenario Three ...................................................................................... 32

Estimating Storage Requirements .................................................................................... 32

Understanding ArcSight Turbo Modes ............................................................................... 33

Chapter 4: Installing SmartConnectors ............................................................................. 35

Installing ArcSight ESM .................................................................................................. 35

Installing the SmartConnector ........................................................................................ 36

Installing SmartConnectors from the Command Line .................................................... 45

Installing SmartConnectors in Silent Mode .................................................................. 45

Upgrading SmartConnectors ........................................................................................... 48

The Upgrade Process ............................................................................................... 48

Upgrade Notes ................................................................................................. 49

Locally Upgrading SmartConnectors .................................................................... 49

Remotely Upgrading SmartConnectors ................................................................ 49

Rolling Back to a Previous Version ............................................................................. 50

Running SmartConnectors .............................................................................................. 50

Standalone ............................................................................................................ 50

As a Windows Service .............................................................................................. 51

As a UNIX Daemon .................................................................................................. 51

Uninstalling a SmartConnector ........................................................................................ 52

Entering Table Parameter Values During Installation .......................................................... 52

Manually Entering Table Parameter Values ................................................................. 53

Manually Entering Parameter Values .......................................................................... 53

Importing and Exporting CSV Files ............................................................................ 53

Chapter 5: Configuring SmartConnectors .......................................................................... 55

Modifying SmartConnector Settings after Installation ......................................................... 55

Changing Connector Parameter Values ....................................................................... 56

Changing Connector Service Settings ......................................................................... 61

Configuring the Connector to Run as a Service ...................................................... 61

Removing a SmartConnector Service ................................................................... 63

Adding a Destination ............................................................................................... 64

Changing Filter Settings through the Wizard ............................................................... 66

4 SmartConnector User’s Guide ArcSight Confidential

Batching .......................................................................................................... 68

Time Correction ................................................................................................ 69

Device Time Auto-Correction .............................................................................. 70

Time Checking .................................................................................................. 71

Cache ............................................................................................................. 72

Network .......................................................................................................... 73

Field Based Aggregation .................................................................................... 77

Filter Aggregation ............................................................................................. 78

Processing ....................................................................................................... 79

Payload Sampling (when available) ..................................................................... 82

Filters ............................................................................................................. 83

Requesting Payload Information ...................................................................................... 86

Working with Payload Data ....................................................................................... 87

Lowering Network Bandwidth Used by the Connector ......................................................... 87

Chapter 6: SmartConnector Destinations .......................................................................... 89

SmartConnector Event Destinations ................................................................................. 89

Additional Destinations ............................................................................................ 90

Configuring Multiple Destinations ..................................................................................... 91

Failover Destinations ..................................................................................................... 93

Adding a Failover Destination .................................................................................... 94

Re-Registering a SmartConnector .................................................................................... 97

Chapter 7: Using SmartConnectors with Connector Appliance ......................................... 101

Managing SmartConnectors on the Connector Appliance ....................................................103

Local (on-board) SmartConnectors ...........................................................................103

Remote Connector Appliance SmartConnectors ..........................................................103

Software-Based SmartConnectors ............................................................................103

Choosing a Deployment Scenario ...................................................................................104

ArcSight Logger .....................................................................................................104

ArcSight ESM .........................................................................................................104

ESM and Logger .....................................................................................................104

Chapter 8: Using SmartConnectors with ArcSight Logger .............................................................................................................. 105

Sending Events from Logger to an ESM Manger ................................................................105

Logger and SmartMessage ......................................................................................106

Sending Events to Logger ..............................................................................................107

Sending Events to Both Logger and an ESM Manager ........................................................108

Forwarding Events from ArcSight ESM to Logger ...............................................................111

Defining SmartConnector Settings in Logger ..............................................................113


Chapter 9: Using SmartConnectors with NSP .................................................................. 117

Overview ....................................................................................................................117

Deploying a Syslog SmartConnector with NSP ..................................................................118

Configuring the Syslog SmartConnectors .........................................................................120

Chapter 10: CEF Destinations .......................................................................................... 121

CEF Syslog ..................................................................................................................121

CEF Encrypted Syslog (UDP) ..........................................................................................123

CEF File ......................................................................................................................124

Installation ............................................................................................................125

File Rotation ..........................................................................................................125

Chapter 11: CSV File Transport Destination .................................................................... 127

Overview ....................................................................................................................127

Installation ..................................................................................................................128

Event Data Rotation .....................................................................................................128

Appendix A: ArcSight Update Packs (AUPs) .................................................................... 131

Defining an AUP ...........................................................................................................131

ArcSight Content AUPs .................................................................................................131

ArcSight ESM ..................................................................................................132

ESM/Logger ....................................................................................................132

Logger ...........................................................................................................132

Connector Appliance .........................................................................................132

ArcSight Connector Upgrade AUP ...................................................................................133

ArcSight ESM ........................................................................................................133

Connector Appliance ...............................................................................................133

ESM Generated AUPs ....................................................................................................134

User Categorization Updates ....................................................................................134

System Zones Updates ...........................................................................................134

User Zones Updates ...............................................................................................134

Appendix B: SmartConnector Frequently Asked Questions ............................................................................................................. 135


About This Book

This book contains information that applies to all SmartConnectors, including installation, deployment, and management of SmartConnectors. Information about installing and configuring individual SmartConnectors is provided in the ArcSight SmartConnector Configuration Guides.

The following topics are discussed in this chapter:

Who Should Read This BookThe audience for this book is primarily security administrators who install SmartConnectors and ensure their connectivity to ArcSight products. This can include administrators for:

Networks

Security

Systems

Databases

If this is the first time you are installing an ArcSight component, ArcSight recommends reading the latest Administrator’s Guide for that component.

“Who Should Read This Book” on page 7

“Related Documentation” on page 8

“ArcSight Customer Support” on page 9


About This Book

Related Documentation ArcSight makes available the following ESM and SmartConnector product documentation. Many of these documents are available for download from the ArcSight ESM Console by choosing the menu option Help > Browse Documentation. The latest and most complete set of documentation is always offered on the ArcSight Customer Support site (https://support.arcsight.com) through the Product Documentation link in the Knowledge Center section.

Document Title Description

ArcSight™ SmartConnector Configuration Guides

Provides vendor-specific instructions for how to install individual SmartConnectors and configure their associated devices.

ArcSight FlexConnector Developer’s Guide

Describes how to design, create, and install custom SmartConnectors. This guide also provides details on how to create additional data mappings.

ESM 101: Concepts for ArcSight™ ESM

ESM 101 introduces the underlying concepts behind how ArcSight ESM works, and provides a roadmap to the tools available in ESM depending on your role in security operations.

ArcSight™ SmartConnector Release Notes

Describes new product features, latest updates, known product issues and work-arounds, and technical support information.

ArcSight™ ESM Installation and Configuration Guide

Explains how to install and configure ArcSight Enterprise Security Management (ESM) components and tools including the ArcSight Database, Manager, Console, and Web applications. Also provides general information about how to plan for, install, and deploy ArcSight SmartConnectors.

ArcSight™ ESM Administrator's Guide

Describes how to configure ArcSight and its network interfaces, and maintain ArcSight for ongoing operations.

ArcSight Logger™ Administrator's Guide

Describes planning, installation, initialization, configuration, and operation of the Logger appliance.

ArcSight™ Connector Appliance Administrator's Guide

Describes planning for, installing, initializing, configuring, and operating the Connector Appliance.

ArcSight™ NSP Installation and Administration Guide

Describes the use and features of Network Synergy Platform (NSP).


https://support.arcsight.com

About This Book

ArcSight Customer SupportYou can obtain a log-in user name and password from your ArcSight Customer Support representative. You can reach ArcSight Customer Support through the following resources:

Resource Description

Support Website http://www.arcsight.com/supportportal/ provides access to ArcSight incident reporting, knowledge base, software downloads, help, and the new Customer Forum.

Protect 724 Community






About This Book


Chapter 1

Introduction to ArcSight Products

ArcSight products comprise several separately installable components working together to process event data from your network. These components are connected throughout your network by way of sensors that report to a series of ArcSight SmartConnectors.

SmartConnectors translate device output into a normalized event schema that becomes the starting point for ArcSight ESM correlation. The following graphic illustrates ArcSight basic components. For complete descriptions of these components, see ESM 101, Concepts for ArcSight ESM v4.0.

Figure 1-1 ArcSight Products

“ArcSight ESM” on page 12

“ArcSight SmartConnectors” on page 13

“ArcSight FlexConnectors” on page 14

“Arcsight Connector Appliance” on page 15

“ArcSight Logger” on page 15

“ArcSight Network Synergy Platforms (NSP)” on page 15


1 Introduction to ArcSight Products

Users interact with ArcSight ESM using the ESM Console or ArcSight Web.

ArcSight SmartConnectors gather and process event data from network devices and pass it to the ESM Manager to be processed and stored in the database.

ArcSight Connector Appliance is a hardware solution incorporating any number of onboard SmartConnectors and a web-based user interface. This tool provides centralized management for SmartConnectors across a number of hosts.

ArcSight NSP uses NCM/TRM software to provide network device inventory, configuration settings, and additional analysis features.

ArcSight Logger is a hardware storage solution optimized for extremely high event throughput.

Arcsight ComponentsArcSight ESM

ArcSight ESM consists of several separately installable components that work together to process event data from your network. These components are described in the following pages.

ESM ManagerAs events stream into the system, the ESM Manager writes them to the ArcSight database. It simultaneously processes the events through the correlation engine, which evaluates each event with network model and vulnerability information to develop real time threat summaries.

ESM DatabaseAs events stream into the ESM Manager from the SmartConnectors, they are written to the ESM Database with a normalized schema. This lets ESM collect all events generated by the devices on your network, which you can analyze and refer to at any time.

The ESM Database is based upon Oracle 10g. A typical installation retains active data online from weeks to months.

ESM ConsoleThe ArcSight ESM Console is a workstation-based interface intended for use by your full-time security staff in a Security Operations Center (SOC) or similar security-monitoring environment. The Console is the authoring tool for building ArcSight ESM filters, rules, reports, Pattern Discovery, dashboards, and data monitors. It also is the interface for administering users and resources.

ArcSight Web ArcSight Web is an independent and remotely installable Web server that provides a secure interface with the ArcSight ESM Manager for browser clients. ArcSight Web is intended for use as a streamlined interface for customers of Managed Service Security Providers (MSSPs), SOC operators, and business users who require access to ArcSight ESM to investigate events from outside the protected network.

The ArcSight ESM Console version should match the ArcSight ESM Manager version to ensure that resources and schemas match.



ArcSight SmartConnectors SmartConnectors are the interface between the ArcSight ESM Manager and the network devices that generate ESM-relevant data on your network.

SmartConnectors collect event data from network devices, then normalize it in two ways. First, they normalize values (such as severity, priority, and time zone) into a common format. Then they normalize the data structure into a common schema. SmartConnectors can filter and aggregate the events to reduce the volume sent to the ESM Manager, which increases ArcSight’s efficiency and reduces event processing time.

In brief, SmartConnectors:

Collect all the data you need from a source device, eliminating the need to return to the device during an investigation or audit.

Parse individual events and normalize event values (such as severity, priority and time zone) into a common schema (format) for use by ArcSight ESM.

Filter out data you know is not needed for analysis, thus saving network bandwidth and storage space.

Aggregate events to reduce the quantity of events sent to the Manager, increasing Arcsight’s efficiency and reducing event processing time.

Pass processed events to the Manager.

Categorize events using a common, human-readable format, saving you time and making it easier to use those event categories to build filters, rules, reports, and data monitors.

Depending upon the network device, some SmartConnectors can also instruct the device to issue commands to devices. These actions can be executed manually or through automated actions from rules and some data monitors.

ArcSight releases new and updated SmartConnectors approximately twice a quarter.

Supported Data SourcesArcSight collects output from data sources with network devices, such as intrusion detection and prevention systems, vulnerability assessment tools, firewalls, anti-virus and anti-spam tools, encryption tools, application audit logs, and physical security logs.

SmartConnectors can be installed either directly on devices or separately on dedicated servers, depending upon the network device reporting to them. The SmartConnector can be co-hosted on the device if the device is a standard PC and its function is entirely software-based, such as IBM/ISS RealSecure devices, Snort devices, and so on. For embedded data sources (such as most Cisco devices and Check Point Firewall appliances), co-hosting on the device is not an option.

During SmartConnector configuration, a SmartConnector is registered to your ArcSight ESM Manager and configured with characteristics unique to the devices it reports on and the business needs of your network.

By default, SmartConnectors maintain a heartbeat with the ESM Manager every 10 seconds. The Manager sends any Console commands or configuration updates to the SmartConnector. The SmartConnector sends new event data to the Manager in batches of 100 events or once every second, whichever comes first. You can configure the time and event count intervals.



Event SeverityDuring the normalization process, the SmartConnector collects data about the level of danger associated with a particular event, as interpreted by the data source that reported the event to the SmartConnector. These data points, device severity and SmartConnector severity, become factors in calculating the event’s overall priority.

Device severity captures the language used by the data source to describe its interpretation of the danger posed by a particular event. For example, if a network IDS detects a DHCP packet that does not contain enough data to conform to the DHCP format, the device flags this as a high-priority exploit.

SmartConnector severity is the translation of the device severity into ArcSight-normalized values. For example, Snort uses a device severity scale of 1-10, whereas Check Point uses a scale of high, medium, and low. ArcSight normalizes these values into a single severity scale. The default ArcSight scale is Low, Medium, High, and Very High.

For example, routine file access and successful authentications by authorized users would be translated into the ArcSight-normalized values as very low severity, whereas a short DHCP packet would be translated as very high severity.

ArcSight FlexConnectorsArcSight’s FlexConnector framework is a software development kit (SDK) that lets you create a SmartConnector tailored to the devices on your network and their specific event data. The following ArcSight FlexConnectors types are available:

CounterACT

File

ID-Based Database

Key-Value File

Multiple Database

Multi-Folder File

Regular Expression File

Regular Expression Folder File

Regular Expression Multiple File

SNMP

Syslog

Time-Based Database

XML File

In addition, beta support is currently available for the following FlexConnectors:

Scanner Database

Scanner XML Reports

Scanner Text Reports

For complete information about these FlexConnectors and how to use them, contact your ArcSight Customer Support representative or see the ArcSight FlexConnector Developer's Guide.



Arcsight Connector ApplianceArcSight Connector Appliance is a hardware solution that incorporates a number of onboard ArcSight SmartConnectors and a web-based user interface that provides centralized management for SmartConnectors across a potentially large number of hosts.

The Connector Appliance centralizes SmartConnector management and offers unified control of SmartConnectors on local and remote Connector Appliances as well as software-based SmartConnectors installed on remote hosts.

ArcSight Connector Appliance includes on-board SmartConnectors that connect event sources to destinations such as ArcSight Logger and ArcSight ESM.

The Connector Appliance delivers the following features and benefits:

Supports bulk operations across all SmartConnectors and is particularly desirable in ArcSight ESM deployments with a large number of SmartConnectors, such as a Managed Security Services Provider (MSSP).

Provides an ArcSight ESM-like SmartConnector management facility in Logger-only environments.

Provides a single interface through which to configure, monitor, tune, and update SmartConnectors. The Connector Appliance does not receive events from the SmartConnectors it manages, and this allows for management of many connectors at one time. The Connector Appliance does not affect working SmartConnectors unless it is used to change their configuration. In some cases, the SmartConnector is commanded to restart.

ArcSight LoggerArcSight Logger is an event data storage appliance optimized for extremely high event throughput. Logger stores security events onboard in compressed form, but can always retrieve unmodified events on demand for forensics-quality litigation data.

Logger can be deployed stand-alone to receive events from syslog messages or log files, or to receive events in Common Event Format from SmartConnectors. Logger can forward selected events as syslog messages to ESM.

Multiple Loggers work together to support high sustained input rates. Event queries are distributed across a peer network of Loggers.

ArcSight Network Synergy Platforms (NSP)ArcSight NSP is an appliance that consists of these two licensed software components.

Network Configuration Manager (NCM)

Threat Response Manager (TRM)

These components build and maintain a detailed understanding of your network’s topology, letting you centrally manage your network infrastructure and rapidly respond to security incidents.



The NCM/TRM solution lets you:

Locate and quarantine any device connected to the network instantly

Apply protocol filters to curb an intrusion attempt

Block specific IP ranges from communicating or block specific protocols

Disable individual user accounts

Manage configuration changes centrally on a single device or a group of devices

Audit the change control process granularity

Build wizards that let you to delegate routine network administration tasks to lower-level administrators.


Chapter 2

SmartConnector Overview

This chapter provides an overview of ArcSight SmartConnectors and how they collect and send events (generated by various vendor devices) to the ArcSight ESM Manager.

The following topics are included in this chapter:

Once SmartConnectors normalize and send events to the ArcSight Manager, the events are stored in the centralized ESM Database. ArcSight ESM then filters and cross-correlates these events with rules to generate meta-events. The meta-events then are automatically sent to administrators with corresponding Knowledge Base articles that contain information supporting their enterprise’s policies and procedures.

SmartConnectors process raw data generated by various vendor devices throughout an enterprise. Devices consist of routers, e-mail servers, anti-virus products, firewalls, intrusion detection systems (IDS), access control servers, VPN systems, anti-DoS appliances, operating system logs, and other sources that detect and report security or audit information.

ArcSight SmartConnectors collect a vast amount of varying, heterogeneous information. Due to this variety of information, SmartConnectors format each event into a consistent, normalized ArcSight message, letting you find, sort, compare, and analyze all events using the same event fields.

Specific SmartConnector Configuration Guides document device-to-ArcSight ESM event mapping information for individual vendor devices, as well as specific installation parameters and configuration information.

“Features” on page 18

“Data Collection Methods” on page 20

“Mapping to Vendor Events” on page 20

“Filter and Aggregate Events” on page 20

“SmartConnector Types” on page 21


2 SmartConnector Overview

FeaturesFor complete information about how the following features work, see the ArcSight ESM v4.0 Administrator’s Guide and ArcSight ESM Console Help.

The following illustration shows the communication between network devices and ArcSight SmartConnectors, and between ArcSight SmartConnectors and ArcSight ESM Manager.

Feature Description

Filtering and Data Reduction

Uses AND/OR based Boolean logic to determine what data is to be included from the device and what data is filtered out when the event is sent to the ESM Manager.

Aggregation Compiles events with matching values into a single event, reducing the number of individual events the ESM Manager must evaluate.

Batching Improves ESM Manager performance by sending a collection of events at one time (rather than after each occurrence).

Time Error Correction

Synchronizes the time between the device and the SmartConnector, and between the SmartConnector and the ESM Manager.

Time Zone Correction Corrects the local time zone, as necessary, to support device-time queries, correlation, and filters.

Categorizer Assigns ArcSight ESM categories to an event.

Resolver Attempts to resolve and reverse-resolve host names and addresses reported by a device.

Data Normalization Converts each event produced by devices to an ArcSight ESM common event format message (or ArcSight message).



Figure 2-1 ArcSight SmartConnector Event Collection and Processing

SmartConnectors both receive and retrieve information from network devices. If the device sends information, the SmartConnector becomes a receiver; if the device does not send information, the SmartConnector retrieves it.

An ArcSight message is created for each event the SmartConnectors collect. Once an event is received, the SmartConnector adds device and event information to the event to complete the message, which is then sent to the ESM Manager.

You can deploy SmartConnectors on a device, on a separate host machine, or on the host machine where the ArcSight ESM Manager system resides.



Data Collection MethodsArcSight SmartConnectors are specifically developed to interoperate with network and security products using multiple techniques, including simple log forwarding and parsing, direct installation on native devices, SNMP, and syslog.

Data collection and event reporting formats for various SmartConnectors include:

Log File Readers (including text and log file)

Syslog

SNMP

Database

XML

Proprietary protocols, such as OPSEC or Cisco PostOffice

The ArcSight Console, Manager, and SmartConnectors communicate using HTTP (HyperText Transfer Protocol) over SSL (Secure Sockets Layer; also referred to as HTTPS).

Vendor device types for which SmartConnectors are available include:

Network and host-based IDS and IPS

VPN, Firewall, router, and switch devices

Vulnerability management and reporting systems

Access and identity management

Operating systems, Web servers, content delivery, log consolidators, and aggregators

For more information about the latest ArcSight SmartConnectors available, visit our website at http://www.arcsight.com and click the Support link.

Mapping to Vendor EventsArcSight SmartConnectors collect the vendor-specific event definitions contained within a network device. This information is mapped to the data fields within the SmartConnector, then sent to the ArcSight ESM Manager.

For specific mappings between the SmartConnector data fields and supported vendor-specific event definitions, see the configuration guide for the device-specific SmartConnector. For example, for mappings for the SmartConnector for Cisco PIX Syslog, refer to the SmartConnector for Cisco PIX Syslog Configuration Guide.

For additional information about mappings and parsing information from third-party devices, see “Advanced Topics” in the FlexConnector Developer’s Guide.

Filter and Aggregate EventsDuring SmartConnector installation and configuration, you can configure the SmartConnector to use filter conditions to focus the events passed to the ESM Manager according to specific criteria. For example, you can use filters to sort out events with certain characteristics, from specific network devices, or generated by vulnerability scanners. Events that do not meet the SmartConnector filtering criteria are not forwarded to the ESM Manager.



You can configure the SmartConnector to aggregate (summarize and merge) events that have the same values in a specified set of fields, either for a specified number of times or within a specified time limit.

SmartConnector aggregation compiles events with matching values into a single event. The aggregated event contains only the values the events have in common plus the earliest start time and latest end time. This reduces the number of individual events the Manager must evaluate.

For example, suppose the SmartConnector is configured to aggregate events with a certain Source IP and Port, Destination IP and Port, and Device Action whenever the events occur 10 times in 30 seconds. If ten events with these matching values are received by the SmartConnector within that timeframe, they are grouped together into a single event with an aggregated event count of 10.

If the 30-second timeframe expires and the SmartConnector has received only two matching events, the SmartConnector creates a single aggregated event with an aggregated event count of two. If 900 matching events were to come in during the 30 seconds, the SmartConnector would create 90 aggregated events, each with an aggregated event count of 10.

Firewalls are a good candidate for aggregation because of the volume of events with similar data coming in from multiple devices.

SmartConnector TypesSmartConnectors are the interface between the ArcSight ESM Manager and the network devices that generate ESM-relevant data on your network.

ArcSight SmartConnectors are generally one of the following types:

File Connectors

Database Connectors

Scanner Connectors

API Connectors

SNMP Connectors

Microsoft Windows Event Log Connectors

Syslog Connectors

FlexConnectors

Model Connectors

SmartConnectors collect event data from network devices, then normalize this data in two ways. First, they normalize values (such as severity, priority, and time zone) into a common format. They then normalize the data structure into a common schema. SmartConnectors can filter and aggregate events to reduce the volume sent to the ESM Manager, which increases ArcSight's efficiency and reduces event processing time.

For general information about ArcSight SmartConnectors, see Chapter 1‚ Introduction to ArcSight Products‚ on page 11.

For installation information and device-specific configuration and mapping information, see the SmartConnector Configuration Guide for the specific device.



File ConnectorsThere are two primary types of log file connector, Real Time and Folder Follower:

Real Time

These connectors can continue to follow a log file that retains its name or changes its name based upon the current date and other factors. The type of real time file connector is based upon the number of files monitored by the connector. There are connectors that monitor a single log file, such as the Snort File connector, and connectors that monitor multiple log files, such as the Cisco Secure ACS and SAP Real Time Audit connectors.

Real Time log file connectors can read normal log files in which lines are separated by a new line character as well as fixed length records in which a file consists of only one line but multiple records of fixed length (such as the SAP Real Time Audit connector).

Folder Follower

Folder follower connectors can follow files deposited into a single folder. There are connectors that monitor a single log file (such as HP-UX or IBM AIX) and connectors that monitor log files recursively (such as F-Secure AntiVirus).

.txt and .xml file types are supported by ArcSight SmartConnectors; which type depends upon the particular device. Text log files are the most common; however, Tripwire and most of the scanner file connectors, such as Nessus, nCircle, and NeXpose are in xml format.

The type of log file connector is not usually part of the connector name unless both types of connector exist for a particular device (such as SAP Audit and SAP Real-Time Audit).

File connectors are normally installed on the device machine, but when the monitored files are accessible through network shares or NFS mounts, the connectors can be installed on remote machines.

For some connectors, a trigger file is required to tell the connector when the file is complete and ready for processing. Typically, this is the same file name with a different extension. Files are renamed by default to increments such as .processed, .processed.1, and so on.

Generally, the only parameter required at installation is the location of the log file or files (the absolute path). When default file paths are known, they are displayed in the installation wizard.

Deleting Log Files After ProcessingIf you choose to delete your log files after the SmartConnector has processed them, you can access the connector's advanced parameters as follows:

1 From the $ARCSIGHT_HOME\current\bin directory in a DOS command window, enter: arcsight connectorsetup

To rename or delete log files, file folders require permissions for the connector.



2 When the following message is displayed, click No.

3 The Agent Configuration Tool window is displayed. From the Options menu, select Show Internal Parameters. The advanced configuration parameters for the connector are displayed as shown in the following figure.

4 To delete log files after processing, change the value for the mode parameter from RenameFileTheSameDirectory to DeleteFile.

5 Click OK.

6 Restart the connector for your change to take effect.

Database ConnectorsDatabase connectors use SQL queries to periodically poll for events. ArcSight SmartConnectors support major database types, including MS SQL, MS Access, MySQL, Oracle, DB2, Postgres, and Sybase.

In addition to the native JDBC driver for each database type, database connectors allow the use of a JDBC ODBC driver for databases that support them, such as MS SQL, Postgres, and MS Access. To use a JDBC ODBC driver, a JDBC ODBC data source is required. For instructions about creating this data source, see the configuration guide for your database connector.



During installation, the installation wizard asks, at a minimum, the following parameter values:

JDBC ODBC Driver

JDBC ODBC Data Source

Database User

Database Password

The database user must have adequate permission to access and read the database. For Audit database connectors, such as SQL Server Audit DB and Oracle Audit DB, system administrator permission is required.

In addition to connectors supporting event collection from a single database, some database connectors support multiple database events such as the Microsoft SQL Server Multiple DB connector. Others collect events from scanner databases, such as SmartConnectors for McAfee FoundScan DB and Mazu Profiler.

There are three major types of database connector:

Time-Based

Queries use a time field to retrieve events found since the most recent query time until the current time.

ID-Based

Queries use a numerically increasing ID field to retrieve events from the last checked ID until the maximum ID.

Job ID-Based

Queries use Job IDs that are not required to increase numerically. Processed Job IDs are filed in such a way that only new Job IDs are added. Unlike the other two types of database connector, Job IDs can run in either Interactive mode or Automatic mode.

Scanner ConnectorsThere are two types of scanner connector, those whose results are retained within a file, and those retrieved from a database. Results for the following types of scanner connectors are retained in a file, making them log file connectors:

XML files (such as Tenable Nessus, nCircle Audit, Qualys Scanner, and Rapid7 NeXpose)

Text files (such as Tenable Nessus NSR, NetRecon NRD)

Other scanners deposit their scanned events in a database and are treated as database connectors, requiring the same installation parameters as database connectors.

Scan reports are converted into base events, which can be viewed on the ESM Console, and aggregated meta events, which are not shown on the Console. Meta events create assets, asset categories, open ports, and vulnerabilities on the ESM Console.



Scanner SmartConnectors can run in either of two modes, automatic or interactive.

Interactive mode

Displays scan reports that can be individually selected to be sent to the connector. This mode is not supported for a connector running as a service.

Automatic mode

The connector checks periodically for any new reports deposited into the folder or any new jobs inserted into the database, then processes them. This mode is supported for both stand-alone applications and services.

Other than the operating mode, other parameter values required for scanner installation depends upon whether a file or database connector has been implemented. For file connectors, the absolute path to and name of the log file is required. For database connectors, see “Database Connectors” on page 23.

API ConnectorsAPI connectors use a standard or proprietary API to pull events from devices. In most cases, a certificate must be imported from the device to authenticate connector access to the device. There are also a number of configuration steps required on the device side. For example, Check Point devices require the configuration of connection type and the importing of a client certificate.

During installation, the following types of parameters are required, although each device's parameters are specific to its API:

Device IP

Service Port

Event types to be pulled

Certificate information

Information specific to the particular API

SNMP ConnectorsSNMP Traps contain variable bindings, each of which holds a different piece of information for the event. They are usually sent over UDP to port 162, although the port can be changed.

SNMP connectors listen on port 162 (or any other configured port) and process the received traps. They can process traps only from one device with a unique Enterprise OID, but can receive multiple trap types from this device.

As with syslog connectors (because SNMP is based upon UDP), there is a slight chance of events being lost over the network.

Parsers use the knowledge of the MIB to map the event fields, but, unlike some other SNMP-based applications, the connector itself does not require the MIB to be loaded.

No parameters are required during connector installation for SNMP devices.



Microsoft Windows Event Log ConnectorsSystem administrators use the Windows Event Log for troubleshooting errors. Each entry in the event log can have a severity of Error, Warning, Information, plus Success Audit or Failure Audit.

There are three default Windows Event Logs:

Application log (tracks events that occur in a registered application)

Security log (tracks security changes and possible breaches in security)

System log (tracks system events)

There are three SmartConnectors for Microsoft Windows Event Log:

SmartConnector for Microsoft Windows Event Log – Unified, this connector can connect to local or remote machines, inside a single domain or from multiple domains, to retrieve and process security and system events.

SmartConnector for Microsoft Windows Event Log – Local, which collects events from the Windows Event Log on your local machine.

SmartConnector for Microsoft Windows Event Log – Domain, which lets you collect Microsoft Windows Event Log events from multiple remote machines and forward them into the ArcSight system (such as multiple occurrences of the same application installed on different machines in one domain).

For details about the Local and Domain connectors deployment, installation, and configuration, see the SmartConnector Configuration Guide for Microsoft Windows Event Log. For mappings, see ArcSight SmartConnector Mappings to Windows Security Events.

For details about the Unified connector, see the SmartConnector Configuration Guide for Microsoft Windows Event Log – Unified. Mappings for this connector are incorporated into its configuration guide.

The SmartConnector for Microsoft Windows Event Log – Unified supports event collection from Microsoft Windows XP, Server 2000/2003/2008 and Vista platforms, as well as support for partial event parsing based upon the Windows event header for all System and Application events. Support for a FlexConnector-like framework that lets users create and deploy their own parsers for parsing the event description for all System and Application events is also provided.

Some individual Windows Event Log applications are supported by the SmartConnector for Microsoft Windows Event Log – Domain, for which Windows Event Log sub-connectors have been developed. These sub-connectors have individual configuration guides that provide setup information and mappings for the particular application. These sub-connectors include:

CA eTrust AntiVirus Windows Event Log

Microsoft Active Directory Service Windows Event Log

Microsoft WINS Windows Event Log

Oracle Audit Windows Event Log

RSA ACE Server Windows Event Log

Symantec Mail Security Windows Event Log



Syslog ConnectorsSyslog messages are free-form log messages prefixed with a syslog header consisting of a numerical code (facility + severity), timestamp, and host name. They can be installed as a syslog daemon, pipe, or file connector. Unlike file connectors, a syslog connector can receive and process events from multiple devices. There is a unique regular expression that identifies the device.

Syslog Daemon connectors listen for syslog messages on a configurable port, using port 514 as a default. It is the only syslog option supported for Windows platforms.

Syslog Pipe connectors require syslog configuration to send messages with a certain syslog facility and severity.

The Solaris platform tends to under perform when using Syslog Pipe connectors. The operating system requires that the connector (reader) open the connection to the pipe file before the syslog daemon (writer) writes the messages to it. When using Solaris and running the connector as a non-root user, using a Syslog Pipe connector is not recommended. It does not include permissions to send an HUP signal to the syslog daemon.

Syslog File connectors require syslog configuration to send messages with a certain syslog facility and severity. For high throughput connectors, Syslog File connectors perform better than Syslog Pipe connectors because of operating system buffer limitations on pipe transmissions.

UNIX supports all three types of syslog connector. If a syslog process is already running, you can end the process or run the connector on a different port.

Because UDP is not a reliable protocol, there is a slight chance of missing syslog messages over the network. TCP is now a supported protocol for syslog connectors.

There is a basic syslog connector, the SmartConnector for UNIX OS Syslog, which provides the base parser for all syslog sub-connectors.

For syslog connector deployment information, see the SmartConnector Configuration Guide for UNIX OS Syslog.

For device-specific configuration information and field mappings, see the SmartConnector Configuration Guide for the specific device. Each syslog sub-connector has its own configuration guide.

During connector installation, for all syslog connectors, choose Syslog Daemon, Syslog Pipe, or Syslog File from the installer selections rather than the name of the syslog sub-connector.

FlexConnectorsArcSight FlexConnectors let you to create custom connectors that can read and parse information from third-party devices and map that information to ArcSight’s event schema. When creating a custom connector, you define a set of properties (a configuration file) that identify the format of the log file or other source that is imported into the ArcSight ESM Manager or ArcSight Logger.

Use of FlexConnectors require the FlexConnector Developer’s Kit.



Model Import ConnectorsRather than collecting and forwarding events from devices, SmartConnectors for Identity Models import user data from an Identity Management system into ArcSight's IdentityView Solution. See the ArcSight Solution Guide IdentityView for information about how Identity Model Import connectors are used.

ArcSight SmartConnectors for Identity Models extract the user identity information from the database and populate the following lists in ArcSight's IdentityView Solution with the data:

Identity Roles Session List

Identity Information Session List

Account-to-Identity Map Active List

These lists are populated dynamically, which means that, as the identity data changes in the Identity Manager, the data in the lists is updated when you refresh the session list.

Identify Model connectors include:

SmartConnector for Microsoft Active Directory Identity Model

SmartConnector for Sun Identity Manager Model

SmartConnector for Oracle IDM Identity Model

Other ConnectorsSome connectors use multiple mechanisms. For example, the SmartConnector for Oracle Audit Database monitors both the database tables and audit files. Other examples of connectors with multiple mechanisms include:

Cisco NetFlow File

Retrieves data over TCP in a Cisco-defined binary format.

ArcSight Streaming Connector

Retrieves data over TCP from ArcSight Logger in an ArcSight-proprietary format.


Chapter 3

Planning for Deployment

Deployment of an ArcSight SmartConnector is based upon the requirements of your network security enterprise. This section outlines possible ArcSight deployments based upon different scenarios.


The scenarios and deployments shown here are only examples of how you might introduce ArcSight ESM into your enterprise. ArcSight ESM is not limited to just these scenarios and deployments.

OverviewArcSight components install consistently across UNIX, Windows, and Macintosh platforms. Whether a host is dedicated to the ArcSight ESM Database, Manager, Console, or other component, ArcSight ESM software is installed in a directory tree under a single root directory on each host (DBMS and other third-party software is not necessarily installed under this directory, however.) The path to this root directory is referred to as $ARCSIGHT_HOME.

In SmartConnector documentation, the 'current' directory is specified rather than presumed to be part of the $ARCSIGHT_HOME location, and the path separator is a backslash (\) (for example, $ARCSIGHT_HOME\current). This is consistent with SmartConnector configuration guide information, and also underscores the fact that ArcSight SmartConnectors are not installed on the same machine as the remaining ArcSight ESM components. Rather, they are typically installed on the same machine as the device whose activity will be monitored.

The directory structure below $ARCSIGHT_HOME is standardized across components and platforms. ArcSight software is generally available in the $ARCSIGHT_HOME\current\bin directory. Properties files, which control the ArcSight configuration, are found in $ARCSIGHT_HOME\config and log files are written to $ARCSIGHT_HOME\logs.

“Overview” on page 29

“Supported Platforms” on page 30

“Deployment Scenarios” on page 30

“Estimating Storage Requirements” on page 32

“Understanding ArcSight Turbo Modes” on page 33


3 Planning for Deployment

ArcSight SmartConnectors collect and process the data generated by various vendor devices throughout your enterprise. Devices consist of routers, email logs, anti-virus products, firewalls, intrusion prevention systems (IPS), access control servers, VPN systems, antiDoS appliances, operating system logs, and other sources where information about security threats are detected and reported.

ArcSight SmartConnectors collect a vast amount of varying, heterogeneous information. SmartConnectors format every raw security event into a consistent, normalized ArcSight event. By creating a consistent message format, you can find, sort, compare, and analyze all events using the same event fields.

When a SmartConnector receives an event, it completes the message by adding device information, then forwarding the event to various components throughout ArcSight ESM.

Supported PlatformsFor information about supported platforms, see the ArcSight SmartConnector Product and Platform Support document that is shipped with each SmartConnector release. Only differences to the support detailed in that document are specified in the device's SmartConnector Configuration Guide.

Deployment ScenariosYou can install SmartConnectors on the ArcSight ESM Manager, a host machine, or a device. Based upon configuration, connectors also can receive events over the network using SNMP, HTTP, syslog, proprietary protocols (such as OPSEC), or direct database connections to the device's repository (such as ODBC or proprietary database connections).

The best deployment scenario for your system depends upon the SmartConnector type, your network architecture, and your operating system.

Scenarios for syslog deployment are documented in the SmartConnector for UNIX OS Syslog Configuration Guide.

Scenarios for deploying Windows Event Log connectors are documented in the SmartConnector for Microsoft Windows Event Log Configuration Guide.

Deployment Scenario OneIn this scenario, there are three ArcSight SmartConnectors residing on three different devices: a firewall, an IPS, and a UNIX operating system. These connectors receive information from the devices or their logs and send captured events to the ESM Manager based upon the connector configuration.

Once events are received by the Manager, it cross-correlates the events using rules, and sends meta-events to the ESM Database and to any ESM Consoles that access the database.



The ESM Manager also can perform preset actions. Events and meta-events within the ESM Database can be played back using the Replay channel to investigate, analyze, or create a report about event history.

Figure 3-1 Three ArcSight SmartConnectors Residing on Three Devices

Deployment Scenario TwoThis scenario is the same as the first, except that the three SmartConnectors reside on a host machine rather than the device itself. The ArcSight SmartConnector need not reside on the device in order to retrieve information from that device. The SmartConnector functions as before, and the ArcSight ESM Manager and Database perform the same functions.

Figure 3-2 Three ArcSight SmartConnectors Residing on a Host Machine



Deployment Scenario ThreeIn this scenario, the ArcSight SmartConnectors reside on the ESM Manager itself, not on a host machine, but still retrieve events from devices in the network. The processing performed by the ArcSight SmartConnector, Manager, and Consoles are identical to the other scenarios.

Figure 3-3 Three ArcSight SmartConnectors Residing on an ESM Manager

Estimating Storage RequirementsUnderstanding the range of devices and SmartConnectors you want to deploy helps in estimating your daily event volume. Log file size is not accurate enough; you need to know how many events are generated during an average day. This varies by the type of device. Not only do different devices generate different event volumes, they also respond differently to various event aggregation policies.

The average size of the data stored for each event depends upon the turbo mode (Fastest, Faster, or Complete) specified for a particular SmartConnector. For detailed information on turbo modes, see the following section, “Understanding ArcSight Turbo Modes”.

SmartConnectors can aggregate events to reduce event traffic. An event that repeats every 500 ms, for example, can be represented by a single event that fires every ten seconds, producing a 20:1 event compression. Individual SmartConnectors can be configured to aggregate events in this manner, reducing event traffic to the ESM Manager and the storage requirements in the Database.

In a distributed environment with multiple ESM Managers, the event volume metric must consider both the SmartConnector feeds to the Manager and the event forwarding from other Managers.



Understanding ArcSight Turbo ModesYou can accelerate the transfer of sensor information through SmartConnectors by choosing one of three turbo modes (Fastest, Faster, or Complete).

The Fastest mode requires the fewest bytes and is most suited to devices such as firewalls, which have relatively little event data. The Faster mode is the Manager default, and requires less storage space. Rich event data sources, such as a network operating system, might use Complete mode, the SmartConnector default. The Complete mode passes all the data arriving from the device, including any custom or vendor-specific (for example, "additional") data.

You can configure SmartConnectors to send more or less event data on a per-SmartConnector basis, and the ESM Manager can be set to read and maintain more or less event data, independent of the SmartConnector setting.

Some events require more data than others. For example, operating system syslogs often capture a considerable amount of environmental data that may not be relevant to a particular security event. Firewalls, on the other hand, typically report only basic information.

ArcSight defines turbo modes as follows:

When a turbo mode is not specified, Mode 3, Complete, is the default. Versions of ArcSight ESM prior to v3.0 run in turbo mode Complete.

The ESM Manager uses its own turbo mode setting when processing event data. If a SmartConnector is set at a higher turbo mode than the Manager, it reports more event data than the Manager requires. The Manager ignores these extra fields.

However, if a Manager is set at a higher turbo mode than the SmartConnector, the SmartConnector has less event data to report to the Manager. The Manager maintains fields that remain empty of event data.

Both situations are normal in real-world scenarios because the Manager configuration must reflect the requirements of a diverse set of SmartConnectors.

Mode Description

Fastest (Mode 1) Recommended for simpler devices, such as firewalls.

Faster (Mode 2) ESM Manager default. Eliminates all but a core set of event attributes to achieve the best throughput. Because the event data is smaller, it requires less storage space and provides the best performance.

Complete (Mode 3) SmartConnector default. All event data arriving at the SmartConnector, including additional data, is maintained.




Chapter 4

Installing SmartConnectors

When you have purchased and are ready to install an ArcSight SmartConnector, see the individual connector’s configuration guide for information specific to the device the connector is monitoring. (For example, when installing a SmartConnector for Windows Event Log, see the SmartConnector Configuration Guide for Microsoft Windows Event Log.)

Individual configuration guides contain installation parameter values to enter, how to configure the particular device to enable SmartConnector event collection, and customized device event mappings to ArcSight ESM fields.


Installing ArcSight ESMBefore you install any ArcSight SmartConnectors, make sure that ArcSight ESM has already been installed correctly. Also, ArcSight recommends reading the ArcSight Installation and Configuration Guide before attempting to install a new ArcSight SmartConnector. For a successful installation of ArcSight ESM, follow this order:

1 Ensure that the ArcSight ESM Manager, Database, and Console are installed correctly.

2 Run the ArcSight ESM Manager. The command prompt window or terminal box displays a "Ready" message when the ESM Manager has started successfully. If the ArcSight ESM Manager is running as a Windows Service or Unix Daemon Service, monitor the server.std.log file located in $ARCSIGHT_HOME\current\logs\default.

3 Run the ArcSight ESM Console. Although not required, it is helpful to have the Console running when installing the SmartConnector to verify successful installation.

“Installing ArcSight ESM” on page 35

“Installing the SmartConnector” on page 36

“Upgrading SmartConnectors” on page 48

“Running SmartConnectors” on page 50

“Uninstalling a SmartConnector” on page 52

“Entering Table Parameter Values During Installation” on page 52


4 Installing SmartConnectors

Before installing the SmartConnector, be sure the following are available:

Local access to the machine where the SmartConnector is to be installed

Administrator privilege

Installing the SmartConnectorFor information regarding operating systems and platforms supported, see the SmartConnector Product and Platform Support document. For complete installation instructions for a particular SmartConnector, see the configuration guide for that connector. The product-specific configuration guide provides device configuration information, installation parameters, and device event mappings to ArcSight ESM fields.

1 Insert the ArcSight Installation CD into your CD-ROM drive or navigate to the location of the ArcSight SmartConnector Installer directory.

2 Start the ArcSight SmartConnector Installer by executing the file for your operating system. Installation files follow the format:

Verify that the ESM Database, Manager, and Console are installed and operating. When the Introduction window is displayed, read the information and click Next when ready.

At a minimum, SmartConnectors must be running version 4021 to communicate with a version 4.0 Manager.

Windows ArcSight-5.0.x.nnnn.y-Connector-Win.exe

Solaris ArcSight-5.0.x.nnnn.y-Connector-Solaris.bin

AIX ArcSight-5.0.x.nnnn.y-Connector-AIX.bin

Linux ArcSight-5.0.x.nnnn.y-Connector-Linux.bin



3 Next, accept the default location for "Where Would You Like to Install?," or click Choose… to select another folder for installation. Click Next when ready.

It is a good practice to develop and use a standard naming convention to specify directory locations, file names, and menu option names for the SmartConnectors you install. Typically, if you install multiple connectors on a particular machine, you should install each SmartConnector in a separate directory.

4 Choose from the following types installation; for most connectors, Typical is the appropriate selection. Click Next.

5 On the following window, accept the default shortcut folder location or select a new or existing Program Group. (Windows users can select the Create Icons for All Users



check box to create icons for all users accessing ArcSight SmartConnectors.) Click Next when you have finished making your selections.

6 Verify your selections on the Pre-Installation Summary window; click Install to begin installation of the SmartConnector core component software.

If the summary is incorrect, click Previous to make changes.

7 An installation process window is displayed during installation of core connector software (click Cancel if you want to cancel the installation).



8 When the installation of ArcSight SmartConnector core component software is finished, the following window is displayed:

9 Make sure ArcSight Manager (encrypted) is selected and click Next.

For information about the ArcSight Logger SmartMessage (encrypted) destination, see Chapter 8‚ Using SmartConnectors with ArcSight Logger‚ on page 105.

For information about NSP Device Poll Listener, see Chapter 9‚ Using SmartConnectors with NSP‚ on page 117.

10 The Wizard first prompts you for Manager certificate information. The default selection is No, the ArcSight Manager is not using a demo certificate. Choose Yes if ArcSight Manager is using a demo certificate. (Before selecting this option, make sure the Manager is, in fact, using a demo SSL certificate. If you are not certain, select No or consult your system administrator.). If your ArcSight Manager is using a self-signed or CA-signed SSL certificate, select No, the ArcSight Manager is not using a demo certificate and click Next.

11 On the next window, replace localhost with the host name of the Manager with which the SmartConnector is to communicate (localhost is appropriate only when the SmartConnector is installed on the same host as the Manager, which is not recommended in a production environment). This name must match the host name in

After completing the SmartConnector installation wizard, remember to manually configure the connector for the type of SSL certificate your Manager is using. See the ArcSight ESM Administrator's Guide for instructions about configuring your SmartConnector when the Manager is using a self-signed or CA-signed certificate and for instructions about enabling SSL client authentication on SmartConnectors so that the Connectors and the Manager authenticate each other before sending data.



the Manager’s certificate, which is usually the fully-qualified name. For example, instead of gabriel, specify gabriel.sales.mycompany.com.

For Manager Port, leave the default value of 8443.

For AUP Master Destination, generally leave this false. If, however, you will have one or more non-ESM destinations, and you want to share this ESM destination's AUP configuration (such as zones) with those destinations, select true. Only do so for one primary destination; if you select true for more than one primary destination or any failover destination, the setting is ignored for all but the first such primary destination.

For Filter Out All Events, select true if you want all events filtered out. This means the connector sends no events to this destination. This is useful when an ESM destination is added solely for the purpose of being the AUP master; this value is usually false unless the AUP Master Destination parameter is set to true.

12 Enter a valid ArcSight user name and password for the ArcSight ESM Manager. This is the same user name and password you created during ESM Manager installation.

13 Select one of the possible SmartConnectors from the window displayed. Scroll down to find the appropriate SmartConnector.



If you are installing a syslog SmartConnector, there are 3 different syslog types: the Syslog Pipe, Syslog File, or Syslog Daemon. For detailed information about syslog SmartConnectors refer to the SmartConnector Configuration Guide for your device.

The SmartConnectors that appear in the list are those that can be installed on the same platform from which you are running the installation program. For example, if you are running on Windows, the list contains a list of those SmartConnectors that are supported on Windows. Similarly, if you are running the installer on a Linux or Solaris-based system, the installer displays a list of SmartConnectors supported on those platforms.

14 After selecting the connector you want to install from the list of SmartConnectors, in this example, SAP Security Audit File, click Next.

15 The next window requests specific parameters for the particular SmartConnector you selected. These parameters vary depending upon the device and are described and explained in the SmartConnector Configuration Guide for the selected SmartConnector.

There are some SmartConnector types (such as Symantec Gateway Security/Enterprise Firewall NG, shown in the following example) that require parameter values to be entered into a table format. You can add this information



manually or import multiple hosts. See “Entering Table Parameter Values During Installation” on page 52 for detailed information.

To manually enter parameter values, click the Add button. See “Manually Entering Table Parameter Values” on page 53 for details.

To locate the .csv file you want to import, click the Import button. Click the Export button to create a .csv file containing the values you have entered in the parameter table. See “Importing and Exporting CSV Files” on page 53 for details.

16 Click Next when you have completed entering data.

17 Give your new SmartConnector a descriptive name to identify it for ArcSight Console users. You also can specify optional location information and add any appropriate comments.

If there are no Import and Export buttons on the parameter entry window for the connector you’ve selected, the parameters are not entered into a table format and this feature does not apply.



In this context, SmartConnector Location refers to the host where you are installing the SmartConnector as well as where within the resource tree this SmartConnector is listed on the ArcSight Console.

Device Location describes the host on which the IDS, syslog, or other software is running. If the device is physical hardware, the Device Location is particularly useful for specifying, for example, a certain position within a specific rack.

18 Click Next when you have finished entering data.

19 Review the summary of data and click Next.

If you choose to configure the SmartConnector to run as a service, the wizard prompts you for the service’s internal and display names.

Each SmartConnector name should be unique. If two similarly named connectors appear in the same SmartConnector Location, an error occurs.



20 Most SmartConnectors can be installed as a Windows service (or Linux/UNIX daemon) so that the SmartConnector runs automatically when the host is restarted. If the SmartConnector is not configured as a service, it must be started manually whenever it is not running. Select Yes or No and click Next.

21 If you choose not to run the SmartConnector as a service, a window such as the following is displayed.

22 Click Finish to complete connector configuration.

For some SmartConnectors, a system restart is required before the configuration settings you made can take effect. If a System Restart window is displayed, read the information and initiate the system restart operation.

Save any work on your computer or desktop and shut down any other running applications, including the ArcSight Console, if it is running; then shut down the system.



Installing SmartConnectors from the Command LineTo install ArcSight SmartConnectors without using the graphical user interface wizard, enter –i console on the command line when you invoke the self-extracting archive. Follow the instructions in the command window.

When the installation has successfully completed, manually run the configuration program by executing runagentsetup.

Installing SmartConnectors in Silent ModeYou can run the ArcSight SmartConnector installation program in silent mode, in which answers to wizard questions are provided by a Properties file. This feature is useful for deploying a large number of identical SmartConnectors.

To use this feature, first install and configure one SmartConnector using the graphical-user interface or the command line. While configuring the first SmartConnector, record its configuration parameters in a Properties file. To install all other SmartConnectors in silent mode, use the Properties file you created to provide configuration information.

To record the configuration of a SmartConnector to a Properties file:

1 Run the SmartConnector Configuration Wizard to extract and install the SmartConnector core files. When the wizard asks you for ESM Manager information, click Cancel.

2 From a command prompt window (from the ARCSIGHT_HOME\current\bin directory), enter the following command to launch the SmartConnector Configuration Wizard in record mode:

On Unix and Linux: ./runagentsetup.sh –i recorderui

On Windows: runagentsetup.bat -i recorderui

ArcSight recommends creating and testing the Properties file on a system other than your in-service, production environment. Recording from such a SmartConnector requires removal.



3 On the window displayed, enter the Silent Properties File Name to select an existing file. Enter the name of the Installation Target Folder to select a location.

4 Continue through all SmartConnector Configuration Wizard windows. The wizard creates a Properties file using the name and location you specified.

Perform the remaining steps on the system on which you want to install the SmartConnector in silent mode:

5 Copy the Properties file from the other system to your current system, preferably to the same directory where you downloaded the installation file.

6 Open the Properties file in an editor of your choice.

7 Find the USER_INSTALL_DIR property in the file and make sure that the path value is the absolute path to the location where you copied the Properties file on this system.

For example, if you copied the Properties file to C:\Program Files\ArcSightSmartConnectors, the path value should be as follows:

ARCSIGHT_AGENTSETUP_PROPERTIES=C\:\\Program Files\\ArcSightSmartSmartConnectors\\silent_properties

8 From the same file, repeat these steps for the ARCSIGHT_AGENTSETUP_PROPERTIES property.

The equal (=) and backslash (\) characters must be preceded by a backslash (\).



9 Find the AgentDetailsPanel.smartConnectorname property in the file and change its value to the name of the SmartConnector you are going to install in silent mode, as shown in the following example:

#====================================================== # Panel 'AgentDetailsPanel' #======================================================

# Select a name for your SmartConnector and specify location parameters.

#

# SmartConnector Name SmartConnectorDetailsPanel.smartConnectorname=SF_SmartConnector1

# Agent Location AgentDetailsPanel.agentlocation=San Francisco

# Device Location AgentDetailsPanel.devicelocation=Site_2.2.223

# Comment AgentDetailsPanel.comment=

#===============================================

10 If appropriate, edit the following properties:

AgentDetailsPanel.agentlocation AgentDetailsPanel.devicelocation AgentDetailsPanel.comment

You can edit any property (Manager Information, user credentials) in the Properties file to suit your needs.

11 Save the Properties file.

12 Download the SmartConnector installation file appropriate for your platform.

13 Run the following command to install the new SmartConnector in silent mode:

ArcSight_Agent_install_file -i silent –f properties_filename



The command launches the InstallShield program and installs the SmartConnector silently.

Example: To install a SmartConnector on Windows platform with the property file name silent_properties, enter:

ArcSight-3.5.x.nnnn.y-Agent-Win.exe –i silent –f silent_properties

Upgrading SmartConnectorsSmartConnectors occasionally may require upgrade. This process can be performed locally or remotely, although remote upgrades from the ESM Console are supported only on Windows, Linux, and Solaris platforms.

The Upgrade ProcessWhen you receive an e-mail notification about new SmartConnector releases from ArcSight Customer Support:

1 Download the latest releases to the ArcSight ESM Manager available for SmartConnector upgrades. Upgrade version files are delivered as .aup files (a compressed file set).

2 Copy the .aup file to ARCSIGHT_HOME\updates\ on a running ArcSight ESM Manager. The Manager automatically unzips the .aup file and copies its content to ARCSIGHT_HOME\repository\.)

3 From the ArcSight ESM Console, select SmartConnectors to be upgraded (one at a time) and launch the upgrade command for each of them.

4 Upon receipt of the upgrade command, the selected SmartConnectors upgrade themselves, restart, and send upgrade results (success or failure) back to the ArcSight ESM Console through the ESM Manager.

• After installing ArcSight SmartConnectors, configure your system’s default file permissions so that files created by ArcSight (events, log files, and so on) are reasonably secure.

• On UNIX systems, file permissions typically are set by adding the umask command to your shell profile. An umask setting of 077, for example, would deny read or write file access to any but the current user. An umask setting of 000 creates an unnecessary security hole.

• If you have installed multiple SmartConnectors in a single JVM, select the first connector installed in the JVM (if you select any other connector the upgrade fails) and launch the upgrade command; this action upgrades all connectors in the JVM.

• If your SmartConnector has multiple ESM Manger destinations, you must perform this process from the primary ESM Console. Any attempt to upgrade from a secondary or non-primary ESM Console destination will fail.



Upgrade Notes If the upgrade is successful, the new SmartConnector starts and reports successful

upgrade status.

If the upgraded SmartConnector fails to start, the original SmartConnector restarts automatically as a failover measure.

SmartConnectors automatically determine their upgrade status when they start.

When upgrading SmartConnectors, be sure to download current versions of the SmartConnector Configuration Guides from the ArcSight Customer Support website. These are the most current configuration guides available and contain information specific to the connector device.

Administrative permission is required to upgrade Connectors.

Versions of the Connectors you want to upgrade must be available on the Manager to which you are connected.

The option for remote upgrade is available only in ArcSight ESM v4.0 or later and only for version 4.0.2.xxxx.0 or newer SmartConnectors. Earlier versions of connectors (formerly known as SmartAgents) must be upgraded manually per the original process by installing a newer version of the SmartConnector.

As a prerequisite to upgrading Connectors, both the ArcSight ESM Manager and the SmartConnector you want to upgrade must be running.

Locally Upgrading SmartConnectorsTo locally upgrade a connector:

1 Stop the running connector and run the ArcSight SmartConnector installer. The installer prompts you for the location to install the connector.

2 Select the location of the SmartConnector that you want to upgrade. The message "Previous Version Found. Do you want to upgrade?" appears.

3 Select the option to continue and upgrade the connector. The original installation is renamed by prefacing characters to the original folder name; the upgraded connector is installed in the location $ARCSIGHT_HOME\current.

Remotely Upgrading SmartConnectors

ArcSight ESM not only provides the ability to centrally manage and configure SmartConnectors, but also to update them remotely. You can use the Upgrade command

• Should this happen, you can review the related logs. Choose Send Command -> Tech Support -> Get Upgrade Logs from the ArcSight Console menus.

• You can also use the Send Logs Wizard to collect and send logs, including upgrade logs, to ArcSight for support help.

Do not attempt to perform a remote upgrade on a secondary ESM destination using a 3.5 version. Using a secondary ESM destination prior to 4.0 causes the upgrade to fail.

Only Windows, Linux, and Solaris platforms are supported for SmartConnector remote upgrade from the ArcSight ESM Console.



on the ArcSight ESM Console to upgrade to newer versions of ArcSight SmartConnector software for managed devices. (You also can use the Rollback command to revert to a previous version of an upgraded SmartConnector.)

The Upgrade command lets you launch, manage, and review the status of upgrades for all SmartConnectors. A failover mechanism launches SmartConnectors with previous versions if upgrades fail. All communication and upgrade processes between components (Console, Manager, and SmartConnectors) take place over secure connections.

The ArcSight ESM Console reflects current version information for all of your ArcSight SmartConnectors.

Rolling Back to a Previous Version You can roll back an upgraded SmartConnector to the previous version with the Rollback command. See the ArcSight Console online Help for details on how to use the Rollback command.

The option for SmartConnector rollback is available only in ArcSight Console v4.0 and on previously upgraded SmartConnector versions 4.0.2.xxxx.0 or newer.

Rollback automatically reinstates the most recent version prior to the currently installed version. You cannot perform a remote rollback on a SmartConnector other than the previously installed version.

For example, if you start with a SmartConnector of version 4.0.2.4793, upgrade to 4.0.2.4794, then upgrade again to 4.0.2.4795, a remote rollback at this point re-installs/starts SmartConnector version 4.0.2.4794. You can only roll back to an earlier version manually.

Running SmartConnectorsSmartConnectors can be installed and run in standalone mode, as a Windows service, or as a UNIX daemon. If installed standalone, the SmartConnector must be started manually, and is not automatically active when a host is re-started. If installed as a Windows service or UNIX daemon, the SmartConnector runs automatically when the host is re-started.

Standalone To run all installed SmartConnectors on a particular host, open a command window, go to ARCSIGHT_HOME\current\bin and run:

arcsight connectors

To view the SmartConnector log, read the file:

Administrative permission is required to roll back SmartConnectors.

Some SmartConnectors require that you restart your system before configuration changes take effect.

SmartConnectors for scanners present a special case. To run a scanner SmartConnector in interactive mode, run in standalone and not as a Windows service or Linux/UNIX daemon.



$ARCSIGHT_HOME\current\logs\agent.log

To stop all SmartConnectors, enter Ctrl+C in the command window.

As a Windows ServiceSmartConnectors installed as a service can be started and stopped manually using platform-specific procedures.

To start or stop SmartConnectors installed as services on Windows platforms:

1 Right-click on My Computer, then select Manage from the Context menu.

2 Expand the Services and Applications folder and select Services.

3 Right-click on the ArcSight SmartConnector service name and select Start to begin running the SmartConnector or Stop to stop running the service.

To verify that a SmartConnector service has started, view the file:

$ARCSIGHT_HOME\logs\agent.out.wrapper.log

To reconfigure a SmartConnector as a service, run the SmartConnector Configuration Wizard again. Open a command window on $ARCSIGHT_HOME\current\bin and run:

runagentsetup

See Chapter 5‚ Changing Connector Service Settings‚ on page 61 for further details.

As a UNIX Daemon SmartConnectors installed as a daemon can be started and stopped manually using platform-specific procedures.

On UNIX systems, when you configure a SmartConnector to run automatically, ArcSight creates a control script in the /etc/init.d directory. To start or stop a particular SmartConnector, find the control script and run it with either a start or stop command parameter.

For example:

/etc/init.d/arc_serviceName {start|stop}

To verify that a SmartConnector service has started, view the file:

$ARCSIGHT_HOME/logs/agent.out.wrapper.log

To reconfigure SmartConnectors as a daemon, run the SmartConnector Configuration Wizard again. Open a command window on $ARCSIGHT_HOME/current/bin and enter:

runagentsetup

See “Changing Connector Service Settings” on page 61 for further details.

On Windows platforms, SmartConnectors also can be run using shortcuts and optional Start Menu entries.



Uninstalling a SmartConnectorBefore uninstalling a SmartConnector that is running as a service or daemon, first stop the service or daemon.

To uninstall on Windows, do the following

1 Open the Start menu.

2 Run the Uninstall SmartConnectors program found under All Programs -> ArcSight SmartConnectors.

3 If SmartConnectors were not installed on the Start menu, locate the $ARCSIGHT_HOME\current\UninstallerData folder and run:

Uninstall_ArcSightAgents.exe

To uninstall on UNIX hosts, do the following

1 Open a command window on the $ARCSIGHT_HOME/UninstallerData directory.

2 Run the command:./Uninstall_ArcSightAgents.

Entering Table Parameter Values During InstallationDuring SmartConnector installation, a connector using table parameters shows the following type of window for entering parameter data.

The parameters for this type of SmartConnector can be entered manually for a few lines of data, or, for a larger number of entries, you can import a .csv file. You can also create a

The UninstallerData directory contains a file .com.zerog.registry.xml with Read, Write, and Execute permissions for everyone. On Windows platforms, these permissions are required for the uninstaller to work. However, on UNIX platforms, you can change the permissions to Read and Write for everyone (that is, 666).

The Uninstaller does not remove all the files and directories under the ArcSight SmartConnector home folder. After completing the uninstall procedure, manually delete these folders.



.csv file by exporting data you’ve already entered. See “Importing and Exporting CSV Files” on page 53 for specific steps.

Manually Entering Table Parameter ValuesNote the following when using this feature:

Columns that contain private data (shown as asterisks), such as passwords, will not appear in exported files after using the Export button.

After importing a .csv file (using the Import button), data in private columns remain hidden (shown as asterisks).

While you can manually enter a private column (either by adding the column to your CSV within a spreadsheet program or by filling it in through the Configuration Wizard), it still will not appear in any exported files. This is a precautionary measure.

Importing data from a .csv file (using the Import button) causes all existing data in the table to be removed and replaced by the incoming data.

Manually Entering Parameter ValuesTo enter parameters manually, use the Add button to create fields and enter the data, as shown below.

If needed, use the Export button to export your parameter table data into an external .csv file to save for later use.

Importing and Exporting CSV FilesAn easy way to quickly populate many lines of parameter data is to create a .csv file, then use the Import button to fill the parameter entry table of the SmartConnector Configuration Wizard.

To use the Import feature:

1 Using a spreadsheet program (such as Microsoft Excel), enter the parameter data into a table and save it as a .csv file.



2 During SmartConnector installation, click the Import button to locate the .csv file you created. The window provides a preview of the CSV’s contents.

3 Click the Import button on the Import window. This populates the SmartConnector parameters fields, as shown below.

4 If you wish, you can add more rows manually (using the Add button) and then export the resulting table (using the Export button) to an external .csv file for later use.

5 If you are finished entering data, click Next.

The example above shows a “Password” column within the Configuration Wizard that does not appear in the original .csv file. This private column does not contain actual password data and will not be included in an exported file.


Chapter 5

Configuring SmartConnectors

This chapter contains configuration tasks you can perform without access to the ArcSight ESM Manager. The following topics are covered:

Modifying SmartConnector Settings after InstallationIf you want to modify any of the ArcSight SmartConnector parameters after installation, including configuring the connector to run as a service or standalone application, you can modify a destination without accessing an ArcSight ESM Manager using the SmartConnector Configuration Wizard.

After first installing a SmartConnector, you can run the wizard again if you want to modify destination settings. From $ARCSIGHT_HOME\current\bin, execute:

runconnectorsetup

The following window is displayed.

“Modifying SmartConnector Settings after Installation” on page 55

“Requesting Payload Information” on page 86

“Lowering Network Bandwidth Used by the Connector” on page 87


5 Configuring SmartConnectors

To make changes to the initial values set during connector installation and configuration, select "I want to change SmartConnector parameters".

To configure the connector to run as a service, or, if already configured to run as a service, to run as a stand-alone application, select "I want to change SmartConnector service settings".

To add a new destination, to configure multiple destinations, or to modify SmartConnector parameters without accessing an ArcSight ESM Manager, select "I want to add/remove/modify ArcSight Manager destinations".

If you did not intend to make any changes, select "I do not want to change any settings".

The remainder of this chapter describes the configuration changes you can make.

Changing Connector Parameter ValuesThis example walks through changing parameters for the SmartConnector for Microsoft Windows Event Log – Local.

1 First, select I want to change SmartConnector parameters.



2 Click Next. You are then asked whether the ArcSight ESM Manager is using a demo certificate. Select the appropriate response and click Next.

3 Enter the requested ArcSight Manager information on the next window.



4 Click Next to continue; enter your user name and password.

5 On the next window, make changes to the connector parameters as desired, then click Next. (This example shows the installation parameters for the Windows Event Log – Local connector.)



6 When you have made your changes, click Next.

7 To update the configuration, click Next, the configuration changes are applied and the following window is displayed.



8 Click Next; on the next window, you are given the opportunity to change your service settings. Select Yes or No as appropriate, then click Next.

This example shows no changes to be made to service settings; the following window is displayed. Click Finish to exit connector configuration.



Changing Connector Service SettingsThis section describes how to run a connector as a service, and how to remove a connector service.

Configuring the Connector to Run as a Service1 First, select I want to change SmartConnector service settings.

2 On the next window, select Yes, I want to configure the SmartConnector to run as a service and click Next.



3 Enter the service parameters and click Next.

4 The following window is displayed indicating the connector has been successfully configured to run as a service. Click Finish to exit the configuration wizard.



Removing a SmartConnector ServiceTo remove the SmartConnector service, after selecting "I want to change SmartConnector Service Settings”, select Yes, I want to remove the SmartConnector service then click Next.

The following window is displayed, indicating successful removal of the connector service; click Finish to leave the configuration wizard.



Adding a Destination1 Select I want to add/remove/modify ArcSight Manager destinations and click

Next.

2 To add a new destination, select Add new destination and click Next.



3 Select a destination type.

See the following chapters for information about configuring these destination types:

Chapter 6‚ SmartConnector Destinations‚ on page 89

Chapter 7‚ Using SmartConnectors with Connector Appliance‚ on page 101

Chapter 8‚ Using SmartConnectors with ArcSight Logger‚ on page 105

Chapter 9‚ Using SmartConnectors with NSP‚ on page 117

Chapter 10‚ CEF Destinations‚ on page 121

Chapter 11‚ CSV File Transport Destination‚ on page 127



Changing Filter Settings through the WizardTo change filter settings using the SmartConnector Wizard:

1 Select I want to add/remove/modify ArcSight Manager destinations and click Next.

2 Leave the default host selection; click Next.



3 Select Modify destination settings as the operation to be performed. Click Next.

The following destination settings selection window is displayed.



BatchingSmartConnectors can batch events to increase performance and optimize network bandwidth. When activated, connectors create blocks of events and send them when they either reach a certain size or the time window expires. You also can prioritize batches by severity, forcing the connector to send the highest severity event batches first and the lowest severity event batches later.

Enable Batching (per event)Creates batches of events of this specified size (5, 10, 20, 50, or 100 events).

Enable Batching (per sec)Sends the events if this time window expires (1, 5, 10, 15, 30, or 60 seconds).

Batch ByThe choices are Time Based and Severity Based. Select Time-Based for the connector to send batches as they arrive (the default); select Severity Based for the connector to send batches based upon severity (batches of highest severity events are sent first).



Time CorrectionThe following settings can be set for Time Correction.

Use Connector Time as Device TimeSelect Yes or No. No is the default value. This setting lets you override the time the device reports, using the time at which the SmartConnector received the event instead. This option assumes that the connector is more likely to report the correct time.

Enable Device Time Correction (in secs)The connector can adjust the time reported by the device Detect Time with this setting. This is useful when a remote device's clock is not synchronized with the ArcSight ESM Manager. This should be a temporary setting. The recommended method for synchronizing clocks between the ESM Manager and devices is through the NTP protocol.

Enable Connector Time Correction (in secs)The connector also can adjust the time reported by the Connector Time SmartConnector itself using this setting. This is for informational purposes only and does not modify the local time on the connector. This should be a temporary setting. The recommended method for synchronizing clocks between ESM Manager and connectors is through the NTP protocol.

Set Device TimeZone ToUsually the original device reports its time zone along with its time. If it does not, it is presumed the connector is doing so. If this is not true, or the device is not reporting correctly, you can switch this option from DIsabled to GMT or to a particular world time zone. That zone is applied to the reported time.



Device Time Auto-CorrectionThe values you set for these fields establish forward and backward time limits that, if exceeded, cause the connector to automatically correct the time reported by the device.

The following settings can be set for Device Time Auto-Correction.

Future ThresholdThe default value is -1. Set to a positive number to activate auto correction. The connector sends the internal alert if the detect time is later than the connector time by Future Threshold seconds.

Past ThresholdThe default value is -1. Set to a positive number to activate auto correction. The connector sends the internal alert if the detect time is earlier than the connector time by Past Threshold seconds.

Device ListEnter a comma-separated list of the devices to which the thresholds are to apply. The default is (ALL) for all devices.



Time CheckingThese are the time span and frequency factors for device time checking.

Future ThresholdThe number of seconds by which to extend the connector's forward threshold for time checking. The default is 300 seconds (5 minutes).

Past ThresholdThe number of seconds by which to extend the connector's rear threshold for time checking. The default is 3600 seconds (1 hour).

FrequencyThe connector checks its future and past thresholds at intervals specified by this number of seconds. The default frequency is 60 seconds (1 minute).



CacheChanging these settings does not affect the events already cached, only new events.

Cache SizeConnectors use a compressed disk cache to hold large volumes of events when the ArcSight ESM Manger is down or when the connector receives bursts of events. This parameter specifies the disk space to use. The default is 1 GB, which, depending upon the connector, can hold about 15 million events, but can also go down to 5 MB (5 MB, 50 MB, 100 MB, 200 MB, 250 MB, 500 MB, 1 GB, 2.5 GB, 5 GB, 10 GB, and 50 GB are the possible values). When this disk space is full, the connector drops the oldest events to free up disk cache space.

Notification ThresholdThe size of the cache's event content at which a trigger notification occurs. The default is 10000.

Notification FrequencySpecifies how often to send notifications once the Notification Threshold is reached. (1 min, 5 min, 10 min, 30 min, and 60 min are the possible values.)



NetworkThe following Network settings are can be modified.

Heartbeat FrequencyThis setting controls how often the connector sends a heartbeat message to the ESM Manager. The default is 10 sec, but can range from 5 seconds to 10 minutes. Note that the heartbeat also is used to communicate with the connector; therefore, if the frequency is set to 10 minutes, it could take as long as 10 minutes to send any configuration information or commands back to the connector.

Enable Name ResolutionSelect Yes, No or Source/Dest Only. The default value is Yes.

When choosing Yes (enabled), the connector attempts to resolve IP addresses to host names and host names to IP addresses if required and when the event rate allows.

Choosing Source/Dest Only causes the device name and IP address to be skipped, which is useful in environments where device IP addresses change frequently.

Name Resolution TTL (secs)Determines how long name resolution is to be in effect. The default value is 3600 seconds.

Wait For Name ResolutionSelect Yes for the connector to wait for name resolution. The default value is No.

Name Resolution Host Name OnlySelect Yes or No. The default value is Yes. When Yes is selected, for reverse resolution (IP Address to Host Name), only the host name field is set. When No is selected, the host name is split and put into both the DNS domain and the Host Name fields. This affects the source, destination, device, and connector name fields.

Name Resolution Domain From E-mailSelect Yes or No. The default value is Yes. When Yes is selected, if the host name and DNS domain fields are empty and the corresponding user name field appears as an email



address, the domain from the email address is put in the DNS domain field. This affects only the source and destination fields.

Clear Host Names Same as IP AddressesSelect Yes or No. The default value is Yes. When Yes is selected, when the host name field is set to an IP address that matches the corresponding IP address field, the host name field is cleared. This affects the source, destination, and device fields.

Don't Resolve Host Names MatchingEnter a regular expression as a value; resolution of host names matching the expression are skipped.

Don't Reverse-Resolve IP RangesEnter a range of IP addresses as a value; reverse resolution for all the IP addresses that fall in that range is skipped.

Remove Unresolvable Names/IPs From CacheSelect Yes, Yes (w/ negative cache), or No.

The default value is No (unresolvable names or IP addresses are not removed from the cache).

Choosing Yes removes unresolvable names or IP addresses from the cache.

Choosing, Yes (w/ negative cache) adds a separate cache to track unresolvable names and IP addresses.

Limit Bandwidth ToWhen Enabled, select from a list of the bandwidth options you can use to constrain the connector's output over the network (1 kbit/sec to 100 Mbits/sec). The default value is Disabled.

Transport ModeYou can configure the connector to cache to disk all the processed events it receives. You can use this setting to delay event-sending during particular time periods. For example, you could use this setting to cache events during the day and send them at night. You also can



set the connector to cache all events except those marked with a very high severity during business hours and send the rest at night. The selections are Normal, Cache, Cache (but send Very High severity events).

Address-Based Zone Population Defaults EnabledThis field applies to ESM version 3.0 ArcSight Managers and is not relevant for ESM v3.5 or 4.0. These versions have integral zone mapping.

Address-Based Zone PopulationThis field applies to ESM v3.0 ArcSight Managers and is not relevant for v3.5 or v4.0 (these versions have integral zone mapping). For v3.0 this table lets you define ranges of IP addresses to map to specific zones. Each row of the table defines a range to map to one zone. The system chooses the first matching range and ranges may overlap, so enter the smaller ranges first. Assuming the default zones are enabled, the zones entered here take precedence over the default zones, so the default zones are used only when none of these ranges match.

Zone Population ModeSelect Normal for zones to be computed and assigned if they are not already set; select Rezone (override) to re-compute and reassign if the zones are already populated; select No Zoning (clear) to clear the zones if they are populated. The default value is Normal.

Customer URIApplies the given customer URI to events emanating from the connector. If the customer resource exists, all customer fields are populated on the ESM Manager. If this particular connector is reporting data that might apply to more than one customer, you can use Velocity templates in this field to conditionally identify those customer.

Source Zone URIWhen populated, this field shows the URI of the zone associated with the connector's source address. This field exists for v3.0 ESM compatibility; it is not relevant in ESM v3.5 or v4.0 because of integral zone mapping.

Source Translated Zone URIWhen populated, this field shows the URI of the zone associated with the connector's translated source address. The translation is presumed to be NAT (network address



translation). This field exists for v3.0 ESM compatibility; it is not relevant in ESM v3.5 or v4.0 because of integral zone mapping.

Destination Zone URIWhen populated, this field shows the URI of the zone associated with the connector's destination address. This field exists for v3.0 ESM compatibility; it is not relevant in ESM v3.5 or v4.0 because of integral zone mapping.

Destination Translated Zone URIWhen populated, this field shows the URI of the zone associated with the connector's translated destination address. This field exists for v3.0 ESM compatibility; it is not relevant in ESM v3.5 or v4.0 because of integral zone mapping.

Agent Zone URIWhen populated, this field shows the URI of the zone associated with the connector's address. This field exists for v3.0 ESM compatibility; it is not relevant in ESM v3.5 or v4.0 because of integral zone mapping.

Agent Translated Zone URIWhen populated, this field shows the URI of the zone associated with the connector's translated address. This field exists for v3.0 ESM compatibility; it is not relevant in ESM v3.5 or v4.0 because of integral zone mapping.

Device Zone URIWhen populated, this field shows the URI of the zone associated with the device's address. This field exists for v3.0 ESM compatibility; it is not relevant in ESM v3.5 or v4.0 because of integral zone mapping.

Device Translated Zone URIWhen populated, this field shows the URI of the zone associated with the device's translated address. This field exists for v3.0 ESM compatibility; it is not relevant in ESM v3.5 or v4.0 because of integral zone mapping.



Field Based AggregationThis feature is an extension of basic connector aggregation. Basic aggregation aggregates two events when and only when the fields of the two events are the same per the fields listed in the description of "Enable Aggregation (in seconds)." However, field-based aggregation implements a more flexible aggregation mechanism; two events are aggregated when only the selected fields are the same for both events.

Field-based aggregation creates a new alert that contains only the fields that were specified, so the rest of the fields are ignored unless "Preserve Common Fields" is set to Yes. Field-based aggregation offers several advantages over basic aggregation, including:

Control over what fields are to be aggregated.

Start and end time set to the earliest start time and latest end time, respectively (rather than taking the values from the first event in the group, as with basic aggregation).

Option to preserve common fields.

Option to sum one or more numeric fields.

Connector aggregation significantly reduces the amount of data received, and should be applied only when you use less than the total amount of information the event offers. For example, you could enable field-based aggregation to aggregate "accepts" and "rejects" in a firewall, but you should use it only if you are interested in the count of these events instead of all the information provided by the firewall.

Time IntervalSelect a time interval, if applicable, to use as a basis for aggregating the events the connector collects. It is exclusive of Event Threshold. Possible values are DISABLED, 1 sec, 5 sec, … 60 min). The default value is DISABLED.

Event ThresholdSelect a number of events, if applicable, to use as a basis for aggregating the events the connector collects. This is the maximum count of events that can be grouped; for example, if 150 events were found to be the same within the time interval selected (contained the same selected fields) and you select an event threshold of 100, you will then receive two



events, one with a count 100 and another with a count of 50. This option is exclusive of Time Interval. Possible values are DISABLED, 10 events, 50 events, … 10000 events. The default value is DISABLED.

Field NamesSelect one or more fields, if applicable, to use as the basis for aggregating the events the connector collects. Use Ctrl + click to select multiple fields. The result is a comma-separated list of fields to monitor. You can use any of the event fields displayed in the wizard; the name can contain no spaces and the first letter should not be capitalized.

Fields to SumIf specified, this set of numeric fields is summed rather than aggregated, preserved, or discarded. The most common fields to sum are bytesIn and bytesOut. Note that if any of the fields listed here are also in the list of field names to aggregate, they are aggregated and not summed.

Preserve Common FieldsSelect Yes or No. The default value is No. Selecting Yes adds fields to the aggregated event if they have the same values for each event. Selecting No ignores non-aggregated fields in aggregated events.

Filter AggregationFilter aggregation is a way of capturing aggregated event data from events that would otherwise be discarded due to a connector filter. Only events that would be filtered out are considered for filter aggregation (unlike field-based aggregation, which looks at all events).

Time IntervalSelect a time interval, if applicable, to use as a basis for aggregating the events the connector collects. It is exclusive of Event Threshold. Possible values are DISABLED, 1 sec, 5 sec, …3600 sec. The default value is DISABLED.

Event ThresholdSelect a number of events, if applicable, to use as a basis for aggregating the events the connector collects. This is the maximum count of events that can be grouped; for example,



if 150 events were found to be the same within the time interval selected (contained the same selected fields) and you select an event threshold of 100, you will then receive two events, one with a count 100 and another with a count of 50. This option is exclusive of Time Interval. Possible values are DISABLED, 10 events, 50 events, … 10000 events. The default value is DISABLED.

Fields to SumIf specified, this set of numeric fields is summed rather than aggregated, preserved, or discarded. The most common fields to sum are bytesIn and bytesOut. Note that if any of the fields listed here are also in the list of field names to aggregate, they are aggregated and not summed.

ProcessingThe following settings for Processing can be modified.

Preserve Raw EventSelect Yes or No. The default value is No. Some devices contain a raw event that can be captured as part of the generated event. If that is not the case, most connectors can also produce a serialized version of the data stream that was parsed/processed to generate the ArcSight event. This feature lets the connector preserve this serialized raw event as a field. This feature is disabled by default since using raw data increases the event size and, therefore, requires more database storage space. You can enable this by changing the Preserve Raw Event setting to Yes. The serialized representation of the raw event is then sent to the ESM Manager and preserved in the Raw Event field.



Turbo ModeIf your configuration, reporting, and analytic usage permits, you can greatly accelerate the transfer of a sensor's event information through connectors by choosing one of three turbo (narrower data bandwidth) modes. Fastest is recommended for simpler devices such as firewalls. Faster eliminates all but a core set of event attributes in order to achieve the best throughput. Because the event data is smaller, it requires less storage space and provides the best performance. Faster is the default value for the ESM Manager. Complete is the SmartConnector default value. All event data arriving at the connector, including additional data, is maintained. When a turbo mode is not specified, Complete is used. In processing events, the ESM Manager's turbo mode trumps that of a connector's turbo mode.

Enable Aggregation (in secs)If you have already used this feature for setting up previous connectors, you can continue to do so. However, ArcSight recommends that you use the new Field Based Aggregation feature as a more flexible option. Selections are Disabled, 1, 2, 3, 4, 5, 10, 30, 60. The default value is Disabled.

Limit Event Processing RateYou can moderate the connector's burden on the processor by reducing its processing rate. This also can be a means of dealing with the effects of event bursts. The choices range from -1 (no limitation on processor demand) to 1 eps (pass just one event per second, making the smallest demand on the processor). Be sure to note that this option's effect varies with the category of connector in use.

Fields to ObfuscateUsing MD5 hashing, this option lets you specify a list of fields for obfuscation in a security event.

Store original time inLets you move the original device receipt time to a specified field if altered by the time correction.

Enable Port-Service MappingSelect Yes or No. No is the default value. If Yes is selected and one of the two fields destination port and application protocol is set and the other is not, the one that is set is used to set the other. For example, if the destination port is 22 and application protocol is not set, then the application protocol is set to ssh.

Uppercase User NamesSelect Disabled or Enabled. The default value is Disabled. If set to any of the enabled settings, the two user name fields are automatically changed to uppercase. Enabled (orig to ID) saves the original values to the sourceUserID and destinationUserID fields, respectively, overwriting any values that may have previously been there. Enabled (orig to ID or Flex) saves the original values in the same fields if they do not already contain values, or to the flexString1 (source) and flexString2 (destination) fields if the ID fields do contain values. Enabled (orig to Add.Data) saves the original values to additional data fields called OrigSrcUsrName and OrigDstUsrName, respectively. The uppercase operation is typically done using the default Locale for the chosen platform. You can set this to a particular Locale by setting the connector.uppercase.user.name.locale property in agent.properties to the desired Locale (using "en_US" for U.S. English, for example).



Enable User Name SplittingSelect Yes or No. The default value is No. If this is set to yes and the destination user name contains commas in the event, this parameter duplicates that event. Each user name in the list is placed in one of the events. For example, if the destination user name in an event is "User 123, User 456," then that event is sent twice, with the destination user name set to "User 123" in the first and "User 456" in the second.

Split File Name into Path and NameSelect Yes or No. The default value is No. If this is set to Yes and an event's file name field is set but its file path field is not, this parameter splits the file name into a path and a name, placing each part into appropriate fields. For example, if the file name field is set to C:\dir\file.ext and the file path is not set, the file path is set to C:\dir and the file name to file.ext. The separator character can be either \ or / as the system looks to the connector to determine the platform.

Event Integrity AlgorithmSelect DISABLED, SHA-256, SHA-1, MD5, or SHA-512. The default value is DISABLED. If this is set to one of the algorithms (such as SHA-256), and the Preserve Raw Event parameter is Enabled, then additional event integrity internal events are generated, normally at a rate of about 1 per 50 normal events.

The crypto signature field also is set in each event in the format: "#seq(alg) :digest", where seq is a persistent event sequence number, alg is the message digest algorithm, and digest is the hexadecimal message digest. These extra events and the crypto signature field values can be used to verify that no events were tampered with after generation.

Generate Unparsed EventsSelect Yes or No. The default value is No. If set to Yes and some incoming event data cannot be parsed (perhaps because a device has been upgraded since the connector parser was written), a special event named "unparsed event" is generated. The raw event appears in the event message field. If set to No, the connector log files indicate the unparsed events.

Preserve System Health EventsSelect Yes or No. The default value is No. If enabled, sends system health events periodically to ESM Manager. Events are named as a "Connector System Health Event" and then generated for several different types of statistics that are collected. Examples include disk usage, memory usage, and processor usage.

Enable Device Status Monitoring (in millisecs)The default value of -1 indicates device status monitoring is disabled. The minimum positive value is 1 min (60000 ms). When enabled, an internal event is sent named "Connector Device Status" for each device tracked by the connector containing the following types of information: the last timestamp when the connector received an event from the device, the total number of events from this device since the connector started, and the number of events sent by this device since the last event of this type.



Payload Sampling (when available)Payload sampling is used by some connectors to send a portion of packet payload (as opposed to the complete packet payload) along with the original event. This portion is retrieved using the on-demand payload retrieval.

See “Requesting Payload Information” on page 86 for information about working with payload data.

Max. LengthThis feature lets you configure the maximum length of the payload sample using the following values: Discard, 128 bytes, 256 bytes, 512 bytes, and 1 Kbyte. When the discard option is chosen, no payload sample is sent inside the original event.

Mask Non-Printable CharactersThis feature lets you mask the non-printable characters in the payload sample.



Filters1 Select I want to add/remove/modify ArcSight Manager destinations and click

Next.

2 Leave the default host selection; click Next.



3 Select Modify destination settings as the operation to be performed. Click Next.

4 Select Filters.

5 In the next window, enter the string that represents your setting modification for any of the settings displayed. Although the graphical modifiers used within the ArcSight ESM Console cannot be used here, you can write filtering strings such as:

Name EQ "Agent"

(name Contains "Super") Or (name EQ "Agent")

attackerAddress Between ("10.0.0.1", "10.0.0.10")



See the Usable Operators table following this procedure for a list of usable operators.

6 Click Next until done and exit the connector configuration wizard.

The following table lists usable operators. For more information about data fields, event mappings, and CEF fields, see the "Data Fields," "Audit Events," "Cases," and "Events" sections in the ArcSight ESM User's Reference.

Usable Operators Description

EQ equal to

NE not equal to

LT less than

LE less than or equal to

GE greater than or equal to

GT greater than

Between compares any specified range

ContainsBits equal to, for bitmap fields

In standard CCE operator for membership test

Contains contains the specified string

StartsWith starts with the specified string

EndsWith ends with the specified string

Like standard CCE operator for simple pattern matching for string type: _ wildcard for single character, % wildcard for any number of characters.

InSubnet for IP address that is not the specified subnet



Requesting Payload InformationPayload refers to the information carried in the body of an event's network packet, as distinct from the packet's header data. The on-demand payload feature is available on the ArcSight ESM Console. Click on any of the vulnerability events sent by the connector and you will see in the Event Inspector whether payload data is available. Click on the Payload tab and you can see additional information, including Description and Recommendation. For success events, you can see Description and Detail.

The following SmartConnectors currently provide payload support:

Barnyard (Snort IDS) File

Cisco Secure IPS SDEE

Enterasys Dragon IDS File

McAfee Vulnerability Manager DB (formerly FoundScan DB)

McAfee Network Security Manager DB

Snort DB

Snort Multiple File

Symantec ManHunt DB

Cisco Secure IDS RDEP

Enterasys Dragon Export Tool File

IBM SiteProtector DB

McAfee IntruShield DB (Legacy)

Snort File (Legacy)

Sourcefire Defense Center eStreamer

Extra information can be retrieved by using the on-demand payload feature on the ArcSight ESM Console. Click on any of the vulnerability events sent by the SmartConnector and you will see in the Event Inspector that Payload data is available; click on the Payload tab and you can see additional information including Description and Recommendation. For services events, you will receive Description and Detail.

You can retrieve, preserve, view, or discard payloads using the ArcSight Console. Because event payloads are relatively large, ArcSight does not store them by default. Instead, you can request payloads from devices for selected events through the Console. If the payload is still held on the device, the ArcSight SmartConnector retrieves it and sends it to the Console.

Payloads are downloaded and stored only on demand; you must configure ArcSight ESM to log these packets. By default, 256 bytes of payload are retrieved.

Whether an event has a payload to store is visible in event grids. Unless you specifically request to do so, only the event's "payload ID" (information required to retrieve the

InGroup for asset in the specified asset category or zone in the specified zone group

Is tests true for the selected state, null or not null

Usable Operators Description



payload from the event source) is stored. Payload retention periods are controlled by the configuration of each source device.

Working with Payload DataThe first step in handling event payloads is to be able to locate payload-bearing events among the general flow of events in a grid view. In an ArcSight Console Viewer panel grid view, right-click a column header and select Add Column <Device> Payload ID. Look for events showing a Payload ID in that column.

To retrieve payloads, in a Viewer panel grid view, double-click an event with an associated payload. In the Event Inspector, click the Payload tab, then click Retrieve Payload.

To preserve payloads, in a grid view, right-click an event with an associated payload, select Payload, then Preserve. Alternatively, in the Event Inspector, click the Payload tab, then the Preserve Payload icon.

To discard payloads, in a grid view, right-click an event with an associated payload, select Payload, then Discard Preserved. You also can use the Event Inspector: In a grid view, double-click an event with an associated payload. In the Event Inspector, click the Payload tab, then click the Discard Preserved Payload icon.

To save payloads to files, in a grid view, double-click an event with an associated payload. In the Event Inspector, click the Payload tab. Click the Save Payload icon. In the Save dialog box, navigate to a directory and enter a name in the File name text field. Click Save.

Lowering Network Bandwidth Used by the ConnectorArcSight SmartConnectors can send event information to the ESM Manager in a compressed format using HTTP compression. The compression technique used provides compression rates of 1 to 10 or greater, depending upon the input data (in this case, the events sent by the connector). Using compression lowers the overall network bandwidth used by connectors dramatically without impacting their overall performance.

By default, all SmartConnectors have compression enabled. To turn it off, add the following line to the agent.properties file (located at ARCSIGHT_HOME\current\user\agent\): http.transport.compressed = false




Chapter 6

SmartConnector Destinations

This chapter provides information about configuring a SmartConnector to send events to one or more destinations. A destination is an ArcSight ESM Manager or ArcSight device that can receive events from a particular SmartConnector. In addition to the five selections displayed during SmartConnector configuration explained below, events can be sent to additional or failover destinations.


SmartConnector Event DestinationsDuring SmartConnector installation, you are asked to select a destination for the events collected by the SmartConnector.

ArcSight Manager (encrypted)When SmartConnectors send events to an ArcSight ESM Manager, the Manager stores the events in a relational database, processes them using its correlation engine, and makes them visible to the ArcSight Console or ArcSight Web interfaces. See Chapter 5‚ Configuring SmartConnectors‚ on page 55 and the ArcSight Online Console Help for complete information.

ArcSight Logger SmartMessage (encrypted)SmartConnectors can send CEF events to ArcSight Logger using an encrypted, optionally compressed channel called SmartMessage. Logger also can receive CEF syslog events from SmartConnectors. For more information, see Chapter 7‚ Using SmartConnectors with Connector Appliance‚ on page 101."

NSP Device Poll ListenerThe Device Poll Listener detects when changes are made to network devices outside of Network Configuration Manager (NCM) / Threat Response Manager (TRM). The SmartConnector captures these changes by collecting syslog output from modified network devices and categorizes the events for ESM. The SmartConnector then initiates an action through NCM/TRM to poll the specific modified network devices to determine the precise changes made to the configuration. For more information, see Chapter 9‚ Using SmartConnectors with NSP‚ on page 117.

“SmartConnector Event Destinations” on page 89

“Configuring Multiple Destinations” on page 91

“Failover Destinations” on page 93

“Re-Registering a SmartConnector” on page 97


6 SmartConnector Destinations

CEF SyslogThis selection sends events in Common Event Format (CEF) (converted to bytes using the UTF-8 character encoding), and provides three protocol options: UDP, TCP, and TLS.

TCP and UDP can be used to send to ArcSight Logger (TLS cannot be used for this purpose). Data received using these protocols are received using a TCP or UDP Receiver. One such receiver can receive from more than one connector.

For more information, see Chapter 8‚ Using SmartConnectors with ArcSight Logger‚ on page 105.

The TLS protocol establishes a secure channel and allows for one-way or two-way authentication. If the TLS protocol is chosen, the events can be received by the SyslogNG Connector.

For more details about this destination, see Chapter 10‚ CEF Destinations‚ on page 121. For more details regarding the SyslogNG Connector, refer to the SmartConnector Configuration Guide for Syslog NG Daemon.

CEF Encrypted Syslog (UDP)This destination sends events in Common Event Format (CEF) through the UDP protocol, providing symmetric-key encryption. This option allows for a “Shared Secret” key that requires configuration to encrypt the data. This data can be decrypted on the receiver side by the CEF Encrypted Syslog (UDP) connector.

For more information on this destination, see Chapter 10‚ CEF Destinations‚ on page 121. For more details on how to decrypt the data, refer to the SmartConnector Configuration Guide for ArcSight CEF Encrypted Syslog (UDP).

CEF FileThis selection allows you to capture security events in a CEF file rather than forwarding them to an ArcSight ESM Manager.

For more detailed information, see Chapter 10‚ CEF Destinations‚ on page 121.

CSV FileThis selection lets you capture events a SmartConnector normally would send to the ArcSight ESM Manager into a CSV file. This is an advanced topic; typical ArCSight configurations do not require the use of external files to communicate events to the ArcSight ESM Manager. For more information, see Chapter 11‚ CSV File Transport Destination‚ on page 127.

Additional DestinationsArcSight SmartConnectors send a copy of events to each additional destination for which it is configured. Additional destinations can be useful, for example, when you have a development ArcSight environment working in parallel with your production environment and you want to test rules and reports.

In such cases, you can configure the SmartConnector to send alerts to both your production Manager and your development Manager to be able to view real-time event flows on both systems. Because the destinations are independent, you do not compromise the events sent to the production Manager.



Configuring Multiple DestinationsTo configure multiple destinations, use the ArcSight SmartConnector Configuration Wizard after installing the ArcSight SmartConnectors.

To start the wizard, execute the following command:

$ARCSIGHT_HOME\current\bin\runagentsetup

To add, remove, or modify a destination:

1 Select the option I want to add/remove/modify ArcSight Manager destinations and click Next.

In this example, the SmartConnector currently installed is ActiveCard AAA Server Accounting Log DB, but the message at the top of the window will be specific to the connector you previously installed.

2 You can either modify the existing destination or you can add a new destination. For this example, select Add new destination and click Next.



3 Select the destination type. For this example, select ArcSight Manager (encrypted) and click Next.

4 Click Add new destination to add a new SmartConnector destination and click Next.



5 Fill in the parameters for the destination you want to add and click Next to finish.

For information about the AUP Master Destination and Filter Out All Events fields, see “Installing the SmartConnector” on page 36.

6 To apply your changes, restart the SmartConnector.

Failover DestinationsEach SmartConnector destination can have a failover destination. A failover destination receives security events from the SmartConnector for which it is configured only when the primary destination (such as the primary ArCSight ESM Manager) is not available or when a network problem occurs. Once these events are backed up in the failover destination. the SmartConnector caches the events and resends them to the primary destination.

A failover destination is active only when the primary destination is unavailable, so the reports and replay features within the secondary Manager could contain incomplete information. This feature performs as a real-time alternative for severe problems with the primary ArCSight ESM Manager.

Failover only works with communication protocols that can detect transmission failure, such as TCP.



Adding a Failover DestinationTo add a failover destination:

1 Run the ArcSight SmartConnector Configuration Wizard and select the option I want to add/remove/modify ArcSight Manager destinations.

2 Select your current destination (Host), as shown below.



3 Select Add fail over destination and click Next.

4 Select a failover destination type. For this example, select ArcSight Manager (encrypted) to set up an alterative Manager in case the production Manager fails.



5 Enter the settings for the failover destination and click Next to continue to the next window.

6 To apply your changes, restart the SmartConnector.

For information about the AUP Master Destination and Filter Out All Events fields, see “Installing the SmartConnector” on page 36.



Re-Registering a SmartConnectorWhen the ArcSight Manager recognizes a SmartConnector, it generates an ID token the SmartConnector uses to identify its security events. If the Manager stops accepting events from a SmartConnector for an unknown reason, or if you have upgraded a SmartConnector but its resource was removed from the database, you may need to re-register the SmartConnector.

To re-register a SmartConnector:

1 Run the ArcSight SmartConnector Configuration Wizard and select the option I want to add/remove/modify ArcSight Manager destinations.

In the example above, the SmartConnector currently installed is "ActiveCard AAA Server Accounting Log DB," but you can use the same procedure for any SmartConnector.

2 Click Next.



3 Run the ArcSight SmartConnector Configuration Wizard and select your current (Host) destination. Click Next.

4 Select the Re-register… option:



5 Log in with a valid User Name on the ArcSight Manager where you are attempting to re-register the SmartConnector. Click Next.

6 Restart the SmartConnector to apply the new ID token.




Chapter 7

Using SmartConnectors with Connector Appliance

The following topics are covered in this chapter:

ArcSight Connector Appliance is a hardware solution that incorporates a number of onboard ArcSight SmartConnectors and a web-based user interface that provides centralized management for SmartConnectors across a potentially large number of hosts.

The Connector Appliance centralizes SmartConnector management and offers unified control of SmartConnectors on

The Connector Appliance

Remote Connector Appliances

Software-based SmartConnectors (installed on remote hosts)

Figure 7-1 ArcSight Connector Appliance includes on-board SmartConnectors that connect event sources to destinations such as ArcSight Logger and ArcSight ESM.

“Managing SmartConnectors on the Connector Appliance” on page 103

“Choosing a Deployment Scenario” on page 104


7 Using SmartConnectors with Connector Appliance

The benefits of Connector Appliance include:

Support of bulk operations across all SmartConnectors, which is particularly desirable in ArcSight ESM deployments with a large number of SmartConnectors, such as a Managed Security Services Provider (MSSP).

An ArcSight ESM-like SmartConnector management facility in Logger-only environments.

A single interface through which to configure, monitor, tune, and update SmartConnectors. The Connector Appliance does not receive events from the SmartConnectors it manages, and this allows for management of many connectors at one time. The Connector Appliance does not affect working SmartConnectors unless it is used to change their configuration. In some cases, the SmartConnector is commanded to restart.

Figure 7-2 Connector Appliance manages all your SmartConnectors

SmartConnectors that forward events to ArcSight ESM can be managed using the ESM Console, so the Connector Appliance is not required if all SmartConnectors have ESM as their only destination. However, the Connector Appliance is very useful when connectors target multiple heterogeneous destinations (for example, when ArcSight Logger is deployed along with ESM), in a Logger-only environment, or when a large number of SmartConnectors are involved, such as in a MSSP deployment.

Connector Appliance SmartConnectors operate within Containers. Each Container runs its own Java Virtual Machine (JVM). Containers contain one or more SmartConnectors.



Managing SmartConnectors on the Connector Appliance

The Connector Appliance manages three types of SmartConnector:

Local (on-board) SmartConnectorsThe Connector Appliance includes multiple Containers and on-board SmartConnectors. The manager interface can be used to manage these local SmartConnectors as well as remote connectors.

Remote Connector Appliance SmartConnectorsThe Connector Appliance can manage SmartConnectors on remote Connector Appliances, as well as other ArcSight hardware solutions such as ArcSight Logger.

Software-Based SmartConnectorsPreviously-installed, software-based SmartConnectors can be remotely managed by some Connector Appliance models, but the remote management feature is disabled on software SmartConnectors by default.

To manage software-based SmartConnectors with the Connector Appliance, enable remote management on them. To do so, add the following property to the user/agent/agent.properties file in the installation directory of each SmartConnector that you want to manage with the Connector Appliance:

remote.management.enabled=true

Restart the SmartConnector for property changes to take effect.

You can also customize the port on which the Connector will be listening. By default, this port is set to 9001, but it can be changed by adding the following property to user/agent/agent.properties:

remote.management.listener.port=9002

In the example above, the Connector listens on port 9002.

High load on on-board connectors may impact performance of the Connector Appliance’s web-based interface.

Only fifth-generation SmartConnectors support remote management, so you will need connector build 4855 (4.0.5.4878.0) or later to use this feature.

Remote Management is not supported on connectors running AIX. This limitation is due to elements within the AIX platform.

Multiple software-based SmartConnectors installed on the same host require a separate port assignment. The default port for ArcSight SmartConnectors is 9001, so the second SmartConnector installed on the same host should use an alternate port. ArcSight recommends using port 9002, 9003, 9004, and so on.



For a complete list of all SmartConnectors supported by the Connector Appliance, see the Connector Appliance Release Notes or visit the ArcSight Customer Support website. New SmartConnectors are added on a regular basis.

Choosing a Deployment ScenarioThe Connector Appliance can be deployed wherever ArcSight SmartConnectors are needed, providing the following benefits:

SmartConnector management without ArcSight ESM (that is, Logger-only environments)

Remote control of runtime parameters, such as bandwidth control

Centralized SmartConnector upgrade management and control

Centralized troubleshooting of specific SmartConnectors

ArcSight Logger ArcSight Logger receives events from and sends to ArcSight SmartConnectors, but lacks the depth of SmartConnector management found in ArcSight ESM.

A Logger-only deployment benefits from the Connector Appliance in many ways, and provides most, but not all, of ESM’s management function (for example, it does not contain the filter designer). The Connector Appliance also offers features that ESM does not, such as bulk operations (enabling control of many SmartConnectors at one time).

Connector Appliance also can configure SmartConnectors with failover destinations, providing central failover control when redundant Loggers are deployed for this purpose. All or some SmartConnectors can be configured to send events to a second Logger or to an event file in the case of communication failure with the primary destination.

For more detailed information about Logger, see Chapter 8‚ Using SmartConnectors with ArcSight Logger‚ on page 105

ArcSight ESMDeploying the Connector Appliance in an ArcSight ESM environment centralizes SmartConnector upgrade, log management, and other configuration issues. For more information, see Chapter 5‚ Configuring SmartConnectors‚ on page 55.

ESM and LoggerConnector Appliance centralizes control when events are sent to ESM and Logger simultaneously. In one scenario, all events are sent to Logger while only high-value events are sent to ESM (for further analysis, for example). In another scenario, all events are sent to both, but Logger implements a longer retention policy.

Although each SmartConnector has specific destination parameters, the Connector Appliance allows for “bulk” management, eliminating the need to manually access each remote SmartConnector host to add or change destinations.

For more detailed information and instructions for using Connector Appliance, refer to the Connector Appliance Administrator’s Guide.


Chapter 8

Using SmartConnectors with ArcSight Logger

ArcSight Logger is a log management solution that is optimized for extremely high event throughput. Logger logs (or stores) time-stamped text messages, called events, at high sustained input rates. Events consist of a receipt time, a source (host name or IP address), and an un-parsed message portion. Logger compresses raw data, but also can retrieve it in an unmodified form for forensics-quality litigation reporting. Unlike ArcSight ESM, Logger does not normalize events.


Multiple Loggers can work together to support an extremely high event volume. Logger can be configured as a peer network with queries distributed across all peer Loggers.

Sending Events from Logger to an ESM MangerLogger’s most basic function is to store a large volume of security events. Logger can send a subset of these events to an ArcSight ESM Manager. It sends syslog or Arcsight Common Event Format (CEF) events directly to ArcSight ESM through a built-in SmartConnector called an ESM Destination. An ESM Destination appears as a SmartConnector on an ESM Console. For more information about ESM Destinations, see the ArcSight Logger Administrator’s Guide.

“Sending Events from Logger to an ESM Manger” on page 105

“Sending Events to Logger” on page 107

“Sending Events to Both Logger and an ESM Manager” on page 108

“Forwarding Events from ArcSight ESM to Logger” on page 111


8 Using SmartConnectors with ArcSight Logger

Logger and SmartMessageSmartMessage is ArcSight technology used by Logger to provide a secure channel between SmartConnectors and ArcSight Logger. SmartMessage provides an end-to-end encrypted secure channel. At one end is an ArcSight SmartConnector, receiving events from the many devices it supports; on the other end is SmartMessage Receiver on Logger.

Figure 8-1 Logger Receivers (R) and Forwarders (F)

The SmartMessage secure channel uses HTTPS (secure sockets layer protocol) to send encrypted events to Logger. This is similar to, but different from the encrypted binary protocol used between SmartConnectors and ArcSight ESM Manager.

Use port 443 (rather than ArcSight's traditional port 8443) because the secure channel uses HTTPS.



Sending Events to Logger1 Set up the SmartMessage Receiver on Logger (see the ArcSight Logger Administrator’s

Guide for detailed instructions).

2 Install the SmartConnector component as documented in the SmartConnector Configuration Guide.

3 Navigate through the panels to the one that states Please select the destination type: and select ArcSight Logger SmartMessage (encrypted). Click Next.

4 Enter the Logger Host Name/IP, leave the port number at default (443), and enter the Receiver Name. This setting should match the Receiver name you created in step 1 so that Logger can listen to events from this SmartConnector. Click Next.



5 Navigate through the subsequent panels until receiving a message that confirms the configuration was successful. Click Finish to complete the process and exit the wizard.

Sending Events to Both Logger and an ESM Manager1 Set up the SmartMessage Receiver on Logger (see the ArcSight Logger Administrator’s

Guide for detailed instructions).

2 Install the SmartConnector component (see the SmartConnector Configuration Guide for your device).

3 Register the SmartConnector with a running ArcSight ESM Manager and test that the SmartConnector is up and running.

4 Using the $ARCSIGHT_HOME\current\bin\runagentsetup script (or arcsight agentsetup -w), restart the SmartConnector configuration program.



5 When the SmartConnector Configuration Wizard is displayed, select I want to add/remove/modify ArcSight Manager destinations and click Next.

6 Select Add new destination and click Next.



7 Select ArcSight Logger SmartMessage (encrypted).

8 Specify the Host Name/IP, the desired Port, and select either Disabled (the default value) or Enabled data compression. Click Next.



9 A message confirms that the configuration was successful. Click Finish to complete the process and exit the wizard.

10 Restart the SmartConnector for changes to take effect.

Forwarding Events from ArcSight ESM to LoggerThe ArcSight Forwarding Connector can read events from an ESM Manager and forward them to Logger using ArcSight’s Common Event Format (CEF).

The Forwarding SmartConnector is a separate installable file, named similarly to this: ArcSight-4.x.x.<build>.x-SuperConnector-<platform>.exe.

Use ArcSight Forwarding Connector build 4810 or later for compatibility with Logger v1.5 or later.



1 Follow the instructions in the SmartConnector Configuration Guide for your device to install the SmartConnector. When you reach step 3, select CEF Syslog and click Next.

2 Specify the required parameters for CEF output. Enter the desired port for UDP or TCP output (TLS is not applicable). These settings should match the Receiver you created in Logger to listen for events from ArcSight ESM.

Parameter Description

Ip/Host IP or host name of the Logger

Port 514 or another port that matches the Receiver

Protocol UDP or Raw TCP



3 Continue with SmartConnector installation.

To configure the ArcSight Forwarding SmartConnector to send CEF output to Logger and send events to another ArcSight ESM Manager at the same time, see “Sending Events to Both Logger and an ESM Manager” on page 108.

Defining SmartConnector Settings in LoggerAfter installing the SmartConnectors to communicate with Logger, you can set up their properties through the SmartConnector Configuration Wizard. Assuming you have installed the SmartConnector component as previously shown (see Chapter 4‚ Installing SmartConnectors‚ on page 35 for detailed instructions), complete these steps:

1 Using the $ARCSIGHT_HOME\current\bin\runagentsetup script (or arcsight agentsetup -w), restart the SmartConnector configuration program.

2 After the SmartConnector Configuration Wizard is displayed, select I want to add/remove/modify ArcSight Manager destinations and click Next.



3 Select ArcSight Logger SmartMessage (encrypted) and click Next.

4 Select Modify destination settings and click Next.



5 The following window provides a choice of destination settings to modify. For this example, select Time Correction and click Next.

6 Each choice opens a unique set of windows to configure. Modify the appropriate settings and click Next.

For detailed descriptions of each configurable setting, see Chapter 5‚ Configuring SmartConnectors‚ on page 55.

For detailed instructions on using the Filter option, see Chapter 11‚ CSV File Transport Destination‚ on page 127.



7 The next window asks whether you want to end the session or select new destination settings to modify. To make additional modifications, select No; to end the session, select Yes.

8 When No is selected, the list of destination settings is redisplayed. When Yes is selected, click Finish to end the session.


Chapter 9

Using SmartConnectors with NSP


OverviewArcSight Network Synergy Platform (NSP) is an appliance that consists of these two licensed software components, also known as managers:

Network Configuration Manager (NCM)

Threat Response Manager (TRM)

These two components build and maintain a detailed understanding of your network’s topology, letting you centrally manage your network infrastructure and rapidly respond to security incidents.

The NCM/TRM solution lets you automate network configuration changes across heterogeneous networks, manage and audit configuration changes on the network from a central console, and obtain quick and easy web-based reports for network device inventory and configuration settings.

The ArcSight Syslog SmartConnector increases NSP’s visibility into the network. It detects network configuration changes in syslog format using SNMP traps, which can then trigger NSP to launch an action to poll the network devices for the complete, new configuration.

The benefits of the NCM/TRM solution include:

Complete visibility into all changes being made to network devices, even where the changes are made directly to the network devices.

Real-time detection and notification for any non-compliant or unauthorized changes.

Ensured compliance with internal standard operating procedures as well as external regulations.


“Deploying a Syslog SmartConnector with NSP” on page 118

“Configuring the Syslog SmartConnectors” on page 120

The following instructions apply to SmartConnector version 4.0.6 and later, which support SmartMessage communication with NSP. If you do not have this or a later SmartConnector build, download the latest from the ArcSight Customer Support Site.


9 Using SmartConnectors with NSP

Deploying a Syslog SmartConnector with NSPWhere you deploy NSP with an ArcSight Syslog Connector, NSP is also connected to ArcSight ESM, letting you use ESM's correlation, trending, reporting, and monitoring tools to track network configuration activity in conjunction with other activity on your network. An ArcSight Syslog Connector can also connect NSP with ArcSight Logger, which provides a clearer picture of network configuration changes happening on your network.

Other uses for deploying syslog SmartConnectors in conjunction with NSP include:

Enabling a hybrid configuration and change control model that permits certain changes to be made directly to network devices, while still maintaining control, visibility, auditing, and compliance for all changes in a central repository (NSP).

Providing a closed-loop solution for capturing network configuration related event information from all sources from which the change can be made (NSP directly, proxied through NSP, or directly to the device) and forwarding this information to ESM in an integrated manner.

The SmartConnector installation wizard contains an NSP Device Poll Listener destination. The Device Poll Listener detects when changes are made to network devices outside of NSP. The SmartConnector captures these changes by collecting syslog output from modified network devices and categorizes the events for ESM.

The SmartConnector then initiates an action through NSP to poll the specific modified network devices to determine the precise changes made to the configuration.

At the same time, NSP can run audits automatically to determine whether the particular change caused the configuration to fall into a non-compliant state. NSP determines this by comparing the current device configuration parameters against the pre-defined policy or benchmark. If there is a deviation from the policy, the audit fails and an alert is sent to the appropriate personnel within the organization, notifying them of the audit failure so they can take immediate action.

You also have the option of forwarding all categorized events to ArcSight ESM or Logger in a normalized format through ArcSight Common Event Format (CEF) for further analysis or storage.

By capturing these changes and immediately prompting NSP to run a device poll or audit at the precise time of the configuration change, this solution provides an automatic, real-time, closed-feedback loop for all configuration changes, even if they are made directly to network devices outside the scope of NSP.



The following diagram depicts the Syslog SmartConnector solution deployed with NSP and ESM.

Figure 9-1 The Syslog SmartConnector solution deployed with ArcSight NSP

Please keep the following in mind when configuring and deploying NSP:

It is optional to run NSP as an audit while the device is polled; however, it does require that audits be currently subscribed to that particular network device or device group.

Alert options include syslog, SNMP, and e-mail.

Remediation is an optional step, as some administrators may simply want to be alerted of the change so they can take their own actions; however, remediation requires that the appropriate remediation links be built in advance.

It is optional to forward events to ESM or Logger. Neither appliance is required for this solution to be fully functional.

For NSP to poll a network device, it must be previously known within the network.

The NSP solution can also be used to remediate the non-compliant device by rolling back to the previous configuration, or by making the specific configuration changes required to return the device into a state that is compliant with the policy or benchmark.



Configuring the Syslog SmartConnectors1 Follow the installation instructions from the SmartConnector Configuration Guide of

your Syslog device through SmartConnector installation step 3.

2 When the window is the displayed asking for you to select the destination type you want to configure, select NSP Device Poll Listener and click Next.

3 Enter the NCM Host name or IP address, the NCM User, and the NCM Password. The NCM Host is the IP address or hostname of the NSP system that will interact with the syslog connector. The NCM User and NCM Password are the user name and password credentials you use to log into the NSP system.

4 Click Next.

5 Continue with SmartConnector installation step 8 from the SmartConnector Configuration Guide of your Syslog device.


Chapter 10

CEF Destinations

This chapter explains the three selections available for sending events in Common Event Format (CEF).

The following topics are discussed:

CEF SyslogThe TCP and UCP destination can be used to send events to ArcSight Logger, where data is received using a TCP or UDP Receiver. One such receiver can receive from more than one connector.

For detailed information, see Chapter 8‚ Using SmartConnectors with ArcSight Logger‚ on page 105.

The TLS protocol provides a means of sending events through a secure channel (an option that does not apply to ArcSight Logger). This data can be received by any application that supports TLS reception. ArcSight supports TLS reception through the SyslogNG connector. If you wish to use the TLS protocol, ensure that you have installed and configured the SyslogNG Connector on the receiver side.

If the connector is not yet installed, refer to the ArcSight SmartConnector Configuration Guide for Syslog NG Daemon for instructions.

“CEF Syslog” on page 121

“CEF Encrypted Syslog (UDP)” on page 123

“CEF File” on page 124


10 CEF Destinations

1 To proceed, run the SmartConnector Installation Wizard and choose CEF Syslog from the selection:

2 Enter the Ip/Host and Port information, then choose TLS from the drop-down menu.

When TLS is chosen and an IP/host and port are specified, a test connection is made. If a SyslogNG connector is running on the specified IP/Port, it returns a certificate.


10 CEF Destinations

If the certificate is not already trusted, the wizard will show you the details of who issued the certificate, and ask you if you want to trust the certificate.

3 Click Yes if you trust this certificate.

4 Proceed with the remainder of the installation.

CEF Encrypted Syslog (UDP)The CEF Encrypted Syslog (UDP) destination allows for events to be sent encrypted over UDP, using a “Shared Secret”.

To decrypt the data on the receiving side, ensure that you have installed and configured the ArcSight CEF Encrypted Syslog (UDP) connector. If the connector is not yet installed, refer to the SmartConnector Configuration Guide for ArcSight CEF Encrypted Syslog (UDP) for instructions.

1 To proceed, run the SmartConnector Installation Wizard and choose CEF Encrypted Syslog (UDP) from the selection. Click Next.

When configuring a connector with the CEF Syslog destination, if the wizard is unable to fetch and import the destination certificate, you can import the certificate manually.

To do so, copy the certificate from the destination to a temporary location, then follow the steps under “Import the Certificate” in the SmartConnector Configuration Guide for Syslog NG Daemon.


10 CEF Destinations

2 The following screen prompts you to configure the destination with a port, IP address, and a 16 character shared key for encryption (Shared Secret). The same Shared Key must be used when configuring the ArcSight CEF Encrypted Syslog (UDP) connector on the receiving side.

3 Click Next and proceed with the installation.

CEF FileThis selection allows you to capture events that a SmartConnector would normally send to the ArcSight ESM Manager, and route them to a file. The format called Common Event Format (CEF) can be readily adopted by vendors of both security and non-security devices. This format contains the most relevant event information, making it easy for event consumers to parse and use them.

For detailed descriptions of field information, see the CEF Standard Document (Common Event Format Guide, version 14).


10 CEF Destinations

InstallationTo create a SmartConnector that logs security events in a CEF file rather than forwarding them to an ArcSight ESM Manager, run the SmartConnector Installation Wizard and, from the selection, choose CEF File.

Enter the following values for these parameters:

After you enter the file transport parameters and click Continue, the SmartConnector Configuration Wizard proceeds as usual.

File RotationEvents are appended to the current file until the rotation time interval expires or the maximum file size is reached. When either condition is exceeded, a new current file is created and the previous current file is renamed (as detailed below).

Event files are named using the timestamp of their creation, and all files, with the exception of the current file, have the text 'done.cef' appended. For example, a typical CEF file set configured to rotate every hour might consist of files named in this manner:

2010-01-28-10-55-33.cef

2010-01-28-09-55-33.done.cef

2010-01-28-08-55-33.done.cef

Parameter What to enter or select

CEF Folder Path where the CEF files are stored

File Rotation Interval

The desired file rotation interval, in seconds. The default is 3,600 (one hour).

File Size File size in megabytes (default: 10 MB)


10 CEF Destinations


Chapter 11

CSV File Transport Destination

This chapter explains how to capture events that a SmartConnector would normally send to the ArcSight ESM Manager, and route them to a file. Typical ArcSight configurations do not require the use of external files to communicate events to the ArcSight ESM Manager.

The following topics are discussed:

OverviewEvent data is written to a file in Excel-compatible comma-separated values (CSV) format, with comments prefixed by ‘#.’ A SmartConnector can be configured to preface the data with a comment line that describes the fields found on a subsequent line. A typical event file might look like this:

#event.eventName,event.attackerAddress,event.targetAddress

"Port scan detected","1.1.1.1","2.2.2.2"

"Worm ""Code red"" detected","1.1.1.1","2.2.2.2"

"SQL Slammer detected","1.1.1.1","2.2.2.2"

"Email virus detected","1.1.1.1","2.2.2.2"

Event data is written to files in the specified folder and can be configured to rotate periodically.

The information in this chapter applies only to SmartConnectors used with ArcSight ESM v4.0.


“Installation” on page 128

“Event Data Rotation” on page 128


11 CSV File Transport Destination

InstallationTo create a SmartConnector that logs security events in a CSV file rather than forwarding them to an ArcSight ESM Manager, run the SmartConnector Installation Wizard and, from the selection, choose CSV File.

Enter the following values for these parameters:

After you enter the file transport parameters and click Continue, the SmartConnector Configuration Wizard proceeds as usual.

Event Data RotationEvents are appended to the current file until the rotation time interval expires, at which time a new current file is created and the previous current file is renamed. One hour is a typical rotation time interval.

Event files are named using the timestamp of their creation, and all files, with the exception of the current file, have the text '.done.csv' appended. For example, a typical CSV file set configured to rotate every hour might consist of files named in this manner:

2007-01-28-10-55-33.csv

Parameter What to enter or select

CSV Path The path to the output folder. If it does not exist, the folder is created.

Fields A comma-delimited list of field names to be sent to the CSV file. Field names are in the form event.targetPort.

File rotation interval

The desired file rotation interval, in seconds. The default is 3,600 (one hour).

Write format header

Select true to send a header row with labels for each column, as described above.



2007-01-28-09-55-33.csv.done

2007-01-28-08-55-33.csv.done

Using the properties file, you can customize the configuration of your CSV SmartConnector to filter and aggregate events as desired.

You also can configure a SmartConnector to send events to a CSV file and an ESM Manager at the same time.




Appendix A

ArcSight Update Packs (AUPs)

This appendix details the different ArcSight Update Packs (AUPs) used in updating content to and from the ArcSight Manager and ArcSight SmartConnectors. AUP files may contain information that applies to SmartConnectors or ESM related updates.

The following topics are discussed in this appendix:

Defining an AUPAUP files provide a way to collect a set of files together and update ArcSight resources as well as distribute parsers to ArcSight SmartConnectors.

For some AUPs, ArcSight provides downloadable packages of new content available to subscribing customers. You can obtain a content subscription through ArcSight Sales or Customer Support. Subscribers also have access to related articles in the ArcSight Customer Support Center's Knowledge Base.

The download files are offered through a special subdirectory on the ArcSight software server. The directory is visible only to subscribers, who receive a notification e-mail from ArcSight Customer Support when files are posted.

ArcSight Content AUPsArcSight continuously develops new SmartConnector event categorization mappings, often called "content." This content is packaged in ArcSight Update Packs (AUP) files.

All existing content is included with major product releases, but it is possible to stay completely current by receiving up-to-date, regular content updates via Arcsight announcements and the Customer Support website (https://software.arcsight.com). Under "Content Subscription Downloads", the files are located in "RELEASE3.X".

“Defining an AUP” on page 131

“ArcSight Content AUPs” on page 131

“ArcSight Connector Upgrade AUP” on page 133

“ESM Generated AUPs” on page 134


A ArcSight Update Packs (AUPs)

Content updates (ArcSight-xxxx-ConnectorContent.aup) are provided by ArcSight and contain data that is then transferred to registered connectors. An AUP can provide updates for:

1 Event categorizations (Category Behavior, Category Object, etc.)

2 Default zone mappings (what IP maps to which zone by default)

3 OS mappings (when a network is scanned, where the asset is created)

As shown below, the method of uploading an AUP varies depending on the ArcSight product.

ArcSight ESMAs an ArcSight customer, you will receive an e-mail notification about content updates from ArcSight support. To update,

1 Download the latest AUP release from the Customer Support website (https://software.arcsight.com).

2 Copy the .aup file to ARCSIGHT_HOME\updates\ onto a running ArcSight ESM Manager. SmartConnectors registered to this ESM automatically download the .aup and, once completed, an audit event is generated.

ESM/LoggerA SmartConnector can send events to ArcSight ESM and Logger simultaneously. In this configuration, it’s helpful to use the AUP Master Destination feature. AUP Master Destination allows ESM to push AUP content to the SmartConnector used for its Logger destination(s). Logger is not capable of storing or pushing its own AUP content.

1 Using the SmartConnector Configuration Wizard, add the ESM destination and set the AUP Master Destination parameter to true (the default is false).

2 If you have not already done so, you can also add the Logger destination.

3 Copy the .aup file to ARCSIGHT_HOME\updates\ on the running ArcSight ESM Manager you added in step 1.

The AUP content is pushed from ESM to the SmartConnector, which then sends an internal event to confirm. Since the AUP Master Destination flag was set for the ESM destination, that AUP content is used by the SmartConnector for Logger or any other non-ESM destinations.

LoggerLogger has no facility to store or forward AUPs to SmartConnectors.

Connector ApplianceConnector Appliance does not support automatic deployment of an AUP. This feature will be included in future releases. Please call customer support for assistance.

The AUP Master Destination flag should be set to true for only one ESM destination at a time. If more than one ESM destination is set and the flag is true for more than one, only the first is treated as master.

Failover ESM destinations cannot be AUP Masters.



ArcSight Connector Upgrade AUPArcSight ESM

1 Download the latest AUP release from the Customer Support website (at https://software.arcsight.com).

2 Copy the .aup file to ARCSIGHT_HOME\updates\ onto a running ArcSight ESM Manager. SmartConnectors registered to this ESM automatically download the .aup and, once completed, an audit event is generated.

3 From the ArcSight Console, select connectors to be upgraded (one at a time) and launch the upgrade command for each of them.

4 Upon receipt of the upgrade command, the selected connectors upgrade themselves, restart, and send upgrade results (success or failure) back to the ArcSight Console through the ArcSight Manager.

a If the upgrade is successful, the new connector starts and reports a successful upgrade status. (The upgraded connector runs in the same home directory as the old one.)

b If the upgraded connector fails to start, the original connector restarts automatically as a fail-over measure. (This is essentially an automatic rollback, and re-start).

Connector ApplianceUploading an AUP through Connector Appliance is performed through it’s web-based user interface. From the Advanced Operations tab, the Connector Upgrade Repository displays upgrades that have been uploaded using the Connector Upgrade command.

To upload .aup updates,

1 Download the latest AUP release from the Customer Support website (at https://software.arcsight.com).

2 From the Advanced Operations tab, click Upgrade, and then click the Upgrade Repositories sub-tab.

3 Click Upload to browse to the downloaded .aup file.

4 Click the Submit button.

The next step is to push this upgrade to one or more containers. To push the upgrade .aup to a container(s),

1 Click the Upgrade Connectors sub-tab.

2 Click the check box for container(s) that you wish to upgrade.

3 Click the Upgrade button.

4 From the drop down menu, select the appropriate upgrade.

5 Click Save.

The file you updated should now appear in the list.

For more detailed information about Connector Appliance, see the Connector Appliance Administrator's Guide.



ESM Generated AUPsSome AUPs are generated by ESM itself for internal maintenance and operation.

User Categorization Updates User Categorization Updates (user-categorizations_user_supplied_00000000001300014581.aup) are generated by ESM when a user modifies the way an event is categorized through the ArcSight Console tools. These updates are then transferred to the registered connectors to update the way the newly sent events will be categorized. This is generally used for categorizing custom signatures for which ArcSight does not provide categorization.

System Zones UpdatesSystem Zones updates (system-zone-mappings_00000000000000000001.aup) are generated by ESM when a change to the ArcSight System zones is detected, then transported to the necessary connectors. It contains the new System-Zone mappings so incoming events are attached to the correct zones or assets in ESM.

As System Zones are always present, all SmartConnectors connected to ESM routinely receive them as an AUP.

User Zones UpdatesUser Zones updates (user-zone-mappings_3RxkkOxYBABDRZlZyr6nrWg==_00000000001700001895.aup) are generated by ESM when a change to a user-created zone configuration is detected, then transported to the necessary connector. It contains the new zone mappings so that incoming events are attached to the correct zones or assets in ESM.


Appendix B

SmartConnector Frequently Asked Questions

The following are a list of frequently asked questions. This section is periodically updated.

“What if my device is not one of the listed SmartConnectors?” on page 136

“My device is on the list of supported products; why doesn't it appear in the SmartConnector Configuration Wizard?” on page 136

“Why isn't the SmartConnector reporting all events?” on page 136

“Why are some event fields not showing up in the Console?” on page 136

“Why isn't the SmartConnector reporting events?” on page 136

“How can I get my database SmartConnector to start reading events from the beginning?” on page 136

“When events are cached and the connection to the Manager is re-established, which events are sent?” on page 137

“Why does the status report the size of the cache as smaller than it should be? For example, I know that a few events have been received by the SmartConnector since the Manager went down, yet the report marks events as zero.” on page 137

“Why does the estimated cache size never change in some SmartConnectors? Why is the estimated cache size negative in others?” on page 137

“Can the SmartConnector cache reside somewhere other than user/agent/agentdata?” on page 137

“Why is my end time always set to a later date and time?” on page 137

“Do our Syslog SmartConnectors support forwarded messages from KIWI or AIX?” on page 138

“What does the T mean in the periodic SmartConnector status lines?” on page 138

“What do Evts and Eps refer to?” on page 138

“Does a file reader SmartConnector reading files over a network share display errors when the network share is disconnected? How can I recognize which error message refers to which file in agent.log and agent.out.wrapper.log?” on page 138

“Are log files accessed sequentially or in parallel?” on page 139

“After reading a log file, can a SmartConnector move them using NFS?” on page 139

“My SmartConnector must read log files from a remote machine through a network share. How can I do this?” on page 139

“Is there any limitation on performance relating to EPS?” on page 139

“How many log files can a SmartConnector access at one time?” on page 139


B SmartConnector Frequently Asked Questions

“What is the recommended maximum number of SmartConnectors per ArcSight ESM Manager?” on page 139

What if my device is not one of the listed SmartConnectors?

ArcSight offers an optional feature called the FlexConnector Development Kit (SDK), which can assist you in creating a custom SmartConnector for your device.

ArcSight can create a custom SmartConnector; contact ArcSight Customer Support for more information.

My device is on the list of supported products; why doesn't it appear in the SmartConnector Configuration Wizard?

SmartConnectors are installable based upon the operating system you are using. If your device is not listed, either it is not supported by the operating system on which you are attempting to install, or your device is served by a Syslog server and is, therefore, a syslog sub-connector. To install a Syslog SmartConnector, select Syslog Daemon, Syslog Pipe, or Syslog File during the installation process.

Why isn't the SmartConnector reporting all events?

Check that event filtering and aggregation setup is appropriate for your needs.

Why are some event fields not showing up in the Console?

Check that the two separate turbo modes for the SmartConnector and the ArcSight ESM Manager are compatible for the specific SmartConnector resource. If the Manager is set for a faster turbo mode than the SmartConnector, some event details will be lost. See “Understanding ArcSight Turbo Modes” on page 33 for detailed information.

Why isn't the SmartConnector reporting events?

Check the SmartConnector log for errors. Also, if the SmartConnector cannot communicate with the Manager, it caches events until its cache is full. A full cache can result in the permanent loss of events.

How can I get my database SmartConnector to start reading events from the beginning?

If it is a FlexConnector for Time-Based DB, set the following parameter in agent.properties:

agents[0].startatdate=01/01/1970 00:00:00

If it is an FlexConnector for ID-Based DB, set the following parameter in agent.properties:

agents[0].startatid=0



When events are cached and the connection to the Manager is re-established, which events are sent?

Events are sent with a 70% live and 30% cached events ratio. If live events are not arriving quickly, the percentage of cached events can be higher. This can reach 100% if there are no live events.

Also, if the settings dictate that certain event severities are not sent at the time connection is restored, those events are never sent. This is true even if they were originally generated (and cached) at a time when they would ordinarily go out.

Why does the status report the size of the cache as smaller than it should be? For example, I know that a few events have been received by the SmartConnector since the Manager went down, yet the report marks events as zero.

Some of the events are in other places in the system, such as the HTTP transport queue. Shut down the SmartConnector and look at the cache size in the .size.dflt file to confirm that the events are really still there.

Why does the estimated cache size never change in some SmartConnectors? Why is the estimated cache size negative in others?

The estimated cache size is derived from a size file that gets read at startup and written at shutdown. If the SmartConnector could not write the size at shutdown (for example, due to an ungraceful shutdown, disk problem, or similar problem) the number could be incorrect. Newer versions will attempt to rebuild this cache size if they find it to be incorrect, but older builds do not.

One solution is to:

1 Stop the SmartConnector.

2 Delete the size file (a file with extension .size.dflt) under current\user\agent\agentdata.

3 Re-start the SmartConnector.

The SmartConnector detects that there is no size file and re-builds the cache size by reading all the cache files.

Can the SmartConnector cache reside somewhere other than user/agent/agentdata?

You can change the folder to contain the SmartConnector cache by adding the following property in agent.properties:

agentcache.base.folder=<relative-folder-path>

where <relative-folder-path> is the path of the folder relative to $ARCSIGHT_HOME.

Why is my end time always set to a later date and time?

ArcSight Manager performs auto time correction for older events. If the end time is older than your retention period, it is set automatically to that lower bound. A warning is displayed and an internal event with the same message is sent to you.



Do our Syslog SmartConnectors support forwarded messages from KIWI or AIX?

Yes.

The property related to KIWI is

syslog.kiwi.forwarded.prefix=KiwiSyslog Original Address

Kiwi adds a prefix with the original address. For example, the message:

Jan 01 10:00:00 myhostname SSH connection open to 1.1.1.1

is converted to

Jan 01 10:00:00 myhostname KiwiSyslog Original Address myoriginalhost: SSH connection open to 1.1.1.1

The SmartConnector strips out the prefix and uses myoriginalhost as the Device Host Name.

The property related to AIX is

syslog.aix.forwarded.prefixes=Message forwarded from,Forwarded from

Similar actions are performed for messages forwarded using AIX.

What does the T mean in the periodic SmartConnector status lines?

"T" is shorthand for "throughput(SLC)." The following lines are in agent.defaults.properties:

status.watermark.stdoutkeys=AgentName,Events Processed,Events/Sec(SLC),Estimated Cache Size,status,throughput(SLC),hbstatus,sent

status.watermark.stdoutkeys.alias=N,Evts,Eps,C,ET,T,HT,S

The SLC stands for Since Last Check, which means "in the last minute," assuming status.watermark.sleeptime=60 has not been overridden.

What do Evts and Eps refer to?

Eps is an acronym for Events Processed and Evts is an acronym for Events/Sec(SLC).

Does a file reader SmartConnector reading files over a network share display errors when the network share is disconnected? How can I recognize which error message refers to which file in agent.log and agent.out.wrapper.log?

If the network share is a Linux/UNIX NFS mount or a Windows network mapped drive, the file reader SmartConnector displays errors in the agent log.

If files are being read using a Windows UNC path that does not require network mapping, the file reader SmartConnector cannot detect a network connection loss.

Error messages related to file access contain the file name, but error messages related to log line parsing do not.



Are log files accessed sequentially or in parallel?

This depends upon the SmartConnector you are using. Some log file connectors process files sequentially and others process log files in parallel.

After reading a log file, can a SmartConnector move them using NFS?

Yes. Folder Follower connectors can rename or move the files using NFS, as long as the folders containing the log files give the correct permissions for the SmartConnector.

My SmartConnector must read log files from a remote machine through a network share. How can I do this?

To establish a network share to a remote machine, you can use network mapping on Windows platforms, and NFS or Samba mounting on Linux/UNIX platforms.

If you are running the SmartConnector as a Windows service, access privileges to the network share are required. To access the user name and password panel:

1 From the Start menu, select Control Panel.

2 Double-click Administrative Tools.

3 Double-click Services.

4 Right-click the name of the appropriate SmartConnector and select Properties.

5 Click the Log on tab, and enter the user name and password for the user with access permissions to the file share. Specify the file path using UNC notation, not as a network mapped drive.

Is there any limitation on performance relating to EPS?

These limitations are subjective and depend upon system resources, number of devices, number of events, and so on.

How many log files can a SmartConnector access at one time?

The SmartConnector can access as many log files as it is configured with. The folders are processed in parallel.

What is the recommended maximum number of SmartConnectors per ArcSight ESM Manager?

There is no hard and fast maximum. The Manager has a restriction of 64 concurrent SmartConnector threads by default. The more threads you add, the more it affects performance, because there is more thread context-switching overhead. The general recommendation is to definitely stay lower than the triple-digit range.




Documents

Smart Connector Users Guide