ActionConnector Cookbook - SEMplicity Inc.€¦ · ActionConnector Cookbook A quick guide to getting the ActionConnector up and running Introduction The ActionConnector is an ArcSight

ActionConnector Cookbook

A quick guide to getting the ActionConnector up

and running

Introduction

The ActionConnector is an ArcSight non-charged component designed to execute

defined tasks based on the commands sent from the ESM. It runs on a separate

server, so as not to impact ESM performance. Console Operators may invoke

ActionConnector commands directly from an Active Channel, passing parameters

from the event fields. Rule Actions may also invoke ActionConnector commands,

passing parameters from aggregated fields. In either case, the ActionConnector

executes a user written program or script and returns the results to the ESM in the

form of an internal ArcSight event. For commands invoked by the Console

Operator, the results of the script also appear in the Viewer pane.

The ActionConnector is very useful for automating common tasks, such as

shunning an IP across all firewalls, generating Remedy tickets based on event

fields, quarantining a device using an intrusion or endpoint protection system,

sending emails to end users and their managers based on event fields, etc. As it is

non-charged, the ActionConnector also provides a low-cost method to achieve

sophisticated automation without purchasing a third-party product.

Objectives

This document contains everything necessary to implement a single sample

ActionConnector application: HelloWorld. Rather than create a number of separate

page 2 © Copyright 2014, 2015 SEMplicity, Inc. All Rights Reserved.

SEMplicity • 61 Chapel Street, Newton, MA 02458 • 401-265-3184 • [email protected] • www.semplicityinc.com

files, we have attempted to embed everything within this document. As the sample

application is relatively simple, the effort required is to cut and paste or manually

enter the information. If you choose to implement this application, at the end of the

process, you will have:

A HelloWorld.py Python program implemented as a line command that

echoes back up the parameters passed to it.

An ActionConnector configured to use your ArcSight ESM as a destination,

with a single command available for testing: HelloWorld.

An ESM Integration Command and related Configuration that you can use

to execute the HelloWorld from the grid viewer editor contexts.

An ESM rule that executes the HelloWorld command automatically, passing

values aggregated from the records which triggered this rule.

A Second-Level Regex Parser implemented on this ActionConnector that

parses output from HelloWorld.py and populates fields in the ArcSight

ActionConnector Command Executed event.

An ActiveChannel to view ActionConnector events and responses.

From here, you will have an understanding of how the ActionConnector works and

a useful framework into which you can add your specific ActionConnector

applications.



Step 1: Create sample program

HelloWorld.py The sample program provided is in the Python programming language, version 2.7. You may

download an appropriate version for your platform here. You will, of course, need to have

Python running on your platform before creating this program. The HelloWorld.py program

below accepts up to three parameter arguments. If not supplied, the program uses a default value.

If the debug level (-d) specified in INFO or DEBUG, it then echoes back each parameter. Simply

copy the text below into a text file called HelloWorld.py.

#!/usr/bin/env python

# encoding: utf-8 '''

HelloWorld -- Short module to echo back passed parameters to stdout

Usage: python.exe HelloWorld.py [-h] [-V] [--d|--debug

DEBUG|INFO|WARNING|ERROR|FATAL] [-l|--log2file] [-1|--argument1

<ARGUMENT_1>] [-2|--argument1 <ARGUMENT_2>] [-3|-- argument3

<ARGUMENT_3>]

@author: George Boitano

@contact: [email protected]

@deffield 6/12/2014 '''

all = []

version = 0.2

date = '2014-06-12'

updated = '2015-07-08'

import sys, os import time

#import misc modules

from argparse import ArgumentParser

from argparse import RawDescriptionHelpFormatter import logging

# create a global logger for this and all sub modules logger =

logging.getLogger()

# set log message header

logFormatter = logging.Formatter('%(asctime)s - HelloWorld -

%(levelname)s -

%(message)s')

# set logging to debug for now. will override when we get the --

debug argument logger.setLevel(logging.DEBUG)

# create the console (STDOUT) logger handler consoleLogger =

logging.StreamHandler(stream=sys.stdout)

consoleLogger.setFormatter(logFormatter)

logger.addHandler(consoleLogger)

logger.debug("console logger created")

def GetArgs(arglist): """

parse and validate the arguments passed to the function, using the

argparser input: arglist

output: namespace containing parsed arguments """

program_version = "v%s" % version program_build_date = str(

updated )

program_shortdesc = "HelloWorld echos back the three passed argument

https://www.python.org/download/releases/2.7.8/

mailto:[email protected]



so syslog" program_license = '''%s

Created by George Boitano on %s.

Distributed on an "AS IS" basis without warranties or conditions

of any kind, either express or implied.

USAGE: HelloWorld.py [-h] [-V] [--d|--debug

DEBUG|INFO|WARNING|ERROR|FATAL] [-l|-- log2file] [-1|--argument1

<ARGUMENT_1>] [-2|--argument1 <ARGUMENT_2>] [-3|--argument3

<ARGUMENT_3>]

''' % (program_shortdesc, str( date )) program_name =

os.path.basename(arglist[0])

program_version_message = '%s %s (%s)' % (program_name,

program_version, program_build_date)

del arglist[0] try:

parser = ArgumentParser(description=program_license,

formatter_class=RawDescriptionHelpFormatter, prog=program_name)

parser.add_argument('-1', '--argument1', dest="argument1",

help="argument 1, indicated by -1 or --argument1 flag",

default='argument1 not specified', nargs='?', const='argument1 flag

specified, no value passed', required=False)









parser.add_argument('-d', '--debug', dest="debug", default="INFO",

help="debug messaging level: DEBUG, INFO, ERROR, FATAL",

choices=['DEBUG','INFO','ERROR','FATAL'],

nargs='?', const='INFO', required=False)

parser.add_argument('-l', '--log2file', dest="log2file",

action="count", default=0, help="log to ./HelloWorld.log file")

parser.add_argument('-V', '--version', action='version',

version=program_version_message, help="display program version

message")

args = parser.parse_args(arglist) except KeyboardInterrupt:

sys.exit(0) except Exception, e:

logger.exception('fatal error parsing arguments, error=' + repr(e) +

". for help please user --help")

raise arglist = [] return args

def main(argv=None):

""" main routine, takes no parameters, returns 0 if successful, 1 if

failure """

# get the arguments, put into dictionary parsedArgs =

GetArgs(sys.argv)

# set the overall logging level baed on debug argument

debugDict = {'DEBUG':logging.DEBUG, 'INFO':logging.INFO,

'WARNING':logging.WARNING, 'ERROR':logging.ERROR,

'FATAL':logging.FATAL}

logger.setLevel(debugDict[parsedArgs.debug])

# if log2file specified in arguments, create a file logger also if

parsedArgs.log2file > 0:

logger.debug("logging to file requested, log file is



HelloWorld.log") fileLogger =

logging.FileHandler('./HelloWorld.log', 'a')

fileLogger.setFormatter(logFormatter)

logger.addHandler(fileLogger) logger.info('file logger created')

if parsedArgs.debug in ['WARNING','ERROR','FATAL']:

logger.warning('debug level ' + parsedArgs.debug + ' will not echo

passed parameters. use level INFO or DEBUG to do this.')

# enumerate passed parameters

logger.info("HelloWorld arguments specified or defaulted:") for

attr, value in parsedArgs. dict .iteritems():

logger.info(" " + attr + ": " + repr(value)) return 0

#priming object

if name == " main ": mainrc = main()

print "HelloWorld.py ended, rc=" + str(mainrc)

sys.exit(mainrc)

After you have created HelloWorld.py, you can test it with the following command. Substitute

the directory of the HellowWorld.py file, and your Python home directory:

<PYTHON_HOME>/python.exe <HELLOWORLD_HOME>/HelloWorld.py –l –d INFO

-1 “Value for Parameter 1” -2

The results should look like this:

2014-08-20 11:24:34,736 - HelloWorld - DEBUG - console logger

created 2014-08-20 11:24:34,743 - HelloWorld - INFO - file logger

created

2014-08-20 11:24:34,744 - HelloWorld - INFO - HelloWorld arguments

specified or defaulted:

2014-08-20 11:24:34,744 - HelloWorld - INFO - debug: 'INFO'

2014-08-20 11:24:34,746 - HelloWorld - INFO - log2file: 1

2014-08-20 11:24:34,746 - HelloWorld - INFO - argument2: 'argument2

flag specified, no value passed'

2014-08-20 11:24:34,746 - HelloWorld - INFO - argument3: 'argument3

not specified'

2014-08-20 11:24:34,747 - HelloWorld - INFO - argument1: 'Test

Parameter 1'

HelloWorld.py ended, rc=0

Step 2: Create Configuration File The actionconnector.counteract.properties file contains definitions for the various commands

supported by this ActionConnector. For this example, it will have a single command defined,

calls HelloWorld.py and passes five parameters. Copy the following text into a file and save it in

any directory for subsequent use when installing the ActionConnector in the next step.

command.count=1 command[0].name=helloworld

command[0].displayname=Hello World command[0].parameter.count=5

command[0].parameter[0].name=parameter1



command[0].parameter[0].displayname=Parameter 1





command[0].parameter[3].name=flags

command[0].parameter[3].displayname=Flags: -l to log to file, -h for

help, -V for version command[0].parameter[4].name=debug

command[0].parameter[4].displayname=Msg Level:

DEBUG/INFO/WARN/ERROR/FATAL

command[0].action="<PYTHON_HOME>/pythonw.exe"

"<HELLOWORLD_HOME>/HelloWorld.py" '${flags}' -d ${debug} -1

'${parameter1}' -2 '${parameter2}' -3 '${parameter3}'

Please note the following for this file:

When you copy this text, be sure there are no smartquotes or em-dashes in the target file.

The ActionConnector only accepts single quotes and hyphens.

The name of the file must be all lower case, with no whitespace, and end with

.counteract.properties. In this instance, it is actionconnector.counteract.properties.

The command[0].name must be all lower case, no spaces.

The command[0].displayname can have mixed case and spaces – this is what will display

within ESM.

The command[0].parameter.count must match the number of parameters defined

immediately below.

The command[0].parameter[X].name values must be all lower case, no spaces. They are

used in the command action, below.

The command[0].parameter[X].displayname values can be mixed case and contain

spaces. They are what you will see in ESM when issuing this command through the

ActionConnector.

In the command action:

o We use pythonw.exe to execute HelloWorld.py because this example is for a

Windows environment. Under Linux/Unix, use python.exe.

o Regardless of whether you running under Windows or Linux/Unix, you can use

the forward slash (“/”) in directory names.

o Specify the parameter names passed to the command, below, as velocity



variables. Specifically, begin the variable with $ and then enclose the parameter

name in curly braces; for example:

${debug}.

o If an argument value can contain spaces or other whitespace, be sure to enclose

the entire velocity variable in single quotes: '${parameter1}', '${parameter2}',

etc.

Step 3: Install and configure

ActionConnector The ActionConnector Development Guide has complete installation instructions, and is available

on Protect724. The abbreviated instructions for this sample application follow below.

1. Upload the latest SmartConnector installer to your ActionConnector host

2. Install the SmartConnector. We will refer to the directory into which you

install the SmartConnector hereafter as <ARCSIGHT_HOME>. When

prompted to Add a Connector, exit the wizard by pressing Cancel.

3. Move the actionconnector.counteract.properties file created in the previous step into: <ARCSIGHT_HOME>/current/user/agent/flexagent/

4. From the DOS command prompt or Unix/Linux prompt, enter the following command: <ARCSIGHT_HOME>/current/bin/arcsight agentsetup -w –sa

https://protect724.hp.com/docs/DOC-2639



5. Choose your ArcSight Manager as the destination. Register the connector, as

you would any other, using appropriate Manager credentials.

6. When prompted, select ArcSight FlexConnector CounterACT from the

list of connectors.

7. When prompted on whether you wish to use the configuration wizard to

create the counteract properties file, answer No.

8. When prompted, enter actionconnector as the name of the configuration

file. Do not enter the entire name of the file; rather, just actionconnector,

omitting “.counteract.properties”.

9. Name the Connector, specify the Manager folder and enter other parameters

as you would for any other SmartConnector. Also, if you want to run the

ActionConnector as a service, specify this when prompted in the wizard. In



this instance, we have named our connector Test ActionConnector 2 and

placed it into the /All Connectors/Site Connectors/ folder.

When finished, in the ESM Console, you should see your ActionConnector in the

Navigation pane under the Connectors tree. If you right click the connector and

select Create Channel with Filter, you should see some standard connector

events.

Step 4: Test from Connectors tree You can manually test any command supported by this ActionConnector from the

Connectors tree in the Navigation tab. This is the easiest way to test new

commands.

1. Right-click the connector and select Send Command/CounterACT/Hello

World:



2. When prompted, enter the parameters to be passed to HelloWorld.py:

3. After you press OK, a Connector Command window opens in the Viewer pane.

From there, you can see the output from your command:



Also, back in our Active Channel, we can see that this command generated an

ArcSight agent:017 event, with the output in the Message field:

Step 5: Create Integration Command

Executing the command from the Navigator pane is useful for testing, but to derive

value from the ActionConnector, it must be accessible as an integration command.



This enables Console operators and ESM rules to run the command based on values

in specific events.

1. In the Navigator page, select Integration Commands. Right-click the folder in

which you want to store your commands, and select New Command:

2. Select your Integration Target Type as Connector.

3. Enter a name your integration and select the Hello World connector command.

Then, for the Parameters value, click the … button and a dialog will appear

prompting for the parameters to pass to HelloWorld.py. These parameters can

be ArcSight fieldnames, local variables, and special fieldnames such as

$selectedField. Below is an example:



Here are some notes on these parameter values:

The values can contain velocity variables, as shown for Parameters 1

through 3.

The special ${selectedField} variable indicates that the ESM should

substitute the field name selected by the Console user when invoking

this command. $selectedItem (not shown) populates the field value

selected. These two variables may be used for Integration Commands

invoked manually by the Console user, but not for Commands called by

Rule Actions.

The velocity variables in Parameters 2 and 3 use ArcSight and variable

field values extracted from an event record: $deviceEventClassId and

$name. This field value comes from an event selected by the Console

user in the View Pane (if invoked manually by a Console user) or from

an aggregated field of a correlated event (if called by a Rule Action).

You can also specify Global Variables, in the form:

${<GLOBAL_VARIABLE_NAME>}

In Program Flags, we have specified -l to have HelloWorld.py send log



records to the HelloWorld.log file in the <HELLOWORLD_HOME>

directory.

4. To invoke an integration command, you need also to define a configuration.

Select the Configuration tab, then select a folder, right-click and select New

Configuration.

5. Next, in the Editor pane, under the Attributes tab, select Connector as the

Integration Configuration type.



6. In the Context tab, select the appropriate ESM contexts from which you

expect to enter this command. As you can see from the various options, there

are many situations for which you can make an Integration Command

available. You can, for example, make a command available in the Viewer

pane on an Active Channel when an event and/or field is selected.

7. In the Commands tab, add the Integration Command you defined in step 3.



8. In the Targets tab, select the ActionConnector you previously created.

Once you save this configuration, the Integration Command becomes available

from any Active Channel by right- clicking a field within an event and selecting

Integration Commands/HelloWorld with Debug. The results show the output

from HelloWorld.py based on the arguments passed in the parameters defined for

the Command.



In this example, we right-clicked on the Name field for an Active Channel event.

As a reminder, our Command Parameters are $selectedField, $name and

$deviceEventClassId. Accordingly, the Command returns these values in the

output:



Of course, issuing an Integration Command manually also generates and agent:017



event, which contains the results in the $message field:

Step 6: Prompting from Commands Integration Commands, when invoked manually (not from rules), can also prompt

the Console user for input. They can also optionally save this input for future use to

the appropriate User resource in ESM, or even to the ESM ActionConnector

resource. Every user has permission to save prompted responses to their User

resource, but only users with Write permission to the ActionConnector resource can

save responses there.

To set up user prompt(s) for an Integration Command, edit the Command and enter

a new variable name for one or more of the parameters. Prompted variable names

must not currently exist in ESM: they cannot be standard ArcSight field names, nor

can they have the same name as any Global Variables you have defined. Beyond

that, we suggest you avoid whitespace and use all lower-case for these prompt

response variables. Below, we have specified three: ${prompt1}, ${prompt2} and

${prompt3}.



Now, when we select a row in an Active Channel and issue the HelloWorld

Integration Command, we see a prompt for three variables:

You may enter values for one or more variables. Notice the Save to Target and

Save to User checkboxes at right. If you enter a value for a variable and check one

or both boxes, ESM will save your response and re-use it on subsequent

Commands. If you Save to User, then whenever you issue this command in the

future, it will not prompt you, but rather populate the value you entered. If you Save

to Target, then whenever anyone issues this command, it will never prompt for this

value, but rather always use the value you entered. ESM saves the variable response

to the User and/or Connector resource under the Integration Parameters tab. If you

want to remove a stored response value, you can edit the User or Connector

resource and delete the value.



Here is an example that demonstrates this functionality, for Save to User.

1. First, we invoke the HelloWorld Integration Command for a row in an Active

Channel:

2. We enter responses to all three prompts. We also check Save to User for

prompt responses 2 and 3, but not the response to prompt 1.

3. This saves the prompt responses 2 and 3 to the Integration Parameters tab of

my User resource:

4. Now, when we issue this command again, we are only prompted for prompt1:



5. The command output, however, shows that the responses stored previously

were passed to HelloWorld.py:

6. If we edit the User resource and delete the Integration Parameters, however,

then the Integration Command will again prompt for all three responses:



Step 7: Invoking Commands from Rules ESM Rules may execute Integration Commands from Actions. The main difference

between this mode of execution and calling the command manually is that Rule

Actions cannot prompt the Console user. Therefore, parameters passed to the

Integration Command must be either hardcoded strings, fields aggregated from Rule

events, or Global/Local Variables. The Integration Command takes these arguments

and executes the appropriate command. It passes results back in the agent:017

records.

Below is an example of configuring a rule to run an Integration Command.



1. First, we must change our Integration Command not to prompt.

In this instance, we have changed the command to use event name, agent host name

and device event class ID. These values are only in effect for manual invocations.

For Rules, as shown below, we will populate the same values from the aggregated

fields.

2. Now, we will create a new rule which fires on agent:050 events produced by



our ActionConnector. These occur approximately every five minutes.

3. Under the Aggregation tab, we must be sure to aggregate all the fields we plan

to pass to the HelloWorld command. If you plan to pass local or global

variables (not shown), you must aggregate them here also.

4. In the Actions tab, Right-Click a Trigger and select Add/Execute Connector

Command.



5. In the Add Execute Connector Command Action pop-up, enter the

Connector and Command. Also, fill in the parameters with either string

constants or aggregated field names from the previous step.

6. Now, when the Rule fires, it will generate a HelloWorld command passing

these parameters. This in turn will send an agent:017 event detailing the

results. Below, we have modified our Active Channel to show both the rule



firing and the resulting agent:017 event.

Step 8: Second-Level Regex Parser Had enough yet? We certainly hope not, because we have one more topic.

Whenever a command is executed on the ActionConnector, the ActionConnector

returns an event to the Manager containing the output from the command in an

agent:017 Command Executed event. With a Second-Level Regex Parser, you can

parse this output into fields within this same event. Using this feature, you can write

sophisticated content to gather information passed back by the ActionConnector and

take subsequent Rule Actions, create Trends or Query Viewers, generate Reports,

etc.

A Second-Level Regex Parser is not exclusive to the ActionConnector. You may

also use this feature in SmartConnectors and FlexConnectors to re-parse selected

fields. This is often a simpler alternative to writing a parser override. In the example

below, we implement a parser to extract the parameter names from the return code

from HelloWorld.py and place it in the deviceCustomNumber1 within the resulting

agent:017 event sent to the Manager.

A regexparser must first specify a field to parse. Then, it parses the field, much like

a normal FlexConnector, and sets the values for other fields. If the field it populates

already contains a value, it overlays that previous value. To implement:



1. On the ActionConnector host, create a new directory: <ARCSIGHT_HOME>/

current/user/agent/fcp/additionalregexparsing/ngflexcounteract/

2. Within this directory, create a new file – regex.0.sdkrfilereader.properties –

and copy the following into it:

source.field=event.message regex=(?s).*?(\\S+) ended, rc=(\\d+).*

token.count=2 token[0].name=ProgramName token[0].type=String

token[1].name=ReturnCode token[1].type=Integer

event.deviceCustomNumber1=ReturnCode

event.deviceCustomNumber1Label=

concatenate(ProgramName,"HelloWorld.py ")

Here are a few notes on this parser:

The source field is the event message. This is where the

ActionConnector places the output from the program it calls.

The regexbegins with the (?s) operator, which causes . to match line

breaks. This allows you to parse across linebreaks in the HelloWorld.py

output.

The regexcaptures two fields, the program name and the return code.

Here is an example of a typical message it will parse:

Message : Standard Output:

2014-08-21 16:34:13,051 - HelloWorld - DEBUG - console logger

created 2014-08-21 16:34:13,065 - HelloWorld - INFO - file

logger created

2014-08-21 16:34:13,065 - HelloWorld - INFO - HelloWorld

arguments specified or defaulted:

2014-08-21 16:34:13,065 - HelloWorld - INFO - debug: 'INFO'

2014-08-21 16:34:13,065 - HelloWorld - INFO - log2file: 1

2014-08-21 16:34:13,065 - HelloWorld - INFO - argument2:

'labhonts1060b' 2014-08-21 16:34:13,065 - HelloWorld - INFO -

argument3: 'agent:050'

2014-08-21 16:34:13,065 - HelloWorld - INFO - argument1:

'Connector Raw Event Statistics'

HelloWorld.py ended, rc=0

The two captured fields are mapped into deviceCustomNumber1 and

deviceCustomNumber1Label.

3. After restarting the ActionConnector, this parse will populate these two fields,



as shown in our Active Channel below.

Conclusion This concludes our tour of the ActionConnector. Possible follow-on topics include:

Debugging techniques and common errors.

Agent.properties values for tuning and handling long-running

commands.

Use of Global Variables in Integration Command parameters,

especially the EvaluateVelocityTemplate function.

Join rules to correlate agent:017 event containing ActionConnector

results with the correlated events which issued the ActionConnector

command.

Existing ActionConnector implementations for vendor products, such

as Mandiant and RSA Aveksa.

Please feel free to contact the author, George A. Boitano of SEMplicity, Inc.

([email protected]) if you have questions, corrections or other ideas for

the ActionConnector. SEMplicity is a specialist consulting firm focusing on all

aspects of the ArcSight product, especially innovative use cases and automation.

mailto:[email protected]

Documents

ActionConnector Cookbook - SEMplicity Inc.€¦ · ActionConnector Cookbook A quick guide to getting the ActionConnector up and running Introduction The ActionConnector is an ArcSight