Upload
jarihd
View
279
Download
8
Tags:
Embed Size (px)
DESCRIPTION
Arcsight Writing Rules
Citation preview
www.arcsight.com 1© 2010 ArcSight Confidential
© 2010 ArcSight, Inc. All rights reserved. ArcSight and the ArcSight logo are trademarks of ArcSight, Inc. All other product and company names may be trademarks or registered trademarks of their respective owners.
SN08: Primer: Writing Rules Not Meant to be Broken
Javier Inclan Worldwide Principal Instructor
September 2010
www.arcsight.com 2© 2010 ArcSight Confidential
Agenda
Rules foundations Understanding rule components
– Conditions– Aggregation– Actions– Triggers
Mastering rules Additional rule features Troubleshooting rules Tuning rules
www.arcsight.com 3© 2010 ArcSight Confidential
Rules Foundations
www.arcsight.com 4© 2010 ArcSight Confidential
First Things First: Rule Definition
What is a rule? Process who is running in the ArcSight ESM manager Evaluates incoming events looking for specific conditions and patterns.
Based on these results it infers meaning about their significance and can initiate actions in response
Applied to events during the correlation evaluation phase of the event lifecycle
Rules are loaded by the ArcSight ESM correlation engine when ArcSight ESM starts up
Rules Foundations
www.arcsight.com 5© 2010 ArcSight Confidential
and or not
Event Definition
JoinCondition
Two additional editor conditions
plus same regular conditions
Rules Foundations
Concepts for Configuring Rules
Constructed using aggregation and Boolean pattern matching within the CCE (AND, OR, NOT)
Rules operate in the real time event stream
Must be activated (save or linked into the real time rules folder)
Moving rules in and out of the “real-time rules” folder triggers the correlation engine to reload the rules
Rules can be scheduled to run at predefined intervals
Scheduled rules do not have to be in the “real-time rules” group
www.arcsight.com 6© 2010 ArcSight Confidential
What Rules Do
Incoming events are compared to conditions and aggregation settings of each enabled rule
Event matches trigger pre-configured actions and a correlation event is generated by default
Rules Foundations
www.arcsight.com 7© 2010 ArcSight Confidential
What is a Correlation Event?
Correlation events become new events to be evaluated by the correlation engine
Rules Foundations
www.arcsight.com 8© 2010 ArcSight Confidential
Identifying Rules Types
Types of Rules Simple Rules
– Match one or more events against one set of conditions
Join Rules– Match more than one event against two or more sets of conditions
Rules Foundations
www.arcsight.com 9© 2010 ArcSight Confidential
Understanding Rule Components
www.arcsight.com 10© 2010 ArcSight Confidential
Understanding Rule Components
A rule definition is based on four components – Conditions– Aggregation– Actions– Triggers
These components are distributed within three different tabs in the rule’s editor
www.arcsight.com 11© 2010 ArcSight Confidential
What Events am I Looking For?
Conditions will define the set of events that I am looking for
Drivers for define this set of conditions could be use case definitions, compliance or computer/network/device security business requirements
Conditions are created using the CCE
Conditions rely on Boolean Logic principles
Understanding Rule Components
www.arcsight.com 12© 2010 ArcSight Confidential
Defining Precise Conditions
What events am I looking for? Events who have met these conditions are going to be named
“matches” Conditions can be loose
attacker address inSubnet 192.168.1.0/24 or target address HasVulnerability xxxx
Conditions can be well defined/precise attacker address=192.168.1.10
Which one is best for performance?
The devil is in the details (i.e… put a = instead of a > and your rule conditions change, whether intentional or !=)
Understanding Rule Components
www.arcsight.com 13© 2010 ArcSight Confidential
Use Categorization Fields in Rules
Develop logical framework for grouping resources Leverage ArcSight event categorization
– Since devices do not utilize a common naming schema for events, ArcSight Connectors map individual signatures to a common taxonomy so that ArcSight ESM can later reason over those events
– Without categories: [ID contains 529 or 621] OR [login and failure and SSH] OR [login and failure and target port 23]
– With categories:
Understanding Rule Components
www.arcsight.com 14© 2010 ArcSight Confidential
Benefits of Network Modeling in Rule’s Conditions
Effective Rules DEPEND on ArcSight ESM product intelligence Enables content that can make informed decisions based on more
detailed information– Asset model describes attributes of the assets
• Vulnerabilities locations active lists asset categories• Increases accuracy of ArcSight priority formula• Identifies assets subject to compliance
Understanding Rule Components
www.arcsight.com 15© 2010 ArcSight Confidential
Understanding Rule Components
Benefits of Network Modeling in Rule’s Conditions
Enables you to build a business-oriented view of dataassets/ranges zones networks customers
ArcSight WITHOUT network, zone, asset modeling, categorization and vulnerability information will produce more false positives and “background chatter” than a mis-configured IDS(aka OPEN THE FLOOD GATES)
How is a vulnerability scanner throwing out traffic on port 23 to 100 servers and analyzing a response differ from a CiscoWorks server using port 23 to push IOS upgrades to 100 switches?
www.arcsight.com 16© 2010 ArcSight Confidential
Aggregation or Aggravation?
Not a mis-spelling; we did not say aggravation
Aggravation might be a symptom if timing parameters and number of events within a specified time frame aren’t well understood
Do you want to aggregate on unique or identical fields?
Before rolling your rule out to production – TEST IT in development, QA or with simulated events fed from a test connector
Understanding Rule Components
www.arcsight.com 17© 2010 ArcSight Confidential
Defining Aggregation
Rule Aggregation Sets required number of event matches within a specified timeframe
– Time frame set here is known as time window expiration Matches only if specified field or fields is unique amongst evaluated events Matches only if specified field or fields is identical amongst evaluated events Values from fields listed in aggregation settings will be carried from base events
to correlation events
Understanding Rule Components
www.arcsight.com 18© 2010 ArcSight Confidential
Rule: Defining Aggregation
What fields to aggregate on?– Generally: event name, attacker/target
Hostname/Address/FQDN/Domain Name/User Name/Zone Resource– Non-aggregated fields can’t be used in dashboards and reports
– Aggregation impacts memory, as aggregation matchesare counted and tracked• Do not aggregate over long periods of time; instead use an active list• Limit the set of aggregated values
!Tip For MSSPs:Aggregate on CustomerResource to ensure eventsfrom the same IP addressare really from thesame machine
Understanding Rule Components
www.arcsight.com 19© 2010 ArcSight Confidential
Defining Aggregation
Use to limit the amount of rule firing for repeat events, or to set thresholds that define certain scenarios This specifies number of matches (threshold) in specified
amount of time by the rule Example – five failed login attempts in two minutes may signify
a brute force
Understanding Rule Components
www.arcsight.com 20© 2010 ArcSight Confidential
Advanced Aggregation
There are four time-evaluation criteria that can affect event-occurrence aggregation and rule-triggering
You can apply these to rules through the aggregation tab and the statement panel of the conditions tab1. Time Frame – establishes the time span for occurrence aggregation
• Event-occurrence aggregation is always controlled by time frame 2. Global Expiration – global expiration applies to an entire rule
• This is the amount of time that qualifying events for all aliases will be retained in memory for evaluation and is based on manager receipt-time
3. Alias Expiration – an alias expiration applies to a single alias within a rule
• This is the amount of time that a qualifying event for this alias will be retained in memory for evaluation and is based on manager receipt-time
4. Matching Time – matching time creates a time-proximity comparison for multiple-alias rules and is based on events' actual creation times
Understanding Rule Components
www.arcsight.com 21© 2010 ArcSight Confidential
Now, We Are Ready for the Action!
Once the rule conditions are met and we meet the threshold requirements set in aggregation, it’s time to take action!
When a rule fires, an action will be taken based on the trigger that you set
You can select single or multiple triggers
Why is my rule firing at weird times?
Why is my rule not firing? (lets look a little closer at timing and triggers)
Understanding Rule Components
www.arcsight.com 22© 2010 ArcSight Confidential
Types of Available Rule Actions
A rule can trigger any combination of the following actions Set event field Send to open view operations Send notification Execute command Execute connecter command Export to external system Create new case Add to existing case Add to active list Remove from active list Add to session list Remove from session list
Understanding Rule Components
www.arcsight.com 23© 2010 ArcSight Confidential
Defining Triggers – Rule Action Triggers
Three types of rule action triggers are available1. Event triggers – act on individual events
– On first event– On every event– On subsequent events
2. Threshold triggers – act on groups of events that satisfy the time frame requirements– On first threshold– On every threshold– On subsequent thresholds
3. Timing triggers – act on timing of events– On Time Unit – triggers on a specified unit of time after a threshold is
met– On Time Window Expiration (TWE) – triggers after the time frame
expires without meeting the number of matches requirement
Understanding Rule Components
www.arcsight.com 24© 2010 ArcSight Confidential
60 sec
1 2 3 4 5 Matches 1 2 3 4 5
1st Threshold
Time
First Threshold
Threshold condition: five matches within two minutes Threshold condition reached Action takes place Threshold time window resets
(2 Minutes)
180 sec
1st Threshold
240 sec
Time Reset
Understanding Rule Components
www.arcsight.com 25© 2010 ArcSight Confidential
60 sec
1 2 3 4 5 6 7 8 9 10 11 12 13 14
100 sec
1st Threshold 2nd Threshold
Time
Every Threshold
Threshold condition: five matches within two minutes Every time the threshold is met Action takes place Continues until TWE
TWE
(2 Minutes)
Understanding Rule Components
www.arcsight.com 26© 2010 ArcSight Confidential
60 sec
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
100 sec
1st Threshold (No Action)
Subsequent Threshold
Time
Subsequent Threshold
Threshold condition: five matches within two minutes After the first threshold is met Waits for second threshold to be met Action takes place at subsequent thresholds Continues until TWE
TWE
(2 Minutes)160 sec
Subsequent Threshold
Understanding Rule Components
www.arcsight.com 27© 2010 ArcSight Confidential
60 sec
1 2 3 4 5 6 7 8
120 sec
1st Threshold
TWE
Time
On Time Unit
Threshold condition: five matches within two minutes with a 30 seconds time unit
Initial threshold is met Action takes place every time the time unit elapses Continues to take action until TWE
30 sec Time Unit
30 sec Time Unit
90 sec
Understanding Rule Components
www.arcsight.com 28© 2010 ArcSight Confidential
60 sec
1 2 3 4 5 6 7 8 9
180 sec
1st Threshold
TWETime
(2 Minutes)
Time Window Expiration
Threshold condition: five matches within two minutes Initial threshold is met Waits until TWE Action takes place
Understanding Rule Components
www.arcsight.com 29© 2010 ArcSight Confidential
Correlation Events Created by Rules
What fields to set in correlation events? These are the fields that will be set in the correlation event Make sure you don’t create a feedback loop
(rules firing on themselves)
!Tip: Agent SeverityUse low for informational rulesthat have indirect consequenceUse medium to very high forrules of direct consequence
Understanding Rule Components
www.arcsight.com 30© 2010 ArcSight Confidential
Don’t Break Your Rule with Excessive Actions
What to do when conditions and thresholdshave been met?– Create a new event, create a case, etc
Use “on first event” or “on first threshold” to avoid excessive rule firing due to heavy attack traffic
Understanding Rule Components
www.arcsight.com 31© 2010 ArcSight Confidential
Don’t Break Your Rule with Excessive Actions
To add all rule firings to a single case, use “on subsequent events” A solution to handle long running continuous attacks would be to
define following triggers– On first threshold – will notify start of attack– On time unit – will periodically notify that the attack is still going on– On time window expiration – will notify end of attack
Understanding Rule Components
www.arcsight.com 32© 2010 ArcSight Confidential
Mastering Rules
www.arcsight.com 33© 2010 ArcSight Confidential
Mastering Rules
Know the business conditions, requirements or use case– That’s how you start to build a rule!
Rule development– ½ science (boolean, timing, action definitions, etc)– ½ art, so keep rule’s conditions as simple and precise as possible
Know our ArcSight event SCHEMA– that’s what you have to work with!
(the fields and the output of those fields)
www.arcsight.com 34© 2010 ArcSight Confidential
Create Multiple Simple Rules Instead of One Complex
Break down the use case requirements by listening for key words Define organizations ArcSight network topology
– Network modeling
Track all user logins, from where and to what device – Rule / session List
Track all user logouts, from where and from what device– Rule / session List
Mastering Rules
www.arcsight.com 35© 2010 ArcSight Confidential
Document Your Rules
Long after you’ve forgotten (maybe six months to a year down the road) when you need to review what you were thinking….
A best business practice when developing rules is to DOCUMENT the use case, business requirements and details of how the rule was developed on the NOTES tab
Possible topics to note: Who requested the rule, who are the stakeholders, original date and time of testing vs. deployment, etc
Mastering Rules
www.arcsight.com 36© 2010 ArcSight Confidential
Use Stock Content and Solutions Foundations
ArcSight provides MANY solutions foundation and stock content rules to facilitate out of box functionality upon installation
If you need to get your bearings this is a good place to start
Remember, any rules enabled in the real time rules folder are LIVE
Mastering Rules
www.arcsight.com 37© 2010 ArcSight Confidential
Additional Rule Features
www.arcsight.com 38© 2010 ArcSight Confidential
Additional Rule Features
Automatic rule disabling– ArcSight automatically disables improperly written rules that would
produce excessive or meaningless events
Clearing rule actions– In a grid view, select a correlation event– Right-click and choose “correlation options”– “Clear rule actions” to clear all actions associated with this rule
Showing rule errors– If rules have errors, the rule icon ( ) changes to indicate it– In the rules resource tree, right-click the rule-error icon and choose
“show error”– The error appears in a dialog box
www.arcsight.com 39© 2010 ArcSight Confidential
Automatic Rule Disabling
Rule disabling factor operation– Alias matches – if an alias is defined, this is the number of events
matching that alias and is independent of other defined aliases in the same rule
– Partial matches – if more than one alias is defined, the number of events matching the aliases defined before the current one, and for the current one, and for their join condition (if present)
– Generated events counts – the number of correlation events generated
– Base event counts – number of base events used to generate correlation events
– Time unit counts – number of time units (minutes) that passed since the rule activated
Above values for rule disabling may be adjusted for your enterprise– ArcSight ESM will disable a rule if the rule exceeds the configured
limits on number of rules triggered per minute or the ratio of base events to triggered rules and is defined in server.defaults.properties file on the manager
Additional Rule Features
www.arcsight.com 40© 2010 ArcSight Confidential
Troubleshooting and Tuning
www.arcsight.com 41© 2010 ArcSight Confidential
Troubleshooting and Tuning
Rules: Troubleshooting
What do you do when the “check engine light” comes on in your car?– Apply the same methodology– Break components down into their most basic form
(don’t digest the entire conditions tab, take it one line or maybe one statement at a time)
Is the data your looking for actually available?– Start back at the basics
(RAW logs from the device prior to hitting our connector and being normalized)
Was the rule imported via an ARB?– If so, was it done on the same revision of ArcSight ESM? – Were resource ID’s exported into the ARB?
Has the rule completed? (partial matching rule?)
www.arcsight.com 42© 2010 ArcSight Confidential
My Rule is Broken!
How would I know? What Clues Do I look for?
Troubleshooting and Tuning
www.arcsight.com 43© 2010 ArcSight Confidential
Check your Condition Logic First
What are your rules dependant on?– Active/session lists, asset/network modeling, variables, etc?
How do you know? Check out https://localhost:8443Resource management and rules to look at details:
Troubleshooting and Tuning
www.arcsight.com 44© 2010 ArcSight Confidential
Identifying Attacks
If a rule is defined to identify the following attacks, it will excessively fire: Denial of Service or Distributed Denial of Service AttackIDS / SIM / SIEM / ESM “Smoke Screening” aka…Copperfield/Angel magic
If rule trigger is activated on EVERY EVENT or EVERY THRESHOLD, it may lead to excessive firing
What would this look like?
Troubleshooting and Tuning
www.arcsight.com 45© 2010 ArcSight Confidential
Potential Issues Related with Timing
Timing is very sensitive in rules firing End time field is a key player during correlation phase. Network latency could lead to potential issues during correlation:
Verify start time, end time, agent receipt time and manager receipt time values – 1-2 min off could be an indicator of network latency
Poor bandwidth or high EPS could produce same results Did something “recently” change that could effect the arrival of events
into the connector?Anything more – could trip an exception error "DCERPC pipe is no longer open" reported in server.log – check the following:– Changed behavior of A/V or HIPS which now blocks remote pipes– Changed network behavior after a Patch (those do get tested first, right? ;> )– Has your OS stopped allowing remote pipe comms? (ie…Windows Firewall or
IPTABLES)– Domain Admin recently tightened access policy or net admin threw a new
ACL/rule
Troubleshooting and Tuning
www.arcsight.com 46© 2010 ArcSight Confidential
Troubleshooting and Tuning
Rules: Troubleshooting
Is your rule recursive ? Starting in Arcsight ESM 4.5.1, rules that trigger themselves recursively will Automatically be disabled temporarily, then re-enabled
(aka – rule bouncing) Has your rule trigger exceeded the max. # of correlated alerts per min. limit?
You would see an error as seen below in your server.log file:[2009-07-30 10:21:59,750][ERROR][default.com.arcsight.rulesengine.actionengine.ActionCommandHandler][onSingleEvent] Too many pending actions 1000, not adding more ....– This is set in server.default.properties as:
• #number of correlated alerts per rule per minute• rules.max.fan-out.time-unit.ratio=1000• Remember persisted settings must be set in server.properties
To reduce excessive firing, consider using ON FIRST and TU/TWE triggers Monitor your rules engine via rcsight ESM dashboards or status monitoring web
page
www.arcsight.com 47© 2010 ArcSight Confidential
Limit Partial Match Storage Using Time Constraints
This condition occurs when using join rules and an event matches one alias
Partial matches for a rule are stored in memory for the specified time window
To limit memory consumption– Limit the aggregation time frame– Use active lists to correlate information from events spaced far in time
! Tip: Partial MatchesCan be monitored usingthe “Rules Status” dash-board in ArcSightAdministration
Troubleshooting and Tuning
www.arcsight.com 48© 2010 ArcSight Confidential
Tuning Rules
ArcSight ESM comes with a dashboard that can enable you to view the statistics of the rules within your environmentThe following data monitors are included:– Partial matching– Top firing rules– Recent fired rules– Rules engine internal stats– Rule error logs
Troubleshooting and Tuning
www.arcsight.com 49© 2010 ArcSight Confidential
More Information?
Rules aren’t something we expect you to be a subject matter expert in by attending this workshop or by attending 3-5 day classes
4.5.1 user guide; chapter 13: Rules Authoring 4.5.1 system content reference guide Talk through your rules
– Engineering 101: “If you can’t explain the process, you don’t understand the process”
ArcSight Protect 724– Content sharing and ARB’s– how are your colleagues writing rules?
Review summary for “SQL look” http://en.wikipedia.org/wiki/De_
Morgan's_laws
Troubleshooting and Tuning
www.arcsight.com 50© 2010 ArcSight Confidential
Your Feedback Builds a Better Conference!
Download session replays after the conference:https://protect724.arcsight.com/community/protect10/sessions
Excellent Good Fair Poor
Rate the speaker a b c d
Rate the content e f g h
Please provide comments: (*) enter any comments/feedback
Text to 32075 (USA & Canada) or 447786204951 (Non-USA) Type ARCS <space> 08 and the letter to each response
SMS body example: ARCS 08ae*your comments
www.arcsight.com 51© 2010 ArcSight Confidential
ArcSight, Inc.Corporate Headquarters: 1 888 415 ARST
EMEA Headquarters: +44 (0)844 745 2068Asia Pac Headquarters: +65 6248 4795
www.arcsight.com