Upload
others
View
75
Download
1
Embed Size (px)
Citation preview
© 2009 ArcSight, Inc. All rights reserved. ArcSight and the ArcSight logo are trademarks of ArcSight, Inc. All other product and company names may be trademarks or registered trademarks of their respective owners.
Deep Dive Into ArcSight ESM Rules
Rob BlockSr. Software Engineer, Correlation TeamSeptember 2009
Overview
Swim– Rule Basics– Rule Matching Overview– Rule Chains
Dive– Rule Partial Matching– Rule Engine Details– Aggregation Engine Details
Deep dive– Excessive Rule Firing– Negated Aliases– Batched Replay and Schedules
www.arcsight.com © 2009 ArcSight Confidentialwww.arcsight.com © 2009 ArcSight Confidential 2
Let’s Swim First
Overview of rules
Overview of rule matching
Rule chains
www.arcsight.com © 2009 ArcSight Confidential 3
ArcSight Rules
Evaluate incoming events for specific conditions and patterns
Correlate information from different events using – Rule correlation – Active lists– Session lists– Threat level calculations
Infer meaning about significance of events, and initiate real-time actions in response
www.arcsight.com © 2009 ArcSight Confidential 4
Correlation EventFields• Type = Correlated• Name = Name of the Rule• File Path = URI of the Rule… +
• Fields set using SetEventField Action +
• Fields used in Aggregation
Structure of a Rule
Condition Matching
Events Matches Execute Actions
On Qualifying Triggers
Audit Events For ActionsFields• Type = Action• Name = Type of Action + Success/Failure
• File Path: URI of the Rule• Device Custom String 4: Trigger Involved
GeneratesRule Chain
Aggregation
www.arcsight.com © 2009 ArcSight Confidential 5
Condition Matching
Conditions
Correlation EventFields• Type = Correlated• Name = Name of the Rule• File Path = URI of the Rule… +
• Fields set using SetEventField Action +
• Fields used in Aggregation
Events Matches Execute Actions
On Qualifying Triggers
Audit Events For ActionsFields• Type = Action• Name = Type of Action + Success/Failure
• File Path: URI of the Rule• Device Custom String 4: Trigger Involved
GeneratesRule Chain
Aggregation
Structure of a Rule
www.arcsight.com © 2009 ArcSight Confidential 6
Conditions
Rule can have multiple sets of conditions, each matching one event
Example: rule with two sets of conditions, matching events– Brute force login attempt– Successful login
Join conditions correlate the aliases themselves; both events came from same attacker
Structure of a Rule
www.arcsight.com © 2009 ArcSight Confidential 7
Aggregation
Events Matches Execute Actions
On Qualifying TriggersAggregation
Specifies number of matches (threshold) needed in specified time frame
Can aggregate based on identical and/or unique fields
Attack is in progress as long as threshold has been met within latest time frame period
Structure of a Rule
Condition Matching
www.arcsight.com © 2009 ArcSight Confidential 8
Triggers
Structure of a Rule
Condition Matching
Events Matches Execute Actions
On Qualifying TriggersAggregation
Trigger typesOn First EventOn Subsequent EventOn Every EventOn First ThresholdOn Subsequent ThresholdOn Every ThresholdOn Time UnitOn Time Window Expiration
www.arcsight.com © 2009 ArcSight Confidential 9
Actions
Structure of a Rule
Condition Matching
Events Matches Execute Actions
On Qualifying Triggers
Action types
Set event fieldSend notificationCreate new caseAdd to caseExecute a commandExecute a connector commandAdd to active listRemove from active listAdd to session listTerminate session
Aggregation
www.arcsight.com © 2009 ArcSight Confidential 10
Audit Events and Correlation Events
Correlation EventFields• Type = Correlated• Name = Name of the Rule• File Path = URI of the Rule… +
• Fields set using SetEventField Action +
• Fields used in Aggregation
Condition Matching
Events Matches Execute Actions
On Qualifying Triggers
Audit Events For ActionsFields• Type = Action• Name = Type of Action + Success/Failure
• File Path: URI of the Rule• Device Custom String 4: Trigger Involved
GeneratesRule Chain
Aggregation
Structure of a Rule
www.arcsight.com © 2009 ArcSight Confidential 11
Rule Chain
Correlation EventFields• Type = Correlated• Name = Name of the Rule• File Path = URI of the Rule… +
• Fields set using SetEventField Action +
• Fields used in Aggregation
Condition Matching
Events Matches Execute Actions
On Qualifying Triggers
Audit Events For ActionsFields• Type = Action• Name = Type of Action + Success/Failure
• File Path: URI of the Rule• Device Custom String 4: Trigger Involved
GeneratesRule Chain
Aggregation
Structure of a Rule
www.arcsight.com © 2009 ArcSight Confidential 12
Rule Chain
Base events corresponding to the rule trigger
Rule chain is not cumulative – Exception: option for on time window expiration
Example: threshold = 3, on every threshold trigger– 3 base events in rule chain for each OET firing
Structure of a Rule
www.arcsight.com © 2009 ArcSight Confidential 13
Rule chain includes the events that contributed to the particular firing of the rule
Consider a rule, with threshold = 3, aggregation time window = 1 minute, and following triggers activated– On first threshold– On time unit (every minute)– On time window expiry
Note: on time unit rule chain only includes events after the first threshold
Rule Chain
12:01 12:02 12:03
First Threshold On Time Unit
12:04
Time Window Expiry
Structure of a Rule
Matching Events
www.arcsight.com © 2009 ArcSight Confidential 14
Let’s Dive
Rule partial matching
Rule engine details
Aggregation engine details
www.arcsight.com © 2009 ArcSight Confidential 15
Rule Partial Matching
Two types of rules– Filter rule: contains single alias– Join rule: contains multiple aliases
An event matching a rule alias generates either a partial match or a full match for the rule, depending on the number of rule aliases
A single event usually matches one rule alias
www.arcsight.com © 2009 ArcSight Confidential 16
Example: Filter Rule
Multiple failed logins on Windows systems
Multiple failed logins on UNIX systems
5 or more failed logins in a minute from same source
on same target
Attempted Brute Force Attack
Attempted Brute Force Attack
Rule Partial Matching
www.arcsight.com © 2009 ArcSight Confidential 17
Example: Join Rule
Correlates two or more different kind of events
Attempted Brute Force Attack + Successful
Login
Attempted Brute Force Attack
Successfullogin to Target system
Rule Partial Matching
www.arcsight.com © 2009 ArcSight Confidential 18
Join Rules and Memory Usage
Join rules usually require more memory than filter rules, due to partial match maintenance
Partial matches– Stored in memory for the specified time window, waiting for
complimentary events– Only minimal event information is stored in the partial match, needed
for join conditions and aggregation
Matching time window for these rules should not be kept too long– Use active lists to correlate information from events spaced far in time
Rule Partial Matching
www.arcsight.com © 2009 ArcSight Confidential 19
Time Constraints in Rules
Aggregation time: time period to use for aggregation (to wait for specified number of matches)
Alias expiration time: how long partially matched events are kept in memory
Rule Partial Matching
www.arcsight.com © 2009 ArcSight Confidential 20
Rule Engine
Rule engine matches incoming security events
against the deployed rules
!
www.arcsight.com © 2009 ArcSight Confidential 21
Rule Engine Structure
Rule Evaluation
Rules
Working Memory(Events)
Relevant Security Events
Rules Engine
Insert
Matches AggregationEngine
Garbage Collector
www.arcsight.com © 2009 ArcSight Confidential 22
Working Memory
Match
Garbage CollectorSuccessful
Login Attempt event2event2
Other Events Matching Filter Rules Eventsevents
Attempted Brute Force Attack Event
event1Insert
AggregationEngine
event1
Rule Evaluation
Rules
Relevant Security Events
Rules Engine
Matches
Rule Engine Example
ExpiredExpiredExpiredExpired
www.arcsight.com © 2009 ArcSight Confidential 23
Rule Aggregation
Recognizes patterns involving repetitive events– Example: five failed logins
Has impact on memory as aggregation matches are counted and tracked
Aggregation cell: set of matching events satisfying the aggregation criteria (identical/unique field values)
Large # of aggregation cells increases memory usage
Once time window expires, inactive matches are cleared
One aggregation tracker per rule
www.arcsight.com © 2009 ArcSight Confidential 24
insert
Aggregation Engine Structure
Matches from Rules Engine
Rules TriggeredAggregation Cells (set of identical
/unique field values)
Matches are added to an aggregation cell based upon values of the aggregated fields
Thresholds Calculated (per aggregation cell)
Aggregation Cell Identified
www.arcsight.com © 2009 ArcSight Confidential 25
AttackerAddress: 192.168.1.5 AttackerZone: InternalZone1
Insert
Matches from Rules Engine
Rules TriggeredAggregation Cells Thresholds Calculated (per aggregation cell)
Aggregation Cell Identified
Aggregation Engine ExampleRule matching 2 failed logins in a minute from same source
AttackerAddress: 192.168.1.2 AttackerZone: InternalZone1
192.168.1.2, InternalZone1
Count = 1
192.168.1.5, InternalZone1
Count = 1
Expire after a minute
192.168.1.5, InternalZone1
Count = 2
Matches are added to an aggregation cell based upon values of the aggregated fields
www.arcsight.com © 2009 ArcSight Confidential 26
Aggregate Only When Unique Option
Allows capturing the cases of widespread problems
Example– A rule to identify widespread computer virus in a corporation– Rule may be written to fire when the virus notifications received from at
least 10 different machines– Here we are interested in events from unique machines
www.arcsight.com © 2009 ArcSight Confidential 27
Apart from more commonly used aggregation on matching values, aggregation can also be done on unique set of values
Deep Dive
Excessive rule firing—understand and avoid
Negated aliases
Batched replay and schedules
www.arcsight.com © 2009 ArcSight Confidential 28
Excessive Rule Firing or Partial Matches
Rule having too relaxed conditions
Triggers not defined judiciously
Single rule recursion– Correlated alerts lead to further firing of the rule
Multiple rule recursion– A set of rules form a recursive cycle
and lead to mutual firing of rules
www.arcsight.com © 2009 ArcSight Confidential 29
Excessive Rule Firing
Excessive rule firing and rule recursion is identified and reported to users using audit events
In excessive rule firing– Rule is temporarily deactivated– Activated again after elapse of time equivalent to rule
aggregation time
In case of rule recursion, the events causing recursion are cut off from the recursion loop
www.arcsight.com © 2009 ArcSight Confidential 30
Example: Excessive Rule Firing
Denial of service attack
An attacker can flood a server with traffic
There may be a rule defined to identify such attacks
If rule trigger is activated for every event or on every threshold it may lead to excessive rule firing
Can make it difficult looking at other rule firings in the channel
www.arcsight.com © 2009 ArcSight Confidential 31
Excessive Rule Firing
On first threshold: will notify start of attackOn time unit: will periodically notify that the attack is still
going onOn time window expiration: will notify end of attack
A solution to handle long running continuous attacks would be to define following triggers
www.arcsight.com © 2009 ArcSight Confidential 32
Negated Aliases
Consider a rule R1 with two aliases E1, E2– E1: matches a server console login event, for a server inside the
server room– E2 (negated): matches a badge scan event to enter the server room
Badge scan event should happen before the server console login event, as the person has to be present in the server room to login
Rule will fire if someone logged on to a console in the server room, without scanning the badge to get inside the room
www.arcsight.com © 2009 ArcSight Confidential 33
Use of negated aliases in join rules provide ability to take action on missing events
Using Negated Aliases to Find Absence of an Event in the Future
It cannot look for missing event in future
Consider a rule R1 with two aliases E1, E2– E1: matches a server reboot event– E2 (negated): matches a server up event
This rule will always fire on receiving a server reboot event, as server up event hasn’t arrived at that point (server up event will happen in future)
www.arcsight.com © 2009 ArcSight Confidential 34
Rules using negated aliases evaluate the absence of the negated event, at the time rule is evaluated
Using Negated Aliases to Find Absence of an Event in the Future
Rule 1 Rule 2 with two aliases E1, E2
• Matches the server reboot event• Trigger defined on (on time window expiration), with suitable aggregation time window (ex: 4 minutes)
• E1: matches Rule 1 event• E2 (negated): matches a server up event
Now Rule 2 will be evaluated when Rule1 fires, which had been time delayed due to on time window expiration trigger
This case can be handled using 2 rules, so as to introduce delay in the event of a matching positive alias
www.arcsight.com © 2009 ArcSight Confidential 35
Batched Replay with Rules
Use case: badge reader events sent to manager in batch once a day
Replay task runs in own rules engine
Events queried from DB– Not all event fields are queried– Essential fields plus fields needed to compute conditions, variables,
aggregation and actions– Use event filter to improve performance
Conditions like InActiveList may fail due to expired list entries
www.arcsight.com © 2009 ArcSight Confidential 36
Rules can process either real-time events, or historical events in batched replay mode
Scheduled Rules Replay
Schedule rule group to run with specified frequency
Reads events with end time in corresponding time period
Can create multiple schedules for same rule group– May get multiple rule firings for same historical events
Can specify time delay between data cutoff and schedule start time– Server property: rules.batched.time.delay (in ms)
9/14 2:00 9/15 2:00 9/15 2:30
Badge reader events9/14 2:00 – 9/15 2:00
Schedule starts
9/15 3:00
Arrive at Manager
Delay = 60x60x1000 ms
End Time query range
Batched Replay with Rules
www.arcsight.com © 2009 ArcSight Confidential 37
Summary
Rule Matching Details
Rule Aggregation Details
Memory Aspects of Rule Matching and Aggregation
Rule Chains
Avoiding Excessive Rule Firings
Use of Negated Aliases
Batched Replay and Schedules
www.arcsight.com © 2009 ArcSight Confidential 38