Real-Time Intrusion Detection with Emphasis on Insider Attacks

Shambhu Upadhyaya Computer Science and Engineering

University at Buffalo

Polytechnic University

October 3, 2003

CEISARE @2

Some Facts & Figures (CSI/FBI 03)

CEISARE @3

Source of Attack

CEISARE @4

Attack Types

CEISARE @5

Actions Taken

CEISARE @6

What Could be Learned From It?

Good prevention techniques must be in place

Good policies must be set up

Need to know what is important

Need to know the application environment

IDS is a must

But there is no IDS that is applicable to all environments

CEISARE @7

Outline of the Talk

General introduction

Evolution of IDS

Major players

Insider threats and how to mitigate?

Conclusion

CEISARE @8

Outline of the Talk

General introduction

Evolution of IDS

Major players

Insider Threats and how to mitigate?

Conclusion

CEISARE @9

What is an IDS?

In its general sense –

Acquires information about its environment to analyze

system behavior

Aims to discover security breaches, attempted breaches,

open vulnerabilities that could lead to potential breaches

Types of information –

Long term info. – a knowledge base of attacks (static)

Configuration info. – a model of the current state (static)

Audit info. – describing the events happening (dynamic)

CEISARE @10

IDS Architecture(Macroscopic View)

System Model ofSystem

Analyzer Visual Presentation

Database,Storage

DATA

Slide adopted from UCDavis, Jeff Rowe

CEISARE @11

IDS Side-effects

False negatives (failed detection)

poor coverage

False positives (wrong indictment)

poor QOS

Degrade normal operation

poor performance

CEISARE @12

Outline of the Talk

General introduction

Evolution of IDS

Major players

Insider Threats and how to mitigate?

Conclusion

CEISARE @13

Evolution of IDS

Paul Innella’s timeline:

CEISARE @14

Current State-of-the-art

1st generation tools are largely signature based

Security is by penetrate and patch

Today’s focus is on detecting novel intrusions

New techniques must consider insider attacks, social

engineering based break-ins etc.,

Need for new paradigms – Design for Security?

New ideas –

Combining IDS with vulnerability analysis

Detection is not fool-proof; must be merged with recovery

CEISARE @15

Outline of the Talk

General introduction

Evolution of IDS

Major players

Insider Threats and how to mitigate?

Conclusion

CEISARE @16

Major Players – Academia Purdue –

CERIAS

UC Davis –

Developed GrIDS (Graph based IDS)

CMU – Home of CERT/CC

Cornell

Language-based security

Columbia

IDS and Data mining

Above list is incomplete

CEISARE @17

Major Players – Industries

IBM Watson

Global Security Analysis Laboratory

Microsoft

Started the Trustworthy Computing initiative in 2002

Cisco

Does research and development

Builds intrusion detection appliances – sensors and software

MAFTIA

European Union of academia and industries

CEISARE @18

Major Players – Labs/Government SRI International –

Developer of EMERALD through funds from ITO, DARPA

Air Force Research Lab –

Defensive Information Warfare Branch

Naval Research Lab –

Center for High Assurance Computer Systems

Multi-level security

National Institute of Standards and Technology –

Computer Security Resource Center

National Security Agency –

Research and education

CEISARE @19

Popular Websites

SANS (System Administration, Networking and Security) Institute

http://www.sans.org/aboutsans.php

CERT/CC

http://www.cert.org/

CERIAS (Center for Education and Research in Information

Assurance and Security)

http://www.cerias.purdue.edu/

NIST (National Institute of Standards and Tech.)

http://csrc.nist.gov/index.html

CEISARE @20

IDS Tools List Mike Sobirey (copyright: Dr. Michael Sobirey)

List of ID Tools from 1995-2000

92 host- and network based Intrusion Detection (&

Response) Systems

Additions are appreciated

NIST Intrusion Detection Tools

Coverage is only up to 1996 (not up-to-date)

About 20+ tools listed

The above two lists have little overlap (cover >110)

CEISARE @21

Recent Releases Responsible for real-time packet capture and analysis (http://www.prelude-ids.org/) on Linux/Unix

Prelude platforms

Portsentry – An IDS that detects and responds to port scans against a target host in real-

time (http://www.psionic.com/products/)

SPADE – Statistical Packet Anomaly Detection Engine (http://www.silicondefense.com/)

inspects recorded data for anomalous behavior based on a computer score

Stealthwatch (Lancope), Stormwatch (Okena)

Stackguard – Protects from stack smashing attacks (

http://www.cse.ogi.edu/DISC/projects/immunix/StackGuard/)

Netscreen -- http://www.netscreen.com/products/

There is no tool that is universally applicable

CEISARE @22

Outline of the Talk

General introduction

Evolution of IDS

Major players

Insider Threats and how to mitigate?

Conclusion

CEISARE @23

Who is an Insider?

CEISARE @24

How to Deal with the Problem?

We focus on the detection only

Model the Insider

Prevention of Insider Misuse

Detection, Analysis and Identification of Misuse

CEISARE @25

IDS with Emphasis on Insider

Current systems are signature-based and they

use audit-trail or rule-based protection

Not effective for insider attack detection

Anomaly detection is applicable, but not very

effective

New theory needed, proactive mechanisms

needed

CEISARE @26

Guidelines for Effective Anomaly Detection

Use the principle of least privilege to achieve better

security

Use mandatory access control wherever appropriate

Data used for intrusion detection should be kept

simple and small

Intrusion detection capabilities are enhanced if

environment specific factors are taken into account

CEISARE @27

Our Approach An out-of-the-box Reasoning Framework for intrusion

detection

Technique used:

Control flow checking from FT (basis for encapsulation of

owner’s intent)

Reasoning based on Theory of Risk Analysis from

Economics

Problem is similar to Pricing Under Uncertainty S. Upadhyaya, R. Chinchani, K. Kwiat, “An Analytical Framework for Reasoning about Intrusions”,

IEEE SRDS 2001

CEISARE @28

User Intent Encapsulation

Obtain the intent of the

user either by inference

or query

Session scope serves as

a certificate

Reduces the search

space during monitoring

CEISARE @29

Illustration of Search Space Reduction

Kernel

Resources

Commands and System

calls

Audit data

reduces as we go

higher up Typical audit data

User

CEISARE @30

InterfaceUser

Sequence of Operations

Resource

Disk Network

CPU Memory

System

Overall Layout of System Operation

CEISARE @31

Expected Sequences

Certain “normal” ways of doing a job

Also, certain “less normal” ways of doing them

A job is completed by performing a sequence of

operations

May not be possible to enumerate all the

sequences

CEISARE @32

Cost Analysis

Cost of Operation = Co

Proportional to the amount of resources used

Cost of Sequence = Cd

Proportional to the difference between current

chosen operation and past history

Cost of Job = *Co + *Cd

CEISARE @33

Job Activity Stochastic

At any stage, a user “chooses an operation”

with a probability

“Choice of an operation” is a random variable

Sequences of operations construct a discrete

stochastic process

CEISARE @34

User Activity as a Martingale

Theorem:

Let the lateral sequence of random

variables for any state i of a sequence of

operations be denoted as:

X1(ti, ), X2(ti, ), … Xn(ti, )

Such a sequence of user activity is a

Martingale

CEISARE @

An Example

nfrm pine exit nop nop

pine ls exit nop nop

mail finger nfrm pine exit

nfrm pine finger exit nop

(nfrm, pine, ls, mail, exit, finger, nop)

CEISARE @36

A Note on Martingale

Martingale uses concepts of conditional probability and

has applications in economics

Model is used to predict market parameters like a share of

a stock

Future price of a commodity depends only on the last

known distribution and not on the entire history of the

prices

There is a parallel between uncertainties in intrusion

detection and the concept of pricing under uncertainty

CEISARE @37

Reasoning

Non-intrusive

Non-deterministic

Intrusive

Monotonically increasing costs

ThTl

CEISARE @38

Cost Scenarios

Low Co + Low Cd

Non-intrusive

Maps into the non-intrusive region

High Co + Low Cd

Intrusive and tending toward a DoS attack on

resources

Maps into the non-deterministic region

CEISARE @39

…contd.

Low Co + High Cd

The intruder??

Maps into the non-deterministic region

High Co + High Cd

The clumsy attack

Maps into the intrusive region

CEISARE @40

Quantification of Thresholds

Threshold Tl

Minimum cost over longest sequence

Threshold Th

Maximum cost over shortest sequence

CEISARE @41

Algorithm: INIT_DISTR(Generates the initial distribution)

Enumerate all possible

sequences

Find the longest sequence

Create a discrete stochastic

process

Generate probabilities at

each stage and shape the

distribution

CEISARE @42

Algorithm: MODIFY_DISTR(Modifies the existing distribution)

Check to see at each stage of the

sequence if the user is conforming to the

profile

At the job termination, if the sequence is

not intrusive, update the frequency

distributions and probabilities

CEISARE @43

Algorithm: DECIDE(Makes a decision in the non-deterministic region)

Calculate the longest sequence from

current stage to complete the job. Move Tl

to that position

The window (Th – Tl) depends on the

gradient of the cost accumulated since

DECIDE was last invoked

CEISARE @44

Sketch of the Overall Algorithm User logs into the system

Chooses the job s/he wishes to performCheck the size of the session scope

If too large,warn userUser wants to change it

Launch inter work-space level monitor

Create workspaces for the jobs

Launch workspace level monitor thread per workspaceLaunch command level monitor thread per command

Authenticate command

Monitor Command

YES

LoopReport command type

Report object accessed

CEISARE @45

Preliminary Implementation

Developed in Java on Solaris 2.8

A university environment was simulated

Monitoring at basic command level

Limited sequence monitoring

Not many scenarios

Perhaps, not realistic for actual deployment

CEISARE @46

Test Cases

User activity collected over two months

Test cases grouped into four categories

1-user, 1-user with multiple logins, multiple users, multiple users

with multiple logins

Two sets of experiments – worst case and average case

Legitimate and intrusive operations

32 attacks

Obvious ones such as transferring /etc/passwd files, exploiting

vulnerabilities such as rdist, perl 5.0.1

Subtle attacks similar to mimicry attacks

CEISARE @47

Summary of Results

Summary 1 User, No Multiple Logins 1 User, With Multiple Logins 2 Users, No Multiple Logins 2 Users, With Multiple LoginsUser Detection 87.50% 78.60% 74.90% 91.90%and Latency 33.4 35 36.1 29User False Positives 12.50% 21.40% 25.10% 8.10%

False Negatives 0% 0% 0% 0%User Detection 98% 89% 100% 94.70%and Latency 0 11 0 9.6

Intruder False Positives 0% 0% 0% 0%False Negatives 2% 11% 0% 5.30%

Intruder Detection 99% 100% 98.20% 100%and Latency 0.4 0.7 0.6 0.5User False Positives 0% 0% 0% 0%

False Negatives 1.40% 0% 1.80% 0%Intruder Detection 56% 81.30% 77.40% 91.50%

and Latency 15.9 14.8 17 27Intruder False Positives 0% 0% 0% 0%

False Negatives 44% 18.70% 22.60% 8.50%

CEISARE @48

Types of Detected Intrusions

It can detect internal attacks -

A cracker logs in and executes commands

Inadvertent operator faults

Internal abuse

External attacks -

Masquerading

Subversion attacks by presenting overly

permissive session-scope (penalty in terms of

reduced QoS)

CEISARE @49

Undetected Intrusive Activity

It cannot contain or detect -

External Denial of Service attacks

Extremely low-level network based attacks

CEISARE @50

Ongoing Research

Addressing outstanding issues like

State explosions due to partial orderings

Scalability

Values of α, β , ??

A more realistic prototype implementation and

testing

Project is currently funded by DARPA

CEISARE @51

Concluding Remarks – Vision Insider threat is very much real

Penetrate and Patch method is not adequate

CMU and other studies show current IDS are not effective

Anomaly detection schemes, that are environment-independent

may be in the focus

Monitoring at user command level has distinct advantages

Conceptually independent of systems and applications

As the no. of threats grows, IDS will become a required element

of system security

CEISARE @52

Concluding Remarks – Research IDS and vulnerability analysis

Effective means of system evaluation

Good metrics for performance, coverage etc.

Return on investment studies

Merge IDS with firewalls

Merge IDS with recovery

It is not possible to detect all intrusions

Protection against unknown threats – Proactive mechanisms

Rapid Incident Response