Alarm management for intrusion detection systems - · PDF fileAbstract Intrusion detection systems (IDSs) are important tools helping the network and system administrators to detect

University of Gothenburg Chalmers University of Technology Department of Computer Science and Engineering Göteborg, Sweden, December 2010

Alarm management for intrusion detection systems Prioritizing and presenting alarms from intrusion detection systems.

Master of Science Thesis, Computer Science Programme

SEBASTIAN KLÜFT EVA LINA STAAF

The Authors grant to Chalmers University of Technology and University of Gothenburg the non-exclusive right to publish the Work electronically and in a non-commercial purpose make it accessible on the Internet. The Authors warrant that they are the authors to the Work, and warrant that the Work does not contain text, pictures or other material that violates copyright law. The Authors shall, when transferring the rights of the Work to a third party (for example a publisher or a company), acknowledge the third party about this agreement. If the Authors have signed a copyright agreement with a third party regarding the Work, the Authors warrant hereby that they have obtained any necessary permission from this third party to let Chalmers University of Technology and University of Gothenburg store the Work electronically and make it accessible on the Internet.

Alarm management for intrusion detection systems

Prioritizing and presenting alarms from intrusion detection systems. Sebastian Klüft Eva Lina Staaf © Sebastian Klüft, December 2010. © Eva Lina Staaf, December 2010. Examiner: Erland Jonsson Supervisor: Magnus Almgren University of Gothenburg Chalmers University of Technology Department of Computer Science and Engineering SE-412 96 Göteborg Sweden Telephone + 46 (0)31-772 1000 Department of Computer Science and Engineering Göteborg, Sweden December 2010

This page has intentionally been left blank.

3

Abstract

Intrusion detection systems (IDSs) are important tools helping the network andsystem administrators to detect intrusions, but have the drawback of many falsepositives. Due to increasing bandwidth, an IDS must process a vast amount ofdata, which results in an ever increasing amount of alarms. For a system ad-ministrator to be able to handle the alarms they must be aggregated, correlatedand ordered into a manageable form and presented in a way which is easy tooverview.

In this thesis we study aggregation, correlation, filtering and ranking as methodsfor managing alarms from IDSs. We have implemented a ranking functionalityin the graphical user interface Snorby, a front end to the open source IDS Snort.Each alarm starts with a basic rank of 0 and the user is able to prioritize ordown prioritize the alarm by pressing either a ’+’ button or a ’-’ button, thusinfluencing its current rank. The rank is calculated from several features, i.e.source IP, destination IP, destination port and alarm signature.

Based on our studies we suggest that ranking systems supported by user voteshave several advantages. First, they allow the user to dynamically change theway the IDS lists the alarms through a very simple means. Second, it shortensthe time required to locate the more important ones, thus reducing the likelihoodthat a serious attack will be missed.

Keywords: intrusion detection, IDS, correlation, fusion, aggregation, filtering,ranking, alarm management

Sammanfattning

Intrangsdetekteringssystem (IDS) ar ett viktigt verktyg som hjalper natverks-och systemadministratorer att upptacka och forebygga intrang, men har en nack-del i manga falsklarm. Pa grund av den vaxande bandbredden maste ett IDSbearbeta en stor mangd data, vilket resulterar i ett standigt okande antal larm.For att en natverks- eller systemadministrator ska ha en mojlighet att bearbetadetta maste larmen aggregeras, korreleras och ordnas till en hanterbar form ochpresenteras pa ett satt som ar latt att ta till sig.

I den har rapporten studerar vi agregering, korrelering, filtrering och rankningsom metoder for att hantera alarm fran ett IDS. Vi har implementerat ett rank-ningssystem i det grafiska granssnittet till Snorby, som ar ett web-granssnitttill Snort. Varje alarm borjar pa rank 0 som anvandaren av systemet sedankan hoja eller sanka med hjalp av tva knappar: ’+’ och ’-’. Varje alarms prio-ritet baseras pa fyra egenskaper: kall-IP, destinations-IP, destinations-port ochalarmsignatur.

Baserat pa vad vi har kommit fram till i vart arbete kan vi dra slutsatsenatt ett system som rankar alarm baserat pa hur anvandarna rostar har flerafordelar. For det forsta later det anvandaren dynamiskt paverka hur IDS:etlistar alarmen, genom valdigt enkla medel. For det andra minskas tiden dettar att hitta de viktiga alarmen, och minskar saledes risken for att en allvarligattack ska forbises.

2

Preface

First of all, this thesis would not have been possible without our mentor andguiding star Magnus Almgren. We are deeply grateful and awed by the effortyou have put into this. The Energizer bunny doesn’t hold a candle to you.

We would also like to thank the following people:

Hakan Nohre, security expert at Cisco, who although being very busy stilltook the time to answer our questions and giving us an interesting tour throughthe different intrusion detection systems from Cisco.

Pierre Kleberger for giving us figures of IDS systems and helping us whenwe had trouble installing BASE.

Emilie Lundin Barse and Ola Soderstrom for giving us some insights intothe world of being an IDS administrator and answering our questions in person.Also an extra thanks to Ola for providing us with some statistics we could useto run our simulations.

Ulf Larson for answering our questions in an entertaining way and giving usa lot of ideas to think about.

Erland Jonsson and Wolfgang John for attending our presentations, givingcritique, and asking relevant questions that helped us find a better focus in ourwork.

Eva Lina wishes to thank her own little cookie monster, spiritual alka-seltzerand gifted toe stepper, Tomas Olovsson, for not asking too often how thisproject was coming along and for knowing who Carl Barks is. Also, thanks forenlightening and entertaining conversations and being an all around nice guy.

3

4

Contents

List of Definitions 11

1 Introduction 131.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131.2 Problem description . . . . . . . . . . . . . . . . . . . . . . . . . 141.3 Goal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141.4 Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141.5 Document Organization . . . . . . . . . . . . . . . . . . . . . . . 14

2 Intrusion Detection Systems 152.1 General Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

2.1.1 Placement . . . . . . . . . . . . . . . . . . . . . . . . . . . 162.1.2 Detection Schemes . . . . . . . . . . . . . . . . . . . . . . 192.1.3 Key Concepts . . . . . . . . . . . . . . . . . . . . . . . . . 202.1.4 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . 21

2.2 Available systems . . . . . . . . . . . . . . . . . . . . . . . . . . . 212.2.1 Snort . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222.2.2 BASE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232.2.3 Snorby . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242.2.4 Cisco MARS . . . . . . . . . . . . . . . . . . . . . . . . . 26

3 Previous Work 273.1 Aggregation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273.2 Correlation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

3.2.1 Using conditions . . . . . . . . . . . . . . . . . . . . . . . 293.2.2 Using probabilities . . . . . . . . . . . . . . . . . . . . . . 30

3.3 Finding the normal . . . . . . . . . . . . . . . . . . . . . . . . . . 303.4 Ranking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 313.5 User input . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

4 Methodology 354.1 How to implement ranking . . . . . . . . . . . . . . . . . . . . . . 364.2 Time based weighing of votes . . . . . . . . . . . . . . . . . . . . 384.3 Implementation of whole system . . . . . . . . . . . . . . . . . . 41

4.3.1 Snort database structure . . . . . . . . . . . . . . . . . . . 424.3.2 How votes are counted . . . . . . . . . . . . . . . . . . . . 444.3.3 Ruby on Rails . . . . . . . . . . . . . . . . . . . . . . . . 454.3.4 Counting Votes . . . . . . . . . . . . . . . . . . . . . . . . 47

5

CONTENTS

4.3.5 Displaying the alarms . . . . . . . . . . . . . . . . . . . . 49

5 Simulations 515.1 Alarm generation . . . . . . . . . . . . . . . . . . . . . . . . . . . 515.2 Simulating clicks . . . . . . . . . . . . . . . . . . . . . . . . . . . 525.3 Details of configuration . . . . . . . . . . . . . . . . . . . . . . . 53

6 Results 576.1 Simulations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

6.1.1 Density . . . . . . . . . . . . . . . . . . . . . . . . . . . . 586.1.2 Ordering . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

6.2 Snorby . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

7 Discussion 677.1 Ranking system . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

7.1.1 Issues outside of the system . . . . . . . . . . . . . . . . . 68

8 Future Work and Conclusions 698.1 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 698.2 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70

Bibliography 72

A Box Plots 77

B Specification of Requirements 79

C Source Code 81C.1 click test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81

D Interviews – Q&A 87D.1 Hakan Nohre . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87D.2 Ola Soderstrom . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89D.3 Emilie Lundin Barse . . . . . . . . . . . . . . . . . . . . . . . . . 100D.4 Ulf Larson . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112

6

List of Figures

2.1 Architecture of a generic IDS [25]. . . . . . . . . . . . . . . . . . 162.2 Architecture of a NIDS. . . . . . . . . . . . . . . . . . . . . . . . 172.3 Architecture of a HIDS. . . . . . . . . . . . . . . . . . . . . . . . 182.4 Architecture of a DIDS. . . . . . . . . . . . . . . . . . . . . . . . 182.5 BASE home screen. . . . . . . . . . . . . . . . . . . . . . . . . . 242.6 Snorby . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

3.1 The Bayesian network used when calculating the final rank scorefor alarms in the M-correlator by Porras et al. [30]. . . . . . . . . 33

4.1 Conceptual design of check boxes for interaction with the rankingsystem. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

4.2 The thumbs up button and the thumbs down button to press forliking respectively disliking a video on YouTube. . . . . . . . . . 37

4.3 Final design of buttons for negative and positive feedback. “+”is for positive and “-” is for negative. . . . . . . . . . . . . . . . . 38

4.4 Figure illustrating how voting measurements can fluctuate overtime. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

4.5 Plotted curves showing the effect of using different smoothingfactors. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

4.6 Figure showing how fast previous observations lose their influencein the EMA formula with three different weights. X-axis is thenumber of time periods since the observation occurred, and theY-axis is the influence in percent that the observation has overthe EMA value. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

4.7 The Snorby database structure. This figure does not show thewhole database structure, but only the parts that are affected byour implementation of the ranking model. Items marked in redindicate tables and values added to the already existing databasestructure. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

4.8 The event view in Snorby showing events fetched from the Snortdatabase sorted in the order they where captured. . . . . . . . . 48

5.1 An example showing how the score for an alarm is calculated. . . 53

6.1 Density of all ten test runs and the average after ranking. . . . . 596.2 Density of all ten test runs and the average before ranking. . . . 606.3 Ordering of alarms of test run two. . . . . . . . . . . . . . . . . . 61

7

LIST OF FIGURES

6.4 Ordering of alarms of test run four. . . . . . . . . . . . . . . . . . 616.5 Ordering of alarms of test run seven. . . . . . . . . . . . . . . . . 626.6 Intervals of the twelve most important alarms. . . . . . . . . . . 636.7 Unmodified version of Snorby displaying the event view. . . . . . 646.9 Three links that allow the user to choose how alarms should be

sorted. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 646.8 The buttons that where added to the Snorby interface. . . . . . . 64

A.1 A generic box plot. . . . . . . . . . . . . . . . . . . . . . . . . . . 77

8

List of Tables

6.1 Number of important alarms generated each test run. . . . . . . 576.2 The IPs and alarm types chosen as important. Whenever an

alarm shows up carrying any of these attributes, it gets a simu-lated click. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

6.3 Density of important alarms listed after i positions. . . . . . . . . 59

9

LIST OF TABLES

10

List of Definitions

Definition 1. Aggregation is the process of grouping alarms together basedon certain user-defined criteria found in alarms. (p. 27)

Definition 2. Correlation is the process of building data models from alarmsusing external knowledge not found within the information carried by the alarmsthemselves. (p. 28)

Definition 3. An important alarm, also referred to as a clicked alarm,is an alarm with attributes of interest to the user. When the user spots thisalarm, she is surmised to click on it, showing her interest. (p. 58)

Definition 4. An unimportant alarm, sometimes referred to as an unclickedalarm, is an alarm lacking attributes of interest to the user. When the userspots this alarm, she is surmised to leave it unclicked, showing her disinterest.(p. 58)

Definition 5. The density D(i), of ranked alarm position i, is the ratio ofimportant alarms found at positions 1 up to i compared to the total number ofalarms found at positions 1 up to i, where i ≤ position of last listed important alarm.(p. 58)

D(i) =# important alarms at positions 1 to i

# alarms at positions 1 to i

Definition 6. Collateral ranking is the act of awarding unimportant at-tributes with higher rank due to their association with important attributes. Thatis, unimportant attributes get points by simply being present in the same alarmsas important attributes. (p. 67)

11

LIST OF TABLES

12

Chapter 1

Introduction

1.1 Background

Since 1966, the syrupy sweet song “It’s a small world after all” have been sungover and over again in the Disney theme parks by the little dolls in the attractionbearing the same name as the song [33]. This sentiment is truer today thanever. The birth of Internet has let us bridge physical distance with the click ofa button. Whether you live in Australia, South Africa or the UK makes littledifference, as with the click of a button millions of computers can be attackedand infected by malware. By the same notion that the world is getting smallerit is also growing; unfathomable proportions of information is being pumpedinto networks connected to the Internet. Individuals, companies, organizationsand governments store their lives, financial records, data and secret documentson social communities and databases.

Much has happened in the world of malware since the Morris worm in 1988 [17].Where Morris operated out of curiosity there are now multi-million dollar opera-tions organized by criminal networks all over the world. Tools for attacking andpenetrating networks are easily bought over the Internet. The “small world”we now live in and the illegal industry dealing in malware and hacking tools allmake it dangerous to simply connect a network cable to your computer.

As a publicly available company or a public authority, a connection to the Inter-net is mandatory and crucial. If you cannot be found on the Internet you mightas well not exist and without an Internet connection, how are your employeessupposed to do their jobs? But a way out is also a way in. Traditional antivirusprograms and firewalls are only able to detect and remove files containing codeassociated with malware and manage incoming and outgoing network communi-cations respectively. There is no way for either of them to discover e.g. passwordguessing since the activity in itself is neither concerned with malicious code nornetwork communications.

What is needed is something auditing the system with a holistic perspective.That something is an intrusion detection system (IDS).

13

1.2. PROBLEM DESCRIPTION

1.2 Problem description

For an intrusion detection system (IDS) to properly perform its task, it has toprocess a huge amount of information from firewalls, logs and audit records. Thedetection rules of the IDS will only single out a small part of the daily events assuspicious and raise an alarm, but a small part of a huge amount may still be alot. As attacks in general are meant to go undetected, attackers try their bestto masquerade their appearance. To accommodate this, the detection rules aredefined in a slightly vague fashion. Unfortunately this vagueness increases thechance of legitimate actions being labeled as suspicious, a prevailing problemknown as false positives.

In many current IDSs, it is the operator that must manually distinguish betweentrue and false positives.1 The fact that false positives easily outnumber the truealarms does not, in general, make the operator any happier. Finding the realthreats can be the equivalence of finding a needle in a haystack.

1.3 Goal

To investigate the problem described in 1.2 and to see what can be done aboutit we intend to:

1. study current methods for reducing the number of alarms.

2. improve and implement one of these methods in an available IDS.

1.4 Limitations

This thesis is not an evaluation of IDSs and their effectiveness, nor is it abouthow to detect suspicious activities in a host or network. The method survey isnot a complete or exhaustive survey and the chosen reports should be viewedas representatives of their respective areas.

1.5 Document Organization

Chapter 2 presents a general overview of the workings of an IDS, as well asa presentation of the IDS Snort and two of its most common interfaces, BASEand Snorby.

Chapter 3 presents a limited survey of methods used for prioritizing and re-ducing alarms.

Chapter 4 presents the methodology behind our work.

Chapter 5 presents our findings and conclusions.

1 In this report we will use the words operator, user and administrator interchangeably.

14

Chapter 2

Intrusion DetectionSystems

This chapter describes Intrusion Detection Systems (IDSs), how they work ingeneral and a brief overview of systems available today.

2.1 General Overview

An IDS is similar to an antivirus program scanning files for malicious code.But where the traditional antivirus program only scans the content of a file theIDS parses and interprets network traffic and/or host activities. An IDS canuse data collected from network packet analysis, logs from routers, firewalls,servers, local systems and system calls to mention a few [1]. Just like theantivirus program an IDS has a set of rules or a database containing knownattack signatures. The IDS compares the signatures against patterns of activity,traffic or behaviour deduced from the data and issues an alarm when the patternor behaviour is a close match. Besides from just issuing an alarm an IDS mayin some cases also take active countermeasures such as shutting down Internetlinks, launch back-traces or make other attempts to identify the attacker andgather evidence. Specifically, an IDS detects unauthorized use of or attacks ona system or network. An IDS can, just like a firewall, either be software-basedor hardware-based.

Figure 2.1 shows an overview of a generic IDS. Target system refers to wherethe sensor is situated, i.e. a router or an ordinary host. Sensor is a piece ofsoftware responsible for collecting data for the IDS. The data can originate froma multitude of sources each with its own format. The pre-processor is responsiblefor weeding out the data required for analysis and transforming it into a formatthat can be understood by the IDS. The analysis engine vets the data againstthe rules and policies to determine whether it is benign or malicious. Post-processing involves correlation of alarms, or construction of attack scenarios,or collection of alarms from other analysis engines. The response unit decidesfor each alarm if the reaction to it should be active or passive [25]. A passive

15

2.1. GENERAL OVERVIEW

Figure 2.1: Architecture of a generic IDS [25].

response includes generation of alarms or log entries but other than that it doesnot interfere with the traffic in any way. An active response on the other hand,may reset the TCP connections in order to disrupt communications, simply droptraffic if the IDS is inline, add the malicious host to blocking lists, or in anyother way disturb the attacker [1].

The typical IDS will not prevent an attack, its function is to detect and alert.However, some IDSs do have this attack-preventing feature and are called IPSs,intrusion prevention systems. But they are a minority, the typical IDS shouldbe viewed as a complement to a firewall, antivirus software, and such and notas a replacement.

2.1.1 Placement

There are generally three different kinds of IDS,

• Network-based IDSs (NIDSs), monitors network links and backbones look-ing for attack signatures.

• Host-based IDSs (HIDSs), monitors the operating system and file systemfor signs of intrusion.

• Distributed IDSs (DIDSs), a number of IDSs used as remote sensors re-porting to some kind of central management system.

There are also IDSs specifically dedicated to gateways monitoring the trafficpassing in and out of the home network and IDSs dedicated to application-specific traffic understanding the flow of the application logic and the underlyingprotocols.

NIDS (Network-based IDS) A NIDS is an IDS that can monitor an entiresegment of the network (see figure on page 17). It is placed on (an) appropriate

16

CHAPTER 2. INTRUSION DETECTION SYSTEMS

Figure 2.2: Architecture of a NIDS.

segment(s) of the network. All traffic seen by the network interface card (NIC)is passed on to the IDS. The advantages of a NIDS is the fact that it doesnot affect either the systems of the network or the network itself since it justpassively records any traffic seen by the NIC. One disadvantage is that it onlysees the traffic by its NIC. Packets taking a different route through the networkwill be missed. Therefore, a NIDS does not always see the whole picture.

HIDS (Host-based IDS) A HIDS protects only the host on which it is situated(see figure on page 18). Due to its location it has access to other types of datathan just network information, including local information such as system calls,file system modifications, and system logs. A plus with the HIDS is the factthat it is possible to adapt the ruleset in great detail for each host. The rulesetonly has to deal with the services specific for that host. A NIDS must haverulesets to deal with everything since it will see everything in traffic on thenetwork. The fine tailored ruleset for HIDS reduces processor overhead andenhances performance for each host. The drawback of a HIDS is the fact thatit steals computing power from its host, unlike a NIDS.

DIDS (Distributed IDS) The standard DIDS is part of a manager/probe ar-chitecture (see figure on page 18). NIDS and/or HIDS sensors placed throughoutthe network report to a centralized management station which compiles the re-ports. Attack logs can be uploaded and new attack signatures downloaded fromthe central management to the sensors as needed. The benefit of a DIDS is theholistic view it presents supplementing the NIDS, unaware of the state of hosts,and the HIDS, unaware of the state of the network. Since all systems have theirdrawbacks, so has also this one. The DIDS uses the internal network to com-municate with the centralized management, and if the network is incapacitated,so is the holistic view.

17


Figure 2.3: Architecture of a HIDS.

Figure 2.4: Architecture of a DIDS.

18


2.1.2 Detection Schemes

There are two different approaches an IDS can take when deciding if an eventshould be reported as an alarm or not, anomaly detection and misuse de-tection.

Anomaly detection, also known as statistical detection, strives to define nor-mal, expected behaviour [35]. It is based on the idea that attacks and intrudersdisplays different behaviour than the normal legitimate user. E.g. a trojan set-ting up a high number of connections where the normal user would yield onlya handful of connections will result in an alarm [24]. To be able to decidewhat actions are anomalous a baseline of normality for user behaviour and sys-tem activity has to be established. The baseline is either predefined by thesystem developers or constructed using statistical sampling, specification-basedapproaches or neural networks [1]. Recording and deciding normal behaviour isa complicated process. It is hard to exhaustively define normal behaviour sinceit is dynamic and changes over time. Homogeneous systems with a moderatenumber of users where the same actions are performed repeatedly are muchmore readily described than a heterogeneous system with maybe thousands ofusers, say a university network. It is also hard deciding what is an anomaly andwhat is a minor deviation in the behaviour of the user. To partly compensatefor this uncertainty, the decision function works with intervals and boundaryvalues. The behaviour is allowed to slide a bit from the expected value withouttriggering alarms. When an alarm is triggered, the alarm is accompanied bya certainty value depending on how far off the behaviour is. This helps whenquickly deciding if an event should be considered as to be a true or false positive,although caution should be exerted. Inappropriately set threshold values are apotential disaster. Set too high and attacks are be missed, set too low and thelevel of false positives skyrockets.

Policing all activities is a costly endeavour since the extensive logging neededby the system to perform has a significant impact on the performance of thehost. This makes anomaly detection less suitable for real-time systems [1].

Anomaly detection has its benefits and drawbacks. Anomalous behaviour doesnot have to be an indication of an attack, the explanation for the altered be-haviour could be as simple as new assignments for the user or a new service onthe network. This results in costly false positives which have to be examined[24]. But this weakness is also the strength of anomaly detection. Althoughunaware of the ways of novel attacks an anomaly detecting IDS is still able toproduce alarms for the novel attacks - as long as the attacks results in a systembehaviour different from the normal. Unfortunately this is not the only obstacleto be conquered. When choosing the parameters to build the baseline, one hasto make a correct selection of parameters and not choose too few or too many.Too few will result in an incomplete model missing attacks and too many willunnecessarily burden the host system.

Misuse detection, also known as signature-based detection or rule-based de-tection, strives to define improper behaviour [35]. It follows the same principlesas an antivirus program. Patterns of misuse called attack signatures are storedin a database and the audited area, may it be log files or network traffic, is

19


vetted against these and any match will result in an alarm. A signature canbe i.e. a string to search for in packets or a sequence of actions. Intrusionschemes are, just like malware, ever changing; the signatures must therefore beconstantly updated as not to miss any attack. Most IDSs adhere to the misuseprinciple [24].

This method has both advantages and disadvantages if compared with anomalydetection. Where misuse detection cannot catch novel attacks, anomaly detec-tion can. Where anomaly detection can never be quite certain of an intrusion,misuse detection can. Misuse detection suffers from similar definition problemsas anomaly detection. The problem with attack signatures is how to define themand still be able to catch all misdeeds without causing false positives to triggeroff on normal traffic.

To summarize; an IDS can adhere to a known-good (anomaly detection) or aknown-bad (misuse detection) policy [1]. Known-good has configuration fileswith the acceptable behaviour explicitly stated. All else yields alarms. Known-bad is much simpler since it does not require a comprehensive model of al-lowed input. Known-bad trigger alarms only on traffic that is known to be bad.Known-bad and known-good both have drawbacks and benefits. Known-bad iseasier on the processors, alerting only when something matches a rule. Unfor-tunately it can only alert on what is stated in the rules. Anything else sailsthrough. Known-good has the benefit of being able to alert on novel attacksas long as they yield a behaviour deviating from the configured acceptable be-haviour. The drawback of known-good is the impossible task of modelling allbehaviour that is acceptable which results in many false positives. On top ofthat, it is completely normal for a user’s behaviour to change over time. Alsoif the system is able to cope with a slowly changing behaviour, who is to sayan intruder would not slowly accustom the system to a behaviour which wouldmask the planned intrusion? Most IDSs use a combination of the two.

2.1.3 Key Concepts

A true positive is an alarm correctly indicating an attack.

A true negative is an event correctly classified as benign.

A false positive is an alarm issued for a benign event mistaken for an attack.This is not uncommon in an anomaly detection system where all events notexplicitly marked as approved issues an alarm [24]. Misuse detection is alsoplagued by erroneous alarms, the cause being badly written detection rules orbenign traffic sporting a high similarity to malicious traffic [16]. Buggy softwareand badly configured hardware adds to the load for both systems, effectivelydrowning the true positives. This is a big problem today [3], and a lot of researchand effort goes into finding a way to minimize the effects false positives have onan operator’s work situation and the system dealing with alarms.A high level of false positives is normal for “young” IDSs and drops off with thetweaking of the system over time [1].

A false negative is an attack the system fails to detect and is much more seri-ous than a false positive [24]. Whereas false positives may obscure the view, the

20


true positives are still there, are still logged and retrievable from the database.An undetected attack is not logged anywhere and thus when and if the intru-sion is detected there is no way of knowing for how long the system has beencompromised, what data to trust, which backup is clean or how much havoc hasbeen wreaked.

2.1.4 Motivation

So why bother having an IDS as it seems like a lot of headache? E.g. say acompany one day discovers that someone has gained unauthorized access totheir database. If they have no IDS, how can they know if they can trust theirdata? Find out when the breach happened? And if any of the backups can betrusted?

An IDS, hopefully, grants the ability to discover reconnaissance attempts, at-tempted system compromise or other malicious activity and this can mean thedifference between being compromised or not. If no one is aware of the attackertrying to break into the system, how can she be stopped? With an IDS ableto warn the administrator of a successful attack, the administrator can takemeasures to limit the damage or take the system offline to examine the extentof the breach.

Another reason to have an IDS is its data mining properties helping the adminis-trator finding subtle trends in large amount of data, which would be impossiblefor a human to find unaided. Correlation between alarms and hosts helps theadministrator to see patterns otherwise hidden.

No matter how big a help the IDS is, skilled analysts will always be needed tosift through the alarms the system produces. They have to be able to separatethe real alarms from false positives. No matter how much the system learns, itwill never be able to assess the level of a threat like a skilled and experiencedanalyst knowing her network.

2.2 Available systems

There are a great many IDS, IPS, and GUI available today from many differentsoftware companies. Many are commercial systems targeted at consumers andtheir home computers such as antivirus and firewall applications offered bySymantec, ESET and others. The systems that we are interested in in thisthesis are however not targeted at home computers. They are designed tohandle loads much greater than what a home computer ever will encounter, andrequire knowledge that most people do not have. During the course of our workwe have studied a few of them, and most of them are open-source systems aswe do not have the funds to buy any licenses.

21

2.2. AVAILABLE SYSTEMS

2.2.1 Snort

Snort [34] is an open source network intrusion detection and prevention systemnow developed by Sourcefire and was originally released in 1998 [31]. It has threemodes of operation. Packet sniffer mode reads packet headers and displaysthem on the console. Packet logger mode will log received packets to a file.Network intrusion detection mode will analyse incoming packets and detectintrusion attempts. Snort analyses network traffic with the help of predefinedrules containing detailed descriptions of what kind of traffic should be consideredharmful to the network. In other words it uses the misuse-based paradigm.These rules may also contain actions to be taken when a specific attack isdetected. Appropriate actions can be to log the alarm to a database, sendan email to an administrator or terminate the offending connection. With thehelp of these rules Snort looks at all parts of incoming network packets (headerand payload) to determine whether each individual packet should be consideredmalicious or not. If a packet is considered malicious the appropriate action forthat type of alarm will be taken.

The alarms that are captured by Snort are by no means guaranteed to be actualattacks or intrusion attempts made by an adversary. Snort is only designed torecognise possible attempted attacks on the network. This means it is essentiallyjust a dumb sensor filtering out traffic matching a set of predefined rules. In turnthis means that there is a relatively high chance that a majority of the alarmsare going to be false positives (depending on how large a target for attacks thenetwork is). The hard work then of analyzing the alarms and see the correlationbetween different alarms is left entirely to the operator to handle.

Snort sensors are most commonly placed in front of the firewall protecting thelocal network so they can capture the unfiltered traffic coming from the Internet.Figure 2.2 on page 17 is a good example of such a sensor placement. If the localnetwork is very big and divided into subnets it might be a good idea to also placeSnort sensors at the border of those subnets. Placing more than one Snort sensorin the network can help tracking the attack as it progresses in the network. Anattack most commonly start with someone attempting to penetrate the outerdefences (firewalls), if this succeeds they might attempt to infect a computer onthe inside. If the computer is successfully infected it will become their bridgeinto the network. Therefore it would be good to have more sensors inside thenetwork to monitor traffic between computers. The Snort sensor at the borderto the internet will only be able to see incoming and outgoing traffic and nottraffic moving between the hosts of the internal network.

Preprocessors

Preprocessors are plug-ins which can be used with Snort to rearrange or modifyincoming packets before they reach the filtering engine. The Snort preprocessorsgenerally fall into two categories. They can be used to either examine packetsfor suspicious activity or modify packets so the detection engine can properlyinterpret them. An “examining” preprocessor has the ability to detect non-signature-based attacks that the detection engine cannot track down. Such

22


attacks include port-scans where a state has to be saved between many differentconnections to determine whether there is a port-scan occurring.

The second kind of preprocessor normalizes incoming traffic so the detectionengine will be able to properly detect attacks. Incoming packets can for examplebe fragmented because the original packet was too large. This creates a problemfor the detection engine since the harmful part of the package might be splitinto two different packets, which results in the detection engine failing to triggeran alarm. Preprocessors are here to mend this situation by stitching the twofragmented packets together again. Packet fragmentation usually happens whenpackets travel between different types of networks. I.e. if a packet travels into anEthernet network and is larger than the MTU (Maximum Transmission Unit) forthat network the packet will be split into two or more packets to be reassembledwhen it reaches its destination.

Rules

Snort rules are divided into two logical sections, the rule header and the ruleoptions. The header contains the rule’s action, protocol, source and destinationIP addresses and netmasks, and the source and destination ports information.The rule option section contains alarm messages and information on which partsof the packet should be inspected to determine if the rule action should be taken.Actions that can be taken include logging the activity, dropping the packet alltogether, reject it (with TCP, RST or ICMP port unreachable), and pass (ignorethe packet).�

1 alert tcp any any -> 192.168.1.0/24 111 \2 (content :"|00 01 86 a5|"; msg:" mountd access ";)� �

Listing 2.1: Example Snort rule

Listing 2.1 defines a rule to generate an alarm if there is traffic coming from anyip or port to a local address 192.168.1.0/24 on port 111 with a payload containingthe binary data “00 01 86 a5”. The binary sequence “00 01 86 a5” can belocated anywhere within the payload for a match to occur. When that happensan alarm will be logged with the message “mountd access”. Backslash is usedto divide the rule onto multiple rows, the default interpretation is otherwise onerow one rule.

2.2.2 BASE

BASE [15] stands for Basic Analysis and Security Engine and is designed toread and display the logged alarms Snort has generated. Alarms are displayedto the user via a web interface written in PHP.

Figure 2.5 gives a statistical overview of what has been logged by the differ-ent sensors such as the distribution of different protocols of the logged alarms.There are also links to the most common views that an administrator will beinterested in keeping an eye on, e.g. most recent alarms and most commonalarms. The views can be further specialised by using the search function where

23


Figure 2.5: BASE home screen.

the administrator can be more precise about what he/she is looking for in thedatabase.

BASE also gives the user the capability to add alarms to groups making it easierfor the user to spot trends in the incoming alarms. Adding alarms to groups isan entirely manual process. Alarms that have been placed in a group can thenbe viewed in a special tab of their own.

To give the user a better overview of the most recent attacks, BASE generatesgraphs using statistics Snort has gathered. The user controls what is displayed inthe graph via comprehensive controls. Simple graphs, such as number of attacksover time can be generated as well as more in-depth graphs only displayingstatistics for a single host or port.

2.2.3 Snorby

Snorby [27] (Figure 2.6) is, like BASE, a web-interface for Snort and has manyof the same features. It is developed using Ruby on Rails [38] and uses manyof the modern web development techniques available today, such as Ajax andFlash. The main difference to BASE is that Snorby is built with collaborationbetween users in mind. All users have two main collaboration tools at theirdisposal, marking events with a star and leaving comments.

Marking events with a star is similar to placing a bookmark. The user simplyclicks on the star icon associated with that particular event to mark it. Starredevents are gathered in a separate list common to all users, meaning if one userstars an event everyone else will be able to see it as well. This promotes col-laboration in that all users can work together to find interesting or important

24


Figure 2.6: Snorby

25


events and investigate these further. This also works well with the other collab-oration tool in Snorby, the commentary system. When starring an event, theuser can leave a comment explaining why that particular event is of interest orneed further investigation. At a later time, when an administrator logs on tothe system she can view those starred alarms, read the comments, review them,and take an appropriate action.

Snorby also has the ability to generate automated reports that are mailed tothe system administrator. These reports contain summary statistics about whatkind of alarms the Snort sensor has generated the previous week as well as morein-depth information about important alarms.

2.2.4 Cisco MARS

MARS [4] stands for Monitoring, Analysis, and Response System, and is anetwork security suite developed by Cisco. In contrast to Snort, BASE, andSnorby this system costs money to buy and install in a network. But it alsohas much more to offer than the previous systems. MARS is more than just anetwork sensor that looks for malicious traffic on the network. It is more like acommand central for all systems that have a security role in the network, suchas firewalls, network sensors, antivirus programs, and much more. It combinesinformation from many different sources into one interface that is easier tooverview, and more than that it also correlates the collected information.

By correlating the information, MARS can detect chains of events that are re-lated to each other across all the different sensors. If an attacker starts attackinga computer and tries to gain root access this can potentially be detected in anumber of different places. The attacker might try to exploit a vulnerabilityon a mail server, which is visible from outside the network. Before the attackreaches the mail server it has to pass the firewall and probably a network sensorsimilar to Snort. If it gets past the first firewall, the mail server itself mighthave a firewall of its own. Lastly the mail server will have a system log thatstores messages about local events, i.e. if someone tries to login on the computera log message will be created. MARS is fed all this information from all thesedifferent sources and correlates them into a form more easily overviewed. Theoperator can then see exactly what has happened from start to finish in theattack, and even if it succeeded or not. MARS will also try to give suggestionsabout where the attack can be mitigated most effectively so it does not happenagain.

Cisco MARS also has the ability to collaborate with other similar MARS systemsthat are deployed around the world. Useful information about attacks are sharedbetween these to help detect new and unknown attacks. For example, blacklistsused to block spam mail are also used to detect suspicious incoming traffic.Traffic sent from a computer known to be a spammer gets logged and reportedto an operator automatically. This can be useful to know as much of the spammail sent today are sent from bot nets operated by criminals. Knowing if suchtraffic suddenly increases might give an early warning that something bad isabout to happen and counter measures can be deployed in time.

26

Chapter 3

Previous Work

This chapter presents different methods helping the operator of an IDS handlethe amount of alarms produced, with a focus on false positives.

The algorithms behind the methods presented below are nothing new and havebeen used within research since the 80s. One can follow how the area matures inthe literature through the decades. The same methods used to find anomalies[36, 10, 22, 14, 11, 32, 21, 20] and detect misuse [36, 13, 23, 19, 37, 18] areused today to form more abstract higher level alarms out of specific lower levelalarms.

Reducing the amount of data is not a novel idea either [12]. Focus has shiftedthough with the growing hardware capacities, from reducing the amount ofdata for the benefit of the system to the benefit of the operator. Frank [12]also breathes the hope that with good enough rules, systems and/or trainingexamples, false positives and false negatives can be brought down to a minimum.This hope is rebutted by Axelsson [3] where he makes calculations on the base-rate fallacy and points out the consequence, that false alarms are something wewill probably never get rid of as the false alarm rate will have to be impossiblylow for that to happen. This seems to be a generally accepted truth and theway to reduce false positives today is to apply the methods presented belowthus hopefully minimizing the effect from false positives.

3.1 Aggregation

The meaning of aggregation varies greatly between reports and we therefore feelthe need to define the word ourselves.

Definition 1. Aggregation is the process of grouping alarms together basedon certain user-defined criteria found in alarms.

Alarms can be aggregated based on any attributes found in alarms such as:destination IP, alarm type, content of data portion.

27

3.2. CORRELATION

Of the methods presented in this chapter, aggregation is the most low level andstraight forward of them all. Hence, it is not common in the literature to focussolely on aggregation. Aggregation is now more of a ubiquitous step in anyscheme processing alarms.

Debar and Wespi are among the few still explicitly mentioning aggregation eventhough very little attention is given to it, mainly focusing on correlation [9].They describe how to aggregate and correlate alarms with the ACC – aggrega-tion and correlation component. The task of which is to present a condensedview of the input received from multiple sensors and ACCs. Incoming eventsare processed by extracting common information and finding a match in pre-vious observations. Correlation relationships are then formed by looking forduplicates and consequences.

A duplicate is an alarm whose cause have already been taken into account bythe ACC algorithm. How alarms are judged to be duplicates is specified in aconfiguration file. A consequence is an alarm that under some conditions givesreason to expect others to follow. For a following alarm to be considered a partof the consequence chain it must occur within a given interval of its predecessor.As with duplicates, consequence chains are declared in a configuration file.

Lastly, the correlated alarms are aggregated into situations, where alarms havecertain characteristics in common. This step is performed since isolated eventsare often considered insignificant. A situation is defined by four terms: alarmsclass, source, target and severity level. The user is presented the situationswhich are fewer in number than the original unprocessed alarms.

3.2 Correlation

Aggregation of alarms still leaves a lot of alarms to be inspected and furthermethods have been explored to crop the number of alarms presented to the user.The term correlation is used for a variety of methods to compress the numberof alarms displayed for the user. To eliminate ambiguity we again feel the needto make a definition.

Definition 2. Correlation is the process of building data models from alarmsusing external knowledge not found within the information carried by the alarmsthemselves.

External knowledge might be such as how alarms appear together, or the im-pact attacks have on the network. The idea is that correlation will suppressthe impact of false positives by lifting the alarms to a higher level of reasonand putting them into context. False positives are marginalized and the truepositives are put into focus.

In this chapter, two methods of correlation are presented, correlation usingconditions and correlation using probabilities.

28

CHAPTER 3. PREVIOUS WORK

3.2.1 Using conditions

Correlation using conditions continues the chain of thought presented by Debarand Wespi [9] for aggregation, some alarms are consequences of others. Theidea is that most intrusions are not isolated and are part of a chain of actionswhere earlier actions are preparing for later. Each alarm is associated witha set of pre-conditions to be fulfilled for the attack to succeed, and a set ofpost-conditions describing the aftermath of the attack, if succeeded. Attackchains are formed by connecting the pre-conditions of one alarm with the post-conditions of one or several other alarms. The attack chains are meant to 1)result in fewer items for the operator to inspect and 2) remove false positivessince they do not belong to any attack chain.

As with the definitions of aggregation and correlation, there is no consensusregarding denomination. Ning et al. [28] have developed a framework usingprerequisites and consequences as opposed to pre- and post- conditions. Thealarms and their connections are modelled onto graphs where nodes correspondsto alarms and edges represents the relationships between them. If an alarm hasbeen decided to prepare for another, they are connected by an edge.

The resulting chains are intended to be analyzed by human users to understandthe correlated alarms as well as the strategies behind them. But the graphs canget fairly big due to the amount of alarms. For a user to be able to managethe graphs they have to be cropped. Ning et al. propose filtering out irrelevantalarms and aggregating alarms of the same type as means. Here, aggregationshows up as a vital part of the process without being given the same focus asin Debar and Wespi [9].

Cuppens et al. [6] have developed something similar to Ning et al. [28] butdo not use graphs as a means to model alarms and relationships. As partof the project MIRADOR, CRIM was developed as a cooperative module forIDSs. It manages, clusters (groups alarms into sets corresponding to the sameoccurrence of an attack), merges (creates a new alarm that is representative ofthe information contained in the various alarms belonging to this cluster) andcorrelates alarms.

Since an intruder, most likely, has to carry out several attacks in a certain orderto reach his goals, Cuppens et al. defines the task of the correlation functionas anticipating the intrusion plan. A set of candidate plans of the currentintrusion is returned by the function. The plans are denominated candidatesince the intruder might not have reached his goal yet. The result is used by thereaction function to help the operator take the best course of actions againstthe intruder.

When an alarm is received, the database is checked to see if the pre- conditionsof the new alarm can be linked to any post-conditions of stored alarms. If itcan, the correlation conditions are checked. An algorithm is then applied to thepair to see if it might be merged into an existing attack scenario, if not a newis started.

Another variation of this theme is the requires/provides model by Zhou etal. [39]. Whereas CRIM by Cuppens et al. [6] and the model suggested by

29

3.3. FINDING THE NORMAL

Ning et al. [28] use predicates to describe the state of the network, Zhou’smodel is based on capabilities which describe what the attacker can accesson the network. Inference rules are drawn between the capabilities, and if allcapabilities of requires (cf. pre-conditions of Cuppens et al. and prerequisitesof Ning et al.) can be inferred from the capabilities of provides (cf. post-conditions of Cuppens et al. and consequences of Ning et al.) the two alarmsare correlated.

3.2.2 Using probabilities

The use of conditions for correlation, see previous section of Ning et al. [28],Cuppens et al. [6] and Zhou et al. [39], is a black and white procedure. Eitherall of the conditions for correlation are fulfilled and the alarms are correlated, orit fails. Correlation using probabilities adopts a grayer view of the correlationprocess. Instead of all or nothing, the probability of an alarm being correlatedto another is calculated. That probability is then used when deciding if thealarms belong together.

Dain and Cunningham [8] have developed an algorithm for fusing and combiningalarms into scenarios. A scenario is described as a sequence of alarms groupedtogether as they share a common cause. Whole scenarios are labeled as falseinstead of individual alarms thus simplifying for the operator. Each new alarmis compared to the existing scenarios using data mining techniques and theprobability of the alarm belonging to that scenario is calculated. The alarmis then added to the most likely candidate, the scenario yielding the highestprobability. If none exists a new one is created.

The most flexible way to fuse an alarm into a scenario would be to consider allpossible ways of combining each new alarm with all previously collected, recal-culating for each alarm how the biggest scenarios can be formed. This wouldquickly become computationally infeasible. Obviously, this is not possible. In-stead an atom model is proposed. The new alarm is compared to all so farconstructed scenarios and merged with the one returning the highest probabilityof correlation. If all scenarios return a probability lower than a preconfiguredthreshold, the alarm joins a new scenario. Once assigned a scenario the alarmis not reconsidered or reassigned, hence the term, atom model. This proceduremight generate errors but greatly simplifies deciding how to assign alarms.

3.3 Finding the normal

Up to 90% of all alarms have benign causes such as misconfigured hardwareand buggy software [16]. By “finding the normal” we refer to the process offinding the alarms caused by benign events. To be able to identify and removethese before applying the correlation methods mentioned in previous sectionholds a lot of promise. The amount of data that is processed by the models issignificantly decreased, as well as the data presented to the user.

A very straight forward method is applied by Gengo and Clifton [5]. By us-ing a data mining technique called frequent episodes, common and recurring

30


sequences of alarms for a given site are identified. These are then manually an-alyzed to see whether they result from normal operations. With the help fromthese alarms, site-specific filters can be constructed which may help reducingthe amount of alarms presented to the administrator.

Julisch [16] is of the same idea as Gengo and Clifton but attacks the problemfrom a higher level of abstraction. Alarms are grouped in clusters and for eachcluster a meta-alarm, a generalized alarm, is derived. A generalized alarm is apattern which the alarm created by the IDS must match in order to belong tothat cluster.

Julisch focuses on bringing alarms into clusters where each cluster representsthe root cause behind the alarms. A root cause is defined as a “problem thataffects components and cause them to trigger alarms”.

To be able to generalize alarms a single rooted and connected directed acyclicgraph (DAG) called a generalization hierarchy is constructed over each attribute,of the alarm. Each parent node in the graph is a more generalized attributethan any of its children nodes. To generalize an alarm, its attributes are locatedin the DAG. The attributes of the alarm are then replaced by the parents. Theproblem with DAGs is that there might be more than one parent to a node.When generalizing alarms one can face problems when there is more than onechoice. Julisch proposes two strategies to resolve this:

• choose-one - user-defined rules decides which parent to choose.

• explore-all - pursues all possible generalizations in parallel and retainsthe one that first leads to a generalized alarm of a user defined size.

A somewhat different approach is used by Manganaris et al. [26] Instead offiltering out or removing the benign alarms as Julisch [16] and Clifton andGengo [5] do above, they are viewed as something normal. The idea is presentedas applying anomaly detection on the output from a misuse detection system. Abase template of what the normal alarms are and how they behave is established,when the “normal” alarms suddenly change in frequency or in sequence, analarm is raised. The basic idea Manganaris et al. have is that repetitive behaviorover long periods of time is in all likelihood normal. Combinations of certainalarms occurring in close temporal proximity of each other and always in thesame order, for an extended period of time are deemed less suspicious than asudden burst of novel alarms. To discover the sets of benign alarms occurringfrequently, association analysis is used on the bursts of alarms.

3.4 Ranking

Correlation in Section 3.2 and filtration in Section 3.3 helps the operator a greatdeal, reducing the number of items to go through. They do not however helpthe operator to rank and process the result in regard of importance. This isaddressed by Almgren and Jonsson [2] and Porras et al. [30]. The two reportshave diametrically opposed methods to rank alarms.

31

3.4. RANKING

Almgren and Jonsson [2] pose the question: “is it possible to capture the pref-erences of the site security officer (SSO) by monitoring her normal interactionwith the IDS?” or put a little differently: “is it possible to prioritize alarmsbased on the feedback from the user?” Their aim is to develop a system thatlearns how to rank alarms based on the input from the user and local operationalconditions. This model assumes the use of a graphical user interface to displaythe alarms and interaction from the user in the form of clicking on alarms. Fourinteraction modes are presented:

• Clicking - the system stores information on what alarms the user clickson. Assumes that alarms clicked on is of greater interest than those nottouched. Ranks those with a larger amount of clicks higher than thosewith a lesser amount.

• Directed learning - this is an extended form of clicking. On top of thenormal procedures the user has the choice of specifying to the system whyshe finds this particular alarm interesting.

• Feedback - the system ranks the alarms and specifies for each a reasonwhy it is raked the way it is. The user then has the option of eitheragreeing or disagreeing with the ranking. This is in a way directed learningin reverse. Instead of the user telling the system, the system is telling theuser.

• Explicit questions - this is another extension of clicking where the sys-tem occasionally asks the user how to rank an alarm.

Almgren and Jonsson estimate the first interaction mode, clicking, to be thebase case of all proposed interaction modes and concentrate on this first. Thework flow of the base case is intended to follow

1. alarms are presented based on the current ranking function.

2. the SSO reviews the ranked list to assess the most important alarms.

3. the SSO investigates the most important alarms by clicking on them.

4. the system evaluates the current ranking function by checking what alarmswere clicked and if these indeed were ranked highest. If not, the systemreevaluates the ranking function using the feedback from the clicks forimprovement.

5. a new view is presented to the user based on the new sample rankingfunction.

The system presented by Almgren and Jonsson operates on an on-the-fly con-figuration. The ranking M-correlator devloped by Porras et al. [30] operates onutterly opposite premises. It is dependent on configuration files describing thetopology of the system, the most critical assets and alarms of interest for theoperator.

As can be seen above, the final ranking score is dependent upon a multitude ofvariables. The correlator starts though with filtering out alarms the operatorhas expressed low interest in. On the alarms passed by the filters a relevancescore is calculated for each, this corresponds to the relevance node in the graph.Each alarm is compared against a map of the topology of the network containing

32


Figure 3.1: The Bayesian network used when calculating the final rank scorefor alarms in the M-correlator by Porras et al. [30].

information about such things as running applications, OS, services, open ports,to see how relevant the alarm is to the network. The M-Correlator includes anIncident Handling Fact Base which provides information about vulnerabilitydependencies. E.g. an alarm pertaining to a Microsoft Server would yield a lowrelevance score in an environment running Apache. The map of the networkis automatically generated by the tool Nmap. The M-Correlator moves on tocalculate the priority of the alarm. This relies on two variables (priority node inthe graph). One is to what degree the alarm is targeting data assets and servicesmost critical for the client users. These are enumerated in a configuration fileby the operator. The other concers which alarms are of greatest interest to theoperator. This is also specified by the operator in a configuration file. Lastly,an incident rank is assigned to each alarm based on the previously calculatedrelvance and priority scores, reflecting the possible impact the attack would haveon the system and the likelihood of its success (outcome in the graph). Once analarm has been given an incident rank the M-Correlator tries to merge relatedalarms together into higher-level security incidents which in turn are ranked.This is done by aid from the Incident Handling Fact Base which also carriesinformation about what types of alarms can be merged.

The relevance score, priority calculation and incident rank are all computed byusing a variation of the Bayes framework for belief propagation in trees.

3.5 User input

Only a few reports mention user input and only Almgren and Jonsson [2] focusexclusively on the interaction between system and operator. The interaction

33

3.5. USER INPUT

can be passive with the system simply observing the user and trying to drawconclusions from these observations and any possible user input, or it can beactive posing specific questions to the user, initiating the interaction. Thismakes the system dynamic in the sense that the algorithm behind the rankingpossibly changes by every interaction.

The time the correlation system of Cunningham et al. [7] is exposed to the useris limited only to the user tagged sets it is trained upon before launch. This isthe only interaction with the system, making it a static one, since beyond theinitial training set, there are no modifications to how the algorithm processesthe alarms.

Attempting to reduce the number of false positives Pietraszek [29] first letsthe user classify a set of alarms, then machine learning techniques are usedon training examples the system derives from the original set of user classifiedalarms to generate rules of how to classify. The alarms classified by the rules arethen sent to the user for optional scrutiny. Alarms are re-classified as needed.Although the user is very involved in the process of classifying alarms, this isvery little highlighted by Pietraszek. Nothing is mentioned about the processof classifying the starting set of alarms, or the process of reclassifying, or abouthow the interaction functions, or what kind of input the system expects andwhat kind of feedback it gives.

As mentioned in Section 3.4, the mission based system of Porras et al. [30]involves the user, although quite extensively, only during start-up, similar toCunningham et al. [7]. The user has to enumerate critical data assets andservices and specify an interest profile describing which classes of alarms are ofgreater concern. However, there is no interaction with the user once the systemis on line and there is no discussion about giving feedback to the system orwhat to do if the user is dissatisfied with the ranking. However, Porras et al.describe an adaptive mode concerning the conditional probability tables (CPTs)under the section describing the Bayes calculation. Here the analyst presentssimulated alarms to the system which are then ranked. For each alarm, theranking is either accepted or changed to a more accurate one by the analyst.The effect will be that the CPTs change to some extent to better mirror thepreferences of the analyst. This adaptive mode is never evaluated or mentionedagain in the remainder of the report.

All schemes have user input somewhere in the chain. Somewhere and some-place there has to be a human involved in the process to either analyze analarm, describe the critical aspects of a network, evaluate performance, sepa-rate unimportant alarms form the important or deciding if it is a true or falsepositive. Clifton and Gengo [5] described in Section 3.3 need the user to deter-mine whether a frequent episode is benign. The same goes for Julisch [16] inthe same section. The generalization hierarchies must all be defined by the userand contains the knowledge of the user about the system and about alarms ingeneral.

In the next chapter we discuss how the methods presented here relate to oursolution and what aspects were taken under consideration.

34

Chapter 4

Methodology

In order to investigate current methods of prioritizing and managing alarms,we have studied current research in form of published reports and books, asdescribed in Chapter 3. We have also interviewed four professionals.1 Basedon this material, we had a number of options to choose from when deciding ona possible improvement to an existing IDS. The choice finally settled on theimplementation of a ranking functionality in the architecture Snort/Snorby.

Choosing method for implementation

Aggregation was the first to be rejected due to its simple and confined nature.Correlation (see Section 3.2) is a labour intensive and time consuming methodin regard to identifying and defining pre-conditions and post-conditions. Thismakes correlation an activity incompatible with the continuous task of inspect-ing incoming alarms. As our interest lies with the everyday interaction a userhas with the IDS, correlation is rejected. Set up of filters is done prior to the de-ployment of the IDS. The setup process is in no way part of everyday operationsmaking this less interesting.

Ranking is a necessary step whether the system employs aggregation, correla-tion, ranking or a collaboration of them all. As long as there are more thanone alarm to inspect, the operator has to make a choice in which alarm to in-spect first. That choice is the essence of ranking - which is the most importantalarm to look at? For these reasons we have chosen to implement a variation ofranking.

Choosing architecture to make implementation on

To be able to make improvements to an existing IDS we needed access to thesource code, which is easier with open source projects. Snort is an obviouschoice because of its longevity and dominance within its field. Of the interfacesavailable for Snort only two seem to be in use today, BASE and Snorby, where

1See appendix D.

35

4.1. HOW TO IMPLEMENT RANKING

we chose the latter for this project. BASE is written in PHP and Snorby inthe framework Ruby on Rails. The latter facilitates implementation and leavesmore time to concentrate on the logic of the implementation. Finally, Snorbyis actively developed, meaning that our contributions may be used in futurereleases.

4.1 How to implement ranking

There are two distinct methods for ranking, either heavily configured [30] or on-the-fly configuration [2]. Out of these, the approach by Almgren and Jonsson [2]seemed the most interesting, focusing on the ongoing interaction between userand system. The system in Porras et al. [30] has a life-cycle similar to filtering,only deployed once in the beginning. Any changes made to the configurationfiles results in a restart of the system. We believe a dynamic system that canlearn is more useful to the operator.

The first two problems immediately encountered after settling on Almgren’sand Jonsson’s [2] mode of interaction, were how to communicate to the systemwhy certain alarms are important and what information of the alarm to usewhen deciding on a rank. There is of a lot of information such as destinationIP, source IP, destination port, source port, alarm signature, and/or patternsin payload that might be of interest to a ranking system. A number of ways,more or less time consuming were discussed during the course of finding a goodinteraction model with the IDS. These are described in the following. For thefinal method, see method 3 on page 37.

Interaction method 1: check boxes

One of the discussed and rejected suggestions was the use of check boxes (Fig-ure 4.1). I.e. each alarm is presented with some check boxes, one each for theattributes chosen for analysis. As we have chosen to follow the attributes des-tination IP, source IP, destination port and alarm signature, our case results infour boxes. The check boxes corresponding to the reason of interest are checkedand the result submitted to the IDS. The same suggestion was also presentedby Emilie Lundin Barse during her interview.2 Hopefully, this will let the IDSmake rather precise predictions about new alarms, whether the operator willfind them important or not. Over time the IDS will gather more informationand improve its performance, saving time for the operator as the critical alarmswill already be flagged. This holds a great benefit over the original clickingmethod proposed by Almgren and Jonsson [2] where the system did not knowwhy an alarm was clicked and in a sense had to guess how to rank the alarms.

The problem and the ultimate reason for its rejection, is that we wanted thenecessary interaction to be as minimal as possible. If this model is extended totrack additional attributes, more check boxes will have to be added and the usersinteraction with the system will become more complicated and time consuming.

2Emilie Lundin Barse (information security consultant, Combitech, interviewed in personby the authors April 12, 2010), see appendix D.3.

36

CHAPTER 4. METHODOLOGY

Figure 4.1: Conceptual design of check boxes for interaction with the rankingsystem.

Interaction method 2: only clicking

A better approach would be to gather information in the background in a morenon-intrusive way but still get adequate information to make meaningful pre-dictions. The first method suggested by Almgren and Jonsson [2], clicking, fitsour objective. It silently records on what alarms the user clicks and tries tomake predictions about rank.

Clicking in its original description has a shortcoming. It is only described asan incrementing value, meaning a value either grows with clicking or remainsas is. But what if one would like to lower the priority of something? I.e. thereis a server with a serious security flaw. All alarms of a certain type targetingthe server is now of immense interest. In all, a few weeks pass by where allsuch alarms are dutifully clicked and examined, until the vendor of the serversoftware releases a patch. All of a sudden, the previously important alarms arenow a hindrance. Here, it would be very beneficial to be able to rank thesealarms down and stop them from cluttering the view.

Interaction method 3: positive and negative feedback

The solution decided on, is used in a number of community news sites such asDigg and Reddit and also by the video-sharing website YouTube. It gives theuser the ability to give positive and negative feedback to the system throughtwo buttons associated with each item. The system will change the rank of thatitem according to the vote. News stories and videos appreciated by a lot ofvotes will be ranked higher than other news stories and videos.

Figure 4.2: The thumbs up button and the thumbs down button to press forliking respectively disliking a video on YouTube.

Our ranking model works in a similar manner, but instead of ranking newsstories it ranks alarms generated by Snort. There are two buttons associatedwith each alarm, a “+” button and a “-” button. If the user clicks on one ofthe buttons she gives that alarm a higher or lower priority respectively.

In reality, alarms are not clicked because of a single attribute.3 Instead, it is3 Explained by Ola Soderstrom (security analyst, Omegapoint, interviewed in person by

37

4.2. TIME BASED WEIGHING OF VOTES

Figure 4.3: Final design of buttons for negative and positive feedback. “+” isfor positive and “-” is for negative.

often a combination of attributes and external knowledge not found in alarmsthat make alarms interesting.4 The ranking system tracks four basic attributes;source IP, destination IP, destination port, and the signature of the alarms.These are chosen since they are assumed to often be of interest to an operator.

There is a counter keeping track of the number of up and down votes a spe-cific attribute receives. When voting for an alarm, each of the four attributesreceives a vote. Prioritizing (giving positive feedback to) an alarm incrementsthe counters for each of the four attributes associated with that alarm by one.I.e. an alarm receives an up vote because of its destination port 79, the counterfor port 79 is incremented together with the counters associated with the valuesof destination IP, source IP and alarm signature. The reason why the countersfor all attributes are incremented when clicking on an alarm is that the systemhas no idea why a user clicks on an alarm. Since there is no way for it to guessthe reason, all it can do is to reward each attribute with one point.

This method of positive and negative feedback is less exact than the methodusing check boxes described above, keeping the element of guess from regularclicking. But, it is possible to add any number of attributes to the chosenmethod without any difference to the user, something not possible with checkboxes as described above. Our hope is that the ranking system will recognizewhat attributes are of importance when the user is able to give negative feedbackon alarms.

4.2 Time based weighing of votes

One problem with this set-up is that old votes influence the current rank asmuch as new votes. I.e. consider two alarms A and B. If A received 50 votessix months ago B would have to receive 51 votes before it is ranked higher bythe system. Most likely, this makes the ranking system slower and slower inresponse to changes in the current level of threat. As new vulnerabilities cropup and get patched continuously, this is not good. The ranking system shouldbe able to respond to new threats and give them a higher rank when relevant,and as the threat disappears, let its rank dissipate over time.

Another problem is that votes for an attribute might fluctuate a lot over time asFigure 4.4 shows. This poses a problem for the ranking system if it were to use

authors April 23, 2010).4 By external knowledge we refer to information such as: is the source responsible for many

attacks recently, has the destination host been present in many alarms recently.

38


0 5 10 15 20 25Time (t)

20

40

60

80N

umbe

r of v

otes

Original data

Figure 4.4: Figure illustrating how voting measurements can fluctuate over time.

the number of votes directly as a score for the attribute. The ranking they getwould be very inconsistent from day to day and very hard to predict. Insteadit would be much better to use some sort of average that has a slower rate ofchange.

To overcome the problem of equal influence of votes cast far apart in time andfluctuations in the number of votes, we use an Exponential Moving Average(EMA). EMA applies weights to data points (or votes in this case), where theweights will become smaller as the data points get older. The formula of EMAis defined as follows:

St = α× Yt + (1− α)× St−1

• Yt is the observation at a time period t.

• St is the value of the EMA at any time period t.

• The coefficient α decides the degree of influence the observed value hason the new EMA value. α is a constant smoothing factor between 0and 1. A higher α discounts older observations faster. Alternatively,α may be expressed in terms of N time periods, where α = 2/(N +1). For example, N = 19 is equivalent to α = 0.1. The half-life of theweights (the interval over which the weights decrease by a factor of two)is approximately N/2.8854 (within 1% if N > 5).

For a given set of observations {Y0, Y1, ..., Yn} this formula returns the EMAvalue St for any time period t ≥ 1. S1 can be initialized in a number of differentways of which the most common is to set S1 = Y0, but a more accurate waywould be to set S1 to be the average of the first 4 or 5 observations. However,

39

4.2. TIME BASED WEIGHING OF VOTES

0 5 10 15 20 25Time (t)

20

40

60

80

Num

ber o

f vot

es

Original dataEMA-5EMA-10EMA-20

Figure 4.5: Plotted curves showing the effect of using different smoothing fac-tors.

the error created by setting S1 = Y0 dissipates over time; a smaller α makes thechoice of S1 relatively more important than a larger α value.

The EMA formula consists of two parts; the first part that calculates the influ-ence of current observations (α× Yt), and the second part which calculates theinfluence of old observations ((1 − α) × St−1). The value α and 1 − α decidesthe influence that each part has on the new EMA value. These two parts arethen added together to form the new EMA value.

The old observations are represented with only one value (St−1) in this formula,which is the previous EMA value that was calculated. However if we expandSt−1 in the second part of the formula we get the following:

St = α× Yt + (1− α)× (α× Yt−1 + (1− α)× (α× Yt−2 + (1− α)× (...)))

We see that it is a series of calculations where St−1 (when expanded) holds thevalue of all previous observations. So to be able to calculate St we first have tocalculate Y0, Y1, ..., Yt−1 until we finally get to St. This is the way it is goingto work in our ranking system. For each new observation of how the users votewe recalculate the EMA value using the current amount of votes, the previousEMA value, and an appropriate α value.

Figure 4.5 shows the behaviour of the EMA formula with three different values ofN . The line called original data5 is the data set {Y0, ..., Yn} that is used as inputto the formula. In our ranking system this would be a set of cumulative numbersrepresenting how the users voted for different attributes each day. EMA-5,EMA-10, and EMA-20 in the figure each represents values produced with the

5This data set is a series of fictional votes an attribute received over a period of time.

40


EMA algorithm using 5, 10, and 20 respectively as values for N in the algorithm.The larger N , the smaller the smoothing factor α is, and as a result, irregularitiesare smoothed out more than with a smaller N .

As we follow the black line we can see that the number of votes for the fictionalattribute fluctuates a lot between different time periods. This variation couldhappen for a number of different reasons, i.e. the attack might have some timeconstraint that makes it happen more on some days than others. This fluctua-tion is not good for our ranking system as it makes the rank very unpredictable.But when the EMA formula is applied the result is much smoother. The bigjumps and drops in the numbers disappears. As big jumps or drops happens inthe line named original data, EMA-5, EMA-10, and EMA-20 will move in thesame direction but not as much. This is because of the α value in the EMAformula, as it weighs the previous EMA value with the current value of theoriginal data line.

As an example we can look at the first point for each of the four lines. They allstart at 59 on the y-axis. The next point on the black line is at 72. To calculatewhere the next point for the EMA-5 line should be we use the following:

α =2

N + 1=

25 + 1

=13

S1 = α× Y1 + (1− α)× S0

= α× 72 + (1− α)× 59

=13× 72 + (1− 1

3)× 59

=13× 72 +

23× 59

= 63 +13

This gives us the second point of the EMA-5 line. The same calculations canbe made for EMA-10 and EMA-20.

Figure 4.6 shows another way to illustrate how much influence a given datapoint has after n time periods. At t = 0 the data point has as much influenceas it can have, it is worth 100% of its original value. As time passes it willlose more and more of its original value in an exponential manner. The linerepresenting EMA-10 will have lost about 50% of its original value after about3.5 days. This corresponds pretty well with the formula for calculating the halflife of an EMA value. With the formula this works out to: 10/2.8854 = 3.4657.

4.3 Implementation of whole system

The implementation of the ranking model is done on top of Snorby. Snorbyadds a few tables on top of the Snort database to support some of the additional

41

4.3. IMPLEMENTATION OF WHOLE SYSTEM

0 5 10 15Time

0

20

40

60

80

%

EMA-5EMA-10EMA-20

Figure 4.6: Figure showing how fast previous observations lose their influencein the EMA formula with three different weights. X-axis is the number of timeperiods since the observation occurred, and the Y-axis is the influence in percentthat the observation has over the EMA value.

features that it offers, i.e. the commentary system. To add the ranking model ontop of Snorby, the number of times a user gives positive and negative feedbackto an alarm has to be counted. We decided that the simplest way to store userfeedback was to add columns and tables to the existing database where Snortstores the alarms. This is accomplished by making two changes to Snorby; twobuttons (“+” and “-”) where added to each event (for positive and negativefeedback), and vote counters where created for each of the four attributes.

Each attribute (i.e. an IP address 10.0.0.5 or a port 80) will have several countersassociated with itself. Each counter will have a variable storing how many votesone attribute received during a day. These values will then be used in the EMAformula so the ranking system will be able to see trends in how the users ofthe system votes. Three different EMA values will be used to highlight votingchanges over short, medium, and long time periods. But before we can do thiswe have to take closer look at the database structure that Snort and Snorbyuses.

4.3.1 Snort database structure

The Snort database stores information about each malicious packet in a struc-ture where the different parts of each packet are saved in different tables. Itbasically stores the whole packet but divided up in its different parts. Our rank-ing system will however only be interested in a small part of this information.When a Snort sensor stores an event (alarms are called events) in the databasea unique identifier is generated for that event. This unique identifier, cid, is aninteger the sensor generates and increments for each new event the sensor stores

42


sidcidsignaturetimestamp

event

sig_idsig_namesig_class_idtot_clicks

signature

iptot_clickscreated_atupdated_at

dst_ip_relevance

sidcidip_srcip_dst

iphdrsidcidtcp_sporttcp_dport

tcphdrsidcidudp_sportudp_dport

udphdr

iptot_clickscreated_atupdated_at

src_ip_relevanceporttot_clickscreated_atupdated_at

dst_port_relevance

* *

11

idcountable_idcountable_typenum_clicksema_5ema_10ema_20created_atupdated_at

click_counters

11

**

1

* 1

1

1

1

1

1

1

*

1 1 1

* *

*

Figure 4.7: The Snorby database structure. This figure does not show the wholedatabase structure, but only the parts that are affected by our implementationof the ranking model. Items marked in red indicate tables and values added tothe already existing database structure.

in the database. To allow for more than one sensor to store events withoutrisking collisions between cid values, a sensor id, sid is stored with every eventas well. These two values (sid and cid) forms the primary key for all the tablesstoring information about captured packets. Figure 4.7 presents an overview ofthe Snort database structure. It also presents the tables that we have added tostore the relevant information for each attribute to accommodate the needs ofthe ranking system.

At the centre of this database structure is the event table. It stores the sid andcid of each packet as well as a signature id and date and time about when itwas captured. The rest of the information about a packet can be retrieved fromother tables in the database as can be seen in Figure 4.7. This figure does notshow all the tables that are available in the Snort database, but only the onesimportant for the ranking model.

43


Signature For each entry in the event table, there is a corresponding entryin the signature table as well, with the same cid and sid. The signature of analarm specifies which filtering rule in Snort matched the current packet, andconsequently caused it to be logged in the database. The signature table storesinformation such as the name of the attack, the id of the signature matching it,priority of the signature, and what version of the filtering rule in Snort capturingit. This can be useful if a faulty filtering rule is detected and rectified, as allevents that where captured with the bad filtering rule can be identified andremoved. However, the only information important in this table to the rankingmodel is sig id, the id of the signature that captured the packet.

Iphdr Snort more or less follows the OSI standard in how it saves the informa-tion about packets caught by one of its signatures. The iphdr table is responsiblefor saving the source and destination IP of each packet. It will always have anentry for every event that is logged in the database as all packets traversing thenetwork are IP packets.

Tcphdr Has only information from captured packets using TCP. This tablehas fields for all the information in a TCP header as defined by the OSI standard.The only information needed for ranking decisions in this table is the destinationport.

Udphdr Similarly to the tcphdr table, this table only saves information aboutpackets using UDP and has a column for every field in the UDP header. Thenecessary information here for ranking decisions is destination port.

4.3.2 How votes are counted

The database tables described above all save information on a per event basis,meaning there is no table that saves information for one specific IP address forexample. This is however exactly what the ranking model needs to do. It has tosave information for each of the four attributes: signature, source and destina-tion IP, and destination port. To count the votes we have added a table namedclick counters, which can be seen at the bottom of Figure 4.7. Each row in thistable represents a counter that keeps track of votes for one specific attribute,such as an IP address or a destination port. To connect these counters to signa-ture, iphdr, tcphdr, and udphdr we need three additional tables: src ip relevance,dst ip relevance, and dst port relevance. These three tables only store a primarykey (i.e. IP address in the case of src ip relevance) and the total vote count(tot clicks6) A new table to connect signature to click counters is not neededas signature already stores information about unique signatures. Every counterhas four values:

6tot clicks is never used in any of the ranking calculations, it is only there because it mightbe interesting to an operator to see what the total voting tally is for an attribute.

44


• num clicks: A cumulative value that saves the amount of “+” or “-” votesone attribute receives. If one attributes receives five pluses and four mi-nuses, the value of num clicks will be 1 since 5 + (−4) = 1.

• ema 5: EMA value with a period of 5 days. Adapts the fastest.

• ema 10: EMA value with a period of 10. Adapts twice as slow as ema 5.

• ema 20: EMA value with a period of 20. Adapts twice as slow as ema 10.

These click counters correspond to the variables Y and S in the EMA formula(section 4.2, page 39) because it stores information about both the raw votecounter (num clicks), and the EMA values (ema 5, ema 10, ema 20). Eachcounter is equivalent to one data point in Figure 4.5. A counter only collectsdata during 24 hours, after 24 hours a new counter is created for the sameattribute. This creates a set of counters that each represents votes cast forone attribute during one day. If a user votes for an event that does not have acounter that is less than 24 hours old a new counter is created. The EMA valuesin the old counter is used together with the current amount of votes stored inthe new counter (in this case 1, since it is new) as input to the EMA formulato create the new EMA values in the counter for that attribute. In the EMAformula these two input variables are called St−1 for the previous EMA value,and Yt for the current vote count. This process of recalculating the EMA valuesalso happens every time a vote is cast by the user, but a new counter is onlycreated if the current one is older than 24 hours.

These counters are only created as needed. When an alarm with attributes thathave never received votes before is clicked a new click counter is created for thatattribute automatically. It is done this way to save space in the database ascreating them all from the start would require a lot of space. Creating clickcounters for all possible IP addresses would require 232 = 4 294 967 296 entriesin the database, which we do not have room for.

4.3.3 Ruby on Rails

Ruby on Rails (often referred to as Rails) facilitates retrieving information fromthe database by creating a so called model of each table in the database. Modelsare classes in Rails with automatically generated methods meant for retrievingand updating data in the table. Each column of the table gets its own set ofmethods. I.e. a table called persons will have a class called Persons in the Railscode, and each column of the table will have a method (i.e. name). Relationsbetween different tables in the database will also be represented in a similarmanner, but here the developer has to define them in the class definition. I.e.the event table is represented by an object Event in Rails that has methods forevery database relation of the table. Calling Event.signature returns a Signatureobject corresponding to the row in the signature table the particular event hada relation with.

The relations between different tables in the database are described in the cor-responding Rails objects by using a function call. There are four basic functioncalls that can be used:

45


• has_one

• has_many

• belongs_to

• has_and_belongs_to_many

These four methods can be used to describe one-to-one, one-to-many, and many-to-many relationships in the database. To describe a relationship between atable containing books and a table containing the libraries that the books belongto would look like this:�

1 class Library < ActiveRecord : : Base2 has many : books3 end4

5 class Book < ActiveRecord : : Base6 be l ong s t o : l i b r a r y7 end� �

The object Library is defined with a call to has_many with argument :books todescribe its relationship with another object Book. The fact that the argumentis :books with a plural ‘s’ is interpreted as a reference to the object Book andnot Books, the ‘s’ just makes the code more readable. On the other end of therelationship the class Book has a call to the method belongs_to with an argu-ment :library to signify that each Book object belongs to one specific Libraryobject. These methods also involve some magic that make it convenient whendesigning a web-application with Ruby on Rails from the start. For exampleall methods defining relationships make a few assumptions about what columnsare present in the tables. The has many method assumes the foreign key in thetable containing books is named library id. If the foreign key has some othername it can be specified as an argument in the method call.

In the case of the tables that have been added to the Snort database, therelationship that src ip relevance has to iphdr would look like:�

1 class SrcIpRelevance < ActiveRecord : : Base2 has one : iphdr , : f o r e i gn k ey => : s r c i p3 . . .4 end5

6 class Iphdr < ActiveRecord : : Base7 be l ong s t o : s r c i p r e l e v a n c e8 . . .9 end� �

This ties the primary key column (named ip) of the src ip relevance table tothe column named src ip in the iphdr table. Rails then knows that all rowsin the iphdr table belongs to one specific row in the src ip relevance table.dst ip relevance and dst port relevance have equivalent relationships defined in

46


their object definitions. Attached to all of these tables is a click counter tableas can be seen in Figure 4.7. This table has what is called a polymorphicrelationship with four tables above it. That means that all of these four tablescan have a relationship with the click counter, and use their respective primarykey as a foreign key in the click counters table. A polymorphic relationship isdefined as follows:�

1 class ClickCounter < ActiveRecord : : Base2 be l ong s t o : countable , : polymorphic => true3 . . .4 end5

6 class SrcIpRelevance < ActiveRecord : : Base7 has many : c l i c k c oun t e r s , as => : countable8 . . .9 end

10

11 class DstIpRelevance < ActiveRecord : : Base12 has many : c l i c k c oun t e r s , as => : countable13 . . .14 end� �

On the second row in the definition of ClickCounter, every object of the classClickCounter is defined to belong to something called :countable. Countableis not a class, but a name for a group of different classes that want to usethis class to store information in its table. On the other side of the relation-ship SrcIpRelevance has a has_many relationship with the class ClickCounter,and with a second argument as => :countable. This tells Ruby on Rails thatSrcIpRelevance wants to belong to the polymorphic relationship named “count-able”. Ergo, the four classes Signature, SrcIpRelevance, DstIpRelevance, andDstPortRelevance can all have a relationship with the same ClickCounter classinstead of having a unique click counter class for each of the four attributeclasses. Rails makes this possible in the database by having an extra column inthe table click counters, in addition to the foreign, key that saves the name ofthe class owning the object/row. This allows other classes to use their primarykey as a foreign key in the click counters table without causing any collisionsbetween foreign keys from different classes.

4.3.4 Counting Votes

Snorby displays a page listing events from the Snort database in the order cap-tured by Snort (shown in Figure 4.8). To allow the user to cast votes on eventsshe thinks are important two buttons have been added, “+” for when the alarmis interesting and “-” for when it is not. Because the vote is cast on an eventand not on each specific attribute, the ranking system cannot know if one, all,or a subset of the attributes are interesting. We decided on the most simplesolution which is to share the vote evenly among the attributes of the alarm.For each vote an alarm receives, each attribute’s score will be incremented ordecremented by one point. Not deciding on individual ratios for each attribute

47


Figure 4.8: The event view in Snorby showing events fetched from the Snortdatabase sorted in the order they where captured.

at this point, i.e. giving source ip two points and destination port only 1, givesus the opportunity to multiply the score later and produce the same results.I.e., if there is an alarm:

Signature: Port scanSource IP: 230.42.25.6

Destination IP: 105.200.156.2Destination Port: 80

“Port scan”, “230.42.25.6”, “105.200.156.2”, and “80” would each get one pointadded to their counters if the user pressed the “+”-button. The vote affectsfour different values in the attributes counter: num clicks, ema 5, ema 10, andema 20. num clicks stores an integer value that is incremented or decrementeddepending on which button was pressed. It starts out with a zero meaningnegative values in a counter are possible. The pseudo code for updating thethree EMA values looks like:�

1 def next ( per iod , current , prev ema )2 mu l t i p l i e r = (2 . 0 / (period+1))

3 return multiplier * current + (1 - multiplier) * prev_ema

4 end� �The next function above follows the EMA algorithm presented on page 39,and produces the next EMA value in the sequence. This function takes threearguments, period is the value controlling the rate at which the EMA value

48


decreases. Second one is current which is the vote count for the attribute atpresent, and prev ema is the previous EMA value for the attribute. Thesethree variables correspond to N , Yt, and St−1 respectively in the EMA formula.Period is used to calculate α in the EMA formula. If a previous EMA valuedoes not exist the EMA value will have the same value as the vote count forthe current counter. This function is called for all three of the EMA values foreach attribute in the alarm that received a click on their “+” or “-” button.

To be able to see how the operator’s interest in different attributes changes overtime, counters are only active for one day, as described in Section 4.3, after thatnew ones are created and the old ones will remain as inactive counters in thedatabase. When a vote is cast on an event and there are no recent counters forthe attributes of that event, new ones will be created. When that happens, thevalues in the old counters are used for calculating the next EMA values for thenew counters.

4.3.5 Displaying the alarms

The EMA values calculated above are used to sort the events in three differ-ent ways, one for each EMA value that each attribute stores. This gives theuser three different lists displaying alarms according to the perceived impor-tance the operator herself has contributed to with her votes. The three sortingalternatives are called “Short”, “Medium”, and “Long”. Short means that itis easily influenced by changes in the short time span. Medium sees changesover a medium time span and long over a long time span. Short uses the valueEMA-5, Medium that of EMA-10, and Long EMA-20.

The actual sorting works as follows. The sorting function is presented with alist with the most recent alarms, in our implementation it is limited to the 100most recent alarms. Each of the alarms has voting information stored for eachof the four attributes that we are interested in. This voting information includesEMA-5, EMA-10, and EMA-20 values. In the case of the short sorting orderthe sorting function looks at the EMA-5 value for each of the four attributesin each alarm. These four EMA-5 values are then added together to form thescore for that alarm. This score is then used by the sorting function to sortthe alarms in the list. The sorted list will then contain all the alarms sortedaccording to their EMA-5 or short sorting order score.

If the operator chooses the Short sorting alternative, alarms high up in the listwill recently have received votes and therefore have a very short term relevanceto the operator. The Medium sorting alternative is a little more long term andthe sorting order will not be affected that much by a lot of votes in the shortterm. Long is even more robust and should reflect what the operator thinks isimportant over a very long time span.

49


50

Chapter 5

Simulations

To evaluate our ranking methodology a series of experiments were performed.A Ruby program was developed for this purpose. As they would be a proofof concept it was decided to start out with a simple base case, which couldeasily be extended by demand. It consists of two parts, 1) an alarm generator,presented in Section 5.1 and 2) a click simulator, presented in Section 5.2. Withconfiguration we can direct the simulation and let it emulate different types ofnetworks and scenarios.

5.1 Alarm generation

The alarm generator generates a list of random alarms with different sets ofattributes. To make it more realistic it uses a configuration which specifiesratios for how likely an attribute is among the generated alarms. Based on ourexpertise, a low security risk signature might for example be more common thana high security risk signature.

The configuration variables pertaining to alarm generation:

• number of alarms,the number of alarms to be generated during a test run.

• alarm ratio high,alarm ratio medium,alarm ratio low,decides the ratios between severities in the number of alarms generated.If high = 15%, then 15% of the number of generated alarms should havean alarm type classed as a high severity type. The total ratio has to addup to 100%.

• number of destination IPs,the number of destinations the generator can choose from when choosinga value for the destination IP attribute of the alarm.

51

5.2. SIMULATING CLICKS

• number of source IPs,the number of sources the generator can choose from when choosing avalue for the source IP attribute of the alarm.

• number of alarm types,the number of alarm types the generator can choose from when choosinga value for the alarm type attribute of the alarm.

• severity ratio high,severity ratio medium,severity ratio low,the ratios of the alarm type severities. Snort labels its alarm types tohave either a high, medium or a low severity. The ratios here decide howmany of our total number of alarm types should be classed as severityhigh, medium and low. The total ratio has to add up to 100%.

As it is difficult to understand the difference between alarm ratio and severity ratioat first, we will now give an example to further explain their functions. Assumethat the following variables have values:

• number of alarms = 10

• alarm ratio high = 20%alarm ratio medium = 30%alarm ratio low = 50%

• number of alarm types = 8

• severity ratio high = 25%severity ratio medium = 50%severity ratio low = 25%

severity ratio x decides how the different alarm types is labeled in terms ofseverity. In our example above, this would mean that out of the eight differentalarm types, 1-8, types 1 and 2 are classed as types of high severity, types 3-6 astypes of medium severity and the remaining two types, 7 and 8, as types of lowseverity. alarm ratio x decides how many of the generated alarms should beof what severity. In the example this results in that two out of the ten generatedalarms should have an alarm type classed as being of high severity. Those twoalarms can then be of either type 1 or 2. Three out of the ten generated alarmsshould have a type classed as being of medium severity. Those three alarms canhave either of the types 3-6. Lastly, the remaining five generated alarms canhave either of the types 7 or 8.

5.2 Simulating clicks

The configuration also allows us to specify how many of the attributes shouldbe considered important in the simulation. This is used later in the simulation,after the random list of alarms has been generated, to simulate which alarms theoperator of the system clicks on. All alarms in the list of generated alarms thathas one or more important attributes will receive a “click”. Each click gives apoint to all the attributes of that particular alarm. The points accumulated for

52

CHAPTER 5. SIMULATIONS

each individual attribute will later determine the score that each alarm receives.The score of an alarm is determined by simply adding up the points for eachindividual alarm as follows:

Attribute PointsSource IP 2Destination IP 10Destination Port 8Alarm type 7Total score 27

Figure 5.1: An example showing how the score for an alarm is calculated.

This score is used to sort the alarms in order of importance. However this wayof sorting does create some anomalies, which we will return to in the Resultschapter (6.1). The anomalies happen when an alarm with less than four im-portant attributes receives a click, because it will also reward any unimportantattributes of the alarm with a point. Unimportant attributes will by this pro-cess receive points simply by the fact that they are associated with an alarmcontaining important attributes.

• source IP click worth,destination IP click worth,alarm severity high click worth,alarm severity medium click worth,alarm severity low click worth,decides how many points each attribute gets added to its ranking scorewhen an alarm is clicked.

• important destinations,how many of the destination IPs should be labeled as interesting to theuser and be clicked when present in an alarm.

• important sources,how many of the source IPs should be labeled as interesting to the userand be clicked when present in an alarm.

• important type high,important type medium,important type low,how many of the alarm types with a certain severity should be labeled asinteresting to the user and clicked when present in an alarm.

5.3 Details of configuration

The configuration behind the tests were set up in collaboration with Ola Soder-strom to as much as possible mimic a real life scenario.1 The numbers repre-

1 Ola Soderstrom(security analyst, Omegapoint, e-mail correspondence on October 28,2010), see appendix D.2.

53

5.3. DETAILS OF CONFIGURATION

senting IPs and alarm types are chosen by us as a means to simulate how clickswould work and have nothing to do with real IP addresses or alarm types.

Basic configuration:

• number of alarms = 100,since 100 is a nice number to have when calculating percentages.

• number of destination IPs = 50,we have decided to emulate a smaller network.

• number of source IPs = 100,logically, the number of external IPs has to be larger than the number ofinternal. We decided on 100 which is more than 50 but still small enoughto produce alarms with the same source.

• number of alarm types = 50,not at all reflecting the actual number real IDSs uses, but a number smallenough to result in alarms having the same type but large enough to givesomething to choose from when generating alarms.

• severity ratio high = 15%,severity ratio medium = 40%,severity ratio low = 45%,15% of the alarms types should be classed as high severity alarms, 45% asmedium severity and 40% as low severity. These numbers are coupled tothe distribution of severity among the alarms.

• alarm ratio high = 15%,alarm ratio medium = 40%,alarm ratio low = 45%,15% of all generated alarms should have an alarm type of high sever-ity, 40% should have an alarm type of medium severity and 45% shouldhave one of low severity. These numbers are chosen based on the directexperience of Soderstrom.

• source ip click worth = 1,destination ip click worth = 1,alarm severity high click worth = 3,alarm severity medium click worth = 1,alarm severity low click worth = 1,when an alarm is clicked, 1 is added to the weight of the destination,source and alarm, with the exception of an alarm of high severity. Inthat case 3 is added to the weight. This is to counter the effect of thesmaller ratio of high severity alarms. In a real system, one has to analyzethe ratios between the different severities before configuring the weights.It is also possible to let the system continuously monitor the ratios andautomatically update the weights.

The initial goal was to reach a clicking frequency of 10-15%, a number Soder-strom usually averaged. However, this was not possible without leaving anattribute unclicked resulting in the following configuration of how to simulateclicks:

54

CHAPTER 5. SIMULATIONS

• important destinations = 4%,4% of the possible destination IPs are selected as important. If a generatedalarm has a destination IP selected as important a click will be simulatedon that alarm and the weights associated with the destination IP, sourceIP and alarm type will all be incremented. This is the smallest percentageresulting in more than one IP being chosen important.

• important sources = 2%,2% of the possible source IPs are selected as important. If the source IP ofan alarm is important the attributes are incremented. This is the smallestpercentage resulting in the system choosing more than one source IP tobe important.

• important type high = 75%,important type medium = 10%,important type low = 5%,75% of the alarm types chosen to be of high severity by severity ratioare selected as important and of interest. These will be clicked if theyshow up in a generated alarm. The same goes for 10% of the alarms ofmedium severity and 5% of the ones of low. 5% is the smallest percentageimportant type low can have and still result in one alarm of severity lowbeing chosen as important. Feeling that alarms of medium severity shouldbe more interesting than alarms of low severity we chose to mark 10% ofthe medium severity alarm as important. This results in more importantalarms than those of low severity but still keeping the number low enoughnot to produce too many important alarms. An alarm of high severityis surmised to most often be clicked. Too low a percentage would seemsilly and too high would generate too many clicked alarms. A compromisebetween the two settled the percentage on 75.

We are aware that the tests do not truly mirror real life. In reality one doesnot judge an alarm important based solely on source IP, destination IP or typeof alarm but at the very least a combination of the three. The attribute sourceIP might as well be interchanged with the triple (source IP, destinationIP, alarm type). The reasons for clicking on an alarm can be complex, de-pending on a range of possibilities, a range we have no possibility to emulate.These tests are here as a proof of concept - that ranking is a viable means tohelp the user manage alarms. We have therefore chosen three simple attributes.

By the same reasoning, we have not simulated the effects of down prioritizingan alarm. As a proof of concept we wanted to keep the experiment simple andconcentrate on only one factor. More importantly, we do not feel there is anyneed for the feature since the user would only utilize this if her interest changesover time or she does something wrong. As the tests are clicked by the programand no time elapse between the alarms, the conditions under which an alarm isdown prioritized are not replicated.

55

5.3. DETAILS OF CONFIGURATION

56

Chapter 6

Results

Here we present the results from Chapters 4 and 5. Section 6.1 presents theperformance of our simulation runs viewed from two angles; how well alarmscarrying important attributes are separated from those who do not and how wellthe ranking order reflects the true severity of the alarms. Section 6.2 presentsthe changes done to Snorby.

6.1 Simulations

Using the configuration desribed in Chapter 5 a series of test runs were per-formed. To smooth the effects of diverging test runs an average of 10 test runswas used. The number of alarms with simulated clicks spans from 22 up to 28averaging 26, making this fairly homogeneous.

Run 1 2 3 4 5 6 7 8 9 10 average# clicked alarms 27 28 28 27 25 25 22 28 24 26 26

Table 6.1: Number of important alarms generated each test run.

The alarm generation ratios given to the click test simulation (destination IP,source IP, and alarm type), resulted in 6 IPs and 8 alarm types being chosen asattributes to be clicked. Table 6.2 summarizes the result.

Dest IPs : 1 - 3Src IPs : 51 - 53alarm high : 1 - 5alarm medium : 8 - 9alarm low : 30

Table 6.2: The IPs and alarm types chosen as important. Whenever an alarmshows up carrying any of these attributes, it gets a simulated click.

57

6.1. SIMULATIONS

Two words frequently recurring within this chapter is important alarms/clicked alarms and unimportant alarms/unclicked alarms.

Definition 3. An important alarm, also referred to as a clicked alarm,is an alarm with attributes of interest to the user. When the user spots thisalarm, she is surmised to click on it, showing her interest.

Definition 4. An unimportant alarm, sometimes referred to as an unclickedalarm, is an alarm lacking attributes of interest to the user. When the userspots this alarm, she is surmised to leave it unclicked, showing her disinterest.

To explain the result, two aspects are highlighted: the density with which theimportant alarms are presented and the ordering.

6.1.1 Density

To be able to understand what is meant by density it has to be defined first.

Definition 5. The density D(i), of ranked alarm position i, is the ratio ofimportant alarms found at positions 1 up to i compared to the total number ofalarms found at positions 1 up to i, where i ≤ position of last listed important alarm.

D(i) =# important alarms at positions 1 to i

# alarms at positions 1 to i

The density after all important alarms have been listed remains the same, nomatter how many unimportant alarms are listed beyond that. This is becausedensity is a measure for how quickly the important alarms are listed. Once theyare all listed, how the rest of the list looks like is unimportant.

Out of the 10 test runs only test run 1, 2, 5 and 10 yielded a density less than100%, meaning that these four runs were the only runs not able to list the im-portant alarms without interspersing them with unimportant alarms. Table 6.3shows the density of a test run averaged from all ten test runs. Positions, i areshown in italics and density, D(i), in verbatim.

On average, the alarm listed on the first position is an important alarm. In the

first 10 positions, 10 important alarms have been listed. D(10) =1010

= 100%.Since this is the average of 10 test runs it is obvious that in all cases, in the first18 positions, all alarms are important. The first change shows up at position19 where the density on average is 99.5% instead of 100%. This means thatin one run out of the ten, an unimportant alarm is listed at index 19. Moreunimportant alarms are interspersed with importants alarms in the followingpositions. This can be deduced from from the decreasing density.

Figure 6.1 shows how well the ranking is able to sort alarms and separate im-portant ones from unimportant. The x-axis represents the positions of the listedalarms. The y-axis represents the density of the important alarms in the first xpositions. Test runs 3, 4, 6, 7, 8 and 9 all managed to achieve a density of 100%and are stacked on top of each other following y = 100. The curves achieving

58

CHAPTER 6. RESULTS

i D(i) i D(i)1 : 100% 18 : 100%2 : 100% 19 : 99.5%3 : 100% 20 : 99%4 : 100% 21 : 99%5 : 100% 22 : 99.1%6 : 100% 23 : 99.1%7 : 100% 24 : 98.8%8 : 100% 25 : 98.8%9 : 100% 26 : 98%

10 : 100% 27 : 97.4%11 : 100% 28 : 97.5%12 : 100% 29 : 97.2%13 : 100% 30 : 95.6%14 : 100% 31 : 96.315 : 100% 32 : 96.4%16 : 100% ... : ...17 : 100% 100 : 96.4%

Table 6.3: Density of important alarms listed after i positions.

less than 100% are from bottom up: test run 1, 2, 10, 5 and the average. All ofthem starts to drop off somewhere in the interval between position 20 and 25.The bump in the curves, most clearly visible in run 2, means that importantalarms was listed once again. This means that the density rises. Once the graphlevels out, all important alarms of that test run (see Table 6.1) have been listed.

Figure 6.1: Density of all ten test runs and the average after ranking.

59

6.1. SIMULATIONS

Figure 6.2: Density of all ten test runs and the average before ranking.

Figure 6.2 shows the densities of the alarms before ordering them after rank.The graph is quite messy but a definitive trend is discernible. As it is purechance when an important alarm shows up, the curves covers a larger intervalin the beginning, 0% ≤ y ≤ 100%. But they level out as the position gets closerto x = 100 and y stabilizes in the interval 20% - 30%. This is expected sincethe average number of important alarms is 26 in a set of 100.

6.1.2 Ordering

The density, how well the important alarms are listed together without beinginterspersed with unimportant alarms, does not tell anything about the orderof the alarms. The clicked alarms can be ordered in a way not reflecting thetrue importance of the alarms, putting the least important clicked alarm at thetop and the most important clicked alarm right above the first unimportantalarm and still achieve 100% density. Since there is no way of averaging theplacement of an alarm, figures of a few illustrative runs were chosen to indicatethe ordering.

Figures 6.3, 6.4 and 6.5 show how well the ordering of test run 2, 4 and 7matches the optimal ordering.1 The x-axis, represents the positions in the list.The y-axis represents the order the alarms should have. The optimal dots show

1The optimal order is simulated by only awarding points to important attributes when analarm receives a click. This results in a ranking score only regarding the attributes of interestto the user. As no unimportant attributes are awarded points, they cannot contribute to therank as in our ranking system. The optimal order is, in other words, the way the user wouldhave ordered them. Note that the optimal order only refers to the order of important alarms,it does not pertain the unimportant alarms.

60

CHAPTER 6. RESULTS

Figure 6.3: Ordering of alarms of test run two.

Figure 6.4: Ordering of alarms of test run four.

how the alarms really should be ordered. The first position of the list, x = 0, iswhere the most important alarm should be found, y = 0. In the second positionthe second most important alarm should be found, in other words, the optimalcase matches y = x. The “unsorted” dots represent the unsorted alarms and inFigure 6.3 the first alarm to be listed (x = 0) is really the 64th most important(y = 63). This is quickly followed by an alarm that should have been at the end

61

6.1. SIMULATIONS

Figure 6.5: Ordering of alarms of test run seven.

of the list and then by one which should have been first. From this it is easy tosee the closer the curve follows the optimal case the better, and the farther offthe curve is the more the alarms are out of correct order.

The “ranked” dots representing the ranked case follow the optimal more closelythan the unsorted case, which is not surprising as the ranking here has onaverage 26 alarms to rank and does so in at most 32 listings (see Figure 6.6).Although not fluctuating as wildly as the the dots representing the unsortedcase, it still fluctuates meaning that the ranked alarms are not in the correctorder. The alarms ranked in test run 4 and 7 have orderings closer to the correctone, than that of test run 2. After around 30 positions, the “ranked” dots losetheir somewhat symmetry to the optimal ordering and start to look like the“unsorted” dots. This happens when all the important alarm have been listedand those left to list is of no importance, hence the ordering is of no interest. Thegraphs are cropped at x = 50 to display the behaviour after all the importantalarms have been listed. The data points between x = 51 and x = 100 are leftout since the same behaviour as in 35 ≤ x ≤ 50.

Another way to illustrate the order is to focus on the rank given to an alarm foreach of the ten test runs. In Figure 6.6, the intervals of the 12 most importantalarms are tracked (read more about box plots in appendix A). The x-axisrepresents the position resulting from the ranking function, the y-axis representsthe position the alarm really should have according to the reason behind theclick. In Figure 6.6 the alarm which should have been first in the list, y = 1,is ranked anywhere between first and 12th with a concentration around thefirst four positions. The alarm which should have been ranked second mostimportant, is most often placed as fifth.

62

CHAPTER 6. RESULTS

Figure 6.6: Intervals of the twelve most important alarms.

6.2 Snorby

The visual changes that have been made to Snorby are quite small and notobvious if the user is unfamiliar with the interface. The reason behind thisis that our goal was to design an interface helping our ranking system butthat at the same time does not interfere too much with the user’s normal wayof interacting with the interface. Figure 6.7 shows the event view in Snorbybefore the modifications, a list of events retrieved from the database whereSnort stores captured packets. Each grey box represents an event with somebasic information about the event, such as name of signature and source anddestination IP. The order in which the events are displayed is the order in whichthey where captured. If the user clicks on one of these events, a page showingmore detailed information about that alarm is displayed.

In the lower left corner of each grey box there is a small yellow box with threeicons; a speech bubble, a star, and a red circle with a white cross. These threeicons are buttons letting the user perform three different tasks; view comments,mark event as favourite, and delete event from database. In Figure 6.8 two newbuttons are present, a “+” and a “-”. These two buttons are what gives the

63

6.2. SNORBY

Figure 6.7: Unmodified version of Snorby displaying the event view.

user the capability to vote for which events she thinks are important. Clickingon “+” tells the ranking system that this alarm and its attributes are importantto the user, a click on “-” has the opposite meaning.

Figure 6.9: Three links that allow the user to choose how alarms should besorted.

Figure 6.8: The buttons that whereadded to the Snorby interface.

Figure 6.9 shows the menu that isadded to the top right corner ofthe event view page. This menuwas added to allow the user tochoose which of the three EMA val-ues that should be used when sort-ing the alarms. “Short”, “Medium”,and “Long” refer to how much in-fluence votes cast a long time agoshould have when sorting the alarms.“Short” uses a shorter time window,“Medium” uses a little larger window, and “Long” will take into account votesthat where cast even longer ago. There is also a link called “Default” resetting

64

CHAPTER 6. RESULTS

the sorting order to the standard order that Snorby uses, in this case sorted bydate in descending order.

65

6.2. SNORBY

66

Chapter 7

Discussion

7.1 Ranking system

The ranking function seems to be performing quite well. Its performancematches optimal density in 6 cases out of 10. In the other four cases the den-sity drops due to unimportant alarms being listed before important ones. Whyunimportant alarms get listed among the important ones goes back to how clicksare handled. As the system has no way of knowing why the user clicks on aspecific alarm it can only add to or subtract from the current score of the at-tributes concerned by the alarm. A phenomena we call collateral rankingis thus created. Unimportant but common attributes get a high ranking scoreonly due to association, as they are often present in alarms with other attributesviewed as important.

Definition 6. Collateral ranking is the act of awarding unimportant at-tributes a higher rank due to their association with important attributes. Thatis, unimportant attributes get points by simply being present in the same alarmsas important attributes.

The phenomena of collateral ranking is even more clear when the ordering isstudied. In Figure 6.6 it is obvious that collateral ranking affects ordering evenmore than the density. The bulk of the box plots hover in the main vicinity ofthe expected placement but the extremities go much farther off. In our smallscale experiment of 100 alarms the effect is not overly drastic but if this were tobe scaled up to 1000 alarms perserving the trends evident in our experiments,the interval of the most important alarm would grow proportionally from [1, 12]to [1, 120]. Figures 6.3, 6.4 and 6.5 show the variation one can be expectedto deal with. Depending on the alarms, their distribution and the clicks of theoperator(s) can end up with a ranking following the expected ordering fairly well,or one can get a ranking having little in common with the expected ordering.

How well the phenomena of collateral ranking translates into a real scenariowe do not know and see the need to make studies in a real environment before

67

7.1. RANKING SYSTEM

definitive conclusions can be drawn. The alarms in our tests are randomlygenerated without internal connections which is not the situation in the realworld. The whole practice of the methods mentioned in Chapter 3.2 is basedon the assumption that alarms are not random. We do not know if the resultsobtained are in any way skewed by the data and tests on real data have to beperformed to see if they support our findings.

7.1.1 Issues outside of the system

The model tracks three attributes in the simulations and four in the implemen-tation in Snorby which is a minuscule subset of all the attributes possible totrack. The question is whether the trends found here manifest themselves in amore complex model, tracking more attributes. Will the ranking become moreaccurate in presenting the expected order or will the collateral ranking be am-plified and wreak even more havoc? As Ola Soderstrom described, alarms arenot clicked because of a single attribute, instead it is often a combination ofattributes which make an alarm interesting.1 These kinds of conditional clicksis not something our model respects. To explicitly tell the system under whatconditions attributes are important, it might be a good idea to apply a datamining technique to the clicked alarms to find relationships between them. Thediscovered relationships can then be displayed for the user and she can eitheragree or disagree with the estimated relationship.

In our ranking system, it is absolutely vital that the operator knows what sheis doing. Voting for unimportant alarms that at first seem important or votingagainst important alarms that at first seem unimportant might lead to the wrongalarms to be highlighted and the alarms needing the most attention being hiddensomewhere at the bottom of the list. Therefore, overconfidence in the rankingis dangerous no matter how good an analyst one views oneself to be, since thereis no way to check if the system ranks alarms correctly.

A precise level of exactness will probably never be attained with a rankingsystem producing estimates based solely on user input. A remedy could be tohave configuration files containing information otherwise difficult to convey tothe system. Difficult information might be conditional clicks, e.g. alarms of thefollowing type: C is only interesting if destination IP = 3, port = 79 andprotocol = UDP. The problem with configuration files is maintenance. Theyhave to be kept up to date to be of any use to the system. Configuration can beanything from minimal, only keeping a few choice decisions on how to rank, orit can be huge (see Porras et al. [30]) containing every detail. Less configurationand more on-the-fly results in a system easy to start using but less exact. Moreconfiguration and less on-the-fly results in a system more exact in its rankingbut much more difficult and time consuming to launch and maintain. Therewill always be a trade-off.

1Ola Soderstrom (security analyst, Omegapoint, interviewed in person by the authors April23, 2010).

68

Chapter 8

Future Work andConclusions

8.1 Future Work

The most obvious and given task in future work is to test our model using morerealistic data and see how it performs.

Our ranking system has a number of weaknesses that are in part inherent tohow it fundamentally works, but it also has a few weaknesses which probablycould be mended with some additional features. Even if an alarm signature iscompletely uninteresting and is categorically down voted, it still might get a lotof points just by association - collateral ranking. This could be mended withan administrative console which allows the operator to manually lock a certainattribute at a lower rank. The opposite is also attractive, allowing the user toraise the rank of an alarm drastically if the need presents itself.

Collateral ranking is also the perpetrator of how the expected order of alarmsis reflected in the ranking. Can this be coped with by adding more attributesto follow?

Point adjustments could also be made depending on which version of a signa-ture generated the alarm. Snort has a column called sig ver in the signaturetable, which saves the version number of the signature that generated the alarm.Adjusting the points awarded depending on the version might be of interest incertain scenarios. A signature is maybe discovered to be causing a lot of falsealarms and patched as a result. The updated signature will receive a new ver-sion number. A penalty can then be attached to the old version of the signatureto lower the rank of all the alarms which where captured with the old version.How useful this would be also depends on how new signatures are deployed. Ifthey are first tested on one Snort sensor while the others continue using the oldone, then it might be of use. But if all Snort sensors start using the updatedversion immediately alarms generated by the old defunct signature will becomeobsolete quickly and giving them a lower score might not have any effect at all

69

8.2. CONCLUSIONS

on the priority of the alarms.

A more advanced addition to the ranking system would be to look at votes ina wider context, as chain of votes. The user of the system might place voteson alarms similar to each other in some regard. These similarities would be ofinterest to the ranking system as they can give more information about whythe user is prioritising one alarm with signature A but not another with theexact same signature. The most recent alarms receiving votes could then beanalysed to find what common traits they have. The user might only prioritizealarms with signature A if the destination IP is 10.0.0.1 and port is 80. This issomething our ranking system at present cannot handle.

8.2 Conclusions

We have studied different techniques to help the user of an IDS manage alarms.Among the techniques for processing alarms, we find aggregation, correlation,filtering and ranking. Out of these, ranking was chosen for an in-depth analysisdue to its many possibilities of being tweaked through normal interaction withthe user. Based on our analysis, we have implemented a ranking system in thearchitecture Snort/Snorby that the user is able to interact with. Our systemtracks four attributes of each alarm: source and destination IP, destination portand alarm signature, and keeps a ranking score for each. The alarms are rankedbased on the total score of these four attributes.

The ranking system seems to be performing quite well based on our generatedtest data. The alarms carrying important attributes are all ranked at the top ofthe list presented to the user. This gives the user a much smaller interval to scanfor important alarms and lets her detect alarms of importance faster. There areno guarantees that the alarms will be sorted according to actual importance,though. The ordering is dependent on how often different attributes occur inalarms and the alarms of highest importance might not occur as often as othersand therefore end up lower down among the important alarms. This behaviouris credited to the phenomena we call collateral ranking (defined on page 67).

This is not a system for operators in training. No guidance is given to the userabout alarms and the user needs to be knowledgeable in this area to be ableto properly use the ranking system. Prioritizing alarms not having any realimportance and down prioritizing crucial alarms will severely skew the final listand might lead to important alarms being missed as they are located at thebottom of the list.

The data used in the simulations is randomly generated and does not originatefrom any real world context. The results therefore have a certain amount ofuncertainty about them. Before any work is done expanding this model, itwould be beneficial to test it in a live network.

Still, our model is based on user input in simple terms and might not catch themore complex reasons why the user decides an alarm is important or unimpor-tant. The phenomena of collateral ranking also disturbs the result. We proposethe use of configuration files but we have not formally tested this. The prob-

70

CHAPTER 8. FUTURE WORK AND CONCLUSIONS

lem with configuration files, though, is the generally time consuming work ofmaintaining them and keeping them up to date. This will always be a trade-off,with less configuration and more on-the-fly operation the system will be read-ily maintained but with a certain amount of uncertainty about it. With moreconfiguration and less on-the-fly operation the system will be more exact in itsranking but harder to maintain.

Overall, our ranking system ranks alarms with minimal configuration and inputfrom the user and shows promising results. We believe it will be a great assetfor intrusion detection operators in their daily work.

71

8.2. CONCLUSIONS

72

Bibliography

[1] Raven Alder, Jacob Babbin, Adam Doxtater, James C. Foster, TobyKohlenberg, and Michael Rash. Snort 2.1 Intrusion Detection, second edi-tion. Syngress Publishing Inc., 800 Hingham Street, Rockland, MA 02370,2004.

[2] Magnus Almgren and Erland Jonsson. Tuning an IDS - learning the securityofficer’s preferences. In 11th Nordic Workshop on Secure IT Systems -NordSec06, pages 43–52, Linkoping, Sweden, 2006.

[3] Stefan Axelsson. The base-rate fallacy and the difficulty of intrusions de-tection. ACM Trans. Inf. Syst. Secur., 3(3):186–205, 2000.

[4] Cisco. Cisco MARS. http://www.cisco.com/en/US/products/ps6241/index.html.

[5] Chris Clifton and Gary Gengo. Developing custom intrusion detectionfilters using data mining. In Military Communications International Sym-posium (MILCOM2000), pages 440–443, 2000.

[6] Frederic Cuppens and Alexandre Miege. Alert correlation in a cooperativeintrusion detection framework. In Proceedings of the 2002 IEEE Symposiumon Security and Privacy, pages 202–, Oakland, California, USA, 2002. IEEEComputer Society.

[7] Oliver Dain and Robert Cunningham. Building scenarios from a hetero-geneous alert stream. In IEEE Workshop on Information Assurance andSecurity, West Point, NY, USA, 2001.

[8] Oliver Dain and Robert Cunningham. Fusing a heterogeneous alert streaminto scenarios. In Proceedings of the 2001 ACM Workshop on Data Miningfor Security Applications, pages 1–13, 2001.

[9] Herve Debar and Andreas Wespi. Aggregation and correlation of intrusion-detection alerts. In RAID ’00: Proceedings of the 4th International Sympo-sium on Recent Advances in Intrusion Detection, pages 85–103, Toulouse,France, 2001. Springer-Verlag.

[10] D.E. Denning. An intrusion-detection model. IEEE Transactions on Soft-ware Engineering, SE-13(2):222–232, 1987.

[11] S. Forrest, S.A. Hofmeyr, A. Somayaji, and Longstaff T.A. A sense ofself for UNIX processes. In Proceedings of the 1996 IEEE Symposium onSecurity and Privacy, pages 120–128, 1996.

73

http://www.cisco.com/en/US/products/ps6241/index.html

http://www.cisco.com/en/US/products/ps6241/index.html

BIBLIOGRAPHY

[12] Jeremy Frank. Artificial intelligence and intrusion detection: Current andfuture directions. In Proceedings of the 17th National Computer SecurityConference, 1994.

[13] P. Helman and G. Liepins. Statistical foundations of audit trail analysisfor the detection of computer misuse. IEEE Transactions on SoftwareEngineering, 19:886–901, 1993.

[14] Harold S. Javitz and Alfonso Valdes. The SRI IDES statistical anomalydetector. In Proceedings of the 1991 IEEE Symposium on Security andPrivacy, pages 316–326. IEEE Computer Society, 1991.

[15] Kevin Johnson, Axton Graham, Jon Hart, Kevin Johnson, Doug Mackie,Sean Muller, Tim Rupp, Christian Svensson, and Max Valdez. Base. http://base.secureideas.net.

[16] Klaus Julisch. Clustering intrusion detection alarms to support root causeanalysis. ACM Trans. Inf. Syst. Secur., 6(4):443–471, 2003.

[17] Brendan P. Kehoe. Zen and the art of the internet. Prentice Hall, 1995.

[18] C. Ko. Logic induction of valid behavior specifications for intrusion detec-tion. In Proceedings of the 2000 IEEE Symposium of Security and Privacy,pages 142–153, 2000.

[19] M. Ko, C. Ruschitzka and K. Levitt. Execution monitoring of security-critical programs in distributed systems: a specification-based approach. InProceedings of the 1997 IEEE Symposium on Security and Privacy, pages175–187. IEEE Computer Society, 1997.

[20] Terran Lane and Carla E. Brodley. Temporal sequence learning and datareduction for anomaly detection. ACM Trans. Inf. Syst. Secur., 2(3):295–331, 1999.

[21] Wenke Lee and Salvatore J. Stolfo. A framework for constructing featuresand models for intrusion detection systems. ACM Trans. Inf. Syst. Secur.,3(4):227–261, 2000.

[22] Wenke Lee and Dong Xiang. Information-theoretic measures for anomalydetection. In Proceedings of the 2001 IEEE Symposium on Security andPrivacy, pages 130–143, Oakland, California, USA, 2001.

[23] Ulf Lindqvist and Phillip A. Porras. Detecting computer and networkmisuse through the production-based expert system toolset (P-BEST). InProceedings of the 1999 IEEE Symposium on Security and Privacy, pages146–161. IEEE Computer Society, 1999.

[24] Alex Lukatsky. Protect your information with intrusion detection. A-ListPublishing, A-LIST, LLC, 295 East Swedesford Rd., PMB #285, Wayne,PA 19087, 2003.

[25] Emilie Lundin Barse. Logging for Intrusion and Fraud Detection. PhDthesis, Chalmers University of Technology, Goteborg, Sweden, 2004.

[26] Stefanos Manganaris, Marvin Christensen, Dan Zerkle, and Keith Hermiz.A data mining analysis of RTID alarms. Comput. Netw., 34(4):571–577,2000.

74

http://base.secureideas.net

http://base.secureideas.net

BIBLIOGRAPHY

[27] Jason Meller. Snorby. http://snorby.org/.

[28] Peng Ning, Yun Cu, and Douglas S. Reeves. Analyzing intensive intru-sion alerts via correlation. In RAID ’02: Proceedings of the 5th Interna-tional Symposium on Recent Advances in Intrusion Detection, pages 74–94,Zurich, Switzerland, 2002. Springer-Verlag.

[29] Tadeusz Pietraszek. Using adaptive alert classification to reduce false pos-itives in intrusion detection. In RAID ’04: Proceedings of the 7th In-ternational Symposium of Recent Advances in Intrusion Detection, pages102–124, Sophia Antipolis, France, 2004. Springer Berlin / Heidelberg.

[30] Phillip A. Porras, Martin W. Fong, and Alfonso Valdes. A mission-impact-based approach to INFOSEC alarm correlation. In RAID ’02: Proceedingsof the 5th International Symposium on Recent Advances in Intrusion De-tection, pages 95–114, Zurich, Switzerland, 2002. Springer-Verlag.

[31] Martin Roesch. Snort - lightweight intrusion detection for networks. InProceedings of USENIX LISA’99, Seattle, Washington, USA, 1999.

[32] R. Sekar, M. Bendre, D. Dhurjati, and P. Bollineni. A fast automaton-basedmethod for detecting anomalous program behaviors. In Proceedings of the2001 IEEE Symposium of Security and Privacy, pages 144–155, Oakland,California, USA, 2001.

[33] Dave Smith. Disney A to Z. Disney Book Publishing Inc., 2006.

[34] Sourcefire. Snort. http://www.sourcefire.com/security-technologies/snort.

[35] William Stallings. Network Security Essentials: Applications and Stan-dards. Pearson Education, Upper Saddle River, NJ 07458, 2007.

[36] Aurobindo Sundaram. An introduction to intrusion detection. Crossroads,2(4):3–7, 1996.

[37] D. Wagner and R. Dean. Intrusion detection via static analysis. In Proceed-ings of the 2001 IEEE Symposium of Security and Privacy, pages 156–168,Oakland, California, USA, 2001.

[38] Dustin Willis Webber. Snorby - a web interface for Snort. https://github.org/Snorby/snorby.

[39] Jingmin Zhou, Mark Heckman, Brennen Reynolds, Adam Carlson, andMatt Bishop. Modeling network intrusion detection alerts for correlation.ACM Trans. Inf. Syst. Secur., 10(1):4, 2007.

75

http://snorby.org/

h

https://github.org/Snorby/snorby

https://github.org/Snorby/snorby

BIBLIOGRAPHY

76

Appendix A

Box Plots

A box plot is dependent on five values:

1. xmin, the minimum value among the data points.

2. xlower, the lower quartile.

3. x0.5, the median.

4. xupper, the upper quartile.

5. xmax, the maximum value among the data points.

x0.5 = the most central value of the data points ordered in ascending order. Ifn is even x0.5 is the average of the two most central data points.

Let the median split the data points in two halves. If n is odd, the median isaccounted to both halves and if n is even, none of the halves.

xlower is the median of the lower half and xupper is the median of the upperhalf.

Figure A.1: A generic box plot.

77

78

Appendix B

Specification ofRequirements

Give positive feedback to an alarmActor: System administratorGoal: The vote should be evenly divided between all or a subset of the attributesof the alarm and saved to a database.Interaction: The user clicks on a button that gives this alarm and alarms similarto this one a higher ranking.Difficulty: Easy

Give negative feedback to an alarmActor: System administratorGoal: The vote should be evenly divided between all or a subset of the attributesof the alarm and saved to a database.Interaction: The user clicks on a button that gives this alarm and alarms similarto this one a lower ranking.Difficulty: Easy

Changing sort orderActor: System administratorGoal: Sorting order should change.Interaction: It should be possible for the user to change the sort order of alarmsbetween three different orders; short, medium, and long. Short will sort thealarms in a way that reflects their short term changes in the ranking. Mediumshould reflect medium term changes in the ranking of alarms, and long is forlong term changes.Difficulty: Hard

79

80

Appendix C

Source Code

C.1 click test

click test is a small program we developed to test how effective it would beto sort alarms using votes from the user. It does not directly mimic how ourimplementation in Snorby works in that it does not simulate any kind of negativefeedback.�

1 #!/ usr / b in / ruby2

3 ### CONFIGURATION ###4

5 # Number o f alarms to genera te6 $num alarms = 1007

8 # Des t ina t ion IPs , how many in t o t a l and what percent o f themtha t w i l l r e c e i v e

9 # c l i c k s10 $num dest ip = 5011 $ impor tan t de s t i p = 4 # in percent12

13 # Source IPs , same dea l as wi th d e s t i n a t i o n IPs14 $num src ip = 10015 $ impo r t an t s r c i p = 2 # in percent16

17 # Alarm type s .18 # How many alarm type s shou ld t he r e be?19 $num alarm types = 5020 # How shou ld they be d i v i d ed between the th r e e s e v e r i t i e s high

, medium and low?21 $ s e v e r i t y r a t i o s = { : high => 15 , # in percent22 : medium => 45 ,23 : low => 40 }24

25 # What percen tages o f the d i f f e r e n t alarm s e v e r i t i e s shou ldr e c e i v e c l i c k s ?

81

C.1. CLICK TEST

26 $ important a larm types = { : high => 75 ,27 : medium => 10 ,28 : low => 5 }29

30 # Alarm genera t ion r a t i o s in percent31 $a l a rm ra t i o s = { : high => 15 ,32 : medium => 40 ,33 : low => 45 }34

35 # How much i s a c l i c k worth?36 $ s r c i p c l i c k = 137 $ d e s t i p c l i c k = 138 $a l a rm type c l i c k = { : high => 3 ,39 : medium => 1 ,40 : low => 1 }41

42 ### END OF CONFIGURATION ###43

44 # IP ranges45 $d e s t i p s = ( 1 . . $num dest ip ) . to a46 $ imp de s t i p s = $d e s t i p s [ 0 . . ( $num dest ip ∗( $ impor tan t de s t i p

/100.0)).round]

47 $src_ips = ($dest_ips.last+1..$dest_ips.last+1+

$num_src_ip).to_a

48 $imp_src_ips = $src_ips [0..( $num_src_ip*($important_src_ip/

100 .0 ) ) . round ]49

50 # Alarm type ranges51 $a larm types = {}52 prev v = 053 $ s e v e r i t y r a t i o s . e a ch pa i r do | k , v |54 new v = prev v + ( $num alarm types ∗( v/100.0)).floor55 $alarm_types[k] = (prev_v+1..new_v).to_a

56 prev_v = new_v

57 end

58

59 # Important alarm types

60 $imp_alarm_types = {}

61 $important_alarm_types.each_pair do |k,v|

62 types = $alarm_types[k]

63 $imp_alarm_types[k] = types[0..(types.size*(v/ 100 .0 ) ) . round−1]

64 end65

66 # Print what was genera ted67 puts "Dest IPs: #{$dest_ips.first}-#{$dest_ips.last}"

68 puts "Important dest. IPs: #{$imp_dest_ips.first}-#{

$imp_dest_ips.last}"

69 puts "\nSource IPs: #{$src_ips.first}-#{$src_ips.last}"

70 puts "Important src. IPs: #{$imp_src_ips.first}-#{$imp_src_ips

.last}"

71 puts "\nAlarm types:"

72 $a larm types . e a ch pa i r do | s , v |73 puts "\t#{s.to_s}: #{v.first}-#{v.last}"

82

APPENDIX C. SOURCE CODE

74 end75 puts "\nImportant alarm types"

76 $imp alarm types . e a ch pa i r do | s , v |77 puts "\t#{s.to_s}: #{v.first}-#{v.last}"

78 end79

80 # two hashes t ha t w i l l ho ld the we i gh t s f o r d i f f e r e n t i p s andtype s o f alarms

81 $s ip w = {}82 $dip w = {}83 $alarm type w = {}84

85 #hashes t ha t w i l l ho ld the r e a l we i gh t s86 $ r e a l s i p w = {}87 $ r ea l d ip w = {}88 $rea l a l a rm type w = {}89

90 # w i l l ho ld a l l the alarms t ha t are generated91 $alarms = [ ]92 # keeps t rack o f how many alarms in t o t a l93 $to t a l a rms = { : high => 0 , : medium => 0 , : low => 0}94

95 class Alarm96

97 a t t r a c c e s s o r : s ev e r i t y , : s r c i p , : d e s t i p , : type , :unordered , : ranked , : opt imal

98

99 def i n i t i a l i z e ( i )100 @id = i101 # random number t ha t w i l l dec ide which alarm to genera te102 random = rand (100)103

104 a l a rm ra t i o s = $a l a rm ra t i o s . s o r t { | a , b | a [ 1 ] <=> b [ 1 ] }105

106 cum r = 0107 a l a rm ra t i o s . each do | i |108 s = i [ 0 ] # s e v e r i t y109 r = i [ 1 ] # ra t i o110

111 i f random < cum r+r then112 $to t a l a rms [ s ] += 1113 # genera te alarm of s e v e r i t y s114 @sever i ty = s115 @src ip = $ s r c i p s [ rand ( $ s r c i p s . s i z e −1) ]116 @dest ip = $d e s t i p s [ rand ( $d e s t i p s . s i z e −1) ]117 types = $alarm types [ s ]118 @type = types [ rand ( types . s i z e −1) ]119 @unordered = 0120 @ranked = 0121 @optimal = 0122

123 # i n i t i a l i z e va l u e s in sip w , dip w and alarm type w124 $s ip w [ @src ip ] = 0125 $dip w [ @dest ip ] = 0

83

C.1. CLICK TEST

126 $alarm type w [ @type ] = 0127 $ r e a l s i p w [ @src ip ] = 0128 $ r ea l d ip w [ @dest ip ] = 0129 $rea l a l a rm type w [ @type ] = 0130 break131 end132 cum r += r133 end134 end135

136 def add c l i c k137 $s ip w [ @src ip ] += $ s r c i p c l i c k138 $dip w [ @dest ip ] += $ d e s t i p c l i c k139 $alarm type w [ @type ] += $a l a rm type c l i c k [ @sever i ty ]140

141 i f $ imp s r c i p s . i n c l ude ?( @src ip )142 $ r e a l s i p w [ @src ip ] += $ s r c i p c l i c k143 end144 i f $ imp de s t i p s . i n c lude ?( @dest ip )145 $ r ea l d ip w [ @dest ip ] += $ d e s t i p c l i c k146 end147 i f $imp alarm types . va lue s . f l a t t e n . i n c lude ?(@type )148 $rea l a l a rm type w [ @type ] += $a l a rm type c l i c k [ @sever i ty

]149 end150 end151

152 def importance153 $s ip w [ @src ip ] + $dip w [ @dest ip ] + $alarm type w [ @type ]154 end155

156 def r ea l impor tance157 $ r e a l s i p w [ @src ip ] + $r ea l d ip w [ @dest ip ] +

$rea l a l a rm type w [ @type ]158 end159

160 def important ?161 $ imp s r c i p s . i n c l ude ?( @src ip ) | |162 $ imp de s t i p s . i n c lude ?( @dest ip ) | |163 $imp alarm types . va lue s . f l a t t e n . i n c lude ?(@type )164 end165

166 def s e t op t ima l ( index )167 @optimal = index168 end169 end170

171 # genera te some random even t s172 1 . upto ( $num alarms ) { | i | $alarms << Alarm . new( i ) }173

174 #keep a counter f o r how many important alarms , used f o rs t a t i s t i c s l a t e r .

175 no imp alarms = 0176

84

APPENDIX C. SOURCE CODE

177 # simu la t e c l i c k s f o r important even t s178 $alarms . each index do | i |179 $alarms [ i ] . unordered = i180 i f $alarms [ i ] . important ?181 $alarms [ i ] . a dd c l i c k182 no imp alarms += 1183 end184 end185

186 # make a copy to pre se rve o r i g i n a l order187 $alarms un = Array . new( $alarms )188

189 $alarms . s o r t ! { | x , y | y . r ea l impor tance <=> x . r ea l impor tance}

190 # record the order o f the op t ima l l y ranked alarms191 $alarms . each index do | i |192 $alarms [ i ] . s e t op t ima l ( i )193 end194

195 # sor t even t s by importance196 $alarms . s o r t ! { | x , y | y . importance <=> x . importance }197

198 # record the order o f ranked alarms199 $alarms . each index do | i |200 $alarms [ i ] . ranked = i201 end202

203 count imp alarms = 0204

205 puts "\nNumber of generated alarms"

206 puts "\tHigh : #{$tot_alarms[:high]}"

207 puts "\tMedium : #{$tot_alarms[:medium]}"

208 puts "\tLow : #{$tot_alarms[:low]}"

209 puts "\tTotal : #{$alarms.size}"

210

211 puts "\nNumber of clicked alarms : #{no_imp_alarms}"

212

213 puts "\nAlarms sorted by importance"

214 puts "\t Legend: <Alarm type >, src ip-->dest ip"

215 $alarms . each do | a |216 pr in t "\n"

217 pr in t "#{a.importance} | " . r j u s t (7 )218 pr in t "#{a.real_importance} | " . r j u s t (7 )219 pr in t "<#{a.type}>" . r j u s t (6 )220 pr in t " (#{a.severity})" . l j u s t (11)221 pr in t "| #{a.src_ip}-->#{a.dest_ip}" . l j u s t (13)222 pr in t " | "

223 i f a . important ?224 pr in t "important |"

225 pr in t " #{(count_imp_alarms += 1)} |" . r j u s t (6 )226 else227 pr in t "|" . r j u s t (11)228 pr in t " #{(count_imp_alarms)} |" . r j u s t (6 )229 end

85

C.1. CLICK TEST

230 pr in t "#{a.unordered} |" . r j u s t (6 )231 pr in t "#{a.ranked} |" . r j u s t (6 )232 pr in t "#{a.optimal}" . r j u s t (4 )233 end234 puts "\n"

235

236 puts "\t Legend: <Alarm type >, src ip-->dest ip"

237 $alarms un . each do | a |238 pr in t "\n"

239 pr in t "#{a.importance} | " . r j u s t (7 )240 pr in t "#{a.real_importance} | " . r j u s t (7 )241 pr in t "<#{a.type}>" . r j u s t (6 )242 pr in t " (#{a.severity})" . l j u s t (11)243 pr in t "| #{a.src_ip}-->#{a.dest_ip}" . l j u s t (13)244 pr in t " | "

245 i f a . important ?246 pr in t "important |"

247 pr in t " #{(count_imp_alarms += 1)}"

248 else249 pr in t "|" . r j u s t (11)250 pr in t " #{(count_imp_alarms)}"

251 end252 end253 puts "\n"� �

86

Appendix D

Interviews – Q&A

D.1 Hakan Nohre

Interview with Hakan Nohre from Cisco via e-mail in Swedish.

Bold = us

Normal = Hakan Nohre

I hur stor utstrackning kan man aggregera alarmen i er IDS?(AGGREGERA = t.ex. visa alla alarm som inbegriper samma kallasom ett alarm sa att man latt kan se om nagon verkligen attackerarnatverket, alltsa sorterar alarmen pa nagot gemensamt attribut somkalla, destination, port, protokoll, etc.)

Kan ske pa olika nivaer:

• I sjalva IPS sa kan alarm aggregeras innan de skickas, sa att t.ex. fleraevent av samma typ mellan samma sort/dest skickas som ett alarm istalletfor flera (minskar bruset).

• I managementverktyget IME kan man latt sortera attacker efter t.ex. perdestination, per source, per attacktyp, per sensor etc. Dvs man kan faen omedelbar vy av alla attacker fran ip adress X, fordelat pa targetT1,T2,T3...TN, och under dessa attacktyper (A1, A2, A3) per X och Ti.

• I management verktyget Cisco MARS aggregeras inte bara IPS alarm,utan aven alarm fran brandvaggar, operativsystem for att aggregeras tillincidenter. En incident kan besta av hundratals events, dar kanske ett parkommer fran IPS, ovriga fran brandvaggsloggar eller serversystem.

I hur stor utstrackning kan man paverka hur alarmen aggregeras?

87

D.1. HAKAN NOHRE

Helt konfigurerbart i alla nivaer.

I hur stor utstrackning kan ert system korrelera alarm/fora upp dempa en hogre niva?(KORRELERA = t.ex. kan man bygga scenarion dar en hel attack-kedja behovs for att uppna hackarens mal. Att kunna se att ett alarmeller flera kan kopplas till ett scenario och varna for scenariot istalletfor att bara visa alarmen var for sig ar ett exempel pa korrelation.)

I management verktyget Cisco MARS aggregeras inte bara IPS alarm, utan avenalarm fran brandvaggar, operativsystem for att aggregeras till incidenter.

En incident kan besta av hundratals events, dar kanske ett par kommer franIPS, ovriga fran brandvaggsloggar eller serversystem.

Vilka fordelar har Ciscos system over konkurrenterna nar det galleratt korrelera/aggregera alarm?

En fordel med Cisco MARS ar att den kan aggregera alarm fran flera olikakallor, inte bara IPS, utan brandvaggar, serversystem. En IPS kan sagas ge brafocus med mycket detaljer om en attack, men ofta kan man tappa den storabilden eftersom IPS bara lyssnar/ser trafiken pa ett par stallen i natverket.

Med MARS kan man alltsa utnyttja information inte bara fran IPS for att fa seen fullstandigare bild. T.ex om IPS visar att A forsoker hacka B : Vilka ovrigamaskiner har A talat med sista 24 timmarna? Denna trafik kanske inte ens hargatt genom IPS, sa man vill alltsa utnyttja brandvaggsloggar eller netflow har.

Kan er IDS lara sig hur viktiga vissa typer av alarm ar for en admi-nistrator genom dennes interaktion med systemet? T.ex. genom attregistrera vilka alarm som klickas pa.

• Ar det nagot ni tillampar i ert system?

• Om ja, hur samlar ert system in denna information?

• Ser du nagra fordelar/nackdelar med detta interaktionssatt?

I vart system bestams viktigheten i ett alarm av ett metavarde, “Risk Rating”.

Risk Rating ar en funktion av:

• event severity (hur allvarlig attack)

• event fidelity (hur saker ar man att det ej ar false positive)

• attack relevancy (om attack ar relevant for OS som attackeras, dar dettafaststalls med passive OS fingerprinting

• target value (administratorsspecifierad vadering av target maskin)

• reputation (kallans rykte pa internet, tas fran Cisco Sensorbase)

88

APPENDIX D. INTERVIEWS – Q&A

Ju hogre RiskRating desto allvarligare ar attacken. Vilka actions en IPS skallta bestams av Risk Rating, t.ex. droppa paket om Risk Rating > 80.

Ser ej sa stora fordelar med att IPS anpassar sig efter anvandarens klickande,detta kan ju variera med tiden.

Hur ser ni pa framtiden? Kommer IDS-system bli battre pa attupptacka nya attacker sjalva eller kommer det forbli en “katt ochratta”-lek dar tillverkaren av IDS-systemet alltid ligger steget efterdom som hittar pa nya attacker?

Ett IPS system med signaturer som ar sant vulnerability baserade (motsats tillexploit baserat) bor hitta aven nya attacker om det ar en kand sarbarhet.

Cisco IPS har aven funktion for ryktesbaserad filtrering, en stor andel av avennya attacker sker idag fran komprometterade hostar, s.k botnets. Dessa attackerkan blockeras aven om de ar helt nya eftersom blockering sker med hjalp avrykteshanteringen.

Vilka problem har era kunder som de vill att ni skall losa nar detgaller intrusion detection och intrusion prevention?

Ofta nagon form av Compliance, t.ex. PCI compliance.

Vilken ny funktionalitet i ert IDS-system ar det era kunder efterfragarmest?

Just nu pagar mycket uppgraderingar av datacenter till 10 gig ut mot servrarna,vilket ger mycket hog total throughput i datacenter. Sa prestandakraven vaxerhela tiden.

D.2 Ola Soderstrom

Transcript of interview we did with Ola Soderstrom from Secode on 23 April2010. The interview was done in person and recorded on an mp3-player.

Vad har du gjort sen du slutade pa Chalmers?

–Nar jag slutade pa Chalmers sa hade jag inget jobb direkt. Sa jag hjalpte litemindre foretag med deras IT-miljoer i storsta allmanhet. Oftast windows miljooch vardagliga problem som dom hade. Och det var val ganska larorikt. Dettavar hosten 2003.

–Sen gick det drygt ett ar och sa fick jag jobb pa Secode som jag har jobbat hossen 2005. Dar har jag haft ungefar samma arbetsuppgifter, har val utvecklatslite kanske men det har blivit mycket sitta och klicka pa alarm. Dar ar jag numen sen nyar ar jag pappaledig.

89

D.2. OLA SODERSTROM

–Vissa saker har man i farskt minne nar man sitter och haller pa med det. Mensom jag nu kanner att vad fort man kommer i fran det tanket och arbetsgangen.Far man tanka efter lite gran sa kommer det fram sa smatt.

Vilka interface har du anvant?

–Cisco Works, Juniper, Tipping Point, ISS (iss.net) och Snort

Ifall du tanker pa dom olika IDS:er som du anvant, i vilken ut-strackning kanner du att du kunnat aggregera alarm.

–Tipping Point har en aggregerings-funktion inbyggt i GUI:t, men den var vallkanske inte sa anvandbar.

–Det man ar intresserad av oftast ar att se langre tillbaks an bara det som manfar in i GUI:t. Det kommer sa stor mangd larm sa att bara aggregera dom somfinns dar fyller inte sa stor funktion.

–Om man daremot hade kunnat haft nagot som haller koll historiken for ensource ip adress sa man vet om den har triggat en viss signatur flera gangeninnan (ett par veckor tillbaka).

I hur stor utstrackning kanner du att du har kunnat paverka hursystemen aggregerar alarmen?

–Ja, det har jag kunnat gora. Har framfor allt kunnat stanga av aggregeringen.Jag har kunnat aggregera pa godtyckliga falt i det GUI:t om jag vill. Nastanalla GUI:n tillater att man sorterar pa en godtycklig kolumn. Som vart systemser ut dar vi samlar allting i ett eget GUI dar man ser alla handelser kommerin. Det har en sorterings funktion som jag anvander ganska ofta. Da kan jagdirekt se om det manga som klumpas ihop under ett attribut. Jag kan sortera pasignaturen och se om det ar manga manga olika ip-adresser som triggar den dakanske den far lite lagre tillforlitlighetsgrad. Sa just sorterings funktionalitetentycker jag ar vard nastan mer an aggregering.

Ett typiskt alarm har information om tid, alarmtyp, kalla/destination,etc. Racker denna information for dig eller behover du mer?

–Det racker for att utesluta en hel mangd larm. Men om det ar nagot som ar liteintressant da kastar jag alltid med mer information for att kunna komma tillnagon slutsats om hur vida det ar intressant i verkligheten eller inte. Da ar dettypiskt att man tittar narmare pa i vilken omgivning som den har signaturentriggar. Om det ar pa ett valdigt begransat natverk eller om det kommer franinternet osv. Omgivningen ar valdigt viktig. Kollar aven pa vilken riktning omattacken sker, pa sjalva signaturen om den ar valdigt gammal osv.

Anvander du nagra kompletterande verktyg till IDS:en? (tcpdump)

90


–Ja det gor man. Tcpdump inte minst. Ar det sa att det ar nagot som artillrackligt intressant da loggar man direkt in pa sensorn och kor igang en tcp-dump.

Har du nagra script som automatiskt gar igang vid ett visst alarm?

–Om man inte kan kolla till en slutsats genom att titta pa larmet. Men omvi talar Juniper, dar finns det tillgang till nagra packet fore och efter alarmeti.o.m. att man kan stalla in att den ska fanga det. Da kan jag ju titta dar ochkomma ganska langt pa det. Skulle inte det vara tillrackligt, da loggar jag inpa sensorn och kor igang en tcpdump for att bilda mig en uppfattning. Narman val har fatt tcpdump:en, antingen sa kollar jag pa den direkt i skalet ellerwireshark om man vill far lite battre koll pa vad som har hant. Annars har vilite hemmabyggda system som dekodar om det ar obfuskerade javascript t.ex.eller om det ar payloads har mycket hexkod o.s.v.

Sebastian: –Har ni det som en plugin/preprocessor till Snort?

–Nej det har vi inte. Utan det ar fristaende sma Python/Pearl script dar vi kanklistra in packeten i for att fa ut nagonting vettigt. For att se vad som daljersig bakom sa att saga.

–Det vi har byggt till Snort det ar verktyg for att hamta pcap:en fran Snorttill skillnad fran Juniper dar man kan klicka pa ett larm och se pcap:en. Sa iSnort har vi konfat det sa att den alltid sparar packet data for varje signatur.Men den sparar det i ett inte sa lattillgangligt format. Det ar svart pcap tillratt signatur/handelse. Sa det har vi byggt verktyg sa att vi kan logga in ochsen komma at det da.

Hur ofta brukar du granska alarmen narmre? Ar det dom flesta ellerbara nagra fa?

–Det ar nog ganska fa som jag gar in och granskar narmare i den bemarkelsen.Jag tror att det ror sig om max 10-15%.

Vad ar det som gor att du inte klickar pa dom andra 90% av alarmen?

–Dels ar det sa att man alltid jobbar med att redusera antalet falsklarm, ochdet arbetet ligger man alltid lite efter med. Sa man far in en del alarm dar manvet att det har borde inte finnas har, men jag har inte tid just nu att filtrerabort det. Och dom som har jobbat fore mig de senaste veckorna har inte hellerhaft tid att filtrera bort det. Sa det finns dar och man vet att det triggar ochar ofarligt for det har man kollat pa.

–Sen ar det da den andra typen av alarm som inte ar falsklarm men som inte arintressanta av andr anledningar. Dom kanske inte forekommer i en omgivningsom ar intressant, t.ex. om man ser maskaktivitet fran externa adresser. Det arju inte falkslarm for dom forsoker faktiskt attackera men man ser dom precishela tiden och dom ar inte intressanta att kolla pa.

–Sen ar det lite graskala dar i mellan av olika anledningar.

91

D.2. OLA SODERSTROM

Hur skulle du beskriva ett typiskt tillvagagangssatt for dig nar du serett alarm?

–Da brukar jag forst och framst gora nagon typ av historikuppslagning/korre-lering pa source-ip-adressen for att se om vi har nagon information om den harvarden. Det kanske ar sa att kunden har meddelat att just den har varden fargora nmap-avsokningar men att vi inte har missat att ge vart backend systemden informationen sa att larmet triggar anda. Eller ocksa kanske det ar sa attkommer det fran dom har vardarna da ar det jattekritiskt, da ska man skickainsidentrapport direkt. Sa forst forsoker jag kolla upp source-ip-adressen i varaegna system och gora nagon typ av historik och korrelering.

–Sen gar jag vidare. Ar det en extern vard da kanske man kollar om den ar ifran Korea/Kina osv. Nasta steg ar nog att titta pa packetdata om sant finnstillgangligt. Vidare kollar jag pa hur signaturen ar definierad. (Aterigen ar Juni-per bra dar till skillnad fran Cisco och TippingPoint dar man inte har tillgangtill hur signaturen ar definierad. Sa kan jag se det reguljara uttryck som domhar anvant for att fanga just den har attacken.) Da kan jag gora en bedomningom just den har signaturen har en hog tendens att falsklarma eller om den arav god kvalite, och matcha det mot den packetdata som jag har for att se omden borde trigga eller inte.

–Sen laser jag pa om sjalva attacken/exploiten som dom forsoker utnyttja. Detfinns oftast i GUI:t dar jag kan komma at CVE-nummer. Behover jag bilda migen battre uppfattning anda kollar jag pa internet och laser pa om attacken ochsarbarheten.

Bygger du sjalv nagra kopplingar mellan alarm eller gor systemet detat dig?

–Nej inte sa mycket faktiskt. Det ar inte sa ofta som vi har den bilden framfoross. Vi har kanske bara en ganska begransad del av natverket som vi ser. Vi serinte vad som hander sen riktigt. Vi vet kanske inte om att det malet som domforsoker attackera att det ar en mail/web-server for den ar inte atkomlig utifran.Nej jag tror inte det, inte just den typen av kopplingar. Klar man bygger ju ihopvissa signaturer som i kombination ar extra intressanta om dom forekommer.Men just attackkedjor pa natverket, kan handa att jag glomt det for att jag intejobbat pa ett tag, men inte sa mycket tror jag.

Eva Lina: –For Emilie hade det i SSIM. Dar kunde hon stalla in sadana korre-leringssaker som att om du har fatt in sahar manga alarm av den har typen dakanske det borjar bli intressant. Da kanske du bor titta om du har fatt in alarmav dom har typerna innan dess. Eller om du far det har alarmet efterat da ardet jattefarligt.

–Lite grann kanske man gor det undermedvetet. Man kollar ju anda den histo-riken som finns nar man far ett alarm. Man kollar source-ip-adresserna och manslar ju aven pa sjalva signaturen for att se vad man far upp. Men det ar valdigtmycket pa manuell basis isf som man gor det.

92


–Fast det ar klart har man proxy-, brandvaggs- och ids-loggar hos en kund,da har vi mycket storre mojlighet att gora den typen av korrelering. Man kankoppla en anvandare till en source-ip-adress t.ex.

Enter Magnus:: meddelar att Ola ar sen...

Sebastian: –Jag tankte pa OSSIM; det hade ju aven tillgang till syslog loggar...Den kunde i syslog:en se om det ar nagon som forsoker gora en root-attack. Dakunde den se i loggen om det ar nagon som far misslyckade login-forsok ellerkanske t.o.m. lyckade login-forsok.

–Precis, men det ar inte sa ofta vi har den mojligheten. Men om vi har detatagandet fran kunden att vi kanske har syslog:ar fran deras servrar som vi kankoppla till handelser i IDS:en osv. Sanna system har kommit ganska starkt pasistone. Hos nagon kund sa har dom ett sadant system som vi loggar in pa ochtar fram information for att gora den korreleringen. Annars sa ar det vara egnasystem dar det ar lite mer eller mindre pa manuell basis som vi gor den hartypen av korrelering och kopplar handelser till varandra.

Eva Lina: –For laser man rapporterna som producerats, da ar det ju dom harstora rosaskimrande landskapen som malas upp hur allt detta skall automatiserasmed machine learning algorigthms. Men det finns ingenting av det ute i denriktiga varlden an da?

–Nja. Det finns men dom saljer skitbra nu och dom kostar enorma summorpengar. En kund dom kor natt som heter Arcsight. Det ar ett system som skallsvalja allting och som skall gora den har typen inbyggda intelligensen i sig da.Men det kanns som att dom missar lite. Dels sa tror jag inte att den har samycket intelligens i sig utan den samlar nog bara ihop allting pa ett stalle ochsa spottar den ur sig det till anvandaren sa far den sitta dar bast den vill. Ochi slutandan ar det operatorens erfarenhet som avgor om man kan faststalla omdet ar ett larm eller inte. For man kan omojligen ga tillverkarens skala pa omdet ar allvarligt eller inte. Det ar sa manga fler parametrar som avgor om det arintressant eller inte. Sa jag tror att det later skitbra for nagon som inte jobbarmed det. Hos oss skulle det inte salja sa bra.

Eva Lina: –Det namnde Emilie ocksa att hon var inte riktigt saker pa den haruniversala losningen. Ett natverk ar som en individ, den ar unik, den finns ingenannanstans i varlden i princip. Sa det finns inget som passar for alla natverk.

–Det kanner man ocksa nar man sitter och jobbar, man lar sig kundernas natverkoch vad som brukar trigga dar och vad som ar legitimt och vad som ar avvikelser idet legitima sa att saga. Sa som jag ser det; visst det ar signaturbaserade systemsom vi jobbar med men pa var niva sa ar det avvikelsedetektering som vi arbetarmed. Hur upptacker vi nagonting som inte brukar forekomma i den har enormastrommen med handelser som vi far emot oss hela tiden. Och da galler det attbygga system som upptacker dom avvikelserna i det normala monstret sa attsaga.

Sebastian: –Sa dom bra systemen ar helt enkelt dom som bara hjalper en att hittaavvikelserna. Det ar inte sa att systemet skall upptacka dom riktiga alarmensjalv utan det skall fungera som ett verktyg at operatoren bara?

93

D.2. OLA SODERSTROM

–Ja. Jag har inte sjalv arbetat med Arcsight och dom har storre systemen sa jaghar sjalv inte sa stor erfarenhet av hur dom fungerar. Det var ju bara nagra arsedan som Unified Threat Management var ett begrepp dar man skulle byggaihop IDS med brandvagg med antivirus, all funktionalitet i ett. Men det slogaldrig igenom riktigt for att ingenting var riktigt bra fran borjan. Bygger manihop manga lite halv-pissiga system tillsammans blir det inte battre.

–Sa nej, jag tror inte sa himla mycket pa det har att man skall aggregera ihopsystem i varandra och fa ett flashigt GUI och sitta och arbeta med. Man mastenog tanka lite modulart. Det skall vara skalbart systemet men anda sa att mankan skraddarsy det sa som man sjalv vill.

Sebastian: –Sa oppna standarder da helt enkelt?

–Oppna standarder ja. Helt klart. Hade vi kunnat arbeta med Snort bara sahade det varit bra men det kan vi inte.

–Men Sourcefire hade varit intressant att titta pa. Det har vi snackat om att viskall ta in i vart testlabb och utverdera. Sa vi far vall se. Det vore intressantatt se skillnaden mellan Snort och Sourcefire och se om det blir valdigt myckethogre kvallite pa larmen an vad Snort har.

Hur kategoriserar du alarm?

–Ar det sa att ett alarm kommer over den dar troskeln dar man inte bara klickarbort det, utan det ar lite intressant. Da forsoker man faststalla vilken kvallitepa det ar pa signaturen helt enkelt.

Sebastian: –Hur ser signaturen ut da. Ar det ett regular expression eller nattannat?

–Om man nu har tillgang till det. I t.ex. TippingPoint har vi inte nagon accessalls till hur signaturen ser ut. Men i Snort och Juniper har man ju det. Detar reguljara uttryck. Dom anvander ett Perl-bibliotek for att parsa dom harreguljara uttrycken. Kan man reguljara uttryck sa kan man bilda sig en upp-fattning om signaturen. Sen finns det lite fler parametrar man kan skruva pa narman skapar en signatur. I Juniper kan man skapa custom signaturer sjalv, omman tycker att den ar usel sa kan man gora en kopia av signaturen och anvandaden som bas for en egen signatur. Da kan man modifiera det reguljara uttrycketom man t.ex. tycker att sql injection bara skapar en massa falsklarm hela tidensa kan man peta in nagonting som minskar falsklarmen.

–Ne, sa det ar inget magiskt alls. Det ar bara att komma at den informationenhelt enkelt och kolla pa det reguljara uttrycket.

Hur kategoriserar du alarm?

• “lag-niva alarm”

• “ointressanta”

• “dubblett”, (orsak redan kand)

94


• “grad av tillforlitlighet”, t.ex. en regel som triggar alarm kanvara valdigt allman och orsaka manga alarm varav de flesta inteberor pa intrang och da ha en lag grad av tillforlitlighet eller enregel kan vara valdigt specifik och darfor ar alarm som genererasutifran den regeln av hog tillforlitlighetsgrad.

• “bekraftelse”, kan man hitta ett nytt event som bekraftar attalarmet ar korrekt?

–Jag tycker att man arbetar lite gran efter alla dom dar. Vissa alarm ar juinte intressanta av olika anledningar och nar man gar lite djupare in pa domda har dom ingen storre tillforlitlighet nar man val kollar pa hur signaturen ardefinierad t.ex.

Eva Lina: –Du har inga egna kategorier forutom dom som vi namnt?

–Nej, faktiskt inte. Jag har inga egna. Jag har nog inte sa mycket mer att sagaom det har utover det som jag redan sagt. Magnus far trakasera mig efterat omhan tycker att han vill ha fler svar.

Hur skulle du betygsatta dom grafiska interface som du anvant? Finnsalla dom saker du vill ha dar eller ar det nagot du saknar?

–Det varierar saklart. Men om man ska sammanfatta det jag har sagt innanlite. Det ar enormt vardefullt att dels kunna se definitionen pa signaturen foratt kunna bedoma om det ar falsklarm eller inte. Och dels att kunna fanga paket,sa att den fangar paket nar signaturen triggar eller helst da innan sa man servad som har skickats. Annars blir man alltid tvaa om man far satta igang denefter att det har hant. Det ar otroligt vardefullt att se paket runtomkring sjalvahandelsen.

Eva Lina: –Emilie ville ha ett enkelt satt att ga direkt fran GUI:t till commandline sa att man kunde kora sql-fragor direkt till databasen. For att ibland kandehon sig hemmad av GUI:t.

–Jo men absolut. Det ar det som vi jobbar med ocksa. Dar finns det satt darman kan ga hur langt som helst for att underlatta for anvandaren. Men i domhar kommersiella systemen dar finns det ju oftast, i alla fall som vi jobbat med,den typen av funktionalitet. Utan det ar ju i vara egna system. Dels sa kravs detatt man har en ganska stor databas med handelser som man kan soka i GUI:t.Men man kan ju inte spara hur mycket som helst for da blir det for langsamt,sa kanske tva manader tillbaka max.

–Men visst i det egenutvecklade granssnittet som vi har dar ar det ju typisktsadana dar saker man vill gora. Man vill hogerklicka pa larmet, satta igang entcpdump pa sensorn for att fanga in mer traffik. Gora uppslag/korreleringardirekt. Just den har typen av korreleringar och historiken dar det ar ju sadanfunktionalitet som jag tycker skall finnas med nar man klickar upp ett larm.Sa att det har tagits om han om av backend-systemet sa att saga. Sa att mansjalv inte skall behovs sitta och gora massa slagningar i nagon databas, utanden informationen finns dar oavsett om man ar ute efter den eller inte.

Sebastian: –Menar du typ vilka 5 kallor som triggade samma alarm och sadar?

95

D.2. OLA SODERSTROM

–Ja, den typen av statistik hade ju varit intressant. Om man tanker pa ensource-ip-adress som har triggat och man vill veta historiskt sett hur det harsett ut pa den har sensorn, har den triggat manga ganger, kanske hur andraoperatorer har bedomt den har handelsen, osv.

–Men visst sen finns det ju mer hognivafragor som man kanske skulle vilja stalla.Hur ser det ut lite mer generellt med signaturer och source och destinationsportar for att blida sig nagon typ utav uppfattning just i den har omgivningenda. Och allting och allting kan man ju inte bygga in i bakomliggande system.

–Att man bygger in i alla fall dom viktigaste grejerna sa att man far upp deninformationen nar man klickar pa alarmen. Typ om man har ett obfuskeratjavascript eller payloads osv, att man far upp det i larmet och kan dekoda detdirekt. Sa att man inte behover url decoda nagonting genom att ga ut pa natetoch klistra in en url, bara for att se vad fan det ar som han har skickar for javlaGET request har eller inte.

–Allting har ju som syfte att reducera den tiden det tar att komma till en slut-sats. Och desto fortare man kan skicka ivag en rapport eller informera kundenom att det hant natt har desto battre. All dom har funktionerna som man kanbygga in i GUI:t som underlattar, det ar liksom det man stravar efter hela tiden.

Du sa att ungefar 10% av alarmen brukar du klicka pa. Hur ofta bru-kar du ga utanfor interfacet och ut pa natet for att hitta informationinnan du kan bestamma dig for hur du ska klassificera ett alarm?

–Dom 10 procenten, da gor jag nog i princip alltid en korrelering i alla fall.Kollar i vara egna system. Historiken pa handelsen och sa dar. Och i 5% avfallen sa gar jag in och tittar pa signaturen och kollar pa pcap datan om denfinns tillganglig. I 2-3% sa gar jag ut pa sensorn och sniffar for att fa annu merinformation.

Vilka forbattringar vill du se for att gora det enklare att upptackaattacker?

–Rent GUI-massigt, ju mer information som kan formedlas direkt till operatorendesto battre. Jag skrev ner tre punkter dar.

–Battre korrelering. Det ar en sadan dar grej som ar sa oerhort tungt och kraversa valdigt mycket hardvara for att uppna det man vill for att det ar sa tungafragor man maste stalla for att fa den dar korreleringen. Men mer korreleringoch kanske battre forstaelse for vilka typer av enskilda signaturer som ar in-tressanta tillsammans med andra signaturer. I sig sa ar dom aldrig intressantamen tillsammans med andra blir dom intressanta. Om tillverkarna kunde sjalvakomma fram med egna metasignaturer mer sa hade det hjalpt till.

–Sen generellt satt sa ar ju kvalliten ganska lag pa signaturer, det ar valdigtvaldigt mycket falsklarm. Om man kunde anstalla nagra nissar som gor lite mertester innan dom slapper signaturerna, sa att det blir lite battre kvalite och truepositives.

96


–Sen skrev jag avvikelsedetektering. Det var lite grann det som jag namndetidigare att man pa natt satt kan bygga system som upptacker avvikelserna idom har enorma strommen av handelser som man far. Dar kan man ju ga inhur djupt som helst pa den source-ip-adressen pa just det har natverket. Varforkommer avvikelsen, ar det pa signatur-niva? osv.

–Men molnet kanske raddar oss och allt kommer bli bra...

Eva Lina: –Ja, hoppas kan vi ju alltid gora.

Skulle du foredra ett system som larsig automagiskt hur du vill hadet, ett dar man maste konfigurera allt (har full kontroll) eller kanskenagot dar i mellan?

–Jag har nog inga problem att slappa kontrollen. Men samtidigt tycker jag attsen sant system som lar sig av hur en opreator klickar pa alarm osv. Det trorjag skulle vara valdigt vardefullt. Men det kanns nodvandigt att det ar nagontyp av IDS-administrator som kan ga in och se vilka beslut som har fattats ochkan overrida dom beslut som har fattats och gora justeringar och ha insyn i denprocessen. Men jag tror att man skulle kunna komma ganska langt bara pa atttitta pa klick om det ar hyfsat erfarna operatorer som sitter dar. Om man seratt de har larmen dom klickar man alltid bort, man hackar alltid bort dom utanatt ens klicka upp dom.

–Nar jag borja dar och man sitter med nytt GUI, sen blir man helt forstord. Alltdet kreativa tankandet bara forsvinner. Men da hade man massor med ideer omvilka forbattringar man skulle kunna gora med just GUI:t. Och da kommer jagihag att en ide var att system skulle lara sig att om fyra operatorer hackar bortett alarm i en veckas tid eller tva dagars tid, det kanske man kan klassa somgront (inte sa intressant). Sen kanske nagon IDS-administrator kan samla allagrona alarm och gora en oversikt och sen saga att; okej dom har alarmen domtar vi inte in i vart system langre. Och sa kanske man har roda alarm som manalltid dubbelklickar pa och man ar alltid pa sensorn och gor en tcpdump. Dykernagonting sadant upp igen da taggas det automatiskt rott.

–Nagon sadan typ av visuella hjalpmedel sa att man direkt kan bilda sig enforsta uppfattning skulle vara bra.

Eva Lina: –Mmm, for det snacka vi om att vi skulle gora. Att vi har toppen meddom hogst rankade, dom roda, sen tankte vi ha en lista med bubblare med domsom okar mest, dar deltat ar som storst. Och sen sen lista med helt nya alarmsom aldrig setts pa natverket innan och det ar ingen som har klickat pa domnagonsin. Och sa naturligtvis sa att man kan fa tillgang till alla alarm. Men attha dom dar tre tyckte vi kandes ratt vettigt.

–Absolut, det kanns jatte vettigt.

Eva Lina: –Men vad bra da var vi inte helt fel ute da iaf.

Ar det viktigt att kunna ge negativfeedback? Att kunna ranka ettalarm lagre och ange anledning(ar) for detta till systemet? Om mant.ex. har en dator i natverket med en kand bugg/svaghet dar det

97

D.2. OLA SODERSTROM

ar viktigt att veta om en attack sker. Men om den senare patchasoch man inte langre vill se den och isf ta bort den m.h.a. negativfeedback.

–Absolut det ar ocksa en funktionalitet som ar valdigt viktig i ett sadant system.Sa att jag ser dom inte som omsesidigt uteslutande dom har olika systemen nihar. Sen kanske man kan tanka sig att i dom fallen det ar en viss typ av sarbarhetoch man vet att nasta tisdag sa skall Microsoft slappa sitt service pack for dethar. Och vi kommer skicka incidentrapporter till kund fram tills dess men efterdet ar det inte intressant. Da skulle man vilja satta nagon life-time pa den harsignaturen: den ar rod tills tisdag, och sen gar den ner pa gron. Sa man kansatta nagon tidstag pa dom.

–Men absolut, manuell negativ feedback. Vem som ska gora det, om operatorenskall kunna gora det sjalv eller om det skall vara en IDS-administrator som skallgora det, det ar vall fragan man kan ta stallning till. Men mojligheten den kannsviktig.

Eva Lina: –Magnus hade ju fyra olika interaktionssatt med IDS:en. Dels baraatt man tittar pa var klickar anvandaren. Dels att anvandaren sjalv kunde angeanledningen till varfor han klicka pa alarmet. Och sa att systemet sjalv gissadepa hur anvandaren ville ha det, sa far man sitta och halla med eller inte hallamed. Och sa tillslut att den fragar explicit om hur den ska ranka alarm.

–Vad sager du om dom har olika graderna av interaktion? Eller vill du bara attden skall halla kaften?

–Det ar ratt stressigt att sitta dar. Man hinner inte ge feedback till systemet saatt det lar sig. Och att sitta och fundera pa var det hamnar pa en skala mellanett och tio, det tror jag inte skulle funka for oss i alla fall.

Eva Lina: –Sa det har med att ange varfor du ar intresserad, det ar inget fordig?

–Nej. Bara ren klickning och sen i efterhand att nagon med stor erfarenhet kanga in och gora dom forandringar som behovs. Det tror jag ar mycket battre.

Eva Lina: –Men det har med att man kan halla med eller inte halla med attman har t.ex. en ja- eller nej-knapp. Sa att man kan tala om for systemet attja det har var bra rankat eller nej det har var daligt. Vad sager du om det?

–Den kraver ju inte sa mycket interaktion. Den ar nog inte sa dum. Ar det ettlarm som klassas rott och sen tycker man att det har har blivit fel. For det arbara en massa urusla operatorer som har suttit och rankat for jag vet att jagvet att det har ar inte intressant. I dom fallen hade nog varit bra att ha denmojligheten i GUI:t.

Eva Lina: –For da har man ju dels det har att det blir ointressant over tid, attman kanske slutar klicka pa det och det langsamt dor bort. Och dels sa kan manaktivt doda det valdigt fort.

–Just det, sa man far bada varldarna dar.

Eva Lina: –Jag antar att den dar sista valdigt verbala och pratglada IDS:et somhela tiden fragar om hur alarm skall rankas inte heller ar nagot for dig?

98


–Nej, det ar absolut ingenting for mig.

Ser du nagra nackdelar eller risker med ranking?

–Ja det ar klart att det finns. Det gor det ju. Det kan vara sa att man lagger forstor tillforlitlighet i den har rankingen. Och sen vaggas man in nagon typ av...Sag att man har en skala pa ett till fyra och sen ar det natt som ar pa tva ochdom ger man fan i att titta pa for att dom bara ar pa tva. Men det finns andraorsaker till att den ar dar nere och den borde inte egentligen vara dar nere, utanden borde vara lite hogre. Hade man inte haft rankingen fran borjan da hademan kanske agnat den storre uppmarksamhet. Det ar dar det kravs nagon medlite storre erfarenhet som kanske gar in med jamna intervall och kollar den hurrankingen sker och av vilken anledning den har fattat besluten sa man kan gorajusteringar.

–Och man kanske ocksa ska ha ett system dar man laser vissa signaturer tillen viss rank for att dom ar alltid intressanta. Det skall inte anvandare ochoperatorer kunna hacka bort, utan dom ar lasta och man vet att dom larmenar kritiska. Men annars kommer jag inte pa nagra omedelbara nackdelar medrankingen.

Eva Lina: –For det var vall det som Hakan Nohre pa Cisco sagade ranking med.Han tyckte att nagon kan ju ranka det fel och da kan det tippa hela systemet.Och vad kommer handa om anvandare andrar sig efter ett tag, vad ska vi goramed det da? Ne det verkar inget bra, tyckte han.

Sebastian: –Ja det ar ju det som ar problemet. Grupppsykologi kan ju fa eninverkan med. Da kanske man far tanker natt i stil med; det har ar ju gron, dakanske det ar nagon som vet battre an mig som har rankat. Man borjar tvivlapa sig sjalv.

–Exakt. Den risken finns dar. Men jag tror att ett sadant system satter lite kravpa dom som jobbar med det, sa att dom har ganska stor erfarenhet. Den harproblematiken den kan man ju komma run ocksa genom att man kanske viktaranvandares mojligheter. Sa att en som bara har jobbat dar i ett ar kanske intehar mojlighet att ranka sa hart. Utan hans klick betyder mindre an nagon somhar jobbat dar i tio ar. Da reducerar man riskerna att en nykomling sabbar helasystemet.

–Men jag tror att den mesta av den problematiken som Cisco snubben tar upp,den kan man nog komma runt om man lurar lite grann. Om man ser det somen hjalpreda och inte en absolut sanning. Utan att man tar det for vad det ar.

Har du nagra ovriga kommentarer eller synpunkter?

–Nej jag tror inte det...

Sebatian: –Jag tankte pa det du sa att nar man klickar pa alarm sa kan manfa upp relaterad information, statistik och annan intressant information. Trordu man skulle kunna anvanda den informationen for att pa natt satt ge enslags rank- eller relevans-poang till alarmet? Om man hittar mycket relaterad

99

D.3. EMILIE LUNDIN BARSE

information och ser att ett source ip har orsakat en javla massa alarm de senastetio minuterna...

–Absolut. All den typen av korrelering som gors den borde ju anvandas. Denhar ju som syfte att man ska bilda sig en battre uppfatting om handelsen ochden hjalper ju. Den skulle ju ocksa kunna vara inbyggt i rankningssystemet.T.ex. om en signatur triggar men source-ip-adressen triggade ocksa en annansignatur igar pa det har natverket, och det gor att den har handelsen klassassom allvarligare an om den bara var isolerad.

–Dar kan man grava jatte mycket kanns det som. Vad ska man titta pa dar. Sig-naturen? Source-ip-adressen? Vilken miljo den forekommer i? Vilka troskelvardenskall man satta? Tillsammans med vilka andra signaturer kan det vara intres-sant? Hur langt tidsspan skall man ha, nagon manad bara eller ett ar?

–Men absolut. Jag tror att det skulle kunna hjalpa om man forde in den infor-mationen i rankningsprocessen.

D.3 Emilie Lundin Barse

Transcript from interview of Emilie Lundin Barse from Combitech. The inter-view was done in person and recorded.

Intervjun borjar med en demonstration av verktyget OSSIM.:

–Snort ar ju egentligen huvudkomponent i OSSIM och det som du far intrangs-detekteringslarmen ifran. Iden med dessa verktyg ar att du ska kunna fa ininformation fran andra kallor och loggar, etc. I den gamla installationen plockarvi in loggarna fran brandvaggen och da kan man korrelera droppade anslutning-ar, till exempel folk som forsoker komma in genom brandvaggen pa portar somar stangda, med larm om de IP-adresserna dyker upp i nagot sammanhang. Detar intrangsdetektering med add-on-information.

–Sa har i den gamla versionen har du BASE-granssnittet men de haller pa attlagga upp sin egen eventviewer som de har valt att kalla den. Dar kan du dela uppsaker i olika flikar. Du kan ocksa installera snare for att plocka in windowsloggaroch OSsec som ar en hostbasered linux-IDS och da poppar de upp har.

–Den kollar lite pa vad som hander pa sjalva sensormaskinen ocksa. Alla larmenhamnar i den har fliken, men detta ar ju det som kallas forensic, det mer attgrava i nar man redan vet att nagot har hant. De har en larmflik dar korrele-ringsregler eller direktiv gor att hogprioriterade larm eller events poppar over ilarmfliken. Tanken ar val att man bara ska kolla pa de som ar hogprioriterade.

Eva Lina: –Staller man in i nagon konfigureringflik vilka som prioriteras?

–Ja, man far lagga in sadana regler. Men den har ett gang inbyggda ocksa. Omman far ett snortlarm pa en attack mot en port har den natverksverktyg somkollar om porten svarar pa trafiken. Om den gor det far det hogre prioritet ochblir till exempel ett larm. Man kan lagga in en massa sadana som de kallarpolicys.

100


–Om man tittar pa direktien, har den sadana har recurrent snort event. Farman valdigt manga snort events av samma typ larmar den pa det.

Sebastian: –Kan man gora motsatsen ocksa? Om man far valdigt manga, kanman ignorera dem da kanske?

–Jo man kan skriva en policy. Man anvander den till direktiven men man lagrardem inte, sa de ligger inte kvar i den har eventfliken. Den gor bara korreleringeni minnet men lagrar inte dem i databasen.

Eva Lina: –Spar ju minne.

–Ja, jo. Den anvander ju p0f som ar en passive OS fingerprinting och dar kanden saga host same och det ar inget man sparar pa. Den har detekterat att detforfarande ar en windowshost, samma maskin, det ar kanske inte nagot man villutreda direkt.

–Om man gor utredningar sa ar det faktiskt ofta de positiva handelserna somman vill ha och de felaktiga. Ta brandvaggsloggar som exempel, om du hittarnagonting som gor att du tror att en server ar hackad sa ar det ju mer intressantmed vilka sessioner som har kommit igenom brandvaggen. Faktum ar val att detofta ar manga utredningar dar det ar viktigare med vad som har lyckats n vadsom har misslyckats. Lyckade inloggningar kan vara viktigare an misslyckade.

–Nar du far upp ett alarm kan du klicka pa det och da fa upp listan over events,det kan ju vara flera events som orsakar samma larm. Vi kan ta och titta panagra olika regler, vi har gjort en del egna, just da med brandvaggsloggarnaoch lite andra saker. Rare but open destination port har lite sadana regler. Omdet ar mer an 30 sekunder som sessionen pagar prioriterar man upp den. Detar ganska bra, att inte bara ha intrangsregler utan ocksa andra saker.

Sebastian: –Kan jag anvanda de har reglerna rekursivt? Om jag da vill ha ettattackscenario, kan jag da saga att det har beteendet liknar den har trojanen,och sedan hander det har. Jag refererar till ett id och sager att det har handerforst och sedan det har?

–Du far lagga ihop det i samma direktiv tror jag, men det gar ju att gora fleraegna grejer i samma direktiv. Men jag vet inte riktigt hur, har inte anvant det.

–OSSIM ar det jag anvant mest. Jag tycker man kan kanna sig lite begransadhar och skulle vilja ha battre mojligheter att sortera sjalva pa vilken parameterman vill och ha ett friare granssnitt, mer ett sql query-granssnitt med stod forregexpar.

Sebastian: –Sa att man kan gora ungefar som views i sql sa att man far en egen?

–Det finns, jag har provat ett par stycken sadana har security informationmanagement-verktyg som ar annu mer inriktade pa att man stoppar in allasin loggar i dom snarare an ids-larm. RSA enVision ar ett verktyg dar de harett sqlgranssnitt. Fast man far lite stod sa att man inte behover skriva queriesutan du har lite hjalp.

Sebastian: –Men finns det inte nagon sokfunktion? Det har en i BASE.

–Du kan ju lagga in tidsperioder. Granssnitter ar ganska krangligt men du kanlagga in tisdperioder och du kan lagga in IP-adresser, vilken sensor larmet kom-

101


mer ifran och sa vidare, men du kan inte soka pa riktigt vad du vill. Man skullevilja ha nagot mer liknande googleindexeringssokning.

–Jag har anvant Splunk ocksa. Det anvands till att stoppa in textbaserade loggari och det gor nagon slags sokmotorindexering av loggarna. Det ar ganska frackt,for man skulle vilja ha den hjalpen i OSSIM.

Eva Lina: –Men du brukar alltsa klicka runt pa alarmen?

–Ja det gor jag. Jag tycker att man nastan alltid behover ga in och titta painnehallet. Den har ju manga kallkodlarm och sql injection-larm och da far mani princip hexkoden dar nere. Jag tycker att det nastan ar alltid man behovertitta pa det for att se om det ar ett falsklarm eller inte, sarskilt nar det gallerkallkod. Om man har det aktiverat falsklarmar det valdigt mycket. Ofta triggarden pa vissa websidor.

Eva Lina: –Tycker du att det racker med informationen som finns har?

–Nej, egentligen inte.

Eva Lina: –Vad brukar du vilja ha forutom detta?

–Sessionsinformation, man ser om det har etablerats en session och hur myckettrafik som har gatt och eventuellt vilka andra maskiner attackeraren har pratatmed. Det andra verktyget jag anvant heter Sguil, ar nastan ett snortgranssnitt.Det kor jag faktiskt hemma. Det har nog utvecklats av det amerikanska forsvaretsom sitter i 24/7-overvakning och verkligen tittar pa varje larm som kommer.Sa i och for sig, det ar gjort mer sa att man verkligen tittar pa varje alarm.

–Har har de markerat ett alarm, mysql insertion overflow attempt. Det ar typisktsadana jag ser hemma ocksa, bakgrundsbrus pa internet kan man val saga. Detar speciellt infekterade maskiner i Asien, jag antar att de inte har nagon updatepa sina maskiner...

Eva Lina: –Det ar val ratt mycket piratkopierat dar borta ocksa?

–Ja, sa de betalar val inte for sina windowslicenser direkt och da har de ingenuppdatering pa dem.

–I det har granssnittet delar dom upp det i hogprioriterade alarm, mellanpriori-terade och lagprioriterade i olika panes. Man kan konfigurera om det lite grannoch sedan kan man klicka upp regeln som har triggat och sa visar den paketin-nehall och sa kan du aven har gora en lookup pa IP-adressen. Om man sokerpa natet pa det som star i paketet ser man att det ar sa har det ser ut nar eninfekterad maskin forsoker sprida sig vidare.

–Det har gor atminstone aggregering, aggregerar pa kall-IP tror jag. Den ar julite korkad da, for vissa saker ill man ju aggregera pa distinationsIP och andrapa kall-IP. Den har count har, sa den har har 64 stycken likadana larm har.

Eva Lina: –Hur manga mojligheter har man har att aggregera annorlunda?

–Har ar det bara att den samlar ihop samma, sa har har man inte sa mangamojligheter. Tanken ar att man ska eskalera, att man tittar pa varje alarm ochsedan kan kategorisera dem som ointressanta och da forsvinner de.

102


–Jag har gatt nan kurs SANS Institute-kurs, de har bra sakerhetskurser. Jaghar gatt deras intrusion detection-kurs och dar berattar de hur de jobbar i dehar verktygen. De har junior analysts som sitter och kollar pa alla alarm somkommer upp. Sedan har de senior analysts som kollar pa det som har eskalerats.

–Man kan ju alltid satta pa coh stanga av snortregler och lagga in egna regler. Ivissa sammanhang har man valdigt fokuserad detektering och da har man baravaldigt specifika regler for saker som man vill upptacka for en viss tjanst.

–Inom forsvaret tittar man oftare pa vissa IP-adresser, overvakar lite vad domsysslar med. Man anvander i princip inte standardreglerna.

Sebastian: –Hur mycket folk har dom som sitter och kollar? Ar det sa de loserproblemet? Att de har senior och junior? Hur mycket later de systemet klura utsjalv vad som ar farligt och inte farligt eller gor de helt enkelt sa att de har enhimla massa folk som tittar pa allt och sa far de bedoma vad som ar farligt ochinte farligt?

–Det blir ju sa, man tittar pa vissa saker mer noga, man har ju begransat medtid. Ett tips jag fick pa hur man kan kolla ar att titta pa de mest allvarligaalarmen och sedan titta pa sadant som dyker upp som enstaka handelser foratt inte missa nagot. Just det har med droppade saker i brandvaggen, vad somar ganska vanligt ar att man tittar pa top 10 och bottom 10. Fa handelserar antagliggen nagon som manuellt forsoker gora nagonting. Top 10 ar oftastscanningar da de bara scannar igenom allt.

–Men Sguil gillar jag. For hemmabruk dar man har ratt lite trafik ar det bra.Det kommer bara att rulla fobi alarm om man har det mesta paslaget som vihar gjort i vissa sammanhang. Det har ar mer gjort for att trimma ner lite pareglerna skulle jag nog tycka, medans i OSSIM har man filosofin att man spararpa alla alarm medans man inte tittar pa alla om man inte tror att nagot specifikthar hant. Det ar lite olika satt att jobba pa.

Sebastian: –Men da funkar inte OSSIM lika bra i hogtrafiksammanhang?

–Jo det gor den ju, for du behover inte titta pa allting som kommer in. vi harhaft lite samarbete med Saab Microwave sa vi har det installerat hos dem. Detblir jattemycket men det gar ratt bra, man far rensa lite sa att diskarna inteblir fulla. Och det finns begransningar, man kan inte ha hur mycket som helsti databasen samtidigt. Men det ar ju problemet med alla databaser. Det blirju ratt segt om man har ett par 100.000 events i databasen. Sokningarna blirvaldigt langsamma. Da kanske man inte kan ha mer an ett par dagars handelsersamtidigt. Man far nastan ha en maskin som importerar och exporterar datafor de tidsperioder man vill soka.

Sebastian: –Det har man ju last om, att da det gar for langsamt att logga tilldatabasen dumpar dom till nagot binart format och sa far andra maskiner staoch tugga igenom den dar filen och lagga den i databasen.

–Man far nastan bygga upp en san struktur, att man har en maskin som samlarin data, en som gor realtidsanalys och sedan en separat maskin dar man gorde har manuella sokningarna eftersom de stor realtidsanalysen. Sa man behovertva analysmaskiner.

103


–Pa Microwave har de kort Snort ganska lange och om du koper Snort har denett litet battre granssnitt an BASE pa den kommersiella versionen.

–Det tar oftast tva ar innan man ar nojd, konfigurerar lite regler, stangt avdet som stor. Ofta upptacker man konfigurationsfel i natverket och da atgardarman kanske de grejerna i borjan, sa det ar valdig tmycket som ligger och stori borjan. Jag har hort det fran flera att det tar cirka tva ar innan man kanneratt man kan jobba i det.

–Men jag kan saga att jag har egentligen inte jobbat med intrangsdetektering.Jag har gjort tva installationer av OSSIM i tva olika nat och vi har gjort vis-sa anpassningar, grejat med regler och policys och lite direktiv, men vi harinte varit med hela vagen tills det blev anvandningsbart. Det blir ju mycketskraphandelser. Det var ju som nagon sade att det forsta man behover konfigu-rera bort ar overvakningsmaskiner som sakerhetsavdelningen anvander, for deligger ofta och spyr ut trafik som triggar regler. Att just konfigurera bort attmaskinen sjalv generar handelser ar en sak man behover gora.

Sebastian: –Da ar de alltsa de hetaste maskinerna man kan infektera i ettforetag? Det ar de maskiner som sakerhetssystemet ignorerar.

–Precis. Faktiskt, det ar valdigt kansliga data om man hittar en sadan databasoch kan ga in och titta. Det har varit vissa sadana verktyg som per defaultvarit oppna mot internet. Att ga in och kunna titta pa ett foretags snortlarmar ju inte bra. Man kan se vad det ar for IP-adresser, vad maskinerna har forfunktion. Jag kommer inte ihag vad verktyget hette, jag tror de tipsade om deti kursen jag laste och fick prova igenom nagra olika verktyg.

Sebastian: –Google ar ju bra pa det dar ocksa, hitta sadana saker alltsa. Jagkan tanka mig att man kan gora en sokquery i google for att hitta alla BASE-installationer.

–Just det, det var nog sa man hittade de har maskinerna som hade det hargranssnittet oppet. Det var nagon sokstarng man kunde anvanda for att hittaalla oppna sajter.

Vilka granssnitt och IDSer har du anvant?

–Granssnitt: Sguil, BASE, det ar snortgranssnitt allihop. Sedan har jag ocksaanvant Cisco MARS. Det kan man val ocksa se som ett granssnitt, men det aregentligen ett security information managment-verktyg. Det ar ju ett ramverk.

Hur tyckte du att man kunde skota aggregering i det?

–Nu korde de ju inte Snort sa jag sag inte riktigt det egentligen, men det hadeandra roliga features i granssnittet. Nar du fick upp ett larm kunde du klickapa nagon liten ikon och fa upp hur trafiken gatt i natverket. Man samladeinte loggar fran routrar, switchar och brandvaggar. Man anvander ju inte CiscoMARS om man inte har ratt mcyket ciscoutrustning, men man kunde fa uppatt “jaha da har det gatt genom den switchen och den routern och blev stoppati den brandvaggen”. Det ar ganska snyggt. Da vet man ju att trafiken faktisktinte gick fram till servern den forsokte attackera. Det ar sadan ovrig information

104


som ar valdigt bra att ha, just det har med brandvaggens logg dar man far vetabade vilka sessioner som har gatt igenom och vilka som blivit stoppade.

Sebastian: –Men det ar bara om man anvander Ciscos? Det finns inga oppnaprogram for att...?

–I OSSIM kan du gora det. Du kan plocka in loggarna fran brandvaggarna ochegentligen gora sparningen sjalv. Cisco har ju tagit det ett steg langer an OSSIM.I OSSIM kan du ocksa se om maskinen har svart pa trafiken.

–I Sguil har en agent for sessionsloggning. Pa det natverkssegementet du lyssnarpa loggar du alla sesioner, vilka IP-adresser som har pratat med varandra pavilka portar och hur mycket trafik som har gatt. Du kan hogerklicka pa ett eventi Sguil och fa upp vilka andra maskiner den attackerade maksinen har pratatmed mer pa natverket. Dar har du kanske annu battre mojligheter till det.

Men om du tanker pa alla interface du anvant, ar det bra aggre-tionsmojligheter eller blir man frustrerad och kanner att man villgora mer och annorlunda?

–Ja, man skulle vilja att de var mycket battre (skratt).

Eva Lina: –Kan man paverka dem pa nagot satt och tala om for dem att “sahar vill jag aggregera och gor det at mig”?

–I princip. OSSIM ar open source sa du kan ju lagga till dina egna plugins. Dukan ju lagga till en egen flik dar du gor en massa annat. Men det ar ju klar, detar ju en massa jobb.

Eva Lina: –Det finns forbattringspotential alltsa?

–Ja, det finns forbattringspotential i alla verktyg. Kan ju lagga till RSAenvisiondar ocksa. Men det ar mer allmant for loggar och jag har anvant ett antal sadanaverktyg. Det finns ett som heter ArcSight ocksa som ar generellt for loggar. Mendessa tva ar generellt for loggar och inte bara for natverkssakerhetsovervakning.ArcSight har jag inte anvant men jag har sett lite av det. De ar ganska mycketfor korreleringsmojligheter, men det ar som med OSSUM, att man far laggain det sjalv. Alla system ar unika sa alla nstallationer av sadana system arflerarsprojekt. Man kan inte bara installera det och tro att det fungerar.

–Du skulle kunna stoppa in snortlarmen i Splunk. Det ar mer googleindexeringpa textloggar, men du skulle kunna stoppa in alarm i den ocksa. Dar har duganska bra stod att tala om vad olika falt ar for nagonting. Dar har man litemer sadana sokningar som jag skulle vilja gora.

Sebastian: –Sa det skulle man kunna kombinera med nagon av de andra?

–Fordelen med Snort ar ju att du kan logga i olika format och inte bara binart.Det finns ett plugin som heter Barnyard, det ar en egen process. Snort loggarbinart och sa har du Barnyard som gor om det till flera olika format. Jag trorpa att man vill ha flera analysverktyg till samma loggdata. Det ar nog inte ettverktyg som ar ratt for alla system.

105


Vad vill du ha ut av ett alarm? Det vanligaste ar ju att du far ut tid,alarmtyp och kalla, destination osv.

–Paketdata vill jag definitivt ha, annars kan man oftast inte saga nagonting omdet.

Eva Lina: –Du sa att du titta pa sessionerna ocksa, ar det nagot mer an det duvill at?

–Egentligen vill man ju ha allt (skratt). Det forsta man gor ar att klicka palarmet, titta pa paketinnehall. Nasta steg ar att kolla vilka andra maskiner denhar maskinenn har pratat med. Vilka andra alarm har den har maskinen orsakat.Nasta steg ar att ga in i serverloggarna och kolla, kan man ha separat, om detser ut som om nagot har hant pa den har maskinen. Om det ser ut som om denlarmar for att det finns nagon bakdorr pa en maskin ar det klart att man garin och kollar om bakdorren finns dar. Men da har man kommit lite utanfor detsom intrangsdetekteringssystemet ska gora.

–Men om det har hant nagonting kan man spela in fullstandig natverkstrafikunder en viss teidsperiod. Jag vet folk som har gjort det. Man kan lagga upp enegen process som spelar borjar spelar in. Nar du far ett visst snortlarm spelardu in all natverkstrafik under kanske 5 minuter for att fa mer information ochda far du ocksa en mer komplett bild utav vad som har hant. Men har du mangalarm kan det bli hur mycket trafik som helst, fast du kan ju valja att bara tafor en viss IP-adress da i och for sig.

Eva Lina: –Ar det nagra andra verktyg forutom de du har visat oss som duanvander for att komplettera bilden?

–Informationsmassigt sett?

Eva Lin: –Magnus pratade om tcpdump...

Tcpdump ar val egentligen standrad 1A, sa det anvander jag. Man kan skrivafilter i det ocksa. Det ar ju ocksa till for konfigurationen, att forsta varfor dentriggar pa vissa saker och for att felsoka. Tcpreplay for att testa, man kan spelaupp en fil md trafik som man vet att det finns attacker i, sa kan man ser attden triggar pa det.

–Vad har man mer? Wireshark har man for att kunna gora ytterligare analys.Men OSSIM innehaller flera andra verktyg, sa den innehaller p0f som ar en pas-sive OS fingerprinting. Den kollar trafiken och ser vad det ar for operativsystempa maskinerna som pratar. Och sa har den NTOP som gor viss trafikanalys saatt man ser vilka protkoll som ga. Man ser trafikmangd.

–Vad som aven ar intressant ar att se hur manga events man far. Det finns ettverktyg som heter pnGraph som tittar pa Snorts statistikfil och ritar grafer padet. Man ser over tiden hur manga alarm man har fatt, och sa ser du trafikmangdover tiden.

–Over huvudtaget trafikbeteende, NTOP visar det. Det ar ett open source-verktyg, men dar far man gora lite konfigurering. Man behover en liten servermen det ingar i OSSIM om man installerar det.

–SGUIL har ju fordelar helt klart for hemmabruk dar man far ganska litehandelser. Du har battre sokmojligheter tycker jag. OSSIM har varit lite buggigt

106


ocksa men det ar ju klart, det har utvecklats en del. Det finns en kommersiellversion med appliance, en hardvarulada som du installerar.

Eva Lina: –Nar det hander nagot och du till exempel slar pa tcpdump, har duautomatiserat det eller gar du in manuellt och startar det nar du ser att nagothander?

–Det har ju varit pilotinstallation det jag har gjort. Jag har ju inte drivit dethela vagen tills det ar fullt fungerande, sa det har bara varit att jag har gat tinmanuellt egentligen. I och for sig, jag har ju gjort ett script som spelar in medtcpdump hela trafiken under en viss tridsperiod efter ett larm. Det har jag gjortnagon gang.

Sebastian: –Laggs det in i snortfilterreglerna?

–Nej, det ar helt separat, du gor en plugin som ar kopplad till snortreglerna.

Sebastian: --Jag har bara last manualen, men dar kan man lagga in samtidigtsom man skriver reglerna for vad det ar den ska reagera pa, saga till den attgora vissa saker.

–Jo, men det var via det. Men det var ganska lange sedan sa det ar knappt attjag kommer ihag hur jag gjorde. Sedan fick man gora ett annat script som sagtill att rensa bort efter en viss tid sa att det inte blev fullt pa disken, for dethar man ju ocksa problem med om man spelar in trafik. Man far ha lite koll pavad man gor.

Eva Lina: –De har verktygen du har anvant, finns det nagon mojlighet for dematt samarbeta, att fa dem att prata med varandra? Far man hacka sjalv eller ardet hopplost?

–Du menar tcpdump...?

Eva Lina: –Jo, och de andra verktygen man kan tanka sig att anvanda.

–Man far hacka lite sjalv, men de ar ju gjorda for det. Tcpdump anvander manofta lite separat. I och for sig, just nar du kopplar det till Snort har den stodfor att du ska kunna kora vilket program du vill nar ett larm trippar.

Sebastian: –Omvant da? Jag ankte pa Ciscos MARSsystem. Dar ar det tvartom,att de helst vill att det bara ska vara inom Ciscos sytem. Eller tillater de ocksaatt man kopplar in lite vad som helst?

–Snort kan du anvanda ihop med Cisco MARS. Cisco har sin egen IDS ochsa stodjer de Snort. Men Snort ar sa spritt att man maste nastan stodja den.Men Cisco stodjer ju inte sina konkurrenters. Men daremot, OSSIM stodjernagra olika IDSer, men de ar ju inte heller bundna till nagon och aven ArcSightoch RSAenvision stodjer flera olika. De ar ju inte heller bundna till nagot IDS-foretag. Tittar man pa till exempel Symantec har de ocksa nagon programvarafor att knyta ihop IDS-larm och andra loggar. Och da stodjer de ju framfor alltsin egen. Men nastan alla stodjer sin egen IDS och Snort i alla fall.

Brukar du kategorisera dina alarm?

107


Eva Lina: –Du visade oss lag-, mellan- och hogprioriterade larm. Brukar dulagga till egna prioriteringar forutom dessa? Magnus hade pa forslag “ointres-santa”, “dubletter - orsak redan kand”. Har du nagra egna indelningar forutomdetta?

–I SGUIL kan man lagga in olika kategorier, fast man kanner alltid att de in-te passar. Man vill ha lite egna. Men sedan har du att du kan lagga in dinegen kommentar till larmet nar du behandlar det. Det ar bra. Over huvudtagetbehover man ha nagot sa att man kan folja upp efterat. Sarskilt om man kon-figurerar alarm maste man ha nagon slags changelog pa vad man har gjort ochvarfor. Annars kommer man inte forsta det. Jag har anvant mest det inbyggdamen man skulle nog vilja gora mer an vad det finns stod for egentligen. Jagskulle vilja gora egna kategorier och kunna flytta over dem och ha flera olikaflika for olika kategorier.

–Jag tror att man altlid kommer vilja ha sa att man kan konfigurera upp sjalv.Sedan har man kanske nagra som foljer med.

Det finns allmanna regler som triggar pa lite allt mojligt och sa finnsdet mer specifika. Har du ngaon indelning av hur tillforlitligt ett larmar? Till exempel att for detta larm finns det 90% sannolikhet att detar riktigt och for ett annat larm ar det bara 20% sannolikhet attdet ar riktigt. Anvander du dig av en sadan indelning pa olika alarmocksa?

–OSSIM har det sjalv. Snort har tre prioritetsnivaer pa larmen men OSSIMhar aven reliabilitynivaer. Jag tror aven att den lagger in det pa snortlarmen,att vissa ar mindre palitliga an andra. Det dar skulle man behova sitta ochkonfigurera upp, for det ar olika for olika system. I OSSIM kan du per larm gain och satta den har siffran och det ar viktigt.

–Problemet ar att vissa opalitliga larm handlar om valdigt viktiga saker sa mankan inte bara ga in och stanga av dem.

Eva Lina: –Den korrelering jag sett far man stalla in sjalv i config-filer. Du harinte haft nagra automagiska system? De lar sig, korrelerar, dumpar ut en vackerfin, bild och talar om for dig att ditt natverk ar helt sakert?

–Napp!

–De kan ju ge sken av att vem som helst skulle kunna anvanda dem da man sergrafer och sa vidare, men i princip sa bygger det alltid pa att du har nagon somverkligen vet vad gor i botten. Sedan kan du gora vissa managementvyer somman kallar det. Man lagger upp vissa grafer, till exempel antal larm over tid.Sadant kan ju management tycka ar intressant. Okar eller minskar hotbilden panagot satt?

Eva Lina: –En allman regel med 20 eller 10% tillforlitlighet, hur bar du dig atom du ska verifiera om det ar riktigt eller ett falsklarm?

–Ibland kan man se i paketinnehallet. Ar det nagot som pagar kan man spela innatverkstrafiken och kolla pa den. Annars ar det bara att ga in pa servern ochkolla om det verkar ha hant nagot, har trafiken ens kommit fram?

108


Eva Lina: –Och de grafiska interfacen? Kanner du att du har nagot att tillaggaeller kanner du dig terapeutiskt rensad?

–Ibland tycker man att kommandoradsgranssnitt har sina fordelar, men det arju bade och. Man vill kunna fa upp grafer over vissa saker. Men att ga in idatabasen och gora sql-fragor eller sql-script anvands ratt mycket. Nar man harsnortlarm ar det ratt ofta man gar in direkt mot databasen och kollar saker.Det gar fortare, det ar mer kraftfullt och du kan soka pa vad du vill. De grafiskagranssnitten brukar ju tyvarr begransa vad man kan soka. Problemet ar att detalltid ar sa unikt hur man jobbar, det ar svart att fa ett granssnitt som passaralla.

–Men jag ser inget problem med att man har ett grafiskt granssnitt som mangor vissa saker i och sedan kan du gora sql-sokningar bredvid. Men det ar juinte sa dumt om det finns en liten flik som gor att du kan gora sokningar pavilka falt du vill i granssnittet ocksa.

Sebastian: –Det ar val det som ar problemet. Det ar tva saker som ar valdigtmotsatta varandra. Att ha ett intuitivt GUI och sedan ha alla kraffulla verktygsom finns under. Det gar inte att fa ihop.

–Det finns inget intragsdetekteringsystem som en idiot kan skota egentligen -idag. Du far ju en massa konstiga falsklarm. Pa en kursen som jag gick, om dukorde en viss p2p-trafik ihop med Junipers IPS tyckte den plotsligt att manhade trojaner pa alla sina system.

–Jag tror inte riktigt pa IPS. Man kan ju i sin brandvagg stoppa viss trafik somman ar saker pa ar dalig. Men IDSen har en annan roll, den gor overvakningoch den ska man helst ha passiv sa att man sjalv kan agera pa det man tyckerar viktigt.

Om du far valja, fantisera fritt, vad vill du ha i framtiden fran dinaIDSsystem? Vad ar onskeegenskapen du saknar idag?

Sebastian: –Vad ar det manuella trakiga arbetet som du helst skulle vilja slippa?

–Jag har ju inte gjort det manuella trakiga arbetet. Det skulle man kunna fragade pa Ccode som sitter med heltidsovervakning till exempel. Det hinner ju blitrakigt. Jag har bara gatt in och tittat ibland. Vi har inte haft nagot uppdragegentligen dar vi haft ett kontinuerligt jobb med att ga in och titta pa det. Mendet ar kanske det jag skulle vilja jobba med. Att man kan lagga en halvdag iveckan pa analys och forbattringar atminstone.

–Man kan ju ha samma information pa flera granssnitt, jag skulle kunna tankamig att kombinera det som OSSIM gor med SGUIL. Man skulle vilja ha ganskamanga funktioner att valja mellan som att man kan gora en googleliknandesokning i loggarna och att man kan gora sql-fragor mot vilken information somhelst i larmen.

Eva Lina: –Vad vill du helst ha, full kontroll eller skulle du kunna tanka dig ettsystem som automagiskt lar sig hur den ska klassificera trafik? Eller kanns detlite laskigt?

109


Sebastian: –Eller som lar sig gora gissninar, gora kvalificerade gissningar vadanvandaren ar ute efter?

–I sa fall tror jag mer pa iden att den fragar mig “jag tror det har” och sa far dusaga ja eller nej. Interaktiv inlarning. Jag har last en artificiell neuronnatkurs ochdom lar sig valdigt konstiga saker. Jag skulle nog inte lite pa det. Sjalvlarandear inte redo for nagon anvandning just nu i alla fall. Men om man kan vara medoch bestamma och forstar varfor den klassar saker a ett visst satt.

Eva Lina: –Da kanske du gillar Magnus ide med att den tittar pa var du klickaroch fragar lite fragor nar den ar osaker.

–Pa det temat sa har ju Stefan Axelsson gjort en artikel, han var i var fors-kargrupp tidigare. Han har gjort just det harmed interaktiv inlarning, att denfragar anvandaren och klassificerar saker. Det kanske kan vara lite intressant fordet har. Som related work tycker jag nog att det kan vara bra.

Sebastian: –Sa systemet ska helst INTE ta nagra egna initiativ utan att fragaforst?

–NA!(skratt) Framfor allt sa vill jag ha kontrollen, jag vill garna logga mycketsaker sa att det finns dar. Men sedan maste man plocka ut vad det ar manska titta pa i overvakningssyfte i realtid. Sedan vill jag kunna ga tillbaka in idatabasen, sa att man prioriterar upp vissa handelser som man faktiskt tittarpa. Man vill ju anda ha den stora hogen med saker, att man gar in dar ibland,kollar vilka kategorier av larm man fatt.

Sebastian: –OSSIM hade det har med relevans. Om man skulle ta och lata denpaverka relevansen, nar den forsoker lara sig vilka saker som ...

–Jo den kan fa lara sig vad som ska poppa upp i larmfliken kanske. Men jag villanda ha den stora massan med allting sa att jag kan ga in och soka i det sjalv.Jag vill ha den kontrollen att allt finns kvar i originalformat och jag kan ga inoch soka i det. Men sedan maste man ha nagot granssnitt for realtidsovervakningfor att pa nagot satt konfigurera det sa att man hinner titta igenom det.

Sebastian: –Det skulle ju kunna vara en grej, att den i overblicken lar sig vadman tycker ar relevant.

–Statistik behover man ocksa. Man behover kunna fa statistik pa IP-adresser,vilka IP-adresser som har genererat mer allvarliga handelser, vilka som genereratmest handelser och vilka som genererat fa men allvarliga till exempel. Sadanasaker behover man kunna fa ut. Man vill kunan gora statistiksokningar i datafor att veta vad manska titta pa. I OSSIM kan du fa upp grafer. Typiska ar ju“top ten attackers”. Men det ar bara pa antal larm. Sedan kanske du skulle viljaha nagra smartare satt, hur allvarliga de ar ocksa.

Eva Lina: –Men om du fick valja mellan de fyra satten, att systemet bara tittarpa vilka alarm du klickar pa och antar att du ar mest intresserad av dem duklickar pa eller foredrar du att man klickar men valjer sjalv nar man sager atsystemet varfor nagot ar viktigt?

–Jag skulle nog vilja ha att nar man klickar upp ett larm, att man kan klicka inagra radio buttons, att man har kanske fyra nivaer eller nagonting, “hur viktigtvar det har?”. Och sa vill man kanske ha den har kategoreseringen pa nagot satt.

110


Kanske med fordefinierade kategorier och kanske kunna lagga till egna. Sa attman enkelt sjalv kunde satta informationen.

Eva Lina: –Det har med att systemet forsoker gissa sig till och istallet for att dusager till den uttryckligen, gissar den och ger anledningen, och sa kan du ange“ja jag haller med” eller “jag haller inte med”?

–Det ar nog bra, sa lange det inte ar dumma fragor(skratt). Man kan ju blivaldigt irriterad.

Sebastian: –Da blir det som gemet i Word. “Du forsoker skriva ett brev!” ’.......nae .......’ “Det ser ut som om du forsoker skriva ett brev. Vill du ha hjalp!”’NEJ!’

–Man maste nog kunna stanga av det dar.

Tycker du det ar viktigt att kunan ge negativ feedback till systemet?Att ett alarm ar viktigt just nu, men sedan kommer det inte att varaviktigt efter att jag har patchat eller bytt ut den har servern. Attman kan sla ner det?

–I ArcSight, RSAenvision och aven OSSIM sa kan du ha en sadan har inventorylist. Den vet vilka IP-adresser som ar servrar som faktiskt finns och den vetvilka operativsystem de kor och vilka tjanster som kor pa maskinerna, vilkaprogramvaror som ar igang pa dem. Men det ar ocksa en massa konfigurationoch saker byts hit och dit, sa helst skulle det uppdatera sig sjalvt, helst skulledet ha nagon koppling till nagot configuration management pa maskinerna saatt man autmatiskt fick in det. Men det ar kackligt att fa det fixat.

–Men jag kan tanka mig att man vill ha en agent ute pa maskinerna som faktisktvet vilka programvaror som kors pa dem, det finns ett antal sadana. Det goratt du kan gora en korrekt korrelering mot snortlarmen och prioritera upp demom det ar mot en sarbar version pa maskinen. Det tycker jag definitvt, just foratt sortera larmen. Sedan kanske man inte slanger bort de andra larmen heller,bara att de inte ar hogt prioriterade.

Kan du komma pa nagra katastrofala nackdelar med rankning?

–Det ar risken att den tar vissa saker och placerar hogt och det andra hamnardar nere och sa tittar man aldrig pa det. Om inte rankningen ar korrekt ar detganska stor risk att man missar nagot. Da upptacker man bara det om nagothander for da gar du kanske in och tittar pa alla larm som ror den maskinenoch forst da hittar du det.

Eva Lina: –Fast det ar klart, den risken finns ju med felskrivna regler ocksa.

–Visst.

Eva Lina: –Undrar vad som ar lattast att hitta, en felskriven regel eller felaktigrankning?

–Testning ar viktigt nar det galler regler. Men det galler kanske aven rankning-en, att man behover en testmetod for att kontrollera vad som hander. Over

111

D.4. ULF LARSON

huvudtaget saknar jag att det inte finns nagon test bench dar man kan spelaupp trafik och fa ett facit pa hur det borde se ut, rankningsmassigt och vilkalarm man over huvudtaget far av IDSen.

–Det ar ganska latt egentligen det jag gor nu. Jag har fatt nagra TCP-dumparnar jag gatt kurserna som man kan spela upp och det skulle vara jattelatt attgora testning med dem.

–Men sedan ar det ju alltid sa att om det finns test bench-data sa kommer deanpassa sig sa att de far bra resultat pa det. Men det ar ratt svart att enbartinstallera de har verktygen och veta att allt fungerar som det ska. Bara det attse att allt ar uppe och snurrar ar steg ett. Men att kunna jamfora olika verktygoch se vilket som ar battre an nagot annat blir ju svart nar det ar kanda data.Man maste ju ta fram nya data till varje test. Men bara det att kolla att allt aruppe och snurar och beter sig som det ska vore jattebra.

D.4 Ulf Larson

Interview with Ulf Larson from Omegapoint via e-mail in Swedish.

Vilka IDS system har du anvant?

• Tillverkare?

• I hur stor utstrackning kanner du att man kan aggregera alarmeni de system du anvant?

• I hur stor utstrackning kanner du att du har kunnat paverka hursystemen aggregerar alarmen?

• I hur stor utstrackning kanner du att du har kunna arbeta medalarmen pa en hog niva / korrelera dem?

• Fordelar/nackdelar med det systemet?

Jag har jobbat en del med Snort. Sedan har jag tittat en del pa RSA envi-sion, som snarare ar ett SIEM (Security Information and Event ManagementSystem), vilket ar ett steroidstint logg- och analysmonster med en IDS-modul.Jag har ocksa anvant pentestningsverktyg som burp-scanner (webbappar) ochnessus (server/infra), vilken innehaller beslutsmoduler (om an lattviktiga) foratt avgora om en mojlig sarbarhet ar en mojlig sarbarhet.

I enVision har man mycket stor mojlighet att aggregera alarm. Mycket godamojligheter att valja bort och filtrera analysdata och larm, samt grafik, ex-empelvis cirkeldiagram, som visar hur stor del av angrepp som kommer fran enviss IP-adress. Burp ar ett mer lattviktigt verktyg och dar har man nagot samremojligheter att paverka. Dock grupperar scannern sjalv resultat av ”samma” typoch utnyttjar en tradstruktur som foljer strukturen pa en skannad webbsite.

Mycket av iden med de stora SIEM-verktygen, som enVision ar att det skallvara enkelt for en operator att sattas framfor verktyget. Om man bygger inmycket vettighet i verktyget skall operatoren inte behova vara expert for att

112


tolka vad som star (det hjalper, givetvis, men det ar inte nodvandigt). Burp arpa lagre niva och erbjuder inte korreleringsmojligheter pa det sattet, eftersomden bara jobbar med en typ av ”alarm”, eller vad den tror ar sarbarheter. Villman korrelera far man utnytta nagot annat verktyg parallellt. EnVision ar ocksaett Enterprize- system, vilket innebar att den i mycket stor utstrackning kankorrelera information fran andra servrar som star ”nan helt annanstans”, ochsom ”gor nat helt annat”. Dessutom (och detta gillar jag MYCKET), kan detkorrelera in information fran sarbarhetsdatabaser, dvs man kan utesluta vissaalarm, da den formodade sarbarheten inte finns. Dessutom (igen) kan man tain information fran sakerhetsskanningar av ex. . . Nessus dara.

For/nackdelar. . . I regel verkar det vara sa att man far det man betalar for, mensamtidigt sa far man ju ocksa vad man fortjanar. . . Mindre verktyg ar lattareatt komma overens med, men till en lagre finkornighet pa installningsfronten.Storre verktyg tar langre tid att lara sig, men kan erbjuda nanokorn(ighet).

Har ni stora problem med “false positives”?

• Hur hanterar du dom isf?

• Hjalper IDS systemet till med det pa nagot satt?

– Hur?

False positives i min nuvarande varld ar en sarbarhetsskanner som sager, woo-hoo, en XSS i rapportgeneratorn. Sen ar det upp till mig att avgora om det ar enfp eller inte. Med aktiv skanning kan verktyget sjalv forsoka verifiera om det aren sarbarhet eller inte, exempelvis med strangen “45987276Isthisreflected234-”,vilket skall forestalla en tillrackligt unik strang som blir reflekterad till motta-garen. (Oj, svarade pa nasta fraga av bara farten).

Man skall givetvis skilja pa ett kritiskt larm, kritisk vard. Jag skulle pastaatt detta ar det latt viktigaste overhuvudtaget (se vidare diskussion angaendeslipsnissar nedan). Det basta sattet att gora detta ar att lata systemet ta emotfeedback fran anvandaren. Systemet kan sjalv avgora om det ar en attack ellerinte (→ alarm), pa en teknisk niva (den kan t.ex se att ett natverkspaket in-nehaller “/bin/sh”). Daremot hur stor paverkan paketet har pa just det egnasystemet skiljer sig mycket och det ar ytterst upp till operatoren att tagga upplarmet i det specifika kontext han/hon sitter. (I extrem forlangningen bordeman tjota ned affarsansvarig som kan tala om att “om attack X gar genom, dagar var business at helvete, direkt”). Det kan vara bra input ocksa, men forsokfa slipsnissarna till det, far du se hur det gar.

I mitt jobb maste jag klicka pa intressanta larm for att gora manuell uppfoljning.

Vilka forbattringar vill du se for att gora det enklare att upptackaattacker?

Grafik ar alltid bra, och givetvis korrelering och aggregering. Feedbackbaseradesystem som tar data fran den miljo de star i. Sen bor man fundera over attinte bara titta pa inkommande trafik utan aven i viss utstrackning pa utgaendetrafik. Om en stor ISP far manga datorer infekterade av en trojan och darefter

113

D.4. ULF LARSON

blir noder i ett botnat maste ISPn kunna marka detta. Dels pa ingaende narkommandocentralen talar om for noderna att borja mangla, och dels pa utgaendenar noderna borjar krakas ut uppkopplingsforsok mot externa malet.

Skulle du foredra ett system som

1. “lar” sig automagiskt hur du vill ha det

2. ett dar man maste konfigurera allt (har full kontroll)

3. eller kanske nagot dar i mellan?

Ett system maste givetvis lara sig under tiden. Inte nodvandigtvis bara av digsom operator, utan aven av hur natverkstrafik beter sig, hur varden beter sig,hur din webbapplikation beter sig. Normalbeteenden ar alltid battre an signa-turer. Skall man bli filosofisk kan man givetvis mena att det gar att konfigureraupp “allt” sa att systemet tar full hansyn till allt dynamiskt beteende. Talarman endast om interaktion med operatoren beror graden av interaktion pa hurbra operatoren ar. En san luffare som jag behover formodligen ett ganska sta-tiskt system eftersom jag inte kan avgora om larm X ar en 1a eller en 10a paallvarlighetsskalan, medan en analysguru som Ola formodligen hade fatt mycketmer ut av systemet om han hade kunnat ge expertfeedback.

Vilket omrade utav de fyra vi tar upp, tror du har storst framtidspo-tential? Ar det enbart ett utav dem eller ar det en kombination?

Jag satter min peng pa feedback, dvs alternativ 3. Jag haller med er om funde-ringarna kring alternativ 2.

Vad tror du om att systemet lar sig anvandarens preferenser genomatt t.ex. fora statistik pa vilka typer av alarm som han/hon klickar pafor att sedan kunna gora antaganden om hur nya alarm skall rankas?

Klickning ar bra.

Samma som ovan men istallet med en tumme upp och tumme nerknapp for alarm man tycker ar viktiga/oviktiga?

Viktning ar bra. Man bor formodligen kombinera viktning med klickning. Fraganar ju om man skall vikta binart, dvs bu (tumme upp) eller ba (tumme ned),eller om man skall ange en siffra mellan 1 och 10, eller snarare -5 till 5.

Ar det viktigt att kunna ge “negativ” feedback? Att kunna ranka ettalarm lagre och ange anledning(ar) for detta till systemet?

Det ar viktigt att vikta negativt. Man far da en grupp larm som man normaltsett kan strunta i (av olika anledningar). Det kanske till och med kan vara saatt man kan vikta ett larm som irrelevant (motsvarar att man plockar bortsignaturen fran konfig-filen. . . typ).

114


Hur skulle man kunna tala om for systemet att fran och med nu skaden inte rakna med de klick som gjorts pa ett alarm? T.ex. innanen patch sa var alarmen som hangde ihop med det problemet viktigaoch man klickade pa vart enda ett. Efter patchen daremot, inte avintresse och bor inte langre tas i berakning nar man rankar alarmen.

Man utnyttjar viktningsparametern, vilket innebar att ett “irrelevant” larmkommer att vara irrelevant oavsett om det tidigare har varit en 10a pa allvar-lighetsskalan och blivit klickad sa hoger musknapp totalhavererat.

Skulle det vara battre/samre om systemet istallet stallde specifikafragor om hur den skall ranka alarm?

• I sa fall, vilka parametrar ska systemet ta hansyn till?

Vet ej. Hur kan ett alarm bara vara intressant forsta gangen (om det inte ar ettspecialfall av ovan fraga)?

Vad ar din omedelbara tanke (bra/dalig) om de fyra satt att rankaalarm som vi tar upp? (clicking, directed learning, feedback, explicitquestions)

Det ar battre, men for att vara generell kan systemet formodligen bara stallafragor utifran rent teknisk paverkan (affarsvarden ar specifika for specifik miljo,ju). En sarbarhetsscanner talar om for testaren att det har angreppet som denprecis identifierat, fungerar pa “det har sattet” och det ar allvarligt “darfor”.Parametrar som man bor ta hansyn till ar hur stor potentiell paverkan en attackkan fa. En attack som kan ge administratorsrattigheter eller lista tabeller i endatabas bor viktas hogt.

Klickning ar jag lite kluven till. Man maste som analytiker forsta varfor manklickar innan man klickar, dock. Detta far man ingen hjalp av fran systemet savitt jag forstar. Inget for nyborjare da systemet formodligen kan bli fellart omman inte riktigt vet vad man klickar pa, eller varfor.

Directed learning ar ocksa svart for en nyborjare. Har maste nyborjaren sjalvtala om varfor han/hon klickar alarmet. Mojligen kan en expert (som Magnus)ge understod genom att forst klicka sig genom en mangd testdata och sedan narsystemet ar tranat (och det forstar varfor), kan man utnyttja “feedback” for atthjalpa mer juniora analytiker (som jag. . . ) att fortsatta jobba med systemet.

Feedback gillar jag. Sakert olika. For nyborjare tror jag att feedback ar denstarkaste kandidaten eftersom den ar den enda som hjalper analytikern att fattavettiga beslut. Detta ar superviktigt da de flesta inte har tid att bli experter(ett uttalande som far sta for mig, men ponera att det finns en sysadmin somskall rodda brandvaggar, AD, IDSer, patchhantering, och som dessutom har 30servrar som gar upp och ned som en. . . karusell som gar upp och ned).

Explicit questions tror jag blir lite samma problematik som directed learning.

115

D.4. ULF LARSON

Ser du nagra nackdelar med rankning?

De tva nackdelarna jag ser ar att systemet kan bli felinlart, samt att om maninte tar hojd for trender, dvs har en korttids- och en langtidsstatistik, kan manmissa viktiga trender.

1. Ett typiskt alarm har information om tid, alarmtyp, kalla/destination,etc.

• Racker denna information for dig eller behover du mer?

– Vad anvander du i sa fall for verktyg for att hamta merinformation? T.ex. tcpdump, full packet capture, fangarall utgaende trafik, minnesdumpning? Sparar du paketgenom natverket?

– Vad ar det for information i sa fall du soker efter?

• Hur ofta granskar ni alarmen narmre?

– Vad ar det for information som spelar roll for dig da?

– Har du tillgang till verktyg som automatiskt hjalper digmed alarm sa att du slipper gora vissa moment sjalv?

• Om du anvander olika verktyg for att komplettera informa-tionen i alarmet, finns det nagon samverkan mellan de olikaverktygen och mellan verktygen och IDSen?

– Om verktygen samverkar, finns den funktionaliteten in-byggd eller ar detta nagot du fatt utveckla/gora sjalv?

2. Hur skulle du beskriva ett typiskt tillvagagangssatt for dig nardu ser ett alarm?

• Vad far du for tankar nar du ser ett alarm?

3. Vad ar den vanliga arbetsgangen? T.ex. jag ser alarm X. Letarefter handelse Y. Om X & Y finns, starta verktyget Z (kanskelater tcpdump logga allt mellan source IP och dest IP?) for attsamla mer information for att bekrafta att alarmet stammer.

4. Hur kategoriserar du alarm?

• “lag-niva alarm”

• “ointressanta”

• “dubblett”, (orsak redan kand)

• “grad av tillforlitlighet”, t.ex. en regel som triggar alarm kanvara valdigt allman och orsaka manga alarm varav de flestainte beror pa intrang och da ha en lag grad av tillforlitligheteller en regel kan vara valdigt specifik och darfor ar alarmsom genereras utifran den regeln av hog tillforlitlighetsgrad.

• “bekraftelse”, kan man hitta ett nytt event som bekraftaratt alarmet ar korrekt?

116


5. Bygger du sjalv kopplingar mellan olika alarm eller gor systemetdet at dig?

6. Hur verifierar du alarm?

7. Ar de grafiska interfacen pa de IDSer du anvant ok?

• Finns all funktionalitet du vill ha dar?

• Saknar du nagon funktionalitet i dem?

8. Hur gar din interaktion till med interfacet?

• Klickar du pa alarm?

• Tittar du enbart utan att klicka?

• Brukar du soka mer information an vad interfacet presente-rar?

Ser ut som om fragan ar hart kopplad till natverksbaserade IDSer. Sannoliktbehover man information om vilken enhet som genererar larmet (om man haren korreleringsmotor och flera insamlande enheter, och vill ha mojlighet att foljafloden genom natet). Sedan ar det val ocksa viktigt att se vilken process somtar emot paketet. For att kunna avgora vad som finns i paketet behover mansannolikt ocksa lite applikationslagerinfo.

Wireshark ar alltid vettigt att plocka fram, i och med att den har sa pass mycketmapp till olika protokoll och liknande.

Foljande fragor besvaras utifran mitt perspektiv som sarbarhetstestare och bortas med en nypa salt. Dock finns likheter.

Tittar man pa min situation med sarbarhetsskanningar tittar jag noga pa varjelarm. Nu ar ju detta inget kontinuerligt jobb utan ett jobb som utfors under enbegransad tid, vilket ger fog for att titta pa allt noga.

Om jag hade anvant olika verktyg hade jag satsat min sista surt forvarvade slantpa ett verktyg som enVision som har mojlighet att plocka in data fran nara 300olika enheter. Sedan hade jag spikat en datainsamlingspolicy (med avseende paatt folja compliance-regler och samtidigt ha mojligheter att upptacka angrepp).Slutligen hade jag kopplat i de olika enheterna och sett till att de samlar in dendatan som policyn foreskriver.

Nar jag ser ett larm tanker jag: Gott! Verktyget hittade en sarbarhet, jagforsvarar min orimlig hoga timpeng. ;-)

Kategorisering – lag, medel, hog. Det ar svart att ge rad angaende huruvida ensarbarhet skall atgardas eller inte. Jag anvander ocksa graderingsmodeller somCVSS v2.

Verifiering av alarm innebar oftast i mitt fall att man forsoker bygga en Poc, dvsbygga en metodik for hur sarbarheten kan utnyttjas, och sedan visa att med me-todikens hjalp kan man palitligt aterskapa attacken som utnyttjar sarbarheten.

117

Documents

Alarm management for intrusion detection systems - · PDF fileAbstract Intrusion detection systems (IDSs) are important tools helping the network and system administrators to detect