INTRUSION DETECTION/PREVENTION

SYSTEM

Project by

Name: V.V.N.Priyanka

Application number: ENGS12443

Guide’s name: Dr. B.M.Mehtre

Guide’s institution: Institute for Development and Research in Banking

Technology (IDRBT)

TABLE OF CONTENTS

1. Introduction

1.1. What is Intrusion Detection System?

1.2. Importance of Intrusion Detection System

1.3. Objective

1.4. Tools used

1.5. Organization of the report

2. Background

2.1. USMA Datasets

2.2. Snort

2.3. Waikato Environment for Knowledge Analysis(WEKA)

2.4. Previous works with USMA Datasets

2.5. Overview of the project

3. Snort as IDS

3.1. Approach to the problem

3.2. Working with Snort

3.3. Working with TCPReplay

3.4. Colasoft Packet Player

3.5. Working with WEKA and JPCAP

3.6. Results

4. Summary

5. Conclusion

5.1. What was accomplished

5.2. Future work

6. References

1. INTRODUCTION

1.1. What is Intrusion Detection System?

Deliberately accessing someone's system without their knowledge is called intrusion and any

attempt to destroy, expose, alter, disable, steal or gain unauthorized access to or make

unauthorized use of an asset in someone's system is called an attack. An intrusion detection

system (IDS) [1] is a device or software application that monitors network or system

activities for malicious activities or policy violations and produces reports to a management

station. The main role of IDS is to detect the intrusions and to prevent the attacks. The key

feature of Intrusion Detection System is their ability to provide a view of unusual activity and

issue alerts notifying administrators and/or block a suspected connection.

Intrusion detection functions include:

Monitoring and analyzing both user and system activities.

Analyzing system configurations and vulnerabilities.

Assessing system and file integrity.

Ability to recognize patterns typical of attacks.

Analysis of abnormal activity patterns.

Tracking user policy violations.

Intrusion detections are classified into three categories:

Signature based detection systems: Very effective against known attacks, and it

depends on the receiving of regular updates of patterns and will be unable to detect

unknown previous threats or new releases.

Anomaly based detection system: Depends on the classification of the network to the

normal and anomalous, as this classification is based on rules or heuristics rather than

patterns or signatures and the implementation of this system we first need to know the

normal behavior of the network.

Specification based detection system: Responsible for monitoring the processes and

matching the actual data with the program and in case of any abnormal behavior will be

issued an alert and must be maintained and updated whenever a change was made on the

surveillance programs in order to be able to detect the previous attacks the unknown.

Three types of Intrusion Detection Systems can be built based on the platforms. They are:

Host based IDS: This type is placed on one device such as server or workstation, where

the data is analyzed locally to the machine and are collecting this data from different

sources. HIDS can use both anomaly and misuse detection system.

Network based IDS: NIDS are deployed on strategic point in network infrastructure.

The NIDS can capture and analyze data to detect known attacks by comparing patterns or

signatures of the database or detection of illegal activities by scanning traffic for

anomalous activity. NIDS are also referred as “packet-sniffers", because it captures the

packets passing through the of communication mediums.

Hybrid based IDS: The management and alerting from both network and host based

intrusion detection devices, and provide the logical complement to NID and HID - central

intrusion detection management.

1.2. Importance of Intrusion Detection System

The fact that we cannot always protect that data integrity from outside intruders in today's

internet environment using mechanisms such as ordinary password and file security, which

leads to a range of issues. Adequate system security is of course the first step in ensuring data

protection. Intrusion detection takes that one step. Placed between the firewall and the system

being secured, a network based intrusion detection system can provide an extra layer of

protection to that system. The Intrusion Prevention System will prevent the attacks disturbing

our system.

Figure 1.1 Block diagram of Intrusion Detection System

1.3.Objective

The primary objective of this project is to study tools like Snort, TCPReplay and WEKA.

The secondary objective is to configure Snort as Intrusion Detection system and add any

possible rule to it to make its functionality even better.

1.4. Tools used

VirtualBox (which includes two Ubuntu virtual machines and one Windows XP

machine)

Snort

TCPReplay

Colasoft Packet Player

Wireshark

Waikato Environment for Knowledge Analysis (WEKA)

1.5. Organization of the report

The document is organized in the following manner. In section 1, the basics of Intrusion

Detection System, its importance and the objective of this document are discussed. In section

2, a detailed discussion about the US Army datasets is discussed along with the research

which was previously done on them. Also, the basic idea of how I am making use of these

datasets to develop IDS is discussed. In addition to these, description about Snort, WEKA,

TCPReplay is also given. In section 3, major concepts like configuring Snort as IDS,

transferring network packets from one virtual machine to another and capturing them with

the help of Snort will be discussed. Later, the results that came will be briefed. In section 4,

summary of the document is discussed. Finally, in section 5, the future enhancements that

can be made to this project are discussed.

2. BACKGROUND

2.1. United States Military Academy(USMA) Datasets

Cyber Defense Exercise (CDX) provided a real world educational exercise that challenged

their university students to build secure networks and defend those networks against

adversarial attacks. They provided about 13 GB PCAP files, each PCAP file containing a

huge number of network packets that were captured from many networks. These files contain

all types of network packets. In my project, I am making use of these PCAP files as the main

data. Figure 2.1 gives an overview of all the IP addresses that were found in the PCAP files

and linked them with their hosts so that it will be convenient for the users of the datasets.

Figure 2.1 IP addresses found in the PCAP files with the IP addresses to hosts on the internal

USMA network

2.2. Snort

Snort is an open source network-based intrusion detection system (NIDS). It has the ability to

perform real-time traffic analysis and packet logging on Internet Protocol (IP) networks. Snort

performs protocol analysis, content searching, and content matching. These basic services have

many purposes including application-aware triggered quality of service, to de-prioritize bulk

traffic when latency-sensitive applications are in use. This program is used on smaller networks

but on larger ones, with Gigabit Ethernet, it may become unreliable. Snort doesn't require that

you recompile your kernel or add any software or hardware to your existing distribution but it

does require that you have root privileges. Snort is an open source Signature based IDS. There

are predefined rules for finding attacks. Snort rules are provided by Sourcefire and can be

downloaded by registered users for their use. To include rules one has to change the

configuration file. When a packet comes, all the rules are checked and if any rule is applicable to

the packet, action specified in the rule will be performed.

The actions can be of four types.

It can be to alert the nodes

Log the incoming matched packet.

Pass the packet without any action.

Drop the packet.

Snort works in three different modes.

Sniffer mode

Packet logger mode

Network intrusion detection mode.

In Sniffer mode snort detects the incoming packets and displays them in console. In packet

logger mode, Snort collects the packets and logs them to disk. In network intrusion detection

system (NIDS) mode alerts will be generated. Alerts can be generated in different ways. It can be

logged or can be displayed to console. Alerts can also be generated in such a way that it will

display only useful information or will display the entire header information.

2.3. WEKA

Waikato Environment for Knowledge Analysis (WEKA) is a workbench that contains a

collection of visualization tools and algorithms for data analysis and predictive modeling,

together with graphical user interfaces for easy access to these functions. WEKA supports

several standard data mining tasks, more specifically, data preprocessing, clustering,

classification, regression, visualization, and feature selection. All of WEKA's techniques are

predicated on the assumption that the data is available as one flat file or relation, where each data

point is described by a fixed number of attributes (normally, numeric or nominal attributes, but

some other attribute types are also supported). The algorithms can either be applied directly to a

dataset or called from your own Java code. In my project, I am using this WEKA tool to classify

the USMA datasets and to analyze any new kind of attack.

2.4. Previous works with USMA datasets

The USMA captured 13GB of datasets which contained many PCAP files. They configured

Snort as Intrusion Detection System and tested that for this data. They logged the alerts which

arose. These alerts were about 10.8 MB which are available in their official website (Cyber

Research Center) along with the datasets.

2.5. Overview of the project

The main idea of this project is to configure Snort as Intrusion Detection System. Test the data

(from USMA Datasets) and generate the alerts. Then analyze the same data using WEKA tool,

classify the data and if any new attack is detected, then the rule for that new attack will be added

to the predefined rule set of Snort. Thus, a new rule can be added to Snort making it even more

efficient. The rest of the report will be about how to configure Snort and make it as IDS, how can

the PCAP files be converted to .csv format and be classified in WEKA tool. Also I will be

discussing about the TCPReplay and Wireshark.

3. SNORT AS IDS

3.1. Approach to the problem

As we are dealing with network packets, to make the scenario look like real time transfer of

network packets, I am using three virtual machines, two Ubuntu machines and one Windows XP.

As the main idea is to make IDS and add a rule to it, the next step to that problem is to configure

Snort as IDS in one of the Ubuntu machines. The use of the remaining virtual machines will be

discussed shortly. The next phase of the project deals with WEKA tool and classification of data

using that tool.

3.2. Working with Snort

In order to use Snort as Intrusion Detection System, firstly we will have to download the Snort

from its official website www.snort.org. Then we must configure Snort by following the

following steps.

Install a virtual machine Ubuntu using VirtualBox.

In this Ubuntu machine, we will have to install and configure Snort.

Open up a terminal by hitting the uppermost icon on the left corner to search for the

terminal application.

Once terminal has been opened, type in the following command (all in one line):“sudo

apt­get install flex bison build­essential checkinstall libpcap­dev libnet1­dev libpcre3­dev

libmysqlclient15­dev libnetfilter­queue­deviptables­dev”

It will ask you for the password. Enter in your login password to the VM.

The selected applications are now being installed. You may occasionally get a prompt

asking to continue. Type “y”’ and continue.

Build and install libdnet from its source code.

1. Type “wget https://libdnet.googlecode.com/files/libdnet­1.12.tgz”. Hit enter.

2. If you type in “ls”, you will see that the file has been downloaded to your home

directory. Issue the following command: tar xvfvz libdnet­1.12.tgz. Hit Enter.

3. This unpacks all the files that were in the libdnet­1­12.tgz file and creates a

libdnet­1­12 directory. Change into the libdnet­1­12 directory.

4. Type: ./configure "CFLAGS=­fPIC". Hit enter. The "­fPIC" C flag is necessary if you

compile it on 64­bit platform.

5. Type “make”. Hit enter.

6. Type “sudo checkinstall”. The checkinstall command above will build .deb package.

And will ask you several questions. Accept default values.

7. Install the .deb package, and create a symbolic link where Snort looks for libdnet.

Type in the following commands: “sudo dpkg ­I libdnet_1.12­1_amd64.deb” and

“sudo ln ­s /usr/local/lib/libdnet.1.0.1/usr/lib/libdnet.1”.

Download, build and Install DAQ (Data Acquisition Library).

1. DAQ can be downloaded from http://www.snort.org/snort­downloads. The

current version is daq­2.0.6. Usually, the downloads are placed in the Downloads

directory of your Ubuntu OS.

2. We are going to repeat the steps we did for the libdnet install ­ unpack the files,

configure, make, and then install.

3. The “sudo checkinstall” command will go through the following steps like it did

from the libdnet procedure. The figures below shows the initial “sudo checkinstall

command” and then the end result.

4. Install the package by running: “sudo dpkg ­i daq_2.0.6­1_amd64.deb”

Download, build and Install Snort.

1. Much like DAQ, Snort can be downloaded :

http://www.snort.org/snort­downloads. The current version is Snort­2.9.8.0.

Again the downloaded file resides in the Downloads directory of your Ubuntu

OS.

2. We are going to repeat the steps we did for the libdnet and daq install -unpack the

files, configure, make, and then install.

3. The “sudo checkinstall” command will go through the following steps like it did

from the libdnet and daq procedures. The figures below shows the initial “sudo

checkinstall command” and then the end result.

4. Install the package by running: “sudo dpkg ­i snort_2.9.8.0­1_amd64.deb”

5. Create a symbolic link for snort by running: “sudo ln ­s

/usr/local/bin/snort/usr/sbin/snort”.

6. Run the ldconfig command, so that dynamic linker run­time bindings for libdnet

and DAQ libraries are properly set up.

7. 8. Verify that snort is installed properly by running “snort ­V”. You will get

something like this then:

Figure 3.1 Output that comes when Snort is successfully configured

By following the above steps, snort will be successfully installed and configures as Intrusion

Detection System.

In order to test the Snort, I added one rule to give an alert when Facebook was accessed. The rule

was:

alert tcp any any -> any any (content:"facebook"; msg="Someone is accessing facebook!!";

sid:1000001;)

So when Facebook was accessed, Snort did generate an alert message as shown in Figure 3.2.

Figure 3.2 Snort giving an alert message when Facebook was accessed

3.3. Working with TCPReplay and WireShark

After Snort was successfully configured as IDS, we can now make use of it by sending the data

i.e., the PCAP files. In order to send those files, I made use of another virtual machine (Ubuntu).

In that machine I installed TCPReplay. Tcpreplay is a suite of GPLv3 licensed tools written by

Aaron Turner for UNIX (and Win32 under Cygwin) operating systems which gives you the

ability to use previously captured traffic in libpcap format to test a variety of network devices. It

allows you to classify traffic as client or server, rewrite Layer 2, 3 and 4 headers and finally

replay the traffic back onto the network and through other devices such as switches, routers,

firewalls, NIDS and IPS's.

The main idea behind using TCPReplay is to transmit the pcap files from one machine to another

machine, so that Snort will detect these as the real time network traffic and generate alerts.

In order to test the working of this TCPReplay, I used one Ubuntu machine in which TCPReplay

was installed and another Windows XP machine in which WireShark was installed. I transmitted

the network packets from Ubuntu machine and captured them using Wireshark in Windows XP

machine. The network diagram for that is represented in Figure 3.3.

Figure 3.3 Network configuration for Windows XP and Ubuntu machines

When the network between Ubuntu machine where TCPReplay was installed and the Windows

XP machine is done in the above fashion, then communication will be established between those

machines. And in that stage we can transmit PCAP files from Ubuntu machine using TCPReplay

as shown in Figure 3.4 and capture them using Wireshark in Windows XP machine as shown in

Figure 3.5.

Figure 3.4 Transmitting packets through TCPReplay from Ubuntu machine

Figure 3.5 Wireshark capturing the transmitted packets in Windows XP machine

The above procedure was implemented to test whether the packets are transmitted from one

machine to another. Now, I tried to implement the same procedure but with Snort instead of

Wireshark. So the network diagram for that configuration is shown in Figure 3.6.

Figure 3.6 Network configurations between two Ubuntu machines

By implementing the above configurations, we can transmit the PCAP files from one Ubuntu

machine and then capture them through Snort and generate the alerts.

But, here I was unable to capture the packets in Snort as there were variations in the IP addresses

of the PCAP files and our machine. So I read the PCAP files directly through Snort and the

output for that was as follows:

WARNING: No preprocessors configured for policy 0.

04/20-18:33:23.155634 10.1.60.187 -> 10.1.60.203

ICMP TTL:255 TOS:0x0 ID:14442 IpLen:20 DgmLen:84

Type:0 Code:0 ID:28844 Seq:4 ECHO REPLY

=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+

==========================================================================

Run time for packet processing was 9.991526 seconds

Snort processed 17625 packets.

Snort ran for 0 days 0 hours 0 minutes 9 seconds

Pkts/sec: 1958

==========================================================================

Memory usage summary:

Total non-mmapped bytes (arena): 782336

Bytes in mapped regions (hblkhd): 21590016

Total allocated space (uordblks): 670080

Total free space (fordblks): 112256

Topmost releasable block (keepcost): 42048

==========================================================================

=====

Packet I/O Totals:

Received: 17625

Analyzed: 17625 (100.000%)

Dropped: 0 ( 0.000%)

Filtered: 0 ( 0.000%)

Outstanding: 0 ( 0.000%)

Injected: 0

==========================================================================

Breakdown by protocol (includes rebuilt packets):

Eth: 17625 (100.000%)

VLAN: 0 ( 0.000%)

IP4: 17449 ( 99.001%)

Frag: 0 ( 0.000%)

ICMP: 7613 ( 43.194%)

UDP: 931 ( 5.282%)

TCP: 8905 ( 50.525%)

IP6: 0 ( 0.000%)

IP6 Ext: 0 ( 0.000%)

IP6 Opts: 0 ( 0.000%)

Frag6: 0 ( 0.000%)

ICMP6: 0 ( 0.000%)

UDP6: 0 ( 0.000%)

TCP6: 0 ( 0.000%)

The above details are the part of output that occurred when packets were captured by Snort.

Snort gives a list showing the number of alerts generated, packets processed and detected,

dropped etc. Also it specifies the memory used by the packets. But in my case, this case also did

not generate any alert messages. So, I transmitted packets through Colasoft Packet Player instead

of TCPReplay.

3.4. Colasoft Packet Player

Colasoft Packet Player is a replay tool which allows you to open captured packet files and

playback to the network. Colasoft Packet Player supports many packet file formats created by

many sniffer software, such as Colasoft Capsa, Ethereal, Network General Sniffer and

WildPackets EtherPeek/OmniPeek etc, it also support burst mode and loop sending feature.

In order to transmit packets to the network, I used this software (as shown in Figure 3.8). Then I

captured the packets by using Snort. With this, I could capture all the files and alerts were

generated for the same. The network diagram for this is shown in Figure 3.7.

Figure 3.7 Network configuration between Ubuntu machine and Windows XP machine

Figure 3.8 Colasoft Packet Player sending packets to the network

In Snort, all the sent packets were captured when the following code was executed:

snort -A console -i eth0 -c /etc/snort/snort.conf -l /var/log/snort -K ASCII

And the output for that was as follows:

Packet I/O Totals:

Received: 16761

Analyzed: 10275 ( 61.303%)

Dropped: 6486 ( 27.900%)

Filtered: 0 ( 0.000%)

Outstanding: 6486 ( 38.697%)

Injected: 0

The above details are the part of output produced which specify the number of received packets,

the number of packets which are analyzed or dropped or filtered or injected among them.

Action Stats:

Alerts: 5040 ( 48.795%)

Logged: 5040 ( 48.795%)

Passed: 0 ( 0.000%)

Limits:

Match: 0

Queue: 0

Log: 0

Event: 0

Alert: 0

Verdicts:

Allow: 10065 ( 60.050%)

Block: 0 ( 0.000%)

Replace: 0 ( 0.000%)

Whitelist: 210 ( 1.253%)

Blacklist: 0 ( 0.000%)

Ignore: 0 ( 0.000%)

Retry: 0 ( 0.000%)

The above part is another part of the output that was generated ehich shows the number of alerts

generated, those which are logged among them. Also it describes about how many packets can

be allowed normally, those which have to be blocked etc. All these results depend upon the

actions that were mentioned in the rules. If the action is to log, then the packets for which those

rules are applicable are only logged. Similarly various actions in the rules provide various

results. The alerts generated above were logged into files as the destination to log the files was

given separately as /var/log/snort. The log files that were generated are shown in the following

figures.

Figure 3.9 Image showing all the logged files for IP address 10.1.60.253

Figure 3.10 Logged ICMP_ECHO file

Figure 3.11 Logged TCP file

As shown in the above figures, many alerts were generated and they were saved.

3.5. Working with WEKA and JPCAP

Another phase in the project was to analyze any other type of intrusion so that it can be added to

Snort as an additional rule. So to do that analysis I used WEKA tool. But WEKA does not accept

pcap formats. So, all the pcap files have to be converted to .csv format. To do this I used JPCAP.

JPCAP: The JPCAP distribution includes both

A tool for real time network traffic capture and analysis.

An API for developing packet capture applications in Java.

So I downloaded jpcap from Sourceforge and wrote java programs which help in converting the

pcap files to .csv files. The algorithm to get the packets and converting them to .csv format is

given below. More details about jpcap can be obtained from Jpcap tutorial (SIP Inspector).

JpcapCaptor captor=JpcapCaptor.openDevice(device[index], 65535, false, 20); //open a file to save captured packets JpcapWriter writer=JpcapWriter.openDumpFile(captor,"yourfilename");

for(int i=0;i<10;i++){ //capture a single packet Packet packet=captor.getPacket(); //save it into the opened file writer.writePacket(packet); } writer.close();

WEKA: The WEKA tool is used for regression analysis, classification and clustering. To

classify my data, I opened a file in WEKA and applied Decision tree classifier. The result of

which is shown in Figure 3.14.

Figure 3.12 WEKA tool displaying attributes when a file is selected.

Figure 3.13 Attribute representation in WEKA

Figure 3.14 J48 decision tree constructed using WEKA tool for one dataset file

Figure 3.15 Classifier error that occurred during above J48 tree construction

Figure 3.10 represents a classifier error. According to that prediction, the data value should

be present in one IP address but it is in another. So the tool generated many such classifier

errors.

3.6. Results

The primary objective of the project to learn tools like Snort, TCPReplay and Weka

was successful.

The secondary objective to configure Snort as IDS was also successful.

Emerging threats rules were added to Snort to detect the malicious packets in USMA

datasets, and the malicious packets were detected.

4. SUMMARY

Deliberately accessing someone's system without their knowledge is called intrusion. An

intrusion detection system (IDS) is a device or software application that monitors network or

system activities for malicious activities or policy violations and produces reports to a

management station. Our systems need to have constant monitoring and frequent analysis to

be effective. So, Intrusion Detection System is an important tool which has to be

implemented in every system.

Even though many anomaly or signature based Intrusion Detection Systems were developed,

the intruders are coming up with new attacks. IDS’s are not able to detect these attacks. Due

to the complexity in these attacks, the rule sets have to be updated to match the attacks.

To overcome the above problem, firstly Snort tool is installed and is configured as IDS. Two

different virtual machines were installed, one Ubuntu machine within which Snort was

configured and another Windows XP machine within which Colasoft Packet Player was

installed. Network packets of US Army datasets are transmitted from XP machine to Ubuntu

machine. The Snort installed in Ubuntu machine captured the packets and detected malicious

packets and logs were created for the same. To enhance the Snort functionality, another rule

set (Emerging threats rules) were also included. In addition to these, the document describes

about TCPReplay tool (used to transmit network packets from one machine to another),

Wireshark (a tool used for capturing data packets) and WEKA tool, a regression analysis

tool.

5. CONCLUSION

5.1. What was accomplished?

Snort was successfully configured as IDS and sample tests on that IDS worked well. Then

using Colasoft Packet Player, the packets were transmitted through the network from

Windows XP machine and are captured through Snort in Ubuntu machine. Alerts were

generated by the Snort and all the alerts are logged into various files.

5.2. Future work

This project can be extending by analyzing the data and implementing more classifiers to the

data using WEKA tool so that any new type of attack can be detected. Thus, when any attack

is detected, Snort’s rule set can be redefined and better IDS can be developed.

6. REFERENCES

1. Intrusion Detection System https://en.wikipedia.org/wiki/Intrusion_detection_system

2. Importance of IDS

http://www.ijser.org/researchpaper%5CImportance_of_Intrusion_Detection_System.pdf

3. Zhou.C, Huang.S, Xiong.N, Yang.S, Li.H, Qin.Y and Li.X , "Design and Analysis of

Multimodal-Based Anomaly Intrusion Detection Systems in Industrial Process

Automation", Systems, Man, and Cybernetics: Systems, IEEE Transactions, Volume:PP,

Issue: 99, 1, 08 April 2015.

4. Al-Jarrah.O, Arafat.A, "Network Intrusion Detection System using attack behavior

classification", Information and Communication Systems (ICICS), 2014 5th International

Conference, 1-6, 1-3 April 2014.

5. WEKA tool http://www.cs.waikato.ac.nz/ml/weka/

6. Snort https://www.snort.org/downloads

7. US ARMY Datasets http://www.usma.edu/crc/sitepages/datasets.aspx

8. Configuration of Snort

http://wiki.aanval.com/wiki/Community:Snort_2.9.2.3_Installation_Guide_for_Ubuntu_1

2.04,_with_Barnyard2,_Pulledpork,_and_Aanval

9. Configuration of Snort - Snort Install on Ubuntu by Shiela Dios(Youtube video)

10. Modes of Snort http://books.gigatux.nl/mirror/snortids/0596006616/snortids-CHP-3-

SECT-4.html

11. Network Intrusion Detection Mode for Snort http://manual.snort.org/node6.html

12. TCPReplay installation http://sourceforge.net/projects/tcpreplay/

13. Colasoft Packet Player www.colasoft.com