Embed Size (px)
Citation preview
9th International ISC Conference on Information Security and Cryptology
September 2012
Iranian Society of
Cryptology University of Tabriz
Anomaly Dtetection Using Artifitial Immune System
Approach
Masoumeh Raji Yazd University
Electrical and Computer Engineering Department
Yazd, Iran
Vali Derhami Yazd University
Electrical and Computer Engineering Department
Yazd, Iran
Reza Azmi
Alzahra University Computer Engineering Department
Tehran, Iran
Abstract— Malicious activities and intrusive to the systems are
major challenges in security of web servers and web-based
applications. Artificial Immune System based detection, inspired
from a hypothetic model of the human immune system, promises
to provide the possibility of detecting novel attacks at a high rate
of detection effectiveness. Web Host Immune Based Intrusion
Detection System introduces immune principles into IDSs to
improve the capability of learning and recognizing novel web
attacks. In this paper immune network and Negative Selection,
two models of artificial immune system, have been compared.
Test and comparison are done on NSL-KDD dataset and
theoretical analysis and experimental evaluation demonstrate
that the Immune Network model is more suitable than Negative
Selection for detecting unknown attacks.
Keywords- Intrusion Detection Systems; Artificial Immune
Systems; Anomaly;
I. INTRODUCTION
Techniques allowing for the detection of novel attacks in
protecting computer systems from intrusions that bypass
preventive countermeasures and evade signature-based misuse
detectors, are mainly based on anomaly detection techniques
or more recently on human immune system [1].In [1], it is
reported that ”Netcraft maintains that 70% of the servers
visible on the internet today are Web servers, with a plethora
of services being added on top of HTTP”. This fact, along
with the security-critical nature of applications being deployed
today via web applications (e.g. e-commerce) and the hostile
environment that the Internet presents (e.g. attacker
anonymity, un-secured message transmission, self-propagating
malware), makes security attacks targeted at web application
hosting machines, a primary concern.
The Artificial Immune System (AIS) is a powerful
paradigm for learning which is originally inspired from the
natural immune system. There are a number of motivations for
using the immune system as inspiration for clustering web
users which include recognition, diversity, memory, self
regulation and learning [3]. The vertebrate immune system is
composed of special type of white blood cells (called B-cells),
which are responsible for detecting antigens and defending
against them. When an antigen is detected by the B-cells, an
immune response is promoted resulting in antigen elimination.
One type of response is the secretion of antibodies by B-cells
(cloning). Antibodies are Y-shaped molecules on the surface of
B-cells that can bind to antigens and recognize them. Each
antibody can recognize a set of antigens which can match the
antibody. The strength of the antigen-antibody interaction is
measured by the affinity of their match [2].
Many artificial immune models have been discussed in
literature such as Negative Selection (NS)[6], Danger Theory
(DT) [11] and Artificial Immune Networks (AINs)[12]. We use
the AIN model which was initially proposed by Jern [4] and
NS and compare results to each other.
Test and comparison are done on NSL-KDD dataset. It is a
new version of KDDcup99 and has some advantages over
KDDcup99. It has solved some of the inherent problems of the
KDD'99[5]. It is considered as standard benchmark for
intrusion detection evaluation [5]. The training dataset of
NSL-KDD similar to KDD99 consist of approximately
4,900,000 single connection vectors each of which contains 41
features and is labeled as either normal or attack type, with
exactly one specific attack type .
The remainder of this paper is organized as follows. Section 2
presents the research context related to immune-based intrusion
detection. In section 3, a review on principles of artificial
immune systems is presented. Section 4 discusses the goals of
this study and introduces algorithm regarding the data
representation. In section 5, the experimental evaluation of the
proposed system is presented. Moreover, the detection ability
of the tow proposed algorithms are tested. Finally, section 6
concludes our study.
9th International ISC Conference on Information Security and Cryptology
II. RELATED WORK
Many artificial immune systems have been proposed and built
over the years. Forrest et al. [6] laid much of the groundwork
for artificial immune systems, including distinguishing “self”
from non-self, using negative selection and applying immune
system principles to security. A network intrusion system
called LYSIS was developed that monitors features of the
TCP/IP packet to detect abnormal traffic. Guangmin [7]
presents an immune based active defense model for web
attacks which is on the basis of the clone selection and hyper-
mutation (IADMW). Http queries are considered as the
antigens. An http query is represented by a vector of attributes
extracted from the http query, with associated weights
represented the importance of the attribute in the http query.
Danforth [8] presents the Web Classifying Immune System
(WCIS) which is a prototype system to detect attacks against
web servers by examining web server requests. Focused on
distinguishing self from non-self and laid the foundations for
the negative selection algorithm. WCIS considers some
features, these features include: length of the URI, number of
variables and distribution of characters. Rassam [9] proposed
an immune network clustering method that is robust in
detecting novel attacks in the absence of labels. The purpose of
this study is to enhance the detection rate by reducing the
network traffic features and to investigate the feasibility of bio-
inspired immune network approach for clustering different
kinds of attacks and some novel attacks. Rough set method was
applied to reduce the dimension of features in DARPA KDD
Cup 1999 intrusion detection dataset. Immune network
clustering was then applied using ainet algorithm to cluster the
data. Previously, we proposed an intrusion detection system,
Based on the principles of the immune system (WHIBIDS) that
can detect known and unknown attacks [10]. The requests
obtained from the preprocessed log files of web server are
presented to the system as antigens. The network of the B-cells
represents a summarized version of the antigens encountered to
the network. Also, they are able to adapt to emerging usage
patterns proposed by new antigens at any time. WHIBIDS
introduces immune network principles into IDSs to improve
the capability of learning and recognizing web attacks,
especially unknown web attacks.
III. ARTIFICIAL IMMUNE SYSTEM
Artificial immune system is derived from the natural
immune system mechanisms that are mainly of two parts:
innate immune and adaptive immune. Innate immune uses
defense tools such as skin and mucous against foreign
substances (antigens). In case of failure of the innate
immune system and the arrival of foreign elements into the
body, Adaptive immune system for a natural reaction is
activated. This part of the natural immune system is able to
identify new types of foreign elements and do appropriate
response to rejection them.
A. Artificial immune network
The immune network theory was proposed by Jerne [4] as
a way to explain the memory and learning capabilities
exhibited by the immune system. The principal hypothesis of
this theory states that immune memory is maintained by B-
cells interacting with each other, even in the absence of foreign
antigens. These interactions can be either excitatory or
inhibitory. The production of a given antibody (elicited by an
external antigen) stimulates/suppresses the production of other
antibodies that stimulate/suppress the production of other
antibodies and so on [11]. Notice that the word antigen denotes
those molecules that the immune cells/molecules are able to
recognize, thus it is necessary to differentiate between self
antigens (antibodies) and non-self antigens. Accordingly with
the notation suggested by Jerne [4], the portion on the antigen‟s
surface that an antibody recognizes is named epitope, the
portion used by an antibody to recognize antigens is named
paratope, and the epitope of an antibody (self antigen) is named
idiotope. Based on Jerne‟s work, some models of immune
network were developed using differential equations to predict
the antibody concentration during and after an immune
response.
An AIN is a bio-inspired computational model that uses
ideas and concepts from the immune network theory, mainly
the interactions among B-cells (stimulation and suppression),
and the cloning and mutation process. Several models have
been proposed for problem solving in areas such as data
analysis, pattern recognition, autonomous navigation and
function optimization.
B. Negative selection
Forrest et al (1994; 1997) proposed and used a negative
selection algorithm for various anomaly detection problems.
This algorithm defines „self‟ by building the normal behavior
patterns of a monitored system. It to each self pattern defined.
If any randomly generated pattern matches a self pattern, this
pattern fails to become a detector and thus it is removed.
Otherwise, it becomes a detector pattern and monitors
subsequent profiled patterns of the monitored system. During
the monitoring stage, if a „detector‟ pattern matches any newly
profiled pattern, it is then considered that new anomaly must
have occurred in the monitored system [6]. The overview of
this algorithm is provided in “Fig. 1”and “Fig. 2”.
9th International ISC Conference on Information Security and Cryptology
Figure 1.Detector Set Generation of a Negative Selection Algorithm
Figure 2.Detection by a Detector Set
So far, various methods have been proposed for modeling the
non-self pattern. Two common methods of NS include
algorithms with fixed radius and variable radius. Detector
production stage in “Fig. 3” has two distinct parts which are
related to two types of algorithms are listed. Blue spheres on
the right side represent self area with variable radius and the
left side fixed radius.
As you can see in “Fig.3”, there is a need for a data set of
normal behaviors in training phase. The advantages of using
variable radius are; first, the low number of non-self detectors
can be used to cover the whole region. Second, very narrow
regions are close to the normal data, that the detector with a
very small radius, can be covered them.
I. PROPOSED METHOD
The proposed Web Host Immune Based Intrusion Detection
System introduces immune principles into IDSs to improve the
capability of learning and recognizing web attacks, especially
unknown web attacks. Antigen and antibodies are represented
same form and their length is equal. Antigen Presenting:
Define each users request as the antigens set Ag. Each request
is represented by a vector of attributes extracted from NSL-
KDD.
Figure 3.Intrusion detection system training phase for producing detectors (adapted from [13])
Affinity function: similarity measure between tow antigen
is Euclidean distance determines the distance between two web
application requests. Precisely, the similarity between two
requests agi and agj is defined as:
2
1
)(),( jn
k
n
inji agagagagdis
(1)
Where k is the number of features is extracted for each
request.
A. Immune network
There are some stimulating and suppressing interactions
between B-cells. The learning process starts by presenting a set
of input data (antigens) to the network of B-cells one at a time.
The system tries to learn an optimal network of linked B-cells
using cloning operation. Each B-cell represents a learned
pattern that can match to an antigen or another B-cell. In
addition, each B-cell represents a softly defined influence zone
that is described in a term of weight function which decreases
with distance from the antigen and the time since the antigen
has presented to the network. The strength of the link between
two B-cells is directly related to their similarity.
The activation of ith
B-cell caused by jth
antigen in the
network after J antigen are presented to the network is defined
by (2). In this equation, dij2 is the distance from antigen j to B-
cell i. σij2 is the scale factor that defines the size of the
Generate
random
detector
Self Pattern
Detect
or Set
Yes
No
Start
End
No
Start
New
pattern
Match
Sufficien
t number
of
detector?
Match with
detector
Self
Anomaly
End
Yes
No
9th International ISC Conference on Information Security and Cryptology
influence zone around a cluster prototype. τ is a constant that
determines the rate of forgetting in immune network.
)2
(2
2
ij
ijd
ij ew
(2)
The stimulation level of a B-cell after presenting J antigen
to the network and the optimal scale of ith
B-cell can be
calculated based on (3) and (4) respectively.
2
1
2
1
2
1
2
1)()(
ij
J
j
ij
ij
N
l
il
ij
N
l
il
ij
J
j
ij
ij
ww
t
w
t
wBB
(3)
J
j
ij
J
j
ijij
ij
w
dw
1
1
2
2
2
(4)
In (3), the first term on the right side of the equation
describes the pure stimulation of B-cell caused by antigen j.
Also, the second and third terms represent co stimulation and
co suppression interactions from other B-cells in the network
respectively. The forth terms indicate the suppression if classes
antigen and B-cell are not the same. The parameter NB is the
maximal number of B-cells in the network. The parameters α(t)
and β(t) are stimulation and suppression coefficient of B-cell
and are updated based on the age(t) of the B-cell. γ is
suppression coefficient between antigen and B-cell. If an
antigen could stimulate the B-cell sufficiently, (wij ≥ wmin),
then the age of this B-cell is refreshed to zero. Otherwise, it
increases by one. The coefficients increase as the age of the B-
cell decreases, hence recently activated B-cells have more
impact on the network [2].
The pseudo code of the proposed algorithm is presented as
following.
1- 1-Fix the Maximal population size NB;
2- Initialize B-cell population and initi 2
using a number of random antigen;
3- Repeat for each antigen;
1. Present antigen to each B-cell;
2. If antigen activated the B-cell
minwwij ;
I. Refresh age(t=0);
II. Add the current B-cell ad
its KNN to working sub-network;
3. Else
I. Increment the age of B-
cell by one;
4. If for all B-cells minwwij ;
I. Create a new B-
cell=antigen;
5. Else
I. Repeat for each B-cell in
working sub-network
i. Compute B-cell
stimulation
ii. Update B-cell2
i
6. If antigens of a session is presented;
I. Clone B-cell based on
their stimulation level;
II. If population size>NB;
i. Remove
extra least
stimulated B-
cells; ALGORITHM 1: THE MODIFIED ALGORITHM OF [2]
As it is shown in proposed algorithm, when an antigen is
unable to activate any B-cell, this antigen may represent a noise
or a new emerging pattern. In this condition, a new B-cell is
created which is a copy of the presented antigen. If this antigen
is a noisy data and does not present a new emerging pattern, it
would not get enough chance to get stimulated by incoming
antigens and is probably eliminated. After each antigen is
presented to the network, the B-cells go under cloning
operation based on their stimulation level. When the population
of the network exceeds a defined threshold, the least stimulated
B-cells are removed from the network.
The distance measure presented in this study is used in all
the steps for calculating the internal and external (B-cell to
antigen) interactions of B-cells. The detailed information about
calculating stimulation level and update it are described by [2].
In the training phase tow profiles of normal and abnormal
behaviors using the proposed algorithm are built. Then, they
are applied to new request in order to detect abnormal
behaviors in the testing phase.
B. Negative selection with variable size detector
The real-valued negative selection algorithm operates on a
unitary hypercube [0, 1] n. A detector dj = (cj, rj) has a center c
∈ [0, 1] n
and a non-self recognition with radius rj ∈ R.
Furthermore, j=1,…,m and m is the number of detectors .Every
self element si = (ci, rs) has a center and a self radius rs,
i=1,…,l and l is the number of self samples. The self-radius
was introduced to allow other elements to be considered as self
elements which lie close to the self-center.
Unlike the self sample, radiuses of detectors (rj) are not fixed.
As it is shown in (5), rj is calculated by Euclidean distance
between the detector center and the closest self sample. If
specified distance is greater than rs, the detector is
removed[14].
sjili
j rccdistr
),(min1
(5)
When the number of detectors has reached a number of
predetermined (Tmax), algorithm is terminated.
The pseudo code of the proposed generation detectors
algorithm is presented as following.
9th International ISC Conference on Information Security and Cryptology
Input: S = Set of self elements, Tmax = max number of V-
Detectors, rs = self radius,
Output: D = Set of generated V-Detectors
1. Select random detectors;
2. Repeat for each detector;
a. Repeat for each self sample;
i. Calculate Euclidean distance
between detector and self sample;
b. Calculate minimum of distance
c. If distance> rs
i. r_detector=r_detector -rs
d. if r_detector is real
i. add this detector to mature detector
set
3. for each non self sample
a. if detector not recognize it
i. add to mature detector
ii. calculate r_detector; ALGORITHM 2: GENERATION DETECTORS
II. EXPERIMENTAL EVALUATION
The empirical evaluation reported in this paper is
performed on NSL-KDD. The original data used in our
experiment, contains 125970 requests for training phase. We
tested tow proposed algorithms on this data set and results are
shown in table I.
TABLE I. EXPERIMENTAL RESULTS OF TOW PROPOSED ALGORITHMS
The maximal population size of the network is set to 50; the
control parameter for the number of nearest neighbors (K) is
set to 3. The activation threshold (wmin) is 0.75, the similarity
threshold θ =0.75 and τ =20. If the weighted distance is greater
than θ, each B-cell is activated. Parameters of NS with variable
size of detector are: TMax=500 and rs=0.1.
The best values of these parameters are obtained via a
genetic algorithm optimizer. Unlike related works these values
were calculated only through trial and error.
Different kinds of metrics are measured to evaluate the
ability of the algorithm to learn the properties of the features of
the data and also detecting the anomaly activities. Detection
rate is the fraction of true positive rates to the number of all
cases that should have been classified as positive. The false
alarm rate can be defined as the proportion of actually normal
cases that were incorrectly classified as anomalous.
We run algorithm 5 times with 5-folds cross validation and
the final values for evaluation measures is the average of these
5 runs.
Table I represent the proposed immune network algorithm
has high capabilities in comparison with NS with variable size
detector. We can claim that the proposed algorithm is
performing high accuracy in detecting malicious activities.
“Fig.4” demonstrates the area under the AIN algorithm curve is
greater than NS algorithm. Algorithm has better performance
if the area under the ROC curve is closer to 1. Column of the
curve shows false alarm rate and row displays detection rate.
Intrusion detection method Artificial Immune System
Negative Selection with
variable size detector
Immune Network
Experimental
conditions
Train phase
Normal data Normal and Abnormal data
Test Phase Normal and Abnormal data Normal and Abnormal data
Evaluation Metrics Accuracy
(%)
False
alarm
rate
(%)
Detection
rate (%)
Accuracy
(%)
False
alarm
rate
(%)
Detection
rate (%)
Results First run 77/13 .03 63.3 93.1 .012 90
Second run 77/7 0.031 63 94 0.15 88.2
Third run 76/5 0.035 64 94.5 0.01 92
Forth run 78/04 0.029 65.1 93.3 0.12 94
Fifth run 76/92 0.033 62.5 95 0.015 91
average 77/285 0.0316 63.58 94.06 0.0128 91.24
9th International ISC Conference on Information Security and Cryptology
Figure 4.ROC Curve for tow proposed algorithms
The network of the B-cells represents a summarized version of
the antigens encountered to the network. Also, they are able to
adapt to emerging usage patterns proposed by new antigens at
any time. Introduced AIN can track different patterns as they
are presented to the network. Also there are some stimulating
and suppressing interactions between B-cells and the system
tries to learn an optimal network of linked B-cells using
cloning operation. Thus, this system is better adapted itself
with new attacks.
III. CONCLUSION
In this paper we proposed two models of artificial immune
system for detecting web anomaly. The results show more
ability of the proposed AIN to clustering web requests to
normal and abnormal than Negative Selection. We compared
AIN and NS with variant size detector that has better
performance than NS with constant radius. In immune network
algorithm the network of the B-cells represents a summarized
version of the antigens encountered to the network. Also, they
are able to adapt to emerging usage patterns proposed by new
antigens at any time. This research is designing an immune
base IDS that has several advantages: (1) Self learning and
immune learning make the model can detect both the known
and unknown web attacks. (2) Ability to detect anomaly in real
time (3) Using immune network algorithm achieved high
detection rates. (4) Can be used as a general classifier. In this
Paper assign a variable value is determined through genetic
algorithm .Future work will determine these parameters by
reinforcement learning and the results will compare with
genetic algorithm.
ACKNOWLEDGMENT
This research was supported by APA center in Yazd
University. The authors would like to thank APA for its
support.
REFERENCES
[1] Vella, M., Roper, M., Terzis, S.,”Characterization of a danger context for detecting novel attacks targetig web-based systems “(2010), http://www.cis.strath.ac.uk/~mv/trep2.pdf
[2] M. Azimpour-Kivi and R. Azmi,” Applying Sequence Align ment in Tracking Evolving Clusters on Web Sessions Data, an Artificial Immune Network Approach”, Computational Intelligence, Communication Systems and Networks (CICSyN) (2011).
[3] B. H. Helmi and A. T. Rahmani, “An AIS algorithm for Web usage mining with directed mutation”, Pro. World Congress on Computational Intelligence (WCCI08) (2008).
[4] N. k. Jerne, “Towards a Network Theory of the Immune System”, Annals of Immunology (1974), 373-389.
[5] M. Tavallaee,E. Bagheri, A. Ghorbani, “A Detailed Analysis of the KDD CUP 99 Data Set”, proceeding of IEEE symposium on computational Intelligence in security and defence application(2009)
[6] S. Forrest, A. S. Perelson, L. Allen, and R. Cherukuri, “self nonself discrimination in a computer”, In Proc. 1994 IEEE ACM Symposium on Research in Security and Privacy, pages 202 – 214, Los Alamitos, CA, USA, 1994..
[7] L. Guangminl, “Modeling Unknown Web Attacks in Network Anomaly Detectio”, International Conference on Convergence and Hybrid Information Technology (2008).
[8] M. Danforth, “Towards a Classifying Arti_cial Immune System for Web Server Attacks”, Department of Computer and Electrical Engineering and Computer Science, International Conference on Machine Learning and Applications (2009).
[9] M. A. Rassam, M. A. Maarof, and A. Zainal, “Intrusion Detection System Using Unsupervised Immune Network Clutering with Reduced Features”, Int. J. Advance. Soft Comput.ppl. 2/2010 (2010).
[10] M.Raji, V.Derhami, R.Azmi. Brewer, “Web Anomaly Intrusion Detection System Using Artificial Immune System Approach”, 6th Internatinal Conference on e-Commerce in Developing Countries(ECDC 2012) in press.
[11] J. Kim; P. J. Bentley, U. Aickelin, J. Greensmith, G. Tedesco, J. Twycross, “Immune system approaches to intrusion detection – a review”, Natural Computing: an international journal, Volume 6 , Issue 4, pp. 413-466,( 2007).
[12] J. C. Galeano; A. VelozaSuan; F. A. González, “a comparative Analysis of Artificial Immune Network Models”, GECCO‟05, Washington, DC, USA.(2005), June 25–29.
[13] R.Azmi, B.Pishgoo, H.Nemati, “Intrusion detection based on supervised by using artificial immune system”,(persian),8thInternatinal ISC Conference on Information Security and Cryptology(ISCISC 2011).
[14] T.Pourhabibi, R.Azmi, “Intrusion Detection Using Negative Selection with varient size of detectors”,(persian), National Conference of Security and Information and Communication,(2010).
0
0.2
0.4
0.6
0.8
1
0 0.2 0.4 0.6 0.8 1
NegativeSelection
Immunenetwork
FA
Dr
