Towards Automatic Traffic Classification

Towards Automatic Traffic Classification

Zhaohong Lai, Alex Galis, Miguel Rio and Chris Todd Department of Electronic and Electrical Engineering, University College London, London, UK

Abstract

Classification of network traffic recently has attracted a great deal of interest as it plays important roles in many areas such as traffic engineering, service class mapping, network management etc. One of the challenging issues for existing detection schemes is that they need prior manual analysis to detect unknown traffic, which is infeasible to cope with the fast growing number of new applications. In this paper, we propose an automatic traffic classification scheme, which is realised by managing traffic detection knowledge with the use of ontologies on the one hand, while developing the self-learning model on traffic detection according to ontologies on the other hand. Also, based on two scenarios, the experiment results demonstrate the automated detection capability for the proposed scheme.

1. Introduction

Classification of network traffic plays important roles in many areas such as network management, traffic engineering, service class mapping, QoS provisioning etc. In the early Internet, traffic classification could be easily realised by reading port numbers. With the rapid evolution of network services, many applications are becoming more difficult to identify as they can be formed in various ways like using unregistered ports or wrapping in other protocols (i.e. HTTP). Currently, there are a number of tools that are able to classify different types of traffic [1]. Some of them are able to detect P2P traffic effectively and others are dedicated to classifying ‘chatting’ traffic etc. They are realised with varying techniques such as payload decoding, signature scanning, traffic statistic analyzing etc.

One of the challenging issues for existing detection schemes is that prior traffic analysis is an essential requirement for them to detect unknown traffic [2]. Given the fast growing number of new applications, prior traffic analysis becomes less reliable. To deal with this issue, there are two important aspects that need to be addressed. One the one hand, a new classification system is required to be able to manage the knowledge of existing detection schemes; as such a

system can be easily shared and reused. On the other hand, it is necessary to investigate how to reason from existing knowledge, on traffic detection, so as to achieve an automatic identification capability.

In this paper, the above two issues are tackled by exploring an automatic traffic classification system. Firstly, how to use the context and ontology means to manage traffic detection knowledge is discussed. Then, this paper goes on to illustrate construction of traffic classification ontology and a model of the flow profile domain ontology. In addition, it reveals the clustering relationship between an IP flow profile and its traffic category. According to this feature, it exploits an automatic clustering engine to cluster flow profile ontologies by using Kohonen’s self-organising map [3].

The paper is structured as follows. Section 2 reviews a variety of existing traffic classification methods and related issues. Section 3 discusses the reasons for employing context and for introducing ontologies and concludes by proposing automatic traffic classification architecture. Section 4 illustrates how to construct both traffic and an IP flow classification ontology. Section 5 exploits the relationship between a flow profile and its traffic category and then discusses how to realise an intelligent learning algorithm with Kohonen’s self-organising map. Section 6 shows how to conduct two testing scenarios and includes related experimental results. Section 7 proposes the application of these schemes to Ambient Networks [4] in the future. Section 8 briefly summarises the conclusions of this paper.

2. Problem Statement

Classifying network traffic is proving challenging. A variety of studies on IP traffic classification have been proposed in the past few years. Firstly, the most direct and efficient method is to use well known ports [5]. The results of this approach are often inaccurate, due to the dramatic increase in network applications using random ports or in tunnelling through HTTP [1]. Secondly, solutions based on payload or signatures are

Third International Conference on Networking and Services(ICNS'07)0-7695-2858-9/07 $20.00 © 2007

TABLE 1. DIFFERENT NETWORK DETECTION SCHEMES

Inputs Prior knowledge Detection algorithms port numbers checking

registered ports rule-based comparison with port number

payload: signature

to know signature value

comparing with pre-known signature

payload: protocol

to learn protocol value

comparing with pre-known protocol

traffic content (payload)

analysing specific traffic (i.e.P2P)

hybrid algorithms

traffic content (header+payload)

to know the statistic feature

Statistics techniques (Naive Bayse, ML etc)

�widely applied. These solutions are widely used, specifically to identify P2P traffic e.g. by using payload payload decoding [6][7] and by signature scanning [8]. More recently a content-based identification scheme has been proposed [2] that effectively integrates all the payload and signature scanning methods. Although high accuracy has been achieved by integration, such methods not only introduce an overhead arising from examination of content details but also a number of intricate issues arising from the integration itself. . Furthermore, one big drawback of these methods is that they are unable access the encrypted content. In [1], the classification of BLINC (BLINd Classification) is implemented from a wider perspective to embrace various properties at three different levels: the social, functional and application level. The use of statistical techniques to detect network applications recently has received a great deal of interest [9][10][11][12][13][14][15][16]. These techniques include Naïve Bayse [11][12], machine learning [17][18] and clustering [14][16]; they rely on features of the traffic statistics e.g. packet size distribution [9][10], packet interval time etc. However, statistics based methods do not include network-related properties such as IP address distribution [7] and the P2P network diameter [19] which are used to detect P2P traffic.

A consequence arising from the use of these techniques is the rapid rise in the quantity of information needed for traffic detection. Some of this information is simple such as port number or IP address and can be directly captured. In contrast, collection of data on flow statistics is a more complex process [12]. Some studies [1][2] have developed hybrid systems aiming to simplify by integrating existing techniques together, but prior manual analysis is essential in their design to detect new traffic. In a step forward some automated and self-learning features of traffic detection have been demonstrated [17][20][21]. Nevertheless, they are currently limited

to some specific statistical methods and may lack the flexibility to adapt to new detection techniques. Therefore, there are some outstanding issues.

• How to manage rising quantities of traffic detection knowledge?

• How to enable different techniques that can be easily accessed and reused?

• How to adapt new detection techniques without changing a classification architecture?

• How to yield high-level knowledge that can be used to automatically detect traffic, and in particular to reduce the need for manual involvement?

3. Towards Automatic traffic identification

In this section, an automatic traffic classification architecture is proposed to support both the ontology management and automatic detection functionalities deployed.

3.1 The use of Context and Ontologies to describe traffic detection knowledge

As discussed in section 2, Table 1 shows a summary of a variety of detection solutions classified by (at least) two types of information. One type includes the kind of input data and any requirements for prior knowledge; the other type of information is the detection algorithm itself. For example, a payload-based technique typically needs information on how many bits are used for signature, what the signature value for a particular application is etc. Then, the traffic detection algorithm is realised by comparing input data with the prior knowledge. Therefore, we need two mechanisms to fully represent the knowledge enshrined in a traffic detection system. One is used to represent prior knowledge and input data and the other is used to describe algorithms on traffic detection. In this paper, such tasks can be achieved by the use of context and ontology as discussed below.

The concept of context has been widely used in many fields [24][25]. A context is defined as any information or knowledge that can be used to customize the situation of an entity [25][26][27]. For instance, an entity can be one specific IP flow or a network node and its related contexts can be inputs information (Table 1) or prior analysis requirements etc. Of these contexts, some of them are explicit, which can be directly obtained from entities while others are implicit, which needs the additional computation like statistic data. The implementation of contexts is realised by network context sensors, which are detailed in later.


On the other hand, by using the ontology means, the algorithm for each detection scheme can be fully expressed. Ontology has been extensively used for various knowledge management purposes in different areas such as Semantic Web [28], flow classification [29], semantic routing in P2P networks [31][32]etc. Gruber defines a specific ontology as the explicit specification of conceptualisations used to help programs and humans share knowledge [30][33]. In general an ontology includes structured knowledge statements that describe the concepts of a domain and their relationship [34][35][36]. Another important term related to ontology is the knowledge base, which is formed by concept instances. There is a direct connection between context and ontology. On the one hand, as stated in [37], the main components of ontology are the domain concepts. Both concepts and concept instances are called entities [35]. On the other hand, according to the context definition, contexts are defined as any information used to characterise such entities. Therefore, the connection between context and ontology is linked by entities, which is shown in fig.1.

Traffic Classification Contexts

Ontology

ConceptsEntities

Relations Knowledge BaseAxioms

Instances

Figure 1. A simplified structural relationship between context, ontology and knowledge base

According to the knowledge statement from ontologies (or knowledge base), we can further infer low-level contexts to yield the high-level data for the traffic detection [38]. To achieve this goal involves a series of task designs, which are presented below.

3.2 An Automatic Traffic Classification Architecture

In this section, we propose an automated traffic detection architecture that is realised in the manner of a modular design. The left box including ontologies and knowledge base is designed to support the seamless knowledge management for traffic detection. The middle two boxes together are to implement automatic traffic identification functionalities. The proposed detection architecture includes four major components as discussed below.

newtraffic?

YES

NO

MAP BIAS

IP flow Profile MAP

Underlying Networks

Service Classes

TrafficCategories

Traffic Classification SensorTraffic

AutomaticClassifierOntologyClustering

Engine

Network Context Sensors

KnowledgeBase

Ontology ManagerCreationInsertion

maintenance

Figure 2. Automatic traffic classification sensor and its architecture

Ontology Manager is the core part of the traffic detection knowledge management. It performs the ontology creation, insertion and maintenance tasks. To successfully achieve these tasks, the key is to have a clear picture about the relationships between different detection techniques. An in-depth analysis on how to construct the traffic detection ontology is given in the next section.

Network Context Sensors are responsible for collection of all the needed information as stated in the ontologies. It monitors not only the tangible objects, like network nodes, but also the intangible entities. Generally, sensors are categorized into two types: the network node sensor and flow context sensor. A node sensor senses all the network activities happened on that node. Such context information is essential to derive information on for example social behaviour [1] and IP address distribution in P2P traffic [7][19]. The flow context sensor collects raw traffic, using the packet capture (pcap) library, and aggregates the data into flows. Then, the raw data will be transformed into contexts, which will be sent to the ontology clustering engine for the next step in data processing.

Ontology Clustering Engine (OCE) aims to learn the traffic characteristics using the data from network context sensors. The learning algorithm is realised with Kohonen’s self-organising map (SOM), A role of the OCE is to cluster contexts into different areas together with the concept statement in the ontologies. To avoid problems arising from the likely existence of high divergence, between different concepts in the ontology, the OCE strategy is to focus primarily on the IP flow profile data; this will be further discussed in section 5.


Automated Traffic Classifier (ATC) is designed to perform the automatic detection functionality. It uses the data from OCE to yield an IP flow profile map (IFPM). The ATC assumes that each traffic category has its own place in an IFPM. However, if the received data overlaps with others, ATC will use the map bias information for the further classification. The bias information refers to additional information obtained by using different detection techniques.

Considering the background of how to deploy such traffic classification functionalities into real networks, a new term called traffic classification sensor is defined to represent the whole detection task. On the top of traffic classification sensors, other services can be derived like service classes mapping (according to traffic categories), overlay routing adaptation etc

4. Traffic classification ontology construction

In this section, firstly, the skeleton of the traffic classification ontology will be given. Having defined the traffic classification ontology, the ontology manager can then perform the creation task to construct ontologies. Due to limited space, the details of ontology manager implementation are not included here. In the second part, how to model an IP flow profile with five concepts has been addressed.

4.1 The ontological view on traffic classification There are two important factors that need to be

carefully considered for the ontology creation. One the one hand, a new designed ontology should include existing major detection techniques. On the other hand, it also needs to support the automatic detection functionality. To deal with the first issue, previously proposed views on ontologies are adopted here [35][39][40]. Naing et al. [39] define an ontology that consists of six elements: },,,,,{ XHARAC RC ,

where C refers to a set of concepts; CA represents attributes for each concept; R represents a set of relationships; RA represents the attributes for a set of relationships; H stands for a concept hierarchy and Xrepresents a set of axioms. According to this definition, a traffic classification ontology is presented in Fig 3, which mainly focuses on concepts C and subClassOf relationships R. In this ontology, network is root and includes two concepts: nodes and traffic, where, in traffic, three concepts are exploited: traffic category,IP flow profile and traffic statistic.

has port

see aslo

s

s

s

s

s

s has traffic

has social behavior

has

flow

pro

file

has statistic features

see also

has applications

has connectionwith other hosts

Network

Nodes

SocialBehavior

Traffic

categories(applications)

IP FlowProfile

trafficstatistic

IP connectionnumber

packet sizedistribution

interval timedistribution

flowevolution

bidirectionalpacket divergence

trafficintensity

portnumber

payloadSignature

payloadProtocol

see

also

has signature has protocol

Figure 3. Traffic classification ontology (s=subClassOf)

As shown in Fig 3, the ‘application’ has various relationships with other concepts: payload signature, payload protocol, traffic statistic, IP flow profile andsocial behaviour. The existing major detection concepts discussed in section 2 have been included in this ontology such as layer 4 port, payload protocol, traffic statistic, IP flow profile, social behaviour etc. It should be noted that here the contexts share the same meaning with CA . For example, for the concept ‘layer 4 port’ the contexts can be TCP or UDP ports, which are further divided into three classes1: well-known port numbers [0-1023], registered port numbers [1024-49151] and unregistered port numbers. Similarly, there exist different sets of contexts for each detection concept.

In Fig 3, a new classification concept called IP flow profile is exploited. As it will be discussed below, a network traffic category is closely related to it is IP flow profile. This is the key to applying automatic detection ability to traffic classification; where an IP flow profile is regarded as a sub-domain ontology in the whole ontology structure. Similarly, other detection concepts such as traffic statistic can also be extended to other sub-domain ontologies.

4.2 IP flow profile ontology modelling The IP flow profile for each application is unique,

as it is affected by various factors e.g. network conditions and application protocols etc. A flow profile can generally be described by a set of concepts like ‘when (time), where (IP address +port number) and what (transport protocol)’. In this paper, the accuracy of a flow profile ontology model is improved by using

1 http://www.iana.org/assignments/port-numbers


five concepts, all of which are closed related to traffic categories. The first two concepts are the distribution of packet size and interval time between packets. Previous studies [9][10] have established that the packet size distribution varies in different traffic categories. Similarly, it is evident that the time gap between packets is diverse between different flows. As shown in Fig.4, the gap value is low and relatively constant in Realplayer but higher and more dynamic in Skype, that is paced by human speech. Thirdly, the traffic volume within a given time, called the flow intensity, is also closely related to its traffic category. In particular, a flow’s intensity is highly driven by the protocol at the application layer. For instance, video-based applications normally produce more traffic than text or voice based applications. The next concept goes to the flow evolution trend, which is defined as the sinusoidal value of the angle between packet length and gap (Fig.5):

22)sin(_

gaplength

lengthtrendflow+

== θ (1)

where the trend value increases when )()( '' gapflengthf > and decreases

if )()( '' gapflengthf < . It is important to select a suitable time unit when performing a trend calculation. If a selected time unit is too small (e.g. a nanosecond) a very small trend value will result. In the common LAN environments, a more ideal result can be achieved if a microsecond time unit is chosen, as this unit is in accord with an IP packet size range of 28 to 1500 bytes. It is clear, there are notable differences between Realplayer and Skype . Skype is mainly affected by the packet length as the gap stays quite stable. By contrast, in the case of Realplayer, it is mostly determined by the packet gap value. Such a distinct phenomenon is illustrated in Fig.5, which also demonstrates that a flow evolution is affected by its traffic category and vice versa.

packet gap difference

0100200300400500600700

0 5 10 15 20 25packet number

10ns

Realplayer-BBCSKYPE-voice

Figure 4. The packet gap comparison:

RealPlayer and Skype

flow trend with sine value

020406080

100120

0 20 40 60 80packet number

Sin

ex10

0

RealplayerSKYPE

Figure 5. The flow evolution trend of Realplayer and Skype(voice)

The fifth concept related to IP flow profile is the ratio of outgoing and incoming packets within a given time, called a flow’s bidirectional packet divergence (fbpd):

numberpacketinnumberpacketoutfbpd

____= (2)

Different applications may produce varying fbpdvalues. For example, the fbpd ratio will be asymmetric in Realplayer but symmetric in many Games applications.

5. Algorithm design for Ontology clustering engine

Having discussed traffic classification ontology, this section will address how to design the self-learning algorithm for the ontology clustering engine. Owing to the high divergence of different concepts in the traffic classification ontology, it is impractical to reason from all of them. Thus, in the first part, the relationship between a traffic category and its flow profile is addressed. Then, in the second part, the application of Kohonen’s self-organising map, to implement the clustering algorithm, is discussed.

5.1 Relationship between IP flow profile ontology and its traffic category

Given a flow profile ontologyO , we know that it consists of a set of concepts iC : OkiCi ⊆= }...1,{where k=5. In some cases, an application can be detected by only analysing a single concept. For example, a series of network applications may be determined only according to the packet size distribution [23]. Nevertheless, it is more often the case that the network application detection is realised by regrouping concepts. For example, according to the definition of fpbd, games applications can be distinguished from multimedia applications2. If the

2 We assume there are only two applications in the target traffic: games and multimedia.


traffic is determined as a multimedia application, further concepts {packet size, flow evolution trend} can be used to identify whether it is primarily a voice or video based application. Since each application has its own IP flow profile features, as discussed in section 4, a corresponding set of related concepts will exist. As the classification accuracy is increased, after each regrouping step or clustering of features among iC ,distinguishability between different IP flow profiles should improve but may not always be unique.

5.2 Clustering IP flow profile ontology with Kohonen’s Self-Organising Map

To cluster data from a flow profile ontology, Kohonen’s Self-Organising Map (SOM) provides a well understood approach. For example the self-organising semantic map (SOSM) has been developed to cluster semantic relationship among different objects [41]. Also, the self-organising feature map (SOFM) has been widely used to achieve a similar goal. The difference between the SOFM and SOSM approaches arises as follows: whereas SOFM clusters data with unlabeled input vectors nℜ , the vectors for SOSM are augmented by class labels. Nevertheless, it has been shown that SOFM possesses the same qualitative information as is found in SOSM [42]. Thus, to avoid the complexity of input labels, SOFM is used as the clustering algorithm in this investigation. After training, an SOFM is capable of clustering various inputs (from the flow profile) into different patterns. SOFM’s learning ability is realised by developing the relationship between input signal vectors nE ℜ∈and weight vectors nU ℜ∈ with the competitive learning algorithm. Here, focus is on the key learning algorithm that enables SOFM is to find the winning neuron je , which forms our target clustering area, as shown below:

dtdU

UU ijoldij

newij += while

∈−=

otherwiseNiue

dtdU ijjij

0)(α

(3)

whereα refers to the learning rate and N represents the clustering area of neuron i. The details of Kohonen’s SOFM algorithm can be found in [3][43].

6. Automatic traffic classification experiments

In order to illustrate the interactions among different components, this section will show the implementations of each component except the

ontology manager and presents related testing results as well. The experiments are conducted in a LAN environment. The testing includes two scenarios. The first scenario shows how to automatically yield an off-line IP flow profile map according to the results produced by the Ontology Clustering Engine. Then, another scenario will show the online automatic traffic classification functionalities.

6.1 Scenario 1: Producing an IP flow profile map

One traffic monitoring machine with Linux OS installed is attached to this LAN. Its network interface card is set to the promiscuous mode so as to capture all the traffic running on this LAN. Three typical unregistered network applications are running on the same network: RealPlayer, MediaPlayer and Pool Game3. The objective of this experiment is to show how to automatically produce the IP flow profile map for the three applications.

Network Context Sensors: The first task of sensors is to capture IP raw traffic and then aggregates them into different flows. Next, it needs to form various contexts that can be used by the Ontology Clustering Engine. The attributes CA of concepts are regarded as having the same nature as contexts. Therefore according to the statement of the IP flow profile ontology, related attributes, or say contexts for each concept, are shown below:

CsizepacketA _ ={incoming packet size, outgoing

packet size, size unit, maximum packet number}; C

timeervalA _int ={incoming packet gap, outgoing packet gap, time unit};

CtensityinflowA ._ = {the incoming sum of packet size,

the outgoing sum of packet size, flow id}; C

evolutionflowA _ ={packet length, packet gap, flow id};

CfpbdA = {incoming packet number, outgoing packet

number, flow id};

As SOFM is used as the clustering algorithm, contexts from sensors need to be transformed into the data format that can be used by SOFM. One typical method is to transform contexts into ratio values (typically between 0 and 1) so as to improve the clustering convergence ability. Table 2 gives seven

3 All the traffic is real data from BBC news, Yahoo entertainment

websites.


ratios, although the number of ratios can be much more. According to this table, context sensors will then produce the ratio value for the OCE. Table 3 shows the ratio context values (R1,2,6) from Realplay traffic.

TABLE 2. SEVEN INPUTS FOR A SOFM

ratio equation context type

R1=uplink_packet_length/1500 packet size

R2=downlink_packet_length/1500 packet size

R3=R1/(R1+R2) packet size

224

gaplengthlengthR

+= flow

evolution trend

R5=gap/max(0, gap) packet gap

nopktinnopktoutR

____6 =

fbpd

1500*/)_(7 NsizepacketRNi

i∈

= flow intensity

TABLE 3. REALPLAY TRAFFIC DATA (UDP)

R1 R2 R6 0.065333 0.712667 0.176471 0.066667 0.682667 0.232877 0.066000 0.676000 0.232877 0.066000 0.673333 0.224490 0.064667 0.680667 0.232877 0.066667 0.684000 0.241379 0.066000 0.687333 0.232877 0.064000 0.676667 0.224490 0.063333 0.678667 0.232877 0.066667 0.680667 0.232877 0.066667 0.640667 0.216216 0.064667 0.503333 0.168831 0.068667 0.506667 0.168831 0.066667 0.510000 0.176471 0.066667 0.510000 0.176471 0.067333 0.516000 0.172185

Ontology Clustering Engine (OCE): After receiving the Real Player data from context sensors, the OCE will start the training process to learn the flow profile. As it can be seen in Fig.6, in a small Kohonen’s 25x25 map, the results are not well clustered as they span various areas. After a series of tests on increasing size maps, we find out that an 80x80 map is able to yield more ideal results (fig.7). This is because that an 80x80 SOM can produce much larger network with 6,400

neurons and 19,200 weight connections (for three inputs, 6400x3=19,200), which leads to a better learning ability than that of 25x25 SOMs. Similarly, two further unregistered applications, Mediaplayer and Pool Games, were tested. Both of them can be readily distinguished with an 80x80 map (which is also an IP flow profile map). These results illustrate how the clustering results of IP flow profile data can be used to identify the relationship with traffic categories.

RealPlayer(24,16), (24,17)(22,14), (10, 0)

MediaPlayer(24,15)

Pool Games(24,0),(24,1(24,4)

Figure 6. The detection results with a 25x25 map

|-- ---- --- ---- ---- --- ---- ---- --- ---- ---- ---- --- ---- ---- --- ---- ---- --- ---- ---- --- -||-- ---- --- ---- ---- --- ---- ---- --- ---- ---- ---- --- ---- ---- --- ---- ---- --- ---- ---- --- -||-- ---- --- ---- ---- --- ---- ---- --- ---- ---- ---- --- ---- ---- --- ---- ---- --- ---- ---- --- -||-- ---- --- ---- ---- --- ---- ---- --- ---- ---- ---- --- ---- ---- --- ---- ---- --- ---- ---- --- -||-- ---- --- ---- ---- --- ---- ---- --- ---- ---- ---- --- ---- ---- --- ---- ---- --- ---- ---- --- -||-- ---- --- ---- ---- --- ---- ---- --- ---- ---- ---- --- ---- ---- --- ---- ---- --- ---- ---- --- -||-- ---- --- ---- ---- --- ---- ---- --- ---- ---- ---- --- ---- ---- --- ---- ---- --- ---- ---- --- -||-- ---- --- ---- ---- --- ---- ---- --- ---- ---- ---- --- ---- ---- --- ---- ---- --- ---- ---- --- -||-- ---- --- ---- ---- --- ---- ---- --- ---- ---- ---- --- ---- ---- --- ---- ---- --- ---- ---- --- -||-- ---- --- ---- ---- --- ---- ---- --- ---- ---- ---- --- ---- ---- --- ---- ---- --- ---- ---- --- -||-- ---- --- ---- ---- --- ---- ---- --- ---- ---- ---- --- ---- ---- --- ---- ---- --- ---- ---- --- -||-- ---- --- ---- ---- --- ---- ---- --- ---- ---- ---- --- ---- ---- --- ---- ---- --- ---- ---- --- -||-- ---- --- ---- ---- --- ---- ---- --- ---- ---- ---- --- ---- ---- --- ---- ---- --- ---- ---- --- -||-- ---- --- ---- ---- --- ---- ---- --- ---- ---- ---- --- ---- ---- --- ---- ---- --- ---- ---- --- -||-- ---- --- ---- ---- --- ---- ---- --- ---- ---- ---- --- ---- ---- --- ---- ---- --- ---- ---- --- -||-- ---- --- ---- ---- --- ---- ---- --- ---- ---- ---- --- ---- ---- --- ---- ---- --- ---- ---- --- -||-- -#-- --- ---- ---- --- ---- ---- --- ---- ---- ---- --- ---- ---- --- ---- ---- --- ---- ---- --- -||--#########------ --- ---- ---- --- ---- ---- ---- --- ---- ---- --- ---- ---- --- ---- ---- --- -||-- ---- -#----#####--- ---- ---- --- ---- ---- ---- --- ---- ---- --- ---- ---- --- ---- ---- --- -||-- ---- --- ---- ---- --- ---- ---- --- ---- ---- ---- --- ---- ---- --- ---- ---- --- ---- ---- --- -||-- ---- --- ---- ---- --- ---- ---- --- ---- ---- ---- --- ---- ---- --- ---- ---- --- ---- ---- --- -||-- ---- --- ---- ---- --- ---- ---- --- ---- ---- ---- --- ---- ---- --- ---- ---- --- ---- ---- --- -||-- ---- --- ---- ---- --- ---- ---- --- ---- ---- ---- --- ---- ---- --- ---- ---- --- ---- ---- --- -||-- ---- -###############------ --- ---- ---- ---- --- ---- ---- --- ---- ---- --- ---- ---- --- -||-- ---- -#----- ---- --- ---- ---- --- ---- ---- ---- --- ---- ---- --- ---- ---- --- ---- ---- --- -||-- ---- --- ---- ---- --- ---- ---- --- ---- ---- ---- --- ---- ---- --- ---- ---- --- ---- ---- --- -||-- ---- --- ---- ---- --- ---- ---- --- ---- ---- ---- --- ---- ---- --- ---- ---- --- ---- ---- --- -||-- ---- --- ---- ---- --- ---- ---- --- ---- ---- ---- --- ---- ---- --- ---- ---- --- ---- ---- --- -||-- ---- --- ---- ---- --- ---- ---- --- ---- ---- ---- --- ---- ---- --- ---- ---- --- ---- ---- --- -||-- ---- --- ---- ---- --- ---- ---- --- ---- ---- ---- --- ---- ---- --- ---- ---- --- ---- ---- --- -||-- ---- --- ---- ---- --- ---- ---- --- ---- ---- ---- --- ---- ---- --- ---- ---- --- ---- ---- --- -||-- ---- --- ---- ---- --- ---- ---- --- ---- ---- ---- --- ---- ---- --- ---- ---- --- ---- ---- --- -||-- ---- --- ---- ---- --- ---- ---- --- ---- ---- ---- --- ---- ---- --- ---- ---- --- ---- ---- --- -||-- ---- --- ---- ---- --- ---- ---- --- ---- ---- ---- --- ---- ---- --- ---- ---- --- ---- ---- --- -||-- ---- --- ---- ---- --- ---- ---- --- ---- ---- ---- --- ---- ---- --- ---- ---- --- ---- ---- --- -||-- ---- --- ---- ---- --- ---- ---- --- ---- ---- ---- --- ---- ---- --- ---- ---- --- ---- ---- --- -||-- ---- --- ---- ---- --- ---- ---- --- ---- ---- ---- --- ---- ---- --- ---- ---- --- ---- ---- --- -||-- ---- --- ---- ---- --- ---- ---- --- ---- ---- ---- --- ---- ---- --- ---- ---- --- ---- ---- --- -||-- ---- --- ---- ---- --- ---- ---- --- ---- ---- ---- --- ---- ---- --- ---- ---- --- ---- ---- --- -||-- ---- --- ---- ---- --- ---- ---- --- ---- ---- ---- --- ---- ---- --- ---- ---- --- ---- ---- --- -||-- ---- --- ---- ---- --- ---- ---- --- ---- ---- ---- --- ---- ---- --- ---- ---- --- ---- ---- --- -||-- ---- --- ----###############--- ---- ---- ---- --- ---- ---- --- ---- ---- --- ---- ---- --- -||-- ---- --- ---- -#-- --- ---- ---- --- ---- ---- ---- --- ---- ---- --- ---- ---- --- ---- ---- --- -||-- ---- --- ---- ---- --- ---- ---- --- ---- ---- ---- --- ---- ---- --- ---- ---- --- ---- ---- --- -||-- ---- --- ---- ---- --- ---- ---- --- ---- ---- ---- --- ---- ---- --- ---- ---- --- ---- ---- --- -||-- ---- --- ---- ---- --- ---- ---- --- ---- ---- ---- --- ---- ---- --- ---- ---- --- ---- ---- --- -||-- ---- --- ---- ---- --- ---- ---- --- ---- ---- ---- --- ---- ---- --- ---- ---- --- ---- ---- --- -||-- ---- --- ---- ---- --- ---- ---- --- ---- ---- ---- --- ---- ---- --- ---- ---- --- ---- ---- --- -||-- ---- --- ---- ---- --- ---- ---- --- ---- ---- ---- --- ---- ---- --- ---- ---- --- ---- ---- --- -||-- ---- --- ---- ---- --- ---- ---- --- ---- ---- ---- --- ---- ---- --- ---- ---- --- ---- ---- --- -|

Pool GamesArea

49

0 79

MediaPlayerArea

RealplayerArea

.

.

.

42

Figure 7. Three traffic categories in an 80x80 flow profile map


TABLE 4. THE ON-LINE DETECTION DATA WITH PACKET NUMBER THRESHOLD =20

no Source IP Destination IP protocol Source PORT

Destination PORT

Packet number

Packet length

1 80.28.18.230 146.87.48.126 UDP 47699 50736 2 94 2 218.49.72.77 146.87.48.126 UDP 52243 50736 1 47 3 146.87.48.126 80.28.18.230 UDP 50736 47699 3 141 4 146.87.48.126 218.49.72.77 UDP 50736 52243 1 139 5 81.48.224.123 146.87.48.126 UDP 46870 50736 2 94 6 146.87.48.126 81.48.224.123 UDP 50736 46870 1 139 7 80.197.152.61 146.87.48.126 UDP 7704 50736 1 47 8 146.87.48.126 80.197.152.61 UDP 50736 7704 2 278 9 212.58.224.64 146.87.49.240 UDP 60538 16411 20 21983 10 146.87.49.240 212.58.224.64 UDP 16411 60538 4 397

6.2 Scenario 2: online automatic traffic detection

Having identified an application’s flow profile, its IP flow profile map is sent to the Automatic Traffic Classifier component, which will perform the on-line traffic detection task, which is detailed as follows.

The Automatic Traffic Classifier (ATC): After receiving the profile clustering results for each flow, ATC will check if the received clustering results are already included in repository. This is realised by comparing clustering positions with those in the repository. If the clustering positions are not found, the detected traffic is tagged as a new type of traffic. The profile position data and its traffic category description (TRD) are stored in the repository with a new flow profile ID (identifier) that is automatically increased by one. However, as mentioned in Section 5 part A, the possibility exists that the clustering results of a particular flow profile overlap with other flow profiles. If this happens, the traffic has to be further classified with other detection solutions as stated in ontology (fig.2). This is realised by setting a series of bias values in addition to IP flow map. For example, P2P traffic can be roughly detected using the ‘IP connection number’. Subsequently, we set the flow profile map bias value as ‘P2PTRAFFIC’. Therefore, even if P2P traffic overlaps other non-P2P traffic, they can be told from bias values. Similarly, other bias values can be assigned with other detection concepts.

It should be noted that the on-line classification is very fast, it only needs a few seconds. Most of the on-line detection time is consumed waiting to produce fbpd ratios. In this experimental design, the waiting time relies on the threshold value of a flow’s packet number. In Table 4, the threshold of the packet number is preset to 20 and a group of ten UDP-based flows have been captured. Therefore, an fbpd ratio will be yielded once either one of the outgoing packet number or incoming packet number reaches 20. From our results (Table 4), it takes 3.74 seconds to get one fbpdvalue from the ninth flow, whose fbpd equals 0.20 (R1=0.066167, R2=0.732767 respectively, as determined by the data in Table 4). For this flow, the

on-line detector produces the responded neuron position of (16,2) which belongs to Realplayer’s area, as shown in Fig.8. As the threshold value is constant, the on-line detection will be faster when the detected traffic rate is higher. For each application, after proper training, the testing results show that an accuracy approaching 90%.

10

55

newtraffic?

YES

NO

IP flow ontology clustering repositoryflow profile 1: (21,0;20,15), [RealPlayer]flow profile 2: (21,6,29,24), [MediaPlayer]flow profile 3: (36,12;45,32),[PoolGames]

Figure 8. Online Automatic Traffic Classification Procedure

7. Future work In future, we are interested in applying the proposed

scheme into a wider network environment. Specifically, we aim to validate it in the project of Ambient Networks [4]. The Ambient Network (AN) project is aimed to foster co-operation between the next generation, heterogeneous wireless networks, in order to gather resources within and across ANs to provide new services [22]. One innovative design in the Ambient Network is to provide the service-aware transport functionalities, which is implemented within a set of service-specific overlays, which are called Service-aware Adaptive Transport Overlays (SATOs). To implement SATO functionalities, one of requirements concerns the interactions between overlay and underlay layers. For example, it is important to know what kind of services are running in ANs. Traffic category information is very useful to address this issue. Specifically, such a task can be realised by


Onode 1Onode 2

Onode4Onode3

AN #1 AN #2 AN #3AN #4

Overlay

Underlay

Path 1

Path 2

SuperPeer

SuperPeer

SuperPeer

SuperPeer Sensor

SensorSensor

Sensor

Congested

Figure 9. A simplified overlay and underlay network structure in ANs

mapping service classes according to traffic categories as stated in [15]. Then, according to different service classes, we can achieve the service adaptation functionality. For instance (Fig-9), when congestion happens on one overlay path like path 1 from Onode(Overlay node) 1 to Onode 4, the time-sensitive traffic detected by sensors will be rerouted to other better paths i.e. path 2. Currently, we propose deploying traffic classification sensors into super peers of each ambient network. After deriving service classes according to information from sensors, we can program service adaptation tasks into codes with high level languages. Then, the programmed services can be delivered automatically to overlay nodes via active networks like the DINA platform [23].

8. Conclusions This paper explores a new type of traffic

classification system with automatic identification capability arising from the investigation of a number of important and remained issues. First of all, the paper shows that, through the use of context and ontologies, the rising quantities of traffic detection knowledge can be effectively managed in a shared and reusable manner. Based on that, the skeleton of traffic classification ontology has been exploited by considering how to include existing traffic detection techniques and how to support the automatic detection feature. In addition, this paper shows that, by deriving traffic features, varying concepts can be used to model IP flow profile ontology. Furthermore it reveals that, the relationship between an IP flow profile and its traffic category, exhibits clustering features; Kohonen’s self-organising map is chosen to fulfil the clustering tasks. Based on two scenarios, the experiment results demonstrate two major gains. The

off-line testing suggests the clustering result of IP flow profile data is a reliable contender to identify its related traffic category. Moreover, the proposed traffic system is able to automatically and rapidly classify traffic.

In future, there are a number of issues that need to be further investigated. First of all, although this paper addresses the overlap issue, which can be resolved by using the flow profile bias technique, more studies needs to be carried out to better manage the flow profile bias values and to establish improved formats etc. In addition, since there is a possibility that an application traffic profile can be faked to interfere with the proposed pattern checking. This is an issue, which requires further work to be resolved. Last but not least, as discussed in section 7, how to deploy such a system and how to deploy traffic classification sensors into a large network (e.g. Ambient Networks) to improve network management and service adaptation still need to be investigated.

AcknowledgmentThis paper describes work partially undertaken in

the context of the Ambient Networks (Phase 2) – Information Society Technologies (IST) project, which is partially funded by the Commission of the European Union. We would like to thank our colleagues in the AN project for insightful comments that helped improve our work. We are also grateful to reviewers for their valuable comments.

References[1] T. Karagiannis, K. Papagiannaki, and M. Faloutsos. BLINC:

Multilevel Traffic Classification in the Dark. In ACM Sigcomm, Philadelphia, PA, August, 2005.

[2] Andrew W. Moore and Konstantina Papagiannaki, "Toward the Accurate Identification of Network Applications", in the Proceedings of Sixth Passive and Active Measurement Workshop (PAM 2005), March/April 2005, Boston, MA.

[3] T. Kohonen, Self-Organization and Associative Memory. Berlin: Springer-Verlag, 1989.

[4] EU-IST project 507134, Ambient Networks, http://www.ambient-networks.org .

[5] Fraleigh, C.,et al.: Packet-level traffic measurements from the sprint IP backbone. IEEE Network (2003).

[6] T. Karagiannis, A. Broido, N. Brownlee,Kc Claffy, and M. Faloutsos. Is p2p dying or just hiding? In Globecom, Dallas, TX, USA, November 2004.

[7] Sen, S.; Jia Wang ;Analyzing peer-to-peer traffic across large networks Networking, IEEE/ACM Transactions on, Volume 12, Issue 2, April 2004 Page(s):219 - 232.

[8] Matthew Roughan, Subhabrata Sen, Oliver Spatscheck, and Nick Duffield. Class-of-service mapping for qos: a statistical signature-based approach to ip traffic classification. In IMC ’04: Proceedings of the 4th ACM SIGCOMM conference. pages 135–148,New York, NY, USA, 2004. ACM Press.

[9] Christian Dewes, Arne Wichmann, and Anja Feldmann. Ananalysis of internet chat systems. In IMC '03: Proceedings


of the 3rd ACM SIGCOMM conference on Internet measurement, pages 51.64, 2003.

[10] Parish, D.J., Bharadia, K., Larkum, A., Phillips, I.W. and Oliver, M., ''Using packet size distributions to identify real-time networked applications'' , IEE Proc. commun. , 1504 , August 2003, pp 221-227, ISSN 1350-2425.

[11] Andrew W. Moore and Denis Zuev, "Internet Traffic Classification Using Bayesian Analysis Techniques" in the Proceedings of the ACM SIGMETRICS June 2005, Banff, Canada.

[12] Denis Zuev and Andrew W. Moore, "Traffic Classification using a Statistical Approach", in the Proceedings of Sixth Passive and Active Measurement Workshop (PAM 2005), March/April 2005, Boston, MA.

[13] A. McGregor, M. Hall, P. Lorier, and J. Brunskill. Flow Clustering Using Machine Learning Techniques. In PAM, 2004.

[14] Effrey Erman, Martin Arlitt, Anirban Mahanti, Traffic Classification Using Clustering Algorithms, SIGCOMM’06 Workshops September 11-15, 2006, Pisa, Italy.

[15] M. Roughan, S. Sen, O. Spatscheck, and N. Duffield. Class-of-Service Mapping for QoS: A Statistical Signature-based Approach to IP Traffic Classification. In ACM/SIGCOMM IMC, November 2004.

[16] L. Bernaille, R. Teuxeira, I. Akodkenou, A. Soule, K. Salamatian, “Traffic Classification on the Fly”, ACM SIGCOMM Computer Communication Review, vol. 36, no. 2, April 2006.

[17] S. Zander, T.T.T. Nguyen, G. Armitage, "Automated Traffic Classification and Application Identification using Machine Learning", Proceedings of IEEE 30th Conference on Local Computer Networks, Sydney, Australia, 15-17 November 2005.

[18] A. McGregor, M. Hall, P. Lorier, J. Brunskill, “Flow Clustering Using Machine Learning Techniques”, Passive & Active Measurement Workshop 2004 (PAM 2004), France, April 19-20, 2004. .

[19] Fivos Constantinou and Panayiotis Mavrommatis,Identifying Known and Unknown P2P Traffic. To appear in theproceedings of The 5th IEEE International Symposium on Network Computing and Applications(IEEE NCA06).

[20] S. Zander, T.T.T. Nguyen, G. Armitage, "Self-learning IP Traffic Classification based on Statistical Flow Characteristics", Passive and Active Measurement Workshop , Boston, USA, March / April 2005.

[21] S Zander, T Nguyen, G Armitage, “A Self-learning System for P2P Traffic Classification”, Proc. of the 6th Passive and Active Network Measurement, 2005.

[22] Galis, A. L.,C K. Jean, et al. "Service-aware Overlay Adaptation in Ambient Networks", International Multi-Conference on Computing in the Global Information Technology (ICCGI) 2006.

[23] Galis, A., Denazis, S., Brou, C., Klein, C. (ed) – “ Programmable Networks for IP Service Deployment“ ISBN 1-58053-745-6; pp450, June 2004; Artech House Books; www.artechhouse.com;

[24] Freedman, J., “The What, Who, Where, When, Why and How of Context-Awareness”, GVU Technical Report, CHI Workshop, work shop 11, 2000.

[25] Dey, A. K., “Understanding and using context, Journal of Personal and Ubiquitous Computing”, Volume 5 (1), pp. 4-7, 2001.

[26] Schmidt, A., Beigl, M., Gellersen, H.W., “There is more to context than location”, Computers & Graphics Journal, Elsevier, Volume 23, No. 6, December 1999.

[27] Schmidt, A., K. van Laerhoven, K., “How to Build Smart Appliances?”, IEEE Personal Communications 8(4), August 2001.

[28] T. Berners-Lee, J. Hendler, and O. Lassila, “The Semantic Web, ” Scientific American, 2001.

[29] Roel Ocampo , Alex Galis , Chris Todd ,Hermann De Meer, “Towards Context-Based Flow Classification”, International Conference on Autonomic and Autonomous Systems , Silicon Valley, USA, July 19-21, 2006.

[30] T.R. Gruber. Towards Principles for the Design of Ontologies Used for Knowledge Sharing. In Roberto Poli Nicola Guarino, editor, International Workshop on Formal Ontology, Padova, Italy, 1993.

[31] Silvana Castano, Alfio Ferrara, Stefano Montanelli, Elena Pagani, Gian Paolo Rossi, ”Ontology-Addressable Contents in P2P Networks”, Proc. 1st Workshop on Semantics in Peer-to-Peer and Grid Computing (SemPGRID'03), Budapest, May 20 2003, pp. 55-68.

[32] Mario Schlosser, Michael Sintek, Stefan Decker, Wolfgang Nejdl , A Scalable and Ontology-Based P2P Infrastructure for Semantic Web Services, Proceedings of the Second International Conference on Peer-to-Peer Computing, 2002.

[33] T. Gruber. “A Translation Approach to Portable Ontology Specification,” Knowledge Acquisition, 5(2), 1993.

[34] Qian Zhao Yu Zhou Perry, M., “Agreement-aware Semantic Management of Services”, Autonomic and Autonomous Systems, 2006. ICAS '06. 2006 International Conference on, 19-21 July 2006.

[35] Christoph Schmitz and Andreas Hotho and Robert Jäschke and Gerd Stumme. Content Aggregation on Knowledge Bases using Graph Clustering. In York Sure and John Domingue, editor(s), The Semantic Web: Research and Applications, (4011):530-544, Springer, Heidelberg, 2006.

[36] Liu Yao Sui Zhifang Chen Xuefei , On Method and Automatic Construction Theory of Domain Ontology Based on Depended Text, Innovative Computing, Information and Control, 2006. ICICIC '06. First International Conference on, Volume: 2, 63- 630-01 Aug. 2006.

[37] R. Stevens, C.A. Goble, and S. Bechhofer. Ontology-based Knowledge Representation for Bioinformatics. Briefings in Bioinformatics, 1(4):398–416, November 2000.

[38] Eun-Jung Ko Hyung-Jik Lee Jeon-Woo Lee, 20-22 Feb. 2006, “Ontology-Based Context-Aware Service Engine for U-HealthCare”, Advanced Communication Technology, 2006. ICACT 2006. The 8th International Conference.

[39] Naing, M.-M. Ee-Peng Lim Dion Goh Hoe-Lian , Ontology-based web annotation framework for hyperlink structures, Web Information Systems Engineering (Workshops), 2002. Proceedings of the Third International Conference on, page(s): 184- 193.

[40] M. Ehrig, S. Handschuh, A. Hotho, et al. Kaon, “Towards a large scale semantic web”. In Proc. 3rd Intl Conf, E-Commerce and Web Technologiese, Aix-en-Provence, Sep. 2002.

[41] H. Ritter and T. Kohonen, “Self-organizing semantic maps,” Biol.Cybern., vol. 61, pp. 241-254, 1989.

[42] Bezdek, J.C. Pal, N.R. , A note on self-organizing semantic maps, Neural Networks, IEEE Transactions on, Publication Date: Sep 1995,Volume: 6, Issue: 5, On page(s): 1029-1036 , ISSN: 1045-9227.

[43] Dayhoff, J., E., 1990, “Neural Network Architectures”, Van Nostrand Reihold, New York, ISBN: 0-442-20744-1.


Documents

Towards Automatic Traffic Classification