Qosmos ixEngine Technology Overview

Enabling True Network Intelligence Everywhere

Qosmos ixEngine Technology

Overview

September 2009

Qosmos ixEngine v5 Technical Overview

Qosmos Network Intelligence Overview

ixEngine Components

Building block approach

Features

Extracted information = Attributes

Traffic metadata & Content

Families of traffic metadata

Protocol Plugin Creator

Introducing Qosmos ixEngine

Technical foundations

The protocol graph core of network intelligence

The Qosmos protocol path

Session signature

Qosmos Application stream

Session correlation

Session drill down (structured attributes)

Dynamic session state

Industry Leading Technology

Protocol plugin independence

Integration in 3rd party system

Functional architecture

Software architecture

Implementation

Architecture

Key Technological Differentiators

Summary

Page 2

Introducing Network Intelligence

Technology

Page 3


What is Network Intelligence Technology?

Mining network

data & events

Delivering

actionable data

Reporting

Network Intelligence

Technology

Application Leveraging


Technology

01010110101010010101011010101001010101101010100110010101011

0101010011001010101101010100101010110101010010101011010101

00110010101011010101001100101010110101010011001010101101010

Action

Page 4


Classifies traffic flows. Traffic flows can be either protocols (HTTP) or application (webmail)

Qosmos Sessionizer™ correlates sessions in order to provide full understanding of each application (rather than just flow understanding)

Extracts in real time data embedded in or computed from the traffic. This Information called Attributes can be either

Metadata

Content

Structures and selectively delivers extracted information in a format ready to be used by 3rd party systems

Filters traffic based on identified context

Network Intelligence Technology Features

Page 5

Technology foundations

The protocol graph: core of network intelligenceThe Qosmos protocol path

Session signature

Qosmos Application stream

Session correlation

Session drill down(Structured attributes)

Dynamic session state

Industry Leading Technology

Page 6


Phase 1: Traffic cleaning

On the fly packet reordering, de-duplication of packets, defragmentation of

packets

Phase 2: Classification

NO use of TCP/UDP ports for classification: protocols identification based on

syntax and semantic analysis

Dynamic parsing of flows according to protocol grammar (unlike static pattern

matching)

De-capsulation of tunneled/encapsulated traffic

Phase 3: Information extraction

Metadata and content extraction and organization into a hierarchical structure

Correlation of information across sessions

Qosmos Network Intelligence Technology Foundations

Page 7


The Protocol Graph: Core of Network Intelligence

Technology

ETH

802.1

IP IPv6

UDP TCPRADIUS

GTP DNS

SIPRTP HTTP

SMTPMSN

GOOGLE GMAIL

Session correlation

Protocol layering

Page 8


The Protocol Graph: Core of Network Intelligence

Technology

ETH

802.1

IP IPv6

UDP TCPRADIUS

GTP DNS

SIPRTP HTTP

SMTPMSN

GOOGLE GMAILGoogle Query:

« restaurant in NY »

Traditional flow analysis

Qosmos Network Intelligence

Page 9


Traditional flow analysis

Traditional 5 tuple session signature

Only pattern matching and port analysis

Qosmos Network Intelligence

Qosmos Ntuple session signature

Syntax and semantic analysis

Qosmos Ntuple Session Signature

@IP

Source@IP Dest

IP

Protocol

Port

SourcePort Dest.

192.168.2.1 192.168.3.2 UDP 3386 3386

Page 10

@IP

Source@IP Dest

IP

Protocol

Port

SourcePort Dest.

192.168.2.1 192.168.3.2 UDP 3386 3386

GTP TeID

50510


Application

Qosmos Sessionizer™

Qosmos Sessionizer

Authentification sessionFlow

Flow

Chat sessionFlow

Flow

Speech sessionSIP Flow

RTP Flow

Qosmos Sessionizer

Qosmos Sessionizer

Qosmos Sessionizer

Page 11


Application


Authentification sessionFlow

Flow

Chat sessionFlow

Flow

Speech sessionSIP Flow

RTP Flow

Qosmos Sessionizer

Qosmos Sessionizer

Qosmos Sessionizer

Traffic

Attributes flow level

Traffic

Attributes session level

Traffic

Attributes application level

Page 12


Nature of Extracted Information


Traffic Classification

Information Extraction

Metadata

Content

Page 13


Metadata:

Structured Information used to generate traffic records

Attributes = Metadata & Content

Session ID Email # Type of attached doc Sender

887765 3 MS WORD [email protected]

86554 1 [email protected]

… … … …

Example of traffic record

Content data :

Data used to recreate a file associated with an application

Data necessary to recreate an email

Data necessary to recreate an attached document

RTP data to recreate VoIP stream

Page 14


Session Correlation (1/3)

RTP

SIP

SBC

Caller

Qosmos

Callee

Page 15



SIPRTP

ETH

802.1

IP IPv6

UDP TCPRADIUS

GTP DNS

HTTP

SMTPMSN

GOOGLE GMAIL

Session correlation

RTP Traffic


SIP Traffic


Page 16



RTP Traffic Attributes

Content Metadata

Attribute Value

Codec G711

Jitter 0.04 ms

SIP Caller 00154853543

SIP Callee 00184285629

SIP Traffic Attributes

Metadata

Attribute Value

SIP Caller 00154853543

SIP Callee 00184285629

Inherited attributes

Correlation key

Correlation key

Page 17


Structured Delivery of Extracted Information

Correlation CorrelationChat

session

Metadata

& Content

Structured

Delivery

Speech

session

Metadata

& Content

Structured

Delivery

Structured

Delivery

Metadata

& Content

Authentication

session

Page 18


Advanced Filtering

Information Extraction

Metadata Content

•Filtering based on

metadata and

content value

•Matching rules

apply to lists of

values

•Black list/white list

support

Structured Delivery

Filter

State

01010110101010010101011010101001010101101010100110010101011

01010100110010101011010101001010101101010100101010110101010

0110010101011010101001100101010110101010011001010101101010

User ApplicationApply filter

Page 19


Delivery: Structured Attributes

SMTP Session #1

Email #1

Sender: [email protected]

Receiver: [email protected]

Content: “Hello Paula, blabla bla bla…”

Email #2

SMTP Session #2

Page 20

ixEngine Components

Building block approach

Features

Extracted information = Attributes

Traffic metadata & Content

Families of traffic metadata

Protocol Plugin Creator

Introducing Qosmos ixEngine

Page 21


Network Intelligence enabled application with embedded

Qosmos ixE

01010110101010010101011010101001010101101010100110010101011

0101010011001010101101010100101010110101010010101011010101

00110010101011010101001100101010110101010011001010101101010

Information

Extraction

Classification

Delivery

Attributes

Forensics Data Mining

Smart Traffic

Management

User Application

Page 22


ixEngine Components

Protocol Plugin API


enabled application

Engineer

Runtime

Runtime

Runtime

Runtime

Page 23


Develop your own customized protocol and application plugins

Plugin API

Your customized

Protocol Plugin

http.plugin

gmail.plugin

facebook.plugin

Plugin API

WWW

Page 24

Plugin independence

Integration in third party system

Functional architecture

Software architecture

Implementation

ixEngine Architecture

Page 25


Plugin Independence

TCP.plugin

HTTP.plugin

gmail.plugin

0101011010101001010101101010100101010110101010011001010101101010100110010

1010110101010010101011010101001010101101010100110010101011010101001100101

01011010101001100101010110101010101001100101011010100110010101101010011001

IP.plugin

ETH.plugin

Dyn

am

ic C

on

so

lid

ati

on

User agent

@IP Source

@ IP Dest

facebook.pluginsender

receiver

WWW

TCP

IP

ETH

Page 26


Functional Architecture (1/2)

01010110101010010101011010101001010101101010100110010101011

0101010011001010101101010100101010110101010010101011010101

00110010101011010101001100101010110101010011001010101101010

Control API

Session

Reassembly

(switchable)Classification

Metadata + Content Extraction

Data API

Packet input Packet output

Data (Metadata & Content)

Traffic Information Management

Traffic Shaping

Policy

Page 27

Qosmos sessionizer™


Functional Architecture (2/2)

Extraction of protocol and applications metadata

Extraction of content

Metadata + Content extraction

Connection with target application

Makes available data extracted by the ixEngine

Data API

Allows to control how ixEngine runs,

and how Network Intelligence is configured

Control API

Reassembly of fragmented, duplicated, de-sequensed

packets into a session

Switchable process

Session reassembly process

Identification and classification of protocols and

applications based on syntax and semantic analysis

Classification

Page 28


Software Architecture (1/2)

01010110101010010101011010101001010101101010100110010101011

0101010011001010101101010100101010110101010010101011010101

00110010101011010101001100101010110101010011001010101101010

Session

Reassembly



Data API

Packet input Packet output

Data (Metadata & Content)

Traffic Information Management

Traffic Shaping

Policy

LibCTL (control) LibFilter

LibAFC

LibData

Page 29

Protocol

Plugin

Library

Protocol

Plugin API

Software Architecture (2/2)

Reassembly process : switchable process that reordonates

fragmented, duplicated, de-sequensed packets

LibAFC does protocol discovery, event and content

extraction, session taging

LibAFC

Library used to define filters. Filters will detect when a

session corresponds to a specific trigger for session and

packet taging

LibFilter

Library used to control the LibAFC

Also used to implement filters in the processing path

LibCTL

Library used to turn binary data produced by libAFC into

alpha numeric data

Libdata is used by the Data API

LibData

Page 30


User Application

Data Handler

Integration of ixEngine in 3rd Party System

0101011010101001010101101010100101010110101010011001010101101010

1001100101010110101010010101011010101001010101101010100110010101

0110101010011001010101101010100110010101011010101010101010101101

Session

Reassembly



Data API

LibCTL (control) LibFilter

LibAFC

LibData

PCAPpacket capture

PCAPinitialisation

File

system

Database

PCAPpacket output

Packet

Session

context

ContentMetadata

Rules

setting

Linux user mode

Page 31


User or kernel mode

Multi-threading support (SMP)

Linux Stand Base 3.x support

Market leading environments supported

List of available platformsStandard platforms

x86_32 Linux User Mode

x86_32 Linux Kernel Mode

x86_64 Linux User Mode

Freescale PowerQUICC/Linux User mode

Freescale 8572/Linux User mode

High performance platformsRMI XLR 7xx / RMI OS

Cavium Octeon / Simple Executive

Tilera TILEPro64

Implementation

Page 32


Summary: Key Technological Differentiators

What What we do best Why it is important

Parsing of flows based

protocol grammar

More than just pattern matching, ixEngine

decodes the full grammar of each protocol

To identify session events and provide

structured information

To avoid false positives

Qosmos Sessionizer™The process tracks each session from

beginning to end, to fully understand usage

of application per user

To understand usages of application

involving correlated/inherited sessions (VoIP

= RTP + SIP)

De-capsulate

encapsulated or

tunneled traffic

We handle major tunneling protocols such

as GTP, GRE, L2TP and many others VJC,

IPv6CP, HTTP

To retain full visibility even when traffic and

applications are encapsulated inside tunnels

Extraction of information

and data from traffic

When an application is identified, the

system extracts all session information

(caller, name of downloaded file etc)

To have a precise vision of usages

To save on storage space (no need to store

entire traffic)

“Database” vision of the

network

Session events information is available in a

database format

Configurable data structure

Easy to use network information to build

powerful solutions

Ability to keep historical vision of all session

events

Session context tagging

with events information

Dynamically enriches session context with

observed events

Ability for developers to use this session

context for other purposes

All intelligence is available to manage

packets (e.g. for intelligent firewalls)

Create customized

intelligence

Unique ability to create your own specific

protocols

Ability to configure Network Intelligence

mechanisms

To fit specific requirements (regional or

custom protocols)

To provide solution vendors with flexible

building blocks

Page 33

Documents

Qosmos ixEngine Technology Overview