The Management The Management Infrastructure of a Network Infrastructure of a Network

Measurement System for Measurement System for QoSQoS ParametersParameters

Alexandru Bikfalvi, Paul PaAlexandru Bikfalvi, Paul Patratrass, , CristianCristian MihaiMihai VanceaVancea, Virgil , Virgil DobrotaDobrota

Technical University of Technical University of ClujCluj--NapocaNapoca, Romania, RomaniaCommunications DepartmentCommunications Department

2626--28 Baritiu Street, 400027 Cluj28 Baritiu Street, 400027 Cluj--Napoca, Romania, Tel/Fax: +40Napoca, Romania, Tel/Fax: +40--264264--597083597083EE--mail: mail: Virgil.Dobrota@com.utcluj.roVirgil.Dobrota@com.utcluj.ro

2

AgendaAgenda

•• Scenario based on existing toolsScenario based on existing tools•• Design of the management infrastructureDesign of the management infrastructure•• Experimental resultsExperimental results•• Conclusions and future workConclusions and future work

3

•• Vendor specific applications Vendor specific applications for specific platformsfor specific platforms

•• Not all applications are QoS Not all applications are QoS capablecapable

•• Some require complex Some require complex commands, scripts, procedural commands, scripts, procedural stepssteps

•• Manual measurement control Manual measurement control (especially on different tools)(especially on different tools)

•• Additional software and Additional software and computations to obtain computations to obtain targeted resultstargeted results

Existing Tools ScenariosExisting Tools Scenarios

Measurement Measurement ApplicationsApplications

Tested NetworkTested Network

4

Our Software SolutionOur Software Solution

Tested NetworkTested Network

Network Network Measurement Measurement SystemSystem

Management Management ConsoleConsole

Measurement Measurement AgentsAgents

Management StationManagement Station

ConsoleConsole•• Straightforward user interfaceStraightforward user interface•• User focuses on scenarios, topology, User focuses on scenarios, topology,

test objectivestest objectives•• Runtime operations performed by Runtime operations performed by

the computerthe computer•• Measured QoS parameters available Measured QoS parameters available

automaticallyautomaticallyInfrastructureInfrastructure•• Software module used by both Software module used by both

applicationsapplications•• Measurement information Measurement information

communicationcommunication•• Accomplishes the task using a Accomplishes the task using a

management protocolmanagement protocol

5

Infrastructure DesignInfrastructure Design

NMS NMS ManagementManagement

Structural LogicStructural Logic

The user interacts with these The user interacts with these items to perform a items to perform a measurementmeasurement

Organizes measurement Organizes measurement operations as a set of operations as a set of abstract itemsabstract items

SessionsSessionsGroupsGroupsTasksTasks

Management MessagesManagement MessagesIdentification messages relay information about Identification messages relay information about agentsagents’’ capabilitiescapabilitiesTask control messages request agents to start a Task control messages request agents to start a measurement taskmeasurement taskData messages are used to collect QoS results from a Data messages are used to collect QoS results from a set of agentsset of agents

6

Working ExampleWorking ExampleDefining Sessions •• Agent (s) involvedAgent (s) involved

•• Generation / AnalysisGeneration / Analysis•• Traffic typeTraffic type•• Packet distributionPacket distribution•• Protocol specificProtocol specific•• Desired resultsDesired results

Session 1Session 1Agent 1 Agent 1 ►► Agent 3Agent 3

Session 2Session 2Agent 2 Agent 2 ►► Agent 4Agent 4

Agent 1Agent 1

Agent 2Agent 2Agent 3Agent 3

Agent 4Agent 4

Task 1Task 1Session 2Session 2 @ 1 o@ 1 o’’clockclock

Task 2Task 2Session 1Session 1 @ 3 o@ 3 o’’clockclock

(pending)(pending)

(pending)(pending)

(running)(running)(finished)(finished)

(running)(running)

QoS

Par

amet

er

TimeTime

Management StationManagement Station

Scheduling and Running Tasks

CollectingResults

7

Software ArchitectureSoftware Architecture

Measurement Measurement ServicesServices

Communication Communication ServicesServices

Management Management ConsoleConsole

Management Management ConsoleConsole

Measurement Measurement AgentAgent

User InterfaceUser Interface

Session Session ManagerManager

Queuing Queuing ServiceService

SNMP ServiceSNMP Service

Management Management ServiceService

•• Management consoleManagement console’’s QoS measurement s QoS measurement intelligenceintelligence

•• Enables user control through various Enables user control through various windows and dialogswindows and dialogs

•• Allows creation of measurement sessions, Allows creation of measurement sessions, groups and tasksgroups and tasks

•• Runs the measurement task; collects and Runs the measurement task; collects and stores resultsstores results

•• Synchronous service for asynchronous Synchronous service for asynchronous messagesmessages

•• Message retransmissions, duplicates Message retransmissions, duplicates deletion, timeoutdeletion, timeout

•• Encapsulation of messages to SNMP PDUsEncapsulation of messages to SNMP PDUs

•• SNMP data verification, buffer overflow SNMP data verification, buffer overflow checking checking

•• A thread for each assigned local network A thread for each assigned local network interfaceinterface

•• Socket I/O operations with SNMP dataSocket I/O operations with SNMP data

8

Management ConsoleManagement Console

Session Manager

The primary application that enables the users to The primary application that enables the users to perform QoS measurementsperform QoS measurements

GUIGUIGUI Task SchedulingTask SchedulingTask Scheduling

Queuing ServiceQueuing ServiceQueuing Service

SNMP ServiceSNMP ServiceSNMP Service Management Service

Management Management ServiceService

Service Control ManagerService Control ManagerService Control Manager

Configuration ServiceConfiguration ServiceConfiguration Service

Har

dwar

e M

anag

erH

ardw

are

Man

ager

Har

dwar

e M

anag

er

Message WrapperMessage WrapperMessage Wrapper

9

Management ServiceManagement Service

•• Basic networking connectivity using UDP transport Basic networking connectivity using UDP transport •• Interface with SNMP service for message encapsulationInterface with SNMP service for message encapsulation•• Interface with queuing service for message distributionInterface with queuing service for message distribution•• Security functions (IPSecurity functions (IP--based filtering)based filtering)

Management Service

Management Management ServiceService

Queuing ServiceQueuing ServiceQueuing Service

OS Socket InterfaceOS Socket InterfaceOS Socket Interface

SNMP ServiceSNMP ServiceSNMP Service

Service Control ManagerService Control ManagerService Control Manager

Service Control ThreadService Control ThreadService Control Thread

Service Control ThreadService Control ThreadService Control Thread

Service Data ThreadService Data ThreadService Data Thread

Control/StatusControl/Status

10

Queuing ServiceQueuing Service

• Temporary storage for management messages awaiting processing

• 4 inbound and 4 outbound priority based FIFO queues

• Message retransmission• Duplicate message

removal• Queue recycling

Queuing ServiceQueuing ServiceQueuing Service

Management ServiceManagement ServiceManagement Service

Session ManagerSession ManagerSession Manager

DispatcherDispatcherDispatcher

DispatcherDispatcherDispatcherServerServerServer

ServerServerServer

Que

ueQ

ueue

Que

ue

Que

ueQ

ueue

Add

ition

al

Serv

ices

Add

ition

al

Add

ition

al

Serv

ices

Serv

ices

11

Session ManagerSession Manager

•• Keeps track on measurements to performKeeps track on measurements to perform•• Generates and listens for related messagesGenerates and listens for related messages•• Discriminates between incoming repliesDiscriminates between incoming replies•• Computes final results and presents them to the userComputes final results and presents them to the user

Contains the measurement related intelligenceContains the measurement related intelligence

SessionsSessions

GroupsGroups

TasksTasks

What to doWhat to do

When to doWhen to do Mea

sure

men

t M

easu

rem

ent

Dat

aD

ata

12

Management ConfigurationManagement Configuration•• The first step after NMS software installationThe first step after NMS software installation•• The user is notified the configuration is absolutely required (aThe user is notified the configuration is absolutely required (all ll

other features are not available)other features are not available)•• Configuration information: manager identification, Configuration information: manager identification,

networking, SNMP and securitynetworking, SNMP and security

Enables manager recognition in Enables manager recognition in multimulti--managing topologiesmanaging topologiesSelection of primary network Selection of primary network interface on multihomed hostsinterface on multihomed hosts

Additional interfaces are Additional interfaces are selectable later onselectable later on

SNMP community names and SNMP community names and associated permissionsassociated permissions

SNMP runs only on selected SNMP runs only on selected network interfacesnetwork interfaces

Internet layer filtering to Internet layer filtering to compensate the lack of security compensate the lack of security for SNMPv1for SNMPv1

13

Services ConfigurationServices Configuration•• Software routines that execute under the same process as the Software routines that execute under the same process as the

management consolemanagement console•• Some have their own threads, other run under the context of Some have their own threads, other run under the context of

other service threadsother service threads

Service Control Manager

Service control: start, stop, pauseService control: start, stop, pause

Service recoveryService recoveryService dependenciesService dependenciesService parametersService parameters

Configuration ServiceConfiguration ServiceConfiguration Service

Session ManagerSession ManagerSession Manager

Queuing ServiceQueuing ServiceQueuing Service

SNMP ServiceSNMP ServiceSNMP Service

Management ServiceManagement ServiceManagement Service

Service startup typeService startup type

14

Agents RegistrationAgents Registration

• Validates an SNMP entity as a NMS measurement agent

UserUser BackgroundBackground•• Identification managed objects Identification managed objects

test the SNMP entity roletest the SNMP entity role•• Advanced information on agent Advanced information on agent

station hardware is collectedstation hardware is collected

•• Agent information is stored and Agent information is stored and updated periodicallyupdated periodically

•• The maximum number of agents The maximum number of agents is 65536is 65536

•• The maximum number of The maximum number of simultaneous tests per agent from simultaneous tests per agent from all managers is 256all managers is 256

15

Create New Session WizardCreate New Session Wizard

SessionSession Session Session GroupGroup TaskTask

Performing MeasurementsPerforming Measurements

New Scheduled Task DialogNew Scheduled Task Dialog

New Session Group DialogNew Session Group Dialog

Create a measurement sessionCreate a measurement sessionSpecify session parametersSpecify session parametersSession is saved but not executedSession is saved but not executedRunning multiple session at onceRunning multiple session at onceGroup of independent sessionsGroup of independent sessionsParametric groupParametric groupCreate a new pending taskCreate a new pending taskSession Manager is now in chargeSession Manager is now in chargeTask may be deleted while pendingTask may be deleted while pending

16

Results InspectionResults InspectionD

epen

denc

ies

Dep

ende

ncie

s

Task TypeTask TypeTask Type

Session Type

Session Session TypeType

SingleSingleSingle

GroupGroupGroup

GenerateGenerateGenerate

AnalyzeAnalyzeAnalyze

BothBothBoth

SummarySummarySummary

Last DataLast DataLast Data

At a glance

TimeTimeTime

Dat

a Fo

rmat

ting

Dat

a Fo

rmat

ting

ParametricParametricParametric

TableTableTable

PlotPlotPlot

ResultsResults

17

Experimental ScenarioExperimental Scenario

Chenin Chenin (management)(management)

Cisco Catalyst Cisco Catalyst 29502950

EnologaEnologa

1Gbps NIC1Gbps NIC

MacabeuMacabeu

ObjectivesObjectives•• ThroughputThroughput

•• Packet delay variationPacket delay variation

•• Packet lossPacket loss

1000 Base SX1000 Base SX

ParametersParameters

•• Frame rate: 200 to 100,000 ppsFrame rate: 200 to 100,000 pps

•• Frame size: 1500 bytesFrame size: 1500 bytes

•• Test duration: 100 secondsTest duration: 100 seconds

•• Packet distribution: periodic and Packet distribution: periodic and PoissonPoisson

•• Evaluate PC performances for traffic generation at Evaluate PC performances for traffic generation at gigabit per second data ratesgigabit per second data rates

Measurement Measurement Endace DAGEndace DAG

CellerCeller

10.198.0.10/2910.198.0.10/29

10.198.0.2/2910.198.0.2/29

147.83.130.175/25147.83.130.175/25

147.83.130.31/25147.83.130.31/2510.198.0.12/2910.198.0.12/29

147.83.130.21/25147.83.130.21/25

•• Traffic type: IEEE 802.3zTraffic type: IEEE 802.3z

18

Example of ResultsExample of Results

•• Series of tests vs. packet rate at 1500 byte framesSeries of tests vs. packet rate at 1500 byte frames

0

1

2

3

4

5

6

7

0 20 40 60 80 100Packet Rate [103 pps]

Num

ber o

f Pac

kets

[106 p

acke

ts]

Tx Packets

Rx Packets

•• Packet countPacket count•• ThroughputThroughput•• Delay variationDelay variation•• Packet loss ratioPacket loss ratio

6 million 6 million packets per packets per 100 seconds 100 seconds testtest 0

100

200

300

400

500

600

700

800

0 20 40 60 80 100Packet Rate [103 pps]

Thro

ughp

ut [M

bps]

Throughput (Mbps)

Almost 700 Almost 700 MbpsMbps

-5-4-3-2-1012345

0 20 40 60 80 100

Packet Rate [103 pps]

Del

ay [m

s]

Avg Delay Variation (ms)Min Delay Variation (ms)Max Delay Variation (ms)

Up to 4 Up to 4 millisecondsmilliseconds

-1

0

1

2

3

4

5

6

7

0 20 40 60 80 100

Packet rate [103 pps]

Pack

et L

oss R

atio

[%]

Packet Loss Ratio (%)

Up to 6.5 Up to 6.5 percentpercent

•• RealReal--time failuretime failure

-1

0

1

2

3

4

5

6

0 20 40 60 80 100

Packet Rate [106 pps]

Real

-tim

e Fa

ilure

s [10

6 tim

es]

Real-time Failure (times)

19

Comparison of Results Comparison of Results • Throughput (Mbps) vs. desired packet rate obtained for different NICs

256

byte

fram

es25

6 by

te fr

ames

1500

byt

e fr

ames

1500

byt

e fr

ames

Intel PRO 1000 VEIntel PRO 1000 VE SysKonnect SK 9843SysKonnect SK 9843

0

50

100

150

200

250

0 100000 200000 300000 400000

Throughput (Mbps)

0

100

200

300

400

500

600

700

800

0 20000 40000 60000 80000 100000

Throughput (Mbps)

0

50

100

150

200

250

0 100000 200000 300000 400000

Throughput (Mbps)

0

100

200

300

400

500

600

700

800

0 20000 40000 60000 80000 100000

Throughput (Mbps)

20

Comparison of Results Comparison of Results • Examining the maximum achieved packet rate vs. desired packet rate

256

byte

fram

es25

6 by

te fr

ames

1500

byt

e fr

ames

1500

byt

e fr

ames

Intel PRO 1000 VEIntel PRO 1000 VE SysKonnect SK 9843SysKonnect SK 9843

0

2000000

4000000

6000000

8000000

10000000

12000000

0 100000 200000 300000 400000

Tx Packets

Rx Packets

0

1000000

2000000

3000000

4000000

5000000

6000000

7000000

0 20000 40000 60000 80000 100000

Tx Packets

Rx Packets

0

2000000

4000000

6000000

8000000

10000000

12000000

0 100000 200000 300000 400000

Tx Packets

Rx Packets

0

1000000

2000000

3000000

4000000

5000000

6000000

7000000

0 20000 40000 60000 80000 100000

Tx Packets

Rx Packets

Maximum 60-100 kpps (depending on packet size)

Maximum 60Maximum 60--100 kpps 100 kpps (depending on packet size)(depending on packet size)

Maximum 40 kpps (regardless of packet size)

Maximum 40 kpps Maximum 40 kpps (regardless of packet size)(regardless of packet size)

• Throughput limitation is due to packet rate limitation of traffic generating software

• NIC driver and OS interface play an important role

• Throughput limitation is due to packet rate limitation of traffic generating software

• NIC driver and OS interface play an important role

21

Software Performance Analysis

• Comparing NMS with MGEN software (Intel NIC)Throughput (Mbps)

0

50

100

150

200

250

300

350

400

0 100000 200000 300000 400000

NMS MGEN

256

byte

fram

es25

6 by

te fr

ames

768

byte

fram

es76

8 by

te fr

ames

Throughput (Mbps)

0

100

200

300

400

500

600

700

0 50000 100000 150000 200000 250000

NMS MGEN

Throughput (Mbps)

0

100

200

300

400

500

600

700

800

0 20000 40000 60000 80000 100000

NMS MGEN

1500

byt

e fr

ames

1500

byt

e fr

ames

• Enhancements needed especially for small packets

• Enhancements needed especially for small packets

22

Conclusions and Future WorkConclusions and Future Work

•• Faster experiment setup and results processingFaster experiment setup and results processingAdvantagesAdvantages

•• No human attendance during testing requiredNo human attendance during testing required•• Extra features to make network testing easier to performExtra features to make network testing easier to perform•• User focuses on experiment target rather on implementationUser focuses on experiment target rather on implementation•• Greater availability of data after a test is completedGreater availability of data after a test is completed

•• Not portable on all existing platformsNot portable on all existing platforms•• Not optimized for local resources usage and all measurement Not optimized for local resources usage and all measurement

proceduresprocedures•• Does not implement a fineDoes not implement a fine--grained analysis to conserve grained analysis to conserve

management bandwidthmanagement bandwidth

DrawbacksDrawbacks

Future WorkFuture Work•• Extend support for other management protocolsExtend support for other management protocols•• Improve communication control procedures between Improve communication control procedures between

management console and agentsmanagement console and agents•• Include support for other networking technologiesInclude support for other networking technologies

23

Thank YouThank You