View
1.239
Download
4
Category
Tags:
Preview:
DESCRIPTION
Activities and Tools on Network Monitoring and Measurements of Heterogeneous Networks
Citation preview
COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 1
NM2 Network Monitoring and
Measurements: some new perspectives (?!?!)
COMICS Research Group Dipartimento di Informatica e Sistemistica
Università degli Studi di Napoli “Federico II”
COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 2
Agenda Ø COMICS (COMputers for Interaction and
CommunicationS ) presentation Ø COMICS research topics Ø Network Monitoring and Measurements
Ø People Involved Ø Approach Ø Contributions
Ø Traffic Monitoring and Analysis Ø Network Measurements
Ø Contacts Ø Publications Ø Large Scale projects
COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 3
COMICS Ø COMICS (COMputers for Interaction and
CommunicationS ) headed by Prof. Giorgio Ventre Ø Work spans 2 laboratories and Spin-Offs:
ü UoN/DIS • @ University of Napoli
ü CINI/ITEM • a research lab of the Italian University Consortium in Computer
Science & Engineering ü Academic Spin-Offs
Ø Funding mainly from EU, Industry, with some money (?) from national and local government
COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 4
People@COMICS
Ø Today around 20 people in the group ü Seven of them with tenure and permanent positions
• Giorgio Ventre • Roberto Canonico • Simon Pietro Romano • Stefano Avallone • Antonio Pescapè • Maurizio D’Arienzo • Salvatore D’Antonio
Ø Collaborations with industries (Telecom Italia, Telefonica O2, Vodafone, H3G, Alcatel, Engineering Ingegneria Informatica, Accenture, Finmeccanica, Selex Sistemi Integrati, Juniper, Ericsson, IBM, Intel, Skylogic, ACCANTO, ALTO, several other SMEs, etc.) and AGCOM, Poste Italiane, and PA.
COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 5
Ø Former EU Projects: ü Guardians ü Cadenus ü Intermon ü E-NET, E-Next ü Cost 263 ü Cost 290 ü OneLab ü NetQoS ü Content ü OneLab2 ü Intersection
Research Projects@COMICS (1/2)
Ø Former National Projects: ü COSMIC ü ESALAB ü NADIR ü QUASAR ü WEBMINDS ü RECIPE ü LATINO
COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 6
Ø Current EU Projects: ü Inspire ü COST Action
IC0703 "Data Traffic Monitoring and Analysis (TMA)
Research Projects@COMICS (2/2)
Ø Current National Projects: ü LINCE
COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 7
Research@Comics Ø Research areas:
ü Traffic Measurements and Analysis ü Network Monitoring and Anomaly Detection ü Perfomance Evaluation of Networked Systems ü Security, Reliability and Resiliency ü QoS and QoE in Heterogeneous Networks ü Analysis and Detection of Network Outages ü Traffic Engineering ü Wireless Mesh Networks
• P2P overlay networks ü Management and control of network infrastructures
• SLA, SLS, Policy based management ü Multimedia services engineering (IETF activities) ü Emulation, Virtualization and Cloud ü Green Networking
COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 8
NM2, Network Monitoring and Measurements
COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 9
Network Monitoring and Measurements (NM2) ² NM2 is part of the COMICS research group of the Dipartimento
di Informatica e Sistemistica at University of Napoli Federico II ² People Involved
Alessio Botta
Alberto Dainotti Walter de Donato
Pietro Marchetta
Giuseppe Aceto Antonio Pescapè
COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 10
NM2 approach and vision
Links
Topologies
Applications
/
Services
Traffic
http://www.grid.unina.it/Traffic/
COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 11
Network/Traffic Monitoring and Analysis (NTMA)
OSS/BSS Integration • Customer Service Assurance
• Perfomance Monitoring
• Service Quality Management
• CRM
Third Parties NTMA
Probe Probe Probe Probe Probe
Network/IT Wired/Wireless Infrasctructure
NM2 philosophy, where we are
Distributed NOC
NM2
NM2
NM2
Other
COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 12
Traffic Monitoring and Analysis
COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 13
NM2: Contributions in Traffic Monitoring and Analysis
Ø Heterogeneous Network Scenarios (Home Networks, 3G/4G, Wireless Metropolitan Mesh Networks, Overlay Networks, gaming consoles, PDAs, household appliances, smartphones, etc)
ü Traffic Capture ü Traffic Characterization
• Novel applications (IPTV, games, streaming video, social networks, etc.)
• Malware traffic ü Traffic Modeling ü Traffic Generation and Active Probing ü Traffic and Service Classification
• New techniques for traffic classification ü Security and Anomaly Detection ü Analysis and Detection of Network Outages
COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 14
NM2: Traffic Capture, Characterization and Modeling (1/2)
Ø Why? ü Application and Service understanding and fingerprinting ü Security ü QoS requirements ü Performance Analysis ü Emulation ü etc.
Ø What? ü High-Speed Packet Capture (COTS, DAG, etc.) ü Statistical characterization and modeling of traffic
properties • Multi-level but with specific focus on packet-level • Per-single application
COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 15
Ø How? Ø Capture and Analysis
Ø Plab http://www.grid.unina.it/software/Plab
Ø Characterization Ø Matlab toolset for statistical
analysis of network traffic http://www.grid.unina.it/Traffic/Tools/statools.php
Ø Modeling Ø Statistical Modeling of traffic
sources
NM2: Traffic Capture, Characterization and Modeling (2/2)
Hidden States
IPT and PS conditional distributions
Hidden Markov Models for different network applications
COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 16
NM2: Active Probing and Application Traffic Generation (1/4)
Ø What? Generation of realistic traffic replicating as accurately as possible real applications and collection of information on how the single packets have been processed by the SUT (system under test).
Ø Why? Ø Network Performance Ø Testing/benchmarking
Ø Network Infrastructure
Ø Device capabilities Ø Quality of Service
(QoS) architectures Ø Queuing disciplines Ø Traffic shapers Ø Etc.
Traffic generation scenario
COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 17
NM2: Active Probing and Application Traffic Generation (2/4)
Ø How? ü D-ITG (Distributed Internet Traffic Generator)
ü http://www.grid.unina.it/software/ITG ü Distributed architecture: traffic senders and receivers can be spread
over the Internet and controlled by a central point ü Generation of traffic according to both statistical models of the
applications and traffic traces of real applications ü High performance, accuracy and flexibility ü Different kinds of hardware and operating systems supported
Trace-based
Analytical model-based
Application-level
Flow-level
Packet-level
Measurement of performance indicators
Open-loop Closed-loop
Automated & Configurable
Repeatabile
COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 18
NM2: Active Probing and Application Traffic Generation (3/4)
COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 19
Ø Since year 2003 D-ITG has being: § Used for the Italian WiMax experimentations (FUB) § Used for Magnets Network (Berlin) design and testing § Used in more than 20 EU research projects (Demo) § Used by more the 50 companies and Telcos for testing their
networking solutions § Used by NASA for the NASA Crew Exploration Vehicle (CEV)
Space communication link sizing § Used in Labs for CISCO certifications § Cited in more than 300 papers/theses worldwide § Included in several Linux distributions: Debian, Slax, OpenWRT,
Linux Microcore, etc.
NM2: Active Probing and Application Traffic Generation (4/4)
COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 20
NM2: Traffic and Service Classification (1/3) Ø Why?
ü Accounting ü QoS ü Security ü Network Analysis ü etc.
Ø What? Traffic and Service Classification/Identification ü (new) Payload Inspection ü Statistical Properties &
Machine Learning ü Multi-Classification
Web report of online traffic classification of a network link
COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 21
NM2: Traffic and Service Classification (2/3) Ø How?
ü TIE – Traffic Identification Engine http://tie.comics.unina.it
ü High-speed platform written in C ü Runs on Linux/FreeBSD/MacOSX
ü Modular and Plugin-based ü Large community
COST-TMA
PacketFilter
SessionBuilder
FeatureExtractor
DecisionCombiner Output
ClassificationPlugin #1
ClassificationPlugin #n...
COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 22
NM2: Traffic and Service Classification (3/3) Ø Novel Classification Technique: PortLoad*
ü Port-based is fast and privacy-friendly because: • It needs the 1st packet only • It uses fixed fields (protocol and port) • It uses few data It can be considered as a special case of packet-classification
techniques developed for routers, flow-monitors, etc.
ü Payload-based is accurate because relies on application-level headers and other information from the payload
• Payload-based signatures
Ø Port + Payload = PortLoad Ø Some interest from industry: Telecom Italia, Seven One
Solutions, ACCANTO, Huawei * Patent N.: NA2010AOOOO11
COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 23
NM2: Security and Anomaly Detection (1/2)
Ø Why? ü Security of network and critical infrastructures ü Security of users
Ø What? ü Traffic Analysis for Network/User Security ü Network Anomaly Detection ü Study of Malware Traffic ü Lawful Interception
Spread of the Slammer Worm in year 2001
COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 24
NM2: Security and Anomaly Detection (2/2) Ø How?
ü Anomaly Detection: traffic analysis through the Wavelet Transform
ü Study of Malware traffic: characterization and detection of computer worms
ü Lawful Interception (traffic monitoring, protocol decapsulation, covert channel detection, …) Witty Worm: Joint PS-IPT observed
from MAWI WIDE link
Detection of a Denial of Service attack through Analysis with the Wavelet Transform
COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 25
NM2: Analysis of Network Outages (1/2)
Ø BGP ü BGP updates from route collectors of RIPE-NCC RIS
and RouteViews ü We combined information from both databases ü Graphical Tools: REX, BGPlay, BGPviz
Ø Active Traceroute Probing
ü Archipelago Measurement Infrastructure (ARK)
ü Manually-initated traceroutes
Ø Internet Background Radiation ü Traffic reaching the UCSD Network Telescope ü Capable of revealing different kinds of blocking
COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 26
NM2: Analysis of Network Outages (2/2)
Telescopes vs BGP Ø Contrasting telescope traffic with
BGP measurements can reveal a mix of blocking techniques that cannot be discovered by looking only at BGP
Ø E.g. the second Libyan outage involved overlapping of BGP withdrawals and packet filtering
0
1
2
3
4
5
6
7
8
02-18 12:00
02-19 00:00
02-19 12:00
02-20 00:00
02-20 12:00
02-21 00:00pa
cket
s pe
r sec
ond
0
2
4
6
8
10
12
14
02-18 12:00
02-19 00:00
02-19 12:00
02-20 00:00
02-20 12:00
02-21 00:00
num
ber o
f vis
ible
pre
fixes
AS30981 AS6762 AS21003
COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 27
Network Measurements
COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 28
NM2: Contributions in Network Measurements Ø Network Performance Analysis and Improvement
ü Hybrid approaches (both active and passive) ü QoS, QoE, KPI ü Informed diversity for performance improvement ü Compression and Reduction of network data
Ø Broadband Benchmarking ü In terms of both QoS parameters and protocols
Ø Network Mapping ü Hybrid and Distributed approaches (routers, links, subnets) ü Accuracy, Discovery time, Intrusiveness
Ø Bandwidth Monitoring ü Wired and Wireless network scenarios ü Distributed and Hybrid approaches ü Accuracy, Discovery time, Intrusiveness
COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 29
Ø Innovative measurement techniques and approaches ü Active → purposely forge synthetic traffic ü Passive → exploit user generated traffic
Ø Able to work in emerging network scenarios ü 3/4G cellular networks, satellite networks, wireless mesh
networks, etc.
Ø Monitored parameters ü One-way delay, round trip time, delay variation (aka jitter),
latency, packet loss, shaping rate, packet reordering, TCP performance (e.g., 0-byte connections, reset segments, out-of-order segments, retransmitted segments, 1-Byte segments retransmitted), etc.
ü Specific Application Performance (DNS, Web, VoIP, IPTV, etc.) ü KPIs synthesized from the parameters above
NM2: Network Performance Analysis (1/3)
COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 30
A novel technique called Multi-layer Root Cause Analysis of TCP connections (MRCA)*
Ø Works analyzing the traffic generated by network users
Ø Allows to infer the performance of the TCP connections and to determine the associated root causes (network, application, OS configuration, etc.)
Ø Improves and integrates different techniques proposed in literature providing an approach integrating different point of view: aggregate, connection, and host Ø Some interest from the industry: Telecom Italia, Skylogic, ACCANTO, Telefonica O2, etc.
* Patent Under Submission
NM2: Network Performance Analysis (2/3)
COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 31
Ø Monitoring and modeling losses ü Characteristics of the loss process on the Internet and on satellite networks
Ø Detecting and analyzing middleboxes
NM2: Network Performance Analysis (3/3)
Internet (through PlanetLab) Satellite network
The effect of a shaper in a cellular network The effect of a PEP in a
satellite network
COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 32
Informed time diversity Ø Allows to reduce loss burstiness, thus improving application performance Ø We developed an application to use the interleaving in real networks
ü Realizes block interleaving ü Has measurement capabilities to automatically configure and adapt to varying network conditions
NM2: Network Performance Improvement (1/2)
COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 33
Informed space diversity Ø Allows to improve performance and reliability using multiple
paths Ø A new packet scheduling policy measuring network status
ü Working at IP layer with decisions on a packet-by-packet basis
Ø A tool to apply path diversity on real networks
NM2: Network Performance Improvement (2/2)
COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 34
NM2: Compression and Reduction of network data Ø Challenges and obstacles due to huge amount of
monitoring data (OSNs, p2p, high speed links, etc) from both active and passive approaches
Ø Compression ü Reduced memory footprint for
stored data ü A set of operations with reduced
time complexity on coded data Ø Reduction
ü There is no need to consider the entire data sets in the processing stage
ü Entropy-based methodology to reduce network traffic data
ü Off-line approach
more details, to answer a specific query that can be put inthe form Y = PX , one can use P instead of P and solveY = PX = TCX . Remember that ||T ||0 and ||C||0 areminimized by construction. Therefore, computing the productCX requires at most ||C||0 multiplications of coefficients.Similarly, computing Y = T (CX) requires no more than||T ||0 multiplications. Thus, the complexity of answering aquery that can be put in the form Y = PX is equal to||T ||0 + ||C||0 operations.Using this factorized format allows to answer a range of
queries. For instance, one can answer any max-k transactionquery to find the k largest transactions in the log file. Thiscan be solved by finding the k largest value of the N ! 1row of P that corresponds to the load. One can similarlyfind the total usage of a specific srcID, by summing allbytes value P (3, i) for which P (1, i) = srcID. The matrixC points to which patterns in T the user calls upon. Thussimilar users will have similar coefficient in the C matrix, andcan be identified by observing this sparse matrix. Conversely,the underlying matrix of patterns T embeds some overallbehavior of the system and can be used to identify abnormalusage. In particular, if after computing T over some period oftime ! at regular intervals, one sees dramatic changes in thecomposition of T , say min! ||T (t2)"!T (t1)||2 > " where !is a column permutation and " a threshold, then it might pointto some abnormal behavior in the system and call for someinvestigation.
In order to compute T and C, we use the techniqueproposed by Zujovic et al [5] in the context of pattern matchingalgorithms (applied to query-by-example image retrieval).
III. EXPERIMENTAL EVALUATION
To evaluate the performance of our technique, we haveconsidered its efficiency in terms of memory footprint reduc-tion (Compression Ratio) and different consequences of theapproximation (Bytes error, ID error, URL error). For theevaluation of the memory footprint, the compression ratio iscompared with the output of the general purpose compressionutility bzip2, employing the Burrows-Wheeler block sortingtext compression algorithm, and Huffman coding. The reportedresults are obtained by varying the value of #, that controlsthe amount of distortion allowed in the approximated factor-ization algorithm (the higher the value of #, the higher thetolerated approximation, but also the more sparsity induced inthe representation matrix); preprocessing stages are identical,with a fixed URL threshold2, while # is varied in the set{0.001, 0.0025, 0.005, 0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1}.
A. Data set
For a proof of concept of the effectiveness of our technique,we use a real traffic trace: the considered data cover atime span of 1 hour, presenting 26965 sessions, with 110different source IDs exchanging 907.024 MB of data with
2We performed an analysis of the effects of the filtering threshold, andfound that 5 is the maximum value for which the accuracy is not significantlyaffected: all presented results refer to this value of URL threshold.
0.08
0.1
0.12
0.14
0.16
0.18
0.2
0.22
0.001 0.01 0.1 1
Com
pre
ssio
n R
atio
lambda
CCS/fullbzip2full/fullbzip2flt/full
Tot/full
Fig. 1. Compression Ratios
1771 different destination URLs. The data set has the formatof a log file, each record of which represents a single HTTPsession, and is constituted by four fields: timestamp (in UNIXepoch time, µs precision), source ID, destination URL, load(in bytes).
B. Results
1) Compression Ratio: The total size of the compressedversion, as well as the size of specific components, is com-pared against the size of the original data. The quantitieswhose ratio is considered are: CCS - size in bytes of Com-pressed Column Sparse representation of C matrix alone; Tot- sum of the size in bytes of CCS, T matrix, bzip2-compressedordered list of URLs, bzip2-compressed ordered list of sourceIDs (this, with a few metadata, is all is needed to rebuild theoriginal data); bzip2flt - size in bytes of the URL-filtered andbzip2-compressed version of the original data; bzip2full - sizein bytes of the bzip2-compressed version of the original data.The CCS component is calculated in bits as:
nnz · (basesize+ #log2(cols)$) + #log2(nnz)$ · (rows + 1)
where nnz is the number of non-zero elements of the matrix,rows and cols are the dimensions of the matrix, and #·$ is theceiling function. This value corresponds to the size occupancyof a sparse matrix, represented as Compressed Column Sparse(or Compressed Sparse Column [6]), where indexes are binarycoded, and each element is represented with basesize bits.In the considered case, basesize is 32, rows and cols arerespectively 3 and 23516. The matrix T is represented asN · K values of length basesize bits each. Fig. 1 shows thecompression ratio as a function of the granularity parameter#. As expected, by increasing the approximation granularity,more sparsity is found in the factor matrix C, and therefore thesize occupancy for CCS representation decreases, causing thesize of total representation to gracefully decrease for growingvalues of #.2) Error on URL decoding: Error on URL decoding is
calculated as the ratio of entries with mistaken URLs versus
the third row of Table II, we obtain a reduction of 59%. In Fig.3(b) and 5(b)) we can see that the approximation is quite goodfor over the 99.9% of the distribution, and mean and standarddeviation are well approximated (see third row of Table II).Figs. 4 and 5(b) show that the two distributions are close inthe main part and in the tail too.
d) PSO: We sketch the Marginal Utility against thenumber of samples in Fig. 3(c). The QQ-plot in Fig. 3(d)shows a good approximation up to about 500 bytes, whichaccounts for 99.2% of the original data set. In the fourth rowof Table II a summary of the conducted analysis is reported.
0 0.005 0.01 0.015 0.02 0.0250
100
200
300
400
500
[s]
Entire setReduced set
0 2 4 6 8 10 12x 10
!3
0
500
1000
1500
[s]
Entire setReduced set
0 20 40 60 80 100 120 1400
0.01
0.02
0.03
0.04
0.05
0.06
0.07
Bytes
Entire setReduced set
0 50 100 150 200 250 3000
0.005
0.01
0.015
0.02
Bytes
Entire SetReduced Set
Fig. 4. Counter-Strike PDFs (clockwise: IAT, IDT, PSO, PSI).
Finally, the outgoing traffic is well approximated by anIDT/PSO series of about 4 millions of samples.
0 0.1 0.2 0.3 0.4 0.5 0.6!8
!6
!4
!2
0
[s]
Entire SetReduced Set
(a) IAT
0 0.2 0.4 0.6 0.8 1 1.2!8
!6
!4
!2
0
[s]
Entire SetReduced Set
(b) IDT
Fig. 5. CCDF of Counter-Strike.
2) Wavelet Analysis of CS Reduced Data Sets: The reduc-tion criterion we use here is based on the analysis of themarginal distributions of traffic data samples. But, in the studyof network traffic also temporal structures and dependencies(e.g. long range dependence and scaling behavior) can beof interest. In this section, we briefly show a time-frequencyanalysis based on the Wavelet Transform, revealing similarbehaviors between the entire and reduced data sets. We usethe Logscale Diagram (LD), which shows the trend of theenergy of the wavelet coefficients at each time scale, allowingto estimate the scaling behavior of the considered process andthe corresponding Hurst parameter (see [12]).
From the Counter-Strike IAT and IDT data sets, we calcu-lated the packet rate time series, with a period of 1 ms, oftraffic flowing in both directions (to and from the server).
2 4 6 8 10 12 14 16 18 20 22
!2
0
2
4
6
8
10
12
14
Octave
Entire SetReduced Set
(a) output packet rate
2 4 6 8 10 12 14 16 18
!4
!2
0
2
4
6
8
10
12
Octave
Entire SetReduced Set
(b) input packet rate
2 4 6 8 10 12 14 16 18 20 2212
14
16
18
20
22
24
26
28
30
Octave
Entire SetReduced Set
(c) output byte rate
2 4 6 8 10 12 14 16 18 20 22
8
10
12
14
16
18
20
22
24
Octave
Entire SetReduced Set
(d) input byte rate
Fig. 6. Logscale Diagram comparison of CS reduced and original data sets.
Let Sj , S1j be the logarithms of the energy of the wavelet
coefficients at scale j of respectively the entire and reduceddata sets. We found Sj =!j S1
j (for j = 1, ..., 17 in the caseof IAT and for j = 1, ..., 22 in the case of IDT) where the=!j operator takes into account their confidence intervals.This can be seen in the LDs in Figs. 6(a) and 6(b), where,at each scale, the confidence intervals of the two diagramsalways intersect. It is worth noting that we found the sameresults for the byte rate time series (Figs. 6(c) and 6(d)), whichwere obtained by combining information from the IAT andPSI series as well as IDT and PSO series. This comparisonis indeed important, since it is highlights properties of realnetwork traffic by combining information on packet arrivaltimes and their size. The analysis in this section shows that,for the considered data sets, the reduction did not heavily affectthe traffic temporal structures.
3) Effects of the Reduction on the Autocorrelation: Besidethe wavelet spectrum of the packet rate and byte rate series,we study the behavior of the Autocorrelation function for bothcomplete and reduced data sets of IDT and PS. This is doneto further assess the impact of the adopted approach on thesamples temporal behavior and their mutual dependencies.
In Fig. 7 the autocorrelation plots, until lag 100, are reportedfor all the data traces. As shown, for all the consideredvariables, the autocorrelation values of the reduced sets arevery close to those of the original sets. In particular, theRoot Mean Square (RMS) value of the error introduced bythe reduction ranges from 0.0128 (for the IAT series) to0.0232 (for the PSO series). To better observe the effect ofthe reduction, a zoomed view of the IAT autocorrelation isreported in Fig. 8. IAT is the variable that presents morecorrelation among the samples, also, its autocorrelation plotreveals an oscillating trend. The view of Fig. 8 allows to verifythat the trend of the reduced-set autocorrelation is very similarto that of the original set. This witnesses that the temporalstructure of the samples is preserved even in the presence ofa such particular behavior.
!"#$%&'((%)*+)%,-,*.%/-$%,**.%.*0#*/*1%-)%)"*%1#.*2)#34%3&%5666%7388'4#2-)#34$%932#*):%$';<*2)%8-))*.%*+,*.)$%&3.%,';(#2-)#34%#4%)"*%577%=>>?%,.32**1#4@$A%
COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 35
NM2: Broadband mapping (1/4)
Ø Different approaches Ø Web-based (Speedtest.net, Netalizr, ...)
Ø easy to use Ø one-shot measure Ø affected by interferences
Ø Client-based (Grenouille, Isposure, HoBBIT, ...) Ø repeated/periodical measures Ø easy large scale deployments Ø active only when the PC is turned on Ø unable to account for interferences
Ø Router-based (SamKnows, BISMark) Ø continous periodical measures Ø observes all traffic passing through network Ø can take into account interfereces Ø difficult to obtain large scale deployments
ISP
Home
network
Local loop
Modem
Router
Host
Ø Measuring from the edge → Independent point of view
COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 36
NM2: Broadband mapping (2/4)
BISMark (router-based) Ø Linux-based firmware
ü customized OpenWRT distro ü Netgear WNDR 3700v2
Ø On-demand access to the router console
Ø Active and passive measurements
Ø Current deployments ü 16 routers in Altanta ü 15 routers in Cape Town
http://projectbismark.net
HoBBIT (client-based) Ø Multi-platform application
ü based on Qt libraries Ø Extensible measurement
framework Ø Supports any underlying
measurement tool Ø Active measurements Ø Current deployment
ü ~100 users in Italy
http://hobbit.comics.unina.it
COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 37
NM2: Broadband mapping (3/4)
BISMark Ø Network measurements taken from the home gateway Ø Both active and passive measurements Ø Main features
ü On-demand remote router control/update
ü Measurements synchronization Ø Allows to monitor
ü Factors affecting performance (Local loop, ISP policies, Home network) ü Usage profiles
COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 38
NM2: Broadband mapping (4/4) Ø Network measurements taken from the users' PC
ü large scale deployments
Ø Active measurements using standard tools ü extensible measurement framework ü geolocation and mapping ü fine-grained management
Ø Main features ü multi-platform ü automatic updates ü per-application measurements
Ø Users can ü monitor their Internet connection ü compare results with others in the
same location
COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 39
NM2: Network Mapping (1/3) Ø Why?
ü Network control and management • Fault isolation, performance analysis, service locations, etc.
ü Network simulations • It is difficult to generate realistic topologies
ü Network aware applications • E.g. to improve the performance
Ø What? ü Automatic discovery of network maps in terms of: routers, links,
subnets, layer-2 devices, etc. ü Achieving
• Completeness (i.e. discover the entire topology) • Accuracy (i.e. make no mistakes) • Low intrusiveness (i.e. reduce both the discovery duration and the
traffic overhead) • Integration with Network Inventory solutions
COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 40
NM2: Network Mapping (2/3) Ø How?
ü Combining multiple passive/active methodologies and techniques ü Hybrid approaches ü Novel techniques based on: IGMP, ParisTraceroute, IP Options, ... ü Hynetd (single vantage point)
• http://www.grid.unina.it/software/TD ü MERLIN (multiple vantage points)
• http://svnet.u-strasbg.fr/merlin
COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 41
NM2: Network Mapping (3/3) MERLIN: MEasure the Router Level of the Internet Ø Target a specific Autonomous System network Ø Multiple techniques integrated and optimized
Ø Improved IGMP probing ü Paris traceroute ü Alias resolution
Ø Several input sources Ø BGP dumps, CAIDA Archipelago
datasets, MaxMind repositories, ... Ø Geo-Location, DNS mapping,
IPtoAS mapping, ...
MERLIN Monitor
MERLIN Coordinator
COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 42
NM2: Bandwidth Monitoring (1/2) Ø Why?
ü Network planning ü QoS ü Admission Control ü Support several kinds of applications
(P2P sharing, overlay networks, CDN, streaming, etc.)
Ø What? Estimation of capacity and available bandwidth in modern heterogeneous networks ü Optimized approaches for each network scenario:
wired, wireless, broadband access, mixed ü Allowing for different deployments:
single probe / edge probes / instrumented path ü Tunable in intrusiveness / accuracy / response speed
COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 43
NM2: Bandwidth Monitoring (2/2) Ø How?
Measurement platform: UANM (Unified Architecture for Network Measurement) http://grid.unina.it/Traffic/uanm.php ü Distributed ü Equipped with state-of-art
techniques ü Plugin-based (easily expandable
with experimental or cutting-edge techniques)
ü Decentralized synchronization for interference avoidance
ü API provided for embedding in applications, monitoring systems, appliances
COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 44
Research Collaborations (not exaustive list)
Ø Cooperative Association for Internet Data Analysis (CAIDA), San Diego,USA
Ø Georgia Tech, Atlanta, USA Ø Eurécom, Sophia Antipolis, France Ø Telefonica O2, (Spain and Germany) Ø TELECOM ParisTech (formerly known as ENST), France Ø Docomo Labs, Palo Alto, Stanford, USA Ø Deutsche Telekom Laboratories, Berlin, Germany Ø UCL, University of Louvain-la-neuve (Belgium) Ø Universitat Politècnica de Catalunya (Barcelona, Spain) Ø etc.
COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 45
Contacts
Antonio Pescape' Dipartimento di Informatica e Sistemistica University of Napoli ''Federico II'' Via Claudio, 21 - 80125, Napoli (Italy) [Room n. 3.10] tel. +39 081 7683856 fax +39 081 7683816 e-mail : pescape@unina.it (or pescape@ieee.org) Personal web-page: http://wpage.unina.it/pescape Teaching web-site (in Italian): http://www.docenti.unina.it/antonio.pescape
COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 46
Selected Publications (not exaustive list) Ø Srikanth Sundaresan, Walter de Donato, Nick Feamster, Renata Teixeira, Sam Crawford,
Antonio Pescapè, "Broadband Internet Performance: A View From the Gateway", to appear in ACM SIGCOMM 2011 proceedings, Toronto, ON, Canada, August 15-19, 2011.
Ø A. Dainotti, A. Pescapé, K. C. Claffy, “Issues and Future Directions in Traffic Classification", IEEE Network, 2011, to appear
Ø Pietro Marchetta, Pascal Mérindol, Benoit Donnet, Antonio Pescapé and Jean-Jacques Pansiot. "Topology Discovery at the Router Level: A New Hybrid Tool Targeting ISP Networks". IEEE Journal on Selected Areas in Communication (JSAC), Special Issue on Measurement of Internet Topologies, 2011, to appear
Ø Alessio Botta, Antonio Pescape', Vinh Bui, Weiping Zhu, "A Markovian Approach to Multi-path Data Transfer in Overlay Networks'', IEEE Transactions on Parallel and Distributed Systems, vol.21, no.10, pp.1398-1411, Oct. 2010
Ø Alessio Botta, Alberto Dainotti, Antonio Pescape', "Do You Trust Your Software-based Traffic Generator?'', IEEE Communications Magazine, vol.48, no.9, pp.158-165, Sept. 2010.
Ø A. Botta, R. Canonico, G. Di Stasi, A. Pescapè, G. Ventre, S. Fdida., "Integration of 3G connectivity in PlanetLab Europe - A step of an evolutionary path towards heterogeneous large scale network testbeds", ACM Springer Mobile Networks and Applications Journal, Special Issue on "Advances In Wireless Test beds and Research Infrastructures", Volume 15, Issue 3, June 2010, Pages 344-355.
Ø Alberto Dainotti, Antonio Pescapè, Giorgio Ventre, "A cascade architecture for DoS attacks detection based on the wavelet transform'', Journal of Computer Security, Volume 17, Number 6/2009, Pages 945-968
COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 47
Selected Publications (not exaustive list) Ø Marco Mellia, Antonio Pescapè, Luca Salgarelli, Traffic classification and its applications to
modern networks, Computer Networks, Volume 53, Issue 6, 23 April 2009, Pages 759-760. Ø A. Thomas Silverston, Olivier Fourmaux, Alessio Botta, Alberto Dainotti, Antonio Pescapè,
Giorgio Ventre, Kavè Salamatian, " Traffic Analysis of Peer-to-Peer IPTV Communities ," Computer Networks, Volume 53, Issue 4, 18 March 2009, Pages 470-484.
Ø Alessio Botta, Antonio Pescapè, Giorgio Ventre, "An approach to the identification of network elements composing heterogeneous end-to-end paths", Computer Networks, Volume 52, Issue 15, 23 October 2008, Pages 2975-2987, Elsevier.
Ø A. Dainotti, A. Pescapè, P. Salvo Rossi, F. Palmieri, G. Ventre, "Internet Traffic Modeling by means of Hidden Markov Models"; Computer Networks (Elsevier), Volume 52, Issue 14, 9 October 2008, Pages 2645-2662
Ø A. Botta, A. Pescapè, R. Karrer, “High-speed backhaul networks: myth or reality?”, Computer Communication Journal (Elsevier), Volume 31, Issue 8, 25 May 2008, Pages 1540-1550.
Ø A. Pescapè, “Entropy-Based Reduction of Traffic Data”, IEEE Communications Letters, pp. 191-193, Vol.11, No.2 - February 2007.
Ø S. Avallone, D. Emma, A. Pescapè, and G. Ventre, “Performance evaluation of an open distributed platform for realistic traffic generation”, Performance Evaluation (Elsevier), ISSN: 0166-5316 – Vol. 60, Issues 1-4, May 2005, pp 359-392
Ø Massimo Bernaschi, Filippo Cacace, Giulio Iannello, Antonio Pescapè, and Stefano Za, “Seamless Internetworking of WLANs and Cellular Networks: architecture and performance issues in a Mobile IPv6 scenario”, IEEE Wireless Communication Magazine (WCM) Journal, pp. 73-80, June 2005
COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 48
Selected Publications (not exaustive list) Ø A. Dainotti, A. Pescapè, C. Sansone, "Early Classification of Network Traffic through Multi-
Classification", Third International Workshop on Traffic Monitoring and Analysis (TMA'11) - April 2011, Vienna (Austria).
Ø A. Botta, A. Pescapè, "Monitoring and measuring wireless network performance in the presence of middleboxes", The 8th International Conference on Wireless On-demand Network Systems and Services (WONS), Bardonecchia (TO), Italy, January 2011. (Download the poster).
Ø A. Pescape', D.Rossi, D. Tammaro, S. Valenti, "On the Impact of Sampling on Traffic Monitoring and Analysis", 22nd International Teletraffic Congress, September 7 - 9, 2010 in Amsterdam, The Netherlands.
Ø A. Botta, A. Pescape', G.Ventre, E. Biersack, S. Rugel, "Performance footprints of heavy users in 3G networks via empirical measurement", The 6th International workshop on Wireless Network Measurements, May 31st, 2010, Avignon, France.
Ø A. Botta, A. Pescapè, G. Aceto, M. D'Arienzo, "UANM: a platform for experimenting with available bandwidth estimation tools", 15th IEEE Symposium on Computer and Communications, June 2010 Riccione (ITALY)
Ø A. Dainotti, F. Gargiulo, L. Kuncheva, A. Pescapè, C. Sansone, "Identification of traffic flows hiding behind TCP port 80", IEEE ICC 2010 - May 2010, Capetown (South Africa)
Ø G. Aceto, A. Dainotti, W. de Donato, A. Pescapè, "PortLoad: taking the best of two worlds in traffic classification", IEEE INFOCOM 2010 - WIP Track - March 2010, San Diego (CA, USA)
Ø V. Carela-Español, P. Barlet-Ros, M. Solè-Simò, A. Dainotti, W. de Donato, A. Pescapè, "K-dimensional trees for continuous traffic classification", 2nd International Workshop on Traffic Monitoring and Analysis (TMA'10), Zurich, Switzerland, April 7, 2010.
COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 49
Selected Publications (not exaustive list) Ø A. Dainotti, W. De Donato, A. Pescapè “TIE: a Community-Oriented Traffic Classification
Platform", International Workshop on Traffic Monitoring and Analysis (TMA'09) @ IFIP Networking 2009 - May 2009, Aachen (Germany)
Ø A. Dainotti, W. De Donato, A. Pescapè, P. Salvo Rossi, "Classification of Network Traffic via Packet-Level Hidden Markov Models", IEEE GLOBECOM 2008 - Dec 2008, New Orleans (LA, USA)
Ø Alessio Botta, Walter de Donato, Antonio Pescapè, Giorgio Ventre, "Networked Embedded Systems: a Quantitative Performance Comparison", IEEE Globecom 2008, New Orleans (LA), USA, 30 November - 4 December, 2008.
Ø Alessio Botta, Roberto Canonico, Giovanni Di Stasi, Antonio Pescapè, Giorgio Ventre, "Providing UMTS connectivity to PlanetLab nodes", 3rd International Workshop on Real Overlays & Distributed Systems, collocated with ACM CoNEXT 2008, Madrid, Spain, 9 - 12 December, 2008.
Ø Alessio Botta, Antonio Pescapè, Vinh Q Bui, Weiping Zhu, "An MDP-based Approach for Multipath Data Transmission over Wireless Networks", 2008 IEEE International Conference on Communications (ICC 2008), page(s): 268 - 274
Ø M.K. Afzal, Aman-Ullah-Khan, A. Pescape', Y. Bin Zikria, S. Loreto, "SCTP vs. TCP Delay and Packet Loss," Multitopic Conference, 2007. INMIC 2007. IEEE International , vol., no., pp.1-5, 28-30 Dec. 2007
Ø Roger Karrer and Antonio Pescape', "2nd generation wireless mesh networks: technical, economical and social challenges". In Proceedings of the 2007 IEEE International Conference on Future Generation Communication and Networking, Jeju Island, Korea, December 2007.
COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 50
Selected Publications (not exaustive list) Ø A. Botta, W. de Donato, A. Pescapé and G. Ventre, “Discovering Topologies at Router Level:
Part II”, Globecom 2007, Washington, D.C., 26-30 November, 2007 Ø Alessio Botta, Antonio Pescapè, Giorgio Ventre, Roger P. Karrer, "High-speed wireless
backbones: measurements from MagNets” in proceedings of the Fourth IEEE International Conference on Broadband Communications, Networks, and Systems (Broadnets), September 2007, Raileigh, North Carolina (USA).
Ø Vinh Q Bui, Weiping Zhu, Antonio Pescape', Alessio Botta, "Long Horizon End-to-End Delay Forecasts: A Multi-Step-Ahead Hybrid Approach", 12th IEEE Symposium on Computers and Communications, 2007
Ø Roger P. Karrer, Istvan Matyasovszki, Alessio Botta, Antonio Pescapè, "MagNets - experiences from deploying a joint research-operational next-generation wireless access network testbed”, TRIDENTCOM 2007, May 2007, Orlando, Florida (USA).
Ø Alberto Dainotti, Antonio Pescapè, Giorgio Ventre, "Worm Traffic Analysis and Characterization", 2007 IEEE International Conference on Communications (ICC 2007)
Ø A. Dainotti, A. Pescapè, P. Salvo Rossi, G. Iannello, G. Ventre, F. Palmieri “An HMM Approach to Internet Traffic Modeling", 2006 IEEE Globecom Conference, Quality, Reliability and Performance Modeling for Emerging Network Services Symposium
Ø A. Dainotti, A. Pescapè, G. Ventre, “Wavelet-based Detection of DoS Attacks", 2006 IEEE Globecom Conference, Network Security Systems Symposium
Ø Giulio Iannello, Francesco Palmieri, Antonio Pescapè, and Pierluigi Salvo Rossi,“End-to-End Packet-Channel Bayesian Model applied to Heterogeneous Wireless Networks”, IEEE Globecom 2005 General Conference - ISBN 0-7803-9415-1 - December 2005, St. Louis (MO, USA)
COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 51
Large Scale Projects
COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 52
NM2: Large Scale Projects Ø BISMark
ü Router-based platform for performing measurements of ISP performance, as well as traffic inside the home
ü http://projectbismark.net ü http://www.bufferbloat.net
Ø HobbIT ü User-based platform for performing measurements of ISP performance ü http://hobbit.comics.unina.it
Ø MERLIN ü Distributed platform to MEasure the Router Level of the Internet ü http://svnet.u-strasbg.fr/merlin
Ø MagNets ü Berlin Wireless MAN design and analysis ü http://www.net.t-labs.tu-berlin.de/~roger/magnets.html
Ø Distributed Monitoring and Measurements Architectures for ü Operational 3G Networks ü Operational Satellite Networks
COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 53
NM2: BISMark (1/3) Ø Network measurements taken from the home gateway
ü A vantage point into the home network
Ø Both active and passive measurements ü Customized to user profile ü Data anonymization
Ø Main features ü On-demand remote router
control/update ü Measurements synchronization
Ø Allows to monitor ü Factors affecting performance
• Local loop • ISP policies • Home network
ü Usage profiles
NOX Box 500Mhz Geode CPU 256 MB RAM 2GB Flash
Custom Debian OS
Netgear WNDR3700 680Mhz MIPS CPU 64 MB RAM 8MB Flash
Custom OpenWrt OS
Currently Supported devices
COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 54
NM2: BISMark (2/3)
COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 55
NM2: BISMark (3/3) Current worldwide deployment status
2 management servers
more than 50 routers
more than 50 measurement servers (Universities, MLab)
COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 56
NM2: HobbIT (1/2) Ø Network measurements taken from the users' PC
ü large scale deployments
Ø Active measurements using standard tools ü extensible measurement framework ü data geolocation and mapping ü fine-grained resource management
Ø Main features ü multi-platform client ü automatic updates ü per-application measurements
Ø Users can ü monitor their Internet connection ü compare results with others in the
same location
About 90 clients in Italy
COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 57
NM2: HobbIT (2/2)
COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 58
NM2: MERLIN MEasure the Router Level of the Internet Ø Target: a specific Autonomous System network Ø Efficient joint among the state-of-art techniques in the router
level topology discovery field: ü Improved IGMP probing ü Traceroute (paris-variant) ü Alias resolution technique
Ø Optimizations: ü Overcome technique's limitation while preserving benefits ü Limit the intrusiveness with a central smart coordination
Ø Several input sources: BGP dumps, CAIDA Archipelago datasets, MaxMind repositories, ...
Ø Geo-Location, DNS mapping, IPtoAS mapping, ...
COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 59
NM2: MERLIN
Sprint Network
MERLIN Monitor
Internet
I want... Sprint?!
MERLIN Coordinator
COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 60
MagNets: Berlin Wireless MAN
Specific active measurement techniques designed to infer Ø Throughput, latency, and loss of the links Ø Impact of enhanced transmission modes Ø Impact of the environment
http://www.net.t-labs.tu-berlin.de/~roger/magnets.html
Comprises a wireless backbone and different wireless mesh networks
Joint research with Deutsche Telekom Laboratories, Berlin
COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 61
Operational 3G networks
Ø Different kinds of (passive ) analyses on the user traffic Ø Traffic classification and application identification Ø TCP performance Ø Root cause analysis Ø Impact of middleboxes
Ø On different operational networks from different European telecom operators
COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 62
Operational satellite networks
Ø Distributed architecture for passive and active monitoring and measurements Ø Different kinds of passive and active analyses Ø End-to-end TCP and UDP
performance Ø TCP performance through
passive analysis Ø Influence of traffic shaping
mechanisms and middleboxes Ø Impact of meteorological
conditions on performance
http://broadband-satellite.atrexx.com/
Recommended