Upload
others
View
10
Download
0
Embed Size (px)
Citation preview
WiScapeA Case for Client-Assisted Approach
to Monitoring Wide-Area Wireless Networks
Sayandeep Sen, Jongwon Yoon, Joshua Hare,Justin Ormont, and Suman Banerjee
University of Wisconsin-Madison
Jongwon Yoon WiScape / IMC 2011
1
Motivation
• One way to monitor wireless performance
- Carrying out drive-by measurement test
Jongwon Yoon WiScape / IMC 2011
2
Labor intensive
Motivation
• One way to monitor wireless performance
- Carrying out drive-by measurement test
Jongwon Yoon WiScape / IMC 2011
3
Does not scale efficientlyfor large-area networks
Motivation
• One way to monitor wireless performance
- Carrying out drive-by measurement test
Jongwon Yoon WiScape / IMC 2011
4
Client Assisted Approach
Client-Assisted Monitoring
• Collecting measured samples from multiple clients
Jongwon Yoon WiScape / IMC 2011
5
Client-Assisted Monitoring
• Collecting measured samples from multiple clients
MeasurementRequests
Jongwon Yoon WiScape / IMC 2011
6
Server
Client-Assisted Monitoring
• Collecting measured samples from multiple clients
MeasuredSamples
Jongwon Yoon WiScape / IMC 2011
7
Benefits
• Leverages multiple users
- Covers more area more frequently
• Captures client experiences
• At locations clients care about
• Helps network operators
- Improves network performance
- Detects connectivity holes
→ in turn helps users
Jongwon Yoon WiScape / IMC 2011
8
Challenges of Client-Assisted Monitoring
Jongwon Yoon WiScape / IMC 2011
9
Challenges of Client-Assisted Monitoring
Measurements discrete in space
Aggregate in SPACE
Jongwon Yoon WiScape / IMC 2011
10
Measurements discrete in space
in SPACE(Zone)?
Challenges of Client-Assisted Monitoring
Battery
Jongwon Yoon WiScape / IMC 2011
11
Data Usage(Mbytes)
Challenges of Client-Assisted Monitoring
Aggregate in TIME(Epoch)?
Measurements discrete in time
Jongwon Yoon WiScape / IMC 2011
12
Challenges of Client-Assisted Monitoring
Aggregate in TIME(Epoch)?
Measurements discrete in time
Jongwon Yoon WiScape / IMC 2011
13
Can we compose
measurements?
Measurements not from a single client
Challenges of Client-Assisted Monitoring
Aggregate in TIME
Measurements discrete
Measurements discrete in space
Aggregate in SPACE(Zone)?
Jongwon Yoon WiScape / IMC 2011
14
Aggregate in TIME(Epoch)?
Measurements discrete in time
Can we compose
measurements?
Measurements not from a single client
WiScape• Characterize the Wireless landscape using client-assistance
• Using small and infrequent amounts of measurement collected by different users
80 packets/ 1hr
100 packets/ 75mins
Jongwon Yoon WiScape / IMC 2011
15
80 packets/ 1hr
120 packets/ 2hrs
90 packets/ 75mins
90 packets/ 80mins
100 packets/ 75mins
WiScapeServer
WiScape• Characterize the Wireless landscape using client-assistance
• Using small and infrequent amounts of measurement collected by different users
Corollary:
Jongwon Yoon WiScape / IMC 2011
16
Corollary: Client-assisted monitoring can capture coarse grained (~1hr) network property
Contributions
• Present a framework for coarse grained monitoring system
• Present applications of coarse grained monitoring of wide-area networksmonitoring of wide-area networks
Jongwon Yoon WiScape / IMC 2011
17
Outline
• Motivation
• Dataset
• Low overhead client-assisted monitoring
– Aggregation in space– Aggregation in space
– Aggregation in time
– Composing of measurements
• Applications
• Related work & Conclusion
Jongwon Yoon WiScape / IMC 2011
18
Outline
• Dataset
Jongwon Yoon WiScape / IMC 2011
19
Data Collection Methodology
Measurement Metrics1) TCP/UDP
Throughput2) Jitter
Net A
Net B
Jongwon Yoon WiScape / IMC 2011
20
CellularBase Stations
Client
Throughput2) Jitter3) UDP Loss rate4) LatencyNet C
Snapshot of TCP performance
Jongwon Yoon WiScape / IMC 2011
21
- City-wide area, NetB. Circle represents 1.1㎢ area
Vehicular Dataset
155 ㎢ city-widearea in MadisonWI
Jongwon Yoon WiScape / IMC 2011
22
- Madison Metro buses for 11 month- Net B and C
Vehicular Dataset
240 km road stretch, Madison, WI to Chicago, IL
- Collected for 6 month- Net B and C
Madison, WI
Jongwon Yoon WiScape / IMC 2011
23
Chicago, IL
Vehicular Dataset
20 km Road stretch, Madison, WI
Jongwon Yoon WiScape / IMC 2011
24
- Active measurement. Collected over 3 month- Net A,B and C
Static & Proximity Dataset
New Brunswick, NJ
Princeton, NJ
Madison, WI
: Static
Jongwon Yoon WiScape / IMC 2011
25
: Proximate
Static: 5 Locations in WI, 2 Locations in NJProximity: Vicinity of the static locations- 5 month in WI using Net A,B and C1 month in NJ using Net B and C
: Static
Dataset
Group Span Months Nets Location
Static 5 locations2 locations
51
A, B, CB, C
Madison, WINew Brunswick, Princeton, NJ
Proximity Vicinity of the static locations
51
A, B, CB, C
Madison, WINew Brunswick,
Jongwon Yoon WiScape / IMC 2011
26
locations 1 B, C New Brunswick, Princeton, NJ
Vehicular 155 ㎢ city-wide 240 Km road155 ㎢ city-wide20km road stretch
6
123
B, C
BA, B, C
Madison, WIMadison-ChicagoMadison, WIMadison, WI
Outline
• Low overhead client-assisted monitoring
– Aggregation in space– Aggregation in space
– Aggregation in time
– Composing of measurements
Jongwon Yoon WiScape / IMC 2011
27
Challenges of Client-Assisted Monitoring
Aggregate in TIME
Measurements discrete
Measurements discrete in space
Aggregate in SPACE(Zone)?
Jongwon Yoon WiScape / IMC 2011
28
Aggregate in TIME(Epoch)?
Measurements discrete in time
Can we compose
measurements?
Measurements not from a single client
Aggregating in SPACE (Zone)
Zone: a region which is small enough to ensure we have similar performance
Jongwon Yoon WiScape / IMC 2011
29
to ensure we have similar performance but big enough to ensure sufficientmeasurements can be collected
Aggregating in SPACE (Zone)
Jongwon Yoon WiScape / IMC 2011
30
Aggregating in SPACE (Zone)
Aggregate in SPACE(Zones)?
What zone size should be used?
Yes
Jongwon Yoon WiScape / IMC 2011
31
Aggregating in SPACE (Zone)
250 meter zone radius 97% of the zone have relative std. dev. < 8%
Jongwon Yoon WiScape / IMC 2011
32
97% of the zone have relative std. dev. < 8%
- TCP throughput for NetB, collected 155 ㎢ Madison(200~10000 measurements were collected per week at all zones)- Relative Standard Deviation = standard dev. of samples / mean of samples
WiScape Framework
Aggregate in SPACE(Zones)?
What zone size should be used?
Aggregate When to re-estimate
Yes
E.g., 250m radius
Yes
Jongwon Yoon WiScape / IMC 2011
33
Aggregate in TIME(Epoch)?
When to re-estimate the metric?
Yes
Measurement Metrics1) TCP/UDP throughput2) Jitter3) UDP Loss rate4) Latency
Aggregating in TIME (Epoch)
• 30 min bins (Coarse time scale)
- TCPthroughputWI
- Jitter, WI
Small degree of variation
Jongwon Yoon WiScape / IMC 2011
34
- UDPThroughputWI
-UDP loss rateWI
• Static-WI/NJ, Net A, B and C
Small degree of variation at coarse time (30min) scale
Aggregating in TIME (Epoch)
• 10 sec bins (Fine time scale)
- Standard Deviation for 10sec/30min time bins
Net B - WI Net C - WI Net B - NJ Net C – NJCoarse30min
Fine10sec
Coarse30min
Fine10sec
Coarse30min
Fine10sec
Coarse30min
Fine10sec
Jongwon Yoon WiScape / IMC 2011
35
30min 10sec 30min 10sec 30min 10sec 30min 10sec
TCP(Kbps)
33 102 36 96 126 408 167 414
UDP(Kbps)
39 82 38 94 153 429 182 365
Jitter(msec)
1.3 2.1 0.7 1.6 0.5 1.6 0.5 1.0
Loss(%)
~0 ~0 ~0 ~0 ~0 ~0 ~0 ~0
Aggregating in TIME (Epoch)
• 10 sec bins (Fine time scale)
- Standard Deviation for 10sec/30min time bins
Net B - WI Net C - WI Net B - NJ Net C – NJCoarse30min
Fine10sec
Coarse30min
Fine10sec
Coarse30min
Fine10sec
Coarse30min
Fine10sec
Jongwon Yoon WiScape / IMC 2011
36
30min 10sec 30min 10sec 30min 10sec 30min 10sec
TCP(Kbps)
33 102 36 96 126 408 167 414
UDP(Kbps)
39 82 38 94 153 429 182 365
Jitter(msec)
1.3 2.1 0.7 1.6 0.5 1.6 0.5 1.0
Loss(%)
~0 ~0 ~0 ~0 ~0 ~0 ~0 ~0High degree of variation at fine time (10sec) scale
Aggregating in TIME (Epoch)
• 10 sec bins (Fine time scale)
- Standard Deviation for 10sec/30min time bins
Net B - WI Net C - WI Net B - NJ Net C – NJCoarse30min
Fine10sec
Coarse30min
Fine10sec
Coarse30min
Fine10sec
Coarse30min
Fine10sec
Jongwon Yoon WiScape / IMC 2011
37
30min 10sec 30min 10sec 30min 10sec 30min 10sec
TCP(Kbps)
33 102 36 96 126 408 167 414
UDP(Kbps)
39 82 38 94 153 429 182 365
Jitter(msec)
1.3 2.1 0.7 1.6 0.5 1.6 0.5 1.0
Loss(%)
~0 ~0 ~0 ~0 ~0 ~0 ~0 ~0Degree of variation is zone specific
Aggregating in TIME (Epoch)
• Zone specific epochs
- Coherence interval of a metric (e.g. TCP throughput,
Jitter, etc.) in a given zone.
- We re-estimate the metric once every time epoch
Jongwon Yoon WiScape / IMC 2011
38
Aggregating in TIME (Epoch)
• Allan deviation
- Determining intervals for over which given metric is
most stable
- Finding the time (coherence) interval at which the difference between bins is minimumdifference between bins is minimum
Jongwon Yoon WiScape / IMC 2011
39
Aggregating in TIME (Epoch)
• Allan deviation
Time duration with the lowest Allan dev.
Jongwon Yoon WiScape / IMC 2011
40
Time duration with the lowest Allan dev. determines the Zone Specific Epoch
Aggregating in TIME (Epoch)
• Zone specific epochs
Jongwon Yoon WiScape / IMC 2011
41
- Proximate-WI/NJ, UDP throughput, Net B
75 min, WI 15 min, NJ
WiScape Framework
Aggregate in SPACE(Zones)?
What zone size should be used?
Aggregate What time epoch
Yes
e.g., 250m radius
Yes
Jongwon Yoon WiScape / IMC 2011
42
Aggregate in TIME(Epoch)?
What time epoch should be used?
How many client-sourced measurements?
Zone specific, O(10s of min)
Can we compose
measurements?
Yes
Yes
WiScape Framework
Aggregate in SPACE(Zones)?
What zone size should be used?
Aggregate What time epoch
Yes
e.g., 250m radius
Yes
Jongwon Yoon WiScape / IMC 2011
43
Aggregate in TIME(Epoch)?
What time epoch should be used?
Zone specific, O(10s of min)
Can we compose
measurements?
Yes
Yes
Zone specific: 100s of packets/ epoch (~10s of min)/zone
[details in paper]
How many client-sourced measurements?
WiScape Framework
Aggregate in SPACE(Zones)?
What zone size should be used?
Aggregate What time epoch
Yes
1) Radius 2) For each zone:
calculate
Zone specific
Yes
e.g., 250m radius
Jongwon Yoon WiScape / IMC 2011
44
Aggregate in TIME(Epoch)?
What time epoch should be used?
Zone specific, O(10s of min)
calculate time epoch
3) For each zone:calculate packet quantity
Yes
Zone specific: 100s of packets/ epoch (~10s of min)/zone
How many client-sourced measurements?
Can we compose
measurements?Yes
Outline
• Applications
Jongwon Yoon WiScape / IMC 2011
45
• Multi Network Interface Systems
Improving Multi-interface Applications
NetA,B and C
Jongwon Yoon WiScape / IMC 2011
46
TCP T
hro
ughput
(kbps)
Zone1 2
• Multi Network Interface Systems
Improving Multi-interface Applications
NetB?NetA,B and C
NetA?
Jongwon Yoon WiScape / IMC 2011
47
TCP T
hro
ughput
(kbps)
Zone1 2
Observations
TCP T
hro
ughput
NetA
NetB NetB
NetC
Persistently better performance
5%
95%
Jongwon Yoon WiScape / IMC 2011
48
TCP T
hro
ughput
(kbps)
Zone1 2
NetB
NetC
NetA
NetB
Observations
• Persistently better performance for a zone
Jongwon Yoon WiScape / IMC 2011
49
- 20 km road stretchMadison, WI
- Vehicular tracesMadison-Chicago
• Multi Network Interface Systems
Improving Multi-interface Applications
Jongwon Yoon WiScape / IMC 2011
50
TCP T
hro
ughput
(kbps)
Zone1 2
NetA
NetB
NetCNetA
NetB
NetC
Improving Multi-interface Applications
• Multi Network Interface Systems
Proportionally use NetA more
Jongwon Yoon WiScape / IMC 2011
51
TCP T
hro
ughput
(kbps)
Zone1 2
NetA
NetB
NetCNetA
NetB
NetC
Improving Multi-interface Applications
• Multi Network Interface Systems
Proportionally use NetC more
Jongwon Yoon WiScape / IMC 2011
52
TCP T
hro
ughput
(kbps)
Zone1 2
NetA
NetB
NetCNetA
NetB
NetC
Improving Multi-interface Applications
• Multi Network Interface Systems
Multisim-WiScape and MAR-WiScape
Jongwon Yoon WiScape / IMC 2011
53
TCP T
hro
ughput
(kbps)
Zone1 2
NetA
NetB
NetCNetA
NetB
NetC
Multisim-WiScape and MAR-WiScape(~30-40% improvement)
[details in paper]
Observations
• Identifying locations with variable performance
• Detecting flash crowd
- Game day,
Jongwon Yoon WiScape / IMC 2011
54
- Game day, Camp Randall, Madison WI
Application for Network Operators
• Identifying regions of high overload quickly
• Detecting flash crowd
- Game day,
Jongwon Yoon WiScape / IMC 2011
55
- Game day, Camp Randall, Madison WI
Detecting overall conditions quickly with small number of measurements
Outline
• Related work & Conclusions
Jongwon Yoon WiScape / IMC 2011
56
Related Work
• Monitoring Cellular Networks
- Mark the Spot (AT&T)
- 3G Test (University of Michigan)
- RootMetrics
Jongwon Yoon WiScape / IMC 2011
57
Future Work
• Full scalable deployment
• Monitoring dense deployments (NYC, LA, etc.)
Jongwon Yoon WiScape / IMC 2011
58
Conclusion
• Present a framework for coarse grained monitoring of wide-area network
- Validate it with data (datasets is available at
www.cs.wisc.edu/~yoonj/wiscape/IMC11_Data.html)
• Applications
- Improving client performance
- Helping operators
Jongwon Yoon WiScape / IMC 2011
59
Thank You!
• Questions?
Jongwon Yoon WiScape / IMC 2011
60
Backup slides
• Backup
Jongwon Yoon WiScape / IMC 2011
61
Aggregating in TIME (Epoch)
TCP T
hro
ughput
(kbps)
Jongwon Yoon WiScape / IMC 2011
62
TCP T
hro
ughput
(kbps)
Timebin size
Aggregating in TIME (Epoch)
TCP T
hro
ughput
(kbps)
Average, Std.
Jongwon Yoon WiScape / IMC 2011
63
TCP T
hro
ughput
(kbps)
Timebin size
30 min bins: Coarse time scale10 sec bins: Fine time scale
Aggregating in TIME (Epoch)
• Allan Deviation
TCP T
hro
ughput
10 min
5 5 5
Jongwon Yoon WiScape / IMC 2011
64
5 10 15 20 25 30
TCP T
hro
ughput
(Mbps)
Time (min)
5 5
5 5
5
Aggregating in TIME (Epoch)
• Allan Deviation
TCP T
hro
ughput
5 min
5 5 5
Jongwon Yoon WiScape / IMC 2011
65
5 10 15 20 25 30
TCP T
hro
ughput
(Mbps)
Time (min)
5 5 5
Aggregating in TIME (Epoch)
• Allan Deviation
TCP T
hro
ughput
15 min
5 10
Jongwon Yoon WiScape / IMC 2011
66
5 10 15 20 25 30
TCP T
hro
ughput
(Mbps)
Time (min)
510