CN R - cs.unh.edu

Empirical Evaluation of UpstreamThroughput in a DOCSIS AccessNetwork

Swapnil Bhatia

(with Radim Bartos and Chaitanya Godsay)

Computer Networks Research Group

Department of Computer ScienceCN

RGand

The InterOperability Laboratory

Research Computing Center

University of New Hampshire

Durham, NH 03824

'

&

$

%

Objectives of this Talk

I Report measurement results from our DOCSIS testbed

I Describe our approach to interpreting results

I Promote discussion of practical aspects of access networks

I Solicit feedback and ideas from the audience

about each of the above

I Promote further collaborative study of access networks

2 of 30 MSAN 2005

'

&

$

%

Outline

I Introduction

� DOCSIS architecture, protocol, enhancers (piggybacking,

concatenation, fragmentation etc.).

I Background of this study

� InterOperability Lab., vendors and providers, complexity of

standard.

I Overview of Experiments

� Testbed, variables and data interpretation.

I Results

� Subset of conclusions.

I Summary and discussion

3 of 30 MSAN 2005

'

&

$

%

DOCSIS Introduction

(Source: DOCSIS 1.1 RFI Specification)

I DOCSIS — Data Over Cable Service Interface Specification

� MAC protocol utilizing existing CATV network

� Developed by CableLabs (Louisville, CO)

� Version 1.0 (pre-1999), 1.1, (1999-), 2.0 (2004-05)

4 of 30 MSAN 2005

'

&

$

%

DOCSIS Introduction (contd.)

I Tree topology

I Downstream vs. upstream

� Separate frequencies

� Broadcast, unicast (resp.)

� TDMA upstream

CM2

CM1

User 1data

User 2data

CMn

User ndata

CM3

User 3data

CATV Plant

CMTS

CM − Cable ModemCMTS − CM Termination System

Splitter/Combiner

WAN

To

I MAP: Periodic downstream control message

� Describes upstream transmission schedule

� Who: Which CM transmits?

� When: Starting when and how long?

� What: What can it transmit?

I Different types of transmission windows

� BW Request (BWR), BWR or Data, Short Data, Long Data,

Maintenance.

5 of 30 MSAN 2005

'

&

$

%



6 of 30 MSAN 2005

'

&

$

%



I Basic Data Transmission Cycle

� Wait for contention-based BW Request window

� Send Request (with retries)

� Retry until MAP received

� Wait for start of MAPped window

� Send data

I Alternatives

� Unicast data or request windows

7 of 30 MSAN 2005

'

&

$

%


Performance Enhancers

I Piggybacking

� Use part of data transmission window to make new requests

I Concatenation

� Transmit more than one data PDU in a single transmission

window

I Fragmentation

� Divide large data PDU to fit into current transmission window

I Header Suppression

� Header of data PDU suppressed at CM, regenerated at CMTS

8 of 30 MSAN 2005

'

&

$

%


Performance Enhancers

I Piggybacking

� Use part of data transmission window to make new requests

I Concatenation

� Transmit more than one data PDU in a single transmission

window

I Fragmentation

� Divide large data PDU to fit into current transmission window

I Header Suppression

� Header of data PDU suppressed at CM, regenerated at CMTS

9 of 30 MSAN 2005

'

&

$

%

Outline

I Introduction




� InterOperability Lab., CableLabs, complexity of standard.



I Results



10 of 30 MSAN 2005

'

&

$

%

Background of this Study

I Supported by the UNH InterOperability Laboratory

� Largest standards compliance testing facility in the country

� 19 consortia (including: iSCSI, SATA, IPv6, WiMax, EFM . . . )

� Industry supported, driven testing and applied research

I Conformance, interoperability and performance

� Previously verified, but in isolation

I Bottomline for vendors and service providers

� Configuration design

� Measurements with real devices

I Benefits to

� Protocol designers

� Equipment manufacturers

� Service providers

11 of 30 MSAN 2005

'

&

$

%

Outline

I Introduction







I Results



12 of 30 MSAN 2005

'

&

$

%

Overview of Experiments

I Goal

� Characterize upstream performance to

answer deployment design questions of the type:

� When is it better to piggyback than concatenate?

� How much is the improvement using concatenation?

� Dependent or independent of CMTS scheduling algorithm

I Independent variables

� Upstream channel rate

� Input packet length

� Performance enhancer

� CMTS

I Dependent variables� Throughput� Latency

Traffic generatorand analyzer

Coaxial cableEthernet

CMTS

Upstream data

RF analyzer

CM

CM

CM

13 of 30 MSAN 2005

'

&

$

%


I Independent variables

I Upstream channel rate

� {0.64, 1.28, 2.56, 5.12, 10.24} Mpbs.

I Packet length

� {64, 128, 256, 512, 768, 1262, 1500} bytes.

I Performance enhancer

� {Concatenation, Piggybacking, Both, Neither} allowed.

I CMTS

� {Vendor-A, Vendor-B}.

I Load

� Constant load of 8 Mbps (saturation).

14 of 30 MSAN 2005

'

&

$

%


I Define a configuration as a tuple

� < rate, length, enhancer, cmts >

I Define a transition as a doubleton of configurations

� {< v1, v2, v3, v4 >, < u1, u2, u3, u4 >} such that

∃! (1 ≤ i ≤ 4) (vi 6= ui)

I Consider a k−tuple of n1, . . . , nk -valued attributes each

I Total number of transitions is

N =k

∑

i=1

(ni(ni − 1)

2·∏

j 6=i

nj

)

=k

∏

i=1

ni ·k

∑

i=1

(ni − 1)

2

15 of 30 MSAN 2005

'

&

$

%


I 2400 cases

� An experiment for each transition

� Capture effect of a single change

� 25 runs per experiment

I Decide whether change improves or worsens performance

� Statistically robust, unbiased data interpretation

� Between and across CMTS

16 of 30 MSAN 2005

'

&

$

%


I Wilcoxon Signed Rank Sum Test (WSRS)

� A popular hypothesis test independent of distribution of data

� Calculates probability of median of sorted ranks being zero

� Null hypothesis (NH): no change in throughput due to a transition

(Toriginal − Tchanged = 0)

� Test provides probability P of NH being true

� Fix desired significance level α = 0.05

� If P ≤ α, reject NH (Toriginal − Tchanged 6= 0)

� i.e., transition affects throughput

� Check one-sided alternative

(Toriginal − Tchanged > 0, Toriginal − Tchanged < 0?)

I Actual α = 0.05/2400 (Bonferroni correction)

17 of 30 MSAN 2005

'

&

$

%

Outline

I Introduction







I Results



18 of 30 MSAN 2005

'

&

$

%

Results

0

0.5

1

1.5

2

2.5

3

0 200 400 600 800 1000 1200 1400 1600

Thr

ough

put (

Mbp

s)

Packet length (bytes)

Channel0.64Mbps1.28Mbps2.56Mbps5.12Mbps10.24Mbps

(a) No enhancers

0

0.5

1

1.5

2

2.5

3

0 200 400 600 800 1000 1200 1400 1600

Thr

ough

put (

Mbp

s)



(b) Both enhancers

I Per CMTS (99% confidence)

� Maximum throughput < 3

Mbps per CM

� Enhancers effective for

smaller packets

19 of 30 MSAN 2005

'

&

$

%

Results

0

0.5

1

1.5

2

2.5

3

0 200 400 600 800 1000 1200 1400 1600

Thr

ough

put (

Mbp

s)



(c) Concatenation

0

0.5

1

1.5

2

2.5

3

0 200 400 600 800 1000 1200 1400 1600

Thr

ough

put (

Mbp

s)



(d) Piggybacking

I Per CMTS (99% confidence)

� Concatenation very effective

for smaller packets

� Piggybacking largely

ineffective

� Need more CMs to see effect

20 of 30 MSAN 2005

'

&

$

%

Results

I When is Piggybacking useful?

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

200 400 600 800 1000 1200 1400

Thr

ough

put (

norm

aliz

ed)

Packet Length (bytes)

No enhancers on 1.28MbpsPiggybacking on 1.28Mbps

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

200 400 600 800 1000 1200 1400

Thr

ough

put (

norm

aliz

ed)


No enhancers on 2.56MbpsPiggybacking on 2.56MbpsNo enhancers on 5.12MbpsPiggybacking on 5.12Mbps

� Larger packet lengths at 1.28

Mbps

� Fewer request windows due to

large packets

21 of 30 MSAN 2005

'

&

$

%

Results

I Is Piggybacking Ever Better than Concatenation?

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

200 400 600 800 1000 1200 1400

Thr

ough

put (

norm

aliz

ed)


Concatenation on 0.64MbpsPiggybacking on 0.64Mbps

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

200 400 600 800 1000 1200 1400

Thr

ough

put (

norm

aliz

ed)


Concatenation on 1.28MbpsPiggybacking on 1.28Mbps

� Yes.

� With large packets on small

channels

22 of 30 MSAN 2005

'

&

$

%

Results

I Is having both enhancers always useful?

� No.

� With small packets it is detrimental.

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0 200 400 600 800 1000 1200 1400

Thr

ough

put (

norm

aliz

ed)


Concatenation on 0.64MbpsBoth Enhancers on 0.64Mbps

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0 200 400 600 800 1000 1200 1400T

hrou

ghpu

t (no

rmal

ized

)Packet Length (bytes)


0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0 200 400 600 800 1000 1200 1400

Thr

ough

put (

norm

aliz

ed)



0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0 200 400 600 800 1000 1200 1400

Thr

ough

put (

norm

aliz

ed)



23 of 30 MSAN 2005

'

&

$

%

Results

I Will an increase in channel rate always help?

� No, with small packets an increase can be detrimental.

0

0.2

0.4

0.6

0.8

1

0 1 2 3

Thr

ough

put (

norm

aliz

ed)

Enhancer combination

5.12 Mbps channel10.24 Mbps channel

24 of 30 MSAN 2005

'

&

$

%

Results

I Suppose small packets on a small channel

� What is the most economical way to increase throughput?

� Enable concatenation alone.

I Mid-sized packets?

� Must increase channel rate.

I Large packets?

� Enable piggybacking, or increase channel rate

25 of 30 MSAN 2005

'

&

$

%

Results

I Do both CMTS agree on all responses?

� No.

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0 200 400 600 800 1000 1200 1400

Thr

ough

put (

norm

aliz

ed)


None on 0.64 MbpsNone on 1.28 Mbps

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0 200 400 600 800 1000 1200 1400

Thr

ough

put (

norm

aliz

ed)


None on 1.28 MbpsNone on 0.64 Mbps

I Anomalies excluded from results

� Useful to respective CMTS vendors

26 of 30 MSAN 2005

'

&

$

%

Outline

I Introduction







I Results


I Summary and Discussion

27 of 30 MSAN 2005

'

&

$

%

Summary

I Characterized upstream performance of a DOCSIS system

� Channel rate, packet sizes, enhancers and CMTS

� Presented only a subset of results

I Empirical Results

� Exhaustive, measurement-based, real system-level

� Valuable tool for configuration design

� Black box approach

I Data currently being analyzed by vendors

� Also available at http://www.cs.unh.edu/cnrg/docsis

I Future work

� Multi-CM characterization

� Other QoS enhancers

� Comparison with analytical models

28 of 30 MSAN 2005

'

&

$

%

Future Work

I Classification and Regression Tree Model

29 of 30 MSAN 2005

'

&

$

%

Discussion and Questions

30 of 30 MSAN 2005

Documents

CN R - cs.unh.edu