Electrical Engineering E6761Computer Communication NetworksLecture 10Active Queue MgmtFairnessInferenceProfessor Dan RubensteinTues 4:10-6:40, Mudd 1127Course URL: http://www.cs.columbia.edu/~danr/EE6761

AnnouncementsCourse EvaluationsPlease fill out (starting Dec. 1st)Less than 1/3 of you filled out mid-term evalsProjectReport due 12/15, 5pmAlso submit supporting work (e.g., simulation code)For groups: include breakdown of who did whatIts 50% of your grade, so do a good job!

OverviewActive Queue ManagementRED, ECNFairnessReview TCP-fairnessMax-min fairnessProportional FairnessInferenceBottleneck bandwidthMulticast TomographyPoints of Congestion

Problems with current routing for TCPCurrent IP routing is non-prioritydrop-tailBenefit of current IP routing infrastructure is its simplicityProblemsCannot guarantee delay boundsCannot guarantee loss ratesCannot guarantee fair allocationsLosses occur in bursts (due to drop-tail queues)Why is bursty loss a problem for TCP?

TCP SynchronizationLike many congestion control protocols, TCP uses packet loss as an indication of congestionTCPRateTimePacket loss

TCP Synchronization (contd)If losses are synchronizedTCP flows sharing bottleneck receive loss indications at around the same timedecrease rates at around the same timeperiods where link bandwidth significantlyunderutilizedFlow 1RateTimeFlow 2Aggregate loadbottleneck rate

Stopping SynchronizationObservation: if rate synchronization can be prevented, then bandwidth will be used more efficientlyQ: how can the network prevent rate synchronization?Flow 1RateTimeFlow 2Aggregate loadbottleneck rate

One Solution: REDRandom Early Detectiontrack length of queuewhen queue starts to fill up, begin dropping packets randomlyRandomness breaks the rate synchronizationAvg. Queue LenDrop Prob10minthmaxthmaxpminth: lower bound on avg queue length to drop pktsmaxth: upper bound on avg queue length to not drop every pktmaxp: the drop probability as avg queue len approaches maxth

RED: Average Queue LengthRED uses an average queue length instead of the instantaneous queue lengthloss rate more stable with timeshort bursts of traffic (that fill queue for short time) do not affect RED dropping rateavg(ti+1) = (1-wq) avg(ti) + wq q(ti+1)ti = time of arrival of ith packetavg(x) = avg queue size at time xq(x) = actual queue size at time xwq = exponential average weight, 0 < wq < 1Note: Recent work has demonstrated that the queue size is more stable if the actual queue size is used instead of the average queue size!

MarkingOriginally, RED was discussed in the context of dropping packetsi.e., when packet is probabilistically selected, it is droppednon-conforming flows have packets dropped as wellMore recently, marking has been consideredpackets have a special Early Congestion Notification (ECN) bitthe ECN bit is initially set to 0 by the sendera congested router sets the bit to 1receivers forward ECN bit state back to sender in acknowledgmentssender can adjust rate accordinglysenders that do not react appropriately to marked packets are called misbehaving

Marking v. DroppingIdea of marking was around since 88 when Jacobson implemented loss-based congestion control into TCP (see Jain/Ramakrishnan paper)Dropping vs. MarkingMarking does not penalize misbehaving flows at all (some packets will be dropped in misbehaving flows if dropping is used)With Marking, flows can find steady state fair rate without packet loss (assumes most flows behave)Status of Marking:TCP will have an ECN option that enables it to react to markingTCPs that do not implement the option should have their packets dropped rather than marked

Network FairnessAssumption: bandwidth in the network is limitedQ: What is / are fair ways for sessions to share network bandwidth?TCP fairness: send at the average rate that a TCP flow would send at along same pathTCP friendliness: send at an average rate less than what a TCP flow would send at along same pathTCP fairness is not really well-definedWhat timescale is being used?What about for multicast? Which path should be used?Which version of TCP?Other more formal fairness definitions?

Max-Min FairnessFluid model of network (links have fixed capacities)Idea: every session has equal right to bandwidth on any given linkWhat does this mean for any session, S?

SsendSrcvS can take use as much bandwidth on links as possiblebut must leave the same amount for other sessions using the linksunless those other sessions rates are constrained on other links

Max-Min Fairness formal defLet CL be the capacity of link LLet s(L) be the set of sessions that traverse link LLet A be an allocation of rates to sessionsLet A(S) be the rate assigned to session S under allocation AA is feasible iff for all L, A(S) CL S s(L)An allocation, A, is max-min fair if it is feasible and for any other allocation B, for every session Seither S is the only session that traverses some link and it uses the link to capacity orif B(S) > A(S), then there is some other session S where B(S) < A(S) A(S)

Max-min fair identification exampleQ: Is a given allocation, A, max-min fair?Write the allocation as a vector of session rates, e.g., A = session 1 is given a rate of 10 under Asession 2 is given a rate of 9 under Athere are 5 sessions in the networkLet B = be another feasible allocationThen A is not max-min fairB(S3) = 5 > 4 = A(S3)There is no other session Si where B(Si) < A(Si) A(S3)The only session where B(Si) < A(Si) is S2but A(S2) = 9 > A(S3)

Max-min fair exampleIntuitive understanding: if A is the max-min fair allocation, then by increasing A(S) by any forces some A(S) to decrease where A(S) A(S) to begin with1061581283

Max-Min Fair algorithmFACT: There is a unique max-min fair allocation!

Set A(S) = 0 for all SLet T = {S: A(S) CL for all L where S s(L) } S s(L)If T = {} then endFind the largest where for all L, A(S) + IS T CL S s(L)For all S T, A(S) += Go to step 2

Problems with max-min fairnessDoes not account for session utilitiesone session might need each unit of bandwidth more than the other (e.g., a video session vs. file transfer)easily remedied using utility functionsIncreasing one sessions share may force decrease in many others:

Max-Min fair allocation: all sessions get 1 By decreasing S1s share by , can increase all other flows shares by

Proportional FairnessEach session S has a utility function, US(), that is increasing, concave, and continuouse.g., US(x) = log x, US(x) = 1 1/xThe proportional fair allocation is the set of rates that maximizes US(x) without links used beyond capacity

S1R1S3R2222S2R2S4R4US(x) = log x for all sessions:xUS(x)

Proportional to Max-Min FairnessProportional Fairness can come close to emulating max-min fairness:Let US(x) = -(-log (x))As , allocation becomes max-min fairutility curve flattens faster: benefit of increasing one low bandwidth flow a little bit has more impact on aggregate utility than increasing many high bandwidth flowsx-(-log (x))

Fairness SummaryTCP fairnessformal definition somewhat unclearpopular due to the prevlance of TCP within the networkMax-min fairnessgives each session equal access to each links bandwidthdifficult to implement using end-to-end meanse.g., requires fair queuingProportional fairnessmaximize aggregate session utilityongoing work to explore how to implement via end-to-end means with simple marking strategies

Network InferenceIdea: application performance could be improved given knowledge of internal network characteristicsloss ratesend-to-end round trip delaysbottleneck bandwidthsroute tomographylocations of network congestionProblem: the Internet does not provide this information to end-systems explicitlySolution: desired characteristics need to be inferred

Some Simple InferencesSome inferences are easy to makeloss rate: send N packets, n get lost, loss rate is n/Nround trip delay:record packet departure time, TDhave receiving host ACK immediatelyrecord packet arrival time, TARTT = TA TDOthers need more advanced techniques

Bottleneck BandwidthA sessions bottleneck bandwidth is the minimum rate at which a its packets can be forwarded through the networkQ: How can we identify bottleneck bandwidth?Idea 1: send packets through at rate, r, and keep increasing r until packets get droppedProblem: other flows may exist in network, congestion may cause packet dropsSsendSrcvbottleneck

Probing for bottleneck bandwidthConsider time between departures of a non-empty G/D/1/K queue with service rate :

Observation 1: packets departure times are spaced by 1/

1/

Multi-queue exampleSlower queues will spread packets apartSubsequent faster queues will not fill up and hence will not affect packet spacinge.g., 1 > 2, 3 > 2

NOTE: requires queues downstream of bottleneck to be empty when 1st packet arrives!!!1232nd packet queues behind 1st 2nd packet queues behind 1st 1st packet exits system before 2nd arrives

Bprobe: identifying bottleneck bandwidthBprobe is a tool that identifies the bottleneck bandwidth:sends ICMP packet pairspackets have same packet size, Mdepart sender with (almost) 0 time spaced between themarrive back at sender with time T between themRecall T = 1/, where is bottleneck rateAssumes is a linear function of packet size, For a packet of size M, = M rr = bit-rate bottleneck bandwidthBottleneck bandwidth = r = M / T

BProbe LimitationsBProbe must filter out invalid probesanother flows packet gets between the packet paira probe packet is lostdownstream (higher bandwidth) queues are non-empty when first packet in pair arrives at queueSolution:Take many sample packet pairsuse different packet sizesNo packet in the middle: estimates come out same with different packet sizesPacket in the middle: estimates come out different

Different Packet SizesTo identify samples where background packet squeezed between the probesLet x be the size of the background packet Let r be the actual available bandwidthLet rest be the estimated available bandwidthWhen background packet gets between probes:rest = M / (x / r + M / r) = M r / (x + M)Let r = 5, x = 10M = 5, rest = 5/3M = 10, rest = 5/2Otherwise, rest = r : different packet sizes yield same estimatedifferent packet sizes yield different estimates!

Multicast TomographyGiven: sender, set of receiversGoal: identify multicast tree topology (which routers are used to connect the sender to receivers)=oror some other configuration?

mtracerouteOne possibility: mtraceroutesends packets with various TTLsrouters that find expired TTL send ICMP message indicating transmission failureused to identify routers along pathProblem with mtracerouterequires assistance of routers in networknot all routers necessarily respond

Inference on packet lossObservation: a packet lost by a shared router is lost by all receivers downstreampoint of packet lossreceivers that lose packetIdea: receivers that lose same packet likely to have a router in common

Q: why does losing the same packet not guarantee having router in common?

Mcast Tomography Steps4 step processStep 1: multicast packets and record which receivers lose each packetStep 2: Form groups where each group initially contains one receiverStep 3: Pick the 2 groups that have the highest correlation in loss and merge them together into a single groupStep 4: If more than one group remains, go to Step 3

loss correlation graph

Tomography Grouping Example{R1}, {R2}, {R3}, {R4}

{R1, R2}, {R3}, {R4}R1R2R3R4{{R1, R2}, R4}, {R3}

Ruling out coincident lossesLosses in 2 places at once may make it look like receivers lost packet under same router Q: can end-systems distinguish between these occurrences?

Assumption: losses at different routers are independent

ExampleActual shared loss rate is .1, but the likelihood that both packets are lost is p1 + (1-p1) p2 p3 = .415ASB123p1 = .1p2 = .7p3 = .5PAPB

A simple multicast topology modelA sender and 2 receivers, A & Bpackets lost at router 1 are lost by both receiverspackets lost at router 2 are lost by Apackets lost at router 3 are lost by BPackets dropped at router i with probability piReceivers compute PAB: P(both receivers lose the packet)PA: P(just rcvr A loses the packet)PB: P(just rcvr B loses the packet) To solve: Given topology, PAB, PA, PB, compute p1,p2,p3

Solving for p1, p2, p3PAB = p1 + (1-p1) p2 p3PA = (1-p1) p2 (1-p3)PB = (1-p1)(1-p2) p3

Let XA = 1 - PAB PA = (1-p1)(1-p2)Let XB = 1 - PAB - PA = (1-p1)(1-p3)Xi = P(packet reaches i)

p2 = PB / XAp3 = PA / XBp1 = 1 PA / (p2 (1-p3))

Multicast Tomography: wrapupApproach shown here builds binary trees (router has at most 2 children)In practice, router may have more than 2 childrenResearch has looked at when to merge new group into previous parent router vs. creating a new parentComments on resulting treerepresents virtual routing topologyonly routers with significant loss rates are identifiedrouters that have one outgoing interface will not be identifedrouters themselves not identified

Shared Points of Congestion (SPOCs)When sessions share a point of congestion (POC)can design congestion control protocols that operate on the aggregate flowthe newly proposed congestion manager takes this approachOther apps:web-server load balancingdistributed gamingmulti-stream applicationsS1R1S2R2Sessions 1 and 2 would share congestion if these links are congestedSessions 1 and 2 would not share congestion if these are the congested links

Detecting Shared POCsQ: Can we identify whether two flows share the same Point of Congestion (POC)?Network Assumptions:routers use FIFO forwardingThe two flows POCs are either all shared or all separate

Techniques for detecting shared POCsRequirement: flows senders or receivers are co-locatedPacket ordering through a potential SPOC same as that at the co-located end-systemGood SPOC candidatesco-located sendersco-located receivers

Simple Queueing Models of POCs for two flowsFG Flow 1FG Flow 2A Shared POCFG Flow 1FG Flow 2Separate POCsBGBGBG

Approach (High level)Idea: Packets passing through same POC close in time experience loss and delay correlations Using either loss or delay statistics, compute two measures of correlation:Mc: cross-measure (correlation between flows)Ma: auto-measure (correlation within a flow) such that if Mc < Ma then infer POCs are separateelse Mc > Ma and infer POCs are shared

The Correlation Statistics...Loss-Corr for co-located senders: Mc = Pr(Lost(i) | Lost(i-1))Ma = Pr(Lost(i) | Lost(prev(i)))

Loss-Corr for co-located receivers: in paper (complicated)Delay: Either co-located topology:Mc = C(Delay(i), Delay(i-1)) Ma = C(Delay(i), Delay(prev(i))

i-4i-2ii-1i-3i+1timeFlow 1 pktsFlow 2 pkts

Intuition: Why the comparison worksRecall: Pkts closer together exhibit higher correlationE[Tarr(i-1, i)] < E[Tarr(prev(i), i)]On avg, i more correlated with i-1 than with prev(i) True for many distributions, e.g., deterministic, anypoisson, poisson

Tarr(prev(i), i)Tarr(i-1, i)

SummaryCovered today:Active Queue ManagementFairnessNetwork InferenceNext time:network security