Upload
others
View
8
Download
0
Embed Size (px)
Citation preview
Network Performance Tools
Jeff Boote Internet2/R&D
June 1, 2008 NANOG 43/ Brooklyn, NY
Overview• BWCTL
• OWAMP
• NDT/NPAD
BWCTL:Whatisit?Aresourcealloca=onandschedulingdaemonforarbitra=onofiperftests
ProblemStatement• Userswanttoverifyavailablebandwidthfromtheirsitetoanother.
Methodology– Verifyavailablebandwidthfromeachendpointtopointsinthemiddletodetermineproblemarea.
TypicalSolu=on• Run“iperf”orsimilartoolontwoendpointsandhostsonintermediatepaths
Typicalroadblocks• NeedsoNwareonalltestsystems• Needpermissionsonallsystemsinvolved(usuallyfullshellaccounts*)
• Needtocoordinatetes=ngwithothers*• NeedtorunsoNwareonbothsideswithspecifiedtestparameters*(*BWCTLwasdesignedtohelpwiththese)
Implementa=onApplica=ons
– bwctlddaemon– bwctlclient
OpensourcelicenseanddevelopmentBuiltuponprotocolabstrac=onlibrary
– Supportsone‐offapplica=ons– Allowsauthen=ca=on/policyhookstobeincorporated
Func=onality(bwctl)bwctlclientapplica=onmakesrequeststobothendpointsofatest
– Communica=oncanbe“open”,“authen=cated”,or“encrypted”(encryptedreservedforfutureuse)
– Requestsincludearequestfora=meslotaswellasafullparameteriza=onofthetest
– Thirdpartyrequests– Ifnoserverisavailableonthelocalhost,clienthandlestestendpoint
– *Mostly*thesamecommandlineop=onsasiperf(someop=onslimitedornotimplemented.)
Func=onality(bwctld)bwctldoneachtesthost
– Acceptsrequestsfor“iperf”testsincluding=meslotandparametersfortest
– Respondswithatenta=vereserva=onoradeniedmessage
– Reserva=onsbyaclientmustbeconfirmedwitha“startsession”message
– Resource“Broker”– Runstests– Both“sides”oftestgetresults
BWCTLExample
BWCTLData(Dash‐Board)
BWCTLData–PathHistory
ResourceAlloca=on(bwctld)• Eachconnec=onis“classified”(authen=ca=on)• Eachclassifica=onishierarchicalandhasanassociatedsetofhierarchicallimits:
– Connec=onpolicy(allow_open_mode)– Bandwidth(allow_tcp,allow_udp,bandwidth)– Scheduling(dura=on,event_horizon,pending)
• A=meslotissimplya=me‐dependentresourcethatneedstobeallocatedjustlikeanyotherresource.Itthereforefollowstheresourcealloca=onmodel.
BWCTL:3‐partyInterac=on
BWCTL:NoLocalServer
TesterApplica=ons• Iperfisprimary“tester”
– Wellknown–widelyused
• Problemsintegra=ngexec’dtool– Serverini=aliza=on(portnumberalloca=on)– errorcondi=ons– Noindica=onofpar=alprogress(Howfullwasthesendbufferwhenthesessionwaskilled?)
• thrulay/nubcpareavailableinlatest‘RC’versionofbwctl
GeneralRequirements• Iperfversion2.0and2.0.2• NTP(ntpd)synchronizedclockonthelocalsystem
– Usedforscheduling– Moreimportantthaterrorsareaccuratethantheclockitself
• Firewalls:– Lotsofportsforcommunica=onandtes=ng
• Endhostsmustbetuned!hbp://www.psc.edu/networking/perf_tune.htmlhbp://www‐didc.lbl.gov/TCP‐tuning/buffers.html
SupportedSystems• FreeBSD4.x,5.x• Linux2.4,2.6• (MostrecentversionsofUNIXshouldwork)
Policy/SecurityConsidera=ons• DoSsource
– ImaginealargenumberofcompromisedBWCTLDserversbeingusedtodirecttraffic
• DoStarget– Someonemightabempttoaffectsta=s=cswebpagestoseehowmuchimpacttheycanhave
• Resourceconsump=on– Timeslots
– Networkbandwidth
PolicyRecommenda=ons• Restric=veforUDP• MoreliberalforTCPtests
• Moreliberals=llfor“peers”
• ProtectAESkeys!
Availability• Currentlyavailablehbp://e2epi.internet2.edu/bwctl/
Maillists:
• bwctl‐[email protected]• bwctl‐[email protected]://mail.internet2.edu/wws/lists/engineering
OWAMP:Whatisit?OWDorOne‐WayPING
• Acontrolprotocol• Atestprotocol• Asampleimplementa=onofboth
WhytheOWAMPprotocol?• Findproblemsinthenetwork
– Conges=onusuallyhappensinonedirec=onfirst…– Rou=ng(asymmetric,orjustchanges)– SNMPpollingintervalsmaskhighqueuelevelsthatac=veprobescanshow
• Therehavebeenmanyimplementa=onstodoOne‐Waydelayovertheyears(Surveyor,Ripe…)
– Theproblemhasbeeninteroperability.– hbp://www.ieo.org/rfc/rfc4656.txt
OWAMPControlprotocol• Supportsauthen=ca=onandauthoriza=on• Usedtoconfiguretests
– Endpointcontrolledportnumbers– Extremelyconfigurablesendschedule
– Configurablepacketsizes• Usedtostart/stoptests• Usedtoretrieveresults
– Provisionsfordealingwithpar=alsessionresults
OWAMPTestprotocol• Packetscanbe“open”,“authen=cated”,or“encrypted”
SampleImplementa=onApplica=ons
– owampddaemon– owpingclient
OpensourcelicenseanddevelopmentBuiltuponprotocolabstrac=onlibrary
– Supportsone‐offapplica=ons– Allowsauthen=ca=on/policyhookstobeincorporated
Func=onality(owpingclient)– owpingclientrequestsOWDtestsfromanOWAMPserver
– Clientcanbesenderorreceiver– Communica=oncanbe“open”,“authen=cated”,or“encrypted”
– Supportsthesetupofmanytestsconcurrently
– Supportsthebufferingofresultsontheserverforlaterretrieval
Func=onality(owampd)owampd
– AcceptsrequestsforOWDtests
– Respondswithaccepted/denied– TestsareformallystartedwithaStartSessionsmessagefromtheclient.
– Runstests– Sessionswithpacketsreceivedattheserverarebufferedforlaterretrieval
OWPINGExample
OWAMPData(Dash‐Board)
OWAMPData(Path)
ResourceAlloca=on• Eachconnec=onis“classified”(authen=ca=on)• Eachclassifica=onisassociatedwithasetofhierarchicallimits
– Bandwidth(bandwidth)– Sessionbuffer(disk)– Datareten=on(delete_on_fetch)– Connec=onpolicy(allow_open_mode)
(no=medependentdimensiontoresourcealloca=oninowampd)
Architecture
GeneralRequirements• NTP(ntpd)synchronizedclockonthelocalsystem
– Specificconfigura=onrequirementsasspecifiedinNTPtalk…
– Strictlyspeaking,owampwillworkwithoutntp.However,yourresultswillbemeaninglessinmanycases
• gnumakeforbuildprocess
SupportedSystems• FreeBSD4.7+,5.x,6.0(64‐bit)• Linux2.4,2.6(64‐bit)• MacOSX10.4.X
• Solaris10.4.5• (MostrecentversionsofUNIXshouldwork)
RecommendedHardware• StableSystemClock
– Temperaturecontrolledenvironment
– NopowermanagementofCPU
• NostrictrequirementsforCPU,Memory,Busspeed
– Moretaskingscheduleswillrequiremorecapablehardware
Opera=onalconcernsTime:
– NTPissuespredominatetheproblems
– Determininganaccurate=mestamp“error”isinmanywaysmoredifficultthangesnga“verygood”=mestamp
– Workingasan“open”serverrequiresUTC=mesource(Forpredefinedtestpeers,otherop=onsavailable)
Firewalls:– Portfiltertrade‐off
• Administratorslikepre‐definedportnumbers• Vendormanufactureswouldprobablyliketo“priori=ze”testtraffic
• Owampdallowsarangeofportstobespecifiedforthereceiver
Policy/SecurityConsidera=ons• Third‐PartyDoSsource• DoStarget• Resourceconsump=on
– Memory(primaryandsecondary)– Networkbandwidth
PolicyRecommenda=ons• Restrictoverallbandwidthtosomethingrela=velysmall
– MostOWAMPsessionsdonotrequiremuch
• Limit“open”teststoensuretheydonotinterferewithprecisionofothertests
Availability• Currentlyavailablehbp://e2epi.internet2.edu/owamp/
Maillists:
• owamp‐[email protected]• owamp‐[email protected]://mail.internet2.edu/wws/lists/engineering
Advancedusertools• NDT
– Allowsuserstotestnetworkpathforalimitednumberofcommonproblems
• NPAD– Allowsuserstotestlocalnetworkinfrastructurewhilesimula=ngalongpath
UnderlyingUserAssump=on• Whenproblemsexist,it’sthenetworksfault!
SimpleNetworkPicture
Bob’s Host
Network Infrastructure
Carol’s Host
Sw
itch
1
Switch 2 Switch 3
NetworkInfrastructure
R1 R3
R4
R2 R7
R6 R9
R8 R5
Switch 4
NDT:Whatisit?Webbrowserinvokedadvanceduserbaseddiagnos=cs
• Allowsuserstotestanetworkpathforalimitednumberofcommonproblems–fromtheirdesktop
• NDTallowsusertogivethenetworkadministratoradetailedviewofexactlywhattheusershostisdoing
• Allowstheusertobeanac=vepar=cipantinthedebuggingprocess–allowsthemtomoredirectlyseehowhostconfigura=oneffectsperformance
Abemptstoanswertheques=ons:Whatperformanceshouldauserexpect?Whatisthelimi=ngfactor?
NDTGoals• Iden=fyrealproblemsforrealusers
– Networkinfrastructureistheproblem
– Hosttuningissuesaretheproblem
• Maketoolsimpletouseandunderstand
• Maketoolusefulforusersandnetworkadministrators
NDTuserinterface• Web‐basedJAVAappletallowstes=ngfromanybrowser
• Command‐lineclientallowstes=ngfromremoteloginshell
NDTsampleResults
FindingResultsofInterest
• DuplexMismatch– Thisisaseriouserrorandnothingwillworkright.Reportedonmainpage,onSta's'cspage,andmismatch:onMoreDetailspage
• PacketArrivalOrder– InferredvaluebasedonTCPopera=on.ReportedonSta's'cspage,(withlosssta=s=cs)andorder:valueonMoreDetailspage
FindingResultsofInterest• PacketLossRates
– CalculatedvaluebasedonTCPopera=on.ReportedonSta's'cspage,(without‐of‐ordersta=s=cs)andloss:valueonMoreDetailspage
• PathBobleneckCapacity– MeasuredvaluebasedonTCPopera=on.Reportedonmainpage
FindingNDTServers
NPAD/pathdiag:Whatisit?• Webbrowserinvokedadvanceduserbaseddiagnos=cs
– Allowsuserstotestalimitedpor=onofthenetworkpathlookingforproblemsthatwouldadverselyeffectlongerpaths
– Abemptstoanswertheques=ons:
• Whatperformanceshouldauserexpect?• Whatisthelimi=ngfactor?
NPAD/pathdiag• AnewtoolfromresearchersatPibsburghSupercomputerCenter
• Findsproblemsthataffectlongnetworkpaths
• UsesWeb100‐enhancedLinuxbasedserver
• WebbasedJavaclient
Sw
itch
1
Switch 2 Switch 3
LongPathProblem
R1 R3
R4
R2 R7
R6 R9
R8 R5
Switch 4
H1
H2
H3 X
1 msec H1 – H2 70 msec H1 – H3
LongPathProblem• E2Eapplica=onperformanceisdependantondistancebetweenhosts
• Fullsizeframe=meat100Mbps– Frame=1500Bytes– Time=0.12msec
– Inflightfor1msecRTT=8packets– Inflightfor70msecRTT=583packets
TCPConges=onAvoidance• Cutnumberofpacketsby½• Increaseby1perRTT
– LAN(RTT=1msec)• Inflightchangesto4packets• Timetoincreasebackto8is4msec
– WAN(RTT=70msec)• Inflightchangesto292packets• Timetoincreasebackto583is20.4seconds
NPADServermainpage
NPADSampleresults
FindingNPADservers
TrythesetoolsNetworkPerformanceToolkit
hbp://e2epi.internet2.edu/network‐performance‐toolkit.html
Knoppixdisk(OSonaCD)thathas:Iperf,thrulay,bwctl,owamp,NDT,NPAD,reverse‐traceroute/ping…
Ques=ons?