NateJohnston - PrincipalEngineer - ComcastDavidShaughnessy - NetworkSoftwareEngineer - Intel

PolicingYourNetworkNeutronDSCP

l WhatisDSCP?

l DSCPUseCases

l ImplementingDSCPinNeutronQoS

l What’sNext?

l Conclusion

PresentationOutline

WhatisDSCP?

DSCPstandsfor“DifferentiatedServicesCodePoint”.

DSCPisaprotocolforspecifyingandcontrollingnetworktrafficbyclasssothatcertaintypesoftrafficgetprecedence- forexample,voicetraffic,whichrequiresarelativelyuninterruptedflowofdata,mightgetprecedenceoverotherkindsoftraffic.

l DSCPisdefinedinRFC2474"DefinitionoftheDifferentiatedServicesField(DSfield)intheIPv4andIPv6Headers”

l DSCPforTunnelsisgovernedbyRFC2983"DifferentiatedServicesandTunnels"

What is DSCP?

DSCPisasix-bitfieldintheIPheader– itcomprisesthehighsixbitsoftheeightbitDS(“DiffServ”)fieldinanIPv4header.TheDSfieldwasformerlyreferredtoastheToS (“TypeofService”)field.

InIPv6,theDSfieldhasbeenrenamedtheTrafficClassfield.ItfunctionsidenticallytotheDSfieldinIPv4.

InIPv4,DSisthethirdfieldintheIPheader,andinIPv6TrafficClassisthesecond,whichindicatestheimportanceofDSCP.

TheDSCPBitsintheIPHeader

DSCPintheIPv4Packet

DSCPintheIPv6Packet

The8bitsoftheDSbytearedividedinto2sections:DSCPhasthehigh6bits,andECN(“ExplicitCongestionNotification”)hasthelast2bits.Forthepurposesofthispresentation,weareignoringtheECNbitsentirely.

The6DSCPbitsarearrangedinto4sections:l Thehighest3bitsareusedasthe“Precedence”setting,whichdefinestheClassSelector.

l Thenexttwobitsdesignate“Delay”and“Throughput”,andcollectivelydefine the“AssuredForwarding”(AF)setting.

l Thelowestbitdesignates“Reliability”andisunused.

ThecontentsoftheDSCPbitsarecollectivelyreferredtoasa“mark”or“codepoint”.

StructureoftheDSByte

l Per-HopBehavior(PHB)describeshowtrafficishandledateachhopbasedontheDSCPvaluethatisset.

l Incaseswherepacketswillbedroppedbecauseofcongestion,trafficwithalowerDSCPmark,ornoneatall,willbedroppedbeforetrafficwithahigherDSCPmark.

l Thereare4kindsofPHBsetting:none(thedefault),ClassSelector1-7,threeAssuredForwardingsubclassesforClassSelectors1-4,andExpeditedForwarding.

l BeforeyouimplementDSCP,youneedtoreallyknowhowthesethingsworkoryoumaycauseunintendedeffects.ContactanetworkengineertoverifyyourDSCPimplementationdetails.

Per-HopBehavior

AllDSCPMarks

DSCPUseCases

Typically,networksoperateonabest-effortdeliverybasis,whichmeansthatalltraffichasequalpriorityandanequalchanceofbeingdeliveredinatimelymanner.Whencongestionoccurs,alltraffichasanequalchanceofbeingdropped.

DSCPallowsyoutoselectspecificnetworktrafficforprioritizationaccordingtoitsrelativeimportanceandusecongestion-managementandcongestion-avoidancetechniquestoprovidepreferentialtreatment.

ThisiswhatDSCPwasdesignedfor.

UseCase1:PreferentialTreatmentUnderCongestion

l DSCPmarks,aspartoftheTCPheader,canbeusedascriteriainfirewallrulesandnetworkdeviceACLs.HereisaCiscoexample:

access-list 101 permit ip any any dscp cs1

l Thereforeyoucouldcomeupwithaconvention,forexample:l CS4isproduction guestsl CS3isQAguestsl Andsoforth…

l ThenensurethattheACLspermittrafficwiththegivenmarkstothenetworkscorrespondingtotheirfunction.

UseCase2:DSCPMarksasSecurityPolicy

ImplementingDSCPinNeutronQoS

TheNeutronDSCPcodedidnotmakeitintotheMitakareleaseofNeutronbecauseoflastminuteissues.

Thoseissueshavebeenresolved,andtheDSCPchangeshavebeenmergedintoNeutronmaster.

DSCPfunctionalitywillbeavailableintheNewtonrelease.

Newton

QoSObjectRelationships

QosDscpMarkingRule

QosRule

QosBandwidthLimitRule

QosPolicy

Port

extendsextends

l CreateaQoS policy

l CreateaDSCPruleforaQoS policy

l AssignaQoS policy toaport

AttachingaQoS PolicytoaPort

# openstack network qos-policy-create ‘urgent’ \--description ‘Deliver now’

# openstack network qos-dscp-marking-rule-create urgent \--dscp-mark 26

# openstack network port-update \48c6256f-9123-4e39-a321-108782807cfc --qos-policy urgent

QoS PolicyAddsandUpdates

Controller "DSCP mark 26, please"

"Update a port, please."

"What are the port details?"

Compute

ML2 Plugin and OVS Mechanism

DriverL2 Agent and

QoS Agent Extension

"Here, including QoS policy <uuid>."

"What are policy <uuid>’s rules?"

"Here they are."

"I’m subscribing to policy <uuid>."

"Hey, policy <uuid> changed!"

QoS ExtensionArchitecture

Controller

Core API

Compute

L2 Agent

1

QoS API Extension

2

ML2 Plugin and OVS Mechanism Driver

3

4

QoS Agent Extension

1 User assigns QoS policy containinga DSCP mark rule, to port

2 OVS driver sends RPC message

3 QoS agent extension receives RPC message

4 QoS agent extension notifies OVS agent

VM1

VM2

management network

"DSCP mark 26, please"

5Open

vSwitchOVS Agent

5 OVS agent sets DSCP mark on port

neutron-openvswitch-agent

ProviderNetworkwithOVS

Legend

Provider network Generic network (vlans) Management network External network

(Parenthetical numbers indicate OpenFlow ofport.)

Compute

Instance 1

VLAN Tagging

eth010.251.2.156

p1p110.1251.1.37

Linux Bridgeqbr

qvb

OVS Integration Bridgebr-int

p1p2 (1)

phy-br-ex (2)

tap

Instance 2

eth010.251.2.157

tap

qvo (2)

DSCP Marking

int-br-ex (1)

br-int (65534)Linux Bridge

qbr

qvbtap qvo (3)

DSCP Marking

Security Groups

Security Groups

tap

VLANs

OVS Provider Bridgebr-ex

br-ex (65534)

phy-br-ex (1)

Physical Network

p1p2

IntroductiontoOpenFlow

OpenFlow Switch

Packet In Packet OutTABLE 0

1 Find highest-priority matching flow entry

2 Execute instructions: - apply action list - update (clear or write) action set

3 Apply action set

Match Fields - ingress port - packet headers - metadata

Actions - modify packet headers - update metadata - send packet to another table or out of the pipeline

2 2 31 1

TABLE N

DSCPinOVS

cookie=1234, table=0, priority=10,arp,in_port=6 actions=resubmit(,24) cookie=1234, table=0, priority=1,in_port=6 actions=mod_nw_tos:104,NORMAL cookie=1234, table=0, priority=0 actions=NORMAL

cookie=1234, table=24, priority=2,arp,in_port=6,arp_spa=10.251.2.136 actions=NORMAL

cookie=1234, table=0, priority=10,arp,in_port=6 actions=resubmit(,24) cookie=1234, table=0, priority=0 actions=NORMAL

cookie=1234, table=24, priority=2,arp,in_port=6,arp_spa=10.251.2.136 actions=NORMAL

OVS Flow Table Before DSCP Marking Added

OVS Flow Table After DSCP Marking Added

DSCPMarkWitnessed

cookie=1234, table=0, priority=10,arp,in_port=6 actions=resubmit(,24) cookie=1234, table=0, priority=1,in_port=6 actions=mod_nw_tos:104,NORMAL cookie=1234, table=0, priority=0 actions=NORMAL

cookie=1234, table=24, priority=2,arp,in_port=6,arp_spa=10.251.2.136 actions=NORMAL

03:36:15.516084 fa:16:3e:41:90:36 > fa:16:3e:41:90:37, ethertype IPv4 (0x0800), length 73: (tos 0x68, ttl 10.251.2.136.58321 > 10.251.2.132.8999: Flags [P.], cksum 0x7a0e (correct), seq 78:85, ack 1, win 229, 03:36:15.516156 fa:16:3e:41:90:37 > fa:16:3e:82:32:e0, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 10.251.2.132.8999 > 10.251.2.136.58321: Flags [.], cksum 0x1b28 (incorrect -> 0x6528), seq 1, ack 85,

OVS Flow Table After DSCP Marking Added

tcpdump Output Reflecting DSCP Mark

DSCPMarkWitnessed:Wireshark

l Thesession IDof theL2agentisusedasthecookievalueinOVSflowentriesl WhenanL2agentreboots,modifiesaportorupdatesafirewallitremovesanyflowswhichflow_cookie_value !=my_session_ID

l BecauseL2agentextensions,suchastheQoS extension,managetheirownOVSflowentries,thechallengeistoensurethatanagentdoesnotremoveextensions’flowentries

L2AgentUpdates:Challenges

l Agentassignseachextensionitsowncookievalue

l Uponagentreboot orportupdateextension-owned flowentriesarepreserved

L2AgentUpdates:Solution

QoS Agent Extension

DVR Agent Extension

SG Agent Extension

OVS Agent Extension

L2 Agent

0x1234 0xefgh0x56780xabcd

L2-Agent-Extensions-API

QoS AgentExtension OVSCookieBridge OVSAgentBridge

Cookie=0x1234In_port=6,actions=mod_nw_tos:104,Normal

In_port=6,actions=mod_nw_tos:104,Normal

In_port=6,actions=mod_nw_tos:104,Normal

Cookie=0x1234,In_port=6,actions=mod_nw_tos:104,Normal

l WeneedtoallowNeutrontouseOVStoapplymultiplefeatures(e.g.,DSCPmarkingandVLANtagging)toasinglepacket

l Weneedtoensurethatafeaturedoesn’thijackotherOVSprocessingbyremovingapacketfromthepipeline

l Weneedtoensurethatfeatureflowsdon’taffect,andaren’taffectedby,anyotherpipelineprocessing,includinganyexistingorfuturefeatures

FeatureIsolation:Challenges

l Usemetadatatocorralpacketsforfeatureapplication

l Apacketenteringthepipelinehasallmetadatafieldssetto0

l Afeature’stable0flowentrywillmatch(inpart)onaparticularmetadatafieldandresubmitmatchingpacketstoa“featuretable”

l Thefeature-tableflowlogicwillapplythefeaturetothepacket,setanon-0valueinthepacket’sfeature-specificmetadatafield,andresubmitthepacketbacktotable0forfurtherprocessing

FeatureIsolation:Solution

OpenStack’s DSCPinOVS

cookie=abcd, table=0, priority=65535,reg2=0x0,in_port=6 actions=resubmit(,10)cookie=1234, table=0, priority=10,arp,in_port=6 actions=resubmit(,24)cookie=1234, table=0, priority=0 actions=NORMAL

cookie=abcd, table=10, priority=0 actions=load:0x37->NXM_NX_REG2[0..5],mod_nw_tos:104,resubmit(,0)

cookie=1234, table=24, priority=2,arp,in_port=6,arp_spa=10.251.2.136 actions=NORMAL

cookie=1234, table=0, priority=10,arp,in_port=6 actions=resubmit(,24)cookie=1234, table=0, priority=0 actions=NORMAL

cookie=1234, table=24, priority=2,arp,in_port=6,arp_spa=10.251.2.136 actions=NORMAL

OVS Flow Table Before DSCP Marking Added

OVS Flow Table After Feature’s DSCP Marking Added

l Additional ruletypesmaycomeavailableduring aserverupgradel DifferentagentsmayusedifferentAPIobjects(e.g.,oneagentmaynotknowaboutDSCPruleswhileanotherdoes)

l Differentagentsmayusedifferentlyversionedobjects(e.g.,oneagentknowsaboutQoSpolicyversion1.0whileanotheragentknowsaboutQoSpolicy1.1)

l Anagentneeds toknowaboutagivenpolicyinstanceid

Server-AgentCommunications:Challenges

Server-AgentCommunications:Solution

ReportVersionedObjects

CreateVersionedFanoutQueue

CreateVersionedFanoutQueue

ReportVersionedObjects

What’sNext?

l IngressDSCPfiltering:WediscussedachangetoSecurityGroupsthatwouldallowingressDSCPfiltering,butSecurityGroupchangesarefraughtwithperil.ThiscouldbecomearoadmapitemforFWaaS atsomepointhowever.

l MarkingencapsulatingpacketswiththeDSCPmarkoftheencapsulatedtraffic:WebelieveitveryunlikelythatDSCP-basedfilteringwilloccurbetweenpartsofaregion,becausesomuchofthatwillbetunneled;theassumptionisthatanyDSCP-relatedbehaviorwillbecomerelevantonlyafterthetrafficexitsthespines.

OmittedfromReferenceImplementation…

l NeutronsupportforExplicitCongestionNotification(ECN)

l Neutrontrafficclassification

l Minbandwidthguarantees

l Ingressbandwidthlimiting

FutureRoadmap

Conclusion

lNeutronserver(neutron.conf)service_plugins = neutron.services.qos.qos_plugin.QoSPlugin

notification_drivers = message_queue

lML2pluginandL2agent(ml2_conf.ini)

[ml2]

extension_drivers = qos

[agent]

extensions = qos

ConfiguringQoS

lDevStack (local.conf)

enable_plugin neutron git://git.openstack.org/openstack.neutron

enable_service q-qos

[[post-config|$NEUTRON_CONF]]

[DEFAULT]

service_plugins=neutron.services.qos.qos_plugin.QoSPlugin

[[post-config|/$Q_PLUGIN_CONF_FILE]]

[ml2]

extension_drivers=qos

[agent]

extensions=qos

ConfiguringQoS,cont.

GeneralDocumentationl NetworkingGuide:UsingOpenStackNetworkingwithQoSl Tokyopresentation“QoS- ANeutronn00bie”

PrerequisitesfortheDSCPChangel L2agentextensionsimplementation: agentAPIl RPCcallbacksrollingupgrades implementationl RPCcallbacksrollingupgrades implementation: reportingandintegration

ChangesAssociatedwiththeDSCPChangel OriginalQoSAPIextensionspecificationl QoSAPIextensionwithDSCPspecificationl ServerandagentDSCPQoSruleimplementationl DSCPimplementation inneutronclientl DSCPinHeat:specificationl DSCPinHeat:implementationoftheQosDscpMarkingRule resource

OtherOpenStack Resources

DiffServ RFCsl RFC2474— DefinitionoftheDifferentiatedServicesField(DSField)intheIPv4andIPv6Headers

l RFC2475— AnArchitectureforDifferentiatedServices

l RFC2597— AssuredForwardingPHBGroupl RFC2983— DifferentiatedServicesandTunnels

l RFC3086— DefinitionofDifferentiatedServicesperDomainBehaviorsandRulesfortheirSpecification

l RFC3140— PerHopBehaviorIdentificationCodes(obsoletes RFC2836)

l RFC3246— AnexpeditedforwardingPHB(obsoletes RFC2598)

l RFC3247— SupplementalInformationfortheNewDefinitionoftheEFPHB(ExpeditedForwarding Per-HopBehavior)

l RFC3260— NewTerminologyandClarifications forDiffserv (updatesRFCs2474,2475,and2597)

l RFC4594— ConfigurationGuidelinesforDiffServ ServiceClasses

l RFC5865— ADifferentiatedServicesCodePoint(DSCP)forCapacity-AdmittedTraffic (updates RFCs4542and4594)

DiffServ ManagementRFCsl RFC3289—ManagementInformationBasefortheDifferentiatedServicesArchitecture

l RFC3290— AnInformalManagementModelforDiffserv Routers

l RFC3317— DifferentiatedServicesQuality ofServicePolicyInformationBase

IfYouWanttoReadMoreaboutDSCP…

AllpatentslistedareU.S.patents.l US20070199064 — Methodandsystemforqualityofservicebasedwebfilteringl US20080089324 — Indicatingorremarkingofa dscp forrtp ofaflow(call) toandfromaserver

l US20080144502 — In-band quality-of-service signaling toendpoints that enforcetraffic policies attraffic sources using policy messages piggybacked onto DiffServbits

l US8767569B2 — Dynamic DSCP availability requestmethodl US20130283379 — System,methodandapparatus thatemployvirtualprivatenetworkstoresistip qos denialofserviceattacks

DSCPUseCasesinPatents

Itwasacollectiveeffort…soourthanksgoesto:

Margaret Frances

Miguel Angel Ajo

Ihar Hrachyshka

Victor Howard

James Reeves

Gary Kotton

John Schwarz

Acknowledgements

l Inteltechnologies’ featuresandbenefitsdependonsystemconfigurationandmayrequireenabledhardware,softwareorserviceactivation.Learnmoreatintel.com,orfromtheOEMorretailer.

l Nocomputersystemcanbeabsolutely secure.

l Testsdocument performanceofcomponents onaparticulartest,inspecific systems.Differencesinhardware,software,orconfigurationwillaffectactualperformance.Consult othersourcesofinformationtoevaluateperformanceasyouconsider yourpurchase. Formorecompleteinformationaboutperformanceandbenchmarkresults, visithttp://www.intel.com/performance.

l Intel,theIntellogoandothersaretrademarksofIntelCorporation intheU.S.and/orothercountries.*Othernamesandbrandsmaybeclaimedasthepropertyofothers.

l ©2016IntelCorporation.

l ©2016ComcastCorporation.

LegalNoticesandDisclaimers

Thank You

Wanttheseslides?