Embed Size (px)
Citation preview
© 2015 Netronome 1
Simon Horman
TC Classification with Open vSwitch
© 2015 Netronome 2
MotivationWould like to:● Partially or fully offload Open vSwitch
○ Software: e.g. TC○ Hardware: e.g. iNIC
● Do so using mechanisms present in upstream
© 2015 Netronome 3
Importance of Offloading
5
10
15
20
25
30
OVS Kernel Datapath with Netdev to VMs
OVS User-Space Datapath with Netdev to VMs
100 Wildcard Rules
1000 Wildcard Rules
10000 Wildcard Rules
64000 Wildcard Rules
Pac
kets
per
sec
ond
(mill
ions
)
12 CPU Cores8 CPU Cores
OVS Offload to iNIC with
PMD to VMs
1 CPU Core
5X Throughput Improvement + 50% CPU Savings
OVS L2/L3 Forwarding to 8 VMs with 64K Flows
OVS Offload to iNIC with
Netdev to VMs
1 CPU Core
© 2015 Netronome 4
eBPF TC Classifier and Actions● eBPF programs supplied to kernel by user-space● Data may be shared with user-space using eBPF maps
Helper
eBPF Classifier
User-Space
Kernel
to furtherprocessing
from priorprocessing
Map Prog
loadquery, update,...
© 2015 Netronome 5
eBPF Offload● eBPF seems well suited to offloading
to programmable hardware○ May be interpreted or;○ JITed and run natively○ Mechanism for very fast and flexible packet handling
© 2015 Netronome 6
TC Offload Possibility● One possibility is to add “ingress -1” support to TC
○ TC qdisc may be added before Ingress○ Managed by Kernel; Executed in Hardware
“Ingress -1”
Ingress
Kernel
iNIC
to furtherprocessingby kernel
from priorprocessingby iNIC
© 2015 Netronome 7
Hinting● Mechanism for offloading Open vSwitch classifier● Offload classifies packet and tags it with metadata● Open vSwitch performs flow lookup using metadata
Classifierpacket
OVSDatapathpacket with
metadatapacket
© 2015 Netronome 8
Modest Proposal for Hinting● TC Classification → metadata added as skb mark● Open vSwitch looks up flows using skb mark ● eBPF TC Classifier allows eBPF maps to synchronise
flow/mark mapping between TC classifier and Open vSwitch● And the eBPF may be offloaded to hardware
© 2015 Netronome 9
Hinting with eBPF Classifier
Hinting Packet Processing Path
ovs-vswitchd
eBPF Classifier
User-Space
Kernel
packet withskb-mark
packet
Map Prog
loadquery, update,...
OVS Datapathpacket
flows, stats, misses….
© 2015 Netronome 10
Offloaded Hinting
Hinting Packet Processing Path
ovs-vswitchd
User-Space
Kernel
loadquery, update,...
OVS Datapath packet
flows, stats, misses….
eBPF Classifier
iNIC
packet
Map Prog packet withskb-mark
© 2015 Netronome 11
Research at NetronomeInvestigating:● Integration of TC classifier with eBPF● Hardware offload of Kernel OVS datapath using switchdev● User-space driven hardware offload of OVS using match-
interface ● ...
© 2015 Netronome 12
QuestionsThanks to the community for many of the ideas presented hereand thanks for your time
© 2015 Netronome 13
Bonus Slides
© 2015 Netronome 14
Possible Flow Handling● Makes use of skb-mark for hinting● Flow handling should work with and without hw offload
© 2015 Netronome 15
eBPF Classifier
ProgMap
1.packet
Possible Flow-Miss Handling
iNIC
ovs-vswitchd
User-Space
Kernel
OVS Datapath
© 2015 Netronome 16
eBPF Classifier
ProgMap
1.packet
2. packet (without skb-mark)
Possible Flow-Miss Handling
iNIC
ovs-vswitchd
User-Space
Kernel
OVS Datapath
© 2015 Netronome 17
eBPF Classifier
ProgMap
1.packet
2. packet (without skb-mark)
Possible Flow-Miss Handling
iNIC
ovs-vswitchd
User-Space
Kernel
OVS Datapath
3. miss: packet, key
© 2015 Netronome 18
eBPF Classifier
ProgMap
1.packet
2. packet (without skb-mark)
4. store: key/mask→skb-mark
Possible Flow-Miss Handling
iNIC
ovs-vswitchd
User-Space
Kernel
OVS Datapath
3. miss: packet, key
© 2015 Netronome 19
eBPF Classifier
ProgMap
1.packet
2. packet (without skb-mark)
4. store: key/mask→skb-mark
5. add flow: key’/mask’→actions exec: packet→actions
Possible Flow-Miss Handling
iNIC
ovs-vswitchd
User-Space
Kernel
OVS Datapath
3. miss: packet, key
© 2015 Netronome 20
eBPF Classifier
ProgMap
1.packet
2. packet (without skb-mark)
4. store: key/mask→skb-mark
6. packet
5. add flow: key’/mask’→actions exec: packet→actions
Possible Flow-Miss Handling
iNIC
ovs-vswitchd
User-Space
Kernel
OVS Datapath
3. miss: packet, key
© 2015 Netronome 21
Possible Flow-Hit Handling
Hinting Packet Processing Path
Kernel
loadquery, update,...
OVS Datapath packet
eBPF Classifier
iNIC
packet
Map Prog packet withskb-mark
ovs-vswitchd
User-Space
Kernel
OVS Datapath 3. packet
eBPF Classifier1.packet
Map Prog2. packet, skb-markiNIC
