OpenStack Neutron NetworkingBD%C9%C0%E5%C8%C6.pdf · With the introduction of the full software-defined networking stack provided by OpenStack Networking (neutron) in the Folsom release,

OpenStack Neutron Networking

Paul Sim Technical Account Manager [email protected]

● OpenStack overview

● OpenStack Components

● Nova-network

● Network as a Service : Neutron

● Network Virtualization - Overlay

● Neutron - Modular Layer 2

● Neutron High Availability

● Distributed Virtual Router

Index

OpenStack overview

OpenStack Components

● Identity Keystone

● Compute Nova

● Block Storage Cinder

● Object Storage Swift

● Image Storage Glance

● Network Quantum/Neutron

● Dashboard Horizon

● Metering Ceilometer

● Orchestration Heat

● Database as a Service Trove

● Hadoop as a Service Sahara

● File-share Service Manila

OpenStack network model

1. Nova-network

a. Flat Network Manager

b. Flat DHCP Network Manager

c. VLAN Network Manager

2. Neutron with plugins

a. ML2 : OpenvSwitch

b. VMware NSX

c. Software Defined Networking

OpenDaylight, Ryu

d. MidoNet

e. OpenContrail

f. ...

OpenStack networking with Nova-network

Controller node

Keystone

Compute node - 2 Compute node - 3

Nova

Glance Horizon

Nova compute

eth1

eth0

eth1

eth0

eth1

eth0

Management

External network

Nova network

Nova compute

Nova network

Compute node - 1

Nova compute

eth1

eth0

Nova network

Nova-network

eth0

Flat DHCP Network Manager VLAN Network Manager

VM VM VM

Bridge dnsmasq

G/W

VM VM VM

Bridge 1 Bridge 2

eth0

vlan 100 vlan 101

dnsmasq dnsmasq

G/W G/W

Nova-network

VM VM VM

Bridge 1 Bridge 2

eth0

vlan 100 vlan 101

dnsmasq dnsmasq

G/W G/W

Compute node-2

VM VM VM

Bridge 1 Bridge 2

eth0

vlan 110 vlan 100

dnsmasq dnsmasq

G/W G/W

Compute node-1

Switch

vlan 100,110

vlan 100,101

Nova-network

Deprecation of Nova Network

With the introduction of the full software-defined networking stack provided by OpenStack Networking (neutron) in the

Folsom release, development effort on the initial networking code that remains part of the Compute component has

gradually lessened. While many still use nova-network in production, there has been a long-term plan to remove the

code in favour of the more flexible and full-featured OpenStack Networking.

An attempt was made to deprecate nova-network during the Havana release, which was aborted due to the lack of equivalent functionality (such as the FlatDHCP multi-host high availability mode mentioned in this guide), lack of a migration path between versions, insufficient testing, and simplicity when used for the more straightforward use cases nova-network traditionally supported. Though significant effort has been made to address these concerns, nova-network will not be deprecated in the Icehouse release. In addition, the Program Technical Lead of the Compute project has indicated that, to a limited degree, patches to nova-network will now again begin to be accepted. This leaves you with an important point of decision when designing your cloud. OpenStack Networking is robust enough to use with a small number of limitations (IPv6 support, performance issues in some scenarios) and provides many more features than nova-network. However, if you do not have the more complex use cases that can benefit from fuller software-defined networking capabilities, or are uncomfortable with the new concepts introduced, nova-network may continue to be a viable option for the next 12 to 18 months. Similarly, if you have an existing cloud and are looking to upgrade from nova-network to OpenStack Networking, you should have the option to delay the upgrade for this period of time. However, each release of OpenStack brings significant new innovation, and regardless of your use of networking methodology, it is likely best to begin planning for an upgrade within a reasonable time frame of each release. As mentioned, there's currently no way to cleanly migrate from nova-network to neutron. We recommend that you keep a migration in mind and what that process might involve for when a proper migration path is released. If you must upgrade, please be aware that both service and instance downtime is likely unavoidable.

http://docs.openstack.org/trunk/openstack-ops/content/nova-network-deprecation.html









Compute Node

Neutron API

Agent

Controller Neutron plugins

Nova Horizon UI

Compute Node

Agent

pSwitch

API, Agent

L4, F/W, VPN

Network as a Service - Neutron

API, Agent

Neutron Plugins

● Modular Layer 2

● OpenvSwitch

● VMware NSX

● Software Defined Networking

o OpenDaylight, Ryu

● MidoNet

● OpenContrail

Controller node

Keystone

Network node Compute node - 1 Compute node - 2

Nova

Glance Horizon

Neutron server

Neutron openvswitch-plugin

Nova compute

eth1 eth2

eth0

eth1 eth2

eth0

eth1 eth2

eth0

eth1 eth2

eth0

Management 192.168.20.0/24

Data 192.168.10.0/24

External network 192.168.122.0/24


Neutron metadata-agent

Neutron L3/dhcp-agent


Nova compute

OpenStack networking with Neutron

Compute node - 3

Compute node - 2

Network node

vRouter A

Network Virtualization

Compute node - 1

Subnet 1

Subnet 2

Subnet 4

vRouter B

vRouter C

vRouter D

Subnet 3

Tenant X

Tenant Y

Tenant Z Subnet 3

Subnet 4

Subnet 2

GRE/VxLAN Tunneling

Network Topology

● ext_net : external network - 192.168.122.0/24 ● net_proj_one : “user_one” tenant - 50.50.1.0/24 ● net_proj_two : “user_one” tenant - 50.50.2.0/24 ● net_proj_new : “user_new” tenant - 60.60.1.0/24

* LibvirtHybridOVSBridgeDriver

libvirt_vif_driver=nova.virt.libvirt.vif.LibvirtHybridOVSBridgeDriver

Network node

net_proj_one net_proj_two net_proj_new

Big picture - Neutron OVS plugin GRE

OpenStack Havana OpenvSwitch plug-in GRE tunneling - LibvirtGenericVIFDriver

Compute node - 1

br-ex

qg~

VM VM

br-

tun

tap~ tag: 1

tap~ tag:2

br-int

Tunnel

qg~

qg~

eth0

qr~

tap~ tap~ tap~

br-int

qr~ qr~

patch

patc

h

br-

tun

patc

h

gre

~

patch

Data 192.168.10.0/24

OVS port

OVS Bridge

● qg~~~ : external gateway interface ● qr~~~ : virtual router interface

Packet conversion

Neutron OVS plugin GRE - Compute node


Compute node - 1

VM VM

tap~ tag: 1

tap~ tag:2

Tunnel

br-

tun

patch

VM

tap~ tag:2

Security Group[1] set_tunnel id

mod_vlan_vid

VM

tap~ tag:3

br-int patch

Neutron OVS plugin GRE - Compute node

janghoon@compute-1:~$ sudo ovs-ofctl dump-flows br-tun

NXST_FLOW reply (xid=0x4):

cookie=0x0, duration=87770.027s, table=0, n_packets=0, n_bytes=0,

priority=3,tun_id=0x1,dl_dst=01:00:00:00:00:00/01:00:00:00:00:00 actions=mod_vlan_vid:1,output:1

cookie=0x0, duration=87770.09s, table=0, n_packets=8786, n_bytes=1893724, priority=4,in_port=1,dl_vlan=1

actions=set_tunnel:0x1,NORMAL


priority=3,tun_id=0x1,dl_dst=fa:16:3e:db:08:63 actions=mod_vlan_vid:1,NORMAL


priority=3,tun_id=0x1,dl_dst=fa:16:3e:e0:73:95 actions=mod_vlan_vid:1,NORMAL

cookie=0x0, duration=87771.753s, table=0, n_packets=2921, n_bytes=951454, priority=1 actions=drop

Packet conversion

Neutron OVS plugin GRE - Network node

janghoon@Network-node:~$ sudo ovs-ofctl dump-flows br-tun

NXST_FLOW reply (xid=0x4):













cookie=0x0, duration=472911.069s, table=0, n_packets=16335, n_bytes=3551350, priority=3,tun_id=0x4,dl_dst=fa:16:3e:89:fd:ce

actions=mod_vlan_vid:3,NORMAL


priority=3,tun_id=0x3,dl_dst=fa:16:3e:d8:d5:29 actions=mod_vlan_vid:2,NORMAL

cookie=0x0, duration=474674.351s, table=0, n_packets=525, n_bytes=52427, priority=3,tun_id=0x3,dl_dst=fa:16:3e:69:ca:97


cookie=0x0, duration=473162.912s, table=0, n_packets=197, n_bytes=19365, priority=3,tun_id=0x4,dl_dst=fa:16:3e:d6:b8:07


cookie=0x0, duration=633937.746s, table=0, n_packets=6207, n_bytes=630043, priority=3,tun_id=0x1,dl_dst=fa:16:3e:c7:ec:bd



priority=3,tun_id=0x1,dl_dst=fa:16:3e:8b:a6:d7 actions=mod_vlan_vid:1,NORMAL

cookie=0x0, duration=636252.069s, table=0, n_packets=163, n_bytes=36046, priority=1 actions=drop

Packet conversion

Namespcae Namespcae Namespcae

Neutron OVS plugin GRE - Network node


br-

tun

Tunnel

eth0

patc

h

gre

~

qr~

tap~

qg~

qr~

qg~

qr~

qg~

br-int

br-ex

patch

Packet conversion

mod_vlan_id

set_tunnel id

tap~ tap~

net_proj_one

net_proj_two

net_proj_new

Network node

Floating-IP(NAT)

Neutron OVS plugin Security Group - GRE

FORWARD

neutron-filter-top

neutron-openvswi-FORWARD

neutron-openvswi-local

neutron-openvswi-sg-chain

neutron-openvswi-iTAP_NUMBER

neutron-openvswi-oTAP_NUMBER

neutron-openvswi-sg-fallback

neutron-openvswi-sg-fallback

Security group is applied here

Neutron OVS plugin Security Group - GRE Chain neutron-openvswi-sg-chain (4 references)

target prot opt source destination

neutron-openvswi-i21767f1f-4 all -- 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-out tap21767f1f-45 --physdev-is-

bridged

neutron-openvswi-o21767f1f-4 all -- 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-in tap21767f1f-45 --physdev-is-

bridged

neutron-openvswi-i7903fd30-7 all -- 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-out tap7903fd30-74 --physdev-is-

bridged

neutron-openvswi-o7903fd30-7 all -- 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-in tap7903fd30-74 --physdev-is-

bridged

ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 Chain neutron-openvswi-i7903fd30-7 (1 references)


DROP all -- 0.0.0.0/0 0.0.0.0/0 state INVALID

RETURN all -- 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED

RETURN icmp -- 0.0.0.0/0 0.0.0.0/0

RETURN tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:22

RETURN udp -- 50.50.1.3 0.0.0.0/0 udp spt:67 dpt:68

neutron-openvswi-sg-fallback all -- 0.0.0.0/0 0.0.0.0/0

Chain neutron-openvswi-o7903fd30-7 (2 references)


DROP all -- 0.0.0.0/0 0.0.0.0/0 MAC ! FA:16:3E:DB:08:63

RETURN udp -- 0.0.0.0/0 0.0.0.0/0 udp spt:68 dpt:67

DROP all -- !50.50.1.2 0.0.0.0/0

DROP udp -- 0.0.0.0/0 0.0.0.0/0 udp spt:67 dpt:68

DROP all -- 0.0.0.0/0 0.0.0.0/0 state INVALID

RETURN all -- 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED

RETURN all -- 0.0.0.0/0 0.0.0.0/0

neutron-openvswi-sg-fallback all -- 0.0.0.0/0 0.0.0.0/0

[1] Note, OpenStack uses iptables rules on the TAP devices such as “tap~~” to implement security groups. However, Open vSwitch is not compatible with iptables rules that are applied directly on TAP devices that are connected to an Open vSwitch port.

Neutron OVS plugin NameSpace - GRE

janghoon@Network-node:~$ sudo ip netns exec qrouter-cf5fe7b7-8fab-45de-ab1c-c0cd404ebed0 ifconfig

lo Link encap:Local Loopback

inet addr:127.0.0.1 Mask:255.0.0.0

inet6 addr: ::1/128 Scope:Host

UP LOOPBACK RUNNING MTU:16436 Metric:1

qg-fa243f49-d6 Link encap:Ethernet HWaddr fa:16:3e:9f:4b:63

inet addr:192.168.122.50 Bcast:192.168.122.255 Mask:255.255.255.0

inet6 addr: fe80::f816:3eff:fe9f:4b63/64 Scope:Link

UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1

qr-bc654dc2-f1 Link encap:Ethernet HWaddr fa:16:3e:c7:ec:bd

inet addr:50.50.1.1 Bcast:50.50.1.255 Mask:255.255.255.0

inet6 addr: fe80::f816:3eff:fec7:ecbd/64 Scope:Link

UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1

janghoon@Network-node:~$ sudo ip netns exec qrouter-cf5fe7b7-8fab-45de-ab1c-c0cd404ebed0 route

Kernel IP routing table

Destination Gateway Genmask Flags Metric Ref Use Iface

default 192.168.122.1 0.0.0.0 UG 0 0 0 qg-fa243f49-d6

50.50.1.0 * 255.255.255.0 U 0 0 0 qr-bc654dc2-f1

192.168.122.0 * 255.255.255.0 U 0 0 0 qg-fa243f49-d6

Neutron OVS plugin Floating-IP(NAT) - GRE

janghoon@Network-node:~$ sudo ip netns show

qdhcp-4c2f2346-ffaa-41a0-ab76-34cadf0163f5

qrouter-e1b88ce4-51e9-4744-be80-d70d04c6a59b

qdhcp-c19e22a0-1700-4b3b-91e5-2c961ef0a353

qrouter-244fff3f-f935-4bdd-949d-739f1ce81dd0

qdhcp-f37b681a-4be8-47b8-8063-3d17d24ee1ae

qrouter-cf5fe7b7-8fab-45de-ab1c-c0cd404ebed0

janghoon@Network-node:~$ sudo ip netns exec qrouter-cf5fe7b7-8fab-45de-ab1c-c0cd404ebed0 iptables -L -n -t nat

Chain neutron-l3-agent-PREROUTING (1 references)


REDIRECT tcp -- 0.0.0.0/0 169.254.169.254 tcp dpt:80 redir ports 9697

DNAT all -- 0.0.0.0/0 192.168.122.51 to:50.50.1.2

Chain neutron-l3-agent-float-snat (1 references)


SNAT all -- 50.50.1.2 0.0.0.0/0 to:192.168.122.51

Chain neutron-l3-agent-snat (1 references)


neutron-l3-agent-float-snat all -- 0.0.0.0/0 0.0.0.0/0

SNAT all -- 50.50.1.0/24 0.0.0.0/0 to:192.168.122.50

Floating-IP(NAT)

NameSpace

Neutron ML2

The Modular Layer 2 (ML2) plugin is a framework allowing OpenStack Networking to simultaneously utilize the variety of layer 2 networking technologies found in complex real-world data centers. It currently works with the existing openvswitch, linuxbridge, and Hyper-V L2 agents, and is intended to replace and deprecate the monolithic plugins associated with those L2 agents.

Neutron

TypeDriver

VLAN

ML2 Plugin

GRE VxLAN Flat

MechanismDriver

OpenvS

witc

h

Hyper-

V

OpenD

aylig

ht

Aris

ta

Cis

co N

exu

s

pSwitch

TypeDriver : TypeDrivers maintain any needed type-specific network state, and perform provider network validation and tenant network allocation. MechanismDriver : The MechanismDriver is responsible for taking the information established by the TypeDriver and ensuring that it is properly applied given the specific networking mechanisms that have been enabled. https://wiki.openstack.org/wiki/Neutron/ML2

https://wiki.openstack.org/wiki/Neutron/ML2

https://wiki.openstack.org/wiki/Neutron/ML2

Neutron ML2 Installation

Network node Compute node - 1 Compute node - 2

Neutron ML2-agent Neutron

ML2-agent

Nova compute

eth0

eth1 eth2 eth1 eth2

eth0

eth1 eth2

eth0

Neutron server



Neutron ML2-agent

Nova compute

Management 192.168.20.0/24

External network 192.168.122.0/24

Data 192.168.10.0/24

Compute node - 3

Compute node - 2

Network node-1

L3 Agent

Neutron Multi network node

Compute node - 1

Tenant A

Tenant B

Tenant A

Tenant C

Tenant D

Tenant C

Network node-2

L3 Agent

Network node-2

Compute node - 3

Compute node - 2

Network node-1

vRouter A - Master

Neutron High Availability(L3 agent)

Compute node - 1

Subnet 1

Subnet 3

Subnet 2

Subnet 5

vRouter B - Backup

vRouter C - Backup

vRouter D - Master

vRouter C - Master

vRouter D - Backup

vRouter A - Backup

Subnet 3

Subnet 4

vRouter B - Master

Tenant X

Tenant Y

Tenant Z

VRRP

Network node-1

Neutron server

eth1 eth2

eth0

Neutron ML2 plugin



External network

Management

Data

KeepAlived

Network node-2

Neutron server

eth1 eth2

eth0

Neutron ML2 plugin



KeepAlived Compute node - 1

Nova compute

eth1 eth2

eth0

Neutron ML2 plugin

Compute node - 2

eth1 eth2

eth0

Neutron ML2 plugin

Nova compute


Namespace OVS bridge

Network node-1

qdhcp-

br-

tun

br-int

qrouter-

ha~

ns~ qr~ qg~

br-ex

Network node-2

qdhcp-

br-

tun

br-int

qrouter-

qr~ qg~

br-ex

ns~

KeepAlived KeepAlived

ha~

ubuntu@ubuntu-5:~$ sudo ip netns exec qrouter-d8625260-88a1-4312-b788-c04fc9094356 tcpdump -n -i ha-27fe59da-

a8

tcpdump: verbose output suppressed, use -v or -vv for full protocol decode

listening on ha-27fe59da-a8, link-type EN10MB (Ethernet), capture size 65535 bytes

16:16:25.213440 IP 169.254.192.2 > 224.0.0.18: VRRPv2, Advertisement, vrid 1, prio 50, authtype none, intvl 2s, length 20





Network node-1

qdhcp-

br-

tun

eth0

br-int patch-tun

patc

h-in

t

qrouter-

tap tap tap

ha~ ns~ qr~

qg~

br-ex

tap

Network node-2

qdhcp-

br-

tun

eth0

br-int patch-tun

patc

h-in

t

gre

~

qrouter-

tap tap tap

ha~ ns~ qr~

qg~

br-ex

tap


gre

~


Network node-1

qdhcp-

br-

tun

eth0

br-int patch-tun

patc

h-in

t

qrouter-

tap tap tap

ha~ ns~ qr~

qg~

br-ex

tap

Network node-2

qdhcp-

br-

tun

eth0

br-int patch-tun

patc

h-in

t

gre

~

qrouter-

tap tap tap

ha~ ns~ qr~

qg~

br-ex

tap


gre

~



Network node-1

qdhcp-

br-

tun

br-int

qrouter-

ha~

ns~ qr~ qg~

br-ex

KeepAlived

ubuntu@ubuntu-5:~$ cat /var/lib/neutron/ha_confs/d8625260-88a1-4312-b788-c04fc9094356/keepalived.conf vrrp_sync_group VG_1 { group { VR_1 } notify_master "/var/lib/neutron/ha_confs/d8625260-88a1-4312-b788-c04fc9094356/notify_master.sh" notify_backup "/var/lib/neutron/ha_confs/d8625260-88a1-4312-b788-c04fc9094356/notify_backup.sh" notify_fault "/var/lib/neutron/ha_confs/d8625260-88a1-4312-b788-c04fc9094356/notify_fault.sh" } vrrp_instance VR_1 { state BACKUP interface ha-27fe59da-a8 virtual_router_id 1 priority 50 nopreempt advert_int 2 track_interface { ha-27fe59da-a8 } virtual_ipaddress { 192.168.10.118/24 dev qg-8fffbd7e-8a } virtual_ipaddress_excluded { 50.50.1.1/24 dev qr-dee474e1-1e } virtual_routes { 0.0.0.0/0 via 192.168.10.51 dev qg-8fffbd7e-8a } }


Network node Tenant A


br-

tun

br-int

qrouter-

ha~

br-ex

KeepAlived

qrouter-

ha~

KeepAlived

HA network : 169.254.192.1 ~ 254

Segmentation id : 0x6

Tenant B

qrouter-

ha~

KeepAlived

qrouter-

ha~

KeepAlived

HA network : 169.254.192.1 ~ 254

Segmentation id : 0x7

● One KeepAlived instance per vRouter

● One HA network per tenant

○ Each HA network has separate

segmentation id

○ allow_overlapping_ips = True

● Maximum 255 HA routers per tenant.


DVR (Distributed Virtual Router) - Installation

Network node

Neutron server

eth1 eth2

eth0

Neutron ML2 plugin



External network

Compute node - 1

Nova compute

eth1 eth2

eth0

Neutron ML2 plugin


Neutron L3-agent

Management

Data

Compute node - 2

Nova compute

eth1 eth2

eth0

Neutron ML2 plugin


Neutron L3-agent

DVR (Distributed Virtual Router) - Packet flow

Compute node - 1

br-

ex

VM VM

GRE Tunnel

VM

br-int

Network node

br-

ex

br-tun

br-int

br-

tun

Compute node - 2

VM VM VM

br-int

br-

tun

1.SNAT

External network

3. East-West traffic

2. Floating IP

OVS bridge

DVR (Distributed Virtual Router) - SNAT : Network node

Namespace

OVS bridge Network node

qdhcp- br-

tun

eth0

br-int patch-tun

patc

h-in

t

gre

~

snat- qrouter-

tap tap tap

sg~ 50.50.6.

2 ns~ qr~

qg~ 192.168.10.109

SNAT br-ex

tap

packet flow

DVR (Distributed Virtual Router) - SNAT : Compute node

Compute node

Namespace

OVS bridge

VM

br-int

br-

tun

qvo~

qbr~

tap~

qvb~

patch-int

qrouter-

qr~ 50.50.6.

1

patch-tun

tap~

Linux bridge

sg~

on

netw

ork

node

packet flow

traffic flow

DVR (Distributed Virtual Router) - SNAT : Compute node

Namespace

OVS bridge

Linux bridge

Compute node

VM

br-int

br-

tun

qvo~

qbr~

tap~

qvb~

patch-int

qrouter-

qr~ 50.50.6.

1

patch-tun

tap~ sg~

(50.5

0.6

.2)

on

netw

ork

node

ubuntu@ubuntu-8:~$ sudo ip netns exec qrouter-

20838b7d-a7ac-4da9-92aa-adec797d600e ip rule

ls

0: from all lookup local

32766: from all lookup main

32767: from all lookup default

842139137: from 50.50.6.1/24 lookup

842139137

ubuntu@ubuntu-8:~$ sudo ip netns exec qrouter-

20838b7d-a7ac-4da9-92aa-adec797d600e ip route

show table 842139137

default via 50.50.6.2 dev qr-9722faba-b7

DVR (Distributed Virtual Router) - Floating IP/DNAT : Compute node

Compute node

Namespace

OVS bridge

VM

br-int

br-

tun

qvo~

qbr~

tap~

qvb~

patch-int

qrouter-

qr~ 50.50.6.

1

patc

h-tu

n

tap~

Linux bridge

packet flow

br-ex

tap

eth0

fip-

fpr~ rfp~

fg~

Route Route

NAT

veth pair


Compute node

Namespace

OVS bridge

VM

br-int

br-

tun

qvo~

qbr~

tap~

qvb~

patch-int

qrouter-

qr~ 50.50.6.

1

patc

h-tu

n

tap~

Linux bridge

packet flow

br-ex

tap

eth0

fip-

fpr~ rfp~

fg~

Route Route

NAT

veth pair

ubuntu@ubuntu-6:~$ sudo ip netns exec qrouter-20838b7d-a7ac-4da9-

92aa-adec797d600e ip rule ls

0: from all lookup local

32766: from all lookup main

32767: from all lookup default

32770: from 50.50.5.5 lookup 16

842138881: from 50.50.5.1/24 lookup 842138881

842138881: from 50.50.5.1/24 lookup 842138881

842139137: from 50.50.6.1/24 lookup 842139137

ubuntu@ubuntu-6:~$ sudo ip netns exec qrouter-20838b7d-a7ac-4da9-

92aa-adec797d600e ip route show table 16

default via 169.254.31.29 dev rfp-20838b7d-a


Compute node

Namespace

OVS bridge

VM

br-int

br-

tun

qvo~

qbr~

tap~

qvb~

patch-int

qrouter-

qr~ 50.50.6.

1

patc

h-tu

n

tap~

Linux bridge

packet flow

br-ex

tap

eth0

fip-

fpr~ rfp~

fg~

Route Route

NAT

veth pair

ubuntu@ubuntu-6:~$ sudo ip netns exec fip-02f9d340-

2caa-4c05-86fb-460c9580f9df ip route show

default via 192.168.10.1 dev fg-f3887d61-2d

192.168.10.114 via 169.254.31.28 dev fpr-20838b7d-a

DVR (Distributed Virtual Router) - East-West traffic flow : Compute node

Compute node-2


VM 50.50.6.3

br-int

br-

tun

qvo~

qbr~

tap~

qvb~

patch-int

qrouter-

qr~ 50.50.6.1

patch-tun

tap~

Linux bridge packet flow

Compute node-1

tap~

qr~ 50.50.5.1

VM 50.50.5.3

br-int

br-

tun

qvo~

qbr~

tap~

qvb~

patch-int

qrouter-

qr~ 50.50.6.1

patch-tun

tap~ tap~

qr~ 50.50.5.1

ICMP Request

ICMP Reply

i.e., ping 50.50.5.3 -> 50.50.6.3

DVR (Distributed Virtual Router) - East-West traffic flow : network topology


Compute node-2


br-int

br-

tun

qvo~

qbr~

tap~

qvb~

patch-int

qrouter-

qr~ 50.50.6.1

patch-tun

tap~


Compute node-1

tap~

qr~ 50.50.5.1

br-int

br-

tun

qvo~

qbr~

tap~

qvb~

patch-int

qrouter-

qr~ 50.50.6.1

patch-tun

tap~ tap~

qr~ 50.50.5.1

VM 50.50.6.3

ubuntu@ubuntu-6:~$ sudo ip netns exec qrouter-20838b7d-

a7ac-4da9-92aa-adec797d600e ip link

2: qr-ecffa2a6-dd: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu

1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen

1000

link/ether fa:16:3e:15:1e:e0 brd ff:ff:ff:ff:ff:ff

5: qr-9722faba-b7: <BROADCAST,MULTICAST,UP,LOWER_UP>

mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default

qlen 1000

link/ether fa:16:3e:71:3d:5a brd ff:ff:ff:ff:ff:ff

ubuntu@ubuntu-8:~$ sudo ip netns exec qrouter-20838b7d-

a7ac-4da9-92aa-adec797d600e ip link

2: qr-ecffa2a6-dd: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu

1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen

1000

link/ether fa:16:3e:15:1e:e0 brd ff:ff:ff:ff:ff:ff

5: qr-9722faba-b7: <BROADCAST,MULTICAST,UP,LOWER_UP>

mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default

qlen 1000

link/ether fa:16:3e:71:3d:5a brd ff:ff:ff:ff:ff:ff

VM 50.50.5.3


Compute node-2


VM 50.50.6.3

br-int

br-

tun

qvo~

qbr~

tap~

qvb~

patch-int

qrouter-

qr~ 50.50.6.1

patch-tun

tap~


Compute node-1

tap~

qr~ 50.50.5.1

VM 50.50.5.3

br-int

br-

tun

qvo~

qbr~

tap~

qvb~

patch-int

qrouter-

qr~ 50.50.6.1

patch-tun

tap~ tap~

qr~ 50.50.5.1

ICMP Request 50.50.5.3 -> 50.50.6.3

Segmentation ID : 50.50.5.0/24 : 0x1 50.50.6.0/24 : 0x3

MAC 50.50.6.3 : fa:16:3e:ff:85:9b 50.50.6.1 : fa:16:3e:71:3d:5a 50.50.5.1 : fa:16:3e:15:1e:e0 50.50.5.3 : fa:16:3e:ce:8c:35

DVR Host MAC : Compute Node -1 : fa:16:3f:5e:a0:cf Compute Node -2 : fa:16:3f:72:60:33

SRC MAC :

fa:16:3e:71:3d:5a

SRC IP : 50.50.5.3

DST MAC : fa:16:3e:ff:85:9b

DST IP : 50.50.6.3

SRC MAC :

fa:16:3e:71:3d:5a

SRC IP : 50.50.5.3


DST IP : 50.50.6.3

SRC MAC :

fa:16:3e:ce:8c:35

SRC IP : 50.50.5.3

DST MAC :

fa:16:3e:15:1e:e0

DST IP : 50.50.6.3

GRE tunnel 0x3

SRC MAC : fa:16:3f:5e:a0:cf

SRC IP : 50.50.5.3


DST IP : 50.50.6.3


Compute node-2


VM 50.50.6.3

br-int

br-

tun

qvo~

qbr~

tap~

qvb~

patch-int

qrouter-

qr~ 50.50.6.1

patch-tun

tap~


Compute node-1

tap~

qr~ 50.50.5.1

VM 50.50.5.3

br-int

br-

tun

qvo~

qbr~

tap~

qvb~

patch-int

qrouter-

qr~ 50.50.6.1

patch-tun

tap~ tap~

qr~ 50.50.5.1

ICMP Reply 50.50.6.3 -> 50.50.5.3

SRC MAC :

fa:16:3e:15:1e:e0

SRC IP : 50.50.6.3


DST IP : 50.50.5.3

SRC MAC :

fa:16:3e:15:1e:e0

SRC IP : 50.50.6.3

DST MAC :

fa:16:3e:ce:8c:35

DST IP : 50.50.5.3

SRC MAC : fa:16:3e:ff:85:9b

SRC IP : 50.50.6.3

DST MAC :

fa:16:3e:71:3d:5a

DST IP : 50.50.5.3




GRE tunnel 0x1

SRC MAC : fa:16:3f:72:60:33

SRC IP : 50.50.6.3

DST MAC :

fa:16:3e:ce:8c:35

DST IP : 50.50.5.3


Compute node-2


VM 50.50.6.3

br-int

br-

tun

qvo~

qbr~

tap~

qvb~

patch-int

qrouter-

qr~ 50.50.6.1

patch-tun

tap~


Compute node-1

tap~

qr~ 50.50.5.1

VM 50.50.5.3

br-int

br-

tun

qvo~

qbr~

tap~

qvb~

patch-int

qrouter-

qr~ 50.50.6.1

patch-tun

tap~ tap~

qr~ 50.50.5.1

ICMP Request 50.50.5.3 -> 50.50.6.3




table=0, n_packets=9178, n_bytes=1009035, idle_age=17470, hard_age=65534, priority=1 actions=NORMAL

table=0, n_packets=2066, n_bytes=214544, idle_age=5, hard_age=65534, priority=1,in_port=1 actions=resubmit(,1)

table=1, n_packets=1765, n_bytes=172970, idle_age=5, hard_age=65534, priority=1,dl_vlan=2,dl_src=fa:16:3e:71:3d:5a

actions=mod_dl_src:fa:16:3f:5e:a0:cf,resubmit(,2)

table=2, n_packets=1849, n_bytes=183458, idle_age=5, hard_age=65534, priority=0,dl_dst=00:00:00:00:00:00/01:00:00:00:00:00

actions=resubmit(,20)

table=20, n_packets=1765, n_bytes=172970, idle_age=5, hard_age=65534, priority=2,dl_vlan=2,dl_dst=fa:16:3e:ff:85:9b

actions=strip_vlan,set_tunnel:0x3,output:3


Compute node-2


VM 50.50.6.3

br-int

br-

tun

qvo~

qbr~

tap~

qvb~

patch-int

qrouter-

qr~ 50.50.6.1

patch-tun

tap~


Compute node-1

tap~

qr~ 50.50.5.1

VM 50.50.5.3

br-int

br-

tun

qvo~

qbr~

tap~

qvb~

patch-int

qrouter-

qr~ 50.50.6.1

patch-tun

tap~ tap~

qr~ 50.50.5.1

ICMP Request 50.50.5.3 -> 50.50.6.3




table=0, n_packets=1789, n_bytes=175146, idle_age=17, hard_age=65534, priority=2,in_port=3,dl_src=fa:16:3f:5e:a0:cf actions=resubmit(,1) table=1, n_packets=1765, n_bytes=172970, idle_age=17, hard_age=65534, priority=4,dl_vlan=2,dl_dst=fa:16:3e:ff:85:9b actions=strip_vlan,mod_dl_src:fa:16:3e:71:3d:5a,output:8


table=3, n_packets=1993, n_bytes=195880, idle_age=18, hard_age=65534, priority=1,tun_id=0x3 actions=mod_vlan_vid:2,resubmit(,9)

table=9, n_packets=1789, n_bytes=175146, idle_age=18, hard_age=65534, priority=1,dl_src=fa:16:3f:5e:a0:cf actions=output:1


Compute node-2


VM 50.50.6.3

br-int

br-

tun

qvo~

qbr~

tap~

qvb~

patch-int

qrouter-

qr~ 50.50.6.1

patch-tun

tap~


Compute node-1

tap~

qr~ 50.50.5.1

VM 50.50.5.3

br-int

br-

tun

qvo~

qbr~

tap~

qvb~

patch-int

qrouter-

qr~ 50.50.6.1

patch-tun

tap~ tap~

qr~ 50.50.5.1

ICMP Request 50.50.5.3 -> 50.50.6.3




table=0, n_packets=1789, n_bytes=175146, idle_age=17, hard_age=65534, priority=2,in_port=3,dl_src=fa:16:3f:5e:a0:cf actions=resubmit(,1) table=1, n_packets=1765, n_bytes=172970, idle_age=17, hard_age=65534, priority=4,dl_vlan=2,dl_dst=fa:16:3e:ff:85:9b actions=strip_vlan,mod_dl_src:fa:16:3e:71:3d:5a,output:8


table=3, n_packets=1993, n_bytes=195880, idle_age=18, hard_age=65534, priority=1,tun_id=0x3 actions=mod_vlan_vid:2,resubmit(,9)

table=9, n_packets=1789, n_bytes=175146, idle_age=18, hard_age=65534, priority=1,dl_src=fa:16:3f:5e:a0:cf actions=output:1

Open Virtual Network project - OVN

● At present, ○ Packet switching -> Linux Bridge, OpenvSwitch ○ Routing -> Policy routing, routing table ○ Security -> iptables, ebtables

● OVN complements the existing capabilities of OpenvSwitch to add native support for virtual network abstractions, such as virtual L2 and L3 overlays and security groups.

● OVN will include logical switches and routers, security groups, and L2/L3/L4 ACLs, implemented on top of a tunnel-based (VXLAN, NVGRE, Geneve, STT, IPsec) overlay network.

Open Virtual Network project - OVN

Compute node

ovs-vswitchd ovsdb-server

ovn-controller

OVN-DB

OVN-Northbound DB

ovs-nbd

OVN plug-in OpenStack (Neutron)

Compute node

ovs-vswitchd ovsdb-server

ovn-controller

OpenFlow OVSDB protocol

OVSDB protocol

OpenFlow

OVSDB protocol

OVSDB protocol

Documents

OpenStack Neutron NetworkingBD%C9%C0%E5%C8%C6.pdf · With the introduction of the full software-defined networking stack provided by OpenStack Networking (neutron) in the Folsom release,