1

Attilla de Groot

Host multitenancy

Attilla de Groot | Sr. Systems Engineer, HCIE #3494 | Cumulus Networks

EVPN to the host

2

Agenda

EVPN to the Host§ Multi tenancy use cases§ Deployment issues§ Host integration with EVPN§ Caveats and future work

3

Multi tenancy use casesVirtualization environments

VM environments§ MLAG to hypervisors§ Vxlan between hosts§ Double overlay§ Dedicated network nodes

Mlag

VM1 VM1

vswitch1 vswitch2

VM1 VM1

vswitch1 vswitch2 vswitch1 vswitch2

vrouter

hypervisor1 hypervisor2 networknode

Mlag

BGP BGP

4

Multi tenancy use casesVM deployment issues

Architecture issues§ MLAG to hypervisors§ Vxlan between hosts§ Double overlay§ Dedicated network nodes§ Orchestration problems

Mlag

VM1 VM1

vswitch1 vswitch2

hypervisor1

bare metal host

Mlag

OVSDBML2

Controller

5

Multi tenancy use casesContainer environments

Container environments§ BGP to hosts§ Host route advertisement § Docker management§ Container overlay

Container1

containervisor1 containervisor2

Container2 Container3 Container4

RoutingDaemon

IP fabric

RoutingDaemon

BGP

BGP BG

P

BGP

ACLsACLs

6

Multi tenancy use casesContainer deployment issues

Architecture issues§ Host networking§ Multi-tenancy§ IP prefix overlap§ ACL management

containervisor

RoutingDaemon

BGP

BGP

ACLs

Container210.1.1.1/32

2001:DB8::1/128

Container110.1.1.1/32

2001:DB8::1/128

tenant1 tenant2

7

Host integration with EVPNOpen standards

EVPN on hosts§ Vlan aware bridge§ VRF in Linux kernel§ VxLAN§ Free Range Routing with EVPN§ Ifupdown2§ Iproute2

cumulus@server01:~$ uname -aLinux server01 4.17.0-041700-generic #201806041953 SMP Mon Jun 4 19:55:25 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

cumulus@server01:~$ vrf list

VRF Table---------------- -----mgmt 1001tenant3 1002tenant2 1003tenant1 1004

8

Host integration with EVPNHost connectivity

Host connectivity§ Routing to hosts§ BGP unnumbered§ RFC5549§ Loopback advertisement§ EVPN address family

auto eth1iface eth1

auto eth2iface eth2

router bgp 65501bgp router-id 10.250.250.1bgp bestpath as-path multipath-relaxneighbor FABRIC peer-groupneighbor FABRIC remote-as externalneighbor FABRIC timers 1 3neighbor eth1 interface peer-group FABRICneighbor eth2 interface peer-group FABRIC!address-family ipv4 unicastredistribute connected route-map loopbacksexit-address-family!address-family l2vpn evpnneighbor FABRIC activateadvertise-all-vniadvertise ipv4 unicastadvertise ipv6 unicast

exit-address-family!route-map loopbacks permit 10match interface lo

interface eth1ipv6 nd ra-interval 10no ipv6 nd suppress-rainterface eth2ipv6 nd ra-interval 10no ipv6 nd suppress-ra

9

Host integration with EVPNL2 tenancy

L2 tenancy§ Vlan-aware bridge§ ARP / ND suppression § L2VNIs§ EVPN Type-2

vlan xxxVM1 eth0 vethX

Vlan aware bridge

Hypervisor1

vlan xxx

Vlan aware bridge

Hypervisor2

VM2vethX eth0

vni xxx vni xxxVxlan tunnel

assigned to local vlan

Vxlan tunnel

assigned to local vl

an

BGPdFRR

BGPdFRR

learn

ed m

acs

adde

d to f

db

learned macs

added to fdb

EVPN Type 2,3 (VNI & MAC/IP) exchange.Messages via spine/leaf nodes

10

Host integration with EVPNDistributed routing

Distributed routing§ Gateway location§ SVIs on host§ Anycast gateway§ Local host routing

vlan yyy

Hypervisor1

VM2

vethX

vrf xxx

vlan xxx

vethX

VM1

eth010.1.1.10/24

svi10.1.1.1/24

vrf yyyl3vni l3vni

eth1 eth2

svi10.1.2.1/24

l2vnil2vni

eth010.1.2.10/24

vlan yyy

Hypervisor2

VM2

vethX

vrf xxx

vlan xxx

vethX

VM1

eth010.1.1.11/24

svi10.1.1.1/24

vrf yyyl3vni l3vni

eth1 eth2

svi10.1.2.1/24

l2vnil2vni

eth010.1.2.11/24

11

Host integration with EVPNL3 tenancy

L3 tenancy§ VRFs on host§ Prefix advertisement§ L3VNIs§ EVPN Type-5

vlan xxxVM1 eth0 vethX

Vlan aware bridge

Hypervisor1

vlan yyy

Vlan aware bridge

Hypervisor2

VM2vethX eth0

Vxlan tunnel

assigned to local VRF

Vxlan tunnel

assigned to local VRF BGPd FRR BGPdFRR

learned macs/ips

added to fd

b and

routing table

learned macs/ips

added to fdb and

routing table

EVPN Type 2,3,5 (VNI, MAC/IP & prefix route) exchange.Messages via spine/leaf nodes

vrf xxxvrf xxx l3vni xxx

svi xxx

l3vni xxx

svi yyy

12

Host integration with EVPNContainer integration

Container integration§ Container IP redistribution§ Host route advertisement§ Prefix overlap§ No ACLs for tenant segregation

Containervisor

container2

vrf xxx

container1

eth010.1.1.10

vrf yyyl3vni l3vni

eth010.1.2.10

Dockerplugin

FRR

routing table X

routing table Y

13

Host integration with EVPNBare metal integration

Bare metal integration§ Integration with RFC standard§ L2 stretching§ L3 tenancy§ Distributed routing

VM1 VM2

hypervisor1

bare metal host

Mlag

FRR

L2VNIL2VNI

BGP

BGP

L2VNI L3VNI L2VNI L3VNI

VTEPAnycast

14

Future workProof of technology

Proof of technology§ Commercial support

§ Software / tools availability

Linux kernel

FRR

§ Demo

cumulus@server01:~$ vrf list

VRF Table

---------------- -----

mgmt 1001

tenant3 1002

tenant2 1003

tenant1 1004

cumulus@server01:~$ bridge vlan

port vlan ids

bridge 1 PVID Egress Untagged

1010

1011

1012

1013

1014

cumulus@server01:~$ uname -a

Linux server01 4.17.0-041700-generic

#201806041953 SMP Mon Jun 4 19:55:25 UTC

2018 x86_64 x86_64 x86_64 GNU/Linux

cumulus@server01:~$ sudo vtysh -c "show bgp evpn vni"

VNI Type RD Import RT Export RT Tenant VRF

* 101010 L2 10.250.250.1:5 65501:101010 65501:101010 tenant1

* 101012 L2 10.250.250.1:3 65501:101012 65501:101012 tenant2

* 101014 L2 10.250.250.1:7 65501:101014 65501:101014 tenant3

* 404001 L3 172.30.11.1:8 65501:404001 65501:404001 tenant1

* 404002 L3 172.30.13.1:9 65501:404002 65501:404002 tenant2

* 404003 L3 172.30.15.1:10 65501:404003 65501:404003 tenant3

https://gitlab.com/CumulusNetworks/evpn-to-the-host

15

Future workOrchestration & Caveats

Orchestration§ Openstack neutron§ Kubernetes / Swarm§ Host / Network management

Head-end replication§ BUM anycast§ Merchant silicon limitations§ Multicast replication§ PIM-SM

16

Future workRoute leaking & Micro segmentation

Route-leaking§ Inter tenant traffic§ Service tenant§ FRR implementation

Micro segmentation§ Host traffic filtering§ Filtering with BPF§ Flowspec for ACL distribution

17

EVPN to host demo

18

EVPN to the Host

Questions ?

19

Thank you!Visit us at cumulusnetworks.com or follow us @cumulusnetworks

© 2018 Cumulus Networks. Cumulus Networks, the Cumulus Networks Logo, and Cumulus Linux are trademarks or registered trademarks of Cumulus Networks, Inc. or its affiliates in the U.S. and other countries. Other names may be trademarks of their respective owners. The registered trademark

Linux® is used pursuant to a sublicense from LMI, the exclusive licensee of Linus Torvalds, owner of the mark on a world-wide basis.