Docker Networking Tutorial – CoreOS Flannel

Srini Seetharamansrini@sdnhub.org

November, 2014

CoreOS Flannel

• Lightweight OS based on Gentoo Linux

• Has a distributed key-value store at the core

• Read-only rootfs. Writeable /etco All services are in containers

CoreOS

• One CIDR subnet per machine, like Kuberneteso Host 1: 10.10.10.0/24

o Host 2: 10.10.11.0/24

• No Docker port-based mapping

• Containers reach each other through IP

• Peer network configs exchanged over etcd

• Packets encapsulated using UDP, and soon VxLAN

Flannel Basic

4

1. Build flannel on each host

2. Set key in etcd for network config

Instructions to Run Flannel

6

$ curl -L http://127.0.0.1:4001/v2/keys/coreos.com/network/config -XPUT -d value='{

"Network": "10.0.0.0/8","SubnetLen": 20,"SubnetMin": "10.10.0.0","SubnetMax": "10.99.0.0","Backend": {"Type": "udp",

"Port": 7890}}

$ git clone https://github.com/coreos/flannel.git$ cd flannel$ docker run -v `pwd`:/opt/flannel -i -t google/golang /bin/bash -c "cd /opt/flannel && ./build"

3. Start flannel. o flanneld port created and route is set for the full flat IP range.

Instructions to Run Flannel (contd.)

7

$ sudo ./bin/flanneld &

Output:I1219 17:34:41.159822 00809 main.go:247] Installing signal handlersI1219 17:34:41.160030 00809 main.go:118] Determining IP address of default interfaceI1219 17:34:41.160579 00809 main.go:205] Using 192.168.111.14 as external interfaceI1219 17:34:41.212157 00809 subnet.go:83] Subnet lease acquired: 10.12.224.0/20I1219 17:34:41.217829 00809 main.go:215] UDP mode initializedI1219 17:34:41.218953 00809 udp.go:239] Watching for new subnet leasesI1219 17:34:41.219349 00809 udp.go:264] Subnet added: 10.13.128.0/20

core@coreos-05 ~ $ route -nKernel IP routing tableDestination Gateway Genmask Flags Metric Ref Use Iface0.0.0.0 192.168.111.1 0.0.0.0 UG 1024 0 0 eth010.0.0.0 0.0.0.0 255.0.0.0 U 0 0 0 flannel010.12.224.0 0.0.0.0 255.255.240.0 U 0 0 0 docker0192.168.111.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0

4. Restart docker daemon with appropriate bridge IP

Instructions to Run Flannel (contd.)

8

$ source /run/flannel/subnet.env$ sudo ifconfig docker0 ${FLANNEL_SUBNET}$ sudo docker -d --bip=${FLANNEL_SUBNET} --mtu=${FLANNEL_MTU} &

• Ping between two bash containers on two different hosts succeeds. The traffic on wire is encapsulated by flanneld

Testing Flannel Networking

9

192.168.111.14

Docker0

10.12.224.1

bash

192.168.111.13

Docker0

10.13.128.1

bash

$ docker run -i -t ubuntu /bin/bashroot@36484def3b03:/# ifconfig eth0eth0 Link encap:EthernetHWaddr 02:42:0a:0c:e0:02inet addr:10.12.224.2 Bcast:0.0.0.0Mask:255.255.240.0

root@36484def3b03:/# ping 10.13.128.2Success!

$ docker run -i -t ubuntu /bin/bashroot@e0b9dd20d146:/# ifconfig eth0eth0 Link encap:EthernetHWaddr 02:42:0a:0d:80:02inet addr:10.13.128.2Bcast:0.0.0.0Mask:255.255.240.0

Packet on the Wire

10

Original ICMP

packet between

the two containers

Flannel

introduced encap

UDP header

• IP address overlap not possibleo VxLAN not used to create container groups

• User-space encapsulation and forwardingo Potential performance bottleneck

Limitations

11

Thank you.

https://github.com/sdnhub/lorispack© 2015 Copyright Reserved