OpenStack networking-sfc Flow 분석

YongYoon.SHINProgrammable Infra Research Section, ETRI

http://uni2u.tistory.com

What a networking-sfc... ^-_-;;

Let’s Talk

• OpenStack Environment

• networking-sfc implementation

• Test

– test case 01

– test case 02

OpenStack Environment

Controller Node(Controller+Network)

eth0

eth1 eth2

br-ex

br-int br-tun

Compute Node 01(Compute)

eth0

eth1 eth2

br-int br-tun

Compute Node 02(Compute)

eth0

eth1 eth2

br-int br-tun

External Network

Management Network

Data Network (Tunnel)

What is Service Function Chaining

SF 01 SF 02 SF 03eth0 eth0eth0 eth1

qbr0 qbr3qbr1 qbr2

qvb0 qvb3qvb1 qvb2

vnet0 vnet3vnet1 vnet2

br-intPort vlan tag A

qvo0 qvo1

int-br-th1

br-eth1

pth-br-th1

eth1

Compute Node 01

SF 04 SF 05eth0eth0 eth1

qbr6qbr4 qbr5

qvb6qvb4 qvb5

vnet6vnet4 vnet5

br-intPort vlan tag C

qvo6qvo4 qvo5

int-br-th1

br-eth1

pth-br-th1

eth1

Compute Node 02

patch-tun

br-tun

patch-int

eth2

patch-tun

br-tun

patch-int

eth2

Port vlan tag B

qvo3qvo2

networking-sfc install

• OpenStack Liberty version

– OVS 2.4.0

• Download networking-sfc

• install networking-sfc

• lunch networking-sfc

$ git clone https://github.com/openstack/networking-sfc.git -b stable/liberty

$ cd networking-sfc$ sudo pip install -e /home/{user}/networking-sfc$ neutron-db-manage --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --subproject networking-sfc upgrade head

# cd networking-sfc# sudo python setup.py install# sudo cp /usr/local/bin/neutron-openvswitch-agent /usr/bin/neutron-openvswitch-agent

제약사항

• must create one virtual router

• SFC vm image

– Lubuntu 버전 dual_nic 이름

• $ sudo sysctl -w net.ipv4.ip_forward=1

• $ sudo iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE

• $ sudo iptables -A FORWARD -i eth0 -o eth1 -m state --state RELATED,ESTABLISHED -j ACCEPT

• $ sudo iptables -A FORWARD -i eth1 -o eth0 -j ACCEPT

Virtual Environment

• Network (from horizon)– External network

• 129.254.194.0/24

– Virtual network• 100.0.0.0/24 (net100)

– Virtual Router• External to net100

• Sf vm add interface (from controller Node)– Controller Node CLI– Neutron port-create --name p1 net100– Neutron port-create --name p2 net100– Neutron port-create --name p3 net100– Neutron port-create --name p4 net100– Neutron port-create --name p5 net100– Neutron port-create --name p6 net100– Nova boot --image cirros --nic port-id={p1_ID} vm1 --flavor m1.tiny– Nova boot --image {forwarding_image} --nic port-id={p2_ID} --nic port-id={p3_ID} sf1 --flavor m1.tiny– Nova boot --image {forwarding_image} --nic port-id={p4_ID} --nic port-id={p5_ID} sf2 --flavor m1.tiny– Nova boot --image cirros --nic port-id={p6_ID} vm2 --flavor m1.tiny

Horizon을 통하여 외부네트워크/가상네트워크/라우터 를 생성함

Controller Node에 접속해서 CLI로 포트 및 VM 생성함• Horizon에서 포트는 같은 서브넷에 대하여 VM이 바인딩

될 하나의 포트만 생성됨• Horizon에서 VM을 생성하면 서로 다른 서브넷이 아닌이상

NIC 2개를 할당 받기 위해 복잡한 절차를 거쳐야 함

• {Port_ID} 가 뒤바뀌지 않도록 주의!• Image Name 확인• vm 이름 확인

• external: external• subnet: ext_sub• 129.254.194.0/24

• network: net100• subnet: sub100• 100.0.0.0/24

• virtual router: vr• setgateway: external• interface: net100

Basic Environment (From Horizon)

SFC Environment (From CLI)

• Create neutron portsdn@controller:~$ neutron port-create --name p1 net100Created a new port:+-----------------------+-----------------------------------------------------------------------------------------------------+| Field | Value |+-----------------------+-----------------------------------------------------------------------------------------------------+| admin_state_up | True || allowed_address_pairs | || binding:host_id | || binding:profile | {} || binding:vif_details | {} || binding:vif_type | unbound || binding:vnic_type | normal || device_id | || device_owner | || dns_assignment | {"hostname": "host-100-0-0-9", "ip_address": "100.0.0.9", "fqdn": "host-100-0-0-9.openstacklocal."} || dns_name | || fixed_ips | {"subnet_id": "acbd4abe-3895-412b-accb-283499c280b9", "ip_address": "100.0.0.9"} || id | 6c5c6a24-ec62-4964-99fa-aeb348c7ed81 || mac_address | fa:16:3e:bd:c2:a6 || name | p1 || network_id | 19bac03e-92a1-4f7f-b6da-c9159cbec0c0 || port_security_enabled | True || security_groups | b2d26c2d-4f23-4398-91ba-b605b892a941 || status | DOWN || tenant_id | 704e6c364c224841ade1122551a506fc |+-----------------------+-----------------------------------------------------------------------------------------------------+

sdn@controller:~$ neutron port-create --name {p2~p6} net100

Controller Node에서 CLI 실행• Horizon을 통해 만든 net100에 포트 생성

Horizon (create port)

net100에 생성된 포트 확인• vm이 바인딩 되지 않았기 때문에 ‘연결 해제됨＇

• Virtual Router와 연결된 인터페이스 포트• DHCP 인터페이스 포트• p1~p6 까지 CLI로 생성한 포트

• 각 포트를 선택하면 포트ID 확인 가능

SFC Environment (From CLI)sdn@controller:~$ nova boot --image cirros --nic port-id=6c5c6a24-ec62-4964-99fa-aeb348c7ed81 src --flavor m1.tiny+--------------------------------------+-----------------------------------------------+| Property | Value |+--------------------------------------+-----------------------------------------------+| OS-DCF:diskConfig | MANUAL || OS-EXT-AZ:availability_zone | || OS-EXT-SRV-ATTR:host | - || OS-EXT-SRV-ATTR:hypervisor_hostname | - || OS-EXT-SRV-ATTR:instance_name | instance-0000003f || OS-EXT-STS:power_state | 0 || OS-EXT-STS:task_state | scheduling || OS-EXT-STS:vm_state | building || OS-SRV-USG:launched_at | - || OS-SRV-USG:terminated_at | - || accessIPv4 | || accessIPv6 | || adminPass | aVq7cExJDRgi || config_drive | || created | 2016-06-14T12:06:52Z || flavor | m1.tiny (1) || hostId | || id | 7d33cc29-bc17-482c-b96a-8792158c9134 || image | cirros (f7e1f270-d39b-481e-a854-eb6d0ed65e24) || key_name | - || metadata | {} || name | src || os-extended-volumes:volumes_attached | [] || progress | 0 || security_groups | default || status | BUILD || tenant_id | 704e6c364c224841ade1122551a506fc || updated | 2016-06-14T12:06:52Z || user_id | 448a618a47ba40ae9af39021e24472fe |+--------------------------------------+-----------------------------------------------+

Controller Node에서 CLI 실행• src 이름의 VM 생성• src vm eth0가 바인딩될 포트는 p1

• 포트 ID를 반드시 확인

SFC Environment (From CLI)sdn@controller:~$ nova boot --image cirros_dual_nic --nic port-id=feb28ff2-f8ea-49e5-b56c-0dbea926abde --nic port-id=78a6f0ee-c5ab-4eeb-844f-d4195b813e4c sf1 --flavor m1.tiny+--------------------------------------+--------------------------------------------------------+| Property | Value |+--------------------------------------+--------------------------------------------------------+| OS-DCF:diskConfig | MANUAL || OS-EXT-AZ:availability_zone | || OS-EXT-SRV-ATTR:host | - || OS-EXT-SRV-ATTR:hypervisor_hostname | - || OS-EXT-SRV-ATTR:instance_name | instance-00000041 || OS-EXT-STS:power_state | 0 || OS-EXT-STS:task_state | scheduling || OS-EXT-STS:vm_state | building || OS-SRV-USG:launched_at | - || OS-SRV-USG:terminated_at | - || accessIPv4 | || accessIPv6 | || adminPass | NiEaStSo6ozs || config_drive | || created | 2016-06-14T12:10:16Z || flavor | m1.tiny (1) || hostId | || id | 8aeb5370-15b2-4e44-9664-003f22404ddd || image | cirros_dual_nic (f3e7a811-c9bf-4c90-8381-d4a2d96e80e6) || key_name | - || metadata | {} || name | sf1 || os-extended-volumes:volumes_attached | [] || progress | 0 || security_groups | default || status | BUILD || tenant_id | 704e6c364c224841ade1122551a506fc || updated | 2016-06-14T12:10:17Z || user_id | 448a618a47ba40ae9af39021e24472fe |+--------------------------------------+--------------------------------------------------------+

Controller Node에서 CLI 실행• sf1 이름의 VM 생성• port-pair를 하여야 하기 때문에 포트 2개를 할당• src vm eth0/eth1이 바인딩될 포트는 p2, p3

• 포트 ID를 반드시 확인

Horizon (create vms)

• net100• port: 8개

• router interface• dhcp• p1 -> src vm (eth0)• p2 -> sf1 vm (eth0)• p3 -> sf1 vm (eth1)• p4 -> sf2 vm (eth0)• p5 -> sf2 vm (eth1)• p6 -> dst vm (eth0)

Basic Flow

Compute02

src sf1

P1(10.0.0.9)

P2(10.0.0.10)

P3(10.0.0.11)

qbr6c5c6a24-ec qbrfeb28ff2-f8 qbr78a6f0ee-c5

qvb6c5c6a24-ec qvbfeb28ff2-f8 qvb78a6f0ee-c5

tap6c5c6a24-ec tapfeb28ff2-f8 tap78a6f0ee-c5

br-int

Vlan tag 5

qvo6c5c6a24-ec qvofeb28ff2-f8

patch-tun

br-tun

patch-int

eth2

qvo78a6f0ee-c5

eth0 eth0 eth1

Compute01

dstsf2

P6(10.0.0.14)

P5(10.0.0.13)

P4(10.0.0.12)

qbr2c85ffec-a4qbrb95acc26-4c qbr5c9e1fa9-ef

qvb2c85ffec-a4qvbb95acc26-4c qvb5c9e1fa9-ef

tap2c85ffec-a4tapb95acc26-4c tap5c9e1fa9-ef

br-int

Vlan tag 4

qvo2c85ffec-a4qvob95acc26-4c

patch-tun

br-tun

patch-int

eth2

qvo5c9e1fa9-ef

eth0eth1 eth0

br-int Flowcookie=0xb9e6db74360a11d5, duration=92915.300s, table=0, n_packets=3623, n_bytes=395762, priority=0 actions=NORMAL (match rule이 없는 경우 normal -> br-tun)

br-tun Flowcookie=0xb9e6db74360a11d5, duration=92977.762s, table=0, n_packets=1994, n_bytes=213002, priority=1,in_port=1 actions=resubmit(,2)cookie=0xb9e6db74360a11d5, duration=92977.265s, table=0, n_packets=616, n_bytes=67276, priority=1,in_port=2 actions=resubmit(,4)cookie=0xb9e6db74360a11d5, duration=92977.198s, table=0, n_packets=1205, n_bytes=134383, priority=1,in_port=3 actions=resubmit(,4)cookie=0xb9e6db74360a11d5, duration=92977.762s, table=2, n_packets=1532, n_bytes=145922, priority=0,dl_dst=00:00:00:00:00:00/01:00:00:00:00:00 actions=resubmit(,20)cookie=0xb9e6db74360a11d5, duration=92977.762s, table=2, n_packets=462, n_bytes=67080, priority=0,dl_dst=01:00:00:00:00:00/01:00:00:00:00:00 actions=resubmit(,22)cookie=0xb9e6db74360a11d5, duration=2841.206s, table=4, n_packets=551, n_bytes=57971, priority=1,tun_id=0x427 actions=push_vlan:0x8100,set_field:4100->vlan_vid,resubmit(,10)cookie=0xb9e6db74360a11d5, duration=92977.761s, table=10, n_packets=1643, n_bytes=179092, priority=1

actions=learn(table=20,hard_timeout=300,priority=1,cookie=0xb9e6db74360a11d5,NXM_OF_VLAN_TCI[0..11],NXM_OF_ETH_DST[]=NXM_OF_ETH_SRC[],load:0->NXM_OF_VLAN_TCI[],load:NXM_NX_TUN_ID[]->NXM_NX_TUN_ID[],output:NXM_OF_IN_PORT[]),output:1cookie=0xb9e6db74360a11d5, duration=12.561s, table=20, n_packets=14, n_bytes=1316, hard_timeout=300, priority=1,vlan_tci=0x0004/0x0fff,dl_dst=fa:16:3e:bd:c2:a6 (p1) actions=load:0-

>NXM_OF_VLAN_TCI[],load:0x427->NXM_NX_TUN_ID[],output:3cookie=0xb9e6db74360a11d5, duration=92977.761s, table=20, n_packets=0, n_bytes=0, priority=0 actions=resubmit(,22)cookie=0xb9e6db74360a11d5, duration=2841.229s, table=22, n_packets=94, n_bytes=13900, priority=1,dl_vlan=4 actions=pop_vlan,set_field:0x427->tun_id,output:2,output:3

br-int Flowcookie=0x975b19bafc6479db, duration=92808.636s, table=0, n_packets=3767, n_bytes=417802, priority=0 actions=NORMAL (match rule이 없는 경우 normal --> br-tun)

br-tun Flowcookie=0x975b19bafc6479db, duration=92470.697s, table=0, n_packets=2121, n_bytes=226311, priority=1,in_port=1 actions=resubmit(,2)cookie=0x975b19bafc6479db, duration=92338.557s, table=0, n_packets=686, n_bytes=75556, priority=1,in_port=3 actions=resubmit(,4)cookie=0x975b19bafc6479db, duration=92280.007s, table=0, n_packets=1072, n_bytes=119551, priority=1,in_port=2 actions=resubmit(,4)cookie=0x975b19bafc6479db, duration=92470.697s, table=2, n_packets=1537, n_bytes=146424, priority=0,dl_dst=00:00:00:00:00:00/01:00:00:00:00:00 actions=resubmit(,20)cookie=0x975b19bafc6479db, duration=92470.696s, table=2, n_packets=584, n_bytes=79887, priority=0,dl_dst=01:00:00:00:00:00/01:00:00:00:00:00 actions=resubmit(,22)

cookie=0x975b19bafc6479db, duration=2157.018s, table=4, n_packets=530, n_bytes=57552, priority=1,tun_id=0x427 actions=push_vlan:0x8100,set_field:4101->vlan_vid,resubmit(,10)cookie=0x975b19bafc6479db, duration=92470.696s, table=10, n_packets=1682, n_bytes=187031, priority=1

actions=learn(table=20,hard_timeout=300,priority=1,cookie=0x975b19bafc6479db,NXM_OF_VLAN_TCI[0..11],NXM_OF_ETH_DST[]=NXM_OF_ETH_SRC[],load:0->NXM_OF_VLAN_TCI[],load:NXM_NX_TUN_ID[]->NXM_NX_TUN_ID[],output:NXM_OF_IN_PORT[]),output:1cookie=0x975b19bafc6479db, duration=269.012s, table=20, n_packets=141, n_bytes=13370, hard_timeout=300, priority=1,vlan_tci=0x0005/0x0fff,dl_dst=fa:16:3e:4a:54:85 (p6) actions=load:0-

>NXM_OF_VLAN_TCI[],load:0x427->NXM_NX_TUN_ID[],output:2cookie=0x975b19bafc6479db, duration=95.910s, table=20, n_packets=0, n_bytes=0, hard_timeout=300, priority=1,vlan_tci=0x0005/0x0fff,dl_dst=aa:04:fa:f6:8d:0c actions=load:0-

>NXM_OF_VLAN_TCI[],load:0x427->NXM_NX_TUN_ID[],output:2cookie=0x975b19bafc6479db, duration=92470.696s, table=20, n_packets=0, n_bytes=0, priority=0 actions=resubmit(,22)cookie=0x975b19bafc6479db, duration=2157.040s, table=22, n_packets=96, n_bytes=14341, priority=1,dl_vlan=5 actions=pop_vlan,set_field:0x427->tun_id,output:3,output:2

Compute01

Compute02

Compute02

src sf1

P1(10.0.0.9)

P2(10.0.0.10)

P3(10.0.0.11)

qbr6c5c6a24-ec qbrfeb28ff2-f8 qbr78a6f0ee-c5

qvb6c5c6a24-ec qvbfeb28ff2-f8 qvb78a6f0ee-c5

tap6c5c6a24-ec tapfeb28ff2-f8 tap78a6f0ee-c5

br-int

Vlan tag 5

qvo6c5c6a24-ec qvofeb28ff2-f8

patch-tun

br-tun

patch-int

eth2

qvo78a6f0ee-c5

eth0 eth0 eth1

Compute01

dstsf2

P6(10.0.0.14)

P5(10.0.0.13)

P4(10.0.0.12)

qbr2c85ffec-a4qbrb95acc26-4c qbr5c9e1fa9-ef

qvb2c85ffec-a4qvbb95acc26-4c qvb5c9e1fa9-ef

tap2c85ffec-a4tapb95acc26-4c tap5c9e1fa9-ef

br-int

Vlan tag 4

qvo2c85ffec-a4qvob95acc26-4c

patch-tun

br-tun

patch-int

eth2

qvo5c9e1fa9-ef

eth0eth1 eth0

Test01 : 100.0.0.0/24 트래픽이 sf-1 경유

port-pair

application 동작• port forwarding• routing• …

sdn@controller:~$ neutron port-pair-create --ingress p2 --egress p3 pp1Created a new port_pair:+-----------------------------+--------------------------------------+| Field | Value |+-----------------------------+--------------------------------------+| description | || egress | 78a6f0ee-c5ab-4eeb-844f-d4195b813e4c || id | cb10e5a1-c47d-4edd-b9de-827fb8111ae4 || ingress | feb28ff2-f8ea-49e5-b56c-0dbea926abde || name | pp1 || service_function_parameters | {"correlation": null} || tenant_id | 704e6c364c224841ade1122551a506fc |+-----------------------------+--------------------------------------+sdn@controller:~$ neutron port-pair-group-create --port-pair pp1 ppg1Created a new port_pair_group:+-------------+--------------------------------------+| Field | Value |+-------------+--------------------------------------+| description | || id | fce7d1f0-d6aa-4900-a0fa-21ca05fb12d2 || name | ppg1 || port_pairs | cb10e5a1-c47d-4edd-b9de-827fb8111ae4 || tenant_id | 704e6c364c224841ade1122551a506fc |+-------------+--------------------------------------+sdn@controller:~$ neutron flow-classifier-create --source-ip-prefix 100.0.0.0/24 --destination-ip-prefix 100.0.0.0/24 --logical-source-port p1 fc1Created a new flow_classifier:+----------------------------+--------------------------------------+| Field | Value |+----------------------------+--------------------------------------+| description | || destination_ip_prefix | 100.0.0.0/24 || destination_port_range_max | || destination_port_range_min | || ethertype | IPv4 || id | 08f9ac3f-38c2-43a4-a01f-2a05fed16f7e || l7_parameters | {} || logical_destination_port | || logical_source_port | 6c5c6a24-ec62-4964-99fa-aeb348c7ed81 || name | fc1 || protocol | || source_ip_prefix | 100.0.0.0/24 || source_port_range_max | || source_port_range_min | || tenant_id | 704e6c364c224841ade1122551a506fc |+----------------------------+--------------------------------------+sdn@controller:~$ neutron port-chain-create --port-pair-group ppg1 --flow-classifier fc1 pc1Created a new port_chain:+------------------+--------------------------------------+| Field | Value |+------------------+--------------------------------------+| chain_parameters | {"correlation": "mpls"} || description | || flow_classifiers | 08f9ac3f-38c2-43a4-a01f-2a05fed16f7e || id | 4774ae78-18a6-4828-90fb-60a9bbfe927a || name | pc1 || port_pair_groups | fce7d1f0-d6aa-4900-a0fa-21ca05fb12d2 || tenant_id | 704e6c364c224841ade1122551a506fc |+------------------+--------------------------------------+

br-int Flowcookie=0x975b19bafc6479db, duration=93931.949s, table=0, n_packets=3826, n_bytes=423416, priority=0 actions=NORMALcookie=0x975b19bafc6479db, duration=404.212s, table=5, n_packets=19, n_bytes=1862, priority=1,ip,dl_dst=fa:16:3e:4c:2b:34 (p2),nw_src=100.0.0.0/24 actions=push_mpls:0x8847,set_field:65791-

>mpls_label,set_mpls_ttl(255),push_vlan:0x8100,set_field:4101->vlan_vid,resubmit(,10)cookie=0x975b19bafc6479db, duration=404.186s, table=5, n_packets=0, n_bytes=0, priority=0,dl_dst=fa:16:3e:4c:2b:34 (p2) actions=push_mpls:0x8847,set_field:65791-

>mpls_label,set_mpls_ttl(255),push_vlan:0x8100,set_field:4101->vlan_vid,set_field:fa:16:3e:00:83:a4 (router interface)->eth_src,resubmit(,10)cookie=0x975b19bafc6479db, duration=404.054s, table=10, n_packets=19, n_bytes=1862, priority=1,mpls,dl_vlan=5,dl_dst=fa:16:3e:4c:2b:34 (p2),mpls_label=65791

actions=pop_vlan,pop_mpls:0x0800,output:15

br-tun Flowcookie=0x975b19bafc6479db, duration=2157.018s, table=4, n_packets=530, n_bytes=57552, priority=1,tun_id=0x427 actions=push_vlan:0x8100,set_field:4101->vlan_vid,resubmit(,10)cookie=0x975b19bafc6479db, duration=92470.696s, table=10, n_packets=1682, n_bytes=187031, priority=1

actions=learn(table=20,hard_timeout=300,priority=1,cookie=0x975b19bafc6479db,NXM_OF_VLAN_TCI[0..11],NXM_OF_ETH_DST[]=NXM_OF_ETH_SRC[],load:0->NXM_OF_VLAN_TCI[],load:NXM_NX_TUN_ID[]->NXM_NX_TUN_ID[],output:NXM_OF_IN_PORT[]),output:1cookie=0x975b19bafc6479db, duration=269.012s, table=20, n_packets=141, n_bytes=13370, hard_timeout=300, priority=1,vlan_tci=0x0005/0x0fff,dl_dst=fa:16:3e:4a:54:85 (p6) actions=load:0-

>NXM_OF_VLAN_TCI[],load:0x427->NXM_NX_TUN_ID[],output:2cookie=0x975b19bafc6479db, duration=95.910s, table=20, n_packets=0, n_bytes=0, hard_timeout=300, priority=1,vlan_tci=0x0005/0x0fff,dl_dst=aa:04:fa:f6:8d:0c actions=load:0-

>NXM_OF_VLAN_TCI[],load:0x427->NXM_NX_TUN_ID[],output:2cookie=0x975b19bafc6479db, duration=92470.696s, table=20, n_packets=0, n_bytes=0, priority=0 actions=resubmit(,22)cookie=0x975b19bafc6479db, duration=2157.040s, table=22, n_packets=96, n_bytes=14341, priority=1,dl_vlan=5 actions=pop_vlan,set_field:0x427->tun_id,output:3,output:2

Compute02

br-int Flowcookie=0xb9e6db74360a11d5, duration=92915.300s, table=0, n_packets=3623, n_bytes=395762, priority=0 actions=NORMAL (match rule이 없는 경우 normal -> br-tun)

br-tun Flowcookie=0xb9e6db74360a11d5, duration=92977.762s, table=0, n_packets=1994, n_bytes=213002, priority=1,in_port=1 actions=resubmit(,2)cookie=0xb9e6db74360a11d5, duration=92977.265s, table=0, n_packets=616, n_bytes=67276, priority=1,in_port=2 actions=resubmit(,4)cookie=0xb9e6db74360a11d5, duration=92977.198s, table=0, n_packets=1205, n_bytes=134383, priority=1,in_port=3 actions=resubmit(,4)cookie=0xb9e6db74360a11d5, duration=92977.762s, table=2, n_packets=1532, n_bytes=145922, priority=0,dl_dst=00:00:00:00:00:00/01:00:00:00:00:00 actions=resubmit(,20)cookie=0xb9e6db74360a11d5, duration=92977.762s, table=2, n_packets=462, n_bytes=67080, priority=0,dl_dst=01:00:00:00:00:00/01:00:00:00:00:00 actions=resubmit(,22)cookie=0xb9e6db74360a11d5, duration=2841.206s, table=4, n_packets=551, n_bytes=57971, priority=1,tun_id=0x427 actions=push_vlan:0x8100,set_field:4100->vlan_vid,resubmit(,10)cookie=0xb9e6db74360a11d5, duration=92977.761s, table=10, n_packets=1643, n_bytes=179092, priority=1

actions=learn(table=20,hard_timeout=300,priority=1,cookie=0xb9e6db74360a11d5,NXM_OF_VLAN_TCI[0..11],NXM_OF_ETH_DST[]=NXM_OF_ETH_SRC[],load:0->NXM_OF_VLAN_TCI[],load:NXM_NX_TUN_ID[]->NXM_NX_TUN_ID[],output:NXM_OF_IN_PORT[]),output:1cookie=0xb9e6db74360a11d5, duration=12.561s, table=20, n_packets=14, n_bytes=1316, hard_timeout=300, priority=1,vlan_tci=0x0004/0x0fff,dl_dst=fa:16:3e:bd:c2:a6 (p1) actions=load:0-

>NXM_OF_VLAN_TCI[],load:0x427->NXM_NX_TUN_ID[],output:3cookie=0xb9e6db74360a11d5, duration=92977.761s, table=20, n_packets=0, n_bytes=0, priority=0 actions=resubmit(,22)cookie=0xb9e6db74360a11d5, duration=2841.229s, table=22, n_packets=94, n_bytes=13900, priority=1,dl_vlan=4 actions=pop_vlan,set_field:0x427->tun_id,output:2,output:3

Compute01

Compute02

src sf1

P1(10.0.0.9)

P2(10.0.0.10)

P3(10.0.0.11)

qbr6c5c6a24-ec qbrfeb28ff2-f8 qbr78a6f0ee-c5

qvb6c5c6a24-ec qvbfeb28ff2-f8 qvb78a6f0ee-c5

tap6c5c6a24-ec tapfeb28ff2-f8 tap78a6f0ee-c5

br-int

Vlan tag 5

qvo6c5c6a24-ec qvofeb28ff2-f8

patch-tun

br-tun

patch-int

eth2

qvo78a6f0ee-c5

eth0 eth0 eth1

Compute01

dstsf2

P6(10.0.0.14)

P5(10.0.0.13)

P4(10.0.0.12)

qbr2c85ffec-a4qbrb95acc26-4c qbr5c9e1fa9-ef

qvb2c85ffec-a4qvbb95acc26-4c qvb5c9e1fa9-ef

tap2c85ffec-a4tapb95acc26-4c tap5c9e1fa9-ef

br-int

Vlan tag 4

qvo2c85ffec-a4qvob95acc26-4c

patch-tun

br-tun

patch-int

eth2

qvo5c9e1fa9-ef

eth0eth1 eth0

Test02 : 100.0.0.0/24 트래픽이 sf-1, sf-2 경유

port-pair port-pair

application 동작• port forwarding• routing• …

application 동작• port forwarding• routing• …

sdn@controller:~$ neutron port-pair-create --ingress p2 --egress p3 pp1Created a new port_pair:+-----------------------------+--------------------------------------+| Field | Value |+-----------------------------+--------------------------------------+| description | || egress | 78a6f0ee-c5ab-4eeb-844f-d4195b813e4c || id | 047ce3a1-e69b-4b78-8730-a00ab414b4c9 || ingress | feb28ff2-f8ea-49e5-b56c-0dbea926abde || name | pp1 || service_function_parameters | {"correlation": null} || tenant_id | 704e6c364c224841ade1122551a506fc |+-----------------------------+--------------------------------------+sdn@controller:~$ neutron port-pair-create --ingress p4 --egress p5 pp2Created a new port_pair:+-----------------------------+--------------------------------------+| Field | Value |+-----------------------------+--------------------------------------+| description | || egress | b95acc26-4c6e-47da-b1b0-1e8bceba01d9 || id | f9ebb5f9-6aaa-43d8-bdb3-9b7f3afba1ce || ingress | 5c9e1fa9-ef66-42a9-9b03-0b3b8407efba || name | pp2 || service_function_parameters | {"correlation": null} || tenant_id | 704e6c364c224841ade1122551a506fc |+-----------------------------+--------------------------------------+sdn@controller:~$ neutron port-pair-group-create --port-pair pp1 ppg1Created a new port_pair_group:+-------------+--------------------------------------+| Field | Value |+-------------+--------------------------------------+| description | || id | fe65f438-c1db-4299-adc1-e3717a90904a || name | ppg1 || port_pairs | 047ce3a1-e69b-4b78-8730-a00ab414b4c9 || tenant_id | 704e6c364c224841ade1122551a506fc |+-------------+--------------------------------------+sdn@controller:~$ neutron port-pair-group-create --port-pair pp2 ppg2Created a new port_pair_group:+-------------+--------------------------------------+| Field | Value |+-------------+--------------------------------------+| description | || id | 2ce8fe6c-7965-4192-9195-b6b8dca9729d || name | ppg2 || port_pairs | f9ebb5f9-6aaa-43d8-bdb3-9b7f3afba1ce || tenant_id | 704e6c364c224841ade1122551a506fc |+-------------+--------------------------------------+sdn@controller:~$ neutron flow-classifier-create --source-ip-prefix 100.0.0.0/24 --destination-ip-prefix 100.0.0.0/24 --logical-source-port p1 fc1Created a new flow_classifier:+----------------------------+--------------------------------------+| Field | Value |+----------------------------+--------------------------------------+| description | || destination_ip_prefix | 100.0.0.0/24 || destination_port_range_max | || destination_port_range_min | || ethertype | IPv4 || id | c203bf1d-2dd8-4d9a-aa06-3b47b7cd1a98 || l7_parameters | {} || logical_destination_port | || logical_source_port | 6c5c6a24-ec62-4964-99fa-aeb348c7ed81 || name | fc1 || protocol | || source_ip_prefix | 100.0.0.0/24 || source_port_range_max | || source_port_range_min | || tenant_id | 704e6c364c224841ade1122551a506fc |+----------------------------+--------------------------------------+sdn@controller:~$ neutron port-chain-create --port-pair-group ppg1 --port-pair-group ppg2 --flow-classifier fc1 pc1Created a new port_chain:+------------------+--------------------------------------+| Field | Value |+------------------+--------------------------------------+| chain_parameters | {"correlation": "mpls"} || description | || flow_classifiers | c203bf1d-2dd8-4d9a-aa06-3b47b7cd1a98 || id | b854e7fc-9428-4730-ab2b-1fb21f7d70af || name | pc1 || port_pair_groups | fe65f438-c1db-4299-adc1-e3717a90904a || | 2ce8fe6c-7965-4192-9195-b6b8dca9729d || tenant_id | 704e6c364c224841ade1122551a506fc |+------------------+--------------------------------------+

br-int Flowcookie=0x975b19bafc6479db, duration=93931.949s, table=0, n_packets=3826, n_bytes=423416, priority=0 actions=NORMALcookie=0x975b19bafc6479db, duration=404.212s, table=5, n_packets=19, n_bytes=1862, priority=1,ip,dl_dst=fa:16:3e:4c:2b:34 (p2),nw_src=100.0.0.0/24 actions=push_mpls:0x8847,set_field:65791-

>mpls_label,set_mpls_ttl(255),push_vlan:0x8100,set_field:4101->vlan_vid,resubmit(,10)cookie=0x975b19bafc6479db, duration=404.186s, table=5, n_packets=0, n_bytes=0, priority=0,dl_dst=fa:16:3e:4c:2b:34 (p2) actions=push_mpls:0x8847,set_field:65791-

>mpls_label,set_mpls_ttl(255),push_vlan:0x8100,set_field:4101->vlan_vid,set_field:fa:16:3e:00:83:a4 (router interface)->eth_src,resubmit(,10)cookie=0x975b19bafc6479db, duration=404.054s, table=10, n_packets=19, n_bytes=1862, priority=1,mpls,dl_vlan=5,dl_dst=fa:16:3e:4c:2b:34 (p2),mpls_label=65791

actions=pop_vlan,pop_mpls:0x0800,output:15

br-tun Flowcookie=0x975b19bafc6479db, duration=2157.018s, table=4, n_packets=530, n_bytes=57552, priority=1,tun_id=0x427 actions=push_vlan:0x8100,set_field:4101->vlan_vid,resubmit(,10)cookie=0x975b19bafc6479db, duration=92470.696s, table=10, n_packets=1682, n_bytes=187031, priority=1

actions=learn(table=20,hard_timeout=300,priority=1,cookie=0x975b19bafc6479db,NXM_OF_VLAN_TCI[0..11],NXM_OF_ETH_DST[]=NXM_OF_ETH_SRC[],load:0->NXM_OF_VLAN_TCI[],load:NXM_NX_TUN_ID[]->NXM_NX_TUN_ID[],output:NXM_OF_IN_PORT[]),output:1cookie=0x975b19bafc6479db, duration=269.012s, table=20, n_packets=141, n_bytes=13370, hard_timeout=300, priority=1,vlan_tci=0x0005/0x0fff,dl_dst=fa:16:3e:4a:54:85 (p6) actions=load:0-

>NXM_OF_VLAN_TCI[],load:0x427->NXM_NX_TUN_ID[],output:2cookie=0x975b19bafc6479db, duration=95.910s, table=20, n_packets=0, n_bytes=0, hard_timeout=300, priority=1,vlan_tci=0x0005/0x0fff,dl_dst=aa:04:fa:f6:8d:0c actions=load:0-

>NXM_OF_VLAN_TCI[],load:0x427->NXM_NX_TUN_ID[],output:2cookie=0x975b19bafc6479db, duration=92470.696s, table=20, n_packets=0, n_bytes=0, priority=0 actions=resubmit(,22)cookie=0x975b19bafc6479db, duration=2157.040s, table=22, n_packets=96, n_bytes=14341, priority=1,dl_vlan=5 actions=pop_vlan,set_field:0x427->tun_id,output:3,output:2

Compute02

br-int Flowcookie=0xb9e6db74360a11d5, duration=93931.949s, table=0, n_packets=3826, n_bytes=423416, priority=0 actions=NORMALcookie=0xb9e6db74360a11d5, duration=404.212s, table=5, n_packets=19, n_bytes=1862, priority=1,ip,dl_dst=fa:16:3e:cc:c4:80 (p4),nw_src=100.0.0.0/24 actions=push_mpls:0x8847,set_field:65791-

>mpls_label,set_mpls_ttl(255),push_vlan:0x8100,set_field:4101->vlan_vid,resubmit(,10)cookie=0xb9e6db74360a11d5, duration=404.186s, table=5, n_packets=0, n_bytes=0, priority=0,dl_dst=fa:16:3e:cc:c4:80 (p4) actions=push_mpls:0x8847,set_field:65791-

>mpls_label,set_mpls_ttl(255),push_vlan:0x8100,set_field:4101->vlan_vid,set_field:fa:16:3e:00:83:a4 (router interface)->eth_src,resubmit(,10)cookie=0xb9e6db74360a11d5, duration=296.252s, table=10, n_packets=19, n_bytes=1862, priority=1,mpls,dl_vlan=4,dl_dst=fa:16:3e:cc:c4:80 (p4),mpls_label=65790

actions=pop_vlan,pop_mpls:0x0800,output:14

br-tun Flowcookie=0xb9e6db74360a11d5, duration=2841.206s, table=4, n_packets=551, n_bytes=57971, priority=1,tun_id=0x427 actions=push_vlan:0x8100,set_field:4100->vlan_vid,resubmit(,10)cookie=0xb9e6db74360a11d5, duration=92977.761s, table=10, n_packets=1643, n_bytes=179092, priority=1

actions=learn(table=20,hard_timeout=300,priority=1,cookie=0xb9e6db74360a11d5,NXM_OF_VLAN_TCI[0..11],NXM_OF_ETH_DST[]=NXM_OF_ETH_SRC[],load:0->NXM_OF_VLAN_TCI[],load:NXM_NX_TUN_ID[]->NXM_NX_TUN_ID[],output:NXM_OF_IN_PORT[]),output:1cookie=0xb9e6db74360a11d5, duration=12.561s, table=20, n_packets=14, n_bytes=1316, hard_timeout=300, priority=1,vlan_tci=0x0004/0x0fff,dl_dst=fa:16:3e:bd:c2:a6 (p1) actions=load:0-

>NXM_OF_VLAN_TCI[],load:0x427->NXM_NX_TUN_ID[],output:3cookie=0xb9e6db74360a11d5, duration=92977.761s, table=20, n_packets=0, n_bytes=0, priority=0 actions=resubmit(,22)cookie=0xb9e6db74360a11d5, duration=2841.229s, table=22, n_packets=94, n_bytes=13900, priority=1,dl_vlan=4 actions=pop_vlan,set_field:0x427->tun_id,output:2,output:3

Compute01

Todo

• networking-sfc 안정화– 기본 동작

• 생성/삭제에 대한 기본 룰 적용 (반영이 안되는 경우 발생)

– Virtual Router 가 반드시 생성될 이유는 없음– Horizon 제공

• 현재 project 가 생겨 작업중 (언제 나올지…)

– agent 제공• neutron-openvswitch-agent를 lunch 하여야 함 (버전 등에 민감)

– OVS version• 기본 ovs 2.4.0 (2.4.9 버전으로 업그레이드 가능성)

• SDN Controller 연동– ODL

• OPNFV 의 SFC 관련 Reference (https://wiki.opnfv.org/display/sfc/SFC+Fuel+Installer+preparation+and+deploy)• NSH 활용

– ONOS• onos vtn Application (onos/apps/vtn/srcmgr/)• https://www.youtube.com/watch?v=9-6XH3wu4dQ