Networking & Policies for Kubernetes by Harmeet Sahni Director PLM Nuage Networks - #NFD12

© 2016 Nokia. All rights reserved. Nuage Networks is a Nokia venture.

Networking & Policies for Kubernetes Networking Field Day 12


Agenda

1. What problems are we trying to solve

2. Kubernetes Overview

3. Networking & Policies for Kubernetes

4. Demo

8/16/2016

2


TIME

Front End MiddleWare SQL DB App Logic Idle

1 2 3 4 5

Container Enviroments Are More Dynamic Than Legacy Virtualized DC

Containers are created and destroyed on the fly. To adapt to the demand SDN needs to follow ,in real time, enforcing the Security, QoS, NAT or service chaining policies for each container.


Control Plane Scale & Convergence For 100K Containers

8/16/2016 4

• 100,000 Containers

• 500 Containers per hypervisor

• 200 Networks in 200 VRFs (router contexts)

• 200 Hypervisors

• 20 Networks per hypervisor

• Total Convergence Time: 9:24 !

Nuage VSC

Nuage VSD

Nuage VSC

Networking Field Day 8


Challenges With Container Networking

5

Integration Complex

Deployments Security Cloud

• App Isolation • Micro-segmentation • Monitoring &

Visibility

• Connect containers

to VMs and bare metal servers

• DC GW Integration • Public breakout

• Integration with Container Orchestration workflows

• Mesos, Docker, Kubernetes, OpenShift

VM

C BM

• Private Cloud • Public Cloud • Hybrid Cloud


The Container Orchestration Ecosystem

8/16/2016 6

Container Orchestration

Containerized PaaS

Cluster Management

Deploy, scale and maintain container

applications

User experience and continuous integration

services

Focuses on isolation of resources and improving

cluster utilization

Nuage offers a comprehensive support matrix of container platforms

What container and PaaS tools are used to manage OpenStack applications?

8/16/2016

7

Production

Dev/QA

PoC

OpenStack User Survey – April 2016

Copyright 2013 Alcatel-Lucent. All rights reserved. CONFIDENTIAL - SOLELY FOR AUTHORIZED PERSONS HAVING A NEED TO KNOW

PROPRIETARY – USE PURSUANT TO COMPANY INSTRUCTION Nuage Networks

KUBERNETES


Kubernetes

Kubernetes – Greek for “helmsman”

Abbreviation: K8S

Open source project originally developed by Google

Platform for automating deployment, scaling, and operations of application containers across clusters of hosts


Kubernetes Architecture

Master

SCHEDULER

API PROXY

AUTH

REPLICATION CONTROLLER

Node

KUBELET SERVICE PROXY

POD (Service 2)

POD (Service 2)

Node

KUBELET SERVICE PROXY

POD (Service 1)

POD (Service 1)

CLIENT

C1 C1

C1 C2 C1 C2

ETCD

Copyright 2013 Alcatel-Lucent. All rights reserved. CONFIDENTIAL - SOLELY FOR AUTHORIZED PERSONS HAVING A NEED TO KNOW

PROPRIETARY – USE PURSUANT TO COMPANY INSTRUCTION Nuage Networks

NUAGE VSP FOR KUBERNETES NETWORKING & POLICIES


Policy-Driven Networking For All Environments

Physical servers Virtual Machines Containers Public Cloud


Overlay-based Virtual Networks Kubernetes Deployment With VSP

Master Node Node

VSD

K A

PI

XMPP

VRS-K8S

Nuage-Kube-Mon

Kubernetes Cluster

Controller

VNI = 100 VNI = 200

Policy Engine

Provides Multi-tenancy and App Isolation

Control over IP Addressing

VRS-K8S


KUBERNETES ON OPENSTACK VMS (POD TO VM COMMUNICATION)

15

Physical Server

Kubernetes Node VM Kubernetes Node

(Physical Server)

Controller

VM VM

Policy Groups


Kubernetes Deployments On Public Cloud

16

Cloud VPC

Docker Swarm Kubernetes

Cloud VM

Cloud VM

Nuage VSP

Network Virtualization Policy Groups Visibility Secure Cloud Interconnect


17

Virtual Routing & Switching

Virtualized Services Controller

Virtualized Services Directory

VPN Connection

VPC

Hybrid Cloud Deployments

Cloud VM

NSG

NSG IPSec Tunnel


Demo

8/16/2016

20


Use Case 1 : Intra-namespace Communication

Web (NGINX)

Web (NGINX)

Web (NGINX)

Default Namespace

TCP/80 TCP/80 TCP/80

Pod to Service communication

Pod-to-Pod communication

Automatic creation of

⁻ subnet(s)

⁻ ACLs to the Default namespace

⁻ - ACLs to access Services


Use Case 2: App Isolation

Web (NGINX)

Web (NGINX)

Web (NGINX)

Default Namespace

TCP/80 TCP/80 TCP/80

Guestbook Namespace

FrontEnd

TCP/80

FrontEnd

TCP/80

FrontEnd

TCP/80

Redis Master

TCP/6379

Redis Slave

TCP/6379

Redis Slave

TCP/6379


Workflow for Network Policies

Kubernetes Master Kubernetes

Node

VSD

K A

PI

XMPP

VRS

Nuage-Kube-Mon

Nuage K8S Plugin

Kubernetes Cluster

Virtualized Services

Controller

Virtualized Services Directory

1. User creates Domain/Zone and defines Network and Security Policies on VSD

2. Labels in a Pod configuration are used to pass metadata to VSD

3. Node Plugin invoked during Pod creation will fetch Labels from Pod configuration

5. VSC gets network and security policy from VSD

6. VSC sends network and security policy to the VRS

4. VRS contacts VSC with Namespace name and metadata information


Use Case 3: Pods Can Talk To Internal Hosts Hosting Some Service With A Specific CIDR

Web (NGINX)

Demo Namespace

TCP/80

Web (NGINX)

Web (NGINX)

Web (NGINX)

TFD Namespace

TCP/80 TCP/80 TCP/80 Policy Group

“Internal Service”

Service CIDR

Web (NGINX)

TCP/80

✔ ️


Other Policy Use Cases

Expose a set of Pods (e.g. a web frontend) so that they are accessible from the Internet

Pods can initiate connections to the Internet but cannot initiate connections to internal hosts


Nuage VSP Addresses Container Networking Challenges

Provides Multi-tenancy and App Isolation

Control over IP addressing

Supports hybrid app environments with containers, VMs and Bare Metal servers

On-prem, Public Cloud and Hybrid Cloud container deployments

Flexible and Granular Security Policy framework


THANK YOU


NUAGE VSP OBJECTS AND KUBERNETES CONCEPTS

Namespace

Labels

Pods

Zone

Policy Groups

VPorts

Cluster Domain

VSP


nuage-kube-mon

• Runs on master node(s)

• Exercises the VSD REST API to ensure that the VSD objects are created • Create delete vsd zones Namespaces

• Create/delete network macros Services • Dynamically scale-up or scale-down subnets

nuage-kubernetes-plugin

• Runs on each of the nodes

• Implementation of the k8s network exec plugin

• Gets invoked when a node is initialized as well as during pod lifecycle events: • Create/delete pod

• Status hook that queries pod’s IP information

NUAGE VSP COMPONENTS


Pod gets a veth interface that maps to a VSP vPort

Pod gets an IP allocated from the subnet pools for that Kubernetes Namespace (VSP Zone)

Pods in a given zone belong to one or more subnets irrespective of which node they are spawned on

Labels are optionally used to do the Security and QoS Policy resolution with the VSD

Pods with VSP

KUBENETES DEPLOYMENT WITH VSP

Technology

Networking & Policies for Kubernetes by Harmeet Sahni Director PLM Nuage Networks - #NFD12