OPNFV Upstreaming -Kubernetes ODL IntegrationPREM SANKAR GOPANNAN, ERICSSON
HTTP://TWITTER.COM/PREMSANKAR
Demo objective
Microservices – Quick intro
CNCF Landscape
Introduce to Kuryr
ODL architecture
Highlight issues of VM and Container Internetworking
Hand-on
Pod Creation
Pinging PODS
Issue OVS commands
Business Requirements
- 5G- IOT- Digital Transformation
Microservices – Quick Intro
• Reference – Martin Fowler, http://martinfowler.com
Principles that drive Microservices architecture
• Services must be loosely coupled so that they can be developed, deployed and scaled independently
• Organized around business capabilities
• API Focused
• Smart endpoints and dumb pipes
• Decentralized Governance
• Decentralized Data Management
• Infrastructure Automation (infrastructure as code)
• Design for failure
• Evolutionary Design
Microservices Design Patterns
• Reference – Chris Richardson http://microservices.io/patterns/microservices.html
Microservices Design Patterns
Decomposition Pattern
Subdomain context
Service discovery Pattern
Database per service Pattern
Keep each microservice’s persistent data private to that service and accessible only via its API
RDBMS design approaches
Private-tables-per-service – each service owns a set of tables that must only be accessed by that service
Schema-per-service – each service has a database schema that’s private to that service
Database-server-per-service – each service has it’s own database server.
Circuit Breaker
API Gateway Pattern
CNCF Landscape
Lifecycle
Requirements
Service Orchestration
Infrastructure Orchestration
Business Process mapping
Audit and Validation
Security
WorkflowManager and Orchestrator
Continuous Integration
ContinuousDeployment
Monitoring
Design and Creation
Deployment and Execution
Operation, Analytics and Maintenance
Opensource Ecosystem
Workflow Manager
and Orchestrator
ONAP
ARIA
OSM
Design and
Deployment
CI/CD
Pipelines
•Jenkins
•Fabric8
(Jenkins,
Kubernetes
and Docker)
•Other options
– goCD,
concourse.ci
Automation
•Ansible/Puppet
/Chef
Scheduler
(w/ docker/cri-o)
Kubernetes
Mesos
Swarm
Service Mesh
ISTIO
Linkerd
Monitoring
Prometheus
Other
alternatives(https://prometheus.io/docs/i
ntroduction/comparison/)
Tracing
Zipkin
Squash
Logging
ELK
Kubernetes
Master components
API Server – Frontend for K8S control plane
Scheduler
Control-manager
Node Controller
Replication Controller
Endpoints Controller
Service account and token controllers
Etcd – All clustered data is stored
Worker Node
Kubelet - primary node agent and watches pod that has been assigned to node
Kube-proxy – enables service abstraction by maintaining network rules on the host
Kubernetes components
Abstraction levels
POD – Encompasses the containers the pods that are related/microservice
Replication Controller – Defines pod count that corresponds to Service
Service – defines logical set of Pods
Ingress – connecting to external world
NFV Service – Deployment 1
PodAPodA
PodA
PodBPodB
NFV Service – Deployment 2
PodBPodAPodBPodA
Kubernetes Networking - Overview
CNI uses CNI Network plugin to setup container networking
(If container runtime is docker, CNM will not be used)
Plugin responsible for creating network interface to container
Plugin calls IPAM to setup IP address
Plugin needs to implement API for network creation and deletion
Kubernetes
CNI
Network Plugin IPAM
VNF Deployment scenario
Neutron Plugin or Gluon/Proton
Linux Linux Linux Linux
Kubernetes
Datacenter Hardware
Linux
OpenStack
OVS
App
Orchestration Systems
OpenStack APIs
OpenDaylight
Kubernetes APIs
CNI Plugin
KVM KVM ContainerRT
Container RT
App App App
VM Application Container Application
AppApp App
ODL COE Architecture
Host OS
Open vSwitch
Host OS
Open vSwitch
VM
Container App
Container App
Container App
VLAN’s
Orchestrator
OpenDaylight
Kubernetes / Docker
Kuryr
Neutron / Gluon
Container App
Container App
Container App
Container Mgt
(docker, kube-pxy)
Iptables / NAT/FW
Native
Openstack Kuryr
Kuryr is an Openstack project aimed at providing network and storage support for hybrid environments
Bridge between container Networking and Openstack Neutron
Two implementation
Kuryr CNI for Kubernetes
Kuryr libnetwork for docker
Kuryr-Kubernetes Architecture
Kuryr components
Kuryr Controller
Watches Kube API resource with a service account
Secure connection with Neutron API server
Kuryr CNI
Communicates with Kube API
Perform local binding of Neutron port
Watches Pod resources for controller-driven vif
Kuryr Kubernetes modes
Baremetal/side by side
VM and Pods are in the same
Nested
Pods within VM
Uses trunk ports to provide neutron port to containers
Uses VLAN segmentation so POD communication goes to vswitch
Neutron – K8S construct mapping
Kubernetes NeutronNamespace Network
Cluster Subnet Subnet Pool
Service Cluster IP Subnet
External Subnet Floating IPExternal NetworkRouter
POD Port
Service Load Balancer
Openstack Kuryr 101
• Controller• Watches K8S API endpoints to
make sure that the corresponding model is maintained in Neutron
• Updates K8S resource endpoints annotations to keep neutron details required by CNI driver
• Watcher• Used by both Controller
and CNI Driver• Connects to K8S API• Observe Registered
Endpoints and invoke call back handlers
• CNI Driver
Kuryr-K8S integration
ODL COE Baremetal
Installing Kuryr AIO
Pull code
git clone https://github.com/openstack/kuryr-kubernetes
Install Vagrant – sudo apt-get install vagrant
Install Virtual Box – sudo apt-get install virtualbox
Edit local.conf.sample
cd $KURYR_HOME/devstack/
cp local.conf.odl.sample to local.conf.sample
By default only Openstack Neutron is enabled and if you want to install Openstack components
Comment out ENABLED_SERVICE=“” statement in local.conf.sample or
ENABLED_SERVICE=<list needed Openstack components>
cd $KURYR_HOME/conrib/vagrant
vagrant up
vagrant ssh