View
229
Download
1
Category
Preview:
Citation preview
1 Proprietary and Confidential 2015
OpenStack Networking L4-L7 Services Deep-dive
Praveen Yalagandula
Avi Networks
2 Proprietary and Confidential 2015
Network Virtualization
Physical Network
Virtual Network Virtual Network Virtual Network …
3 Proprietary and Confidential 2015
Network Virtualization Layers
Switches, LAN (Broadcast Domain) L2 Neutron: Network
Routers, IP Subnets L3 Neutron: Router, Subnet
Firewalls Load Balancers (ADC)
DNS VPN Servers
… L4 – L7
Neutron Services: FWaaS LBaaS
Designate VPNaaS
….
Physical World Virtual World
4 Proprietary and Confidential 2015
Load Balancers Core goals: High Availability and Performance
Internet
Web Server
Users
5 Proprietary and Confidential 2015
Load Balancers Core goals: High Availability and Performance
Internet
Web Servers
Users
6 Proprietary and Confidential 2015
Load Balancers Core goals: High Availability and Performance
Internet
Web Servers
Users
App Servers Database Servers
North-South Traffic
East-West Traffic
7 Proprietary and Confidential 2015
Load Balancer: Advanced features Application Delivery • Session persistence
– Preserve a single user’s session to the same backend server – Requires LBs to be stateful
• Intelligent load balancing – L3 load balancing not good enough – Look at L7 payload and spread traffic – E.g., Policy-based load balancing for HTTP – Requires TCP Termination
• SSL Termination – Move onus of SSL handshakes to central place – Easy to apply enterprise security policies – which ciphers are accepted, which protocols, what
length keys, etc.
Request URI contains Backend Pool
https://…./browse/… Regular Servers
https://…/cart/... https://…/checkout/…
Premium Servers
8 Proprietary and Confidential 2015
LBaaS in Neutron APIs
• LBaaS v1.0 API – Introduced in Grizzly – Lacks several key advanced features: SSL support, rules based switching
• LBaaS v2.0 API – Introduced in Kilo – Currently in progress
• Reference implementation and Horizon support coming in Liberty
10 Proprietary and Confidential 2015
LBaaS v1.0 Model • Pool is the central object • VIP defined by port_id • Supported protocols: TCP,
HTTP, HTTPS • LB Methods:
– Round-robin – Least-connections – Source_ip
• Session Persistence: – Source_ip – Http_cookie – App_cookie
- vip_id - name - description - subnet_id - protocol - lb_method - members - monitors - provider - stats - vip - admin_state_up
Pool
- pool_id - address - protocol_port - weight - admin_state_up
Member
- name - description - port_id - protocol_port - protocol - pool_id - session_persistence - admin_state_up - connection_limit
Vip
- Type (ping, TCP, HTTP, HTTPS) - delay - timeout - max_retries - http_method - url_path - expected_codes - admin_state_up - pools
HealthMonitor
- pool_id - bytes_in - bytes_out - active_connections - total_connections
PoolStatistics
0..1
1
1
1*
**
0..1
11 Proprietary and Confidential 2015
LBaaS v2.0 Model
- name - description - healthmonitor_id - protocol - lb_algorithm - members - admin_state_up - provisioning_status - operating_status - session persistence
Pool
- pool_id - address - protocol_port - weight - admin_state_up - subnet_id - provisioning_status - operating_status
Member
- name - description - vip_port_id - vip_subnet_id - vip_address - provisioning_status - operating_status - provider
LoadBalancer
- Type (ping, TCP, HTTP, HTTPS) - delay - timeout - max_retries - http_method - url_path - expected_codes - provisioning_status - admin_state_up
HealthMonitor
- loadbalancer_id - bytes_in - bytes_out - active_connections - total_connections
LB Statistics
1
*1
1
*
1
10..1
- name - description - default_pool_id - loadbalancer_id - protocol - protocol_port - default_tls_container_id - sni_containers - connection_limit - provisioning_status - operating_status - admin_state_up
Listener
- listener_id - tls_container_id - position
SNI Container
1
0..1
1
*
12 Proprietary and Confidential 2015
Initial Implementations
1 Hypervisor-process based E.g., Reference LBaaS implementation (using HAProxy)
2 Appliance-based E.g., Traditional ADC solutions
13 Proprietary and Confidential 2015
Reference Implementation (HAProxy)
• One HAProxy process per Pool/VIP • Running on Network Node
VM
VM VM VM VM
VM VM
VM VM
VM
VM VM VM
VM
Compute Nodes Network Node(s)
Keystone
Controller Node(s) Neutron w/LBaaS
……
LBaaS Agent
HAProxy HAProxy
HAProxy HAProxy
North-South Traffic
East-West Traffic
14 Proprietary and Confidential 2015
Reference Implementation (HAProxy)
Reference Implementation (Haproxy) Scalability Limited
• Runs on shared Neutron nodes, creating a large fan-in • Traffic “tromboning” • Complex to manage multiple Neutron nodes / HAProxy instances
High Availability None • Will need other solutions (e.g., PaceMaker) for achieving HA
Tenant Isolation Best effort; No strong guarantees • No per-tenant SLA service • Common pool of resources: network nodes
Not suitable for enterprise-grade clouds
15 Proprietary and Confidential 2015
Appliance-based Implementation
• Appliances located “next” to OpenStack servers
• Plug into underlay o Need to understand
underlay protocols VM
VM VM VM VM
VM VM
VM VM
VM
VM VM VM
VM
Compute Nodes Network Node(s)
Keystone
Controller Node(s) Neutron w/LBaaS
……
L3 Agent
router router
router router
North-South Traffic
East-West Traffic
LB1 LB2 LB3 LB4
16 Proprietary and Confidential 2015
Appliance-based Implementations
Tenant isolation and SLA? One or more appliances per Tenant and per Availability Zone
Operational and Management Costs
High, due to complexity • Individual device management è too many touch points • Appliances external to OpenStack è traffic tromboning • Complex routing setup due to dependence on underlay network • Limited scalability, especially for virtual appliances • Per appliance (expensive) support agreements
Complex & Expensive
18 Proprietary and Confidential 2015
Service-VM Architecture Distributed load balancer with a centralized control plane
LB1 LB2 LB3 LB4
OpenStack Legacy Next Generation
OpenStack
VM
VM VM VM VM
VM VM
VM VM
VM
VM VM VM
VM
VM
VM VM VM VM
VM VM
VM VM
VM
VM VM VM
VM
Controllers
Service Engine
20 Proprietary and Confidential 2015
Avi Networks Cloud Application Delivery Platform
AVI SERVICE ENGINES
AVI CONTROLLER
AVI UI
Self-Service Multi-Tenancy Single Point of Control
Single Point of Visibility Real-Time Analytics Google-like search for app traffic
Enterprise-grade ADC features App/Tenant Isolation Auto-Scaling
REST API
OpenStack
Neutron LBaaS
Keystone
Load Balancer Configuration
Server, Tenant, & Network
Configuration Nova
21 Proprietary and Confidential 2015
Avi LBaaS Overview Ignoring a lot of details
Admin creates 1) Avi Controller VM 2) SE Management Network Admin configures Avi Controller with OpenStack credentials
OpenStack Controller(s)
Neutron LBaaS
Keystone
Nova
Glance Avi
Controller
SE Mgmt Network
22 Proprietary and Confidential 2015
Avi LBaaS Overview Ignoring lot of details
OpenStack Controller(s)
Neutron LBaaS
Keystone
Nova
Glance Avi
Controller Server 1
Server 2
SE Mgmt Network
Server Network
Service Engine2
Service Engine1
User has two servers and requests LB service
Avi Controller dynamically creates SEs as needed to • Ensure high availability • Meet performance SLAs
traffic
23 Proprietary and Confidential 2015
Avi LBaaS Overview Ignoring lot of details
OpenStack Controller(s)
Neutron LBaaS
Keystone
Nova
Glance Avi
Controller Server 1
Server 2
SE Mgmt Network
Server Network
Service Engine2
Service Engine1
User has two servers and requests LB service
traffic
Service Engine3
Strong isolation guarantees possible by separating out the service engines
of different tenants
24 Proprietary and Confidential 2015
Service-VM Architecture: Flexible Deployments
• SEs in a user’s tenant context
• SEs in a common Service tenant context – Exclusive SEs per tenant – Shared SEs
OpenStack Controller(s)
Neutron LBaaS
Keystone
Nova
Glance Avi
Controller
Server 1
Server 2
SE Mgmt Network
Server Network
Service Engine2
Service Engine1
Service Engine3
Tenant 1
Tenant 2 Server 1
25 Proprietary and Confidential 2015
Service-VM Architecture: Flexible Deployments
• SEs in a user’s tenant context
• SEs in a common Service tenant context – Exclusive SEs per tenant – Shared SEs
OpenStack Controller(s)
Neutron LBaaS
Keystone
Nova
Glance Avi
Controller
Server 1
Server 2
SE Mgmt Network
Server Network
Service Engine2
Service Engine1
Service Engine3
Tenant 1
Tenant 2 Server 1
26 Proprietary and Confidential 2015
Service-VM Architecture: Flexible Deployments
• SEs in a user’s tenant context
• SEs in a common Service tenant context – Exclusive SEs per tenant – Shared SEs
OpenStack Controller(s)
Neutron LBaaS
Keystone
Nova
Glance Avi
Controller
Server 1
Server 2
SE Mgmt Network
Server Network
Service Engine2
Service Engine1
Service Engine3
Tenant 1
Tenant 2 Server 1
27 Proprietary and Confidential 2015
Service-VM Architecture: Flexible Deployments
• SEs in a user’s tenant context
• SEs in a common Service tenant context – Exclusive SEs per tenant – Shared SEs
OpenStack Controller(s)
Neutron LBaaS
Keystone
Nova
Glance Avi
Controller
Server 1
Server 2
SE Mgmt Network
Server Network
Service Engine2
Service Engine1
Tenant 1
Tenant 2 Server 1
28 Proprietary and Confidential 2015
Public Cloud Deployment
Avi CONT
Avi SE
Avi SE
Avi SE tenant
Web
Web
Web
Web
Consumer tenants Tenant Cloud Service Cloud
Keystone, Glance, Nova, Neutron
Avi LBaaS Driver
Horizon with SSL
SSL cert management
End User
SE mgmt network
Avi CONT Avi
CONT
29 Proprietary and Confidential 2015
Benefits over Reference Implementation (HAProxy) Reference Implementation (HAProxy) Service VM Architecture
Scalability Limited • Runs on shared Neutron nodes, creating a large fan-
in / networking bottleneck • Complex to manage multiple Neutron nodes /
HAProxy instances
Elastic Auto-scale • On-demand performance increase per App
– spin up new VM • Optimized for both East-West and North-
South traffic
High Availability None • Will need other solutions (e.g., PaceMaker) for
achieving HA
Inbuilt High Availability • Active/Active and N+1 • Automatically correct SE failures
Tenant Isolation Best effort; No strong guarantees • No per-tenant SLA service • Common pool of resources: network nodes
Strong Guarantees • SLA-based application delivery for tenants • Full data path isolation
30 Proprietary and Confidential 2015
Challenges for Service-VM model
• Limited number of VNICs per VM • VM Network performance
31 Proprietary and Confidential 2015
Challenge: Limited number of VNICs per VM
• Enterprises with 1000s of applications è Inefficient to create use a VNIC per LB: will need 100s of Service Engines
• One solution: Multiple fixed IP addresses on port – Caveat: Some plugins support it and some don’t – Caveat: Most plugins don’t support same IP address on multiple ports
• If we move IP address from VIP port, then other side effects
• Solution for ML2: – Allowed-address pair extension for outgoing traffic – ARP/Gratuitous ARP for incoming traffic
32 Proprietary and Confidential 2015
Challenge: Network Performance
• Depends on plugins • VETH pair bottleneck in
older kernels (Ubuntu 12.04)
• Older OVS performance issues (2.0.2 shipped with even recent distributions)
• Net Filter ConnTrak issues – Limited number of entries – PPS limits
• All fixable issues
ML2 with OVS/VLAN
Recommended