How to ensure SDN/NFV doesn't break your network?
United States +1 (646) 558-2111
United Kingdom +44 (0) 330 221 9914
Access Code: 674-242-804
Bryon Lowen
Vice-President Telco at
QualiTest
Alex Henthorn-Iwane
Vice-President Marketing at
QualiSystems
Webinar Presenters
Agenda
QualiTest & QualiSystems partnership
Challenges with NFV –what, when and why to verify with SDN/NFV?
SummaryCreating an Automated
SDN / NFV verification as a Service
Building SDN / NFV assurance process
QualiTest & QualiSystems partnership
Challenges with NFV –what, when and why to verify with SDN/NFV?
SummaryCreating an Automated
SDN / NFV verification as a Service
Building SDN / NFV assurance process
QualiTest & QualiSystems partnership
•Pure Play Testing focus•Passion for testing•World’s 2nd largest •Telco expertise
•Automation software•Cloud management•Test & CI automation•Carrier DevOps
QualiTest & QualiSystems partnership
Challenges with NFV –what, when and why to verify with SDN/NFV?
SummaryCreating an Automated
SDN / NFV verification as a Service
Building SDN / NFV assurance process
• End-to-end services are already often quite complexo Many service scenarios won’t change except for the presence of VNFs
o Consider topology below for end-to-end (E2E) network service (NS) including IMS-core enabled voice services in Cable MSO
What is a Service?
CMTS
eDVA
PCMM
S_CSCFI-CSCFP-CSCF
MRFC/MRFP
HSS TAS
VM
CAS
CCF OSS
BGCF/ESCF
MGCF STP
SRP
CMS
SBC
SIP PeerNetworks
PSTN
TNS
• Endpoints aren’t just handsets, home gateways, fixed CPEo Clusters of VMs o VMWare, OpenStack/KVM
• New WAN topology constructs/protocols are emergingo Federation of controllers. OpenFlow? Network Virtualization Overlays? PCEP? Segment Routing?
What is a Service? How Cloud/NFV Changes Things
Cloud DC1 Cloud DC2
Data Center Fabric
Storage
VM
VM
VM
VM
VM
VM
VM
VM
VM
Hypervisor Hypervisor Hypervisor
Compute
Network
Cloud Orchestration
Data Center Fabric
Storage
VM
VM
VM
VM
VM
VM
VM
VM
VM
Hypervisor Hypervisor Hypervisor
Compute
Network
Cloud Orchestration
MPLS/SDNCore
• Performance of VNFs must be characterized and certified on hypervisor/server hardware
• North-south and east-west dimensions to performance testingo How does traffic perform when going just through the hypervisor?o How about when it exits a physical port and goes to a PNF device?
What are New Performance Testing Challenges
IaaS/PaaS/SaaS IaaS/PaaS/SaaS
Data Center Fabric
Storage
VM
VM
VM
VM
VM
VM
VM
VM
VM
Hypervisor Hypervisor Hypervisor
Compute
Network
Cloud Orchestration
Data Center Fabric
Storage
VM
VM
VM
VM
VM
VM
VM
VM
VM
Hypervisor Hypervisor Hypervisor
Compute
Network
Cloud Orchestration
When & Why: SDN/NFV DevTest Speed & Scale Implications
Network as Utility
• Waterfall timeframes
• Long certification cycles
• Manual, little automation OTT NFV SDN OSS/BSS
Dev/Test Cycle
App Lifecycle is Agile
• Agile timeframes
• Implies automation
VS
Service chain permutations could be endlesso Design permutation of vendor solution when defining the architecture (interoperability)
o Operational permutation at the time of the service instantiation (service differentiation)
Agility = IT rate of change x 100o Performance and scale testing
How do you qualify and certify these service in an agile fashion?
When & Why: NFV DevTest Scale Implications
Admins make 500 updates / day
peak
APIs support 10,000’s updates /
day peak
Cloud Service
no admin - dynamic & automated
TodayOperations staff are involved
QualiTest & QualiSystems partnership
Challenges with NFV –what, when and why to verify with SDN/NFV?
Q&ACreating an Automated
SDN / NFV verification as a Service
Building SDN / NFV assurance process
Building SDN / NFV assurance process
• Reliable Service
• Right Price
• Good SLAs
• Increasingly Complex
• Renegotiate contracts
• Not affect QOS
Build strong assurance process to capitalize SDN / NFV potential
QualiTest NFV validation process
• QualiTest aims to create a unified end-to-end approach to service assurance and network performance
NFV Management
Orchestration Infrastructure Migration
Assure full compliance to ESTI ISG Standards.
Assure full compliance to NFV requirements for any network equipment provider and streamline the ability of vendors and users to be able to implement compatible NFV solutions
Build a comprehensive Test Environment, including Control plane and Data plane workload simulation tools, network analytics, QoS and QoE monitoring probes
Test management and Orchestration prior to VNF migration, including each component (e.g. Orchestrator, Vnfm, VIM)
Test the VNF architecture and implementation to assure smooth transition
ESTI NFV ISG Compliance (NFV-Co)
NFV Compliance for Vendors (NFV-Ven)
NFV Test Environment (NFV-TE)
MANO Testing (MANO-T)
VNF Migration (VNFM-T)
QualiTest NFV validation process
Assure full compliance to ESTI ISG Standards.
ESTI NFV ISG Compliance (NFV-Co)
NFV Compliance for Vendors (NFV-Ven)
NFV Test Environment (NFV-TE)
MANO Testing (MANO-T)
VNF Migration (VNFM-T)
(NFV-Co) ETSI NFV Compliance
ETSI ISG Standards Test procedure
Assuring compliance aims to find the deviations from standards• MANO• VNF/NE
• Analyse ISG current standards • Prepare compliance requirements • Analyse each NFV component / int• Define gap points and analyse • Repeat steps 1-6 for each VNF and NE
Assure full compliance to NFV requirements for any network equipment provider and streamline the ability of vendors and users to be able to implement compatible NFV solutions
ESTI NFV ISG Compliance (NFV-Co)
NFV Compliance for Vendors (NFV-Ven)
NFV Test Environment (NFV-TE)
MANO Testing (MANO-T)
VNF Migration (VNFM-T)
(NFV-Ven) Compliance for Vendors
MANO VNF
• VNF Descriptor format • VNF redundancy model • VNF state transitions • VNF Package • Allocation of addresses• Scaling event types and format• Maintain records on VNF
Packages
• VNF design patterns• VNF update and upgrade• VNF properties• VNF Topological Characteristics • VNF States and Transitions• VNF fault management
Build a comprehensive Test Environment, including Control plane and Data plane workload simulation tools, network analytics, QoS and QoE monitoring probes
ESTI NFV ISG Compliance (NFV-Co)
NFV Compliance for Vendors (NFV-Ven)
NFV Test Environment (NFV-TE)
MANO Testing (MANO-T)
VNF Migration (VNFM-T)
(NFV-TE) NFV Test Environment
(NFV-TE) NFV Test EnvironmentNFV provides great flexibility:• Build parallel test environments • Scale up and down compute resources • Copy any existing environment
Environment build guidelines:• Management and Orchestration should be
similar to the production• NS and VNF Instantiation should be similar to
production environment • Same level of resources should be allocated.
Test Appliances:• Simulate workload traffic on the client / server • Simulate both data plane & control plane traffic • Measure key metrics data & control plane
Appliances vs. Physical Test Appliances:• Virtual test appliances are more cost effective • Physical test appliances are recommended for
highest levels of data-plane performance or microsecond-level timing accuracy
Test management and Orchestration prior to VNF migration, including each component (e.g. Orchestrator, Vnfm, VIM)
ESTI NFV ISG Compliance (NFV-Co)
NFV Compliance for Vendors (NFV-Ven)
NFV Test Environment (NFV-TE)
MANO Testing (MANO-T)
VNF Migration (VNFM-T)
(MANO-T) MANO Testing
(MANO-T) MANO Testing
The first step is to test NFV Management and Orchestration Architectural Framework
NFV Orchestrator
VNF Manager
(VIM) and NFV Infrastructure (NFVI)
VIM– VNF Manager (Vi-Vnfm)
Orchestrator – VIM (Or-Vi)
The VNF is deployed as one or more Virtual
Machines; source: ETSI GS NFV 002 v1.2.1
Test the VNF architecture and implementation to assure smooth transition
ESTI NFV ISG Compliance (NFV-Co)
NFV Compliance for Vendors (NFV-Ven)
NFV Test Environment (NFV-TE)
MANO Testing (MANO-T)
VNF Migration (VNFM-T)
(VNFM-T) VNF Migration
Static testing VNF-Mano Integration
Conformance testing VNF Internal functionality
Scalability/Performance End-to-end/system test
QualiTest & QualiSystems partnership
Challenges with NFV –what, when and why to verify with SDN/NFV?
SummaryCreating an Automated
SDN / NFV verification as a Service
Building SDN / NFV assurance process
Goal is Continuous Processes: Continuous Integation/Delivery
Automated BuildCode Commit Automated Unit Test
Automated Sanity TestAutomated & Scaled Full Regression TestAutomated Test Results
Collection
Automated BI Report Generation
Automated Build, Deploy, Test
Auto-scaled regression tests
Applied to service & network
changes
QualiSystems Summarized
Processes Infrastructure Test
Page 25
Automation Platform
• Self-Service System
• Infrastructure Modelling
• Reservation / Scheduling
• Provisioning Automation
• Reporting / Utilization
• Test Automation
• Test Results Reporting
• CI Integration
Step 1: Automate all Infrastructure
Physical Labs
MaaS
Virtual Labs
IaaS & PaaS
• Build virtual labs for VNF-only scenarios• Consolidate or federate virtual & physical labs into private/hybrid clouds
CloudShell Resource Management
• Any resource: virtual, physical, logical (IP prefix, software license)
• User-defined resource tagging to easily find the type of device you need
• Eliminates manual searching
• Customizable resource abstractions enable efficient resource pooling
All resources managed in a single repository
Ensure Orchestration can Handle Network Topologies
=
Carriers adopting infrastructure orchestration to build DevOps have foundthat most cloud management platforms only support compute concepts
CloudShell orchestration is capable of handling E2E NS topologies
Replace static diagrams with a live workspace
Live Topology
• Drag & drop devices
• Draw required connections and activate
• Powerful search capabilities based on device attributes
• Abstract topologies
• Power on/off devices by controlling PDUs
Step 2: Create Lab as a Service Cloud
• CloudShell allows you to create self-service access to full network environments built and provisioned on demand
• Make available to internal and external users:
• Including vendors, contractors
Key goals:
• Increase efficiency utilization of test lab resources
• Reduce ratio of setup to test time
Select environment
Verify availability
Reserve
SetupTesting
activities
Collaborate
Teardown
Reservation & Scheduling
• Plan ahead lab operations
• Automatically resolve allocation conflicts
• Guarantees availability of devices for tests
• Only lock up resources when they are needed for testing
• Promotes efficient sharing
Improved sharing through reservation &
scheduling
• “One-Click” is first step toward continuous
• Self-service users can easily launch entire certification from UI
• Integrate with ALM to ensure consistent reporting
• Certification can be reused for many stages:
• New technology and early stage service proof of concept testing
• New service introduction
• Vendor interoperability testing
• Contractor code qualification
Step 3: Create “One-Click” Certification
Tight Integration between TestShell & CloudShell
Create an Environment
Develop a Test
Create the Automation
Suite
Execute and Monitor the
Suite
Develop test in TestShell Studio in conjunction
with a live CloudShell environment for greater accuracy,
easier debugging
Suite = reservations & associated tests
What Happened when a Cable MSO Built an “As-a-Service” Approach
Overall: Complete change in the way devtest team conducted business
Measured:
• 15% annual operations cost savings
• 20% immediate reduction in certification cycles
• 33% immediate increased in certification coverage and improving
• 100% increase in resource utilization due to 24/7 service availability
Ongoing observations:
• Earlier certification by internal departments and vendors moves quality to the left in the cycle—still measuring outcomes
• Eliminated need for extensive travel time and costs on part of vendors and contractors
• Increased compliance with their methodology and standards via automation
• Increased pace of innovation
QualiTest & QualiSystems partnership
Challenges with NFV –what, when and why to verify with SDN/NFV?
SummaryCreating an Automated
SDN / NFV verification as a Service
Building SDN / NFV assurance process
Summary
• Combined approach, building on a methodology of test processes and automation.
• There will also be a skills pivot as organisations transition to this virtualised environment.
• Increased pace of innovation.
Thankyou