Summit 16: Multi-site OPNFV Testing Challenges

  • View

  • Download

Embed Size (px)


  • June 2023, 2016 | Berlin, Germany

  • Multi-site OPNFV Testing ChallengesTrevor Cooper (Intel)

    Fatih Degirmenci (Ericsson)

    Morgan Richomme (Orange)

  • Agenda

    Multi-site Testing Today - Trevor- Pharos Infrastructure

    - CI Today

    Multi-site Testing Needs - Morgan- Advanced Test-cases

    - Interoperability

    Multi-site Testing Plans - Fatih- Challenges and Requirements

    - Future Plans and Vision for Infra and Testing

    Summary and Call to Action - Fatih

  • Multi-site Testing Today

  • Multi-site Testing Introduction

    Motivations for Multi-site Testing Geographically and technically diverse test environments

    Test resources independent of installers and scenarios promotes robust platform

    Inter-deployment testing for interoperability between data-centers

    Future real-world use-cases and deployment scenarios

    Pharos Infrastructure Infra Working Group: Pharos, Octopus, Releng, Genesis

    Infrastructure labs, process, tools and features

    CI / CD / CT Environments increasing in number and complexity

    Process to improve platform robustness

    Challenges of a Distributed Infrastructure Managing a distributed resources

    Automating testing

    Distributing tests

  • Pharos Infrastructure

    Current priorities: Allocation and management of Infra resources for Colorado

    Pharos change process to manage special requirements

    Lab resource booking and usage tool/dashboard

    Pharos compliance verification/validation

    Dealing with todays multi-site challenges Challenges and how we solved them

  • Production Today

    CI Yesterday CI Today

    POD1Installer XScenario Y



    Deploy, run tests

    POD1Installer XScenario Y



    Deploy, run tests

    Lab 1

    Lab 2

    Lab 1

    Lab 2

  • Production Evolution - Tomorrow

    POD1Installer XScenario Y



    Deploy, run tests

    Lab 1

    Lab 2

    POD1Installer XScenario Y


    Lab 3

    CI Tomorrow

  • Infra Improvements

    Lab as a Service (LaaS) Accessible virtual sandboxes for development and smoke tests

    More homogeneous environments for development work

    Off-load shortage of bare-metal resources

    Efficient use of Lab Resources Strive for CD (provide weekly stable versions)

    Provide dynamic use of lab resources (Dev and CI)

    Verify/validate POD setup for automatic configuration

    All labs/pods/installer configurations for automatic POD configuration

    Jenkins / CI improvements - permissions, scheduling, sandboxes, etc.

  • Multisite Testing Needs

  • Test cases

    Advanced test cases OPNFV Multisite Project - evolving

    Brahmaputra -> Use case definition and bug fixes

    Colorado -> Single machine/Multiple OpenStack Instances, Functional Testing, Kingbird

    Future -> Go after true Multisite

    Discussion on multi tenant vIMS for D river

    Interoperability testing

  • An advanced test case: vIMS

    Today for Functest test: Metaswitch vIMS + Cloudify orchestrator installed in a single tenant on one OPNFV POD.

    Not acceptable in real life (SPOF)

    Geographical redundancy on current production IMS system

    Architecture Cloud ready (Cassandra ring)

    Possible evolutions: 1) test in multi-tenants (local redundancy)2) Test in multi-sites (geographical redundancy)

  • An advanced test case: vIMS

    Need of Multi-sites to evaluate the connectivity issues (VPN/Flat) the consequence on performance The accuracy of placement algorithm:

    Performance (DPDK/ODP area) Location (legal constraint)

    Disaster/recovery scenarios/constraints

    Similar considerations on vCDN for next releases

  • An interoperability test

    OPNFV is a great place to automate interoperability testing

    Several PODs with several scenarios

    And a centralized CI

  • An interoperability test: bgpvpn

    SDNVPN project deals with the implementation of the OpenStack bgpvpn API

    Several possible backends ODL

    Neutron Bagpipe


    Use OPNFV to perform interoperability testing would be valuable

  • An interoperability test: bgpvpn



    Deploy, run tests


  • Multi-site Testing Plans

  • Lessons learned

    Distributed infrastructure is not trivial

    Automating tests/ state synchro towards multi-sites

    We learn by doingfirst requests on multi-sites needed raised in Brahmaputra => Multi-site project

  • Next steps

    Apply learnings Bringing up, managing and using distributed infrastructure

    Evolve continuously No dramatic changes

    E2E View and holistic approach Features will come together one day, creating more complete

    Multisite solutions

  • Next steps

    Infrastructure Provision, manage and use resources cleverly

    Increase the utilization rather than burning CPU uselessly

    Get ready to enter uncharted territories - security and access policies

    CI Evaluate CI to see if it can support Multisite use cases

    Provide feedback that fits to purpose rather than bunch of log files

    Continue applying best practices

    Testing Test what needs to be tested when it is needed to be tested

  • Summary

    We already are challenged by what we had in the past and what we currently have

    The complexity will just increase, requiring us as a community to work together like a community

  • Call to Action