Next Gen Infrastructure Core (NGIC) & M-CORD - schd.ws - M-CORD -- CORD...• Investigate functionality for new usage models, e.g. Connectionless IOT, Multi-Radio Access Technologies,

Intel Labs

Next Gen Infrastructure Core (NGIC) & M-CORD

CORD Build Event - November 7-9th, 2017 – San Jose, CA

Christian Maciocco

Principal Engineer, Director of Telecom Systems Research – Intel Labs

Ashok Sunder Rajan, Kannan Babu Ramia, Sai Edupuganti, Jacob Cooper, Karla Saur – Intel Labs

2

Legal Disclaimer

Intel Confidential

• This presentation contains the general insights and opinions of Intel Corporation (“Intel”). The information in this presentation is provided for information only and is not to be relied upon for any other purpose than educational. Use at your own risk! Intel makes no representations or warranties regarding the accuracy or completeness of the information in this presentation. Intel accepts no duty to update this presentation based on more current information. Intel is not liable for any damages, direct or indirect, consequential or otherwise, that may arise, directly or indirectly, from the use or misuse of the information in this presentation.

• Intel technologies’ features and benefits depend on system configuration and may require enabled hardware, software or service activation. Learn more at intel.com, or from the OEM or retailer.

• No license (express or implied, by estoppel or otherwise) to any intellectual property rights is granted by this document.

• Intel, the Intel logo and Xeon are trademarks of Intel Corporation in the United States and other countries.

• *Other names and brands may be claimed as the property of others.

• © 2017 Intel Corporation.

Intel Labs3

Agenda

•vEPC Background

•Next Generation Infrastructure Core (NGIC)

•CORD High Level View

•NGIC and M-CORD

•Summary / Next Steps

Intel Labs4

SGWMME

vEPC Background (1/2)

S11

S1U

Control Data

PGW

• Operators’ real traffic (San Jose, Houston, Chicago, …)

• Identified system’s bottleneck

• “Understanding Bottlenecks in Virtualizing Cellular Core Network functions”, IEEE LANMAN ‘15

• No independent control or data plane scaling

User Data

”Other names and brands may be claimed as the property of others” User plane capacity reduction as control plane interference increases

Intel Labs5

vEPC Background (2/2)

”Other names and brands may be claimed as the property of others”

IVB

SKT0 HT=1 2x12=24 LCORES

MME HSS

IVB

SKT1 2x12=24 LCORES

PGWS1MME

Mngt IF

S1U SGi

NODE1

S1MME

S1U

1GbE

SW

IVB


IVB

SKT1 2x12=24 LCORES

Mngt IF

S1U SGi

NODE2

10GbE

SW10GbE

SW

IVB


IVB

SKT1 2x12=24 LCORES

Mngt IF

S1U SGi

NODEN

SGi

PktGen1

GTP-U

UDP/IP

PktGen4

GTP-U

UDP/IP

U/L

eNB

RAN Emulator

Signaling & U/L User Plane

Traffic generator

App

Server(Video,

Web, …)

App

Server(Video,

Web, …)

App

Server(Video,

Web, …)

App

Server(Video,

Web, …)

D/L Application Load

Generator

PktGen1

IP

PktGen4

IP

SGW

PGWSGW

PGWSGW

EPC Setup

User Plane Traffic Model

User plane and control plane queue flooding

Intel Labs6

Agenda

•vEPC Background



•NGIC and M-CORD


Intel Labs7

What is it ? Why ?

• Research and collaborations to understand telecom workloads behavior on Intel Architecture

• Research to address software/hardware platform improvements

• Released learnings and optimized reference software into open source community (DPDK, CORD)

• S/P-GW, Cuckoo Hash for optimized lookup from collaborations with industry or academic partners

• MME, HSS, PCRF will be released later

• Investigate functionality for new usage models, e.g. Connectionless IOT, Multi-Radio Access Technologies, etc

• NGIC is not a product and will not be a product from Intel

Next Generation Infrastructure Core (NGIC)

https://gerrit.opencord.org/#/admin/projects/ngic

Intel Labs8

SGWMME

Next Generation Infrastructure Core (NGIC)

S11

S1U

Control Data

PGW

• Operators’ real traffic (San Jose, Houston, Chicago, …)

• Identified system’s bottleneck

• “Understanding Bottlenecks in Virtualizing Cellular Core Network functions”, IEEE LANMAN ‘15

• No independent control or data scaling

SW Defined Network (Separation of Control and

Data)

SGW

DataONLY

MME & S/PGW Control ONLY

Control Data

PGW

DataONLY

User Data

• SDN based architecture

• Match/Action semantic data plane

• Independent & scalable control & data

• Functional EPC per operator’s requirements

Traditional EPC Architecture Disaggregated Architecture

User Data

SDN Controller


Intel Labs

Data Plane9

Functional EPC per Operator’s Requirements - NGIC

SGi

S1U

UL Uplink (UL)

Downlink (DL)

DL

Tables and Policies

Spirent

RAN Emulator

Spirent

Apps Server

GTP-U Decap

Rule Identif.

Bearer Mapping

SDF Metering

Charging IP Forward

IP ForwardGTP-U Encap

DL Data Indic.

Charging Metering(SDF, APN)

Bearer Mapping

Rule Identif.

Control Plane

SDN Controller

MME

Spirent

Sprint

Intel IETF DMM FPC (Forwarding & Policy Control)

ZeroMQ, OF 1.3 previously w/ Vendors Ext.

MMEHSS PCRF



SW to be released later

10©2016 Sprint. This information is subject to Sprint policies regarding use and is the property of Sprint and/or its relevant affiliates and may contain restricted, confidential or privileged materials intended for the sole use of the intended recipient. Any review, use, distribution or disclosure is prohibited without authorization.

11/9/2017 2:02 PMVersion 1.0

Linear Scaling Characteristics

0

1000000

2000000

3000000

4000000

5000000

6000000

7000000

8000000

9000000

1 2 3 4

PA

CK

ETS

PER

SEC

ON

D

NUMBER OF PACKET PROCESSINGCORES

Scaling with Number of Cores

50k Subs

250k Subs

500k Subs

• 1 packet processing core• 2.125 Million pps (50k subs)• 1.625 Million pps (500k subs)

• 4 packet processing cores• 8.2 Million pps (50k subs)• 6.5 Million pps (500k subs)

All packet processing cores implement all network functions (PGW, SGW, DPI, Child Protection,CG-NAT, static FW & SFC)

• Additional (overhead) cores for C3PO EPC Data Plane Node• Rx Core• Tx Core• Load-balancing Core • Master Core• Interface Core

• Packet processing cores are in addition to the overhead cores

From Sprint’s Presentation at NFV World Congress, May 2017

Intel Labs11

Agenda

•vEPC Background



•NGIC and M-CORD


Enabling the Programmable 5G Edge Cloud

M-CORD

Source: Open Networking Foundation

M-CORD: Enabling the Programmable 5G Edge Cloud

Open Source

Disaggregation

Virtualization

Cloudification

Programmability

PROGRAMMABLE 5G EDGE CLOUD


Edge Cloud

EPC

Wireless Access Edge Cloud Core

eNodeBRadio

After with SDN & NFV

Before with dedicated hardware

DISAGGREGATE RAN

VIRTUALIZE RAN components in the Cloud

SDN-ize it for PROGRAMMABILITY

Virtualized RAN

Control

DISAGGREGATE Core

VIRTUALIZE all components in the Cloud

SDN-ize it for PROGRAMMABILITY

Virtualized EPC

PROGRAMMATICALLY instantiate slices for different use cases


M-CORD as the Edge Cloud

RRU + DU

CU-U

CU-c Low

ONOS xRAN

vMME

vHSS

vSGW-U

vSGW-C

vPGW-U

vPGW-C

OpenStack

vPGW-u

vPGW-c

HSS PCRF

vSGW-u

vSGW-c vMME

Core Cloud

PDN

M-CORD Edge Cloud


XOS

ONOS

vPCRF

M-CORD Usage Models

Virtualized RAN

Control

Virtualized EPC

SD-RAN:

xRAN + ONOS

Commercial Small Cells

CBRS Spectrum

Multi-Radio (MRAT)

WiFi / LTE Steering

WiFi RAN Slicing

Open Source EPC

Infrastructure Acceleration

Multi-Access CORD

Diversification of Hardware Choices

End-to-End Network Slice Stitching

End-to-End Solution PoCs & Capabilities

Mobile Core

RAN and Access


Intel Labs17

Agenda

•vEPC Background



•NGIC and M-CORD


NGIC (vEPC) & M-CORD


NGIC & M-CORD

Internet

• Millions of Any-G

Connections

• Full Mobility with Traffic

• Call Modeling

• Topology Emulation

• Full virtual

• ETSI Compliant

M-CORD

Compute Networking Storage

Infrastructure Virtualization (Openstack)

(OvS/DPDK based)

ONOS

XOS

S/P-GW-u

(VM)

S/P-GW-c

(VM)MME

(VM)

SGiS1-APS1-U

HSS PCRF


vLSSpirent Landslide

Virtual

S/P-GW-c

(VM)

S/P-GW-u

(VM)

CP-DP_net

S11_net S1U_net SGI_net

MME

(VM)

Emulated

eNBApp Server

LOGICAL NETWORK CONNECTIVITY IN CORD

vSwitch (OVS / OVS-DPDK)

S/P-GW-uS/P-GW-c

A B C D E F G

Switch

vLS RAN +

MME + App server

Legend:A,D management MANAGEMENT

B,E CP-DP_net PRIVATE

C S11_net ACCESS_AGENT

F S1U_net ACCESS_AGENT

G SGI_net ACCESS_AGENT

PHYSICAL NETWORK CONNECTIVITY


(OvS/DPDK based)

ONOS

XOS

S/P-GW-u

S/P-GW-cMME

HSS PCRF

CORD InstallerCORD-1483

CORD-1484

CORD-1486

networking-onosCORD-1488

nova-computeCORD-1684

CORD-1485

CORD-1842

CORD-1483: Configure Nova with DPDK-enabled flavors

CORD-1484: Add nova config options to juju_config.yml

CORD-1485: Install OVS-DPDK through nova-compute charm

CORD-1486: Set kernel commandline parameters for compute nodes

CORD-1488: Add VIF details to handle VHOST_USER port type.

CORD-1684: Backport libvirt driver changes, to enable memAccess=shared tag required for correct functioning of VHOST_USER

CORD-1842: Replace default QEMU with patched QEMU 2.7.0 for correct functioning of VHOST_USER

M-CORD using OvS/DPDK and Enhanced Platform Awareness

https://jira.opencord.org/browse/CORD-1483







M-CORD using OvS/DPDK and Enhanced Platform Awareness

- Features used to achieved high performance on CORD- OvS/DPDK

- Enhanced Platform Awareness:

- VM memory backed by Huge Pages

- CPU isolation

- Core pinning

- vHost user data path

L2FWD Throughput in CORD w/ OvS/DPDK vs. CORD w/ OvS(Higher is better)

L2 FWD Latency in CORD w/ OvS/DPDK vs. CORD w/ OvS(Lower is better)

Modules: vEPC & M-RAT (1/2)

InternetEmulated eNB

Compute Networking Storage


(OvS/DPDK based)

ONOS

XOS

S/P-GW-u

S/P-GW-cMME

HSS PCRF

M-RAT

client

LTE Wi-Fi

Ctrl

WiFi User Data

LTE User Data

Multi-RAT Data-Plane

Encapsulation &

Tunneling

Sequencing

& Splitting

Concatenation &

Fragmentation

Buffering &

Retransmissions

Multi-RAT Control-Plane

E2E Protocols

Measurements

management

Client / Session

Management

Data-Plane / Traffic

Managements



Intel Labs26

Agenda

•vEPC Background



•NGIC and M-CORD


Intel Labs27

Summary / Next Steps

•NGIC is a functional, DPDK based, EPC operating in bare-metal, VMs, or containers and orchestrated in M-CORD

•Additional EPC components, MME, HSS, PCRF, will be made available to CORD over time

Intel Labs28

NGIC Hands-on Tutorial Next

Prerequisites for Hands-on Tutorial

• Install Docker (1.13 or higher) and Docker Compose

• Docker images for NGIC control, data plane and traffic generationdocker pull ngiccorddemo/ngic-cp

docker pull ngiccorddemo/ngic-dp

docker pull ngiccorddemo/ngic-traffic

• Demo foldergit clone https://github.com/ngiccorddemo/cordbuild2017.git

Intel Labs29

Thank You

Intel Confidential – Internal Use Only

Intel Labs31

Backup

Intel Labs32

Match/Action : Optimized Table Lookup with Cuckoo Hashing[Pagh 01]

• “Scalable, High Performance Ethernet Forwarding with CuckooSwitch”, Dong Zhu, Bin Fan, Dave Anderson (CMU), Michael Kaminsky (Intel)

H1(x) H2(x)

X

1

H1(x) H2(x)

X

2

H1(x) H2(x)

X

Y

3

H1(x) H2(x)

X

Y

4

H1(x) H2(x)

X

Y

Z

5

H1(x) H2(x)

X

Y

Z

6

One Insert may move a lot of items especially at high table occupancy.Optimal multi-writer insertion using Intel® TSX

collision

Efficient MATCH/Action Semantic Data Plane (1/2)

Intel Labs33

Traditional hash

0

1

2

3

4

5

6

7

8

9

0% 10% 20% 30% 40% 50% 60% 70% 80%

Inse

rts

/ S

eco

nd

Mil

lio

ns

Table Load

Cuckoo Hash Insert Performance per CPU Core

32M Random

8M Random

2M Random

32M Sequential

15X

0.00

500.00

1,000.00

1,500.00

2,000.00

2,500.00

3,000.00

10 1M 4M

Memory bandwidth vs. # of entries

rte_hash

Cuckoo

Previous hash

0.00

2.00

4.00

6.00

8.00

10.00

12.00

14.00

16.00

18.00

1M 2M 4M 8M 16M 32M

Throughput vs. # of entries

rte_hash

Cuckoo

Previous hash

Efficient MATCH/Action Semantic Data Plane (2/2)


Documents

Next Gen Infrastructure Core (NGIC) & M-CORD - schd.ws - M-CORD -- CORD...• Investigate functionality for new usage models, e.g. Connectionless IOT, Multi-Radio Access Technologies,