D5.3 Demonstration and Evaluation of the 5G-XHaul ...5G-XHaul Deliverable H2020-ICT-2014-2 671551 Page 8 of 49 14. Aug. 2018 Executive Summary To address the high bandwidth and low

Dynamically Reconfigurable Optical-Wireless Back-haul/Fronthaul with Cognitive Control Plane for Small Cells and Cloud-RANs

D5.3 Demonstration and Evalua-

tion of the 5G-XHaul Integrated

Prototype

This project has received funding from the European Union’s Framework

Programme Horizon 2020 for research, technological development

and demonstration

Advanced 5G Network Infrastructure for the Future Internet

Project Start Date: July 1st, 2015 Duration: 36 months

H2020-ICT-2014-2 671551 14th August 2018 – Version 1.0

Project co-funded by the European Commission

Under the H2020 programme

Dissemination Level: Public

5G-XHaul Deliverable

H2020-ICT-2014-2 671551 Page 2 of 49 14. Aug. 2018

Grant Agreement Number: 671551

Project Name: Dynamically Reconfigurable Optical-Wireless Backhaul/Fronthaul with Cognitive Control Plane for Small Cells and Cloud-RANs

Project Acronym: 5G-XHaul

Document Number: D5.3

Document Title: Demonstration and Evaluation of the 5G-XHaul Integrated Prototype

Version: 1.0

Delivery Date: 30th June 2018 (14th August 2018)

Responsible: University of Thessaly (UTH)

Editor(s): Paris Flegkas (UTH)

Authors:

Kostsas Choumas (UTH), Daniel Camps-Mur, Joan Josep Aleixendri (I2CAT), Arash Farhadi Beldachi, Anna Tzanakaki (UNIVBRIS-HPN), David Jones (BIO), Peter Legg (BWT), Jim Zou (ADVA), Jay-Kant Chaudhary (TUD), Jens Bar-telt (AIR), Nebojsa Maletic (IHP), Vladica Sark (IHP), Jesús Gutiérrez (IHP).

Keywords: Bristol city testbed, final demo, wireless SDN, converged fronthaul and backhaul, optical and wireless heterogeneous transport

Status: Final

Dissemination Level Public / Confidential

Project URL: http://www.5g-xhaul-project.eu/

http://www.5g-xhaul-project.eu/


H2020-ICT-2014-2 671551 Page 3 of 49 14. Aug. 2018

Version History

Rev. N Description Author Date

0.0 First draft with ToC Paris Flegkas (UTH) 14/06/2018

0.1 Included draft Section 3.2 Daniel Camps-Mur (i2CAT),

Joan Josep Aleixendri (i2CAT) 13/07/2018

0.2 Included Section 4.1

Included Section 3.1

Nebojsa Maletic (IHP), Vladica Sark (IHP)

Arash Farhadi Beldachi (UNIVBRIS-HPN)

23/07/2018

0.3 Included input in Section 3.1 Jim Zou (ADVA), Jens Bartelt (AIR) 24/07/2018

0.4 Included Section 4.2 Peter Legg (BWT) 25/07/2018

0.5 Revision of the document Kostas Choumas (UTH) 27/07/2018

0.6 Updated 3.1 Jim Zou (ADVA) 30/07/2018

0.7 Integrated version of the document Paris Flegkas (UTH) 31/07/2018

0.8 Per Section review ALL 09/07/2018

0.9 Final Revision

Paris Flegkas (UTH)

Daniel Camps-Mur (i2CAT),

Jesús Gutiérrez (IHP)

10/08/2018

1.0 Version for submission Jesús Gutiérrez (IHP) 14/08/2018


H2020-ICT-2014-2 671551 Page 4 of 49 14. Aug. 2018

Table of Contents

EXECUTIVE SUMMARY .............................................................................................................................. 8

1 INTRODUCTION ................................................................................................................................... 9

2 BIO TESTBED DESCRIPTION AND FINAL DEMO TOPOLOGY ............................................. 10

2.1 Bristol Is Open testbed infrastructure............................................................................................................ 10

2.2 Final Demonstrator overall topology and scenario ........................................................................................ 10

3 FINAL DEMONSTRATION SCENARIOS AND EXPERIMENTAL RESULTS.......................... 13

3.1 Converged front- and backhaul demo and results ......................................................................................... 13 3.1.1 Scenario / Topology Description .................................................................................................................. 13 3.1.2 Deployment .................................................................................................................................................. 18 3.1.3 Integration/Validation Tests ........................................................................................................................ 19 3.1.4 Conclusions .................................................................................................................................................. 25

3.2 SDN Wireless Backhaul demo and results ...................................................................................................... 25 3.2.1 Scenario / Topology Description .................................................................................................................. 25 3.2.2 Deployment .................................................................................................................................................. 26 3.2.3 Integration/Validation Tests ........................................................................................................................ 28 3.2.4 Results of SDN wireless demonstration ....................................................................................................... 34

4 INDIVIDUAL LAB DEMOS ............................................................................................................... 39

4.1 Demonstrations related to the 5G-XHaul transceiver solution ...................................................................... 39

4.2 Point to MultiPoint (P2MP) mmWave transmission ...................................................................................... 42

5 SUMMARY AND CONCLUSIONS .................................................................................................... 46

6 REFERENCES ...................................................................................................................................... 47

7 ACRONYMS ......................................................................................................................................... 48


H2020-ICT-2014-2 671551 Page 5 of 49 14. Aug. 2018

List of Figures

Figure 2-1: Bristol is Open Infrastructure. ....................................................................................................... 10

Figure 2-2: Final Demonstrator Topology in the city of Bristol including 5G-XHaul extensions. .................... 11

Figure 2-3: Full network diagram of the 5G-XHaul Demo topology. Devices in blue and Sub-6 devices have been introduced by 5G-XHaul partners. .................................................................................................. 12

Figure 3-1: Overall setup diagram of the converged front- and backhaul demo. ........................................... 13

Figure 3-2: TSON 5G-XHaul implementation architecture for 5G-XHaul final demonstration........................ 14

Figure 3-3: a) Massive MIMO RU deployed in HPN laboratory, b) Baseband unit deployed at HPN laboratory. ................................................................................................................................................................. 15

Figure 3-4: Comparison of required transport data rate for classical CPRI and 5G-XHaul split A. ................ 15

Figure 3-5: TUD’s BBU and USRP. ................................................................................................................ 16

Figure 3-6: Flexible time frequency resource grid of GFDM. .......................................................................... 16

Figure 3-7: Properties of GFDM waveform. .................................................................................................... 17

Figure 3-8: USRP deployed at HPN lab. ........................................................................................................ 17

Figure 3-9: GUI illustrating 16-QAM demodulated signal in real time of the GFDM FPGA transceiver. ....... 18

Figure 3-10: on-site photo of the front- and backhaul deployment. ................................................................ 18

Figure 3-11: on-site photo of the front- and BH deployment. ......................................................................... 19

Figure 3-12: Ethernet BER measurements. .................................................................................................... 20

Figure 3-13: CPRI LVC measurements. ......................................................................................................... 20

Figure 3-14: TSON node1 CPRI reference clock and TSON node2 CPRI recovered clock. ......................... 21

Figure 3-15: CPRI recovered clock spectrum. ................................................................................................ 21

Figure 3-16: Power and extinction ratio at 10 Gb/s of 16 channels. ............................................................... 22

Figure 3-17: BER of PRBS-31 at 10 Gb/s after 10 km fibre transmission. ..................................................... 22

Figure 3-18: One-way transmission latency results of the cross-connect (transponder). .............................. 22

Figure 3-19: MEMS-VCSEL SFP+ wavelength tuning procedure. Left) coarse sweep to find the correct channel grid; Right) fine tune sweep to optimise the central wavelength according to the received PT power at the OLT. .................................................................................................................................... 23

Figure 3-20: Bit error test on CPRI link ........................................................................................................... 23

Figure 3-21: Bit error test on CPRI link with interim disconnect. .................................................................... 24

Figure 3-22: FH latency measurement with approx. 5 metres of fibre length. ................................................ 24

Figure 3-23: FH latency measurement with approx. 8 km of fibre length. ...................................................... 24

Figure 3-24: Physical topology deployed in the Bristol city front. ................................................................... 25

Figure 3-25. SDN topology discovered by the SDN controller. ...................................................................... 26

Figure 3-26: I2CAT Sub-6 link between M Shed and Pero’s bridge. .............................................................. 27

Figure 3-27: BWT SDN-enabled 60GHz nodes at Waterfront and Millennium Square. ................................. 27

Figure 3-28: iPASOLINK 60 GHz nodes in M Shed and Wapping Wharf. ..................................................... 28

Figure 3-29: Main network path followed by slice 1 and slice 2. .................................................................... 30

Figure 3-30: End-to-end latency of slice 1 and slice 2. Latency contribution from the Sub-6 and mmWave links. ................................................................................................................................................................. 31


H2020-ICT-2014-2 671551 Page 6 of 49 14. Aug. 2018

Figure 3-31: CDF of the end-to-end throughput experienced by slice 1 and slice 2. ..................................... 32

Figure 3-32. CDF of throughput sustained through the Sub-6 and mmWave segments of the network. ....... 32

Figure 3-33: Pero’s Bridge – Millennium Square performance metrics. ......................................................... 33

Figure 3-34: Performance metrics for Millennium Square to AT Bristol. ........................................................ 34

Figure 3-35: The 5G-XHaul Dashboard. ......................................................................................................... 35

Figure 3-36: Slice 1 UL and DL paths after Sub-6 link breaks. ...................................................................... 36

Figure 3-37: Slice 1 and Slice 2 throughput after link breaks. ........................................................................ 37

Figure 3-38. Slice 1 TCP out of order packets received after link breaks. ..................................................... 38

Figure 3-39. Broken SDN topology detcted by the SDN controller. ................................................................ 38

Figure 4-1: 60 GHz beamforming RF front-end solution. ............................................................................... 39

Figure 4-2: RF front-end boards attached to the metal carriers for use in demonstration activities. .............. 39

Figure 4-3: Three 60 GHz beamforming RF boards attached to baseband SDR platforms deployed in the HPN lab. ........................................................................................................................................................... 40

Figure 4-4: Illustration of 60 GHz beam steering demo. ................................................................................. 40

Figure 4-5: 60 GHz beam steering demo GUI showing the received QPSK signal constellation at the Slave 1 node. ........................................................................................................................................................ 41

Figure 4-6: Illustration of 60 GHz ranging demo. ............................................................................................ 41

Figure 4-7: 60 GHz ranging demo GUI showing the distance estimate in metres between the Master and Slave 1 nodes. ................................................................................................................................................... 42

Figure 4-8: Indoor P2MP configuration of BWT Typhoon modules, (a) Typhoon unit 01-2f, (b) layout of units, plan view, (c) photograph of the three units. ........................................................................................... 43

Figure 4-9: Metrics for link to STA 01-13. ....................................................................................................... 44

Figure 4-10: Metrics for link to STA 01-2f. ...................................................................................................... 45


H2020-ICT-2014-2 671551 Page 7 of 49 14. Aug. 2018

List of Tables

Table 3-1: Ethernet upstream/downstream latency. ....................................................................................... 20

Table 3-2: CPRI upstream/downstream latency. ............................................................................................ 21


H2020-ICT-2014-2 671551 Page 8 of 49 14. Aug. 2018

Executive Summary

To address the high bandwidth and low latency requirements of 5G networks, 5G-XHaul proposed converged optical and wireless networks technologies that form a flexible transport infrastructure supporting both back-haul and fronthaul services managed by a single unified Software Defined Networking (SDN)-based control plane. 5G-XHaul’s Work Package 5 (WP5) focuses on the demonstration and evaluation of the main architec-tural functionalities and features described in detail in previous technical WPs. These demonstration activities have been carried out through a set of planned validation experiments that took place in the project testbed facilities, as reported in deliverables D5.1 and D5.2.

In this context, deliverable D5.3 reports on the results stemming from the final 5G-XHaul demonstrator de-ployed and integrated in the state-of-the art “City of Bristol” transport network infrastructure (Bristol Is Open). More specifically, all wireless and optical data technologies and most of the control plane technologies devel-oped within the project have been interconnected with the existing infrastructure to assess the end-to-end 5G-XHaul proposed solution and its performance evaluation in a realistic 5G environment.


H2020-ICT-2014-2 671551 Page 9 of 49 14. Aug. 2018

1 Introduction

The 5G-XHaul solution involves a flexible transport infrastructure consisting of optical and wireless networks technologies which support both backhaul (BH) and fronthaul (FH) services managed by a single unified Soft-ware Defined Networking (SDN)-based control plane.

WP5 is the 5G-XHaul work package responsible for the experimentation and demonstration activities of the project aiming to prove the feasibility of the 5G-XHaul solution. More specifically, WP5 focuses on the demon-stration and performance evaluation of the main architectural functionalities and features described in detail in the technical WPs.

The approach followed for evaluating the proposed solution in WP5 was first to use the available lab testbeds in the project, namely NITOS from University of Thessaly and HPN lab from University of Bristol, to validate individual solutions in isolated lab conditions. The results from the lab tests and demos of the developed SDN functionality over Sub-6 and millimetre wave (mmWave) wireless transport technologies, performed in NITOS were reported in deliverable D5.1. The results from the integration of wireless (Massive MIMO) and optical (WDM-PON and TSON) converged transport technologies supporting backhaul and fronthaul traffic, which took place in HPN lab, have been included in deliverable D5.2.

Subsequently, the approach entailed the test and demonstration of an the integrated 5G-XHaul solution in a realistic environment in the city of Bristol testbed, called Bristol is Open (BIO).

In this context, this deliverable reports on the results from the final 5G-XHaul demonstrator deployed in the state-of-the art “City of Bristol” transport network infrastructure (BIO), where all the wireless and optical data and control plane technologies, developed within the project, have been integrated and deployed. The smart city testbed was used for the assessment of the end-to-end 5G-XHaul proposed solution and its performance evaluation in a realistic 5G environment.

Organisation of the document

This deliverable is structured in five main sections. Following the introduction section, Section 2 provides a high-level presentation of the smart city testbed in Bristol and the specific network topology and scenario set up in the final demonstration. Section 3 concentrates on the results from two main scenarios. First, the con-verged BH and FH scenario using the massive MIMO radio unit and the Baseband Unit (BBU) emulator over the optical technologies developed within the project, namely WDM-PON and TSON. Second, the SDN wire-less BH scenario demonstrating the developed control plane functionality over a Sub-6 and mmWave transport network. This section mainly describes the details of the specific scenarios, the deployment of the new com-ponents in BIO and the results collected from specific experiments for every use case. Section 4 presents results from individual lab tests, i.e. the 60 GHz transceiver solution developed by IHP/TES demonstrating beam steering and ranging capabilities, and the point to multi-point capabilities of the BWT mmWave nodes. Finally, Section 5 provides a summary and the main conclusions of the deliverable.


H2020-ICT-2014-2 671551 Page 10 of 49 14. Aug. 2018

2 BIO testbed description and final demo topology

2.1 Bristol Is Open testbed infrastructure

Bristol is Open (BIO) provides an experimental infrastructure around the Bristol Harbourside area. It consists of a research network integrating optical, wireless, IoT and computing to provide a unique open and program-mable experimental platform in the centre of Bristol. BIO assets include four main locations connected by a multi-fibre ring with switching and OpenStack compute facilities at each site. Wireless equipment is deployed mostly on lampposts and adjacent street cabinets along with some buildings and sites, which are connected through either fibre or wireless links to the main locations. The University of Bristol building (Merchant Ventur-ers Building) in Woodland Road is the location where the main firewall and monitoring systems are deployed through which VPN and internet access can be provided.

BIO offers the testbed as a utility for experimentation. The objective of 5G-XHaul demonstration activities was to provide the necessary extensions to the existing BIO testbed, in order to evaluate the performance of the end-to-end proposed solution in a realistic environment. In Figure 2-1 we can see an overview of the trial area without the 5G-XHaul extensions.

Figure 2-1: Bristol is Open Infrastructure.

2.2 Final Demonstrator overall topology and scenario

The objective of 5G-XHaul is to provide a single transport network infrastructure supporting both FH and BH services under a unified SDN control plane. To achieve this goal, and starting from the testbed described in Section 2.1, we mapped to this testbed our architectural vision leaning on the available data plane technolo-gies from the various 5G-XHaul partners.

The technologies that were integrated included Wavelength Division Multiplexing Passive Optical Network (WDM-PON) and the Time-Shared Optical Network (TSON) in the optical domain; to mmWave and Sub-6 technologies (BH and massive MIMO) in the wireless domain.

The exact locations of the deployed equipment are highlighted in the map shown in Figure 2-2. In this figure, we can see the SDN enabled 60 GHz and Sub-6 wireless nodes, connected through a fibre link up to the HPN laboratory, where the optical technologies and the C-RAN implementation using a Massive MIMO RU and a Baseband Unit (BBU) emulator are located.


H2020-ICT-2014-2 671551 Page 11 of 49 14. Aug. 2018

Figure 2-2: Final Demonstrator Topology in the city of Bristol including 5G-XHaul extensions.

During our demonstration, BH traffic was created from Media Servers located at the HPN laboratory to clients connected in the available Access Points (APs) in Wapping Wharf and M Shed passing through all the avail-able wireless technologies in the city centre as well as the optical technologies in the HPN lab. At the same time, FH traffic was generated from the BBU passing again through the WDM-PON and TSON optical nodes located in the HPN laboratory, thus demonstrating a converged infrastructure supporting both Common Public Radio Interface (CPRI) and video (backhaul) traffic. Finally, a SDN controller was deployed at a compute node in the HPN lab managing all the SDN enabled wireless devices of the network providing flexibility and resili-ence to link failures through mechanisms deployed in WP3.

Figure 2-3 shows the detailed logical network topology with all the different components integrated in the 5G-XHaul demonstration. In the next section, we provide a detailed description of the scenarios involving the converged optical and FH infrastructure deployed at the HPN lab as well as the SDN-enabled wireless transport infrastructure deployed in the city of Bristol. This document provides photos of the deployment as well as results collected during the final demonstration.


H2020-ICT-2014-2 671551 Page 12 of 49 14. Aug. 2018

Figure 2-3: Full network diagram of the 5G-XHaul Demo topology. Devices in blue and Sub-6 devices have been introduced by 5G-XHaul partners.


H2020-ICT-2014-2 671551 Page 13 of 49 14. Aug. 2018

3 Final demonstration scenarios and experimental results

In this section we present the results from the two main scenarios. The first scenario consists of the converged BH and FH scenario using the massive MIMO radio unit and the Baseband Unit (BBU) emulator over the optical technologies developed within the project, namely WDM-PON and TSON. The second scenario con-sists of the SDN wireless BH scenario demonstrating the developed control plane functionality over a Sub-6 and mmWave transport network. We describe the details of the specific scenarios, the deployment of the new components in BIO, and the results collected from specific experiments for every use case

3.1 Converged front- and backhaul demo and results

This demonstration aims at evaluating the functionality and performance of: (a) the optical technologies de-veloped within the project used to transport for both FH and BH services, i.e. dynamic and elastic time-shared optical network (TSON), and flexible and wavelength-agnostic wavelength division multiplexing passive optical network (WDM-PON); and (b) the beamforming-enhanced massive MIMO radio unit (RU), the universal soft-ware radio peripheral (USRP) as a 5G user equipment (UE) and a BBU emulator realising our FH network. In the following subsections, detailed results of each subsystem will be elaborated.

3.1.1 Scenario / Topology Description

As shown in Figure 3-1, the green frame depicts the converged FH/BH setup in the HPN laboratory, where two Ethernet BH streams and one FH CPRI stream are transported over the described infrastructure. The CPRI data containing the I/Q samples of MIMO radio signals is generated by a BBU emulator, while the BH data using the 10GbE protocol (i.e. two video streams) is generated from the video servers and VLAN-tagged by an Edge Transport Node (ETN)1 therein.

Figure 3-1: Overall setup diagram of the converged front- and backhaul demo.

Both front- and backhaul data are terminated by the first TSON node, and two in-line traffic analysers are used to monitor the throughput and bit error. The TSON network converges and transports these two services to the second TSON node. In the downstream, the second TSON node disaggregates two Ethernet BH streams to two 10GbE egress ports according to the corresponding VLAN tags, while it aggregates again the backhaul upstream to a single 10GbE stream.

The WDM-PON further multiplexes the front- and backhaul streams onto separate DWDM wavelengths with 100 GHz spacing in the C-band. Since the downstream and upstream are divided into two wavelength groups,

1 The interested reader is referred to deliverables D3.1 and D3.2 for a description of the 5G-XHaul virtualization solution

TSON 1 TSON 2 ADVAOLT

ADVA

RNADVA ONU 3

BBU Emulator

RRHAIR

I/Q

sa

mp

les

BIO

Si

ngl

e D

ark

Fib

re

BIO Dark Fibres

SFP+

SFP+

2 fibre

HPN Lab

SFP+

SFP+ SFP

+SFP+

I/Q samples

2 fibres

2

2

5G UE

Conducted RF


H2020-ICT-2014-2 671551 Page 14 of 49 14. Aug. 2018

only a single dark fibre is required between the OLT and RN, which saves the fibre resource. On the ONU side, ONU 1 and 2 forward the BH streams to another remote NEC switch, which further connects to the wireless segment of the architecture, while ONU 3 is connected to the massive-MIMO RU. In addition, two spare ONUs, not transporting any service, are used to demonstrate the autonomous wavelength adaptation to the OLT.

All client interfaces between different sub-systems are optical SFP+ modules operating at 1310 nm.

Time-shared optical network (TSON)

Figure 3-2 depicts the TSON architecture for the 5G-XHaul final demonstration to support both optical FH and BH services. The configuration involves two TSON edge nodes employing Xilinx VC709 evaluation boards. TSON edge node 1 is connected to an ADVA OLT with three clients comprising two Ethernet and one CPRI ports. TSON edge node 2 is connected to the video servers and BBU emulator with two clients, including one Ethernet and one CPRI ports, respectively. In the upstream scenario, TSON Edge node 1 receives the Ether-net and CPRI traffic from the ADVA OLT, aggregates two Ethernet traffic streams into one flow and sends both CPRI and the aggregated Ethernet packets to the TSON Edge node 2. TSON Edge node 2 receives the aggregated Ethernet and CPRI traffic and directs it to the video servers and the BBU emulator, respectively.

Figure 3-2: TSON 5G-XHaul implementation architecture for 5G-XHaul final demonstration.

In the downstream scenario, TSON edge node 2 receives Ethernet and CPRI traffic from the video servers and the BBU emulator respectively and sends them to the TSON edge node 1. TSON edge node 1 receives the traffic, and segregates the Ethernet traffic in two ports, based on the VLAN tags, while delivering both Ethernet and CPRI traffic to the ADVA OLT.

WDM-PON

The WDM-PON enables a wavelength-based point-to-point (P2P) connectivity between BBUs and RUs, while being also able to carry backhaul traffic. The system employed in this demonstration leverages the results in deliverables D4.2 [12] and D5.2 [7], where a tuneable laser technology based on the micro-electromechanical system (MEMS) vertical-cavity surface-emitting laser (VCSEL) offers a capacity of 10 Gbit/s per wavelength over 10 km single dark fibre.

The tuneable MEMS-VCSEL is packaged in a SFP+ form factor with an additional 9 mm housing length. The MEMS-VCSEL is a much lower cost option than used in other commercially available tuneable SFP+, due to its simpler tuning and testing (1 input instead of 3+ inputs used in tuneable edge emitters), as well as simpler epitaxy and fabrication processes. Moreover, the module also has integrated an embedded communication channel for receiving the control signal from the OLT and responding to its commands to find the edge of the optical filter, and then fine tune the laser wavelength to the centre of the band.


H2020-ICT-2014-2 671551 Page 15 of 49 14. Aug. 2018

Figure 3-3: a) Massive MIMO RU deployed in HPN laboratory, b) Baseband unit deployed at HPN la-boratory.

Massive MIMO Radio Unit

Figure 3-3 a) shows a picture of the deployed massive MIMO radio unit (RU) developed by AIR. It is an upgraded version of the hardware described in deliverables D4.12 [6] and D5.2 [7]. Notably, the RU consists of an 8x8 antenna array with cross-polarised antenna elements. Up to 8 independent beams can be formed to support spatial multiplexing. Four 20 MHz carriers are supported, resulting in a total occupied bandwidth of 80 MHz. The RU features a FH interface in the form of four optical SFP interfaces supporting CPRI. The RU furthermore features a 1 GbE control interface.

The main feature, however, is the integrated PHY-layer processing. The antenna implements digital beam-forming on FPGA within the antenna, which corresponds to 5G-XHaul’s functional Split A as described in e.g. D2.2 [1]. In classical CPRI, one data stream would need to be transported per each of the 64 antenna elements. With split A, only one data stream for each of the 8 user beams per carrier needs to be transported. This yields a reduction in required transport capacity as highlighted in Figure 3-4. With split A, the required 39.3 Gb/s can be transported on the four optical FH interfaces, each featuring CPRI line rate 7 (9.8 Gb/s). The classical CPRI split however, would have required a transport data rate of 314.6 Gb/s, which would be require 32 optical interfaces with CPRI line rate 7. Note, that for split A, the CPRI protocol can still be used, as time domain samples are transported. However, they are no longer on an antenna-carrier basis, but instead on a beam-carrier basis. For the demonstration, only a single carrier and a single beam were used, hence one optical interface with CPRI line rate 5 (4.9 Gb/s) was sufficient.

Figure 3-4: Comparison of required transport data rate for classical CPRI and 5G-XHaul split A.

Baseband Unit (BBU)

The BBU consists of two parts: a real time Generalized Frequency Division Multiplexing (GFDM) baseband, implemented on USRP, and a BBU emulator, implemented on Kintec-7 FPGA KC705 board. The KC705 implements the CPRI protocol on FPGA and provides the optical interface towards the massive MIMO RU. It stores IQ samples provided by the GFDM baseband on RAM memory and can then forward them to the RU in real time.

1 GbE control interface

Optical FH interface (CPRI)

Power supply

Radio Unit

Mounting pole

RF interface to UE

BBU Emulator

Fronthaul interface (back side)

39,32

314,57

0,00 50,00 100,00 150,00 200,00 250,00 300,00 350,00 400,00Fronthaul data rate in Gbps

CPRI

Split A


H2020-ICT-2014-2 671551 Page 16 of 49 14. Aug. 2018

Figure 3-5: TUD’s BBU and USRP.

Figure 3-6: Flexible time frequency resource grid of GFDM.

TUD provides GFDM based IQ samples to the AIR BBU emulator which after being processed at the BBU emulator, are transported to the AIR massive MIMO RU, as illustrated in Figure 3-5. GFDM is a block-based multicarrier modulation technique that employs circular filtering [2] instead of linear filtering to achieve low OOB. GFDM was among the initially considered candidate waveforms for 5G. For the sake of completeness, we provide next a brief introduction to GFDM.

Let us consider a resource block of time duration 𝑇, and frequency bandwidth 𝐵. The total available bandwidth

is divided into 𝐾 equally spaced subcarriers with subcarrier spacing 𝑇sub =𝐵

𝐾, and the available time is divided

into 𝑀 subsymbols with subsymbol spacing ∆𝑓 =𝑇

𝑀 such that we have ∆𝑓𝑇sub = 1. Here, 𝑁 = 𝐾𝑀 gives total

number of symbols and by properly tuning these parameters, we can obtain different possible waveforms, e.g., by setting 𝑀 = 1, GFDM turns into conventional OFDM. Similarly, SC-FDE (single carrier frequency do-main equalization) can be obtained when 𝐾 = 1. Hence, we can say GFDM is quite flexible due to its flexible time frequency resource grid shown in Figure 3-6. Besides that, it offers other promising properties, which are highlighted and illustrated in Figure 3-7.


H2020-ICT-2014-2 671551 Page 17 of 49 14. Aug. 2018

Figure 3-7: Properties of GFDM waveform.

Figure 3-8: USRP deployed at HPN lab.

In GFDM, the transmit samples are generated as follows:

𝑥[𝑛] = ∑ ∑ 𝑔𝑘,𝑚[𝑛]𝑑𝑘,𝑚

𝑀−1

𝑚=0

𝐾−1

𝑘=0

, 𝑛 = 0,1, … , 𝑁 − 1 (1)

where 𝑔(𝑛) is a prototype filter and 𝑔𝑘,𝑚[𝑛] refers its time and frequency shifted version. Parameter 𝑑 denotes

a data block that consists of 𝑁 = 𝐾𝑀 symbols. The details of GFDM transceiver design and its properties can be found in [2].

The demodulated signal from the AIR massive MIMO antenna is received via cable to one of the receiving ports of the USRP shown in Figure 3-8, since there is no provision of wireless transmission due to licensing issues. The USRP deployed at the HPN lab is the NI USRP 2954R.

The GUI as shown in Figure 3-9 depicts the final 16 QAM demodulated signal. The constellation diagram shows real demodulated signal received through TSON, WDM-PON and the AIR massive MIMO antenna, thus confirming the excellent performance of the integrated FH chain. Although for the integration tests we have adopted 16-QAM, the GFDM transceiver module is flexible enough to support any suitable higher mod-ulation schemes.


H2020-ICT-2014-2 671551 Page 18 of 49 14. Aug. 2018

Figure 3-9: GUI illustrating 16-QAM demodulated signal in real time of the GFDM FPGA transceiver.

3.1.2 Deployment

Overall Deployment

Figure 3-10 depicts the overall deployment of the converged front- and backhaul demo setup in the HPN laboratory, where each key sub-system developed in the project is highlighted.

Figure 3-10: On-site photo of the front- and backhaul deployment.

TSONNode 1

TSONNode 2

ADVAOLT&ONU

AIRMassiveMIMO

TUDUE

BBU


H2020-ICT-2014-2 671551 Page 19 of 49 14. Aug. 2018

WDM-PON Deployment

Figure 3-11 shows the WDM-PON demo setup in the HPN laboratory. At the OLT side, a cross-connect tran-sponder card was used to convert the grey light interface of CPRI and BH Ethernet data to the DWDM wave-lengths, which were emitted from the tuneable MEMS-VCSEL SFP+s. After a multiplexing stage and trans-mission over an 8 km fibre spool emulating a typical FH distance, the wavelengths were demultiplexed at the remote node and fed into the ONU line card. In the demo system, each ONU was effectively a tuneable MEMS-VCSEL SFP+, and the line card as a transponder converted the DWDM wavelength back to the grey light for each ONU.

Moreover, the transmitted wavelength of those two spare ONUs were monitored on the optical spectrum an-alyser (OSA). Once these two ONUs were connected to any of the provisioned ports on the remote node, they started receiving the out-of-band communication commands from the OLT, and the emitted wavelength was autonomously tuned to the paired upstream wavelength, as confirmed on the OSA.

Figure 3-11: WDM-PON demo setup in the HPN laboratory.

Massive MIMO Radio Unit Deployment

Figure 3-3 a) shows the massive MIMO RU deployed in the HPN laboratory. The RU was mounted on a short pole. On the bottom, the optical FH interface, the Ethernet control interface, and the power supply are visible. The radome has been cut away to allow access to the antenna ports. The RF cable connecting the RU to the UE is also visible.

Figure 3-3 b) shows the baseband unit. The KC705 board is inside the grey box, the FH interface is on the back side of the box.

3.1.3 Integration/Validation Tests

TSON Tests

We have evaluated the TSON 5G-XHaul implementation architecture for both FH and BH services. Three different scenarios are considered for TSON experimental evaluation. The first scenario includes both TSON Edge nodes connected back-to-back with short fibres. In the second and third scenarios, the proposed tech-nologies are evaluated over the Bristol City Metro Fibre of 8 km and 16 km, respectively, of standard single-mode fibre (SSMF).

Remote node

ONUs

OLT

Fibrespool

OSA


H2020-ICT-2014-2 671551 Page 20 of 49 14. Aug. 2018

The Ethernet performance parameters under consideration include Bit Error Rate (BER) and latency. A traffic analyser (Anritsu MT1100) generates/receives two Ethernet traffic streams to/from the TSON edge node 1 and 2 at 4.4 Gb/s with fixed frames of 1500B length. Figure 3-12 shows the Ethernet BER measurements as a function of received optical power for the different considered scenarios. The BER curves show that the penalty observed for the case of 8 km of SSMF transmission over the Bristol City Metro Fibre Infrastructure compared to the back-to-back (B2B) performance is negligible. A penalty lower than 1dB is observed for a 16 km transmission.

Table 3-1 displays the upstream/downstream latency for Ethernet scenario. TSON edge node worst-case latency is less than 4%.

The CPRI performance parameters considered include Line Code Violations (LCV) (8B/10B decoding) ratio and latency. The traffic analyser generates/receives CPRI traffic to/from the TSON node 1 and 2 at 4.9152 Gb/s. Figure 3-13 demonstrates the CPRI LVC measurements as a function of received optical power for the different scenarios considered. The LVC curves reveal that the penalty for both 8 km and 16 km compared to the B2B case is as low as <1dB.

Figure 3-12: Ethernet BER measurements.

Table 3-1: Ethernet upstream/downstream latency.

Destination Ethernet Latency [µs]

B2B 1.57

8 km 43

16 km 84.54

Figure 3-13: CPRI LVC measurements.


H2020-ICT-2014-2 671551 Page 21 of 49 14. Aug. 2018

The VC709 includes a Silicon Labs Si5324 jitter attenuator on the board. The Si5324 is used for CPRI clock recovery from a user-supplied SFP/SFP+ module and use the jitter-attenuated recovered clock to drive the reference clock inputs of a GTH transceiver. Figure 3-14 illustrates the TSON node 1 CPRI reference clock and TSON 2 recovered clock. As observed in Figure 3-14, the CPRI clock is successfully recovered. Figure 3-15 shows the CPRI recovered clock spectrum.

Figure 3-14: TSON node1 CPRI reference clock and TSON node2 CPRI recovered clock.

Figure 3-15: CPRI recovered clock spectrum.

Table 3-2: CPRI upstream/downstream latency.

Destination CPRI Latency [µs]

B2B 0.45

8 km 42.04

16 km 83.38

Table 3-2 shows the upstream/downstream latency for CPRI scenario. TSON edge node worst-case latency is less than 1.5%.

WDM-PON Tests

Since the tuneable MEMS-VCSEL SFP+ plays the key role in the WDM-PON transmission performance, the latest-released modules used in the demonstration were thoroughly investigated. The modules are capable of a 14+ nm tuning range with 10 Gb/s performance (equivalent to 16+ channels spaced at 100 GHz) and a side-mode suppression ratio (SMSR) exceeding 50 dB. They have ~+1~-3 dBm output power across their tuning range as shown in Figure 3-16. Uniform extinction ratios of 5±0.1 dB are also seen across the tuning range of the device at 10 Gb/s.


H2020-ICT-2014-2 671551 Page 22 of 49 14. Aug. 2018

Figure 3-16: Power and extinction ratio at 10 Gb/s of 16 channels.

To measure the BER, a 10 Gb/s pseudorandom binary sequence (PRBS) with the pattern length of 231-1 was modulated on one of the ONUs. After 10 km fibre transmission, the BER as a function of received optical power at the OLT transceiver is shown in Figure 3-17. Since the MEMS-VCSEL SFP+ does not have a clock and data recovery (CDR) functionality, the BER performance could be slightly improved if the client device is capable of CDR. The downstream performance from OLT to ONUs exhibited the similar performance, as modules used on both sides had the same released version.

Figure 3-17: BER of PRBS-31 at 10 Gb/s after 10 km fibre transmission.

Figure 3-18: One-way transmission latency results of the cross-connect (transponder).

2

4

6

8

10

12

-27 -26 -25 -24 -23 -22 -21 -20 -19 -18 -17 -16 -15

-Lo

g(B

ER)

Received Optical Power (dBm)

CDR off

CDR on

0

1

2

3

4

5

6

0 2 4 6 8 10 12

Late

ncy

(n

s)

Data Rate (Gb/s)

Mean Latency C > N

Mean Latency N > C


H2020-ICT-2014-2 671551 Page 23 of 49 14. Aug. 2018

One way latency of the cross-connect at a transponder was characterized for different line rates, and the two forwarding directions between client ports (C) and network ports (N) showed same results, where the latency decreased down to 2.57 ns as the line rate increased to 10.1376 Gb/s. Results are shown in Figure 3-18.

The module has integrated preliminary ITU-T G.698.4 (ex G.metro) [11] functionality implemented for auto-matically changing channels and fine-tuning the centre wavelength of the device within a channel. A control signal is sent from the OLT to the ONU module as a low amplitude (~8%), low speed signal (500 b/s) on top of the high-speed signal (not transmitted in this demo case). The Main Control Unit (MCU) of the SFP+ re-ceives the signal after low speed filter from the Receiver Optical Sub-Assembly (ROSA) and decodes the signal, and then responds to the command, either changing channels or tuning within the channel to fine tune the wavelength.

Figure 3-19: MEMS-VCSEL SFP+ wavelength tuning procedure. Left) coarse sweep to find the correct channel grid; Right) fine tune sweep to optimise the central wavelength according to the received PT

power at the OLT.

The centralized wavelength locker at the OLT monitors a unique pilot tone (PT) label superimposed on the upstream data of each ONU, in order to precisely tune and constantly track each upstream wavelength. Figure 3-19 illustrates the tuning procedure of the tuneable MEMS-VCSEL SFP+ at the ONU. More elaborated details were provided in deliverable D4.2 [12].

Massive MIMO Radio Unit Tests

For the massive MIMO RU, first, general functionality of the setup was evaluated. Using the CRC check of the CPRI protocol, the number of byte errors on the CPRI interface were analysed. Figure 3-20 shows a screen-shot of the corresponding control interface. As can be seen, no errors are occurring, showing the setup was running perfectly.

The error test was performed with a duration of up to 10 minutes and still showed zero errors. At a line rate of 4.9152 Gb/s, this shows that the bit error rate was less than BER < 1/(10 ⋅ 60 s ⋅ 4.9152 Gb/s) = 3.4 ⋅ 10−13, which is better than the bit error rate of 10-12 required by the CPRI standard.

Figure 3-20: Bit error test on CPRI link

Furthermore, it was shown that the CPRI link can recover after a disconnect. For this, the optical cable was disconnected while the link was running, and then disconnected. In Figure 3-21, a screenshot of the corre-sponding measurement is shown. As can be seen, first, no errors are occurring. After the disconnect, the error count increases. After the reconnect the error count stops increasing, showing that no additional errors are occurring, i.e. that the link is again working perfectly.


H2020-ICT-2014-2 671551 Page 24 of 49 14. Aug. 2018

Figure 3-21: Bit error test on CPRI link with interim disconnect.

Figure 3-22: FH latency measurement with approx. 5 metres of fibre length.

Next the latency on the CPRI link, including PON and TSON was evaluated. For this, CPRI frames were send from the BBU to the RU and immediately sent back to the RU. With this, the BBU could estimate the round-trip delay time.

First the measurements were performed with only approximately 5 metres of fibre length. Accordingly, the measured delay should mainly consist of the processing delay of the CPRI protocol and intermediate nodes. The measure delay is shown in Figure 3-22, which indicates delay of 218 samples. At the sample rate of 122.88 MHz, this corresponds to 1.77 µs, which is significantly below the CPRI requirement of 5 µs.

Next, a 8 km fibre was included in the link. Now, the delay should be dominated by the propagation time of the signal on the optical fibre. With 16 km of round-trip distance and a speed of light on fibre of approximately

2 x 108 m/s, this should yield a latency of approximately 16 km/2 ⋅ 108 m/s = 80 µs. The corresponding meas-urement is given in Figure 3-23, which shows a latency of 10472 samples or 85.22 µs.

Figure 3-23: FH latency measurement with approx. 8 km of fibre length.


H2020-ICT-2014-2 671551 Page 25 of 49 14. Aug. 2018

3.1.4 Conclusions

Overall, we successfully integrated and demonstrated the complete converged front- and backhaul service scenario over a single transport infrastructure capable of delivering the CPRI data for the massive MIMO RH and two Ethernet backhaul services for the wireless SDN at the same time. The performance of each sub-system has been thoroughly analysed in this demonstration measuring the round-trip delays and throughput for both BH and FH traffic. The correct reception of the 16-QAM constellation, while concurrently transporting Ethernet traffic, validates the ability of the 5G-XHaul architecture to jointly transport BH and FH services.

3.2 SDN Wireless Backhaul demo and results

3.2.1 Scenario / Topology Description

Figure 3-24 shows the specific part of the demo topology concerning the SDN-controlled wireless nodes and links:

Figure 3-24: Physical topology deployed in the Bristol city front.

Three MERU APs were deployed at Wapping Wharf and M Shed to provide connectivity to the end users in the city. These APs were announcing two different SSIDs (Service Set Identifier), 5GXHAUL2128 and 5GXHAUL2129.

The SDN controlled components of the network were located on M Shed east, Millennium Square, Pero’s Bridge and AT Bristol (aka. We The Curious). The SDN controlled devices were Linux based systems with an OpenVSwitch (OvS) instance running, and connected to an instance of OpenDaylight (ODL, Boron distribu-tion) on a Virtual Machine (VM) instantiated on an OpenStack compute node located at HPN lab; the VM was a Linux-based system (Ubuntu 14.04.5 LTS). The devices connected to the controller were:

BWT Typhoon (60 GHz).

I2CAT Sub-6 (Wi-Fi).

BLU switch – a commercial solution offering six 1 Gb/s Ethernet ports.

In Figure 3-25 we can see the topology detected by the SDN controller:


H2020-ICT-2014-2 671551 Page 26 of 49 14. Aug. 2018

Figure 3-25. SDN topology discovered by the SDN controller.

Figure 3-25 shows a chain of 10 SDN switches, including:

- The node at M Shed East (MSE) (openflow:...2093).

- The Sub-6 link (nodes 5660, 6775).

- The two BWT links: 6922 on Pero’s Bridge connecting to 6914 on Millennium Square, and node 6482 on Millennium Square connected to 6956 on AT Bristol.

- The BLU switch located on AT Bristol.

Finally, the Media Servers were two different OpenStack instances (VMs) running on a compute node located at the HPN lab. The Media servers were a Linux Based system (Ubuntu 14.04.5 LTS) that acted as a DHCP and repository for media content; for the DHCP server we installed the dnsmasq software and for the Media Server we installed and configured VLC (VideoLAN media player, link) to stream in a loop fashion the same video with a fixed quality (720p).

The DHCP servers were configured to provide the clients connected to the MERU APs IP address on the same network/broadcast domain as the Media Server itself; these where the networks defined:

Media Server 1 (slice 1): 192.168.128.0/24.

Media Server 2 (slice 2): 192.168.129.0/24.

3.2.2 Deployment

The photos below show the deployed nodes across the city. Figure 3-26 shows the two I2CAT Sub-6 nodes, one on the roof of M Shed, and the other on a lamp post in Pero’s Bridge, which is collocated with the SDN mmWave node from BWT. SDN-enabled 60 GHz nodes from BWT were also deployed in the Waterfront and at We The Curious in Millennium Square (see Figure 3-27).

https://www.videolan.org/vlc/


H2020-ICT-2014-2 671551 Page 27 of 49 14. Aug. 2018

Figure 3-26: I2CAT Sub-6 link between M Shed and Pero’s Bridge.

Figure 3-27: BWT SDN-enabled 60GHz nodes at Waterfront and Millennium Square.

Finally, iPASOLINK 60 GHz links [14], already included in the BIO network, were also used located in M Shed and Wapping Wharf (Figure 3-28):


H2020-ICT-2014-2 671551 Page 28 of 49 14. Aug. 2018

Figure 3-28: iPASOLINK 60 GHz nodes in M Shed and Wapping Wharf.

3.2.3 Integration/Validation Tests

To demonstrate the delivery of backhaul services supported by the 5G-XHaul SDN control plane, the 5G-XHaul network was sliced in two different end-to-end paths as shown in Figure 3-29. Slice 1 was announced through the 5GXHAUL2128 SSID, and slice 2 was announced on 5GXHAUL2129 SSID; both slices used WPA2 for security reasons and had a different password, and both SSIDs were announced on the 2.4 GHz and the 5 GHz bands. Each SSID was mapped to a unique 802.1q VLAN which was added to the header of the packets received from the clients (i.e. upstream traffic to the Media Servers). The VLAN IDs were 2128 for slice 1 and 2129 for slice 2.

The node located at MSE was configured as an Edge Transport Node (ETN) and was responsible for the mapping between 8021q VLANs representing each slice and the 5G-XHaul end-to-end label switched paths. The translation turned the slice-related VLAN into an internal tunnel ID which was used by the SDN controlled nodes to match on the 8021q header and chose the path for each slice. The translation in the ETNs was configured as follows:

Traffic coming from the MERU VLAN tagged as 2128 would turn into VLAN tagged traffic with VLAN ID 2111 and sent to the Sub-6 node.

Traffic coming from the MERU VLAN tagged as 2129 would turn into VLAN tagged traffic with VLAN ID 2113 and sent through the IPASOLINK (connected to the CISCO and NEC ATB).

Traffic coming from the Sub-6 VLAN tagged as 2112 would turn into VLAN tagged traffic with VLAN ID 2128 and sent to the MERU APs.

Traffic coming from the IPASOLINK node VLAN tagged as 2114 would turn into VLAN tagged traffic with VLAN ID 2129 and sent to the MERU APs.


H2020-ICT-2014-2 671551 Page 29 of 49 14. Aug. 2018

The traffic tagged with VLAN ID 2111 was considered upstream traffic going to the Media Servers. This traffic followed the following path. First, the Sub-6 nodes, and the BWT links up to the BLU switch named ATB (Figure 3-29). Second, from ATB it was forwarded to the NEC switch at ATB with the same VLAN id, and after being forwarded through the optical part of the network. Finally, it reached the OpenStack compute node, where the 802.1q header was removed, and traffic was deliver it to the Media Server 1 VM.


H2020-ICT-2014-2 671551 Page 30 of 49 14. Aug. 2018

Figure 3-29: Main network path followed by slice 1 and slice 2.


H2020-ICT-2014-2 671551 Page 31 of 49 14. Aug. 2018

On the other hand, the upstream for slice 2 with VLAN ID 2113 followed this path. First, it was resent to the CISCO switch and onwards to the iPASOLINK node between M Shed east and AT Bristol. Second, from there it was forwarded through the optical network. Finally, it reached the OpenStack compute node, where the 8021q header was removed, and traffic was delivered to the Media Server 2.

The downstream for slice 1, with VLAN ID 2112, between the Media Servers and the MERU APs deployed on the streets followed this path. First, it was VLAN tagged as 2112 by the OpenStack compute node. Second, it was forwarded through the optical part of the network to reach the ATB switch. Finally, it was forwarded through the BWT and I2CAT wireless links to reach the node at MSE.

The downstream for slice 1, with VLAN ID 2114, followed a similar path than the slice 1 downstream, but it used the IPASOLINK nodes to reach MSE instead of going through the BWT and I2CAT wireless links. The NEC ATB was configured to deliver the packets from this VLAN ID directly to the iPASOLINK.

We now assess the performance of this infrastructure measuring end to end delay and throughput for each slice, and studying separately the delay and throughput performance of the Sub6 and 60GHz technologies of the wireless segment.

Figure 3-30 depicts the end-to-end latency experienced by slices 1 and 2 through the 5G-XHaul infrastructure, together with the contribution to this latency from the Sub-6 and mmWave links. Results are plotted using a Cumulative Distribution Function (CDF). To obtain these results we left an Internet Control Message Protocol (ICMP) ping running for 120 seconds and plot the resulting Round-Trip Time (RTT) delay for different parts of the topology. Slice 1 is going through the MERU AP, the Sub-6 link and the two mmWave links before reaching the optical part of the network, while slice 2 goes through the MERU AP, the iPASOLINK and the optical segment. We can see in Figure 3-30 how the two slices differ in latency; slice 1 has an average latency of 9-10 ms while slice 2 seems to be stable at an average of 5-6 ms. We posit that this small difference on average latency is due to the additional wireless hops incurred by slice 1. Latency is kept below 15 ms for both slices in 80% of the cases, but there is a long latency tail reaching up to 100 ms for the remaining 20% of the packets. In order to discern the reasons for this latency we plot the CDFs of the Sub-6 and mmWave segments (yellow and blue lines respectively). We can see how the Sub-6 and mmWave segments introduce an average latency of 3-4 ms, and their worst-case latency is well below 15 ms. Therefore, we posit that the long latency tail is due to either delays introduced in the compute platform, recall that the ICMP processes are launched from two VMs, or from delays introduced in the wireless access, where a regular Wi-Fi network, subject to interfer-ence, is being used.

Figure 3-30: End-to-end latency of slice 1 and slice 2. Latency contribution from the Sub-6 and mmWave links.

To continue our benchmark of the 5G-XHaul network, we look at the end-to-end throughput experienced by slice 1 and slice 2, and assess the throughput capability available in each segment of the wireless network, whereas an assessment of the optical segment can be found in the previous section.


H2020-ICT-2014-2 671551 Page 32 of 49 14. Aug. 2018

To assess the end-to-end throughput available to each slice, we run an iperf from a Raspberry PI device directly connected to the MERU AP on M Shed, and an iperf server on each of the Media Servers deployed in the BiO VMs. We observe in Figure 3-31 that the throughput is quite similar for both slices, with the iperf streams being more or less stable around 25 Mb/s. The reason for this low end-to-end throughput is the fact that the bottleneck is located on the link between the Raspberry PI and the MERU AP.

Figure 3-31: CDF of the end-to-end throughput experienced by slice 1 and slice 2.

In order to assess the full capability of the 5G-XHaul wireless transport segment, we evaluate individually the throughput performance of the Sub-6 and the mmWave segments, again using the iperf tool. We can see in Figure 3-32 show the Sub-6 link has some variations on throughput but it seems to be stable at around 125 - 140 Mb/s. Regarding the mmWave segment, we observe a stable throughput around 750 Mb/s. It is worth noticing that in the mmWave case the iperf traffic is traversing two hops, namely “Pero’s Bridge – Millennium Square”, and “Millennium Square – AT Bristol”. The bottleneck in this case the “Pero’s Bridge – Millennium Square” link, which had worse LoS conditions. Figure 3-33 and Figure 3-34 depict the signal metrics and individual throughput for each mmWave link, where we indeed observe a worse signal quality in the “Pero’s Bridge – Millennium Square” link. The BSSs are configured in half-band mode, using (different) halves of the 802.11ad channel 3.

Figure 3-32. CDF of throughput sustained through the Sub-6 and mmWave segments of the network.

The performance of the mmWave links was evaluated using TCP iperf traffic, sent in both directions of each link. The performance of the links “Pero’s Bridge – Millennium Square” and “Millennium Square – AT Bristol”


H2020-ICT-2014-2 671551 Page 33 of 49 14. Aug. 2018

are shown in Figure 3-33 and Figure 3-34, respectively. The aggregate rate on each link is approximately 900 Mb/s.

Figure 3-33: Pero’s Bridge – Millennium Square performance metrics (BSSs configured in half-band mode).


H2020-ICT-2014-2 671551 Page 34 of 49 14. Aug. 2018

Figure 3-34: Performance metrics for Millennium Square to AT Bristol (BSSs configured in half-band mode).

3.2.4 Results of SDN wireless demonstration

We now describe an experiment designed to demonstrate in an operational environment some of the capabil-ities of the WP3 SDN mechanisms designed to manage traffic on the wireless segment of the network.

For this experiment, two different tablets were connected to the two SSIDs corresponding to each slice. We connected a Samsung Galaxy Tab 8’’ to 5GXHAUL2128 and a Lenovo Yoga Tab 3.8’’ to 5GXHAUL2129. Once both tablets were connected and had an IP address provided by the Media Servers, a 720HD video


H2020-ICT-2014-2 671551 Page 35 of 49 14. Aug. 2018

streaming was launched for each tablet. We used the VLC program [8] as a client on the tablets and connected it to a Media server that was streaming in loop mode for each slice.

After a few seconds of launching the video streams, the 5G-XHaul SDN controller’s dashboard started to display the tablets throughput; the links the dashboard was monitoring were the Sub-6 link and the iPASOLINK going from MSE to NEC ATB, reporting respectively aggregate traffic for slice 1 and slice 2 respectively. Traffic statistics were collected by the SDN controller via OpenFlow statistics, and exported to a Grafana based dashboard using Prometheus [8]. Figure 3-35 depicts the 5G-XHaul dashboard.

Figure 3-35: The 5G-XHaul Dashboard.

Once enough data was gathered to observe the video streaming throughput in the dashboard, we simulated a link break on the Sub-6 link; the main path for the slice 1 upstream and downstream traffic. The link break was simulated with issuing an ‘ifconfig mesh0 down’ command, which disables the wireless interface on one side of the mesh link and leaves the two Sub-6 devices without connection.

In order to quickly restore connectivity after the link break, we used the Fast Local Link Reroute (FLLR) agent introduced in deliverable D3.2 [3]. Using FLLR, each slice is configured to have a main path and a backup path, which occupy a disjoint set of network elements. Upon a link break, the local FLLR agent is notified by the wireless driver, and reconfigures the network elements to start using the backup path. The uplink and downlink backup paths for slice 1 are depicted in Figure 3-36. Notice how the slice 1 uplink backup path needs to reach up to the Sub-6 device at M Shed, to realize that the link is broken and be forwarded instead through the iPASOLINK. Once a packet arrives to the Sub-6 device and the link is broken, a regress rule sends the packet back to the MSE device (Figure 3-36). Subsequent packets are directly forwarded through the iPASO-LINK devices. This process may introduce a slight packet reordering, which will be analysed in our experiment. The slice 1 downlink backup path, follows its regular path through the mmWave devices, but when a packet reaches the Sub-6 device and detects the broken link, this packet is returned to the SDN switch at Pero’s Bridge, which starts forwarding slice 2 through the NEC switch and the iPASOLINK.


H2020-ICT-2014-2 671551 Page 36 of 49 14. Aug. 2018

Figure 3-36: Slice 1 UL and DL paths after Sub-6 link breaks.


H2020-ICT-2014-2 671551 Page 37 of 49 14. Aug. 2018

Figure 3-37 depicts the instantaneous throughput experienced by the video streams on slice 1 (upper plot) and slice 2 (lower plot), while we break the Sub-6 link. This data is obtained using the TCPdump tool to capture packets on the Sub-6 and the iPASOLINK simultaneously. The upper plot depicts in blue colour the throughput experienced by slice 1 while traversing its main path, i.e. the path through the Sub-6 link. In red, we plot the slice 1 video stream throughput through the backup path. The video stream is transmitted using TCP, which explains the two throughput peaks of 10-12 Mb/s corresponding to TCP being in Slow Start. When the Sub-6 link is broken, at around second 43, packets start flowing immediately through the downlink backup path, but we observe a second TCP Slow Start phase. To understand why TCP enters Slow Start upon the link break, we depict in Figure 3-38 the Wireshark capture on the iPASOLINK device when the Sub-6 link breaks. We can see that we have a TCP stream between IPs 192.168.128.109 (the Media Server) and 192.168.128.208 (the tablet connected to slice 1), where TCP packet reordering occurs. Packet reordering is introduced by FLLR because the packets in transit between the SDN switch at Pero’s Bridge (PRB in Figure 3-36) and the Sub-6 device when the PRB switch detects the first regressed packet, and switches to the backup path, are overcome by subsequent packets. Despite this minimal reordering that causes TCP Slow Start, the slice 1 video kept flowing smoothly at all times, which validates the practical performance of FLLR. The lower plot of Figure 3-37 depicts the throughput experienced by the video stream in slice 2. This stream kept playing smoothly at all times, even after slice 1 is rerouted to go through the same link as slice 2. The throughput levels observed 2-4 Mb/s, correspond to the used quality of 720HD.

Figure 3-37: Slice 1 and Slice 2 throughput after link breaks.


H2020-ICT-2014-2 671551 Page 38 of 49 14. Aug. 2018

Figure 3-38. Slice 1 TCP out of order packets received after link breaks.

Finally, in Figure 3-39 we can see the resulting topology from the SDN controller’s point of view with the broken Sub-6 link, splitting the topology split into two parts. It is worth noticing that the SDN controller detects the broken link seconds after the link is actually broken. The reason is the Link Layer Discovery Protocol (LLDP) protocol used by OpenFlow to discover topology. The SDN controller periodically, e.g. every 5 seconds, in-structs the SDN switches to transmit an LLDP packet to their next hop to verify that links are still alive. In addition, a timeout is introduced before the SDN controller declares a link as broken. After the SDN controller detects that the link is broken, the network could be further reconfigured if needed. Thus, our FLLR agent ensured that communication could be maintained even until the SDN controller realizes about the new network state.

Figure 3-39. Broken SDN topology detcted by the SDN controller.

We complement the description of this experiment provided in this report, with a video of the overall demo, where it can be observed how both slice 1 and slice 2 videos keep playing smoothly while the Sub-6 link is broken. This video is available in the 5G-XHaul YouTube channel [10].


H2020-ICT-2014-2 671551 Page 39 of 49 14. Aug. 2018

4 Individual Lab demos

Apart from the overall 5G-XHaul demonstrations in the field, some project partners have demonstrated addi-tional developments in a laboratory environment. Below we describe two of the demonstrations that have been carried out.

4.1 Demonstrations related to the 5G-XHaul transceiver solution

In the 5G-XHaul project, IHP and TES Electronic Solutions have developed a 60 GHz RF beamforming RF front-end. This work is reported in deliverables D4.9 [1] and D4.10 [5]. The main building blocks of this solution are the 60 GHz antenna array designed by TES, and the 60 GHz beamforming IC (BFIC) and up-/down con-version IC (UDCIC) developed by IHP. The RF front-end board was designed by TES, and the final integration of all components was carried out by IHP. The developed 60 GHz RF front-end board is shown in Figure 4-1.

The front-end module with an antenna reflector was attached to a specially designed carrier from aluminium for the final demo. Figure 4-2 depicts three of such carriers with the RF front-end boards attached.

Figure 4-1: 60 GHz beamforming RF front-end solution.

Figure 4-2: RF front-end boards attached to the metal carriers for use in demonstration activities.


H2020-ICT-2014-2 671551 Page 40 of 49 14. Aug. 2018

Finally, Figure 4-3 shows three 60 GHz nodes deployed in the HPN lab.

Figure 4-3: Three 60 GHz beamforming RF boards attached to baseband SDR platforms deployed in the HPN lab.

The functionality of the developed 60 GHz beamforming RF front-end module was demonstrated in the HPN laboratory in Bristol. The RF module was connected to a baseband (BB) FPGA platform developed by IHP, which is suitable for SDR (Software-defined Radio) applications.

Two demos were shown, namely:

1) 60 GHz beam steering demo, and

2) 60 GHz ranging demo.

In the beam steering demo, the beam steering functionality of the developed solution was shown. In this demo three nodes were used, one node being master node and two others being the slave nodes. The Master node and one slave node (aka Slave 1) were positioned face to face at a distance of 1 m, while the second slave node (aka Slave 2) was placed at the angle of 30 degrees left from the Master node at approximately the same distance. The demonstration setup is illustrated in Figure 4-4.

Figure 4-4: Illustration of 60 GHz beam steering demo.

BB

BB

BB

Master Slave 1

Slave 2


H2020-ICT-2014-2 671551 Page 41 of 49 14. Aug. 2018

The Master node was transmitting a QPSK modulated OFDM signal to Slave 1, to then, subsequently, switch its beam from Slave 1 to Slave 2. Three BB FPGA platforms were connected to a host PC running MATLAB script for signal generation and processing. The script generates signal frames to be transmitted from the Master, triggers signal transmission and reception, captures signals at two slave nodes and processes them. In the designed MATLAB Graphical User Interface (GUI) (see Figure 4-5), the QPSK signal constellation is first seen at Slave 1, to then, after beam switching, it appeared at the Slave 2.

Figure 4-5: 60 GHz beam steering demo GUI showing the received QPSK signal constellation at the Slave 1 node.

In the ranging demo, precise distance estimation between Master and Slave 1 using Two-way ranging (TWR) [13] was performed. This demonstration corresponds to the localization techniques developed in WP3. Simi-larly to the first demo, two BB FPGA nodes are connected to a host PC with a script running in MATLAB. The script triggers the TWR procedure from the Master node to estimate the distance to Slave 1. The Master was positioned at a trolley to easily change the distance between the nodes and to assess the accuracy of the ranging in a moving scenario.

The demo set up is illustrated in Figure 4-6, while the GUI that controls the demo is shown in Figure 4-7.

Figure 4-6: Illustration of 60 GHz ranging demo.

BB BB

Master Slave 1

TWR d

d


H2020-ICT-2014-2 671551 Page 42 of 49 14. Aug. 2018

Figure 4-7: 60 GHz ranging demo GUI showing the distance estimate in metres between the Master and Slave 1 nodes.

4.2 Point to MultiPoint (P2MP) mmWave transmission

The Blu Wireless mmWave technology supports point-to-multipoint transmission. This means that a PBSS can contain more than one STA, with the PCP/AP able to schedule air time for each non-PCP STA to transmit and receive traffic to/from the PCP/AP. A 3-member PBSS was setup in the BWT office, as shown in Figure 4-8. Each non-PCP STA is approximately 8m from the PCP, and is given equal air time. The air time for Tx and Rx is also equal. TCP iperf traffic is generated in each direction on each link, and metrics are captured using Grafana, Figure 4-9 and Figure 4-10. It can be seen that all links are operating at a MCS from 10-12 (12 is the maximum supported by the device). The data rates are not equal because the iperf traffic is generated on the Typhoon network processors, and rates are dictated by the CPU loading.


H2020-ICT-2014-2 671551 Page 43 of 49 14. Aug. 2018

(a) (b)

(c)

Figure 4-8: Indoor P2MP configuration of BWT Typhoon modules, (a) Typhoon unit 01-2f, (b) layout of units, plan view, (c) photograph of the three units.

Glass panel

Interior wall

STA 01-2f

PCP

STA 01-13

PCP

STA 01-2f

STA 01-13


H2020-ICT-2014-2 671551 Page 44 of 49 14. Aug. 2018

Figure 4-9: Metrics for link to STA 01-13.


H2020-ICT-2014-2 671551 Page 45 of 49 14. Aug. 2018

Figure 4-10: Metrics for link to STA 01-2f.


H2020-ICT-2014-2 671551 Page 46 of 49 14. Aug. 2018

5 Summary and conclusions

WP5 is the 5G-XHaul work package responsible for the experimentation and demonstration activities of the project aiming to prove the feasibility of the 5G-XHaul solution. More specifically, WP5 focused on the demon-stration of the main architectural functionalities and features described in detail in the technical WPs i.e. through a set of planned validation experiments that took place in the project testbed facilities.

In this context, D5.3 reports on the results from the final demonstrator of 5G-XHaul deployed in the state-of-the art “City of Bristol” transport network infrastructure (BIO). All wireless and optical data plane technologies and most of the control plane technologies have been integrated and deployed. We leveraged the smart city testbed for the assessment of the end-to-end 5G-XHaul proposed solution and its performance evaluation in a realistic 5G environment. The results from the validation and the measurements collected proved the validity of the proposed solution, providing a flexible converged end-to-end optical and wireless transport infrastruc-ture supporting both BH and FH services.


H2020-ICT-2014-2 671551 Page 47 of 49 14. Aug. 2018

6 References

[1] 5G-XHaul deliverable D2.2, “System Architecture Design”, July 2016.

[2] N. Michailow et al., "Generalized Frequency Division Multiplexing for 5th Generation Cellular Networks," in IEEE Transactions on Communications, vol. 62, no. 9, pp. 3045-3061, Sept. 2014.

[3] 5G-XHaul deliverable D3.2, “Design and evaluation of scalable control plane, and of mobility aware capabilities and spatio-temporal demand prediction models”

[4] 5G-XHaul deliverable D4.9, “Initial report on mm-Wave circuits and systems for high rate point to mul-tipoint links”, December 2016.

[5] 5G-XHaul deliverable D4.10, “Final report on mm-Wave circuits and systems for high rate point to mul-tipoint links”, March 2018.

[6] 5G-XHaul deliverable D4.12, “Advanced Antenna System with Integrated L1 Processing”, December 2016.

[7] 5G-XHaul deliverable D5.2, “Evaluation of wireless-optical converged functionalities at UNIVBRIS testbed”, June 2016.

[8] VideoLAN Organization (VLC): https://www.videolan.org

[9] Prometheus: https://prometheus.io/

[10] 5G-XHaul YouTube channel: https://www.youtube.com/channel/UCT-uBXp-yM8bO3bhNOsHUzg

[11] ITU-T Std. G.698.4, “Multichannel bi-directional DWDM applications with port agnostic single-channel optical interfaces”.

[12] 5G-XHaul deliverable D4.2, “Optical Fronthauling Solution”, November 2017.

[13] 5G-XHaul deliverable D4.13, “Synchronization and Localization for Cooperative Communications”, April 2017.

[14] https://www.nec.com/en/global/prod/nw/pasolink/products/ipasolinkSX.html

https://www.videolan.org/

https://prometheus.io/


H2020-ICT-2014-2 671551 Page 48 of 49 14. Aug. 2018

7 Acronyms

Acronym Description

5G Fifth Generation Networks

AAS Advanced Antenna System

ADC Analogue-to-Digital Converter

AFE Analogue Front-End

AWG Arrayed Waveguide Grating

BB Baseband

BBU Baseband Unit

BER Bit Error Rate

BH Backhaul

BIO Bristol is Open

BS Base Station

CDR clock and data recovery

CPRI Common Public Radio Interface

C-RAN Cloud Radio Access Network (aka Cloud-RAN )

CO Central Office

DAC Digital-to-Analogue Converter

DCF Dispersion Compensating Fibre

EDFA Erbium Doped Fibre Amplifier

EVM Error Vector Magnitude

FH Fronthaul

FLLR Fast Local Link Reroute

FPGA Field Programmable Gate Array

GFDM Generalized Frequency Division Multiplexing

GUI Graphical User Interface

IQ In-phase Quadrature-phase

KPI Key Performance Indicator

LLDP Link Layer Discovery Protocol

LTE Long Term Evolution

LUT Lookup Table

LCV Line Code Violations

MAC Medium Access Control

MCU Main Control Unit

MEMS micro-electromechanical system

MIMO Multiple-Input Multiple-Output


H2020-ICT-2014-2 671551 Page 49 of 49 14. Aug. 2018

mmWave Millimetre Wave

NGFI Next Generation Fronthaul Interface

OBSAI Open Base Station Architecture Initiative

OLT Optical Line Terminal

ONU Optical Network Unit

P2P Point-to-Point

PHY Physical layer

PoE Power over Ethernet

PON Passive Optical Network

PT Pilot Tone

PRBS pseudorandom binary sequence

RFIC Radio Frequency Integrated Circuits

RRH Remote Radio Head

RN Remote Node

ROSA Receiver Optical Sub-Assembly

RTT Round-Trip Time

RU Radio Unit

Rx Receiver

SDN Software Defined Networking

SMF Single Mode Fibre

SSMF Standard Single Mode Fibre

TDM Time Division Multiplexed

TSON Time-Shared Optical Network

Tx Transmitter

UE User Equipment

VHDL VHSIC Hardware Description Language

WDM Wavelength Division Multiplexing

WP Work Package

Documents

D5.3 Demonstration and Evaluation of the 5G-XHaul ...5G-XHaul Deliverable H2020-ICT-2014-2 671551 Page 8 of 49 14. Aug. 2018 Executive Summary To address the high bandwidth and low