QorIQ OpenDataPlane v16

QorIQ OpenDataPlane v16.08

NXP Semiconductors Document Number: QORIQODP1608

Rev. 0, Aug 2016

Contents

Chapter 1 Release Notes................................................................................ 41.1 What's New.......................................................................................................................... 41.2 Licensing.............................................................................................................................. 41.3 How to Obtain this Release..................................................................................................5

Chapter 2 Known Issues/Limitations............................................................ 6

Chapter 3 Unsupported ODP API's................................................................7

Chapter 4 QorIQ OpenDataPlane (ODP) v16.08 User Manual......................84.1 Conventions.........................................................................................................................84.2 Introduction..........................................................................................................................8

4.2.1 Intended audience................................................................................................................ 84.2.2 Definitions and acronyms......................................................................................................84.2.3 Supported platforms............................................................................................................. 9

4.3 Product Description............................................................................................................. 94.3.1 Product diagram....................................................................................................................9

4.4 Build ODP Through Yocto.................................................................................................. 104.4.1 How to install the SDK and ODP tarball..............................................................................104.4.2 Building ODP applications...................................................................................................114.4.3 Building ODP applications only........................................................................................... 11

4.5 Using ODP Applications.....................................................................................................114.5.1 LS1043ARDB Board Preparation and Bring-up...................................................................114.5.2 LS2085ARDB Board Preparation and Bring-up.................................................................. 124.5.3 odp_pktio application.......................................................................................................... 14

4.5.3.1 Overview............................................................................................................................... 144.5.3.2 Test setup..............................................................................................................................154.5.3.3 Running odp_pktio on DUT...................................................................................................154.5.3.4 Test description.....................................................................................................................15

4.5.4 odp_generator application.................................................................................................. 154.5.4.1 Overview............................................................................................................................... 164.5.4.2 Test setup..............................................................................................................................164.5.4.3 Running odp_generator on DUT...........................................................................................16

4.5.5 odp_l3fwd application......................................................................................................... 174.5.5.1 Overview............................................................................................................................... 174.5.5.2 Test setup..............................................................................................................................184.5.5.3 Running odp_l3fwd on DUT..................................................................................................18

4.5.6 ODP ipsec sample application (odp_ipsec, odp_ipsec_proto)............................................184.5.6.1 Overview............................................................................................................................... 184.5.6.2 Test setup..............................................................................................................................194.5.6.3 Running ODP ipsec applications on DUT.............................................................................20

4.5.7 odp_classifier application....................................................................................................214.5.7.1 Overview................................................................................................................................214.5.7.2 Test setup.............................................................................................................................. 224.5.7.3 Running odp_classifier on DUT.............................................................................................22

4.5.8 odp_timer application..........................................................................................................23

Contents

QorIQ OpenDataPlane v16.08 Documentation, Rev. 0, Aug 20162 NXP Semiconductors

4.5.8.1 Overview-odp_timer..............................................................................................................234.5.8.2 Test setup-odp_timer............................................................................................................ 234.5.8.3 Running odp_timer on DUT.................................................................................................. 24

4.5.9 odp_lpmfwd application...................................................................................................... 244.5.9.1 Overview...............................................................................................................................244.5.9.2 Running odp_lpmfwd on DUT...............................................................................................284.5.9.3 Test description.....................................................................................................................28

4.6 ODP over Virtual Machine (LS2080A)...............................................................................284.6.1 Building Images for Virtual Machine................................................................................... 294.6.2 Running Virtual Machine.................................................................................................... 304.6.3 Running ODP Application in VM.........................................................................................33

4.7 Troubleshooting................................................................................................................. 34

Contents

QorIQ OpenDataPlane v16.08 Documentation, Rev. 0, Aug 2016NXP Semiconductors 3

Chapter 1Release Notes

NXP Digital Networking is pleased to announce the release of QorIQ OpenDataPlane v16.08 supporting Layerscape DPAAbased QorIQ family of processors. This release can be used for both LS2080ARDB and LS1043ARDB boards. See list ofnew features and changes in OpenDataPlane v16.08 in the What's New on page 4 section below. See downloadinstructions in How to Obtain this Release on page 5.

1.1 What's New

Highlights

• Supports ODP API v1.10.1

• Following ODP applications have been verified:

• odp_pktio

• odp_generator

• odp_l3fwd

• odp_ipsec

• odp_ipsec_proto ( NXP extenssion API for Crypto acceleration )

• odp_classifier

• odp_timer

• odp_lpmipfwd (Longest Prefix Match IP Forward)

• ODP over VM support ( odp_pktio and odp_l3fwd only )

Processor Support

• LS2080A

• LS1043A

Yocto and Toolchain

Same as of SDK2.0 release

U‐Boot


Linux Kernel Core and Virtualization


1.2 LicensingThe majority of the software included in the QorIQ SDK is licensed under a form of open source license (e.g. GPL, BSD).Some software is licensed under the Freescale End User License.

Release Notes

What's New


1.3 How to Obtain this ReleaseQorIQ OpenDataPlane v16.08 can be downloaded as a combination of tar ball and SDK2.0 ISO files (see details below).SDK documentation is available on the NXP web site (see link below).

The SDK consists of the following ISOs and tarball:

• Source ISO ‐ contains SDK source code, Yocto recipes, and SDK user documentation.

• Yocto Cache Binary ISOs – contains pre ‐ built Yocto cache binary files that will accelerate building the SDK.

• Images – contains the flash and hard drive images for each supported platform.

• tarball - QorIQ OpenDataPlane v16.08.tgz contains ODP support specific changes on top of SDK2.0 release.

QorIQ SDK ISOs can be downloaded from this location:

http://www.nxp.com/products/software‐and‐tools/run‐time‐software/linux‐sdk/linux‐sdk‐for‐qoriq‐processors:SDKLINUX

QorIQ SDK User documentation can be located online here:

Knowledge Center

Contact/Support:

nxp.com/support

Release Notes

How to Obtain this Release


http://www.nxp.com/products/software-and-tools/run-time-software/linux-sdk/linux-sdk-for-qoriq-processors:SDKLINUX

https://freescale.sdlproducts.com/LiveContent/web/pub.xql?%0Ac=t&action=home&pub=QorIQ_SDK&lang=en%E2%80%90US

http://www.nxp.com/support/sales-and-support:SUPPORTHOME

Chapter 2Known Issues/Limitations

• Ordered queue is not supported

• Route option with subnet is not supported in odp_ipsec and odp_ipsec_proto application

• Pktio in burst/queue mode will work on only 4 1G ports for LS1043A

• Segmented buffers are supported only for forwarding applications like pktio, l3fwd

• ODP application using schedule mode cannot be restarted.

• CRYPTO – DPAA hardware is an asynchronous hardware. Hence only asynchronous mode is supported for cryptooperations

• CRYPTO – Only ESP sessions are supported (crypto algorithm supported : 3DES, AES, authentication algorithm :SHA1, MD5).

• CRYPTO – New buffer/Out of place crypto mode is not supported

• CRYPTO – Sessions with requirement of authentication before cipher are not supported

• Classifier – Only 8 types of pmr counts are supported in LS2080A. There is no support for pmr cascading

• Application restart does not work on LS1043A and will require the system re-start

• odp_timer_test work with only 4 cores on LS2080A. ( Limited by Resources allocated for ODP applications)

• odp_generator in udp mode work with non-zero interval ( -i ) option

• odp_classifier: Fail to classify packets with L2 and L3 priority on LS1043A

• odp_classifier: Pool packet counts are not proper on LS1043A

• Traffic Management is not supported

Known Issues/Limitations


Chapter 3Unsupported ODP API's

Following ODP API's are not supported in this release

odp_packet_add_dataodp_packet_rem_dataodp_packet_extend_headodp_packet_trunc_headodp_packet_extend_tailodp_packet_trunc_tailodp_packet_copy_from_pktodp_packet_concatodp_packet_splitodp_packet_copy_partodp_packet_alignodp_packet_move_dataodp_packet_copy_dataodp_packet_has_tsodp_packet_has_ts_clrodp_packet_ts_setodp_packet_tsodp_cos_drop_setodp_cls_cos_pool_setodp_pktio_skip_setodp_cls_cos_poolodp_cos_drop

Unsupported ODP API's


Chapter 4QorIQ OpenDataPlane (ODP) v16.08 User Manual

4.1 ConventionsThis document uses the following conventions:

courier Is used to identify commands, explicit command parameters, code examples, expressions, datatypes, and directives.

4.2 IntroductionThis document provides information about ODP Sample Applications built on ODP API 1.10.1 implementation. User canexperience the main functionalities of OpenDataPlane (ODP) with these applications and can also learn how to use the ODPAPI from source code of these applications.

The document explores the following target applications:

• ODP generator sample application

• ODP pktio sample application

• ODP ipsec transport and tunnel sample applications

• ODP packet classify sample application

• ODP timer sample application

• ODP L3 Forwarding sample application

• ODP LPM IP Forwarding sample application

More information about ODP introduction, architecture and user guide are available on Linaro web site:

OpenDataPlane API Reference Manual for linux-generic

OpenDataPlane (ODP) User Guide

OpenDataPlane (ODP) Implementers Guide

4.2.1 Intended audienceThis document is intended for software developers and architects who want to develop ODP applications on QorIQ basedplatforms. The document assumes that users are familiar with Linux-based software development, and also with the ODPconcepts and APIs.

4.2.2 Definitions and acronyms

SDK Software Development Kit

RDB Reference Design Board

Table continues on the next page...

QorIQ OpenDataPlane (ODP) v16.08 User Manual

Conventions


http://docs.opendataplane.org/snapshots/odp-publish/generic/dox_html/latest/next/linux-generic-doxygen-html/index.html

http://www.opendataplane.org/api-documentation/users-guide/

http://www.opendataplane.org/api-documentation/implementers-guide/

Table continued from the previous page...

DUT Device Under Test

FMan Frame Manager

PBL Pre Boot Loader

RCW Reset Configuration Word

UDP User Datagram Protocol

SIP DIP Source Internet Protocol and Destination Internal Protocol

4.2.3 Supported platformsCurrent release is supported on the following QorIQ processing platforms:

• LS2080A

• LS1043A

4.3 Product Description

4.3.1 Product diagramThe figure below identifies how ODP application runs in a typical network processor software architecture.


Product Description


ODP Applications ODP Applications

PKTIO, Queue IO, Crypto, Classifier, Runtime services

Extended API

Resource-mgmt, discovery, Eth-config

Crypto-PACAIOP Mgmt & Comm

NW-services – KNI, L2. L3

NW Services*

Run-Time ServicesIO & Acceleration

Queue, Scheduler

ClassifierOffload* Crypto PKTIO

Resource Mgmt*

Timers Memory

SyncBuffers

Arch/ ARM

KNI

Routing ARP

VFIO

SEC

PME

AIOPDPAA 1

DPAA 2

LS-B

us

Res

-Mgr

Pla

tform

spe

cific

Fram

ewor

kA

PI L

ayer

App

licat

ions

ODP API

Figure 1. ODP on QorIQ

4.4 Build ODP Through Yocto

4.4.1 How to install the SDK and ODP tarball1. Download and Install the following SDK V2.0 ISOs:

QorIQ Linux SDK v2.0 Source.isoQorIQ Linux SDK v2.0 AARCH64 Yocto Cache.iso

For detailed information on ISO installation, see “Essential Build Instructions” under “Getting

Started with Yocto Project” within the QorIQ SDK documentation.

NOTE


Build ODP Through Yocto


2. Download and install the ODP tarball (QorIQ OpenDataPlane v16.08.tgz) to the ISO installation folder:

$ tar -zxf QorIQ-OpenDataPlane-v16.08.tgz$ cd QorIQ-OpenDataPlane-v16.08$ ./install

4.4.2 Building ODP applications1. $ cd <path>/QorIQ-SDK-V2.0-20160527-yocto

2. $ . ./fsl-setup-env -m <platform-name> (Configure compilation environment for the requested board.)

3. bitbake fsl-image-kernelitb

<platform-name>: ls2080ardb for LS2080A and ls1043ardb for LS1043A.

After successful compilation, images will be generated at tmp/deploy/images/<board-name>/ folder. This compiles andinstalls the ODP applications in the rootfs.

4.4.3 Building ODP applications onlyODP applications can also be compiled alone using Yocto once you have ran "bitbake fsl-image-kernelitb". The steps areas follows:

1. cd build_<board-name>/

2. source SOURCE_THIS

3. Clean ODP: bitbake -f -c cleansstate odp

4. Make ODP: bitbake -f -c compile odp

The above step will only compile the odp applications. For adding them to rootfs and creating a

Kernelitb, use the command given below.

NOTE


4.5 Using ODP ApplicationsExecute the below commands to allocate resources for ODP applications.

4.5.1 LS1043ARDB Board Preparation and Bring-up

Board Bring-up

The QorIQ SDK 2.0 documentation contains Software Deployment Guides for all the supported reference boards. Refer tothe LS1043A Software Deployment Guide for instructions on how to deploy U-Boot, Linux kernel and root file system to theboard.

Interface Naming Conventions

Throughout the ODP User Manual, ODP application commands are listed. These commands use intf-1, intf-2, …, intf-x asgeneric interface names. Use fm0-mac<X> in placeof intf-<x> when running ODP commands on LS1043A.


Using ODP Applications


Example command listed in user manual:

$ ./odp_pktio -c 2 -m 2 -i <intf-1>,<intf-2>

Corresponding command to run on LS1043A:

$ ./odp_pktio -c 2 -m 2 -i fm0-mac1,fm0-mac2

The table below provides the mapping between the generic interface names and the names to be used in the ODP commandsfor LS1043A.

Generic Interface Names LS1043A Interface Names LS1043ARDB Ports

intf-1 fm0-mac1 QSGMII0


intf-3 fm0-mac3 RGMII0

intf-4 fm0-mac4 RGMII1



Resource Allocation Commands

Execute the following commands to allocate the resources required to run the ODP applications.

$ export DEF_CFG_PATH=/usr/odp/pcd/odp_perf_config_ls1043.xml$ export DEF_PCD_PATH=/usr/odp/pcd/odp_pktio_pcd.xml

4.5.2 LS2085ARDB Board Preparation and Bring-up

Board Bring-up

The QorIQ SDK 2.0 documentation contains Software Deployment Guides for all the supported reference boards. Refer tothe LS2085ARDB Software Deployment Guide for instructions on how to deploy U-Boot, Linux kernel and ManagementComplex and root file system to the board. LS2085ARDB supports both LS2085A (non-production SoC) and LS2080A(supports ODP).

Interface Naming Conventions

Throughout the ODP User Manual, ODP application commands are listed. These commands use intf-1, intf-2, …, intf-x asgeneric interface names. Use dpni.<x> in placeof intf-<x> when running ODP commands on LS2080A.

Example command listed in user manual:


Corresponding command to run on LS2080A/LS1088A:

$ ./odp_pktio -c 2 -m 2 -i dpni.1,dpni.2




The table below provides the mapping between the generic interface names and the names to be used in the ODP commandsfor LS2080A:.

Generic Interface Names LS2085ARDB Ports LS2085ARDB Interface Names

intf-1 eth4 dpmac.1(dpni.x*)








(*) denotes dpni.x to dpmac.<index> mapping is auto-generated by the resource allocation

“dynamic_dpl.sh” script. See the steps below for details.

NOTE

Resource Allocation Commands

Execute the following commands to allocate the resources required to run the ODP applications.

$ cd /usr/odp/scripts$ source ./dynamic_dpl.sh dpmac.1 dpmac.2 (Reference command)

In order to use multiple interfaces, user can run the following command:

source ./dynamic_dpl.sh dpmac.1 dpmac.2 …… dpmac.8

NOTE

The output log of the reference command is shown below:

##################### Container dprc.2 is created ####################

Container dprc.2 have following resources :=>

* 4 DPBP * 5 DPCON * 1 DPSECI * 3 DPNI * 10 DPIO * 10 DPCI

######################### Configured Interfaces #########################

Interface Name Endpoint Mac Address




============== ======== ==================dpni.1 dpmac.1 00:00:00:00:0:1dpni.2 dpmac.2 00:00:00:00:0:2dpni.3 dpni.3 00:00:00:11:11:11

Note: dynamic_dpl.sh script allocates default number of resources like buffer pools ( DPBP ), I/O thread context ( DPIO ),S/W queues ( DPCI ) etc. as shown in output above.

To increase number of resources requred by an application below environment variables shall be exported to overwrite thedefault configuration before executing dynamic_dpl.sh script.

DPIO_COUNT : Defines the number of I/O contexts that are available to the application. Current implementation allocates one DPIO for each timer thread spawned internally; in addition to allocating a DPIO for a worker thread. Example - If an application initiates 8 worker threads which internally may spawn 8 timer threads; DPIO count for that case shall be at least 16. If application is not spawning timer threads then the DPIO count would be equal to the number of worker threads.

Usage : export DPIO_COUNT=<n> eg. "export DPIO_COUNT=10"

DPBP_COUNT : Defines number of buffer pools that can be allocated in an application.

Usage : export DPBP_COUNT=<n> e.g "export DPBP_COUNT=4"

DPCI_COUNT : Defines the number of queues that can be created in an application.

Usage : export DPCI_COUNT=<n> e.g "export DPCI_COUNT=10"

DPCONC_COUNT : Defines number of scheduler groups that can be created in an application. 3 DPIO's are created by default for all applications to create Control, Worker and 'All' scheduler group.

Usage : export DPCONC_COUNT=<n> eg. "export DPCONC_COUNT=6"

4.5.3 odp_pktio application

4.5.3.1 OverviewThe odp_pktio application is a sample packet input/output application which receives packets from external traffic source andreflects back the packets after swapping the IP addresses and MAC addresses.

The application will be configured using its available command line options:

Usage: %s OPTIONS E.g. %s -i eth1,eth2,eth3 -m 0 OpenDataPlane example application. Mandatory OPTIONS: -i, --interface Eth interfaces (comma-separated, no spaces) Optional OPTIONS -c, --count <number> CPU count. -m, --mode 0: Receive and send directly (no queues) 1: Receive and send via queues. 2: Receive via scheduler, send via queues.




-h, --help Display help and exit. environment variables: ODP_PKTIO_DISABLE_SOCKET_MMAP ODP_PKTIO_DISABLE_SOCKET_MMSG ODP_PKTIO_DISABLE_SOCKET_BASIC can be used to advanced pkt I/O selection for linux-generic

4.5.3.2 Test setup

Interface#1 brd_if#1

brd_if#2Interface#2

Spirent TestCenter

DUT

odp_pktio

Figure 2. Test setup (odp-pktio)

4.5.3.3 Running odp_pktio on DUTExecute below commands on DUT to run odp_pktio:


Interfaces intf-1 can be dpni.<index> or fm<X>-mac<Y>. See LS1043ARDB Board Preparation and

Bring-up on page 11 and LS2085ARDB Board Preparation and Bring-up on page 12 for interface

details as per board in use.

NOTE

For additional details on options, see the command line options mentioned above.

4.5.3.4 Test descriptionThe application will receive traffic from the traffic generator – and this will be seen in the application console, on one interfaceor on multiple interfaces where the application is running. Once a packet is received, its destination Ethernet addresses willbe swapped (and so the IP addresses) and the packet will be sent back on the port it came from, to the packet originator.

As a reference, packet scheduler mode is chosen.

A number of 100000 packets will be sent on one port, from the Spirent test center/traffic generator.

Packet format

Eth source Eth destination Ip source Ip destination Proto

Any Any Any Any Ipv4/UDP

The application will send back 100000 packets to the Spirent test center, each packet having its Ethernet and IP addressesswapped.

4.5.4 odp_generator application




4.5.4.1 OverviewThe odp_generator application is a sample application which either receives packets from external traffic source or generatesthe traffic and sends it to the network on the basis of mode of operation.


Usage: %s OPTIONS E.g. %s -I eth1 -r OpenDataPlane example application. Work mode: 1.send udp packets 2.receive udp packets 3.works like ping Mandatory OPTIONS: -I, --interface Eth interfaces (comma-separated, no spaces) -a, --srcmac src mac address -b, --dstmac dst mac address -c, --srcip src ip address -d, --dstip dst ip address -s, --packetsize payload length of the packets -m, --mode work mode: send udp(u), receive(r), send icmp(p) -n, --count the number of packets to be send -t, --timeout only for ping mode, wait ICMP reply timeout seconds -i, --interval wait interval ms between sending each packet default is 1000ms. 0 for flood mode Optional OPTIONS -h, --help Display help and exit. environment variables: ODP_PKTIO_DISABLE_SOCKET_MMAP ODP_PKTIO_DISABLE_SOCKET_MMSG ODP_PKTIO_DISABLE_SOCKET_BASIC can be used to advanced pkt I/O selection for linux-generic

4.5.4.2 Test setupFigure 3. Test setup (odp_generator)


brd_if#2Interface#2

Spirent TestCenter

DUT

odp_generator

4.5.4.3 Running odp_generator on DUTExecute below commands on DUT to run odp generator application:




Receive mode:

$ ./odp_generator -I <intf-1> -m r -w 1

UDP mode:

$ ./odp_generator -I <intf-1>--srcmac 00:00:00:00:00:0X --dstmac00:00:10:00:94:02 --srcip 192.85.2.2 --dstip 192.85.1.2 --packetsize 64 -m u

Ping mode:

$ ./odp_generator -I <intf-1> --srcmac 00:00:00:00:00:0X --dstmac00:10:94:00:00:02 --srcip 192.85.2.2 --dstip 192.85.1.2 --packetsize 64 -mp




NOTE


4.5.5 odp_l3fwd application

4.5.5.1 OverviewThe odp_l3fwd application is a sample application which demonstrates the TCP/IP layer 3 forwarding based on configuredroute.

The application uses two matching tables, one for route entries and second for flow cache entries. Once a packet is received,lookup is done and based on lookup result the packet is forwarded onto a particular interface.

Following are detailed information about the application:

• Application configures route table entries and maximum number of flows to be catered at init time.

• Flows are maintained in a hash table using 2-tuple/5-tuple (option at compile time) hash calculation.


Usage: %s OPTIONS Mandatory OPTIONS: -i, --interface Eth interfaces (comma-separated, no spaces) -r, --route SubNet:Intf:NextHopMAC o SubNet: IPaddress with mask bits i.e. aa:bb:cc:dd/maskbitso Intf: Interface name i.e. dpni-0o NextHopMAC: Destination Mac Address for next hop Optional OPTIONS -c, --count <number> CPU count. -m, --mode Mode of application0: Burst send & receive packets (no queues)1: Send & receive packets through ODP Schedular.Default: Packet burst mode. -f, --flows <number> Maximum number of Route cache flows.Default: 100000. -h, --help Display help and exit




4.5.5.2 Test setup

Interface#1DUT

odp_l3fwd

brd_if#1


Spirent TestCenter

Figure 4. Test setup (odp_l3fwd)

4.5.5.3 Running odp_l3fwd on DUTExecute the below commands on DUT to run odp layer 3 forwarding application:

$./odp_l3fwd -i <intf-1>,<intf-2> -r192.85.0.0/16:<intf-1>:08.00.27.76.B5.01 -r 192.86.0.0/16:<intf-2>:08.00.27.F5.8B.02 -m 0 -c 4

Interfaces intf-1 and intf-2 can be dpni.<index> or fm<X>-mac<Y>. See LS1043ARDB Board

Preparation and Bring-up on page 11and LS2085ARDB Board Preparation and Bring-up on page

12 for interface details as per board in use.

NOTE


4.5.6 ODP ipsec sample application (odp_ipsec,odp_ipsec_proto)

4.5.6.1 OverviewThe “odp_ipsec” application functions as a simple L3 IPv4 router with support for IPsec 3DES cipher in both transmit andreceive directions. Note that both IPsec "transport" and “tunnel” modes are supported. These applications authenticate andencrypts/decrypts traffic from external source and forwards the encrypted/decrypted packets.

In addition, the odp_ipsec_proto application demonstrates IPSec ESP protocol offload advantages over standard algorithm-oriented crypto API.

The applications will be configured using its available command line options:

Usage: %s OPTIONS E.g. %s -i eth1,eth2,eth3 -m 0 OpenDataPlane example application. Mandatory OPTIONS: -i, --interface Eth interfaces (comma-separated, no spaces) -m, --mode 0: SYNC 1: ASYNC_IN_PLACE 2: ASYNC_NEW_BUFFER Default: 0: SYNC api mode Routing / IPSec OPTIONS: -r, --route SubNet:Intf:NextHopMAC -p, --policy SrcSubNet:DstSubNet:(in|out):(ah|esp|both)




-e, --esp SrcIP:DstIP:(3des|aes|null):SPI:Key192 -a, --ah SrcIP:DstIP:(md5|sha1|null):SPI:Key128 Where: NextHopMAC is raw hex/dot notation, i.e. 03.BA.44.9A.CE.02 IP is decimal/dot notation, i.e. 192.168.1.1 SubNet is decimal/dot/slash notation, i.e 192.168.0.0/16 SPI is raw hex, 32 bits KeyXXX is raw hex, XXX bits long Examples: -r 192.168.222.0/24:p8p1:08.00.27.F5.8B.DB -p 192.168.111.0/24:192.168.222.0/24:out:esp -e 192.168.111.2:192.168.222.2:3des:201:656c8523255ccc23a66c1917aa0cf30991fce83532a4b224 -a 192.168.111.2:192.168.222.2:sha1:201:2122232425262728292a2b2c2d2e2f3031323334

4.5.6.2 Test setupThis test uses the hardware setup described in below figures:

Host1 DUT

brd_if#0

192.168.11

brd_if#1

192.168.222.1

Host2

ETH0

192.168.222.2

Crypto SubnetClear Subnet

ETH0

192.168.111.2

Figure 5. Transport mode setup

DUT DUT

brd1_if#0

192.168.2.1

brd1_if#1

192.168.160.1

brd2_if#0

192.168.160.2

brd2_if#1

192.168.3.1

Crypto Subnet

Figure 6. Tunnel mode setup




4.5.6.3 Running ODP ipsec applications on DUT

Transport mode

Execute below commands on DUT for odp_ipsec application in transport mode.

$ ./odp_ipsec -i brd_if#0, brd_if#1 -c <Num of cores> -m 1 -r 192.168.111.2/32: brd_if#0:00.11.00.00.00.01 -r 192.168.222.2/32: brd_if#1:00.22.00.00.00.02 -p 192.168.111.0/24:192.168.222.0/24:out:esp -e192.168.111.2:192.168.222.2:3des:201:656c8523255ccc23a66c1917aa0cf30991fce83532a4b224-p 192.168.222.0/24:192.168.111.0/24:in:esp –e192.168.222.2:192.168.111.2:3des:301:c966199f24d095f3990a320d749056401e82b26570320292

Execute below commands on Host1:

• ifconfig eth0 192.168.111.2 up

• ifconfig eth0 hw ether 00:11:00:00:00:01

• ip route add 192.168.222.0/24 via 192.168.111.1

• arp -s 192.168.111.1 00:00:00:00:00:01

Execute below commands on Host2 for odp_ipsec application in transport mode

• ifconfig eth0 192.168.222.2 up

• ifconfig eth0 hw ether 00:22:00:00:00:02

• ip route add 192.168.111.0/24 via 192.168.222.1

• arp -s 192.168.222.1 00:00:00:00:00:02

Host2 has the following setkey configuration file applied. Use reference setkey script as below:

#!/usr/sbin/setkey -f # Flush the SAD and SPD flush; spdflush;

add 192.168.111.2 192.168.222.2 esp 0x201 -E 3des-cbc 0x656c8523255ccc23a66c1917aa0cf30991fce83532a4b224;

add 192.168.222.2 192.168.111.2 esp 0x301 -E 3des-cbc 0xc966199f24d095f3990a320d749056401e82b26570320292;

spdadd 192.168.111.2 192.168.222.2 any -P in ipsec esp/transport//require;

spdadd 192.168.222.2 192.168.111.2 any -P out ipsec esp/transport//require;

Tunnel mode

Execute the below commands for tunnel mode in odp_ipsec_proto application

On Left board(DUT):

$ ./odp_ipsec_proto -i brd1_if#0,brd1_if#1 -r 192.168.2.2/32:brd1_if#0:02.00.c0.a8.3c.01 -r 192.168.3.2/32:brd1_if#1:<next hop (mac addr of brd2_if#0)> -p 192.168.2.0/24:192.168.3.0/24:out:both -e192.168.2.2:192.168.3.2:aes:1:0102030405060708090a0b0c0d0e0f10 -a




192.168.2.2:192.168.3.2:sha1:1:2122232425262728292a2b2c2d2e2f3031323334 -t192.168.2.2:192.168.3.2:192.168.160.1:192.168.160.2 -p 192.168.3.0/24:192.168.2.0/24:in:both -e192.168.3.2:192.168.2.2:aes:2:0102030405060708090a0b0c0d0e0f10 -a192.168.3.2:192.168.2.2:sha1:2:2122232425262728292a2b2c2d2e2f3031323334 -t192.168.3.2:192.168.2.2:192.168.160.2:192.168.160.1 -c 1 -m 1

On Right board(Host2):

$ ./odp_ipsec_proto -i brd2_if#0,brd2_if#1 -r 192.168.3.2/32:brd2_if#1:02.00.c0.a8.3c.02 -r 192.168.2.2/32:brd2_if#0:< next hop (mac addr of brd1_if#1> -p 192.168.2.0/24:192.168.3.0/24:in:both -e192.168.2.2:192.168.3.2:aes:1:0102030405060708090a0b0c0d0e0f10 -a192.168.2.2:192.168.3.2:sha1:1:2122232425262728292a2b2c2d2e2f3031323334 -t192.168.2.2:192.168.3.2:192.168.160.1:192.168.160.2 -p 192.168.3.0/24:192.168.2.0/24:out:both -e192.168.3.2:192.168.2.2:aes:2:0102030405060708090a0b0c0d0e0f10 -a192.168.3.2:192.168.2.2:sha1:2:2122232425262728292a2b2c2d2e2f3031323334 -t192.168.3.2:192.168.2.2:192.168.160.2:192.168.160.1 -c 1 -m 1

For opd_ipsec same command can be used, user just need to modify the application name on both left and right board commands.

1. For LS2085ARDB:

brd1_if#0 dpni.<DPNI_INDEX_1>brd2_if#0 dpni.<DPNI_INDEX 2>brd2_if#1 dpni.<DPNI_INDEX_1>brd1_if#1 dpni.<DPNI_INDEX_2>

2. For LS1043ARDB:

brd1_if#0 fm0-mac3brd2_if#0 fm0-mac4brd2_if#1 fm0-mac3brd1_if#1 fm0-mac4


4.5.7 odp_classifier application

4.5.7.1 Overviewodp_classifier is a sample application to demonstrate classification among the queues based on configured packet matchingrules. Currently implementation demonstrates classification based on following parameters:

1. IP Source address

2. IP QoS

3. VLAN Priority

Functional behavior of the application:

• Application configures multiple packet matching rules specifying the queue as a target place.

• Traffic, matching with the configured rules, will be distributed among the target queues.

• Remaining traffic will be forwarded to default queue.

Application is supported for two modes as given below:

• Drop mode: All the received frames are dropped at application.




• Reply mode: Received frame are reflected back onto the same interface after swapping its Ethernet SMAC & DMAC andSIP & DIP addresses.


Usage: %s OPTIONS Mandatory OPTIONS: -i, --interface Eth interfaces (comma-separated, no spaces) -p, --policy <odp_pmr_term_e>:<value>:<mask bits>:<queue name>odp_pmr_term_e: Type of packet matching rulevalue: Value of rule to be matchedmask bits: Masking bits for matching rulesqueue name: Target Queue where packet is to be received -l, --l2_policy <VLAN Priority Value>:<queue name>VLAN Priority Value: L2 VLAN priority value:Queue Name: Target Queue where packet is to be received -q, --l3_policy <L3 QoS Value>:<queue name>L3 QoS Value: L3 QoS or IP TOS (One byte) valueQueue Name: Target Queue where packet is to be received Optional OPTIONS -c, --count <number> CPU count. -m, --mode Mode of application0: Packet Drop mode. Received packets will be dropped1: Packet Reply mode. Received packets will be sent backDefault: Packet Drop mode. -t, --timeout Time to run the classifier1: Time for which the classifier will be run in seconds0: Runs in infinite loopDefault: Runs in infinite loop. -a, --l3_policy_precedence Flag to flip precedence between L2 and L3 Policy.1: L3 policy rule will take precedence over L2 Policy rules0: L2 policy rule will take precedence over L3 policy rulesDefault: 0 -h, --help Display help and exit

4.5.7.2 Test setup


Spirent TestCenter

DUT

odp_classifier

Figure 7. Test setup (odp_classifier)

4.5.7.3 Running odp_classifier on DUTExecute below commands on DUT to odp classifier application:

Reply mode:$./odp_classifier –i <intf-1> -p ODP_PMR_SIP_ADDR:192.85.1.1:FFFFFFFF:queue1 -l 1:queue2 -q 40:queue3 -m 1Drop Mode:




$./odp_classifier –i <intf-1> -p ODP_PMR_SIP_ADDR:192.85.1.1:FFFFFFFF:queue1 -l 1:queue2 -q 40:queue3 -m 0

Interface intf-1 can be dpni.<index> or fm<X>-mac<Y>. See LS1043ARDB Board Preparation and



NOTE

For LS2080ARDB, DPBP objects are required to be increaesed to 16 for configuring multiple rules

for classifciation. Refer section LS2085ARDB Board Preparation and Bring-up on page 12 to

increase object count in resource container.

NOTE


4.5.8 odp_timer application

4.5.8.1 Overview-odp_timerodp_timer is a sample application that will enqueue an event after a certain time period, when the timer expires and dequeuesthat event and validates it.

The application will create number of threads, given by –c option and allocate a timer for each thread. Each timer will beadded by the application with ticks given by –p option.

Feasibility of ticks will be checked by min and max values given by options –m and –x.

When timer expires, a timeout buffer will be enqueued into a queue that will be received by application using odp_schedule()API. After that, buffer will be validated and timer will reset if more timeouts remain given by –t option.


Usage: %s OPTIONS -c, --count <number> CPU count); -r, --resolution <us> timeout resolution in usec -m, --min <us> minimum timeout in usec -x, --max <us> maximum timeout in usec -p, --period <us> timeout period in usec -t, --timeouts <count> timeout repeat count -h, --help this help

4.5.8.2 Test setup-odp_timer

DUT

odp_timer_test

Figure 8. Test setup (odp_timer)




4.5.8.3 Running odp_timer on DUTExecute below commands on DUT to run the odp timer application

$./odp_timer_test –c 2 –t 2


4.5.9 odp_lpmfwd application

4.5.9.1 Overviewodp_lpmfwd application demonstrates the IPv4 packets forwarding based upon Longest Prefix Match methodology. The LPMbased IPv4 forwarding application is a multi-threaded application that routes IPv4 packets from one Ethernet interface toanother. The packets that reach the LPM application can be forwarded to destination IP address using LPM route algorithm.The LPM routing algorithm uses the prefix for destination IP address to do the route look up.

Longest Prefix Match Algorithm:

Instead of using traditional Radix-Trie algorithm, here we choose to use one simpler LPM algorithm. It uses 5 level tables:First level table – 65536 entries array, indexed by the top 16 bits of IP address. Second level table – 32 entries array, indexedby bit12~15 of IP address. Third level table – 32 entries array, indexed by bit8~11 of IP address. Fourth level table – 32 entriesarray, indexed by bit4~7 of IP address. Fifth level table – 32 entries array, indexed by bit0~3 of IP address. The 2nd level to5th level tables are only created when its first level table entry has valid value. See below figure:

At init time, only the first level (i.e. top16 bits) array is created which contains 65536 null entries. While adding route entriesto the FIB table, the 2nd level to 5th level arrays will be created accordingly. This is a typical ASIC design algorithm of LPMwhich is fast and simple to search while costs far more memory. The worst case is to index and compare 5 times whensearching an IP address, but it’s still fast enough.

Examples:

1) Add one Class A route 10.0.0.0/8 to the route table. (Gateway is 1.1.1.1, destination port is 1) The first 16bit of 10.0.0.0(0x0a000000) is 0x0a00 (2560). And the mask is 8 bit which is smaller than the 1st level bit-length (16b), so below entries(from 0x0a00 to 0x0aff) will be created in the FIB table:




From entry No.2560 to No.2815 (total 255 entries) are filled with same content (flag, gwIP, dstPort, ptr …). Now if a packetwith DIP of 10.1.1.1 comes in, its first 16 bit value is 0x0a01 (2561). So, the No.2561 entry of the 1st level table will be checked.If it’s a ‘leaf’ node (now it is), then the best-match is found. And the packet will be forwarded to the ‘dstPort’ after replacingthe SMAC and DMAC. And, any DIP of 10.x.x.x will all be forwarded to port 1 with gwIP 1.1.1.1 based on above table.

2) Now, a new route 10.1.1.0/24 is added to the FIB table. The table will be like this:

The new route will overwrite the No.2561 (0x0a01) entry with ‘flag’ of ‘nonLeaf’ and a pointer to ‘next-level-table’. Now a 16-entry-block memory will be allocated as the ‘next-level-table’ because the 2nd level is 4bit indexed. And the base address ofthis new block will be set to the ‘nxtLvlTblPtr’ of No.2561. Because the next 4 bit of the new route is 0 (bit16 to bit19 of0x0a010100), so the 1st entry in the 2nd level table is used as another ‘nonLeaf’ entry. While all the other entries in the 2ndlevel table should be filled with the same value of its ‘parent’ route (10.0.0.0/8). The netmask is 24bit which is larger than16+4, so the 3rd level table should also be allocated (16 entries). And the next 4bit of the new route is 1 (bit20 to bit23 of0x0a010100), so the 2nd entry will be used for the new route. And because the netmask (24bit) is no-larger-than 16+4+4, sothis entry will be the ‘Leaf’ entry of this new route (see above figure in green). And the according values (gwIP, dstPort, etc.)will be filled in that entry. Now a frame with DIP of 10.1.1.100 comes. There will have 3 lookups to get the final result:

• Index with first 16bit of DIP, whose value is 0x0a01. ‘Non-leaf’ means to continue the next-level lookup.

• Index with the next 4bit of DIP, whose value is 0. Then ‘Non-leaf’ again.

• Index with the next 4bit of DIP, whose value is 1. Then the ‘leaf’ node is found and the lookup reaches an end.




Now a frame with DIP of 10.1.192.10 comes. You can see it will find the ‘leaf’ node in the 2nd level table and get the route ofnet-address 10.0.0.0/8. And a frame with DIP of 10.1.10.10 will find its ‘leaf’ node in the 3rd level table and also get the routeof net-address 10.0.0.0/8 as we expected. The multi-branch trie algorithm provides a very fast way of route-lookup but arelatively complicated way of route-add/deletion.

Application Usage:

There are two binaries for LPM based IPv4 packets forwarding:

odp_lpmfwd : Main binary which will do forwarding.

odp_lpmfwd_config : Helper binary to configure the routes, interfaces and ARPs.

Both binaries will communicate via Linux message queues.

odp_lpmfwd usage:

./odp_lpmfwd -h

OpenDataPlane LPM forwarding application.

Usage: odp_lpmfwd OPTIONS E.g. odp_lpmfwd -i eth1,eth2 -m 0 -c 1 In the above example, eth1 and eth2 are the interfaces from which pkts will be forwarded depends upon the routes

Mandatory OPTIONS: -i, --interface eth interfaces (comma-separated, no spaces)

Optional OPTIONS -m, --mode 0: Burst send & receive packets (no queues) 1: Send & receive packets through ODP Schedular. Default: Packet burst mode. -c, --count <number> CPU count. -h, --help Display help and exit.

odp_lpmfwd_config usage:

./odp_lpmfwd_config --help

Usage: odp_lpmfwd_config[OPTIONS…] -B, --routeadd=TYPE adding a route -E, --showintf=TYPE show interfaces -F, --intfconf=TYPE change intf config -G, --arpadd=TYPE adding a arp entry -P, --PID=TYPE pid to hook with (Mandatory param) -?, --help Give this help list --usage Give a short usage message -V, --version Print program version

Help for show all enabled interfaces:

./odp_lpmfwd_config –P <pid of odp_lpmfwd> -E –help

Usage: -E [OPTION...] -a, --a=ALL All interfaces -?, --help Give this help list --usage Give a short usage message -V, --version Print program version




Example: ./odp_lpmfwd_config -P 2234 -E -a true

Help for assign IP addresses to interfaces:

./odp_lpmfwd_config –P <pid of odp_lpmfwd> -F –help

Usage: -F [OPTION...] -a, --a=IPADDR IP Address -i, --i=IFNUM If Number -?, --help Give this help list --usage Give a short usage message -V, --version Print program version

Example: ./odp_lpmfwd_config -P 2234 -F -a 192.168.222.1 -i 1Where i is the index of the interface.

e.g.: For following command: ./odp_lpmfwd –i intf-2,intf-1 In odp_lpmfwd_config to assign the IP addresses “i” should be 1 for intf-2 and 2 for intf-1

Help for ARP entry addition:

./odp_lpmfwd_config –P <pid of odp_lpmfwd> -G –help

Usage: -G [OPTION...] -m, --m=MACADDR MAC Address -r, --r=Replace Replace Exiting Entry - true/ false {Default:false} -s, --s=IPADDR IP Address -?, --help Give this help list --usage Give a short usage message -V, --version Print program version

Example: ./odp_lpmfwd_config -P 2234 -G -s 192.168.111.2 -m 00:00:00:00:00:11 -r true

Help for route entry addition:

./odp_lpmfwd_config –P <pid of odp_lpmfwd> -B –help

Usage: -B [OPTION...]

-c, --c=FIBCNT Number of FIB entries -d, --d=DESTIP Destination IP -g, --g=GWIP Gateway IP -n, --n=NETMASK netmask length to be used by LPM -?, --help Give this help list --usage Give a short usage message -V, --version Print program version

Example: ./odp_lpmfwd_config -P 2234 -B -d 192.168.111.0 -g 192.168.222.2 -n 24 -c 256

Mandatory or optional arguments to long options are also mandatory or optionalfor any corresponding short options.




4.5.9.2 Running odp_lpmfwd on DUTExecute below commands on DUT to run odp_lpmfwd:

$ ./odp_lpmfwd –i <intf-1>,<intf-2> &




NOTE

To configure the interfaces, routes and ARPs, run the following commands:

$ ./odp_lpmfwd_config -P $1 -E -a true$./odp_lpmfwd_config -P $1 -F -a 192.168.222.1 -i 1$./odp_lpmfwd_config -P $1 -F -a 192.168.111.1 -i 2$./odp_lpmfwd_config -P $1 -G -s 192.168.222.2 -m 00:00:00:00:00:10 -r true$./odp_lpmfwd_config -P $1 -G -s 192.168.111.2 -m 00:00:00:00:00:11 -r true$./odp_lpmfwd_config -P $1 -B -d 192.168.111.0 -g 192.168.222.2 -n 24 -c 256$./odp_lpmfwd_config -P $1 -B -d 192.168.222.0 -g 192.168.111.2 -n 24 -c 256

Where $1 is PID of the process odp_lpmfwd.

4.5.9.3 Test descriptionWhen a packet is received on one of the input interfaces, it will be routed and forwarded according to the LPM algorithm. Thepacket will be changed - destination Ethernet address will be updated according to received command line options, the TTL(Hop Limit) will be decremented and the checksum will be recalculated. Finally the packet will be transmitted on the outgoinginterface.

Packet format

Stream 1 on interface#1: IPv4 stream, DIP 192.168.222.4

Stream 2 on interface#2: IPv4 stream, DIP 192.168.111.4

4.6 ODP over Virtual Machine (LS2080A)On LS2080A, ODP is supported over Virtual Machine (VM) with DPAA2 Ethernet Interfaces directly assigned to the VirtualMachine.

The following ODP applications are supported:

• odp_pktio

• odp_l3fwd

ODP_on_VM Setup:


ODP over Virtual Machine (LS2080A)


4.6.1 Building Images for Virtual MachineRefer Build ODP Through Yocto on page 10 section for installing SDK ISO for ODP.

Execute the commands below to build images for ODP on VM in <INSTALL_PATH_FOR_ISO>/QorIQ-SDK-V2.0-20160527-yocto:

1. . ./fsl-setup-env -m ls2080ardb

2. echo 'ROOTFS_IMAGE = "fsl-image-virt"' >> conf/local.conf

3. bitbake -c cleansstate odp

4. bitbake -c patch odp

5. Uncomment the below line in file “<INSTALL_PATH_FOR_ISO>/build_ls2080ardb/tmp/work/ls2080ardb-fsl-linux/odp/git-r0/git/platform/linux-dpaa2/Makefile.inc”

a. # DPAA2_CFLAGS += -DODP_USE_PHYS_IOVA


After compiling successfully, the images will be generated at ‘<INSTALL_PATH_FOR_ISO>/build_ls2080ardb/tmp/deploy/images/ls2080ardb/’.

The following images are used for LS2080ARDB and Virtual Machine Bring-up:

• u-boot-ls2080ardb.bin

• PBL_0x2a_0x41_1867_533_1867.bin (rcw)

• mc_9.0.5_ls2080a_20160414.itb

• dpc-0x2a41.dtb

• dpl-odp.0x2A_0x41.dtb

• kernel-fsl-ls2080a-rdb.itb

• Image

• fsl-image-virt-ls2080ardb.ext2.gz.u-boot




4.6.2 Running Virtual MachineThe virtual machine runs inside the host Linux system and it is launched by an application called QEMU. In order to run avirtual machine instance we have to do some preparation steps:

1. Download the Guest Kernel and rootfs Image from Image server.

2. Mount Huge TLBFS on host so that QEMU can use huge pages.

3. Bind Guest’s DPRC to the VFIO driver on host.

Please note that console logs for the Guest Linux do not appear on the host Linux console (i.e UART). The guest logs areexposed through telnet, and they can be accessed by doing telnet on the host boards IP Address (IP_ADDR_BRD) andGUEST_CONSOLE_TELNET_PORT. Each Virtual machine that is run on a single host is allocated a differentGUEST_CONSOLE_TELNET_PORT, and this port number is specified by user running Virtual machine through QEMUcommand line.

Steps for running Virtual machine

1. Log-in into to the host Linux.

2. Assign IP_ADDR_BRD to network interface connected to Image server and to the machine from which guest consoleis accessed.

# ifconfig eth0 <IP_ADDR_BRD>

3. Mount Huge TLB FS

Execute the commands below to initialize the resources required by VM to be run:

# echo "hugetlbfs /hugetlbfs hugetlbfs defaults,mode=0777 0 0" >> /etc/fstab# mkdir /hugetlbfs# mount /hugetlbfs

4. Create DPRC Containers for VM

a. Create container for VM kernel

# ./vm_dpl_setup.sh vm_dpl_setup_parent.conf

b. Create container for VM user space

# ./vm_dpl_setup.sh vm_dpl_setup_child.conf <DPMAC1> <DPMAC2>

Please note that DPMAC1 and DPMAC2 have to be replaced with the DPMACs which have to be given to DPDK runningin VM.

The configuration files for DPL setup i.e. vm_dpl_setup_child.conf and vm_dpl_setup_parent.conf provide the sampleconfiguration for single VM running on 8 cores. For other configurations, the environment variables in configuration filehave to be modified accordingly.

In case of running two virtual machines, step a) and step b) have to be run twice. Each run will create the container forone virtual machine. In the second run the PARENT_DPRC env variable in vm_setup_child.conf has to be updated withthe DPRC created in second run of step a).

5. Check the new IOMMU groups created for child dprc container of Guest VM root container. dprc.1 is the system DPRCroot container which always belongs to Host Linux. In this example dprc.2 is the child of dprc.1 and this is going to bethe root container of VM.

# ls sys/devices/platform/80c000000.fsl-mc/dprc.1/dprc.2/ dpbp.1 dpcon.1 dpio.1 dpmcp.23 dpmcp.27 dprc.3 power dpbp.2 dpcon.2 dpio.2 dpmcp.24 dpmcp.28 driver rescan




dpbp.3 dpcon.3 dpio.3 dpmcp.25 dpmcp.29 driver_override subsystem dpbp.4 dpcon.4 dpio.4 dpmcp.26 dpni.1 iommu_group uevent

DPRC.3 is the only child container of DPRC.2, Check the iommu-group of dprc.3 # readlink /sys/devices/platform/80c000000.fsl-mc/dprc.1/dprc.2/dprc.3/iommu_group

Output../../../../../../kernel/iommu_groups/7 IOMMU group “6” and “7” should be available in /dev/vfio/

# ls /dev/vfio/6 7 vfio

:- dprc.2 and dprc.3 are resource container that contain pre-allocated resources like ethernet

interfaces to be used by virtual machines.

NOTE

The child’s IOMMU group has to be given to VM while launching. In above example child’s IOMMU group 7 has to begiven to VM during launch. Child’s IOMMU group is referred as CHILD_IOMMU_GROUP in further steps.

6. Go to a directory that has space to copy files. Copy Guest linux kernel and Guest root file system to the directory:

# cd /tmp/# scp <USER>@<IP_ADDR_IMAGE_SERVER>:<TFTP_BASE_DIR>/Image .# scp <USER>@<IP_ADDR_IMAGE_SERVER>:<TFTP_BASE_DIR>/ fsl-image-core-ls2085ardb.ext2.gz .

7. Launch the Virtual Machine

# qemu-system-aarch64 -m 4096 -nographic -mem-path /hugetlbfs -cpu host-machine type=virt -kernel ./Image -enable-kvm -initrd ./fsl-image-ls2085ardb.ext2.gz -append 'root=/dev/ram0 rw console=ttyAMA0,115200 rootwait earlyprintk ramdisk_size=1000000' -serial tcp::4446,server,telnet -device vfio-fsl-mc,host=dprc.2 -device vfio-smmu,x-group=<CHILD_IOMMU_GROUP> -smp 8 -S

NOTE

• Please note that GUEST_CONSOLE_TELNET_PORT is given as 4446 in above example, if you are running twovirtual machines you can give 4446, 4447 to the two virtual machines respectively/

• Value of CHILD_IOMMU_GROUP is 7 in the example that is taken in this document.

The following logs appear on Host UART console upon launching the Virtual Machine

QEMU 2.4.0 monitor - type 'help' for more information(qemu) QEMU waiting for connection on: disconnected:telnet::4446,serverqemu-system-aarch64: -device vfio-fsl-mc,host=dprc.2: Failed to create KVM VFIO device: No such deviceqemu-system-aarch64: -device vfio-fsl-mc,host=dprc.2: Failed to create KVM VFIO device: No such deviceOnly One Root DPRC can exists

Please note that QEMU logs come completely only when you log-in to the guest machine using telnet as specified insection “Accessing virtual machine console”.Also note that The log saying “Failed to create KVM VFIO device: No suchdevice” is currently a known issue, it can be ignored all functionality works even with this log.The –S option mentioned inthe qemu command stops the Virtual Machine bootup after initial setup. The QEMU threads needs to be affined to coressequentially (this is needed because of few H/W limitations). Run ‘info cpus’ command on QEMU CLI interface to see theQEMU threads.




Console To access the console for launched VM, telnet to the Host Linux ip address with telnet port as specified in theQEMU command while launching the VM

$ telnet <IP_ADDR_BRD> 4446Trying 192.168.1.102...Connected to 192.168.1.102.Escape character is '^]'.

Console print on UART where VM was launched After connecting to the VM through telnet, qemu command prompt willappear:

(qemu) QEMU waiting for connection on: disconnected:telnet::4446,serverqemu-system-aarch64: -device vfio-fsl-mc,host=dprc.2: Failed to create KVM VFIO device: No such deviceqemu-system-aarch64: -device vfio-fsl-mc,host=dprc.2: Failed to create KVM VFIO device: No such deviceOnly One Root DPRC can exists[Press Enter to drop to Qemu shell](qemu)

Affine VM core threads to Physical CPU

The Virtual Machine core threads needs to be affined to physical cores sequentially to avoid VM core migrating on physicalcores.

Run ‘info cpus command on QEMU CLI interface to get the VM threads ids

cid="dWNBT">(qemu) info cpus* CPU #0: thread_id=<tid1> CPU #1: (halted) thread_id=<tid2> CPU #2: (halted) thread_id=<tid3> CPU #3: (halted) thread_id=<tid4> CPU #4: (halted) thread_id=<tid5> CPU #5: (halted) thread_id=<tid6> CPU #6: (halted) thread_id=<tid7> CPU #7: (halted) thread_id=<tid8>

Open a new SSH session to the host Linux to affine VM threads to physical cores ----------- Affine core for VM on physicalcores 0,1,2,3,4,5,6,7 ---------------

~# taskset -p 0x1 <tid1>~# taskset -p 0x2 <tid2>~# taskset -p 0x4 <tid3>~# taskset -p 0x8 <tid4>~# taskset -p 0x10 <tid5>~# taskset -p 0x20 <tid6>~# taskset -p 0x40 <tid7>~# taskset -p 0x80 <tid8>

*****Continue launching and booting the Virtual Machine******

On the console where qemu was launched, run

(qemu) c

Once the VM is launched, prompt will come on telnet console

---------Booting Linux on physical CPU 0x0-----------[ 0.000000] Initializing cgroup subsys cpu




[ 0.000000] Linux version 4.1.8+gda8455a (b27504@b27504-OptiPlex-790) (gcc version 4.9.3 20150311 (prerelease) (Linaro GCC 4.9-2015.03) ) #1 SMP PREEMPT Fri May 20 16:59:34 IST 2016[ 0.000000] CPU: AArch64 Processor [411fd071] revision 1[ 0.000000] Detected PIPT I-cache on CPU0[ 0.000000] alternatives: enabling workaround for ARM erratum 832075[ 0.000000] alternatives: enabling workaround for ARM erratum 834220[ 0.000000] efi: Getting EFI parameters from FDT:……………….……………….Starting kernel log daemon...0Starting internet superserver: xinetd.

QorIQ SDK (FSL Reference Distro) 2.0 ls2080ardb /dev/ttyAMA0

ls2080ardb login:

8. Launching two Virtual machines

In case of two virtual machines, please take care of following:

• Memory assigned to each virtual machine should not exceed the total number of huge pages assigned on system. Inabove example (step 8) giving 2048 Mega bytes to each virtual machine works fine.

• Console telnet port of both virtual machine must be different, for example 4446 could be console of VM1 and 4447could be console for VM2.

Following are the example launch commands for two VMs:

VM1:

qemu-system-aarch64 -m 2048 -nographic -mem-path /hugetlbfs -cpu host -machine type=virt -kernel ./Image -enable-kvm -initrd ./fsl-image-ls2085ardb.ext2.gz -append 'root=/dev/ram0 rw console=ttyAMA0,115200 rootwait earlyprintk ramdisk_size=1000000' -serial tcp::4446,server,telnet -device vfio-fsl-mc,host=dprc.2 -device vfio-smmu,x-group=<CHILD_IOMMU_GROUP> -smp 8 –S

VM2:

qemu-system-aarch64 -m 2048 -nographic -mem-path /hugetlbfs -cpu host -machine type=virt -kernel ./Image -enable-kvm -initrd ./fsl-image-ls2085ardb.ext2.gz -append 'root=/dev/ram0 rw console=ttyAMA0,115200 rootwait earlyprintk ramdisk_size=1000000' -serial tcp::4447,server,telnet -device vfio-fsl-mc,host=dprc.2 -device vfio-smmu,x-group=<CHILD_IOMMU_GROUP> -smp 8 -S

ConsoleTo access the console for launched VM, telnet to the Host Linux ip address with telnet port as specified in theQEMU command while launching the VM machine.

4.6.3 Running ODP Application in VMResources Allocated to VM

Physical interfaces are assigned to VM through the resource container (dprc.x) assigned to the VM while executing the QEMUcommand

DataPathLayout ( dpl) file used to boot up the system contains two resource containers dprc.2 and dprc.4. Below table liststhe interfaces allocated to these two resource containers and below mentioned interface names shall be used while executingthe ODP applications.




Host - dprc.2

( dprc.3 – vm dprc )

Host dprc.4

( dprc.5 – vm dprc )

Physical Port Port Name to beused in ODP

Mac Address Physical Port Port Name to beused in ODP

Mac Address

eth4 dpni-1 02:00:c0:a8:47:01 eth6 dpni-3 02:00:c0:a8:47:03

eth5 dpni-2 02:00:c0:a8:47:02 eth7 dpni-4 02:00:c0:a8:47:04

Execute the commands below on the Virtual Machine before running the ODP application:

# echo 512 > /proc/sys/vm/nr_hugepages# cd /usr/odp/scripts/# export DPRC=<dprc.x> # ./bind_dprc.sh <dprc.x># export HOST_START_CPU=0

For <dprc.x> use:

• ‘dprc.3’ for VM launched with dprc.2

• ‘dprc.5’ for VM launched with dprc.4

For HOST_START_CPU :

• First phyical core which is assigned to VM. Default value is 0

NOTE

See Using ODP Applications for instructions on how to run ODP Applications.

4.7 Troubleshooting1. LS1043ARDB

• Please ensure to use correct bootargs:

bootargs console=ttyS0,115200 root=/dev/ram0 earlycon=uart8250,0x21c0500,115200 bportals=s0 qportals=s0 isolcpus=1-3

• In case application is not able to run please ensure that the path for scripts has been set in environment variable.

2. LS2085ARDB

• Please ensure to use correct bootargs:

bootargs console=ttyS1,115200 root=/dev/ram0 earlycon=uart8250,mmio,0x21c0600,115200 ramdisk_size=0x2000000 default_hugepagesz=2m hugepagesz=2m hugepages=256

• In case application is not able to run please ensure dpni.<index> is used correctly in the command line.

Common steps:

In case packets are not flowing:

• Ensure that traffic generator to board connectivity is proper.


Troubleshooting


• Ensure that the stream is configured properly having the destination MAC address of that interface where the stream isdestined.

• Ensure that there should be no space when interface are specified in application command.

Using DEBUG TOOL to get hardware stats for DPAA2 platforms:

plat_debug_tool is used to display stats of hardware blocks like QBMAN, Crypto, etc.

How to use plat_debug_tool

The plat_debug_tool is only applicable to DPAA2 platforms.

NOTE

1. Enabling Debug Server in ODP applications

By default debug server is disabled in ODP applications. To enable debug server, execute following commands

$ export PLAT_DEBUG_THREAD=1$ export PLAT_DEBUG_PORT=<x>

• <x> : UDP port number on which debug server will listen upon.

• Valid range: 1024 to 65535.

• Default port is 10000.

2. Running plat_debug_tool client

Before running plat_debug_tool, execute following command:

$ export PLAT_DEBUG_PORT=<x>

• <x> : UDP port number on which client will bind to.

• Default port is 10000.

UDP port number should be same as given for server.

NOTE

For running plat_debug_tool, execute following command.

$ ./plat_debug_tool -d dpni.1 -o 0 -c 0 -i 192.168.10.10 (Reference Command)

• Usage: ./plat_debug_tool OPTIONS

• Mandatory OPTIONS:

• -d, --device Device name like dpni.1, dpbp.1

Execute restool command to identify the HW devices used by ODP application

NOTE

Supported devices:

• dpni.x - for hardware interfaces

• dpbp.x - for buffer pools

• dpseci.x - for ipsec

option not required for object ID 20 (will be ignored if given)

NOTE


Troubleshooting


• -o, --obj_id all: Display all the stats for a given device.

Stats for dpni.x

• 0: Dpni Stats

• 1: Dpni Attributes

• 2: Dpni Link State

• 3: Dpni Max Frame Length

• 4: Dpni MTU

• 5: L3 chksum hardware offload (enable/disable)

• 6: L4 chksum hardware offload (enable/disable)

• 7: Dpni Primary Mac Addr

• 8: Congestion Group Id for FQs

• 9: Scheduling Priority for FQs

• 10: Tail Drop Threashold for FQs

• 11: FQ Context

• 12: FQ State

Stats for dpbp.x

• 13: Qbman frame count

• 14: Qbman byte count

• 15: Qbman has free buffers or not

• 16: Qbman buffer pool is depleted or not

• 17: Number of free buffers in qbman

Stats for dpsec.x

• 18: DPseci Attributes

• 19: DPseci counters

• 20: Per SA stats (only for odp_ipsec and odp_ipsec_proto apps)

To use per SA stats option, It is required to enable the ipsec-debugs at compile time. Following are

the steps to enable the ipsec-debugs in yocto:

1. Go to the following path:

a. cd sources/meta-fsl-dataplane/recipes-extended/odp

2. Add a following line in the file "odp_git.bb"

a. For board ls2080ardb:

i. EXTRA_OECONF_append_ls2080ardb += "--enable-ipsec-debug=yes"

b. For board ls2088ardb:

i. EXTRA_OECONF_append_ls2088ardb += "--enable-ipsec-debug=yes"

3. Go back to the build directory and run the following commands to recompile the ODP:

a. bitbake -f -c clean odp

b. bitbake -f -c compile odp

NOTE


Troubleshooting


• -c, --command

• 0: get

• 1: reset

• -i, --dest_ip

• IP address of debug server

• Default: local host (127.0.0.1)

• -h, --help

• Display help and exit.


Troubleshooting


How To Reach Us

Home Page:

nxp.com

Web Support:

nxp.com/support

Information in this document is provided solely to enable system and software implementers

to use Freescale products. There are no express or implied copyright licenses granted

hereunder to design or fabricate any integrated circuits based on the information in this

document. Freescale reserves the right to make changes without further notice to any

products herein.

Freescale makes no warranty, representation, or guarantee regarding the suitability of its

products for any particular purpose, nor does Freescale assume any liability arising out of the

application or use of any product or circuit, and specifically disclaims any and all liability,

including without limitation consequential or incidental damages. “Typical” parameters that

may be provided in Freescale data sheets and/or specifications can and do vary in different

applications, and actual performance may vary over time. All operating parameters, including

“typicals,” must be validated for each customer application by customer's technical experts.

Freescale does not convey any license under its patent rights nor the rights of others.

Freescale sells products pursuant to standard terms and conditions of sale, which can be

found at the following address: nxp.com/SalesTermsandConditions.

Freescale, the Freescale logo, and Kinetis are trademarks of Freescale Semiconductor, Inc.,

Reg. U.S. Pat. & Tm. Off. All other product or service names are the property of their respective

owners. ARM and Cortex are registered trademarks of ARM Limited.

Ⓒ 2016 Freescale Semiconductor, Inc.

QORIQODP1608Rev. 0

12 Aug 2016

http://www.nxp.com

http://www.nxp.com/support

http://www.nxp.com/SalesTermsandConditions

Documents

QorIQ OpenDataPlane v16