Application Notes for Configuring Amazon Lex and Polly

TLC; Issue 1.3 Application Note 1 of 22 ©2019 Avaya Inc. All Rights Reserved

Application Notes for Configuring Amazon Lex and Polly with Avaya Experience Portal Version 7.2.2 Issue 1.3

Abstract

These Application Notes describe the configuration steps required to integrate Amazon Lex and

Amazon Polly with Avaya Aura® Experience Portal. Amazon Lex is an Amazon Web Services

service for providing Omni-channel bot capabilities within a contact center. This service can be

used to create automated bot system that can service chat, SMS and voice applications. These

applications can provide automated self-service in an Avaya contact center.

These notes describe what is required to set and configure both the Avaya Experience Portal and

the Amazon Lex application to all these solutions to interoperate. These notes assume that the basic

Avaya Experience Portal solution is set up, configured and connected to the voice telephony

system, an SMS gateway, and a web application server.


Table on Contents

1. Introduction ............................................................................................................................ 4

2. General Test Approach and Test Results ................................................................................ 4

3. Reference Configuration ......................................................................................................... 4

4. Equipment and Software Validated ........................................................................................ 6

5. Configure Amazon Lex ............................................................................................................ 6

5.1. Prerequisites – setting up your account ............................................................................. 6

5.2. Create Bot Example – Book a Hotel .................................................................................... 7

5.3. Amazon Lex Configuration – Alias and keys ....................................................................... 8

6. Configure Orchestration Designer Application ..................................................................... 10

6.1. Libraries to add to the project .......................................................................................... 10

6.1. Add Libraries to Build Path ............................................................................................... 11

6.2. Configure the Trust Store in Orchestration Designer ....................................................... 12

6.3. Configurable Parameters .................................................................................................. 12

6.4. Simulation Profile .............................................................................................................. 12

6.5. Chat Flow .......................................................................................................................... 13

6.6. SMS Flow ........................................................................................................................... 14

6.7. Voice Flow ......................................................................................................................... 14

6.8. Deployment Considerations ............................................................................................. 15

6.9. Libraries ............................................................................................................................. 15

6.10. Certificates and Trust Store .......................................................................................... 15

7. Configure Avaya Experience Portal ....................................................................................... 16

7.1. Voice Application .............................................................................................................. 16

7.2. SMS Application ................................................................................................................ 16

7.3. Chat Application ................................................................................................................ 17

8. Verification Steps .................................................................................................................. 18

8.1. HTML Chat Application ..................................................................................................... 18

8.2. SMS application................................................................................................................. 18

8.3. Voice Bot Application ........................................................................................................ 19

9. Limitations of the Current Integration Model ...................................................................... 20

10. Conclusion ......................................................................................................................... 20


11. Additional References ....................................................................................................... 20


1. Introduction These notes detail the integration with Amazon Lex and Avaya Experience Portal. Amazon Lex is an Amazon Web Services (AWS) service for speech and text recognition. It can be used in conjunction with another AWS service called Polly which provides Text to Speech services. Together these along with the Amazon Lex tools provide the basis of a speech enabled “bot”. This service is more than just speech as it also can take any textual input and run it through the same bot logic. Thus, a bot can be created that will provide chat, SMS and speech via a single bot. These application notes detail how this Amazon Lex bot can be integrated with Avaya Experience Portal and Orchestration Designer to leverage the different communication channels, contact center integration, and dialog capabilities inherent in AAEP to integrate Omni-Channel Amazon Lex bots into the contact center automation experience.

2. General Test Approach and Test Results The approach is to take an Avaya Experience Portal that is configured with voice channel licenses and connectivity; SMS channels and connectivity to an SMS gateway and HTML channels and a Tomcat application server for hosting mobile web chat. To this configuration the applications that integrate to Amazon Lex are added to be tested using these channels

3. Reference Configuration Figure 1 below show the high-level architecture and configuration. The communication channels (voice, SMS, HTML chat) are provided by the Avaya Experience Portal platform. The AAEP platform then exposes these channels to the Orchestration Designer dialog framework. The Orchestration Designer dialogs can then communicate with the Amazon API Gateway to provide the Omni-Channel bot interaction.


Figure 1 - High Level Configuration

Figure 2 below provides a more detailed configuration and architecture for the interactions with the end user, the AAEP system, and the Amazon Lex Bot. The interaction elements for each of the media are detailed below. Text Chat: The text chat interface is implemented via an HTML5 chat interface within a web browser. This HTML5 application is created with Orchestration Designer. The web application is hosted on a web application server and is accessed via a reverse proxy connection. The Orchestration Designer provides the dialog that interfaces with the user and interfaces to the Amazon Application Gateway to send and receive the responses to the Lex bot. SMS: The SMS interface is implemented using the SMS channels in Avaya Experience Portal. This provides either a SMPP or HTTPS interface to a standards-based SMS gateway. The SMS channel in AAEP provides a channel interface to Orchestration Designer application for SMS dialogs. As part of processing the SMS dialog, the Orchestration Designer application interfaces with the Amazon Application Gateway to send and receive the responses to the Lex bot. Since SMS is a session-less communication mechanism, we are using the conversation store within Avaya Experience Portal to provide the state of the conversation between turns. Voice: The voice interface is provided by the Media Processing Platform (MPP). This provides the voice interface and integrates with the VoiceXML browser which interprets the VoiceXML dialogs that are written in Orchestration Designer. With the voice application, the system prompts the user for input. The user’s response is captured via a record node. This voice is sent to Lex for processing. The Lex bot responds back with a Polly (Amazon Text to Speech) audio file that is played back to the user using the MPP.


Figure 2 - Detailed Configuration

4. Equipment and Software Validated

Equipment/Software Release/Version Avaya Experience Portal 7.2.2

Avaya Orchestration Designer 7.2.2

Amazon Lex - Amazon Polly -

Context Store Snap-In 3.5 Aws-java-api 1.11.573

5. Configure Amazon Lex

5.1. Prerequisites – setting up your account


There are some basic steps to setting up and Amazon Lex account for developing a bot application using an Amazon AWS infrastructure.

1. Set up an Amazon Developer Account. Go to https://developer.amazon.com/alexa Click on the Sign In button in the upper right and either sign in or click on the Create you Amazon Developer account button. This will create your Amazon developer account.

2. Set up and Amazon Web Services Account: Go to https://aws.amazon.com This will create your account for AWS.

3. If you do not already have a GitHub account, you can create one here: https://github.com, otherwise there are a set of sample applications and tutorials at: https://github.com/alexa

5.2. Create Bot Example – Book a Hotel Once you have your account set up (see previous section), navigate to the Amazon Lex service under AWS services. This will bring you to the Lex Console From the Lex Console – choose Create. You can choose to either Custom Bot or Try a Sample. We want to choose BookTrip under the Try a Sample section. As shown below. Name your Bot as you would like and select Create at the bottom.

This will create a sample Bot that you can test in the AWS Console. Step 1 – Click on Build – this will create the Bot – Note the bot name will be used later in configuration. Step 2 – Once built, you can test your bot using the Test Bot panel on the right.

https://developer.amazon.com/alexa

https://aws.amazon.com/

https://github.com/

https://github.com/alexa

https://console.aws.amazon.com/lex/home?region=us-east-1


Step 3 – Once you are happy with your bot you can publish by clicking on the Publish button in the upper left. – Note this Alias will be used in configuration. Alias are very useful in creating multiple independent versions of the applications.

5.3. Amazon Lex Configuration – Alias and keys In the AWS Console (back to top level) click on IAM (Identity and Access Management). Start with Add user to add a user that will be accessing the bot functionality. Give them Programmatic access.


Set the permissions for the user, by attaching the following existing policy: AmazonLexRunBotsOnly

Create your access keys: Copy the keys for use in configuration in Avaya Experience Portal!!

When you configure your application within Orchestration Designer you will need: Amazon Access Key: ____________________ Amazon Secret Key: ____________________ Amazon Bot Name: ____________________ Amazon Bot Alias: _____________________ Amazon User: _________________________ These will go into the configurable variables for the system as shown below:


6. Configure Orchestration Designer Application Note that with this discussion the base release is assumed to be what is specified Release table above. The sample applications are included with this application note to illustrate the functionality, and are not intended as real production applications.

6.1. Libraries to add to the project In order to access the Amazon API gateway functionality, the following library must be added to the tomcat/conf/lib folder:

• aws-java-sdk-1.11.573.jar This file has the following dependencies:

• commons-codec-1.10.jar

• ion-java-1.0.2.jar

• jmespath-java-1.11.573.jar

• joda-time-2.8.1.jar

• httpclient-4.5.5.jar

• httpcore-4.4.9.jar

• Jackson-annotations-2.6.0.jar

• Jackson-core-2.6.7.jar

• Jackson-databind-2.6.7.2.jar

• Jackson-dataformat-cbor-2.6.7.jar

• joda-time-2.8.1.jar These can be obtained from the Amazon Lex API site. Or they are available from Avaya as a download to support Amazon Lex integration.

https://aws.amazon.com/lex/developers/


Within this API bundle, there is a lib directory that contains the main aws jar:

• aws-java-sdk-1.11.573.jar There is also a third party lib directory that contains the dependencies:

• commons-codec-1.10.jar

• ion-java-1.0.2.jar

• jmespath-java-1.11.573.jar

• httpclient-4.5.5..jar

• httpcore-4.4.9.jar

• jackson-annotations-2.6.0.jar

• jackson-core-2.6.7.jar

• jackson-databind-2.6.7.2.jar

• jackson-dataformat-cbor-2.6.7.jar

• joda-time-2.8.1.jar

6.1. Add Libraries to Build Path Add the libraries detailed to the project build path. To do this go to Project -> Properties -> Java Build Path. Click on Libraries tab Click on Add Variable Choose TOMCAT_HOME Click Extend Click on the lib directory Choose the libraries to add The resulting build path should look like:


6.2. Configure the Trust Store in Orchestration Designer In Orchestration Designer go to Windows -> Preferences -> Avaya Aura -> Orchestration Designer -> Certificates. If not set already to a keystore, set this to the Java keystore. Note the default password for doing this with OD is changeit

6.3. Configurable Parameters Within all the sample Orchestration Designer projects, there are a set of configurable variables that are defined for the project that enable you to have a generic application with a configurable bot. To enable these, first enable the AVP/AEO Configurable Application Variables under Project – Properties – Orchestration Designer – Pluggable Connectors.

6.4. Simulation Profile


A nice way to set the configurable variables that are required for AWS communication is to set them in a Simulation Profile. To do this, click on the pencil icon to the right of the simulation profile in Application Simulation tab in Orchestration Designer. Choose New, and name the profile (Amazon Lex in the example below). Click on the Variable Administration tab. Click on the Import Variables button to import the configurable variables from your project. Once they are imported, you can set them with the configuration for your bot. Save this information, and they use this profile with your simulation to be able to access your bot from Orchestration Designer.

6.5. Chat Flow The sample chat flow is an Orchestration Designer HTML web project the will greet and prompt the user with the initial question. This question is then fed to the bot using the AWS Lex API using the bot name, alias and credentials specified by the configuration. This provides the initial intent to the bot. The flow then cycles though a loop, with the bot providing the prompts, and the Orchestration Designer application collecting the input and sending it to the bot. when the all the slots are filled, the Orchestration Designer application will display the results and exit.


The GetChatData is java application that interacts with the AWS Lex API to pass information back and forth to Lex.

6.6. SMS Flow The sample SMS flow is a Messaging application within Orchestration Designer. Since SMS is a stateless communication, the state of the conversation is maintained in this example within Conversation Store within the Orchestration Designer/Avaya Experience Portal environment. Conversation Store maintains the state of the dialog between SMS interactions. In the flow below, the message comes in. The ManageConversation node gets the state and the dialog. Next the message is sent to Lex, and results and presented back to the user. The check conversation node will check if the conversation state is completed and ready for fulfillment.

6.7. Voice Flow With the sample voice flow, the flow uses the record capability to capture the user voice input to send to the Amazon API gateway. In this case, the Polly audio is played back to the user via the Play node. Like in the previous cases, the completion/ready for fulfillment state is used to determine of the bot is done, and ready for fulfillment.


6.8. Deployment Considerations The Orchestration Designer application can be packaged and deployed using the standard deployment options in Orchestration Designer.

6.9. Libraries The project can contain the AWS libraries, however, this will result in a very large package, since all the AWS libraries will be included. To address this, the set of libraries that were specified in Section 6.1 should be installed on the application server under $TOMCAT_HOME/lib

6.10. Certificates and Trust Store On the application server, access the Orchestration Designer runtimeconfig configuration web page: http://<App-Server>:<port>/runtimeconfig/ Default login/pw = ddadmin/ddadmin Change the keystore to point to the application server cacerts file. For the default application this is /opt/tomcat/lib/cacerts


7. Configure Avaya Experience Portal These application notes assume the basic Avaya Experience Portal configuration, and only detail the incremental configuration for the Amazon Lex integration.

7.1. Voice Application The Lex Voice application is deployed on the Avaya Experience Portal as a VoiceXML application. This needs to be assigned an incoming phone number. In this case, the application does not use ASR or TTS resources, thus no ASR or TTS needs to be configured. In order for this application to communicate with your bot you need to specify the bot name and credentials – click on the pencil icon on the far right of the application configuration screen.

7.2. SMS Application The SMS LEX application is deployed on the Avaya Experience Portal as a SMS application. It needs to be assigned to an incoming SMS number (or a specified launch pattern if the number is shared).


In order for this application to communicate with your bot you need to specify the bot name and credentials – click on the pencil icon on the far right of the application configuration screen. Since this is an SMS application, there is also conversation lease time. This is the time that the conversation stays active in conversation store in Avaya Experience Portal

7.3. Chat Application The LEX chat application is deployed on the Avaya Experience Portal as a HTML application. It has the URL on the application server (internal URL) and that of a redirector application for external access (External URL) In order for this application to communicate with your bot you need to specify the bot name and credentials – click on the pencil icon on the far right of the application configuration screen. Note that this looks just like the voice application.


8. Verification Steps For verification we will start with the simplest to test first to be able to eliminate connectivity and configuration issues from the verification.

8.1. HTML Chat Application To test the HTML chat application access the HTML application URL: http://<App_server_ip>:port/Application_Name/Start This will bring up the first page of the chat application. Continue though the interaction as shown below:

8.2. SMS application To test the SMS application, send an inbound SMS application to the SMS number administered for your Bot. This will then start the dialog, which you can continue as shown below.


8.3. Voice Bot Application To test the voice bot application, dial the number for the voice bot application that was configured in Section 7.1. The system should answer with a prompt and you can have the following dialog:

System User How can I help you?

Book a room What city will you be staying in?

Chicago What day to you want to check in?

Next Tuesday How many nights will you be staying?

Three What type of you would you like: King, Queen of Deluxe?

King OK, I have you down for a 3-night stay in Chicago starting on <date>. Shall I book the reservation?

Yes


9. Limitations of the Current Integration Model As you have seen with the applications and the integration, there are few limitations with the current integration mechanism. For the voice integration, the current integration is to record the utterance and then sent it to the Amazon for recognition. This has a few limitations: 1 – This integration mechanism does not support the standard speech function “barge-in” or the ability to interrupt or barge in to the playing prompt. 2 – Because of this store and forward methodology, there is the potential for latency in the communication, especially if there are transmission and/or processing delays. Another potential issue has to do with having a single bot for SMS, Chat and Voice. While this is a development and maintenance cost advantage to have a simple bot for all the different media, there will undoubtedly be limitations do to the differences in the communication media. Those who work with speech IVR have long ago learned that the spoken language is different from the written language in phrasing and usage. This may become an issue with more sophisticated bots.

10. Conclusion Amazon Lex and the AWS toolkit provide an easy way to integrate in Omni-Channel bot functionality into Avaya Experience Portal.

11. Additional References Orchestration Designer Developer Guide – Provides detailed information on developing applications in Orchestration Designer Orchestration Designer on-line help. Accessed from within Orchestration Designer by selecting Help – Help Contents – Avaya Aura Orchestration Designer. Avaya Experience Portal Administration Guide – Provides detailed information on configuring and administering the Avaya Experience Portal. Getting Started with Amazon Lex – Getting started guide Alexa Design Guide – How to design Alexa Applications Alexa Tutorial – Tutorial on developing Alexa Skills Alexa Skill Testing Tool – Online tool to simulate Alexa on your desktop Amazon Lex API – Amazon Lex API and documentation

https://support.avaya.com/css/P8/documents/101046172

http://support.avaya.com/css/P8/documents/101045870

https://aws.amazon.com/lex/getting-started/

https://github.com/alexa/skill-sample-nodejs-city-guide

https://echosim.io/welcome

https://aws.amazon.com/lex/developers/


Amazon Lex Documentation – Complete set of documentation – look at the Developer Guide here. AWS Toolkit for Eclipse - Amazon Lex Tools for Eclipse

https://aws.amazon.com/documentation/lex/

https://docs.aws.amazon.com/lex/latest/dg/what-is.html

https://aws.amazon.com/eclipse/


©2017 Avaya Inc. All Rights Reserved.

Avaya and the Avaya Logo are trademarks of Avaya Inc. All trademarks identified by ® and TM

are registered trademarks or trademarks, respectively, of Avaya Inc. All other trademarks are the

property of their respective owners. The information provided in these Application Notes is

subject to change without notice. The configurations, technical data, and recommendations

provided in these Application Notes are believed to be accurate and dependable but are presented

without express or implied warranty. Users are responsible for their application of any products

specified in these Application Notes.

Documents

Application Notes for Configuring Amazon Lex and Polly