Using Predictive Analytics for Customer Intent Mining

Using Predictive Analytics for Customer Intent Determination

INFORMS Annual 2014

© 2013 24/7 Customer, Inc. All rights reserved. CONFIDENTIAL

Saturday, April 15, 2023

Samik Raychaudhuri, Ph.D.Ravi Vijayaraghavan, Ph.D.

Introduction

© 2013 24/7 Customer, Inc. All rights reserved. CONFIDENTIAL 2

• [24]7 is a software company based out of Bay area, US and Bangalore, India, delivering customer support solutions enhanced by predictive technologies

• Using predictive models to drive enhanced customer experience is an emerging and niche area of application of analytics and big data

• Our machine learning models on big data predict the customer intent across various touchpoints in real time, helping us provide an intuitive experience when the customers (of our clients) contact us

3© 24/7 Customer, Inc. 2013. All rights reserved. CONFIDENTIAL

2.5BDigital Interactions/Year

4.5TBInteraction Data/Week

90%+CSAT across channels

100M Visitors/Year

1stTrue Multi-modal Solution

1stOmni-channel Solution

We deliver a cloud-based software platform that uses predictive analytics and big data to make company-to-

consumer connections intuitive.

[24]7 - World’s Largest Self-Service Network

4© 24/7 Customer, Inc. 2013. All rights reserved. CONFIDENTIAL

[24]7 – Recent TechCrunch Article

http://techcrunch.com/2014/11/10/14-years-in-the-making-247-buys-intelliresponse-for-its-customer-service-suite/

5

Assist (for Chat)

Smart chat platform for online and mobile engagement

Assist (for IVR)

Call deflection to mobile web chat for higher NPS and ROI

Assist (for Voice)

Smart voice agent platform for multi-modal engagement of voice callers

SELF SERVICE

PRODUCTS

ASSISTED SERVICE

PRODUCTS

© 2014. 24/7 Customer, INC. All rights reserved. CONFIDENTIAL

Predictive Sales

Drive higher incremental revenue and customer acquisition

Predictive Service

Reduce customer effort to increase CSAT and NPS in customer service

Chat Agents

Chat agent services that engage customers and help reduce costs, generate revenue, and improve CSAT

Voice Agents

Voice agent services that engage customers and help reduce costs, generate revenue, and improve CSAT

SOLUTIONS

SERVICES

Social

Social sharing

Mobile

Mobile self-service

Vivid Speech

Mobile for IVR

Speech

Speech self-service IVR

[24]7 iLabs: A Quick Snapshot

Data Sciences @ [24]7 iLabs

Chief Data Scientist

R&D Data Infra Structure Speech Science Client Analytics

Client and Delivery Orientation

IP Asset Generation

25%PhDs

80+

DataScientists

50

Patents

75%Masters+

• Areas of Expertise - Machine Learning, Data Mining, Statistics, Operations Research, Speech Recognition,

Natural Language Processing, Econometrics, Math Modeling.

• IP assets created in critical areas such as – Natural Language and Speech Recognition, Omni-channel Intent

Prediction, Design of Experiments, Agent performance, Text Mining, Social Mining etc.

Data Science and Customer Experience

© 2014 24/7 Customer, Inc. All rights reserved. 77

V0.0•No Data•No Tools•No Scale•Ad Hoc Metrics

V1.0•Structured Data•BI Tools•Offline Analysis & Decisioning •Scale•Metrics on Efficiency, Quality and Compliance

V2.0•Structured and Unstructured Data•Analytics tools (SAS)•Offline Analysis & Decisioning•Scale•Metrics on segment level CSAT, NPS, loyalty, value•Siloed Channels

V3.0•Structured and Unstructured Data•BIG DATA infra/tools•Offline Analysis and Real-time Decisioning•Massive Scale•Metrics on Individual Customers - preferences, issues, sentiments•Integrated Channels and Devices

Dat

a S

cien

ce M

atu

rity

Data Science – What it means for [24]7


fn (Customer type,

location, Identity, interaction context, journey, behavior …)

Intent: Purchase; issue with product or service, …

Customer Intent Engine

Intent Models

fn (Identity, ntent type,

history, channel affinity, customer value…)

Measure: usage, containment, repeat…

Engagement Engine

Guided self-

service

“ ”

Chat

Phone

Sales

Resolution

Experience

Retention

Metrics: conversion rate, revenue, CSAT, …

Outcomes

Machine Learning At Scale

Creating Personalized Intuitive Consumer Experiences

Anticipate Simplify Learn

Big Data in [24]7


Big Data

• Web & IVR Logs• Web Journeys• Transcripts• Social media• CRM• Customer history• Product mix

• Surveys• Switch data• Agent performance• Agent dispositions• Agent notes

Big Data Platform: Technologies


• We use varieties of open-source or free technologies to power our platform. Some of the technologies in use:

• Real Time Data Platform (RTDP)

• Apache Cassandra ring [http://cassandra.apache.org/]

• Jetty server for execution [http://www.eclipse.org/jetty/]

• BDP

• Apache Hadoop [http://hadoop.apache.org/]

• Apache Hive [http://hive.apache.org/]

• Apache Spark [http://spark.apache.org/] [Upcoming]

• Others

• Apache Kafka [http://kafka.apache.org/]

• Apache Avro [http://avro.apache.org/]

• HP Vertica database [http://www.vertica.com/]

• Apache Pig [http://pig.apache.org/]

• Apache Storm [https://storm.apache.org/]

Use case of intent prediction: Web visits


• For our clients in the retail vertical, we provide chat agents who are experienced in providing differentiated support

• The differentiation is based on:• Current phase of the journey

• Specific persona of the visitor

• Essentially using targeted web chat to drive an intuitive experience

12

How to Engage - All customers are not the same...

At any point of time on a website..

Geeks

• Attention to details

Deal seekers

• Looks for discounts

Convenience buyers

• Has a specific need• Wants to finish sale

fast

Chatty-Cathy

• Gives a lot of context on the purchase

Novice

• Doesn’t have a specific need in mind

We find customers with different behaviors

Each persona needs a different kind of engagement.

27%

6%

8%

14%

15%

13

How to Engage - Personalizing experiences improves engagement and increases revenue

Conv

ersi

on R

ate

Geeks

• Attention to details

Chatty-Cathy

• Gives a lot of context for every action

Geeks when engaged with more technical details, are more likely to

purchase

Customers with Chatty-Cathy persona when engaged with more

questions, are more likely to purchase

Geeks Others0.0%0.5%1.0%1.5%2.0%2.5%3.0%3.5%4.0%4.5%

Less technical details More technical details

Chatty-Cathy Others0.0%

0.5%

1.0%

1.5%

2.0%

2.5%

Less probing questions More probing questions

1.38X 1.46X

38%46%

0.3

0.3

0.9

0.8

0.8

0.5

0.3

0.3

0.8

0.30.20.1

0.3

0.3

0.3

0.9

0.6

0.3

0.7

0.6

0.40.3

0.10.9

0.3

0.8

0.30.6

0.2

0.8

0.7

0.10.5

0.4

0.3 0.10.2

0.30.3

0.7 0.50.3

0.3

0.9

Targeting with Business Rules

14

Rule 1

Rule 2

Rule 3

0.3

0.3

0.9

0.8

0.8

0.5

0.3

0.3

0.8

0.30.20.1

0.3

0.3

0.3

0.9

0.6

0.3

0.7

0.6

0.40.3

0.10.9

0.3

0.8

0.30.6

0.2

0.8

0.7

0.10.5

0.4

0.3 0.10.2

0.30.3

0.7 0.50.3

0.3

0.9

Targeting with ML Models

15

The Inside Story: Data Fusion to Predicting Intent


Web Self Service

•User ID•Search•Referrer•Journey•Behavior•Time

Speech Self Service

•User ID•Journey•Geo•Effort•Recognition•Completion

Chat

•User ID•Transcript•Agent Tags•Experience•Resolution

Mobile Self Service

•User ID•Geo•Journey•Behavior

Big Data

Identity

Location

Prior Context

Current Journey

Intent Models

17

Web Self Service: Data from Weblogs

© 2014 24/7 Customer, Inc. All rights reserved.17

1.23.172.207 - - [30/May/2012:00:08:21 -0400] "wid=1338350897094&mid=0&vid=d47c88d3-6e3c-4206-a780-1b7c240a808&bsid=1338350897097-379718&ts=1338350897131&...&title=Essential G570 15.6" Laptop | Shop | [24]7| US& meta_NumRating=37&meta_AvgRating=4.3& meta_TaxoTyp=SubSeriesPage& meta_ModelName=G Series& meta_ModelNum=[24]7 G570 - Best & prod_List=$699.00`& prod_ECoupCode=DOORBUSTER0524`& vc=1& ref=http://www.google.com/url?sa=t&rct=j& q=[24]7 Essential G570 &source=web&cd=1&ved=0CMkEEBYwAA&url=…&"Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/536.5 (KHTML, like Gecko) Chrome/19.0.1084.52 Safari/536.5"

RAW WEBLOGS

•Search term•Organic/Paid•Search Engine•Campaign•Referrer•Geo•……….

•Journey/Path•History•Internal Search•Time•Product•Behavior•………..

+

Web Logs

PRE-DOMAIN DOMAIN

Web Self Service: Interaction Data


• We also capture interaction data from web self-service journeys

• page-level activity

• Vertical specific activity

• Retail:

• products wishlisting

• products checkout

• Banking

• Opening new account

Model Trained to Extract Intent from Chat Interaction


Intent Type: PurchaseProduct Choice - Tablet

Collaborative Tagging

Customer: I'm looking to buy a tablet?

Machine Learning

Availability Check forVisualpad A170e

Availability and price check for Visualpad tablet

Promotions for Ideapad A745 Accessory Availability

INTENT Classification

Agent: Thank you for contacting us. How may I help you today?Customer: I am looking to buy a tablet.Agent: May I know the price range you are considering?Customer: Around 400$Agent: I would recommend X series.Customer: Ok.Agent: I need your fist and last name to create a quote.Customer: Phillip Jones.

Customer Intent

Chat Transcripts

Specific Case Study – Example Types of Models

20

Chat Propensity

Inte

nt

to P

urc

hase Target

Population: Chat &

Purchase

Target Population:

Chat & Purchase

Low

High

Chat Propensity ModelLog (Chat/No Chat) = a + b* # of pages + c*present

page + d*previous pages + e*interaction history +

f*referral + g*search

Purchase Propensity ModelLog (Purchase/No Purchase) = a + b* time on

session + c*present page + d*previous pages + e*referral

+ f*landing page + g*browser

Assist if X & Y> Thresholds

Evaluation for cut-off

score

Visitor Purchase

Propensity Score - X

Visitor Chat PropensityScore - Y

Iteration Every 10 secs

A Computer Manufacturer with Global Reach

21

Nov-Dec '12 Jan-Mar'13 Apr-Jun'13 July-Sep'13 Oct-Dec'130

1

2

3

4

5

6

7

8

Results: Steady revenue growth via deployed models

© 2014 24/7 Customer, Inc. All rights reserved.

*Monthly revenues averaged for quarter*Program started in Nov’12

First set of target and when to invite models

for conversion and engagement

Campaign segment target models for e-mail,

display ads and SEO with contextual invites

Staffing models and User experience initiatives

around look and feel of chat invite

Segmented targeting models by key domains

Segmented targeting models for bouncers and Repeat Visitors

22

And Incremental Performance

© 2014 24/7 Customer, Inc. All rights reserved.

1 20

1

2

3

4

5

6

7

Accurate targeting, measurement and data driven performance

management ensured that incremental lift is created through chat

Con

vers

ion

X

5.94X

Hot LeadSelf Serve

Hot LeadChat

Technology


• The trained models are made available on our big-data platform

• The platform uses Apache Cassandra for data storage and Apache Storm for real-time reporting

• Data from Cassandra is used to evaluate the prediction function in real time

Cassandra DB Raw Data Transformed Data Prediction

Challenges and Future Research


• Data preparation• Quality framework and dashboarding

• Performance of models in real-time

• Continuous monitoring

• Automated retraining of models• Determining session length

• Standard 30 mins from last interaction

Questions


Documents

Using Predictive Analytics for Customer Intent Mining