Using Facebook Messenger Bots for Market Research

By Kate DuHadwayMICHIGAN STATE UNIVERSITY MASTER’S OF SCIENCE IN MARKET RESEARCH

A Proof of Concept Case Study

%a

AgendaA BRIEF HISTORY OF CHATBOTS

• What are they?

• What do they mean for the industry?

WHAT WE DID

• Look and feel of a FacebookMessenger-based survey

KEY FINDINGS

• Devices used and overall experience

• Demographics and tech adoption

• Costs and response rates

• Best practices and opportunities for further research

What Are Chatbots?

MINI APPLICATIONS THAT EXIST WITHIN

MESSAGING SERVICES

RESEMBLEONE-TO-ONE OR

PERSON-TO-PERSON CONVERSATIONS

USUALLY INCLUDE SOME FORM OF

ARTIFICIAL INTELLIGENCE

Applications that are designed to respond to conversational language - like narrow versions of digital assistants like Apple's Siri, Amazon's Alexa or Google Assistant, designed to perform specific tasks.Steve Ranger, ZDNet

A Brief History of Chatbots

I N T H E U . S . , C H AT B O T S H AV E T A K E N O F F J U S T W I T H I N T H E L A S T Y E A R

ELIZA,acomputerprogramdesignedbyJosephWeizenbaum,

professorofcomputerscienceatM.I.T.,iscreatedasa

therapist(butdoesn’tpassthe

Turingtest)

ClippyofferstipsforusingMicrosoftOffice

WeChatopensitsplatformtochatbots

inChina

FacebookopensMessengerplatformtobotdevelopers

inApril,nowhasover10,000 bots

MicrosoftlaunchesbotplatformforBing,Skype

AmazonlaunchesbotdevelopmentplatformforAlexa

Siricreatedin2008,debutsoniPhonein2010

KiklaunchesbotshopinApril2016,chatbotstakeoffontheplatform

1966 1996 2013 2016 20172014

AmazonlaunchesitsEchodigitalassistant,poweredbyAlexa

FacebooklaunchesGroupchatbots

GoogleannouncesChatbase,chatbotanalyticsplatform

2008

Are Chatbots theNext Mobile Apps?

*Source:Companies,BIIntelligence,2016,Global

45%

18%

10%

#1mostusedapp

#2mostusedapp

#3mostusedapp

Smartphone app use - % of time

TOTAL 73%Of time spent on top 3 mobile apps

TOP MOBILE APPS 150M

130M

116M

Montlyuniquevisitors

ComScore2016MobileAppreport,U.S.18+

Chatbots and Market Research

% %a

Using chatbots for market research is something that

hasn’t been fullyexplored yet

Researchers need to be where their consumers

are, which is increasingly mobile

Messaging appsare a natural and

intuitive way to reach people on mobile

Proof of Concept:What We Did

METHODOLOGYTwo experimental groups recruited via Facebook news feed promotions to take a survey on Facebook Messenger

INCENTIVES: POINTS FOR PANELISTS, AMAZON GIFT CARD FOR NON-PANELISTS

One control group recruited from our existing panel to take the same 10 min survey on our website

1ST GROUPActive panelists (targeted via email list)

2nd GROUPAnyone but active panelists

959TOTAL COMPLETES

VIA MESSENGER BOT

Total completes via website survey: n=1660

Look and Feel

Keys to a GoodUser Experiencea Simple, clear instructions

a Short response options

a Limited open ends

a Use of “quick reply” feature

Look and Feel

a Simple, clear instructions

a Short response options

a Limited open ends

a Use of “quick reply” feature

Keys to a GoodUser Experience

Look and Feel

a Simple, clear instructions

a Short response options

a Limited open ends

a Use of “quick reply” feature

Keys to a GoodUser Experience

Look and Feel

a Simple, clear instructions

a Short response options

a Limited open ends

a Use of “quick reply” feature

Keys to a GoodUser Experience

86% 80% 80% 77%

68% 69% 79%

74%

Smartphone Tablet Desktop Laptop

Overall Survey Experience by Device used, Top 3 Box

FacebookMessengersurvey Web-basedsurvey

Denotessignificanceatthe95%confidencelevel

Base:AlltechnologyintendersWeb-basedsurvey:n=1660FacebookMessengersurvey:n=620

User Experience

Easy to use and understand. The questions were simple and straight forward. Then the answers that I wanted to choose were easy to choose…didn't even have to use a drop down menu. –Danielle H.

76%

35%

Key Findings – Devices

DEVICES USEDTO TAKE SURVEY

9%

7%

7%

23%

32%

9%

Denotessignificanceatthe95%confidencelevelBase:Alltechnologyintenders.Web-basedsurvey:n=1660.FacebookMessengersurvey:n=620

Typically take our web-based surveys via smartphone

1 in 3

Smartphone

Desktop

Laptop

Tablet

Web-based survey

Facebook Messenger survey

Key Findings – Costs

Comparing Apples to Apples

Cost comparisons

reflect cost to recruit

a Messenger-based panel

With opt-in, we can

send notifications for new

surveys directly to panelists’

Messenger inboxes, pre-

screened based on previous

survey responses

Total costs were similar to

slightly

lower than previous panel

acquisition campaigns via

Facebook

BEST PRACTICE

Using traditional panel for

longer/more complicated

survey designs, using

Facebook Messenger

for quick reads,

in-the-moment surveys,

mobile audiencea

Significantly more cost effective to target our current active panelists due to name/brand recognition

$

Key Findings – Response Rates

Qualified completesas a percentageof survey starts

%

39%42%Facebook Messenger Web-Based Survey

34%

41%

25% 25%

35% 40%

18to34 35to54 55andolder

General population (US) Messenger surveyrespondents

Key Findings –Demographics

Target:Gen Pop U.S.

U.S. FACEBOOK USERS

Female

52%Male

48%

MESSENGER SURVEY

Female

Male - 29%

71%

Data Findings – Intent to Purchase Yes/NoSignificant differences only among TV, VR Headset, and Streaming Device purchase intenders

10%

13%

16%

16%

16%

16%

18%

25%

31%

50%

19%

15%

12%

7%

23%

20%

17%

34%

28%

44%

Streamingdevice

Desktop

Smartwatch

VRheadset

Tablet

GamingConsole

FitnessWearable

TV

Laptop

Smartphone

In the next 6 months, do you intend to purchase a …?

Web-basedsurvey FacebookMessengersurvey

3%

3%

5%

6%

7%

7%

9%

10%

10%

5%

7%

2%

9%

5%

11%

6%

17%

16%

e-Reader

HandheldGamingDevice

ARglasses

Smartwear

Wearablecamera

DigitalHomeAssistant

2-in-1

Speaker/stereosystem

Homevideoplayer

In the next 6 months, do you intend to purchase a …?

Denotessignificanceatthe95%confidencelevel

Base:AlltechnologyintendersWeb-basedsurvey:n=1660FacebookMessengersurvey:n=620

Tech Adoption

§ Facebook Messenger respondents were more likely to be innovators when it comes to tech adoption

§ Web-based survey respondents were more likely to be part of the late majority

31%

15%

31%

13%

7%

2%

12% 12%

33%

23%

14%

5%

I’mactivelyonthelookoutto

buynewtechnologydevicesandservices

I'malwayskeentousenewtechnologyproductsassoonastheyenterthemarket

Iliketogetnewtechnology

productsafterthey’vebeenoutforawhile

IsometimesbuynewtechnologyproductsbutonlywhenIreallyneed

them

Ionlyreplacetechnology

productswhentheygowrongorarebroken

Noneofthese

FacebookMessengersurvey Web-basedsurvey

Denotessignificanceatthe95%confidencelevel

Base:AlltechnologyintendersWeb-basedsurvey:n=1660FacebookMessengersurvey:n=620

Questions?

Learnings andFuture ResearchOpportunities

Bot experience works best within

Facebook’s native Messenger app, or

within the Messenger-specific interface

in a browser.

Using 3rd-party scripting tool

vs developing a bot

from scratch

Biggest challenge:

Data collection

and analysis

Incentives – what amount is

ideal for this medium?

Qualitative research:

Seamless live chat

integration

Varied user inputs:

Audio, images, video, opt-in geolocation, app

data, or wearable data

Group chatbots for

Messenger-based

MROCs

Gamified research:

Chatbot games and

market research integrations