Personalization Strategy WorkshopLos Angeles

August 18, 2016

Travis Bryant, Vice President of Sales, Optimizely Ian Thiel, Head of New Products, OptimizelyMatty Wishnow, Founder & CEO, Clearhead

+

Agenda

2:30 PM Registration & Refreshments

3:00 PM Open/WelcomeTravis Bryant, Vice President of Sales, Optimizely

3:15 PM Personalization Capabilities Overview & Demo Ian Thiel, Head of New Products, Optimizely

3:45 PM Break & Networking

4:00 PM Personalization Strategy Best PracticesMatty Wishnow, Founder & CEO, Clearhead

5:00 PM Group Discussion, Networking & Happy Hour

Welcome and Opening Remarks

Travis Bryant, Vice President of Sales, Optimizely

Travis Bryant

Vice President of Sales Optimizely

Personalization Strategy & Demo

Ian Thiel, Head of New Products, Optimizely

Ian Thiel, Head of New Products,

Optimizely

What are my peers doing?2016

Behavioral retargeting produces 4x ROI

Personalized campaign

Used behavioral targeting to show buy-one-get-one tire promotion to customers who had viewed tires.

Results

• 21% increase in overall site sales• Sold out of tire stock 2 months before expected• 4x return on investment in Optimizely and solutions

partner services

PaywallsTargeted offersAccount Based Marketing (ABM)Product recommendationsCategory affinity

What are my peers doing?

Recap

Demo!2016

How does Optimizely personalize?

Personalization Strategy Workshop

Matty Wishnow, Founder & CEO, Clearhead

Matty Wishnow Founder & CEO

Clearhead

August

solving problems with personalization

an introduction to PSM

2

problem / solution mapping A unifying framework for continuously optimizing around a common set of goals, problems & solutions, researched & validated with data.

1

2

3

goals

PS

Ps

pS

ps

problems

1

2

3

4

hypothesesprioritized by data

1

2

3

4

experiments

validated by dataoutcomes

UX

product

merchandising

marketing

3

it’s still an experiment

everything is an experiment

promotions + campaigns

merchandising changes

new features & functions

UX + IA changes

new SaaS additions

personalized experiences

everything

if everything is an experiment,

then these are questions to live by

What problems are the changes solving?

1

How will you know if the change was successful?

2

4

what is personalization?The customization, targeting or adaptation of content and/or experiences for end users based on implicit or explicit attributes of more refined segments.

what is a/b testing?A method of comparing a variation to a control to determine if the differences observed in the sample are statistically likely to survive in a larger, general population or data set.

1

2

3

a

b

ab testing to segmentation segmentation to a/b testing

a

b

1

2

a

b

5

sizing the prize

large segment

effort

oppo

rtun

ity

small segment

algorithmic manual

target smart phone users with ability to send cart to

their desktop

people who bought this long tail item also

bought this

target un-authenticated people who come from pinterest with an email

opt-in modal

6

“A good archer is known not by his arrows but by his aim.” -Thomas Fuller

7

increase mobile conversion rate by 10% by the end of 2016.

ask yourself: • target set? • clearly understood? • time based? • realistic?

reduce mobile bounce rate by 15% by the end of 2016.

increase mobile revenue per visit by 5% by the end of q3 2016.

improve mobile net promoter score by 10% by the end of q1 2017.

goals

8

“If I had one hour to save the world, I would spend fifty-five minutes defining the problem… and only five minutes finding the solution.” -Albert Einstein

9

increase mobile conversion rate by 10% by the end of 2016.

problems

ask yourself: • is it a root problem? • who does it impact? • where and when

does it impact them? • how do you know it

is a problem?

users find it hard to click on our filter & facet functionality on their smart phone.

it is challenging for users to look at alternative product shots on our mobile PDP because the thumbnails are so tiny.

we frustrate mobile phone users with two extra steps — interstitial cart and account options — before getting them to checkout.

goal

10

“Determine the thing that can and shall be done, and then we shall find the way.” -Abraham Lincoln

11

solution hypotheses

ask yourself: • I believe that… • If I am right then… • could a designer/

developer/analyst reasonably begin work based on the hypothesis?

I believe that if we skip the interstitial cart page for smart phone users and redirect them to checkout once they add something to their cart, they will be less likely to waver in their journey and bounce. If I am right, then, mobile conversion rates for for smart phone users will increase by 5%.

I believe that if we eliminate the “sign up” option at the beginning of check-out for all unauthenticated users on smart phones, they will be less intimidated by the prospects of filling out extra form fields and will be more likely to purchase. If I am right, then mobile conversion rates for for smart phone users will increase by 10%.

we frustrate mobile phone users with two extra steps — interstitial cart and account

options — before getting them to checkout.

problem

Consistently require all changes to be mapped to problems & hypotheses.

Determine if additional problem research is needed.

hypotheses experiments outcomesproblemsgoals

Determine what should be researched or tested vs “just do it”

Review & iterate regularly.

2

4

5

3

Agree to the “Problems Worth Solving.”

1

what do you do once you’ve got your map?

goals problems hypotheses outcomes

13

define clear success metrics, that include targets, time basis & exceptions reporting.

1

capture business & end user problems.2

organize all problems into clusters.3

research & prioritize problems.4

agree how much to invest in “problems we want to solve.”

5

develop solution hypotheses for “problems we want to solve.”6

prioritize solution hypotheses.7

ux, product & stakeholders plan for feedback & experiments.

8

solutions are tested at various stages.9

experiments

analysis & data stories around specific solution hypotheses.10

ongoing measurement of KPIs related to goals.

11

qualitative tracking of “problems worth solving”

12

expand investment in those solutions that worked!

13

move on to new hypotheses if old ones proved invalid.

14

use data to regularly re-align around problems & solution hypotheses.

15

mapping details

14

segments: let’s get real

where should we start?

how do you get from qualitative personas to definable

data segments?

is predictive segmentation a real thing?

anybody doing amazingomni-channel personalization?

for problem research

for hypothesis development

should we just randomly explore segments?

15

When you are personalizing, you are still experimenting.

Don’t personalize just because you can. Endeavor to solve problems that relate to critical goals.

Key considerations for personalization • Segment size • Segment value • Manual v algorithmic

key takeaways

Segment definition and exploration takes time. There’s no magic button (yet).

Experiments come with risk and investment.

Multi-channel customer data layers are increasingly a practical reality!

1

2

3

4

5

6

Thanks!

+