Rise of the machine (learning algorithms)

Ris

e of

the

Mac

hine

Lea

rnin

g A

lgor

ithm

s

Rise of the Machine(learning algorithms)

data driven website optimization

Frank van LankveltSenior Big Data Engineer / Architect

Ris

e of

the

Mac

hine

Lea

rnin

g A

lgor

ithm

s

The B2B Customer Journey

discover compare

consider - business

consider - technical

buy

Ris

e of

the

Mac

hine

Lea

rnin

g A

lgor

ithm

s

Inbound / Advertising

discover

● SEA● Display● Affiliate● Social● Native

Ris

e of

the

Mac

hine

Lea

rnin

g A

lgor

ithm

s

Machine based Optimization

Ris

e of

the

Mac

hine

Lea

rnin

g A

lgor

ithm

s

SEA, Display

Ris

e of

the

Mac

hine

Lea

rnin

g A

lgor

ithm

s

Multi-armed Bandits

balancingExploitation

withExploration

Ris

e of

the

Mac

hine

Lea

rnin

g A

lgor

ithm

s

Bayes’ Theorem

Proposition A and evidence B,P(A), the priorP(A|B), the posteriorthe quotient P(B|A)/P(B) represents the support B provides

for A.

Ris

e of

the

Mac

hine

Lea

rnin

g A

lgor

ithm

s

Conversion rate Distribution

hit: 0miss: 0

hit: 10miss: 40

hit: 1miss: 4

hit: 100miss: 400

Ris

e of

the

Mac

hine

Lea

rnin

g A

lgor

ithm

s

With multiple options

Multiple distributionsA - the incumbentB - the challenger

How often should B be shown?

Thompson Sampling:sample distributionsshow variant with highest conversion rate Red: old configuration

Blue: new configuration

Ris

e of

the

Mac

hine

Lea

rnin

g A

lgor

ithm

s

Beyond the banner

discover compare

● A/B testing● content experiments

Ris

e of

the

Mac

hine

Lea

rnin

g A

lgor

ithm

s

Rinse & Repeat?

Ris

e of

the

Mac

hine

Lea

rnin

g A

lgor

ithm

s

Content Experiments - Setup

A B

Ris

e of

the

Mac

hine

Lea

rnin

g A

lgor

ithm

s

Content Experiments - Reporting

Experimentgoal / conversionconversion rateshistorical servings

Multi-armed / Contextual Bandit

Ris

e of

the

Mac

hine

Lea

rnin

g A

lgor

ithm

s

Train Model Visits

BanditModel

Content Experiments - Data flow

Site

Sessionize

Requestlog

Ris

e of

the

Mac

hine

Lea

rnin

g A

lgor

ithm

s

Personalizing

compare● targeting● personalization● recommendation● email marketing

consider - business

consider - technical

Ris

e of

the

Mac

hine

Lea

rnin

g A

lgor

ithm

s

Getting personal?

Ris

e of

the

Mac

hine

Lea

rnin

g A

lgor

ithm

s

Targeting & Personalization

Default

Amsterdam

Ris

e of

the

Mac

hine

Lea

rnin

g A

lgor

ithm

s

Behavioral Targeting

Use metadata / context on documents or visitor to personalize

Visitor looks mostly at clothing?show more clothing

Visitor looks mostly at shoes?show more shoes

I.e. use meta-data provided by the editor (a human) to augment the experience.

Based on rules, metadata => still much control for humans

Ris

e of

the

Mac

hine

Lea

rnin

g A

lgor

ithm

s

Audience Experiment

does audience configuration

improveconversion?

Ris

e of

the

Mac

hine

Lea

rnin

g A

lgor

ithm

s

Contextual bandit

Conversion rate depends on context:

x the contextw the weights𝚽 cdf of normal

dist.

Ris

e of

the

Mac

hine

Lea

rnin

g A

lgor

ithm

s

CTR prediction - under the hood

Ris

e of

the

Mac

hine

Lea

rnin

g A

lgor

ithm

s

CTR Context / Features

Ad featuresbid phrasesad titlead textlanding page URLlanding page itselfa hierarchy of advertiser, account,

campaign, ad group and ad

Query featuressearch keywordspossible algorithmic query expansioncleaning and stemming

Context featuresdisplay locationgeographic locationtimeuser datasearch history

Cardinalities varies● gender (2 values)● userID (billions)

x, w very large (sparse) vectors

Ris

e of

the

Mac

hine

Lea

rnin

g A

lgor

ithm

s

Persuasion Principles

Persuasion Principles

● Social Proof● Scarcity● Authority● Reciprocity● Commitment● Liking

Ris

e of

the

Mac

hine

Lea

rnin

g A

lgor

ithm

s

Persuasion API > In pictures

Ris

e of

the

Mac

hine

Lea

rnin

g A

lgor

ithm

s

Modeling the Visitor

Multi-armed bandit per visitor

● each principle gets an arm● model persuasion principle susceptibility

raise conversion by

10%

Authority

Ris

e of

the

Mac

hine

Lea

rnin

g A

lgor

ithm

s

The B2B Customer Journey

is this real?

or just fantasy?

Ris

e of

the

Mac

hine

Lea

rnin

g A

lgor

ithm

s

Work in Progress

Ris

e of

the

Mac

hine

Lea

rnin

g A

lgor

ithm

s

Mapping the Customer Journey

Man: behavioral targeting

annotate pages withpersona andphase

Machine: derived from data

cluster visitors - persona

cluster pages in visit - phase

do these agree?

Ris

e of

the

Mac

hine

Lea

rnin

g A

lgor

ithm

s

Log Likelihood Ratio - relatedness of pages

A not A

B x 20 - x 20

not B 40 - x 140 + x 180

40 160 200

LLR A, B

total # visitors

visitors of B

visitors of A

visitors of A & B

LLR as “weight” between vertices

Ris

e of

the

Mac

hine

Lea

rnin

g A

lgor

ithm

sonehippo.com pages

(inverse) distanceLLR

colorJourney phase(pink: no phase)Computer

says NO

Exploratory Analysis - Do man & machine agree?

Ris

e of

the

Mac

hine

Lea

rnin

g A

lgor

ithm

s

Summarizing

buy?

Algorithms to use in anger

● contextual / multi-armed banditparticularly in (realtime) reinforcement learning

● Log Likelihood Ratio

The Customer Journey is complex

● should it be left to man?

Ris

e of

the

Mac

hine

Lea

rnin

g A

lgor

ithm

s

Questions?