Keynote Ton Wesseling - How to apply analytics within A/B-testing to maximize your growth

H o w t o a p p l y a n a l y t i c s w i t h i n A / B - t e s t i n g t o m a x i m i z e y o u r g r o w t h

C o n v e r s i o n C o n f e r e n c e L o n d o n , O c t o b e r 3 0 t h 2 0 1 4 To n W e s s e l i n g – C E O – Te s t i n g . A g e n c y

MAXIMIZE YOUR GROWTH

Ton Wesseling | @ TonW Maximize your growth #convcon

Ton Wessel ing | @ TonW

Growth Hacker

Optimization Lunatic

Knowledge Lover


Ton Wessel ing - @tonw

ü  More testing ü  Quality Assurance ü  Lower pricing


Onl ine Exper iments – A/B-test ing

Winner!"

Variation B"

Variation A"

Default"


Hyper l ink junky


Digging through log fi les


Trying to improve


Over and over again


That is:

0 / 1 Testing


But i t should be l ike:


Scient ific approach

A / B Testing Live Online Research


Exper iments

“If you double the number of experiments you do per year

you’re going to double your inventiveness”

Jeff Bezos, CEO Amazon – 2004!


Over and over again


Growth l ike Amazon G

erm

an E

com

mer

ce m

arke

t, th

ank

you

Andr

é M

orys


Growth


Start-up wor ld


They don’t have the t ime to do this:


Invest ing in Start-ups


Grow fast


Or…


Speed up – more exper iments


Speed up – more people


I t takes many exper iments to win


Why is your visitor here?


How can you

help your visitor

to fullfil his needs?


I t ’s not about conversion opt imizat ion

Growth is about learning how to serve your v is i tors!


Automate the non learning stuff !


Test ing costs t ime

Use it to LEARN


FA C T & A C T – 7 s t e p s t o l e a r n

&

Tell Conclude Analyze

Test Create Analyze Find


And how we use Analyt ics


FA C T & A C T

&


Test Create Analyze Find Find


Set up your user journeys


Create segments to report on journeys


Translate them to funnels


Entry / Exi t surveys


Event tracking

Events

Pageviews

Sessions

User User

Session

Pageview Pageview

Session

Pageview

Event Event


Compare segments


Now you’ve got specific spots to opt imize


FA C T & A C T

&


Test Create Analyze Find Analyze


Enough traffic & conversions?


Conversions v ia your opt imizat ion spot


ABtestguide.com/calc


Low significance levels

ü Big chance the winner is not a real winner


Low power levels

ü Big chance a real winner is not recognized


Tested percentage is a spread!


Example test-setup


Average power: 40%


So:


I f there is enough traffic to test


On page behaviour


More detai ls needed?


Event flow report ing


To have a clear v is ion what’s going on


Combine th is with persuasion knowledge

Wheel-of-Persuasion.com


FA C T & A C T

&


Test Create Analyze Find Test Create


Now we use the A/B-tst ing software


Front end development


We measure in Google Analyt ics


They do have analyt ics – but not enough


I t a lso keeps you away from this fa i lure


Event tracking var iat ion behaviour


We can also use Google Tag Manager


FA C T & A C T

&



Analyze


Another error warning:

ü You just go and dig unitil you find something (you will always find something that seems true BUT is not!)


Analyze your fixed test length:

ü  With up front stated segments, conversion points and significance level

ü Custom reports

ü Google Analytics API

ü Excel plugins


Not s ignificant

ü It’s not a winner, stop

ü But, it’s does not mean it’s a loser!

ü Average: 30% of your tests will have winners


FA C T & A C T

&

Tell Combine Analyze


Tell Conclude


Conclude

ü Do nothing? or

ü Re-test? or

ü Implement? or

ü Scale? and

ü Continue or change test roadmap


Knowledge l ibrary!


Tel l !

I t ’s about learning how to serve your v is i tors!


Exper iments

“If you double the number of experiments you do per year

you’re going to double your inventiveness”

Jeff Bezos, CEO Amazon – 2004!


DIA

Data Insights Action


Without act ion, no growth


Be l ike Bi l l , and win!


Happy test ing! G

erm

an E

com

mer

ce m

arke

t, th

ank

you

Andr

é M

orys


Tw i t t e r : To n W L i n k e d i n : To n W e s s e l i n g

E m a i l : To n . W e s s e l i n g @ Te s t i n g . A g e n c y

SLIDES - #convcon

Data & Analytics

Keynote Ton Wesseling - How to apply analytics within A/B-testing to maximize your growth