Learn from failed A/B tests to create winning results

Profitable ‘A-ha!’ moments everyday.™

THE CONNECTED CONVERSION OPTIMIZATION PLATFORM

Keep up with the new platform at vwo.com/evolution

© 2007-2017 WiderFunnel Marketing Inc.

Tweet this: @chrisgoward #Awesome

Learn From Failed A/B Tests to Create Winning Results

How to Turn Losses into Wins



Before we get started. . .

• This presentation deck and recording will be available to you after the webinar

• Use the General Chat panel to ask questions or make comments in real time

• Stay to the end for the chance to win a free digital copy of You Should Test That! Type question here


Tweet to: @chrisgoward #Awesome

One of you will win the book today.

Get a free chapter at: YouShouldTestThat.com



• Developed the LIFT Model®, PIE Framework, and Infinity Optimization Process™

• Keynotes at conferences and seminars around the world

• Author of You Should Test That!Get a free chapter at: YouShouldTestThat.com

Chris GowardFounder & CEO, WiderFunnel



23.4% more orders for IBM Softlayer servers



40.5% more ink subscriptions for HP Instant Ink



115% more qualified leads for Magento Enterprise



210% lead generation lift for Equity Trust



$1 million per month additional revenue for BuildDirect.com





But not all tests are winners, right?







Result: Both variations lost.

Var A Var B





But! The experimental design delivered valuable insights.



Var A Var B



Clarity: Shoppers want to refine their results by color.

..but, inconsistent use of “color” and “palette”creates cognitive load.



There are no losing experiments......if you use the right experiment process.



?



?

A B C





?

A B C

15.6% Lift 9.8% Lift23.6% Lift



Great results aren’t random

The right process makes magic happen.



Infinity Optimization Process



A B C ê 7.7%é 0.6%



A B C



A B C é 14.1%é 8.9%

?



ValidateDesign of Experiments is used to maximize the growth and insights results from experiments.



Your challenge: Balance results and insights



Unless you have infinite traffic...You can't isolate every variable - tests will take too long.

You can't only test one variation - you'll learn nothing.

You can find the ideal balance with Design of Experiments.



The results and insights spectrum

Results Insights

A/B Multivariate



The results and insights spectrum

Results Insights

A/B MultivariateFractional factorial



Why is this important?



It allows you to gain important insights.



Use Factorial Testing to understand your customers’ psychology







Infinity Optimization Process



Explore

Web analytics – done right – can feed insights into user experience hypotheses.



Conversion Rate: -2.23%Revenue per visitor: -2.95%





Conversion Rate: -9.97%Average Donation: +1.57%Revenue per visitor: -8.55%



Ikea effect

Consumers place a

disproportionately high value on

products they partially create.

Could interacting with the slider

create a sense of ownership of

the donation?



Explore

Your visitors convert in their minds before they convert with their click.



#AlternativeFacts



Loss Aversion

Studies have shown that losing

an amount is twice as

psychologically powerful as

gaining the same amount.

- Amos Tversky and Daniel Kahneman



Conversion Rate: +2.04%Average Donation: +5.94%Revenue per visitor: +8.01%



Consistency Principle

We like to keep consistent with

what we think, say and do, and will

change to ensure this is so.

- Cialdini, Influence.







Conversion Rate: +1.97%Average donation value: +3.07%Revenue per visitor: +5.10%



SelectedRegular donor

Selected Occasional

donor

Selected First-time

donor

Did not select an option

14% of visitors68% of donors

86% of visitors32% of donors



Now, we’re building on these insights to deliver Personalized data-informed experiences.



Q&A àOr email: [email protected]



Want new ideas to spark your testing program?

Download the new graphical cheat-sheet with 17 new A/B test ideas:

WiderFunnel.com/17ideas

Email your questions to: [email protected]



Thank You!You will receive a recording of the webinar

in the next few days.

Profitable ‘A-ha!’ moments everyday.™