View
218
Download
0
Category
Preview:
Citation preview
#thinkppcHOSTED BY:
The 11 13 Biggest A/B Testing Mistakes and How To
Avoid Them
Featuring Peep Laja
#thinkppc
Presenters
• Peep Laja– Guest Presenter– Founder of Conversion XL
– @peeplaja
• Roopa Carpenter– Moderator– CRO Manager at Hanapin Marketing
– @roopacarpenter
#thinkppc
Join the conversation
• Include the hashtag #thinkppc in your Twitter tweets.
Or use the webinar question box to send us questions.
#thinkppc
Live Poll Question #1
How long have you been doing CRO?#thinkppc
A. Less than 1 yearB. 1-3 yearsC. 3-5 yearsD. 5+ years
#thinkppc
Live Poll Question #2
How do you manage your account(s)?#thinkppc
a) I’m part of an in-house PPC team.b) I do all the marketing myself.c) I work with an agency.d) I’m a consultant.
#thinkppcPeep Laja, ConversionXL @peeplaja
Typical scenario: A company runs 100 tests / year. Lots of
wins. But no difference on the bank account.
#thinkppc
Precious time wasted on stupid
tests
Testing mistake #1:
#thinkppc
You don’t have a data-driven,
learning-oriented hypothesis
Testing mistake #2:
#thinkppc
Learn more: http://conversionxl.com/how-to-come-up-with-more-winning-tests-using-data/
#thinkppc
You think you know what will
work
Testing mistake #3:
#thinkppc
Your sample size is way too small
Testing mistake #4:
#thinkppc
#thinkppc
Calculate needed sample size in advance:
#thinkppc
I won’t even look at results until there’s at least 250-350
conversions PER variation.
Way more if you want to segment the data
#thinkppc
Your tests don’t run long enough
Testing mistake #5:
#thinkppc
#thinkppc
You don’t test full weeks at a time
Testing mistake #6:
#thinkppc
#thinkppc
You run tests when you have no
traffic
Testing mistake #7:
#thinkppc
•Learn more: http://conversionxl.com/how-to-do-conversion-optimization-with-very-little-traffic/
#thinkppc
Test data is not sent to third-party
analytics
Testing mistake #8:
#thinkppc
#thinkppc
(function(){ var experimentID = 2809164105; window.optimizely = window.optimizely || [];
if (typeof window.optimizely.data.experiments[experimentID] !== 'undefined') { var experimentName = window.optimizely.data.experiments[experimentID].name; var variationName = window.optimizely.data.state.variationNamesMap[experimentID]; window.ga = window.ga || function() {(window.ga.q = window.ga.q || []).push(arguments);}; window.ga.l =+ new Date(); window.ga('send', 'event', 'Optimizely', experimentID + ' ' + experimentName, variationName, {'nonInteraction': 1}); } })();
#thinkppcPeep Laja, ConversionXL @peeplaja
#thinkppc
You give up after your first test for
a hypothesis fails.
Testing mistake #9:
#thinkppc
#thinkppc
Full case study: http://conversionxl.com/case-study-how-we-improved-landing-page-conversion/
79.3% uplift in conversions
#thinkppc
#thinkppc
#thinkppc
#thinkppc
#thinkppc
#thinkppc
#thinkppc
You’re not aware of validity threats
Testing mistake #10:
#thinkppc
History effectInstrumentation
effectSelection effect
#thinkppc
You’re ignoring small gains
Testing mistake #11:
#thinkppc
5% monthly increase in your conversion rate will result in 80% uplift for the year. That’s how
math works.
#thinkppc
You copy competitors /
leaders / other people’s tests
Testing mistake #12:
#thinkppc
Which Test Won?
#thinkppc
WhichTestWon?
WhichTestWasNotRunProperly?
#thinkppc
Somebody else’s test results are
useless
#thinkppc
“Site shows stats from various A/B tests - Finally I've got evidence to show clients on a load of design decisions!”
#thinkppc
#thinkppc
The process, the method, the
analysis
#thinkppc
You’re not running tests at all times
Testing mistake #13:
#thinkppc
PPC Retainers
Need some assistance for your PPC? We offer on-going and one-time retainers.
Learn More:
http://www.hanapinmarketing.com/ppc-retainers/
#thinkppc
Live Q&A Time!
#thinkppc
Have more questions?
Thank you for attending our webinar! #thinkppc
• Contact us Directly:
» Hanapin Feedback: marketing@hanapinmarketing.com
Recommended