Upload
vwo
View
146
Download
0
Embed Size (px)
Citation preview
Scaling your Testing Program
Vivek VilatiaSenior Product Manager, VWO
Anshul GurbaxaniAssociate Product Manager, VWO
Prem KujurCustomer Success Manager, VWO
What is A/B testing
In the most simplest form, A/B
Testing is comparing two versions
of a web page to see which version
performs better for a defined
conversion goal
Type of Tests
A/B TestChange any element or section
of a webpage and test both
versions.
Split URL TestTest performances of separate
URLs if the landing pages are
significantly different.
Multivariate TestRun tests on multiple variation
combinations together. Needs
sufficient traffic.
Simple A/B tests Advanced A/B tests
➔ Great starting point
➔ Smaller teams
➔ Easy to setup
➔ Quick wins
➔ Test changes like copy,
arrangement of elements,
CTA colors etc
➔ Test advanced use cases with
greater impact
➔ Conversion optimization program
➔ Self-sufficient team
➔ The right tool-set
➔ Management buy-in
➔ Tons of perseverance
Test pages | Defining a set of similar pages for your test
Asterix * in the URL matches pattern would replace the variable part of the URL
Represented by * in the URL matches pattern
Audience | Complex targeting rules
Visitors logged in on
your website
Are either from India or have their
preferred currency as INR
Run your test for at least a week.
Along with a primary goal, have related secondary goals.
Ensure that your primary goal represents the direct and
exclusive impact of the changes being tested.
Making sense of your A/B testing reports - Best Practices
Slice and Dice your test results to learn the trends and
behaviour of various user segments.
Testing radical changes at times might just do the trick
Identify micro-conversions and important correlative metrics
increase in a given variation, implement it.
Inconclusive Tests
Ensure that you don’t use segments with very less data.
Make sure there is no error in the test setup.
Ensure that you are not running multiple tests
simultaneously in the same part of the journey you are A/B
testing
Inconclusive Tests - Pitfalls