Embed Size (px)
Citation preview
How to Increase Your Tes0ng Success by Combining Qualita0ve and Qualita0ve Data
Hiten Shah, KISSmetrics
Hiten Shah Co-‐Founder KISSmetrics @hnhshah
Hiten Shah Co-‐Founder KISSmetrics @hnhshah
There’s a big problem with A/B tes5ng!
You’ve tested
the color of
your bu?ons
You’ve watched your compe5tors
You’ve tried tes5ng everyone’s opinion
You’ve hit the limits of your current design
You’ve run experiments inspired by blog posts
But…
Only 1 out of 5 tests win.
4 out of 5 tests cost you money.
$
There are no magic bullets or pixie dust
You need to double your win rate.
Tools
5 tac5cal 5ps to double your win rate
1. Find out why people don’t convert 2. Use the exact words customers do 3. Start with a hypothesis then priori5ze 4. Create experiments that are small 5. Con5nuously experiment
Discover what informa5on is missing
Ask people when they try to leave
Email a survey
Ask people why they cancelled
Get feedback right when people cancel
1. Find out why people don’t convert 2. Use the exact words customers do 3. Start with a hypothesis then priori5ze 4. Create experiments that are small 5. Con5nuously experiment
Learn what frustrates them
Find out how they describe your product
Test very specific call to ac5ons (CTAs)
1. Find out why people don’t convert 2. Use the exact words customers do 3. Start with a hypothesis then priori5ze 4. Create experiments that are small 5. Con5nuously experiment
“A solid test hypothesis is an informed solution to a real problem – not an arbitrary guess. The more research and data you have to base your hypothesis on, the be!er it will be.”!— Michael Aagaard, h!p://kiss.ly/abthypo
Answer key ques5ons for each experiment
1. Which top metric are we trying to improve? 2. What can you anchor your metric against? 3. What is your hypothesis about why this metric is at x%? 4. What do we expect from this experiment? 5. What was the result and what did we learn from it?
hEp://kiss.ly/6qsab
1. Find out why people don’t convert 2. Use the exact words customers do 3. Start with a hypothesis then priori5ze 4. Create experiments that are small 5. Con5nuously experiment
Waste less code !
Small experiments help you focus on learning instead of wri0ng code.
Know what works !
Small experiments help you isolate and understand what impacts conversions.
Validate your ideas !
Small experiments help you validate assump0ons about your product.
1. Find out why people don’t convert 2. Use the exact words customers do 3. Start with a hypothesis then priori5ze 4. Create experiments that are small 5. Con5nuously experiment
Con5nuous Experimenta5on Small, measurable changes.
Keeps us honest. Prevents us from breaking things.
hEp://kiss.ly/etsycont
Dan McKinley @mcfunley
@etsy
One more thing...
Learn more… hEp://grow.kissmetrics.com/op0con
NEW! KISSmetrics
Customizable
A/B Tes5ng R
eport
THANK YOU
Hiten Shah @hnshah
Hiten Shah Co-‐Founder KISSmetrics @hnhshah
