Drastically Improve Your Users’ Online Experiences With A/B Testing

HOSTED BY: &

Your Presenters

▸ Samantha Kerr• CRO Manager at

Hanapin Marketing

• PPC Hero Blogger

• @samantha__kerr

▸ Kate Wilcox• CRO Coordinator

at Hanapin Marketing

• PPC Hero Blogger• @katewilcoxppc

▸ Duane Brown• Performance

Marketer at Unbounce

• Hero Conf Speaker• @duanebrown

#thinkppc

Join the conversation

#thinkppc

Include the hashtag #thinkppc in your Twitter tweets.

Or use the webinar question box to send us questions.

#thinkppc

Who is Hanapin?• Run the world’s most popular PPC

blog and conference.• We manage and optimize global paid

search, paid social, and display programs.

• Within 12 months, brands can expect a 250% increase in their growth trajectory.

#thinkppc

Our Clients

Live Poll question #1

#thinkppc

How long have you been in PPC?#thinkppc

A.Less than 1 yearB.1-3 yearsC.3-5 yearsD.5+ years

Live Poll question #1

#thinkppc

How long have you been doing CRO?#thinkppc

A.Less than 1 yearB.1-3 yearsC.3-5 yearsD.5+ years

#thinkppc

Our job is to make the customer experience better. An increased CR is a by product in the end.

Why Do We Test…

#thinkppc

#thinkppc

Why You Would Do Testing➔ Increase performance

➔ Customer insights

➔ Rotate offers (remove banner blindness)

There is always room for improvement…. If you’ve enough traffic to run tests.

#thinkppc

Why You Would Not Do Any Testing

➔ Risk: You don’t know if you’ll do better or worse.

➔ You don’t have enough traffic

There is no reward if you don’t take any kind of risk.

A/B Testing

#thinkppc

#thinkppc

What to A/B Test?How do you capitalize on that –Hypothesis:

• In science, a hypothesis is an idea or explanation that you then test through study and experimentation.

• What do you think will happen?• …okay to be wrong.

#thinkppc

Define SuccessKnow what the end looks like:

• Once a test is live. Netflix starts tracking success metrics:

Streaming hours AND Retention

• Increased sales• Increased VOV• More downloads• Better Lead

#thinkppc

Let data lead the way. Don’t let a HIPPO dictate the path to success.

#thinkppc

Opportunity• Sample Size

• Size of Opportunity

• Confidence level: 95%?• Margin of error• Run test for 2-3 weeks at

least

• Control vs. Treatment group

#thinkppc

A/B Tests 201• Changing a button or colours

• Can you? Sure. Should you?

• Small changes vs. large changes

• Radical different designs helps drive faster change as you can see what works

#thinkppc

Better Experiences• Create better experiences for

your customers

#thinkppc

Old Design

New Design

Conversion Rate2.3 % VS 8.7%

#thinkppc

New Design

Using Eisenberg

#thinkppc

#thinkppc

Eisenberg Hierarchy of Optimization

#thinkppc

Does it work?• Elements with no action

• Goal accomplishment

Fix it, don’t test it.

#thinkppc

Can everyone reach the site?

• 404 errors

• Age

• Device

16% CR lift

#thinkppc

Can users understand how to use the site?

• Navigation

• Stereotype elements

57.09% lift in element engagement

#thinkppc

Does it flow?• Narrative

• Language

• “Keep it simple stupid”

• Answer questions

• Build confidence

92.78% CR lift120.32% lift in call

rate

#thinkppc

Is the site motivating?• Motivation

• Psychology

• Messaging & imagery 76% lift page engagement

120% lift in download rate

Tips & Tricks

#thinkppc

#thinkppc

Best Practices Are Guidelines.• And “hacks” should be tested. Always.

• 17.82% drop in call rate• 5.05% drop in form submission

rate

#thinkppc

Hotjar + Customer Form• Finding bugs

• Filling out forms

#thinkppc

Never forget your users.• Conversions don’t exist without human

beings.

• Data can lead you in the right direction, but you have to remember that the data was created by people.

• “But, why?”

#thinkppc

Keep it simple.• When A/B testing – only change one thing

>1 change = no validity or causationBad juju

#thinkppc

But sometime complex is fun.• Multivariate tests can combine your efforts.• Hello efficiency!• Need traffic…

#thinkppc

A/B Testing Tips• Be data informed; not gut or

personal opinion.• Netflix = hours of play &

retention• Unbounce = customer signups

• Don’t test too much at once. Dirty Data.

#thinkppc

HomeworkNetflix: https://channels.theinnovationenterprise.com/presentations/quasi-experimentation-at-netflix-beyond-a-b-testing

www.slideshare.net/xamat/netflix-recommendations-beyond-the-5-stars

Would you like an offer?

#thinkppc

A. Yes! I'd like a copy of Unbounce's new e-book “The 7 Principles of Conversion-Centered Design"

B. Get an account analysis by the experts at Hanapin (For accounts spending 15k+/month)

C. No Thanks!

Live Q&A Time!

#thinkppc

Contact us Directly:

Hanapin Feedback: marketing@hanapinmarketing.com

Unbounce Feedback: info@unbounce.com

#thinkppc

Thank you for attending our webinar!