32
YOU’RE TESTING WHAT?! FOUR TYPES OF TESTS YOU SHOULD BE RUNNING MATT LACEY Head of Optimisation @matt_lacey12 17 TH SEPTEMBER 2013 PRWD Reveal MANCHESTER

You're testing what?! Four Types of Tests You Should be Running. Matt Lacey, Head of Optimisation at PRWD

  • View
    651

  • Download
    1

Embed Size (px)

DESCRIPTION

Matt Lacey Head of Optimisation at PRWD

Citation preview

Page 1: You're testing what?! Four Types of Tests You Should be Running. Matt Lacey, Head of Optimisation at PRWD

YOU’RE TESTING

WHAT?!

FOUR TYPES OF TESTS

YOU SHOULD BE

RUNNING

MATT LACEY

Head of Optimisation

@matt_lacey12

17TH SEPTEMBER 2013

PRWD Reveal

MANCHESTER

Page 2: You're testing what?! Four Types of Tests You Should be Running. Matt Lacey, Head of Optimisation at PRWD

@PRWD #PRWDREVEAL

What should you be testing?

Pricing & Price Framing

Proposition Testing

Optimising for Engagement

Optimising for Conversion

TALK OVERVIEW

Page 3: You're testing what?! Four Types of Tests You Should be Running. Matt Lacey, Head of Optimisation at PRWD

@PRWD #PRWDREVEAL

WHAT SHOULD YOU BE TESTING?

Page 4: You're testing what?! Four Types of Tests You Should be Running. Matt Lacey, Head of Optimisation at PRWD

@PRWD #PRWDREVEAL

WHAT’S

WRONG WITH

TESTING

BUTTON

COLOURS

Disclosure: We test buttons. Size, Prominence, Copy, Positioning and yes, Colour.

But:

There are likely to be constraints that limit the number of tests that you can run based on design & developer resource

In most situations we find that there are tests that can be run that will deliver more interesting results and more value

Page 5: You're testing what?! Four Types of Tests You Should be Running. Matt Lacey, Head of Optimisation at PRWD

@PRWD #PRWDREVEAL

What is the data telling me about visitor behaviour?

- Top landing pages- Pages with high traffic- Key leak or conversion points

What are users telling me about their experiences? What can we observe about their behaviour?

- Usability errors- Influential proposition messaging

What does the business know about customer struggles or business challenges?

- Customer service insights- Merchandising team

HOW TO

CHOOSE WHAT

TO TEST

Page 6: You're testing what?! Four Types of Tests You Should be Running. Matt Lacey, Head of Optimisation at PRWD

@PRWD #PRWDREVEAL

Triage- Just Do It – JDI- Single Feature Test- Batch Testing- Radical Redesign Testing- Larger Redesign projects

Triangulation- Multiple research methods confirm or strengthen the same issues

Potential Impact vs Effort Required- Likelihood to impact key metrics

vs

- Design/Development effort- Build time- Development schedule

HOW TO

PRIORITISE

YOUR TESTS

Page 7: You're testing what?! Four Types of Tests You Should be Running. Matt Lacey, Head of Optimisation at PRWD

@PRWD #PRWDREVEAL

1. PRICE & PRICE FRAMING

TESTS

Page 8: You're testing what?! Four Types of Tests You Should be Running. Matt Lacey, Head of Optimisation at PRWD

@PRWD #PRWDREVEAL

Through moderating or observing hundreds of user testing sessions we have often asked:

WHY TEST

PRICING?“If you could buy the same product from two different websites, what factors would be important in helping you choose which to buy from?”

User consistently respond with:

1. Price

2. Delivery ( price and timescales)

Page 9: You're testing what?! Four Types of Tests You Should be Running. Matt Lacey, Head of Optimisation at PRWD

@PRWD #PRWDREVEAL

1. Pricing should be based on value, not cost

2. Prices should be tangible

3. Prices should be comparable

4. To change your prices, you must reframe

5. Price differentiation is the key enabler of profit

6. Pricing communication shapes the customers perception of value

7. You must be prepared to lose some sales.

S o u rce: L ei g h Cal d wel l : T h e P s y ch o l o g y o f P ri ce: H o w t o u s e p ri ce t o i n creas e d em an d , p ro f i t an d cu s t o m er s at i s f act i o n

SEVEN

PRINCIPLES OF

PRICING

Page 10: You're testing what?! Four Types of Tests You Should be Running. Matt Lacey, Head of Optimisation at PRWD

@PRWD #PRWDREVEAL

Three Principles of Anchoring

1. The perception of price is often based on the context.

2. Decision making and the power of suggestion

3. Avoiding extremes

S o u rce: P ri ce In t el l i g en t l yhttp://blog.priceintelligently.com/blog/bid/181199/Price-Anchoring-to-Optimize-Your-Pricing-Strategy

TECHN IQUE:

PRICE

ANCHORING

Page 11: You're testing what?! Four Types of Tests You Should be Running. Matt Lacey, Head of Optimisation at PRWD

@PRWD #PRWDREVEAL

Four Options:

1. Control

2. Variation A – (SAVE £22)

3. Variation B – (SAVE 38%)

4. Variation C – remove ‘(cancel anytime)’ from monthly

WHICH TEST

WON?

12.5% overall increase in

subscriptions for Variation A (SAVE £22)

Page 12: You're testing what?! Four Types of Tests You Should be Running. Matt Lacey, Head of Optimisation at PRWD

@PRWD #PRWDREVEAL

Consider the range of price options available and test.

• This could apply to product or service costs, delivery costs, etc

Run tests to make price more transparent

• What do people get for their money• Test messaging that explains what

people are paying for.

Control comparisons

• Test using competitor prices for comparisons

PRICE & PRICE

FRAMING

TAKEAWAYS

Page 13: You're testing what?! Four Types of Tests You Should be Running. Matt Lacey, Head of Optimisation at PRWD

@PRWD #PRWDREVEAL

2. PROPOSITION TESTING

Page 14: You're testing what?! Four Types of Tests You Should be Running. Matt Lacey, Head of Optimisation at PRWD

@PRWD #PRWDREVEAL

Watch clip: http://www.youtube.com/watch?feature =p layer_embedded&v =ps7bB9-ksMY

The second a visitor comes to you they have in mind: “Why should I buy from you versus all your competitors?”

You need to display clear unique value propositions or unique campaign propositions.

Everyone who is successful in online conversion is doing this. If you’re not, you’re wasting your time.

Bryan Eisenberg, Godfather of CRO

Page 15: You're testing what?! Four Types of Tests You Should be Running. Matt Lacey, Head of Optimisation at PRWD

@PRWD #PRWDREVEAL

BEFORE

AFTER

4.35% increase in CR

This equated to a six-figure increase in revenue.

Page 16: You're testing what?! Four Types of Tests You Should be Running. Matt Lacey, Head of Optimisation at PRWD

@PRWD #PRWDREVEAL

GET REVISING

SIGN-UP TABLE

Price doesn’t put me off, although the basic package is more than enough for me.

I think the site has already got everything I need.

Moderated User testing participants

Page 17: You're testing what?! Four Types of Tests You Should be Running. Matt Lacey, Head of Optimisation at PRWD

@PRWD #PRWDREVEAL

Page 18: You're testing what?! Four Types of Tests You Should be Running. Matt Lacey, Head of Optimisation at PRWD

@PRWD #PRWDREVEAL

GET REVISING

SIGN-UP TABLE

185% increase in premium subscriptions

with a small drop in free subscription. This was seen to be a good trade-off.

You have more features held back to encourage people to subscribe at a later date.

Page 19: You're testing what?! Four Types of Tests You Should be Running. Matt Lacey, Head of Optimisation at PRWD

@PRWD #PRWDREVEAL

Test what features/functionality is given for free and what is paid for

Test how your proposition is displayed/framed on your site

Test offering different thresholds for free delivery

Test introducing urgency to the process

PROPOSITION

TESTING

TAKEAWAYS

Page 20: You're testing what?! Four Types of Tests You Should be Running. Matt Lacey, Head of Optimisation at PRWD

@PRWD #PRWDREVEAL

3 RADICAL REDESIGNS

OPTIMISING FOR ENGAGEMENT

Page 21: You're testing what?! Four Types of Tests You Should be Running. Matt Lacey, Head of Optimisation at PRWD

@PRWD #PRWDREVEAL

This opens up the top of the conversion funnel.

This is likely to be the point where you have the largest number of potential test participants.

Search for landing or key site pages with high bounce rates or those that rarely lead to conversions

Key Areas to Test

Homepage

Top landing pages

OPTIMISING FOR

ENGAGEMENT

Page 22: You're testing what?! Four Types of Tests You Should be Running. Matt Lacey, Head of Optimisation at PRWD

@PRWD #PRWDREVEAL

BEFORE AFTER

10.2% performance

improvement in booking conversion rate

Page 23: You're testing what?! Four Types of Tests You Should be Running. Matt Lacey, Head of Optimisation at PRWD

@PRWD #PRWDREVEAL

BEFORE AFTER

10% improvement in CR

Page 24: You're testing what?! Four Types of Tests You Should be Running. Matt Lacey, Head of Optimisation at PRWD

@PRWD #PRWDREVEAL

BEFORE AFTER

6.8% improvement in CR

Page 25: You're testing what?! Four Types of Tests You Should be Running. Matt Lacey, Head of Optimisation at PRWD

@PRWD #PRWDREVEAL

Let users know that they have landed in the right place

Make sure that pages have a clear focus and visual hierarchy. Users need to know where to begin.

Test both simplifying the experience and adding in more detail

A smaller amount of highly qualified visitors can trump large amounts of un-qualified visitors and convert better

ENGAGEMENT

OPTIMISATION

TAKEAWAYS

Page 26: You're testing what?! Four Types of Tests You Should be Running. Matt Lacey, Head of Optimisation at PRWD

@PRWD #PRWDREVEAL

4 RADICAL REDESIGNS

OPTIMISING FOR CONVERSION

Page 27: You're testing what?! Four Types of Tests You Should be Running. Matt Lacey, Head of Optimisation at PRWD

@PRWD #PRWDREVEAL

Testing towards the end of your funnel helps to convert users who are closest to converting and tend to have higher intent.

Improvements here opens up the bottom of the conversion funnel.

Simplifying and removing issues that slow momentum is crucial.

Key Areas to Test

Checkout (login, delivery, payment)

Contact/Sign-up forms

OPTIMISING FOR

CONVERSIONS

Page 28: You're testing what?! Four Types of Tests You Should be Running. Matt Lacey, Head of Optimisation at PRWD

@PRWD #PRWDREVEAL

BEFORE AFTER

72% improvement in

Registrations from the homepage

Page 29: You're testing what?! Four Types of Tests You Should be Running. Matt Lacey, Head of Optimisation at PRWD

@PRWD #PRWDREVEAL

BEFORE AFTER

7.4% CR improvement from

sign-up page

Page 30: You're testing what?! Four Types of Tests You Should be Running. Matt Lacey, Head of Optimisation at PRWD

@PRWD #PRWDREVEAL

Don’t be afraid to offer visitors an opportunity to convert at an early stage

Use persuasive techniques to pull users through (not push!)

Simplify forms to the bare minimum fields required

Reduce errors. There’s a huge correlation between users getting an error and abandoning the funnel

Provide key supporting proposition messaging throughout the process

Re-enforce positive messages (e.g. security in checkout)

CONVERSION

OPTIMISATION

TAKEAWAYS

Page 31: You're testing what?! Four Types of Tests You Should be Running. Matt Lacey, Head of Optimisation at PRWD

@PRWD #PRWDREVEAL

With the right kind of research and preparation the odds are stacked in your favour.

Therefore:

Be data driven. Be bold. High risk, high reward.

FINAL

TAKEAWAY

Page 32: You're testing what?! Four Types of Tests You Should be Running. Matt Lacey, Head of Optimisation at PRWD

@PRWD #PRWDREVEAL

Are there any questions?

THANKS FOR

LISTENING

MATT LACEY

Head of Optimisation

@matt_lacey12

17TH SEPTEMBER 2013

PRWD Reveal

MANCHESTER