Rightio - 2016
Rightio Landing Page Experimentation
A quick recap of our original goals and objectives
Introduction
User Interaction
How are users interacting with the current landing pages?
Bounce rate
What is/could be causing high Bounce rates on certain pages?
Visual Differentiation
Will visual differentiation of landing pages create higher conversion
rates?
Key Areas of Focus
Agreed & defined landing page requirements for 383 to investigate, explore & understand
21 3
What we discovered from the existing site?
Site Analysis
Existing Interactions
7% of users check their area with the Coverage map.
15% Of mobile users click on a
phone call to action
1 minute Users tend to spend just under one minute on site
and view less than 2 pages
Bounce RateRightio asked us to examine bounce rate, which prior to our changes, stood at 76%.
We then implemented click to call event tracking on the site and found that bounce rate dropped by 6%, to 70%
This led us to surmise that users are engaging with the landing pages, but their conversion is taking place offline (picking up the phone and calling).
On mobile, bounce rate dropped by 13%, from 84% to 71% - which is where click to call event tracking would predominately be triggered.
Bounce rate is actually therefore much lower than the recorded figure in Google Analytics. Similarly, as the sites are geared towards lead generation, bounce rate shouldn’t be a key metric for Rightio.
Location Selection
4 geographically & demographically varied regions, specifically chosen to maximise
learnings
Our Regions
London Predominately Mosaic Profile A, Rightio’s customer-base here are career-driven, time-sensitive individuals.
Swansea With no clear Mosaic profile emerging from Rightio’s Swansea customer base, this audience is truly mixed.
Leeds Predominately Mosaic Profile O & J, these individuals are price-sensitive.
2
1
4
3
Brighton Rightio’s customer base in Brighton is both Mosaic Profile A & O.
Landing page variation creation
Creation of 4 very different landing page variants, each with a different core focus on key Rightio
proposition attributes
Our Variants
Trust We dialled up trust, highlighting how Rightio is a reliable service.
Price We showed pricing for typical Rightio services, by listing the typical price for common plumbing jobs.
Speed We focused on Rightio as a speedy service for time-sensitive individuals.
2
1
4
3
Wildcard We dialled up all of the key messaging from the previous variants to see if users respond to a mix of messages.
Trust This landing page variant promoted Rightio as a trustworthy, reliable service.
Speed This landing page variant highlighted Rightio’s speedy service, perfect for time-sensitive individuals.
Price This landing page variant provided pricing details about typical Rightio jobs.
Wildcard This landing page variant contained a combination of messages, as well as a new layout.
So why the Wildcard?
Layout
The Wildcard allowed us to test a completely
different layout.
1
Significance
We hypothesised that if the other variants didn’t
yield significant results, a different layout would.
3
Messaging The Wildcard allowed us to combine messaging
for regions with no clear Mosaic profile.
2
Secondary Data
We would be able to understand if phone
number positioning was leading conversion, via
Crazy Egg.
4
Extra Learnings
Testing a Wildcard provided Rightio with
further analysis on how a different layout could influence conversion.
5
Our testing toolkit
Optimizely Leading customer experience optimisation software, which allowed us to manage traffic allocation and integrate with other tools
Crazy Egg A heat map provider which allows us to understand where users click
Google Analytics Using custom dimensions we were able to easily monitor traffic and on-page performance for each variation
Inspectlet Session recording software which allows us to view recordings of user activity
Google Analytics
Session and on site activity data
Testing methodology
Our approach to testing and reporting
1 Month
Testing ran for 4.5 weeks (1 calendar month)
Call Stats
Reviewing inbound calls to validate conversion
2 tailed testing
2 tailed testing to validate statistical
significance
Metric Definitions
Calls to Booking The percentage of calls
which result in a booking
Visits to Calls The percentage of
sessions which result in a call.
Why Calls to Bookings Matters
Our primary goal is to increase LP Visits > calls, however, we also need to consider the quality and
qualification of leads generated through the messaging presented. At present, Calls to Bookings is
the only metric that can validate the quality of the leads we are generating.
Why Calls to Bookings Matters cont.
We appreciate and understand the level of coverage in any given location can significantly impact final Calls to Bookings performance, however, when testing began, we had no other metric to indicate the qualification of
leads generated.*Leeds example to be
discussed in findings
An understanding of our statistical learnings, and the insights we can base upon them
Our learnings
What the research told us:
Based on Mosaic findings, we know that in London, Rightio's customer base is predominately Mosaic A, City Prosperity.
With this in mind, we hypothesised that the Speed variant would perform best, as Rightio’s customers in London are time-sensitive individuals.
London
London Findings
Visits to Calls 1
6.12% 12.77% 16.63% 28.21%27.91%
London Findings
Calls to final bookings 2
0% 33.33% 20% 45.45%25%
The Wildcard variant performed best in London, both in terms of Visits to Calls & Calls to Book.
We know that Rightio’s London customer base is predominately made up of Mosaic A individuals. However, this does not necessarily mean that Rightio is attracting only City Prosperity customers, it just means they do the best job of converting these leads.
Rightio’s adverts for the London test URL received clicks from areas as varied as Kennington and Maida Vale. There is a £7,638 difference in take-home salary in these areas. This highlights the divide of budgets per borough and also suggests that across such a varied region, users might have different priorities in mind.
In regions with a split of demographics, we believe the wildcard will perform best as it contains a wide mix of messaging designed to appeal to the price and time-conscious.
What this means
Key Observation
London is a clear example of a densely populated, highly diverse region, which doesn't lend itself to
one messaging category.
What the research told us:
Rightio’s customer base in Leeds swings towards Mosaic O, Rental Hubs.
Our thoughts were that these customers are likely to be price sensitive and searching for the best deal. In comparison to London, these leads will be less focused on Speed and more on Price.
Leeds
Leeds Findings
Visits to Calls 1
13.10% 48.89% 12.73% 18.18%24.14%
Leeds Findings
Calls to final bookings 2
45.45% 22.73% 57.14% 41.67%21.43%
Focusing on Visits to Calls, the Trust variant performed better than control and our other top performing variant Speed in a statistically significant way.
Price had the lowest Visits to Calls conversion rate of all the variants, at 12.7%. However, inversely this variant had the best calls to booking conversion rate. This suggests that while less leads from Leeds came through to the call centre, those that did were more qualified & more likely to convert.
This suggests that Rightio have a choice - Push more traffic through to call centres via the Trust variant or generate more viable leads via Price.
What this means
What the research told us:
From Mosaic analysis, we can see that the variations from the UK average are less significant than other regions.
There is some emergence of the mosaic M profile, but generally the Mosaic profiles are mixed. With this in mind, we hypothesised that Swansea might respond well to the Wildcard.
Swansea
Swansea Findings
Visits to Calls 1
5.17% 7.32% 5.26% 40.00%27.03%
Swansea Findings
Calls to final bookings 2
66.67% 66.67% 50.00% 0%60.00%
Swansea is a varied region, in that Rightio has no clear mosaic profile for its customer base within the city.
With this in mind, it does make sense that the Wildcard would perform best within Swansea.
As we have seen with London, in regions with a split of demographics, the Wildcard will perform best as it contains a wide mix of messaging designed to appeal to the price and time-conscious.
However, we can see that the calls to booking conversion rate stands at 0%, suggesting that while the Wildcard can attain leads for Rightio, they might not necessarily convert.
The Trust variant does the best job of converting in terms of visits to call and then calls to book, and therefore would be the preferred variant for both.
What this means
What the research told us:
From the Mosaic insights, we know that 27% of Rightio customers in Brighton are Mosaic profile O (Rental Hubs) with 15% at profile A (City Prosperity).
Therefore, Brighton mimics Leeds in terms of its biggest audience profile (O), but its second top Mosaic profile is A.
With this in mind, it would be logical to expect the Wildcard to perform well, given the mix of audiences (prosperous and more limited budgets).
Brighton
Brighton Findings
Visits to Calls 1
15.79% 36.11% 23.81% 33.33%22.00%
Brighton Findings
Calls to final bookings 2
66.67% 53.85% 60.00% 50.00%63.64%
In Brighton, the top Visit to Call variants, Trust & Wildcard, are relatively tied in terms of conversion.
Interestingly, in Leeds where the top profile was O, we saw Trust perform best. In London, where the Mosaic profile is A, the Wildcard performed best.
It therefore makes sense that in a region where we see O & A as the top profiles, Trust & Wildcard would be so closely tied, with just a 3% difference in Visits to Calls conversion performance.
However, while these variants are more likely gain leads, they aren’t the most likely to convert - the Control still does the best job of this (although there is not much difference between Control & Trust: just 13%).
What this means
Wildcard Observations
With the Wildcard variation winning in 2 locations, we performed a deep dive using Heatmapping & Scroll Depth indicators to
identify any potential anomalies
Wildcard Observations
Heat maps show that Wildcard users are viewing the whole page, engaging with &
viewing content below the fold
Wildcard Observations
A CTA above the fold on our Wildcard variant is not causing users to convert higher, as all evidence points to users
engaging with the whole page
Potential next steps and recommendations for Rightio landing page optimisation
Key learnings & Opportunities
Key recommendations
What are our immediate recommendations for locations
tested?
Implement best performing variant;
Wildcard is also best performing for final conversion.
London
LeedsTo generate an increase in inbound calls;
To generate better qualified inbound calls;
SwanseaTo generate an increase in inbound calls;
To generate better qualified inbound calls;
BrightonTo generate an increase in inbound calls;
To generate better qualified inbound calls;
Key Learning
We can clearly increase conversion rates of visits to calls through tailored messaging aligned to
Mosaic profile for specific* regions
What insights have we gathered regarding the Mosaic profiling content?
In 50% of cases, Mosaic profiling delivered the right result as expected for generating the most amount of conversions. However, this logic is not totally sound, especially for densely populated areas that speaking to one very specific audience subset will simply not work.
In these cases, the Wildcard variant won, specifically because it is a mixture of all key messaging and the absence of one focal message point.
*Specific Region Definition
Key Learning
Conversely, we can actively reduce Visit to Call conversions during
periods of low coverage or high call wait times
Turning the taps off during periods of low coverage, without impacting the overall customer experience
Recommended rollout strategy
1. We know the Wildcard page always outperforms the current Control landing page, in all locations.
Therefore, rollout Wildcard to all locations as the new landing page.
We anticipate an uplift of around 16.3% of calls to be generated based upon existing findings and results.
Recommended rollout strategy
2. For any regions that do not see an uplift of around 16.3% of calls with the move to the Wildcard - or any regions that see a dip in conversions - revert to the Mosaic indicated variant and monitor performance.
Through this approach, we will softly “find the specific regions” that react best to highly focused messaging with no detrimental effect to current conversion rates.
OK Good Better Best
Recommended rollout strategy
16.3%
Extended possibilities with DLP
PPC DG Dynamic LP CC Support Coverage
Reduced failed booking during times of low coverage
Extended possibilities with DLP
Dynamic LP CC Support
Key Hypothesis: Customised Call Scripts based upon landing page variation leading to higher end conversion
Core Motivations
We have distilled a core
understanding of our customers
In summary
What have we learned and delivery through this process
1
Increased Conversions
Increase in conversions
& understood why
3
Variant landing Pages
4 Designs are created,
ready to implement
2
Rollout Strategy
A clear direction for next
steps
4
Created possibilities
Created new
considerations for DLP integration to wider
business
5
Site rebuild
Replatform the Rightio site with DLP and multivariate testing at heart
Platform integration to CC Deep integration of web platform to CC working practices and scripts allowing
customised engagement
Coverage integration
Creation of coverage checker and integration driving wider DG &
fulfilment activities
Future considerations
Wider considerations for Rightio web presence
21 3
Copyright 2016 383 project ltd. All Rights Reserved.
The contents of this document are the property of 383. They represent the intellectual property in the form of, but not limited to, processes, ideas and creative designs. They may not be used without prior written agreement and only upon full compensation to 383 for the use or partial use of any of the material contained.
Thank you