Joshua G Rice. Improving Conversion Rates at 80 Percent … · 2020. 1. 13. · Visual Website Optimizer is website optimization tool that over 1200 companies have used to increase

Joshua G Rice. Improving Conversion Rates at 80 Percent Solutions. A Master’s Paper for the M.S. in I.S degree. August, 2014. 44 pages. Advisor: Paul Jones

This experiment used A/B testing to attempt to improve three tracked conversion goals--“Engagement”, “Paypal Submission”, and “Purchase Clicks” --on 80 Percent Solution’s product website www.MacFreedom.com. The participants for this study were web users from any location or device who visited www.MacFreedom.com during the testing period.

Three randomized controlled experiments were executed using Visual Website Optimizer to test for statistical significance of the conversion goals. The results of these experiments showed that:

• Changing the site’s tagline demonstrated significant success.

• Changes to landing page call-to-action buttons exhibited negative statistical significance.

• Variations of the purchase page call-to-action buttons demonstrated significant success.

Headings:

A/B testing

Quantitative marketing research

Information services -- Marketing

Response rates

User interfaces (Computer systems)

IMPROVING CONVERSION RATES AT 80 PERCENT SOLUTIONS

by Joshua G Rice

A Master’s paper submitted to the faculty of the School of Information and Library Science of the University of North Carolina at Chapel Hill

in partial fulfillment of the requirements for the degree of Master of Science in

Information Science.

Chapel Hill, North Carolina

August 2014

Approved by

_______________________________________ Paul Jones

1

Table of Contents

Introduction ........................................................................................................................ 2

Hypothesis .......................................................................................................................... 3

Review of Literature ........................................................................................................... 3

Method ............................................................................................................................... 9 Experiment 1: Tagline Text Variations ...................................................................................... 11 Experiment 2: CSS Call-‐To-‐Action Button & Background Div Variations .................................. 13 Experiment 3: Purchase Page “Buy Now” CTA Button Variations ............................................ 19

Results .............................................................................................................................. 21

Discussion ......................................................................................................................... 24

Limitations ........................................................................................................................ 26

Conclusion ........................................................................................................................ 27

Areas for further research ................................................................................................ 28

References ........................................................................................................................ 29

Appendix ........................................................................................................................... 37 1.1 ............................................................................................................................................. 37 1.2 ............................................................................................................................................. 39 1.3 ............................................................................................................................................. 41

2

Introduction

The use of controlled experiments has allowed Amazon.com to “innovate quickly

and effectively” (Kohavi, Henne, & Sommerfield, 2007). Numerous experiments are

thought to be “running at any given time” on Amazon.com “because of its relatively low

cost,” according to Jennifer Cardello, director at Nielsen Norman Group (2014).

Amazon.com, a multibillion-dollar e-commerce company, uses its website as its main

marketplace to sell products thus tracking user engagement to measure company success.

Not just large companies like Amazon use controlled experiments to enhance their

website, but many small e-commerce companies rely on their sites as their main source of

revenue.

For this project, I worked with 80 Percent Solutions, a small digital software

company located in Chapel Hill, North Carolina. 80 Percent Solutions has one e-

commerce website responsible for the majority of the company’s profits. Like many

small web companies, 80 Percent Solutions has limited resources for marketing and

advertising much less running controlled experiments on the primary product website. I

predict that running controlled experiments will be extremely important to the company’s

success, leading directly to increased sales.

3

Hypothesis

By manipulating 80 Percent Solution’s MacFreedom.com website tagline, landing

page call-to-action button, and purchase page “Buy Now” call-to-action buttons, I will

produce positive statistically significant conversion rates in comparison with the controls

for the following conversion goals: “Engagement”, “Paypal Submissions”, and “Purchase

Clicks”.

Review of Literature

Most successful 21st century companies have a presence on the Internet usually in

the form of a company website. Web users who visit a company’s website have the

ability to obtain information about the company and potentially purchase associated

goods. Companies can reach customers across physical barriers previously impossible

before the emergence of the World Wide Web. When businesses “trade products or

services conducted via computer networks,” this is known as e-commerce (E-commerce,

n.d.). E-commerce websites can be extremely important for some companies to sell their

products and services, as the site may exist as the only medium to sell.

If companies sell their products solely through e-commerce websites, this makes

the content and design elements of the website very valuable to the success of the

company. Websites should be carefully planned to provide a very intuitive user

experience enabling customers to purchase goods successfully. A customer’s user

experience on a website is extremely important for companies to optimize profit, because

it enables them to get from point A to point B, converting the user from a visitor to a

4

customer. Any lack of clarity in a website’s landing page or home page can result in a

user immediately navigating away from the site and thus not buying the product.

Companies that use a web presence as the only market to sell products should focus on

the design and content of their website to maximize sales.

After doing a large amount of research on how to optimize e-commerce websites,

call-to-actions showed up repeatedly as the key element for optimization. Call-to-actions

(CTAs) within websites are usually buttons or large graphics that prompt users to

perform specified actions. Whether this action is “to download a PDF, fill out a form, buy

a product, or even just click through to another page,” CTAs are the medium through

which the user interactions take place (Aagaard, 2013). There are a few elements

common across various sources that make up CTAs: “size, message (text), placement and

color” (Adlakha, 2013). The size of the CTAs should be “large” as well as consistent in

the same way that “you want to keep your branding consistent” (Bakkestuen, 2013). The

size and placement elements of the CTA button should combine to create buttons that

“stand out without overwhelming the design” (Chapman, n.d.). Numerous sources use the

term “Above The Fold” when describing the general placement and location of CTA

buttons on landing pages (Fishleder, 2014). Jakob Nielsen, a famous website usability

guru, defines the “Fold” as the space on a web page that is “viewable without further

action” (2010). He also makes any information placed in the fold seem important by

pointing out that “Web users spend 80% of their time looking at information above the

page fold” (Nielsen, 2010).

According to the popular analytics, marketing, and testing blog Kissmetrics, color

is the “strongest and most persuasive” visual cue (Smith, 2014). Despite the existing

5

theory and field of science known as color psychology, the color of CTA buttons “should

be in stark contrast to background colors” in order to capture attention (Chawla, 2013).

The text or message of a CTA button should give the user “a reason to click” as well as

“tell them clearly what they’ll get” without the use of “vague words” (Babbar, 2014).

Paras Chopra, chief executive officer of a popular marketing and analytics software

company, wrote a case study describing the increase of customers from a slight change of

text in a CTA button. The takeaway lesson learned from the case study was that if the text

within a CTA button “pushes too hard at the beginning,” a user “might feel distracted,

uncomfortable and leave the site” (Paras, 2011).

In this paper, I focused on creating variations of CTA buttons along with

changing landing page tagline text with the intent to improve defined conversion goals.

Winning variations will directly increase sales and conversions for 80 Percent Solutions.

80 Percent Solutions is a small digital software company based out of Chapel Hill,

North Carolina that uses an e-commerce website to sell software applications. 80 Percent

Solutions provides productivity tools with the main purpose of reducing digital

distractions. The company was founded in 2011 and is single-handedly run by Fred

Stutzman, Ph.D. The company currently is based in a Chapel Hill technology incubator

called Launch Chapel Hill.

Freedom, the company’s first product developed, has roughly 50,000 paid

customers and around 1,500 unique visitors every day. As of June 2014, the site’s main

product web page MacFreedom.com holds a top position on the first page of Google

results for the search term “Freedom”, which signifies the amount of daily traffic to the

website. The application works by allowing users to restrict how long they would like to

6

be without Internet, cell data, or Bluetooth access. There is also the ability to schedule

blocked periods in advance on specific days and times of the week. Freedom has been

proven successful by its popularity and effectiveness since it was first released to the

public for free in its early stages. News stories and articles about Freedom have appeared

across the web and in print at multiple mainstream media outlets such as The New York

Times, USA Today, and Time Magazine (Stutzman, n.d.).

Because of the company’s size and limited resources, no advertising or marketing

techniques are currently in place to help attract new customers. This means that most new

customers find out about Freedom through 80 Percent Solution’s website. The company

is trying to improve its website and increase sales by optimizing three conversion goals.

The first goal 80 Percent Solutions seeks to improve is user conversions, which is

the number of website visitors who buy Freedom and convert to paid customers. This

goal leads directly to increased sales and is the most important conversion goal to

improve.

Before a visitor of the site is able to buy Freedom, they must click to the purchase

page. The next conversion goal needing improvement is the number of clicks to this

page. This also plays a significant role as a conversion goal, because it can determine

whether or not the users are getting close to buying the product. As one would imagine,

the more clicks to the purchase page is probably related to more clicks to buy the product.

The last targeted conversion goal is related to the user’s activity when he or she

first lands on the website. Users who make a click or complete any action on the site are

tracked differently than users that leave the site immediately after they visit. If a user

visits MacFreedom.com and then immediately leaves the website, this would count as a

7

user who did not interact with the site and would thus contribute to a metric known as

bounce rate. In other words, this is the rate at which users visit and leave without

engaging with the website. The more engagement, the lower the bounce rate.

Fortunately, there are solutions and testing tools available that can optimize

usability. These tools allow website administrators to record and gather information about

their site. This may be as simple as tracking the number of clicks or submissions from an

html form. The tools work by embedding snippets of code in the website that collect data

on visiting users’ behavior. By tracking behavior, trial and error comparison tests can be

run to determine which version of the website produces the best results. These tools can

effectively optimize a website to increase user engagement with the site, ultimately

leading to increased purchase rates on the website.

Visual Website Optimizer is website optimization tool that over 1200 companies

have used to increase their sales and conversions (“What is Visual Website Optimizer

and how is it used?”, 2012). There are approximately 100 case studies written up on the

tool’s website that highlight how the tool was used to increase sales and conversions at

other companies. According to Becky Simanowski, an A/B Testing and Optimization

Program Manager for SAS Institute’s main company website, this tool is popularly used

as an “industry standard” (personal communication, May 1, 2014). Visual Website

Optimizer has a comprehensive number of features, but the main one I will be looking at

is the A/B test.

An A/B test is buzzword in marketing and web analytics that simply means a

randomized controlled experiment. In a basic A/B test, you have a control (A) and a

variation (B) that you are testing against each other to see the difference in whatever goal

8

or goals you are attempting to track. With the amount of traffic that 80 Percent Solutions

gets on their primary product website MacFreedom.com, an A/B/n test or A/B split test is

of better use. In an A/B split test, more than one variation can be tested against the

control because there is sufficient website traffic. This traffic is randomized and split up

between each variation and the control. Although very simple in terms of the type and

complexity of the test, A/B tests can provide a large number of advantages to

organizations (Nielsen, 2005).

There have been a few A/B tests run on the landing page since the website first

launched in 2011. None have been run recently, because the company is in a transitional

growth stage trying to hire employees and gather funding for the development into a

software as a service (SaaS) platform. Although the previous tests were done using

Visual Website Optimizer, they are no longer available for reference by the time of my

tests.

After fully understanding the intention of the product previously as a research

intern with the company, I decided to choose two areas of MacFreedom.com that need

improvement. Both areas have the potential to enhance website conversion rates.

The first area of improvement is the text of the website’s tagline. A tagline is a

short “memorable phrase or sentence that is closely associated with a particular person,

product, movie, etc.” (Tagline, n.d.). Jakob Nielsen emphasizes that a website’s tagline

should “tell us what they actually sell” versus using a more general phrase to describe a

benefit of that particular product or service (2001). Macfreedom.com’s current website

tagline is “Freedom for Productivity”, which does not clearly indicate what the company

is selling to the user.

9

The other area of improvement is the call-to-action buttons. CTAs “form a crucial

element of your landing page and the conversion funnel” (Chawla, 2013). The CTA

buttons on a website should stand out while maintaining consistency with other CTAs

used on the site (Bakkestuen, 2013). Currently, MacFreedom.com’s CTA buttons on the

landing page and purchase page vary in color and design consistency.

In this paper, I focused on creating variations of the landing page tagline and CTA

buttons with the intent to improve the above three conversion goals. Winning variations

will directly increase sales and conversions for the company.

Method

Three experiments were run testing three conversion goals set using the online

software tool Visual Website Optimizer. 80 Percent Solutions had an existing license for

the software. Each experiment was tested for “Engagement” and “Paypal Submissions”

set as goals within Visual Website Optimizer. Experiment 1 and Experiment 2 were also

tested for “Purchase Click” as a conversion goal. Because Experiment 3 involved a

variation change already located on the purchase page (MacFreedom.com/purchase), this

experiment tested for “Google Play Click” as the third conversion goal. This conversion

goal tracks clicks to the Google Play “Buy Now” CTA button

(https://play.google.com/store/apps/details?id=com.eightypct.freedom).

I used the term “Engagement” to mean the same definition as engagement within

the Visual Website Optimizer knowledgebase ("What does “engagement” ”, 2013). This

term measured when a user either filled out an html form or clicked on any of the

following seven html elements:

10

1. Anchor element(<a></a>)

2. Any element which is a descendent of Anchor element(e.g. click on img in

<a><img /></a>)

3. Button Element(<button></button>)

4. Any element which is a descendent of button element(e.g. click on span in

<button><span></span></button>)

5. Input element with type button (<input type=”button” />)

6. Input element with type submit (<input type=”submit” />)

7. Input element with type image (<input type=”image” />) ("What does

"engagement on the page" means?", 2013).

“Purchase Click” was defined as a conversion goal to mean the number of times a

user clicked to purchase Freedom. These clicks came from the landing page CTA buttons

located in the top right and bottom middle part of the page. Any clicks to the purchase

page (located in the navigation bar at the top of the web page) also were counted. These

clicks did not necessarily mean that users purchased Freedom. The next conversion goal

tracked this measure.

“Paypal Submissions” was the number of times a visitor confirmed and submitted

payment for a purchase. Ultimately this was the most important goal to track because it

related directly to increasing total revenue and sales.

Three experiments were run one at a time in numerical order, each starting on a

new Monday of the week. A screenshot of each experiment’s variation(s) and control has

been provided along with what was needed to create the variations. Three experiments

were run until a level of statistical significance was reached.

11

Variations were considered successful if the difference of the specified conversion

rate goal and the control was statistically significant. The conversion rate goal was

“represented by a binomial random variable,” which basically means that it either has a

value of converting or not converting (Chopra, 2010). If a variation of any of the

experiments did very poorly and proved statistically significant, the variation was

disabled so that it did not hurt sales for 80 Percent Solutions.

Experiment 1: Tagline Text Variations

Experiment 1 started on June 9th, 2014 and was an A/B split test. Three variations

of the website’s tagline were compared to the control tagline “Freedom For Productivity”

shown below.

12

• Variation 1: “Eliminate Digital Distractions”

o Text changed in front page html file

• Variation 2: “Block Digital Distractions”

13

o Text changed in front page html file

• Variation 3: “Freedom From Digital Distractions”

o Text

o Text changed in front page html file & “font-size: 25px” changed

within master.min.css file.

Experiment 2: CSS Call-To-Action Button & Background Div Variations

This experiment was also a split A/B test comparing three different variations of

the landing page top right CTA button with the control shown below.

14

• Variation 1: CTA button layered with soft grey background div

o Within master.min.css file, edited #buttonDarkBG

§ Replaced existing background-color value #2b333d to #eff1f3

• Variation 2: Green CTA button layered with dark background div

15

o Within master.min.css, edited a.buttonLink, a.buttonLinkWithImage

§ Removed existing background css: background:url("https://dlvfa1owjktk.cloudfront.net/freedom/web/844f58909f6c1a1ce71fc411b0ed70a0/img/buttonBGOut.png") 0 0 repeat-x;

§ Added new background gradient css: background: background: #49d100;

background: url(data:image/svg+xml;base64,PD94bWwgdmVyc2lvbj0iMS4wIiA/Pgo8c3ZnIHhtbG5zPSJod…EiIGhlaWdodD0iMSIgZmlsbD0idXJsKCNncmFkLXVjZ2ctZ2VuZXJhdGVkKSIgLz4KPC9zdmc+); background: -moz-linear-gradient(top, #49d100 0%, #00ad00 100%); background: -webkit-gradient(linear, left top, left bottom, color-stop(0%,#49d100), color-stop(100%,#00ad00)); background: -webkit-linear-gradient(top, #49d100 0%,#00ad00 100%); background: -o-linear-gradient(top, #49d100 0%,#00ad00 100%); background: -ms-linear-gradient(top, #49d100 0%,#00ad00 100%); background: linear-gradient(to bottom, #49d100 0%,#00ad00 100%); filter: progid:DXImageTransform.Microsoft.gradient( startColorstr='#49d100', endColorstr='#00ad00',GradientType=0 );

16

o Within master.min.css file, edited a.buttonLink:hover,a.buttonLinkWithImage:hover

§ Removed existing button hover css: background:url("https://dlvfa1owjktk.cloudfront.net/freedom/web/844f58909f6c1a1ce71fc411b0ed70a0/img/buttonBGOver.png") 0 0 repeat-x

§ Added new background gradient hover css: background: #00ad00; /* Old browsers */

/* IE9 SVG, needs conditional override of 'filter' to 'none' */ background: url(data:image/svg+xml;base64,PD94bWwgdmVyc2lvbj0iMS4wIiA/Pgo8c3ZnIHhtbG5zPSJodHRwOi8vd3d3LnczLm9yZy8yMDAwL3N2ZyIgd2lkdGg9IjEwMCUiIGhlaWdodD0iMTAwJSIgdmlld0JveD0iMCAwIDEgMSIgcHJlc2VydmVBc3BlY3RSYXRpbz0ibm9uZSI+CiAgPGxpbmVhckdyYWRpZW50IGlkPSJncmFkLXVjZ2ctZ2VuZXJhdGVkIiBncmFkaWVudFVuaXRzPSJ1c2VyU3BhY2VPblVzZSIgeDE9IjAlIiB5MT0iMCUiIHgyPSIwJSIgeTI9IjEwMCUiPgogICAgPHN0b3Agb2Zmc2V0PSIwJSIgc3RvcC1jb2xvcj0iIzAwYWQwMCIgc3RvcC1vcGFjaXR5PSIxIi8+CiAgICA8c3RvcCBvZmZzZXQ9IjEwMCUiIHN0b3AtY29sb3I9IiM0OWQxMDAiIHN0b3Atb3BhY2l0eT0iMSIvPgogIDwvbGluZWFyR3JhZGllbnQ+CiAgPHJlY3QgeD0iMCIgeT0iMCIgd2lkdGg9IjEiIGhlaWdodD0iMSIgZmlsbD0idXJsKCNncmFkLXVjZ2ctZ2VuZXJhdGVkKSIgLz4KPC9zdmc+); background: -moz-linear-gradient(top, #00ad00 0%, #49d100 100%); /* FF3.6+ */ background: -webkit-gradient(linear, left top, left bottom, color-stop(0%,#00ad00), color-stop(100%,#49d100)); /* Chrome,Safari4+ */ background: -webkit-linear-gradient(top, #00ad00 0%,#49d100 100%); /* Chrome10+,Safari5.1+ */ background: -o-linear-gradient(top, #00ad00 0%,#49d100 100%); /* Opera 11.10+ */ background: -ms-linear-gradient(top, #00ad00 0%,#49d100 100%); /* IE10+ */ background: linear-gradient(to bottom, #00ad00 0%,#49d100 100%); /* W3C */ filter: progid:DXImageTransform.Microsoft.gradient( startColorstr='#00ad00', endColorstr='#49d100',GradientType=0 ); /* IE6-8 */

• Variation 3: Green CTA button layered with soft grey background div

17

o Within master.min.css file, edited a.buttonLink,

a.buttonLinkWithImage § Removed existing background css:

background:url("https://dlvfa1owjktk.cloudfront.net/freedom/web/844f58909f6c1a1ce71fc411b0ed70a0/img/buttonBGOut.png") 0 0 repeat-x;

§ Added new background gradient css: background: background: #49d100;

background: url(data:image/svg+xml;base64,PD94bWwgdmVyc2lvbj0iMS4wIiA/Pgo8c3ZnIHhtbG5zPSJod…EiIGhlaWdodD0iMSIgZmlsbD0idXJsKCNncmFkLXVjZ2ctZ2VuZXJhdGVkKSIgLz4KPC9zdmc+); background: -moz-linear-gradient(top, #49d100 0%, #00ad00 100%); background: -webkit-gradient(linear, left top, left bottom, color-stop(0%,#49d100), color-stop(100%,#00ad00)); background: -webkit-linear-gradient(top, #49d100 0%,#00ad00 100%); background: -o-linear-gradient(top, #49d100 0%,#00ad00 100%); background: -ms-linear-gradient(top, #49d100 0%,#00ad00 100%); background: linear-gradient(to bottom, #49d100 0%,#00ad00 100%);

18

filter: progid:DXImageTransform.Microsoft.gradient( startColorstr='#49d100', endColorstr='#00ad00',GradientType=0 );

o Within master.min.css file, edited

a.buttonLink:hover,a.buttonLinkWithImage:hover § Removed existing button hover css:

background:url("https://dlvfa1owjktk.cloudfront.net/freedom/web/844f58909f6c1a1ce71fc411b0ed70a0/img/buttonBGOver.png") 0 0 repeat-x

§ Added new background gradient hover css: background: #00ad00; /* Old browsers */

/* IE9 SVG, needs conditional override of 'filter' to 'none' */ background: url(data:image/svg+xml;base64,PD94bWwgdmVyc2lvbj0iMS4wIiA/Pgo8c3ZnIHhtbG5zPSJodHRwOi8vd3d3LnczLm9yZy8yMDAwL3N2ZyIgd2lkdGg9IjEwMCUiIGhlaWdodD0iMTAwJSIgdmlld0JveD0iMCAwIDEgMSIgcHJlc2VydmVBc3BlY3RSYXRpbz0ibm9uZSI+CiAgPGxpbmVhckdyYWRpZW50IGlkPSJncmFkLXVjZ2ctZ2VuZXJhdGVkIiBncmFkaWVudFVuaXRzPSJ1c2VyU3BhY2VPblVzZSIgeDE9IjAlIiB5MT0iMCUiIHgyPSIwJSIgeTI9IjEwMCUiPgogICAgPHN0b3Agb2Zmc2V0PSIwJSIgc3RvcC1jb2xvcj0iIzAwYWQwMCIgc3RvcC1vcGFjaXR5PSIxIi8+CiAgICA8c3RvcCBvZmZzZXQ9IjEwMCUiIHN0b3AtY29sb3I9IiM0OWQxMDAiIHN0b3Atb3BhY2l0eT0iMSIvPgogIDwvbGluZWFyR3JhZGllbnQ+CiAgPHJlY3QgeD0iMCIgeT0iMCIgd2lkdGg9IjEiIGhlaWdodD0iMSIgZmlsbD0idXJsKCNncmFkLXVjZ2ctZ2VuZXJhdGVkKSIgLz4KPC9zdmc+); background: -moz-linear-gradient(top, #00ad00 0%, #49d100 100%); /* FF3.6+ */ background: -webkit-gradient(linear, left top, left bottom, color-stop(0%,#00ad00), color-stop(100%,#49d100)); /* Chrome,Safari4+ */ background: -webkit-linear-gradient(top, #00ad00 0%,#49d100 100%); /* Chrome10+,Safari5.1+ */ background: -o-linear-gradient(top, #00ad00 0%,#49d100 100%); /* Opera 11.10+ */ background: -ms-linear-gradient(top, #00ad00 0%,#49d100 100%); /* IE10+ */ background: linear-gradient(to bottom, #00ad00 0%,#49d100 100%); /* W3C */

19

filter: progid:DXImageTransform.Microsoft.gradient( startColorstr='#00ad00', endColorstr='#49d100',GradientType=0 ); /* IE6-8 */

o Within master.min.css file, edited #buttonDarkBG

§ Replaced existing background-color value #2b333d to be #eff1f3

Experiment 3: Purchase Page “Buy Now” CTA Button Variations

The last experiment run was an A/B test comparing one variation of the “Buy

Now” buttons on the purchase page with the control shown below.

• Variation: Blue “Buy Now” buttons with adjusted Google Play button size.

20

o Within master.min.css file, added the following new css:

§ Added new input css for new input id blue CTA button: input[id='blueCTA'] {

background:url("https://dlvfa1owjktk.cloudfront.net/freedom/web/844f58909f6c1a1ce71fc411b0ed70a0/img/buttonBGOut.png") 0 0 repeat-x; font-family:"Droid Sans", "Arial", sans-serif; font-size:18px; color:white; padding:10px 30px; text-decoration:none; -moz-border-radius:5px; -webkit-border-radius:5px; -khtml-border-radius:5px; border-radius:5px; border:none; position:relative; top:-5px; cursor: pointer; } input[id='blueCTA']:hover {

21

background:url("https://dlvfa1owjktk.cloudfront.net/freedom/web/844f58909f6c1a1ce71fc411b0ed70a0/img/buttonBGOver.png") 0 0 repeat-x; cursor: pointer; }

o Replaced existing gplaygr.png file with the following to remove translucent pixels in order to properly sync with new button size:

§ https://dl.dropboxusercontent.com/u/23344752/gplaygr.png

o Updated html file on purchase page. § For each of the 5 “Buy Now” buttons, the 3rd input tag line within

each form was replaced with the following: <input id="blueCTA" type="submit" name="submit" value="Buy Now">

//The original line was replaced to look something like this with a different source:

<input type="image" src="./Permanent_files/btn_buynow_LG.gif" border="0" name="submit" alt="PayPal - The safer, easier way to pay online!" style="padding-bottom:5px;">

Results

Experiment 1 was run from June 9th, 2014 to June 11th, 2014. Within that 3-day

window of time, there were 1,509 visitors and 60 total Paypal submissions. The tagline

variation “Block Digital Distractions” was a winning variation with a +102.71%

percentage improvement over the control tagline “Freedom for Productivity”. Below is an

exported table of the results for each conversion goal. See Appendix 1.1 for additional

figures.

22

Experiment 2 was run from June 17th, 2014 to June 20th, 2014. It started on a

Tuesday instead of a Monday because of technical difficulties importing the variations

into Visual Website Optimizer. Over four days, there were 483 visitors and 7 Paypal

submissions. Variation 1 of this experiment was disabled due to statistical significance of

-60.82% and -37.96% percentage improvement over the control. This experiment ended

up being disabled completely by Fred Stutzman on June 20th, 2014 because of the risk of

losing sales and traffic. Below is an exported table of the results for each conversion

goal. See Appendix 1.2 for additional figures.

23

Experiment 3 was run from June 23rd, 2014 to June 28th, 2014. Within the testing

period, there were 916 visitors and 203 Paypal submissions. The change to blue CTA

buttons reached success with a +26.04% percentage improvement over the control Paypal

CTA buttons. On June 25th, 2014, Visual Website Optimizer launched a new platform for

their software tool. This new platform brought along a new interface, which is why the

results below differ in style. See Appendix 1.3 for additional figures.

24

Discussion

Creating new variations can be a difficult task because the experimenter has to

make blind guesses to understand why the current design elements are not producing high

enough conversion goals. Even the results of previous A/B tests lead the experimenter to

make more educated guesses. I will try to explain and analyze each experiment as best I

can, while remembering that any reasoning behind statistical significance cannot be

proven.

Experiment 1 produced variation 2 as a statistically significant variation with a

+120.71% improvement of the Paypal Submissions conversion goal. Variation 1 and 3

both showed above a +60% increase of Paypal Submissions, although they did not reach

significance. All three variations showed positive results for the other conversion goals:

Purchase Click and Engagement. I believe the success of Experiment 1 was because the

tagline variations better explain what Freedom does more directly than the control

25

tagline, which proclaims somewhat vaguely that Freedom can be used to increase

productivity.

Experiment 2 had minimal positive results for Paypal submissions with variation

2 and 3 having a few conversions each. For reasons other than not reaching statistical

significance, the results are not very useful. First, the control had 0 Paypal submissions,

which makes the variations hard to compare with. Second, the change was made to the

landing page and Paypal submissions occur on the purchase page (after being directed

through the Paypal website). Lastly, with the Engagement conversion goal showing an

-8% or more decrease in improvement for the two enabled variations, the positive results

of the Paypal Submissions does not seem like they were caused by variations. Although

in this experiment Paypal Submissions is the most important conversion goal, the result

of the other two conversion goals decrease the value of the Paypal Submissions success.

This is a perfect example for why multiple conversion goals should be run

simultaneously.

By changing the landing page CTA buttons in Experiment 2, I hoped to maintain

design consistency. New variations were created to be more closely aligned with the

existing color palette used on the website. The color choices and removal of the

background border may have taken user attention and focus away from the CTA button.

Although the variations may look better visually, the prominence of the CTA buttons has

more influence and a higher priority. It would have been interesting to let the experiment

run for a longer period of time to see extended results.

Experiment 3 had only one variation running against the control, as well as a

slightly different conversion goal because the test ran on the purchase page instead of the

26

landing page. The variation reached conclusive results by improving the Paypal

Submissions conversion goal. The winning variation of the CTA button maintained

design consistency of colors and button design style across the website compared to the

control CTA buttons that resembled out-of-date Paypal “Buy Now” buttons. The

consistency of CTA buttons work in this experiment over the previous, because users

have already made the decision to purchase Freedom and therefore, the CTA buttons do

not require as much attention. There was also a 4.85% improvement of Engagement over

the control.

The third conversion goal for Experiment 3, the Google Play Click saw a

-17.3% decrease in improvement. This conversion goal did not carry as much weight as

the other two conversion goals in this experiment, because the Google Play Click only

tracked clicks to one of the six CTA buttons on the purchase page. This decrease in

improvement may be because the blue CTA “Buy Now” buttons pulled attention away

from the “Google Play” button and focused more attention on themselves, thus increased

overall user conversions (Paypal Submissions).

Limitations

There are a few general limitations to standard A/B tests that apply to this

experiment. A/B tests are normally focused on short-term optimization, meaning that a

variation can prove statistically significant over a short period, but over a longer period

the control may prove to be more successful. As Nielsen states, if you are “basing your

decisions on short-term numbers…they can lead you astray” (2005). The winning

variation from Experiment 1 should be tested continually over an extended period of time

27

to prove that it does reach statistical significance over the control. The same logic can be

applied to the disabled variation in Experiment 2, but the sales of the company may be

heavily affected in the process of trying to prove significance over the extended

timeframe.

Another major limitation already touched on is that A/B tests do not provide any

understanding or reasoning behind why variations are effective. If variation A is a winner

over variation B, it may increase the tested conversion goals, but may not be helpful for

transitioning other parts of the website in the future. Nielsen compares the narrowness of

A/B testing with an “open-ended method like user testing, where users often reveal

stumbling blocks you never would have expected” (2005). Additional types of testing

may be needed to better understand the user and meaning behind successful and

unsuccessful variations.

Conclusion

By focusing on specific design elements and measuring the impact of changes by

tracking conversion goals, this study has proved 2 of 3 experiments statistically

successful. Although the experimenter does not venture any strong behavioral insights as

to why the variations were successful, A/B tests have and can continue to effectively

increase sales at 80 Percent Solutions. A/B tests are most useful for optimizing tracked

conversion rates and offer an additional benefit of further educating the experimenter

about the product being tested. Other than improved conversion rates, A/B tests can lead

to better-quality decisions and more successful tests in the future.

28

Areas for further research

Follow-up experiments to this study would be very beneficial, as they can provide

more data to strengthen winning variation significance. Outside of following up with A/B

split testing, other methods of increasing usefulness include using tracking, surveys, and

user personas (Cleanthous, 2014). If possible, a longer testing period could certify that

the negative results from Experiment 2 are repeatable. However, this validation may not

immediately occur after an additional timeframe, because as Chopra states “conversion

rate constantly changes…because your traffic mix changes from day to day” (2012).

Testing new landing page CTA button variations may eventually provide further support

one way or the other, but at the risk of losing sales if the variations produce negative

results.

In Experiment 2, I believe the proper balance of design consistency and user

attention is currently missing within the front page CTA buttons. This level of balance

may be present within Experiment 3 from conclusive results, but follow-up experiments

should still be run to increase significance. Further research should be done looking at the

causal relationship between the findings and established behavioral research relating to

colors, style, and attraction.

29

References

Aagaard, M. (2013, May 23). How To Design Call to Action Buttons That Convert.

Retrieved June 2, 2014, from http://unbounce.com/conversion-rate-

optimization/design-call-to-action-buttons/

Adlakha, K. (2013, December 10). Designing Call-to-Action Buttons that Customers

Click. Retrieved June 2, 2014, from

http://www.affinityexpress.com/2013/12/10/designing-call-action-buttons-

customers-click

Babbar, S. (2014, March 25). 9 Insanely Simple Ways to Optimize Your PPC Landing

Page. Retrieved June 25, 2014, from https://vwo.com/blog/ways-to-optimize-ppc-

landing-page

Bakkestuen, M. (2013, February 12). What Makes a Great Call to Action? Retrieved June

25, 2014, from http://blog.acumium.com/2013/02/12/what-makes-a-great-call-to-

action/

Benun, I. (2005, June). How to Create a Winning Tagline. Retrieved June 2, 2014, from

http://www.usabilityprofessionals.org/upa_publications/upa_voice/volumes/2005/

june/winning_tagline.html

Birch, N. (2013, June 6). Clever and Conspicuous Tagline is a First Step to a Good

Acquaintance. Retrieved June 2, 2014, from http://designmodo.com/tagline-

typography/

30

Bustos, L. (2011, January 20). Is Your PayPal Button Hurting Conversions? Retrieved

June 5, 2014, from http://www.getelastic.com/pay-pal-fud/

Cardello, J. (2014, April 20). Define Stronger A/B Test Variations Through UX Research

Retrieved June 5, 2014, from http://www.nngroup.com/articles/ab-testing-and-ux-

research/

Chapman, C. (n.d.). “Call To Action” Buttons: Guidelines, Best Practices And Examples.

Retrieved June 5, 2014, from http://www.hongkiat.com/blog/call-to-action-

buttons-guidelines-best-practices-and-examples/

Chartrand, J. (n.d.). How to Create a Rock-Solid Tagline That Truly Works. Retrieved

June 5, 2014, from http://www.copyblogger.com/create-a-tagline/

Chawla, S. (2013, April 18). Call to Action Buttons: The Ultimate Guide on Which Ones

Convert and Why. Retrieved June 25, 2014, from https://vwo.com/blog/call-to-

action-buttons-ultimate-guide/

Chopra, P. (2010, January 1). How reliable are your split test results? Retrieved June 25,

2014, from https://vwo.com/blog/how-reliable-are-your-split-test-results/

Chopra, P. (2010, January 26). What you really need to know about mathematics of A/B

split testing. Retrieved June 25, 2014, from https://vwo.com/blog/what-you-

really-need-to-know-about-mathematics-of-ab-split-testing/

Chopra, P. (2011, October 18). ‘Try demo’ or ‘Buy now’: A/B testing finds which button

increased clickthroughs by 47%. Retrieved June 25, 2014, from

https://vwo.com/blog/testing-call-to-action-buttons/

Chopra, P. (2012, February 23). 5 most common A/B testing misconceptions. Retrieved

June 5, 2014, from https://vwo.com/blog/ab-testing-misconceptions/

31

Cleanthous, A. (2014, March 31). 10 Guidelines To Improve The Effectiveness Of A/B

Testing. Retrieved June 1, 2014, from http://usabilitygeek.com/10-guidelines-

improve-effectiveness-a-b-testing/

Cookson, G. (2011, July 28). What makes a good call to action button? Best practice tips.

Retrieved June 25, 2014, from http://blog.respondhq.com/call-to-action/what-

makes-a-good-call-to-action-button-best-practice-tips/

DeMers, J. (2013, July 25). The Definitive Guide to Crafting Winning Calls to Action in

Your Content. Retrieved June 25, 2014, from

http://www.huffingtonpost.com/jayson-demers/the-definitive-guide-to-

c_b_3648475.html

E-Commerce. (n.d.). In Wikipedia. Retrieved June 9, 2014, from

http://en.wikipedia.org/wiki/E-Commerce

Erin, A. (2014, March 11). Rethink The Psychology of Your Calls to Action. Retrieved

June 5, 2014, from http://sazbean.com/category/strategy/usability-strategy/

Fishleder, A. (2014, March 20). The Call To Action: Being at the Right Place at the Right

Time. Retrieved June 5, 2014, from http://www.maximizer.com/blog/the-call-to-

action-being-at-the-right-place-at-the-right-time/

Gardner, O. (2009, December 3). The Future of the Marketing Call-To-Action Button:

Big, Shiny & Animated. Retrieved June 5, 2014, from

http://unbounce.com/conversion-rate-optimization/the-future-of-marketing-call-

to-action-buttons/

Geisheker, P. (2013, July 29). The Truth About the Best and Worst Call to Action Button

Colors for Your Website. Retrieved June 5, 2014, from

32

https://smallbusiness.yahoo.com/advisor/truth-best-worst-call-action-button-

colors-website-212506540.html

Gelberg, J. (2010, March 15). How to Choose a Tag Line That Delivers Your Company’s

Message Effectively. Retrieved June 5, 2014, from

http://www.bluefountainmedia.com/blog/tag-line-delivers-message/

Gube, J. (2009, October 13). Call to Action Buttons: Examples and Best Practices.

Retrieved June 10, 2014, from

http://www.smashingmagazine.com/2009/10/13/call-to-action-buttons-examples-

and-best-practices/

Hamill, D. (2009, March 19). Good Call-To-Action Buttons. Retrieved June 10, 2014,

from http://www.uxbooth.com/articles/good-call-to-action-buttons/

Harrison, K. (n.d.). Ensure your home page tagline tells what you actually do. Retrieved

June 10, 2014, from http://www.cuttingedgepr.com/articles/name-your-home-

page-tagline.asp

How many types of conversion goals are present in VWO? (n.d.). Retrieved June 5, 2014,

from http://visualwebsiteoptimizer.com/knowledge/how-many-types-of-

conversion-goals-are-present-in-vwo/

Kohavi, R., Henne, R. M., & Sommerfield, D. (2007, August). Practical guide to

controlled experiments on the web: listen to your customers not to the hippo. In

Proceedings of the 13th ACM SIGKDD international conference on Knowledge

discovery and data mining (pp. 959-967). ACM.

33

Krug, S. (2014, April 30). Steve Krug on Why Nothing Beats a Good Tagline!™.


http://www.peachpit.com/articles/article.aspx?p=2209312

Linstroth, D. (n.d.). Test Your Message To Build The Best Landing Page. Retrieved June

5, 2014, from http://blog.launchrock.co/test-your-message-to-build-the-best-

landing-page/

Moth, D. (2013, May 30). Seven tips for designing an effective ecommerce call-to-action

Retrieved June 4, 2014, from https://econsultancy.com/blog/62829-seven-tips-for-

designing-an-effective-ecommerce-call-to-action

Nielsen, J. (2001, July 22). Tagline Blues: What's the Site About? Retrieved June 4,

2014, from http://www.nngroup.com/articles/tagline-blues-whats-the-site-about/

Nielsen, J. (2003, November 10). The Ten Most Violated Homepage Design Guidelines.

Retrieved June 4, 2014, from http://www.nngroup.com/articles/most-violated-

homepage-guidelines/

Nielsen, J. (2005, August 15). Putting A/B Testing in Its Place. Retrieved June 7, 2014,

from http://www.nngroup.com/articles/putting-ab-testing-in-its-place/

Nielsen, J. (2010, March 22). Scrolling and Attention. Retrieved July 7, 2014, from

http://www.nngroup.com/articles/scrolling-and-attention/

Nielsen, J. (2013, November 24). Conversion Rates. Retrieved June 7, 2014, from

http://www.nngroup.com/articles/conversion-rates/

Niggulis, O. (2013, January 16). Which Color Converts The Best? Retrieved June 10,

2014, from http://conversionxl.com/which-color-converts-the-best/#.

34

Olyslager, P. (n.d.). Call To Action Buttons and the Psychology of Color. Retrieved June

10, 2014, from http://www.paulolyslager.com/call-to-action-buttons-psychology-

color/

Overgoor, J. (2014, May 27). Experiments at Airbnb. Retrieved June 10, 2014, from

http://nerds.airbnb.com/experiments-at-airbnb/

Raby, J. (2013, October 22). Call To Actions – What Are They and Why You MUST Use

Them on Your Website. Retrieved June 10, 2014, from

http://www.combustible.ca/en/2013/10/call-to-actions-what-are-they-and-why-

you-must-use-them-on-your-website/

Robinson, N. (n.d.). Crafting the Perfect Buy Button. Retrieved June 5, 2014, from

http://www.trinityinsight.com/resources/crafting-perfect-buy-button/

Ryan, P. (2014, February 14). Web Site Usability and Conversion Rates. Retrieved June

5, 2014, from http://www.ryandesignstudio.com/news/2014/web-site-usability-

conversion-rates/

Sharafeddine, L. (2012, October 4). Call To Action Button Best Practices: Internal

Factors. Retrieved June 5, 2014, from http://blog.market8.net/b2b-web-design-

and-inbound-marketing-blog/Call-To-Action-Button-Best-Practices-Internal-

Factors

Simmons, G. (2014, April 1). Ten Timely Tips for Terrific Title Tags. Retrieved June 5,

2014, from http://searchenginewatch.com/article/2348627/Ten-Timely-Tips-for-

Terrific-Title-Tags

35

Smith, J. (2014, June 6). How to Use the Psychology of Color to Increase Website

Conversions. Retrieved July 7, 2014, from

http://blog.kissmetrics.com/psychology-of-color-and-conversions/

Stutzman, F. (n.d.). Praise for Freedom. Retrieved June 26, 2014, from

https://macfreedom.com/press

Tagline. (n.d.). Retrieved June 26, 2014, from http://www.merriam-

webster.com/dictionary/tagline

Thomason, L. (2002, August). Promotion Tip: Explain Your Site With A Tagline.


http://www.netmechanic.com/news/vol5/promo_no15.htm

Tseng, A. (2011, April 25). Why Users Click Right Call to Actions More Than Left

Ones. Retrieved June 5, 2014, from http://uxmovement.com/buttons/why-users-

click-right-call-to-actions-more-than-left-ones/

Vertommen, K. (2012, August 8). How to Reduce Your Website’s Bounce Rate.

Retrieved June 25, 2014, from http://www.instantshift.com/2012/08/08/how-to-

reduce-your-websites-bounce-rate/

Walker, T. (2010, February 24). 25 Examples of Convincing Call-to-Action Buttons.

Retrieved June 25, 2014, from http://designshack.net/articles/inspiration/25-

examples-of-convincing-call-to-action-buttons/

Weber, H. (2012, October 10). The new ‘Pay with Stripe’ button makes online payments

feel less like a Darth Vader death grip. Retrieved June 25, 2014, from

http://thenextweb.com/dd/2012/10/10/the-new-pay-with-stripe-button-makes-

online-payments-not-feel-like-death/

36

Weil, D. (2002, December 3). Top Tips for Taglines That Engage First-Time Visitors.

Retrieved June 25, 2014, from http://www.marketingprofs.com/2/weil3.asp

What does "engagement on the page" means? (2013, May 16). Retrieved June 25, 2014,

from http://visualwebsiteoptimizer.com/knowledge/what-does-engagement-on-

the-page-means/

What is Visual Website Optimizer and how is it used? (2012, June 7). Retrieved June 25,

2014, from http://visualwebsiteoptimizer.com/knowledge/what-is-visual-website-

optimizer-and-how-is-it-used/

37

Appendix

The figures below show results of the experiments in more detail.

1.1

38

Paypal Submission

Purchase Click

Engagement

39

1.2

40

Paypal Submission

Purchase Click

Engagement

41

1.3

42

Paypal Submission

Google Play Click

Engagement

Campaign N

ame: Josh Rice Test 1

Campaign Id: 38

‐‐‐ Aggregate Data ‐‐‐Engagem

entControl

20.86%2.69%

‐‐

37478

Engagement

Eliminate Digital Distractions

24.32%2.86%

16.63%87%

37090

Engagement

Block Digital Distractions23.41%

2.68%12.27%

81%410

96Engagem

entFreedom

From Digital Distractions

23.10%2.86%

10.75%77%

35582

Paypal Submission

Control2.41%

1.01%‐

‐374

9Paypal Subm

issionElim

inate Digital Distractions4.59%

1.39%90.93%

95%370

17Paypal Subm

issionBlock Digital Distractions

4.88%1.36%

102.71%97%

41020

Paypal Submission

Freedom From

Digital Distractions3.94%

1.32%63.88%

88%355

14Purchase Click

Control11.23%

2.09%‐

‐374

42Purchase Click


13.78%2.29%

22.74%85%

37051

Purchase ClickBlock Digital Distractions

15.12%2.26%

34.66%95%

41062

Purchase ClickFreedom


12.96%2.28%

15.39%76%

35546

‐‐‐Daywise Data‐‐‐

Goal

Variation Nam

eDate

VisitorsConversions

Conversion RateEngagem

entControl

9/6/201467

1319.40%

Engagement

Control10/6/2014

19438

19.59%Engagem

entControl

11/6/2014113

2723.89%

Engagement


9/6/201473

1317.81%

Engagement


10/6/2014189

4825.40%

Engagement


11/6/2014108

2926.85%

Engagement

Block Digital Distractions9/6/2014

8025

31.25%Engagem

entBlock Digital Distractions

10/6/2014224

4720.98%

Engagement


10624

22.64%Engagem

entFreedom


9/6/201468

1522.06%

Engagement

Freedom From

Digital Distractions10/6/2014

17340

23.12%

Engagement

Freedom From


11427

23.68%Paypal Subm

issionControl

9/6/201467

34.48%

Paypal Submission

Control10/6/2014

1944

2.06%Paypal Subm

issionControl

11/6/2014113

21.77%

Paypal Submission


9/6/201473

34.11%

Paypal Submission


10/6/2014189

63.17%

Paypal Submission


11/6/2014108

87.41%

Paypal Submission


804

5%Paypal Subm

issionBlock Digital Distractions

10/6/2014224

94.02%

Paypal Submission


1067

6.60%Paypal Subm

issionFreedom


9/6/201468

22.94%

Paypal Submission

Freedom From


1736

3.47%Paypal Subm

issionFreedom


11/6/2014114

65.26%

Purchase ClickControl

9/6/201467

68.96%


10/6/2014194

2713.92%


11/6/2014113

97.96%

Purchase ClickElim

inate Digital Distractions9/6/2014

737

9.59%Purchase Click


10/6/2014189

2412.70%

Purchase ClickElim

inate Digital Distractions11/6/2014

10820

18.52%Purchase Click


8017



22427



10618


Freedom From


6810


Freedom From


17320


Freedom From


11416

14.04%

Campaign N

ame: Josh Rice Test 2

Campaign Id: 45

‐‐‐ Aggregate Data ‐‐‐Paypal Subm

issionControl

0%0%

‐‐

1340

Paypal Submission

Light Background (1)0%

0%0%

0%72

0Paypal Subm

issionGreen Button (2)

3.03%1.91%

0%0%

1324

Paypal Submission

Green Button/W

hite BG (3)

2.07%1.51%

0%0%

1453

Engagement

Control22.39%

4.61%‐

‐134

30Engagem

entLight Background (1)

13.89%5.22%

‐37.96%6%

7210

Engagement

Green Button (2)

20.45%4.49%

‐8.64%35%

13227

Engagement

Green Button/W

hite BG (3)

18.62%4.14%

‐16.83%22%

14527


14.18%3.86%

‐‐

13419

Purchase ClickLight Background (1)

5.56%3.46%

‐60.82%2%

724

Purchase ClickGreen Button (2)

12.88%3.73%

‐9.17%38%

13217

Purchase ClickGreen Button/W

hite BG (3)

13.79%3.67%

‐2.72%46%

14520


Goal

Variation Nam

eDate

VisitorsConversion

Conversion RatePaypal Subm

issionControl

17/06/1433

00%

Paypal Submission

Control18/06/14

430

0%Paypal Subm

issionControl

19/06/1437

00%

Paypal Submission

Control20/06/14

210

0%Paypal Subm

issionLight Background (1)

17/06/1437

00%

Paypal Submission

Light Background (1)18/06/14

350

0%Paypal Subm

issionGreen Button (2)

17/06/1433

13.03%

Paypal Submission

Green Button (2)

18/06/1439

25.13%

Paypal Submission

Green Button (2)

19/06/1444

12.27%

Paypal Submission

Green Button (2)

20/06/1416

00%

Paypal Submission

Green Button/W

hite BG (3)

17/06/1431

26.45%

Paypal Submission

Green Button/W

hite BG (3)

18/06/1441

00%

Paypal Submission

Green Button/W

hite BG (3)

19/06/1441

00%

Paypal Submission

Green Button/W

hite BG (3)

20/06/1432

13.13%

Engagement

Control17/06/14

339

27.27%Engagem

entControl

18/06/1443

1330.23%

Engagement

Control19/06/14

374

10.81%Engagem

entControl

20/06/1421

419.05%

Engagement


377

18.92%Engagem

entLight Background (1)

18/06/1435

25.71%

Engagement


01‐

Engagement

Green Button (2)

17/06/1433

721.21%

Engagement

Green Button (2)

18/06/1439

717.95%

Engagement

Green Button (2)

19/06/1444

818.18%

Engagement

Green Button (2)

20/06/1416

531.25%

Engagement

Green Button/W

hite BG (3)

17/06/1431

825.81%

Engagement

Green Button/W

hite BG (3)

18/06/1441

614.63%

Engagement

Green Button/W

hite BG (3)

19/06/1441

512.20%

Engagement

Green Button/W

hite BG (3)

20/06/1432

825%


17/06/1433

515.15%


18/06/1443

716.28%


19/06/1437

38.11%


20/06/1421

419.05%


17/06/1437

00%


18/06/1435

38.57%


19/06/140

1‐


17/06/1433

412.12%


18/06/1439

410.26%


19/06/1444

715.91%


20/06/1416

212.50%


hite BG (3)

17/06/1431

412.90%


hite BG (3)

18/06/1441

512.20%


hite BG (3)

19/06/1441

49.76%


hite BG (3)

20/06/1432

721.88%

Campaign N

ame: Josh Rice Test #3

Campaign Id: 47

‐‐‐ Aggregate Data ‐‐‐Engagem

entControl

55.06%2.18%

‐‐

850468

Engagement

Blue Buttons59.93%

2.20%8.84%

98%811

486Paypal Subm

issionControl

20.24%1.76%

‐‐

850172

Paypal Submission

Blue Buttons24.04%

1.92%18.82%

97%811

195Google Play Click

Control5.18%

0.97%‐

‐850

44Google Play Click

Blue Buttons4.69%

0.95%‐9.48%

32%811

38


Goal

Variation Nam

eDate

VisitorsConversion

Conversion RateEngagem

entControl

23/06/1410

330%

Engagement

Control24/06/14

11171

63.96%Engagem

entControl

25/06/14109

6155.96%

Engagement

Control26/06/14

9055

61.11%Engagem

entControl

27/06/1485

5160%

Engagement

Control28/06/14

6327

42.86%Engagem

entControl

29/06/1472

3244.44%

Engagement

Control30/06/14

7541

54.67%Engagem

entControl

1/7/201483

4048.19%

Engagement

Control2/7/2014

8146

56.79%Engagem

entControl

3/7/201462

3759.68%

Engagement

Control4/7/2014

94

44.44%Engagem

entBlue Buttons

23/06/149

666.67%

Engagement

Blue Buttons24/06/14

11062

56.36%Engagem

entBlue Buttons

25/06/14105

5855.24%

Engagement


8051

63.75%Engagem

entBlue Buttons

27/06/1490

5763.33%

Engagement


5435

64.81%Engagem

entBlue Buttons

29/06/1455

2749.09%

Engagement


7045

64.29%Engagem

entBlue Buttons

1/7/201475

4965.33%

Engagement


7740

51.95%Engagem

entBlue Buttons

3/7/201479

4962.03%

Engagement


77

100%Paypal Subm

issionControl

23/06/1410

110%

Paypal Submission

Control24/06/14

11131

27.93%Paypal Subm

issionControl

25/06/14109

2119.27%

Paypal Submission

Control26/06/14

9017

18.89%Paypal Subm

issionControl

27/06/1485

1618.82%

Paypal Submission

Control28/06/14

636

9.52%Paypal Subm

issionControl

29/06/1472

68.33%

Paypal Submission

Control30/06/14

7523

30.67%Paypal Subm

issionControl

1/7/201483

1922.89%

Paypal Submission

Control2/7/2014

8115

18.52%Paypal Subm

issionControl

3/7/201462

1625.81%

Paypal Submission

Control4/7/2014

91

11.11%Paypal Subm

issionBlue Buttons

23/06/149

222.22%

Paypal Submission


11028

25.45%Paypal Subm

issionBlue Buttons

25/06/14105

2120%

Paypal Submission


8023

28.75%Paypal Subm

issionBlue Buttons

27/06/1490

2426.67%

Paypal Submission


5413

24.07%Paypal Subm

issionBlue Buttons

29/06/1455

1221.82%

Paypal Submission


7021

30%Paypal Subm

issionBlue Buttons

1/7/201475

912%

Paypal Submission


7718

23.38%Paypal Subm

issionBlue Buttons

3/7/201479

2025.32%

Paypal Submission


74

57.14%Google Play Click

Control23/06/14

101

10%Google Play Click

Control24/06/14

1118


Control25/06/14

1096

5.50%

Google Play Click

Control26/06/14

904


Control27/06/14

853


Control28/06/14

632


Control29/06/14

726


Control30/06/14

751


Control1/7/2014

834


Control2/7/2014

816


Control3/7/2014

623


Control4/7/2014

90

0%Google Play Click


90

0%Google Play Click


1103



1054



805



902



545



551



703



758



772



794



71

14.29%

Documents

Joshua G Rice. Improving Conversion Rates at 80 Percent … · 2020. 1. 13. · Visual Website Optimizer is website optimization tool that over 1200 companies have used to increase