Split Test Master Guide · your costs will eventually rise proportionally to the amount of traffic you get, which will eventually lead to less money in the bank. After 9 years in

SSpplliitt TTeesstt MMaasstteerr GGuuiiddee Learn How to Split Test Like a Pro! Written By Justin Harrison

Page 1 of 55

Get Paid to Refer the Split Test Master Guide to your friends Visit: http://www.splittestmasterguide.com/jv/

Table of Contents:

Table of Contents .........................................................................2

Legal Notices ................................................................................4

Foreword .......................................................................................5

About Justin Harrison..................................................................6

1. Split Test Master Guide Introduction......................................7

1.1. Why this Guide? ....................................................................................7 1.2. Why Split Test? .....................................................................................8

2. Split Testing Basics ...............................................................11

2.1. What is Split Testing?..........................................................................11 2.2. Important Split Testing Terminology....................................................11 2.3. Key Testing Metrics.............................................................................13 2.4. Always test against your control page .................................................19 2.5. Making Your Split Tests Statistically Valid...........................................20

3. What to Test ............................................................................23

3.1. What should you be testing? ...............................................................23 3.2. Test Scenarios ....................................................................................24

4. Different Types of Split Tests................................................27

4.1. A/B Split Testing Explained .................................................................27 4.2. Multilevel Split Testing Explained ........................................................27 4.3. Multivariate Split Testing Explained.....................................................28 4.4. Which Split Testing Method Should You Use? ....................................29

5. Designing Your Split Test......................................................31

5.1. Introduction..........................................................................................31 5.2. Conversion Mapping ...........................................................................31 5.3. Defining Your Target Goals.................................................................34 5.4. Testing With Traffic Consistency .........................................................34 5.5. The Biggest Secret to Successful Split Testing ...................................35

Page 2 of 55

Get Paid to Refer the Split Test Master Guide to your Friends Visit: http://www.splittestmasterguide.com/jv/

Split Test Master Guide

6. How to Interpret Data .............................................................38

6.1. Introduction..........................................................................................38 6.2. Calculating Your Lift ............................................................................38 6.3. The Noise Factor.................................................................................39 6.4. Making Allowance for Latent Conversions ..........................................40 6.5. Interpreting Your Key Metrics ..............................................................40 6.6. Monitoring Visitor Activity ....................................................................42 6.7. The Relationship between Split Tests and Analytics ...........................43

7. Important Considerations......................................................44

7.1. SEO Considerations............................................................................44 7.2. PPC Considerations ............................................................................44

8. Choosing a Split Testing Platform........................................47

9. Closing Thoughts ...................................................................49

10. Resources .............................................................................50

11. Bonus Chapter......................................................................52

11.1. All time Biggest Split Testing Mistakes:.............................................52 11.2. Landing Page Tips: ...........................................................................54

12. Get Paid to Promote This Guide .........................................55

Page 3 of 55 Get Paid to Refer the Split Test Master Guide to your Friends

Visit: http://www.splittestmasterguide.com/jv/

Legal Notices:

Copyright information:

Copyright © 2008 Justin Harrison. All rights Reserved. This document may not be copied in part or in full without the express written permission from the author. All violations will be prosecuted to the fullest extent of the law.

Information Disclaimer:

All the information provided is for knowledge purposes only, and does not constitute a legal contract between the author or any distributing party, person, or entity. Although every reasonable effort is made to present current and accurate information, the author or any distributing party makes no guarantees of any kind. In no event shall the author or any distributing party be responsible or liable directly or indirectly, for any damage or loss caused or alleged to be caused by, or in connection with, the use of, or reliance on any such content, goods, or services available on or through any such channel.

Income Disclaimer:

In no way does the information represented in this document constitute a representation of earnings and, as with any business, your results may vary and will be based on your individual capacity, business experience, expertise, and level of desire. The use of the information, products, and services should be based on your own due diligence and you agree that author, or any distributing party, is not liable for any success or failure of your business that is directly or indirectly related to the purchase and use of our information, products, and services.

Printing Notice:

You may only print this document for personal use.

Page 4 of 55


Foreword:

First and foremost, I would like to thank my close friends, family, clients, partners, co-workers, affiliates, and suppliers for their dedicated support and unwavering belief in my abilities, vision, and passion. Secondly, and most importantly, to my beautiful wife who has accepted the fact that I have a mistress called “online marketing”. I thank you from the bottom of my heart for your patience, understanding and support. I continue to be awed by your grace.

Last but not least, to my two dogs, Rubbish and Patches. You are indeed the best friends a guy could ever have. Thank you for enduring those long days and nights in my study while I thumped away at this guide. You have never left my side, and I will never leave yours.

For the past 9 years I have been enthralled, engaged, and captivated by the online market. I am privileged to be positioned alongside some of the most respected clients, suppliers, and partners in the industry. Every business venture, every friendship made, and every lesson learned has been a journey unto itself and I am humbled by the privileges bestowed upon me. I thank you for taking the time to read this book. I know beyond doubt that if you apply the information within this book there will be a marked improvement in your business.

Page 5 of 55


About Justin Harrison:

I won’t go into any extensive details about my background since you can find detailed information on my consulting career and business at both my International and South African internet marketing consulting websites. However, since some background information in context to this book is required, here are some key items you should know:

• Over the past 9 years I have been extensively involved in online marketing, both from within my business and as a consultant to many of the leading online brands.

• I never intended on consulting. However, due to the early success of

my business, word got around to some influential executives and before long I was requested to consult for these companies.

• The consulting arm of my business continues to grow via word of

mouth and through existing client relationships.

• I currently command $950.00 per hour for web analytics and data interpretation, with most of my clients requesting long-term contracts for my services in excess of 5 years.

• My business is successful, reaping seven figures a year, and is

primarily focused on niches within the online information and software sectors.

A willingness to always learn, plenty of self-sacrifice, and most importantly, an emphasis on building strong relationships throughout my career, has without question been the cornerstone of my business success. Away from business and consulting, I am pretty much a regular guy. I enjoy the outdoors with a healthy dose of action-related sports including quad biking, jetskiing, and body boarding. I have a passion for animals, real estate, and investing. I am somewhat of a social butterfly, and always enjoy meeting new people. I believe in working hard, being loyal, and honoring commitments; I add more value to relationships than I take, and most of all, I believe that true success comes from dedicating effort to one single cause at a time.

Page 6 of 55


http://www.justinharrison.com/

http://www.justinharrison.co.za/

1. Split Test Master Guide Introduction:

1.1. Why this Guide? I am an aggressive split testing professional. This is what I do for a living in my business and for clients. I know first hand what a dramatic impact split testing can have on the bottom line of a business. The problem with split testing is that, for the most part, there appears to be nothing “sexy” about crunching numbers. Most online businesses either ignore the importance of split testing altogether, or they fall into the trap of thinking that they know enough to simply jump in and test. Split testing on the surface is one of those subjects that may seem overly simplistic. Scratch a little deeper though, and you are bound to find yourself wrapped up a in a web of technical acronyms and “geeky” plurals. Additionally, the internet is filled with so-called “gurus” who purport to know more than they actually do. This is perhaps the most alarming fact of all because it translates into misinformation for the people who need it most. I have always lived my life by a simple philosophy, which really is at the root of what split testing is all about.

“I believe that if you don’t take care of what you have (no matter how little that might be), then the universe won’t give you anymore.”

Now that might sound a bit too philosophical, and I am not trying to impose any of my beliefs on you, but that statement really does resonate with the entire concept of improving the bottom line of your business. You can get lots of traffic, but if you don’t optimize your conversion process your costs will eventually rise proportionally to the amount of traffic you get, which will eventually lead to less money in the bank. After 9 years in this business I have learned that no matter what the small steps are in between, or the greater vision of your online project, it all ultimately boils down to traffic and conversions.

Page 7 of 55



The problem is that so many business owners are putting “the cart before the horse”. Their entire focus is on more traffic, more visitors, and more advertising, when in fact it should be on optimizing their offers. Obviously, you need some traffic before you can start optimizing. However, before you go out and start a full blown marketing campaign, you should have already tested and optimized your offers.

Knowing and doing are really two different things, and the Split Test Master Guide will prepare you for success by not only enabling you to fully understand how to correctly split test, but most importantly, how to interpret the results. Regardless of your experience, I fully encourage you to read this guide from top to bottom and to make notes along the way. As a final word, I would like to point out that I can (at times) agree that split testing is not the “sexiest” subject around, in fact, it can be downright boring to some, so I will do my utmost to keep the reading light. That being said, I personally think that any effort or event that results in your bank account growing is “SUPER SEXY”, so excuse me if I get a little overzealous at times; I hope some of my enthusiasm rubs off.

1.2. Why Split Test? The obvious reason for split testing is to increase conversions, and in so doing increase revenue (money in the bank). However, there are quite a few less obvious reasons as to why you should split test.

Ultimately it all comes down to more money in the bank. Like split testing, business is about taking one step at a time, and this is where refining your process and optimizing your business becomes essential. Here are 6 examples as to why you should split test, aside from simply seeking to increase conversions to get more money in the bank:

1. Advertising Costs: Split testing decreases your advertising costs and decreases the cost per client, or per lead, acquisition.




The affect is you can grow your advertising budget organically by ensuring adequate margin in your conversion cost.

2. Traffic Targeting: Split Testing your various forms of traffic allows you to accurately target the traffic that converts best, and enables you to make important decisions about which traffic sources to invest in.

3. Operating Costs:

Split Testing allows you to drive down operating costs as you refine your traffic and lead sources by ensuring your business only targets prospects that are most likely to do business with you. This means fewer phone calls and email inquiries from prospects that are unlikely to do business with you.

4. Affiliate Retention:

One of the biggest mistakes made by merchants is putting an affiliate offer to market before it has been optimized. An affiliate offer that converts well will ensure affiliate retention.

5. Market Intelligence:

Split testing enables you to gather key intelligence on your market, which can be used to formulate important decisions. For example, you might discover that your market responds to strong risk reversals as a single element in your copy, which would then prompt you to use more social proof and testimonials on your site. Market Intelligence is one of the most overlooked aspects of split testing. This is a useful way to get to know your market.




6. Product Development: Split testing gives you strong insight into which products and services your prospects and market responds to, and that enables you to make informed decisions regarding future products. For example, if you had a site that sold DVD’s and CD’s and you discovered that your DVD section converted 300% higher than the CD section, you would make the decision to prioritize your time and resources on building your DVD section.

On the surface, split testing seems straight forward; however, nothing could be further from the truth. Split testing is, without any doubt in my mind, the single biggest opportunity to grow and establish any business.



2. Split Testing Basics:

2.1. What is Split Testing?

Broadly, the term “split testing” refers to testing one page against another with the objective of discovering which page converts better.

Illustration 1 – A Page Split Test

More specifically, the term “split testing” refers to testing one “element” or a set of elements against each other, with the objective of discovering which element or group of elements converts best.

Split testing is one of the most powerful strategies for increasing the effectiveness of any web based offer.

2.2. Important Split Testing Terminology

Protocol:

A test “protocol” refers to a plan for your test. It describes specific methods and rules for your test environment, such as your split test name, test dates, test type, and most importantly, your test goal. The Control: The control refers to the original page or element that you used. Typically a new version is tested against the control. Once you find a page or element

Page 11 of 55



that converts better than the original control page, or element, that becomes the new control. The Signal: The signal is your baseline conversion rate on your control page. You must have a signal to test against in order to be able to determine if your test decreases or increases conversion. Sample Size: Sample size refers to the number of hits each page or element will be exposed to before making a statistically valid decision.

Action: An action is any step or process that requires visitors to make within the conversion process, such as subscribe, buy, click-through, etc. Conversion Rate: The conversion rate refers to the ratio of unique visits (hits) to a page in relation to the number of desired actions resulting from those visits (hits), usually expressed as a percentage.

Noise: The term noise refers to any variable outside the test parameters that will have an influence on results, such as dips or spikes in traffic, or a change in traffic sources. Noise is one of the biggest challenges faced in any test. Lift: Lift refers to the percentage increase between the control page or element and any new page or element.




Latency: The term latency in a split testing environment refers specifically to a delay. Latent Actions: The term latent action refers to actions that happen after the first contact by a visitor. For example, if a visitor goes through the split test today and only takes action upon return tomorrow, this is known as a latent action.

2.3. Key Testing Metrics

There are many well known testing metrics such as click-through rate and conversion rate, as well as many less known metrics such as engagement rate and pageviews per session. However, not all of these metrics are of importance and many of them are simply useful for making various marketing assumptions, while others are critical to optimization and split testing performance. Below is a list of key testing metrics that you need to be familiar with:

Conversion Rate: The conversion rate is the ratio of visitors (prospects) to actions (conversions). This conversion could be a sale, an opt-in, or any other action that you require visitors to make, and the rate is always a measure of unique visitors to unique actions. Conversion rate is calculated as follows:

Illustration 2 – Conversion Rate




For example:

If your website received 1000 unique visitors which resulted in 123 newsletter signups that would equal a conversion rate of 12.3%.

“123 divided by 1000 multiplied by 100 equals 12.3%

When and where to use it? Conversion rate tracking is valid anytime you want to track and improve the performance of specific actions that you want your visitors to perform, such as subscribe, buy, register, etc. If you have a multiple page ordering process, you can also use conversion rate tracking at each stage. This will enable you to monitor not only the overall sales conversion, but the individual processes within the “checkout” process itself.

Click-through Rate:

Most people are familiar with the term “click-through rate” because of the popularity and frequent use of this term in pay-per-click advertising. The click-through rate is the ratio of the number of people who viewed a link (known as impressions) to the number of people who clicked on a link (such as a banner, button or text link). The click-through rate is always a measure of unique impressions to unique clicks. Click-through rate is calculated as follows:

Illustration 3 – Click-through Rate




For example:

If your advertisement received 1000 unique impressions which generated 75 clicks, that would equal a click-through rate of 7.5%


When and where to use it? Click-through rate is not only valid in your advertising that leads to your site, such as PPC and banners, but it is also valid within the context of your site. This is especially true for affiliate marketing or for monitoring the performance of internal links that lead to your own offers. Specifically, click-through rates can be used to optimize the performance of links on your site.

Engagement Rate: The engagement rate is a metric that is used to quantify the ratio of people who spend enough time on a page to have “received the core message”. Naturally, this metric is somewhat subjective, but a good measure is to work on the basis that if someone spends more than 5 seconds on a web page they have had enough time to read the core headline, thus receiving the core message of the page. Engagement rate is calculated as follows:

Illustration 4 – Engagement Rate

For example:




If you had 6500 visitors on your website and 1000 of them spent more than 5 seconds on your landing page, this would indicate an engagement rate of 15.3% on the landing page.


When and where to use it? Although most split testing focuses heavily on conversion rate testing, in my experience the single, biggest improvement in overall conversion comes from improving your engagement rate. You should use this metric to test the effectiveness of your site design, load times, and most importantly, your page headlines.

Average Pageviews per Session: Average pageviews per session is a measurement of site “stickiness”, and indicates how effective your site is at keeping your visitors on the site. Pageviews per session is calculated as follows:

Illustration 5 – Pageviews per Session

For example:

If your site receives 1500 pageviews from 100 visitors, this would amount to an average of 15 pageviews per session.

“1500 divided by 100 equals 15”

When and where to use it?




Average pageviews per session is typically useful when your primary focus is on selling advertising space, since the more pageviews you have the greater your advertising inventory is. You should use this metric if you sell advertising space and you are reliant on the number of pageviews. Typically, this metric can help you improve your navigation structures and your site hierarchy design.

Average Session Time:

Average session time is another measurement of site “stickiness”, and indicates how effective your site is at keeping visitors on your site. Average session time is calculated as follows:

Illustration 6 – Average Session Time

For example:

If the total time of 100 visitors on your site is 400 minutes then the average session time equals 4 minutes per session.

“400 divided by 100 multiplied equals 4”

When and where to use it? As in the case with pageviews per session, average session time is a useful metric for optimizing your site for selling advertising space. This is especially true if you run a site that revolves banners every couple of seconds, since higher-than-average session times will give you more advertising inventory. Typically, this metric can help you improve your navigation structures, content, and your site hierarchy design.

Average Order Value:




Average order value is a measurement of the average monetary value of transactions occurring on a site. Average order value is calculated as follows:

Illustration 7 – Average Order Value

For example:

If your website received 28 orders, which resulted in total revenue of $3,300.00, then the average order value equals $117.86.

“$3,300.00 divided by 28 equals $117.86”

When and where to use it? Average order value is particularly useful in an eCommerce scenario where you have multiple products, or if you sell single item products with up sell or cross sell options. Average order value, as a metric, is very useful for increasing your maximum revenue per sale, and increasing your total profitability.

Revenue per Visitor: Revenue per visitor is a measurement of how effective your site is at monetizing visitors, and it describes the value of each unique visitor. Revenue per visitor is calculated as follows:




Illustration 8 – Revenue per Visitor

For example:

If your website received 1100 visitors, which resulted in total revenues of $3,300.00, then the revenue per visitor equals $3.00.

“$3,300.00 divided by 1100 equals $3.00”

When and where to use it? Average revenue per visitor is one of the most important metrics available because it allows you to make key decisions about purchasing or investing in additional traffic streams. By knowing what each visitor is worth, you are able to purchase traffic (clicks) at a lower rate than you currently make per visitor, and in so doing keep your traffic streams profitable.

2.4. Always test against your control page

In any split test it is imperative to have an existing result to test against, also known as testing against your control (the original version). In order to be able to make any accurate assumptions about split test results, you must first have a baseline result to measure against. In most instances, when you begin a new split test you will have the original version that you want to test, but you might not have any data on the actual performance of that version. In this instance, it is wise to setup your split test and define the original version to send your traffic to, as this will accumulate click and conversion data, and that will form the baseline of your control data.




Additionally, once you begin split testing; ALWAYS include your control page. This will expose the control page to the same noise and elements within the testing environment as your new page.

2.5. Making Your Split Tests Statistically Valid

All forms of split testing rely on statistical probability, measured and expressed as a percentage of certainty. As a general assumption, the percentage of certainty is commonly and mistakenly assumed to be 100%, which by mere virtue of the fact of “statistical probability” is not possible.

For Example:

If we split test Page A against Page B and page A converts at a 5% higher rate than page B, we can, as a matter of certainty, assume that page A will continue to convert at a 5% higher rate. “This assumption relies on statistical probability”.

The problem is that over time, things change. Market forces and various other influencers can have an impact on that “statistical probability”. In essence, the “percentage of certainty” of any statistical probability is a variable amount, and remains somewhat subjective.

For Example:

As in the example above, depending on how tests are produced and preformed, you may be 90% or 95% certain that page A will continue to perform 5% higher than page B.

”A 100% certainty is not possible”.

Accordingly, in order to produce the highest percentage of certainty within your statistical probability testing (split tests), it is important to pay special attention to certain key factors (known as variables) within your tests to ensure that you can make accurate “assumptions”.




The following are key factors (variables) you need to consider in factoring the statistical probability and validity of your tests:

Sample Size:

Ensure that your sample size is large enough to be considered statistically valid. This is obviously subjective and relevant on a case by case basis. However, a general rule is to have sampled at least 20%-30% of your average monthly traffic for your statistical assumptions to be valid.

For Example:

If your site receives 1000 visitors in a month, you should aim to have sampled at least 200 to 300 hits to both your control page and your new page.

Obviously, the longer you run your test, the more statistically valid it will be, and the closer you will get being 100% certain (although this cannot exist) the results are statistically probable.

Sample Size Consistency: In order to make an accurate assessment and assumption between your versions within a split test, it is important to have a level of consistency in your sample size.

For Example:

It is not statistically sound or valid to draw a comparison between a page that has received 1000 hits versus a page that has received only 25 hits.

Since the control page will have more hits than the new version in any split test, it is suggested you expose the new sample page to at least 30%-50% of the hits that you generated to your control page.




Test Distribution Consistency:

One of the biggest mistakes made in split testing is the lack of consistency in the distribution of test elements (pages). It is of paramount importance that elements (pages) are distributed evenly between visitors.

For Example:

* Visitor 1 gets Page A * Visitor 2 gets Page B * Visitor 3 gets Page A * Visitor 4 gets Page B ”An even distribution in your test ensures consistency in results”.

Additionally, it is just as important that repeat visitors (latent actions) are exposed to the same split test route (recipe) that they first encountered to keep test results consistent (usually done via a cookie).

Noise and Traffic Consistency:

One of the biggest reasons for inaccurate and invalid split test data is noise and inconsistent traffic patterns.

For Example:

If during a split test the versions in your split test are exposed to an untargeted email campaign, this would result in an overall watering down of conversion data. In this case, testing new versions the following week is likely to produce invalid and inconclusive results.

Reducing noise in your split tests, and having consistency in your traffic patterns during any test, is the key to statistical validity.



3. What to Test:

3.1. What should you be testing?

The general rule is “anything that can be tested should be tested”, however this list can become somewhat impractical at a point, so it is useful to have a quick list of items that you should be testing. Some of these elements will be specific to lead generation landing pages (also known as squeeze pages). Others will be valid to sales pages and some will be valid to all types of pages.

Copy Specific Elements:

• Headlines • Sub-headlines • Opening statements • Bullet points • Copy length • Copy tone • Risk reversal (the guarantee) • Scarcity factors (limited time offers) • Social proof • Calls to action • Price points • Product descriptions

Design Specific Elements:

• Use and placement of images • Navigation structure • Design and layout • Page size and loading time • Order form layouts • Order buttons and text

Trust Specific Elements:

• Security seals and logos • Placement of security seals and logos

Page 23 of 55



• Use and placement of live support icons • Use and placement of telephone numbers

3.2. Test Scenarios

Each split test scenario is unique; it starts with defining a goal and knowing what to improve. It is particularly useful and important to create a conversion map (described in detail in chapter 6). However, there are some common test scenarios that are performed during split testing and optimization:

Lead Generation Page Testing:

Lead generation pages are often referred to as “squeeze pages” and these pages are designed to “push” (shepard) your visitors into signing up for your newsletter, or to place their contact details on your prospect list. Typically, these kinds of landing pages go hand in hand with some type of free offer in exchange for your visitor contact information. Contrary to many expert opinions and many of my colleagues, I believe measuring the conversion rate of a lead generation page as a single element, without factoring in your entire sales funnel, is actually counter-productive, since these lead generation pages are essentially designed to eliminate untargeted traffic and prospects. Essentially, the process of conversion tracking without monitoring the overall result of actual sales is totally counterproductive. A higher lead conversion rate could dramatically reduce your sales, while the opposite could also be true. Accordingly, it is always my advice that when doing conversion testing on your lead generation pages, you keep a close watch on your sales conversion rates as well.

Single Page Sales Copy:

Single page sales copy, normally referred to as a “sales page”, is fairly easy to test. You are expecting the user to make one of two choices, either to buy or reject your offer.




However, one of the bigger challenges faced in testing sales copy, or more specifically long sales copy, is testing the affects of single element changes since there are so many variables and copy elements associated with long sales copies. This is where metrics, such as your engagement rate and average visitor session time, become imperative to testing the effectiveness of sales copy, and establishing which copy and design elements have the greatest individual impact on the overall result.

Limiting your testing variables in long sales copy is of particular importance and will also help give you valuable insight into the mindset of your market, as you see which elements your visitors respond strongest to.

Multiple Page Sales Copy:

Conventionally, single page sales copy converts much better than multiple page sales copy, and it is my personal advice that you always try to keep your core message to one page at a time. However, there are times when there is so much information that needs to be conveyed in the sales process that it becomes feasible from a conversion perspective to spread your copy across multiple pages. In instances where you have a multiple page sales copy, each step (page) should be seen as a conversion point and monitored accordingly, since the goal of the previous page is always to get the visitor to take action and progress to the next step (page). However, it is vitally important that when you monitor individual conversion points within a multiple sales page copy, that you factor these conversion points back to the overall key conversion objective (sales).

Checkout Processing:

Checkout processing, which is typically experienced in an ecommerce environment, is much the same conversion monitoring process as in the case above. As a rule you will want to track and monitor specific checkout points (add to cart, confirm order, etc.), then factor these individual conversion points back against the overall key conversion objectives (sales).




Key Point: Conversion lifts across multiple conversion points compound the overall conversion result. This is one of the biggest secrets to getting huge conversion lifts.

Home Landing Page: The home page is one of the most difficult pages to test and optimize because the purpose of the home page is rather generalized. However, this does not mean you should not test your home page. Generally, traffic to a website’s home page is quite diversified. It shifts in traffic patterns, such as search engine ranking and keyword positioning, which produces a significant amount of noise within the testing environment. Additionally, hits to a website’s home page are not always from targeted prospects, and you will often find competitors and job hunters frequently landing on the home page of a website. This results in testing data inaccuracies. Perhaps most importantly, any changes made to your home page could have a dramatic impact on your search engine rankings. Changes can be risky and should always be carefully considered. However, in spite of all of this, a website’s home page provides the benefit of accessing a large sample size (many hits), and traffic diversity. Although this does present a challenge to test, it can provide a useful broad base sample. A website’s home page has the potential to make the single biggest impact (impression) on visitors. It is certainly worth undergoing the testing challenges. When deciding on how, and if, to undergo home page testing, it is vitally important to consider all aspects and to review your goals.



4. Different Types of Split Tests:

4.1. A/B Split Testing Explained

A/B split testing is one of the simplest forms of split testing. This technique enables you to compare two page variations against each other. In an A/B split test environment the audience (visitors) of a marketing message are divided between two variations of a message. Real time calculation of the performance results provides instant feedback on which version is performing best. A/B split testing is also called OFAT testing, (one factor at a time), on account of the fact that A/B split testing works best when you make single variable changes between split testing versions.

Illustration 9 – A/B Split Test

4.2. Multilevel Split Testing Explained

Multilevel split testing, or A/B/C+ split testing, is almost exactly the same as traditional A/B split testing, except that there are multiple (more than 2) creative recipes being tested. As in the case with traditional A/B split testing, it is always best to test single variable elements (i.e. the headline, price, etc.) between all versions.

Page 27 of 55



Additionally, Multilevel split testing should only be used on pages with high traffic volume, as this will enable an adequate sample size within a short period of time in order to establish a credible result.

Illustration 10 – Multilevel Page Split Test

4.3. Multivariate Split Testing Explained

Multivariate split testing, which is also referred to as Multivariable split testing, is an advanced method which enables you to split test more than one page variable (element) at a time. Multivariate split testing uses mathematical formulas (algorithms) to rotate, test, and measure the performance of multiple variables within a single page, and conclude which group of variables performs best. Multivariate split testing is an effective way to rapidly split test the relationship between page elements, providing a view on obtaining the best converting set of variable elements.




In essence, Multivariate split testing shows which combination of elements converts best.

4.4. Which Split Testing Method Should You Use?

There is debate about whether or not A/B split testing has any place in the modern realm of conversion testing. Many fellow experts and authorities suggest that A/B split testing is somewhat archaic, and that the ONLY way of the future is Multivariate split testing. Despite all the debate, hype, and overuse of fancy terms, I am here to dispel all myths and tell you the facts in plain and simple layman’s terms. The reality is that each split testing method has its place and application. Traditional A/B split testing is just as effective as it has always been, and it will continue to be for as long as the internet is around. Additionally, Multivariate split testing has its own place and application. However, it is NOT an improvement on traditional A/B split testing but rather an additional resource in your testing kit. It is inaccurate to draw any comparisons between Multivariate split testing, and A/B split testing, since these two have very different applications.

A/B split testing seeks to test single conversion elements against each other. Multivariate split testing seeks to optimize (group) the best set of elements together.

Both methods of split testing have their own application. One method might be superior whereas the other could quite well be inferior. Much to the surprise of many of my colleagues, in 85% of my test cases I prefer traditional A/B split testing or Multilevel (A/B/C+) split testing. Not because of any personal opinion, but because I have proven over the past 9 years that this remains one of the most effective testing methods. A/B split testing is useful for simple tests which need to be launched quickly, and will provide you with useful market insight as you discover which elements your market responds to. Multivariate split testing on the other hand is very useful in ecommerce environments where A/B split testing is simply too time consuming. It then




becomes practical to test several hundred variable scenarios simultaneously in order to establish the best variable relationships. I frequently use Multivariate split testing in demanding ecommerce environments, and with detailed planning, Multivariate split testing is certainly a very useful tool.

Generally, ecommerce environments are most suited for Multivariate split testing because they rely heavily on the relationship between elements and this form of split testing can greatly increase the speed of your testing. However, one of the major drawbacks with Multivariate split testing is that it takes major planning and implementing. Multivariate split tests run the risk of testing too many combinations. This can result in tests having to run for many months before running through all the variable grouping options. Whoever said “there is genius in simplicity” was certainly right, and when it comes to split testing, you will do well to remember this. More often than not, you will find your need will bring you back to traditional A/B split testing and Multilevel (A/B/C+) split testing.



5. Designing Your Split Test:

5.1. Introduction

Designing your split test can seem like a daunting task, however if you know what you want (the goal) and you combine that with a logical approach (the plan) then you are likely to find it a breeze. Since I have highlighted that most of your split testing will be based on A/B split testing and Multilevel (A/B/C+) split testing, the remainder of this document will focus on these forms of test environments. If you run a large ecommerce operation, then you should consider hiring a professional to plan and implement your Multivariable split test, as this can be a cumbersome and overwhelming task.

5.2. Conversion Mapping

Conversion mapping is one of the most important split testing steps, and without knowing and breaking down your conversion process, it is nearly impossible to effectively implement a split testing plan. Conversion mapping might seem like a laborious and time consuming task, but the truth is that a little planning upfront can literally save you months of work down the line. I like to think of a conversion map as “a plan for guiding people through the sales process”. This is often also referred to as the “sales funnel”.

Page 31 of 55



Illustration 11 – Basic Conversion Map

The example above shows a very simplistic conversion map; you have traffic on your website, the traffic arrives at your landing page, the traffic is required either to opt-in or purchase.




In the example below, you can view a more detailed and intricate conversion map, which will help you develop your own conversion mapping strategies.

Illustration 12 – Advanced Conversion Map

As you can see in the example above, traffic is funnelled through a conversion process, towards the end goal (sale). However, in between getting the traffic and making the sale, there are a number of conversion points (steps) that can and should be tracked. This includes traffic types, traffic sources, landing pages, sales pages, and order processing, to name but a few.




5.3. Defining Your Target Goals Once you have developed your conversion map, defining your target goals is easy. The best way to think about target goals is to consider the actions you want your website visitors to take (i.e. where do you want them to end up on your website). For example, if you are running a squeeze page as your landing page, then your target goal (action) is to get your visitors to opt-in or provide you with their contact details. The tricky part about defining your target goals is that actions can only be tracked after the goal has been actioned. This is where many people new to split testing get confused.

Illustration 13 – Defining Target Goals

Once you have a clear conversion map with your target goals defined then you are ready to begin split testing.

5.4. Testing With Traffic Consistency

As you may have gathered, eliminating noise in your split tests is critical to making accurate assessments on your split test results. One of the best ways to do this is to use consistent traffic sources to gain statistical validity. One of the best ways to do this is to use pay-per-click advertising such as Google Adwords or MSN Adcenter.




Of course, there are a number of other pay-per-click services. I would recommend you use Google Adwords, as this has proven to be the most consistent traffic sample. Additionally, I would advise you only use one pay-per-click service at a time to test and sample, because a mixture of more than one source will often lead to imbalanced results. Another good idea when you design your split test is to setup a separate and unique test environment (landing pages, sales pages etc) that will not be exposed to general traffic on your website. This will allow you to not only expose your split test to a very constant traffic source but will enable you to make accurate assessments.

5.5. The Biggest Secret to Successful Split Testing

As I have mentioned before, A/B and Multilevel (ABC+) split testing remains one of the greatest split testing formulas. However, the opposite is often reported by naïve “experts” who overlook one of the greatest and most powerful aspects of A/B split testing. The primary focus of your business is to get to know your target audience better than any of your nearest competitors, and most savvy businesses know that market research and extensive customer surveys is key. It stands to reason that being able to interpret your websites visitor and buyer behaviour holds a lot of merit in the overall objective of getting to know your prospects and customers. I have dedicated an entire chapter (chapter 6) to interpreting data, , however, one of the biggest hidden secrets to split testing is knowing which elements your audience reacts strongest to. Most split testers simply test to find out which version converts better, with the single-minded approach of increasing conversions. One reason why I have become one of the most sought after split testers, is the fact that I have chosen to look beyond the stats (if you will) and find the reasoning behind results. Despite the fact my colleagues and even some clients say that A/B testing is dead and out-of-date, here lies the secret and truth as to why this will never be the case.




A/B split testing requires some level of manual testing in comparison to Multivariate testing; it requires you to test one element at a time. While this might seem like the greatest weakness of this testing method, it is in fact one of its greatest strengths for those who understand its value. Over the years I have consistently outperformed my colleagues and competitors who have used the more “automated” Multivariate testing method. Most are still at a loss as to why this is the case. By testing single variable elements (although it is a process) the time and effort you invest in getting to know your market is truly invaluable, and over time you will see various patterns emerging. Although legal agreements and restraint of trade prohibits me from mentioning any specific names, there is one example that I will never forget. It goes to show how important getting to know and understand your target audience is. One of my global financial service clients has, over the years, aggressively used split testing to increase conversions on their products. They were rather successful in their objectives prior to my involvement, managing an impressive 31% lift in conversion. They were using a combination of Multilevel and Multivariate split testing, which ran almost 100% automatically and with very little human interaction or data interpretation. Naturally, when I began testing and much to everyone’s surprise, I reverted to traditional A/B split testing. For the first few months I had marginal success, with small increases and decreases here and there. However, during these increases and decreases, certain patterns began to emerge. Because I was manually split testing elements such as headlines, security statements, risk reversal, etc., I started to understand which elements had the greatest impact on their target audience. Needless to say, the more “automated” Multivariate split testing was, which successfully grouped the best elements together, the more it was unable to interpret the actual mindset of the audience in order to suggest key changes or areas that required emphasis. The result was that I noticed a pattern, whereby the target audience responded more acutely (both negatively and positively) to the variations on a specific test element.




This test element happened to be what we call the “hero shot”, best described as a combination between an image of a person and introductory text designed at “personalizing” the visitor’s experience. In testing image variations, I noticed inclines and declines whenever the age of the person in the image varied noticeably. Accordingly, I decided to find out the average age of prospective clients by checking the lead database and found the most popular age group was 35-45. I decided to match the age profile of our hero shot to the prospects visiting the site. I theorized that neutrality in the character presented was of key importance because it would not challenge or impair the prospect’s attention, and it would better serve the client’s objective of coming across as personable and approachable. The result was an initial lift of 26% across the site. But what’s more important is that this one element brought us closer to understanding what prospects in this specific market wanted and needed. While trust, risk reversal, and all the other elements were just as important, this one element suggested that the target audience needed, wanted and demanded a more personal tone of communication. As a result we simplified the terminology, changed the copy tone, and even threw in a bit of “slang” (amidst strong challenges from the company’s corporate executives) to make sure we connected with prospects. And the result… a staggering 1276% lift.



6. How to Interpret Data:

6.1. Introduction Data interpretation is one of the most important facets of split testing. If you fail to correctly interpret the results, or worse don’t interpret your split test data at all, then you could spend years going around in circles. You can, and will, receive marginal gains without doing any data interpretation. You might even experience the occasional fluke where you are able to drastically improve your results. However, if you want to consistently increase conversion, you must accept and learn to do draw conclusions and interpret the data from your split test results, as this is the only sure way to increase your conversions. As important as it is to interpret data, it is equally important to correctly interpret this data, as incorrect theories can, and will, lead to decreased conversions. Fortunately, split testing and data interpretation has a self mending approach. If you act upon incorrect theories, your split test will always reveal this approach. You don’t need to be a rocket scientist to get it right. Split testing and data interpretation is as logical as breathing, and the more you do it, the better you become, and the faster you will learn about the single most important part of your business… the customer.

6.2. Calculating Your Lift As you progress through your split tests you will have a running aggregate of the split test (average conversion percentage of pages combined) as well as the actual conversion percentage of each page. However, you may find it useful to manually calculate the lift (conversion difference between pages). This will give you an objective idea of how one page (version) has preformed against another.

Page 38 of 55



Illustration 14 – Calculating Your Lift

For example:

If Page A (the old page) converts at 1.34% and Page B (the new page) converts at 2.16% then the conversion lift from the old page to the new page is 61.2%

“2.16 less 1.34 divided by 1.34 Multiplied by 100 equals 61.2%”

6.3. The Noise Factor Noise is an unavoidable factor in any split test because it’s nearly impossible to limit noise altogether. As already mentioned in this guide, there are certain steps you can take to prevent noise in your tests. However, the fact remains that there will always be variants in your traffic patterns. For example:

• Time of day • Day of week • Season • Holidays • Negative / Positive media on your industry • Natural disasters • Etc.

It is very important that when you notice big changes in your split test results (both up and down) you check to see if you can attribute this to any outside influences (noise) other than the reasoning of one page or element versus another. Many experts (myself included) have celebrated early victories or losses in their split test results, only to discover the true result of a test was a result of noise factors.




Take your time when considering and comparing results and don’t be hasty to draw conclusions.

6.4. Making Allowance for Latent Conversions Latent conversion occurs when someone visits your website, and upon being presented with your “action page” (landing page, sales page etc), decides to leave your site, returning later (hours, days, or even weeks) and then performs the action you required on the first visit. Latent conversions happen more regularly than you would expect, so it is very important you make allowances for this in all your split tests. The first and most obvious step is to ensure that returning visitors are always presented with the same version of the original page they received upon first contact on your site. This is done via a cookie, and although there is a margin for error, and some visitors may clear their cookies before the next visit, this step greatly reduces imbalances in your results. The most important aspect is interpreting your latent conversions. Firstly, you need to always check the percentage of latent conversions versus your immediate conversions. Some products naturally have latent conversions. However, if you are expecting immediate conversions and a large number of your conversions are latent, then you need to look closely at your copy and processes. Unexpected latent conversions often indicate some form of distrust or misunderstanding from your target audience. Secondly, when assessing your results, always allow a few days grace period (or the equivalent time based on your average latency), especially for sales and lead funnels that have a higher percentage of latency, as this will ensure the accuracy of your results.

6.5. Interpreting Your Key Metrics

Each of your key metrics provides a valuable and unique way to look at the results between your tests. Each metric has their own purpose and objective, making it important to consider them collectively.




As you have probably gathered by now, data interpretation is one of the most important and challenging aspects of split testing. Each conclusion, opinion, and theory is subjective and open-ended. For this reason it is imperative that you factor your key metrics in conjunction with one another against the overall result, as this will keep your hypothesis honest and true. Let’s quickly recap your key metrics:

• Conversion Rate • Click-through Rate • Engagement Rate • Average Pageviews per Session • Average Session Time • Average Order Value • Revenue per Visitor

The mere nature of split testing relies on variable elements and there are a limitless number of variations and possibilities for both the outcome and your hypothesis. This is why it is near impossible to develop “copy and paste” split testing plans. Rather than train you how to be a “parrot”, I think there is far more value in learning how to think and discover for yourself. As I mentioned, the variations and variables are limitless, but to get your “split testing juices flowing” here are a few quick examples of how observing and factoring your key metrics against the overall result can help you become a split testing wizard.

Example 1:

Let’s assume you are split testing a squeeze page, with the main objective of capturing your visitors name and email address. By looking at not only your conversion rate between split testing versions, but also the engagement rate, and your average visitor session time, you will be able to discover which elements and parts of your message have the greatest impact toward the actual conversion objective. Depending on the message and objective, a shorter summary message or longer, more descriptive message within your copy may be required. These metrics will help guide your ideas and strategies.




Example 2:

Let’s assume you are split testing a squeeze page, with the main objective of capturing your visitors name and email address.

By looking at not only your conversion rate, engagement rate, and average visitor session between split testing versions, but also your click-through rate from advertising leading to your landing page, you will be able to determine the effectiveness of your advertising message.

“More clicks don’t always mean more conversions”

Example 3:

Let’s say you are split testing a sales landing page with the objective of increasing your overall sales conversion. By looking at not only your sales conversion rate, but also your average value per visitor you will be able to work out what each visitor is worth, and what each traffic source is worth against the other.

6.6. Monitoring Visitor Activity

Monitoring your visitor’s activity and interactivity with your site and landing pages is one of the hidden gems for learning how to create better landing pages and sites, and therein lies the key to better conversions. Monitoring visitor activity is always manual labour, and this is one of the reasons why so many people overlook this important aspect. However, if you spend the time getting know how your visitors interact with your site, then you will be handsomely rewarded. For example, do you currently know your major entry and exit pages on your website? Do you know which pages your visitors spend more time on?

On the outside, it seems like meaningless, run-of-the-mill, boring statistics. However, if you take the time to look into these stats, you will discover interesting sociological and behavioural patterns that you can use not only to improve conversion, but improve your visitor experience.




The first rule of business is “know your customer”. Monitoring your visitor activity and behaviour is as close as you will ever come to knowing and developing a deep understanding of your customers. Wherever you can, you should and must spend time monitoring the visitor activity on your site. In fact, I have made this a daily routine within my business, and I can tell you it is worth every second invested.

6.7. The Relationship between Split Tests and Analytics Obviously, there has and always will be, a direct correlation between split testing and analytics (general web statistics). The problem is that as split testing has become popularized, the two have become viewed as separate entities all together. There are now even stand-alone split testing packages or scripts, and however well intentioned these “solutions”, they often cause more harm than good to those people using them. The problem with stand-alone or “independent” split testing platforms is there is no immediate way to correlate or factor analytic results against split test results. Meaning, people are getting only half the picture at any given time. Here is a practical example. Do you think that a major drop off in organic ranking and traffic would affect your conversions? Or do you think that longer page loading times would affect your visitor engagement rate and ultimately conversions? Of course it would, and this is where having the whole picture is critical to your decision making. In order to be able to make key decisions, you must always be in a position to factor the relationship between your analytics and your split testing results.

The key is not just about knowing which version converts better or worse, but about knowing why they convert better or worse!



7. Important Considerations:

7.1. SEO Considerations

Perhaps one of the biggest challenges faced during both split testing and the results of a split test, is that at some point this will have an affect on your search engine rankings / positioning. If your website does not yet rely on organic traffic, then this is obviously not an issue. However, in most cases it is. I always recommend that you design a unique split test environment away from the core of your website, and that you run these tests via pay-per-click traffic, such as Google Adwords first. This enables you to avoid contestant changes to your “exposed pages” during your test, which would obviously affect your rankings and positioning. Additionally, it is important to consider the margin of lift within your split test when deciding to roll out a new version into the front end of your site. If there is a very small margin of lift, then you might be better off leaving your existing version intact, as changes could negatively affect your rankings in the short term and thus not be worth the sacrifice. Whenever you roll out changes as a result of your split test result, always look at the possible SEO impact and weigh up the odds before rushing to correct any changes on your site.

7.2. PPC Considerations

If you have established PPC listings that have a significant history built up within your PPC account, then you need to plan your split testing strategy carefully, or you could find yourself inadvently being penalized. The way traditional split testing works is to redirect and divide traffic between pages from a single URL.

Page 44 of 55



The problem is that even though Google Adwords and other PPC engines do allow redirected URLs to be used in campaigns under specific circumstances, often times you can end up being penalized.

Excerpt from Googles Terms of Service:

In line with our existing policy, we will continue to require that your ads display URL match its destination URL (the URL of your landing page).

For example, if your destination URL is www.google.com, your display URL must also be www.google.com. The following would not be acceptable display URLs for an ad for www.google.com:

We do understand that many advertisers utilize tracking URLs within the destination field of their ads. Therefore, if the URL of your landing page matches that of your display URL, your ads will be approved.

For example:

Display URL: www.google.com/adwords Destination URL: www.trackingurl.com/google123 --> Landing page URL: www.google.com would be acceptable

Display URL: www.google.com/adwords Destination URL: www.trackingurl.com/google123 --> Landing page URL: www.trackingurl.com would not be acceptable

We all know it takes time and work to create a perfect campaign, and your account history plays a major role in the overall success, bids, and placement of your ads, so a little caution is always good.

If you have an existing campaign running and you want to use the traffic from that campaign for your split testing, you have the following options:



http://www.google.com/adwords

http://www.trackingurl.com/google123

http://www.google.com/

http://www.google.com/adwords

http://www.trackingurl.com/google123

http://www.trackingurl.com/


Option 1:

If the traffic from your campaign goes to a single URL, then this is an easy workaround. All you need to do is create an .htaccess file that redirects traffic from your current landing page to the new split test URL. The great thing about this workaround is that there is no need to touch or change anything in your Adwords account, and there is no direct impact on your adverts or account history. Just remember that PPC engines are always on the lookout for abuse, so make sure the destination URL (where the visitor ends up) always matches the domain in the display URL of your adverts. Many affiliates try and abuse this system by advertising one URL then redirecting visitors to an entirely different domain, and this is where you can get severely penalized if you partake in this kind of redirection.

Option 2:

If you have multiple adverts leading to multiple landing pages, you may prefer to actually change the URL on the PPC advert itself. In this case, I recommend that you setup an exact copy of your ad (leaving the old advert as is) and adding your new split test URL to the new (second) advert in your campaign. This will ensure that existing ads are not affected and that your ads retain their history in your account. Just be certain to ensure that traffic is divided equally between the two adverts so that your new advert can begin building some history and data in your account.



8. Choosing a Split Testing Platform

One of the biggest questions posed by split testers, aside from which split testing software or script to use, is whether or not to invest in a hosted solution, or a server based solution.

The difference between hosted and server based solutions: A hosted solution resides on a service providers servers and you rent the use of the platform via the service provider.

A server based solution resides on your own server, and you own the software.

Both hosted and server based solutions have their advantages and disadvantages. Your selection will be based on technical abilities, technical resources, and unique business circumstances. Let’s take a quick look at the pros and cons of each:

Hosted Solution Pros:

• Quick and instant setup • No capital outlay on software and hardware • No technical resources required • No software and hardware maintenance • Access to automatic and free software updates • Access to dedicated service support

Hosted Solution Cons:

• You don’t own the software • Requires you to pay a monthly service fee

Server Based Pros:

• You own the software • No monthly service fees to pay • You have exclusive access to your data

Page 47 of 55



Server Based Cons:

• Requires significant capital outlay • Requires extensive setup and maintenance • Requires technical ability and resources • Requires ongoing self support • Requires installation updates

As you can see, each has its own merits and your choice depends entirely on your unique business situation. If you run a small to mid-size business that has 0-15 websites, then the hosted solution is usually the best option. On the other hand if you have access to dedicated resources such as competent IT staff, hardware, and you run a number of sites in excess of 15-20, then it becomes feasible to consider a server based option.

Note: Please see the resources section for details on recommended software, scripts etc.



9. Closing Thoughts:

Writing this guide presented some very real challenges, and I hope that I have done enough within these pages to educate and inspire you to bring fact and science to your marketing endeavors. Perhaps the greatest challenge in writing this guide is that the mere subject of split testing is rather technical and bland. I have done my utmost to steer away from overly technical terms, and tried at all levels to keep things simplified and to the point. More challenging than that, is the fact that there are no hard and fast rules to split testing, and certainly no handbook that can tell you exactly how and what to split test in your business. Just as split testing is about testing variables, so is the subject itself full of variables and options. In order to become good at split testing, you must prepare yourself to think and grow your interpretation skills, as these two elements are integral to your success. This guide alone is by no means a comprehensive or definitive handbook to split testing, but it will give you the basic foundations. I encourage you to stay subscribed, as I have made it my life’s cause to help businesses grow and prosper by using split testing to increase conversions and bottom line profit. To date, I have yet to find any other vehicle in a business that can so rapidly, methodically, and consistently grow the bottom line of a business. The results and the logic are undeniable. If you are not aggressively split testing your offers, actively monitoring your analytics, and constantly learning about your visitor’s behaviour, then you might as well be driving a bus down the wrong side of the road blindfolded. The time to start testing and tracking was yesterday!

Page 49 of 55


10. Resources:

Over the years I have used and purchased just about every analytic and split testing resource I could find. Some were good, and some not so good, but the one lesson I have learned is that there is no substitute for the right thing. In my mind, there is only one platform that is simple enough, yet effective enough to provide any real value. So much so that I have invested the future of my entire business in this software. It is the only platform I use to track and test both in my business and for all my blue chip clients. But before you get swept up in the hype of what I have just said, I want to express that this is no “magic bullet”. It’s just downright good software (that has a slight learning curve), but will provide you with all the essentials. There is still a great deal of improvement that can be done to this software, and the developers and I have a good working relationship that will hopefully continue to mold and evolve the software. I am not going to bombard you with hundreds of resources and options, nor am I about to draw comparisons with the software I use, versus any other options on the market. Suffice to say, this is the only software I will recommend, because I know from personal experience that it is the best.

Server Based Option:

If you are looking for a server based option and have the need, technical resources, and capital to invest, then my personal suggestion is Stuffed Tracker. Stuffed Tracker is developed by a group of developers in Russia and their work has made it one of the best split testing and analytics programs ever developed. The server based solution is rather expensive. There are scalable options available but your investment can be up to $1500.00 or more. The one thing you need to be aware of however is that aside from the software license and the hardware costs you incur, you will also need to pay a monthly or annual service fee for a GEO tracking database server such as GEOIP, to get detailed geographic stats.

Page 50 of 55


http://www.splittestmasterguide.com/review/stuffed.php



You can view my full review of Stuffed Tracker at: http://www.splittestmasterguide.com/review/stuffed.php Note: I have negotiated a special deal that is limited to Split Test Master Guide Readers which you can find at:


Hosted Option:

Unfortunately, Stuffed Tracker does not provide a hosted solution That’s why, after much consideration, I decided to negotiate with the team at Stuffed Tracker to setup and run a hosted version myself. This hosted version provides all the same features as the server based solution, plus as an added bonus, there are detailed training videos and resources that will only be available to members of the hosted version. The good news is that there is no capital outlay whatsoever, and for a nominal monthly fee you will get access to this world class software and the bonus resources aimed at drastically cutting the learning curve. For more information go to: http://www.splittestmasterguide.com/recommends/stuffed-online.php





http://www.splittestmasterguide.com/recommends/stuffed-online.php

11. Bonus Chapter

11.1. All time Biggest Split Testing Mistakes:

Over the years I have noticed that newcomers to the split testing world tend to make similar mistakes, so I hope that by listing the top mistakes here, you can avoid the same fate. Incorrect testing and mistakes lead to inconclusive results, and this ultimately leads to a loss of revenue. You would be wise to avoid the mistakes I am about to list below:

Mistake #1: Using ad variations on Google Adwords to split test:

I’ve actually seen this “free” method of split testing listed on some of the popular forums like “Webmasterworld” and others. The problem with this system of testing is that it is fundamentally flawed. The basic guideline of this testing is that you setup exact advert copies in your Google Adwords account, then split the traffic using the Google Adwords adverts. The problem is there is no system or way to recall the person’s original page or ad visit should they decide to return to your site, as is the case with latent conversions. This dramatically obscures results and leads to irrelevant conversion stats.

Mistake #2: Starting on the home page:

Despite the obvious challenges already mentioned in testing the home page, and the possible impact on search engine rankings, many novice split testers often choose the home page as a starting point. See chapter 3.2. Test Scenarios (Home Page Testing)

Mistake #3: Testing multiple variables at once:

Many novice split testers choose to test two completely unique pages against each other, and this means that there is no conclusive evidence as to why one page performs better or worse than the other. See chapter 5.5. The Biggest Secret to Successful Split Testing

Page 52 of 55



Mistake #4: Not running tests long enough: I have lost count on the number of times people have celebrated early victories or losses only to be disappointed or proven wrong down the line. For conclusive results you must let your tests run long enough. See chapter 2.5. Making Your Split Tests Statistically Valid

Mistake #5: Testing non-influential elements:

Many novice and experienced split testers spend a countless amount of time testing elements that either have little or no impact on conversions. This prolongs or sometimes totally avoids the goal of getting to the highest possible conversion. See chapter 3.1. What Should You Be Testing?

Mistake #6: Not interpreting the key metrics:

Split testing inherently revolves around increasing conversions, and because of this, many people ignore the importance of the relationship between key metrics and actual conversion. See chapter 6.5. Interpreting Your Key Metrics

Mistake #7: Ignoring latency:

Depending on the product or service, latent conversion can have a drastic impact on conversions, but many split tests don’t make allowance for latent actions. See chapter 6.4. Making Allowance for Latent Conversions

Mistake #8: Ignoring or not recognizing noise:

Noise is unavailable in any split test however, many novice split testers don’t take the time to find out if any of their split test results are affected by excessive experimental noise levels. See chapter 6.3. The Noise Factor




11.2. Landing Page Tips:

Any page on your website is a potential landing page, especially when you consider the amount of search engine traffic that arrives on the deep links within your website. However, the landing page tips I am going to provide below are specifically for campaign landing pages where you are driving high volumes of traffic directly at a specific page, for a specific purpose, such as lead generation, or to make a sale. How to create perfect landing pages:

• Keep it clean and simple • Avoid the use of links on the page if possible • Use bold attention grabbing headlines • Make your key benefit points quickly • Have a strong call to action • Make your opt-in / order form simple • Make your opt-in / order form recognizable • Use plenty of white space • Always ensure that your forms are above the fold

Additionally, the following elements have the greatest impact on conversions on a landing page: • Headline • Feature benefit bullets • Hero shot (Image) • Security seals and guarantees • Risk reversal statements • Scarcity generators • Calls to action

Always remember that landing pages are designed to instantly give your visitors what they are looking for when they arrive at your website. If you do this successfully your conversion rate will improve.



12. Get Paid to Promote This Guide

I wrote the Split Test Master Guide to help as many people as possible increase the net worth of their efforts and their business through split testing and increasing their conversions. However, I need your help in getting the Split Test Master Guide out there and into the people’s hands who need it most. I would like to invite you to become one of my joint venture partners, and, in the process, you can profit from helping to get the word out about the Split Test Master Guide. This way we all benefit! I will personally pay you $1.00 for every person you directly refer and who downloads the Split Test Master Guide. You can even recruit your own army of affiliates, and I will pay you an additional $0.50 for every person your affiliates refer to download the Split Test Master Guide.

For more information on this incredible opportunity and to register please visit: http://www.splittestmasterguide.com/jv/

Or click on the button below:

http://www.splittestmasterguide.com/jv/

Page 55 of 55





Documents

Split Test Master Guide · your costs will eventually rise proportionally to the amount of traffic you get, which will eventually lead to less money in the bank. After 9 years in