SEO split tests you should run - Will Critchlow

  • View
    7.265

  • Download
    0

Embed Size (px)

Text of SEO split tests you should run - Will Critchlow

  • SEO split testsyou should be running

    Will Critchlow / @willcritchlow

    BrightonSEO

  • Robert Liston famously:Carried out an operation with a 300% mortality rate

    Via: reddit

    https://www.reddit.com/r/funny/comments/1lz27p/robert_liston_performed_a_operation_with_a_300/

  • He amputated:1. The patients leg2. His assistants fingers

    Via: reddit

    https://www.reddit.com/r/funny/comments/1lz27p/robert_liston_performed_a_operation_with_a_300/

  • Both of them died,along with a bystander

    Via: reddit

    https://www.reddit.com/r/funny/comments/1lz27p/robert_liston_performed_a_operation_with_a_300/

  • Before germ theory, 25-50% of patients died from infections

    (Speed also used to be a prized surgical skill pre-anaesthetic)

    It wasnt always confidence-inspiring

  • Liston could amputate a leg in 2 minutes

  • Liston could amputate a leg in 2 minutes(but in his enthusiasm he once cut off the patients testicles too)

  • Welcome.Ill be your doctor today.

    Confidence inspiring stuff

  • The Liston of site migrations

    Step 1: fail to put redirects in place

  • The Liston of site migrations

    Step 2: rel=canonical every page to the homepage

  • Good for the patientBad for the patient

    Accidental

    Deliberate

  • Good for the patientBad for the patient

    Accidental

    Deliberate

    Mercury for syphilis

  • Good for the patientBad for the patient

    Accidental

    Deliberate

    Mercury for syphilis

    Not washing hands

  • Good for the patientBad for the patient

    Accidental

    Deliberate

    Mercury for syphilis

    Not washing hands Garlic + Onion

  • Of course a lot of deliberate things were neither harmful nor beneficial

  • Cargo cult:During WW2, Pacific islanders who had never seen

    manufactured equipment saw modern military planes bring cargo to their remote islands.

    Read Richard Feynmans speech

    http://calteches.library.caltech.edu/51/2/CargoCult.htm

  • Cargo cult:After the war, cults developed that tried to recreate

    the conditions that brought the planes(runways, control towers, military uniforms)

    without understanding what had really happened.

    Read Richard Feynmans speech

    http://calteches.library.caltech.edu/51/2/CargoCult.htm

  • Do we have our own cargo cults?

  • Do you recommend changing h2 to h1?

    Do you have a good reason why?

    Even if it does help, does it help enough to be worth it?

  • Lets start washing our hands

  • The scientific method

    Step 1:Generate hypotheses

  • In medicine, old wives tales are a great place to start. Looking at things that appear to work, but we have

    no idea why is a good source of hypotheses.

    A great example is this 1,000-year-old spell that included garlic, onion and cows stomach, and turned

    out to kill MRSA.

    I guess the SEO equivalent is to ask the old timers.

    http://www.bbc.co.uk/news/uk-england-nottinghamshire-32117815

  • In all areas of science, you are at an advantage if you can figure from first principles. Richard Feynman

    famously used to draw what became known as Feynman diagrams to understand sub-atomic

    interactions through thought experiments alone.

    The SEO equivalent is to stay abreast of information retrieval and ML papers and formulate hypotheses

    based on an understanding of how the algorithm likely works.

  • Finally, you can go mining the data.

    The obvious SEO equivalent is the various correlation studies into ranking factors.

    In both medicine and SEO, you obviously have to be wary of spurious correlations. Blindly mining data

    can get arbitrarily high correlations (the example above has a correlation of 0.993!).

    https://www.newscientist.com/article/mg22329844-000-supercomputers-make-discoveries-that-scientists-cant/http://www.tylervigen.com/spurious-correlations

  • The scientific method

    Step 2:Try things in the lab

    Problem: results may not hold, or may come with new side effects

  • In the SEO space, this is work like that done by

    IMEC labs.

    It involves attempting to run controlled

    experiments on test domains and / or with

    volunteer participants. The outcomes are

    normally not improved rankings or traffic that

    participants care about.

    https://moz.com/rand/imec-lab/https://moz.com/rand/imec-lab/

  • Potential pitfalls

  • What works in tests may not work in the real world

    Source: National Institutes of Health

    http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2964774/

  • Do you recommend http https migrations?

    All else being equal, secure is better. All else is never equal.

    Side effects may include ranking fluctuations, traffic drops, difficult conversations with your boss.

    Side effects

    may include headache, nausea, vomiting, death, dizziness, dysentery, cardiac

    arrhythmia, mild heart explosions, varicose veins, darkened stool, darkened

    soul, lycanthropy, trucanthropy, more vomiting, arteriosclerosis,

    hemorrhoids, mild discomfort, vampirism, spontaneous dental hydroplosion,

    sugar high, even more vomiting, and mild rash.

  • The scientific method

    Step 3:Gold standard scientific trials

  • TL;DR scurvy bad, science hard

    You should read the story of one of the first controlled scientific experiments that proved lemons could

    cure scurvy (in 1747!). The incredible story of how the discovery supported British naval supremacy, and

    then how compounding errors involving the colonial supply-chain, faster steam-powered ships, and polar

    bear offal led to the loss of the knowledge, the death of polar explorers, and the eventual rediscovery of

    vitamin C.Source: idlewords

    http://idlewords.com/2010/03/scott_and_scurvy.htm

  • How SEO split tests work

  • You might have seen @TomAnthonySEO tweeting about the platform weve built to make this easy

  • https://www.distilledodn.com

  • Excuse a brief diversion into geeky details

  • Instead of comparing the performance of the control pages directly with the variant pages, we build a

    forecast of whats called the counterfactual which is an estimate of what would have happened if we hadnt

    made the change. We use the control group to make a counterfactual forecast that takes into account

    seasonality and site-wide changes.

    The black line on the chart above is the actual organic traffic to the variant pages. The blue line is the

    counterfactual.

    More: Distilled blog post and free forecasting tool

    https://www.distilled.net/resources/testing-titles-and-h1s-seo-impact/https://www.distilled.net/resources/statistical-forecasting-for-seo-analytics-and-a-free-tool/

  • Its easiest to analyse the results by looking at the cumulative difference over time between the actual

    organic traffic and the counterfactual.

    The pale blue area is the 95% confidence interval.

    We can see a (statistically) zero effect for an initial time while Google crawls and indexes the test,

    followed by steady growth. A couple of weeks in, the confidence interval goes above zero and we have a

    winning test.More: Distilled blog

    https://www.distilled.net/resources/testing-titles-and-h1s-seo-impact/

  • Its easiest to analyse the results by looking at the cumulative difference over time between the actual

    organic traffic and the counterfactual.

    The pale blue area is the 95% confidence interval.

    We can see a (statistically) zero effect for an initial time while Google crawls and indexes the test,

    followed by steady growth. A couple of weeks in, the confidence interval goes above zero and we have a

    winning test.More: Distilled blog

    Hashtag winning

    https://www.distilled.net/resources/testing-titles-and-h1s-seo-impact/

  • Further reading for those interested:

    Predicting the present with Bayesian structural time series [PDF] Inferring causal impact using Bayesian structural time series [PDF] CausalImpact R package Finding the ROI of title tag changes

    More: Distilled blog

    http://people.ischool.berkeley.edu/~hal/Papers/2013/pred-present-with-bsts.pdfhttp://people.ischool.berkeley.edu/~hal/Papers/2013/pred-present-with-bsts.pdfhttp://static.googleusercontent.com/media/research.google.com/en//pubs/archive/41854.pdfhttp://static.googleusercontent.com/media/research.google.com/en//pubs/archive/41854.pdfhttp://google-opensource.blogspot.cz/2014/09/causalimpact-new-open-source-package.htmlhttp://google-opensource.blogspot.cz/2014/09/causalimpact-new-open-source-package.htmlhttp://markedmondson.me/finding-the-roi-of-title-tag-changes-using-googles-causalimpact-r-packagehttp://markedmondson.me/finding-the-roi-of-title-tag-changes-using-googles-causalimpact-r-packagehttps://www.distilled.net/resources/testing-titles-and-h1s-seo-impact/

  • What should you be testing?

  • Add structured dataOne of the easiest tests to run is the addition of structured data - we recommend schema.org via JSON-LD.

  • smokymountains.com

  • We got one of the fastest and clearest uplifts we have seen so far with

    the addition of structured data to detail pages. This chart shows the

    uplift from adding location-based data to individual property pages.

  • Improve your organic advertsAdvert testing plays a huge part in PPC. Looking at typical meta descriptions, it appears its rarely