Embed Size (px)
Citation preview
@dom_woodman
This talk is about split testing
Money.
Talked ourselves out of money
How to get the money back
Talked ourselves out of money
How to get the money back
How did we talk ourselves out of the
money?
Part 1
Not knowing things is scary
?
We’re paid experts, of course we should know
3
Part 2
Fear
SEO is hard to measure
Before
Before
After
Not money. At least we’re on sessions
Still not money. Now we’re on clicks.
What on earth is search visibility?
Seasonality
Seasonality
Industry Demand
Seasonality
Industry Demand
Seasonality
Industry Demand
Ranks
Ranks
CTR estimates
Ranks
CTR estimates
Handwaving
No-one cares when the going is good
Fear
Measurement
Part 3
Part 3: SEO has become more individual
More ranking factors
Which interact in more complex ways
Fear
Measurement
Complexity
So how does this all fit together
So how does this all fit together
So how does this all fit together
So how does this all fit together
So how does this all fit together
So how does this all fit together
So how does this all fit together
So how does this all fit together
So how does this all fit together
So how does this all fit together
So how does this all fit together
So how does this all fit together
So how does this all fit together
So how does this all fit together
So how does this all fit together
So how does this all fit together
So how does this all fit together
So how does this all fit together
SEO Manager: Jan 2016 - Jan 2017● SEO Manager at T-Shirts and Shorts. ● Completed 4 major projects. Reduced Dev
backlog.● Removed a penguin penalty. Regularly audited
backlinks. ● Organic traffic went up from 100K ~ 150K
What does my CV look like?
SEO Manager: Jan 2016 - Jan 2017● SEO Manager at T-Shirts and Shorts. ● Completed 4 major projects. Reduced Dev
backlog.● Removed a penguin penalty. Regularly audited
backlinks.
What does my CV look like? -- If I’m unlucky
Talked ourselves out of money
How to get the money back
Stop making guesses about our results
CRO and growth hacking are doing it
right
Learning from split tests
What does an SEO split test look like?
Control
Variant
This is a traditional CRO test
Control
Variant
I’m about to show you 2 graphs a lot. This is the first.
Black is the variant, treat blue as the control
In this case the blue line is higher, it’s a negative test
This is the start date
The second graph shows the total additional sessions
The fans show margin of error
The test is significant when the fan crosses 0
In this case it happened here:
Here they are together.
1.Adding ALT tags2.Removing SEO category text
3.Using JS to show content4.Setting exact match title tags
5.Adding structured data
1.Adding ALT tags2.Removing SEO category text
3.Using JS to show content4.Setting exact match title tags
5.Adding structured data
Established wisdom would say ALT tags on images are good for SEO
We found adding ALT tags to images had no effect
1.Adding ALT tags2.Removing SEO category text
3.Using JS to show content4.Setting exact match title tags
5.Adding structured data
How does SEO text on category pages perform?
E-commerce site number 1 - No effect/negative effect
E-commerce site number 2 ~ 3.1% increase in organic sessions
1.Adding ALT tags2.Removing SEO category text
3.Using JS to show content4.Setting exact match title tags
5.Adding structured data
We can render Javascript!
What happens if your content is only visible with Javascript?
Javascript EnabledJavascript Disabled
Making it visible increased organic sessions by ~ 6.2%
1.Adding ALT tags2.Removing SEO category text
3.Using JS to show content4.Setting exact match title tags
5.Adding structured data
Title tag before: Which TV should I buy? - ArgosTitle tag after: Which TV to buy? - Argos
What happens when you match title tags to the greatest search volume?
Organic sessions decreased by an average of 8%
1.Adding ALT tags2.Removing SEO category text
3.Using JS to show content4.Setting exact match title tags
5.Adding structured data
Category pages have lots of images and not much text
Adding structured data to category pages
Organic sessions increased by 11%
Running your own tests
1.How much traffic do you need?
2.How long should you run a test for
3.What kind of things are easy to test?
4.How can you do all the maths?
1.How much traffic do you need?
2.How long should you run a test for
3.What kind of things are easy to test?
4.How can you do all the maths?
Average of 1,000 organic sessions per
day to your set of pages
1.How much traffic do you need?
2.How long should you run a test for
3.What kind of things are easy to test?
4.How can you do all the maths?
Typically takes 2 - 3 weeks to reach
significance.
1.How much traffic do you need?
2.How long should you run a test for
3.What kind of things are easy to test?
4.How can you do all the maths?
Anything where you’re making changes to a
page
1.How much traffic do you need?
2.How long should you run a test for
3.What kind of things are easy to test?
4.How can you do all the maths?
distilled.net/diy-splittester
bit.ly/testing-lessons
