Tracking Errors in GA

@timlb#spwk

Started working on web sites in 1993

http://www.bryaneisenberg.com/hidden-secrets-of-amazon/

In 2014 I find myself asking:

“After 20 years of the web, why are so many sites still so hard to use?”

For 20 years we have been

● Building web sites

● Using web sites

● Measuring and testing web sites

But sites are still…

● Hard to navigate

● Difficult to understand

● Counter-intuitive

● Sometimes broken

And full of…

Errors!

Question: what is ‘bad’?

Web analysts start by asking the business:

“what is ‘good’?”

Answer defined as desired commercial outcome:● orders / revenue● leads● sign-ups● content-views

Web analyst then defines GA goals to matchWe then measure ‘good’ in terms of percentage of visits from various channels which complete goals

Easy to then think ‘bad’ = ‘not converting’

The analytics trap

Tempting to measure ‘good’ in ways which encourage us to ignore the site itselfOur numbers seldom record the things that people using the site would complain about(Or praise!)

The conversion rates in GA can say that a site is ‘good’ in terms of macro-goal completionPeople using it might say it was ‘bad’ meaning that they found it difficult to do what they wanted

So, why track errors?Reduce errors = make sites ‘less bad’

Errors can be useful indication of usability issues

● ‘Usability’ normally = lab tests or remote usability testing= sampling & not real visitors

● Error messages = real visitors, no sampling● Error messages = measurable

Errors are good (for insights)

Consistent, measurable, indicator of usability problemsSo a valuable source of insight and prioritisation for usability improvements

Benefits● Short term measurable improvements (online, this session)

● May well have more important long-term benefits○ Lifetime value○ Brand perception off-line and on-line

Each error is a small cry of pain from your visitorEach error is a point of friction which damages your brand

So, how many errors do your visitors see?

In most cases you won’t find the answer in GA (or other WA tools)

How to track errors in GA

What can you already track?

How to track the others

Universal Example of Error:404 page not found

You can’t stop people posting corrupt links, so every site will have 404 pages

● Ugly truth: in real life you may even have bad internal links

● Ugly truth: your redirects to mobile site may break

Can you count 404s in GA?

Early webmasters used to check for 404 errors and fix links

Now some sites make it hard to see 404 errors

Examples: ● Wordpress & Magento serve 404 page using the requested URI● But Title field usually includes ‘not found’● Tip: use a view filter to add 404 detail to URI

Quick 404 test

404 is so common that people compile galleries of examples and write blog posts on ‘best practice’

But are these pages tracked?

Helpful example of a good 404 gallery:https://creativemarket.com/blog/2013/07/22/the-best-404-pages-on-the-internet

Test: visit sample sites requestinghttp://thesite.com/404pageTrackingTest

and see what is sent to GA

Of 11 sites, 2 could not even see 404 in GA

6 could only see 404 in TitleWould need view filter for goals

How does tracking 404 help

Fix any internal broken links(bonus hint: look for redirects to mobile site causing 404s)Contact external sources and suggest fix

Experiment with more helpful 404 page to improve: ● bounce rate● exit rate● conversion rate for visits including a 404

Moving on from 404s

Checkout forms are another common source of errors

What’s your most common error?What’s your most annoying error?

And the big checkout errors are…Not what you’d expect?

What if it really is “user error”

How do you deal with ‘email already exists?

Examples of errors which can be fixed● Very valuable: Confusing promotional codes

● Mandatory fields○ Is it really needed○ If yes, can it be made more obvious

● Formats for post codes and phone numbers○ Use better validation and intelligent templates

● Requiring selection of options when there is only one● Requiring double-selection of option (e.g address lookup)● No default quantity on product pages● Site search 0 results

○ nb search ‘enter search term here’ prompt

So, where do we start with GA?

First: check if any other errors visible in GAFew errors visible in GA since they are dynamic elements within the page, or even modal dialog boxes

But check in case page reloads in a way which can be identified in GAExample: Magento onepage checkout login fail reloads with /index appended to URI (and note unique pageviews…)

If not in GA: what we need to know?

Need to add custom GA tracking in most cases to record

● What the error was● Where the visitor experienced it

Prioritise errors if you’re adding to an existing site and need to limit budget or are very worried about hit quota● Start with pages with high pageviews● Check usability tests to see which errors are common

Big decision:Virtual Pageviews or Events

VPVFor

● Can be used as goal● Can be seen in funnel visualization● Has an exit rate● Can be seen in flow reports● Can be used in content grouping

Against● Inflates pageviews● Need to track page explicitly● Breaks convention of using pageviews for

units of content which correspond with visitor seeing a new chunk of material

EventsFor

● Can be used as goal (now)● Doesn’t inflate pageviews● Built in reporting of event page ● Can be seen in behavior flow● Category / Action / Label structure● Event value can be used in goal

Against● Not visible in funnel visualisation● No exit rate● Cannot be used for Content Grouping

(unless you explicitly code them)

How to track “What the error was”Simple version:

Record verbatim text of message+ Easy to specify and treat as code library: “if this error text is displayed, then send it to GA”+ Standard rule means cheapest to implement throughout site on a new build

- Reports can be difficult to read: too many words- If same text is used multiple times on same form (‘invalid value’) then report is less valuable

Last two points = lesson on quality of error message

Real text means reports are constant reminder of how bad micro-copy is!

Advice on good error messages here: http://bit.ly/SuperWeek2014ErrorIdeas

Real-life tip: it may be worth adding a unique reference code to each error if there are multiple versions of same error on same page

Where the error happened:VPV directory structure thoughts

Choice:

Start with path or error?

/original/path/errors/’the message’(like page’s own URI)or/errors/original/path/’the message’

Think about:

Content Drilldown ReportContent Grouping

Reporting on errors in GA

Goals and Intelligence AlertsDashboardsCustom Reports(Segmentation)(Content Grouping)

Why use Goals?Powerful standard reports● Conversion rate everywhere in GA● Reverse Goal Path Report

○ What were people looking at in steps before error○ Very useful for internal broken links!

● Goal URLs Report○ Great if Goal has been configured to match more than one error

Bonus: Intelligence Alerts● Configure an Intelligence Alert for key ‘error’ goals

● Set up an automatic daily email alert○ Will arrive far too late in this timezone○ But still useful in case you did not check or notice

● Special reason: the Intelligence Alert system will attempt to identify the source of the change for you

○ Not always successful but sometimes works very well

Choice: which view for goals

Should you use a separate view for ‘error goals’?

● For: if you normally use the GA ‘All goals’ rolled-up metrics then having goals for ‘bad’ things will mess up your reports

● Against: if you keep ‘error goals’ outside your normal working views, how often will you check (e.g. you can’t include in standard dashboards)

I don’t use ‘all goals’ so I prefer to have ‘read and react’ goals in working views

Examples of Read and React Goals

Characteristics:● You can take action in hours● Can change rapidly from day to day● Can be caused by off-site or user-generated content

404 page● If conversion rate jumps● Find source of broken link● Fix if on-site● Contact source if off-site (or post good link if it’s user-generated content and use chance to

engage with potential visitors!)

Rejected promo code● If ‘bad’ code is common because it’s confusing (0/o/O, i/I/l) configure an extra matching code● Learn lesson and don’t use ambiguous codes● If ‘bad’ code is one-off on user-generated site visit the site, apologise and post a ‘good’ code

(engage with visitors)

Dashboards: really powerfulGreat example of when to use a Real Time Widget:● RT: active visitors with errors● RT: the active 404 error pages● RT: the other active error pages● External source/medium of

errors● External referring sites (only)● Adwords target errors● Error conversion rate● Error completions● Source of internal bad links

(Goal previous step!)● Bad link requested (Goal

completion URL)Include just conversion in other dashboards

Custom Reports: VitalFilter to focus on what counts

Pick only the metrics you need

Those checkout errors

Comparison view?

Actual numbers = actual money

Different people - (source/medium drill down)

Segment ideas

‘User’ scope segments:Potential to compare effect of error messages over different visits

‘Sequence’ segment rules:Possibility for looking at impact of errors at different stages of visit

Original segments:Great for looking at impact of with/without error(s) on macro-conversion

Content Grouping Thoughts

Same question as with VPV path

Group errors with original pageorGroup errors as dedicated group

Summary

Errors are a proxy for usability problems

Each error message is a small cry of pain from your visitorEach error message is a tiny dent in your brand’s reputation

Tracking errors in GA lets you see where the points of friction are

Measuring errors allows you to prioritise improvements

Improvements likely to bring enough short-term income to pay for work

Real big pay-off will be in customer lifetime value in multi-channel world

Resources

Bundle of all my Superweek2014 Links: http://bit.ly/Superweek2014TLB

Great article on form validation: http://bit.ly/Superweek2014FormsIncludes link to Luke Wroblewski post

“When compared to our control version, the inline validation form with the best performance showed compelling improvements across all the data we measured. Specifically, we saw:

● a 22% increase in success rates,● a 22% decrease in errors made,● a 31% increase in satisfaction rating,● a 42% decrease in completion times, and● a 47% decrease in the number of eye fixations.”

Great advice on error messages: http://bit.ly/SuperWeek2014ErrorIdeas

My first blog post on the subject: http://bit.ly/Superweek2014ErrorsIncludes very useful discussion thread