The Screen Size Challenge Report

8/7/2019 The Screen Size Challenge Report

http://slidepdf.com/reader/full/the-screen-size-challenge-report 1/30

THE SCREEN SIZE CHALLENGEA study on how traditional web analytics tools typically track mobile websites

and applications versus niche mobile analytics vendors.

Steve JacksonDirector – Business Insights

Kwantic

http://www.kwantic.com/



The Screen Size Challenge

Table of Contents

Introduction Mobile Analytics and Web Analytics ................................... ......................................... ......................................... ......................................... ......................3

Mobile Analytics Challenges.....................................................................................................................................................................................................................4

1) Multiple Operating Systems..........................................................................................................................................................................................................4

2) Operators... ......................................... ......................................... ......................................... ........................................ ......................................... ......................4

3) Privacy..........................................................................................................................................................................................................................................64) Devices & Applications .................................................................................................................................................................................................................6

Data Collection Methods..........................................................................................................................................................................................................................7

Benefits of Mobile Tracking......................................................................................................................................................................................................................8

Tracking Behaviour...........................................................................................................................................................................................................................8

Data .................................................................................................................................................................................................................................................9

Information.... ......................................... ......................................... ......................................... ........................................ ......................................... ......................9

Action...............................................................................................................................................................................................................................................9

Start at the top of the pyramid .................................... ......................................... ......................................... ......................................... ........................................ 10

Intelligent Communication .............................................................................................................................................................................................................11

Segmentation and Voice of Customer ....................................................................................................................................................................................................12

Segmentation..................... .......................................... ........................................ ......................................... ......................................... ........................................ 12

Voice of Customer ......................................... ......................................... ......................................... ......................................... ......................................... .............12

Data Integration.....................................................................................................................................................................................................................................13

The Tool Comparison Method........... ......................................... ......................................... ........................................ ......................................... .................................. 14

Example: Score for the Excel interface ...................................... .......................................... ......................................... ........................................ ...........................14

The Tool Scores...... ......................................... ......................................... ........................................ ......................................... ......................................... ....................15

Bango (Mobile analytics vendor) ....................................................................................................................................................................................................15

Commentary...................................................................................................................................................................................................................................16

Product capabilities ‐ Strengths ............ ............. ........... ............. ............. ............ ............. ............ ............. ............. ........... ............. ............ ............. ............. ............ ... 16

Product capabilities ‐ Weaknesses...................................................................................................................................................................................................... 16

Service & support – Strengths............................................................................................................................................................................................................. 17

Service & support – Weaknesses... ............. ............. ............ ............. ........... ............. ............. ............ ............. ............ ............. ............. ........... ............. ............. ......... 17

Pricing & Licensing Strategy – Strengths............................................................................................................................................................................................. 17

Pricing & Licensing Strategy – Weaknesses ........................................................................................................................................................................................ 17

Market Localization – Strengths.......................................................................................................................................................................................................... 18

Market Localization – Weaknesses.... ............. ............. ............ ............. ........... ............. .............. ........... ............. ............ ............. ............. ........... ............. ............. ..... 18

General Comments & Recommendations........................................................................................................................................................................................... 18

CEM4Mobile (Mobile analytics vendor) .................................... ......................................... .......................................... ......................................... ..........................19

Commentary...................................................................................................................................................................................................................................19










Google Analytics (GA) (Web analytics vendor) ........................................ ......................................... ......................................... ........................................ ..............23

Commentary...................................................................................................................................................................................................................................23










Adobe SiteCatalyst (Web analytics vendor).....................................................................................................................................................................................26

Commentary...................................................................................................................................................................................................................................26





Pricing & Licensing Strategy – Strengths............................................................................................................................................................................................. 27Pricing & Licensing Strategy – Weaknesses ........................................................................................................................................................................................ 27




Overall Conclusions................................................................................................................................................................................................................................29

About the author ..................................... ......................................... ......................................... ......................................... ........................................ ...........................30

About Kwantic .......................................................................................................................................................................................................................................30




Introduction Mobile Analytics and Web Analytics

This study was done to see how mobile analytics vendors compare to

traditional web analytics vendors and also comments on the general challengesfaced and how the tools deal with these challenges. I’ve also included a section

discussing the benefits of mobile tracking and how you might approach doing

this using the AID model (action, information & data).

The reason I put this together is because Kwantics’ own market is heavily

involved with the mobile sector, either they manufacture devices, provide

network services or are simply taking the first steps in mobile marketing and

advertising. As a company we’re constantly asked questions many of which we

needed to understand in more depth ourselves.

The study is by no means comprehensive. I’ve studied 2 pure play mobile

analytics solutions and 2 traditional web analytics solutions as they come out of

the box. The idea is to see where the emerging vendors have improved their

solutions over the traditional vendors.

One of the vendors (Omniture) has multiple add‐ons that could be used (i.e

Insight & Survey) that could easily raise their score and the price of the solution.

However I thought it prudent to compare the out of the box versions of each

tool rather than extra applications.

Hopefully the study will give you a bit more insight into what’s involved in the

mobile analytics space, what you need to take into account, how the tools

compare against each other and how you go about doing it yourself.




Mobile Analytics Challenges

There are 4 main challenges to take into consideration before you even start

tracking the mobile web.

1) Multiple Operating Systems

There are a number of different operating

systems in the Smartphone market.

1 ‐ Gartner is an information technology research and advisory company providing technology related insight.

NYSE: IT (http://www.gartner.com)

The data shown is subject to

change. In Q2 2009 Symbiancommanded more than 50%

market share as oppose 41%.

Since that time Nokia (Symbian)

have lost market share largely to

Android – an open source

platform created by Google.

Android has gone from 2.8% in

2009 to 17% in 2010, due largely

to manufacturers such as Sony,Samsung, HTC and Motorola

taking advantage of the free

operating system and designing

devices that suit it.

In terms of tracking it is clear that

mobile websites need to take

into account many different

operating systems to cover all

their bases.




2) Operators

In addition to operating systems there are hundreds of operators worldwide

that have different ways to allow access to behavioural data of their

subscribers. The competitive landscape has changed considerably over the last

5‐6 years.

Fig 1 & 2 Source: The Future of Mobile Telecommunications http://www.prtm.com

The operators have different ways to safeguard privacy and cookies in

particular are an extremely important way to identify visitors in traditional web

analytics tools.

Due to the mobile phone being a very personal device it means cookies are not

as anonymous as with the fixed web where only your browser version is

tracked. Operators can for instance block cookies from being sent through their

networks meaning a whole new challenge to track behaviour.




It’s also extremely important to detect the access point, for instance has the

visitor used WAP, internet apn or wifi to view the content? How the user

accesses content can effect measurement considerations.

3) Privacy

As mentioned mobile phones are very personal pieces of kit. In a tracking sense

they are largely associated with individuals, meaning personally identifiable

information is embedded into the phone. Bills to the operator, mobile

applications with transactional data and technology such as embedded GPS

mean that potentially anyone can know who, where and what you’re doing.

Mobile operators are however tightly regulated and very responsible when it

comes to your personal data. It is not in any operators’ interest to risk violating

privacy laws and therefore they don’t.

While privacy should be the holy grail, worldwide standards have not yet been

defined about how mobile operators can legally and safely allow data to be

gathered. This leads to a bit of a wild west situation with each operator having

its own methodology, some allowing reasonable access to data and some

blocking everything.

4) Devices & Applications

Apps’ that can be accessed offline present a new challenge when trying to

measure the engagement the user has with the application. Not only that but

app development also often means more difficult technical requirements for

actual tracking.

Due to the huge increase in iPhone, Android & tablet applications appearing on

the market visitors need to be detected and identified correctly. Additionally

Symbian, Java and Flash apps all have their own requirements.

Only smartphones actually have image and javascript tracking enabled by

default. In 2010 there are still a large percentage of older models that don’tsupport tag based solutions, while some low end devices overwrite or lose

cookies and need to be detected by different means.

Some high end devices like the iPhone block 3rd

party cookies and therefore 1st

party cookies need to be used.

All of these challenges have to be considered when looking at the tracking of

mobile usage which brings us to the data collection question.




Data Collection Methods

Due to the different challenges it’s worth discussing the data technologies that

exist today in order to track how mobile sites and apps are tracked.

1. Log based ‐ The server log files for mobile websites contain information in the

headers such as IP & URL strings that can identify mobile browsers. The

benefit of using logs is that the data is easily accessible (it sits on a server

you can access without any need to tag anything). The downside is you will

have to carefully set‐up the data into some readable form and you may not

get good information out. Some telecom companies are providing analytics

via log data to their advertisers.

2. Image tags ‐ Requires tagging of the mobile website but you can avoid most

of the caching issues with varying parameters. Requires image support via

the mobile devices and operators to collect data.

3. JavaScript tags ‐ A more sophisticated method of writing the image tag that

enables more information to be processed (the JavaScript asks the device

for it), but also has more limited support in mobile devices (smartphones

mostly). In many cases JavaScript isn’t supported by older mobile phones.

JavaScript also has a bigger delay when sending data that can cause missing

data if the script doesn't have time to execute properly.

4. Packet sniffing solutions ‐ Captures the data from the traffic when in passes

through the network meaning that tags are not required thus solving the

cookie/tagging problems. It does however require a physical platform

between the server and the users device. This means it’s challenging to

setup. It also still requires a simple image tag to avoid proxy issues and track

event based things (like applications on the phone).

5. Application measurement ‐ Mainly works on smart phones (iPhone, Android,

Windows CE, S60 3rd edition + etc.) and means that the application hasinbuilt tracking to send tracking hits to the server. These hits can be cached

to measure "offline" usage as well, but the phone must be connected to the

Internet to send the data. The technology used to track applications is very

similar to event based tracking in the fixed web such as flash applications

and movies.

In terms of tracking tools, the more ways that a vendor can gather the data the

better due to different operators, different device adoption, geography etc.






Data

There is lots of data, reams of data, data that you can pull from everywhere.

You have competitive data, marketplace data, sales data, historic customerdata, network traffic data, server data, behavioural data, platform data,

research data and probably thousands more sources of data that I could list but

won’t.

The problem with all of this data is that it’s usually all over the place in different

silos’, disconnected and not in context. The first job is to gather requirements,

what do you need? Then define the data sources that can help you answer

those needs, look for context and extract information from the data.

Information

Information from data is less evident though there is still quite a lot of it. This is

one of the reasons analysts are in high demand. You have to first have an action

in mind and second have relevant data sources before you can find information.

This sounds simple but it isn’t and why we talk about key performance

indicators (KPIs) so much because they direct us toward the right direction for

action (or they should). KPIs really should start with the actions you want to

take (or around the actions you want your users to take).

One thing about information is that while there is still quite a lot of it, it isn’t

always relevant or useful. The key is to learn how to distinguish the useful

information from the useless.

Action

This is the entire point. You get some information that allows you to make a

decision resulting in an action. You should not take action without having data

that backs up your information. Or you should be consistently taking action that

is giving you new data to put into context and take new actions. This is called

testing (for instance A/B or MVT).

Unfortunately there is far less action going on than we would like and a lot of

data reporting.

So getting back to tracking behaviour what is it that you want to do with the

information about your mobile users? To illustrate lets create a fictional

scenario.




Start at the top of the pyramid

What action do you want to take?

‐ I want to know if there is something stopping mobile users from

registering to our site and if so I want to do something about it.

What information do you need?

‐ Data about how people sign up, where they come from and what they

do as well as if possible what bugs them or delights them about our

offers.

What KPIs are we looking at?

‐ Sign up completions (segmented by device type)

‐ Abandonment (through the sign up process)

‐ Good and bad engagement (high bounce rates/single page or event

access segmented by device type)

So how does this work?

Firstly set‐up the KPIs in your analytics tool to get the data.

Second examine it and collate the information. Then act on what’s useful.

For example; 10000 visits hit the mobile website and 500 of them signed up via

the mobile phone application process. Each sign up is worth €20.

A 5% conversion rate – This is your average benchmark.

Your segments show the following conversion rates;

Nokia N73 iPhone Blackberry HTC

1% 7% 3% 8%

So now you have data in some context and are creating information. The

iPhone and the HTC are better than the benchmark while the Blackberry and

Nokia are worse. This should lead to your first “why?” question. Why is this

occurring?

Second you check your registration abandonment and notice that of the 5000

that start the process the largest place that people 50% left from was the first

page and the largest proportion were using a Nokia N73.

This means that the lowest conversion rate comes from the highest amount of visitors. Again you have another reason to ask why this is?




Finally you look at the kind of visit that leaves the mobile site without doing

anything and notice that from Blackberry visits as well as Nokia N73 visits had

higher bounce rates.

You decide to ask your N73 visitors. 65% of 200 respondents from the N73

visitors tell you via your mobile survey that they really like and want your

service but they can’t register on their phones easily due to having a much

smaller screen size than the site design is optimized for.

You realize that both the N73 and the Blackberry have smaller screens than the

iPhone and HTC. You now have information in context that answers your

question and allows you to act.

Action: Optimize the site for smaller screen sizes to convert more of your

audience. Upon doing this your conversion rate on those devices goes up andyou earn €20K per month more in new registrations.

Intelligent Communication

Don’t say this to the decision maker:

We checked the conversion rates of our sign up procedure and only 1% of N73

visitors convert. The bounce rate across N73 & Blackberry devices were higher

than average and the abandonment rate was higher in the registration process.

This led us to do a survey where 63% of our visitors told us there were technical

issues with our website in their devices.

Say this:

Our mobile websites aren’t working on small screen mobile phones and our

customers are becoming frustrated. If we fix this we estimate a €240K lift in

revenue from new registrants per year.

Keep the technical stuff back until you’re asked to prove it.




Segmentation and Voice of Customer

You’ll notice in the scenario we did two very important things to get to the

actionable part of the pyramid.

We looked at subsets (segments) of the data (the device types) and we asked

the visitors to the mobile site what they thought of the service.

These two tactics along with formulating KPIs also help to guide your path

towards actionable information.

Segmentation

Segmentation is one of the most useful tools in any analytics system and allows

you to put data into context. You can segment by type of device, operating

system, browser, traffic source, extremely engaged traffic, poorly engaged

traffic, traffic that has taken certain actions, geography, city, operator – in fact

pretty much anything.

When segmenting it’s useful to always have a KPI to guide you. Bounce rate or

conversion rate across those individual segments allows you to put things into

context. It allows you to compare across different variables. You should take a

benchmark of the site average (as I showed with 5%) and then compare across

segments to gain insights.

If the information isn’t useful don’t use it. There will be a lot of segmented

information that you just can’t use, it’s all part of your investigation.

Voice of Customer

Voice of customer data is an incredibly useful source of information because

while Analytics systems will tell you what the problem is they won’t tell you

why the problem exists. Voice of customer will tell you why.

When users give you qualitative data on what works and what doesn’t there are

few sources of information that are better. The key is in asking the right

question. As you saw with the example there came a time when you know what

is happening (you saw N73 and Blackberry visitors weren’t converting for a

variety of reasons) but you didn’t know why. The survey told you that the

smaller screen size was an issue.

When you don’t know why is when you engage in a qualitative study such as a

survey. If people are bailing out at a particular point in the process ask them

right there as they’re about to leave.




The responses you get may give you exactly the answers you need to improve

your visitors experience and increase your own revenues.

Data Integration

Integration of data is becoming increasingly important. An obvious advantage of

combining mobile data with user information is the ability to serve ads based

on location. There are many ways that combining behavioural data with other

data sources can considerably improve online revenue generation.

The usual methods to do this are the ability to export to excel, CSV, XML, or PDF

but many vendors now open direct APIs to their raw data meaning that

anything that is recorded can be extracted as needed.

It is desirable from a tracking point of view that vendors have ways to importand export data so that correlations or comparisons can be made.




The Tool Comparison Method

Through a series of questions to the vendor and direct access to the tools the

author was able to score each tool independently across a set of pre‐definedcriteria.

The scoring method is unambiguous meaning you score well or you score poorly

depending on the tools ability to answer each question. Zero was the worst

score with 5 being the best across each question.

Example: Score for the Excel interface

Q: Can Excel‐based reports be refreshed automatically when the workbook is

opened?

0: No ability to generate Excel reports

1: XLS file generation from within the tool

3: API through third‐party tools to excel

5: API from within Excel to the analytics database

There were 53 different questions asked of each tool. The subjects of these

questions were around the tool itself, the services and support, the pricing

strategy and the market localization.

Tool questions were focused on data collection, data storage, data processing,data quality, data security, data segmentation, reporting, mobile specific data

reporting, usability and marketing measurement.

Services & support questions focused on technical support availability as well as

mobile measurement consulting services.

Pricing & Licensing Strategy questions focused on the cost and licensing options

of the tool while market presence focused on language localization and support

for Asian character sets.

The scoring was weighed 70% in favour of the tool questions with 10% weight

assigned equally to services, strategy and market presence due to no actual

client requesting this assessment.

This weighting would differ in a situation where a client asked Kwantic to assess

the best tool for their measurement strategy because then we would be able to

weigh the questions based on what was important to that client.

While I have scored these tools in the fairest manner possible the study was

designed to compare how mobile analytics vendors compare to web analyticsvendors. I would add that your overall strategy is more important than this




simple comparison and it makes a huge difference to the overall scores when

your own companies’ needs are taken into account.

The Tool Scores




Bango (Mobile analytics vendor)

Introduction – taken from bango.com.

http://www.bango.com/

Bango provides the technology that powers commerce for businesses targeting

the growing market of internet enabled mobile phone users.

Bango’s products collect payment from mobile users for on‐line content and

services, and provide accurate analytics for mobile marketing campaigns and

sites. The world's leading brands plus thousands of smaller content providers

and developers use Bango products to run their mobile businesses.

Commentary

Product capabilities ‐ Strengths

Bango scored higher than traditional web analytics tools due to the fact that it

has more methods to collect data than Google or Omniture, it has superior

tracking across widgets & applications and the data warehouse is out of the

box. Identification of individual sessions is also stronger than in GA and

Omniture.

Out of the mobile tools Bango has slightly better reporting due to having

slightly better scenario analysis, a better excel interface and stronger partner

integration.

Bango was neck and neck with CEM4Mobile across all the Mobile Analytics

support sections (both tools scoring 5 across all questions) save the user

feedback mechanisms that Bango does not have (or focus on).

Bango beat GA & Omniture hands down in Mobile analytics support scoring top

marks across handsets, OS and access points, better customer/user reporting,

good coverage across being able to measure applications, widgets and excellent

ability to distinguish the users environment (geo location, wireless/non

wireless, operator visited from etc).

Product capabilities ‐ Weaknesses

Both Omniture and GA have stronger partner networks and therefore

integration capabilities as well as path and scenario visualization.

Bango was weaker than Omniture for measuring segments though it should be

noted that there are extra costs involved for segmented traffic in SiteCatalyst.




Both Omniture and GA were stronger at measuring online marketing such as

media advertising & search engine marketing as well as benchmarking.

Something that traditional web analytics tools have focused on.

Service & support – Strengths

Technical support availability, service levels and training curriculum richness

scored top marks for Bango. They have focused strongly on these areas having 4

tiers of support with 24/7/365 availability at the top tier (with a higher price

tag). Major US brands have selected Bango on its ability to resolve and respond

to issues fast and we found numerous examples of good customer testimonials

across their website.

Bango also has support based out of multiple offices around the world and

global free support via their forums and knowledge base.

Since Bango is supporting mobile analytics for the purpose of payment

platforms for mobile applications and websites it has much stronger support

than a typical web analytics vendor in terms of training for the end user around

mobile expertise.

Service & support – Weaknesses

Bango has a relatively small consulting service around the tool preferring to use

partners to deliver value adding services.

Pricing & Licensing Strategy – Strengths

In terms of the 4 vendors listed Bango is moderately priced starting at $499 per

year for a basic starter pack. The pricing is based around events (server

requests) and the more events you register the more you pay. $499 would buy

you about 250,000 events per month which is enough for most small

businesses.

If you’re in the enterprise bracket registering 10 million events per month then

you could expect to pay around $900 per month plus service packages (silver isapproximately $200 per month). Over 10 million events per month are bulk

discounted further.

Pricing & Licensing Strategy – Weaknesses

There is only 1 way to license Bango and that’s a hosted ASP which may not be

possible for some companies who need to keep data on internal servers.

Additionally just like Google Analytics, Bango bears no responsibility for

intellectual property losses due to data loss.




Market Localization – Strengths

Global character support is available. This is important especially if you have

presence in Asian or Arabic markets.

Market Localization – Weaknesses

Based on current demand the tool is only available in Western languages.

General Comments & Recommendations

The cool thing about Bango is the reason it came into being. It’s designed to

collect payments from mobile websites or applications. Because of this they

needed a good analytics tool and Bango Analytics was therefore necessary.

Bango is better than the traditional web analytics tools at measuring mobileanalytics at the time of writing.

The customer profiles that tend to have the most success using Bango appear

to be those that are monetizing their mobile websites, have applications that

generate revenue or do heavy mobile campaigning. If you’re billing via apps or

your mobile website and therefore need good analytics alongside then Bango

might be a good choice to consider.




CEM4Mobile (Mobile analytics vendor)

Website Introduction – From cem4mobile.com.

http://www.cem4mobile.com/

CEM4Mobile is the Customer Experience Management solution for companies

providing Mobile Content and Value Added Services for individual or business

users. CEM4Mobile enables mobile service distributors in all industry verticals

to measure and analyze interactions between end‐users and mobile services,

optimize their business and ensure successful customer relations.

CEM4Mobile concentrates on mobile content services and detects all the

transactions from wireless devices. Both browsing and application based

services are supported. The statistical data is generated directly from the

content portal or application server, thus making the integration fast and

straightforward. CEM4Mobile is offered on a Software‐as‐a‐Service (SaaS)

basis, and the reports can be accessed easily through the Internet.

Commentary


CEM4Mobile scored highest in this study across product capabilities overall for

measuring the success or failure of mobile websites and applications.

CEM4Mobile was neck and neck with Bango across all the Mobile Analytics

support sections (both tools scoring 5 across all questions) but the main

difference was user feedback mechanisms and cross analysis in which

CEM4Mobile outscored Bango.

This is the key difference between CEM4Mobile and other tools on the market,

the ability to be able to gauge the customer experience by fetching opinions

and values from end users. They can then link analytics and voice of customer

data via pretty powerful cross analysis functions.

CEM4Mobile also beat GA & Omniture hands down in Mobile analytics support

scoring top marks across handsets, OS and access points, better customer/user

reporting, good coverage across being able to measure applications, widgets

and excellent ability to distinguish the users environment (geo location,

wireless/non wireless, operator visited from etc).

CEM4Mobile also has strong segmentation capabilities on a par with Omniture

with no extra cost.

User profiling was strongest across all 4 tools with CEM4 mobile. This is due to

the nature of being able to segment opt‐in visitors who have viewed the survey

(or other opted in visitors) and therefore being able to build profiles around




known users.


All the tools have stronger partner networks and therefore integration

capabilities as well as stronger path and scenario visualization.

At the time of writing CEM4Mobile also doesn’t have an API nor very strong

excel integration. While the tool does generate Excel files you would then need

to cut and paste those files into other formats if you were reporting mobile

statistics as part of a wider set of KPIs (or build KPIs around an exported excel

from CEM4Mobile).

Both Omniture and GA were stronger at measuring fixed web online marketing

such as media advertising & search engine marketing as well as benchmarking.

Something that traditional web analytics tools have focused on.


CEM4Mobile has the strongest services and consulting team around mobile

analytics than any of the 4 tools in the study. The professional support service

levels involve meeting customers face to face, a telco around set‐up and

technical support as well as going through findings from the reports with the

customers’ own datasets.

As the focus is around customer experience management (CEM) and mobileanalytics this is outstanding when compared to Omniture (where extra costs are

incurred for service) or Google/Bango who service through 3rd

party partners.


While the level of service is good availability is limited to 10 hours during the

business day (CET) and on call back‐up/email support. Also while global support

is free via forums and email, local markets are only covered remotely during

those 10 hours, so no local offices nor 24/7/365 global service.


In terms of the 4 vendors listed CEM4Mobile is moderately priced starting at

around €2000 per year for a basic starter pack. The pricing is based around

events (server requests) and the more events you register the more you pay.

€2000 would buy you about 50,000 events per month. Deals are also available

for shorter periods (even monthly) starting from €190 per month.

If you’re in the enterprise bracket registering 10 million events per month then

you could expect to pay around €1000 per month. Over 10 million events permonth are bulk discounted further.




There are 3 licensing options, hosted/ASP (as with all the vendors in this study),

partially distributed ASP where the customer is hosting a proxy that calls the

CEM4Mobile server and a fully distributed ASP where the customer hosts the

software.

This flexibility is extremely useful allowing for customers who want to keep

their information in house.

The final strength is the data ownership and security. Data is owned by the

client and fully guaranteed on security around intellectual property losses.

There are banks using the system because of the full indemnification.


Aside from having to pay for the product there are no real weaknesses to speakof. Google Analytics won on pricing because it’s free but that was the only

reason.


CEM4Mobile comes in 4 languages, English, Spanish, Finnish and Hindi. The

localization includes the UI and all the functions. However, the help files,

documentation, some parts of the reports and training material is currently

offered only in English.

The tool also supports non‐roman character sets (such as Asian and Arabic

characters).


The weakness exists if you need the tool configured in a language that is not

supported.





CEM4Mobile has taken steps to answer the “why” question when I discussed

the benefits of mobile tracking. The fact that you can add a survey to your

mobile website and have a strong feedback mechanism from visitors to your

mobile website with this tool allows you to qualify their level of experience.

With this one measurement solution you could literally find out the problems

faced by users in the whole scenario I outlined earlier. You could see the KPIs,

segment the data into different device sets, see the differences between

segments and then ask the user to fill in a very short survey in return for a

months free service. This would give you the complete answer to that scenario

(that an N73 or Blackberry was too small).

In addition to Analytics, CEM4Mobile boasts IAB approved Ranking for mobile

audience measurement and Surveys.

When you want to measure mobile websites, applications and user experience

then the clients having success with CEM4Mobile suggest to us you should take

a look at this tool.




Google Analytics (GA) (Web analytics vendor)

Website Introduction from google.com/analytics.

http://www.google.com/analytics

Google Analytics is the enterprise‐class web analytics solution that gives you

rich insights into your website traffic and marketing effectiveness. Powerful,

flexible and easy‐to‐use features now let you see and analyze your traffic data

in an entirely new way. With Google Analytics, you're more prepared to write

better‐targeted ads, strengthen your marketing initiatives and create higher

converting websites.

Commentary


Reporting in Google Analytics is strong as the tool has been built since 2005 for

usability based on user input from enterprises across the globe.

The integration with partners is very strong, there are good APIs and even open

source developments around the tool. For instance survey companies like 4Q

and Kampyle integrate with GA for allowing you to get voice of customer data.

Report visualization, path visualization the ability to bring in testing tools and

other features at no cost are all admirable strengths.

It’s quite easy to use Google data within the enterprise as part of a wider set of

KPIs.

Security also has a very strong track record with very few leaks being reported

in 5 years of enterprise operation.

Perhaps the strongest ability in terms of Mobile Analytics is the ability tomeasure all forms of marketing very well including SMS/MMS, media

advertising, banner advertising as well as providing global benchmarking.


The mobile measurement tools surpassed GA in how they track mobile critical

metrics. While GA have things like different handsets, screen sizes, brands and

such they don’t measure the individual or allow user feedback.

There are other weaknesses around knowing the operator networks and accesspoints. The mobile vendors regularly check operator & access point databases.




They then update their tools to suit (monthly at least) and there is little

evidence Google are doing the same.

Exporting of data from Google can’t be done. You can get the data via APIs and

downloads but it will remain Google’s property which may be a problem for

some clients.


Google have good help files, good online education and knowledge bases but

primarily rely on 3rd

parties to help support their tool.

The wide number of partners in their network and open source movement

around Google Analytics means solutions to problems can be found online

where in many cases the same can’t be said of other vendors.


There is no official Google Analytics service or support aside from the online

education and partner network. While the partner network is very capable

there will always be extra costs involved and mobile analytics expertise in these

partner networks is limited at the time of writing.

There is no official training program or curriculum all of this being handled by

3rd parties or relying on online Q&A.


The tool itself is free. The cost of set‐up and running the tool is limited only by

your own organizations IT set‐up and processes to get the data.

In companies with either a very large amount of data being collected or low

budgets this extremely cost effective.


Google owns the data and despite very few leaks being reported since it began

reporting in 2005 it bears no responsibility for loss of intellectual property or

data. This is the price of free.


GA is the world leader at this time localized in 32 languages with full character

set support.





None, a perfect score by Google here.


I have to say I love Google Analytics. I still think it was the best thing Google

ever did giving the tool away for free. The thing with free is that people think

cheap, but in GA’s case this is far from true, it is an enterprise class product

with the ability to measure more than just web analytics out of the box.

Is it the best in class for measuring mobile data? No. I think I’ve seen enough of

other systems to show that GA is not as focused on the mobile niche as say

Bango or CEM4Mobile.

However for a free tool 3.16 out of 5 is not bad all things considered.

I would recommend GA if you have a low budget, you’re just starting out with

analytics or you’re not sure what you need. GA may be enough in the short or

even longer term.

Consider the entire strategy, if you’re measuring just mobile apps or mobile

sites GA may not be the best option, but if mobile is just one channel in a much

wider marketing plan and you don’t need in depth information then GA might

be good to consider.




Adobe SiteCatalyst (Web analytics vendor)

Website Introduction (from omniture.com)

http://www.omniture.com/

Adobe SiteCatalyst is the foundational product for the Adobe® Online

Marketing Suite, powered by Omniture®. It provides you with actionable, real‐

time intelligence regarding your online strategies and marketing initiatives. It

also helps you quickly identify the most profitable paths through your Web site,

where visitors are dropping off, what's driving critical success events, and how

different segments of visitors interact with your Web site.

Commentary


Segmentation in Omniture is strong though you will need the ability to use ASI

slots (depending on your installation this may cost more) to get a good view of

mobile only traffic across the report suite.

The partner network is very strong with a huge base of add‐ons that can be

integrated from the Adobe Partner Program. The ability to upgrade to a wide

variety of different marketing measurement programs is one of Omniture’s

strengths. Pretty much any online marketing function you want can be added to

the SiteCatalyst platform (although these cost extra).

The reporting interface also scored highest with good integration via API, good

export and import capabilities and perhaps the strongest integration with Excel

available today with report builder.

Perhaps the strongest ability in terms of SiteCatalyst Mobile Analytics is the

ability to measure all forms of marketing as well as SMS/MMS, media

advertising, banner advertising.


SiteCatalyst will do the basics across mobile analytics scoring moderately well

across all except user feedback mechanisms (which could be served via another

add‐on) but it doesn’t track as well as the niche players (CEM4Mobile & Bango).

Again like GA, when it comes to measuring access points, operator networks,

distinguishing between wi‐fi & network visits, individual user reporting and

profiling it falls a little short of the niche players.

Without the rest of the suite to help it SiteCatalyst does the basics but really to

the best from the platform you want to add data warehouse, discover, search




center and test & target. This would be a much more powerful suite but would

cost a lot more.


Technical support availability is good. Worldwide and available 24/7/365 via

phone, email, ticketing or live via Twitter and chat.

Service levels are significantly above average, most queries are answered same

day (same hour quite often).

Professional services are generally high standard and offered with pretty much

every deal but do come at extra cost.

Each client usually meets a representative, is assigned an account manager, a

technical support person as well as online ticketing, knowledge base andforums. There are also many white papers and other downloads which can help

in the early phases.


Omniture don’t specialize in mobile services and don’t therefore have strong

dedicated support around mobile web tracking, therefore paying additional for

professional consulting services around this area may not be as good as around

web tracking.

The weakness however would only come in comparison to a niche player that

offers professional services (like for instance CEM4Mobile).


The data can be extracted and belongs to the client, not to Omniture.

Omniture offer full indemnification towards intellectual property losses and

have a strong record around security of data.


The set‐up costs and ongoing cost of the tool is the most expensive from the 4

tools compared.

There is only 1 licensing option the ASP model with SiteCatalyst which means if

you want to host your data internally you can’t use SiteCatalyst to do so.





Omniture does support Non‐English character sets which is useful if you have

operations in Aisa or Arabic countries.


The tool is only available in English.


I also love Omniture SiteCatalyst. The service and support is good and I have to

say I know a lot of folks at Omniture and they are all dedicated to the work.

Again is SiteCatalyst the best in class for measuring mobile data? No. To be

honest the power of Omniture lies in the combined suite.

As a standalone product it just about shades Google Analytics and falls short of

the two niche products.

In fact I’ll go out on a limb and say to anyone reading that buying SiteCatalyst

alone is a waste of resources. You need at least the datawarehouse and ASI

slots to make SiteCatalyst worth the investment otherwise segmentation is

quite weak.

Again with Omniture it’s more about the entire strategy and the suite additions.You will pay more but you could have the ability to manage all your online

search marketing (Search Center), SEO (DigitalPulse), Email (through a number

of partners), CRM (for instance SalesForce Integration), testing (Test & Target)

Surveys (Survey), as well as standard analytics and pretty decent mobile

analytics (3.38/5) from one place. The question then becomes do you have the

budget and the people to help you?

I would recommend Omniture when you have a large budget (in excess of €50K

per year for the tool and significant investment in people/process) and have

greater needs than mobile measurement alone.




Overall Conclusions

One purpose of this study was to find out if mobile analytics vendors were

better than pure play web analytics vendors in measuring mobile websites andapplications and I think at the moment I would answer yes.

There is more to mobile tracking than traditional web vendors have taken into

account and looking into the future I see even more upcoming challenges.

There is considerable diversity in the fast moving mobile industry due to heavy

competition between the handset platforms. For example Android as an open

source environment has been heavily tailored while equally fierce competition

in mobile content and marketing technologies means the measurement

vendors have to change at a rapid pace. Larger vendors are slower at reactingto these shifts in the ecosystem while new, faster paced start‐ups can often be

more agile.

Both mobile application and browsing usage is growing exponentially.

Tablets are exploding the usage of both mobile browsing and applications. They

connect primarily via wifi but operators are coming up with special tablet

contracts meaning you can expect even more growth in this sector, especially if

Apple has anything to do with it. Expect other companies that sell hardware to

get in on the act as well. For instance Amazon have a free mobile networkwhich helps their adoption in low wifi areas.

Application SDKs are becoming more flexible and this will allow cross platform

apps that become even more popular. Nokia for instance have just launched an

SDK said to be more powerful than any other allowing developers to easily

create application UIs across Symbian, Android and iOS. More information can

be found here;

http://www.paniccode.com/2010/10/nokia‐has‐already‐won‐the‐developers‐

battle/

At the core of all this development is the battle for the hearts and minds of the

consumer and knowing not only what the consumer wants but also

understanding their experience is key to the success of the mobile net.

Measuring that user experience, finding useful information from those

experiences and taking action that improves your bottom line is where the wise

money will be spent in the coming years around mobile analytics.




About the author

Steve is a well known web analyst, author and speaker

living in Helsinki, Finland. A pioneer since 2002, he

established one of the first European web analytics

consultancies (Aboavista), later acquired by Satama

(now Trainers' House) in 2006.

He now serves as Director of Business Insights at Kwantic a business

optimization agency focusing on improving e‐commerce, generating more or

better leads and driving more engaged traffic to media websites to improve

advertising revenues.

In 2008 he wrote his first book Cult Of Analytics published on May 14th 2009.

You can get 1/3rd

of it for free from kwanitc.com.

He also writes his own blog (blackbeak.com) focusing on web analytics and was

co‐founder of the Web Analytics Association Finnish presence in 2005, the

successful model then being adopted across the Nordics.

About Kwantic

Kwantic is an online business optimization agency specializing in Analytics, User

Research & Marketing. Our business model is based around success fees we

charge our clients when their performance levels have improved.

We're currently 20 people who specialize in web analytics, analysis, market

research, marketing, sales and usability.

Kwantic believes that all businesses have too much data and not enough high

quality information. It's like you can't see the wood for the trees.

Our mission is to cut through the crap to find helpful information from your

data and help you to use it to improve your business.

In 2009 the Kwantic Advantage™ earned its clients 4 euros for every euro they

spent with us. Currently we have one client earning 17 euros for every euro

they spend. You could too.

More information;

http://www.kwantic.com/

http://www.kwantic.fi

Documents

The Screen Size Challenge Report