Upload
andreia-pinto-de-sousa
View
227
Download
0
Embed Size (px)
Citation preview
7/29/2019 Remote Usability Testing - Buyers Guide 2013
1/36
7/29/2019 Remote Usability Testing - Buyers Guide 2013
2/36
Opening
Thoughts
7/29/2019 Remote Usability Testing - Buyers Guide 2013
3/36
Foreword
ii
Remote usability testing has been around for some time now.
Slowly but surely, as company budgets get tighter and the thirst for
insights grows, innovation will keep finding new opportunities to
create necessary solutions to help us excel as professionals.
Remote usability testing is such a solution. With laboratory
usability studies often starting with 5-figure costs, remote
usability testing offers an affordable alternative for companieskeen on learning from their customers and website visitors.
Yes, I agree, remote usability testing is not comparable to regular
ol laboratory usability testing, but that does not mean that it
cannot function as a surrogate, or additional research method.
Many usability specialists argue that moderated tests can lead to
more effective and valuable insights, than (unmoderated) remote
usability testing.
Although I understand the arguments, consideration must be given
to the quality/price ratio. Even with a professional performing the
analysis a five person remote usability study can be up to 10
times more cost effective than a laboratory usability study.
With lower costs to getting valuable insights, remote usability
testing inarguably opens up the market for all companies to start
researching. There are less rocks to hide under, less excuses to
convey. Companies can simply no longer deny the fact that
remote usability research helps improve their companysservice, and ultimately their companys bottom line.
This guide wont tell you which provider is the best. It is my goal
to remain objective. All the service providers mentioned offer
different solutions, with different features, at different prices. This
guide is an attempt to point out to you, from a practitioners and
business point of view, whats important to know about a service
provider prior to doing business with them.
I have a lot of respect for the service providers who offer us these
amazing solutions for performing remote usability research. Even
though some would say that a market leader has been established,
new companies are already working hard to challenge that
status quo.
I hope that the providers of remote usability testing tools willcontinue to innovate and excel. Lets face it... in the end, it is the
customer who benefits, and that is all that really counts.
Matthew Niederberger
Founder actualinsights*
www.actualinsights.com
http://www.actualinsights.com/?utm_source=ebook&utm_medium=foreword+link&utm_campaign=Remote+Usability+Testing+Buyers+Guide+2013http://www.actualinsights.com/?utm_source=ebook&utm_medium=foreword+link&utm_campaign=Remote+Usability+Testing+Buyers+Guide+20137/29/2019 Remote Usability Testing - Buyers Guide 2013
4/36
People say I make strange choices, but
they're not strange for me. My sickness is
thatI'm fascinated by human behavior,
by what's underneath the surface, by the
worlds inside people.
- Johnny Depp
7/29/2019 Remote Usability Testing - Buyers Guide 2013
5/36
Chapter 1
What I do.
actualinsights* is a one-stop solution companygeared towards supplying small, medium, and
enterprise size companies with pragmatic insights
into conversion and user experience opimization.
7/29/2019 Remote Usability Testing - Buyers Guide 2013
6/36
A single goalI started actualinsights* with one goal:
To help companiesgain affordable and effective insights to help
themincrease revenue and customer satisfaction online.
Whats in it for you?With so many tools in the market today, it is hard to find the right match for you
and your company. My goal is to help you find the right tool for your research
needs. Even if you are just starting, or if you are an Enterprise company looking to
leverage other forms of data such as web analytics.
By working closely with you and your company in a pragmatic fashion I can help
you gain insights into customer attitudes and behavior. These insights will help you
prioritize your optimization efforts, helping you fix or improve those online features
and functionalities, and increase revenue and customer satisfaction.
How my experience fits in with your goalsMy experiences range from online user research studies to web monitoring
activities such as interaction, performance, voice of customer, and analytics.
With many years of experience under my belt from working with various major
companies I am confident that I can help you succeed, too.
HOW I CAN HELP YOU SUCCEED
1. Services for every budget
2. Receive actionable insights
3. Pragmatic research approach
4. Vast experience in multiple verticals
5. Internationally focused
5
Section 1
What I do
How to find me?e: [email protected]
t: +31648227889
http://www.facebook.com/actualinsightshttp://www.facebook.com/actualinsightshttp://twitter.com/actualinsights/http://nl.linkedin.com/in/matthewniederberger/http://nl.linkedin.com/in/matthewniederberger/http://www.facebook.com/actualinsightshttp://www.facebook.com/actualinsightshttp://twitter.com/actualinsights/http://twitter.com/actualinsights/http://www.facebook.com/actualinsightshttp://www.facebook.com/actualinsightshttp://www.facebook.com/actualinsightshttp://twitter.com/actualinsights/http://twitter.com/actualinsights/http://nl.linkedin.com/in/matthewniederberger/http://nl.linkedin.com/in/matthewniederberger/mailto:[email protected]:[email protected]7/29/2019 Remote Usability Testing - Buyers Guide 2013
7/36
Section 2
Services
6
Section 2
Services
Conversion OptimizationTake action from insight, but if you are not
sure how, then let me help you. From
mockups to prototypes, and projects plans
to success. Lets get started today.
Website Usability TestingTest your own website and see how real
visitors interact and think about it. Or test a
competitor, and see what they like there.
Mobile Usability TestingHave a mobile app or website? Perform tests
a various mobile and tablet devices and learn
what visitors think of it.
Customer Feedback AnalysisListen, analyze, and optimize. There is
nothing more powerful than the voice of the
customer to help you improve your business.
Website Performance ResearchFind out how your websites performance in
terms of page load and server response
times are affecting your visitors experience.
Web AnalyticsAre you looking beyond visits and page
views? Make sure you really make the most
of web analytics today.
WorkshopsTired of constantly hiring expensive external
consultants? *wink* I am more than happy to
train you and your team on various topics.
Public SpeakingFrom Google to Travelport, Finland to Spain,
I will speak and share my insights and
motivations during your exclusive event.
7/29/2019 Remote Usability Testing - Buyers Guide 2013
8/36
Section 3
Testimonials
7
Theres something fresh about Matthew Ive always appreciated as hes willing to share knowledge
and skills through his blog as well as through informal chats and internationally recognized publicspeaking events, at which he excels. His community focused efforts reflect his no-nonsense ap-
proach to gaining and sharing insights.
Matthew is able to focus on business and customer values, drawing on his broad knowledge of
qualitative and quantitative analytics tools hes gained through the many years in our industry. His
unique blend of digital analytics knowledge and user experience make him an ideal, pragmatic
candidate for any company aiming to take their online presence and data driven decision making
processes one step further.
Matthew is one of a kind; few are as dedicated, focused and knowledgeable as him. And make no
doubt, Matthew knows what he wants. But above all, Matthew gets things done with contagious
warmth and passion that Ive seldom experienced. Matthew Niederberger is no doubt one of the
fastest rising stars on the European ecommerce arena.
Tony Markovski
MassRelevance
Aurlie PolsMind Your Privacy
7/29/2019 Remote Usability Testing - Buyers Guide 2013
9/36
Matthew is on top of new developments in UX and analytics and very eager to try out new methods
and tools. He has a very clear picture of developments in the UX landscape. The variety of tools inhis toolbox help him to gain a very good understanding of his users and the way they behave and
think. Matthew is eager to learn, a smart guy, professional, and I look forward working with him in
the future.
Paul VeugenFounder Usabilla
Matthew is a thought leader in his niche, Ive enjoyed his keynotes several times. We have regular
talks about the tools in the market place and share our experiences in testing, analytics and travel
in general. I always learn something new!
Wouter BlokGoogle
7/29/2019 Remote Usability Testing - Buyers Guide 2013
10/36
Chapter 2
Remote
UsabilityTesting
What exactly is Remote Usability Testing and howcan your business benefit from it? Being a relative
new technology can be beneficial for some,
confusing for others. With a handful of service
providers in todays market, how can you best
choose who to deal with?
7/29/2019 Remote Usability Testing - Buyers Guide 2013
11/36
Remote Usability Testing, also known as unmoderated usabilitytesting, is a technique used by usability and user experience
professionals to conduct usability tests without a lab, and not in
the presence of the participant.
Simply put, it is an alternative to laboratory usability
testing where tests are:
setup and launched via an online portal
completed by participants sitting in the
comfort of their own home, in theory,
anywhere in the world
analyzed by the test owner**, online, back
behind his/her own desk (possibly at home as
well)
As in any usability test, the participants screen and
audio (voice) are recorded for later analysis. In some cases, the
participant is recorded as well, giving the test owner a complete
view of the participants behavior online, thoughts spoken
allowed, and possible facial expressions.
The latter is often overlooked, but often used as an argument infavor of laboratory usability testing. Recent technological
breakthroughs, however, have proven that even facial
expressions in the form of confusion, surprise, frustration, and
more, can be analyzed with remote usability testing.
Personally, I think the biggest differentiator between
the offline vs. online forms of usability testing is
the communication between the host and theparticipant.
Remote usability testing allows not only
enterprise size companies, but even a one-
man businesses, to discover critical insights in
web/mobile user behaviors.
Where tools like iPerceptions, Qualaroo, Usabilla,
and Mouseflow give us insights through customer
feedback forms and interaction analysis, remote usability testing
tools will give you a more richer experience in looking over the
shoulder and into the mind of the participant.
And to think, we are still in early phases of it all.
Section 1
What is it?
10
** Some providers offer enhanced services where they will analyze thevideos for you saving you time (it can take an hour to fully analyze a20 minute video), but also leveraging their expertise.
7/29/2019 Remote Usability Testing - Buyers Guide 2013
12/36
In a recent study* conducted among user experience specialists, in house and inagencies, the question was asked:
What one thing do you think would help most improve user experience for your
users?
The majority of the respondents on both sides answered that user testing would
help the most. It would help in not only improving the user experience but also in
improving return on investments. A better understanding, proof of visitors
struggling online, will help fight the common myths that:
user experience is simply a buzz word
companies know best what customers want
user testing costs too much
To help you better understand how remote usability testing can benefit you, I have
listed several of them here. The benefits have helped me over the years, so again
from a practitioners point of view, they should help you, too.
Better UnderstandingOver the past years my biggest observation with usability testing in general is its
unique power to create empathy among colleagues, but more importantly among
stakeholders.
Section 2
TOP 5 BENEFITS FOR YOU
1. Perform affordable usability research
2. Fast study turn around times
3. Uses less internal resources
4. Helps gain valuable insights
5. Innovative technology
How can it benefit you?
11
* eConsultancy User Experience Survey Report 2013
7/29/2019 Remote Usability Testing - Buyers Guide 2013
13/36
Empathy often leads to a willingness to better understand the
users experience on your website or mobile app. This alone
pushes your knowledge and advantage over competitors to a
higher level.
Cost Eff
ectivenessIf you have any experience at all with usability research, then I am
sure that you at one point in time have seen the bill for one of
these studies. I have never seen a study cost less than a significant
5-figure total.
Without alternatives, these kind of numbers would seem normal.
Lets do the math for a remote usability research study, using an
external consultant to perform the analysis:
10 participants $49 = $490
8 hours consultancy services $125/hour = $1000
- 2 hours for test preparation and setup
- 5 hours for video analysis (30 minutes per video)
- 1 hour post study consultation
So for $1490 you can, in theory, conduct a remote usability
research study. Yes, yes, I am underselling the consultancy hours,but in its basic form the numbers match up.
Dont forget, it is not only the time that your are saving with setup
and deployment of the research study. Laboratory usability
research studies require even more time to prepare, especially with
regards to recruiting panel members.
Panel RecruitmentAlright, great technology, but like other online user research tool,
such as customer feedback surveys, I will need a huge panel to
get some significant insights.
The above statement is what withholds a lot of people/companies from expanding user research efforts. A medium level
survey tool can cost anywhere between $49 to $99 a month, but
without visitors giving feedback it wont do you much use.
Remote usability testing tools like those mentioned in this guide
dont suffer from this.
All the service providers mentioned will allow you to make use of
their panel members for testing. The time consuming task ofrecruiting your own panel members is removed from the
equation.
Furthermore, all service providers give you control over
segmentation options for their panel. From age to income,
education to online skill levels.
12
7/29/2019 Remote Usability Testing - Buyers Guide 2013
14/36
Need more? It is quite common to see the feature allowing you to
invite your own panel members for your study.
TIP The service providers often guarantee a high level of quality
with regards to the videos, so keep in mind that when using your
own panel members this guarantee is voided.
Mobile UsabilityTalking about mobile adoption rates these days is like talking
about the weather. Everybody does it and everyone still acts
surprised as if it is an unexpected thunderstorm that will ruin their
weekend plans.
Mobile usability testing using remote research tools is a fact.
Testing can be performed on both Android and iOS devices
without the use of special software. The most common solution is
by using a webcam connected to a computer, pointed towards
the mobile device (often top-down).
Using this method allows you not only to see and hear the basics,
ie. the participants device screen and voice, but also to see the
movements of the hand.
Mobile remote usability testing can no longer be ignored. Not only
can you test mobile websites, but also apps (paid & free). Some
service providers even offer capabilities for testing unreleased
iOS apps, perfect for those moments where you want to start
early with your optimization cycle.
On both the technical and logistical spectrum the service
providers will arrange everything allowing you, as a business
owner, focus on setting up the test and reaping the benefits from
the insights.
Setup and AnalysisHas usability test engineering become a thing of the past? To a
certain extent, yes it has. With todays remote usability testing
tools there is definitely a lot that you can do yourself. It doesnt
matter if you are a freelancer testing for a client, or an in-house
specialist at some large company, the tools offer solutions that
will help anyone get started today.
Setting up a test is in all cases nothing more than a process of
filling in forms. Even with test templates to help you get started in
under a minute. If you are testing a specific hypothesis then some
offline work will be required to figure out how to setup the test,but even this is a one-time thing.
The more you setup tests, the more you learn what capabilities
there are, the more efficient you as a practitioner will become,
saving yourself, or your company, valuable time and money.
13
7/29/2019 Remote Usability Testing - Buyers Guide 2013
15/36
I often compare it to baking. Simple cupcakes dont require much
preparation and can be prepared in a breeze, but a multi-layer
cake with all the trimmings will take more time and effort. There is
no getting around it, neither in baking, nor remote usability test
setups.
Analyzing the results is what gives you the insights. Luckily mostservice providers offer a collection of online analysis tools to
make your life easy. Using these online analysis tools will help you
keep note of any interesting behaviors and comments simply by
adding annotations to the video.
Not stopping there, some service providers allow you to create
small clips from the video. These clips can later be viewed in a
highlight video. The highlight videos look and feel like compilationvideos. Time is money, so spend it wisely and by using these
form of compilation videos will allow you to see several insightful
moments from all participants in a single video clip.
Are you not sure if you have the skills to analyze remote usability
tests? I can understand. In the study I mentioned earlier, many
companies indicated that a lack of resources and/or specialist
knowledge was holding them back from starting with user testing.
Luckily for them professional services are available, not only from
myself, but also from the service providers themselves. The
biggest benefit to you is, is that if you want to analyze yourself,
you can.
Sharing InsightsAfter performing your analysis online you need to take action.
Action in the sense that your insights are worthless if no one actsupon them.
Maybe you found a technical problem during your analysis, or
maybe the majority of the participants had trouble finding your
hero products call-to-action button. Sharing these insights with
people responsible for the design/interaction of the website or
mobile app is essential.
Sharing can be done in several different ways, depending on the
service provider you choose. The most widely available feature is
downloading the videos.
Now although downloading sounds great, how many of your
colleagues will actually sit and watch several 15 minute videos?
Not many, believe me.
14
7/29/2019 Remote Usability Testing - Buyers Guide 2013
16/36
The two best options for sharing video are:
sharing an individual clip from a single participant
creating a compilation video from multiple clips
Because of the compilation options that several service providers
offer, you can easily create videos with highlights from all the
sessions, or highlights on a specific insights, and share these.
The last important sharing option, is the option to share your
annotations. As youve just read, annotating the video with your
insights during the analysis is a common feature. Downloading
those annotations, strangely, is not. Some service providers offer
it and leverages your capability to sharing insights with your team
to a stellar level.
Post Test SurveysCollecting user experience data is what it is all about with remote
usability research. This qualitative form of data is immensely
valuable but costs a lot of time and effort to collect, sort, and
report.
To help you get an idea of the participants thoughts after having
participated in a study, service providers offer you the opportunity
to ask questions. These questions are often asked at the end of
the video/audio recording part, and participants must type their
answers in a form.
Ranging from 3 to 100 post test questions, there is a differenceamong service providers in terms of limits. Either way it is an
extra chance for you to pick the participants brain.
Question types range from open ended questions to radio button
form fields, from checkboxes to likert scales. The most common
question you see among these surveys are NPS related
questions. In some cases scores from some of the survey
questions, or custom questions provided by the service providersthemselves can be benchmarked against other companies in your
industry.
Preparation, and some caution, should be observed with the
survey questions. Survey questions can be a powerful form of
insights, but asking the proper question in an unambiguous
manner is key to gaining significant insights from survey
responses.
Time To InsightsDid I save the best for last? Maybe, that is for you to judge. When
I talk about remote usability testing I do seem to talk about turn
around times a lot. With that, I mean the time it takes for a study
15
7/29/2019 Remote Usability Testing - Buyers Guide 2013
17/36
to be completed, from when a study is launched, until all the
participants have completed and uploaded it.
I put on a smirk every time I hear marketeers talk about their bi-
annual usability research studies. Studies that take weeks to
prepare, and weeks to analyze. Yes, there are benefits to
laboratory research over the remote kind, but every day thedistinction between the two is gradually disappearing.
Why wait 2 weeks, a month, or even more to get your study
results. When looking at todays remote usability research
providers, the longest turn around time is still only a mere 72
hours. Of course, you still need to analyze the results or have
them analyzed for you by a providers professional services, but
like I mentioned early, through the use of online analysis toolseven this becomes less time consuming, giving you the time to
focus more on your business.
16
7/29/2019 Remote Usability Testing - Buyers Guide 2013
18/36
Chapter 3
Service
Providers
With three main players in the market, it can get
confusing who is the right match for you. Although
the technology seems similar, there is quite a
difference between features, pricing, quality, and
even turn around times.
7/29/2019 Remote Usability Testing - Buyers Guide 2013
19/36
UserTesting.com provides the fastest and most affordable web, desktop and
mobile app testing in the market. In less than one hour, the company gives
marketers, product managers and UX designers, on-demand access to users in
their target audience, who deliver audio, video and written feedback on websites
or apps. UserTesting.com is also the only service that allows customers to test
unreleased apps and get detailed feedback within a day.
Enterprise solutions are also available for clients who need a more robust testing
solution. In addition to UserTesting.coms standard service, enterprise clients haveaccess to a dedicated research team, an expanded recruiting pool with millions of
real people, user intercepts, moderated testing, and access to industry experts like
Steve Krug, Brian Eisenberg, Tim Ash and more.
UserTesting.com customers include
Section 1
TOP 5 KEY FEATURES
1. Video, audio, and written feedback within
an hour
2. Test any website or mobile app, including
your competitors
3. Observe participants in their natural
environment
4. Large panel ensures that participants
match your target audience
5. Improve your conversion rate or your
money back
18
VIEW SAMPLE VIDEO
http://youtu.be/WDDNp146G9shttp://youtu.be/WDDNp146G9shttp://youtu.be/WDDNp146G9shttp://youtu.be/WDDNp146G9shttp://youtu.be/WDDNp146G9s7/29/2019 Remote Usability Testing - Buyers Guide 2013
20/36
Used by the top 10 web properties in the U.S., UserTesting.com
has run hundreds of thousands of usability tests.
UserTesting.com is headquartered in Mountain View, CA. User
panels are currently available in the U.S., Canada, and the UK.
For more information, visitwww.usertesting.com orPhone:1-888-877-1882
Email: [email protected]
Twitter: @UserTesting
19
http://twitter.com/usertestinghttp://twitter.com/usertestingmailto:[email protected]:[email protected]://www.usertesting.com/http://www.usertesting.com/7/29/2019 Remote Usability Testing - Buyers Guide 2013
21/36
Based in Europe, WhatUsersDo is one of the most established providers in the
Remote Usability market. Serving over 1,500 Digital Professionals from their
London HQ they have been operating for more than five years - regularly testing
across 20+ countries in multiple languages.
WhatUsersDo clients are as diverse as EasyJet, Telefonica, Tesco, Virgin Money
and Sage as well as thousands of SMEs any organisation that wants to put user
insight at the heart of their decision-making.
WhatUsersDo are transforming the working practices of organisations who
recognise that basing design decisions on user insight, rather than internal
opinions or hunches, makes for a better experience for their customers and more
successful digital properties. They do this by answering the why of user
behaviour (e.g. Why do people leave my website or App?) that completes the
partial picture that analytics packages provide (e.g. the what and how many of
Google Analytics).
Whatusersdo customers include
Section 2
TOP 5 KEY FEATURES
1. International panel
2. SmartPhone & Tablet Testing
3. Enterprise Plans & Agency Packs
4. Recruitment options
5. Analysis by WhatUsersDo
20
VIEW SAMPLE VIDEO
http://youtu.be/Ipp3v8fFG6Qhttp://youtu.be/Ipp3v8fFG6Qhttp://youtu.be/Ipp3v8fFG6Qhttp://youtu.be/Ipp3v8fFG6Qhttp://youtu.be/Ipp3v8fFG6Qhttp://youtu.be/Ipp3v8fFG6Q7/29/2019 Remote Usability Testing - Buyers Guide 2013
22/36
WhatUsersDo offers two levels of service:
1. Pay As You Go self-serve testing for SMEs who need
actionable feedback from users a few times per year.
2. Managed Service for larger clients and agencies who want to
embed UX Testing in their processes and test throughout the
Development Lifecycle.
The Managed Service plan offers an enhanced service level that
includes expert advice in test design, analysis of results and
advanced testing allowing clients to truly embed UX Testing in
their design and development processes. On these enhanced
plans customers benefit from:
- enhanced panel options: selecting users from the WhatUsersDo
international consumer panel, invite their own customers or
recruit on-demand
- full-lifecycle testing: from concept through to wireframe to live
site benchmarking
- multi-channel testing: including desktop, tablet and SmartPhone
(or even all three)
- international testing: run multi-lingual tests with analysis (andparticipants spoken thoughts) presented in their own language.
For more information on service levels contact
[email protected] or call +44 (0)207 036 1950 or visit
www.whatusersdo.com
Twitter: @whatusersdo
21
http://twitter.com/whatusersdohttp://www.whatusersdo.com/mailto:[email protected]://twitter.com/whatusersdohttp://twitter.com/whatusersdohttp://www.whatusersdo.com/http://www.whatusersdo.com/mailto:[email protected]:[email protected]7/29/2019 Remote Usability Testing - Buyers Guide 2013
23/36
YouEye is an online research platform that features remote usability testing.
YouEye empowers companies to quickly identify the biggest opportunities to
improve their products, websites or services. With webcam-based usability
testing, study analytics and customized feedback communities, it's never been
easier or more affordable to build amazing experiences.
YouEye has worked with leading companies including Unilever, Walmart Labs,
SapientNitro, Airbnb and others, and offers customized offerings to enterprise and
agency customers based on their specific needs and business goals. YouEye also
offers a touchless solution for rapid feedback. Participants for these studies start
at $39, and most results are available within 48 hours.
YouEye customers include
Section 3
TOP 5 KEY FEATURES
1. Emotion Recognition technology
2. Webcam recording
3. Dedicated research team for study creation
and analysis
4. Custom tester demographics
5. Human review of each result
forguaranteedquality
22
VIEW SAMPLE VIDEO
http://youtu.be/iNKkNkH9UPIhttp://youtu.be/iNKkNkH9UPIhttp://youtu.be/iNKkNkH9UPIhttp://youtu.be/iNKkNkH9UPIhttp://youtu.be/iNKkNkH9UPI7/29/2019 Remote Usability Testing - Buyers Guide 2013
24/36
By implementing the YouEye solution into your organization, you
can gain a huge edge on competitors, delight your customers and
increase your conversions.
You can learn more about YouEye athttp://www.youeye.com.
Reach out to us at [email protected], andfollow @YouEyeTests on
Twitterto keep up with the latest updates.
23
http://twitter.com/youeyetestshttp://twitter.com/youeyetestsmailto:[email protected]:[email protected]://www.youeye.com/http://www.youeye.com/7/29/2019 Remote Usability Testing - Buyers Guide 2013
25/36
Chapter 4
The
Comparison
How do these service providers weigh up against
each other? On the following pages critical features,
from a practitioners point of view, will be compared.
From Pricing Plans to Test Setup, and from Analysis
to Customer Service.
S i 1
7/29/2019 Remote Usability Testing - Buyers Guide 2013
26/36
Whatusersdo
Start price per participant $39 30 (approx $45) $39
Offers free trial
Offers bulk discount
Offers Premium plans
Section 1
Plans & Pricing
25
S ti 2
7/29/2019 Remote Usability Testing - Buyers Guide 2013
27/36
Whatusersdo
Browser testing
Prototype websites
Upload prototype files **
Mobile/Tablet apps
Unreleased apps ** ** **
Screen & Audio Recording
Webcam Recording
Emotion Registration
Max recording time per test *** 15 mins 20 mins 15 mins
= Innovative and/or Unique Feature
Section 2
Test & Recording Types
26
Legend
* iOS only
** Premium accounts only (ie. Enterprise level plans)
*** Suggested limits
Section 3
7/29/2019 Remote Usability Testing - Buyers Guide 2013
28/36
Whatusersdo
Panel Members Included
Panel Member Nationalities US, CA, UKUS, UK, AUS, FR,
DE, IE, IT, NL, ES, PLUS
Panel Segmentation
Use own panel members
Screen Participants ** *** **
Rate participants
= Innovative and/or Unique Feature
Section 3
Panel Recruitment
27
Legend
** Premium accounts only (ie. Enterprise level plans)
*** UK panel members only
Section 4
7/29/2019 Remote Usability Testing - Buyers Guide 2013
29/36
Whatusersdo
Setup wizard/walkthrough
Test templates
Task library
Multi language interface
Post Task Survey
NPS
Add video markers
Add annotations
Create clips
Create Highlight Video
Section 4
Test Setup & Analysis
28
Legend
** Premium accounts only (ie. Enterprise level plans)
Section 5
7/29/2019 Remote Usability Testing - Buyers Guide 2013
30/36
Whatusersdo
Money back guarantee
Quality Assurance
Project Management ** ** **
Analysis Service ** ** **
Average test turn around time < 1 hr
7/29/2019 Remote Usability Testing - Buyers Guide 2013
31/36
Chapter 5
Thoughts for
2014
Whats in store for 2014? Where will new
technological advances take the field of remote
usability testing? Let the service providers and
myself share our thoughts with you.
7/29/2019 Remote Usability Testing - Buyers Guide 2013
32/36
Co-Founder Dave Garr
Computing is moving from one screen a computer accessing the web to four screens: a computer, a tablet,
a phone, and a smart TV. The next big UX trend is understanding that user experience does not mean just web
site experience or mobile experience or phone experience or store experience.
Companies will stop designing each experience in a vacuum. Theyll start putting together the holistic
understanding of the entire user experience.
I also see a lot of work being done on sentiment analysis -- software that mines text for meaning and insight. Its
often used to extract opinions and emotions from social media to help companies determine how people feel
about their products.
Sentiment analysis is an extremely difficult problem, but its a problem that will eventually be solved. When the
problem is solved, itll cull through a massive amount of text and automatically call attention to the biggest, most
frequently mentioned issues. This will increase the value of qualitative tools like open-ended survey questions and
transcripts of user testing.
7/29/2019 Remote Usability Testing - Buyers Guide 2013
33/36
CEO Kyle Henderson
The future of usability testing will look and feel like the analytics products we use today.
We will see big data meet qualitative feedback. New technologies and forms of measurement will appear.
Cross discipline collaboration for discovering insights will also move to the web.
We'll have teams engaging around UX quality and improvements through online social interactions.
The future of usability is in the cloud.
7/29/2019 Remote Usability Testing - Buyers Guide 2013
34/36
Founder Lee Duddell
In many ways, were only just starting to see remote UX testing become widely adopted. A survey we ran with
Econsultancy at the end of 2012 showed that although 78% of brands were committed to providing the best
possible UX, over 60% of them were still basing digital design decisions on hunches and opinions. Its clear from
this that, although available for many years, lab-based approaches do not provide the answer to closing this gap.
In just the past 12 months weve seen remote UX testing becoming truly embedded with our clients who now see
it as an essential part of both their development process as well as a means to conducting user research toinform strategic decisions.
In the coming months and years we will be developing WhatUsersDo to support even better decision-making and
ultimately improved return from digital channels. While I cant reveal all of the details, expect to see:
- increased multi-channel capabilities
- more support for beyond usability insight and testing
- integration with third party tools
- an emphasis on user-generated, rather than purely task-based, insights to better understand users natural
behaviour.
7/29/2019 Remote Usability Testing - Buyers Guide 2013
35/36
Founder Matthew Niederberger
As a practitioner I have been lucky enough to have tested all the tools in this guide. One thing to keep in mind
when considering one of the fabulous tools is that no tool is truly 100% alike. Each tool has its own set of useful
features, each suited for specific needs, your needs.
In 2014 I expect many of these differences to grow. The feature gap between the tools is what makes them
unique, and I think that every single service provider will continue to strive for that uniqueness, even though the
basics will remain the same.
A trend that I am starting to see and one that I hope continues to grow is more quantifiable data, more usability
metrics. Analyzing user videos takes a lot of time and effort and remains subjective to the professional analyzing
it. By incorporating a small amount of metrics into the tools will help speed up the understanding and adoption of
the solutions within companies.
I think that 2014 will see more expansion of these solutions into the European arena offering more multi-lingual
panels. The service companies, mainly the US based ones, will have to look for alternative recruiting methods,
beyond the often used Mechanical Turk.
There are challenges ahead, but also plenty of opportunities.
C i h
7/29/2019 Remote Usability Testing - Buyers Guide 2013
36/36
Copyright
xxxv
2013 Matthew Niederberger/actualinsights*