User Research Techniques by Vikram Rao, RSA

User Research User Research TechniquesTechniques

Vikram Rao Vikram Rao STC India UX Session - STC India UX Session -

BangaloreBangalore2525thth June 2011 June 2011

AgendaAgenda Context around User Research Context around User Research User Research Techniques User Research Techniques

Evaluate Evaluate somesome User Research User Research TechniquesTechniques

Pros and Cons of the TechniquesPros and Cons of the Techniques Choosing a User Research Choosing a User Research

TechniqueTechnique

Context around Context around User Research User Research

Fact of the matter…Fact of the matter… What are the problem we discover What are the problem we discover

almost every time we analyze a almost every time we analyze a project going awry?project going awry? Badly Define RequirementsBadly Define Requirements We don’t know much about our We don’t know much about our

users and their needs users and their needs Poor Communications Poor Communications Stakeholder politics Stakeholder politics etc. etc.

““KnowingKnowing and and caringcaring about about users and their lives is users and their lives is

absolutely at the core of absolutely at the core of user-centered design.”user-centered design.”

What/Why? How?

Summarizing User Summarizing User ResearchResearch

KnowingKnowingthe Userthe User

wellwell enoughenough

>>>> UserUserResearchResearch



Data Mining / Analysis Data Mining / Analysis Data mining is the process of Data mining is the process of extracting extracting

hidden patternshidden patterns from data. from data. As more data is gathered, with the As more data is gathered, with the

amount of data doubling every three amount of data doubling every three years, data mining is becoming an years, data mining is becoming an increasingly important tool to transform increasingly important tool to transform this data into information.this data into information.


Pros of using Data Mining / Analysis Pros of using Data Mining / Analysis Leveraging existing sources (internal or Leveraging existing sources (internal or

external) of informationexternal) of information Cons of using Data Mining / AnalysisCons of using Data Mining / Analysis

Very time consuming Very time consuming The raw data needs to be scanned, The raw data needs to be scanned,

sorted and analyzed – for which we sorted and analyzed – for which we need skilled UX Researchers or need skilled UX Researchers or Analysts to perform the exerciseAnalysts to perform the exercise


User TripUser Trip A User Trip is a technique where A User Trip is a technique where youyou

take a ‘trip’ through the whole processtake a ‘trip’ through the whole process of using a particular product or system, of using a particular product or system, making yourself a critical observant making yourself a critical observant user. user.


Pros of using User Trip Pros of using User Trip It would help us gain a good understanding of the It would help us gain a good understanding of the

requirements and provide suggestions towards requirements and provide suggestions towards additional features in the proposed application. additional features in the proposed application.

Detailed review of the UI would help us to Detailed review of the UI would help us to understand the pain points with the existing understand the pain points with the existing scenario/ application.scenario/ application.

We can preempt a lot of issues with the UIWe can preempt a lot of issues with the UI Cons of using User Trip Cons of using User Trip

The environment or situation in which the actual The environment or situation in which the actual user is likely to perform the task cannot be user is likely to perform the task cannot be simulated.simulated.

If the exact user profile is not considered during If the exact user profile is not considered during simulation by the person who is conducting the user simulation by the person who is conducting the user trip, it might not be effective.trip, it might not be effective.


Observation ExerciseObservation Exercise An observation Exercise is a technique An observation Exercise is a technique

where where you ‘observe’ actual users or you ‘observe’ actual users or representative users through the whole representative users through the whole processprocess of using a particular product or of using a particular product or system. system.


Pros of using Observation Exercise Pros of using Observation Exercise Collection of detailed, descriptive narrative data Collection of detailed, descriptive narrative data

about the user’s profile, product usage, etc. about the user’s profile, product usage, etc. Can be referred to throughout the year; useful Can be referred to throughout the year; useful

for instructional planning. for instructional planning. Cons of using Observation Exercise Cons of using Observation Exercise

Needs to be conducted in classroom or work Needs to be conducted in classroom or work environment which may its own limitations / environment which may its own limitations / distractionsdistractions

Scheduling the sessions (time required, when to Scheduling the sessions (time required, when to observe) is a challenge observe) is a challenge

Great deal of expertise and objectivity required.Great deal of expertise and objectivity required.


User InterviewsUser Interviews User Interviews User Interviews help us in better help us in better

understanding the user’s ‘goals’ and understanding the user’s ‘goals’ and ‘needs’‘needs’ by actually talking to them. by actually talking to them.

The information would help us make The information would help us make better decisions in further improving better decisions in further improving the user experience and capabilities of the user experience and capabilities of the product or system. the product or system.


Pros of using User Interviews Pros of using User Interviews Interviews are a great way to find in-depth Interviews are a great way to find in-depth

information about your users. information about your users. They're fairly easy to organize.They're fairly easy to organize. Typically need very little facilities / tools and Typically need very little facilities / tools and

can also be done remotelycan also be done remotely.. Cons of using User InterviewsCons of using User Interviews

Laborious process if done with many usersLaborious process if done with many users Getting the right users that match the persona Getting the right users that match the persona

could be a challenge.could be a challenge.


Contextual InquiryContextual Inquiry Contextual inquiry involves collecting Contextual inquiry involves collecting

detailed information about customer detailed information about customer work practice by work practice by ‘observing’ and ‘observing’ and ‘interviewing’ the user‘interviewing’ the user while they while they actually work. actually work.

The researcher should stay in the The researcher should stay in the background and let the user lead the background and let the user lead the situation as much as possible. situation as much as possible.


Pros of using Contextual Inquiry Pros of using Contextual Inquiry A chance to see what really happens and also A chance to see what really happens and also

clarify in contextclarify in context Provides data about environmental factorsProvides data about environmental factors Critical source of information for Persona and Critical source of information for Persona and

scenario development.scenario development. Cons of using Contextual InquiryCons of using Contextual Inquiry

Analysis relies on the capabilities of the Analysis relies on the capabilities of the observerobserver

Possibility of no follow-up or probing means it's Possibility of no follow-up or probing means it's possible to make wrong inferencespossible to make wrong inferences

Only provides data on the observed work tasks Only provides data on the observed work tasks and environment.and environment.


Split A/B TestingSplit A/B Testing Split A/B testing is a technique of Split A/B testing is a technique of

finding out which changes or options finding out which changes or options help your users' performancehelp your users' performance. .

It provides a controlled method of It provides a controlled method of measuring the effectiveness (or not) of measuring the effectiveness (or not) of alterations to your site, product or alterations to your site, product or system. system.


Pros of using Split A / B Testing Pros of using Split A / B Testing Low risk approach Low risk approach Cheaper than other methods such as focus groups Cheaper than other methods such as focus groups Provides proof Provides proof Great way to do 'test run' new designs before full roll-out (to Great way to do 'test run' new designs before full roll-out (to

avoid negative surprises on the launch day) avoid negative surprises on the launch day) Can solve internal disputesCan solve internal disputes

Cons of using Split A / B TestingCons of using Split A / B Testing New designs might have to 'wear in' before you can measure New designs might have to 'wear in' before you can measure

their real performance (visitors' initial response might be their real performance (visitors' initial response might be negative because they're used to the old solution) negative because they're used to the old solution)

You can only compare 2 versions with a single factor that You can only compare 2 versions with a single factor that differentiates both designs (for more factors / variations you differentiates both designs (for more factors / variations you need to deploy multivariate testing which is more difficult to need to deploy multivariate testing which is more difficult to analyze) analyze)

It takes technical know-how to set up and analyze the results It takes technical know-how to set up and analyze the results You're testing in a live environment so external factors might You're testing in a live environment so external factors might

have an impact on the outcome have an impact on the outcome If your site / system is really in bad shape, this technique does If your site / system is really in bad shape, this technique does

not help as you'll still need to do a proper overhaul.not help as you'll still need to do a proper overhaul.


Diary StudyDiary Study Diary Study is a technique where Diary Study is a technique where

participants collect data about events participants collect data about events on site as they happenon site as they happen..

Researcher would have to define Researcher would have to define templates to be handed out to templates to be handed out to systematically collect data. systematically collect data.


Pros of using Diary Study Pros of using Diary Study Need very little time to plan and organizeNeed very little time to plan and organize Can be done parallely with other ongoing Can be done parallely with other ongoing

tasks and research activities tasks and research activities Cons of using Diary StudyCons of using Diary Study

Highly dependant on the users recording Highly dependant on the users recording the right information the right information

Users need to be motivated to record Users need to be motivated to record activities and pass it alongactivities and pass it along


Focus Groups Focus Groups A focus group could be defined as a A focus group could be defined as a

group of interacting individuals having group of interacting individuals having some common interest or some common interest or characteristics, brought together by a characteristics, brought together by a moderator, who uses the group and its moderator, who uses the group and its interaction as a way to interaction as a way to gain information gain information about a specific or focused issueabout a specific or focused issue. .


Pros of using Focus Groups Pros of using Focus Groups Collection of detailed, descriptive narrative Collection of detailed, descriptive narrative

data about the user’s profile, product usage data about the user’s profile, product usage experience, etc. experience, etc.

Very cost effective in terms of gathering Very cost effective in terms of gathering primary data and are very much time efficient primary data and are very much time efficient

Cons of using Focus GroupsCons of using Focus Groups It is difficult to gather users together for this It is difficult to gather users together for this

exerciseexercise Opinions might get biased based on comments Opinions might get biased based on comments

from other participating membersfrom other participating members


User Surveys / QuestionnairesUser Surveys / Questionnaires User Survey is an offline means of User Survey is an offline means of

finding out how the software or web site finding out how the software or web site is likely to be used by a specific set of is likely to be used by a specific set of users, some of usability pain points and users, some of usability pain points and other generic information. other generic information.


Pros of using User Surveys and Questionnaires Pros of using User Surveys and Questionnaires Need very little time to plan and organizeNeed very little time to plan and organize A cheap and effective way of capturing information. A cheap and effective way of capturing information.

Cons of using User Surveys and QuestionnairesCons of using User Surveys and Questionnaires It is a one sided. We typically get answers to the It is a one sided. We typically get answers to the

questions that we have – we not actually listening questions that we have – we not actually listening to the user.to the user.

You can only have a limited set of questions as You can only have a limited set of questions as surveys and questionnaires cannot be very long. surveys and questionnaires cannot be very long.

Choosing a User Research Choosing a User Research TechniqueTechnique

Choosing a techniqueChoosing a techniqueYou need to consider the following You need to consider the following

when choosing a user research when choosing a user research technique:technique:

GoalsGoals: Why you’re doing the research?: Why you’re doing the research? Information RequiredInformation Required: Qualitative vs : Qualitative vs

Quantitative?Quantitative? ScheduleSchedule: When to do the research?: When to do the research? BudgetBudget: How much it’s going to cost?: How much it’s going to cost?

Choosing a techniqueChoosing a techniqueUser Research Technique

New Product or

System

Obtain Specific Results

Have Time on Hand

High Budget Availability

Access to Users

Need Qualitative

Data

Data Mining / Analysis X

User Trip X

Observation Exercise X X X X

User Interviews X X X X

Contextual Inquiry X X X X X

Split A / B Testing X X X

Diary Study X

Focus Groups X X X XUser Surveys /

Questionnaires X X X

Questions…Questions… You can also reach me at: You can also reach me at:

[email protected]@gmail.com | | 98456949799845694979

mailto:[email protected]

Design

User Research Techniques by Vikram Rao, RSA