PowerPoint Presentation

How to conduct field research for mobile appsMAY 2016

Workshop GoalsUnderstand the pros/cons and different considerations for field researchHow to develop a research plan for mobile field researchPractice conducting a field studyPresent findings and reflect on field studies for mobile apps

Schedule5:30 6:15pmDesigning, planning and conducting field research6:45 7:30pmForm groups and develop a field study plan7:30 8:00pmBreak8:00 8:45pmConduct field study8:45 9:30pmAnalyze and present findings and reflections

Suzanne BoydFOUNDER & CEO

Introductions

Ian WyosnickUSER RESEARCHER

INTRODUCTIONSWere a user-centered design consultancy focused on government agencies, nonprofits and enterprises with a social-impact mission.

5

Clients and impact

6

User-centered design process

UCD takes the guesswork out of designing usable experiences

A philosophy that places users at the center of the development processA systematic approach to gathering, understanding and applying user data throughout the planning, design and development stages of a website.

There are five basic characteristics that define the UCD process:Goal-driven and user focused: Identify who the target users are, how they work and what their goals are. Align user goals with business goals to lay the foundation for a successful project. Empirical: Based on data not opinions. Usability, usefulness and value are measured by target users behavior with the web site, not opinions about the web siteIterative: Try out ideas in a low-risk format. Success requires drafts and revisions in an agile and customer-driven process. We investigate solutions with users prior to coding by creating prototypes and mockups. When problems are found in the web site or applications during usability studies, they are fixed in subsequent versions before the system is launched.Collaborative: Multidisciplinary, requiring a team approach. Leverage a variety of skills sets, such as project management, user-centered design, graphic design, interaction design, writing, and coding. Scalable: Adapts to fit resources and time. The user-centered design process scales to different challenges such as budget constraints, scheduling, and resource constraints.

7

Getting out of the lab and into the field

The field

Coffee shopsLibrariesConferencesHospitalsCourt houseFerry terminalsFarmers marketsBusses, trains and light railStations and bus stops

Ian Wyosnick () - recommending this picture since we re-use the original one (with Tori, Mike and Melanie) later in the deckBehaviors and products we studyWayfindingTrip planningPurchasingInfo seekingGamingWeb sites / prototypesKiosks Apps

Focus on mobile apps

Some statsInternet usage 51% mobile / 42% desktopTime spent on screens 2.8 hours mobile / 2.4 hours desktop90% of mobile media time is via apps. Top apps

Mobile Marketing Statistics Compilation by Dave Chaffey - 2016Internet usage 51%42%2.8 hrs / day2.4 hrs / dayTime on screens

80% of internet users own a smartphoneMobile includes smartphone and tablet. Usage includes both work and home.

Facebood and other social mediaYoutubeEntertainment and gaming

13

Why do field studies?

What is a field study? A field study is a general method for collecting data about users, user needs, and product requirements that involves observation and interviewing. Data are collected abouttaskflows, inefficiencies, and the organizational and physical environments of users.Investigators in field studies observe users as they work, taking notes on particular activities and often asking questions of the users. Observation may be either direct, where the investigator is actually present during the task, or indirect, where the task is viewed by some other means like a video recorder set up in an office. The method is useful early in product development to gather user requirements. It is also useful for studying currently executed tasks and processes.

Usability Body of Knowledge

Research methodsAnalyticsEye trackingObservationsSurveysInterviewsUsability testing

Components of field research

Contextual inquiry (CI) is a user-centered design (UCD) research method, part of the Contextual Design methodology. A contextual inquiry interview is usually structured as an approximately two-hour, one-on-one interaction in which the researcher watches the user do their normal activities and discusses what they see with the user.

Ethnography (from Greek ethnos "folk, people, nation" and grapho "I write") is the systematic study of people and cultures. It is designed to explore cultural phenomena where the researcher observes society from the point of view of the subject of the study. An ethnography is a means to represent graphically and in writing the culture of a group. The word can thus be said to have a "double meaning," which partly depends on whether it is used as a count noun or uncountably.[1] The resulting field study or a case report reflects the knowledge and the system of meanings in the lives of a cultural group.[2][3][4

ethnography: Ethnographic field research involves the study of groups and people as they go about their everyday lives. First-hand participation in some initially unfamiliar social world and the production of written accounts of that world by drawing upon such participation [is the core of ethnographic research].Emerson, Fretz, and Shaw - See more at: http://www.uxmatters.com/mt/archives/2010/06/ethnography-in-ux.php#sthash.s8oqsD2N.dpuf17

Ian Wyosnick () - Surveys, interviews, usability testing and observations can all be part of field researchThe spectrum of field researchRosenbaum & Kantner (2007)exploratoryresearch

structuredresearch

ethnography

contextual inquiry

field usability study

user + toolenvironment / contextuser + toolThe field

The field is the environment / context in which your users interact with or use your product.

If youre doing research with planning trips busses an stopsIf youre researching social media while in transport, waiting anywhereIf youre researching a grocery app If youre researching photo appIf youre researching 19

Lets brainstormPublic transit trip planner appSocial media appGrocery list appAir travel appGame appBusses, trains, light rail, stations, bus stopsIf you are researchingThen the field may be

Your thoughts

What have you heard about field studies?

Contextual inquiry (CI) is a user-centered design (UCD) research method, part of the Contextual Design methodology. A contextual inquiry interview is usually structured as an approximately two-hour, one-on-one interaction in which the researcher watches the user do their normal activities and discusses what they see with the user.

Ethnography (from Greek ethnos "folk, people, nation" and grapho "I write") is the systematic study of people and cultures. It is designed to explore cultural phenomena where the researcher observes society from the point of view of the subject of the study. An ethnography is a means to represent graphically and in writing the culture of a group. The word can thus be said to have a "double meaning," which partly depends on whether it is used as a count noun or uncountably.[1] The resulting field study or a case report reflects the knowledge and the system of meanings in the lives of a cultural group.[2][3][4

ethnography: Ethnographic field research involves the study of groups and people as they go about their everyday lives. First-hand participation in some initially unfamiliar social world and the production of written accounts of that world by drawing upon such participation [is the core of ethnographic research].Emerson, Fretz, and Shaw - See more at: http://www.uxmatters.com/mt/archives/2010/06/ethnography-in-ux.php#sthash.s8oqsD2N.dpuf21

Benefits of field studies

Study context of useEfficientAncillary findings

Early morning at King Street Station | Seattle, WA

Directly observe context of use

Context of use People LightWeather Other apps ArtifactsOthers?

24

EfficientResearch with many people in a dayMinimal recruitment costRepresentative participantsNo lab costs

Ancillary findingsPain pointsUnmet needsBehaviors

Ian Wyosnick () - I thought this image would represent 'ancillary findings' better as she is looking at something away from mobileChallenges of field studies

Logistics and organizationWeatherPrivacy and recordingOff topic conversationDifficult participantsRejectionObserversFatigue

DesignOPTIONS AND CONSIDERATIONS

Considerations for your study designRecordedNotrecordedgenerativeevaluativeSoloTeamStaying putRecruited in advance InterceptObservationScenariosRemoteIn personWhy are you conducting research?summativeformativeWho is conducting the research? soloteamrecruitedinterceptHow will you find participants?How will you capture the data?video/audiono video/audioWhere will you go? on the goset up shopWhose device will you use? participantresearcher

Generative researchhelps you define the problem youd like to design a solution for.Evaluative researchevaluates an existing design (in prototype form or in final form).

30

+ Go where the participants are+ Find a captive audience- Less control over circumstances and permissions.

+More control+Easier to record+Use your own devices-Less ancillary findings

ON THE GOSET UP SHOP+Go where the participants are+Find a captive audience-Less control

Ian Wyosnick () - Aligned the word and added the negative bullet to the 'Set Up Shop'Journal studiesRecruited participants log their behaviors with the app over a period of timeCapture experiences whenever and wherever they happenMany emerging tools available

Recording Equipment CameraiPadClip boardPaper and pencilSled

Recording Software & apps

Input is the appInput is the video camera

34

Prepare LOGISTICS

Prepare study materials

Study plan and research questionsRecruitment criteria and screenerConsent formFacilitator scriptScenario(s)Note taking sheetsPost-test interview and metricsHonoraria

Work the logisticsObtaining permissionsElectricity availability and considerationsInternet availability and considerationsOfficial identificationStaff schedules and points of contact in the fieldScheduling and communication during the studyTip: Schedule sync-ups during the day to revise study plan accordingly

Dont forget the detailsWeather appropriate clothingExtra pens and pencilsClipboardsCamera/Recording devicesBackpacks or other bags to carry everythingSnacks to keep you goingClear instructions for any recruited participants on where park and meet up

Conduct the study

Recruit and screen participants

Find willing participantsWhat are the behaviors or approximate personas you are looking forHook emGive them the pitchScreen for fit to user profile

Facilitating sessionsKeep sessions short and within the promised amount of time(i.e. if you say it will take 15 minutes, make sure it does)Stay on script as it will ensure some rigor and consistency to the study

Take good notesTake a picture of the participantAgree on codes with other researchersUse forms Leverage checkboxesAnticipate usability issues and count instancesIf the study is free flowing, blank note sheets may be best

Example

Usability IssuesP1P2P3Participant has Yelp downloaded on phonexxParticipant uses multiple apps to complete scenarioxParticipant does not use filtersxxx

Scenario 1P1P2P3CompletionSFSPost scenario metric score425# of requests for help010# of errors131

Capture and synch up on themes as you go

Log problems and positives after each participantDebrief with team along the way

Recruitment and screeningTimeline

Specificbehavior

Random

Agrees to participate

Declines to participate

How do I select a participant?

Ok thankyou, have a good day!

Hook em!

Initial Screener

Qualified

Disqualified

Consent

Agrees to participate

Declines to participateStart Study!15sec15sec30sec

Interview and study facilitationTimeline cont

Qualified

Disqualified

Taking all qualified participants

Result of recruitment & screening

Ok thankyou, have a good day!

Follow up demographicsThankyou, have a good day!2min10min

Optional:Do they fit in a needed user segment?NoYes

Determine scenarioRun usability testPost-study interview and metrics2min15min total

Analyze

Analyze dataCompile quantitative dataTask success, Time on task, Satisfaction, Error rates, etc. Analyze qualitative dataProblems experiences, comments / recommendations, pathways, think a loud protocolDevelop exact and concise problem statementsGood problem statement: 12/14 participants clicked on notifications instead of newsfeedPoor problem statement: Clicked on wrong link.Poor problem statement: Was confused about links.

ReminderFocus on what participants did and said.

Your field research mission

MissionYou have just signed a contract to do usability work for Yelp. They have been experiencing a decrease in users and realize that they may need to improve their product. Yelp wants you to do field research to get qualitative and quantitative feedback on user behavior with mobile devices when deciding to find a restaurant while on the go.Design a field research study to understand how apps or websites are determining a users choice of where to eat, how they use those apps or websites, and how they may integrate with other apps or websites to do auxiliary tasks (i.e. getting directions and travelling to the restaurant).

You have just signed a contract to do usability work for Yelp. They have been experiencing a decrease in users and realize that they may need to improve their product. They are interested in doing field research to get qualitative and quantitative feedback on user behavior with mobile apps when deciding to find a restaurant while on the go.

Design a field research study to understand how apps are determining a users choice of where to eat, how they use those apps and how they may integrate with other websites or apps to do auxiliary tasks (i.e. getting directions and travelling to the restaurant).52

StepsCreate study planResearch questions, roles and responsibilities, scenariosConduct field research (at least 3 participants)Analyze dataReport on findingsStudy approachFindingsLessons learned and takeaways

Your thoughts

What have you learned about field studies?

Feedback survey linkwww.uxpa2016.org/sessionsurvey?sessionid=292

Contextual inquiry (CI) is a user-centered design (UCD) research method, part of the Contextual Design methodology. A contextual inquiry interview is usually structured as an approximately two-hour, one-on-one interaction in which the researcher watches the user do their normal activities and discusses what they see with the user.

Ethnography (from Greek ethnos "folk, people, nation" and grapho "I write") is the systematic study of people and cultures. It is designed to explore cultural phenomena where the researcher observes society from the point of view of the subject of the study. An ethnography is a means to represent graphically and in writing the culture of a group. The word can thus be said to have a "double meaning," which partly depends on whether it is used as a count noun or uncountably.[1] The resulting field study or a case report reflects the knowledge and the system of meanings in the lives of a cultural group.[2][3][4

ethnography: Ethnographic field research involves the study of groups and people as they go about their everyday lives. First-hand participation in some initially unfamiliar social world and the production of written accounts of that world by drawing upon such participation [is the core of ethnographic research].Emerson, Fretz, and Shaw - See more at: http://www.uxmatters.com/mt/archives/2010/06/ethnography-in-ux.php#sthash.s8oqsD2N.dpuf54

Thank you.