Observations from behind the scenes

A presentation to NUX Monday 7 April 2014

2004 First PC based qualitative user-tests recruited and accommodated in studio and off-site on behalf of comparison site clients and digital agencies

2014 Recruiting participants and accommodating qualitative user -testing on mobile, PC and tablet for clients, digital agencies and research companies

A man with a hammer

The User – Testing Landscape •  Increasingly different environments

–  In-home –  At Work –  Virtually In-home –  Virtually At Work –  On-street –  On-site –  Scientific research spaces –  Market research spaces

–  It’s Horses for Courses…

Where, when, why? Place + - In Home or In Workplace

• Provides more qualitative insight into a user’s environment • Immediate tech support for user from Moderator • Some respondents are more relaxed in-home/at work • Quick to set up and get into the field – can be recruited in as little as 3 days • Cheaper to incentivise respondents in their own environment • No venue hire cost

• Enables meetings with hard to reach users

• Less control of the environment – interruptions from colleagues, dogs, children etc • Some respondents feel less relaxed being observed in home or at work • Time consuming– travel time limits numbers per day, interruptions may extend interview time •  Minimal live observation by stakeholders requires greater time for sharing outcomes and getting buy-in •  Higher moderator travel costs

On Street

• Short lead time – hire fieldworkers to recruit on-street by the day • Reduces margin of error re. user and their own technical issues • Consistent equipment • Quick conversations • Reduced incentive cost • Suitable for projects with a wide target audience • Moderator focus on interview rather than respondent management

• Requires high-footfall sites e.g city centres, shopping malls • Requires amenable management and convenient locations and free reliable wifi • More targetted respondents may be less available • Unsuitable for longer conversations • Minimal live observation by stakeholders requires greater time for sharing outcomes and getting buy-in

Where, when, why? Place + -

On Site

• Short lead times – no venue hire required • Technical support in place • Low Cost – no venue hire, no equipment hire, no respondent host, catering optional • Proximity to testing and moderation by agency personnel encourages ownership of outcomes

• Objectivity is compromised – respondents and agency/client are not on neutral territory • Facilities may inhibit process e.g no parking, waiting areas, hospitality services • Minimal live observation by all stakeholders may require greater time for sharing outcomes and getting buy-in

• User Testing Space

• Neutral territory . Facilities designed to optimise process •  Meeting room style observation rooms big enough for Agile workshops etc. Viewed via mirror • Direct involvement of all stakeholders shortens implementation time, encourages joint ownership and may require less documentation of outcomes • Involvement of all stakeholders strengthens relationships encouraging better communication/understanding within teams • Adaptable spaces recreate home or office settings to put respondents at ease as far as possible •  If required, consistent and accurately specified multi platform test equipment available

• Venue hire cost is higher relative to other options • Longer lead time to coordinate diaries • Perceived as ‘un-natural’

Where, when, why Low Cost

Tighter Timescales

Low Observer Involvement

High Observer Involvement

Shorter Interview

Longer Interview

Hard to find users

Natural Habitat

Neutral Territory

In Home

On Street

On Site

User Testing space

Checklist 1. Basics

Basic Materials Discussion guide? Stimulus Material? Hard and soft copies? Incentives? Respondent details and contacts?

Your Own Equipment Laptop/Mobile/tablet/smartTV Chargers, adaptors, digital audio recorder back up, batteries, memory stick?

The Venue’s Equipment Monitors, cables, mice, keyboards, splitters, mobile capture, laptop, mics, web cam. Technical support? Do they provide SmartTV/ laptop/mobile/tablet? Room layout?

Capturing the Interview Are you confident with the software? Set up folders? Installed lupdates?

Checklist 2. Venue

Venue Specification How long do you need to set up? Will there be any technical support? Who will look after the respondents and clients? Is it easy to find? Is there any parking? How big is the observation/respondent space? Is stationery provided? What catering is covered in the basic price? Do you need to leave at a certain time? Is there a secure place to leave equipment overnight? How private is the work space? How flexible is the respondent space – office and/or lounge set up? How quickly are recordings available, in what format?

Venue Technical Specification (If using their equipment) do you need a specific search engine, operating system, resolution, sound card, aspect ratio, router, wireless or wired broadband?

Streaming to Remote Viewers Streaming account details – Team Viewer? Wireless capacity?

Checklist 3. People

Participants Participant profiles including names, times, incentive, mobile numbers/address? Who will meet and greet? Participant incentives and signing sheets - correct denominations in cash? Non-disclosure documents, do you need them? Do you need a spare recruit if there are any no-shows? Are you insured for lone workers in-home? Petty cash for coffees?

Observers Observers invited? Number of observers, catering required? Special dietary requirements? Schedule of attendance? Directions? Respondent profiles less contact details? Do they need to sign NDA?

Interviewer How far will you travel between each interview? How much time needs to be left between each one? Should you have a driver? Where will you eat?

Take the prospect of off site user testing from this…

www.thetalkingshop.co.uk @talkingshopMCR ktw@thetalkingshop.co.uk … to this