Embed Size (px)
DESCRIPTION
Your management/client is demanding that you do interface testing with actual users but you don’t have the time or money to conduct a full blown usability study. And, here’s another catch: it takes more than a single session in front of the interface for users to truly understand and appreciate – and evaluate -- its features and experience. What do you do? This presentation uses an iPhone app case study to demonstrate tools and tips for conducting an effective online diary study, combined with an online focus group session. We take you through all of the steps of this methodology that we’ve used successfully to test iPhone applications (for companies like Franklin Covey and Deepak Chopra) and other online interfaces that require a deeper user involvement in the interface.
Citation preview
Fast, Cheap and In Control:
Using an Online Diary/Focus Group Methodology to Gauge Meaningful Experiences
David Kozatch and Susanna Kirk
DIG
1. Short on $ budget and time2. Single session with UI not enough3. You want to assess more than just ‘usability”
Q: How?
Is there a “budget” UI test that will gauge meaningful experiences?
Case Study
Who would like to be happier?
No problem
“There’s an app
for that”
Signal Patterns “Live Happy” for the iPhone – a mobile happiness building application
Why is this woman smiling?
Sonja Lyubormirsky
• What makes people happy?• Benefits of happiness?• How can we make people happier?
And, she wrote a book about it
•Savor
•Restore and Cultivate Relationships
•Goal setting
•Be grateful
•Cultivate optimism
Timeline: record your progress
• Personality survey • Happiness and Mood survey• Activity “Fit” Survey
Q: Sounds great -- but how do you put …
?
A: Partner with a company that knows about technology …
The Project
• Assess “proof of concept”
• Assess “usability” • Assess preference of
guided vs. unguided navigation
• Assess price sensitivity/value
• Would a donation help?
Client Objectives:
And, by the way …• Highly specialized
audience
• Requires at least a week of use in order to realize its benefits
• Short time window prior to launch
• Difficult to distribute the app to users
• Limited budget for research
“Can we get a copy of the app and take it for a spin?”
Not quite ready for prime-time
Please stand by….
Photo: Diane Diederich
1. Does the application provide personal significance for users?
2. Is the interface helpful? Respectful? Interested? Forgiving?
3. Does the app take full advantage of the personalized features and interface provided by the iPhone?
Reframe Objectives to include 3 additional questions re: meaning:
Recommendation:
Qualitative assessment
• Sync to iPhone/iPod Touch• Use Live Happy prototype for 7-10 days• Provide detailed feedback via Google Docs
every day during the test period
Focus Group:• Participate in follow-up discussion online via
WebEx and telephone• Include in-depth discussion and instant
polling
Methodology:
Diary:
Why not remote UI study?
• Limited research budget: requires greater time commitment
• Remote testing limited to a few tasks
• Too much focus on “errors” • Need detailed feedback
about experiences
Keep sample small (N=10-12)Use client resources: user or
prospect email lists, etc.Use online social networks (e.g.,
Facebook) and related user pages (e.g., ipod lounge)
Ask qualifying questions relating to specific behaviors/attitudes (e.g., price sensitivity)
Only include those with some interest in overall concept
Recruiting tips
Base $ amount on difficulty of participation – not just audience type
Cash is king: coupons or merch seldom work
Don’t be afraid to overpay to insure participation – you’ve only got one shot
Spread out payments: e.g., 50% after complete diary
entries 50% after complete focus
group
$ Incentive tips
The Diary Study
• UI issues in a “real world” environment
• Users choose their own pace
• “Unfiltered” responses• Adjust questions during
test• Easy, continuous client
involvement
Lose benefits of “direct observation” but gain …
• Flexible and Free • Can reveal a small portion of the diary questions at a time via skip patterns
• Responses automatically populate Google spreadsheet for easy review and sharing
• All responses are dated• Users can upload links to pictures
• Users trust Google
Why we like
Keep questions simple and shortUse mix of closed and open endsGive user the ability to decide
which questions they’d like to answer
Provide final screen text that invites user to go back and answer more q’s
For picture uploads, have respondents use http://imageshack.us Free No registration required Don’t have to share own flickr
photos
Google Docs diary tips
Stay in touch!Use friendly language Praise and cajoleFoster competition, if possibleRemind them that they are getting
paid
More diary tips
Focus Group
Why bother doing a focus group?
• Allows users to share experiences
• Allows client to hear direct feedback and probe deeper
• Adds more depth re: user experiences beyond initial diary entries
• More opportunities to assess emotional language
• Can test additional interface ideas/fixes
• Provides confirmation of hypothesis learned from diaries
… can better assess meaning and user engagement
And …
• Was the (interface) helpful?• Define value of the (interface) in terms of how
it made you feel• Did you get a sense that the (interface) was
listening/responding to your needs as you completed each task?
• How does this (application) fit with how you perform related activities in your life?
Q:
• Relatively low cost• Has video capability• Can switch from PPT to live Web easily if needed
• Private chat feature allows clients to participate silently
• Has polling feature with instant display of results
• Can record session and stream to client or save as .wmv file
Why we like
Length: 1 ½ - 2 hoursKeep text on PPT slides simple
and unclutteredMake use of screen
shots/imagesUse opportunity to follow up on
individual diary postsUse polling sparinglyOpen poll prior to group
discussion to nail down individual responses
Focus Group moderating tips
Users liked the Live Happy prototype …
•Well designed and inviting
•Can choose your own path or follow a sequence
•Takes advantage of personalized iPhone features
•Fast learning curve
Conclusions
But initial prototype didn’t support higher price point desired by the client.
Why?
• Respond to my activities with affirmations/positive comments
• Clearer indications that suggestions are targeted and unique
• Suggestions across various media
• Integrate LiveHappy with an online account for data entry and backup
• Add more content from Sonja's book that is inquiry based
• Track my change for me
Research showed that users wanted a more meaningful experience …
Isn’t that what we all want?
How do we get there?
“Budget” solution
•Speed (“Fast”)
•Value (“Cheap”)
Performance
Contact Information
• David Kozatch, Principal
Ph 646.416.6784
• Susanna Kirk, Senior Analyst
Ph 617.475.5105
Web: www.digsmarter.com
Blog: blog.digsmarter.com
