Upload
hortense-fox
View
214
Download
0
Embed Size (px)
Citation preview
Interviews & Focus Groups
KSE966/986 Seminar - Fall 2012/Spring 2013 May 10, 2013Uichin Lee
Contents
1. Pros and cons of interviews2. Applications of interview research3. Who to interview4. Interview strategies5. Interviews vs focus groups6. Types of questions7. Conducting an interview8. Electronically mediated interviews 9. Analyzing interview data
What is interview?Gathering direct feedback from
interested individualsSurvey: reaching out a large number
of people, but often restrictive ◦Open-ended questions receive short (or
no) answersAlternative: direct discussions with
concerned participations◦ Interviews with individuals◦Focus groups involving multiple users at
one time
1. Pros and cons of interviews
Pros◦Go deep◦Flexible
Cons◦Skill to manage (unbounded discussion?
Setting up meetings, listening/taking notes)
◦Time and resource intensive (both interviewing and analyzing content)
◦Recall problems (similar to surveys)
2. Applications of interview researchInitial exploration
Requirements elicitation
Evaluation and Subjective Reactions
2. Applications of interview researchInitial exploration
Starting from clean slate, try to understand users’ goals/needs, to probe new possibilities, etc. by asking broader questions
Requirements elicitation
Evaluation and Subjective Reactions
Breaking the Disposable Technology Paradigm: Opportunities for Sustainable Interaction Design for Mobile Phones (CHI 2008)Goals:
◦ How people understand the lifespan of their phones
◦ What factors, such as style, service contracts, and functionality, affect how they attribute value to their phones, and their awareness and actions regarding mobile phone sustainability
Methods: ◦ a 34-question web survey (n=79) ◦ a series of in-depth, semi-structured phone
interviews with participants selected based on survey responses (who suggested an interesting history or experience) (n=10)
How do people organize their desks? Implications for the design of office information systems (ACM TOIS 1983)
Goals:◦ In the early 1980s, desktop info sys is relatively new; understanding how
people organize info was incomplete◦ Goal is to understand how people organize info
Methods: Interviewed 10 workers in their offices
◦ Workers are asked to describe the layout of the office (indicating where info is stored and why) Only clarification questions are asked Some are asked to find some documents..
◦ Also asked about information practices: How well organize (Likert scale) Biggest problems? Do you keep lists of things to do? Do you keep a calendar of appointments? How often are you unable to do…
Results: files (structured) vs. piles (unorganized, but also acting as reminders)
2. Applications of interview researchInitial exploration
Requirements elicitation◦Try to understand needs/concerns/goals
during the early stage of design
Evaluation and subjective reactions◦Try to get summative evaluations of
completed products (e.g., usability surveys) (say for planning design revision)
3. Who to interview Beyond users – Stakeholders
◦ Anyone who may be affected by the use of a system◦ There could be multiple types of users (or related people)
E.g., hospital info systems; patients and their family members
Interview representatives of different groups from different perspectives
Key informants who are knowledgeable about how work is done◦ Particularly useful and informative individuals◦ Must be selected carefully to avoid bias
When selecting, respect the participants (e.g., reaching out homeless people on the street or people with disabilities)
4. Interview strategies:Types of Interviews
Fully Structured
Semi-Structured
Unstructured
Less structure: greater difficulty in conducting and interpreting interview
But
More opportunity for insight
4. Interview strategies:Types of InterviewsFully structured – Orally-presented survey
◦Stick with the script.◦Good for comparison across individuals
Semi-structured – pre-specified questions serve as starting point for discussion◦Digression is okay
Unstructured – initial question, possible list of topics, but no pre-defined script ◦Less structure good for open ended
exploration
4. Interview strategies:Focused & Contextual InterviewsGo beyond asking questionsAsk participant to
◦ Demonstrate use of technology (instead of explaining how to do)
◦ Rely on artifacts (papers, photos, etc.) and context ◦ Use external aids to elicit feedback or reaction (known
as “probes” e.g., props or prototypes)
Example:◦ Asking a user to show the office and tell how to store
things◦ Asking a user to show how tech is used at home in
detail◦ Asking to use a functional prototype and complete
some tasks
5. Interviews vs focus groups
Interviews take time◦1 hour or more/response◦Several hours for analyzing notes
Focus groups◦More people in less time ◦Up to 8-12 people at once.
5. Interviews vs focus groupsFocus Groups: Pros & Cons
Pros◦Broad range of viewpoints and insights◦Each group will likely have at least one
person who will stimulate others to talk
Cons◦Hard to manage group dynamics◦Generally can't be fully structured◦May need to ask fewer questions◦Selection can be challenging
6. Types of questions Closed-ended Questions
◦ “On a scale of 1-10, 10 being best, how did you like the web page?”
◦ Easy to analyze, but may not be informative
Yes/no questions◦ Remember, the goal is to get interviewees to give you useful
insight◦ Simple questions get simple answers◦ “Did you like the home page?” You'll get a one-word answer
Open-ended questions◦ Invite elaboration, discussion: e.g., What did you think about the
web page?”◦ Ask users to complete a sentence: “My favorite web browser
feature is...”◦ Conceptual mapping: draw pictures or layouts to describe
understanding of a situation or problem
6. Types of questionsSimple questions – no jargon
Avoid compound questions with multiple parts◦Not "“What were the strengths and weaknesses
of the menu layout and the toolbar?”◦Ask two separate questions instead
Avoid judgmental phrasing or tone◦Not “Don’t you think this is hard to use?”◦Possible bias, instead ask “What did you think?”
6. Types of questionsQuestions in un- or semi-
structured interviews◦Often, questions are generated in
response to participant comments◦Can be hard to do this well◦Start with structured interviews
Get a few under your belt before moving to unstructured
7. Conducting an interview
PreparationRecording responsesDuring the interview
◦Rapport◦Introduction◦Getting down to business◦Promoting discussion◦Debriefing
7. Conducting an interviewPreparationPilot test – w/ colleagues and
participants◦Good for logistics and for confidence
Write an interview guide listing what to do and when◦Particularly good if multiple
researchers are involvedLogistical backups
◦Batteries for laptop, audio recorder, extra paper, etc.
7. Conducting an interviewRecording responses: notes
• Audio and video recordings are fine, but
• Paper notes are still important– Record insights, non-verbal responses,
etc.– Try to record what you can, but– Don't do so at the expense of listening
• Summarize written notes as soon as possible after the interview
before you forget...
7. Conducting an interviewRecording responses: audio/videos
• Complete, but expensive (transcribing and interpreting responses)
• Video is tricky, but gets useful information
• Consider audio + still pictures• Respect privacy and anonymity
7. Conducting an interviewDuring the Interview
• You're the Host: Build Rapport• First few minutes are crucial• Be professional, friendly, respectful, nonjudgmental
• If so, interviewees are inclined to trust you and provide honest and useful feedback
• Listen carefully• Providing refreshments is a good idea (e.g., water, snacks)
• Avoid loud, crunchy food though
• Outline• Briefly introduce research goals (use script if possible to
be consistent)• Tell that you’re recording the session and how• Complete paperwork (informed consent)• Simple questions first, hard questions later
7. Conducting an interviewDuring the Interview
• Be flexible• If your interview is not fully
structured• But, try to keep things on track• Explain why you are asking each
question• Define terms, avoid jargon• Ask for clarification
7. Conducting an interviewDuring the Interview
Read between the lines…◦Is the interviewee telling you what
they think you want to hear? (hoping to please researchers)
◦If so, make a note of it◦Might want to downplay in
interpretation
7. Conducting an interviewDuring the Interview
Challenges of focus groups◦Manage the room. Be prepared to
deal with Digressions Arguments
◦Give everyone a chance to talk Address them directly “Joan, what do you think about...?”
7. Conducting an interviewDuring the Interview
Promoting discussion◦What if they won't talk?◦Fully-structured – not much to do◦Otherwise
Rephrase questions Dig deeper into specifics
◦Use props and probes to stimulate feedback
◦Focus groups – ask for dissenting or concurring feedback
7. Conducting an interviewClosing it out: DebriefingAsk for any final comments
◦ Interviewees might make additional useful comments
Provide more details about research goals
Brief summary of findingsSay “thanks”!After thanking the users, spend some
time to reflect and summarize notes immediately
Consider turning off recording devices
8. Electronically mediated interviews Phone, web chat, email,
conference calls◦Pros
Easy, inexpensive Reach out more people with less effort
◦Cons Lack of face-to-face contact Fewer non-verbal cues Pacing can be harder
9. Analyzing interview dataDo it as soon as possibleAvoid “cherry-picking”Fully-structured, closed-ended: tabulate
answersOpen-ended questions require coding
(audio transcribe data, written notes)◦Organizing responses to various categories
e.g., affinity diagrams: hierarchical groupings of structures and themes, built from the bottom up
Critical incidence analysis: if interviews are used for examining stories (e.g., a notable failure of an excising system)
Summary
1. Pros and cons of interviews2. Applications of interview research3. Who to interview4. Interview strategies5. Interviews vs focus groups6. Types of questions7. Conducting an interview8. Electronically mediated interviews 9. Analyzing interview data