CSCD 487/587Human Computer Interface

Winter 2013

Lecture 14Observation and Ethnography

Outline There are benefits and challenges of

different types of observations How to observe as an onlooker, a

participant, and as an ethnographer

Observation can be Direct

– You observe what users do or

Indirect– You use diary studies and

logging

What and When to Observe

Observation is usable at all stages during product development

Goals and questions determine the paradigms and techniques used

Observers can be– Onlookers– Participants– Ethnographers

Degree of immersion that evaluators adopt varies across a broad outsider-insider spectrum

Types of Observation

ObservationObservationControlled Controlled

Environment Environment (e.g. Lab)(e.g. Lab)

Field Field Environment Environment (e.g. Natural)(e.g. Natural)

Outsider looking on

““Quick and Dirty” Quick and Dirty” oror

In usability In usability testingtesting

““Quick and Dirty” Quick and Dirty” or or

In field testingIn field testing

InsiderInsider N.AN.A..

Participant Participant observation (e.g. observation (e.g. in ethnography)in ethnography)

Approaches to Observation

“Quick and Dirty” Observation– It can occur anywhere, anytime– Least formality involved

Observation in Usability Testing– Video and interaction logs capture all user

operations– One-way mirrors or remote TV screen– Data is used to analyze what users are doing and

provide insight into users’ reactions– Can augment other techniques– Users can't always explain what they do

• Thus, limits of surveys and interviews !!!– Direct observation helps fill in details

Approaches to Observation

Observation in Field Studies– Observer may be

• Passive observer,• Participant observer or• Ethnographer

– Goal is to cause as little disruption as possible

Approaches to Observation

Passive or Outside Observer Example: An observer in class of boys and

girls whose primary job to keep track of how much time a particular technology is used by each gender

Participant Observer Participates in social conventions of a group,

combines participation with maintenance of professional distance for unbiased observation

Ethnography Becomes one of the group, takes weeks,

months or longer Inside information of community activity is

obtained

How to ObserveIn Controlled Environment

Practical Issues Need to Plan Advance– Decide where users are located so that equipment

can be set up in advance– Decide about modes of data capture e.g. video,

interaction logs– Equipment testing, audio and video– Get legal user consent– Prepare a script to guide users through set of

questions – It is important to make users feel comfortable– Problem with this approach

• Observers do not know what users are thinking

Controlled Environment Think-Aloud Technique

User is trying to perform certain task Observer wonders, what is going on, what is he thinking, why did he do that, Externalize thought process

Controlled Environment Think Aloud Technique

Checklist of things to Plan Before Going into Field

Select a framework to guide activities in field Decide how to record events Think about how to gain acceptance and trust of users under observation Be prepared to refine and refocus study for same or new user groups Think about how to handle sensitive issues Consider checking notes with a participant and plan to look at situation from a different perspective

Frameworks for In the field

There are Frameworks to guide observation, provides structureSimple– The person. Who? – The place. Where?– The thing. What?

The Goetz and LeCompte (1984) framework:– Who is present? – What is happening? – When does the activity occur?– Where is it happening? – Why is it happening? – How is the activity organized?

Frameworks forIn the field

The Robinson (1993) framework– Space. What is the physical space like?

– Actors. Who is involved?

– Activities. What are they doing?

– Objects. What objects are present?

– Acts. What are individuals doing?

– Events. What kind of event is it?

– Goals. What do they to accomplish?

– Feelings. What is the mood of the group and of individuals?

Data Collection Techniques

Notes plus still camera– The least technical way of collecting data– Transcription the first step in data analysis– Photographs, sketches, etc. can be easily collected

Audio recording plus still camera +– Less intrusive than video – More flexible and mobile – One drawback is transcribing the data which can

be onerous if many hours have to be transcribed

Data Collection Techniques

Video– Advantage of both audio and

video data, but more intrusive– Attention becomes focused on

what is seen through lens – Analysis of video can be long

and time consuming– Sound may get muffled when

recording in noisy conditions

Comparison

Table source: Preece, Rogers, Sharp: Interaction Design: Beyond Human-Computer Interaction. Wiley

Comparison (contd.)

Table source: Preece, Rogers, Sharp: Interaction Design: Beyond Human-Computer Interaction. Wiley

Indirect Observation

Diaries Diaries provide record of What users did, When they did it and What they thought about their interactions with technology Useful when users are scattered and unreachable Inexpensive and requires no special equipment or

expertise They rely on participants being reliable and

remembering to complete them

Indirect Observation

Interaction Logging - Includes analysing data from key presses, mouse movements etc.

- Usually synchronized with video and audio logs - Time stamped to calculate how much time a user

spends on a particular task

E.g. Explicit counters that record visits to a website

If you want to find out if adding a bulletin board increases number of visits, comparison of traffic before and after is useful

- Unobtrusive and large volumes of data can be collected

Ethnography

Ethnography’s is notion of participant observation

– Idea that you learn about other people's cultural practices by

– Going there, being there, and by doing it with them

– Most traditional anthropologists who would consider themselves to be ethnographers have spent years living in other cultures with people, and not just watching what they do, but actually doing it too

21

Ethnography

Ethnography

• Ethnography is an analytic perspective that is associated with doing interactive fieldwork- Term was coined or invented by anthropologist

Bronislaw Malinowski circa 1922 to describe his approach towards understanding other people’s cultures through fieldwork

- Term “ethnography” is often used to describe fieldwork in systems design

Design Scenario

Task for our Ethnographic study:– Design an email system for the 21st Century

networked home– We need to first look at how mail is currently

delivered in the home

An exampleHandling mail in the home

- Letterbox, pigeon hole, porch, front door

- In other words a known in common delivery / collection point … … …

- Where does mail arrive?

Known in common collection point

Contingent • But known in common by

household members

What happens next?

Mail arrives then what?

- Someone collects it. Who?

- But can anybody open it? No

- So it gets put some place where members can see that new mail has arrived

- A known in common place where members can sort through it to see if any mail has arrived for them

Known in common sorting point

• Just about anyone (even the dog :-)

26www.id-book.com

Who is it for?Specifically, how do we determine who is entitled to open particular mail items?- Name and ….

- Logos, handwriting, postmarks, senders’ address, etc.

• All articulate entitlement to open (not necessarily name)

Sorting the mail: members can see at-a-glance who it is for

- Visible features of envelopes enable members to determine at-a-glance who an item is for and what to do with it• junk it, leave it, open it

What if it is for someone else?What happens then? Do we leave the mail at the sorting point?

- Members usually put mail for others in places that are relevant to them

Relevant recipient point

• Places the recipient usually sits

– At the kitchen table, on their favourite chair, on the arm of the sofa, or nest of tables next to it, etc.

• Or places where the recipient cannot miss it (e.g., outside the bedroom door of your teenage son)• Mail items that are not for but may be of interest to other members are also placed in this way (postcards, letters, leaflets, etc.)

What do we do with it?

Lots of mail requires that we do something with it (e.g., pay a bill). Members place mail to reflect its action status.

- Mail requiring immediate action is placed such that members can see that at-a-glance

• By the mobile phone charger, or by the car keys, or in porch next to the front door, etc., if it’s has to be taken out of the home and dealt with

• Or by a computer if its to be dealt with online

Immediate action point

What if a response isn’t needed?

Not all mail requiring action requires that we respond to it

– e.g., bank statements, insurance certificates, mortgage payment records, etc.

- We may still need to take action on it though (e.g., checking and archiving).

- Mail where action may need to be taken is placed to show that action is pending.

Pending action point

30www.id-book.com

What if a response will wait?

Other mail items require action but not until a later date

– e.g., appointments, invitations, promotional offers, etc.

- They are placed to display their relevance to up and coming events

- e.g., on notice boards next to calendars, shopping lists, takeaway menus, etc.

Pending event points

31www.id-book.com

What if it’s Special?

Not all of the mail we receive is utilitarian bills, bank statements, appointments,We also receive “special” mail items: birthday cards, thank you cards, congratulations, etc.

- Special items are placed to display their essentially social character

• On window sills, mantelpieces, cupboards, dressers, etc.

Social display points

Naturally Accountable Organization

The organization of mail handling in the home, observable and reportable by members1. Collecting mail from a known in common delivery / collection point

2. Placing mail at a known in common sorting point

3. Sorting mail based on household member’s entitlement to do so, which is read off the visible

features of envelopes and not just the name

4. Placing sorted mail items in places relevant to their recipients

5. Placing mail to indicate its action status

Immediate action

Pending action

Pending event

6. Placing mail to display its special social status

33www.id-book.com

Design scenario revisited

What does the naturally accountable organization of mail handling suggest about the design of email for the 21st Century networked home?

– How might the field study “inform” design?• What are the main organizational features or “findings” of the

study?

• Thought experiment part two: who factored the distribution of email across, and its display at, different locations into their design sketches?- Can you see the value of looking at relevant activities in the real

world?

- Mail is distributed around the home- Mail is distributed through its physical placement and display- The display of mail performs various functions:

• At-a-glance it announces that new mail has arrived, that mail has arrived for you, that immediate action needs taking, and so on

Summary

Many techniques for collecting user data

Observation, survey, questionnaires, think aloud and ethnographic studies

Need to capture user's interactions with the object of study

Time, expense will certainly affect choice of method

And, ultimately what you need to learn from the users

35

End

Usability testing with Earth based interface

See Assignments page for short, fun assignment