The Aphasia The Aphasia ProjectProjectDesigning Technology For and Designing Technology For and

With People who have AphasiaWith People who have Aphasia

Maria Klawe, Maria Klawe, Sonya Nikolova, Jordan Boyd-Sonya Nikolova, Jordan Boyd-GraberGraber

Princeton UniversityPrinceton University

Marilyn Tremaine, Rutgers UniversityMarilyn Tremaine, Rutgers University

Joanna McGrenere, Joanna McGrenere, Peter Graf, Barbara Purves,Peter Graf, Barbara Purves,Karyn Moffatt, Meghan Allen Karyn Moffatt, Meghan Allen University of British ColumbiaUniversity of British Columbia

OutlineOutline

What is Aphasia?What is Aphasia? Problems with existing solutionsProblems with existing solutions The evolution of ESI plannerThe evolution of ESI planner Broader impactsBroader impacts

Aphasia is a loss of words – not intelligence

Acquired language Acquired language disorderdisorder Caused by brain Caused by brain

damage (e.g. damage (e.g. stroke, trauma, etc.)stroke, trauma, etc.)

Impairment of Impairment of communication communication abilitiesabilities

Relative sparing of Relative sparing of other cognitive other cognitive abilitiesabilities

What is Aphasia?What is Aphasia?

Impact of AphasiaImpact of Aphasia

More common than Parkinson’s More common than Parkinson’s diseasedisease

Withdrawal from societyWithdrawal from society

Problems with existing Problems with existing solutionssolutions

Limited to communication; lack Limited to communication; lack of higher level applicationsof higher level applications

Communication too slow for Communication too slow for most aphasicsmost aphasics

Stigma of relying on laptops in Stigma of relying on laptops in publicpublic

Poor user-interface of Poor user-interface of handheldshandhelds Buttons too smallButtons too small Confusing navigationConfusing navigation Poor organization of multi-media Poor organization of multi-media

datadata

The Aphasia ProjectThe Aphasia Project

Goal: understand how individuals with Goal: understand how individuals with aphasia communicate and seek aphasia communicate and seek opportunities where technology can better opportunities where technology can better support individuals in their daily lifesupport individuals in their daily life

Interdisciplinary project: computer Interdisciplinary project: computer science, psychology, speech language science, psychology, speech language pathologypathology

Multi-site: UBC, Princeton, aphasia Multi-site: UBC, Princeton, aphasia centers, centers,

Types of SubprojectsTypes of Subprojects

I: Participatory design of application I: Participatory design of application prototypesprototypes

Daily plannerDaily planner HandheldHandheld Combined handheld and laptop systemCombined handheld and laptop system

Recipe book (laptop or tablet)Recipe book (laptop or tablet) File system (handheld)File system (handheld)

II: Evaluation of commercial PDAsII: Evaluation of commercial PDAs Long term use by SkipLong term use by Skip Short term use of camera functionShort term use of camera function

Special challenges with Special challenges with research involving aphasicsresearch involving aphasics Access to participantsAccess to participants Communication with participantsCommunication with participants Right hemiparesisRight hemiparesis Lack of experience with computers, Lack of experience with computers,

PDAPDA Uncertain future of PDAsUncertain future of PDAs

I: Participatory Design of I: Participatory Design of High-Level Application High-Level Application

PrototypesPrototypes

Anita Borg, Anita Borg, 1949 - 20031949 - 2003

• daily planner • interactive cookbook

Karyn Karyn MoffattMoffatt

ESI Planner 1ESI Planner 1The Enhanced with Sound and Images Planner

PDA application (iPaq from HP)PDA application (iPaq from HP) A computerized daily planner A computerized daily planner

designed for people with designed for people with aphasia aphasia

Uses images, sound, and text Uses images, sound, and text to represent people and places to represent people and places in appointmentsin appointments

I: I: Images, Sound, Text…Images, Sound, Text…

Say you have an appointment with Queen Say you have an appointment with Queen Elizabeth, at the Eiffel Tower on November Elizabeth, at the Eiffel Tower on November 6, 2003 from 8:00am to 10:00am6, 2003 from 8:00am to 10:00am

ESI Planner would display it like this:ESI Planner would display it like this:

I: I: Participatory DesignParticipatory Design

Four phasesFour phases

I: I: Participatory DesignParticipatory Design

Four phasesFour phases Idea Idea

brainstormingbrainstorming InterviewsInterviews Identified needs: Identified needs:

Daily plannerDaily planner Recipe bookRecipe book

I: I: Participatory DesignParticipatory Design

Four phasesFour phases Idea brainstormingIdea brainstorming Paper low-fi Paper low-fi

prototypesprototypes

1 - Drawn by hand

2 - Using computer tools

6

7

8

9

101112

1

2

3

4

5

6

7

8

June: 9/6/03

7:00 am - 9:30 am

Marilyn Monroe Eiffel Tower

11 : 1510

20

1: 00

05

Starts

Ends

I: I: Participatory DesignParticipatory Design

Four phasesFour phases Idea Idea

brainstormingbrainstorming Paper low-fi Paper low-fi

prototypesprototypes Medium-fi Medium-fi

prototypesprototypes

I: I: Participatory DesignParticipatory Design

Four phasesFour phases Idea brainstormingIdea brainstorming Paper low-fi prototypesPaper low-fi prototypes Medium-fi prototypesMedium-fi prototypes High-fi prototypes and formal High-fi prototypes and formal

evaluation in labevaluation in lab

I: High-fi PrototypesI: High-fi Prototypes

ESI Planner

NESI Planner

I: Formal EvaluationI: Formal Evaluation

8 participants 8 participants Session 1:Session 1:

30 minutes with each Planner (ESI and 30 minutes with each Planner (ESI and NESI)NESI)

10 tasks (retrieval, creation, 10 tasks (retrieval, creation, modification)modification)

# of tasks completed# of tasks completed # of tasks completed # of tasks completed correctlycorrectly

Session 2: Western Aphasia BatterySession 2: Western Aphasia Battery

I: ResultsI: Results

Preference: 5 ESI, 3 NESI.Preference: 5 ESI, 3 NESI.

Total Tasks Correct by Participant

0

2

4

6

8

10

12

SR CW CB WV JP GP NF MM

Participant

To

tal T

as

ks

Co

rre

ct ESI NESI Significantly Significantly

more tasks more tasks completed completed correctly with correctly with ESI Planner.ESI Planner.

Limitations of ESI Limitations of ESI Planner 1Planner 1

Need for field studyNeed for field study Interface for input and modification Interface for input and modification

of appointmentsof appointments Lack of stable memory on iPAQ after Lack of stable memory on iPAQ after

power losspower loss Lack of library of icons and soundsLack of library of icons and sounds

ESI Planner 2ESI Planner 2

Input on laptopInput on laptop Lingraphica library of icons and Lingraphica library of icons and

soundssounds Stable memoryStable memory Built-in camera in iPAQBuilt-in camera in iPAQ Participatory design with speech Participatory design with speech

language pathologists (SLPs)language pathologists (SLPs) Four week field study Four week field study

LingraphicaLingraphica

Current Lingraphica systemCurrent Lingraphica system Laptop-based (easier interface)Laptop-based (easier interface)

Used at home for practiceUsed at home for practice

IssuesIssues Expensive, restricted to AAC insurance Expensive, restricted to AAC insurance

diagnosisdiagnosis

StigmaStigma

A Lingraphica phraseA Lingraphica phrase

Initial goal for prototypeInitial goal for prototype

Download appointments and phrases Download appointments and phrases from Lingraphica to ESI planner on from Lingraphica to ESI planner on handheldhandheld

Anticipated usesAnticipated uses Visits to doctors, etc.Visits to doctors, etc. ShoppingShopping Medication Medication

ESI Planner 2 homeESI Planner 2 home

appointmentsappointments

phrasephrase

The need for LgLiteThe need for LgLite

Lingraphica (Lg) too complexLingraphica (Lg) too complex Inability to add photos in Lg Inability to add photos in Lg No Windows version of LgNo Windows version of Lg

LgLite interfaceLgLite interface

LgLite interfaceLgLite interface

The Pilot Field StudyThe Pilot Field Study

Seven aphasics at Adler Aphasia Seven aphasics at Adler Aphasia CenterCenter

Four weeks, one videotaped meeting Four weeks, one videotaped meeting with each participant per weekwith each participant per week

Log of interactions with iPAQ and Log of interactions with iPAQ and computercomputer

OutcomesOutcomes

Little use of appointmentsLittle use of appointments Input only possible at Adler CenterInput only possible at Adler Center Desire for input on iPAQDesire for input on iPAQ

Extensive use of cameraExtensive use of camera Desire for better photo management on Desire for better photo management on

iPAQiPAQ Problems with sound playbackProblems with sound playback Range of text and speech abilitiesRange of text and speech abilities Enthusiasm for continued use of iPAQsEnthusiasm for continued use of iPAQs

Next stepsNext steps

Add input on iPAQAdd input on iPAQ Improve photo management on iPAQImprove photo management on iPAQ Improve sound playback Improve sound playback Improve LgLite interfaceImprove LgLite interface Another field study at AdlerAnother field study at Adler

Broader impactBroader impact

Dual usefulness between aphasics Dual usefulness between aphasics and elderly (poor vision, memory and elderly (poor vision, memory loss, Alzheimers, etc.)loss, Alzheimers, etc.)

Other visual language applicationsOther visual language applications

For More Info…For More Info…

http://www.cs.ubc.ca/projects/http://www.cs.ubc.ca/projects/Aphasia/Aphasia/