Amela Karahasanović

Research Scientist, SINTEF IKT

Adjunct Associate Professor, DESIGN

amela@sintef.no

www.sintef.no

Sintef IKT, Forskningsveien 1

http://heim.ifi.uio.no/~amela/

•User modeling

•Usability and user

experience

•Methods for empirical

evaluation in HCI and SE

•Software maintenance

•Program comprehension

Technology for a better society

•Largest Scandinavian independent research organisation

•More than 2100 employees from 67 countries

•Generated knowledge for 2.8 billions of NOK in 2010

•Eight research divisions

•About 16% of our turnover is derived from international projects

2

HCI research group

– Psychology

– Software Engineering

– Media science

3

Wednesday 19th September 10:15 – 12:00

http://gemini-centre-user-experience.origo.no/

– Bobble experiment (3D technology)

– Living Lab studies

– Collaboration tool

4

Plan for today

Chapter 10 + some examples

• What is usability testing

• Usability testing versus traditional research

• Types of usability testing

• User-based testing

• Questions for discussion and homework

5

What is usability testing?

• Involving representative users, representative tasks, representative environments

– Testing paper prototypes

– Wizard of Oz

– Testing working version before it is released

– Testing working system

– Different devices (smart phones, laptops…)

6

Goal of usability testing

• Improve the quality of an interface by finding flaws in it

• Flaws that cause problems for the majority of people

7

8

Usability versus research?

• Approach similar as one used in classic research

• Different goals

• Usability testing is involved in building a successful product (resources, time, trade-offs)

– Practical, large impact

– Optimize interface after each iteration

• Usability study – Research study - Research on usability methods

9

Usability engineering

Activities aiming to improve the ease of use of an interface

• Expert-based testing (usability inspection)

• Automated testing (usability inspection)

• User-based testing (usability testing)

10

Expert-based testing

• Structured inspections done by interface experts • Before tests with users • Confusing wording, inconsistent layout, obvious flaws • Heuristic review

– Compare interface with the rules

• Consistency inspections – Series of screens or web pages inspected

• Cognitive walkthrough – Experts perform the tasks (high-frequency and

important/seldom)

• Guidelines review – Web Content Accessibility Guidelines

11

Automated usability testing

• Software that compare interface with the guidelines

• Produce report and/or fix the code

• Manual check often needed

• <alt> tag but not if the text is appropriate

• Number of fonts, avg. font size, deepest level of a menu

• RAMP, InFocus

12

User-based testing

– Select representative users

– Select the setting

– Decide what tasks users should perform

– Decide what type of data to collect

– Before the test session (informed consent, etc.)

– During the test session

– Debriefing after the session

13

Questions – Part I

Go to the alarm app on you phone

– Difference between a summative and a formative usability test of this app

– Should we first test it with users or with experts? What do you think? Why?

– What are the differences between a usability test and an experiment with this app?

14

Stages

– Users perform tasks (early or later in the development)

– Formative testing • Low-level fidelity prototypes

• How the user perceives an interface component?

• Low-cost of paper prototypes, users comfortable to criticize

– Summative testing • Evaluate the effectiveness of specific design choices

• High fidelity prototype

– Validation test • Before release, compare to benchmarks

– Formal approach 15

Usability evaluation Talhonia

time

16

Usability evaluation Talhonia – cont.

17

How many users?

– 5 users will find approximately 80% of problems

– 7 for small projects, 15 for medium-large projects

– Goal to find the major flows , that will cause the most problems and must be fixed

– How many users can we afford? How many users can we get? How many users do we have time for?

18

Location

– Lab

– Portable usability lab

– Remote studies (time, place)

19

Tasks

• Clear; no need for further explanation

• Tested

• One clear answer – way to do things

• Tasks that are performed often

• Critical tasks – logging

• No private, financial information

• Must be clear how to go to the next task

20

Oppgave 4: Du er hjemme og du ser NRK Nett-TV på din PC ( http://www.nrk.no/nett-tv ).

Se på episoden 10:12 av Matlyst og svar følgende spørsmål:

• Hva heter kvinnen som er kledd i lilla genser og brunt forkle?

• Når en potet frøs om vinteren, hva ble den da?

21

Measurement

– Task performance

– Time performance

– User satisfaction (validated survey tool)

– Average time to recover from an error

– Time spent using help

– Number of visits to the search feature

– Time spent on specific web page

– Typing speed

– Qualitative data

0

5

10

15

20

25

30

35

40

45

1 3 5 6 7 10 11 12 13 17 18 20 21 23 24 25 26

Unified Library Application - Quit

Unified Library Application

Title Information

ReturnItem

Reserve Title

Lend Item

Insert Title Window

Find Title

Compilation

Subject_id 15 Task 2

Sum of Seconds

Minutes

Visited

22

Starten av dagen: • Sjekke at alt er på plass i laben; gavekort, utstyr, godteri, brus, batterier, pen og papir osv. • Skrive ut eksperimentet material som skal deles til deltagerne • Skrive ut tlf.nr. til de som kommer (mail fra Nordstat) • Informere resepsjonen at vi venter deltakere. Hvis deltaker kommer veldig tidlig, be resepsjonisten

henvise dem til sofa i resepsjonen. • Logge seg på PC og teste alt virker (ingen updates skal dukke opp i løpet av eksperimentet) • Gå inn på facebook og twitter og sjekk at ”thomtor” er logget ut • Heng IKKE FORSTYR på døra • Ha alltid mobilen med deg Eksperimentet (før deltakeren kommer) • Start SINTEF verktøyet (r2d2exp.jar). Det kan ta litt tid – IKKE KLIKK FLERE GANGER! • Start opp camtasia studio (dobbelt klikk) • Start opp uLog (høyre klikk på ikonet i ”system tray” og start) • Eksperiment (hent deltaker) • Ta med deltageren ned; hilse; introduksjon • Konfidensialitet skjema signeres • Husk å sette mobilen på vibrasjon • Start opptak i camtasia studio (høyreklikk, klikk ‘record the screen’, klikk record button) • …

23

24

Making sense of the data

– Write up the results and help influence the design of the specific interface

– Presentation to developers and managers

– Report should • Include all flaws

• Priorities

• For each flaw: describe the problem, present the data, priority, suggest a fix, estimate time/efforts for the fix

– Report structure • How you did usability testing and how you prepared

• What happened during the testing

• The implications and recomendations

25

Questions – Part II Alarm app usability test

• What is important to remind participants before the usability test?

• What is the "think aloud" protocol? Should it be used in summative or formative studies?

• Properties of good tasks?

26

Assignment

Make a PLAN for a summative evaluation of the ‘Contacts’ functionality on your mobile phone in a usability lab with 10 participants.

27