The launch of usability testing at the Cheng Library:

What I learned

Ray Schwartz: schwartzr2@wpunj.eduWilliam Paterson University

VALE Users' ACRL-NJ/NJLA CUS ConferenceFriday, January 6, 2017

Outline

1. What is Usability Testing?2. Why Usability Testing?3. How is it done?4. Our ‘Foray’ into Usability Testing.5. Test Preparation.6. Results.7. Response from the Library.

2

What is usability testing?

• “Usability testing is a way to see how easy to use something is by testing it with real users.”

• "Users are asked to complete tasks, typically while they are being observed by a researcher, to see where they encounter problems and experience confusion. If more people encounter similar problems, recommendations will be made to overcome these usability issues. “

What is usability testing? (2016). Retrieved from http://www.experienceux.co.uk/faqs/what-is-usability-testing/

3

• “Watching people try to use what you’re creating/designing/building (or something you’ve already created/designed/built), with the intention of (a) making it easier for people to use or (b) proving that it is easy to use.”

• “This element of actual use is what makes usability testing very different from things like surveys, interviews, and focus groups where you’re asking people for their opinions about things or their past experiences using things.”

Krug, Steve (2010). Rocket surgery made easy. Berkeley, CA: New Riders, p. 13.4

Usability testing is not a quantitative study

• Usability testing is conducted solely for the purposes of improving library products and services. It also does not need to be submitted to your University’s IRB for review. An IRB only reviews research as defined by federal regulations as “a systematic investigation, including research development, testing and evaluation, designed to develop or contribute to generalizable knowledge.” (45 CFR 46.102(d)).

• With usability testing, the people involved are usually called “test participants”, not “test subjects” to remind ourselves that we are not testing them, but the services they are they using (Krug 2010, p. 13).

5

Why usability testing?

• Usability testing brings users into your design process.

• This means that what you make remains relevant to the people who will have to use it.

• Uncovers problems while they’re still cheap to fix.

6

How is it done?

• In this case we are talking about DIY usability testing. This is not the “full blown” usability testing that you carefully select your participants and take 1 to 2 days testing, with a week to prepare a briefing and then followed by deciding what to fix.

Krug, Steve (2010). Rocket surgery made easy. Berkeley, CA: New Riders, p. 14.7

Steve Krug’s version of DIY

A morning a month.• Which includes testing, debriefing and deciding what to fix.

Start earlier than you think would make sense.• This means to start testing before you even changed anything.• Start testing other sites. Even the best sites have usability problems.

Recruit loosely and grade on a curve.• Doing testing frequently is more important than carefully targeting your

audience.

8

Make it a spectator sport.• As many people should observe from another room (e.g., via video) as

possible. Seeing is believing.

Focus ruthlessly on a small number of the most important problems.• Intensely focus on fixing the most serious problems first. Pick the three

worst problems.

When fixing problems, always do the least you can do.• Tweak, don’t redesign.• Take something away.

9

Basic TenetsMatthew Reidsma (Web Services Librarian)

from Grand Valley State University.

• Do one test every month.• Focus on what users do, not what we’d like them to do.• No more than 5 questions for a 30 minute test.• Test no more than 3 students or faculty a month.• Invite everyone from the library to observe. EVERYONE.

Matthew Reidsma : How we do usability testing. (2011). Retrieved September 16, 2016, from https://matthew.reidsrow.com/articles/13

10

• First, the presentation to Faculty Librarians’ Meeting to inform and build support.

• Our modified tenets• One a month – or when a question arises or as part of a project.• Two to six questions for a 15 minute test.• Test no more than three to five stakeholders a month.• Use screen/voice recording software for librarians to observe later.

• Next, test preparation.

Our ‘Foray’ into Usability Testing

11

Test Preparation

• Equipment - Windows laptop and SnagIt software for screen and voice recording.

• Location – Food Court of the Student Center.

• Opening script and release form.

• Prepared tasks.

12

Examples from the script

“The first thing I want to make clear right away is that we’re testing the site, not you. You can’t do anything wrong here. In fact, this is probably the one place today where you don’t have to worry about making mistakes.”

Krug, Steve (2010). Rocket surgery made easy. Berkeley, CA: New Riders, p. 147-148.13

“As you use the site, I’m going to ask you as much as possible to try to think out loud: to say what you’re looking at, what you’re trying to do, and what you’re thinking. This will be a big help to us.”

“If you have any questions as we go along, just ask them. I may not be able to answer them right away, since we’re interested in how people do when they don’t have someone sitting next to them to help.”

Krug, Steve (2010). Rocket surgery made easy. Berkeley, CA: New Riders, p. 147-148.14

Recording consent form

Thank you for participating in our usability research.We will be recording your session to allow [ORGANIZATION NAME] staff members who are unable to be here today to observe your session and benefit from your comments.Please read the statement below and sign where indicated.- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

I understand that my usability test session will be recorded. I grant [ORGANIZATION NAME] permission to use this recording for internal use only, for the purpose of improving the designs being tested.

Signature: _______________________________Print your name: __________________________Date: __________________

Krug, Steve (2010). Rocket surgery made easy. Berkeley, CA: New Riders, p. 153.15

Creating Tasks

• Figure out what you’re testing and who the stakeholders are

• Set goals and from them, create tasks.

• User goal: Look up grades.• Good task: Look up the results of your midterm exams.• Poor task: You want to see the results of your midterm exams. Go to the

website, sign in, and tell me where you would click to get your transcript.

Grguric, Eka (2016) Guerilla Usability Testing and Communicating Value, NCSU Libraries Fellow, Code4Lib conference, 8 March 2016.

16

Example of a verbal prompt

• "Maybe you should try logging out?”

Better

• "Can you describe to me what you are expecting to see / happen right now?”

Avoid verbal prompts

17

Our Tasks

• Find peer reviewed articles on the topic of climate change.

• Find books on the topic of climate change

18

Tested three different sites

• http://www.wpunj.edu/library/

• https://catalog.wpunj.edu/Combined

• http://www.montclair.edu/library/

19

Make some action items

Items that were always (3 M’s):

• Missed • Misunderstood• Misused

Quickly summarize the key findings

20

Results

• Eight students tested the three sites – 3 were freshmen, 3 were seniors, 1 was a junior and 1 was a sophomore.

• Use of facets varied by class and interface. • Correct use of tabs varied by interface.• Drop down boxes were used.• Search options link was not used.• Confusion with some labels on the tabs.

21

Response from the Library

• Presented results at Faculty Librarians’ Meeting• Solicited volunteers for performing usability tests – one volunteer.• Results and design of future tasks will be main topic of next Library

Assessment Committee meeting.• Some agreement on what changes need to be made as a result of tests.

• Peer-review checkbox• Renaming tab labels(e.g., removing the word “Articles”)

• At this time, there is no mechanism for debriefing and how to decide on fixing usability issues.

22

Videos• Peachpit TV. (2010). Rocket Surgery Made Easy by Steve Krug: Usability Demo. Retrieved from

https://www.youtube.com/watch?v=QckIzHC99Xc&feature=player_embedded• Grguric, Eka (2016). Guerilla Usability Testing and Communicating Value, Code4Lib conference, 8 March

2016, https://www.youtube.com/watch?v=Dd04w--7EuY&feature=youtu.be&t=7999• UX tutorial: What is usability testing? | lynda.com - YouTube. (2013). Retrieved September 16, 2016,

from https://www.youtube.com/watch?v=48dMCEDJ1gM

Books• Krug, S. (2010). Rocket surgery made easy / the do-it-yourself guide to finding and fixing usability

problems.• Krug, S. (2006). Don't make me think! : a common sense approach to Web usability. Berkeley, Calif :

New Riders Pub., [2006].• Nielsen, J., & Loranger, H. (2006). Prioritizing Web usability. Berkeley, Calif. : New Riders, [2006].• Nielsen, J. (2000). Designing Web usability. Indianapolis, Ind. : New Riders, [2000].• Rubin, J. (1994). Handbook of usability testing : how to plan, design, and conduct effective tests. New

York : Wiley, [1994].

References and Other Resources

23

Websites• Advanced Common Sense - Steve Krug’s Web site. (2016). Retrieved September 16, 2016, from

http://www.sensible.com/• Matthew Reidsma : How we do usability testing. (2011). Retrieved September 16, 2016, from

https://matthew.reidsrow.com/articles/13

Software• Camtasia Studio. (2016, December 11). In Wikipedia. Retrieved from

https://en.wikipedia.org/w/index.php?title=Camtasia_Studio&oldid=754262442• Silverback 3. (n.d.). Retrieved March 8, 2016, from https://silverbackapp.com/• Snagit. (2016, September 5). In Wikipedia. Retrieved from

https://en.wikipedia.org/w/index.php?title=Snagit&oldid=737929379

24