Usability testing in an agile development process

How our approach to testingmight work for you too…

Neil Allison, UX ManagerUniversity of Edinburgh

UCISA Manchester17 March 2016

@usabilityed#ucisa16 #usability

This presentation isn’t about Agile

• It’s about regular, rapid, inclusive usability testing with minimal overheads

• It just so happens that the agile process we used to develop the new University CMS (EdWeb) forced me to work this way– It can work for you regardless

#ucisa16 #usability

Usability testing

It’s easy, right?

http://bit.ly/krug-test-demo #ucisa16 #usability

What’s challenging

• Getting the go ahead to use time & money on usability testing

• Getting colleagues & customers to take on board what you uncover

• Getting fixes to problems implemented

(Why usability problems go unfixed: http://bit.ly/LvrGoq)

#ucisa16 #usability

Unless you’re…

Flickr credit: randychiu#ucisa16 #usability

My challenges as UX Lead for EdWeb

• It’s not a formal role in Information Services

• Misconception that it’s the UX Lead’s job to ‘decide what’s usable’

• Team is too close to the product, with not enough exposure to CMS users

• Striking a balance between delivering new functionality and improving what we have

#ucisa16 #usability

So what did we do?

1. Get the right people in a room

2. Watch a small number of short sessions with users doing something

3. Prioritise the issues they see

4. Collaboratively consolidate their priority lists

5. Agree actions for usability issues

6. Repeat every few weeks#ucisa16 #usability

Who are the right people?

• Everyone with a stake in the product– No exceptions

http://bit.ly/1I1lZfQ

“Have you had your recommended dose

of research?” #ucisa16 #usability

What did we watch?

• Real CMS users doing real tasks

• Facilitated usability testing sessions

• Focus of testing agreed collaboratively in team

“Research shows that teams make better services when everyone on a project team observes users first hand.”

http://bit.ly/1I1rlYI #ucisa16 #usability

How many did we watch?

“The most striking truth of the curve is that zero users give zero insights.”

• As many as you can fit into the time you have (so probably not very many)

http://bit.ly/1vQ7eHD

MYTH?

#ucisa16 #usability

How did we prioritise?

“Running a usability test has been compared with taking a drink from a fire hydrant…”

• Rocket Surgery template:1. Individual notes while observing2. Distil to 3 issues after each participant

http://bit.ly/1I1mCWW

“If you prioritise usability problems using 'gut feel' or intuition, you run the risk

of being exposed as a fraud…”

How did we consolidate?

Then what?

• Usability issues prioritised, not solutions

• Agree actions based on:– Is a solution “obvious”?– Is there an easy development solution?– Is there an alternative to development?

#ucisa16 #usability

Minor issue for users

Major issue for

users

Easy solution available

No easy solution available

Prioritisation matrix

#ucisa16 #usability

Recap: our process for EdWeb

In advance• Agree test focus with team

• Write and pilot test script

• Recruit 3 participants to turn up on the day

On the day• 3 sessions:

– 30 minutes max– 15 minutes between

• Observers prioritise their notes between sessions

• Final 30(ish) minutes spent prioritising top observations & agreeing actions

#ucisa16 #usability

Alternatives to live testing

• Record in advance for smoother event mgt– Running usability tests and facilitating the

observation group at same time is challenging

• Use a remote testing service– www.usertesting.com– www.whatusersdo.com– There are others…

#ucisa16 #usability

What was good about our process

For the team• Closer to our CMS users –

immediate impact

• Shared insight & experience

• Ownership of the priority issues– What to fix immediately– What we can live with that

we thought was a problem

For me• Process keeps set up and

organisation of session to a minimum

• No report writing

• Moves the culture of the team on, emphasising CMS usability on the agenda

#ucisa16 #usability

Ongoing challenges

• How we reduce usability problems occurring in the system in the first place– Developer time at a premium– Limited time for collaborative forward planning

• Getting more of the right people in the room– For longer and more frequently

#ucisa16 #usability

Everything you need

• Steve Krug’s Rocket Surgery resources:http://bit.ly/1I1muXo

• David Travis’ prioritisation flowcharthttp://bit.ly/1I1mCWW

• My blog article on this process:http://bit.ly/uoe-agile-usability

#ucisa16 #usability

Epilogue…

• EdWeb CMS is a huge success– More useful, usable & satisfying CM experience– Significant ROI in terms of staff time spent on top tasks,

training & support – Big uptake of system beyond the customer base of our

legacy CMS

• New usability testing service available to product development and service management teamshttp://bit.ly/UoE-usability-service

#ucisa16 #usability

Thank you

Questions?

Neil.Allison@ed.ac.ukWebsite Programme UX Manager

University of EdinburghTwitter: @usabilityed

#ucisa16 #usability