Usability Testing

David Rashty

"I find that a great part of the information I have was acquired by looking up something and finding something else on the way."

Franklin P. Adams (1881-1960)

Sounds familiar?

• Not being able:– to find the information you are looking for.– to find a page you know is out there.– to return to a page you once visited.– to determine where you are.– to visualize where you have

been and where you can go.

What is usability?

• Usability is closely related to: – Ease of use

effective performance of tasks.– Efficiency

resources expended in relation to the accuracy of goals achieved.

– Satisfactionthe comfort and acceptability of the interaction.

Usability testing

• Usability testing is a methodology that employs potential users to evaluate the degree to which a website/software meets predefined usability criteria.

Usability testing – basic elements

• Problem statement• Testing methods• Getting test users• Observation and

testing• Output and results

Problem statement

• Before conducting a testing one should clarify the purpose of the test since it will guide the rest of the testing procedures.

• Important issues:– What do you want to achieve?

– Are there any navigational and organizational problem areas within the site?

– Is there any confusing terminology within the site?

Testing methods

MethodShort descriptionUsability laboratoryA room with computer equipment, a place for

an observer to sit and a special observation area.

Web-basedOnline evaluation with live feedback from users.

Thinking aloudA test subject thinks aloud while navigating the site.

ObservationVisiting the users and observing them work.

QuestionnairesSite or email questionnaires are an effective way of measuring user satisfaction.

Testing methods

MethodShort description

InterviewsWell suited to exploratory studies where one does not know yet what one is looking for.

Focus groupsUsers are brought together to discuss new concepts and identify important issues.

Heuristic evaluationLooking at an interface and trying to come up with an opinion about pros and cons about it.

Log file analysisThe computer automatically collect statistics about the detailed use of the system.

User feedbackShows the immediate and pressing concerns, is an ongoing process and is adapts quickly.

An example: usability labs

• Requirements:– Provide test room– Provide data analysis

tools– Have a supportive

environment– Provide portable tools– Have space for group

testing, focus groups and task analysis.

• Staff– Usability engineer for

administering the activities in the lab

– A psychologist and a computer specialist for analyzing the data from the experiments

An example: usability labsThe subject room must contain office furniture, video tape equipment, a microphone and a computer with appropriate software.

The observer side must contain a powerful computer to collect the usability data and analyze it. A one-way mirror separates the rooms.

An example: heuristic evaluation

• A technique for finding usability problems with a user interface.

• A small number of trained evaluators separately evaluate a user interface by applying a set of broad directives that are relevant to the case in question.

• They then combine their results and classify each problem according to its importance defining the solution process.

An example: heuristic evaluation

• Jakob Nielsen identified 10 heuristics which are broadly helpful in spotting the vast majority of problems.

1. visibility of system status 2. match between the system

and the real world 3. user control and freedom 4. consistency and standards 5. error prevention 6. recognition rather than

recall 7. flexibility of use 8. minimalist design 9. help users diagnose and

recover from errors 10. help and documentation

Getting test users

• It is important to identify target users and then recruit them to participate in the usability test.

• Research indicates that five to seven participants are enough to show trends.

Observation and testing

• The test is divided into four phases:– Preparation

• Test room is ready, computer in start state and written material available at hand.

– User explanation• Purpose of the test, computer setup, test procedure.

– Running the test• Test according to method choice.

– User feedbacks• Satisfaction questionnaires

Output and results

• A usability report should contain references to the following topics:– A statistical analysis on the usage of the

different regions in a website;– A qualitative and quantitative comparison of

different models for the main elements in the site;

– Recommendations with respect to labeling, searching, organization and navigation tools.

Output and results

• What does your organization do after receiving a usability report?– Implement changes

• The implementation requires a concentrated effort in order to fix all usability problems.

• The implementation may require further consultation with specialists in user interface design and information architecture in order to implement the proposed changes.

Usability testing: costs

• Website usability review by Nielsen: $30,000

• Average cost of an advanced usability lab:$20,000

• Web-based usability testing:

$20,000

Usability study: MOF case

• The Israeli Ministry of Finance previous website had 5,000 pages, was updated daily and had more than 40,000 visitors monthly.

• But…– 60% of users stayed less

than a minute in the site.

– Only 8% stayed more than 10 minutes.

Usability study: MOF case

• The results of a thorough usability study show that:– Users spent 30% more time

in the site.– The number of pages

visited grew by 25%.– The number of users that

left the site after visiting the home page get smaller by 50%.

– Navigation and searching systems more intuitive and simple.

Conclusions

• Usability is essential to survival.

• Usability is the key technique for superior customer relationships.

• Attention to usability increases the percentage of those who complete a purchase after visiting a website.

“Usability rules the Web. Simply stated, if the customer can’t find a product then he or shewill not buy it.”

“The Web is the ultimate customer-empowering

environment. He or she who clicks the mouse gets to decide everything. It is so easy to go elsewhere; all the competitors in the world are but a mouseclick away."

Jakob Nielsen

References

• Sites on usability– www.useit.com: Jakob Nielsen’s website– www.usableweb.com: 793 links about web usability– www.usabilityfirst.com: A guide to usability resources– www.webword.com: Usability and human factors– www.usability.org: Usability and general resources – www.uie.com: User Interface Engineering – www.webpagesthatsuck.com: Good counter examples– www.vividence.com: Offers online testing– www.microsoft.com/Usability: Usability at Microsoft