Upload
others
View
4
Download
0
Embed Size (px)
Citation preview
# USABILITY TESTING
Usability
Definition of Usability in ISO/IEC 9241-11 Standard
The extent to which a product can be used by specified
users to achieve specified goals with effectiveness,
efficiency and satisfaction in a specified context of use.
Objective
Device testing
Performance testing
Security testing
• Ease of use
• Ease of learning with the system
• Satisfaction with the entire experience
Documents• Ethics documents
• Consent form
• Post-test questionnaire
Participants
• Be representative of the target users
• Own a target device + OS
• Be available at the time and place
• Be ready to sign a consent form
Devices
Procedure
Step Time
Welcome / Signing of the consent form 5 min
Pre-test interview 5 min
Carrying out the test tasks 35 min
Post-test questionnaire / interview 10 min
Debriefing 5 min
What is debriefing?
• Going through and analyzing the actions performed by the
participant (conducted with the participant)
• Test session identifies problems, debriefing session provides
insight on why those problems occurred
Testing Tasks
• Register an account
• Sign into your account
• Log your diary data
• Retrieve Fitbit data
• Visualize data
Data CollectionQuantitative data
Qualitative data
• Screen recorder records the foot stage of clicks
• Timer recorders how long it took participants to complete each task
• Bio-sensors record physiological metrics
• Can you talk out loud …?
• Can you walk me through…?
• What do you think of …?
• Could you suggest 3 changes ?
Performance metrics
ISO/IEC 9126-4 Metrics
Effectiveness The accuracy and completeness with which users
achieve specified goals
Efficiency The resources expended in relation to the accuracy and
completeness with which users achieve goals.
Satisfaction The comfort and acceptability of use.
Effectiveness
Completion Rate
𝑒𝑓𝑓𝑒𝑐𝑡𝑖𝑣𝑒𝑛𝑒𝑠𝑠 =𝑁𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑡𝑎𝑠𝑘𝑠 𝑐𝑜𝑚𝑝𝑙𝑒𝑡𝑒𝑑
𝑇𝑜𝑡𝑎𝑙 𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑡𝑎𝑠𝑘𝑠× 100%
Number of Errors
Efficiency
Time-based Efficiency
𝑇𝑖𝑚𝑒 𝐵𝑎𝑠𝑒𝑑 𝐸𝑓𝑓𝑖𝑐𝑖𝑒𝑛𝑐𝑦 =
σ𝑗=1𝑅 σ𝑖=1
𝑁 𝑛𝑖𝑗𝑡𝑖𝑗
𝑁𝑅
N: total number of tasks
R: total number of users
𝑛𝑖𝑗: the result of task i by user j; 𝑛𝑖𝑗=0 if a user fails to complete the task; 𝑛𝑖𝑗
=1 if a user complete the task
𝑡𝑖𝑗: the time spent by user j to complete task i
Efficiency
Overall Relative Efficiency
𝑂𝑣𝑒𝑟𝑎𝑙𝑙 𝑅𝑒𝑙𝑎𝑡𝑖𝑣𝑒 𝐸𝑓𝑓𝑖𝑐𝑖𝑒𝑛𝑐𝑦 =σ𝑗=1𝑅 σ𝑖=1
𝑁 𝑛𝑖𝑗 ∙ 𝑡𝑖𝑗
σ𝑗=1𝑅 σ𝑖=1
𝑁 𝑡𝑖𝑗× 100%
N: total number of tasks
R: total number of users
𝑛𝑖𝑗: the result of task i by user j; 𝑛𝑖𝑗=0 if a user fails to complete the task; 𝑛𝑖𝑗 =1
if a user complete the task
𝑡𝑖𝑗: the time spent by user j to complete task i
Satisfaction
Measured through standard satisfaction questionnaires
Task Level Satisfaction
• Questionnaires should be given to participants immediately
upon they attempt a task (irrespective of whether they
manage to achieve its goal)
• Popular questionnaires
SEQ: Single Ease Question (1 question)
UME: Usability Magnitude Estimation (1)
SMEQ: Subjective Mental Effort Questionnaire (1)
ASQ: After Scenario Questionnaire (3)
NASA-TLX: NASA’s Task Load Index (5)
Satisfaction
Test Level Satisfaction
• Questionnaires should be given to participants at the end of the
test session
• Popular questionnaires:
SUS: System Usability Scale (10 questions)
SUPR-Q: Standardized User Experience Percentile Rank Questionnaire (13)
CSUQ: Computer System Usability Questionnaire (19)
QUIS: Questionnaire for User Interaction Satisfaction (24)
SUMI: Software Usability Measurement Inventory (50)
Results
• Background summary: what, where, who was tested,
who was the testing team, what equipment was used
• Methodology: demographic data, task scenario,
metrics collected
• Test results: quantitative data (completion rates, error,
task times, satisfaction ratings) and qualitative data
(problems experienced, good/bad points, suggestions).
• Findings and recommendations: good and bad
usability, what and how to improve.
To Do
• Prepare documents (consent forms, questionnaires)
• Create task list
• Conduct usability testing with 3 participants
(If your app is not ready for user test today, you can continue working
on your app and do a user test later after the class)
• Write usability testing report
Submit Usability Testing Report to [email protected]
by 9 Jan Tuesday 12:00 pm.