Upload
ecaterina-moraru-valica
View
273
Download
0
Embed Size (px)
DESCRIPTION
Between 27 May - 12 June 2014 we tested XWiki with 9 users. The testing sessions were recorded and held face-to-face in Iasi, Romania. This is a presentation that contains some conclusions and some video snapshots from the testing sessions. More details http://design.xwiki.org/xwiki/bin/view/Proposal/UsabilityScenario4Report/
Citation preview
Usability Testing2014 Highlights & Findings
Ecaterina Moraru — 26 July 2014 —
Why is good to do usabilitysessions?
You confirm issues you've suspected
Get amazed by new things you discover
“”
·
·
SCENARIOScenario #4
PERIOD27 May - 12 June 2014
VERSIONXE 6.0
SKINColibri
DURATION9:45:45 h
FEATURESProfile, Navigation, Pages, Wikis,Applications
Usability Sessions
3/44
Tasks
1 | First Impressions
2 | Profile Editing
3.1 | Wiki Navigation
3.2 | Page Editing
3.3 | Page Hierarchy
4 | Sending Message
6 | Welcome Editing
7.1 | Wiki Creation
7.2 | Application Install
5 | Conclusions
Tasks
5/44
Task 1
INSTRUCTIONS HYPOTHESES
What are your firstimpressions of XWiki?
What do you think you cando with it?
You can scroll, hover, butplease don't click onanything yet.
Introduction
Testing: Homepage
Status: Not-logged-in
Status: Logged-in
Task 1
·
·
·
·
7/44
Task 2
INSTRUCTIONS HYPOTHESES
Imagine this is your firstday at work and XWiki isused internally.
You want to customiseyour profile and add someinformation about yourself.How can you do that?
UCUC11 : Measure the performance of 'menu-
button' vs. 'icon-link' (#tmEdit vs.
.editProfileCategory)
9/9 'icon-link' usage
Testing: Profile
9/9 Success
Task 2
·
·
·
9/44
(video)
Add hints to Profile Preferences Specify default values
Task 2 - Profile Editing
Safari Power SaverClick to Start Flash Plug-in
· · 10/44
Task 3
INSTRUCTIONS SUBTASKS
You've been given the assignment to add somecontent (text+image) for the Marketing team.
Task 3.1
Task 3.2
Task 3.3
INSTRUCTIONS HYPOTHESES
First let's browse a bit theproduct and locate wherethe Marketing material isfound.
UCUC22 : Measure the performance of Search,
Search Suggest vs. Wiki Index
7/9 Search Suggest
2/9 Wiki Index
Testing: Navigation
3/9 couldn't find
Marketing without
help
Task 3
Task 3.1
·
·
·
·
12/44
(playlist)
No suggest for partial words
No Wiki results in Search
Pressing Enter too fast
3/9 Suggest worked in
first try
4/9 Suggest failed in
first try
Task 3.1 - Navigation - Search Suggest
Safari Power SaverClick to Start Flash Plug-in
·
·
·
·
·13/44
(video)
Harder to find under menu
No Wikis gadget on Homepage
No Applications panel entry
1/9 Used only Wiki Index
2/9 Found it alone after
Suggest fail
Task 3.1 - Navigation - Wiki Index
Safari Power SaverClick to Start Flash Plug-in
·
·
·
·
·
14/44
Task 3.2
INSTRUCTIONS HYPOTHESES
You need to create a pagecalled 'XWiki Enterprise' (inthe Marketing location).
As a result, the page shouldlook similar to:
UCUC33 : In order to edit, you need to be a
member of the wiki. Measure task
success.
UCUC44 : Measure 'Add image' dialog naming.
UCUC55 : Measure table performances.
Testing: Page Creation
Testing: Page Editing
Extended: wiki syntax
Extended: macro usage
Task 3.2
·
·
·
·
16/44
EXPECTED RESULT0/9 expected result
3/9 created Wiki
2/9 edited Sandbox
ACTUAL RESULTS
x2 x2
Task 3.2 - Page Creation
·
·
·
17/44
(video)
EXPECTED RESULT ACTUAL RESULTS
Task 3.2 - Page Creation
Safari Power SaverClick to Start Flash Plug-in
18/44
(video)
EXPECTED RESULT ACTUAL RESULTS
Task 3.2 - Page Creation
Safari Power SaverClick to Start Flash Plug-in
19/44
(video)
EXPECTED RESULT ACTUAL RESULTS
Task 3.2 - Page Creation
Safari Power SaverClick to Start Flash Plug-in
20/44
(video)
ACTUAL RESULTS ACTUAL RESULTS
Task 3.2 - Page Creation
Safari Power SaverClick to Start Flash Plug-in
21/44
Task 3.2 - Page Creation
Challenges:Finding the Add buttonCreating a Page and not something elseCreating the page in the right location
Issues:Problems finding the Add buttonMissing Add and Edit actions from IndexesDifficulties in finding Wiki Index and understand current locationConfusion between Page and Wiki conceptsDepending on the current location, having automatically Parent setMissing permissions by not completing the Join this wiki step
····
·······
22/44
(video)
4/9 used 'Join this wiki'
Confusing between Move - Copy -Rename - Export
Missing 'Add - Page' option
Missing Rights to move pages
Task 3.2 - Join this wiki
Safari Power SaverClick to Start Flash Plug-in
·
·
·
·
23/44
(playlist)
Preference for 'External image'
Save confirmation needed
Improvements: Single click, Pre-filled label, etc.
Task 3.2 - Page Editing
Safari Power SaverClick to Start Flash Plug-in
·
·
·
24/44
Task 3.3
INSTRUCTIONS HYPOTHESES
Create another page called'Products'.
Link the 2 pages one fromanother in order to be ableto navigate between them.
Is it possible also to makean hierarchy?
(where 'Products' page ison an upper level than'XWiki Enterprise' page)
UCUC66 : Measure discoverability for 'Page
Parent' functionality.
UCUC77 : Asses the clearness of hierarchy
display (breadcrumb, document
index tree, document information,
etc.).
Testing: Page Linking
7/9 WYSIWYG Page
Linking
Testing: Page Parent
1/9 Page Parent
Task 3.3
·
·
·
·
26/44
(video)
Missing visibility for Parent
Cross wikis issues
Tree is hidden and ambiguous
Task 3.3 - Page Parent
Safari Power SaverClick to Start Flash Plug-in
·
·
·
27/44
Task 4
INSTRUCTIONS HYPOTHESES
You would like to let themanager know that thecontent has been addedand the page is ready.
How would you do that?
UCUC88 : Evaluate alternative communication
methods.
8/9 Send Message
1/9 Send Page by mail
Testing: Send Message
Testing: Page
Comments
Testing: Send Page by e-
Task 4
·
·
·
·
·
29/44
Task 6
INSTRUCTIONS HYPOTHESES
Go to the Home page andremove the first paragraphfrom the 'Welcome to yourwiki' section.
UCUC99 : As simple users the ability to edit
inline is not available. Since the
default editor is WYSIWYG, the user
will not be able to remove the
paragraph without removing the
whole Dashboard macro.
8/8 failed to edit the text
2/8 changed the title
2/8 deleted gadget
Testing: Dashboard
Testing: Inline Editing
Task 6
·
·
·
·
·
31/44
(playlist)Task 6 - Welcome Editing - Main
Safari Power SaverClick to Start Flash Plug-in
32/44
(playlist)Task 6 - Welcome Editing - Dashboard
Safari Power SaverClick to Start Flash Plug-in
33/44
Task 7.1
INSTRUCTIONS HYPOTHESES
You would like to create adedicated place forDevelopment material, justlike the one existing for theMarketing team.
Create such a location.
UCUC1010 : Evaluate the preference for wiki or
space solution.
3/9 Space instead of Wiki
UCUC1111 : Evaluate Distribution Wizard's
impact and friendliness.
6/9 Later
1/9 Never
2/9 Install
Testing: Wiki Creation
Testing: Space Creation
Testing: Distribution
Wizard
Prereq: X user has 'Add
Wiki' right
Task 7.1
·
·
·
·
·
·
·
·
35/44
(video)Task 7.1 - Wiki Creation
Safari Power SaverClick to Start Flash Plug-in
36/44
Task 7.2
INSTRUCTIONS HYPOTHESES
The Development teamcould use a Calendarapplication.
Find out how to install thisapplication on your wiki.
UCUC1212 : Evaluate multiple Calendar
applications naming ambiguity.
6/8 Full Calendar Macro
1/8 Calendar Panel
Application
1/8 Mocca Calendar
Application
Testing: Administration
Testing: Application
Panel
Testing: Extension
Manager
Task 7.2
·
·
·
·
·
·
38/44
(playlist)
Display only Applications
Add a Search button
Provide entries in Suggest
Task 7.2 - Application Install
·
·
·
39/44
Task 5
INSTRUCTIONS HYPOTHESES
Thinking about your overallexperience with XWiki,what did you like about theproduct?
What did you not like?
Would you use it in thefuture? Why or why not?
Conclusions
Task 5
·
41/44
Questions?
Thank you to all the participants and for your attention
Ecaterina Moraru — 26 July 2014 —