USABILITY H
U
M
A
N
F A
C
T
O
R
S
&
T
E
C
H
N
O
L
O
G
Y
Marko Nieminen
Helsinki University of Technology, Finland
Shop-floor Usability
Marko Nieminen
Helsinki University of Technology
http://www.interactive.hut.fi/persons/mniemi
USABILITY H
U
M
A
N
F A
C
T
O
R
S
&
T
E
C
H
N
O
L
O
G
Y
Marko Nieminen
Helsinki University of Technology, Finland
Shop Floor Environment
Co-workersSupervisors
Order ManagementProduction ControlProduct Revisions
Work InstructionsProduction Control
Assembly Tips
Assembly Cell
“INFORMATION ENVIRONMENT”
Tools &Materials
USABILITY H
U
M
A
N
F A
C
T
O
R
S
&
T
E
C
H
N
O
L
O
G
Y
Marko Nieminen
Helsinki University of Technology, Finland
Formative Usability Testing
User Observation
USABILITY H
U
M
A
N
F A
C
T
O
R
S
&
T
E
C
H
N
O
L
O
G
Y
Marko Nieminen
Helsinki University of Technology, Finland
Usability(Nielsen 1993)
SystemAcceptability
SocialAcceptability
PracticalAcceptability
Price
Compatibility
Reliability
etc.
Utility
Functional and OrganisationalUtility
Usability Learnability
Effective to use
Easy to remember
Few Errors
Subjectivelypleasant
USABILITY H
U
M
A
N
F A
C
T
O
R
S
&
T
E
C
H
N
O
L
O
G
Y
Marko Nieminen
Helsinki University of Technology, Finland
Usability Studies
� Heuristic evaluation
� User observation with video logging
� User interviews
� User questionnaires
� Computer event logging with a log-
file
USABILITY H
U
M
A
N
F A
C
T
O
R
S
&
T
E
C
H
N
O
L
O
G
Y
Marko Nieminen
Helsinki University of Technology, Finland
User Profiles
• Author Users
– The ones who create and
manage the content
information that is
presented in the
Interactive Task Support
System
– Experienced computer
users
– Need to be trained to their
tasks: no previous
knowledge about task
analysis
• Reader Users
– Shop-floor employees
– No computer experience
– Specialists in assembly
work: capable of
performing complex
assembly tasks, familiar
with assembly
environment
– No extensive training may
be provided
USABILITY H
U
M
A
N
F A
C
T
O
R
S
&
T
E
C
H
N
O
L
O
G
Y
Marko Nieminen
Helsinki University of Technology, Finland
Usability Goals
� Learnability: ITSS must be learned within 30 minutes (with 15 minutes tutorial)
� Efficiency: minimal effort for finding required information
� Memorability: see learnability
� Errors: prevention of (1) operation and (2) goal errors
� Attitude: subjectively pleasing
USABILITY H
U
M
A
N
F A
C
T
O
R
S
&
T
E
C
H
N
O
L
O
G
Y
Marko Nieminen
Helsinki University of Technology, Finland
Results
Questionnaires
• Questions
– 1 Experience in using ITSS, 2 Computer literacy, 3
Attitude, 4 Screen, 5 Messages & texts, 6 Pictures, 7
Audio, 8 Learning, 9 System performance
• Portions of the questions were transformed into an interview (free form questions)
• Results
– Overall attitude (negative/positive): 6.0
– Getting started (difficult/easy): 3.0
– Quality of speech (ambiguous/unambiguous): 2.0
– Quality of pictures (unclear/clear): 4.3
USABILITY H
U
M
A
N
F A
C
T
O
R
S
&
T
E
C
H
N
O
L
O
G
Y
Marko Nieminen
Helsinki University of Technology, Finland
Log files
• Logging of user initiated events from the user
interface
• Structure
– (1) Date and time, (2) Section of the system user is in, (3)
Current "page" name (content heading), (4) Function that
the user selected in that context
• Allows detailed level follow-up studies, even
“replays”
• Ethical questions!
– users must know about the logging
– for research and development purposes only, not for
performance evaluations
USABILITY H
U
M
A
N
F A
C
T
O
R
S
&
T
E
C
H
N
O
L
O
G
Y
Marko Nieminen
Helsinki University of Technology, Finland
Log File Results
Frequency of Use
0
50
100
150
200
1 8
15
22
29
36
43
Date (15.12.-28.1.)
B
0
300
600
900
12001
31
61
91
12
1
Date (7.6.-8.11.)
Am
ou
nt
of
ev
en
ts
A
USABILITY H
U
M
A
N
F A
C
T
O
R
S
&
T
E
C
H
N
O
L
O
G
Y
Marko Nieminen
Helsinki University of Technology, Finland
Log File Results
What Information is Used
"Last month" InitialHands-ontraining
Orientationtraining
Detailed assembly instructions 75 % 73 % 60 %General assembly instructions 13 % 14 % 15 %Start screen 10 % 5 % 15 %Tools 2 % <1% 10 %Parts and materials 1 % 7 % 0 %General organisationalinformation
1 % <1% 0 %
USABILITY H
U
M
A
N
F A
C
T
O
R
S
&
T
E
C
H
N
O
L
O
G
Y
Marko Nieminen
Helsinki University of Technology, Finland
Log File Results
What Functions are Used
"Last Month" InitialHands-ontraining
Orientationtraining
Button: Next 74 % 34 % 42 %Button: first detailed description 10 % 6 % 8 %Button: Previous 6 % 8 % 2 %Button: Picture 2 5 % 18 % 1 %Button: Contents 4 % 5 % 3 %Button: Sound / voice 1 % 6 % 0 %
USABILITY H
U
M
A
N
F A
C
T
O
R
S
&
T
E
C
H
N
O
L
O
G
Y
Marko Nieminen
Helsinki University of Technology, Finland
Conclusions
• ITSS is used continuously even though the frequency of use
decreases. Decrease may indicate learning in selected
assembly task, continuous use may indicate the search of
detail information.
• Users use only those functions that are essential to their work.
• Log files provide an easy way to gather detailed information
about the actual use of the system. Supplemental information
is, however, required by e.g. observation studies and
interviews to get a full picture of the situation. In addition, log
file analysis requires more automated analysis tools. Even
short time log-file analyses can indicate the style of real use.
• Questionnaires as well need to be supported by other
evaluation methods that provide more detailed information
about use context.
Recommended