USABILITY H

U

M

A

N

F A

C

T

O

R

S

&

T

E

C

H

N

O

L

O

G

Y

Marko Nieminen

Helsinki University of Technology, Finland

Shop-floor Usability

Marko Nieminen

Helsinki University of Technology

http://www.interactive.hut.fi/persons/mniemi

Marko.Nieminen@hut.fi

USABILITY H

U

M

A

N

F A

C

T

O

R

S

&

T

E

C

H

N

O

L

O

G

Y

Marko Nieminen

Helsinki University of Technology, Finland

Shop Floor Environment

Co-workersSupervisors

Order ManagementProduction ControlProduct Revisions

Work InstructionsProduction Control

Assembly Tips

Assembly Cell

“INFORMATION ENVIRONMENT”

Tools &Materials

USABILITY H

U

M

A

N

F A

C

T

O

R

S

&

T

E

C

H

N

O

L

O

G

Y

Marko Nieminen

Helsinki University of Technology, Finland

Formative Usability Testing

User Observation

USABILITY H

U

M

A

N

F A

C

T

O

R

S

&

T

E

C

H

N

O

L

O

G

Y

Marko Nieminen

Helsinki University of Technology, Finland

Usability(Nielsen 1993)

SystemAcceptability

SocialAcceptability

PracticalAcceptability

Price

Compatibility

Reliability

etc.

Utility

Functional and OrganisationalUtility

Usability Learnability

Effective to use

Easy to remember

Few Errors

Subjectivelypleasant

USABILITY H

U

M

A

N

F A

C

T

O

R

S

&

T

E

C

H

N

O

L

O

G

Y

Marko Nieminen

Helsinki University of Technology, Finland

Usability Studies

� Heuristic evaluation

� User observation with video logging

� User interviews

� User questionnaires

� Computer event logging with a log-

file

USABILITY H

U

M

A

N

F A

C

T

O

R

S

&

T

E

C

H

N

O

L

O

G

Y

Marko Nieminen

Helsinki University of Technology, Finland

User Profiles

• Author Users

– The ones who create and

manage the content

information that is

presented in the

Interactive Task Support

System

– Experienced computer

users

– Need to be trained to their

tasks: no previous

knowledge about task

analysis

• Reader Users

– Shop-floor employees

– No computer experience

– Specialists in assembly

work: capable of

performing complex

assembly tasks, familiar

with assembly

environment

– No extensive training may

be provided

USABILITY H

U

M

A

N

F A

C

T

O

R

S

&

T

E

C

H

N

O

L

O

G

Y

Marko Nieminen

Helsinki University of Technology, Finland

Usability Goals

� Learnability: ITSS must be learned within 30 minutes (with 15 minutes tutorial)

� Efficiency: minimal effort for finding required information

� Memorability: see learnability

� Errors: prevention of (1) operation and (2) goal errors

� Attitude: subjectively pleasing

USABILITY H

U

M

A

N

F A

C

T

O

R

S

&

T

E

C

H

N

O

L

O

G

Y

Marko Nieminen

Helsinki University of Technology, Finland

Results

Questionnaires

• Questions

– 1 Experience in using ITSS, 2 Computer literacy, 3

Attitude, 4 Screen, 5 Messages & texts, 6 Pictures, 7

Audio, 8 Learning, 9 System performance

• Portions of the questions were transformed into an interview (free form questions)

• Results

– Overall attitude (negative/positive): 6.0

– Getting started (difficult/easy): 3.0

– Quality of speech (ambiguous/unambiguous): 2.0

– Quality of pictures (unclear/clear): 4.3

USABILITY H

U

M

A

N

F A

C

T

O

R

S

&

T

E

C

H

N

O

L

O

G

Y

Marko Nieminen

Helsinki University of Technology, Finland

Log files

• Logging of user initiated events from the user

interface

• Structure

– (1) Date and time, (2) Section of the system user is in, (3)

Current "page" name (content heading), (4) Function that

the user selected in that context

• Allows detailed level follow-up studies, even

“replays”

• Ethical questions!

– users must know about the logging

– for research and development purposes only, not for

performance evaluations

USABILITY H

U

M

A

N

F A

C

T

O

R

S

&

T

E

C

H

N

O

L

O

G

Y

Marko Nieminen

Helsinki University of Technology, Finland

Log File Results

Frequency of Use

0

50

100

150

200

1 8

15

22

29

36

43

Date (15.12.-28.1.)

B

0

300

600

900

12001

31

61

91

12

1

Date (7.6.-8.11.)

Am

ou

nt

of

ev

en

ts

A

USABILITY H

U

M

A

N

F A

C

T

O

R

S

&

T

E

C

H

N

O

L

O

G

Y

Marko Nieminen

Helsinki University of Technology, Finland

Log File Results

What Information is Used

"Last month" InitialHands-ontraining

Orientationtraining

Detailed assembly instructions 75 % 73 % 60 %General assembly instructions 13 % 14 % 15 %Start screen 10 % 5 % 15 %Tools 2 % <1% 10 %Parts and materials 1 % 7 % 0 %General organisationalinformation

1 % <1% 0 %

USABILITY H

U

M

A

N

F A

C

T

O

R

S

&

T

E

C

H

N

O

L

O

G

Y

Marko Nieminen

Helsinki University of Technology, Finland

Log File Results

What Functions are Used

"Last Month" InitialHands-ontraining

Orientationtraining

Button: Next 74 % 34 % 42 %Button: first detailed description 10 % 6 % 8 %Button: Previous 6 % 8 % 2 %Button: Picture 2 5 % 18 % 1 %Button: Contents 4 % 5 % 3 %Button: Sound / voice 1 % 6 % 0 %

USABILITY H

U

M

A

N

F A

C

T

O

R

S

&

T

E

C

H

N

O

L

O

G

Y

Marko Nieminen

Helsinki University of Technology, Finland

Conclusions

• ITSS is used continuously even though the frequency of use

decreases. Decrease may indicate learning in selected

assembly task, continuous use may indicate the search of

detail information.

• Users use only those functions that are essential to their work.

• Log files provide an easy way to gather detailed information

about the actual use of the system. Supplemental information

is, however, required by e.g. observation studies and

interviews to get a full picture of the situation. In addition, log

file analysis requires more automated analysis tools. Even

short time log-file analyses can indicate the style of real use.

• Questionnaires as well need to be supported by other

evaluation methods that provide more detailed information

about use context.