View
18
Download
0
Category
Preview:
Citation preview
Xin Wang, Senior Lecturer (PhD), University of North Texas Brian Rennick, Associate Dean for Information Technology
(M.S.)., Brigham Young University Library
Applying User Experience Metrics for Optimizing Library Floor Map Application
使用用户计量数据优化图书馆地图软件设计
1
2
Outline
ØWhat are user experience (UX) metrics?
ØWhy are they important?
ØHow do you choose and apply them to inform the design decisions?
ØA real life example: Design floor map through incorporating user experience metrics
ØWhat are user experience research?
3
§ User-Centered Design and Development (UCD) Process for Information Architecture (IA)
Fig. 1: Three clouds of IA processDing, W. & X. Lin.(2010). IA research, design ,and evaluation. In Information Architecture: The design and integration of information spaces (pp.23-39). San Rafael: Morgan & Claypool Publishers.
Research
§Understand business goals and contexts§Employ user research methods to solicit user feedback§Analyze research results§Develop design strategies
Design
§Develop blueprints, wireframes, low/high-fidelity prototypes for user interfaces
Evaluation
§Perform log analysis or web usage analysis§Conduct usability evaluation
1. UX Research Framework
Research
§Understand business goals and contexts§Employ user research methods to solicit user feedback§Analyze research results§Develop design strategies
Design
§Develop blueprints, wireframes, low/high-fidelity prototypes for user interfaces
Evaluation
§Perform log analysis or web usage analysis§Conduct usability evaluation
4
§ UX Research Framework
Context
Content Users
Business goals, funding, technology, human resources
Document/data types, metadata, volume, existing structure
Audiences, tasks, needs, Information seeking behavior, vocabularies, experience
Table 1. UX Research Techniques
1. UX Research Framework
Source: Ding, W. & X. Lin.(2010). IA research, design ,and evaluation. In Information Architecture: The design and integration of information spaces (pp.23-39). San Rafael: Morgan & Claypool Publishers.
ContextBackground Research
Presentations and meetings
Stakeholder interviews
Technology assessment
ContentHeuristic Evaluation
Content analysis
Content mapping
Comparative Benchmarking
UsersCard Sorting User Interviews
and user testingContextual inquiry
Search log and clickstream analysis
Survey Focus group
Table 2. IA Research Methods
5
Personas
1. UX Research Techniques
6
2. What are User Experience Metrics?
ØA metric is a way of measuring a particular phenomenon or thing;
ØUX metrics are based on a reliable system of measurement;ØUX metrics reveal the interaction between a user and a
product; § UX metrics are observable and quantifiable;§ UX metrics are “People-oriented”§ Interaction § Attitude§ Behaviors
Source: Tullis, T. & Albert, B. (2013). Measuring the User Experience: Collecting, Analyzing, and Presenting Usability metrics. 2nd Edition. MA: Elsevier.
Type of UX Research Task
Suc
cess
Task
Tim
e
Erro
rs
Effic
ienc
y
Lear
nabi
lity
Issu
e-Ba
sed
Met
rics
Self-
Repo
rted
Met
rics
Beha
vior
al &
Psy
chic
al
Met
rics
Com
bine
d &
Com
para
tive
Met
rics
Live
Web
site
Met
rics
Card
-Sor
ting
Met
rics
1.Completing a transaction X X X X X 2. Comparing products X X X X 3. Evaluating frequent use of the same product X X X X X 4. Evaluating navigation and/or information architecture X X X X5. Increasing awareness X X X 6. Problem discovery X X 7. Maximizing usability for a critical product 8. Creating an overall positive user experience X X 9. Evaluating the impact of subtitle changes X 10. Comparing alternative designs X X X X X
7
Table 3: Ten Types of Common Usability Study and the Metrics that Maybe Most Appropriate for each
Ø Commonly Used UX Metrics
2. What are User Experience Metrics?
Source: Tullis, T. & Albert, B. (2013). Measuring the User Experience: Collecting, Analyzing, and Presenting Usability metrics. 2nd Edition. MA: Elsevier.
• Task Success• Task Time• Errors • Efficiency (Time)• Learnability• Issue-Based Metrics• Behavioral & Psychical Metrics• Combined & comparative
Metrics• Live Website Metrics• Card-sorting Metrics
8
§ Card sorting: a technique of organizing the elements of an information system in a way that makes sense to users
§ Implementing card sorting:1. Open/Closed sorts2. Cards are content items from a site or an information system3. Number of cards: 50-60 or less4. Number of participants: “15” is a good target5. Granularity: cards can be high-level or detailed (items exist
at a similar level is better for one-round sorting)6. Cross-listing: defining primary hierarchy or exploring
alternate navigation paths?7. Quantitative and qualitative data analysis
4. Apply UX Metrics into Design Decisions
9
Fig 1: BYU Library Responsive Website and the Floor Maps Application
5. A Real Life Example: a Floor Map Application
ØIn 2014, BYU libraries converted the website to one responsive design that performed equally well on all devicesØThe library wayfinding system was converted to a
responsive design as part of the overall redesign of the library website.
10
§ OptimalSort : a powerful tool to find out how users think your content should be organized
5. A Real Life Example: a Floor Map Application
11
1. Open card sorts: define the application structure with users’ perspectives
2. Number of cards: 503. Number of participants: 164. Granularity: high-level (content topics of main-page categories)5. Cross-listing: were not allowed6. Data analysis: Both quantitative and qualitative data analyses
5. A Real Life Example: a Floor Map Application
12
This sorting activity is like a fun game !
Fig.2 Instructions for the Card Sorting Activity
§ Participants can complete the card sort process from their own computers§ The intriguing process of card sorting increases the response rate
During the data analysis process, you probably have noticed that the
card sorting is not a completely objective and precise technique. Many times, you need to make
some subjective decisions. Different participants will sort cards in different ways, so the
cluster analysis and dendrogram won't produce the ideal
information architecture for you. However, OptimalSort can help you
start examining the patterns and the strength of the relationships. In order to pull together a suitable
grouping of items, you need to refer to the Cluster analysis, the
Dendrogram, the raw sort results plus participants provided grouping names. Of course, there might also
be business rules or real world contraints that mean certain items
have to go in certain places. Sometimes, unfortunately, politics
gets in the way of a good information architecture. Luckily,
you can also use the data from your sessions to help convince
management that it's time for a change. I like to print all the
information off, arrange it on my desk, and just absorb it for awhile.
Then, I try creating groups that seem to best match the majority view based on the sort results. I'll check the groups I create against the individual raw data, in case
there was some people who sorted an entirely different way. But my
aim is to make a hierarchy that will be acceptable to everyone who
participates in the sort. In the end, you need to apply a combination of
knowledge from the cluster analysis, dendrogram, and what
participants said during the sort, in order to create a good first pass at an information architecture. You should also use other data you have, such as usability studies,
customer support data, search logs, and web logs, to inform your
analysis. Don't blindly follow the statistical output. Think about what
participants said, and about real world implications. Back up your
decisions with this data. If you can't find data to back up your
decision, it indicates you might not have it right.
The next step, after we have our abstract hierarchy, is to refine it by testing it with a reverse sort. That's
the next topic.
13
§ Standardize similar categories
Fig. 3 The Category Page in OptimalSort
5. A Real Life Example: a Floor Map Application
Standardized category name
Participants’ category names
14
§ Find out the frequency of a specific card being placed in a normalized category§ Pick out cards with higher frequency§ Form a user-centered category with corresponding subcategories
15 out of 16 people placedThe “Family History”under this category.
Fig. 4 A Standardized Category
5. A Real Life Example: a Floor Map Application
Help Centers Count
Family History 15Accounting Lab 12
Teaching & Learning Lab 12
Microforms Reference 12
American Heritage Lab 11
Research & Writing Center 10
Social Science/Education 9
Business & Economics 9Science/Maps 8
15
Figure 5: Similarity Matrix of Card Sorting Data
Identifying obvious cluster “Library Services”
5. A Real Life Example: a Floor Map Application
Similarity Matrix: shows the percentage of participants who agree with each card paring.
16
5. A Real Life Example: a Floor Map Application
Figure 6: Standardization Grid
Ø Standardization Grid: shows the distribution of cards across the standardized categories you have defined. Each table cell shows the number of times a card was sorted into the corresponding scandalized category.
17
5. A Real Life Example: a Floor Map Application
Dendrograms: are used to illustrate the clusters. A “X%” score tells % of participants agree with this grouping.
Figure 7 : Standardization Grid: Dendrograms
18
§ Proposed New Categories for Global & Secondary Navigation
Figure 8: Newly Proposed Navigation and Labeling Systems of the Floor Map Application
5. A Real Life Example: a Floor Map Application
19
Ø The card sorting is not a completely objective and precise technique. Many times, you need to make some subjective decisions.
Ø OptimalSort can help you start examining the patterns and the strength of the relationships.
Ø There are also be business rules or real world constraints that mean certain items have to go in certain places. Sometimes, unfortunately, politics gets in the way of a good information architecture.
Ø To create a good first pass at an information architecture. You should also use other data you have, such as usability studies, customer support data, search logs, and web logs, to inform your analysis.
Ø Don't blindly follow the statistical output. Think about participants’ feedback, and about real world implications. Back up your decisions with this data
Ø After we have our abstract hierarchy, is to refine it by testing it with a reverse sort.
5. Conclusions
20
Recommended