Upload
farrukh-sahar
View
125
Download
1
Tags:
Embed Size (px)
Citation preview
Comparing the Effectiveness of
Electronic Diary and UX Curve Methods
in Multi-Component Product Study
Farrukh Sahar
Jari Varsaluoma
Sari Kujala
Contents
• Introduction
• Motivation and problem statement
• Methods
• Results
• UX Evaluation Implications for UX
Practitioners
• Conclusion
4.11.2014 2
Introduction
• In this study, the goal is to measure the effectiveness of
an electronic diary and the UX Curve method as remote
research methods for collecting qualitative data.
• The cost-effectiveness of the methods was evaluated
using two factors: the amount of time required by 1) the
researchers in employing the methods and 2) the
participants to report their experiences with the method.
• Also, to determine the quality of the reported
experiences, their content was compared between the
methods.
4.11.2014 3
Motivation and problem
statements
• in the literature there is lack of research on measuring
and comparing the effectiveness of electronic diary and
the UX Curve method as remote research method in the
long-term UX evaluation.
• In addition, there is a need to measure and compare the
effectiveness of methods in order to guide UX
practitioners on selecting an appropriate and effective
research method that enables valid and reliable
evaluation of long-term UX.
4.11.2014 4
Methods and studied
product
• 25 participants
• we conducted an eight-week-long electronic diary
study and subsequently employed the UX Curve
method.
• The studied sports watch system had four components.
The sports watch was a main component with three
other components: 1) a speed sensor; 2) a heart rate
belt; and 3) a sports website. All the participants had
bought the products for themselves and had been using
the product at least five times per week.
4.11.2014 5
Sports watch system
4.11.2014 6
Electronic diary and the UX
Curve method
4.11.2014 7
Electronic diary and the UX
Curve method
4.11.2014 8
Results=>Positive and
negative experience items
• In the positive experience items’ utility subcategory, the
participants reported more utilities related to the sports
watch system during the electronic diary study than with
the UX Curve method.
• Interestingly, there were 11 stimulation category
experiences related to the sports watch that users
reported only with the UX Curve method. Furthermore,
there were 12 stimulation category experiences related
to the sports website that users reported only with the
UX Curve method.
4.11.2014 9
Results =>Positive and
negative experience items
• In addition, the findings suggest that the electronic diary
was more effective than UX Curve method in capturing
the missing functionalities and technical faults.
• The findings show that even though the structures of
both methods were different, both methods captured
more experiences related to sports watch and sports
website compared with speed sensor and heart rate belt.
4.11.2014 10
Results =>Cost-effectiveness
• Comparison of cost-effectiveness of the electronic diary
study and the UX Curve method
4.11.2014 11
Results =>Cost-effectiveness
• Comparison of cost-effectiveness of the electronic diary
study and the UX Curve method
4.11.2014 12
Results =>Cost-effectiveness
• Comparison of cost-effectiveness of the electronic diary
study and the UX Curve method
4.11.2014 13
Results =>Cost-effectiveness
• Comparison of cost-effectiveness of the electronic diary
study and the UX Curve method
4.11.2014 14
Results =>Cost-effectiveness
• Comparison of cost-effectiveness of the electronic diary
study and the UX Curve method
4.11.2014 15
• With the electronic diary study the participants reported
their experiences in more detail than with the UX Curve
method.
• In our electronic diary study motivating the
participants and getting a good user response rate was
a challenge for the researchers because participants
were required to fill the weekly diary for two months. On
the other side, the UX Curve method required less
efforts by the researchers because participants were
required to draw the UX curves only once at the end of
the study.
4.11.2014 16
Results =>Cost-effectiveness
Results =>Workload
• During the UX Curve method data analysis, the
researchers invested efforts in deciphering the
handwriting of participants and transferring the data into
the electronic format.
• We experienced that because of the huge number of
details in the 200 collected diaries the data analysis
process was extensively laborious in nature and
consumed more time than the UX Curve method.
4.11.2014 17
UX Evaluation Implications
for UX Practitioners
1) In order to effectively reveal utility related experiences,
the UX practitioners should use an electronic diary
study rather than the UX Curve method.
2) In order to capture the missing functionalities and
technical faults, the UX practitioners should use an
electronic diary study rather than the UX Curve method.
3) The UX practitioners could use an electronic diary study
instead of the UX Curve method to reveal experiences
related to dissatisfaction.
4.11.2014 18
UX Evaluation Implications
for UX Practitioners
4) In order to reveal experiences associated with
effectiveness, the UX practitioners could use an electronic
diary study instead of the UX Curve method.
5) It seems that the UX practitioners could use the UX
Curve method rather than an electronic diary study in order
to capture the positive experiences related to stimulation.
4.11.2014 19
Conclusion
• Our results suggest that both methods are effective in
revealing specific type of information and have their
strength and weaknesses.
• The electronic diary was more effective than the UX
Curve method in revealing utilities, missing
functionalities and technical faults related to sports
watch system.
• On the other hand, the UX Curve method was more
effective than electronic diary in capturing the positive
experience items related to stimulation category.
4.11.2014 20
Limitations and Future
Studies
• The reported experiences with the electronic
diary study could have affected on the
remembered experiences with the UX Curve
method.
• In the future, it is important to study the
effectiveness of other methods during the
remote UX evaluation of multi-component
products.
4.11.2014 21