Bas arkun loh-usability

Baş, T., & Arkün-Kocadere, S. (2016). Usability studies of “Labours of Hercules”: The mythological mobile game. In G. Pop (Ed.).

Proceedings of the Contemporary World Challenges for the European Citizen Conference (pp. 310-313). Cluj-Napoca, Romania.

USABILITY STUDIES OF "LABOURS OF HERCULES":

THE MYTHOLOGICAL MOBILE GAME

Turgay Baş, Hacettepe University, [email protected]

Selay Arkün Kocadere, Hacettepe University, [email protected]

Abstract: Labours of Hercules (LOH) mobile game is an intellectual output of the “Searching

for the Labours of Hercules” (SFTLOH) Erasmus+ Project which focuses on the integration of

information and communication technologies (ICT) into teaching and learning process. This

study aims to explain the usability studies that were performed in the development process of

the mobile game. Usability is defined as the degree to which a product can be used free of

problems. Usability tests are performed either during the development process and/or at the

end of the process. Usability studies performed at the beginning of the development process

and repeated until completion limit the number of problems and prevent extra development

costs or time. While developing LOH, 8 different usability test in 3 phases were performed with

the participation of 3 team members, 22 users, and 1 expert. In addition to the usability tests

that have already been completed, the 4th phase will be performed with the participation of

real users as an end product test.

Keywords: usability, Labours of Hercules, Erasmus+ project, mobile game, educational game

Introduction

Usability is generally defined as the degree to which a product can be used without encountering

difficulties or errors. Usage without problems result in higher satisfaction, ease of use, and aim

completion. Indicators of usability include not getting an error, solving an error easily,

understanding the usage of product, learning how to use it without help or without having to

think about it, and leaving the usage process satisfied. Usable products usually serve their

design goals (Dumas & Redish, 1999, www.usability.gov, ISO 9241-11).

Usability is generally associated with the field of human-computer interaction and is assumed

to be a user interface feature of computers and computer-like devices such as tablets, phones,

etc. Usability studies are important to help create more user friendly environments by

facilitating interfaces that are errorless, compatible with the target audience, and more

qualitative.

The main aim of usability studies is to determine the degree of usability of a product and to

develop a strategy to improve it. Although various evaluation methods have been described in

the literature, two in particular are more commonly used. The first, user evaluation, is performed

by the representatives of the target group, based on their user experiences. The second method,

expert evaluation, is based on criterion and is performed by experts who have qualifications in

the field of human-computer interaction (Dix, Finlay, Abowd, & Beale, 2004; Barnum, 2010).

Usability studies are also categorized according to their timing, either during the design process

or at the end of it. While usability tests run during the development process generally aim to

improve the usability of product, others aim to determine the usability degree of product.

Ideally, usability studies are performed both within and at the end of the process with the

participation of both users and experts (Rubin & Chisnell, 2008; Barnum, 2010).

mailto:[email protected]

mailto:[email protected]

http://www.usability.gov/



This study aims to explain the usability studies performed during the design process of the

Labours of Hercules (LOH) mobile game. LOH is an intellectual output of the “Searching for

the Labours of Hercules” (SFTLOH) project supported by the European Union Erasmus+

Program Key Action 2, Mixed Strategic Partnership for School Education. A total of 8

institutions (5 high schools, 1 non-governmental organization, and 2 universities) collaborated

under the leadership of Mehmet Çelikel High School in Zonguldak / Turkey.

Erasmus+ Project: Searching for the Labours of Hercules

Project SFTLOH is based on the mythological story of Hercules and his travels throughout

Europe to complete the Twelve Labours. The Labours of Hercules represents the common

heritage of European countries and aims to promote common heritage amongst cultural

diversity and common heritage of the 6 partner countries of Turkey, Romania, Italy, Spain,

Greece, and Hungary.

The focus of Project SFTLOH is the integration of information and communication

technologies (ICT) into teaching and learning processes. For this purpose, various opportunities

for improving and facilitating the use of technology in instructional processes were provided to

teachers and students. ICT related workshops were given to teachers and students and two

intellectual outputs were designed by two partner universities. These outputs were planned to

produce qualitative examples of the use of ICT for learning.

In this study, we will focus on one of these two examples, the LOH mobile game. Educational

games have been used in teaching since the 1980s and demand for educational games has not

diminished over the years. On one hand, technology, especially mobile technology, has grown

enormously while on the other learners of today have become defined as the game generation.

Such developments have made mobile games an inevitable part of not just for daily life but also

education.

In addition to creating an example of ICT integration into the learning environment in general,

LOH was developed with the aim of producing a learning material for teaching mythology, in

particular the story of the “12 Labours of Hercules”. The game was made up of 12 levels

corresponding to one labour with each level beginning with the runner phase and ending with

a boss fight. Each level included a challenge corresponding to that level’s labour.

Usability studies in design process

Through an integrated approach, the usability studies implemented for the LOH mobile game

were performed in different phases of the design process in different devices by different users

(Table 1). This study only includes the tests not performed by the game developers.

Table 1: Usability studies in design process

1st Phase

Paper Prototype

2nd Phase

Digital Prototype

3rd Phase

Version 1.0

Design Team Test

User Test

Expert Evaluation

Design Team Test (with test devices)

User Test (with test devices)

Design Team (with personal devices)

User Test (with personal devices)

User Test

Expert Evaluation



1st Phase – Paper Prototype

The first usability studies were performed progressively using a paper prototype by 3 members

of the design team, 5 students representing the target group, and 1 expert. The game’s main

screens were presented to all testers. The aim of this test was to determine any complicated or

problematic parts of the user interface and to solve any resultant issues. The majority of the

items reported as problematic were buttons, icons, and direction messages, as well as

complexity of main page. After discussing this feedback with the graphic designers and coders,

the design team made changes considering the usability test reports. At the end of the first

phase, a digital prototype was developed.

2nd Phase – Digital Prototype

In the 2nd phase, a draft of the digital version was tested by 3 members of the design team and

10 students representing the target group. First, test devices were used, both on the Android and

IOS platforms, and users were asked to download the game and test it on their own devices.

While few problems were determined in tests on the test devices, there were many technical

problems on the personal devices. The majority of problems were about downloading, loading,

and performance. Results of the phase 2 usability tests were reflected in Version 1.0 of the

game.

3rd Phase – Version 1.0

Version 1.0 of the game (Appendix) was developed and uploaded to Google Play and the App

Store. Usability tests of this version were completed by 7 students and 1 expert. Thanks to the

previous tests, results were not related with technical issues in this phase but instead provided

feedback for improving the user experience. Considering the tests in this phase, the determined

problems were fixed in Version 1.1.

4th Phase – Target Group

In the final phase usability studies with real users to improve the game regarding the thoughts

of the target group are planned.

Conclusion

Usability studies performed on the LOH mobile game were completed during the design

process through various methods and are planned for the end stage. In addition to the design

team, 22 volunteer students and an expert participated in the studies. The majority of the

usability problems were prevented and not presented to real users. As the usability studies

started at the beginning of the design process and proceed progressively to the end, they served

to prevent extra cost and time in the revision stages in addition to helping to lower the incidence

of problems.

While running tests without real users might be considered a limit of the study, the participation

of volunteer students served to speed up the testing and fixing process. On the other hand, the

real target group will be included in usability studies performed with the end product available

on the mobile app markets in the 4th phase, also mitigating this limitation.

Acknowledgment

“Labours of Hercules” mobile game is funded by the Erasmus+ Program of the European Union

as a part of project “Searching for the Labours of Hercules” (2014-1- TR01-KA201- 012990)

and Hacettepe University Scientific Research Coordination Unit with the project titled

“Development of an Educational 3D Mobile Game” (SUA-2016- 9609)”. This educational

game is designed and developed in cooperation with Pixega Game Studio.



Appendix: Screen shots of the game



REFERENCES

Barnum, C. M. (2010). Usability Testing Essentials: Ready, Set, Test. Morgan Kaufmann.

Dix, A., Finlay, J., Abowd, D. G. & Beale, R. (2004). Human-Computer Interaction. (Third

Edition). Prentice Hall.

Dumas, J. S. & Redish, J. (1999). A Practical Guide to Usability Testing. Greenwood

Publishing Group Inc. Westport, CT, USA.

ISO 9241-11 (1998). Ergonomics Requirements for Office Work with Visual Display Terminals

(vdts) part 11: Guidance on Usability.

Rubin, J. & Chisnell, D. (2008). Handbook of Usability testing. (Second Edition). Wiley.

USA Official Web Site of Usability. Retrieved December 19, 2016, from www.usability.gov

http://www.usability.gov/