Mobile Legends and Interface Usability: Eye Tracking

Heads-Up-Displays

                     Josh Beitle Steve Si            M.S. Game Science  and Design                                 M.S Game Science and Design           Northeastern University                                                       Northeastern University

                    Victor Ma                                                    Rudra Trivedi   M.S. Game Science  and Design                                   M.S Game Science and Design         Northeastern University                                                            Northeastern University

Abstract

Multiplayer Online Battle Arena games (MOBA) have dominated PC gaming in recent years and

have begun expanded into the mobile realm. When running on mobile devices, MOBA games

must integrate all of the genres notoriously complex control interfaces to a cell phone size

screen, such as movement, attack, ability buttons, and mini-maps. Our research is focusing on

Mobile Legends, a popular mobile MOBA game. Our study aimed to investigate whether the

user interface on Mobile Legends now is effective and usable to make recommendations on how

it might be improved. Through the ‘Eye Tracking’ technology and gameplay feedback survey,

we concluded that most parts of the user interface are intuitive for new players to learn easily,

but still has room to perfection by removing potentially distracting stimulus from the high action

areas of the screen.

Introduction

Multiplayer Online Battle Area games, or MOBAs for short, are a very popular video

game genre in the world. Lead by League of Legends, there are countless competition events that

offer an actual cash reward for players. In our project, we are focusing on one of the most

popular Mobile MOBA games, Mobile Legends, which is running on a mobile device such as a

cell phone or iPad. MOBA games as a gaming genre on mobile devices is relatively new,

however, they arose from a huge following from PC MOBA games and a desire for more

frequent gameplay experiences. The game rules are simple that players must invade and take the

opposing team’s side by destroying their base. However, the control of the game is not that easy

and clear as PC MOBAs. Instead of using a mouse and keyboard to play, players in Mobile

Legends have to control the whole game through a small phone size touch screen, making it a lot

more difficult to recognize all of the gameplay information on the screen. Hence, to make sure

that players can play the game on mobile devices easily and make players’ skills level be

reasonably separated, the usability of Mobile Legends becomes very important for its success.

For Mobile Legends, is that players must manage a complex interface that includes an

attack button, 4 to 6 ability buttons, a mini-map, scoreboard, joystick, and chat room all on a cell

phone size screen. Even if played on a tablet like the iPad-Mini, the screen’s size only has

approximately 7.5 inches. Mobile Legends requires players’ constant interaction during the

game, game designers must pay careful attention to usability issues. Failure to design usable

game interfaces can interfere with the larger goal of creating a compelling experience for users

and can have a negative effect on the overall quality and success of the game. Thus, usability is a

critical element for Mobile Legends and mobile games in general. Usability in video games is

about maximizing effectiveness, efficiency, and satisfaction. This definition originates from the

traditional software industry, but it translates well to game development. In games, usability is

about delivering a better and deeper experience with less unnecessary interruptions or challenges

that have not been designed by the developers. In a nutshell, game usability is the degree to

which a player can learn, control, and understand a game (Pinelle).

To begin playing Mobile Legends, players are required to play through 30-40 minutes’

worth of tutorials, which explain all aspects of the game clearly. However, obviously, it is a lot

of time for players to spend 40 minutes in a tutorial before playing their game. To test the game’s

innate usability and intuitiveness, we tested a group of players’ first experiences with Mobile

Legends by having them play a match with no tutorial while tracking their eyesight trails to learn

about how they focus during the game processing. We are going to ask them to think out loud

during their gameplay and finish a survey about their specific feelings about the game. A

combined analysis of participant’s attitudes towards certain featured and the time spent looking

at each interface section will show us which pieces of the game were intuitive for new players.

Hypothesis: The user interfaces in Mobile Legends are well layed out, making gameplay

intuitive for new users through clear iconography.

Background

A study was done in 2017 named “Design and implementation of a gameplay UI for a

MOBA mobile game” which analyzed and compared a lot of mobile MOBA games through their

control UI layout and game rules. To learn more about the usability of mobile MOBA games, the

researcher built their own mobile MOBA game, changing the different game UI layout across

participants to gauge the user interface usability (Wahlroos).

This gives us a model about the user interface of mobile MOBA games preference. The

author came out that the left-bottom to right-up direction layout and all most all the button

arrangement is successful. This gameplay loop is reflected in Mobile Legends where the player’s

team always starts from the bottom-left corner of the map, fighting their way upward. This

research is mostly processed through playtesting and without an analyzation of players’ focus on

the screen. This gives a lot of information to us about a new way to research the game’s usability

through eye tracking software.

Methodology

For this research we are looking to analyze how intuitive the interface of Mobile Legends

is. So, for this reason we used the methods of survey, thinking out loud, and biometrics reading

using eye tracking technology.

To begin, we recruited 12 participants who had not played Mobile Legends before. The

participants were asked to directly start playing the game mode without going through a tutorial.

However, the players were briefed about some background game mechanics that we deemed

would be intrusive to the experience without prior knowledge. To prevent the participants from

getting overwhelmed by the difficulty of the game and to create a uniform experience across

participants, we made them play against the A.I, which is easier than fighting real players. Every

participant was instructed to use the same character, a ranged marksman named Layla. Layla is a

simple character that does not use many complex ability mechanics and is the character used in

the game’s actual tutorial. Participants were playing on I-Pad, as it is easier to track eye

movement on a bigger screen than a smaller mobile screen.

We used the Tobii Pro 2 eye tracking glasses to track participants’ eye movements. After

the game, players were asked to fill out a posttest survey which had questions regarding the

game and their attitudes towards different game features.

Analysis of the data:

To draw inference from the data collected through eye tracking we planned to focus on 7

Areas of Interest. These 7 AOIs were attack buttons, joystick, mini-map, item shop, player

character and scoreboard. After the AOIs were mapped the time spent on each AOI was

analyzed.

Figure 1: The user interface partitioned by the seven Areas of Interest for eye tracking

We got the results in percentage time spent on each AOI so there was no need of

normalizing the data. The result obtained were plotted into bar graphs with error bars using

Excel. The data from eye tracking combined with coded survey results acted as basis for the

claims that some features of UI were not so useful.

Results

For our results, we are focusing on player’s behavior and performance, and how they are

related with each other. In our research, there’s 12 people in total, they all have different

experiences on mobile games and MOBA games. Everyone will act differently because they

have different experience before. So, the core for our research is the relationship among present

behavior and afterward performance. We want to figure out if this game did a good job directing

player’s behavior when they have a various experience on mobile games and MOBA games.

Usability -> Behavior -> Performance

Before talking about the relationships between different objectives, the data’s quality is

important to know.

Performance

In general, we have 12 matches, and two players lost their game, so it’s about 83% of

winning rate. The play time for all matches is in the range of 8:43 to 15:18, it’s not very long.

Figure 2: Grouped averages of player match times.

Each match takes about 12:29 minutes on average to finish, it’s a fast game.

In an extreme situation, we have a participant who won the game without killing anyone and

finished the game in 14 minutes. In MOBA games, performance can be evaluated by kills and

deaths.

0 1 2 3 4 5 6 7 8 9

1

0

1

2

1

0

2

1

2 2

Number of Player Deaths

Parti

cipa

nts

Figure 3: grouped player deaths, excluding ineffective outliers.

The match summary turns out to have average separate kills and extremely focused

deaths. Almost everyone died 1 to 2 times, but there are three players that perform terribly bad

and got up to 11 deaths. Looking at the K/D ratio, we found players can be divided into two

categories: bad and good at this game.

<1 >10123456789

10

3

9

Kill-Death Ratio

Parti

cipa

nts

Figure 4: Grouped player average Kill-Death ratio. Greater than 1 is considered a successful

participant.

The kill-death ratio less than one can be considered as bad performance. When we look

through the data, we found there is a very obvious gap between these two categories. A large

percentage of the participants were learning the game quick enough to play well in the short time

limit.

Behavior

We can split behavior section into two datasets: eye-tracking and survey. They focus on

different perspectives and is recorded in different procedures. When players are playing the

game, eye-tracking is responsible for recording the real time reaction without consciousness.

After finishing the game, survey is for players to think about their behavior afterward with

consciousness and objectiveness.

Let’s talk about eye movement first. Along with the play process, they look at different

places. When they just started the game, they look everywhere to learn the UI and eventually

settle down for the game action in the center to upper right right areas of the screen. When they

are attacking, eyes will move to the target. By the end of the game, they became focused on

different objectives such as turret, enemy champion and jungle monsters, which means they

already know the UI layout and enjoy the game. The learning curve for this game is not very

steep.

Although we split players into two categories, it seems that everybody’s eye movement is

quite the same. In the methodology part, we drew 7 ROI regions, each region stands for different

places they look.

Figure 5: Averaged percentage time spent for player gaze in each of the Areas of Interest.

As we can see, the two concentrated part are battle area and hero herself, this means

participants use most of their time looking at the center of the screen. Also, people tend to not

look at the score board and joystick, maybe because they are well designed, and players can use

these features unconsciously. However, it seems not very good to use when it comes to the item

and ability UI. Their functions are not to show something to players, so they need to be very

intuitive and obvious to players.

Eye-tracking glasses are limited to know what players are thinking, so there comes the

survey. When players first enter the game, their impressions for it is very important. Almost

every people who had game experience before considering this game very familiar and like other

MOBA games, but those who have not played any games before found this game very frustrating

and hard. When playing the game, everybody feels satisfying when they managed a kill, but the

frustrating point for them varies.

As people who has played MOBA games before will know, the item system in MOBA

games is very complex and sometimes they are one of few core mechanics. However, this

game’s item system seems not simpler than PC MOBA games. Participants said they know the

shiny little button (which is item system) is important, but they don’t know what the use of that

is, they just pushed the button to make the button disappear because it would frequently obstruct

the action of the game, resulting in confusing moments

Usability

In general, people think this game is not very hard to start.

83% of participants consider this game easy to use at the first time and they can get their hands

on very quick. Although some participants complained about buttons, 66% of them still agree

that the button layout is very intuitive and clean. They feel very confident about this game after

the first round. Players indicated that while they would have preferred a tutorial, it was not

strictly necessary for this game.

Conclusion

Overall, players were learning how to play this game successfully. All but two

participants managed to win their games and end with positive KDAs. The system was intuitive

enough for participants to make a working concept as new players. All of our participants

generally agreed on what the game’s most successful and least successful components were.

They all enjoyed killing their opponents and felt most engaged when using their abilities and

fighting the enemy. All player’s reported combat as the game’s best feature and our eye tracking

analysis showed that players fixated either on the battle area or their ability button AOIs during

25% of their match. We conclude that this is the most effective UI element of the game because

of the intuitive iconography and player success. We can also conclude from the lack of gaze

fixation yet effective usages that the joystick and the attack buttons located at each thumb were

effective stand-ins to what might be perceived as a standard controller.

Conversely, players complained about frequent game notifications obscure a large

portion of the battle area, making confusion at crucial moments. Notifications obscured action,

sometimes causing deaths and confusion of what was happening in the game. Most of the

participants agreed that this was the biggest obstacle in learning to play the game. Our biggest

recommendation would be to move these notifications to unused left half of the screen or reduce

their size to support the combat flow state. As new players, the minimap was difficult to

understand and ping notifications were annoying, so both went largely went unused. The players

still experienced success despite not taking advantage of these features (that higher-level players

perceive as essential) because of the low challenge of the AI opponents. This is okay in the

context of our experiment because we were focused on observing the usability of the more

central interface features. In summary, we conclude that most of the interface features included

in Mobile Legends are intuitively effective for new players at an entry level, but still has room

for perfection in moving potentially distracting stimuli away from the action.

Discussion

It seems that across the board, the participants generally enjoyed the game and thought it

was easy to learn while playing. However, 92% of our participants indicated in their surveys that

they had at least minimal familiarity with the MOBA genre and its conventions. Even though

they had not played Mobile Legends specifically, most of the participants had at least a working

knowledge of what they were supposed to do in the game, potentially skewing our results of

what was effective in learning the game or not. If we were to conduct this study again in the

future, we recommend either recruiting players with minimal MOBA experience or have a

secondary pool of more experienced players. This will separate results to see both what

inexperienced and advanced players identify as important in-game information.

In addition, our eye tracking procedure was not as successful as desired. Area of interest

heatmaps could not be made because the participants would frequently move during testing

despite warnings. The recording eye tracking glasses and I-pad were in different and moving

positions for each participant. This made it very difficult to find reliable recording segments to

conduct AOI analysis, severely reducing the amount of usable gameplay footage. In a future

study, experimenters should either find a way to make participants stay motionless during the

study or conduct the study on a fixed screen where eye tracking AOIs can be captured more

reliably.

References

David Pinelle, Nelson Wong, and Tadeusz Stach. 2008. Heuristic evaluation for games: usability

principles for video game design. In Proceedings of the SIGCHI Conference on Human

Factors in Computing Systems (CHI '08). ACM, New York, NY, USA, 1453-1462.

Wahlroos, Valtteri, Design and implementation of a gameplay UI for a MOBA mobile game,

Aalto doc, May 2018

Appendix

Figure 6: The script read to participants before playing.

Figure 7: The posttest survey, two pages.