Usability Testing and Redesigning the Alcatel Lucent IP Touch 4018

Chameleon Innovation Group

David Güiza Caicedo - io1110532

Steven Fokkinga - io1096354

Marjolein Hartog - io1214322

Tine Lavrysen - io1186345

Meike Mak - io1171712

Amine Rhord - io1171860

 1

Index Executive summary 2

Exploring the IP Touch 4018 4

Visual aesthetics analysis 5

Researching the usability of the IP Touch 4018 6

Problem definition 7

Design specification 7

Design proposals 9

Redesign 15

Aesthetics of the redesign 16

Icons 17

Second usability study 17

Results from the usability test of the redesign 18

IP Next 20

Evaluation of design proposal 24

Strategic advice 25

Bibliography 26

Appendices 27

 2

Executive summary Introduction The goal of this project is to evaluate the usability of the Alcatel-Lucent IP Touch 4018 office phone, within the environment of the employees at the TU Delft (a technical university in The Netherlands), and to propose a re-design of the phone according to the findings of the performed tests. The Alcatel-Lucent IP Touch 4018 Product Description Manufacturer: Alcatel – Lucent Reference: IP Touch 4018 Description: Landline telephone terminal working on voice over IP technology, to be sold as part of a complete communications solution which includes the terminals and the network in which they work. Product functions The telephone, along with its underlying network, is capable of a very complete set of functions including (but not limited to): Voicemail service, Recent call log, Telephone book, ‘Hands free’ calling, Conference calling, Call forwarding (connecting the current call to a different terminal) and Call redirecting (redirecting an incoming call to a different terminal or external number). Usability research To find out how intuitive the IP Touch 4018 is and what usability problems emerge out of normal office use at the Delft University of Technology, a usability test was done. The test consisted of an online questionnaire, filled in by 50 employees of the TU Delft, an interview with four employees who filled in the questionnaire and a usability test with the physical phone and six participants who had no prior experience with the phone. General conclusions Most of the primary functions are not used, either because they are not needed in their average day work (such as forwarding a call), or because their use is too complex and the users do not know how to operate them (such as the phonebook). The phone is not sufficiently intuitive in most use scenarios. Often buttons are not understood at all or are misinterpreted and the menu structure is perceived as extensive and unclear. Voice feedback and text feedback in the display are needed while operating the menu, but these are often unclear or absent, and when present there is a general inconsistency in its use (sometimes voice is used, sometimes text). Audio and visible feedback (such as lighting buttons) is also unclear, inconsistent in use, or simply not present when performing some functions (such as dialing tones while calling a number). The phone’s looks and ergonomics are perceived as good enough and fitting the office environment, yet a more attractive visual aesthetic could be achieved.

 3

The Redesign The redesign of the phone offers a ‘downgrade’ from the IP Touch 4018, in the sense that it does not support as many functions as the IP Touch 4018, but just the basic functionality that could be used by an average office employee. It bears a basic and simple form, keeping in equilibrium straight and curved surfaces and shapes. The phone is black and the color blue is used as secondary color not only for decoration, but also for certain usecues. The functions are directly accessible through buttons with icons. The icons are selected on the basis of an online test with 257 participants. The buttons are grouped according to the moment of suggested or available use. This idea is further enhanced by the use of blue colored guiding lights surrounding each group of buttons. The guiding lights guide the user through the steps that he has to take in order to complete a chosen action. The redesign has a 30 character dot matrix display and text larger than 30 characters will automatically scroll. To set the phone in hands free mode, the user must place the handset sideways on the blue slot above the base (see picture). This way of putting the phone in handsfree is a more physical action and the user can direct his/her voice towards something in stead of a ‘closed box’. Usability research of the redesign To find out how intuitive the redesign is, a second usability test was done. Six participants tested a flash model and a wooden model of the design. General conclusion The usability test showed that the redesign is an improvement of the original phone. The functions which were put under direct accessible keys were faster reachable, the feedback was clearer and the blue guiding lights helped participants to look in the right direction. Also some new problems were found, though most of them are easily solved in the final design but some, like the icons used, need some further research. The final design: ‘IP Next’ Product Description: Reference: IP Next Description: Landline telephone terminal working on voice over IP technology, to be sold as part of a complete communications solution which includes the terminals and the network in which they work. Recommendations for Alcatel After the research we did to the IP Touch 4018 there are two main recommendations to Alcatel. The first is that usability problems are caused by the phone and not by the user: In an ideal situation a product provides no hurdles in use. The second is that things like consistency, sufficient feedback and small menus make products qualitatively better. The IP Next is a product that is not more expensive in production costs than the IP touch 4018. Its usability was confirmed by the people who used the prototype and also its looks made a good impression. The IP Next has no conventional looks, but it is still marketable as an office phone. It has the potential to replace the IP Touch 4018 and be the starting point of new design line and vision.

 4

Exploring the IP Touch 4018 Introduction The goal of this project is to evaluate the usability of the Alcatel-Lucent IP Touch 4018 office phone, within the environment of the employees at the TU Delft (a technical university in The Netherlands), and to propose a re-design of the phone, if considered appropriate, according to the findings of the performed tests. This report consists roughly of 2 parts, the first part is about the research that was done to the IP touch 4018, and the second part is about the redesign that was made. Alcatel-Lucent IP TOUCH 4018 Product Description Manufacturer: Alcatel – Lucent Reference: IP Touch 4018 Description: Landline telephone terminal working on voice over IP technology, to be sold as part of a complete communications solution which includes the terminals and the network in which they work. Product functions The telephone, along with its underlying network, is capable of a very complete set of functions including (but not limited to):

• Voicemail service • Recent call log • Telephone book • ‘Hands free’ calling • Conference calling • Call forwarding (connecting the current call to a different terminal) • Call redirecting (redirecting an incoming call to a different terminal or external number) • (A more detailed flow chart of how these functionalities are connected to each other can be

found as an appendix 2). Product Operation The telephone can be divided into the following operational parts:

The “menu navigation” section works as an interface for the different menus built into the system in order to operate the phone. The pre-programmed function keys include a shortcut to the phonebook,

Hand set

Menu navigation

Alpha numeric keypad

Pre-programmed functions keys

Programmable buttons

 5

voicemail, last call log and a help button. The programmable keys work as customizable shortcuts for any given functionality as a means of saving time with often used functions (e.g. forwarding a call to a specific number). The audio keys are used to control the volume of the handset and the loudspeaker, and to activate/deactivate the hands free function.

Visual aesthetics analysis People differ in cultural background, experience, knowledge and taste and therefore will also differ in what they consider visually pleasant. Nevertheless, there are some universal principles about what makes an object visually pleasant, which seem to apply for all human beings. We will discuss these aspects for the IP Touch 4018. Symmetry makes objects visual attractive. It is said that this has to do with the reduction of the cognitive load on people while ‘reading’ and ‘understanding’ the object, hence achieving a maximum effect for minimum means. The telephone itself is not symmetrical, but some symmetrical aspects can be found. People also like to look for patterns to create order and to see relationships. The ‘gestalt laws’ suggest that we perceive objects as well-organized, whole structures rather than as separated, isolated parts. People tend to classify objects into groups and have a representation of these groups in their head, in a process called proto-typicality. The IP Touch 4018 matches the general prototypical image that people have of a desk phone, and by doing so it is easier for the user to categorize it and this leads to a bigger chance that he will find it pleasant.

Ergonomics The IP Touch 4018 follows most of the suggested physical ergonomic properties that are expected from a device of this type:

• The buttons all have an adequate size and separation from each other, to ensure that the user will not have difficulties operating the keys or pressing buttons by mistake.

• The keys and display are also placed on a surface with an adequate angle in which the user can easily and effectively see them and operate them without having to change his body position (such as leaning forward on his chair or standing up).

• The handset is also of an adequate size and seems to pose no problems for people with different hand sizes and its shape does not seem to induce a hand posture that can lead to RSI (repetitive strain injury) problems.

 6

Researching the usability of the IP Touch 4018 To find out how intuitive the IP Touch 4018 is and what usability problems emerge out of normal office use1 at the Delft University of Technology a usability study was conducted. The results are used to propose a redesign for the phone. Method To get a good insight in the different problems users are facing, the usability study consisted of three parts. The first part was an online questionnaire among TU Delft employees to get more insight in which functions are used and with what frequency. Participants were also asked to rate the looks and usability of the phone. The second part included in depth interviews with long time users that had previously filled in the questionnaire. Participants were asked why they do not use specific functions and what problems they run into with everyday use. The third and last part of the study consisted of a usability test with first time users. Participants were asked to complete several tasks with the phone, which are common to normal office use. A complete description of the research questions and the method of the test can be found in the appendix 3. General conclusions from the research The main conclusion from the research is that the overall use of the IP Touch 4018 is perceived as complicated and it is not inviting to be explored. Most of the primary functions are not used. Two reasons exist for this. First of all, not all employees need all of the offered functionality in their average day work (such as forwarding a call). Secondly, the use of several functions is too complex and the users do not know how to operate them (such as the phonebook). Exact numbers of the proportion between these reasons are not present, but trough the in depth interviews it became clear that most employees do not understand how to use the majority of the functions. Other important conclusions are:

• The phone is not sufficiently intuitive in most use scenarios. Often buttons are not understood at all or misinterpreted and the menu structure is perceived as extensive and unclear.

• Feedback is required while operating the menu, but is often unclear or absent, and when present there is a general inconsistency in its use (sometimes voice is used, sometimes text).

• Auditory and visual feedback (such as lighting buttons) is unclear, inconsistent in use, or simply not present when performing some functions (such as dialing tones while calling a number).

The looks of the phone and its ergonomics are perceived as good enough and fitting the office environment, yet a more attractive visual aesthetic could be achieved. Recommendations after the research A general recommendation could be to limit the functionalities of the phone to those which are most often used, and to simplify the operation of those which are currently not often used due to their complexity (such as voicemail or the phonebook). By following the division suggested by Norman (1988) in which he makes a distinction in usage between (1) managing to do something (often through trial and error) and (2) actually knowing how to do it, it can be concluded that by simplifying the ease of use of the phone and providing the appropriate feedback to the user when needed, he will have a better experience with the phone and improve his productivity with it. A new styling of the phone could be implemented to enhance the pleasant visual aesthetic properties, which adds to a better user experience. Also some extra functionality has been suggested by the participants of our research: Employees want to track all their calls, not only the missed calls, but also calls they received and dialed and at what date and time this happened (much like a mobile phone). As they are on the same network, the link between the phone and the computer network should be more beneficial, for example by synchronizing the user’s outlook contacts directly with the telephone’s phonebook.

                                                            

1 We define “normal office use” as the average use by an employee, so we exclude people that use the phone as a part of their job description, like secretaries, janitors etc. We want to highlight that these employees use a different phone than the IP Touch 4018.

 7

The sending of DTMF tones is unclear. The term does not describe the function in a way that laymen can understand and the participants think that it either should always be enabled or it should have its own button. For a more elaborate overview of all usability problems from this test, please consult appendices 4 and 5.

Problem definition Several aspects of this phone are problematic. One of the most striking problems is the complex navigation for the majority of the functions, which is an important reason for the little use of those functions. Another problem concerns the feedback given by the phone. Often it is unclear, for example when the phone gives a description of a function. The phone also gives inconsistent feedback. Sometimes it is visual feedback on the display and sometimes it is voice feedback. The last major problem concerns the different modes the phone can be in. There is no clear feedback that the user changes mode, for example when the hand set is picked up. The problems with the phone concern several (groups of) users. First of all there is the everyday user (employee of TU Delft) who needs a user friendly communication system. Secondly, the company who bought the phones (TU Delft) wants a communication system that provides the best price/quality relationship to realize a pleasant and efficient working environment for their employees. The last party concerned is the manufacturer of the phone (Alcatel), who wants to improve their reputation in communication systems, and maximize their present and future orders in this area. Design Goal The design goal is therefore to create a telephone that forms a good interface between the user and the possibilities of a communication network. The focus will be on designing a physical telephone. Neither the network nor the concept of a communication system will be redesigned. In the process of designing new solutions, creating new problems should be avoided.

 8

Design specifications A list with the most important demands and wishes for the redesign was made. The full list can be found in Appendix 6 (Design Specifications), but below the highlights are described. As was concluded from the first usability test, the menu structure was too extensive and complicated. In the redesign, the maximum number of presented options must be four, and the maximum depth must be three levels (Menu > Sub-menu > Sub-sub-menu). The telephone should also be as consistent as possible in its required actions (and feedback) to complete all tasks (i.e. it should not require a certain action from the user at one step in a task, which it doesn't require at a similar step in another task). As the telephone will be used in an office environment, it is important that it can be used by a population as large as possible. Therefore, 99% of the office employees must be able to use the operational elements of the telephone. This means buttons, handset, visual feedback, auditory feedback etcetera should be perceived and used by 99% of the population of 20 to 60 year old men and women, left and right handed. Full specifications on this topic can be found in Appendix 7 (Physical Ergonomics Data). As the telephone will be sold globally, no static text (for example on the buttons) is allowed. Icons will be used where appropriate, but for icons, universal understanding is more difficult to achieve. Therefore the icons used in the phone must be comprehensible for at least 70% of the office employees, and in combination with text on the display, the comprehensibility of the icons should be at least 90%. To achieve a high score on the aesthetics, the layout of the telephone should comply as much as possible to the aesthetic consequences of the Gestalt laws of: symmetry, similarity, proximity, closure, continuity and common fate. In the second usability test, participants will be asked to rate the redesign, so a comparison with the Alcatel IP Touch 4018 from the first usability test is possible. A group of at least 6 participants should rate the looks of the telephone on average with at least a 4 on a 7-point scale and the usability on average with at least a 5. And 80% should agree that they would see the telephone fit in their office environment based on looks and based on usability. To assist the users in achieving their goal, provided use cues should guide (but not limit) users as much as possible in their use of the phone and whenever possible, the feedback on errors should contain a clear description of the error, the cause of the error, and the solution to the error.

 9

Design proposal In the early phase of designing, six concepts for a new phone were developed:

 10

 11

 12

 13

 14

From the six concepts we chose to further develop the concept called ‘Physics’. The idea selection can be found in the appendix 8. This eventually led to concept ‘IP Next’.

 15

Redesign Of all the concepts the ‘Physics’ concept was chosen, and we developed the following redesign:

 

 16

Aesthetics of the redesign Even though the new design is very different from the IP Touch 4018, this phone still doesn’t differ a lot from the prototypical image of a desk phone. This principal, called prototypically, is based on the principle that people tend to classify object into groups and have a representation of these groups in their mind. If the telephone come close to the prototypical representation people have of a phone it is easier to categorise it and people are likely to like it better. This also has to do with the familiarity principle which says that if you already have seen a desk telephone like this, you are more likely to like it. A principle that is related to this is the principle of most advanced, yet acceptable (MAYA). This principle says that if you develop a new product a familiar, typical product is most acceptable for future users. But, you want to make an innovative, advanced product so you should find a balance between this innovativeness and the level to which people will accept it. The redesign looks very familiar, because of 2 important typicality’s; the presence of a base with buttons and the presence of another product part placed on the base which has a wire which is connected to the base. These aspects make this phone familiar but it has a very advanced look. As mentioned before there are some universal principles about what makes an object visually attractive. When designing the redesign these principles were taken into account, to create a balanced and appealing product. The redesign was made symmetric, the buttons that were related were made a unity, and the design style was kept the same throughout the design. These things make the redesign appealing on a basic level, which apply to any user. Conclusion After this analysis you can conclude that the redesign is based on many of the aesthetic principles which make it easy on the eye and more appealing. This has been the starting point from the beginning, and it worked out pretty well at the end. The ratings given to the looks of the phone by the participants of the usability test can be found in appendix 9

Symmetry Good continuation

Similarity Proximity

 

 17

Icons Because the aim was to make the phone as universal as possible, the use of text on buttons was avoided. Using static text would mean that different phones need to be produced for different languages. This would mean a serious increase on the production costs. Therefore icons will be used to make several buttons and functionalities clear. In our previous research it was found that some of the icons used on the current phone are not clear and do not explain their functionality well. Therefore NEN-ISO 9186-1 (Graphical symbols - Test methods - Part 1: Methods for testing comprehensibility (ISO 9186-1:2007,IDT)) was used to set up an online icon test to judge the icons we have designed for the nine main functionalities (see appendix 10-A). Results 257 people from different countries filled in the questionnaire. The results of the questionnaire can be found in appendix 10-B. With the result of the questionnaire we choose the following icons to be used for the following functions:

Phone settings Automatic Redirect Do Not Disturb

Hold Forward Conference

Call Log Contacts / Phone Book Voicemail.

Second usability study Unlike the first usability study, only one type of research was used in the second usability study: a usability test. Because the second study was about a new design, no long term users could be contacted for a questionnaire or in depth interviews. The second usability test was held with first time users and was very similar to the usability test of the IP Touch 4018. However, several important differences were present. In the second usability test a physical, non-interactive, model was used next to an interactive model on a touch screen. Physical model

The physical model was used to test the visual impression, most physical ergonomics and the hands free function. Properties such as button displacement and sound effects were not present in this model. It is thought that the absence of a dial tone has influenced the comprehension of the hands free function.

 

 18

Interactive model

The interactive model on the touch screen was used to test the interaction and functionalities of the redesign. Most interactions were implemented; at least the ones needed for the test. When a participant wanted to use a function that was not implemented (such as ring tone or contrast), he was told that this function was not available in the prototype. A disadvantage of this interactive model is that it is not completely comparable with a full functioning physical model. A result of the usability test of the IP Touch 4018 was that participants randomly picked up the hand set while navigating through the menu. This is something that was not found in the second usability test, but can be ascribed to the use of the touch screen model. Participants sometimes even forgot to take up the hand set for tasks of which they knew the hand set should be picked up. Also the tasks in the second usability test were slightly different than the ones in the first test. A more detailed overview of the method and tasks of the second test can be found in appendix 11.

Results from the usability test of the redesign The usability test showed the strengths of the redesign. Although there were slightly more tasks than in the test of the Alcatel-Lucent IP Touch 4018, the overall time for the participants to finish them was nearly halved on average. There was also less frustration and only a few times participants expressed the need for a manual (which was not available) to find a specific function. The participants in the test of the IP Touch 4018 had on the other hand regularly consulted the leaflet-manual. The time for participants to reach the functions that alter the phone’s behaviour when in either passive or in-call mode (phone settings, automatic redirect, do not disturb, forward, on hold and conference calling), were found significantly faster than with the IP Touch 4018. The latter relied on a system of those functions in different menus, on different levels, or by direct access with the programmable keys (which were very little used by long-time users), which was in our opinion unclear, inconsistent and indirect. On the redesign, the icons on the direct access-buttons (the six buttons below the display) were in most cases recognized and used. Especially the icons and buttons for on hold, do not disturb, conference calling and phone settings were understood by almost all participants the first time. Forwarding and automatic redirect were harder to immediately understand, half of the participants used those buttons the first time, and the rest of them after some exploring. The blue guiding lights did help here to distinct the two, since the light around Forwarding is only available while calling and the light around automatic redirect only when the phone is not in calling-mode. There was a little bit more trouble with the button group which consisted of phonebook, call log and voicemail. The icon for call log was not always understood and sometimes mistaken for another function. However, after people got their missed call-task, in which the button lights up, they all understood its function. The other two buttons were also mistaken for other functions, but in a lesser

 

 19

extent. At the end of the test all participants could clearly distuinghish the buttons and icons in this group. The guiding lights helped most participants well in the intended way; all participants discovered its use either immediately or some minutes into the test. From that point on, whenever searching for a function, almost all the participants looked at the active lights first to see which buttons were available, to proceed in their process to complete the action. Four participants expressed that this was a very useful function. The feedback of the phone was much better understood than on the IP Touch 4018 (only 7 feedback-related problems against 34 on the IP Touch 4018). The wider, scrollable text display could give more information to users without resorting to abbreviations, and was not perceived as annoying or inconvenient. The handsfree function was not understood immediately by all participants, however two (out of six) participants got it right the first time (before any other tasks), two got it right after spending time with the digital model (which did not include any handsfree-related tasks), and the other two did not get it without explanation from the researchers. Even though this was a satisfactory, but not perfect result, the main premise of this design feature was not to have first time users to reach the handsfree function faster (since it was one of the functions on the IP Touch 4018 that participants did find easily), but to improve the experience of putting the phone in a position which was both physically different from the passive mode, and to give the user an opportunity to talk with an actual microphone and speaker, instead of with a ‘black box’. During this test it was hard to see if people would agree with this premise, since it is more of a satisfaction that comes with longer term usage. However, all participants that were asked expressed that they preferred the physical handsfree-functions over the button-handsfree of the original model. Five participants expressed they liked the looks of the new phone, two even very much so. One participant did not like the aesthetics and preferred a more conventional phone look. Half of the participants expressed that the buttons and the icons on the phone were quite big (compared to other phones), but none of them thought this was a problem. Besides these significant differences compared to the original phone, there were also some new problems that occurred with this design. Some of them were small design-related errors (which were easily fixable after the usability test – for the final redesign), some of them were prototype-related errors, because of the division in a physical model and a digital-cognitive model, and because of some limitations of the models, there were some problems that would not arise with an actual phone, and some were more fundamental problems with the concept (of features) into which some more research would be needed. The most occurred design-related problem was that the back-button for leaving menus and functions was not found in a lot of cases, which proved to be a problem with the icon on it (an arrow pointing backwards). With a more conventional icon this probably would not have been a problem in these cases. There was also some confusion over the use of numbers for menu items, since most participants expected that these functions were to be selected with the numerical buttons, which was not the case. Also, the use of the numerical-letter keys to select the first letter of a name in the phonebook was not supported in this design. The problems that were not so easily fixed included that not all participants understood that the phonebook-menu changed when they picked up the handset (when it is picked up or in handsfree, it goes straight to the call-contact menu, otherwise it goes to the preceding menu of add contact, edit contact, etc.). Some users had some problems understanding why the menu was different from the last time (this had also been the case with the IP Touch 4018). The observed problems from the usability test of the redesign were clustered into categories, which resembled the original clustering very closely. A list of all observed problems during the test can be found in Appendix 12. Conclusion

 

 20

The usability test showed the improvement of the redesign over the original phone. The functions were much faster reachable, the feedback was clearer, and the guiding lights helped participants to look in the right direction. The handsfree function was not unanimously detected, however, participants liked the idea of it. Some new problems were found with this design, though most of those are easily solved. Some others remain subject of further investigation.

 

 21

 

 22

 

 23

 

 24

A full overview of the final menu structure of the IP Next can be found in appendix 13

 

 25

Evaluation of design proposal The redesign was evaluated using the earlier stated design specifications which can be found in Appendix 6 (Design Specifications). The menu structure was significantly reduced, so it became less complicated to navigate trough. The largest number of menu options available in the redesign is four and the largest number of menu layers is three. The redesign has a menu setup with the least possible steps, therefore the most used functions are available after one step and the least used/less important functions are at a maximum of three steps. To further increase understanding of the navigation, every menu, sub menu and sub-sub menu item has an “>” sign in front of it for every level of the menu (> for menu, >>> for sub-sub menu). At the current state the prototype is as consistent as possible for the user. This is however debatable, because it cannot be consistent in every aspect and there is a balance to be found between consistency and redundancy. As the handset can be placed with the wire on the right or left hand side, it is optimal for both left and right handed users. The size of the hand set (in the middle, where the user holds it) is 90 x 44 x 9 mm. This size meets the ergonomic demands regarding grip circumference and hand breadth. The size of the smallest buttons is 18 x 12 x 1 mm. This size meets the ergonomic demands regarding forefinger breadth. The height of a Latin character on the display is 5 mm and the height of the smallest icon is 6 mm. These sizes meet the requirements regarding legibility. All ergonomic data can be found in Appendix 7(Physical Ergonomics Data). The comprehensibility of the current icons does not reach 70% as concluded from the online icons test. Further research is needed on the design and comprehensibility of the icons. Also the comprehensibility of the current icons in combination with text is not tested in a way that a valid percentage can be assigned. It cannot be said that the icons on the phone are as universal as possible. This is because there was not enough time for an extensive world wide test. As previously discussed, in the redesign the following Gestalt principles are present: symmetry, similarity, proximity, closure, continuity and common fate. In the usability test, the participants rated the looks of the telephone on average with a 4,83 on a 7-point scale and the usability on average with a 5,25. And 100 % agreed that they would see the telephone fit in their office environment based on looks and 100 % based on usability. This complies with the previously stated requirements. In the usability test most of the functions were clear after some exploration of the prototype, only the hands free function was not understood immediately by all participants. This is probably partly due to lack of auditory feedback (dial tone) on the prototype, because the cues that were supposed to guide the user (colour and shape), were noticed. In the usability test the light edges were commented on to be guiding and helpful. Furthermore, when a button is pressed with unavailable functionality a message is displayed with a description of, the cause of and the solution to the error, which was also perceived as very helpful. As an overall conclusion, the redesign meets the requirements stated in the design specification but on some topics, more research is needed. This includes an extensive, world wide test of the used icons to get a more valid result. There are also a few icons that need a redesign, and off course these redesigns have to be tested for comprehensibility as well. Furthermore, a test with a full working model to test the convenience and understanding of the hands free function is also needed.

Strategic advice

 

 26

Research This report contains a lot of results and insights, some of more importance then others for the future of Alcatel-Lucent. There are two parts to be separated in this search for information to benefit from, the research and its results and the solutions presented, from the designers mind. First of all it should be mentioned, that in the future it would be of great benefit if Alcatel Lucent started to conduct more thorough research on the usability of their products. There have been a lot of discussions about where the problem lies when a user can’t complete an action. Some of it may lie in the product, and some of it in the users’ hands, but we believe that it is possible to create a product that provides no hurdles in use. It should be the starting point in product design at any time, because the less time the user spends in frustration, the more time can be spend on appreciating the product at hand. Many of results of both usability tests are universal principles. Things like consistency, sufficient feedback and keeping menus small are conclusions that can be transferred into new designs. These things really make the usability and thus the quality of the products better. So it’s recommended that there will be more research about these topics and that they will be used in future designs. The IP NEXT The concept of the IP Next was originally to make a phone that isn’t more expensive than The IP Touch 4018, and would contain a few common parts (like a hand set, a one lined display and normal buttons). With these constraints it showed it was possible to create a phone with better usability, without making things too complicated. At the end of this project, there is no doubt that the redesign is an improvement. The IP Next is therefore an interesting product for the market. Not only its usability has been praised by the people who got the chance to use the interactive model, also its appearance has made a good impression. The IP Next can be seen as a model that’s has no conventional looks but still is marketable as an office phone. So the IP Next has the potential to replace the IP Touch 4018 and the IP Next can be a starting point of a new design line and vision. This vision contains simplicity and power that can be used by any user and this can be the foundation of a powerful campaign.

 

 27

Bibliography

- Griffeth, N., “Making a simple interface complex: interactions among telephone features”,

Conference companion on Human factors in computing systems: common ground, p.244-245, April 13-18, 1996, Vancouver, British Columbia, Canada

- Halstead-Nussloch, R., “The Design of Phone-Based Interfaces for Consumers”, Human Factors in Computing Systems (CHI'89), Austin, Texas, 1989, pp. 347-352.

- Resnick, P., & Virzi, R. A. (1993). “Skip and scan: Cleaning up telephone interfaces”. Proceedings of CHI ‘93, New York: ACM Press.

- John D. Gould , Clayton Lewis, “Designing for usability: key principles and what designers think”, Communications of the ACM, v.28 n.3, p.300-311, March 1985

- Jennifer Ferreira , Pippin Barr , James Noble, “The semiotics of user interface redesign”, Proceedings of the Sixth Australasian conference on User interface, p.47-53, January 30-February 03, 2005, Newcastle, Australia

- Norman, D.A., (1988).Design of everyday things. New York, NY: Currency Doubleday.

- Savage, P., Designing a GUI for business telephone users, interactions, v.2 n.1, p.32-41, Jan. 1995

 

 28

Appendices

 

 29

Appendix 1 – Phone Keys’ Identification.

 

 30

Appendix 2 – Functionality Flowchart

.

 

 31

Appendix 3 – Method Usability Test IP Touch 4018 Research questions What usability problems are the participants facing? Which role is played by form and functional product characteristics in the emergence of usability problems? To what extent are observed usability problems ephemeral? Are there usability problems which might not be obvious at first but that emerge after prolonged usage? Which primary functions2 of the phone are used in a normal office use at the Delft University of Technology? What do participants think of the phone in terms of aesthetics and ergonomics? Methods Online questionnaire: An online questionnaire was set up with questions for long time users of the IP Touch 4018 at the Delft University of Technology. This questionnaire was intended as a mean to acquire some general insights of the experience that these users had after working with the phone in an office environment for a prolonged period of time. This included enquiries such as: If and how often are the primary functions used? How do the different functionalities fit their normal office environment, and how would they rate it? How does the look of the phone fit their office environment, and how would they rate it?

two screenshots from the online test

Interviews with long time users: We performed several personal interviews with some long time users of the phone, in order to get a more in depth insight of the aforementioned experience of long time users. Usability test: We performed several usability tests with participants with no previous experience with the phone. This was aimed at assessing how intuitive the usage of the phone is for a novice user and what kind of problems he might face while performing some common tasks of an office environment, such as:

• Dialing a number • Forwarding a call to a colleague • Adding and removing numbers to the built in phone book

                                                            

2 We define “primary functions of the phone” as: calling, receiving calls, using voicemail, forwarding and redirecting a call, putting a call on hold, hands free operation, conference calling and the use of the phonebook.

 

 32

• Redirecting a call to another number while away • Basic usage of missed calls and voicemail functionalities • (The full list of tasks can be found as appendix) • We also took the chance to ask the participants for their opinion regarding the general • usage of the phone and its looks.

Participants All participants were between 20 and 65 years of age, and they can be divided as follows:

• 50 employees from different faculties at the Delft University of Technology, with an experience of minimum 1 month with the phone, completed the online questionnaire. (4 of these employees also participated in the interviews.)

• 5 participants with no previous experience with the phone took part of the usability test. • 1 long time user took part of the usability test.

Research results A) Some of the most important findings regarding the problems that the users face are:

• The phone often gives unclear or deceptive feedback or users sometimes don’t notice the feedback provided by the phone.

• Phone functionalities and buttons are misinterpreted. • It’s easy for the users to lose themselves in the complex menu structure. • Many users think the phone is too complex and therefore they stop trying to use certain

functions. B) Some of the most common problems regarding the formal and functional characteristics include:

• Misinterpretation of buttons. The icons represented on the buttons often do not reflect well the function they stand for (e.g. the ‘hang up’ button is often interpreted as ‘put on hold’)

• The layout of the buttons also plays a role in confusion, as for example the programmable keys are perceived as belonging to the telephone book button because they are close together and look like a list.

• The complex menu structure is seen as too extensive and often unclear. Also, the content of the menu changes if the handset is on or off its base, adding to the user’s confusion.

C) It was found that most of the usability problems are not ephemeral, as many problems that emerged with novice users in the usability test tend to continue being present with experienced users. In most cases, this is due to the overall complexity of the phone. It was also found that there are problems we did not discover during the usability test, which emerge after a prolonged usage of the phone, mostly dealing with missing functionality such as the ability to add a name to the programmed phone numbers or a proper call log with incoming and outgoing calls with time stamps. D) The majority of the primary functions of the phone are hardly used by the average users in the office environment at the Delft University of Technology, as can be seen on the following chart:

 

 33

From the interviews, we know some of these functions (namely voice mail and phone book) are not used because it is too hard to understand how they work. If they were easy to use, they would be used. E) Most of the users think that the look of the phone is basic and a little dull, but they think that those looks fit the office environment. As for ergonomics, the physical part of the phone is appreciated by most users. The cognitive part (the logic of its operation) is rated below average. This can all be seen from the following two charts:

 

 34

Appendix 4 – Main Cluster table, results per participant

 

 35

Appendix 5 – Graphical clustering of all usability problems

 

 36

Appendix 6: Design Specifications DEMANDS The redesign solution must be in the form of a physical telephone station. Usability test The redesign and/or elements thereof should be based on findings from the usability test. Specific functionalities If a (sub-) menu structure is used in the interface of the telephone, the maximum number of presented option is four. If a (sub-) menu structure is used in the interface of the telephone, the maximum depth is three levels (Menu > Sub-menu > Sub-sub-menu). The steps necessary to complete very basic tasks like calling someone and picking up an incoming call should be understood immediately after start (of the task) by all novice users in a usability test. All other functions (forward, conference, hands free, hold, automatic redirect, do not disturb, phone settings [like language settings], phone book, voice mail, call log, etc.) do not have to be understood immediately the first time, but should be clear after exploration of the telephone. At consecutive times all other functions should be understood faster than the first time. Physical ergonomics Appendix 11: Physical ergonomics data P0.5-P99.5 of the user population should be physically able to use the operational elements (e.g. buttons) on the telephone, if applicable. 99% of the user population should be able to perceive the visual feedback given by the telephone from 75cm distance. 99% of the user population should be able to perceive the auditory feedback given by the telephone, at 1m distance from the source when using the hands free function (when applicable), and at 1cm distance from the source when using the handset (when applicable). Cognitive ergonomics No static text (for example on the buttons) is allowed on the telephone. The icons used on the phone should be comprehensible for at least 70% of the office employees. In combination with text on the display the comprehensibility of the icons should be at least 90%. Novice users must be able to understand the auditory and visual feedback given by the telephone, without the use of an instruction manual. After getting feedback, 80% of novice users should be able to understand what the desired action is to proceed in completing the specific task. Rating A group of at least 6 participants should rate the looks of the telephone on average with at least a 4 on a 7-point scale, when confronted with a physical model and/or rendering of the design. Based on how it looks, at least 80% of these participants should state that they would see the telephone fit in their office environment A group of at least 6 participants should rate the usability of the telephone on average with at least a 5 on a 7-point scale, after doing a usability test with it. Based on the usability, at least 80% of these participants should state that they would see the telephone fit in their office environment

 

 37

WISHES Menu structure If there is a distinction between functions that are easier to find and faster to reach (less steps), and functions that are harder to find and take more steps to reach, then the most used functions should be assigned to the first category, and the least used functions to the latter category. When confronted with a navigational menu-structure, users should be able to tell which (sub) menu they are currently in. Cognitive ergonomics Whenever possible, the feedback on errors should contain a clear description of the error, the cause of the error, and the solution(s) to the error. The feedback given by the telephone should be activated as quickly as possible after the action that triggered it. Provided use cues should guide (but not limit) users as much as possible in their use of the phone. Consistency The telephone should be as consistent as possible in its required actions to complete all tasks (i.e. it should not require a certain action from the user at one step in a task, which it doesn't require at a similar step in another task). The telephone must give auditory and/or visual feedback that is as consistent as possible throughout the use of the phone. The icons used on the telephone should be as consistent as possible. The icons used on the telephone should be as universal as possible for world wide understanding. Aesthetics The layout of the telephone should comply as much as possible to the aesthetic consequences of the Gestalt laws of: symmetry, similarity, proximity, closure, continuity and common fate.

 

 38

Appendix 7- Physical ergonomics data Physical measures (Source: www.dined.nl) For the following data the data set `Dutch adults’ population `DINED 2004 (20-60 years)’ is used. To serve as many people as possible, the physical properties of the telephone should comply with the P0.5 of women and the P99.5 of men. Therefore several measures are summed up below so the final design can be tested to those. All measures are in mm.

Measure Telephone property Female P0.5 Male P99.5

Hand breadth without thumb Hand set length 70 104 Forefinger breadth Button breadth 11 23

For the handset length, the hand breath is too wide, because fingers are not as wide as the widest point of the hand. Therefore, a handset length of 90 mm will be sufficient. For the button breath, the total of the button plus open space next to it should be minimum 23 mm, so no other buttons can be touched accidentally. For the grip circumference no data is available in the population `DINED 2004 (20-60 years)’. Therefore the grip circumference of the population `DINED 2004 (20-30 years)’ is used. To show this is legitimate the hand length of both populations are compared.

Measure Population Female P0.5 Male P99.5

Hand length DINED 2004 (20-30 years) 157 225 Hand length DINED 2004 (20-60 years) 158 224

The above hand length comparison justifies the use of the grip circumference of the population `DINED 2004 (20-30 years)’.

Measure Telephone property Female P0.5 Male P99.5

Grip circumference Hand set circumference 31 53

Legibility (Source: Ergonomics of human-system interaction - Part 303: Requirements or electronic visual displays [ISO/DIS 9241-303:2006,IDT]) The system should be able to show a Latin character height of 20-22 minutes of arc. With an in the demands specified distance of 750 mm, the character height should be at least 4.4-4.8 mm. The height of the icons should be at least 20 minutes of arc, but 25-35 minutes of arc is preferred. With a distance of 750 mm, the icon height is preferred to be 5.5-7.6 mm. Audibility An average spoken conversation on a distance of 1 meter has a loudness of 65 dB. (Dirken, Productergonomie). We will take this as a standard for the volume of the handset. Sounds are largely dependent on the environment they are in. A sound can remain unnoticed in a loud environment, so important information can be missed. But a sound can also be annoying the user in a quiet environment. Also user’s hearing capacities can vary. We don’t want to damage user’s ears, but users whose ears are already damaged should also be able to receive sound feedback. Therefore it is very important that the volume of the handset and the volume of the auditory feedback can be adjusted to the user’s personal needs.

 

 39

Appendix 8 – Idea selection Design The final design can’t differ too much of what is expected by the user: it must follow the MAYA-principle (Most Advanced Yet Acceptable) The layout of the telephone should comply as much as possible to the aesthetic consequences of the Gestalt laws of: symmetry, similarity, proximity, closure, continuity and common fate. Production The time required for the detailing of the design must fit into our planning. The final production costs may not be significantly higher than those of the current IP Touch 4018 model, unless it is justified by eventual advantages Use The telephone must be used in a network, so it should be centrally organized If a (sub-) menu structure is used in the interface of the telephone, it should be clear and not present too many options, where four is the optimal number. Evaluation of the concepts Concept LCD

This concept introduces a new way of arranging numeric buttons. The design follows the Gestalt laws very well. It is fairly easy to develop and produce. It has a very clear menu structure (full text, no abbreviations or icons) and is suitable for network use.

 

 40

Concept Projection

In this concept, the buttons constantly change meaning, which may lead to confusion with the user. It also doesn’t comply with some Gestalt laws like symmetry and continuity. Furthermore, the projection makes it more expensive than the IP Touch 4018 phone, and the development time would also be longer. But once used to the new menu structure, it is very clear, and it fits well into a network.

Concept Modular

The modular concept complies with Maya and Gestalt principles. It is also fairly easy to develop and produce. As the modules bring you directly to a certain menu, the structure is very clear. The largest problem is the logistics. Modules are very easily lost, and the problems for the administration are ample: How many modules does everyone want? Who has which module?

Concept Mobile

 

 41

The mobile concept is also very easy, but its largest problem is the network connection. As it is wireless, network coverage cannot be guaranteed in large office buildings. This will lead to frustration with the user.

Concept Physics

The Physics concept is very similar to the IP Touch 4018 model, but it adds to that a very intuitive way of putting it on hands free. Some research has to be done on the used icons, as they are the key to the user’s understanding.

Concept Touch screen

 

 42

The Touch Screen concept is probably the most futuristic one. But since it introduces a whole new way of interacting with the phone, it doesn’t comply with the Maya principle. As it uses a very large touch screen, the production costs will rise significantly.

Based on this comparison and after extensive discussion with Frank Blaauw (Alcatel-Lucent), we chose Concept Physics to be the concept we wanted to further develop.

 

 43

Appendix 9 - Results Final Usability Test – Ratings After the usability test, the participants were asked to rate both the looks and usability of the redesign they just worked with. They were asked to give a score on a 7-point scale. They were also asked whether they would see the phone fit into their office environment, based on the looks and the available functionalities. As previously specified, the rating for looks should score an average of at least 4 on a 7-point scale and the rating for usability should score an average of at least 5.

When they were asked whether the functionalities of the phone would fit their office environment, two participants made the remark that they would rather see it wireless. One participant remarked that it has all the necessary functions and is easy to handle. As she stated it herself: “No needless crap”. One participant noted that she thinks first time users will need some time to get used to all the buttons and their meaning. About the looks, everyone agrees it would fit into their office environment. Everyone agrees that black is a good color for the phone. One participant expressed a particular preference for the blue accents,

 

 44

but she also remarked that that she thought the screen and buttons looked more basic, less design than the rest of the telephone. When comparing this to the Alcatel IP Touch 4018, the ratings are significantly higher. The IP Touch 4018 scored an average of 3,52 on looks and an average of 2,94 on usability. With the rating of 4,83 the redesign scored 37 % higher on looks and with the rating of 5,25 on usability the redesign scored 78% higher! A small remark on this comparison is that the IP Touch 4018 results included scores from 50 long-time users, while the redesign was evaluated by 6 new users.

 

 45

Appendix 10 – Icons. A - Design proposals for the icons

A: Phone settings B: Automatic redirect C: Do not disturb D: Forward E: Conference F: Hold G: Contacts H: Call log I: Voicemail For our test the judgment test-variant of the norm was used, in which participants are asked to judge how suitable an icon is, by stating how large a percentage of office employees they think will understand the icon. Icons were made for nine referents: contacts, settings, hold, conference, automatic redirect, forward, do not disturb, voice mail, call log. Each referent is explained and on every page the assignment is repeated. The order in which the referents and icons appear is random for every participant. Below is an example of one page of the test.

 

 46

Figure X: example of online icon test question B - Results

Graph 1: Origin of the respondents

The following graphs illustrate the average percentage of office employees that would understand what the icon means according to the respondents of the online test.

 

 47

.

 

 48

Appendix 11 – Method of second usability test Six participants were invited to test our design in a lab at the faculty of IDE. The usability test was performed in a simulated office environment. The participants were be seated at a desk with a touch screen (under an angle of approximately 70 degrees) running the software prototype and the physical model on it. Behind the participant’s chair will be a video camera which will be focused on the physical model of the phone and the participant’s hands. First, our participants were welcomed with this text: “Welcome and thank you for your time! For the course Usability Testing and Redesign we are doing research on this telephone. We would like you to perform some tasks with this telephone so that we can see how this telephone functions. During these tasks you cannot do anything wrong. Take your time and please speak your thoughts out loud. During these tasks you will be recorded. These recordings will only be used in this course and you will remain anonymous. We kindly ask you to sign this declaration of secrecy and permission to film. At the end of the tasks we will ask you a few questions.“ Then the functioning of the touch screen was explained to the participants. Tasks were each given on a separate piece of paper. Participants were asked to read them out loud and the perform them. Tasks 1 to 10 were to be performed on the software prototype, task 0 was to be performed on the physical model. These were the tasks given to the participants:

• Task 0: What would you do to call hands free? • Task 1: Your phone is set up in Dutch at this point and you would like to change it to English so

that you can understand it. Can you please show us how you would do this? Task 2: Look at the telephone, what do you think happened? (By whom and at what time were you called?)

• Task 3: You now want to add John Smith, your colleague, to the built in phone book of your phone. Could you please show us how you would do that? His phone number is 015-2785621. Could you now show us how to call John using your phone book?

• Task 4: You want to call the following number 015-015-242980. Could you please show us how you would do this? Could you show us how to add your colleague John Smith to the conversation as well?

• Task 5: In a minute you will be receiving a call from us and after answering, you want that particular conversation to be continued by your colleague John Smith. Could you please answer the phone and show us how you would send the current call to John’s phone.

• Task 6: Could you show us how to remove John from your telephone book? • Task 7: You will be absent for a few days and you would like your incoming calls to be sent to

your mobile phone so you can answer your calls anywhere. • Can you show us how you would do this? • Task 8: Imagine you are very busy and you don’t want the telephone ringing to disturb you.

How would you do this? • Task 9: Look at the telephone, what do you think is going on here? (How would you listen to

this message?) Task 10: You want to know what the current welcoming text is for your voicemail service, how would you do that?

After they finished this, some specific questions on the test with the software model were posed, like: “You clicked 5 wile in the telephone book menu to access “J” faster. This doesn’t work, do you miss that function?”.

 

 49

Then the following questions about the physical model were posed: What do you think about the size of the buttons?What do you think about the size of the handset? Do you like the feel of the handset? What do you think about the angle of the base?

During the usability test some bugs appeared and were fixed between participants. After the first participant, we also decided that we should add a part where we test whether participants can put the phone on hold. We didn’t include that in the original test. We changed Task 5 to: “In a minute you will be receiving a call from us and after answering, you want to put it on hold, and afterwards you want that particular conversation to be continued by your colleague John Smith. Could you please answer the phone and show us how you would send the current call to John’s phone.” As too many participants had problems with the voicemail button, we also decided between test sessions that the voicemail button needed a coloured light as well, and that the text “Please pick up the handset first” should be displayed when this was not yet the case.

 

 50

Appendix 12 Following is a list of the most important problems which were observed multiple times during the usability test. Please refer to the graphical clusters and the problem-table for a more elaborate overview of the observed problems.

• The back button (icon) is not understood. • Some participants press numbers instead of “ok” when trying to access a menu option, because

the menu entries always start with a number. • Some participants press numbers when they are in the telephone book, because they want to

have fast access to a name (example: 5 for J). This function is not available. • Some participants confirm menus by clicking “ok”, while this is not necessary. As they press the

button, they are already in that menu, the name of the menu is only there to clarify the meaning of the button.

• Some participants confirm phone numbers by clicking “ok”, while actually the phone will automatically start calling, and no confirmation is necessary.

• If some participants want to call a contact from their phonebook, they do not understand that they should pick up the hand set first.

• The same problem occurs with voicemail. Participants do not understand they should pick up the handset before making a connection with voicemail.

• Some participants try to jump from menu to menu (partly because they don’t know where the back button is), but this function is not available.

• Some participants do not understand the hands free function. • Some participants found the handset to be too flat, not nice to hold in your hand.

 

 51

Head-clusters Clusters

Sub-clusters Name Short description

Occurrence

Sub-totals

clusters

Subtotals

head-clusters

FEED

BAC

K

1: missing feedback Missing feedback The user is looking for/expecting feedback, but there is none

2 2

7 2: unclear feedback

a Unclear text feedback The user gets feedback from the text-display but doesn't understand it

2

5

b Unclear usecue for handsfree (NEW)

The user gets insufficient feedback to understand the action to put the phone in handsfree mode

3

FUN

CTI

ON

S &

BU

TTO

NS

3: Unfamiliarity Unknown backbutton

The user does not notice/use the backbutton, so the navigation gets very hard

15 15

51

4: misinterpretation Misinterpreted button

The user misinterprets a button of the telephone and thinks it will lead him to the right place

9 9

5: unavailable functionality

a Switching functions while active

The user wants to use a certain functionality or property of the phone that doesn't exist

4

27 b

Pressing numbers for navigation

The user tries to navigate through a menu by pressing the numbers corresponding to the menu-entry number

16

c Pressing OK for phonebook

The user tries to access the phonebook by pressing the OK button

7

NA

VIG

ATI

ON

& M

ENU

S

6: tedious use

Using handset for navigation

The user does not know how to exit a menu or change a condition of the phone, so he picks up and puts down the handset to return to standby mode

11 11

51 7: wrong navigation a Wrong (place in) menu

The user tries to carry out a certain (sub)task, but is in the wrong place or menu to do it

23

35

b Wrong mode of phone The user tries to find a certain menu, but the phone is in another mode because the handset is either on- or off- the hook

12

8: random buttons Random buttons

The user is not sure how to finish a certain (sub)task and tries different buttons to see if they are helpful

5 5

MIS

C. 9: final product-

unrelated errors

User error The user makes an error which is not caused by the phone 3 3 6

Prototype bugs (NEW) The prototype of the phone contains a bug that will not occur in the final design

3 3

115

 

 52

 

 53

Appendix 13 - IP Next – Available functions and navigation structure

o VOICEMAIL [Button]* - (presented as a textual menu instead of a voice menu) List of messages

• [Chronologically organized set of voicemail messages with name/number, date and time ]

o Listen to the message o Delete the message o Call the person back

Welcoming message options

• Listen to current welcoming message

• Edit welcoming message o CALL HISTORY [Button]*

Incoming calls (name/number, date and time) Outgoing calls (name/number, date and time) Missed calls (name/number, date and time)

o PHONE BOOK [Button]* Add a new contact (name and number) Edit a contact Delete a contact Call a contact

• (choose a contact by using the navigation arrows)

• (choose a contact by using the dial pad to navigate to the starting letter of the contact)

o PHONE SETTINGS [Button] Language Ring tones menu

• Ring tone volume

• Change ring tone

• Ringing style Contrast Turn guiding lights off/on

o AUTOMATIC FORWARD OF INCOMING CALLS [Button] To a contact To a number To an item from the call history

o “DO NOT DISTURB” MODE [Button] On / off

o HOLD [Button] On / Off

o FORWARDING OF AN ACTIVE CALL [Button] To a contact To a number To an item from the call history

o CONFERENCE CALL [Button] With a contact With a number With an item from the call history

* There are however 3 types of scenarios where some of the buttons will react differently when pressed:

 

 54

1) When the hand set is picked up or hands free is activated and the user is ready to make a call, the “Phone book” button will let him choose directly a contact to call instead of showing the phone book menu.

2) When there is a missed call present and the user presses on the “Call history” button, he will be taken directly to the latest entry of the “Missed calls” section of the call history menu.

3) When there is a new voicemail message and the user presses on the “Voicemail” button, he will be taken directly to the latest entry of the “List of messages” menu.