A Usability Review of

1

Verizon WirelessFamily Locator

A. Davis, C. Stern, D. EdingerK. Burgh, P. Raj, R. Waindle

2

• Executive Summary• Positive Findings•Usability Issues by Severity and Scope• Conclusions and Next Steps

Agenda

Executive Summary

3

The primary goals of the study were to:• Evaluate the intuitiveness and overall flow of the interface

• Evaluate the effectiveness of the terminology

• Gather first impressions of the application from test participants

• Determine which features are useful and which are not

• Identify additional features that would be useful to users

• Evaluate navigation within the application

• Identify areas of improvement and make recommendations

4

EXECUTIVE SUMMARY

Objectives

5

We studied ten participants.• Four participants were tested in the

Bentley usability lab• Six participants were tested remotely

Participants were asked to complete the following tasks:• Edit profile information (family

member’s name and avatar)• Locate family members• Find out how fast a family member is

traveling• Set up a Scheduled Location Update• Set up an Arrival/Departure Update• Set up a new Location

Participants were asked to follow a standard “think aloud” protocol.• Participants verbally expressed what

they were thinking as they worked through the tasks.

EXECUTIVE SUMMARY

MethodologyFor each task, we collected:• Usability issues encountered during

execution• Whether the participant was able to

complete the task• Whether the participant needed

assistance from the facilitator to complete the task

• Feedback from the participant while “thinking aloud” during the task

Participants also completedpre- and post-test questionnaires.• Results from the post-test assessment

questionnaire (SUS) are in the Appendix.

In addition, we conducted an expert review of the application.• Findings and recommendations have

been incorporated into this report

Note: We tested the live version of Family Locator, as the updated version was inaccessible due to technical issues.

EXECUTIVE SUMMARY

Recommendations

6

Replace the term “Update.”Users think “Update” refers to software updates, and therefore tend to ignore the Updates section. Consider using another term, such as “Alert” or “Notification.”

Move Update functions to the Family Members section.When asked to create an Update for a particular family member, nearly all users expected to find this function in the family member’s profile. To be consistent with the user’s mental model, Update functions should be accessible from family member profiles.

Clarify and simplify the procedure for creating a new location.Many users had trouble with this task, and the difficulties came in several forms. Consider ways to make this procedure more transparent and bullet-proof.

Reduce the reliance on wordy instructions.Some tasks, such as creating an Update, rely too much on text to guide the user. Consider ways to subtly guide the user with minimal instruction.

Consolidate and simplify the help system.Help text is spread across three systems – the Help pages, the Message Center, and the User Guide. These three systems should be consolidated into one. In addition, the help text should be reviewed for concision and clarity.

Label the avatars on the map with family members’ names.Labels would make it much easier for the user to identify family members on the map.

7

The Concept Itself• Parents liked knowing that their children were safe and

could be located easily.• Participants liked the idea of receiving notifications as to

their family members’ whereabouts.• Participants liked that they could send an email or text

message to a family member directly from the application.

Site Customization• The variety of map views accommodates user preferences.• Participants liked the ability to customize the avatar for

each profile.• Participants liked being able to view the locations of all

family members on the map.

Positive FindingsEXECUTIVE SUMMARY

Prioritization by Severity and Scope

8

EXECUTIVE SUMMARY

Usability issues are categorized according to:

Severity• High – Prevents or greatly hinders

task completion• Medium – Somewhat hinders

task completion• Low – Does not hinder task

completion but adds confusion or frustration, or does not conform to best practices

Scope • Global – Impact greater than a

single screen• Local – Impact limited to a single

screen

Issues by Priority

Severity Scope # of Issues

High Global 2

High Local 5

Medium Global 3

Medium Local 2

Low Global 1

Low Local 8

High Severity IssuesGlobal in Scope

9

Signed Out and Not Returned to Sign-in PageISSUEThe user is unexpectedly signed out of the application, and is not redirected to the sign-in page.

DETAILSThe user is unexpectedly signed out of the application. This can happen due to a system error or the session timing out. The user is then redirected to the Family Locator main page and must find the link to the sign-in page to sign in again.

RECOMMENDATIONSWhen the application signs a user out, it should display the sign-in page so the user can easily sign in again. Consider adding a session timer. The user should also be alerted a minute or so before the session times out due to inactivity. Consider lengthening the session time.

10

Updates TabISSUEUsers who wanted to set up scheduled and arrival/departure notifications did not see the Updates section as the place to perform this task.

DETAILSParticipants had difficulty figuring out where they could set up update notifications. Several needed assistance or were unable to complete the task at all. Most users ignored the “Updates” panel, assuming that it had to do with software updates; several noted that the amount of text dissuaded them from exploring that area. Many users looked first in the Profiles area, reasoning that since they wanted to set up a notification for a particular family member, they would be able to do so from the family member’s profile .

RECOMMENDATIONSChange the name of the Updates area. Add some pictures or iconography to replace or minimize the text. Allow users to set up notifications from family members’ profiles.

I was thinking Update was an update to the technology. I have an update to every single app on my phone. I don’t need this.”Notifications: where do I set them?”

11

“

“

High Severity IssuesLocal in Scope

12

“Unavailable” StatusISSUE“Unavailable” status is not entirely clear and potentially disconcerting. Selecting the profile button for additional information on the family member results in an error message.

DETAILSThe meaning of the “Unavailable” status is not clear, especially for new users. If the user attempts to get more information by clicking the family member’s profile, they receive an error message. With emotions possibly high due to the inability to locate a family member, displaying an impersonal error message can add frustration.

RECOMMENDATIONSTake into account the use case (a parent cannot find a child) and provide a better explanation of the reasons for and implications of the “Unavailable” status. Offer more information in the profile on what was last known about the family member.

“

13

Sam is ‘unavailable.’ I don’t like that. I don’t know what that means. . . If one of my kids was really unavailable and couldn’t be found, I would want something to jump out and tell me.”

Adding a Location

ISSUEWhen adding a location, there is no clear option for the user to manually enter an address. Also, there is no feedback when the user clicks Add Location other than a change in the button’s label.

DETAILSAfter users click Add Location, there is no indication as to what they should do next (i.e., click a location on the map or use the map’s search feature ). This caused significant delay and puzzlement.

RECOMMENDATIONSProvide prompts when Add Location is clicked to make it clear what the user needs to do next. In particular, make it clearer that the user has the option to enter an address manually.

14

Updates: Saved Location RequiredISSUEWhile setting up an Arrival/Departure Update, the user is informed that they should have first set up a location. The user must cancel the setup process (thereby losing the setup data they’ve entered so far), set up a location, and redo the update setup.

RECOMMENDATIONSNotify users before they embark on the setup that location data is required. Allow users to type in an address from the Update window.

That’s annoying.”

Adding location was unnecessarily complicated.”

I would say the system is starting to make me feel dumb.”

“

“

“

15

Update Request FailureISSUESetting up an update can fail if the family member’s phone is in use, off, or out of the coverage area. After failure, no information is saved and all must be re-entered when the user tries again.

DETAILSUsers expect to be able to add updates regardless of the phone status. Users may spend time setting up (through numerous steps) only to find that their data will be lost.

RECOMMENDATIONNotify users that phone status may preclude setup. Add a queue for failed Update requests so user data is not lost, or reprocess requests automatically and notify the user when successful.

If I can’t schedule an Update just because the phone is being used, that is a real problem… I’d expect it to push out the update even though somebody was talking on the phone. …It would be nice if they gave me a section of things I tried to schedule and couldn’t.”

“

16

Medium Severity IssuesGlobal in Scope

17

Help Text is Verbose and Not CentralizedISSUEUsers are resistant to reading large amounts of text in order to understand the features of the application. Furthermore, there are too many different locations where help is provided.

RECOMMENDATIONAvoid presenting users with text-heavy descriptions of options. Centralize help text to one location. Consider using rollover-style tooltips.

18

DETAILSHelp text is spread across the help pages, the Message Center, and the User Guide. In general, help and instructional text should be succinct and centralized.

Access to User GuideISSUEUsers faced challenges when attempting to access the User Guide for help with a task.

DETAILSUsers had trouble finding the link to the User Guide. When one user finally found the link and attempted to open the guide, an Adobe Acrobat error message was displayed. Since the User Guide opened in a new browser window, the user found it difficult to alternate between the guide and the application to follow the step-by-step instructions for the task.

RECOMMENDATIONSThe User Guide might be more useful if it were more integrated with the application. Since the application does not fill up the browser window, the space on either side of the application could be used to display task-specific help.

19

Saving Changes & ConfirmationISSUEUsers were unsure of how to proceed after entering new profile information. In addition, no feedback is displayed confirming that the entered information was saved. Finally, the Enter key does not save data.

DETAILSUsers had no trouble updating profile information, but did not realize that they needed to click the Save button at the bottom of the input box to save their changes.

RECOMMENDATIONSProvide user feedback after saving. Narrow the width of the dialog box so that the Save and Cancel buttons are not so distant from the data-entry fields. Allow Enter key to activate Save.

20

Browser Back ButtonISSUEThe browser’s Back button selection automatically initiates the phone location process.

RECOMMENDATIONMake phone location a process the user must explicitly initiate. Add programming so that when the user clicks the Back button, the current state of the session is preserved.

21

Medium Severity IssuesLocal in Scope

22

Searching for an Address for a New LocationISSUEThe user was confused by the presence of multiple search tools.

DETAILSWhen attempting to type in a street address to create a location for a notification, the user used the site-wide search tool at the top of the page instead of the search tool at the top of the map. The user was able to successfully complete the task using the correct search tool after receiving assistance from the facilitator.

RECOMMENDATIONSConsider a popup box to enter new location information vs. entry of information via a the map’s search tool. Change the wording in the instructions, which prompts the user to use the search tool “at the top of the page.”

23

Location Search Results Are Difficult to FindISSUEIt is difficult for users to distinguish between saved locations and the results of a search request.

DETAILSThe user identified a location using the map’s search tool and then moved the mouse pointer onto the map, where it showed a “Click to Place” prompt. The user did not understand why after entering an address they had to also select a location on the map. In addition, users failed to notice the search results listed in the Location panel.

RECOMMENDATIONSAfter search, display search results in a more prominent location. Disable the “Click to Place” feature while the search results displayed.

24

Low Severity IssuesGlobal in Scope

25

Unclear TerminologyAvatarThis term might be unfamiliar to some. Users are more likely to understand terms like “picture,” “image,” or “icon.”

HistoryThe History panel displays only location history and not, for example, message history. Change to “Location History.”

Message CenterThis feature incorporates frequently asked questions and the user guide. Its name causes confusion as to its purpose – users tend to think it has something to do with text messaging.

Refining/UnavailableThese terms are unclear.

26

Low Severity IssuesLocal in Scope

27

Difficulty FindingLocation-Related InformationISSUE Users didn’t realize that they could user can click the avatar to find more location information such as speed, current location, etc.

DETAILSUsers were unaware that they could click on an avatar to find additional information, such as the family member’s speed and location.

RECOMMENDATION Add a conventional “i” symbol on the avatar which would prompt the user to click on it for additional information. Also, display location-related information within the user profile.

28

29

Avatars Are Not LabeledISSUE Users can forget how avatars are assigned, and some avatars are very similar.

RECOMMENDATIONLabel the avatars on the map with either the names of family members or phone numbers by default, so that it is easy to distinguish family members on the map.

Edit Profile: Cannot Enter Full Name ISSUE The user cannot enter the full name of a family member.

DETAILSProfile names are limited to twelve characters. Users cannot enter names exceeding this limit.

RECOMMENDATIONSAllow more than 12 characters for the family member’s name or notify users of limit by displaying a sample (e.g. greyed text in field or sample text elsewhere).

30

Location Circles ISSUE The meaning or significance of the various circles surrounding family-member icons on the map is unclear. Their relationship to the Locate process is not obvious.

DETAILS Users assumed that the yellow circle referred to the possible location, but there was uncertainty about its significance. The distinctions among circles of different colors and line formats (solid vs. dashed) were not clear.

RECOMMENDATIONS Offer some additional cues to explain not only what the circles represent, but also why they are (or are not) displayed. If the Locate button status is linked to the display of the circle, create visual cues to unite the two.

Now I have yellow circles for some reason. I think they are here, but it still says Locating, so I am really not sure.”

“

31

Contrast Between Avatar and MapISSUE It can be difficult to differentiate some avatars from the background of the map, especially when the map was set to aerial view (e.g. green frog on green map).

RECOMMENDATIONSDifferentiate the avatar from the busy map background by adding a halo or drop shadow to separate images. Possibly make map background lighter.

It’s trickier to see… It didn’t jump right out at me initially.”

“

32

Speed and Movement / DirectionISSUEUsers are unsure of the accuracy and timeliness of the speed measurement – for example, does a 0-10 speed mean that family member is moving at all? They expressed a desire to see the direction in which might be traveling.

RECOMMENDATIONSProvide better feedback about how current the data may be (even though there is a timestamp next to this info, users often did not connect them). If possible, also show the direction of travel.

Taylor is at the same place as a few minutes ago, so I am not sure she is going that speed [30-40mph]? …[l would like to see] a track of where Taylor has been. Breadcrumbs.”

“

33

DETAILSUsers were skeptical about both the accuracy and the timeliness of the speed data displayed in the location information. They also expressed a desire to see the direction in which family members might be traveling.

Add Update Request:Two-Minute DelayISSUE Add Update can make users wait up to two minutes. After this delay, the process can still fail.

DETAILSUser have to wait several minutes before being informed of the status of the update. The user may wait two minutes only to find that the update was not successful and their setup data was lost. While the update is being processed, users are prevented from performing other tasks. Users expect the process to run in the background and then notify them of success or failure upon completion.

RECOMMENDATIONSIf possible, determine whether updates can be configured before users spend time on setup. Run the update operation in the background. Allow users to continue to use the application while waiting.

34

35

Conclusionsand Next Steps

35

36

Recommendations• Replace the term “Update.”

• Move Update functions to the Family Members section.

• Clarify and simplify the procedure for creating a new location.

• Reduce the reliance on wordy instructions.

• Consolidate and simplify the help system.

• Label the avatars on the map with family members’ names.

Overall, users liked the product.• Many users indicated that they’d be interested in using

Family Locator.

• Participants liked what the system can do and were sometimes surprised at the depth of functionality.

Users were able to learn the product with assistance.• If the most severe and moderate usability issues were

addressed, users should be able to set up and use the application without difficulty.

• With some simple changes to terminology, users should be able to learn the product without assistance.

37

Conclusions

38

Next StepsShort-term• Change the term “Update” to “Alert” or “Notification.”• Enable users to add a new location while they are creating

a new update.• Add a visible session timer or warn the user when a

session timeout is imminent.

Long-term• Move “Update” and other functions to the family member

profiles section.• Consolidate and simplify the help systems.• Prioritize and address the remaining severe and moderate

issues.• Conduct another usability study after changes have been

made.

AppendixList of Usability Issues

Tasks from the Usability Test

SUS Scores• By wireless carrier• Graphical representation

Participant Demographic Information

MindBridge Associates Contact Information

39

List of Usability Issues

40

LIST OF USABILITY ISSUES

Scope Severity Issue Slide #

Global High Signed Out and Not Returned to Login 10

Global High Update Tab 11

Global Medium Access to Help Guide 8

Global Medium Saving Changes & Confirmation 9

Global Medium Browser Back Button 10

Global Low Terminology 12

Global High, Medium, and Low Severity Issues

41

Scope Severity Issue Slide #

Local High “Unavailable” Status 14

Local High Adding Location 15

Local High Too Much Text to Read for Selecting an Option 16

Local High Updates Tab 17

Local High Updates: Saved Location Required 18

Local High Update Request: Failure 19

Local Medium Location Information Difficult to Find 21

Local Medium Searching for an Address for a New Location 22

Local Medium Location Search Results Unclear 23

LIST OF USABILITY ISSUES

Local High and Medium Severity Issues

42

Scope Severity Issue Slide #

Local Low Difficulty in Finding Location Related Information 25

Local Low Identical Default Avatars 26

Local Low Edit Profile Choosing Ariel or Road View 27

Local Low Profile Name: Cannot Enter Full Name 28

Local Low Location Circles 29

Local Low Avatar Background Differentiation 30

Local Low Speed and Movement/Direction 31

Local Low Update Request: Two Minute Delay 32

LIST OF USABILITY ISSUES

Local Low Severity Issues

43

Scope Severity Issue Slide #

Local Medium “Unavailable” Status 21

Local High Adding Location 15

Local High Too Much Text to Read for Selecting an Option 16

Local High Updates 17

Local High Updates: Saved Location Required 18

Local High Update Request: Failure 19

LIST OF USABILITY ISSUES

Local Low Severity Issues (continued)

44

APPENDIXTasks Presented to Study Participants

Participants were asked to suppose they had twochildren: Taylor (12 years old) and Sam (18 years old).

Completed, Without

Assistance

Completed, But With

AssistanceUnable to Complete

1 Change the name of the “781-879-3290” listing to match your name. Change the picture associated with the phone and add your email address. 10 of 10 0 of 10 0 of 10

2 Change the name associated with Taylor’s phone (“781-879-3291”) to “Taylor”; change the picture as well. Do the same for Sam’s phone (“781-879-3292”). 10 of 10 0 of 10 0 of 10

3 Use Family Locator to find out where Taylor and Sam are right now. 8 of 10 2 of 10 0 of 10

4 You know that Sam is right now driving home from a friend’s house. Use Family Locator to find out how fast he’s driving. 6 of 10 4 of 10 1 of 10

5 Taylor is taking a special drawing class at 3pm every weekday. To get to the class, Taylor walks by herself from the school to the arts center. You want Family Locator to notify you of Taylor’s location via text message at 3pm every weekday through the end of the year.

6 of 10 4 of 10 1 of 10

6 Taylor’s soccer team is at an away game. You want to be at her school to pick her up when the team returns, but you don’t know how long the game will last. You want Family Locator to notify you by text message when Taylor is within 1 mile of the school. The school is at 8 Forest Street, Waltham, MA 02452.

10 of 10 0 of 10 0 of 10

45

46

APPENDIXSUS Results: Bar Chart

Average Scores

47

48

APPENDIX

DemographicsWho is your current

wireless carrier?Which do you use more often - a Mac or a PC?

What kind of mobile phone do you have?

Do any of your children have mobile phones?

AT&T Both iPhone No

Virgin Mobile Mac Android No

Sprint PC Android No

T-Mobile PC Android No

Verizon PC Android No

AT&T Mac Sanyo No

AT&T PC Motorola No

Verizon Mac Feature phone Yes

AT&T PC Don't know Yes

Verizon Mac iPhone Yes