Tag-It Snag-it or Bag-it Combining Tags Threads and Folders in E-mai

Tag-it, Snag-it, or Bag-it: Combining Tags, Threads, and Folders in E-mail

Abstract John C. Tang

IBM Research

650 Harry Road

San Jose, CA 95210 USA

[email protected]

Eric Wilcox

IBM Research

650 Harry Road


[email protected]

Julian A. Cerruti

IBM Research

650 Harry Road


[email protected]

Hernan Badenes

IBM Research

650 Harry Road


[email protected]

Stefan Nusser

IBM Research

650 Harry Road


[email protected]

Jerald Schoudt

IBM Research

650 Harry Road


[email protected]

We describe the design of bluemail, a web-based email system that provides message tagging, message threading, and email folders. We wanted to explore how this combination of features would help users manage and organize their email. We conducted a limited field test of the prototype by observing how users triage their own email using bluemail. Our study identified ways in which users liked tagging, threading, and foldering capabilities, but also some of the complex ways in which they can interact. Our study elicited early user input to guide the iterative design of these features. It also involved a user study researcher, designer, and developer in the field test to quickly integrate different perspectives during development.

ACM Classification Keywords H5.3. Group and Organization Interfaces: Asynchronous interaction—email

Keywords Email, web 2.0, threading, tagging, folders, web-based email, user study.

Introduction For many enterprises, email has become a mission-critical application. Knowledge workers spend several hours every day interacting with email, using it not only for communication but also for personal information

Copyright is held by the author/owner(s).

CHI 2008, April 5–10, 2008, Florence, Italy.

ACM 978-1-60558-012-8/08/04.

CHI 2008 Proceedings · Case Studies April 5-10, 2008 · Florence, Italy

2179

management, task management, and interacting with any of the corporate business processes now transacted via email [3]. Especially in a corporate environment, more research and innovation in email is needed to yield increased user efficiency and productivity.

Prior research on the use of enterprise email (cf. [17]) has shown that users employ a variety of strategies in using folders to manage their email. However, maintaining a folder structure is a difficult and time-consuming task. Recent developments in full text search capability have helped users find information in email more easily, but search does not replace the organizing and orienteering attributes of folders [7].

Tagging has emerged as a mechanism for casual organization of information in social networking services such as Flickr [5] and del.icio.us [2]. Tagging offers many desirable properties that make it easy to:

Assign multiple tags to a single resource

Create new tags on the fly with minimal cognitive burden on the user

Represent information indexed by tags through intuitively usable tag clouds

Furthermore, by providing a mechanism for sharing tags, social software systems can enable individual participants to leverage other users’ tagging efforts.

Despite the advent of Web 2.0 and its focus on social software and lightweight collaboration tools such as wikis or blogs, there has been only limited innovation in email. Google’s GMail [6] service stands out as providing a refreshingly different approach to email.

GMail’s view is based on the thread of conversational exchanges over time, rather than the more conventional view based on individual messages.

We were, however, surprised that GMail’s labels are markedly different from both the well-established folder metaphor and the new Web 2.0 concept of tags. It seems that people with established foldering habits will miss the organizing structure of their folders and the capability to move messages out of the inbox view. Meanwhile, proficient taggers will lack the ability to spontaneously create new labels without switching views out of the context of the current message.

In light of these trends, we took some new design approaches in the bluemail prototype. We use a threading model that is inspired by GMail but complement it with a conventional folder mechanism in light of the useful purpose folders serve for some corporate email users [7]. We also provide a tag-based mechanism for categorizing and retrieving emails as well as a tag cloud visualization that shows the list and frequency of tags used. This set of mechanisms enables a variety of behaviors, allowing users to organize email conversations into folders, switch to a purely tag-based model, or combine the two.

Related work Studies of the use of email have consistently found that users view email as more than just an interpersonal communication tool and develop personal and diverse habits around using email. Mackay’s [10] early study of email usage found that it was also used for time and task management. She found that the use of email was amazingly diverse, suggesting that email designers


2180

should offer flexible “primitives” that users can adopt and customize to their personal preferences of usage.

Whittaker & Sidner [17] described “email overload”—how email was also used for task management and personal archiving. They characterize three common strategies for dealing with email overload:

No filers: No use of email folders, relying on full-text search to find information

Frequent filers: Actively minimize number of messages in their email inbox by frequently filing into a large number of folders

Spring cleaners: Intermittently (typically every 1-3 months) clean out inbox into a large number of folders

Fisher et al.’s [4] more recent study of email added a fourth potential strategy of users who kept their inboxes small by filing into just a small number of folders. Furthermore, they concluded that email users do not tend to distinctly follow only one strategy, but may try a variety of strategies at different times.

Ducheneaut & Bellotti [3] described email as a hub for managing the diverse kinds of information received via email. Given the amount of time spent in email and the range of tasks accomplished in it, they characterized email as a habitat for information.

Wattenberg et al. [15] focused on email use in the enterprise. Beyond confirming more evidence for the idiosyncratic use of email, they also reflected on the challenges of introducing new features into email. Since it is such a critical application, it is important for email prototypes deployed into everyday use to be complete

and reliable enough to avoid any disruption of use. Developing new email features must often deal with software development constraints and integration with legacy email infrastructure and data.

These studies have identified several design features for improving the email interface. First of all, the diverse and idiosyncratic ways in which people use email suggest the need for iteratively eliciting real user feedback to guide the introduction of novel email features. Yet, the critical nature of email makes it difficult to try new features in actual use without completely implementing all of the functionality users expect in email. This challenge requires clever ways of integrating the software development process with eliciting realistic feedback on new email features.

Regarding specific email features to explore, grouping messages together according to conversational threads (history of messages on a related topic among a similar list of recipients) has been a recurring design implication [17, 15]. A thread-based view of a folder can use space more efficiently while at the same time giving the user additional context when focused on an individual message. GMail [6] uses a clean and consistent model of threads, rather than individual messages, as the basic organizing unit.

Meanwhile, popular social networking applications have used tagging as a mechanism for helping annotate and retrieve information. Tags are unstructured, one-word labels that users apply (through a lightweight interface) to some object of digital information. Studies of the use of tagging [1, 11] have demonstrated both the individual benefits in finding information and the social benefits of seeing others’ tags and collectively indexing


2181


Figure 1: The bluemail interface, showing views, folders, and tag cloud on the left and message list and preview in the center.

information. We wanted to explore how tagging might also be useful in managing information in email.

bluemail interface Figure 1 provides an overview of the main interface of the bluemail client. The layout follows a pattern common to many email clients with views and folders

listed on the left and a central content area with a message list on top and a message preview panel on the bottom. A button strip across the top of the list provides a set of standard email actions such as ‘New Message’, ‘Reply’, ‘Forward’, and, ‘Delete’. Bluemail also adds a tag cloud in the left pane to show the relative frequency of tags applied to messages.

2182

Bluemail is built using the open source Dojo Web 2.0 based toolkit and leverages AJAX techniques that dynamically update the web page without requiring a complete page refresh in the browser.

Design and development process The design and development process for bluemail has followed an agile practice as described by Wilcox et al. [16]. Major features are developed independently in sandboxes. Each sandbox is a complete instance of the bluemail system, running parallel to the main code branch. Changes and bug fixes are dynamically merged among sandboxes and the main branch. This allows features such as threading and tagging to be explored and tested in a fully functional instance that remains almost completely current to the ongoing development of the system. We have been able to support over 200 active users while still developing features, soliciting feedback and measuring actual usage along the way.

Threading and tagging in bluemail Message threads A thread is defined as the unique set of messages that result from the natural reply-to chain in email. Threads are often represented as a tree structure with the originating message acting as the root node in the tree. If a node receives more than one reply, the thread structure branches. For the purpose of this discussion, we will call one particular path from an end node in the tree back to the root a conversation, shown in Figure 2.

Threads are calculated by applying heuristics that combine both RFC 2822 [13] reply-to fields and subject field similarity matching. As messages are loaded either from local cache or from the network, the heuristics are incrementally applied, dynamically building the threads

and incrementally updating the display list presented to the user in a web-based client. Our approach contrasts with traditional threading algorithms [18] that calculate thread information in a monolithic, “batch” process. We chose this approach since JavaScript code and data structure support in the browser would not gracefully manage the load of processing thousands of messages in a single procedure. By processing messages in smaller increments, the main processing thread of the client is not locked for long periods of time. Incremental feedback to the user also provides a sense of progress.

Figure 2: This figure illustrates how a single conversation in a thread corresponds to the direct path from an end node (F) back up to the root node (A). Messages in other branches of the thread (such as C, E, and G) would not be represented in this particular conversation view.

In bluemail, threads are calculated against all the messages in a user’s email database. When viewing an inbox, folder, or other subset of messages, each entry is checked to see if it is part of a thread. The entire thread is displayed including messages that may have been stored in different folders. For example, if three messages that belong to thread1 are moved by the user into folder1, then the next time a reply to thread1 is received, the inbox will show thread1 as a thread of four messages that appear in the list at the order in time of the most recently received message. Figure 3 shows how threads are represented in the list view. Each thread is gathered and collapsed into a single row in the list. A thread icon column (see Figure 3A) is used to further distinguish threads from single message entries. A user can toggle the view between showing threads in the interface versus a flat list of messages by clicking on the icon in the column header. Hovering over the thread icon presents a tooltip-style popup listing all the messages in the thread (see Figure 4). The user can click on any message listed in the hover to view that specific message in the preview.


2183

list by toggling the icon column header, (B) new entries are shown in bold, d is too long, senders of earlier messages are elided to conserve space.

irst names of order they d with the

re the most ing to an essage is es sent by is replaced

in a thread conserve ot message

the list, which is preserved to maintain grounding, the oldest message(s) are elided.

When a thread in the message list is selected, the preview panel displays only the most recent message. But, since many email programs default to appending

replies, messages in a lier messages in the chain of ntents of each message is conversation view of that Figure 5. In this view, the top (in contrast to GMail, t the bottom). Earlier dded in the message are ding the sender’s name and directory, and set off by an r. We present this design as goal of presenting a messages in a thread (not sages with their embedded use the conversation view of st release to get initial user

ead icon tht.


2184

Figure 3: (A) Threads can be viewed in the and (C) when the list of senders in a threa

The Who column for threads contains the fsenders of the individual messages in the were received. The list of senders is orderemost recent on the left (unlike GMail, wherecent is rightmost). Any name correspondunread message remains bold until that mread (see Figure 3B). Additionally, responsthe user are included, and the user’s nameby the text ‘me’. When the list of sendersbecomes too long, the names are elided tospace, as shown in Figure 3C. After the ro

Figure 4: When the user hovers over a thrmessages of the thread are shown in a lis

in

conversation history duringthread typically embed earreplies. In bluemail, the coparsed and displayed as a single message, as seen inmost recent reply is at the where the most recent is acontributions that are embeseparated by a header inclupicture from the corporate alternating background coloa step towards our ultimateconversation view of all thejust parsing individual mesresponses). We wanted to individual messages in a fir

e

Figure 6: The tag entry field is located in the mail header of the message preview pane.

Figure 5: The conversation view displays a parsed message as a linear set of contributions, most recent at the top.

feedback from this study. This compromise allowed us to get user feedback before investing the time needed

Figure 6 shows a detailed view for the tag entry field. The field is situated in the header area for a message in either preview or full view mode. To tag a message, a user types one or more space-separated words into the field. Tags can be created spontaneously while viewing the message, without interrupting the user’s train of thought or current activity. As the user begins typing a tag, a list of matching suggestions appears beneath the field, allowing the user to select a previously used value. Upon hitting the return or tab key, the tags are applied to the message. Other elements such as the tag cloud and message list are also updated to reflect the new addition. When the user hovers over a tag in the message header, a small circle icon with an ‘x’ provides an affordance for removing the tag. Clicking directly on the tag text (here or anywhere a tag appears) switches the main content area to a filtered list view that shows all messages across a user’s mail tagged with the corresponding tag value. Figure 7 provides a view of how tags appear in the message list. Single messages (see Figure 7A) pre-pend tag values in a smaller font size and blue color to the beginning of the subject text. For threaded messages (see Figure 7B), all tags from the corresponding messages are aggregated and pre-pended to the front of the subject text. As a user tags messages, the values are aggregated into a tag cloud as shown in Figure 8. Tags in the cloud that are used more often relative to other tags are displayed in a larger and heavier weighted typeface.


2185

to develop a more complete thread viewer. Deleting and foldering a thread item from the list view results in a batch operation on all messages in the thread. If items were previously added to folders, they will be moved or deleted as a result of the user’s most recent action on the thread. Users can always switch from threaded view to individual message view for more fine-grained control of specific messages.

Message tagging The interface for message tagging consists of four main elements: a tag entry field and display in the message view, tag display in the list view, a tag cloud, and a message list view filtered by tag. Tags and folders coexist in bluemail; users may apply the same tag to messages contained in different folders.

We were interested in exploring the benefits of such a tag cloud in relationship to a user’s own email. Tag clouds are primarily considered to be useful in describing the topic or overall gist of a set of peer-generated data such as blog posts, web bookmarks, etc. [14]. We also hypothesize that the simple frequency metric used to weight the cloud might be better enhanced for email if it were to take into consideration other factors such as recency or social-networking relationship information. At this stage, tags in bluemail are private. When a message is forwarded or replied to, the tag meta-data does not propagate as part of the message. Recipients of the message cannot see the tags applied by other users of a message. We first focused on the use of private tags, with the intent to explore shared tagging after we gain further insights from user input. Limited field test of the prototype Since we were introducing a novel combination of threading, foldering, and tagging features into email, we wanted to get user feedback on the design direction as early in the process as possible. We hoped that these features would fundamentally change the way users interact with their email, but were wary of the challenges of introducing new features to such a critical

application that exhibited such diverse and idiosyncratic use practices [15]. We wanted to get user feedback that was both realistic in how they would actually use them and early in the design process so that we could iterate the design. We felt it was important to observe them using bluemail with their actual email data, but we did not want to wait until we had a fully functional prototype that could be deployed into everyday use.

Figure 7: (A) shows how tags are pre-pended to the subject column entry in the list, and (B) shows how tags are aggregated and displayed for all messages in a thread.

Figure 8: The tag cloud aggregates all the tags a user has applied to messages throughout his or her mail. Tags that are used more frequently are displayed in a larger and heavier weight font.

We conducted a limited field test of the bluemail prototype that was a hybrid of traditional empirical testing and walkthrough methods [8]. Our software development architecture enabled us to quickly implement working prototypes of specific features and plug them into a working, web-based email system [16]. While the working prototype demonstrated the new features we were adding, it did not have all the features of a full email client, and was not robust enough to be deployed for everyday use. But, we were able to load up the user’s email into the bluemail prototype and observe them using it to process their own email. In this way, we were able to go from designing the features to getting realistic user feedback on them in a span of about two months.

Methodology For the field test prototype, we recruited a diverse set of fifteen participants across different geographical sites, job roles, professional experience, and even native languages. This range of participants resulted from the involvement of people in three different roles in the bluemail project to conduct the study:

User study researcher—developed the field study to collect user input and identify design implications, based in California


2186

Designer—designed the user interface for the bluemail working prototype, based in California, but had recently worked in Massachusetts

Developer—architected and implemented the working prototype, based in Argentina

All three interviewers started with the same interview guide, and two interviewers were able to conduct two user studies jointly before doing the remaining studies independently. The techniques and results of those studies were shared with the third interviewer so that he could also follow a similar pattern. Each interviewer recruited participants from their own network of contacts at our large, multi-national company, yielding experienced email users with a variety of backgrounds.

Each study session was conducted in the participant’s typical working environment using their own computer. Before the study began, participants were invited to temporarily move aside any email that they did not want to be viewed as part of the study for privacy reasons. The participant’s email was loaded into the bluemail prototype. This meant that the participant was operating on their own email as currently organized into the inbox and any folders that they maintained and as currently marked with unread status or flags.

After collecting some demographic information about the user’s job role and number of years of professional experience, we then demonstrated the features of how bluemail handled threading, foldering, and tagging. We stepped them through the process of viewing and operating on threads, moving messages and threads into folders, and applying tags and filtering by tag.

We then observed the participants as they triaged and processed their email for about an hour. This activity included processing new mail that they had just received or cleaning out their inbox or other email folder. As they went along, we asked them to describe what they were doing as they processed their mail. From time-to-time, we would probe them about their interactions with bluemail. For example, we might check whether they realized that deleting or foldering a thread would act on all the messages of the thread, or probe if they would like to tag any messages, or ask how they would like tagging and foldering to interact.

In total, the study session lasted about 1½ hours. While most sessions were conducted face-to-face in the participant’s work office, three were conducted remotely using a telephone and NetMeeting for screensharing. All sessions were audio recorded to capture the users’ comments.

Results After all fifteen sessions were completed, the three interviewers compiled their notes to identify common patterns and interesting reactions among the users. Our observations focused on the following topics:

Usability of threading

How threading collections of messages interacts with the conversation view within a single message

How threading and foldering interact

How tagging and foldering interact

Usability of threading Several studies of email have proposed threaded views as a way to help users manage email [17, 15], spurring


2187

research on developing algorithms for identifying email threads (cf. [9]). Even though threading mechanisms have become available in several email clients, we are unaware of empirical studies of how people actually use threading in email clients.

In our field test of the bluemail prototype, we identified a few usability issues with (at least our implementation of) threading in email. As Figure 1 shows, threading does “compress” the view of the message list and allows a more efficient view of more messages. It also groups related messages together, which can make it easier to keep up to date with related messages that might arrive at different times throughout the day. Of the thirteen users who expressed a preference about threading, nine (69%) indicated that they would probably use the threaded view of bluemail to process their email messages.

Some users did not like the way threads grouped together not only received messages on the topic but also the messages the user sent as part of the thread. This behavior violated their model of the “Sent” folder as a separate folder from the inbox. This reaction was more common among users who had not experienced GMail, which has a similar threading behavior.

Some of the threading mechanisms can also interfere with established work practices of reading email. For example, in order to compress more information of the thread into a single line of the message list, bluemail abbreviates the sender’s name to just their first name (similar to GMail). However, two participants mentioned that this strategy did not scale up well to a large enterprise where many people can share common names. One participant pointed to a thread which

simply showed “Dave, Dave, Dave, Dave”, which actually consisted of an exchange of messages between two people who were both named Dave.

Furthermore, the entry for a thread would be sorted by the sender of the most recent message in a thread. Yet, when looking for a particular email message, many users look for it by remembering the sender of that message. Thus, if they were to look for messages by sender, either by scanning for it in the sender column or actually sorting by sender, it would be harder to find that message if it were presented as part of thread whose most recent message was sent by someone else.

For these reasons, several users liked that the threaded view could be toggled on and off (by clicking on the thread icon at the top of the thread column). They particularly contrasted that toggle feature with GMail, which always shows a threaded view of messages. One user in particular mentioned that the ability to toggle the threaded view on and off was a useful feature in migrating from a message view to a threaded view.

We also discovered that about one-third of the participants had a practice of deleting all but the most recent message of a thread. Some users would then file just that one message in a folder rather than the entire collection that composed the thread. While this strategy reduced the clutter and storage space that email threads could occupy, it required considerable manual effort that had to be updated as new messages came in on that thread. In light of the way bluemail moves entire threads into folders (and keeps threads together within that folder) two of these participants felt fine simply moving the entire thread into the folder without deleting earlier messages from the thread.


2188

How threading and conversation view interact In addition to collapsing threads of messages into a single entry in the message list, bluemail also represents embedded replies within a single message as a conversation view. Figure 5 shows how each embedded reply within a message is distinguished with a thumbnail picture of the sender and alternating shading of the background. Selecting a thread in the message list shows the most recent message of the thread in a conversation view.

While there is much overlap of information in the conversation view of the most recent message in a thread and the collection of all messages that compose the thread, there can be some subtle differences that can lead to user confusion. As users began to grasp the difference between the conversation view of the most recent message and the collection of messages in a thread, they realized that if there was a branch in the thread, they could miss that branch if they only looked at the conversation view of the most recent message. Referring back to Figure 2, the highlighted conversation with message F as the most recent message would not show the branches that included messages C, E, or G.

Users became concerned about the potential for missing messages, and suggested ways of detecting such branching in threads and highlighting messages that would not be included in the conversation of the most recent message. Some users began to question how threads were defined, revealing that subtle differences in the user’s conceptual model of a thread would lead to different user expectations. For example, if any branching in a thread resulted in generating a separate thread, that could make it easier for users not

to miss unique messages, at the cost of adding more clutter to the message list.

Based on this user input, we have since designed a conversation view for the entire thread (rather than just for individual messages). This thread viewer collects all the messages of a thread together in reverse chronological order. It also highlights unread contributions by displaying the sender’s name in bold in the message list and collapsing all previously read contributions when viewing the thread conversation.

How threading and foldering interact Watching how users moved email threads into folders raised several interesting issues about how those features interacted. A basic question was whether operating on a thread applied to all the messages in the thread, or just the message currently displayed in the message preview. This issue arose both in deleting threads or moving a thread to a folder.

In the study, we demonstrated that such operations on threads applied to all the messages in the thread. Consequently, when we observed a user operating on a thread, we asked whether they intended to delete (or folder) all of the messages in the thread. Although most users replied that operating on all the messages is what they expected, two users were unpleasantly surprised by that. We have since implemented a warning dialog that pops up when these operations occur on a threaded collection of messages (with the option of suppressing such warnings in the future).

Users also reacted to the way threads included earlier messages in the thread that had been already filed into a folder. Several of our users moved messages into


2189

folders to get them out of the inbox view. This practice was especially common among those who use their inbox as a “to-do” list and want to only leave messages in their inbox that are reminders to do some action on them. Upon seeing threads in bluemail, users wondered what would happen to earlier messages in the thread that they had moved into a folder. Would they re-appear as part of a thread when a new reply arrived into the inbox that was part of that thread? If so, that could disrupt users’ model of removing messages that they have already dealt with out of the inbox view.

That situation did not occur during the user study, since we just observed the use of email in one sitting. However, users’ encounters with a working prototype and their experiences of foldering messages and threads were sufficient to raise the question and elicit feedback on how to handle this situation. We described one possible design approach of showing all the messages contained in the thread, but indicating those messages that had been moved into a folder with that folder name. This approach would not add clutter to the message list, since each thread still only occupied one line in the list, but would provide feedback that the user had already filed earlier messages away. This mechanism also naturally suggests where to file the new message, based on prior foldering behavior.

We got mixed reactions to this potential design. Some users, especially those who were frequent filers, simply did not want to see messages that had been previously moved to folders “return” to the inbox view. Others thought that our proposed design would work for them. This is an issue that we will want to watch closely when we can do a long-term deployment into everyday use, and may have to be governed by a user preference.

Tagging and foldering Tagging was perhaps the most novel feature that bluemail presented to our users. Although everyone easily understood how tagging worked in bluemail from our explanation, 40% of the participants had not had any personal experience with tagging computer objects. Tagging got the most mixed reactions from the participants, as only seven participants (47%) said that they would probably use tagging. Six participants (40%) were unsure if they would use tagging or not, and two (13%) predicted that they would not tag at all.

For those that were considering using tags, most saw tags as complementary to folders. Four participants said they would continue to use folders, although two more said that they might not need folders in the future with tagging. Some saw a clear advantage of being able to apply more than one tag to a message (a limitation that can be frustrating with traditional email folders). Some even saw tags as a way of cutting across folders to allow grouping together messages from different folders by giving them the same tag. Others saw tags as a way to indicate sub-topics in a folder, especially those with many messages. As one frequent filer, who had an extensive folder structure, applied tags to messages, it appeared that tags served as “adjectives” to the folder names, which served as “nouns”. A few mentioned GMail’s lack of folders as one of the major frustrations that discouraged them from using GMail.

Tagging provides another orienteering mechanism, allowing people to pivot in and out of filtered views in order to locate messages. Also, tagging provides further grounding for search. Many of the participants noted that they would primarily use tagging to help with search, since they often file things away and spend


2190

a lot of time looking for them again. Tagging important messages should make them easier to find through a keyword search than orienteering through folders.

Once again, the common email practice of moving messages out of the inbox view by putting them into folders arose in imagining how they would use tagging. If they wanted the act of applying a tag as a way of indicating that they had processed a message and no longer had to deal with it, they would also want a view of “untagged” messages as an analog to the inbox that served as a to-do list.

But for those users who were not interested in using tagging, they mainly thought that the existing indicators of unread marks, flagging, and search were adequate in managing their email. They did not see the need for doing the extra work of tagging (or foldering, for that matter). Even though the interface of typing in a tag was very lightweight (moreso than the GMail interface for applying labels), these users (mostly no filers) did not want to expend the extra effort.

A couple other suggestions arose from introducing tagging into users’ email ecology of existing messages and folders. Several users wanted a “batch tagging” mechanism for applying tags to multiple messages. Since the current interface only allowed applying tags when reading a message, each message would have to be selected individually to be able to apply a tag. We have since enabled selecting multiple messages in the list and dragging and dropping them onto a tag in the tag cloud. This tagging gesture is consistent with selecting and dragging multiple messages onto folders.

Another suggestion was to leverage existing folders by automatically or implicitly applying the folder name as a tag to all messages contained in the folder. This would allow all searching and filtering mechanisms that took advantage of tags to also exploit the already existing name of the containing folder. Both ideas suggested the need to consider how to ease the migration from the existing structure of users’ email when introducing a novel mechanism such as tagging.

Mundane usability issue—keyboard shortcuts While not germane to the issues of threading, tagging, and foldering, one mundane usability issue that arose was the expectation of having keyboard shortcuts. Most users tried to navigate in bluemail using the keyboard shortcuts (arrows for moving the selection up and down in the message list and the <delete> key for deleting messages). It was interesting to note how widespread the use of keyboard shortcuts was in email. Perhaps because people spend so much time in email, it has been worth the investment to learn and remember the keyboard shortcuts for basic functions in email.

Reflecting on our field test Our limited field test of bluemail identified several design issues that shaped our ongoing work. Getting timely user feedback even before the entire design was complete confirmed our approach of iteratively testing just-implemented features with users.

Complexity of integrating threads, folders and tags Our field test revealed several ways in which threading, foldering, and tagging can interact in complex ways. While each feature clearly brings rich benefits on its own, combining them together can lead to some unexpected interactions. Perhaps this explains why


2191

prior email clients have yet to offer this combination of features. For example, since GMail does not offer folders, it can present a simpler user interface and conceptual model that avoids some of the complex interactions that we encountered.

From our field test, we realized that the interface has to communicate a clear model of threading, and especially help the user detect all new contributions to a thread. The conceptual model for threading was being confused by the collection of individual messages, each with its own conversation view. While we believe that the thread viewer that we have since designed helps clarify the model of threading, we plan study this further when we actually deploy bluemail.

Threading also surprised some users in the way that it can cut across several folders. Threads could end up displaying messages from the inbox, from the Sent folder, and from folders that the user previously filed earlier messages in the thread. Our redesigned thread viewer denotes if messages of a thread reside outside of the current collection in view so that users can both get quick access to the entire thread of messages but also maintain an accurate organizational model of where the messages are stored. Especially when introducing threading to an existing email ecology where users may have a considerable investment in a folder organization, it is important to be able to leverage that prior work in representing threads of new incoming mail.

The combination of tagging and foldering led to the widest range of reactions. Some users imagined using tagging to modify or complement folders. Two thought that tagging would replace their use of folders over

time and some imagined using neither tags nor folders. It seems like the users’ idiosyncratic email practices will lead to the full range of using tagging or not in conjunction with other email features. The interface needs to accommodate how tagging potentially interacts with other features.

We expect that offering a combination of these features (or “email primitives” as characterized by Mackay [10]) could enable rich new practices for managing email. In light of the idiosyncratic use of email, we expect users to discover which combination of features would best match their preferences in processing email. Thus, it is important to clearly communicate how various features might interact with each other. Our limited field test uncovered several interactions that might be surprising, so it is important to convey those clearly to the user. Furthermore, it should be easy to selectively turn on or off specific features, such as toggling the threaded view in the message list.

User study methodology Our limited field test affirmed the value of user testing even before the design was fully implemented. This approach allowed us to get user input early in the design process when we could still make fundamental adjustments. It was also important to use a working prototype operating on the user’s own email to elicit some of the reactions we observed. For example, working with their existing folders helped users notice potential interactions with both threading and tagging.

Our field test also involved different project roles in conducting the study. This diversity not only brought multiple perspectives to bear on the analysis of the


2192

data but also broadened each of our perspectives by observing users interacting with bluemail first hand:

User study researcher—While this is the role traditionally involved with conducting user studies, he found it valuable to understand what observations emerged as important to the other two roles.

Designer—While many usability practices intentionally avoid involving the designer in the user study, for fear of potential bias, our designer found it very valuable to be personally involved in the study. He found that it helped him separate the more superfluous design elements from the core design attributes that needed more focus. Participating in the user study helped him better communicate the important design ideas, and he likened it to the process of giving a talk or writing a paper as a way of crystallizing one’s work into a form that more clearly communicates to others.

Developer—He appreciated having a broader role in the project, and felt that his experience helped develop his intuitions on the priorities for future development work. He likened the experience to that of a parent watching their child perform in front of an audience. It was exciting to see the implemented features that the users liked, but painful to see them stumble on known bugs (or discover new ones). This experience heightened his sensitivity to how his work directly affects the experience users have with bluemail.

Especially since our software development process allowed features to be developed somewhat independently of each other, conducting a user study that integrated a combination of features together and involved the diverse perspectives of different project roles helped bring more cohesion to the project overall.

Future work We have already followed through on a number of design issues that our field test identified. In fact, among the three project members who conducted the user study, we have followed up with six of the original participants to show them the revised design for bluemail. While these interviews validated the major design changes around the thread viewer and operations on threads, they also identified new design and usability issues on which to iterate.

In the future, we would like to expand tagging, enabling users to apply tags when composing messages as well as reading them. We also want to go beyond privately applied tags to experiment with shared tags. Allowing tags applied by the sender of a message to be seen by the recipients may afford social convergence on indexing and organizing email among the correspondents. Sharing tags with others opens several interesting and challenging possibilities, including distinguishing between shared and private tags.

Ultimately, we want to deploy a fully functional version into everyday use and see how these features are used over time. In this deployment, users will be expected to discover how to use bluemail’s features through an introductory message (rather than the personal demonstration given in the field test). We are working through the engineering needed to make sure that the implementation is complete and reliable enough to sustain daily use of a critical application. We are also instrumenting it to log its own usage so we can study how patterns evolve over time. Especially given the diverse practices in using email, a full deployment is the only way to understand how these new features will be integrated into actual email practice.


2193

Acknowledgements We thank all fifteen of our anonymous users for taking the time to give us valuable feedback on the bluemail prototype. We also thank Steve Farrell and Tessa Lau for their input on many of these ideas.

References [1] Ames, Morgan, & Mor Naaman, “Why We Tag: Motivations for Annotation in Mobile and Online Media”, Proc. CHI 2007, ACM Press (2007), pp. 971-980.

[2] del.icio.us, “del.icio.us/about”, http://del.icio.us/about/, verified September 28, 2007.

[3] Ducheneaut, Nicolas & Victoria Bellotti, “E-mail as Habitat”, Interactions, Sept. + Oct. (2001), pp. 30-38.

[4] Fisher, Danyel, A.J. Brush, Eric Gleave, and Marc A. Smith, “Revisiting Whittaker & Sidner’s ‘Email overload’ Ten Years Later”, Proc. CSCW 2006, ACM Press (2006), pp. 309-312.

[5] Flickr, “About Flickr”, http://flickr.com/about/, verified September 28, 2007.

[6] GMail, “About GMail”, http://mail.google.com/mail/help/about.html, verified September 21, 2007.

[7] Jones, William, Ammy J. Phuwanartnurak, Rajdeep Gill, and Harry Bruce, “Don't take my folders away! Organizing personal information to get things done”, Extended Abstracts CHI 2005, (2005), pp. 1505-1508.

[8] Karat, Clare-Marie, Robert Campbell, and Tarra Fiegel, “Comparison of empirical testing and walkthrough methods in user interface evaluation”, Proc. CHI 1992, ACM Press (1992), pp. 397-404.

[9] Lewis, David D. & Kimberly A. Knowles, “Threading electronic mail: A preliminary study”, Information Processing and Management: An International Journal, Vol. 33, Iss. 2, (March 1997), pp. 209-217.

[10] Mackay, Wendy, “Diversity in the use of electronic mail: A preliminary inquiry”, ACM Transactions on Office Information Systems, Vol. 6, No. 4 (1988), pp. 380-397.

[11] Millen, David R., Jonathan Feinberg, and Bernard Kerr, “Dogear: Social Bookmarking in the Enterprise”, Proc. CHI 2006, ACM Press (2006), pp. 111-120.

[12] Nusser, Stefan, Julian Cerruti, Eric Wilcox, Steve Cousins, Jerald Schoudt, Sergio Sancho, “Enabling Efficient Orienteering Behavior in Webmail Clients”, Proc. UIST 2007, ACM Press (2007), pp. 139-148.

[13] Resnick, Paul, “RFC 2822 (rfc2822)”, http://www.faqs.org/rfcs/rfc2822.html, verified September 25, 2007.

[14] Rivadeneira, A.W., Daniel M. Gruen, Michael J. Muller, David R. Millen, “Getting Our Head in the Clouds: Toward Evaluation Studies of Tagclouds”, Proc. CHI 2007, ACM Press (2007), pp. 995-998.

[15] Wattenberg, Martin, Steven L. Rohall, Daniel Gruen, and Bernard Kerr, “E-Mail Research: Targeting the Enterprise”, Human-Computer Interaction, Vol. 20, Nos. 1 & 2 (2005), pp. 139-162.

[16] Wilcox, Eric, Stefan Nusser, Jerald Schoudt, Julian Cerruti, Hernan Badenes, “Agile development meets strategic design in the enterprise”, Proc. Agile Processes in Software Engineering and Extreme Programming XP 2007, Spring (2007), pp. 208-212.

[17] Whittaker, Steve & Candace Sidner, “Email overload: Exploring personal information management of email”, Proc. CHI 1996, ACM Press (1996), pp. 276-283.

[18] Zawinsky, Jamie, “Message Threading”, http://www.jwz.org/doc/threading.html, verified September 25, 2007.


2194

Documents

Tag-It Snag-it or Bag-it Combining Tags Threads and Folders in E-mai