Evaluation of Vibrotactile Pattern Design Using Vibrotactile ScoreJaebong Lee Seungmoon Choi∗

Haptics and Virtual Reality LaboratoryDepartment of Computer Science and Technology

Pohang University of Science and Technology (POSTECH)Republic of Korea

ABSTRACT

Despite the plethora of vibrotactile applications that have impactedour everyday life, how to design vibrotactile patterns efficientlycontinues to be a challenge. Previously, we proposed a vibrotac-tile score as an intuitive and effective approach for vibrotactile pat-tern design. The vibrotactile score is adapted from common mu-sical scores and preserves the metaphor of musical scores for easylearning. In this paper, we investigate the usability of the vibro-tactile score, focusing on its learnability, efficiency, and user pref-erence. Experiment I was to compare the vibrotactile score andthe current dominant practice of vibrotactile pattern design usingprogramming or scripting. The results gained from programmingexperts validated the substantially superior performance of the vi-brotactile score. Experiment II compared the vibrotactile score withwaveform-based design implemented in a few recent graphical au-thoring tools. Regular users without programming backgroundsparticipated in this experiment, and the results substantiated the im-proved performance of the vibrotactile score.

Index Terms: H.5.2 [Information Interfaces and Presentation]:User Interfaces—Haptic I/O; D.2.2 [Software Engineering]: De-sign Tools and Techniques—User interfaces

1 INTRODUCTION

Efforts to make good use of vibrotactile feedback date back to asearly as 1920s, when Gault envisioned to transfer speech into vi-brotactile stimuli for the hearing impaired [8]. Since then, vibro-tactile rendering has greatly expanded its application areas [7], forexample, confirmation feedback for the touchscreen [12, 15], in-formation delivery in the vehicle [23, 18], and user experience en-hancement for the mobile device [21] (all these references presentan extensive literature review on the respective topic). In particular,the recent adoptions of vibrotactile feedback in commodity prod-ucts have further stimulated the research interest of relevant fieldssuch as HCI, VR, robotics, and consumer electronics. Nonethe-less, the design process of vibrotactile signals, which inevitably re-quires repeated implementations and tests, remains the same. Themajority of researchers and designers still rely on either direct pro-gramming or an in-house waveform-based editor (see Section 1.1for details). Simple, intuitive, comprehensive means that can fa-cilitate vibrotactile pattern design can greatly contribute to relevantresearch and industry, but no de facto standards exist yet.

As an alternative, we previously proposed a symbolic designmethod adapted from musical scores [11], motivated by the sig-nificant similarity between sound and tactile vibration. In this ap-proach, low, signal-level details and high, metaphoric features aredecoupled. Power users can use all the functions with improved de-sign efficiency, while regular users without sufficient engineering

∗e-mail: {novaever, choism}@postech.ac.kr.

background can focus on the latter, analogously to music compo-sition using musical scores. In this article, we are now concernedwith the usability of the score-based vibrotactile pattern design.

1.1 Waveform-Based Editing

As described earlier, there has been increasing demand for softwaretools that can aid the design and assessment of vibrotactile patterns.Notable responses include the Hapticon Editor [6], Haptic Icon Pro-totyper [24], MOTIV studio (previously called TouchSense Studioand VibeTonz Studio) [9], and posVibEditor [17]. All of these ed-itors offer GUI (Graphical User Interface) for the user to directlyedit the waveform of an input signal to actuators, along with someadditional features. See Figure 1(a) for a conceptual illustration.

The Hapticon Editor and its upgraded version, the Haptic IconPrototyper, were developed for haptic icons to be played with aone degree-of-freedom force-feedback device (e.g., a haptic knob)[6, 24]. Even though the target attribute was a force profile, the twoeditors can be easily adapted for vibrotactile pattern design. TheMOTIV studio, a commercial editor from Immersion Corp., wasdeveloped for mobile devices [9]. It provides a template of pat-tern elements and a timeline interface in which the elements can becombined to make more complex vibrotactile patterns. It also has aconvenient feature of automatic vibrotactile pattern generation frommusic files in the MIDI format. A simplified version was includedin a Haptic Phone (Samsung Electronics; model SCH-W420), sothat the users could create their own vibrotactile patterns. In ad-dition, our research group released the posVibEditor, which hadseveral advanced functions [17]. For instance, it supports multiplevibration actuators, which are common in HCI and VR research,using a multi-channel timeline interface. The posVibEditor alsoprovides a unique design mode called perceptually transparent ren-dering [20]. The method enables to minimize perceptual deviationfrom the intended effect [16, 22] using a psychophysical magnitudefunction for vibration perception (e.g., see [21]).

These waveform-editing tools are analogous to a sound compo-sition program that allows the user to directly manipulate soundwaveforms. The designer can, and should, manually refine allsignal-level characteristics, such as carrier signal frequency, carriersignal shape, and amplitude envelope, for each vibration actuator.This low-level access provides the greatest flexibility in waveformshaping, but composing music in this way is far from being intuitiveor time-efficient. Moreover, the designer may suffer from improperunderstandings on the limited actuator performance and the result-ing perceptual consequences. This drawback is especially impor-tant for the editing software intended for regular consumers.

1.2 Score-Based Editing

To compose music or audio icons, we usually rely on a musicalscore, i.e., a sequence of symbols. Based on the same metaphor, weproposed, for vibrotactile effect design, to use a vibrotactile score,which represents vibrotactile signals using the symbols adaptedfrom musical scores [11]. Our vibrotactile score is based on a pianoscore with a few features borrowed from a guitar tablature. Using

231

IEEE Haptics Symposium 20124-7 March, Vancouver, BC, Canada978-1-4673-0809-0/12/$31.00 ©2012 IEEE

(a) Waveform-based design (b) Score-based design (c) Vibrotactile clef

Figure 1: Conceptual illustration of two vibrotacile pattern design paradigms. (a) and (b)+(c) result in the same pattern. See the text for details.

Figure 2: User interface of the VibScoreEditor.

the score, we can represent the desired pitch, strength, and dura-tion of a vibrotactile note, and compose “vibrotactile music” in thesame way as we do with musical scores. As shown in Figure 1(b),the pitch of a note is represented by its vertical location on the stafflines, and its duration is determined by its shape. We denote thestrength of the note by an integer inside its head.

How the score symbols are translated to actual vibration signalsis determined by a vibrotactile clef. Analogously to the musicalclef, the vibrotactile clef defines a mapping from the position andintensity of a vibrotactile note to the pitch and strength (or asso-ciated physical parameters) of a vibrotactile signal, respectively. Italso includes the tempo variable that stores the duration of the quar-ter note in seconds. The durations of other notes are scaled accord-ingly. For instance, the first quarter note in Figure 1(b) is on thesecond staff line from the bottom, so corresponding to 100 Hz ac-cording to the clef shown in Figure 1(c). Similarly, its amplitude 5is converted to 2.0 V, and its duration is 0.08 s (tempo 0.08 s in theclef). In this way, the vibrotactile score and clef shown in Figures1(b) and 1(c) result in the same waveform as Figure 1(a).

Note that vibrotactile clef design must take into account the dy-namic performance of an actuator. Thus, vibrotactile clefs are betterdesigned by experienced users or product manufacturers. Regularusers or consumers can easily design effective and favorable vi-brotactile patterns using vibrotactile scores based on predesignedvibrotactile clefs. This decoupled approach enables the designer toput more effort into determining the sequence of metaphoric sym-bols which deems most effective, leading to improved efficiency.

We embodied these concepts in a graphical editor named the Vi-brotactile Score Editor (VibScoreEditor). Its primary features are(1) an intuitive GUI, (2) decoupled structure of vibrotactile scoreand clef, (3) a run-time player for on-the-fly test, (4) support of dif-ferent vibration actuators, and (5) data management in the XMLformat. The VibScoreEditor also provides three more symbols:crescendo and decrescendo, which express gradual increases anddecreases of vibration strength, respectively, and legato, which in-dicates no pause between adjacent notes. Its screen shot is shownin Figure 2. Further details can be found in [11].

1.3 Paper OverviewWe expected that score-based design using the VibScoreEditorcould be intuitive and easy to learn, leading to the efficient de-sign of vibrotactile patterns even for non-experts. In order to as-sess the validity of this hypothesis, we experimentally evaluatedthe usability of the vibrotactile score in two user experiments. Theexperiments differed in target user population and the design meth-ods used for comparison. In Experiment I, the conventional designmethods, programming and scripting, were compared with editingwith the VibScoreEditor by expert users. In Experiment II, the twographical vibrotactile pattern authoring methods, waveform editingand vibrotactile score editing, were compared by regular users. Theresults of both experiments indeed demonstrated largely improvedperformance of the vibrotactile score in terms of learnability, effi-ciency, and user preference. The rest of this paper reports details ofthe two experiments followed by general discussion.

232

2 EXPERIMENT I

The sample user group of Experiment I assumed experts in vibro-tactile pattern design, who could provide authoritative and informedassessments. The design method used for the comparison was pro-gramming; it is still the dominant and most powerful method forvibrotactile pattern design. However, recruiting a large number ofparticipants with expertise in both fields was infeasible because vi-brotactile pattern design experts are still scarce. As an alternative,we used programming experts for this experiment. In addition, us-ing a simple script, instead of making a full program, was alsocompared. Scripting is a powerful option, especially in terms ofefficiency for programming experts.

2.1 Methods

2.1.1 Participants

Twelve participants (all male; 21-28 years old) took part in thisexperiment and were paid for their efforts. All participants wereseniors or graduate students with a computer science major attend-ing the authors’ institution. They had considerable knowledge andexperience in programming with the C language and in coding withHTML or XML, but no prior exposure to vibrotactile pattern de-sign. They also passed a simple screening test to confirm their fa-miliarity with basic musical scores and symbols.

2.1.2 Experimental Conditions

The participants implemented vibrotactile patterns using threemethods: programming with C, scripting with XML, and graph-ical editing with the VibScoreEditor. For the C programming, theparticipants were provided with a complete sample program that in-cluded all working functions, e.g., those for device initialization, si-nusoidal wave generation, and communication. They were allowedto modify, copy, and paste codes as much as necessary. For theXML scripting, templates of the XML documents used in the Vib-ScoreEditor were given along with example files. The VibScoreEd-itor defines two XML schemas for vibrotactile score and clef, re-spectively [11]. A separate playback program for XML documentswas also provided for testing designed vibrotactile patterns. Theparticipants also could select their favorite text editor. For graphi-cal editing with the VibScoreEditor, the participants began with anempty document. The preparation ensured that only the time nec-essary for the implementation and test of vibrotactile patterns wasmeasured. The time required to make the initial working codes wasexcluded for the C programming and XML scripting methods.

For each design method, the participants completed three tasks.Task 1 was implementing simple 5-second long compound tactonsthat consisted of three elements for action, object, and result, re-spectively, taken from [1]. The participants were given the graphi-cal instructions shown in Figure 3. No musical notations were usedsince that could have been advantageous to the vibrotactile score.The participants implemented two compound tactons (e.g., ‘movefile success’). No elementary tactons were used twice for the sameparticipant. One elementary tacton for action was selected as ‘cre-ate’ or ‘delete,’ and the other was selected as ‘copy’ or ‘move.’ Intask 2, the participants were asked to make a long vibrotactile pat-tern (13.6 seconds) by combining 42 short sinusoidal vibrations. Agraphical illustration similar to those in Figure 3 was used to definethe pattern. Task 3 was to compose vibrotactile patterns based onmusic. The participants were presented with the musical score ofa very popular Korean pop song (‘Tell me’ by Wonder Girls), andthey were asked to compose a vibrotactile music piece that matchedthe first four measures of the song. To help with the task, a fewguidelines were also given; a high-pitch note should have higherfrequency than a low-pitch note, a quarter note should be playedfor 0.8 seconds, and the first note of each measure should have alarger amplitude than the following notes.

Figure 3: Graphical instructions used in task 1 of Experiment I. Thelength and darkness of a bar denote the length and intensity of vibra-tion, respectively.

To illustrate the complexity of the tasks, graphical instructionsand example solutions for the C language and the XML preparedby the experimenter are included in the supplementary material ofthis paper. Those for the VibScoreEditor are presented in Figure 4.

To objectively declare task completion, all of the tasks involvedimplementing specified vibrotactile patterns rather than designingnew creative patterns. The latter, which corresponds to the actualdesign process, would require much more repetitions of implemen-tation and test. Thus, the differences in some usability metrics, suchas task completion time, may be significantly amplified in practice.

2.1.3 ProceduresThe experiment took place in three consecutive days per partici-pant. The experiment of each day consisted of a training sessionfollowed by three main sessions for the three tasks, all done usingone design method. During the training, the experimenter explainedthe design method based on a script, written and memorized priorto the experiment, to regulate the amount of knowledge providedto the participants. Then, as exercise, the participants implementedthree simple vibrotactile patterns that contained patterns useful forthe main tasks. After the training, the participants performed tasks1, 2, and 3 in the increasing order of expected difficulty. It usuallytook 60 – 90 min to complete the one-day sessions. The order ofthe design methods used on the three days were balanced across theparticipants using a Latin square to avoid any order effects [26].

The participants were closely monitored in all sessions. Theywere given a hint if they did not make any progress for more than1 min during the exercise and for more than 10 min during themain tasks. After the participant declared task completion, the im-plementation results were inspected. If errors were identified, theparticipant was asked to fix them. The task completion time wasrecorded after the participant fixed all of the errors.

After finishing all experimental sessions each day, the partici-pants filled out a questionnaire to assess the easiness to learn anduse, intuitiveness, and efficiency of the design method used in theday. After completing the entire experiment, they were asked fortheir subjective preference for each design method and each task.All of the questions were rated on a seven-level Likert scale.

2.2 Results and DiscussionThe experimental results are summarized in Figure 5, and the re-sults of the one-way ANOVA with the design method as an inde-

233

(a) Task completion time (b) Subjective metrics (c) Preference

Figure 5: The results of Experiment I. The error bars represent the Tukey multiple comparison intervals. A line above two adjacent bars indicatesthat the two design methods were not statistically different in the corresponding metric.

(a) Task 1

(b) Task 2

(c) Task 3

Figure 4: Example solutions of Experiment I in vibrotactile scores.

pendent variable are shown in Table 1. Figure 5(a) shows that theaverage task completion times of the C language condition were555–1097 s (about 9–18 min), those of the XML condition were377–578 s (about 6–10 min), and those of the VibScoreEditor con-dition were 228–487 s (about 4–8 min). The VibScoreEditor re-sulted in the smallest average task completion time for all tasks.The design method was statistically significant for the task com-pletion time in all tasks (Table 1). Tukey’s HSD test indicated thatthe task completion times were all statistically different, except fortask 2 in a comparison between XML and the VibScoreEditor (Fig-ure 5(a)).

The task completion times had large individual variances, espe-cially in the C language condition, due to the nature of program-ming. In particular, they were distributed between 545–1878 s fortask 2. In spite of the large variances, the vibrotactile score wasmore efficient than C programming for all tasks with a statisticalsignificance. The C programming required a 2.4 times longer com-pletion period than the vibrotactile score editing in task 1 whichwas the simplest, and 4.3 times longer in task 3 where a vibrotactilepattern was designed from the musical score. The efficiency gain ofthe VibScoreEditor is expected to be even higher in actual use when

Table 1: ANOVA results at significance level α = 0.05.Task completion time F2,22 p

Exercise 23.39 < 0.0001*Task 1 45.52 < 0.0001*Task 2 19.89 < 0.0001*Task 3 63.78 < 0.0001*

Subjective metric F2,22 pEasiness to learn 4.16 0.0293*Easiness to use 7.37 0.0036*Intuitiveness 5.60 0.0108*

Efficiency 7.59 0.0031*

Preference F2,22 pTask 1 5.26 0.0136*Task 2 1.83 0.1847Task 3 12.45 0.0002*Overall 2.49 0.1060

* Statistically significant cases are marked by *.

vibrotactile patterns are designed and tested repeatedly. It shouldalso be reiterated that all the pre-made functions were given for theC programming, which would not be always the case in practice.

The XML condition shared exactly the same data structure as theVibScoreEditor. The major difference between them was whethera text editor or the GUI was used to input data. The VibScoreEd-itor showed shorter average task completion times than the XMLcondition with a statistical significance, except in task 2. At theindividual data level, XML scripting had slightly smaller task com-pletion times in only 3 cases out of the 48 trials (12 participants ×(1 exercise + 3 tasks)). This suggests that the use of the score GUIimproves efficiency in vibrotactile pattern design.

It should be noted that even in the exercise, the VibScoreEditorresulted in significantly lower task completion times, indicating itwas easy to learn. Before the experiment, the participants werealready familiar with C programming and XML scripting, but notwith the VibScoreEditor.

In the subjective evaluation (Figure 5(b)), the VibScoreEditor ex-hibited the best ratings in all of the subjective metrics: easinessto learn, easiness to use, intuitiveness, and efficiency. The designmethod had a statistically significant effect in all the subjective met-rics (Table 1). For user preference (Figure 5(c)), the participantspreferred the VibScoreEditor for all tasks except task 2 and alsoin the overall rating. The effect of design method was statisticallysignificant in tasks 1 and 3 (Table 1).

The raw data of the subjective evaluation showed very large in-dividual variances. This seems correlated with the preference andskills of each participant. The participants were programming ex-perts who usually have strong habits and preferences that have de-

234

Table 2: Summary of verbal comments.

Advantages of the C language◦ Provides the maximum flexibility in pattern design.◦ Easy to learn and use since it is a very familiar language.Disadvantages of the C language◦ The code tends to grow very long.◦ All details including memories must be managed by a programmer.◦ It is difficult to debug.Advantages of the XML◦ Familiar text editors (e.g. Vim) can be utilized.◦ Errors can be easily found and fixed.Disadvantages of the XML◦ Many new keywords must be memorized.◦ Less convenient than the VibScoreEditor, and less flexible than the C language.Advantages of the VibScoreEditor◦ Easy to learn and use.◦ Vibrotactile patterns can be managed intuitively.◦ Errors can be easily found.◦ No need to be concerned with memory management and programming details.◦ Allows consistent parameter management via a vibrotactile clef.◦ Has many convenient features (e.g. legato).Disadvantages of the VibScoreEditor◦ A vibrotactile clef must be referred to frequently.◦ Takes time to learn.

veloped over the years. For example, one participant who was anexpert at HTML coding using Vim (a popular VI clone) exhibitedan extraordinary preference for the XML condition. After finishingtask 2, the participant reported that he felt using the VibScoreEditortook longer than using XML since he was new to the VibScoreEd-itor. In the measured task completion times, however, he spent 64more seconds for XML scripting. A few other participants showedsimilar responses; they thought that they spent more time in the Vib-ScoreEditor condition than in the C programming or XML scriptingcondition, but actually they did not. It is likely that these tendenciesled to the individual differences observed in the subjective ratings.

Verbal comments collected from the participants are summarizedin Table 2. Most of the comments agreed with our expectations.Noticeably, it was reported that in tasks 1 and 2 the VibScoreEditorwas inconvenient since they needed to refer to the vibrotactile cleffrequently to convert the properties of vibration to a correspond-ing note. This was because the vibration properties were speci-fied in terms of numbers for frequency, amplitude, and duration. Itseems easier for experienced programmers to copy and paste thenecessary text codes and then change several numerical parame-ters in them, which can the most effective strategy for coding withC or XML. We however note that such an implementation is notcommon in actual vibrotactile pattern designs. Instead, several pre-defined properties are combined for “design,” with little need toremember their physical definitions. For instance, to design tac-tile icons, [2] used three roughnesses and three rhythms, [13] usedthree waveform shapes, four frequencies, and three amplitudes, and[18] used three frequencies and two amplitudes. The vibrotactilescore and clef could be much more useful for making a sequence ofpredefined properties.

3 EXPERIMENT II

In this experiment, two GUI-based vibrotactile authoring methods,waveform editing and score editing, were comparatively assessedby common users without any prior experience in programming.

Figure 6: Modified posVibEditor for Experiment II.

Figure 7: Modified VibScoreEditor for Experiment II.

3.1 Methods3.1.1 Participants

Twelve participants (ten males and two females; 17-25 years old)took part in the experiment and were paid for their efforts. All theparticipants were undergraduate students enrolled in the authors’university. They had no prior knowledge or experiences of vibro-tactile pattern design. They had moderate skills in computer use(e.g. Microsoft Office), but no programming experience. The par-ticipants passed a simple screening test to confirm their familiaritywith basic musical scores and symbols. No participants had specialmusic skills. Besides regular school education, the participants didnot receive extra music training for more than three years.

3.1.2 Experimental Conditions

The participants implemented vibrotactile patterns using two graph-ical authoring tools: the posVibEditor for waveform editing [19]and the VibScoreEditor for score editing. The posVibEditor wasthe only waveform-based authoring tool that we had access to atthe source code level. In addition, it supports the vibration motor inwhich input voltage level determines both frequency and amplitudeof the resulting vibration. For instance, the simple line-connectedinput waveform shown in Figure 6 produces a rather complex si-nusoidal output of co-varying frequency and amplitude (like Fig-ure 1(a)) [21]. Hence, a vibration motor is considerably easier for

235

Figure 8: Instruction for task 2.

Figure 9: Instruction given as a waveform.

Figure 10: Instruction given as a musical score.

waveform design than other actuators, such as voice-coil or piezo-electric actuator, which require full input waveform composition.

The different functionalities of the two editors were balanced asfollows. In the posVibEditor, the multichannel timeline interfacefor multiple actuators was removed since the VibScoreEditor sup-ports only one actuator. Note that the resulting function, as shownin Figure 6, is equivalent to other waveform editors such as the Vi-beTonz Studio or “My Haptic”1. In the VibScoreEditor, only onestaff line was used (Figure 7) since vibration motors require oneinput attribute only. Using multiple staff lines without in-head in-tensities, just like the piano score, was an alternative. We chose theformer since it less resembles musical scores and thus is expectedto show worse performance than the latter. As a result, both editorswere simplified for comparison, but we intended a higher advantagefor waveform-based design.

The participants completed four tasks with each design method.Task 1 was essentially the same as task 1 of Experiment I. Thegraphical instructions in Figure 3 were slightly modified. The fre-quencies were removed, and the voltages were slightly adjusted forthe vibration motor used in this experiment. Task 2 was also similarto task 2 of Experiment I. The participants made a long vibrotactilepattern (12 s) as a combination of 27 short sinusoidal vibrations(Figure 8). The next two tasks were to examine the effect of in-structions given in different forms. In the literature, several nota-tions were used to describe vibrotactile patterns in the design step,e.g., detailed description [4], waveform [5, 14], and musical score

1See http://farm4.static.flickr.com/3248/2973623996 e82319a1c0.jpgfor a picture. This smart phone editor has been shipped with several hapticphones (Korean domestic models) made by Samsung Electronics.

Table 3: ANOVA results at significance level α = 0.05.Task completion time F1,11 p

Exercise 34.12 0.0001*Task 1 47.39 < 0.0001*Task 2 131.02 < 0.0001*

Waveform 23.43 0.0005*Score 28.11 0.0003*

Subjective metric F1,11 pEasiness to learn 0.65 0.4382Easiness to use 0.19 0.6742Intuitiveness 0.03 0.8742

Efficiency 3.88 0.0745Fun 1.35 0.2691

Preference F1,11 pTask 1 1.05 0.3283Task 2 5.03 0.0464*Overall 7.37 0.0201*

* Statistically significant cases are marked by *.

[2, 3]. Both tasks consisted of combining 12 short sinusoidal vibra-tions to make a 9.6-s long pattern (Figures 9 and 10). In the taskgiven as a musical score, we also instructed the participants to playa quarter note for 0.8 s and to make the voltages of four notes ineach measure 3.6, 1.2, 2.4, and 1.2 V, respectively. The participantswho used waveform editing solved the task given as a waveformfirst and then the task given as a musical score. The order for theparticipants using vibrotactile score editing was reversed.

3.1.3 ProceduresEach participant carried out the experiment on two consecutivedays. Six participants used waveform editing on the first day andscore editing on the next day. The other six participants followedin the opposite order. On each day, the participants finished themain sessions consisting of the four tasks in addition to a trainingsession. A vibration motor was used as an actuator. The other pro-cedures were the same as Experiment I.

The questionnaire was also similar. In particular, the participantswere asked to answer the questions based on the editing methodol-ogy itself and not on the functions of the two editors. As the targetpopulation was common users, the question for overall subjectivepreference was changed to be more practical: “How much do youlike waveform editing (or vibrotactile score editing) as a part of thefunctions in your mobile phone?”.

3.2 Results and DiscussionThe experimental results are shown in Figure 11. The results of aone-way ANOVA performed with the design method as an indepen-dent variable are summarized in Table 3.

For all tasks, vibrotactile score editing led to lower average taskcompletion times than waveform editing (Figure 11(a)). The av-erage task completion time of waveform editing and score editingwere in the range of 155–416 s (about 3–7 min) and 77–199 s (about1–3 min), respectively. The differences were statistically signifi-cant for all tasks (Table 3). Waveform editing took 1.8 times longerthan vibrotactile score editing for the simplest task (task 1), and 2.1times longer for the complex and long task (task 2). This suggeststhat as a task becomes more difficult, the difference in task comple-tion time may increase further. Even when the task was given as awaveform, score editing showed a significantly lower task comple-tion time than waveform editing. The task was given as a musicalscore showed a more evident performance gain.

In the subjective evaluation, score editing received better ratingsin all the subjective metrics: easiness to learn, easiness to use, in-tuitiveness, efficiency, and fun (Figure 11(b)). Although the ratingsof score editing were consistently higher, the difference was morenoticeable for efficiency. The differences due to the design method,

236

(a) Task completion time (b) Subjective metrics (c) Preference

Figure 11: The results of Experiment II. The error bars represent standard errors. The labels of waveform and score represent the tasks theinstructions of which were given as a waveform and a musical score, respectively.

Table 4: Summary of verbal comments.Advantages of the waveform editing◦ Easy to learn and use.◦ Intuitive and easy to read since it shows changes in vibration strength over time

using a graph.Disadvantages of the waveform editing◦ Frequent use of mouse is annoying.◦ Takes time to design pattern.Advantages of the vibrotactile score editing◦ Easy to learn and use.◦ Expresses vibration rhythm effectively and intuitively using musical notations.◦ Efficient since it expresses strength and duration concisely.◦ Interesting and fresh design method.Disadvantages of the vibrotactile score editing◦ Confusing to novice who is unaccustomed to musical score.◦ Difficult to read strength and duration simultaneously.

however, were not statistically significant in any metric (Table 3;albeit marginally significant for efficiency). In addition, the subjec-tive ratings of this experiment were higher than those of C program-ming and XML scripting of Experiment I (compare Figures 11(b)and 5(b)). This indicates that both methods are easy, intuitive, effi-cient, and fun, but just have different characteristics. For example,some participants commented that waveform editing was more intu-itive since it showed changes in vibration strength using a graph. Incontrast, other participants reported the opposite since score edit-ing effectively expressed vibration rhythm using familiar musicalnotations (Table 4).

In terms of user preference, the participants preferred vibrotac-tile score editing more than waveform editing (Figure 11(c)). Itsstatistical significance was confirmed by ANOVA except in task 1(Table 3). It appears that the preference of score editing is moreapparent in complex and long tasks. In terms of overall preference,the participants gave much higher ratings for the score editing task.In the individual data, 10 out of the 12 participants preferred vi-brotactile score editing. Most participants reported that vibrotactilescore editing was a fresh and new idea and felt more efficient thanwaveform editing (Table 4). Verbal comments collected from theparticipants are summarized in Table 4.

4 GENERAL DISCUSSION

In the two usability experiments, the vibrotactile score demon-strated quantitative and qualitative performances superior to theother existing vibrotactile pattern design methods of programming,scripting, and waveform editing. This was unanimous in both usergroups of programming experts and regular users, even for the tasks

unfavorable to score-based editing. We believe that the strengthof the vibrotactile score stems from its concise and abstract repre-sentation of various vibration attributes, adapted from the musicalsymbols that have been refined for centuries. The vibrotactile scoremay prove more useful in actual authoring where pattern designsand tests are repeated many times.

Another unique strength is that the vibrotactile score is ade-quate by its nature for designing vibrotactile patterns from musicalsources, as demonstrated in the usability experiments. This fea-ture can be useful for several important applications, such as tactileicons for information delivery using rhythm variations [12], tactilemelodies transformed from music [25], and tactile stimuli for thehearing impaired to feel music [10].

In addition, the vibrotactile score can be valuable in small elec-tronic devices such as mobile devices. Obviously, programming orscripting is not an option in mobile devices. Waveform editing isessentially a continuous process and is not suitable for a mobile de-vice with a limited number of buttons. It is likely that a touch-screeninterface with a stylus would be necessary for waveform editing. Incontrast, a vibrotactile score consists of discrete symbols, and there-fore it can be easily managed with a small number of buttons and/orwith a GUI interface.

An apparent drawback of the vibrotactile score is the need tolearn symbols for the users unfamiliar with musical scores. For thisreason, our vibrotactile score only uses the basic and simple musicalsymbols that are usually included in regular elementary education.In fact, we attempted to assess the learnability of our vibrotactilescore in a formal experiment. However, we were unable to recruita sufficient number of the participants who could use a computerbut not read basic musical symbols. This can be due to the factthat Korean regular education includes music for at least nine years.Instead, we asked two additional participants, who volunteered toExperiment II but did not pass the screening test about basic musi-cal notations, to solve the tasks of Experiment II. They memorizedthe basic musical symbols for only five minutes before the experi-ment. Interestingly, the task completion times were similar to thosereported in Experiment II. Their errors, however, were about fivetimes higher than the participants of Experiment II, who passed thescreening test. Most mistakes were made on the durations of a noteand a rest. Although informal, these results also demonstrate thedecent learnability of the vibrotactile score, which originates fromthe use of the musical symbols already familiar to most people.

5 CONCLUSIONS

In this paper, we have evaluated the usability of vibrotactile scoreas a way of authoring vibrotactile patterns in two user experiments.Experiment I aimed at assessing the usability of score-based edit-ing in comparison with the present practice of programming and

237

scripting, assuming an expert user group who can make the mostauthoritative judgments. The vibrotactile score outperformed theother two by far in all of task performance, subjective measures, anduser preference. In Experiment II where regular users with no pro-gramming background participated, the target of comparison waswaveform-based design, the graphical alternative becoming morepopular recently. Again, score-based design exceeded waveform-based design in all metrics. In particular, task completion time de-creased more than twice for a long task. All of the experimentalresults suggested that score-based design can be a superior designmethod to currently existing alternatives and is worth for persistentdevelopment and evaluation.

The source code of the current VibScoreEditor is available fordownloading at http://hvr.postech.ac.kr/?page id=120. At present,we are upgrading the VibScoreEditor so as to support multiple vi-bration actuators.

ACKNOWLEDGMENTS

This work was supported in parts by an NRL program 2010-0018454 and a BRL program 2010-0019523 both from NRF andby an ITRC program NIPA-2011-C1090-1111-0008, all funded bythe Korean government.

REFERENCES

[1] S. Brewster and L. M. Brown. Tactons: Structured tactile messages fornon-visual information display. In Proceedings of the 5th AustralasianUser Interface Conference, pages 15–23, 2004.

[2] L. M. Brown, S. A. Brewster, and H. C. Purchase. A first investigationinto the effectiveness of tactons. In Proceedings of the World HapticsConference, pages 167–176, 2005.

[3] L. M. Brown, S. A. Brewster, and H. C. Purchase. Multidimensionaltactons for non-visual information presentation in mobile device. InProceedings of International Conference on Human Computer Inter-action with Mobile Devices and Services (MobileHCI), pages 231–238, 2006.

[4] A. Chan, K. MacLean, and J. McGrenere. Learning and identifyinghaptic icons under workload. In Proceedings of the First Joint Euro-haptics Conference and Symposium on Haptic Interfaces for VirtualEnvironment and Teleoperator Systems (World Haptics 2005), pages432–439, 2005.

[5] M. Enriquez, K. MacLean, and C. Chita. Haptic phonemes: Basicbuilding blocks of haptic communication. In Proceedings of the In-ternational Conference on Multimodal Interfaces (ICMI), pages 302–309, 2006.

[6] M. J. Enriquez and K. E. MacLean. The Hapticon Editor: A tool insupport of haptic communication research. In Proceedings of the In-ternational Symposium on Haptic Interfaces for Virtual Environmentand Teleoperator Systems, pages 356–362. IEEE, 2003.

[7] A. Gallace, H. Z. Tan, and C. Spence. The body surface as a com-munication system: The state of the art after 50 years. Presence,16(6):655–676, 2007.

[8] R. Gault. Progress in experiments on tactual interpretation of oralspeech. Journal of Abnormal Psychology and Social Psychology,19(2):155–159, 1924.

[9] Immersion Corp. MOTIV studio, 2011. http://www2.immersion.com/developers/index.php?option=com content&view=section&layout=blog&id=17&Itemid=504.

[10] M. Karam, F. Russo, C. Branje, E. Price, and D. I. Fels. Towardsa model human cochlea: Sensory substitution for crossmodal audio-tactile displays. In Proceedings of Graphics Interface Conference,pages 267–274, 2008.

[11] J. Lee, J. Ryu, and S. Choi. Vibrotactile score: A score metaphorfor designing vibrotactile patterns. In Proceedings of World HapticsConference, pages 302–307, 2009.

[12] K. E. MacLean. Foundations of transparency in tactile informationdesign. IEEE Transactions on Haptics, 1(2):84–95, 2008.

[13] K. E. MacLean and M. Enriquez. Perceptual design of haptic icons.In Proceedings of Eurohaptics, pages 351–363, 2003.

[14] T. McDaniel, S. Krishna, D. Colbry, and S. Panchanathan. Using tac-tile rhythm to convey interpersonal distances to individuals who areblind. In Proceedings of the ACM International Conference on Hu-man Factors in Computing Systems, pages 4669–4674, 2009.

[15] G. Park, S. Choi, K. Hwang, S. Kim, J. Sa, and M. Joung. Tac-tile effect design and evaluation for virtual buttons on a mobile de-vice touchscreen. In Proceedings of the International Conference onHuman-Computer Interaction with Mobile Devices and Services (Mo-bileHCI), pages 11–20. ACM, 2011.

[16] J. Ryu and S. Choi. Benefits of perceptually transparent vibrationrendering in mobile device. Lecture Notes on Computer Science (Eu-roHaptics 2008), 5024:706–711, 2008.

[17] J. Ryu and S. Choi. posVibEditor: Graphical authoring tool of vibro-tactile patterns. In Proceedings of the IEEE International Workshopon Haptic, Audio and Visual Environments and Games, pages 120–125, 2008.

[18] J. Ryu, J. Chun, G. Park, S. Choi, and S. H. Han. Vibrotactile feedbackfor information delivery in the vehicle. IEEE Transactions on Haptics,3(2):138–149, 2010.

[19] J. Ryu, J. Jung, and S. Choi. Perceived magnitudes of vibrations trans-mitted through mobile device. In Proceedings of the Symposium onHaptic Interfaces for Virtual Environment and Teleoperator Systems,pages 139–140, 2008.

[20] J. Ryu, J. Jung, S. Kim, and S. Choi. Perceptually transparent vibra-tion rendering using a vibration motor for haptic interaction. In Pro-ceedings of the IEEE International Symposium on Robot & HumanInteractive Communication, pages 310–315, 2007.

[21] J. Ryu, J. Jung, G. Park, and S. Choi. Psychophysical model for vi-brotactile rendering in mobile devices. Presence: Teleoperators andVirtual Environments, 19(4):1–24, 2010.

[22] J. Ryu, C.-W. Lee, and S. Choi. Improving vibrotactile pattern identifi-cation for mobile devices using perceptually transparent rendering. InProceedings of the International Conference on Human-Computer In-teraction with Mobile Devices and Services (MobileHCI), pages 257–260. ACM, 2010.

[23] C. Spence and C. Ho. Tactile and multisensory spatial warning signalsfor drivers. IEEE Transactions on Haptics, 1(2):121–129, 2008.

[24] C. Swindells, E. Maksakov, K. E. Maclean, and V. Chung. The roleof prototyping tools for haptic behavior design. In Proceedings of theSymposium on Haptic Interfaces for Virtual Environment and Teleop-erator Systems, pages 161–168. IEEE, 2006.

[25] J. B. van Erp and M. M. Spape. Distilling the underlying dimensionsof tactile melodies. In Proceedings of Eurohaptics, pages 111–120,2003.

[26] B. J. Winer, D. R. Brown, and K. M. Michels. Statistical Principles inExperimental Design. McGraw-Hill, 3rd edition, 1991.

238