HAVE 2008 - IEEE International Workshop onHaptic Audio Visual Environments and GamesOttawa, Canada, 18-19 October 2008

posVibEditor: Graphical Authoring Tool of Vibrotactile Patterns

Jonghyun Ryu and Seungmoon Choi

Haptics and Virtual Reality LaboratoryDepartment of Computer Science and Engineering,

POSTECH, KoreaE–mail: {fall, choism}@postech.ac.kr

Abstract – This paper presents an authoring tool, posVibEditor, forquick and easy design of vibrotactile patterns for vibration motors.The tool supports the drag-and-drop design paradigm so that noviceusers can easily learn and interact with the tool. Vibration patternsare managed in a database using XML formats in order to improvetheir reusability and extensibility. The multi-channel timeline inter-face is also developed for designing time-synchronized multiple vi-brotactile patterns for multiple vibration motors. In addition, an in-ternal vibration player is incorporated to allow the user to test andevaluate designed patterns immediately. The last unique feature is amodule for perceptually transparent rendering that can guarantee thedelivery of perceptually correct vibration effects. The authoring toolis suitable to mobile devices that contain a single vibration motor aswell as applications in virtual reality and haptics that frequently em-ploy multiple vibration motors.

Keywords – Vibrotactile pattern, Vibration authoring, Mobile device.

I. INTRODUCTION

Research in haptics has shown that information transmis-sion via vibrotactile stimulation can be effective for many ap-plications in virtual reality [1][2], human-computer interaction[3], and games [4][5][6]. Vibrotaction alone is useful in de-livering information in a direct and uncluttered manner. Itscross-modal use with vision and audition often allows impres-sive performance increase and/or richer sensory experience.In particular, the recent adoption of vibrotactile feedback in acommercial mobile phone (so-called “Haptic Phone” by Sam-sung Electronics) has proven that the technology is becomingmature.

Given an application, a key to successful design of vibrotac-tile effects is two-fold. First, a vibration actuator that is tech-nologically feasible to the application should be chosen, whiletaking into account the range of physical vibrotactile stimuli itcan create. There exist several types of vibration actuators suchas vibration motor, voice-coil actuator, and piezoelectric actua-tor [7], but none of them are perfect in the sense that each actu-ator has a range of vibrotactile stimuli that the actuator cannotcover (e.g., see [8]). Second, the perceptual effects of resultingvibration patterns need to be carefully evaluated. Otherwise,

a vibrotactile signal may fail to accomplish its goal, especiallydue to the fairly limited human performance of vibrotactile per-ception (see [9] for a review). These limitations in both actua-tor technology and human perception usually require a signif-icant number of iterations in designing and testing vibrotactileeffects.

Despite the numerous applications of vibrotaction (see [7]for review), effort for developing software tools that enableeasy and quick authoring of haptic effects has been somewhatscarce. MacLean et al. presented the “Hapticon Editor” inwhich haptic icons for a rotary knob can be graphically de-signed [10]. This editor was subsequently extended to the hap-tic icon prototyper [11]. In this editor, force profiles with re-spect to time can be designed using graphical user interface(GUI), and the designed haptic icons can be saved and reusedusing a palette. A player of the haptic icons is also included.The editor also supports the import capability of force profilescaptured from a real knob. This editor can also be used fordesigning vibration effects, although it is more suited to kines-thetic feedback with a haptic knob.

Another recent notable authoring tool is the VibeTonz sys-tem [12] developed by Immersion Corp. This commercial toolprovides GUIs for designing vibrotactile effects to be used inmobile devices. Drag-and-drop is also supported. The targetvibration actuator is a vibration motor inside a mobile device,thus vibration patterns are designed in terms of voltage appliedto the motor [13]. A unique feature of the VibeTonz systemis autonomous generation of vibrotactile patterns from a MIDIfile containing music, mostly relying on the rhythm informa-tion. To the best of our knowledge, we are not aware of vi-bration editing software other than the Hapticon Editor and theVibeTonz.

In this paper, we present a graphical authoring tool of vibra-tory patterns for multiple vibration motors, named “posVibEd-itor”. The posVibEditor supports• Graphical editing of vibration patterns,• Drag-and-drop design paradigm,• Systematic management of vibration patterns,• XML formats in data files for extensibility,• Multichannel vibration pattern design,

978-1-4244-2669-0/08/$25.00 ©2008 IEEE

TABLE I. Summary of the functionalities of the Haptic icon prototyper, the VibeTonz, and the posVibEditor.

Haptic Icon Prototyper [11] VibeTonz [12] posVibEditor

Authoring stimulus Force Vibration VibrationUsed actuator Rotary knob Vibration motor Vibration motor

Designable pattern type Various Trapezoidal form only VariousDrag-and-drop O O OInternal player O O O

Multichannel timeline 4∗ X OMultiple actuators X X O

Perceptually transparent rendering X X O∗ Timeline interface for multiple haptic attributes is included for a single channel haptic knob.

• Quick test of designed patterns by using a internal player,and

• Perceptually transparent rendering [13].Table I summarizes the main functionalities of the HapticonEditor, the VibeTonz, and our posVibEditor.

The rest of this paper is organized as follows. Sec. II in-troduces vibration actuators and two design modes supportedby the posVibEditor. The overview of user interface and inter-nal structure is given in Sec. III. The following three sections(Secs. IV, V, and VI) explain in full details the three majorcomponents of the posVibEditor: the vibration pattern man-ager, vibration pattern editor, and multichannel timeline inter-face, respectively. We conclude the paper in Sec. VII.

II. VIBRATION ACTUATORS AND DESIGN MODES

The posVibEditor has been developed for vibrotactile pat-terns that can be played with a single or multiple vibrationmotors. Thus, the editor can be used for mobile phone ap-plications that usually contains one vibration motor, as well asfor VR and haptics research where many vibration motors areoften used for various purposes.

In the posVibEditor, a user can choose one of two designmodes that were conceived based on the driving principle of avibration motor. In the vibration motor, voltage applied to itdetermines the angular velocity of its rotor that has eccentricmass distribution to induce vibration. The angular velocity ofthe rotor is proportional to the linear frequency of the result-ing sinusoidal vibration [14]. The vibration amplitude is alsodecided uniquely by the applied voltage. Thus, the commonpractice for the vibration motor has been to design a timed se-quence of applied voltage that will be sent to the vibration mo-tor, and this is the first mode supported by the posVibEditor. Acritical drawback of this mode, however, is that it can be verydifficult for the user to imagine what vibration stimulus wouldbe generated from a voltage pattern being designed. An ex-ample is given in Fig. 1, where the red smooth line representsvoltage command to a vibration motor and the blue oscillat-ing line represents acceleration measured from the motor. It is

evident from the figure that the slowly-changing voltage com-mand results in the sinusoidal vibration waveform with varyingamplitude and frequency, even in significant actuation delays.

A much more intuitive design mode supported by theposVibEditor is based on the concept of Perceptually Trans-parent Rendering (PTR) previously proposed by our group[13][15]. The common practice prior to PTR was that vibra-tion patterns are designed for a physical variable (e.g., appliedvoltage) that will be sent to a vibration display (e.g., a mobilephone with a vibration motor), as shown in Fig. 2(a). Theexerted vibration by the display is then processed by the hu-man central nervous system. However, the dynamics of thevibration display and the complex human perception may se-riously distort the perceived vibration effect from the originaldesign intention. For example, when a vibration pattern thatincreases linearly with time is transmitted without PTR as is inFig. 2(a), a target perceptual attribute, the magnitude of vibra-tion perceived by the user, may grow nonlinearly (see [13] forconcrete data). This makes it difficult for the designer to pre-dict what perceptual effects will be obtained by the vibrationpatterns s/he is designing.

In PTR, the forward relation with vibration command as in-put and target percept as output is modeled, and its inverserelation is used for vibration rendering, as shown in Fig. 2(b).Thus, the user can design a vibration effect with regards tothe target percept, and the vibration command that will inducethe intended percept is autonomously computed in a renderingprogram. This strategy minimizes the perceptual distortion ofvibration rendering, and was shown to improve the pairwisediscriminability of vibratory stimuli in a mobile phone [15].

The design mode for PTR implemented in the posVibEditorconsiders vibration voltage command to a mobile device as in-put and perceived vibration magnitude as output, based on ourprevious findings on their relations [13], [16]. A vibration pat-tern can be authored for the perceived magnitude of vibrationthat the user would experience. Then, a module responsible forPTR included in the form of a script in the posVibEditor auto-matically converts the pattern into voltage values to be sent toa vibration motor in the mobile device. This capability for per-

Fig. 1. Example of voltage applied to a vibration motor (LG Innotek; MVMFA345A) in a mobile phone (LG Electronics; KH-1000) and generated vibration.

Fig. 2. Conceptual comparison of vibration rendering (a) without and (b) with perceptually transparent rendering.

ceptually transparent rendering is one of the features unique tothe posVibEditor.

III. OVERVIEW OF USER INTERFACE AND INTERNALSTRUCTURE

The posVibEditor consists of three major panels for userinterface. They are labeled with (a), (b), and (c) in Fig. 3.Panel (a) is named as the vibration pattern manager, and isworkspace for managing and grouping formerly designed vi-bration patterns. Multiple groups of vibration patterns can beadministrated using the tabs located in the bottom of the panel,improving the reusability. Panel (b) is the vibration pattern ed-itor in which a vibration pattern can be designed via drag-and-drop by moving control points overlaid on the pattern. Severalpre-defined pattern templates are also supported. Panel (c) isthe multichannel timeline interface. A designed pattern canbe inserted into this timeline interface for composing complexmultichannel vibration patterns that would be synchronouslyplayed via multiple vibration motors. We note that all of thethree panels support drag-and-drop for quick and easy designof vibration patterns.

The three user interface components interact with five in-ternal functional modules, the data structure manager module,XML loader, PTR module, player module, and hardware com-munication module (Fig. 4). Further details on each user inter-

face component and associated internal modules will be pre-sented in the rest of this paper.

IV. VIBRATION PATTERN MANAGEMENT

All files used in the posVibEditor, e.g., for vibration pat-terns, database for vibration pattern management, and mul-tichannel configurations, are saved in the XML (eXtensibleMarkup Language) formats using MSXML (Microsoft CoreXML Service) 6.0 . The XML file format has three major char-acteristics – structurability, extensibility, and interoperability[17][18]. Since the XML format supports a hierarchical deepnested structure, vibration patterns that have many parameterscan be saved in a highly structured form. The XML format canbe easily extended to include new parameters by simply addingnew corresponding tags defined freely by the developer, with-out changing the file structure or the program for loading thefiles. The interoperability of the XML file format also allowsdesigned vibration patterns to be used in all platforms, such asWindows, Linux, Mac, and even embedded system.

The XML schema used to represent vibration patterns isshown in Fig. 5(a). A vibration pattern file should have ‘VIB-SEG’ tag as a root node with ‘BASETIME’, ‘NAME’, ‘TYPE’,and ‘CTRPOINT LIST’ tags as child nodes. The ‘BASE-TIME’ tag defines the start time of a vibration pattern and canbe ignored if the start time is zero. The ‘NAME’ tag defines the

Fig. 3. Screenshot of the posVibEditor.

Fig. 4. Internal structure of the posVibEditor.

unique name for the vibration pattern. The ‘TYPE’ tag spec-ifies which of the three types of vibration patterns designablein the vibration editor (see Sec. V for further details) is usedfor the vibration pattern. For this, the current posVibEditorsupports three strings: ‘curve’, ‘line’, and ‘sine’. The ‘CTR-POINT LIST’ tag is to store the list of control points consist-

Fig. 5. The XML schema for saving (a) a vibration pattern and (b) amultichannel configuration.

ing in the vibration pattern and has one more ‘CTRPOINT’tags. The ‘CTRPOINT’ tag has ‘TIME’ and ‘INTENSITY’ at-tributes to describe the start time and vibration magnitude of acontrol point.

Vibration patterns can be administrated using the vibrationpattern manager. Vibration patterns designed using the vibra-tion pattern editor (see Sec. V) can be registered in the databaseof the vibration pattern manager, grouped for efficient man-

agement, and reused later. To register a vibration pattern intothe manager, the designer can select the corresponding tab onthe multi-tab panel in the bottom of the vibration pattern ed-itor, and drag and drop the tab into an adequate location inthe vibration pattern manager. The registered vibration patternis automatically grouped and sorted as an item in the activevibration group. The vibration pattern manager enables thestructured storage of vibration patterns and improves the over-all reusability of the patterns, in the efficient manner using thefairly standard GUI.

Among the five internal modules shown in Fig. 4, two mod-ules, the data structure manager module and the XML loaderare responsible for the vibration pattern manager. As impliedin their names, the data structure manager module is in chargeof dealing with the data for vibration patterns and multichan-nel timeline. The XML loader saves and reads data in the XMLschema to and from an external storage such as a hard disk.

V. VIBRATION PATTERN EDITING AND PLAY

The vibration pattern editor of the posVibEditor (see panel(b) in Fig. 3) provides user interface for designing vibrationpatterns and playing them on the spot. The designer can beginpattern design by choosing one of predefined types of vibrationpatterns. The current version supports three types: curve, line,and sine waveform (see Fig. 6). A segment in the curve patternconsists of two control points, and the control points are inter-polated using Catmull-Rom spline [19]. The line waveformalso uses two control points for each segment. The sine wave-form requires three control points. The time and value of thefirst control point defines the start time and DC offset, respec-tively. The amplitude of the sine waveform is determined bythe difference between the values of the first and second con-trol points. The last control point determines the finish time.

To edit a vibration pattern, the designer can move controlpoints overlaid on the pattern using a mouse. The left button ofthe mouse is used for mostly moving a control point to a newposition, and the right button for adding and deleting a con-trol point. To add a new control point, the designer presses theright button on a control point near which s/he wants to add anew control point. Then a new control point is created besidethe selected control point, and the designer can drag the newcontrol point to a desired position while keeping the right but-ton pressed. A control point can be deleted by pushing the leftor right button on it and dragging it outside of the editor panel.Multi-tab interface (see the bottom of the editor) is also pro-vided for simultaneous design of multiple vibration patterns.This intuitive multi-tab interface with the drag-and-drop leadsto high accessibility, convenience, and efficiency in vibrationpattern design, even for novice users.

A designed vibration pattern can be played and tested usingthe internal player. The player interface is located in the rightof the vibration editor window. When PTR is used, the internalplayer converts a desired perceived magnitude profile into thecorresponding voltage sequence using the PTR module, and

Fig. 6. Three types of vibration pattern supported in the posVibEditor: (a)curve, (b) line, and (c) sine waveform.

then transmits the voltage command to the vibration hardwarethrough serial communication. When PTR is not used, a volt-age vibration pattern is directly sent to the hardware. Using theinternal player, the designer can repeat the design and test ofvibration patterns quickly and easily, significantly acceleratingthe whole design process.

The internal player uses the multimedia timer supported inMicrosoft Windows. Since the multimedia timer has 1 ms reso-lution for timing update, the maximum sampling rate for send-ing a vibration waveform to the hardware is 1 kHz. At present,we set the sampling rate to 200 Hz considering the compu-tation time for the vibration pattern interpretation, the com-munication time with the vibration hardware, and the typicalactuation delay of vibration motors.

Three internal modules, the PTR, internal player, and hard-ware communication modules, take care of the user interfacefor vibration pattern editing and playing. The current com-munication module uses the serial communication port in aPC, but can be easily extended to support more communica-tion means if necessary.

VI. MULTICHANNEL TIMELINE INTERFACE

The multichannel timeline interface (panel (c) in Fig. 3)is incorporated in the posVibEditor in order to support the de-sign and test of simultaneously controlled vibration patterns formultiple vibration motors. Using a number of vibration motorsfor interesting and effective vibration feedback is very com-mon in haptics and VR research (see [20] for a recent exam-ple). The multichannel timeline configuration is also saved inthe XML format following the schema shown in Fig. 5(b). Theroot node ‘MULTICHANNEL’ has ‘CHANNEL’ tags. Each ofthe ‘CHANNEL’ tag has ‘CHAN NAME’ tag, ‘MOTOR ID’

tag, and several ‘VIBSEG’ tags, which define a unique chan-nel name, the ID of a vibration motor corresponding to thechannel, and vibration patterns to be played in the channel (asdefined in Fig. 5(a)), respectively.

Using the multichannel timeline interface, a designer cancreate multiple channels each of which corresponds to a spe-cific vibration motor, and arrange desired vibration patterns onthe timeline corresponding to the channel. This multichanneltimeline design is commonly used in video and audio editingtools. Our multichannel timeline interface for vibration edit-ing is conceptually the same to that of traditional video andaudio editors, in that multiple vibration “clips” can be orga-nized along the common timeline and that an individual vi-bration channel is mapped to a vibration motor similarly to anaudio channel mapped to a speaker in the audio system. Thedrag-and-drop manipulation is also supported in the multichan-nel interface. To our knowledge, this multichannel feature isunique in the posVibEditor among haptic authoring tools.

For communicating with the vibration hardware, we use se-rial communication from a PC to our custom controller formultiple vibration motors. Information for driving multiple vi-bration motors is encoded in a custom-designed protocol usingseven bytes for each channel in every frame. One byte is usedfor a starting flag, and three bytes are for device number, mo-tor number, and vibration level, respectively. The other threebytes duplicate the data bytes for validation. This protocol canbe easily extended to take care of other types of vibration hard-ware if necessary.

VII. CONCLUSION

We developed the vibration pattern authoring tool named“posVibEditor” for quick and easy design of multiple vibrationpatterns. The user interface supports drag-and-drop to maxi-mize the user’s convenience. A vibration pattern can be graph-ically designed using the predefined templates, and registeredto a database for reuse. The multichannel timeline interfaceis also included for designing synchronized multichannel vi-bration patterns for multiple vibration motors, along with thevibration player embedded for rapid test and evaluation. Theextensibility of supported vibration patterns is also guaranteedby using the XML format for data management.

Furthermore, the posVibEditor supports perceptually trans-parent rendering in which desired vibration effects are de-signed in the perceived intensity of vibration and automaticallyconverted to the corresponding sequence of applied voltageto be sent to a vibration motor [13]. This method minimizesany interference of physical and perceptual characteristics ofthe vibration generation and perception processes on the user’spercept. Consequently, the novice designer can concentrate onthe perceptual effect of a vibration pattern only, which makesthe authoring process highly intuitive and convenient.

The current version of posVibEditor only supports vibrationmotors as actuators, but we plan to upgrade it to handle voice-coil vibration actuators.

ACKNOWLEDGMENTS

This work was supported in parts by the Korea Science andEngineering Foundation (KOSEF) grant (No. R0A-2008-000-20087-0) (MOST) and by the ITRC (Information TechnologyResearch Center) support program (No. IITA-2008-C1090-0804-0002) supervised by the IITA, both funded by the Koreagovernment.

REFERENCES

[1] J. Ryu and G. J. Kim, “Using a vibro-tactile display for enhanced colli-sion perception and presence,” in Proceedings of the ACM InternationalConference on Virtual Reality Software and Technology, 2004, pp. 89–96.

[2] R. W. Lindeman, J. N. Templeman, J. L. Sibert, and J. R. Cutler, “Han-dling of virtual contact in immersive virtual environments: Beyond visu-als,” Virtual Reality, vol. 6, no. 3, pp. 130–139, 2002.

[3] L. M. Brown and T. Kaaresoja, “Feel who’s talking: Using tactons formobile phone alerts,” in CHI ’06 Extended Abstracts on Human Factorsin Computing Systems, 2006, pp. 604–609.

[4] Sony Computer Entertainment Inc., “Playstation,” 2007, http://www.playstation.com.

[5] Microsoft Corp., “Xbox,” 2007, http://www.xbox.com.[6] Nintendo, “Wii,” 2007, http://wii.com.[7] M. Benali-Khoudja, M. Hafez, J.-M. Alexandre, and A. Kheddar, “Tac-

tile interfaces. a state-of-the-art survey,” in Proceddings of 35th Interna-tional Symposium on Robotics (ISR’04), 2004, pp. 721–726.

[8] B. J. P. Mortimer, G. A. Zets, and R. W. Cholewiak, “Vibrotactile trans-duction and transducers,” Journal of the Acoustical Society of America,vol. 121, no. 5, pp. 2970–2977, 2007.

[9] L. A. Jones and S. J. Lederman, Human Hand Function. New York,NY, USA: Oxford University Press, Inc., 2006.

[10] M. J. Enriquez and K. E. MacLean, “The hapticon editor: A tool in sup-port of haptic communication research,” in In Proceedings of the 11thSymposium on Haptic Interfaces for Virtual Environment and Teleoper-ator Systems, 2003, pp. 356–362.

[11] C. Swindells, E. Maksakov, K. E. Maclean, and V. Chung, “The role ofprototyping tools for haptic behavior design,” in Proceedings of Sym-posium on Haptic Interfaces for Virtual Environment and TeleoperatorSystems, 2006, pp. 161–168.

[12] Immersion Corp., “Mobility overview,” Immersion Corp., 2007, http://www.immersion.com/mobility.

[13] J. Ryu, J. Jung, S. Kim, and S. Choi, “Perceptually transparent vibrationrendering using a vibration motor for haptic interaction,” in Proceed-ings of IEEE International Symposium on Robot and Human InteractiveCommunication, 2007, pp. 310–315.

[14] J. Jung and S. Choi, “Perceived magnitude and power consumption ofvibration feedback in mobile devices,” Lecture Notes on Computer Sci-ence (Proceedings of the International Conference on Human-ComputerInteraction, Part II, HCII 2007), vol. 4551, pp. 354–363, 2007.

[15] J. Ryu and S. Choi, “Benefits of perceptually transparent vibration ren-dering in mobile device,” Lecture Notes on Computer Science (EuroHap-tics 2008), vol. 5024, pp. 706–711, 2008.

[16] J. Ryu, J. Jung, and S. Choi, “Perceived magnitudes of vibrations trans-mitted through mobile device,” in Proceedings of The 16th Symposiumon Haptic Interfaces for Virtual Environments and Teleoperator Systems,2008, pp. 139–140.

[17] T. Bray, J. Paoli, and C. M. Sperberg-McQueen, “Extensible markup lan-guage (xml) 1.0,” World Wide Web Consortium (W3C) Recommendation,http://www.w3.org/XML/, 1998.

[18] C. F. Goldfarb and P. Prescod, The XML handbook. Upper Saddle River,NJ, USA: Prentice Hall PTR, 1998.

[19] E. Catmull and R. Rom, “A class of local interpolating splines,” Com-puter Aided Geometric Design, pp. 317–326, 1974.

[20] H.-Y. Chen, J. Santos, M. Graves, K. Kim, and H. Z. Tan, “Tactor local-ization at the wrist,” Lecture Notes on Computer Science (EuroHaptics2008), vol. 5024, pp. 209–218, 2008.