The Programmable Haptic Rhythm TrainerLiya Grace Ni

School of EngineeringCalifornia Baptist UniversityRiverside, California 92504Email: gni@calbaptist.edu

Abstract—Mastering rhythm and hand coordination are bigchallenges in piano learning. This demonstration presents a novelembedded haptic-audio device, which allows a user to enter therhythm through the keypad, and feel the different rhythm on lefthand and right hand simultaneously. Meanwhile it also producessteady beat with clicking sound and blinking LED. This devicehas the potential to significantly enhance at-home piano practiceor online piano learning.

I. INTRODUCTION

Rhythm is the heart beat of a music piece. Yet masteringrhythm is challenging for many musical instrument players.Particularly in piano playing, the challenge of playing thecorrect rhythm normally is combined with another challenge- hand coordination. Piano teachers often ask students to clapor tap the rhythm of a new composition they started to learn,before they actually play it on the piano. This method isquite effective since it involves multisensory learning, in otherwords, the rhythm is learnt through visual, auditory and hapticchannels simultaneously. For practice at home or online pianolessons, in which cases the piano teacher is not available tocorrect the rhythm clapping or tapping, it is desirable to havea device to assist this multisensory rhythm learning approach.

Previous research on applying haptics technology in mu-sic education has been mainly focused on improving pos-ture/gesture during the performance of a music instrument. Forexample, the Haptic Guidance System (HAGUS) for musicalmotor learning, particularly drum playing [1], and the real-time vibrotactile feedback system to correct a novice violinstudent’s posture and bowing action [2].

The programmable haptic rhythm trainer proposed here isan embedded haptic device that improves rhythm learningand hand coordination in piano playing. It integrates hapticguidance on the rhythm of a specific composition with theauditory and visual clues similar to a regular metronome. Aprogramming interface with keypad and LCD is provided fora user to enter the rhythm and other options such as speed andtime signature.

II. SYSTEM DESCRIPTION

The Dragon12-Plus training board (www.evbplus.com) witha Freescale MC9S12DG256 microcontroller was adopted forthe prototype development of the haptic rhythm trainer. Theonboard LCD and keypad provide an interface for a user toinput the rhythm of a certain segment of music and otheroptions. Two TowerPro SG-50 servo motors are utilized to

generate the haptic guidance of rhythm, one for the left handand one for the right hand. Haptic guidance of rhythm isintegrated with the audio and visual clues that are provided bythe onboard speaker and LED. Similar to a regular metronome,the audio and visual outputs are in the form of clickingsound and blinking LED in steady beats. Figure 1 shows theDragon12-Plus training board with the onboard speaker, LED,keypad, LCD and two TowerPro SG-50 servo motors mountedon the top.

Fig. 1. Dragon12-Plus Demo Board with Servo Motors

The LCD displays instructions for user input, and willdisplay error message for any invalid entry. Help messageswill be displayed when the user press the “#” key at anytime during the input stage. By following the instructions, theuser can enter the tempo, time signature, right hand rhythm,left hand rhythm, and number of repeats by using the 4-by-4 keypad. The rhythm is entered using the number keys, forexample, 1 for a whole note, 4 for a quarter note, and so on.Any number followed by a “0” indicates a rest instead of anote. After the programming is done, the haptic, audio, visualoutputs are generated in a synchronized pattern. Correspondingto the user specified time signature, the first beat in everymeasure is played in a higher pitch from the speaker and therest in a lower pitch. The switching of LED between on andoff is synchronized with the clicking sound. Each motor isswitching between two angular positions. Corresponding tothe “down”s and “up”s in the action of tapping, the angularposition for “down” is denoted as θ1, and the position for “up”is denoted as θ2. For a rest, the servo motor is controlled to

978-1-4244-6509-5/10/$26.00 ©2010 IEEE

be staying at position θ2 for the whole duration. For a note, itstays at position θ1 for 100 milliseconds less than the duration,and goes back to θ2 and stays for the rest 100 milliseconds.The two motors move corresponding to the different rhythmon the left hand and right hand respectively. The same rhythmwill be repeated according to the number of repeats enteredby the user. To feel the rhythm, the user need to rest his/herfinger tip on the plastic piece mounted on the motor shaft,then he/she will feel gentle taps corresponding to the rhythm.

III. CONTRIBUTIONS AND FUTURE WORK

This device can significantly enhance the piano learningexperience due to the following advantages:

• Normally a piano student only has up to one hour perweek time contact with the teacher. Many adults alsochoose to take online piano lessons. The haptic rhythmtrainer can be programed and play the correct rhythmautomatically, thus benefits at-home practice and onlinepiano lessons.

• Instead of providing haptic guidance while the user isplaying piano, the rhythm training is a separate stepbefore playing. This is similar to the popular exercise ofclapping or tapping rhythm before playing the music onthe piano. This way the rhythm can be played correctlybut not too mechanically, and the piano player can stillfocus on other things like the melody, finger numbering,dynamics, etc.

• In addition to rhythm learning, this device can also helpimproving hand coordination by tapping the differentrhythm on left hand and right hand simultaneously.

There is still a big gap between a consumer product andthe prototype version of the programmable haptic rhythmtrainer presented in this paper. First of all, the Dragon12-Plus demo board provides a lot more features that are notneeded by the haptic metronome, which results in a highercost. The size of the Dragon12-Plus demo board itself isalmost four times of a regular metronome. In order to make acompact and less expensive device, a PCB with the peripheraldevices (LCD, speaker, servos, keypad and LED) and themicrocontroller need to be designed to replace the demo board.The programming of rhythm will be more convenient if therhythm exercises provided by music educational publisherscan be directly downloaded from their websites to the hapticrhythm trainer. This will require a USB interface and asoftware module to support interpreting the downloaded file.The rhythm file should also be written in a compatible format.Based on future evaluation of the device, other new featuresand modifications will be incorporated to the design.

REFERENCES

[1] Grindlay, G., “Haptic Guidance Benefits Musical Motor Learning”, Pro-ceedings of the 2008 IEEE Symposium on Haptic Interfaces for VirtualEnvironments and Teleoperator Systems, Reno, Nevada, pp. 397-404.

[2] Van Der Linden, J., Schoonderwaldt, E. and Bird, J., “Good vibrations:Guiding body movements with vibrotactile feedback”, Proceedings ofthe Third International Workshop on Physicality, September 2009, Cam-bridge, UK, pp. 13-18.