Robotically Augmented Electric Guitar for Shared Control

Takumi OgataGeorgia Institute of Technology

840 McMillan St.Atlanta, GA

togata3@gatech.edu

Gil WeinbergGeorgia Institute of Technology

840 McMillan St.Atlanta, GA

gilw@gatech.edu

ABSTRACTRobotically Augmented Electric Guitar is a novel roboticguitar that establishes shared control between human per-formers and mechanical actuators. Unlike other mecha-tronic guitar instruments that perform pre-programmed mu-sic automatically, this guitar allows the human and actua-tors to produce sounds jointly; there exists a distributedcontrol between the human and robotic components. Theinteraction allows human performers to have full controlover the melodic, harmonic, and expressive elements of theinstrument while mechanical actuators excite and dampenthe string with a rhythmic pattern. Guitarists can still ac-cess the fretboard without the physical interference of amechatronic system, so they can play melodies and chordsas well as perform bends, slides, vibrato, and other expres-sive techniques. Leveraging the capabilities of mechani-cal actuators, the mechanized hammers can output com-plex rhythms and speeds not attainable by humans. Fur-thermore, the rhythmic patterns can be algorithmically orstochastically generated by the hammer, which supportsreal-time interactive improvising.

Author KeywordsNIME, musical applications of robotics, augmented instru-ment, actuated instrument, electric guitar

ACM ClassificationH.5.5 [Information Interfaces and Presentation] Sound andMusic Computing, J.5 [Arts and Humanities] PerformingArts.

1. INTRODUCTIONRobotic musicianship is situated between two primary re-search areas — musical mechatronics and machine musi-cianship. Musical mechatronics focuses on sound produc-tion through mechanical means while machine musician-ship develops algorithms for music perception, composition,performance, and theory [1]. Integrating between thesetwo areas, researchers in the field of robotic musicianshipdesign robots with musical intelligence that can algorith-mically generate music in an effort expand and enhancethe creativity and expression of their human co-players.

Licensed under a Creative Commons Attribution4.0 International License (CC BY 4.0). Copyrightremains with the author(s).

NIME’17, May 15-19, 2017, Aalborg University Copenhagen, Denmark.

The Robotically Augmented Electric Guitar (RAEG) is de-signed to incorporate elements from mechatronic instru-ments and robotic musicianship, exploring the interactionthat is situated between playing a passive instrument andperforming with a robotic musician. Unlike other mecha-tronic guitars, RAEG allows a human performer to havefull control over the melodic, harmonic, and expressive ele-ments of the instrument, while collaborating and interact-ing with the mechanized hammer to produce the sound.Human guitarists can still access the fretboard without thephysical interference of a mechanical system, and can playmelodies and chords as well as perform bends, slides, vi-brato, and other expressive techniques while the mechani-cal actuators excite and dampen the string with a rhythmicpattern. While leveraging the complex and fast movementof actuators, the expressivity of performing the guitar is re-tained. Much like how robotic musician’s output can resultin an inspiring real-time interaction, algorithmically gener-ated rhythmic patterns played by mechanized portion of theRAEG can influence and surprise the performer.

2. RELATED WORKSIn the field of mechatronic instrument design, researchershave developed a number of mechanically actuated guitars,such as GuitarBot by LEMUR [4]. Most of these mecha-tronic guitars utilize a DC motor with plectrums for excit-ing the string. A few of these projects, like GuitarBot,consist of a sliding pitch shifter that manipulates pitch.Another commonality between these related instruments isthat they play music automatically without any human in-terference. For example, GuitarBot is controlled via MIDIfor performing live; performance is pre-programmed and au-tomatic. On the other hand, previous work in robotic mu-sicianship includes robots that perform alongside humans.Weinberg’s Shimon, an anthropomorphic robot, improviseson a marimba in response to a human performer’s inputwhile the robot’s output influences the performer [1]. Utiliz-ing complex algorithms, the generated musical phrases aredesigned to inspire human creativity. In the middle groundbetween mechatronic instruments and robot musicians arerobotic instruments with shared control. Much like Logan-Greene’s Brainstem Cello1, in which a mechanical systemmanipulates pitch or dampen strings in real time as a hu-man performs on a cello, robotically augmented instrumentsallow a human performer and robotic mechanisms to sharecontrol of the sound production [1]. Gurevich and Sheffieldexplored this field by designing a non-traditional instrumentthat is mechanically augmented and establishes distributedcontrol [2, 3]. Both instrument designers investigate sharedcontrol between human and mechanized motion within asingle instrument; their behaviors are influenced by each

1http://zownts.com/brainstem.html

487

other in this dynamic system. RAEG further explores thefield of shared control by incorporating robotics into an elec-tric guitar to promote the interaction of actuators and ahuman which then jointly produce sounds.

Figure 1: a) Full instrument, b) Damper, c) Ham-mer

3. DESIGN APPROACHThe design of RAEG retains the shape and size of a conven-tional electric guitar with enough space next to the bridgefor picking and strumming. The instrument contains sixmotorized hammers that hit the string while the guitaristmanipulates pitch and other parameters on the fret. A ma-jor advantage of integrating mechanical components intoa musical system is the ability to output musical phrasesthat are difficult or impossible to physically produce as ahuman performer. Mechanical sound production capabil-ities can allow musicians to explore speed, rhythm, har-monic, melodic, and timbre controls not attainable by hu-mans alone [1]. A servo motor was chosen to support thesefunctionalities, since unlike motorized plectrum in Guitar-Bot and other mechatronic guitars, it can also support dy-namic and timbre manipulation. The starting position ofthe hammer can affect the dynamics. In order to exploretimbral variations, the amount of damping can be con-trolled, which could affect the timbre similar to the palmmute technique. In addition, servos led to the motorizedhammer design, which produces “hammering” sounds thatare difficult for human to produce, especially on multiplestrings simultaneously. Furthermore, larger size and move-ment of the hammers provide a better visual cue for theplayers and audiences. The prototype of this robotic guitarwas designed to be simple in order to create an intuitive in-teraction. The software was also designed to be intuitive sothat the user can input rhythmic patterns with ease. Thesimple design for both hardware and software is crucial forexperimenting with the instrument for all users.

3.1 Hardware DesignThe guitar body houses six motorized hammers and six mo-torized dampers, controlled by an Arduino MEGA. A servomotor is used to mechanically move a 3D printed hammer tohit the strings. In order to enhance playing speed, a smallservo with low torque but high speed was chosen. Similar toGuitarBot, RAEG contains electro-mechanical dampers foreach string. The dampers are implemented in the design tomake sure that all the notes are pronounced cleanly, espe-cially when playing fast. This also allows for playing shortnotes, which are difficult to be performed by a human, whilethe amount of dampening can affect the dynamic and tim-bre.

3.2 Software DesignMax/MSP is used to program the patterns of hammer move-ments. The current design of the software consists of six se-quencers that allow a 16th note rhythmic patterns for eachstring to be inputted by users, who can easily click on thesequencers to input patterns in real time. Other parame-ters include BPM and activation time between the hammerand damper. In order to incorporate the crucial elementof robotic musicians, users can decide to let the computergenerate rhythmic patterns instead.

4. MODES OF INTERACTIONOne of the main interaction modes allows the hammers toperform a rhythmic pattern while the human guitarist isfretting a bassline or chord-progression with the left handand melody with the right hand. By leveraging the al-gorithmic nature of musical robots, patterns can be pro-duced stochastically or algorithmically. Due to these vary-ing rhythmic patterns, music with the same progression andmelody sounds different each time it is performed. Also, thepatterns can be generated in a way that a human would notnormally play or is too complex to execute. Furthermore,advanced users can explore rhythmic patterns that are hu-manly impossible to execute. For example, one hammercan perform a triplet, another hammer play a quintuplet,and another hammer play a 16th note pattern. While aseasoned finger-style player could perform this with enoughpractice, this system can immediately output this patternand also at a much faster speed than humanly possible. Asfor composing and improvising with the instrument, mu-sicians can input a pattern, let the hammers play it, andstart fretting to come up with chords and melodies. Theycan also decide to let the computer generate a pattern andthen perform along with generated patterns. The softwareallows users to choose if a new algorithmic pattern will begenerated every measure, every two measures, or every fourmeasures.

5. FUTURE WORKIn order to better compose, perform, and improvise withRAEG, a custom foot-pedal will be designed. This pedalwill be used to trigger various patterns, change the modeof interaction, and turn on/off the system. Furthermore, aresponsive feature will be added to the instrument. Becausethe guitarist can choose to pick and strum the strings, therhythm performed by the human can affect how the roboticcomponents generate rhythm.

6. REFERENCES[1] M. Bretan and G. Weinberg. A survey of robotic

musicianship. Communications of the ACM,59(5):100–109, 2016.

[2] M. Gurevich. Distributed control in a mechatronicmusical instrument. In Proceedings of NIME, pages487–490, 2014.

[3] E. Sheffield and M. Gurevich. Distributed mechanicalactuation of percussion instruments. In Proceedings ofthe international conference on New Interfaces forMusical Expression, pages 11–15. The School of Musicand the Center for Computation and Technology(CCT), Louisiana State University, 2015.

[4] E. Singer, K. Larke, and D. Bianciardi. Lemurguitarbot: Midi robotic string instrument. InProceedings of the 2003 conference on New interfacesfor musical expression, pages 188–191. NationalUniversity of Singapore, 2003.

488