1

Playpens, Fireflies and Squeezables: New Musical

Instruments for Bridging the Thoughtful and the Joyful

Gil Weinberg

Final Draft – will be published in Leonardo Music Journal December 2002

ABSTRACT The author discusses research in music cognition and education indicating that novices and untrained students perceive and learn music in a fundamentally different manner than do expert musicians. Based on these studies, he suggests implementing high-level musical percepts and constructionist learning schemes in new expressive musical instruments that would provide thoughtful and joyful musical activities for novices and experts alike. The author describes several instruments---the Musical Playpen, Fireflies and Squeezables---that he has developed in an effort to provide novices with access to rich and meaningful musical experiences and recounts observations and interviews of subjects playing these instruments.

Gil Weinberg is a doctoral candidate in the Hyperinstrument group at the Massachusetts Institute of Technology Media Laboratory. He holds an undergraduate degree from Tel Aviv University in music and computer science and a graduate degree in media arts and sciences from MIT. His work, which was recently featured in Ars Electronica, focuses on designing and building networked musical instruments for novices and experts. Weinberg's latest composition, Nerve, premiered in February 2002 with the Deutsches Symphonie-Orchester Berlin.

Gil Weinberg (composer, researcher), MIT Media Laboratory, 20 Ames

Street E15-445, Cambridge, MA 02139, U.S.A. E-Mail: <gili@media.mit.edu>.

2

The 20th century saw the diffusion of a number of powerful musical ideas from the "high-art" academic world into popular music. Varèse’s exploration of timbre, Reich’s minimalism and Cage’s happenings are some examples of pioneering 20th-century high-art musical experimentations that were diffused into the wider world and contributed to the formation of popular electronic music as we know it today. Musicians such as Frank Zappa, Philip Glass and Michael Nyman brought ideas and trends from both worlds into their music. At the same time, technical innovations such as FM synthesis grew from laboratory experiments into commercial products, providing novices with the power to create and manipulate sounds in a manner that had previously been accessible only to a small group of experts. These recent cases, as well as such earlier examples as the popular reach of opera in the 17th and 18th centuries, contradict the notion of an inherent separation between what is usually considered popular music for the masses and what many regard as serious or high-art music for the elite. There are no intrinsic reasons why rich and thoughtful “high-art” musical ideas and experiences should not be available to and enjoyed by novices and the general public, nor why the unique expressive and emotional quality that is usually affiliated with popular music should not be part of the high-art experience. However, when investigating fields such as music perception and music education, we do find strong evidence that novices and experts perceive and relate to music differently [1] and that beginners learn music in a fundamentally different manner than do advanced students [2]. Other studies suggest that this differentiation also bears social implications, stressing that the remarkable proliferation in the consumption of popular music in everyday life tends to manifest itself as incidental listening, passive participation and utilitarian consumption, such as music in shopping malls or aerobics classes [3]. These experiences, while they expose more people to more music for longer periods of time, tend to lack the rich and thoughtful aspects of concentrated listening and active creation of music. Fields of Study In an effort to combine thoughtfulness and expression in musical experiences that are accessible to all, I have investigated research in cognition and perception, music education and instrument interaction, which I believe can provide several possible directions for bridging these different modes of perception and learning. Cognition and Perception David Smith has presented a number of studies that show how a significant number of musical percepts, which are regarded as fundamental and obvious by expert musicians, are not perceived as such by novices [4]. For example, he shows that novices cannot perceive octave equivalence; they do not identify or categorize intervals, diatonic hierarchy or transposition, and do not follow structure and shape in the same way that experts do. Smith and Marla also conducted a quantitative study showing that while experts’ aesthetics focus on syntactic musical aspects, novices’ aesthetics stem much more from referential, sensual and emotional sources [5]. These studies encouraged me to explore what I regard as high-level musical percepts---composite musical elements such as stability, contour or tension---that have been proved to be perceived and understood by novices, but

3

that also bear a rich analytical core, which can intrigue the experienced musician. For example, various psychoacoustics studies show the perceptual significance of the general outline of a melody’s pitch curve, also known as melodic contour [6]. In one case, it was shown that novices were able to retain the contour of a semi-known melody much more easily than its specific pitches [7]. Trehub et al. demonstrated that melodic contour can be perceived by infants as young as 1 year old, strengthening the inference that this percept is well ingrained in human cognition [8]. These studies suggest that by providing an intuitive introduction to the playing and manipulation of contour, we can create a bridge between the expressive manner in which novices relate to melodic curves and the analytical manner in which experts perceive the lower-level relationships between pitches and intervals. I believe that providing novices with the power to create and phrase a melody by manipulating its contour, regardless of its exact pitches and intervals, offers them a unique creative experience that is usually reserved for experts and that can serve as an entry point for further investigations into more advanced concepts such as harmony and counterpoint. Another example of a high-level musical percept that can serve as an intuitive and expressive bridge for deeper musical investigations is stability. It has been shown, for example, that the cognitive perception of structural stability is influenced by musical parameters such as tempo, pitch commonality, dissonance and rhythmic variation [9]. Here too, an algorithm that would allow players to manipulate these parameters (and therefore control stability) can provide a unique expressive experience that could lead to deeper musical analyses. Similarly, an algorithmic implementation of the high-level percept of harmonic tension, based on Lerdahl’s tonal tension theory [10], can lead to a more detailed and analytical comprehension of harmony. Music Education With many musical education systems, such as Dalcroze [11], Orff-Schulwerk [12] and Suzuki [13], educators find it difficult to combine theory and technique with the expressive and emotional aspects of playing and creating music. Jeanne Bamberger addresses this difficulty, asserting that separating the formal from the expressive might alienate children from music making altogether [14]. Bamberger shows that preteens are inclined to process music in a “figural” manner, in which they tend to focus on “know-how”---intuitive aspects such as the global features of melodic fragments, the “felt” features of contour, rhythm and grouping, etc. Most education programs, however, require children in their early teens to process music in what Bamberger defines as the formal mode, in which musical notation, theory and analysis are abruptly introduced. As a result of this “know-that” approach certain important musical aspects that came naturally in the figural mode may be hidden, at least temporarily, when children try to superimpose formal knowledge upon figural intuitions. If this “crisis” is not acknowledged and the gap between the different modes is not negotiated, it can lead players to give up music altogether [15]. My approach for addressing this conflict is informed by Seymour Papert’s Constructionist Learning theory---an educational philosophy based on the notion that learning is most effective when the student constructs personally meaningful artifacts. The theory states that intuitive and emotional connections with personally created artifacts, students can construct their knowledge and obtain powerful theoretical ideas. Papert

4

emphasizes the ability of the computer to provide such personal and meaningful learning experience to a wide variety of learners. "Because it (the computer) can take on a thousand forms and can serve a thousand functions, it can appeal to a thousand tastes" [16]. Instrument Interaction I believe that children, novices and diverse audiences can gain access to rich and insightful musical phenomena through active participation in creating music. But in order to allow such access without compromising the figural intuitions, educators should focus on designing and building new musical instruments that can serve as expressive and enjoyable gates to deeper musical experiences. It is well known that children construct an important part of their knowledge about the world by interacting with instruments and physical objects of various kinds. Piaget, for example, showed the critical role of interaction with tangible objects in the development of human cognition. He demonstrated how processes such as the development of spatial locomotion, definition of the self, and abstract representation are connected to and enhanced by interacting with physical objects [17]. Various researchers at the Massachusetts Institute of Technology (MIT) Media Laboratory have built on these findings by exploring the cognitive effects of interacting with electronically and digitally enhanced physical objects. Papert focused on the idea that learning can be enriched by embedding technology in objects and allowing children to interact with the objects’ behavior, not only with their physicality [18]. Mitchel Resnick et al. concentrated on designing programmable digital toys that treated the players as collaborators in the design process and not just as “users” [19]. Tod Machover brought similar ideas to the musical realm by initiating the Hyperinstruments project. The project’s preliminary goal was to build “digitally expanded musical instruments in an effort to provide extra power and finesse to virtuosic performers” [20]. It was expanded later to the design of interactive musical instruments for non-professional musicians, students, music lovers and the general public [21]. Current Hyperinstrument research is attempting to push the envelope in both these directions by designing high-level professional systems that measure subtle and sophisticated human performance and by building interactive entertainment systems for novices and the general public. Instruments and Applications Based on the research in these fields of study, I have formulated a hypothesis suggesting that by embedding high-level percepts and constructionist learning schemes in intuitive and compelling musical instruments we can provide expressive and pleasurable musical experiences to children, novices and the general public without compromising richness, thoughtfulness and artistic depth. In order to address this hypothesis, I have developed three kinds of musical instruments---the Musical Playpen, Fireflies and Squeezables---in which I embedded algorithms for high-level musical control and constructionist learning in an effort to highlight the intuitive and expressive nature of some thoughtful musical experiences.

5

The Musical Playpen The Musical Playpen was the framework for my preliminary experimentations with designing and evaluating algorithms for high-level musical control [22]. I chose to design the instrument for toddlers and infants, so that I could investigate whether children that young can participate in a meaningful, active musical experience. I began by installing two high-level controllers, one for contour and the other for rhythmic stability, in an environment that the subjects would find familiar and fun regardless of any musical functionality---a 1.5-x-1.5-m playpen filled with about 400 colorful plastic balls. This particular musical playpen, however, did offer musical responses in correlation to children’s activity in the space. Players’ movements around the playpen propagated from ball to ball and triggered four piezo-electric accelerometers that were hidden inside four selected balls in each corner of the playpen. The balls’ ability to transmit hits to neighboring balls, combined with the accelerometers’ high sensitivity, allowed for almost any delicate movement around the playpen to be captured by at least one sensor. The analog signal was then digitized and sent to a Macintosh computer running Max [23], where it was mapped to musical output played from speakers below the playpen (Fig. 1). I mapped two of the four corners to control the musical contour of an Indian raga, so that the more energetic the players’ movements in these corners were, the higher the Indian raga pitches became. Children could therefore create melodic phrases and manipulate their curves by changing the intensity of their body movements in these corners. Player’s movements in the other two corners were mapped to an algorithm that controlled the tempo, rhythmic variation, and timbre of percussive sequences in an effort to provide access to controlling rhythmic stability. The more energetic the players were when near these corners, the higher the probability was for eighth notes, triplets, sixteenths and quintuplets to be added to the well-ordered quarter notes of the default pattern. The tempo curve also fluctuated more sharply (between 100 and 180 beats per minute), as did the rate of timbral change (the sounds used included those of bass drums, tablas, snares, and cymbals of various kinds). The observation sessions conducted with the playpen at MIT and at the Boston Children’s Museum from 1998 to 1999 produced a wide range of responses to the new instrument and the high-level musical control that it offered. For example, a 1-year-old infant started her session by triggering a sequence of notes as she was placed near one of the Indian-raga corners. The infant looked in the direction of the sound source and tried to move her hand towards that corner, seemingly trying to repeat the music she heard. When she succeeded and another melodic phrase was played, she smiled, took one ball and tried to shake it, obviously without audible results. Frustrated, she then threw the ball towards a rhythmic corner, generating a short percussive sequence. She approached this corner while moving her torso back and forth, laughing when discovering that her movements controlled the music. After stopping for a while, as if considering her next move, the infant started to slowly move her body again back and forth, gradually accelerating her movements, generating less and less stable percussive sequences. Only after repeating this behavior in another corner did the infant seem to be ready to use more expressive, less restricted gestures all over the playpen.

6

This constructive, almost analytical approach did not repeat itself with another 16-month-old toddler. The first play patterns that he demonstrated were turbulent movements all over the playpen, kicking and waving his arms, throwing balls all over and accompanying himself by singing and screaming joyfully. After this expressive explosion, the toddler gradually started to explore the different responses in the different corners around the playpen. He then performed several abrupt jumps from one corner to another. Towards the end of the observation, the toddler seemed to have developed a unique play pattern: His “compositions” included ecstatic random parts in the center of the playpen, which were interrupted by gentle exploratory parts near the corners (Fig. 2). In a environment, where he was placed in a playpen that was disconnected from its musical output, no organized play patterns were observed [24]. It can be seen then that for the infant, the controlled and restrained exploration led to more impulsive and joyful play, while for the toddler it worked the other way around---the spontaneous explosion led to more thoughtful and structured behavior. These responses strengthened my belief that with the right instruments and controls, young children can have access both to spontaneous, expressive music-making as well as to more serious and thoughtful musical explorations. The findings also encouraged me to develop a new set of instruments, which I entitled “Musical Fireflies”, in an effort to address older children and novices who are more cognitively developed and can discuss their impression of the experience. Musical Fireflies The Musical Fireflies [25] are palm-sized digital musical instruments, designed to introduce users to mathematical music concepts such as beat, rhythm and polyrhythm without requiring any prior knowledge of music theory or instruction. They were designed to provide a constructionist musical experience that would bridge between the figural and the formal learning modes, by offering players expressive and fun rhythmical experiences that can be easily transformed into analytical and formal exploration. The Fireflies allow players to construct their own rhythmic patterns, embellish the patterns in real time, synchronize patterns among different players and trade instrument sounds with their peers. For a single player, the instrument provide figural and formal familiarization with musical concepts such as accents, beats, rhythmic motifs and timbre. The multi-player interaction introduce players to more advanced musical concepts such as polyrhythm [26]. Each Firefly has two input buttons connected to a "Cricket" [27]---a Logo-programmable microprocessor responsible for processing, mapping, and communication. Using these buttons, players can enter rhythmic patterns that the microprocessor convert to Cricket Logo general MIDI commands and send through a serial bus port to a “MidiBoat” [28]---a small General MIDI circuit that is connected to a top-mounted speaker. All the electronic components are embedded in the Firefly’s 20-x-14-x-2.5cm 3D-printed case (Fig. 3). Interaction with the Musical Fireflies occurs in two distinct modes–--the Single Player mode, in which players convert numerical patterns into rhythmical patterns, and the Multi Player mode, where collaboration with other players enhance the basic patterns into polyrhythmic group compositions. In the Single Player mode, tapping the left and right buttons records accented and non-

7

accented notes respectively. After two seconds of inactivity, the Firefly plays back the entered pattern in a loop using a default tempo. This activity provides players with a tangible interface for entering and listening to the rhythmical output of any numerical pattern they envision. For example, the numerical pattern 4 3 5 2 2 would be entered and played back as follows: x y y y x y y x y y y y x y x y (loop) x = Accented note played by the left button. y = Non-accented note played by the right button.) During playback players can input a second layer of accented and non-accented notes in real time, using a different timbre. Each tap on a button plays a note aloud and recorded its quantified position so that the note becomes part of the rhythmic loop. When two playing Fireflies "see" each other (i.e. when their infrared signals are exchanged) they automatically synchronize their rhythmic patterns (a similar interaction occurs when real fireflies synchronize their light pulses to communicate in the dark). This activity provides participants with a richer rhythmical composition and allows for an informal introduction to polyrhythm. For example, if one Firefly play[s] a 7-beat pattern (x y y y x y y) and another plays a 4-beat pattern (x y y y), then the players can hear the process of divergence and convergence as the patterns go in and out of phase every 28 beats, the smallest common denominator (Table 1). While two Fireflies are synchronized, players can also initiate a "Timbre Trade," in which instrument sounds are traded between the devices (Fig. 4). Pressing either the left or right button trades both layers of the accented or non-accented timbre respectively. This allows for a higher level of musical abstraction, because the rhythmical patterns become separated from the specific timbre in which they were created. Because the interaction becomes richer as each Firefly gains more timbres, the system encourages collaborative play as participants become motivated to trade and collect timbres from their peers. Observations of play sessions with the Musical Fireflies during the year 2000 were followed by discussions with the players. Participants were asked about the expressive and educational aspects of the session as well as their suggestions for improvements. A Max-based software version of the application was prepared so that players would be able to compare the software version with the tangible interaction that they had with the physical Fireflies. Both novices and experienced players found the concrete aspects of playing with a physical object more compelling than the computer-based graphical user interface, mentioning that the unmediated connection with the instrument contributed to the feeling of personal connection to the music they created. Experts seemed to be most intrigued by the construction of group polyrhythmic compositions. Most novices, however, found it a bit confusing, suggesting that there should be a more moderate learning curve between single and multi-player modes. Other weaknesses mentioned were the lack of variety in rhythmic values and rests, the lack of continuous development of the recorded patterns and the inability to play in larger groups. These weaknesses are currently being addressed in the “Beatbug Network” project (See “Future Work” section below).

8

Squeezables In the Squeezables project [29], I attempted to combine all three aspects of my hypothesis---high-level musical controllers, constructionist interaction and a musical instrument that could provide expressive and intuitive control---into a complete artistic experience that would culminate in the performance of a musical composition. My goal was to allow a group of players, consisting of novices and experts, to interdependently collaborate in constructing a meaningful musical composition. The instrument, therefore, was made of six squeezable and retractable gel balls mounted on a small podium (Fig. 5), which Players could simultaneously squeeze and pull to manipulate a set of low- and high-level musical percepts. The combination of pulling and squeezing allowed players to utilize familiar and expressive gestures and to control multiple synchronous and continuous musical channels. Several materials were tested as means of such soft and expressive control. For the final prototype, I chose soft gel balls, which proved to be robust and responsive, providing a sense of force feedback control that derived from the elastic qualities of the gel. Buried inside each ball was a 0.5-x-2.0-cm plastic block covered with five pressure sensors that were protected from the gel by an elastic membrane. The analog pressure values from these sensors were transmitted to a digitizer and converted to MIDI. The pulling gestures were sensed by six variable resistors that were installed under the table. An elastic band connected to each ball added opposing force to the pulling gesture and helped to retract the ball back onto the tabletop (Fig. [6]). In order to better evaluate the high-level algorithms in the instrument, I decided to implement some straightforward mappings that controlled relatively low-level musical parameters. For example, one of the balls formed a one-to-one connection between squeezing and pulling to the modulation rate and range of two low-frequency oscillators, respectively. For other balls I developed higher-level algorithms to control percepts such as contour and stability. Based on Dibben’s findings, I mapped the gestures of pulling and squeezing of the “Arpeggiator” ball to control a combination of musical parameters, such as tempo, pitch commonality, dissonance and rhythmic variation, so that the more the ball was squeezed and pulled, the more unstable the arpeggiated sequence became. In order to provide a coherent constructionist interaction, I divided the balls into five accompaniment balls and one melody soloist. The five accompaniment balls were fully autonomous---no input from other balls influenced their output. However, these balls’ output not only was mapped to the accompaniment parameters but also significantly influenced the “melody” ball. While pulling the melody ball manipulated its own contour so that the higher it was pulled, the higher the melody pitches became, the actual pitches, as well as the MIDI velocity, duration and pan values, were determined by the level of pulling and squeezing of the accompaniment balls. This allowed the accompaniment balls to “shape” the character of the melody while maintaining a comprehensive scheme of interaction among themselves, leading to a real-time collaborative constructionist effort by all. I composed a short piece for three Squeezables players, which is based on the tension between the accompaniment balls and the melody ball that is shaped by them [30]. The piece, which was featured in Ars Electronica 2000, starts with a high-level of instability and builds gradually towards a repetitive rhythmic peak. Special notation was

9

created for the piece. Two continuous graphs were assigned for each one of the six balls. One graph indicated the level of squeezing over time and the other indicated the level of pulling (Fig. [7]). The process of writing and performing the piece (See Fig. [8]) served as a useful tool for evaluating the mapping and sensing techniques that were used. In addition, I held discussions with novices and professionals after they played with the instrument. In general, children and novices were more inclined to prefer playing the balls that provided high-level control such as contour and stability. They often stated that these balls allowed them to be more expressive and less analytical. Musicians, on the other hand, often found the high-level control somewhat frustrating, because it did not provide them with direct and precise access to specific desired parameters. Some experts complained that their personal interpretation of the high-level controllers for stability differed from the one that I implemented in designing the instrument. Both novices and professional players found the multiple-channel synchronous control expressive and challenging and the pulling and squeezing gestures comfortable and intuitive. These gestures allowed delicate and easily learned control of many simultaneous parameters, which was especially compelling for children and novices. The organic and responsive nature of the balls was one of the features mentioned as contributing to this expressive experience. When asked about the interdependent constructionist activity, one melody ball player described her experience as a constant state of trying to expect the unexpected. To another player, the experience felt almost like controlling an entity with a life of its own. Playing the accompaniment balls led to different responses. In doing so, players could control and manipulate the melody without being significantly influenced themselves. However, full constructionist collaboration with the other accompaniment players was essential for substantially affecting the melody. In a manner similar to chamber music group interaction, body and facial gestures served an important role in coordinating the accompaniment players' gestures and establishing an effective outcome. Such collaborations turned out to be especially compelling for children, who found the accompaniment balls conducive to social interaction, intuitive and easy to play with. Some complaints were made, however, regarding the difficulty for a specific player to significantly influence the melody without trying to coordinate such an action with the other accompanying players. Some players felt that this interaction prevented them from expressing their individual voices. Future Work The observations and interviews that I conducted were helpful in evaluating certain aspects of my hypothesis, such as whether a particular high-level controller proved to be effective, whether players learned an advanced musical concept that would have been inaccessible to them otherwise, or whether a specific gesture or material proved to be expressive and intuitive. Further cognitive and educational studies will be required in order to investigate psychological aspects such as the experience of pleasure, and the long-term transferability of the learned musical material. I plan to conduct such studies with a set of newly developed instruments called Beatbugs (Fig. [9]). The Beatbugs allow for a group of eight players to create, develop, and share rhythmic motifs through a simple interface [31]. I composed a piece for eight Beatbug players, entitled Nerve, for Tod

10

Machover's Toy Symphony [32], which was performed during 2002 with the Deutsches Symphonie-Orchester Berlin, [the Irish National Symphony Orchestra and the BBC Scottish Symphony Orchestra (Fig. 10)]. Week-long workshops that address educational and cognitive aspects were held before each concert with groups of local children, educators and professional musicians. I intend to publish an overview of these workshops and concerts in the future.

11

References 1.J.D. Smith, “Conflicting Aesthetic Ideals in a Musical Culture,” Music Perception <B>4<D> (1987) pp. 373--392. 2. J. Bamberger, "Intuitive and Formal Musical Knowing: Parables of Cognitive Dissonance," In S.S. Madeja (ed.), The arts, cognition and basic skills (St. Louis: Cemrel Inc., 1978) pp. 173—213. 3. T. DeNora, Music in Everyday Life (Cambridge, U.K.: Cambridge Univ. Press, 2000). 4. D.J. Smith, “The Place of Musical Novices in Music Science,” Music Perception <B>14<D>, No. 3, 227--262 (1997). 5. D.J. Smith and R. Marla, “Aesthetic Preference and Syntactic Prototypicality in Music: ‘Tis the Gift to Be Simple,” Cognition <B>34<D> (1990) pp. 279--298. 6. M.A. Schmuckler, “Testing Models of Melodic Contour Similarity,” Music Perception <B>16<D>, No. 3, 295--326 (1999). 7. J. Sloboda, The Musical Mind: The Cognitive Psychology of Music (Oxford, U.K.: Clarendon Press, 1987). 8. S.E. Trehub, D. Bull and L.A. Thorpe, “Infants' Perception of Melodies: The Role of Melodic Contour,” Child Development <B>55<D> (1984) pp. 821--830. 9. N. Dibben, “The Perception of Structural Stability in Atonal Music: The Influence of Salience, Stability, Horizontal Motion, Pitch Commonality, and Dissonance,” Music Perception <B>16<D>, No. 3, 265--294 (1999). 10. F. Lerdahl, “Calculating Tonal Tension,” Music Perception <B>13<D>, No. 3, 319--363 (1996). 11. Dalcroze Society of America web site, <http://www.dalcrozeusa.org> (2001). 12. B. Warner, Orff-Schulwerk: Applications for the Classroom (ENGLEWOOD CLIFFS NJ: Prentice-Hall, 1991). 13. S. Suzuki, Nurtured by Love (New York: Exposition Press, 1969). 14. J. Bamberger, “Growing up Prodigies: The Mid-Life Crisis,” New Directions for Child Development <B>17<D> (1982) pp. 61--78. 15. H. Gardner, Frames of Mind (New York: Basic Books, 1983) pp. 110--112. 16. S. Papert, Mindstorm (New York: Basic Books, 1980). 17. J. Piaget, The Principles of Genetic Epistemology (New York: Basic Books, 1972)

12

18. S. Papert, The Children's Machine: Rethinking Schools in the Age of the Computer (New York: Basic Books, 1993). 19. M. Resnick, F. Martin, R. Sargent and B. Silverman, “Programmable Bricks: Toys to Think With,” IBM Systems Journal <B>35<D>, Nos. 3--4, 443--452 (1996). 20. T. Machover, "Hyperinstruments: A Progress Report," (1992) p. 3. Available from the Media Laboratory of the Massachusetts Institute of Technology Unpublished. 21. T. Machover, the Brain Opera web site, <http://brainop.media.mit.edu> (2000). 22. G. Weinberg, "The Musical Playpen: An Immersive Digital Musical Instrument," Personal Technologies Vol. 3, No. 3, 132--136 (1999). 23. M. Puckette, "The Patcher," Proceedings of International Computer Music Association (San Francisco, CA: PUBLISHER?, 1988) pp. 420--429. See also – http://www.cycling74.com/products/max.html 24. View a video clip of children playing the Musical Playpen at <http://www.media.mit.edu/~gili/playpenclip.mov>. 25. G. Weinberg, T. Lackner and J. Jay, “The Musical Fireflies---Learning About Mathematical Patterns in Music through Expression and Play,” Proceedings of XII Colloquium on Musical Informatics 2000 (A'quila, Italy: Instituto Gramma, 2000). 26. S. Handel, “Using Polyrhythms to Study Rhythm,” Music Perception <B>11<D>, No. 4, 465--484 (1984). 27. F. Martin, B. Mikhak and B. Silverman, “Metacrickets---A Designer’s Kit for Making Computational Devices,” IBM System Journal <B>39<D>, Nos. 3--4, 795--815 (2000). 28. J. Smith, The MiniMidi web site, <http://www.media.mit.edu/~jrs/minimidi> (1998). 29. G. Weinberg and S. Gan, “The Squeezables: Toward an Expressive and Interdependent Multi-player Musical Instrument,” Computer Music Journal <B>25<D>, No. 2, 37--45 (2001). 30. Listen to the Squeezables performance at <http://www.media.mit.edu/~gili/publications/Squeezadelic.mp3>. 31. G. Weinberg, R. Aimi and K. Jennings, "The Beatbug Network---A Rhythmic System for Interdependent Group Collaboration," Proceedings of NIME 2002 (Limeric Ireland: University Of Limeric, Department of Computer Science and Information Systems, 2002). 32. T. Machover, the Toy Symphony web site, <http://www.toysymphony.org>. Manuscript received 28 December 2001.

13

Figure Captions Fig. 1. The Musical Playpen system. Signals from four piezo-electric sensors in four selected balls are digitized and sent to a computer to [activate] algorithms for melodic contour and rhythmic stability. MIDI commands from the computer are sent to a synthesizer that plays through four speakers under the playpen. (<c> Gil Weinberg) Fig. 2. A toddler playing the Musical Playpen. (Photo: Gil Weinberg) Fig. 3. The Musical Fireflies' electronics. Signals from two buttons are sent to the "Cricket" microprocessor within, which maps them to MIDI commands and sends them to a "MidiBoat” general MIDI unit. Audio from the MidiBoat is sent through an amplifier to an embedded speaker. (Photo: Gil Weinberg) Fig. 4. Two players synchronize their patterns and trade timbres through the Fireflies' infra-red port. (Photo: Gil Weinberg) Fig. 5. The Squeezables: Six squeezable and retractable gel balls mounted on a podium. (Photo: Gil Weinberg) Fig. 6. Playing the Squeezables: A combination of squeezing and pulling gestures by all players. (Photo: Gil Weinberg) Fig. 7. The Squeezables composition notation: Twelve separate graphs indicate the level of pulling and squeezing of each ball. (<c> Gil Weinberg) Fig. 8. A Squeezable performance. (Photo: Gil Weinberg) Fig. 9. The Beatbugs: A velocity sensitive piezo-electric trigger and two bend-sensor antennae allow for entering rhythmic motifs and continuously manipulating their pitch, timbre and rhythm. When connected to a central computer system 8 Beatbug players can share their rhythmic motifs with their peers and develop them into a full composition. (<c> Gil Weinberg) Fig. 10. “Nerve” for an interconnected network of Beatbugs, as performed with the BBC Scottish Symphony Orchestra. Glasgow June 2002.] (Photo: [Tod Machover]) Table 1. An example for a polyrhythmic exercise with the Musical Fireflies 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 7/4

x y y y x y y x y y y x y y x y y y x y y x y y y x y y

4/4

x y y y x y y y x y y y x y y y x y y y x y y y x y y y

A 7 beat pattern and a 4 beat pattern (played by two synchronized Fireflies respectively) diverge and converge every 28 beats.