Evolutionary Music Composition

Evolutionary Music Composition

Martin Dostal

Abstract. Evolutionary music composition refers to machine-based generation ofmusical pieces by means of evolutionary computation techniques. In this chap-ter we discuss machine-based music composition from several viewpoints. First,techniques used to machine-based music composition are reviewed. Second, mu-sic composition subtasks, such as generating melodies, harmonization of musicalphrases and generating rhythm patterns or composition of a rhythm accompanimentare introduced. Third, two distinct approaches to the evaluation of generated mu-sic are discussed: interactive evaluation based on human mentors judgement andautonomous evaluation of generated musical material by the system itself. The restof the chapter describes six recognized evolutionary systems for music composi-tion in detail. In particular, the description is focused to the design of evolutionaryalgorithms behind these systems.

1 Introduction

Evolutionary techniques have attained high popularity in machine learning and opti-mization. These techniques have a wide body of applications including less conven-tional areas such as machine-based creating of (art)works. However, the term art-work may seem as overstatement, depending on your definition on what an artworkreally is, so we can use rather the term machine creativity for machine generatedpaintings, music or poetry.

In this book chapter we discuss machine-based generation of music using evo-lutionary techniques. We assume that the reader is at least somewhat familiar withmusic, not necessarily as composer of performer, but at least as a consumer. Alsosome familiarity with evolutionary techniques is expected.

Martin DostalDept. Computer Science, Palacky University Olomouc, 17. Listopadu 12,77146 OLOMOUC, Czech Republice-mail: [email protected]

I. Zelinka et al. (Eds.): Handbook of Optimization, ISRL 38, pp. 935964.springerlink.com c Springer-Verlag Berlin Heidelberg 2013

936 M. Dostal

Although it may seem unmusical or somewhat provocative, music compositioncan also be understood as an optimization process. The human composer searchesin the space of all possible music compositions that he is able to compose, for onesuch that satisfies his own artistic or other criteria, such as a style, length or instru-mentation, for instance. Technically speaking, something like input constrains canbe more visible in performing applied arts; the artist works with some requirementsand constraints specified by the client and the artist is expected to create such an(art)work that meets the requirements. In particular, evolutionary techniques areused in music composition in such a way, that is, as a mechanism for creating orvariating musical pieces according to some specified criteria.

The chapter is organized as follows. First, music composition subtasks, such ascomposing melodic material, harmonization, or generating rhythm accompanimentwill be introduced and a brief survey of previous work in these fields will be pro-vided. Next section discusses commonly used techniques in machine-based musiccomposition. The following section discusses two distinct approaches used in cre-ative systems, namely interactive and autonomous systems. Remaining sections willintroduce some recognized evolutionary systems for music composition in detail.Used representation of musical knowledge, the design of evolutionary algorithmand recombination operators will be discussed. In melodic development, GP-Music,GenDash and GenJam will be described. The following section will describe Phon-Amnuaisuks system for harmonization. In generating rhythm accompaniment twosystems, namely CONGA and GeneticDrummer, will be described.

1.1 Brief HistoryHuman composers use very different approaches to music composition: from ratherformal, utilizing various musical knowledge and compositional theories, to intuitive,rehearsal-based composition of music. Not surprisingly, the techniques resemblingalgorithmic composition had been used yet prior to the rise of computers. For in-stance, Mozart proposed Musikalisches Wurfelspiel (Musical Dice Game), a sys-tem where a dice was used to random selection of short musical patterns that werecombined together [1]. Some other composers used choosing of random numbersin assembling their compositions, such as Johann Philipp Kirnberger in The EverReady Composer of Polonaises and Minuets (1757, 1783) or Joseph Haydn in thePhilharmonic Joke (1790). Reportedly, Mozart, Bach and Bartok used Fibonaccinumbers and golden section [36] in their compositions. However, it is somewhatquestionable whether the mathematical background was used on purpose or ratherintuitively by these composers.

A considerable growth of interest into computer-based composition (also includ-ing other areas of arts) has begun in late 80-ties of 20. century along with thewidespread popularity of machine learning and optimization techniques, especiallyknowledge-based systems, neural networks and genetic algorithms. One of the firstsuccessful attempts to utilize a computer in music composition was the Illiac Suiteby Hiller, Isaacson and Baker [1]. The computer was used to imitate the composition

Evolutionary Music Composition 937

process in a somewhat similar way to the later used evolutionary techniques. IannisXenakis, a Romanian-born Greek composer, was one of the pioneers of algorithmiccomposition. He used stochastic techniques in music composition and developed acomputer program that helped him in composition [53, 54].

1.2 Music Composition Subtasks

In this section we describe the main application areas of music composition systemsand provide a brief survey on previous work.

1.2.1 Melody (Motivic) CompositionAutomatic composition of musical pieces is apparently one of the most challeng-ing problems in machine art. In past twenty years several systems were developedand some of them also successfully used in the composition of true musical pieces(in contrast to experimental, or toy-like nature of some other systems) or live per-formances. Rodney Waschka II, an american composer known for usage of algo-rithmic composition methods, created a composition tool called GenDash [49, 50].GenDash creates variations of musical motives using a genetic algorithm. Waschkais one of composers that used his system successfully in publicly performed com-positions including Saint Ambrose, Sapphos Breath (both chamber operas) [51]or String Quartet: Ha! Fortune or Eclogues for Woodwind Quintet [50]. We descibeGenDash in more detail in Section 3. Jacob [31, 32] also experimented with creatingvariations of melodic motives using genetic algorithm. Jacobs system also consid-ered an ear module for listening and evaluating composed melodies in contrastto Waschkas system which omitted the fitness evaluation at all. GaMusic [39] wasa system for composing of two-octave melodies on the basis of interactive geneticalgorithm. Another attempt to evolve melodies using interactive genetic algorithmhas also been made by Ralley [40].

Johanson and Poli created the GP-Music system that evolves short melodic se-quences according to both mentors preferences and autonomous measures by thesystem itself [33]. GP-Music is a good example of a toy-like experimental system.The system is based on genetic programming and we provide detailed descriptionof the system in Section 2.

Spector and Alpern [42, 43] proposed the GenBebop; a system for composingshort jazz improvisations, the so called trading fours. Trading fours refers to four-bar improvisations on a reply to recently played four bars by another player. How-ever, generating bebop improvisations is only a particular application of Spector andAlperns approach. Their aim was more general: they attempted at a system whichwould enable a construction of artificial artists with some aesthetic tradition and par-ticular musical preferences. Similar application area has also been investigated byBiles who developed GenJam [8, 10, 11, 13]. It is one of the most referred evolution-ary composition systems so far. GenJam is an interactive genetic algorithm-basedsystem for trading-fours improvisation. The system is, however, able to improvisein real time, thus Biles used GenJam successfully at a lot of live performances with

938 M. Dostal

his Virtual Quintet, which is a combo of John A. Biles on trumpet or flugelhorn andGenJam on tenor sax and other instruments. GenJam also participated on severalBiles music records with the Virtual Quintet.

Thywissen [44] introduced GeNotator, a hybrid-algorithmic composition tool forcomputer assisted composition of music. Gibson and Byrne [21] designed a sys-tem for composing short musical pieces using diatonic, four-part Western harmony.The system has been called NEUROGEN and utilized a genetic algorithm to com-pose music and neural networks to evaluate the quality of generated results. Unemi[47, 48] created SBEAT, an interactive system for breeding short musical phrases(e.g., rhythm patterns) in terms of musical patterns with various rhythm, pitch andvelocity.

A distinctive part of music composition has been investigated by Horner andGoldberg who developed a system for thematic bridging [29]. Thematic bridging isa process that lies in transforming one musical part into another musical part overan interval of time.

1.2.2 Harmonization

Several researchers aimed at automated harmonization of musical phrases. Ebicoglu[20] developed CHORAL, a system for the harmonization of chorales in the styleof J.S.Bach. Phon-Amnuaisuk et al. developed a system for the four part (soprano,alto, tenor, bass) harmonization of user-specified melodies [5, 6]. The system wasbased on genetic algorithm and supported also an autonomous evaluation of gen-erated harmonies. We describe this approach in more detail in Section 5. Authorswere also interested in a comparison of methods that could be used to compose har-monic accompaniment. They have proposed a different version of the system; it wasbased on a rule-based inference system instead of genetic algorithm [6]. Accordingto experimental results, the authors concluded that the rule-based implementationproduced better results than the genetic algorithm-based one.

Horner and Ayers [28] have developed a system that composes a harmonic pro-gression using genetic algorithm using a rule-based approach. The rules are usedas constraints upon the search space of possible harmonies. The constraint rules areof two types: first type defines how individual chords can be voiced and the secondtype of rules defines how voices are allowed to change from a chord to next one[15]. The process of harmonization is executed consecutively for each of the groupof constraint rules. McIntyre [38] also used genetic algorithm to harmonization. Hissystem generates chord sequences according to the rules of how chords should bespelled and it is also used to autonomous evaluation of generated harmonies.

Hoover et al. [27] have proposed a system for harmonization of existing musi-cal phrases called scaffolds. The system produces harmonic variations and rhythmchanges using a combination of evolutionary computation and neural networks. Theneural network is responsible for the scaffold transformation in terms of used pitchesand rhythm.

Issues related to machine harmonization of music had been critically discussedby Wiggins [52]. Wiggins discusses various machine learning techniques used to


harmonization and points-out that the task of harmonization is not just a matterof filling of the notes according to the rules of harmonization but also creating amusical structure with a musical intention behind. The latter point is, however, nottaken into account in most systems that perform harmonization. In fact, a local per-forming of algorithmic composition systems (e.g., typically on the current measureonly) without a broader context, such as planning a structure of the composition,is the property of most current systems. For instance, GeneticDrummer is such alocally performing system. It generates a phrase with rhythm accompaniment byindividual measures. During the generating process the system does not see the pre-ceding nor following measures so that the composition process is entirely local. Theconsistence of resultant phrase is, however, ensured by the application of the sameconstrains (specified by the fitness function) on each measure present in the phrase.

1.2.3 Rhythm

Horowitz [30] used interactive genetic algorithm for composing short rhythm pat-terns. The system used also some fitness criteria (e.g., syncopation, density, down-beat, beat repetition) that were derived from the mentors evaluation. Tokui and Ibadeveloped a system called CONGA [48], an evolutionary drum machine that evolvesmulti-timbre rhythm patterns. CONGA is an interactive system extended with a neu-ral network which performs as an evaluation assistant. The system combines geneticalgorithms and genetic programming: individual measures are generated using ge-netic algorithm whereas the complete phrases are bred using genetic programming,see Section 6 for detailed description of the system. NEAT Drummer [26] evolvesrhythm patterns using a combination of neural networks and interactive evolutionarycomputation.

GeneticDrummer [1719] is one of a few evolutionary systems that perform au-tonomous evaluation of generated results using various fitness operators. The systemevolves a rhythm accompaniment played on a common drumset. Another differenceto the most of other existing systems is that the rhythm accompaniment is evolvedto an existing musical phrase in addition to users specified preferences. That is, thegenerated accompaniment should match the musical phrase to which is the rhythmaccompaniment being generated. We describe GeneticDrummer in detail in Sec-tion 7.

1.3 Methods and Techniques for Algorithmic CompositionIn the previous section we surveyed briefly application areas of algorithmic com-position. Another interesting point can be found in the employment of methodsand techniques behind algorithmic composition systems. Among widely used evo-lutionary techniques, such as genetic algorithms (GaMUsic, GenDash, GenJam,GaNotator, Neurogen, GeneticDrummer . . . ) and genetic programming (GP-Music,GenBeBop), various other techniques and methods has also been utilized in algo-rithmic composition. For instance, stochastic processes (including Markov chains)[2, 16, 53, 54] has been widely used before the rise of evolutionary techniques.

940 M. Dostal

Cybernetic composer [4] is a representative example of systems based on Markovchains. The system deduces the rhythm of a melody using Markov chains and com-poses a new melody at a later stage. Some attempts have also been made withchaotic, non-linear systems [23, 24].

Knowledge based systems have been successfully used in harmonization [20, 46].Harmonization as a musical task is supported with well grounded theory that can beapplied as a set of constrains and rules in a knowledge-based system. Some attemptshave also been made with grammar-based systems, see [34, 41] for more detail.

Neural networks have also attained certain popularity in algorithmic composi-tion. Neural networks have been used in several hybrid systems (typically with ge-netic algorithms or genetic programming) to assist the composition process [14, 21].In evaluation assistance tasks, neural networks learn responses from the mentor inan interactive system and later helps the mentor with recommendations or a pre-filtering the results [14, 33, 43, 45]. Another quite common usage of neural net-works is learning from input data such as a melody, a harmony [25] or a rhythm[26]. Baggi used also neural networks to composition of swing musical pieces [7].

Since this book chapter focuses on utilization of evolutionary computation tech-niques in various subtasks of music composition, the following text will discusssome representative evolutionary systems and the evolutionary techniques behindthese systems. Genetic algorithms and genetic programming are obviously the twomost popular evolutionary techniques used in algorithmic composition. Althoughthe basic shape of these algorithms behind algorithmic composition systems is prac-tically the same as in other more common or more formal applications of evolu-tionary computation, some differences can be identified. First, evolutionary musiccomposition systems typically use small populations, commonly between 10-50 in-dividuals. Interactive systems, as discussed above, use small populations owing totime consuming and demanding evaluation of individuals for the mentor. However,systems with autonomous evaluation of individuals also use relatively small popu-lations in comparison to other application areas of evolutionary computation. Ini-tial populations in music composition systems, if not generated randomly, oftenuse typical examples (e.g., phrases, patterns or motives) of a particular genre, styleor rhythm, for instance. These populations contain typically tenths of individuals.Broadly speaking, populations can be construed as an impression of ones musi-cal invention (similarly to viewing an audio sample of a drum as impression of thesound of such a musical instrument, for instance) and the evolutionary algorithmitself as a creative machinery responsible for the composers hard work. The crucialpart of evolutionary systems is the fitness evaluation. In musical systems, the fitnessfunction is either based on some musically meaning criteria or an evaluation by thementor, as it is discussed in Section 1.4 in more detail. The mentors evaluation isa quite specific property of many musical systems in a broad context of evolution-ary systems and their applications. Also mutation operators are in musical systemsoften designed to be musically meaning rather than purely random. The reason isobvious, the mutation should produce at least acceptable individuals. However theproduction of musically improper changes that breaks the rules may also be advan-tageous to some extent. The role of mutation in musical systems is often high in


comparison to occurrence of mutation in nature. Mutation is, beside crossover, thecrucial mechanism for creating variations of musical pieces.

Another important viewpoint for evolutionary techniques is the appropriatenessof individual techniques for particular musical subtasks. Seemingly, evolutionarytechniques can be great in musical tasks less constrained with complex rules. Typi-cally, evolutionary techniques seem very appropriate for evolving of rhythm accom-paniment. Why do we argue for that? The creation of appealing rhythm patterns orgrooves can be based on quite simple rules and the partial results can be creativelycombined into even better rhythms and patterns. Also establishing a rhythmicalcoordination between a musical phrase and a rhythm accompaniment generatedevolutionary is not that a complex task. On the other side, usage of evolution-ary techniques to harmonization seems less appropriate than using more formalapproaches, such as rule-based systems, for instance. Composing a harmony re-quires a wider and deeper musical knowledge in order to produce good sounding,musically correct harmonies. Similar observation holds for generating melodic mo-tives. In these musical tasks, a musically meaning structure for a musical piece ora musical intention behind the composition must be devised evolutionary, whichseems to be a much more difficult task than creating appealing rhythm patterns. Forcomposing melodic motives or harmonization, knowledge-based approaches seemmore appropriate.

1.4 Interactive versus Autonomous Systems

1.4.1 Interactive Systems

Interactive evolutionary systems use evolutionary algorithms as a technique tomachine-based generating or combining of compositions. However, one importantpart of the system, namely the evaluation of generated music, is not performed au-tonomously by the system itself but rather it is substituted by the decision of theuser. The user acts as a human mentor that assigns a rating that specifies the qualityof generated results. Typically, the quality is expressed by a number in a specifiedrange, or to simplify the process, it can be reduced to categorization of individu-als into keepers or losers. Keepers survive to next generation, whereas losers areremoved. The justification for using a human mentor is obvious: the mentor canevaluate the aesthetic quality of individuals that seems almost impossible to be eval-uated algorithmically. The mentor is also subjective, which could be an advantageor not, depending on the viewpoint.

One common difficulty in the practical use of interactive systems is the human fa-tigue. Basically, it has two dominant causes; first, the amount of evaluations is higheven in small populations. The mentor must evaluate each individual in each gener-ation of the composition process which is pretty time consuming. Second, listeningcarefully and critically to music fatigues quickly the mentor. Similarly, recordingengineers are faced daily with this phenomenon. It is not possible to hold long, con-tinuous recording or mixing sessions for the engineer. Arguably, the task for thementor is about the same arduousness as for the engineer.

942 M. Dostal

Interactive systems thus typically use small populations in order to keep theamount of evaluations admissible. Nonetheless, the task for the mentor is stilltime consuming and fatiguing. Several interactive systems address this issue witha machine-based mentor that helps the human in evaluation. These systems [14,33, 43, 45] typically use an artificial neural network that learns from the mentorsresponses and provide a preliminary evaluation or pre-filtering of results (e.g. au-tonomous identification of losers).

1.4.2 Autonomous Systems

These systems aims at the autonomous evaluation of generated music in terms ofquality. Though the autonomous evaluation of aesthetic quality or musical pleasing-ness seems not to be much viable for the computer, various quantitative or structure-based measures were devised to evaluate the music on such a basis. Autonomoussystems use typically heuristic rules, knowledge-based rules (typically coming fromthe music theory) or some kind of learning from the mentor as it was mentionedearlier. In evaluation of melodic material a variety of measures including horizontalintervals, melodic contours, musical staff density, rhythm structure or syncopationwere devised [3, 36, 38, 42, 44]. In evaluation of generated harmonies also a rule-based approach was successfully used [6, 52]. Rhythm seems to be quite good evalu-able using heuristic rules since the rhythm is to a large extent determined by struc-ture of durations. No such a complex problems arise in rhythm evaluation in com-parison to melodic or harmonic material. Also, the rhythm accompaniment seemsless subject to thematic development and thus can be reasonably evaluated at thelevel of individual measures instead of evaluating long musical phrases. Horowitz[30] evaluated syncopation, density, downbeat or beat repetition on the generatedshort rhythm patterns. Dostal [17] proposed fitness operators that evaluate rhythmconsonance or dissonance with a particular musical pattern, dynamic consonance orrhythm density of individual percussion instruments available in a drumset. Theseoperators are described in detail in Section 7.

Listening

1. Jacobss system for music composition. The webpage contains several samplesincluding a description.http://www.eng.umd.edu/

blj/algorithmic_composition/

2. GenBebop. The webpage contains a few jazz improvisations performed by theSpectors and Alperns system.http://hampshire.edu/

lasCCS/genbebop.html

3. SBEAT. Over twenty examples of various styles and length.http://www.intlab.soka.ac.jp/

unemi/sbeat/sample.html


2 GP-Music

Johanson and Poli created the GP-Music system that evolves short melodic se-quences according to both the mentors preferences and autonomous measures ofthe system itself [33]. GP-Music is based on genetic programming. Basically, thesystem is interactive, however the automated measures can be performed using anextended version of the system that combines genetic programming with a neuralnetwork. The network is trained using the mentors responses and performs an au-tonomous evaluation of generated music afterwards.

Individuals, that is musical sequences, are represented as program trees as widelyused in genetic programming. Terminals represent individuals notes (the followingsubset of chromatic scale is available: C-4 C#4 D-4 D#4 E-4 F-4 F#4 G-4 G#4 A-5A#5 B-5), chords (C-Chord, D-Chord, E-Chord, F-Chord, G-Chord, A-Chord, B-Chord) or rest. Notes have a constant duration. The function set contains operationsto constitute the sequence structure:

Play two (arity 2) concatenates two sequences together. Add space (arity 1) inserts a rest after each event present in the input sequence. Play twice (arity 1) repeats the sequence passed as parameter. Shift up (arity 1) shifts notes in the sequence one note up. Shift down (arity 1) shifts notes in the sequence one note down. Mirror (arity 1) reverses the original sequence. Play and mirror (arity 1) concatenates the original and reversed sequence

together.

Fig. 1 and Fig. 2 depicts an individual that represents the following sequence: [F-4rest C-4 rest F#4 C#4 rest C D].

(PLAY-TWO(PLAY-TWO(ADD-SPACE (PLAY-TWO F-4 C-4))F\#4)

(PLAY-TWO(ADD-SPACE C\#4)(PLAY-TWO C D)))

Fig. 1 Example of an individual in GP-Music

GP-Music uses standard genetic programming for evolving melodic sequences[35]. The mentor evaluates individuals on a 1-100 scale in each generation. Thusthe same individual may obtain a different fitness during the repeated evaluation bythe mentor. Automated measures are performed using a modified back-propagationneural network, see [33] for more detail about the network architecture.

944 M. Dostal

Fig. 2 Example of an individual in GP-Music

Listening

Authors provide several sound examples including a description of the parametersetting.http://graphics.stanford.edu/

bjohanso/gp-music/

gp_music-old.html

3 GenDash

GenDash [49, 50] is a compositional tool developed and used by american composerRodney Waschka II. The system itself is intended for composing musical pieces ina highly collaborative way with the human composer. Waschka successfully usedGenDash to compose complete musical pieces as well as to generate parts of piecesthat were further processed and arranged into musical pieces by the author.

GenDash is rather a music composition tool for the human composer than anautonomous music composition system. The system is intended primarily to cre-ate variations of measures of music but without the evaluation of generated musi-cal pieces by the human mentor or a fitness measure during the evolution process.It means that GenDash lacks one fundamental part of an evolutionary system andsuggests that it is a compositional tool which the composer uses to generate somemusical pieces that he afterwards deliberately decides to use it in a piece or discard.

The technique behind GenDash is basically a genetic algorithm without fitnessevaluation. That is, individuals to be reproduced to the next generation are selectedat random. Individuals represent measures of music. The initial population of in-dividuals is predefined by the user. For instance, Waschka used a population oftwenty-six measures of music drawn from typical Greek and Medieval songs for


composing Sapphos Breath, a one-act opera comprised of an overture and twelvescenes [51].

New individuals are created using standard one-point crossover between two par-ents. GenDash also allows for mutation. Two types of mutation are possible. Firsttype is a mutation performed by the system, such as changing a pitch or a durationof note. Second is the so called wildcard mutation that is based upon the altering ofthe individual by the human composer.

Waschka reported successful usage of GenDash to compose ten musical piecesincluding small- to medium-scale pieces up to 30-40 min duration. The authortweaked the system individually for each one composition. Some details on Gen-Dash usage on particular musical pieces can be found in [50, 51].

Listening

1. Waschka II, R. (1996). Empty Frames. Raleigh: Borik Press.2. Waschka II, R. (2000). Saint Ambrose. Raleigh: Borik Press.3. Waschka II, R. (2001). Sapphos Breath. Raleigh: Borik Press.

4 GenJamGenJam (a short for Genetic Jammer) is a system for generating jazz melodic impro-visations that has been developed by John A. Biles [8, 10, 11, 13]. The system is ableto improvise full-chorus solos and chase choruses in real time. Chasing choruses is astandard practice in jazz improvisation where soloists improvise on four-bar phraseson a tune in a way that a soloist chases the previously played phrase by anothersoloist by developing a new four-bar phrase on the basis of the previously playedone. In GenJam, the chorus chasing is performed between a four-bar phrase playedlive by the performer and GenJam which chases the performers improvisation. Thehuman performer plays a melody on an instrument that is converted in real-time tocorresponding MIDI events using the Roland GI-10 pitch-to-MIDI-converter. Thesedata are further processed by GenJam as it will be explained below.

GenJam is based on interactive genetic algorithm that breeds new melodies. Sincethe system is interactive, the evaluation of generated phrases is performed by the hu-man mentor. The mentor listens critically to generated results and provides on-goingfeedback on the perceived quality by evaluating phrases as good, bad or neutral. Asit was mentioned, the system interacts with the performer and the mentor. Addi-tionally, GenJam uses some other data required for the interactive genetic algorithmoperation:

Measure population that contains chromosomes. Each chromosome holds onemeasure long melodic phrase and its fitness.

Phrase population. Chromosomes in Phrase population contains the phrase fit-ness and identifiers to corresponding measure chromosomes that constitute thephrase.

946 M. Dostal

Rhythm sequence. It is a standard MIDI file containing usually three to fivetracks, typically piano, bass, drums and strings or guitar. The file is played backduring the live performance. It is used to evoke an illusion of a jazz quintet withtwo soloists, that is the human performer and GenJam, and accompanying rhythmtrio.

Head sequence. Similar to the rhythm sequence, the head sequence is a standardMIDI file that is played back during the live performance. The sequence containsthe original melody of a tune.

Chord progression. It contains the harmonic progression of the tune. The infor-mation about the harmonic structure is used by GenJam to ensure production ofharmonically correct notes.

Choruses. It provides information to GenJam how each of the choruses can behandled. In example, a full-chorus solo, chase chorus or a rest.

GenJam operates in three modes, learning, breeding or demo. Learning mode isused to evaluate phrases in population. Phrases are played-back in random orderand the mentor provides a feedback by responding g for good, b for bad, or a neutralrating is considered if the mentor does not provide any feedback on the phrase.Good evaluation from the mentor increases the fitness of the phase by one, whilebad evaluation decreases it by one. No response on the phrase does not affect itsfitness. Fitness values are bounded to interval < 30,30 >. More details can befound in [9].

Demo mode is used to select phrases for a live performance. The selection isperformed using a tournament selection. No feedback from the mentor is requiredin this mode.

Breeding mode is intended for creating new phrases using recombination opera-tions. In each application of the breeding mode one half of phrases in population isreplaced by newly bred phrases.

GenJam is designed to be able to operate on a live performance which is a quiteunique feature among music composition systems. Human player plays four barsof a melody that GenJam listens to. The playing is transformed to MIDI eventsand quantized into eight eighth-note-length intervals, depending on internal repre-sentation used by the genetic algorithm. In last thirty milliseconds of the humansfour bar phrase playing, the system breeds a new phrase. GenJam is aware about thebegin/end of the humans playing since the complete tune structure including thechord progression and accompanying tracks is predefined. In other words, impro-visations by the human and GenJam are performed on the beforehand determinedparts of the tune.

4.1 Population

GenJam uses two populations; a phrase population and a measure population.Both populations are of fixed size: phrase population contains 48 individuals andmeasure population holds 64 individuals. Individuals are encoded using bit-stringchromosomes.


A chromosome in phrase population encodes an identifier, fitness value and iden-tifiers of individuals in the measure population that constitute the phrase. For exam-ple, consider a phrase chromosome depicted on Fig. 3. The chromosome has iden-tifier 10, fitness -5 and the phrase is composed of four measure chromosomes withthe following identifiers: 20, 34, 11 and 23.

Fig. 3 Example of a phrase chromosome

A chromosome in measure population encodes its identifier (that is used to ref-erencing measure chromosomes in phrase chromosomes), a fitness value and one-measure-long melodic phrase in 4/4 time. A phrase is encoded in a series of eightevents, each representing one eighth-note-length event. These events are furthertransformed to MIDI events and played by the synthesizer. An event may representeither a note, a rest or a hold. Hold event has the effect of holding the previous event.For example, if a note event is followed by one hold event, the note will be playedwith quarter-note duration. Note events encode a relative note (pitch) of the event(see below). Since the events are internally represented as four-bit strings, GenJamis limited to 14 note scale at one time, 2 of 16 possible combinations are reserved tothe rest and hold events. The rest event is encoded in chromosome as 0, hold as 15and note events as 1-14. See Fig. 4 which depicts four chromosomes referenced inthe phrase chromosome from Fig. 3. Fig. 4 contains chromosomes with ids 20, 34,11 and 23 whose fitnesses are: -3, 12, -22 and 4.

Fig. 4 Examples of measure chromosomes

As it was stated above, note events encode relative notes while 14 different notesare available. These relative notes are mapped using predefined scale maps and thechord progression of the tune. Chord progression defines harmonic structure of thetune (limited to two chords per measure at maximum). A note map is an array ofpitches representing a scale appropriate to the given chord. Since the scale maynot be composed roughly of 14 notes, the scale is extended to 14 notes using scale

948 M. Dostal

repetition. For example, a scale consisting of [C D E G A Bb] would be extendedto [C3 D E G A Bb C4 D E G A Bb C5 D]. Biles synthesized the scale maps fromsome jazz educational literature. The resultant scale map is depicted in Table 1 [11].Another, a more complex map can be found in [10]. Using chord progression andscale maps, GenJam produces harmonically correct pitches only.

Table 1 Scale map

Chord Scale NotesCmaj7 Major (avoid 4th) C D E G A BC7 Mixolydian (4th) C D E G A BbCm7 Minor (avoid 6th) C D Eb F G BbCm7b5 Locrian (2nd) C Eb F Gb Ab BbCdim W/H Diminished C D Eb F F# G# A BC+ Lydian Augmented C D E F# G# A BC7+ Whole Tone C D E F# G# A#C7#11 Lydian Dominant C D E F# G A BbC7#9 Altered Scale C Db Eb E F# G# BbC7b9 H/W Diminished C Db Eb E F# G A BbCm7b9 Phrygian C Db Eb F G A BbCmaj7#11 Lydian C D E F# G A B

Obviously, this kind of representation is limited in several aspects. First, it can en-code only eighth-note multiplies. Second, there is not possible to encode a dynamicarticulation of notes. Third, there are only 14 pitches available at the time. Finally,the humans playing is quantized to eighth-note-length intervals which limits therhythmical expressivity of the system.

The populations are initialized at random at the outset of evolution. Each chro-mosome has assigned a fitness value equal to zero. In measure chromosomes, thestaff part is initialized with events. Each position is initialized either with a rest, ahold or a note event. The probability of a note event is set to 124 and the probabilityof a rest or and a hold event is set to 524 . Biles set the probabilities on the basis ofempirical observations.

4.2 Recombination Operators

In GenJam, selection and replacement of individuals is based on a modified tour-nament selection. A tournament is performed between four randomly selected indi-viduals (i.e. without regard to their fitnesses). Two of four individuals with highestfitnesses are selected as parents that undergo mating using crossover and mutation.These newly produced offsprings are used as replacement of the latter two individ-uals (those two with lowest fitnesses) in the tournament.

Crossover operation is a slightly different version of standard single-point cross-over. Crossover is performed at one randomly selected position in the event-part of


chromosomes. One of the offsprings then undergo mutation, while the other one iskept unmodified.

Mutation is the most interesting operation of recombination operations in Gen-Jam. It does not operate on a bit-level as many conventional, general-purpose mu-tation operations used in genetic algorithms, but rather at the level of individualevents. The point is to provide musically meaningful manipulation of measures en-coded in chromosomes. GenJam performs mutation on both phrase and measurechromosomes. On measure chromosomes, the following operations can be applied:

Identity. Returns identical, unmodified chromosome. Reversion. Produces a chromosome with reverse order of events. For example,

a reversion of chromosome [20 -3 14 13 14 12 15 9 0 0] would re-sult in [20 -3 0 0 9 15 12 14 13 14].

Right Rotation. The event-part of a chromosome is rotated by randomly chosennumber (between 1 and 7) of positions to the right. For example, a right rotationof chromosome [20 -3 14 13 14 12 15 9 0 0] of 3 positions wouldresult in [20 -3 9 0 0 14 13 14 12 15].

Inversion. Each event is changed to 15-. For example, an inver-sion of chromosome [20 -3 14 13 14 12 15 9 0 0] would result in[20 -3 1 2 1 3 0 6 15 15].

Ascending Sort. Events in chromosome are sorted in ascending order. For ex-ample, ascending sort of chromosome [20 -3 14 13 14 12 15 9 0 0]would result in [20 -3 0 0 9 12 13 14 14 15].

Descending Sort. Events in chromosome are sorted in descending order. For ex-ample, ascending sort of chromosome [20 -3 14 13 14 12 15 9 0 0]would result in [20 -3 15 14 14 13 12 9 0 0].

Transposition. This operation is performed on a randomly selected number ofsteps (between 1 and 4) on the direction of greater minimum distance betweenthe note-event and upper (i.e., 14) or lower (i.e., 1) bound. Hold and rest eventsare unaffected. For example, a transposition of chromosome [20 -3 14 1314 12 15 9 0 0] of 3 positions would result in [20 -3 11 10 11 912 6 0 0].

On phrase chromosomes the following operations can be applied:

Identity. Returns identical, unmodified chromosome. Reversion. Produces a chromosome with reverse order of events. For example, a

reversion of chromosome [10 -5 20 34 11 23] would result in [10 -523 11 34 20].

Right Rotation. The event-part of a chromosome is rotated by randomly chosennumber (between 1 and 7) of positions to the right.

Genetic Repair. Replaces the reference to the worst chromosome in phrase withrandomly selected another one.

Super Phase. Generates completely new phrase. On each position a tournamentof three measure chromosomes is performed and a reference to the winning chro-mosome is stored at the corresponding position in phrase chromosome. Note thatthis operation usually does not preserve any phrase from the parents.

950 M. Dostal

Lick Thinner. This operation replaces a reference to measure in the phrase chro-mosome that occurs in the phrase population most frequently. The reference isreplaced by another one selected at random. The role of the operator is to elimi-nate some occurrences of highly fit measures from the phrase population.

Orphan Phase. It operates opposite to the Super phase operation, that is the lessfit chromosomes are considered as winners from tournaments. This operation isintended to increase the diversity of population.

Since GenJam is interactive system, the evaluation cannot be performed in everygeneration of the breeding process. In Demo mode, only selection is performed. InBreeding mode, all recombination operators are applied. Fitness values are assignedinteractively by the mentor in Learning mode. Biles also experimented with a dif-ferent version of GenJam that eliminates the evaluation of generated results by thementor, see [12] for more detail. Biles and Anderson [14] have also made an attemptto enhance the interactive evaluation by the mentor with a neural network that is ableto discover worst individuals in population. However, the experiments did not resultin much success.

Listening

The webpage contains several nice music examples including a description.http://www.ist.rit.edu/

jab/GenJam.html

5 Phon-Amnuaisuks Harmonization System

Phon-Amnuaisuk et al. developed a system for harmonizing user-specified melodies[5, 6]. The system produces four-part (while one part is the input melody) homo-phonic tonal harmony using genetic algorithm. The genetic algorithm alsoautonomously evaluates generated harmonies, so that in contrast to interactive sys-tems, the evaluation does not rely completely on the user.

The system operates with four voices, soprano, alto, tenor and bass. Sopranovoice is assumed to be a melody voice. Chromosomes are represented in a matrix-like structure. Each chromosome contains five rows; the first four rows representsvoices (soprano, alto, tenor, bass) while the last row describes the duration of thestaff in individual voices. It follows that all voices have the same rhythm. Individualchromosomes represent four-voice phrases and the recombination operations oper-ate on the complete chromosome, not the individual voices.

Genetic algorithm uses standard single point crossover. Mutation of chromo-somes is possible using the following musically meaning operations: Peturb. Transposes alto, tenor and bass voices up or down by one tone or semi-

tone. The selection of transposition direction is done at random. Swap. The operation swaps two randomly picked voices (alto, tenor or bass).

It enables changing chords between open (open voicing is one that uses eachpart on the closest harmonizing note) and closed (closed voicing refers to anarrangement of the notes of chords within a narrow range) positions and changing


of inversions. Both open and closed harmony are common techniques for voicingin harmonization.

Rechord. It generates a new chord from the melody stored in the soprano voice.A chord is constituted by the note in soprano voice as a root note, and 3rd or 5th.

PhraseStart. Mutates the beginning of each phrase to start with tonic root posi-tion on a down beat.

PhraseEnd. Mutates the end of each phrase to end with a chord in root position.Obviously, crossover and mutation can produce harmonically unpleasant results.However, such individuals should be eliminated in the reproduction process.

The most interesting part of the system is the fitness evaluation of generated har-monic structures. Fitness evaluation is performed autonomously by the system. Fit-ness function utilizes common harmonization knowledge used in standard four-partharmony writing. Harmonization knowledge in the system captures various types ofpossible harmonic structures while some of them are preferred and some are not.The evaluation of harmonic occurrences is performed on two levels, within voicesand between voices. For instance, within voices a stepwise progression is preferredover large leaps, while dissonant chords and some kinds of leaps are penalized. Be-tween voices, occurrences of parallel unison, parallel octaves or progression fromdiminished 5th to perfect 5th, or crossing voices are penalized. More details on fit-ness evaluation can be found in [6].

Since the system evaluates only the harmonic movements between two adjacentchords, it is not possible to compose large-scale and sophisticated harmonic pro-gressions.

Authors also have developed a different version of the system which uses a rule-based inference engine instead of genetic algorithm. A comparison between thesystems has been made and the rule-based implementation has delivered substan-tially better results [6]. It is a quite presumable result, since the rule-based approachseems to be more appropriate for harmonization tasks than genetic algorithm. Theprocess of writing harmonies is a well examined part of music composition theory,but genetic algorithms provide rather an unstructured search in the problem domain.

6 CONGA

CONGA (COmposition iN Genetic Approach) [45] is an evolutionary drum ma-chine that generates multi-timbre rhythm patterns. To generating rhythms, a combi-nation of genetic algorithm and genetic programming is used. Genetic algorithm isused to breed individuals, usually one measure long rhythm phrases that are furtherarranged into 416 measures long rhythm patterns using genetic programming.

Basically, the system is interactive so that the evaluation of generated phrasesrelies on the mentor. Note also that the system generates rhythm patterns withoutany regard to other music, so that its purpose is much closer a drum machine ratherthan a rhythm accompaniment composition system. However, CONGA also utilizesa neural network that learns the mentors responses and provides an assistance inevaluation of individuals.

952 M. Dostal

6.1 Generating Phrases

Phrases are generated using genetic algorithm. Individuals in population representshort phrases, usually one measure or half-measure long ones. Individuals are repre-sented in two-dimensional chromosomes with a matrix-like structure. Rows repre-sent individual timbres (drum instruments) and columns represent individual eventsin the phrase. In fact, the representation is close to the way how the classic pattern-oriented sequencers (e.g., ROLAND TB-303) were programmed by the user. Eachcolumn corresponds to a predefined note duration according to the chosen resolu-tion of the system, e.g., one sixteenth or one eighth note. So that, according to theresolution and meter, the chromosome has the corresponding length, e.g, for 1/16note resolution and 3/4 meter, the chromosome length will be equal to 12. Eventsare described by integer values, 0 is used for rest, and 1-127 for note-events. Thevalue describes the strength of the stroke that corresponds to the velocity parameterused in MIDI devices. The number of timbers is also user-definable so that the chro-mosome will contain as many rows as the number of individual timbres. We depictan example of chromosome and corresponding notation on Fig. 5. The chromosomecontains three timbres, hi-hat (HH), snare (SD) and bass drum (BD). The resolutionwas set to 1/16 note. This simple representation, however, does not enable to cap-ture some other nuances of drum playing, such as different stroke types (e.g, doublestroke, flam of drag stroke) or possible instrumentations of particular drum timbers(e.g., head or rim stroke on drums, or bell or bow stroke on cymbals).

Fig. 5 Example of a phrase chromosome in CONGA

In CONGA, two crossover operations are available. First, standard one pointcrossover which exchanges selected parts between parental chromosomes and sec-ond, a crossover operation that exchanges selected timbers between the parents.Likewise many other genetic algorithm-based generative systems, GONGA usesseveral musically meaning mutation operators:

Random. Mutates selected chromosome positions with randomly selected values(0-127).


Rotation. Rotates a timbre in the chromosome. Reverse. Reverses a timbre in the chromosome. Timbre exchange. Exchange selected timbre parts with the selected chromosome.

6.2 Arranging Phrases to Patterns

Individual phrases generated by the above outlined genetic algorithm are arrangedinto 416 measures long patterns using genetic programming [35]. Thus, the pat-terns are represented as programs with a tree structure. Terminal nodes representreferences to individual phrases or the Random function, and the non-terminal nodesrepresent functions from the function set that constitute phrases. The following func-tions are available:

Sequence. Operates on two nodes. Plays NODE1 and NODE2 consecutively. Repetition. Operates on two nodes. Repeats NODE1 until the length of NODE1

is equal to length of NODE2. Concatenation. Operates on two nodes. Plays the first half of NODE1 and the

second half of NODE2 consecutively. Reversion. Operates on one node which is played in reverse order. Random. Plays a randomly selected phrase individual available in the pattern

individual.

Fig. 6 depicts an example of phrase in tree representation. Two phrases (i.e., I-4 andI-5) appear in the pattern chromosome that corresponds to the sequence [firsthalf of I-5] [second-half of I-5 or I-4] [I-4] [I-5].

Fig. 6 Example of a pattern chromosome in CONGA

6.3 Operating with CONGA

The user is allowed to parametrize CONGA in various aspects, such as populationsizes or length of generated patterns or used timbers. The system outputs to MIDIand can be synchronized with another MIDI devices using MTC (MIDI time clock).

954 M. Dostal

Using that, the mentor is able to evaluate the evolved patterns with consideration ofsome other music played by another MIDI devices.

As it was mentioned above, CONGA utilizes also a neural network that pro-vides an evaluation assistance to the mentor. A three-layer back-propagation net-work learns the responses from the mentor. The assistance is provided in terms ofpreselection of highly fit individuals to the mentor in order to reduce the psycholog-ical burden caused by evaluation of each evolved individual.

7 GeneticDrummer

GeneticDrummer is an evolutionary composer of rhythm accompaniment. The sys-tem has been developed between 2000 and 2002 by Dostal [17], the author ofthis chapter. The system composes a matching rhythm accompaniment to the givenmusical piece. Internally, it uses genetic algorithm that performs, contrary to mostevolutionary rhythm generators, autonomous quality evaluation of generated accom-paniment using a fitness function. The user, however, specifies the fitness criteria inorder to define the preferred aspects of resulting rhythm accompaniment. Genetic-Drummer listens also to one selected instrument part of the musical piece (e.g., abass-guitar or rhythm piano part) to which a rhythm accompaniment will be com-posed in order to produce an accompaniment that matches with the music. In Gene-ticDrummer, the aspect of musically meaning design of the system and the ability toperform autonomously is strongly emphasized. The system uses musically meaningrepresentation that captures typical drum playing techniques and instrumentationpossibilities of individual drums available in the system. Also, the design of muta-tion and fitness operations attempts to be musically meaning. These aspects will beintroduced in more detail later in this section.

7.1 Representation of RhythmIn GeneticDrummer, the representation enables to express fine nuances of drumplaying technical possibilities utilized by human drum players and also the indi-vidual instrumentation possibilities of percussion instruments. The representationcaptures a conventional drumset consisting of a snare drum, a bass drum, four tom-toms, a hi-hat, two crash cymbals, a splash, and a ride cymbal. Each drum instrumentenables some instrumentation techniques. For instance, on snare drum a head drumstroke, a rim stroke or a rimshot (that is a hit to the rim and head simultaneously) ispossible, whereas on bass drum only a kick pedal stroke is available. Another goodexample of instrumentation possibilities can be provided with cymbals. Ride cym-bal is usually played on the bow, edge or bell while the hi-hat is played on bow oredge. On hi-hat, the level of cymbals openness is controlled using a foot pedal. Allof these individual instrumentation differences are captured in GeneticDrummer.However, drum technique nuances utilized by human players are also particularlyimportant for musically meaning and flexible representation. When human drum-mer plays drums with drumsticks, it is possible (and also commonly used) to play


more than one note using one drum stroke. These kinds of drum strokes are great tocoloring the rhythm accompaniment with fine nuances. Fig. 7 depicts types of drumstrokes available in GeneticDrummer. That is, a drum stroke is the basic musical el-ement in GeneticDrummer, not a note as commonly used in other rhythm generationsystems. Note also, that this approach is commonly used in drum notation, wheremultiple drum strokes are identified by a particular symbol instead of writing-downindividual notes corresponding to the drum stroke type. In a pattern, drum stroketypes can be easily interchanged (e.g., single stroke to double stroke or vice versa)and thus used to produce naturally sounding rhythm variations easily.

Fig. 7 Drum stroke types available in GeneticDrummer

Drum strokes of individual drums or cymbals are encoded with integer valuescalled rhythm elements which are stored at individual positions in chromosomes. Arhythm element encodes a drum stroke type and a drum stroke volume. Drum strokevolume is specified on a six-point scale from piano to forte fortissimo. Each druminstrument has assigned an interpretation table that is used to encode and decoderhythm elements for individual drum instruments. For instance, Table 2 depicts theinterpretation table for the snare drum. There is also one special rhythm element:a rest, that is encoded with the zero value. Other interpretation tables and furtherdetails on rhythm elements can be found in [19].

956 M. Dostal

Table 2 Snare drum interpretation table

playing technique p mp mf f ff fffsingle drumhead stroke 11 12 13 14 15 16double drumhead stroke 21 22 23 24 25 26triple drumhead stroke 31 32 33 34 35 36drumhead flam stroke 41 42 43 44 45 46drumhead drag stroke 51 52 53 54 55 56drumhead ruff stroke 61 62 63 64 65 66single side-stick stroke 71 72 73 74 75 76

Individuals in a population are represented using multichromosomes. Multichro-mosomes hold one measure long rhythm patterns. A multichromosome is a liststructure composed of chromosomes. Each chromosome corresponds to a particularpercussion instrument and is composed of rhythm elements, see Fig. 8.

((chromosome representing splash cymbal)(chromosome representing crash cymbal 2)(chromosome representing crash cymbal 1)(chromosome representing ride cymbal)(chromosome representing hi-hat)(chromosome representing snare drum)(chromosome representing bass drum)(chromosome representing small tom-tom)(chromosome representing medium tom-tom)(chromosome representing large tom-tom)(chromosome representing floor-tom))

Fig. 8 Multichromsome structure

Chromosomes are encoded in a pattern-like style, that is each position corre-sponds to a particular duration depending on the internal resolution of the systemand the metre of multichromosome. The most commonly used internal resolution inmultichromosomes is one sixteenth-note. Using that resolution a chromosome with4/4 metre will contain 16 positions. Fig. 9 depicts a sample multichromosome in 4/4metre and the corresponding notation. The (empty) element represent a chromo-some composed of rests, that is (00 00 00 00 00 00 00 00 00 00 0000 00 00 00 00).

The pattern-oriented representation of rhythm is easy to use in computers, butit does not enable to encode triplets directly. To encode triplets we use a specialvalue 300 that is added to the rhythm element value if a note should be shifted to atriplet division. For instance, Fig. 10 depicts such an example of triplet coding. Thismethod of triplet coding was inspired with triplet coding used in pattern sequencers.


((empty)(empty)(empty)(empty)(14 00 14 00 14 00 14 00 44 00 74 00 24 00 15 00)(00 00 00 00 14 00 00 00 00 00 00 00 14 00 00 00)(14 00 00 00 00 00 14 00 14 00 00 00 00 00 00 00)(empty)(empty)(empty)(empty))

Fig. 9 Multichromosome and notation example

score:

chromosome:(14 314 314 0 14 314 314 0 14 314 314 0 14 314 314 0)

Fig. 10 Triplet coding example

As it was stated above, individuals in a population are encoded with multichro-mosomes. GeneticDrummer uses two populations, one for generating the accompa-niment (the so called groove populaion) and one for generating rhythm breaks (theso called break population). Rhythm breaks may appear at the last measure of agenerated phrase. Needless to say that as a groove population can also be used a setof individuals representing drum breaks, if the user has an intention to generate alonger drum solo.

From the musical standpoint, the population contains one measure long drum pat-terns of particular rhythm and drum-playing style. The populations can be composedof typical grooves of a particular human player, for instance. In this case, the Gene-ticDrummer is able to, to some extent, imitate the playing style of the human player.Another good approach to build-up populations, is to take a drum-practice book andrewrite typical patterns of particular rhythm into multichromosomes. It was actuallythe most frequently used approach to build-up the populations for experimentingwith GeneticDrummer. The last, a quite appealing, possibility is to record drumpatterns played live by the human drummer on drum pads using a MIDI drumtrigger module and afterwards transform these MIDI data into the format of

958 M. Dostal

multichromosomes. Unfortunately, the transformation must be done manually atthe present since the conversion from MIDI to multichromosomes is not yet imple-mented in GeneticDrummer.

7.2 Genetic Algorithm

Basically, GeneticDrummer utilizes the well known Simple Genetic Algorithm(SGA) [22] that is adjusted to the above introduced representation, musically mean-ing fitness and mutation. Standard two parent, one-point crossover is used to createnew individuals. Crossover performs on multichromosomes, that is each chromo-some within the multichromosome is crossed at the same position. Two offspringmultichromosomes are produced as a result of crossover. Reproduction is performedusing the standard roulette-wheel proportional selection.

Mutation and fitness is the crucial part of GeneticDrummers genetic algorithmthat is most responsible for producing acceptable results. Both mutation and fitnessare functions (written in CommonLisp as the GeneticDrummer itself) that performmutation of multichromosomes or fitness evaluation of multichromosomes, respec-tively. In mutation, several the so called mutation operators can be used. Mutationoperators are built-in functions that perform mutation at the level of chromosomes.Similar to mutation operators, a fitness function can utilize several built-in fitnessoperators. Fitness operators also operate on chromosomes. Thus, a fitness functionshould return a number between -100 and 100. The higher value is returned thehigher quality the multichromosome is. Fitness values above 100 are cut to 100 andvalues below -100, non-numeric values or evaluations that resulted into an error areset to -100. Since the majority of the fitness and mutation operators performs onchromosomes with regard to the selected track of accompanying music, the opera-tors usually take a chromosome, a measure of accompanying music and some othervalues specific to the particular operator as parameters. Some fitness operators, how-ever, does evaluate the chromosome according to the users specified ideal resultinstead of a measure of accompanying music. The fitness operators are following(the ai parameter represents a measure of accompanying music, chromosomerepresents a chromosome, that is a measure of a particular percussion instrument,ideal-value parameter represents the desired level of particular musical effectevaluated by the fitness operator, such as the rhythm density for instance. Ideal valueis a number between 0 and 1, where 0 represents no occurrence of a particular effectand 1 represent maximum occurrence of such effect): (rhythm-conformity ai chromosome) evaluates a rhythm consonance

between the chromosome and the accompanying instrument. This operator isused typically to evaluate a rhythm consonance between the bass drum and bassguitar, for instance. It is also great in generating solos.

(rhythm-unconformity ai chromosome) is an inverse operator torhythm-conformity. Rhythmically dissonant chromosomes are preferred.

(dynamic-conformity ai chromosome) evaluates the dynamic con-formity between the chromosome and accompanying music. This operator is


used to evaluate dynamic phrasing between drums and accompanying music orevaluating crescendos.

(instrument-quantity ideal-value chromosome) evaluates thedensity of drum strokes according the user specified ideal-value. The operatoralso takes care about multiple drum strokes, thus double strokes are consideredmore rhythmically dense than single strokes, for instance. This operator is usefulto let the user specify how dense drum breaks or bass-drum (e.g., double bassdrum cadences typical to heavy metal drums) patterns are preferred.

(instrument-quantity-on-strong-beats ideal-valuestrong-beats chromosome) operates similar to the previous operator, butthe evaluation is performed on the strong beats only. This operator is used forevaluating a syncopation.

Return values of the fitness operators are between 0 and 1. Zero means least suc-cess and 1 means maximum success. Mutation operators counterpart fitness oper-ators. The following mutation operators perform on individual positions (rhythm-elements) in chromosomes with a given probability. On each position a probabilistictest is performed in order to decide whether to apply the operator on the position orlet the position unchanged. The following mutation operators are available:

(mut-rhythm-element-change p chromosome) alters a drum stroketype with given probability p. For instance, it can alter single stroke with the flamstroke.

(mut-accentuate p ai chromosome) accentuates rhythm elements ac-cording to the rhythm of accompanying music with given probability p. Thisoperator is useful in generating breaks to imitate the phrasing of accompanyingmusic.

(mut-bass-teamwork p ai chromosome) improves the rhythm con-sonance between a chromosome (that is a particular percussion instrument) andaccompanying music. The operator is applied with a given probability p.

(mut-syncopize p ai chromosome strong-beats) performs sim-ple syncopation on the chromosome by means of moving a rhythm element froma strong beat one position forward or back. This operator is particularly used inswing tunes to imitate the effect of syncopation.

Fitness as well as mutation operators are described in much more detail [17] in-cluding the corresponding formulas. Fig. 11 depicts a typical example of a muta-tion function in GeneticDrummer. This mutation function with probability of 6%changes the rhythm elements of crash cymbal 1 (notes-cc-1) and bass drum(notes-bass) to the rhythm of accompanying instrument. This is typical for gen-erating the end of multi-measure breaks, when the break ends with some ostinatostrokes on cymbals and bass drum.

Fig. 12 depicts an example of a fitness function. This fitness function prefers alight accompaniment on snare notes-snare) and tom-toms (notes-tt-10,notes-tt-12, notes-tt-13) that rhythmically follows the accompanyinginstrument.

960 M. Dostal

(lambda (music)(mutation-bass-teamwork notes-cc-1 0.06 music)(mutation-bass-teamwork notes-bass 0.06 music))

Fig. 11 Mutation function example

(lambda (music)(+ (* 10 (instrument-quantity notes-snare 0.2))

(* 10 (instrument-quantity notes-tt-10 0.2))(* 10 (instrument-quantity notes-tt-12 0.2))(* 10 (instrument-quantity notes-tt-13 0.2))(* 20 (rhythm-conformity notes-snare music))(* 20 (rhythm-conformity notes-tt-10 music))(* 20 (rhythm-conformity notes-tt-12 music))(* 20 (rhythm-conformity notes-tt-13 music))))

Fig. 12 Fitness function example

One run of the genetic algorithm produces one measure long rhythm accompani-ment to one measure of accompanying music. The inputs to genetic algorithm arefollowing: a population, a fitness function, a mutation function, a measure of accom-panying music and the parameters applied to genetic algorithm such as the numberof generations, or starting position for generating a break. After passing the givennumber of generating the individual with highest fitness is returned as result.

The rhythm accompaniment to a complete phrase is generated by the repeatedapplication of the genetic algorithm with the same parameters except the measurewith accompanying instrument which is obviously set to the next measure beforegenerating next measure of rhythm accompaniment. After generating all measuresof the rhythm accompaniment, the genetic algorithm is executed once again in or-der to generate a break. A break population is used instead the groove populationand also the fitness and mutation is defined specifically for generating a break. Af-terwards the last measure of generated groove and generated break are combinedtogether according to the system settings: a user can set at which bar should thebreak start.

7.3 Interaction with GeneticDrummer

To generate a rhythm accompaniment to a complete tune, the user is obliged to selectan accompanying instrument to which the drum accompaniment is to be generated.Then the user cuts the track with the accompanying instrument into phrases of arbi-trary length. Typically, the phrases correspond to verses and choruses. Phrases canhave different length. For each phrase, the user defines parameters to the geneticalgorithm, as it was described earlier. The real interaction with GeneticDrummer


is relatively easy for the user. GeneticDrummer communicates with the CakewalkProAudio, a well known DAW application.

The author is developing a new, more advanced version of GeneticDrummer thatis about to be introduced in 2012.

Listening

GeneticDrummer has been used in various genres, such as jazz, rock or latin. Note,that the below presented results represent a rough result from the GeneticDrummerand thus they were not edited afterwards by the user in any way, except for soundprocessing.

1. Chicken, funk classic by Pee Wee Ellis.http://dostal.inf.upol.cz/data/mp3/chicken-2011.mp3The bass guitar part was used as music accompanying track to which the drumaccompaniment was evolved.

2. A Night in Tunisia, a jazz standard written by Dizzy Gillespie.http://dostal.inf.upol.cz/data/mp3/nit.mp3

3. Blue Bossa, an instrumental jazz tune written by Kenny Dorham.http://dostal.inf.upol.cz/data/mp3/bluebossa.mp3

4. Summertime, from Porgy and Bess opera written by George Gershwin.http://dostal.inf.upol.cz/data/mp3/summertime.mp3

5. Sockshop, a typical rock and roll tune.http://dostal.inf.upol.cz/data/mp3/sockshop.mp3Used groove population:http://dostal.inf.upol.cz/data/mp3/sockshop-groove-pop.mp3Poulation used for breaks:http://dostal.inf.upol.cz/data/mp3/sockshop-break-pop.mp3.

8 Conclusion

Evolutionary music composition has become widely used and popular techniquein algorithmic composition. Seemingly, the interest of researchers is growing; thenumber of various conferences, workshops and books on algorithmic compositionis growing. Also, there are artists interested in utilization of algorithmic compositiontechniques in their artworks. Some music composition systems, such as GenDash orGenJam have appeared in various compositions or live performances. GenJam hasbeen used on around 300 various jazz tunes. The system is also used by the authorin his VirtualQuintet on various recordings, tutorials and live gigs.

However, there is a field of problems that require further systematic investigationincluding finding promising representation, measurement of aesthetic criteria or de-veloping a theoretical background on generative art. McCormack in his work [37]formulates and discusses five open problems in evolutionary art in detail.

962 M. Dostal

References

1. Alpern, A.: Techniques for algorithmic composition of music (1995)2. Ames, C.: The markov process as a compositional model: A survey and tutorial3. Ames, C.: Quantifying musical merit. Interface 21, 5393 (1992)4. Ames, C., Domino, M.: Cybernetic composer: an overview, pp. 186205. MIT Press,

Cambridge (1992)5. Amnuaisuk, S.P., Tuson, A., Wiggins, G.: Evolving musical harmonisation. In: Proceed-

ings of the 4th International Conference on Artificial Neural Networks and Genetic Al-gorithms, Springer (1999)

6. Amnuaisuk, S.P., Wiggins, G.A.: The four-part harmonisation problem: A comparisonbetween genetic algorithms and a rule-based system. In: Proceedings of the AISB 1999Symposium on Musical Creativity, pp. 2834. AISB (1999)

7. Baggi, D.L.: Neurswing: an intelligent workbench for the investigation of swing in jazz.Computer 24, 6064 (1991)

8. Biles, J.A.: GenJam: A genetic algorithm for generating jazz solos (1994)9. Biles, J.A.: Genjam populi: Training an iga via audience-mediated performance (1995)

10. Biles, J.A.: Interactive genjam: Integrating real-time performance with a genetic algo-rithm. In: Proceedings of the 1998 International Computer Music Conference, ICMC1998 (1998)

11. Biles, J.A.: Life with genjam: Interacting with a musical iga. In: Proceedings of the 1999IEEE International Conference on Systems, Man, and Cybernetics (1999)

12. Biles, J.A.: Autonomous GenJam: Eliminating the fitness bottleneck by eliminating fit-ness (2001)

13. Biles, J.A.: GenJam: evolution of a jazz improviser, pp. 165187. Morgan KaufmannPublishers Inc., San Francisco (2002)

14. Biles, J.A., Anderson, P.G., Loggi, L.W.: Neural network fitness functions for a musi-cal ga. In: Proceedings of the International ICSC Symposium on Intelligent IndustrialAutomation (IIA 1996) and Soft Computing (SOCO 1996), pp. 3944. ICSC AcademicPress (1996)

15. Burton, A.R., Vladimirova, T.: Generation of musical sequences with genetic techniques.Computer Music Journal 23(4), 5973 (1999)

16. Camboropoulos, E.: Markov chains as an aid to computer assisted composition. MusicalPraxis 1(1), 4152 (1994)

17. Dostal, M.: Genetic algorithms as a model of musical creativity - on generating of ahuman-like rhythmic accompaniment. Computers and Artificial Intelligence 24(3), 321340 (2005)

18. Dostal, M.: The genetic drummer plays funk! In: ArtEscapes: Variations of Life in theMedia Arts, Catalogue of the EvoMUSART 2007 Exhibition (2007)

19. Dostal, M.: Towards representation of rhythm in genetic algorithm. In: New Trends inArtificial Intelligence, Proceddings of EPIA 2007, 13th Portuguese Conference on Arti-ficial Intelligence. Associacao Portuguesa para a Inteligencia Artificial (APPIA) (2007)

20. Ebcioglu, K.: An expert system for harmonizing four-part chorales, pp. 385401. MITPress, Cambridge (1992)

21. Gibson, P.M., Byrne, J.A.: NEUROGEN: musical composition using genetic algorithmsand cooperating neural networks. In: Second International Conference on Artificial Neu-ral Networks, pp. 309313. IEEE, New York (1991)

22. Goldberg, D.E.: Genetic Algorithms in Search, Optimization and Machine Learning, 1stedn. Addison-Wesley Longman Publishing Co., Inc., Boston (1989)


23. Harley, J.: Algorithms adapted from chaos theory: Compositional considerations.In: Proc. of the 1994 International Computer Music Conference, Aarhus, Denmark,pp. 209212 (1994)

24. Herman, M.: Deterministic chaos, iterative models, dynamical systems and their appli-cation in algorithmic composition. In: Proceedings of the International Computer MusicConference (1993)

25. Hild, H., Feulner, J., Menzel, W.: Harmonet: A neural net for harmonizing chorales in thestyle of j. s. bach. In: Moody, J.E., Hanson, S.J., Lippmann, R. (eds.) NIPS, pp. 267274.Morgan Kaufmann (1991)

26. Hoover, A.K., Rosario, M.P., Stanley, K.O.: Scaffolding for interactively evolving noveldrum tracks for existing songs. In: Proceedings of the 2008 Conference on Applicationsof Evolutionary Computing, Evo 2008, pp. 412422. Springer, Heidelberg (2008)

27. Hoover, A.K., Szerlip, P.A., Stanley, K.O.: Interactively evolving harmonies throughfunctional scaffolding. In: Proceedings of the 13th Annual Conference on Genetic andEvolutionary Computation, GECCO 2011, pp. 387394. ACM, New York (2011)

28. Horner, A., Ayers, L.: Harmonisation of musical progressions with genetic algorithms.In: Proceedings of the 1995 International Computer Music Conference, pp. 483484.ICMA, San Francisco (1995)

29. Horner, A., Goldberg, D.E.: Genetic algorithms and Computer-Assisted music composi-tion. In: International Conference on Genetic Algorithms, pp. 437441 (1991)

30. Horowitz, D.: Generating rhythms with genetic algorithms. In: Proceedings of the 1994International Computer Music Conference, pp. 142143. ICMA, San Francisco (1994)

31. Jacob, B.L.: Composing with genetic algorithms. In: International Computer Music Con-ference (January 1995)

32. Jacob, B.L.: Algorithmic composition as a model of creativity. Org. Sound 1, 157165(1996)

33. Johanson, B., Poli, R.: GP-music: an interactive genetic programming system for mu-sic generation with automated fitness raters. In: Proceedings of the Third InternationalConference on Genetic Programming (1998)

34. Keller, R.M., Morrison, D.R.: A grammatical approach to automatic improvisation.Technical report

35. Koza, J.R.: Genetic programming - on the programming of computers by means of nat-ural selection. In: Complex Adaptive Systems. MIT Press (1993)

36. Livio, M.: The Golden Ratio: The Story of Phi, the Worlds Most Astonishing Number.Broadway Books (2002)

37. McCormack, J.: Open Problems in Evolutionary Music and Art. In: Rothlauf, F., Branke,J., Cagnoni, S., Corne, D.W., Drechsler, R., Jin, Y., Machado, P., Marchiori, E., Romero,J., Smith, G.D., Squillero, G. (eds.) EvoWorkshops 2005. LNCS, vol. 3449, pp. 428436.Springer, Heidelberg (2005)

38. McIntyre, R.A.: Bach in a box: The evolution of four part baroque harmony using thegenetic algorithm. In: International Conference on Evolutionary Computation, pp. 852857 (1994)

39. Moore, J.H.: Gamusic: Genetic algorithm to evolve musical melodies (1994)40. Ralley, D.: Genetic algorithms as a tool for melodic development. In: Proceedings of the

International Computer Music Conference, Banff, Canada, pp. 501502 (1995)41. Roads, C.: Grammars as representations for music. In: Roads, C., Strawn, J. (eds.) Foun-

dations of Computer Music, pp. 403442. MIT Press, Cambridge (1985)

964 M. Dostal

42. Spector, L., Alpern, A.: Criticism, culture, and the automatic generation of artworks. In:AAAI 1994: Proceedings of the Twelfth National Conference on Artificial Intelligence,vol. 1, pp. 38. American Association for Artificial Intelligence, Menlo Park (1994)

43. Spector, L., Alpern, A.: Induction and recapitulation of deep musical structure. In: Pro-ceedings of the IFCAI 1995 Workshop on Artificial Intelligence and Music, pp. 4148(1995)

44. Thywissen, K.: Genotator: an environment for investigating the application of geneticalgorithms in computer assisted composition. In: Proceedings of the 1996 InternationalComputer Music Conference, pp. 274277. ICMA, San Francisco (1996)

45. Tokui, N., Iba, H.: Music composition with interactive evolutionary computation. In:Proceedings of the Generative Art International Conference, Milan, Italy (2000)

46. Tsang, C.P., Aitken, M.: Harmonizing music as a discipline of constraint logic program-ming. In: Proceedings of then International Computer Music Conference (1991)

47. Unemi, T.: A design of genetic encoding for breeding short musical pieces. In: Workshopon Artificial Life Models for Musical Applications II: Search for Musical Creativity,pp. 2529 (2002)

48. Unemi, T.: Sbeat3: a tool for multi-part music composition by simulated breeding. In:Proceedings of the Eighth International Conference on Artificial Life, pp. 410413. MITPress, Cambridge (2003)

49. Waschka, R.: Avoiding the fitness bottleneck using genetic algorithms to composeorchestral music. In: Proceedings of the International Computer Music Conference,pp. 201203 (1999)

50. Waschka, R.: Composing with Genetic Algorithms: GenDash, ch. 6, pp. 117136.Springer (2007)

51. Waschka, R.: Theories of evolutionary algorithms and a new simplicity opera: Makingsapphos breath. In: Artificial Life Models for Musical Applications, pp. 7986 (2011)

52. Wiggins, G.A.: Automated generation of musical harmony: whats missing? In: Pro-ceedings of the International Joint Conference in Artifical Intelligence 1999, IJCAI 1999(1999)

53. Xenakis, I.: Formalized music thought and mathematics in composition. Indiana univer-sity press (1971)

54. Xenakis, I.: Formalized music: thought and mathematics in composition. Harmonologiaseries. Pendragon Press (1992)

Evolutionary Music CompositionIntroductionBrief HistoryMusic Composition SubtasksMethods and Techniques for Algorithmic CompositionInteractive versus Autonomous Systems

GP-MusicGenDashGenJamPopulationRecombination Operators

Phon-Amnuaisuk's Harmonization SystemCONGAGenerating PhrasesArranging Phrases to PatternsOperating with CONGA

GeneticDrummerRepresentation of RhythmGenetic AlgorithmInteraction with GeneticDrummer

ConclusionReferences

Documents

Evolutionary Music Composition