SavantTony Wu – Jay Ni – Deri Kusuma

Automated music improvisation

1

Special Thanks to: Prof. Plummer and Cristina Popand our testers: Tim Lambert, Adriana Miu, Jon Lau, and You Li

Architecture

2

Algorithms

3

Music Representation

4

- Screenshot of Cakewalk Music Creator 3

Note

Time

Pitch

Note Representation

5Struck time

Pitch

Release time

Velocity = struck volume

Time

Volume (db)

Music Timing

6

=120

@ 120 beats per minute

1 tick ≈ 125 ms

1 beat ≈ 0.5 sec

1 measure

Tempo (beats per minute)

@ 4 ticks per beat

Logic

Architecture

7

GUI

Input buffer

Libraries and intermediate data structures

Output buffer

MIDI controllerMIDI input file

Input manager

Decision Engines

Analyzers

Improvisers

Output manager

Synthesizer

MIDI output file

Logic threadInput thread

Output thread

Visualization

Audible sound

GU

I thr

ead

Real-time Music Storage

8

Circular chunklistPress note O(1)

Release note O(1)

Range query O(A + n)

Time overhead O(n) per tick

Memory usage O(NL)

A: number of answersn: avg. number of notes per tickN: total number of notes storedL: avg. length of notes in tick

Chord analysis – basic chord types

9

These are like basic colors in painting

Other chords are simply enrichment / inversion / transposition of these basic feels

major minor dim aug dom half dim sus4C:

Enrichment Inversion Transposition (of C major)

Chord analysis – note domains

10

Important for generation!

C: I d: VIIb G: IVAffinity to A, B

Distance from F, Ab Affinity to F, Ab Affinity to F#

These are all C major chords:

Chord Analysis – Note Desirability

11

C major: Only C,E,G will sound good down here

F and F# only sound good with

high octaves

D,A,B sound good if not too

low

Higher pitch has less chord determining

capability

Chord Analysis – Capturing Richness

12

G dominant 7 b9 b13G

F

B

EbAb

Bass

Strong notes

Higher notes

Storing the note distribution of input captures richness beyond basic types (e.g. major, minor)

C, Db, D, E, F#, A, Bb Absent

Chord Analysis – Capturing Ambiguity

13

We compute a score for each possible chord interpretation, and then pick the

highest score to determine chord.

C Db D Eb E F F# G Ab A Bb B

M .93 … … … … .50 … … … … … …

m .47 … … … … … … … … .70 … …

dim -.44 … … … … … … … … .34 … …

aug -.76 … … … … … … … … … … …

dom .23 … … … … … … … … … … …

halfdim -.45 … … … … … … … … .48 … …

… … … … … … … … … … … … …

Decoupling Motif and Chord

14

C major

G 7

+

+

+ Bb minor 7 9 / F

=

=

=The same motif

Offline Motif Fitting

15

CBBbAAbGF#FEEbDDbCBBbAAbGF#FEEbDDbC

One-to-one pitch mapping algorithmwith dynamic programming Maximize objective

function:

REWARDchordal fitness as a function of motif and destination chord

PENALTYabsolute displacement ,relative displacement

TIME BUDGET60-120 ms per motif

Real-time Motif Fitting

16

Pitch

Time

Maximize objective function per assignment, given assistance level

REWARDchordal fitness as a function of destination chord,consistency to assignment history

PENALTYabsolute displacement,collision with assignment history for neighboring notes

TIME BUDGET< 1 ms per note

The present

input

mapped

Determining Key of Piece

17

HOW?The current key is determined by the discrete convolution of the input notes with each major / minor filter.

Performing Key Changes• Abrupt: Using dominant chord of new tonic• Smooth: Leverage current chord progression

C: I – V – vi – iii – F:I – V – vi - … (C:IV – I – ii - …)

Pleasant Chord Transitions

18

Chord State Machine Chord type heuristic

I M, M7, M9

II m7

III m, D7

IV M, M7, m

V D7

VI m, m7

VII dim, dim7, halfdim7

(From music theory)

Chord Phrasing

19

Forcing a phrase to end in a chord (e.g. V7) produces clearer phrase boundaries.

C: I – iii – vi – IV – ii – V7 – I – V7

If this isn’t possible…

C: I – iii – vi – IV – ii – I – iii – V7

Take 2 steps back!

C: I – iii – vi – IV – ii – – – V7

Savant Modes

20

Mode Description

Co-op Duet with human on harmony, human on melody. Melody is fitted to match the harmonic progression.

AI Co-op Duet with AI on harmony, human on melody. Melody is fitted to match the harmonic progression.

Remix Analyzes motif and chords of input, and generates a remixed version with motif and chord substitutes. User can play along and choose to influence direction of generation, or to have input fitted to chord progression.

Call & Response User plays a musical phrase, and computer will respond with an equal length phrase.

Empty No logic, just like a real piano!