Upload
melanie-walsh
View
222
Download
5
Tags:
Embed Size (px)
Citation preview
Algorithms for pattern discovery and pitch spelling in
music
David MeredithGoldsmiths CollegeUniversity of London
Overview of Research Interests
Music information retrieval• Managing musical data and retrieving useful information from it
Automatic music transcription• Computing a score from a recording of a musical passage
Computational music cognition and analysis
• Constructing computational models that extract structures that listeners hear and music analysts find interesting
Evaluation• sound methodologies and “gold-standard” test collections
Musical pattern discovery and pitch spelling
Musical pattern discovery•Finding themes and other perceptually important repeated patterns
•Useful for indexing in music information retrieval
Pitch spelling•Predicting the pitch names (e.g., C#4, B@3) of notes in a “piano-roll” representation (e.g., MIDI)
•Essential for transcribing music from MIDI (or audio) to notation
Uses of musical pattern discovery algorithms Indexing
• Store themes, motives and other memorable patterns in index to enable sub-linear retrieval times
Transcription and music analysis• Beat tracking and metrical structure analysis - similar patterns have similar metrical structure
• Grouping and phrasing - “parallellism” (Lerdahl and Jackendoff, 1983) most important factor in grouping
Composer’s assistant, automatic improvisation
• Cure composer’s block by suggesting new material based on patterns discovered in music already written
• Automatically create new music that develops themes discovered in music already played
Importance of repeated patterns in music analysis and cognition Schenker (1954. p.5):
•repetition “is the basis of music as an art”
Bent and Drabkin (1987, p.5):•“the central act” in all forms of music analysis is “the test for identity”
Lerdahl and Jackendoff (1983, p.52):
•“the importance of parallelism [i.e., repetition] in musical structure cannot be overestimated. The more parallelism one can detect, the more internally coherent an analysis becomes, and the less independent information must be processed and retained in hearing or remembering a piece”
Most musical repetitions are neither perceived nor intended
Interesting musical repetitions are structurally diverse Want to discover all and only interesting repeated patterns
Class of interesting repeated patterns is structurally diverse because
•patterns vary widely in structural characteristics
•many ways of transforming a musical pattern to give another pattern that is perceived to be a version of it• e.g., truncated, augmented, diminished, inverted, embellished and even reversed
Example of repeated motive
Example of thematic transformation
String-based algorithms for discovering musical patterns Most previous approaches assume music represented as strings
•each string represents a voice or part
•each character represents a note or an interval between two consecutive notes in a voice
Similarity between two patterns measured in terms of edit distance calculated using dynamic programming
•see, e.g., Lemstrom (2000), Hsu et al. (1998), Rolland (1999)
Problems with the string-based approach - Edit distance
B is an embellished version of A
If both patterns represented as strings each symbol represents pitch of note
then edit distance between A and B is 9
If allow pattern with 9 differences to count as a match, then get many spurious hits
Problems with string-based approach - Polyphony
If searching polyphonic music and• do not know voice to which each note belongs (e.g., MIDI format 0 file); or
• interested in patterns containing notes from 2 or more voices
then• combinatorial explosion in number of possible string representations
• if don’t use all possible representations then may not find all interesting patterns
Using multidimensional point sets to represent music (1)
Using multidimensional point sets to represent music (2)
SIA - Discovering all maximal translatable patterns (MTPs)
Pattern is translatable by vector v in dataset if it can be translated by v to give another pattern in the dataset
MTP for a vector v contains all points mapped by v onto other points in the dataset
O(kn2 log n) time, O(kn2) space O(kn2) average time with hashing (Lemstrom)
SIATEC - Discovering all occurrences of all MTPs
Absolute running times of SIA and SIATEC
SIA and SIATEC implemented in C run on a 500MHz Sparc on 52 datasets (6≤n≤3456, 2≤k≤5)
< 2 mins for SIA to process piece with 3500 notes
13 mins for SIATEC to process piece with 2000 notes
Need for heuristics to isolate interesting MTPs 2n patterns in a dataset of size n SIA generates < n2/2 patterns
• => SIA generates small fraction of all patterns in a dataset
Many interesting patterns derivable from patterns found by SIA
BUT many of the patterns found by SIA are NOT interesting
• 70,000 patterns found by SIA in Rachmaninoff’s Prelude in C# minor
• probably about 100 are interesting
=> Need heuristics for isolating interesting patterns in output of SIA and SIATEC
Heuristics for isolating musical themes and motives
Cov=6CR=6/5
Cov=9CR=9/5
Comp = 1/3 Comp = 2/5 Comp = 2/3
€
Coverage = Number of points covered by occurrences of the pattern
Compactness = Number of points in pattern
Number of points in region spanned by pattern
Compression ratio = Coverage
Number of points in pattern + Freq. of occurrence of pattern -1
COSIATEC - Data compression using SIATEC
Start
Dataset
SIATEC
List of <Pattern, Translator_set> pairs
Print out best pattern, P, and its translators
Remove occurrences of P from dataset
Is dataset empty?
End
No
Yes
Using COSIATEC for finding themes and motives in music
First iteration Second iteration
SIAM - Pattern matching using SIA
O(knm log(nm)) time
O(knm) space O(knm) average time with hashing
Query pattern
Dataset
Improving SIAM - Ukkonen, Lemström & Mäkinen (2003) Use sweepline-like scanning of the dataset (Bentley and Ottmann, 1979)
Generalized to approximate matching of sets of horizontal line-segments
Improved running time to O(mn log m) (without hashing) and working space to O(m)
Implemented as algorithm P2 on C-BRAHMS demo web site
• <http://www.cs.helsinki.fi/group/cbrahms/demoengine/>
Improving SIAM - Clifford (In preparation) Finds best match in O(n log n) time Reduce problem to one dimension by randomised projection
Reduce length of problem by uniform hashing
Perform pattern matching using FFTs Find best match and check in O(m) time exactly how many points match at the location that can be inferred from this match
Pitch spelling algorithms (1)
Pitch spelling algorithms (2)
Pitch spelling in tonal music
Pitch name depends on harmonic structure and voice-leading structure
Pitch name chosen so that score correctly represents the way the music is intended to be perceived and interpreted
(Piston, 1978, p.8)
Comparative analysis of pitch spelling algorithms Algorithms analysed, evaluated and (in some cases) improved
•Longuet-Higgins (1976, 1987, 1993)•Cambouropoulos (1996,1998, 2001, 2003)
•Temperley (2001)•Chew and Chen (2003, 2005)•Meredith (2003, 2005, 2006)
Test corpus•195972 notes, 216 movements, 8 baroque and classical composers
•almost exactly equal number of notes (24500) for each composer
Evaluation criteria and performance metrics Evaluation criteria
• Spelling accuracy - how well an algorithm predicts the pitch names
• Style dependence - how much spelling accuracy depends on style
Performance metrics• Note accuracy - proportion of notes in corpus spelt correctly
• Style dependence - standard deviation of note accuracies over 8 composers
Robustness to temporal deviations• Best versions of algorithms also run on version of test corpus in which onsets and durations were randomly adjusted
Longuet-Higgins’s algorithm (1976,1987,1993)
Uses 6 rules to predict pitch names• Rule 1: pitch names as close to tonic on line of fifths
• Rules 2-6: deal with chromatic intervals and key changes
• Rule 2 incorrectly implemented in music.p
6 versions of algorithm tested• Original and two versions with Rule 2 “corrected”
• Same three algorithms with pitch names not restricted to being between G double sharp and A double flat
Two versions of test corpus• Voices arranged “end-to-end” (should be better)
• Voices “interleaved” with notes sorted by onset and pitch
Longuet-Higgins’s algorithm - Results Correcting Rule 2 implementation lowered note accuracy
Made half as many errors when voices end-to-end
Allowing pitch names to be anywhere on the line of fifths doubled number of errors
Original version performed best (NA = 98.21%; SD = 1.79)
Cambouropoulos’s algorithm (1996,1998,2001,2003)
Three published versions of algorithm
Input changed to sequence of MIDI note numbers
Shifting overlapping window• improves running time and avoids boundary errors
Computes all spellings for each window
• 128 spellings for each 9-note window
Spelling penalised if• contains intervals that are rare in tonal scales
• contains double sharps or double flats
Cambouropoulos’s algorithm - Evaluation
18 ways in which two versions of the algorithm could differ
• e.g., variable or fixed length window
26 versions implemented and tested• goal to estimate optimal combination of variable features
Window: Variable-length better than fixed-length
• Best variable-length window version: NA = 99.07%; SD = 0.46
Increasing window size • increases accuracy but exponentially increases running time
• 12 note window is practical maximum
Algorithm with ‘optimal’ combination of features: NA = 99.15%; SD = 0.47
Temperley and Sleator’s pitch spelling algorithm (2001)
Temperley and Sleator’s algorithm - Evaluation
Output of meter program depends on tempo System tested on 6 versions of corpus, each with different tempo
Best on natural tempo or half-speed corpora NA = 99.30%; SD = 1.13 (without enh. change)
NA = 97.79%; SD = 4.57 (with enh. change) Highly sensitive to tempo
at 4 times natural tempo, NA = 74.58% • worse than just spelling all black notes randomly as either sharp or flat!
Simple implementation of TPR 1 alone achieved NA = 99.04%; SD = 0.65
Chew and Chen’s algorithm (2003,2005)
Based on “spiral array” = line of fifths coiled up
Tonic represented by center of effect = Centroid of positions in spiral array of pitch names in preceding window
First spelt so close to global CE, then re-spelt so close to weighted average of local and cumulative CEs
Chew and Chen’s algorithm - Evaluation New implementation allows user to
• use line of fifths instead of spiral array• consider notes starting in each window instead of notes sounding in each window when computing CEs
• change aspect ratio of spiral array
Run 1296 times on test corpus, each time with different parameter value combination
Best 12 versions scored NA=99.15%, SD=0.4
• worked best when all three CEs used, local and cumulative CEs weighted equally and chunks small
Line of fifths worked just as well as the spiral array
PS13s1 (Meredith, 2003,2005,2006)
Pitch name implied by a tonic is one that is closest to the tonic on the line of fifths
Strength with which tonic implied proportional tofrequency of occurrence
Strength with which pitch name implied proportional to sum of frequencies of occurrence of tonics implying pitch name
PS13s1 - Results Takes two parameters:
• Precontext (Kpre): number of notes preceding note to be spelt included in context
• Postcontext (Kpost): number of notes following note to be spelt included in context
PS13 run with all values of Kpre and Kpost between 1 and 50
• PS13s1 run with 17 best values obtained with PS13
• Made 15-19% fewer errors than PS13 for these parameter values
Some results: • NA = 99.44%, SD = 0.49 (Kpre=10, Kpost=42)• NA = 99.44%, SD = 0.45 (Kpre=33, Kpost=25)• NA = 99.19%, SD = 0.51 (Kpre=40, Kpost=1)
Summary of pitch spelling resultsAlgorithm Clean
corpusNoisy corpus
NA% SD NA% SD
PS13s1x 99.44 0.45 99.41 0.50
Temperley* 99.27 1.30 97.43 1.69
Chew and Chen+ 99.15 0.42 99.12 0.47
Cambouropoulos+
99.15 0.47 99.07 0.53
Longuet-Higgins§
98.21 1.79 98.25 1.71
xKpre= 33, Kpost= 25*Two-pass, natural tempo corpus, without enh. change+New optimized versions§Only when music processed a voice at a time.
Future work Pattern discovery and pattern matching
• Compare SIA algorithms with methods developed in other more mature fields (e.g., computer vision, graph matching)
• Improve time complexity of SIA algorithms with advanced algorithmic techniques (e.g., randomized projection, hashing)
• Adapt algorithms for approximate matching and scaling (matching at different tempi)
• Adapt SIA and SIATEC for early pruning of uninteresting patterns
Pitch spelling• Incorporate PS13s1 into complete MIDI-to-notation transcription system
• Use PS13s1 for key-tracking and harmonic analysis
• Use PS13s1 for feature extraction on audio data
Acknowledgements and further details Thanks to
•Chris Bishop and Stephen Robertson for inviting me to give a talk
•Geraint Wiggins for suggesting SIAM•Kjell Lemstrom for developing SIAM further
•Raphael Clifford for developing SIAM further still
•EPSRC for funding • GR/S17253/02, GR/N08049/01
Further details: http://www.titanmusic.com