Outline

• Introduction• Music Information Retrieval• Classification Process Steps • Pitch Histograms• Multiple Pitch Detection Algorithm• Musical Genre Classification• Implementation• Future Work

Why do we classify?

• Increasing importance of digital music distribution• Effectively navigating through large web-based music

collections• Structuring on-line music stores & radio stations• Creating intelligent Internet music search engines and

Peer-to-Peer systems• Can be used in other type of analysis like similarity

retrieval or summarization

Audio Classification

Jazz

Rock

Classical

Country

Electronica

Reggae

WorldFolk New Age

?

?

? ? ?

??

?

?

?

?

?

?

Audio Classification (cont.)

Audio Classification (cont.)

Music Information Retrieval (MIR)

The process of indexing and searching music collections.

• Symbolic MIR – Structured signals such as MIDI files are used.

– Melodic information is typically utilized.• Two different approaches: Query-by-melody (manual) and Query-by-humming

• Audio MIR – Arbitrary unstructured audio signals are used.

– Timbral and rhythmic (beat) information is utilized.

What is MIDI?

• Musical Instrument Digital Interface• A music definition language • Communication protocol• supports 128 different voices• includes 16 channels

Classification Process Steps

MIDI file Audio-from-MIDI file Arbitrary Audio file

Pitch Histogram

4D Feature Vector(Pitch Content Feature Set)

Multiple Pitch Detection Algorithm

Labeled Feature Vectorsused by Statistical Classifiers

Histogram Construction Algorithm

Timbral & Rhythmic Features

Genre Classification Result by comparing the feature vectors

Pitch Histograms

• Unfolded Histogram– an array of 128 integer values (bins) indexed by MIDI note numbers

– showing the frequency of occurrence of each note in a musical piece

– contains information regarding the pitch range of the music

• Folded Histogram– All notes are transposed into a single octave and mapped to a circle of

fifths

– an array of 12 integer values

– contains information regarding the pitch content of the music

Folded Pitch Histogram – Index Numbers

Index Numbers

0 1 2 3 4 5 6 7 8 9 10 11

12 13 14 15 16 17 18 19 20 21 22 23

24 25 26 27 28 29 30 31 32 33 34 35

36 37 38 39 40 41 42 43 44 45 46 47

48 49 50 51 52 53 54 55 56 57 58 59

60 61 62 63 64 65 66 67 68 69 70 71

72 73 74 75 76 77 78 79 80 81 82 83

84 85 86 87 88 89 90 91 92 93 94 95

96 97 98 99 100 101 102 103 104 105 106 107

108 109 110 111 112 113 114 115 116 117 118 119

120 121 122 123 124 125 126 127      

Unfolded Pitch Histograms

Fig.1 - Unfolded Pitch Histograms of 2 Jazz pieces (left) and 2 Irish songs (right).

Pitch Histogram features

• Four dimensional feature vector– PITCH-Fold– AMPL-Fold– PITCH-Unfold– DIST-Fold

Pitch Histogram Calculation

• For MIDI files:– The algorithm increments the corresponding note’s frequency

counter while using linear traversal over all MIDI events in the file.

– Normalization

• For arbitrary audio files:– Multiple Pitch Detection Algorithm

Multiple Pitch Detection Algorithm

Fig.2 – Multiple Pitch Detection Flow Chart

Experiment Details

• Types of music contents:– symbolic (refers to MIDI)– audio-from-MIDI (generated using a synthesizer playing a MIDI file)– audio (digital audio files like mp3’s found on the web)

• Five musical genres are used:– Electronica, Classical, Jazz, Irish Folk and Rock

• Experiment Set:– A set of 100 musical pieces in MIDI format for each genre– A set of 100 audio-from-MIDI pieces for each genre– A set of 100 general audio files

• KNN(3) Classifier

Classification Results in MIDI

Fig.3 – Classification accuracy comparison of random and MIDI

Classification Results in MIDI

Classification Results in MIDI

Fig.4 – Pair-wise evaluation in MIDI

Classification Results in MIDI

Fig.5 – Average classification accuracy as a function of the length of input MIDI data

Classification Results in Audio-from-MIDI

Fig.6 - Classification accuracy comparison of random and Audio-from-MIDI

Classification Results in Audio-from-MIDI

Comparison of Classification Results

Fig.7 – Classification accuracy comparison

Implementation

MARSYAS – MusicAl Research SYstem for Analysis and Synthesis– the software used for audio Pitch Histogram calculation and

musical genre classification.– Three distinct modes of visualization:

• Standard Pitch Histogram plots

• 3D pitch-time surfaces

• Projection of the pitch-time surfaces onto a 2D bitmap

MARSYAS Visualization

Fig.8 – Examples of grayscale pitch-time surfaces. Jazz (top) and Irish Folk music (bottom)

Summary

• Symbolic representation is more preferable in the sense of computing Pitch Information.

• This work can be viewed as an attempt to bridge the two distinct MIR approaches by using Pitch Histograms.

• Pitch Histograms do carry a certain amount of genre-identifying information.

• Multiple Pitch Detection Algorithm is not perfect, but it works by a certain degree.

Future Work

• Real-time running version of Pitch Histogram.– for better classification performance.– to conduct more detailed harmonic analysis such as figured bass

extraction, tonality recognition, and chord detection.

• The features derived from Pitch Histograms might be applicable to the problem of content-based audio identification or audio fingerprinting.

• Alternative feature sets are needed.

• Query-based retrieval mechanism for audio music signals.

Thanks

• Cosku Turhan for the art work on my slides…• 4 Non Blondes for their song, “What's Up?” :)