Music Analyser

Music AnalyserMusic Analyser

MSc in Information Engineering

Jean-Marie Froux

MSc Information Engineering 2003

Music Analyser - JM Froux City University - 2003

2

Abstract

Transcription of music : from the sound to the paper through computer.

The technologies used to develop :

C#, C++, DirectX


3

• Introduction

• 1: Theory

• 2: Wave file analyser (in C#)

• 3: Music analyser (in C++)

• Conclusion

• Questions

Contents


4

Introduction

• What is music transcription?

• Music and computers

• Why is it interesting ?

• The programs developed


5

1/ Musical Theory

• 1.1 The physic of sound

• 1.2 The Fourier Analysis


6

1.1 The physic of sound

• There are 4 parameters to describe a sound: – Duration, (s).– Intensity, (dB).– Height, (Hz).


7

Frequency and notes (1)

Ear sensibility : 20 Hz to 20 kHz

represents 10 octaves

A flute can play between 130.81 Hz and 1046.5 Hz

Change octave by doubling frequency



8


• 12 intervals in an octave • Tone is an interval of T = 62 • Semitone is an interval of T = 122

(=1.0594)

• We obtain thus the scale for Major C:

Note C D E F G A B C

Frequency f1 21/6f1 21/3f1 25/12f1 27/12f1 23/4f1 211/12f1 2f1



9


All change possible with sharps and flats.

The basic (reference) note is “A3” 440Hz.

Other are deduced by multiplication with

T= 122



10

The physic of sound

• There are 4 parameters to describe a sound: – Duration, (s).– Intensity, (dB).– Height, (Hz).– Tone


11

The tone (1)

• A sound is made of many frequencies :

–The fundamental

–Harmonics (multiples of the fundamental)



12

The tone (2)

Each instrument produces a specific spectrum :

2 4 6 8 Frequency

Magnitude

3 5 7 Frequency

Magnitude

A sound played by a clarinet

The same sound played by a oboe



13

1.2 Fourier transform

• Any pseudoperiodic signal can be broken up into an infinite sum of sinusoidal signals of frequency multiple of a basic frequency.

1

0 00)( 2sin2cosk

tfkbtfkaa kktx


14

Interest

• The FFT allows us to see the frequencies and magnitudes of sinusoidal signals.

1.2 Fourier transform

frequency

Magnitude

time

Frequency


15

2/ Wave file Analyser

• 2.1 Design

• 2.2 Implementation

• 2.3 Testing


16

2.1 Design

Record data

2.1 Wave Analyser Design

Analyse data

Write score


17

Record data2.1 Wave Analyser Design

Select capture device

Create capture buffer

Wave format tag (compression)

Average bytes per second

Samples per second

Bits per sample

Block Align

Channel(s)Start


18

Analysis2.1 Wave Analyser Design

Open the file

Get the file size

Create an array of the same size

Copy the data in the array

Decode the header chunk

Analyse the data chunk


19

Analysis (2)

• The program the should analyse the sample with an FFT and find the notes.

• We will see the algorithm with the second program.

• Once the notes found, they are stored in a file.



20

Write notes


Open file

Read file

title

signature

notesDraw the score


21

The .notes fileRules :

– First line: Title: Title– Second line : Signature1: Int– Third line : Signature: Int– Then the notes described by letters A to G.

• 3 octaves lower (add: l), medium (m=default), and higher (h)

• Sharp (s) and flat (f)• Timing: 16th (add x), 8th (add e), quarter is

default, half (a), three-quarter (t), whole (w).



22

Testing • The software:

2.2 Wave Analyser Testing


23

Testing • Capture devices and record:



24

Testing • Analysis



25

Testing • The score:



26

Conclusion

• C# is a very good programming language, but it needs the latest version of DirectX9.

• As it is very new, the help is poor.

• That’s why I have decided to change my way, and I passed to C++.

2.Wave Analyser


27

3/ The sound analyser in real time

• 3.1 Sampling.

• 3.2 FFT.

• 3.3 Search for the note.

• 3.4 Display.

• 3.5 Save.

• 3.6 The recognition of rhythm.


28

3.1 Sampling

• tell the soundcard to start sampling the input from the microphone

• store the samples in a client buffer • The client creates a separate thread

which is suspended waiting on an event• Once the buffer is full, notifies the client • The client thread does the necessary

work and waits for the next event.



29

3.2 FFT

• The FFT buffer is filled with samples in bit-reversed order

• The bit reversal is done, converts real samples into complex numbers (with the imaginary part set to zero)

• Then the FFT calculation is organized into bunches.



30

3.3 Find the note

• For each FFT

Recover the frequency and magnitude of the highest peak

if magnitude is above the noise, seek if there is a peak at frequency divided by 2.



31

3.3 Find the note (2)

• If yes:The peak of higher magnitude isn’t the

fundamental (but the first harmonic)


Example with a guitar: spectrum of an A2:

In this case, the highest peak is the first harmonic, we obtain the fundamental by dividing by 2 this frequency


32

3.3 Find the note (3)

• If not:

seek a peak at frequency multiplied by 2 (the first harmonic).

If there is one, the fundamental is the peak of higher magnitude.

Then find the note with the table of frequencies



33

3.3 Display the note (1)

• Problem: not display several times the same note played once.

• If for 2 consecutive FFT– The note found is the same– The magnitude of the second is not higher

It is the same note.• If the magnitude changes, it means that the

note is played again.



34

3.3 Display the note (2)• The notes are represented by an “o” and a

# if needed. To help the position of the note, I have adapted it to the octave:


Frequency octave

f<131 Hz 1

131f<261 2

261f<523 3

523f<1046

4

f1046 5

•The notes go from the "G" of octave 1 until "A" of octave 3

•For the too high or too low notes the octave is modified in order to display the note on the scale.


35

3.3 Save the notes (1)

• Important to allow musician to recover their work in another software.

• First step to MIDI file, write the data in a .txt file



36

3.3 Save the notes (2)• Example:

MThd | Format=0 | # of Tracks=1 | Division=96Track #0

**************************************** Time Event1: 1: 0 |On Note | chan= 1 | pitch=D 1 | vol=127 2: 0 |Off Note | chan= 1 | pitch=d 1 | vol=100 | On Note | chan= 1 | pitch=A 2 | vol=127 3: 0 |Off Note | chan= 1 | pitch=a 2 | vol=100 |End of track|



37

3.4 Rhythm recognition• Metronome

– Help the musician when recording– Can give the tempo of the piece

• Trace.txt: Save the tempo, the note, the value of the timer and the amplitude.

• E.g.:


60.C. 0. 608.C#. 47. 1407.D. 94. 1212.D. 141. 1675.D. 188. 1880.

Next step is the analysis of this file.


38

Testing• The software:



39

Testing• The software:



40

Conclusion

• With C++ I found complete help on the Internet

• It gives a program which works well.

• Some improvements can be done to continue this work.



41

Further Work

• Rhythm recognition

• Detection of tonality

• Polyphonic recognition?


42

QUESTIONS??? ???


43

Thank you for your attention

Music Analyser

Jean-Marie FROUX

MSc Project

City University

Supervisor: D J STYLES

Documents

Music Analyser