HHT and Applications in Music Signal Processing

1

HHT and Applications in Music Signal Processing

電信一 R01942128 陳昱安

2

Research area: MER

Not quite good at difficult math

About the Presenter

3

HHT : abbreviation ofHilbert-Huang Transform

Decided after the talk given byDr. Norden E. Huang

About the Topic

4

Fourier is nice, but not good enough Clarity Non-linear and non-stationary signals

Why HHT?

5

Hilbert-Huang Transform

Hilbert Transform Empirical Mode Decomposition

6

Hilbert Transform

dttutuHt

)(1)}({)(

Not integrable at τ=t Defined using Cauchy principle value

7

Dealing with 1/(τ-t)

-∞ ∞τ=t

=0

8

Input u(t) Output H{u}sin(t) -cos(t)cos(t) sin(t)exp(jt) -jexp(jt)exp(-jt) jexp(-jt)

Quick Table

9

I know how tocompute

Hilbert Transform

10

That’s cool…SO WHAT?

11

exp(jz) =cos(z) + jsin(z)

exp(jωt) =cos(ωt) + jsin(ωt)

θ(t) = arctan(sin(ωt)/cos(ωt))

Freq.=dθ/dt

12

S(t) = u(t) + jH{u(t)}θ(t) = arctan(Im/Re)Freq.=dθ/dt

What happen if u(t) = cos(ωt) ?Hint:

H{cos(t)} = sin(t)

13

Input : u(t) Calculate v(t) = H{u(t)} Set s(t) = u(t) + jv(t) θ(t) = arctan(v(t)/u(t)) fu(t)= d θ(t) /dt

Frequency Analysis with HT

14

Congrats!!!Forgot something?

15

Hilbert-Huang Transform

Hilbert Transform Empirical Mode Decomposition

16

0 8

F = 1Hz

17

0 8

F = 1Hz

18

0 8

F = 1Hz

19

1Hz

0 8

F = 1/8Hz

20

=+

21

To makeinstantaneous frequency

MEANINGFUL

22

Need to decomposesignals

into “BASIC” components

23

Decompose the input signal Goal: find “basic” components Also know as IMFIntrinsic Mode Functions BASIC means what?

Empirical Mode Decomposition

24

1) num of extrema - num of zero-crossings≤ 1

2) At any point, the mean value of the envelope defined by the local maxima and the envelope defined by the local minima is zero.

Criteria of IMF

25

TO BESHORT

26

IMFsare signals

27

Oscillate

28

Around 00

29

Empirical Mode Decomposition Used to generate IMFs

Review: EMD

EMD

30

Empirical Mode Decomposition Used to generate IMFs

Review: EMD

EMD

Hint:Empirical means

NO PRIOR KNOWLEDGES

NEEDED

31

Application I:Source Separation

32

Problem

33

Problem

Source Separatio

n

34

What if…We apply STFT, then

extract different componentsfrom different freq. bands?

35

Problem Solved?No!

36

Gabor Transform of piano

37

Gabor Transform of organ

38

Gabor Transform of piano + organ

39

I see…So how to make sure we do it right?

40

41

How to win Doraemonin paper-scissor-stone?Easy. Paper always win.

42

The tip is to know the answer first!

43

Single-MixtureAudio Source Separation

by Subspace Decompositionof Hilbert Spectrum

Khademul Islam Molla, and Keikichi Hirose

44

Approximation of sources

Desired result

45

PHASE I:Construction of

possible source model

46

HilbertSpectra

IMFsEMDHilbert

TransformOriginal Signal

IMF 1IMF 2IMF 3

∶

Spectrum of

47

X1

X2

X3

X4

X5

X6

Spectrum of original signal

X1

X2

X3

X4

X5

X6

Spectrum of IMF1

X1

X2

X3

X4

X5

X6

Spectrum of IMF2 ..

.frequ

ency

48

Original Signal

IMF1

IMF2Projection 1

Projection 2

49

AFTERSOME

PROCESSING

50

RESULTSIN

51

INDEPENDENTBASIS

52

Frequency Band I

Freq

uenc

y Ba

nd II

53

Frequency Band I

Freq

uenc

y Ba

nd IIHint:

Data points are different

observations

54

Frequency Band I

Freq

uenc

y Ba

nd IISo…

What does this basis

mean?

55

Frequency Band I

Freq

uenc

y Ba

nd II

7F1 +2F2

3F1 +4F2

56

Gabor Transform of piano

F(piano) = 10F1 + 9F2 + F3

3F1 + 4F2

7F1 + 2F2

3F2 + F3

57

FindSourceModels

58

ClusterBasis

Vectors

59

Cluster

60

Raw data

61

Clustered

62

Approximated SourcesIn hand!

63

The “figure” of sources obtained We have been through

1) EMD : Obtain IMFs2) Hilbert Transform : Construct spectra3) Projection : Decompose signal in frequency space4) PCA and ICA : Independent vector basis5) Clustering : Combine correlated vectors together6) Voila!

Finally

64

PHASE II:Reconstruction of

separated source signals

65

Spectrum of each source is a linear combination of the vector basis generated

Reconstruction

]... [];... [ , 2121

1

aaaAyyyYYAH

ayH

T

i

Tii

SignalSpectrum

Combinationof sources’

spectra

66

Let the clustered vector basis to be Yj

Then the weighting of this subspace is

Reconstruction

1Tj j jA Y H

67

Tjjj AYH

68

Why HHT?◦EMD needs NO PRIOR KNOWLEDGE◦Hilbert transform suits for non-linear and non-stationary condition

However, clustering…

Conclusions

69

Application II:Fundamental

Frequency Analysis

70

ProblemSTFT of C4(262Hz)

Music Instrument Samples of U. Iowa

71

FUNDAMENTAL FREQUENCY

ESTIMATION FOR MUSIC SIGNALS WITH

MODIFIED HILBERT-HUANG TRANSFORMEnShuo Tsau, Namgook Cho and C.-C. Jay Kuo

72

Basic Idea

EMD

73

Mode mixingExtrema finding

◦Boundary effect◦Signal perturbation

Problems of EMD

74

1. Kizhner, S.; Flatley, T.P.; Huang, N.E.; Blank, K.; Conwell, E.; , "On the Hilbert-Huang transform data processing system development," Aerospace Conference, 2004. Proceedings. 2004 IEEE , vol.3, no., pp. 6 vol. (xvi+4192), 6-13 March 2004

2. Md. Khademul Islam Molla; Keikichi Hirose; , "Single-Mixture Audio Source Separation by Subspace Decomposition of Hilbert Spectrum," Audio, Speech, and Language Processing, IEEE Transactions on , vol.15, no.3, pp.893-900, March 2007

3. EnShuo Tsau; Namgook Cho; Kuo, C.-C.J.; , "Fundamental frequency estimation for music signals with modified Hilbert-Huang transform (HHT)," Multimedia and Expo, 2009. ICME 2009. IEEE International Conference on , vol., no., pp.338-341, June 28 2009-July 3 2009

4. Te-Won Lee; Lewicki, M.S.; Girolami, M.; Sejnowski, T.J.; , "Blind source separation of more sources than mixtures using overcomplete representations," Signal Processing Letters, IEEE , vol.6, no.4, pp.87-90, April 1999

References

75

Q&A請把握加分的良機

76

Thank you for your attention!

THE END

77

Recycle bin

78

Input u(t) Output H{u}sin(t) -cos(t)cos(t) sin(t)exp(jt) -jexp(jt)exp(-jt) jexp(-jt)

Quick Table

Insight:Hilbert transform

rotate input by π/2on complex plane

79

EMD

80

～！＠＃＄％︿＆＊

Spectrum of original signal

＊＆︿％＄＃＠！～

Spectrum of IMF1

～＠！＃＄︿％＆＊

Spectrum of IMF2

…

81

Original Signal

IMF1

IMF2Projection 1

Projection 2

82

Data Spread on Vector Space

83

PCA ICA

84

PCAICA

85

PCAICA

Fact:PCA & ICA are

linear transforms

86

FAQ

87

Q: Why Hilbert Transform?