Mixture Model

Hong Chulju

Index•Gaussian Distribution

• Multivariate Gaussian Distribution

• Latent Variable Model

• Mixture Model

• EM Algorithm

• Gaussian Mixture Model Example

• Gaussian Mixture Model vs k-Means Clustering

• Mel Frequency Cepstrum Coefficient

• Speaker identification

• Speaker diarization

Review : Gaussian Distributiona.k.a. Normal Distribution

Gaussian Distribution Likelihood

N(165, 5)

N(165, 10)

N(165, 15)

Which one is the fittest?

Gaussian Distribution Likelihood

Multivariate Gaussian Distribution

Latent Variable Model a.k.a. Hidden Variable Model

관찰되지 않고 숨겨져 있는 변수

관찰된 변수로부터 추정

Latent Variable

Latent Variable Model

숨겨진 변수로부터 관찰된 변수가 도출되었다고 가정

관찰된 변수 + 숨겨진 변수를 이용하여 모델링

Latent Variable Model a.k.a. Hidden Variable Model

Observable: Height

Hidden: Gender

Mixture Model전체 모집단에 있는 여러 개의 부분 모집단을 표현하기 위한 확률 모델

Mixture Model Example

Gaussian Mixture Model

Guassian distribution pdf

Mixture Model Revisited

Gaussian Mixture Model

How to solve it?

Expectation-Maximization Algorithm

EM Algorithm : GMM example

EM Algorithm : GMM example

EM Algorithm : GMM example

GMM vs k-Means Clustering

GMM

k-Means Clustering

Latent Variable based clustering

Observable Variable based clustering

More

Generalization of Mixture Model

(http://en.wikipedia.org/wiki/Hidden_Markov_model)

Hidden Markov Model

Algorithm for decoding HMM

(http://en.wikipedia.org/wiki/Viterbi_algorithm)

Viterbi Algorithm

MFCCsMel Frequency Cepstrum Coefficients

1. Mel Frequency?

2. Cepstrum?

CepstrumSpectrum > cepStrum

http://www.speech.cs.cmu.edu/15-492/slides/03_mfcc.pdf

1. Spectrum?

2. Why Cepstral analysis?

Mel FrequencyMel-Frequency analysis of speech is based on human perception experiments

MFCCs revisitedMel Frequency Cepstrum Coefficients

1. 주어진 신호(time-amplitude)를 푸리에 변환한다.

2. 푸리에 변환된 신호(frequency-amplitude)를 Mel-Scale로 변환한다.

3. Mel-Scale로 변환된 신호에 log를 취한다.

4. 여기에 이산 코사인 변환을 취한다.

구하는 방법

Speaker Diarization

Features (Observable): MFCCs 12+a elements, etc

Latent Variable : Speaker

#Components : # Speakers

http://cslu.ohsu.edu/~bedricks/courses/cs655/hw/hw4/hw4.html

Practice 1: GMM

Practice 2: Speaker Diarization

Hmm

Questions?