Mixture model

Mixture Model

Hong Chulju

Index•Gaussian Distribution

• Multivariate Gaussian Distribution

• Latent Variable Model

• Mixture Model

• EM Algorithm

• Gaussian Mixture Model Example

• Gaussian Mixture Model vs k-Means Clustering

• Mel Frequency Cepstrum Coefficient

• Speaker identification

• Speaker diarization

Review : Gaussian Distributiona.k.a. Normal Distribution

Gaussian Distribution Likelihood

N(165, 5)

N(165, 10)

N(165, 15)

Which one is the fittest?

Gaussian Distribution Likelihood

Multivariate Gaussian Distribution

Latent Variable Model a.k.a. Hidden Variable Model

관찰되지 않고 숨겨져 있는 변수

관찰된 변수로부터 추정

Latent Variable

Latent Variable Model

숨겨진 변수로부터 관찰된 변수가 도출되었다고 가정

관찰된 변수 + 숨겨진 변수를 이용하여 모델링

Latent Variable Model a.k.a. Hidden Variable Model

Observable: Height

Hidden: Gender

Mixture Model전체 모집단에 있는 여러 개의 부분 모집단을 표현하기 위한 확률 모델

Mixture Model Example

Gaussian Mixture Model

Guassian distribution pdf

Mixture Model Revisited

Gaussian Mixture Model

How to solve it?

Expectation-Maximization Algorithm

EM Algorithm : GMM example

GMM vs k-Means Clustering

k-Means Clustering

Latent Variable based clustering

Observable Variable based clustering

Generalization of Mixture Model

(http://en.wikipedia.org/wiki/Hidden_Markov_model)

Hidden Markov Model

Algorithm for decoding HMM

(http://en.wikipedia.org/wiki/Viterbi_algorithm)

Viterbi Algorithm

MFCCsMel Frequency Cepstrum Coefficients

1. Mel Frequency?

2. Cepstrum?

CepstrumSpectrum > cepStrum

http://www.speech.cs.cmu.edu/15-492/slides/03_mfcc.pdf

1. Spectrum?

2. Why Cepstral analysis?

Mel FrequencyMel-Frequency analysis of speech is based on human perception experiments

MFCCs revisitedMel Frequency Cepstrum Coefficients

1. 주어진 신호(time-amplitude)를 푸리에 변환한다.

2. 푸리에 변환된 신호(frequency-amplitude)를 Mel-Scale로 변환한다.

3. Mel-Scale로 변환된 신호에 log를 취한다.

4. 여기에 이산 코사인 변환을 취한다.

구하는 방법

Speaker Diarization

Features (Observable): MFCCs 12+a elements, etc

Latent Variable : Speaker

#Components : # Speakers

http://cslu.ohsu.edu/~bedricks/courses/cs655/hw/hw4/hw4.html

Practice 1: GMM

Practice 2: Speaker Diarization

Questions?

Mixture model

Technology

Very Fast EM-Based Mixture Model Clustering Using ...papers.nips.cc/paper/...based-mixture-model-clustering-using-multiresolution-kd-trees.pdfVery Fast EM-Based Mixture Model Clustering

The subspace Gaussian mixture model – a structured model ... · The subspace Gaussian mixture model – a structured model for speech recognition Daniel Poveya, Luk´aˇs Burgetb,

depmixS4: An R Package for Hidden Markov Models · Keywords: hidden Markov model, dependent mixture model, mixture model, constraints. 1. Introduction Markov and latent Markov models

BAYESIAN CLASSIFICATION USING GAUSSIAN MIXTURE … · BAYESIAN CLASSIFICATION USING GAUSSIAN MIXTURE MODEL AND ... Bayesian Classiﬁcation Using Gaussian Mixture Model and EM Estimation:

On the mixture model for multiphase flow

A Gaussian Mixture Model Spectral Representation for ...mi.eng.cam.ac.uk/~mjfg/thesis_mns25.pdf4.1 Gaussian mixture model representations of the speech spectrum 45 4.1.1 Mixture models

MIXTURE-MODEL INTENSITY WINDOWING (MMIW)

Choosing a ‘correct’ mixture model

Gaussian Mixture Models (GMM) and ML Estimation Examples Materials/gmm... · 2020-02-28 · Gaussian Mixture Model • GMM Gaussian Mixture Model • Probabilistic story: Each cluster

Gaussian Mixture Model and Hidden Markov Model

A Mixture Model for Expert Finding

A Gaussian Mixture Model Spectral Representation for ...mi.eng.cam.ac.uk/~mjfg/thesis_mns25.pdf · A Gaussian Mixture Model Spectral Representation for Speech Recognition ... 2.1.2

Clustering：k-means, expect-maximization and gaussian mixture model

Probabilistic analog of clustering: mixture models We model uncertainty in the identification of objects to mixture components Each mixture component can

LING 696B: Mixture model and linear dimension reduction

Part 3 Vector Quantization and Mixture Density Model

MIXTURE MODEL CLUSTERING FOR VERY LARGE …studentsrepo.um.edu.my/6108/1/Mixture_Model_Clustering_for_Very... · MIXTURE MODEL CLUSTERING FOR VERY LARGE DATA ... 2010 . MIXTURE MODEL

Penalized multinomial mixture logit model - Springer · Penalized multinomial mixture logit model 123 the multinomial logit model is a logistic regression of the categorical response

GAUSSIAN MIXTURE MODEL CLASSIFIERS FOR DETECTION AND

Mixture Markov regression model with application to ...huaiping/paper/2017_Gao_et_al-2017-Biometrical... · Mixture Markov regression model with application to mosquito surveillance