Algorithm Apply MESSAGE - Stanford University

EMAlgorate cont'dRECAPGeneric Em Algorithm

Apply to Gmms MESSAGE Gmm algo is Ema START FACTORAnalysis

WE EXAMINED GMMS

Given X X ERd AND K 20De fnd Plz j for i i n j i K

A SOFTAssignment

WE called Z A LATENT VARIABLE NOT OBSERVED DIRECTLY

TODAY Generalize MODEL RELATE IT to MLE EM Algorithm

f PARAMETERSLfo II dog Pix o

EI log PCx Z o

PKtonralfeneecealgor.tkt xwLie

g Lela Ello Howerow

z Oct LCDGetOut Aegnfax

10 HIE Lt f easy Tooptimize

RoughAlgoritumCESTEP 1 Find 103 given 0

C 7 l d a g 0M STEP 2

tArgnfax Lf

WE go term logzPCx 2 jo term fix a particular c

dog Pex z o dog Ez pcxinz.es

Pick Q z s 1 Q Z L Q723 o V z.ua

log PCxji.IEJREcalelogCECx3 7tIEloslxD Great

Es this holds 3 log P

forAny Cyoice Qthat satisfies c Q log P x z

Q ZPropertgt

Pick Q Z satisfy Property 2

LIQ't OH tight CFS

want dog Pex zjo Q t log Pcx oQ Ct

TALE Cz Plz x Q Too

q pCx z P x jo doesnot depend

See piazza post 822 for •more discussion.

I s dpplz Ix D

z onz

dog Pix D Qi

Note Z VARIES for Evey point

we call ELBOCK Q D Qe by P

WE'VE Stown hot Z 1 ELBO x Q o

OH IE ELBO xli Q je'twith our SELECTIONofQ

WARID Mixture of Gaussians EM RECOVERS AD40CAlgonitta

PIX 2 7 pfx.us ziDjPCzlilIndustries Z n Multinomial OI Oi QQ _7

clustermeans X 2 j Nfu r32Ci LATENT VARIABLE

WHATS EMHERE Gmm IntenseCj Plz j x jo PCx z g o

µ by

µVIA BAYESRok

MSTE.se fiMax EIEa Q t t lo pcxlis.fiME Q.cz'D

Wag Of jMEALconman

fto GE w log iz xuEjcx uBdw

0µg fifa YEO w logexpf klxcis.ee Cx aDle

deanna E Eg we Ej'Cx noAssume exists Cfull RANK

Setting to 0 Mj w jfcfGMMUPDATEJE.wsMEALconman

fto GE w log z iz x uEjx Bdw

qq.fiQ Mos w logdj log isconstrained

j 1 constraintLANGNEAN

Dogfiles 0djfweiIlogOjtA mI.dj D

Detour WANT To find critical Points off R s t X Xz glx D

u

as

no IE x.txz tPOINT

If 2 is A critical POINT Tew Off't is Painted2g PgCz Dfct X Oge

exists

Ux fix t X guy 1

Q L Ofc x 40gex o Parallel Condition

gcx I 0 Constraint is satisfied

og.li wg'logolit7CII m i

EI w A o d EEEwiIj

since s i e g tx a

dj Wiiis

MF eE Cm Recovers Gmm Algorithm

NB Z is discrete Here Can Replace Sums of Intgads

TaetoRAnalysisMANY FEWER points than dimensions need

nod Guns

t.fowdoesthishapr.esPdAeESENSORS All over campus RECORD temperature

RS n campus co eboos of locations do 10005

But Only Record for 30 days CN C dWANT TO FIT A Density but seems hopeless

KegIDEI Assumethere is some latent rv thatIS not Complex AND Explains behavior

lets SEE the problems w filling A single Gaussiansmaller there

Given x x'm Rdµ To X SEEMS OK

E In EI Cx a exec afRANK E E n C d NOT Fold RANK

181 0 7NOTdefined

PCxjµ E z lk exp.se xustE'cx u3zero

WE will fix issues by Examining theesimplermodelsSpoiler Combine All thee our finalmodel

BuldwgBlocSuppose INDEPENDENT Indentical v.v

E of near E c row

ma II Waste coin Eoo

Assumption o E C n T n t dog owE o2I

2 02 E x ex t d togZ

Oz z C tu.dz o e d 22

02 End

141 IC I od