PRML 13.2.2: The Forward-Backward Algorithm

PRML 13.2.2

The forward-backward algorithm

August 4, 2014 by Shinichi TAMURA


Today's topics

The forward-backward algorithm 1.  E-step of EM algorithm for HMM 2.  Multiple shorter sequences 3.  Predictive distribution

Aug 4, 2014 PRML 13.2.2 Shinichi TAMURA


Today's topics




Today's topics




E-step of EM algorithm for HMM

Quick review HMM is trained by EM algorithm E-step: Evaluate γ(zn) and ξ(zn-1, zn) M-step: Update parameters


!k ="(z1k)!Kj=1 "(z1 j)

,

Ajk =

!Nn=2 #(zn!1, j, znk)!K

l=1!Nn=2 #(zn!1, j, znl)

.



Quick review HMM is trained by EM algorithm E-step: Evaluate γ(zn) and ξ(zn-1, zn) M-step: Update parameters


!k ="(z1k)!Kj=1 "(z1 j)

,

Ajk =

!Nn=2 #(zn!1, j, znk)!K

l=1!Nn=2 #(zn!1, j, znl)

.




ξ(zn-1, zn) γ(zn)




ξ(zn-1, zn) γ(zn) ? ?



How to calculate γ(zn), ξ(zn-1, zn) ? Basic idea is message passing on tree (It will be studied later, and we shall do conventional style now)




Evaluation of γ(zn)


!(zn) = p(zn|x1:N)

=p(x1:N |zn)p(zn)

p(x1:N)

=p(x1:n|zn)p(zn)p(xn+1:N |zn)

p(x1:N)





!(zn) = p(zn|x1:N)

=p(x1:N |zn)p(zn)

p(x1:N)


p(x1:N)





!(zn) = p(zn|x1:N)

=p(x1:N |zn)p(zn)

p(x1:N)


p(x1:N)




zn+1 zNzn-1 znz1

xn+1 xNxn-1 xnx1

Head TailTail





!(zn) = p(zn|x1:N)

=p(x1:N |zn)p(zn)

p(x1:N)


p(x1:N)





!(zn) = p(zn|x1:N)

=p(x1:N |zn)p(zn)

p(x1:N)


p(x1:N)





!(zn) = p(zn|x1:N)

=p(x1:N |zn)p(zn)

p(x1:N)


p(x1:N)

β(zn) α(zn)




ξ(zn-1, zn) γ(zn) ? ?




ξ(zn-1, zn) γ(zn)

β(zn) α(zn) ?

?

?

p(x1:N) ?



Now problem is changed into

How to calculate α(zn), β(zn) ?




Evaluation of α(zn)


!(zn) = p(x1:n|zn)p(zn)= p(xn|zn)p(x1:n!1|zn)p(zn)= p(xn|zn)p(x1:n!1, zn)= p(xn|zn)

!

zn!1

p(x1:n!1, zn|zn!1)p(zn!1)

= p(xn|zn)!

zn!1

p(x1:n!1|zn!1)p(zn|zn!1)p(zn!1)

= p(xn|zn)!

zn!1

!(zn!1)A.






!

zn!1


= p(xn|zn)!

zn!1


= p(xn|zn)!

zn!1

!(zn!1)A.




zn-1 znz1

xn-1 xnx1

HeadTail






!

zn!1


= p(xn|zn)!

zn!1


= p(xn|zn)!

zn!1

!(zn!1)A.






!

zn!1


= p(xn|zn)!

zn!1


= p(xn|zn)!

zn!1

!(zn!1)A.






!

zn!1


= p(xn|zn)!

zn!1


= p(xn|zn)!

zn!1

!(zn!1)A.






!

zn!1


= p(xn|zn)!

zn!1


= p(xn|zn)!

zn!1

!(zn!1)A.






!

zn!1


= p(xn|zn)!

zn!1


= p(xn|zn)!

zn!1

!(zn!1)A.




zn-1 znz1

xn-1 xnx1

Head

TailTail






!

zn!1


= p(xn|zn)!

zn!1


= p(xn|zn)!

zn!1

!(zn!1)A.






!

zn!1


= p(xn|zn)!

zn!1


= p(xn|zn)!

zn!1

!(zn!1)A.






!

zn!1


= p(xn|zn)!

zn!1


= p(xn|zn)!

zn!1

!(zn!1)A.






!

zn!1


= p(xn|zn)!

zn!1


= p(xn|zn)!

zn!1

!(zn!1)A.





!(zn) = p(xn|zn)!

zn!1

!(zn!1)A.

k = 1

k = 2

k = K

...

n = 1 n = 2 n = N...

O(K)





!(zn) = p(xn|zn)!

zn!1

!(zn!1)A.

k = 1

k = 2

k = K

...

n = 1 n = 2 n = N...

O(K2)





!(zn) = p(xn|zn)!

zn!1

!(zn!1)A.

k = 1

k = 2

k = K

...

n = 1 n = 2 n = N...

O(K2N)



Evaluation of α(zn) Start of recursion?


!(z1) = p(x1|z1)p(z1) =

!""""""""""#

"1p(x1|!1)...

"N p(x1|!N)

$%%%%%%%%%%&.



Evaluation of α(zn) Start of recursion?


!(z1) = p(x1|z1)p(z1) =

!""""""""""#

"1p(x1|!1)...

"N p(x1|!N)

$%%%%%%%%%%&.




ξ(zn-1, zn) γ(zn)

β(zn) α(zn) ?

?

?

p(x1:N) ?




ξ(zn-1, zn) γ(zn)

β(zn) α(zn) ?

?

p(x1:N) ?



Evaluation of β(zn)


!(zn) = p(xn+1:N |zn)=!

zn+1

p(xn+1:N |zn, zn+1)p(zn+1|zn)

=!

zn+1

p(xn+1:N |zn+1)p(zn+1|zn)

=!

zn+1

p(xn+2:N |zn+1)p(xn+1|zn+1)p(zn+1|zn)

=!

zn+1

!(zn+1)p(xn+1|zn+1)AT.





!(zn) = p(xn+1:N |zn)=!

zn+1


=!

zn+1


=!

zn+1


=!

zn+1






!(zn) = p(xn+1:N |zn)=!

zn+1


=!

zn+1


=!

zn+1


=!

zn+1





zn+2 zNzn zn+1

xn+2 xNxn xn+1

HeadTail





!(zn) = p(xn+1:N |zn)=!

zn+1


=!

zn+1


=!

zn+1


=!

zn+1






!(zn) = p(xn+1:N |zn)=!

zn+1


=!

zn+1


=!

zn+1


=!

zn+1






!(zn) = p(xn+1:N |zn)=!

zn+1


=!

zn+1


=!

zn+1


=!

zn+1





zn+2 zNzn zn+1

xn+2 xNxn xn+1

TailTail





!(zn) = p(xn+1:N |zn)=!

zn+1


=!

zn+1


=!

zn+1


=!

zn+1






!(zn) = p(xn+1:N |zn)=!

zn+1


=!

zn+1


=!

zn+1


=!

zn+1






!(zn) = p(xn+1:N |zn)=!

zn+1


=!

zn+1


=!

zn+1


=!

zn+1






!(zn) = p(xn+1:N |zn)=!

zn+1


=!

zn+1


=!

zn+1


=!

zn+1






!(zn) =!

zn+1


k = 1

k = 2

k = K

...

n = 1 n = 2 n = N...

O(K2N)



Evaluation of β(zn) Start of recursion?


p(zN |x1:N) =p(x1:N |zN)p(zN)!(zN)

p(x1:N)

! !(zN) =!1 · · · 1

"T.



Evaluation of β(zn) Start of recursion?


p(zN |x1:N) =p(x1:N |zN)p(zN)!(zN)

p(x1:N)

! !(zN) =!1 · · · 1

"T.




ξ(zn-1, zn) γ(zn)

β(zn) α(zn) ?

?

p(x1:N) ?




ξ(zn-1, zn) γ(zn)

β(zn) α(zn)

?

p(x1:N) ?



Evaluation of p(x1:N) It appears on the denominator of γ(zn).


!(zn) ="(zn)#(zn)p(x1:N)



Evaluation of p(x1:N) It appears on the denominator of γ(zn). In fact, we don't need it for update because it cancel out.


!k ="(z1k)!Kj=1 "(z1 j)

,

Ajk =

!Nn=2 #(zn!1, j, znk)!K

l=1!Nn=2 #(zn!1, j, znl)

.



Evaluation of p(x1:N) It appears on the denominator of γ(zn). In fact, we don't need it for update because it cancel out. However, we need to evaluate it, because it's LIKELIHOOD, which is monitored.




Evaluation of p(x1:N)


!(zn) ="(zn)#(zn)p(X) .

!!

zn

!(zn) ="zn "(zn)#(zn)p(X) .

! p(X) =!

zn

"(zn)#(zn) (!!! = 1).

" p(X) =!

zN

"(zN) =!

k"k(zN).





!(zn) ="(zn)#(zn)p(X) .

!!

zn

!(zn) ="zn "(zn)#(zn)p(X) .

! p(X) =!

zn

"(zn)#(zn) (!!! = 1).

" p(X) =!

zN

"(zN) =!

k"k(zN).





!(zn) ="(zn)#(zn)p(X) .

!!

zn

!(zn) ="zn "(zn)#(zn)p(X) .

! p(X) =!

zn

"(zn)#(zn) (!!! = 1).

" p(X) =!

zN

"(zN) =!

k"k(zN).Any n will do





!(zn) ="(zn)#(zn)p(X) .

!!

zn

!(zn) ="zn "(zn)#(zn)p(X) .

! p(X) =!

zn

"(zn)#(zn) (!!! = 1).

" p(X) =!

zN

"(zN) =!

k"k(zN).




ξ(zn-1, zn) γ(zn)

β(zn) α(zn)

?

p(x1:N) ?




ξ(zn-1, zn) γ(zn)

β(zn) α(zn)

?

p(x1:N)

Nth



Evaluation of ξ(zn-1, zn)


!(zn!1, zn) = p(zn!1, zn|x1:N)

=p(x1:N |zn!1, zn)p(zn!1, zn)

p(x1:N)

=p(x1:n!1|zn!1, zn)p(xn|zn!1, zn)p(xn+1:N |zn!1, zn)p(zn!1, zn)

p(x1:N)

=p(x1:n!1|zn!1)p(xn|zn!1, zn)p(xn+1:N |zn!1, zn)p(zn!1, zn)

p(x1:N)

=p(x1:n!1|zn!1)p(xn|zn)p(xn+1:N |zn!1, zn)p(zn!1, zn)

p(x1:N)...





!(zn!1, zn) = p(zn!1, zn|x1:N)


p(x1:N)


p(x1:N)


p(x1:N)


p(x1:N)...





!(zn!1, zn) = p(zn!1, zn|x1:N)


p(x1:N)


p(x1:N)


p(x1:N)


p(x1:N)...




zn+1 zNzn-1 znz1

xn+1 xNxn-1 xnx1

Head TailTailTail





!(zn!1, zn) = p(zn!1, zn|x1:N)


p(x1:N)


p(x1:N)


p(x1:N)


p(x1:N)...





!(zn!1, zn) = p(zn!1, zn|x1:N)


p(x1:N)


p(x1:N)


p(x1:N)


p(x1:N)...





!(zn!1, zn) = p(zn!1, zn|x1:N)


p(x1:N)


p(x1:N)


p(x1:N)


p(x1:N)...




zn+1 zNzn-1 znz1

xn+1 xNxn-1 xnx1

Head

TailTail





!(zn!1, zn) = p(zn!1, zn|x1:N)


p(x1:N)


p(x1:N)


p(x1:N)


p(x1:N)...





!(zn!1, zn) = p(zn!1, zn|x1:N)


p(x1:N)


p(x1:N)


p(x1:N)


p(x1:N)...





!(zn!1, zn) = p(zn!1, zn|x1:N)


p(x1:N)


p(x1:N)


p(x1:N)


p(x1:N)...




zn+1 zNzn-1 znz1

xn+1 xNxn-1 xnx1

HeadTail





!(zn!1, zn) = p(zn!1, zn|x1:N)


p(x1:N)


p(x1:N)


p(x1:N)


p(x1:N)...





!(zn!1, zn) = p(zn!1, zn|x1:N)


p(x1:N)


p(x1:N)


p(x1:N)


p(x1:N)...





!(zn!1, zn) = p(zn!1, zn|x1:N)


p(x1:N)


p(x1:N)


p(x1:N)


p(x1:N)...





...


p(x1:N)

=p(x1:n!1|zn!1)p(xn|zn)p(xn+1:N |zn)p(zn!1, zn)

p(x1:N)

=p(x1:n!1|zn!1)p(xn|zn)p(xn+1:N |zn)p(zn|zn!1)p(zn!1)

p(x1:N)

=!(zn!1)p(xn|zn)p(zn|zn!1)"(zn)

p(x1:N).





...


p(x1:N)


p(x1:N)


p(x1:N)


p(x1:N).




zn+1 zNzn-1 znz1

xn+1 xNxn-1 xnx1

HeadTail





...


p(x1:N)


p(x1:N)


p(x1:N)


p(x1:N).





...


p(x1:N)


p(x1:N)


p(x1:N)


p(x1:N).





...


p(x1:N)


p(x1:N)


p(x1:N)


p(x1:N).

β(zn) α(zn)





...


p(x1:N)


p(x1:N)


p(x1:N)


p(x1:N).




ξ(zn-1, zn) γ(zn)

β(zn) α(zn)

?

p(x1:N)

Nth




ξ(zn-1, zn) γ(zn)

β(zn) α(zn)

p(x1:N)

Nth



Summary HMM is trained by EM algorithm E-step: Evaluate α(zn) and β(zn)

M-step: Update parameters


!(zn) = p(xn|zn)!

zn!1

!(zn!1)p(zn|zn!1),

"(zn) =!

zn+1

"(zn+1)p(xn+1|zn+1)p(zn+1|zn).


Today's topics




Today's topics




Multiple shorter sequences

In some practical situation, we can't get a long sequence.




In some practical situation, we can't get a long sequence. Instead, we get multiple shorter sequences.




In some practical situation, we can't get a long sequence. Instead, we get multiple shorter sequences. In this situation, we are still able to use forward-backward algorithm (w/ bit modification).




E-step Just evaluate independently


!(z(r)n ) = p(z(r)n |X(r), !old),"(z(r)n!1, z

(r)n ) = p(z

(r)n!1, z

(r)n |X(r), !old).



E-step Just evaluate independently


!(z(r)n ) = p(z(r)n |X(r), !old),"(z(r)n!1, z

(r)n ) = p(z

(r)n!1, z

(r)n |X(r), !old).



M-step Just add them all

etc.


!k =

!Rr=1 "(z

(r)1k )!R

r=1!K

j=1 "(z(r)i j ),

Ajk =

!Rr=1!Nrn=2 #(z

(r)n!1, j, z

(r)nk )

!Rr=1!Kl=1!Nrn=2 #(z

(r)n!1, j, z

(r)nl )




etc.


!k =

!Rr=1 "(z

(r)1k )!R

r=1!K

j=1 "(z(r)i j ),

Ajk =

!Rr=1!Nrn=2 #(z

(r)n!1, j, z

(r)nk )


(r)n!1, j, z

(r)nl )




etc.


!k =

!Rr=1 "(z

(r)1k )!R

r=1!K

j=1 "(z(r)i j ),

Ajk =

!Rr=1!Nrn=2 #(z

(r)n!1, j, z

(r)nk )


(r)n!1, j, z

(r)nl )

Note: p(X) no longer cancel out


Today's topics




Today's topics




Predictive distribution

Once you trained the HMM, next thing you want to do may be PREDICTION. I.e., evaluation of p(xN+1|X)





p(xN+1|X) =!

zN+1

p(xN+1, zN+1|X)

=!

zN+1

p(xN+1|zN+1,X)p(zN+1|X)

=!

zN+1

p(xN+1|zN+1)p(zN+1|X)

=!

zN+1

p(xN+1|zN+1)!

zN

p(zN , zN+1 |X)

=!

zN+1

p(xN+1|zN+1)!

zN

p(zN+1|zN ,X)p(zN |X)

...




p(xN+1|X) =!

zN+1

p(xN+1, zN+1|X)

=!

zN+1


=!

zN+1


=!

zN+1

p(xN+1|zN+1)!

zN

p(zN , zN+1 |X)

=!

zN+1

p(xN+1|zN+1)!

zN


...




p(xN+1|X) =!

zN+1

p(xN+1, zN+1|X)

=!

zN+1


=!

zN+1


=!

zN+1

p(xN+1|zN+1)!

zN

p(zN , zN+1 |X)

=!

zN+1

p(xN+1|zN+1)!

zN


...




zN zN+1z1

xN xN+1x1

HeadTail




p(xN+1|X) =!

zN+1

p(xN+1, zN+1|X)

=!

zN+1


=!

zN+1


=!

zN+1

p(xN+1|zN+1)!

zN

p(zN , zN+1 |X)

=!

zN+1

p(xN+1|zN+1)!

zN


...




p(xN+1|X) =!

zN+1

p(xN+1, zN+1|X)

=!

zN+1


=!

zN+1


=!

zN+1

p(xN+1|zN+1)!

zN

p(zN , zN+1 |X)

=!

zN+1

p(xN+1|zN+1)!

zN


...




p(xN+1|X) =!

zN+1

p(xN+1, zN+1|X)

=!

zN+1


=!

zN+1


=!

zN+1

p(xN+1|zN+1)!

zN

p(zN , zN+1 |X)

=!

zN+1

p(xN+1|zN+1)!

zN


...




p(xN+1|X) =!

zN+1

p(xN+1, zN+1|X)

=!

zN+1


=!

zN+1


=!

zN+1

p(xN+1|zN+1)!

zN

p(zN , zN+1 |X)

=!

zN+1

p(xN+1|zN+1)!

zN


...




...

=!

zN+1

p(xN+1|zN+1)!

zN


=!

zN+1

p(xN+1|zN+1)!

zN

p(zN+1|zN)p(zN |X)

=!

zN+1

p(xN+1|zN+1)!

zN

p(zN+1|zN)p(zN ,X)p(X)

=1

p(X)!

zN+1

p(xN+1|zN+1)!

zN

p(zN+1|zN)!(zN).




...

=!

zN+1

p(xN+1|zN+1)!

zN


=!

zN+1

p(xN+1|zN+1)!

zN

p(zN+1|zN)p(zN |X)

=!

zN+1

p(xN+1|zN+1)!

zN


=1

p(X)!

zN+1

p(xN+1|zN+1)!

zN

p(zN+1|zN)!(zN).




zN zN+1z1

xN xN+1x1

Head

TailTail




...

=!

zN+1

p(xN+1|zN+1)!

zN


=!

zN+1

p(xN+1|zN+1)!

zN

p(zN+1|zN)p(zN |X)

=!

zN+1

p(xN+1|zN+1)!

zN


=1

p(X)!

zN+1

p(xN+1|zN+1)!

zN

p(zN+1|zN)!(zN).




...

=!

zN+1

p(xN+1|zN+1)!

zN


=!

zN+1

p(xN+1|zN+1)!

zN

p(zN+1|zN)p(zN |X)

=!

zN+1

p(xN+1|zN+1)!

zN


=1

p(X)!

zN+1

p(xN+1|zN+1)!

zN

p(zN+1|zN)!(zN).




...

=!

zN+1

p(xN+1|zN+1)!

zN


=!

zN+1

p(xN+1|zN+1)!

zN

p(zN+1|zN)p(zN |X)

=!

zN+1

p(xN+1|zN+1)!

zN


=1

p(X)!

zN+1

p(xN+1|zN+1)!

zN

p(zN+1|zN)!(zN).




...

=!

zN+1

p(xN+1|zN+1)!

zN


=!

zN+1

p(xN+1|zN+1)!

zN

p(zN+1|zN)p(zN |X)

=!

zN+1

p(xN+1|zN+1)!

zN


=1

p(X)!

zN+1

p(xN+1|zN+1)!

zN

p(zN+1|zN)!(zN).




...

=!

zN+1

p(xN+1|zN+1)!

zN


=!

zN+1

p(xN+1|zN+1)!

zN

p(zN+1|zN)p(zN |X)

=!

zN+1

p(xN+1|zN+1)!

zN


=1

p(X)!

zN+1

p(xN+1|zN+1)!

zN

p(zN+1|zN)!(zN).

Only depend on α(zN) i.e., whole X don't need




ξ(zn-1, zn) γ(zn)

β(zn) α(zn)

p(x1:N)

Nth




ξ(zn-1, zn) γ(zn)

p(x1:N)

β(zn) α(zn) p(xN+1|X) Nth

Nth


Today's topics




Today's topics




Today's topics



Technology

PRML 13.2.2: The Forward-Backward Algorithm