Efficient Belief Propagation for Image Restoration

Qi ZhaoMar.22,2006

References

• Pedro F. Felzenszwalb and Daniel P. Huttenlocher. Efficient Belief Propagation for Early Vision. To appear in the International Journal of Computer Vision.

• Y.Weiss and W.T. Freeman. On the optimality of solutions of themax-product belief propagation algorithm in arbitrary graphs. IEEE Transactions on Information Theory, 47(2):723–735, 2001.

• L.I. Rudin, S. Osher, and E. Fatemi, "NONLINEAR TOTAL VARIATION BASED NOISE REMOVAL ALGORITHMS", PHYSICA D 60 (1-4): 259-268 Nov. 1, 1992.

Outline

• MRF model for Image Restoration• Image Restoration using Efficient Belief

Propagation• Experimental Results(Demo)

– Additive noise removal– Image Inpaiting

Markov Model

• Motivation– Markov random field models provide a robust and

unified framework for early vision problems.

MRF Models for Image Restoration

• : set of pixels in an image • : a finite set of labels, which correspond to the underlying

intensities of the pixels.• E.g., , where• Objective: Finding a labeling that minimizes the energy

corresponds to the MAP estimation problem for the defined MRF.– Neighborhood system: 4-neighborhood system– Prior: pair-wise potential cliques– Likelihood energy:

e.g. ,where

– Posterior energy:

gf

i i ig f n 2(0, )in N

2( , ) min{( ) , }, ( , )i i i i i iV f f f f f f

1 2

1( | ) ( | ) exp( )( 2 )

Ni ii N

p g f p g f V

2 2( ) ( ) / 2i i ii i

V V f f g ( ) ( | ) ( ) ( , )i i ii iE f E f g V f V f f

Loopy Belief Propagation: Max-Product

• Let be the message that node sends to a neighboring node at iteration , we have

• Finally, the label that maximizes is individually selected for each node.

ti im

i i

t0 0;i im

1min ( ( ) ( , ) ), ( )i

t ti i f i i i i im V f V f f m i i i

( )( ) ( ) ti i i i ii N ib f V f m

*if ( )i ib f

2( )O nk T

Speed Up Techniques (1)

• Computing a Message Update in Linear Time– Computing , where

• Potts Model:

( ) min ( ( , ) ( ))i

ti i i f i i im f V f f h f

1( ) ( ) ( )t

i i i i ih f V f m f 2( )O k ( )O k

2( ) min ( ( ) ( ))i

ti i i f i i im f c f f h f

( ) min( ( ),min ( ) )i

ti i i i f im f h f h f d

0, 0( )

, 0x

V xd x

• Firstly, compute the lower envelope of the parabolas;

• Secondly, fill in the value of by checking the height of the lower envelope at each grid location .

( )im f

Speed Up Techniques (2)

• BP on the Grid Graph

– The grid graph is bipartite– Two groups of nodes: A & B– Time t:

• Msg from Nodes A -> Nodes B– Time t+1:

• Msg from Nodes B -> Nodes A

Speed Up Techniques (3)

• Multi-Grid BP– Problem in BP: it takes many iterations for information to

flow over large distances in the grid graph. – Basic Idea: to perform BP in a coarse-to-fine manner, so that

long range interactions between pixels can be captured by short paths in coarse graphs.

Experiments (1)

• Noise Removal2( ) min(( ) , )i i i i iV f f f f d 2( ) (( ) )i i i iV f g f

0.05,20,1

TL

20 Original Image

BP Restored Image

• Parameters

Experiments (2)

20

0.05 1 0.2 0.01

20,1

TL

Original Image

Experiments (3)

• Comparisons

0.05,20,1

TL

20

0.2,100T

TV Restored Image

BP Restored Image

Experiments (4)

• Image Impainting– For pixel in the masked region,( ) 0iV f i

(a) Noised, masked image (b) L=1,T=25 (c) L=1,T=14 (d) L=5,T=5

Efficiency Improved by the Coarse-to-Fine Technique!

Thank You!