Discrete MRF Inference of Marginal Densities for Non-uniformly Discretized Variable Space

Masaki Saito, Takayuki Okatani, Koichiro Deguchi Tohoku University

Outline

1. Introduction 2. Algorithms for a non-uniformly discretized variable space 3. Dynamic discretization method 4. Experimental results

Outline

1. Introduction 2. Algorithms for a non-uniformly discretized variable space 3. Dynamic discretization method 4. Experimental results

Markov Random Fields

MAP inference • Directly computes the

maximum of • Equivalent to

minimizing

MPM inference • Estimates the marginal density

(then computes its maximum ) • Necessary in some problems

– e.g. Learning CRF models

Continuous and Discrete MRF

• Mean Field(MF) and Belief Propagation(BP) are only choices for the estimation of marginal densities

• Two formulations: Discrete and Continuous – The variable x at each site takes a continuous or a discrete

value

Applicability Computational cost (per iter.)

Discrete Wide O((# of labels)^2) Continuous Limited Efficient

Continuous and Discrete MRF

Discretization

• A common practice is to convert a continuous problem to a discrete one by discretizing the variable space uniformly – A continuous density is approximated by a discrete one

Continuous and Discrete MRF

• What if the variable space can be non-uniformly discretized so that the accuracy is maximized with a minimum number of discretization bins? – Will achieve a better trade-off between accuracy vs.

computational cost – Will be useful for non-Euclidean variable spaces

uniform non-uniform Spherical surface

Outline

1. Introduction 2. Algorithms for a non-uniformly discretized variable space 3. Dynamic discretization method 4. Experimental results

• ‘s may have arbitrary positions and sizes • ‘s may have different number of mixtures

Remark:

Our approach

1. Represent the marginal density as a mixture of rectangular densities

2. Follow the variational method to derive MF/BP algorithms

weight rectangular density

Our approach

1. Represent the marginal density as a mixture of rectangular densities

2. Follow the variational method to derive MF/BP algorithms

Introduce P such that marginal densities are

easily computed

Find P minimizing the KL distance

between P and Q

Compute the marginal densities

MF

[Textbook of Graphical Models by Koller-Friedman]

Conventional continuous-discrete conversion

1. Discretize the variable space into discrete labels (uniformly or non-uniformly)

2. Define an energy E(x) in the discrete domain Original continuous energy (if any) is discarded Step 2 does not care about how the space was discretized

conventional define a new discrete energy function

Approximate joint density

Models for the marginal densities Constraints

MF

BP

Derivation of new MF & BP algorithms

New MF and BP updating rules

• A few terms are newly added; they compensate for the non-uniformity of descretization – They vanish when the discretization is uniform

New updating rules Conventional updating rules

MF

BP

&

: newly added terms

Outline

1. Introduction 2. Algorithms for a non-uniformly discretized variable space 3. Dynamic discretization method 4. Experimental results

Dynamic discretization of the variable space

• If we have a prior knowledge about where is important in the variable space, the application of our method is straightforward

• Our “dynamic discretization method” is targeted at the case where no such knowledge is available

1D

2D

Coarse-to-fine block subdivision

1. Discretize the variable space coarsely (maybe uniformly)

2. Perform MF or BP iterations to obtain the estimates of the marginal densities

3. Identify the block whose mixture weight is the largest, and divide the block into subblocks – Weights and messages are adjusted

4. Go to 2 until a desired result is obtained

Outline

1. Introduction 2. Algorithms for a non-uniformly discretized variable space 3. Dynamic discretization method 4. Experimental results

Effect of non-uniform discretization

• A simple Gaussian grid MRF

• Non-uniform discretization; asymmetric with respect to the origin

5

5

-1 -2 2 1 0

64 16

64+16 = 80

Effect of non-uniform discretization

Mean Field

Belief Propagation

Stereo matching

• The proposed method is used to adaptively and recursively discretize the variable space

• Conventional: divide the variable space into 16 fixed blocks • Proposed: initially divide into 8 blocks, and finally divide into

16 blocks with 8 subdivision procedures

Conventional Proposed Input images

Ground truth x8

Stereo matching(BP)

…

Conventional Blocks=16

Proposed Blocks=8 Blocks=10 Blocks=16

Input images Ground truth

Blocks=12

Estimated marginal densities

Our method gives a better result than conventional one with only 10 blocks

Disparity maps (maximum of marginal)

Conclusions

• We have derived new MF and BP algorithms for estimating marginal densities for pairwise MRFs that can properly deal with non-uniform discretization

• We have also presented a method that can dynamically discretize the variable space

• These tools are expected to make it possible to achieve a better trade-off between accuracy and computational cost

• The idea is to represent the marginal densities with a mixture of rectangular densities and then apply the variational method

• Our approach also gives a rigorous method for converting continuous MRFs to discrete ones