Upload
wayne-paul
View
18
Download
1
Embed Size (px)
DESCRIPTION
Embedding Gestalt Laws in Markov Random Fields by Song-Chun Zhu. Purpose of the Paper. Proposes functions to measure Gestalt features of shapes Adapts [Zhu, Wu Mumford] FRAME method to shapes Exhibits effect of MRF model obtained by putting these together. Recall Gestalt Features. - PowerPoint PPT Presentation
Citation preview
Embedding Gestalt LawsEmbedding Gestalt Laws
in Markov Random Fieldsin Markov Random Fields
by Song-Chun Zhuby Song-Chun Zhu
Purpose of the PaperPurpose of the Paper
Proposes functions to measure Gestalt features of shapes
Adapts [Zhu, Wu Mumford] FRAME method to shapes
Exhibits effect of MRF model obtained by putting these together.
Recall Recall GestaltGestalt Features Features(à la [Lowe], and others)
Colinearity
Cocircularity
Proximity
Parallelism
Symmetry
Continuity
Closure
Familiarity
FRAMEFRAME[Zhu, Wu, Mumford]
F ilters
R andom fields
A nd
M aximum
E ntropy
A general procedure for constructing MRF models
Three Main PartsThree Main Parts
Data
Learn MRF models from data
Test generative power of learned model
Elements of DataElements of Data
A set of images representative of the chosen application domain
An adequate collection of feature measures or filters
The (marginal) statistics of applying the feature measures or filters to the set of images
Data: ImagesData: Images
Zhu considers 22 animal shapes and their horizontal flips
The resulting histograms are symmetric
More data can be obtainedBut are there other effects?
Sample Animate ImagesSample Animate Images
Contour-based Feature MeasuresContour-based Feature MeasuresGoal is to be generic
But generic shape features are hard to find
φ1 = κ(s), the curvature
κ(s) = 0 implies the linelets on either side of Γ(s) are colinear
φ2 = κ'(s), its derivative
κ'(s) = 0 implies three sequential linelets are cocircular
“Other contour-based shape filters can be defined in the same way”
Zhu's Symmetry FunctionZhu's Symmetry Function
Ψ(s) pairs linelets across medial axesDefined and computed by minimizing an energy functional constructed so that
Paired linelets are as close, parallel and symmetric as possible, and
There are as few discontinuities as possible
Region-based Feature MeasuresRegion-based Feature Measures
φ3(s) = dist(s, ψ(s))
Measures proximity of paired linelets across a region
φ4(s) = φ3'(s), the derivative
φ4(s) = 0 implies paired linelets are parallel
φ5(s) = φ'4(s) = φ3''(s)
φ5(s) = 0 implies paired linelets are symmetric
Another Possible Shape FeatureAnother Possible Shape Feature
φ6(s) = 1 where ψ(s) is discontinuous
0 otherwise
Counts the number of “parts” a shape has
Can Gestalt “familiarity” be (statistically?) measured?
The StatisticThe Statistic
The histogram of feature φ over curve Γ is
H(z; φk, Γ) = ∫δ(z-φk(s)) ds
δ is the Dirac function: mass 1 at 0, and 0 otherwise
μ(z; φk) denotes the average over all images
Zhu claims μ is a close estimation of the marginal distribution of the “true distribution” over shape space, assuming the total number of linelets is small.
Statistical ObservationsStatistical Observations
φ1 at scales 0, 1, 2
φ3 φ4 φ5
On 22 images and their flips
Construct a ModelConstruct a Model
Ω is the space of shapes
Φ is a finite subset of feature filters
We seek a probability distribution p on Ω
∫Ω p(Γ) dΓ = 1 (1)
That reproduces the statistics for all φ in Ω
∫Ω p(Γ) δ(z-φ(s)) dΓ = μ(z; φ) (2)
Construct a Model, 2Construct a Model, 2Idea: Choose the p with maximal entropy
Seems reasonable and fair, but is it really the best target/energy function?
Lagrange multipliers and calculus of variations lead to
p(Γ; Φ, Λ) = exp(–∑φЄΦ ∫ λφ(z) H(φ, Γ, z) dz) / Zwhere Z is the usual normalizing factor
Λ = { λφ | φЄΦ }
It's a Gibbs DistributionIt's a Gibbs Distribution
In other words, it has the form of a Gibbs distribution, and therefore determines a Markov Random Field (MRF) model.
Markov Chain Monte CarloMarkov Chain Monte Carlo
Too hard to compute λ's and p analytically
Idea: Sample Ω according to the distribution p, stochastically update Λ to update p, and repeat until p reproduces all μ(z; φ) for φ Є Φ
Monte Carlo because of random walk
Markov Chain in the nature of the loop
Markov Chain Monte Carlo, 2Markov Chain Monte Carlo, 2
From the sampling produce μ'(z; φ)Same as μ(z; φ) except based on a random sample of shape space
For the purposes of today's discussion, the details are not important
For φ Є Φ
μ'(z; φ) = μ(z; φ)
Zhu et al. assume there exists a “true underlying distribution”
The Nonaccidental StatisticThe Nonaccidental Statistic
For φ' not in the set Φ we expect
μ'(z; φ') ≠ μ(z; φ')
μ'(z; φ') is the accidental statistic for φ'It is a measure of correlation between φ' and Φ
The “distance” (L1, L2, or other) between μ'(z; φ') and μ(z; φ') is the nonaccidental statistic for φ'
It is a measure of how much “additional information” φ' carries above what is already in Φ
The Algorithm (simplified)The Algorithm (simplified)
Enter your set Γ = { γ } of shapes
Enter a (large) set { φ } of candidate feature measures
Compute μ(φ, Γ) for all φ in Φ
Compute μ'(φ) relative to a uniform distribution on Ω
Until the nonaccidental statistic of all unused features is small enough, repeat:
Algorithm, 2Algorithm, 2
Of the remaining φ , add to Φ one with maximal nonaccidental statistic
Update:Set of Lagrange multipliers Λ = { λ }
Probability distribution model p(Φ, Λ)
The μ'(φ) for remaining candidate features φ
Experiments and DiscussionExperiments and Discussion
Let my description of these experiments stimulate your thoughts on such issues as
Are there better Gestalt feature measures?
What is the best possible outcome of a generative model of shape?
What feature measures should be added to the Gestalt ones?
How useful were these experiments and what other might be worth doing?
Experiment 1Experiment 1
When the only feature used is the curvature the model generated
Experiment 1, continuedExperiment 1, continued
A Gaussian model (with the same κ-variance) produced
Experiment 2Experiment 2
Experiment 2 uses both κ and κ'
The nonaccidental statistic of κ' with respect to the model based on κ can be seen here
Experiment 2, continuedExperiment 2, continued
This time the model generated these shapes, purported to be smoother and more scale invariant
Experiment 3Experiment 3
The nonaccidental statistics of the three region-based shape features relative to the model produced in Experiment 2
Experiment 3, continuedExperiment 3, continued
So r'' was omitted, this model has
Φ = { κ, κ', r, r' }
Experiment 3, continuedExperiment 3, continued
This model produced such shapes as
Concluding DiscussionConcluding Discussion
Zhu acknowledges that the selection of training shapes might introduce a bias; but
Discussion, continuedDiscussion, continued
Zhu acknowledges that the paucity of Gestalt features limits the possible neighborhood structures used to define a MRF.
Zhu acknowledges that these models do not account for high-level shape properties, and suggests that a composition system might address this problem.
Questions and CommentsQuestions and Comments
Although it is in the nature of an MRF-model to propagate local properties, I think there needs to be a higher-level basis (than linelets) for measuring the Gestalt features of a shape!
Are there better Gestalt feature measures?
What feature measures should be added to the Gestalt ones?
More Questions for DiscussionMore Questions for Discussion
What is the best possible outcome of a generative model of shape? Is such a thing worth pursuing?
How useful were Zhu' experiments and what others might be worth doing?