Upload
noah-sawyer
View
218
Download
1
Tags:
Embed Size (px)
Citation preview
Mean-Field Theory and Its Applications In Computer Vision5
1
Global Co-occurrence Terms
2
• Encourages global consistency and co-occurrence of objects
Without cooc
With co-occurrence
Global Co-occurrence Terms
3
• Defined on subset of labels• Associates a cost with each possible subset
Properties of cost function
4
Non-decreasing
0.2 0.2 0.2
3.0 3.0 3.0
5.0
Properties of cost function
5
We represent our cost as second order cost function defined on binary vector:
Complexity
6
• Complexity: O(NL2)
• Two relaxed (approximation) of this form• Complexity: O(NL+L2)
Our model• Represent 2nd order cost by binary latent variables • Unary cost per latent variable
7
1l
2l
3l
label level variable node (0/1)
Our model• Represent 2nd order cost by binary latent variables • Pairwise cost between latent variable
8
1l
2l
3l
Global Co-occurrence Cost• Two approximation to include into fully connected CRF
9
Global Co-occurrence Terms• First model
10
1l2l
3l
Global Co-occurrence Terms• Model
11
1l2l
3l
Global Co-occurrence Terms• Constraints (lets take one set of connections)
12
1l2l
3l
1l2l
3l
If latent variable is on, atleast one of image variable take that label
If latent variable is off, no image variable take that label
Global Co-occurrence Terms• Pay a cost K for violating first constraint
13
1l2l
3l
Global Co-occurrence Terms• Pay a cost K for violating second constrait
14
1l2l
3l
Global Co-occurrence Terms• Cost for first model:
15
1l
3l
Global Co-occurrence Terms• Second model
• Each latent node is connected to the variable node
16
1l2l
3l
Global Co-occurrence Terms• Constraints (lets take one set of connections)
17
1l2l
3l
1l2l
3l
If latent variable is on, atleast one of image variable take that label
If latent variable is off, no image variable take that label
Global Co-occurrence Terms• Pay a cost K for violating the constraint
18
1l2l
3l
Global Co-occurrence Terms• Cost for second model:
19
1l2l
3l
Global Co-occurrence Terms• Expectation evaluation for variable Yl• Case 1: Y_l takes label 0
20
1l2l
3l
Global Co-occurrence Terms• Expectation evaluation for variable Yl• Case 1: Y_l takes label 0
21
1l2l
3l
Global Co-occurrence Terms• Expectation evaluation for variable Yl• Case 1: Y_l takes label 0
22
1l2l
3l
Global Co-occurrence Terms• Expectation evaluation for variable Yl• Case 1: Y_l takes label 1
23
1l
2l
3l
Global Co-occurrence Terms• Expectation evaluation for variable Yl• Case 1: Y_l takes label 1
24
1l
2l
3l
Global Co-occurrence Terms• Expectation evaluation for variable Yl
25
Global Co-occurrence Terms• Latent variable updates:
26
Global Co-occurrence Terms• Latent variable updates:
27
Global Co-occurrence Terms
Pay a cost K if variable takes a label l and corresponding latent variable takes label 0
28
1l2l
3l
ComplexityExpectation updates for latent variable Y_l
29
ComplexityExpectation updates for latent variable Y_l
30
Overall complexity:
Does not increase original complexity:
PascalVOC-10 dataset
31Qualitative analysis: observe an improvement over other comparative methods
PascalVOC-10 dataset
32
Algorithm Time (s) Overall Av. Recall Av. I/U
AHCRF+Cooc 36 81.43 38.01 30.09
Dense CRF 0.67 71.43 34.53 28.40
Dense + Potts 4.35 79.87 40.71 30.18
Dense + Potts + Cooc
4.4 80.44 43.08 32.35
Observe an improvement of almost 2.3% improvement Almost 8-9 times faster than alpha-expansion based method
Mean-field Vs. Graph-cuts
33
• Measure I/U score on PascalVOC-10 segmentation • Increase standard deviation for mean-field• Increase window size for graph-cuts method
• Both achieve almost similar accuracy
Window sizes
34
Algorithm Model Time (s) Av. I/U
Alpha-exp (n=10) Pairwise 326.17 28.59
Mean-field pairwise 0.67 28.64
Alpha-exp (n=3) Pairwise + Potts 56.8 29.6
Mean-field Pairwise + Potts 4.35 30.11
Alpha-exp (n=1) Pairwise + Potts + Cooc
103.94 30.45
Mean-field Pairwise + Potts + Cooc
4.4 32.17
• Comparison on matched energy
Impact of adding more complex costs and increasing window size
PascalVOC-10 dataset
35
Algorithm bkg plane Cycle bird Boat
AHCRF+Cooc
82.5 43.2 4.9 17.4 27.1
Dense + Potts + Cooc
82.9 44.6 15.8 18.9 26.3
Algorithm bottle Bus car cat Chair
AHCRF+Cooc
31.3 49.4 51.0 29.3 7.1
Dense + Potts + Cooc
31.7 48.9 55.2 33.3 7.9
Per class Quantitative results
PascalVOC-10 dataset
36
Algorithm Cow Dtb dog horse Mbike
AHCRF+Cooc
26.7 8.3 17.0 24.0 27.1
Dense + Potts + Cooc
27.0 16.1 16.8 23.4 43.8
Algorithm
pson Plant sheep sofa train TV Av
AHCRF+Cooc
41.9 21.8 25.2 16.4 43.8 43.4 30.9
Dense + Potts + Cooc
38.4 21.1 30.9 15.5 44.0 36.8 32.35
Per class Quantitative results
Mean-field Vs. Graph-cuts
37
• Measure I/U score on PascalVOC-10 segmentation • Increase standard deviation for mean-field• Increase window size for graph-cuts method
•Time complexity very high, making infeasible to work with large neighbourhood system