Upload
others
View
1
Download
0
Embed Size (px)
Citation preview
PhD Viva-Voce
Some Applications of the Power WatershedFramework to Image Segmentation and Image
Filtering
Sravan Danda∗
Supervisor: B.S.Daya Sagar∗Joint Supervisor: Laurent Najman†
∗SSIU, Indian Statistical Institute, Bangalore Centre†LIGM, Universite Paris Est, ESIEE, France
October 11, 2019
PhD Viva-Voce
Overview
1 Watershed Segmentation on Graphs
2 Power Watershed (PW) Framework
3 PW for Fast Isoperimetric Image Segmentation
4 Mutex Watershed : PW Limit of Multi-Cut Graph Partitioning
5 PW for Image Filtering
6 Perspectives
PhD Viva-VoceWatershed Segmentation on Graphs
A synthetic gray-scale image
27 29
29
26
28
28
27
29
30 27
26 28 29 28 26
73 69 68 70 68
72 70 71 71 73
67 74 71 72 70
PhD Viva-VoceWatershed Segmentation on Graphs
Gradient Image2
2
3
1
2
1
1
2 2
1
3
1
0
3
2
3
2 1
2
47
1
41
1
39
2
42 42
4
1
1
1
2
3
2
1 5
2
5
1
4
0
0
2
1 3
7 3 1 2
PhD Viva-VoceWatershed Segmentation on Graphs
Flooding Simulation1
0 1
1
0
0
PhD Viva-VoceWatershed Segmentation on Graphs
Flooding Simulation
1 1
1
1 1
0 1
1 1
1
1
1 1
1 0
0 1
PhD Viva-VoceWatershed Segmentation on Graphs
Flooding Simulation2
1 1
1
1 1
0 2 1
2 1 1
1
1
1
2
1
1 0
0
2
1
2
PhD Viva-VoceWatershed Segmentation on Graphs
Flooding Simulation2
1 1
1
1 1
0 2 1
2 1 1
1
1
1
2
1
1 0
0
2
1
3 2
PhD Viva-VoceWatershed Segmentation on Graphs
Flooding Simulation2
1 1
1
1 1
0 2 1
2 1 1
1
1
1
2
1
5
1 0
0
2
1
3 2
PhD Viva-VoceWatershed Segmentation on Graphs
Watershed as a Graph-cut
Watershed Cut:Minimum Spanning Forest Cut w.r.t. Minima 1
1Watershed cuts: Minimum spanning forests and the drop of waterprinciple, J Cousty, G Bertrand, L Najman, M Couprie, IEEE Transactions onPattern Analysis and Machine Intelligence 31 (8), 1362-1374
PhD Viva-VoceWatershed Segmentation on Graphs
Watershed as a Limit of Total Variation Minimizers
Power Watershed 2
limp→∞ x(p) where
x(p) = arg minx
(Q(p)(x))
Q(p)(x) =∑eij∈E
wpij |xi − xj |2 +
∑i∈Seed
wpi |xi − fi |2
2Power watershed: A unifying graph-based optimization framework, CCouprie, L Grady, L Najman, H Talbot, IEEE transactions on pattern analysisand machine intelligence 33 (7), 1384-1399
PhD Viva-VoceWatershed Segmentation on Graphs
Power Watershed: Fast Watershed Cut
Power Watershed: Seeded Image Segmentation 3
3Power watershed: A unifying graph-based optimization framework, CCouprie, L Grady, L Najman, H Talbot, IEEE transactions on pattern analysisand machine intelligence 33 (7), 1384-1399
PhD Viva-VocePower Watershed (PW) Framework
Power Watershed Framework
Let 0 < λ1 < λ2 < · · · < λk
Q(x) =∑
iλi Qi (x)
PhD Viva-VocePower Watershed (PW) Framework
Power Watershed Framework
Let 0 < λ1 < λ2 < · · · < λk
Q(p)(x) =∑
iλp
i Qi (x)
PhD Viva-VocePower Watershed (PW) Framework
Power Watershed Framework
Let 0 < λ1 < λ2 < · · · < λk
Q(p)(x) =∑
iλp
i Qi (x)
x(p) = arg minx
Q(p)(x)
PhD Viva-VocePower Watershed (PW) Framework
Power Watershed Framework
Let 0 < λ1 < λ2 < · · · < λk
Q(p)(x) =∑
iλp
i Qi (x)
x(p) = arg minx
Q(p)(x)
x(p) → x∗ (?)
PhD Viva-VocePower Watershed (PW) Framework
Power Watershed - Generic Algorithm
Algorithm 1 Generic Algorithm to compute limit of minimizers 4
Input: Function Q(p)(x) =∑k
i=1 λpi Qi (x), where λk > λk−1 >
· · · > λ1 > 0.Output: x∗ :
1: Mk = arg min Qk(x) where x ∈ C2: for i from k − 1 to 1 do3: Compute Mi = arg min Qi (x) where x ∈ Mi+14: end for
4Extending the Power Watershed Framework Thanks to Γ-Convergence,LNajman, SIAM Journal on Imaging Sciences 10 (4), 2275-2292
PhD Viva-VocePower Watershed (PW) Framework
Power Watershed Framework - Why?
Relates Watershed-Cuts with Random Walker and ShortestPath Segmentation
Results in a faster Watershed-Cut algorithm
PhD Viva-VocePower Watershed (PW) Framework
Power Watershed Framework - Why?
Relates Watershed-Cuts with Random Walker and ShortestPath SegmentationResults in a faster Watershed-Cut algorithm
PhD Viva-VocePower Watershed (PW) Framework
Laplacian Matrix
a
bc
d
e f
3
3
3
3
3
32
1 1
a b c d e fa 7 −3 0 −1 0 −3b −3 6 −3 0 0 0c 0 −3 7 −3 0 −1d −1 0 −3 9 −3 −2e 0 0 0 −3 6 −3f −3 0 −1 −2 −3 9
PhD Viva-VocePower Watershed (PW) Framework
Fast Spectral Clustering using PW Framework
minimizeH∈Rn×m
Tr(HtLH)
subject to HtH = I(1)
PhD Viva-VocePower Watershed (PW) Framework
Laplacian Matrix: Decomposition
a
bc
d
e f
3
3
3
3
3
3
a b c d e fa 6 −3 0 0 0 −3b −3 6 −3 0 0 0c 0 −3 6 −3 0 0d 0 0 −3 6 −3 0e 0 0 0 −3 6 −3f −3 0 0 0 −3 6
PhD Viva-VocePower Watershed (PW) Framework
Laplacian Matrix: Decomposition
a
bc
d
e f
2
a b c d e fa 0 0 0 0 0 0b 0 0 0 0 0 0c 0 0 0 0 0 0d 0 0 0 2 0 −2e 0 0 0 0 0 0f 0 0 0 −2 0 2
PhD Viva-VocePower Watershed (PW) Framework
Laplacian Matrix: Decomposition
a
bc
d
e f
1 1
a b c d e fa 1 0 0 −1 0 0b 0 0 0 0 0 0c 0 0 1 0 0 −1d −1 0 0 1 0 0e 0 0 0 0 0 0f 0 0 −1 0 0 1
PhD Viva-VocePower Watershed (PW) Framework
Fast Spectral Clustering using PW Framework
minimizeH∈Rn×m
∑i
wpi Tr(HtLi H)
subject to HtH = I(2)
PhD Viva-VocePower Watershed (PW) Framework
Scalability of Spectral Clustering Algorithms
Traditional Spectral Clustering: O(n 32 )
Power Spectral Clustering: O(nlogn)
where n are non-zero entries in L
PhD Viva-VocePower Watershed (PW) Framework
Scalability of Spectral Clustering Algorithms
Traditional Spectral Clustering: O(n 32 )
Power Spectral Clustering: O(nlogn)
where n are non-zero entries in L
PhD Viva-VocePower Watershed (PW) Framework
PW Framework for other Image Segmentation Algorithms?
Can we obtain faster algorithms for other image segmentationmethods?
Yes!
PhD Viva-VocePower Watershed (PW) Framework
PW Framework for other Image Segmentation Algorithms?
Can we obtain faster algorithms for other image segmentationmethods?Yes!
PhD Viva-VocePW for Fast Isoperimetric Image Segmentation
Image: Similarity Graph
2
1
1
3 3
2
2
3
2 1
13 10
wij = 100 exp(− ||i−j||σ )
PhD Viva-VocePW for Fast Isoperimetric Image Segmentation
Isoperimetric Partitioning
2
1
1
3 3
2
2
3
2 1
13 10
PhD Viva-VocePW for Fast Isoperimetric Image Segmentation
Isoperimetric Partitioning
Isoperimetric Cost(A) = W (A,A)min{|A|, n − |A|}
PhD Viva-VocePW for Fast Isoperimetric Image Segmentation
Isoperimetric Partitioning
xi ={
0 if vi ∈ A1 if vi ∈ A
PhD Viva-VocePW for Fast Isoperimetric Image Segmentation
Isoperimetric Partitioning
xi ={
0 if vi ∈ A1 if vi ∈ A
xtLx = xtDx− xtW x=
∑i ,j
xi dijxj −∑i ,j
xi wijxj
=∑i ,j
wijx2i −
∑i ,j
xi wijxj
=∑i ,j
wij(x2i − 2xi xj + x2
j )
=∑i ,j
wij(xi − xj)2
PhD Viva-VocePW for Fast Isoperimetric Image Segmentation
Isoperimetric Partitioning
xi ={
0 if vi ∈ A1 if vi ∈ A
xtLx = W (A,A)
PhD Viva-VocePW for Fast Isoperimetric Image Segmentation
Isoperimetric Partitioning
xi ={
0 if vi ∈ A1 if vi ∈ A
|A| = xt1
n − |A| = (1− x)t1
PhD Viva-VocePW for Fast Isoperimetric Image Segmentation
Isoperimetric Partitioning
Discrete Formulation:
minimizex
xtLxmin{xt1, (1− x)t1}
subject to xi ∈ {0, 1} ∀i(3)
PhD Viva-VocePW for Fast Isoperimetric Image Segmentation
Isoperimetric Partitioning
Continuous Relaxation:
minimizex
xtLxmin{xt1, (1− x)t1}
subject to xi ∈ [0, 1] ∀i(3)
Select best threshold!
PhD Viva-VocePW for Fast Isoperimetric Image Segmentation
Isoperimetric Partitioning
Seed Constraint xj = 0
minimizex
xt−jL−jx−j
min{xt−j1, (1− x−j)t1}
subject to xi ∈ [0, 1] for i 6= j(3)
Select best threshold!
PhD Viva-VocePW for Fast Isoperimetric Image Segmentation
Isoperimetric Partitioning
Lagrange Multipliers 5
L−jx−j = 1 (3)
5Isoperimetric graph partitioning for image segmentation, L Grady, ELSchwartz, IEEE Transactions on Pattern Analysis and Machine Intelligence,469-475
PhD Viva-VocePW for Fast Isoperimetric Image Segmentation
Fast Isoperimetric Partitioning
Solve 6
LMaxST−j x−j = 1 (4)
6Fast, quality, segmentation of large volumes - isoperimetric distance trees,L Grady, European Conference on Computer Vision, 449-462
PhD Viva-VocePW for Fast Isoperimetric Image Segmentation
Fast Isoperimetric Partitioning
2
3 3
2 3
2
13 10
PhD Viva-VocePW for Fast Isoperimetric Image Segmentation
Fast Isoperimetric Partitioning: Computational Cost
O(n) : where n are non-zero entries in L
PhD Viva-VocePW for Fast Isoperimetric Image Segmentation
Why does the MST heuristic work?
Power Watershed Framework ⇒ Enough to solve on UMaxST! 7
7Revisiting the Isoperimetric Graph Partitioning Problem, S Danda, AChalla, BD Sagar, L Najman, available athttps://hal.archives-ouvertes.fr/hal-01810249
PhD Viva-VocePW for Fast Isoperimetric Image Segmentation
Fast Isoperimetric Partitioning Using PW
2
3 3
2
2
3
2
13 10
PhD Viva-VocePW for Fast Isoperimetric Image Segmentation
Are Solutions on MST and UMST the same?
a c
e
db
g
1
1
2
1
3
3
0.9
Original Graph
PhD Viva-VocePW for Fast Isoperimetric Image Segmentation
Are Solutions on MST and UMST the same?
a c
e
db
g
1
1
2
1
3
3
UMST Graph
PhD Viva-VocePW for Fast Isoperimetric Image Segmentation
Are Solutions on MST and UMST the same?
a c
e
db
g
1
2
1
3
3
a c
e
db
g
1
1
2
3
3
a c
e
db
g
1
2
1
3
3
All Possible MSTs
PhD Viva-VocePW for Fast Isoperimetric Image Segmentation
Solving Linear System on Graph, UMST and an arbitraryMST are different!
Node Original UMST MST1 MST2 MST3g 0.00 0.00 0.00 0.00 0.00a 1.69 2.68 5.00 4.00 11.16b 1.54 2.32 9.66 1.00 5.00c 2.04 3.52 7.00 5.50 10.66d 2.09 3.64 8.66 6.50 9.00e 1.94 3.74 8.00 6.16 10.00
PhD Viva-VocePW for Fast Isoperimetric Image Segmentation
How different are Solutions on MST and UMST?
LemmaLet Tumst and Tmst denote the operators on UMST and MSTrespectively, as defined above. Then there exists two positiveconstants K1 and K2 such that
K1
k∑i=1
(ui −mi )2w2i ≤ ‖Tumst − Tmst‖ ≤ K2
k∑i=1
(ui −mi )2w2i . (5)
PhD Viva-VocePW for Fast Isoperimetric Image Segmentation
Results in Practice
F-Score
Comparison of MaxST and UMaxST as a sufficient statistic!
PhD Viva-VocePW for Fast Isoperimetric Image Segmentation
Results in Practice
Adjusted Rand Index
Comparison of MaxST and UMaxST as a sufficient statistic!
PhD Viva-VocePW for Fast Isoperimetric Image Segmentation
Results in Practice
Scatter plot of Normalized values of solutions
Strictly increasing plot implies perfectly consistent solutions!
PhD Viva-VocePW for Fast Isoperimetric Image Segmentation
Results in Practice
Inversions
Comparison of MaxST and UMaxST as a sufficient statistic!
PhD Viva-VocePW for Fast Isoperimetric Image Segmentation
Results in PracticeData Reduction
0 20 40 60 80 100Reduction (%)
0
20
40
60
80
100
120
Coun
t (To
tal o
f 500
)
PhD Viva-VocePW for Fast Isoperimetric Image Segmentation
Contributions
1 Detailed Analysis of the relaxed Cheeger Cut problem2
3
4
PhD Viva-VocePW for Fast Isoperimetric Image Segmentation
Contributions
1 Detailed Analysis of the relaxed Cheeger Cut problem2 Establish using PW framework that considering UMST graph
acts as a sufficient statistic3
4
PhD Viva-VocePW for Fast Isoperimetric Image Segmentation
Contributions
1 Detailed Analysis of the relaxed Cheeger Cut problem2 Establish using PW framework that considering UMST graph
acts as a sufficient statistic3 Establish bounds between UMST and MST based
implementations4
PhD Viva-VocePW for Fast Isoperimetric Image Segmentation
Contributions
1 Detailed Analysis of the relaxed Cheeger Cut problem2 Establish using PW framework that considering UMST graph
acts as a sufficient statistic3 Establish bounds between UMST and MST based
implementations4 Empirically establish that UMST based reduction is robust
compared to MST based implementation
PhD Viva-VoceMutex Watershed : PW Limit of Multi-Cut Graph Partitioning
Mutex Watershed: 8 The Setup
G = (V ,E ,W )
8Steffen Wolf, Constantin Pape, Alberto Bailoni, Nasim Rahaman, AnnaKreshuk, Ullrich Kothe, and Fred A. Hamprecht. The mutex watershed:Efficient, parameter-free image partitioning. In Vittorio Ferrari, Martial Hebert,Cristian Sminchisescu, and Yair Weiss, editors, Computer Vision - ECCV 2018 -15th European Conference, Munich, Germany, September 8-14, 2018,Proceedings, Part 4, volume 11208 of Lecture Notes in Computer Science,pages 571–587. Springer, 2018
PhD Viva-VoceMutex Watershed : PW Limit of Multi-Cut Graph Partitioning
Mutex Watershed: 8 The Setup
G = (V ,E ,W )f : E → {−1,+1}
8Steffen Wolf, Constantin Pape, Alberto Bailoni, Nasim Rahaman, AnnaKreshuk, Ullrich Kothe, and Fred A. Hamprecht. The mutex watershed:Efficient, parameter-free image partitioning. In Vittorio Ferrari, Martial Hebert,Cristian Sminchisescu, and Yair Weiss, editors, Computer Vision - ECCV 2018 -15th European Conference, Munich, Germany, September 8-14, 2018,Proceedings, Part 4, volume 11208 of Lecture Notes in Computer Science,pages 571–587. Springer, 2018
PhD Viva-VoceMutex Watershed : PW Limit of Multi-Cut Graph Partitioning
Mutex Watershed: 8 The Setup
G = (V ,E ,W )f : E → {−1,+1}W : E → R+
8Steffen Wolf, Constantin Pape, Alberto Bailoni, Nasim Rahaman, AnnaKreshuk, Ullrich Kothe, and Fred A. Hamprecht. The mutex watershed:Efficient, parameter-free image partitioning. In Vittorio Ferrari, Martial Hebert,Cristian Sminchisescu, and Yair Weiss, editors, Computer Vision - ECCV 2018 -15th European Conference, Munich, Germany, September 8-14, 2018,Proceedings, Part 4, volume 11208 of Lecture Notes in Computer Science,pages 571–587. Springer, 2018
PhD Viva-VoceMutex Watershed : PW Limit of Multi-Cut Graph Partitioning
Mutex Watershed: State-of-the-art on ISBI 2012
PhD Viva-VoceMutex Watershed : PW Limit of Multi-Cut Graph Partitioning
Mutex Watershed: Algorithm
Algorithm 2 Mutex Watershed
Initialize A = ∅.for each edge e in descending order of W (e) do
if A ∪ e does not violate the mutex condition thenA← A ∪ e
end ifend forreturn Subgraph induced by {e ∈ A|f (e) = +1}
PhD Viva-VoceMutex Watershed : PW Limit of Multi-Cut Graph Partitioning
Mutex Watershed: An Example
1 21
6
5
3-2
7
2
43
8
-4
54
9
2
10
9
2
11
9
1
12
-7
-3
13
5
8
14
-6
15
-7
-8
16
-5
9
17
2
5
18
8
4
19
5
20
7
2
21
3
9
22
1
6
23
9
2
24
8
25
2
1 3 4 7
PhD Viva-VoceMutex Watershed : PW Limit of Multi-Cut Graph Partitioning
Mutex Watershed: Walk-Through
1 2 3 4 5
10
9
6
11
9
7 8 9
12 139
14 15
16 17 189
23
9
19 20
21 22 24 25
PhD Viva-VoceMutex Watershed : PW Limit of Multi-Cut Graph Partitioning
Mutex Watershed: Walk-Through
1 2 3 4 5
10
9
6
11
9
7 8 98
12-8
139
18
8
14 15
16 179
23
9
19
24
8
20
21 22 25
PhD Viva-VoceMutex Watershed : PW Limit of Multi-Cut Graph Partitioning
Mutex Watershed: Walk-Through
1 2 3 4 5
10
9
6
11
9
7
12
-7
8 98
15
-7
-813
9
18
8
14
20
7
16 179
23
9
19
24
8
21 22 257
PhD Viva-VoceMutex Watershed : PW Limit of Multi-Cut Graph Partitioning
Mutex Watershed: Walk-Through
1 2 3 4 5
10
9
6
11
9
7
12
-7
8 98
14
-6
15
-7
-813
9
18
8
20
7
16 179
196
23
9
24
8
21 22 257
PhD Viva-VoceMutex Watershed : PW Limit of Multi-Cut Graph Partitioning
Mutex Watershed: Walk-Through
1
6
5
2 3 4 5
10
9
11
9
7
12
-7
8
13
5
98
14
-6
15
-7
-8
16
-5
9 5
18
8
20
7
179
196
23
9
24
8
21 22 257
PhD Viva-VoceMutex Watershed : PW Limit of Multi-Cut Graph Partitioning
Mutex Watershed: Walk-Through
1
6
5
2 3
8
-4
4 54
10
9
11
9
7
12
-7
13
5
98
14
-6
15
-7
-8
16
-5
9 5
18
8
4
20
7
179
196
23
9
24
8
21 22 257
PhD Viva-VoceMutex Watershed : PW Limit of Multi-Cut Graph Partitioning
Mutex Watershed: Walk-Through
1
6
5
2 3 43
8
-4
54
10
9
11
9
7
12
-7
9-3
13
5
8
14
-6
15
-7
-8
16
-5
9 5
18
8
4
20
7
21
3
179
196
23
9
24
8
223
257
PhD Viva-VoceMutex Watershed : PW Limit of Multi-Cut Graph Partitioning
Mutex Watershed: Walk-Through
1
6
5
2 3-2
7
2
43
8
-4
54
10
9
2
11
9
12
-7
9-3
13
5
8
14
-6
15
-7
-8
16
-5
9 5
18
8
4
20
7
172
21
3
919
6
23
9
24
8
223
257
PhD Viva-VoceMutex Watershed : PW Limit of Multi-Cut Graph Partitioning
Mutex Watershed: Segments
1
6
5
2
7
2
3 43
54
10
9
2
11
9
8
13
5
98
129
145
18
8
154
20
7
16 172
21
3
919
6
23
9
24
8
223
257
PhD Viva-VoceMutex Watershed : PW Limit of Multi-Cut Graph Partitioning
Multi-Cut Graph Partitioning
Q(a) = mina∈{0,1}|E |
−∑e∈E
aewe
s.t C1(A) = ∅ with A = {e ∈ E |ae = 1}(6)
NP-Hard!
PhD Viva-VoceMutex Watershed : PW Limit of Multi-Cut Graph Partitioning
Mutex Watershed: PW Limit of Multi-Cut
Q(p)(a) = mina∈{0,1}|E |
−∑e∈E
aewpe
s.t C1(A) = ∅ with A = {e ∈ E |ae = 1}(7)
PhD Viva-VoceMutex Watershed : PW Limit of Multi-Cut Graph Partitioning
Mutex Watershed: PW Limit of Multi-Cut
Gk = (V ,Ek , W |Ek)
mina∈{0,1}|Ek |
−∑
e∈Ek
ae
s.t C1(A) = ∅ with A = {e ∈ Ek |ae = 1}(8)
denote the solution space by Ak .
PhD Viva-VoceMutex Watershed : PW Limit of Multi-Cut Graph Partitioning
Mutex Watershed: PW Limit of Multi-Cut
Gk−1 = (V ,Ek−1, W |Ek−1)
mina∈{0,1}|Ek−1|
−∑
e∈Ek−1
ae
s.t C1(A) = ∅ with A = Ak ∪ {e ∈ Ek−1|ae = 1}(9)
denote the solution space by Ak−1.
PhD Viva-VoceMutex Watershed : PW Limit of Multi-Cut Graph Partitioning
Mutex Watershed: PW Limit of Multi-Cut
Repeat until all edges are processed.
PhD Viva-VoceMutex Watershed : PW Limit of Multi-Cut Graph Partitioning
Mutex Watershed: PW Limit of Multi-Cut
Sub-problems can be handled with a ‘Union-Find’ datastructure.
Sub-problems are tractable!
PhD Viva-VoceMutex Watershed : PW Limit of Multi-Cut Graph Partitioning
Mutex Watershed: PW Limit of Multi-Cut
Sub-problems can be handled with a ‘Union-Find’ datastructure.Sub-problems are tractable!
PhD Viva-VoceMutex Watershed : PW Limit of Multi-Cut Graph Partitioning
Contributions
Mutex Watershed is PW limit of Multi-cut partitioning
PhD Viva-VocePW for Image Filtering
PW Framework for Image Filtering?
Can we relate image filtering tools?
Yes!
PhD Viva-VocePW for Image Filtering
PW Framework for Image Filtering?
Can we relate image filtering tools?Yes!
PhD Viva-VocePW for Image Filtering
Shortest Path Filter
SPF i =∑j∈V
gi (j)Ij ,
PhD Viva-VocePW for Image Filtering
Shortest Path Filter
SPF i =∑j∈V
gi (j)Ij ,
where
gi (j) =exp
(−Θ(i ,j)
σ
)∑
k∈V exp(−Θ(i ,k)
σ
)
PhD Viva-VocePW for Image Filtering
Shortest Path Filter
2
2
2
2
1
i
2
1
1 30
2
1
1
10
2
j
14
27
19
2
11
10
1
2
1
2
2
1
2 1
2 1 2
Θi (j) = 3
PhD Viva-VocePW for Image Filtering
Tree Filter 9
TFi =∑
jti (j)Ij
9Linchao Bao, Yibing Song, Qingxiong Yang, Hao Yuan, and Gang Wang.Tree filtering: Efficient structure-preserving smoothing with a minimumspanning tree. IEEE TIP, 23(2): 555-569, 2014.
PhD Viva-VocePW for Image Filtering
Tree Filter 9
TFi =∑
jti (j)Ij
where
ti (j) =exp(−D(i ,j)
σ )∑q exp(−D(i ,q)
σ )
9Linchao Bao, Yibing Song, Qingxiong Yang, Hao Yuan, and Gang Wang.Tree filtering: Efficient structure-preserving smoothing with a minimumspanning tree. IEEE TIP, 23(2): 555-569, 2014.
PhD Viva-VocePW for Image Filtering
Tree Filter on a Synthetic Graph
1
2
2
3
1
2
2
2 2
2
2
1
3
3
2
1
2 2
2
11
3
11
2
10
1
10 11
2
1
2
3
2
1
2
2 2
1
1
1
2
2
100
2
2 2
3 100 101 3
Gradient image
PhD Viva-VocePW for Image Filtering
Tree Filter on a Synthetic Graph
1
2
2 1
2
2
2
2
1
i
2
1
2
2 1
10
1
j2
1
2
2
1
1
1
2
2
100
2
2 2
ti (j) = 9
PhD Viva-VocePW for Image Filtering
Tree Filter on a Synthetic Image
L to R: Noisy Image, TF, TF + BF
PhD Viva-VocePW for Image Filtering
Can the Tree Filter be explained?
Power Watershed Framework ⇒ Tree Filter is an approximate limitof Shortest Path Filters 10
10Some Theoretical Links between Shortest Path Filters and MinimumSpanning Tree Filters, S Danda, A Challa, BD Sagar, L Najman, Journal ofMathematical Imaging and Vision, January 2019
PhD Viva-VocePW for Image Filtering
Limit of Shortest Path Filters
j2
1
1
i
4 4
2
1
3
2 1
13 10
j4
1
1
i
16 16
4
1
9
4 1
169 100
L to R: Image Graph, Image Graph with edge weights raised topower 2
PhD Viva-VocePW for Image Filtering
Limit of Shortest Path Filters: Characterization
LemmaLet G = (V ,E ,W ). For every pair of pixels i and j in V , thereexists p0 ≥ 1 such that, a path P(i , j) is a shortest path between iand j in G(p) for all p ≥ p0 if and only if P(i , j) is a smallest pathw.r.t. reverse dictionary order between i and j in G. Further, p0 isindependent of i and j.
PhD Viva-VocePW for Image Filtering
Reverse Dictionary Order: Illustration
1
S
1
1
1
T
1
1 2
1
S
1
1
1
T
1
1 2
PhD Viva-VocePW for Image Filtering
Limit of Shortest Path Filters: Characterization
LemmaEvery smallest path w.r.t. reverse dictionary order between any twoarbitrary nodes in G = (V ,E ,W ) lies on an MST of G and henceon the UMST of G.
PhD Viva-VocePW for Image Filtering
Limit of Shortest Path Filters: UMST Filter
1 21
6
2
32
41
8
2
52
9
2
10
2
72
11
2
1
13
2
1
14
2
15
2
122 2
18
10
1
19
10
16 172
21
1
2 2
23
1
202
24
2
25
2
221
26
1
1
27
2
2
28
100
2
29
2
30
2
100
PhD Viva-VocePW for Image Filtering
UMST Filter: Characterization
LemmaFor every pixel i in the image I, there exists a spanning tree Ti(termed as adaptive spanning tree), such that Ti contains asmallest path with respect to reverse dictionary ordering betweenpixels i and any other pixel j in I.
PhD Viva-VocePW for Image Filtering
UMST Filter: Adaptive Spanning Trees
1
1
1
1 1
1
10
2
14 19
2 2
1
1
1 1
1
10
2 2
1
1
1
1
1
10
2 2
PhD Viva-VocePW for Image Filtering
UMST Filter: Depth-Based Approximation
1 21
32
41
52
9
2
6 72
11
2
81
13
2
101
14
2
122 2
151
19
10
16
21
1
17 182 2
23
1
202
24
2
221
26
1
1
27
2
28
100
252
29
2
30
2
PhD Viva-VocePW for Image Filtering
UMST Filter: Order-Based Approximation
1 21
32
41
52
9
2
6 72
11
2
81
13
2
101
14
2
122 2
151
19
10
16
21
1
17 182 2
23
1
202
24
2
221
26
1
1
27
2
28
100
252
29
2
30
2
PhD Viva-VocePW for Image Filtering
Results in Practice
Salt and Pepper Noise
L to R: House Image, Bilateral Filter, Tree Filter, OurApproximation to Limit of SPFs
PhD Viva-VocePW for Image Filtering
Results in Practice
Structural Similarity Indices
Mean SSIM on Salt Pepper NoiseBF TF UMSTF
House 0.69 0.80 0.83Barbara 0.72 0.66 0.72Lena 0.69 0.75 0.79Pepper 0.62 0.74 0.74Mean 0.68 0.74 0.77
PhD Viva-VocePW for Image Filtering
Results in Practice
SSIM: Tree Filter vs UMST Filter 11
0.50 0.55 0.60 0.65 0.70 0.75 0.80 0.85 0.90Tree Filter
0.50
0.55
0.60
0.65
0.70
0.75
0.80
0.85
0.90Ou
r App
roximation
11Some theoretical links between shortest path filters and minimum spanningtree filters, S Danda, A Challa, BSD Sagar, L Najman, available athttps://hal.archives-ouvertes.fr/hal-01617799v5
PhD Viva-VocePW for Image Filtering
Contributions
1 Establish UMST filter as a limit of shortest path filters2
3
PhD Viva-VocePW for Image Filtering
Contributions
1 Establish UMST filter as a limit of shortest path filters2 Tree filter as an approximation to UMST filter3
PhD Viva-VocePW for Image Filtering
Contributions
1 Establish UMST filter as a limit of shortest path filters2 Tree filter as an approximation to UMST filter3 Implement Depth-based and Order-based approximations of
UMST filter
PhD Viva-VocePW for Image Filtering
Perspectives
1 Adaptive Spanning Trees can be processed in parallel!2
PhD Viva-VocePW for Image Filtering
Perspectives
1 Adaptive Spanning Trees can be processed in parallel!2 Can we learn edge-aware features using the adaptive spanning
trees?
PhD Viva-VocePerspectives
Perspectives
1 Can we speed-up other tree-based algorithms such as scale-setanalysis?
2
3
4
PhD Viva-VocePerspectives
Perspectives
1 Can we speed-up other tree-based algorithms such as scale-setanalysis?
2 Total Variation ↔ Cheeger Cut. Application to TVminimization!
3
4
PhD Viva-VocePerspectives
Perspectives
1 Can we speed-up other tree-based algorithms such as scale-setanalysis?
2 Total Variation ↔ Cheeger Cut. Application to TVminimization!
3 Understanding working principle behind PW framework?4
PhD Viva-VocePerspectives
Perspectives
1 Can we speed-up other tree-based algorithms such as scale-setanalysis?
2 Total Variation ↔ Cheeger Cut. Application to TVminimization!
3 Understanding working principle behind PW framework?4 PW implies UMST is a sufficient statistic for image
segmentation and filtering. Can we obtain sufficient statisticsfor graph-modelled data in general?
PhD Viva-VocePerspectives
I would like to thank my advisors, Prof. B S Daya Sagar and Prof.Laurent Najman all their support, anonymous reviewers of myarticles and anonymous examiners and CCSD faculty for theirsuggestions, and Indian Statistical Institute for providing me
fellowship to pursue this research.