Image Foresting Transform

1

Image Foresting Transformfor Image Segmentation

Presented by:Michael FangWeilong Yang

2

A Few Things to RecallImage Segmentation

◦Finding homogeneous regionsGraph-based Methods

◦Treating images as graphsImage Foresting Transform

◦Unification◦Efficiency◦Simplicity

3

Directed GraphsA directed graph is a pair (I, A), where I is a set of nodes and A is a set of ordered pairs of nodes.

4

PathsA path is a sequence t1, t2, …, tk

of distinct nodes in the graph, such that (ti, ti+1) A for 1 i k – 1.

A path is trivial if k = 1;Path denotes the

concatenation of two paths, and , where ends at t and begins at t.

Path = s, t denotes theconcatenation of the longest prefix of and the last arc (s, t).

5

Path CostsA path-cost function is a mapping

that assigns to each path a cost (), in some ordered set of cost values.

A function is said to be monotonic-incremental (MI) when

(t) = h(t),( s, t) = () (s, t),

where h(t) is a handicap cost value and satisfies: x’ x x’ (s, t) x (s, t) and x (s, t) x, for x, x’ and (s, t) A.

6

Examples of MI Cost Functions

Additive cost function sum(t) = h(t),

sum( s, t) = sum() + w(s, t),where w(s, t) is a fixed non-negative arc weight.

Max-arc cost function max(t) = h(t),

max( s, t) = max{max(), w(s, t)}, where w(s, t) is a fixed arc weight.

7

Predecessor Map and Spanning ForestA predecessor map is a function P that

assigns to each node t I either some other node in I, or a distinctive marker nil I – in which case t is the root of the map.

A spanning forest is a predecessor map which takes every node to nil in a finite number of iterations (i.e., it contains no cycles).

8

Paths of the Forest PFor any node t I, there is a path

P*(t) which is obtained in backward by following the predecessor nodes along the path.

P*(c) = a, b, c, where P(c) = b, P(b) = a,

P(a) = nil

P*(i) = i, where P(i) = nil

9

Optimum-path Forest

An optimum-path forest is a spanning forest P, where (P*(t)) is minimum for all nodes t I. Consider cost function sum in the example below.

10

An Image as a Directed GraphA grayscale image I is a pair (I, I),

where I is a finite set of pixels (points in Z2) and I assigns to each pixel t I a value I(t) in some arbitrary value space.

An adjacency relation A is a binary relation between pixels of I, which is usually translation-invariant.

Once A has been fixed, image I can be interpreted as a directed graph, whose nodes are the image pixels in I and whose arcs are defined by A.

11

Seed Pixels

In some applications, we would like to use a predefined path-cost function but constrain the search to paths that start in a given set S I of seed pixels. This constraint can be modeled by defining

12

IFT Algorithm for Image Segmentation

1. Path Cost

2. Four-Connected Adjacency

13

IFT algorithm with FIFO policy(1)

Initialization

It

C(t)

14

016

0

013

0

5 5 5 5

5 5 5 5

5 5 5 5

0 5 5 0

1 2 3 4

5 6 7 8

9 10 11 12

13 14 15 16

∝ ∝ ∝ ∝

∝ ∝ ∝ ∝

∝ ∝ ∝ ∝

0 ∝ ∝ 0

IFT algorithm with FIFO policy(2)

15

Growing Process

IFT algorithm with FIFO policy(3)

16

016

0

59

5

514

5

5 5 5 5

5 5 5 5

5 5 5 5

0 5 5 0

1 2 3 4

5 6 7 8

9 10 11 12

13 14 15 16

∝ ∝ ∝ ∝

∝ ∝ ∝ ∝

5 ∝ ∝ ∝

0 5 ∝ 0

IFT algorithm with FIFO policy(4)

17

59

5

514

5

515

5

512

5

5 5 5 5

5 5 5 5

5 5 5 5

0 5 5 0

1 2 3 4

5 6 7 8

9 10 11 12

13 14 15 16

∝ ∝ ∝ ∝

∝ ∝ ∝ ∝

5 ∝ ∝ 5

0 5 5 0

IFT algorithm with FIFO policy(4)

18

514

5

512

5

515

5

55

5

510

5

5 5 5 5

5 5 5 5

5 5 5 5

0 5 5 0

1 2 3 4

5 6 7 8

9 10 11 12

13 14 15 16

∝ ∝ ∝ ∝

5 ∝ ∝ ∝

5 5 ∝ 5

0 5 5 0

IFT algorithm with FIFO policy(4)

19

515

5

512

5

55

5

510

5

5 5 5 5

5 5 5 5

5 5 5 5

0 5 5 0

1 2 3 4

5 6 7 8

9 10 11 12

13 14 15 16

∝ ∝ ∝ ∝

5 ∝ ∝ ∝

5 5 ∝ 5

0 5 5 0

IFT algorithm with FIFO policy(4)

20

515

5

55

5

510

5

58

5

511

5

5 5 5 5

5 5 5 5

5 5 5 5

0 5 5 0

1 2 3 4

5 6 7 8

9 10 11 12

13 14 15 16

∝ ∝ ∝ ∝

5 ∝ ∝ 5

5 5 5 5

0 5 5 0

IFT algorithm with FIFO policy(4)

21

5 5 5 5

5 5 5 5

5 5 5 5

0 5 5 0

1 2 3 4

5 6 7 8

9 10 11 12

13 14 15 16

5 5 5

5 5 5 5

0 5 5 0

5

5 5 5 5

IFT algorithm with FIFO policy(4)

22

Another Example

23

Input Image

Gradient Image

Seeds Labeling

IFT

Framework of Image segmentation by IFT

24

Experiment Results (1)

25

Experiment Results (2)

26

Experiment Results (3)

27

Experiment Results (4)

28

SummaryBasic concept of the Image

Foresting TransformIFT for image segmentationExperiment results

29

References (1)1. Martin, D., Fowlkes, C., Tal, D., and Malik, J., A database of human segmented natural

images and its application to evaluating segmentation algorithms and measuring ecological statistics, ICCV(2), No. 7-14, January 2001, pp. 416–425.

2. Vincent, L., and Soille, P.[Pierre], Watersheds in Digital Spaces: An Efficient Algorithm Based on Immersion Simulations, PAMI(13), No. 6, June 1991, pp. 583-598.

3. Makrogiannis, S., Economou, G., Fotopoulos, S., A region dissimilarity relation that combines feature-space and spatial information for color image segmentation, SMC-B(35), No. 1, February 2005, pp. 44-53.

4. Haris, K., Efstratiadis, S.N., Maglaveras, N., Katsaggelos, A.K., Hybrid Image Segmentation Using Watersheds And Fast Region Merging, IP(7), No. 12, December 1998, pp. 1684-1699.

5. Shi, J.[Jianbo], Malik, J.[Jitendra], Normalized Cuts and Image Segmentation, PAMI(22), No. 8, August 2000, pp. 888-905.

6. Zabih, R.[Ramin], Kolmogorov, V.[Valdimir], Spatially coherent clustering using graph cuts, CVPR04(II: 437-444).

7. Boykov, Y.Y.[Yuri Y.], Jolly, M.P.[Marie-Pierre], Interactive Graph Cuts for Optimal Boundary and Region Segmentation of Objects in N-D Images, ICCV01(I: 105-112).

8. Andrew, Y.N., Jordan, M. and Weiss, Y., On Spectral Clustering: Analysis and an Algorithm, NIPS(14), 2002, pp. 849–856.

9. Weiss, Y.[Yair], Segmentation using Eigenvectors: A Unifying View, ICCV99(975-982).

10. Felzenszwalb, P.F.[Pedro F.], Huttenlocher, D.P.[Daniel P.], Image Segmentation Using Local Variation, CVPR98(98-104).

30

References (2)11. Felzenszwalb, P.F.[Pedro F.], Huttenlocher, D.P.[Daniel P.], Efficient Graph-Based

Image Segmentation, IJCV(59), No. 2, September 2004, pp. 167-181.

12. Haxhimusa, Y.[Yll] and Kropatsch, W.G.[Walter G.], Segmentation Graph Hierarchies, SSPR&SPR(18-20) August 2004, pp. 343–351.

13. Haxhimusa, Y.[Yll], Ion, A.[Adrian], Kropatsch, W.G.[Walter G.], Illetschko, T.[Thomas], Evaluating Minimum Spanning Tree Based Segmentation Algorithms, CAIP05(579).

14. Falcão, A.X.[Alexandre X.], Stolfi, J.[Jorge], de Alencar Lotufo, R.[Roberto], The Image Foresting Transform: Theory, Algorithms, and Applications, PAMI(26), No. 1, January 2004, pp. 19-29.

15. Falcão, A.X.[Alexandre X.], Bergo, F.[Felipe] and Miranda, P.[Paulo], Image Segmentation by Tree Pruning, Proceedings of the Computer Graphics and Image Processing, XVII Brazilian Symposium, 2004, pp. 65–71.

16. Ding, J., Ma, R., Chen, S. and Wang, B., A Fast Directed Tree Based Neighborhood Clustering for Image Segmentation, NIP(4233), 2006, pp. 369–378.

17. Li, K.[Kang], Wu, X.D.[Xiao-Dong], Chen, D.Z.[Danny Z.], Sonka, M.[Milan], Optimal Surface Segmentation in Volumetric Images: A Graph-Theoretic Approach, PAMI(28), No. 1, January 2006, pp. 119-134.

18. Grady, L.[Leo], Random Walks for Image Segmentation, PAMI(28), No. 11, November 2006, pp. 1768-1783.

19. Pednekar, A.S.[Amol S.], Kakadiaris, I.A.[Ioannis A.], Image Segmentation Based on Fuzzy Connectedness Using Dynamic Weights, IP(15), No. 6, June 2006, pp. 1555-1562.

20. Luo, Q., Khoshgoftaar, T.M., Unsupervised Multiscale Color Image Segmentation Based on MDL Principle, IP(15), No. 9, August 2006, pp. 2755-2761.

31

References (3)21. Arbeláez, P.A.[Pablo A.], Cohen, L.D.[Laurent D.], A Metric Approach to Vector-

Valued Image Segmentation, IJCV(69), No. 1, August 2006, pp. 119-126.22. Cremers, D.[Daniel], Osher, S.J.[Stanley J.], Soatto, S.[Stefano], Kernel Density

Estimation and Intrinsic Alignment for Shape Priors in Level Set Segmentation, IJCV(69), No. 3, September 2006, pp. 335-351.

23. Bresson, X.[Xavier], Vandergheynst, P.[Pierre], Thiran, J.P.[Jean-Philippe], A Variational Model for Object Segmentation Using Boundary Information and Shape Prior Driven by the Mumford-Shah Functional, IJCV(68), No. 2, June, 2006, pp. 145-162.

24. Boykov, Y.Y.[Yuri Y.], Funka-Lea, G.[Gareth], Graph Cuts and Efficient N-D Image Segmentation, IJCV(70), No. 2, November 2006, pp. 109-131.

25. Bresson, X.[Xavier], Vandergheynst, P.[Pierre], Thiran, J.P.[Jean-Philippe], Multiscale Active Contours, IJCV(70), No. 3, December 2006, pp. 197-211

26. Tu, Z.W.[Zhuo-Wen], Zhu, S.C.[Song-Chun], Parsing Images into Regions, Curves, and Curve Groups, IJCV(69), No. 2, August 2006, pp. 223-249.

27. Seghers, D., Loeckx, D.[Dirk], Maes, F.[Frederik], Vandermeulen, D., Suetens, P.[Paul], Minimal Shape and Intensity Cost Path Segmentation, MedImg(26), No. 8, August 2007, pp. 1115-1129.

28. Papandreou, G., Maragos, P., Multigrid Geometric Active Contour Models, IP(16), No. 1, January 2007, pp. 229-240.

29. Tao, W., Jin, H., Zhang, Y., Color Image Segmentation Based on Mean Shift and Normalized Cuts, SMC-B(37), No. 5, October 2007, pp. 1382-1389.

30. Wu, J., Chung, A.C.S., A Segmentation Model Using Compound Markov Random Fields Based on a Boundary Model, IP(16), No. 1, January 2007, pp. 241-252.

32

References (4)31. Pavan, M.[Massimiliano], Pelillo, M.[Marcello], Dominant Sets and Pairwise Clustering,

PAMI(29), No. 1, January 2007, pp. 167-172.32. Tai, Y.W.[Yu-Wing], Jia, J.[Jiaya], Tang, C.K.[Chi-Keung], Soft Color Segmentation and Its

Applications, PAMI(29), No. 9, September 2007, pp. 1520-1537.33. Pyun, K., Lim, J., Won, C.S., Gray, R.M., Image Segmentation Using Hidden Markov

Gauss Mixture Models, IP(16), No. 7, July 2007, pp. 1902-1911.34. Arias, P., Pini, A., Sanguinetti, G., Sprechmann, P., Cancela, P., Fernandez, A., Gomez, A.,

Randall, G., Ultrasound Image Segmentation With Shape Priors: Application to Automatic Cattle Rib-Eye Area Estimation, IP(16), No. 6, June 2007, pp. 1637-1645.

35. Nikou, C.[Christophoros], Galatsanos, N.P.[Nikolaos P.], Likas, A.C.[Aristidis C.], A Class-Adaptive Spatially Variant Mixture Model for Image Segmentation, IP(16), No. 4, April 2007, pp. 1121-1130.

36. Benboudjema, D.[Dalila], Pieczynski, W.[Wojciech], Unsupervised Statistical Segmentation of Nonstationary Images Using Triplet Markov Fields, PAMI(29), No. 8, August 2007, pp. 1367-1378.

37. Michailovich, O.V.[Oleg V.], Rathi, Y.[Yogesh], Tannenbaum, A.[Allen], Image Segmentation Using Active Contours Driven by the Bhattacharyya Gradient Flow, IP(16), No. 11, November 2007, pp. 2787-2801.

38. Zoller, T.[Thomas], Buhmann, J.M.[Joachim M.], Robust Image Segmentation Using Resampling and Shape Constraints, PAMI(29), No. 7, July 2007, pp. 1147-1164.

39. Qiu, H.J.[Huai-Jun], Hancock, E.R.[Edwin R.], Clustering and Embedding Using Commute Times, PAMI(29), No. 11, November 2007, pp. 1873-1890.

40. Rivera, M., Ocegueda, O., Marroquin, J.L., Entropy-Controlled Quadratic Markov Measure Field Models for Efficient Image Segmentation, IP(16), No. 12, December 2007, pp. 3047-3057.

33

References (5)41. Chen, S.F.[Shi-Feng], Cao, L.L.[Liang-Liang], Liu, J.Z.[Jian-Zhuang], Tang, X.

[Xiaoou], Iterative MAP and ML Estimations for Image Segmentation, CVPR07(1-6).

42. Allili, M.S.[Mohand Said], Ziou, D.[Djemel], Object of Interest segmentation and Tracking by Using Feature Selection and Active Contours, CVPR07(1-8).

43. Liu, T.[Tie], Sun, J.[Jian], Zheng, N.N.[Nan-Ning], Tang, X.[Xiaoou], Shum, H.Y.[Heung-Yeung], Learning to Detect A Salient Object, CVPR07(1-8).

44. Cremers, D.[Daniel], Rousson, M.[Mikael], Deriche, R.[Rachid], A Review of Statistical Approaches to Level Set Segmentation: Integrating Color, Texture, Motion and Shape, IJCV(72), No. 2, April 2007, pp. 195-215.

45. Tai, X.C.[Xue-Cheng], Christiansen, O.[Oddvar], Lin, P.[Ping], SkjÆlaaen, I.[Inge], Image Segmentation Using Some Piecewise Constant Level Set Methods with MBO Type of Projection, IJCV(73), No. 1, June 2007, pp. 61-76.

46. Riklin-Raviv, T.[Tammy], Kiryati, N.[Nahum], Sochen, N.A.[Nir A.], Prior-based Segmentation and Shape Registration in the Presence of Perspective Distortion, IJCV(72), No. 3, May 2007, pp. 309-328.

47. Sofou, A., Maragos, P.[Petros], Generalized Flooding and Multicue PDE-Based Image Segmentation, IP(17), No. 3, March 2008, pp. 364-376.

48. Ding, J., Ma, R., Chen, S., A Scale-Based Connected Coherence Tree Algorithm for Image Segmentation, IP(17), No. 2, February 2008, pp. 204-216.

49. Du, X., Bui, T.D., A New Model for Image Segmentation, SPLetters(15), 2008, pp. 182-185.

50. Orbanz, P.[Peter], Buhmann, J.M.[Joachim M.], Nonparametric Bayesian Image Segmentation, IJCV(77), No. 1-3, May 2008, pp. 25-45.

Thank You!Questions?