Upload
others
View
15
Download
0
Embed Size (px)
Citation preview
International Journal of Computer Applications (0975 – 8887)
Volume 118– No.12, May 2015
27
Saliency Detection with VORONOI Diagram
Dao Nam Anh Department of Information Technology
Electric Power University 235 Hoang Quoc Viet road
Hanoi, Vietnam
Nguyen Huu Quynh Department of Information Technology
Electric Power University 235 Hoang Quoc Viet road
Hanoi, Vietnam
a. Input image b. Voronoi diargram c. Salient region
Figure 1: Example of saliency detection with the Voronoi diagram
ABSTRACT
Many applications are serviced by the Voronoi tessellation
required to split image into Voronoi regions. An automatic
method to learn and detect salient region for color image with
support of the Voronoi diagram is presented. Salient regions
are modeled as flexible circumstance corresponding to centers
of mass. The centers are predicted by local contrast-based
representation with local maxima. Results are demonstrated
that are very competitive with other recent saliency map
detection schemes and show robustness to capture visual
attention objects. Our major contributions are the local
maxima based method for allocation of Voronoi centroids and
the Gaussian-based filter for estimating attention degrees. To
show the effectiveness of the approach, saliency maps are
detected for images of MSRA saliency object database by
some state-of-the-art methods. The strengths and the
weaknesses of the approach are considered, with a special
focus on the context based salient regions a challenging task
which can be found in wide range of applications addressed in
computer vision.
General Terms
Pattern Recognition, Algorithms.
Keywords
Voronoi tessellation, salient region.
1. INTRODUCTION Computational geometry has attracted research interest in the
past decade not least because of important application in, but
not limited to, astronomy, biology, cartography, chemistry,
forestry, linguistics, marketing, meteorology, physics,
physiology and statistics. Voronoi diagrams [1][2] being a
geometric construct of a discrete set of points and line
segments have interesting and surprising properties in
computational geometry and geometric modeling [3] and
particularly in point pattern analysis [4].
The spectrum of the Voronoi diagram research for the image
processing covers many areas including shape representation,
fundamental segmentation, texture analysis, vector fields and
special k-clustering. Features of Voronoi edges, vertices and
circles generated by k-points can be used for optimum k-
clustering [5] that provides solutions for color quantization
and image compression [6]. A metric based on Voronoi
regions of each point is used to define distance from a pixel to
Voronoi points for image segmentation [7]. A particular
Voronoi tessellation is addressed in [8], where the generators
of the Voronoi regions in the tessellation are also the centers
of mass. Thus, spatially distributed vector fields are clustered
by a the Voronoi regions-based similarity measure function in
both the spatial and vector spaces.
Voronoi tessellation is the base for a texture segmentation
algorithm [9], where feature vector is calculated for each
Voronoi polygon and then used to identify the interior and the
border regions of textures. Voronoi skeletons in form of
diagram of a polygon are constructed in [10] to derive
accurate and robust skeletons for planar shapes. The skeletons
possess the properties of connectivity as well as Euclidean
metrics in the method.
Besides, visual saliency [11] is of high interest in many
computer vision applications, ranging from image
segmentation [12], content-aware compression [13], object
detection [14] in web images [15] and focus of attention in
human robot interaction [16].
One of our motivations for this research is to construct
saliency map (example in Fig.1c) with advantages of Voronoi
diagram (Fig.1b) in presenting image features such as contrast
by computational geometry. This paper introduces a special
method of creating Voronoi tessellation for saliency detection
purpose. The Voronoi points are defined from analysis of
local contrast. A local maxima filter is described for building
Voronoi region.
To the best of our knowledge, an algorithm for detection of
visual attention region is demonstrated with an standard
benchmark designed for salient objects. As shown in an
implementation, comparing with a range of different methods,
this algorithm achieves respectable performance on the
International Journal of Computer Applications (0975 – 8887)
Volume 118– No.12, May 2015
28
standard benchmark, and exhibits graceful ability to select
detect salient regions.
2. OUTLINE OF PAPER Section 3 reviews the Voronoi diagram and saliency detection
research that relates to our method. Section 4 describes the
proposed approach. Experiment results are reported in Section
5, followed by discussion, directions for future work and
conclusion in Section 6, 7, 8.
3. PRIOR WORK This section first reviews methods aiming for image
segmentation using Voronoi tessellation. Then, methods that
apply the Voronoi diagram to detect salient region/object are
reported.
3.1 VORONOI SEGMENTATION Segmentation of human faces is addressed by Voronoi
Diagram approach [17] that applies a distance transformation
to segment face features. The approach employs a greedy
search algorithm looking for a particular face candidate by
focusing its action in elliptical-like regions. A segmentation
method by region growing is presented in [18], where seed
points are derived from the color edge image as the peaks in
the associated Voronoi diagram. Regions are grown using
gates on pixel color relative to region central color and region
edge pixel color.
Voronoi graph is used for autonomous place detection that is
sensor and domain independent [19]. The skeleton of the free
space in the local surround is found by a real-time calculated
Voronoi graph. An algorithm based on the edge-weighted
centroid Voronoi tessellation which uses propagation of the
inter-slice consistency constraint is presented in [20]. It can
segment a 3D superalloy image, slice by slice, to obtain the
underlying grain microstructures.
An interactive image segmentation method based on structural
pattern recognition is introduced in [21]. A model graph is
constructed from an over-segmentation of an image and from
traces given by user. The over-segmentation is controlled by
markers provided by centroidal Voronoi diagrams.
3.2 VORONOI SALIENCY Generalized Voronoi tessellation is outlined in low-level
segmentation for vector valued images [22] where
segmentation procedure uses an accordant pseudo-metric from
the image data and the choice of a set of sources. The method
predesigns saliency image which is related to the contrast
ultra-metric and then estimation of the edges is based on the
metric.
An iterative segmentation voting method, which uses oriented
kernels for deriving visual attention as it relates to symmetry
is outlined in [23]. Voronoi tessellation, voting, and level set
methods are joined to outline blob-like structures in the
method, implemented for biological data.
Perceptual grouping and domain meshing are addressed in
[24]. The article shows a solution for the problems by
Voronoi diagrams and their dual Delaunay complexes,
defined with geodesic distances over 2D Riemannian
manifolds.
The Voronoi diagram, traditionally used for point patterns, is
applied to define region neighbors in [25] for an object
description. The generalized Voronoi diagram partitions an
image into polygons, each covering a region, representing its
area of influence.
A segmentation method for multi-regions is carried out in two
phases using the Voronoi supports at vertices and the implicit
integration is demonstrated in [26]. The volume belonging to
each patch segment can be achieved with the support of
Voronoi diagram computed in the voxel space.
4. VORONOI SALIENCY DETECTOR Denote color image u(x) on space : 3:)( xu (1)
Saliency map s(x) encodes local visual attention:
1]1,0[:)( us (2)
Voronoi diagram V divides space into a number of cells C
for a set of seed generators G. },{ CGV (3)
The generators G are predefined by function g:
},}{ 1 GzzG i
k
ii
Gzfalsezg
Gztruezgzg
\,)(
,)()( (4)
There is one Voronoi cell for each seed generator:
iii CzcellCGzcell )(,:)( (5)
iii zCgeneratorGCCgenerator )(,:)( (6)
A Voronoi cell or tessellation consists of all pixels closer to
that seed than any other:
i
k
iji
i
k
ii
CjiCC
kiCCC
1
1
,,
},..1{,,}{
(7)
},,..1,,,{ ijkjGzzzzzCzC jjiii (8)
Creating Voronoi cells from Voronoi generators is noted by
CGGVoronoi :)( (9)
4.1 LOCAL MAXIMA FILTER To detect Voronoi seed generators a Gaussian-based local
maxima filter is described as follows:
)y(
2exp
2
1max)x(
2
2
)( dxy
agrdf GxNy
(10)
The filter provides searching for a neighbor pixel )(xNy
that gets local relative maximum: y near x and the values d(z)
of d for all neighbor of y are all less than d(y). Then assign
value d at y to d at x.
4.2 LOCAL VARIATION Local variation of a pixel x with its neighbors )(xNy is
defined as follows:
)(
2
2
2
1 )()(2
exp2
1)(
xNyW yuxuxy
xv
(11)
Local variation mean of a region is calculated from (11):
)())((1 )()(
xNyxNnumel yvxm (12)
Thus, divergence at a pixel x is described by difference of the
variation v(x) over the mean variation m(x):
2)()()( xmxvxd (13)
Set of local maxima M on d(x) can be found from (13):
}},{ jj xxM (14)
International Journal of Computer Applications (0975 – 8887)
Volume 118– No.12, May 2015
29
where d(x) achieves local maxima at xi over all its neighbors
)(xNy
Mxydxd ixNyi i ),(max)( )( (15)
The Voronoi generators G (4) is found by the set of local
maxima M: G:=M
(16)
4.3 SALIENT CELLS To find salient regions the Voronoi cells with their divergence
d(x) are implemented. Salient value for the pixel x that is a
generator (x=zi) is the same its divergence:
)(:)( xdxs (17)
For other pixel x, find corresponding generator zi by the local
maxima filter f(x)(10):
)(xfzi (18)
then assign salient value d(zi) for s(x):
)())(()( izdxfdxs (19)
Salient map s(x) is continuous, with the Otsu threshold:
)(:)( ohistogramso (20)
salient region can be detected:
sb
s xsx
\
})(:{
(21)
4.4 VORONOI SALIENCY ALGORITHM We now elaborate above local maxima filter and Voronoi
diagram into the Voronoi Saliency Algorithm (VSA).
Start: given an input image u(x),
1.Local variation calculation: define local variation v(x)
and divergence d(x) by (11)(13).
2.Local Maxima detection: use (14)(15).
3.Voronoi diagram creation: apply (16)(18).
4.Saliency map: get s(x) by saliency filter (17)(19).
5.Thresholding and export output by (20)(21).
Figure 2 illustrates the VSA algorithm working with an image
(Fig.2a) from MSRA Salient Object Database [27]. Step 1
produces local variation v(x) showed in Fig.2a and the
divergence d(x) - Fig.1c. Step 3 creates Voronoi generators G
(Fig.1d) and cells C (Fig.1e, Fig.1f). Step 4 gives saliency
map s(x) presented in Fig.1g, followed by step 5, that exports
final extracted salient region (Fig.1f).
SALIENCY DETECTION WITH VORONOI DIAGRAM
1. Calculate local variation
2. Detect local maxima
Input image u(x)Local variation v(x) and divergence d(x)
Location with local maxima M
4. Fill variation in Voronoi cells
Saliency map s(x)
3. Create Voronoi diagram
Voronoi cells C
Figure 2: Algorithm of Saliency Detection with Voronoi Diagram
a) Input u(x): 28_28988.jpg
[27]
b) Local variation v(x) by (11) c) Divergence d(x) by (13) d) Local maxima M, generator G
International Journal of Computer Applications (0975 – 8887)
Volume 118– No.12, May 2015
30
e) Voronoi cells C f) Voronoi cells C, zoom in g) Saliency map s(x) by (19) f) Extracted salient region s
(21)
Figure 3: Example of saliency detection with the Voronoi diagram
a) 27_27017.jpg b) Extracted object c) Voronoi diagram d) Saliency map
e) 28_28907.jpg f) Extracted object g) Voronoi diagram h) Saliency map
i) 28_28003..jpg j) Extracted object k) Voronoi diagram l) Saliency map
m) 27_27186.jpg n) Extracted object o) Voronoi diagram p) Saliency map
q) 27_27330.jpg r) Extracted object s) Voronoi diagram t) Saliency map
International Journal of Computer Applications (0975 – 8887)
Volume 118– No.12, May 2015
31
u) 28_28209.jpg v) Extracted object w) Voronoi diagram x) Saliency map
y) 28_28717.jpg z) Extracted object aa) Voronoi diagram bb) Saliency map
Figure 4: Example of saliency detection with the Voronoi diagram for color images from [27]
5. EXPERIMENTS Figure 3 shows experimental results of the saliency detection
method with Voronoi diagram. Input images given by [27] are
displayed in the first column. They belong to different image
categories like nature, day and night. The Voronoi diagram
which is calculated in VSA’s the third step with generator and
cells are presented in the third column. Saliency map
produced by the fourth step for each test image is outlined in
the fourth column.
Finally, salient region extracted by VSA’s the fourth step is
demonstrated in the second column next to its input. Note that
the Voronoi generators zi are local maxima of divergence d(x)
and the saliency value s(x) for each Voronoi cell Ci is the
divergence value at the generator zi of the cell.
Results of initial experiments on the MSRA Salient Object
Database [27] are associated with the some state-of-the-art
methods [28][29][30].
Rarity-based saliency detection [28] is sequential bottom-up
features extraction. The features of the model are both low-
level (luminance and chrominance) and medium-level
(orientations).
Saliency Detection by Self-Resemblance [29] is a bottom-up
saliency detection algorithm. This employs local steering
kernels and applies a nonparametric kernel density estimation
based on matrix cosine similarity.
Saliency For Image Manipulation [30] approach is based on
four principles: pixel distinctness, pixel reciprocity, object
association and multi-layer saliency. Basic saliency map is
constructed from a distinctness map, based on the first and
second principles, and a prominent object probability map,
based on the third principle.
The VSA and these three methods are tested on a set of
images of the salient object benchmark [27]. The database
provides three user inputs for each test image. The inputs are
in a form of rectangular covering region of visual attention.
This format is not right for all the cases, particularly if a
salient object is not fit well in a rectangular. Though, test with
the database could give a relative performance comparison.
Mean squared error (MSE) and the sum of absolute
differences (SAD) are selected metrics for this test. MSE
measures the average of the squares of errors which is the
difference between the estimator and the object for
measurement. SAD weights the similarity between image
blocks by taking the absolute difference between each pixel in
the original block and the corresponding pixel in the block
being used for comparison.
Examples of the results are displayed in Fig.7. Note that
saliency map have a continuous range [0,1].
Depending on rounding and thresholding methods, extraction
of salient regions could be variable for the same input images.
Though, the statistics from the experiment in Fig.6
demonstrate that the algorithm for saliency detection with
Voronoi achieves competitive performance on this standard
dataset.
6. DISCUSSION The Voronoi diagram is suitable method to present salient
degrees. In our algorithm the degree is based on contrast
estimation expressed by variation and divergence for a local
frame around each pixel. The size of frame and the Gaussian
deviation are the main parameters of the algorithm. They can
be chosen by statistics on performance for selected
combinations of the parameters.
7. FUTURE WORK The principle of Voronoi diagram is simple and effective.
This article shows that the problem of saliency detection can
be solved by the Voronoi diagram. Though our method is just
one of several possible solutions. Further research on visual
attention and Voronoi cells could open new results.
8. CONCLUSION A method for detection saliency by Voronoi diagram is
proposed by this paper. The method estimates divergence and
local maxima on the divergence to build up the Voronoi
generators. Tessellations are generated from the generators by
a local maxima filter. The tessellations are the base for salient
map.
The method is effective and simple to manage. It is tested at a
standard saliency object database with a competitive
performance on the MSE and SAD metrics.
9. ACKNOWLEDGMENTS The authors would like to thank MSRA [27] for the database
of salient objects.
International Journal of Computer Applications (0975 – 8887)
Volume 118– No.12, May 2015
32
Special thanks authors of Rarity-based saliency detection [28],
Saliency Detection by Self-Resemblance [29] and Saliency
For Image Manipulation [30] for providing source codes that
are used in our experimental implementation.
The authors are grateful to anonymous internal reviewers for
their constructive comments on this manuscript.
Figure 5: Statistics on experiment for saliency detection by methods [28][29][30] and VSA
a.
Inpu
t im
age
[2
7]
29_29329.jpg
29_29490.jpg
29_29729.jpg
29_29788.jpg
b.
Sal
ien
cy b
y u
ser
Iipu
t
User input
User input
User input
User input
c.
Met
hod
[28
]
MSE=0.10, SAD=0.20
MSE=0.09, SAD=0.21
MSE=0.21, SAD=0.32
MSE=0.17, SAD=0.29
d.
Met
hod
[29
]
MSE=0.32, SAD=0.47
MSE=0.16, SAD=0.28
MSE=0.13, SAD=0.20
MSE=0.23, SAD=0.36
Method MSE SAD
A. Rarity-based saliency detection [28]
0.15 0.24
B. Saliency Detection by Self-Resemblance [29]
0.17 0.25
C. Saliency For Image Manipulation [30]
0.15 0.23
D. Saliency detection with Voronoi diagram (VSA)
0.14 0.23
MSE 0.00
0.10
0.20
0.30
A B C D
MSE
SAD
International Journal of Computer Applications (0975 – 8887)
Volume 118– No.12, May 2015
33
e.
Met
hod
[30
]
MSE=0.05, SAD=0.10
MSE=0.06, SAD=0.13
MSE=0.15, SAD=0.24
MSE=0.12, SAD=0.22
f.
VS
A m
ap
MSE=0.09, SAD=0.21
MSE=0.10, SAD=0.18
MSE=0.17, SAD=0.31
MSE=0.15, SAD=0.31
g.
VS
A o
bje
ct
Figure 6: Example of saliency detection by methods [28][29][30] and VSA
10. REFERENCES [1] G. Voronoi. Nouvelles applications des paramètres
continus à la théorie des formes quadratiques. Journal für
die Reine und Angewandte Mathematik,
133:97178,1907.
[2] Chew, L. P. 1986. Building Voronoi diagrams for convex
polygons in linear expected time. Technical Report PCS-
TR90-147, Dept. Math. Comput. Sci., Dartmouth
College, Hanover.
[3] Franz Aurenhammer, Voronoi diagrams - a survey of a
fundamental geometric data structure, ACM computing
surveys, vol.23, no.3, 1991, pages 345-405.
[4] Atsuyuki Okabe, Barry Boots, Kokichi Sugihara & Sung
Nok Chiu (2000). Spatial tessellations – concepts and
applications of Voronoi diagrams. 2nd edition. John
Wiley, 2000, 671 pages.
[5] S. Hasegawa, H. Imai, M. Inaba, N. Katoh, and J.
Nakano, Efficient algorithms for variance-based k-
clustering, in Proceedings of the First Pacific Conference
on Computer Graphics and Applications, World
Scientific, River Edge, NJ, 1993, pp. 75–89.
[6] M. Inaba, H. Imai, and N. Katoh, Experimental results of
randomized clustering algorithms, in Proceedings of the
12th ACM Symposium on Computational Geometry:
Communication Section, 1996.
[7] Thouis R. Jones, A Carpenter, and P Golland. 2005.
Voronoi-Based segmentation of cells on image
manifolds. In Proc of the First inter. Conf. on Com.
Vision for Biomed Image App. (CVBIA'05), pp 535-543.
[8] Q. Du and X. Wang, Centroidal Voronoi tessellation
based algorithms for vector fields visualization and
segmentation, Proc. IEEE Visualization, pp. 43-50, 2004.
[9] ]M. Tüceryan and Anil K. Jain. 1990. Texture
segmentation using Voronoi polygons. IEEE Trans.
Pattern Anal. Mach. Intell. 12, 2 (1990), 211-216.
[10] Niranjan Mayya and V. T. Rajan. 1996. Voronoi
diagrams of polygons: a framework for shape
representation. J. Math. Im. Vis. 6, 4 (1996), 355-378.
[11] L. Itti, C. Koch, and E. Niebur. A model of saliency-
based visual attention for rapid scene analysis. IEEE
Trans. On PAMI, 20(11), 1998.
[12] J. Han, K. Ngan, M. Li, and H. Zhang. Unsupervised
extraction of visual attention objects in color images.
IEEE Transactions on Circuits and Systems for Video
Technology, 16(1):141.145, 2006.
[13] Zund, F. , Pritch, Y. , Sorkine-Hornung, A. , Mangold,
S., Content-aware compression using saliency-driven
image retargeting, Image Processing (ICIP), 2013.
[14] A. Shokoufandeha, I. Marsicb, Sven J. Dickinsona,
View-based object recognition using saliency maps,
Image and Vision Computing 17 (1999) 445–460.
[15] P. Wang, J. Wang, G. Zeng, J. Feng, Salient object
detection for searched web images via global saliency,
(CVPR), 2012.
[16] G. Schillaci, S. Bodiroža, V. Hafner, Evaluating the
effect of saliency detection and attention manipulation in
human-robot interaction, Int. J. of Social Robotics, 2013,
Vol. 5, Is. 1, pp 139-152.
[17] A. Cheddad, D. Mohamad and A. Abd Manaf, Exploiting
Voronoi diagram properties in face segmentation and
features extraction, Pattern Recognition, 41 (12) (2008)
3842-3859, Elsevier Science.
[18] Sinclair, D.: Voronoi seeded colour image segmentation.
Technical Report 3, AT&T Laboratories Cambridge
(1999).
International Journal of Computer Applications (0975 – 8887)
Volume 118– No.12, May 2015
34
[19] P. Beeson, N. K. Jong, and B. Kuipers, Towards
autonomous topological place detection using the
extended Voronoi graph, IEEE Inter. Conference on
Robotics and Automaton (ICRA), 2005.
[20] Y. Zhou, L. Ju, Y. Cao, Jarrell W. Waggoner, Y. Lin, J.
Simmons, S. Wang. Edge-weighted centroid Voronoi
tessellation with propagation of consistency constraint
for 3d grain segmentation in microscopic superalloy
images. CVPR Workshops 2014: 258-265.
[21] Narendra Ahuja, Sinisa Todorovic, Connected
segmentation tree –a joint representation of region layout
and hierarchy, Proc. IEEE Conf. Computer Vision and
Pattern Recognition (CVPR), 2008.
[22] Arbelaez, P. A., & Cohen, L.D. (2003). Generalized
Voronoi tessellations for vector-valued image
segmentation. Proc. 2nd IEEE Workshop on Variational,
Geometric and Level Set Methods in Computer Vision
(VLSM'03) (pp.49-56).
[23] H. Chang, Q.Yang, B. Parvin, Segmentation of
heterogeneous blob objects through voting and level set
formulation, Pattern Recognit Lett. 2007, 28(13).
[24] Sébastien Bougleux, Gabriel Peyré, Laurent D. Cohen.
Anisotropic geodesics for perceptual. grouping and
domain meshing. ECCV 2008.
[25] Matthew Cook, Probabilistic reasoning and decision
making in sensory-motor systems. The Neuromorphic
Engineer (2009).
[26] Ran Fan, Xiaogang Jin, and Charlie C.L. Wang, Multi-
region segmentation based on compact shape prior. IEEE
Trans. on Automation Science and Engineering, 2014.
[27] Tie Liu, Jian Sun, Nan-Ning Zheng, Xiaoou Tang,
Heung-Yeung Shum. Learning to detect a salient object.
In Proc. IEEE Cont. on Computer Vision and pattern
Recognition (CVPR), 2007.
[28] N. Riche, M. Mancas, M. Duvinage, M. Mibulumukini,
B. Gosselin, T. Dutoit, RARE2012: A multi-scale rarity-
based saliency detection with its comparative statistical
analysis. Sig. Proc.: Image Comm. 28(6): 642-658
(2013).
[29] Hae Jong Seo, and Peyman Milanfar, Static and space-
time visual saliency detection by self-resemblance, J.
Vision 9(12):15,1-27.
[30] R. Margolin, L. Zelnik-Manor, A. Tal. Saliency for
image manipulation, The Visual Computer, 2012.
IJCATM : www.ijcaonline.org