Extraction of Salient Extraction of Salient Contours in Color ImagesContours in Color Images

Extraction of Salient Extraction of Salient Contours in Color ImagesContours in Color Images

Vonikakis Vasilios, Ioannis Andreadis and Vonikakis Vasilios, Ioannis Andreadis and Antonios Gasteratos Antonios Gasteratos

Democritus University of Thrace Democritus University of Thrace 2006 2006

Democritus University of Thrace Democritus University of Thrace 2006 2006

Presentation OverviewPresentation Overview

Problem definition

Biological background

Description of the model

Results

Conclusions

Definitions: Salient ContoursDefinitions: Salient Contours

Salient contours:

The most evident contours that draw the attention of an observer Problem

definition

Applications of salient contoursApplications of salient contours

Create the ‘primal sketch’ of the image

Filter the optical data and keep only the significant information

Reduce the amount of visual information that a visual system processes

Problem definition

The Human Visual SystemThe Human Visual System

Biological background

RetinaRetina Visual CortexV1, V2…

Visual CortexV1, V2…

Optic nerveOptic nerve

light(ganglion cells)

Double opponent cellsDouble opponent cells

Biological background

They are located in area V1

Two chromatic and one achromatic

They have a center-surround receptive field

They receive opposite signal to center and surround

They respond only to changes between center and surround – edges detectors

R+G-B

B-R-G

G-R

R-G

-R-G-B

R+G+B

Blue-Yellow Red-Green Black-White

The primary visual cortex V1The primary visual cortex V1

Biological background

The visual cortex analyses the retinal output in 3 different maps:

1. Color (double-opponent cells) 2. motion-depth3. orientation of edges

At every position of the visual field, the V1 has cells (orientation filters) of all possible orientations

Favorite connections of a

horizontal orientation cell Biological background

Orientation cells prefer to be connected with others that create co-circular paths

This favors the smooth continuity of contours

Connection of orientation cellsConnection of orientation cells

Block diagram of the modelBlock diagram of the model

Description of the model

InputImage

InputImage

Extraction of color edges

SalientContoursnetwork

Extracting color edgesExtracting color edges

Description of the model

Center Surround

9x9 mask

Similar to the double-opponent cells of V1

Extracting color edgesExtracting color edges

Description of the model

max { RG, BY, BW }

RG BY BW

Orientation filtersOrientation filters

Description of the model

• 60 kernels

• 10×10 pixel size

• 12 orientations

• all possible positions within every orientation

The image is divided to 10×10 non-overlapping regions

For every region all 60 kernels are convolved

The higher response defines the kernel that best describes the orientation of the region

Objective: to encode the orientation of the edgesObjective: to encode the orientation of the edges

Encoded edgesEncoded edges

Description of the model

Color edge image

Image with oriented filters

Kernel 24:75° Kernel 19:135°

Computing the connection matrixComputing the connection matrix

Description of the model

We have calculated the connection matrix of all the 60 kernels, in a 5×5 kernel neighborhood

Kernel 6 Kernel 17 Kernel 54Connection matrix: weight [60] [5] [5] [60]Connection matrix: weight [60] [5] [5] [60]

weight [m] [ i ] [ j ] [n] : connection from kernel ‘m’ in the weight [m] [ i ] [ j ] [n] : connection from kernel ‘m’ in the center of the center of the 5×5 region, to kernel ‘n’ in the (i,j) position to kernel ‘n’ in the (i,j) position

Connection matrix: weight [60] [5] [5] [60]Connection matrix: weight [60] [5] [5] [60]

weight [m] [ i ] [ j ] [n] : connection from kernel ‘m’ in the weight [m] [ i ] [ j ] [n] : connection from kernel ‘m’ in the center of the center of the 5×5 region, to kernel ‘n’ in the (i,j) position to kernel ‘n’ in the (i,j) position

i

j

m

n

n

nn

n

nnn

nn

n

Influence between kernelsInfluence between kernels

Description of the model

Basic influence equation of kernel m (i,j) to kernel n

outoutnn(t) = out(t) = outnn(t-1) + weight(t-1) + weightm(i,j)→nm(i,j)→n× out× outmm(t)(t)

If weightm(i,j)→n>0 (kernel m is in the favorite curves of n) the influence is excitatory (outn(t)>outn(t-1))

If weightm(i,j)→n<0 (kernel m is not in the favorite curves of n) the influence is inhibitory (outn(t)<outn(t-1))

Activation function of kernel kActivation function of kernel k

Description of the model

Only the kernels with equal excitation in both lobes achieve high output

This favors the good continuation of salient contours

FLobe 1

Lobe 2 L1: total excitatory influence to Lobe 1

L2: total excitatory influence to Lobe 2

inh: total inhibitory influencek

IterationsIterations

Description of the model

Oriented filters

t=0 t=1 t=9 t=19

Salient contour kernels gradually increase their values

Kernels of non-salient contours gradually decrease their values

Usually 10 iterations are necessary

Results Results

Results

Original image Color edges Salient contours

700×700: 2.7 sec

700×576: 2.3 sec

More results More results

Results

Original image Color edges Salient contours

1000×768: 4.8 sec

1000×768: 4.8 sec

More results More results

Results

Original image Color edges Salient contours

500×750: 2.1 sec

672×496: 1.8 sec

ConclusionsConclusions

The proposed extraction of edges exhibits better results, especially for isoluminant areas, than the gradient of R,G and B

The proposed kernel set is an adequate way of coding the orientation of edges

The proposed method successfully extracts some of the most salient contours of the image

The execution time of the method when executed by a conventional PC is small compared to other saliency algorithms in the field

Thank you! Thank you!