8D040 Basis beeldverwerking Feature Extraction Anna Vilanova i Bartrolí Biomedical Image Analysis...

8D040 Basis beeldverwerking

Feature Extraction

Anna Vilanova i Bartrolí

Biomedical Image Analysis Groupbmia.bmt.tue.nl

N=M=30

What is an image?

Image is a 2D rectilinear array of pixels (picture element)

N=M=256

L=15(4 bits)L=255 (8 bits)

What is an image?No continuous values - Quantization

255

170

15

8

Binary Image L=1 (1 bit)L=3 (2 bits)

An image is just 2D?

No! – It can be in any dimension

Example 3D:

Voxel-Volume Element

Segmentation

Reduction of dimensionality

Why feature extraction ?

Pixel levelImage of 256x256 and

8 bits

256 65536 ~ 10 157826

possible images

• Incorporation of cues from human perception• Transcendence of the limits of human perception

• The need for invariance

Why feature extraction ?

Apple detection …

Transformation (Rotation)

cos( ) sin( ) 0sin( ) cos( ) 0

0 0 1

2a

1a

1b2b

P

( )T P

1 2( , ,1)P x x

1

1 2 2

cos( ) sin( ) 0( ) ( , ,1) sin( ) cos( ) 0

0 0 1 A

aT P x x a

O

1 1 2cos( ) sin( )b a a

2 1 2sin( ) cos( ) b a a

cos( )

sin( )

sin( )

cos( )

How do we transform an image?

• We transform a point P

• How do we transform an image f(P)?

1

1 2 2( ) ( , ,1)

A

aT P x x a

OT ( ) .T P P T

( ) ( )newf Q f Pnewff

QPHow do we know

which Q belongs to P?

1b

2

a

1

a

2

b

.Q P T

How do we transform an image?

• How do we transform an image f(P)?

1b

1. Q PT

We know T which is the transformation we want to achieve.

1( ) ( . )newf Q f Q T

( ) ( )newf Q f Pnewff

QP2

a

1

a

2

bHow do we know

which Q belongs to P?

Apple detection …

Feature Characteristics

• Invariance (e.g., Rotation, Translation)• Robust (minimum dependence on)• Noise, artifacts, intrinsic variations• User parameter settings

• Quantitative measures

We extract features from…

Region of InterestSegmented Objects

Classification

Features

Texture Based(Image & ROI)

Shape(Segmented objects)

Shape Based Features

• Object based• Topology based (Euler Number)• Effective Diameter

(similarity to a circle to a box)• Circularity• Compactness• Projections• Moments (derived by Hu 1962)

• …

4-neighbourhood of

8-neighbourhood of

Adjacency and Connectivity – 2D

pp

p

*( ) ( ) { }k kN p N p p

Notation: k-Neighbourhood of is p ( )kN p

[ , ]p i j

Adjacency and Connectivity – 3D

6-neighbourhood 18-neighbourhood 26-neighbourhood

Objects or Components (Jordan Theorem)

• In 2D – (8,4) or (4,8)-connectivity• In 3D – (6,26)-,(26,6)-,(18,6)- or

(6,18)-connectivity

Connected Components Labeling

Each object gets a different label

Connected Components Labeling

A

B

CRaster Scan

Note: We want to label A. Assuming objects are 4-connected B, C are already labeled.

Cortesy of S. Narasimhan

Connected Components Labeling

1

0

0 label(A) = new label

0

X

X label(A) = “background”

1

0

C label(A) = label(C)

1

B

0 label(A) = label(B)

1

B

C If

label(B) = label(C)then, label(A) = label(B)

Cortesy of S. Narasimhan

2

4 2 2 2 3

4 4 4 4 ?

2

2 2 2 2 3

2 2 2 2 2

2

2 2 2 2 3

2 2 2 2 2 2 2 3

2 2 2 2 2 2 2

2 2 2

What if label(B) not equal to label(C)?

Connected Components Labeling

1

B

C

Connected Components Labeling

Each object gets a different label

Classification

Features

Texture Based(Image & ROI)

Shape(Segmented objects)

Topology based – Euler Number

E C H

Euler Number E describes topology.

C is # connected components

H is # of holes.

Euler Number 3D

E C Cav G

Euler Number E describes topology. C is # connected components

Cav is # of cavities

G is # of genus

E=1+0-1=0 E=1+0-1=0 E=1+1-0=2

Euler Number 3D

E=2+0-0=2 E=1+1-0=2

Euler Number E describes topology. C is # connected components

Cav is # of cavities

G is # of genusE C Cav G

3D Euler Number

• The Euler Number in 3D can be computed with local operations• Counting number of vertices, edges and faces of the

surfaces of the objects

1

# # #

C Cav

i i ii

E vertices edges faces

Simple Shape Measurements

• 2D area - 3D volume • Summing elements

• 2D perimeter - 3D surface area• Selection of border elements • Sum of elemets with weights

• Error of precision

,

( , )x y

A f x y

1 22

where #ele. with 4c background ele.i

P N N

N i

Similarity to other Shape

• Effective Diameter

2

4 AC

P

2PComp

A

4

• Circularity (Circle C=1)

• Compactness – (Actually non-compactness)(Circle Comp= )

Ar

2A r

2P r

Moments

• Definition

• Order of a moment is• Moments identify an object uniquely

• ? is the Area• Centroid

pqr

, ,

[ , , ]p q rpqr

x y z

x y z f x y z p q r

000

100 010 001

000 000 000

( , , ) ( , , )

x y zc c c

[ ] : [1, ] {0,1}nf x N

Central Moments

• Moments invariant to position

• Invariant to scaling

, ,

( ) ( ) ( ) [ , , ]p q rpqr x y z

x y z

c x c y c z c f x y z

, ,

13

000

( ) ( ) ( ) [ , , ]p q rx y z

x y zpqr p q r

x c y c z c f x y z

Moments to Define Shape and Orientation

( , , )x y zc c c200

020

002

110

101

011

xx xy xz

yx yy yz

zx zy zz

xx

yy

zz

xy yx

xz zx

yz zy

I I I

I I I I

I I I

I

I

I

I I

I I

I I

Inertia Tensor

Eigenanalysis of a Matrix

• Given a matrix S , we solve the following equation

( ) =S I x 0 j j jSv = λ v

we find the eigenvectors and eigenvalues

• Eigenvectors and eigenvalues go in couples an usually are ordered as follows:

jjv

det( ) =S I 0

1 2 3

Eigenanalysis of the Inertia Matrix

Eigenanalysis

Sphere

Flatness

Elongated

( , , )x y zc c c

1 1v

2 2v

3 3v3

2

1

3

1

2

Eigenanalysis of the Inertia Matrix

Eigenanalysis

Sphere

Flatness

Elongated

( , , )x y zc c c

1 1v

2 2v

3 3v2

3

1

3

1

2

1 2 3

1 2 3

1 2 3

Orientation in 2D

• Using similar concepts than 3D

• Covariance or Inertia Matrix

• Eigenanalysis we obtain 2 eigenvalues and 2 eigenvectors of the ellipse

20 11

11 02IC

Moments Invariance

• Translation• Central moments are invariant

• Rotation• Eigenvalues of Inertia Matrix are invariant

• Scaling• If moment scaled by (3D) (2D) 1

3000

p q r

12

00

p q

Moments invariant rotation-translation-scaling

• For 3D three moments (Sadjadi 1980)

For 2D seven moments

1 200 020 002

2 2 22 200 020 200 002 020 002 101 110 011

23 200 020 002 002 110 110 101 011

2 2020 101 200 011

2

J

J

J

Classification

Features

Texture Based(Image & ROI)

Shape(Segmented objects)

Image Based Features

• Using all pixels individually• Histogram based features

− Statistical Moments (Mean, variance, smoothness)− Energy− Entropy− Max-Min of the histogram− Median

• Co-occurrance Matrix

Gonzalez & Woods – Digital Image ProcessingChapter 11 – 11.3.3 Texture

Histogram

0 1 2 3 4 5 6 7 8 9

Quantized , 0,...,

( ) # of voxels [ ]

total # of voxels

i

i i

b i L

N b f x b

N

L=9

( ) ( ) /i iP b N b N

bi

P(bi)

How do the histograms of this images look like?

Bimodal Histogram

60 80 100 120

0.1

0.2

0.3

0.4

0.5

Trimodal Features

50 100 150 200 250

0.005

0.01

0.015

0.02

0.025

0.03

0.035

Histogram Features

• Mean

• Central Moments0

( )

L

i ii

m b P b

0

( ) ( )

L

nn i i

i

c b m P b

Histogram Features

• Mean

• Variance

• Relative Smoothness

• Skewness

0

( )

L

i ii

m b P b

2 22

0

( ) ( )

L

i ii

b m P b c

2

11

1R

33

0

( ) ( )

L

i ii

c b m P b

Histogram Features

• Energy (Uniformity)

• Entropy

2

0

[ ( )]

L

ii

E P b

20

2

( ) log ( ( ))

if ( ) 0 then ( ) log ( ( )) 0

L

i ii

i i i

H P b P b

P b P b P b

Examples of Energy and Entropy

0

0,1

0,2

0,3

0 2 4 6 8 10 12 14 bi

P(bi)

0

0,02

0,04

0,06

0,08

0 2 4 6 8 10 12 14 bi

P(bi)

0

0,1

0,2

0,3

0,4

0 2 4 6 8 10 12 14 bi

P(bi)

0 2 4 6 8 10 12 14

00.10.20.30.40.50.60.70.80.9

11.1

bi

P(bi) Energy=1Entropy=0

Energy=0,111Entropy=3,327

Energy=0,255Entropy=2,018

Energy=0,0625Entropy= 4

Examples

Texture Mean std R 3rd moment Energy Entropy

1 82.64 11.79 0.002 -0.105 0.026 5.434

2 143.56 74.63 0.079 -0.151 0.005 7.783

3 99.72 33.73 0.017 0.750 0.013 6.374

1 2 3

The next slides were not given, during the lecture and will not be asked in the exam

Intensity Co-occurrance Matrix

• Operator Q defines the position between two pixels (e.g, pixel to the right)

• Co-occurance matrix G is (L+1) x (L+1) (6x6). Counts how often Q occurs

0 1 4

0 4 5

2 3 1

2 3 4

4

4

1

1

6

3

5

1

1 6 55 1

0 1 2 3 4 5 6

0 0 1 0 0 1 0 0

1 0 0 2 0 1 0 1

2 0 0 0 2 0 0 0

3 0 1 0 0 1 0 0

4 2 1 0 0 0 1 1

5 0 2 0 1 0 0 0

6 0 0 0 0 0 1 0

Image G

Example

• L=256• Q “one pixel immediately to the right”

Image

G - Matrix

Features based on the co-ocurrence Matrix

• The elements of G (gij) is converted to probability (pij) by dividing by the amount of pairs in G

• Based on the probability density function we can use• Maximum• Energy (uniformity)• Entropy

0 0

1

L L

iji j

p

Features based on the co-ocurrence Matrix

• Homogenity – closeness to a diagonal matrix

• Contrast

0 0 1 | | L L

ij

i j

p

i j

2

0 0

( )

L L

iji j

i j p

Features based on the co-ocurrence Matrix

• Correlation – measure of correlation with neighbours

0 0

( )( )L Lr c ij

i j r c

i m j m p

2 2

0 0 0

( ) ( ) ( ) ( )

L L L

i ij r i r r ii i i

P i p m iP i i m P i

2 2

0 0 0

( ) ( ) ( ) ( )L L L

j ij c j c c ij j i

P j p m jP j j m P i

Example

• L=256• Q “one pixel immediately to the right”

Image

G - Matrix

Example

Image

G - Matrix

Correlation Contrast Homogeneity

1 - 0.0005 10838 0.0366

2 0.9650 570 0.0824

3 0.8798 1356 0.2048

Moments

• Definition

• Order of a moment is• Moments identify an object uniquely

• Centroid

, ,

[ , , ]p q rpqr

x y z

x y z f x y z pqr p q r

100 010 001

000 000 000

( , , ) ( , , )x y zc c c

[ ] :[1, ] [1, ]nf x N L

Central Moments

• Moments invariant to position

• Normalized central moments, ,

( ) ( ) ( ) ( , , )p q rpqr x y z

x y z

c x c y c z c f x y z

, ,

13

000

( ) ( ) ( ) ( , , )p q rx y z

x y zpqr p q r

x c y c z c f x y z

Moments invariant rotation-translation-scaling• For 3D three moments (Sadjadi 1980)

• For 2D seven moments (Hu’s 1962)

1 200 020 002

2 2 22 200 020 200 002 020 002 101 110 011

23 200 020 002 002 110 110 101 011

2 2020 101 200 011

2

J

J

J

Moments invariant rotation-translation-scaling-mirroring (within minus sign)

1 2 3 4

5 6

, , , ,

,

are all

equal Mirroring

7 7

7 7

or

Projections

1

( ) ( , )

yN

xy

proj x f x y

1

( ) ( , )xN

yx

proj y f x y

x

y