Histogram equalization, geometric transformations/file/E0005E-lecture2... · 2012-09-10 ·...

2012-09-10 E0005E, Lecture 2 1

INSTITUTIONEN FÖR SYSTEMTEKNIK

LULEÅ TEKNISKA UNIVERSITET

2012-09-10 E0005E, Lecture 2 2

Histogram equalization, geometric transformations and neighborhood operations

Matthew Thurley

E0005E Industrial Image Analysis

2012-09-10 E0005E, Lecture 2 3

Course Details

• Course web page - http://www.ltu.se/edu/course/E00/E0005E/E0005E-Industriell-bildanalys-1.36192?l=en

- Course guide – what you need to know

- Lecture notes

- Instructions for each development phase

- Other information

• Course textbook - Digital Image Processing, Rafael Gonzalez and Richard woods.

Pearson Education, 3rd edition

• Fronter for submission of reports

Last Lecture

• Digital Images

• Image Sampling & Quantisation

- Spatial Resolution

- Intensity Resolution

• Histograms

• Point Operations

- Lookup tables

- Contrast stretching

- Thresholding

2012-09-10 E0005E, Lecture 2 4

This Lecture

• Histogram Equalisation (you used this in Lab 1)

• Geometric Transformations

• Point Operations

• Neighbourhood Operations

• Noise Filtering

- Linear filtering

- Median Filtering

- Edge Preserving Filtering

2012-09-10 E0005E, Lecture 2 5

2012-09-10 6

Histogram Equalization

Speaking Notes: Histograms are the basis for many spatial domain processing algorithms,

image enhancement, thresholding, segmentation. They are simple to calculate and good for

real-time processing

Figures from DIP 3.3 Histogram Processing

2012-09-10 E0005E, Lecture 2 7

Histogram Equalization

2012-09-10 E0005E, Lecture 2 8

Histogram Equalization

Arbitrary histogram of an image

expressed as a probability function

If we adjusted the image intensities so that all

intensities occured at equal probabilties

(uniform distribution) then structure in the

image should be significantly more visible.

Quiz

Give a single intensity transformation function for spreading the intensities of an image so the lowest intensity is 0 and the highest L-1

What effect would setting to zero the lower-order bit planes have on the histogram of the image in general?

What effect on the histogram would setting the high-order bit planes to zero have instead?

2012-09-10 E0005E, Lecture 2 9

2012-09-10 E0005E, Lecture 2 10

Histogram Equalization Howto

• Calculate Intensity Histogram

• Calculate the Probability Function

• Calculate the cumulative sum of the probability function

• Scale the cumulative probability function by (L-1) the number of intensities

• The scaled cumulative probability function is the lookup table to achieve histogram equalisation

Intensity Frequency Probability

2012-09-10 E0005E, Lecture 2 11

Histogram Equalization Example

This is the scaled cumulative probability function, scaled from range [0..1] to [0..7], the range of intensities.

2012-09-10 E0005E, Lecture 2 12

Histogram Equalization

Many not be invertible as the left graph shows

GEOMETRIC TRANSFORMATIONS

E0005E Industrial Image Analysis

Matthew Thurley

2012-09-10 E0005E, Lecture 2 13

2012-09-10 E0005E, Lecture 2 14

Geometric Transformations

• Geometric transformations permit elimination of distortion in a captured image.

• Allow for matching of two different images of the same object

• Problem: Transform the distorted image to the correct geometric shape

• Textbook: linear only example in DIP 2.6 pg. 87

• Textbook: “Image Processing, Analysis, and Machine Vision” Sonka et. al. 4.2

Geometric Distortion in Images

• What we see in the image depends on the position of the camera

• Objects in the image may be affected by a noticable perspective distortion (such as this whiteboard, where the right side of the board looks shorter than the left side)

• It might be easier to read if we could transform the image so that the whiteboard appears rectangular again.

Images courtesy of Zhang & He (2003)

Transformed

Geometric Distortion in Images

• We might have a data projector which projects a rectangular image but we want to position the projector at an angle to the wall.

• The projected image will be affected by a noticable distortion (such as this image from a projector attached to the ceiling)

• It will be easier to read if we could transform the image so that when it is projected onto the wall it appears rectangular (modern data projectors can do this transformation)

• The transformation is function that can convert an image from one shape into another

Images courtesy of Wikipedia: Keystone effect

The image we want to see

Image as it appears on the wall

Undistorted image

Transforming the Image

• Here is a general equation to convert an arbitrary four sided polygon to another arbitrary four sided polygon

• We apply this equation to our images to calculate the corresponding ( , ) in the distorted image from any (i,j) in the undistorted image.

• We will use this to correct the distortion

• (i,j)

• ( , ) j ^ ^

i

i

j

j ^ ^

i Distorted image

Undistorted image

Transforming the Image

• The general equation contains 8 unknown values denoted a1, a2, a3, a4 and b1, b2, b3, b4 which we need to define so that the equation will transform between our images

• To solve for these 8 unknowns we need to create 8 simultaneous equations where we know the values of i,j, ,

• (i,j)

• ( , ) j ^ ^

i

i

j

Distorted image

j ^ ^

i

Transforming the Image

Undistorted image • We define 4 points in the undistorted image (i,j) , and 4 matching points in the distorted image ( , ) and we have all the values we need to create our 8 simultaneous equations

• Substituting these values into the general equation we can get our 8 simulataneous equations

•

(0,18)

j ^ ^

i

i

j

Distorted image

(0,0)

• •

• (60,0)

(60,40) (0,40)

•

•

• •

(54,0)

(60,32)

(15,32)

)32,15()40,0(

)32,60()40,60(

)0,54()0,60(

)18,0()0,0(

)ˆ,ˆ(),(

jiji

Undistorted image •

(0,18)

i

j

Distorted image

(0,0)

• •

• (60,0)

(60,40) (0,40)

•

•

• •

(54,0)

(60,32)

(15,32)

)32,15()40,0(

)32,60()40,60(

)0,54()0,60(

)18,0()0,0(

)ˆ,ˆ(),(

jiji

Solve the Equation

The Transformation Method

A. Create a new image that will be our new undistorted image.

B. For each pixel (i,j) in our new image

1. Use the transform function to calculate the corresponding position ( , ) in the distorted image

2. Get the color of the pixel at ( , )

3. Set the color of pixel (i,j) to this value

j ^ ^

i

j ^ ^

i

1820

7

10

3

400

3ˆ

8

3

10

9

800

3ˆ

jiijj

jiiji

Undistorted image

• (i,j)

• ( , ) j ^ ^

i

i

j

Distorted image

Correcting Distortion: Whiteboard example

Images courtesy of Zhang & He (2003)

Transformed

• (i,j)

• (i,j)

• ( , ) j ^ ^

i

2012-09-10 E0005E, Lecture 2 23

Geometric Transformations

• The recorded image is distorted (warped) due to some non-linear optic transfer function.

• Problem: Transform the distorted image to the right geometric shape

Distorted geometry Correct geometry

2012-09-10 E0005E, Lecture 2 24

Geometric Transformations

Barrel-distorted image A set of corresponding point pairs,

‘anchor points’

2012-09-10 E0005E, Lecture 2 25

Geometric Transformations

Points should be uniformly

distributed around the perimter of

the image Coordinates in the

correct image

Coordinates in the

distorted image

2012-09-10 E0005E, Lecture 2 26

Geometric Transformations Notes If the transformation is bi-linear (linear along rows and columns) such as in the earlier example, then m=1 and we require only 4 points (perhaps 3) Non-linear requires higher values of m (m=2 in this example) If the transform does not change rapidly based on position, then low order values of m, 2 or 3 can be used requiring 6 or 10 points.

2012-09-10 E0005E, Lecture 2 27

2012-09-10 E0005E, Lecture 2 28

2012-09-10 E0005E, Lecture 2 29

Geometric Transformations

• As part of development phase 2 you will work with an example with only 4 ‘anchor-points’

• So your transform equations will have the simpler linear form

2012-09-10 E0005E, Lecture 2 30

Geometric Transformations

A Need in Iron Ore Mining

• In mining the mountain is blasted to break it up into fragmented rocks

• An excavator digs out the fragmented rocks and dumps them in a shute that eventually leads to the processing plant for extraction of the iron ore

• The driver of the excavator would like to know if the bucket contains iron ore, or if it contains mostly waste rock

• The driver knows the approximate weight of the rock in the bucket

• If the volume of rock was known, then the rock density could be estimated

• High density would indicate iron ore, low density indicates waste rock

• Waste rock should not be put into the shute as it creates a lot of subsequent processing for no value

• Lots of waste rock would indicate it is time to stop excavating

• How to calculate volume? Image courtesy of LKAB, Underground excavator used at LKAB Kiruna

What is a height image

• A height image is another kind of digital image where the pixel color doesn’t represent the color of the objects in the scene, but the height of the objects.

• Below are two height maps of an excavator bucket viewed from the ceiling of the tunnel as the excavator drove underneath.

• The left image is a bucket containing rocks, the right image is an empty bucket. (White values are higher than gray values)

A solution for improved sustainability and productivity in mining

• As the images were collected at different times the excavator is not in the same position or orientation in the two images.

• Therefore we can apply our algorithm for correcting distortion in images

• Choose the empty bucket to be our undistorted image, the full bucket to be our distorted image

• Use the corner points in both images to build the transformation function and apply

Full bucket Empty bucket Full bucket transformed to match the empty bucket

A solution for improved sustainability and productivity in mining

• Subtract empty bucket from the transformed full bucket to get the volume image

• Calculate the bucket volume by adding all the height values in the volume image multiply by the known area of pixel

• Use the volume with the known weight of the bucket to estimate density and therefore rock type (iron ore or waste rock)

Empty bucket Volume image Full bucket transformed to match the empty bucket

POINT OPERATIONS AGAIN

E0005E Industrial Image Analysis

Matthew Thurley

2012-09-10 E0005E, Lecture 2 35

2012-09-10 36

Point operations (lecture 1)

• A point operation is an operation where the gray level gi at a certain pixel (i,j) is replaced with a new gray level go according to some mapping go = F(gi)

(i,j)

gi = I(i,j)

go = F(gi)

I(i,j) = go

E0005E, Lecture 2

2012-09-10 37

Point operations (lecture 1)

• Point operations are implemented as a Look Up Table

- If the mapping F(g) is independent of pixel position

2 9 0 1

0 7 4 3

9 8 1 6

3 0 6 2

Original image New image

Look Up Table

0 2 4 8 16 16 16 16 32 64

0 1 2 3 4 5 6 7 8 9

E0005E, Lecture 2

NEIGHBOURHOOD OPERATIONS

E0005E Industrial Image Analysis

Matthew Thurley

2012-09-10 E0005E, Lecture 2 38

Liner versus Non-linear operations

• Some operations, like point-operations will be described as linear operations, and other operations will be described as non-linear.

• Consider a general operator H, that produces an output image Ig, given an input image If, then

Ig = H[If]

H is a linear operator if

H[ a1.If1 + a2.If2 ] = a1.H[If1] + a2.H[If2]

= a1.Ig1 + a2.Ig2

This simply says that if we sum two scaled images and then apply the operator, it is the same as applying the operator to the images individually first and then scaling and summing

For example; the summation operator is linear, the max and min operators are non-linear (consider if a1 = -1)

2012-09-10 E0005E, Lecture 2 39

DIP 2.6.2

2012-09-10 40

Neighbourhood operations

• A square neighbourhood N(i0,j0) to the pixel (i0, j0)

the group of pixels (i0 – k, j0 – l )

where –m ≤ k ≤ m and –n ≤ l ≤ n

are odd constants

• Example

m = n = 1

Size 3x3

E0005E, Lecture 2

2012-09-10 41

Neighbourhood operations

• The graylevel gi at a certain pixel (i0, j0) is replaced with a new graylevel go according to some mapping

- go = F(N(i, j))

E0005E, Lecture 2

Operation on the whole

neighbourhood N

2012-09-10 42

Linear sliding-window operations

• Neighbourhood operation - linear sum of the graylevels in a neighbourhood N

E0005E, Lecture 2

2012-09-10 43

Non-linear sliding-window operations

• Maximum in a neighbourhood N

• Minimum in a neighbourhood N

E0005E, Lecture 2

• You need to consider how to handle the image boundary.

+ Can use only the part of the mask that intersects the image. Matlab does this with imfilter if you require a resultant image that is the same size.

+ For edge filters: Could pad the resultant edge image with zeros

A Note about Neighbourbood Operations

2012-09-10 E0005E, Lecture 2 44

NOISE FILTERING

E0005E Industrial Image Analysis

Matthew Thurley

2012-09-10 E0005E, Lecture 2 45

Noise Filtering – Image Restoration

• We now consider two types of noise and two noise filters

- Guassian white noise

- Salt and pepper noise

Textbook: DIP 5.3 Restoration in the Presence of Noise Only - Spatial Filtering

2012-09-10 E0005E, Lecture 2 46

2012-09-10 47

Noise – Additive (guassian) white noise

• At every pixel in the original image I a random value ξ is added to the original grayvalue.

- ξ = N(0, σ2)

- The random variable is normally-distributed and the noise values for any two pixels are independent.

• A noise contaminated image D is given by

D(i, j) = I(i, j)+ξ(i, j)

E0005E, Lecture 2

2012-09-10 48

Noise – additive white noise

E0005E, Lecture 2

D(i, j) I(i, j)

2012-09-10 49

Noise – filtering white noise

E0005E, Lecture 2

• Averaging filter mask reduce additive white noise

2012-09-10 50

Noise – additive white noise

E0005E, Lecture 2

D(i, j) I(i, j) Filtered image

2012-09-10 51

Noise – salt and pepper noise

• At certain pixels in the original image I the original grayvalue has been replaced with either a very high or a very low value.

• An binary expression of salt and pepper noise

E0005E, Lecture 2

2012-09-10 52

Noise – salt and pepper noise

E0005E, Lecture 2

D(i, j) I(i, j)

2012-09-10 53

Noise – filtering salt and pepper noise

E0005E, Lecture 2

• Median filter mask reduce salt and pepper noise

Median value (middle value)

Sorted array of graylevels

2012-09-10 54

Noise – salt and pepper noise

E0005E, Lecture 2

D(i, j) I(i, j) Filtered image

Neighbourhood Operations – Matlab Example

• Sliding window operations, both linear and non-linear can be performed in many different ways in MATLAB.

• To implement a median filter we first create a filter function > funcM = ’median(x(:))’;

• We can use this in a useful general function where m and n control the neighbourhood size > ImM = nlfilter(Im,[2*m+1 2*n+1],funcM)

• To implement a linear white noise filter shown earlier we create a different filter function, and then use that > funcA = ’((1/15)*x(:)’*[1 2 1 2 3 2 1 2 1]’)’;

> ImA = nlfilter(Im,[2*m+1 2*n+1],funcA)

2012-09-10 E0005E, Lecture 2 55

2012-09-10 56

Edges (more next lecture)

• Edges are one of the most important features in an image since the edges are strong indicators of object boundaries both for humans and in segmentation algorithms.

• Edge preserving filters attempt to filter noise while maintaining important edge information

• Textbook: Sonka 4.3.1 pg 73 Averaging using a rotating mask (very similar)

• Textbook: DIP 5.3.3 contains Adaptive reduction filter that is also variance based but more complicated

E0005E, Lecture 2

2012-09-10 57

Edge Preserving Filters

E0005E, Lecture 2

2012-09-10 58

Edge Preserving Filters

E0005E, Lecture 2

Neighborhood with vertical orientation

Neighborhood with horizontal orientation

2012-09-10 59

Direction-of-Least-Variance

• The sample variance of a vector (line) L is defined as

E0005E, Lecture 2

Consider the 10 pixels along the red line, 2 black, 2 white ...

Intensity profile along the line L (the red line)

2012-09-10 60

Direction-of-Least-Variance

• Check the variance in M (in this case 4) directions in the neighbourhood

E0005E, Lecture 2

Mean(L)

Do the smoothing in the direction of least variance, in this case vertically, perhaps using a filter

0 1 0

0 1 0

0 1 0

2012-09-10 61

DVL – example of corner detail

E0005E, Lecture 2

References & Further Reading

• Zhengyou Zhang & Li-wei He, Whiteboard Scanning and Image Enhancement, Microsoft Research, Technical Report MSR-TR-2003-39, June 2003, http://research.microsoft.com/en-us/um/people/zhang/papers/tr03-39.pdf

• Keystone effect, Wikipedia, Jan 2010, http://en.wikipedia.org/wiki/Keystone_effect

• Gonzalez R. and Woods R., Geometric Spatial Transformations and Image Registration, pg. 87, Digital Image Processing, Third Edition, Pearson Education, ISBN-10: 0-13-505267-X

• Sonka M., Hlavac V., Boyle R., Geometric Transformations, section 4.2, Image Processing, Analysis, and Machine Vision, Second Edition, International Thomson Publishing Inc., ISBN: 0-534-95393

• Contact me: matthew.thurley@ltu.se

This lecture contains some examples of how to include references and cite sources

Summing Up

• Consider the following three questions;

- What do I need to work on?

- What have I learnt today?

- What was the main point left unanswered today?

• Write your answers in the provided journal. Write the lecture number 2 on top of the page. Write your name and student number on the front of the book

2012-09-10 E0005E Industrial Image Analysis 63