Motivation - University of Hong Kong

Image Intensity and Point Operations

Dr. Edmund Lam

Department of Electrical and Electronic EngineeringThe University of Hong Kong

ELEC4245: Digital Image Proessing(Second Semester, 2015–16)

http://www.eee.hku.hk/˜elec4245

E. Lam (The University of Hong Kong) ELEC4245 Jan–Apr, 2016 1 / 43

Motivation

A digital image is

a matrix of numbers, each corresponding to certain brightness.


Motivation

A/D converter D/A convertersensor displaydigital image

finite dynamic range finite representation finite dynamic range

These numbersare called intensity, or gray levelsmust be nonnegativemust fall within a range of discrete values (dynamic range)are measured by the number of bits.


Gray Levels

How many gray levels are “enough”?Often, 8-bit. 28 = 256 levelsYour computer likes it: 8 bits = 1 byteFor an image of size X × Y, your computer can store it with XYbytes (each pixel needs 1 byte to store its intensity). In reality, weneed much less, due to compression.

But other values exist:Printing: We may only have 1-bit (“ink” or “no ink” at a specificlocation)High dynamic range (HDR) imaging: With better sensors anddisplays, we may record and show a wider rangeWhat’s the limit? Our eyes can see about 14 orders of magnitude!


Gray Levels

8-bits 7-bits 6-bits 5-bits

4-bits 3-bits 2-bits 1-bit


Point operations

1 Point operations

2 Image histogram

3 Combining images


Point operations

Gray level mapping

We will focus on discussing gray level images as the notations andconcepts are much easier to understand. For color images, we canalways perform such operations on the luminance channel. (morelater)

Let the input image be represented by Iin(x, y). We process theimage, and the output is represented by Iout(x, y).The simplest kind of processing is a point-wise operation:

Iout(x, y) = T {Iin(x, y)

}

where Tcan be a one-to-one mapping (reversible)can be a many-to-one mapping (irreversible)cannot be a one-to-many mapping

For every pixel, we change the intensity from value input to output


Point operations

Gray-level mapping

The algorithm can be represented by an input-output plot

input intensity

output intensity

It can usually be implemented as a look-up table (LUT) for maximumefficiency.


Point operations

Gray-level mapping

LUT is most flexible. But conceptually, let’s consider formulas:

Iout(x, y) =

{0 Iin(x, y) < T

255 Iin(x, y) ≥ T Threshold (1)

Iout(x, y) = 255 − Iin(x, y) Negative (2)

Iout(x, y) = c log[1 + Iin(x, y)

]Logarithm (3)

Iout(x, y) = c[Iin(x, y)

]γ Power-law (4)

Pick c and γ so that Iout(x, y) is within [0, 255].


Point operations

Gray-level mapping

input

output

Thresholdinput

output

Negative

input

output

Logarithminput

output

Power-law


Point operations

Threshold

Original Modified


Point operations

Threshold

Output is a binary imageT can be set at the mid-point of the intensity range (i.e., 128), butany other number is also fine.Theoretically, we lost 7/8 of the total information! Butsurprisingly, we retain most of the useful information.Thresholding is often used as part of a computer vision process,e.g., in pattern recognition or defect detection.


Point operations

Negative

Original Modified

Not used often. For “ordinary” images it would look funny.

More useful for images we don’t normally see, such as medical images.E. Lam (The University of Hong Kong) ELEC4245 Jan–Apr, 2016 13 / 43

Point operations

Logarithm

Original Modified


Point operations

Power-law

Example (γ = 1.5):

Original Modified


Point operations

Bit-plane slicing

Represent each pixel value in binary, and then create a binary imagefor each bit. Each such image is called a bit-plane.

1 0 1 1 0 1 0 0180 =

0 0 1 1 0 1 0 153 =...

plane 8 plane 1

Plane 8 is “most significant” while plane 1 is “least significant”E. Lam (The University of Hong Kong) ELEC4245 Jan–Apr, 2016 16 / 43

Point operations

Bit-plane slicing

Bit 8 Bit 7 Bit 6 Bit 5

Bit 4 Bit 3 Bit 2 Bit 1


Point operations

Multiple bit-planes

Bit 8 Bit 8–7 Bit 8–6 Bit 8–5

Bit 8–4 Bit 8–3 Bit 8–2 Bit 8–1


Point operations

Application: Watermarking

Replace bit-plane 1 with another binary image as digital “watermark”

Bit 8–2 Bit 1

+ =

Slicing this image:

Bit 8 Bit 7 Bit 6 Bit 5 Bit 4 Bit 3 Bit 2 Bit 1


Point operations

Application: Watermarking

This is one form of digital watermarking: hiding information digitallyOften used for authentication: for example, to show that a certainpicture is owned by youA fancy word — steganography: art or practice of concealing amessage, image, or file within another message, image, or fileThe method using bit-plane slicing is simple, easy to implement,and the watermark is easy to detectDrawback: the watermark is not robust: it can easily be destroyedor replacedThere are much more sophisticated schemes


Image histogram

1 Point operations

2 Image histogram

3 Combining images


Image histogram

Histogram

Each pixel has a value (intensity). By collecting all the pixels together,we can form a histogram.

The spatial information is lost!The histogram can give us a vague idea of the intensityconcentrations.


Image histogram

Histogram

Original Too dark Too bright Equalized


Image histogram

Histogram

The histogram can be helpful to provide the “curve” for gray-levelmapping.

Histogram equalization: output image has (roughly) the samenumber of pixels of each gray level (hence “equalized”)

Good thing: make use of all available gray levels to the maximumextentReality: this is only approximate because we are not allowedone-to-many mapping (see the next example)Conceptually: (for 8-bit) lowest 1/256 intensity of all pixels map tointensity 0; next 1/256 map to intensity 1; next map to 2, etc.Mainly works when the illuminating condition has problem


Image histogram

Histogram equalization

Example: 3-bit image, 64 × 64 pixels. Assume the following:

gray level number of pixels

0 7901 10232 8503 6564 3295 2456 1227 81


Image histogram


1 Gray levels: [0, . . . , 7], total 4096 pixels2 Proportion of input pixels at level 0: 790/4096 ≈ 0.19. We need to

fill the entire range of 0 to 7, so such pixels should map to0.19 × 7 ≈ 1.33. Round to the nearest integer, we map them to 1.

3 Proportion of input pixels at level 0 and 1: (790 + 1023)/4096 ≈ 0.44.Level 1 should map to 0.44 × 7 ≈ 3.08→ 3.

4 Proportion of input pixels at level 0 to 2:(790 + 1023 + 850)/4096 ≈ 0.65. Level 2 should map to0.65 × 7 ≈ 4.55→ 5.

5 Similarly: Level 3→ 6, Level 4→ 6, Level 5→ 7, Level 6→ 7,Level 7→ 7


Image histogram


Generally,Assume L levels, and j = 0, . . . ,L− 1; image is of size M×N. Let n jdenote the number of pixels at level j.We compute, for each k,

sk =L − 1MN

k∑

j=0

n j k = 0, 1, . . . ,L − 1 (5)

so each sk is the ideal output level for an input level k.We are limited to integer output levels, so we quantize sk.


Image histogram


input intensity

count

input intensity

output intensity

00

1

1

2

2

3

3

4

4

5

5

6

6

7

7

output intensity

count

Note that output histogram is roughly flat, but not strictly.E. Lam (The University of Hong Kong) ELEC4245 Jan–Apr, 2016 28 / 43

Image histogram


Research: What is undesirable, and how to improve the algorithm?

Original Histogram equalization


Image histogram

Adaptive histogram equalization

Modification: adaptive histogram equalizationHistogram equalization based on a portion of the image, e.g.,every non-overlapping 16 × 16 block (tile). Limit contrastexpansion in flat regions by clipping values. Smooth blending(bilinear interpolation) between neighboring tiles.

Original Global equalization Adaptive equalization


Combining images

1 Point operations

2 Image histogram

3 Combining images


Combining images

Pointwise operations

We can perform point-by-point (also known as pointwise) operations tocombine several images.

Assume the images are of the same size:

addition: I(x, y) = a(x, y) + b(x, y)

subtraction: I(x, y) = a(x, y) − b(x, y)

multiplication: I(x, y) = a(x, y) · b(x, y)

division: I(x, y) =a(x, y)b(x, y)


Combining images

Addition and averaging

Average images to reduce noise

1 image 8 images 32 images


Combining images


Assume each image is corrupted by additive white Gaussian noise:

fi(x, y) = g(x, y) + ni(x, y) (6)

g(x, y) is the “ideal” noise-free imagefi(x, y) is what we capture (subscript i to denote the ith one)ni(x, y) is the noise. Every pixel of the noise follows a Gaussiandistribution with mean zero and the same standard deviation σ. Thestandard deviation (or variance σ2) of the noise indicates howsevere the image is corrupted.We use the expected value E, such that

E[ni(x, y)] = 0 (7)

E[n2i (x, y)] = σ2 (8)


Combining images


Example

σ2 = 0.001 × 2552 σ2 = 0.01 × 2552 σ2 = 0.1 × 2552


Combining images


Assume we now have K images, f1(x, y), . . . , fK(x, y)

f̃ (x, y) =1K

K∑

i=1

fi(x, y)

Noise in one image:

E[( f1(x, y) − g(x, y))2] = E[n21(x, y)] = σ2 (9)


Combining images


Noise in the averaged image:

E[( f̃ (x, y) − g(x, y))2] = E[(1K

K∑

i=1

fi(x, y) − 1K

K∑

i=1

g(x, y))2] (10)

= E[(1K

K∑

i=1

ni(x, y))2] (11)

=1

K2

K∑

i=1

E[(ni(x, y))2] (12)

=1

K2 · K · σ2 =1Kσ2 (13)

Eq. (12) is valid provided E[nin j] = 0 when i , j


Combining images

Subtraction

Spot the difference:

No defect, f1(x, y) With defect f2(x, y)


Combining images

Subtraction

Take the difference: | f1(x, y) − f2(x, y)|No alignment Properly aligned and thresholded

Research: How to align?E. Lam (The University of Hong Kong) ELEC4245 Jan–Apr, 2016 39 / 43

Combining images

Multiplication

We can think about how an image is formed: (the imaging process)

f (x, y) = i(x, y)r(x, y) (14)

i(x, y) is the illumination source: 0 < i(x, y) < ∞r(x, y) is the reflectance: 0 < r(x, y) < 1Some images are formed with transmission (e.g. x-ray), thenr(x, y) is the transmissivityf (x, y) are confined to the available dynamic range when capturedby a detector


Combining images

Other combinations

High dynamic range (HDR) imaging: combining images from differentexposures


Combining images

Other combinations

Removing occlusion

−→

(source: Herley, “Automatic occlusion removal from minimum number ofimages,” ICIP 2005)


Combining images

Summary

We looked at image enhancement with one or more images as input.

We consider each pixel location as unrelated to its neighbors.Next: We look at image processing that involves the neighborpixels.


Documents

Motivation - University of Hong Kong