COURSE NAME

DIGITAL IMAGE PROCESSING

COURSE CODE: EC 802

Dr. Mrutyunjay RoutDept. of Electronics and communication Engineering

NIT Jamshedpur

1NIT Jamshedpur

NIT Jamshedpur 2

Course Description

Chapter-I: Basic of Digital Image Processing and Fundamental Concepts Digital

Image Processing : Digital Image Representation, Fundamental Steps in Digital Image

Processing, elements of an Digital Image Processing System, Visual Perception,

Structure of the Human Eye, Image Formation in the Eye, Brightness Adaptation and

Discrimination, Image Model, Sampling and Quantization, Pixel Relationships, image

geometry and basic transformation, photo graphic film.

Chapter-II: Image Transformations: Fourier and fast Fourier transform, other

transforms, two dimensional transform and their properties (Walsh, Hadamard and

discrete cosine transforms etc.).

Chapter-III: Image Enhancement: Technique, enhancement by point processing,

spatial filtering, enhancement in frequency domain colour processing.

Chapter-IV: Image Restoration: Degradation model, algebraic approach to

restoration, uniform linear motion, Weiner filter, least square restoration, geometric

transformation, spatial transformation, grey level interpolations.

Chapter-V: Image Compression: Data reduency, fidelity criteria, encoder and

decoder, variable length coding, bit phase coding, lossless predictive coding, lossy

predictive coding, transform coding, image compression standards.

NIT Jamshedpur 3

Books

Text Books:

1. Rafael C Gonzalez and Richard E Woods, “Digital Image

Processing”, 2nd Edition, Pearson Education, 2003.

2. Anil K. Jain, “Fundamentals of Digital Image Processing”,

Prentice Hall, 1989.

Reference Books:

1. William K. Pratt, “Digital Image Processing”, 3rd Edition”, John

Wiley, 2001.

2. S. Chanda and Dutta Majumdar, “Digital Image Processing and

Applications”, Prentice Hall of India, 2000.

3. Kenneth R. Castleman, “Digital Image Processing”, Prentice

Hall, 1996.

Lecture: 1 to 8

➢What is an image

➢ Image Formation

➢ Image Formation in the Eye

➢ Image Representation

➢Application and Example

➢Types of Image

➢ Image Taxonomy

4NIT Jamshedpur

Introduction to Digital Image

Processing

• Manipulation of an image by means of a processor

• Processing of an image by means of computer

• Digital processing of any 2-D data

5NIT Jamshedpur

Imageacquisition

Digitalstorage

ImageProcessing by

Digital computer

DisplaySample

and Quantize

• An Image is a 2D function, f (x, y), Where x and y are

spatial coordinates. Amplitude of f at any pair of coordinates

(x, y) is called the intensity or gray level of the image.

• When spatial coordinates and amplitude values are all finite,

discrete quantities, the image is called as digital image.

6NIT Jamshedpur

Introduction to Digital Image

Processing

NIT Jamshedpur 7

Structure of Human Eye

Image Formation in the Eye

NIT Jamshedpur 8

• Brightness

Overall lightness or darkness of an image

• Contrast

Difference in brightness between objects or regions

Why do we process images

• To facilitate their storage

Efficient storage in digital cameras

Video streaming on the internet

• To prepare them for display or printing

• To enhance or restore themImprove visibility of features

Repair photographic errors

• To extract information from them

Face recognition

9NIT Jamshedpur

Application and Example

• Noise Filtering

• Content enhancement

• Contrast enhancement

• Deblurring

• Remote sensing

10NIT Jamshedpur

Filtering

11

Noisy Image Filtered Image

NIT Jamshedpur

Image Enhancement

12

Low Contrast Image Enhanced image

NIT Jamshedpur

Image DeblurringDefocused Motion blurring

Deblurred

13NIT Jamshedpur

Remote Sensing

14NIT Jamshedpur

Types of Digital image• Black-n-White Image

Binary Image (two-tone) Gray level Image (gray-tone)

15NIT Jamshedpur

Types of Digital image• Color Image

• Another classificationStill ImageMovie Image (Video Sequences)

16NIT Jamshedpur

Image Process Applications

Image processing

Forensic field

Entertainment

Remote sensing

Medical field

17NIT Jamshedpur

Image Processing Taxonomy

18NIT Jamshedpur

Image Processing Taxonomy

19NIT Jamshedpur

Image Processing Taxonomy

20NIT Jamshedpur

Lecture: 9 to 16

➢ Image sensing and Acquisition

➢ Image Formation Model

➢ Image Sampling and Quantization

➢ Image Representation

➢Basic Relationships between pixels

21NIT Jamshedpur

Image Sensing and Acquisition

• Image acquisition using a single Sensor

• Image acquisition using sensor strips

• Image acquisition using sensor array

22NIT Jamshedpur

23NIT Jamshedpur

Image sensor

NIT Jamshedpur 24

Single Sensor with Motion

Sensor Strips

NIT Jamshedpur 25

Image Formation Model

26NIT Jamshedpur

A simple image model

27NIT Jamshedpur

• Image: A two-dimensional light-intensity function

; f(x, y) = i(x, y) r(x, y) Why? (the 1st, the 2nd)

where, illumination 0 < i(x, y) < ∞,

reflectance components 0 < r(x, y) < 1

- illumination: the amount of source light incident on the scene being viewed

- reflectance: the amount of light reflected by the objects in the scene

The interval [Lmin, Lmax] : called the gray scale → [0, L]

Concepts in sampling and

quantization

28NIT Jamshedpur

Sampling and Quantization

29NIT Jamshedpur

Continuous image projected

onto a sensor array

Result of image sampling and

quantization

Image Representation

30NIT Jamshedpur

Basic Relationships between Pixels

31NIT Jamshedpur

• Neighbors of a pixel

4-neighbors [ 𝑁4(𝑃) ]

Diagonal neighbors [ 𝑁𝐷(𝑃) ]

8-neighbors [ 𝑁8(𝑃) ]

• Adjacency, Connectivity, Regions, Boundaries

and path

Lecture: 17 t0 26

➢Mathematical tools used in digital image

processing

➢ Image Transformation

➢2D Fourier Transform

➢Walsh and Hadamard Transform

➢Discrete Cosine Transform

32NIT Jamshedpur

Mathematical tools used in digital

image processing

• Array Versus Matrix Operations

• Linear Versus Non-linear Operations

• Arithmetic Operations

• Set Operations

• Logical Operations

• Spatial Operations

• Vector and Matrix Operations

33NIT Jamshedpur

34NIT Jamshedpur

Set Operations

NIT Jamshedpur 35

Set Operations

Logical Operations

NIT Jamshedpur 36

Spatial Operations

37NIT Jamshedpur

• Single Pixel Operations

• Neighborhood Operations

• Geometric Spatial Transformations

Spatial Operations

38NIT Jamshedpur

• Single Pixel Operations

Spatial Operations

39NIT Jamshedpur

• Neighborhood Operations

Spatial Operations

40NIT Jamshedpur

• Geometric Spatial Transformations

➢ Translation

➢ Scaling

➢ Rotation

➢ Shear

Spatial Operations

41NIT Jamshedpur

• Image Rotation

NIT Jamshedpur 42

Image Registration

NIT Jamshedpur 43

Formation of vector for color Image

Image Transformation

44NIT Jamshedpur

NIT Jamshedpur 45

2-D DFT

2/122

1

0

1

0

)(2

1

0

1

0

)(2

),(),(),(

),(),(

),(1

),(

vuIvuRvuF

evuFyxf

eyxfMN

vuF

M

u

N

v

yN

vx

M

uj

M

x

N

y

N

yv

M

xuj

+=

=

=

−

=

−

=

+

−

=

−

=

+−

Fourier Spectrum

−

=

−

=

=1

0

1

0

),(1

)0,0(M

x

N

y

yxfMN

F

F(0,0) is the average intensity of an image

46

2-D DFT

NIT Jamshedpur

• Fourier spectrum: 2/122 ),(),(),( vuIvuRvuF +=

• Phase:

= −

),(

),(tan),( 1

vuR

vuIvu

• Power spectrum: ),(),(),(),( 222vuIvuRvuFvuP +==

47

2-D DFT

NIT Jamshedpur

48

2-D DFT

NIT Jamshedpur

49

2-D DFT Properties

NIT Jamshedpur

50

2-D DFT Properties

NIT Jamshedpur

51

2-D DFT Properties

NIT Jamshedpur

52

2-D DFT Properties

NIT Jamshedpur

1. Multiply the input image by (-1)x+y to center the transform

2. Compute F(u,v), the DFT of the image from (1)

3. Multiply F(u,v) by a filter function H(u,v)

4. Compute the inverse DFT of the result in (3)

5. Obtain the real part of the result in (4)

6. Multiply the result in (5) by (-1)x+y

Basic Filtering in Frequency Domain

NIT Jamshedpur 53

Low Pass Filter attenuate high

frequencies while “passing”

low frequencies.

High Pass Filter attenuate

low frequencies while

“passing” high frequencies.

54

Low-pass and High-pass Filters

NIT Jamshedpur

How to Perform 2-D DFT by Using 1-

D DFT

f(x,y)

1-D DFT

by row F(u,y)

1-D DFTby column

F(u,v)NIT Jamshedpur 55

How to Perform 2-D DFT by Using 1-

D DFT

1-D DFT

by rowF(x,v)

f(x,y)

1-D DFTby column

F(u,v)

Alternative method

2-D Walsh Transform

NIT Jamshedpur 57

• Fourier transform is based on trigonometric terms

whereas the Walsh transform consists of basis functions

whose values are only 1 and -1.

• They have the form of square waves.

• The 2-D Walsh transform is separable and symmetric.

• Therefore it can be implemented as a sequence of two 1-

D Walsh transforms, in a fashion similar to that of the 2-D

DFT.

• These functions can be implemented more efficiently in

a digital environment than the exponential basis

functions of the Fourier transform.

2-D Walsh Transform

NIT Jamshedpur 58

• 2-D Walsh transform as a straightforward extension of the

1-D transform:

• The above is equivalent to:

2-D Inverse Walsh Transform

NIT Jamshedpur 59

• Inverse 2-D Walsh transform:

• The above is equivalent to:

2-D Hadamard Transform

NIT Jamshedpur 60

• 2-D Hadamard is similar to the 2-D Walsh transform

• The above is equivalent to:

2-D Inverse Hadamard Transform

NIT Jamshedpur 61

• Inverse 2-D Hadamard transform

• The above is equivalent to:

Recursive Relationship of the

Hadamard Transform

NIT Jamshedpur 62

• An important property of Hadamard transform is that,

letting HN represent the Hadamard matrix of order N the

recursive relationship holds :

• Therefore, starting from a small Hadamard matrixwe can

compute a Hadamard matrix of any size.

• This is a good reason to use the Hadamard transform

Image of 1-D Hadamard Matrices

NIT Jamshedpur 63

Discrete Cosine Transform (DCT)

NIT Jamshedpur 64

• note that

– the better the energy compaction

– the larger the number of

coefficients that get wiped out

– the greater the bit savings for the

same loss

• this is why the DCT

is important

• we will do

mostly the 1D-

DCT

– the formulas are

simpler the insights

the same

– as always, extension

to 2D is trivial

Discrete Cosine Transform (DCT)

NIT Jamshedpur 65

• the first thing to note is that there are various versions of

the DCT

– these are usually known as DCT-I to DCT-IV

– they vary in minor details

– the most popular is the DCT-II

2-D DCT

NIT Jamshedpur 66

• the extension to 2D is trivial

• the procedure is the same

How to Perform 2-D DCT by Using

1-D DCT

NIT Jamshedpur 67

1) Create intermediate

sequence by computing

1D-DCT of rows

2) Computing 1D-DCT of

columns

Lecture: 27 to 31

IMAGE ENHANCEMENT

68NIT Jamshedpur

Why Image Enhancement?

• The aim of image enhancement is to improve thevisual appearance of an image, or to provide a“better transform representation for futureautomated image processing.

• Many images like medical images, satellite images,aerial images and even real life photographs sufferfrom poor contrast and noise.

• It is necessary to enhance the contrast and removethe noise to increase image quality.

• Enhancement techniques which improves thequality (clarity) of images for human viewing,removing blurring and noise, increasing contrast,and revealing details are examples of enhancementoperations.

69NIT Jamshedpur

70NIT Jamshedpur

WHAT IS IMAGE ENHANCEMENT?• Processing an image to enhance certain features of the image

• Result become more suitable than the original image for specific

application

• Processing techniques are highly application dependent

• e.g Best technique for the enhancement of X‐ray images may not

be the best one for microscopic images.

NIT Jamshedpur 71

Different Enhancement TechniquesThe existing techniques of image enhancement can be

classified into two categories:

➢ Spatial Domain Techniques• Work on image plane itself

• Direct manipulation of the image pixels

➢ Frequency Domain Techniques• Modify the Fourier Transform coefficients of an image

• Take inverse Fourier Transform of the modified coefficients to

obtain the enhanced image

Examples Of Image Enhancement

Techniques

NIT Jamshedpur 72

➢ Noise removal

➢ Contrast adjustment

Spatial Domain Enhancement

Techniques

73NIT Jamshedpur

• These techniques operate directly on the image pixels

• More efficient computation and requires less processingresources to implement

• Spatial Domain Process is defined by

Input

ImageOUTPUT

ImageT is an operator on f defined over

a neighborhood of point (x,y)

𝑔 𝑥, 𝑦 = 𝑇 [𝑓 𝑥, 𝑦 ]

Image Enhancement in Spatial

Domain

74NIT Jamshedpur

Again can be broadly categorized into following types• Point Processing Techniques

• Histogram Based Techniques

• Mask Processing Techniques

• 1x1 neighborhood operation is called as point processing and is

represented by the transformation function s= T(r). Where s and r

represents the intensity of g and f respectively

Contrast stretching

75NIT Jamshedpur

• Low-contrast images can result from poor illumination, lack of

dynamic range in the image sensor, or even wrong setting of a lens

aperture.

• The idea behind contrast stretching is to increase the dynamic range

of the gray levels in the image being processed.

• The general form is:

𝑆 =1

1 + Τ𝑚 𝑟𝐸

where, r are the input image values, s are the output image values, m is

the thresholding value and E the slope.

Contrast stretching( cont…)

76NIT Jamshedpur

Figure shows the effect of the variable E:

• If E = 1 the stretching became a threshold transformation.

• If E > 1 the transformation is defined by the curve which is smoother and

• When E < 1 the transformation makes the negative and also stretching.

Contrast stretching( Example)

77NIT Jamshedpur

Image Negative

78NIT Jamshedpur

Let the image has an intensity level in the range [0 L-1],

then the intensity transformation is given by s=L-1-r

Image thresholding

79NIT Jamshedpur

• Thresholding is the simplest segmentation method.

• The pixels are partitioned depending on their intensity value T.

• Global thresholding, using an appropriate threshold T.

• Imagine a poker playing robot that needs to visually interpret the

cards in its hand:

Noise reduction

80NIT Jamshedpur

➢ Median Filtering• The median filter is normally used to reduce noise in an image by

preserving useful detail in the image.

• The median filter considers each pixel in the image in turn and looks at its

nearby neighbors to decide whether or not it is representative of its

surroundings.

• The median is calculated by first sorting all the pixel values from the

surrounding neighborhood into numerical order and then replacing the

pixel being considered with the middle pixel value.

➢ Noise removal using Averaging• Image averaging works on the assumption that the noise in your image is

truly random.

• This way, random fluctuations above and below actual image data will

gradually even out as one averages more and more images.

Histogram equalization

81NIT Jamshedpur

• Histogram Equalization is a technique that generates a gray map which

changes the histogram of an image and redistributing all pixels values to

be as close as possible to a user – specified desired histogram.

• It allows for areas of lower local contrast to gain a higher contrast.

HistogramOriginal Image

Equalized ImageEqualized Histogram

Both images are

quantized to 64

grey levels

Intensity Level slicing

82NIT Jamshedpur

➢ Highlighting specific range of intensities• Enhancing features such as masses of water in the satellite

imagery

• Enhancing flaws in X-ray images.

Lecture: 32 to 37

IMAGE RESTORATION

83NIT Jamshedpur

Image Restoration

• Correction of degraded images

• Reconstruction of an image that has been degraded by using a

priori knowledge of the degradation phenomenon

• Restore an image that has been degraded in some way.

• Make a model of the degeneration process and use inverse

methods.

• Purpose of image restoration is to restore a degraded/distorted

image to its original content and quality.

84NIT Jamshedpur

85NIT Jamshedpur

Image Restoration

➢ Model

NIT Jamshedpur 86

Cause of Degradation

➢ Bad focusing

➢ Motion Blur

➢ Non-linearity of the sensor

➢ Noise

➢ Atmospheric Turbulence

Noise Models

• Most types of noise are modeled as known probability density functions

• Noise model is decided based on understanding of the physics of the sources of noise.

– Gaussian: poor illumination

– Rayleigh: range image

– Gamma, exp: laser imaging

– Impulse: faulty switch during imaging,

– Uniform is least used.

• Parameters can be estimated based on histogram on small flat area of an image

88NIT Jamshedpur

Image Restoration

➢ Possible approaches:

• Inverse filtering.

• Try to model degradation effect.

• Use Fourier-domain methods and identify which frequencies

are related to the degrading effect.

89NIT Jamshedpur

Inverse Filtering

90NIT Jamshedpur

Problems with Inverse Filtering

At deconvolution, the FT of the image is divided by the FT

of the degrading effect.

➢ Problems:

• Small values of H(u, v) can cause overflow (usually small at HF).

• If noise is included, it can be dominating.

➢ Solutions:

• Perform division only in a limited part of the (u, v)-plane.

• Use weights to limit the effect at division with small numbers.

91NIT Jamshedpur

Inverse Filtering Examples

92NIT Jamshedpur

Wiener Filtering

93NIT Jamshedpur

Wiener Filtering

NIT Jamshedpur 94

Filter Comparison

NIT Jamshedpur 95

Drawback with Wiener Filtering

96NIT Jamshedpur

➢ Drawback:• Must know (or approximate) the power spectra of undegraded

• image and noise.

➢ Better choice:

❑ Constrained Least Squares Filter (CLSF).• Need only knowledge of mean and variance of noise (apart from

degradation function).

• Only one parameter, which can be iteratively computed foroptimality.

Lecture: 38 to 42

IMAGE COMPRESSION

97NIT Jamshedpur

Image Compression

➢ The problem of reducing the amount of data

required to represent a digital image.

• Redundant data reduction

• Uncorrelated data confirms redundant data

elimination

➢ From a mathematical viewpoint: transforming

a 2-D pixel array into a statistically

uncorrelated data set.

98NIT Jamshedpur

99NIT Jamshedpur

Why do We Need Compression

➢ For data STORAGE and data TRANSMISSION

• DVD

• Remote Sensing

• Video conference

• FAX

• Control of remotely piloted vehicle

➢ The bit rate of uncompressed digital cinema

data exceeds one Gbps.

NIT Jamshedpur 100

Taxonomy of image compression

methods➢ Based on data interpretation

• Methods are dependent on data semantics.

• Higher compression ratios are achieved.

• Decompression does not reconstruct the input image fully.

➢ Without data interpretation

• Compression can be used for any image, regardless of its

semantics.

• Statistical redundancy and (possibly) irrelevance for human

viewing is explored.

NIT Jamshedpur 101

Without data interpretation image

compression methods

➢ Lossless methods

• Only the statistical redundancy is removed/suppressed.

• A full reconstruction of the original signal/image is possible.

➢ Lossy methods

• Irrelevant information is removed.

• Such information is removed which is unimportant in a given

context(e.g. high frequencies, details in intensity, which is

unobservable by a human eye).

• Only partial reconstruction of the original signal/image is

possible.

NIT Jamshedpur 102

Image compression

and its backward reconstruction

• Transformation T reduces

redundancy and is often

invertible.

E.g., cosine transformation, run

length encoding (RLE).

• Quantizing Q removes

irrelevance and is not invertible.

E.g., neglecting cosine

transformation coefficient matching

to high frequencies.

• Coding C and decoding C−1 are

invertible and lossless.

Error-Free Compression

108NIT Jamshedpur

➢ Applications

• Archive of medical or business documents

• Satellite imaging

• Digital radiography

➢ Coding• Huffman coding

• LZW Coding

• Bit-plane coding

• Loss-less Predictive Coding

➢ They provide: Compression ratio of 2 to 10.

Loss-less Predictive Coding

109NIT Jamshedpur

Loss-less Predictive Coding

110NIT Jamshedpur

Loss-less Predictive Encoder

Loss-less Predictive Decoder

Lossy Compression

111NIT Jamshedpur

➢ Lossy encoding is based on the concept ofcompromising the accuracy of thereconstructed image in exchange forincreased compression.

➢ Lossy encoding techniques are capable ofreproducing recognizable mono-chromeimages from data that have been compressedby more than 100:1 and images that arevirtually indistinguishable from the original at10:1 to 50:1 .

➢ Lossy Compression• Spatial Domain Methods

• Transform Coding

Lossy Predictive Coding

112NIT Jamshedpur

➢Predictive Coding: Transmit the difference

between estimate of future sample & the sample itself.

• Delta modulation

• DPCM

• Adaptive predictive coding

• Differential frame coding

Lossy Predictive Coding

113NIT Jamshedpur

Lossy Predictive Encoder

Lossy Predictive Decoder

Differential Pulse Code

Modulation (DPCM)

114NIT Jamshedpur

DPCM Result Images

115NIT Jamshedpur

DPCM Prediction Error

116NIT Jamshedpur

Transform Coding

117NIT Jamshedpur

Transform Coding System Encoder

Transform Coding System Decoder

The goal of the transformation process is to decorrelate the pixels of

each sub-image, or to pack as much information as possible into the

smallest number of transform coefficients

Transform Coding

118NIT Jamshedpur

Transform Coding

NIT Jamshedpur 119

Transform Coding

120NIT Jamshedpur