1

What is Image Quality Assessment?

Image quality is a characteristic of an image that measures the perceived image degradation

It plays an important role in various image processing application.

Goal of image quality assessment is to supply quality metrics that can predict perceived image quality automatically.

Two Types of image quality assessment– Objective quality assessment – Subjective quality assessment

2

Two Types of image quality assessment

3

Objective (MSE, PSNR, SSIM) [1,2]– Automated– Low cost– Less accurate

Subjective: Mean Opinion Score (MOS) [4]– Involving human participants– Expensive– More accurate and ecologically valid

Subjective Quality Measure

The best way to find the quality of an image is

to look at it

because human eyes are the ultimate viewer.

Subjective image quality is concerned with how image is perceived by a viewer and give his or her opinion on a particular image.

4

Subjective Quality Measure

The mean opinion score (MOS) has been used for subjective quality assessment for many years.

In Audio standard subjective test listeners (both male and female, professional and non-professional) rate the heard audio quality over the communication medium being tested.

In case of image, the image quality is measured by viewers (both male and female, professional and non-professional) on the display devise or printed media.

However, this approach is too inconvenient, time consuming and expensive.

5

Example of mean opinion score (MOS)

The MOS is generated by averaging the result of a set of standard, subjective tests.

MOS is an indicator of the perceived image quality.

Mean Opinion Score of 1 is worst image quality and 5 is best [4].

Mean Opinion Score (MOS)

MOS Quality Impairment

5 Excellent Imperceptible

4 Good Perceptible but not annoying

3 Fair Slightly annoying

2 Poor Annoying

1 Bad Very annoying

6

Objective Quality Measure

8

Objective Quality Measure

Mathematical models that approximate the results of subjective quality assessment.

Goal of objective evaluation is to develop quantitative measure that can predict perceived image quality.

9

Objective Quality Measure

Objective quality measure plays variety of roles:

– To monitor and control image quality for quality control systems

– To benchmark image processing systems;– To optimize algorithms and parameters;

– To help home users better manage their digital photos and evaluate their expertise in photographing.

10

Objective evaluation

Three types of objective evaluation

It is classified according to the availability of an original image with which distorted image is to be compared

– Full reference (FR)– No reference –Blind (NR)– Reduced reference (RR)

11

Full reference quality metrics

MSE and PSNR: the most widely used video/image quality metrics during last 20 years.

SSIM: new metric (was suggested in 2004) shows better results than PSNR with reasonable computational complexity increasing.

A great effort has been made to develop new objective quality measures for image/video that incorporate perceptual quality measures by considering the human visual system (HVS) characteristics.

12

HVS – Human visual system

Quality assessment (QA) algorithms predict visual quality by comparing a distorted signal against a reference, typically by modeling the human visual system.

The objective image quality assessment is based on well defined mathematical models that can predict perceived image quality between a distorted image and a reference image.

These measurement methods consider human visual system (HVS) characteristics in an attempt to incorporate perceptual quality measures.

13

MSE – Mean square error

MSE and PSNR are defined as

Where x is the original image and y is the distorted image.

M and N are the width and height of an image.

L is the dynamic range of the pixel values.

If the MSE decrease to zero, the pixel-by-pixel matching of the

images becomes perfect.14

Same MSE but different qualities

MSE for all images = 21515

SSIM – Structural similarity index

Recently proposed approach for image quality assessment.

A metric for measuring the similarity between two images.

Its value lies between [0,1]

The SSIM is designed to improve on traditional metrics like PSNR and MSE, which have proved to be inconsistent with human eye perception. Based on human visual system.

16

SSIM measurement system

17

Assume we have two images x and y, where xi and yi ,

i = 1, 2, …, N are their elements. The aim is to determine how similar images x and y are.

Mean and SD for image x is defined as follows:

How to measure SSIM

18

How to measure SSIM

To compare the similarity of two images x and y,

we need to compare their

Luminance : l Contrast: c and Structure: s

By combining them in a weighted function as follows: S(x,y) = f ( l(x,y), c(x,y), s(x,y) )

19

How to measure SSIM

S(x,y) = f ( l(x,y), c(x,y), s(x,y) )So we need to define the above three functions l,c,s and also the function f(.).

However, the similarity measure S(x,y) should satisfy the following conditions:

1- Symmetry: S(x,y) = S(y,x)

2- Boundedness: S(x,y) ≤ 1

3- Unique maximum: S(x,y) = 1 if and only if x=y, that is

xi = yi, for all i=1,2, …, N)

20

How to measure SSIM

S(x,y) = f ( l(x,y), c(x,y), s(x,y) )

Luminance comparison is defined as:

Where the constant C1 to avoid division by a very small (or zero) number.

Where L is the dynamic range of pixel values (255 for 8-bit gray scale image), and K1 « 1 is a small constant.

21

𝑙 (𝑥 , 𝑦 )=2𝜇𝑥𝜇𝑦+𝐶1

𝜇𝑥2 +𝜇𝑦

2 +𝐶1

How to measure SSIM

S(x,y) = f ( l(x,y), c(x,y), s(x,y) )

Contrast comparison is defined as:

Where the constant C1 to avoid division by a very small (or zero) number.

Where L is the dynamic range of pixel values (255 for 8-bit gray scale image), and K2 « 1 is a small constant.

22

𝑐 (𝑐 , 𝑦 )=2𝜎𝑥𝜎 𝑦+𝐶2

𝜎 𝑥2 +𝜎 𝑦

2 +𝐶2

How to measure SSIM

S(x,y) = f ( l(x,y), c(x,y), s(x,y) )

Structure comparison is defined as:

Where the constant C3 to avoid division by a very small (or zero) number.

23

𝑠 (𝑥 , 𝑦 )=𝜎𝑥𝑦+𝐶3

𝜎 𝑥𝜎 𝑦+𝐶3

𝑆𝑆𝐼𝑀 (𝑥 , 𝑦 )= [ 𝑙(𝑥 , 𝑦 ) ]𝛼 . [𝑐 (𝑥 , 𝑦 )]𝛽 . [ 𝑠 (𝑥 , 𝑦 )]𝛾

How to measure SSIM

Where α > 0, β > 0, γ > 0 are parameters used to adjust the relative importance of the components.

It is easy to verify that this definition of SSIM satisfies the three conditions:1- Symmetry: S(x,y) = S(y,x)

2- Boundedness: S(x,y) ≤ 1

3- Unique maximum: S(x,y) = 1 if and only if x=y,

24

𝑆𝑆𝐼𝑀 (𝑥 , 𝑦 )= [ 𝑙(𝑥 , 𝑦 ) ]𝛼 . [𝑐 (𝑥 , 𝑦 )]𝛽 . [ 𝑠 (𝑥 , 𝑦 )]𝛾

How to measure SSIM

To simplify the above equation, we set α = β = γ = 1

SSIM ε [0,1], Values closer to 1 indicate higher similarity.

25

𝑆𝑆𝐼𝑀 (𝑥 , 𝑦 )= [ 𝑙(𝑥 , 𝑦 ) ]𝛼 . [𝑐 (𝑥 , 𝑦 )]𝛽 . [ 𝑠 (𝑥 , 𝑦 )]𝛾

𝑆𝑆𝐼𝑀 (𝑥 , 𝑦 )=(2𝜇𝑥𝜇𝑦+𝐶1)(2𝜎 𝑥𝑦+𝐶2)

(𝜇𝑥2 +𝜇 𝑦

2 +𝐶1)(𝜎 𝑥2 +𝜎 𝑦

2 +𝐶2)

How to measure SSIM

If we divide images x and y into blocks and then compute the similarity of each two corresponding blocks, and give a weight to the similarity measure of each block, then the average SSIM over all blocks is a new measure calledMSSIM

Where M is the number of blocks, and xi and yj are two corresponding image blocks.

26

𝑆𝑆𝐼𝑀 (𝑥 , 𝑦 )=(2𝜇𝑥𝜇𝑦+𝐶1)(2𝜎 𝑥𝑦+𝐶2)

(𝜇𝑥2 +𝜇 𝑦

2 +𝐶1)(𝜎 𝑥2 +𝜎 𝑦

2 +𝐶2)

MSE=0, MSSIM=1 MSE=225, MSSIM=0.949 MSE=225, MSSIM=0.989

MSE=215, MSSIM=0.671 MSE=225, MSSIM=0.688 MSE=225, MSSIM=0.723

MSE VS MSSIMoriginal Image

MSE vs. MSSIM

28Sparkle noise causes artificial artifacts in digital images. It appears as a bright spots which have a typical intensity of 40% of maximum intensity.

MSE vs. MSSIM simulation result

Type of Noise MSE MSSIM

Salt & Pepper Noise 228.34 0.7237

Spackle Noise 225.91 0.4992

Gaussian Noise 226.80 0.4489

Blurred 225.80 0.7136

JPEG compressed 213.55 0.3732

Contrast Stretch 406.87 0.9100

29

MSE vs. MSSIM

MSE=226.80 MSSIM =0.4489 MSE = 225.91 MSSIM =0.4992

30

MSE vs. MSSIM

MSE = 213.55 MSSIM = 0.3732 MSE = 225.80 MSSIM =0.7136

31

MSE vs. MSSIM

MSE = 226.80 MSSIM = 0.4489 MSE = 406.87 MSSIM =0.910

32

Why MSE is poor?

MSE and PSNR are widely used because they are simple and easy to calculate and mathematically easy to deal with for optimization purpose.

There are a number of reasons why MSE or PSNR may not correlate well with the human perception of quality.

33

Why MSE is poor?

– Digital pixel values, on which the MSE is typically computed, may not exactly represent the light stimulus entering the eye.

– Simple error summation, like the one implemented in the MSE formulation, may be markedly different from the way the HVS and the brain arrives at an assessment of the perceived distortion.

– Two distorted image signals with the same amount of error energy may have very different structure of errors, and hence different perceptual quality.

34

Conclusion

We consider the proposed SSIM indexing approach as a particular implementation of the philosophy of structural

similarity, from an image formation point of view.

35

References

[1] Z. Wang and A. C. Bovik, “Image quality assessment: from error visibility to structural similarity,” IEEE Trans. Image Processing, vol. 13, pp. 600 – 612, Apr. 2004. www.ece.uwaterloo.ca/~z70wang/publications/ssim.html

[2] Z. Wang and A. C. Bovik, “Modern image quality assessment”, Morgan & Claypool Publishers, Jan. 2006.

[3] X. Shang, “Structural similarity based image quality assessment: pooling strategies and applications to image compression and digit recognition” M.S. Thesis, EE Department, The University of Texas at Arlington, Aug. 2006. 

[4] ITU-R Rec. BT. 500-11: Methodology for the Subjective Assessment of Quality for Television Pictures. June 2002.

36