Prof. YingLi TianOct. 17, 2018

Department of Electrical EngineeringThe City College of New York

The City University of New York (CUNY)

Lecture 6: Image Restoration

1

I2200: Digital Image processing

Thanks to G&W website and Lexing Xie for slide materials

Announcement: Michael will present HW2 Today. HW3 is out today, due on 11/06. Peter will present HW3 on 11/07. Midterm Exam: Oct. 24, 2018.

Open notes No electronic device is allowed except calculator

Final project: Send me the following info before 10/31, 2018 The project title and a short description if you choose your own

project (must be image processing related). Your partner if you want to team with someone.

2

We have covered …

3

Outline What is image restoration

Scope, history and applications A model for (linear) image degradation

Restoration from noise Different types of noise Examples of restoration operations

Restoration from linear degradation Inverse and pseudo-inverse filtering Wiener filters Blind de-convolution

Geometric distortion and its corrections

Degraded images

5

Image Restoration Image restoration is to "compensate for" or

"undo" defects which degrade an image. Degradations:

motion blur noise camera defocus distortion …

6http://www.owlnet.rice.edu/~elec539/Projects99/BACH/proj2/intro.html

Image Restoration VS Enhancement

7

Image restoration

8

A model for image distortion

9

A model for image distortion

Image restoration Use a priori knowledge of the degradation Modeling the degradation and apply the

inverse process Formulate and evaluate objective criteria of

goodness

10

Usual assumptions for the distortion model

111

Common noise models

12More details, G&W p313-319

(Impulse)

The visual effects of noise

13

Recovering from noise

14

Example: Gaussian noise & mean filter

15

Example: salt-and-pepper noise & median filter

16

Recovering from Periodic Noise

17

Recovering from Periodic Noise in Frequency domain

18

Example of bandreject filter

19

Example of bandpass filter

20Hbp(u, v) = 1 - Hbr(u, v)

Notch filter

21

Example of notch filter

22

Example of notch filter

23

Outline What is image restoration

Scope, history and applications A model for (linear) image degradation

Restoration from noise Different types of noise Examples of restoration operations

Restoration from linear degradation Inverse and pseudo-inverse filtering Wiener filters Blind de-convolution

Geometric distortion and its corrections

Recover from linear degradation

250

Estimate the degradation Function

26

By Image Observation By Experimentation By modeling

Example of turbulence model

27

28

Image blur due to motion

G&W Equ. 5.6-11

Image Restoration by Inverse filter

29

Inverse filtering example

30

Inverse filtering example

31

Input image

Inverse filtering under noise

32

Pseudo-inverse filtering

33

Back to the original problem

34

Wiener (mini mean square error) filter

35

Wiener filter

36

SNR and MSE measurement

37

Signal-to-noise ratio:

Mean square error:

Observations about Wiener filter

38

39

Wiener Filter example

40

Wiener filter example

41

Wiener filter example

42

Wiener filter: when does it not work

43

Improve Wiener filters

44

45

Example of improved wiener filter

46

Wavelet Restoration

47

Two step: Fourier-domain inverse filtering Wavelet-domain image denoising.

When the blurring function is not invertible, the algorithm is not applicable.

Wavelet-based deconvolution technique for ill-conditioned systems

48

Two step: modifying the Wiener filter

Wavelet-domain image denoising.

49

Blurred Lena ImagePSNR = 23.2993, MSE = 304.1938

Restored Lena Image (WaRD) PSNR = 19.5115, MSE = 727.7

Restored lenaImage (Wavelet)PSNR = 16.8552, MSE = 1341.4

Restored Lena Image (Subband) PSNR = 20.1223, MSE = 632.2

Example of Wavelet Restoration

Power Spectrum Equalization (PSE) We assumed that we knew the blurring

function h, what should we do if they were unknown? Wiener Filter

Power Spectrum Equalization -- restore the power spectrum of the degraded image

50

Example of PSE

51

Lenna after Blurring Plus Noise, Mean Squared Error = 1.0660e+05

Lenna restored using Wiener Filter, Mean Squared Error =

123.2

Lenna restored using PSE, Mean Squared Error = 419.5

Blind deconvolution

52

Restore image without knowledge of our blurring function

Estimate H (Jain approach)

Wiener filter

http://www.owlnet.rice.edu/~elec539/Projects99/BACH/proj2/blind/bd.html

Example of Blind deconvolution

53

Lenna after Blurring Plus NoiseMean Squared Error = 1.0660e+05

Lenna restored using Jain ApproachMean Squared Error = 283.1

Outline What is image restoration

Scope, history and applications A model for (linear) image degradation

Restoration from noise Different types of noise Examples of restoration operations

Restoration from linear degradation Inverse and pseudo-inverse filtering Wiener filters Wavelet Restoration Blind de-convolution

Geometric distortion and its corrections

Geometric distortions

55

Geometric/spatial distortion examples

56

Recovery from geometric distortion

57

Recovery from geometric distortion

58

Rahul Swaminathan, Shree K. Nayar: Nonmetric Calibration of Wide-Angle Lenses and Polycameras. IEEE Trans. Pattern Anal. Mach. Intell. 22(10): 1172-1178 (2000)

Omnicamera

59

http://www.columbia.edu/cu/record/23/20a/omnicamera.html

Estimating distortions

60

Summary

61

A image degradation model Restoration from noise Restoration from linear degradation Inverse and pseudo-inverse filters, Wiener filter, constrained

least squares, wavelet restoration, Geometric distortions Readings

G&W Chapter 5.1 – 5.10. M. R. Banham and A. K. Katsaggelos "Digital Image

Restoration“, IEEE Signal Processing Magazine, vol. 14, no. 2, Mar. 1997, pp.24-41

Who said distortion is a bad thing?

62

Suggestions for Final Project -- 1 Each student or team works on different projects Send me by email about the TITLE and a brief

description if you will choose the final project by yourself before 10/31/2018 (must be image processing related).

Two students can work on same project Team work Need to show the contributions for each student Send me by email for the partner you choose before

10/31. Otherwise, I’ll assume you prefer to do the final project by yourself.

63

Suggestions for Final Project -- 2 For the project:

Code Report Presentation

64

HW2 Presentation

65