Image Processing and Analysis (ImagePandA) 5 – Image Restoration and Reconstruction Christoph...

Image Processing and Analysis (ImagePandA)

5 – Image Restoration and Reconstruction

Christoph Lampert / Chris Wojtan

Based on slides by Selim Aksoy, Bilkent University

Image interpolation

Image interpolation

Image interpolation

Nearest neighbor interpolation

Image interpolation

Image interpolation

Nearest neighbor interpolationFirst, find the x and y values of the nearest pixels

int[] px = new int[4];int[] py = new int[4];px[0] = Math.floor(x); py[0] = Math.floor(y);px[1] = Math.ceil(x); py[1] = Math.floor(y);px[2] = Math.floor(x); py[2] = Math.ceil(y);px[3] = Math.ceil(x); py[3] = Math.ceil(y);

Nearest neighbor interpolationNext, find the nearest pixel

int nx, ny; // x,y coords of nearest neighbordouble minDist = Double.MAX_VALUE;for(int i=0; i<4; i++) { double dist = Math.abs(x-px[i])+Math.abs(y-py[i]); if(dist<minDist) { minDist = dist; nx=px[i]; ny=py[i]; }}

Finally, assign the value of the nearest pixel

pixelVal = image.get(nx,ny);

Nearest neighbor interpolation

Piecewise linear interpolation

Piecewise Linear interpolation

0 1

Piecewise linear interpolation

Piecewise cubic interpolation

Piecewise Cubic (Hermite) interpolation

2D interpolation

Nearest neighbor Bi-linear Bi-cubic

17

Resizing images

How can we generate a half-sized version of a large image?

Adapted from Steve Seitz, U of Washington

18

Resizing images

Throw away every other row and column to create a 1/2 size image (also called sub-sampling).

1/4

1/8

Adapted from Steve Seitz, U of Washington

19

Resizing images

Does this look nice?1/4 (2x zoom)

1/8 (4x zoom)

1/2

Adapted from Steve Seitz, U of Washington

20

Sampling and aliasing

Adapted from Steve Seitz, U of Washington

Sampling and aliasing

http://www.youtube.com/watch?v=ckmtG8CZjDA

22

Sampling and aliasing

Errors appear if we do not sample properly. Common phenomenon:

High spatial frequency components of the image appear as low spatial frequency components.

Examples: Wagon wheels rolling the wrong way in movies. Checkerboards misrepresented in ray tracing. Striped shirts look funny on color television.

23

Resizing images

Throw away every other row and column…… regular sampling of a high-frequency function!

1/4

1/8

Adapted from Steve Seitz, U of Washington

24

Resizing images

Does this look nice?1/4 (2x zoom)

1/8 (4x zoom)

1/2

Adapted from Steve Seitz, U of Washington

ALIASING

25

Resizing images Regular sampling of a high-frequency image causes

aliasing! Solution: smooth the image (remove high

frequencies) first!

Gaussian 1/4

Gaussian 1/8

Gaussian 1/2

Adapted from Steve Seitz, U of Washington

26

Resizing images

Gaussian 1/4 (2x zoom)

Gaussian 1/8 (4x zoom)

Gaussian 1/2

Adapted from Steve Seitz, U of Washington

28

Gaussian pyramids

Adapted from Michael Black, Brown University

29

Gaussian pyramids

Adapted from Michael Black, Brown University

De-noising

30

Removing Noise

31

Inverting the Degradation Function

32

Inverting the Degradation Function

33

http://www.sweaglesw.com/cs448/

Original motion-blurred imagePSF Clean image of The Beehive Nebula

Inverting the Degradation Function

34

Inverting the Degradation Function

35

Texture Synthesis

36

Incrementally growing pixels Given an incomplete neighborhood Find similar neighborhoods from the example, copy pixels

?

Texture Synthesis

37

Incrementally growing pixels Given an incomplete neighborhood Find similar neighborhoods from the example, copy pixels

Texture Synthesis

38

Incrementally growing pixels Given an incomplete neighborhood Find similar neighborhoods from the example, copy pixels

Image Quilting Cut up the example into patches (not just pixels) Choose patches which line up the best Stitch the patches together along optimal seems

Texture Synthesis

39

Incrementally growing pixels Given an incomplete neighborhood Find similar neighborhoods from the example, copy pixels

Image Quilting Cut up the example into patches (not just pixels) Choose patches which line up the best Stitch the patches together along optimal seems

Can use this to fill holes in an image Instead of some example, use the image itself http://www.youtube.com/watch?v=NH0aEp1oDOI

Super-resolution

40

How do we increase image size?

Interpolation? Need to fill in missing details

EMBIGGEN

INTERPO

LATION

Super-resolution

41

How do we increase image size? Interpolation?

Need to fill in missing details Combine multiple images at sub-pixel off-sets Synthesize details from examples

Steal details from other images Copy details from same image (at different scales)

http://www.wisdom.weizmann.ac.il/~vision/SingleImageSR.html