HQVcis580/Spring2017/Lectures/...3URMHFWLRQ RI SLQKROH ZLPJ KLPJ SS [ \ ,PDJH SULQFLSDO SRLQW [XX S Z Z FFGLPJ FFG LPJ \ YY S K K FFGLPJ FFG LPJ ' 3RLQW 3URMHFWLRQ 3L[HO 6SDFH &&

$Page 1: HQVcis580/Spring2017/Lectures/...3URMHFWLRQ RI SLQKROH ZLPJ KLPJ SS [ \ ,PDJH SULQFLSDO SRLQW [XX S Z Z FFGLPJ FFG LPJ \ YY S K K FFGLPJ FFG LPJ ' 3RLQW 3URMHFWLRQ 3L[HO 6SDFH &&$
3D object

Lens

CCD sensor

Slide by HyunSoo Park

3D Point Projection (Metric Space)3D point

mf

( , , )X Y Z

: Focal length in meter

Pinhole-fm

ccd X

u fZm

ccd Y

v fZm

ccd ccd X Y

u v f fZ Zm m( , ) ( , )

2D projection onto CCD plane

( , , )X Y Z

Projection plane

ccd ccdu v( , )


mf

( , , )X Y Z

: Focal length in meter

Pinhole=

center of projection-fm

ccd X

u fZm

ccd Y

v fZm

ccd ccd X Y

u v f fZ Zm m( , ) ( , )


( , , )X Y Z

Projection plane

( , )u vccd ccd

fm

ccd ccdu v( , )

http://www.joshuanava.biz/perspective/in-other-words-the-observer-simply-points-in-the-same-direction-as-the-lines-in-order-to-find-their-vanishing-point.html

recall we can paint a picture by

1) placing an image plane in front of us,2) draw a ray from our eye to a point in the image plane, to form a ray3) intersect that ray with the 3D world, and mark the intersection point,4) take the color of that 3D point, and paint it onto the 2D pixel

The ray itself, extended infinitely long, can be also thought of representative of a point in infinity of all 3D lines in direction, and we will see that would allow us to recover the orientation of the camera.

Locating Center of Projection

we will demonstrate how to find COP of your cell phone camera.

1) draw a set of radiating lines on a paper2) place the camera vertical to the paper

the radiating lines on paper should look like this, in the 3d world


1) using a live displaying mode, we can see these radiating lines as transformed in to the camera image plane. what does it look like?2) make sure we keep the camera vertical and move it forward and backward, until the imaged lines look all vertical 3) when we see this we can mark the front of the camera.



-- tracing the radiating lines to their intersection, we obtained a rough estimated of the physical center of projection of your camera

-- we now have an idea of where is the optical center of the camera, how about the focal length itself

Focal Length

Digital SLR cameras have focal length at order of 35mm, or 50 mm. Our cell phone, because it is very thin, has very short focal lengths, on order of a few mm.

Nikon’s 1200-1700mm f/5.6-8P lens.Professional photographer has lens with much longer focal lengths, on order of 1-2meters

Zooming, the process of changing focal length, can amplifying a scene. We can do the same with a cell phone, just we have to walk a lot to get those images.

Large Focal Length compresses depth

© 1995-2005 Michael Reichmann

400 mm 200 mm 100 mm 50 mm 28 mm 17 mm

zooming does not only create a sense of magnification, it can also create a sensation of moving,shown here, from 400mm, to 17mm, we can see the size of background objects is much reduced


mf

( , , )X Y Z

: Focal length in meterccd

Xu f

Zm

ccd Y

v fZm

ccd ccd X Y

u v f fZ Zm m( , ) ( , )


( , , )X Y Z

Projection plane

( , )u vccd ccd

fm

Center of Projection


mf

( , , )X Y Z

: Focal length in meterccd

Xu f

Zm

ccd Y

v fZm

ccd ccd X Y

u v f fZ Zm m( , ) ( , )


( , , )X Y Z

Projection plane

( , )u vccd ccd

fm

( , )u vimg img

img img ccd ccdu v u v( , ) ( , ) ?

MetricPixel

3D Point Projection (Pixel Space)

CCD sensor (mm) Image (pixel)



u vccd ccd( , )

u vimg img( , )



u vccd ccd( , )

ccdw

ccdh

u vimg img( , )





u vccd ccd( , )

ccdw

ccdh

u vimg img( , )



u vccd ccd( , )

ccdw

ccdh

(0,0)

Projection of pinhole

u vimg img( , )



u vccd ccd( , )

ccdw

ccdh

(0,0)


imgw

imgh

u vimg img( , )



u vccd ccd( , )

ccdw

ccdh

(0,0)


(0,0) imgw

imgh

x yp p( , ): Image principal point

u vimg img( , )



u vccd ccd( , )

u vimg img( , )

ccdw

ccdh

(0,0)


(0,0) imgw

imgh


xu pu

w wimgccd

ccd img



u vccd ccd( , )

u vimg img( , )

ccdw

ccdh

(0,0)


(0,0) imgw

imgh


xu pu

w wimgccd

ccd img

yv pv

h himgccd

ccd img



u vccd ccd( , )

u vimg img( , )

ccdw

ccdh

(0,0)


(0,0) imgw

imgh


xu pu

w wimgccd

ccd img

yv pv

h himgccd

ccd img

x

wu u p

wimg

img ccdccd

y

hv v p

himg

img ccdccd


O

( , )u vccd ccd

mf

Projection plane ( , , )X Y Z

Z

ccd ccd m m( , ) ( , )X Y

u v f fZ Z


u vccd ccd( , )

u vimg img( , )

ccdw

ccdh

(0,0)

(0,0) imgw

imghx yp p( , )

: Metric projection


O

( , )u vccd ccd

mf


Z


u v f fZ Z


u vccd ccd( , )

u vimg img( , )

ccdw

ccdh

(0,0)

(0,0) imgw

imghx yp p( , )

: Metric projection

x x

w w Xu u p f p

w w Zimg img

img ccd mccd ccd

y y

h h Yv v p f p

h h Zimg img

img ccd mccd ccd

Pixel projection


O

( , )u vccd ccd

mf


Z


u v f fZ Z


u vccd ccd( , )

u vimg img( , )

ccdw

ccdh

(0,0)

(0,0) imgw

imghx yp p( , )

: Metric projection

x x

w w Xu u p f p

w w Zimg img

img ccd mccd ccd

y y

h h Yv v p f p

h h Zimg img

img ccd mccd ccd

Pixel projection


O


Z


u v f fZ Z

x x

w w Xu u p f p

w w Zimg img

img ccd mccd ccd

y y

h h Yv v p f p

h h Zimg img

img ccd mccd ccd


u vccd ccd( , )

u vimg img( , )

ccdw

ccdh

(0,0)

(0,0) imgw

imghx yp p( , )

: Metric projection

Focal length in pixel

Pixel projection

img imgu v( , )

f


O


Z


u v f fZ Z

x x

w w Xu u p f p

w w Zimg img

img ccd mccd ccd

y y

h h Yv v p f p

h h Zimg img

img ccd mccd ccd


u vccd ccd( , )

u vimg img( , )

ccdw

ccdh

(0,0)

(0,0) imgw

imghx yp p( , )

: Metric projection


xf

yf

imgm

ccd

x

wf f

wimg

mccd

y

hf f

hwhere

Pixel projection

img imgu v( , )

f


O


Z


u v f fZ Z

x x

w w Xu u p f p

w w Zimg img

img ccd mccd ccd

y y

h h Yv v p f p

h h Zimg img

img ccd mccd ccd


u vccd ccd( , )

u vimg img( , )

ccdw

ccdh

(0,0)

(0,0) imgw

imghx yp p( , )

: Metric projection


f

f

img imgm m

ccd ccd

w h

f f fw h

where

Pixel projection

img img

ccd ccd

w h

w hif

img imgu v( , )

f


O


Z

img x

Xu f p

Z

img y

Yv f p

Z


u vccd ccd( , )

u vimg img( , )

ccdw

ccdh

(0,0)

(0,0) imgw

imghx yp p( , )

img imgu v( , )

f


O


Z

img x

Xu f p

Z

img y

Yv f p

Z


u vccd ccd( , )

u vimg img( , )

ccdw

ccdh

(0,0)

(0,0) imgw

imghx yp p( , )

img imgu v( , )

f

img xZu fX p Z

img yZv fY p Z


O


Z

img x

Xu f p

Z

img y

Yv f p

Z


u vccd ccd( , )

u vimg img( , )

ccdw

ccdh

(0,0)

(0,0) imgw

imghx yp p( , )

img imgu v( , )

f

img xZu fX p Z

img yZv fY p Z

img

img

11

x

y

u f p X

Z v f p Y

Z


O


Z

img x

Xu f p

Z

img y

Yv f p

Z


u vccd ccd( , )

u vimg img( , )

ccdw

ccdh

(0,0)

(0,0) imgw

imghx yp p( , )

img imgu v( , )

f

img xZu fX p Z

img yZv fY p Z

img

img

11

x

y

u f p X

v f p Y

Z

Metric spacePixel space


O


Z

img x

Xu f p

Z

img y

Yv f p

Z


u vccd ccd( , )

u vimg img( , )

ccdw

ccdh

(0,0)

(0,0) imgw

imghx yp p( , )

img imgu v( , )

f

img xZu fX p Z

img yZv fY p Z

img

img

11

x

y

u f p X

v f p Y

Z


+Camera intrinsic parameter: metric space to pixel space


O


Z

img x

Xu f p

Z

img y

Yv f p

Z


u vccd ccd( , )

u vimg img( , )

ccdw

ccdh

(0,0)

(0,0) imgw

imghx yp p( , )

img imgu v( , )

f

img xZu fX p Z

img yZv fY p Z

img

img

11

x

y

u f p X

v f p Y

Z

K

+Camera intrinsic parameter: metric space to pixel space


Camera Intrinsic Parameter

O

Projection plane

Z

img

img

11

x

y

u f p X

v f p Y

Z

K

Metric space

( , , )X Y Z


O

Projection plane

Z

img

img

11

x

y

u f p X

v f p Y

Z

K

Metric space

( , , )X Y Z

f


O

Projection plane

Z

img

img

11

x

y

u f p X

v f p Y

Z

K

Metric space

( , , )X Y Z

f

img imgu v( , )

Pixel space

2D Inverse Projection

O

Projection plane

img

img

11

x

y

u f p X

v f p Y

Z

K

Metric space

f

img imgu v( , )

Pixel space

u X

v Y

Z

K

img

-1img

1


O

Projection plane

img

img

11

x

y

u f p X

v f p Y

Z

K

Metric space

f

img imgu v( , )

Pixel space

u X

v Y

Z

K

img

-1img

1

u

vK

img

-1img

1

2D point == 3D ray

3D ray


O

Projection plane

img

img

11

x

y

u f p X

v f p Y

Z

K

Metric space

f

img imgu v( , )

Pixel space

u X

v Y

Z

K

img

-1img

1

u

vK

img

-1img

1

( , , )X Y Z

2D point == 3D ray

3D ray


O

Projection plane

img

img

11

x

y

u f p X

v f p Y

Z

K

Metric space

f

img imgu v( , )

Pixel space

u X

v Y

Z

K

img

-1img

1

u

vK

img

-1img

1

( , , )X Y Z

2D point == 3D ray

3D ray

The 3D point must lie in the 3D ray passing through the origin and 2D image point.

Exercise

324 m

1280 pix

960 pix

fm ?

What f to make the height of Eifel tower appear 960 pixel distance?

21.8 mm

ccd size

1500 m

Exercise

324 m

1280 pix

960 pix

fm ?


m

hY Yy f f

Z h Zimg

imgccd

21.8 mm

ccd size

1500 m

Exercise

324 m

1280 pix

960 pix

fm ?


m

hY Yy f f

Z h Zimg

imgccd

21.8 mm

ccd size

1280 324960 0.0757m

0.02181500 m mf f

1500 m

Focal Length

Exercise

324 m

Z ?

1280 pix

960 pix

What Z to make the height of Eifel tower appear 960 pixel distance?

m

hY Yy f f

Z h Zimg

imgccd

21.8 mm

ccd size

fm = 50 mm

Exercise

324 m

Z ?

1280 pix

960 pix

What Z to make the height of Eifel tower appear 960 pixel distance?

m

hY Yy f f

Z h Zimg

imgccd

21.8 mm

ccd size

ZZ

1280 324960 0.05 990.826m

0.0218

fm = 50 mm

Exercise

324 m

1500

What Zp to make the height of Eifel tower appear twice of the person?

fm = 50 mm

Z p

1.7 m

Exercise

324 m

1500


Y

h fZe

fm = 50 mm

Z p

h e

h p

1.7 m Y

h fZ

pp

p

h

h ep 2

s.t.

Exercise

324 m

1500


Y

h fZe

Y Y

f fZ Z

p

p 2

fm = 50 mm

Z p

h e

h p

1.7 m Y

h fZ

pp

p

h

h ep 2

s.t.

p

1.72 1500 15.74m

324 Z

Where Was I?

Z ?

1.7 m

Sensor size Sensor spec

img

ccd

m

hY Yy f f

Z h Z

Where Was I?

Z ?

1.7 m

Sensor size Sensor spec

img

ccd

m

hY Yy f f

Z h Z

Where Was I?

Circa 1984

Where Was I?

Circa 1984

250 pix

670 pix

Where Was I?

Circa 1984

324 m

fm = 50 mm

0.9 m

250 pix

670 pix

Where Was I?

Circa 1984

324 m

Z 1

0.9 m

img img1 1 11 1

1 ccd 1 ccd 1

1280 0.90.05 8.03m

0.0218 250 m m

h hY Y Yy f f Z f

Z h Z h y

250 pix

670 pix

fm = 50 mm

Where Was I?

Circa 1984

324 m

fm = 38 mm

Z 1

0.9 m

img img1 1 11 1

1 ccd 1 ccd 1

1280 0.90.05 8.03m

0.0218 250 m m

h hY Y Yy f f Z f

Z h Z h y

250 pix

670 pix

Where Was I?

Circa 1984

324 m

Z 1

0.9 m

img img1 1 11 1

1 ccd 1 ccd 1

1280 0.90.05 8.03m

0.0218 250 m m

h hY Y Yy f f Z f

Z h Z h y

250 pix

670 pix

fm = 50 mm

Where Was I?

Circa 1984

324 m

Z 1

0.9 m

Z 2

250 pix

670 pix

m m

h hY Y Yy f f Z f

Z h Z h yimg img2 2 2

2 22 ccd 2 ccd 2

1280 3240.05 1079m

0.0218 670

img img1 1 11 1

1 ccd 1 ccd 1

1280 0.90.05 8.03m

0.0218 250 m m

h hY Y Yy f f Z f

Z h Z h y

fm = 50 mm

Where Was I?

Circa 1984

250 pix

670 pix

m m

h hY Y Yy f f Z f

Z h Z h yimg img2 2 2

2 22 ccd 2 ccd 2

1280 3240.05 1079m

0.0218 670

200m

400m

600m

800m

1000m

800 m

Where Was I?

Focal Length

f d

Strong perspective

Focal Length

f d

Weak perspective

Focal Length

f

Orhographic

d

Focal Length



Dolly Zoom (Vertigo Effect)

Dolly Zoom (Vertigo Effect)

Dolly Zoom

Given focal length (fm=100mm),what Z100 to make the height of the person remain the same as fm=50mm?

f50 = 50 mm

h p H p

Z 50 =157.41m

Dolly Zoom


Y

h fZ50 50

50

h p

Y

h fZ100 100

100

h h100 50s.t.

H p

Z 50 =157.41m

f50 = 50 mm

Dolly Zoom


Y

h fZ50 50

50

f

Z Zf100

100 5050

f100 = 100 mm

Z 50 =157.41m

h p

Y

h fZ100 100

100

h h100 50s.t.

Z100

100157.41 314.8m

50

H p

Z 100 ?

Where am I with Dolly Zoom?

1 400pixh

2 120pixh


1 400pixh

2 120pixh

How far I need to step back with zoom factor x2?

How will h2 change?


A

B

C

camera

Image plane

Z

1 400pixh

2 120pixh

Top view

1 4mH

2 6mH

How will h2 change?



1 400pixh

2 120pixh

A

B

C

camera

Image plane

Z

f = 2md1d

Top view


1 4mH

2 6mH

Unknowns: f, d1How will h2 change?


1 400pixh

2 120pixh

A

B

C

camera

Image plane

Z

2f 1d

Top view

1 4mH

2 6mH

Unknowns: f, d1,

d

d

= 2md

How will h2 change?



1 400pixh

2 120pixh

A

B

C

camera

Image plane

Z

2f 1d

Top view

1 4mH

2 6mH

d

1

11 h f

H

d

Equations:

= 2md

Unknowns: f, d1, dHow will h2 change?



1 400pixh

2 120pixh

A

B

C

camera

Image plane

Z

2f 1d

Top view

1 4mH

2 6mH

d

1

11 h f

H

d

Equations:

1

11 2

+

H

h fd d

= 2md

Unknowns: f, d1, d How will h2 change?



1 400pixh

2 120pixh

A

B

C

camera

Image plane

Z

2f 1d

Top view

1 4mH

2 6mH

d

1

11 h f

H

d

Equations:1

11 2

+

H

h fd d 1 d d

= 2md




1 400pixh

2 120pixh

A

B

C

camera

Image plane

Z

2f 1d

Top view


1 4mH

2 6mH

d

1

11 h f

H

d

Equations:1

11 2

+

H

h fd d

2

12 +

Hh f

d d

= 2md


1 d d

A

B

C

camera

Image plane

Z

2f 1d

Top view

1 4mH

2 6mH

d

1

11 h f

H

d

Equations:1

11 2

+

H

h fd d 1 d d

21

1

1 2

1= 2.5m

1

d

h HhH

d

= 2md

Unknowns: f, d1, d

2

12 +

Hh f

d d

1 400pixh

2 120pixh

2 120pixh 2 200pix h

2.5m d

A

B

C

camera

Image plane

Z

2f 1d

Top view

1 4mH

2 6mH

d

1

11 h f

H

d

Equations:1

11 2

+

H

h fd d 1 d d

21

1

1 2

1= 2.5m

1

d

h HhH

d

= 2md

Unknowns: f, d1, d

2

12 +

Hh f

d d

1 400pixh

2 120pixh

2 120pixh

2.5m d 250pixf

1 400pixh

2 120pixh

A

B

C

camera

Image plane

Z

2f 1d

Top view


1 4mH

2 6mH

d

1

11 h f

H

d

Equations:1

11 2

+

H

h fd d

2

12 +

Hh f

d d

= 2md


2

12 2 200pix

+ + f

d d

Hh

d

250pixf1 2.5m d d

1 400pixh

2 120pixh

A

B

C

camera

Image plane

Z

2f 1d

Top view


1 4mH

2 6mH

d

1

11 h f

H

d

Equations:1

11 2

+

H

h fd d

2

12 +

Hh f

d d2

12 2 429pix

+ + f

d d

Hh

d

= 2md


250pixf1 2.5m d d

Documents

HQVcis580/Spring2017/Lectures/...3URMHFWLRQ RI SLQKROH ZLPJ KLPJ SS [ \ ,PDJH SULQFLSDO SRLQW [XX S Z Z FFGLPJ FFG LPJ \ YY S K K FFGLPJ FFG LPJ ' 3RLQW 3URMHFWLRQ 3L[HO 6SDFH &&