Sterio Vision

8/18/2019 Sterio Vision

1/45

ISAN-DSP GROUP

IMAGE REGISTRATION


2/45

ISAN-DSP GROUP

Course Syllabus

Date and Time : MW 11.00-12.00 EN 4510, LAB1 TU17-20, LAB2 TH17-20

Assessments:

Attendane ! H"#e$"%& 5'

La( and H"#e$"%& )5'

M*dte%# )0'

+*na )0'

Grading:5-100' A, 75-5' B, 70-75' B, /5-70' , /0-/5' ,

55-/0' , 50-55' , 0-50' +

References:

1. Raae . G"n3ae3 and R*a%d E. W""d, 6**ta I#ae 8%"e*n9,

Add*"n Wee, 1;;2.

2. An* nda#enta " **ta I#ae 8%"e*n9,

8%ent*e-Ha, In., 1;;.

). W**a#


3/45

ISAN-DSP GROUP

Course Outline

1. Int%"d>t*"n

2. **ta I#ae +>nda#enta

). I#ae T%an"%#

4. I#ae Enane#ent

5. I#ae Se#entat*"n

/. I#ae "#?%e*"n

7. I#ae M"%?""


4/45

ISAN-DSP GROUP

a?te% 1

Int%"d>t*"n t" I#ae 8%"e*n


5/45

ISAN-DSP GROUP

What is Digital Image Processing ?

8%"e*n " a #>t*d*#en*"na ?*t>%e ( a d**ta "#?>te%

การประมวลผลสญญาณรปภาพโดยใชดจตอลคอมพวเตอร

Why we need Digital Image Processing ?

1. เพ อ!"#ก$ละจดเก%ภาพ2. เพ อปรปร&'ภาพใ(ด)*# +!โดยใช

กระว!การ"า'คณต,าสตร). เพ อช-วยใ!การวเคราะ(รปภาพ4. เพ อส'เคราะ(ภาพ5. เพ อสรา'ระการมอ'เ(%!ใ(ก

คอมพวเตอร


6/45


7/45

ISAN-DSP GROUP

Visal Perce!tion" #man $ye

8*t>%e %"# M*%""t Ena%ta 2000


8/45

ISAN-DSP GROUP

Chapter 2: Digital Image Fundamentals


9/45

ISAN-DSP GROUP

Visal Perce!tion" #man $ye %cont&'

1. Te lens "nta*n /0-70' $ate%, /' " at.

2. Te iris d*a?%a# "nt%" a#">nt " *t tat ente% te ee.

3. Light receptors *n te retina

- A(">t /-7 #**"n cones "% (%*t *t C**"n aed photopic - en*t " "ne * a(">t 150,000 ee#ent##2.

- "ne *nC"Ce *n ""% C**"n.

- "ne a%e "nent%ated *n fovea a(">t 1.5D1.5 ##2.

- A(">t 75-150 #**"n rods "% d*# *t C**"n aed scotopic

- R"d a%e en*t*Ce t" "$ eCe " *t and a%e n"t *nC"Ced ""% C**"n.

4. Blind spot * te %e*"n " e#e%ene " te "?t* ne%Ce %"# te ee.


10/45

ISAN-DSP GROUP



11/45

ISAN-DSP GROUP

Image (ormation in #man $ye


12/45

ISAN-DSP GROUP



13/45

ISAN-DSP GROUP

)rightness Ada!tation o* #man $ye

Position

I n t e n s i t y

Inten*t*e " >%%">nd*n ?"*nt eet ?e%e*Ced (%*tne at ea ?"*nt.

In t* *#ae, ede (et$een (a% a??ea% (%*te% "n te %*t *de and da%&e%

On te et *de.


14/45

ISAN-DSP GROUP

In a%ea A, (%*tne ?e%e*Ced * da%&e% $*e *n a%ea B *

(%*te%. T* ?en"#en"n * aed Mach Band Effect .

)rightness Ada!tation o* #man $ye %cont&'

8"*t*"n

I n t e n 5 * t :

AB


15/45

ISAN-DSP GROUP

)rightness Ada!tation o* #man $ye %cont&'

Simultaneous contrast . A #a F>a%e aCe eDat te a#e *nten*t

(>t te a??ea% ?%"%e*Ce da%&e% a (a&%">nd (e"#e *te%.


16/45

ISAN-DSP GROUP

Imaging Geometry" Pers!ecti+e ,rans*ormation

Len ente%

x,X y,Y

x, y

z,Z

X,Y,Z @ $"%d ""%d*nate

λ

x,y,z @ a#e%a ""%d*nate te#

X,Y,Z

λ @ "a ent

I#ae ?ane

λ λ

λ λ

−−=

−−=

Z

Y y

Z

X x

EF. 1.1


17/45

ISAN-DSP GROUP

−

−

−

=

=

Z Z

Z

Y Z

X

z

y

x

c

λ λ

λ

λ λ

λ

Imaging Geometry" Pers!ecti+e ,rans*ormation %cont&'

>et*"n: H"$ an $e ?%"et te %ea $"%d "(et at ,J,K "nt"te *#ae ?ane > a ?"t"%a?* *#

An$e%: Reat*"n (et$een a#e%a ""%d*nate x,y,z and $"%d ""%d*nate

X,Y,Z a%e *Cen (

S*ne "n te *#ae ?ane z * a$a 3e%", z=0, $e "n*de% "n D,

$*e 3 * neeted.

EF. 1.2


18/45

ISAN-DSP GROUP


EF>at*"n 1.2 * n"t *nea% (ea>e " K *n te d*C*de% " $e

*nt%"d>e te homogeneous coordinate t" "Ce t* ?%"(e#.

=

Z

Y

X

wa%te*an

""%d*nate

=

k kZ

kY

kX

whH"#"ene">

""%d*nate

k @ n"n3e%" "ntant

T" "nCe%t %"# te "#"ene"> ""%d*nate wh t" te a%te*an

""%d*nate w, $e d*C*de te *%t ) "#?"nent " wh ( te ">%t

"#?"nent.


19/45

ISAN-DSP GROUP

−−

=

−

==

λ

λ

λ

100

0100

0010

0001

1 Z k kZ

kY

kX

k

kZ

kY

kX

Pwc hh


−

=

100

0100

0010

0001

1λ

P

Te ?e%?et*Ce t%an"%#at*"n #at%*D "% te "#"ene"> ""%d*nate:

8e%?et*Ce t%an"%#at*"n (e"#e:

EF. 1.)


20/45

ISAN-DSP GROUP

−−

=

λ

λ Z k kZ

kY kX

c h


=

−

−

−

=

−−⋅

−−⋅

−−⋅

=

z

y

x

Z

Z Z

Y Z

X

Z k kZ

Z k kY

Z k kX

c

λ

λ

λ

λ

λ

λ

λ

λ

λ

λ

λ

λ

+%"# "#"ene">

""%d*nate

We et a#e%a ""%d*nate *n te *#ae ?ane:


21/45

ISAN-DSP GROUP

Imaging Geometry" In+erse Pers!ecti+e ,rans*ormation

=−

100

01000010

0001

1

1

λ

P hh c P w

1−= $e%e EF. 1.4


22/45

ISAN-DSP GROUP

+"% an *#ae ?"*nt x0 ,y0, *ne "n

te *#ae ?ane z @0, $e aCe

=

k

kykx

ch0

0

0

In+erse Pers!ecti+e ,rans*ormation %cont&'

LLL

0

"%0 0

0

0

0

1 →

=

== − y

x

w

k

ky

kx

c P whh

We et te $"%d ""%d*nate :

Since the perspective transformation maps 3-D coorinates to !-D "oorinates,

we cannot #et the inverse transform $n%ess we have aitiona% information&


23/45

ISAN-DSP GROUP

T" *nd te ">t*"n, et

=

k

kz

ky

kx

ch0

0


We et

+=

+

=

+=

=

+

== −

z

z Z

z

yY

z

x X

w

z k kz

ky

kx

c P w hh

λ

λ λ

λ λ

λ

λ

λ

0

0

0

0

1 "%

BA

EF. 1.5


24/45

ISAN-DSP GROUP

0

0

Z y

Y

Z x X

−=

−=

λ λ

λ λ


+%"# EF. 1.5,

Z

Z z

−=

λ

λ We et EF. 1./

S>(t*t>t*n EF. 1./ *nt" EF.1.5, $e et

EF. 1.7

EF>at*"n 1.7 "$ tat *nCe%e ?e%?et*Ce t%an"%#at*"n %eF>*%e

*n"%#at*"n " at eat "ne "#?"nent " te $"%d ""%d*nate " te ?"*nt.


25/45

ISAN-DSP GROUP


BA and BA00

Z

y

Y Z

x

X −=−= λ

λ λ

λ Tee eF>at*"n:"$ tat ?"*nt "n L*ne ' *n te $"%d ""%d*nate

#a? t" a *ne ?"*nt *n te *#ae ?ane.

Len ente%

x,X y,Y

z,Z

x, y

λ

I#ae ?ane

X ! ,Y ! ,Z !

X 3 ,Y 3 ,Z 3

8"*nt X ( ,Y ( ,Z ( , X ! ,Y ! ,Z !, and X 3 ,Y 3 ,Z 3 #a? t"

8"*nt x,y *n te *#ae ?ane.

X ( ,Y ( ,Z (

Line L


26/45


27/45

ISAN-DSP GROUP

Stereo Imaging" #ow we get de!th in*ormation *rom

eyes %cont&'

Problem: $e &n"$ a#e%a ""%d*nate " te "(et "n et and%*t *#ae ?ane x(, y( and x!, y! and $ant t" "$ a% %"# te

a#e%a te "(et * "ated.

Note: $en -aD* * ?a%ae t" te %">nd, $e aCe y( = y!

x!, y!

x(, y(

λ

λ ) w

Z

I#ae 2

I#ae 1

8ane "

"ntant Z

X

O%**n " $"%d

""%d*natete#


28/45

ISAN-DSP GROUP

1. +%"# te *nCe%e ?e%?et*Ce t%an"%#, $e "#?>te X ( and X !:

BA and BA 22

211

1 Z x

X Z x

X −=−= λ λ

λ λ

). S*ne et and %*t ene a%e e?a%ated ( d*tane ) , $e aCe

) X X += 12

2. Z ( and Z ! #>t (e eF>a, $e et 21 Z Z =

Stereo Imaging" #ow we get de!th in*ormation *rom

eyes %cont&'

4. +%"# 1, 2 and ), $e et

BA and BA 211

1 Z x ) X Z x X −=+−= λ λ

λ λ

S"C*n K *ed

12 x x

) Z

−−= λ

λ


29/45


30/45

ISAN-DSP GROUP

Imaging Geometry " A**ine ,rans*ormations

1. Translation

2. Scaling

3. otating


31/45

ISAN-DSP GROUP

Image Geometry" ,ranslation o* O./ect

Y

Z

X

*?ae te "(et ( Cet"% X 0,Y 0, Z 0 $*t %e?et t"

*t "d ?"*t*"n.

=

11000

100

010

001

1

0

0

0

Z

Y

X

Z

Y

X

Z

Y

X

0

0

0

Z Z Z

Y Y Y

X X X

+=

+=

+=

X X 0,Y Y 0, Z Z 0

X ,Y , Z

A X 0 , Y

0 , Z 0 B


32/45

ISAN-DSP GROUP

Image Geometry" ,ranslation o* (rame

Y

Z

X

0,0,0

T%anate te "%**n ?"*nt " te %a#e ( X 0,Y 0, Z 0

$*t %e?et t" te "d %a#e

−

−

−

=

11000

100

010

001

1

0

0

0

Z

Y

X

Z

Y

X

Z

Y

X

0

0

0

Z Z Z

Y Y Y

X X X

−=

−=

−=

Y ∗

Z ∗

X ∗ A X 0

, Y 0 , Z

0 B

Te "(et t* ta at te a#e ?"*t*"n. On te %a#e * #"Ced.


33/45


34/45

ISAN-DSP GROUP

Image Geometry" Rotating an o./ect a.ot 0-a1is

R"tate an "(et a(">t X -aD* ( x *n a ">nte%"&$*e

d*%et*"n.

Y

Z

X= X ∗

x

−=

11000

0"*n0

0*n"0

0001

1

Z

Y

X

Z

Y

X

x x

x x

θ θ

θ θ

N"te : In t* ae te "(et * #"Ced. On y and z

a%e aned $*e x t* te a#e.


35/45

ISAN-DSP GROUP

Image Geometry" Rotating a *rame a.ot 0-a1is

R"tate te %a#e a(">t X -aD* ( x *n a ">nte%"&$*e

d*%et*"n.

Y

Z

x

−

=

110000"*n0

0*n"0

0001

1

Z

Y

X

Z

Y

X

x x

x x

θ θ

θ θ Y ∗

Z ∗

N"te : In t* ae te "(et * n"t #"Ced. Te %a#e

* %"tated *ntead.

X= X ∗


36/45

ISAN-DSP GROUP


R"tate an "(et a(">t Y -aD* ( y *n a ">nte%"&$*e

d*%et*"n.

Z

X

y

−=

11000

0"0*n

0010

0*n0"

1

Z

Y

X

Z

Y

X

y y

y y

θ θ

θ θ

N"te : In t* ae te "(et * #"Ced. On x and z

a%e aned $*e y t* te a#e.

Y= Y ∗


37/45

ISAN-DSP GROUP


R"tate te %a#e a(">t Y -aD* ( y *n a ">nte%"&$*e

d*%et*"n.

Y= Y ∗

Z

X

y


* %"tated *ntead.

Z ∗

X ∗

−

=

11000

0"0*n

0010

0*n0"

1

Z

Y

X

Z

Y

X

y y

y y

θ θ

θ θ


38/45

ISAN-DSP GROUP


R"tate an "(et a(">t Z -aD* ( z *n a ">nte%"&$*e

d*%et*"n.

Y

Z

X

z

N"te : In t* ae te "(et * #"Ced. On x and y

a%e aned $*e z t* te a#e.

−

=

11000

0100

00"*n

00*n"

1

Z

Y

X

Z

Y

X

z z

z z

θ θ

θ θ


39/45

ISAN-DSP GROUP


R"tate te %a#e a(">t Z -aD* ( z *n a ">nte%"&$*e

d*%et*"n.

Y

Z= Z ∗

X

z

Y ∗


* %"tated *ntead.

−=

11000

0100

00"*n

00*n"

1

Z

Y

X

Z

Y

X

z z

z z

θ θ

θ θ

X ∗


40/45

ISAN-DSP GROUP

X

Y

Z

W"%d ""%d*nate Ste#

Image Geometry" #ow to com!te a !oint on an image

!lane *rom the world coordinate

Pro*%em+ we (now t)e location o" t)e ob*ect

an! want to (now w)ere it will be +ro*ecte!

on t)e "ilm ,image +lane-.

y

z

x

0 a # e % a 0 " " % d * n a t e

S : 5 t e #

nswer+

1. Trans"orm t)e worl! coor!inate tot)e camera coor!inate

2. Per"orm t)e +ers+ectie trans"ormation


41/45

ISAN-DSP GROUP

Image Geometry" #ow to com!te a !oint on the image

!lane *rom the world coordinate %cont&'

Len ente%

x

y

x, y

z λ

x,y,z @ a#e%a ""%d*nate, X,Y,Z @ W"%d ""%d*nate

I#ae ?ane z @0

#e"ore using t)e +ers+ectie trans"ormation/

t)e worl! a0es X--! must coinci!e wit) t)e camera

a0es x-y-z / ,we nee! some trans"ormations-.

X,Y,Z. X

Z

Y


42/45

ISAN-DSP GROUP

Image Geometry" 4om!te the camera coordinate *rom

the world coordinate

X

Y

Z

W"%d ""%d*nate Ste#

y

z

x

0 a # e % a 0 " " % d * n a t e

S : 5 t e #

2

3

'imbal center

1. Translate by "#

Note : +er"orm on t)e "rame

2. Pan t)e camera

,rotate about ! a0is-

. Translate by 4 5 r

Ste+s "rom 'on4ale46s boo(

3. Tilt t)e camera

,rotate about X a0is-

7. Com+ute t)e

+ers+ectie Tr.

1 " #

r


43/45

ISAN-DSP GROUP

&ormula "rom 'on4ale46s boo(

hh w/ " P c ⋅⋅⋅⋅=

Pers+ectie tr.

Translate to t)e

image +lane

Center by z 5 r

Translate to t)egimbal center "#

8orl! coor!inate

Camera coor!inate


the world coordinate %cont&'

otate by

Pan , z -

an! Tilt , x -


44/45

ISAN-DSP GROUP

X

Y

Z

W"%d ""%d*nate Ste#

y

z

x

0 a # e % a 0 " " % d * n a t e

S : 5 t e #

2

3

'imbal center

1. Translate by "$

2. Pan t)e camera

3. Tilt t)e camera

Note : +er"orm on t)e "rame

'eneral case

. Twist t)e camera

,rotate about a0is-

7. Com+ute t)e

+ers+ectie Tr.

"$ 1




45/45

ISAN-DSP GROUP



'eneral case

hh w1 P c ⋅⋅⋅=

Pers+ectie tr.

Translate tot)e image

+lane center "$

8orl! coor!inate

Camera coor!inate

otate by

Pan ,

z -/Tilt , x -/

Twist , y-

Documents

Sterio Vision