Pca Classify

8/12/2019 Pca Classify

1/13

A. Geigel CPSC 320 Vision IRC/November 2004 1

Recognizing American Sign Language Letters Using

Principal Component Analysis

A. Geigel

Undergraduate Computer Science Department, Hiram College, Hiram, Ohio

Presented November 18 2004

Abstract: !"e b#sis $or #n im#ge re%ognition s&stem is 'resented t"#t %#n %l#ssi$& sign l#ng(#ge letters (sing

Prin%i'#l Com'onent An#l&sis te%"ni)(es. Strong em'"#sis is 'l#%ed ('on (nderst#nding met"od t"eor& #nd

'ro%ed(re o$ t"e s&stem. A%t(#l im'lement#tion #nd res(lts #re $orm#ll& dis%(ssed.

INTR!UCTIN

*ver t"e '#st de%#des # %onsider#ble #mo(nt o$ time #nd

e$$ort "#s been e+'ended rese#r%"ing #nd develo'ing

vision s&stems (sing Prin%i'#l Com'onent An#l&sis ,PCA-te%"ni)(es. !"ese s&stems "#ve "#d in%re#sing s(%%ess

d(e in '#rt to te%"nologi%#l #dv#n%es "i%" #$$ord )(i%er%om'(t#tion #nd more e$$i%ient met"ods $or e+tr#%ting

res(lts. PCA vision s&stems e+%el #t re%ogniing sm#ll b(t%r(%i#l di$$eren%es beteen %olle%tions o$ im#ges o$ #

single %l#ss "ile #t t"e s#me time not s(%%(mbing to

obst#%les t"#t o(ld norm#ll& "#rm ot"er vision s&stems.

!"e most not#ble rese#r%" in t"is $ield involvesre%ogniing "(m#n $#%esot"erise non #s igen$#%e

s&stems. All PCA s&stems de'end ('on m#t"em#ti%#ll&

e+'e%ted re'etitions $o(nd in simil#r im#ges in # given

ensemble 10. Per$orming eigen de%om'osition on #rel#ted #ver#ge %ov#ri#n%e m#tri+ e+'loits t"ese

red(nd#n%ies #nd (ltim#tel& 'rovides # m#t"em#ti%#l

model $rom "i%" to b#se %l#ssi$i%#tion d(ring t"ere%ognition '"#se.

!"e 'resent investig#tion emb#rs ('on #nsering "et"er

t"e s(%%ess #nd res(lts $o(nd in igen$#%e rese#r%"

'er$ormed in t"e l#te 1580s 6 #nd e#rl& 1550s 7 %#n bee+tended in # simil#r #& to re%ognie Ameri%#n Sign

#ng(#ge letters. Re%ogniing sign l#ng(#ge letters (sing

PCA %#n 'rovide # b#sis $or $(t(re rese#r%" in re%ogniing

ot"er "#nd gest(res i$ s(%" res(lts #re desired.

9(rt"ermore # s(%%ess$(l sign l#ng(#ge re%ognition

s&stem bolsters s(''ort $or t"e belie$ t"#t PCA %#n be (sed

to re%ogniesub-class di$$eren%es o$ # single %l#ss. :"ileigen$#%e s&stems de'ended gre#tl& on t"e $#%t t"#t t"e

e&es mo(t" nose #nd ot"er $#%i#l $e#t(res rem#ined

rel#tivel& st#ti% 6 in #n im#gesign l#ng(#ge letters "#ve

obvio(s di$$eren%es beteen $inger 'ositions #nd $ists"#'e. !"(s t"ere is no st#ti% %onsisten%& #mong all

im#ges in # sign l#ng(#ge ensemble.

:e s"#ll begin b& dis%(ssing t"e t"eoreti%#l #s'e%ts o$ #PCA s&stem simil#r to t"ose t"#t ere $irst rese#r%"ed

"#t it is "& it ors "#t it tells (s#nd t"en $ollo;

(' ver& brie$l& it" # s"ort "istor& o$ t"e rese#r%"

'er$ormed in %om'(ter vision rel#ting to PCA te%"ni)(es.

9in#ll& # $(ll& $(n%tion#l #nd %om'leted s('ervised signl#ng(#ge re%ognition s&stem b#sed on PCA te%"ni)(es ill

be 'resented.


2/13


Given # n(mber ,N- o$ simil#r gr#&s%#le tr#ining im#ges

,>- e#%" %om'osed o$ #n (n%"#nging n(mber o$ %ol(mns

#nd ros e b(ild # ne m#tri+ %#lled #n ensemble

denoted ?>n@ t"#t %ont#ins $or e#%" o$ its ros t"e entire'i+el d#t# o$ # given > 6. 9ig(re 2 gives # vis(#l

inter'ret#tion o$ t"is tr#ns$orm#tion.

%igure ' !"e ensemble m#tri+ ?>n@ is b(ilt $rom l#&ering entire 'i+el

d#t# o$ ever& tr#ining im#ge #s ros.

!"e %ol(mn dimensions o$ # %om'leted ensemble ?>n@ #re

e)(iv#lent to t"e n(mber o$ 'i+els in #n im#ge it" t"e

n(mber o$ ros e)(iv#lent to "o ever m#n& im#ges ere

in%l(ded in t"e ensemble. An ensemble is im'ort#ntbe%#(se it #ligns 'i+el d#t# in # $orm "i%" %#n be

e+'loited l#ter.

%igure ( A vis(#l re'resent#tion o$ # %om'leted ensemble ?>n@.

et (s ens(re t"#t e#%" element o$ ?>n@ is o$ # simil#r %l#ss

#nd t"#t ?>n@ is b(ilt it" no ot"er > o$ #n e+tremel&

di$$erent %l#ss. :e no t"#t st#tisti%#l red(nd#n%ies e+ist

in im#ges o$ # simil#r %l#ss #nd #re m#gni$ied "en%onstr#ints #re 'l#%ed ('on s%#le tr#nsl#tion #nd rot#tion

10. 9or # PCA im#ge s&stem t"e di$$eren%es #mong t"e

im#ges in ?>n@ #re im'ort#nt. :e strengt"en t"ese

di$$eren%es b& %#l%(l#ting $or ever& > in ?>n@ t"e de'#rt(re$rom t"e #ver#ge > 6. !"e #ver#ge > is &ielded (sing t"e

$olloing e)(#tion "ere N is e)(#l to t"e n(mber o$

im#ges (sed to b(ild ?>n@. !"e de'#rt(re $rom t"e me#n >

#nd res(lting ensemble is s"on in )(#tion 2.

=

=N

n nN1

@?1

#)uation & C#l%(l#tion o$ t"e #ver#ge im#ge $rom t"e ensemble. !"e

#ver#ge > is (sed to %onstr(%t t"e %om'leted ensemble .

= @?n

#)uation ' !"e $in#l ensemble is &ielded b& t#ing ever& element o$

t"e (n#ltered ensemble ?>n@ #nd s(btr#%ting t"e #ver#ge >.

!"is #ltered version o$ ?>n@ re'resents t"e di$$eren%es

#mong #ll o$ t"e im#ges. :e denote t"e nel& %onstr(%tedensemble #s . #%" ro o$ %orres'onds to 'i+el d#t#

"i%" #re %ommonl& re$erred to #s %#ri%#t(res.

*(r #ttention no t(rns to e+tr#%ting eigenve%tors "i%"ill be (sed t"ro(g"o(t t"e rest o$ t"e tr#ining '"#se #nd in

t"e re%ognition '"#se. !"e %om'(ted ort"onorm#l

eigenve%tors ,#nd rel#ted eigenv#l(es- o$ #n #ver#ge

%ov#ri#n%e m#tri+ b(ilt (sing 'rovides # b#sis todes%ribe t"e im'ort#nt $e#t(res in . Bielding t"e

eigenve%tors #nd eigenv#l(es re'resent#tive o$ re)(irest"e %re#tion o$ #n #ver#ge %ov#ri#n%e m#tri+.

An #ver#ge %ov#ri#n%e m#tri+ ,- is e#sil& %#l%(l#ted b&

t#ing t"e tr#ns'ose o$ #nd m(lti'l&ing b& 8. !"e

res(lting obDe%t is # s&mmetri% s)(#re #ver#ge %ov#ri#n%e

m#tri+ obt#ined $rom 5. !"e dimensions o$ #ree)(#l to t"e n(mber o$ %ol(mns in .

=

T

#)uation ( Com'(ting t"e #ver#ge %ov#ri#n%e m#tri+ given #n

ensemble. )(#l to t"e tr#ns'ose o$ times itsel$.

igen de%om'osition is t"en 'er$ormed on . !"e

$olloing e)(#tion E rel#tes t"e #ver#ge %ov#ri#n%e

m#tri+ to its %orres'onding eigenv#l(es ,F- #nd

eigenve%tors ,u-.

nu

nnu =

#)uation * Rel#ting t"e nel& %onstr(%ted %ov#ri#n%e m#tri+ to

eigenve%tors #nd %orres'onding eigenv#l(es.

igen de%om'osition &ields #s m#n& eigenve%tors #s t"ere

#re n(mber o$ %ol(mns in #nd #n e)(#l n(mber o$eigenv#l(es. #%" eigenve%tor m#t"em#ti%#ll& re'resents #

sele%t $e#t(re in #ll elements o$ . !"ere #re "oever

some eigenve%tors "i%" #re more e+'ressive #ndt"ere$ore re'resent more im'ort#nt $e#t(res in . !"ese

$irst K eigenve%tors ,%orres'onding to t"e l#rgest

eigenv#l(es- #re sele%ted #s t"e 'rin%i'#l %om'onents #nd

s#ved $or $(t(re %om'(t#tions.


3/13


!"e re%entl& #%)(ired K eigenve%tors #re t"e onl&

eigenve%tors needed sin%e #n #dmi+t(re o$ t"em #nd o(r

#ver#ge > #lone %#n s($$i%ientl& #''ro+im#te #n& im#ge in

t"e origin#l ensemble ?>n@. !"is e %#n re%#ll is t"e #imo$ # PCA s&stem #nd #$$ords # %om'(t#tion#ll& e#sier #&

to re'resent im#ges. :e re#lie too t"#t (tili#tion o$ all

eigenve%tors o(ld res(lt in #n e+#%t re%re#tion o$ t"e

origin#l im#ge 6.

Nnu

N

n ncoef =

=

+ 1

#)uation + :e %#n #''ro+im#te #n origin#l im#ge > b& line#rl&%ombining 'rim#r& eigenve%tors m(lti'lied b& # rel#ted %oe$$i%ient #nd

#dding t"e #ver#ge > 'i+el v#l(es. Com'lete re%onstr(%tion %#n o%%(r i$

#ll eigenve%tors #re (sed. 9or # PCA vision s&stem "oever onl&K

n(mber o$ eigenve%tors s"o(ld be (sed.

A$ter %"oosing # Kn(mber o$ 'rim#r& eigenve%tors e

move on to %om'lete t"e tr#ining '"#se "i%" re)(ires%#l%(l#ting t"e %oe$$i%ients o$ ever& ro ,%#ri%#t(re- in .

!"e %oe$$i%ients #re %#l%(l#ted b& t#ing t"e dot 'rod(%t o$#llKeigenve%tors it" # given %#ri%#t(re #nd storing t"e

res(lts in # loo(' t#ble. A$ter #ll N ros o$ #re

'ro%essed in t"is $#s"ion t"e loo(' t#ble is %onsidered

$inis"ed. #%" %olle%tion o$ K %oe$$i%ient res(lts $or #given ro in #re given # %l#ssi$i%#tion l#bel ,L-. In #ll

t"ere ill beKn(mber o$ %oe$$i%ients $orNn(mber o$ test

im#ge entries. !"is %#n be%ome %om'(t#tion#ll& time

%ons(ming to "#ndle d(ring t"e re%ognition '"#se.

*$ten %l(stering o$ simil#r L im#ges o%%(rs in t"e

%oe$$i%ient $e#t(re s'#%e. !"e #dvent o$ %l(stering is

desir#ble sin%e e %#n red(%e o(r 'revio(sl& %onstr(%tedloo(' t#ble to onl&Ln(mber o$ %l#sses. AllK%oe$$i%ient

v#l(es $or e#%" %l#ss L#re #ver#ged #nd t"ese res(lting

#ver#ge %oe$$i%ients #re stored #s t"e d#t# in t"e red(%ed

loo(' t#ble. It is im'ort#nt to note t"#t e m#& not be sol(%& #s to ala!sdetermineLn(mber o$ %l(sters. !"ere

%o(ld be #n& n(mber o$ se'#r#te %l(sters t"#t 'ert#in to t"e

s#me L overl#''ing %l(sters "i%" m#e t"is #''ro#%"

in$e#sible or s'or#di% 'lots in t"e $e#t(re s'#%e.A''ro'ri#te inter'ret#tion o$ res(lts is vit#l be$ore

'rogressing it" t"is met"od.

Re%#ll t"#t t"e &ielded %oe$$i%ients #re # res(lt o$ t#ing t"e

dot 'rod(%t o$ #ll 'rin%i'#l eigenve%tors it" 'i+el d#t##sso%i#ted it" # ro in . !"e Kres(lting %oe$$i%ients

re'resent t"e n(mbers b& "i%" e#%" eigenve%tor m(st bem(lti'lied b& to &ield t"e #''ro+im#tion o$ t"#t $e#t(re $or

# given im#ge. !"is s"o(ld m#e sense sin%e t"e

eigenve%tors trul! are t"e $e#t(res o$ t"e entire ensemble

#nd re)(ire some #lter#tion to #''ro+im#te t"e im#ge. !"e#ddition o$ t"e m(lti'lied eigenve%tor v#l(es is not

s($$i%ient &et #s t"e #ver#ge > m(st #lso be #dded. !"e

b#sis o$ $(t(re %l#ssi$i%#tion relies on t"e "o'e t"#t im#ges

o$ # simil#r %l#ss ill re)(ire # simil#r r#nge o$ %oe$$i%ients

t"#t m(st be (sed to re%re#te t"e im#ge.

!"e l#st '#rt o$ # PCA vision s&stem is str#ig"t$or#rd #nd

int(itive. Given # test im#ge ,T- 'rovided to t"e s&stem e

$ind its %orres'onding %#ri%#t(re ,Tc- b& s(btr#%ting t"e

#ver#ge >. Ne+t e (tilie e#%" o$ o(rKeigenve%tors #ndt#e t"e dot 'rod(%t it" Tcto &ield t"e %oe$$i%ients o$ Tc.!"ese #re t"en %om'#red to o(r 'revio(sl& %onstr(%ted

l#bel/%oe$$i%ient loo(' t#ble #nd # ne#rest neig"bor

#lgorit"m is 'er$ormed b#sed on (%lide#n dist#n%es in t"e$e#t(re s'#%e 8. :"i%"ever dist#n%e is determined to be

t"e sm#llest dire%tl& identi$ies "i%" %l#ss t"e im#ge

s"o(ld be #sso%i#ted it". Com'(t#tion#l time is red(%ed

even more i$ t"e loo(' t#ble %onsists o$ t"e #ver#ges o$t"e %oe$$i%ients $or e#%" %l#ss.

PR#,IUS R#S#ARC-

!"e rese#r%" o$ PCA te%"ni)(es in %om'(ter vision d#tes

b#% to t"e l#te 1580s it" initi#l or 'er$ormed b&Sirovi%" #nd irb& #t Hron niversit&. Sirovi%" #nd

irb& $o%(sed on (sing # %om'(ter to re%ognie %ro''ed

versions o$ "(m#n $#%es #nd %om'#red t"eir res(lts to t"#to$ re%ogniing $(ll $#%es. !"eir s&stem "#d good res(lts

#nd more im'ort#ntl& 'roved t"e 'r#%ti%#lit& o$ s(%" #

s&stem 6. !"eir 'ioneering s(%%ess #s $olloed in 1550

"en t"e& e+tended t"eir s&stem o$ re%ogniing $(ll $#%esb& do(bling t"eir ensemble sie t"ro(g" t"e (se o$ mirror

im#ges 7. !"is gre#tl& red(%ed error r#te in re%ognition

o$ test im#ges. Sirovi%" "#s %ontin(ed rese#r%" #nd "#s

'(blis"ed #s re%entl& #s 2000 10.

:"ile e ere (n#ble to lo%#te # %o'& o$ t"e or'er$ormed b& !(r #nd Pentl#nd e o(ld be remiss $or

not re$eren%ing it "ere. !"eir 1551 v#ri#tion o$ Sirovi%"#nd irb&Js PCA s&stem relied on %#l%(l#ting eig"ts

#sso%i#ted it" eigen'i%t(res ,inste#d o$ %oe$$i%ients- #nd

stored t"ese v#l(es in # retriev#ble d#t#b#se. !"e go#l o$

t"eir s&stem #s to s(%%ess$(ll& determine i$ # $#%e #s'resent it"in # given im#ge or not. Interestingl& t"eir

ensemble not onl& in%l(ded K"e#d onL s"ots b(t #lso

in%l(ded di$$erent "e#d sies orient#tions #nd lig"ting

%onditionsm#ing it rob(st #nd reli#ble $or sever#l'ossible t&'es o$ test im#ges 10.

9ive &e#rs l#ter Sets #nd :eng ere #ble to develo' #PCA b#sed s&stem t"#t v#ried $rom ot"er %ontem'or#r&s&stems ,"i%" ere de'endent ('on t"e di$$eren%es

beteen im#ges-. !"e& inste#d (sed dis%rimin#ting

$e#t(res &ielded $rom dis%rimin#nt #n#l&sis to m#e

%l#ssi$i%#tions 8. !"eir rese#r%" #s not onl& #ble tore%ognie t"e 'resen%e o$ # $#%e b(t to #lso %l#ssi$& the

person in t"e im#ge m#ing it im'ressive #nd ort"

mentioning "ere 10.


4/13


:e $in#ll& note #nd %redit t"e s(%%ess t"#t Mog"#dd#m #nd

Pentl#nd "#d b& being t"e $irst to develo' # PCA s&stem

t"#t relied ('on 'rob#bilisti% densit& estim#tion. !"is

%l#ssi$i%#tion met"od %ontr#sts it" t"e b#si% (%lide#n%l#ssi$i%#tion met"ods (sed be$ore"#nd. !"eir '#'er #s

'(blis"ed in 155E #long it" sever#l ot"er '(bli%#tions

#ttem'ting to %re#te better #&s to %l#ssi$& im#ges e#%"

"#ving t"eir on st#'le o$ merit #nd %ontrib(tion to t"ere#lm o$ PCA rese#r%" 10.

!"e investig#tion $or o(r in)(ir& did not re)(ire t"e

#%)(isition o$ some o$ t"ese ors d(e to t"e $#%t t"#t o(rs&stem is b#sed e+tensivel& on Sirovi%" #nd irb&Js e#rl&

rese#r%" "i%" is no %onsidered old. In %onsider#tion o$

o(r 'rim#r& investig#tion e le#ve $(rt"er "istori%#l

rese#r%" to t"e interested re#der or dire%t t"em to t"eiter#t(re S(rve& 10 %om'lied b& "#o #nd ot"ers "i%"

o$$ers # %om're"ensive "istor& o$ PCA st(dies.

I"PL#"#NTATIN

A %om'leted s('ervised sign l#ng(#ge re%ognition s&stem#s develo'ed (sing PCA te%"ni)(es simil#r to t"ose

rese#r%"ed b& Sirovi%" #nd irb&. 9or o(r s&stem

"oever e devi#ted $rom some o$ t"eir stri%ter%onstr#ints es'e%i#ll& in d#t# #%)(isition #nd 're'#r#tion.

Code $or im'lement#tion #s 'rogr#mmed in COO #nd #s

%om'#rtment#lied into t"ree sm#ller 'roDe%ts $or

re#d#bilit& e#se #nd #bstr#%tion. All 're'#r#tor& tr#ining#nd testing im#ges ere t#en #nd 're;'ro%essed b& "(m#n

intervention.

!"e $irst '#rt o$ t"e s&stem (sed im#ges lo%#ted in # lo%#ldire%tor& to b(ild #n ensemble o$ %#ri%#t(res. !"en t"e

#ver#ge %ov#ri#n%e m#tri+ #s %om'(ted be$ore eigende%om'osition beg#n. 'on %om'letion o$ eigen

de%om'osition #ll eigenve%tors ere sorted #nd stored toindivid(#l $iles #long it" %orres'onding eigenv#l(es.

!"e se%ond '#rt o$ t"e s&stem (sed some o$ t"is o(t'(t to

e+tr#%t ne%ess#r& %oe$$i%ients to be s#ved #nd (tilied in

t"e testing '"#se. 9in#ll& t"e testing '"#se re%eived #llre)(ired elements to m#e # %l#ssi$i%#tion D(dgment o$ #n

(ntr#ined im#ge. :e s"#ll dis%(ss in det#iled ste's t"e

im'lement#tion #nd individ(#l res(lts o$ t"e se'#r#te '#rts

o$ t"e s&stem.

:e is" to m#e t"e re#der ##re be$ore %ontin(ing o$

t"e s%o'e o$ o(r s('ervised s&stem. In #n #ttem't to limitt"e e+tent o$ t"e 'roDe%t so #s to &ield res(lts in # timel&m#nner #nd to ill(str#te t"e 'otenti#l s(%%ess o$ # sign

l#ng(#ge re%ognition s&stem t"e de%ision #s m#de to

tr#in onl& $ive ,6- letters. :e %"ose t"e voels o$ t"e

Ameri%#n nglis" #l'"#bet= A I * #nd . S(%%ess o$t"is limited s&stem o(ld 'rovide t"e b#se or #nd

%on$iden%e to e+tend t"e s&stem to ot"er letters. :it" t"is

in mind e %ontin(e.

PR#LI"INAR$ PR#PARATIN

Some ne%ess#r& gro(ndor #s needed be$ore

im'lement#tion %o(ld be %#rried o(t. Im#ge %#'t(ring #s

'er$ormed b& t"ree individ(#ls= # '"otogr#'"er # 'erson"olding # lig"t #nd t"e signer o$ t"e letter. nlie irb&

#nd Sirovi%" in%redible %#re #s not ta"ento st#bilie t"e

%#mer# or m#int#in # st#ti% lig"t so(r%e. !"e '"otogr#'"er

(sed onl& "is "#nds #nd #rms to st#bilie t"e %#mer# #ndt"e lig"ter "eld t"e lig"t in # semi;st#ti% 'osition. !"e

signer "eld t"eir rig"t "#nd in $ront o$ # d#r bl(e b#%dro'

#nd signed t"e #''ro'ri#te letter. !"ere #s onl& one

signer o$ t"e letters# C#(%#si#n m#le #nd t"e'"otogr#'"er #s instr(%ted to vis(#ll& %enter t"e $ist #re#

o$ t"e "#nd #s m(%" #s t"e& %o(ld. A tot#l o$ 30 'i%t(res

'er letter %l#ss ere t#en $or # gr#nd tot#l o$ 160 im#ges.

*nl& 140 o$ t"ese im#ges o(ld %om'rise o(r tr#ining set"oever "ile t"e ot"er 10 ,to o$ e#%" letter %l#ss-

o(ld be s#ved $or o(r testing '"#se. !"e res(lting 28

tr#ining im#ges 'er letter %l#ss ere # s($$i%ient #mo(nt $or

o(r PCA vision s&stem to or.

A Son& 3.2 Meg#'i+el C&bers"ot br#nd %#mer# #s (sed

to t#e t"e '"otogr#'"s. He$ore 'i%t(res ere t#en t"e

%#mer# #s set to # resol(tion o$ 740 b& 480 #nd t"egr#&s%#le o'tion #s #%tiv#ted. All res(lting im#ges ere

s#ved in PG $orm#t. Ne+t e#%" im#ge #s o'ened in

Adobe P"otos"o' CS #nd %onverted to # tr(e gr#&s%#le

version "ile t#ing into %onsider#tion 'ro'er rot#tions toorient t"e "#nd 'osition %orre%tl& ,im#ges ere n#tivel&

t(rned 50Q %lo%ise-. tili#tion o$ t"e P"otos"o'

%ro''ing/resiing tool red(%ed o(r sign l#ng(#ge letter to #

sie o$ 74 'i+el ros b& 48 'i+el %ol(mns. !"e red(%edim#ge #s s#ved on%e #g#in to PG $orm#t #t m#+im(m

)(#lit&. !"e $in#l ste' to 're'#ring o(r im#ge set #s to%onvert #ll im#ges to PNM $ile $orm#t. !"e PNM $ile

%onversion s#ved 'i+el d#t# #s ASCII v#l(es "i%"#$$orded (s t"e #bilit& to e#sil& or it" #nd inter'ret

$(t(re %om'(t#tions. !"e GIMP o'en so(r%e gr#'"i%s

editing so$t#re #s (se$(l $or t"is %onversion #nd #s

(sed t"ro(g"o(t im'lement#tion. !"e $in#lied versions o$t"e im#ges ere t&'e P2 ,gr#&s%#le- #nd "#d # m#+im(m

gr#& v#l(e o$ 266. *t"er t"#n "#t #s des%ribed #bove

no ot"er m#ni'(l#tion $ilters or editing #s done.

!r#ining im#ges ere n#med in # s'e%i$i% $orm#t "i%"

in%l(ded t"e letter #nd t"e inde+ o$ t"#t inst#n%e o$ t"e

im#ge. An e+#m'le o$ # $ilen#me o(ld be #$%&N' orO()%&N'. !"ro(g"o(t im'lement#tion $iles #re o$teno(t'(t to t"e s&stem t"#t #re #sso%i#ted it" t"eir origin#l

im#ges. !"ese $iles #l#&s %ont#in # re$eren%e to t"eir

letter %l#ss #nd inde+. :e no dis%(ss t"e im'lemented

tr#ining '"#se.

TRAINING P-AS#

Code #s develo'ed to "#ndle in'(t o$ # d&n#mi% n(mbero$ tr#ining im#ges even t"o(g" o(r s&stem s'e%i$i%#ll&


5/13


"#ndled onl& 140 im#ges. A to;dimension#l ve%tor d#t#

str(%t(re 2 #s (sed to "old t"ese v#l(es. *n%e 'i+el

d#t# #s s#ved into memor& t"e #ver#ge 'i+el v#l(es $rom

t"e ensemble ere ev#l(#ted. !"is #ver#ge im#ge #ss#ved b#% to t"e s&stem in # $ile %#lled a*g+*arphi%&N'

$or vis(#l re$eren%e. 9ig(re 6 s"os t"is #ver#ge im#ge.

Note t"e bl(rring e$$e%t re'resent#tive o$ t"e %om'osite o$

#ll signed letters %om'rising t"e ensemble.

%igure * Aver#ge im#ge t#en b& s(mming #ll 'i+el v#l(es in o(rensemble o$ 140 im#ges #nd dividing b& 140. !"is im#ge d#t# is l#ter

s(btr#%ted $rom t"e elements in t"e ensemble to &ield o(r $in#l ensemble.

:e re%#ll t"#t $or PCA vision te%"ni)(es to or o'tim#lln ensemble is re)(ired to %onsist o$ t"e di$$eren%e o$

v#l(es $rom t"e origin#l im#ges #nd t"e nel& retrieved#ver#ge v#l(es. rib& #nd Sirovi%" remind (s t"#t it is in

o(r interest $or e$$i%ien%& t"#t e %om'(te t"is neensemble 6. S(%" #n ensemble %onsisting o$ %#ri%#t(res

#lso #ids %onsider#bl& to red(%ing time "en &ielding t"e

#ver#ge %ov#ri#n%e m#tri+ 8. !"e res(lting Ktr(eL

ensemble is t"(s m#de (' o$ t"e di$$eren%e im#ges. !"esedi$$eren%e im#ges ere o(t'(t to $ile d(ring e+e%(tion $or

vis(#l re$eren%e. All 140 o$ t"e im#ges ere o(t'(t to $ile

in t"e $orm+carac%&N'. 9ig(re 7 ill(str#tes # s#m'ling

o$ origin#l im#ges it" t"eir %orres'onding %#ri%#t(res.Note t"#t # %#ri%#t(re %#n #%t(#ll& "#ve neg#tive v#l(es

"i%" #re D(st #s im'ort#nt #s 'ositive v#l(es. In GIMP

"oever neg#tive 'i+el v#l(es #re tre#ted #s b#%gro(nd,bl#%- so e do not KseeL t"ese v#l(es.

%igure + 9rom le$t to rig"t signed letters A I * it" origin#ls on

to' #nd %#ri%#t(res on bottom. :e see "ere t"e interesting '#rts to e#%"letter s"oing #s t"e lig"ter '#rts o$ t"e "#nd 'roviding # good b#sis $or

o(r eigenve%tors to ret(rn t"e most e+'ressive $e#t(res o$ t"e ensemble.

A$ter t"e ensemble #s %om'leted t"e #ver#ge %ov#ri#n%e

m#tri+ needed to be %om'(ted. In order to e+tr#%t

eigenve%tors #nd %orres'onding eigenv#l(es # s&mmetri%s)(#re m#tri+ m(st be (sed E 5 #nd more s'e%i$i%#ll&

o(r needs re)(ired #n #ver#ge %ov#ri#n%e m#tri+ b#sed

$rom o(r ensemble. Attem'ts to %#l%(l#te t"is m#tri+ b&

met"odi%#l br(te $or%e ere $o(nd to be %om'(t#tion#ll&

ine$$i%ient #nd t"(s %om'letel& in$e#sible. An eleg#nt

sol(tion #s 'ro'osed 8 #nd re$eren%ed e#rlier "i%"

bro(g"t %om'(t#tion time don to liter#ll& # $e se%onds.It too #dv#nt#ge o$ t"e m#t"em#ti%#l 'ro'ert& t"#t t"e

tr#ns'ose o$ o(r ensemble m(lti'lied b& t"e ensemble

res(lts in # 30E2 ,ros- b& 30E2 ,%ol(mns- m#tri+

re'resenting t"e #ver#ge %ov#ri#n%e m#tri+. S(%" #'ro'ert& e+ists #s # res(lt o$ #lre#d& "#ving t#en t"e

di$$eren%e o$ o(r im#ges in t"e ensemble. :e e#gerl&

e+'loited t"is re#li#tion to o(r bene$it

!"e %onstr(%tion o$ t"e %ov#ri#n%e m#tri+ in o(r %ode #s

"#ndled ver& ni%el& b& $(n%tions 'rovided b& t"e

Nem#t10 ,NM10- libr#r& %re#ted b& Robert


6/13


t"e NM10 #ver#ge %ov#ri#n%e m#tri+ to # NRIC s(''orted

m#tri+.

In #n #ttem't to #''e#se (sers o$ bot" NM10 #nd NRICt"e NM10 libr#r& in%l(ded m#tri+ t&'es t"#t ere t"o(g"t

to %oo'er#te #t le#st initi#ll& it" b#si% NRIC $(n%tions.


7/13

A. Geigel CPSC 320 Vision IRC/November 2004 E

!"e se%ond '"#se o$ o(r 'roDe%t t"o(g" still %onsidered

t"e tr#ining '"#se served to '(r'oses= 1- it %on$irmed t"e

initi#l belie$ t"#t t"ree eigenve%tors ere s($$i%ient to (se

$or $(t(re %l#ssi$i%#tion #nd 2- it b(ilt o(r l#bel/%oe$$i%ientloo(' t#ble to be (sed $or $(t(re %l#ssi$i%#tion.

Re%#ll t"#t t"e &ielded eigenve%tors #re # m#t"em#ti%#l

re'resent#tion o$ t"e most e+'ressive $e#t(res o$ o(rensemble. As s(%" t"e eigenve%tors t"emselves %o(ld be

vieed #s im#ges. !"e vis(#l inter'ret#tion o$ t"e

eigenve%tors ill(str#tes "#t t"e most e+'ressive $e#t(res

#re #nd #''e#r #s t"e brig"t #re#s. 9ig(re E s"os todi$$erent #&s o$ re'resenting t"e eigenve%tors #s im#ges

ot"erise non #s eigen'i%t(res. !"e to' ro #re

eigen'i%t(res t"#t t#e into #%%o(nt neg#tive v#l(es

e+"ibiting "#t #re tr(l& 'ositive #nd neg#tive it" t"ebottom ro #ltern#tivel& s"oing t"e #bsol(te v#l(e o$ t"e

eigen'i%t(re t"(s ill(str#ting t"e K"iddenL e)(#ll&

im'ort#nt #re#s o$ o(r eigenve%tors.

%igure 1 !"e t"ree 'rin%i'#l eigenve%tors s"on vis(#ll& #seigen'i%t(res. !"e to' ro s"os # version it" 'ositive #nd neg#tive

'i+el v#l(es ,#%t(#ll& (sed in $(rt"er %#l%(l#tions- #nd t"e bottom ro

s"os #n #bsol(te v#l(e version o$ t"e eigen'i%t(re ,(sed to vis(#ll&

e+'ose im'ort#nt neg#tive v#l(e #re#s-.

Progr#mmed %ode $or (se in t"e se%ond '"#se o$ o(r

'roDe%t #s (sed to gener#te t"ese im#ges "oever it "#s

sin%e been %ommented o(t sin%e t"e& #re not #l#&s

needed. !"e t"ree 'rim#r& eigenve%tor $iles gener#tedd(ring eigen de%om'osition served #s t"e in'(t $or t"e t"is

'"#se. A b#si% loo' #s em'lo&ed to $ill # to

dimension#l ve%tor t&'e d#t# str(%t(re it" t"e v#l(es o$

e#%" 'rim#r& eigenve%tor. I$ t"e eigen'i%t(res eredesired t"en e m(lti'lied e#%" n(meri%#l element o$ t"e

eigenve%tor b& 10000 #nd $or%ed #n integer t&'e. !"is

e$$e%tivel& t(rned t"e de%im#l $lo#ting 'oint n(mbers

retrieved $rom e#%" eigenve%tor into vie#ble 'i+el v#l(es.

A tem'or#r& integer t&'e v#ri#ble m#int#ined "#t t"e

m#+im(m 'i+el v#l(e #s #nd too ne%ess#r& ste's too(t'(t t"e d#t# to #n im#ge $ile. !"ese $iles ere n#med in

t"e $olloing $orm#t e*ec+pic+2%&N'#nd #re re'resented

in t"e to' ro o$ 9ig(re E. An #ltern#te version too t"e

#bsol(te v#l(e o$ 'i+el d#t# #nd #s o(t'(t to $iles$olloing t"e %onvention e*ec+pic+abs+2%&N'. !"ese #re

re'resented #s t"e bottom im#ges in 9ig(re E.

I$ eigen'i%t(res ere notdesired t"e %ode need sim'l& be%ommented o(t. It is im'ort#nt to re%ognie t"#t t"e 'i+el

d#t# re'resenting t"e eigen'i%t(res #s not (sed $or

%#l%(l#ting t"e %oe$$i%ients. !r(e $lo#ting 'oint v#l(es

ere (sed $or t"is.

+tr#%ting t"e %oe$$i%ients $or e#%" im#ge in t"e tr#ining

ensemble #s str#ig"t$or#rd. !o #%%om'lis" t"is e

%o'ied #ll o$ t"e %#ri%#t(re o(t'(t $iles t"#t ere gener#ted

d(ring eigen de%om'osition to t"e %(rrent oringdire%tor&. !"e %oe$$i%ients o$ # given im#ge ere 'rod(%ed

b& t#ing t"e dot 'rod(%t o$ # given %#ri%#t(re im#ge #nd

#ll 'rim#r& eigenve%tors.

Coding t"is '#rt o$ t"e s&stem #s #%"ieved b& #g#in

rel&ing on t"e NM10 libr#r&. !"ree %ol(mn ve%tors ere

%re#ted o$ t&'e Col(mnVe%tor #nd $illed it" #''ro'ri#teeigenve%tor elements. !"en # b#si% loo' #s (sed to

retrieve ever& tr#ining %#ri%#t(re im#ge $rom $ile #nd to

(se #not"er Col(mnVe%tor t&'e d#t# str(%t(re to "old t"e

'i+el d#t# o$ t"e %(rrentl& 'ro%essed im#ge. Re)(ired ne+tere t"e e+tr#%tion o$ t"e t"ree #sso%i#ted %oe$$i%ients $or

t"is im#ge. !"e NM10 $(n%tionDot&roductre%eived toCol(mnVe%tor t&'e in'(ts ,t"e %#ri%#t(re im#ge #nd

eigenve%tor- #nd ret(rned # de%im#l n(mber o$ NM10 t&'eRe#l 3. !"(s $or e#%" %#ri%#t(re im#ge in o(r dire%tor&

t"ree %orres'onding %oe$$i%ients ere %om'(ted.

!"ese %oe$$i%ients #nd t"eir #sso%i#ted %l#ss l#bels,retrieved $rom t"e $ilen#me- ere o(t'(t to #

l#bel/%oe$$i%ient loo(' $ile entitled coef+db%T1T$or (se in

t"e %l#ssi$i%#tion testing '"#se. ere in 9ig(re 5 e s"o

#n e+#m'le o$ t"e $irst to elements o$ t"e $ilecoef+db%T1T.


8/13


Re2: Cl#ss A Green:Cl#ss 3lue:Cl#ss I Purple:Cl#ss * Cyan:Cl#ss

%igure 4 !"e t"ree %oe$$i%ients &ielded $rom t#ing t"e dot 'rod(%t o$ t"e e#%" 'rin%i'#l eigenve%tor it" # given im#ge $rom t"e ensemble. !"e res(lting

%l(stering o$ t"e di$$erent letter %l#sses ill(str#tes t"#t t"e $irst t"ree eigenve%tors are sufficient$or (se in re%ogniing $(t(re test im#ges 'resented to t"e s&stem.

!"e %(bes in t"e gr#'" re'resent t"e #ver#ge %oe$$i%ients $or # '#rti%(l#r letter %l#ss. Cl#ss A "#s to #ver#ges %orres'onding to t"e to lig"ting/orient#tion

s(bsets it" t"e s'"ere in t"e %enter re'resenting t"e glob#l #ver#ge $or letter %l#ss A. !"is glob#l #ver#ge is not (sed in t"e testing '"#se b(t is 'resented

"ere $or in$orm#tive '(r'oses. Ne#rest neig"bor %#l%(l#tion is (sed it" t"e si+ #ver#ges in t"e testing '"#se to determine t"e letter %l#ss o$ t"e im#ge.

A

1=;835.073

1115.64

253.E85

A2=

;77E.5E5

1088.42;E5.6755

%igure 5 9irst to elements in t"e coef+db%T1T$ile to s"o "o %l#ss

l#bels ere #sso%i#ted it" %orres'onding %oe$$i%ients.

Note t"#t t"e %l#ss l#bel #s on one line $olloed b& #ninteger n(mber re'resenting t"e im#ge inde+ $or t"#t letter

%l#ss #nd $in#ll& t"e t"ree %oe$$i%ients $rom t"e dot

'rod(%t %om'(t#tion. !"e $irst v#l(e %orres'onded to t"e

res(lt $rom t"e $irst eigenve%tor #nd so on. !"is #s

%om'(ted $or #ll 140 tr#ining im#ges.

It #s #t t"is time "en e ere $irst #ble to $eel %on$ident

in "#ving %"osen to (se t"e $irst t"ree eigenve%tors #s o(r

'rim#r& eigenve%tors. :e de%ided to 'lot t"ese %oe$$i%ientv#l(es in # 3< $e#t(re s'#%e (sing t"e x%oordin#te $or t"e

$irst %oe$$i%ient y$or t"e se%ond %oe$$i%ient #nd z$or t"e

t"ird. !"e 'rogr#m %ode #s brie$l& #ltered to s#ve t"e%oe$$i%ient d#t# to # $ile in # #& t"#t %o(ld be e#sil&'#rsed b& t"e Gr#'"is so$t#re '#%#ge. Gr#'"is %#n ver&

e$$e%tivel& "#ndle in'(t d#t# in'(t $iles #nd inter'ret

%"#r#%ters to '#rse to 're'#re t"e d#t# $or t"ree

dimension#l 'lotting. !"is %ommented o(t %ode #s notdeleted #nd still e+ists b(t is not norm#ll& ne%ess#r&.

!"e res(lts $rom gr#'"ing ere im'ressive#t le#st $or

$o(r o$ t"e letter %l#sses. 9ig(re 8 ill(str#tes "o ell t"e


9/13


%oe$$i%ients %l(stered toget"er. S(%" %l(stering 'rovided

strong eviden%e t"#t t"ree eigenve%tors ere indeed

s($$i%ient $or testing $(t(re im#ges. !"e 'lots $or one letter

%l#ss "oever ere Ks're#d o(tL #nd not ell gro('edtoget"er. !"e letter %l#ss A seemed to "#ve to se'#r#te

%l(stering #re#s in t"e $e#t(re s'#%e. 'on revie #nd

investig#tion #s to "& t"is mig"t "#ve o%%(rred e

re#lied t"#t lig"ting #nd "#nd orient#tion di$$eren%es $ort"e letter %l#ss A %"#nged some"#t dr#m#ti%#ll& beteen

(3%&N'#nd(4%&N'W in e$$e%t %re#ting to versions o$

t"e letter %l#ss A "i%" ere le#rned b& t"e s&stem. :e

'rovide t"eir im#ges "ere $or %om'#rison. !"e #d;"o%sol(tion #s to sim'l& tre#t t"e to versions o$ A #s

se'#r#te %l(sters.

%igure 6 *n t"e le$t is im#ge(3%&N'#nd on t"e rig"t is im#ge(4%&N'. A %"#nge in "#nd orient#tion #nd lig"ting $or letter %l#ss A #t

t"is moment res(lted in seeing to di$$erent letter %l#ss A %l(sters in #

$e#t(re s'#%e 'lot o$ %oe$$i%ients.

Sin%e t"e $e#t(re s'#%e s"oed %l(stering it #s 'r#%ti%#l

to %#l%(l#te t"e #ver#ge %oe$$i%ients $or e#%" letter %l#ss

#nd store t"ese res(lts in o(r loo('/%oe$$i%ient t#bleinste#d o$ (sing #ll 140 %om'#risons d(ring t"e testing

'"#se. Re%#ll t"#t t"e ne#rest neig"bor %#l%(l#tion %#n be

time %ons(ming $or # l#rge set o$ %om'#risonsit is to o(r

#dv#nt#ge to red(%e t"is %#l%(l#tion to onl& si+%om'#risons. Sin%e t"ere ere no to versions o$ %l#ss

A t"ere o(ld besi5'ossible %l#ssi$i%#tions $o(r o$ "i%"

#re I * #nd #nd to o$ "i%" o(ld be $or A.

Aver#ge %oe$$i%ient %#l%(l#tion sim'l& re)(ired # slig"t

modi$i%#tion to t"e e+isting %ode. Inste#d o$ 'rinting o(t

t"e %oe$$i%ient res(lts #s t"e 'rogr#m e+e%(ted # tot#l #s

m#int#ined #nd ('on $inis"ing it" one letter %l#ss t"etot#l #s divided b& 28 or t"e tot#l n(mber o$ im#ges $or #

letter %l#ss. !"is met"od #s im'lemented $or t"e letter

%l#sses I * #nd b(t not $or letter %l#ss A. !o

%#l%(l#te t"e #ver#ges o$ t"e to versions o$ letter %l#ss Ae modi$ied t"e %ode ver& s'e%i$i%#ll& to &ield t"e #ver#ge

%oe$$i%ients $or im#ges(%&N'to(3%&N' #nd t"en $or

im#ges $rom (4%&N' to /6%&N'. *n%e t"e res(lts

ere gener#ted #nd re%orded t"e %ode #s modi$ied b#%to its origin#l $orm. !"e res(lting $ile %re#ted #s

coef+a*g+db+true%T1T. :e 'resent t"e %ontents o$ t"is $ile

"ere in its entiret&.

A

;E28.154

8E2.476182.532

A

615.E38

;68E.7E3E7.7EE8

873.24E

;1676.E370.201

I

;681.411

;118.452;1668.13

*

1710.37

1243.48

242.105

;1E07.68

152.785830.374

%igure &7 !"e %om'lete %ontents o$ t"e $ile coef+a*g+db+true%T1T. !"e

$ile %ont#ins t"e #ver#ge %oe$$i%ients $or ever& letter %l#ss t"#t #str#ined in%l(ding to versions o$ letter %l#ss A.

:e note "ere t"#t b& sim'l& vieing t"e #ver#ge

%oe$$i%ients $rom t"e $ile one %#n g#in t"e sense t"#t e#%"

letter %l#ss does re'resent # (ni)(e $e#t(re s'#%e. !"ese#ver#ges #re re'resented in 9ig(re 8 b& t"e %(bes.

!"e %ode #sso%i#ted it" t"is '"#se 'rovided #ddition#l

s(''ort t"#t t"e $irst t"ree eigenve%tors ere s($$i%ient.:e %o(ld re%re#te or #t le#st #''ro+im#te #n& origin#l

'i%t(re $rom # letter %l#ss b& m(lti'l&ing # given im#geJs

%#l%(l#ted %oe$$i%ients it" t"eir res'e%tive eigenve%tors#nd s(mming #ll v#l(es #long it" t"e #ver#ge im#ge. As

t"e n(mber o$ eigenve%tors (sed in t"e #''ro+im#tion

'ro%ess in%re#ses t"e re%onstr(%tion o$ t"e im#ge be%omes

better. I$ #ll eigenve%tors ere (sed t"e im#ge o(ld bere%onstr(%ted in its origin#l $orm.

Re%onstr(%tion #s 'er$ormed d(ring e+e%(tion o$ t"e

%oe$$i%ient %#l%(l#tion 'ro%ess des%ribed D(st # moment#go. In order to #%%om'lis" t"is in %ode e re)(ired t"e

(se o$ t"ree ne NM10 Col(mnVe%tor t&'e ve%tors to "old

t"e ne v#l(es derived $rom m(lti'l&ing e#%" element in

e#%" 'rim#r& eigenve%tor b& t"e %orres'onding %oe$$i%ient.:e #lso re)(ired t"e v#l(es o$ t"e #ver#ge im#ge o$ t"e

ensemble #nd so "#ndled t"is in'(t #%%ordingl&. A b#si%

loo' m#int#ined t"e s(mm#tion o$ t"e $o(r re)(ired

elements #nd $or%ed #n integer t&'e $or t"e d#t#. Integerd#t# #s $or%ed so t"e o(t'(t to #n im#ge $ile %o(ld be


10/13


vieed. !"ese $iles $olloed t"e n#ming %onvention

#($+eigen+recon+%&N'#nd served #s # vis(#l ev#l(#tion

#s to "et"er t"e t"ree sele%ted eigenve%tors ere good

eno(g" to re'resent #ll t"e im#ges in t"e ensemble. :e'rovide # sele%tion o$ t"e res(lts no.

%igure && +#m'les o$ origin#l im#ges o$ e#%" letter %l#ss ,to'- it"

t"eir %orres'onding eigen re%onstr(%ted #''ro+im#tions ,belo-. Note

t"e to entries $or %l#ss A #nd "o simil#r t"e re%onstr(%tion o$ t"e

se%ond %l#ss A is to t"e re%onstr(%tion o$ %l#ss.

!"e res(lts bot" %on$irm #nd r#ise do(bt to o(r sele%tion o$t"ree eigenve%tors. !"e $irst version o$ letter %l#ss A #s

"orribl& re%onstr(%ted #nd one %#n 'rob#bl& not even tell"#t t"e origin#l im#ge #s s(''osed to be. !"e

re%onstr(%tion o$ #n im#ge $rom t"e se%ond version o$

letter %l#ss A #s some"#t better &et it #''e#red to loo

more lie # member o$ letter %l#ss . A$ter letter %l#ss At"e re%onstr(%tions ere #t le#st KgoodL it" t"e

re%onstr(%tions s"oing t"#t t"e m#Dor distin%tive $e#t(res

o$ t"e letter %l#sses ere in $#%t %#'t(red in t"e $irst t"ree

eigenve%tors.

!"e de%ision #s m#de t"#t be%#(se o$ t"e strengt" o$ t"e

%l(stering res(lts #nd t"e re%onstr(%tions o$ origin#l

im#ges ere #t le#st KgoodL t"#t e o(ld b#se o(r testing'"#se o$ t"e 'roDe%t o$$ o$ t"e #ver#ge %oe$$i%ients obt#ined

e#rlier #nd t"e t"ree 'rim#r& eigenve%tors.

R#CGNITIN 0 CLASSI%ICATIN T#STING

!"e re%ognition/%l#ssi$i%#tion '"#se #s t"e e#siest '"#se

to im'lement in %ode. !"ree items ere re)(ired to test

t"e s(%%ess o$ t"e develo'ed s&stem. !"e& in%l(ded= t"e

#ver#ge l#bel/%oe$$i%ient loo(' t#ble t"e t"ree 'rim#r&eigenve%tors #nd # test im#ge "i%" needed to be

re%ognied #nd/or %l#ssi$ied 8.

Ag#in e relied on t"e NM10 libr#r& to t#e t"e dot

'rod(%t o$ o(r test im#ge it" e#%" o$ o(r 'rim#r&eigenve%tors. !"e test im#ge $ilen#me #s 'rom'ted $or

#nd lo#ded into # NM10 Col(mnM#tri+ t&'e d#t# str(%t(reD(st #s e#%" o$ t"e t"ree eigenve%tors ere. !"e test

im#ges e (sed $olloed t"e n#ming %onvention

#/+test%&N'"oever t"e (ser #s onl& re)(ired to enter

t"e letter #nd n(mber to m#e $or more e$$i%ient testing#nd #n#l&sis. A$ter t"e Dot&roduct$(n%tion #s (tilied

t"e res(lting %oe$$i%ients ere t"en %om'#red to t"e

%oe$$i%ient/loo(' t#ble "i%" #s #lso lo#ded $rom $ile

into # to dimension#l ve%tor t&'e d#t# str(%t(re. A ver&

element#r& ne#rest neig"bor %#l%(l#tion #s #''lied to

&ield "i%" %l#ss t"e '#rti%(l#r test im#ge s"o(ld be

#sso%i#ted it". In %ode t"is #s #%%om'lis"ed b&s(mming t"e s)(#res o$ t"e di$$eren%e o$ t"e t"ree

%oe$$i%ients o$ t"e test im#ge it" t"e %oe$$i%ients $or #

given letter %l#ss. :e o'ted to not $ind t"e actualdist#n%e

so #s to #void (nneeded %om'(t#tion time dedi%#ted to%#l%(l#ting t"e s)(#re root.

!"e dist#n%es ere t"en 'rinted to t"e s%reen $or vis(#l

inter'ret#tion #nd si+ %ondition#l st#tements ,one $orever& letter %l#ss- ere (sed to determine "i%" letter

s"o(ld be 'rinted to t"e s%reen #s t"e g(ess. :e dis%(ss

t"e s(%%ess #nd res(lts o$ o(r s&stem ne+t.

R#SULTS

In retros'e%t t"e im'lement#tion o$ t"e sign l#ng(#ge

re%ognition s&stem #s s(%%ess$(l in ne#rl& ever& re#lm

#nd #t ever& m#Dor milestone. #%" '"#se 'er$ormed #s it

s"o(ld "#ve #nd delivered reli#ble res(lts it" "i%" tob#se $(t(re investig#tion #nd 'rogress. !"e testing '"#se

#s no e+%e'tion to t"is r(le.

Re%#ll t"#t #t t"e beginning o$ o(r 'roDe%t 160 im#ges ere

t#en it" 140 o$ t"ose %om'rising t"e tr#ining d#t# #nd

t"e ot"er 10 set #side #s test im#ges to be (sed #$ter s&stem

%om'letion. !"e $olloing $ig(re s"os t"e 10 test im#gest"#t ere set #side ,to im#ges $or ever& letter %l#ss- it"

t"e to' ro %onsisting o$ t"e $irst im#ges tested #nd t"e

bottom ro t"e se%ond.

%igure &' !"e 10 test im#ges s#ved sin%e t"e beginning #nd (sed in t"e$in#l '"#se o$ t"e 'roDe%t. Re%ognition o$ t"ese im#ges to t"eir res'e%tive

letter %l#sses ere 100X #%%(r#te.

Cl#ssi$i%#tion s(%%ess #s 100X $or #ll 10 o$ t"e testim#ges. !"e s&stem #s #ble to %orre%tl& ret(rn to t"e (ser

#n ASCII letter KAL KL KIL K*L or KL %orres'ondingto t"e letter t"#t #s signed in t"e test im#ge. :e in%l(de

"ere t"e %#l%(l#ted %oe$$i%ients o$ e#%" o$ t"e test im#ges#nd t"e letter %l#ss t"#t t"e s&stem g(essed.


11/13


Test C#% & C#% ' C#% ( Guess

A1 351.15 ;850.37E E76.424 A

A2 608.E51 ;746.601 252.286 A

1 327.713 ;1482.34 568.374

2 1025.2 ;1607.48 336.432

I1 ;362.885 ;273.653 ;1554.64 I

I2 ;178.0E1 ;114.304 ;1576.E1 I*1 2266.2 107E.11 715.637 *

*2 2225.85 E57.478 670.542 *

1 ;1315.12 641.35 728.3E3

2 ;1420.84 ;116.058 708.E28

Table ' !"e %#l%(l#ted %oe$$i%ients $or e#%" o$ t"e 10 test im#ges #nd t"e

PCA vision s&stemJs g(ess #s to "i%" letter %l#ss e#%" belonged to.

!"ese res(lts ere not s(r'rising "en vis(#ll&

re'resented in o(r $e#t(re s'#%e. #%" o$ t"e to test

im#ges 'er letter %l#ss ere 'lotted in t"e $e#t(re s'#%e tosee "o %lose to t"e #ver#ge %oe$$i%ients t"e& ere.

Presented "ere #re $ive $e#t(re s'#%e 'lots %orres'onding toe#%" letter %l#ss. !"e %oe$$i%ients o$ t"e test im#ges #re

re'resented in t"e $e#t(re s'#%e #s ('side don %ones it"t"e ti' 'ointing to t"e lo%#tion in t"e $e#t(re s'#%e t"#t it

o(ld be 'lotted.

%igure &( 9ive se'#r#te $e#t(re s'#%e 'lots e#%" de'i%ting t"e%oe$$i%ients o$ test im#ges $or # given letter %l#ss. Coe$$i%ients o$ test

im#ges #re 'lotted (sing inverted %ones it" t"e 'oint o$ t"e %one

s"oing "ere t"e test im#ge o(ld be 'lotted.

&*8A: Letter Class A 9

&*8#: Letter Class # 9

&*8I: Letter Class I 9


12/13


&*8: Letter Class 9

&*8U: Letter Class U 9

!"ese $e#t(re s'#%e 'lots re#$$irm o(r tr(st in t"e

develo'ed s&stem. 9or ever& 'lotted test im#ge e %#n

int(itivel& re%ognie t"#t t"e&shouldbe o$ # %ert#in letter%l#ss. n$ort(n#tel& onl& t"e se%ond version o$ t"e letter

%l#ss A #s (sed #s # test im#ge. It o(ld "#ve been

desir#ble to test im#ges $rom bot" versions o$ letter %l#ss

A. Interestingl& t"o(g" e noti%e t"#t t"e 'lots $or t"ese%ond version ere ver& %lose to t"e 'lots $or letter %l#ss

. !"is s"o(ld %ome #s no s(r'rise "oever sin%e o(r

re%onstr(%ted im#ges $or t"is version o$ A ere ver&simil#r to t"#t o$ t"e letter . In #ddition t"e simil#rities

beteen t"e se%ond version o$ letter %l#ss A #nd ere

even more #''#rent "en looing #t t"e (%lide#n dist#n%e

o(t'(t $or A.

!"e %#l%(l#ted dist#n%e $or(+test%&N'to t"e #ver#ge o$

letter %l#ss A #s 682615 %om'#red to # %#l%(l#ted

dist#n%e to t"e #ver#ge o$ letter %l#ss "i%" #s 843118.

And t"en $or /+test%&N' t"e dist#n%e #s 45560.7%om'#red to 5EE01E. Most dist#n%e %#l%(l#tions 'l#%ed #

given test im#ge %le#rl& in one %l#ss b(t t"ese res(lts ere

m(%" %loser. #d 'ro'er t"o(g"t gone into setting #side #

test im#ge $rom e#%" version o$ letter %l#ss A e %o(ld beeven more %on$ident in o(r res(lts.

It is im'ort#nt to note too t"#t no reDe%tion %#se #s

'rogr#mmed $or t"is s&stem. No n(meri%#l t"res"old o$(%lide#n dist#n%e #s in%l(ded t"#t o(ld de%ide

"et"er #n im#ge s"o(ld not be #sso%i#ted it" #n& %l#ss.

!"ere$ore o(r s&stem ill #l#&s #ttem't to %l#ssi$& #n

im#ge even i$ it does not %ont#in # signed letter.

PR3L#"S 0 CNC#RNS

An& %om'le+ vision s&stem %#n be e+'e%ted to "#ve

develo'ment#l 'roblems. *(r im'lement#tion #s noe+%e'tion #nd in t"e interest o$ %om'leteness e ill

e+'ose t"e (ni)(e tro(bles e en%o(ntered. :e s"#ll

"oever $orego re$eren%ing ever& min(te %oding $#(lt t"#t

%#n 'l#g(e 'rogr#mming e+e%(tion s(%" #s lin #nd%om'ile errors. In #ddition to t"is e o(ld lie to 'oint

o(t t"#t no 'roblem t"#t be$ell o(r s&stem #s "#rm$(l to

t"e e+tent o$ "#lting 'rogress #nd &ielding (nreli#ble

res(lts.

!"e tr#ining '"#se #s res'onsible $or gener#ting t"egre#test #mo(nt o$ 'roblems. !"is '"#se #s t"e $irst '#rt

o$ t"e 'roDe%t to be 'rogr#mmed #nd re)(ired t"e gre#test#mo(nt o$ time #nd te#ing. !"is #s most 'rob#bl& #

res(lt o$ "#ving to le#rn "o t"e to m#tri+ m#ni'(l#tion

%l#sses ,NM10 #nd NRIC- ored #nd "o t"e& %o(ld be

m#de to or toget"er. A gre#t de#l o$ e$$ort #s #stedtr&ing to (se NM10 m#tri+ t&'es t"#t ere s(''osed to

or it" NRIC. !"e version o$ t"e NRIC libr#r& e (sed

#s too ne so %onversions beteen d#t# t&'es "#d to be

"#ndled m#n(#ll& #s #s dis%(ssed e#rlier in t"eIm'lement#tion se%tion.

!"e eigen de%om'osition o$ t"e #ver#ge %ov#ri#n%e m#tri+too )(ite # long time#ro(nd 13 "o(rsto %om'lete #nd'rod(%e reli#ble res(lts. :e onl& mention t"is "ere

be%#(se #n #ltern#tive met"od #s 'ro'osed "i%" (sed #

red(%ed %ov#ri#n%e m#tri+ #nd 'romised sm#ller e+e%(tion

time b(t #ttem'ts to (nderst#nd t"e rel#tions"i' #mong t"ered(%ed m#tri+ #nd t"e origin#l m#tri+ ere "#rd to $ollo

#nd not ever %om're"ended 8. 9or t"is re#son e o'ted

to (se t"e br(te $or%e #''ro#%" to "#ndle eigen


13/13


de%om'osition o$ t"e l#rger #nd more time %ons(ming

m#tri+.

Prob#bl& t"e l#rgest "(m#n error #s rel#ted to t"e im#gest"#t ere (sed $or t"e ensemble. Re%#ll t"#t # tot#l o$ 160

im#ges ere t#en #t t"e beginning o$ t"e 'roDe%t. Initi#ll&

e "#d "o'ed to (se #ll 160 im#ges $or tr#ining '(r'oses

#nd t"en on%e %om'leted e o(ld sim'l& %re#te ne testim#ges. *(r $irst tr#ining #ttem't #%t(#ll& (sed #ll 160

im#ges "oever t"is %re#ted # 'roblem. A$ter o(r s&stem

#s tr#ined e #ttem'ted to t#e ne 'i%t(res in t"e s#me

$#s"ion e "#d #t t"e beginning o$ t"e 'roDe%t.n$ort(n#tel& t"e nel& %re#ted test im#ges ere too

di$$erent in orient#tion #nd lig"ting %onditions to be

re%ognied %orre%tl&. 9or t"is re#son e %"ose to retr#in #

se%ond time (sing 140 im#ges #nd to "old #side toim#ges $rom ever& letter %l#ss "i%" o(ld serve #s o(r

test im#ges. Sin%e t"ese im#ges ere t#en #t t"e s#me

time #s t"e rest o$ t"e ensemble t"e& "#d simil#r

orient#tions #nd lig"ting %onditions. As s(%" t"e&

'rovided # better b#sis to test o(r s&stem #nd 'rod(%e moreinteresting res(lts. I$ time "#d 'ermitted e o(ld "#ve

lied to tr#in m(lti'le versions #nd orient#tions o$ #ll letter

%l#sses to m#e t"e s&stem #s rob(st #s 'ossible.Interestingl& t"e tr#ining '"#se $or t"e 160 im#ge

ensemble 'rod(%ed t"ree 'rim#r& eigenve%torst"e s#me

n(mber #s t"e 140 ensemble.

:"ile t"e %om'#rison o$ re%onstr(%ted im#ges to origin#l

im#ges 'rovided 'roo$ t"#t t"e %"osen t"ree eigenve%tors

ere s#tis$#%tor& it o(ld "#ve been even better to in%l(de

error estim#tion. Sirovi%" #nd irb& 6 did t"is #nd ere#ble to s#& it" %ert#int& t"#t # re%onstr(%ted im#ge #s

it"in # %ert#in 'er%ent#ge o$ simil#rit& o$ t"e origin#l.!ime %onstr#ints 'revented t"is $rom being done #nd

em'"#sis #s inste#d 'l#%ed ('on delivering res(lts o$more interesting #nd im'ort#nt '#rts o$ t"e s&stem.

!"e most obvio(s im'rovement to o(r s&stem o(ld be t"e

in%l(sion o$ more letter %l#sses. S%o'e limited t"e n(mbero$ letters to onl& voels $or o(r investig#tion b(t

%om'(t#tion#ll& t"e b(ilding o$ # ne #ver#ge %ov#ri#n%e

m#tri+ #nd t#ing t"e eigen de%om'osition $or more

tr#ining d#t# is not b#d. Re%#ll t"#t t"e tr#ns'osing ste' to&ield t"e #ver#ge %ov#ri#n%e m#tri+ ill #l#&s ret(rn #

30E2 b& 30E2 m#tri+ no m#tter "o m#n& tr#ining im#ges

#re in%l(dedW 'rovided t"e im#ge sie is 74 ros b& 48%ol(mns.

*t"er 'ossible im'rovements %o(ld in%l(de sign l#ng(#ge

re%ognition $rom # movie stre#m #nd t"e #bilit& to

inter'ret #nd %l#ssi$& more idel& (sed 'i%t(re $ile $orm#tss(%" #s PG or GI9.

CNCLUSIN

:e "#ve s"on t"#t b& (sing 'rimitive %om'(ter vision

PCA te%"ni)(es t"#t ere rese#r%"ed #lmost to de%#des

#go e %o(ld still %re#te # oring sign l#ng(#gere%ognition s&stem. *(r limited s(%%ess it" t"is rese#r%"

"el's to 'rove t"#t t"ese te%"ni)(es #re still (se$(l tod#&

#nd %#n be e+tended $(rt"er to re%ognie ot"er "#nd s"#'es

#nd/or gest(res.

R#%#R#NC#S

1 Anton o#rd. lement#r& ine#r Algebr#. Et" ed.

Ne Bor= o"n :ile& Y Sons In%. 1554.

2 Co"oon #mes #nd #%

Documents

Pca Classify