Experimental Study on Performance Evaluation of ...maeum.ipdisk.co.kr/publist/VOL1/maeumnet/data/Adobe PDF 로우데이터...Experimental Study on Performance Evaluation of Fingerprint

Experimental Study on Performance Evaluation of

Fingerprint Readers

김학일 ⋅ 전성욱 ⋅ 강효섭 (Inha Univ.)정용화 ⋅ 길연희 ⋅ 안도성 (ETRI)김재성 ⋅ 신대철 ⋅ 심상옥 (KISA)

Background (1/2)

§ Why do we need performance evaluation?u To compare technologies quantitatively and

objectivelyu To reduce developing-cycle by automating

evaluation processu To increase user’s reliability to technologiesu To develop and protect the market

Background (2/2)

§ Requirements for Performance Evaluation u Generalityu Expertise u Fairness u Reliability

§ Component of Performance Evaluationu Evaluation Target u Evaluation Method u Evaluation Index u Database

Research Objectives

§ To study comparatively on characteristics of fingerprint readers of various sensing mechanisms§ To provide a framework of evaluating the

performance of fingerprint readers§ To demonstrate our own Best Practices in

testing and reporting performance of fingerprint readers

Research Scopes

§ Comparative evaluation of performance in liveness detection§ Quantitative comparison of fingerprint

image quality according to environmental factors as well as human factors§ Evaluation of proper resolution for infant

fingerprints

Process of Performance Evaluation

Evaluation Target

Formation of Evaluation Environment

& Database

EvaluationMethod

Evaluation Index

Evaluation

Report of Result

Evaluation Targets(Fingerprint Readers)

Sensor

Image

XXOXLivenessDetection

500403250500DPI

DCBACompany

ThermalTactileCapacitiveOptical

Performance Evaluation according to Environmental and Human Factors

Technical Evaluation

Fingerprint Sensor Module

Formation of Evaluation Environment & Database

TemperatureHumiditySkin HumidityPressure

Image Quality

Comparison byVisual Inspection

Image Quality Measures

§ Why do we need to measure image quality?u It affects the performance of feature extraction.u It is often utilized for failure-to-enroll or failure-to-

acquire.u It is affected by human factors as well as environmental

factors.u It indicates the performance of fingerprint readers

according to changes in user behavior and environment.

Criteria for Quality of Fingerprint Image [4-7]

§ How distinct ridges and valleys appear in gray levels.§ How apparent ridge directions are. § How large the portion of foreground is

compared to the background.

Methods of Image Quality Measurement [10]

2.23 (44.6)0.07 (1.4)0.05 (1)Processing Time(sec)

ComputationThresholding involved

Unable to reflect ridge-valley structure

Thresholding involved

Weakness

Feature extraction algorithm using Gabor filter

Robust to ridge-valley structures

Simple, fastStrength

Definition of ridge direction

Contrast of gray values between ridge and valley

Distribution and deviation of local gray values

Measuring factor

Energy distribution of 8-direction Gabor filter

Distinctiveness in ridge-valley structure

Ratio of foreground and background

Underlying idea

Gabor filter based

Directional contrast

Standard deviation

based

Standard Deviation-based

3216

Block Scanning

16

32 ( ) ( )

indexscan :

block i of average gray value :

),(at pixel of gray value :

,...,1 641

th

32

1

32

1

2

N

G

yxG

NiGGV

i

xy

x yixyi =−= ∑∑

= =

constant gnormalizin :

1

1

C

VC

IN

ii∑

==


Slit Window

( ) ( )

( )NkSSD

iPGS

k

jiji

,,1

8,,1

minmax

4

1

K

K

=−=

== ∑=

∑=

=N

kkD

CI

1

1

Gabor filter based

1θg2θg

3θg4θg

5θg6θg

7θg8θg

)2exp()()(

21

exp),,,,,( 2

2

2

2

ky

k

x

kyxi fxi

yxfyxh θ

θθ πσσ

σσθ ×

+−=

∑ ∑−

−=

−

−=

++=12/

2/

12/

2/

),,,,,(),(),,,,,(w

wx

w

wxyxkyxk fyxhyYxXIfYXg σσθσσθ

i=1,2,3… .m : m direction, m=8, f=frequency, θk : kth direction of gabor filter

ω= 23 : size of gabor filter

100),,,,,(

×=NC

fYXgI yxk σσθ

I : image quality measureN : scan number, C: normalize constant

List of Factors [8]

71 ~ 100%High

36 ~ 70%Middle

0 ~ 35%LowSkinHumidity

Strongly pressingHigh

Normally pressingMiddle

Softly pressingLow

Pressure

Hum

an

0 ~ 100%Humidity

SummerAbove 30

Room Temperature20~30

Spring or fall10~20

Beginning of the spring or end of the fall0~10

WinterBelow 0Tem

peratu

re(oC

)

Environ

men

t

StateFactor

Database Distribution(Sensor A, 504 images)

-5 0 51015 2025

3035

40

1

2

3

020

4060

80100

Skin Hum

idity(%

)

Fin

ger

Pre

ssur

e

Temperature(? )

Visual Inspection (1/8)

§ Temperature : 17.3℃ (Sensor D)

Image

422911Quality Measure (DC)

GoodNormalBadQuality


Image Quality : Good(G), Normal(N), Bad(B)Image Quality : Good(G), Normal(N), Bad(B)

Temperature

0

10

20

30

40

50

60

70

80

90

100

-1.8(

B)7.5

(B)

18.3(

B)

26.7(

B)

35.2(

B)

-1.8(

N)7.5

(N)

18.3(

N)

26.7(

N)

35.2(

N)

-1.8(

G)

7.5(G

)

18.3(

G)

26.7(

G)

35.2(

G)

Qua

lity

A B

C D


§ Skin Humidity : 35%~70% (Sensor B)

Image





Skin Humidity

0

10

20

30

40

50

60

70

80

90

100

L(B) M(B) H(B) L(N) M(N) H(N) L(G) M(G) H(G)

Quality

A B

C D


§ Low Pressure (Sensor A)

Image





Pressure

0

10

20

30

40

50

60

70

80

90

100

L(B) M(B) H(B) L(N) M(N) H(N) L(G) M(G) H(G)

Qua

lity

A B

C D

Image




§ Air Humidity : 40%~50% (Sensor D)



Air Humidity

0

10

20

30

40

50

60

70

80

90

100

20~3

0(B)

30~4

0(B)

40~5

0(B)

50~6

0(B)

20~3

0(N)

30~4

0(N)

40~5

0(N)

50~6

0(N)

20~3

0(G)

30~4

0(G)

40~5

0(G)

50~6

0(G)

Quality

A B

C D

Effect of Temperature to Optical Sensor

Temperature

0

2

4

6

8

10

12

14

16

18

20

8 12 16 20 24 28 32 36 40 44 48 52 56 60 64 68 72 76 80 84 88

Image Quality

Perc

ent

(%)

-1.8(90)7.5(108)18.3(144)26.7(126)35.2(126)

Below zero

Effect of Air Humidity to Optical Sensor

Air Humidity

0

2

4

6

8

10

12

14

16

18

20

8 12 16 20 24 28 32 36 40 44 48 52 56 60 64 68 72 76 80 84 88

Image Quality

Perc

ent

(%)

20~30(90)30~40(108)40~50(270)50~60(126)

Effect of Finger Pressure to Optical Sensor

Pressure

0

5

10

15

20

25

8 12 16 20 24 28 32 36 40 44 48 52 56 60 64 68 72 76 80 84

Image Quality

Perc

ent

(%)

LOW(198)

MIDDLE(198)

HIGH(198)

Effect of Skin Humidity to Optical Sensor

Skin Humidity

0

2

4

6

8

10

12

14

16

18

20

8 12 16 20 24 28 32 36 40 44 48 52 56 60 64 68 72 76 80 84 88

Image Quality

Perc

ent

(%)

LOW(111)

MIDDLE(257)HIGH(226)

Effect of Temperature to Semiconductor Sensor

Temperature

0

2

4

6

8

10

12

14

16

18

20

28 32 36 40 44 48 52 56 60 64 68 72 76 80 84 88 92 96 100

Image Quality

Perc

ent

(%)

-1.8(90)7.5(108)18.3(144)26.7(126)35.2(126)

Effect of Air Humidity to Semiconductor Sensor

Air Humidity

0

2

4

6

8

10

12

14

16

18

28 32 36 40 44 48 52 56 60 64 68 72 76 80 84 88 92 96 100

Image Quality

Perc

ent

(%)

20~30(90)30~40(108)40~50(270)50~60(126)

Effect of Finger Pressure to Semiconductor Sensor

Pressure

0

2

4

6

8

10

12

14

16

18

20

28 32 36 40 44 48 52 56 60 64 68 72 76 80 84 88 92 96 100

Image Quality

Perc

ent

(%)

LOW(198)

MIDDLE(198)

HIGH(198)

Effect of Skin Humidity to Semiconductor Sensor

Skin Humidity

0

2

4

6

8

10

12

14

28 32 36 40 44 48 52 56 60 64 68 72 76 80 84 88 92 96 100

Image Quality

Perc

ent

(%)

LOW(150)

MIDDLE(312)

HIGH(132)

Effect of Temperature to Tactile Sensor

Temperature

0

5

10

15

20

25

30

35

8 12 16 20 24 28 32 36 40 44 48 52

Image Quality

Perc

ent

(%)

-1.8(90)7.5(108)18.3(144)26.7(126)35.2(126)

Effect of Air Humidity to Tactile Sensor

Air Humidity

0

5

10

15

20

25

30

35

8 12 16 20 24 28 32 36 40 44 48 52

Image Quality

Perc

ent

(%)

20~30(90)30~40(108)40~50(270)50~60(126)

Effect of Finger Pressure to Tactile Sensor

Pressure

02468

101214161820222426

8 12 16 20 24 28 32 36 40 44 48 52

Image Quality

Perc

ent

(%)

LOW(198)

MIDDLE(198)

HIGH(198)

Effect of Skin Humidity to Tactile Sensor

Skin Humidity

0

5

10

15

20

25

30

8 12 16 20 24 28 32 36 40 44 48 52

Image Quality

Perc

ent

(%)

LOW(166)

MIDDLE(245)

HIGH(183)

Effect of Temperature to Thermal Sensor

Temperature

0

2

4

6

8

10

12

14

16

18

20

8 12 16 20 24 28 32 36 40 44 48 52 56 60 64 68 72 76 80 84

Image Quality

Perc

ent

(%)

-1.8(90)7.5(108)18.3(144)26.7(126)35.2(126)

Effect of Air Humidity to Thermal Sensor

Air Humidity

0

2

4

6

8

10

12

14

16

18

20

8 12 16 20 24 28 32 36 40 44 48 52 56 60 64 68 72 76 80 84

Image Quality

Perc

ent

(%)

20~30(90)30~40(108)40~50(270)50~60(126)

Effect of Finger Pressure to Thermal Sensor

Pressure

0

2

4

6

8

10

12

14

16

18

20

22

12 16 20 24 28 32 36 40 44 48 52 56 60 64 68 72

Image Quality

Perc

ent

(%)

LOW(198)

MIDDLE(198)

HIGH(198)

Effect of Skin Humidity to Thermal Sensor

Skin Humidity

0

2

4

6

8

10

12

14

16

18

20

8 12 16 20 24 28 32 36 40 44 48 52 56 60 64 68 72 76 80 84

Image Quality

Perc

ent

(%)

LOW(76)

MIDDLE(252)

HIGH(266)

Discussions (1/3)

§ Environmental Factorsu Temperature

l Quality decreases when the temperature goes below zero due to dryness of skin

l On the whole, all sensors produce good quality images above the room temperature

u Air Humidityl No change according to humidity levels

Discussions (2/3)

§ Human Factorsu Pressure

l Image quality is always good with the middle levell For optical sensor, foreground image gets smaller for

the low pressure.l Semiconductor sensor produces good images not

only with the middle pressure but also with the high pressure

l Tactile gives better image at the low pressure than at the high pressure.

Discussions (3/3)

§ Human Factorsu Skin Humidity

l Skin humidity affects most to the image quality of optical and tactile sensors.

l Because of sweeping type, thermal sensor is less affected.

Evaluation of Proper Resolution for Infant Fingerprints

Technical Evaluation

Fingerprint Sensor Module

Formation of Evaluation Environment & Database

Resolution

Image Quality&

Accuracy in Feature Extraction

DET curves

Evaluation Targets

USB

211mA

-20~65℃

19.3x14.2

240x315

400

Tactile

USBUSBUSBUSBInterface

< 200mW<110mA< 150 mW20mWPower

consumption

0~50℃-40~60℃-40~85℃0~70℃Operational temperature

11.4x11.412.9x15.29.2x13.80.4x14Sensor size(mm)

300x300248x292256x288312x452Image size

500500508500Resolution (dpi)

SemiconOpticalCCDThermalType

Infant fingerprint database§ Total 4500 images

(50 peopleⅹ6 fingersⅹ 3 trialsⅹ5sensors)

§ Composition

Sample Image (Optical)

Adult

Infant

BadNormalGood

Sample Images (Semiconductor)

bad

Adult

Infant

NormalGood

Adult

Infant

BadNormalGood

Sample Images (Tactile)

Sample Images (Thermal)Bad

Adult

Infant

NormalGood

Sample Images (CCD)Bad

Adult

Infant

NormalGood

Comparison of Image Quality Measures (Infant)

00.090(c)Bad

0.480.430.2(b)Normal

0.740.640.47(a)Good

Infant

Gabor filterBased


Standard deviation

Based

Image quality measure

(a) (b) (c)

Comparison of Image Quality Measures (Adult)

0.390.070.32(c)Bad

0.630.390.68(b)Normal

0.790.530.75(a)Good

Adult

Gabor filterBased


Standard deviation

Based


(a) (b) (c)

Comparison of IQ Measures (Thermal)

1

dir_iq

Correlation

1gabor_iq

.957031std_iq

.85706.90106dir_iq

gabor_iqstd_iqinfant

(Infant)

00.05.05(c)Bad

0.180.230.17(b)Normal

0.570.650.47(a)Good

Infant

Gabor filterBased


Standard deviation

Based


(a) (b) (c)

(Adult)

00.010(c)Bad

0.490.480.29(b)Normal

0.740.860.82(a)Good

Adult

Gabor filterBased


Standard deviation

Based


(a) (b) (c)

Comparison of IQ Measures (Optical)

1

.86105

dir_iq

1gabor_iq

.93099std_iq

.772991dir_iq


Correlation

(Infant)

0.080.120.06(c)Bad

0.330.330.23(b)Normal

0.810.770.56(a)Good

Infant

Gabor filterBased


Standard deviation

BasedImage quality measure

(a) (b) (c)

(Adult)

0.290.20.07(c)Bad

0.710.70.56(b)Normal

111(a)Good

Adult

Gabor filterBased


Standard deviation


(a) (b) (c)

Comparison of IQ Measures (Semiconductor)

1

dir_iq

1gabor_iq

.756851std_iq

.3958.71154dir_iq


Correlation

(Infant)

0.140.140.1(c)Bad

0.410.310.27(b)Normal

0.50.390.45(a)Good

Infant

Gabor filterBased


Standard deviation


(a) (b) (c)

(Adult)

0.190.070.08(c)Bad

0.480.350.35(b)Normal

0.610.540.56(a)Good

Adult

Gabor filterBased


Standard deviation


(a) (b) (c)

Comparison of IQ Measures (Tactile)

1

dir_iq

1gabor_iq

.924251std_iq

.85194.89556dir_iq


Correlation

(Infant)

0.0600(a)Bad

0.10.150.04(b)Normal

0.310.340.18(a)Good

Infant

Gabor filterBased


Standard deviation


(a) (b) (c)

(Adult)

0.120.120.03(a)Bad

0.330.390.21(b)Normal

0.640.510.47(a)Good

Adult

Gabor filterBased


Standard deviation


(a) (b) (c)

Estimation of Ridge Distance[12]

pixelsadandpixelsgaand

pixelsdefdabcaghig

3030

15

≤≤

======

Target point

Target point

Thinned ridge

Ridge distance estimationComputational structure of ridge distance

Comparison of Ridge Distances

Method

7.0707.15.87.687.7Manual

6.9956.966.77.057.27Automatic Infant

8.7588.67.78.99.83Manual

8.2958.367.738.438.66Automatic Adult

AVERAGEOpticalTactileSemiconThermal

Infant:Adult= 0.8:1

Evaluation by Feature Extraction Accuracy

§ TM : Number of true minutiae on fingerprint image§ PM : Number of true minutiae extracted by algorithm§ DM : Number of true minutiae lost by algorithm§ FM : Number of false minutiae extracted by algorithm

TM

PM=ratioextractionminutiaeTrue

TM

DM=ratioextractionminutiaeDropped

FMPM

FM

+=ratioextractionminutiaeFalse

Performance Evaluation of Feature Extraction

0.5940.7490.594 Adult

0.2090.2470.209 InfantOptical

0.4830.7160.530 Adult

0.1790.1890.169 InfantSemicon

0.5910.5400.611 Adult

0.3320.2470.304 InfantThermal

1 - Dropped minutiae

extraction ratio

1 - False minutiaeextraction

ratio

True minutiae extraction

ratioSensor

For 500 dpi



Ridge distance setting

0.4200.5650.420 6.250

0.2090.2470.209 8.750 Optical

0.3340.4310.334 6.250

0.1790.1190.169 8.750 Semicon

0.5440.4490.544 7.000

0.3320.2470.304 8.750 Thermal

1 - Dropped minutiae

extraction ratio

1 - False minutiae

extraction ratio

True minutiae extraction ratioSensor

For infant fingerprints


Performance Evaluation of Matching

Performance Evaluation of Matching

Score Distribution according to Resolution

EER according to Resolution

Discussions

§ Experimental comparison of ridge intervalu Adult : infant ≅ 1 : 0.8u Fingerprint image resolution affects the

performance of feature extraction as well as matching accuracy.

u It is demonstrated that the increase in image resolution of infant fingerprints improves matching accuracy.

u Different sensors require different levels of compensation for image resolution.

Conclusions

§ Carried out evaluation and comparison of characteristics of fingerprint readers in terms of liveness detection, image quality, and matching errors§ Demonstrated experimental procedures

for performance evaluation of fingerprint reader modules§ Experimental results can be utilized for

developing fingerprint reader modules.

References (1/3)

[1] Stephanie Schuckers, Larry Hornak, Tim Norman, Reza Derakhshani and Sujan Parthasaradhi, "Issues for Liveness Detection in Biometrics", Proceedings of the Biometric Consortium Conference, Vol.1, 2002.

[2] Tsutomu Matsumoto, Hiroyuki Matsumoto, Koji Yamada and Satoshi Hoshino, "Impact of Artificial "Gummy" Fingers on Fingerprint System", Optical Security and Counterfeit Deterrence Techniques IV, Vol. 4677, pp275-289, 2002.

[3] Dosung Ahn, “Study on fingerprint recognition algorithm based on cliques and its performance evaluation system”, Ph D dissertation Submitted to Inha University, 2001.

[4] ANSI/NIST ITL 1-2000:American National Standard for information System-Data Format for interchange of Fingerprint , Facial & Scar Mark & Tattoo (SMT) Information,2000

References (2/3)

[5] Standard for the Electronic Submission of Fingerprint Cards to the FBI, online at http://www.biometrics.org/REPORT/FBIfp.html

[6] CJIS(Criminal Justice Information Services)-Electronic Fingerprint Transmission Specification, Appendix F&G, January, 1999

[7]Interim IAFIS Image Quality Specification for Scanners, online at, http://www.engr.sjsu.edu/biometrcs/publications_appendixg.html

[8] CESG/BWG Biometric Test Programme, online at, http://www.cesg.gov.uk/biometrics/pdfs/BiometricTestReportpt1.pdf.

References (3/3)

[9] A.M. Bazen and S.H. Gerez, “Segmentation of Fingerprint Images”, Proc. ProRISC2001, 12th Ann. Workshop Circuits, Systems and Signal Processing, Nov. 2001.

[10] Wonchurl Jang, “An Adaptive Feature Extraction Algorithm for Automatic Fingerprint Recognition”, Master’s Thesis, Yonsei University, 2002.

[11] L. L. Shen, A. Kot and W.M.Koo,“Quality Measures of Fingerprint Images”, Third International Conference, AVBPA 2001, Halmstad, Sweden, Proceedings, pp. 266-271, Jun. 2001.

[12] Jisung Yang, ”False Minutiae Removal Algorithm for Automatic Fingerprint Recognition Systems”, Master’s Thesis,

Inha University, 2000.

Documents

Experimental Study on Performance Evaluation of ...maeum.ipdisk.co.kr/publist/VOL1/maeumnet/data/Adobe PDF 로우데이터...Experimental Study on Performance Evaluation of Fingerprint