85
FINGERPRINT ENHANCEMENT Frequency Domain

FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

  • Upload
    hanhi

  • View
    228

  • Download
    0

Embed Size (px)

Citation preview

Page 1: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

FINGERPRINT ENHANCEMENTFrequency Domain

Page 2: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

Outline

Fingerprint Image Enhancement Prior Related WorkProposed Algorithm: STFT AnalysisExperimental Evaluation

Page 3: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

Need for Enhancement

What you see What you ‘think’ you see

Page 4: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

Reality: What you usually get..

High contrast print Typical dry print

Low contrast print Typical Wet Print Creases

Faint print

Page 5: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

Prior Related Work

ChallengesFingerprint image is non stationary (has dominant local orientation and frequency)General purpose image processing algorithms are not usefulTraditional operators and filters assume gaussian noise model‘Noise’ in fingerprint images consists mostly of ridge breaks

Contextual FiltersExisting techniques are based on ‘contextual’ filteringFilter parameters are adapted to each local neighborhoodFilters themselves may be spatial or Fourier domain basedFilter parameters in ‘unrecoverable’ regions can be interpolated based on its neighbors

Page 6: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

Spatial Filtering(Yang et.al 1996, Greenberg et. Al 1999) proposed local anisotropic filteringFilter kernel adapts at each pixel location

( ) ( )⎭⎬⎫

⎩⎨⎧ −

+⋅−

−+=⊥

)().(

)()(exp)(),(

02

20

02

20

00 xnxx

xnxxxxVSxxf

σσρ

Parameters: radial extent of the filter: vector parallel to the ridge direction ridge direction: vector perpendicular to the ridge direction, , : shape parameters

In our case, S= -2, V = 10 , = 4, = 2

ρn⊥n

ρ )( 02 xσ

)( 02 xσ )( 0

2 x⊥σ)( 0

2 x⊥σ

Page 7: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

Spatial Filtering (cont.)

Even Symmetric Kernel

Fourier spectrum showing the localization

( ) ( )[ ]yxi

xyxx

eeyxG 002

20

2

20

21),( νξβσ

π

πσβ+⎥

⎥⎦

⎢⎢⎣

⎡ −+

−−

=

Hong et al, 96/98 proposed the use of Gabor filters for enhancementGabor filter has the best joint space-frequency localizationThe filter is aligned with the direction of the ridgesDoes not handle high curvature regions well due to block wise approach. Angular and radial bandwidths are constant.

Page 8: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

Fourier Domain FilteringSherlock et al 94, proposed the use of Fourier domain filtering The image is convolved with a filter bank of directionally selective filtersImage enhanced by selecting a linear combination of filter responsesHas high space complexity, requires estimation of core/delta locations

Watson et al. 94, proposed the use or ‘root filtering’ for enhancement.(Pseudo matched filter)Does not require the computation of orientation images

{ }{ }),(),(

),(),(),( 1

yxIFFTvuFvuFvuFFFTyxI k

enh

=

= −

Root Filtering

Fourier Domain Filtering

Page 9: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

Traditional Approaches

Local Ridge SpacingF(x,y)

Projection Based Method

EnhancementFrequency/Spatial

Local Orientationθ(x,y)

Gradient Method 87] Witkinsand [Kaas

tan21

),(),(

),(),(2

1

22

⎟⎟⎠

⎞⎜⎜⎝

⎛=

−=

=

∈ ∈

∈ ∈

∑∑

∑∑

xx

xy

Wu Wvyxxx

Wu Wvyxxy

GG

vuGvuGG

vuGvuGG

θ

[Ratha et al 95]

Page 10: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

Proposed Approach: Overview

STFT Analysis

Frequency Image

RegionMask

Orientation Image

CoherenceImage

Fourier domain Enhancement

Page 11: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

Surface Wave ModelFingerprint ridges can be modeled as an oriented wave

[ { } ]

=

=+=

),(

),(

)sin()cos(2cos),(

yx

yx

f

yxfAyxiθ

θθπLocal ridge orientation

Local ridge frequency

Local Neighborhoods

Validity of the model

Surface wave

Page 12: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

STFT Analysis

Fingerprint image is non stationary, so we require both space and frequency resolution: time frequency analysisSTFT in 1D

STFT in 2D

dtetwtxX tjωτωτ −∞

∞−

∗∫ −= )()(),(

∫ ∫∞

∞−

∞−

+−∗ −−= dxdyeyxwyxIX yxj )(212121

21),(),(),,,( ωωττωωττ

Page 13: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

Parameter Estimation

Paradigm: The Fourier domain response can be viewed as a distribution of surface waves. Each term F(r, θ) corresponds to a surface wave of frequency 1/r and orientation θWe seek to find the most likely surface wave and hence estimate the dominant direction and frequencyWe can represent the Fourier spectrum in polar form as F(r,θ) The power spectrum is reduced to a joint probability density function using

The angular and frequency densities are given by marginal density functions

∫ ∫=

r

drdrF

rFrp

φ

φθ

θθ 2

2

),(

),(),(

∫=r

drrfp ),()( θθ ∫=θ

θθ drfrp ),()(

Page 14: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

Ridge Orientation Image

⎟⎟⎟⎟

⎜⎜⎜⎜

=Ε∫

∫−

φ

φ

θθθ

θθθθ

dp

dp

)()2cos(

)()2sin(tan

21}{ 1

( )( ) ⎟⎟

⎞⎜⎜⎝

⎛∗∗

= −

),(),(2cos),(),(2sintan

21),(' 1

yxWyxOyxWyxOyxO

Page 15: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

Region Mask

⎟⎟⎠

⎞⎜⎜⎝

⎛= ∫∫ θθ drdrFE

2),(log

The surface wave approximation does not hold in the background regionThe region mask is obtained by simple thresholding of the block energy image

Page 16: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

Coherence Image

• Block processing is unreliable in regions of high curvature• Sherlock and Monro 94, relax filter parameters near the singular locations• Estimation of singular point is difficult in poor images!• We use an angular coherence measure proposed by Rao and Jain 90

WW

yxOyxOyxC Wji

×

−=∑∈),(

00

00

)),(cos()),(cos(),(

Page 17: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

Frequency Image

∫ ⋅=Εr

drrprr )(}{∑ ∑

∑ ∑+

−=

+

−=

+

−=

+

−== 1

1

1

1

1

1

1

1

),(),(

),(),(),(),(' x

xu

y

yv

x

xu

y

yv

vuIvuW

vuIvuWvuFyxF

[Jain et al 00]

Page 18: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

Enhancement

1994] al, et.[Sherlock frequency mean :}{r n,orientatiomean :}{bandwidth,angular :,bandwidth radial:

otherwise 0

if 2

)(cos)( )()(

)()(

)()(),(

c

2

2222

2

rEEr

Hrrrr

rrrH

HrHrH

c

BWBW

BWcBW

C

nc

nBW

nBW

r

r

==

⎪⎩

⎪⎨⎧ ≤−

−=

−+=

⋅=

φφφ

φφφφφφπ

φ

φφ

φ

φ

Page 19: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

Additional Enhancement Results

Page 20: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

Qualitative Comparison

Original Image Root Filtering

Page 21: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

Qualitative Comparison(cont.)

Gabor Filter based Enhancement Proposed Approach

Page 22: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

Objective Evaluation

• We evaluated the effect of enhancement on 800 images from FVC2002 DB3• The evaluation consists of 2800 genuine test and 4950 impostor tests• It can be seen that the matcher performance improves with enhancement

Page 23: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

Feature Extraction

Page 24: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

Outline

Minutia Feature ExtractionPrior Related WorkChain code contourExperimental Evaluation

Page 25: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

Background

Minutiae represent local discontinuities in ridge flowMinutiae features are the most widely used fingerprint representationThere are several standards such as CBEFF (file format) and ANSI-NIST (interchange format) standards for minutiae based fingerprint representation

Minutiae extraction approaches may be broadly categorized intoBinarization based approachesDirect gray scale extraction

Page 26: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

Prior Related Work

Binarization ApproachesMINDTCT,NIST NFIS, (Garris et. Al, 02)

Directionally adaptive binarizationTemplate matching is used to detect minutiaeGreedy approach to minutia detection leads to false positives.Extensive post processing is required to eliminate false positives

Adaptive Flow Orientation technique (Ratha et. al., 95)Binarization is performed by peak detection Peak detection leads to false positives in regions of poor ridge constrastThinning and morphological post processing shift minutia location.

Peak Detection (Domeniconi 98)Fingerprints is treated as a 3D surfaceRidges are detected as peaks and saddle points on this surfaceUses Hessian matrix of the gradients to determine dominant directionsCannot compensate for ridge breaks

Direct Gray Scale Ridge Following Ridge Following (Maio and Maltoni 97, Jiang and Yau 01)

Based on ridge pursuitHas low computational complexity.Cannot handle poor contrast prints and images with poor ridge structure. Relies on a good orientation map for ridge pursuit

Page 27: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

Binarization Method

Binarization Thinning Minutia DetectionAcquisition

Page 28: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

Proposed Approach: Chain Code Contours

Provides a lossless description of the contour and also gives direction and curvature information.Translation and rotation invariantUsed in computer vision for encoding object boundariesUsed for character recognition (Madhavanth et. al 99)

Page 29: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

Minutiae Detection using Chain Codes

Minutiae are encountered as points of ‘significant’ turn on the contourLeft turn: Ridge endingRight turn: Bifurcation ( )

( ) 0 : Left turn

0:Right turn

OUTIN

OUTIN

PPsign

PPsignrr

rr

Page 30: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

Determining Turn Points

Th

OUTIN

OUTIN

PPPP

θθ

θ

>

•=

:t turnSignifican

)cos( rr

rr

⎟⎟⎠

⎞⎜⎜⎝

⎛=

+−=

X

Y

OUTIN

PP

PPP

1tan

2

φ

rrr

Page 31: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

Post processing

• Feature Extraction errors• Missing minutiae • Spurious minutiae

• Spurious minutia can be removed using post processing• Heuristic rules:1. Merge minutiae that are a certain distance of each other and have similar angles2. Discard minutiae whose angles are inconsistent with ridge direction3. Discard all border minutia4. Discard opposing minutiae within certain distance of each other

Page 32: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

Results

Page 33: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

Results (cont.)

Page 34: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

Experimental Evaluation

Test Data150 prints from FVC2002(DB1) were randomly selected for evaluation.Ground truth was established using a semi automated truthing tool. Results compared using NIST NFIS open source software.

MetricsProposed by Sherlock et. Al 94Sensitivity: Ability of the algorithm to detect true minutiaeSpecificity : Ability of the algorithm to avoid false positivesFlipped : Minutiae whose type has been exchanged

Truth Ground:N Exchanged, :E positives, False :FP Negatives, False :FN

,1,1NEFlipped

NFPySpecificit

NFNySensitivit =−=−=

Page 35: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

Quantitative Analysis : Results

Examples

File Name NIST Proposed method

Actual TP FP M F TP FP M F

10_8.tif 18 16 8 2 1 17 0 1 1

11_6.tif 50 40 4 10 2 41 4 9 4

12_8.tif 29 22 5 7 3 22 3 7 1

13_6.tif 35 28 10 7 4 28 10 7 2

14_6.tif 44 34 12 10 6 37 13 7 5

15_7.tif 38 37 7 1 5 37 3 1 0

16_7.tif 41 35 12 6 5 36 8 5 8

17_6.tif 43 35 16 8 11 36 7 8 11

18_8.tif 34 31 7 3 4 32 6 2 1

19_7.tif 35 26 8 9 3 31 6 4 5

Page 36: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

Results

Summary resultsCount TP(ANSI) > proposed : 40 of 150Count E(ANSI) < proposed : 40 of 150

Metric NIST Proposed

Sensitivity(%) 82.8 83.5

Specificity(%) 77.2 76.8

Flipped(%) 12.0 10.9

Sensitivity distribution Overall statistics

Page 37: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

FINGERPRINT MATCHING

Page 38: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

Outline

Fingerprint Image Enhancement Minutiae ExtractionMatching Algorithm

Prior Related WorkNew Representation: K-pletLocal Matching: Dynamic ProgrammingConsolidation: Coupled BFSExperimental Evaluation

Page 39: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

Minutiae Based Matching

ChallengesMinutiae extraction is error prone is low quality imagesNot robust to non-linear distortion.Intra-user variation

Page 40: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

Challenges: Non-linear Distortion

Page 41: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

Challenges: Quality and Intra-user variance

Variation in quality

Intra-user variation

Page 42: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

Prior Related Work

Minutiae based matching algorithms can be broadly categorized as

Global Matching : All points are aligned at onceImplicit Alignment: Alignment and matching done simultaneouslyExplicit Alignment: Has a separate pre-alignment stage

Local Matching: Only local neighborhoods are matchedFeatures are chosen to be rotation and translation invariantMore resilient to non-linear deformationChallenges in local matching approaches

Representation Local Matching Consolidation

Page 43: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

Prior Related Work (cont.)

Global MatchingPoint correspondences not known : combinatorial problem

Relaxation approach (Ranade and Rosenfield 93)Likelihood of each match is either decreased or increased at each iteration based on compatibility of rest of the pointsIterative approach makes it too slow to be practical

Generalized Hough Transform (Ratha et al. 96)All possible transformation represented as a quantized search spaceSearches for the most optimal transform in the search spaceVery fast

Ridge Alignment (Jain et al. 97)Performs explicit alignment before matchingEach minutiae is associated with its ridge (represented by a curve)The alignment is based on ridge correspondenceGlobal matching is then performed using string edit distance

Page 44: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

Deformation ModelsRigid Transformation

Transformation parameters:

Affine Transformation

Non-rigid TransformationBazen and Garez (2002) used TPS to find arbitrary transformation. Point correspondences(pre-alignment) is still required

⎥⎦

⎤⎢⎣

⎡ΔΔ

+⎥⎦

⎤⎢⎣

⎡⎥⎦

⎤⎢⎣

⎡ −=⎥

⎤⎢⎣

⎡yx

yx

yx

)cos()sin()sin()cos(

''

θθθθ

[ ]θ,, yx ΔΔ

⎥⎦

⎤⎢⎣

⎡ΔΔ

+⎥⎦

⎤⎢⎣

⎡⎥⎦

⎤⎢⎣

⎡=⎥

⎤⎢⎣

⎡yx

yx

aaaa

yx

2221

1211

''

Page 45: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

Local Matching

(Jiang and Yau 00)11 dimensional local features derived from reference minutiae and two closest neighborsBest match is used only for explicit alignment

(Jea and Govindaraju 04)5 dimesional features Si (ri0, ri1, φi0, φi1, δi) derived from two closest neighborsAlignment is still required

(Ratha et al. 00)‘Star’ representation derived from all minutiae within a particular radiusConsolidation by checking consistency

(Garris et. al 03: BOZORTH3)Line featuresConsolidation by linking consisting matches

iM

1N

0N

0iϕ

1iϕ

0ir

1ir

Page 46: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

Proposed Algorithm

Representation K-PletFeatures invariant to rotation and translationLocal relationship formally represented by a directed graph

Local MatchingPosed as a string alignment problem and solved by dynamic programmingMatches all neighbors simultaneously

ConsolidationCoupled Breadth First SearchBreadth first search is used to propagate the matchesSimilar to human verificationNo explicit alignment required at any stage

Page 47: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

Neighborhood Representation: K-plet

Page 48: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

K-plet

r

Θ

Φ

Page 49: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

The ‘Graphical’ View

Page 50: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

Local Matching

All local neighbors have to be matched simultaneously. Greedy approach does not work when conflicts occurThese can solved by finding the alignment through optimization process such as by solving a string alignment problem

1Tm

2Tm

1Im

2Im

r

Example of alignment: S (acbcdb) – (ac__bcdb)T (cadbd) - (_cadb_d_)

Trivial solution requires exponential timeEach match is given a cost. Alignment solved through recurrence relation

⎪⎩

⎪⎨

⊗+−⊗+−

+−−=

])[,(]1,[)],[(],1[

])[],[(]1,1[max],[

jtjiDisjiD

jtisjiDjiD

σσσ

{ }( )

0])[,(,0)],[(otherwise

bounds within if ])[],[()(

=⊗=⊗⎩⎨⎧

=++−

jtisMISMATCHeMATCHjtis

dCdrCdC r

σσ

σφθ φθ

Page 51: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

Graphical Matching: Coupled BFS

Page 52: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

Coupled BFS

Page 53: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

Graphical Matching: Coupled BFS

Page 54: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

Coupled BFS

Page 55: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

Graphical Matching: Coupled BFS

Page 56: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

Graphical Matching: Coupled BFS

Page 57: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

Graphical Matching: Coupled BFS

Page 58: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

Experimental Evaluation800 prints from FVC2002(DB1)2800 genuine tests,4950 impostor testsCompared with BOZORTH 3

Error RatesBOZORTH3: 3.6% EER, 5.0% FMR100Proposed: 1.5% EER, 1.65% FMR100

Page 60: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

Correspondence Problem

1Tm

2Tm

1Im

2Im

r

a) Non-linear distortion

b) Non-trivial correspondence

c) Global consistency

d) Dynamic thresholds

e) No singular feature registration

Page 61: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

Brute-force MatchingChoose any pair of minutiae x and y from the images Q and R(Assume they are aligned)Convert minutiae in Q into polar coords w.r.t. to xConvert minutiae in R into polar coords w.r.t. to y

Compute COST matrix of minutiae pairs (A,B)

COST (rx, ry, φx, φy, δx, δy)O(N2) (Worst case)

Use the MCF to find best correspondence and total costO(nm2) == O(n3) (m = no. of edges; m ~ n)

Repeat for all pair alignments x and y

Choose the alignment <x,y> and correspondence with the lowest cos

Complexity: O(N2).(N2 + N3) = O(N5)

r0

r1

φ0

φ1

δ0δ1

A

BO

Page 62: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

[Capacity constraint]

[Skew symmetry]

[Flow conservation]

Minimum Cost Flow (MCF)

s t

1

2

4

3 5

16 (5.8)

13 (4.0)

5 (4.6)

10 (6.0)

8 (3.3)

5 (5.8)15 (4.6)

25 (7.2)

6 (6.8)

10 (4.5)

),(),(,, vuwvufVvu <=∈∀),(),(,, vufuvfVvu −=∈∀

∑∈

=−∈∀Vv

vuftsVu 0),(},,{

Ford-Fulkerson algorithm [1962] O(nm2)Cycle-canceling algorithm [Orlin et al., 2000] O(m (m + n log n) log (nU))

U is the maximum capacity of links (1 in our case)n = number of nodes; m = number of edges

Page 63: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

s t

Nq nodes from Q Nr nodes from R

1 (cij)

1 (0) 1 (0)

MCF and Fingerprint Matching

Links with cost less than a threshold are added to network.

Output:

Best correspondence

And its cost

Input:

COST matrix

Page 64: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

Similarity Scores

Only number of matched minutiae (n) is not reliable

Use NN to combine the following

Number of matched minutiae (n)Numbers of features in overlapping parts of the convex hulls formed by the matched minutiae: Oq, Or

Cost from MCF

Traditionally: Alternate:rqNN

n2

rq NNn+2

Page 65: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

15.88

1.96

1.58

Min.TER

N/A

2.49

3.32

Min.TER

N/A

N/A

1.88

Min.TER

19.96

5.46

6.55

Min.TER

9.28N/AN/A11.11FVC2002 DB3

1.161.57N/A3.37FVC2002 DB2

1.062.131.014.67FVC2002 DB1

EEREEREEREERDatabases

K-pletsmatch

Genuine test: 138ms

Imposter test: 136ms

Triplet match

Genuine test: 179ms

Imposter test: 7ms

Brute-Force+MCF

Bozorth3

Experimental Results

Page 66: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

Adaptive Matching SystemMinimize Alignment Step

Matching path adapts to number of minutiae features

No

Nr > αNq > α

Triangle feature matching

Template database

Result

Yes

Yes

Yes

No

No

Triangle feature matching

Has a match?

Brute-force matching

Nq < αNr <α

Query fingerprint

Minutiae extraction

Nq Nr

Page 67: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

Triangle Feature Matching

Secondary FeaturesTriplets of MinutiaeS (r0, r1, φ0, φ1, δ0, δ1)

Derived only form minutiae

No ridge countNo minutiae type

Purely localized featureDoes not rely on global landmark

Transformation invariant

No pre-alignment needed

(a) (b)

1ϕ0r

1r

0M

1MCM

Page 68: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

Triangle Feature MatchingSingle Alignment Step

Generate COST matrix of triangles in Q and R O(n2)Find correspondence of triangles and cost using MCF O(n3)Remove globally inconsistent matchesIdentify the max bin in histogram (10o)Average the orientation differences of triangle pairs from the max bin and neighbors (10o, 20o, 350o) = 0.78o

Same could be done for translation

Move back to minutiae from trianglesConvert ALL minutiae to polar coordsusing best matched triangle McShift all orientations by average orientation Compute the COST matrix O(n2)Use MCF to obtain correspondence and cost O(n3)Complexity is O(n3) + O(n2) + O(n3) + O(n2) = O(n3)

False Matched TrianglesGood Matched TrianglesTriangle: central minutiae Mc

OD Distrubtion

0

2

4

6

8

10

12

0 30 60 90 120

150

180

210

240

270

300

330

Degree

OD=0.7865°

Page 69: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

15.88

1.96

1.58

Min.TER

N/A

2.49

3.32

Min.TER

N/A

N/A

1.88

Min.TER

19.96

5.46

6.55

Min.TER

9.28N/AN/A11.11FVC2002 DB3

1.161.57N/A3.37FVC2002 DB2

1.062.131.014.67FVC2002 DB1

EEREEREEREERDatabases

K-pletsmatch

Genuine test: 138ms

Imposter test: 136ms

Triplet match

Genuine test: 179ms

Imposter test: 7ms

Brute-ForceBozorth3

Experimental Results

Page 70: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

Experimental Results (Partial Fingerprints)Adaptive Matching System

0

10

20

30

40

50

60

70

10 20 30 40 50 60 70 80 90

Image size (%)

Err

or ra

te (%

)

TER EER

FVC2002 DB1:

110 fingers – 8 impressions

(2 normal)

1 x 5 % area = 550 samples

FRR rate

7 x 550 samples tested for genuine matches

Use 1,3-8 impressions are ref

Use 2nd impression for query

FAR rate

Use any 2 of 5 samples as query

Use all 1st impressions as ref

Note: values are taken from individualROCs

Page 71: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

Improve Robustness of triangle featuresUse notion of clusters of minutiaeImprove speed and global consistency by indexing reference templates

K-plet Matching with Secondary Features

MOD

Page 72: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

Sensitivity of Triangle Features

(a) (b)

(c) (d)

O

A

BC

OB

A

C

O

A

BC x

O

A

C

(a) (b) (c)

O

A

BC

O

A

BC x

O

A

C

Sensitivity of Triangles to Minutiae Extraction

Associate K triangles

Page 73: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

Verification in clustersAnalysts’ Examination Method

Page 74: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

Additional K Triangle Features

S= (r0, r1, φ0, φ1, δ0, δ1)

r0

r1

φ0

φ1

δ0

δ1

A

BC

D

O

SOABSOCASODASOBCSODBSOCD

features.secondary Max. 2KC

Remove secondary features that are too wide (close to 180°) or too narrow (close to 0°)

K is decided by the size of the fingerprint

If Minutiae > 30Set k as 6

If Minutiae < 20Set k as 10

If Minutiae in [20, 30]Set k as 7

Page 75: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

Triangle IndexingNeighboring minutia labeled with two quadrant tags (bins) clockwise.

Total bins is 32 (4 x 8)

SOAB has index labelQ0Q2, Q1Q2, Q0Q3, and Q1Q3

Each triangle in reference is in 2-4 binsEach query triangle can have 1 bin

92% reduction in matches (580-47)/580 8% of edges in COST matrix input

Avg. Triangles on a fingerprint

580

Avg. Triangles in a bin 47

Min. Triangles in a bin 4

Max. Triangles in a bin 102

A

B

O

Q0Q1

Q2

Q3Q4

Q5

Q6

Q7

Page 76: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

si <•••>

s <•••>s <•••>

s <•••>s <•••>s <•••>s <•••>

Label: Q0Q1

s <•••>s <•••>

Label: Q3Q6

s <•••>s <•••>

s <•••>s <•••>

Label: Q7Q2

s <•••>s <•••>s <•••>

Query fingerprint

Reference fingerprint

Indexing bins

with label Q3Q6

Triangle Matching

Page 77: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

Coupled K-plet AlgorithmFind lists of matched triangles

Construct the cost matrix using the binning scheme (8% of edges)Apply MCF to obtain matched triangles: Lr and Lq.

Identify clusters Form cliques within Lr and Lq

BFS: Start with any pair of corresponding mr and mqRemove mr and mq from Lr and LqBreadth First Extended MatchAdd matched mr and mq into matched queues Qr and Qq

Find the no. of matched minutiae (score) in the BFS sequence

If there are unvisited minutiae in lists LRepeat the BFS procedure and find new score

Return with best sequence in terms of number i ti

Page 78: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

Link Clusters

B

Cb

c

Aa0 1

2

345

67

89

1011

12

02

3

1

4

678

9

1011

12

II RR

Link the seedsTo form Cliques

Link the seedsTo form Cliques

Let k =4

Lq [A,B,C]

Returned by MCF

A(0,1,2,4)

B(12,11,9,10)

C( …)

4(5,6,7,8)

Page 79: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

Coupled Breadth-First Matching

B

Cb

c

Aa0 1

2

345

67

89

1011

12

02

3

1

4

678

9

1011

12

II RR

B

Cb

c

Aa0 1

2

345

67

8 910

11

12

02

3

1

46

78

9 1011

12

II RR

A a A0124 a0124

Page 80: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

Coupled Breadth-First Search Matching (2)

B

Cb

c

Aa0 1

2

345

67

8 910

11

12

02

3

1

46

78

9 1011

12

II RR

B

Cb

c

Aa0 1

2

345

67

8 910

11

12

02

3

1

46

78

9 1011

12

II RR

A,0,1,2,4,B A,0,1,2,4,b A,0,1,2,4,B,7,8,6,12,11,9,10

a,0,1,2,4,b,7,8,6,12,11,9,10

Page 81: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

Extended Matching Result

B

Cb

c

Aa0 1

2

3456

78 9

1011

12

02

3

1

46

78

9 1011

12

II RR

B

Cb

c

Aa0 1

2

3456

78 9

1011

12

02

3

1

46

789

101112

II RR

Without Seed Linking Seed Linking

A,0,1,2,4,B,7,8,6,3,12,11,9,10

14 matched minutiae

A,0,1,2,4,7,8,6,3

9 matched minutiae

Page 82: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

15.88

1.96

1.58

Min.TER

N/A

2.49

3.32

Min.TER

N/A

N/A

1.88

Min.TER

19.96

5.46

6.55

Min.TER

9.28N/AN/A11.11FVC2002 DB3

1.161.57N/A3.37FVC2002 DB2

1.062.131.014.67FVC2002 DB1

EEREEREEREERDatabases

K-pletsmatch

Genuine test: 138ms

Imposter test: 136ms

Triplet match

Genuine test: 179ms

Imposter test: 7ms

Brute-ForceBozorth3

Experimental Results

Page 83: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

Experimental Results

10−4

10−3

10−2

10−1

100

10−0.04

10−0.03

10−0.02

10−0.01

100

False accept rate

Gen

uine

acc

ept r

ate

ROC

Bozorth3ProposedMethod

10−4

10−3

10−2

10−1

100

10−0.04

10−0.03

10−0.02

10−0.01

100

False accept rate

Gen

uine

acc

ept r

ate

ROC

Bozorth3ProposedMethod

FVC2002 DB1 FVC2002 DB2

Page 84: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

5.163.167.934.7211.778.28211.03146.4150

10.176.1611.176.2619.6912.35188.70130.9040

16.209.7917.199.5533.1825.63163.36113.3130

30.9220.9728.0216.1268.4043.90133.3092.4220

Min. TER(%)

EER(%)Min. TER(%)

EER(%)Min.TER(%)

EER(%)

Coupled K-plets MatchTriplet MatchBozroth3Avg. Width(pixels)

Avg. Height(pixels)

Sizes(%)

Partial Fingerprint Experimental Results

Page 85: FINGERPRINT ENHANCEMENT Frequency Domaingovind/CSE666/fall2007/sharat-class.pdf · Outline Fingerprint Image Enhancement Prior Related Work Proposed Algorithm: STFT Analysis Experimental

Conclusion

ContributionsNew Fingerprint Image Enhancement using STFT Analysis.

Simultaneously estimates all intrinsic imagesIncreases recognition rate of existing matchers

New Feature Extraction Algorithm using Chain code ContourObviates need for thinningPerforms favorably with NIST feature extractor

New Graph based matching algorithm Robust to non linear distortionFormal technique for propagating local matchesPerforms better than NIST BOZORTH3 matcher over FVC DB1 database