(2011) Face Image Quality

Biometric Standards, Performance, and Assurance Laboratory |

Purdue University

www.bspalabs.org

www.twitter.com/bspalabs

www.slideshare.net/bspalabs

www.linkedin.com/bspalabs

Comparison of Face Image Quality MetricsSSCI 2011| Paris| April 15th, 2011

http://www.biotown.purdue.edu/index.html

Biometric Standards, Performance & Assurance Laboratory www.bspalabs.org | www.twitter.com/bspalabs | www.slideshare.net/bspalabs |

http://www.linkedin.com/companies/bspa-labs

Agenda

IntroductionMotivation – why are we doing this?Related WorkMethodologyResultsConclusions and Future WorkComments / Questions




Introduction: Standard Face Compliance

Geometric RequirementsISO/IEC 19794-5

Token Face ImageStandardGeometric Characteristics




Introduction

Face Recognition Performance

constrained by Quality

Public Data sets contribution Face Recognition Vendor

Test- FERET database

Pose, Illumination, and Expression (PIE)

Operational Data- regulations FERET Database PIE Database





PIE Database FERET Database

Geometric RequirementsISO/IEC 19794-5

Token Face Image

StandardGeometric Characteristics





StandardGeometric Characteristics

FERET Database IDOC Electronic Database

IDOC Legacy Database




Motivation

Visited INDHS on a project to discuss whether face recognition was ready to use in an operational setting

Indiana Dept. of Corrections (IDOC) asked us to look at their datasets to see if compliant in the FRS

Our IRB would allow performance to be ran on IDOC database but we looked at quality

To analyze IDOC photographs to identify problematic standard metrics

Specify face images because no agreement on a standard face recognition template




Methodology: Legacy Data

Captured from 1970’s - now 9,233 images

Operational Environment exposure to typical environmental conditions

Scanned- Kodak i1220 auto-feed scanner in color at 300 DPI

Printed on archival-quality paper

Holes from rings in binders

Change of color due to age

Cropped for optimal quality




Methodology: Electronic Data

49,694 images- used 48,786 Images couldn’t be found (blank or invalid)

All used with same camera

Operational EnvironmentDigitally collected

JPEG

Collected around 2009/2010




Methodology: FERET Data

4,063 imagesDirect comparison between different algorithms Testing commercially available datasets and

prototype face recognition technologiesPublically availableColored imagesControlled environment




Methodology: Metrics

28 metrics examined Scores between 0 and 10 0-3.9= Poor

4-6.9= Average

7-10= Good




Methodology: Hypothesis

The hypothesis for this experiment was as follows: H0: µiqcL = µiqcE = µiqcF

Ha: µiqcL ≠ µiqcE ≠ µiqcF

iqc is the individual image quality metric

L is the Legacy data set

E is Electronic, and

F is the FERET

P-value set at .05

If < .05 we reject the null hypothesis(statistically significant) and accept the alternative hypothesis




Requirements Specified in ISO/IEC 19794-5




Results: Overall

Aware Quality Metric Legacy Electronic FERET

Overall 7.14 6.24 7.28

Results indicate artifacts other than acquisition affect quality

Legacy better than Electronic

Electronic - majority between 3 and 3.5

Clear difference in distribution




Results: Scene

Quality Metric Legacy Electronic FERETEyes Clear 9.53 9.45 8.77Glare Free 6.58 6.63 6.56Sunglasses 6.81 5.28 6.01Eyes Open 8.38 7.94 7.77Shadows in the Eye Sockets 8.64 8.08 8.13Uniform Lighting 5.39 4.82 4.61Hot Spots 5.36 4.92 6.09Facial Shadows 7.02 7.56 8.12Background Uniformity 3.85 6.86 5.94Background Brightness 4.00 3.38 5.31Background Shadows 6.27 5.10 7.94Frontal Pose 6.90 7.37 8.40

Eyes Clear- Indicates whether or not the subject is wearing glasses

Glare Free- Indicates glare, which generally results from a subject wearing glasses.

Sunglasses- acceptable only for medical reasons.

Eyes Open- ISO standard requires that the iris and pupil of the eye should be clearly seen

Shadows in Eye Sockets- Measures the likelihood that no shadows appear in the eye-sockets.

Uniform Lighting- Indicates whether or not the lighting is equally distributed on the face.

Background Uniformity- Indicates whether the background of a facial image contains a uniform color or a single color pattern

Background Brightness- Measures the average grayscale value of pixels over the background area

Background Shadows- Indicates whether the background of a facial image contains shadows caused by either the face or the imaging devices

Frontal Pose- constrained by less than +/-5 degrees from frontal.




Results: Photographic Requirements

Quality Metric Legacy Electronic FERETCentered 5.89 5.28 6.60

Cropping 9.95 9.95 9.99

Focus 7.63 4.24 5.07

Motion Blur 8.01 7.96 8.41

Exposure 7.21 6.96 7.74

Unnatural Color 6.93 7.38 6.62

Centered- poor measurement of this quality indicates that the token face image will be cropped by the image borders

Cropping- subject's entire head (face) is in the frame

Focus- blurry images, which may be a result of the camera being out of focus

Motion blur- movement from the subject or the camera

Exposure- overexposure and underexposure on the subject's face

Unnatural Color- facial skin area using flesh tone detection




Results: Digital Requirements

Quality Metric Legacy Electronic FERET

Contrast 6.39 6.90 6.48

Scanning Artifacts 6.68 7.12 7.79

Interlaced 9.77 7.28 8.78

Sensor Noise 7.35 6.80 5.64

Contrast- poor contrast value may lack detail from too little or too much contrast in the image

Scanning Artifacts- degradation of performance of recognition algorithm that uses high resolution face images.

Interlaced- possibly extracted from interlaced video frames

Sensor Noise- contains color speckle noise and lacks sufficient color depth

acquired using cell-phone cameras or captured under low illumination conditions




Results: Format Requirements


Compression Artifacts 6.74 6.79 5.08

Compression Artifacts- Stored with visually average quality and with a compression ratio that could degrade the face recognition performance.

Measures the compression ratio and detects JPEG blockiness artifacts of a compressed image.

Medium quality for all three data sets




Results: Algorithmic


Faceness 8.94 9.37 9.43

Texture 7.54 3.47 4.18

These metrics indicate the suitability of a face image with Identix’s face recognition algorithms

Faceness- clear and suitable for face recognition. An obscured face has a low quality score and is therefore likely to degrade the face recognition performance

Texture- effective resolution of the subject’s face for use with high-resolution face recognition algorithms.




Results: Failure to Extract

Database Legacy Electronic FERET

FTX 810 1791 8

Total Images 9232 49692 4063

FTX rate 8.77% 3.60% 0.19%

Overall quality score 7.14 6.24 7.28

Software’s inability to extract features

Sorted all images in Excel and found which had no scores (FTX)

Legacy scored higher overall and in many of the individual characteristics compared to the electronic dataset

Still had higher FTX Rate




Conclusions and Future Work

Room for improvement in image quality of operational data

Comparison of operational to publically available data sets FERET- better overall and better results for majority of metrics

Algorithmic developers adjust to operational dataAnalyze performance of IDOC




Authors and Primary Contact Information

Authors Kevin O’Connor

Undergraduate Researcher at BSPA Lab [email protected]

Stephen Elliott, Ph.D. BSPA Lab Director & Associate Professor [email protected]

Gregory T. Hales Graduate Research at BSPA Labs [email protected]

Jonathan Hight Undergraduate Researcher at BSPA Labs [email protected]

Contact Information

Stephen Elliott, Ph.D.

Associate Professor

Director of BSPA Labs

[email protected]

mailto:[email protected]







Any Questions?

Follow the discussion on the research blog after the conference

www.bspalabs.org/


Technology

(2011) Face Image Quality