4798 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL ...4798 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 22, NO. 12, DECEMBER 2013 Face Recognition Using Ensemble String Matching Weiping

4798 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 22, NO. 12, DECEMBER 2013

Face Recognition Using Ensemble String MatchingWeiping Chen and Yongsheng Gao, Senior Member, IEEE

Abstract— In this paper, we present a syntactic string matchingapproach to solve the frontal face recognition problem. Stringmatching is a powerful partial matching technique, but is notsuitable for frontal face recognition due to its requirementof globally sequential representation and the complex natureof human faces, containing discontinuous and non-sequentialfeatures. Here, we build a compact syntactic Stringface repre-sentation, which is an ensemble of strings. A novel ensemblestring matching approach that can perform non-sequential stringmatching between two Stringfaces is proposed. It is invariantto the sequential order of strings and the direction of eachstring. The embedded partial matching mechanism enables ourmethod to automatically use every piece of non-occluded region,regardless of shape, in the recognition process. The encouragingresults demonstrate the feasibility and effectiveness of usingsyntactic methods for face recognition from a single exemplarimage per person, breaking the barrier that prevents stringmatching techniques from being used for addressing compleximage recognition problems. The proposed method not onlyachieved significantly better performance in recognizing partiallyoccluded faces, but also showed its ability to perform directmatching between sketch faces and photo faces.

Index Terms— Ensemble string matching, face recognition,occlusion, partial matching, stringface, sketch recognition,syntactic.

I. INTRODUCTION

RECOGNIZING human faces with appearance changesfrom one exemplar photo per person is an important

and challenging problem, both in theory and for practicalapplications. A recent question to computer vision researchersis whether we can recognize partially occluded faces inan unconstrained environment when only one example viewper person is available, for example, if only a passport oridentity card photograph [54] is available for each person inthe database. It is clear that the locations, sizes and shapesof occlusions are not predictable in real applications andthus their effect cannot be removed by employing predefinedregions. If, however, we can introduce a mechanism duringthe similarity calculation process to intelligently determinethe occluded areas and exclude them from the similaritymeasurement, it seems possible to solve this problem.

Although there are studies [8], [49]–[52], [56], [57] thathave attempted to solve the problem of recognising partiallyoccluded faces, a number of important theoretical and practical

Manuscript received August 28, 2012; revised January 30, 2013 andMay 3, 2013; accepted July 18, 2013. Date of publication August 15, 2013;date of current version September 27, 2013. This work was supported bythe Australian Research Council under Discovery Grant DP0877929. Theassociate editor coordinating the review of this manuscript and approvingit for publication was Prof. Theo Gevers.

The authors are with Griffith School of Engineering, Griffith Univer-sity, Brisbane, QLD 4111, Australia (e-mail: [email protected];[email protected]).

Color versions of one or more of the figures in this paper are availableonline at http://ieeexplore.ieee.org.

Digital Object Identifier 10.1109/TIP.2013.2277920

issues in this task remain unsolved. For example, how to betterdetermine the locations and shapes of the occluded regionsand how to effectively exclude them from the similaritymeasurement are still unclear.

In this study, we attempt to develop a new strategy forface recognition that (1) can recognize faces with partialocclusions of arbitrary shapes and locations, (2) is suitablefor single model based recognition, (3) works as a generalface recognition method, and (4) matches sketch faces andphoto faces directly without needing to know which image isa sketch.

We propose a novel Stringface representation and matchingconcept for face recognition with one exemplar image perperson. The Stringface is a high-level syntactic representationof a face image, which can be constructed from a singleface image, without any training. In our implementation, thelow-level edge pixels used in early vision are considered as“letters” to be grouped into intermediate-level straight lineswith additional local structural information. These straightlines are considered as “words” in a language. A string isa “sentence”, which is a chain of straight lines, connectedtogether, representing an edge curve on the face. A Stringfaceis a “body text” composed of many strings, which holds asyntactic description about the identity of the face.

A novel Ensemble String Matching scheme (ESM) isdesigned, which can handle the uncertainty problems regardingthe direction of each string and the order of different stringswhen matching two Stringfaces. Different from conventionalstring matching techniques that provide a single matchingscore, the similarity between two Stringfaces is the aggregationof an ensemble of similarities between all matched stringpairs. This is believed to be the first piece of work thatuses a syntactic string representation and matching concept toaddress the frontal face recognition problem. Some of the ideaspresented in this paper were initially reported in [53]. In thisstudy, we report the complete and updated formulation, alongwith an extensive experimental evaluation of our technique.

The rest of the paper is organized as follows: Section 2gives a brief review of the related primitive-based recognitiontechniques and string matching methods. The Stringface rep-resentation and matching concepts are proposed in Section 3,with a detailed description of a novel Ensemble String Match-ing approach. In Section 4, we report the extensive experi-ments that have been conducted to validate the feasibility andeffectiveness of the proposed method. Finally, the paper isconcluded in Section 5.

II. BACKGROUND

In practical applications, faces often have to be recognizedfrom a single exemplar image per person, for example, whenjust a passport or identity card photograph is available as the

1057-7149 © 2013 IEEE

CHEN AND GAO: FACE RECOGNITION USING ENSEMBLE STRING MATCHING 4799

sample of a person. This presents a further challenge to theface recognition problem, due to the additional restrictions onthe training data that can be used and the recognition techniqueitself.

Primitive-based methods have been proven to be effectivewhen dealing with the single sample problem [13]. Accordingto the types of primitives, these approaches can be classifiedinto three groups: low level, intermediate level, and highlevel primitive-based approaches. In low level primitive-basedapproaches, a face is represented by a group of identity relatedpoints whose spatial information (i.e., the locations of theselow level primitives) is used for recognition [32], [55]. Lowlevel primitive-based approaches rely only on the spatial infor-mation of an image, but lack the capacity to represent the localstructure of a face. In order to deal with this problem, prim-itives with additional structural information are introduced.In [24], a face is described using a set of Directional CornerPoints (DCPs) with three attributes, which integrate spatialfeatures and additional structural information about the con-nectivity to their neighbors. The structural attributes of DCPsenhance the discriminative power of the descriptor, allowing itto achieve not only a noticeable accuracy increase over its lowlevel edge map counterpart, but a lower computational cost andmemory requirement, through the use of sparse points. LineEdge Map (LEM) [10] is another intermediate level primitive-based face representation and recognition approach, whoseprimitives are line segments grouped from the pixels of anedge map. The additional local structural attributes of orienta-tion and length provide both better discriminative power, andgreater robustness to noise than low level primitives.

In theory, a high level primitive-based approach that fur-ther considers the interrelationship between intermediate levelprimitives can achieve a higher level of performance byproviding a meaningful complete interpretation of the image.An analogy in understanding a language can be drawn toillustrate the concept. The low level primitives (e.g., pixels)and structural primitives (e.g., straight lines) can be viewed asthe letters and words of a language, while high level primitives(e.g., curves) are regarded as sentences of the language thatare generated from the words according to a grammar. Hence,a high level primitive-based approach is sometimes called asyntactic approach.

String matching is a syntactic and structural method for sim-ilarity measurement between strings or vectors, and has beenwidely used for pattern matching in molecular biology [25],speech recognition [26], and file comparison [27], [28]. Stringscan be classified into two categories: symbolic strings andattributed strings. The goal of string matching algorithms is tofind a sequence of elementary edit operations that transformone string into another at a minimal cost. The elementaryoperations are often deletion, insertion, and substitution ofstring symbols. The most well-known application of symbolicstring matching is the spelling checker algorithms built intoword processing applications such as Microsoft Word. Theuse of symbols was found to be inadequate for achievingcomplex shape recognition [29] because symbols are discretein nature while most problems of shape recognition dealwith attributes that are basically continuous. To overcome

this, attributed string matching [29]–[31] was proposed formeasuring the similarity of shapes by tracing the contour of ashape using a series of primitives, then converting them into astring. One of the prominent advantages of string matchingis its ability to perform partial matching without requiringprior knowledge about the location and size of the missingdata in the string. Furthermore, it is also very robust (ofteninvariant) to spurious outliers and noise. This is particularlyappealing when recognizing complex objects such as humanfaces because in real applications, occlusions can occur atany location, and with arbitrary shapes and sizes. However,with the current state of technology, string matching cannot beused for recognizing natural images containing complex dis-continuous features, as it only works on matching a single 1 Dsequentially ordered pattern (which only can represent a singlecontinuous contour/silhouette of the shape) to another one. Toour knowledge, there is no frontal face recognition using stringmatching techniques. The most related work is [30] for humanface profile recognition. It encodes a continuous face profilesilhouette into a chain of connected profile line segmentsto perform string-to-string matching, which outperformed thepoint-to-point [32] and line-to-line [10] matching methods.However, it can only represent the continuous silhouette ofa profile face, and other important but unconnected distinctivefeatures such as the eyes, eyebrows, mouth, and ears, cannotbe used. Also, it performs a global alignment on two sequencesof lines and fails to work when a face profile has occlusions.

III. ENSEMBLE STRING MATCHING OF STRINGFACES

In this study, we propose a novel high-level Stringfacerepresentation that describes the complex discontinuous facialfeatures in a human face as an ensemble of attributed stringsby integrating the structural connectivity information of a faceimage. A new Ensemble String Matching technique, whichis invariant to the order of strings and is able to performforward-backward string matching, is developed to address thediscontinuity and order uncertainty problems in Stringface.

A. Stringface

The primitive string of a visual object is an abstractand high-level representation. However, conventional stringmatching techniques are limited to continuous (or connected)primitive representation and matching. For example, stringmatching [30] was used to measure the similarity betweentwo continuous silhouettes of profile faces by ignoring otherimportant but unconnected distinctive features, such as eyes,eyebrows, mouth, and ears. A human frontal face containscomplex discontinuous features, and it was often assumed thatstring matching techniques were not suitable for frontal facerecognition.

In this study, we define a syntactic representation of ahuman face, Stringface (SF), which is an ensemble of stringswhere each string describes a facial component. Strings in SFare separated by null primitives φ as

SF = S1φS2φ · · ·φSn−1φSn , (1)


Fig. 1. Visual illustration of an example Stringface using line segments asprimitives.

where n is the number of strings in SF . Each string Sp ,p = 1, . . . , n is a sequence of connected primitives as

Sp = Lq Lq+1 · · · Lq+m p , (2)

where Lq is the qth primitive in SF and (m p + 1) is thenumber of primitives in Sp .

Cognitive psychological studies [33], [34] indicated thathumans recognize line drawings as quickly and almost asaccurately as gray-level images since the line drawings pre-serve most of the important feature information. In this study,line segments, which are obtained by applying polygonalline fitting [35] to the edge map of a face image, areemployed as building units (i.e., “word” primitives) of aStringface. Each primitive, L(l, θ, x, y), is associated withfour attributes l, θ , x , and y, representing the length, orien-tation, and (x,y) midpoint location of the line, respectively.The line orientation θ is defined as the angle with respectto the horizontal. The Stringface represents not only thelocal structural information (by attributes of primitives) butalso the global structure (by strings of primitives) of a face.Fig. 1 gives a visual illustration of an example StringfaceSF = S1φS2φ · · ·φS26φ · · ·φS29φ · · ·φSn , where S29 =L134L135L136L137L138 L139.

B. Ensemble String Matching

Let SF A = S A1 · · · S A

n1= L A

1 · · · L AN1

and SF B =SB

1 · · · SBn2= L B

1 · · · L BN2

be two Stringfaces representing thetest face and the model face, respectively. n1 and n2 arethe numbers of strings in SF A and SFB . N1 and N2 arethe numbers of primitives including null primitives in SF A

and SF B .1) Cost Functions: Attributed string matching (one string

matched against another string) has been used in contour[29] and curve [30] matching. Their cost functions have

proven to be effective for handling segmentation errors and theinconsistency problem in corner detection. In this study, weuse the same cost functions for change and merge operationsas in [30]. Here, we briefly summarise the two cost functionsfor the convenience of reading this paper. Readers can referto [30] for more details.

Let Ak and Bl < j > be two primitives merged fromstring segments A〈i− k+1 : i〉 = L A

i−k+1 L Ai−k+2 · · · L A

i (withk primitives) and B〈 j − l + 1 : j〉 = L B

j−l+1 L Bj−l+2 · · · L B

j(with l primitives) in Stringfaces SF A and SF B respectively.The cost function of a change operation from Ak toBl < j > is defined as

C( Ak → Bl < j >) = |lk − ll | + f (�(θ k, θ l))

+∥∥∥(x, y)k, (x, y)l

∥∥∥ . (3)

It measures the length differences (|lk − ll |), the intersectingangle of orientations (�(θ k, θ l)), and the distance betweenthe midpoints (

∥∥(x, y)k, (x, y)l

∥∥) of Ak and Bl <

j >. When k = 1 and l = 1, no merge is performed and theabove change operation reduces to the conventional one-to-onechange operation A → B < j >.

The cost of merging a sequence of k primitives A〈i−k+1 :i〉 into one merged primitive Ak is defined as

C(A〈i − k + 1 : i〉 → Ak )

= f

⎛

⎝k − 1

lk

i∑

q=i−k+1

�(θ k, θq)× lq

⎞

⎠, (4)

where lk and θ k are the length and the orientation of themerged primitive Ak , lq and θq are the length and theorientation of primitive Lq in A〈i − k+1 : i〉 before merging.Every �(θ k, θq) is weighted by its normalized length lq

/

lk byassuming the contribution of angle difference of a primitiveis proportional to its length. The number of merge operations,k−1, is also factored into the merge cost.

The insert and delete operations used in [29] and [30] arenot needed due to the embedded partial matching mechanismin the proposed ESM (see Section III.B.3).

2) Bounding Merge Operation by Null Primitive: As aStringface is composed of multiple strings delimited by nullprimitives, the merge operation has to be restricted in thesame string, that is, the primitives in string Sp cannot bemerged with primitives in its neighboring strings Sp−1 andSp+1. For a primitive Lq ∈ Sp , the merge operation hasto be stopped by the first φ primitive it meets. Hence themaximum number of primitives that can be merged with Lq isbounded by

m_boundq = q −p−1∑

t=1

|St | − (p − 1), (5)

where |St | is the number of primitives in the t th string.3) S-M Matrixes: The similarity matching between the two

Stringfaces can be characterized by the edit operation costsusing dynamic programming, which generates an S-M matrixpair, i.e., a similarity matrix S and a merge step matrix M.


The S-M matrix pair has a size of (N1 + 1) × (N2 + 1) asfollows.

S =

⎛

⎜⎜⎜⎝

s(0, 0) s(0, 1) . . . s(0, N2)s(1, 0) s(1, 1) . . . s(1, N2)

......

......

s(N1, 0) s(N1, 1) . . . s(N1, N2)

⎞

⎟⎟⎟⎠

, (6)

where each element s(i, j) records a similarity value betweenthe primitives in SF A and SF B along the path determined byits corresponding element m(i, j) in M.

M =

⎛

⎜⎜⎜⎝

m(0, 0) m(0, 1) . . . m(0, N2)m(1, 0) m(1, 1) . . . m(1, N2)

......

......

m(N1, 0) m(N1, 1) . . . m(N1, N2)

⎞

⎟⎟⎟⎠

, (7)

where each element m(i, j) contains two merge step values kand l (see Eq. (10)).

The element s(i, j) stores the maximum accumulated sim-ilarity between two segments (a segment is a portion of astring) of Stringfaces starting from L A

1 and L B1 , and ending at

L Ai and L B

j respectively, as defined in Eq. (8).

s(i, j) ={

maxk, l

(T (k, l)) if L Ai �= φ and L B

j �= φ

0 otherwise,(8)

where T (k, l) is the accumulated similarity resulting from editoperations as defined below.

T (k, l) ={

s(i − k, j − l) + S if S ≥ 00 otherwise,

(9)

The merge steps that produce the maximum T (k, l) arerecorded in m(i, j) as

m(i, j) = arg maxk,l

(T (k, l)). (10)

If either L Ai or L B

j is a null primitive, s(i, j) is reset to zero(see Eq. (8)). This is to ensure that T (k, l) will not increaseits value by edit operations on primitives beyond the currentstring. To seamlessly include the partial matching mechanismin the design of ESM, the similarity parameter S in Eq. (9) isdefined as

S = λ− C(A〈i − k + 1 : i〉 → Ak )

+ C(B〈 j − l + 1 : j〉 → Bl < j >)

+ C( Ak → Bl < j >),

(11)

where λ is a threshold to decide the strength of partialmatching. If the combined cost of merging and changingprimitives is greater than λ (which gives S < 0), theseprimitives are considered as outliers and discarded from ESM.This is achieved by setting T (k, l) = 0 when S < 0 (see Eq.(9)). The detailed algorithm of creating the S-M matrix pairis given in Algorithm 1.

4) Similarity Measurement: The similarity of associatinga group of segments from Stringface SF A with a group ofsegments from Stringface SF B is computed as:

S(SF A, SF B) = ξ × (1

f∑

i=1τiλ

×f

∑

i=1

Scorei ) (12)

Algorithm 1. The proposed S-M matrix pair generation in ESM.

where Scorei is the similarity score of the i th matched stringsegments between SF A and SF B , and f is the numberof matched string segments (There may be more than onematched segment, or none, in a single string). λ is thethresholding value to decide the strength of a partial matching(see Eq. (11)). τi is the number of change operations for cor-responding the i th matched segments in the two Stringfaces.ξ is a weight term taking the importance of matching a largerpercentage of both Stringfaces in the similarity measurementdesign, according to the observation that humans considerthe percentage of similar content between two objects whenjudging the quality of matching.

ξ= 1

2

(length o f matched SF A

length of SF A+ length of matched SF B

length o f SF B

)

(13)The pairs of matched segments are found by first locating

the maximal element in S. Other matrix elements leading tothis maximal value are then sequentially determined by M witha traceback procedure ending with an element of S equallingzero. The maximum value is recorded as the similarity scoreof this matched segment pair. Before searching for the nextmatched segment pair, all the elements in S related to thepervious matched primitives are set to zero to avoid matchinga primitive in one Stringface to multiple primitives in the otherStringface. The above traceback procedure is repeated until allthe elements in S are 0 to find all Scorei for Eq. (12). (seeAlgorithm 2).

5) Analysis and Example: Different from a conventionalstring that guarantees the order of primitives is maintained ineach instance, Stringface is actually an ensemble of stringsin which both the order of these strings, and the startand end of each string are interchangeable. For example,


Fig. 2. An example of the ensemble string matching between two Stringfaces. The squares and the circles represent the starts and the ends of strings, whichdecide the sequence directions of the strings.

Algorithm 2. The proposed S-M matrix pair generation in ESM.

an object X in three different pictures is encoded into SFsSF A = S A

1 φS A2 = L1 L2 L3 L4 L5 L6L7φL9 L10L11(L8 = φ),

SF B = SB2 φSB

1 = L9 L10 L11φL1 L2 L3 L4 L5 L6 L7, SFC =←−SC

1 φSC2 = L7 L6 L5 L4 L3 L2 L1φL9 L10 L11. Note that there is

an order change of strings between SF A and SF B , and a direc-tion change between the 1st string of SF A and the 1st string ofSFC . The proposed ensemble string matching gives similarityvalues of S(SF B, SF A) = 1 (Score1 = 7λ, Score2 = 3λ,

Fig. 3. Matrix S and traceback result.

τ1 = 7, τ2 = 3, f = 2, ξ = 1) and S(SFC , SF A) = 1(Score1 = 3λ, Scorei = λ(i = 2, ..., 8), τ1 = 3, τi =1 (i = 2, ..., 8), f = 8, ξ = 1), equalling the 100% self-match of S(SF A, SF A) = 1 (Score1 = 7λ, Score2 = 3λ,τ1 = 7, τ2 = 3, f = 2, ξ = 1). This provides the requiredstring location and direction mutation invariances.

We now show a real example of the proposed Stringfacematching approach. Fig. 2(a) and (b) visualize StringfacesSF A and SF B of two different face images from the sameperson in the AR face database. Fig. 2(c) displays the 27th string of SF A with 5 primitives (L A

111L A112 L A

113L A114L A

115).And Fig. 2(d) and (e) are the 29th and the 26th strings ofSF B with 6 and 2 primitives, respectively. The S-M matrixpair calculated by Algorithm 1 is shown in Fig. 3 and Fig. 4.First, we locate the maximal element of matrix S which is


Fig. 4. Matrix M and traceback result.

s(L A115, L B

138) =31.01 in Fig. 3. The merge step of this entryis m(L A

115, L B138) = (1, 2) (see Fig. 4), which means that L A

115is not merged with any other primitives and primitives L B

138and L B

137 are merged into a new primitive in the matchingprocess. Then we traceback by one primitive in SF A and twoprimitives in SF B (see Algorithm 2) to entry (L A

114, L B136),

with similarity value s(L A114, L B

136) = 25.97 and merge stepm(L A

114, L B136) = (2, 1). A whole path (marked in red in Fig. 3

and Fig. 4) can be traced by repeating the above process. Themerged primitives in strings S A

27 and SB29 during the matching

process are shown in Fig. 2(f) and (g), respectively. Thesimilarity score between S A

27 and SB29 is 31.01. Note that the

similarity score between S A27 and SB

26 is 0.

IV. EXPERIMENTAL RESULTS

Three publicly available databases (the AR face database[37], the FRGC ver2.0 face database [38], and the CUHK facesketch database [2]) were used to evaluate the effectivenessof the proposed approach. The AR database consists of over4,000 frontal view images of 126 persons (70 male and 56female). Each person is represented by 26 images capturedin two different sessions with a two-week time interval.Each session contains 13 face images under different lightingconditions (right light on, left light on and both lights on),different facial expressions (smiling, angry, and screaming),and partial occlusions (sunglasses and scarf). Some imageswere found missing or corrupted for a few persons. The photosof 117 persons having complete sets of images (64 male and 53female) covering all conditions were used in our experiments.Fig. 2 shows an example set of images from one person in theAR database used for our experiments.

The FRGC ver2.0 database contains 50,000 recordingsdivided into training and validation sets. The training setconsists of 12,776 images from 222 persons, which were takenin 9–16 sessions. The validation set contains images from 466

Fig. 5. A sample set of AR face images from one person. (a) Neutral faceunder controlled/ideal conditions taken in the first session; (b-d) faces withdifferent facial expressions taken in the first session; (e-g) faces under differentlighting conditions taken in the first session; (h-i) faces with different partialocclusions taken in the first session; (j) neutral face under controlled/idealconditions taken in the second session; (k-l) faces with different partialocclusions taken in the second session.

Fig. 6. Example sketches for faces from (a) the AR database and (b) theCUHK student database.

Fig. 7. Samples of the cropped faces used in our experiments.

persons collected in 4,007 sessions. The number of sessionsused for taking photos of each person is different, ranging from1 to 22 sessions (410 persons were photographed in 2 or moresessions). In each session were recorded four controlled stillimages, two uncontrolled still images, and one 3 D scan. In ourexperiments, all the 466 persons in the validation set were usedfor testing. Different from the AR database, there are multipleneutral expression images for each person in the first sessionof FRGC. The first image in the first session was used asthe single model of the person. One test image per personwas randomly selected from the remaining neutral expressionimages (after having taken out the model image) of the personin all sessions.

The CUHK face sketch database (CUFS) contains sketchesdrawn by an artist based on 123 faces from the AR database[37], 188 faces from the Chinese University of Hong Kong(CUHK) student database, and 295 faces from the XM2VTSdatabase [39]. All the 311 sketches for the faces in thepublicly available AR and CUHK databases were used in ourexperiments (see examples in Fig. 6).

In all the experiments, a preprocess was applied to normal-ize (in scale and orientation) the original face images suchthat the two eyes were manually aligned roughly at the sameposition with a distance of 80 pixels. Then the facial regionswere cropped to a size of 160 × 160 for recognition. Someexample cropped faces are displayed in Fig. 7. Note that allthe experiments employ a single model per person matchingscheme.


Fig. 8. The effect of λ on the recognition rate under ideal conditions.

The performances of the proposed method are com-pared with the major benchmark approaches that can handleappearance changes including partial occlusions and/or canperform face sketch recognition. They are: (1) partitionedSparse Representation-based Classification (p-SRC) [40],Independent Component Analysis Architecture I (ICA-I) [41],Adaptively Weighted Patch Pseudo Zernike Moment Array(AWPPZMA) [42], Local Probabilistic approach (LocPb) [8],which achieved state-of-the-art performances in handling par-tial occlusions; (2) Eigenface [17], which is the most popularbenchmark in performance comparison of face recognitionapproaches; (3) Line Edge Map (LEM) [10] and DirectionalCorner Point (DCP) [24] methods, which use descriptorsbased on the same source of edge information (but in anintermediate-level primitive manner) to benchmark the dis-criminative power of the proposed high-level representationand matching. Experimental results in all the tables are pre-sented in groups according to the above three categories foreasy comparison.

A. The Effect of λ

The effect of λ in Eq. 11 on the recognition accuracywas investigated on the AR face database, in which theneutral faces under controlled/ideal conditions taken in the firstsession (e.g., Fig. 5(a)) were selected as the model set, andthe neutral faces under controlled/ideal conditions taken in thesecond session (e.g., Fig. 5(j)) were used as the test set. Therecognition rate is plotted against the values of λ in Fig. 8.It is found that Stringface with a low thresholding value (i.e.λ = 2) performed badly. It improved quickly and reached theoptimal value when λ ranged from 8 to 14. For all the otherexperiments in this paper, λ was set as 10.

B. Face Recognition Under Controlled/Ideal Conditions

To have a complete performance evaluation across all recog-nition conditions, the system performance under controlledconditions was first evaluated on both AR and FRGC ver2.0databases. The recognition rate of the proposed approachtogether with those of p-SRC, AWPPZMA, ICA-I, Eigenface,LEM, and DCP approaches are summarized in Table I.

From Table I, it can be seen that the proposed approachperformed similar to p-SRC, AWPPZMA, LEM, and DCPmethods, and significantly outperformed ICA-I and Eigenfacemethods on the AR face database. On the large FRGC ver2.0

TABLE I

PERFORMANCE COMPARISON UNDER CONTROLLED/IDEAL CONDITIONS

database, the Stringface and p-SRC methods achieved muchbetter accuracies than the rest of the methods. This may bebecause the images of the same person under ideal conditionsin the FRGC ver2.0 database have larger variations than thosein the AR database, and AWPPZMA, LEM, and DCP methodsare not as robust as Stringface and p-SRC to these variations.The Eigenface and ICA-I methods work well as long as thetest image is “similar” to the ensemble of training samplesand the training set should include multiple images of eachperson with some variations to obtain better performance.Here, all experiments employed a single model per personmatching scheme. Hence, the single model based methods,i.e. Stringface, LEM, DCP, and AWPPZMA, achieved higherrecognition rates than Eigenface and ICA-I. Partitioned SRC[40] is the state-of-the-art method that can recognize faces withvarying expressions, varying illuminations, and occlusions.Although there is only one sample image per person, itstill performed very well (97.43% and 81.76%), as did theproposed approach (96.58% and 82.62%) on both databases.

C. Face Recognition Under Partial Occlusions

The proposed Stringface approach is, in theory, able toeffectively find the most discriminative local parts (strings)for recognition without requiring any prior knowledge orassumption of the occlusions. The capacity of the proposedapproach to recognize occluded faces was studied using twopublicly available databases. The AR Face database was usedto evaluate the system performance on real occluded faces,while the FRGC ver2.0 database was employed to investigatethe sensitivity to the size of occlusion using the same strategyas in [43], [45], [46].

1) Recognition With Real Data: In this experiment, theneutral face images in the first session (i.e., Fig. 5(a)) wereused as the models, while the occluded faces with sunglassesand scarfs in both the first and second sessions (i.e., Fig. 5(h), (k), and Fig. 5 (i), (l)) were used as the tests respectively.It is observed from Table II that occlusions affected theperformances of all methods with an accuracy drop ranging9.40%–66.67% and 1.71%–88.03% for faces (Session 1) withsunglasses and scarfs, respectively. As expected, the proposedmethod and the baseline methods that can handle occlusionsperformed better than other benchmark methods. However,the Stringface (86.54% on average) and p-SRC (83.55% onaverage) performed significantly better than other methodsincluding those robust to partial occlusions, and the proposed


TABLE II

PERFORMANCE COMPARISON FOR SUNGLASSES AND SCARF OCCLUDED

FACES. AD: ACCURACY DECREASE DUE TO OCCLUSION COMPARED TO

THAT UNDER IDEAL CONDITIONS IN TABLE I

Fig. 9. Examples of model and occluded test faces. (a) Model image; (b-k)The corresponding test images with occluding blocks of sizes 10 × 10, 20 ×20,…, 100 × 100 respectively at a random location.

Stringface consistently obtained the highest accuracies underall four conditions (2 sessions and 2 occlusions). The sun-glasses and scarfs (Session 1) only resulted in a small drop ofaccuracies (9.40% and 1.71% for Stringface, and 11.96% and6.83% for p-SRC, respectively). It is interesting to note that theproposed method only has a 1.71% accuracy decrease whenthe faces are occluded by scarfs, showing its superior capacityin handling partial occlusions. The performance drops, due tothe fact that occlusions in the session 2 images were biggerthan those in the session 1 images, are understandable asthe intra-class variations increase when taking the session 2pictures after a two week time interval. As we expected, theresults of all algorithms that can handle occlusions (Stringface,p-SRC, AWPPZMA, ICA-I, and LocPb) show that recognizingoccluded faces with scarfs is easier than with sunglasses,because the eye areas carry more identity related informationthan the mouth and nose areas of a person’s face, despitethe fact that the scarfs covered a larger facial area than thesunglasses.

2) Sensitivity to Occlusion Size: Sensitivity to occlusionsize is often used to quantitatively evaluate the robustness of analgorithm to occlusions. In this evaluation test, we employedthe same experimental strategy as in [43]–[47] on the largeFRGC ver2.0 database. The test images used in Section IV.Bwere occluded by a square of s×s at a random location, whilethe model images remained unchanged without occlusion (seeexamples in Fig. 9). The size of the occlusion ranged froms = 10 (0.39% of the image) up to s = 100 (39.06% of theimage) with a step size of 10.

The accuracies of the proposed method and the benchmarkapproaches are plotted against the occlusion size in Fig. 10.The proposed Stringface method demonstrated an ability supe-rior to all other methods when recognizing partially occludedfaces. For block sizes from 10 × 10 up to 50 × 50, theperformances of Stringface and p-SRC were nearly invariant to

Fig. 10. Recognition under varying sizes of random occlusion (10 × 10, 20 ×20, …, 100 × 100 of occluding blocks).

occlusions, while all other benchmark algorithms experiencedbig drops in accuracy. For block sizes from 70 × 70 upwards,the performance gap between Stringface (73.39%) and p-SRC(68.45%) started to increase quickly, and recognition ratesof the other methods dropped significantly, to below 35%.With a block size of 100 × 100, the Stringface approachcan still achieve an impressive recognition rate of over 60%,compared to p-SRC’s 38.41%. These results demonstrate thesignificant superiority of the proposed Stringface method inhandling partial occlusions. The superior performance of theStringface method is believed to be due to its content-basedpartial matching scheme (which can take every piece ofnon-occluded content, within regions of any shape, into therecognition process automatically), instead of the subregion-based matching scheme used by other methods (which can-not differentiate occluded and non-occluded areas within asubregion).

D. Face Sketch Recognition

Computerized sketch-photo recognition has attracted muchattention recently [2]–[7], [58]. This section examines how theproposed Stringface method performs for directly recognizinga face sketch with only a single sample photo per person.We randomly selected 100 persons from the AR database,and 150 from the CUHK student database. The neutral facephotos from the first session (e.g., Fig. 5(a)) and the secondsession (e.g., Fig. 5(j)) of the AR database, and from theCUHK student database were used as the single samples ofthe persons, respectively. Their sketches were used as tests.Fig. 11 and Fig. 12 show the cumulative match score (CMS)of the proposed approach and the benchmark algorithms onfaces from the AR database and the CUHK student database,respectively. Note that the recognition rates of p-SRC inFig. 11 and Fig. 12 are only for rank = 1 as it can onlyperform rank 1 matching (In rank N matching, a correctmatch is counted only when the model face from the sameperson as the test face is among the best N matched modelfaces).

Due to differences between sketches and photos and theunknown psychological mechanism of sketch generation [2],face sketch recognition is much harder than normal face recog-nition. It is interesting to observe from Fig. 11 and Fig. 12 that


Fig. 11. Comparative performance of the proposed method and the bench-mark algorithms for directly recognizing sketches of faces from the AR facedatabase. (a) Matching results between sketches and photos from the samesession. (b) Matching results between sketches and photos from differentsessions.

the proposed Stringface approach obtained very encouragingaccuracies when directly applying it to sketch recognition,considering it as a general face recognition technique with-out using any sketch-photo synthesis techniques as used in[2]–[6]. Its better performance against benchmark methodsmay be attributed to the structural representation in Stringfacesand the flexible partial matching mechanism. A sketch drawnby an artist (1) depicts the structural characteristics of the per-son’s face, and (2) contains some shadow textures to abstractlyreflect its 3 D shape information. A pencil can replicate thestructural features of a face photo quite well. Stringfacesgenerated from facial structures in nature selectively take thereliable information of a sketch for recognition. This result,obtained from a computer vision system, seems in accordancewith the findings of cognitive psychological studies [33], [34]that show that humans recognize line drawings as quickly andalmost as accurately as gray level pictures. Some even conjec-ture that caricatures may be better representations than naturalimages [2] because caricatures contain some kind of “super-fidelity” due to its accentuated structure in addition to theessential minimum of identity information [48]. However, theexperimental results in this section cannot be over-interpretedas supporting the above hypothesis because the benchmarkapproaches are those best at handling partial occlusions (plusthe most popular baseline Eigenface and approaches based

Fig. 12. Comparative performance of the proposed method and the bench-mark algorithms for directly recognizing sketches of faces from the CUHKstudent database.

on similar source of features), instead of all face recognitionalgorithms, upon which a meaningful argument could bemade.

Although sketches reproduce the shape and structural fea-tures better than textures, they inevitably exaggerate somedistinctive facial features like caricatures, which involve shapedeformation [2]. For example, it is observed that most ofthe noses drawn in the sketches of CUHK faces are biggerthan those in photos. The flexible partial matching mecha-nism in ESM can automatically exclude the largely distortedareas from the calculation of matching score, thus eliminatingthe negative effect of large distortions in a sketch. This isevidenced by the much higher performance of Stringfacecompared to non-partial matching LEM and DCP approaches,which use similar types of structural information at theintermediate level. The differences in accuracies in Fig. 12are larger than those in Fig. 11 because the sketches ofCUHK student faces have larger distortions from their photosthan the sketches of AR faces (This also resulted in loweraccuracies on the CUHK sketches compared to those on theAR sketches). As the sketches of AR faces were drawn basedon the photos taken in Session-1, the recognition rates formatching the sketches and photos from the same session(Fig. 11(a)) are consistently higher than those from differentsessions (Fig. 11(b)).

E. Face Recognition Under Varying Lighting Conditions

The sensitivity to illumination changes is one of the impor-tant issues for the evaluation of face recognition systems. Theproposed Stringface employs a high-level syntactic primitiverepresentation derived from intermediate-level representationof lines grouped from low-level edge pixels. We performed anexperimental study to evaluate how well Stringface performsfor recognizing faces under varying lighting conditions. Theneutral face images in the AR database taken in the firstsession (e.g., Fig. 5(a)) were used as the single models ofthe persons. The face images under three different lightingconditions (e.g., Fig. 5 (e), (f), and (g)) taken in the samesession were used as test images. The results are summarizedin Table III.

It can be seen from Table III that the proposed approachachieved an average accuracy improvement of 1.43% over


TABLE III

EXPERIMENTAL RESULTS UNDER VARYING LIGHTING CONDITIONS

TABLE IV

EXPERIMENTAL RESULTS UNDER VARYING FACIAL EXPRESSION

CHANGES

the LEM method, which is one of the best illuminationinsensitive methods based on facial edges for single modelimage recognition. This is believed to be due to the factthat the LEM method employs an intermediate-level imagerepresentation derived from low-level edge map representa-tion, while the proposed Stringface approach represents notonly the local structural information but also their inter-relationships within a face. Table III also shows that theproposed approach significantly outperformed DCP, AWP-PZMA, ICA-I, and Eigenface methods under different lightingconditions.

F. Face Recognition Under Varying Facial Expressions

Similar experiments were conducted to evaluate the effectsof different facial expressions (smiling, angry, and screaming)on the system performance. The faces with neutral expressionin the first session (e.g., Fig. 5 (a)) were used as the modelimages; the face images with three different expressions (e.g.,Fig. 5 (b), (c), and (d)) taken in the same session were usedas the test images.

The experimental results are summarized in Table IV. Thesmiling and the angry expressions caused the recognition rateof Stringface to drop by 11.11% and 9.40% compared to theneutral expression in Table I. The large expression changes ofscreaming significantly affected the performance of Stringface.The Eigenface and the AWPPZMA methods are found the leastsensitive to the facial expression changes.

V. CONCLUSION

In this paper, we presented a new method for humanfrontal face recognition using a high-level string representa-tion and matching strategy. A new Stringface representation

is proposed, which groups low-level “letter” primitives intointermediate-level “word” primitives and then further inte-grates their relational organization into high-level “sentence”strings. The similarity measurement of two faces is performedby matching two Stringfaces through a new Ensemble StringMatching scheme, which is able to effectively find the mostsimilar segments between two Stringfaces, regardless of thesequential order of strings and the direction of each stringin a Stringface. Its embedded partial matching mechanismcan recognize faces with occlusions of arbitrary shapes andlocations without requiring any prior knowledge or assumptionof the missing regions. The proposed approach is comparedagainst the benchmark algorithms using three databases (AR,FRGC ver2.0, and CUFS). The promising experimental resultsdemonstrate the superiority of the proposed method in recog-nising partially occluded faces, and the feasibility of using ahigh-level syntactic method for face recognition, indicating anew strategy for face coding and matching.

REFERENCES

[1] G. U. J. Robert and N. da Vitoria Lobo, “Recognizing a facial imagefrom a police sketch,” in Proc. 2nd IEEE Workshop Appl. Comput. Vis.,Dec. 1994, pp. 129–137.

[2] X. Wang and X. Tang, “Face photo-sketch synthesis and recognition,”IEEE Trans. Pattern Anal. Mach. Intell., vol. 31, no. 11, pp. 1955–1967,Nov. 2009.

[3] P. C. Yuen and C. H. Man, “Human face image searching system usingsketches,” IEEE Trans. Syst., Man Cybern. A, Syst. Humans, vol. 37,no. 7, pp. 493–504, Jul. 2007.

[4] B. Xiao, X. Gao, D. Tao, and X. Li, “A new approach for facerecognition by sketches in photos,” Signal Process., vol. 89, no. 8,pp. 1576–1588, 2009.

[5] X. Gao, J. Zhong, D. Tao, and X. Li, “Local face sketch synthesislearning,” Neurocomputing, vol. 71, nos. 10–12, pp. 1921–1930, 2008.

[6] B. Xiao, X. Gao, D. Tao, Y. Yuan, and J. Li, “Photo-sketch synthesisand recognition based on subspace learning,” Neurocomputing, vol. 73,nos. 4–6, pp. 840–852, 2010.

[7] B. Klare and A. K. Jain, “Sketch to photo matching: A feature-basedapproach,” Proc. SPIE, vol. 7667, pp. 766702-1–766702-10, Apr. 2010.

[8] A. M. Martinez, “Recognizing imprecisely localized, partially occluded,and expression variant faces from a single sample per class,” IEEE Trans.Pattern Anal. Mach. Intell., vol. 24, no. 6, pp. 748–763, Jun. 2002.

[9] X. Tang and X. Wang, “Face sketch recognition,” IEEE Trans. CircuitsSyst. Video Technol., vol. 14, no. 1, pp. 50–57, Jan. 2004.

[10] Y. Gao and M. K. H. Leung, “Face recognition using line edge map,”IEEE Trans. Pattern Anal. Mach. Intell., vol. 24, no. 6, pp. 764–779,Jun. 2002.

[11] P. N. Belhumeur, J. P. Hespanha, and D. J. Kriegman, “Eigenfaces vs.Fisherfaces: Recognition using class specific linear projection,” IEEETrans. Pattern Anal. Mach. Intell., vol. 19, no. 7, pp. 711–720, Jul. 1997.

[12] I. Naseem, R. Togneri, and M. Bennamoun, “Linear regression for facerecognition,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 32, no. 11,pp. 2106–2112, Nov. 2010.

[13] X. Tan, S. Chen, Z. Zhou, and F. Zhang, “Face recognition froma single image per person: A survey,” Pattern Recognit., vol. 39, no. 9,pp. 1725–1745, 2006.

[14] T. Kanade, “Picture processing system by computer complex and recog-nition of human faces,” Ph.D. dissertation, Dept. Inf. Sci., Kyoto Univ.,Kyoto, Japan, Nov. 1973.

[15] R. Brunelli and T. Poggio, “Face recognition: Features versus templates,”IEEE Trans. Pattern Anal. Mach. Intell., vol. 15, no. 10, pp. 1042–1052,Oct. 1993.

[16] M. Kirby and L. Sirovich, “Application of the Karhunen-Loeve proce-dure for the characterization of human faces,” IEEE Trans. Pattern Anal.Mach. Intell., vol. 12, no. 1, pp. 103–108, Jan. 1990.

[17] M. A. Turk and A. P. Pentland, “Eigenfaces for recognition,” J. Cognit.Neurosci., vol. 3, no. 1, pp. 71–86, 1991.

[18] J. Wu and Z. H. Zhou, “Face recognition with one training image perperson,” Pattern Recognit. Lett., vol. 23, no. 14, pp. 1711–1719, 2002.


[19] J. P. Phillips, H. Moon, S. A. Rizvi, and P. J. Rauss, “The FERETevaluation methodology for face-recognition algorithms,” IEEE Trans.Pattern Anal. Mach. Intell., vol. 22, no. 10, pp. 1090–1104, Oct. 2000.

[20] S. C. Chen, D. Q. Zhang, and Z.-H. Zhou, “Enhanced (PC)2A for facerecogniton with one training image per person,” Pattern Recognit. Lett.,vol. 25, no. 10, pp. 1173–1181, 2004.

[21] T. Ahonen, A. Hadid, and M. Pietikainen, “Face description with localbinary patterns: Application to face recognition,” IEEE Trans. PatternAnal. Mach. Intell., vol. 28, no. 12, pp. 2037–2041, Dec. 2006.

[22] B. Zhang, Y. Gao, S. Zhao, and J. Liu, “Local derivative pattern versuslocal binary pattern: Face recognition with high-order local patterndescriptor,” IEEE Trans. Image Process., vol. 19, no. 2, pp. 533–544,Feb. 2010.

[23] A. Pentland, B. Moghaddam, and T. Starner, “View-based and modulareigenspaces for face recognition,” in Proc. IEEE Comput. Soc. Conf.Comput. Vis. Pattern Recognit., Jun. 1994, pp. 84–91.

[24] Y. Gao and Y. Qi, “Robust visual similarity retrieval in single modelface databases,” Pattern Recognit., vol. 38, no. 7, pp. 1009–1020, 2005.

[25] S. C. Chan and A. K. C. Wong, “Synthesis and recognition ofsequences,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 13, no. 12,pp. 1245–1255, Dec. 1991.

[26] L. Rabiner and B.-H. Juang, Fundamentals of Speech Recognition.Englewood Cliffs, NJ, USA: Prentice-Hall, 1993.

[27] P. Heckel, “A technique for isolating differences between files,” Com-mun. ACM, vol. 21, no. 4, pp. 264–268, 1978.

[28] L. Bahl and F. Jelinek, “Decoding for channels with insertions, deletions,and substitutions with applications to speech recognition,” IEEE Trans.Inf. Theory, vol. 21, no. 4, pp. 404–411, Jul. 1975.

[29] W. H. Tsai and S. S. Yu, “Attributed string matching with merg-ing for shape recognition,” IEEE Trans. Pattern Anal. Mach. Intell.,vol. PAMI-7, no. 4, pp. 453–462, Jul. 1985.

[30] Y. Gao and M. K. H. Leung, “Human face profile recognition usingattributed string,” Pattern Recognit., vol. 35, no. 2, pp. 353–360, 2002.

[31] S. W. Chen, S. T. Tung, C. Y. Fang, S. Cheng, and A. K. Jain, “Extendedattributed string matching for shape recognition,” Comput. Vis. ImageUnderstand., vol. 70, no. 1, pp. 36–50, 1998.

[32] B. Takács, “Comparing face images using the modified hausdorffdistance,” Pattern Recognit., vol. 31, no. 12, pp. 1873–1881, 1998.

[33] I. Biederman and G. Ju, “Surface versus edge-based determinants ofvisual recognition,” Cognit. Psychol., vol. 20, no. 1, pp. 38–64, 1988.

[34] V. Bruce, E. Hanna, N. Dench, P. Healey, and M. Burton, “Theimportance of ‘mass’ in line drawings of faces,” Appl. Congnit. Psychol.,vol. 6, no. 7, pp. 619–628, 1992.

[35] M. K. H. Leung and Y. H. Yang, “Dynamic two-strip algorithm in curvefitting,” Pattern Recognit., vol. 23, nos. 1–2, pp. 69–79, 1990.

[36] A. Leonardis and H. Bischof, “Robust recognition using eigenimages,”Comput. Vis. Image Understand., vol. 78, no. 1, pp. 99–118, 2000.

[37] A. M. Martinez and R. Benavente, “The AR face database,” Tech.Rep. 24, 1998.

[38] P. J. Phillips, P. J. Flynn, T. Scruggs, K. W. Bowyer, J. Chang,K. Hoffman, J. Marques, J. Min, and W. Worek, “Overview of theface recognition grand challenge,” in Proc. IEEE Comput. Soc. Conf.Comput. Vis. Pattern Recognit., vol. 1. Jun. 2005, pp. 947–954.

[39] K. Messer, J. Matas, J. Kittler, J. Luettin, and G. Maitre, “XM2VTSDB:The extended M2VTS database,” in Proc. Int. Conf. Audio- Video-BasedPerson Authenticat., 1999, pp. 72–77.

[40] J. Wright, A. Y. Yang, A. Ganesh, S. S. Sastry, and Y. Ma, “Robust facerecognition via sparse representation,” IEEE Trans. Pattern Anal. Mach.Intell., vol. 31, no. 2, pp. 210–227, Feb. 2009.

[41] M. S. Bartlett, J. R. Movellan, and T. J. Sejnowski, “Face recognition byindependent component analysis,” IEEE Trans. Neural Netw., vol. 13,no. 6, pp. 1450–1464, Nov. 2002.

[42] H. R. Kanan, K. Faez, and Y. Gao, “Face recognition using adaptivelyweighted patch pzm array from a single exemplar image per person,”Pattern Recognit., vol. 41, no. 12, pp. 3799–3812, 2008.

[43] J. Kim, J. Choi, J. Yi, and M. Turk, “Effective representation using ICAfor face recognition robust to local distortion and partial occlusion,”IEEE Trans. Pattern Anal. Mach. Intell., vol. 27, no. 12, pp. 1977–1981,Dec. 2005.

[44] X. Tan, S. Chen, Z.-H. Zhou, and J. Liu, “Face recognition underocclusions and variant expressions with partial similarity,” IEEE Trans.Inf. Forensics Security, vol. 4, no. 2, pp. 217–230, Jun. 2009.

[45] H. J. Oh, K. M. Lee, and S. U. Lee, “Occlusion invariant face recognitionusing selective local non-negative matrix factorization basis images,”Image Vis. Comput., vol. 26, no. 11, pp. 1515–1523, 2008.

[46] H. Jia and A. M. Martinez, “Support vector machines in face recognitionwith occlusions,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit.,Jun. 2009, pp. 136–141.

[47] S. Z. Li, X. W. Hou, H. J. Zhang, and Q. S. Cheng, “Learning spatiallylocalized, parts-based representation,” in Proc. IEEE Comput. Soc. Conf.Comput. Vis. Pattern Recognit., vol. 1. Dec. 2001, pp. I-207–I-212.

[48] J. Benson and D. I. Perrett, “Perception and recognition of photographicquality facial caricatures: Implications for the recognition of naturalimages,” Eur. J. Cognit. Psychol., vol. 3, no. 1, pp. 105–135, 1991.

[49] X. Tan, S. Chen, Z. Zhou, and F. Zhang, “Recognizing partiallyoccluded, expression variant faces from single training image per personwith SOM and soft k-NN ensemble,” IEEE Trans. Neural Netw., vol. 16,no. 4, pp. 875–886, Jul. 2005.

[50] K. Ohba and K. Ikeuchi, “Detectability, uniqueness, and reliability ofeigen windows for stable verification of partially occluded objects,”IEEE Trans. Pattern Anal. Mach. Intell., vol. 19, no. 9, pp. 1043–1047,Sep. 1997.

[51] Y. Saito, Y. Kenmochi, and K. Kotani, “Estimation of eyeglassless facialimages using principal component analysis,” in Proc. Int. Conf. ImageProcess., vol. 4. 1999, pp. 197–201.

[52] B. W. Hwang and S. W. Lee, “Reconstruction of partially damaged faceimages based on a morphable face model,” IEEE Trans. Pattern Anal.Mach. Intell., vol. 25, no. 3, pp. 365–372, Mar. 2003.

[53] W. Chen and Y. Gao, “Recognizing partially occluded faces from a singlesample per class using string-based matching,” in Proc. 11th Eur. Conf.Comput. Vis., vol. 3. 2010, pp. 496–509.

[54] T. Bourlai, A. Ross, and A. Jain, “On matching digital face imagesagainst scanned passport photos,” in Proc. 1st IEEE Int. Conf. Biomet-rics, Identity Security, Sep. 2009, pp. 1–10.

[55] M. Bicego, A. Lagorio, E. Grosso, and M. Tistarelli, “On the use of SIFTfeatures for face authentication,” in Proc. Conf. Comput. Vis. PatternRecognit. Workshop, Jun. 2006, p. 35.

[56] X. Li and F. Da, “Efficient 3 D face recognition handling facialexpression and hair occlusion,” Image Vis. Comput., vol. 30, no. 9,pp. 668–679, 2012.

[57] X. Li, X. Zhang, and C. Ren, “Structured sparse error coding for facerecognition with occlusion,” IEEE Trans. Image Process., vol. 22, no. 5,pp. 1889–1900, May 2013.

[58] B. Klare, Z. Li, and A. K. Jain, “Matching forensic sketches to mugshot photos,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 3,pp. 639–646, Mar. 2011.

Weiping Chen received the B.Sc. degree in elec-tronic engineering and automation from the ZhejiangUniversity of Science and Technology, Hangzhou,China, in 2001, and the M.Sc. and Ph.D. degreesin electronic engineering from Griffith University,Brisbane, Australia, in 2007 and 2012, respectively.His current research interests include face recogni-tion, biometrics, pattern recognition, and computervision.

Yongsheng Gao received the B.Sc. and M.Sc.degrees in electronic engineering from ZhejiangUniversity, Hangzhou, China, in 1985 and 1988,respectively, and the Ph.D. degree in computerengineering from Nanyang Technological University,Singapore. Currently, he is a Professor with theSchool of Engineering, Griffith University, Brisbane,Australia. His current research interests include facerecognition, biometrics, biosecurity, image retrieval,computer vision, pattern recognition, environmentalinformatics, and medical imaging.

Documents

4798 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL ...4798 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 22, NO. 12, DECEMBER 2013 Face Recognition Using Ensemble String Matching Weiping