Eyebrow Shape Analysis by Using a Modified Functional Curve Procrustes
Distance
Yishi Wang, Cuixian Chen, Midori Albert, Yaw Chang, Karl Ricanek
University of North Carolina Wilmington
{wangy, chenc, albertm, changy, ricanekk}@uncw.edu
Abstract
To tackle the problem of automatic recognition of hu-
man eyebrow, a novel approach for shape analysis based
on frontal face images is proposed in this paper. First, eye-
brow curves are acquired by fitting cubic splines based on
landmark points. Next, we propose to use a modified func-
tional curve procrustes distance to measure the similarities
among the cubic splines, and finally a multidimensional
scaling method is adopted to evaluate the effectiveness of
the distance. This work extends previous work in analyzing
the eyebrow for both human and machine recognition by
providing a framework based on shape contours. Further
this work demonstrates the effectiveness of eyebrow shape
for discrimination when teamed with the appropriate metric
distance.
1. Introduction
As the most common facial hairs among men and
women, eyebrows can convey both subtle and bold expres-
sions such as fear, anger, surprise, contempt, happiness,
sadness, etc. However, the human eyebrow is an often over-
looked facial component thought to not to have much value
in for automatic recognition. In the area of identity sciences
including biometrics, where the major interests lie in iris,
ear, palm, fingerprint, and the face as a whole, there have
been very few studies focused on the eyebrow for recogni-
tion and/or soft-biometrics. There has been some interest in
the periocular recognition and soft-biometrics, which may
or may not include the eyebrow. It is this inconsistency of
definition for the periocular region, which has led to this
study.
In a forensic context, faces of suspects captured in
surveillance photographs may be partially covered, such as
by masks or sunglasses. An interesting question is: when
occlusions exist, based on an exposed part, such as eye-
brows, would it be possible to use eyebrow as a reliable
tool to identify a person?
Figure 1. Example of importance of Eyebrow in face recognition.
1.1. Prior work
As indicated in Fig 1, It is a challenge to examine the
magnitudes of difference in the eyebrow between different
individuals as well as among different images of the same
individual. The role of eyebrows in face recognition has
been studied in [10], where the work revealed that the eye-
brow was far more important than eyes for human recog-
nition. However, in [10] only human perception was dis-
cussed, rather than using biometric modeling. Bharadwaj et
al. [1] demonstrated that better automatic recognition was
possible with the periocular region if the eyebrow was in-
cluded. Since the focus of this work was on the texture
from the periocular region, it is unclear whether shape was
a factor in this work. In [4], eyebrow region was manually
segmented, and features were extracted and calculated ac-
cording to three categories: global shape feature, local area
feature, and critical point feature. Classifications were then
conducted based on the three feature categories. These fea-
tures, especially the global shape feature, maybe subjective
and not general enough to be finely quantified. Li et al. [8]
studied an automatic human eyebrow recognition system
via fast template matching and Fourier spectrum distance,
and concluded that eyebrow can serve as an independent
biometric for human recognition.
1.2. Contribution of work
In this work, we study the shape of the human eyebrows
by using a modified functional curve procrustes distance to
1
Figure 2. Framework for eyebrow classification.
develop a eyebrow classification system. The system may
have a great potential for person identification or verifica-
tion. Our major research questions are: (1) how do we rep-
resent the shape of eyebrows quantitatively and effectively,
(2) how can we measure the distance between different eye-
brows, and most importantly, (3) how can we present the
difference in an efficient and meaningful way?
To answer all these questions, we design multiple ex-
periments to discover whether or not eyebrow measures are
unique enough to be a good identifier. Through systematic
evaluation, theoretical and experimental results suggest that
our novel functional curve procrustes distance is an effec-
tive metric distance. The proposed framework is shown in
Figure 2. First, the eyebrows are labeled by ten landmarks
that best describe the general boundaries. Then we fit those
top five points by using cubic spline. Hereafter we have
a set of functions with both ending points known. In this
work, we only consider the top boundaries of left eyebrows,
focusing on the curve comparison. This approach can easily
be extended to address the shape comparison of the entire
eyebrows.
The next challenge is how to compare different eyebrow
curve shapes. In the literature, there have been numerous re-
search on comparing/contrasting shape of objects and pla-
nar curves, such as Hausdorff distance, Frechet distance,
procrustes distance [5], inner distance and etc. In this work,
we propose the modified functional curve procrustes dis-
tance because of its simplicity for calculation and effective-
ness for distance representation. From the previous steps,
with n eyebrows, we end up with n(n − 1)/2 distances. A
natural question is how can we present these distances in
an effective way, i.e., can we project the distances into a
lower dimensional space? We use multidimensional scaling
to reduce the dimension of the distances.
This contribution can be summarized as: (1) We develop
a descriptor of the landmarks shown in Figure 3 and their
location for the quantitative analysis. (2) In our preprocess-
ing step, we use cubic spline to fit the landmarks and ob-
tain a smooth curve. (3) We propose a modified functional
Figure 3. Landmarking scheme for left eyebrow and example of
landmarking.
curve shape procrustes distance to measure the differences
of eyebrow curves, to overcome the difficulties caused by
the relative flat curvatures. (4) To evaluate our proposed
approach systematically, first we manually select a set of
28 Caucasian female eyebrows from the UIUC-PAL dataset
for automatic eyebrow classification. Next, we compare the
results from the proposed approach with human perception
results. Finally, the proposed approach is evaluated on 437
Caucasian eyebrows from PAL-dataset. Our experiment on
PAL dataset suggests that our proposed approach to au-
tomatic eyebrow classification achieves higher recognition
rates than humans.
The organization of this paper is laid out as follows: Sec-
tion 2 presents landmarking scheme and cubic-spline curve
fitting. Section 3 introduces the properties of the new func-
tional curve shape procrustes distance to measure the dis-
tance between functional curves. Section 4 presents the
techniques of Multidimensional Scaling (MDS). The exper-
iment results are presented in Section 5, and conclusions are
drawn in final section of this paper.
2. Eyebrow shapes
In our preliminary study, we find out that it is very
difficult to determine the boundaries of human eyebrows.
Some people have very bushy eyebrows, while it is com-
mon for the seniors to have sparse ones. Therefore, we
develop a descriptor of the landmarks shown in Fig 3 and
their location for the quantitative shape analysis with the
top left eyebrows of n subjects, (xi, yi), for i = 1, · · · , n,
where xi = (xi,0, ..., xi,4)T be the x coordinates and yi =
(yi,0, ..., yi,4)T be the y coordinates of the labeled landmark
points 0-4. Due to the nature of the human eyebrows, it is
easier to consider the general eyebrow shapes as intersec-
tions of two continuous curves.
Next, we use statistical curve fitting to approximate the
boundaries of eyebrows. The general idea of curve fitting
with given landmark points is to find a continuous function
that passes through those landmark points while being rel-
atively smooth. The way to achieve such a goal is to find a
twice differentiable function such that among all functions
f(x), the following objective function is minimized:
L(f, λ) =5
∑
i=1
(yi − f(xi))2 + λ
∫
(f′′
(x))2dx.
2
According to Green and Silverman (1994) [6], under certain
conditions, the above loss function has a unique minimizer:
a natural cubic spline that passes through (xi,j , yi,j) for
each fixed i and j = 0, · · · , 4. In our preliminary study, we
test on multiple curve fitting techniques and find out that the
cubic spline is effective and the resulting curves are good
approximation of the true eyebrow curves. With a group
of curves, our next goal is to define the distances between
these curves, such that the distances can be used to measure
the similarities and differences of the shapes among the eye-
brows.
Let F = {fi : fi is the top-left eyebrow spline function
of the ith subject} be a set of continuous function both end-
ing points provided. Without loss of generality, we may as-
sume that the slopes of the straight lines connecting the two
ending points are zero and the Euclidean distance is one.
3. Distances of planar functional curves
We propose to use Modified Functional Curve Procrustes
Distance, to measure the similarities between different con-
tinuous curves. The definitions were proposed as following:
Definition 1. (Equally spaced heights with zero ending
points). Let f : [0, 1] 7→ R be a continuous function
with f(0) = f(1) = 0, and let 0 = t1 < ... <tn = 1 be n equally spaced points on [0, 1] , then F :=(f(t1), f(t2), · · · , f(tn))T is called equally spaced heights
of f on [0, 1].
Hereafter, we assume that all curves considered are zero
at ending points of their domain. Therefore, when the
equally spaced heights with zero ending points of a function
is a constant, it indicates that the constant is zero. Gener-
ally, the equally spaced heights may be positive or negative,
depending on the curve f and where the sampled points are
located.
Definition 2. (Equally spaced distance (ESD) between two
continuous functions) Let g and h be two continuous curves
on closed domains, and let U = (u1, ..., un)T and V =(v1, ..., vn)T be the equally spaced heights with zero ending
points of g and h respectively. When ||U ||2 · ||V ||2 > 0, the
equally spaced distance between curves g and h is
Dn(U, V ) :=
√
1 −< U, V >2
||U ||22 · ||V ||22, (1)
where < ·, · > is the inner product of two vectors, and || · ||2is the L2 norm.
Let f1, f2 and f3 be continuous functions on [0, 1] with
both ending points equal to zero, let X = (x1, ..., xn)T ,
Y = (y1, ..., yn)T and Z = (z1, ..., zn)T be their corre-
sponding equally spaced heights, with ||X|| · ||Y || · ||Z|| >
0, it is obvious that Dn(f1, f1) = 0, and D(f1, f2) =D(f2, f1) ≥ 0. If D(f1, f2) = 0, it indicates that∑n
i=1 |xiyi| = ||x||||y||. By Cauchy-Schwartz inequality,
we have X = rY , where r is a scalar. As to the triangle
inequality, it can be proved that
Theorem 3. With the previous notations ,
Dn(f1, f2) ≤ Dn(f1, f3) + Dn(f2, f3).
Therefore Dn is a pseudo metric.
Notice that for (1), if we replace U and V by function gand h respectively, change the inner product and L2 norm
of vectors to the ones for functions, we have
D(g, h) :=
√
1 −< g, h >2
||g||22 · ||h||22
, (2)
where
< g, h >=
∫ 1
0
g(x)f(x)dx, and ||g||22 =
∫ 1
0
g2(x)dx.
In fact, the distance D(g, h) in (2) is the limit of Dn(g, h)in (1) as n approaches to infinity. It can be verified that
D(g, h) is also a pseudo metric with the following theorem.
Theorem 4. With the conditions and notation in Theorem 3,
D(f1, f2) ≤ D(f1, f3) + D(f2, f3).
The distance D(g, h) is similar with the the distance pro-
posed in [7], in which the functions take a complex format
to accommodate both the x and y variables. In this work,
because of the way we register our function, we can only
focus on the y variable, which is exactly the reason why
all we need is the vertical heights used in the definition of
Dn in (1). This simplification should reduce the compu-
tation time since only y coordinates are involved. Similar
with the distance proposed in [7], all the above mentioned
distances do not allow constant functions or vectors. How-
ever, in many real world application, such a limitation can
be a big hurdle. There is hardly a function that is constant,
but there are many planar curves that are very similar with
constant curves. For example, in our eyebrow classification
experiment, some eyebrows demonstrate relatively flat cur-
vature. In order to overcome this difficulty, we propose to
add a constant parameter k > 0 to U , V , g and h in (1) and
(2), and therefor the revised distance of (1) and (2) are:
Dkn(U, V ) :=
√
1 −< U + k, V + k >2
||U + k||22 · ||V + k||22, (3)
which we name as modified discrete procrustes distance,
and
3
Dk(g, h) :=
√
1 −< g + k, h + k >2
||g + k||22 · ||h + k||22, (4)
which we name as modified functional curve procrustes dis-
tance. It can be proved that both (3) and (4) are metric dis-
tance and now it allows for U or V , or g or h to be con-
stant(s).
Let fi : [0, 1] 7→ R for 0 ≤ i ≤ n be n cubic splines (or
let F1, ..., Fn be the corresponding equally spaced heights),
we may calculate the modified functional curve procrustes
in (4) for each pair, and hence there aren(n−1)
2 distances
among all pairs. A n × n matrix S is often used with the
(i, j) element represents the distance between the ith and
jth functions, i.e., si,j = Dk(Fi, Fj). Obviously, S is sym-
metric with diagonal elements equal to zero.
4. Multidimensional scaling
Because of the nature of the matrix S, it is difficult to
have a visual idea on the relationships among all functions,
in terms of the similarities of shapes. The challenge is
then to reduce the dimension of the distances fromn(n−1)
2to two or three dimensions where visual understanding of
the similarities is easier. In this work, we use Multidimen-
sional Scaling (MDS) [11] to reduce the dimension. With
the aforementioned distance matrix Sn×n = (si,j), MDS
searches vectors z1, · · · , zn ∈ Rd to minimize the follow-
ing objective function:
LM (z1, · · · , zn) :=∑
i 6=j
(si,j − ||zi − zj ||2)2. (5)
The major idea of MDS is to use zi, for i ∈ 1, · · · , n as a
lower dimensional representation of the data that preserves
the pairwise distances, si,j , as much as possible. For vari-
ous objective functions for different types of MDS, [2] con-
tains a great amount of information. For (5), essentially we
are looking for the approximation that
s2i,j ≈ ||zi − zj ||
22 = ||zi||
22 + ||zj ||
22 − 2zT
i zj
Since it is the inner product of zi and zj that we are most
interested in, the above expression is equivalent with
−1
2s2
i,j ≈ zTi zj − ||zi||
22/2 − ||zj ||
22/2.
Let A = (− 12s2
i,j) and let B be the centered version of A in
the following sense
B = HAH where H = In − Jn/n,
and Jn is an n × n matrix of one.
Thus, the optimization of objective function in (5) be-
comes an eigenvalue problem. The minimizer of (5) has an
explicit solution by using the largest d eigenvalues of matrix
B and the corresponding eigenvectors. Let λ1, · · · , λd be
the largest eigenvalues of matrix B with associated eigen-
vectors e1, · · · , ed, then
(zT1 , · · · , zT
n )T = (√
λ1e1, · · · ,√
λded).
In the following experiments, we will present the effec-
tiveness of the distance in (4) by using MDS with d = 2.
0.0 0.2 0.4 0.6 0.8 1.0
0.0
0.1
0.2
0.3
0.4
0.5
0.6
seq(0, 1, length = 100)
sim
.fit[1
, ]
A
B
C
DE F
Figure 4. Simulated eyebrows: flat (type A), slightly round (type
B), moderately round (type C), highly round (type D), skewed to
the left/arched (type E), and skewed to the right/arched (type F).
0.0 0.2 0.4 0.6 0.8 1.0
0.0
0.1
0.2
0.3
0.4
0.5
0.6
seq(0, 1, length = 100)
rea
l.T
L.f
it[i,
]
Figure 5. Top-left eyebrows for 28 manually selected females on
PAL dataset.
5. Experiment
In this section we shall systematically evaluate the ef-
fectiveness of eyebrow classification system based on the
modified functional curve procrustes distance.
5.1. Simulated eyebrows
In order to access the effectiveness of the distances in (4)
as well as for shape classification purpose, six types of sim-
ulated eyebrows are generated: flat (type A), slightly round
(type B), moderately round (type C), highly round (type D),
skewed to the left (type E), and skewed to the right (type F).
Note that the simulated eyebrows are generated from func-
tions of probability density function of Uniform distribu-
tion, Normal distribution and F distribution independently.
4
For each type of eyebrow, we generate four variations by
adding normal random noises. Thus we have 24 simulated
eyebrow totally. Here the random noises are carefully cho-
sen, so that the general shape is not affected. The shapes of
the simulated eyebrows are in Figure 4. We aim to reveal
which categories, A-F, our real eyebrows fall into or close
to. It is worth to emphasize that in our human perception
experiment, we only classify the real eyebrows into three
categories: flat (type A), round (type B-D), angled (type E,
F).
Figure 6. PAL Sample images: flat, round, round, and angled. Im-
age ID from left to right: 9, 17, 27, 13. We conduct two sets of
human perceptions to classify the eyebrows into three categories:
flat, round, and angled.
5.2. Face and eyebrow database
The Productivity Aging Laboratory (PAL) face database
[9] is selected for this experiment due to its quality of im-
ages. Only the frontal images with neutral facial expression
are selected for our eyebrow classification algorithm. (See
Figure 6 for sample images.)
First, for our eyebrow classification and human percep-
tion tasks, we manually select 28 Caucasian female images,
with age range 18-86, to control ethnicity and gender ef-
fects, as shown in Figure 5. In our preliminary study, we de-
velop a descriptor of the landmarks assigned and their loca-
tions for quantitative analysis, which is sufficient to capture
the details of the shape of eyebrow, yet not redundant. Then
each eyebrow is manually labeled with landmarks assigned
as points along the eyebrow. Next, for our 28 females im-
ages, all top left eyebrows are fitted by using cubic splines
discussed in section 2. Finally, the proposed approach is
test on 437 Caucasian eyebrows from PAL-dataset.
5.3. Experiment results
Compared with the curves from the top left eyebrows
of our 28 subjects, the simulated ones cover most of the
variations that we have from the real ones. In the following
experiments, we use the distance defined (4) with k = 0.5.
0.0 0.2 0.4 0.6 0.8 1.0
0.0
0.1
0.2
0.3
0.4
0
0
0.0 0.2 0.4 0.6 0.8 1.0
0.0
0.1
0.2
0.3
0.4
0
0
0.0 0.2 0.4 0.6 0.8 1.0
0.0
0.1
0.2
0.3
0.4
0
0
0.0 0.2 0.4 0.6 0.8 1.0
0.0
0.1
0.2
0.3
0.4
0
0
Simulated eyebrows
Flat eyebrows
Round eyebrows
Angled eyebrows
Figure 7. Eyebrow classification results based on 3D MDS of eye-
brow shapes by the Modified Functional Curve Shape Procrustes
Distance.
Image ID
green 7, 9, 11, 21, 24, 26, 15, 10, and 18
black 1-6, 16, 17, 19, 20, 23, 25, and 27
blue 8, 12, 13, 14, 22, and 28.Table 1. Summary to eyebrow classification results on 28 selected
female eyebrows.
First, we aim to explore the simulated eyebrows only.
Figure 8 clearly shows that the four symmetric types (A, B,
C, D) stay in the center, while type E and type F are on the
two sides. The layout of type A through type D is according
to the round levels of the simulated eyebrows, with type A
and type D on the two far ends and type B and type C in
the middle. Moreover, the variations of each eyebrow type
stay much closer to the ones of its own type than the ones
of the other types. This figure validates the effectiveness of
the distance in (4) for the simulated eyebrows.
Next, when we consider the 28 selected eyebrows with
the simulated ones from Figure 8, it suggests that the eye-
brow shapes are within the triangle of type A, Type C and
Type F. The most flat (type A) eyebrows are from images
with ID: 9 and 21, the most round eyebrow is from im-
age with ID 17, and the 28th and 13th images are the most
skewed ones. However, no eyebrow is as skewed to the
right as the 13th skewed to the left. Eyebrows from subject
number 9, 3, 27, 17, 13 etc. are among the most extreme
ones. Moreover, from the face and eyebrow sample images,
it can be clearly seen that the eyebrow shapes change con-
tinuously and smoothly, from bottom (flat) to top (round),
from left to right (change of skewness). The bottom left
corner image shows the shapes of simulated eyebrows, il-
lustrating modes of variation in shapes.
For human perception purpose, three professional and
well-trained anthropology scholars involve in our human
5
Figure 8. Distribution of the 24 simulated eyebrows and 28 female top-left sample eyebrows from PAL dataset. Simulated eyebrows include
Types of A to F with four variations in each type by adding normal random noise.
perception study. We conduct two sets of human percep-
tions to classify the eyebrows into 3 categories as: flat,
round, and angled. In the first experiment, three scholars
are required to classify the global eyebrow features with all
facial features. In the second experiment, three scholars are
required to classify the fitting curves from landmarks as lo-
cal eyebrow features without any facial features. In the first
experiment, there is only 16 out of 28 eyebrow shapes are
classified by all three scholars consistently, which gives us a
57.14% 3-way agreement. While in the second experiment,
there is a significant improvement of 3-way agreement of 22
out of 28, that is 78.57%. From this perception study, we
find out that it is subjective for humans to classify shapes
even for well trained professional anthropologists and even
when the facial features are removed.
We then consider to build an automatic classifier to clas-
sify real eyebrows into three categories: flat, round, and an-
gles. We consider the Libsvm as our 3-class classifier [3].
Hereafter, we adopt the first three coordinates from MDS
to preserve more information. We use four simulated flat
eyebrows (type A) as our class 1 training set; four varia-
tions from type C as class 2, and four simulated angled eye-
brows (type F) for class 3. Due to lack of ground truth of
the shape classification from the human perception study, to
efficiently illustrate our classification outputs, we compare
the classification outputs of the real eyebrow samples to the
simulated data in Fig 7. The red curves are the simulated
eyebrow curves, while the green, black, and blue curves are
the real eyebrows that are classified as flat, round and an-
gled, respectively. Table 1 shows the specific classification
results. As we compare the classification results from Fig-
ure 7 with the 2D projection in Figure 8, the results are very
consistent and promising.
Finally, we evaluate our proposed approach on the PAL-
dataset with 437 Caucasian eyebrows. The results are
shown in Figure 9, which is similar to Figure 8.However,
it is interesting to find out the majority of eyebrows stay in
between round and skewed to the left, which is the same
finding from our perception study. From both Figure 7 and
Figure 9, similar eyebrows are clustered together while the
extraneous ones are around the boundaries, such as ID 362,
304, 355, 208, 247, etc. It indicates that our proposed ap-
proach provides an effective metric distances among eye-
brow curves for further classification task.
6. Conclusion and future research direction
In this work, a novel curve shape classification approach
is proposed and applied to the analysis of human eye-
brows, based on a modified functional curve procrustes dis-
tance.Experiment results suggest that, when combined with
MDS, the intrinsic distances between convex 2D shapes
can be efficiently represented with a two dimension space.
6
Figure 9. Distribution of the 24 simulated eyebrows and 437 Caucasian top-left sample eyebrows from PAL dataset.
Further research on this work includes but is not lim-
ited to: combination all portions of eyebrow curves with
length/width, area and color of eyebrows; evaluating the
performance on person identification/verification; investi-
gating the gender/age differences with eyebrows.
7. Acknowledgment
This work has been partially funded by Army Research
Lab in conjunction with the Center for Advanced Studies in
Identity Sciences and the FBI Biometric Center of Excel-
lence.
The authors would like to thank SAMSI for its workshop
series. The authors would also like to thank Christopher
Maier, Jessica Daley, Olivia Smith for assistance in eyebrow
landmarking and human perceptions.
References
[1] S. Bharadwaj, H. Bhatt, M. Vatsa, and R. Singh. Periocular
biometrics: When iris recognition fails. In BTAS 2010: 1-6.
[2] I. Borg and P. Groenen. Modern Multidimensional Scaling:
theory and applications. New York: Springer-Verlag, 2005.
[3] C.-C. Chang and C.-J. Lin. LIBSVM: a library for
support vector machines, 2001. Software available at
http://www.csie.ntu.edu.tw/ cjlin/libsvm.
[4] Y. Dong and D. Woodard. Eyebrow shape-based features for
biometric recognition and gender classification: A feasibil-
ity study. In International Joint Conference on Biometrics,
2011.
[5] I. L. Dryden and K. V. Mardia. Statistical Shape Analysis.
John Wiley and Sons, 1998.
[6] P. J. Green and B. W. Silverman. Nonparametric Regres-
sion and Generalized Linear Models: A roughness penalty
approach. Chapman and Hall, 1994.
[7] R. Larsen. Functional 2d procrustes shape analysis. In
SCIA’05, pages 205–213, 2005.
[8] Y. Li, H. Li, and Z. Cai. Human eyebrow recognition in
the matching-recognizing framework. Computer Vision and
Image Understanding, 117, 2013.
[9] M. Minear and D. C. Park. A lifespan database of adult facial
stimuli. Behavior Research Methods, Instruments, & Com-
puters, 36:630–633, 2004.
[10] J. Sadr, I. Jarudi, and P. Sinha. The role of eyebrows in face
recognition. Perception, 32(3):285–293, 2003.
[11] W. S. Torgerson. Multidimensional scaling: I. theory and
method. Psychometrika, 17(4):401–419, 1952.
7