Proceedings of the 2012 International Conference on Wavelet Analysis and Pattern Recognition, Xian, 15-17 July, 2012

INDIVIDUAL RECOGNITION BY GAIT IN VIRTUAL SPACE

YUAN-YUAN ZHANG, SHU-MING JIANG, ZID-QIANG WEI, JIAN-FENG ZHANG, SID-JIE XU

Information Research Institute, Shandong Academy of Sciences, Jinan 250014, ChinaE-MAIL: zhangyy@sdas.org,jsm@sdas.org, weizhq@sdas.org, zhangjf@sdas.org, xusj@sdas.org

(1)

Abstract:Individual recognition is the technique which recognizes

person's identity through his gait. Gait energy image (GEl) isa classical gait representation and it can be decomposed intostructural part and detailed part. Then virtual gait energyimage (VGEI) can be constructed in virtual space byintegrating those two different parts. The generalizedprincipal component analysis (GPCA) is applied to VGEI toreduce the dimension. The classical Euclidean distance is usedto measure the similarity of different gait features, and thenearest neighbor classifier is adopted to discriminate differentpatterns. The experiments on CASIA database testify theeffectiveness of the proposed algorithm.

Keywords:Individual recognition; Gait energy image; Virtual GEl;

Virtual space

1. Introduction

Gait, is the manner of human walking. Gait is aparticularly attractive modality for visual surveillancebecause it can be detected from a long distance andmeasured at low resolution. Recently, the study of gaitrecognition, which concerns recognizing individuals by theway they walk, has received an increasing interest fromresearchers in the computer vision community.

Generally, gait recognition techniques can be dividedinto two broad categories, namely model-basedapproaches[I-2] and model-free approaches[3-5], accordingto whether the gait features are extracted from models ornot. However, the majority of current techniques aremodel-free approaches. They typically analyze the imagesequence by motion or shape and characterize the wholemotion pattern of the human body by a compactrepresentation regardless of the underlying structure. Forexample, Wang[3] employed eigen-shapes to represent gaitsequences using the method of procrustes shape analysis.Zhang[4] extended eigen-shapes to a better description byshape context descriptor. Han[5] proposed a spatiotemporalgait representation called gait energy image (GEl) tocharacterize human walking properties.

978·1-4673·1535·7/121$31.00 ©2012 IEEE

In the domain of face recognition, Wang[6] employedthe method of bit-plane feature fusion to extract outlinefeatures and texture features respectively from people'sfrontal images. Then a new virtual face is constructed bycombining those two features. Inspired by Wang[6] andbased on GEl, this paper proposes a novel gaitrepresentation called virtual GEl (VGEI) to facilitate thetask of individual recognition by gait in virtual space.

2. Gait energy image

GEl is a spatiotemporal gait representation constructedfrom binary silhouettes [5]. It is obtained by averaging thesilhouettes of a pedestrian over one gait cycle. It can becomputed by the following formula:

1 NGEI(x,y)=-LIt (x,y)

N t=1

where, It is the tth binary silhouette image of a sequence, xand y are values in the 2D image coordinate, and N is theframe number of one gait cycle. GEl is the time-normalizedaccumulative energy image of human walking in acomplete cycle, and it is a robust feature since randomnoise has been reduced in the average process. Therepresentation also helps to save storage space andcomputation time. However, GEl is not suitable to bedirectly used because of its high dimension. Principalcomponent analysis (PCA)[7] is used here to obtain severalprincipal components to represent the original gait featuresfrom a high-dimensional measurement space to alow-dimensional eigen-space.

3. Virtual gait energy image

3.1. Bit-plane decomposition

From formula (1), we know that GEl is a gray image.Usually, the gray level of this kind of image is 256, andeach pixel of it can be represented by one byte memorycomposed of 8 binary elements.

170

Proceedings of the 2012 International Conference on Wavelet Analysis and Pattern Recognition, Xian, 15-17 July, 2012

Assumed that the resolution of GEl is XXV, then GElcan be stored by a 3-dimensional matrix with size of X X YX 8. According to this matrix, GEl can be easilydecomposed to 8 binary images with size of X X Y whenthe third dimension varies eight times. The binary imagesare called bit-planes of GEL Denote Bi represent the ithbit-plane, where i varies from 0 to 7. BO represents thebit-plane corresponding to the lowest bit, and B7 representsthe bit-plane corresponding to the highest bit.

In Figure 1, a typical GEl is shown in the first row andthe corresponding 8 bit-planes are shown in the second row.The left most bit-plane is BO and the right most is B7.

Figure 1.A typical GEl and its 8 bit-planes

3.2. Virtual GEl

As we can see in Figure 1, it is obviously that differentbit-plane captures different visual information about theoriginal GEL Specifically, B4, B5, B6, and B7 capture morestructural characteristics about human motion, while theothers capture more detailed features.

Similar to Wang's work[6], two independent imagescan be constructed using those bit-planes. One is structuralimage which contains the relatively more stable informationof GEl, and the other is detailed image which contains themore likely changeable information of GEL Then a virtualimage can be obtained by integrating the structural imageand the detailed image in virtual space. The virtual image iscalled virtual GEl (VGEI) in this paper.

For any GEl, the corresponding VGEI can becomputed by the following formula:

VGEI(x,y) =S(x,y)+ jD(x,y)7 7 (2)

=LPiBi(X,y)+ jLqiBi(X,y)i=O i=O

where, j denotes the imaginary unit, and x, y are values inthe 2D image coordinate. Sand D are the structural imageand the detailed image of GEl, respectively. The parameterspi and qi are the percentage of structural image and detailedimage to which the ith bit-plane belongs. pi and qi shouldsatisfy the following formula:

Pi +qi =1 (3)where, i varies from 0 to 7. Empirically, pi becomes largeras i grows, and qi becomes smaller as i grows.Theoretically, pi and qi is computed according to thecontribution to structural image and detailed image of Bi.However, it remains to further study in the followingresearch. These parameters are chosen by observation atthis stage.

Figure 2 gives two examples of VGEI of two differentpeople. In each row, from left to right, those 5 imagesdenote: the original GEl, the structural image S, the detailedimage D, the modulus image ofVGEI, and the phase imageofVGEI, respectively.

Figure 2. Original GEl and its structural image, detailed image,modulus image and phase image of VGEI of two different subjects

4. Generalized peA

We defme a n-dimensional complex-vector space asG={a+jPla, PER}, where a and P are n-dimensionalvectors in real space. Now, if we define the inner product as(x,y)=xty, where x, y E G. The complex-vector space inwhich the above inner product has been defined is calledunitary space.

In unitary space, given a set ofL classes including rol,ro2, ... , roL, and pattern x is an n-dimensionalcomplex-vector. The total scatter St can be expressed as

(4)

L

where, "'0 =E {x} =L P ( flJi ) mi denotes the total samplei=l

mean in unitary space, and P(roi) is the priori of class i,

while mi =E {X IflJi } is the sample mean of class i.

By the definition of formula (4), it is easy to verifythat St is an Hermite matrix, and it is of non-negativedefmite[8]. In unitary space, the corresponding PCAdiscriminate criterion is J(qJ)=qJTSUp, where qJ is ann-dimensional non-zero complex-vector. J(qJ) is thediscriminate criterion ofgeneralized PCA (GPCA).

The vector qJ that maximizing J( qJ) is called the

171

Proceedings of the 2012 International Conference on Wavelet Analysis and Pattern Recognition, Xian, 15-17 July, 2012

optimal projection axis. In fact, the optimal projection axisis the eigenvector corresponding to the maximal eigenvalueof St. Generally, a set of projection axes are selected tomaximize the J(qJ) and also subject to the orthonormalconstraints

rprSlPj =0, Vi *- j, i,j =1,...,d (5)

In fact, the optimal proj ection axes qJ1, qJ2, ... , qJd, ofGPCA can be selected as the orthonormal eigenvectors ofStassociated with the first d largest eigenvalues. Then, theGPCA transformation can be represented as

(6)

where, U=[qJl, qJ2, ... , qJd]T, f is a column matrix thattransformed from VGEI according to its rows, and c is theprojection of in GPCA subspace.

Figure 3 shows the comparison of eigengaits(eigenvectors represented as an image) between traditionalPCA and GPCA. The images in the first row are eigengaitsof GEl in traditional PCA subspace. The images in thesecond and the third row are modulus and phase ofeigengaits of VGEI in GPCA subspace, respectively. It isobvious that, the eigengaits derived from VGEI emphasizethe detailed information, while those from GEl emphasizethe holistic outline information.

Figure 3. The comparison of eigengaits between traditional PCA andGPCA

5. Gait recognition in virtual space

In the scheme of our work, the binary silhouettes ofgait sequence are the input of the system. The GEl of eachsequence can be easily obtained. Then the GEl can bedecomposed into eight bit-planes. Those bit-planes can becombined to structural image and detailed image accordingto different weights respectively. The corresponding VGEI

can be obtained by integrating the structural image anddetailed image in virtual space. GPCA is applied to VGEIto reduce the dimensions.

Gait recognition is a traditional pattern classificationproblem which can be solved by measuring similaritiesamong gait sequences. The classical Euclidean distance isused to measure the similarity of different gait features. Atlast, the nearest neighbor (NN) classifier is adopted todiscriminate different patterns. It determines the probepattern to have the same class with the gallery which is thenearest.

The block diagram in Figure 4 summarizes the majorsteps in a gait recognition system in virtual space.

Figure 4. The block diagram of proposed algorithm

6. Experimental results

We use dataset-B in CASIA gait database[9] to verifythe effectiveness of the proposed algorithm. This datasethas 124 different subjects. Each subject walked along astraight line for 10 times (6 times for normal walking, 2times with a bag, and 2 times with a coat). Some examplesof this dataset are shown in Figure 5. Here, we use all thesequences of 6 times' normal walking at lateral view. In theexperiment, the parameter p is set as [0 0 0 0.25 0.5 0.75 11], and q is set as [1 1 1 0.75 0.5 0.25 0 0].

Figure S. Samples in datllset-B of CASIA database

We use the leave-one-out cross-validation rule toobtain the unbiased estimate of the Correct ClassificationRate (CCR). Each time we leave one sequence out as aprobe sample and train on the remainder. The CCR of theproposed algorithm is reported in Table 1. For the benefit ofcomparison, the results of some recently publishedalgorithms are also listed in Table 1. From this table, it canbe easily seen that the proposed algorithm is outperformothers.

172

Proceedings of the 2012 International Conference on Wavelet Analysis and Pattern Recognition, Xian, 15-17 July, 2012

TABLE 1. COMPARISON TO OTHER ALGORITHMS IN TERMS OF CCR

Method CCR Method CCR

Wang, 2003 [3] 88.98% Zhang, 2009[4] 94.49%

Chen, 2009[10] 91.10% Zhang, 2010[13] 95.83%

Wang, 2009[11] 93.27% Han, 2006[ 5] 97.04%

Zhang,93.54% Proposed algorithm 98.25%

2009[12]

The recognition performance is also evaluated byCumulative Match Score (CMS). It is defmed as thecumulative probability that the real class of a testmeasurement is among its top matches. The rank is plottedalong the horizontal axis, and the vertical axis is thepercentage of correct matches. The CMS ofVGEI is shownin Figure 6. For comparison, the results of Wang[3] andHan[5] are also shown in this figure, and they are labeled as"Original-PMS" and "Original-GEl" respectively.

Figure 6. CMS on datllset-B of CASIA database

We also estimate False Acceptance Rate (FAR) andFalse Reject Rate (FRR) via the leave-one-out rule in termsof verification performance. The Receiver OperatingCharacteristic curves (ROC) is used to reflect the balancelevel of FAR and FRR, and it is shown in Figure 7. Theverification performance is better if the area below ROCcurve is smaller. It is obvious that the proposed VGEI isbetter than others.

Figure 7. ROC on datllset-B of CASIA database

7. Conclusions

This paper proposes a novel individual recognitionalgorithm using virtual gait energy image. Generalized PCAis used to reduce the dimension ofVGEL The experimentalresults on CASIA database indicate that the proposedalgorithm is simple but effect.

References

[1] S. L. Dockstader, M. J. Berg, A. M. Tekalp,"Stochastic kinematic modeling and feature extractionfor gait analysis", IEEE Transactions on I.P., Vol.l2,No.8, pp. 962-976, 2003.

[2] R. Urtasun, P. Fua, "3D Tracking for gaitcharacterization and recognition", Proceeding ofFG2004 Conference, Seoul, pp.17-22, May 2004.

[3] L. Wang, T. T. Tan, W. M. Hu, H. Z. Ning,"Automatic gait recognition based on statistical shapeanalysis", IEEE Transactions on I.P., Vol.12, No.9, pp.1120-1131, 2003.

[4] Y. Y. Zhang, X. J. Wu, Q. Q. Ruan, "Combiningprocrustes shape analysis and shape context descriptorfor silhouette-based gait recognition", ElectronicsLetters, Vo1.45, No.13, pp. 674-675, June 2009.

[5] J. Han, B. Bhanu, "Individual recognition using gaitenergy image", IEEE Transactions on P.A.M.I.,Vo1.28, No.2, pp. 316-322,2006.

[6] H. Wang, Y. Leng, Z. Wang, X. Wu, "Application ofimage correction and bit-plane fusion in generalizedPCA based face recognition", Pattern RecognitionLetters, Vo1.28, No.16, pp. 2352-2358, 2007.

[7] L. Sirovich, M. Kirby. "Low-dimensional procedurefor the characterization of human faces", Journal of

173

Proceedings of the 2012 International Conference on Wavelet Analysis and Pattern Recognition, Xian, 15-17 July, 2012

the Optical Society of America A, Vo1.4, No.3, pp.519-524. 1987.

[8] J. Yang, J. Y. Yang, "Generalized K-L transformbased combined feature extraction", PatternRecognition, Vo1.35, No.1, pp. 295-297, January2002.

[9] CASIA gait database. Online available:httpc//www.cbsr.ia.ac.cn/

[10] C. Chen, J. Liang, H. Zhao, H. Hu, J. Tian, "Framedifference energy image for gait recognition withincomplete silhouettes", Pattern Recognition Letters,Vol.30, No.11, pp. 977-984,2009.

[11] K. J. Wang, X. Y. Ben, L. L. Liu, W. Chen, "Gaitrecognition using information fusion of energy",

Journal of Huazhong University of Science andTechnology, Vol. 37, No.5, pp. 14-17, May. 2009. (inChinese)

[12] E. H. Zhang, H. B. Ma, J. W. Lu, Y. J. Chen, "Gaitrecognition using dynamic gait energy and PCA+LPPmethod", Proceeding of ICMLC 2009, vol.1, pp.50-53,2009

[13] Y. Y. Zhang, X. J. Wu, Q. Q. Ruan, "Statistical gaitrecognition based on tangent angle features", PatternRecognition and Artificial Intelligence, Vol.23, No.3,pp. 539-545. August 2010. (in Chinese)

174