Person Appearance Modeling and Orientation Estimationusing Spherical Harmonics

M. C. Liem and D. M. Gavrila

Abstract— We present a novel approach for the joint estima-tion of a person’s overall body orientation, 3D shape and tex-ture, from overlapping cameras. Overall body orientation (i.e.rotation around torso major axis) is estimated by minimizingthe difference between a learned texture model in a canonicalorientation and a texture sampled using the current 3D shapeestimate (i.e. torso and head). The estimated body orientationsubsequently allows to update the 3D shape estimate, taking intoaccount the new 3D shape measurement obtained by volumecarving. Our main contribution is a method for estimating aperson’s relative body orientation while simultaneously gener-ating a basic Spherical Harmonics based model of the person’sshape and texture.

Experiments show that the proposed method outperformstwo state-of-the-art orientation estimation methods: one com-bining a fixed 3D shape model with a generate-and-test texturematching approach and one using a classifier based approach.

I. INTRODUCTION

3D human pose is an important feature for human activityand social behavior analysis. As part of the 3D pose, aperson’s body orientation (i.e. rotation around 3D torso majoraxis) conveys much important information about the person’scurrent activity, whether indicating the person’s direction offocus, direction of movement or interaction level with otherpeople in the scene.

In this paper, we focus on the estimation of overall personbody orientation from few, overlapping cameras. To obtaina more accurate orientation estimate, we jointly estimate itwith a 3D shape and texture representation of a person’storso-head, under a rigidity assumption. By projection ontoa basis of Spherical Harmonics (SH), a low dimensionalappearance model is created that can cope with object defor-mations while retaining the spatial information captured in atextured 3D representation. By comparing the texture modelof the person at consecutive points in time, the relative bodyorientation can be estimated elegantly using the propertiesof the SH. This in turn allows to update the 3D shape andtexture model of a person.

Apart from facilitating an accurate orientation estimate,the proposed 3D shape and texture model has furthermorethe potential to offer improved track disambiguation in amultiple person scenario, compared to a simpler 2D basedmodel. It could also provide an initialization for applicationsaiming at full articulated 3D pose recovery.

M. C. Liem and D. M. Gavrila are with the Intelligent Systems Labo-ratory, University of Amsterdam, 1098 XH Amsterdam, The Netherlands{M.C.Liem,D.M.Gavrila}@uva.nl

This research has received funding from the EC’s Seventh FrameworkProgramme under grant agreement number 218197, the ADABTS project.

II. PREVIOUS WORK

Estimating person orientation has been addressed in mul-tiple ways. Several 2D single frame approaches combineorientation-specific person detectors [1], [2], [3]. Work by[4] estimates body and head pose in batch mode, couplingthe output of underlying classifiers. In [5], a texture sampledfrom a cylinder surrounding the person is shifted along therotational axis to find the best matching orientation andsimultaneously form an appearance model.

Modeling person appearance is most commonly donebased on single view information. Color histograms repre-senting the full extent of a person’s appearance are often used[6], while more sophisticated methods split a single person’sappearance up into several layers, incorporating some spatialinformation in the descriptor [7], [8]. Some approaches alsocombine histograms created at multiple viewpoints [7].

Since the human body shape is largely non-rigid, modelingbody texture is not straightforward. One approach is to ignorethe shape changes over time and map a full-body texture ontoa rigid approximation of the body shape (e.g. a cylinder [5]).An alternative approach is to aim for full articulated 3D bodypose and model estimation, with all the challenges involved(robustness, computational cost), as done in [9], [10], [11].

SH have been used in order to perform face recognitionunder varying lighting conditions [12]. Representing 3D ob-jects by projecting them onto a SH basis has been researchedmainly with respect to exemplar based 3D object retrieval. Acollection of rotationally invariant 3D object descriptors hasbeen compared in [13]. Recently, a SH decomposition of a3D extension of the HOG descriptor was presented in [14].

In this paper, we take an intermediate approach betweena fixed body shape and full articulated pose. We estimatethe largest rigid partition (core) of the body over time byregarding all non-rigid elements of the human body (arms,legs) as ‘noise’ around the rigid body core (torso-head).Our main contribution is a method for estimating a person’srelative body orientation while simultaneously generating abasic model of the person’s shape and texture. The estimateis made on a single frame basis using an on-line learnedappearance model consisting of low dimensional SH shapeand texture representations, without the need for any a-priorishape or texture information. By using SH as a basis, orienta-tion estimation can be performed elegantly, without the needof explicitly testing for different orientations. Furthermore,the low dimensional representation gives us an appearancemodel that is robust against local changes in shape or textureby implicitly smoothing the models.

(a) SH components for geometric deformation

(b) SH components for (color) gradient

Fig. 1. Spherical basis functions for the first three SH bands (L = 2).SH components can be visualized by representing spherical function f asa deformation of a sphere (a) or as a texture gradient (b).

III. SPHERICAL HARMONICS

In order to simplify the estimation of a person’s orientationat time t based on a shape model St and texture model Tt, werepresent both in a Spherical Harmonic (SH) subspace. SHare the equivalent of Fourier transformations on a 3D sphere;they decompose a spherical function f(u, v) (a functiondefined on the surface of a sphere) into a linear combinationof orthonormal spherical basis functions. A SH subspaceconsists of L bands, and M components (spherical basisfunctions) per band, where for a certain band 0 ≤ l ≤ L,M = 2l + 1. The SH decomposition of a spherical functionf computes SH parameters A, where Al,m is the weight ofcomponent m in band l such that SFA is the reconstructionof f from A, thus SFA(u, v) = f(u, v). In practice we usea limited number of bands such that SFA is a band limitedapproximation of f .

If a 3D shape is represented by fS as the protrusion of asphere along vertices (u, v), then its SH decomposition canbe seen as a linear combination of canonical shapes of whichfig. 1(a) shows the first 9. Analogous, the SH decompositionof a texture represented by fT as color values at vertices(u, v) can be seen as a linear combination of gradient mapsseen in fig. 1(b). More details on SH and how to decomposea 3D object into SH components can be found in [15].

Equivalent to the translational invariance of Fourier trans-formations, SH are rotationally invariant. This means thatany rotation of a 3D object described by SH components isthe same as the rotation of its SH components. Because SHform an orthonormal basis, a rotation of an object projectedonto a SH basis results in a shift of the components weights

in each band [16]. Rotation of a 3D object along the verticalaxis in SH is done according to the following function:

Al,me−imφ = Bl,m. (1)

Here Al,m is the weight of component m in band l beforerotation, i is the imaginary unit and Bl,m is the weightof component m in band l after rotation over angle φ.For notational simplicity, we will sometimes leave out thesuperscript l,m when describing the rotation of a completeSH object and use Aφ to denote B. In those cases, rotationis assumed to be applied to all components l,m of A.

In case a SH parametrization A and B of same objectis given, the rotation φ between them can be found byminimizing the following sum squared error (SSE) using aquasi-Newton approach like BFGS [16]:∑

1≤l≤L

∑1≤m≤l

(Al,me−imφ − Bl,m)2. (2)

Here, L is the number of bands selected for representingthe object. The 0th component of every band is not neededfor this estimation since it is not influenced under rotation.

IV. 3D APPEARANCE

At each time step t, C images I1:Ct are captured from

C different, overlapping viewpoints. In our experiments,C = 3. Foreground estimation is done for each image anda volumetric reconstruction of the scene is created usingvolume carving. While there is only one person in the scene,all reconstructed volumes too small to represent a person arediscarded. The remaining set of voxels is the person’s visualhull H. The person’s measured position xt is based on theground plane projection of the center of the principal axis ofH. A constant acceleration Kalman filter is used to filter xtover time and gives the filtered person position xt.

In order to use SH to model the person’s shape, H istransformed into spherical shape function fS . To create fS ,a unit sphere is centered on the person location xt and thesphere’s surface is discretized into a uniformly distributed setof surface points (in our experiments we use 55×55 surfacepoints). For each surface point (u, v), a ray r is cast from xt,trough (u, v). The spherical function is defined by fS(u, v) =d, where d is the distance between xt and the most distantvoxel inH along r. This is similar to the method described in[17] except that, in order to maintain rotational information,we do not normalize the 3D shape. To compensate for thefact that volume carving tends to over-estimate the shape, thespherical function is scaled down 10% to prevent samplingof the texture outside the object during texture generation.

A sampled texture consists of a spherical texture functionfT , created by projecting the surface points of a shapefunction fS onto images I1:C

t and sampling the image valuesat these locations. Texture is only sampled from imageregions containing foreground and the color of surface pointsvisible in multiple cameras is averaged over these cameras.

The goal is to model the person’s appearance, consisting ofa SH shape model St and a SH texture model Tt, and estimate

Algorithm 1: Appearance modeling and orientation es-timation

Input: St−1, Tt−1, xt−1, φt−1, I1:Ct , HOutput: St, Tt, xt, φt

1 xt = person position based on average voxel position of H;2 x−t = Kalman filter prediction of person position using xt−1;3 xt = Kalman filter update of x−t using measurement xt;4 φ−t = Kalman filter prediction of orientation using φt−1;5 ∆φ =∞; Intermediate orientation difference;// Determine texture model Tt and orientation φt

6 while ∆φ ≥ 0.001 do7 if not first iteration then φt = φt + ∆φ;8 else φt = φ−t ;9 Rotate shape model: Sφt−1 = St−1e

imφt ;10 Reconstruct spherical function SFSφ from Sφt−1;11 Position SFSφ at xt;12 Project surface points SFSφ (u, v) onto I1:Ct ;13 Sample texture from I1:Ct at projected surface points (i, j), creating

fT (u, v) = 1C

∑C I

ct (i, j);

14 Compute SH representation Tt of fT ;15 ∆φ = arg minφ[

∑λ

∑l

∑m (T l,m,λt e−imφ − T l,m,λt−1 )2];

16 φt = Kalman filter update of φ−t using measurement φt;17 if t < 1/αT then18 Tt = t−1

t Tt−1 + 1t Tt;

19 else20 Tt = (1− αT )Tt−1 + αT Tt;

// Compute shape model St based on φt21 Position a unit sphere with 55× 55 discretized surface points at xt;22 foreach surface point (u, v) do23 Cast ray r from xt trough (u, v);24 Determine distance d between xt and the voxel in H along r furthest

away from xt;25 Compute scaled fS using fS(u, v) = 0.9d;

26 Compute the SH representation St of fS ;27 ∀l,m ∈ St : Sφt = Ste−imφt ;28 S0 = S0; St = (1− αS)St−1 + αS Sφt ;

the person’s orientation φt (rotation along the vertical axis)based on the estimated appearance.

Algorithm 1 describes the SH based appearance modelingand orientation estimation method. The different parts of thismethod will be explained in this section.

A. Texture Model

First, a constant acceleration Kalman filter is used to geta prediction φ−t of person orientation φt based on φt−1.Using the shape model St−1 from the previous timestep, theSH texture measurement Tt at time t is computed and theperson’s orientation φt is estimated using this measurement.

Texture function fT at t is created using a rotated sphericalshape function SFSφ , reconstructed from the SH shape modelSφt−1 acquired by rotating St−1 using the predicted orienta-tion according to line 9 of alg. 1. When positioned at xt,SFSφ ’s discretized surface points can be used to sample theperson’s texture from images I1:C

t . Texture regions withoutcolor information due to occlusions or sampling outside theforeground regions are filled using the average color alonghorizontal scanning lines over the texture. This way, artifactsin the SH texture representation due to lacking informationare prevented while vertical color variance is maintained inthe texture. Tt is computed by projecting fT onto a 9 bandSH subspace (L = 8), reducing the dimensionality of thetexture space is by 98% and smoothing the texture. To use

RGB color, the three color channels λ = {R,G,B} aremodeled as three separate SH.

Since rotation of the SH texture is simplified accordingto (1), finding the most likely orientation φt given the SHtexture model from the previous timestep Tt−1 and thecurrent texture measurement Tt is straightforward and canbe solved by minimizing the SSE on line 15 of alg. 1 for φtusing a quasi-Newton approach like BFGS [16]. A Kalmanfilter update of φ−t using φt gives the final orientation φt.

The time-based texture model Tt is learned by exponentialdecay using learning rate αT according to the equation online 20 of alg. 1. However, to prevent overrepresentationof the first sampled textures in Tt, iterative averaging isused to learn Tt as long as t < 1

αT, as shown on line 18

of alg. 1. Combining models and measurements over timerequires each measurement to be rotated into a canonicalorientation, matching the model. By sampling the textureusing the rotated shape model Sφt−1, Tt is in a canonicalorientation and can directly be combined with Tt−1. Anexample of a sampled texture, its SH reconstruction and alearned texture after 140 frames can be found in fig. 2(f).

B. 3D Shape Model

Using orientation φt, the SH shape model St can becomputed. First, the SH shape measurement St at time t isconstructed by transforming visual hull Ht into a sphericalfunction fS and projecting this function onto the SH space.To reduce feature dimensionality and simultaneously smooththe shape representation, the first 17 bands of the SHspace (L = 16) are used for projection, reducing featuredimensionality by 90%. Examples of a person’s visual hull,the spherical function derived from that visual hull andthe reconstruction of the SH representation of the sphericalfunction can be seen in fig. 2(a)-(c). The top-down viewsshowing the person facing to the right clearly show the excessvolume created by volume carving due to shape ambiguities.

Estimating the rigid body over time is done by averagingshape estimates over time, decreasing the influence of non-rigid body parts on the estimated body shape. Since armsand legs will have different positions over time, they willbe averaged out in the final shape estimate as can be seenin fig. 2(d). To combine shape models and measurements,they have to be in canonical orientation. Since the shape isrepresented in SH components, rotation is straightforward (asmentioned in sec. III) and does not yield rotation artifactswhich would occur when rotating objects in the discretevolume space consisting of cubic voxels in a regular grid.The rotated SH shape measurement Sφt is computed as shownin alg. 1 on line 27, in accordance with (1).

The rigid body shape model St at time t is computedover time by iteratively combining Sφt for all timestepsunder exponential decay, as shown in alg. 1 on line 28. Theshape learning rate, αS , is split up in two parts: one forgrowing the model and one for shrinking it. Because of thenature of volume carving, it is much more likely that carvingartifacts consist of extrusions of the actual shape instead ofindentations. To handle artifacts efficiently, the learning rate

(a) (b) (c) (d) (e) (f)

Fig. 2. Examples taken from the person shown in fig. 3(c). (a)-(d) show a top-down view of the person facing to the right (top) as well as a cameraperspective view (bottom). (a) Person visual hull H. (b) Spherical shape function using 55× 55 surface points. (c) 17 band SH shape representation. (d)Learned shape model after 140 frames, mapped with sampled texture. (e) Person modeled. (f) top-to-bottom: sampled texture (55× 55 pixels), 9 band SHtexture representation, learned SH texture after 140 frames.

for surface points that are more indented in Sφt than in St−1

is set to 0.4. The learning rate for more extruded surfacepoints is set to 0.01. An example of a learned shape modelcan be found in fig. 2(d), showing that the model learned agood approximation of the actual person’s volume.

C. Iterative Orientation Estimation

The estimate of the current body orientation made usingthe equation on line 15 of alg. 1 allows for an iterativeorientation optimization scheme. Since the texture measure-ment Tt is sampled using the reconstruction of St−1, orientedaccording to the Kalman filter prediction φ−t instead of thefinal estimate φt, the sampled texture might be distorted.

In order to optimize the estimation of the person orienta-tion, texture sampling and orientation estimation are repeatedmultiple times. Each time, the reconstruction of St−1 isrotated using the most likely orientation estimate φt of theprevious iteration. While the estimate gets closer to the trueorientation, the SSE gets smaller and the difference betweenorientation estimates reduces. Optimization is stopped whenthe difference between consecutive orientation estimates isless than 0.001 rad or 10 iterations have been done.St−1 could be updated every iteration using Sφt , rotated

using the previous iteration’s estimate of φt. However, whilethe orientation estimate is suboptimal, an incorrectly rotatedversion Sφt might result in a more distorted shape estimateinstead of an improved estimate.

V. EXPERIMENTS

We evaluated how the SH texture representation, shape in-formation and the iterative estimation procedure influence thequality of the estimated orientation. To this end, our methodwas compared to two state of the art approaches. The firstmethod is based on the Panoramic Appearance Map (PAM)[5]. A fixed size cylinder with a diameter of 35 cm and aheight of 2 m, fitting tightly around a person’s torso is usedfor sampling the person’s texture. Orientation is estimated ina generate-and-test fashion by sampling 360 textures using

1◦ orientation difference and comparing all textures to alearned texture model. The orientation of the best matchingtexture sample is used as the object orientation. Alternatively,we combined the cylinder based shape model with our SHbased texture representation and used the iterative orientationestimation from the previous section.

The second state of the art method was the classifierbased orientation estimation method introduced by [2]. Thismethod is complementary to our method, using a fundamen-tally different way of orientation estimation. The classifiercombines HOG-based features to get an absolute orientationestimate per frame. It uses a mixture of four orientationexperts, each trained on images showing pedestrians in one offour canonical orientations (front, back, left, right). The ex-perts’ classification results are used as weights in a GaussianMixture Model, creating a full orientation probability densityfunction (pdf ). A maximum likelihood estimate derived fromthis pdf is used as the final orientation estimate. The expertswere trained using data kindly provided by the authors of [2].We generated regions of interest in the 2D images based onthe projection of the volume space onto the camera plane.Orientation estimates per camera are combined into a 3Dorientation estimate using the camera calibration.

The following settings are used for the experiments. Tex-tures are sampled in standard RGB colorspace. Preliminaryexperiments were done using both normalized RGB and C-invariant color spaces [18], but overall best results weregained using standard RGB. The voxelspace has a resolutionof 2 × 2 × 2 cm/voxel. Textures and shapes are sampled ata resolution of 55× 55 surface points and a texture learningrate αT = 0.05 is used. For the methods using SH basedtextures 9 SH bands are used for texture representation. Forthe SH shape representation 17 SH bands are used. For allmethods, each frame’s orientation estimate was constrainedto be within 30◦ of the previous frame’s estimate. Applyingthis constraint for the classifier based approach was done bylimiting the range of the pdf when computing the maximum

(a) (b) (c) (d) (e) (f) (g) (h) (i)

(j)

Fig. 3. (a)-(i) The 9 people used in the 12 scenarios. (j) Samples of scenarios, showing the 3 camera viewpoints and 3 scenarios.

likelihood orientation. Since the classifier based approachis time independent by nature, we also tested this methodwithout applying the 30◦ orientation constrained.

All methods were evaluated on about 2800 frames, dis-tributed over 12 scenarios recorded in an open, outdoorsenvironment with uncontrollable illumination. Three cameraswith fully overlapping views are used recording at 20 Hz andat a resolution of 752 × 560 pixels. While some scenariosfeature multiple people, orientation estimation was onlyperformed for one person. The 9 different people involved inthe scenes can be found in fig. 3(a)-(i), while fig. 3(j) showssample frames from 3 scenarios and all camera viewpoints.

Ground-truth (GT) was created by labeling 17 distinctivelocations (joint and head positions) on each person’s bodyin each camera and using these to compute the pose of a 3Dperson model. The annotated torso orientation was taken tobe the person body facing direction. Doing this for a regularinterval of frames results in a positional accuracy of about 4cm. We use GT foreground segmentations created using theannotated 3D pose to eliminate the effect of segmentationartifacts and to solve the problem of selecting the object ofinterest when multiple objects are available in the scene.

Fig. 4(a) shows the moving RMS error at time t: the RMSerror over frames 1 to t of all scenarios combined. Since notall sequences have the same length, vertical bars indicatethe points where scenarios end and the RMS error is takenover fewer scenarios. Please note that since the first threemethods all provide relative orientation estimates w.r.t. thefirst frame, the error is 0 at this point. Because the classifierbased approach gives an absolute orientation estimate foreach frame, its error is not 0 for the first frame. Furthermore,since the moving RMS error for the first few frames has avery small sample size, the lower range of classifier basedresults is not representative. Comparing the classifier basedapproach to the other methods is best done at the later frames.

Measured over all scenarios, our method consistently out-performs both other relative orientation estimation methods.In the first 40 frames the error rises quickly because of sub-optimal shape and texture models. A comparison with themethod using the cylinder together with SH texture shows

50 100 150 200 250 300 350 4000

10

20

30

40

50

60

70

frame

RM

S e

rro

r (d

eg

)

SH shape and texture

PAM

Cylinder shape, SH texture

Classifier based approach

Classifier based approach (constr)

(a) Moving RMS error evolution over time

0 20 40 60 80 100 120 140 160 1800

0.2

0.4

0.6

0.8

1

Absolute orientation error (deg)

Fra

ction o

f fr

am

es

SH shape and texture

PAM

Cylinder shape, SH texture

Classifier based approach

Classifier based approach (constr)

(b) Cumulative distribution of absolute orientation error

Fig. 4. (a) Moving RMS error over all scenarios, vertical lines markscenario end frames. (b) Cumulative absolute error distributions over allscenarios.

that our method benefits most from modeling the shape ofthe person. Representing the texture in low-dimensional SHspace gives a comparable performance to PAM’s full textureapproach, especially when considering that the 9 SH bandsreduce the number of features by 98%. However, when theperson being tracked has a low-contrast appearance like theone shown in fig. 3(e), the SH texture lacks detail.

Constraining the orientation difference per frame for theclassifier based method gives a significant performance im-provement. While the unconstrained method is outperformedby all methods, the constrained version outperforms bothPAM and the cylinder with SH texture. However, our method

20 40 60 80 100 120 140 160 180−300

−250

−200

−150

−100

−50

0

frame

orie

nta

tio

n (

de

g)

Orientation estimation scenario 8

Ground truth

Full SH

PAM

Cylinder shape, SH texture

Classifier based approach (constrained)

Fig. 5. (top) Example of the estimated orientation of the person fromfig. 3(c) using the four methods. (bottom) Frames 48, 98 and 148 form thisscenario. Notice some occlusion has occurred between frames 98 and 148.

still shows a lower RMS error compared to the constrainedclassifier when measured over all frames. While the qualityof its orientation estimates are independent of time, theconstrained classifier’s performance seems to be slightlybetter between frame 100 and 175. This can be explainedby some shorter scenarios featuring low contrast people (e.g.fig. 3(b) and (e)) whose orientation can be estimated betterwith the less texture-sensitive classifier based approach.

Fig. 4(b) shows each method’s cumulative error distri-bution, obtained by binning the errors over all frames.Our method outperforms all other methods with respect tothe error distribution. Most noticeable is the gap in thefraction of frames between 40◦ and 80◦ orientation error,compared to the PAM and cylinder shape method, meaningthat our method has significantly less errors > 80◦. Theunconstrained classifier approach shows more errors around180◦, matching the results from fig. 6 of [2]. These errors arecaused by the ambiguity in person shape when viewed fromthe front or the back. The constrained classifier based methodhas less large errors than the other alternative methods butis still outperformed by our method.

Fig. 5 shows an example of the orientation estimationperformance on a frame-to-frame basis in one scenario. Inorder not to clutter the graph too much, we only showthe constrained classifier results here, since it gave thebest overall RMS score in the previous experiments. Ourmethod is shown to follow the GT orientation closely, whileboth PAM and the cylinder with SH texture method driftaway from the GT. This drift becomes larger at frame 110,around which point in the scenario the tracked person isoccluded in one of the cameras. Our method shows morerobustness to this occlusion. The classifier based approachdoes a reasonable job following the GT, but shows a bit moreerratic behavior, due to its frame-wise estimation approach.

All experiments were done on a single core of a 2.6 GhzIntel Xeon CPU. Average processing times were: 9.5 s/fr forour method, 1 s/fr for PAM, 1.7 s/fr for the cylinder with SH

texture and 0.7 s/fr for the classifier based approach. Volumecarving and computation of the spherical shape functiontook about 8 s of the 9.5 s our method needs per frame,using a crude implementation. A large speed improvementis possible by using a GPU implementation. The classifierbased approach was implemented in C++, while the othermethods partly contain unoptimized Matlab code.

VI. CONCLUSION

We presented a novel approach for estimating the relativeperson orientation, based on a low dimensional shape andtexture representation using Spherical Harmonics. Orienta-tion estimation experiments show that this approach outper-forms several alternative methods. In future work, resultscould be improved by using more advanced inference meth-ods like particle filters instead of the maximum likelihoodapproach for selecting the position and orientation used here.Furthermore, the fusion of our relative orientation estimateswith classifier based absolute orientation estimates mightfurther improve performance.

REFERENCES

[1] T. Gandhi and M. M. Trivedi, “Image based estimation of pedestrianorientation for improving path prediction,” in Proc. of the IEEE IV,2008, pp. 506–511.

[2] M. Enzweiler and D. M. Gavrila, “Integrated pedestrian classificationand orientation estimation,” in Proc. of the IEEE CVPR, 2010, pp.982–989.

[3] H. Shimizu and T. Poggio, “Direction estimation of pedestrian frommultiple still images,” in Proc. of the IEEE IV, 2004, pp. 596–600.

[4] C. Chen and J. Odobez, “We are not contortionists: Coupled adaptivelearning for head and body orientation estimation in surveillance,” inProc. of the IEEE CVPR, 2012, pp. 1544–1551.

[5] T. Gandhi and M. M. Trivedi, “Person tracking and reidentification:Introducing panoramic appearance map (PAM) for feature representa-tion,” MVA, vol. 18, no. 3, pp. 207–220, 2007.

[6] H. Ben Shitrit, J. Berclaz, F. Fleuret, and P. Fua, “Tracking multiplepeople under global appearance constraints,” in Proc. of the IEEEICCV, 2011, pp. 137–144.

[7] M. Liem and D. M. Gavrila, “Multi-person localization and trackassignment in overlapping camera views,” in Proc. of the DAGM, 2011,pp. 173–183.

[8] A. Mittal and L. S. Davis, “M 2 tracker: a multi-view approach tosegmenting and tracking people in a cluttered scene,” IJCV, vol. 51,no. 3, pp. 189–203, 2003.

[9] A. Balan et al., “Detailed human shape and pose from images,” inProc. of the IEEE CVPR, 2007, pp. 1–8.

[10] J. Gall et al., “Motion capture using joint skeleton tracking and surfaceestimation,” in Proc. of the IEEE CVPR, 2009, pp. 1746–1753.

[11] M. Hofmann and D. M. Gavrila, “3D human model adaptation byframe selection and Shape-Texture optimization,” CVIU, 2011.

[12] Z. Yue, W. Zhao, and R. Chellappa, “Pose-encoded spherical harmon-ics for face recognition and synthesis using a single image,” EURASIP,vol. 2008, pp. 65:1–65:18, Jan. 2008.

[13] B. Bustos et al., “Feature-based similarity search in 3D objectdatabases,” ACM Computing Surveys, vol. 37, pp. 345–387, Dec. 2005.

[14] K. Liu et al., “3D Rotation-Invariant description from tensor operationon spherical HOG field,” in Proc. of the BMVC, 2011.

[15] R. Green, “Spherical harmonic lighting: The gritty details,” in GameDevelopers Conference, 2003.

[16] A. Makadia and K. Daniilidis, “Direct 3D-rotation estimation fromspherical images via a generalized shift theorem,” in Proc. of the IEEECVPR, vol. 2, 2003, pp. II– 217–24.

[17] D. Saupe and D. V. Vranic, “3D model retrieval with sphericalharmonics and moments,” in Proc. of the DAGM, 2001, pp. 392–397.

[18] J. M. Geusebroek, R. van den Boomgaard, A. W. M. Smeulders, andH. Geerts, “Color invariance,” IEEE Trans. on PAMI, vol. 23, no. 12,pp. 1338–1350, Dec. 2001.