The perception and content of cast shadows:an interdisciplinary review

Hannah M. Dee∗and Paulo E. Santos†

November 11, 2009

AbstractRecently, cognitive psychologists have turned their attention to the

formerly neglected study of cast shadows and the information theypurvey. These studies show that the human perceptual system val-ues information from shadows very highly in the perception of certainqualities, sometimes even to the detriment of other cues. From shad-ows, it is possible to ascertain characteristics of the light source, thecasting object, and the screen that the shadow is cast upon, and soshadows can provide a rich source of additional perceptual informa-tion. This can hold even if the light source and the casting objectare out of the field of view. Indeed, in some cases, shadows are theonly source of perceptual information about the presence, shape andmotion of an object. However with a few notable and very recentexceptions, computer vision systems have treated cast shadows notas signal but as noise. The aim of this paper is to provide a conciseyet comprehensive review of the literature on cast shadow perceptionfrom across the cognitive sciences, elucidating both the theoretical in-formation available from cast shadows, the perception of shadows inhuman and machine vision, and the ways in which the human percep-tual system makes use of information from shadows. In this way wehope to highlight the way in which the human visual system exploitsthis rich source of information and the computer vision tools currentlyavailable for detection and inference using shadows.

∗Hannah Dee is with GIPSA Lab, Institut National Polytechnique de Grenoble, 961Rue de la Houille Blanche, BP 46, 38402 Saint Martin d’Heres, France.†Paulo Santos is with FEI, Av. Humberto de Alencar, Sao Bernardo do Campo-SP,

Brazil

1

1 Introduction

Cast shadows are caused when a caster comes between a light source anda surface or screen. The information content in these types of shadows cantherefore be used to provide knowledge about any or all of these three ele-ments. As a very elementary example, if we assume that the light does notmove very fast and that the screen is flat and horizontal, we can draw conclu-sions about the size, motion and shape of casting objects by looking at theirshadows. In this way, shadows were used as powerful tools in early astro-nomical research for the determination of solstices and equinoxes, to providean approximation of the distances from the Earth of the sun and moon, andto estimate the size of celestial bodies. Keeping caster and screen constant,the motion of a light source has been used for thousands of years to measuretime. Keeping the light source and screen constant, the use of shadows toinform about moving objects out of sight has been known for millennia –Plato’s allegory of the cave concerns just this situation [54]. In Galilean-eraobservations of the sky, shadows (or, more precisely, eclipses) were used toshow that the moon and the known planets were of the same nature as theEarth and that light has a finite speed and spreads by diffraction (as well asrefraction and reflection). In the 20th century, shadows were used to verifythe relativistic predictions of the deviation of light in the presence of massand to suggest the hypothesis that the speed of the earth’s rotation is slowingdown [9].

In this paper we shall concentrate on the information content of castshadows rather than self shading (where an object casts a shadow upon it-self), and for the sake of brevity we shall refer to cast shadows as simply“shadows”. These shadows are largely used by the human perceptual systemto draw conclusions about everyday scenes, and as we shall see later in thispaper, some of these conclusions suggest that information from shadows canoverride conflicting depth cues present in the visual world [36]. This im-plies that our perception of space is biased towards using information fromshadows in certain situations. In spite of this, computer vision systems havelargely placed shadows in the position of noise to be filtered out1. In thispaper we contrast the information available from shadows with computer vi-

1There is a large sub-field of vision research that deals with shape-from-shading wherebyan object’s self-shadowing is used to determine its shape [38]. This line of research,however, does not take into account cast shadows. For more detail on shape from shading,see the recent review paper [21].

2

sion methods for shadow filtering and shadow segmentation in order to makeexplicit the gap between what human perception deems as important to ex-tract when constructing a spatial representation from a visual scene, andwhat current autonomous computer vision systems are designed to extract.

In order for the information content in shadows be used as knowledge wenote two difficult problems that a perceptual system has to solve first, whichin turn give rise to a number of interesting questions that intersect cognitivescience and computer vision research. The first problem is how shadows canbe detected in the first place – some shadows have clear outlines and seemvery “solid”, yet we do not tend to misperceive shadows as objects. Othershadows have vague borders and therefore should be harder to perceive, buthumans do not have any difficulty in doing so. Secondly, there has to be aconsideration of the Shadow Correspondence Problem: given perceived ob-jects and perceived shadows in one scene, how can shadows be unambiguouslyanchored to their casters?

Once the detection and the shadow correspondence problems are solved,we have a series of high-level reasoning tasks to consider. Section 2 describesthe various things we can learn from the investigation of shadows, drawingon optics and geometry but also upon the way in which shadows are usedin painting and computer graphics to convey various aspects of the visualscene. Section 3 moves on from the theoretical possibilities of shadow per-ception to consider evidence from the fields of psychology and neuroscienceon the ways in which humans perceive shadows and the ways in which weactually use them – the hows, whats and whys of human shadow perception.Section 4 considers the detection and use of shadows in computer vision,artificial intelligence and robotics and finally Section 5 brings together thevarious interdisciplinary threads, draws some conclusions and provides point-ers to open research questions.

2 The information content in shadows

Assuming that an environment has one strong light source (the primarylight) and any other light sources are weak or diffuse (secondary light) theanatomy of a shadow cast upon a uniform screen is fairly simple: it consistsof a main part (which we will call the shadow body), and a less dark fringe(the penumbra). The perceived darkness and any perceived colour of theshadow depends upon the colour of the screen, the intensity of the primary

3

Figure 1: The anatomy of a shadow. The shadowed area is totally occludedfrom the light source by the caster, and the penumbra is partially occluded(that is, from the penumbra it is possible to see some part of the light source).Black lines indicate lines of sight. With a point light source, there is nopenumbra.

light source and the intensity and colour of any secondary (or “ambient”)illumination. The width of the penumbra depends upon the size of the pri-mary light source, and the distance of the caster to the screen. A diagramof the shadow formation process is given in Figure 1. The situation becomesmore complicated in the presence of multiple strong light sources, but similarprinciples apply.

In real-world scenes a detailed model of shadow formation needs to takeinto account a number of different factors, related to the caster, light sourceand screen:

• Caster information:

– The shape and size of the caster determine size and shape ofshadow;

– The position (and pose) of the caster, particularly with respect tothe light source, affects the shape, size and location of the shadow;

– Opaque objects cast solid shadows, but translucent objects castcoloured or weak shadows.

• Light information:

– The shape and size of the light source determine characteristics ofthe penumbra;

4

– The position of the source (along with the position of the caster)determines location of the shadow;

– Light source intensity determines the contrast between shaded andnon-shaded areas;

– The intensity of any ambient illumination also affects contrast;

– The colour of ambient illumination determines the colour of theshadow.

• Screen information:

– Screen orientation with regard to light source determines the de-gree of distortion in shadow shape;

– The shape and location of background clutter can cause shadowsto split, distort, or merge.

(a) (b)

Figure 2: In some situations shadows carry information about objects outsideof view, via the “viewpoint” of the light source. In a), we can concludethat there is someone out of the scene behind the observer, and in b) wecan conclude that there is an object “hiding” behind the pot. Photo 2(a)shows the artwork Shadow Stone, by Andy Goldsworthy, a work of art whichencourages viewers to play with shadows.

5

By making assumptions about or keeping constant some of these factors,shadows can be used to determine various aspects of the visual scene. Casatiin [6] overviews the information encoded in shadows, which is not necessarilyexploited by our perceptual system. For instance, it is possible to hypothesiseon the presence and location of the light source, the caster can be hypothe-sised (whether they are inside or outside the observer’s field of view or not),the distance between the caster and the screen, and whether or not the casterand the shadow are in contact with each other. The observation of a shadowin a scene, but not its caster, indicates the presence of objects outside thevisual field (or occluded objects). Shadows indicate the direction in whichthe light source can be found, and intensity of the source (or the relative in-tensity of multiple sources). The width of the penumbra informs the angularsize of the source, and the distortion of the shadow outline (with respect tothe shape of the caster) indicates the texture of the screen. Shadow motioncarries information about the 3D structure of a caster, about the caster’smotion in depth or about the geometry of the screen. Another importantfact about the information content of shadows is that they can be seen asproviding the observer with a second viewpoint : that of the light source, asthe shadow depicts the projection of the caster’s terminator line.

A glimpse of how the human perceptual system uses the information of(static) cast shadows can be obtained from the analysis of artistic depictionsof the natural world. As Conway and Livingstone point out [15], in order totranslate a convincing impression of the external world, artists explore rulesof perspective, of colour perception or visual illusions.

Some of the information found in cast shadows was intensely exploredby Renaissance painters, mainly in order to depict the position of importantobjects in scenes or to represent relative depth [8,9,19]. Indeed Leonardo daVinci himself carried out many observations into the way in which shadowsare cast (for example, explaining why shadows cast by the sun on a whitewall tend to look blue) [25] and was also probably the first to relate theappearance of shadows with occlusion, when he says “no luminous body eversees the shadows that it generates” [20].

In particular, it was through the investigation of how the 3D world couldbe depicted in 2D paintings that projective geometry came to be developedin the 15th century, although it is unclear whether the observation of shadowsas projections played a central role in the development of this discipline [19].

It is worth mentioning the near complete absence of any representation

6

of cast shadows in pre-Renaissance art or in non-Western cultures2, andit is also worth mentioning that a great number of shadow depictions arephysically impossible [5,7,9] showing that once adopted by painters, the useof shadows was far from straightforward. This neglect of shadows in artisticrepresentations of the world could be explained by the inherent difficulty ofdepicting the right characteristics of luminosity (and imprecise borders) tomake dark patches on canvas be perceived as shadows [9], or it may be dueto the fact that the human perceptual system is simply insensitive to someof the information provided by static cast shadows [32].

Jacobson and Werner [32] describe a visual search experiment where hu-man viewers had to determine impossible shadows in distinct scenes witha number of casters and shadows. The results of this experiment indicatethat the subjects were insensitive to inconsistencies in cast shadows, fromwhich the authors concluded that the inclusion of cast shadows is not criti-cal to the understanding of pictorial art. Cavanagh [13] suggests that thosetransgressions of standard physics in visual art that pass unnoticed by theviewers’ understanding (such as inconsistent shadows) indicate that our per-ceptual system uses a simplified physics to interpret the world. This simpli-fied physics facilitates an efficient assessment of the visual world. Taking adifferent view, Casati argues that impossible shadows, often drawn as replicasof objects, are better cues for the localisation of casters in scene depictionsthan a more realistic shadow [8]. This observation seems to contrast with Ca-vanagh’s hypothesis of simplified physics, as the visual processing of replicasof objects corresponds to a more complex visual situation than that found ineveryday life.

Closely related to the painters’ need to depict shadows, computer graphicsis also interested in the rendering of the spatio-temporal structure of scenes,and therefore it has considered the determination and rendering of cast shad-ows in great depth. The first survey about shadow algorithms for computergraphics was presented in [16], which describes a classification of the earlymethods. A more up-to-date survey is presented in [72], where shadow algo-rithms are classified by the type of shadows they produce: hard shadows, softshadows, shadows of transparent objects and shadows for complex modelingprimitives. In general, the large majority of shadow algorithms are based on

2Although a very worthwhile mention here has to go to Chinese shadow puppetry.Whilst this art, strictly speaking, is concerned with using and not depicting shadows, ithas been around for millennia.

7

the following methods (as cited in [72]): area subdivision, ray tracing, radios-ity, shadow volumes and shadow maps (or z-buffers). [28] presents a surveyof real-time soft-shadow algorithms which is a comprehensive discussion ofalgorithms derived from the previous two methods cited.

Rendering soft shadows realistically is a hard problem, and none of themodern algorithms cope with all the difficulties involved in this task [28].Instead of trying to produce realistic shadows, [63] evaluates the level ofdetail required to produce shadows that are sufficiently detailed to be ac-ceptable by the human perceptual system. The final aim of this research isto use simplified models of scene objects to reduce the complexity of shadowrendering.

The next section presents empirical investigations into the human per-ception of shadows.

3 The psychology and neuroscience of cast

shadow perception

The psychological work reported in [36,44,51] discusses experimental resultswhich suggest that the human perceptual system prefers cues provided byshadows over other information in order to infer 3D motion of objects. Sur-prisingly, shadows are trusted more than changes in apparent object size. Inone experiment discussed in [36], a number of human subjects were presentedwith a computer simulation in which the shadow of a static square (cast ona chequered screen) moves away from its caster. Most subjects reported per-ceiving the square moving towards and away from the background accordingto the shadow motion, even though the size of the square remained unchangedthroughout the experiment (this was clear from the static chequered back-ground). It is worth pointing out that, geometrically, there are a number ofpossible competing hypotheses for the shadow motion that would be morecoherent than object motion as an explanation in this case (e.g. the motionof the light source). However, subjects even reported having the illusion ofthe objects changing in size according to the shadow’s motion. See Figure 3for a visual example of this effect.

Further situations were explored in [36], to verify the effect of shadowperception on the perception of motion in depth using a new scenario: thesubjects were shown two distinct animations of a ball moving inside of a box.

8

Figure 3: Still image version of Kersten and Mamassian experiment [36], inwhich the shadow changes but we perceive the square as moving.

In the first animation, the ball was made to move along a diagonal insidethe box, whilst the ball’s shadow described an horizontal trajectory in theimage. In the second situation, the ball’s trajectory was the same, but theball’s shadow moved in such a way that it was aways connected to its caster.Even though the ball’s trajectory was identical in both situations, and therewas no change in the size of any objects in the scene during motion, allobservers interpreted the ball as rising above the floor in the situation wherethe shadow motion was horizontal, but as receding in depth in the other.These findings (summarised in [44]) suggest that, in some cases at least,the human perceptual system is biased to use shadow information for theinterpretation of 3D motion and that shadow information can even overridenotions of conservation of object size.

As well as providing a strong cue about motion in depth, cast shadowsprovide information that could be used in the interpretation of surface shapeof the screen, however experimental findings suggest that this information isnot used by the human perceptual system [37,44].

Psychological studies investigating the relationship between shadow per-ception and object recognition tell a less clear story. Braje et al. [3] reportresults from three experiments involving the recognition of natural objectswith shadows in several experimental conditions, and suggest that humanobject recognition is not affected by the presence of shadows. The authorsconclude that the results are consistent with a feature-based representation ofobjects, where shadows may be filtered out as noise. However, it may also bethe case that the results obtained are dependent on the type of stimuli used in

9

the experiments (simple familiar objects), which contained much redundantinformation that could reduce the importance of the information providedby object’s shadows. Castiello in [11] reports an experiment with contrast-ing results, in which the perception of objects is hindered when presentedwith incongruent cast shadows (wrong shadow) or incongruent lighting withrespect to the shadows (shadow on the wrong side with respect to the lightsource). There are two competing explanations for these findings: eitherthe perception of shadows is used to improve object recognition in certainsituations, or incongruent shadows work as distractors in the scene.

The relation between the perception of shadows and the determination ofoptical contact3 is also subject of psychological research [37,51]. In particular,[51] investigates the difference in depth perceptions of a floating object withrelation with an object on the ground following it “like a shadow”. Theauthors want to address three fundamental questions. What are the featuresthat make a shadow be perceived as such? What is the effect of objectseparation in the perception of depth from shadows? In situations withmultiple shadows, what are the features that make us associate one particularshadow with an object? They investigate these questions by varying thelight intensity of the lower object, its thickness and its motion relative tothe casting object. Perhaps unsurprisingly, darker objects are more readilyperceived as shadows. Common motion of object and shadow is an importantfeature for shadow association and, in situations with multiple shadows, theauthors suggest that common speeds decide which shadow is associated withan object (relative size being a secondary concern).

Psychological research also suggests that our perceptual system uses castshadows as a coarse cue: it does not matter if the shadow is the wrongshape for the casting object, it just has to be associated with the casterand telling a coherent story about the object motion or location. Bonfiglioliet al. in [2] carried out a naturalistic study using real objects with fakeshadows, and discovered that shadows do not affect our verbal reports ofwhat is going on, but can affect the way we reach for an object - shadowswhich are the “wrong” shape affect our physical behaviour but not our verbalreaction times. Ostrovsky et al [52] , investigating shadows which arise frominconsistent illumination present results which contradict earlier studies (i.e.[23]). These studies involve the presentation of an array of identical objects

3Optical contact is the place where an object is connected to the background in a 2Dprojection of a 3D scene.

10

with consistent shadows and shading, but one drawn as though it is lit from adifferent direction. Whilst the earlier studies suggested that the illuminationchange was easy to detect (indeed, it popped out) the later study shows thiswas an artifact of the regularity of the array. When objects are arranged ina more random fashion, the shadow difference is harder to perceive (as longas the shadow is plausible).

Rensink and Cavanagh in [59] present compelling evidence for the hy-potheses that shadows are processed early in the visual pathway and thendiscarded, and that we use an assumption of a single overhead light source indoing this. Using a visual search methodology, they show that the detectionof shadow-like shapes consistent with an overhead light source takes longerthan the detection of the exact same shape in other situations. If the shape isaltered so it is not shadow-like (it’s lighter, or has the wrong texture, or thewrong edge features to be a shadow), or the shape is shadow-like but is con-sistent with illumination from below, visual search is much quicker. Casaticomes to the same conclusion in [7] through the observation that dark patchesin paintings, sometimes bearing no resemblance to real shadows, suffice toenhance the perception of depth. The human perceptual system seems to ex-tract position information from shadows early on in processing, then filtersthem out in order to avoid interpreting shadows as objects in further spatialinferences.

A related question considered in [12] is that of whether shadow processingis implicit (i.e. without conscious awareness) or not. Through studying castshadow perception in groups of people with brain injuries, Castiello et al. tryto localise cast shadow processing in the brain and to determine whether con-scious awareness is necessary. They show that the performance in a simpleobject recognition task is hindered if the shadow is missing or incongruent(does not match the object). This effect exists even in brain-injured pa-tients suffering from visual neglect, who are not aware of the existence ofthe shadow. These findings suggest that our ability to process and deal withcast shadows is not dependent upon our conscious awareness of them and,therefore, is an implicit process. Furthermore, the authors test the hypoth-esis that shadow processing in the human brain is located in the temporallobe (following some previous evidence that an analogous process occurs inmonkeys’ temporal areas). For that, a number of patients suffering fromleft visual field neglect caused by lesions in the temporal lobe are subjectto the same object recognition task with incongruent shadows as patientswith frontal lobe lesions. In this case, the temporal lobe patients had lower

11

reaction times when presented with shadows the left of an object, providingsome support for the hypothesis.

Whether shadow processing is implicit or explicit, there is evidence thatshadows cast by a person’s own body parts are used more effectively injudgements about extrapersonal space than shadows from other objects car-rying analogous information. This has been observed by Pavani and Castielloin [53], where the judgement of distances from shadows of the subject’s ownhand diverged from similar judgements when the subjects were wearing apolygonal glove. Following a similar experimental setup, Galfano and Pavanifind support for the hypothesis that body-shadows act as cues for atten-tion [27].

To summarise the psychological and neurological evidence, it appearsthat our visual systems use shadows in early vision4 as coarse indicators ofdepth and 3D position in space. The perception of shadows seems to relyon the assumption that there is a single overhead stationary light source[35,43,44], but the human perceptual system probably does not rely on castshadows to determine shape, and nor does it seem to use shadows as a sourceof information about the screen upon which they are cast. It seems thatthe human perceptual system makes assumptions about the appearance ofshadows, and these early vision processes only perceive a stimulus as shadowif it is a fairly homogeneous region, darker than the background, withoutinternal edge features. Despite these constraints upon what we consider tobe shadow, we are still able to handle major variations in the appearanceof shadows, perceiving as shadows those stimuli arising from inconsistentillumination or shape [22], or even thick dark patches in scenes [51].

The use of the information content of cast shadows, however, presupposesthe solution of the shadow correspondence problem [8,42], which involves thesegmentation of shadows in scenes and the connection of shadows to theirrelative casters [44]. Shadows, like holes, are dependent objects – without acaster, they do not occur. Matching shadows to their casters is a hard prob-lem for various reasons: there may be various competing possible matchesbetween shadows and objects in a complex scene (i.e. the shadow correspon-dence problem is underconstrained); the screen may not be planar, whichmay turn a point-to-point matching into a complex non-linear registration

4By “early vision” we adopt the definition used in [59] and mean a process carriedout in the first few hundred milliseconds of processing and not involving stimulus-specificknowledge

12

procedure; and shadows of close objects may merge. In agreement with thehypothesis of early-visual processing for shadows, psychological experimentssuggest that the human visual system utilises an initial coarse matching be-tween shadows and their casters that would allow for a re-evaluation givenevidences from high-level reasoning procedures [42]. However, this seems toconstrast with cases investigated in [8] where some evidence is presented tosupport that the perceptual system solves the shadow correspondence prob-lem even when the shadows depicted represent a more complicated situationthan naturally observed.

The next section overviews the main algorithms of computer vision forshadow detection, giving more emphasis to those that use shadows as infor-mation, instead of filtering them as noise. We also discuss some literature inartificial intelligence and robotics in which cast shadows are considered.

4 Cast shadows in computer vision, artificial

intelligence and robotics

Much shadow detection work in computer vision is centred around the ideaof shadow as noise. Two broad approaches are affected by shadows: the firstdeals mainly with single images and is associated with the segmentation ofimages into the objects that they depict; and the second deals with videoand is concerned with the identification of moving objects. Shadows areproblematic in both cases – they cause spurious segmentations in the firstinstance, and spurious foreground objects in the second. Perhaps the simplestshadow detection method proposed is that of [66], in which a grey-scaleimage is simply thresholded and the darker pixels are labelled “shadow”. Inthe archaeological images the authors deal with, this works reasonably well;however, for more complex images more sophisticated algorithms are calledfor.

Those algorithms dealing with single images use colour and texture infor-mation to group image pixels into regions that correspond to single elementsin the real world (such as grass, or trees). This can be seen as an exercise incolour constancy ; the aim is to determine the colour of the underlying objectin various light conditions, and in this context shadows are merely one ofthese light conditions rather than an object of study in themselves. The ex-istence of strong shadows can cause spurious segmentations, and so shadow

13

detection is performed in order to classify shaded pixels as part of the screen,rather than as shadow. An example of this sort of work is that of Vasquez etal [67] who engage not so much in shadow detection as in shadow blindness.The aim is a segmentation in which image components are classified regard-less of self shading and inter-shading; this is achieved by identifying “ridges”in colourspace which are characteristic of a particular dominant colour underdiffering lighting conditions. Whilst these ridges could conceivably be usedas part of a shadow detection algorithm, this is not part of their currentwork. We propose that shadow removal algorithms such as those introducedby Finlayson and colleagues [24] fall in a similar category – they are con-cerned with shadow blindness, and only work on individual images (and areoften too slow to be considered useful for video processing).

The second major consideration of shadows within computer vision comeswhen detecting moving objects. This is commonly done by subtracting“background” from video to find objects of interest, where background isdetected by finding those pixels or image regions which do not change muchin colour. In doing this, shadows become a major source of false positives asa cast shadow will make an otherwise uninteresting pixel change colour.

Thus in computer vision, shadow detection almost always involves somemodel of the colour of the screen, or background, and then detection is per-formed using a model of shadows characterising them as “roughly the samecolour as background, but darker”. Prati in [56] provides an overview anda taxonomy of shadow detection techniques, dividing them into model-basedand non-model-based and then further into parametric and non-parametrictechniques. This categorisation does not apply so well to more recent works,many of which can be thought of as “ensemble methods”. Thus we makea different distinction, between methods which detect shadows based uponcolour information alone, and those which incorporate some form of spatialinformation (such as the relationship between pixels classified as shadow, orthe spatial relationship between known objects and shadow regions).

Cucchiara et al. in [17] take as their starting point detected movingobjects and a background model. The pixel values of moving objects areconverted to the HSV (Hue, Saturation and Value) colour space, and thenobserved values of all three HSV components are compared to those of thebackground model. The particular calculations they make are the differencebetween foreground and background values for H and S, and the ratio ofthe two V values. This captures the intuitive observations that shadows areabout the same hue as the same part of the scene unshadowed, slightly more

14

saturated, and darker. Stauder, Mech and Ostermann in [64] use assumptionsabout the background (it will dominate the scene), the nature of shadowsand luminance (shadows are darker and tend to have uniform shading) andthe presence of moving and static edges. In addition to these they use thewidth of edges to detect a shadow’s penumbra: in a world without point lightsources, shadows have fuzzy edges – so those regions bound by broad edgesare candidates for shadows as the edges could be penumbra.

Martel-Brisson and Zaccarin, in [45] and [46], present a Gaussian-mixturemodel-based approach for shadow detection. They use three types of modelin coordination to find the shadows: one of these models physical character-istics of shadows, and the other two models are statistical. The simplest is aphysical model of shadow appearance, which essentially expresses the famil-iar notion that shadows are similarly coloured to background but darker, andthey use some of the models described earlier [17, 29]. This physical modelalone is insufficient, and they augment it with statistical learning to try tominimise false shadows. Using a Gaussian mixture model (GMM) with fourGaussians to model the distribution of pixel colours in the background, theyassume the most stable component is the actual background and all othersforeground. As observations accrue various other colours will be capturedby the GMM as occurring at this one particular pixel. However, the shad-owed value can be assumed to be the most stable foreground Gaussian asit will occur more frequently than any foreground colour caused by movingobjects or noise. This most stable foreground component is then comparedto the physical shadow model, and if it is a plausible shadow colour, thelearning parameter of that particular Gaussian is increased so that distri-butions which are plausible shadow colours at a particular pixel convergemore quickly. Their third component (the Gaussian Mixture Shadow Model,or GMSM) stores the parameters of up to three previously learned stableshadow Gaussians, which avoids the “forgetting” of shadow characteristicsin periods of great foreground motion or changing illumination.

Joshi and Papanikolopoulos [33] present work which uses a support vectormachine (SVM) to perform classification of image regions into shadow andnon-shadow categories. As with many of the papers we discuss here, theirstarting point is a GMM of background appearance and a “weak classifier”.Their classifier is based upon colour and edge features, and is used to trainthe SVM. This allows for more variation in shadow appearance than manyother approaches, as an SVM can learn a more complicated discriminatoryfunction. In [34] this approach is extended using a co-training framework.

15

In co-training, a small set of labelled examples are used to train a pair ofclassifiers, and then for previously unseen and unlabelled examples the outputof each classifier is used as new labelled data to train the other. The twoclassifiers presented in this work use edge features and colour features, andthus those patches which are confidently classified as shadow based uponcolour are used as new examples for training the classifier based upon edgefeatures, and vice versa. The presented results are very impressive.

In [50] a technique for shadow detection which is part statistical andpartly based upon physical attributes is described. This is a seven-stagealgorithm which is novel within computer vision as it models the physicalcharacteristics of shadows from two light sources: “diffuse” and “point” (thesky and the sun respectively). They start with a GMM based moving objectdetector, then reduce the detected pixels by getting rid of those which arebrighter than the corresponding background pixel. Next, the detected area isthresholded on saturation, and if not too saturated they keep pixels which arebluer (shadows are assumed to be illuminated only by sky, not sun). Theythen use a new “spatio-temporal albedo” measure which looks at neighbour-ing pixels in time and space, searching for those which are uniform. Remain-ing pixels are candidate shadow pixels, and the difference between these andbackground pixels is used to discard those which are actually background.The penultimate step estimates body colour from a segmented region, andthe final step matches body colour against learned body colours from thescene. This technique seems to work well on the author’s test data, but islimited to outdoor situations.

This approach of using a physics-based techniques and features has be-come more popular in the last two years. Martel-Brisson and Zaccarin [47]take a simplified reflectance model and use it to learn the way in whichcolours change when shaded, and Huang and Chen [31] have also incorpo-rated a richer, physics-based colour model for shadow detection based uponthe work of Maxwell et al [48]. Maxwell presents a bi-illuminant dichromaticreflection model, which enables the separation of the effects of lighting (di-rect and ambient) from the effects of surface reflectance. Huang and Chensimplify this model in several ways, such as assuming that the ambient illumi-nation is constant, which enables them to implement shadow detection basedupon the simplified model in a video analysis task. Their system involves aglobal shadow model which is a GMM representing the change in colour ofa pixel when shaded (based upon the ambient illumination), and a per-pixelcolour model. The use of a global model means that their approach is very

16

fast to train and robust to low frame rate videos. Results are presented whichshow that this approach performs comparably to other methods [41,46,47].

4.1 Using spatial information within shadow detection

We now move on to techniques which incorporate spatial information. Thesimplest way to do this is to use some measure of “spatial coherence” (shad-owed pixels tend to be next to other shadowed pixels), but some authors usemore sophisticated spatial models of shadow location. Porikli and Thorntonin [55] present a method which is similar in spirit to that of Martel-Brissonand Zaccarin. They also use a physical model of shadows as a weak clas-sifier (shadows are darker than the expected background), and use thosepixels which satisfy this condition for updating their Gaussian shadow mod-els. However they introduce a spatial coherence condition in addition tocolour information.

Micic et al in [49] also use spatial coherence. This is enforced by smooth-ing and morphological operations, to eliminate small shadow regions that oc-cur inside foreground or background. Their colour based classifier is foundedupon the observation that the colour change due to shading can be ap-proximated by a diagonal matrix transformation in colour space. Rittscheret al [60] enforce spatial coherence through the use of a Markov RandomField (MRF); they also use temporal continuity constraints in their shadowand foreground detection. Salvador et al [61] also exploit spatial coherence.Shadow pixels are initially detected based upon colour difference to a ref-erence pixel5. They use an observation window rather than working at thelevel of the individual pixel to reduce noise, and a Gaussian distribution tomodel the difference between shadow and non-shadow pixel colours. Theythen use spatial constraints to remove spurious object pixels classified asshadow (shadow regions cannot be entirely surrounded by object regions),and a final information integration stage makes the decision as to whether apixel depicts a shadow or not.

A similar effect is obtained by Wang et al. in [69] and extended to incor-porate edge information in [70], who use a statistical approach based uponHidden Markov Models and Markov Random Fields. They combine these twomodels in a Dynamic Hidden Markov Random Field (DHMRF). The dynamic

5In video, the reference pixel is the same spatial location but from the backgroundmodel, in a still image, the reference pixel is a neighbour

17

segmentation is modelled within the Hidden Markov Model framework, andspatial constraints are handled by the Markov Random Field. This has theeffect of making a pixel more likely to be shadow if its neighbours are shaded.They model background variation using a GMM; when a foreground pixelis discovered they use the DHMRF framework (based upon colour, spatialcoherence and also edges) to decide whether that pixel is background (andhence update the GMM) or whether it is shadow or foreground. A furtherenhancement is introduced in [71] in which additional latent variables areintroduced further stabilising the segmentations. As the authors are consid-ering road traffic scenes alone they impose a further constraint by assumingthat foreground objects are rectangular. Similarly, Liu et al [41] use GMMsand weak spatial information, but they encode the spatial information usinga Markov Random Field formalisation to smooth detected shadow pixels.

Hsieh et al. in [30] propose a technique which uses a stronger form ofspatial information (as well as colour information). They assume that theobject casting the shadow is a pedestrian (or more than one pedestrian),and that the shadow is being cast onto the ground (flat planar surface).They first perform a background subtraction then morphological operationsto obtain the moving objects (people) and their shadows. On this segmentedarea they then calculate the centre of gravity and orientation (estimated bytaking moments of the area). This allows them to find a rough segmentationof person from shadow by finding the bottom of the person and drawinga diagonal line (oriented to match the orientation of the entire segmentedarea): see Figure 4(a) for an illustration of this. Given this rough shadowsegmentation they then build a Gaussian model of the colour distribution ofthe shadow pixels, allowing colour based refinement of the shadow model.

Renno et al. in [58] describe another shadow detection technique whichuses spatial information to augment a colour based shadow segmentation. Inthis paper they deal with the characteristic quadruple shadows cast by foot-ball players under floodlights, and a novel skeletonisation approach is usedto distinguish those foreground detections due to the cast shadows (whichappear on the floor) and those which are due to actual foreground motion.Those pixels which are most likely to be shadow are used to train the shadowGMM, and the others to train the foreground models. Figure 4(b) illustratesthis approach.

Both [30] and [57] use strong spatial information, but also make somestrong assumptions about the light, the screen, and the caster. Hsieh et al.have difficulty in detecting shadows where there are overlapping pedestrians,

18

(a) (b)

Figure 4: Two computer vision techniques which exploit spatial as well ascolour information to perform shadow removal: 4(a) shows the method ofHsieh et al. [30], and 4(b) shows the skeletonisation (top) intermediate seg-mentation (middle) and final results of shadow removal (bottom) from Rennoet al. [57]

19

or pedestrians assuming unusual poses (sticking their arms out, for example),and explicitly only model shadows cast on a planar surface by people. Rennoet al. make similar assumptions – given their domain (soccer tracking) thisis a reasonable thing to do as soccer players are invariably human and soccerpitches are planar with a characteristic lighting pattern. In scenes in whichthese assumptions do not hold, these approaches will naturally have difficulty.

Cucchiara et al in [18] describe an extension to their 2001 work [17] inwhich a higher level reasoning component classifies regions as one of Mov-ing object, Background, Shadow, Ghost or Ghost shadow. Regions as shadowhave to be adjacent to moving object regions. This prevents spurious shadowsunattached to casters being “invented” by the software. In their terminology,a ghost is an artifact of the tracking system and can correspond to an erro-neous foreground detection or a an erroneous shadow detection. By using thereasoning component to work out where shadows and ghosts should appear,they handle these problems well.

When we consider systems which use shadows, instead of filtering themout, there are only a handful: [4] use known 3D locations and their cast shad-ows to perform camera calibration and light location (using known castersand screen to tell about light source); [10] uses the moving shadows cast byknown vertical objects (flagpoles, the side of buildings) to determine the 3Dshape of objects on the ground (using the shadow to tell about the shape ofthe screen).

Balan et al. [1] use shadows as a source of information for detailed hu-man pose recognition: they show that using a single shadow from a fixedlight source can provide disambiguation in a similar way to using additionalcameras. They estimate human pose from a single calibrated camera, us-ing a strong light source to cast shadows on the ground. The shape of theshadow and the shape of the observed silhouette taken together enable de-tailed recovery of the pose of the human. In this work they also discuss theestimation of light source position (given pose and shape), and the surfacereflectance of the person under consideration.

To the best of our knowledge the first paper to attempt a formalisationof shadows (from an artificial intelligence standpoint) in order to accomplishautomatic scene recognition is [68]. This paper presents a number of com-puter programs capable of reconstructing 3D descriptions from line drawingsof objects and their shadows. After an initial identification and grouping ofshadow lines and regions from line drawings, the system proposed is capableof extracting high-level relations representing contact, support and orienta-

20

tion between objects.

4.2 Shadows within robotics

Within robotics the emphasis of the computer vision task shifts from thepassive interpretation of a scene to active exploration of the visual world andthe robot’s place within it. Perhaps unsurprisingly the use of shadows withinrobotics is therefore more common than within mainstream computer vision.There are several systems which make use of cast shadows for informing aboutthe location of the robot or the robot’s manipulators, and the relationshipbetween the robot and its environment.

Two systems [14,26] have used the shadow cast by a robot’s arm to refinethe robot’s estimation of limb location. When a robot wishes to move itsarm from A to B in the real world, it has various sources of informationabout the motion. Visual feedback is a central part of this and these recentpapers have incorporated shadows into the visual element of robot motioncontrol, inspired in part by [12] who showed that humans use the shadowsof their own limbs in a similar fashion. Fitzpatrick and Torres-Jara in [26]track the position of a robotic arm and its shadow cast on a table to derivean estimate of the time of contact between the arm and the table. Shadowsare detected in this work using a combination of two methods: in the first,a background model of the workspace is built without the arm and thenused to determine light changes when the arm is within the camera view.The second method compares subsequent frames in order to detect movingregions of light change. The authors motivate their work pointing out thatdepth from shadows and stereopsis may work as complementary cues forrobot perception, while the latter is limited to surfaces rich in textures, theformer works well in smooth (or even reflective) surfaces. Cheah et al. [14]present a novel controller for a robot manipulator, providing a solution tothe problem of trajectory control in the presence of kinematic and dynamicuncertainty. In order to evaluate their results, an industrial robot arm wascontrolled using the visual observation of the trajectory of its own shadow.Kunii and Gotoh [39] propose a Shadow Range Finder system that uses theshadow cast by a robot arm on the surface of a terrain in order to obtaindepth information around target objects. In planetary explorations this typeof system may provide low-cost, energy-saving sensors for the analysis of theterrain surrounding rock samples of interest.

Within the field of robotics planning and navigation, Tompkins et al.

21

[65] describe an autonomous path planning system that takes into accountvarious conditions of the robot‘s state, including pecularities of the terrainand lighting. In this context, the information about shadows cast by terrainirregularities allows the rover to plan a trajectory that maximises the trade-offbetween the exposure of the solar cells to sun light and the limited resources(including time) in planetary missions. More recently, Santos et. al. [62]describe an initial representation of cast shadows in terms of a spatial logicformalising occlusion relations. This initial representation is used in a mobilerobot self-localisation procedure in office-like environments to determine therelative locations of light source, caster, and robot. Lee and colleagues [40]use cast shadows inside pipes to detect landmarks: by fitting bright lightsto the front of their pipe inspection robot, they can determine when a pipebends by detecting cast shadows.

5 Conclusions and open questions

As discussed in Section 3, the findings from the psychological literature sug-gest that the human perceptual system uses shadow information in the in-terpretation of 3D motion and that shadow information can even overridenotions of conservation of object size. Indeed, we seem to have a majorperceptual bias towards shadows for determining this sort of information.We also make some major assumptions in the detection of shadows (com-mon motion, overhead light source, darker than surroundings) and use theseassumptions to reduce the complexity involved in understanding the visualworld.

We know of no work to date within artificial intelligence or computervision that uses shadows in the same way that human systems do – thatis, using a coarse shadow representation early in processing, to determinespatial relationships between elements of the 3D scene and to assist in depthperception. Within computer vision we can now find shadow detection algo-rithms using similar visual features to the human perceptual system (colourand edge based features) and some spatial features (e.g. spatial coherence).However it remains the case that the aim of the vast majority of these systemsis the ability to ignore shadows, not to use them.

This could be due to the passive nature of many computer vision systems,which aim to model and understand the world from a single static camera,and we find the recent developments in robotics heartening. However we

22

would like to argue that a considered reading of the psychological evidencejustifies the development of a shadow processing stage in any cognitivelyplausible vision system, and that no systems to date come close to this.

Acknowledgments

The authors would like to thank the British Council for funding exchangevisits between the authors during which these ideas were developed. HannahDee was partially supported by EPSRC Grant LAVID, EP/D061334/1, UK.Paulo Santos achnowledges support from FAPESP and CNPq, Brazil.

References

[1] Balan A.O., Black M.J., Haussecker H. and Sigal L. ‘Shining a lighton human pose: On shadows, shading and the estimation of pose andshape.’ In: Proc. International Conference on Computer Vision (ICCV).Rio de Janeiro, Brazil, 2007.

[2] Bonfiglioli C., Pavani F. and Castiello U. ‘Differential effects of castshadows on perception and action.’ Perception, Vol 33(11), pp. 1291–1304, 2004.

[3] Braje W.L., Legge G.E. and Kersten D. ‘Invariant recognition of naturalobjects in the presence of shadows.’ Perception, Vol 29, pp. 383–398,2000.

[4] Cao X. and Foroosh H. ‘Camera calibration and light source orienta-tion from solar shadows.’ Computer Vision and Image Understanding ,Vol 105(1), pp. 60–72, 2007.

[5] Casati R. ‘Methodological issues in the study of the depiction of castshadows: A case study in the relationships between art and cognition.’Journal of Aesthetics and Art Criticism, Vol 62(2), pp. 163–174, 2004.

[6] Casati R. ‘The shadow knows: a primer on the informational structureof cast shadows.’ Perception, Vol 33(11), pp. 1385–1396, 2004.

[7] Casati R. ‘The cognitive science of holes and cast shadows.’ Trends inCognitive Science, Vol 10(2), pp. 54–55, 2006.

23

[8] Casati R. ‘The copycat solution to the shadow correspondence problem.’Perception, 2007.

[9] Casati R. Shadows: Unlocking Their Secrets, from Plato to Our Time.Vintage, 2007.

[10] Caspi Y. and Werman M. ‘Vertical parallax from moving shadows.’ In:Proc. Computer Vision and Pattern Recognition (CVPR). New York,USA, 2006.

[11] Castiello U. ‘Implicit processing of shadows.’ Vision Research, Vol 41,pp. 2305–2309, 2001.

[12] Castiello U., Lusher D., Burton C. and Disler P. ‘Shadows in the brain.’Journal of Cognitive Neuroscience, Vol 15:6, pp. 862–872, 2003.

[13] Cavanagh P. ‘The artist as neuroscientist.’ Nature, Vol 434, pp. 301–307,2005.

[14] Cheah C.C., Liu C. and Slotine J.J.E. ‘Adaptive tracking control forrobots with unknown kinematic and dynamic properties.’ I. J. RoboticRes., Vol 25(3), pp. 283–296, 2006.

[15] Convay B.R. and Livingstone M.S. ‘Perspectives on science and art.’Current Opinion in Neurobiology , Vol 17, pp. 1–7, 2007.

[16] Crow F. ‘Shadow algorithms for computer graphics.’ In: SIGGRAPH’77: Proceedings of the 4th annual conference on Computer graphics andinteractive techniques , pp. 242–248. ACM Press, 1977.

[17] Cucchiara R., Grana C., Neri G., Piccardi M. and Prati A. ‘Thesakbot system for moving object detection and tracking.’ In: Video-based Surveillance Systems: Computer Vision and Distributed Process-ing (Part II - Detection and Tracking), pp. 145–158. Kluwer AcademicPublishers, 2001.

[18] Cucchiara R., Piccardi M. and Prati A. ‘Detecting moving objects,ghosts and shadows in video streams.’ IEEE transactions on PatternAnalysis and Machine Intelligence (PAMI), Vol 25(10), pp. 1337–1342,2003.

24

[19] da Costa Kauffmann T. ‘The perspective of shadows: The history ofthe theory of shadow projection.’ Journal of the Warburg and CourtauldInstitutes , Vol 38, pp. 258–287, 1979.

[20] da Vinci L. Notebooks of Leonardo Da Vinci . Dover, New York, 1978.

[21] Durou J.D., Falcone M. and Sagona M. ‘Numerical methods forshape-from-shading: A new survey with benchmarks.’ Computer Vi-sion and Image Understanding , Vol 109(1), pp. 22 – 43, 2008. ISSN1077-3142. URL http://www.sciencedirect.com/science/article/

B6WCX-4PRYG8D-1/2/4ab79e3dac220aa9f08212aa84d5dbbf.

[22] Elder J., Trithart S., Pintilie G. and MacLean D. ‘Rapid processing ofcast and attached shadows.’ Perception, Vol 33, pp. 1319–1338, 2004.

[23] Enns J.T. and Rensink R.A. ‘Influence of scene based properties onvisual search.’ Science, Vol 247, pp. 721–723, 1990.

[24] Finlayson G.D., Hordley S.D., Drew M.S. and Lu C. ‘On the removalof shadows from images.’ IEEE transactions on Pattern Analysis andMachine Intelligence (PAMI), Vol 28(1), pp. 59–68, 2006.

[25] Fiorani F. ‘The colors of Leonardo’s shadows.’ Leonardo, Vol 41(3),pp. 271–278, 2008.

[26] Fitzpatrick P. and Torres-Jara E. ‘The power of the dark side: using castshadows for visually-guided touching.’ In: Proc. of the 4th IEEE/RASInternational Conference on Humanoid Robots , pp. 437– 449. 2004.

[27] Galfano G. and Pavani F. ‘Long-lasting capture of tactile attention bybody shadows.’ Experimental Brain Research, Vol 166, pp. 518–527,2005.

[28] Hasenfratz J.M., Lapierre M., Holzschuch N. and Sillion F. ‘A sur-vey of real-time soft shadows algorithms.’ Computer Graphics Forum,Vol 22(4), pp. 753–774, 2003.

[29] Hoprasert T., Harwood D. and Davis L.S. ‘A robust background sub-traction and shadow detection.’ In: Proc. Frame Rate workshop, Inter-national Conference on Computer Vision. 1999.

25

[30] Hsieh J.W., Hu W.F., Chang C.J. and Chen J.S. ‘Shadow elimination foreffective moving object detection by gaussian shadow modeling.’ Imageand Vision Computing , Vol 21(6), pp. 505–516, 2003.

[31] Huang J.B. and Chen C.S. ‘Moving cast shadow detection using physics-based features.’ In: Proc. Computer Vision and Pattern Recognition(CVPR). 2009.

[32] Jacobson J. and Werner S. ‘Why cast shadows are expendable: insen-sitivity of human observers and the inherent ambiguity of cast shadowsin pictorial art.’ Perception, Vol 33, pp. 1369–1383, 2004.

[33] Joshi A. and Papanikolopoulos N. ‘Learning of moving cast shadows fordynamic environments.’ In: Proc. of the IEEE International Conferenceon Robotics and Automation, pp. 987–992. 2008.

[34] Joshi A. and Papanikolopoulos N. ‘Learning to detect moving shadowsin dynamic environments.’ IEEE transactions on Pattern Analysis andMachine Intelligence (PAMI), Vol 30(11), pp. 2055–2063, 2008.

[35] Kersten D., Knill D., Mamassian P. and Bulthoff I. ‘Illusory motionfrom shadows.’ Nature, p. 31, 1996.

[36] Kersten D., Mamassian P. and Knill D. ‘Moving cast shadows and theperception of relative depth.’ Technical Report 6, Max-Planck-Institutfur biologische Kybernetik, 1994.

[37] Kersten D., Mamassian P. and Knill D. ‘Moving cast shadows induceapparent motion in depth.’ Perception, Vol 26, pp. 171–192, 1997.

[38] Kriegman D.J. and Belhumeur P.N. ‘What shadows reveal about objectstructure.’ In: ECCV ’98: Proceedings of the 5th European Conferenceon Computer Vision-Volume II , pp. 399–414. Springer-Verlag, London,UK, 1998.

[39] Kunii Y. and Gotoh T. ‘Evaluation of Shadow Range Finder: SRF forPlanetary Surface Exploration.’ In: Proc. of the IEEE InternationalConference on Robotics and Automation, pp. 2573–2578. 2003.

[40] Lee J.S., Roh S.G., Kim D.W., Moon H. and Choi H.R. ‘In-pipe robotnavigation based upon the landmark recognition system using shadow

26

images.’ In: Proc. of the IEEE International Conference on Roboticsand Automation. 2009.

[41] Liu Z., Huang K., Tan T. and Wang L. ‘Cast shadow removal combin-ing local and global features.’ In: Proc. Computer Vision and PatternRecognition (CVPR). 2007.

[42] Mamassian P. ‘Impossible shadows and the shadow correspondenceproblem.’ Perception, Vol 33, pp. 1279–1290, 2004.

[43] Mamassian P. and Goutcher R. ‘Prior knowledge on the illuminationposition.’ Cognition, Vol 81(1), pp. B1–B9, 2001.

[44] Mamassian P., Knill D.C. and Kersten D. ‘The perception of cast shad-ows.’ Trends in cognitive sciences , Vol 2(8), pp. 288–295, 1998.

[45] Martel-Brisson N. and Zaccarin A. ‘Moving cast shadow detection froma Gaussian mixture shadow model.’ In: Proc. Computer Vision andPattern Recognition (CVPR), pp. 643–648. 2005.

[46] Martel-Brisson N. and Zaccarin A. ‘Learning and removing cast shadowsthrough a multidistribution approach.’ IEEE transactions on PatternAnalysis and Machine Intelligence (PAMI), Vol 29 (7), pp. 1134–1146,2007.

[47] Martel-Brisson N. and Zaccarin A. ‘Kernel-based learning of cast shad-ows from a physical model of light sources and surfaces for low-levelsegmentation.’ In: Proc. Computer Vision and Pattern Recognition(CVPR). 2008.

[48] Maxwell B., Friedhoff R. and Smith C. ‘A bi-illuminant dichromaticreflection model for understanding images.’ In: Proc. Computer Visionand Pattern Recognition (CVPR). 2008.

[49] Mikic I., Cosman P., Kogut G. and Trivedi M. ‘Moving shadow andobject detection in traffic scenes.’ In: Proc. International Conferenceon Pattern Recognition (ICPR), pp. 321–324. Barcelona, Spain, 2000.

[50] Nadimi S. and Bhanu B. ‘Physical models for moving shadow and objectdetection in video.’ IEEE transactions on Pattern Analysis and MachineIntelligence (PAMI), Vol 26(8), pp. 1079– 1087, 2004.

27

[51] Ni R., Braunstein M. and Andersen G.J. ‘Perception of scene layout fromoptical contact, shadows and motion.’ Perception, Vol 33, pp. 1305–1318, 2004.

[52] Ostrovsky Y., Cavanagh P. and Sinha P. ‘Perceiving illumination incon-sistencies in scenes.’ Perception, Vol 34, pp. 1301–1314, 2005.

[53] Pavani F. and Castiello U. ‘Binding personal and extrapersonal spacethrough body shadows.’ Nature Neuroscience, Vol 7(1), pp. 13–14, 2004.

[54] Plato. The Republic. Penguin Classics, 2007. Translated by H.D.P. Lee,Desmond Lee, written 360 BC.

[55] Porikli F. and Thornton J. ‘Shadow flow: A recursive method to learnmoving cast shadows.’ In: Proc. International Conference on ComputerVision (ICCV). 2005.

[56] Prati A., Mikic I., Trivedi M. and Cucchiara R. ‘Detecting moving shad-ows: algorithms and evaluation.’ IEEE transactions on Pattern Analysisand Machine Intelligence (PAMI), Vol 25(7), pp. 918–923, 2003.

[57] Renno J.R.R., Orwell J. and Jones G.A. ‘Evaluation of shadow classifi-cation techniques for object detection and tracking.’ In: IEEE Interna-tional Conference on Image Processing . Singapore, 2004.

[58] Renno J.R.R., Orwell J., Thirde D.J. and Jones G.A. ‘Shadow classifi-cation and evaluation for soccer player detection.’ In: Proc. British Ma-chine Vision Conference (BMVC), pp. 839–848. Kingston upon Thames,UK, 2004.

[59] Rensink R.A. and Cavanagh P. ‘The influence of cast shadows on visualsearch.’ Perception, Vol 33, pp. 1339–1358, 2004.

[60] Rittscher J., Kato J., Joga S. and Blake A. ‘A probabilistic backgroundmodel for tracking.’ In: Proc. European Conference on Computer Vision(ECCV), pp. 336–350. 2000.

[61] Salvador E., Cavallaro A. and Ebrahimi T. ‘Cast shadow segmentationusing invariant color features.’ Computer Vision and Image Understand-ing (CVIU), Vol 95(2), pp. 238–259, 2004.

28

[62] Santos P.E., Dee H.M. and Fenelon V. ‘Qualitative robot localisation us-ing information from cast shadows.’ In: Proc. of the IEEE InternationalConference on Robotics and Automation. 2009.

[63] Sattler M., Sarlette R., Mucken T. and Klein R. ‘Exploitation of humanshadow perception for fast shadow rendering.’ In: APGV ’05: Pro-ceedings of the 2nd symposium on Applied perception in graphics andvisualization, pp. 131–134. ACM, New York, NY, USA, 2005.

[64] Stauder J., Mech R. and Ostermann J. ‘Detection of moving castshadows for object segmentation.’ IEEE Transactions on multimedia,Vol 1(1), pp. 65–76, 1999.

[65] Tompkins P., Stentz A. and Whittaker W.L. ‘Automated surface mis-sion planning considering terrain, shadows, resources and time.’ In: Pro-ceedings of the 6th International Symposium on Artificial Intelligence,Robotics and Automation in Space (i-SAIRAS ’01), Montreal, Canada..2001.

[66] Troccoli A. and Allen P.K. ‘A shadow based method for model reg-istration.’ In: Computer Vision and Pattern Recognition Workshops .2004.

[67] Vazquez E., van de Weijer J. and Baldrich R. ‘Histogram-based imagesegmentation in the presence of shadows and highlights.’ In: EuropeanConference on Computer Vision. Marseille, France, 2008.

[68] Waltz D. ‘Understanding line drawings of scenes with shadows.’ In: ThePsychology of Computer Vision, pp. 19–91. McGraw-Hill, 1975.

[69] Wang Y., Loe K.F., Tan T. and Wu J.K. ‘A dynamic Hidden MarkovRandom Field Model for foreground and shadow segmentation.’ In:Proc. IEEE Workshop on Applications of Computer Vision. 2005.

[70] Wang Y., Loe K.F. and Wu J.K. ‘A dynamic conditional random fieldmodel for foreground and shadow segmentation.’ IEEE transactions onPattern Analysis and Machine Intelligence (PAMI), Vol 28, pp. 279–289,2006.

29

[71] Wang Y. and Ye G. ‘Joint random fields for moving vehicle detec-tion.’ In: Proc. British Machine Vision Conference (BMVC). Leeds,UK, 2008.

[72] Woo A., Poulin P. and Fournier A. ‘A survey of shadow algorithms.’IEEE Computer Graphics and Applications , Vol 10(6), pp. 13–32, 1990.

30