Object Persistence for Synthetic CreaturesDamian A. Isla Bruce M. Blumberg

MIT Media Lab,77 Massachusetts Ave. NE18-5th fl.

Cambridge, MA 02139-4307 1-617-452-5603

{naimad, bruce}@media.mit.edu

ABSTRACTWe present methods for anticipatory behavior in simulatedgraphical creatures. We discuss in general terms theimportance of anticipatory behavior through explicitexpectation formation. We present an in-depth description of aspecific type of expectation-formation, namely location-expectation, or object persistence. A new representation – theProbabilistic Occupancy Map (POM) – is presented, and it isshown how this representation can be used to maintainestimations of the positions of mobile objects in the worldbased on both positive and negative knowledge provided bythe creature’s perceptual system. Finally a number ofillustrative results are presented that show Duncan, oursimulated dog, successfully performing a number of tasks thatrequire a high degree of spatial common sense.

Categories and Subject DescriptorsI.2.0 [Artificial Intelligence]: General – cognitive simulation

General TermsAlgorithms, Design, Theory.

KeywordsAutonomous agents, graphical agents, object persistence,spatial common sense.

1. INTRODUCTIONAs we attempt to build increasingly sophisticated autonomousinteractive agents (or “synthetic creatures” as we will call themin this paper) an important contributor to a life-likeappearance is the ability to anticipate. Anticipation – orexpectation formation – can be thought of as the ability tomake decisions and react to aspects of the world state that, forone reason or another, cannot be directly perceived. This mightinclude events that occur outside the field of view, areoccluded, or that have not yet happened (but are expected tohappen). There are many potential sources for theseexpectations, such as reliably observed event correlations(when the button is pushed, the elevator doors are expected toopen), theory of mind (given an assumed state of mind,another creature might be expected to perform action A),physical intuition (when a ball is released, it is expected tofall), or, as will be discussed at length here, spatial structure (ifa ball rolls behind a wall, it is expected to come out again aftera certain delay). Not only do these types of expectations makea creature seem more intelligent, but their absencesignificantly impairs any pretensions it might have to

common sense. If the ball disappears behind a wall, it wouldappear either broken or colossally stupid for the creature to

then not know where to look for it.

Our research group has, over the past few years, experimentedwith various aspects of expectation in graphically embodiedcreatures such as Duncan the Virtual Terrier (Figure 1). Duncanlives in a graphical environment which he perceives through asynthetic perception system which includes simulatedaudition and point-of-view rendering (synthetic vision).Duncan is a platform for various aspects of our research,including models of operant and classical conditioning andmotor learning. In the work described in this paper, Duncanwas given a sense of object persistence which allows him tomaintain reasonable models of the locations of mobile objectsin the world.

Section 2 is a brief discussion of expectation theory ingeneral. Section 3 discusses in depth the problem of location-expectation. Section 4 introduces Probabilistic OccupancyMaps (POMs) as a means of maintaining location-expectationdistributions. Section 5 presents some example results andconcluding remarks are given in Section 6.

2. EXPECTATION THEORY[8] provides an excellent overview of expectation theory. Thisdiscussion will largely parallel its conclusions.

In general, it is an observation of some sort – perhaps of anunusually salient stimulus – that leads to an expectationformation by a specialized predictor unit. These expectationsare accompanied by a degree of confidence. The validity of anexpectation is ascertained through verification. The process ofverification can have three possible outcomes:

• Verifiably true: The expectation turned out to be correct.

Figure 1: Duncan the Highland Terrier

• Verifiably false: The expectation turned out to beincorrect. This is an expectation violation.

• Unverifiable: The accuracy of the prediction could not bedetermined.

Typically, a verifiable prediction-outcome is followed bysome form of belief revision. The predictor that generated theexpectation might also be revised, for example, its reliabilityrating might be modified according to whether its outcomeproved accurate or not. If the prediction accuracy wasunverifiable then neither the expectation nor the predictoritself is impacted. This entire process is summarized in Figure2.

The above formulation applies to discrete true/falsepredictions. Some expectations, however, are better formulatedas a space of predictions that might naturally be expressed as aprobability distribution over the possible world states, as inFigure 3. Within this space, each infinitesimal element can be

considered an individual prediction to be verified as true orfalse. These individual predictions can affect each other in twoways: first, in that an observation and confirmation of oneprediction is equivalent to a negation of all the otherpredictions, resulting in a “tightening” of the distributionaround the observed value (Figure 3b). Second, a verifiednegation of a sub-region of the state-space (effectively aculling of probability from the distribution) results in arenormalization of the rest of the distribution (Figure 3c).

2.1 Expectation Violations and SalienceWe define salience as the degree to which an observationviolates expectation. As [8] points out there are two types ofexpectation violation: unexpected observations and negatedexpectations (verified false predictions). For the first kind, astraightforward inverse relationship exists between salienceand confidence at the observed value (thus unexpectedobservations are more salient):

1()()()cxsxcx-=

(1.1)

This maps salience into the range [0,∞] (assuming that c(x) hasrange [0,1]). We call this form of salience surprise.

For expectation violations of the second kind, the salience canbe considered proportional to the amount of confidence culledby the observation. This might be expressed as

()()Rsxcx=Ú(1.2)

where R is the region of the state-space of x that has beenverified as false (see Figure 3c). We call this form of salienceconfusion.

It is notable that many secondary emotions imply some formof expectation: fear, delight, disappointment, anticipation,confusion, dread, worry, etc. They also imply an extendedrange of possible interactions. Creatures that can formexpectations can be tricked, teased, deceived and more.

3. OBJECT PERSISTENCEIn this section we introduce an extended example ofexpectation-formation: that of location-expectation. We arguethat the ability to maintain reasonable location-expectationsis tantamount to a sense of object persistence. Objectpersistence, as discussed by the psychologist Piaget [Piaget,1954 #15], refers to the persistence of mental images ofobjects after they have stopped being perceived. It alsoimplies the ability to make basic deductions about whereobjects could be, and to act on those deductions. If a childturns toward a toy that has been hidden from it, it doesn’tforget about the toy, or remain staring at the toy’s last knownposition. Instead it searches for the toy systematically, basedon its last observed position and based on the physicalstructure of the environment – could it have been hiddenbehind the box, or under the table? The basic problem ofobject persistence, which we would like our synthetic creaturesto be able to solve, is: given that an object is not currentlyvisible, where does the creature presume that object to be?

Expectation

Verification

FalseTrue Unverifiable

...

Predictor

Predictor

Predictor

Predictor

Reward

Salient Event

Punish

Figure 2: The expectation/verification loop

p(x)

x

(a)

p(x)

x

(b)

x

p(x)

(c)Figure 3: Incorporating positive and negative

observations.

A distribution (a) represents a space of predictions with theindicated confidence. An observation of the true state (b)results in a tightening of the distribution while thenegation of some region of the state space (c) zeroes thedistribution in that region, and scales the rest of thedistribution up as appropriate.

3.1 Distribution RepresentationIn forming probability distributions over space, we must comeup with some way of representing those distributions. Onenatural representation might be the gaussian distribution –after all, it is compact, and mathematically convenient, withnotions of confidence and “best guess” built in.

The difficulty with the straight gaussian representation is thatit does not lend itself easily to the incorporation of negativeknowledge. Figure 4 shows an example of this problem. Figure4a, shows the distribution for an occluded object. A great dealof negative information – namely, those locations observed tobe empty – is being thrown out. The correct distribution, theone in which all probability has been zeroed in all visiblyempty space, is shown in Figure 4b. This distribution i sclearly not gaussian – it is even disjoint. Since the shape ofthe distribution relies ultimately on the physical structure ofthe environment, it can be arbitrarily complex – and arbitrarilymessy.

3.2 Probabilistic Occupancy MapsThe general strategy we use to overcome the difficulties of thegaussian representation is to discretize the location-distribution, through the use of a Probabilistic OccupancyMap (POM). In this formulation the environment is partitionedinto discrete locations which can be used as “buckets” ofprobability. Thus the compact gaussian representation isreplaced with a vector of discrete activations, in which eachactivation corresponds to the probability that the target objectis contained in that location. As Figure 4c shows, this schemehas no problem representing oddly-shaped distributions,though its accuracy obviously depends on the resolution ofthe map itself.

We use a hexagonal grid overlaid on the environment as thegeneral form of the map. Each node is connected to sixneighbors, and is denoted by a pair of (i,j) indices (to maintaincorrect topology, the connectivity of these nodes alternates oneven and odd rows). Each node has a world-space position, andrepresents all space for which it is the nearest node. Each nodeis also annotated with a probability vector, each elementindicating the probability that the node contains a particularobject. For simplicity, in the following discussion we willassume that there is only one target object to be tracked, andthat each node is annotated with a single probability, p(ni, j), ofcontaining that object.

In some experiments the maps themselves were learned, suchthat higher resolution was used in more “interesting” parts ofthe environment, such as around prominent landmarks. Thisstrategy is very useful in terms of representational economy.However, it is not essential to the basic implementation, andso will not be discussed here. See [6] for details.

3.2.1 Probability DiffusionOn timesteps in which the target object is not observed, adiscrete diffusion step is carried out to reflect the decreasedconfidence in the target’s location.

Simple isotropic diffusion works well. The update expressionfor a single map element, or node, can be given as

1'()()(1)()(')6tttnneighborsnpnpnpnll+Œ=-+ (1.3)

where l

is a diffusion constant in the range [0,1] and

()tpn

is the probability of the node n at time t. Thus at each timestep,each node passes some fraction of its own activation to itsneighbors. The diffusion constant should be roughlyproportional to the mobility of the object (such that an objectthat has been observed to be stationary has a low diffusionconstant, and so on).

When the target was last observed in motion, the probabilityshould diffuse preferentially in the direction of the target’slast observed velocity. This can be used in the example inwhich the target object disappears behind an occluder – thecreature can form an expectation about when it shouldreappear. The most obvious way to achieve this is to modifythe diffusion rates so as to favor diffusion in the correctdirection.

Assuming that a target object’s velocity can be estimatedthrough simple averaging of position changes over recenttimesteps, we might set the diffusion rate at a specific node as

(a)

(b)

(c)

Figure 4: Distribution Representation

A gaussian distribution (a) cannot incorporate negativeinformation about regions of space observed to be empty(b). In this work, we discretize the space itself, and alloweach element to become a “bucket” of probability (c).

2max0,iicivllll⋅=+ʈÁ˜Ë¯

(1.4)

where v is the velocity vector, li is the position offset betweenthe current node and the node’s ith neighbor, and li is the

diffusion rate along the ith connection. lc is a constantdiffusion rate, ensuring that some probability is diffused toevery neighbor, even if that neighbor does not lie in thedirection of the velocity vector.

At each timestep, therefore, each node gives li of its totalactivation to each of its i neighbors, and keeps the remainingactivation for itself. If the sum of these rates is more than 1,then the rates are normalized to add to 1 (since a node cannotgive away more probability than it has). Note that if thevelocity v is zero, then the entire expression is equivalent toequation (1.3).

To keep distributions from shooting off into infinity, we alsoslowly decay the velocity vector over all timesteps in whichthe object is not observed.

3.2.2 ObservationOn timesteps in which the target object was observed, we pickthe node n* that is closest to the target’s position. We tightenthe distribution around this node by setting its probability to1, and zeroing the probabilities of all other nodes:

*,,0,()ijijnnpn"≠¨(1.5)

*1()pn¨(1.6)

As in section 2.1, we determine a salience for the observationwith the expression

**1()()pnspn-=

(1.7)

While this gives a valid mapping for a salience in the range[0,∞], it has one undesirable property: that when thedistribution is very diffuse, even the most likely location (thelocation with the highest p *) returns a high salience ifconfirmed. Since the desired behavior should be for the mostexpected prediction not to be considered salient whenconfirmed, we normalize the previous expression by phighest, theprobability of the most likely location. Thus,

**))((highestppnspn-=

(1.8)

3.2.3 VerificationVerification in this domain requires us to use informationabout locations observed to be empty. If a map node with acertain probability is considered a prediction about anobject’s location, then the observation of that locationwithout the corresponding observation of the object can beconsidered a negation of that prediction. In such a case, theprobability content of the node is zeroed. Once all visiblelocations are zeroed, the remaining distribution in non-visiblelocations is renormalized.

On timesteps in which the target object was not observed, themap nodes are divided into a visible set, V, and a non-visibleset, N . First a total amount of culled probability pculled i scalculated as

()cullednVppnŒ=Â(1.9)

Then the probabilities of all visible nodes are set to zero:0,()nVpn¨"Œ

(1.10)

Finally the remaining probabilities are renormalized1,()()1cullednNpnpnp"ά-

(1.11)

Note that by the definition given by equation (1.2), the valuepculled is also the measure of the surprisingness of the failure toobserve the target object.

4. IMPLEMENTATIONThe POM scheme was implemented on top of C4, a charactersimulation platform described in [2] and [7]. Currently,Duncan (Figure 1) is our most sophisticated character, and i twas in his brain that most of the following was implementedand tested.

4.1 Synthetic VisionA simple model of synthetic vision was used as the primarysource of perceptual information for Duncan. Similar toschemes used in [1] and [18]. The vision model consisted of arendering of the world from the point of view of Duncan’s lefteye. This rendering was color-coded (as shown in Figure 5)such that individual objects were recognized by color (i.e. noshape-analysis was performed). From these renderings, screen-space object-centroids were extracted, and combined with thecontents of the depth-buffer to produce eye- and then world-space coordinates. Two implications of this strategy are that anobject is considered visible if any portion of that object i svisible, and that the location is taken as the centroid of thevisible portion.

4.1.1 The Test-point MethodAnother crucial function of the vision system is to determinethe visibility of map-locations for the purposes of negativeverification (i.e. in separating the N and V sets, from section

Figure 5: Duncan’s Synthetic Vision

3.2.3). A number of schemes are possible here, the “truest” oneprobably involving pixel-sampling over the expected screen-space extent of the node. This scheme is not used due to run-time considerations.

Instead, we use a vastly simplified Test-Point approximation,in which a single point is imagined floating half a body-height above the world location of the map-node to be tested.The world-coordinates of this point are transformed back intoNDC coordinates. The depth-buffer is then queried to seewhether an object at that point would be visible. If the depth ofthe pixel to which the test-point transformed is less then theNDC-depth of the test-point, then the test-point is hidden, andthe entire location is considered non-visible (and placed in theN set). If the depth is greater, then the location is consideredvisible (and placed in the V set).

Unlike object-observation, in which the view of any part of theobject counts as a full observation, it is generallyadvantageous in location-visibility to err on the side of non-visibility. In other words, it is advantageous to demand thatthe entire location be visible before the corresponding node i sconsidered visible. The danger is that if the target object is in alocation that is considered visible without being observed,then its true location will be discounted as a possible location.On the other hand, the risk of mistaking a visible location for anon-visible location is only that a location the creature is notat might be considered for a longer period than otherwise,especially since the likely reaction will be to approach thatlocation and examine it further. Note, however, that our test-point method does not follow this guideline.

4.2 Working Memory and Object MatchingDuncan maintains a Working Memory model that consists of arecent perceptual history of all objects in the world. On eachtimestep, the perceptual input from each object – color, shapeand location – from the vision system is bundled up into abelief object. These beliefs are compared to a list of persistentbeliefs stored in the creature’s Working Memory. Since manyof the new beliefs probably represent a new observation of anold belief (if it represents the newest data from an object thathas been tracked over multiple timesteps, for example), allincoming beliefs undergo a process of belief matching, inwhich the “distances” between the new belief and each of theold ones is found. If the lowest of these “distances” is below athreshold then, the two are considered a match, and the datacontained in the new belief is added onto the perceptualhistory contained in the old one. If the lowest distance i sabove the threshold, then the new belief is considered torepresent a new object, and is itself added to Working Memory.

In finding the distance between two perception tuples (eachconsisting of <color, shape, location>), each component inone tuple is compared with the corresponding component ofthe other. The total distance is then considered the sum of thecomponent-distances. For the purpose of this system, bothshape and color are symbolic binary matches returningdistance infinity or zero depending on whether there is anexact symbolic match or not (e.g. “red” = “red”).

In comparing location however, we can again make use of thelocation distributions of the POM, since ultimately we aremore interested in the likelihood of the target appearing at thenew location, than the straight distance between the newobservation and the last one (although straight distance mightbe an economical stop-gap). This likelihood is exactly what

the POM gives us. When a new observation is made, theincoming location data can be ascribed a nearest node, n*. Thisnode holds some probability, p(n*) of containing the targetobject. To find the “distance” between a location-distribution,and a new observation at n*, we use

**))((highestppndpn-=

(1.12)

where highestp

is again the highest probability value of any

node in the map. This is, of course, simply a re-casting of thesalience function of equation (1.8). This follows the intuitionthat if an observation is too unexpected, we might reject i taltogether. Thus location and location-expectations are centralto Duncan’s ability to distinguish between different objects inthe world.

When a new belief is merged into the old one, the appropriatedistribution tightening, as per equations (1.5) and (1.6)occurs. For all non-visible objects, the appropriatedistribution culling is carried out, as in equation (1.10),followed by the diffusion of all location-distributions, as inequation (1.3).

4.3 Action Selection and Motor ControlHaving updated his perceptual image of the world, Duncanmust decide what to do. Though the problem of ActionSelection is a major focus of our research (particularly withregards to learning), it is largely irrelevant to this work. Inmany of the experiments conducted on this system, Duncanwas given just three behaviors: “look at target”, “approachtarget” and “approach shepherd” (the shepherd being a user-controlled character inhabiting the same space). Duncandecided between these behaviors deterministically at theinstruction of the shepherd. These high-level behaviors aretranslated by the action selection system into low-levelnavigation and motor commands. A Navigation System wasuse to control locomotion toward a goal and obstacleavoidance, and a Motor System was used as the lowest-levelinterface with the graphics system to control Duncan’sanimation.

5. RESULTSThis section presents a few scenarios as examples of Duncan’snew spatial capabilities with the POM system in place.

5.1 Salient Moving ObjectsOne result of the POM system is that moving objects–especially irregularly moving objects – automaticallybecome salient. If an object remains still, its distributionlooks like a roughly gaussian cluster of probability centeredon the observed location of the object. When the object startsmoving, the object is matched into a relatively unlikely part ofthis distribution – thereby making that observation salient.When velocity-based (anisotropic) diffusion is used, an objectthat zigzags is more salient than an object that moves at aconstant speed in a constant direction.

5.2 Search BehaviorsThere are a number of examples of “emergent search” that fallout of the POM system almost for free.

5.2.1 Emergent Look-AroundIntelligently moving the eyes over a visual scene, from ananimation point of view, can be important in maintaining theillusion of life [16]. However, there is no practical point to thisif the negative information about empty space is not used.Typically, the “look-around” behavior is a canned animation.However, a convincing “look-around” happens naturally withthe POM system. Consider a situation in which a target (in thiscase, a sheep) has not been seen in some time (and has beenmoved since it was last observed), and is represented by awide, diffuse distribution. When instructed to look at thesheep, Duncan turns towards the most likely position for thesheep, namely the center of the distribution. Not finding itthere, he looks first one way, then the other, always looking atthe current most likely position. The result is an alternatingsweep-left, sweep-right behavior, until the sheep is found at itsnew location.

5.2.2 Emergent SearchIf Duncan is instructed to approach the sheep, rather than justlook at it, then the emergent look-around becomes an emergentsearch. As in the emergent look-around, Duncan approaches atevery point the most likely location for the target object. He i spreceded by his gaze however, which rules out open areas ofthe environment. This leaves only areas occluded by barriers,and the resulting behavior is what looks like a systematicsearch of these occluded areas.

In one example, Duncan is introduced to the environmentshown in Figure 6. After a brief view of the environment andthe objects in it, Duncan is called to the shepherd. Behind hisback, the sheep is moved behind the nearest wall. When told toapproach the sheep, Duncan immediately goes to searchbehind the correct wall. Duncan is called away again, and thistime the sheep is moved from its original hiding place to a newone behind the other wall. When again instructed to approachthe sheep, Duncan first looks in the original hiding place. Notfinding it there, he comes out and again scans the scene, thistime concluding that the only place the sheep can now be i sbehind the second wall. Again, he finds the sheep in its hidingplace.

What is perhaps most interesting about this example, is that ata behavioral level – at the level of the Action Selectionmechanism – Duncan has only two behaviors to chose from:“approach the sheep” and “approach the shepherd”. However,the result is much more sophisticated, and has the appearanceof deliberative reasoning.

This “emergent search” behavior is extremely important forkeeping the process of action-selection clean. If “search”needed to be included at this level of the system, it would needto be listed explicitly as a strategy for “approaching thesheep,” “approaching the shepherd,” “biting the sheep”,“eating the sheep” etc. Matters would be even worse if weexpected learning to occur at this level of control – thuspushing onto the learning mechanism the burden of decidingwhen “biting the sheep” necessitates a “search.” With thearchitecture described here – in which some of the decision-making has been pushed up into the perceptual/memorysystem and down into the navigation/motor system – somedegree of search capability is implied in a l l targetedbehaviors.

5.3 Distribution-Based Object-MatchingTo show the advantage of distribution-based object-matching,a scenario was set up as shown in Figure 7. Like in a previousexample, Duncan tracks a moving sheep as it passes behind along wall. This time, however, the wall has a large gap in it. Thescene is staged, because at a certain time where the sheep,unimpeded, might have expected to come out from behind theright wall, a second sheep, initially hidden, appears there. Asingle sheep would have been observed to pass through thegap. Since it did not, the intelligent observer is forced toconclude that there are, in fact, two separate sheep. Happily,Duncan makes the same determination, since his constantviewing of the space between the two walls keeps any of theprobability from diffusing from behind one wall to behind theother. Thus when the second sheep appears, it appears in alocation for which the first sheep’s location-distribution i szero. Since it is impossible for the two observations torepresent the same object, the belief-matching step fails and anew persistent belief is added to Working Memory. This isapparent, because when Duncan is subsequently asked toapproach the nearest sheep, he looks behind the left wall first.

This is an exciting result for two reasons. First, it shows a verydifferent kind of “reasoning” from the previous examples – akind of negative reasoning, reflecting the ultimatedetermination that the two objects are not the same. Second, i tshows a kind of reasoning that is highly distributed: thevision system provides observation data, the VerificationSystem extracts visible locations, the Spatial Systemmaintains the POM itself, and the Working Memory systemmakes the ultimate decision to allocate a new persistent belief.In the best sprit of the Society of Mind [9], it is the confluenteffect of all these systems that results in the “intelligent”behavior.

Figure 6: The search task environment

Figure 7: The object-matching task environment

5.4 Emotional BehaviorDuncan maintains a list of variables to represent variousaspects of internal state. Some of these variables are explicit“emotional” variables. In one series of experiments, Duncanwas given the variables of “surprise”, “confusion” and“frustration”. The first two of these were simply the outputs ofequations (1.8) and (1.9) run through an asymmetric low-passfilter (for sharp rising edges and soft falling ones). The“frustration” variable grew by some factor on every timestepthat Duncan’s target was not directly observed. Figure 8 showsa stereotypical trace of these variable values over time in a“search for the sheep” task.

Currently, the only effect that these variables have is to changeDuncan’s facial expression (as shown in Figure 9). However,the ultimate hope would be to feed these emotional variablesback into Duncan’s basic decision-making processes, suchthat, for example, frustration could act as a signal to “stopwhatever you’re doing and try something else”, and surprisecould indicate “examine more carefully your object ofattention” etc.

Beyond being interesting from a procedural animation pointof view, we consider this type of emotional behavior to be animportant element of the appearance of intelligence. Evennaïve viewers can observe Duncan’s actions and facialexpression and come to fairly sophisticated high-levelconclusions about what Duncan is doing, what he wants andwhat he expects of the world. We are all very accustomed tomaking assumptions and reasoning about expectations.Allowing a synthetic creature like Duncan not only to have butalso to express those expectations gives observers morereason to ascribe to him the characteristic of intelligence.

6. CONCLUSIONSIn this paper, we have presented a model of object persistenceand a representation – the Probabilistic Occupancy Map – thatimplements it. We draw a number of major conclusions.

Architecturally, the POM is an important lesson in multi-layerdecision-making. The section on emergent search shows thateven with a simple explicit behavior-selection layer, manycomplicated behaviors can result with the rightperception/space-modeling mechanisms. The behavioral levelcould be completely user-directed, or scripted, or could be runby a complex automatic behavior simulation system. Whateverthe case, there is clear benefit to separating out this series ofspatial competencies into separate layers.

Another important point is that a good representation is wortha lot. In this case, the POM structure was tuneable enough to beuseful, but also general enough to account for manyinteresting effects. For example, it was fairly simple toincorporate a sense of momentum for velocity-basedprobability diffusion. It was also general enough thatunforeseen behaviors emerged without being designed for.

Most importantly, this work is an illustration of theexpectation theory that is presented in section 2. It shouldshow that (a) representations based on this theory areconceptually simple and easy to implement, (b) therepresentations are powerful, and lead to interesting behaviorsand abilities and (c) these new abilities contributesubstantially to a creature’s believability and apparentcommon sense. It is hoped ultimately that this work can serveas a model for how other kinds of expectation formation

abilities might be incorporated into a behavior simulationframework.

7. RELATED WORKSThe field of artificial life is relatively new, but already hassuch seminal works as [1] and [15]. More recently, work byTerzopoulos and his colleagues (e.g. [5] and [18] and) hasproved very impressive, and relevant to the current work fortheir concern with higher-level cognitive processes andcognitive modeling.

There has been work on expectation theory. [12] points outanother source of surprise: events for which there were noexpectations but that are inherently unusual. The examplegiven by the author is that of a brick flying through a window:no prior expectation existed over where or when such an eventwould occur, but it should nonetheless register a surprise. Wedo not treat this form of surprise in this work, though webelieve that such models of inherent salience could easily beincorporated into this framework. As already mentionedbefore, [8] was very influential to this work.

Drescher did work specifically on cognitive processes leadingto object persistence [3] though only in a toy world. His workwas itself an exploration of the computational side of Piaget’stheories (as in [13]).

Time

Var

iabl

e V

alue

Confusion FrustrationSurprise

Duncan instructed to approachsheep; discovers sheep is notin last-observed location

Sheep found inunexpected location.

Figure 8: Trace of emotional variables in a search task.

(a) (b)

(c) (d)

Figure 9: Duncan (a) neutral (b) confused (c) surprised(d) frustrated

Probabilistic Occupancy Maps are clear derivatives of theOccupancy Grids, or Evidence Grids used in the mobilerobotics literature (e.g. [10]). The major difference is that the“probability” held by each spatial element in Occupancy Gridsrefers to the probability of that element being occupied, ratherthan the probability that it should contain any particularobject. Occupancy Grids are not position-distributions overspace, rather they are a method for sensor-fusion over space.

More generally, there has been a tremendous amount of workdone on the modeling of emotions, such as [11], [14], and [19]to name but a few. Recent work on modeling emotional socialagents has come out of the Synthetic Characters group itself,with [17]. In terms of emotion modeling, the current workseeks to go beyond the simple happiness/sadness models ofemotion (e.g. [4]) to secondary emotions such as confusion orsurprise that express more subtle aspects of a character’sinternal state.

8. ACKNOWLEDGMENTSMany thanks to the Synthetic Characters Group: Geoff Beatty,Matt Berlin, Robert Burke, Marc Downie, Scott Eaton, JesseGrey, Matt Grimes, Yuri Ivanov, Michael Johnson, AileenKawabe, Ben Resner, Lilly Shirvanee, Bill Tomlinson andChris Kline.

9. REFERENCES[1] B. M. Blumberg, "Old Tricks, New Dogs: Ethology

and Interactive Characters." PhD. Thesis, MIT MediaLab, 1996.

[2] R. Burke, D. Isla, M. Downie, Y. Ivanov, and B.Blumberg, "CreatureSmarts: The Art and Architectureof a Virtual Brain," presented at The Game DevelopersConference, San Jose, CA, 2001.

[3] G. Drescher, Made-Up Minds: A ConstructivistApproach to Artificial Intelligence, MIT Press, 1991.

[4] P. Ekman, "An Argument for Basic Emotions," inCognition and Emotion, 1992.

[5] J. Funge, X. Tu, and D. Terzopoulos, "CognitiveModeling: Knowledge, Reasoning and Planning forIntelligent Characters," presented at Siggraph 99, LosAngeles, 1999.

[6] D. Isla, "The Virtual Hippocampus: Spatial CommonSense for Synthetic Creatures." Masters Thesis, MIT,2001.

[7] D. Isla, R. Burke, M. Downie, and B. Blumberg, "ALayered Brain Architecture for Synthetic Creatures,"presented at The International Joint Conference onArtificial Intelligence, Seattle, WA, 2001.

[8] C. Kline, "Observation-based Expectation Generationand Response for Behavior-based Artificial Creatures."Masters Thesis, MIT Media Lab, 1999.

[9] M. Minsky, The Society of Mind. New York, NY,Simon and Schuster, 1985.

[10] H. Moravec, "Sensor Fusion in certainty grids formobile robots," presented at Sensor Devices andSystems for Robotics, Catalonia, Spain, 1987.

[11] A. Ortony, G. Clore, and A. Collins, The CognitiveStructure of Emotions, Cambridge University Press,1988.

[12] A. Ortony and D. Partridge, "Surprisingness andExpectation Failures: What's the Difference?" presentedat Internation Joint Conference on ArtificialIntelligence, 1987.

[13] J. Piaget, The Construction of Reality in the Child.New York, Norton, 1954.

[14] R. Picard, Affective Computing, MIT Press, 1997.

[15] C. Reynolds, "Flocks, Herds, and Schools: ADistributed Behavioral Model," Computer Graphics,vol. 21, 1987.

[16] F. Thomas and O. Johnson, The Illusion of Life:Hyperion, 1995.

[17] B. Tomlinson and B. Blumberg, "Social Behavior,Emotion and Learning in a Pack of Virtual Wolves,"presented at AAAI Fall Symposium, 2001.

[18] X. Tu and D. Terzopoulos, "Artificial Fishes: Physics,Locomotion, Perception, Behavior," presented atSIGGRAPH, Orlando, FL, 1994.

[19] J. D. Velasquez, "Modeling emotions and othermotivations in synthetic agents," presented at AAAI97, 1997.