Layered dynamic probabilistic networks for spatio-temporal modelling

Layered dynamic probabilistic networks for spatio-temporalmodelling

Hung H. Bui *, Svetha Venkatesh, Geo� WestDepartment of Computer Science, Curtin University of Technology, Perth, WA 6001, Australia

Received 14 March 1999; received in revised form 7 July 1999; accepted 20 July 1999

Abstract

In applications such as tracking and surveillance in large spatial environments, there is a need for representing

dynamic and noisy data and at the same time dealing with them at di�erent levels of detail. In the spatial domain, there

has been work dealing with these two issues separately, however, there is no existing common framework for dealing

with both of them. In this paper, we propose a new representation framework called the Layered Dynamic Probabilistic

Network (LDPN), a special type of Dynamic Probabilistic Network (DPN), capable of handling uncertainty and

representing spatial data at various levels of detail. The framework is thus particularly suited to applications in wide-

area environments which are characterised by large region size, complex spatial layout and multiple sensors/cameras.

For example, a building has three levels: entry/exit to the building, entry/exit between rooms and moving within rooms.

To avoid the problem of a relatively large state space associated with a large spatial environment, the LDPN explicitly

encodes the hierarchy of connected spatial locations, making it scalable to the size of the environment being modelled.

There are three main advantages of the LDPN. First, the reduction in state space makes it suitable for dealing with wide

area surveillance involving multiple sensors. Second, it o�ers a hierarchy of intervals for indexing temporal data. Lastly,

the explicit representation of intermediate sub-goals allows for the extension of the framework to easily represent group

interactions by allowing coupling between sub-goal layers of di�erent individuals or objects. We describe an adaptation

of the likelihood sampling inference scheme for the LDPN, and illustrate its use in a hypothetical surveillance sce-

nario. Ó 1999 Elsevier Science B.V. All rights reserved.

Keywords: Dynamic probabilistic networks; Reasoning with di�erent levels of abstraction; Wide-area surveillance

1. Introduction

In many real-world problems such as tracking and surveillance, there is the need to deal with alarge spatial environment. In this type of application, the representation of the environment needsto address several issues, such as the ability to handle uncertainty in a dynamic environment, andthe ability to represent and process information about the environment at various levels of detail.

www.elsevier.com/locate/ida

Intelligent Data Analysis 3 (1999) 339±361

* Corresponding author.

E-mail addresses: [email protected] (H.H. Bui), [email protected] (S. Venkatesh), geo�@cs.curtin.edu.au (G. West)

1088-467X/99/$ - see front matter Ó 1999 Elsevier Science B.V. All rights reserved.

PII: S 1 0 8 8 - 4 6 7 X ( 9 9 ) 0 0 0 2 7 - X

In the spatial domain, there has been existing work that deals with these two requirementsindependently. To handle the uncertainty inherent in many applications, many researchers haveused Bayesian networks as the underlying knowledge representation framework [4,6,15,24].Others have emphasised the importance of a hierarchical representation of a large spatial envi-ronment [5]. However, in applications such as wide-area surveillance, there is a need to addressboth issues in a common representation framework.

In this paper, we propose a new architecture called the Layered Dynamic Probabilistic Net-work (LDPN), a special type of Dynamic Probabilistic Network (DPN) [8,10,23], as an integratedframework for representing and dealing with uncertain spatial data at various levels of detail.Given a multi-level representation and many alternatives, the state space could turn out to belarge. For example, data about a single track of one person or object can be represented by aHMM, however, the state space then would be the set of all possible positions of that object onthe ground. To avoid the problem of a relatively large state space associated with a wide spatialenvironment, the LDPN explicitly encodes the hierarchy of connected spatial locations in theenvironment. The layers in the LDPN correspond to paths through the environment at variouslevels of detail. An abstract or intermediate path in¯uences the evolution of the more detailedpath, all from the top-level to the coordinate level. The LDPN can take direct advantage of thelocal connectivity and neighbourhood of a spatial environment, making it scalable to largerspatial layouts.

The paper is structured as follows. The next section reviews various existing Bayesian networkarchitectures that have been used in the spatio-temporal domain. We then describe the repre-sentation of the LDPN, and provide an inference algorithm based on the likelihood samplingscheme. Finally, we illustrate the working of the LDPN in an experimental surveillance scenario.

2. Related background

The Bayesian network (also known as probabilistic network or belief network) [16,25] is aprobabilistic framework for representing and reasoning with uncertainty. Due to its concreteformal foundation and the availability of a variety of computational techniques, the Bayesiannetwork has been used in many di�erent types of applications where there is a need to deal withuncertain data. A Bayesian network is a directed acyclic graph (DAG) where the set of nodesrepresents the set of domain variables, and the links from the parents to a particular node rep-resent the causal dependency or in¯uence. These causal links are parameterised by the conditionalprobability distribution of the variable represented by the current node, given the parent vari-ables. Given a set of domain variables X1; . . . ;Xn, such a network structure encodes the jointprobability distribution of all the domain variables given by

Pr�x1; . . . ; xn� �Yn

i�1

Pr�xijpa�xi��;

where pa�xi� denotes the set of parents of xi.Probabilistic reasoning with Bayesian networks is termed inferencing. The basic inference

task on a Bayesian network is to calculate the conditional probability of a variable given thevalues of a set of other variables (the evidence). Many computation techniques have been

340 H.H. Bui et al. / Intelligent Data Analysis 3 (1999) 339±361

developed for this task, ranging from exact inference algorithms [9,17,22] that compute theexact value of the conditional probability required based on analytical transformation, toapproximative inference algorithms [12,14,26,29,30] that compute an approximation of therequired probability, usually based on stochastic simulation of the network. The latter has theadvantage of being applicable on all types of network structure, and sometimes is the onlytechnique available especially for large networks when exact computation becomes intractable[7].

For applications that need to deal with the temporal dynamics of the environment, the Dy-namic Probabilistic Network (DPN) [8,10,23] is a special Bayesian network architecture forrepresenting the evolution of the domain variables over time. A DPN consists of a sequence oftime-slices where each time-slice contains a set of variables representing the state of the envi-ronment at the current time. Thus, each time-slice is a Bayesian network on its own, and the samenetwork structure is replicated at each time-slice. In addition to the links within the same time-slice, the DPN encodes the temporal dynamics of the environment via the evolution model,represented by the causal links from the current time-slice to the next. The noise associated withthe observation of the environment at each time-slice can be modelled by attaching observationnodes to the variables being observed and specifying the conditional probability of the obser-vation given the actual value of the variable (the observation model). A special case of the DPNwhere, in each time-slice, there is only a single state variable and an observation node, is the well-known Hidden Markov Model (HMM) [27].

The structure of the DPN can be used to make predictions about the future state variables(predicting), or about the unobserved variables in the past (smoothing) [20]. These tasks can becast as types of inference problem. A concept that plays an important role in the inferenceproblem on the DPN is the so-called belief state [2], which is the joint probability distribution ofall the current state variables conditioned on the observations. For example, in the predictiontask, due to the Markov property of the DPN (the current time-slice renders the past and thefuture probabilistically independent), the current belief state completely summarises all infor-mation about the past observations that is needed to draw predictions regarding the futurevariables. All existing inference schemes for the DPN [2,19,21] involve maintaining and updatingeither the current belief state, or one of its approximation forms. When a new observation isreceived, the current belief state is rolled over one time-slice ahead following the evolution model,then conditioned on the new observation to obtain the updated belief state. An exact computationscheme for doing this such as in dHugin [21] could be expensive, especially when the number ofnodes joining the two adjacent time-slices is large. Meanwhile, approximative methods whichwork by maintaining only an approximation of the belief state, have been found promising [2]. In[21], the likelihood weighting sampling inference scheme [12,29] for Bayesian networks has beenextended to work with the DPN, however the algorithm has only been tested on a simple networktopology. In Section 5, we adapt this algorithm to work with our proposed more complex LDPNarchitecture.

Our LDPN can also be thought of as a type of Coupled Hidden Markov Model (CHMM) [3]where the network consists of a number of interacting Markov chains. In our LDPN, each layercan be thought of as a Markov chain acting both as the goal sequence for the layer below, and thegoal realisation sequence for the layer above. Thus each chain in our LDPN only interacts withthe chains directly below and above it. This type of interaction is more local than the other types

H.H. Bui et al. / Intelligent Data Analysis 3 (1999) 339±361 341

of interactions that have been considered [3,13,18]. However, we note that a more local type ofinteraction among the chains does not necessarily lead to more e�cient inferencing algorithms [2].

There have been a large number of applications of DPN and Bayesian networks for dealingwith noisy data in spatio-temporal domains which we do not attempt to describe in detail here.The DPN has been used in monitoring and surveillance of tra�c scenes [6,15]. The HMM is usedto reconstruct and classify vehicle trajectories from noisy observations [11]. The CHMM is used totrack human movement on the ground and some limited group behaviours such as following,meeting, etc [24]. Similar techniques can also be used for recognising and classifying humangestures [4]. In all these applications, the domains are usually locally restricted, e.g. tracking andsurveillance is carried out independently within a single room or a single ground space region.Thus, the need for dealing with di�erent levels of detail does not arise. The proposed LDPNarchitecture is capable of representing dynamic data such as human or vehicle trajectories atdi�erent levels of detail, and thus facilitates monitoring and surveillance at a larger scale over awide spatial environment and with multiple sensors.

3. Modelling a large spatial environment

For a large spatial environment, a simple representation using ground-plane coordinates doesnot su�ce. It is important to pay attention to the special landmarks within the environment andtheir spatial relationships. These are the locations that people/agents will visit, such as doorways,furniture, windows, etc. In tracking or surveillance, the set of locations is also the set of goals ordestinations that an agent might be following. In this section, we introduce a simple graph-basedhierarchical representation of a spatial environment that focuses on the connectivity relationshipsbetween the locations of interest.

Let the set of locations of interest be L, the environment is represented as a directed graphG whose nodes denote the locations, and links are established between neighbouring and con-nected locations (a directed link between two location indicates that there is a travelling path fromone location to the other without going through a third location). We call such a graph a con-nectivity graph. For example, given the o�ce ¯oor plan in Fig. 1, the set of locations of impor-tance might be the set of doors between rooms. Each door can act as an entrance or exit, and thuswe represent each door i by two nodes Ini and Outi in the connectivity graph. From In1, there arethree links to In2, In3, and Out1 respectively (Fig. 1). The link �In1; In2� represents a path from door

Fig. 1. A ¯oor layout.


1 to door 2, the link �In1; In3� represents a path from door 1 to door 3, and the link �In1;Out1�represents a circular path that enters through door 1, turns around somewhere inside the roomand exits through the same door. Note that the use of a directed graph allows us to model lo-cations that permit only one-way tra�c, such as entrance-only or exit-only doors, one-way streets,etc.

To facilitate working at di�erent levels of detail, it is also important to have a hierarchy of theset of locations. To model this, we assign to each location a number representing the level of detailthat the location belongs to. For example, we can assign all the room doors to level 1, all the ¯oorentrances (staircases) and exits to level 2, and all the main building entrances and exits to level 3.When tracking from room to room, we need to consider all these locations. However, whentracking from ¯oor to ¯oor, we need only consider locations whose level numbers are greater thanor equal to 2, etc.

Formally, a level ranking of location is a function r : L! f1; 2; . . .g. The set of all locationsof interest at level k is then Lk � fl 2L j r�l�P kg (thus we have L1 �L). At level 1, wehave a connectivity graph G1 which is identical to G. The connectivity graphs at the higherlevels can be constructed from the connectivity graph at a lower level using the followingprinciple: two nodes at level k � 1 are connected if and only if there is a direct path betweenthem at level k.

Formally, given two graphs Gk � �Vk;Ek� and Gk�1 � �Vk�1;Ek�1� where Vk�1 � Vk, we will de-®ne the condition for which Gk�1 is an abstraction of the connectivity graph Gk.

De®nition 1 (Direct path). A path in Gk is called a direct path if apart from the start and the endnodes, it does not contain any nodes that belong to Vk�1.

If there is a direct path between two nodes in Gk, they are called locally connected.

De®nition 2 (Vicinity). Given a node v 2 Vk�1. The vicinity of v, denoted by vic�v�; u is the set of allnodes in Vk that are locally connected to v:

vic�v� � fu 2 Vkj9 a direct path in Vk from u to vg:

De®nition 3 (Reduced graph). A graph Gk�1 is called a reduced graph of Gk i� Vk�1 � Vk, and�x; y� 2 Ek�1 i� x and y are locally connected in Gk.

A reduced graph Gk�1 of Gk acts like an abstraction of Gk at the higher level, since all con-nectivity relationships in Gk are summarised in Gk�1. Indeed, a path in Gk corresponds to anabstract path in Gk�1. This relationship is termed path refinement and is de®ned formally asfollows:

De®nition 4 (Path refinement). Given a path p � �u1; . . . ; un� in Gk�1 and a path q � �v1; . . . ; vm� inGk. The path q is said to be a re®nement of p i� there exists a set of indices 16m1; . . . ;mn6m suchthat:· vmi � ui;8i � 1; . . . ; n,· vmiÿ1

; vmiÿ1�1; . . . ; vmi 2 vic�ui�; 8i � 1; . . . ; n,· vmn ; vmn�1; . . . ; vm 2 vic�un�.


Thus, given a hierarchy of locations fL1; . . . ;LKg, and a connectivity graph at level 1, G1,the connectivity graphs at the higher levels are constructed so that Gk�1 is a reduced graph of Gk.The sequence of reduced graphs fG1; . . . ;GKg represents the connectivity relationship between thelocations at di�erent levels of details. Fig. 2 shows an example of the connectivity graphs at threedi�erent levels of abstraction.

4. Representation of the LDPN

The above model of the environment is purely static. In applications such as tracking andsurveillance, we have to deal with dynamic objects that evolve over time, e.g. trajectories of hu-mans or moving vehicles. Furthermore, the representation must be able to handle the uncer-tainties arising from noisy sensory data.

The modelling of a dynamic and uncertain environment is particularly suited to the DynamicProbabilistic Network (DPN), and a number of applications in the spatial domain have relied onthe DPN as their underlying representations [4,6,15,24]. However, in this work, the representationof the spatial environment is a ¯at structure, and no hierarchy of locations is present. Althoughsu�cient in small spatial environments, a ¯at dynamic model of the environment su�ers fromthree major drawbacks when applied to a wide-area environment:1. The tracks that evolve through a wide-area environment can be non-stationary at the ground

level.2. The state space is large, and thus hinders e�cient computation.3. A ¯at structure is unable to handle queries at di�erent levels of abstraction.

Fig. 2. Connectivity graphs at di�erent levels of details.


In this section, we describe the LDPN, a special type of DPN for representing tracking datawhere the hierarchy of locations is encoded explicitly. We show that LDPN is able to deal with allthe above problems associated with a wide-area environment. We ®rst illustrate the basic prin-ciples of the LDPN model through the most simple form of LDPN that has only two layers. Thefull LDPN representation can then be built by extending the two-layer LDPN following the samebasic principles.

4.1. A two-layer LDPN

In this section, we present a simple LDPN that has only two layers. Our aim is to use this two-layer LDPN to discuss two basic ideas in the formulation of the full LDPN: (1) the translation ofa sequence of sub-goals to a sequence of stationary intervals at the state level, and (2) the utili-sation of local connectivity between di�erent states and their hierarchy.

4.1.1. From sub-goals to stationary chains of statesAn issue in representing the trajectory data of movement in a wide spatial environment is the

problem of non-stationarity. For example, consider the problem of representing the trajectory of aperson moving from location A to location B and then continuing to location C (Fig. 3). We cande®ne the state space as the set of coordinates on the ground-plane, and represent this trajectoryas a chain of points through the state space, where the current point in¯uences the next point via atransition probability table. Since the transition probabilities during the AB interval are di�erentfrom the transition probabilities during the BC interval, the entire ABC chain is clearly non-stationary. However, it can be thought of as a composite of two stationary chains, one from A toB, and the other one from B to C. The non-stationarity problem means that simple HMMs arenot a suitable representation, since the HMM assumes a stationary Markov chain in the hiddenlayer. Intuitively, a suitable model for the trajectory in the above example would involve a`concatenation' of two (stationary) HMMs together.

To overcome the non-stationary problem, it is necessary to maintain not only the currentposition of the agent on the ground-plane, but also its current goal or destination as well. Thus,our two-layer LDPN is an extension of the HMM where there are two hidden layers: the statelayer representing the sequence of ground-plane positions, and the goal layer representing thesequence of intermediate goals (destinations) that the agent follows. The observation layer rep-resents noisy observation of the state layer as in the HMM.

If an HMM represents noisy observation of a stationary Markov chain, our two-layer LDPNrepresents how a sequence of goals or a plan is realised in a state sequence, and how noisy

Fig. 3. Trajectory between rooms.


observation of the state sequence is obtained. Formally, given a plan an g � �g1; . . . ; gn� repre-senting an ordered sequence of goals that the agent needs to complete, the model will specify theconditional probability that a particular observation sequence ~o � �o�1�; . . . ; o�T �� is observed overtime: Pr�~o j g�.

While executing a plan, the current goal dictates the current action of the agent, which in turna�ects the current state of the environment. We encode this causal dependency by the causal linksfrom the current goal and the current state to the next state. These causal links constitute theSEM, or State Evolution Model. This type of in¯uence coming from the goal layer down to thestate layer is termed evolution influence, and the associated causal links are termed evolution links.Formally, an SEM is a set of parameters r�s; s0; g� representing the conditional probability thatthe current state is s, given the previous state is s0 and the current goal is g:

r�s; s0; g� � Pr�s�t�1� � s j s�t� � s0; g�t�1� � g�: �1�In executing the plan, the agent also needs to monitor the current goal and replace it with the nextgoal when appropriate. The model for this is called the GPM or Goal Persistence Model. Weassume that a goal persists until it is achieved by events at the state level. In other words, thecurrent state determines whether or not the current goal is achieved. This type of in¯uence comingfrom the state layer up to the goal layer is termed persistence influence, and the associated causallinks are termed persistence links. Formally, let a�t� be a binary variable representing whether thecurrent goal g�t� is achieved. The GPM is the set of parameters c�g; s� which represents theprobability that the current goal g is achieved by the current state s:

c�g; s� � Pr�a�t� � true j g�t� � g; s�t� � s�: �2�If a�t� is true, we assume that in the next time instant, the current goal will be replaced by the goalsucceeding it in the plan g, otherwise, the current goal persists until the next time instant:

Pr�g�t�1� � g j a�t� � false; g�t� � g� � 1; �3�Pr�g�t�1� � succ�g� j a�t� � true; g�t� � g� � 1: �4�

Finally, the current state a�ects how its noisy observation is formed. This is modelled by theObservation Model (OM), which is a set of parameters x�o; s� representing the conditionalprobability that an observation o is obtained given the real state is s:

x�o; s� � Pr�o�t� � o j s�t� � s�: �5�A two-layer LDPN is de®ned in terms of its SEM, GPM and OM, together with an initial dis-tribution for the initial state as follows:

De®nition 5. A two-layer LDPN is a tuple k � hr�0�;r; c;xi such that:· r�0� is the distribution of the initial state s�0�:

r�0��s� � Pr�s�0� � s�:· r is the state evolution model:

r�s; s0; g� � Pr�s�t�1� � s j s�t� � s0; g�t�1� � g�:


· c is the goal persistence model:

c�g; s� � Pr�a�t� � true j g�t� � g; s�t� � s�:· x is the observation model:

x�o; s� � Pr�o�t� � o j s�t� � s�:

The essence of this model is as follows. The agent starts with the ®rst goal g1 in the plan g(g�0� � g1), its initial state s�0� is chosen according to the initial state distribution r�0�. Then, fort � 0 to T ÿ 1:· An observation o�t� is generated from current state s�t�. This models the observation of the

agent's state by an outsider plus associated observation noise.· The next goal g�t�1� is generated from g�t� and s�t�1�. This is done by ®rst examining the current

goal achievement status a�t�, and replacing the current goal with the next goal in the plan ifa�t� � true.

· The next state s�t�1� is generated from s�t� and g�t�1�. This models the behaviour of the agenttrying to achieve g�t� while the current state is s�t� plus associated ``execution'' noise.The full Bayesian network corresponding to a two-layer LDPN k is shown in Fig. 4. At each

time slice t, there are four nodes corresponding to the four variables g�t�, a�t�, s�t� and o�t� whichrepresent the current goal, the achievement of the current goal, the current state, and the ob-servation of the current state respectively. The links encode the ``parent-child'' relationship whichrepresents the probabilistic dependencies of these variables. The node s�t�1� has two parents s�t�

and g�t�1�, representing the causal links from the current goal and state to the next state. Theselinks are parameterised by the SEM r (Eq. (1)). The node a�t�1� has two parents g�t� and s�t�.These links are parameterised by the GPM c (Eq. (2)). The node g�t�1� has two parents g�t� anda�t�, and these links are parameterised by the conditional probabilities in Eqs. (3) and (4). Finally,the node o�t� has a single parent s�t�, and this link is parameterised by the OM x (Eq. (5)). Thenetwork encodes the following factorisation of the joint probability distribution of all thevariables:

Fig. 4. The two-layer LDPN.


Pr o�0�; s�0�; a�0�; g�0�; . . . ; o�T �; s�T �; a�T �; g�T � j g� ��

YT

t�0

Pr�o�t� j s�t��Pr�a�t� j g�t�; s�t��" # YT

t�1

Pr�s�t� j s�tÿ1�; g�t��Pr�g�t� j a�tÿ1�; g�tÿ1��" #

Pr s�0�ÿ �

�6�Alternatively, the model can be thought of as a concatenation of multiple HMMs, each corre-sponding to the interval of achieving a goal in the plan g. Within each interval, since the currentgoal remains the same, the transition probability matrix of the evolution of states is stationary.Each interval ends when the corresponding goal has been reached. For example, Fig. 5 illustratesthe process of executing a plan consisting of ®ve sub-goals: g1; g2; . . . ; g5. The entire time-windowcan be divided into ®ve intervals, where interval i corresponds to the period of achieving sub-goali. Each interval can be represented by a HMM, and the entire process is a concatenation of 5di�erent HMMs, where each HMM ends when the corresponding sub-goal has been achieved.

4.1.2. Exploiting local connectivityModelling a wide spatial environment typically needs a relatively large state space to encode all

the possible positions of the agent on the ground-plane. This can lead to very demanding com-putation. However, the special characteristics of the spatial environment mean that connectivityamong the states is low and locally restricted. We term this property state locality. In addition, thegoal space is simply the set of possible interesting destinations, and is a subset of the state spaceitself. It is reasonable to assume that two consecutive goals in a plan must be adjacent in theconnectivity graph of the goal space. We term this property goal locality. Furthermore, in thecourse of following the plan, the activities involved in achieving a particular goal mostly occurwithin the region in the state space surrounding that goal. We term this property goal-state lo-cality. In dealing with a wide spatial environment, it is desirable to exploit these locality con-straints so that computation can be carried out more e�ciently.

To model the local connectivity, a state space will be modelled by a connectivity graph Gs. Weassume that a state can only evolve to one of the neighbouring states. Thus, the transitionprobability matrix only contains positive entries corresponding to the edges of the graph, and iszero everywhere else: r�s; s0; g� > 0 i� �s0; s� is an edge of the state space connectivity graph Gs.

Similarly, the goal space is the set of destinations, or interesting locations in the environments,and is modelled as a subset of the state space together with its connectivity graph Gg. A link inGg represents the existence of a direct path between the corresponding locations. Thus, Gg is areduced graph of Gs.

Fig. 5. Intervals of achieving goals.


The three locality constraints can then be translated into the following constraints in our two-layer LDPN model:

Assumption 1 (State locality). A state sequence s must be a path in Gs. In other words, a state is onlyallowed to evolve to one of its neighbouring states in Gs.

Assumption 2 (Goal locality). A plan g must be a path in Gg. In other words, adjacent goals in theplan g must also be adjacent in Gg.

Assumption 3 (Goal-state locality). If a state s is within the vicinity of a goal g, then g can onlyin¯uence the evolution of s to a neighbouring state also within vic�g�. Thus, if s 2 vic�g�, thenr�s0; s; g� � 0 for all s0 62 vic�g�.

Assumption 3 drastically reduces the size of the representation of the two-layer LDPN. Incomparison with the general model, we only need to specify the state evolution parametersr within the vicinity of each goal. Thus, the size of the representation of the SEM parameter isreduced from j G jj S j2 to j G jj Vic j2 where j Vic j is the size of the vicinity of a goal in G. In amodel of a wide spatial environment, the size of the state space j S j can become very large whilethe size of the vicinity of a goal j Vic j remains relatively small. Thus, the constrained modelprovides a signi®cant reduction in terms of the size of the parameters, and is much more scalableto wide-area environments.

Furthermore, since the goals are destinations and are part of the state space, we assume thatthe goal persistence model is such that a goal g is achieved if and only if it has been reached in thestate layer (i.e. an agent has reached a destination if and only if its current position is at thedestination). Thus, c�g; s� � 1 if s � g, and c�g; s� � 0 if s 6� g.

Under these assumptions, the following proposition holds.

Proposition 1. A plan g can only give rise to a sequence of states s which is a refinement of g.

Pr�s j g� > 0) s refines g:

To prove this proposition, we can divide the sequence of states s into intervals of achieving eachsub-goal in the plan g as we have done before (see Fig. 5). From Assumption 3, the state chainwithin each interval lies entirely within the vicinity of the corresponding sub-goal. Furthermore,each of these chains must end at the state that achieves the sub-goal of the corresponding interval(see Fig. 6). Thus, s is a re®nement of g by De®nition 4.

4.2. The general LDPN

In the two-layer LDPN, we assume that a goal sequence or plan is given and concentrate onhow this sequence of goals is re®ned into a sequence of states at the state level. The same idea canbe applied in the generation of the plan itself, from for instance, a higher level plan. We thus couldhave more than one layers of goal and sub-goals at di�erent levels of abstraction, paralleling thehierarchy of locations in a spatial environment as has been described previously.

In this section, we describe the full LDPN where we allow for more than one goal layers. Weuse the same basic principle as the two-layer LDPN to model the re®nement of a goal sequence at


a higher level, to a goal sequence at a lower level, and to a state sequence where observations canbe made.

4.2.1. Full Bayesian network representationSimilar to the two-layer LDPN, the full LDPN is a dynamic Bayesian network where the set of

variables at time-slice t is c�t� � fo�t�; g�t�0 ; g�t�1 ; . . . ; g�t�K ; a

�t�1 ; . . . ; a�t�K ; g

�t�topg where g�t�0 denotes the

current state, o�t� denotes the observation of the current state; for k P 1, g�t�k denotes the currentgoal at level k, a�t�k denotes the achievement of the goal g�t�k ; and g�t�top denotes the top-level goal. Weassume that the top-level goal remains unchanged over time, and thus the goal sequence at the toplevel collapses to just one variable: for all t, g�t�top � gtop.

In spirit, the full LDPN model retains much of what has been discussed before. A sequence ofgoal at level k receives two types of causal in¯uence from other variables: (1) evolution in¯uence,coming from the goal sequence at the higher level, and (2) persistence in¯uence, coming from thegoal sequence at the lower level. The persistence in¯uence determines if the current goal (at levelk) is achieved, and if that is the case, the evolution in¯uence determines what the next goal will be.

Formally, the evolution links at level k are parameterised by the evolution model rk:rk�gk; g0k; gk�1� is the conditional probability of the current level-k goal is gk given that the previouslevel-k goal is g0k, the current level-�k � 1� goal is gk�1, and the goal g0k has been achieved.

rk�gk; g0k; gk�1� � Pr�g�t�1�k � gk j g�t�k � g0k; g

�t�1�k�1 � gk�1; a

�t�k � true�: �7�

When k � 0, r0 is identical to the state evolution model (Eq. (1)), that is:

r0�g0; g00; g1� � Pr�g�t�1�0 � g0 j g�t�0 � g00; g

�t�1�1 � g1�:

Thus, the form of Eq. (7) can also be used for our convenience when k � 0 if we let a�t�0 � true.When k � K, k � 1 becomes the top level goal. Thus we have:

rK�gK ; g0K ; g� � Pr�g�t�1�K � gK j g�t�K � g0K ; gtop � g; a�t�K � true�:

The persistence links at level k are parameterised by the persistence model ck: ck�gk; gkÿ1� is theconditional probability that the current level-k goal is achieved given the current level-k goal is gk,the current level-�k ÿ 1� goal is gkÿ1, and gkÿ1 has been achieved.

ck�gk; gkÿ1� � Pr�a�t�k � true j g�t�k � gk; g�t�kÿ1 � gkÿ1; a

�t�kÿ1 � true�: �8�

The LDPN can be formally de®ned in terms of the evolution and persistence parameters asfollows:

Fig. 6. An interval of achieving a goal.


De®nition 6 (LDPN). An LDPN is a tuple k � h�r�0�k �; �rk�; �ck�;xi where:

· r�0�k is the prior distribution of the initial goal at level-k g�0�k :

r�0�k �gk� � Pr�g�0�k � gk�:· rk is the evolution model for the level-k goals.· ck is the goal persistence model for the level-k goals.· x is the observation model:

x�o; g0� � Pr�o�t� � o j g�t�0 � g0�:The network structure for the LDPN is shown in Fig. 7. From the network structure, we note twoimportant types of conditional independence. Time-wise, given values of the sub-goals at all levelsat some time t, all the other variables in the past are probabilistically independent of all othervariables in the future. The set of sub-goals at all levels at time t then acts like a slice in a DPNmodel. Layer-wise, given the values of all the sub-goals at a layer k, all the variables at the lowerlayers are probabilistically independent of the variables at the higher layers.

Similarly to the two-layer LDPN, the full LDPN can also be visualised in terms of the intervalsof achieving each sub-goal in the sub-goal hierarchy. Fig. 8 depicts the visualisation of the evo-lution of goal sequences at three levels. At the top-level, we have only one interval of the same top-level goal throughout. This top-level goal is re®ned into two intervals level 2, corresponding to thegoal sequence �g21; g22�. Each of these intervals in turn, is divided into smaller intervals corre-sponding to the persistence duration of goals at level 1.

Fig. 7. The hierarchical model.


Probabilistically, the process can be viewed as follows. We start with a single variable at the toplevel. This variable is re®ned into a stationary Markov chain at level-K. Then, each variable in thisnew Markov chain is re®ned into a stationary chain at level K ÿ 1, and the re®nement process isrepeated recursively until the last goal level. Thus looking at each level separately, what we have isa concatenation of some stationary Markov Chains, and the lower level is a re®nement of thechains at the higher level.

4.2.2. Representation of local connectivitySimilar to the two-layer LDPN, we can have a more compact LDPN model by exploiting the

local connectivity within the state space representing the environment. We model the connectivityin the state space and at di�erent goal levels by a sequence of graphs �G0; . . . ;GK� where G0 is theconnectivity graph of the state space, Gk is the connectivity graph of the level-k goal space, andGk�1 is a reduced graph of Gk. For example, G0 can represent the connectivity graph of the en-vironment at the coordinate level, G1 the connectivity graph between doors, G2 the connectivitygraph between building entrances and exists, etc. Given this connectivity model, we introduce theadditional locality constraints similar to those in a two-layer LDPN:1. A sequence of sub-goals gk must be a path in Gk. In other words, a goal is only allowed to

evolve to one if its neighbouring goals in Gk.2. If a level-k goal gk is within the vicinity of a level-�k � 1� goal gk�1, then gk�1 can only in¯uence

the evolution of gk to a neighbouring goal also within vic�gk�1�. Thus, if gk 2 vic�gk�1�, thenrk�g0k; gk; gk�1� � 0 for all g0k 62 vic�gk�1�.

3. The goal persistence model is such that a sub-goal gk ends if and only if it has been reached inthe level-�k ÿ 1�. Thus, ck�gk; gkÿ1� � 1 if gk � gkÿ1, and ck�gk; gkÿ1� � 0 if gk 6� gkÿ1.Given the above constraints, the following proposition holds:

Proposition 2. A sequence of sub-goals at level-k gk can only give rise to a sequence of level-�k ÿ 1� sub-goals gkÿ1 which is a refinement of gk.

Pr�gkÿ1 j gk� > 0) gkÿ1 refines gk:

The visualisation of this constrained LDPN is as follows. First the top-level goal is re®ned into astationary chain of level-K goals that ends in the top-level goal itself. Then, each new goal isre®ned into a stationary chain at the lower level that ends in the current goal itself. This process isrepeated down to the last goal level, and to the state level. The result is a hierarchy of Markovchains. Since the transition matrix needs only be speci®ed between goals that lie in the same

Fig. 8. The interval view of the hierarchical structure.


vicinity, the size of the representation is still manageable even when a large state space (e.g. re-sulting from a wide spatial environment) is considered.

5. Monitoring problems with LDPN

In tracking, surveillance, and other types of monitoring applications, an important problem isto predict the future evolution of the tracked objects, given the observations made in the past. Inprobabilistic terms, this problem can be translated to the computation of the conditional prob-ability of various future variables, given the sequence of observations up to the current time. Inthe case of the LDPN, the future variables whose values we might want to examine could be thetop-level goal, or the current and future goal node at various abstraction levels, depending on thenature of the query. For example, if we are interested in the agent's ®nal destination, we need toexamine the top-level goal; if we are interested in the nearest location that the agent is heading to,we need to examine the current goal at level 1, etc.

Computing these conditional probabilities for our LDPN can be cast as the prediction problemin Dynamic Probabilistic Networks. The main approach to solve this problem is to maintain andupdate a current belief state, i.e. the joint probability distribution of all the variables of the currenttime-slice given the history of observation. Due to the Markov property of DPN (each time slicerenders the past and future independent), such a current belief state summarises all the infor-mation contained in the past observations, and is su�cient to draw prediction about variables infuture time slices. When a new observation arrives, the current belief state is updated by rolling itone time-slice ahead and then conditioning on the new observation. Although this procedure issimple, maintaining an exact belief state becomes very expensive when the dimensionality of thebelief state is large [2,19]. For our LDPN, we need to maintain the joint probability distribution ofall the current goal and state variables, given the current sequence of observations. Thus, thedimension of the belief state is K � 2 (we have K intermediate goal variables, one state variable,and one top-level goal variable). Thus, the exact computation method is infeasible in LDPN withmany layers.

When exact computation is infeasible, approximative or sampling methods are often foundvaluable [7]. One of the sampling schemes, likelihood weighting [12,29] has been extended to workwith DPN [19]. In the remainder of this section, we describe a similar algorithm to work with theLDPN model.

5.1. Likelihood weighting sampling

The basic likelihood weighting method maintains an approximate belief state represented as aset of samples of the current time-slice together with their weights. To move to the next time-slice,for each sample, a new sample value at the next time-slice is generated according to the transitionprobabilities. The likelihood of the next observation given the new sample value is then used toupdate the weight of the sample. The procedure is given in Algorithm 1.

Algorithm 1 (Likelihood weighting, from [19]).Algorithm LW-samplingBegin

for i � 1 . . . N


wi � 1for t � 0 . . . T

Instantiate o�t�

for i � 1 . . . Nsamplei � GenerateNextSample(samplei,t)wi � wi � Likelihood�o�t� j samplei�

Normalise wi

End

One problem of the basic likelihood weighting method is that the observations are not used inthe process of generating new samples (they are only used to update the weights of the generatedsamples). Because of this, samples generated for a network with a weak evolution model willdiverge and quickly become irrelevant. To avoid this problem, a modi®cation of the basic algo-rithm called Evidence Reversal (ER) has been proposed [12,19]. Before the sampling is carried out,ER reverses the direction of the link from state to observation, so that the sample of a state can beconditioned on its observation. In the next section, we describe an adaptation of this procedurefor the LDPN.

5.2. LW sampling with ER on LDPN

To apply ER on the LDPN, before sampling at time t, we use the arc-reversal technique forBayesian networks [28] to reverse the link from the current state node g�t�0 to o�t� so that after thereversal, o�t� becomes a parent of g�t�0 . The side e�ect of such an arc-reversal procedure is that o�t�

now inherits all the parents of g�t�0 which are g�tÿ1�0 and g�t�1 . This is illustrated in Fig. 9 with the

dotted lines representing the new links obtained after the procedure.The parameters on the new established links are computed from the old parameters as follows:

Fig. 9. Evidence reversal on the LDPN.


Pr�o�t� j g�tÿ1�0 ; g�t�1 � �

Xg�t�

02vic�g�t�

1�Pr�o�t� j g�t�0 �Pr�g�t�0 j g�tÿ1�

0 ; g�t�1 �; �9�

Pr�g�t�0 j g�tÿ1�0 ; g�t�1 ; o

�t�� / Pr�o�t� j g�t�0 �Pr�g�t�0 j g�tÿ1�0 ; g�t�1 �: �10�

After performing the evidence reversal step, we are ready to generate a new value at time-slice tfor the current sample. In the LDPN, a sample is an instantiation of the current state and all thegoal variables �g�t�0 ; g

�t�1 ; . . . ; g�t�K ; gtop�, and thus is a tuple �g0; g1; . . . ; gK ; g� where g0 is a state

sample, gk is a sample of the current goal at level k, and g is a sample of the top-level goal. Togenerate a sample at the next time-slice, we assume that the current state and goals are instan-tiated to their current sampled values. From the current state and goals, we then determine thestatus of the achievement of these goals a�t�k . Finally, from �g�t�0 ; g

�t�1 ; . . . ; g�t�K ; gtop� and �a�t�1 ; . . . a�t�K �,

the new sample is generated for the next time-slice according to the conditional probabilities of theevolution model. In doing this, we note that if a goal at level k has not been achieved (a�t�k � false)then all the goals above it have also not been achieved (a�t�l � false for all l P k), and thus for alll P k, the current goal at level l persists to the next time-slice: g�t�1�

l � g�t�l . Thus, we only need togenerate random samples for all the goals up to the ®rst goal that has not been achieved. Togenerate a new value for the state variable g�t�0 , ®rst we need to perform the ER procedure for thecurrent time-slice t. Then, the new value for g�t�0 is sampled according to the distribution obtainedin (10). The likelihood value obtained in (9) is then used to update the weight of the currentsample. This procedure is given in Algorithm 2.

Algorithm 2. Generating the next sample valueAlgorithm GenerateNextSample(CurrentSample, NewSample, t)Input CurrentSample � �g0; g1; . . . ; gK ; g�Begin

Compute largest k such that a�t�k � trueFor l � k � 1; . . . ;K

g0l � gl

For l � k; . . . ; 1Sample g0l from the distribution given by the parameters rl�g0l; gl; g0l�1�

Sample g00 from the distribution obtained from (10) after ER procedureNewSample � �g00; g01; . . . ; g0K ; g�

End

Finally, the overall LW-ER procedure on the LDPN is as follows. We ®rst generate N samplesfor the initial time-slice according to the initial distributions. Then, for each new time-slice t, thesample population and their weights are updated as in Algorithm 3.

Algorithm 3. LW-ER on LDPNAlgorithm LW-ER-LDPNBegin

Instantiate the new observation o�t�.Perform evidence reversal at time-slice t.


For i � 1; . . . ;NGenerate NewSamplei � �g00; . . . ; g0K ; g� from Samplei � �g0; . . . ; gK ; g�Compute likelihood � Pr�o�t� j g�tÿ1�

0 � g0; g�t�1 � g01� (from (9))

weighti � weighti � likelihoodEnd

The current sample population and their weights then provide the approximation to theprobability distribution of all the current goal variables. For example, to estimate the distributionof the top-level goal gtop, we only have to look at all the values of gtop present in the samplepopulation and normalise their weights.

6. Experiments and results

In this section, we illustrate the use of the LDPN model via a simulated tracking problem in acomplex spatial environment. The task involves tracking the movement of an object througha building consisting of 8 connected rooms (Fig. 10). Each room is modelled by a set of cells on a5� 5 grid, and two adjacent rooms are connected via a door in the centre of their common edge.The four entrances to the building are labelled north (N), west (W), south (S) and east (E). Inaddition, the door in the centre of the building (C) acts like an entrance between the building'snorth wing and south wing.

The hierarchy of this spatial environment is constructed as follows. The state level (level 0)consists of the set of connected cells of all the rooms. Level 1 consists of the set of all doors andentrances. Level 2 consists of the set of building entrances and the wing entrance (N, W, S, E, C).The top level (level 3) consists of only the four entrances to the building. The evolution parametersat the state level specify the transition probabilities between cells within a single room, given theroom door that the agent is heading to. Similarly, at the higher levels, the evolution parametersspecify the transition probabilities between doors, given the main building exits that the agent isheading to. Note that with a ¯at structure, we would have to specify the transition probabilities

Fig. 10. The environment.


between cells in the entire building, given the main building exits the agent is heading to, resultingin many redundancies in the model parameters and an unnecessarily large state space.

To generate the hypothetical data used in the tracking task, ®rst, the parameters of the LDPNare selected manually. Then, the LDPN structure is used to generate a random sequence of cells atthe bottom level of the hierarchy to simulate an agent's path. Fig. 11 shows an example of agenerated track entering the building via the West entrance and exiting the building via the Eastentrance. The number shown next to a position on the track representing the time when the agentis at that position. Finally, an observation model speci®ed as a distribution (Fig. 12) is used toperturb each point in the generated track over a 3� 3 cell neighbourhood to obtain the ®nalobservation data points.

With this set of generated observation points as input data, we implement and run the LDPNinference algorithm described in the previous section to answer queries about the tracked object.At each time-slice, we look at three queries at di�erent levels of abstraction: (1) which main en-trance the object is heading to, (2) which room the object is currently in, and (3) which nearestdoor/entrance the object is currently heading to. The scope of the ®rst query is the entire envi-ronment, whereas the scopes of the other two queries are limited to the immediate surroundings ofthe current position of the tracked object.

At each time-slice, the answer to the ®rst query is a probability distribution on the set of fourbuilding exits N, S, E, W; this is obtained by computing the conditional probability of the top-level goal node in the LDPN structure given the past sequence of observations. The answer to thesecond query is a probability distribution on the set of rooms 0; 1; . . . ; 7; this is obtained bycomputing the conditional distribution of the current coordinate-level node in the LDPN. Theanswer to the third query is a probability distribution on the set of all doors and entrances; this isobtained from the conditional probability of the level-1 goal node. The probabilities in the queryanswer can be interpreted as the degrees of certainty, or they can be used as inputs to an externalmodule for optimal decision making.

Fig. 11. Synthetic track through the building.


We ran the stochastic inference algorithm with the sample population size of 10000 on the sameinput data for 100 times to get the mean and standard deviation of the output probabilities. Theprobabilities obtained from the ®rst query, averaged over the 100 runs, are plotted in Fig. 13. Wenote some interesting behaviours of the predicted probabilities as the track evolves. The fourprobabilities are initially the same, with Pr�W � slightly higher since W is the closest exit. As thetrack moves away from W, its probability however quickly diminishes. As the track crosses thedoor at time 7, Pr�S� becomes smaller, and diminishes after the track has crossed the wing en-trance at time 16. At around time 25, Pr�N� becomes dominant since the track is heading upward,however, when it turns back, Pr�E� becomes dominant instead. Towards the end, Pr�E� increasesdramatically after the track has crossed the door at time 50 and moves closer to the East exit. Fig.14 shows the standard deviation of the output probabilities for the East exit. At the beginning, thestandard deviation is arti®cially small since the probabilities are initialised to 0:25. It then growsand stabilises to within an acceptable interval and diminishes as the ®nal destination becomescertain. The results obtained from the second and third level queries are plotted in Fig. 15 and Fig.16. Note that since there are many rooms and doors, we only plot the probabilities for room 6 androom 7 in Fig. 15, and the probabilities for the three doors exiting from room 7 (front, right, back)in Fig. 16. Fig. 15 shows that from around time 15 to time 50, the object is moving from room 7 toroom 6, and then back to room 7. Fig. 16 shows the probabilities of the di�erent doors that the

Fig. 12. Observation model.

Fig. 13. Probabilities of di�erent exits over time.


object uses to exit from room 7 at the two intervals: from time 15 to time 25, and from time 35 totime 50.

7. Conclusion

In this paper, we have presented a new Bayesian network architecture, the LDPN, for repre-senting uncertain data in spatial domains and dealing with them at di�erent levels of detail. Toachieve this, the LDPN explicitly encodes the hierarchy of the spatial locations in the environ-ment, enabling it to take advantage of the inherent neighbourhood and hierarchical structure ofthe set of spatial locations. By dividing the environment into regions corresponding to vicinities of

Fig. 14. Mean and standard deviation of the probabilities for East exit.

Fig. 15. Probabilities that the agent is at Room 6 and Room 7.


the locations at the next higher level, data such as trajectories can be assumed to be stationary,given the goal of reaching the location at the higher level. Furthermore, the transition proba-bilities need only be speci®ed at the vicinity surrounding each destination, thus making the size ofthe transition probability tables relatively constant and the model scalable to wide-area envi-ronments. For the inferencing task on the new network architecture, we have adapted the sam-pling inference scheme for DPN [19] to work with the LDPN. The algorithm has been tested onsynthetic data in a surveillance scenario.

In future work, we plan to investigate the possibility of learning and automatically acquiringthe parameters of the LDPN from a set of training data. If the training data also contains valuesfor the higher level goals, the evolution model of each goal can be learn separately using thestandard HMM parameter re-estimation procedure. However, if the higher level goals are com-pletely hidden, more complex methods for learning with hidden variables [1] will be needed.Another research direction is to use coupled LDPNs for modelling complex group behaviours.Since the intermediate sub-goals are explicitly represented in the LDPN, a group behaviour can bespeci®ed by coupling some of these individual goals together. Finally since the LDPN is partic-ularly suited for modelling a large spatial environment, we are currently using the LDPN as theunderlying framework in a wide-area surveillance system involving multiple cameras in multipleconnected rooms and areas.

Acknowledgements

This research reported here was supported by an ARC grant from the Australian ResearchCouncil. We were bene®tted from the stimulating correspondents with Prof. Terry Caelli at theDepartment of Psychology, The University of Alberta, Canada. Also, thanks to the anonymousreviewers for their helpful comments.

Fig. 16. Probabilities that the agent is exiting Room 7 via the di�erent doors.


References

[1] J. Binder, D. Koller, S. Russell, K. Kanazawa, Adaptive probabilistic networks with hidden variables, Machine Learning 29

(1997) 213±244.

[2] X. Boyen, D. Koller, Tractable inference for complex stochastic processes, in: Proceedings of the 14th Annual Conference on

Uncertainty in Arti®cial Intelligence, 1998.

[3] M. Brand, Coupled hidden Markov models for modeling interacting processes, Tech. Report 405, Media Lab, MIT, Cambridge.

[4] M. Brand, N. Oliver, A. Pentland, Coupled hidden Markov models for complex action recognition, in: IEEE Computer Society

Conference on Computer Vision and Pattern Recognition, 1997.

[5] Hung H. Bui, S. Venkatesh, D. Kieronska, Constructing hierarchical abstractions for qualitative representations of space and

time, Australian Journal of Intelligent Information Systems (1995) 36±45.

[6] H. Buxton, S. Gong, Advanced visual surveillance using Bayesian networks, in: Proceedings of the IEEE International Conference

on Computer Vision, 1995.

[7] E. Castillo, J.M. Guti�errez, A.S. Hadi, Expert systems and probabilistic network models, Springer, Berlin, 1997.

[8] P. Dagum, A. Galper, E. Horvitz, Dynamic network models for forecasting, in: Proceedings of the Eighth Annual Conference on

Uncertainty in Arti®cial Intelligence, 1992, pp. 41±48.

[9] B. D'Ambrosio, Incremental probabilistic inference, in: Proceedings of the Ninth Annual Conference on Uncertainty in Arti®cial

Intelligence, 1993, pp. 301±308.

[10] T. Dean, K. Kanazawa, A model for reasoning about persistence and causation, Computational Intelligence 5 (3) (1989) 142±150.

[11] R. Fraile, S.J. Maybank, Vehicle trajectory approximation and classi®cation, in: British Machine Vision Conference, 1998.

[12] R. Fung, K.C. Chang, Weighting and integrating evidence for stochastic simulation in Bayesian networks, in: Proceedings of the

Fifth Conference on Uncertainty in Arti®cial Intelligence, 1989.

[13] Z. Ghahramani, M.I. Jordan, Factorial hidden Markov models, Machine Learning 29 (1997) 245±273.

[14] M. Henrion, Propagating uncertainty in Bayesian networks by probabilistic logic sampling, in: J. Lemmer, L. Kanal (Eds.),

Uncertainty in Arti®cial Intelligence, North-Holland, Amsterdam, 1986.

[15] T. Huang, D. Koller, J. Malik, G. Ogasawara, B. Rao, S. Russell, J. Weber, Automatic symbolic tra�c scene analysis using belief

networks, in: Proceedings of the National Conference on Arti®cial Intelligence (AAAI-94), 1994.

[16] F. Jensen, An Introduction to Bayesian Networks, Springer, Berlin, 1996.

[17] F. Jensen, S. Lauritzen, K. Olesen, Bayesian updating in recursive graphical models by local computations, Computational

Statistics Quarterly 4 (1990) 269±282.

[18] M.I. Jordan, Z. Ghahramani, L.K. Saul, Hidden Markov decision trees, in: M.C. Mozer, M.I. Jordan, T. Petsche (Eds.),

Advances in Neural Information Processing Systems 9, MIT Press, Cambridge, MA, 1997.

[19] K. Kanazawa, D. Koller, S. Russell, Stochastic simulation algorithms for dynamic probabilistic networks, in: Proceedings of the

11th Annual Conference on Uncertainty in Arti®cial Intelligence, 1995, pp. 346±351.

[20] U. Kjaerul�, A computational scheme for reasoning in dynamic probabilistic networks, in: Proceedings of the Eighth Annual

Conference on Uncertainty in Arti®cial Intelligence, 1992, pp. 121±129.

[21] U. Kjñrul�, dHugin: A computational system for dynamic time-sliced Bayesian networks, International Journal of Forecasting 11

(1995) 89±111.

[22] S. Lauritzen, D. Spiegelhalter, Local computations with probabilities on graphical structures and their application to expert

systems, Journal of the Royal Statistical Society B 50 (1988) 157±224.

[23] A.E. Nicholson, J.M. Brady, The data association problem when monitoring robot vehicles using dynamic belief networks, in:

Proceedings of the 10th European Conference on Arti®cial Intelligence, 1992, pp. 689±693.

[24] N. Oliver, B. Rosario, A. Pentland, A Bayesian computer vision system for modeling human interactions, in: Proceedings ICVS-

99, 1999.

[25] J. Pearl, Probabilitic Reasoning in Intelligent Systems: Networks of Plausible Inference, Morgan Kaufmann, San Mateo, CA,

1988.

[26] J. Pearl, Evidential reasoning using stochastic simulation of causal models, Arti®cial Intelligence 32 (1987) 245±257.

[27] Lawrence R. Rabiner, A tutorial on Hidden Markov Models and selected applications in speech recognition, Proceedings of the

IEEE 77 (2) (1989) 257±286.

[28] R. Shachter, Evaluating in¯uence diagrams, Operations Research 34 (1986) 871±882.

[29] R.D. Shachter, M.A. Peot, Simulation approaches to general probabilistic inference on belief networks, in: Proceedings of the

Fifth Conference on Uncertainty in Arti®cial Intelligence, 1989.

[30] J. York, Use of Gibbs sampler in expert systems, Arti®cial Intelligence 56 (1992) 115±130.


Documents

Layered dynamic probabilistic networks for spatio-temporal modelling