TRACTABLE PARTICLE FILTERS FOR ROBOT FAULT DIAGNOSIS · Vandi Verma Robotics Institute Carnegie Mellon University Pittsburgh, PA 15213 Oct 25, 2004 Submitted in partial fullment of

TRACTABLE PARTICLE FILTERS FOR ROBOTFAULT DIAGNOSIS

Vandi Verma

Robotics InstituteCarnegie Mellon University

Pittsburgh, PA 15213

Oct 25, 2004

Submitted in partial fulfilment ofthe requirements for the degree of

Doctor of Philosophy

Thesis Committee:Reid Simmons, Chair

Richard Dearden, NASA Ames Research CenterGeoff Gordon

Sebastian Thrun, Stanford University

c© VANDI VERMA, 2004

ABSTRACT

EXPERIENCE has shown that even carefully designed and tested robots may en-counter anomalous situations. It is therefore important for robots to monitor theirstate so that anomalous situations may be detected in a timely manner. Robotfault diagnosis typically requires tracking a very large number of possible faults

in complex non-linear dynamic systems with noisy sensors. Traditional methods eitherignore the uncertainly or use linear approximations of nonlinear system dynamics. Suchapproximations are often unrealistic, and as a result faults either go undetected or becomeconfused with non-fault conditions.

Probability theory provides a natural representation for uncertainty, but an exact Bayesiansolution for the diagnosis problem is intractable. Monte Carlo approximations have demon-strated considerable success in application domains such as computer vision and robot lo-calization and mapping. But, classical Monte Carlo methods, such as particle filters, cansuffer from substantial computational complexity. This is particularly true with the pres-ence of rare, yet important events, such as many system faults.

The thesis makes contributions to the theory of probabilistic state-estimation to solvethis problem. It presents a novel approach that outperforms existing algorithms (in termsof computational efficiency) for reliably estimating the state of general (nonlinear, non-Gaussian) dynamic systems in real-time in the presence of uncertainty (including rareevents such as faults).

The thesis presents a set of complementary algorithms that provide an approach forcomputationally tractable fault diagnosis. These algorithms leverage probabilistic approachesto decision theory and information theory to efficiently track a large number of faults in ageneral dynamic system with noisy measurements. The problem of fault diagnosis is rep-resented as hybrid (discrete/continuous) state estimation. Taking advantage of structurein the domain it dynamically concentrates computation in the regions of state space thatare currently most relevant without losing track of less likely states. Experiments with adynamic simulation of a six-wheel rocker-bogie rover show a significant improvement inperformance over the classical approach.

TABLE OF CONTENTS

ABSTRACT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ii

LIST OF FIGURES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vi

LIST OF TABLES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viii

NOTATION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix

CHAPTER 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.1. Robot Fault Detection, Identification, and Monitoring . . . . . . . . . . . . . 2

1.1.1. Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.1.2. Challenges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

1.2. Particle Filters for Monitoring Faults . . . . . . . . . . . . . . . . . . . . . . . 41.2.1. Fault Monitoring as Recursive State Estimation . . . . . . . . . . . . . . 51.2.2. Bayesian Filtering for Recursive State Estimation . . . . . . . . . . . . . . 51.2.3. Classical Particle Filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61.2.4. Hybrid Representation for Fault Monitoring . . . . . . . . . . . . . . . . 71.2.5. Conditional Dependencies . . . . . . . . . . . . . . . . . . . . . . . . . . 81.2.6. Bayesian Representation for Fault Monitoring . . . . . . . . . . . . . . . 91.2.7. Classical Particle Filter for Fault Monitoring . . . . . . . . . . . . . . . . 91.2.8. Limitations of Classical Particle Filter for Fault Monitoring . . . . . . . . 9

1.3. Thesis Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91.4. Thesis Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

CHAPTER 2. Bayesian State Estimation . . . . . . . . . . . . . . . . . . . . . . . . . 122.1. Bayesian Filtering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132.2. Parametric Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

2.2.1. Kalman Filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132.2.2. Extended Kalman Filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142.2.3. Gaussian Quadrature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152.2.4. Unscented Transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162.2.5. Unscented Kalman Filter . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

2.3. Mixture of Gaussian Belief . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172.3.1. Pruning Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172.3.2. Sampling Methods – Rao-Blackwellized Particle Filter . . . . . . . . . . . 182.3.3. Collapsing Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

2.4. Monte Carlo Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202.4.1. Importance Sampling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212.4.2. Resampling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222.4.3. Classical Particle Filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232.4.4. Effective Sample Size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

TABLE OF CONTENTS

2.4.5. Auxiliary Particle Filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . 262.4.6. Unscented Particle Filter . . . . . . . . . . . . . . . . . . . . . . . . . . . 262.4.7. Factored Particle Filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . 272.4.8. Mixture Proposal Distribution . . . . . . . . . . . . . . . . . . . . . . . . 27

2.5. Smoothing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 282.5.1. Kalman Smoothing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 282.5.2. Expectation Propagation . . . . . . . . . . . . . . . . . . . . . . . . . . . 292.5.3. Monte Carlo Smoothing . . . . . . . . . . . . . . . . . . . . . . . . . . . . 292.5.4. Assumed Density Smoothing . . . . . . . . . . . . . . . . . . . . . . . . . 29

CHAPTER 3. Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 303.1. Related Work in Hybrid State Estimation . . . . . . . . . . . . . . . . . . . . . 30

3.1.1. Multiple Model Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . 303.1.2. Mixture Component Pruning Methods . . . . . . . . . . . . . . . . . . . 313.1.3. Probabilistic Hybrid Automata . . . . . . . . . . . . . . . . . . . . . . . . 313.1.4. Rao-Blackwellized Particle Filter . . . . . . . . . . . . . . . . . . . . . . . 313.1.5. Particle Boosting Methods . . . . . . . . . . . . . . . . . . . . . . . . . . 32

3.2. Non-temporal Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323.3. Qualitative Model-based Methods . . . . . . . . . . . . . . . . . . . . . . . . 323.4. Related Work in Robotics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

3.4.1. Safety Engineering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 333.4.2. Redundancy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 343.4.3. Expert Systems and Fault Tree Analysis . . . . . . . . . . . . . . . . . . . 343.4.4. Partially Observable Markov Decision Process . . . . . . . . . . . . . . . 35

3.5. Related work in Hierarchical Methods . . . . . . . . . . . . . . . . . . . . . . 353.5.1. Structural Abstraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353.5.2. Functional Abstraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 363.5.3. Hierarchical Clustering . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

3.6. Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

CHAPTER 4. Decision Theoretic Particle Filter . . . . . . . . . . . . . . . . . . . . . 374.1. Risk-sensitive Sampling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 384.2. The Risk Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 394.3. Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

4.3.1. Faults in a Simulation of the Hyperion Rover . . . . . . . . . . . . . . . . 424.3.2. Stuck Wheel Faults in a Rocker-Bogie Rover Simulation . . . . . . . . . . 42

CHAPTER 5. Variable Resolution Particle Filter . . . . . . . . . . . . . . . . . . . . . 475.1. Variable Resolution State Space Model . . . . . . . . . . . . . . . . . . . . . . 485.2. Belief State Estimation at a Fixed Resolution . . . . . . . . . . . . . . . . . . . 485.3. Bias-Variance Trade-off . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 505.4. Dynamically Varying Resolution . . . . . . . . . . . . . . . . . . . . . . . . . 525.5. Variable Resolution Particle Filter Algorithm . . . . . . . . . . . . . . . . . . 525.6. Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

CHAPTER 6. Look-ahead Particle Filters . . . . . . . . . . . . . . . . . . . . . . . . 586.1. Variable Resolution Unscented Particle Filter . . . . . . . . . . . . . . . . . . 58

6.1.1. Optimal Proposal Distribution . . . . . . . . . . . . . . . . . . . . . . . . 596.1.2. Approximation of Optimal Posterior using Unscented Kalman Filters . . 596.1.3. Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

6.2. Decision-Theoretic Monte Carlo Smoothing . . . . . . . . . . . . . . . . . . . 626.3. Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

[Draft: 19 October 2004—19:11 ] iv

TABLE OF CONTENTS

CHAPTER 7. Robot Fault Diagnosis Experiment . . . . . . . . . . . . . . . . . . . . 667.1. Experimental setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66

7.1.1. Simulator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 667.1.2. Faults modeled . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

7.2. Experimental results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 697.2.1. Classical Particle Filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . 707.2.2. Decision-Theoretic Particle Filter . . . . . . . . . . . . . . . . . . . . . . . 727.2.3. Decision-Theoretic Variable Resolution Particle Filter . . . . . . . . . . . 74

CHAPTER 8. Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 838.1. Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 838.2. Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84

8.2.1. Model-based Diagnosis . . . . . . . . . . . . . . . . . . . . . . . . . . . . 848.2.2. Physics Based Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . 848.2.3. Probabilistic Diagnosis . . . . . . . . . . . . . . . . . . . . . . . . . . . . 858.2.4. Interface to Planners/Controllers . . . . . . . . . . . . . . . . . . . . . . 858.2.5. Lessons Learned from Experiments with Real Rovers . . . . . . . . . . . 868.2.6. Role of simulation in rover fault diagnosis . . . . . . . . . . . . . . . . . 86

8.3. Future Directions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 878.3.1. Adaptive Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 878.3.2. Switching Hierarchies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 878.3.3. Factored Representation . . . . . . . . . . . . . . . . . . . . . . . . . . . 88

BIBLIOGRAPHY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89

APPENDIX A. Simulation of Hyperion rover for Fault Detection and Identification . 97

APPENDIX B. List of Fault Names and Corresponding Numbers . . . . . . . . . . . 100

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105

[Draft: 19 October 2004—19:11 ] v

LIST OF FIGURES

1.1 Dante II being rescued by helicopter . . . . . . . . . . . . . . . . . . . . 31.2 Mars Polar Lander . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.3 Spirit continuing operation on five wheels as a result of detecting

lubricant leak fault . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41.4 Graphical model representing fault monitoring . . . . . . . . . . . . . . 8

2.1 Graphical model representing a state space model . . . . . . . . . . . . 122.2 Trajectory depletion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

4.1 Hyperion Robot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 424.2 FDI on Hyperion with a classical particle filter . . . . . . . . . . . . . . 434.3 Decision-theoretic particle filter . . . . . . . . . . . . . . . . . . . . . . 434.4 Six-wheel rocker bogie rover in Darwin2K simulator . . . . . . . . . . . 444.5 Change in rover trajectory induced by independent stuck wheel faults . 444.6 HMM representing fault transitions for stuck wheels . . . . . . . . . . . 454.7 Performance comparison between DTPF and CPF with stuck-wheel

faults in simulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 454.8 Degradation in DTPF performance with a larger number of similar faults 45

5.1 General Markov model representing physical states . . . . . . . . . . . 495.2 Abstract Markov model . . . . . . . . . . . . . . . . . . . . . . . . . . . 495.3 Markov Model with multiple levels of abstraction . . . . . . . . . . . . 505.4 HMM representing abstraction of 4.6 . . . . . . . . . . . . . . . . . . . 555.5 HMM representing a refinement of an abstract state in 5.4 . . . . . . . . 555.6 Graph comparing number of particles vs. error for CPF and VRPF on a

simple experiment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 565.7 Graph comparing wall clock time vs. error for CPF and VRPF on a

simple experiment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

6.1 Graph comparing number of particles vs. error for CPF, UPF, and VUPFon a simple experiment . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

6.2 Graph comparing wall clock time vs. error for CPF, UPF, and VUPF on asimple experiment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

6.3 Decision-Theoretic cost function smoothing . . . . . . . . . . . . . . . . 64

7.1 Side view of rocker-bogie rover in Darwin2K . . . . . . . . . . . . . . . 67

LIST OF FIGURES

7.2 Top view of rocker-bogie rover in Darwin2K . . . . . . . . . . . . . . . 677.3 View of rocker-bogie rover in Darwin2K . . . . . . . . . . . . . . . . . . 677.4 View of rocker-bogie rover in Darwin2K . . . . . . . . . . . . . . . . . . 687.5 Slipping wheel on Hyperion rover during a field test in the Atacama

desert . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 697.6 Stuck ATV . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 707.7 The K9 rover stuck during a field test . . . . . . . . . . . . . . . . . . . 707.8 Distribution over all states for 10 timesteps after the occurrence of a fault

with a CPF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 717.9 Delay in fault detection with a CPF. . . . . . . . . . . . . . . . . . . . . 727.10 Section of figure 7.9 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 737.11 Section of figure 7.9 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 737.12 Distribution over all states for 10 timesteps after the occurrence of a fault

with a DTPF. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 747.13 Delay in fault detection with a DTPF. . . . . . . . . . . . . . . . . . . . 757.14 Section of figure 7.13 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 757.15 Dendrogram representing the hierarchy for the normal state and all one,

two, three, and four wheel slipping, stuck, and lubricant leak faults . . 767.16 The top third of the hierarchy from figure 7.15 at a higher resolution . . 777.17 The middle segment of the hierarchy from figure 7.15 at a higher

resolution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 787.18 The bottom third of the hierarchy from figure 7.15 at a higher resolution 797.19 Distribution over all states for 10 timesteps after the occurrence of a fault

with a DT-VRPF. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 807.20 Delay in fault detection with a DT-VRPF. . . . . . . . . . . . . . . . . . 817.21 Comparison of DTPF and DT-VRPF for normal driving, and every

combination of one, two, three, and four wheels slipping, stuck, orwith a lubricant leak. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82

8.1 Schematic of fault detection and identification module in system architecture 858.2 Health Monitoring on the Hyperion rover during a field test . . . . . . 87

A.1 Schematic view of the Hyperion rover . . . . . . . . . . . . . . . . . . . 97

[Draft: 19 October 2004—19:11 ] vii

LIST OF TABLES

2.1 Algorithm for systematic resampling . . . . . . . . . . . . . . . . . . . 222.2 Classical particle filter for fault monitoring . . . . . . . . . . . . . . . . 24

4.1 Decision-theoretic particle filter algorithm . . . . . . . . . . . . . . . . . 40

5.1 Variable resolution particle filter algorithm . . . . . . . . . . . . . . . . 54

NOTATION

• S : the state space• st : state at time t, where st ∈ S. st = 〈dt, xt〉• st : sequence of states up to time t. st = {s0, s1, . . . st}• s1...t : sequence of states up to time t. s1...t = {s0, s1, . . . st}• dt : discrete state (fault or operational state) at time t• xt : continuous state (variables for tracking dynamics) at time t• ut : control at time t.• Z : the observation space• zt : the measurement (also called observation) at time t, zt ∈ Z• zt : sequence of measurements up to time t. zt = {z0, z1, . . . zt}• z1...t : sequence of measurements up to time t. z1...t = {z0, z1, . . . zt}• p(s) : a distribution over states.• p(st | st−1) : the probability distribution over states at time t, given (conditioned

on) the distribution over states at time t− 1• p(zt | st): probability of obtaining measurement zt in state st.• s[i]

t : value of state for particle i at time t, where s[i]t ∈ S

• d[i]t : value of discrete state for particle i at time t, where d[i]

t ∈ D• x[i]

t : value of continuous state for particle i at time t• w[i]

t : weight of ith particle at time t• b : a full belief, a high dimensional representation of p(s)• R : a reward function• N : total number of particles in particle filter• s[i]

t ∼ p(st | s[i]t−1): draw ith state particle s[i]

t from the conditional distribution overstates p(st | st−10[i]), given the state at time t− 1.

• π(s): prior probability distribution over states s

CHAPTER 1

Introduction

THIS thesis presents several complementary algorithms for detecting faults on-

board operating robots. Here, a fault is defined as a deviation from expected

behavior. Experience has shown that even carefully designed and tested robots

may encounter faults (Carlson and Murphy, 2003). One of the reasons for this

is that components degrade over time. Another is that the developers of the robot rarely

have complete knowledge of the environment in which it operates and hence may not have

accounted for certain situations.

Fault Detection and Identification (FDI) for robots is a complex problem. This is be-

cause the space of possible faults is very large, robot sensors, actuators, and environment

models are uncertain, and there is limited computation time and power. Probability the-

ory provides a natural representation of the uncertainty in the rover domain, but an exact

Bayesian solution to FDI is intractable. Traditional methods address this intractability by

approximating the rover dynamics and/or by ignoring uncertainty. Often these approx-

imations are unrealistic, and either faults go undetected, or an unreasonable number of

false positives are triggered.

The algorithms presented in this thesis use Monte Carlo methods to gain accuracy.

Classical Monte Carlo methods for dynamic systems, such as particle filters, are capable

of tracking complex nonlinear systems with noisy measurements. The problem is that

estimates from a particle filter tend to have a high variance for small sample sets. Using

large sample sets is computationally expensive and defeats the purpose.

This dissertation presents several complementary algorithms for improving the accu-

racy of fault monitoring with a computationally tractable set of samples in a particle filter.

Experimental results in the rover domain show significant improvement in accuracy over

the classical approach.

1.1 ROBOT FAULT DETECTION, IDENTIFICATION, AND MONITORING

The algorithms described in this thesis enable monitoring of a wider range and larger

number of faults during robot operation than has hitherto been possible. Furthermore,

these algorithms provide the probability of the robot being in each of the fault and op-

erational states given the sensor data. They can handle noisy sensors, non-linear, non-

Gaussian models of behavior, and are computationally efficient.

Another advantage of the probabilistic approach to FDI is that it allows flexibility in

the types of planners/controllers that may be used for controlling the robot and for recov-

ering from faults. For example, such distributions are compatible with classical conditional

planners or Markov Decision Processes (MDPs), which may use the most likely state to de-

termine which action to take, and also Partially Observable Markov Decision Processes

(POMDPs) (Sondik, 1971; Kaelbling et al., 1998), which use the distribution over the entire

state space.

1.1. Robot Fault Detection, Identification, and Monitoring

A fault is defined as a deviation from the expected behavior of the system. A failure is a

complete interruption of the system’s ability to perform the required operation (Isermann

and Balle, 1997). Fault detection is defined as the process of determining that a fault has

occurred. Fault identification is the process of determining exactly which exception or fault

occurred. Fault detection and identification are typically passive, i.e. they do not alter

control actions. Fault monitoring is the process of providing a distribution over fault and

operational states when there is uncertainty in the domain. All the algorithms presented

in this thesis perform fault monitoring.

The faults addressed here include mechanical component failures, such as broken mo-

tors and gears; faults due to environmental interactions, such as a wheel stuck against a

rock; and sensor failures, such as broken encoders.

1.1.1. Motivation

In a number of application domains, such as planetary exploration, search and res-

cue, mine mapping, nuclear waste cleanup, and demining, robots operate in environments

where human intervention is expensive, slow, unreliable, or infeasible. It is therefore es-

sential for the robots to monitor their behavior so that faults may be addressed before they

result in catastrophic failures. An example of this is the Dante II robot (Bares and Wetter-

green, 1999). In 1994, Dante II was deployed in a remote Alaskan volcano to demonstrate

remote robotic exploration. While ascending out of the crater, it encountered steep slope

[Draft: 19 October 2004—19:11 ] 2

1.1 ROBOT FAULT DETECTION, IDENTIFICATION, AND MONITORING

Figure 1.1. Dante II being rescued by a helicopter after it slipped while ascending avolcano it was studying in Alaska1.

Figure 1.2. Mars Polar Lander turned off its landing thrusters because of a sensor spike.2

and cross-slope conditions that changed the system dynamics. Failure to identify this re-

sulted in the robot falling on its side. Dante II was unable to self-right and had to be rescued

by helicopter (figure 1.1).

Another example is the Mars Polar Lander (figure 1.2). It is hypothesized that a sensor

spike made it turn off its landing thrusters before had actually landed. Since the engine

turned off too soon, the spacecraft fell to the surface at about 50 miles per hour, and crash-

landed (NASA, 2000).

A recent example is from the Mars Exploration Rover, Spirit. There is a lubricant leak

in one of the wheels on Spirit. This fault was detected by the large team of engineers who

painstakingly analyze rover telemetry every night. A fully autonomous rover would be

required to detect this fault autonomously. The detection of this fault has allowed the team

to modify the control algorithm and continue operation with five wheels as shown in figure

1.3.

Not only are robots venturing into areas inaccessible or dangerous for humans, but

they are also increasingly becoming a part of day to day life. It is also important for these

robots to detect faults in a timely manner, since failure to do so may result in expensive

consequences, both monetary and in terms of consumer trust that may be hard to regain.

If faults go undetected, autonomous robots in real-world environments may behave in

[Draft: 19 October 2004—19:11 ] 3

1.2 PARTICLE FILTERS FOR MONITORING FAULTS

Figure 1.3. Spirit continuing operation on five wheels as a result of detecting a lu-bricant leak fault3

an unpredictable or dangerous manner. On the other hand, detecting and recovering from

faults can considerably improve the performance of the robots (Bererton and Khosla, 2002).

To maximize successful operation the emphasis needs to be on designing to minimize faults

as much as possible, and to include algorithms to detect and recover from faults when they

do occur. The focus of this dissertation is on the fault detection aspect of the problem.

1.1.2. Challenges

Interpreting certain faults requires context sensitive interpretation of sensor data that

can be obtained only by monitoring the dynamics of the system over time, which tend to

differ according to operating conditions. For example, for a rover, an increase in the power

required for locomotion on flat ground may be a cause for concern, but a similar increase

on a slope might be perfectly acceptable. Sensors do not directly report these dynamics

because they are noisy and limited, i.e., they do not have complete information about the

state of the rover and the environment that it is operating in. Control actions do not pro-

vide complete information about state transitions either, since faults and environmental

interactions induce involuntary transitions. In addition, there are a large number of com-

ponents that can fail in various combinations at any instant in time and the computational

resources are too limited to consider all possible combinations.

1.2. Particle Filters for Monitoring Faults

Our formulation of the fault monitoring problem requires estimating robot and en-

vironmental state, as it changes over time, from a sequence of sensor measurements that

provide noisy, partial information about the state. The Bayesian approach to dynamic state

[Draft: 19 October 2004—19:11 ] 4


estimation addresses this problem. Computing the exact Bayesian posterior analytically is

intractable for the fault monitoring problem. Hence, we use a particle filter approximation

in this thesis. Particle filters are a Monte Carlo approximation method for dynamic state

estimation. Particle filters have been extensively used for Bayesian state estimation in non-

linear systems with noisy measurements (Isard and Blake, 1998; Fox et al., 1999; Doucet

et al., 2001). They approximate the probability distribution with a set of samples or parti-

cles. The algorithms presented in this dissertation all use particle filters.

1.2.1. Fault Monitoring as Recursive State Estimation

State estimation is the process of determining the state of a system from a sequence of

data. Fault monitoring has a natural interpretation as a state estimation problem. Possible

fault and operational modes of the systems are represented as explicit states. The sequence

of measurements is then used to determine the state of the system. There are two main

classes of state estimation methods: batch estimation methods and recursive estimation

methods.

Batch methods treat all the data with equal importance and find an optimal estimate

of the state given the entire sequence. However, full batch estimation is computationally

expensive and gets slower as the number of observations increases. It is therefore not

suitable for fault monitoring.

Recursive state estimation methods incorporate the data as it becomes available and

replace the data with a statistic. Estimates at subsequent timesteps use this statistic instead

of the history of data for state estimation. A majority of the popular recursive state es-

timation methods make a Markov assumption, i.e., the past and future are conditionally

independent given the current state. If the statistic is a sufficient statistic, the performance

of recursive state estimation is equivalent to batch estimation.

This thesis concentrates on a discrete time, first order Markov formulation of the state

estimation problem. A distribution over the unobservable discrete fault and operational

states of a rover is obtained from noisy measurements of continuous sensor readings. This

problem is addressed in this thesis using a probabilistic technique called Bayes filtering.

1.2.2. Bayesian Filtering for Recursive State Estimation

Filtering is defined as the forward process of estimating the state, st, at time t, given

the sequence of data up to time t, z0 . . . zt. In a forward process the state estimate at time

t − 1 may only depend on measurement up to time t − 1 and not on the measurement at

time t. Bayes filters are based on Bayes rule from probability theory.

[Draft: 19 October 2004—19:11 ] 5


Here the multivariate state at time t is denoted as st and measurements or observa-

tions as zt. Since we use the discrete time, first order Markov formulation of the dynamic

state estimation problem, the state at time t is a sufficient statistic for the history of mea-

surements. That is, p(st | s0...t−1) = p(st | st−1) and the observations depend only on the

current state, i.e. p(zt | s0...t) = p(zt | st). The posterior distribution at time t, p(s1...t | z1...t),

includes all the available information up to time t and provides the optimal solution to the

state estimation problem. We are interested in the filtering distribution, p(st | z1:t), which

is a marginal of this distribution. The recursive filter is defined as follows (for derivation

see section 2.1):

p(st | z1...t) = ηt p(zt | st)∫p(st | st−1) p(st−1 | z1...t−1) dst−1 (1.1)

This process is known as Bayesian filtering, optimal filtering, or stochastic filtering and

may be characterized by three distributions: (1) a transition model p(st | st−1), (2) an

observation model p(zt | st), and, (3) an initial prior distribution, π(s0).

Although the mathematical structure of the optimal Bayesian solution is well under-

stood (Bar-Shalom and Fortmann, 1988), the computational complexity of the optimal fil-

tering algorithm limits its practical applicability using even the fastest computers available

in the foreseeable future. In general, robots (especially planetary rovers) tend to be even

more power and computation constrained than the state of the art. Therefore, we are inter-

ested in tractable methods that compute approximations to this posterior.

1.2.3. Classical Particle Filters

Particle filters (Metropolis and Ulam, 1949; Gordon et al., 1993a; Kanazawa et al., 1995;

Doucet et al., 2001; Isard and Blake, 1998) perform Monte Carlo integration to approximate

the posterior distribution over the states with a set of samples or “particles”. Monte Carlo

integration is a technique for computing an integral over a complicated domain using sam-

ples. The particle approximation converges to the true Bayesian posterior in the limit as

the number of samples goes to infinity.

Particle filters have several characteristics that make them attractive for fault monitor-

ing on robots: they are non-parametric (can represent arbitrary distributions), can handle

hybrid state spaces, can handle noisy sensing and motion, and can easily be extended to an

anytime approach where the number of particles (and hence the estimation accuracy) can

be adjusted to match available computation.

[Draft: 19 October 2004—19:11 ] 6


Particle filters approximate the posterior with a set of N fully instantiated state sam-

ples or particles, Pt = {x[1]t . . . x

[N ]t } as follows:

p(st | z1:t) ≈1

N

N∑

i=1

δ(st − x[i]t ) (1.2)

where δ(.) denotes the Dirac delta function. It can be shown that as N → ∞ the approxi-

mation in equation 1.2 approaches the true posterior density (Doucet and Crisan, 2002). In

general it is difficult to draw samples from p(st | z1:t); instead, samples are drawn from

a more tractable distribution, q(.), called the proposal, or importance, distribution. Each

particle is assigned a weight, w[i], to account for the fact that the samples were drawn

from a different distribution (Rubin, 1988; Rubinstein, 1981). There are a large number

of possible choices for the proposal distribution, the only condition being that its support

must include that of the posterior. The common practice is to sample from the transition

probability, p(st | st−1), in which case the importance weight is equal to the likelihood,

p(zt | st)(Avitzour, 1995; Gordon et al., 1993a; Kitagawa, 1993) .

1.2.4. Hybrid Representation for Fault Monitoring

A hybrid (discrete/continuous) state representation (Bar-Shalom, 1988; R. B. Wash-

burn, 1985) provides a natural formulation for fault monitoring. The hybrid state captures

the notion that the behavior of the system cannot be characterized at all times by a single

model of behavior (differential equation), but a finite number of models can describe the

behavior in different modes of operation. Each discrete state corresponds to a behavior

mode (differential equation model), which is tracked with continuous variables. The sys-

tem switches from one behavior mode (model) to another according to a transition proba-

bility matrix.

1.2.4.1. Discrete fault states. Let D= {d(1) . . . d(K)} represent K discrete fault and

operational states of the robot, where dt ∈D is the discrete state of the robot at time t and

dt = {d1...t} the discrete, first order Markov chain representing the evolution of the state

over time. The problem of monitoring the state of the system consists of providing a belief

(a distribution over the state set D) at each time step as it evolves based on the following

transition model:

p(dt = j | dt−1 = i), (i, j ∈D) (1.3)

1.2.4.2. Continuous state variables. In addition to the discrete states there are also

continuous states. Let xt ∈ Rnx denote the multivariate continuous state at time t.

[Draft: 19 October 2004—19:11 ] 7


xt-1

dt-1

xt

ut ut+1

zt-1 zt

dt

Figure 1.4. The graphical model representing fault monitoring as a hybrid stateestimation problem. The continuous state transitions (xt−1 to xt) are conditionallyindependent of the discrete state transitions (dt−1 to dt ). ut and zt represent thecontrol and measurement respectively at time t.

Each of the discrete fault and operational modes changes the dynamics of the rover.

Let xt ∈ Rnx denote the multivariate continuous state of the rover at time t. The non-linear

conditional state transition models are denoted by p(xt | xt−1, dt). The state of the rover

is observed through a sequence of measurements, {zt}, based on the measurement model

p(zt | xt, dt), zt ∈ Rnz .

1.2.5. Conditional Dependencies

As a consequence of the above the following factored representation is obtained:

p(xt, dt | xt−1, dt−1) = p(xt | xt−1, dt)p(dt | dt−1) (1.4)

Here, the discrete state transition is conditionally independent of the continuous state tran-

sitions given the previous discrete state. This is not an exact factorization, but is reasonable

in most cases.

[Draft: 19 October 2004—19:11 ] 8

1.3 THESIS CONTRIBUTIONS

1.2.6. Bayesian Representation for Fault Monitoring

Figure 1.4 depicts fault monitoring as a graphical model. A graphical model [a recent

introduction may be found in (Jordan., 2003)] is a probabilistic model defined in terms of a

directed or undirected graph. The nodes in the graph represent random variables and the

dependencies and independencies between variables are represented by edges.

The Bayes filter for estimating the factored state is thus:

p(xt, dt | z1...t) = ηt p(zt | xt)∫ ∑

dt−1

p(xt | xt−1, dt) p(dt | dt−1) dxt−1 (1.5)

where ηt is a normalizing constant. Here, control is omitted in equations for brevity.

1.2.7. Classical Particle Filter for Fault Monitoring

As discussed in section 1.2.3 particle filters may be used to provide a non-parametric

approximation of the posterior in equation 1.4. A set of N fully instantiated state samples,

or particles, {(d[1]t , x

[1]t ) . . . (d

[N ]t , x

[N ]t )}, represents a hybrid (discrete/continuous) posterior

as follows:

pN (xt, dt | z1...t) =N∑

i=1

w[i]t δ

x[i]t ,d

[i]t

(xt, dt) (1.6)

The importance weight for the hybrid posterior represented by x[i]t , d

[i]t is:

w[i]t =

p(x[i]t , d

[i]t | x[i]

t−1, d[i]t−1, zt)

q(x[i]t , d

[i]t )

(1.7)

1.2.8. Limitations of Classical Particle Filter for Fault Monitoring

The main problem with classical particle filters for fault monitoring is that a large

number of particles is often needed to enable detection of rare events. Reliably tracking

multiple low-probability fault states, which requires maintaining and updating large num-

bers of particles, is typically not practical due to limited computation. Small particle sets

do not provide reasonable approximations because: (1) they are unlikely to represent low

probability fault states, (2) their estimates are likely to have a high variance, particularly

when there are a large number of possible state transitions.

1.3. Thesis Contributions

This dissertation makes contributions to the theory of probabilistic state-estimation

to solve the problem of fault monitoring. It presents a novel approach that outperforms

existing algorithms (in terms of computational efficiency and scalability) for reliably esti-

mating the state of general (nonlinear, non-Gaussian) dynamic systems in real-time in the

[Draft: 19 October 2004—19:11 ] 9

1.3 THESIS CONTRIBUTIONS

presence of uncertainty (including rare events such as faults). This thesis presents three

novel algorithms that address these issues:

1. The Risk Sensitive Particle Filter (RSPF) reduces the number of particles required

to track unlikely states by incorporating potential loss in the proposal distribution

for updating particles.

2. The Variable Resolution Particle Filter (VRPF) reduces the number of particles re-

quired by grouping multiple low-probability states based on similarity, differenti-

ating them only when it is useful.

3. The Variable Resolution Unscented Particle Filter (VUPF) looks at the expected sensor

measurement one step ahead in time to improve the state estimate provided by a

limited number of particles.

The RSPF incorporates a model of cost when generating particles. This approach is

motivated by the observation that the cost of not tracking hypotheses is related to risk.

Not tracking a rare, but risky, state may have a high cost, whereas not tracking a rare but

benign state may be irrelevant. Incorporating a cost model into particle filtering improves

the tracking of states that are most critical to the performance of the robot. Automatic

calculation of the cost model is implemented by solving a Markov Decision Process (MDP)

to estimate the value of tracking a particular state.

The VRPF maintains samples in the state space at dynamically varying resolution for

computational efficiency. Resolution within state space varies by region, depending on the

belief that the true state lies within each region. Where belief is strong, resolution is fine.

Where belief is low, resolution is coarse, abstracting multiple similar states together. The

resolution of the state space is dynamically updated as the belief changes. The algorithm

makes an explicit bias-variance trade-off to select between maintaining samples in a biased

generalization of a region of state space versus in a high variance specialization at fine

resolution. Samples are maintained at a coarser resolution when the bias introduced by

the generalization to a coarse resolution is outweighed by the gain in terms of reduction

in variance, and at a finer resolution when it is not. Maintaining samples in abstraction

prevents potential hypotheses from being eliminated prematurely for lack of a sufficient

number of particles. Empirical results show that the VRPF requires significantly lower

computation for performance comparable to a classical particle filter.

The VUPF reduces the variance of the particle filter estimate by taking into account

the next measurement when generating particles. This approach combines the Unscented

Kalman Filter (UKF) with the Variable Resolution Particle Filter. It computes an approxi-

mately optimal proposal distribution for each transition using an UKF (Julier and Uhlmann,

[Draft: 19 October 2004—19:11 ] 10

1.4 THESIS OUTLINE

1996). Particles are then generated from this proposal distribution. The VRPF is used to im-

prove the scalability of this approach. The VRPF reduces the number of possible transitions

to the next state and hence UKF computations. Experimental results show a significant im-

provement in efficiency over classical approaches.

This thesis includes the first application of particle filters to robot fault diagnosis,

which has since become an active area of research.

1.4. Thesis Outline

• Chapter 2 provides a background on Bayesian state estimation, with a focus on

methods relevant to this dissertation.

• Chapter 3 discusses related work in more detail.

• Chapter 4 presents the Risk Sensitive Particle Filter algorithm.

• Chapter 5 presents the Variable Resolution Particle Filter algorithm.

• Chapter 6 presents the Variable Resolution Unscented Particle Filter and demon-

strates how the variable resolution particle filter in conjunction with the Unscented

particle filter results in an efficient algorithm. In addition this chapter also presents

a particle filter that uses Decision-theoretic Monte Carlo smoothing to improve ac-

curacy.

• Chapter 7 discusses results from experiments with diagnosing locomotion faults

in simulation.

• Chapter 8 presents a concluding discussion and suggests future work.

[Draft: 19 October 2004—19:11 ] 11

CHAPTER 2

Bayesian State Estimation

BAYESIAN state estimation is a method for determining the state of a system

from a sequence of data based on Bayes Rule. A state space representation

assumes that there is some underlying hidden state (of the robot and environ-

ment) that generates the observations, and that this hidden state evolves in

time, possibly as a function of control inputs. Batch estimation methods find an optimal

estimate of the state given the entire sequence of data (measurements and control). There

are certain conditional independence assumptions made by most of these methods, as de-

picted in the state space model shown in figure(2.1) - one is that the past and future states

are conditionally independent given the state at time t (the Markov Assumption) and the

other is that the measurements are conditionally independent given the state. As noted in

the previous chapter, full batch estimation is computationally expensive and gets slower

as the robot accumulates increasing volumes of data. This chapter describes recursive state

estimation methods that incorporate data incrementally. It also describes approximate fil-

tering methods such as particle filtering, and different representations for fault monitoring

as a state estimation problem such as hidden Markov models and dynamic Bayes nets.

Figure 2.1. Graphical model representing a state space model

2.2 PARAMETRIC METHODS

2.1. Bayesian Filtering

Filtering (also called tracking or monitoring) is the process of computing a posterior

distribution over the state set given the sequence of data up to the current time step. Fil-

tering computes, p(st | z0 . . . zt), an estimate of the state st at time t, based on a partial

sequence of data, z0 . . . zt. A recursive filter for this problem is derived below. Equation 2.1

uses Bayes rule and equations 2.2 and 2.4 use the Markov assumption.

p(st | z1...t) = ηt p(zt | st, z1...t−1) p(st | z1...t−1) (2.1)

= ηt p(zt | st) p(st | z1...t−1) (2.2)

= ηt p(zt | st)∫

p(st | z1...t−1, st−1)p(st−1 | z1...t−1) dst−1 (2.3)

= ηt p(zt | st)∫p(st | st−1) p(st−1 | z1...t−1) dst−1 (2.4)

This process, known as either Bayesian filtering, optimal filtering, or stochastic filtering,

may be characterized by three distributions: (1) a transition model p(st | st−1), (2) an

observation model p(zt | st), and, (3) an initial prior distribution, π(s0).

2.2. Parametric Methods

2.2.1. Kalman Filter

Kalman filters (Kalman, 1960) (also called linear dynamical systems, or LDSs) are a

popular method for tracking the state of a dynamic system and are optimal if the process

and measurement equations are linear and the noise is Gaussian. The Kalman filter as-

sumes that the state, measurements, and control are continuous, st ∈ RNs , zt ∈ RNz , ut ∈RNu . The Kalman filter represents the distribution over states using only the first two

central moments of the distribution (the mean and the covariance). Given the mean and

the covariance at one time step, the Kalman filter computes the posterior distribution at

the next time step by first making a prediction of the state at the next time step using the

current state, and then updating the estimate using the next measurement.

The dynamic model in a general Kalman filter can be written as follows:

st = Ast−1 +But + vt (2.5)

zt = Cst +Dut + wt (2.6)

where, vt ∼N (µs, Q), and wt ∼N (µz, R) are independent Gaussian noise terms represent-

ing the process and measurement noise respectively. Matrix A is Nz × Nz , B is Ns × Nu,

C is Nz ×Ns, D is Nz ×Nu, Q is Ns ×Nu, and R is Nz ×Nz .

[Draft: 19 October 2004—19:11 ] 13


The state and measurement functions in a Kalman filter are linear Gaussian as shown

in equation(2.7) and equation(2.8) respectively.

p(xt | xt−1, ut) = N (st;Axt−1 +But, Q) (2.7)

p(zt | xt, ut) = N (zt;Cx+Dut, R) (2.8)

Equation 2.5 represents the process model (or time update step). This step propagates

the prior mean, st−1 and covariance, Pt−1, at time t− 1 to predict the next time step mean,

st|t−1, and covariance, Pt|t−1, but uses no new measurements. Here, st|t−1 is used to denote

an estimate of the mean of st conditioned on the partial sequence z0 . . . zt−1. Equation

2.6 represents the measurement update step that incorporates the new measurement at

time t and updates the predicted mean and covariance to get the posterior mean, st|t, and

covariance, Pt|t.

The complexity of Kalman filtering is O (min(N 2s , N

3z ). This is because the matrix

multiplication for prediction takesN 2s and the matrix inversion takesN 3

z . This computation

can be significantly reduced for sparse matrices.

2.2.2. Extended Kalman Filter

Robot dynamics are typically nonlinear, and thus it is often not possible to use the

Kalman filter directly to track such systems. The extended Kalman filter (EKF) is a pop-

ular method that linearizes such systems by approximating the nonlinear function using

a first order Taylor series expansion about the current state estimate. The accuracy of the

approximation depends on how well the linearization approximates the true function in

the local area around the current state estimate, where the size of this locality depends on

the variance of the current state.

The main disadvantage of the EKF is that the linearization is often a poor approxi-

mation unless the second and higher order terms in the locality of the linearization are

negligible. Often the nonlinerities are higher order and the EKF produces poor estimates.

To improve the accuracy, it is possible to use higher order terms in the Taylor series expan-

sion, but this can be expensive and is rarely used.

Another problem with the EKF is that for the Taylor series expansion the Jacobian

must be computed. Some nonlinear functions are not differentiable or may be difficult to

differentiate, and this prevents the use of the EKF.

[Draft: 19 October 2004—19:11 ] 14


2.2.3. Gaussian Quadrature

2.2.3.1. Numerical Integration Methods. Numerical integration refers to the use of

numerical techniques for computing integrals that cannot be computed analytically. Exam-

ples of numerical integrations methods that are relevant to this thesis are Gaussian quad-

rature and Monte Carlo methods.

The EKF linearizes the nonlinear process and measurement models using the first

order terms of a Taylor series expansion. It then uses this linearization to compute the

posterior distribution. Numerical integration methods directly approximate the posterior

distribution resulting from the nonlinear function. Consider the following nonlinear func-

tion f :

xt+1 = f(xt) (2.9)

Say we wish to approximate p(xt+1, xt) = p(xt+1 | xt)p(xt) as a Gaussian. The following

integrals provide the first and second order moments that allow us to construct the optimal

Gaussian approximation of the distribution:

E[xt+1] =

∫f(xt)p(xt)dxt (2.10)

E[x2t+1] =

∫f2(xt)p(xt)dxt (2.11)

E[xtxt+1] =

∫xtf(xt)p(xt)dxt (2.12)

Often there are no closed form solutions to these integrals and numerical integration meth-

ods must be used.

2.2.3.2. Quadrature. Quadrature refers to any numerical integration method that

replaces an integral with a sum; see (Deans, 2002) for an introduction.

Newton-Cotes rules optimize only the weights but not the location of the points at

which the function is evaluated. Hence each sample has only a single degree of freedom. If

the function is a first or second order polynomial, Newton-Cotes rules such as Simpson’s

rule and the trapezoidal rule provide an exact solution.

2.2.3.3. Gaussian Quadrature. Gaussian quadrature refers to quadrature methods

that specify the location of the points at which the function should be evaluated in addition

to the weights. Each sample thus has two degrees of freedom (the location and the weight),

which provides more flexibility and hence higher order approximations are possible with

the same number of samples. Gaussian quadrature is designed to compute integrals of

[Draft: 19 October 2004—19:11 ] 15


polynomials multiplied with a known non-negative weight function as follows:∫

x

w(x)f(x)dx ≈∑

i

wif(xi) (2.13)

The points and weights are chosen such that the integral is exact if f is a polynomial of

degree 2n− 1 or less. Suppose the integral is multidimensional, i.e. in Rd instead of R. For

multiple dimensions the Gaussian quadrature rule may be applied to the integral in each

dimension. Thus a n point quadrature in d dimensions requires nd points. This quickly

gets computationally expensive in high dimensions.

2.2.4. Unscented Transform

The Unscented Transform (UT) is a method for computing an integral in d dimenstions

without using an exponential number of weights. UT assumes that the weight function is

a Gaussian.

Like Gaussian Quadrature, the UT uses deterministic sampling to compute the inte-

grals in equations 2.10–2.12 above numerically. The integrals are computed using a set of

deterministically chosen weighted samples (called sigma points) that replace the analytic

integral with a numerical sum. Each sigma point is independently propagated through the

process and measurement models, and the set of propagated sigma points is analyzed to

provide a posterior Gaussian approximation. The process of calculating, propagating, and

analyzing the sigma points is called an Unscented Transform or UT. In the UT, the sigma

points are chosen to match the moments of the Gaussian distribution.

These sigma points and weights are used to match some constraints. One constraint

specifies that the weights must sum to one:

∑

i

wi = 1 (2.14)

Another constraint requires that the moments of the sigma points match the moments of a

Gaussian distribution. In d dimensions 2d+ 1 sigma points are deterministically chosen to

capture the true mean and covariance. There is a single central sigma point, and a sigma

point is generated on either side of the central point along each dimension. The weights

are set to κ/(d + κ) for the central point and 1/(2(d + κ) for all other points. (Julier and

Uhlmann, 1996) recommends that κ should be chosen such that d+ κ = 3.

In (Julier, and J. K. Uhlmann, 2002) the authors present the scaled unscented trans-

form that allows a filter designer to match higher order moments in the distribution at the

expense of no longer matching some of the lower order moments as precisely.

[Draft: 19 October 2004—19:11 ] 16

2.3 MIXTURE OF GAUSSIAN BELIEF

2.2.5. Unscented Kalman Filter

Like the Kalman filter and EKF the Unscented Kalman Filter (UKF) is a recursive min-

imum mean square error estimator, but it often provides an efficient improvement over the

Extended Kalman Filter (EKF) for nonlinear models. The UKF does not approximate the

nonlinear process and measurement models. It uses the actual models and instead approx-

imates the distribution of the state variable as a Gaussian. It does so by using an UT to

approximate the integral.

2.3. Mixture of Gaussian Belief

Recently models have been introduced that represent transitions (switches) from one

discrete state to another. These models are variously called switching Kalman filters,

switching linear systems, jump-Markov models, jump-linear system, switching state-space

models, or Switching Linear Dynamic systems. They all share the property that, condi-

tioned on the discrete state, the continuous state dynamic model is linear. That is, for each

history or discrete states that the system can be in, a separate Kalman filter could be main-

tained.

If a Kalman filter is maintained for every trajectory, the number of mixture compo-

nents increases exponentially with time. When the belief represented by a set of M Gaus-

sians, each representing the posterior of a Kalman filter in a bank of M Kalman filters, is

propagated to the next time step, t, we get a different Gaussian for every combination of

Gaussian belief at time t−1 and possible next discrete state dt that the current discrete state

dt−1 may switch to. In the worst case, when the transition matrix is dense, this may result

in M Gaussians at time t for every Gaussian at time t− 1, or a total of M 2 Gaussians in the

posterior at time t. If the tracking were done for T timesteps , we would end up with M T

Gaussians. While in the fault diagnosis problem the discrete state transition matrix is fairly

sparse, there is still a need to reduce the number of mixture components from increasing

exponentially.

To address this, researchers have developed algorithms that which propagate the exact

posterior for one step, giving a large Gaussian mixture, and then reduce the mixture as

needed. Methods for reducing the mixture are listed below:

2.3.1. Pruning Methods

These algorithms drop the Gaussians with low posterior probability. They keep the

Gaussians with the M highest posterior probabilities and prune the rest.

[Draft: 19 October 2004—19:11 ] 17


2.3.2. Sampling Methods – Rao-Blackwellized Particle Filter

This approach samples mixture components proportional to their posterior likelihood.

RBPFs factor the state space, so that a subset of the variables may be computed analytically

given the values of the rest of variables, which are obtained from particle filtering (Liu,

2000; Doucet et al., 1999). Since our representation of the fault diagnosis problem has a

natural factorization and the discrete state transitions p(dt | dt−1) are conditionally inde-

pendent, Rao-Blackwellized particle filters may be used to track the state space. Discrete

states are sampled as in a classical particle filter, and conditioned on these discrete samples,

d[i]t the continuous state x[i]

t is computed analytically.

The most common representation for the conditional probability density p(xt | x[i]t−1, d

[i]t )

is a Gaussian, (µ[i]t ,∑[i]t ), where µ[i]

t and∑[i]t represent the mean and covariance of the

Gaussian conditioned on the discrete sample d[i]t .

2.3.3. Collapsing Methods

These methods repeatedly merge the most similar pair of Gaussian mixture compo-

nents (Lerner et al., 2000) until only M Gaussians remain. They still have to generate the

full exact posterior Gaussian mixture, before collapsing them. Collapsing algorithms fall

into the three main classes, described below:

2.3.3.1. Generalized Pseudo-Bayesian. (GPB) (Ackerson and Fu, 1970; Chang and

Athanas, 1978; Tugnait, 1982): A GPB algorithm of order k, called (GPBk), maintains at

most Mk−1 Gaussians in the posterior belief state, where M is the number of discrete

states (behavior models), and k is the number of timesteps over which the discrete state

transitions are considered simultaneously. GPBk needs to run M k Kalman filters. So,

GPB1 (i.e. k = 1), runs M Kalman Filter and has at most 1 Gaussian in its belief state and

GPB2 (ie. k = 2), runsM 2 Kalman Filters has has at mostM Gaussian in its belief state etc.

In GPB1, the 1 Gaussian in the belief state is propagated to the next time. Given that

there are at most M discrete states, dt, at time t that it may transition to, it may have to

run up to M Kalman Filters. All the posterior Gaussians are then collapsed to get a single

Gaussian. Note that the Gaussians being collapsed together may be in different discrete

states. The most likely discrete state is maintained. This algorithm does not simultane-

ously track multiple hypotheses and is not very accurate, but it does allow the system to

dynamically transition (switch) to a different discrete state.

[Draft: 19 October 2004—19:11 ] 18


In GPB2, when Gaussians are propagated to the next time step, all the Gaussians that

transition to the same next discrete state, dt, are collapsed. Note, the Gaussians being col-

lapsed together may have been in different discrete states at time t − 1. Since there are M

modes in the system, there are at most M Gaussians in the posterior belief state, and at

most one Gaussian in each discrete state.

At each time step GPB2 needs to generate up to M 2 Gaussians, which is computa-

tionally expensive. Although GPB2 is more accurate than GPB1, it is still not accurate

enough for robot fault diagnosis. This is because Gaussians in the same posterior discrete

state at time t may actually be very different. The classic example of this is a robot that

works or is faulty, and when it is faulty it does not move. Say the prior belief consists of

two Gaussians one in the normal and the other in faulty state. At the next timestep both

these Gaussians transition to the faulty state, but the position represented by these Gaus-

sians is very different. On collapsing we result in an average position, which is entirely

incorrect. GPB2 is prone to collapse dissimilar Gaussians, which is not desirable.

2.3.3.2. Interacting Multiple Model. (IMM) (Blom, 1984; Blom and Bar-Shalom,

1988) achieves a compromise between performance and computational complexity. It main-

tains at most M Gaussians, like GPB1, but its performance is comparable to GPB2. The

computational saving is obtained by collapsing Gaussians before propagating them to the

next time step with a filter update. For each discrete state at time t, a prior Gaussian at time

t − 1 is obtained by mixing all M Gaussians at time t − 1. Hence, M new prior Gaussians

at time t − 1 are obtained. Note that the mixture weights for obtaining each of the prior

Gaussians depends on its corresponding next discrete state dt. The weight is proportional

to the probability that the discrete state at time t− 1 represented by the prior Gaussian will

transition to the discrete state dt at time t.

IMM collapses together all the Gaussians, at time t, before they are propagated to the

next time step, but this is done by weighting the component Gaussians by the probability of

the discrete state transition, (dt−1, dt) that generated the Gaussian. GPBk and IMM reduce

the exponential increase in computation over time, but they can still be computationally

inefficient if the number of discrete states (M ) at a single time step is itself large, which is

often the case in fault diagnosis. In addition, this algorithm is prone to collapsing dissimilar

Gaussians, as with GPB2.

[Draft: 19 October 2004—19:11 ] 19

2.4 MONTE CARLO METHODS

2.3.3.3. Collapsing Based on Similarity. (Lerner et al., 2002): This algorithm col-

lapses posterior Gaussians based on similarity. Posterior Gaussians at time t are enumer-

ated in order of likelihood. The symmetric Kullback Liebler (KL) divergence of this Gauss-

ian is computed against all the other Gaussians in the posterior. If the symmetric KL di-

vergence is less than a user defined threshold, the Gaussians are collapsed. This prevents

dissimilar Gaussians from being collapsed, unlike the GPBk algorithm.

Say there are G Gaussians that need to be collapsed down to M Gaussians. Sorting

these Gaussians in order of likelihood takes O(GlogG) and computing the symmetric KL-

divergence takes O(Gn3x), where nx is the dimensionality of each Gaussian. Finding the

set of Gaussians with KL-divergence less than the threshold c takes O(GMd2). Hence the

algorithm takes O(GlogG + Gd3 + GMd2). But if the original set consists of G = M 2

Gaussians as in GPB2, this can be inefficient.

Since the Gaussians that are collapsed may be in different discrete states, a latent vari-

able H is introduced. Each hypothesis variable H corresponds to a Gaussian and to a

distribution over the discrete states. In other words, for each Gaussian, there is a cor-

responding distribution over the discrete states, which is maintained by the hypothesis

variable. The problem with this approach is that corresponding to each Gaussian, a prob-

ability distribution must be maintained over all the discrete states. In (Lerner et al., 2002),

the discrete state space is therefore decomposed into weakly interacting sub-systems based

on the Boyan-Koller (BK) (Boyen and Koller, 1998) algorithm. The hypothesis variables are

factored, and maintain distributions over, mutually exclusive sub-sets of discrete and con-

tinuous variables. The distribution over the continuous variables is a Gaussian mixture

with mixture components determined by the corresponding distribution over the discrete

variables. Discrete and continuous variables in one sub-set may be conditionally depen-

dent upon discrete or continuous variables in other sub-sets as long as there are no loops.

2.4. Monte Carlo Methods

Monte Carlo methods are computational methods that use random samples of a prob-

ability distribution to convert integrals to sums. An introduction to Monte Carlo methods

may be found in (McKay, 2003; Doucet et al., 2001). Assume that we want to evaluate

the following integral representing the expectation of function f(x) under the distribution

p(x). If it is sufficiently complex that we cannot evaluate it exactly:

E[f(x)] =

∫p(x)f(x)dx (2.15)

[Draft: 19 October 2004—19:11 ] 20


Samples x[i], i ∈ 1 . . . N are drawn at random according to the probability distribution p(x).

These samples are used to approximate the integral in equation(2.15) as follows:∫p(x)f(x)dx ≈ 1

N

∑

i

f(x[i]) ;x[i] ∼ p(x) (2.16)

Monte Carlo methods can compute arbitrary integrals to arbitrary precision given suf-

ficient computation. Monte Carlo methods are statistically unbiased, but have finite vari-

ance that decreases as the sample size increases. These methods are shown to converge in

the limit as the number of samples goes to infinity.

2.4.1. Importance Sampling

Often it is not easy to draw samples from the target distribution p(x). Uniformly sam-

pling the support of the target distribution is extremely inefficient and not recommended

unless the target distribution is itself uniform. This is because in high dimensional state

spaces probability mass is typically concentrated in a small region of the state space and a

large number of samples would be required to ensure sampling on those regions.

If the target distribution cannot be directly sampled, but it is possible to evaluate the

density p(x) to within a multiplicative constant then importance sampling may be used.

Importance sampling is a general method for estimating the expectation of a function

(equation(2.15)).

In importance sampling, a simpler probability distribution q(x), also known as the

proposal distribution is used to generate samples. Given that samples are drawn from a

different distribution than the target distribution, p(x), values of x where the proposal

distribution q(x) is greater than p(x) will be over-represented (i.e. more samples than are

justified under p(x) will be drawn for these values), and values of x where q(x) is less than

p(x) will be under-represented. To account for this discrepancy, over-represented samples

are down-weighted with respect to under-represented samples. These weights, known as

importance weightsw[i], i ∈ 1 . . . N , are computed as a ratio between the value of x evaluated

at the true distribution and the proposal distribution as follows:

w[i] =p(x[i])

q(x[i])(2.17)

Using importance sampling, the integral in equation(2.15) is computed as follows:∫p(x)f(x)dx =

∫q(x)

p(x[i])

q(x[i])f(x)dx ≈ 1

N

∑

i

p(x[i])

q(x[i])f(x[i]) ;x[i] ∼ q(x) (2.18)

It is important that q(x) have support wherever p(x) is non-zero, or the importance weight,

p(x)/q(x), can diverge. In high dimensional state spaces importance sampling may per-

form poorly because a large number of samples may be required to get a representative

[Draft: 19 October 2004—19:11 ] 21


1. Compute cumulative mass function Q2. q0 = 03. for i = 1 to N do4. qi = qi−1 + w[i]

5. end6. Resample N particles7. u0 ∼ uniform(1, 1/N)8. j = 0, c = 09. for i = 1 to N do

10. c = c+ 1/N11. while c > qi do12. j = j + 113. end14. sample sj15. set weight for sj to 1/N16. end

Table 2.1. Algorithm for systematic resampling

sample from p(x) unless q(x) is a good approximation of p(x), and because importance

weights are likely to have a high variance and hence only a few samples will dominate

the estimate. A large number of samples are thus effectively removed from the estimation

process since their importance weights result in a numerically insignificant contribution to

the estimate in equation (2.16).

2.4.2. Resampling

Importance sampling could potentially be used for a dynamic system by multiplying

the importance weights over time. In (Kong et al., 1994) and (Doucet et al., 1999), the

authors show that the variance of the importance weights increases over time.

To avoid the degeneracy introduced by high variance importance weights, samples

with low importance weights are eliminated and samples with high importance weights

are duplicated. This technique is known as resampling. Resampling associates with each

sample x[i] a number of children ni, ni ∈ 0 . . . N , such that∑i ni = N . There are numerous

resampling algorithms in the literature (Gordon et al., 1993b; Kitagawa, 1993; Doucet, 1998;

Carpenter et al., 1999; Liu and Chen, 1998; Crisan and Lyons, 1997). In all our experiments

we use systematic resampling (Kitagawa, 1993; Liu and Chen, 1998; Crisan and Lyons, 1997),

which guarantees that ni is within ±1 of E (Xi]).

Table 2.1 shows the algorithm for systematic resampling. The algorithm stars by com-

puting a cumulative mass function of the normalized weights of all the particles taken in a

random order.

[Draft: 19 October 2004—19:11 ] 22


2.4.3. Classical Particle Filter

As discussed in 1.2.3 a particle filter (Metropolis and Ulam, 1949; Gordon et al., 1993a;

Kanazawa et al., 1995) is a Monte Carlo approximation of the posterior in a Bayes filter.

The advantages of particle filters are:

• It is an anytime algorithm where the accuracy of the estimate can be made to match

available computation

• Converges to the true distribution in the limit

• Can represent non-parametric distributions

• Can track non-linear non-Gaussian dynamic systems

As discussed in 1.2.3, it is difficult to draw samples from p(st | z1:t); instead, samples are

drawn from a more tractable distribution, q(.), called the proposal, or importance, distri-

bution. Each particle is assigned a weight, w[i] to account for the fact that the samples

were drawn from a different distribution (Rubin, 1988; Rubinstein, 1981). There are a large

number of possible choices for the proposal distribution, the only condition being that its

support must include that of the posterior. The common practice is to sample from the

transition probability, p(st | st−1), in which case the importance weight is equal to the

likelihood, p(zt | st), as shown below:

w[i]t ∝ p(x

[i]t | z1...t)

q(x[i]t | x[i]

t−1, z1...t) p(x[i]t−1 | z1...t−1)

=p(zt | x[i]

t , z1...t−1) p(x[i]t | x[i]

t−1) p(x[i]t−1 | z1...t−1)

q(x[i]t | x[i]

t−1, z1...t) p(x[i]t−1 | z1...t−1)

=p(zt | x[i]

t , z1...t−1) p(x[i]t | x[i]

t−1)

q(x[i]t | x[i]

t−1, z1...t)(2.19)

A common problem with particle filters is trajectory depletion. Trajectory depletion is

caused/aggravated by peaked measurement likelihood when the measurement appears

in the tail of a prior. This is shown in figure 2.2. Only a few samples (that may be poor

estimates of the true posterior) are assigned a high weight. On resampling only these

samples survive and the filter estimate may diverge. A poor proposal distribution (i.e. a

proposal distribution that is very different from the posterior distribution) can also result

in trajectory depletion.

[Draft: 19 October 2004—19:11 ] 23


1. t = 0 ; i = [1 . . . N ]

2. d[i]0 ∼ r(d0) π(d0)

3. x[i]0 ∼ p(x0 | d0) ; i = [1 . . . N ]

4. ∀t > 05. for i = 1 to N do6. sample d

[i]t ∼ p(Dt|d[i]

t−1)

7. sample x[i]t ∼ p(Xt|x[i]

t−1, d[it )

8. set w[i]t = p(zt|x[i]

t )9. end

10. predicted state = {d[i]t , x

[i]t }i=[1...N ]

11. for i = 1 to N do12. pick 〈d[i]

t , x[i]t 〉 from predicted state with probability proportional to w[i]

t

13. endTable 2.2. The algorithm for fault monitoring with a classical particle filter.

Figure 2.2. Figure depicting trajectory depletion. The prior is shown as a blue dot-ted line and the likelihood as a red solid line. Particles are shown in black. Only afew samples are assigned a high weight in this scenario. This results in a depletionof independent particle trajectories on resampling.

2.4.4. Effective Sample Size

The effective sample size is a measure of the accuracy with which statistics may be ob-

tained from the approximate posterior distribution computed by a particle filter. The effec-

tive sample size provides an estimate of the performance of the particle filter. Informally it

may be thought of as effectively how many particles the filter actually has. The larger the

effective sample size the more reliable the filter is.

The effective sample size is defined as the sample size that would be required for a

random sample from the target distribution to achieve the same estimating precision as

the particle filter. Since each approximation will estimate some statistics well and others

poorly, the effective sample size depends on the statistic being estimated.

[Draft: 19 October 2004—19:11 ] 24


Often the statistic of interest is the posterior expectation:

E(xt) =

∫f(xt)p(xt | z1,... ,t)dxt (2.20)

Let the particle filter estimate of the posterior mean, µt and variance σ2t be µt, and σ2

t

respectively.

µt =

N∑

i=1

w[i]t s

[i]t (2.21)

and,

σ2t =

N∑

i=1

w[i]t (s

[i]t )2 − µ2

t (2.22)

The mean of f(xt) using a simple random sample of size Neff from p(xt | z1,... ,t) is µt

and the variance is σ2t /Neff

In (Carpenter et al., 1999) the authors present a method to compute the effective sam-

ple size based on the classical analysis of variance technique:

1. Collect data from M independent runs, each with N samples in the particle filter

being analyzed.

2. For each run j; j ∈ {1, . . . ,M}, compute the particle based estimate of the mean

µ(j)t and variance σ2(j)

t .

3. In addition, compute the average mean, µt, and variance, σ2 over the M runs put

together.

4. The effective sample size Neff is obtained by equating the two estimates of the

variance of µt, the between sample variance (or the variance between different runs

of the particle filter) and the other based on the variance that an estimate of the

mean based on a simple random sample would have:

1

M

M∑

j=1

(µ(j)t − µt)2 =

σ2

Neff(2.23)

Solving for Neff , gives Neff = Mσ2

∑Mj=1(µ

(j)t −µt)2

It should be noted that the effective sample size does not provide any estimate of the

convergence to the true distribution. All it provides is an estimate of the convergence to

some distribution. Hence, a noisy biased filter may have a large effective sample size, but

it will be a poor approximation of the true posterior. Also, computing effective sample size

is expensive since it requires M runs of the filter and hence is not a viable online method

for estimating the performance of filters.

[Draft: 19 October 2004—19:11 ] 25


2.4.5. Auxiliary Particle Filter

The Auxiliary Particle Filter (APF) (Pitt and Shephard, 1999) is a method that uses

future data to bias sampling to regions with high posterior likelihood. Subsequent variants,

such as the Unscented Particle Filter (van der Merwe et al., 2000), present alternate methods

for estimating the posterior likelihood. The bias is compensated for by introducing an

additional term in the importance weight.

In the APF, particles (trajectories) {x[i]t−1} that are expected to be the most promising at

the next time step are propagated to the next time step t. Particles are evaluated based on

their predictive likelihood p(zt | xt−1).

Typically the analytical computation of the predictive likelihood is intractable and an

approximation is needed.

p(zt | xt−1) =

∫p(zt | xt)p(xt | xt−1)dxt (2.24)

(Pitt and Shephard, 1999) describe a simple approximation where:∫p(zt | xt)p(xt | xt−1)dxt ≈ p(zt, µxt|xt−1

) (2.25)

where µxt|xt−1is the mode or mean of p(xt | xt−1). At times, this approximation can be

poor (especially when the state space is hybrid) and may lead to degradation of perfor-

mance over the classical particle filter by biasing sampling toward uninteresting regions.

Another approach is to use a second-stage Monte Carlo method for each particle, but

this is computationally intensive and can introduce an additional variance.

2.4.6. Unscented Particle Filter

The Unscented Particle Filter (UPF) (van der Merwe et al., 2000) is a form of Auxiliary

Particle Filter that uses a UKF to approximate the predictive likelihood in equation 2.24.

For each particle, the UPF computes a posterior mean and covaraiance using a UKF.

The prediction step in the UKF uses only prior information, but the measurement update

step incorporates the measurement and updates the predicted mean and covariace to ob-

tain the posterior mean and covaraince. Particles generated based on a proposal distribu-

tion generated from the UKF posterior are more likely to generate particles that are closer to

the measurement, and the particle filter is thus less likely to suffer from particle starvation.

A disadvantege of the UPF is that it is computationally expensive. A UKF must be

computed at each time step for each particle, which is an O(n2xnz) computation per parti-

cle, where nx is the dimensionality of the state space and nz is the dimensionality of the

measurement space. In section 6.1 we present enhancements to this algorithm.

[Draft: 19 October 2004—19:11 ] 26


2.4.7. Factored Particle Filter

In (Ng et al., 2002) the authors utilize a factored representation of a purely discrete

state space, where subsets of variables are approximated as being independent. Factored

Particle filters (FPF) model the state space as weakly interacting clusters of variables that

are nearly independent of each other.

Similar to the Boyen-Koller (BK) (Boyen and Koller, 1998, 1999) algorithm, variables

with weak interactions are approximated as being independent. The belief state is parti-

tioned into k subsets of beliefs each over a different cluster of variables, C = {c1, . . . , ck}.The belief state about the entire system is represented as a set of factored beliefs over lo-

calized subsystems. The global or joint belief state is represented as a product of marginal

beliefs over the subsets:

p(st | z1:t) ≈∏

C

p(sct | z1:t) (2.26)

where sct represents the partial state represented by the variables in cluster c.

FPF maintains an approximation of the factored belief over each subsystem using a

set of factored particles, {x[i]t,c, . . . , x

Nct,c }, where Nc is the number of particles for cluster c;

c ∈ {c1, . . . , cK}. Factored particles specify values for a subset of the state variables. Fac-

tored particles may be thought of as maintaining an approximate marginal distribution

over variables in the set. Equation (2.26), representing the Boyen-Koller approximation,

can be further approximated as:

p(st | z1,... ,t) ≈∏

C

1

Nc

Nc∑

i=1

δ(st,c − x[i]t,c) (2.27)

An advantage of using FPF is that it increases the diversity of the sample set. In ad-

dition, sampling from p(xc1)p(xc2) can reduce the Monte Carlo error over sampling from

p(xc1, xc2).

Projection to a factored space, followed by propagation and then a global join (to get a

joint distribution over the entire state space), is done at each time step. Projection and join

are done regardless of the belief and number of particles in the state space.

Chapter 5 presents the VRPF algorithm that uses an explicit bias-variance tradeoff to

generalize more selectively and efficiently.

2.4.8. Mixture Proposal Distribution

The mixture proposal distribution presented in (Thrun et al., 2000) improves posterior

particle estimates, particularly in situations where sensors are highly accurate. The pro-

posal distribution is a mixture of the next state distribution, q ∼ p(st | st−1), and a sensor

[Draft: 19 October 2004—19:11 ] 27

2.5 SMOOTHING

based distribution, q ∼ p(zt | st):

(1− α)q + αq (2.28)

where, 0 ≤ α ≤ 1. q samples possible states give the sensor measurements and assigns

importance weights based on the prior belief.

One of the issues with this approach is that it is often not easy to sample from the

sensor model. Although, when possible, this approach is useful in fault diagnosis where

sensor measurements can be very informative and can narrow down the space of possible

hypothesis considerably.

2.5. Smoothing

In fault diagnosis, sometimes it is important to estimate past states, p(st−l | z1:t); l > 0,

given the measurements up to the current time. For example, it may be important to infer

whether a steering potentiometer on a robot broke l timesteps ago, given the current sensor

reading that shows an erroneous heading. This approach of inferring states l timesteps in

the past is called either fixed-lag smoothing or hindsight.

Smoothed estimates are available after a delay of l timesteps. If the delay is acceptable,

smoothing provides superior estimates to filtering.

The filtering distribution, p(st | z1:t), is a marginal of the posterior distribution, p(s0:t |z1:t). The recursive filter as defined in equation(2.4) is as follows:

p(st | z1:t)

= ηt p(zt | st)∫p(st | st−1) p(st−1 | z1:t−1) dst−1

(2.29)

Bayesian filtering provides an optimal estimate of the posterior, at time t, given the se-

quence of data [z0, . . . , zt] up to time t. It is, not however, the optimal estimate, given

future data. A batch estimate of a sequence of data from time [0 . . . T ], where T > t, would

provide a more accurate estimate.

Smoothing is a backward process of updating past estimates based on future data,

i.e. p(st | s0:t+l); l > 0. It achieves a balance between the efficiency of filtering and the

optimality of batch methods. Bayesian smoothing is defined as follows:

p(st | z1:t+l) = γt

∫p(st+1 | st)p(st | z1:t)p(st+1 | z1:t+l)dst+1 (2.30)

2.5.1. Kalman Smoothing

Kalman smoothing is the best known, but may be used only in filters that linearize the

state equations (such as the Kalman filter, and Extended Kalman Filter).

[Draft: 19 October 2004—19:11 ] 28

2.5 SMOOTHING

Linearization of the measuremtnes is done after the state has been estimated using

more data, which reduces the variance of the state estimate before linearization. In addi-

tion, including a larger amount of data results in a posterior that is more Gaussian. Hence

the Gaussian approximation to the posterior is more accurate.

For a lag of L timesteps, the computational requirements for Kalman filtering vs.

smoothing are O (N2x) and O (N2

xL) respectively and the space requirements are O (Nx)

and O (NxL) respectively.

The Rauch-Tung-Striebel (RTS) algorithm (Rauch et al., 1965) performs fixed-interval

offline smoothing.

2.5.2. Expectation Propagation

Expectation Propagation (Minka, 2001) is an improvement over Kalman smoothing

that can be used with methods that linearize the measurement equations (such as EKF,

UKF). Expectation-propagation re-linearizes the measurement equations until a globally-

stable solution is reached. There are fixed lag variants for linear-Gaussian models, such as

the Variable State Dimension filter (VSDF) (McLauchlan, 1999).

2.5.3. Monte Carlo Smoothing

There are also methods in the literature for Monte Carlo smoothing. Forward particle

filtering is performed up to time t + l. Particles at time t are then re-weighted and resam-

pled, based on a sampled particle from time t + 1 to obtain an updated estimate st|t+1

(Doucet et al., 2000).

Particles in a particle filter represent trajectories through state space. Resampling, al-

though essential for reducing the variance of importance weights, also reduces the number

of independent trajectories, (or the effective sample size (Carpenter et al., 1999)). The aim

of Monte Carlo smoothing is to ensure long independent trajectories, else the particle filter

can often degenerate to tracking a single hypothesis. It does so by re-weighting particles

to assign higher weights to particles that are more likely to survive, given future data.

2.5.4. Assumed Density Smoothing

There are also methods that re-weight particles at time t, using a density fitted to the

particles at time t + 1 (Koller and Fratkina, 1998). Forward sampling is then performed

using an updated proposal distribution q(st|t+1) = p(st+1 | st|t+1), based on the smoothed

state estimate. Expectation propagation is also an assumed density smoothing method.

[Draft: 19 October 2004—19:11 ] 29

CHAPTER 3

Related Work

THIS chapter discusses previous work that is relevant to this dissertation. Re-

lated work in the area of hybrid state estimation is presented in section 3.1.

Diagnosis is one of the classical problems in Artificial Intelligence. Section 3.2

discusses some of these classical non-temporal diagnosis methods. Section 3.3

covers qualitative model-based systems. This is the state-of-the-art in currently deployed

systems. Related work in the area of robot fault tolerance is presented in section 3.4. In

addition, relevant hierarchical approaches are discussed in section 3.5.

3.1. Related Work in Hybrid State Estimation

There is a rich literature of work in the area of probabilistic state estimation. Histori-

cally, most of the research has been done in the field of signal processing. In robotics, great

strides have been made in the area of mobile robot localization and mapping using prob-

abilistic methods (Fox et al., 1999; Thrun et al., 2001). Most of the work in mobile robot

localization has been in continuous state spaces. This section focuses on relatively recent

work in the area of hybrid state estimation, since this thesis uses a hybrid representation

for fault diagnosis.

3.1.1. Multiple Model Methods

Several approaches were developed in the 1960s that run a bank of independent Kalman

Filters to track multiple candidate hypotheses, i.e., multiple states that are possible given

the uncertainty. Filters with high residual (poor match with measurements) are pruned

away. This approach is known as the static (non-switching) Multiple Model (MM) ap-

proach in the target tracking literature.

In (Roumeliotis et al., 1998) this approach is applied to robot fault detection and iden-

tification. A probabilistic combination of the residual from a bank of Kalman filters is used

3.1 RELATED WORK IN HYBRID STATE ESTIMATION

to determine the correct fault state. In a later extension, a backpropagation Neural Network

was used to process these residuals and identify the fault (Goel et al., 2000). In general, this

approach is not very scalable since all the filters have to be run simultaneously. In addi-

tion, this approach does not allow the system to dynamically transition between various

operating and fault modes.

3.1.2. Mixture Component Pruning Methods

These algorithms drop the Gaussians with high residual. They keep M Gaussians

with the highest posterior probabilities and prune the rest. MaKSI (Washington, 2000) is

an example of this approach. This method (Washington, 2000) uses a Partially Observable

Markov Decision Process (POMDP) to represent the discrete state model. The method does

not explicitly compute a POMDP policy or actively act in response to the estimated belief.

In fact there is no uncertainty associated with actions. Actions are passively observed and

result in discrete state transitions. Pruning mixture components is not very robust to noise.

Pruning is done based on the likelihood from a single sensor measurement and the correct

hypothesis may easily get pruned away.

3.1.3. Probabilistic Hybrid Automata

(Hofbaur and Williams, 2002) use Probabilistic Hybrid Automata (PHA), which ex-

tends hidden Markov Models (HMMs) with continuous dynamic models. In PHA. each

discrete state is associated with a set of equations describing continuous state dynamics.

In addition to probabilistic discrete state transitions, as in a HMM, PHA allows discrete

state transitions to be conditioned on the continuous state of the system. These conditional

transitions are implemented though Boolean expressions over the continuous state of the

system called guard conditions. This is a also a mixture component pruning method that

maintains only a finite set of the most likely hypotheses.

3.1.4. Rao-Blackwellized Particle Filter

The approach presented in (de Freitas, 2002) uses a RBPF for FDI. In (Morales-Mendendez

et al., 2002) this approach is extended to include one-step look-ahead, which significantly

improves performance. Funiak and Williams (Funiak and Williams, 2003) combine this

approach with Probabilistic Hybrid Automata (Hofbaur and Williams, 2002). All these ap-

proaches, however, are restricted to linear Gaussian models for each fault state. Hutter and

Dearden (Hutter and Dearden, 2003) further extend this approach to moderately non-linear

models by using UKFs (Julier and Uhlmann, 1996) instead of Kalman filters. Although the

[Draft: 19 October 2004—19:11 ] 31

3.3 QUALITATIVE MODEL-BASED METHODS

conditional density is no longer exact and convergence guarantees can no longer be made,

it is found to work well in practice. In addition, like a classical particle filter, all these

approaches suffer from particle starvation when the discrete state space is large.

3.1.5. Particle Boosting Methods

(Dearden and Clancy, 2002) address the issue of low probability faults by using an

“oracle” to provide a set of candidate states that the system might end up in, given the

current distribution over the state space. On resampling, they ensure that there are always

some particles in the states provided by the oracle. For sparse discrete state transitions,

they do a one step look-ahead to determine these states. This can be expensive in large

state spaces.

3.2. Non-temporal Methods

Diagnosis is one of the fundamental problems in Artificial Intelligence. But most clas-

sical diagnosis work is non-temporal. In other words, these methods use values of vari-

ables (sensor measurements) at a single instance in time to detect faults. In contrast, one of

the main aims of this thesis is to diagnose faults that can be detected only by observing the

time varying values of key variables. Hence, while interesting, most of these approaches

are insufficient.

Geometrical classifiers divide the space of sensor measurements into different nominal

and fault states (Dasarthy, 1990). These methods typically do not take temporal informa-

tion into account, making them unsuitable for robot diagnosis.

Rule-based systems, such as MYCIN (Shortliffe, 1976), and a number of expert sys-

tems (Harmon et al., 1988) have been extensively used in diagnosis. Relevant literature that

covers the application of these methods specifically to robot fault diagnosis is discussed in

section 3.4.3.

Bayesian belief networks have also been used for diagnosis (Szolovits and Pauker,

1978, 1993). Most applications of belief nets do not include temporal dependencies. A re-

cent approach extends these ideas to dynamic systems using Dynamic Bayesian Networks

(Lerner et al., 2000).

3.3. Qualitative Model-based Methods

Model-based reasoning (Brown et al., 1982) inspired an entire field of research in di-

agnosis (de Kleer and Williams, 1987; McIlraith, 1998; Ghallab, 1985). Model-based ap-

proaches are those that reason about explicit models of the system. Model-based diag-

nosis systems have been extended to handle temporal constraints in dynamic systems,

[Draft: 19 October 2004—19:11 ] 32

3.4 RELATED WORK IN ROBOTICS

XDE (Hamscher, 1991), SIDA (Guckenbiehl and G, 1990), GMODS (Holtzblatt et al., 1991),

SHERLOCK (de Kleer and Williams, 1989) and DIANA (Dague et al., 1990).

To address continuous dynamic systems, systems such as, e.g TEXSYS (Glass et al.,

1991), MIMIC (Dvorak and Kuipers, 1989) and the Livingstone system (Williams and Nayak,

1996) use qualitative models. State-estimation in Livingstone is a search over the transi-

tions of the hardware model to find a state that is consistent with the sensor measurements.

The state constraints are formalized as logic formulae defined over the space of discretized

variables representing the state of the system. Livingstone assumes that there is no un-

certainty in the discretized sensor measurements and uses them as hard constraints. This

results in a null hypothesis, or incorrect identification, if the true trajectory is not the most

likely at any point in the history of the sensor measurements.

Later enhancements made in Livingstone II (Kurien and Nayak, 2000) regenerate past

hypotheses based on saved history. Livingstone was successfully tested on the Deep Space

I spacecraft, but when tested on a rover it triggered numerous false positives in the pres-

ence of noise (Verma, 2001).

3.4. Related Work in Robotics

Although almost every robot generally has to address some set of faults, in general

fault diagnosis on robots is implemented in a manner that may be described as exception

handling in the control software.

A workshop on Robot Fault Diagnosis at the 2004 IEEE International Conference on

Robotics and Automation (ICRA) brought together researchers and practitioners that have

relevant long-term experience with challenges faced in application domains, approaches

that have been successful, and unresolved issues that need to be addressed. (Carlson, 2004)

provides a 26 page report on this workshop.

The section provides an overview of research in robot fault diagnosis. Some of the

approaches described in this section, such as Safety Engineering and Redundancy (sub-

section 3.4.1 and 3.4.2) are not really fault detection and identification approaches, but are

included here since they are commonly used for addressing robot faults..

3.4.1. Safety Engineering

Pre-emptive engineering of the system (Scheding, 2000; Silberman, 1999) is the most

common approach for addressing faults in robotics. Scheding presents an application of

this approach at an autonomous port in Australia that uses crane robots and container

[Draft: 19 October 2004—19:11 ] 33

3.4 RELATED WORK IN ROBOTICS

carrying mobile robots. Although this approach can be fairly successful, along with un-

certainty flexibility and robustness to uncertain environments are often engineered out of

the system. For example, in (Scheding, 2000) there is a extremely well-defined protocol

for interaction with ships and no human is allowed within the perimeter of the port. If an

entry is detected, the entire port shuts down.

It should be pointed out that the work in this thesis is complementary to safety engi-

neering. Fault diagnosis is not a replacement for careful design and software verification.

But, numerous examples (including a long list of spacecraft that undergo rigorous safety

engineering) have shown that it is a necessary complement. On the other hand, recovering

from faults is a computationally intensive and time consuming process, and hence even

with 100% detection accuracy it is still important to design systems to minimize faults.

3.4.2. Redundancy

Another common approach, which may be considered a variant of safety engineering,

is redundancy. Through redundancy, robot sensors, components, or even entire robots (as

in multi-robot systems) are expendable since the same functionality is provided by more

than one component or robot.

Fault tolerance is often listed as a motivating factor in multi-robot systems (Parker,

1998). A relevant variant on traditional multi-robot approaches in that of modular repairable

robots (Bererton and Khosla, 2002), where components are replaceable. Some robots in such

approaches may be considered redundant since they may take on the task of repairing

other robots.

In the case of sensors redundancy is often not implemented as a selection between

sensors, but instead integration of sensor measurements is performed (Murphy and Hersh-

berger, 1996). Common approaches are grid-based voting methods (Djath et al., 2000; Soika,

1997) and Bayesian information integration.

3.4.3. Expert Systems and Fault Tree Analysis

Expert systems have been extensively applied to fault diagnosis. One of the earli-

est approaches in this area was developed to detect navigation faults (Stuck, 1992, 1995).

Classical approaches use rule (or knowledge) bases. Variants that use fuzzy logic and neu-

ral nets have been developed to handle noise (Yan, Ota, Nakamura, Arai and Kuwahara,

Yan et al.; Lamine and Kabanza, Lamine and Kabanza; Hamilton, Lane, Taylor and Brown,

Hamilton et al.). (Simmons, 1994) applies these methods to fault detection on the robots

Xavier and Ambler.

[Draft: 19 October 2004—19:11 ] 34

3.5 RELATED WORK IN HIERARCHICAL METHODS

An approach for diagnosing robot manipulators based on expert systems and fault

tree analysis is presented in (Visinsky et al., 1995). Fault tree analysis is extensively used for

diagnosing industrial systems and there are a number of commercial packages available.

Fault tree analysis is based on the assumption that Boolean logic can capture inter-

actions between component failures. Probabilities may be used to represent component

failures. Interaction between failures are captured using a tree structure of Boolean op-

erators that decompose high level failures into combinations of base level events. Fault

tress provide a hypothesized combination of events (minimal cut set) leading up to a given

failure. An extension that uses a fuzzy Markov models of the fault tree is presented in

(Leuschen, Walker and Cavallaro, Leuschen et al.).

Another example is the SFX-EX architecture that uses a partial causal model of sensor,

environment, and task interactions to diagnose faults (Murphy and Hershberger, 2000).

This approach has also been applied to multiple robots (Long, Murphy and Parker, Long

et al.).

3.4.4. Partially Observable Markov Decision Process

An approach that addresses uncertainty and uses a POMDP representation is pre-

sented in (Fernandez, 2000). This approach focuses on recovery from discrete fault events

and was demonstrated in the context of indoor mobile robot navigation.

3.5. Related work in Hierarchical Methods

Hierarchical methods have been used in the areas of planning, classification, and

learning. These methods aim to utilize structure in the domain to make a large state space

tractable. Hierarchical methods in diagnosis and classification are discussed below since

they are the most relevant to this thesis.

3.5.1. Structural Abstraction

In diagnosis, hierarchical methods have traditionally been used for structural abstrac-

tion, where components are aggregated to describe the system at varying levels of system

detail. A hypothetical diagnosis in such a system may proceed by first determining that

the fault is in the power subsystem, and then proceed to find the particular battery that is

at fault, etc. An example of structural abstraction applied to model-based diagnosis may

be found in (Chittaro and Ranon, 2004).

[Draft: 19 October 2004—19:11 ] 35

3.6 SUMMARY

3.5.2. Functional Abstraction

In this approach to diagnosis, a hierarchical sequence of tests is performed to search

for the correct hypothesis. Each layer in the hierarchy performs a test. The outcome of

each test determines which branch of the hierarchy to search (test) further (Koppolu and

Chatterjee, 2001).

3.5.3. Hierarchical Clustering

Another area in which hierarchies are commonly used is in clustering. Hierarchical

clustering recursively clusters data points into a partition of clusters, each of which is itself

hierarchically clustered.

The two main approaches for building hierarchical clusters are agglomerative and di-

visive. Agglomerative hierarchies are built bottom-up, starting with the leaf nodes and

merging nodes until a root node is reached. Divisive hierarchies are built top-down by

recursively partitioning the root node (consisting of all the nodes) until leaf nodes are

reached.

The hierarchy used in this thesis clusters states with similar dynamics together in a

manner similar to hierarchical clustering. The purpose of clustering in this thesis is very

different from typical applications of hierarchical clustering. Continuous sensor measure-

ments are not clustered to assign discrete fault labels, as is the case with the approach

presented in section 3.2. Instead, the hierarchy is used to cluster fault models into a multi-

resolution hierarchy a priori to enable tracking at variable resolutions. Further details are

provided in chapters 5 and 7.

3.6. Summary

Related work in hybrid state estimation (section 3.1) does not provide a method that

enables efficient tracking of general nonlinear, non-Gaussian systems. Classical diagnosis

approaches from section 3.2 cannot detect faults that require tracking the dynamics of the

system over time. Qualitative model based methods do not provide a representation that

can easily handle noisy hybrid systems.

In robotics, there are very few examples of systems that autonomously detect com-

plex faults. Safety engineering and redundancy are the most commonly used approaches.

Expert systems have also been used extensively in robotic systems for fault detection and

identification, but these systems too cannot track faults that require switching dynamics.

This dissertation aims to address the need for an efficient fault monitoring approach

that can detect robot faults that require tracking dynamics over a period of time.

[Draft: 19 October 2004—19:11 ] 36

CHAPTER 4

Decision Theoretic Particle Filter

THIS chapter presents the Decision Theoretic Particle Filter (DTPF) (Verma et al.,

2001; Thrun et al., 2001), which generates particles by factoring in cost to effi-

ciently track rare high risk events. Section 4.1 presents a particle filter that

generates particles factoring in a risk-function, section 4.2 presents an ap-

proach for computing a risk function, and section 4.3 presents experimental results.

Faults are low-probability, high-cost events. The classical particle filter (CPF) gener-

ates particles proportional only to the posterior probability of an event. It is insensitive to

costs that might arise from the particle approximation. Monitoring a system to detect and

identify faults based on a CPF therefore requires a very large number of particles, making

it computationally expensive. By taking cost of missing a diagnosis into account the DTPF

makes it more likely that high-cost, low probability events will have samples generated

to represent them, which means that these events can be reliably tracked with a smaller

number of particles than the CPF.

In our representation, the state estimate is probabilistic, i.e. a probability distribution

over states (also known as a belief) is provided. A full decision-theoretic approach would

require computing the cost for every every possible distribution over the belief state, which

is computationally intractable. Hence, an approximation that maintains a single distribu-

tion over states, which is the difference between the expected future cost from taking differ-

ent actions from the same state is used. Intuitively, when the difference in cost from taking

different actions from a state is high, it is more important to track that state accurately so

that the optimal action may be selected. If an important state is not tracked accurately,

an action would be selected based on an erroneous state estimate. Although the action

selected may be optimal for the erroneous state, it is likely to be sub-optimal for the true

state and will result in a high cost. The difference in cost from taking different actions is

called the “risk”, r(x). For efficiency, the risk is defined as a function of the state alone and

4.1 RISK-SENSITIVE SAMPLING

not the state and control. Since fault monitoring is typically passive and merely observes

measurements and control to provide a state estimate, this is reasonable.

The DTPF, which is a technique for maintaining particles according to a distribution

proportional to r(x)p(x) is presented in section 4.1. A heuristic approximation for gener-

ating the risk function from the instantaneous future loss, C(x, u), is presented in section

4.2.

Let C(x, u) ∈R be a cost function that assigns real-valued costs to states and control.

From a decision theoretic point of view, the goal or risk sensitive sampling is to gener-

ate particles that minimize the cumulative increase in cost due to the particle approxima-

tion. The classical particle filter is modified to generate particles in a risk-sensitive manner,

where risk is a function of the cost C. An approximate risk function is defined that ap-

proximates the cumulative expected cost relative to tracking individual states. This risk

function is calculated using value iteration.

Related approaches include prior boosting (Rubin, 1987) and prior biasing (Dearden and

Clancy, 2002). In prior boosting, kN samples are drawn in the update stage instead of the

usual N samples. On resampling only N samples are maintained. This approach produces

results similar to a classical particle filter with kN samples and has computational require-

ments comparable to a filter with nearly kN samples and therefore does not adequately

address our problem – of improving tracking of low-probability high-risk states without

unduly increasing computation. Prior biasing always generates some samples in high-risk

states, but requires an oracle to determine which states are high-risk. The risk-function

proposed in this chapter could potentially be used as this oracle.

4.1. Risk-sensitive Sampling

Risk-sensitive sampling generates particles factoring in a risk function, r(x). The only

constraint on this function is that it be positive and finite everywhere. Not all risk functions

are equally useful. Hence, it is important to derive the “right” risk function. Decision

theory gives us a framework for deciding what the “right” action is in any given state.

By considering approximation errors due to Monte Carlo sampling using decision theory

and making a sequence of rough approximations the risk function r(x) is obtained. This is

discussed further below.

Decision-theoretic particle filters generate samples that are distributed according to:

γtr(dt)p(xt, dt | zt) (4.1)

Here γt = [∑d r(d)p(d | zt)]−1 is a normalizing constant that ensures that the term in 4.1

is a probability distribution. Thus, the probability that a discrete state sample d[i]t is part

[Draft: 19 October 2004—19:11 ] 38

4.2 THE RISK FUNCTION

of the state at time t is not only a function of its posterior probability, but also of the risk

r(d[i]t ) associated with that sample.

Sampling from equation 4.1 is achieved by the following two modifications to the

basic particle filter algorithm in table 2.2. First, the initial set of particles (line 2 in table 2.2)

d[i]0 is generated from the distribution:

γ0 r(d0) p(x0d0) = γ0 r(d0) p(x0 | d0) p(d0) (4.2)

and recursive prediction (line 6 in table 4.1) is replaced with:

d[i]t ∼

r(d[i]t )

r(d[i]t−1)

p(Dt|d[i]t−1) (4.3)

The complete decision-theoretic particle filter for fault monitoring is presented in table 4.1.

We conjecture that this simple modification results in a particle filter with samples dis-

tributed according to γtr(dt)p(dt, xt | zt). Our conjecture is obviously true for the base

case t = 0, since the risk function r was explicitly incorporated in the construction of D0

(equation 4.2). By induction, let us assume that the particles at time t − 1 are distributed

according to:

γt−1 r(dt−1) p(dt−1, xt−1 | zt−1) (4.4)

Then Line 6 of the modified algorithm generates:

d[i]t ∼ γt−1

r(dt)

r(dt−1)r(dt−1) p(dt | dt−1) (4.5)

= r(dt) p(dt | dt−1) (4.6)

Line 7 gives us:

x[i]t ∼ γt−1 r(dt) p(xt | dt, xt−1, z

t−1) p(dt | dt−1) (4.7)

Particles generated in Line 8 are distributed as follows:

w[i]t γt−1 r(dt) p(xt | xt−1, dt, z

t−1) p(dt | dt−1) (4.8)

= γt−1 r(dt) p(zt | xt) p(xt | xt−1, dt) p(dt | dt−1) (4.9)

Using equation 1.5 this term is, up to a normalizing constant γtηtγ−1t−1, equivalent to the

desired distribution in (4.1), which proves our conjecture. Thus, the DTPF successfully

generates samples from a distribution that factors in the risk r.

4.2. The Risk Function

The risk function determines how important it is to track a given state x. Our approach

is based on the assumption that there are two possible situations, one in which the state is

tracked well, and one in which the state is tracked poorly, In the first situation, we assume

that any controller will basically choose the right control, whereas in the second situation,

[Draft: 19 October 2004—19:11 ] 39

4.2 THE RISK FUNCTION

1. t = 0 ; i = [1 . . . N ]

2. d[i]0 ∼ γ0 r(d0) p(d0)

3. x[i]0 ∼ p(x0 | d0) ; i = [1 . . . N ]

4. ∀t > 05. for i = 1 to N do6. sample d

[i]t ∼ r(d

[i]t )

r(d[i]t−1)

p(Dt|d[i]t−1)

7. sample x[i]t ∼ p(Xt|x[i]

t−1, d[i]t )

8. set w[i]t = p(zt|x[i]

t )9. end

10. predicted state = {d[i]t , x

[i]t }i=[1...N ]



t

13. endTable 4.1. The algorithm for a decision-theoretic particle filter.

it is reasonable to assume that the controls are selected randomly. To complete this model

it is assumed that that with small probability, the state estimator might move from a “well-

tracked” to “lost-track” and vice versa.

Using these assumptions a Markov Decision Process (MDP) is formulated to model

the effect of tracking accuracy on the expected cost. The MDP is defined over an aug-

mented state space 〈x, c〉, where c ∈ 0, 1 is a binary state variable that models whether the

estimator tracks the state with sufficient (ct = 1) or insufficient (ct = 0) accuracy. The

various probabilities of the MDP are obtained from known probability distributions based

on the assumption that the tracking accuracy c is conditionally independent of the system

state 〈dt, xt〉. Although most of the equations presented so far have ignored control u, it is

specified here for completeness:

p(〈xt, dt, ct〉 | ut, 〈xt−1, dt−1, ct−1〉) = p(xt, dt | ut, xt−1, dt−1) p(ct | ct−1) (4.10)

p(zt | 〈xt, ct〉) = p(zt | xt) (4.11)

p(〈x0, d0, c0〉) = p(x0, d0) p(c0) (4.12)

C(〈xt, dt, ct〉, ut) = C(xt, dt, ut) (4.13)

The L.H.S. expressions in equations 4.10–4.13 define all the necessary components of the

augmented model. The only unspecified terms on the R.H.S. are the initial tracking prob-

ability p(c0) and the transition probabilities for the state estimator p(ct | ct−1). The former

is set in accordance to the initial knowledge state. It is set to 1 if the initial system state is

known and 0 if it is unknown. For the tracking accuracy, p(ct | ct−1), we adopt a model

[Draft: 19 October 2004—19:11 ] 40

4.3 EXPERIMENTAL RESULTS

where with high likelihood the tracking state is retained (p(ct = ct−1) = 0.95) and with

low likelihood it changes (p(ct 6= ct−1) = 0.05).

The MDP is solved using value iteration (Sondik, 1971). Value iteration uses iterations

of dynamic programming to compute increasingly accurate cost (value) for each state. To

model the effect of poor racking on the control policy, our approach uses the following

value iteration rule (stated here without discounting for simplicity), in which V denotes

the value-function, which maps states to values, and Q is Q-function, which provides the

cost of every action possible from each state:

V (〈d, c〉) =

minuQ(〈d, c〉, u) if c = 1

β[maxuQ(〈d, c〉, u)] + (1− β)[∫Q(〈d, c〉, u)] du if c = 0

(4.14)

Q(〈d, c〉, u) = C(d, u) +

1∑

c′=0

∑

d′

V (〈d′, c′〉) p(c′ | c) p(d′ | u, d) (4.15)

This value iteration rule considers two cases: one when c=1, i.e., the state is estimated

sufficiently accurately, it is assumed that the controller selects actions that minimize cost.

However, if c = 0, in the worst case the controller picks an action that maximizes cost and in

the best case it selects random control actions. These two options are traded off by the gain

factor β, which controls the “pessimism” of the approach. β = 1 suggests that poor state

estimation leads to the worst possible control. β = 0 is more optimistic, in that the control

is assumed to be random. An alternative approach for solving the problem addressed in

this paper would be to analyze the estimation process as a Partially Observable Markov

Decision Process (POMDP) (Astrom, 1965; Sondik, 1971). Unfortunately, existing POMDP

algorithms are extremely computationally demanding.In addition, the risk function r is

modeled as a function of states, and not of beliefs, which suggests an MDP solution.

Finally, the risk-function is defined using the costs of each states computed above.

The risk r is defined as the difference between the value function that arises from accurate

versus inaccurate state estimation:

r(x) = V (x, c = 0)− V (x, c = 1) (4.16)

The value function assigns a real-valued cost to states and control. The control se-

lected, given the exact state, results in the minimum cost. The approximate nature of the

particle representation may result in sub-optimal control and hence increased cost.

[Draft: 19 October 2004—19:11 ] 41


Figure 4.1. Hyperion Robot

4.3. Experimental Results

4.3.1. Faults in a Simulation of the Hyperion Rover

In a simulation of the Hyperion robot (Wettergreen et al., 2002) shown in figure 4.1

faults were explicitly introduced and a sequence of controls and measurements are recorded.

This sequence was then tracked by a DTPF and a classical particle filter (see Appendix A

for simulation details). In the experiment the robot was driven with a variety of differ-

ent control inputs in the normal operation mode. For this experiment, the measurements

were the rover pose (x, y, θ) and steering angle. At the 17th timestep, wheel #3 becomes

stuck and locked against a rock. The wheel is then driven in the backward direction, fixing

the problem. The robot returns to the normal operation mode and continues to operate

normally until the gear on wheel #4 breaks at the 30th time step. Figure 4.3 shows the

performance with a DTPF. The state numbers represent different fault modes, with state 1

corresponding to normal operation. Figure 4.2 shows performance with a CPF. The DTPF

yields superior results to CPF. Even though failures are very unlikely, the DTPF success-

fully identifies them due to the high risk associated with such failures; the CPF essentially

fails to do so. The estimation error is shown in the bottom row of figures 4.3 and 4.2. It is

practically zero for the DTPF when 1, 000 or more particles are used. CPF exhibits non-zero

error even with 100, 000 particles.

4.3.2. Stuck Wheel Faults in a Rocker-Bogie Rover Simulation

The second experiment was to diagnose locomotion faults in a physics-based simu-

lation of a six-wheel rocker-bogie rover. Figure 4.4 shows a snapshot of the rover in the

Darwin2K (Leger, 2000) simulator. In simulation stuck wheel faults at each wheel were

modeled. Each of these faults cause a change in rover dynamics as shown in figure 4.5.

[Draft: 19 October 2004—19:11 ] 42


0 20 400

5

101000 samples

0 20 40

0

2

4

6

8

0 20 40

0

0.5

1

Time step −>

0 20 400

5

10

Mos

t Lik

ely

Sta

te

100 samples

0 20 40

0

2

4

6

8

Sam

ple

Var

ianc

e

0 20 40

0

0.5

1

Err

or u

sing

1−

0 lo

ss

Time step −>

0 20 400

5

1010,000 samples

0 20 40

0

2

4

6

8

0 20 40

0

0.5

1

Time step −>

0 20 400

5

10100,000 samples

0 20 40

0

2

4

6

8

0 20 40

0

0.5

1

Time step −>

Figure 4.2. Tracking faults on the Hyperion rover with a classical particle filter

10 20 30 400

5

10

Mos

t lik

ely

stat

e

100 samples

0 20 400

5

10

15

Avg

. sam

ple

varia

nce

10 20 30 400

0.5

1

Med

ian

erro

r (1

−0

loss

)

10 20 30 40−0.1

0

0.1

Time step −>

Err

or v

aria

nce

0 20 400

5

10

1000 samples

0 20 400

5

10

15

0 20 40−1

0

1

0 20 40−1

0

1

Time step −>

0 20 400

5

10

10000 samples

0 20 400

5

10

15

0 20 40−1

0

1

0 20 40−1

0

1

Time step −>

Figure 4.3. Tracking faults on the Hyperion rover with a decision-theoretic particle filter

[Draft: 19 October 2004—19:11 ] 43


Figure 4.4. Six-wheel rocker bogie rover in Darwin2K simulator

60 65 70 75 80 85 90 95 100 105

75

80

85

90

95

100

105

110

X−>

Y−

>

NDRFSRMSRRSLFSLMSLRS

Figure 4.5. Change in rover trajectory induced by independent stuck wheel faults

The Markov model representing the discrete state transitions consists of 7 physical

states. As shown by the HMM representing the discrete state transitions in figure 4.6 the

normal driving (ND) state may transition back to the normal driving state or to any one

of six fault states: right front (RF), right middle (RM), right rear (RR), left front (LF), left

middle (LM) and left rear (LR) wheel stuck. For this experiment, the measurements are

changes in robot position and orientation.

In the first experiment in this domain only two of the six faults were represented (and

induced): one on a wheel on the right and the second on a wheel on the left. Figure 4.7

shows the improvement in performance with the DTPF. When there are numerous sim-

ilar fault states with high risk, the performance of the DTPF degrades. To demonstrate

this all six faults shown in Figure 4.6 were represented and tracked. The degradation in

performance is shown in Figure 4.8. This degradation occurs because the DTPF works by

trying to populate all of the fault states with particles; when there are too many fault states

[Draft: 19 October 2004—19:11 ] 44


ND

RF RR

RM

LF

LM

LR

Figure 4.6. HMM representing fault transitions for stuck wheels. Self transitions ineach state are not shown in the figure.

0 20 40 60 80 100 1200

100

200

300

400

500

600

Number of particles

KL

dive

rgen

ce

Num. particles vs. Error

Classical PFDTPF

Figure 4.7. Reduction in error in terms of KL-divergence from using the DTPF overCPF for a small number of faults

20 40 60 80 100 1200

500

1000

1500

2000

2500

Number of particles

KL

dive

rgen

ce


Classical PFDTPF

Figure 4.8. Degradation in DTPF performance with a larger number of similar faults

[Draft: 19 October 2004—19:11 ] 45


compared to the number of particles there is no way to populate all of them. A variable

resolution particle filter (VRPF) was designed to address this difficulty; chapter 5 describes

the VRPF.

[Draft: 19 October 2004—19:11 ] 46

CHAPTER 5

Variable Resolution Particle Filter

THE variable resolution particle filter (VRPF) (Verma et al., 2003) introduces the

notion of abstract particles, in which a particle may represent an individual

state or sets of similar states. With this method, a single abstract particle si-

multaneously tracks multiple similar states. A limited number of particles are

therefore sufficient for representing large portions of the state space when likelihood of

occupying this part of the state space is low. When the likelihood of the grouped states

increases and it is important to determine the relative importance of these states, particles

are specialized to represent individual states. This chapter demonstrates that tracking mul-

tiple states by grouping them together does not always increase the state estimation error;

at times, it reduces the error. This is because the variance of the state estimate can often

be dramatically reduced by abstraction, with only a minor increase in bias. A quantita-

tive decomposition of the error in terms of bias and variance is presented. A bias-variance

trade-off is made to dynamically refine and abstract states to change the resolution to min-

imize the state estimation error. As a result, reasonably low variance posterior estimates

can be obtained with a relatively small number of particles.

Consider the problem of diagnosing locomotion faults on a robot. Motors on any

of the wheels may stall at any time, but the probability of a stalled motor at any time is

low. Failures in wheels on the same side of the robot generate similar observations (drift

toward the side with the stuck wheel). Under these conditions, a classical particle filter

with small number of particles is likely to produce an estimate with high variance resulting

in identifying some arbitrary wheel fault on the same side rather than the correct fault.

With the VRPF the wheel faults on the same side of the rover would be aggregated together

into an abstract fault. Given a fault, the abstract state representing the side on which the

fault occurs would have high likelihood. The samples in this state would be assigned a high

importance weight. This would result in multiple copies of these samples on resampling

5.2 BELIEF STATE ESTIMATION AT A FIXED RESOLUTION

proportional to weight. Once there are sufficient particles to populate all the refined states

represented by the abstract state, the resolution of the state would be changed to the states

representing the individual wheel faults. At this stage, the correct hypothesis is likely to

be included in this particle based approximation at the level of the individual states and

hence the correct fault is likely to be detected.

The variable resolution particle filter requires: (1) A variable resolution state space

model that defines the relationship between states at different resolutions, (2) an algorithm

for state estimation given a fixed resolution of the state space, (3) a basis for evaluating

resolutions of the state space model, and (4) and algorithm for dynamically altering the

resolution of the state space.

In sections 5.1–5.4 the continuous state xt is omitted for clarity since only the resolu-

tion of the discrete states is varied.

5.1. Variable Resolution State Space Model

The VRPF requires a variable resolution state space model. We use a multi-layered

hierarchy: each physical (non-abstract) state corresponds to a leaf of the hierarchy. Sets

of states with similar state transition and observation models are aggregated together at

each non-leaf node in the hierarchy to form abstract states (see figures 5.1, 5.2, and 5.3).

In addition to the physical state set D, the variable resolution model consists of a set of

abstract states A ∈ {a(1) . . . a(M)} that represent sets of states (including other abstract

states):

a(j) ⊆ D ∪A

5.2. Belief State Estimation at a Fixed Resolution

This section describes the algorithm for estimating a distribution over the state space,

given a fixed resolution for each state, where different states may be at different fixed reso-

lutions. For each particle in a physical state, a sample is drawn from the predictive model

for that state p(dt | dt−1). It is then assigned a weight proportional to the likelihood of

the measurement given the prediction. For each particle in an abstract state, Aj , one of

the physical states, dt, that it represents in abstraction is selected proportional to the prob-

ability of the physical state under the stationary distribution, π(dt). The predictive and

measurement models for this physical state are then used to obtain a weighted posterior

sample. The particles are then resampled proportional to their weight. Based on the num-

ber of resulting particles in each physical state a Bayes estimate with a Dirichlet(1) prior is

[Draft: 19 October 2004—19:11 ] 48

5.2 BELIEF STATE ESTIMATION AT A FIXED RESOLUTION

Figure 5.1. General Markov model representing physical states

Figure 5.2. Abstract Markov model with similar states grouped. Circles that encloseother circles represent abstract states

obtained as follows:

p(dt | zt) =n(dt) + π(dt)

| Nt | +1,∑

dt

π(dt) = 1 (5.1)

where, n(dt) represents the number of samples in the physical state dt and | Nt | represents

the total number of particles in the particle filter. The distribution over an abstract state Aj

[Draft: 19 October 2004—19:11 ] 49

5.3 BIAS-VARIANCE TRADE-OFF

Figure 5.3. Markov model showing multiple levels of abstraction of the physicalmodel in figure 5.11

at time t is estimated as:

p(Aj | zt) =∑

dt∈Ajp(dt | zt) (5.2)

5.3. Bias-Variance Trade-off

The loss l from a particle based approximation p(dt | zt), of the true distribution p(dt |zt) is:

l = E[p(dt | zt)− p(dt | zt)]2 (5.3)

= {p(dt | zt)− E[p(dt | zt)]}2 +

{p(dt | zt)2 − E[p(dt | zt)]2} (5.4)

= b(p(dt | zt))2 + v(p(dt | zt)) (5.5)

where b(p(dt | zt)) is the bias and v(p(dt | zt)) is the variance.

The posterior belief state estimate from tracking states at the resolution of physical

states introduces no additional bias. But the variance of this estimate can be high especially

with small sample sizes. An approximation of the sample variance at the resolution of the

physical states may be computed as follows:

v(dt) = p(dt | zt)p(dt | zt) [1− p(dt | zt)]

n(dt) + π(dt)(5.6)

[Draft: 19 October 2004—19:11 ] 50

5.3 BIAS-VARIANCE TRADE-OFF

The loss of an abstract stateAj , is computed as the weighted sum of the loss of the physical

states dt ∈ Aj , as follows2:

l(Aj) =∑

dt∈Ajp(dt | zt) l(dt) (5.7)

The generalization to abstract states biases the distribution over the physical states to the

stationary distribution. In other words, the abstract state has no information about the

relative posterior likelihood, given the data, of the states that it represents in abstraction.

Instead it uses the stationary distribution to project its posterior into the physical layer. The

projection of the posterior distribution p(Aj | zt), of abstract state Aj , to the resolution of

the physical layer p(dt | zt), is computed as follows:

p(dt | zt) =π(dt)

π(Aj)p(Aj | zt) (5.8)

where, π(Aj) =∑x∈Aj π(x).

As a consequence of the algorithm for computing the posterior distribution over ab-

stract states described in section 5.2, a posterior over physical states dt with no bias from

abstraction is available at no extra computation, as shown in equation (5.1). This assumes

that the prior had no bias. The bias b(Aj), introduced by representing the set of physical

states dt ∈ Aj , in abstraction as Aj is approximated as follows:

b(Aj) =∑

dt∈Ajp(dt | zt) [p(dt | zt)− p(dt | zt)]2 (5.9)

It is the weighed sum of the squared difference between the posterior p(dt | zt), computed

at the resolution of the physical states and the biased posterior p(dt | zt), computed at the

resolution of abstract state Aj .

An approximation of the variance of abstract state Aj is computed as a weighted sum

of the projection to the physical states as follows:

v(Aj) =∑

dt∈Ajp(dt | zt)

[π(dt)

π(Aj)

]2p(Aj | zt)[1− p(Aj | zt)]

n(Aj) + π(Aj)(5.10)

The loss from tracking a set of states dt ∈ Aj at the resolution of the physical states is thus:

lp =∑

dt∈Ajv(dt) (5.11)

The loss from tracking the same set of states in abstraction as Aj is:

la = b(Aj) + v(Aj) (5.12)

There is a gain in terms of reduction in variance from generalizing and tracking in abstrac-

tion, but it results in an increase in bias. Here, a trade-off between bias and variance refers

2The relative importance/cost of the physical states may also be included in the weight

[Draft: 19 October 2004—19:11 ] 51

5.5 VARIABLE RESOLUTION PARTICLE FILTER ALGORITHM

to the process of accepting a certain increase in one term for a larger reduction in the other

and hence in the total error.

5.4. Dynamically Varying Resolution

The variable resolution particle filter uses a bias-variance trade-off to make a decision

to vary the resolution of the state space. A decision to abstract to the coarser resolution of

abstract state Aj , is made if the state space is currently at the resolution of states Ai, and

the combination of bias and variance in abstract state Aj is less than the combination of

bias and variance of all its children Ai, as shown below:

b(Aj) + v(Aj) ≤∑

Ai∈{children(Aj)}[b(Ai) + v(Ai)] (5.13)

On the other hand if the state space is currently at the resolution of abstract state Aj , and

the reverse of equation (5.13) is true, then a decision to refine to the finer resolution of states

Ai is made. The resolution of a state is left unaltered if its bias-variance combination is less

than its parent and its children. To avoid hysteresis, all abstraction decisions are considered

before any refinement decisions.

Each time a new measurement is obtained the distribution of particles over the state

space is updated. Since this alters the bias and variance trade-off, the states explicitly repre-

sented at the current resolution of the state space are each evaluated for gain from abstrac-

tion or refinement. Any change in the current resolution of the state space is recursively

evaluated for further change in the same direction.

5.5. Variable Resolution Particle Filter Algorithm

For clarity of presentation a somewhat simplified algorithm for abstracting and re-

fining states in VRPF is presented in table 5.1. In reality a number of optimizations are

possible.

Lines 2–4 is the initialization at time t = 0. The initial set of particlesB0 = {a[i]0 , x

[i]0 }i=1...N

are drawn from the prior distributions. First, the prior discrete state, a[i]0 , (which may be an

abstract or a physical state, depending on the resolution of the prior) is drawn from π(a0).

Then the continuous state, x[i]0 , is drawn from the conditional prior p(x0 | a0). Then R0 is

set to the set of unique states (physical or abstract) represented in B0.

Lines 6–45 are repeated in sequence for all t > 0.

Lines 6–7 projects all the particles to physical states to use the physical transition and

measurement models. If a particle, a[i]t−1 = d(j) ∈ Bt−1, is in an abstract state, one of its

descendant physical states {d(j)}, is selected proportional to the prior probability of the

physical states. π(d) represents the prior probability of state d.

[Draft: 19 October 2004—19:11 ] 52


Lines 8–16 update the projected particles exactly as in steps 5–13 of the CFP algorithm

from table 2.2.

Lines 17–23 re-assign particles to the appropriate abstractions of the physical states to

reflect the current resolution of the state space (represented by Rt−1).

Lines 6–23 estimate a probability distribution over the state at a fixed resolution, rep-

resented by Rt−1.

Lines 24–45 vary the resolution of the abstraction to trade bias against variance as

detailed in sections 5.3 and 5.4.

Lines 24–34 compute bias-variance statistics.

Lines 35–45 trade-off bias and variance to select the new resolution Rt of the state

space. If the loss for representing using the parent is lower than that for the current state,

the state is abstracted to the parent state. If the loss for using a child is lower than that for

the current state, the state is refined to the child states.


As in section 4.3.2 the problem domain for the experiment with VRPF involves diag-

nosing locomotion faults in a physics-based simulation of a six-wheel rocker-bogie robot.

Figure 4.4 shows a snapshot of the robot in the Darwin2K (Leger, 2000) simulator.

The Markov model representing the discrete state transitions consists of 7 physical

states. As shown in figure 4.6 the normal driving (ND) state may transition back to the

normal driving state or to any one of six fault states: right front (RF), right middle (RM),

right rear (RR), left front (LF), left middle (LM) and left rear (LR) wheel stuck. Each of

these faults cause a change in the robot dynamics, but the faults on each side (right and

left), have similar dynamics.

Given that the three wheels on each side of the robot have similar dynamics, a hi-

erarchy that clusters the fault states on each side together is used. Figure 5.4 shows this

hierarchical model, where the abstract states right side fault (RS), and left side fault (LS)

represent sets of states {RF, RM, RR} and {LF, LM, LR} respectively. The highest level of

abstraction therefore consists of nodes {ND, RS, LS}. Figure 5.5 shows how the state space

in figure 5.4 would be refined if the loss in abstract state RS given the number of particles

is greater than the combined loss of the physical states RF, RM and RR.

For this experiment, the measurements are noisy changes in robot position and orien-

tation. The particle set Bt consists of N particles, where each particle a[i]t is a hypothesis

about the current discrete state of the system. When particle filtering is performed with

the variable resolution particle filter, the particles are initialized at the highest level in the

[Draft: 19 October 2004—19:11 ] 53


1. t = 02. a

[i]0 ∼ π(a0) ; i = [1 . . . N ]

3. x[i]0 ∼ p(x0 | a0) ; i = [1 . . . N ]

4. R0 = {a[i]0 , x

[i]0 }i=1...N

5. t > 06. if a[i]

t−1 is an abstract (non-leaf),

7. p(d[i]t−1) ∼ π(d(j))∑

d(k)π(d(k))

; dj , dk ∈ a[i]t−1

8. for i = 1 to N do9. d

[i]t ∼ p(dt|d[i]

t−1)

10. x[i]t ∼ p(Xt|x[i]

t−1, d[it )

11. w[i]t = p(zt|x[i]

t )12. end13. predicted state = {d[i]

t , x[i]t }i=[1...N ]



t

16. end17. for i = 1 to N do18. if d[i]

t ∈ Rt−1,19. set a[i]

t = d[i]t

20. else21. find p = parent(d[i]

t ) ; p ∈ Rt−1

22. set a[i]t = p

23. end24. foreach a(k) ∈ Rt−1

25. if a(k) = dt is a physical (leaf) state,26. p(dt | zt) = n(dt)+π(dt)

N+1 ,∑dtπ(dt) = 1

27. bias(dt) = 0

28. v(dt) = p(dt | zt) p(dt|zt) [1−p(dt|zt)]n(dt)+π(dt)

29. else30. p(a(k) | zt) =

∑dt∈a(k)

p(dt | zt)31. bias(a(k)) =

∑dt∈a(k)

p(dt | zt)[p(dt | zt)− p(dt | zt)]2

32. v(a(k)) =∑dt∈a(k)

p(dt | zt)[π(dt)π(a(k))

]2 p(a(k)|zt)[1−p(a(k)|zt)]n(a(k))+π(a(k))

33. end34. end35. while Rt−1 not empty36. if b(p) + v(p) ≤∑ai∈{children(p)}[b(ai) + v(ai)] , where p = parent(a(k))

37. add p to Rt−1

38. remove a(k) and siblings(a(k)) from Rt−1

39. else if b(a(k)) + v(a(k)) ≥∑ai∈{children(a(k))}[b(ai) + v(ai)]

40. remove a(k) from Rt−1

41. add children(a(k)) to Rt−1

42. else43. add a(k) to Rt44. end45. end

Table 5.1. The algorithm for variable resolution particle filter.

[Draft: 19 October 2004—19:11 ] 54


ND

RF RR

RM

LF

LM

LR

Figure 5.4. HMM representing abstract state transitions for the stuck wheel Markovmodel from 4.6. Self transitions in each state are not shown in the figure.

ND

RF RR

LS

LM

RM

LF

LR

Figure 5.5. HMM representing a refinement of an abstract state in 5.4. Self transi-tions in each state are not shown in the figure.

abstraction hierarchy. When an RF fault occurs, this should result in a high likelihood of

particles in RS. These particles should multiply, which should result in the bias in RS ex-

ceeding the reduction in variance in RS over RF, RM and RR, thus favoring tracking at the

finer resolution. Additional observations should then assign a high likelihood to RF.

Figure 5.6 shows a comparison of the error from monitoring the state using a CPF that

tracks the full state space, an abstract filter that only tracks an abstract state space (never

refining states), and the VRPF that varies the resolution of the state space (and switches

between abstract and physical states). The X axis shows the number of particles used,

the Y axis shows the error in terms of KL-divergence from an approximation of the true

[Draft: 19 October 2004—19:11 ] 55


0 200 400 600 800 1000 12000

50

100

150

200

250

Number of particles

KL

dive

rgen

ce


Classical PFVRPFAbstract

Figure 5.6. Comparison of number of particles vs. error in terms of KL-divergencebetween a CPF, an abstract filter (which never refines to a finer resolution), and theVRPF. TheX−axis shows the number of particles and the Y− axis shows the error.

Figure 5.7. Comparison of wall clock time vs. error in terms of KL-divergence be-tween a CPF and VRPF. TheX−axis shows wall clock time and the Y− axis showsthe error.

[Draft: 19 October 2004—19:11 ] 56


posterior computed using a large number of particles. 1, 000, 000 particles were used to

compute an approximation to the true distribution. The KL divergence is computed over

the entire length of the data sequence and is averaged over multiple runs (varying from 50

to 5 as particle size was increased) over the same data set. The data set includes normal

operation and each of the six faults.

Figure 5.6 demonstrates that for small particle sizes the abstract filter performs better

than the CPF. This is because, with small particle sets, the variance of the CPF is much

higher than the bias introduced by abstraction. But, with large particle sets, the perfor-

mance of the CPF is superior to the abstract filter since there are sufficient particles to track

this simple model and the variance is minimal, but the abstract filter continues to have the

same bias and performance. The performance of the VRPF is superior for both small and

large particle sets. This is because with small particle sets, it maintains particles in abstract

states, but as soon as the bias-variance trade-off indicates a reduction in variance that justi-

fies refinement to a finer resolution the VRPF refines the state space. In addition figure 5.7

shows the KL-divergence along the Y axis and wall clock time along the X axis. Here too,

the performance of the VRPF is preferable to the CPF or the abstract filter.

[Draft: 19 October 2004—19:11 ] 57

CHAPTER 6

Look-ahead Particle Filters

THIS chapter describes various enhancements for improving fault monitoring

with particle filters. Enhancements include looking at the measurement one

step ahead in time to focus the generation of fault hypotheses, using future

data (i.e. delaying estimation) to improve past estimates (also known as smooth-

ing), and using future data to alter the risk-function.

One limitation of classical particle filters is that they are purely predictive because

it doesn’t take the next measurement into account. There are a large number of possible

faults at a given instance, but only a small subset of these faults have a high probability

given the next measurement. This chapter presents methods that use future observations

to generate particles thus focusing the filter on faults with higher likelihood.

Some of these enhancements introduce a small delay in fault detection and identifica-

tion because they require future measurements, but the improvement in performance more

than offsets this in some situations.

6.1. Variable Resolution Unscented Particle Filter

The Variable Resolution Unscented Particle Filter (VUPF) filter uses the next mea-

surement to weight hypotheses. Using the next measurement can dramatically improve

performance. This section describes a particle filter that generates particles at time t using

a proposal distribution that takes into account the measurement at time t, in addition to

the state at time t − 1. This proposal distribution, p(xt, dt | x[i]t−1, d

[i]t−1, zt) is the “optimal”

proposal distribution, that is, the distribution that minimizes the variance of the impor-

tance weights conditioned on x[i]t−1, d

[i]t−1 and zt. Unfortunately, it is difficult to sample from

this proposal distribution exactly, so the VUPF approximates it using Unscented Kalman

Filters (UKFs) (Julier and Uhlmann, 1996)(see section 2.2.5). This approximation is similar

to an unscented particle filter (van der Merwe et al., 2000), but it takes into account the fact

6.1 VARIABLE RESOLUTION UNSCENTED PARTICLE FILTER

that some of our state variables are discrete while others are continuous. To compute these

approximations each possible pair of i and dt must be examined separately; this process

can be computationally expensive, a complaint that is addressed by utilizing the variable

resolution particle filter algorithm from chapter 5.

The VUPF combines the UKF (see section 2.2.5) and VRPF (see chapter 5). The VRPF

tracks abstract states that may represent single states or sets of states. There are many

fewer transitions between states when they are represented in abstraction. The VRPF in

conjunction with a UKF proposal improves performance and may potentially be used in

large state spaces. Experimental results show a significant improvement in efficiency.

6.1.1. Optimal Proposal Distribution

Doucet et al. (Doucet, 1998) show that p(xt, dt | x[i]t−1, d

[i]t−1, zt) is the “optimal” proposal

distribution, that is, the distribution that minimizes the variance of the importance weights

conditioned on x[i]t−1, d

[i]t−1 and zt. Since it is difficult to sample from exactly this proposal

distribution UKFs (Julier and Uhlmann, 1996) are used instead to approximate it.

Bayes’ rule and our conditional independence assumptions imply

p(xt, dt | x[i]t−1, d

[i]t−1, zt) = η[i] p(zt | xt, dt) p(xt, dt | x[i]

t−1, d[i]t−1)

= η[i] p(zt | xt, dt) p(xt | x[i]t−1, dt) p(dt | d

[i]t−1)

= η[i] η[i]dtp(xt | x[i]

t−1, dt, zt) p(dt | d[i]t−1)

The normalizing constants in the above equations are

η[i] = 1/p(z | x[i]t−1, d

[i]t−1)

η[i]dt

= p(zt | x[i]t−1, dt)

η[i] may be ignored since it doesn’t depend on xt or dt. The discrete transition probability

p(dt | d[i]t−1) is known and UKFs are used to approximate η[i]

dtand p(xt | x[i]

t−1, dt, zt) for each

particle i and possible discrete transition dt. To compute these approximations we need to

examine each possible pair of i and dt separately.

6.1.2. Approximation of Optimal Posterior using Unscented Kalman Filters

In our case, we use an Unscented Transform (UT) to approximate p(xt | x[i]t−1, dt, zt) as

a Gaussian. This approximation will usually be excellent because we are conditioning on a

single previous state and a single possible fault. Given this approximation, the values for

µ[i]dt

(the mean of xt), P[i]dt

(the covariance of xt), and η[i]dt

(the observation likelihood) can be

computed analytically .

[Draft: 19 October 2004—19:11 ] 59


Only the last of these quantities is not given by the standard Kalman filter equations.

It is:

η[i]dt∝ 1

|S[i]dt|12

exp(− 12 (zt − z[i]

dt)T (S

[i]dt

)−1(zt − z[i]dt

))

where S[i]dt

is the innovation covariance and z[i]dt

is the predicted observation (both computed

from the UT).

Once the UTs are computed, samples are drawn from the proposal distribution by first

drawing dt and then xt according to equation 6.1–6.2, then computing importance weights

via equation (1.7).

p(dt | d[i]t−1) ∼ p(dt | d[i]

t−1) η[i]dt

(6.1)

p(xt | x[i]t−1, dt) ∼ N (µ

[i]dt, P

[i]dt

) (6.2)

In order to find η[i]dt

for every dt and i, a UKF is computed for each particle and every

possible next discrete state transition. Given that there are potentially a large number of

faults to transition to at any step, this may be inefficient.

While in the traditional case, each transition would require an additional UKF, using

the VRPF reduces computation by reducing the number of state transitions. This is be-

cause the VRPF groups multiple particles with similar transitions.The one-step look-ahead

algorithm with VRPF is called VUPF.

6.1.3. Experimental Results

The faults and the abstraction hierarchy considered in this experiment are the same as

in the experiment in section 5.6. The measurements in this experiment were noisy estimates

of the robot position and orientation. The particle set representing the state consists of N

particles, where each particle [a[i]t , x

[i]t ] is a hypothesis about the current state of the system.

a[i]t is the discrete fault or normal state and x

[i]t is the multi-dimensional continuous state

representing the change in position and orientation of the robot.

Figures 6.1 and 6.2 show a comparison of the error from monitoring the state using

a CPF, a UPF, and a VUPF. The X axis shows the number of particles used, the Y axis

shows the error in terms of KL-divergence from an approximation of the true posterior

computed using a large number of particles. For the experiment in figures 6.1 and 6.2, the

continuous measurements were noisy absolute robot positions. 1, 000, 000 particles were

used to compute an approximation to the true distribution. The KL divergence is computed

over the entire length of the data sequence and is averaged over multiple runs over the

same data set. The data set included normal operation and each of the six faults. Figure 6.1

demonstrates that using the VUPF significantly reduces the error for small particle sets.

[Draft: 19 October 2004—19:11 ] 60


0 100 200 300 400 5000

10

20

30

40

50

60

Number of particles

KL

dive

rgen

ce


Classical PFUKF with ClassicalUKF with VRPF

Figure 6.1. Comparison of number of particles vs. error in terms of KL-divergencebetween CPF, UPF(labeled as UKF with classical), and VUPF(labeled as UKF withVRPF). The X−axis shows the number of particles and the Y− axis shows theerror.

0 20 40 60 80 100 1200

5

10

15

20

25

30

Time

KL

dive

rgen

ce

Time vs. Error

Classical PFUKF with ClassicalUKF with VRPF

Figure 6.2. Comparison of wall clock time vs. error in terms of KL-divergence be-tween CPF, UPF(labeled as UKF with classical), and VUPF(labeled as UKF withVRPF). The X−axis shows the number of particles and the Y− axis shows theerror.

[Draft: 19 October 2004—19:11 ] 61

6.2 DECISION-THEORETIC MONTE CARLO SMOOTHING

Figure 6.2 shows the error along the Y axis and wall clock time along the X axis. The

performance of the VUPF is superior in this graph too.

The improvement in performance of the VUPF over UPF is expected to be even greater

when the variable resolution state space model is larger and results in a larger reduction

in the size of the state space than the simple experiment presented here. This is because a

UKF needs to be computed for every possible discrete state transition, and the number of

discrete state transitions between the abstract state in the VRPF are fewer than the discrete

state transitions between the physical states in the underlying physical model.

6.2. Decision-Theoretic Monte Carlo Smoothing

The estimation of the state of the system using future measurements is termed smooth-

ing (see section 2.4.1). The idea behind decision-theoretic Monte Carlo smoothing (DTMS)

is to maintain a time varying risk-function. Future measurements are used to smooth the

risk-function to discourage sampling of states that are unlikely given future measurements.

This approach is a compromise between the purely predictive decision-theoretic theoretic

approach in chapter 4, which is efficient but inaccurate (for a large number of faults), and

the auxiliary particle filter (see section 2.4.5), which is more accurate, but computationally

expensive.

The aim of Monte Carlo smoothing is to ensure long independent trajectories, else

the particle filter can often degenerate to tracking a single hypothesis. It does so by re-

weighting particles to assign higher weight to particles that are more likely to survive,

given future data. It is important to note that the backward pass in classical smoothing

does not generate any new hypotheses. In addition, forward sampling from smoothed es-

timates is not guided by the smoothing process. Smoothing only improves the marginal

distribution at time t − L, where L is the length of the fixed-lag over which smoothing is

performed. In some applications such a hybrid-state estimation for fault diagnosis, where

there are a large number of possible faults that may occur at any instance, states that re-

sulted in long trajectories in the forward pass, may not do so subsequent forward passes

using smoothed estimates.

As discussed in section 6.1, the problem with using Auxiliary particle filter variants

(such as UPF) for hybrid state estimation is that it is computationally very expensive. A

estimate of the posterior likelihood (such as with a UKF), needs to be computed for every

possible next discrete state transition. Measurement noise makes it difficult to find a rea-

sonable estimate of a threshold that may be used for selecting posterior states with high

likelihood without an exhaustive search.

[Draft: 19 October 2004—19:11 ] 62


In DTMS, each particle in addition to the discrete-continuous state, {(d[1]t , x

[1]t ) . . . (d

[N ]t , x

[N ]t )},

and importance weight, {w[i]t }, also maintains an estimate of the risk-function, r[i]

t , at time

t. As discussed in section 4.2, the initial risk function is computed for each discrete state

dt, t = 0, using a MDP, which assumes that complete information about the current state is

available This is an efficient approximation. The process is actually a Partially Observable

Markov Decision Process (POMDP) (Sondik, 1971; Kaelbling et al., 1998), but computing a

value function for large state spaces in a POMDP is computationally expensive. A POMDP

is a model from operations research (OR) literature for decision making using information

states (distributions over states) when complete information about the current state is not

available. Smoothing the risk-function improves the approximation of the risk-function at

time t, given a more accurate estimate of the probability distribution over states at time

t+ 1.

The aim of smoothing the risk function at time t is not merely to reinforce the discrete

states that are already represented in the particle set at time t + 1, but to encourage explo-

ration of alternate high-risk states. Hence, instead of increasing the risk of transitions that

have a high posterior likelihood we decrease the risk of transitions that have low posterior

likelihood. A fixed lag history of particles is maintained. If a trajectory terminates at time

t + 1, i.e. the particle is eliminated during re-sampling, the risk function at time t for that

particle is updated to reduce the risk of state t+ 1. Subsequent forward sampling therefore

encourages sampling alternate high-risk states, which now have a relatively higher risk.

This process is iterated until the trajectory survives resampling at time t+ 1 or all possible

transitions have been explored. The advantage of this approach is that extra computation

is required only when the particle estimate starts to degenerate and not otherwise.


Preliminary experimental results show an improvement in performance for small par-

ticle sets when using DTMS over decision-theoretic sampling without smoothing. For our

experiments we used data from simulating stuck wheel faults on a six-wheel rocker-bogie

rover in the Darwin2k (Leger, 2000) simulator. There are 7 discrete states in this simple

example – a normal operation state and a state representing each of the stuck wheel faults.

The continuous state tracks the [x, y, θ] position of the rover in global co-ordinates and

measurements are noisy estimates of the position. Each experiment described below was

run with the true state being each of the 7 discrete states. Except when specified otherwise,

each experiment was repeated 1000 times to compute the standard error bars shown. The

[Draft: 19 October 2004—19:11 ] 63


5 10 15 20 25 30 35 40 45 50 550

500

1000

1500

2000

2500

3000

Number of particles

KL

dive

rgen

ce


RBPFDecision−theoretic RBPFDecision−theoretic smoothing

Figure 6.3. Number of particles vs. error in terms of KL-divergence for Rao-Blackwelized particle filter, Decision-theoretic RBPF, and Decision-theoretic RBPFwith smoothed loss function

experiments measure error in terms of Kulback-Leibler divergence from the true distribu-

tion (computed using 106 particles with a classical filter. Since the size of the discrete state

space in this preliminary experiment is small, small particle sets were used to highlight the

relative differences between the approaches.

In the first experiment, shown in figure 6.3 as a dashed black line, a Rao-Blackwellized

particle filter was used. The discrete states were sampled and the conditional continuous

states were obtained using a UKF since the process is nonlinear (GPF). As expected the filter

tracked the normal state well, but tracked fault states poorly since the probability of faults

is low and no particles jumped to fault states. This experiment was only repeated 50 times

since additional runs would not have made a visible difference to the graph. The reason

the KL-divergence increases slightly with more particles is because we use a Dirichlet (1)

prior and the effect of the prior decreases with larger sample sizes.

In the second experiment, the Rao-Blackwellized filter above was enhanced to use

decision-theoretic sampling. The results were good when the experiment was limited to

one or two highly distinct fault states. When the full experiment was performed, perfor-

mance degraded as shown in figure 6.3 as the dash-dot blue line. This is because all faults

[Draft: 19 October 2004—19:11 ] 64


have high risk and an incorrect fault (that explained the measurement relatively better than

the normal state) was often confused when the correct fault state was not sampled.

In the third experiment, shown in figure 6.3 as a solid red line, decision-theoretic

smoothing of the risk function was performed with the GPF.

[Draft: 19 October 2004—19:11 ] 65

CHAPTER 7

Robot Fault Diagnosis Experiment

TTHIS chapter presents experiments with diagnosing a variety of faults in a sim-

ulation of a rocker-bogie rover. These experiments aim to demonstrate that

the approach presented in this thesis:

• Improves performance/accuracy ratio relative to previous approaches for compa-

rable computation.

• Can track a larger number of faults in real-time than previous approaches

• Can handle nonlinear faults which degrade performance in other approaches

• Can handle faults that may only be observed through tracking dynamics over time,

which cannot be handled with other approaches

• Can handle noisy measurements, which some other approaches cannot.

7.1. Experimental setup

7.1.1. Simulator

The experiments in this thesis were performed using the Darwin2K simulator. Dar-

win2K is a free, open-source toolkit for robot simulation and automated design (Leger,

Leger).

Darwin2K’s simulation capabilities are tailored to support engineering design and

controller prototyping for robotic application. For example, it includes detailed motor and

gear head models and provides full dynamic simulation capabilities.

For the experiments in this thesis many new features were added to the base simulator.

These features will be released in the future as an open source plugin for Darwin2K. The

enhancements include new controllers (e.g. Ackermann steering), a new rover model (six-

wheel rocker-bogie with steering control), an interface for logging a large number of rover

parameters, the ability to introduce a number of different locomotion faults (eg. stuck

wheels, slipping wheels, lubricant leak, etc.), and some terrain and soil models.

7.1 EXPERIMENTAL SETUP

Figure 7.1. Side view of rocker-bogie rover in Darwin2K

Figure 7.2. Top view of rocker-bogie rover in Darwin2K

Figure 7.3. View of rocker-bogie rover in Darwin2K

A six-wheel rocker bogie rover with actuated steering was used in the experiments

discussed in section 7.2. Figures 7.1 to 7.4 show images of the rover used.

[Draft: 19 October 2004—19:11 ] 67

7.1 EXPERIMENTAL SETUP

Figure 7.4. View of rocker-bogie rover in Darwin2K

7.1.2. Faults modeled

The following locomotion faults are included in the results presented in section 7.2:

• Slipping wheel: has no traction but continues to spin. This fault was a common

occurrence on almost all the field robots that were studied during the course of

this thesis. Figure 7.5 shows one such instance from a field test with the Hyperion

rover in the Atacama desert. If left undetected it may result in the robot getting

seriously stuck, as shown in figure 7.6.

Wheel velocity computed from encoder measurements does not provide evidence

of this fault, therefore PID output at each wheel was measured (this is because the

simulation does not provide measurements of current and voltage). The output at

the slipping wheel is less than expected for nominal traction and the output at the

other wheels is slightly higher than expected for nominal traction. The effect of

multiple wheels slipping is not independent, since all the wheels are attached to a

single rigid rover body.

• Faulty encoder: reports a value of 0 encoder ticks and hence 0 wheel velocity even

when the wheel is functioning normally. 0 wheel velocity indicates a stuck wheel,

which alters the rover trajectory and hence heading. To detect this fault measure-

ments of the rover heading are used. Nominal heading measurements indicate

that the encoder and not the wheel is at fault. The effect of multiple wheels with a

faulty encoder is independent since a faulty encoder only alters measurements at

a single wheel and does not affect the overall rover dynamics.

• Stuck wheel: causes the wheel to not rotate (or move intermittently) and alters the

rover trajectory.

[Draft: 19 October 2004—19:11 ] 68


Figure 7.5. Slipping wheel on Hyperion rover during a field test in the Atacama desert

Multiple observations of this fault have also been made during the course of this

thesis. For example the K9 rover on locked a wheel on multiple occasions as a re-

sult of a rock that lodged itself in the metal wheel. If left undetected this fault can

potentially cause the “rampant-K9” configuration shown in figure 7.7. A stuck

front wheel causes the middle wheel to drive faster than the front wheel and

pushes the bogie up into this air. This is a stable fault configuration where the

rover only has three wheels in contact with the ground.

To differentiate this fault from faulty encoders the rover heading is tracked. The

effect of multiple stuck wheels is not independent.

• Lubricant leak: nonlinearly increases the internal friction of the motor (H. Ols-

son and Lischinsky, 1999; Elsharkawy and Guedouar, Elsharkawy and Guedouar).

This fault occurred on the Spirit rover on Mars (daily report, 2004). A quote from

the Mars Exploration Rover report dated July 16,2004: That sixth wheel drags rather

than rolls, causing the whole vehicle to veer to the right.. In the experiments presented

here a constant volume of lubricant was modeled as leaking with time. Rover

heading and PID output are tracked to detect this fault.

7.2. Experimental results

The modes in the experiments in this section include normal driving, encoder faults at

each of the six wheels, and all combinations of one, two, three, and four wheel slip, stuck

and lubricant leak faults. This set of discrete states may be referred to as all faults, all modes

or all discrete states for the rest of this section. Currently there are 169 discrete faults and 21

continuous variables in the model.

[Draft: 19 October 2004—19:11 ] 69


Figure 7.6. Stuck ATV1

Figure 7.7. The K9 rover stuck during a field test 2

7.2.1. Classical Particle Filter

Figure 7.8 shows the distribution over all discrete states for 10 timesteps after the

occurrence of a fault. Along theX axis is the true state and along the Y axis is the estimated

state. 10 iterations of experiments with each fault were performed with 1000 particles in

each experiment.

Perfect performance would result in a diagonal running from the top left corner to

the bottom right corner of the plot. The performance of the CPF is quite poor. There is no

probability mass in most of the multiple fault states. Only some of the most likely faults

[Draft: 19 October 2004—19:11 ] 70


True state

Est

imat

ed s

tate

CPF

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

Figure 7.8. Distribution over all states for 10 timesteps after the occurrence of afault with a CPF. Along the X-axis the true fault state and along the Y-axis is theestimated state

have some posterior probability, but even then similar faults are easily confused by the

CPF.

Figure 7.9 shows the delay in fault detection, where a fault is considered detected

when the posterior probability of the fault is greater than or equal to 0.6. This is a simplistic

method of fault detection. This threshold may be varied to match mission requirements.

In addition different controllers may choose more sophisticated statistics on the posterior

distribution to detect a fault. But the figure does provide a reasonable indication of the

performance of the filter.

All the discrete states are along the X axis and the delay in fault detection (measured

in timesteps since the occurrence of the fault) is along the Y axis. In the plot a delay value

of 20 implies non-detection. The figure shows a box and whisker plot. The box has lines

at the lower quartile, median, and upper quartile values. The whiskers are lines extending

from each end of the box to show the extent of the rest of the data. Outliers are data with

values beyond the ends of the whiskers that are represented with the + sign.

Figure 7.9 shows that most of the faults go undetected. Only the normal state (state 7,

shown at the bottom left corner of the plot) is shown to be detected instantaneously. One

other state is sometimes detected. Detection of the normal state is not very useful, but is

[Draft: 19 October 2004—19:11 ] 71


2

4

6

8

10

12

14

16

18

20

CPF

Del

ay in

faul

t det

ectio

n

Fault number

Figure 7.9. Box and whisker plots showing the delay in fault detection with a CPF.In these plots a delay value of 20 implies non-detection. The box has lines at thelower quartile, median, and upper quartile values. The whiskers are lines extend-ing from each end of the box to show the extent of the rest of the data. Outliers aredata with values beyond the ends of the whiskers that are represented with the +

sign.

shown in the plot for completeness. There are some fault detections, but most of them are

+ signs indicating that they are mostly noise.

Figure 7.10 shows a section of figure 7.9 at a higher resolution. The section that is

shown at a higher resolution is depicted in the left figure in a dashed box. The only state

that has no delay in detection is a nominal state (state 7).

Figure 7.11 shows a higher resolution of the dashed section depicted in figure 7.9.

Fault 127 is detected a few times with a delay of 7, but the median delay in detection for

127 is 20, which implies a non-detection.

7.2.2. Decision-Theoretic Particle Filter

Figure 7.12 is similar to the matrix shown in figure 7.8, but with a Decision-Theoretic

particle filter (DTPF). The performance of the DTPF is a substantial improvement over the

performance of the CPF, but there is a lot of probability mass scattered around the diagonal.

The boxes around the diagonal are a result of fault ordering: because of the clustering

described below, similar states are grouped together.

[Draft: 19 October 2004—19:11 ] 72


2

4

6

8

10

12

14

16

18

20

CPF

Del

ay in

faul

t det

ectio

n

Fault number

(a) Section

7 110 109 108 107 116 115 106 105 120 119 104 103 118 117 102

2

4

6

8

10

12

14

16

18

20

CPF

Del

ay in

faul

t det

ectio

nFault number

(b) Zoom

Figure 7.10. A section of figure 7.9, shown in a dashed box in (a), is shown at ahigher resolution in (b)

2

4

6

8

10

12

14

16

18

20

CPF

Del

ay in

faul

t det

ectio

n

Fault number

(a) Section

148 147 125 124 141 140 139 138 127 126 123 122 71 48 68 47 70 62

2

4

6

8

10

12

14

16

18

20

CPF

Del

ay in

faul

t det

ectio

n

Fault number

(b) Zoom


Figure 7.13 shows the delay in fault detection. This figure is similar to figure 7.9, but

with the DTPF. The performance is better than the CPF in that faults are at least detected.

But the variance in the delay is fault detection is very high and often faults are not detected

at all.

Figure 7.14 shows a higher resolution of the dashed section depicted in figure 7.13.

[Draft: 19 October 2004—19:11 ] 73


True state

Est

imat

ed s

tate

RSPF

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

Figure 7.12. Distribution over all states for 10 timesteps after the occurrence of afault with a DTPF. Along the X-axis the true fault state and along the Y-axis is theestimated state

7.2.3. Decision-Theoretic Variable Resolution Particle Filter

7.2.3.1. Multi-resolution Hierarchy. For the Variable Resolution Particle Filter (VRPF)

(described in chapter 5), a hierarchy is required. For the VRPF used in this chapter the hi-

erarchy was computed automatically. A simple algorithm was used, the Jeffry-divergence

(also called the symmetric KL divergence) between the posterior distributions using pre-

dictive models of each physical state (conditioned on the same prior) was computed. States

with minimal Jeffreys-divergence were greedily clustered into an abstract state to create a

binary tree. Figure 7.15 shows a dendrogram representing the hierarchy for the normal

state and all one, two, three, and four wheel slipping, stuck, and lubricant leak faults. A

listing of physical fault names and their corresponding fault numbers is provided in Ap-

pendix B. The top most fault shown in figure 7.15 is the normal state. Figure 7.16 shows

the top third segment of figure 7.15 at a higher resolution. Figures 7.17 and 7.18 show the

subsequent two sections in order at higher resolutions.

Random hierarchies (where states are clustered ignoring similarity to create a bal-

anced tree) were created to determine if it is necessary to cluster states or not. The per-

formance of the algorithm with random hierarchies is poorer than that with a well chosen

hierarchy. This is because it isn’t always the true state that is sampled when projecting

[Draft: 19 October 2004—19:11 ] 74


2

4

6

8

10

12

14

16

18

20

DTPF

Del

ay in

faul

t det

ectio

n

Fault number

Figure 7.13. Box and whisker plots showing the delay in fault detection with aDTPF. In these plots a delay value of 20 implies non-detection. The box has lines atthe lower quartile, median, and upper quartile values. The whiskers are lines ex-tending from each end of the box to show the extent of the rest of the data. Outliersare data with values beyond the ends of the whiskers that are represented with the+ sign.

2

4

6

8

10

12

14

16

18

20

DTPF

Del

ay in

faul

t det

ectio

n

Fault number

(a) Section

172 171 152 151 150 149 133 132 175 143 129 128 162 161 158 157 156 155 137

2

4

6

8

10

12

14

16

18

20

DTPF

Del

ay in

faul

t det

ectio

n

Fault number

(b) Zoom


[Draft: 19 October 2004—19:11 ] 75


1 2 3 4 5 6 7 8 9 10

4353394142574951

631266322373456

132

53623665060

2253052

324

433276129464555586428673859355444654069627047684871

122123126127138139140141124125147148130131134135136137155156157158161162128129143175132133149150151152171172163164169170142144145146153154165166159160167168173174176177

72178

8788818283848586

121179

959699

1007374

180181

7576

182183

77787980899091929394

111112

9798

113114101102117118103104119120105106115116107108109110

7

Figure 7.15. Dendrogram representing the hierarchy for the normal state and allcombinations of one, two, three, and four wheel slipping, stuck, and lubricant leakfaults. Fault names are provided in Appendix B

[Draft: 19 October 2004—19:11 ] 76


1 2 3 4 5 6 7

72178

8788818283848586

121179

959699

1007374

180181

7576

182183

77787980899091929394

111112

9798

113114101102117118103104119120105106115116107108109110

7

Figure 7.16. The top third of the hierarchy from figure 7.15 at a higher resolution

[Draft: 19 October 2004—19:11 ] 77


1 2 3 4 5 6 7

122123126127138139140141124125147148130131134135136137155156157158161162128129143175132133149150151152171172163164169170142144145146153154165166159160167168173174176177

Figure 7.17. The middle segment of the hierarchy from figure 7.15 at a higher resolution

[Draft: 19 October 2004—19:11 ] 78


1 2 3 4 5 6 7

4353394142574951

631266322373456

132

53623665060

2253052

324

433276129464555586428673859355444654069627047684871

Figure 7.18. The bottom third of the hierarchy from figure 7.15 at a higher resolution

[Draft: 19 October 2004—19:11 ] 79


True state

Est

imat

ed s

tate

VRPF

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Figure 7.19. Distribution over all states for 10 timesteps after the occurrence of afault with a DT-VRPF. Along the X-axis the true fault state and along the Y-axis isthe estimated state

to the physical layer for propagating particles. If the particles are projected to “similar”

states the continuous posterior is closer to the true posterior than when they are projected

to very different states. If the active resolution of the state space is not the physical layer,

on abstraction particles are mixed between states.

7.2.3.2. Results. Figure 7.19 is similar to the matrix shown in figure 7.8, but with a

VRPF that uses a decision-theoretic proposal distribution (DT-VRPF). The DT-VRPF com-

bines the decision theoretic and variable resolution particle filters. As shown by the almost

clean diagonal, the distribution over 10 time steps after the occurrence of a fault is almost

completely accurate with 1000 particles. This is a big improvement over just the DTPF

shown in figure 7.12.

Figure 7.20 shows the delay in fault detection. This figure is similar to figure 7.9, but

with the DT-VRPF. As shown in the figure a large proportion of the faults are detected in

the first two timesteps after the occurrence of a fault.

Figure 7.21 shows a comparison of the error from monitoring the state using a Decision-

theoretic particle filter (DTPF), shown as a blue dotted line, that tracks the full state space,

and the DT-VRPF, shown as a red solid line, that varies the resolution of the state space

(and switches between abstract and physical states).

[Draft: 19 October 2004—19:11 ] 80


2

4

6

8

10

12

14

16

18

20

VRPF

Del

ay in

faul

t det

ectio

n

Fault number

Figure 7.20. Box and whisker plots showing the delay in fault detection with a DT-VRPF. In these plots a delay value of 20 implies non-detection. The box has linesat the lower quartile, median, and upper quartile values. The whiskers are linesextending from each end of the box to show the extent of the rest of the data.Outliers are data with values beyond the ends of the whiskers that are representedwith the + sign.

The X axis shows the number of particles used, the Y axis shows the error in terms of

KL-divergence from an approximation of the true posterior computed using a large num-

ber of particles. The KL divergence is computed over a fixed length of the data sequence

and is averaged over multiple runs (varying from 10 to 3 depending on particle size). For

fault modes (discrete states) the KL-divergence is computed over 10 timesteps after the

occurrence of the fault.

As shown in figure 7.21, for small particle sets, the mean error of the DT-VRPF over

all the experiments performed is lower than that of the DTPF. As the particle size increases

the difference in error decreases. The performance of the DT-VRPF with a 100 particles is

far better than the performance of the DTPF with 2000 particles.

[Draft: 19 October 2004—19:11 ] 81


0 2000 4000 6000 8000 10000 120000

200

400

600

800

1000

1200

Number of particles

KL

dive

rgen

ce

All faults

Figure 7.21. Comparison of DTPF and DT-VRPF for normal driving, and everycombination of one, two, three, and four wheels slipping, stuck, or with a lubri-cant leak.

[Draft: 19 October 2004—19:11 ] 82

CHAPTER 8

Conclusion

8.1. Contributions

This dissertation presents a novel approach that outperforms existing algorithms (in

terms of computational efficiency and scalability) for reliably estimating the state of general

(nonlinear, non-Gaussian) dynamic systems in real-time in the presence of uncertainty (in-

cluding rare events such as faults). This thesis presented three novel algorithms to address

these issues:

1. The Decision Theoretic Particle Filter (DTPF) reduces the number of particles re-

quired to track unlikely states by incorporation potential loss in the proposal dis-

tribution for updating particles.

2. The Variable Resolution Particle Filter (VRPF) reduces the number of particles re-

quired by grouping multiple low-probability states based on similarity, differenti-

ating them only when it is useful.

3. The Variable Resolution Unscented Particle Filter (VUPF) looks at the expected sensor

measurement one step ahead in time to improve the state estimate provided by a

limited number of particles.

This thesis includes the first application of particle filters to robot fault diagnosis,

which has since become an active area of research. It also provides a comprehensive study

of issues specific to robot fault diagnosis and presents approaches for addressing them.

This thesis increases the scope of possible faults that may be tracked. The observation

from early work in implementing Health Monitoring for various robots was that faults with

dramatic impact on performance (that are typically irrecoverable) are easily tracked using

those methods but that more common, more subtle faults like wheel slip go undetected.

In addition, extremely rare faults were not considered to be worth monitoring. This thesis

presents a set of methods that can accurately identify a large number of faults, including

subtle faults, efficiently.

8.2 DISCUSSION

8.2. Discussion

This section presents a discussion based on various lessons learned and observations

made during the course of this dissertation.

8.2.1. Model-based Diagnosis

Often faults are defined as deviations from the nominal behavior of the system. This

definition naturally lends itself to model-based diagnosis, where a model of nominal sys-

tem behavior is maintained and deviations from this behavior imply an anomaly.

This thesis uses a model-based approach. In addition to modeling nominal behavior,

the behavior of the system with different types of faults is also modeled. This enables

the system to not only detect that there is a fault, but also to identify the particular fault.

In some ways human experts also maintain models of system behavior and a model of

behavior in the presence of an anomaly (based on prior experience or information) makes

it easier to detect the anomaly.

The down side of model-based methods is that building models (particularly of com-

plex nonlinear systems) is very difficult. Building system models is where a large pro-

portion of the time in any model-based diagnosis system is spent. (Willeke and Dearden,

Willeke and Dearden) discusses some specific problems encountered during building hy-

brid models for a rover and presents an approach to address them.

Since model based methods can only identify modeled faults a key issue is to deter-

mine what faults to track. (Carlson and Murphy, 2003) presents a study of a set of indoor

mobile robots that logs failures over a period of time. Data from real robots is hard to ob-

tain since faults are very infrequent and sometimes data logging is not turned on when

the fault does occur. Using simulations for this purpose is a much more practical, but less

realistic method (Baydar, 2002).

8.2.2. Physics Based Approach

A physics based approach, such as the one utilized in this thesis, provides an intu-

itive way to map dynamics (continuous behavior) to fault modes. The models are based

on variables that have a physical interpretation as do the faults that occur in the system.

The main drawback is that physics based models are complicated and hard to build. It is

not possible to represent the types of faults addressed in this thesis with a simple set of

qualitative constraints and hence the physics based approach provides a crucial capability.

[Draft: 19 October 2004—19:11 ] 84

8.2 DISCUSSION

SENSORSENVIRONMENTFDI CONTROLLER(RECOVERY)

CONTROLLER

ROBOT

measurements

control commands

Figure 8.1. Schematic of how the fault detection and identification (FDI) componentfits in with robot control and sensing

8.2.3. Probabilistic Diagnosis

Probability theory provides a natural representation for the uncertainty inherent in

real-world robotic systems. As a consequence of ignoring uncertainty when diagnosing

real-world systems a large number of false positives may be triggered due to sensor noise,

actuator noise, or uncertain environmental interactions. In addition some faults may seem

unlikely given a single measurement, but a sequence of measurements may reinforce the

fault. Probability theory provides a convenient representation for maintaining fault hy-

pothesis that may not instantly seem like the most likely state. This dissertation presents a

probabilistic approach to diagnosis.

8.2.4. Interface to Planners/Controllers

The approach presented in this thesis assumes a diagnosis system that is independent

of the controller that is controlling the rover. This is currently the typical architecture for

most NASA rovers. One of the reasons for the architecture is that it is modular and allows

a lot of flexibility in the types of planners/controllers that may be used for controlling the

rover and for recovering from faults. A schematic diagram of the system is shown in figure

8.1. The fault detection and identification component receives control input and sensor

measurements and provides a distribution over the fault and normal operation states. This

information may be used by a recovery system/controller to recover from the fault. Both

the controllers shown may even be combined; they would certainly communicate. This

thesis has focused on the aspects of fault diagnosis. The ultimate goal is to allow robots to

recover from faults, which will be an area of future work.

For rovers with low computational power and high cost associated with failure the

modular architecture from figure 8.1 is likely to be the only practical approach for some

time. One of the advantages of the approach presented in this thesis is that it can easily be

[Draft: 19 October 2004—19:11 ] 85

8.2 DISCUSSION

adapted to an anytime approach where the accuracy or the estimate can be made to match

available computation.

8.2.5. Lessons Learned from Experiments with Real Rovers

The research in this dissertation is based on experience with a variety of different

rovers.

The research was inspired by the desire to address shortcomings of a discrete model-

based diagnosis system called Livingstone (Williams and Nayak, 1996) that performed

poorly at diagnosing faults on the K9 rover at NASA Ames. There was a need for a method

that could handle a hybrid state space (since the purely qualitative models were too coarse)

and uncertainty.

Around the same time K9 experienced a couple of instances of stuck wheel faults that

caused the “rampant-K9” configuration shown in figure 7.7. Data logging was not active at

the time the faults occurred. Recreating the same scenario on K9 proved to be impossible

since rover engineers were understandably reluctant to put the rover in a configuration

they thought was dangerous. In general this is a major problem with collecting data with

faults on a real rover. As a result of this experience constant data logging whenever K9 was

operational was put in effect.

In another early experiment a simple constraint based Health Monitoring system was

implemented on the Hyperion rover. Real-world field tests in the Arctic and the Atacama

desert demonstrated a clear need for a system that could handle uncertainty. Far too many

false positives were triggered with the constraint based system as shown in figure 8.2. In

addition choosing thresholds proved to be challenging since these thresholds sometimes

differ based on the mode of operation. Another challenge when working with data from

real rovers was that ground truth for faults is generally not available. In a number of cases

additional sensors or components may be used to collect ground truth data. For example a

unactuated free-wheel attached to the back of a rover may be used to collect ground truth

for slip.

8.2.6. Role of simulation in rover fault diagnosis

As discussed in section 8.2.5, it is often very difficult to obtain data from real rovers in

fault modes. This data is required for building probabilistic models for fault monitoring.

One way to get around this is to build analytical models, but this is not always feasible

when faults involve complex interactions with the environment.

[Draft: 19 October 2004—19:11 ] 86

8.3 FUTURE DIRECTIONS

1 2 3 5 6 7 9 10 11 17 21 320

50

100

150

200

250

300

350

Fault number

Num

ber

of o

ccur

renc

es

Faults detected by Health Monitor over the entire experiment

NO NAV ARCSHIGH ROLLHIGH PITCHOFF NAV MAPBATTERY LO/HIMISSED MISSIONPLAN WAYPOINT

(a) Faults detected

1 2 3 5 6 7 9 10 11 17 21 320

100

200

300

400

500

600

700

Fault number

Num

ber

of o

ccur

renc

es

Proportion of false positives for each fault

False positives

(b) False positives

Figure 8.2. (a) Faults detected by the Health Monitor on board the Hyperion roverduring a field test. (b) False positives encountered during the same field test.

Often a high fidelity dynamic simulation is the only option. In the future it will be

useful to develop techniques for learning models in simulation that may then be updated

based on available real data.

8.3. Future Directions

8.3.1. Adaptive Models

Most systems degrade over a period of time. There is a fine line between changing a

normal model to reflect changes in the rover vs the rover gradually moving to a fault mode.

A method for online adaptation of the models could improve the accuracy of diagnosis for

operation over a long duration.

8.3.2. Switching Hierarchies

When creating a hierarchy, faults may cluster differently depending on the control

mode. An interesting line of future research would be to learn a number of possible clusters

and extend the hierarchical method to deal with multiple abstractions of the same states.

Since the VRPF maintains a distribution over physical states, when a hierarchy is

switched particles would just be percolated up to the right level of abstraction in the new

hierarchy.

[Draft: 19 October 2004—19:11 ] 87

8.3 FUTURE DIRECTIONS

8.3.3. Factored Representation

When diagnosing a complex rover system, there are likely to be variables that are

independent of other variables. A factored representation could take advantage of these

independencies and make the computation more efficient.

In a factored state space variables in the state space are represented independently

in the physical model. E.g., if there are K fault and operational variables in the system,

then the discrete state of the system would be represented by K variables. A non-factored

Hidden Markov Model style representation uses a single random variable, dt ∈ {1 . . .m}.It represents the state of the system as a distribution over m discrete modes. The struc-

tured PF represents the state using a set of factored random variables {D1 . . . DK}. Both

representations cover the occurrence of every possible combination of faults, but the fac-

torized representation does so in compact manner. The factored representation may have

exponentially fewer parameters than the HMM representation. This factored representa-

tion may also be called a Dynamic Bayes Net (DBN). The factored HMM, (and any DBN in

general), may be converted to a non-factored HMM by creating a single variable that is the

Cartesian product of the component state variables. Note that the factored representation

provides a compact representation of the process, but is still not a compact representation

of the belief.

In the future it would be interesting to implement a filter that used a structured repre-

sentation that combines abstraction (as in the VRPF) with factorization.

For example, consider a state space with three discrete variables X1, X2, and X3. If X1

and X2 were (conditionally) independent from X3. The state space could be factored as:

X1X2 and X3, where X1X2 represents a joint state space. In addition if there were subsets

of states in the joint space X1X2 that were more similar, they could be clustered into a

variable resolution hierarchy. A similar structure could also be applied to the continuous

states.

[Draft: 19 October 2004—19:11 ] 88

BIBLIOGRAPHY

Ackerson, G. and Fu, K. (1970). On state estimation in switching environments. In IEEE

Transactions on Automatic Control, pages 10–17.

Astrom, K. (1965). Optimal control of markov decision processes with incomplete state

estimation. In J. Math. Anal. Appl.

Avitzour, D. (1995). A stochastic simulation bayesian approach to multitarget tracking. In

IEEE Transaction on Aerospace and Electronic systems.

Bar-Shalom, L. C. Y. (1988). Control of discrete-time hybrid stochastic systems. In IEEE

Transactions on Automatic Control, volume 37, pages 1522–1527.

Bar-Shalom, Y. and Fortmann, T. R. (1988). Tracking and Data Association. Academic Press.

Bares, J. E. and Wettergreen, D. S. (1999). Dante ii: Technical description, results and lessons

learned. In International Journal of Robotics Research, volume 18, pages 621–64, July.

Baydar, C. (2002). Diagnosis of complex failures in robotic assembly systems using vir-

tual factories. In AAI/KDD/UAI-2002 Joint Workshop on Real-Time Decision Support and

Diagnosis Systems.

Bererton, C. and Khosla, P. (2002). An analysis of cooperative repair capabilities in a team

of robots. In Proceedings of the IEEE International Conference on Robotics and Automation.

Blom, H. (1984). An efficient filter for abruptly changing systems. In In Proceedings of the

IEEE Conference of Decision and Control, pages 656–658.

Blom, H. and Bar-Shalom, Y. (1988). The interacting multiple model algorithm for systems

with markovian switching coefficients. In IEEE Transactions on Automatic Control, pages

780–783.

Boyen, X. and Koller, D. (1998). Tractable inference for complex stochastic processes. In

Proceedings of the 14th Annual Conference on Uncertainty in Artificial Intelligence (UAI-98),

pages 33–42, San Francisco, CA. Morgan Kaufmann Publishers.

Boyen, X. and Koller, D. (1999). Exploiting the architecture of dynamic systems. In Pro-

ceedings of the sixteenth national conference on artificial intelligence and eleventh innovation

BIBLIOGRAPHY

applications of AI conference on Artificial intelligence and innovative applications of artificial

intelligence, pages 313–320. American Association for Artificial Intelligence.

Brown, J. S., Burton, R. R., and de Kleer, J. (1982). Pedagogical, natural language and

knowledge engineering techniques in sophie i, ii and iii. In Intelligent Tutoring Systems.

Carlson, J. (2004). Technical report for the safety security rescue center: Icra 2004 robot

fault diagnosis workshop. Technical report, University of South Florida.

Carlson, J. and Murphy, R. (2003). Reliability analysis of mobile robots. In Proceedings of the

IEEE International Conference on Robotics and Automation.

Carpenter, J., Clifford, P., and Fearnhead, P. (1999). Improved particle filter for non-linear

problems. In IEE Proceedings - Radar, Sonar and Navigation, number 147, pages 2–7.

Chang, C. and Athanas, M. (1978). State estimation for discrete systems with switching

parameters. In IEEE Transactions on Aerospace and Electronic Systems, pages 418–425.

Chittaro, L. and Ranon, R. (2004). Hierarchical model-based diagnosis based on structural

abstraction. Artif. Intell., 155(1-2), 147–182.

Crisan, D. and Lyons, T. (1997). A particle approximation of the solution of the kushner

strantonovitch equation. Technical report, Imperial College, London, UK.

Dague, P., Jehl, O., and Taillibert, P. (1990). An interval propagation and conflict recogni-

tion engine for diagnosing continuous dynamic systems. In Expert Systems in Engineer-

ing:Lecture Nites in AI.

daily report, M. (2004). About face: Rover engineers change the rules for driving.

http://marsrovers.jpl.nasa.gov/spotlight/20040716.html.

Dasarthy, B. (1990). Nearest neighbor pattern classification techniques. In IEEE Computer

Society Press.

de Freitas, N. (2002). Rao-blackwellised particle filtering for fault diagnosis. In IEEE

Aerospace.

de Kleer, J. and Williams, B. C. (1987). Diagnosing multiple faults. In Readings in Nonmono-

tonic Reasoning.

de Kleer, J. and Williams, B. C. (1989). Diagnosis with behavioral modes. In In Proceedings

of the 11th International Joint Conference on Artificial Intelligence.

Deans, M. (2002). Bearings-Only Localization and Mapping. PhD thesis, Carnegie Mellon

University.

Dearden, R. and Clancy, D. (2002). Particle filters for real-time fault detection in planetary

rovers. In Twelfth International Workshop On Principles Of Diagnosis, DX-01.

Djath, K., Dufaut, M., and Wolf, D. (2000). Mobile robot multisensor reconfiguration. In

IEEE Intelligent Vehicles Symposium.

[Draft: 19 October 2004—19:11 ] 90

BIBLIOGRAPHY

Doucet, A. (1998). On sequential monte carlo sampling methods for bayesian filtering.

Technical report, Cambridge University Engineering Department, Cambridge, England.

Doucet, A. and Crisan, D. (2002). A survey of convergence results on particle filtering for

practitioners. In IEEE Trans. Signal Processing.

Doucet, A., de Freitas, N., and Gordon, N. (Eds.). (2001). Sequential Monte Carlo Methods in

Practice. Springer-Verlag.

Doucet, A., Godsill, S. J., and West, M. (2000). Monte carlo filtering and smoothing with

application to time-varying spectral estimation. In IEEE International Conference on Acous-

tics, Speech and Signal Processing.

Doucet, A., Gordon, N., and Krishnamurthy, V. (1999). Particle filters for state estimation

of jump markov linear systems. Technical report, Cambridge University.

Dvorak, D. and Kuipers, B. (1989). Model-based monitoring of dynamic systems. In IJCAI.

Elsharkawy, A. and Guedouar, L. An inverse analysis for steadystate elastohydrodynamic

lubrication of onelayered journal bearings.

Fernandez, J. (2000). Supervision, detection, diagnosis and exception recovery in autonomous

mobile robots. PhD thesis, University of Vigo.

Fox, D., Burgard, W., and Thrun, S. (1999). Markov localization for mobile robots in dy-

namic environments. In jair, volume 11, pages 391–427.

Funiak, S. and Williams, B. (2003). Multi-modal particle filtering for hybrid systems with

autonomous mode transitions. In Workshop on Princlipples of Diagnosis.

Ghallab, M. (1985). Task execution monitoring by compiles production rules in an ad-

vanced multi-sensor robot. In Robotics Research, the second International Symposium.

Glass, B., Erickson, W., and K.J.Swanson (1991). Texsys: A large-scale demonstration of

model-based real-time control of space station subsystem. In 7th International conference

on AI applications.

Goel, P., Dedeoglu, G., Roumeliotis, S., and Sukhatme, G. S. (2000). Fault detection and

identification in a mobile robot using multiple model estimation and neural network. In

n Proc. IEEE International Conference on Robotics and Automation.

Gordon, N., Salmond, D., and Smith, A. (1993a). Novel approach to nonlinear/non gauss-

ian bayesian state estimation. In IEE Proceedings-F, volume 140, pages 107–113.

Gordon, N., Salmond, D., and Smith, A. (1993b). Novel approach to nonlinear/non-

Gaussian Bayesian state estimation. In IEEE Proceedings of Radar, Sonar, and Navigation,

volume 140, pages 107–113.

Guckenbiehl, T. and G, S.-R. (1990). Sidia: Extending prediction-based diagnosis of dy-

namic models. Technical report, Price waterhouse technology center.

[Draft: 19 October 2004—19:11 ] 91

BIBLIOGRAPHY

H. Olsson, K. J. Astrom, C. C. d. W. M. G. and Lischinsky, P. (1999). Friction models and

friction compensation. In European Journal of Control, volume 4, pages 176–195, Dec.

Hamilton, K., Lane, D., Taylor, N., and Brown, K.

Hamscher, W. (1991). Modeling digital circuits for troubleshooting. In AI.

Harmon, P., Maus, R., and Morrissey, W. (1988). Expert Systems: Tools and Applications. John

Wiley and Sons.

Hofbaur, M. and Williams, B. (2002). Mode estimation of probabilistic hybrid systems. In

Hybrid Systems: Computation and Control.

Holtzblatt, L., Neiberg, M., and Piazza, R. (1991). Temporalresoning in an atms. Technical

report, MITRECorporation.

Hutter, F. and Dearden, R. (2003). The gaussian particle filter for diagnosis of non-linear

systems. In Proceedings of the 5th IFAC Symposium on Fault Detection, Supervision and Safety

of Technical Processes.

Isard, M. and Blake, A. (1998). CONDENSATION: conditional density propagation for

visual tracking. International Journal of Computer Vision, 29(1), 5–28.

Isermann, R. and Balle, P. (1997). Trends in the application of model-based fault detection

and diagnosis of technical processes. In Control Eng. Practice.

Jordan., M. I. (2003). Graphical models. In Statistical Science (Special Issue on Bayesian Statis-

tics).

Julier, S. J. and Uhlmann, J. K. (1996). A general method of approximating nonlinear trans-

formations of probability distributions. Technical report, Robotics Research Group, De-

partment of Engineering Science, University of Oxford.

Julier, and J. K. Uhlmann, S. J. (2002). The Scaled Unscented Transformation. In Proceedings

of the IEEE American Control Conference, pages 4555–4559, Anchorage AK, USA. IEEE.

Kaelbling, L., Littman, M., and Cassandra, A. (1998). Planning and acting in partially

observable stochastic domains. Artificial Intelligence, 101.

Kalman, R. (1960). A new approach to linear filtering and prediction problems. Journal of

Basic Engineering, pages 34–45.

Kanazawa, K., Koller, D., and Russel, S. (1995). Stochastic simulation algorithms for dy-

namic probabilistic networks. In In Proceedings of Uncertainly in Artificial Intelligence, vol-

ume 89.

Kitagawa, G. (1993). A monte carlo filtering and smoothing method for non-gaussian non-

linear state space models. In Proceedings of the 2nd U.S.-Japan Joint Seminar on Statistical

Time Series Analysis.

[Draft: 19 October 2004—19:11 ] 92

BIBLIOGRAPHY

Koller, D. and Fratkina, R. (1998). Using learning for approximation in stochastic processes.

In ICML.

Kong, A., Liu, J., and Wong, W. H. (1994). Sequential imputations and bayesian missing

data problems. In Journal of the American Statistical Association.

Koppolu, S. and Chatterjee, A. (2001). Hierarchical diagnosis of identical units in a system.

In IEEE Transactions on Computers.

Kurien, J. and Nayak, P. (2000). Back to the future with consistency-based trajectory track-

ing. In aaai.

Lamine, K. and Kabanza, F. History checking of temporal fuzzy logic formulas for moni-

toring behavior-based mobile robots.

Leger, C. Darwn2k. http://darwin2k.sourceforge.net/.

Leger, P. (2000). Darwin2K: An Evolutionary Approach to Automated Design for Robotics.

Kluwer Academic Publishers.

Lerner, U., Moses, B., Scott, M., McIlraith, S., and Koller, D. (2002). Monitoring a complex

physical system using a hybrid dynamic bayes net. In uai.

Lerner, U., Parr, R., Koller, D., and Biswas, G. (2000). Bayesian fault detection and diagnosis

in dynamic systems. In aaai.

Leuschen, M., Walker, I., and Cavallaro, J. Robotic reliability through fuzzy markov mod-

els.

Liu, J. and Chen, R. (1998). Sequential Monte Carlo methods for dynamic systems. In jasa,

volume 93.

Liu, R. C. S. (2000). Mixture kalman filters. In Journal of Royal Statistical Society.

Long, M., Murphy, R., and Parker, L.

McIlraith, S. (1998). Explanatory diagnosis: Conjecture actions to explain observations. In

In Proceedings of the 6th International Conference on Principles of Knowledge Representation

and Reasoning.

McKay, D. (2003). Information Theory, Inference, and Learning Algorithms, chapter 29–30,

pages 356–398. Cambridge University Press.

McLauchlan, P. (1999). The variable state dimension filter applied to surface-based struc-

ture from motion. Technical report, University of Surrey.

Metropolis, N. and Ulam, S. (1949). The monte carlo method. In Journal of the American

Statistical Association, volume 44.

Minka, T. (2001). A family of algorithms for approximate Bayesian inference. PhD thesis, MIT.

Morales-Mendendez, R., de Freitas, N., and Poole, D. (2002). Real-time monitoring of com-

plex industrial processes with particle filters. In NIPS.

[Draft: 19 October 2004—19:11 ] 93

BIBLIOGRAPHY

Murphy, R. and Hershberger, D. (2000). Handling sensing failures in autonomous mobile

robots. In ijrr.

Murphy, R. R. and Hershberger, D. (1996). Classifying and recovering from sensing failures

in autonomous mobile robots. In aaai.

NASA (2000). Mars program independent assessment team summary report. Report pub-

lished by investigation team.

Ng, B., Peshkin, L., and Pfeffer, A. (2002). Factored particles for scalable monitoring. In

UAI.

Parker, L. E. (1998). Alliance: An architecture for fault tolerant multi-robot cooperation. In

IEEE Transactions on Robotics and Automation.

Pitt, M. and Shephard, N. (1999). Filtering via simulation: auxiliary particle filter. In jasa,

volume 94.

R. B. Washburn, T. G. Allen, D. T. (1985). Performance analysis for hybrid state estimation

problems. In In Proceedings of the American Control Conference, pages 1047–1053.

Rauch, H., Tung, F., and Striebel, C. (1965). Maximum likelihood estimates of linear dy-

namic systems. In American Institute of Aeronautics and Astronautics Journal.

Roumeliotis, S. I., Sukhatme, G., and Bekey, G. (1998). Sensor fault detection and identi-

fication in a mobile robot. In IEEE/RSJ International Conference on Intelligent Robots and

Systems. Vancouver, Canada.

Rubin, D. (1987). Comment on “The calculation of posterior distributions by data augmen-

tation” by tanner, m.a. and wong w.hmscher. In jasa, volume 82, page 543.

Rubin, D. (1988). Using the SIR algorithm to simulate posterior distributions. Oxford University

Press.

Rubinstein, R. Y. (1981). Simulation and Monte Carlo method. In John Wiley and Sons.

Scheding, S. (2000). High-integrity navigation: a frequency-domain approach. In IEEE

Transactions on Control Systems Technology.

Shortliffe, E. (1976). Mycin: Computer-based medical consultations. In Elsevier, New York.

Silberman, J. (1999). Robot Orthogonal Defect Classification Toward an In-Process Measurement

System for Mobile Robot Development. PhD thesis, Carnegie Mellon University.

Simmons, R. (1994). Becoming increasingly reliable. In Proc. of 2nd Intl. Conference on Arti-

ficial Intelligence Planning Systems.

Soika, M. (1997). A sensor failure detection framework for autonomous mobile robots. In

IEEE/RSJ International Conference on Intelligent Robots and Systems.

Sondik, E. J. (1971). The optimal control of partially observable Markov Decision Processes. PhD

thesis, Stanford.

[Draft: 19 October 2004—19:11 ] 94

BIBLIOGRAPHY

Stuck, E. R. (1992). Detecting and Diagnosing Mistakes in Inexact Vision-Based Navigation. PhD

thesis, Department of Computer Science, University of Minnesota.

Stuck, E. R. (1995). Detecting and diagnosing navigational mistakes. In Proceedings of the

IEEE International International Conference on Intelligent Robotics and Systems.

Szolovits, P. and Pauker, S. G. (1978). Categorical and probabilistic reasoning in medical

diagnosis. In Artificial Intelligence.

Szolovits, P. and Pauker, S. G. (1993). Categorical and probabilistic reasoning in medical

diagnosis revisited. In Artificial Intelligence.

Tanner, M. A. (1996). Tools for Statistical Inference. Springer Velag.

Thrun, S., Fox, D., and Burgard, W. (2000). Monte Carlo localization with mixture proposal

distribution. In Proceedings of the AAAI National Conference on Artificial Intelligence. AAAI.

Thrun, S., Fox, D., Burgard, W., and Dellaert, F. (2001). obust monte carlo localization for

mobile robots. In Artificial Intelligence.

Thrun, S., Langford, J., and Verma, V. (2001). Risk sensitive particle filters. In nips.

Tugnait, J. K. (1982). Detection and estimation of abruptly changing systems. In Automatica,

pages 607–615.

van der Merwe, R., A. Doucet, N. d. F., and Wan, E. (2000). The unscented particle filter. In

In Advances in Neural Information Processing Systems (NIPS13).

van der Merwe, R., de Freitas, N., Doucet, A., and Wan, E. (2000). The unscented parti-

cle filter. Technical report, Cambridge University Engineering Department, Cambridge,

England.

Verma, V. (2001). Anecdotes from rover field operations. RIACS Summer student research

program report- NASA Ames research center.

Verma, V., Langford, J., and Simmons, R. (2001). Non-parametric fault identification for

space rovers. In isairas.

Verma, V., Thrun, S., and Simmons, R. (2003). Variable resolution particle filter. In Interna-

tional Joint Conference of Artificial Intelligence.

Visinsky, M., Cavallaro, J., and Walker, I. (1995). A dynamic fault tolerance framework for

remote robots. In IEEE Transactions on Robotics and Automation.

Washington, R. (2000). On-board real-time state and fault identification for rovers. In icra.

Wettergreen, D. S., Dias, M., Shamah, B., Teza, J., Tompkins, P., Urmson, C., Wagner, M.,

and Whittaker, W. (2002). First experiment in sun-synchronous exploration. In Interna-

tional Conference on Robotics and Automation.

Willeke, T. and Dearden, R.

[Draft: 19 October 2004—19:11 ] 95

BIBLIOGRAPHY

Williams, B. C. and Nayak, P. (1996). A model-based approach to reactive self-configuring

systems. In aaai.

Yan, T., Ota, J., Nakamura, A., Arai, T., and Kuwahara, N. Concept design of remote fault

diagnosis system for autonomous mobile robots.

[Draft: 19 October 2004—19:11 ] 96

APPENDIX A

Simulation of Hyperion rover for Fault Detection andIdentification

Hyperion is a mobile robot that was developed at Carnegie Mellon University. It was ori-

gianlly developed for Sun-synchronous navigation (Wettergreen et al., 2002) in the Arctic.

It has also been used recently for an investigation of life in extreme environments in the

Ataama desert in Chile. Figure A.1 shows a schematic top view of the robot that was used

in our model. Hyperion has four actuated wheels. The back frame is rigid and the front

axle is steerable, but not actuated. W1, W2, W3 and W4 are the four wheels and v1, v2, v3

and v4 are their respective transitional velocities. α is the steering angle and θ (not shown

in the diagram) is the vehicle’s rotation with respect to the world coordinate frame. Rx and

Ry represent x and y coordinates in the rover frame and Sx and Sy represent the x and y

coordinates in the steering frame. L is the wheel base and B is the track or rover width.

B

L

Ry

Rx

α

SySx

W1

W2

W4W3

v1

v2

v3 v4

Figure A.1. Schematic view of the Hyperion rover

APPENDIX A. HYPERION

The fault model for Hyperion included a very small subset of possible failures. We

were interested in determining if any of the four wheel motors were faulty, any wheel was

broken and locked, or if any of the drive gears were broken. This is important because in

the event of any of the above faults, the commanded velocity at one or more wheels has to

be altered to achieve the desired motion. The problem was to determine if any one of these

faults occurred given the commanded velocity at each wheel, noisy measurements of the

position and orientation of the rover, and the steering angle. For experimental purposes

we used a simple kinematic model of Hyperion. The wheel positions, P1, P2, P3 and P4 of

each wheel in the rover coordinate frame are as follows:

P1 =

(B

2cosα,

L

2− B

2sinα

)

P2 =

(B

2cosα,

L

2+B

2sinα

)

P3 =

(−B

2,−L

2

)

P4 =

(B

2,−L

2

)(A.1)

The instantaneous motion of the robot is represented by the vector M = [∆x,∆y,∆θ,∆α]T.

The kinematic equations for the instantaneous motion at each wheel were:

M1 =

[∆x+

(L

2+B

2sinα

)∆θ +

B

2∆α

]x+

[∆y − B

2cosα∆θ − B

2cosα∆α

]y

M2 =

[∆x−

(L

2− B

2sinα

)∆θ +

B

2∆α

]x+

[∆y +

B

2cosα∆θ +

B

2cosα∆α

]y

M3 =

[∆x+

L

2∆θ

]x+

[∆y − B

2∆θ

]y

M4 =

[∆x+

L

2∆θ

]x+

[∆y +

B

2∆θ

]y (A.2)

The ten discrete states that were estimated were:

1. normal state

2. wheel1 and wheel2 motor or gear broken

3. wheel3 broken

4. wheel4 broken

5. wheel1 stuck

6. wheel2 stuck

7. wheel3 stuck

8. wheel4 stuck

9. wheel3 gear broken

10. wheel4 gear broken

[Draft: 19 October 2004—19:11 ] 98

APPENDIX A. HYPERION

The reason that wheel1 or wheel2 motor or gear broken were lumped together as a single fault

was because when either of the front wheels could not be controlled, there was no way to

control the heading of the rover and the mission would have to be aborted. Aggregating

faults that result in the same response reduced the dimensionality of the state space. On

the other hand, if a motor on one of the back wheels was determined to be broken, then the

control input could still have been modified to produce the desired motion. The dynamics

of the rover were different in each of the discrete states. The continuous variables used

to track system dynamics is represented by the vector C = [x, y, θ, α]T and control by A =

[u1, u2, u3, u4]T, where the uiswere commanded wheel velocities. The sensor measurement

is the rover pose (x, y, θ) and the steering angle α from the steering potentiometer. The

measurement matrix was O = [x, y, θ, α].

[Draft: 19 October 2004—19:11 ] 99

APPENDIX B

List of Fault Names and Corresponding Numbers

1 Left Rear wheel slipping while driving straight2 Left Middle wheel slipping while driving straight3 Left Front wheel slipping while driving straight4 Right Rear wheel slipping while driving straight5 Right Middle wheel slipping while driving straight6 Right Front wheel slipping while driving straight7 Normal straight driving

22 Left Front and Right Front wheels slipping while driving straight23 Left Front and Right Middle wheels slipping while driving straight24 Left Front and Right Rear wheels slipping while driving straight25 Left Middle and Left Front wheels slipping while driving straight26 Left Middle and Right Front wheels slipping while driving straight27 Left Middle and Right Middle wheels slipping while driving straight28 Left Middle and Right Rear wheels slipping while driving straight29 Left Rear and Left Front wheels slipping while driving straight30 Left Rear and Left Middle wheels slipping while driving straight31 Left Rear and Right Front wheels slipping while driving straight32 Left Rear and Right Middle wheels slipping while driving straight33 Left Rear and Right Rear wheels slipping while driving straight34 Right Middle and Right Front wheels slipping while driving straight35 Right Rear and Right Front wheels slipping while driving straight36 Right Rear and Right Middle wheels slipping while driving straight37 Left Front, Right Middle, and Right Front wheels slipping while driving straight38 Left Front, Right Rear, and Right Front wheels slipping while driving straight39 Left Front, Right Rear, and Right Middle wheels slipping while driving straight40 Left Middle, Left Front, and Right Front wheels slipping while driving straight

APPENDIX B. FAULT NAMES

41 Left Middle, Left Front, and Right Middle wheels slipping while driving straight42 Left Middle, Left Front, and Right Rear wheels slipping while driving straight43 Left Middle, Right Middle, and Right Front wheels slipping while driving straight44 Left Middle, Right Rear, and Right Front wheels slipping while driving straight45 Left Middle, Right Rear, and Right Middle wheels slipping while driving straight46 Left Rear, Left Front, and Right Front wheels slipping while driving straight47 Left Rear, Left Front, and Right Middle wheels slipping while driving straight48 Left Rear, Left Front, and Right Rear wheels slipping while driving straight49 Left Rear, Left Middle, and Left Front wheels slipping while driving straight50 Left Rear, Left Middle, and Right Front wheels slipping while driving straight51 Left Rear, Left Middle, and Right Middle wheels slipping while driving straight52 Left Rear, Left Middle, and Right Rear wheels slipping while driving straight53 Left Rear, Right Middle, and Right Front wheels slipping while driving straight54 Left Rear, Right Rear, and Right Front wheels slipping while driving straight55 Left Rear, Right Rear, and Right Middle wheels slipping while driving straight56 Right Rear, Right Middle, and Right Front wheels slipping while driving straight57 Left Front, Right Rear, Right Middle, and Right Front wheels slipping while driving

straight58 Left Middle, Left Front, Right Rear, and Right Middle wheels slipping while driving

straight59 Left Rear, Left Middle, Left Front, and Right Rear wheels slipping while driving straight60 Left Rear, Left Middle, Left Front, and Right Front wheels slipping while driving

straight61 Left Rear, Left Middle, Right Middle, and Right Front wheels slipping while driving

straight62 Left Rear, Right Rear, Right Middle, and Right Front wheels slipping while driving

straight63 Left Middle, Right Rear, Right Middle, and Right Front wheels slipping while driving

straight64 Left Rear, Left Front, Right Rear, Right Middle wheels slipping while driving straight65 Left Middle, Left Front, Right Rear, and Right Front wheels slipping while driving

straight66 Left Rear, Left Middle, Left Front, and Right Middle wheels slipping while driving

straight67 Left Rear, Left Middle, Right Rear, Right Front wheels slipping while driving straight68 Left Rear, Left Front, Right Middle, and Right Front wheels slipping while driving

straight69 Left Middle, Left Front, Right Middle, and Right Front wheels slipping while driving

straight70 Left Rear, Left Middle, Right Rear, and Right Middle wheels slipping while driving

straight

[Draft: 19 October 2004—19:11 ] 101


71 Left Rear, Left Front, Right Rear, and Right Front wheels slipping while driving straight72 Left Rear wheel stuck while driving straight73 Left Middle wheel stuck while driving straight74 Left Front wheel stuck while driving straight75 Right Rear wheel stuck while driving straight76 Right Middle wheel stuck while driving straight77 Right Front wheel stuck while driving straight78 Left Front and Right Front wheels stuck while driving straight79 Left Front and Right Middle wheels stuck while driving straight80 Left Front and Right Rear wheels stuck while driving straight81 Left Middle and Left Front wheels stuck while driving straight82 Left Middle and Right Front wheels stuck while driving straight83 Left Middle and Right Middle wheels stuck while driving straight84 Left Middle and Right Rear wheels stuck while driving straight85 Left Rear and Left Front wheels stuck while driving straight86 Left Rear and Left Middle wheels stuck while driving straight87 Left Rear and Right Front wheels stuck while driving straight88 Left Rear and Right Middle wheels stuck while driving straight89 Left Rear and Right Rear wheels stuck while driving straight90 Right Middle and Right Front wheels stuck while driving straight91 Right Rear and Right Front wheels stuck while driving straight92 Right Rear and Right Middle wheels stuck while driving straight93 Left Front, Right Middle, and Right Front wheels stuck while driving straight94 Left Front, Right Rear, and Right Front wheels stuck while driving straight95 Left Front, Right Rear, and Right Middle wheels stuck while driving straight96 Left Middle, Left Front, and Right Front wheels stuck while driving straight97 Left Middle, Left Front, and Right Middle wheels stuck while driving straight98 Left Middle, Left Front, and Right Rear wheels stuck while driving straight99 Left Middle, Right Middle, and Right Front wheels stuck while driving straight

100 Left Middle, Right Rear, and Right Front wheels stuck while driving straight101 Left Middle, Right Rear, and Right Middle wheels stuck while driving straight102 Left Rear, Left Front, and Right Front wheels stuck while driving straight103 Left Rear, Left Front, and Right Middle wheels stuck while driving straight104 Left Rear, Left Front, and Right Rear wheels stuck while driving straight105 Left Rear, Left Middle, and Right Front wheels stuck while driving straight106 Left Rear, Left Middle, and Right Middle wheels stuck while driving straight107 Left Rear, Left Middle, and Right Rear wheels stuck while driving straight108 Left Rear, Right Middle, and Right Front wheels stuck while driving straight109 Left Rear, Right Rear, and Right Front wheels stuck while driving straight110 Left Rear, Right Rearm, and Right Middle wheels stuck while driving straight

[Draft: 19 October 2004—19:11 ] 102


111 Left Frint, Right Rear, Right Middle, and Right Front wheels stuck while drivingstraight

112 Left Middle, Left Front, Right Middle, and Right Front wheels stuck while drivingstraight

113 Left Middle, Left Front, Right Rear, and Right Front wheels stuck while driving straight114 Left Middle, Left Front, Right Rear, and Right Middle wheels stuck while driving

straight115 Left Middle, Left Rear, Right Middle, and Right Front wheels stuck while driving

straight116 Left Middle, Left Rear, Right Rear, and Right Front wheels stuck while driving straight117 Left Middle, Left Rear, Right Rear, and Right Middle wheels stuck while driving straight118 Left Rear, Left Front, Right Middle and Right Front wheels stuck while driving straight119 Left Rear, Left Front, Right Rear, and Right Front wheels stuck while driving straight120 Left Rear, Left Front, Right Rear, Right Middle wheels stuck while driving straight121 Left Rear, Right Rear, Right Middle, and Right Front wheels stuck while driving straight122 Lubricant leak in Left Rear wheel123 Lubricant leak in Left Middle wheel124 Lubricant leak in Left Front wheel125 Lubricant leak in Right Rear wheel126 Lubricant leak in Right Middle wheel127 Lubricant leak in Right Front wheel128 Lubricant leak in Left Frontm and Right Front wheels129 Lubricant leak in Left Front and Right Middle wheels130 Lubricant leak in Left Front and Right Rear wheels131 Lubricant leak in Left Middle and Left Front wheels132 Lubricant leak in Left Middle and Right Front wheels133 Lubricant leak in Left Middle and Right Middle wheels134 Lubricant leak in Left Middle and Right Rear wheels135 Lubricant leak in Left Rear and Left Front wheels136 Lubricant leak in Left Rear and Left Middle wheels137 Lubricant leak in Left Rear and Right Front wheels138 Lubricant leak in Left Rear and Right Middle wheels139 Lubricant leak in Left Rear and Right Rear wheels140 Lubricant leak in Right Middle and Right Front wheels141 Lubricant leak in Right Rear and Right Front wheels142 Lubricant leak in Right Rear and Right Middle wheels143 Lubricant leak in Left Front, Right Middle, and Right Front wheels144 Lubricant leak in Left Front, Right Rear, and Right Front wheels145 Lubricant leak in Left Front, Right Rear, and Right Middle wheels146 Lubricant leak in Left Middle, Left Front, and Right Front wheels147 Lubricant leak in Left Middle, Left Front, and Right Middle wheels148 Lubricant leak in Left Middle, Left Front, and Right Rear wheels149 Lubricant leak in Left Middle, Right Middle, and Right Front wheels150 Lubricant leak in Left Middle, Right Rear, and Right Front wheels

[Draft: 19 October 2004—19:11 ] 103


151 Lubricant leak in Left Middle, Right Rear, and Right Middle wheels152 Lubricant leak in Left Rear, Left Front, and Right Front wheels153 Lubricant leak in Left Rear, Left Front, and Right Middle wheels154 Lubricant leak in Left Rear, Left Front, and Right Rear wheels155 Lubricant leak in Left Rear, Left Middle, and Left Front wheels156 Lubricant leak in Left Rear, Left Middle, and Right Front wheels157 Lubricant leak in Left Rear, Left Middle, and Right Middle wheels158 Lubricant leak in Left Rear, Left Middle, and Right Rear wheels159 Lubricant leak in left Rear, Right Middle, and Right Front wheels160 Lubricant leak in Left Rear, Right Rear, and Right Front wheels161 Lubricant leak in Left Rear, Right Rear, and Right Middle wheels162 Lubricant leak in Right Rear, Right Middle, and Right Front wheels163 Lubricant leak in Left Front, Right Rear, Right Middle, and Right Front wheels164 Lubricant leak in Left Middle, Left Front, Right Rear, and Right Middle wheels165 Lubricant leak in Left Rear, Left Middle, Left Front, and Right Rear wheels166 Lubricant leak in Left Rear, Left Middle, Left Front, and Right Front wheels167 Lubricant leak in Left Rear, Left Middle, Right Middle, and Right Ftont weels168 Lubricant leak in Left Rear, Right Rear, Right Middle, and Right Front wheels169 Lubricant leak in Left Middle, Right Rear, Right Middle, and Right Front wheels170 Lubricant leak in Left Rear, Left Front, Right Rear, and Right Middle wheels171 Lubricant leak in Left Middle, Left Front, Right Rear, and Right Front wheels172 Lubricant leak in Left Rear, Left Middle, Left Front, and Right Middle wheels173 Lubricant leak in Left Rear, Left Middle, Right Rear, and Right Front wheels174 Lubricant leak in Left Rear, Left Front, Right Middle, and Right Front wheels175 Lubricant leak in Left Middle, Left Front, Right Middle, and Right Front wheels176 Lubricant leak in Left Rear, Left Middle, Right Rear, and Right middle wheels177 Lubricant leak in Left Rear, Left Front, Right Rear, and Right Front wheels178 Left Rear wheel encoder faulty179 Left Middle wheel encoder faulty180 Left Front wheel encoder faulty181 Right Rear wheel encoder faulty182 Right Middle wheel encoder faulty183 Right Front wheel encoder faulty

[Draft: 19 October 2004—19:11 ] 104

Index

Q-function, 41“optimal” proposal distribution, 59Ackerson and Fu (1970), 18, 89Astrom (1965), 41, 89Avitzour (1995), 7, 89Bar-Shalom and Fortmann (1988), 6, 89Bar-Shalom (1988), 7, 89Bares and Wettergreen (1999), 2, 89Baydar (2002), 84, 89Bererton and Khosla (2002), 4, 34, 89Blom and Bar-Shalom (1988), 19, 89Blom (1984), 19, 89Boyen and Koller (1998), 20, 27, 89Boyen and Koller (1999), 27, 89Brown et al. (1982), 32, 90Carlson and Murphy (2003), 1, 84, 90Carlson (2004), 33, 90Carpenter et al. (1999), 22, 25, 29, 90Chang and Athanas (1978), 18, 90Chittaro and Ranon (2004), 35, 90Crisan and Lyons (1997), 22, 90Dague et al. (1990), 33, 90Dasarthy (1990), 32, 90Deans (2002), 15, 90Dearden and Clancy (2002), 32, 38, 90Djath et al. (2000), 34, 90Doucet and Crisan (2002), 7, 91Doucet et al. (1999), 18, 22, 91Doucet et al. (2000), 29, 91Doucet et al. (2001), 5, 6, 20, 91Doucet (1998), 22, 59, 90Dvorak and Kuipers (1989), 33, 91Elsharkawy and Guedouar (Elsharkawy and Gue-

douar), 69, 91Fernandez (2000), 35, 91Fox et al. (1999), 5, 30, 91Funiak and Williams (2003), 31, 91Ghallab (1985), 32, 91Glass et al. (1991), 33, 91Goel et al. (2000), 31, 91Gordon et al. (1993a), 6, 7, 23, 91Gordon et al. (1993b), 22, 91Guckenbiehl and G (1990), 33, 91H. Olsson and Lischinsky (1999), 69, 91Hamilton, Lane, Taylor and Brown (Hamilton et al.),

34, 92Hamscher (1991), 33, 92Harmon et al. (1988), 32, 92Hofbaur and Williams (2002), 31, 92Holtzblatt et al. (1991), 33, 92

Hutter and Dearden (2003), 31, 92Isard and Blake (1998), 5, 6, 92Isermann and Balle (1997), 2, 92Jordan. (2003), 9, 92Julier and Uhlmann (1996), 10, 16, 31, 58, 59, 92Julier, and J. K. Uhlmann (2002), 16, 92Kaelbling et al. (1998), 2, 63, 92Kalman (1960), 13, 92Kanazawa et al. (1995), 6, 23, 92Kitagawa (1993), 7, 22, 92Koller and Fratkina (1998), 29, 92Kong et al. (1994), 22, 93Koppolu and Chatterjee (2001), 36, 93Kurien and Nayak (2000), 33, 93Lamine and Kabanza (Lamine and Kabanza), 34, 93Leger (2000), 42, 53, 63, 93Leger (Leger), 66, 93Lerner et al. (2000), 18, 32, 93Lerner et al. (2002), 20, 93Leuschen, Walker and Cavallaro (Leuschen et al.),

35, 93Liu and Chen (1998), 22, 93Liu (2000), 18, 93Long, Murphy and Parker (Long et al.), 35, 93McIlraith (1998), 32, 93McKay (2003), 20, 93McLauchlan (1999), 29, 93Metropolis and Ulam (1949), 6, 23, 93Minka (2001), 29, 93Morales-Mendendez et al. (2002), 31, 93Murphy and Hershberger (1996), 34, 94Murphy and Hershberger (2000), 35, 93NASA (2000), 3, 94Ng et al. (2002), 27, 94Parker (1998), 34, 94Pitt and Shephard (1999), 26, 94R. B. Washburn (1985), 7, 94Rauch et al. (1965), 29, 94Roumeliotis et al. (1998), 30, 94Rubinstein (1981), 7, 23, 94Rubin (1987), 38, 94Rubin (1988), 7, 23, 94Scheding (2000), 33, 34, 94Shortliffe (1976), 32, 94Silberman (1999), 33, 94Simmons (1994), 34, 94Soika (1997), 34, 94Sondik (1971), 2, 41, 63, 94Stuck (1992), 34, 94Stuck (1995), 34, 95

105

INDEX

Szolovits and Pauker (1978), 32, 95Szolovits and Pauker (1993), 32, 95Tanner (1996), 95Thrun et al. (2000), 27, 95Thrun et al. (2001), 30, 37, 95Tugnait (1982), 18, 95Verma et al. (2001), 37, 95Verma et al. (2003), 47, 95Verma (2001), 33, 95Visinsky et al. (1995), 35, 95Washington (2000), 31, 95Wettergreen et al. (2002), 42, 95, 97Willeke and Dearden (Willeke and Dearden), 84, 95Williams and Nayak (1996), 33, 86, 95Yan, Ota, Nakamura, Arai and Kuwahara (Yan et al.),

34, 96daily report (2004), 69, 90de Freitas (2002), 31, 90de Kleer and Williams (1987), 32, 90de Kleer and Williams (1989), 33, 90van der Merwe et al. (2000), 26, 58, 95

Abstract Markov model, 49abstract states, 48Assumed Density Smoothing, 29Auxiliary Particle Filter, 26

Bayesian Filtering, 13

Classical Particle Filter, 23cost function, 38

Dante II, 3Darwin2K, 67Decision Theoretic Particle Filter, 37decision-theoretic Monte Carlo smoothing, 62DTMS, 62DTPF, 37

Effective Sample Size, 24EKF, 14Expectation Propagation, 29expected cost, 40Extended Kalman Filter, 14

Factored Particle Filter, 27failure, 2Fault detection, 2Fault monitoring, 2Filtering, 13fixed-lag smoothing, 28

Gaussian Quadrature, 15Generalized Pseudo-Bayesian, 18GPB, 18

Hyperion robot, 42

IMM, 19Importance Sampling, 21importance weights, 21Interacting Multiple Model, 19

jump-linear system, 17jump-Markov models, 17

Kalman Filter, 13Kalman Smoothing, 28

LDS, 13linear dynamical systems, 13

Mars Polar Lander, 3measurement update step, 14Mixture of Gaussian Belief, 17Mixture Proposal Distribution, 27monitoring, 13Monte Carlo Methods, 20

Numerical Integration Methods, 15

Parametric Methods, 13prior biasing, 38prior boosting, 38process model, 14proposal distribution, 21Pruning Methods, 17

Quadrature, 15

Rao-Blackwellized Particle Filter, 18Resampling, 22risk function, 39risk-function, 41Risk-sensitive sampling, 38

scaled unscented transform, 16Smoothing, 28switching Kalman filters, 17Switching Linear Dynamic systems, 17switching linear systems, 17switching state-space models, 17systematic resampling, 22

tracking, 13trajectory depletion, 23

UKF, 17Unscented Kalman Filter, 17Unscented Particle Filter, 26Unscented Transform, 16UT, 16

value iteration, 41value-function, 41variable resolution particle filter, 47variable resolution state space model, 48VRPF, 47VUPF, 60

[Draft: 19 October 2004—19:11 ] 106

Document Log:

Manuscript Version 0 — Jan 11, 2004

Typeset by AMS-LATEX — 19 October 2004

VANDI VERMA

ROBOTICS INSTITUTE, CARNEGIE MELLON UNIVERSITY, 5000 FORBES AVE., PITTSBURGH, PA 15213,USA, Tel. : (412) 268-1858

E-mail address: vandi@@ri.cmu.edu

Typeset by AMS-LATEX

Documents

TRACTABLE PARTICLE FILTERS FOR ROBOT FAULT DIAGNOSIS · Vandi Verma Robotics Institute Carnegie Mellon University Pittsburgh, PA 15213 Oct 25, 2004 Submitted in partial fullment of