LEARNING BY SIMPLIFIED COST-REFERENCE PARTICLE FILTERING

USING BIASED DATA

Monica F. Bugallo, Ting Lu, and Petar M. Djuric,

Department of Electrical and Computer Engineering

Stony Brook University, Stony Brook, NY 11794, USA

e-mail: {monica,tinglu,djuric}@ece.sunysb.edu

ABSTRACT

In this paper we address the problem of online learning by

cost-reference particle filtering combined with Kalman fil-

tering. We propose an efficient learning scheme applicable

to problems where some of the unknowns of a dynamic sys-

tem of interest are linear given the remaining unknowns,

which are nonlinear. To that end, we exploit a concept

that is analogous to Rao-Blackwellization, and we imple-

ment it by using only one Kalman filter. The resulting al-

gorithm is tested and compared to standard particle filtering

for the problem of target tracking using bearings-only mea-

surements acquired by two sensors.

1. INTRODUCTION

Many methods for learning belong to the class of sequential

methods, where learning is implemented recursively. The

latest measurements containing information about the un-

knowns of interest modify the existing knowledge about the

unknowns following a certain theory. One such method-

ology, known as particle filtering (PF), was introduced to

the engineering community in the early nineties [1]. Since

then it has advanced considerably so that today it is mostly

well understood with all its advantages and disadvantages.

PF is primarily used for learning in scenarios where the un-

knowns are nonlinear and evolve with time and where the

noises used in the models may be non-Gaussian.

PF is based on Bayesian theory and therefore uses as-

sumptions about the probability distributions of the noises

in the modeling equations [2]. The objective of PF is very

ambitious and it amounts to tracking the posterior distri-

butions of the unknowns since they capture the complete

knowledge about the unknowns. PF is based on approximat-

ing these distributions by discrete random measures, which

are composed of samples (particles) and weights associated

to the particles. More recently, a PF method that is not based

This work has been supported by the National Science Foundation

under CCF-0515246 and the Office of Naval Research under Award

N00014-06-1-0012.

on probabilistic assumptions was proposed in [3]. In other

words, the probability distributions of the noises in the state

and observation equations are assume unknown and no at-

tempts are made to estimate them. It is also based on parti-

cles but instead of weights, they have costs that measure the

“quality” of the weights according to an adopted criterion.

The original name of the methodology is cost-reference par-

ticle filtering (CRPF).

The philosophy of CRPF is practically the same as that

of PF. The complete knowledge about the unknowns is

described by a cost function defined on the space of the

unknowns. This function is approximated by a discrete

measure that is updated with every measurement. In its

original form, CRPF in spirit is close to auxiliary PF [4].

Since CRPF does not require probabilistic assumptions, it

is not surprising that it is much more robust than PF.

In many problems some of the unknowns are con-

ditionally linear on the remaining (nonlinear) unknowns.

For such problems there is a methodology known as

Rao-Blackwellization (RB) [5], which combines PF with

Kalman filtering (KF) [6]. According to the scheme, ev-

ery particle stream has its Kalman filter which takes care of

the linear unknowns. RB allows for more accurate estimates

of the unknowns because the dimension of the space that is

explored with particles is reduced and therefore it is much

better searched.

In this paper we propose to use an analogous scheme for

CRPF. That is, the nonlinear unknowns are processed by us-

ing particles and the linear ones by using KF. Moreover, we

propose to use only one Kalman filter for all the particle

streams instead of as many as there are streams1. We il-

lustrate the use of the method on a target tracking problem

where the tracking is achieved by bearings-only measure-

ments obtained by two sensors. The linear unknowns are

the biases of the measurement devices.

The paper is organized as follows. The learning prob-

lem is stated in Section 2. In Section 3 we describe the

1Here we assume that the cost function used for computing the costs is

unimodal. In cases of multimodality the number of KFs corresponds, in

general, to the number of modes.

1-4244-1566-7/07/$25.00 ©2007 IEEE. 402

new CRPF for a system with conditionally linear parame-

ters which makes use of only one Kalman filter in its im-

plementation. Simulation results regarding the problem of

bearings-only tracking are given in Section 4. Final conclu-

sions are given in Section 5.

2. PROBLEM FORMULATION

Many learning problems in signal processing can be stated

in terms of estimating an unobserved discrete-time random

state in a dynamic system of the form

xt = f(xt−1,ut) (1)

yt = g(xt) + bt + vt (2)

where, in equation (1), xt is a nonlinear unknown and

represents the system state at time t; f(·) is a known vector

function, which, in general, may be nonlinear; and ut is

a zero-mean noise vector whose probability distribution is

unknown. The measurements, yt, are functions of the

unknown state modeled using (2) where g(·) denotes a

known vector function of the state; bt = b represents a

vector of unknown biases2; and vt is another zero-mean

noise vector with an unknown probability distribution. The

objective is to obtain the posterior probability distribution

of the state, xt, given the sensor measurements, y1:t, i.e., to

acquire p(xt|y1:t) in the presence of the unknown biases b.

3. PROPOSED METHOD

We seek a solution to the learning problem based on

a probabilistic-free approach, i.e., we do not make any

assumptions about the probability distributions of the noise

vectors. We only assume that the mean of the state noise

vector is known. Moreover, for the state-space model

described by (1)-(2), we apply the RB philosophy that

reduces the variance of the nonlinear state estimate. We

can do that because the vector of biases b constitutes a

linear parameter given xt. Note, however, that to apply

such scheme by means of KF and using CRPF reasoning,

one has to circumvent properly the need for knowledge of

the covariances of the noises required by the traditional RB

approach. We include a method that estimates the necessary

covariances. Also, unlike traditional RB-PF approaches

where one Kalman filter is used per particle stream, we

use a single Kalman filter for the entire filter. We have

reported the use of one Kalman filter for RB in the context

of standard PF in [7] and [8].

Although CRPF is a probabilistic-free method, its se-

quential procedure follows a similar structure as that of

standard PF (SPF) [3]. In particular, the discrete measure

2In the paper we consider a vector of unknown constant biases. With a

slight modification of the algorithm, dynamic biases can also be treated.

in SPF is composed of particles and weights associated to

them, and in CRPF, it also contains particles, but its weights

represent costs that capture the quality of the state particles

computed by using the observations.

In CRPF, we denote the discrete measure by ζt ={x

(m)0:t , c

(m)t

}M

m=1, where x

(m)0:t represent the particle

streams and c(m)t are the costs associated to x

(m)0:t . To al-

low for online processing of the observations, the costs are

updated following [3]

c(m)t = c(x

(m)0:t |y1:t)

= λc(x(m)0:t−1|y1:t−1) + �c(x

(m)t |yt) (3)

where λ is a control factor, which prevents assignment

of excessive weight to past particles (0 ≤ λ ≤ 1), and

�c(x(m)t |yt) is an incremental cost, which measures the

“quality” of the particles by the latest measurement. Here

we use as incremental cost function

�c(x(m)t |yt) = ||yt − y

(m)t ||q

where y(m)t is an estimate of the observation based on the

particle x(m)t , and q > 0. Other forms of �c(·) can also be

considered.

In this paper the novelty is the special treatment of the

linear parameters in the system (the unknown biases). We

propose tracking them by using a Kalman filter-type algo-

rithm which can be used because the biases are condition-

ally linear parameters. In particular, the proposed method

can be implemented as follows. At time instant t and using

the new observation yt, the random measure ζt−1 is updated

using the next five steps:

1. Particle selection: Compute the risks of the particles,

i.e., the prediction of the costs of propagating the

particles from the previous step3. One possibility in

computing the risks is the use of

r(m)t = λc

(m)t−1 + ||yt − g(f(x

(m)t−1),0) − bt−1||

q.

The particles are resampled according to a probability

mass function (pmf) defined by π(k)t ∼ μ(r

(k)t ),

where μ(·) is a monotonically decreasing function.

As a result, we generate a new resampled random

measure

ζt−1 ={

x(m)0:t−1, c

(m)t−1

}M

m=1.

Note that the computational complexity of this step,

mainly due to the calculation of the pmf, can be

reduced employing other alternative selection tech-

niques which are based on sorting [9].

3This initial step mimics the idea of auxiliary particle filtering [4].

403

2. Particle propagation: We draw new particles ac-

cording to a proposal distribution

x(m)t ∼ pt(xt|x

(m)t−1)

where pt(·) is an appropriate probability density func-

tion (pdf). Then we evaluate the corresponding costs

following the expression

c(m)t = λc

(m)t−1 + �c

(m)t .

3. State estimation: This step can be carried out by

using different schemes. One possible estimate is the

mean-square error (MSE) estimate,

xt =M∑

m=1

π(m)t x

(m)t

with π(m)t ∝ μ(c

(m)t ) [3]. Another alternative is the

minimum cost particle [9].

4. Observation noise estimation: Here we estimate the

covariance matrix of the observation noise in order to

apply a blind Kalman filter that does not know the

value of that matrix. The following derivations allow

us to obtain it

zt = yt − g(xt) − bt−1

zt =t − 1

tzt−1 +

1

tzt

Czt=

t − 1

tCzt−1

+1

t(zt − zt)(zt − zt)

�

Cbt=

t − 1

tCbt−1

+1

tCbt−1

Cv,t = Czt− Cbt

.

5. Measurement update of the linear states: The

equations of the blind Kalman filter then become

Kt = Cbt−1

(Cbt−1

+ Cv,t

)−1

bt = bt−1 + Kt zt

Cbt= Cbt−1

(I − Kt

).

4. EXAMPLE

In this section we present computer simulations that show

the validity and performance of the proposed algorithm.

4.1. Bearings-only tracking problem

For illustration purposes, we considered the problem of

tracking one target using two static sensors which collected

sensor 2

target

sensor 1

y2,ty1,t

Fig. 1. A system with two static sensors and their bearings-

only measurements.

bearings-only measurements. Figure 1 depicts the geometry

of the problem, whose mathematical formulation is stated as

xt = Gxxt−1 + Guut (4)

yt = h(xt) + b + vt (5)

where t = 1, 2, · · · , represents discrete-time index.

In (4), x�

t = [x1,t x2,t x1,t x2,t]� denotes the state vec-

tor composed of the coordinates and velocity components

of the target at time t; Gx and Gu are state- and noise-

transition matrices defined by

Gx =

⎡⎢⎢⎣

1 0 Ts 00 1 0 Ts

0 0 1 00 0 0 1

⎤⎥⎥⎦ , Gu =

⎡⎢⎢⎣

T 2s /2 00 T 2

s /2Ts 00 Ts

⎤⎥⎥⎦

with Ts being the sampling interval; and ut ∈ �2 is the state

noise vector modeled according to the following mixture

Gaussian distribution:

ut ∼ .3N (0, I2) + .5N (0, .25I2) + .2N (0, .01I2).

In our experiment, the observation vector y�

t =[y1,t y2,t]

� in (5) consists of collected measurements where

the function h(·) is defined as

h(xt) =

⎡⎣arctan

(x2,t−l2,1

x1,t−l1,1

)

arctan(

x2,t−l2,2

x1,t−l1,2

)⎤⎦

where (l1,1, l2,1) and (l1,2, l2,2) denote the positions of

sensors 1 and 2, respectively; the vector b� = [b1 b2]�

is composed of unknown biases from each of the sensors;

and the observation noise, vt ∈ �2, independent from ut,

is also modeled as a mixture Gaussian distribution, but of

the form

vt ∼ .5N (0, 10−2I2) + .4N (0, 10−4I2) + .1N (0, 10−5I2) .

404

Assuming that the observations from the sensors are

sent to a fusion center, the objective is to apply the pro-

posed algorithm for estimation of the target’s position as

accurately as possible.

−1200 −1000 −800 −600 −400 −200 0 200 400−500

0

500

1000

1500

2000

2500

3000

x1 (m)

x 2 (m

)

True trajectorySPFnCRPF−1KF

Fig. 2. Trajectory of the target and the estimates obtained

by the proposed CRPF-1KF and the SPFn methods.

4.2. Algorithms

We applied the algorithm proposed in the previous section

(the algorithm is labeled as CRPF-1KF to indicate that we

used one cost-reference particle filter and one Kalman fil-

ter) for the considered bearings-only tracking problem. For

comparison and benchmarking purposes, we also imple-

mented the following algorithms:

• A CRPF that, by imitating the traditional marginal-

ized PF, used one Kalman filter per particle (labeled

as CRPF-MKF to indicate that we used M Kalman

filters)

• A CRPF that made a wrong assumption by consider-

ing that there were no biases (labeled as CRPFn)

• A standard particle filter that considered complete

knowledge of the noise probability distributions and

of the biases (labeled as SPF), and therefore it did not

have to include them in the estimation problem

• A standard particle filter (labeled as SPFn) that

assumed (a) no biases and (b) wrong noise probability

distributions given by

ut ∼ N (0, 0.25I2)

wt ∼ N (0, 10−4I2).

The implemented CRPF algorithms used Gaussian

propagation4, N (0, 4I2), and λ = 0, q = 2 and μ(x) =1/x.

0 50 100 150 200 250 3000

0.2

0.4

0.6

0.8

t (s)

bias

0 50 100 150 200 250 30010

−4

10−2

100

102

t (s) st

d

true bias sensor 1estimated bias sensor 1

0 50 100 150 200 250 3000

0.1

0.2

0.3

0.4

t

bias

0 50 100 150 200 250 30010

−4

10−2

100

102

t

std

true bias sensor 2estimated bias sensor 2

Fig. 3. Estimates and standard deviations of the biases

obtained by CRPF-1KF. Top: results corresponding to the

bias of sensor 1, b1. Bottom: results corresponding to the

bias of sensor 2, b2.

4.3. Results

We simulated a scenario where the target evolved from the

origin for T = 300s with a sampling period of Ts = 1s and

zero initial velocity. The coordinates of the sensors were

placed at (l1,1, l2,1) = (−12000, 13000) and (l1,2, l2,2) =(10000, 15000), and their biases were set to b1 = 0.606and b2 = 0.225, respectively. In the implementation of the

4Note that other propagation distributions can be used [3].

405

0 50 100 150 200 250 30010

0

101

102

103

104

105

t (s)

MS

E lo

catio

n

SPFnSPFCRPF−1K

0 50 100 150 200 250 30010

0

101

102

103

104

105

t (s)

MS

E lo

catio

n

CRPFnCRPF−MKCRPF−1K

Fig. 4. MSE in m2 for the position of the target obtained by the different methods.

various methods we used M = 500 particles and for the

methods which use KF we set b0 = 0 and Cb0 = 100I2.

Figure 2 shows the trajectories of the target and the

obtained estimates in the two-dimensional space resulting

from a single simulation of the dynamic system. To

demonstrate the robustness of the proposed method, we

also show the result obtained by the SPFn. It is clear

that the proposed algorithm remained locked to the state

trajectory while the SPFn had a poor performance. It is

important to remark that the proposed algorithm did not use

the probabilistic information about the system and still had

a very good performance.

Although the biases were considered nuisance parame-

ters for the estimation of the nonlinear state, the algorithm

can still provide their estimates (see step 5 of the previous

Section). Figure 3 illustrates the estimates of the biases and

the evolution of the standard deviation of the estimated bi-

ases, which as expected, decreased with time.

We also computed the mean-square error (MSE) of the

position of the target measured in square meters according

to the formula

MSEt =1

2

1

J

J∑j=1

[(xj

1,t − xj1,t)

2 + (xj2,t − xj

2,t)2]

where [xj1,t xj

2,t]� was the true position of the target at time

t in the j-th run, and [xj1,t xj

2,t]� was the corresponding

estimate obtained by the filter. The MSE plots were

computed by averaging J = 50 independent simulations.

Figure 4 depicts the obtained results. For clarity in the

presentation, we show a comparison of the proposed method

with other SPFs on the left and a comparison of the

proposed method with other CRPFs on the right. We clearly

see that the worst performance was achieved by the particle

filters that assumed there were no biases, i.e., SPFn and

CRPFn. The SPF, which assumed complete knowledge of

the biases, achieved the best performance and constituted

a lower bound of the estimation problem. The proposed

method showed a performance close to the bound and

very similar to the CRPF-MKF that used one Kalman filter

per particle. We reiterate that the new method had such

performance even though it did not use any probabilistic

assumption unlike the SPF. Note also that since the method

only uses one Kalman filter, its computational complexity is

considerably reduced compared to the CRPF-MKF .

5. CONCLUSIONS

We presented a new simplified cost-reference particle

filtering-based algorithm for online learning in problems

with biased measurements. The proposed method, unlike

standard statistical learning techniques, does not use any

knowledge about the probabilistic distributions of the noises

in the system. Besides, following the Rao-Blackwellization

philosophy, the new algorithm treats the biases as nuisance

parameters and marginalizes them out of the estimation

problem using only one Kalman filter. The validity of the

method was tested through computer simulations by ap-

plying it to a bearings-only tracking problem. The results

showed that the new method clearly outperforms the par-

ticle filter that does not assume biased sensors and sup-

poses wrong probabilistic information and is close to the

performance of the standard particle filter that has complete

knowledge of the biases. Furthermore, when compared to

the traditional implementation of Rao-Blackwellized cost-

reference particle filter, it performs practically the same,

406

while at the same time it requires much less computations.

6. REFERENCES

[1] N. J. Gordon, D. J. Salmond, and A. F. M. Smith,

“Novel approach to nonlinear/non-Gaussian Bayesian

state estimation,” IEE Proceedings-F, vol. 140, no. 2,

pp. 107–113, 1993.

[2] A. Doucet, N. de Freitas, and N. Gordon, Eds., Sequen-

tial Monte Carlo Methods in Practice, Springer, New

York, 2001.

[3] J. Mıguez, M. F. Bugallo, and P. M. Djuric, “A new

class of particle filters for random dynamical systems

with unknown statistics,” EURASIP Journal on Applied

Signal Processing, vol. 2004(15), pp. 2278–2294, 2004.

[4] M. Pitt and N. Shepard, “Filtering via simulation:

auxiliary particle filters,” Journal of the American

Statistical Association, vol. 94, no. 446, pp. 590–599,

June 1999.

[5] C. P. Robert and G. Casella, Monte Carlo Statistical

Methods, Springer, New York, 1999.

[6] T. Schon, F. Gustafsson, and P. Nordlund, “Marginal-

ized particle filters for mixed linear/nonlinear state-

space models,” IEEE Transactions on Signal Process-

ing, vol. 50, pp. 2279–2289, 2005.

[7] M. F. Bugallo, T. Lu, and P. M. Djuric, “Tracking

with biased measurements of signal strength sensors,”

in Proceedings of the 15th International Conference on

Digital Signal Processing.

[8] M. F. Bugallo, T. Lu, and P. M. Djuric, “Biased sensors:

Localization and data fusion,” in Proceedings of the

15th European Signal Processing Conference.

[9] M. F. Bugallo, J. Mıguez, and P. M. Djuric, “Positioning

by cost reference particle filters: study of various imple-

mentations,” in Proceedings of the 2005 International

Conference on “Computer as a tool”, EUROCON, Bel-

grade, Serbia and Montenegro, 2005.

407