Brief Paper Review

B.Balaji (2009)“Nonlinear filtering and quantum physics A Feynman path integral perspective”

Kohta IshikawaOct 30. 2011 第3回 確率の科学研究会

1

Outline• Continuous Filtering• DMZ equation (Solution of Continuous Filtering)• Yau equation (approximating DMZ eq)• Path Integrals and Stochastic Processes• Path Integral Representation of Continuous Filtering

• Continuous Filtering and Quantum Physics

2

Continuous Filtering• Discrete Filtering

• Continuous-Discrete Filtering

• Continuous-Continuous Filtering

System ModelObservation Model

dx(t) = f(x(t), t)dt+ e(x(t), t)dw(t) System Model(SDE)y(tk) = h(x(tk), tk) + wk

xk = f(xk�1, tk�1)xk�1 + e(xk�1, tk�1)

yk = h(xk, tk) + wk

Observation Model

dx(t) = f(x(t), t)dt+ e(x(t), t)dv(t)

dy(t) = h(x(t), t)dt+ dw(t)

System Model(SDE)

Observation Model(SDE)

3

• Optimal Estimation of Unnormalized Probability Density under Observation

Duncan-Mortensen-Zakai equation

unnormalized probability density

d�(t, x) = LY �(t, x)dt+mX

i=1

hi(x)�(t, x)dyi(t)

LY �(t, x) = �nX

i=1

@

@xi(fi(x)�(t, x)) +

1

2

nX

i=1

@

2�(t, x)

@x

2i

� 1

2

mX

i=1

h

2i (x)�(t, x)

�(0, x) = �0(x)

• Probability density is given as a conditional expectation under observation(4)

• Then, term is difficult to handledy(t)

4

Robust DMZ equation• Eliminate term dy(t)

u(t, x) = exp

�

mX

i=1

hi(x)yi(t)

!�(t, x)

@u(t, x)

@t

=1

2

nX

i=1

@

2u(t, x)

@x

2i

+nX

i=1

0

@�fi(x) +mX

j=1

yj(t)@hj(x)

@xi

1

A @u(t, x)

@xi

�✓ nX

i=1

@fi(x)

@xi+

1

2

mX

i=1

h

2i (x)�

1

2

mX

i=1

yi(t)�hi(x)

+mX

i=1

nX

j=1

yi(t)fj(x)@hi(x)

@xj� 1

2

mX

i,j=1

nX

k=1

yi(t)yj(t)@hi(x)

@xk

@hj(x)

@xk

◆u(t, x)

Then, DMZ equation translates to

u(0, x) = �0(x)

5

Yau equation• Approximated DMZ equation• Anyway, we can only handle observation term discretely :P

• Consider DMZ equation valid between each observation interval

• Replace y(t) as followsy(t) !

⇢y(⌧l)y(⌧l�1)

post-measurement formpre-measurement form

6

Yau equation• In the interval @ul(t, x)

@t

=1

2

nX

i=1

@

2ul(t, x)

@x

2i

+nX

i=1

0

@�fi(x) +mX

j=1

yj(⌧l)@hj(x)

@xi

1

A @ul(t, x)

@xi

�✓ nX

i=1

@fi(x)

@xi+

1

2

mX

i=1

h

2i (x)�

1

2

mX

i=1

yi(⌧l)�hi(x)

+mX

i=1

nX

j=1

yi(⌧l)fj(x)@hi(x)

@xj� 1

2

mX

i,j=1

nX

k=1

yi(⌧l)yj(⌧l)@hi(x)

@xk

@hj(x)

@xk

◆ul(t, x)

ul(⌧l, x) = ul�1(⌧l�1, x)

solution of previous step

⌧l�1 t ⌧l

• Observation term is constant in the equation• This approximates DMZ equation well

observation is available at times {⌧0, ⌧1, · · · }

7

Yau equation• Eliminate observation term from coefficientsul(t, x) = exp

mX

i=1

yi(⌧l)hi(x)

!ul(t, x)

@ul(t, x)

@t

=1

2

nX

i=1

@

2ul(t, x)

@x

2i

�nX

i=1

fi(x)@ul(t, x)

@xi

�

nX

i=1

@fi(x)

@xi+

1

2

mX

i=1

h

2i (x)

!ul(t, x)

ul(⌧l�1, x) = exp

mX

i=1

yi(⌧l)hi(x)

!ul�1(⌧l�1, x)

observation term onlycontributes to initial condition

• previous observation is also usableyi(⌧l�1)

it approximatesdirectly

�(t, x)

8

Yau algorithm1. Solve Yau eq along with initial condition at latest observation

2. Derive a probability density at next time step

3. Earn a observation at the time

4. Repeat from 1. with a time advancing

This is one of the few efficient algorithms for continuous nonlinear filtering

9

Path Integrals and Stochastic Processes• Stochastic Process, Path Measure and Fokker-Planck equation

Fokker-Planck equation Path Integral representation

Stochastic Process

path measure,functional differentiation

probability density of state

green’s function

10

Path Integral Representation of Continuous Filtering• Formal Solution directly wrote down from SDE

xi(t) = fi(x(t)) + vi(t)

yi(t) = hi(x(t)) + wi(t)

SDE for continuous filtering

Probability densityfunctional jacobian

functional measureP (t, x, y|t0, x0.y0) =

Z[d⇢(v(t))][d⇢(w(t))]

⇥ [dx(t)]�(x(t)� f(x(t))� v(t))

�x(t)�

n(x(t)� f(x(t))� v(t))

⇥ [dy(t)]�(y(t)� h(x(t))� w(t))

�y(t)�

m(y(t)� h(y(t))� w(t))

⇥ �

n(x(t)� x)|x(t0)=x0

�

m(y(t)� y)|y(t0) = y0

11

Path Integral Representation• Probability density with Gaussian noise measure

[d⇢(v(t))] = [Dv(t)] exp

� 1

2~v

nX

i=1

Z t

t0

vi(t)2dt

!

[d⇢(w(t))] = [Dw(t)] exp

� 1

2~w

mX

i=1

Z t

t0

vi(t)2dt

!

P (t, x, y|t0, x0.y0) =

Zy(t)=y

y(t0)=y0

Zx(t)=x

x(t0)=x0

[Dv(t)][Dw(t)][Dx(t)][Dy(t)]

⇥ exp

� 1

2~v

nX

i=1

Zt

t0

v

2i

(t)dt�nX

i=1

Zt

t0

@f

i

(x(t))

@x

i

dt� 1

2~w

mX

i=1

Zt

t0

w

2i

dt

!

⇥�

n

(x(t)� f(x(t))� v(t))�

m

(y(t)� h(y(t))� w(t))

=

Zy(t)=y

y(t0)=y0

Zx(t)=x

x(t0)=x0

[Dx(t)][Dy(t)] exp (�S)

S =1

2

Z t

t0

dt

"1

~v

nX

i=1

(xi(t)� fi(x(t)))2 +

nX

i=1

@fi(x(t))

@xi+

1

~w

mX

i=1

(yi � hi(x(t)))2

#

~v, ~wvariance of noise(assumed diagonal)

:

jacobian term

12

Path Integral Representation• Approximation with discrete measurements

measurement term in the action S

state independent(absorbed in the measure)

� 1

2~w

Z ti

ti�1

dt

mX

i=1

(yi � hi(x(t))) = � 1

2~w

Z ti

ti�1

dt

mX

i=1

⇥y

2i (t) + h

2i (x(t))� 2hi(x(t))yi(t)

⇤

measurementindependent

relevant term1

~w

Z ti

ti�1

dt

mX

j=1

hj(x(t))yj(t) ⇠⇢ 1

~w

Pmj=1 hj([x(ti) + x(ti�1)]/2)[yj(tj)� yj(tj�1)]

1~w

Pmj=1 hj([x(ti) + x(ti�1)]/2)[yj(tj�1)� yj(tj�2)]

13

Path Integral Representation• Approximation with discrete measurements

leads to approximated probability density

P (ti, xi, yi|ti�1, xi�1, yi�1) ⇠ ˜

P (ti, xi|ti�1, xi�1)

⇥⇢

exp

⇣1~w

Pmj=1 hj([x(ti) + x(ti�1)]/2)[yj(tj)� yj(tj�1)]

⌘

exp

⇣1~w

Pmj=1 hj([x(ti) + x(ti�1)]/2)[yj(tj�1)� yj(tj�2)]

⌘

˜

P (t

i

, x

i

|ti�1, xi�1) =

Zx(ti)=xi

x(ti�1)=xi�1

[Dx(t)] exp(�S(t

i�1, ti))

S(ti�1, ti) =1

2

Z ti

ti�1

dt

"1

~v

nX

i=1

(xi(t)� fi(x(t)))2 +

nX

i=1

@fi(x(t))

@xi+

1

~w

mX

i=1

h

2i (x(t))

#

satisfies Yau equation with δfunction initial condition (fundamental solution)P (t, x|ti�1, xi�1)

14

Continuous Filtering and Quantum Physics• Action term is interpreted as integration of Lagrangian

˜

P (t, x|t0, x0) =

Zx(t)=x

x(t0)=x0

[Dx(t)] exp

✓� 1

~v

S

◆

L = T � V

S =1

2

Zt

t0

dt

"nX

i=1

[x2i

(t) + f

2i

(x)� 2xi

(t)fi

(x(t))] + ~v

nX

i=1

@f

i

(x(t))

@x

i

+~v

~w

mX

i=1

h

2i

(x(t))

#

=1

2

Zt

t0

dt

"nX

i=1

[x2i

(t) + f

2i

(x)] + ~v

nX

i=1

@f

i

(x(t))

@x

i

+~v

~w

mX

i=1

h

2i

(x(t))

#

�nX

i=1

Zx(t)

x(t0)dx

i

(t)fi

(x(t))

⌘ 1

2

Zt

t0

dtL�nX

i=1

Zx(t)

x(t0)dx

i

(t)fi

(x(t))T =

1

2

Z t

t0

dt

nX

i=1

x

2i (t)

�V =1

2

Z t

t0

dt

"nX

i=1

f

2i (x) + ~v

@fi(x(t))

@xi

�+

~v~w

mX

i=1

h

2i (x(t))

#

15

General Advantages of Path Integral Representation• Straightforward to develop numerical algorithms• A lot of (quantum mechanical) approximation methods are applicable

• classical approximation (MAP estimation in Bayesian context)

• perturbation methods

16

Summary• Derive Yau equation for nonlinear continuous filtering

• Develop the Path Integral representation of the solution

17

Another Example of Path Integral Methods• Stochastic Optimal Control (3)• Hamilton-Jacobi-Bellman(HJB) equation

dx = (b(x, t) +Bu)dt+ dv

C(xint

, t

int

, u) =

⌧�(x(t

f

)) +

Ztf

tint

dt

✓1

2u(t)TRu(t) + V (x(t), t)

◆�

xint

J(x, t) = minu(t!tf )

C(x, t, u(t ! tf ))

Dynamics (with control)

Cost Function

Solution of the problem is optimal control function which minimizes the cost

end cost control potential

(Optimal control function is derived from minimized cost function)

18

Optimal Control• HJB equation and path integral representation

�@J

@t

= �1

2

@J

@x

�TBR

�1B

T @J

@x

+ V + b

T @J

@x

+1

2Tr

⌫

@

2J

@x

2

�

J(x, t) = �� log

Z

x(t)=x

[Dx(t)] exp

✓�S

�

◆

S = �(x(tf )) +

Z tf

td⌧

✓1

2(x(⌧)� b(x(⌧), ⌧))TR(x(⌧)� b(x(⌧), ⌧)) + V (x(⌧), ⌧)

◆

V (x(t), t) could be non-quadratic(difficult to solve with traditional methods)

Solution of the equation could be wrote as path integral under some conditions

19

References(1) B. Balaji, “Universal Nonlinear Filtering Using Feynman Path Integrals Ⅱ: The Continuous-Continuous Model with Additive Noise” PMC Physics A (2009) 3:2

(2) Shing-Tung Yau and Stephen S.-T. Yau, “Real Time Solution of Nonlinear Filtering Problem Without Memory Ⅰ” Mathematical Research Letters 7, 671-693 (2000)

(3) H. J. Kappen “Path Integrals and Symmetry Breaking for Optimal Control Theory” Journal of Statistical Mechanics (2005) P11011

(4) A. H. Jazwinski “Stochastic Processes and Filtering Theory”

20