Upload
others
View
43
Download
0
Embed Size (px)
Citation preview
16.412 / 6.834 Lecture, 15 March 2004 1
Hybrid Mode Estimation andGaussian Filtering with Hybrid HMMs
Stanislav Funiak
16.412 / 6.834 Lecture, 15 March 2004 2Hybrid Mode Estimation and Gaussian Filtering with Hybrid HMM Models
References
Hofbaur, M. W., and Williams, B. C. (2002). Mode estimation of probabilistic hybrid systems. In: Hybrid Systems: Computation and Control, HSCC 2002.Funiak, S., and Williams, B. C. (2003). Multi-modal particle filtering for hybrid systems with autonomous mode transitions. In: DX-2003, SafeProcess 2003.Lerner, U., R. Parr, D. Koller and G. Biswas (2000). Bayesian fault detection and diagnosis in dynamic systems. In: Proc. of the 17th
National Conference on A. I.. pp. 531-537.V. Pavlovic. J. Rehg, T.-J. Cham, and K. Murphy. A Dynamic Bayesian Network Approach to Figure Tracking Using Learned Dynamic Models. In: Proc. ICCV, 1999.H.A.P. Blom and Y. Bar-Shalom. The interacting multiple model algorithm for systems with Markovian switching coefficients. IEEE Transactions on Automatic Control, 33, 1988.
16.412 / 6.834 Lecture, 15 March 2004 3Hybrid Mode Estimation and Gaussian Filtering with Hybrid HMM Models
Hybrid Models
Hidden Markov model
on
failed
off
p(x0)p(xt | xt-1)p(zt | xt)
Dynamic systems
?
Applications:- target tracking- localization and mapping- …
Applications:- topological localization
16.412 / 6.834 Lecture, 15 March 2004 4Hybrid Mode Estimation and Gaussian Filtering with Hybrid HMM Models
Outline
Applications: fault diagnosis, visual trackingSwitching linear Gaussian models + exact filteringProbabilistic Hybrid Automata + filteringApproximate Gaussian filtering with hybrid HMM models
fx(t1)|z(t1)(x|z1)
fx(t2)|z(t2)(x|z2)
fx(t2)|z(t1),z(t2)(x|z1,z2)
XZ1
x1
x2
Z1 Z2 Z1 Z2 X
σ
σ
σ
µ
Process model isxt = Axt-1 + But-1 + qt-1
Measurement model iszt = Hxt-1 + rt
Image adapted from Maybeck.
16.412 / 6.834 Lecture, 15 March 2004 5Hybrid Mode Estimation and Gaussian Filtering with Hybrid HMM Models
Scenario 1: Wheel monitoring for planetary rovers
Discrete variable: wheel failed (if any)Continuous variables: linear and angular velocity
Normal trajectory and trajectorieswith fault at each wheel
Courtesy NASA JPL Courtesy of Vandi Verma. Used with permission.
16.412 / 6.834 Lecture, 15 March 2004 6Hybrid Mode Estimation and Gaussian Filtering with Hybrid HMM Models
Scenario 2: Diagnosing subtle faults [H&W 2002]
Discrete variables: operational modeContinuous variables: CO2 flow, CO2 & O2 conc.
{closed, open, stuck-closed, stuck-open}
Courtesy NASA JSC
16.412 / 6.834 Lecture, 15 March 2004 7Hybrid Mode Estimation and Gaussian Filtering with Hybrid HMM Models
Scenario 3: Visual pose tracking Discrete variables: type of movementContinuous variables: head, legs, and torso position
Courtesy Pavlovic. J. Rehg, T.-J. Cham, and K. Murphy
16.412 / 6.834 Lecture, 15 March 2004 8Hybrid Mode Estimation and Gaussian Filtering with Hybrid HMM Models
Scenarios 1-3: Common properties
1. Continuous dynamics2. Finite set of behaviors, determines dynamics
Continuous state hiddenNoisy observationsUncertainty in the model
System may switchbetween behaviors
Need continuous statistical estimation
Need to track discrete changes
Need both
16.412 / 6.834 Lecture, 15 March 2004 9Hybrid Mode Estimation and Gaussian Filtering with Hybrid HMM Models
Outline
Applications: fault diagnosis, visual trackingSwitching linear Gaussian models + exact filteringProbabilistic Hybrid Automata + filteringApproximate Gaussian filtering with hybrid HMM models
16.412 / 6.834 Lecture, 15 March 2004 10Hybrid Mode Estimation and Gaussian Filtering with Hybrid HMM Models
Hybrid models – Desired properties
State evolution:Stochastic continuous evolution (uncertain model)Gaussian noise (for KF)Probabilistic discrete transitionsContinuous observations, discrete and continuous actions
Interaction of discrete and continuous state:Discrete state affects continuous evolutionContinuous state affects discrete evolution
Large systems
1 2
uc1
ud1
ud2
wc1
3
yc2
yc1
vs1 vs3
vo1
vo2
A A
CA
A
vs2
16.412 / 6.834 Lecture, 15 March 2004 11Hybrid Mode Estimation and Gaussian Filtering with Hybrid HMM Models
Graphical models revisited
a1
b1
Z1
x1
b2
Z2
x2
),|( iij xaxp
Actions
Beliefs
Observations
States
ObservableHidden
)|( ii xzp
Model:
Transition distribution
Observation distribution)|( ii xzp
),|( iij xaxp
16.412 / 6.834 Lecture, 15 March 2004 12Hybrid Mode Estimation and Gaussian Filtering with Hybrid HMM Models
Review: Hidden Markov models
Discrete states, actions, and observationsTransition & observation p. written as tables
a1
b1
Z1
x1
b2
Z2
x2
),|( iij xaxp
Actions
Beliefs
Observations
States
Observable
Hidden
117
5
9
77MassAve
Belief update:
16.412 / 6.834 Lecture, 15 March 2004 13Hybrid Mode Estimation and Gaussian Filtering with Hybrid HMM Models
Review: Linear models, Kalman Filter
Continuous states, actions, and observationsLinear (linearized) process and measurement model
a1
b1
Z1
x1
b2
Z2
x2
),|( iij xaxp
Actions
Beliefs
Observations
States
Observable
Hidden
111 tttt qBuAxx
ttt rHxz
Belief update:
16.412 / 6.834 Lecture, 15 March 2004 14Hybrid Mode Estimation and Gaussian Filtering with Hybrid HMM Models
Switching Linear Systems (SLDS)
Discrete and continuous stateAlso known as jump Markov linear Gaussian model
a1
b1
Z1
x1
b2
Z2
x2
),|( 1 tt xaxp
Actions
Beliefs
Observations
Continuousstates
ObservableHidden
)|( tt xzp
d1 d2Discretestates
Discrete states (modes):
00
1
)Pr(
)|Pr(
d
dd tt
Continuous state:
)(
)(
)()(
000
11
111
dvx
rHxy
dq
udBxdAx
ttt
tt
ttttt
)|( 1 tt ddp
16.412 / 6.834 Lecture, 15 March 2004 15Hybrid Mode Estimation and Gaussian Filtering with Hybrid HMM Models
Example: Acrobatic robot tracking
ok
failed
0.005
0.995
1.0
µ
µ
T
2121 ,,,
noisetf
noisetf
noiset
noiset
kkkkyesk
kkkkyesk
kkk
kkk
),,,,(
),,,,(
,2,2,1,1,21,2
,2,2,1,1,11,1
,2,21,2
,1,11,1
noisetf
noisetf
noiset
noiset
kkkknok
kkkknok
kkk
kkk
),,,,(
),,,,(
,2,2,1,1,21,2
,2,2,1,1,11,1
,2,21,2
,1,11,1
)(okA
)( failedA
)(okB
)( failedB
)(okq
)( failedq
)|Pr( 1 tt dd
)01.0,0(,
0
0
1
0
NrH
16.412 / 6.834 Lecture, 15 March 2004 16Hybrid Mode Estimation and Gaussian Filtering with Hybrid HMM Models
Example cont’d
The actuator works until t=20, then breaks
ok ok ok ok
failed
)(okA)(okB)(okq
)( failedA)( failedB)( failedq
)(okA)(okB)(okq
)(okA)(okB)(okq
)(okA)(okB)(okq
t=20t=19t=18t=17
…
t=21
16.412 / 6.834 Lecture, 15 March 2004 17Hybrid Mode Estimation and Gaussian Filtering with Hybrid HMM Models
What questions?
Mode estimation:Given a1z1 … atzt, estimate dt
Application: fault diagnosis
Continuous state estimationGiven a1z1 … atzt, estimate xt
Application: tracking
Hybrid state estimationGiven a1z1 … atzt, estimate xt, dt
16.412 / 6.834 Lecture, 15 March 2004 18Hybrid Mode Estimation and Gaussian Filtering with Hybrid HMM Models
Exact filtering for SLDS
Main idea:Do estimation for each sequence of mode assignmentsSum up the assignments that end in the same modeEach Gaussian has an associated weight (probability)
ok
ok
failed
ok
failed
ok
failed
Maths:
)...|...,(
)...|,(
11...
1
11
11
ttdd
tt
tttt
zazaddxp
zazadxp
t
)...|...(
)...,...|(
)...|...,(
111
111
111
ttt
tttt
tttt
zazaddp
zazaddxp
zazaddxp
Continuous tracking
Probability of a mode sequence
16.412 / 6.834 Lecture, 15 March 2004 19Hybrid Mode Estimation and Gaussian Filtering with Hybrid HMM Models
Continuous tracking
Back to our example:
ok ok ok ok
failed)(okA)(okB)(okq
)( failedA)( failedB)( failedq
)(okA)(okB)(okq
)(okA)(okB)(okq
)(okA)(okB)(okq
t=19t=18t=17
…
Know:1. Model A,B each each t2. Observations3. Actions
Can do Kalman filteringas before
)()( ,ˆ it
it Cx
Sequence (i)
Kalman filter: ),ˆ()...,...|( )()(111
it
ittttt CxNzazaddxp
)(ˆ itx
)(itC
)(1ˆ i
tx
t=20
16.412 / 6.834 Lecture, 15 March 2004 20Hybrid Mode Estimation and Gaussian Filtering with Hybrid HMM Models
Probability of a mode sequence (1/2)
Challenge: Can we update efficiently?)...|...( 111 ttt zazaddp
a1
b1
Z1
x1
b2
Z2
x2
),|( 1 tt xaxp
Actions
Beliefs
Observations
Continuousstates
ObservableHidde
n
)|( tt xzp
d1 d2Discretestates )|( 1 tt ddp
ok ok ok ok
failed
)...|......( 111111 ttt zazaokokddp
)...|......( 111 ttt zazafailedokokddp
1. Prediction:
)...,...|(
)...|...(
)...|...(
111111
111111
11111
tttt
ttt
ttt
zazadddp
zazaddp
zazaddp
)|()...|...( 1111111 ttttt ddpzazaddp
conditionalprobability
independence
discrete transition probability
16.412 / 6.834 Lecture, 15 March 2004 21Hybrid Mode Estimation and Gaussian Filtering with Hybrid HMM Models
Probability of a mode sequence (2/2)
Challenge: Can we incorporate the latest observation?
ok ok ok ok
failed
)...|......( 111 ttt zazafailedokokddp2. Update:Bayes rule + Kalman Filter
)...|......( 11111 ttt zazafailedokokddp
)...|...( 11111 ttttt zazazaddp
Observation likelihoodgiven the mode sequence Prediction
),ˆ()...,...|( )()(11111
iittttt CxNzazaddxp1.
Sequence (i)
2. ),ˆ()...,...|( )()(11111 RHHCxHNzazaddzp Tii
ttttt
S
TrrSeS
15.02/1|2|
1
constzazaddpzazaddzp ttttttt )...|...()...,...|( 1111111111
16.412 / 6.834 Lecture, 15 March 2004 22Hybrid Mode Estimation and Gaussian Filtering with Hybrid HMM Models
Outline
Applications: fault diagnosis, visual trackingSwitching linear Gaussian models + exact filteringProbabilistic Hybrid Automata + filteringApproximate Gaussian filtering with hybrid HMM models
16.412 / 6.834 Lecture, 15 March 2004 23Hybrid Mode Estimation and Gaussian Filtering with Hybrid HMM Models
Probabilistic Hybrid AutomataSLDS + nonlinear dynamics, guarded transitionsHofbaur & Williams 2002
has-ball
true
false
7.01
0.1
0.9
7.01
0.5
7.010.5
7.01
1.0
0.5
0.5
2121 ,,,
Continuous dynamics:
noisetf
noisetf
noiset
noiset
kkkkyesk
kkkkyesk
kkk
kkk
),,,,(
),,,,(
,2,2,1,1,21,2
,2,2,1,1,11,1
,2,21,2
,1,11,1
noisetf
noisetf
noiset
noiset
kkkknok
kkkknok
kkk
kkk
),,,,(
),,,,(
,2,2,1,1,21,2
,2,2,1,1,11,1
,2,21,2
,1,11,1
µ
µ
T
ball:
a1
b1
Z1
x1
b2
Z2
x2
),|( 1 tt xaxp
Actions
Beliefs
Observations
Continuousstates
ObservableHidd
en
)|( tt xzp
d1 d2Discretestates
Discrete transition now dependson the continuous state
),|( 1 ttt xddp (also, the observation function g depends on discrete state)
16.412 / 6.834 Lecture, 15 March 2004 24Hybrid Mode Estimation and Gaussian Filtering with Hybrid HMM Models
Discrete prediction in PHA
Main idea:compute transition probability from continuous estimate
constant for all xt-1 thatsatisfy a given guard
)(1
)(1,ˆ i
ti
t Cx
true true true true
false
)...,...|()...|...(
)...|...(
111111111111
11111
ttttttt
ttt
zazadddpzazaddp
zazaddp
)...|...( 111111 ttt zazaddp
)...|...( 111111 tttt zazadddp
O
def
tttttX ttt Pdxzazaddxpxddp 1111111111 )...,...|(),|(
Cannot simplify as easily as before
Instead:
= p(transition ) p(guard for satisfied | previous estimate)
16.412 / 6.834 Lecture, 15 March 2004 25Hybrid Mode Estimation and Gaussian Filtering with Hybrid HMM Models
Example
0.7 1meanof estimated 1 guard boundary
probabilityof guard c
7.01
7.01
7.01
7.01
)(1
)(1,ˆ i
ti
t Cx
P(true -> false | estimate i) = 0.1* + 0.5*
16.412 / 6.834 Lecture, 15 March 2004 26Hybrid Mode Estimation and Gaussian Filtering with Hybrid HMM Models
Concurrent Probabilistic Hybrid Automata
PHA + composition
2121 ,,,
µ
µ
T
ball:
Gaussian filtering:Merge the difference equations in component models
e.g., equation solver
Track sequences of full mode assignments as beforeCompute the transition probability component-wise
16.412 / 6.834 Lecture, 15 March 2004 27Hybrid Mode Estimation and Gaussian Filtering with Hybrid HMM Models
Putting it all together
Estimate the continuous state for each mode sequenceUpdate the posterior probability of each mode sequence1. prediction (transition expansion)2. observation
Hybrid update equationsHofbaur & Williams 2002
Po
Pt
prediction observation
)(1
)(111 ,ˆ),...( i
ti
tt CxddbOld estimate: New estimate:
)()(1 ,ˆ),...( i
ti
tt Cxddb
ottt PPddbddb )...()...( 111
16.412 / 6.834 Lecture, 15 March 2004 28Hybrid Mode Estimation and Gaussian Filtering with Hybrid HMM Models
Outline
Applications: fault diagnosis, visual trackingSwitching linear Gaussian models + exact filteringProbabilistic Hybrid Automata + filteringApproximate Gaussian filtering with hybrid HMM models
16.412 / 6.834 Lecture, 15 March 2004 29Hybrid Mode Estimation and Gaussian Filtering with Hybrid HMM Models
Gaussian filtering for hybrid HMMs
Tracks mode sequences with a bank of Kalman Filters
Problem: the number of possible mode sequences increases exponentiallyin time(also exponentially in # components)Exponential computational complexity
2 strategies:1. pruning (truncating)2. collapsing (merging)
16.412 / 6.834 Lecture, 15 March 2004 30Hybrid Mode Estimation and Gaussian Filtering with Hybrid HMM Models
Pruning: Selecting relevant mode sequences
Two methodsK-best filtering
Looking for a set of leading sequencesObtained efficiently with A* search
Particle filteringSelecting trajectories probabilistically by samplingRao-Blackwellised particle filtering
x2x2
x2
x2
x2
x2
x2
x1
x1x0
x2
x2(1)
(2)
(3) x2(3)
(1)
(2)
x1(3)
(1)
(4)
(5)
(6)
(7)
(8)
(9)
16.412 / 6.834 Lecture, 15 March 2004 31Hybrid Mode Estimation and Gaussian Filtering with Hybrid HMM Models
K-best Filtering
A* search through space of possible successors [H&W 2002]Evaluated in the order of
)()()( nhngnf
Probability (cost) oftrajectory so far
Admissible heuristics:upperbound on probabilityfrom nv onward
16.412 / 6.834 Lecture, 15 March 2004 32Hybrid Mode Estimation and Gaussian Filtering with Hybrid HMM Models
Rao-Blackwellised Particle Filtering
Principle: Decrease the computational complexity by reducing dimensionality of sampled space
Sample variables rClosed-form solution for variables s
r
s
r
)|( )(:0itrp s
s
component 1component 2
PT2
nv
PT1PT1 Po
h(k)h(k-1)(k-1)
transition expansion estimation
component l
xx(k)
16.412 / 6.834 Lecture, 15 March 2004 33Hybrid Mode Estimation and Gaussian Filtering with Hybrid HMM Models
RBPF for Hybrid HMMs
mok,0
(1) Initialization step
(2) Importance sampling step
(3) Selection (resampling) step
mok,0 mok,1mok,1
mok,0mok,1 mf,0 mf,1
(4) Exact (Kalman Filtering) step
draw samples from the initial distribution over the modes
initialize the corresponding continuous state estimates
evolve each sample trajectory according to the transition model and previous continuous estimates
determine transition & observ-ation model for each sample
update continuous estimates
mok,0mok,0
mok,0mok,1
mok,1mok,1
mok,1mf,1
mok,1mok,1
mok,0mok,0
mok,0mok,1
mok,1mok,1
Funiak & Williams 2003
16.412 / 6.834 Lecture, 15 March 2004 34Hybrid Mode Estimation and Gaussian Filtering with Hybrid HMM Models
Do we really need hybrid?
Alternatives:HMM, grid-based methods: Course discretizationineffective for tracking a dynamic system! Kalman Filter: Unimodal distribution too weakParticle filter: Sample size too large
16.412 / 6.834 Lecture, 15 March 2004 35Hybrid Mode Estimation and Gaussian Filtering with Hybrid HMM Models
What you should know
The form of hybrid modelsThe “exact” algorithm for computing the belief state over mode sequences + continuous estimates
And how to use it for common diagnostic tasks
The strategies for pruning mode sequences(conceptually)