Bisimulation and Metrics for Labelled Markov Processesprakash/Talks/aalborg_2012.pdf · The transitions could be indeterminate (nondeterministic). Panangaden (McGill) Bisimulation

Bisimulation and Metrics for Labelled MarkovProcesses

Prakash Panangaden1

1School of Computer ScienceMcGill University

13th June 2012, University of Aalborg

Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 1 / 46

Outline

1 Introduction


Outline

1 Introduction

2 Labelled transition systems


Outline

1 Introduction


3 Ordinary bisimulation


Outline

1 Introduction



4 Discrete probabilistic transition systems


Outline

1 Introduction




5 Labelled Markov processes


Outline

1 Introduction





6 Probabilistic bisimulation


Outline

1 Introduction






7 * Proof of the logical characterization theorem


Outline

1 Introduction







8 Metrics


Outline

1 Introduction







8 Metrics

9 Continuous-state systems


Outline

1 Introduction







8 Metrics

9 Continuous-state systems

10 Conclusions


Introduction

Summary of Results

Probabilistic bisimulation can be defined for continuousstate-space systems. [LICS97]


Introduction

Summary of Results


Logical characterization. [LICS98,Info and Comp 2002]


Introduction

Summary of Results



Metric analogue of bisimulation. [CONCUR99, TCS2004]


Introduction

Summary of Results




Approximation of LMPs. [LICS00,Info and Comp 2003, CONCUR2005]


Introduction

Summary of Results





Weak bisimulation. [LICS02,CONCUR02]


Introduction

Summary of Results






Real time. [QEST 2004, JLAP 2003,LMCS 2006]


Introduction

Summary of Results







Event Bisimulation [I and C, 2006]


Introduction

Summary of Results







Event Bisimulation [I and C, 2006]

Abstract MPs [ICALP 2009]


Introduction

Collaborators

Josée Desharnais


Introduction

Collaborators

Josée DesharnaisGheorghe Comanici


Introduction

Collaborators

Josée DesharnaisGheorghe ComaniciAlexandre Bouchard-Côté


Introduction

Collaborators

Josée DesharnaisGheorghe ComaniciAlexandre Bouchard-CôtéPhilippe Chaput


Introduction

Collaborators

Josée DesharnaisGheorghe ComaniciAlexandre Bouchard-CôtéPhilippe ChaputVincent Danos


Introduction

Collaborators

Josée DesharnaisGheorghe ComaniciAlexandre Bouchard-CôtéPhilippe ChaputVincent DanosAbbas Edalat


Introduction

Collaborators

Josée DesharnaisGheorghe ComaniciAlexandre Bouchard-CôtéPhilippe ChaputVincent DanosAbbas EdalatNorm Ferns


Introduction

Collaborators

Josée DesharnaisGheorghe ComaniciAlexandre Bouchard-CôtéPhilippe ChaputVincent DanosAbbas EdalatNorm FernsVineet Gupta


Introduction

Collaborators

Josée DesharnaisGheorghe ComaniciAlexandre Bouchard-CôtéPhilippe ChaputVincent DanosAbbas EdalatNorm FernsVineet GuptaRadha Jagadeesan


Introduction

Collaborators

Josée DesharnaisGheorghe ComaniciAlexandre Bouchard-CôtéPhilippe ChaputVincent DanosAbbas EdalatNorm FernsVineet GuptaRadha JagadeesanKim Larsen


Introduction

Collaborators

Josée DesharnaisGheorghe ComaniciAlexandre Bouchard-CôtéPhilippe ChaputVincent DanosAbbas EdalatNorm FernsVineet GuptaRadha JagadeesanKim LarsenFrancois Laviolette


Introduction

Collaborators

Josée DesharnaisGheorghe ComaniciAlexandre Bouchard-CôtéPhilippe ChaputVincent DanosAbbas EdalatNorm FernsVineet GuptaRadha JagadeesanKim LarsenFrancois LavioletteRadu Mardare


Introduction

Collaborators

Josée DesharnaisGheorghe ComaniciAlexandre Bouchard-CôtéPhilippe ChaputVincent DanosAbbas EdalatNorm FernsVineet GuptaRadha JagadeesanKim LarsenFrancois LavioletteRadu MardareGordon Plotkin


Introduction

Collaborators

Josée DesharnaisGheorghe ComaniciAlexandre Bouchard-CôtéPhilippe ChaputVincent DanosAbbas EdalatNorm FernsVineet GuptaRadha JagadeesanKim LarsenFrancois LavioletteRadu MardareGordon PlotkinDoina Precup


Labelled transition systems

The definition

A set of states S,



The definition

A set of states S,

a set of labels or actions, L or A and



The definition

A set of states S,


a transition relation ⊆ S ×A× S, usually written

→a⊆ S × S.

The transitions could be indeterminate (nondeterministic).



The definition

A set of states S,


a transition relation ⊆ S ×A× S, usually written

→a⊆ S × S.

The transitions could be indeterminate (nondeterministic).

We write s a−−→ s′ for (s, s′) ∈→a.


Ordinary bisimulation

Bisimulation

s and t are states of a labelled transition system. We say s is bisimilarto t – written s ∼ t – if

s a−−→ s′ ⇒ ∃t ′ such that t a

−−→ t ′ and s′ ∼ t ′

andt a−−→ t ′ ⇒ ∃s′ such that s a

−−→ s′ and s′ ∼ t ′.



Bisimulation relations

Define a (note the indefinite article) bisimulation relation R to bean equivalence relation on S such that





sRt means ∀a, s a−−→ s′ ⇒ ∃t ′, t a

−−→ t ′ with s′Rt ′

and vice versa.







and vice versa.

We define s ∼ t if there is some bisimulation relation R with sRt .







and vice versa.

We define s ∼ t if there is some bisimulation relation R with sRt .

This is the version that is used most often.



An example

s0

a

a

a

;;

;;;;

;

s1

b

s2

b

c

;;

;;;;

;s3

c

s4 s5

t0

a

t2b

c

88

8888

8

t4 t5P1 P2



An example

s0

a

a

a

;;

;;;;

;

s1

b

s2

b

c

;;

;;;;

;s3

c

s4 s5

t0

a

t2b

c

88

8888

8

t4 t5P1 P2

Here s0 and t0 are not bisimilar.



An example

s0

a

a

a

;;

;;;;

;

s1

b

s2

b

c

;;

;;;;

;s3

c

s4 s5

t0

a

t2b

c

88

8888

8

t4 t5P1 P2

Here s0 and t0 are not bisimilar.

However s0 and t0 can simulate each other!



How do we know that two processes are not bisimilar?

Define a logic as follows:





φ ::== T|¬φ|φ1 ∧ φ2|〈a〉φ





φ ::== T|¬φ|φ1 ∧ φ2|〈a〉φ

s |= 〈a〉φ means that s a−−→ s′ and t |= φ.





φ ::== T|¬φ|φ1 ∧ φ2|〈a〉φ


We can define a dual to 〈〉 (written []) by using negation.





φ ::== T|¬φ|φ1 ∧ φ2|〈a〉φ


We can define a dual to 〈〉 (written []) by using negation.

s |= [a]φ means that if s can do an a the resulting state mustsatisfy φ.



Examples of HM Logic

T is satisfied by any process, F is not satisfied by any process.





s |= 〈a〉T means s can do an a action.






s |= ¬〈a〉T or s |= [a]F means s cannot do an a action.






s |= ¬〈a〉T or s |= [a]F means s cannot do an a action.

s |= 〈a〉(〈b〉T ) means that s can do an a and then do a b.



The logical characterization theorem

Two processes are bisimilar if and only if they satisfy the sameformulas of HM logic.





Basic assumption: the processes are finitely-branching (otherwiseyou need infinitary conjunctions).





Basic assumption: the processes are finitely-branching (otherwiseyou need infinitary conjunctions).

To show that two processes are not bisimilar find a formula onwhich they disagree.



The role of negation

Consider the processes below:

s0

a

a

;;

;;;;

;

s1 s2

b

s3

t0

a

t2

b

t3





s0

a

a

;;

;;;;

;

s1 s2

b

s3

t0

a

t2

b

t3

s0 |= 〈a〉¬〈b〉T but t0 does not.





s0

a

a

;;

;;;;

;

s1 s2

b

s3

t0

a

t2

b

t3


s0 and t0 agree on all formulas without negation.





s0

a

a

;;

;;;;

;

s1 s2

b

s3

t0

a

t2

b

t3


s0 and t0 agree on all formulas without negation.

Note that [a] has an implicit negation.


Discrete probabilistic transition systems


Just like a labelled transition system with probabilities associatedwith the transitions.





(S,L,∀a ∈ L Ta : S × S −→ [0,1])





(S,L,∀a ∈ L Ta : S × S −→ [0,1])

The model is reactive: All probabilistic data is internal - noprobabilities associated with environment behaviour.



Bisimulation for PTS: Larsen and Skou

Consider

t0a[ 1

3 ]

a[ 2

3 ]

88

8888

8

t1 t2

b[1]

t3

s0a[ 1

3 ]

a[ 13 ]

a[ 13 ]

;;

;;;;

;

s1 s2

b[1]

s3

b[1]

s4

P1 P2




Consider

t0a[ 1

3 ]

a[ 2

3 ]

88

8888

8

t1 t2

b[1]

t3

s0a[ 1

3 ]

a[ 13 ]

a[ 13 ]

;;

;;;;

;

s1 s2

b[1]

s3

b[1]

s4

P1 P2

Should s0 and t0 be bisimilar?




Consider

t0a[ 1

3 ]

a[ 2

3 ]

88

8888

8

t1 t2

b[1]

t3

s0a[ 1

3 ]

a[ 13 ]

a[ 13 ]

;;

;;;;

;

s1 s2

b[1]

s3

b[1]

s4

P1 P2

Should s0 and t0 be bisimilar?

Yes, but we need to add the probabilities.



The Official Definition

Let S = (S,L,Ta) be a PTS. An equivalence relation R on S is abisimulation if whenever sRs′, with s, s′ ∈ S, we have that for alla ∈ A and every R-equivalence class, A, Ta(s,A) = Ta(s′,A).





The notation Ta(s,A) means “the probability of starting from s andjumping to a state in the set A.”





The notation Ta(s,A) means “the probability of starting from s andjumping to a state in the set A.”

Two states are bisimilar if there is some bisimulation relation Rrelating them.


Labelled Markov processes

What are labelled Markov processes?

Reactive systems: Larsen and Skou





Labelled Markov processes are probabilistic versions of labelledtransition systems. Labelled transition systems where the finalstate is governed by a probability distribution - no otherindeterminacy.






All probabilistic data is internal - no probabilities associated withenvironment behaviour.







We observe the interactions - not the internal states.







We observe the interactions - not the internal states.

In general, the state space of a labelled Markov process maybe a continuum.



Motivation

Model and reason about systems with continuous state spaces orcontinuous time evolution or both.

hybrid control systems; e.g. flight management systems.



Motivation



telecommunication systems with spatial variation; e.g. cell phones



Motivation




performance modelling,



Motivation





continuous time systems,



Motivation





continuous time systems,

probabilistic process algebra with recursion.



The Need for Measure Theory

Basic fact: There are subsets of R for which no sensible notion ofsize can be defined.





More precisely, there is no translation-invariant measure definedon all the subsets of the reals.





More precisely, there is no translation-invariant measure definedon all the subsets of the reals.

Actually there is if you only require finite additivity.



Stochastic Kernels

A stochastic kernel (Markov kernel) is a function h : S × Σ−→ [0,1] with (a) h(s, ·) : Σ −→ [0,1] a (sub)probability measureand (b) h(·,A) : X −→ [0,1] a measurable function.



Stochastic Kernels


Though apparantly asymmetric, these are the stochasticanalogues of binary relations



Stochastic Kernels


Though apparantly asymmetric, these are the stochasticanalogues of binary relations

and the uncountable generalization of a matrix.



Formal Definition of LMPs

An LMP is a tuple (S,Σ,L,∀α ∈ L.τα) where τα : S × Σ −→ [0,1] isa transition probability function such that



Formal Definition of LMPs

An LMP is a tuple (S,Σ,L,∀α ∈ L.τα) where τα : S × Σ −→ [0,1] isa transition probability function such that

∀s : S.λA : Σ.τα(s,A) is a subprobability measureand∀A : Σ.λs : S.τα(s,A) is a measurable function.


Probabilistic bisimulation

Larsen-Skou Bisimulation

Let S = (S, i ,Σ, τ) be a labelled Markov process. An equivalencerelation R on S is a bisimulation if whenever sRs′, with s, s′ ∈ S,we have that for all a ∈ A and every R-closed measurable setA ∈ Σ, τa(s,A) = τa(s′,A).Two states are bisimilar if they are related by a bisimulationrelation.



Larsen-Skou Bisimulation

Let S = (S, i ,Σ, τ) be a labelled Markov process. An equivalencerelation R on S is a bisimulation if whenever sRs′, with s, s′ ∈ S,we have that for all a ∈ A and every R-closed measurable setA ∈ Σ, τa(s,A) = τa(s′,A).Two states are bisimilar if they are related by a bisimulationrelation.

Can be extended to bisimulation between two different LMPs.



Logical Characterization

L ::== T|φ1 ∧ φ2|〈a〉qφ




L ::== T|φ1 ∧ φ2|〈a〉qφ

We say s |= 〈a〉qφ iff

∃A ∈ Σ.(∀s′ ∈ A.s′ |= φ) ∧ (τa(s,A) > q).




L ::== T|φ1 ∧ φ2|〈a〉qφ

We say s |= 〈a〉qφ iff

∃A ∈ Σ.(∀s′ ∈ A.s′ |= φ) ∧ (τa(s,A) > q).

Two systems are bisimilar iff they obey the same formulas of L.[DEP 1998 LICS, I and C 2002]



That cannot be right?

s0

a

a

;;

;;;;

;

s1 s2

b

s3

t0

a

t1

b

t2

Two processes that cannot be distinguished without negation.The formula that distinguishes them is 〈a〉(¬〈b〉⊤).



But it is!

s0a[p]

a[q]

;;

;;;;

;

s1 s2

b

s3

t0

a[r ]

t1

b

t2

We add probabilities to the transitions.



But it is!

s0a[p]

a[q]

;;

;;;;

;

s1 s2

b

s3

t0

a[r ]

t1

b

t2


If p + q < r or p + q > r we can easily distinguish them.



But it is!

s0a[p]

a[q]

;;

;;;;

;

s1 s2

b

s3

t0

a[r ]

t1

b

t2


If p + q < r or p + q > r we can easily distinguish them.

If p + q = r and p > 0 then q < r so 〈a〉r 〈b〉1⊤ distinguishes them.


* Proof of the logical characterization theorem

Digression on Analytic Spaces

An analytic set A is the image of a Polish space X (or a Borelsubset of X ) under a continuous (or measurable) function f : X−→ Y , where Y is Polish. If (S,Σ) is a measurable space where Sis an analytic set in some ambient topological space and Σ is theBorel σ-algebra on S.





Analytic sets do not form a σ-algebra but they are in thecompletion of the Borel algebra under any measure. [Universallymeasurable.]





Analytic sets do not form a σ-algebra but they are in thecompletion of the Borel algebra under any measure. [Universallymeasurable.]

Regular conditional probability densities can be defined onanalytic spaces.



Amazing Facts about Analytic Spaces

Given A an analytic space and ∼ an equivalence relation suchthat there is a countable family of real-valued measurablefunctions fi : S −→ R such that

∀s, s′ ∈ S.s ∼ s′ ⇐⇒ ∀fi .fi(s) = fi(s′)

then the quotient space (Q,Ω) - where Q = S/ ∼ and Ω is thefinest σ-algebra making the canonical surjection q : S −→ Qmeasurable - is also analytic.



Amazing Facts about Analytic Spaces

Given A an analytic space and ∼ an equivalence relation suchthat there is a countable family of real-valued measurablefunctions fi : S −→ R such that

∀s, s′ ∈ S.s ∼ s′ ⇐⇒ ∀fi .fi(s) = fi(s′)

then the quotient space (Q,Ω) - where Q = S/ ∼ and Ω is thefinest σ-algebra making the canonical surjection q : S −→ Qmeasurable - is also analytic.

If an analytic space (S,Σ) has a sub-σ-algebra Σ0 of Σ whichseparates points and is countably generated then Σ0 is Σ! TheUnique Structure Theorem (UST).



The Quotient

Given (S,Σ, τa) an LMP, we define s ≃ s′ if s and s′ obey exactlythe same formulas of L0.



The Quotient


The functions I[[φ]] : S −→ R defined by I[[φ]](s) = 1 if s |= φ and 0otherwise are a countable family of measurable functions suchthat s ≃ s′ if and only if all the functions agree on s and s′. Thusthe quotient space (Q,Ω) is analytic.



The Quotient


The functions I[[φ]] : S −→ R defined by I[[φ]](s) = 1 if s |= φ and 0otherwise are a countable family of measurable functions suchthat s ≃ s′ if and only if all the functions agree on s and s′. Thusthe quotient space (Q,Ω) is analytic.

We define an LMP (Q,Ω, ρa) where ρa(t ,U) := τa(s,q−1(U));s ∈ q−1(t).



ρ is well defined - I

Easy to check that q−1(q([[φ]])) = [[φ]]:s ∈ q−1(q([[φ]])) implies that q(s) ∈ q([[φ]]), i.e. ∃s′ ∈ [[φ]].s ≃ s′, so s |= φ so s ∈ [[φ]].





Thus q([[φ]]) is measurable.






Thus the σ-algebra generated -say, Λ - by q([[φ]]) is asub-σ-algebra of Ω.






Thus the σ-algebra generated -say, Λ - by q([[φ]]) is asub-σ-algebra of Ω.

Λ is countably generated and separates points so by UST it is Ω.Thus q([[φ]]) generates Ω.



ρ is well defined - II

The collection q([[φ]]) is a π-system (because L0 has conjunction)and it generates Ω; thus if we can show that two measures agreeon these sets they agree on all of Ω.





If q(s) = q(s′) = t then τa(s, [[φ]]) = τa(s′, [[φ]]) (simpleinterpolation).





If q(s) = q(s′) = t then τa(s, [[φ]]) = τa(s′, [[φ]]) (simpleinterpolation).

Thus τa(s,q−1(q([[φ]]))) = τa(s′,q−1(q([[φ]]))) and hence ρ is welldefined. We have ρa(q(s),B) = τa(s,q−1(B)).



Finishing the Argument

Let X be any ≃-closed subset of S.





Then q−1(q(X )) = X and q(X ) ∈ Ω.





Then q−1(q(X )) = X and q(X ) ∈ Ω.

If s ≃ s′ then q(s) = q(s′) and

τa(s,X ) = τa(s,q−1(q(X ))) = ρa(q(s),q(X )) =

ρa(q(s′),q(X )) = τa(s′,q−1(q(X ))) = τa(s′,X ).


Metrics

A metric-based approximate viewpoint

Move from equality between processes to distances betweenprocesses (Jou and Smolka 1990).


Metrics



Formalize distance as a metric:

d(s, s) = 0,d(s, t) = d(t , s),d(s,u) ≤ d(s, t) + d(t ,u).

Quantitative analogue of an equivalence relation.


Metrics



Formalize distance as a metric:

d(s, s) = 0,d(s, t) = d(t , s),d(s,u) ≤ d(s, t) + d(t ,u).

Quantitative analogue of an equivalence relation.

Quantitative measurement of the distinction between processes.


Metrics

Criteria on Metrics

Soundness:d(s, t) = 0 ⇔ s, t are bisimilar


Metrics

Criteria on Metrics


Stability of distance under temporal evolution:“Nearby states stayclose forever.”


Metrics

Criteria on Metrics


Stability of distance under temporal evolution:“Nearby states stayclose forever.”

Metrics should be computable (efficiently?).


Metrics

Bisimulation Recalled

Let R be an equivalence relation. R is a bisimulation if: s R t if:

(s −→ P) ⇒ [t −→ Q,P =R Q]

(t −→ Q) ⇒ [s −→ P,P =R Q]

where P =R Q if

(∀R − closed E) P(E) = Q(E)


Metrics

A putative definition of a metric analogue ofbisimulation

m is a metric-bisimulation if: m(s, t) < ǫ ⇒:

s −→ P ⇒ t −→ Q, m(P,Q) < ǫ

t −→ Q ⇒ s −→ P, m(P,Q) < ǫ


Metrics



s −→ P ⇒ t −→ Q, m(P,Q) < ǫ

t −→ Q ⇒ s −→ P, m(P,Q) < ǫ

Problem: what is m(P,Q)? — Type mismatch!!


Metrics



s −→ P ⇒ t −→ Q, m(P,Q) < ǫ

t −→ Q ⇒ s −→ P, m(P,Q) < ǫ

Problem: what is m(P,Q)? — Type mismatch!!

Need a way to lift distances from states to a distances ondistributions of states.


Metrics

A detour: Kantorovich metric

Metrics on probability measures on metric spaces.


Metrics



M: 1-bounded pseudometrics on states.


Metrics




d(µ, ν) = supf

|

∫

fdµ−

∫

fdν|, f 1-Lipschitz


Metrics




d(µ, ν) = supf

|

∫

fdµ−

∫

fdν|, f 1-Lipschitz

Arises in the solution of an LP problem: transshipment.


Metrics

An LP version for Finite-State Spaces

When state space is finite: Let P,Q be probability distributions. Then:

m(P,Q) = max∑

i

(P(si)− Q(si))ai

subject to:∀i .0 ≤ ai ≤ 1∀i , j . ai − aj ≤ m(si , sj).


Metrics

The Dual Form

Dual form from Worrell and van Breugel:


Metrics

The Dual Form


min∑

i ,j

lijm(si , sj ) +∑

i

xi +∑

j

yj

subject to:∀i .

∑

j lij + xi = P(si)

∀j .∑

i lij + yj = Q(sj )∀i , j . lij , xi , yj ≥ 0.


Metrics

The Dual Form


min∑

i ,j

lijm(si , sj ) +∑

i

xi +∑

j

yj

subject to:∀i .

∑

j lij + xi = P(si)

∀j .∑

i lij + yj = Q(sj )∀i , j . lij , xi , yj ≥ 0.

We prove many equations by using the primal form to show onedirection and the dual to show the other.


Metrics

Example

Let m(s, t) = r < 1. Let δs(δt) be the probability measureconcentrated at s(t). Then,

m(δs, δt) = r


Metrics

Example


m(δs, δt) = r

Upper bound from dual: Choose lst = 1 all other lij = 0. Then

∑

ij

lijm(si , sj ) = m(s, t) = r .


Metrics

Example


m(δs, δt) = r

Upper bound from dual: Choose lst = 1 all other lij = 0. Then

∑

ij

lijm(si , sj ) = m(s, t) = r .

Lower bound from primal: Choose as = 0,at = r , all others tomatch the constraints. Then

∑

i

(δt (si)− δs(si ))ai = r .


Metrics

The Importance of the Example

We can isometrically embed the original space in the metric space ofdistributions.


Metrics

Return from Detour

Summary of detour: Given a metric on states in a metric space, can liftto a metric on probability distributions on states.


Metrics

Metric “Bisimulation”


s −→ P ⇒ t −→ Q, m(P,Q) < ǫ

t −→ Q ⇒ s −→ P, m(P,Q) < ǫ


Metrics



s −→ P ⇒ t −→ Q, m(P,Q) < ǫ

t −→ Q ⇒ s −→ P, m(P,Q) < ǫ

The required canonical metric on processes is the least such: ie.the distances are the least possible.


Metrics



s −→ P ⇒ t −→ Q, m(P,Q) < ǫ

t −→ Q ⇒ s −→ P, m(P,Q) < ǫ

The required canonical metric on processes is the least such: ie.the distances are the least possible.

Thm: Canonical least metric exists. Usual fixed-point theoryarguments.


Continuous-state systems

What about Continuous-State Systems?

Develop a real-valued “modal logic” based on the analogy:Program Logic Probabilistic LogicState s Distribution µFormula φ Random Variable fSatisfaction s |= φ

∫

f dµ





∫

f dµ

Define a metric based on how closely the random variables agree.





∫

f dµ

Define a metric based on how closely the random variables agree.

We did this before the LP based techniques became available.



Real-valued Modal Logic

f ::= 1 | max(f , f ) | h f | 〈a〉.f




f ::= 1 | max(f , f ) | h f | 〈a〉.f

1(s) = 1 Truemax(f1, f2)(s) = max(f1(s), f2(s)) Conjunctionh f (s) = h(f (s)) Lipschitz〈a〉.f (s) = γ

∫

s′∈S f (s′)τa(s, ds′) a-transition

where h 1-Lipschitz : [0,1] → [0,1] and γ ∈ (0,1].




f ::= 1 | max(f , f ) | h f | 〈a〉.f


∫



d(s, t) = supf |f (s)− f (t)|




f ::= 1 | max(f , f ) | h f | 〈a〉.f


∫



d(s, t) = supf |f (s)− f (t)|

Thm: d coincides with the canonical metric-bisimulation.



Finitary syntax for Real-valued modal logic

1(s) = 1 Truemax(f1, f2)(s) = max(f1(s), f2(s)) Conjunction(1 − f )(s) = 1 − f (s) Negation

⌊fq(s)⌋ =

q , f (s) ≥ qf (s) , f (s) < q

Cutoffs

〈a〉.f (s) = γ∫


q is a rational.


Conclusions

Recent results and related work

Markov Decision Processes with continuous state spaces. [Ferns,P., Precup]


Conclusions


Markov Decision Processes with continuous state spaces. [Ferns,P., Precup]Sampling based techniques for approximating the metric.


Conclusions


Markov Decision Processes with continuous state spaces. [Ferns,P., Precup]Sampling based techniques for approximating the metric.Logical characterization for simulation via domain theory, but weneed disjunction. [DGJP]


Conclusions


Markov Decision Processes with continuous state spaces. [Ferns,P., Precup]Sampling based techniques for approximating the metric.Logical characterization for simulation via domain theory, but weneed disjunction. [DGJP]Approximation theory for continuous space LMPs. [DGJP, BFPP,DD, DDP]


Conclusions


Markov Decision Processes with continuous state spaces. [Ferns,P., Precup]Sampling based techniques for approximating the metric.Logical characterization for simulation via domain theory, but weneed disjunction. [DGJP]Approximation theory for continuous space LMPs. [DGJP, BFPP,DD, DDP]Domains of LMPs, universal LMP [DGJP]


Conclusions


Markov Decision Processes with continuous state spaces. [Ferns,P., Precup]Sampling based techniques for approximating the metric.Logical characterization for simulation via domain theory, but weneed disjunction. [DGJP]Approximation theory for continuous space LMPs. [DGJP, BFPP,DD, DDP]Domains of LMPs, universal LMP [DGJP]Metrics [van Breugel, Worrel]


Conclusions


Markov Decision Processes with continuous state spaces. [Ferns,P., Precup]Sampling based techniques for approximating the metric.Logical characterization for simulation via domain theory, but weneed disjunction. [DGJP]Approximation theory for continuous space LMPs. [DGJP, BFPP,DD, DDP]Domains of LMPs, universal LMP [DGJP]Metrics [van Breugel, Worrel]Weak bisimulation and metric analogue [DGJP]


Conclusions


Markov Decision Processes with continuous state spaces. [Ferns,P., Precup]Sampling based techniques for approximating the metric.Logical characterization for simulation via domain theory, but weneed disjunction. [DGJP]Approximation theory for continuous space LMPs. [DGJP, BFPP,DD, DDP]Domains of LMPs, universal LMP [DGJP]Metrics [van Breugel, Worrel]Weak bisimulation and metric analogue [DGJP]Duality theory for LMPs. [Worrell et al.]


Conclusions


Markov Decision Processes with continuous state spaces. [Ferns,P., Precup]Sampling based techniques for approximating the metric.Logical characterization for simulation via domain theory, but weneed disjunction. [DGJP]Approximation theory for continuous space LMPs. [DGJP, BFPP,DD, DDP]Domains of LMPs, universal LMP [DGJP]Metrics [van Breugel, Worrel]Weak bisimulation and metric analogue [DGJP]Duality theory for LMPs. [Worrell et al.]Decision procedure for metric without discount. [van Breugel et al.]


Conclusions


Markov Decision Processes with continuous state spaces. [Ferns,P., Precup]Sampling based techniques for approximating the metric.Logical characterization for simulation via domain theory, but weneed disjunction. [DGJP]Approximation theory for continuous space LMPs. [DGJP, BFPP,DD, DDP]Domains of LMPs, universal LMP [DGJP]Metrics [van Breugel, Worrel]Weak bisimulation and metric analogue [DGJP]Duality theory for LMPs. [Worrell et al.]Decision procedure for metric without discount. [van Breugel et al.]Beautiful coinduction principle for stochastic processes due toKozen.


Conclusions


Markov Decision Processes with continuous state spaces. [Ferns,P., Precup]Sampling based techniques for approximating the metric.Logical characterization for simulation via domain theory, but weneed disjunction. [DGJP]Approximation theory for continuous space LMPs. [DGJP, BFPP,DD, DDP]Domains of LMPs, universal LMP [DGJP]Metrics [van Breugel, Worrel]Weak bisimulation and metric analogue [DGJP]Duality theory for LMPs. [Worrell et al.]Decision procedure for metric without discount. [van Breugel et al.]Beautiful coinduction principle for stochastic processes due toKozen.New completeness theorems: Cardelli, Larsen, Mardare [ICALP2011]Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 45 / 46

Conclusions

Work in Progress

New approach to approximation based on averaging [Chaput,Danos, P., Plotkin, ICALP 2009]


Conclusions

Work in Progress


Approximations of the logic and convergence properties [Larsen,Mardare, P.]


Conclusions

Work in Progress


Approximations of the logic and convergence properties [Larsen,Mardare, P.]

New Stone-type duality theory [Larsen, Mardare, P.]


Documents

Bisimulation and Metrics for Labelled Markov Processesprakash/Talks/aalborg_2012.pdf · The transitions could be indeterminate (nondeterministic). Panangaden (McGill) Bisimulation