Learning Büchi Automata and Its Applications · MSO NFA Both MSO and NFA de ne the class of regular expressions. ... vol. 131 of LNCS, pages 52-71. Springer-Verlag, 1981. ... Part

Learning Buchi Automata and Its Applications

Lijun Zhang

Institute of Software, Chinese Academy of Sciences

9th April 2018

Overview

Part 1 Motivations

Part 2 The ins and outs of Buchi automata

Part 3 Learning Algorithms for finite and Buchi automata

Part 4 Applications

1 / 243

• Who is Buchi?

• Why he introduced Buchi automata?

• What is Buchi automata?

• Is it useful?

2 / 243

Julius Richard Buchi

• a Swiss logician and mathematician (1924-1984)• received his diploma in mathematics and theoretical physics at

ETH Zurich (Prof. Hopf)• went to home (St. Gallen) for eight months to work on a

problem• 1950: showed the works to Prof. Hopf, Prof. Bernays

3 / 243

Automata and Logic

Definition (Automata and Logic, Buchi60, Elgot61)

MSO ≡ NFABoth MSO and NFA define the class of regular expressions.Proof: Effective

• From NFA to MSO (A→ ϕA)

• From MSO to NFA (ϕ→ Aϕ)

what about the infinite dimension?

4 / 243

Automata and Logic

Definition (Automata and Logic, Buchi60, Elgot61)

MSO ≡ NFABoth MSO and NFA define the class of regular expressions.Proof: Effective

• From NFA to MSO (A→ ϕA)

• From MSO to NFA (ϕ→ Aϕ)

what about the infinite dimension?

4 / 243

Why he introduced Buchi automata?

• Buchi, J.R. (1962). ”On a decision method in restrictedsecond order arithmetic”. Proc. International Congress onLogic, Method, and Philosophy of Science. Stanford: StanfordUniversity Press: 1-12.

5 / 243

Part I

Motivation

1 Program Termination Analysis

2 Temporal Logic

3 Model Checking

4 Model & Specification Learning

6 / 243

Program Termination Analysis

Does this program terminate?

program fun( ):

`1: while (i>0 and y>0):

`2: if input()=1 then

`3: x := x-1

`4: y := y+1

`5: else

`6: y := y-1

`7: fi

`8: done

7 / 243

Entscheidungsproblem (The Decision Problem)

• Hilbert-Ackermann, 1928: Entscheidungsproblem, decide if agiven first-order sentence is valid (dually, satisfiable).

• Church-Turing Theorem, 1936: The Decision Problem isunsolvable.

• Turing, 1936: Defined computability in terms of Turingmachines (TMs)

• Proved that the halting problem for TMs is unsolvable

• Reduced halting problem to Entscheidungsproblem.

8 / 243

Halting Problem

It takes as input a computer program and input to the programand determines whether the program will eventually stop when runwith this input.

• If the program halts, we have our answer.

• If it is still running after any fixed length of time has elapsed,we do not know whether it will never halt or we just did notwait long enough for it to terminate.

program loop(int i):

`1: while (i>0):

`2: skip

9 / 243

Undecidability of the Halting Problem

10 / 243

Termination

B. Cook, A. Podelski, and A. Rybalchenko, 2011, CACM: ProvingProgram Termination.

• “in contrast to popular belief, proving termination is notalways impossible”

• The Terminator tool can prove termination or divergence ofmany Microsoft programs.

• Tool is not guaranteed to terminate! Explanation:

• Most real-life programs, if they terminate, do so for rathersimple reasons.

Andrey Rybalchenko, at 32, 2010: Innovators under 35, MITTechnology Review.

11 / 243

MIT Technology Review

Andrey Rybalchenko has developed (LICS’04) a new method forfinding software bugs

• automated testing systems detect when programs do ”badthings” that lead to crashes, forcing the program to quit.

• misses bugs that allow the software to keep running but leaveit unable to accept new input or do anything useful.

• In essence, Rybalchenko instead tries to identify when aprogram is doing ”good things”, such as making progressthrough loops or responding to other programs.

• with Microsoft, in 2006, Rybalchenko incorporated hismethods into Terminator, a commercial program used to findbugs in the device drivers.

12 / 243

Safety & Liveness Lamport

Mutual Exclusion Examples

• always not (CS1 and CS2): safety

• always (Request implies eventually Grant): liveness

• always (Request implies (Request until Grant)): liveness

13 / 243

Terminator tools: starte-of-the-art

SV-COMP: Intl. Competition on Software Verification held atTACAS 2018

• Goal of the competition: Provide a snapshot of thestate-of-the-art in software verification to the community

14 / 243

Terminator: starte-of-the-art tools

• AProVE: based on reduction to term rewritting system

• Terminator: based on transition invariants

• T2, CPA-Seq: based on transition invariants

• UAutomizer: based on

15 / 243

Part I

Motivation


2 Temporal Logic

3 Model Checking


16 / 243

Program Behaviours

• Does the program terminates?

• Is the program safe (buffer overflow, zero pointer, deadlock,mutual exculsion)?

• Is the protocol safe (same ip property in IEEE Zeroconfprotocol)?

17 / 243

18 / 243

Program Behaviours

Amir Pnueli (1941-2009)

• He studied mathematics at the Technion during 1958-1962

• He continued directly to PhD studies in the WeizmannInstitute of Science in Israel

• During 1967 and 1968, postdoc at Stanford University and atIBM research center in Yorktown Heights, New York

• During a sabbatical at the University of Pennsylvania he wasintroduced to the work of the philosopher Arthur Prior

Arthur Prior: Past, Present, and Future in 1967

19 / 243

Arthur Prior (1914-1969)

Consider the statement ”I am hungry”. It maybe true today, butfalse tomorrow.Prior, born in New Zealand, introduced tense logic (Past, Present,and Future):

ϕ ::= a | ¬ϕ | ϕ ∧ ϕ | Gϕ | Fϕ | Pϕ | Hϕ

20 / 243

Program Behaviours

Amir: the first to realize the potential implications of applyingPrior’s work to computer programs.

• Amir Pnueli 1977 seminal paper The Temporal Logic ofPrograms

• revolutionized the way computer programs are analyzed

In mathematics, logic is static. It deals with connections amongentities that exist in the same time frame. When one designs adynamic computer system that has to react to ever changingconditions,..., one cannot design the system based on a static view.It is necessary to characterize and describe dynamic behaviors thatconnect entities, events, and reactions at different time points.Temporal Logic deals therefore with a dynamic view of the worldthat evolves over time.”

21 / 243

Program Behaviours

Definition (The Temporal Logic of Programs)

• Pnueli introduced Linear temporal logic (LTL) as a logic forthe specification of programs

• investigated Model checking problem: via reduction to MSO

In 1996, Pnueli received the Turing Award for seminal workintroducing temporal logic into computing science and foroutstanding contributions to program and systems verification.

22 / 243

Model Checking LTL Properties

• the MSO based algorithm has nonelementary complexity

• the most efficient algorithm for checking LTL formulae isbased on

23 / 243

Part I

Motivation


2 Temporal Logic

3 Model Checking


24 / 243

Model Checking

Clarke and his student E. Allen Emerson saw an importantpossibility in temporal logic: it could be directly checked bymachine.

• E.M. Clarke and E.A. Emerson. Design and synthesis ofsynchronization skeletons using branching time temporal logic,In: Proceedings of the Workshop on Logics of Programs, vol.131 of LNCS, pages 52-71. Springer-Verlag, 1981.

• used to synthesize abstractions of concurrent programs

• model checking presented as a secondary result.

• Queille, J. P.; Sifakis, J. (1982), ”Specification andverification of concurrent systems in CESAR”, InternationalSymposium on Programming

• Working independently, Jean-Pierre Queille and Joseph Sifakisdeveloped similar ideas

25 / 243

Model Checking Turing Award 2007

Model Checker: given a finite state model of the system and aformal property, automatically checks whether such a propertyholds for (a given state in) that model.

“does a program behave as intended?”

• mathematical model M (e.g., Kripke structure, transitionsystem), specification ϕ, and automatic proof or refutation of:M ϕ

• applicable for hardware, software, protocols

• potential push-button technology: software tools

s0

error

26 / 243

The state space explosion

• application to practical systems was severely limited: thenumber of states to be explored.

• the number of states a memory location can assume is toomuch

• From the literature, McMillan found an efficient encoding,BDD

• Symbolic model checker

Kenneth L. McMillan, Bell Labs, Cadence Berkeley Laboratories,Microsoft Research: CAV award for a series of fundamentalcontributions resulting in significant advances in scalability ofmodel checking tools.

27 / 243

Futurebus+ Cache Coherence Protocol Clarke Bell Lab. et al. 1995

The first industrial scale case study using model checking

• Edmund M. Clarke, Orna Grumberg, Hiromi Hiraishi, SomeshJha, David E. Long, Kenneth L. McMillan, Linda A. Ness

• Futurebus+: bus architecture for high-performance computers

• Cache coherence protocol: insure consistency of data inhierarchical systems

• 2300 lines of SMV code

• challenge: model construction, property specification (CTL)

• hierarchical, nondeterminism, abstraction

• state explosion: largest configuration verified has 3 bussegments, 8 processors 1030 states

• find potential erros in the protocol

28 / 243

Some major techniques against the explosion

• symbolic algorithms (open-source BDD manipulation librariessuch as CUDD)

• bounded model checking algorithm: unroll the system for afixed number of steps and do the checking

• bisimulation reduction: reduce the system to its bisimulationquotient

• partial order reduction: reduce the number of independentinterleavings of concurrent processes that need to beconsidered

• abstraction: prove the property on the simplified system

• CEGAR: Counterexample guided abstraction refinement

• learning

29 / 243

Part I

Motivation


2 Temporal Logic

3 Model Checking


30 / 243

How are the models obtained?

• from source codes, protocols, circuits ...

• often abstraction applied to achieve a model of modest size

• how faithfully are they representing the original system?

31 / 243

one can learn the model

32 / 243

Angluin-Style Exact Learning Framework Angluin 1987

Learning an automaton A efficiently using membership andequivalence query

33 / 243

Model Learning Peled et al. Steffen et al. 2002

• SUL: System Under Learning

• Black box, active learning

• Assumption: we can bring it back to initial state

• Membership query is easy to answer

• Equivalence query: exploit conformance testing via testqueries

34 / 243

Model Checking & Model Learning Peled et al. 2002

• Goal: to check a system SUL satisfies a set of propertiesϕ1,. . . ,ϕk

• Learn M using model learning

• Equivalence query• M satisfies all ϕi : pass it through the conformance tester• otherwise: analyse counterexample (spurious, or real)

35 / 243

Compositional/AG verification Cobleigh, Giannakopoulou, and Pasareanu

TACAS’03

• Goal: to check a composed system M ‖ M ′ |= ϕ

• Divide & Conquer: find an abstraction A of M

• A preserves/abstracts M

• A should be much smaller than M

• check A ‖ M ′ |= ϕ instead

Design learning algorithm to learn the abstraction A

36 / 243

Learning for Probabilistic model checking

Probability is the core part for several systems and situations:

• randomized algorithms (exploited in protocols)

• reliability, performance

• probabilistic programming

• optimization

• system biology

We will discuss how it can be used in this setting.

37 / 243

Part II

The ins and outs of Buchi automata

5 Nondeterministic Finite Automata

6 Automata and Logic

7 Buchi automata

38 / 243

Automaton & Regular Language

• The regular languageL = Σ∗a

• automaton A = (Σ,Q, q0, δ,F ) accepting L

39 / 243

Regular Language

For a given set of letters (alphabet) Σ,

• ε, ∅, a ∈ Σ are regular expressions

• if E ,F are regular expressions, E .F , E ∪ F , and E ∗ are regularexpressions

• The language

L = u ∈ a, b+ | the number of b in u is 4n + 3

is regular

• Regular expression for L:

(a∗.b.a∗.b.a∗.b.a∗).(b.a∗.b.a∗.b.a∗.b.a∗)∗

40 / 243

Nondeterministic Finite Automata

A nondeterministic finite automata (NFA) is a tupleA = (Σ,Q, I , ρ,F ) where

• Q is a finite set of states

• Σ is the set of alphabet

• I ⊆ Q is the set of initial states

• ρ : Q × Σ→ 2Q is the transition relation

• F ⊆ Q is the set of accepting states

We omit Σ if it is clear from the context. We say A isdeterimnistic if ρ : Q × Σ→ Q.

41 / 243

Examples of NFA

q0 q1B1 :

ba

a

b

r0 r1 r2B2 :

b

a

a

a

b

b

a

42 / 243

Semantics of NFA

Given w = a0a1 . . . an−1 ∈ Σ∗, a run π of A on w is an finitesequence of states π = q0q1 . . . qn such that q0 ∈ I and for alli = 0, . . . , n − 1, qi+1 ∈ ρ(qi , ai )

The run π is accepting if qn ∈ F . A word w ∈ Σ∗ is accepted by Aif there exists an accepting run π on w

The language of A is the set of all accepted words:L(A) = w ∈ Σ∗ | A has an accepting run on w

43 / 243

Semantics of NFA

Given w = a0a1 . . . an−1 ∈ Σ∗, a run π of A on w is an finitesequence of states π = q0q1 . . . qn such that q0 ∈ I and for alli = 0, . . . , n − 1, qi+1 ∈ ρ(qi , ai )

The run π is accepting if qn ∈ F . A word w ∈ Σ∗ is accepted by Aif there exists an accepting run π on w

The language of A is the set of all accepted words:L(A) = w ∈ Σ∗ | A has an accepting run on w

43 / 243

Examples of NFA

q0 q1A1 :

ba

a

b

r0 r1 r2A2 :

b

a

a

a

b

b

a

44 / 243

What is the NFA for the language L = Σ∗aΣn?

45 / 243

Operations on NFA: Union

Given two NBAs A1 and A2, there exists an NBA A such that

L(A) = L(A1) ∪ L(A2) and |A| ∈ O(|A1|+ |A1|)

q0 q1A1 :

ba

a

b

r0 r1 r2A2 :

b

a

a

a

b

b

a

A = A1 ∪ A2

46 / 243

Operations on NFA: Intersection

Given two NFAs A1 and A2, there exists an NFA A such that

L(A) = L(A1) ∩ L(A2) and |A| ∈ O(|A1| · |A1|)

The intersection is simpler with product automaton

47 / 243

Subset Construction

For an NFA A = (Q, I , ρ,F ), with subset construction we have aDFA defined by

• set of states: 2Q

• initial state: I

• transition: ρ(S , a) =

• set of final states:

What is the DFA for the language L = Σ∗aΣn?

48 / 243

Subset Construction

For an NFA A = (Q, I , ρ,F ), with subset construction we have aDFA defined by

• set of states: 2Q

• initial state: I

• transition: ρ(S , a) =

• set of final states:

What is the DFA for the language L = Σ∗aΣn?

48 / 243

Complementation

A = (Q, I , ρ,F )

• If A is an DFA

• If A is an NFA

49 / 243

Emptiness

Nonemptiness Problem: Decide if given A, L(A) is nonempty.Directed Graph GA = (S ,E ) of NFA A = (Σ,Q,Q0, ρ,F ):

• Nodes: S = Q

• Edges: E = (s, t) : t ∈ ρ(s, a) for some a ∈ ΣIt holds: A is nonempty iff there is a path in GA from Q0 to F .Decidable in time linear in size of A, using breadth-first search ordepth-first search.

50 / 243

Part II




7 Buchi automata

51 / 243

An example

Consider the alphabet Σ = a, b, c, and the those words over Σsuch that

• no a is succeeded by b,

• any b is succeeded by a,

• a is the last letter

An automaton for it:

52 / 243

An example

Consider the alphabet Σ = a, b, c, and the those words over Σsuch that

• no a is succeeded by b,

• any b is succeeded by a,

• a is the last letter

A formula in first order logic (FOL) for it:

• variables x , y for letter positions

• S(x , y): successor predicate

• Pa(x): the position x carries a

• last(x) := ¬∃yS(x , y)

• ¬∃x∃y(S(x , y) ∧ Pa(x) ∧ Pb(y)

• ∀x(Pb(x)→ ∃yS(x , y) ∧ Pa(y))

• ∃x(last(x) ∧ Pa(x))

53 / 243

Syntax of First Order Logic

The well-formed formulas of FOL are constructed according to thefollowing grammar:

ϕ ::= x < y | Pa(x) | S(x , y) | ¬ϕ | ϕ→ ϕ | ∀xϕ

where x , y are variables.

• ∀xϕ: variable x is bound, ϕ is in the scope of quantifier ∀x .

• ϕ(x , y): formula ϕ has (only) free variables x , y (not in thescope of some quantifiers)

• a sentence if a formula without free variables

Some formulas:

• last(x) := ¬∃yS(x , y)

• ¬∃x∃y(S(x , y) ∧ Pa(x) ∧ Pb(y)

• ∀x(Pb(x)→ ∃yS(x , y) ∧ Pa(y))


54 / 243

Syntax of First Order Logic


ϕ ::= x < y | Pa(x) | S(x , y) | ¬ϕ | ϕ→ ϕ | ∀xϕ


• ∀xϕ: variable x is bound, ϕ is in the scope of quantifier ∀x .

• ϕ(x , y): formula ϕ has (only) free variables x , y (not in thescope of some quantifiers)


Some formulas:

• last(x) := ¬∃yS(x , y)

• ¬∃x∃y(S(x , y) ∧ Pa(x) ∧ Pb(y)

• ∀x(Pb(x)→ ∃yS(x , y) ∧ Pa(y))


54 / 243

Finite Word Models

Definition (Finite Words)

View finite word w = a0, ..., an−1 over alphabet Σ as amathematical structure:

• Domain: D = 0, 1, . . . , n − 1• Dyadic predicate: <

• Monadic predicates: Pa : a ∈ Σ

55 / 243

Semantics of FOL


ϕ ::= x < y | Pa(x) | S(x , y) | ¬ϕ | ϕ→ ϕ | ∀xϕ


• (w , p1, . . . , pm) |= ϕ(x1, . . . , xm): formula ϕ is satisfied in wwhen free variables x1, . . . , xm are interpreted byp1, . . . , pm ∈ D

Consider

• last(x) := ¬∃yS(x , y)


56 / 243

Semantics of FOL


ϕ ::= x < y | Pa(x) | S(x , y) | ¬ϕ | ϕ→ ϕ | ∀xϕ


• (w , p1, . . . , pm) |= ϕ(x1, . . . , xm): formula ϕ is satisfied in wwhen free variables x1, . . . , xm are interpreted byp1, . . . , pm ∈ D

Consider

• last(x) := ¬∃yS(x , y)


56 / 243

An example

Consider the alphabet Σ = a, b, and the those words over Σsuch that

• any two occurrences of b (with no b between them) areseparated by an odd number of letter a

An automaton for it:

57 / 243

An example

Consider the alphabet Σ = a, b, and the those words over Σsuch that

• any two occurrences of b (with no b between them) areseparated by an odd number of letter a

A formula in monadic second order logic (MSO) for it:

• between such two b: there is a set of positions containing thefirst b, then every second position, and finally the last b

• variables X ,Y vary over set of positions

• atomic formula X (y): y ∈ X

• ∀x∀y(Pb(x) ∧ x < y ∧ Pb(y) ∧ ∀z(x < z ∧ z < y → ¬Pb(z))

• ∃X (X (x) ∧ ∀u∀v(S(u, v)→ (X (u)↔ ¬X (v))) ∧ X (y))

58 / 243

Syntax of Monadic Second Order Logic

The well-formed formulas of MSO are constructed according to thefollowing grammar:

ϕ ::= X ⊆ Y | Sing(X ) | Pa(x) | S(X ,Y ) | X ⊆ Pa | ¬ϕ | ϕ→ ϕ | ∀Xϕ

where X ,Y are second order variables.

• ∀Xϕ: variable X is bound, ϕ is in the scope of quantifier ∀X .

• ϕ(X ,Y ): formula ϕ has (only) free variables X ,Y (not in thescope of some quantifiers)


• X (y): y ⊆ X

• x < y :¬x = y ∧∀X (X (x)∧∀z∀z ′(X (z)∧S(z , z ′)→ X (z ′))→ X (y))

• ∀x(...): ∀X (Sing(X ) ∧ ...)

59 / 243

Syntax of Monadic Second Order Logic




• ∀Xϕ: variable X is bound, ϕ is in the scope of quantifier ∀X .

• ϕ(X ,Y ): formula ϕ has (only) free variables X ,Y (not in thescope of some quantifiers)


• X (y): y ⊆ X

• x < y :¬x = y ∧∀X (X (x)∧∀z∀z ′(X (z)∧S(z , z ′)→ X (z ′))→ X (y))

• ∀x(...): ∀X (Sing(X ) ∧ ...)

59 / 243

Finite Word Models

Definition (Finite Words)

View finite word w = a0, ..., an−1 over alphabet Σ as amathematical structure:

• Domain: 0, ..., n − 1

• Dyadic predicate: <

• Monadic predicates: Pa : a ∈ Σ

60 / 243

Semantics of MSO




• (w ,P1, . . . ,Pm) |= ϕ(X1, . . . ,Xm): formula ϕ is satisfied in wwhen free variables X1, . . . ,Xm are interpreted byP1, . . . ,Pm ⊆ D.

• Equivalently, extend alphabet Σ′ = Σ ∪ 0, 1m: label(a, c1, c2, . . . , cm) of position p ∈ D means p ∈ Pi iff ci = 1.

Consider

• X ⊆ Y

61 / 243

Semantics of MSO




• (w ,P1, . . . ,Pm) |= ϕ(X1, . . . ,Xm): formula ϕ is satisfied in wwhen free variables X1, . . . ,Xm are interpreted byP1, . . . ,Pm ⊆ D.

• Equivalently, extend alphabet Σ′ = Σ ∪ 0, 1m: label(a, c1, c2, . . . , cm) of position p ∈ D means p ∈ Pi iff ci = 1.

Consider

• X ⊆ Y

61 / 243

Automata and Logic Buchi60, Elgot61

MSO ≡ NFA. Both MSO and NFA define the class of regularexpressions.Proof: From NFA to MSO (A → ϕA). Assume A = (Q, q0, ρ,F )with Q = 0, 1, . . . , k and q0 = 0.

• w = a0a1 . . . an−1 ∈ L(A): π = q0q1 . . . qn such that q0 = 0and for all i = 0, . . . , n − 1, qi+1 ∈ ρ(qi , ai ), and qn ∈ F .

• we code states q0, . . . , qn−1 by a tuple (X0, . . . ,Xk) ofpairwise disjoint subsets of 0, . . . , n − 1 such that: Xi

contains those positions of w where state i is assumed

• ϕ = ∃X0 . . . ∃Xk(ϕ1 ∧ ϕ2 ∧ ϕ3 ∧ ϕ4)

• ϕ1 = ∧i 6=j∀x¬(Xi (x) ∧ Xj(x))

• ϕ2 = ∀x(first(x)→ X0(x))

• ϕ3 = ∀x∀y(S(x , y)→ ∨(i ,a,j)∈ρ(Xi (x) ∧ Pa(x) ∧ Xj(y)))

• ϕ4 = ∀x(last(x)→ ∨(i ,a,j)∈ρ and j∈F (Xi (x) ∧ Qa(x)))

62 / 243


MSO ≡ NFA. Both MSO and NFA define the class of regularexpressions.Proof: From NFA to MSO (A → ϕA). Assume A = (Q, q0, ρ,F )with Q = 0, 1, . . . , k and q0 = 0.

• w = a0a1 . . . an−1 ∈ L(A): π = q0q1 . . . qn such that q0 = 0and for all i = 0, . . . , n − 1, qi+1 ∈ ρ(qi , ai ), and qn ∈ F .

• we code states q0, . . . , qn−1 by a tuple (X0, . . . ,Xk) ofpairwise disjoint subsets of 0, . . . , n − 1 such that: Xi


• ϕ = ∃X0 . . . ∃Xk(ϕ1 ∧ ϕ2 ∧ ϕ3 ∧ ϕ4)

• ϕ1 = ∧i 6=j∀x¬(Xi (x) ∧ Xj(x))

• ϕ2 = ∀x(first(x)→ X0(x))



62 / 243


MSO ≡ NFA. Both MSO and NFA define the class of regularexpressions.Proof: From MSO to NFA (ϕ→ Aϕ). Let ϕ(X1, . . . ,Xn) be aMSO formula. We construct an NFA accepting w ∈ Σ× 0, 1nsatisfying ϕ.

• atomic formulas Xj ⊆ Xi : checks when 1 occurs in j-thsequence, it also do so for i-th sequence

• Sing(X ),Suc(Xj ,Xk),Xj ⊆ Qa

• ϕ1 ∧ ϕ2

• ϕ1 ∨ ϕ2

• ¬ψ• ϕ(X1, . . . ,Xn) = ∃Xn+1ψ(X1, . . . ,Xn+1): We have A forψ(X1, . . . ,Xn+1) over Σ×0, 1n+1. Nondeterministicly guessthe sequence defining the n + 1-th additional components,and work on it over like A.

63 / 243


MSO ≡ NFA. Both MSO and NFA define the class of regularexpressions.Proof: From MSO to NFA (ϕ→ Aϕ). Let ϕ(X1, . . . ,Xn) be aMSO formula. We construct an NFA accepting w ∈ Σ× 0, 1nsatisfying ϕ.



• ϕ1 ∧ ϕ2

• ϕ1 ∨ ϕ2

• ¬ψ• ϕ(X1, . . . ,Xn) = ∃Xn+1ψ(X1, . . . ,Xn+1): We have A forψ(X1, . . . ,Xn+1) over Σ×0, 1n+1. Nondeterministicly guessthe sequence defining the n + 1-th additional components,and work on it over like A.

63 / 243

MSO Satisfiability

Definition (MSO Satisfiability - Finite Words)

Satisfiability: models(ψ) = ∅Satisfiability Problem: Decide if given ψ is satisfiable.It holds: ψ is satisfiable iff Aψ is nonnempty.It holds: MSO satisfiability is decidable.

• Translate ψ to Aψ.

• Check nonemptiness of Aψ .

Computational Complexity:

• Naive Upper Bound: Nonelementary Growth 2 to the power ofthe tower of height O(n)

• Lower Bound [Stockmeyer, 1974]: Satisfiability of FO overfinite words is nonelementary (no bounded-height tower).

64 / 243

So what happens for infinite words?

65 / 243

Infinite Word Models

Definition (Infinite Word Models)

View finite word w = a0, a1, . . . over alphabet Σ as a mathematicalstructure:

• Domain: D = 0, 1, . . ., i.e., natural numbers.

• Dyadic predicate: ≤• Monadic predicates: Pa : a ∈ Σ

Interpretations of FOL or MSO formulae are the same. Consider:

• last(x) := ¬∃yS(x , y)

• ∀x∃y(x < y ∧ Pa(y))

• ∃x∀y(x < y → ¬Pa(y))

66 / 243

Automata and Logic: The infinite case

Lemma (Automata and Logic, Buchi62)

MSO ≡ BABoth MSO and NFA define the class of ω-regular expressions.Proof: Effective

• From BA to MSO (A→ ϕA)

• From MSO to BA (ϕ→ Aϕ)

67 / 243

Part II




7 Buchi automata

68 / 243

Omega-regular languages

An ω language is regular if it corresponds to the language of anω-regular expression

U1V ω1 + U2V ω

2 + · · ·+ UnV ωn

where Ui ⊆ Σ∗, Vi ⊆ Σ+ are regular languages

69 / 243

What Buchi automata are

Buchi automata are the simplest automata accepting ω-regularlanguages

A nondeterministic Buchi automaton is a tuple B = (Q, I , ρ,F )where




• F ⊆ Q is the set of accepting states

70 / 243

Examples of Buchi automata

q0 q1B1 :

ba

a

b

r0 r1 r2B2 :

b

a

a

a

b

b

a

71 / 243

Semantics of Buchi automata

Given w = a0a1 . . . ∈ Σω, a run π of B on w is an infinitesequence of states π = q0q1 . . . such that q0 ∈ I and for all i ∈ N,qi+1 ∈ ρ(qi , ai )

A run π = q0q1 . . . is accepting if Inf(π) ∩ F 6= ∅, whereInf(π) = q ∈ Q | ∀i ∈ N∃j > i : qj = q

A word w ∈ Σω is accepted by B if there exists an accepting run πon w

The language of B is the set of all accepted words:L(B) = w ∈ Σω | B has an accepting run on w

72 / 243

Examples of Buchi automata

q0 q1B1 :

ba

a

b

r0 r1 r2B2 :

b

a

a

a

b

b

a

• ababaω ∈ L(B1)

• ababaω ∈ L(B2)

• (ab)ω ∈ L(B1)

• (ab)ω /∈ L(B2)

• abababω /∈ L(B1)

• abababω /∈ L(B2)

73 / 243

Operations on Buchi automata: Union

Given two NBAs B1 and B2, there exists an NBA B such that

L(B) = L(B1) ∪ L(B2) and |B| ∈ O(|B1|+ |B1|)

q0 q1B1 :

ba

a

b

r0 r1 r2B2 :

b

a

a

a

b

b

a

B = B1 ∪ B2

74 / 243

Operations on Buchi automata: Intersection


L(B) = L(B1) ∩ L(B2) and |B| ∈ O(|B1| · |B1|)

The intersection is simpler with generalized Buchi automata

75 / 243

Generalized Buchi automata

A nondeterministic generalized Buchi automaton with k acceptingsets is a tuple B = (Q, I , ρ,F) where




• F = Fj ⊆ Q | j ∈ 1, . . . , k is the set of k sets ofaccepting states

76 / 243

Examples of generalized Buchi automata

q0

1

q1

2

B1 :

b

a

a

b

r0

1

r1

2

r2B2 :

b

a

a

a

b

b

a

77 / 243

Semantics of generalized Buchi automata

Given w = a0a1 . . . ∈ Σω, a run π of B on w is an infinitesequence of states π = q0q1 . . . such that q0 ∈ I and for all i ∈ N,qi+1 ∈ ρ(qi , ai )

A run π = q0q1 . . . is accepting if Inf(π) ∩ F 6= ∅ for each F ∈ F

A word w ∈ Σω is accepted by B if there exists an accepting run πon w

The language of B is the set of all accepted words:L(B) = w ∈ Σω | B has an accepting run on w

78 / 243

Examples of generalized Buchi automata

q0

1

q1

2

B1 :

b

a

a

b

r0

1

r1

2

r2B2 :

b

a

a

a

b

b

a

• ababaω /∈ L(B1)

• ababaω /∈ L(B2)

• (ab)ω ∈ L(B1)

• (ab)ω /∈ L(B2)

79 / 243

Buchi automata vs. generalized Buchi automata

Each Buchi automaton is trivially a generalized Buchi automaton

B = (Q, I , ρ,F ) B′ = (Q, I , ρ,F = F)

Are generalized Buchi automata more powerful than Buchiautomata?

80 / 243

Converting generalized Buchi automata to Buchi automata

Given a generalized Buchi automaton B = (Q, I , ρ,F) withF = F1, . . . ,Fk, it is equivalent to the Buchi automatonB′ = (Q ′, I ′, ρ′,F ′) where

• Q ′ = Q × 1, . . . , k• I ′ = I × 1

• ρ′((q, j), a) =

ρ(q, a)× j if q /∈ Fj

ρ(q, a)× (j mod k) + 1 if q ∈ Fj

• F ′ = F1 × 1

81 / 243

Converting generalized Buchi automata to Buchi automata

q0

1

q1

2

b

a

a

b

q0, 1 q1, 1

q0, 2 q1, 2

b

a

ab

ba

a

b

82 / 243

Operations on Buchi automata: Intersection


L(B) = L(B1) ∩ L(B2) and |B| ∈ O(|B1| · |B1|)

Idea: convert NBAs to GBAs, intersect GBAs, convert back toNBA

83 / 243

Operations on generalized Buchi automata: Intersection

Intersection is based on the synchronous product of B1 and B2

Given two GBAs B1 = (Q1, I1, ρ1,F1) and B2 = (Q2, I2, ρ2,F2),their synchronous product B = B1 × B2 is the GBAB = (Q, I , ρ,F) where

• Q = Q1 × Q2

• I = I1 × I2

• ρ((q1, q2), a) = ρ1(q1, a)× ρ2(q2, a)

• F = F1 × Q2 | F1 ∈ F1 ∪ Q1 × F2 | F2 ∈ F2

84 / 243

Operations on generalized Buchi automata: Intersection

q0

1

q1

2

B1 :

b

a

a

b

r0

1

r1

2

r2B2 :

b

a

a

a

b

b

a

q0, r0 q0, r1 q0, r2

q1, r0 q1, r1 q1, r2

b

aa

b

a

b

ab

a

a

a

bb

a

F1 × Q2

F2 × Q2

Q1 × F1 Q1 × F2

B1 × B2 :

85 / 243

Operations on Buchi automata: Emptiness check

Given an NBA B,

check whether L(B) = ∅ in time O(|B|)

Idea: compute the strongly connected components reachable fromthe initial states, and check whether at least one contains anaccepting state

86 / 243

Inclusion checking

Given two NBAs B1 and B2, check whether

L(B1) ⊆ L(B2)

87 / 243

Operations on Buchi automata: Difference


L(B) = L(B1) \ L(B2)

Idea: replace language difference with complementation andintersection, since L(B1) \ L(B2) = L(B1) ∩ L(Bc2)

88 / 243

Operations on Buchi automata: Complementation

Given an NBA B, there exists an NBA Bc such that

L(Bc) = Σω \ L(B)

Ramsey-based approach

• Buchi shows that ω-regular language has the form ∪i∈IUiVωi

• Ui ,Vi are both regular languages, I finite

• Combinatorial approach (Ramsey’s Theorem): thecomplement language is also of this form

• thus the complementation can also be characterized by aBuchi automaton

• complexity 22O(n)

As for NFAs, can determinisation be used for thecomplementation?

89 / 243

Operations on Buchi automata: Complementation

Given an NBA B, there exists an NBA Bc such that

L(Bc) = Σω \ L(B)

Ramsey-based approach

• Buchi shows that ω-regular language has the form ∪i∈IUiVωi

• Ui ,Vi are both regular languages, I finite

• Combinatorial approach (Ramsey’s Theorem): thecomplement language is also of this form

• thus the complementation can also be characterized by aBuchi automaton

• complexity 22O(n)

As for NFAs, can determinisation be used for thecomplementation?

89 / 243

Determinization

Deterministic Buchi automaton is not powerful enough

• Σ∗aω

Thus, Buchi automaton is not closed under determinization.

90 / 243

Why complementing Buchi automata

For termination analysis of a program P

• Synthesize B1, . . . ,Bn, each with a termination argument

• Check L(P) ⊆ L(B1) ∪ · · · ∪ L(Bn)

For proving the connection to MSO.

91 / 243

Automata and Logic Buchi62

MSO ≡ BA. Both MSO and BA define the class of ω-regularexpressions.Proof: From BA to MSO (B → ϕB). Assume B = (Q, q0, ρ,F )with Q = 0, 1, . . . , k and q0 = 0.

• w = a0a1 . . . an−1 ∈ L(B): π = q0q1 . . . qn such that q0 = 0and for all i = 0, . . . , n − 1, qi+1 ∈ ρ(qi , ai ), and qn ∈ F .

• we code states q0, . . . , qn−1 by a tuple (X0, . . . ,Xn−1) ofpairwise disjoint subsets of 0, . . . , n − 1 such that: Xi


• ϕ = ∃X0 . . . ∃Xk(ϕ1 ∧ ϕ2 ∧ ϕ3 ∧ ϕ4)

• ϕ1 = ∧i 6=j∀x¬(Xi (x) ∧ Xj(x))

• ϕ2 = ∀x(first(x)→ X0(x))



92 / 243

Automata and Logic Buchi62

MSO ≡ BA. Both MSO and BA define the class of ω-regularexpressions.Proof: From MSO to BA (ϕ→ Bϕ). Let ϕ(X1, . . . ,Xn) be a MSOformula. We construct an NFA accepting w ∈ Σ× 0, 1nsatisfying ϕ.



• ϕ1 ∧ ϕ2

• ϕ1 ∨ ϕ2

• ¬ψ• ϕ(X1, . . . ,Xn) = ∃Xn+1ψ(X1, . . . ,Xn+1): We have B forψ(X1, . . . ,Xn+1) over Σ×0, 1n+1. Nondeterministicly guessthe sequence defining the n + 1-th additional components,and work on it over like B.

93 / 243

Part III

Learning algorithms for Finite & Buchi

Automata

8 Learning Finite Automata

9 Learning Buchi Automata

94 / 243

DFA & Regular Language

• The regular language

L = u ∈ a, b+ | the number of b in u is 4n + 3

• Regular expression for L:

(a∗.b.a∗.b.a∗.b.a∗).(b.a∗.b.a∗.b.a∗.b.a∗)∗

• DFA M = (Σ,Q, q, δ,F )

q0 q1 q2 q3

a

b

a

b

a

b

a

b

95 / 243

Right Congruence for DFA

For a DFA M, we define x ∼M y iff δ(q, x) = δ(q, y)

• The relation ∼M is an equivalence relation.

• Some states are irrelevant for the accepted language

• L(M) is the union of

96 / 243

Right Congruence for RE

For a language L, we define a relation x vL y such that for eachv ∈ Σ∗, xv ∈ L⇔ yv ∈ L

• The relation ∼L is an equivalence relation.

• Some equivalence classes are irrelevant for L

• L is the union of

97 / 243

Bisimulation & Σ∗a

98 / 243

2n & Σ∗aΣn

99 / 243

Right Congruence

• A relation R is a right congruence over Σ∗ if x R y impliesxv R yv for all v ∈ Σ∗

• A regular language L is recognised by R if it can be written asa union of sets of the form [u].

100 / 243

Myhill-Nerode Theorem Myhill’57 & Nerode’58

The following statements are equivalent:

1 L is a regular language on Σ

2 there exists a right congruence relation over Σ∗ such that ithas finitely many equivalent classes, and L can be expressedas a union of some of the equivalences

3 ∼L has finitely many equivalent classes

Moreover, for regular language, |Σ∗/∼L| equals the number of

states of the smallest DFA recognizing L.

101 / 243

Access String

For a given target (minimal) DFA M, we have:

• Access string: M[x ] := δ(q, x)

• we use the access string x to access the state M[x ]

• in general, many access strings access the same state

• Distinguishing string: if xv /∈ L and yv ∈ L or vice versa

• two access strings x , y access different states if such v exists

102 / 243

Syntactic DFA Nerode

Given a regular language L, a syntactic DFA M of L is defined as:

• consider function tL : Σ∗ → F,TΣ∗ , defined bytL(u)(v) = L(uv)

• tL(u) corresponds to the residual language after reading u

• states can be considered as the image of tL(u) | u ∈ Σ∗• δ(tL(u), a) =

We know M is finite, but the domain Σ∗ is infinite.

• M = (Σ,Σ∗/vL, [ε]vL

, δ), where δ([u]vL, a) = [ua]vL

for allu ∈ Σ∗ and a ∈ Σ

103 / 243

Syntactic DFA Nerode

Given a regular language L, a syntactic DFA M of L is defined as:

• consider function tL : Σ∗ → F,TΣ∗ , defined bytL(u)(v) = L(uv)

• tL(u) corresponds to the residual language after reading u

• states can be considered as the image of tL(u) | u ∈ Σ∗• δ(tL(u), a) =

We know M is finite, but the domain Σ∗ is infinite.

• M = (Σ,Σ∗/vL, [ε]vL

, δ), where δ([u]vL, a) = [ua]vL

for allu ∈ Σ∗ and a ∈ Σ

103 / 243

Approximation by Observation Table Gold Automatica’72

• We maintain an observation table: T : (S ∪ SΣ)→ F,TE ,where S is prefix closed

• T is closed and consistent

ε bab

ε F Fb F Ta F F

ba F Tbb F F

⇒ε b

a

b

a

b

104 / 243

Approximation by Observation Table Gold Automatica’72

• We maintain an observation table: T : (S ∪ SΣ)→ F,TE ,where S is prefix closed

• T is closed and consistent

• if not closed: move sa above

• if not consistent: add a distinguishing string

Lemma (Gold)

For S1 ⊆ S2 . . . and E1 ⊆ E2 . . ., both in the limit equating to Σ∗,it holds that there exists an i such that the automaton derivedfrom (Sj ,Ej) is isomorphic to target automaton M.

105 / 243

L* based on Observation Table

Lemma (Gold)

For S1 ⊆ S2 . . . and E1 ⊆ E2 . . ., both in the limit equating to Σ∗,it holds that there exists an i such that the automaton derivedfrom (Sj ,Ej) is isomorphic to target automaton M.

• index i now known

• Arbib & Zeiger Automatica’69: makes an assumption |M| ≤ n

• Angluin Infor.&Control’81: shows that with this assumption iis bounded (exponentially)

• Angluin I&C’87: another assumption, equivalence query• YES: done• NO: provides a counterexample, use the counterexample to

update the table

• Rivest & Schapire I&C’93: improved version, andnon-restarting scenario with homing sequence

106 / 243

Overview of the L* learning framework for DFAs

w ∈? L

L(C ) =? L

DFA TeacherDFA Learner

e1 e2 · · ·v1 0 1 · · ·v2 0 0 · · ·v3 1 1 · · ·

......

w1 · · ·w2 · · ·w3 · · ·

......

Observation table MQ(w)

yes/no

EQ(C )

noCE: w ∈ L L(C ) yes

automaton C

107 / 243

Example

Target language isL = u ∈ a, b+ | the number of b in u is 4n + 3

ε

ε Fa Fb F

⇒ ε

a

b

For a counterexample bbab ∈ L: we find a new experiment bab todistinguish ε and b

ε bab

ε F Fb F Ta F F

ba F Tbb F F

⇒ε b

a

b

a

b

108 / 243

Example

We again receive bbab as the counterexample and find ε and bbcan be distinguished by ab

ε bab ab

ε F F Fb F T F

bb F F Tbbb T F F

a F F Fba F T F

bba F F Tbbba T F Fbbbb F F F

⇒ ε b bb bbb

a

b

a

b

a

b

a

b

109 / 243

L* based on Classification Trees Kearns & Vazirani’94

ε bab ab

ε F F Fb F T F

bb F F Tbbb T F F

a F F Fba F T F

bba F F Tbbba T F Fbbbb F F F

⇒

ε

bab bbb

ab b

ε bb

110 / 243

L* based on Classification Trees

1 Root is labelled with ε, and one of the leaf node should be ε

2 A tree T induces a DFA

3 A tree induces equivalent classes over the states of the targetautomaton

4 Use counterexample for refinement

111 / 243


ε

ε bbabε bbab

a

b

a

b

• A tree induces a DFA

• Property of the initial automaton: all accepting states arerepresented by one state, non-accepting states are representedby another state.

112 / 243


ε

ε bbabε bbab

a

b

a

b

• A tree induces a DFA

• Property of the initial automaton: all accepting states arerepresented by one state, non-accepting states are representedby another state.

112 / 243

A tree induces equivalent classes over the states of the targetautomaton

• for each string s: one can walk down the tree withmembership queries, and will reach a bottom string t

• state t represents all such strings• transitions are constructed by transitions from the

representations

ε

ε bbabε bbab

a

b

a

b

ε b bb bbab

a

b

a

b

a

b

a

b

113 / 243

How is the automaton related to the target minimal DFA?

ε

ε bbabε bbab

a

b

a

b

ε b bb bbab

a

b

a

b

a

b

a

b

114 / 243

Counterexample based refinement

Let M be the target minimal DFA, M the current automaton.

• A counterexample is a string γ ∈ Σ∗ such that when playedon M and M, exactly only one of them accepts γ.

• Note since ε is an access string, the starting states aresynchronized

• Find the smallest prefix γ[i ] resulting in different states• M[γ[i ]] denotes the state in the current automaton: it can be

obtained easily• M[γ[i ]] denotes the state in the original automaton: whether it

is represented by M[γ[i ]]?

115 / 243

Counterexamples

Let M denote the target minimal DFA, and M denote the currentautomaton.

• γ[i − 1] is a new access string, it should be separated fromstring M[γ[i − 1]]

• the distinguishing string is γid where d is the distinguishingstring for M[γ[i ]] and M[γ[i ]]

116 / 243


ε

ε bbabε bbab

a

b

a

b

ε b bb bbab

a

b

a

b

a

b

a

b

• Counterexample babb: accepting in M, but rejecting in M

• babb is the smallest prefix, thus bab the new access string.The distinguishing string is b.

117 / 243




Experiment b can distinguish ε and bab

ε

b bbab

ε bab

⇒ ε bab bbab

a

b

a

b

a

b

• Still counterexample babb: accepting in M, but rejecting in M

• bab is the smallest prefix: reach access string bab, but ε in M.

• thus ba the new access string. The distinguishing string is bb.

118 / 243




Experiment b can distinguish ε and bab

ε

b bbab

ε bab

⇒ ε bab bbab

a

b

a

b

a

b




118 / 243





Experiment ab can distinguish ε and bbε

b bbab

bb bab

ε ba

⇒ ε ba bab bbab

a

b

a

b

a

b

a

b

119 / 243

Myhill-Nerode is the key of L*

120 / 243

Part III

Learning algorithms for Finite & Buchi

Automata

8 Learning Finite Automata

9 Learning Buchi Automata

121 / 243

Buchi Automata & ω-Regular Expressions

• Buchi Automaton B = (Σ,Q, q, δ,F )

• Our goal is to learn a Buchi automaton recognizing theω-regular language L = Eω withE = u ∈ a, b+ | the number of b in u is 4n + 3

q0start q1 q2 q3

a

b

a

b

a

b

a

b

122 / 243

Buchi Automata & ω-Regular Expressions

• Given an ω-regular language L, the right congruence vL of Lis defined such that x vL y iff ∀w ∈ Σω. xw ∈ L⇐⇒ yw ∈ L.

• Problem: no corresponding Myhill-Nerode theorem.a, b∗aω cannot accepted by a (Buchi) automaton inducedby vL

123 / 243

Ultimately Periodic Words

For an ω-regular language L, let UP(L) denote the set of allultimately periodic words uvω | u ∈ Σ∗, v ∈ Σ+.• Buchi62: For ω-regular languages L, L′, it holds L = L′ iff

UP(L) = UP(L′)

• For LTL model checking problem, it is sufficient to considerUP words.

124 / 243

Learning ω-regular Language

1 Trakhtenbrot’62, Staiger’83: Myhill-Nerode theorem does nothold for ω-regular language.

2 Maler & Pnueli’95: extension to subset of ω-languages wrt.deterministic co-Buchi automata.

3 Arnold’85: A syntactic congruence for ω-languages.

4 Maler & Staiger STACS’93, revision’08: Syntacticcongruences for ω-languages through a family ofright-congruences.

5 Calbrix, Nivat & Podelski MFPS’93: equivalentcharactersation using L$.

6 Angluin & Fisman ALT’14: Learning Lω based on FDFA andrecurrent FDFA.

125 / 243

Family of right-congruences (FORC) Maler & Staiger’93

DefinitionA family of right-congruences (FORC) is a pairR = (∼, ≈u[u]∈Σ∗/∼) such that

1 ∼ is a right-congruence relation on Σ∗,

2 ≈u is a right-congruence relation for every [u] ∈ Σ∗/ ∼,

3 for all u, x , y ∈ Σ∗, x ≈u y implies ux ∼ uy .

An ω-regular language L is recognised by R if it can be written asa union of sets of the form [u]([v ]u)ω such that uv ∼ u.

126 / 243

Family of right-congruences (FORC) Maler & Staiger’93

Definition (Syntactic FORC)

Let L ⊆ Σω, and let u, x , y ∈ Σ∗. For each [u] ∈ Σ∗/∼L, define

• x ≈uS y iff ux ∼L uy and for all v ∈ Σ∗ if uxv ∼L u then

u(xv)ω ∈ L⇔ u(yv)ω ∈ L

The syntactic FORC is defined as (∼L, ≈uS[u]∈Σ∗/∼L

).

Theorem (Myhill-Nerode theorem for ω-languages)

An ω-language is regular iff it is recognized by a finite FORC.Moreover, its syntactic FORC is the coarsest FORC recognising it.

127 / 243

Family of DFAs Angluin & Fisman ALT’14

FDFAs F = (M, Aq) over an alphabet Σ consists of

• a leading automaton M = (Σ,Q, q, δ) and

• progress DFAs Aq = (Σ,Qq, sq, δq,Fq) for each q ∈ Q.

λstart

M a

b

λstart

a

b

a

Aλ a

b

a

b

b

a a

b

Σ∗(aω + bω)

128 / 243

Syntactic FDFAs

Given an ω-regular language L, a syntactic FDFA F = (M, Aq)of L is defined as follows.

• The leading automaton M is the tuple (Σ,Σ∗/vL, [ε]vL

, δ),where δ([u]vL

, a) = [ua]vLfor all u ∈ Σ∗ and a ∈ Σ.

• The progress automaton Au is the tuple(Σ,Σ∗/≈u

S, [ε]≈u

S, δS ,FS), where δS([u]≈u

S, a) = [ua]≈u

Sfor all

u ∈ Σ∗ and a ∈ Σ. The accepting states FS is the set ofequivalence classes [v ]≈u

Sfor which uv vL u and uvω ∈ L.

129 / 243

Canonical FDFAs

Given an ω-regular language L. We define periodic (respectively,syntactic and recurrent) FDFA F = (M, Aq) of L. We define theright congruences ≈u

P ,≈uS , and ≈u

R :

x ≈uP y iff ∀v ∈ Σ∗, u(xv)ω ∈ L⇐⇒ u(yv)ω ∈ L,

x ≈uS y iff ux vL uy and ∀v ∈ Σ∗, uxv vL u =⇒ (u(xv)ω ∈ L⇐⇒ u(yv)ω ∈ L),

x ≈uR y iff ∀v ∈ Σ∗, uxv vL u ∧ u(xv)ω ∈ L⇐⇒ uyv vL u ∧ u(yv)ω ∈ L.

The progress automaton Au is the tuple (Σ,Σ∗/≈uK, [ε]≈u

K, δK ,FK ),

where δK ([u]≈uK, a) = [ua]≈u

Kfor all u ∈ Σ∗ and a ∈ Σ. The

accepting states FK is the set of equivalence classes [v ]≈uK

forwhich uv vL u and uvω ∈ L when K ∈ S ,R and the set ofequivalence classes [v ]≈u

Kfor which uvω ∈ L when K ∈ P.

130 / 243

Learning Algorithm for FDFAs based on Observation Table

Leading DFA Learner L∗M

(x1, y1) (x2, y2) · · ·u1 · · ·u2 · · ·

......

Leading Table

Progress DFA Learner L∗Au1

u1 e1 e2 · · ·v1 · · ·v2 · · ·

......

Progress Table


u2 e1 e2 · · ·v1 · · ·v2 · · ·

......

Progress Table

· · ·

131 / 243

Learning Algorithm for FDFAs based on Classification Trees

Leading DFA Learner L∗M

...

(x , y) ...

u1 u2

Leading Tree


...

e ...

v1 v2

u1

Progress Tree


...

e ...

v1 v2

u2

Progress Tree

· · ·

For syntactic FDFA , the progress trees are K -ary trees.132 / 243

Learning Buchi Automata via FDFA TACAS’17

Mem

ber

Eq

uivalen

ce

FDFA learner FDFA teacher

BA

teacher

Table-based

Tree-based

• PeriodicFDFA

• SyntacticFDFA

• RecurrentFDFA

FDFA F to BA B

• Under-Approx. B

• Over-Approx. B

Analyze CE

• Under-Approx. B

• Over-Approx. B

F

MemFDFA(u, v) MemBA(uvω)

yes/no

EquFDFA(F ) EquBA(B)

yes

Output a BA recognizing the target language

no + uvωno +(u′, v ′)

133 / 243

Counterexample Analysis for FDFA Learner

• Positive counterexample uvω: uv ∼M u, uvω ∈ L and (u, v) isnot accepted by F .

• Negative counterexample uvω: uv ∼M u, uvω 6∈ L and (u, v)is accepted by F .

L

F

uvω

uvωuvω

134 / 243

Why not Build a Precise Buchi Automaton

We have UP(F) =⋃∞

n=0a, b∗ · (abn)ω for followingnon-canonical FDFA F . We assume that UP(F) characterizes anω-regular language L. We can show that the right congruence ≈εPof a periodic FDFA of L is of infinite index. Observe thatabk 6≈εP abj for any k , j ≥ 1 and k 6= j , becauseε · (abk · abk)ω ∈ UP(F) and ε · (abj · abk)ω /∈ UP(F). It followsthat ≈εP is of infinite index.

εstart

Ma

b

εstart a

b

Aεa

b

b

aa b

135 / 243

Approximating Ultimately Periodic Words of FDFA

Let F = (M, Au) be an FDFA where M = (Σ,Q, q, δ) andAu = (Σ,Qu, su,Fu, δu) for every u ∈ Q. Then

UP(F) =⋃

u∈Q,v∈Fu

L(Mqu ) · N(u,v)

where A(u,v) = vω | uv vM u ∧ v ∈ L((Au)suv ).We approximate UP(F) by approximating A(u,v):

• Over-Approximation. N(u,v) = L(P(u,v))ω where

P(u,v) = (Σ,Qu,v , su,v , fu,v, δu,v ) = Muu × (Au)suv .

• Under-Approximation. N(u,v) = L(P(u,v))ω whereP(u,v) = Mu

u × (Au)suv × (Au)vv .

136 / 243

Approximating Ultimately Periodic Words of FDFA

εstart

M a

b

εstart a

Aε

a, b

a

b

In the example, we can see that bω ∈ UP(F) whilebω /∈ UP(L(B)).

q0start q1 q2

q′2

Ba

b

ε a, b

a

b εε

q0start q1 q2

q3

q′2

q4

Ba

b

ε a

b

a

b

ε

ab

a, b

ε

137 / 243

Counterexample Analysis for FDFA Teacher

• Target L = aω + bω, the conjectured FDFA F depicted below.

• Suppose the BA teacher returns a negative counterexample(ba)ω.

• (ba, ba) is accepted by F while (bab, ab) is not.

• the FDFA teacher has to find a decomposition of (ba)ω thatF accepts.

εstart

M a

b

εstart a

Aε

a, b

a

b

138 / 243


For a given F , we define:

• an FA D1 withL(D1) = u$v | u ∈ Σ∗, v ∈ Σ∗, uv vM u, v ∈ L(AM(u)),and

• an FA D2 withL(D2) = u$v | u ∈ Σ∗, v ∈ Σ∗, uv vM u, v /∈ L(AM(u)).

For uvω, an FA Du$v withL(Du$v ) = u′$v ′ | u′ ∈ Σ∗, v ′ ∈ Σ+, uvω = u′v ′ω.

139 / 243


• counterexamples for under-approximations

LB

F

uvω

uvωuvω

140 / 243


• counterexamples for over-approximations

LF

B

uvω

uvω

uvω

141 / 243


tradeoff:

• Under-approximation is complete in dealing with spuriouscountereexamples.

• Over-approximation may not terminate, but is smaller.

142 / 243

Experimental Results

We implemnent a library to learn ω-regular language ROLL(Regular Omega Language Learning)http://iscasmc.ios.ac.cn/roll/

Models L$ LPeriodic LSyntactic LRecurrent

Struct.&Approxi.

Table TreeTable Tree Table Tree Table Tree

under over under over under over under over under over under over

#Unsolved 4 2 3 0/2 2 0/1 1 4*/5 0 3*/3 1 0/1 1 0/1#St. 3078 3078 2481 2468 2526 2417 2591 2591 2274 2274 2382 2382 2400 2400#Tr. 10.6k 10.3k 13.0k 13.0k 13.4k 12.8k 13.6k 13.6k 12.2k 12.2k 12.7k 12.7k 12.8k 12.8k#MQ 105k 114k 86k 85k 69k 67k 236k 238k 139k 139k 124k 124k 126k 126k#EQ 1281 2024 1382 1351 1950 1918 1399 1394 2805 2786 1430 1421 3037 3037Timeeq(s) 146 817 580 92 186 159 111 115 89 91 149 149 462 465Timetotal(s) 183 861 610 114 213 186 140 144 118 120 175 176 499 501EQ(%) 79.8 94.9 95.1 80.7 87.3 85.5 79.3 79.9 75.4 75.8 85.1 84.6 92.6 92.8Mem(MB) 25k 28k 25k 24k 26k 25k 26k 26k 26k 26k 25k 24k 28k 27k

143 / 243

http://iscasmc.ios.ac.cn/roll/

Part IV

Applications

10 Complementation


12 Probabilistic Model CheckingDiscrete time Markov chainsLogicsPCTL Model CheckingPCTL for MDPs

13 PLTL Model CheckingLearning Based Probabilistic Model Checking

144 / 243

Determinization

Buchi automaton is not closed under determinization. Example.

Thus subset construction fails. Consider the automaton acceptingΣ∗, all leading to an absorbing accepting state qf .

• in the subset construction: a state is a set of states, referredto as a macrostate.

• a run visits an accepting states infinitely often

• but no infinitely run exists in the original automaton.

145 / 243

Determinization

Buchi automaton is not closed under determinization. Example.

Thus subset construction fails. Consider the automaton acceptingΣ∗, all leading to an absorbing accepting state qf .

• in the subset construction: a state is a set of states, referredto as a macrostate.

• a run visits an accepting states infinitely often

• but no infinitely run exists in the original automaton.

145 / 243

Determinization based Construction

More general accepting conditions are proposed

• Muller63: Muller condition, Inf(π) ∈ F

• Rabin69,72: Rabin condition, for some i , Inf(π) ∩ Ri = ∅ andInf(π) ∩ Ai 6= ∅ with F = (Ri ,Ai ) | i = 1, 2, . . . , k

• Street82: Street condition, for all i , Inf(π) ∩ Ri 6= ∅ orInf(π) ∩ Ai = ∅ with F = (Ri ,Ai ) | i = 1, 2, . . . , k

LemmaNondeterministic Buchi, Rabin, Street, Muller all recognize thesame class of ω-regular languages. Moreover, nondeterministicRabin, Street, Muller are closed under determinization.

146 / 243

Determinization based Construction

• McNaughton66: transform Buchi into deterministic Muller,with improvements by many, size 2O(n2)

• Safra88: transform Buchi into deterministic Rabin, nO(n)

• Vardi et al.01,06: Ranking based approach, O((0.96n)n)

• Qiqi Yan ICALP’06: lower bound example Ω((0.76n)n)

• Sven Schewe STACS’09: improved algorithm to meet thelower bound

147 / 243

Safra’s complementation approach

Let B = (Q, I , ρ,F ) be a Buchi automaton.

• run subset construction

• a thread of macrostate is split off whenever final states areencountered

• organized in trees, handled simultaneously using subset• if final states f1 . . . , fn are encountered, introduce f1, . . . , fn

as its new son• horizontal merge: delete q if it exists in its older brothers.

Thus: sons are disjoint, subset of their parents• vertical merge: if the union of sons is the same as the parent,

delete all sons. We reach a breakpoint

148 / 243



• states: set of Safra trees

• initial state I

• transition ρ(T , a)• for any macro state in T , add a new son if needed• applying subset construction to all nodes of T• apply horizontal and vertical merges• label macrostate with ! if it reaches a breakpoint

• F is an accepting set if all trees in it contain the samemacrostate which is marked with !.

149 / 243



• McNaughton’s theorem 66: NBA can be transformed intodeterministic Muller automaton.

• Rabin’s accepting pairs bounded by O(|Q|).

• (Rk ,Ak): Rk contains trees without node k, Ak trees with!-labelled node k

150 / 243

Some new results about Buchi automata

• Frantisek Blahoudek, Matthias Heizmann, Sven Schewe, JanStrejcek, Ming-Hsien Tsai: Complementing Semi-deterministicBuchi Automata. TACAS 2016: 770-787

• Semi: parts starting from accepting states are deterministic

• faster direct construction

• simpler subset construction (N,C , S ,B)

• Joel Allred and Ulrich Ultes-Nitsche, LICS’18: A Simple andOptimal Complementation Algorithm for Buchi Automata

151 / 243

Learning the complementation VMCAI’17


• we apply our learning algorithm to build the complementation

• membership is trivial

• equivalence query: we adapt the algorithm for the teacher

152 / 243

Learning framework for Buchi complementation

uvω /∈? L(B)

L(B(F)) ∩ L(B) =? ∅

L(B(Fc))⊆? L(B)CE

anal

ysis

Buchi Teacher

complement B(F)

Buchi LearnerF

DFA

lear

ner

MQ(u, v)

yes/no

EQ(F)

noxyω yes

noxyω

yes

CE: (u, v)

153 / 243

Experimental evaluation

• Comparison between GOAL, SPOT, and Buechic oncomplementing Buchi Store

• learning complement automata is working well in practice

• double complementation: we gained advantage over thecompetitor algorithms, when the complement automata werelarge

• for semi-deterministic automata: we are competitive with thespecialised method for SNBA

BlockExperiments GOAL

Buechic SPOT(States, Transitions) Ramsey Determinisation Rank Slice

1287 NBAs

(928, 2071)

|Q| 21610 3919 21769 4537 2428 1629|ρ| 964105 87033 179983 125155 35392 13623tc 992 300 203 204 105 6

25 NBAs

(55, 304)

|Q|–to–

926 38172 1541 165 495|ρ| 21845 384378 50689 5768 4263tc 28 42 12 474 <1

32 NBAs(20, 80)

|Q|–to– –to–

27372 11734 96 2210|ρ| 622071 1391424 6260 102180tc 56 152 7 1

154 / 243

Part IV

Applications

10 Complementation




155 / 243

Program Termination

Termination problem: we requre that a terminating tool returnsanswers that are correct, but we donot neccessarily require ananswer.

• trivial to build a tool: returns unknown simply.

• goal: keeping the unknown answers as low as possible

• Turing49: classical approach for proving termination• termination argument search• termination argument checking (easy)

156 / 243

Program Termination

Challenge:

• hard to find a single ranking function

• often forced to use ranking function into complex well-orders

program fun( ):



`3: x := x-1

`4: y := y+1

`5: else

`6: y := y-1

`7: fi

`8: done

157 / 243

Turings’s Classic Method Turing’49

Idea: map from program into a program known to terminate, suchthat first program has analogous step in the send one.

• A binary relation R on X is well-founded if every subset of Xhas a minimal element

• Equivalently: ontains no countable infinite descending chains

• To prove a program’s transition relation R is well-founded• find a ranking function (progress measure) f from program

state to a well-order (S , >)• f is a termination argument for the set

T = (s, t) | f (s) > f (t)• prove the program transition relation R is a subset of it:

R ⊆ T• Here: well-order (S , >): total order > is a total order, and a

well-founded relation, such as (N, >)

158 / 243

Transition Invariant Podelski & Rybalchenko LICS’04

Challenge:



program fun( ):



`3: x := x-1

`4: y := input()

`5: else

`6: y := y-1

`7: fi

`8: done

159 / 243

Transition Invariant Podelski & Rybalchenko LICS’04

Challenge:



• use a set of ranking functions (disjunctive terminationargument)

• termination argument checking becomes complex: anynumber of unrolling should be considered

• exploit assertion checking techniques/tools

• or finding disjunctive termination argument: correct byconstruction

• logical foundation for size-change graphs

160 / 243

Program Termination Analysis Heizmann, Hoenicke & Podelski CAV’14

`1

`2

`3

`4

`5

i>0

j:=1

j<ij++

j>=i

i--

Consider the CFG of P as a Buchi automaton, and derivetermination

• Synthesize B1, . . . , Bn,each with a termination argument

• Check L(P) ⊆ L(B1) ∪ · · · ∪ L(Bn)

• Usually, construct Q0, . . . , Qn+1 so that• L(Q0) = L(P)• L(Qi ) = L(Qi−1) \ L(Bi ) = L(Qi−1) ∩ L(Bci )• L(Qn) = ∅

161 / 243


How to show that this program terminates

program sort(int i):

`1: while (i>0):

`2: int j:=1

`3: while (j<i):

// if (a[j]>a[i]):

// swap(a[j],a[i])

`4: j++

`5: i--

162 / 243


`1

`2

`3

`4

`5

i>0

j:=1

j<ij++

j>=i

i--

How to show that this program terminates

program sort(int i):

`1: while (i>0):

`2: int j:=1

`3: while (j<i):

// if (a[j]>a[i]):

// swap(a[j],a[i])

`4: j++

`5: i--

162 / 243

Analyzing Executions

Given a CFG, extract a single lasso execution

`1

`2

`3

`4

`5

i>0

j:=1

j<ij++

j>=i

i--

163 / 243


q1

q2

q3

q4

i>0

j:=1

j<ij++

Given a CFG, extract a single lasso execution

`1

`2

`3

`4

`5

i>0

j:=1

j<ij++

j>=i

i--

163 / 243


Assign a ranking function to the single execution

q1

q2

q3

q4

i>0

j:=1

j<ij++

164 / 243


q1 oldrnk =∞

q2 oldrnk =∞

q3 i − j < oldrnk

q4 0 ≤ i − j ≤ oldrnk

i>0

j:=1

j<ij++

Assign a ranking function to the single execution frnk = i − j

q1

q2

q3

q4

i>0

j:=1

j<ij++

164 / 243


Generalize the automaton: merging states with the same labelling

q1 oldrnk =∞

q2 oldrnk =∞

q3 i − j < oldrnk


i>0

j:=1

j<ij++

165 / 243


q1 oldrnk =∞

q3 i − j < oldrnk


i>0

j:=1

j<ij++

Generalize the automaton: merging states with the same labelling

q1 oldrnk =∞

q2 oldrnk =∞

q3 i − j < oldrnk


i>0

j:=1

j<ij++

165 / 243


Generalize the automaton

q1 oldrnk =∞

q3 i − j < oldrnk


i>0

j:=1

j<ij++

166 / 243


q1 oldrnk =∞

q3 i − j < oldrnk


Σ

Σ

j0

Generalize the automaton

q1 oldrnk =∞

q3 i − j < oldrnk


i>0

j:=1

j<ij++

166 / 243

Buchi automaton with rank Certificate

• one initial state, one final state

• each time the final state is reached, the value of the rankingfunction is decreased

• all runs in the automaton terminate

• it covers all runs (OUTER + INNER)∗INNERω

What are the remaining traces?

• in control flow refinement, a multi-path loop can bedecomposed:

(a + b)+ = (b∗ab∗)+ + b+

• similarly,

(a + b)ω = (a + b)∗(b∗ab∗)ω + (a + b)∗b+

• thus the CFG can be bounded by the two Buchi automata

167 / 243

Buchi automaton with rank Certificate

• one initial state, one final state

• each time the final state is reached, the value of the rankingfunction is decreased

• all runs in the automaton terminate

• it covers all runs (OUTER + INNER)∗INNERω

What are the remaining traces?

• in control flow refinement, a multi-path loop can bedecomposed:

(a + b)+ = (b∗ab∗)+ + b+

• similarly,

(a + b)ω = (a + b)∗(b∗ab∗)ω + (a + b)∗b+

• thus the CFG can be bounded by the two Buchi automata167 / 243

Buchi automaton with rank Certificate PLDI’18

• Correctness holds if the following inclusion holds

L(P) ⊆ L(B1) ∪ · · · ∪ L(Bn)

• despite the expensive algorithm, it performs already quite well

• bottleneck: the complementation algorithm

• multi-layer construction, and exploit semi-deterministicautomata

168 / 243


Generalize the automaton: deterministic automaton

q1 oldrnk =∞

q3 i − j < oldrnk


i>0

j:=1

j<ij++

169 / 243


q1 oldrnk =∞

q3 i − j < oldrnk


∅true

Σ

j++ j0

j:=1

i>0j0

j:=1

j<ij++

169 / 243


Generalize the automaton: semideterministic automaton

q1 oldrnk =∞

q3 i − j < oldrnk


i>0

j:=1

j<ij++

170 / 243


q1 oldrnk =∞

q1, q40 ≤ i − j ≤ oldrnk =∞

q3 i − j < oldrnk


∅true

j++ j:=1 i>0

Σ

j<ij0

j++

j:=1

Σ

j++ j0

j:=1

i>0j0

j:=1

j<ij++

170 / 243

Part IV

Applications

10 Complementation




171 / 243

Why probability is important: reliability

Zeroconf protocol objectives

• network protocol for address assignment

• new devices joining the network get a unique IP address

• no user interaction needed

Zeroconf protocol overview

1 randomly choose one of the 65 024 addresses available in theprivate B-class 169.254.0.0/16

2 Loop: as long as the number of sent probes is less than n

3 broadcast the probe message “who is using the chosenaddress”?

4 got a reply? Go to 1

5 no reply within r > 0 time units:• if n probes have been sent: use the address• otherwise go to 2 172 / 243


A simplified model for the Zeroconf protocol is:

s0 s1 s2 . . . sn−1 sn

ok err

1− qq p

1− p

p

1− p

p

1− p

p

1− p

p

1− p

q: probability of choosing an address already in use,q = #devices

65024

p: probability of message loss

173 / 243


s0 s1 s2 . . . sn−1 sn

ok err

1− qq p

1− p

p

1− p

p

1− p

p

1− p

p

1− p

What is the probability that

• an IP address is eventually obtained?

• an unused IP address is eventually obtained?

• an already in use IP address is eventually obtained?

174 / 243

Discrete time Markov chains

A (Discrete time) Markov chain (MC) is a tuple M = (S , s, L,P)where

• S is a finite set of states

• s is the initial state

• L : S → Σ is a labelling function

• P : S × S → [0, 1] is the transition probability matrix

P is such that∑

s′∈S P(s, s ′) ∈ 0, 1 for each s ∈ S .

175 / 243

Example of Markov chain

s0

s123 s456

s ′123s23 s45 s ′456

s1 s2 s3 s4 s5 s6

12

12

12

12

12

12

12

12

12

12

12

12

12

12

M = (S , s, L,P)

176 / 243

Computing probabilities

s0

s123 s456

s ′123s23 s45 s ′456

s1 s2 s3 s4 s5 s6

12

12

12

12

12

12

12

12

12

12

12

12

12

12

What is the probability of finally reaching the state s2?

P(s0 s123 s23 s2 )

+ P(s0 s123 s ′123 s123 s23 s2 )

+ P(s0 s123 s ′123 s123 s ′123 s123 s23 s2 )

+ P(s0 s123 s ′123 s123 s ′123 s123 s ′123 s123 s23 s2 )

+ P(s0 s123 s ′123 s123 s ′123 s123 s ′123 s123 s ′123 s123 s23 s2 )

+ P(s0 s123 s ′123 s123 s ′123 s123 s ′123 s123 s ′123 s123 s ′123 s123 s23 s2 )

. . .

=∞∑n=0

P(s0 s123 (s ′123 s123)n s23 s2 )

177 / 243


s0

s123 s456

s ′123s23 s45 s ′456

s1 s2 s3 s4 s5 s6

12

12

12

12

12

12

12

12

12

12

12

12

12

12


P(s0 s123 s23 s2 )

+ P(s0 s123 s ′123 s123 s23 s2 )

+ P(s0 s123 s ′123 s123 s ′123 s123 s23 s2 )

+ P(s0 s123 s ′123 s123 s ′123 s123 s ′123 s123 s23 s2 )

+ P(s0 s123 s ′123 s123 s ′123 s123 s ′123 s123 s ′123 s123 s23 s2 )

+ P(s0 s123 s ′123 s123 s ′123 s123 s ′123 s123 s ′123 s123 s ′123 s123 s23 s2 )

. . .

=∞∑n=0

P(s0 s123 (s ′123 s123)n s23 s2 )

177 / 243

Markov Chains

• analysis of systems that exhibiting probabilistic behaviour

• randomized algorithms, protocols, modelling system failure(fault trees)

• properties: probabilistic termination, expected duration ofcertain events

• Markov (memoryless): the future states depend only on thecurrent state, not on its past states

178 / 243


s0

s123 s456

s ′123s23 s45 s ′456

s1 s2 s3 s4 s5 s6

12

12

12

12

12

12

12

12

12

12

12

12

12

12

How can we compute P(s0 s123 s23 s2 )?

Intuitively, it is

P(s0 s123 s23 s2 ) = P(s0, s123) · P(s123, s23) · P(s23, s2 )

Formally, it is

Probability of a path

A finite path ξ is a finite sequence of states ξ = s0s1s2 . . . sn suchthat for each 0 ≤ i < n, P(si , si+1) > 0.

The probability P(ξ) of ξ is defined as P(ξ) =∏n−1

i=0 P(si , si+1).

179 / 243


s0

s123 s456

s ′123s23 s45 s ′456

s1 s2 s3 s4 s5 s6

12

12

12

12

12

12

12

12

12

12

12

12

12

12

How can we compute P(s0 s123 s23 s2 )?

Intuitively, it is

P(s0 s123 s23 s2 ) = P(s0, s123) · P(s123, s23) · P(s23, s2 )

Formally, it is

Probability of a path

A finite path ξ is a finite sequence of states ξ = s0s1s2 . . . sn suchthat for each 0 ≤ i < n, P(si , si+1) > 0.

The probability P(ξ) of ξ is defined as P(ξ) =∏n−1

i=0 P(si , si+1).

179 / 243


s0

s123 s456

s ′123s23 s45 s ′456

s1 s2 s3 s4 s5 s6

12

12

12

12

12

12

12

12

12

12

12

12

12

12


∞∑n=0

P(s0 s123 (s ′123 s123)n s23 s2 )

=∞∑n=0

1

2· 1

2·(

1

2· 1

2

)n

· 1

2

=1

2· 1

2· 1

2·∞∑n=0

(1

2· 1

2

)n

=1

8·∞∑n=0

(1

4

)n

=1

8· 1

1− 14

=1

8· 1

34

=1

8· 4

3=

1

6

180 / 243

The probabilistic branching time logic PCTL Hansson & Jonsson 94

The logic PCTL expresses properties about the branching structureof the system.Examples:

• with probability 1, an IP address is eventually obtained

• two processes are in the critical section at the same time withprobability 0

• if a process wants to enter the critical section, with probability1 it will eventually enter

181 / 243

Syntax of the PCTL logic

The formal syntax of PCTL is as follows:

ϕ ::= a | ϕ ∧ ϕ | ¬ϕ | P./p[Ψ]

Ψ ::= Xϕ | ϕU ϕ

where a ∈ AP is an atomic proposition, ./ ∈ <,≤,=,≥, >, andp ∈ [0, 1] ∩Q.

ϕ is called a state formula while Ψ a path formula.

Other common operators can be derived:

false = a ∧ ¬a

true = ¬falseϕ1 ∨ ϕ2 = ¬(¬ϕ1 ∧ ¬ϕ2)

ϕ1 → ϕ2 = ¬ϕ1 ∨ ϕ2

Fϕ = true U ϕ

182 / 243

Examples of PCTL formulas

• with probability 1, an IP address is eventually obtainedP=1[FIP]

• two processes are in the critical section at the same time withprobability 0P=0[F(c1 ∧ c2)]

• if a process wants to enter the critical section, with probability1 it will eventually enter∧2

i=1(wi → P=1[Fci ])

183 / 243

Informal semantics of the PCTL logic

a . . .a

Xa . . .a

a U b . . .a ∧ ¬b a ∧ ¬b a ∧ ¬b b

Fa . . .¬a ¬a ¬a a

P./p[Ψ] if P( ξ ∈ Paths | ξ |= Ψ ) ./ p

184 / 243

The probabilistic linear temporal logic PLTL

The logic LTL expresses properties about sequences of events.Examples:

• an IP address is eventually obtained

• it never happens that two processes are in the critical sectionat the same time

• whenever a process wants to enter the critical section, it willeventually enter

The probabilistic extension PLTL considers the probability of suchsequences of events.

185 / 243

Syntax of the PLTL logic

The formal syntax of PLTL is as follows:

ϕ ::= a | ϕ ∧ ϕ | ¬ϕ| Xϕ | ϕU ϕ

PLTL ::= P./p[ϕ]

where a ∈ AP is an atomic proposition, ./ ∈ <,≤,=,≥, >, andp ∈ [0, 1] ∩Q.Other common operators can be derived:

false = a ∧ ¬a

true = ¬falseϕ1 ∨ ϕ2 = ¬(¬ϕ1 ∧ ¬ϕ2)

ϕ1 → ϕ2 = ¬ϕ1 ∨ ϕ2

Fϕ = true U ϕ

Gϕ = ¬F¬ϕ

186 / 243

Examples of PLTL formulas

• an unused IP address is eventually obtained with probability atmost 0.01P≤0.01[FunusedIP]

• with probability 1, it never happens that two processes are inthe critical section at the same timeP=1[G¬(c1 ∧ c2)]

• with probability at least 0.99, whenever a process wants toenter the critical section, it will eventually enterP≥0.99[G(

∧2i=1 wi → Fci )]

187 / 243

Semantics of the PLTL logic

ϕ . . .ϕ

Xϕ . . .ϕ

ϕ1 U ϕ2 . . .ϕ1 ∧ ¬ϕ2ϕ1 ∧ ¬ϕ2ϕ1 ∧ ¬ϕ2 ϕ2

Fϕ . . .¬ϕ ¬ϕ ¬ϕ ϕ

Gϕ . . .ϕ ϕ ϕ ϕ ϕ

P./p[ϕ] if P( ξ ∈ Paths | ξ |= ϕ ) ./ p

188 / 243

Semantics of the PLTL logic: some example

Are these properties satisfied by the following paths?

FGb . . .¬b ¬b ¬b b b · · ·

189 / 243



FGb . . .¬b ¬b ¬b b b · · ·

GFb . . .b ¬b b ¬b b · · ·

189 / 243



FGb . . .¬b ¬b ¬b b b · · ·

GFb . . .b ¬b b ¬b b · · ·

G(a U b) . . .a ∧ ¬b b b a ∧ ¬b b · · ·

189 / 243



FGb . . .¬b ¬b ¬b b b · · ·

GFb . . .b ¬b b ¬b b · · ·

G(a U b) . . .a ∧ ¬b b b a ∧ ¬b b · · ·

GF(a U b) . . .a ∧ ¬b b ¬a ∧ ¬b a ∧ ¬b b · · ·

189 / 243



FGb . . .¬b ¬b ¬b b b · · ·

GFb . . .b ¬b b ¬b b · · ·

G(a U b) . . .a ∧ ¬b b b a ∧ ¬b b · · ·

GF(a U b) . . .a ∧ ¬b b ¬a ∧ ¬b a ∧ ¬b b · · ·

G(Fa ∧ Fb) . . .a ∧ ¬b b ¬a ∧ ¬b a ∧ ¬b b · · ·

189 / 243



FGb . . .¬b ¬b ¬b b b · · ·

GFb . . .b ¬b b ¬b b · · ·

G(a U b) . . .a ∧ ¬b b b a ∧ ¬b b · · ·

GF(a U b) . . .a ∧ ¬b b ¬a ∧ ¬b a ∧ ¬b b · · ·

G(Fa ∧ Fb) . . .a ∧ ¬b b ¬a ∧ ¬b a ∧ ¬b b · · ·

GF(a ∧ Fb) . . .a ∧ ¬b b ¬a ∧ ¬b a ∧ ¬b b · · ·

189 / 243



FGb . . .¬b ¬b ¬b b b · · ·

GFb . . .b ¬b b ¬b b · · ·

G(a U b) . . .a ∧ ¬b b b a ∧ ¬b b · · ·

GF(a U b) . . .a ∧ ¬b b ¬a ∧ ¬b a ∧ ¬b b · · ·

G(Fa ∧ Fb) . . .a ∧ ¬b b ¬a ∧ ¬b a ∧ ¬b b · · ·

GF(a ∧ Fb) . . .a ∧ ¬b b ¬a ∧ ¬b a ∧ ¬b b · · ·

F(Xa) . . .a ¬a ¬a ¬a ¬a · · ·

189 / 243

Mixing PCTL and PLTL: PCTL* Bianco & deAlfaro95

• PCTL is a branching time logic used for stating properties onthe branching structure of a system

• PLTL is a linear time logic used for stating properties on thetemporal behaviour of a system

How can we state properties on both branching and temporalaspects of a system?

Just use a combination of PCTL and PLTL: PCTL*.

190 / 243

Syntax of the PCTL* logic

The formal syntax of PCTL* is as follows:

ϕ ::= a | ϕ ∧ ϕ | ¬ϕ | P./p[Ψ]

Ψ ::= ϕ | Ψ ∧Ψ | XΨ | Ψ U Ψ

where a ∈ AP is an atomic proposition, ./ ∈ <,≤,=,≥, >, andp ∈ [0, 1] ∩Q.

ϕ is called a state formula while Ψ a path formula.

Other common operators can be derived:

false = a ∧ ¬a

true = ¬falseψ1 ∨ ψ2 = ¬(¬ψ1 ∧ ¬ψ2)

ψ1 → ψ2 = ¬ψ1 ∨ ψ2

FΨ = true U Ψ

GΨ = ¬F¬Ψ

191 / 243

Examples of PCTL* formulas

• With probability 1, a state which is followed by an error statewith probability at most 0.01 is reached infinitely oftenP=1[GFP≤0.01[Xerror ]]

• with probability 1, it is never the case that with probabilitygreater than 0.1 an error state is eventually reachedP=1[G¬P≥0.1[Ferror ]]

• with probability at least 0.99, whenever a process wants toenter the critical section, it will enter within 4 steps withprobability at most 0.25P≥0.99[G(

∧2i=1 wi → P≤0.25[ci ∨X(ci ∨X(ci ∨X(ci ∨Xci )))])]

192 / 243

Reachability probabilities in finite MCs

Problem statement

Data: a MC M, a state s, and a set of states G ⊆ S

Aim: compute P(s |= FG ) = Ps(FG )

Remember thatPs(FG ) = Ps( ξ ∈ Paths(M) | ∃n ∈ N : ξ[n] ∈ G )

Characterization of reachability probabilities

• For each state s ∈ S , consider the variable xs representing theprobability of satisfying FG in s, i.e., xs = P(s |= FG )

• For each state s ∈ S ,• if s ∈ G , then xs = 1• if s can not reach G , then xs = 0• for each s ∈ pred∗(G ) \ G ,

xs =∑g∈G

P(s, g)︸︷︷︸reach G in one step

+∑

t∈S\G

P(s, t) · xt︸︷︷︸reach G via t ∈ S \ G

193 / 243

Predecessor states of a MC

Given a MC M, for G(M) and a set of vertices U ⊆ V , the set of

• immediate predecessors of U ispred(U) = v ∈ V | ∃u ∈ U : (v , u) ∈ E

• pred∗ is the reflexive and transitive closure of pred.

194 / 243


s0

s123 s456

s ′123s23 s45 s ′456

s1 s2 s3 s4 s5 s6

12

12

12

12

12

12

12

12

12

12

12

12

12

12

What is the probability of finally reaching the state s2 , i.e.,P(F s2 )?By using the previous construction, we have:

• xs1 = xs3 = xs4 = xs5 = xs6 = 0 and x s2= 1

• xs456 = xs′456= xs45 = 0

• xs23 = 12 · x s2

+ 12 · xs3

• xs′123= 1

2 · xs123 + 12 · xs1

• xs123 = 12 · xs′123

+ 12 · xs23

• xs0 = 12 · xs123 + 1

2 · xs456

By solving the system, we obtainxs23 = 1

2 , xs123 = 13 , xs′123

= 16 , and xs0 = 1

6

195 / 243

Reduced linear equation system

The linear equation system can be reduced by considering onlystates in pred∗(G ) \ G :

• let S? = pred∗(G ) \G be the states not in G that can reach G

• let A = (P(s, t))s,t∈S?be the restriction of P to only states in

S?

• let b = (bs)s∈S?be the probability to reach G in 1 step:

bs =∑

g∈G P(s, g)

Then x = (xs)s∈S?with xs = P(s |= FG ) is the unique solution of

x = A× x + b or, equivalently, (IS?− A)× x = b

196 / 243


s0

s123 s456

s ′123s23 s45 s ′456

s1 s2 s3 s4 s5 s6

12

12

12

12

12

12

12

12

12

12

12

12

12

12

What is the probability of finally reaching the state s2 , i.e.,P(F s2 )?

By using the previous matrix construction, we have:

• S? = s0, s123, s′123, s23

•

1 −1

2 0 00 1 −1

2 −12

0 −12 1 0

0 0 0 1

×

s0

s123

s ′123

s23

=

00012

By Gaussian elimination, we obtainxs23 = 1

2 , xs123 = 13 , xs′123

= 16 , and xs0 = 1

6

197 / 243

Improving reachability probability computation

In practice, S=1 and S=0 should be chosen as large as possible, soto have S? (and the corresponding equation system) as small aspossible:

S=1 = s ∈ S | P(s |= FG ) = 1 andS=0 = s ∈ S | P(s |= FG ) = 0

They can be obtained in linear time by analysing the underlyinggraph G(M).

198 / 243

Iterative computation of reachability probabilities

The reduced linear equation system consists of:• S? = pred∗(G ) \G being the states not in G that can reach G• A = (P(s, t))s,t∈S?

being the restriction of P to only states inS?

• b = (bs)s∈S?being the probability to reach G in 1 step:

bs =∑

g∈G P(s, g)

Then x = (xs)s∈S?with xs = P(s |= FG ) is the unique solution of

x = A× x + b

This gives an iterative approach to compute x:

x(0) = 0 and x(i+1) = A× x(i) + b for i ∈ N.

Then

1 x(n)(s) = P(s |= F≤nG ) for s ∈ S?

2 x(0) ≤ x(1) ≤ x(2) ≤ . . . ≤ x3 x = limn→∞ x(n)

where F≤nG = ξ ∈ Paths(M) | ∃i ≤ n : ξ[i ] ∈ G is thebounded version of FG .

199 / 243

Iterative computation of reachability probabilities

The sequence of approximate solutions x(0) = 0 andx(i+1) = A× x(i) + b for i ∈ N can be obtained by an algorithmcomputing iteratively x(i+1) = A× x(i) + b and stopping when

maxs∈S?

|x (i+1)s − x

(i)s | < ε for some small tolerance ε

This method ensures convergence.

For practical computation, other methods are used, likeGauss-Siedel or Jacobi.

200 / 243

Constrained reachability probabilities in finite MCs

Problem statement

Data: a MC M, a state s, and two sets of states B,G ⊆ S

Aim: compute P(s |= B U G ) = Ps(B U G )

Remember that Ps(B U G ) = Ps( ξ ∈ Paths(M) | ∃n ∈ N :ξ[n] ∈ G ∧ ∀i < n : ξ[i ] /∈ B )

Characterization of reachability probabilities

• For each state s ∈ S , consider the variable xs representing theprobability of satisfying B U G in s, i.e., xs = P(s |= B U G )

• For each state s ∈ S ,• if s ∈ G , then xs = 1• if s can not reach G via F , then xs = 0• for each s ∈ (pred∗(G ) ∩ F ) \ G ,

xs =∑g∈G

P(s, g) +∑

t∈S\G

P(s, t) · xt

In practice, the same constructions and optimizations for FG apply.

201 / 243

PCTL model checking problem

We have now all the ingredients for solving the PCTL modelchecking problem:

PCTL model checking problem

Input: a finite MC M = (S , s, L,P), a state s ∈ S , and a(state) PCTL formula ϕ

Output: yes, if s |= ϕ; no, otherwise

Basic algorithm

The decision about s |= ϕ is made by:

1 Compute the satisfaction set Sat(ϕ) = s ∈ S | s |= ϕ :this is done recursively by a bottom-up traversal of the parsetree of ϕ:

• the nodes of the tree are the sub-formulas of ϕ• for each node, i.e., for each subformula η of ϕ, compute Sat(η)• compute Sat(η) by means of the satisfaction sets of its

children, like:

Sat(η1 ∧ η2) = Sat(η1) ∩ Sat(η2) and Sat(¬η) = S \ Sat(η)

2 Return s ∈ Sat(ϕ)

202 / 243

Core part of the PCTL model checking algorithm

The satisfaction set Sat( · ) is defined by structural induction asfollows:

Sat(a) = s ∈ S | a ∈ L(s) Sat(ϕ1 ∧ ϕ2) = Sat(ϕ1) ∩ Sat(ϕ2)

Sat(¬ϕ) = S \ Sat(ϕ)

Sat(P./p[Ψ]) = s ∈ S | Ps(s |= Ψ) ./ p

where s |= Ψ = ξ ∈ Paths(M) | ξ |= Ψ ∧ fst(ξ) = s .

203 / 243

The X next operator case: P(s |= Xϕ)

The basic result underlying P(s |= Xϕ) is:P(s |= Xϕ) =

∑s′∈Sat(ϕ) P(s, s ′).

This gives the following algorithm:

• consider all states simultaneously

• in matrix representation, it is

(P(s |= Xϕ))s∈S = P× bϕ

where bϕ(t) = 1 if t ∈ Sat(ϕ), 0 otherwise.

In practice, the next operator case reduces to a matrix-vectormultiplication.

204 / 243

Example of the X next operator case

s0 s1

s2

s3

try

fail

succ1

0.1

0.80.1

1

1

Consider the MC on the right andthe PCTL formula P≥0.9[X(¬try ∨ succ)].

We have that

• Sat(succ) = s3• Sat(try) = s1• Sat(¬try) = S \ s1 = s0, s2, s3• Sat(¬try ∨ succ) = Sat(¬try) ∪ Sat(succ) = s0, s2, s3

We know that (P(s |= Xϕ))s∈S = P× bϕ where ϕ = ¬try ∨ succ .This yields to:

(P(s |= Xϕ))s∈S =

0 1 0 00 0.1 0.1 0.81 0 0 00 0 0 1

×

1011

=

0

0.911

Thus, Sat(P≥0.9[X(¬try ∨ succ)]) = s1, s2, s3.

205 / 243

The U until operator case: P(s |= ϕ1 U ϕ2)

The algorithm is as follows:

• compute S=1 = Sat(P=1(ϕ1 U ϕ2)) by graph analysis

• compute S=0 = Sat(P=0(ϕ1 U ϕ2)) by graph analysis

• solve the equation system for S?

• assign states to Sat(P./p[ϕ1 U ϕ2]) accordingly

On the importance of pre-computing using graph analysis

1 reduces the number of variables in the linear equation system

2 ensures the uniqueness of the solution of the linear equationsystem

3 gives exact solutions for states in S=0 and S=1

4 for qualitative properties, no need to solve the linear equationsystem

In practice, the until operator case reduces to simple graph analysisand the solution of a linear equation system.

206 / 243

Complexity analysis of the PCTL model checking algorithm

Let |ϕ| be the size of a (state) PCTL formula, i.e., the number oftemporal and logical operators in ϕ.

Complexity theorem

For a finite MC M and state PCTL formula ϕ, the PCTL modelchecking problem can be solved in time

O(Poly(|S |) · |ϕ|)

Informal analysis

1 The linear complexity in |ϕ| comes from the need to solve themodel checking problem for each node of the parse tree (i.e.,computing Sat(η) for each subformula η of ϕ).

2 The worst-case complexity comes from the U until operator:

1 Computing S=0 and S=1 can be done in linear time.2 Solving the equation system on S? is in Θ(|S?|3).

207 / 243

Markov decision processes

• Markov chains are well suited for simple synchronousprobabilistic systems

• MDPs are extensions of Markov chains with non-deterministicchoices

• concurrency: parallel components with asynchronousexecution

• planning: find optimal decisions

• abstraction

208 / 243

Probabilistic models: Markov decision processes

A Markov decision process (MDP) is a tuple M = (S , s, L,Act,P)where

• S is a finite set of states

• s is the initial state

• L : S → Σ is a labelling function

• Act is a finite set of actions

• P : S × Act × S → [0, 1] is the transition probability matrix

P is such that∑

s′∈S P(s, a, s ′) ∈ 0, 1 for each s ∈ S anda ∈ Act.

209 / 243

Computing probability

s

h

t

beepedfair

12

fair

12unfair

15

unfair

45

beep

What is the probability of having beeped ?

210 / 243

Computing probability by resolving nondeterminism

In order to compute the probability of a path, we need a means toresolve nondeterminism.Nondeterminism is resolved by a scheduler (policy, adversary,strategy, . . . ) on a given path.Paths now include the performed actions.

SchedulerGiven an MDP M, a scheduler is a functionυ : Paths∗(M)→ Dist(Act) such that whenever υ(ξ)(a) > 0, then∑

s′∈S P(last(ξ), a, s ′) = 1.

The probability of a finite path ξ = s0 a1 s1 a2 . . . sn is then definedrecursively as:

P(s0a1s1a2 . . . sn) =

1 if ξ = s,

0 if ξ = s0 6= s,

P(ξ′) · υ(ξ′)(an) · P(last(ξ′), an, sn) if ξ = ξ′ an sn.

211 / 243

Schedulers

• also known as policies, adversaries

• for an MDP, a scheduler induces a Markov chain• unwinding of the original MDP• with states as paths in the MDP

• some of the important sub-classes:• memoryless or with memory• randomisation or deterministic• finite memory

212 / 243

Computing probability

s

h

t

beepedfair

12

fair

12unfair

15

unfair

45

beep

What is the probability of having beeped ?It is P(s fair h beep beeped ) + P(s unfair h beep beeped ).

For the scheduler υ choosing fair in s, it is 12 .

For the scheduler υ choosing unfair in s, it is 15 .

For the scheduler υ choosing uniformly between fair and unfair ins, it is 7

20 .

213 / 243

MDP Reachability Probabilistic

In probabilistic model checking, there are some efficientpre-computations

• the 1-states

• the 0-states

To extend them to MDPs, one needs the notion of endcomponents

• an end component is an sub-MDP such that one can stayinside with probability one

• can be computed efficiently (more complex than SCCcomputations)

• MEC: maximal end components

• the maximal reachability probability is the same for stateswithin the same MEC

214 / 243



• the 1-states

• the 0-states

To extend them to MDPs, one needs the notion of endcomponents

• an end component is an sub-MDP such that one can stayinside with probability one

• can be computed efficiently (more complex than SCCcomputations)

• MEC: maximal end components

• the maximal reachability probability is the same for stateswithin the same MEC

214 / 243


Maximal/minimal reachability:

maxυPυs (FG )

• Bianco& deAlfaro: memoryless deterministic schedulersenough

• finite many schedulers (but exponential)

• Puterman: policy iteration & value iteration

• LP reduction

• LP solver

215 / 243

Part IV

Applications

10 Complementation




216 / 243

Semantics of P

For MDPss |= PJ(ϕ) iff Prυs (ϕ) ∈ J for all υPrυs (ϕ) := Prυs σ ∈ Paths | σ |= ϕ

denote the probability of the set of paths satisfying ϕ

Thus it suffices to compute the maximal and minimal probabilitysupυ PM,υ(ϕ) and infυ PM,υ(ϕ).

217 / 243

Classical Approach

To check s |= PJ(ϕ), the main steps are:

1 translate the formula ϕ into an equivalent DRA A = det(ϕ);2 build (the reachable fragment of) the product automatonM⊗ A;

3 for each bottom SCC S, check whether S is accepting. Let Ube the union of these accepting SCCs;

4 infer the probability PM⊗A(3U).

Challenges

• the complexity of the transformation from LTL to Buchiautomata and then to deterministic Rabin automata is in2O(2n+log2 n)

• the problem itself is complete for double exponential time(Courcoubetis,Yannakakis).

What to do?

• efficient algorithm for the Buchi automata translation

218 / 243

Classical Approach





Challenges



What to do?

• efficient algorithm for the Buchi automata translation

218 / 243

Classical Approach





Challenges



What to do?

• efficient algorithm for the Buchi automata translation218 / 243

Our approach CONCUR’14

1 translate the formula ϕ into an equivalent DRA A = det(ϕ);

2 build (the reachable fragment of) the product automatonM⊗ A;



On-the-fly

• we construct a quotient structure of M⊗ A on-the-fly

• we unwind certain sub-model only if needed

• worst-case complexity?

219 / 243

Why it performs very good?


• the 1-states

• the 0-states

Complexity: linear to the size of the model.

Observation: the size of the product automata is the bottleneck

We exploit it in the following sense:

• we build a property-driven product automata on-the-fly

• so most of the 0 states are not included in this construction

• most of the 1 states can be determined in the cheap quotientstructure

• only few part needs to be fully determinised

220 / 243



• the 1-states

• the 0-states








220 / 243



• the 1-states

• the 0-states








220 / 243

Buchi Automata

• For an LTL formula ϕ, one can construct a Buchi automatonBϕ such that L(ϕ) = L(Bϕ).

Vardi & Wolper 1986

• One can apply off-the-shelf established tools like LTL3BA orSPOT to translate an LTL specification to a TGBA.

Remark

• It is well-known that TGBAs are more expressive than LTLformulas.

• Subset construction does not work for determinisation ofBuchi automata.

221 / 243

Buchi Automata

• For an LTL formula ϕ, one can construct a Buchi automatonBϕ such that L(ϕ) = L(Bϕ).

Vardi & Wolper 1986

• One can apply off-the-shelf established tools like LTL3BA orSPOT to translate an LTL specification to a TGBA.

Remark

• It is well-known that TGBAs are more expressive than LTLformulas.

• Subset construction does not work for determinisation ofBuchi automata.

221 / 243

How does our approach work?

• We apply state-of-the-art tool SPOT to construct the Buchiautomata for ϕ

• We build abstraction automata: subset automata, breakpointautomata, such that

• smaller number of states• admits efficient symbolic data structures

222 / 243

Subset construction

• Rabin states: tree structure, labelled with Buchi statesw , x , y , z

x , y

x

z

∈ A

w , x , y , z

x , y

x

zm,

∈M⊗ A

• subsume states to quotient statew , x , y , z

x , y

x

zm,

w , x , y , z

x , zm, m, w , x , y , z

• obtain quotient model from Buchi automaton

223 / 243

Subset product

SCCSCC

SCC

4 74

• build product of model with subset states

• compute SCCs

• states represented by one SCC either all accepting or allrejecting

• transient probabilities also obtained here

224 / 243

Subset decision

SCCSCC

SCC

4 7?

• try to decide SCCs by under-/overapproxing acceptancecondition

• all/some states in the subset are accepting (adapted totransition accepting)

w , x , y , z

w , x , y , z

w , x , y , z

w , x , y , z

w , x , y , z

w , x , y , z

w , x , y , z

w , x , y , z

w , x , y , z

in Buchi under over

225 / 243

Breakpoint decision

SCCSCC

SCC

4 7?

m, w , x , y , z , ∅

• otherwise: choose some state• then apply a breakpoint construction (refined approximation)• breakpoint state: subset-states, union-of-rabin-tree-states

w , x , y , z

x , y

x

zm,

w , x , y , z

x y , zm, m, (w , x , y , x), (x , y , z)

226 / 243

Rabin decision

SCCSCC

SCC

4 7?

m, w , x , y , z ,

• if this still not works: choose state and apply Rabindeterminisation

• otherwise: choose some state• then apply Rabin determinisation starting from this state• still better than Rabin determinisation from scratch:• Rabin automata smaller than those from direct

determinisation• transient probability construction in quotient model

227 / 243

Optimisations

• use transition-labelled, generalised Buchi automataaacc1,acc2

aacc1,acc3

a, acc1

• have (additional) BDD-based implementation

v1

v2 v2

v3 v3

0 1

• on-the-fly exploration of automata states a b a b a bc

c

c

c

• on-the-fly SCC decompositionSCC SCC

SCC

228 / 243

Conclusion

Our lazy determinisation approach is effective and practicallyapplicable.

It can often severly speed up the decision algorithm for complexLTL formulas on probabilistic models.

229 / 243

Learning Framework for Compositional Verification[POPL2015]

230 / 243

Assume Guarantee Framework

The assume-guarantee reasoning proof rule for classical systems:

M0 A A‖M1 |= P

M0‖M1 |= P(1)

• M0 A means that A simulates all behaviors of M0

• to show the composed system satisfying P, it suffices to find aclassical assumption A such that A simulates M0, and Acomposed with M1 satisfies P as well.

231 / 243

Finding classical assumptions by active machine learning

• A learning algorithm infers a representation of an unknowntarget by making queries to a teacher.

• The learning-based framework thus devises a mechanicalteacher to answer such queries.

• Together with a learning algorithm, the framework is able tofind assumptions automatically.

• For classical systems, the L∗ learning algorithm for regularlanguages suffices to infer classical finite automata as classicalassumptions.

232 / 243

Running Example

Property: P≤0.01[ψfailed ] where ψfailed stands for reaching the state〈s1

3 s23 〉.

233 / 243

A Purported Assumption

On receiving the weighted assumption A, the mechanical teacher

• It first checks premise node1 e A

• It then checks the other premise by model checking.

234 / 243

Model Checking

• Since A is a weighted assumption, A‖node2 need not be anMDP.

• We need to lift the probabilistic model checking algorithm toweighted assumptions.

• After model checking, we find that the property P≤0.01[ψfailed ]does not hold on A‖node2.

• A witness to A‖node2 6|= P≤0.01[ψfailed ] is constructed.

235 / 243

Witness Checking

• A weighted witness to A‖node2 6|= P≤0.01[ψfailed ] is notnecessarily a witness to node1‖node2 6|= P≤0.01[ψfailed ].

• The mechanical teacher checks if the witness toA‖node2 6|= P≤0.01[ψfailed ] is spurious or not.

• the corresponding path in node1‖node2 has probability0.64× 0.1× 0.1 = 0.0064 ≤ 0.01.

• The witness to A‖node2 6|= P≤0.01[ψfailed ] is hence spurious.

• The mechanical teacher then should help the learningalgorithm revising the weighted assumption by sending acounterexample.

236 / 243

Selecting Counterexamples

To remove the spurious witness in A, the mechanical teacherselects a transition in the weighted assumption A whichcontributes most to the spurious witness.

• The transitions s10

start−→ s11 and s1

1

go1−→ s13 in the weighted

assumption A contribute to the spurious witness.• The mechanical teacher can send either of the transitions as a

counterexample to the learning algorithm.

• Assume the mechanical teacher sends the transition s11

go1−→ s13

as the counterexample.• The learning algorithm will then update the weight of the

selected transition in revised weighted assumptions.

237 / 243

Learning Assumption

• After receiving a counterexample, the learning algorithm willpurport another weighted assumption, say A′.

• We have node1 e A′.• A′‖node2 |= P≤0.01[ψfailed ] holds by model checking.• Thus node1‖node2 satisfies the intended probabilistic property.• Note A′ is a not a probabilistic assumption. If only

probabilistic assumptions were considered, assume-guaranteereasoning would not be effective.

238 / 243

An MTBDD Learning Algorithm

We develop an MTBDD learning algorithm to infer weightedassumptions. Let f (x) be an unknown target MTBDD. We assumea teacher to answer the following types of queries:

• On a membership query MEM(ν) with a valuation ν of x, theteacher answers f (ν);

• On an equivalence query EQ(F)(g) with a conjectureMTBDD g(x), the teacher answers YES if f = g . Otherwise,she returns a valuation ν of x with f (ν) 6= g(ν) as acounterexample.

• In order to learn general MTBDD’s, we modify the L∗

algorithm to generate k-DFA.

• When the modified L∗ algorithm gives a k-DFA in anequivalence query, the MTBDD learning algorithm transformsthe automaton into an MTBDD.

239 / 243

Learning Framework for Compositional Verification

240 / 243

241 / 243

Conclusion: Buchi automata are quite useful

• This is joint work with many colleagues, and students YongLi, Depeng Liu

• Advanced Automata-based Algorithms for Program TerminationChecking, Chen, Y.-F.; Heizmann, M.; Lengal, O.; Li, Y.; Tsai,M.-H.; Turrini, A. and Zhang, L. In PLDI, 2018.

• Learning to Complement Buchi Automata, Li, Y.; Turrini, A.;Zhang, L. and Schewe, S. In VMCAI, pages 313-335, LNCS 10747,2018.

• A Novel Learning Algorithm for Buchi Automata based on Family ofDFAs and Classification Trees, Li, Y.; Chen, Y.-F.; Zhang, L. andLiu, D. In TACAS, pages 208-226, LNCS 10205, 2017.

• Leveraging Weighted Automata in Compositional Reasoning aboutConcurrent Probabilistic Systems, He, F.; Gao, X.; Wang, B.-Y. andZhang, L. In POPL-15, pages 503-514, ACM, 2015.

• Lazy Probabilistic Model Checking without Determinisation, Hahn,

E. M.; Li, G.; Schewe, S.; Turrini, A. and Zhang, L. In CONCUR,

pages 354-367, LIPIcs 42, 2015.

242 / 243

Conclusion

• Dikstra: testing can never prove the absence of bugs

• Simple & elegant

• Not only do good research, but be its loyal, continuous user!

• Difficult things are usually not easy to be put into practice

• Termination problem• Buchi complementation & determinization problem

243 / 243

Conclusion

• Dikstra: testing can never prove the absence of bugs

• Simple & elegant

• Not only do good research, but be its loyal, continuous user!

• Difficult things are usually not easy to be put into practice• Termination problem• Buchi complementation & determinization problem

243 / 243

Documents

Learning Büchi Automata and Its Applications · MSO NFA Both MSO and NFA de ne the class of regular expressions. ... vol. 131 of LNCS, pages 52-71. Springer-Verlag, 1981. ... Part