Efficient Controller Synthesis for Consumption Games with Multiple Resource Types

Efficient Controller Synthesisfor Consumption Games

with Multiple Resource Types

Tomas Brazdil1 Krishnendu Chatterjee2 Antonın Kucera1

Petr Novotny1

1Masaryk University 2IST AustriaBrno, Czech Republic Klosterneuburg, Austria

CAV, BerkeleyJuly 10, 2012

Consumption Games Tomas Brazdil, Krishnendu Chatterjee, Antonın Kucera, Petr Novotny

Outline

Motivation

→

Consumption games

→

Results & comparison with multi-energy games


Motivation

• Agent in unpredictableenvironment

• Resource consumption

• Resource reloads

• Goal: avoid depletion of theagents resources.


Motivation






Motivation






Motivation (2)

Example: public transportation - emergency team.

Resources: Reloads:fuel (natural gas) • • gas stations

spare parts • • depot

worker’s energy • • various establishments

Repairs feasible? Size of fuel tank/storage?


Motivation (2)







Motivation (2)







Motivation (2)





Repairs feasible? Size of fuel tank/storage?Consumption Games Tomas Brazdil, Krishnendu Chatterjee, Antonın Kucera, Petr Novotny

Game modelPlayers: � (controller)

� (environment)Setting: finite graph + load vector (∈ Nd

0 )

Configuration: (s, (5, 9, 3))

u

s•

t v

(−1,−2, 0)

(−2,−1, 0) (0,−3, 0)

(−2, ω, 0)

(ω,−1, ω)(0, ω, 0)

(−1,−1,−1)

Goal of player �: maintain the load vector positive in everycomponent.




0 )

Configuration: (s, (5, 9, 3))

u

s•

t v

(−1,−2, 0)

(−2,−1, 0) (0,−3, 0)

(−2, ω, 0)

(ω,−1, ω)(0, ω, 0)

(−1,−1,−1)





0 )

Configuration: (u, (4, 7, 3))

u•

s

t v

(−1,−2, 0)

(−2,−1, 0) (0,−3, 0)

(−2, ω, 0)

(ω,−1, ω)(0, ω, 0)

(−1,−1,−1)





0 )

Configuration: (u, (4, 7, 3))

u•

s

t v

(−1,−2, 0)

(−2,−1, 0) (0,−3, 0)

(−2, ω, 0)

(ω,−1, ω)(0, ω, 0)

(−1,−1,−1)





0 )

Configuration: (t, (2, 6, 3))

u

s

t• v

(−1,−2, 0)

(−2,−1, 0) (0,−3, 0)

(−2, ω, 0)

(ω,−1, ω)(0, ω, 0)

(−1,−1,−1)



Resource updates

Labeling function L : Edges → X d , for some set X .Multi-energy games1: X = Z, updates = addition (integersrepresented in binary)

s t

(−1, 3)

(2,−1)

(s, (2, 1))

(−1,3)−→ (t, (1, 4))

(2,−1)−→ (s, (3, 3))

(−1,3)−→ (t, (2, 6))

(2,−1)−→ (s, (4, 5))

But reloads are often atomic actions!

⇒ ⇒

1Chatterjee, K., Doyen, L., Henzinger, T.A., Raskin, J.F.: Generalizedmean-payoff and energy games. In: Proceedings of FSTTCS’10.


Resource updates


s• t

(−1, 3)

(2,−1)

(s, (2, 1))

(−1,3)−→ (t, (1, 4))

(2,−1)−→ (s, (3, 3))

(−1,3)−→ (t, (2, 6))

(2,−1)−→ (s, (4, 5))


⇒ ⇒



Resource updates


s t•(−1, 3)

(2,−1)

(s, (2, 1))(−1,3)−→ (t, (1, 4))

(2,−1)−→ (s, (3, 3))

(−1,3)−→ (t, (2, 6))

(2,−1)−→ (s, (4, 5))


⇒ ⇒



Resource updates


s• t

(−1, 3)

(2,−1)

(s, (2, 1))(−1,3)−→ (t, (1, 4))

(2,−1)−→ (s, (3, 3))

(−1,3)−→ (t, (2, 6))

(2,−1)−→ (s, (4, 5))


⇒ ⇒



Resource updates


s t•(−1, 3)

(2,−1)

(s, (2, 1))(−1,3)−→ (t, (1, 4))

(2,−1)−→ (s, (3, 3))

(−1,3)−→ (t, (2, 6))

(2,−1)−→ (s, (4, 5))


⇒ ⇒



Resource updates


s• t

(−1, 3)

(2,−1)

(s, (2, 1))(−1,3)−→ (t, (1, 4))

(2,−1)−→ (s, (3, 3))

(−1,3)−→ (t, (2, 6))

(2,−1)−→ (s, (4, 5))


⇒ ⇒



Resource updates


s• t

(−1, 3)

(2,−1)

(s, (2, 1))(−1,3)−→ (t, (1, 4))

(2,−1)−→ (s, (3, 3))

(−1,3)−→ (t, (2, 6))

(2,−1)−→ (s, (4, 5))


⇒ ⇒



Resource updates (2)

Labeling function L : Edges → X d , for some set X .

Consumption games:X = {0,−1,−2, . . . } ∪ {ω}• integers represented in binary• ω only on edges outgoing from �’s vertices

• consumption = subtraction• reloads = � increases the load to an arbitrary value

s t

(ω,−1)

(s, (2, 1))

(−1,ω)−→ (t, (1,

11

))




Consumption games:X = {0,−1,−2, . . . } ∪ {ω}• integers represented in binary• ω only on edges outgoing from �’s vertices• consumption = subtraction• reloads = � increases the load to an arbitrary value

s• t

(−1, ω)

(ω,−1)

(s, (2, 1))

(−1,ω)−→ (t, (1,

11

))





s t•(−1, ω)

(ω,−1)

(s, (2, 1))(−1,ω)−→ (t, (1,

11

))





s t•(−1, ω)

(ω,−1)

(s, (2, 1))(−1,ω)−→ (t, (1, 11))


Algorithmic problems

How to control the system in order to avoid resourcedepletion?

Parameters:

• n – number of vertices of the game graph

• d – dimension of the game (i.e., of the load and updatevectors)

• ` – max{|n| | n ∈ Z is a component of some update vector}

Energy games: intractable even for d = 3 and ` = 1.

For consumption games, the fundamental problems are tractablefor every fixed d and `.




Parameters:









Parameters:







Emptiness problem for Safe(s)

Safe(s) = {α | � has a winning strategy in (s, α)}

Problem: For a given s, is Safe(s) 6= ∅?

Is there a car allowing a response to any accident, wherever ithappens?

Proposition (reduction to Streett games)

Safe(s) 6= ∅ ⇔ � can assure that for every 1 ≤ i ≤ d , the i-thresource is either decreased only finitely often or reloaded infinitelyoften.

Complexity: O(d! · nd+1). (Including the computation of thewinning strategy.)Auxiliary resources can encode Streett winning conditions.


Emptiness problem for Safe(s)


Problem: For a given s, is Safe(s) 6= ∅?

Is there a car allowing a response to any accident, wherever ithappens?

Proposition (reduction to Streett games)

Safe(s) 6= ∅ ⇔ � can assure that for every 1 ≤ i ≤ d , the i-thresource is either decreased only finitely often or reloaded infinitelyoften.

Complexity: O(d! · nd+1). (Including the computation of thewinning strategy.)Auxiliary resources can encode Streett winning conditions.


Membership problem for Safe(s)


Problem: For a given s and α, is α ∈ Safe(s)?

It is sufficient to start with 12 litres of fuel and 2 boxes of spareparts?

Solution: compute all minimal elements of Safe(s).(α ∈ Safe(s)⇔ α ≥ some minimal element of Safe(s))

Finite reload property

Let B = n · d · `.1 If � can win from (s, α), then he can win from (s, α) without

reloading any resource above B.

2 If α is a minimal element of Safe(s), then α ≤ (B, . . . ,B).






















Membership problem for Safe(s) (2)

Encode configurations with load vectors between (0, . . . , 0) and(B, . . . ,B) into vertices (game size O(n · Bd)):

t

s

u

(ω,−2)

(−1, 0)(0, 0)

(−1, ω)(−1, 0)

B = 3 · 2 · 2 = 12(12, 2) ∈ Safe(s)?

→

[s, (12, 2)]

[u, (11, 2)]

1

...

[u, (11, 12)]

[t, (12, 0)]

Avoid!

From the finite reload property: If α ≤ (B, . . . ,B), thenα ∈ Safe(s)⇔ � can win the safety game from [s, α].

Complexity: (n · d · `)O(d) (incl. strategy computation).




t

s

u

(ω,−2)

(−1, 0)(0, 0)

(−1, ω)(−1, 0)

B = 3 · 2 · 2 = 12(12, 2) ∈ Safe(s)?

→

[s, (12, 2)]

[u, (11, 2)]

1

...

[u, (11, 12)]

[t, (12, 0)]

Avoid!






t

s

u

(ω,−2)

(−1, 0)(0, 0)

(−1, ω)(−1, 0)

B = 3 · 2 · 2 = 12(12, 2) ∈ Safe(s)?

→

[s, (12, 2)]

[u, (11, 2)]

1

...

[u, (11, 12)]

[t, (12, 0)]

Avoid!






t

s

u

(ω,−2)

(−1, 0)(0, 0)

(−1, ω)(−1, 0)

B = 3 · 2 · 2 = 12(12, 2) ∈ Safe(s)?

→

[s, (12, 2)]

[u, (11, 2)]

1

...

[u, (11, 12)]

[t, (12, 0)]

Avoid!




Storage capacities - Cover(s)

Finite reload property ⇒ storage sizes (n · d · `, . . . , n · d · `) aresufficient for winning (if � can win at all).

Is (1, . . . , 1, n · d · `+ 1) sufficient?

It is sufficient to have a 250-litre gas tank and a space for 2 boxesof spare parts?

Definition

Vector α covers s if � has a winning strategy in (s, α) s.t. underthis strategy the load vector is always ≤ α (and ≥ (1, . . . , 1)).

Cover(s) = {α | α covers s}


Storage capacities - Cover(s)

Finite reload property ⇒ storage sizes (n · d · `, . . . , n · d · `) aresufficient for winning (if � can win at all).

Is (1, . . . , 1, n · d · `+ 1) sufficient?

It is sufficient to have a 250-litre gas tank and a space for 2 boxesof spare parts?

Definition

Vector α covers s if � has a winning strategy in (s, α) s.t. underthis strategy the load vector is always ≤ α (and ≥ (1, . . . , 1)).

Cover(s) = {α | α covers s}


Membership problem for Cover(s)

Proposition

If α is a minimal element of Cover(s), then α ≤ (C , . . . ,C ), whereC = (n · d · `)d!.

(z1, . . . , zd) ∈ Cover(s)⇔ (min{z1,C}, . . . ,min{zd ,C}) ∈ Cover(s)

t

s

u

(ω,−2)

(−1, 0)(0, 0)

(−1, ω)(−1, 0)

C = (3 · 2 · 2)2 = 144(250, 2) ∈ Cover(s)⇔ (144, 2) ∈ Cover(s)⇔ � wins the safety game from [s, (144, 2)]

→

[s, (144, 2)]

[u, (143, 2)][t, (144, 0)]

Complexity: (n · d · `)O(d ·d!) (even for computing all minimalelements + winning strategy).

Fact

There are games where minimal covering vectors havecomponents exponential in d .

Hypothesis

Minimal elements of Cover(s) are bounded byC ∈ (n · d · `)O(d). Thus, all minimal elements can becomputed in time (n · d · `)O(d2)



Proposition



t

s

u

(ω,−2)

(−1, 0)(0, 0)

(−1, ω)(−1, 0)


→

[s, (144, 2)]

[u, (143, 2)][t, (144, 0)]


Fact


Hypothesis




Proposition



t

s

u

(ω,−2)

(−1, 0)(0, 0)

(−1, ω)(−1, 0)


→

[s, (144, 2)]

[u, (143, 2)][t, (144, 0)]


Fact


Hypothesis



Iterative algorithms for Safe(s)

Iterative algorithms exist for:

• One-player games (� controls all vertices).

• Decreasing games (every resource is either reloaded ordecreased along every cycle).

Pros:

• Avoid construction of the large safety game.

• Potentially terminate much earlier.

Future: develop similar algorithms for Cover(s).


Conclusion

Consumption games:

• ...are a natural model for resource-dependent systems.

• ...admit more efficient algorithmic analysis than multi-energygames.

• ...admit efficient algorithmic analysis if the number ofresources and the magnitude of decrements is small.

Future work:

• Prove better complexity bounds and design iterativealgorithms for Cover(s).

• ...and many other possibilities, thanks to the naturalness ofthe model.

Thank you!


Conclusion

Consumption games:




Future work:



Thank you!


Conclusion

Consumption games:




Future work:



Thank you!


Technology

Efficient Controller Synthesis for Consumption Games with Multiple Resource Types