Models@run.time forSelf-adaptive Reactive Systems:A Controller Synthesis-based Approach

Kenji TeiAssociate Professor

National Institute of Informatics, Japan

tei@nii.ac.jphttp://researchmap.jp/teikenji/?lang=english

Joint work with Moeka Tanabe, Ezequiel Castellano, Leandro Nahabedian, Nicolas D’Ippolito, Shinichi Honiden

How does the system adapt to changes?

cloud robots

Continuous real-time data

Internet of Thingsmobile

Sytems face changes at runtime

IaaSUsers system

Sudden increase ofuser traffic

Unstableperformance

How do we ensure correctness of software system?

IoT devices

Device failure

environment

controller(software)

goalcontrol

monitor

Assurance at Development Time

, |=EC GKind of requirements guaranteed depends on model and method adopted

Motivation

environment

controller(software)

goal

Assurance at Development Time

|| |=E G

c1

u2

c2

u4

u3c4

u3c1 u1

u2

u4

c3c2

u3c4

[] p1<> p2

���

C

control

monitor

Motivation

Environment Modeling for Reactive System

environment

moveToWmoveToEpickupputdown

arriveAtWarriveAtM

pickupSuccess

putSuccess

arriveAtE

pickupFailputdFail

���� �����������

������ ���������� �����������

||E = (MAP||W_ROBOT).

Environment Modeling for Reactive System

MAP=(arrive[’w] -> MAP[’w]),

MAP[’w]=( move[’e] -> arrive[’m] -> MAP[’m]

| move[’w] -> arrive[’w] -> MAP[’w]

| putdown -> putsuccess -> MAP[’w]

| pickup -> pickupfail -> MAP[’w]),

MAP[’m]=( move[’e] -> arrive[’e] -> MAP[’e]

| move[’w] -> arrive[’w] -> MAP[’w]

| putdown -> putfail -> MAP[’m]

| pickup -> pickupfail -> MAP[’m]),

MAP[’e]=( move[’e] -> arrive[’e] -> MAP[’e]

| move[’w] -> arrive[’m] -> MAP[’m]

| putdown -> putfail -> MAP[’e]

| pickup -> pickupsuccess -> MAP[’e]).

W_ROBOT=(arrive[’w] -> ROBOT),

ROBOT= (move[Direction] -> arrive[Locations] -> ROBOT

| pickup -> (pickupsuccess -> ROBOT | pickupfail -> ROBOT)

| putdown -> (putsuccess -> ROBOT | putfail -> ROBOT)

| ended -> reset -> ROBOT).

moveToE

arriveM

moveToE

arriveE

pickupSuccess

pickup

models

environment

controller(software)

goal

Assurance at Development Time

|| |=E G

c1

u2

c2

u4

u3c4

u3c1 u1

u2

u4

c3c2

u3c4

[] p1<> p2

���

C

control

monitor

Motivation

environment

goalcontroller(software)

E may become invalid at runtime

System may no longer work,or may continue, but without any assurances

… -> arriveAtW -> moveToE -> arriveAtW ->…unforseen!!

|| |=c1

u2

c2

u4

u3c4

u3c1 u1

u2

u4

c3c2

u3c4

[] p1<> p2

���

control

monitor

Motivation

Environment is Uncertain

Sudden increase of user traffic

Unstable performance

Mulfunction

Location change

Disconnection

Security attack

Machine

Obstacles

Slippyfloor

Sensor/Actuator

Cloud / External Service

Physical entity

User

Service down

Assuming More Realistic Environment

...MAP[’w]=( move[’e] -> arrive[’m] -> MAP[’m]| move[’w] -> arrive[’w] -> MAP[’w]| putdown -> putsuccess -> MAP[’w]| pickup -> pickupfail -> MAP[’w]),

...

...MAP[’w]=( move[’e] -> (arrive[’m] -> MAP[’m]

| arrive[’w] -> MAP[’w])| move[’w] -> arrive[’w] -> MAP[’w]| putdown -> putsuccess -> MAP[’w]| pickup -> pickupfail -> MAP[’w]),

...

[]!(<pickupsuccess,putsuccess>0&&0pickup)0

[]!(!<pickupsuccess,putsuccess>0&&0putdown)0

[](pickup7>AT[’e])0

[](putdown7>AT[’w])0

[](<ended,reset>07>0(<pickupsuccess,{reset}>0&&0<putsuccess,{reset}>))0

[]((AT[’e]0&&0X(move[’w]))07>0X(!arrive[’e]0W0putsuccess))0

[]((AT[’w]0&&0X(move[’e]))07>0X(!arrive[’w]0W0pickupsuccess))0

[]!(<pickupsuccess,putsuccess>0&&0pickup)0

[]!(!<pickupsuccess,putsuccess>0&&0putdown)0

[](pickup7>AT[’e])0

[](putdown7>AT[’w])0

[](<ended,reset>07>0(<pickupsuccess,{reset}>0&&0<putsuccess,{reset}>))0

[]((AT[’e]0&&0X(move[’w]))07>0X(!arrive[’e]0W0putsuccess))0

[]((AT[’w]0&&0X(move[’e]))07>0X(!arrive[’w]0W0pickupsuccess))0

How Much Should We Assume?

Eoptimistic Grich

GpoorEpessimistic

���

high

low

functionalityrich

poor

risk

Everythingcan go wrong

Everythingworks ideally �

��

Use models at runtime!

Graceful Degradationby Self-adaptation with Models

environment

goalcontrol

monitor

controller(software)

adaptationengine

EC G

E’ G’C’

relaxedgoal

Context of My Approach

c1

u2

c2

u4

u3c4

u3c1 u1

u2

u4

c3c2

u3c4

[] p1<> p2

���

LTL GoalsLTS Env. ModelLTS Controller

Self-adaptation by Models@run.time

System

Monitor

Analyzer Planner

Executer

executiontraces

E

Gi

C1. updateenv. model

enactment

C

2. determine req. level

4. hot-swapcontroller

3. generatecontroller

control

Adaptation Engine

knowledge

cachedcontrollers

Decision Making

GN�G1� ����

Motivation

Discrete Controller Synthesisas a Planner

Synthesize C by solving a control problem <E,G>(E || C |= G)

Nicolas D'Ippolito, et al., Synthesis of Live Behaviour Models, FSE2010Nicolas D‘Ippolito, et al., Synthesis of live behaviour models for fallible domains, ICSE2011

c1

u2

u4

c3c2

u3c4

u3

ControllerSynthesizer

Et+1Mt+1

c1

u2

c2

u4

u3c4

u3

� p1�� p2�Gt+1

Ct+1

Feedback control for discrete event systems- driven not by time but rather by events- represented as automata, Petri nets, and the like

Synthesis as Two-Player Game- Discrete Controller Synthesis-

What’s a Game?A game is composed of an arena and a winning condition

- Discrete Controller Synthesis-

Winning RegionThe winning region for Player 1 is a set of states of the arena in which Player 1 can always win

- Discrete Controller Synthesis-

StrategyA (winning) strategy for Player 1 is defined asa finite number of steps that Player 1 will take to ensure reaching the goal from the initial state, no matter what Player 2 does.

Strategies for reachability are directed-acyclic graphs

- Discrete Controller Synthesis-

Tool Support• MTSA (Modal Transition System Analyzer)

- Discrete Controller Synthesis-

http://mtsa.dc.uba.ar

Enact Model

• Interpret controllermodel

• Map actions in modelto concrete implementation

Enactment framework

V.Braberman et al., Controller synthesis: From modelling to enactment, ICSE 2013

- Discrete Controller Synthesis-

Self-adaptation by Models@run.time

System

Monitor

Analyzer Planner

Executer

executiontraces

E

Gi

C1. updateenv. model

enactment

C

2. determine req. level

4. hot-swapcontroller

3. generatecontroller

control

Adaptation Engine

knowledge

cachedcontrollers

Decision Making

GN�G1� ����

Motivation

Self-adaptation by Models@run.time

System

Monitor

Analyzer Planner

Executer

executiontraces

E

Gi

C1. updateenv. model

enactment

C

2. determine req. level

4. hot-swapcontroller

3. generatecontroller

control

Adaptation Engine

knowledge

cachedcontrollersGN�G1� ����

Motivation

SituationAwareness

Updating LTS-based Environment Modelat Runtime

Inconsistent

cons

isten

t

cons

isten

t

Situationchanges

ModelUpdate

Purpose- Environment Model Update -

Updating LTS-based Environment Modelat Runtime

c1

u2

u4

c3c2

u3c4

c1

u2

u4

c3c2

u3c4

u3

executiontrace

ModelLearner

EtEt+1

… ->u2->c2->u3->…

Online update should beaccurate and efficient

Purpose

ModelUpdater

- Environment Model Update -

������������������������

����

����

Proposal- Environment Model Update -

����

Construct a model with all traces in the windowUse gradient descent algorithm

- repeat computation until convergence

Update the model with the latest traceUse stochastic gradientdescent-based algorithm- the latest data is used for update

instead of random picking

����

Existing LTS-model Update Our online LTS-model update���!��� � ����������� ��� ������������

���� ������� ��������������� �������������������� ����� ��

���� !��������� "

��������������

�!���������� ��

����������

�����

���� ������ ��

"pre-condition, action, {post-condition α, β, γ,…}#

e.g. "arrive.w1, move.e, {arrive.m1, arrive.w1}#

����� ��������� �����������������!������"

! ����

���������������

������

�� ������

�����

Proposal- Environment Model Update -

Evaluation

( )

())

()

��������������� �����������������

����������� �

��������� ��� �������

���������� �� ��������

Accuracy and settling time

Computational Overhead

- Environment Model Update -

Self-adaptation by Models@run.time

System

Monitor

Analyzer Planner

Executer

executiontraces

E

Gi

C1. updateenv. model

enactment

C

2. determine req. level

4. hot-swapcontroller

3. generatecontroller

control

Adaptation Engine

knowledge

cachedcontrollersGN�G1� ����

Motivation

Other Key Techniques

1. Environment model learning

3. Controller synthesis

c1

u2

u4

c3c2

u3c4

c1

u2

u4

c3c2

u3c4

u3

executiontrace

ModelLearner

EtEt+1

… ->u2->c2->u3->…

c1

u2

u4

c3c2

u3c4

u3

ControllerSynthesizer

Et+1Mt+1

c1

u2

c2

u4

u3c4

u3

� p1�� p2�Gt+1

2. Goal relaxationc1

u2

u4

c3c2

u3c4

u3

Relaxer

GNG1 ���

Gj

Gicurrentlevel

Et+1

make E neither optimistic nor pessimistic

avoid unnecessary degradation

generate an assured controller

4. Controller updateswap controller to new one

���� ���

������

���

�

��� �

�����

�����

c1�

u1�

u2�

c2�

u4�

u3�c4�

u3�

Ct+1

Ct+1 [SEAMS2016]

[SAC2017]

Ongoing work

Summary• Context– Environment will change at runtime– How do we ensure correctness of software?

• Tech. Topics– How does the system generate a correct controller for

unforeseen situation?

=> Models@run.time approach enables decision making when more information is available

=> Update the environment model andsynthesize a correct controller at runtime!