1

A Formal Monitoring-based Framework for Software Development and Analysis

Feng ChenMarcelo D'AmorimGrigore Rosu

University of Illinois at Urbana-Champaign

2

Motivation :Increase Software Reliability

Traditional formal verification methodsHigh confidence, do not always scale up well

TestingScalable, but adhoc, no correctness guarantees

What do we really want … ?A.Prove a program correct?B.Correct execution!

A => B … but isn’t the price too big?

3

Overview

• Monitoring Software Systems

• Monitoring-Oriented Programming (MOP)

• Java-MOP

• An HTTP-Server Example

• Conclusion and Future Work

4

Avoiding Complexity whileEnsuring Safety by Monitoring

(Example by Lui Sha - Simplex)

Joe is a student in an algorithms classFinal exam is to develop a sorting algorithm

If incorrect then Joe fails the classIf provably correct but inefficient then he gets BIf both correct and efficient then he gets A

Hm, heap sort is efficient but hard to prove correct; insertion sort is trivial to prove correct but it is not efficient

Hm, heap sort is efficient but hard to prove correct; insertion sort is trivial to prove correct but it is not efficient

5

Joe’s Solution

Heapsort

O(n log(n))

Monitor ifvector issorted

yes

O(n)

Insertionsort

no

O(n2)provablycorrect

Joe has an efficient and provably correct sorting algorithm!He avoided proving heap sort correct, which is hard!

6

Monitoring in Engineering

Most engineering disciplines take monitoring

as basic design principleFire alarms in buildings

Fuses in electronics

Watchdogs in hardware

Why not to do the same in software eng.?Monitoring-oriented programming

7

Monitoring-Oriented Programming (MOP)Proposed at RV 2003

Monitoring already used in software developmentDesign By Contract, Runtime Verification (MaC, PaX, …)

MOP underlying motivations and challenges• Can one generically and formally capture the vario

us approaches to monitoring and runtime verification in a unified manner?

• Monitoring should be a design and software development principle. Write programs having monitoring in mind!

8

What are the foundational principles of monitoring programs?

Program

S1 S2 . . . . . Sn

1. Observation

Specification

2. Checking

3. Recovery

9

Goal of MOP

The different approaches to RV rely on implicit,carefully chosen instance of the 3 principles

1. Observation2. Checking3. Recovery

MOP aims at separation and individual advancementof the three principles

Three ways to look at MOP1. Merging specification and implementation2. Extending languages with logic-based statements3. A light weighted formal method

10

MoP Example: A traffic light monitor

Following monitor can appear in a Java program.../*@ logic=FTLTL { predicate red = tlc.state.getColor() == 1; predicate green = tlc.state.getColor() == 2; predicate yellow = tlc.state.getColor() == 3; formula : []( green -> (! red U yellow)); Violation handler : … any code}@*/...

It is pre-compiled into actual Java code...switch(bttFsmState) {

case -1 : break; case 1 : bttFsmState = tlc.state.getColor()==3 ? 1 : tlc.state.getColor()==2 ?

tlc.state.getColor()==1 ? 0 : 2 : 1; break;

case 2 : bttFsmState = tlc.state.getColor()==3 ? 1 : tlc.state.getColor()==1 ? 0 : 2;

break; }

if(bttFsmState == 0) { … the “recovery” code …}}

...

11

MoP Features andDesign Principles

12

Extending Languages with Logics

• Do not modify host languages– Able to reuse heritage systems– Evolve with languages and compilers

• Specifications can be – Annotations in the source code

• Especially for checkpoints

– In separated specification files• Separation of concerns• Improved reusability

13

MOP Specification Syntax

/************** Heading starts ****************/[attribute]* <Type> <Name> logic= <Logic Name> {

/************* Body starts ********************/

... Specification Body ...

/************* Handler starts *****************/[Violation Handler: ...handling the violation...]

[Validation Hander: ...handling the validated...]

}

class-inv logic = ERE {

var int flag = -1;event suspend: called(void suspend()) {flag = 1;};event resume: called(void resume()) {flag = 2;};formula: (suspend resume)*

Violation Handler: if (flag == 2) { System.out.println("resume() called before suspend() in HttpClient! Adding client back to queue ..."); synchronized(suspendedClients){ suspendedClients.add(thisObject); } } else { System.out.println("suspend() called twice before resume() in HttpClient!"); }}

DefinitionExample

14

MOP Specification Syntax

• Attributes– static, inline/outline, sync/async

• Types of specifications:– class invariants, interface constraints, method

pre-post conditions, checkpoints.

• User-defined handlers– Violation and/or validation handlers

• Not all combinations always possible

15

Sorting Example

method (HeapSort(int[] a)) logic=JML {

ensures isOrdered(a);

Violation handler: InsertionSort(a);}

…

if (! isOrdered(a)){ InsertionSort(a);}

…

16

Security Example

class-inv logic=ERE {

event auth = authentication();event use = accessResource();formula: (! auth)* use;

Validation handler: authentication();}

switch ($state) { case 0 : $state = use ? 1 : ! auth ? 0 : -1; break; }

if ($state == 1) authentication();

17

Extensible Logic Framework

18

Workflow of MOP

19

Logic Plug-ins

• No silver-bullet logic to express any requirements• To add a new MOP requirements formalism, all on

e needs to do is to provide a logic plug-in– Logic plug-in consists of a language shell and a logic en

gine; several logic plug-ins can share a logic engine

• The I/O of the logic plug-in is standardized to facilitate extensibility

• Logic plug-in encodes a monitor synthesis algorithm: formula -> monitor

• WWW repository of logic plug-insfsl.cs.uiuc.edu/mop

20

Current Logic plug-ins

• Linear Temporal Logic (LTL)– Past Time LTL– Future Time LTL

• Extended Regular Expressions (ERE)• Design By Contract: Jass, JML (partly)• To be supported:

– RTL, MTL, …– Users can easily add new logics via provided standar

dized interface

21

Future time LTL plug-in

Input:[] (green -> (! red) U yellow)

Output:Declaration. int state;Initialization. state = 1;Monitoring Body. switch (state) { case 1 : state = yellow ? 1 : green ? (red ? -1 : 2) : 1; case 2 : state = yellow ? 1 : red ? -1 : 2; }Failure Condition. state == -1

Algorithm jointly with Havelund(RV’01)

Algorithm jointly with Havelund(RV’01)

22

Past time LTL plug-in

Input:start(P) -> [Q, end(R \/ S))

Output:Declaration. boolean now[3], pre[3]Initialization. now[3] = R || S; now[2] = Q; now[1] = P;Monitoring Body. now[3] = R || S; now[2] = (pre[2] || Q) && (now[3] || (! pre[3])); now[1] = P;Failure Condition. now[1] && (! pre[1]) && (! now[2])

Algorithm jointly with Havelund(TACAS’02)

Algorithm jointly with Havelund(TACAS’02)

23

Extended RE plug-in

Input:~((~ empty)(green red)(~ empty))

Output:Declaration. int state;Initialization. state = 0;Monitoring Body. switch (state){ case 0 : state = (yellow || red) ? 0 : green? 1 : -1; case 1 : state = green ? 1 : yellow ? 0 : -1; }Failure Condition. state == -1

Algorithm jointly with Sen (RV’03)

Algorithm jointly with Sen (RV’03)

24

Java-MOP

A prototype supporting part of the desired MOP features in Java

25

Java-MOP Overview

• Provides GUI and command-line interfaces– Online interface: http://fsl.cs.uiuc.edu/mop

• Supports FTLTL, PTLTL, ERE, Jass, JML

• Uses AspectJ for monitor integration– Limitations

• Can only monitor scalar fields of classes for class invariants

• The event generation and analysis are not atomic

26

Java-MOP Architecture

27

Java-MOP at Work

A buggy HTTP server

- taken from an IBM repository of buggy concurrent Java programs

28

29

Request access to the server

Granted?

work Put itself into the waiting queue

Get a client from the waiting queue, and resume it

suspend itself

Exit

Yes No

1

23

30

2 suspend2

Clienti Clientj

Exit

suspend

Has been removed from

the waiting queue!

SuspendedClients

Clienti

1

3Stopped

add

remove & resume

31

A bad trace

32

The Bug

• Caused by “unexpected” thread interleaving– Not easy to analyze and test

• Root cause– Violation of the basic assumption for suspend()

and resume(): (suspend() resume())*

• Can we fix it on-the-fly in case of violation?• Yes: put the client back into the queue to

allow another client to resume it later.

33

34

Trace with Monitor

35

Conclusion and Future Work

• MOP: a uniform logic-independent monitoring framework.

• Java-MOP: a prototype for Java

• Incorporate more logics/paradigms, especially those that are widely accepted– Complete JML, RTL, MTL, …– WWW repository of logic plug-ins

• Turn Java-MOP into a mature tool– Currently a prototype– Use it on real and large applications

• Support more programming languages