A Fault Tolerant Java Virtual Machine

A Fault Tolerant Java Virtual Machine

Malte Tiedje

SeminarZuverlassigkeit von Software in sicherheitskritischen Systemen

28. Juni 2005

Malte Tiedje ( Seminar Zuverlassigkeit von Software in sicherheitskritischen Systemen)A Fault Tolerant Java Virtual Machine 28. Juni 2005 1 / 25

Introduction Fault-tolerance

Fault-tolerance

What is Fault-tolerance ?

Definition

... is the property of a system that continues operating properly in theevent of failure of some of its parts.

www.wikipedia.org


Introduction Fault-tolerance

Fault-tolerance

What is Fault-tolerance ?

Definition

... is the property of a system that continues operating properly in theevent of failure of some of its parts.

www.wikipedia.org


Introduction Why Java?

Why Java?

Java is ...

portable

secure: strong-typing, ...

distributed: RMI

and of course: OO, simple, wide-used

but Java is not fault-tolerant



Why Java?

Java is ...

portable


distributed: RMI





Why Java?

Java is ...

portable


distributed: RMI





Why Java?

Java is ...

portable


distributed: RMI





Why Java?

Java is ...

portable


distributed: RMI





Why Java?

Java is ...

portable


distributed: RMI




The approach

4 Steps

4 Steps

1. Define a deterministic state machine a unit of replication

2. Implement independently failing replicas of the state machine

3. Ensure all replicas start from identical states and perform the samesequence of state transitions

4. Ensure each output-producing transition yields in a single output tothe environment


The approach

4 Steps

4 Steps






The approach

4 Steps

4 Steps






The approach

4 Steps

4 Steps






The approach State-Machines

State Machines

State Machines are ..

a set of state variables and a sequence of commands

A command ...

reads a subset of state variables (read set values = rsvs)

modifies a subset of states variables (write set values = wsvs)

A command is deterministic ...

when a comand produces a deterministic wsvs and outputs an givenrsvs

A deterministic state machine ...

reads fixed sequence of deterministic commands



State Machines



A command ...









State Machines



A command ...









State Machines



A command ...









State Machines



A command ...









Fault-tolerance by duplication

Replication

Definition

Providing multiple identical instances of the same system, directing tasksto all of them in parallel, and choosing the correct result on the basis of aquorum

www.wikipedia.org

Each replica undergoes the sames sequence of state transitions andproduces the sames output!



Fault-tolerance by duplication

Replication

Definition

Providing multiple identical instances of the same system, directing tasksto all of them in parallel, and choosing the correct result on the basis of aquorum

www.wikipedia.orgEach replica undergoes the sames sequence of state transitions andproduces the sames output!


The approach JVM as State Machine

JVM as State Machine I

Implement replica coordination in the JVM:

3 Challenges

1. not all commands executed by the JVM are deterministic

2. replicas of a JVM do not in general execute identical sequence ofcommands

3. the read-set for a given command is not guaranteed to containidentical values at all replicas





3 Challenges

1. not all commands executed by the JVM are deterministic2. replicas of a JVM do not in general execute identical sequence of

commands

3. the read-set for a given command is not guaranteed to containidentical values at all replicas





3 Challenges

1. not all commands executed by the JVM are deterministic2. replicas of a JVM do not in general execute identical sequence of

commands3. the read-set for a given command is not guaranteed to contain

identical values at all replicas



JVM as State Machine II

Problem: JVM is multi-threaded and a state-machines typical are not

Solution: every thread is a state-machine and the JVM is a set ofcooperating state-machines

In particular: BEE (Bytecode Execution Engines) as set of functionsthat define together a replica

Napper 2003



JVM as State Machine II

Problem: JVM is multi-threaded and a state-machines typical are not

Solution: every thread is a state-machine and the JVM is a set ofcooperating state-machines

In particular: BEE (Bytecode Execution Engines) as set of functionsthat define together a replica

Napper 2003


Details Non-deterministic commands

Non-deterministic commands

Exclusively invoked by Java Native Interface (JNI)

e.g read the hardware clock

Problem: the replicas have different input values, because the input isperformed outside the scope of the JVM

Solution: the protocol forces the backup to adopt the writes-setvalues produces by the primary

But: this is not enough: we have to restrict the behavior of the nativemethods



























Restriction 1 and 2

Restriction 1

Native methods must not produce non-deterministic output to theenvironment

Example

native void DoNotDo() {lc = read time of day ();print ( lc );

}

native long Input () {return read time of day ();

}native void Output(long lc) {

print ( lc );}



Restriction 1 and 2

Restriction 1


Example


}



print ( lc );}



Restriction 1 and 2

Restriction 1


Example


}



print ( lc );}



Restriction 1 and 2

Restriction 2

Native methods must invoke other methods deterministically

Example

native void DoNotDo() {lc = read time of day ();if ( lc > 17:24:32)

acquire lock ();}


}void do(long lc ) {

lc = Input ();if ( lc > 17:24:32)

acquire lock ();}



Restriction 1 and 2

Restriction 2


Example


acquire lock ();}



lc = Input ();if ( lc > 17:24:32)

acquire lock ();}



Restriction 1 and 2

Restriction 2


Example


acquire lock ();}



lc = Input ();if ( lc > 17:24:32)

acquire lock ();}



Implementation

Checked all native methods in JRE librariesless then 100 are non-deterministic

Stored signature of each method in hash table(class, method, arguments)

When primary invokes native method, check hash table

On match, send backup return values and modified arguments

On recovery, backup may use logged values



Implementation








Implementation








Implementation








Implementation







Details Non-deterministic rsvs

Non-deterministic Read Sets I

Because of Multi-Threading in the JVM the values of shared variablesare non-deterministic

Solution I:

All access to shared data is wrapped by correct use of monitors (usingsynchronized)therefore we need replicating the Lock Synchronization





Solution I:

All access to shared data is wrapped by correct use of monitors (usingsynchronized)

therefore we need replicating the Lock Synchronization





Solution I:

All access to shared data is wrapped by correct use of monitors (usingsynchronized)therefore we need replicating the Lock Synchronization



Implementation I

Napper 2003

Definition

< tid , tasn, lid , lasn >

tid thread id of the locking thread

asn thread acquire sequence number recording the number oflocks acquired so far by thread tid

lid lock id

lasn lock acquire sequence number recording the number of timeslid has been acquired so far



Implementation I

Napper 2003

Definition




lid lock id




Implementation I

Napper 2003

Definition




lid lock id




Implementation I

Napper 2003

Definition




lid lock id




Implementation II

Hard to create unambiguous ids

Cannot use object address as lid

Cannot use order of events at primary

Napper 2003



Implementation II




Napper 2003



Implementation II




Napper 2003



Implementation II




Napper 2003



Non-deterministic Read Sets II

Solution I: many programs do not meet this condition (not even Sun’sJRE)

Example

class Example {

static Formatter shared data = null ;

String toString (){if ( shared data == null){

shared data = new Formater();synchronized method();...

}}}




Solution I: many programs do not meet this condition (not even Sun’sJRE)

Example

class Example {

static Formatter shared data = null ;

String toString (){if ( shared data == null){

shared data = new Formater();synchronized method();...

}}}




Solution II:

A thread has exclusives access to all shared variables while scheduled

therefore we need to replicate the thread scheduling




Solution II:

A thread has exclusives access to all shared variables while scheduled

therefore we need to replicate the thread scheduling



Implementation

Definition

< brcnt , pcoff ,moncnt , lasn, tid >

brcnt counts the control flow changes executed (e.g. branches,jumps, and methods invocations)

pcoff records the bytecode offset of the PC within the methodcurrently executed by t

moncnt counts the monitor acquisitions and releases performed by t

lasn records the lock acquisition sequence number when t isrescheduled while waiting on a lock

tid the thread id of the next scheduled thread



Implementation

Definition









Implementation

Definition









Implementation

Definition









Implementation

Definition








Details Output to the environment

Output to the environment

Objective: Simulate a single, fault-tolerant state-machine

In general impossible

Restriction 3

All native method output to the environment is either idempotent ortestable

therefor we need a Side Effect Handler






Restriction 3








Restriction 3








Restriction 3





Side Effect Handler

register: method’s signature, what should be logged, etc

test: called on testable, uncertain commands

log & receive: how primary and backup exchange state

restore: called at the backup during recovery



Side Effect Handler







Side Effect Handler







Side Effect Handler







Side Effect Handler






Evaluation

Evaluation

Overhead: depends on application and Rep. Lock-Sync / Rep. ThreadSched.

Experiments: SPEC JVM98 benchmark (i.a: compress, db, raytracerrendering)

Qualitative: differ from 5% up to 375%, average 60% for rts, 140%for rla


Evaluation

Evaluation





Evaluation

Evaluation





Evaluation

Eval.: Replicated Lock Acquisition

Napper 2003


Evaluation

Eval.: Replicated Thread Scheduling

Napper 2003


Evaluation

Conclusion

A fault-tolerant JVM (at a reasonable cost)

Write Once, Run Anywhere

A framework for replicating multi-threaded SMs


Evaluation

Conclusion





Evaluation

Conclusion





Evaluation

References

A Fault-Tolerant Java Virtual Machine: Jeff Napper, LorenzoAlvisi, Harrick Vin

http://www.cs.utexas.edu/users/jmn/papers/napper03fault.ppt

www.wikipedia.org


Technology

A Fault Tolerant Java Virtual Machine