View
222
Download
0
Category
Tags:
Preview:
Citation preview
Global States
Topics
Usefulness of global state Difficulties in determining snapshot Determining global state Examples
Readings
Van Steen and Tanenbaum: 5.3 Coulouris: 10.5“Distributed Snapshots: Determining Global
States of Distributed Systems”, K. Mani Chandy and Leslie Lamport, ACM Transactions on Computer Systems, vol 3, no 1, February 1985.
Motivating Example
Assume that we have processes interacting in a client-server mode.
Client makes request to server Waits for response While waiting for response, client simply
blocks; does not answer to requests from other nodes.
A
B
C
This is a deadlock
Motivating Example
Assume that you have a centralized server.
It queries each node Each node responds with a list of requests
that are pending (requests for which a response has not been sent).
Centralized server can then build a “waits-for” graph. Cycle in graph implies deadlock
This is an example where being able to determine the global state is useful.
Other Motivating Examples
Distributed Debugging Distributed Garbage Collection
An object is considered to be garbage if there are no longer any references to it anywhere in the distributed system.
p2p1
message
garbage object
objectreference
Global State Occasionally it is useful to know the global state
of a distributed system. The global state of a distributed system consists
of the local state of each process, together with the messages that are currently in transit, that is, that have been sent but not delivered.
Exactly what a local state of a process is, depends on our interest. In a distributed database system, the local state may
consist of only those records that form part of the database and excludes temporary records used in computation.
Global State in a Single Process Application
Assume that we have the following code:x = y = z =0;x = z+1;y = x;z = y
The state can be represented as the values of x,y,z. The sequence of states for the execution of the above
program is the following:000, 100, 110, 111
Deterministic Computation At any point in computation there is at most one event
that can happen next. There is only one sequence of states for the same
input.
Global State for a Multiple Process Application
Say in process P1 you have a code segment as follows:
1.1 x = 5;1.2 y = 10*x;1.3 send(y,P2);
1.4 z = 8;
Say in process P2 you have a code segment as follows:
2.1 a=8;2.2 b=10*a;2.3 rcv(c,P1);
2.4 b = b+c;
Global State for a Multiple Process Application
Non-Deterministic Computation At any point in computation there can be
more than one event that can happen next There is more than one possible
sequence of states for the same input. Assume that a state is represented as
(x,y,z,a,b,c). Observation: In this example, regardless
of the sequence, eventually all sequences reach these global states (5,50,0,8,80,0), (5,50,0,8,80,50)
Global State for Multiple Process Application
0,0,0,0,0,0
5,0,0,0,0,0
5,0,0,8,0,0
5,0,0,8,80,0
0,0,0,8,0,0
5,0,0,8,0,0
5,50,0,8,0,0
5,50,0,8,80,50
5,50,8,8,80,50
5,50,8,8,80,130
5,50,0,8,80,130
5,50,8,8,80,130
5,50,0,8,80,0
A Non-Deterministic Computation
Each of the paths represent a possible sequence of events that could have occurred during execution. This assumes that the same inputs were
applied. Different paths occur for different reasons that
include machines that have different loads at different runs, the network load at different runs, etc;
The different paths correspond to a different sequence of global states that the distributed program goes through.
Viewing Global States
A process in a distributed system has only a limited view of the system, which consists of its local state and the messages it has sent or received.
Given this, is it possible for processes in the system to determine a global system state?
Instantaneous recording not possible No global clock: Distributed recording of local
states cannot be synchronized based on time. Random network delays; What messages are
in transit?
Viewing Global States
The main difficulty is that global state computation requires computation and communication, and by the time such effort is completed, the global state has changed.
A snapshot of the system is a single configuration of the system. It is not necessarily a global state that has actually occurred, but one that could have occurred during the computation.
A Naive Snapshot Algorithm
Processes record their state at any arbitrary point
A designated process collects these states
Simple, but not correct.
Example of Naive Snapshot Algorithm in Use
P records its state
P Q
X = 5
Example of Naive Snapshot Algorithm in Use
P Q
m
Let m be a message to update a variable Y in Q by incrementing it by the value of X.
Example of Naive Snapshot Algorithm in Use
Q records its state after receiving m but before applying the operation.
P Q
m
Example of Naive Snapshot Algorithm in Use
Process P records its state before it actually sends m
Process Q records its state after it has received m.
This is bad. The result is a global state that has
recorded the receive event but no send event.
Distributed Snapshot Represents a state in which the
distributed system might have been in. A distributed snapshot should reflect a
consistent state. If we have recorded that a process P has
received a message from a process Q, then we should have also recorded that process Q had actually send that message.
Otherwise, a snapshot will contain the recording of messages that have been received but never sent.
The reverse condition (Q has sent a message that P has not received) is allowed.
Distributed Snapshot
The notion of a global state can be graphically represented by what is called a cut (example on the next slide).
A cut represents the last event that has been recorded for each process.
Consistent State
a) A consistent cutb) An inconsistent cut
Assumptions for Algorithm
The algorithm to determine global states records process states locally and the states are collected by a designated server.
No failures in channels and processes – exactly once delivery.
Unidirectional channel and FIFO ordered message delivery.
Always a path between two processes. Global snapshot initiation at any process at
any time. No process activity halt during snapshot.
Features of Algorithm
Does not promise us to give us exactly what is there i.e., an actual state
However does give us a consistent state
Algorithm P starts by recording its own local state P subsequently sends a marker along each of its
outgoing channels When Q receives a marker through channel C, its
action depends on whether it had already recorded its local state: Not yet recorded: It records its local state and sends the
marker along each of its outgoing channels Already recorded: The marker on C indicates that the
channel’s state should be recorded: all messages received before this marker and since the last time Q records its own state.
Q is finished when it has received a marker along each of its incoming channels.
Algorithm
a) Organization of a process and channels for a distributed snapshot
Algorithm
b) Process Q receives a marker for the first time and records its local state
c) Q records all incoming messagesd) Q receives a marker for its incoming channel and finishes
recording the state of the incoming channel
Algorithm
How do we know when we are done? If a process Q receives the marker
requesting a snapshot for the first time, it considers the process that sent the marker as its predecessor
When Q completes its part of the snapshot, it sends its predecessor a DONE message.
By recursion, when the initiator of the distributed snapshot has received a DONE message from all its successors, it knows that the snapshot has been completely taken.
Example
Assume two process p1 and p2 connected by two unidirectional channels, c1 and c2.
The two processes trade in widgets. Process p1 sends orders for widgets over c2 to p2,
enclosing payment at the rate of $10 per widget. Process p2 sends widgets along channel c1 to p1. The processes have the initial states shown in the
figure on the next slide. It is assumed that p2 has already received an order for 5 widgets, which it will shortly dispatch to p1.
Example
p1 p2c2
c1
account widgets
$1000 (none)
account widgets
$50 2000
This is the initial state when the algorithm was initiated;Before initiation, the process p2 had received an order for 5 widgets;Thus account has $50
Example
Upon request for global snapshot, process p1 records its state in the actual global state, S0, when p1’s state is <$1000,0>.
p1 p2(empty)<$1000, 0> <$50, 2000>
(empty)
c2
c1
1. Actual Global State S0
Example
Process p1 then emits a marker message over its outgoing channel c2 before it sends the next application-level message: (Order 10, $100) over channel c2.
The system enters actual global state S1.
2. Actual Global state S1
p1 p2(Order 10, $100), M<$900, 0> <$50, 2000>
(empty)
c2
c1
Example
Before p2 receives the marker, it emits an application message (5 widgets) over c1 in response to p1’s previous order, yielding a new actual global state S2.
3. Actual Global State S2 p1 p2(Order 10, $100), M<$900, 0> <$50, 1995>
(five widgets)
c2
c1
Example
Now process p1 receives p2’s message (5 widgets) and p2 receives the marker. p2 records its state as <$50,1995> and that of channel c2 as the empty sequence.
p2 sends a marker message over c1. When process p1 receives p2’s marker
message, it records the state of c1 which is the single message (5 widgets) that it received after it first recorded its state.
Example
The final recorded state is p1: <$1000,0>; p2:<$50,1995>;c1:<(five widgets)>; c2:<>
4. Actual Global state S3
p1 p2(Order 10, $100)<$900, 5> <$50, 1995>
(empty)
c2
c1
Example
The sequence of actual global states was the following: S0: p1: <$1000,0>; p2:<$50,2000>;c1:<>;
c2:<>
S1: p1: <$900,0>; p2:<$50,2000>; c1:<>; c2:<(order 10, $100)>;
S2: p1: <$900,0>; p2:<$50,1995>; c1:<five widgets>; c2:<(order 10, $100)>;
S3: p1: <$900,5>; p2:<$50,1995>; c1:<empty>;c2:<(order 10, $100)>;
Example
A possible sequence of global states is the following: S0: p1: <$1000,0>; p2:<$50,2000>;c1:<>;
c2:<>
S1’: p1: <$1000,0>; p2:<$50,1995>;c1:<(five widgets)>; c2:<>
Recorded State: p1: <$1000,0>; p2:<$50,1995>;c1:<(five
widgets)>; c2:<>
Example
S0
S1
S2
S3
S1’
………
Example
The recorded state is not the same as one of the actual states.
The recorded state is one that was possible in one of the possible sequences of global states.
Why is this useful? What can we do with this? To illustrate we will look at another
example.
Example
What if p1 initiates the snapshot algorithm after it issues a request for the 10 widgets and before it receives the 5 widgets?
This would imply that p2 gets the order for the 10 widgets before the marker. This causes p2 to update its account and number of widgets to $150 and 1985 respectively. p2 will record this state when it receives the marker. p2 sends the 10 widgets.
The state of c1 is <(5 widgets),(10 widgets)> and c2 is marked as empty.
Example 2Say in process P1 you
have a code segment as follows:
1.1 a = 5;1.2 recv(b, P2);
1.3 c = a+ b;1.4 send(c,P2);
Say in process P2 you have a code segment as follows:
2.1 x = 5;2.2 recv(y,P1);
2.3 z = x+y;2.4 send(z, P1);
State includes values of a,b,c,x,y,z and for each process records what that process is waiting (if there is anything).
Example 2
p1 p2c2
c1
a=5b=0c=0
This is the state when the algorithm was initiated;1.1 has been executed but 2.1 has not; marker is sent out after 1.1has been executed but before trying to execute 1.2Channels are empty
x=0y=0z=0
S0
Example 2
p1 p2c2
c1
a=5b=0c=0
2.1 has been executed before it receives the marker from p1
Upon receiving the marker it records that c2 is empty , thatx=5,y=0,z=0 and that it is waiting for a message from p1
It sends out a marker on c1
x=5y=0z=0
S1
p2 is waiting for message from p1
Example 2
p1 p2c2
c1
a=5b=0c=0
p1 receives a marker on c1
It records the state of its incoming channel as empty.The final state recorded is this:a=5,b=0,c=0, x=5,y=0,z=0; p2 is waiting for a message from p1 and channels are empty.
x=5y=0z=0
S2
p2 is waiting for message from p1
Example 2
So far the deadlock has not been detected. Let’s say that after P1 starts waiting for a message
from P2 that it initiates the global snapshot algorithm again.
P1 records its state as a=5,b=0,c=0 and that it is waiting for a message from P2
P1 sends the marker to P2 . P2 records its state as x=5,y=0,z=0, incoming
channel is empty and it is waiting for a message from P1
P2 sends a marker on c1; When P1 receives the marker it records the state of channel c1 as empty
Example 2
At this point the global state recorded includes the following: a=5,b=0,c=0 x=5,y=0,z=0 Both channels are empty P1 is waiting for a message from P2 and P2 is
waiting for a message from P1
This recorded state has enough information to determine that there is a deadlock.
If the algorithm is initiated again the global state still shows a deadlock.
Example 3
p0 p1
p2
M
B
C
A
M
B
A
C
M
Send A
Send B
Record State
Send M
Recv B
Recv A
Recv C
Recv M, Record State, Channel (2,1)empty
Recv M, Record State, Channel (0,1)empty, Send M
Recv M, Record State, Channel (0,1)A
Send C
Stable Properties
The global state computed using the algorithm described can be used for detecting stable properties, e.g.,Distributed deadlock detectionDistributed garbage collection
A stable property is a property that once it holds true in a state Si then it holds true in state Sj where j > i
Stable Properties
Let’s look at deadlock detection. Assume each process has a list waits-for which records the list of processes that the process is waiting for a message from.
The global state includes values of the waits-for list from each process.
Let’s assume two processes p1 and p2. Let p1.waits-for = {p2} and p2.waits-for = {p1}.
This represents a deadlock. If there is a deadlock in state Si then there is a deadlock in Sj (where j > i).
Stable PropertiesS0
S1
S2
S3
S1’
S2’
If deadlock has happened in S3 then it happens in subsequent states.
Summary
No shared memory in distributed systems
Very difficult to determine exactly what a system’s current state is.
Determining the global state of a distributed system can be done by synchronizing all processes so that each collects its own local state along with the messages that are currently in transit.
Recommended