Upload
abraham-curtis
View
229
Download
0
Tags:
Embed Size (px)
Citation preview
The Complexity of Distributed Algorithms
Common measures
Space complexityHow much space is needed per process to run an algorithm?(measured in terms of N, the size of the network)
Time complexityWhat is the max. time (number of steps) needed to complete theexecution of the algorithm?
Message complexityHow many message are exchanged to complete the execution of the
algorithm?
An example
Consider broadcasting in an n-cube (here n=3). Process 0 is the source, broadcasting the value of x[0]. Let x[0] = v. N = total number of processes = 2n = 8
0 1
2
3
4
5
6
7
source
Each process j > 0 has a variable x[j], whose initial value is arbitrary.
Finally, x[0] = x[1] = x[2] = … = x[7] = v
Broadcasting using message passing
{Process 0} m.value := x[0]; send m to all neighbors
{Process i > 0}repeat receive m {m contains the value}; if m is received for the first time then x[i] := m.value; send x[i] to each neighbor j > I else discard m end ifforever
What is the (1) message complexity(2) space complexity per process?
0 1
2
3
4
5
6
7
m
m
m
Number of edgeslog2N
Broadcasting using shared memory
{Process 0} x[0] := v{Process i > 0}repeat
if there exists a neighbor j < i : x[i] ≠ x[j] then x[i] := x[j] (PULL DATA){this is a step} else skip
end ifforever
What is the time complexity?(i.e. how many steps are needed?)
0 1
2
3
4
5
6
7
Arbitrarily large. Why?
Broadcasting using shared memory
{Process 0} x[0] := v{Process i > 0}repeat
if there exists a neighbor j < i : x[i] ≠ x[j] then x[i] := x[j] (PULL DATA){this is a step} else skip
end ifforever
0 1
2
3
4
5
6
7
15
10
12
27
9932
1453
Node 7 can keep copying from 5, 6, 4 indefinitely long before the value in node 0is eventually copied into it.
Broadcasting using shared memory
Now, use “large atomicity”, where
in one step, a process j reads the state x[k]
of each neighbor k < j, and updates x[j]
only when these are equal, but
different from x[j].
What is the time complexity?
How many steps are needed?
The time complexity is now O(n2)
(n = dimension of the cube)
0 1
2
3
4
5
6
7
Time complexity in rounds
Rounds are truly defined for synchronous
systems. An asynchronous round consists
of a number of steps where every eligible
process (including the slowest one)
takes at least one step. How many rounds
will you need to complete the broadcast
using the large atomicity model?0 1
2
3
4
5
6
7
An easier way to measure complexity in rounds is to assume that processes executing their steps in lock-step synchrony
Representing distributed algorithms
Why do we need these?
Don’t we already know a
lot about programming?
Well, you need to capture the notions of
atomicity, non-determinism, fairness etc.
These concepts are not built into languages
like JAVA, C++, python etc!
Syntax & semantics: guarded actions
<guard G> <action A>
is equivalent to
if G then A
(Borrowed from E.W. Dijkstra: A Discipline of Programming)
Syntax & semantics: guarded actions
• Sequential actions S0; S1; S2; . . . ; Sn
• Alternative constructs if . . . . . . . . . . fi
• Repetitive constructs do . . . . . . . . . od
The specification is useful for representing abstract algorithms, not executable codes.
Syntax & semantics
Alternative construct
if G1 S1
[] G2 S2
…
[] Gn Sn
fi
When no guard is true, skip (do nothing). When multiple guards are true, the choice of the action to be executed is completely arbitrary.
Syntax & semantics
Repetitive construct
do G1 S1
[] G2 S2
.
[] Gn Sn
od
Keep executing the actions until all guards are false
and the program terminates. When multiple guards
are true, the choice of the action is arbitrary.
Example: graph coloring
0 1
0
1
{program for process i}
c[i] = color of process i
do
j neighbor(i): c(j) = c(i) c(i) := 1- c(i)
od
Will the above computation terminate?
There are four processes and two colors0, 1. The system has to reach a configuration in which no two neighboring processes have the same color.
Consider another example
program uncertain;define x : integer;initially x = 0do x < 4 x := x + 1[] x = 3 x := 0od
Question. Will the program terminate?
(Our goal here is to understand fairness)
The adversary
A distributed computation can be viewed as a game between the system and an adversary. The adversary may come up with feasible schedules to challenge the system (and cause “bad things”). A correct algorithm must be able to prevent those bad things from happening.
Non-determinism(Program for a token server - it has a single token}repeat
if (req1 token) ∧ then give the token to client1 else if (req2 token) ∧ then give the token to client2
else if (req3 token) ∧ then give the token to client3 forever
Now, assume that all three requests are sent simultaneously.Client 2 or 3 may never get the token! The outcome could have been different if the server makes a non-deterministic choice.
1 23
Token server
Examples of non-determinism
Determinism caters to a specific order and is a special case of non-determinism.
If there are multiple processes ready to execute actions, then who will execute the action first is nondeterministic
Message propagation delays are arbitrary and the order of message reception is non-deterministic
Atomicity (or granularity)
Atomic = all or nothingAtomic actions = indivisible actions
do red message x:= 0 {red action} [] blue message x:=7 {blue action}od
Regardless of how nondeterminism ishandled, we would expect that the value of x will be an arbitrary sequence of 0's and 7's.Right or wrong?
x
Atomicity (continued)
do red message x:= 0 {red action} [] blue message x:=7 {blue action}od
Let x be a 3-bit integer x2 x1 x0, so x:=7 means (x2:=1, x1:= 1, x2:=1), and x:=0 means (x2:=0, x1:= 0, x2:=0)
If the assignment is not atomic, then manyinterleavings are possible, leading to any possible value of x between 0 and 7
x
So, the answer depends on the atomicity of the assignment
Atomicity (continued)
Does hardware guarantee any form of atomicity? Yes! (examples?)
Transactions are atomic by definition (in spite of process failures). Also, critical section codes are atomic.
We will assume that G A is an “atomic operation.” Does it make a difference if it is not so?
y
x
if x ≠ y y:= x fi
if x ≠ y y:= x fi
Atomicity (continued){Program for P}define b: booleaninitially b = true
do b send msg m to Q[] ¬ empty(R,P) receive msg;
b := falseod
Suppose it takes 15 seconds tosend the message. After 5 seconds,P receives a message from R. Will it stop sending the remainder of themessage? NO.
P QR
b
Fairness
Defines the choices or restrictions on the scheduling of actions. No such restriction implies an unfair scheduler. For fair schedulers, the following types of fairness have received attention:
– Unconditional fairness
– Weak fairness
– Strong fairness
Scheduler / demon /adversary
Fairness
Programtest
define x : integer
{initial value unknown}
do true x : = 0
[] x = 0 x : = 1
[] x = 1 x : = 2
od
An unfair scheduler may never schedule the second (or the third actions). So, x may always be equal to zero.
An unconditionally fair scheduler will eventually give every statement a chance to execute without checking their eligibility. (Example: process scheduler in a multiprogrammed OS.)
Weak fairness
Program test
define x : integer
{initial value unknown}
do true x : = 0
[] x = 0 x : = 1
[] x = 1 x : = 2
od
• A scheduler is weakly fair, when it eventually executes every guarded action whose guard becomes true, and remains true thereafter
• A weakly fair scheduler will eventually execute the second action, but may never execute the third action. Why?
Strong fairness
Programtest
define x : integer
{initial value unknown}
do true x : = 0
[] x = 0 x : = 1
[] x = 1 x : = 2
od
A scheduler is strongly fair, when it eventually executes every guarded action whose guard is true infinitely often.
The third statement will be executed under a strongly fair scheduler. Why?
Study more examples to reinforce these concepts