Randomized Algorithms Morteza ZadiMoghaddam Amin Sayedi

Randomized Randomized AlgorithmsAlgorithms

Morteza ZadiMoghaddamMorteza ZadiMoghaddam

Amin SayediAmin Sayedi

Types of Randomized algorithmsTypes of Randomized algorithms

Las VegasLas Vegas Monte CarloMonte Carlo

Las VegasLas Vegas

Always gives the true answer.Always gives the true answer. Running time is random.Running time is random. Running time is bounded.Running time is bounded. Quick sort is a Las Vegas algorithm.Quick sort is a Las Vegas algorithm.

Monte CarloMonte Carlo

It may produce incorrect answer!It may produce incorrect answer! We are able to bound its probability.We are able to bound its probability. By running it many times on By running it many times on

independent random variables, we independent random variables, we can make the failure probability can make the failure probability arbitrarily small arbitrarily small at the expense of at the expense of running timerunning time..

Monte Carlo ExampleMonte Carlo Example

Suppose we want to find a number Suppose we want to find a number among n given numbers which is among n given numbers which is larger than or equal to the median.larger than or equal to the median.


Suppose ASuppose A11 < … < A < … < Ann..

We want AWe want Aii, such that i ≥ n/2., such that i ≥ n/2.

It’s obvious that the best deterministic It’s obvious that the best deterministic algorithm needs O(n) time to algorithm needs O(n) time to produce the answer.produce the answer.

n may be very large! n may be very large! Suppose n is 100,000,000,000 !Suppose n is 100,000,000,000 !


Choose 100 of the numbers with Choose 100 of the numbers with equal probabilityequal probability. .

find the maximum among these find the maximum among these numbers.numbers.

Return the maximum.Return the maximum.


The running time of the given The running time of the given algorithm is O(1).algorithm is O(1).

The probability of Failure is 1/(2The probability of Failure is 1/(2100100).). Consider that the algorithm may Consider that the algorithm may

return a wrong answer but the return a wrong answer but the probability is very smaller than the probability is very smaller than the hardware failure or even an hardware failure or even an earthquake!earthquake!

Monte CarloMonte Carlo

Suppose the output is Yes or No.Suppose the output is Yes or No. One sided error.One sided error. Two sided error.Two sided error.

RP ClassRP Class( randomized polynomial )( randomized polynomial )

Bounded polynomial time in the Bounded polynomial time in the worst case.worst case.

If the answer is Yes; Pr[ return Yes] > If the answer is Yes; Pr[ return Yes] > ½.½.

If the answer is No; Pr[ return Yes] = If the answer is No; Pr[ return Yes] = 0.0.

½ is not actually important.½ is not actually important.

PP ClassPP Class( ( probabilistic polynomial probabilistic polynomial ))

Bounded polynomial time in worst case.Bounded polynomial time in worst case. If the answer is Yes; Pr[ return Yes] > ½.If the answer is Yes; Pr[ return Yes] > ½. If the answer is No; Pr[ return Yes] < ½.If the answer is No; Pr[ return Yes] < ½. Unfortunately the definition is weak Unfortunately the definition is weak

because the distance to ½ is important because the distance to ½ is important but is not considered.but is not considered.

Routing ProblemRouting Problem

There are n computers.There are n computers. Each computer has a packet.Each computer has a packet. Each packet has a destination D(i).Each packet has a destination D(i). Packets can not follow the same Packets can not follow the same

edge simultaneously.edge simultaneously. An An oblivious algorithm oblivious algorithm is required.is required.


For any deterministic oblivious For any deterministic oblivious algorithm on a network of N nodes algorithm on a network of N nodes each of outdegree d, there is an each of outdegree d, there is an instance of permutation routing instance of permutation routing requiring (N/d) requiring (N/d) ½½. .


Pick random intermediate destination.Pick random intermediate destination. Packet i first travels to the Packet i first travels to the

intermediate destination and then to intermediate destination and then to the final destination.the final destination.

With probability at least 1-(1/N), every With probability at least 1-(1/N), every packet reaches its destination in 14n packet reaches its destination in 14n of fewer steps in Qof fewer steps in Qnn..

The expected number of steps is 15n.The expected number of steps is 15n.

Maximum SatisfiabilityMaximum Satisfiability

You have m clauses and n boolean You have m clauses and n boolean variables.variables.

Each clause contains some of Each clause contains some of variables or some of complements.variables or some of complements.

A clause is satisfied if at least one of A clause is satisfied if at least one of it’s variables are satisfied.it’s variables are satisfied.

We want to set the variables such that We want to set the variables such that the number of satisfied clauses is the number of satisfied clauses is maximized.maximized.

Example for Maximum SatExample for Maximum Sat

There are 3 variables A, B and C.There are 3 variables A, B and C. MM11 = (A) or (B) = (A) or (B) MM22 = (A) or (not B) or (not C) = (A) or (not B) or (not C) MM33 = (C) = (C) MM44 = (B) or (not C) = (B) or (not C) MM55 = (not C) = (not C)

Example of Maximum SatExample of Maximum Sat

Set A = TrueSet A = True Set B = TrueSet B = True Set C = FalseSet C = False

Four of the clauses are satisfied.Four of the clauses are satisfied.

Maximum SatMaximum Sat

This problem is a famous problem This problem is a famous problem which has no polynomial time which has no polynomial time algorithm yet. It’s NP-hard.algorithm yet. It’s NP-hard.


For any set of m clauses, there is For any set of m clauses, there is truth assignment for the variables truth assignment for the variables that satisfies at least m/2 clauses.that satisfies at least m/2 clauses.


Let ZLet Zi i =1 if the i-th clause is satisfied =1 if the i-th clause is satisfied and 0 otherwise.and 0 otherwise.

Set the variables in a random way.Set the variables in a random way. The probability of a clause with k The probability of a clause with k

variables to be true is 1- (1/(2variables to be true is 1- (1/(2kk)) >= ½.)) >= ½. So E[ZSo E[Z11]+…+E[Z]+…+E[Zmm] >= ½.] >= ½. Thus there exist at least one Thus there exist at least one

assignment such that Zassignment such that Z11+…+Z+…+Zmm >= ½. >= ½.

Maximum Sat algorithmMaximum Sat algorithm

This problem is NP-hard so we seek for This problem is NP-hard so we seek for approximation algorithms.approximation algorithms.

We have an algorithm that produces an We have an algorithm that produces an answer which is at least ½ of the best answer which is at least ½ of the best answer.answer.

If all clauses have at least 2 literals then If all clauses have at least 2 literals then we have an algorithm that produces an we have an algorithm that produces an answer which is at least ¾ of the best one.answer which is at least ¾ of the best one.


We want to maximize ZWe want to maximize Z11+…+Z+…+Zmm.. We have some inequalities:We have some inequalities: ∑∑ yyii (if X (if Xii is in Z is in Zjj and is uncomplemented) and is uncomplemented) ∑ ∑ (1-(1-yyii) (if X) (if Xii is in Z is in Zjj and is complemented). and is complemented). This inequality must be hold:This inequality must be hold: ∑∑ yyii + + ∑ (1-∑ (1-yyii) >= Z) >= Zjj This problem could be solved using integer This problem could be solved using integer

linear programming.linear programming. We have to use linear programming.We have to use linear programming.


Solve the problem using linear Solve the problem using linear programming.programming.

You get a real number for each yi or zYou get a real number for each yi or z ii.. Assign XAssign Xii true with the probability y true with the probability yii.. The expected number of clauses that The expected number of clauses that

are satisfied is (1- 1/e) of the best are satisfied is (1- 1/e) of the best answer.answer.


Using both algorithms and choosing Using both algorithms and choosing the better answer gives us an answer the better answer gives us an answer which is at least ¾ of the best which is at least ¾ of the best answer!!! Which is better than ½ and answer!!! Which is better than ½ and 1- 1/e.1- 1/e.

2-Sat2-Sat

Every clause has at most 2 literals.Every clause has at most 2 literals. We want to check if all clauses can be satisfied.We want to check if all clauses can be satisfied. It has polynomial algorithm.It has polynomial algorithm. Assign random values to the variables.Assign random values to the variables. If all clauses are satisfied we are finished.If all clauses are satisfied we are finished. If there is an unsatisfied clause, the value of If there is an unsatisfied clause, the value of

one of it’s literals is different from the best one of it’s literals is different from the best answer.answer.

Change the value of one of the variables in this Change the value of one of the variables in this clause. You may make a good change or bad clause. You may make a good change or bad one.one.

2-Sat2-Sat

You are walking on a path.You are walking on a path. If you are on 0 you go to 1.If you are on 0 you go to 1. If you are on i you go to i+1 or i-1 If you are on i you go to i+1 or i-1

with equal probability.with equal probability. The expected number of steps to The expected number of steps to

reach the end of the path is O(nreach the end of the path is O(n22).). So the given algorithm is O(nSo the given algorithm is O(n33).).

Graph ConnectivityGraph Connectivity

You want to check if two vertices u and v You want to check if two vertices u and v are in the same connected component.are in the same connected component.

Start a random walk from v.Start a random walk from v. Have a random walk of length 2nHave a random walk of length 2n33.. If you haven’t visited u, the probability of u If you haven’t visited u, the probability of u

to be in this component is less than ½.to be in this component is less than ½. By repeating this algorithm, you can make By repeating this algorithm, you can make

the probability of failure arbitrarily small.the probability of failure arbitrarily small.

Graph ConnectivityGraph Connectivity

Running time of algorithm is O(nRunning time of algorithm is O(n33).). Required space is O(logn).Required space is O(logn).

Diameter of a Point SetDiameter of a Point Set

You want to find the diameter of set of You want to find the diameter of set of n points in the space.n points in the space.

Suppose I(x) is the convex body formed Suppose I(x) is the convex body formed by the intersection of n sphere centered by the intersection of n sphere centered at n points with radius x.at n points with radius x.

F(p) is distance between p and the point F(p) is distance between p and the point in the set that is farthest from p.in the set that is farthest from p.


Consider I(x) when x=F(p).Consider I(x) when x=F(p). For any q in S, if q is in I(x) then For any q in S, if q is in I(x) then

F(q)<F(p).F(q)<F(p). And if q is not in I(x) then F(p)<F(q).And if q is not in I(x) then F(p)<F(q).


Pick a point p in s at random. Pick a point p in s at random. Computer F(p). [O(n)]Computer F(p). [O(n)]

Set x=F(p). Compute I(x). [O(n logn)]Set x=F(p). Compute I(x). [O(n logn)] Find the points outside I(x). Call this Find the points outside I(x). Call this

set T. [O(n logn)]set T. [O(n logn)] If T is empty return x as the answer, If T is empty return x as the answer,

else continue on T.else continue on T.


The running time of algorithm above The running time of algorithm above is O(n log n).is O(n log n).

In each step, all points that have In each step, all points that have smaller F(x) than the chosen point smaller F(x) than the chosen point are removed.are removed.

All-Pairs Shortest PathsAll-Pairs Shortest Paths

Let G(V,E) be an undirected, Let G(V,E) be an undirected, connected graph with V={1,…,n} and connected graph with V={1,…,n} and |E|=m.|E|=m.

The adjacency matrix A is an n The adjacency matrix A is an n n 0-1 n 0-1 matrix with Amatrix with Aijij=A=Ajiji=1 if and only if the =1 if and only if the edge (i,j) is present in E.edge (i,j) is present in E.

We are going to compute matrix D We are going to compute matrix D which Dwhich Dijij equals the length of a equals the length of a shortest path from vertex i to vertex j.shortest path from vertex i to vertex j.

All-Pairs Distances All-Pairs Distances

Z AZ A22

Compute matrix A’ such that A’Compute matrix A’ such that A’ijij=1 if and =1 if and only if i ≠ j and (Aonly if i ≠ j and (Aijij=1 or Z=1 or Zijij>0)>0)

If A’If A’ijij=1 for all i ≠ j then return D = 2A’-A.=1 for all i ≠ j then return D = 2A’-A. Recursively compute the APD matrix D’ for Recursively compute the APD matrix D’ for

the graph G’ with adjacency matrix A’.the graph G’ with adjacency matrix A’. S AD’S AD’ Return matrix D with DReturn matrix D with Dijij=2D’=2D’ij ij if Sif Sijij≥D’≥D’ijijZZii ii , ,

otherwise Dotherwise Dijij=2D’=2D’ijij-1.-1.

APSPAPSP

The APD algorithm computes the The APD algorithm computes the distance matrix for an n-vertex graph distance matrix for an n-vertex graph in time O(MM(n)log(n)) using integer in time O(MM(n)log(n)) using integer matrix multiplication algorithm.matrix multiplication algorithm.

Matrix multiplication algorithm Matrix multiplication algorithm running in time O(nrunning in time O(n2.3762.376).).

Boolean Product Witness Boolean Product Witness MatrixMatrix

Suppose A and B are nSuppose A and B are nn boolean n boolean matrices and P=AB is their product matrices and P=AB is their product under Boolean matrix multiplication.under Boolean matrix multiplication.

A witness for PA witness for Pijij is an index k is an index k {1, {1,…,n} such that A…,n} such that Aikik=A=Akjkj=1. Observe =1. Observe that Pthat Pijij=1 if and only if it has some =1 if and only if it has some witness k.witness k.

BPWMBPWM

W -ABW -AB 2. for t=0,…, 2. for t=0,…, log(n)log(n) do do 2.1. r 22.1. r 2tt

2.2. Repeat 2.2. Repeat 3.77log(n)3.77log(n) times times 2.2.1. Choose random R 2.2.1. Choose random R {1,…,n} with |R|=r . {1,…,n} with |R|=r . 2.2.2. Compute A2.2.2. Compute ARR and B and BR R .. 2.2.3. Z A2.2.3. Z ARRBBRR . . 2.2.4. for all (i,j) do2.2.4. for all (i,j) do

if Wif Wijij < 0 and Z < 0 and Zijij is witness then W is witness then Wijij Z Zijij

3. for all (i,j) do3. for all (i,j) do if Wif Wijij < 0 then find witness W < 0 then find witness Wijij by brute force. by brute force.

BPWMBPWM

The BPWM algorithm is a Las Vegas The BPWM algorithm is a Las Vegas algorithm for the BPWM problem with algorithm for the BPWM problem with expected running time expected running time O(MM(n)logO(MM(n)log22(n)).(n)).

The probability that no witness is The probability that no witness is found for Pfound for Pijij before the end of Step 2 before the end of Step 2 is at mostis at most

(1-1/2e)(1-1/2e)3.77log(n)3.77log(n) 1/n. 1/n.

Determining Shortest PathDetermining Shortest Path

A successor matrix S for an n-vertex A successor matrix S for an n-vertex graph G is an n graph G is an n n matrix such that n matrix such that SSijij is the index of a neighbor of vertex is the index of a neighbor of vertex i that lies on a shortest path from i to i that lies on a shortest path from i to j.j.

APSPAPSP

Compute the distance matrix Compute the distance matrix D=APD(A).D=APD(A).

for s={0,1,2} dofor s={0,1,2} do Compute 0-1 matrix DCompute 0-1 matrix Dkjkj

(s)(s)=1 if and only if =1 if and only if DDkjkj+1 = s (mod 3)+1 = s (mod 3)

Compute the witness matrix Compute the witness matrix WW(s)(s)=BPWM(A,D=BPWM(A,D(s)(s)).).

Compute successor matrix S for G.Compute successor matrix S for G.

APSPAPSP

Algorithm APSP computes the Algorithm APSP computes the successor matrix for an n-vertex successor matrix for an n-vertex graph G in expected time graph G in expected time O(MM(n)logO(MM(n)log22(n)).(n)).

Algorithm contractAlgorithm contract

H GH G While H has more than 2 vertices doWhile H has more than 2 vertices do

Choose an edge (x,y) uniformly at Choose an edge (x,y) uniformly at random from the edges in H.random from the edges in H.

F F F F {(x,y)}. {(x,y)}. H H / (x,y).H H / (x,y).

(C,V/C) the sets of vertices (C,V/C) the sets of vertices corresponding to the two meta-corresponding to the two meta-vertices in H=G/F.vertices in H=G/F.

FastCutFastCut

n |v|n |v| if n if n 6 then compute min-cut of G by brute- 6 then compute min-cut of G by brute-

force enumeration elseforce enumeration else t t 1+n/1+n/22 Using Algorithm Contract, perform two Using Algorithm Contract, perform two

independent contraction sequences to obtain independent contraction sequences to obtain graphs Hgraphs H11 and H and H22 each with t vertices. each with t vertices.

Recursively compute min-cuts in each of HRecursively compute min-cuts in each of H11 and and HH2.2.

Return the smaller of the two min-cuts.Return the smaller of the two min-cuts.

FastcutFastcut

Algorithm Fastcut succeeds in finding Algorithm Fastcut succeeds in finding a min-cut with probability a min-cut with probability (1/log(n)).(1/log(n)).

Algorithm Fastcut runs in O(nAlgorithm Fastcut runs in O(n22log(n)) log(n)) time and uses O(ntime and uses O(n22) space.) space.

MSTMST

Finding MST in a graph with n Finding MST in a graph with n vertices and m edges has a Las vertices and m edges has a Las Vegas algorithm which has the Vegas algorithm which has the expected running time O(n+m).expected running time O(n+m).

But We don’t have enough time to But We don’t have enough time to Explain it !!!Explain it !!!

Research problemsResearch problems

Devise an algorithm for the all-pairs Devise an algorithm for the all-pairs shortest paths problem that does not use shortest paths problem that does not use matrix multiplication and runs in time O(nmatrix multiplication and runs in time O(n3-3-

) for a positive constant ) for a positive constant .. Devise an algorithm for computing the Devise an algorithm for computing the

diameter of an unweighted graph that diameter of an unweighted graph that does not use matrix multiplication and does not use matrix multiplication and runs in time O(nruns in time O(n3-3-) for a positive constant ) for a positive constant ..

Research problemsResearch problems

Devise a Las Vegas or a deterministic Devise a Las Vegas or a deterministic algorithm for min-cuts with running algorithm for min-cuts with running time close to O(ntime close to O(n22).).

Is there a randomized algorithm for Is there a randomized algorithm for min-cuts with expected running time min-cuts with expected running time close to O(m)? close to O(m)?

Documents

Randomized Algorithms Morteza ZadiMoghaddam Amin Sayedi