1

Randomized Algorithms CHAPTER 5: The Probabilistic Method

2

content

basic principles of the probabilistic methodmaximum satisfiability (MAX-SAT) problem. expanding graphs problem of oblivious routingLovasz Local Lemma, method of conditional probabilities

3

Two principles 1) Any random variable assumes at least one value that is no smaller

than its expectation, and at least one value that is no greater than its expectation. We know of many intuitive versions of this principle in real life - for instance, if we are told that the average annual income of theoretical computer scientists is $20,000, we know that there is at least one theoretical computer scientist whose income is $20,000 or greater.

2) If an object chosen randomly from a universe satisfies a property with positive probability, then there must be an object in the universe that satisfies that property. For instance, if we were told that a ball chosen randomly from a bin is red with probability 1/3, then we know that the bin contains at least one red ball.

4

Example 1

Theorem 1.2: The expected size of the autopartition produced by RandAuto is O(n log n).

Example 5.1: for any set of n disjoint line segments in the plane, there is always an autopartition of size O(n log n). This follows directly from the fact that if we were to run the RandAuto algorithm, the random variable defined to be the size of the autopartition can assume a value that is no more than its expectation; thus, there is an autopartition of this size on any instance.

Example 2

5

Theorem 2.1: Given any instance of T2k, the expected number of steps for the game tree randomized algorithm is at most 3 k.Since n = 4k the expected running time of our randomized algorithm is nlog

4 3,which we bound by n0.793.

Example 5.2: Any algorithm for game tree evaluation that produces the correct answer on every instance develops a certificate of correctness: for each instance, it can exhibit a set of leaves whose values together guarantee the value it declares is the correct answer.By Theorem 2.1, the expected number of leaves on any instance of T2k is at most n0.793, where n = 22k. It follows that on any instance of T2k, there is a set of n0.793 leaves whose values certify the value of the root for that instance. Note that we assert the existence of such a certificate with certainty, even though the technique used for establishing it was probabilistic.

6

Example 3

Example 5.3: We saw that for every n x n 0-1 matrix A, for a randomly chosen vector b Є {-1, +1}n, we have ||Ab||∞ ≤ 4√n lnn, with probability at least 1 -2/n.From this we may conclude that for every such matrix A, there always exists a vector b Є {-1,+1}n such that ||Ab||∞ ≤ 4√n lnn.

Example 4

7

finding a large cut in a graph: Given an undirected graph G(V,E) with n vertices and m edges, we wish to partition the vertices of G into two sets A and B so as to maximize the number of edges (u, v) such that u Є A and v Є B. The problem of finding an optimal max- cut is NP_hard;.

Theorem 5.1: For any undirected graph G(V,E) with n vertices and m edges, there is a partition of the vertex set V into two sets A and B such that |{(u,v) Є E | u Є A and v Є B}| ≥ m/2.proof: Consider the following experiment. Each vertex of G is independently and equiprobably assigned to either A or B. For an edge (u,v), the probability that its end-points are in different sets is 1/2. By linearity of expectation, the expected number of edges with end-points in different sets is thus m/2. It follows that there must be a partition satisfying the theorem

8

Question:Having shown existence of a combinatorial object using the probabilistic method, can we find the object efficiently?

Answer: 1) we have a deterministic polynomial-time algorithm that finds the

combinatorial object whose existence is guaranteed by the probabilistic method.

2) we have a randomized polynomial-time algorithm that works with high probability.

3) we have a deterministic or randomized algorithm, but that is non-uniform. 4) we have instances where we know of no efficient algorithm for finding the

object in question.

5.2. Maximum Satisfiability

9

Maximum satisfiability : rather than decide whether there is an assignment that satisfies all the clauses, we instead seek an assignment that maximizes the number of satisfied clauses. This problem, called the MAX-SAT problem, is known to be NP-hard,

Note: We may assume without loss of generality that no clause contains both a literal and its complement, since such clauses are satisfied by any truth assignment. m/2 is the best possible universal guarantee, since the instance may consist of the two clauses x and x, in which case no better guarantee is possible.

10

Max SAT

Theorem 5.2: For any set of m clauses, there is a truth assignment for the variables that satisfies at least m/2 clauses.

Proof: set each variable to true or false independently and equiprobably. For 1 < i < m, let Zi = 1 if the ith clause is satisfied and 0 otherwise. For any clause containing k literals, the probability that it is not satisfied by this random assignment is 2-k,the probability that a clause with k literals is satisfied is 1 — 2-k ≥ 1/2, implying that E[Zi] ≥ 1/2 for all i. The expected number of clauses satisfied by this random assignment is ∑i=0 m E[Zj] ≥ m/2. Thus, there exists at least one assignment of values to the variables for this.

11

Max SAT

Given an instance I, m*(I) : the maximum number of clauses that can be satisfiedmA(I) : the number of clauses satisfied by an algorithm A. The performance ratio of an algorithm A = mA(I)/m*(I). If A achieves a performance ratio of a, we call it an a-approximation algorithm. For a randomized algorithm A, we replace mA(I) by E[mA(I)]

Theorem 5.2 yields an algorithm whose performance guarantee is 1 – 2-k, where every clause contains at least k literals . It follows that we have a randomized 3/4-approximation algorithm for in- instances of MAX-SAT in which every clause has at least two literals.How we can yield a randomized 3/4-approximation in clauses with one literal?

12

Second Algorithm

formulating the problem as an integer linear program, solve the linear programming relaxation, and then using the randomized rounding technique. For each clause Cj in the instance, we associate an indicator variable zj Є {0,1} to indicate whether or not that clause is satisfied. For each variable xi, we use an indicator variable yi to indicate the value assumed by that variable; thus yi = 1 if the variable xi is set true, and yi = 0 otherwise. Let Cj

+ be the set of indices of variables that appear in the uncomplemented form in clause Cj, and Cj

- be the set of indices of variables that appear in the complemented form in clause Cj.;

13

formulating the MAX-SAT problem

we allow yi and Zj to assume real values in the interval [0,1]. Let ŷi be the value obtained for variable yi by solving this linear program, and let žj be the value obtained for Zj .first : the expected number of clauses satisfied by linear program is at least (1 — 1/e)∑j žj Second: the number of clauses satisfied by the better of these two solutions is at least 3/4 ∑j žj

)()1(

),}(1,0{,

max1

jzyy

subject

jizywhere

z

jj ci jici i

ji

m

jj

14

Max SAT

Lemma 5.3: Let Cj be a clause with k literals. The probability that it is satisfied by randomized rounding is at least βk žj that βk= 1-(1-1/k)k

proof: Since we are focusing on a single clause Cj, we may assume without loss of generality that all the variables contained in it appear in uncomplemented form. So we have Clause Cj remains unsatisfied by randomized rounding only all yi’s is rounded to 0. this occurs with probability . remains to show that is minimize when it suffices to show that 1-(1 - z/k)k ≥βk z for all positive integers k and 0≤ z ≤1function f(x) = 1-(1 - x/k)k is a concave function, to show that it is never less than a linear function g(x) over the interval [0,1], it suffices to verify the inequality at the end-points x = 0 and x = 1 . Applying this principle to the linear function g(z) = βk z , the lemma follows

jk zyy ˆˆ...ˆ1

k

1ii )y-(1

jk z)y-(11k

1ii

k

1ii )y-(11 kzy ji /ˆˆ

Note: βk= 1-(1-1/k)k ≥1-1/e so the expected number of clauses satisfied by randomized rounding is at least 1-1/e ∑j žj

15

Comparing two algorithms

n1 denote the expected number of clauses that are satisfied when each variable is independently set to 1 with probability ½ and n2 denote the exexpected number of clauses that are satisfied when we use the linear programming followed by randomized rounding.

16

17

5.3. Expanding Graphs

expanding graph is a graph in which the number of neighbors of any set of vertices S is larger than some positive constant multiple of |S|.

Definition 5.1: An (n,d,α,c) OR-concentrator is a bipartite multigraph G(L, R, E), with the independent sets of vertices L and R each of cardinality n, such that 1. Every vertex in L has degree at most d. 2. For any subset S of vertices from L such that |S| < αn, there are at least c |S| neighbors in Rit is desirable to have d as small as possible and c as large as possible

18

Expanding Graphs

Theorem 5.6: There is an integer n0 such that for all n > n0, there is an (n, 18,1/3,2) OR-concentrator

Proof: Consider a random bipartite graph on the vertices in L and R, in which each vertex of L chooses its neighbors by sampling (with replacement) d vertices independently and uniformly from R and we discard all but one copy of multiple edges.Let εs denote the event that a subset of s vertices of L has fewer than cs neighbors in R. We will first bound Pr[ εs], and then sum Pr [εs] over the values of s no larger than αn to obtain an upper bound on the probability that the random graph fails to be an OR-concentrator with the parameters we seek.

19

Expanding Graphs

Fix any subset S L of size s, and any subset T R of size cs. There are (n

s) ways of choosing S, and (ncs) ways of choosing T. The probability that T

contains all of the at most ds neighbors of the vertices in S is (cs/n)ds. Thus, the probability of the event that all the ds edges emanating from some s vertices of L fall within any cs vertices of R is bounded as follows

for α = 1/3 and using s < αn, we have

Let r = 2/3I8 *3e3, and note that r < 1/2. We obtain that :

scdccddscss

kneknds

i cens

ncs

csne

sne

ncs

csn

sn k

][]Pr[ 11

)/(

s

cd

s

cds

cdccd

i eecce

3

182,1811

1

)3(32)3(

331]Pr[

11

]Pr[11

r

rrs

s

si

20

: میشود اثبات زیر گراف وجود احتماالتی متد باهای بخش شامل بخشی دو و nبا Lگراف هر 2log2nبا Rراس درجه که راس

:d = 2log2n(4 log2 n)/nحداکYثر Lراس دارد را زیر خصوصیت دو و استبا 1. مجموعه زیر در n/2هر به Lراس 2log2حداقل

n - n راسR. است وصلراس 2. از Rهیچ بیش log2 12به

n راسL. نیست وصل

الگوریتم رشته Aمانند RPیک یک وجود تشخیص زبان xبرای ازای Lدر r≥ 0به≤n-1:

•X اعداد از نیمی حداقل ازای به باشد رشته ، rعضو A(x,r) جواب بدهد 1باید•X مقادیر همه ازای به نباشد زبان rعضو ، A(x,r) بدهد صفر جواب باید

فصل رندم با tدر 3الگوریتم تکرار هر در خطای 2log nتکرارو احتمال با بیت1/t. میدهد جواب

تکرار تعداد با فصل این با 1در log2وn خطای احتمال با میخواهیم به n/1بیت

. برسیم جواب

21

: گراف این خاصیت به توجه بابا 1. مجموعه زیر در n/2هر به Lراس 2log2حداقل

n - n راسR. است وصلاز Rراس n/2log2nتعداد یک هیچ به که دارد داده n/2امکان تخصیص اعداد مقدار

به نباشد Lشده وصلمرتبه از کمتر خطا احتمال .n/n log nپس است

الگوریتم:در راس هر ازای بازه ]Lبه در عدد در[ n-1, 0یک راس هر ازای به عدد Rو یک

بازه ] میدهد[ 2log2n, 0در تخصیصlog2با

n از راس یک در vمانند Rبیت راس آن با متصل مقادیر و شود می انتخابL های ورودی عنوان به .A(x,r)به rرا میدهیم همه Xاگر ازای به روش این نباشد رشته احتمال rعضو و میدهد صفر هاجواب

. ندارد خطاX مقادیر از نیمی ازای به باشد زبان باید rعضو ،A(x,r) . با بدهد یک جواب باید

احتمالی همه A(x,r)چه ازای میدهد؟ rبه صفر جواب ورودیش های

Routing

22

N_Scheme ازای به که است جایگشتها از ای مجموعه یابی مسیر مسله برایدر شود انتخاب جایگشت یک تصادفی صورت به اگر ها مقصد و مبدا از نمونه هر

از .15nکمتر برسند مقصد به ها بسته همه گام

رساندن برای یابی مسیر مکعب N درمسئله یک در مقصد به گفته nبسته بعYدینظر در رندم بصورت میانی مقصد یک میتوان مقصد و مبدا هر برای که شد

احتمال با کYه از N/1-1گرفت کمتر .15nبا میرسند مقصد به ها بسته همه گاممقصد و مبدا هر میانی های مقصد کردن مشخص برای نیاز مورد رندم بیت تعداد

Nn( میباشد تک(. N=2nبیت تک انتخاب بجای میتوان است زیاد ها بیت تعدادها جایگشت تعYداد ولی شود انتخاب رندم بصورت جایگشت یک میانی های مقصد

( است (.NNزیاد

میشود ثابت احتماالتی متد از طبق رندم صورت به اگر N3جایگشت، NNکهرندم صورت به اگر مقصدها و مبدا اولیه نمونه هر شود،برای انتخاب جایگشت

این احتمال N3از با بسته هر شود، انتخاب جایگشت یک کمتر N/2-1جایگشت در.15nاز میرسد مقصد به گام

نیاز مورد رندم بیت .3nتعداد میباشد

23

The Lovasz Lemma

در که لم قرار این از دارد کاربرد بسیار احتماالتی های روش: است زیر

. ” هم ” داریم بد پیشامدهای از احتمالی فضای کنیم فرضپیشامدها از یک هر دادن رخ احتمال کنید فرض چنین

به pحداکثر حداکثر پیشامد هر عالوه به و پیشامد dباشد . از یک هیچ که این احتمال آنگاه است وابسته دیگر

. است مثبت نیفتد اتفاق بد پیشامدهایو E1,..Enاگر: 5.12نتیجه باشند احتمال فضای های رخداد

هر . i pr[Ei]≤pبرای رخداد هر اگر به حدباشد رخداد dاکثرو باشد وابسته آنگاه ep(d+1) ≤1دیگر pr[∩nباشند

i=1 Ei]>0. است

24

مسئله برای وگاس الس الگوریتم K_SATیکمسئله در K_SATدر حداکثر متغیر هر یک 2k/50اگر میتوان شود ظاهر عبارت

. نشدن ارضا احتمال مسله این در کرد طراحی وگاس الس تصادفی الگوریتمعبارت همه d=k2k/50و p=2-kیک اینکه احتمال و برقرار لم شرایط پس است

است مثبت شوند ارضا مناسب عبارات مقداردهی یک احتماالتی متد طبق پس. دارد الگوریتم:وجود

: است نامشخص متغYیر هر مقدار ابتدا در یک فازمیشود یک یا صفر آن مقدار و شده انتخاب رندم به متغیر یک مرحله هر در

عبارت یک در اگر مرحله هر آن k/2در و باشد شده دهی مقدار آن متغیرآن های متغیر بقیه و شده مارکدار عYبارت آن نشده ارضا هنوز عبارت

. میشوند گذاشته کنار Yنامشخص: دوم فاز

وبا میگیریم نظر در را مارکدار های عبارت و نامشخص های متغیر فقطسایز با مجزا های بخش به ها عبارت این زیاد اند O(log n)احتمال شده تبدیل

عبارت همه تا میشود چک ها متغیر های دهی مقدار تک تک بخش هر برای حال. شوند ارضا مارکدار های

25

پذیرند؟ ارضا مارکدار عبارتهای آیا دوم مرحله اجرای زمان آیا و است چقدر Yبخش هر سایز

است؟ ای جمله چند ارضا: 1پاسخ احتمال و میباشد برقرار را همه لم شرایط

است مثبت بخش هر احتماالتی شدن متد طبق هر پس. است پذیر ارضا بخش

احتمال: 2پاسخ سایز o(1)-1با بخش و O(log n)هر داردنیز ) Yبخش هر نامشخص های متغیر و O( log nتعداد است

.2O(log n)باید شود چک دهی مقدار :تکرار اول فاز بود زیاد ها بخش از یکی سایز اگر نکته

میشود.

26

شرطی احتمال متدکرد . استفاده شرطی احتمال متد از میتوان مسایل برخی برای

بردار یافتن مسئله برای مثال درخت شود Ab||<4√n ln n||که bبرای. محاسبه بردار یک برگ هر تا ریشه مسیر بگیرید نظر در و bرا میدهد را

آن با متناظر دادن نادرست جواب احتمال برگ هر در است bمیدانیم صفر یا. یک یا و

یک به رسیدن میانی bاحتمال نود هر برای چپ aبد فرزند به توجه با و cراراست :dفرزند میکنیم مشخص

P(a)= (p(c) + p(d))/2کنید سطر Eiفرض که باشد ماتریس iرخدادی در Aام بیشتر bکه شود ضرب

:n ln n√4از میکنیم تعریف باشد

از استفاده بجای از p(a)حال مرحله هر الگوریتم P^(A)در که میکنیم استفاده. میشود محاسبه ای جمله چند زمان در و است معYین الگوریتم یک

)|()(ˆ aEprapi i