Algorithms for optimization problems typically go through a

CHAPTER 16

Greedy Algorithms

Dec 2015

eng-hs.netنبزةاوةنعلىنلةمةلكنق نسنلاتناكمنقعنمجاسا نمننستلام نسست نلكتلونسة نلا

61862999باكستودابنأستفلنبةاسننناكجمعة ناكوئةستة أننلصنةونن61941842أما ناكهسدست نأستفلنصاكنننوسة نناكفوعفينتلنمننلصنةوننمجاسا ناكسنلاتنملنفوةن

net.hs-eng ,com.hs-engنباكمنقعةننمجاسا ناكسنلاتنملنفوةن hs.com-enginfo@ (2 )ن9 4444 260ن .نحمادةنشعبان

Algorithms for optimization problems typically go through a

sequence of steps, with a set of choices at each step.

For many optimization problems, using dynamic programming to

determine the best choices is overkill; simpler, more efficient

algorithms will do.

Definition

A greedy algorithm always makes the choice that looks best at

the moment. That is, it makes a locally optimal choice in the hope

that this choice will lead to a globally optimal solution.

The greedy method is quite powerful and works well for a

wide range of problems. There are many algorithms that can be

viewed as applications of the greedy method, including

minimum-spanning-tree algorithms Dijkstra’s algorithm for

shortest paths from a single source

16.1 An activity-selection problem

Our first example is the problem of scheduling several

competing activities that require exclusive use of a common

resource, with a goal of selecting a maximum-size set of mutually

compatible activities. ،عد ها ب لم نر هار مل الأن أج

عد، ها ب لم نقرأ تب أجمل الك

.أجمل أيام حياتنا لم تأتٍ بعد

Dec 2015




Suppose we have a set S = {a1, a2, ..., an} of n proposed

activities that wish to use a resource, such as a lecture hall, which

can be used by only one activity at a time. Each activity a has a

start time s1 and a finish time fi, where 0 ≤ si < fi < ∞ .

If selected, activity a, takes place during the half-open time

interval [si, fi). Activities ai and aj are compatible if the intervals

[si , fi) and [sj , fj) do not overlap (i.e., ai and aj are compatible if

si ≥ fi or sj ≥ fi ). The activity-selection problem is to select a

maximum-size subset of mutually compatible activities.

For example, consider the following set S of activities, which

we have sorted in monotonically increasing order of finish time:

i 1 2 3 4 5 6 7 8 9 10 11

si 1 3 0 5 3 5 6 8 8 2 12

fi 4 5 6 7 8 9 10 11 12 13 14

For this example, the subset {a3, a9, a11} consists of mutually

compatible activities. It is not a maximal subset, however, since

the subset {a1, a4, a8, a11} is larger. In fact, {a1, a4, a8, a11} is a

largest subset of mutually compatible activities; another largest

subset is {a2, a4, a9, al1}.

لب البشرر ييير ف رب قربا أغ

ية، له نها ليس ليس يه رابح و

.ولا يمكنهم الخروج منه

Dec 2015




We shall solve this problem in several steps.

We start by formulating a dynamic- programming solution to this

problem in which we combine optimal solutions to two

subproblems to form an optimal solution to the original problem.

We consider several choices when determining which

subproblems to use in an optimal solution.

We shall then observe that we need only consider one choice —

the greedy choice— and that when we make the greedy choice,

one of the subproblems is guaranteed to be empty, so that only

one nonempty subproblem remains.

Based on these observations, we shall develop a recursive

greedy algorithm to solve the activity-scheduling problem.

We shall complete the process of developing a greedy solution by

converting the recursive algorithm to an iterative one.

Although the steps we shall go through in this section are more

involved than is typical for the development of a greedy

algorithm, they illustrate the relationship of greedy

algorithms and dynamic programming.

أغلب الناس تبذل جهدها باليد

.الذي يكفب لبقائهم، مجرد بقاء

Dec 2015




The optimal substructure of the activity-selection problem

As mentioned above, we start by developing a

dynamic-programming solution to the activity-selection problem.

Our first step is to find the optimal.

The optimal substructure of this problem is as follows.

Suppose now that an optimal solution Aij to Sij includes activity ak.

Then the solutions Aik to Sik and Akj to Skj used within this optimal

solution to Sij must be optimal as well.

The usual cut-and-paste argument applies, If we had a solution

A’ik to Sik that included more activities than Aik, we could cut out

Aik from Aij and paste in A’ik, thus producing another solution to Sik

with more activities than Aij Because we assumed that Aij is an

optimal solution, we have derived a contradiction.

Similarly, if we had a solution A’kj to Skj with more activities than

Akj, we could replace Akj by A’kj to produce a solution to Sij with

more activities than Aij.

يدوث، نادر ال شبء أف نييا ذلك

اليقيقررررررة أف معاررررررم النرررررراس

.مت اجدوف قط، هذا كل شبء

Dec 2015




Converting dynamic-programming solution to a greedy solution

There is a pattern to the activities that we choose.

Because we always choose the activity with the earliest finish time

in Smi , n+1, the finish times of the activities chosen over all

subproblems will be strictly increasing over time.

Moreover, we can consider each activity just once overall, in

monotonically increasing order of finish times.

The activity that we choose when solving a subproblem is

always the one with the earliest finish time that can be legally

scheduled. The activity picked is thus a “greedy” choice in the

sense that, intuitively, it leaves as much opportunity as possible

for the remaining activities to be scheduled.

That is, the greedy choice is the one that

maximizes the amount of unscheduled

time remaining.

تتسم اليياة دائماً بالغم ض والمشاكل

إذا رأيت الطريق أمامك ممهداً اعلم

.أنه ليس الطريق الصييح

Dec 2015




A recursive greedy algorithm

RECURSIVE-ACTIVITY- SELECTOR (s, f, i, n) 1 m ← i+1 2 while m ≤ n and Sm < fi > Find the first activity in Si,n +1. 3 do m ← m + 1 4 if m ≤ n 5 then return {am } U RECURSIVE-ACTIVITY- SELECTOR(s, f, m, n( 6 else return Ø

An iterative greedy algorithm

GREEDY-ACTIVITY-SELECTOR (s, f)

1 n ← length[s]

2 A ← {a1}

3 i ← 1

4 for m ← 2 to n

5 do if sm ≥ fi

6 then A ← A U {am}

7 i ← m

8 return A

تتك ف اليياة من لياات رحيل

.كثيرة واحدة تل أخرى

Dec 2015




عررد الييرراة بررف أهررداأ جري ررة أحررد ت

.ص ر الم ت لأناس على قيد اليياة

Dec 2015




Exercise 16.1-2

Suppose that instead of always selecting the first activity to finish, we instead select the last activity to start that is compatible with all previously selected activities. Describe how this approach is a greedy algorithm, and prove that is yields an optimal solution.

Solution

The proposed approach - selecting the last activity to start that is compatible with all previously selected activities - is really the greedy algorithm but starting from the end rather than the beginning. Another way to look at it is as follows. We are given a set S = {a1, a2, . . . , an} of activities, where ai = [Si , fi ), and we propose to find an optimal solution by selecting the last activity to start that is compatible with all previously selected activities. Instead, let us create a set S’ = {a1

’ , a2’ , . . . , an

’ }, where ai

’ = [ fi , si ).

That is, ai’ is ai in reverse. Clearly, a subset of {ai1 , ai2 , . . . , aik } ⊆

S is mutually compatible if and only if the corresponding subset

{ a1’ , a2

’ , . . . , ak’ } ⊆ S’ is also mutually compatible.

Thus, an optimal solution for S maps directly to an optimal solution for S’ and vice versa. The proposed approach of selecting the last activity to start that is compatible with all previously selected activities, when run on S, gives the same answer as the greedy algorithm from the text - selecting the first activity to finish that is compatible with all previously selected activities - when run on S’. The solution that the proposed approach finds for S corresponds to the solution that the text’s greedy algorithm finds for S’, and so it is optimal.

خفصة اليكمة: عش وأنت على

.قيد اليياة ولا تمت قبل م تك

Dec 2015




Exercise 16.1-3

Suppose that we have a set of activities to schedule among a large number of lecture halls. We wish to schedule all the activities using as few lecture halls as possible. Give an efficient greedy algorithm to determine which activity should use which lecture hall.

Solution

Move through the activities according to starting and finishing times. Maintain two lists of lecture halls: Halls that are busy at time t and halls that are free at time t. When t is the starting time for some activity schedule this activity to a free lecture hall and move the hall to a the busy list. Similarly, move the hall to the free list when the activity stops. Initially start with zero halls. If there are no halls in the free list, create a new hall.

The above algorithm uses the fewest number of hall possible: Assume the algorithm used m halls. Consider some activity a that was the first scheduled activity in lecture hall m. i was put in the mth hall because all of the m – 1 halls were busy, that is, at the time a is scheduled there are m activities occurring simultaneously. Any algorithm must therefore use at least m halls, and the algorithm is thus optimal.

The algorithm can be implemented by sorting the activities. At each start of finish time we can schedule the activities and move the halls between the lists in constant time. The total time is thus dominated by sorting and is therefore Θ(n lg n).

عدى إخماد شر لا تت حياة أغلب الب

عض حرائق ح لهم والتغلب على ب

.المشاكل الطارئة، هذا كل شبء

Dec 2015




Exercise 16.1-4

Not just any greedy approach to the activity-selection problem produces a maximum-size set if mutually compatible activities. Give an example to show that the approach of selecting the activity of least duration from those that are compatible with previously selected activities does not work. Do the same for the approaches of always selecting the compatible activity that overlaps the fewest other remaining activities and always selecting the compatible remaining activity with the earliest start time.

Solution

Consider the following figures for three examples with

other greedy strategies that go wrong:

Selecting the activity with the least duration from example a will result in selecting the topmost activity and none other. Clearly, this is worse than the optimal solution obtained by selecting the two activities in the second row.

The activity with the minimum overlap in example b is the middle activity in the top row. However, selecting this activity eliminates the possibility of selecting he optimal solution depicted in the second row. Selecting the activity with the earliest starting time in example c will yield only the one activity in the top row.

(c)

(a)

(b)

طالما أف الم ت ليس هد ا ب حد

ه، لماذا نعيش حياة الأم ات، ذات

وكأنه الهدأ الأوحد ب حياتنا؟

Dec 2015




16.2 Elements of the greedy strategy

A greedy algorithm obtains an optimal solution to a problem

by making a sequence of choices. For each decision point in the

algorithm, the choice that seems best at the moment is chosen.

This strategy does not always produce an optimal solution, but as

we saw in the activity-selection problem, sometimes it does.

This section discusses some of the general properties of greedy

methods.

The process that we followed to develop a greedy algorithm

was a bit more involved than is typical. We went through the

following steps:

1. Determine the optimal substructure of the problem.

2. Develop a recursive solution.

3. Prove that at any stage of the recursion, one

of the optimal choices is the greedy choice.

Thus, it is always safe to make the greedy choice.

طر الناس على الف ز، لكنهم يجاهدوف

.أنفسهم كب يؤلف ها على الهزيمة

Dec 2015




4. Show that all but one of the subproblems induced by having made the greedy choice are empty.

5. Develop a recursive algorithm that implements the greedy strategy.

6. Convert the recursive algorithm to an iterative algorithm.

How can one tell if a greedy algorithm will solve a particular

optimization problem? There is no way in general, but the

greedy-choice property and optimal substructure are the two key

ingredients. If we can demonstrate that the problem has these

properties, then we are well on the way to developing a greedy

algorithm for it.

Greedy-choice property

The first key ingredient is the greedy-choice property:

a globally optimal solution can be arrived at by making a locally

optimal (greedy) choice. In other words, when we are considering

which choice to make, we make the choice that looks best in the

current problem, without considering results from subproblems.

ررررب النهايررررة، العبرررررة ليسررررت

بالسررن ات التررب عشررتها، بررل ررب

.نبض اليياة ب تلك السن ات

Dec 2015




Here is where greedy algorithms differ from dynamic

programming. In dynamic programming, we make a choice at

each step, but the choice usually depends on the solutions to

subproblems. Consequently, we typically solve

dynamic-programming problems in a bottom-up manner,

progressing from smaller subproblems to larger subproblems.

In a greedy algorithm, we make whatever choice seems best at

the moment and then solve the subproblem arising after the

choice is made. The choice made by a greedy algorithm may

depend on choices so far, but it cannot depend on any future

choices or on the solutions to subproblems. Thus, unlike dynamic

programming, which solves the subproblems bottom up, a greedy

strategy usually progresses in a top-down fashion, making one

greedy choice after another, reducing each given problem

instance to a smaller one.

Of course, we must prove that a greedy choice at each step

yields a globally optimal solution, and this is where cleverness

may be required. Typically, the proof examines a globally optimal

solution to some subproblem. It then shows that the

solution can be modified to use the greedy

choice, resulting in one similar but smaller subproblem.

كررل ال قررائي تشررير إلررى أف أغلررب النرراس

.لعالم لكب لا تفعل شي ام ج دوف ب هذا ا

Dec 2015




Optimal substructure

A problem exhibits optimal substructure if an optimal

solution to the problem contains within it optimal solutions to

subproblems. This property is a key ingredient of assessing the

applicability of dynamic programming as well as greedy

algorithms. As an example of optimal substructure, recall how we

demonstrated that if an optimal solution to subproblem Sij

includes an activity ak, then it must also contain optimal solutions

to the subproblems Sik and Skj. Given this optimal substructure, we

argued that if we knew which activity to use as ak, we could

construct an optimal solution to Sij by selecting ak along with all

activities in optimal solutions to the subproblems Sik and Skj.

لليياة معاف كثيرة لم يتعرأ

.أغلب الناس على معامها

Dec 2015




Greedy versus dynamic programming

Because the optimal-substructure property is exploited by

both the greedy and dynamic-programming strategies, one might

be tempted to generate a dynamic- programming solution to a

problem when a greedy solution suffices, or one might mistakenly

think that a greedy solution works when in fact a

dynamic- programming solution is required.

To illustrate the subtleties between the two techniques, let us

investigate two variants of a classical optimization problem.

The 0-1 knapsack problem is posed as follows.

A thief robbing a store finds n items; the ith item is worth vi

dollars and weighs wi pounds, where vi and wi are integers.

He wants to take as valuable a load as possible, but he can carry

at most W pounds in his knapsack for some integer W.

Which items should he take? (This is called the 0-1 knapsack

problem because each item must either be taken or

left behind; the thief cannot take a fractional

amount of an item or take an item more

than once.) معاررررم المرررردخنين لا يقلعرررر ف عررررن

الترردخين إلا إذا تعلررق الأمررر بييرراتهم،

.وعلى نفس الني تسير باقب الأم ر

Dec 2015




In the fractional knapsack problem, the setup is the same,

but the thief can take fractions of items, rather than having to

make a binary (0-1) choice for each item.

Both knapsack problems exhibit the optimal-substructure

property. For the 0-1 problem, consider the most valuable load

that weighs at most W pounds. If we remove item j from this load,

the remaining load must be the most valuable load weighing at

most W — wj that the thief can take from the n — 1 original items

excluding j. For the comparable fractional problem, consider that

if we remove a weight w of one item j from the optimal load, the

remaining load must be the most valuable load weighing at most

W — w that the thief can take from the n — 1 original items plus

wj — w pounds of item j.

Although the problems are similar, the fractional knapsack

problem is solvable by a greedy strategy,

whereas the 0-1 problem is not.

نة دوف ياة ممك يه أف الي شك ما لا م

شرية قعادة، الأغلبية العامى من الب

.عية أو رغما عنهميفعل ف ذلك ط ا

Dec 2015




To solve the fractional problem, we first compute the value

per pound vi /wi for each item. Obeying a greedy strategy, the

thief begins by taking as much as possible of the item with the

greatest value per pound. If the supply of that item is exhausted

and he can still carry more, he takes as much as possible of the

item with the next greatest value per pound, and so forth until he

can’t carry any more. Thus, by sorting the items by value per

pound, the greedy algorithm runs in 0 (n Ig n) time.

To see that this greedy strategy does not work for the 0-1

knapsack problem, consider the problem instance illustrated in

the figure. There are 3 items, and the knapsack

can hold 50 pounds. Item 1 weighs 10 pounds

and is worth 60 dollars. Item 2 weighs 20

pounds and is worth 100 dollars. Item 3

weighs 30 pounds and is worth120 dollars.

ندما قت ع ية ال ساف أهم يدرك الإن

ته، يعيش اللياات الأخيرة ب حيا

.أغلبنا ينتار هذا ال قت كب يندم

Dec 2015




Thus, the value per pound of item 1 is 6 dollars per pound,

which is greater than the value per pound of either item 2

(5 dollars per pound) or item 3 (4 dollars per pound).

The greedy strategy, therefore, would take item 1 first.

As can be seen from the case analysis in the figure(b), however,

the optimal solution takes items 2 and 3, leaving 1 behind.

The 2 possible solutions that involve item 1 are both suboptimal.

For the comparable fractional problem, however, the greedy

strategy, which takes item 1 first, does yield an optimal solution,

as shown in the figure(c). Taking item 1 doesn’t work in the 0-1

problem because the thief is unable to fill his knapsack to

capacity, and the empty space lowers the effective value pound of

his load. In the 0-1 problem, when we consider an item for

inclusion in the knapsack, we must compare the solution to the

subproblem in which the item is included with the solution to the

subproblem in which the item is excluded before we can make the

choice. The problem formulated in this way gives

rise to many overlapping subproblems - a

hallmark of dynamic programming - and indeed,

dynamic programming can be used to solve the

0-1 problem.

يرررتكلم بعرررض النررراس أ نرررراء

نرررر مهم، أمررررا الم يا ررررروف

. يتكلم ف أ ناء ن م الآخرين

Dec 2015




Exercise 16.2-1

Prove that the fractional knapsack problem has the greedy-choice

property.

Solution

Assume there’s an optimal solution A but it does not take as much

possible from the first item ( highest value/weight ration).

- We can remove the part of one of the other items and

replace it with the first item.

- We get a set of items with higher value.

- A is not optimal.

- Contradiction!

- Solution A must contain as much as possible of the first item,

which is the greedy choice.

- By induction on the number of choices made, making the

greedy choice at every step produces an optimal solution.

- The greedy choice property is proven.

يقررال إف أقصررر قصررة كتبهررا إنسرراف:

قد أف رجل ولد وعاش ومات. وأنا أعت

خرررين بشرربء مررن التط يررل: قرريرة

.رجل ولد ولم يعش ومي ذلك قيم ت

Dec 2015




Exercise 16.2-2

Give a dynamic-programming solution to the 0-1 knapsack

problem that runs in O(n W) time, where n is number of items and

W is the maximum weight of items that the thief can put in his

knapsack.

Solution

Let i be the highest numbered item among 1,…,n items in an

optimal solution S for W with value v(S). Then S’ = S – {i} is an

optional solution for W – wi with value v(S’) = v(S) – vi.

We can express this in the following recursion. Let c(i,w) denote

the value of the solution for items I,…,i and maximum weight w.

0 if i = 0 or w = 0

c[i,w] = C(i-1,w) if wi > w

max(vi + c(i-1,w-wi),c(i-1,w)) if i>0 and w ≥ wi

Notice that the last determines whether or not the ith element

should be included in an optimal solution.

We can use this recursion to create a straight

forward dynamic programming algorithm

as follows:

إف المررردارس ينبررر ر المعر رررة: يأتيهرررا

الرربعض لينهلرر ا منهررا، والرربعض لأخررذ

.نفيات، لكن الأغلب للتمسح بها لا أكثر

Dec 2015




Input: Two sequences v = <v1,…vn> and w = <w1,…,wn>, the

number of items n and the maximum weight w.

Output: The optimal value of the knapsack.

for w ← 0 to W do

c[0,w] ← 0

end for

for i ← 1 to n do

c[i,0] ← 0

for w ← 1 to W do

if wi ≤ w then

if vi + c[i-1,w-wi] > c[i-1,w] then

c [i,w] ← vi + c[i-1,w-wi]

else

c[i,w] ← c[i-1,w]

end if

else

c[i,w] ← c[i-1,w]

end if

end for

end for

return c[n,W]

ي لد الإنساف جاهفً وليس غبياً، لكن

.أغلب أنامة التعليم تي له إلى غبب

Dec 2015




Exercise 16.2-5

Describe an efficient algorithm that, give a set {x1,x2,…,xn} of

points on the real line, determines the smallest set of unit-length

closed intervals that contains all of the given points.

Argue that your algorithm is correct.

Solution

Consider the following simple algorithm: Sort the points obtaining

a new array {y1,…,yn}. The first interval is given by [y1,y1+1].

If yi is the leftmost point not contained in any existing interval, the

next interval is [yi,yi+1] and so on.

This greedy algorithm does the job since the rightmost element of

the set must be contained in an interval and we can do no better

than the interval [y1,y1+1]. Additionally, any subproblem to the

optimal solution must be optimal. This is easily seen by

considering the problem for the points greater than y1+1 and

arguing inductively.

ا للسعادة أحد العناصر التب لا غنى عنه

.ه ا تقاد المرء لبعض ما يريد

Dec 2015




Exercise 16.2-6

Show how to solve the fractional knapsack problem in O(n) time.

Solution

Use a linear-time median algorithm to calculate the median m of

the vi/wi ratios. Next, partition the items into three sets:

G = {i : vi/wi > m}, E = {i : vi/wi = m}, and L = {i : vi/wi < m}; this step

takes linear time. Compute WG = ∑iG wi and WE = ∑iE wi, the total

weight of the items in sets G and E, respectively.

If WG > W, then do not yet take any items in set G, and

instead recurs on the set of items G and knapsack capacity

W.

Otherwise (WG ≤ W), take all items in set G, and take as

much of the items in set E as will fit in the remaining

capacity W – WG.

If WG + WE ≥ W (i.e., there is no capacity left after taking all

the items in set G and all the items in set E that fit in the

remaining capacity W – WG), then we are done.

Otherwise (WG + WE < W), then after taking all

the items in sets G and E, recurs on the set

of items L and knapsack capacity W – WG – WE.

البعض ينشر السعادة أينما ذهب،

.البعض يخلفها وراءه متى ذهب

Dec 2015




Continue

To analyze this algorithm, note that each recursive call takes

linear time, exclusive of the time for a recursive call that it may

take. When there is a recursive call, there is just one, and it’s for a

problem of at most half the size. Thus, the running time is given

by the recurrence T(n) ≤ T(n/2) + Θ(n), whose solution is

T(n) = O(n).

العجيررب أف أغلبنررا يسررعى قررط

.لتجنب الألم، لا لإيجاد السعادة

Dec 2015




Exercise 16.2-7

Suppose you are given two sets A and B, each containing n

positive integers. You can chose to reorder each set however you

like. After reordering, let ai be the ith element of set A, and let bi

be the ith element of set B. You then receive a payoff of ∏i=1 aibi.

Give an algorithm that will maximize your payoff. Prove that your

algorithm maximizes the payoff, and state its running time.

Solution

Sort A and B into monotonically decreasing order.

Here’s a proof that this method yields an optimal solution.

Consider any indices I and j such that i < j, and consider the terms

aibi and aj

bj. We want to show that it is no worse to include these

terms in the payoff that to include aibj and aj

bi, i.e., that aibi aj

bj ≥ aibj

ajbi. Since A and B are sorted into monotonically decreasing order

and i < j, we have ai ≥ aj and bi ≥ bj. Since ai and aj are positive

and bi – bj is nonnegative, we have aibi-bj ≥ aj

bi-bj.

Multiplying both sides by aibjaj

bi yields aibi aj

bj ≥ aibj aj

bi.

Since the order of multiplication doesn’t matter,

sorting A and B into monotonically increasing order

works as well.

إف الذين لا يجيدوف الابتسام قد ينته ف

.إلى تشريي البكاء والدع ة إليه كعبادة

Documents

Algorithms for optimization problems typically go through a