Greedy Method

unit5 1

Greedy Algorithm

Lecturer notes is purely design and developed by Er. Mohd. Arif Siddique

Lecturer Dept. of Computer Science & ITRGEC, Meerut

Unit 3 Greedy Algorithms 2

General form of Greedy Algorithms

Algorithm greedy (C) //C is an input set having n element

{

S // S is a solution set

while not solution (S) and C do

x extract best element from C.

if (x is feasible)

Then C C - {x} // extract X from input set C

S S{x} // add X into solution

if solution (S) = then return “no solutions”

Time complexity excluding sorting: O(n)


A Greedy Algorithm

A greedy algorithm always makes the choice that looks best at the moment. In dynamic programming, the optimal solution is described in a recursive

manner, and then is computed ``bottom up''. Dynamic programming is a powerful technique, but it often leads to algorithms with higher than desired running times.

An alternative design technique, called greedy algorithms. This method typically leads to simpler and faster algorithms, but it is not as powerful or as widely applicable as dynamic programming.

The greedy concept make the choice that looks best at the moment in this hope that local optimal choices lead to global optimal solution

Even when greedy algorithms do not produce the optimal solution, they often provide fast heuristics (non-optimal solution strategies), and are often used in finding good approximations.


Problem 1:Problem 1: Activity-Selection ProblemActivity-Selection Problem

Problem:Given a set S = {1, 2, …, n} of n proposed activities, with a

start time si and a finish time fi for each activity i, select a maximum-size

set of mutually compatible activities.

Compatible Activities:

Activities i and j are compatible if the internal [si, fi) and [sj, fj) do not

overlap, i.e, i and j are compatible if si≥ fj and sj ≥ fi

Goal: Select a maximum-size set of mutually compatible activities


Greedy Activity-Selection AlgorithmSort the input activities by increasing finishing time. f1 ≤ f2 ≤ . . . ≤ fn

Algo GREEDY-ACTIVITY-SELECTOR (s[ ], f[ ]){n = length [s] A={i} j = 1 For(i = 2 to n ){

if (si ≥ fj ){

A= AU{i} j = i

}}

return set A }

Time complexity excluding sorting: O(n)


ExampleGiven 11 activities A p q r s t u v w x y z

s 1 3 0 5 3 5 6 8 8 2 12

f 4 5 6 7 8 9 10 11 12 13 14

p

q

r

s

t

u

v

w

x

y

z

0 1413121110987654321

Here all input activities are already sorted by increasing finishing time.


Example cont…

0p

q

r

s

t

u

v

w

x

y

z

1413121110987654321

Initialization A={p}

Activity ={p}


p

q

r

s

t

u

v

w

x

y

z

0 1413121110987654321

Activity ={p}

Activity ‘p’ and ‘q’ are not compatible (overlapped) So

don't add it in to solution

Example cont…


p

q

r

s

t

u

v

w

x

y

z

0 1413121110987654321

Activity ={p}

Activity ‘p’ and ‘r’ are not compatible (overlapped) So


Example cont…


p

q

r

s

t

u

v

w

x

y

z

0 1413121110987654321

Activity ={p, s, }

Activity ‘p’ and ‘s’ are compatible, So

add ‘s’ in to solution

Example cont…


p

q

r

s

t

u

v

w

x

y

z

0 1413121110987654321

Activity ={p, s, }

Activity ‘s’ and ‘t’ arenot compatible (overlapped) So


Example cont…


p

q

r

s

t

u

v

w

x

y

z

0 1413121110987654321

Activity ={p, s, }

Activity ‘s’ and ‘u’ arenot compatible (overlapped) So


Example cont…


p

q

r

s

t

u

v

w

x

y

z

0 1413121110987654321

Activity ={p, s, }

Activity ‘s’ and ‘v’ arenot compatible (overlapped) So


Example cont…


p

q

r

s

t

u

v

w

x

y

z

0 1413121110987654321

Activity ={p, s, w, }

Activity ‘s’ and ‘w’ are

compatible, So add

‘w’ in to solution

Example cont…


p

q

r

s

t

u

v

w

x

y

z

0 1413121110987654321


Activity ‘s’ and ‘x’ arenot compatible (overlapped) So


Example cont…


p

q

r

s

t

u

v

w

x

y

z

0 1413121110987654321


Activity ‘w’ and ‘y’ arenot compatible (overlapped), So

do not add ‘y’ in to solution

Example cont…


p

q

r

s

t

u

v

w

x

y

z

0 1413121110987654321

Activity ‘w’ and ‘z’ arecompatible, So add ‘z’

in to solution

Example cont…

Activity ={p, s, w, z }


p

q

r

s

t

u

v

w

x

y

z

0 1413121110987654321

Activity ‘w’ and ‘z’ arecompatible, So add ‘z’

in to solution

Example cont

Activity ={p, s, w, z }


Activity Selection (Interval Scheduling)

Time0

A (3)

C (2)

F (6)

B (1)

D (5)

G (7)

E (4)

1 2 3 4 5 6 7 8 9 10 11

H (8)

0 1 2 3 4 5 6 7 8 9 10 11



Time0

A (3)

C (2)

F (6)

B (1)

D (5)

G (7)

E (4)

1 2 3 4 5 6 7 8 9 10 11

H (8)

0 1 2 3 4 5 6 7 8 9 10 11

B



Time0

A (3)

C (2)

F (6)

B (1)

D (5)

G (7)

E (4)

1 2 3 4 5 6 7 8 9 10 11

H (8)

0 1 2 3 4 5 6 7 8 9 10 11

B

C (2)

C



Time0

A (3)

C (2)

F (6)

B (1)

D (5)

G (7)

E (4)

1 2 3 4 5 6 7 8 9 10 11

H (8)

0 1 2 3 4 5 6 7 8 9 10 11

B

C (2)

A



Time0

A (3)

C (2)

F (6)

B (1)

D (5)

G (7)

E (4)

1 2 3 4 5 6 7 8 9 10 11

H (8)

0 1 2 3 4 5 6 7 8 9 10 11

B

C (2)

E



Time0

A (3)

C (2)

F (6)

B (1)

D (5)

G (7)

E (4)

1 2 3 4 5 6 7 8 9 10 11

H (8)

0 1 2 3 4 5 6 7 8 9 10 11

B

C (2)

ED



Time0

A (3)

C (2)

F (6)

B (1)

D (5)

G (7)

E (4)

1 2 3 4 5 6 7 8 9 10 11

H (8)

0 1 2 3 4 5 6 7 8 9 10 11

B

C (2)

E F



Time0

A (3)

C (2)

F (6)

B (1)

D (5)

G (7)

E (4)

1 2 3 4 5 6 7 8 9 10 11

H (8)

0 1 2 3 4 5 6 7 8 9 10 11

B

C (2)

E G



Time0

A (3)

C (2)

F (6)

B (1)

D (5)

G (7)

E (4)

1 2 3 4 5 6 7 8 9 10 11

H (8)

0 1 2 3 4 5 6 7 8 9 10 11

B

C (2)

E H


Elements of the Greedy Strategy

․ When to apply greedy algorithms? Greedy-choice property: A global optimal solution can be

arrived at by making a locally optimal (greedy) choice. Dynamic programming needs to check the solutions to

subproblems. Optimal substructure: An optimal solution to the problem

contains within its optimal solutions to subproblems. E.g., if A is an optimal solution to S, then A' = A - {1} is an optimal

solution to S' = {i S: si f1}.

․ Greedy algorithms (heuristics) do not always produce optimal solutions.

․ Greedy algorithms v.s. dynamic programming (DP) Common: optimal substructure Difference: greedy-choice property DP can be used if greedy solutions are not optimal.


Problem 2:Problem 2: Fractional Knapsack Problem

The 0-1 knapsack problem

(Each item is either taken or not taken)

The fractional knapsack problem

(Allow to take fraction of items.)

Knapsack Problem

Classification of Knapsack Problem:

There are n items in a store. For i =1,2, . . . , n, item i has weight

wi > 0 and benefit bi > 0. Thief can carry a maximum weight of W

pounds in a knapsack. In this version of a problem the items can

be broken into smaller piece, so the thief may decide to carry only

a fraction of object i.


These line takes θ(n)

These line takes θ(n)

Fractional Knapsack Problem cont..Algo Greedy-fractional-knapsack (Item[n], w[ ], b[ ], W){Knap=0Weight = 0Benefit=0for each item i v[i] = b[i] / w[i] // value per pound

while (weight <=W) { i= Extract item of maximum value from list if(weight + w[i] ≤ W) { Knap=Knap U Item[i] weight = weight + w[i] benefit= benefit +v[i] } else{ Knap=Knap U Item[i] weight = W benefit = (W - weight) * v[i] / w[i] // Factoring item, W – weight is needed weight

}} return x}


Fractional Knapsack Problem cont..Algo Greedy-fractional-knapsack (Item[n], w[ ], b[ ], W){Knap=0Weight = 0Benefit=0for each item i v[i] = b[i] / w[i] // value per pound

while (weight <=W) { i= Extract item of maximum value from list if(weight + w[i] ≤ W) { Knap=Knap U Item[i] weight = weight + w[i] benefit= benefit +v[i] } else{ Knap=Knap U Item[i] weight = W benefit = (W - weight) * v[i] / w[i] // Factoring item, W – weight is needed weight

}} return Knap}

So time co

mplexity e

xcluding so

rting: O

(n)


Example

Item A B C D E F G

Benefits (Rs.) 12 10 8 11 14 7 9

Weight (Kg.) 4 6 5 7 3 1 6

Let S ={a, b, c, d, e, f, g} denote a set of objects with weights and benefits

as given in the table below. What is an optimal solution to the fractional

knapsack problem for S assuming that we have a sack that can hold objects

with total weight 18?

Carrying capacity W = 18 Kg.


Example cont…

First we must calculate the “value" for the each items,which is defined as value = benefits/weight. I do this andgive them in the table below:

Item A B C D E F G

benefits 12 10 8 11 14 7 9

Weight 4 6 5 7 3 1 6

Value 3 1.67 1.6 1.57 4.67 7 1.5

value = benefit / weight


Example cont…

Item A B C D E F G

benefits 12 10 8 11 14 7 9

Weight 4 6 5 7 3 1 6

Value 3 1.67 1.6 1.57 4.67 7 1.5

Sort this table according to decreasing value

Item F E A B C D G

benefits 7 14 12 10 8 11 9

Weight 1 3 4 6 5 7 6

Value 7 4.67 3 1.67 1.6 1.57 1.5


Example cont…

Item F E A B C D G

benefits 7 14 12 10 8 11 9

Weight 1 3 4 6 5 7 6

Value 7 4.67 3 1.67 1.6 1.57 1.5

W=18

Initially

Knapsack

weight = 0

benefits =0

W=18


1.5

6

9

G

1.57

7

11

D

1.6

5

8

C

1.67

6

10

B

3

4

12

A

4.67

3

14

E

7

1

7

F

Weight

Value

benefits

Item

W=18

FKnapsack

weight = 1

benefits =7

W=18

Select maximum valued item ‘F’, Here (weight+w[F])<W

Put whole item ‘F’ into knapsack. Add weight[F] with weight

and benefit[F] with benefit.

Example cont…


1.5

6

9

G

1.57

7

11

D

1.6

5

8

C

1.67

6

10

B

3

4

12

A

4.67

3

14

E

7

1

7

F

Weight

Value

benefits

Item

W=18

F, EKnapsack

weight = 1+3 =4

benefits =7+14 =21

W=18

Now select next maximum valued item ‘E’, Here (weight+w[E])<W Put whole item ‘E’ into knapsack. Add weight[E] with weight and benefit[E] with benefit.

Example cont…


1.5

6

9

G

1.57

7

11

D

1.6

5

8

C

1.67

6

10

B

3

4

12

A

4.67

3

14

E

7

1

7

F

Weight

Value

benefits

Item

W=18

F, E, AKnapsack W=18

Now select next maximum valued item ‘A’, Here (weight+w[A])<=W Put whole item ‘A’ into knapsack. Add weight[A] with weight and benefit[A] with benefit.

Example cont…

weight = 1+3 =4+4=8

benefits =7+14 =21+12=33


1.5

6

9

G

1.57

7

11

D

1.6

5

8

C

1.67

6

10

B

3

4

12

A

4.67

3

14

E

7

1

7

F

Weight

Value

benefits

Item

W=18

F, E, A, BKnapsack W=18

Now select next maximum valued item ‘B’, Here (weight+w[B])<=W Put whole item ‘B’ into knapsack. Add weight[B] with weight and benefit[B] with benefit.

Example cont…

weight = 1+3 =4+4=8+6=14

benefits =7+14 =21+12=33+10=43


F, E, A, B, CKnapsack W=18

Now select next maximum valued item ‘C’, Here (weight+w[C])

⊀W Put whole item ‘B’ into knapsack. And calculate weight and benefit as follows:

1.5

6

9

G

1.57

7

11

D

1.6

5

8

C

1.67

6

10

B

3

4

12

A

4.67

3

14

E

7

1

7

F

Weight

Value

benefits

Item

W=18

Example cont…

benefits =7+14 =21+12=33+10=43+(needed weight)* value[C]

=43+(4*1.6) = 43+6.4 = 49.4

needed weight = W – weight = 18 – 14 = 4Put whole item ‘B’ into knapsack.

weight = W = 18


Example cont…

Item F E A B C D G

benefits 7 14 12 10 8 11 9

Weight 1 3 4 6 5 7 6

Value 7 4.67 3 1.67 1.6 1.57 1.5

W=18

Remaining items D, G could not put into Knapsack (bag) because bag is full

i.e weight = W

F, E, A, B, CKnapsack =

Weight in bag = W = 18

Benefits = Rs. 49.4


Complexity Analysis

If the items are already sorted into decreasing order of vi / wi, then the while-loop takes a time in O(n); Therefore, the total time including the sort is in O(n log n).

If we keep the items in heap with largest vi/wi at the root. Then

creating the heap takes O(n) time

while-loop now takes O(log n) time (since heap property must be restored after the removal of root)


Huffman coding is an algorithm used for lossless data compression developed by David A. Huffman as a PhD student at MIT in 1952, and published in A Method for the Construction of Minimum-Redundancy Codes.

Professor David A. Huffman (August 9, 1925 - October 7, 1999)

"Huffman Codes" are widely used applications that involve the compression and transmission of digital data, such as: fax machines, modems, computer networks, and high-definition television (HDTV), etc.

Introduction


Reducing the space required to store files on disk or tape

Reducing the time to transmit large files.

Huffman savings are between 20% - 90%

Motivation


Basic Idea

Huffman Codes:

Suppose we wish to save a text (ASCII) file on the disk or to transmit it though a network using an encoding scheme that minimizes the number of bits required. In fixed-size-encoding -scheme, without compression, characters are typically encoded by their ASCII codes with 8 bits per character. We can do better if we variable-size-encoding. In the variable-size-encoding-scheme we assign different code of different length according to their frequencies of occurrences.


Example:

Suppose you have a file with 100K characters.

For simplicity assume that there are only 6 distinct characters in the file from a through f, with frequencies as indicated below.

We represent the file using a unique binary string for each character.

a b c d e f

Frequency(in 1000s)

45 13 12 16 9 5

Fixed-length codeword

000 001 010 011 100 101

Space = (45*3 + 13*3 + 12*3 + 16*3 + 9*3 + 5*3) * 1000

= 300K bits


Can we do better ??

By using variable-length codes instead of fixed-length codes.

Idea : Giving frequent characters short codewords, and infrequent characters long codewords.

i.e. The length of the encoded character is inversely proportional to that character's frequency.

YES !!

Space = (45*1 + 13*3 + 12*3 + 16*3 + 9*4 + 5*4) * 1000=224K bits

a b c d e f Bit Required

Frequency(in 1000s)

45 13 12 16 9 5

Fixed-length codeword

000 001 010 011 100 101 300K bits

Variable-length codeword

0 101 100 111 1101 1100 224K bits

Savings = 25%


Codes in which no codeword is also a prefix of some other codeword.

("prefix-free codes" would have been a more appropriate name)

Variable-length codeword

0 101 100 111 1101 1100

It is very easy to encode and decode using prefix codes.

No Ambiguity !!

It is possible to show (although we won't do so here) that the optimal data compression achievable by a character code can always be achieved with a prefix code, so there is no loss of generality in restricting attention to prefix codes.

PREFIX CODES


Huffman invented a greedy algorithm that constructs an optimal prefix code called a Huffman code. The algorithm builds the tree T corresponding to the optimal code in a bottom-up manner.

It begins with a set of |C| leaves and performs a sequence of |C| - 1 "merging" operations to create the final tree.

Greedy Choice?

The two smallest nodes are chosen at each step, and this local decision results in a globally optimal encoding tree.

In general, greedy algorithms use small-grained, or local minimal/maximal choices to result in a global minimum/maximum

Constructing a Huffman code


Greedy Algorithm for Huffman Tree

Algo Greedy-Huffman(C,f){n length[C]Q C:ffor (i 1 to n-1) { z Allocate-Node x left[z] Extract-Min(Q) y right[z] Extract-Min(Q) f[z] f[x]+f[y] Insert(Q, z) }return Extract-Min(Q)}


Constructing a Huffman Coding Tree

a:45b:13e:9f:5 c:12 d:16

14

a:45b:13c:12 d:16

e:9f:5

14

From following Lines

z Allocate-Node

x left[z] Extract-Min(Q)

y right[z] Extract-Min(Q)

f[z] f[x]+f[y]

From following LinesInsert(Q, z)



25

a:45b:13c:12 d:16

e:9f:5

14

a:4525

b:13c:12

d:16

e:9f:5

14


z Allocate-Node



f[z] f[x]+f[y]



30


a:4525

b:13c:12

d:16

e:9f:5

14

30 a:45

d:16

e:9f:5

14

25

b:13c:12


z Allocate-Node



f[z] f[x]+f[y]



55

3025

14

a:45

b:13

e:9f:5

c:12 d:16



z Allocate-Node



f[z] f[x]+f[y]

Insert(Q, z)


100

55

3025

14

a:45

b:13

e:9f:5

c:12 d:16



z Allocate-Node



f[z] f[x]+f[y]

Insert(Q, z)


Adding binary code in this tree

100

55

3025

14

a:45

b:13

e:9f:5

c:12 d:16

00

0

0

11

1

1

10

Add ‘0’ with left child, and ‘1’with right child of each node


100

55

3025

14

a:45

b:13

e:9f:5

c:12 d:16

00

0

0

11

1

1

10

Finding binary code from this tree

Char Freq Codeword

a 45 0

b 13 101

c 12 100

d 16 111

e 9 1101

f 5 1100


The analysis of the running time of Huffman's algorithm assumes that Q is implemented as a binary min-heap.

• For a set C of n characters, the initialization of Q in line 2 can be performed in O(n) time using the BUILD-MIN-HEAP procedure.

• The for loop in lines 3-8 is executed exactly |n| - 1 times. Each heap operation requires time O(log n). The loop contributes = (|n| - 1) * O(log n) = O(nlog n)

Thus, the total running time of HUFFMAN on a set of n characters = O(n) + O(nlog n)

= O(n log n)

Running Time analys


Thank You

Documents

Greedy Method