IT2201 Algorithm Design and Analysis UNIT 5

WWW.VIDYARTHIPLUS.COM

WWW.VIDYARTHIPLUS.COM V+ TEAM

ALGORITHM

NOTATIONS

Asymptotic Notations

Asymtotic notations are method used to estimate and represent the efficiency of an algorithm using simple formula. This can be useful for seperating algorithms that leads to different amounts of work for large inputs.

Comparing or classify functions that ignores constant factors and small inputs is called as asymtotic growth rate or asymtotic order or simply the order of functions. Complexity of an algorithm is usually represented in O, o, , O (big oh), (big omega), (big theta) notations.

Let f(n) & g(n) be any non-negative functions defined on the set of natural numbers.



f(n) will an algorithms running time indicated by its basic operation count C(n) g(n) will be some simple function to compare the count with.

BIG - OH NOTATION (O)

This is a standard notation that has been developed to represent functions which bound the computing time for algorithms and it is used to define the worst case running time of an algorithm and concerned with very large values of N.

Definition : - A function f(n) = O(g(n)) [f(n) O(g(n))], iff there exist positive constants c & no such that f(n)cg(n) for all n, nno.

Example: The function f(n) = 8n - 2 is O(n).

Justification: By the big-Oh definition, we need to find a real constant c > 0 and an integer constant n0 >= 1 such that 8n - 2 = n0. It is easy to see that a possible choice is c = 8 and n0 = 1.

More examples: 2n + 10 is O(n), 7n - 2 is O(n), 3n3 + 20n2 + 5 is O(n3), 3log n + 5 is O(log n)

The big-Oh notation allows us to say that a function f(n) is less than or equal to another function g(n) up to a constant factor and in the asymptotic sense as n grows towards infinity. If f(n) is of the form of An + B, where A and B are constants. It's called a linear function, and it is O(n).



The big-Oh notation gives an upper bound on the growth rate of a function. The statement "f(n) is O(g(n))" means that the growth rate of f(n) is no more than the growth rate of g(n).

BIG - OMEGA NOTATION ()

This notation is used to describe the best case running time of algorithms and concerned with very large values of N.

Definition : - A function f(n) = (g(n)) [f(n) (g(n))], iff there exist positive constants c & no such that f(n) cg(n) for all n, nno.

Example, show that n2 is (nlogn).

BIG - THETA NOTATION ()

This notation is used to describe the average case running time of algorithms and concerned with very large values of n.

Definition : - A function f(n) = (g(n)) [f(n) (g(n))], if there exist positive constants c1,c2 & no such that c1g(n) f(n) c2g(n) for all n, nno.



Example: 3nlogn + 4n + 5logn is (nlogn).

Justification: 3nlogn



(little omega) f(n) is W (g(n)) if f(n) is asymptotically >g(n)

For example, let us consider sequential search ALGORITHM SequentialSearch (A[0...n-1],K) // Input : An array A[0..n-1] and a search key k. // Output : Returns the index of the first element of A that matches R or -1 if there are no matching elements.

while i < n and A[i] # k do i i + 1 if i < n return i else return - 1 Here, the best - case efficiency is 0(1) where the first element is itself the search element and the worst - case efficiency is 0(n) where the last element is the search element or the search element may not be found in the given array.

RECURSIVE ALGORITHMS:

A Recursive function is a function that is defined in terms of itself. Similarly, an algorithm is said to be recursive if the same algorithm is

invoked in the body. An algorithm that calls itself is Direct Recursive. Algorithm A is said to be Indirect Recursive if it calls another algorithm

which in turns calls A. The Recursive mechanism, are externally powerful, but even more

importantly, many times they can express an otherwise complex process very clearly. Or these reasons we introduce recursion here.

The following 2 examples show how to develop a recursive algorithms.

In the first, we consider the Towers of Hanoi problem, and in the second, we generate all possible permutations of a list of characters.



Towers of Hanoi:

.

.

.

Tower A Tower B Tower C

It is Fashioned after the ancient tower of Brahma ritual. According to legend, at the time the world was created, there was a

diamond tower (labeled A) with 64 golden disks. The disks were of decreasing size and were stacked on the tower in

decreasing order of size bottom to top. Besides these tower there were two other diamond towers

(labeled B & C). Since the time of creation, Brehman priests have been attempting to

move the disks from tower A to tower B using tower C, for intermediate storage.

As the disks are very heavy, they can be moved only one at a time. In addition, at no time can a disk be on top of a smaller disk. According to legend, the world will come to an end when the priest have

completed this task. A very elegant solution results from the use of recursion. Assume that the number of disks is n. To get the largest disk to the bottom of tower B, we move the remaining

n-1 disks to tower C and then move the largest to tower B. Now we are left with the tasks of moving the disks from tower C to B.



To do this, we have tower A and B available. The fact, that towers B has a disk on it can be ignored as the disks larger

than the disks being moved from tower C and so any disk scan be placed on top of it.

Algorithm:

1.Algorithm TowersofHanoi(n,x,y,z) 2. //Move the top n disks from tower x to tower y. 3. { . . . 4.if(n>=1) then 5. { 6.TowersofHanoi(n-1,x,z,y); 7.Write(move top disk from tower X ,to top of tower ,Y); Towersofhanoi(n-1,z,y,x); } }

Relationship between P, NP, NP-Complete and NP-Hard

problems.



NP-Hard problems NP-Hard are problems that are at least as hard as the hardest problems in NP. Note that NP-Complete problems are also NP-hard. However not all NP-hard problems are NP (or even a decision problem), despite having 'NP' as a prefix. That is the NP in NP-hard does not mean 'non-deterministic polynomial time'. Yes this is confusing but its usage is entrenched and unlikely to change.

NP-hard problems may be of any type: decision problems, search problems, or optimization problems Problem H is at least as hard as L, because H can be used to solve L If there is a polynomial algorithm for any NP-hard problem, then there are polynomial algorithms for all problems in NP, and hence P = NP; If P NP, then NP-hard problems have no solutions in polynomial time If an optimization problem H has an NP-complete decision version L, then H is NP-hard All optimization versions of difficult combinatorial problems are NP-Hard Very small problems can be solved by an exhaustive search algorithm, relatively small problems can be solved by dynamic programming and large problems can be solved by Branch and Bound An example of an NP-hard problem is the decision version of SUBSET-SUM which is this: Given a set of integers, does any non empty subset of them add up to zero? That is a yes/no question, and happens to be NP-complete.

INTRODUCTION TO ALGORITHM DESIGN TECHNIQUES

Algorithm Design Paradigms are the General approaches to the construction

of efficient solutions to problems.

Such methods are of interest because:

They provide templates suited to solving a broad range of diverse problems.

They can be translated into common control and data structures provided by

most high-level languages.



The temporal and spatial requirements of the algorithms which result can be

precisely analyzed.

Although more than one technique may be applicable to a specific problem, it

is often the case that an algorithm constructed by one approach is clearly

superior to equivalent solutions built using alternative techniques.

Some of the algorithm design techniques:

Greedy Algorithms Divide and conquer Dynamic programming Backtracking Branch and bound Randomized algorithm

Greedy Algorithm

The solution is constructed through a sequence of steps, each expanding a partially constructed solution obtained so far. At each step the choice must be locally optimal this is the central point of this technique. Greedy techniques are mainly used to solve optimization problems. They do not always give the best solution.

Examples

Minimal spanning tree

Shortest distance in graphs

Greedy algorithm for the Knapsack problem

The coin exchange problem

Huffman trees for optimal encoding

We have already seen three greedy algorithms in Dijkstra's, Prim's, and Kruskal's algorithms in unit 5 Graphs. Greedy algorithms work in phases. In



each phase, a decision is made that appears to be good, without regard for future consequences. Generally, this means that some local optimum is chosen. This "take what you can get now" strategy is the source of the name for this class of algorithms. When the algorithm terminates, we hope that the local optimum is equal to the global optimum. If this is the case, then the algorithm is correct; otherwise, the algorithm has produced a suboptimal solution. If the absolute best answer is not required, then simple greedy algorithms are sometimes used to generate approximate answers, rather than using the more complicated algorithms generally required to generate an exact answer.

Huffman tree construction :

characters A B C D -

possibilities 0.35 0.1 0.2 0.2 0.15

Step 1 : Arrange the characters in ascending order of their possibilities or

probabilities.

0.1 B

Add the probabilities of the two least characters based on their possibilities

and then rearrange them again in ascending order

B+C= 0.1 + 0.15 =0.25

0.2 C

0.2 D

0.35 A

0.15 -



0.2 C

Step 2 :

0.4

Add the next two least values C & D and reaarange the characters.

0.25 0.4

Step 3 : 0.6

0.4 0.25

0.2 D

0.1 B

0.15 -

0.35 A

0.2 D

0.2 C

0.1 B

0.15 C

0.2 C

0.2 D

0.35 A

0.35 A

0.2 C

0.2 D

0.1 B

0.15 C

0.25



0.2

5

1 0

1 0 1 0

1 0

Finally after adding all the characters we get a tree which is the Huffmann

tree.And label all the left child nodes as 0 and the right child nodes as 1 to find

out the coding of the characters.

Step 4 :

1

0.4 0.6

A 11

B 100

C 00

D 10

Divide and conquer

Another common technique used to design algorithms is divide and conquer. Divide and conquer algorithms consist of two parts:

Divide: Smaller problems are solved recursively (except, of course, base cases).

0.2 C

0.2 D

0.35 A

0.1 B

0.15 -



Conquer: The solution to the original problem is then formed from the solutions to the subproblems.

Divide and conquer is probably the beat known general algorithm design technique. It makes according to the following general plan. 1. A problems instance is divided into several smaller instances of the same problem,ideally of about the same size. 2. The smaller the instances are solved(typically recursively,though sometimes a different algorithm is employed when instances become small enough) 3. If necessary,the solutions obtained for smaller instances are combined to get a solution to the original problem.

Problem of

size n

Subproblem 1

of size n/2

Subproblem 2

of size n/2

Solution to subproblem

1

Solution to

subproblem 2

Solution to the

original problem



A problems instances of size n is divided into the instances of size n/2.More

generally ,an instances of size n can be can be divided into several instances of

size n/b,with a of them needed to be solved.(Here a &b are constances ; a>=1

and b>=1).Assuming that size n is a power of b, to simplify the analysis, we get

the following recurrence for the running time T(n).

T(n)=aT(n/b)+f(n),(general divide and conquer recurrence)

Where f(n) is a function that accounts for the time spent on dividing the

problem into smaller ones and on combining the solution.

Some of the algorithms in Divide and conquer

1.Quick sort

2.Merge sort

3.Binary search

Quick sort (partition exchange sort):

Pivot element is used to do the division process

Procedure:

1.Select the pivot from the given set of elements.

2.To divide the array pivot is selected, two index variables i and j are taken for

manipulation.

If first element =pivot, then

i=1 and j=n-1[n=no. of elements]

If last element=pivot, then

i=0 and j=n-2



3. The job of index variable i, is to search an element that is greater than the

pivot,so i is incremented by 1 till the value stored at at i is greater than pivot.

4. Similarly j needs to search an element that is smaller than the pivot,so, the

value j is decremented by 1 till the value stored at j is smaller than the pivot.

5. When the elements are found, they are interchanged. Again from the

current position i and j are incremented and decremented resp and exchanges

are made appropriate if desired.

6. The process ends whenever the index variables cross over or meet each

other , provided that the respective conditions are satisfied.

7. If the pivot is taken as the first element

if i and j meets each other (or)cross over do the swap between the j

th element and the pivot element.

If the pivot is taken as the last element

If i and j meet each other or cross over do the swap between the i th

element and the pivot element.

8. As a result, the whole array is divided into two parts,where all the elements

before the pivot are lesser and after the pivot are greater.

9. Now the same procedure is applied recursively for the two subarrays to get

the elements in sorted order.

NOTE:

Sometimes pivot selection will be the best case (or) the average case

(or) the worst case. It purely depends on the nature of elements.

Example:



n=6

0 1 2 3 4 5

56 91 35 75 48 68

Pivot i j j

Pivot

i i i j j

j

cross over(respective conditions are satisfied)

[while (a[i]pivot)

j--;]

There is a crossover between i and j, so the process ends.Make a swap

between the pivot and the j th element.

(Ele

56 48 35 75 91 68

0 1 2 3 4 5

35 48 56 75 91 68



ments before pivot are lesser ) (elements after the pivot are greater)

recursive quicksort

Therefore all the elements are in the sorted order.

ROUTINE:

Void main()

{

quicksort (a,0,n-1);

}

void quicksort(int a[],int low,int high)

{

int temp,i,j,pivot;

35 48 56 68 75 91



if(low



quicksort(a,low,j-1);

quicksort(a,j+1,high);

}

}

Mergesort(Divide and conquer):

It is a perfect example of the divide and conquer technique.

It sorts a given array A[0.n-1] by dividing it into two halves A[0[n/2]-

1] and A[[n/2].n-1],sorting each of them recursively,and then merging the

two smaller sorted arrays into a single sorted one.

ALGORITHM:

Mergesort(A[0n-1])

// sort array A[0n-1] by recursive mergesort



//input:An array A[0n-1] of orderable elements

//output:Array A[0n-1] sorted in non descending order

if n>1

copy A[0[n/2]-1] to B[0[n/2]-1]

copy A[[n/2]n-1] to C[0[n/2]-1]

mergesort(B[0[n/2]-1])

mergesort(C[0[n/2]-1])

merge(B,C,A)

Example:

Do mergesort for the following sets of elements:

8 3 13 6 2 14 5 9 10 1 7 12 4

Start doing the division, until the division is not possible.



So after sorting the output is

1 2 3 4 5 6 7 8 9 10 12 13 14

Example 2:



Binary Search:

Algorithm

1. Bin search(a,n,x) 2. // Given an array a[1:n] of elements in non-decreasing



3. //order, n>=0,determine whether x is present and 4. // if so, return j such that x=a[j]; else return 0. 5. { 6. low:=1; high:=n; 7. while (low



14 14 14

Found

x=-14 low high mid

1 14 7

1 6 3

1 2 1

2 2 2

2 1 Not found

x=9 low high mid

1 14 7

1 6 3

4 6 5

Found

Theorem: Algorithm Binsearch(a,n,x) works correctly.

Proof:

We assume that all statements work as expected and that comparisons such

as x>a[mid] are appropriately carried out.

Initially low =1, high= n,n>=0, and a[1]



Otherwise, the range is narrowed by either increasing low to (mid+1) or decreasing high to (mid-1).

Clearly, this narrowing of the range does not affect the outcome of the search.

If low becomes > than high, then x is not present & hence the loop is exited.

Dynamic Programming

Its is a technique for solving problems with overlapping subproblem.

Typically,there subproblems arise from a recurrence relating a solution to a

given problem with solution to its smaller subproblems of the same type.

Dynamic programming suggests solving each smaller subproblems once and

recording the results in a table from which a solution to the original problem

can be then obtained.

Some algorithms are:

Warshalls algorithm

Floyds algorithm

Optimal binary search tree

Warshalls algorithm is used for finding the transitive closure

Floyds algorithm is used for the all-pairs shortest-paths roblems

Dynamic programming can be used for constructing an optimal binary

search tree given set of keys and known probabilities of searching for them.

WARSHALLS ALGORITHM

Warshalls algorithm is for computing the transitive closure of a directed

graph.The transitive closure of a directed graph with n vertices can be

defined as the n-by-n Boolean matrix T={tij},in which the element in the ith

row (1



directed path (is a directed path of a positive length) from the ith vertex to the

jth vertex;otherwise, tij is 0.

RULES

Directed matrix Adjacency matrix

0 1 0 0

0 0 0 1

0 0 0 0

1 0 1 0

TRANSITIVE CLOSURE

Warshalls algorithm constructs the transitive closure of a given

digraph with n vertices through a series of n-by-n Boolean matrices.

R(0),.R(K-1),R(K),.R(N)

0 0 1 1 1 0

1 1

a b

c d



R(0) is just adjacency matrix boxed row and column are used for getting R(1)



Floyds algorithm for the all- pairs shortest path problem

Given a weighted connected graph (undirected or directed ),the all-pair shortest

path problem asks to find the distances (the length of the shortest paths)from

each vertex to all other vertices.

It is convenient to record the length of shortest path in an n-by-n matrix D

called the distances matrix : the elements dij in the ith row and the jth column of

this matrix indicates the length of the shortest path from the ith vertex to the jth

vertex (1



3 6 7

1

a

b

c

d

a b c d

Implements floyds algorithm for the all- pairs shortest path

problem

INPUTS: The weight matrix W of a graph

OUTPUT:The distance matrix of the shortest paths lengths

D



a b c d

a

b

c

d

3 6

3 2

a b c d

a

b

c

d

Distance Matrix :

The matrix indicates the length of the shortest path from the ith vertex to

the jth vertex.

Steps followed for Initial Input Weighted Matrix D0

D0 = 0 3

2 0

7 0 1

6 0

Let D1 be a matrix which contains length of the shortest path with

intermediate vertices number, not higher than one.

(i) a intermediate

b ___ a ___ c => 5

d ___ a ___ c => 9

D1 = 0 3

2 0 5

7 0 1

6 9 0

D2 is a matrix which contains length of the shortest path with

intermediate vertices numbers not higher than two.



a b c d

a

b

c

d

a

b

c

d

a

b

c

d

a

b

c

d

a b c d

a

b

c

d

a b c d

a

b

c

d

(ii) (a,b)

D2 0 3

2 0 5 c b a => 9

9 7 0 1

6 9 0


intermediate vertices number not higher than three.

(iii) (a,b,c)

D2 = 0 10 3 4 b a c d => 6

2 0 5 6 d a c b => 16

9 7 0 1 a c d => 4

6 16 9 0 a c b => 10


intermediate vertices number, not higher than four.

(iv) (a,b,c,d)

D2= 0 10 3 4



2 0 5 6 c d a => 7

7 7 0 1

6 16 9 0

Optimal Binary Search Tree

Dynamic approach minimizes the average no of comparisons for a

successful search.

As an example ,consider four keys A,B,C,D to be searched for with

probabilities 0.1,0.2,0.4,0.3 respectively.

The following diagram depicts two out of 14 possible binary search trees

containing tree keys.

The average no of comparisons in a successful search in the first tree is

probability x level

0.1*1+0.2*2+0.4*3+0.3*4=2.9

While for second tree is

0.1*2+0.2*1+0.4*2+0.3*3=2.1

Neither of there two trees is optimal

We could find the optimal tree by generating all 14 binary

search trees with there keys.( total no of binary search

trees with n keys is equal to the nthcatalan number

(eg) c(n)=[2n] 1 form n>0,c(o)=1

n=4(no of key)

=14possible binary search trees .

This exhaustive search approach is unrealistic .



So let a1,..an be distinct keys ordered from the smallest to the

largest and let p1,..........pn be the probabilities of searching for them

Let c[i,j] be the smallest average number of comparisons made in a

successful search in a binary trees Tji made up of keys ai,,..aj,where i

,j are some integer indices,1



j

Let us illustrate the algorithm by applying it to the four-key set

Key A B C D

probability 0.1 0.2 0.4 0.3

The initial tables look like this

0 1 2 3 4

Main table Root table

Step 2: Do c[i,i-1]=o for 1



c[4,3]=0 1 < 4 < 5

c[5,4]=0 1< 5 < 5

Step 3:

Do c[i,j]=pi for 1



Do R[i,i]=I for 1< i for 1



=0.5

When k=2 c[1,1]+c[3,2]+ 2s=1 ps

=0.1+0+0.3

=0.4

c[1,2]=min{when k=1 is 0.5

when k=2 is 0.4}

=0.4

Update c[1,2]=0.4 in the main table and

Update the respective k values (i.e)k=2 in the root table for

r[1,2].

1 2 2 3 4

Root table

Main table

Thus the average number of key comparisons in the optimal tree is equal

to 1.7.

Since r[1,4]=3, the root of the optimal tree contain

0 0.1

0.4

0 0.2

0 0.4

0 0.3

0



the third key (i.e) C.Its left subtree is made up of keys A&B. Its right

subtree contains just key D.

To find the specific structre of threse subtrees,we find first thir roots by

consulting the root table again as follows since R[1,2]=2the root n of the

optimal tree containing A&B is B, with A being itsleft child ( and the root

of the one node tree: R[1,1]=1.) Since R[4,4]=4,the root of their one node

optimal tree is its only key D.

BACKTRACKING

It is one of the most general algorithm design techniques. Many problems which deal with searching for a set of solutions or for a

optimal solution satisfying some constraints can be solved using the backtracking formulation.

To apply backtracking method, tne desired solution must be expressible as an n-tuple (x1xn) where xi is chosen from some finite set Si.

The problem is to find a vector, which maximizes or minimizes a criterion function P(x1.xn).

The major advantage of this method is, once we know that a partial vector (x1,xi) will not lead to an optimal solution that (mi+1..mn) possible test vectors may be ignored entirely.

Many problems solved using backtracking require that all the solutions satisfy a complex set of constraints.

These constraints are classified as: i) Explicit constraints.

ii) Implicit constraints.

1) Explicit constraints: Explicit constraints are rules that restrict each Xi to take values only

from a given set.

Some examples are,

Xi0 or Si = {all non-negative real nos.}

Xi =0 or 1 or Si={0,1}.



LiXiUi or Si= {a: LiaUi}

All tuples that satisfy the explicit constraint define a possible solution space for I.

2) Implicit constraints: The implicit constraint determines which of the tuples in the

solution space I can actually satisfy the criterion functions.

Algorithm:

Algorithm IBacktracking (n)

// This schema describes the backtracking procedure .All solutions are

generated in X[1:n]

//and printed as soon as they are determined.

{

k=1;

While (k 0) do

{

if (there remains all untried

X[k] T (X[1],[2],..X[k-1]) and Bk (X[1],..X[k])) is true ) then

{

if(X[1],X[k] )is the path to the answer node)

Then write(X[1:k]);

k=k+1; //consider the next step.

}



else k=k-1; //consider backtracking to the previous set.

}

}

All solutions are generated in X[1:n] and printed as soon as they are determined.

T(X[1]..X[k-1]) is all possible values of X[k] gives that X[1],..X[k-1] have already been chosen.

Bk(X[1]X[k]) is a boundary function which determines the elements of X[k] which satisfies the implicit constraint.

Certain problems which are solved using backtracking method are,

1. Sum of subsets.

2. Graph coloring.

3. Hamiltonian cycle.

4. N-Queens problem.

1.SUM OF SUBSETS:

We are given n positive numbers called weights and we have to find all combinations of these numbers whose sum is M. this is called sum of subsets problem.

If we consider backtracking procedure using fixed tuple strategy , the elements X(i) of the solution vector is either 1 or 0 depending on if the weight W(i) is included or not.

If the state space tree of the solution, for a node at level I, the left child corresponds to X(i)=1 and right to X(i)=0.



Example:

Given n=6,M=30 and W(16)=(5,10,12,13,15,18).We have to generate all possible combinations of subsets whose sum is equal to the given value M=30.

In state space tree of the solution the rectangular node lists the values of s, k, r, where s is the sum of subsets,k is the iteration and r is the sum of elements after k in the original set.

The state space tree for the given problem is,

In the state space tree, edges from level i nodes to i+1 nodes are labeled with the values of Xi, which is either 0 or 1.



The left sub tree of the root defines all subsets containing Wi. The right subtree of the root defines all subsets, which does not include

Wi.

Generation of State space tree

Maintain an array X to represent all elements in the set. The value of Xi indicates whether the weight Wi is included or not. Sum is initialized to 0 i.e., s=0. We have to check starting from the first node.

Assign X(k)



//generate right child and evaluate Bk.

If ((S+ r- W[k]>=m)and(S+ W[k+1]



The backtracking algorithm helps to find Hamiltonian cycle for any type of graph.

Procedure:

Define a solution vector X(Xi..Xn) where Xi represents the I th visited vertex of the proposed cycle.

Create a cost adjacency matrix for the given graph. The solution array initialized to all zeros except X(1)=1,bcoz the cycle

should start at vertex 1.

Now we have to find the second vertex to be visited in the cycle. The vertex from 1 to n are included in the cycle one by one by checking

2 conditions, 1.There should be a path from previous visited vertex to current vertex.

2.The current vertex must be distinct and should not have been visited earlier.

When these two conditions are satisfied the current vertex is included

in the cycle, else the next vertex is tried.

When the nth vertex is visited we have to check, is there any path from

nth vertex to first 8vertex. if no path, the go back one step and after the

previous visited node.

Repeat the above steps to generate possible Hamiltonian cycle.

Algorithm:(Finding all Hamiltonian cycle)

Algorithm Hamiltonian (k)

{

Loop

Next value (k)

If (x (k)=0) then return;

{



If k=n then

Print (x)

Else

Hamiltonian (k+1);

End if

}

Repeat

}

Algorithm Nextvalue (k)

{

Repeat

{

X [k]=(X [k]+1) mod (n+1); //next vertex

If (X [k]=0) then return;

If (G [X [k-1], X [k]] 0) then

{

For j=1 to k-1 do if (X [j]=X [k]) then break;

// Check for distinction.

If (j=k) then //if true then the vertex is distinct.

If ((k



}

3.QUEENS PROBLEM:

This 8 queens problem is to place n-queens in an N*N matrix in such a

way that no two queens attack each otherwise no two queens should be in the

same row, column, diagonal.

Solution:

The solution vector X (X1Xn) represents a solution in which Xi is the column of the th row where I th queen is placed.

First, we have to check no two queens are in same row. Second, we have to check no two queens are in same column. The function, which is used to check these two conditions, is [I, X (j)],

which gives position of the I th queen, where I represents the row and X (j) represents the column position.

Third, we have to check no two queens are in it diagonal. Consider two dimensional array A[1:n,1:n] in which we observe that

every element on the same diagonal that runs from upper left to lower right has the same value.

Also, every element on the same diagonal that runs from lower right to upper left has the same value.

Suppose two queens are in same position (i,j) and (k,l) then two queens lie on the same diagonal , if and only if |j-l|=|I-k|.

STEPS TO GENERATE THE SOLUTION:

Initialize x array to zero and start by placing the first queen in k=1 in the first row.

To find the column position start from value 1 to n, where n is the no. Of columns or no. Of queens.

If k=1 then x (k)=1.so (k,x(k)) will give the position of the k th queen. Here we have to check whether there is any queen in the same column or diagonal.



For this considers the previous position, which had already, been found out. Check whether

X (I)=X(k) for column |X(i)-X(k)|=(I-k) for the same diagonal.

If any one of the conditions is true then return false indicating that k th queen cant be placed in position X (k).

For not possible condition increment X (k) value by one and precede d until the position is found.

If the position X (k) n and k=n then the solution is generated completely.

If kn then k th queen cannot be placed as the size of the

matrix is N*N. So decrement the k value by one i.e. we have to back track and after the

position of the previous queen.

Algorithm:

Algorithm place (k,I)

//return true if a queen can be placed in k th row and I th column. otherwise it

returns //

//false .X[] is a global array whose first k-1 values have been set. Abs

returns the //absolute value of r.

{

For j=1 to k-1 do

If ((X [j]=I) //two in same column.

Or (abs (X [j]-I)=Abs (j-k)))

Then return false;

Return true;

}



Algorithm Nqueen (k,n)

//using backtracking it prints all possible positions of n queens in n*n

chessboard. So

//that they are non-tracking.

{

For I=1 to n do

{

If place (k,I) then

{

X [k]=I;

If (k=n) then write (X [1:n]);

Else nquenns(k+1,n) ;

}

}

}

Example: 4 queens.

Q

Q

Q



Two possible solutions are

Solutin-1 Solution 2

(2 4 1 3) (3 1 4 2)

4. GRAPH COLORING:

Let G be a graph and m be a given positive integer. If the nodes of G can be colored in such a way that no two adjacent nodes have the same color. Yet only M colors are used. So its called M-color ability decision problem.

The graph G can be colored using the smallest integer m. This integer is referred to as chromatic number of the graph.

A graph is said to be planar iff it can be drawn on plane in such a way that no two edges cross each other.

Suppose we are given a map then, we have to convert it into planar. Consider each and every region as a node. If two regions are adjacent then the corresponding nodes are joined by an edge.

Consider a map with five regions and its graph.

Q

Q

Q

Q

Q



1 is adjacent to 2, 3, 4.

2 is adjacent to 1, 3, 4, 5

3 is adjacent to 1, 2, 4

4 is adjacent to 1, 2, 3, 5

5 is adjacent to 2, 4

Steps to color the Graph:

First create the adjacency matrix graph(1:m,1:n) for a graph, if there is an edge between i,j then C(i,j) = 1 otherwise C(i,j) =0.

The Colors will be represented by the integers 1,2,..m and the solutions will be stored in the array X(1),X(2),..,X(n) ,X(index) is the color, index is the node.

He formula is used to set the color is, X(k) = (X(k)+1) % (m+1)

First one chromatic number is assigned ,after assigning a number for k node, we have to check whether the adjacent nodes has got the same values if so then we have to assign the next value.

Repeat the procedure until all possible combinations of colors are found.

The function which is used to check the adjacent nodes and same color is,

1

3

5 4

A

l

g

o

r

it

h

m

m

C

o

l

o

r

i

n

g

(

k

)

2



If(( Graph (k,j) == 1) and X(k) = X(j))

Example:

N= 4

M= 3

Adjacency Matrix:

0 1 0 1

1 0 1 0

0 1 0 1

1 0 1 0

Problem is to color the given graph of 4 nodes using 3 colors.

Node-1 can take the given graph of 4 nodes using 3 colors.

The state space tree will give all possible colors in that ,the numbers which

are inside the circles are nodes ,and the branch with a number is the colors of

the nodes.

State Space Tree:

1

3

2

4



Algorithm:

Algorithm mColoring(k)

// the graph is represented by its Boolean adjacency matrix G[1:n,1:n] .All

assignments //of 1,2,.,m to the vertices of the graph such that adjacent

vertices are assigned //distinct integers are printed. k is the index of the next

vertex to color.

{

repeat

{

// generate all legal assignment for X[k].

Nextvalue(k); // Assign to X[k] a legal color.

If (X[k]=0) then return; // No new color possible.

If (k=n) then // Almost m colors have been used to color the n

vertices



Write(x[1:n]);

Else mcoloring(k+1);

}until(false);

}

Algorithm Nextvalue(k)

// X[1],X[k-1] have been assigned integer values in the range[1,m] such

that //adjacent values have distinct integers. A value for X[k] is determined in

the //range[0,m].X[k] is assigned the next highest numbers color while

maintaining //distinctness form the adjacent vertices of vertex K. If no such

color exists, then X[k] is 0.

{

repeat

{

X[k] = (X[k]+1)mod(m+1); // next highest color.

If(X[k]=0) then return; //All colors have been used.

For j=1 to n do

{

// Check if this color is distinct from adjacent color.

If((G[k,j] 0)and(X[k] = X[j]))

// If (k,j) is an edge and if adjacent vertices have the same color.

Then break;

}



if(j=n+1) then return; //new color found.

} until(false); //otherwise try to find another color.

}

The time spent by Nextvalue to determine the children is (mn)

Total time is = (mn n).

6.9.4.1 KNAPSACK PROBLEM USING BACKTRACKING:

The problem is similar to the zero-one (0/1) knapsack optimization problem is dynamic programming algorithm.

We are given n positive weights Wi and n positive profits Pi, and a positive number m that is the knapsack capacity, the is problem calls for choosing a subset of the weights such that,

ni

WiXi1

m and ni

PiXi1

is Maximized.

Xi Constitute Zero-one valued Vector.

The Solution space is the same as that for the sum of subsets problem. Bounding functions are needed to help kill some live nodes without

expanding them. A good bounding function for this problem is obtained by using an upper bound on the value of the best feasible solution obtainable by expanding the given live node.

The profits and weights are assigned in descending order depend upon the ratio.

(i.e.) Pi/Wi P(I+1) / W(I+1)



Solution :

After assigning the profit and weights ,we have to take the first object weights and check if the first weight is less than or equal to the capacity, if so then we include that object (i.e.) the unit is 1.(i.e.) K 1.

Then We are going to the next object, if the object weight is exceeded that object does not fit. So unit of that object is 0.(i.e.) K=0.

Then We are going to the bounding function ,this function determines an upper bound on the best solution obtainable at level K+1.

Repeat the process until we reach the optimal solution.

Algorithm:

Algorithm Bknap(k,cp,cw)

// m is the size of the knapsack; n no.of weights & profits. W[]&P[] are

the //weights & weights. P[I]/W[I] P[I+1]/W[I+1].

//fwFinal weights of knapsack.

//fp final max.profit.

//x[k] = 0 if W[k] is not the knapsack,else X[k]=1.

{

// Generate left child.

If((W+W[k] m) then

{

Y[k] =1;

If(k fp) and (k=n)) then

{



fp = cp + P[k];

fw = Cw+W[k];

for j=1 to k do X[j] = Y[j];

}

}

if(Bound(cp,cw,k) fp) then

{

y[k] = 0;

if(kfp) and (k=n)) then

{

fp = cp;

fw = cw;

for j=1 to k do X[j] = Y[j];

}

}

}

Algorithm for Bounding function:

Algorithm Bound(cp,cw,k)

// cp current profit total.

//cw current weight total.



//kthe index of the last removed item.

//mthe knapsack size.

{

b=cp;

c=cw;

for I =- k+1 to n do

{

c= c+w[I];

if (c



5+1

6.

Fp = (-1)

13 & 0+4 6 cw = 4,cp = 5,y(1) =1

k = k+2

23 but 7>6 so y(2) = 0

So bound(5,4,2,6)

B=5

C=4

I=3 to 3

C=6

6 6

So return 5+(1-(6-6))/(2*1)

5.5 is not less than fp. So, k=k+1 (i.e.) 3.

3=3 & 4+2 6



cw= 6,cp = 6, y(3)=1.

K=4.

If 4> 3 then Fp =6,fw=6,k=3 ,x(1) 1 0 1

The solution Xi 1 0 1

Profit 6

Weight 6.

BRANCH AND BOUND

Its is an algorithm design technique that enhances the idea of generating a

state-space tree with the idea of estimating the best value obtained from a

current node of the decision tree .If such an estimate is not superior to the

best solution seen up to that point in the processing, the node is eliminated

from further consideration.

Note that in the standard terminology of optimization problems, a feasible

solution is a point in the problems search space that satisfies all the

problems constraints.

(eg, a Hamiltonian circuit in the traveling saluman problem, a subset of item

whose total weight does not exceed the knapsacks capacity).

While an optimal solution is a feasible solution with the best value of the

objective function.(eg., the shortcut Hamiltonian circuit, the most valuable

subset of items that fit the knapsack).

Compared to backtracking, branch and bound required two additional items:



A way to provide, for every node of a state- space tree, a bound on the best

value of the objective function on any solution that can be obtained by

adding further components to the partial solution represented by the node.

The value of the best solution seen so far If this information is available, we

can compare a nodes bound value with the value of the best solution seen to

far . If the bound value is not better than the best solution seen so fa-(ie)., not

smaller for a minimization problem and not larger for a maximization

problem- the node is nonpromising and can be terminated. This is the

principal idea of the branch-and bound technique.

In general, we terminate a search path at the current node in a state-space

tree of a branch-and-bound algorithm for any one of the following three

reasons:

The value of the nodes bound is not better than the value of the best

solution seen so far.

The node represents no feasible solution because the constraints of the

problem are already violated.

The subset of feasible solution represented by the node consists of a

single point.

Branch-and- Bound algorithms

1. Assignment problem.

2. Knapsack problem.

3. Travelling saleman problem.

Assignment problem



Let us illustrate the branch-and-bound approach by applying it

to the problem of assigning n people to n jobs so that the total cost of the

assignment is as small as possible.

Job1 job2 job3 job4

C= 9 7 8 person a

6 4 7 person b

5 8 8 person c

7 6 9 person d

Lb=2+3+1+4

Lb=10

Best- first branch- and- bound

(the lower bound value(lb) is taken into account)

We start with the root that corresponds to no elements selected from

the cost matrix.

The lower bound value for the root, denoted lb is 10.

Select one element in each row of the matrix to that no two selected

elements are in the same column and their sum is the smallest possible.

4

2

2 3

1



Job1 job2 job3 job4

9 7 8 person a

C= 6 4 3 7 person b

5 8 8 person c

7 6 9 person d

There elements 3 and 1 are selected from the same column, but it can be

taken for the initial construction.

lb (lower bound)=2++3+1+4=10

Level 0

no of branches=4{1,2,3,4}nodes or branches.

0

Level1

1 2 3 4

Note: we have 4 live leaves (node 1 through 4)that may contain an optimal

solution.the most promising oh them is node 2 because it has the smallest

lower bound values.

start lb=2+3+1+4=10

1

4

4

4

4

4

2

2

3

a2(job2)

lb=2+3+1+4

=10

aaaaa

l

a 1(job 1)

lb=9+3+1+

4=17

a3(job3)

lb=7+4+5+4=

20

a4(job4)

lb=8+3+1+6=

18



=5+2+3+4

=14

( if the values selected from the same row, dont consider, go for the next

choice. If the value selected from the same colum, consider it for the initial

construction)

Note: node5 has the smallest lower bound value,so proceed with this.



6+2+1+ 4=(13)

= 6+2+9+8

=25

If c is assigned job3 a is assigned job2

Then d is assigned job4 b is assigned job1

C is assigned job3

d is assigned job4

Note: Thus based on the smallest lower bound value , the respective branch

is extended on order to find the optimal solution for other cases.

TRAVELLING SALESMAN PROBLEM

INTRODUCTION:

It is algorithmic procedures similar to backtracking in which a new

branch is chosen and is there (bound there) until new branch is choosing for

advancing.

This technique is implemented in the traveling salesman problem [TSP]

which are asymmetric (Cij Cij) where this technique is an effective

procedure.

STEPS INVOLVED IN THIS PROCEDURE ARE AS FOLLOWS:

STEP 0: Generate cost matrix C [for the given graph g]

STEP 1: [ROW REDUCTION]

For all rows do step 2



STEP2: Find least cost in a row and negate it with rest of the

elements.

STEP 3: [COLUMN REDUCTION]

Use cost matrix- Row reduced one for all columns do STEP 4.

STEP 4: Find least cost in a column and negate it with rest of the elements.

STEP 5: Preserve cost matrix C [which row reduced first and then column

reduced] for the i th time.

STEP 6: Enlist all edges (i, j) having cost = 0.

STEP 7: Calculate effective cost of the edges. (i, j)=least cost in the i th

row excluding (i, j) + least cost in the j th column excluding (i, j).

STEP 8: Compare all effective cost and pick up the largest l. If two or more

have same cost then arbitrarily choose any one among them.

STEP 9: Delete (i, j) means delete ith row and jth column change (j, i) value

to infinity. (Used to avoid infinite loop formation) If (i,j) not

present, leave it.

STEP 10: Repeat step 1 to step 9 until the resultant cost matrix having

order of 2*2 and reduce it. (Both R.R and C.C)

STEP 11: Use preserved cost matrix Cn, Cn-1 C1

Choose an edge [i, j] having value =0, at the first time for a

preserved matrix and leave that matrix.

STEP 12: Use result obtained in Step 11 to generate a complete tour.

EXAMPLE: Given graph G



MATRIX:

1 2 3 4 5

1

2

3

4

5

PHASE I

STEP 1: Row Reduction C

25 40 31 27

5 17 30 25

19 15 6 1

9 50 24 6

22 8 7 10



C1 [ROW REDUCTION:

1 2 3 4 5

1

2

3

4

5

STEP 3: C1 [Column Reduction]

1 2 3 4 5

1

2

3

4

5

STEP 5:

Preserve the above in C1,

0 15 6 2

0 12 25 20

18 14 5 0

3 44 18 0

15 1 0 3

0 15 3 2

0 12 25 20

18 14 2 0

3 44 18 0

15 1 0 3



1 2 3 4 5

1

2

3

4

5

STEP 6:

L= (5,4)(5,3),(4,5),(3,5),(2,1),(1,2),

STEP 7:

Calculation of effective cost [E.C]

(1,2) = 2+1 =3

(2,1) = 12+3 = 15

(3,5) = 2+0 =2

(4,5) = 3+0 = 3

(5,3) = 0+12 = 12

(5,4) = 0+2 = 2

STEP 8:

L having edge (2,1) is the largest.

0 15 3 2

0 12 22 20

18 14 2 0

3 44 18 0

15 1 0 3



STEP 9: Delete (2,1) from C1 and make change in it as (1,2) if exists.

Now Cost Matrix =

2 3 4 5

1

3

4

5

STEP 10:

The Cost matrix 2 x 2.

Therefore, go to step 1.

PHASE II:

STEP1: C2(R, R)

2 3 4 5

1

3

4

5

15 3 2

14 2 0

44 18 0

1 0 0

13 1 0

14 2 0

44 18 0

1 0 0



STEP 3: C2 (C, R)

2 3 4 5

1

3

4

5

STEP 5: Preserve the above in C2

C2 =

2 3 4 5

1

3

4

5

STEP 6:

L= {(1,5), (3,5), (4,5), (5,2), (5,3), (5,4)}

STEP 7: calculation of E.C.

(1,5) = 1+0 =1

13 1 0

13 2 0

43 18 0

0 0 0

13 1 0

13 2 0

43 18 0

0 0 0



(3,5) = 2+0 =2

(4,5) = 18+0 =18

(5,2) = 0+13 =13

(5,3) = 0+13 =13

(5,4) = 0+1 =1

STEP 8: L having an edge (4,5) is the largest.

STEP 9: Delete (4,5) from C2 and make change in it as (5,4) =

if exists.

Now, cost matrix

2 3 4

1

3

5

STEP 10: THE cost matrix 2x2 hence go to step 1

PHASE III:

STEP 1: C3 (R, R)

2 3 4

13 1

13 2

0 0



1

3

5

STEP 3: C3 (C, R)

2 3 4

1

3

5

STEP 5: preserve the above in C2

STEP 6: L={(1,4), (3,4), (5,2), (5,3)}

STEP 7: calculation of E.C

(1,4)=12+0=12

(3,4)=11+0=11

(5,2)=0+11=11

(5,3)=0+12=12

STEP 8: Here we are having two edges (1,4) and (5,3) with cost = 12. Hence

arbitrarily choose (1,4)

12 0

11 0

0 0

12 0

11 0

0 0



STEP 9: Delete (i,j) (1,4) and make change in it (4,1) = if exists.

Now cost matrix is

2 3

2

3

STEP 10: We have got 2x2 matrix

C4 (RR)=

2 3

3

5

C4 (C, R) =

2 3

3

5

Therefore,C4 = 2 3

11

0 0

0

0 0

0

0 0



3

5

STEP 11: LIST C1, C2, C3 AND C4

C4 2 3

3

5

C3 2 3 4

1

3

5

C2 =

1

3

4

5

C1 =

0

0 0

0

0 0

12 0

11 0

0 0

13 1 0

13 2 0

43 18 0

0 0 0



1 2 3 4 5

1

2

3

4

5

TEP 12:

i) Use C4 =

2 3

3

5

Pick up an edge (I, j) =0 having least index

Here (3,2) =0

Hence, T (3,2)

Use C3 =

2 3 4

0 15 3 2

0 12 22 20

18 14 2 0

3 44 18 0

15 1 0 0

0

0 0

12 0



1

3

5

Pick up an edge (i, j) =0 having least index

Here (1,4) =0

Hence, T(3,2), (1,4)

Use C2=

2 3 4 5

1

3

4

5

Pick up an edge (i, j) with least cost index.

Here (1,5) not possible because already chosen index i (i=j)

(3,5) not possible as already chosen index.

(4,5) 0

Hence, T (3,2), (1,4), (4,5)

11 0

0 0

13 1 0

13 2 0

43 18 0

0 0 0



Use C1 =

1 2 3 4 5

1

2

3

4

5

Pick up an edge (i, j) with least index

(1,2) Not possible

(2,1) Choose it

HENCE T (3,2), (1,4), (4,5), (2,1)

SOLUTION:

From the above list

32145

This result now, we have to return to the same city where we started (Here 3).

Final result:

321453

0 15 3 2

0 12 22 20

18 14 2 0

3 44 18 0

15 1 0 0



COST IS 15+15+31+6+7=64

6.9.5.2 KNAPSACK PROBLEM :

Knapsack problem (np-hard problems)

Given n items of know weights w1.wn of values v1vn and a knapsack

of capacity w, find the most valuable subset of the items that fit into the

knapsack.

Do the possible subsets and find the total weight that should fit into the

knapsack according to its capacity (10) and at the same time select the

highest profit (is most valuable ones)

Subset Total weight Total value

{1} 7 $42

{2} 3 $12

{3} 4 $40



{4} 5 $25

{1,2} 10 $36

{1,3} 11 not feasible x

{1,4} 12 not feasible x

{2,3} 7 $52

{2,4} 8 $37

{1,2,3} 14 not feasible x




{1,2,3,4} 19 not feasible x

Therefore, the items 3&4can be placed into the knapsack with more profit

Rearrange the total weight of items 3 &4 is 9 & total value is $65.

Knapsack problem (Branch and bound)

Let us now discuss how we can apply the branch and bound technique to

solve the knapsack problem.

Given n items of known weight wi and values vi , i=1,2,.n and a knapsack of

capacity w , find the most valuable subset of the items that fit in the

knapsack.

Note:

{3,4 } 9 $65



It is convenient to order the items of an given instance in descending order by

their value- to weight ratios

V1 > v2 > .> vn

W1 w2 wn

Ub = V + (W - w) (Vi+1 / Wi+1)

where W = capacity

w = weight

Vi+1 = value

Wi+1 = weight

1.It is natural to structure the state-space tree for this problem as a binary

tree constructed as follows.

2.Each node on the ith level of this tree, 0 < i < n, represents all the subset of

n items that include a particular selection made from the first I ordered

items.

This particular selection is uniquely determined by a path from the root to

the node: a branch going to the left indicates the inclusion of the next item

while the branch going to the right indicate its exclusion.

3.we record the total weight w and the total value v of this selection in the

node, along with some upper bound ( ub) on the value of any subject that can

be obtained by adding zero or more items to this selection .

4. A single way to compute the upper bound us is to add to v1 the total value

of the items already selected, the product of the remaining capacity of the

knapsack(W-w) & the best per unit pay off among the remaining items

,which is vi+1\wi+1.



Not feasible. Since weight > 0

Eg :-

item 1 2 3 4

weight w=7 w=3 w=4 w=5

value v=42 v=12 v=40 v=25

Capacity weight = 10

Solution :

(3,4)

TW = 9

TV = 65

Branch and Bound Technique :

Item Weight Value Value/Weight

1 4 $40 10

Subset Total Weight

Total Value

1 7 42 2 3 42 3 4 40 4 5 25 1,2 10 54 1,3 11 82 1,4 12 67 2,3 7 52 2,4 8 37 3,4 9 65 1,2,4 15 79 1,2,3 14 94 1,3,4 16 107 2,3,4 12 117 1,2,3,4 19 119



1

6 5

4 3

2 1

0

1

W0

2 W 2

W0 1 W 1

8 7

1 1

1

W0

4 W 4

Wo

3

W

3

2 7 $42 6 Average in descending order

3 5 $25 5 of V/W, for items

4 3 $12 4

Capacity Weight = 10

i = 0 W=0 V=0

Ub = 0 + (10 -0)(10) = 100

W=0 ; V=0 Ub = 100

W=4 ; V=40 Ub = 76

W=0 ; V=0 Ub = 60

W=11

W=4 ; V=40 Ub = 70

W=9 ; V=65 Ub = 69

W=4 ; V=40 Ub = 64

W=12

W=9 ; V=65 Ub = 65



i = 1 W=4 V=40

Ub = 40 + (10-4)(6) = 76

i = 2 W = 7+4 = 11 > 10 Therefore Not feasible.

i = 2 w/0 2 W=4 V=40

Ub = 40 + (10-4)(5) = 70

i = 3 W=9 V=65

Ub = 65 + (10-9)(4) = 69

i = 3 w/o W=4

Therfore Solution is (1,3)

RANDOMIZED ALGORITHM :

Skip List

Skip Lists were invented by William Pugh in 1989. It is an alternative to BST

and other balanced trees.

Definition : A Skip list is a probabilistic data structure, where elements are

kept sorted by key. It allows quick search, insertions and deletions of

elements with simple Algorithms.



It is basically a linked list with additional pointers, such that intermediate

nodes can be skipped.

It uses a Random number generator to make some decision.

it randomly chooses a level for the nodes, according to the node distribution,

that is, toss a coin until you see a head, and use the no. of tosses as the node

level.

Skip List Algorithm :

Search :

Start from the highest level and move downwards to perform the

proper search.

Insertion :

Find the position, randomly select a node level and insert it.

Deletion :

Start from the highest level; move downwards; find the position;

delete it

Randomised Algorithm :

It uses random number during the Algorithm.

Why do we need Randomised Algorithm?

Algorithms sometimes suffer from bad inputs. But we do not want to

get caught. Sometimes deterministic algorithms are too slow and not even



feasible, but randomized algorithms casn give us good results with higher

probability.

It is not possible to have true randomness. But Pseudo Randomness is

possible for more than random number generation is required.

Documents

IT2201 Algorithm Design and Analysis UNIT 5