Asymptotic Notation, Review of Functions & SummationsBy, Nikhilesh Walde CSE Department PIET26 April 2010 Comp 122, Spring 2004

Asymptotic Complexity Running time of an algorithm as a function of input size n for large n. Expressed using only the highest-order term in the expression for the exact running time. Instead of exact running time, say 5(n2). Describes behavior of function in the limit.

Written using Asymptotic Notation.

asymp - 1

Comp 122

Asymptotic Notation 5, O, ;, o, [ Defined for functions over the natural numbers. Ex: f(n) = 5(n2). Describes how f(n) grows in comparison to n2. Define a set of functions; in practice used to compare two function sizes. The notations describe different rate-of-growth relations between the defining function and the defined set of functions.

asymp - 2

Comp 122

5-notationFor function g(n), we define 5(g(n)), big-Theta of n, as the set:

5(g(n)) = {f(n) : positive constants c1, c2, and n0, such that n u n0, we have 0 e c1g(n) e f(n) e c2g(n)

}Intuitively: Set of all functions that have the same rate of growth as g(n).

g(n) is an asymptotically tight bound for f(n).asymp - 3 Comp 122

5-notationFor function g(n), we define 5(g(n)), big-Theta of n, as the set:

5(g(n)) = {f(n) : positive constants c1, c2, and n0, such that n u n0, we have 0 e c1g(n) e f(n) e c2g(n)

}Technically, f(n) 5(g(n)). Older usage, f(n) = 5(g(n)). Ill accept either f(n) and g(n) are nonnegative, for large n.asymp - 4 Comp 122

Example5(g(n)) = {f(n) : positive constants c1, c2, and n0, such that n u n0, 0 e c1g(n) e f(n) e c2g(n)}

10n2 - 3n = 5(n2) What constants for n0, c1, and c2 will work? Make c1 a little smaller than the leading coefficient, and c2 a little bigger. To compare orders of growth, look at the leading term. Exercise: Prove that n2/2-3n= 5(n2)asymp - 5 Comp 122

O-notationFor function g(n), we define O(g(n)), big-O of n, as the set: O(g(n)) = {f(n) : positive constants c and n0, such that n u n0,we have 0 e f(n) e cg(n) }

Intuitively: Set of all functions whose rate of growth is the same as or lower than that of g(n).

g(n) is an asymptotic upper bound for f(n).f(n) = 5(g(n)) f(n) = O(g(n)). 5(g(n)) O(g(n)).asymp - 6 Comp 122

; -notationFor function g(n), we define ;(g(n)), big-Omega of n, as the set:

;(g(n)) = {f(n) : positive constants c and n0, such that n u n0,we have 0 e cg(n) e f(n)}

Intuitively: Set of all functions whose rate of growth is the same as or higher than that of g(n).

g(n) is an asymptotic lower bound for f(n).f(n) = 5(g(n)) f(n) = ;(g(n)). 5(g(n)) ;(g(n)).asymp - 7 Comp 122

Relations Between 5, O, ;

asymp - 8

Comp 122

Relations Between 5, ;, OTheorem : For any two functions g(n) and f(n), f(n) = 5(g(n)) iff f(n) = O(g(n)) and f(n) = ;(g(n)).

I.e., 5(g(n)) = O(g(n)) ;(g(n)) In practice, asymptotically tight bounds are obtained from asymptotic upper and lower bounds.

asymp - 9

Comp 122

Asymptotic Notation in Equations Can use asymptotic notation in equations to replace expressions containing lower-order terms. For example, 4n3 + 3n2 + 2n + 1 = 4n3 + 3n2 + 5(n) = 4n3 + 5(n2) = 5(n3). How to interpret? In equations, 5(f(n)) always stands for an anonymous function g(n) 5(f(n)) In the example above, 5(n2) stands for 3n2 + 2n + 1.

asymp - 10

Comp 122

o-notationFor a given function g(n), the set little-o:o(g(n)) = {f(n): c > 0, n0 > 0 such that n u n0, we have 0 e f(n) < cg(n)}.f(n) becomes insignificant relative to g(n) as n approaches infinity:

lim [f(n) / g(n)] = 0npg

g(n) is an upper bound for f(n) that is not asymptotically tight. Observe the difference in this definition from previous ones. Why?asymp - 11 Comp 122

[ -notationFor a given function g(n), the set little-omega:

[(g(n)) = {f(n): c > 0, n0 > 0 such that n u n0, we have 0 e cg(n) < f(n)}.f(n) becomes arbitrarily large relative to g(n) as n approaches infinity:npg

lim [f(n) / g(n)] = g.

g(n) is a lower bound for f(n) that is not asymptotically tight.asymp - 12 Comp 122

Longest Common Subsequence (LCS) In biological application, to compare the DNA of two or more different organism A strand of DNA consist of string of molecules called bases A strand of DNA can be expressed over the finite set { A,C,G,T}

26 April 2010

Comp 122, Spring 2004

Longest Common Subsequence (LCS) Example: S1 = ACCGGTCGAGTGCGCGGAAGCCGGCCGAA S2 = GTCGTTCGGAATGCCGTTGCTCTGTAAA FIND S3, in which the bases in S3 appear in each of S1 and S2. The bases must appear in the same order but not necessarily consecutively. S3 = GTCGTCGGAAGCCGGCCGAA---- LCSasymp - 14 Comp 122

Longest Common Subsequence (LCS) Example : X = ( A, B, C, B, D, A, B) Y = ( B, D, C, A, B, A) Z1 = (B, C, A) Z2 = (A, B, A) Z3 = (B, C, B, A)

asymp - 15

Comp 122

LCS_ LENGTH (X,Y)1 2 3 4 5 6 m length [X] n length [Y] for i 1 to m do c[i,0] 0 for j 0 to n do c[0,j] 0

asymp - 16

Comp 122

LCS_ LENGTH (X,Y) 7 for i 1 to m 8 do for j 1 to n 9 do if Xi = Yi 10 then c [i, j] c [i-1,j-1] + 1 11 b[i, j] Inclined Arrow 12 else if c [ i-1, j] >= c [ i, j-1] 13 c [i, j] c [i-1, j] 14 b[i, j] Upward Arrow 15 else c [i, j] c [i, j-1] 16 b[i, j] Leftward Arrow 17 Return c and bComp 122

asymp - 17

Print_LCS (b, X, i, j) 1 if i = 0 or j = 0 2 then return 3 if b[i, j] = Inclined arrow 4 then Print_LCS(b, X, i-1, j-1) 5 Print Xi 6 elseif b[i, j] = upward Arrow 7 then Print_LCS(b, X, i-1, j) 8 else Print_LCS(b, X, i, j-1)asymp - 18 Comp 122

Matrix Chain Multiplication Given a sequence (chain) < A1,A2,A3.An> We wish to find the product of n matrices. Use a standard algorithm for multiplying pair of matrices as a subroutine once we have parenthesized them. For e.g. The Product can be fully parenthesized in five different ways:asymp - 19 Comp 122

Matrix Chain Multiplication (A1 (A2(A3A4))) (A1 ((A2A3 )A4)) ((A1A2)(A3A4)) ((A1 (A2A3))A4) ((A1A2)A3) A4) Matrix multiplication is associative All Parenthesization yield the same product.

asymp - 20

Comp 122

Matrix_Multiply (A,B) 1 if column [A] row [B] 2 then error multiplication cannot perform 3 else for i 1 to row [A] 4 do for j 1 to column [B] 5 do C[i , j] 0 6 for k 1 to row [B] 7 do C[i,j] C[i,j]+ A[i,k]* B[k,j] 8 Return Casymp - 21 Comp 122

Matrix Chain Multiplication If A[p,q] and B[q,r] then resultant matrix C[p,r] The time to compute C is dominated by the number of scalar multiplication in Line 7 Which is p x q x r. Different cost is incurred by different parenthesization of a matrix product. For e.g. A1 = 10 x 100, A2 = 100 x 5, A3 = 5 x 50 ((A1.A2).A3) and (A1.(A2.A3))asymp - 22 Comp 122

Matrix Chain Multiplication Matrix A1 A2 A3 A4 A5 A6 Dimension 30 x 35 35 x 15 15 x 5 5 x 10 10 x 20 20 x 25

asymp - 23

Comp 122

Matrix_Chain_Order (p) 1 n length[p] 1 2 for i 1 to n 3 do m[i, i] 0 4 for 2 to n // is the chain length. 5 do for i 1 to n - +1 6 do j i + -1 7 m[i, j] infinity

asymp - 24

Comp 122

Matrix_Chain_Order (p)8 for k i to j 1 9 do q m[i,k] + m[k+1,j] + Pi-1 PkPj 10 if q < m[i, j] 11 then m[i, j] q 12 s[i, j] k 13 return m and s

asymp - 25

Comp 122

Print_Optimal _Parens(s,i,j) 1 if i = j 2 then print A 3 else Print ( 4 Print_Optimal_Parens (s,i,s[i,j]) 5 Print_Optimal_Parens (s,s[i,j]+1, j) 6 Print ) Output = ((A1(A2A3))((A4A5)A6)).asymp - 26 Comp 122

Assembly Line Scheduling Dynamic programming solves a manufacturing problem The Colonel Motors Corporation produces automobiles in a factory that has two assembly lines An automobile chassis enters each assembly line, has parts added to it at a number of stations, and a finished auto exits at the end of the lineasymp - 27 Comp 122

Assembly Line Scheduling

asymp - 28

Comp 122

Assembly Line Scheduling Each assembly line has n stations, numbered j = 1, 2, ..., n. We denote the jth station on line i (where i is 1 or 2) by Si,j. The jth station on line 1 (S1,j) performs the same function as the jth station on line 2 (S2,j). The stations were built at different times and with different technologies, however, so that the time required at each station varies, even between stations at the same position on the two different linesasymp - 29 Comp 122

Assembly Line Scheduling Chassis enters station 1 of one of the assembly lines, and it progresses from each station to the next. Entry time ei for the chassis to enter assembly line i and an exit time xi for the completed auto to exit assembly line i. Find the fastest way through a factory After going through the jth station on a line, the chassis goes on to the (j + 1)st station on either line.asymp - 30 Comp 122

Assembly Line Scheduling There is no transfer cost if it stays on the same line, but it takes time ti,j to transfer to the other line after station Si,j. The problem is to determine which stations to choose from line 1 and which to choose from line 2 in order to minimize the total time through the factory for one auto. Generally, It is used in rush Order.

asymp - 31

Comp 122

Assembly Line Scheduling

asymp - 32

Comp 122

Assembly Line Scheduling

asymp - 33

Comp 122

Assembly Line Scheduling

asymp - 34

Comp 122

FASTEST-WAY(a, t, e, x, n)1 f1[1] e1 + a1,1 2 f2[1] e2 + a2,1 3 for j 2 to n 4 do if f1[j - 1] + a1,j f2[j - 1] + t2,j-1 + a1,j 5 then f1[j] f1[j - 1] + a1, j 6 l1[j] 1 7 else f1[j] f2[j - 1] + t2,j-1 + a1,j 8 l1[j] 2asymp - 35 Comp 122

FASTEST-WAY(a, t, e, x, n)9 if f2[j - 1] + a2,j f1[j - 1] + t1,j-1 + a2,j 10 then f2[j] f2[j - 1] + a2,j 11 l2[j] 2 12 else f2[j] f1[j - 1] + t1,j-1 + a2,j 13 l2[j] 1 14 if f1[n] + x1 f2[n] + x2 15 then f* = f1[n] + x1 16 l* = 1 17 else f* = f2[n] + x2 18 asymp - 36 Comp 122 l* = 2

PRINT-STATIONS(l, n)1 2 3 4 5 i l* print "line " i ", station " n for j n downto 2 do i li[j] print "line " i ", station " j - 1

asymp - 37

Comp 122

Assembly Line Scheduling PRINT-STATIONS would produce the output line 1, station 6 line 2, station 5 line 2, station 4 line 1, station 3 line 2, station 2 line 1, station 1

asymp - 38

Comp 122

Huffman codes Huffman codes are a widely used and very effective technique for compressing data. Savings of 20% to 90% memory, depending on the characteristics of the data being compressed. We consider the data to be a sequence of characters. Huffman's greedy algorithm uses a table of the frequencies of occurrence of the characters to build up an optimal way of representing each character as a binary string.asymp - 39 Comp 122

Huffman codes Suppose we have a 100,000-character data file that we wish to store compactly. We observe that the characters in the file occur with the frequencies given by Figure. That is, only six different characters appear, and the character a occurs 45,000 times. If each character is assigned a 3-bit Fix Length codeword,the file can be encoded in 300,000 bits. Using the variable-length code shown, the file can be encoded in 224,000 bits.asymp - 40 Comp 122

Huffman codes

Frequency (in thousands) Fixed-length codeword Variable-length codeword

a b c d e f 45 13 12 16 9 5 000 001 010 011 100 101 0 101 100 111 1101 1100

asymp - 41

Comp 122

Fixed-length codeword

asymp - 42

Comp 122

Variable-length codeword

asymp - 43

Comp 122

Huffman codes

asymp - 44

Comp 122