Embed Size (px)
Citation preview
Chapter Two
Algorithm Analysis Empirical vs. theoretical Space vs. time Worst case vs. Average case Upper, lower, or tight bound Determining the runtime of programs What about recursive programs?
What’s the runtime?int n;
cin >> n;
for (int i=0; i<n; i++)
for (int j=0; j<n; j++)
for (int k=0; k<n; k++)
cout << “Hello world!\n”;
What if the last line is replaced by:
string *s=new string(“Hello world!\n”);
O(n3) runtime
O(n3) time and space
2n3+n2+n+2?
Resource Analysis Runtime: we’d like to count the steps –
but that would be machine dependent
Space: we may also be interested in space usage
ignore constant factors, use O() notation count steps equivalent to machine language
instructions
count the bytes used
Asymptotic notation g(n) is said to be O(f(n)) if there exist constants c and n0
such that g(n) < c f(n) for all n > n0
g(n) is said to be (f(n)) if there exist positive constants c and n0 such that 0 <= c f(n) < g(n) for all n > n0
g(n) is said to be (f(n)) if g(n) = O(f(n)) and g(n) = (f(n))
O: like <= for functions (asymptotically speaking) : like >= : like =
for all n > n0
ignore constant factors, lower order terms
Asymptotic notation: examples Asymptotic runtime, in terms of O, ,
? Suppose the runtime for a function is
n2 + 2n log n + 40 0.0000001 n2+ 1000000n1.999
n3 + n2 log n n2.0001 + n2 log n 2n+ 100 n2
1.00001n+ 100 n97
Asymptotic comparisons 0.0000001 n2 = O(1000000n1.999 )?
n1.000001 = O(n log n)?
1.0001n = O(n943)?
lg n = (ln n)?
(Compare the limit of the quotient of the functions)
No – a polynomial with a higher power dominates one with a lower power
No – all polynomials (n.000001) dominate any polylog (log n)
No – all exponentials dominate any polynomial
Yes – different bases are just a constant factor difference
What’s the runtime?int n;
cin >> n;
for (int i=0; i<n; i++)
for (int j=0; j<n; j++)
for (int k=0; k<n; k++)
cout << “Hello world!\n”;
for (int i=0; i<n; i++)
for (int j=0; j<n; j++)
for (int k=0; k<n; k++)
cout << “Hello world!\n”;
(n3) + (n3) = (n3)Statements or blocks in sequence: add
What’s the runtime?int n;
cin >> n;
for (int i=0; i<n; i++)
for (int j=n; j>1; j/=2)
cout << “Hello world!\n”;
Loops: add up cost of each iteration(multiply loop cost by number of iterations
if they all take the same time)
log n iterations of n steps (n log n)
What’s the runtime?int n;
cin >> n;
for (int i=0; i<n; i++)
for (int j=0; j<i; j++)
cout << “Hello world!\n”;
Loops: add up cost of each iteration
1 + 2 + 3 + … + n = n(n+1)/2 = O(n2)
What’s the runtime?template <class Item>
void insert(Item a[], int l, int r)
{ int i;
for (i=r; i>l; i--) compexch(a[i-1],a[i]);
for (i=l+2; i<=r; i++)
{ int j=i; Item v=a[i];
while (v<a[j-1])
{ a[j] = a[j-1]; j--; }
a[j] = v;
}
}
What’s the runtime?void myst(int n)
{ if (n<100)
for (int i=0; i<n; i++)
for (int j=0; j<n; j++)
for (int k=0; k<n; k++)
cout << “Hello world!\n”;
else
for (int i=0; i<n; i++)
for (int j=0; j<n; j++
cout << “Hello world!\n”;
}
Estimate the runtime Suppose an algorithm has runtime (n3)
suppose solving a problem of size 1000 takes 10 seconds. How long to solve a problem of size 10000?
Suppose an algorithm has runtime (n log n)
suppose solving a problem of size 1000 takes 10 seconds. How long to solve a problem of size 10000?
runtime 10-8 n3; if n=10000, runtime 10000s = 2.7hr
runtime 10-3 n lg n; if n=10000, runtime 133 secs
Worst vs. average case You might be interested in worst, best, or
average case analysis of an algorithm You can have upper, lower, or tight bounds on
each of those functions. Eg. For each n, some problem instances of
size n have runtime n and some have runtime n2.
Worst case: Best case: Average case:
(n2), (n), (log n), O(n2), O(n3)
(n), (log n), O(n2), (n)
(n), (log n), O(n2), O(n3)
Average case: need to know distribution of inputs
The Taxpayer Problem Tax time is coming up. The IRS needs to
process tax forms. How to access and update each taxpayer’s info?
ADT? ADT Dictionary: find(x), insert(x),
delete(x) Implementation?
Array Implementation Insert(x):
Find(k):
Delete(I):
Records[numRecs++] = x;Runtime: O(1)
For (I=0; I<numRecs; I++) if (records[I].key == k)
return I;Runtime: O(n)
records[I]=records[--numRecs];Runtime: O(1)
Time for nOperations?
O(n2)
Sorted Array Implementation Find(x):
Runtime?
int bot=1, top=numRecs-1, mid;
while (bot <= top) {
mid = (bot + top)/2;
if (data[mid]==x) return mid;
if (data[mid]<x) top=mid-1; else bot=mid+1;
}
return –1;
Analysis of Binary Search How many steps to search among n
items? Number of items eliminated at each step? Definition of lg(x)? Runtime?
O(log n)
Sorted Array, cont. Insert(x)?
Delete(x)?
Time for n insert, delete, and find ops?
O(n)
O(n)
O(n2)
Which implementation is better?
find(x) insert(x) delete(x)ArrayS. Array
Worst case for n operations? Array: Sorted Array:
O(n2)
What if some operations are more frequent than others?
O(n2)
O(log n) O(n) O(n)
O(n) O(1) O(1)
Molecule viewer example Java demos: molecule viewer
Example1 Example2 Example3
Molecule Viewer Source Snippet/* * I use a bubble sort since from one iteration to the next, the sort * order is pretty stable, so I just use what I had last time as a * "guess" of the sorted order. With luck, this reduces O(N log N) * to O(N) */
for (int i = nvert - 1; --i >= 0;) { boolean flipped = false; for (int j = 0; j <= i; j++) {
int a = zs[j];int b = zs[j + 1];if (v[a + 2] > v[b + 2]) { zs[j + 1] = a; zs[j] = b; flipped = true; }
} if (!flipped) break;}
Merge sort runtime?void mergesort(first, last) {
if (last-first >= 1) {
mid=(last-first)/2 + first;
mergesort(first, mid);
mergesort(mid+1,last);
merge(first, mid, last);
}
}
T(n) = 2T(n/2) + c nT(1) = b;
Called a recurrence relation
Recurrence relationsIn Discrete Math: you’ll learn how to solve these.
In this class: we’ll say “Look it up.”
But you will be responsible for knowing how to write down a recurrence relation for the runtime of a program.
Divide-and-conquer algorithms like merge sort that divide problem size by 2 and use O(n) time to conquer
T(n) = 2T(n/2) + c n
have runtime O(n log n)
Hanoi runtime?void hanoi(n, from, to, spare {
if (n > 0) {
hanoi(n-1,from,spare,to);
cout << from << “ – “ << to << endl;
hanoi(n-1,spare,to,from);
}
}
T(n) = 2T(n-1) + cT(0) = b
Look it up: T(n) = O(2n)
Hanoi recurrence solutionT(n)=2T(n-1)+cT(n-1) = 2T(n-2) + cT(n-2) = 2T(n-3) + c_______T(n) = 2T(n-1) + c = 2 [ 2T(n-2) + c ] + c = 22 T(n-2) + 2 c + c = 23 T(n-3) + 22 c + 21 c + 20 c … = 2k T(n-k) + 2k-1 c + 2k-2 c + … + 21 c + 20 c = 2k T(n-k) + c(2k – 1)Done when n-k=0 since we know T(0). T(n) = 2n b + c 2n - c = (2n)
Binary Search recurrence?Recurrence relation?
T(n)=T(n/2)+c; T(1) = b
Look it up: (log n)
