LECTURE NOTES OF ALGORITHMS: DESIGN ...Asymptotic Analysis 4 March 2017 Ass. Prof. Dr. Samaher Al_Janabi Notes of Lecture #3 Example #3: Problem: Do arrays A, B have a number in common?

Faculty of Science for Women( SCIW), University of Babylon, Iraq

[email protected]

By

Ass. Prof. Dr. Samaher Al_Janabi

LECTURE NOTES OF ALGORITHMS:

DESIGN TECHNIQUES AND

ANALYSIS

Department of Computer Science University of Babylon

4 March 2017

Outlines

• Asymptotic Analysis

• The Gist

• Big-oh Notation

• Big Omega Notation

• Big- Theta Notation

• Basic Examples and claim need prof

• Fibonacci numbers

• Prefix averages for set of numbers

• Fibonacci numbers using recursion

Ass. Prof. Dr. Samaher Al_JanabiNotes of Lecture #3

4 March 2017

The Gist

Motivation

• Analysis provides basic vocabulary for discussing the design and analysis inalgorithms. More, it is a mathematical concept it is by no means math formaths sake. For example, You will very frequently hear serious programmerssaying that such and such code runs at O(n) time, where such and such othercode runs in o ( n^2) times. It's important you know what programmersmean? when they make statements like that? .

• A Sweet spot for discussing the high level performance(reasoning) ofalgorithms. {What I mean by that is, it is on the one hand coarse enough tosuppress all of the details that you want to ignore. Details that depend on thechoice of architecture, the choice of programming language, the choice ofcompiler} .

• On the other hand, it's sharp enough to be useful. In particular, to make predictive comparisons between different high level algorithmic approaches to solving a common problem. This is going to be especially true for large inputs ( e.g. Sorting and integer multiplication)

Ass. Prof. Dr. Samaher Al_JanabiNotes of Lecture #3

4 March 2017

Asymptotic Analysis

High level idea

suppresses constant factors (I.e. too system dependent ) and lower-order terms (I.e. irrelevant oflarge input). Well lower-order terms basically by definition become increasingly irrelevant as youfocus on large inputs.

Ass. Prof. Dr. Samaher Al_Janabi Notes of Lecture #34 March 2017

Example #1: Problem: Does array A contain the integer t? Given A (array of length n) and t (an integer).Algorithm 1

1: for i = 1 to n do

2: if A[i] == t then

3: Return TRUE

4: Return FALSE

What is the running time of this piece of code? [hint: this example have one loop]

a. O(1)

b. O(logn)

c. O(n)

d. O(n2)

Example #2: Given A, B (arrays of length n) and t (an integer). [Does A or B contain t?]Algorithm 2

1:for i = 1 to n do

2: if A[i] == t then

3: Return TRUE


5: if B[i] == t then

6: Return TRUE

7: Return FALSE

What is the running time of this piece of code? [hint: this example have one loop]

a. O(1)

b. O(logn)

c. O(n)

d. O(n2)

Asymptotic Analysis


Example #3: Problem: Do arrays A, B have a number in common? Given arrays A, B of length n.Algorithm 3


2: for j = 1 to n do

3: if A[i] == B[j] then

4: Return TRUE

5: Return FALSE

What is the running time of this piece of code? [hint: this example have two loop]

a. O(1)

b. O(logn)

c. O(n)

d. O(n2)

Example #4: Problem: Does array A have duplicate entries? Given arrays A of length n.Algorithm 4


2: for j = i+1 to n do

3: if A[i] == A [j] then

4: Return TRUE

5: Return FALSE

What is the running time of this piece of code? [hint: this example have two loop]

a. O(1)

b. O(logn)

c. O(n)

d. O(n2)

Big-oh Notation


Let T(n) = function of n=1,2,……n

[usually, the worst case running time an algorithm]

Q: when is T(n)=O(F(n))?

Answer : it means eventually, for all sufficiently large values of n, T(n) is bounded above by a

constant multiple of f(n).

Formal Definition : T(n)=O(f(n))

If and only if (iff) there exit two constants c,n0 such that

T(n)<= cf(n)

For all n>=n0

Note : in this formal chose c,n0 can't depend on n .

T(n)=O(f(n))

Big-oh Notation


Example #5: 2n+3= O(n) let f(n)=n

Where, T(n)≤ cf(n), let c=4, then n=1,…n,

2n+3≤ 4*1-----------------False (5<=4)

2n+3 ≤ 4*2----------------True(7<=8 Then n>=2)

Example #6: 10n^2+5n+3= O(n^2) let f(n)=n^2, c=11

Where, T(n)≤ cf(n), then try to use n=1,…n,

10n^2+5n+3 ≤ 11 *n^2, when n=1 --------------- False (18<=11)

………………………. , when n=2 --------------- False

………………………. , when n=3 -------------- False

………………………. , when n=4 --------------- False

………………………. , when n=5 --------------- False

………………………. , when n=6 ---------------True(360+30+3<=11*36)= (393<=396)

Big-Omega Notation


Formal Definition : T(n)= (f(n))

If and only if (iff) there exit two constants c,n0 such that

T(n) ≥ cf(n)

For all n>=n0

Ω

T(n)= (f(n))ΩBig-Theta Notation

Formal Definition : T(n)= Θ(f(n))

If and only if (iff) there exit three positive constants c1,c2,n0 such

that [T(n) =O(F(n)) and T(n)= (f(n))]

c1f(n) ≤ T(n) ≤ c2f(n)

For all n>=n0

Ω

Example #7:


Let 𝑇 𝑛 =1

2𝑛2 + 3𝑛. Which of the following statements are

true? (Check all that apply.)

𝑇 𝑛 = Θ(𝑛2).

𝑇 𝑛 = 𝑂 𝑛 .

𝑇 𝑛 = Ω 𝑛 .

𝑇 𝑛 = 𝑂(𝑛3).

Example #8:


Example #9:

Example #9:


Hint:

we used two functions F(n) and G(n) therefore; we need two constant C1 and C2.

Proof: [max [f.g]=Θ(f(n)+g(n))]

For every n , we have

C1[f(n)+g(n)] ≤ max [f(n).g(n)]≤ c2[f(n)+g(n)]

This meaning:

max [f(n).g(n)] ≤ [f(n)+g(n)]

And

2max [f(n).g(n)] ≥ f(n)+g(n) move 2 to other side

max [f(n).g(n)] ≥ ½ [f(n)+g(n)]

½ [f(n)+g(n)] ≤ max [f(n).g(n)] ≤ [f(n)+g(n)] for all n>=1

This lead to

[max [f.g]=Θ(f(n)+g(n))]

Where,[ n0=1,c1=1/2,c2=1]

Proof the Following :

Fibonacci Numbers


the Fibonacci numbers are the numbers in the following integer sequence, called

the Fibonacci sequence, and characterized by the fact that every number after the first two is

the sum of the two preceding ones Often, especially in modern usage, the sequence is

extended by one more initial term: 0, 1,1,2,3,5,8,13,21,34,55,…….

Time complexity for Fibonacci sequence using number of steps method

Space complexityNote : In this case, weuse four constants ( Fib,fnum1,fnum2,I ) as aconstant variable

Prefix averages for set of numbers


Time complexity for Prefix averages using number of steps method

Fibonacci numbers using recursion


2n,)2n(fibo)1n(fibo

2n,n)n(fibo

Time complexity

Fibonacci numbers using recursion


2n,)2n(fibo)1n(fibo

2n,n)n(fibo

Time complexity

Max number of node in depth K is

Fibonacci numbers # Activation

7 1

6 1

5 2

4 3

3 5

2 8

1 13

Hint: for find Fib(30), we need approximation 500 activations

Documents

LECTURE NOTES OF ALGORITHMS: DESIGN ...Asymptotic Analysis 4 March 2017 Ass. Prof. Dr. Samaher Al_Janabi Notes of Lecture #3 Example #3: Problem: Do arrays A, B have a number in common?