7/29/2019 Module 1(AA)

1/32

Course Instructor : Inayat khan

Department: Shaikh Zayed Islamic Centre

University Of Peshawar

Analysis of Algorithm

Lecture-01: Introduction to analysis of

Algorithm and calculating running time

7/29/2019 Module 1(AA)

2/32

Inayat khan2

Algorithm

AlgorithmInput Output

An algorithm is a step-by-step procedure forsolving a problem in a finite amount of time.

7/29/2019 Module 1(AA)

3/32

Introduction What is Algorithm?

a clearly specified set of simple instructions to be followed to

solve a problem

Takes a set of values, as input and

produces a value, or set of values, as output

May be specified

In English

As a computer program

As a pseudo-code

Data structures Methods of organizing data

Program = algorithms + data structures

Inayat khan3

7/29/2019 Module 1(AA)

4/32

Introduction

Why need algorithm analysis ?

writing a working program is not good enough

The program may be inefficient!

If the program is run on a large data set, then the

running time becomes an issue

Inayat khan4

7/29/2019 Module 1(AA)

5/32

Example: Selection Problem

Given a list of N numbers, determine the kth

largest, where k N.

Algorithm 1:

(1) Read N numbers into an array

(2) Sort the array in decreasing order by some

simple algorithm

(3) Return the element in position kInayat khan5

7/29/2019 Module 1(AA)

6/32

Example: Selection Problem

Algorithm 2:

(1) Read the first k elements into an array and sort

them in decreasing order

(2) Each remaining element is read one by one

If smaller than the kth element, then it is ignored

Otherwise, it is placed in its correct spot in the array,

bumping(hiting) one element out of the array.

(3) The element in the kth position is returned as the

answer.

Inayat khan6

7/29/2019 Module 1(AA)

7/32

Example: Selection Problem

Which algorithm is better when N =100 and k = 100?

N =100 and k = 1?

What happens when N = 1,000,000 and k =

500,000?

There exist better algorithms

Inayat khan7

7/29/2019 Module 1(AA)

8/32

Algorithm Analysis We only analyze correctalgorithms

An algorithm is correct

If, for every input instance, it halts with the correct output

Incorrect algorithms

Might not halt at all on some input instances

Might halt with other than the desired answer

Analyzing an algorithm

Predicting the resources that the algorithm requires

Resources include Memory

Communication bandwidth

Computational time (usually most important)

Inayat khan8

7/29/2019 Module 1(AA)

9/32

Algorithm Analysis

Factors affecting the running time computer : How fast is computer

Compiler : Which algorithm is used

Algorithm used: Which type algorithm is used howefficient is?

input to the algorithm The content of the input affects the running time

typically, the input size(number of items in the input) is themain consideration

E.g. sorting problem the number of items to be sorted

E.g. multiply two matrices together the total number ofelements in the two matrices

Inayat khan9

7/29/2019 Module 1(AA)

10/32

Running Time of an Algorithm

How many time an algorithm takes.

E.g. running time can be calculated in term of---

N time, N2 time LogN etc.

Running time of an algorithm depends on the

number of steps or instruction in the algorithm.

7/29/2019 Module 1(AA)

11/32

Example of Algorithm Running Time

Calculate

Lines 1 and 4 count for one unit each

Line 3: executed N times, each time four units Line 2: (1 for initialization, N+1 for all the tests, N for

all the increments) total 2N + 2

total cost: 6N + 4 O(N)------This is running time of

algorithm

N

i

i1

3

1

2

3

4

1

2N+2

4N

1

Inayat khan11

7/29/2019 Module 1(AA)

12/32

Running Time Cases

There are three main cases of the running time:

1. Worst Case running time

2. Average Case running time

3. Best Case running time

7/29/2019 Module 1(AA)

13/32

Worst- / average- / best-case Worst-case running time of an algorithm

The longest running time forany input of size n

An upper bound on the running time for any input

guarantee that the algorithm will never take longer

Example: Sort a set of numbers in increasing order; and thedata is in decreasing order

The worst case can occur fairly often

E.g. in searching a database for a particular piece of information

Best-case running time

sort a set of numbers in increasing order; and the data isalready in increasing order

Average-case running time

May be difficult to define what average means

Inayat khan13

7/29/2019 Module 1(AA)

14/32

Running-time of algorithms

Bounds are for the algorithms, rather than programs

programs are just implementations of an algorithm,

and almost always the details of the program do not

affect the bounds

Bounds are foralgorithms, rather than problems

A problem can be solved with several algorithms,

some are more efficient than othersInayat khan14

7/29/2019 Module 1(AA)

15/32

Inayat khan15

Pseudocode

High-level descriptionof an algorithm

More structured thanEnglish prose

Less detailed than aprogram

Preferred notation fordescribing algorithms

Hides many programdesign issues

7/29/2019 Module 1(AA)

16/32

Inayat khan16

Pseudocode Details

Control flow

ifthen [else]

whiledo

repeatuntil fordo

Method declaration

Algorithmmethod(arg [,arg])Input

Output

Method callvar.method(arg [,arg])

Return value

returnexpression ExpressionsAssignment

(like in Java)

Equality testing

(like in Java)n2Superscripts and other

mathematical formattingallowed

7/29/2019 Module 1(AA)

17/32

Pseudocode Example

Example: find maxelement of an array

AlgorithmarrayMax(A,n)

InputarrayA ofn integersOutputmaximum element ofA

currentMaxA[0]

fori1ton 1do

ifA[i] currentMaxthencurrentMaxA[i]

returncurrentMax

7/29/2019 Module 1(AA)

18/32

Inayat khan18

Counting Primitive

Operations

By inspecting the pseudocode, we can determine themaximum number of primitive operations executed by analgorithm, as a function of the input size

AlgorithmarrayMax(A,n) # operationscurrentMaxA[0] 2

fori1ton 1do loopn 1 times

ifA[i] currentMaxthen 2 ops (index, comp)

currentMaxA[i] 2 ops (index, asmt)

{ increment counter i } (n 1)

returncurrentMax 1

4n 1

7/29/2019 Module 1(AA)

19/32

Inayat khan19

Where Did That Come From?

Incrementing loop is (n-1) ops, plus a check

each time (n-1) more ops.

Each time through body of loop is either 2ops or 4 ops

36Total14

1)1(4)1(22Total1)1(2)1(22

nn

nnnn

7/29/2019 Module 1(AA)

20/32

Inayat khan20

Summations

The analysis involved computing a summation.

Given a finite sequence of values a1, a2, a3, ,

an. Their sum a1 + a2 + a3+ + an isexpressed in summation notation as

n

i

ia1

7/29/2019 Module 1(AA)

21/32

Inayat khan21

Summations

If c is a constant

and

n

i

i

n

i

i acca11

n

i

i

n

i

i

n

i

ii baba111

)(

7/29/2019 Module 1(AA)

22/32

Inayat khan22

Arithmetic Series

)(2

)1(

...4321

2

1

nnn

ni

n

i

7/29/2019 Module 1(AA)

23/32

Inayat khan23

Quadratic Series

)(6

32

...941

3

23

1

22

n

nnn

nin

i

7/29/2019 Module 1(AA)

24/32

Inayat khan24

Geometric Series

If 0 < x < 1 then this is (1), and

if x > 1, then this is (xn)

)(1

1

...1

2

)1(

2

1

nx

x

xxxx

n

nn

i

i

7/29/2019 Module 1(AA)

25/32

Inayat khan25

Harmonic Series

For n 0

)(ln

ln

1

...3

1

2

1

1

1

1

n

nn

iH

n

i

n

7/29/2019 Module 1(AA)

26/32

Inayat khan26

Calculate running Time of Nested

Loops-1 Example

for i 1 to n

do

for j 1 to 2ido

k = j

while (k 0)

do

k = k -1

7/29/2019 Module 1(AA)

27/32

Inayat khan27

Example: Nested Loops-2

We write out the loops as summations and thensolve the summations.

To convert loops into summations, we work from

inside out.

7/29/2019 Module 1(AA)

28/32

Inayat khan28

Example: Nested Loops-3

for i 1 to n

do

for j 1 to 2ido

k = j

while (k 0)

do

k = k -1

Inner while loop isexecuted for

k=j, j-1, j-2, , 0

Time spent in the innerwhile loop is constant.

Let I() be the time

spent in the while loop.Thus:

j

k

jjI0

11)(

7/29/2019 Module 1(AA)

29/32

Inayat khan29

Example: Nested Loops-4

for i 1 to n

do

for j 1 to 2ido

k = j

while (k 0)

do

k = k -1

Let M() be thetime spent in thefor loop.

i

j

jIiM2

1

)()(

7/29/2019 Module 1(AA)

30/32

Inayat khan30

Example: Nested Loops-5

ii

iii

j

jjIiM

i

j

i

j

i

j

i

j

32

22

)12(2

1

)1()()(

2

2

1

2

1

2

1

2

1

Arithmetic Series

7/29/2019 Module 1(AA)

31/32

Inayat khan31

Example: Nested Loops-6

for i 1 to n

do

for j 1 to 2ido

k = j

while (k 0)

do

k = k -1

The outer most forloop:

Let T() be running

time of the entirealgorithm:

n

i

iMnT1

)()(

7/29/2019 Module 1(AA)

32/32

Inayat khan32

Example: Nested Loops-7

)(

6

11154

2

)1(3

6

322

32

)32()()(

3

23

3

11

2

1

2

1

2

n

nnn

nnnnn

ii

iiiMnT

n

i

n

i

n

i

n

i

6

11154

12

2230812

18184128

12

1818

12

4128

2

33

6

264

2

)1(3

6

322

2

2

2

2

2

2

3

3

23

23

23

3

nnn

nnn

nnnnn

nnnnn

nnnnn

nnnnn

Quadratic Series