DSA Book.pdf

7/29/2019 DSA Book.pdf

1/367

Data Structures and Algorithms

Jrg Endrullis

Vrije Universiteit AmsterdamDepartment of Computer Science

Section Theoretical Computer Science

20092010

http://www.few.vu.nl/~tcs/ds/

7/29/2019 DSA Book.pdf

2/367

The Lecture

Lectures:

Tuesday 11:00 12:45(weeks 3642 in WN-KC159)

Wednesday 9:00 10:45(weeks 3641 in WN-Q105, 42 in WN-Q112)

Exercise classes:

Thursday 11:00 12:45 (weeks 3642 in WN-M607(50))

Thursday 13:30 15:15 (weeks 3642 in WN-M623(50))

Thursday 15:30 17:15 (weeks 3642 in WN-C121)

Exam on 19.10.2009 at 8:45 11:30.

Retake exam on 12.01.2010 at 18:30 21:15.

7/29/2019 DSA Book.pdf

3/367

Important Information: Voortentamen

Voortentamen (pre-exam): Tuesday 29 september, 11:00 12:45 in WN-KC159

The participation is not obligatory, but recommended.

The result can influence the final grade only positively:

Let V be the pre-exam grade, and T the exam grade.The final grade is calculated by:

max(T,2T + V

3)

Thus if the final exam grade T is higher than the pre-examgrade V, then only the final grade T counts. But if the pre-examV is higher then it counts with 13 .

7/29/2019 DSA Book.pdf

4/367

Contact Persons

lecture:

Jrg [email protected]

exercise classes:

Dennis [email protected]

Michel [email protected]

Atze van der [email protected]

7/29/2019 DSA Book.pdf

5/367

Literature

7/29/2019 DSA Book.pdf

6/367

Introduction

Definition

An algorithm is a list of instructions for completing a task.

Algorithms are central to computer science. Important aspects when designing algorithms:

correctness

termination

efficiency

7/29/2019 DSA Book.pdf

7/367

Evaluation of Algorithms

Algorithms with equal functionality may have huge

differences in efficiency (complexity)

Important measures:

time complexity

space (memory) complexity

Time and memory usage increase with size of the input:

input size

time

best case

average case

worst case

10 20 30 40

2s

4s

6s

8s

10s

7/29/2019 DSA Book.pdf

8/367

Time Complexity

Running time of programs depends on various factors: input for the program

compiler

speed of the computer (CPU, memory)

time complexity of the used algorithm

Average case often difficult to determine.

We focus on worst case running time:

easier to analyse

usually the crucial measure for applications

7/29/2019 DSA Book.pdf

9/367

Methods for Analysing Time Complexity

Experiments (measurements on a certain machine):

requires implementation

comparisons require equal hardware and software

experiments may miss out imporant inputs

Calculation of complexity for idealised computer model:

requires exact counting

computer model: Random Access Machine (RAM)

Asymptotic complexity estimation depending on input size n:

allows for approximations

logarithmic (log2 n), linear (n), . . . , exponential (an), . . .

7/29/2019 DSA Book.pdf

10/367

Random Access Machine (RAM)

A Random Access Machine is a CPU connected to a memory:

potentially unbounded number of memory cells

each memory cell can store an arbitrary number

CPU Memory

primitive operations are executed in constant time

memory cells can be accessed with one primitive operation

7/29/2019 DSA Book.pdf

11/367

Pseudo-code

We use Pseudo-code to describe algorithms:

programming-like, hight-level description

independent from specific programming language

primitive operations:

assigning a value to a variable

calling a method

arithmetic operations, comparing two numbers

array access

returning from a method

7/29/2019 DSA Book.pdf

12/367

Counting Primitive Operations: arrayMax

We analyse the worst case complexity of arrayMax.

Algorithm arrayMax(A, n):Input: An array storing n 1 integers.Output: The maximum element in A.

currentMax = A[0]for i = 1 to n 1 do

if currentMax < A[i] then

currentMax = A[i]

done

return currentMax

1 + 1

1 + n+ (n 1)1 + 1

1 + 11 + 1

1

Hence we have a worst case time complexity:

T(n) = 2 + 1 + n+ (n 1) 6 + 1

= 7 n 2

7/29/2019 DSA Book.pdf

13/367

Counting Primitive Operations: arrayMax

We analyse the best case complexity of arrayMax.

Algorithm arrayMax(A, n):Input: An array storing n 1 integers.Output: The maximum element in A.

currentMax = A[0]for i = 1 to n 1 do

if currentMax < A[i] then

currentMax = A[i]

done

return currentMax

1 + 1

1 + n+ (n 1)1 + 1

01 + 1

1

Hence we have a best case time complexity:

T(n) = 2 + 1 + n+ (n 1) 4 + 1

= 5 n

7/29/2019 DSA Book.pdf

14/367

Important Growth Functions

Examples of growth functions, ordered by speed of growth: log n

n

n log n

n2

n3

an

logarithmic growth

linear growth

(n log n)-growth

quadratic growth

cubic growth

exponential growth (a> 1)

We recall the definition of logarithm:

logab = c such that ac = b

S d f G h Pi i l

7/29/2019 DSA Book.pdf

15/367

Speed of Growth, Pictorial

x

f(x)

f(x) = log x

f(x) = x

f(x) = x log x

f(x) = x2

f(x) = ex

S d f G th

7/29/2019 DSA Book.pdf

16/367

Speed of Growth

n 10 100 1000 104 105 106

log2 n 3 7 10 13 17 20

n 10 100 1000 104 105 106

n log n 30 700 13000 105

106

107

n2 100 10000 16 108 1010 1012

n3 1000 16 19 1012 1015 1018

2n 1024 1030 10300 103000 1030000 10300000

Note that log2 ngrowth very slow, whereas 2n growth explosive!

Ti d di P bl Si

7/29/2019 DSA Book.pdf

17/367

Time depending on Problem Size

Computation time assuming that 1 step takes 1s(0.000001s).

n 10 100 1000 104 105 106

log n < < < < < 0.00002s

n < < 0.001s 0.01s 0.1s 1s

n log n < < 0.013s 0.1s 1s 10sn2 < 0.01s 1s 100s 3h 1000h

n3 0.001s 1s 1000s 1000h 100y 105y

2n 0.001s 1023y > > > >

Here < means fast (< 0.001s), and > more than 10300 years.

A problem for which there exist only exponential algorithms areusually considered intractable.

S h i A

7/29/2019 DSA Book.pdf

18/367

Search in Arrays

We analyse the worst case complexity of search.

Algorithm search(A, n, x):Input: An array storing n 1 integers.Output: Returns true if A contains x and false, otherwise.

for i = 0 to n 1 do

if A[i] == x then

return true

done

return false

1 + (n+ 1) + n1 + 1

(not taken for worst case)1 + 1

1

Hence we have a worst case time complexity:

T(n) = 1 + (n+ 1) + n 4 + 1

= 5 n+ 3

Search in Arrays

7/29/2019 DSA Book.pdf

19/367

Search in Arrays

We analyse the best case complexity of search.

Algorithm search(A, n, x):Input: An array storing n 1 integers.Output: Returns true if A contains x and false, otherwise.

for i = 0 to n 1 doif A[i] == x then

return truedone

return false

1 + 11 + 1

1

Hence we have a best case time complexity:

T(n) = 5

Search in Sorted Arrays: Binary Search

7/29/2019 DSA Book.pdf

20/367

Search in Sorted Arrays: Binary Search

Algorithm binSearch(A, n, x):Input: An array storing n integers in ascending order.

Output: Returns true if A contains x and false, otherwise.

low = 0high = n 1while low high do

mid = (low + high)/2y = A[mid]

if x < y then high = mid 1if x == y then return true

if x > y then low = mid+ 1

done

return false

1 2 2 4 6 7 8 11 13 15 16

low highmid

x = 8y = 7

Search in Sorted Arrays: Binary Search

7/29/2019 DSA Book.pdf

21/367

Search in Sorted Arrays: Binary Search

Algorithm binSearch(A, n, x):Input: An array storing n integers in ascending order.

Output: Returns true if A contains x and false, otherwise.

low = 0high = n 1while low high do

mid = (low + high)/2y = A[mid]

if x < y then high = mid 1if x == y then return true

if x > y then low = mid+ 1

done

return false

1 2 2 4 6 7 8 11 13 15 16

highlow mid

x = 8y = 13

Search in Sorted Arrays: Binary Search

7/29/2019 DSA Book.pdf

22/367

Search in Sorted Arrays: Binary Search

Algorithm binSearch(A, n, x):Input: An array storing n integers in ascending order.

Output: Returns true if A contains x and false, otherwise.

low = 0high = n 1while low high do

mid = (low + high)/2y = A[mid]

if x < y then high = mid 1if x == y then return true

if x > y then low = mid+ 1

donereturn false

1 2 2 4 6 7 8 11 13 15 16

low highmid

x = 8y = 8

Search in Sorted Arrays: Binary Search

7/29/2019 DSA Book.pdf

23/367