DSA Book.pdf

  • View
    252

  • Download
    1

Embed Size (px)

Text of DSA Book.pdf

  • 7/29/2019 DSA Book.pdf

    1/367

    Data Structures and Algorithms

    Jrg Endrullis

    Vrije Universiteit AmsterdamDepartment of Computer Science

    Section Theoretical Computer Science

    20092010

    http://www.few.vu.nl/~tcs/ds/

    http://www.few.vu.nl/~tcs/ds/http://www.few.vu.nl/~tcs/ds/
  • 7/29/2019 DSA Book.pdf

    2/367

    The Lecture

    Lectures:

    Tuesday 11:00 12:45(weeks 3642 in WN-KC159)

    Wednesday 9:00 10:45(weeks 3641 in WN-Q105, 42 in WN-Q112)

    Exercise classes:

    Thursday 11:00 12:45 (weeks 3642 in WN-M607(50))

    Thursday 13:30 15:15 (weeks 3642 in WN-M623(50))

    Thursday 15:30 17:15 (weeks 3642 in WN-C121)

    Exam on 19.10.2009 at 8:45 11:30.

    Retake exam on 12.01.2010 at 18:30 21:15.

  • 7/29/2019 DSA Book.pdf

    3/367

    Important Information: Voortentamen

    Voortentamen (pre-exam): Tuesday 29 september, 11:00 12:45 in WN-KC159

    The participation is not obligatory, but recommended.

    The result can influence the final grade only positively:

    Let V be the pre-exam grade, and T the exam grade.The final grade is calculated by:

    max(T,2T + V

    3)

    Thus if the final exam grade T is higher than the pre-examgrade V, then only the final grade T counts. But if the pre-examV is higher then it counts with 13 .

  • 7/29/2019 DSA Book.pdf

    4/367

    Contact Persons

    lecture:

    Jrg [email protected]

    exercise classes:

    Dennis [email protected]

    Michel [email protected]

    Atze van der [email protected]

  • 7/29/2019 DSA Book.pdf

    5/367

    Literature

  • 7/29/2019 DSA Book.pdf

    6/367

    Introduction

    Definition

    An algorithm is a list of instructions for completing a task.

    Algorithms are central to computer science. Important aspects when designing algorithms:

    correctness

    termination

    efficiency

  • 7/29/2019 DSA Book.pdf

    7/367

    Evaluation of Algorithms

    Algorithms with equal functionality may have huge

    differences in efficiency (complexity)

    Important measures:

    time complexity

    space (memory) complexity

    Time and memory usage increase with size of the input:

    input size

    time

    best case

    average case

    worst case

    10 20 30 40

    2s

    4s

    6s

    8s

    10s

  • 7/29/2019 DSA Book.pdf

    8/367

    Time Complexity

    Running time of programs depends on various factors: input for the program

    compiler

    speed of the computer (CPU, memory)

    time complexity of the used algorithm

    Average case often difficult to determine.

    We focus on worst case running time:

    easier to analyse

    usually the crucial measure for applications

  • 7/29/2019 DSA Book.pdf

    9/367

    Methods for Analysing Time Complexity

    Experiments (measurements on a certain machine):

    requires implementation

    comparisons require equal hardware and software

    experiments may miss out imporant inputs

    Calculation of complexity for idealised computer model:

    requires exact counting

    computer model: Random Access Machine (RAM)

    Asymptotic complexity estimation depending on input size n:

    allows for approximations

    logarithmic (log2 n), linear (n), . . . , exponential (an), . . .

  • 7/29/2019 DSA Book.pdf

    10/367

    Random Access Machine (RAM)

    A Random Access Machine is a CPU connected to a memory:

    potentially unbounded number of memory cells

    each memory cell can store an arbitrary number

    CPU Memory

    primitive operations are executed in constant time

    memory cells can be accessed with one primitive operation

  • 7/29/2019 DSA Book.pdf

    11/367

    Pseudo-code

    We use Pseudo-code to describe algorithms:

    programming-like, hight-level description

    independent from specific programming language

    primitive operations:

    assigning a value to a variable

    calling a method

    arithmetic operations, comparing two numbers

    array access

    returning from a method

  • 7/29/2019 DSA Book.pdf

    12/367

    Counting Primitive Operations: arrayMax

    We analyse the worst case complexity of arrayMax.

    Algorithm arrayMax(A, n):Input: An array storing n 1 integers.Output: The maximum element in A.

    currentMax = A[0]for i = 1 to n 1 do

    if currentMax < A[i] then

    currentMax = A[i]

    done

    return currentMax

    1 + 1

    1 + n+ (n 1)1 + 1

    1 + 11 + 1

    1

    Hence we have a worst case time complexity:

    T(n) = 2 + 1 + n+ (n 1) 6 + 1

    = 7 n 2

  • 7/29/2019 DSA Book.pdf

    13/367

    Counting Primitive Operations: arrayMax

    We analyse the best case complexity of arrayMax.

    Algorithm arrayMax(A, n):Input: An array storing n 1 integers.Output: The maximum element in A.

    currentMax = A[0]for i = 1 to n 1 do

    if currentMax < A[i] then

    currentMax = A[i]

    done

    return currentMax

    1 + 1

    1 + n+ (n 1)1 + 1

    01 + 1

    1

    Hence we have a best case time complexity:

    T(n) = 2 + 1 + n+ (n 1) 4 + 1

    = 5 n

  • 7/29/2019 DSA Book.pdf

    14/367

    Important Growth Functions

    Examples of growth functions, ordered by speed of growth: log n

    n

    n log n

    n2

    n3

    an

    logarithmic growth

    linear growth

    (n log n)-growth

    quadratic growth

    cubic growth

    exponential growth (a> 1)

    We recall the definition of logarithm:

    logab = c such that ac = b

    S d f G h Pi i l

  • 7/29/2019 DSA Book.pdf

    15/367

    Speed of Growth, Pictorial

    x

    f(x)

    f(x) = log x

    f(x) = x

    f(x) = x log x

    f(x) = x2

    f(x) = ex

    S d f G th

  • 7/29/2019 DSA Book.pdf

    16/367

    Speed of Growth

    n 10 100 1000 104 105 106

    log2 n 3 7 10 13 17 20

    n 10 100 1000 104 105 106

    n log n 30 700 13000 105

    106

    107

    n2 100 10000 16 108 1010 1012

    n3 1000 16 19 1012 1015 1018

    2n 1024 1030 10300 103000 1030000 10300000

    Note that log2 ngrowth very slow, whereas 2n growth explosive!

    Ti d di P bl Si

  • 7/29/2019 DSA Book.pdf

    17/367

    Time depending on Problem Size

    Computation time assuming that 1 step takes 1s(0.000001s).

    n 10 100 1000 104 105 106

    log n < < < < < 0.00002s

    n < < 0.001s 0.01s 0.1s 1s

    n log n < < 0.013s 0.1s 1s 10sn2 < 0.01s 1s 100s 3h 1000h

    n3 0.001s 1s 1000s 1000h 100y 105y

    2n 0.001s 1023y > > > >

    Here < means fast (< 0.001s), and > more than 10300 years.

    A problem for which there exist only exponential algorithms areusually considered intractable.

    S h i A

  • 7/29/2019 DSA Book.pdf

    18/367

    Search in Arrays

    We analyse the worst case complexity of search.

    Algorithm search(A, n, x):Input: An array storing n 1 integers.Output: Returns true if A contains x and false, otherwise.

    for i = 0 to n 1 do

    if A[i] == x then

    return true

    done

    return false

    1 + (n+ 1) + n1 + 1

    (not taken for worst case)1 + 1

    1

    Hence we have a worst case time complexity:

    T(n) = 1 + (n+ 1) + n 4 + 1

    = 5 n+ 3

    Search in Arrays

  • 7/29/2019 DSA Book.pdf

    19/367

    Search in Arrays

    We analyse the best case complexity of search.

    Algorithm search(A, n, x):Input: An array storing n 1 integers.Output: Returns true if A contains x and false, otherwise.

    for i = 0 to n 1 doif A[i] == x then

    return truedone

    return false

    1 + 11 + 1

    1

    Hence we have a best case time complexity:

    T(n) = 5

    Search in Sorted Arrays: Binary Search

  • 7/29/2019 DSA Book.pdf

    20/367

    Search in Sorted Arrays: Binary Search

    Algorithm binSearch(A, n, x):Input: An array storing n integers in ascending order.

    Output: Returns true if A contains x and false, otherwise.

    low = 0high = n 1while low high do

    mid = (low + high)/2y = A[mid]

    if x < y then high = mid 1if x == y then return true

    if x > y then low = mid+ 1

    done

    return false

    1 2 2 4 6 7 8 11 13 15 16

    low highmid

    x = 8y = 7

    Search in Sorted Arrays: Binary Search

  • 7/29/2019 DSA Book.pdf

    21/367

    Search in Sorted Arrays: Binary Search

    Algorithm binSearch(A, n, x):Input: An array storing n integers in ascending order.

    Output: Returns true if A contains x and false, otherwise.

    low = 0high = n 1while low high do

    mid = (low + high)/2y = A[mid]

    if x < y then high = mid 1if x == y then return true

    if x > y then low = mid+ 1

    done

    return false

    1 2 2 4 6 7 8 11 13 15 16

    highlow mid

    x = 8y = 13

    Search in Sorted Arrays: Binary Search

  • 7/29/2019 DSA Book.pdf

    22/367

    Search in Sorted Arrays: Binary Search

    Algorithm binSearch(A, n, x):Input: An array storing n integers in ascending order.

    Output: Returns true if A contains x and false, otherwise.

    low = 0high = n 1while low high do

    mid = (low + high)/2y = A[mid]

    if x < y then high = mid 1if x == y then return true

    if x > y then low = mid+ 1

    donereturn false

    1 2 2 4 6 7 8 11 13 15 16

    low highmid

    x = 8y = 8

    Search in Sorted Arrays: Binary Search

  • 7/29/2019 DSA Book.pdf

    23/367