1

Algorithms & Pseudocode Orders of Magnitude Sequential Searching

Finding Minimum value Linear Search

Sorting Insertion Sort Selection Sort

CSE 30331CSE 30331Lecture 3 – Algorithms ILecture 3 – Algorithms I

2

Algorithms & Pseudocode Algorithm

Well defined computational process (sequence of steps) that takes some input(s) and produces some output(s)

Why discuss algorithms? Facilitate design of software modules Analyze costs (time and space) of software module in a manner

that is independent of platform

Pseudocode Method of writing algorithms that is independent of

programming language

3

Constant Time Algorithms

An algorithm is Θ(1) when its running time is independent of the number of data items. The algorithm runs in constant time.

The storing of the element involves a simple assignment statement and thus has efficiency Θ(1).

fro nt rear

D irec t In se r t a t R ear

4

Linear Time Algorithms

An algorithm is Θ(n) when its running time is proportional to the size of the list.

When the number of elements doubles, the number of operations doubles.

S equential S e arch fo r the Minim um E lem ent in an A rray

32 46 8 12 3

m in im u m elem en t fou n d in th e list a fter n com p a rison s

n = 51 2 3 4 5

5

Less Efficient Algorithms

Quadratic Algorithms: Θ(n2) practical only for relatively small values of n. When n

doubles, running time quadruples.

Cubic Algorithms: Θ(n3) efficiency is generally poor; doubling the size of n

increases the running time eight-fold.

Exponential: Θ(2n) Awful, but some algorithms have no better algorithmic

solution

6

Algorithm EfficiencyAlgorithm Efficiency

n log2n n log2n n2 n3 2n 2 1 2 4 8 4 4 2 8 16 64 16 8 3 24 64 512 256 16 4 64 256 4096 65536 32 5 160 1024 32768 4294967296 128 7 896 16384 2097152 3.4 x 1038

1024 10 10240 1048576 1073741824 1.8 x 10308

65536 16 1048576 4294967296 2.8 x 1014 Forget it!

7

Logarithmic Time Algorithms

The logarithm of n, base 2, is commonly used when analyzing computer algorithms.

Ex. lg(2) = log2(2) = 1

lg(1024) = log2(1024) = 10

When compared to the functions n and n2, the function lg(n) grows very slowly.

nn 2

lo g 2n

8

A Simple Linear Algorithm

Example: Find the minimum element in an array A Assume A contains n elements (n >= 1) Assume elements are not ordered Basic technique is sequential

min(A,n) cost times smallest = A[0] c1 1 for i = 1 to (n-1) c2 n if A[i] < smallest c3 n-1 smallest = A[i] c4 0..n-1 return smallest c5 1

9

Analysis of min(A,n) min(A,n) cost times smallest = A[0] c1 1 for i = 1 to (n-1) c2 n if A[i] < smallest c3 n-1 smallest = A[i] c4 0..n-1 return smallest c5 1

Best: A[0] is min T(n) = c1 + c2(n) + c3(n-1) + 0 + c5 T(n) = (c2+c3)(n) + (c1+c5-c3) = an + b linear or Θ(n)

Worst: A[n-1] is min and A[i] > A[i+1] for all i=[0,n) T(n) = c1 + c2(n) + c3(n-1) + c4(n-1) + c5 T(n) = (c2+c3+c4)(n) + (c1+c5-c3-c4) = cn + d linear or Θ(n)

10

Linear Search

Example: Find the target element t in an array A Assume A contains n elements (n >= 1) Assume elements are not ordered Basic technique is linear search Return value is index of smallest element in range 0..n, where n

indicates not found

linearSearch(A,n,t) cost times i = 0 c1 1 while (i < n) and (A[i] != t) c2 1..n+1 i = i + 1 c3 0..n return i c4 1

11

Linear Search linearSearch(A,n,t) cost times i = 0 c1 1 while (i < n) and (A[i] != t) c2 1..n+1 i = i + 1 c3 0..n return i c4 1

Worst: t is not in A T(n) = c1 + c2(n+1) + c3(n) + c4 T(n) = (c2+c3)(n) + (c1+c2+c4) = an + b linear or Θ(n)

Average: A[k] is t, where k = n/2 T(n) = c1 + c2(k+1) + c3(k) + c4 T(n) = (c2+c3)(k) + (c1+c2+c4) = cn + d T(n) = ((c2+c3)/2)(n) + (c1+c2+c4) = cn + d linear or Θ(n)

12

Insertion Sort

On each pass the next element is inserted into the portion of the array previously sorted

The sort is “in place”

The array is logically separated into two parts [..sorted..] [..unsorted..]

The index j marks the location of the key and separates sorted from unsorted portions of the array

13

Example of Insertion Sort

[ ] [ 6 2 5 4 3 ] original array [ 6 ] [ 2 5 4 3 ] prior to 1st iteration [ 2 6 ] [ 5 4 3 ] after 1st iteration [ 2 5 6 ] [ 4 3 ] after 2nd iteration [ 2 4 5 6 ] [ 3 ] after 3rd iteration [ 2 3 4 5 6 ] [ ] after last iteration

14

Insertion Sort

Invariant: A[0..j-1] is the same collection of elements as in

the previous iteration BUT they are now sorted On each iteration

A[j] is the element that is inserted into A[0..j-1] producing a sorted A[0..j]

A[j+1..n-1] remains unsorted

15

Insertion Sort Algorithm Sort n elements of A into non-decreasing order

insertionSort(A,n) cost times for j = 1 to (n-1) c1 n key = A[j] c2 n-1 // insert A[j] into sorted sequence A[0..j-1] i = j – 1 c3 n-1 while (i >= 0) and (A[i] > key) c4 f A[i+1] = A[i] c5 g i = i – 1 c6 g A[i+1] = key c7 n-1

f = g =

1

1

n

j

jt

1

1

)1(n

j

jt

16

Insertion Sort Analysis Best: already presorted

tj is number of times the while loop checks limits and pairs of elements during shifting of elements in A[0..j-1]

The loop terminates on first check since A[j] <= A[j+1] for all j T(n) = c1(n) + (c2+c3+c7)(n-1) + c4(f) + (c5+c6)(g)

tj = 1, so f = = (n-1) and g = = 0

T(n) = (c1+c2+c3+c4+c7)n – (c2+c3+c4+c7)

Linear Θ(n)

1

1

n

j

jt

1

1

)1(n

j

jt

17

Insertion Sort Analysis

Worst: presorted in reverse order Shifting loop has to compare and shift every element in A[0..j-1]

because initial order ensures A[k] > A[j] for all k < j

T(n) = c1(n) + (c2+c3+c7)(n-1) + c4(f) + (c5+c6)(g)

tj = j, so f = = n(n-1)/2 and g = = (n-1)(n-2)/2

T(n) = c1(n) + (c2+c3+c7)(n-1) + c4(n(n-1)/2 ) + (c5+c6)((n-1)(n-2)/2) T(n) = an2 + bn + c

Quadratic Θ(n2)

1

1

n

j

jt

1

1

)1(n

j

jt

18

Selection Sort On each pass the smallest element in the unsorted

sub-array is selected for appending to the sorted sub-array

The sort is “in place”

The array is logically separated into two parts [..sorted..] [..unsorted..]

The index k separates sorted from unsorted portions of the array

19

Example of Selection Sort

[ ] [ 6 2 5 4 3 ] original array [ 2 ] [ 6 5 4 3 ] after 1st iteration [ 2 3 ] [ 5 4 6 ] after 2nd iteration [ 2 3 4 ] [ 5 6 ] after 3rd iteration [ 2 3 4 5 ] [ 6 ] after last iteration

20

Selection Sort

Invariant: A[0..k-1] is sorted and all elements in A[0..k-1] are

less than or equal to all elements in A[k..n-1] On each iteration

A[smallIndex] is the element that is appended to A[0..k-1] producing a sorted A[0..k]

A[k+1..n-1] remains unsorted

21

Selection Sort Algorithm

Sort n elements of A into non-decreasing order

selectionSort(A,n) cost times for k = 0 to n-2 c1 n // scan unsorted sublist to find smallest value smallIndex = k c2 n-1 for j = k+1 to n-1 c3 f if A[j] < A[smallIndex] c4 g smallIndex = j c5 h // swap smallest value with leftmost if smallIndex != k c6 n-1 swap(A[k],A[smallIndex]) c7 0..n-1 f = g = h =

1

1

n

k

kt

1

1

)1(n

k

k

1

1

n

k

k

22

Selection Sort Analysis Best: already presorted

tk is 0, since the leftmost element of the unsorted sub-array is always the smallest element

No swaps are performed, since the array is already in order T(n) = c1(n) + (c2+c6)(n-1) + c3(f) + c4(g)

f = = n(n-1)/2 and g = = (n-1)(n-2)/2

T(n) = (c1+c2+c6)n – c6 + c3(n(n-1)/2) + c4((n-1)(n-2)/2) T(n) = an2 + bn + c

Quadratic Θ(n2)

1

1

n

k

k

1

1

)1(n

k

k

23

Selection Sort Analysis

Worst: presorted in reverse order Searching for smallest requires changing smallIndex on every

comparison in A[0..j-1] because initially A[k] > A[j] for all k < j A swap occurs n-1 times

T(n) = c1(n) + (c2+c6+c7)(n-1) + c3(f) + c4(g) + c5(h)

tk = k, so f = = n(n-1)/2 and g = h = = (n-1)(n-2)/2

T(n) = c1(n) + (c2+c6+c7)(n-1) + c3(n(n-1)/2) + (c4+c5)((n-1)(n-2)/2 ) T(n) = an2 + bn + c

Quadratic Θ(n2)

1

1

n

j

k

1

1

)1(n

j

k

24

Binary Search Algorithm

Case 1: A match occurs. The search is complete and mid is the index that locates the target.

if (midValue == target) // found match return mid;

m idfirs t

target

C as e 1: target = m id valueS earc h is d o ne

las t-1 las t

25

Binary Search Algorithm

Case 2: The value of target is less than midvalue and the search must continue in the lower sublist.

if (target < midvalue) // search lower sublist

<reposition last to mid>

<search sublist arr[first]…arr[mid-1]

las t-1firs t

target

C as e 2: target < m id valueS earc h lo w er s ub lis t

m id -1 las t

26

Binary Search Algorithm

Case 3: The value of target is greater than midvalue and the search must continue in the upper sublist .

if (target > midvalue)// search upper sublist

<reposition first to mid+1>

<search sublist arr[mid+1]…arr[last-1]>

C as e 3: target > m id valueS earc h up p er s ub lis t

las t-1new firs t = m id + 1

firs t

target

las t

27

Successful Binary Search

Search for target = 23

Step 1: Indices first = 0, last = 9, mid = (0+9)/2 = 4.

Since target = 23 > midvalue = 12, step 2 searches the upper sublist with first = 5 and last = 9.

m id

-7 3 5 8 12 16arr

0 1 2 3 4 5

23 33 55

6 7 8 9

28

Successful Binary Search

Step 2:

Indices first = 5, last = 9, mid = (5+9)/2 = 7.

Since target = 23 < midvalue = 33, step 3 searches the lower sublist with first = 5 and last = 7.

m id

-7 3 5 8 12 16arr

0 1 2 3 4 5

23 33 55

6 7 8 9

29

Successful Binary Search

Step 3:

Indices first = 5, last = 7, mid = (5+7)/2 = 6.

Since target = midvalue = 23, a match is found at index mid = 6.

m id

-7 3 5 8 12 16arr0 1 2 3 4 5

23 33 55

6 7 8 9

30

Unsuccessful Binary Search

Search for target = 4.

Step 1: Indices first = 0, last = 9, mid = (0+9)/2 = 4.

m id

-7 3 5 8 12 16arr

0 1 2 3 4 5

23 33 55

6 7 8 9

Since target = 4 < midvalue = 12, step 2 searches the lower sublist with first = 0 and last = 4.

31

Unsuccessful Binary Search

Step 2:

Indices first = 0, last = 4, mid = (0+4)/2 = 2.

Since target = 4 < midvalue = 5, step 3 searches the lower sublist with first = 0 and last 2.

m id

-7 3 5 8 12 16arr0 1 2 3 4 5

23 33 556 7 8

32

Unsuccessful Binary Search

Step 3: Indices first = 0, last = 2, mid = (0+2)/2 = 1.

Since target = 4 > midvalue = 3, step 4 should search the upper sublist with first = 2 and last =2. However, since first >= last, the target is not in the list and we return index last = 9.

m id

-7 3 5 8 12 16arr

0 1 2 3 4 5

23 33 55

6 7 8 9

33

Binary Search Implementationint binSearch (const int arr[], int first, int last,

int target)

{

int origLast = last; // original value of last

while (first < last) {

int mid = (first+last)/2;

if (target == arr[mid])

return mid; // a match so return mid

else if (target < arr[mid])

last = mid; // search lower sublist

else

first = mid+1; // search upper sublist

}

return origLast; // target not found

}

34

Binary Search Analysis

Each comparison divides the problem size in half

Assume for simplicity that n = 2k

So, worst case … How many times do we divide n in half before

there are no more values to check? T(n) = 1+ k = 1 + lg n Logarithmic Θ(lg n)