1

Data Structures and Algorithms

Sorting I

Gal A. Kaminka

Computer Science Department

2

What we’ve done so far

We’ve talked about complexity O(), run-time and space requirements

We’ve talked about ADTs Implementations for:

Stacks (2 implementations) Queues (2 implementations) Sets (4 implementations)

3

Sorting

Take a set of items, order unknown Set: Linked list, array, file on disk, …

Return ordered set of the items

For instance: Sorting names alphabetically Sorting by height Sorting by color

4

Sorting Algorithms

Issues of interest: Running time in worst case, other cases Space requirements

In-place algorithms: require constant space The importance of empirical testing

Often Critical to Optimize Sorting

5

Short Example: Bubble Sort

Key: “large unsorted elements bubble up”

Make several sequential passes over the set Every pass, fix local pairs that are not in order

Considered inefficient, but useful as first example

6

(Naïve)BubbleSort(array A, length n)

1. for in to 2 // note: going down

2. for j2 to i // loop does swaps in [1..i]

3. if A[j-1]>A[j]

4. swap(A[j-1],A[j])

7

Pass 1: 25 57 48 37 12 92 86 33

25 48 57 37 12 92 86 33

25 48 37 57 12 92 86 33

25 48 37 12 57 92 86 33

25 48 37 12 57 86 92 33

25 48 37 12 57 86 33 92

8

Pass 2: 25 48 37 12 57 86 33 92

25 37 48 12 57 86 33 92

25 37 12 48 57 86 33 92

25 37 12 48 57 33 86 92

Pass 3: 25 37 12 48 57 33 86 92

25 12 37 48 57 33 86 92

25 12 37 48 33 57 86 92

9

Pass 4: 25 12 37 48 33 57 86 92

12 25 37 48 33 57 86 92

12 25 37 33 48 57 86 92

Pass 5: 12 25 37 33 48 57 86 92

12 25 33 37 48 57 86 92

Pass 6: 12 25 33 37 48 57 86 92

Pass 7: 12 25 33 37 48 57 86 92

10

Bubble Sort Features

Worst case: Inverse sorting Passes: n-1 Comparisons each pass: (n-k) where k pass number Total number of comparisons:

(n-1)+(n-2)+(n-3)+…+1 = n2/2-n/2 = O(n2) In-place: No auxilary storage Best case: already sorted

O(n2) Still: Many redundant passes with no swaps

11

BubbleSort(array A, length n)

1. in

2. quitfalse

3. while (i>1 AND NOT quit) // note: going down

4. quittrue

5. for j=2 to i // loop does swaps in [1..i]

6. if A[j-1]>A[j]

7. swap(A[j-1],A[j]) // put max in I

8. quitfalse

9. ii-1

12

Bubble Sort Features

Best case: Already sorted O(n) – one pass over set, verifying sorting

Total number of exchanges Best case: None Worst case: O(n2)

Lots of exchanges:

A problem with large items

13

Selection Sort

Observation: Bubble-Sort uses lots of exchanges These always float largest unsorted element up

We can save exchanges: Move largest item up only after it is identified More passes, but less total operations

Same number of comparisons Many fewer exchanges

14

SelectSort(array A, length n)

1. for in to 2 // note we are going down

2. largest A[1]

3. largest_index 1

4. for j1 to i // loop finds max in [1..i]

5. if A[j]>A[largest_index]

6. largest_index j

7. swap(A[i],A[largest_index]) // put max in i

15

Initial: 25 57 48 37 12 92 86 33

Pass 1: 25 57 48 37 12 33 86 | 92

Pass 2: 25 57 48 37 12 33 I 86 92

Pass 3: 25 33 48 37 12 I 57 86 92

Pass 4: 25 33 12 37 I 48 57 86 92

Pass 5: 25 33 12 I 37 48 57 86 92

Pass 6: 25 12 I 33 37 48 57 86 92

Pass 7: 12 I 25 33 37 48 57 86 92

16

Selection Sort Summary Best case: Already sorted

Passes: n-1 Comparisons each pass: (n-k) where k pass number # of comparisons: (n-1)+(n-2)+…+1 = O(n2)

Worst case: Same. In-place: No external storage Very few exchanges:

Always n-1 (better than Bubble Sort)

17

Selection Sort vs. Bubble Sort

Selection sort: more comparisons than bubble sort in best case O(n2) But fewer exchanges O(n) Good for small sets/cheap comparisons, large items

Bubble sort: Many exchanges O(n2) in worst case O(n) on sorted input

18

Insertion Sort

Improve on # of comparisons Key idea: Keep part of array always sorted

As in selection sort, put items in final place As in bubble sort, “bubble” them into place

19

InsertSort(array A, length n)

1. for i2 to n // A[1] is sorted

2. y=A[i]

3. j i-1

4. while (j>0 AND y<A[j])

5. A[j+1] A[j] // shift things up

6. jj-1

7. A[j+1] y // put A[i] in right place

20

Initial: 25 57 48 37 12 92 86 33

Pass 1: 25 | 57 48 37 12 92 86 33

Pass 2: 25 57 I 48 37 12 92 86 33

25 48 | 57 37 12 92 86 33

Pass 3: 25 48 57 | 37 12 92 86 33

25 48 57 | 57 12 92 86 33

25 48 48 | 57 12 92 86 33

25 37 48 | 57 12 92 86 33

21

Pass 4: 25 37 48 57 | 12 92 86 33

25 37 48 57 | 57 92 86 33

25 37 48 48 | 57 92 86 33

25 37 37 48 | 57 92 86 33

25 25 37 48 | 57 92 86 33

12 25 37 48 | 57 92 86 33

22

Pass 5: 12 25 37 48 57 | 92 86 33

Pass 6: 12 25 37 48 57 92 | 86 33

12 25 37 48 57 86 | 92 33

Pass 7: 12 25 37 48 57 86 92 | 33

12 25 37 48 57 86 92 | 92

12 25 37 48 57 86 86 | 92

12 25 37 48 57 57 86 | 92

12 25 37 48 48 57 86 | 92

23

Pass 7: 12 25 37 48 48 57 86 | 92

12 25 37 37 48 57 86 | 92

12 25 33 37 48 57 86 | 92

24

Insertion Sort Summary

Best case: Already sorted O(n) Worst case: O(n2) comparisons

# of exchanges: O(n2) In-place: No external storage In practice, best for small sets (<30 items)

BubbleSort does more comparisons! Very efficient on nearly-sorted inputs

25

Divide-and-Conquer

An algorithm design technique: Divide a problem of size N into sub-problems Solve all sub-problems Merge/Combine the sub-solutions

This can result in VERY substantial improvements

26

Small Example: f(x)

1. if x = 0 OR x = 1

2. return 1

3. else

4. return f(x-1) + f(x-2)

What is this function?

27

Small Example: f(x)

1. if x = 0 OR x = 1

2. return 1

3. else

4. return f(x-1) + f(x-2)

What is this function?

Fibbonacci!

28

Divide-and-Conquer in Sorting

Mergesort O(n log n) always, but O(n) storage

Quick sort O(n log n) average, O(n^2) worst Good in practice (>30), O(log n) storage