Sorting Algorithms and Average Case Time Complexity

• Simple Sorts O(n2)– Insertion Sort– Selection Sort– Bubble Sort

• More Complex Sorts O(nlgn)– Heap Sort– Quick Sort– Merge Sort

Heap Sort

• Remember the heap data structure . . .• Binary heap = – a binary tree that is – Complete– satisfies the heap property

Heap Sort (cont’d)

• Given an array of integers:– First, use MakeHeap to convert the array into a

max-heap. – Then, repeat the following steps until there are no

more unsorted elements:• Take the root (maximum) element off the heap by

swapping it into its correct place in the array at the end of the unsorted elements• Heapify the remaining unsorted elements (This puts

the next-largest element into the root position)

Heap Sort (cont’d)

HeapSort(A, n)MakeHeap(A)for i = n downto 2

exchange A[1] with A[i]n = n-1Heapify(A,1)

Heapsort Example

• 2 8 6 1 10 15 12 11

Heap Sort (cont’d)

• Time complexity?– First, MakeHeap(A)• Easy analysis: O(nlgn) since Heapify is O(lgn)• More exact analysis: O(n)

– Then:• n-1 swaps = O(n) • n-1 calls to Heapify = O(nlgn)

– O(n) + O(n) + O(nlgn) = O(nlgn)

Other Sorting Ideas

• Divide and Conquer!

Quicksort

• Recursive in nature. • First Partition the array, and recursively

quicksort the subarray separately until the whole array is sorted

• This process of partitioning is carried down until there are only one-cell arrays that do not need to be sorted at all

Quicksort Algorithm

• Given an array of integers:If array only contains one element, returnElse

Pick one element to use as the pivotPartition elements into 2 subarrays:

Elements less than or equal to the pivotElements greater than or equal to the pivot

Quicksort the two subarrays

Quicksort Example

Quicksort Example

• There are a number of ways to pick the pivot element. Here, we will use the first element in the array

Quicksort Partition

• Given a pivot, partition the elements of the array such that the resulting array consists of:– One sub-array that contains elements <= pivot– Another sub-array that contains elements >= pivot

• The sub-arrays are stored in the original array• Partitioning loops through, swapping elements

below/above pivot

Quicksort Example: Partition Result:

QuickSorttemplate<class T>void quicksort(T data[], int first, int last) { //partition such that left <= pivot, right >= pivot int lower = first+1, upper = last; swap(data[first],data[(first+last)/2]); T pivot = data[first]; while (lower <= upper) { //find the position of the first > pivot element on the left while (data[lower] < pivot) lower++; //find the position of the first < pivot element on the right while (pivot < data[upper]) upper--; if (lower < upper) swap(data[lower++],data[upper--]); else lower++; } swap(data[upper],data[first]); if (first < upper-1) quicksort (data,first,upper-1); //quicksort left if (upper+1 < last) quicksort (data,upper+1,last); //quicksort right}

Quicksort - Recursion

• Recursion: Quicksort sub-arrays

Quicksort: Worst Case Complexity

• Worst case running time?

• Assume first element is chosen as pivot.

• Assume array is already in order:

<

– Recursion:1. Partition splits array in two sub-arrays:• one sub-array of size 0• the other sub-array of size n-12. Quicksort each sub-array– Depth of recursion tree? O(n)– Number of accesses per partition? O(n)

Quicksort: Best Case Complexity

• Assume that keys are random, uniformly distributed.

• Best case running time: O(n log2n)

Merge Sort

• Quicksort : worst case complexity in the worst case is O(n2) because it is difficult to control the partitioning process

• The partitioning depends on the choice of pivot, no guarantee that partitioning will result in arrays of approximately equal size

• Another strategy is to make partitioning as simple as possible and concentrate on merging sorted (sub)arrays

Merge Sort• Also recursive in nature• Given an array of integers, apply the divide-and-

conquer paradigm– Divide the n-element array into two subarrays of n/2

elements each– Sort the two subarrays recursively using Merge Sort– Merge the two subarrays to produce the sorted array

• The recursion stops when the array to be sorted has length 1

Merge Sort

• Merge Sort– Divide array into two halves– Recursively sort each half– Merge two halves to make sorted whole

Merging• Merging: Combine two pre-sorted lists into a

sorted whole• How to merge efficiently?– Use temporary array– Need index 1 (for array 1), index 2 (for array 2), 3 (for

array tmp)1 3 19 25 5 7 8 9 29

1 3 5 7

Merging

Merge(A, left, middle, right)Create temporary array temp, same size as Ai1 = left;i2 = middle;i3 = 0;while ((i1 != middle) && (i2 != right))

if (A[i1] < A[i2])temp[i3++] = A[i1++]

else temp[i3++] = A[i2++]

copy into temp remaining elements of Acopy into A the contents of temp

Merging

• Time Complexity Analysis• T(n) = c1 if n =1

• T(n) = 2 T(n/2) + c2n

merge

sort

divide

O(n)

2T(n/2)

O(1)

Merge Sort

MergeSort(A, left, right)

if (left < right)

{

middle = (left + right)/2 MergeSort(A, left, middle)

MergeSort(A, middle+1, right)

Merge(A, left, middle+1, right)

}