Embed Size (px)
Citation preview
1
Chapter 7: Sorting
Example: EASYQUESTION AEEINOQSSTUY
Run-time: # of comparisons # of swaps
O(N2),O(N logN) & O(N) Algorithms
1
Sorting: Summary
1. Selection sort2. Bubble sort3. Insertion sort (Section 7.1)
4. Shellsort (Section 7.4)
5. Heapsort (Section 7.5)
6. Mergesort (Section 7.6)
7. Quicksort (Section 7.7)
8. Counting sort9. Radix sort (Section 7.11)
10. External sorting (Section 7.12)
Note: Selection sort and bubble sort are not in the textbook. Already covered in CS 26/110?
Internalsorting
O(N2)
O(N logN)
O(N) – one of the most creative CS topics!
2
2
1. Selection sort: concept
Select the smallest element from the file, select the next smallest element from the remaining file, (repeat).
Select the smallest element from thissubfile and swap itwith the leftmostElement ( ).
3
Selection Sort: ExampleSource: Sedgewick
File: EASYQUESTIONSize: 12 items
#comps #swapsEASYQUESTIONAESYQUESTION 11 1AESYQUESTION 10 1AEEYQUSSTION 9 1AEEIQUSSTYON 8 1AEEINUSSTYOQ 7 1AEEINOSSTYUQ 6 1AEEINOQSTYUS 5 1AEEINOQSTYUS 4 1AEEINOQSSYUT 3 1AEEINOQSSTUY 2 1AEEINOQSSTUY 1 1Total 66 11
123456789
1011
#pass
#swaps = 11 if identical elements are swapped.
4
3
Selection sort: analysis
Which factors depend on the sorting keys? The number of passes?
The number of comparisons?
The number of swaps?
Run-time: Run-time metric: number of comparisons between
two elements Θ(N2) always (no worst, average, or best case).
no (always N-1)
no (always N-i in the i-th pass)
no (always 1 in each pass)
5
Recursive pseudo code for Selection sort
selectionSort(S,T){// S: source string; T: target string if S.length==0 then // Base case (first 1/3){ T <-S; return;
}
smallest(S, s1, S_Rest);
// select the smallest, s1, from S, and S_Rest is the remaining string
selectionSort(S_Rest,T_Rest);// Recursion with assumption (second 1/3): // when N = k-1, S_Rest -> T_Rest
T <- append(s1 , T_Rest); // (Third 1/3): when N = kreturn;
}
Run-time analysis: O(N2).6
4
2. Bubble sort: concept
“Bubble up” from the last element to a[0], from the last to a[1], from the last to a[2], etc.
Bubble up from the last position of this subfile to the next position of the alreadysorted subfile.
In each pass, the bubble contains the smallest element seen so far from the bottom.
7
File: EASYQUESTIONSize: 12 items
#comps #swapsEASYQUESTION worst worstAEESYQUISTNO 11 (11) 8 (11)AEEISYQUNSTO 10 (10) 6 (10)AEEINSYQUOST 9 (9) 6 (9)AEEINOSYQUST 8 (8) 4 (8)AEEINOQSYSUT 7 (7) 3 (7)AEEINOQSSYTU 6 (6) 2 (6)AEEINOQSSTYU 5 (5) 1 (5)AEEINOQSSTUY 4 (4) 1 (4)AEEINOQSSTUY 3 (3) 0 (3)
(2) (2) (1) (1)
Total 63 (66) 31 (66)
#pass
123456789
Bubble Sort: ExampleSource: Sedgewick
8
5
Bubble sort: analysis (1/5)
Which factors depend on the sorting keys (of the input file)? The number of passes
The number of comparisons*
The number of swaps
Footnote (*): No in each pass. The total number of comparisons may be affected, as the sorting may terminate early when the number of swaps in any pass equals 0.
yes
noyes
9
Bubble sort: analysis (2/5)
Worst-case: Run time?
An example of the worst case? Elements are in the reverse order, e.g., JIHGFEDCBA
Number of passes?
Number of comparisons?
Number of swaps?
Θ(N2)
N-1
(N-1)N/2
(N-1)N/2
10
6
Bubble sort: running time (3/5)
Best case: Run time?
An example of the best case? Elements are already in the sorted order, e.g.,
ABCDEFGHIJ
Number of passes?
Number of comparisons?
Number of swaps?
Θ(N)
1
N-1
0
11
Bubble sort: running time (4/5)
Average case: Run time: When the elements are in a random order. Number of passes depends on the input file,
and is no more than N-1. For simplicity, consider (N-1)/2 passes on
average, then: Number of comparisons ~= (3/8)N2 – N/2
(see the next slide)
Number of swaps ~= (N-1)N/4 (about half the worst case).
Θ(N2)
12
7
Bubble sort: running time (5/5)
The number of comparisons in the average case: Let p be the number of passes until the file is completely
sorted. Then, the total number, C, of comparisons is:
C = (N-1) + (N-2) + … + (N-p)= (N-1)N/2 – (N-p-1)(N-p)/2
Assume p = (N-1)/2, that is, half the maximum possible number of passes. Then,
C = (3/8)N2 – N/2 + 1/8.
Exercise: Write a recursive function for Bubble sort.
13
3. Insertion sort: concept
For each element, insert it into the (already sorted) subfile on the left.
14
8
File: EASYQUESTIONSize: 12 items
#comps #swapsEASYQUESTION worst worstAESYQUESTION 1 (1) 1 (1)AESYQUESTION 1 (2) 0 (2)AESYQUESTION 1 (3) 0 (3)AEQSYUESTION 3 (4) 2 (4)AEQSUYESTION 2 (5) 1 (5)AEEQSUYSTION 5 (6) 4 (6)AEEQSSUYTION 3 (7) 2 (7)AEEQSSTUYION 3 (8) 2 (8)AEEIQSSTUYON 7 (9) 6 (9)AEEIOQSSTUYN 7 (10) 6 (10)AEEINOQSSTUY 8 (11) 7 (11)Total 41 (66) 31 (66)
#pass
123456789
1011
Insertion Sort: ExampleSource: Sedgewick
15
Insertion sort: running time (1/2)
Which factors depend on sorting keys (of the input file)? The number of passes: no (always N-1)
The number of comparisons: yes
The number of swaps: yes
Worst case run-time: O(N2) When elements are in the reverse order, e.g.,
JIHGFEDCBA.
(N-1)N/2 comparisons and (N-1)N/2 swaps.
16
9
Insertion sort: running time (2/2)
Best case running time: O(N) When elements are already in the sorted order,
e.g., ABCDEFGHIJ.
One comparison and no swap at each pass. Hence, N-1 comparisons and 0 swap.
Average case running time: O(N2) Elements are in a random order.
Half of the worst case run-time.
17
Insertion sort: exercise
Sort the file ILOVERECURSION.
18
10
4. Shellsort: concept - presorting
An extended version of the insertion sort
With insertion sort, the number of swaps generally increases as the number of passes increases because the sorted subfile in each pass has more
elements on the left as the passes progress.
The Shellsort alleviates this problem by ‘presorting’ the elements at a regular interval --typically in a sequence of intervals (called the increment sequence) h1, h2, …, ht.
19
Shellsort: example (1/3)Data file: EASYQUESTION (size = 12)Assume an increment sequence 4, 1.
#comps #swapsEASYQUESTIONEASYQUESTION 2 0EASYQIESTUON 3 1EAEYQIOSTUSN 3 2EAENQIOSTUSY 3 3Subtotal 11 6
Phase 1: increment = 4
20
11
Shellsort: example (2/3)
#comps #swapsEAENQIOSTUSYAEENQIOSTUSY 1 1AEENQIOSTUSY 1 0AEENQIOSTUSY 1 0AEENQIOSTUSY 1 0AEEINQOSTUSY 3 2AEEINOQSTUSY 2 1AEEINOQSTUSY 1 0AEEINOQSTUSY 1 0AEEINOQSTUSY 1 0AEEINOQSSTUY 3 2AEEINOQSSTUY 1 0Subtotal 16 6
Phase 2: increment = 1
21
Shellsort: example (3/3)
Total 27 comparisons and 12 swaps. Why smaller than in the insertion sort? Effect of presorting, which reduces the number of
swaps required in the second phase (i.e., with the increment 1)
Typically an increment sequence longer than two numbers is used.
22
12
Side note: Rationale for presorting
Observation: Insertion sort becomes more efficient as the elements become closer to being sorted.
Idea: Shellsort rearranges the elements so that the file is partially sorted at regular intervals and, then, uses insertion sorting on the resulting file.
As a result, the total run-time is shorter than doing insertion sort from the beginning.
23
Shellsort: increment sequence and run-time
The increment sequence influences the running time. Shell’s increment sequence: O(N2)
hk = hk-1/2. That is, 1, 2, 4, 8, 16, 32, …, N/2 .
Knuth’s increment sequence: O(N3/2) hk = 1+3*hk-1. That is, 1, 4, 13, 40, 121, 364, … N/2.
Hibbard’s increment sequence: O(N3/2) hk = 2k-1. That is, 1, 3, 7, 15, 31, … N/2.
Sedgewick’s increment sequence: O(N4/3) hk = 9*4k – 9*2k + 1 (k = 0, 1, 2, …) for odd positions,
4k – 3*2k + 1 (k = 2, 3, 4, …) for even positionsThat is, 1, 5, 19, 41, 109, 209,… N/2.
Footnote. The running time is theta (Θ), precisely speaking.24
13
Shellsort: increment sequence and run-time
Sedgewick’s increment sequence is the best known in practice.
Shell’s increment is the worst among the four. This is caused by the fact that the numbers are not
prime to each other.
The Shellsort has been empirically demonstrated to be more efficient than the insertion sort. A precise running time analysis for Shellsort is
known to be infeasible.
25
Shellsort: exercise
Sort the file ILOVERECURSION using the increment sequence 4,1.
26
14
5. Heapsort: concept
Build a max heap and repeat deleting the maximum element until the heap is empty. Phase 1: buildMaxHeap
Phase 2: do { deleteMax } until empty
The file is sorted in the reverse order if min-heap (and deleteMin) is used.
27
Heapsort: example (1/3)
File: EASYQUESTION
Phase 1: buildMaxHeap
E
A S
Y Q U E
S T I O N
E A S Y Q U E S T I O N0 1 2 3 4 5 6 7 8 9 10 11 12
Y
T U
S Q S E
E A I O N
Y T U S Q S E E A I O N0 1 2 3 4 5 6 7 8 9 10 11 12
28
15
Heapsort: example (2/3)
deleteMaxU
T S
S Q N E
E A I O
U T S S Q N E E A I O Y0 1 2 3 4 5 6 7 8 9 10 11 12
deleteMax
Phase 2: deleteMax until empty
Y
T U
S Q S E
E A I O N
Y T U S Q S E E A I O N0 1 2 3 4 5 6 7 8 9 10 11 12
29
Heapsort: example (3/3)
T
S S
O Q N E
E A I
T S S O Q N E E A I U Y0 1 2 3 4 5 6 7 8 9 10 11 12
deleteMax
A E E I N O Q S S T U Y0 1 2 3 4 5 6 7 8 9 10 11 12
. . .
30
16
Heapsort: running time
Phase 1 (buildMaxHeap): O(N) Proof: See Weiss’ book Theorem 6.1.
Phase 2 ({deleteMax}): O(N logN) Proof:
Percolating down the root in a binary heap of k nodes performs 2logk comparisons.
So, the total number of comparisons = 2 log(N-1) + 2 log(N-2) + …+ 2 log2 < 2 logN + 2 logN +… + 2 logN = 2(N-2)logN = O(N logN).
The number of swaps is no more than the number of comparisons.
31
Heapsort: exercise
Sort the file ILOVERECURSION.
32
17
6. Mergesort: concept
Divide a file into subfiles, sort each subfile, and merge the sorted subfiles. This is a divide-and-conquer algorithm.
This algorithm needs temporary space for storing merged subfiles, so is not in-place sorting (We could do without the temporary space at the cost of
higher complexity of the algorithm, but it’s not covered here.)
In-place sorting: no need for a temporary space.
33
Mergesort: example (1/2)
File: EASYQUESTION (size = 12)
EASYQUESTION
EASYQU ESTION
EAS YQU EST ION
E AS Y QU E ST I ON
A S Q U S T O N
Divide
34
18
Mergesort: example (2/2)
AEEINOQSSTUY
AEQSUY EINOST
AES QUY EST INO
E AS Y QU E ST I NO
A S Q U S T O N
Merge (conquer!)(The subfiles colored in blue are those whose merge results are different from a simple concatenation of the two input subfiles.)
35
Mergesort: algorithm// a: the file (i.e., array) of items to be sorted// temp: a temporary file for storing the merge output// left: the left-most index of the subfile// right: the right-most index of the subfilemergesort(a, temp, left, right) {
if (left < right) {int center = (left+right)/2; // integer part onlymergesort(a, temp, left, center);mergesort(a, temp, center+1, right);merge(a, temp, left, center+1, right);
}}
Source: Weiss
36
19
Mergesort: mergemerge(a, temp, leftpos, rightpos, rightend) {
leftend = rightpos – 1; tmppos = leftpos;numElements = rightend – leftpos + 1;// Merge the left and the right halves of a[] and store the result in temp[].while (leftpos <= leftend && rightpos <= rightend) {
if (a[leftpos] <= a[rightpos])temp[tmppos++] = a[leftpos++]
else temp[tmppos++] = a[rightpos++]}// Copy either the left half or the right half, whichever was left over.while (leftpos <= leftend) temp[tmppos++] = a[leftpos++]; // left halfwhile (rightpos <= rightend) temp[tmppos++] = a[rightpos++]; // right half// Copy temp[] back to a[].for (i=0; i < numElements; i++, rightend--)
a[rightend] = temp[rightend];}
37
Mergesort: merge example
Left half: A G I N O R S T
Right half: E E F L M P X Y
A G I N O R S T
E E F L M P X Y
38
20
Mergesort: algorithm tracing
EASYQUESTIONEASYQUESTIONEASYQUESTIONEASYQUESTIONEASYQUESTIONAESYQUESTIONAESYQUESTIONAESYQUESTIONAESYQUESTIONAESYQUESTIONAESQUYESTIONAEQSUYESTION
AEQSUYESTIONAEQSUYESTIONAEQSUYESTIONAEQSUYESTIONAEQSUYESTIONAEQSUYESTIONAEQSUYESTIONAEQSUYESTIONAEQSUYESTINOAEQSUYESTINOAEQSUYEINOSTAEEINOQSSTUY
Show how the order of elements changes (after merging) as the sorting progresses.
39
Mergesort: running time (1/3)
Let T(N) be the number of comparisons between two elements for sorting a file of N elements. Then,
T(N) = T(N/2) + T(N/2) + NT(1) = 0
Note: running time of mergeIt takes ~N comparisons to merge two files of N/2 elements each.e.g., 14 comparisons for merging two files of 8 elements each:
A G I N O R S T (8 elements)
E E F L M P X Y (8 elements)
40
21
Mergesort: running time (2/3)
Let N = 2n. Then,T(2n) = 2T(2n-1) + 2n
2T(2n-1) = 22T(2n-2) + 2n
22T(2n-2) = 23T(2n-3) + 2n
. . .
. . .+) 2n-1T(21) = 2nT(20) + 2n___________________________________
T(2n) = 2n · n
Hence, T(N) = O(N logN)
0
T(N) = 2 T(N/2) + N for N > 1= 0 for N = 0
Note: always O(N logN), i.e., no best, average, or worst case.
41
Side note: Mergesort running time (3/3)
See the textbook for another – more elegant – technique for solving this recurrence equation.
Let S(N) = T(N)/N. Then, the recurrence relation of T(N) is the same as:
S(N) = S(N/2) + 1 S(1) = 0
42
22
Mergesort: exercise
Sort the file ILOVERECURSION using mergesort.
43
7. Quicksort: concept
It’s quick!
Partition the file into two subfiles so that: all elements in one subfile pivot
all elements in the other subfile > pivot.
Repeat this partitioning until each partition has only one element.
This is a divide-and-conquer algorithm.
44
23
Quicksort: algorithm
void quicksort(File a[], int left, int right) {if (right <= left) return;// pick the partitioning element (a.k.a., pivot)int i = partition(a, left, right); quicksort(a, left, i – 1); // sort the left subfilequicksort(a, i+1, right); // sort the right subfile
}
45
Quicksort: partitioning
The partition function divides the elements into those smaller than the pivot and those larger than the pivot.
{a[j] | a[j] a[i]for all j [left, i-1]}
{a[k] | a[k] > a[i]for all k [i+1,right]}
a[i]
Result ofquicksort(a, left, i-1)
Result ofquicksort(a, i+1, right)
Result isPartitioning element (a.k.a., pivot)
46
24
Quicksort: partitioning algorithmint partition (a[], left, right) {// a[]: element keys.// left: array position of the leftmost element.// right: array position of the rightmost element.
Choose the pivot and swap it with a[right];int i = left, j = right;repeat {
// scan from i= left to the right until find the 1st element > pivot.while (a[++i] ≤ pivot) {};// scan from j= right to the left until (1) find the 1st element ≤ pivot,// or (2) j falls off the left position (i.e., j < left).while (a[--j] > pivot) if (j == left) break;if (i < j) // i and j did not cross each other
then swap a[i] and a[j];else // i and j crossed each other or j < left.
break;}swap a[i] and a[right];return i; //Here, a[i] is between the left and the right subfiles.
}
47
Quicksort: partitioning exampleFile: EASYQUESTION (size = 12). Pivot: N
E A S Y Q U E S T I O N
Scan until a[i] > a[right] Scan until a[j] a[right]
i jSwap(a[i], a[j])
E A I Y Q U E S T S O Ni jSwap(a[i], a[j])
E A I E Q U Y S T S O Ni
jSwap(a[i], a[right])
(i and j crossed each other.)
E A I E N U Y S T S O Q
left subfile right subfile48
25
Quicksort: pivot selection
Option 1: always choose either the first or the last element. pro: fast con: if the file is already sorted, then it never partitions into
two subfiles.
Option 2: randomly choose an arbitrary element. pro: on average, a file is divided into two equal-sized
halves. con: incurs the overhead of random-number generation.
Option 3: choose the median of three (i.e., left, center, and right), or five, seven, etc. This is a compromise between option 1 and option 2.
49
Quicksort: Exercise
Sort the file ILOVERECURSION. Choose the median-of-three as the pivot.
50
26
Quicksort: running time (1/3)
Let T(N) be the number of comparisons between two elements for sorting a file of N elements.
Then, for any i ∈ [1:N],T(N) = T(i) + T(N-i-1) + N for N > 1T(0) = T(1) = 0where: T(i) and T(N-i-1) are for the recursive quicksort of
the two subfiles of sizes i and N-i-1 each. N is for the partitioning (which involves a linear
scan starting from the two ends).
51
Quicksort: running time (2/3)
Worst case: O(N2) The partitioned file sizes are 0 and N-1. In other
words, the pivot is always the smallest (or the largest) element.
T(N) = T(0) + T(N-1) + N = T(N-1) + N
Best case: O(NlogN) The partitioned file sizes are equal (= N/2).
T(N) = 2T(N/2) + N
52
27
Quicksort: running time (3/3)
Average case: O(N logN) The partitioned file sizes can be any value in 0 ~
N-1 (with equal probability).
T(N) = i=0N-1 p(i) (T(i) + T(N-i-1)) + N
= (1/N) ( i=0N-1T(i) + i=0
N-1T(N-i-1) ) + N
= (2/N) j=0N-1T(j) + N
53
Quicksort: variant
Small-subfile cutoff Quicksort does not perform as well as insertion
sort for a small file. So, use insertion sorting when subfile size < M. (M = 5 ~ 20 is known to be small enough.)
54
28
Quicksort vs. mergesort Comparison with the mergesort:
Both mergesort and quicksort are divide-and-conquer algorithms.
Mergesort does the actual sorting in the ‘conquer’ (i.e., merge) phase - while returning from recursive calls to mergesort.
Quicksort does the actual sorting in the ‘divide’ (i.e., partition) phase - while making recursive calls to quicksort.
Complexity: O(NlogN) Best case in quicksort Worst case in mergesort
Then, why quicksort? Quickselect etc – no need to sort all elements.
55
Quickselect
Selection of the k-th smallest element, using quicksort.
quickselect(file, k) {Pick a pivot.Partition the file into the left subfile L and
the right subfile R.If k size(L) then return quickselect(L, k).else if k == size(L)+1 then return the pivot.else return quickselect(R, k-size(L)-1).
}
56
29
Quickselect: running time
Worst case: O(N2) T(N) = T(N-1) + N for N ≥ 1
T(0) = 0
Best case: O(N) T(N) = T(N/2) + N for N ≥ 1
T(0) = 0
Average case: O(N) T(N) = (1/N)i=0
N-1 T(i) + N for N ≥ 1
T(0)= 0
Exercise.
57
8 & 9. Bucket sort
Concept
8. Counting sort
9. Radix sort
58
30
Bucket sort: concept
Distribute the elements into buckets based on the elements’ key values. Then, sort each bucket separately and concatenate them. The key: a small integer or convertible to one (with
O(1)). Insert an element with the key a[i] into bucket[ f(a[i]) ],
where f is a key conversion function.
Bucket sort: does not compare & swap elements but distribute them.
Bucket sort uses buckets and is not in-place sorting.
59
8. Counting sort: concept
Use a counter array (CA): The array size is the maximum possible key value among the
elements sorted. Each counter array cell makes one bucket.
Algorithm:// Distribution:For each element a[i] in the input file {
compute the key value f(a[i]);increase the counter CA[ f(a[i]) ] by one;
}// Scan:For each element CA[j] in the counter array {
repeat the corresponding key value CA[j] times. }
Note: it is the value of a key, not the number of keys, that determines the size of the counter array. So, the array size may be significantly large if the maximum key-value is large. 60
31
Counting sort: example
E A S Y Q U E S T I O N (12 elements)(a[ ] = 69, 65, 83, 89, 81, 85, 69, 83, 84, 73, 79, 78)
1 0 0 0 2 0 0 0 1 0 0 0 0 1 1 0 1 0 2 1 1 0 0 0 1 0
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25A B C D E F G H I J K L M N O P Q R S T U V W X Y Z
AEEINOQSSTUY
Distribute into the counter array CA, using f([a[i]) = a[i] – 65.
Scan the counter array
CA:
Key:
61
Counting sort: exercise
Sort the file ILOVERECURSION
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25A B C D E F G H I J K L M N O P Q R S T U V W X Y Z
CA:
Key:
62
32
Counting sort: running time
O(N+M) always O(N) for the distribution, where N is the input file
size.
O(M) for the scan, where M is the counter array size.
Note: O(N+M) = O(max(N,M)) = O(N) if M = O(N), i.e., M
increases at most linearly with N. In the examples we’ve seen, M = 26, constant.
63
9. Radix sort: concept
The key is a radix r integer. Do a bucket sort from the least significant digit
(LSD) first. Called the LSD radix sort.
Distribute the keys piece by piece (not in their entirety) for better efficiency. The piece is a digit if the key is a number and a character if
the key is a string. Effective for long keys.
64
33
LSD radix sort: algorithm
For each digit from the least significant digit first {1. Distribute the elements in the input file into buckets
by the current digit.2. Collect the elements in the buckets back into the
file.}
65
LSD radix sort: algorithm
LSDradixSort(a[], d) {// a: array of the element keys// d: key size (i.e., the number of digits in a key)
for i = 1 to d { // for each digit, with the LSD firstclear bucket[0], bucket[1],…, bucket[r-1]; // for radix r.// Distributefor j = 0 to N-1 // for each element of a[ ]
insert a[j] at the end of bucket[i-th_digit(a[j])];// Collectinsert the contents of bucket[0], bucket[1],…,bucket[r-1]into a[];
}}
66
34
LSD radix sort: example (1/2)File: EASYQUESTION (12 elements)
(= 69, 65, 83, 89, 81, 85, 69, 83, 84, 73, 79, 78)
Distribute into digit_bucket[1st_digit(a[i])].
Buckets
0 1 2 3 4 5 6 7 8 9
81 83 83 73
84 65 85
69 89 69 79
78
Collect
QSSITAUNEYEO(= 81, 83, 83, 73, 84, 65, 85, 78, 69, 89, 69, 79)
Radix = 10
dd
2nd digit 1st digit
67
LSD radix sort: example (2/2)
Distribute into digit_bucket[2nd_digit(a[i])].
Buckets
0 1 2 3 4 5 6 7 8 981 83 83 84 85 89
73 78 79
65 69 69
AEEINOQSSTUY(= 65, 69, 69, 73, 78, 79, 81, 83, 83, 84, 85, 89)
Collect
QSSITAUNEYEO(= 81, 83, 83, 73, 84, 65, 85, 78, 69, 89, 69, 79)
68
35
LSD radix sort is “stable”.
Stable sorting When a file is sorted by multiple sorting keys in
sequence, the sorting is stable if the order of elements sorted by one key is preserved when the file is sorted by another key.
In the case of LSD radix sort, the multiple sorting keys are mulitple digits (from the least significant one first).
69
LSD radix sort: exercise
LSD radix sort the file with integer keys: 220, 294, 003, 528, 129, 398, 123, 210, 513, 567, 834, 019. Start with the least significant digit.
How many buckets do we need? 10 buckets, one for each digit 0 to 9.
70
36
LSD radix sort: running time
O(d*N) always Where d is the key size (e.g., the number of digits,
the number of characters) and N is the sorted file size (i.e., the number of elements)
Note: O(d*N) = O(N) if d is fixed and much smaller than
N.
71
Side note: MSD radix sort
Algorithm:
1. Bucket := file.
2. For each digit from the MSD to the LSD {
Distribute elements in each bucket into smaller r buckets by the current digit.
}
72
37
10. External sort
Data are stored in an external storage device like the disk (not in main memory). Textbook considers files in tape, but we consider
files in disk here. No difference for our purpose because we assume the sorted elements are stored back to back in contiguous disk pages.
The disk page I/O time (not the CPU time) is the dominant running time metric. Here, we will use a simplified metric, the number
of disk pages accessed.
73
Side note: Disk I/O cost metric
The number of disk pages accessed (i.e., read or written) during the execution is used commonly for disk I/O cost analysis.
It, however, misses out the effect of the actual layout of disk pages on disk.
But, positively speaking, counting only the number of pages accessed is neutral to different possible layouts of the disk pages.
74
38
External sort: algorithm
Two phases: initial sorting and progressive merging.
ExternalSort(a[], b, m) {// a[]: data file (Note: Each array element is a disk page.)// b: file size (pages); m: main memory buffer size (pages)// 1. Sort the file to generate an initial set of subfiles.Sort a[] with m pages each time. // Generates n = b/m sorted subfiles.// 2. Merge the subfiles progressively.d = m – 1; // degree of merging (e.g., d = 2, i.e., binary, if m = 3). for p = 1 to logdn // each pass
for i = 1 to n/dpMerge d subfiles of size mdp-1 pages into one sorted file.
}
Why d = m –1 ? Because one page is reserved for the output.
75
External sorting: example
1st pass
2nd pass
3rd pass
4th pass
Data file (31 pages)
Main memory buffer (m = 3 pages)
Initial sorting
In each pass, the size of a subfile doubles while the number of subfiles halves. (Note: d = m – 1 = 2)
76
39
External sorting: running time
Disk I/O cost is Initial sorting: 2b page I/O’s (b page reads and b page
writes) Merging: logdn = logdb/m passes with 2b page I/O’s in
each pass, where: b: the data file size (pages) m: the main memory buffer size (pages) n (= b/m ): the number of initial subfiles generated d: the degree of merging (= m – 1).
In summary, O(2b + 2b logdb/m) = O(b logdb/m)
= O(b logdb) if m is fixed and m << b.
77
Sorting running time: wrap-up (1/3)
Selection sort: O(N2) always Bubble sort: O(N2) average, worst; O(N) best Insertion sort: O(N2) average, worst; O(N) best Shellsort*: O(N4/3) best known worst Heapsort: O(NlogN) always Mergesort: O(NlogN) always Quicksort: O(NlogN) average; O(N2) worst Counting sort: O(N) always Radix sort: O(N) always External sorting: O(blogb)) for a file of b pages.
Shellsort has different worst case run-time depending on an “increment sequence”.78
40
Sorting running time: wrap-up (2/3)Run-time Average Best worst
Selection* O(N2) O(N2) O(N2)
Bubble O(N2) O(N) O(N2)
Insertion O(N2) O(N) O(N2)
Shell O(N1.3) O(N) O(N1.3)
Heap* O(NlogN) O(NlogN) O(NlogN)
Merge* O(NlogN) O(NlogN) O(NlogN)
Quick O(NlogN) O(NlogN) O(N2)
Counting* O(N) O(N) O(N)
Radix* O(N) O(N) O(N)
*: no average/best/worst case. 79
Sorting running time: wrap-up (3/3)
Insertion sorting preferable for a small file (e.g., ~50 elements), especially if the file is nearly sorted already.
Quicksort preferable for a large file (e.g., ~million elements).
Radix sort preferable for a large file with a long sorting key, provided that enough memory space is available for buckets.
80
41
Sorting: trivia questions
For each sorting algorithm: Does its running time depend on the order of
the elements in a file?
How does it sort a file whose elements are all duplicates,.e.g., AAAAAAAAA?
Exercise!
81
Sorting: More Topics Internal sorting vs. external sorting
depending on where the file resides: main memory vs. disk
Array sorting vs. linked list sorting depending on the data structure of the file. Direct access to an array, whereas sequential access to
a linked list.
Direct sorting vs. indirect sorting depending on whether we move the actual items in the
file or only the pointers (or indices) to them
In-place sorting: no need for a temporary space. We assumed internal, array, direct and in-place
sorting unless stated otherwise.
82
