Algorithm Design and Analysisudel.edu/~amotong/teaching/algorithm design/lectures/(Lec...Radix Sort Version: Summer 2018 Amo G. Tong 27 Title PowerPoint Presentation Author Tong Amo

Algorithm Design and Analysis

Version: Summer 2018 Amo G. Tong 1

Lecture 1-1Non-comparison Sorting• Lower bound of comparison Sorting

• Counting Sort

• Radix Sort


• Comparison Sort• In a comparison sort, we use only comparisons between

elements to gain order information.• Insertion sort, merge sort, heapsort, quicksort,…

• How does each comparison help us to find the correct permutations?• There are totally n! permutations.

Sorting


• Comparison Sort• In a comparison sort, we use only comparisons between

elements to gain order information.• Insertion sort, merge sort, heapsort, quicksort,…

• How does each comparison help us to find the correct permutations?• There are totally n! permutations

• Each comparison eliminates half of the current feasible permutations.

Sorting


• Comparison Sort• How does each comparison help us to find the correct

permutations?• There are totally n! permutations


Sorting


Example:

Input: 0, 9, 2, 1𝑎1, 𝑎2, 𝑎3, 𝑎4

Group1={permutations with 𝑎1 < 𝑎2}; Group2={permutations with 𝑎1 > 𝑎2}.After comparing 𝑎1 and 𝑎2 , we know 𝑎1 < 𝑎2 and thus only have to consider

Group1.




• So the number of comparisons needed in the worst-case is 𝐥𝐨𝐠𝒏!, and thus 𝑻(𝒏) = 𝛀(𝐥𝐨𝐠𝒏!)

Sorting


Some Math: 𝑛! ≥𝑛

2

𝑛/2⇒ 𝑇 𝑛 = Ω(𝑛 log 𝑛)




• So the number of comparisons needed in the worst-case is log 𝑛!, and thus 𝑇(𝑛) = Ω(log𝑛!)

• We cannot do better than 𝚯(𝒏 𝐥𝐨𝐠𝒏) by comparing elements.

Sorting


Some Math: 𝑛! ≥𝑛

2

Τ𝑛 2⇒ 𝑇 𝑛 = Ω(𝑛 log 𝑛)

• Counting Sort

Counting Sort


• Counting Sort• Input: a sequence 𝑎1, … , 𝑎𝑛 of integers in the range 0,… , 𝑘 − 1.• For each value 𝑥 ∈ {0,… , 𝑘 − 1}, count the number of 𝑥 in

𝑎1, … , 𝑎𝑛 .

• List the output.

Counting Sort


• Counting Sort

Counting Sort


Example:

Input: 0, 9, 2, 1 with 𝑘 = 10;Step 1: 𝒄[𝒙] = 𝟎 for 𝒙 ∈ {𝟎,… , 𝒌 − 𝟏}.

• Counting Sort

Counting Sort


Example:

Input: 0, 9, 2, 1 with 𝑘 = 10;Step 1: 𝑐[𝑥] = 0 for 𝑥 ∈ {0, … , 𝑘 − 1}.Step 2: 𝒄 𝟎 = 𝒄 𝟎 + 𝟏;

𝒄 𝟗 = 𝒄 𝟗 + 𝟏;𝒄 𝟐 = 𝒄 𝟐 + 𝟏;𝒄 𝟏 = 𝒄 𝟏 + 𝟏;

• Counting Sort

Counting Sort


Example:

Input: 0, 9, 2, 1 with 𝑘 = 10;Step 1: 𝑐[𝑥] = 0 for 𝑥 ∈ {0, … , 𝑘 − 1}.Step 2: 𝑐 0 = 𝑐 0 + 1;

𝑐 9 = 𝑐 9 + 1;𝑐 2 = 𝑐 2 + 1;𝑐 1 = 𝑐 1 + 1;

Step 3: 𝒄 𝟎 =1, 0;𝒄 𝟏 =1, 0, 1;𝒄 𝟐 =1, 0, 1, 2;𝒄 𝟑 =0, 0, 1, 2;….𝒄 𝟗 =1, 0, 1, 2, 9;

• Counting Sort

Counting Sort


Pseudocode:

COUNTING-SORT(𝐴, 𝑛, 𝑘)

1 for 𝑖 = 0 to 𝑘 − 12 𝑐 𝑖 = 03 end4 for 𝑖 = 1 to 𝑛5 𝑐 𝐴[𝑖] ++6 end7 𝑖𝑛𝑑𝑒𝑥 = 1;8 for 𝑖 = 0 to 𝑘 − 19 for 𝑗 = 0 to 𝑐 𝑖10 𝐵 𝑖𝑛𝑑𝑒𝑥 = 𝑖11 𝑖𝑛𝑑𝑒𝑥++12 end13 end14 Output B;

• Counting Sort

Counting Sort


Pseudocode:


1 for 𝑖 = 0 to 𝑘 − 12 𝑐 𝑖 = 0 //initialize 3 end4 for 𝑖 = 1 to 𝑛5 𝑐 𝐴[𝑖] ++ //count 6 end7 𝑖𝑛𝑑𝑒𝑥 = 1;8 for 𝑖 = 0 to 𝑘 − 19 for 𝑗 = 0 to 𝑐 𝑖10 𝐵 𝑖𝑛𝑑𝑒𝑥 = 𝑖 //copy the output to B11 𝑖𝑛𝑑𝑒𝑥++12 end13 end14 Output B;

• Counting Sort

Counting Sort


Pseudocode:


1 for 𝑖 = 0 to 𝑘 − 12 𝑐 𝑖 = 0 Θ(𝑘)3 end4 for 𝑖 = 1 to 𝑛5 𝑐 𝐴[𝑖] ++ Θ(𝑛)6 end7 𝑖𝑛𝑑𝑒𝑥 = 1;8 for 𝑖 = 0 to 𝑘 − 19 for 𝑗 = 0 to 𝑐 𝑖10 𝐵 𝑖𝑛𝑑𝑒𝑥 = 𝑖 Θ(𝑐[𝑖] + 1)11 𝑖𝑛𝑑𝑒𝑥++12 end13 end14 Output B;

Θ 𝑐 0 + 𝑐 1 +⋯+ 𝑐 𝑖 − 1 + 𝑘= Θ(𝑛 + 𝑘)

Totally, Θ(𝑛 + 𝑘)

• Counting Sort• Input: a sequence 𝑎1, … , 𝑎𝑛 of integers in the range 0,… , 𝑘 − 1.• For each value 𝑥 ∈ {0,… , 𝑘 − 1}, count the number of x in

𝑎1, … , 𝑎𝑛 .


• Totally, 𝚯(𝒏 + 𝒌), beating 𝚯(𝒏 𝐥𝐨𝐠𝒏). WHY?• Essentially, we have more information about the input.

Counting Sort


• Counting Sort• Input: a sequence 𝑎1, … , 𝑎𝑛 of integers in the range 0,… , 𝑘 − 1.• For each value 𝑥 ∈ {0,… , 𝑘 − 1}, count the number of x in

𝑎1, … , 𝑎𝑛 .


• Totally, 𝚯(𝒏 + 𝒌), beating 𝚯(𝒏 𝐥𝐨𝐠𝒏). WHY?• Essentially, we have more information about the input.

Counting Sort


• A sort is stable if it preserves the input order among equal elements.

Stable Sort


0 9 2 2 1

𝑎1 𝑎2 𝑎3 𝑎4 𝑎5input

output

output

𝑎1 𝑎5 𝑎2 𝑎3 𝑎4

𝑎1 𝑎5 𝑎2 𝑎4 𝑎3

stable

not stable

• A sort is stable if it preserves the input order among equal elements.

• Counting sort can be implemented to be stable.(Exercise)

Stable Sort


• Sort the elements on each digit from the lowest digit position to the highest digit position.

• Use a stable sort for each sorting.

Radix Sort


020911291299

020911291299



Radix Sort


020911291299

020911291299

911020291299



Radix Sort


020911291299

020911291299

911020291299

020291299911

• Why a stable sort?

Radix Sort


20119199

20119199

• Why a stable sort?

Radix Sort


20119199

20119199

20119199

20119991

stable

not stable

• Correctness:

• Suppose that the numbers are sorted by their low-order 𝑡 – 1 digits.

• They will be correctly sorted by the low-order 𝑡 digits.

Radix Sort


• Correctness:



• Two numbers that differ in digit t are correctly sorted, because digit 𝑡 is the most significant so far.

• Two numbers have the same digit t are correctly sorted, because we use a stable sort.

Radix Sort


• Correctness:



• Two numbers that differ in digit t are correctly sorted, because digit 𝑡 is the most significant so far.

• Two numbers have the same digit t are correctly sorted, because we use a stable sort.

• The correctness follows inductively.

Radix Sort


Documents

Algorithm Design and Analysisudel.edu/~amotong/teaching/algorithm design/lectures/(Lec...Radix Sort Version: Summer 2018 Amo G. Tong 27 Title PowerPoint Presentation Author Tong Amo