Lecture 4 Sorting in Linear Time - UMass Amherst 241 F19 Lecture 4.pdf · Lecture 4 Sorting in Linear Time ECE 241 –Advanced Programming I Fall 2019 Mike Zink

Lecture4SortinginLinearTime

ECE241– AdvancedProgrammingIFall2019MikeZink

ECE 241 – Adv. Programming I Fall 2018 © 2019 Mike Zink

Overview

1

• CountingSort• RadixSort• BucketSort


Objective

• Understand thatcomparison isnottheonlymethodtosortlists

• Learnaboutsortingalgorithmsthatruninlineartime


Introduction

3

• NowknowseveralalgorithmsthatcansortinO(n log n)time

• MergeSort andHeapsort achieveupperbound inworstcase

• Quicksortachieves thisonaverage• Property:thesortedordertheydetermineisbasedonlyoncomparisonbetweentheinputelements->comparisonsorts


Lower Bounds for Sorting

4

• Comparison sort:• Useonlycomparison betweeninputsequence <a1,a2,…,an>togainorderinformation

• Givenai andaj,perform testsai <aj,ai <=aj,ai = aj,ai >=aj,orai >aj

• Novalueinspection, nogainingoforderinformationinanyotherway


Lower Bounds for Sorting

5

• Assumption: allinputelementsaredistinct• Comparisonai =aj isuseless• Comparisons, ai <aj ,ai <=aj,ai >=aj,andai >aj allequivalentsincetheyyieldidenticalinformationaboutrelativeorderofai andaj

• Thereforeallcomparisons haveformai <=aj


Decision Tree Model

6

≤

1:2

2:3 1:3

2:31:3<1,2,3>

<1,3,2> <3,1,2,> <2,3,1> <3,2,1>

<2,1,3>

>≤

≤

≤

≤>

>

>

>

• a1 =6,a2 =8,a3 =5


Lower Bound for Worst Case

7

• Worstcasenumberofcomparisons:• Longestpathfromroottoanyreachableleave

• Heightofdecision treeislowerbound


Counting Sort

8

• Assumesninputelements(allintegers)inrange0tok

• Basicidea:• Determineforeachinputelementx,numberofelements lessthanx

• Informationcanbeusedtoplaceelement x directlyintoitspositioninoutputarray

• E.g.,ifthereare17elements lessthanx,itbelongsinoutputposition18.


Counting Sort - Algorithm

9

• InputArrayA[1..n]• B[1..n]holdsstoredoutput• C[0..k]provides temporaryworkingstorage

ECE 241 – Adv. Programming I Fall 2018 © 2019 Mike Zink 10

Counting-Sort(A, B, k) 1 fori <- 0tok

2 doC[i]<- 0

3 forj<- 1tolength[A]

4 doC[A[j]]<- C[A[j]]+1

5 #C[i]nowcontainsthenumberofelementsequaltoi.

6 fori <- 1 tok

7 doC[i]<- C[i]+C[i - 1]

8 #C[i]nowcontainsthenumberofelementslessthanorequaltoi.

9forj<- 1tolength[A]downto 1

10 doB[C[A[j]]]<- A[j]

11 C[A[j]]<- C[A[j]]- 1


Counting Sort - Example

11

2 5 3 0 2 3 0 3

1 2 3 4 5 6 7 8

2 0 2 3 0 1

0 1 2 3 4 5

A

C

2 2 4 7 7 8

0 1 2 3 4 5

C


Counting Sort - Example

12

3

1 2 3 4 5 6 7 8

2 2 4 6 7 8

0 1 2 3 4 5

B

C

0 3

1 2 3 4 5 6 7 8

1 2 4 6 7 8

0 1 2 3 4 5

B

C

0 3 3

1 2 3 4 5 6 7 8

1 2 4 5 7 8

0 1 2 3 4 5

B

C


Counting Sort - Analysis

13

• forlooplines1-2:O(k)• Forlooplines3-4:O(n)• Forlooplines6-7:O(k)• Forlooplines9-11:O(n)• Total:O(k+n)• Usewhenk=O(n)->O(n)


Radix Sort

14

• Sortnumbers inacolumndigit-by-digit• Startingwiththeleastsignificantbit

329457657839436720355

720355436457657329839

720320436839355457657

329355436457657720839


Radix Sort

15

• Incomputer,used tosortrecordsofinformationkeyedbymultiplefields

• E.g.,sortdatebythreekeys:year,month,day• Compareyears, iftiecomparemonths, iftiecomparedays


Radix Sort

16

2017-12-012016-04-162017-05-052013-01-102016-06-012014-11-142011-03-30

2011-03-302013-01-102014-11-142016-04-162016-06-012017-12-012017-05-05

2011-03-302013-01-102014-11-142016-04-162016-06-012017-05-052017-12-01

• Sortdates


Radix Sort

17

Radix-Sort(A, d) 1 fori <- 1tod

2 do useastablesorttosortarrayAondigiti


Radix Sort - Analysis

18

LemmaGivennd-digitnumbersinwhicheachdigitcantakeonuptok possiblevalues,Radix-SortcorrectlysortsthesenumbersinO(d(n+ k))timeifthestablesortitusestakesO(n+ k)time.

ProofThecorrectnessofRadix-Sortfollowsbyinductiononthecolumnbeingsorted.Theanalysisoftherunningtimedependsonthestablesortusedastheintermediate sortingalgorithm.Wheneachdigit isintherange0tok-1(sothatitcantakeonk positivevalues),andk isnottoolarge,countingsortistheobviouschoice.Eachpassovernd-digitnumbersthentakestimeO(n+k).Therearedpasses,sothetotaltimeforradixsortisO(d(n+k)).


Bucket Sort

19

• Assumption aboutinput• Input isdrawnfromuniformdistribution• Input isgeneratedbyrandomprocess thatdistributeselementsuniformlyandindependently over interval[0,1)

• (InCountingSort:inputconsistsofintegers inasmallrange)


Bucket Sort

20

• Idea:divideinterval[0,1)intonequal-sized sub-intervalsorbuckets

• Distributeninputnumbers intobuckets• Sinceinputuniformlyandindependentlydistributedover[0,1)=>expectnottoomanynumbers fallintoonebucket

• Output:simplysortnumbers ineachbucketandgothroughbuckets inorder


Bucket Sort

21

Bucket-Sort(A) 1 n <- length[A]

2 fori <- 1ton

3 doinsertA[i]intolistB[⌊n A[i]]

4 fori <- 0ton- 1

5 do sortlistB[i]withinsertionsort

6 concatenatethelists B[0],B[1],…,B[n- 1]togetherinorder


Bucket Sort - Example

22

A1 .782 .173 .394 .265 .726 .947 .218 .129 .23

10 .68

B0 /1234 /5 /678 /9

.12 .17 /

.21 .23 .26 /

.39

.68 /

.72 .78 /

.94


Bucket Sort – Analysis

23

• Alllinesexcept line5take0(n)timeinworstcase

• AnalysisofcostofinsertionsortrevealsthatexpectedtimeforBucketSortis:O(n)+n*O(2– 1/n)=O(n)

• Runsinlinearexpected time


Next Steps

24

• NextlectureonTuesday• Discussions immediatelyfollowingthislecture• Homework1duetonightat11PM

ECE 241 – Data Structures Fall 2018 © 2018 Mike Zink25

Documents

Lecture 4 Sorting in Linear Time - UMass Amherst 241 F19 Lecture 4.pdf · Lecture 4 Sorting in Linear Time ECE 241 –Advanced Programming I Fall 2019 Mike Zink