Searching and Sorting Searching: Sequential, Binary Sorting: Selection, Insertion, Shell

Searching and SortingSearching and Sorting

Searching: Sequential, BinarySearching: Sequential, BinarySorting: Selection, Insertion, ShellSorting: Selection, Insertion, Shell

OutlineOutline

SearchingSearching unsorted listsunsorted lists sorted listssorted lists Java’s Comparable interfaceJava’s Comparable interface

SortingSorting insertioninsertion selectionselection shellshell

Searching an Unsorted ListSearching an Unsorted List

Finding out whether an item is in a listFinding out whether an item is in a list contains methodcontains method

Linear search:Linear search: loop/recur thru list looking at each itemloop/recur thru list looking at each item

» stop when item found or no more list to look atstop when item found or no more list to look at

to iterLinS(arr, item): for i = 0..arr.len-1: if arr[i]=item: return true return false

to recLinS(arr, len, item): if len = 0: return false if arr[len-1] = item: return true; return recLinS(arr, len-1, item)

Linear SearchLinear Search

Worst case? It’s not there!Worst case? It’s not there! total of N comparisons total of N comparisons O(N) O(N)

But if it is there?But if it is there? best case: it’s first (1 comparison): O(1)best case: it’s first (1 comparison): O(1) worst case: it’s last (N comparisons): O(N)worst case: it’s last (N comparisons): O(N) average case: any position equally likely!average case: any position equally likely!

» 1 comparison, 2 comparisons, 3, 4, …, N1 comparison, 2 comparisons, 3, 4, …, N» average = (1 + 2 + 3 + … + N) / Naverage = (1 + 2 + 3 + … + N) / N

= (N(N+1)/2) / N = (N+1)/2: O(N)= (N(N+1)/2) / N = (N+1)/2: O(N)

Best, Worst, AverageBest, Worst, Average

Number of operations often variableNumber of operations often variable depends on specific values being useddepends on specific values being used no single formula that states amount of work!no single formula that states amount of work!

Formulas for best, worst may be easy to getFormulas for best, worst may be easy to get Formula for average usually a bit harderFormula for average usually a bit harder Most interested in worst and averageMost interested in worst and average

best is best is nicenice, but we don’t expect/worry about it, but we don’t expect/worry about it expectexpect average, average, prepare for prepare for worstworst

Searching a Sorted ListSearching a Sorted List

Improved linear search:Improved linear search: stop when we pass where it should have beenstop when we pass where it should have been

should have been before 17should have been before 17so it must not be thereso it must not be there

Saves very little time, actuallySaves very little time, actually best & worst still the samebest & worst still the same average still O(N)average still O(N)

5 7 9 12 13 17 22 25 27 28 42

15

Cutting in HalfCutting in Half

Binary search better for sorted lists:Binary search better for sorted lists: look at middle itemlook at middle item if it is bigger than what we’re looking for, then if it is bigger than what we’re looking for, then

we only need to look in the lower halfwe only need to look in the lower half if it’s smaller than what we’re looking for, then if it’s smaller than what we’re looking for, then

we only need to look in the upper halfwe only need to look in the upper half if it’s what we’re looking for – return the indexif it’s what we’re looking for – return the index

Binary SearchBinary Search

Find midpoint, compare it to the itemFind midpoint, compare it to the item too big too big search lower part of array search lower part of array too small too small search upper part of array search upper part of array otherwise otherwise just right! just right!

5 7 9 12 13 17 22 25 27 28 42

9


Repeat until found…Repeat until found… too big too big search lower part of array search lower part of array too small too small search upper part of array search upper part of array otherwise otherwise just right! just right!

5 7 9 12 13 17 22 25 27 28 42

9


Find midpoint, compare it to the itemFind midpoint, compare it to the item too big too big search lower part of array search lower part of array too small too small search upper part of array search upper part of array otherwise otherwise just right! just right!

5 7 9 12 13 17 22 25 27 28 42

4


Repeat until found…Repeat until found… too big too big search lower part of array search lower part of array too small too small search upper part of array search upper part of array otherwise otherwise just right! just right!

5 7 9 12 13 17 22 25 27 28 42

4


Repeat until found…Repeat until found… or until nowhere left to lookor until nowhere left to look (return fail)(return fail)

5 7 9 12 13 17 22 25 27 28 42

4


int binaryFind(T item, T v[], int lo, int hi)int binaryFind(T item, T v[], int lo, int hi)if (lo > hi)if (lo > hi)

return –1;return –1; // not found// not foundint mid = lo + (hi – lo)/2;int mid = lo + (hi – lo)/2;if (item<v[mid])if (item<v[mid])

return binaryFind(item, v, lo, mid–1);return binaryFind(item, v, lo, mid–1);if (v[mid]<item)if (v[mid]<item)

return binaryFind(item, v, mid+1, hi);return binaryFind(item, v, mid+1, hi);return mid;return mid; // found// found

Binary Search in JavaBinary Search in Java

Want it to work for any kind of arrayWant it to work for any kind of array need a generic methodneed a generic method

» similar to generic class: public class Pair<T>similar to generic class: public class Pair<T> add <T> before return typeadd <T> before return typepublic static <T> int binaryFind(T item, T[] v, int lo, int public static <T> int binaryFind(T item, T[] v, int lo, int

hi)hi) problem: doesn’t compileproblem: doesn’t compileif (if (item < v[mid]item < v[mid]))bad operand types for binary operator ‘<’!

Comparison in JavaComparison in Java

The < operator only works for numbers:The < operator only works for numbers:if (if (“this string” < “that string”“this string” < “that string”))

Need to use compareTo methodNeed to use compareTo methodif (“this string”.compareTo(“that string”) < 0)if (“this string”.compareTo(“that string”) < 0)

The T for binaryFind must The T for binaryFind must alsoalso have the have the compareTo methodcompareTo method and must have told Java that it doesand must have told Java that it does

» recall: interfacesrecall: interfaces

bad operand types for binary operator ‘<’!

The Comparable InterfaceThe Comparable Interface

Tells Java that this object can be comparedTells Java that this object can be compared generic interface: what can it be compared generic interface: what can it be compared toto?? usually to other objects of same classusually to other objects of same classpublic class Person implements Comparable<Person>public class Person implements Comparable<Person>

Only method required is compareToOnly method required is compareTopublic int compareTo(Person other)public int compareTo(Person other)

» returns < 0 if this < otherreturns < 0 if this < other» returns 0 if this == otherreturns 0 if this == other» returns > 0 if this > otherreturns > 0 if this > other

The Comparable InterfaceThe Comparable Interface

Suppose sorting by height (short to tall)Suppose sorting by height (short to tall)public int compareTo(Person other) {public int compareTo(Person other) { return this.height – other.height;return this.height – other.height;}}

» returns < 0 if this.height < other.heightreturns < 0 if this.height < other.height» returns 0 if this.height == other.heightreturns 0 if this.height == other.height» returns > 0 if this.height > other.heightreturns > 0 if this.height > other.height

Suppose sorting by nameSuppose sorting by namepublic int compareTo(Person other) {public int compareTo(Person other) { return this.name.compareTo(other.name);return this.name.compareTo(other.name);}}

Binary Search in JavaBinary Search in Java

Problem was Problem was item < v[mid]item < v[mid] needed to change it to needed to change it to item.compareTo(v[mid]) < 0item.compareTo(v[mid]) < 0 item must compare itself to objects of type Titem must compare itself to objects of type T item’s type must be item’s type must be Comparable<T>Comparable<T>public static <T> int binaryFind(Comparable<T> item,public static <T> int binaryFind(Comparable<T> item,

T[] v, int lo, int hi)T[] v, int lo, int hi) That worksThat works

works for any type that can compare itself to Tworks for any type that can compare itself to T» so we could look for a Student in a Person[]so we could look for a Student in a Person[]

binaryFind in JavabinaryFind in Java

public static <T> int binaryFind(Comparable<T> item, public static <T> int binaryFind(Comparable<T> item, T[] v, int lo, int hi) {T[] v, int lo, int hi) {

if (lo > hi) { return -1; }if (lo > hi) { return -1; } int mid = lo + (hi - lo) / 2;int mid = lo + (hi - lo) / 2; if (item.compareTo(v[mid]) < 0) {if (item.compareTo(v[mid]) < 0) { return binaryFind(item, v, lo, mid - 1);return binaryFind(item, v, lo, mid - 1); } else if (item.compareTo(v[mid]) > 0) {} else if (item.compareTo(v[mid]) > 0) { return binaryFind(item, v, mid+1, hi);return binaryFind(item, v, mid+1, hi); } else { return mid; }} else { return mid; } // found it!// found it! }}

Complexity of Binary SearchComplexity of Binary Search

Worst case, we get half the list every time!Worst case, we get half the list every time! assume N is a power of 2: N == 2assume N is a power of 2: N == 2kk

» 1 comparison reduces list from 21 comparison reduces list from 2k k to 2to 2k-1 k-1

» 1 more comparison reduces from 21 more comparison reduces from 2k-1k-1 to 2 to 2k-2k-2

» ……» 1 more comparison reduces from 21 more comparison reduces from 211 to 2 to 200 (i.e. 1) (i.e. 1)» 1 more comparison reduces from 21 more comparison reduces from 200 to 0 to 0

• #comparisons = k+1 = 1 + log N = O(log N)#comparisons = k+1 = 1 + log N = O(log N) shorter lists: 1 + ceiling(log N) = O(log N)shorter lists: 1 + ceiling(log N) = O(log N)

Sorted ListsSorted Lists

Sorting a list gives Sorting a list gives bigbig performance benefit performance benefit O(N) vs. O(log N) for N = 1,000,000?O(N) vs. O(log N) for N = 1,000,000?

» 1,000,000 vs. 201,000,000 vs. 20 Sorting lists is a very high prioritySorting lists is a very high priority

lots of different sorting methodslots of different sorting methods simplest methods not usually very goodsimplest methods not usually very good

» bubble sort, anybody?bubble sort, anybody? fastest methods usually hard to explainfastest methods usually hard to explain

Some Simple Sorting MethodsSome Simple Sorting Methods

Selection sortSelection sort ““select” items and place in their final positionselect” items and place in their final position

Insertion sortInsertion sort ““insert” items into a sorted sub-vectorinsert” items into a sorted sub-vector

Shell sortShell sort improved insertion sort moves items fasterimproved insertion sort moves items faster

Selection SortSelection Sort

Based on “copying” a list in orderBased on “copying” a list in order select the smallest item in the listselect the smallest item in the list add it to end of new listadd it to end of new list cross it off the old listcross it off the old list

But do it all in one arrayBut do it all in one array front part of list is sortedfront part of list is sorted won’t need to be changed again – everything won’t need to be changed again – everything

left “unsorted” is biggerleft “unsorted” is bigger

–23


Start at low endStart at low end

For each position in the list…For each position in the list… find the smallest unsorted elementfind the smallest unsorted element swap it into positionswap it into position

6 100 3 –2 8 686 –2 1003 100 6 1001008

6 100 3 –2 8

Selection Sort Top-Level TraceSelection Sort Top-Level Trace

lst lst [6, 100, 3, -2, 8]; [6, 100, 3, -2, 8];SelectionSort( lst );SelectionSort( lst ); 00 1 1 2 2 3 3 4 4 lst == lst == 6 6 100 3 -2 8100 3 -2 8

1st pass:find smallest item in listswap it with first item in list

i.e. swap -2 with 6


lst lst [6, 100, 3, -2, 8]; [6, 100, 3, -2, 8];SelectionSort( lst );SelectionSort( lst ); _0_0 1 1 2 2 3 3 4 4 lst == lst == 6 6 100 3 -2 8100 3 -2 8lst == lst == [-2] [-2] 100100 3 6 83 6 8

2nd pass:find smallest remaining item in rest of listswap it into the second position

i.e. swap 3 with 100


lst lst [6, 100, 3, -2, 8]; [6, 100, 3, -2, 8];SelectionSort( lst );SelectionSort( lst ); 00 1 1 2 2 3 3 4 4 lst == lst == 6 6 100 3 -2 8100 3 -2 8lst == lst == [-2] [-2] 100100 3 6 83 6 8lst == lst == [-2 3] [-2 3] 100 6 8100 6 8


lst lst [6, 100, 3, -2, 8]; [6, 100, 3, -2, 8];SelectionSort( lst );SelectionSort( lst ); 00 1 1 2 2 3 3 4 4 lst == lst == 6 6 100 3 -2 8100 3 -2 8lst == lst == [-2] [-2] 100100 3 6 83 6 8lst == lst == [-2 3] [-2 3] 100 6 8100 6 8lst == lst == [-2 3 6] [-2 3 6] 100 8100 8


lst lst [6, 100, 3, -2, 8]; [6, 100, 3, -2, 8];SelectionSort( lst );SelectionSort( lst ); 00 1 1 2 2 3 3 4 4 lst == lst == 6 6 100 3 -2 8100 3 -2 8lst == lst == [-2] [-2] 100100 3 6 83 6 8lst == lst == [-2 3] [-2 3] 100 6 8100 6 8lst == lst == [-2 3 6] [-2 3 6] 100 8100 8lst == lst == [-2 3 6 8] [-2 3 6 8] 100100

ExerciseExercise

Show the evolution of the following array Show the evolution of the following array under selection sort. Show the result after under selection sort. Show the result after each (top-level) passeach (top-level) pass [15, 3, 21, 45, 7][15, 3, 21, 45, 7]

Finding the SmallestFinding the Smallest

Each pass requires finding the smallest item Each pass requires finding the smallest item remaining unsortedremaining unsorted need its location in the array, so we can swap itneed its location in the array, so we can swap it on on iith pass search locations th pass search locations ii–1 .. N–1–1 .. N–1 can start assuming first of those is smallestcan start assuming first of those is smallest


to SelectionSort( List a )to SelectionSort( List a )for i for i 0 .. Length(a) – 2 0 .. Length(a) – 2

p p i; i;for j for j i + 1 .. Length(a) – 1 i + 1 .. Length(a) – 1

if (a[j] < a[p])if (a[j] < a[p])p p j; j;

Swap(a[p], a[i]);Swap(a[p], a[i]);

ExerciseExercise

List all the comparisons made List all the comparisons made between between array elementsarray elements while selection-sorting the while selection-sorting the following list:following list: [2, 8, 12, 4, 9][2, 8, 12, 4, 9]

Insertion SortInsertion Sort

Based on inserting into an ordered listBased on inserting into an ordered list For when you’re creating a listFor when you’re creating a list Keep it sorted right from the startKeep it sorted right from the start Insert each item into its proper place...Insert each item into its proper place... ...shifting other items up as required...shifting other items up as required

But all items already in the arrayBut all items already in the array ““split” array into sorted list part and input partsplit” array into sorted list part and input part

Insertion SortInsertion Sort

Start at low endStart at low end

For each unsorted element…For each unsorted element… find its placefind its place make spacemake space insert itinsert it

6 100 3 –2 8

61006 100 3 –2 86 100 36 100 3 –23 6 100 8–2 1008

Insertion Sort, Top Level LoopInsertion Sort, Top Level Loop

Multiple passes thru the listMultiple passes thru the list At start of At start of iith pass, first th pass, first ii items are in order items are in order

list of length 1 always “in order”list of length 1 always “in order” At end of At end of iith pass, first th pass, first ii+1 items in order+1 items in order

thus need only N – 1 passes in allthus need only N – 1 passes in all On On iith pass, “insert” th pass, “insert” ii+1+1stst item into sorted item into sorted

part of listpart of list

Insertion Sort Top-Level TraceInsertion Sort Top-Level Trace

lst lst [6, 100, 3, -2, 8]; [6, 100, 3, -2, 8];InsertionSort( lst );InsertionSort( lst ); _0_0 _1 _1 2 2 3 3 4 4 lst == lst == [ 6] [ 6] 100 3 -2 8100 3 -2 8

1st pass:list from 0 to 0 is sortedinsert item 1 into the sorted part of the list

i.e. insert 100 into [6]


lst lst [6, 100, 3, -2, 8]; [6, 100, 3, -2, 8];InsertionSort( lst );InsertionSort( lst ); _0_0 _1 _1 2 2 3 3 4 4 lst == lst == [ 6] [ 6] 100 3 -2 8100 3 -2 8lst == lst == [ 6 100] [ 6 100] 3 -2 83 -2 8

2nd pass:list from 0 to 1 is sortedinsert item 2 into the sorted part of the list

i.e. insert 3 into [6, 100]


lst lst [6, 100, 3, -2, 8]; [6, 100, 3, -2, 8];InsertionSort( lst );InsertionSort( lst ); _0_0 _1 _1 2 2 3 3 4 4 lst == lst == [ 6] [ 6] 100 3 -2 8100 3 -2 8lst == lst == [ 6 100] [ 6 100] 3 -2 83 -2 8lst == lst == [ 3 6 100] [ 3 6 100] -2 8-2 8


lst lst [6, 100, 3, -2, 8]; [6, 100, 3, -2, 8];InsertionSort( lst );InsertionSort( lst ); _0_0 _1 _1 2 2 3 3 4 4 lst == lst == [ 6] [ 6] 100 3 -2 8100 3 -2 8lst == lst == [ 6 100] [ 6 100] 3 -2 83 -2 8lst == lst == [ 3 6 100] [ 3 6 100] -2 8-2 8lst == lst == [-2 3 6 100] [-2 3 6 100] 88


lst lst [6, 100, 3, -2, 8]; [6, 100, 3, -2, 8];InsertionSort( lst );InsertionSort( lst ); _0_0 _1 _1 2 2 3 3 4 4 lst == lst == [ 6] [ 6] 100 3 -2 8100 3 -2 8lst == lst == [ 6 100] [ 6 100] 3 -2 83 -2 8lst == lst == [ 3 6 100] [ 3 6 100] -2 8-2 8lst == lst == [-2 3 6 100] [-2 3 6 100] 88lst == lst == [-2 3 6 8 100][-2 3 6 8 100]

ExerciseExercise

Show the evolution of the following array Show the evolution of the following array under insertion sort. Show the result after under insertion sort. Show the result after each (top-level) passeach (top-level) pass [15, 3, 21, 45, 7][15, 3, 21, 45, 7] [17, 4, 8, 3][17, 4, 8, 3]

Insertion on Pass Insertion on Pass ii

Move Move ii+1+1stst item into temporary storage item into temporary storage Starting at the “end” of the sorted part…Starting at the “end” of the sorted part… ……move items up until you find where move items up until you find where

“new” item goes“new” item goes find a smaller itemfind a smaller item fall off the front of the listfall off the front of the list

Insertion on Pass 2 TraceInsertion on Pass 2 Trace

lst == [6, 100], 3, -2, 8lst == [6, 100], 3, -2, 8Insert 3 into [6, 100]Insert 3 into [6, 100] 00 1 1 2 2 temptemplst == lst == [ 6 [ 6 100100 33]] 3 3lst == lst == [ [ 66 100 100 100] 100] 3 3lst == lst == [[ 66 6 100] 6 100] 3 3lst == lst == [ 3 6 100] [ 3 6 100] 3 3

Insertion on Pass 2 TraceInsertion on Pass 2 Trace

lst == [-2, 3, 6, 100], 8lst == [-2, 3, 6, 100], 8Insert 8 into [-2, 3, 6, 100]Insert 8 into [-2, 3, 6, 100] 00 1 1 2 2 3 3 4 4 temptemp[ -2 3 6 [ -2 3 6 100100 88] ] 8 8[ -2 3 [ -2 3 66 100100 100] 100] 8 8[ -2 3 6 8 100][ -2 3 6 8 100] 8 8

Insertion Sort Pseudo-CodeInsertion Sort Pseudo-Code

to InsertionSort( List a )to InsertionSort( List a )for i for i 0 .. Length(a) – 2 0 .. Length(a) – 2

p p i + 1 i + 1temp temp a[p]; a[p];while (p > 0 && a[p – 1] > temp)while (p > 0 && a[p – 1] > temp)

a[p] a[p] a[p – 1]; a[p – 1];p p p – 1; p – 1;

a[p] a[p] temp; temp;

ExerciseExercise

Show the evolution of the following array Show the evolution of the following array under insertion sort. Show the result after under insertion sort. Show the result after each assignment into the array (that is, inner each assignment into the array (that is, inner as well as outer loops):as well as outer loops): [17, 4, 8, 3][17, 4, 8, 3]

Work in Selection SortWork in Selection Sort

to SelectionSort( List a )to SelectionSort( List a )for i for i 1 .. Length(a) – 1 1 .. Length(a) – 1

p p i; i;for j for j i + 1 .. Length(a) i + 1 .. Length(a)

if (a[j] < a[p])if (a[j] < a[p]) p p j; j;temp temp a[p]; a[p];a[p] a[p] a[i]; a[i];a[i] a[i] temp; temp;

Assignment

Comparison

Counting Ops in Selection SortCounting Ops in Selection Sort

Outer loop runs i from 1 to N – 1Outer loop runs i from 1 to N – 1 inner runs j from i+1 to N (inner runs j from i+1 to N (alwaysalways))

i=1i=1N–1N–1 (3 + (3 + j=i+1j=i+1NN 1) 1)

= = i=1i=1N–1N–1 (3 + (3 + j=1j=1NN 1 – 1 – j=1j=1ii 1) 1) = = i=1i=1N–1N–1 (3 + N – i) (3 + N – i) = (3 + N)(N – 1) – (N – 1)(N)/2= (3 + N)(N – 1) – (N – 1)(N)/2 = N= N22 + 2N – 3 – (N + 2N – 3 – (N22 – N)/2 = (N – N)/2 = (N22 + 5N – 6)/2 + 5N – 6)/2

Work in Insertion SortWork in Insertion Sort

to InsertionSort( List a )to InsertionSort( List a )for i for i 1 .. Length(a) – 1 1 .. Length(a) – 1

p p i + 1 i + 1temp temp a[p]; a[p];while (p > 1 && a[p – 1] > temp)while (p > 1 && a[p – 1] > temp)

a[p] a[p] a[p – 1]; a[p – 1];p p p – 1; p – 1;

a[p] a[p] temp; temp;Assignment

Comparison

Counting Ops in Insertion SortCounting Ops in Insertion Sort

Let N be the size of the array/vectorLet N be the size of the array/vector number of elements in the listnumber of elements in the list

Outer loop iterates N – 1 timesOuter loop iterates N – 1 times contains two assignmentscontains two assignments

Inner loop iterates Inner loop iterates at most at most i timesi times list was in reverse order – need to compare alllist was in reverse order – need to compare all contains one comparison & one assignmentcontains one comparison & one assignment

Counting Ops in Insertion SortCounting Ops in Insertion Sort

Outer loop runs i from 1 to N – 1Outer loop runs i from 1 to N – 1 inner runs p from i+1 down to 2 (“worst case”)inner runs p from i+1 down to 2 (“worst case”)

i=1i=1N–1N–1 (2 + (2 + p=2p=2i+1i+1 2) = 2) = i=1i=1N–1N–1 (2 + 2i) (2 + 2i)

= 2(N – 1) + 2(N – 1)(N)/2= 2(N – 1) + 2(N – 1)(N)/2 = 2N – 2 + N= 2N – 2 + N22 – N – N = N= N22 + N – 2 + N – 2

Comparing Selection & InsertionComparing Selection & Insertion

TTSelection sortSelection sort(N) = (N(N) = (N22 + 5N – 6)/2 + 5N – 6)/2 TTInsertion sortInsertion sort(N) = N(N) = N22 + N – 2 + N – 2 N = 1000 gives:N = 1000 gives:

Selection sort: 502,987 operationSelection sort: 502,987 operation Insertion sort: 1,000,998 operationsInsertion sort: 1,000,998 operations

Selection looks fasterSelection looks faster But this is worst-case timeBut this is worst-case time

Worst Worst vs.vs. Average Average

When we did Insertion sort, we said “at When we did Insertion sort, we said “at most” for the inner loopmost” for the inner loop the most work it could do – “worst case”the most work it could do – “worst case” usuallyusually will do less will do less

For Selection, we said “always”For Selection, we said “always” never does any lessnever does any less

Measuring ComplexityMeasuring Complexity

Worst caseWorst case imagine that everything goes badly for usimagine that everything goes badly for us longest possible time for this algorithm to runlongest possible time for this algorithm to run

Average caseAverage case how much work would we how much work would we expectexpect to do to do

Best caseBest case everything goes well – rarely consideredeverything goes well – rarely considered

Average Case AnalysisAverage Case Analysis

Sometimes hard to doSometimes hard to do not this timenot this time

On On averageaverage, we expect inner loop of , we expect inner loop of insertion to only go ½ way to the frontinsertion to only go ½ way to the front

i=1i=1N–1N–1 (2 + ½(2i)) = 2(N–1) + (N–1)(N)/2 (2 + ½(2i)) = 2(N–1) + (N–1)(N)/2

= 2N – 2 + (N= 2N – 2 + (N22 – N)/2 – N)/2 = (N= (N22 + 3N – 4)/2 + 3N – 4)/2

Average Selection & InsertionAverage Selection & Insertion

TTSelection averageSelection average(N) = (N(N) = (N22 + 5N – 6)/2 + 5N – 6)/2 TTInsertion averageInsertion average(N) = (N(N) = (N22 + 3N – 4)/2 + 3N – 4)/2 N = 1000 gives:N = 1000 gives:

Selection sort: 502,987 operationSelection sort: 502,987 operation Insertion sort: 501,498 operationsInsertion sort: 501,498 operations

They do about the same amount of workThey do about the same amount of work and both are O(Nand both are O(N22))

ExerciseExercise

Show the evolution of each of the following Show the evolution of each of the following arrays under both insertion and selection arrays under both insertion and selection sortssorts [99, 14, 21, 12, 5][99, 14, 21, 12, 5] [3, 7, 14, 5, 2, 8][3, 7, 14, 5, 2, 8] [9, 8, 7, 6, 5, 4, 3, 2, 1][9, 8, 7, 6, 5, 4, 3, 2, 1]

Shell SortShell Sort

Sort far apart items firstSort far apart items first then sort the ones that are closer togetherthen sort the ones that are closer together

Sort items separated by a given “gap”Sort items separated by a given “gap” every fifth element, for exampleevery fifth element, for example

Use any sorting methodUse any sorting method insertion sort, insertion sort, e.g.e.g.

Reduce the gap size & repeatReduce the gap size & repeat stop when you’ve sorted with a gap of 1stop when you’ve sorted with a gap of 1

Shell Sort: Gap = 5Shell Sort: Gap = 5

17241232119481 15757741182829

2481

8124Swap

1794

9417Swap

2911 OK

2832

3228Swap

1812 OK

Start at location 6Up by 1 each timeUse insertion sort

Shell Sort: Gap = 5 (cont)Shell Sort: Gap = 5 (cont)

94811228111724 15757741183229

4181

8141Swap

7794

9477Swap

7529 OK

1532

3215Swap

24 OK

17 OK

28

2815Swap


77411215111724 32759481182829

List is now “5-sorted”List is now “5-sorted”

7717 94

4124 81

12 18

15 3228

11 7529


77411215111724 32759481182829

Continue with a smaller gapContinue with a smaller gap77

41

12

15

11

17

24

32

75

94

81

18

28

29

SwapSwap

OKOK

OKSwap (41)


77411215111724 32759481182829

List is now 3-sortedList is now 3-sorted

Note: it’s still 5-sortedNote: it’s still 5-sorted

24

28

17

18

11

12

15

81

77

94

32

75

41

29

2815 322412 94

24281718111215 81779432754129


Now it’s just normal insertion sortNow it’s just normal insertion sort but everything’s “pretty close” to where it’s but everything’s “pretty close” to where it’s

going to end upgoing to end up

Number of assignments:Number of assignments: shell sort:shell sort: 13 + 13 + 15 = 4113 + 13 + 15 = 41 insertion sort:insertion sort: 5757

24281718111215 81779432754129

15 1512 151211 18 1817 28 2824 4129 75 947532 9477 9481

Gap SizesGap Sizes

Any descending sequence will doAny descending sequence will do so long as it ends at 1so long as it ends at 1

Some sequences:Some sequences: N/2, N/4, N/8, …, 1 N/2, N/4, N/8, …, 1 not especially goodnot especially good A = 2A = 2log Nlog N – 1, A/2, A/4, …, 1 – 1, A/2, A/4, …, 1 N/3, N/9, N/27, …, 1 N/3, N/9, N/27, …, 1 …… other sequences may be even betterother sequences may be even better

Gap SizesGap Sizes

N = 13N = 13 insertion sort = 52 comps, 65 storesinsertion sort = 52 comps, 65 stores 11stst gap sequence: 6, 3, 1 gap sequence: 6, 3, 1 41 comps, 77 stores41 comps, 77 stores 22ndnd gap sequence: 7, 3, 1 gap sequence: 7, 3, 1 46 comps, 81 stores46 comps, 81 stores 33rdrd gap sequence: 4, 1 gap sequence: 4, 1 37 comps, 63 stores37 comps, 63 stores

N = 130 (random values)N = 130 (random values) 4487c, 4618s4487c, 4618s 11stst: 65, 32, 16, 8, 4, 2, 1: 65, 32, 16, 8, 4, 2, 1 1383c, 2236s1383c, 2236s 22ndnd: 127, 63, 31, 15, 7, 3, 1: 127, 63, 31, 15, 7, 3, 1 1059c, 1792s1059c, 1792s 33rdrd: 43, 14, 4, 1: 43, 14, 4, 1 1082c, 1592s1082c, 1592s

17241232119481 157741182829

Shell SortShell Sort

to ShellSort( List a )to ShellSort( List a )gap gap Length(a) Length(a) 2; 2; /* or … *//* or … */while (gap while (gap 1) 1)if (gap % 2 == 0) ++gap;if (gap % 2 == 0) ++gap; /* much better! *//* much better! */for i for i gap .. Length(a)–1 gap .. Length(a)–1p p i, i, temp temp a[p]; a[p];while (p while (p gap && a[p–gap] > temp) gap && a[p–gap] > temp)a[p] a[p] a[p–gap], p a[p–gap], p p–gap; p–gap;a[p] a[p] temp; temp;gap gap gap gap 2; 2; /* or … *//* or … */

ExerciseExercise

Show the evolution of the following arrays Show the evolution of the following arrays under shell sort. Use length/3 as the under shell sort. Use length/3 as the starting gap:starting gap: [15, 3, 21, 45, 7, 17, 4][15, 3, 21, 45, 7, 17, 4] [13, 11, 20, 15, 16, 6, 5, 8][13, 11, 20, 15, 16, 6, 5, 8]

Complexity of Shell SortComplexity of Shell Sort

Depends on the gap sequenceDepends on the gap sequence O(NO(N22) for ) for N/2 versionN/2 version

» O(NO(N3/23/2) if we do the only-odd-numbers version) if we do the only-odd-numbers version O(NO(N3/23/2) for 2) for 2log Nlog N – 1 version – 1 version

» ……, 63, 31, 15, 7, 3, 1, 63, 31, 15, 7, 3, 1» some others have this as wellsome others have this as well

some versions have O(Nsome versions have O(N4/34/3)) one version has O(None version has O(N1+sqrt(8ln(5/2)/ln(N))1+sqrt(8ln(5/2)/ln(N)))) others we don’t even know!others we don’t even know!

CommentsComments

All O(NAll O(N22) in worst case) in worst case Shell sort can be O(NShell sort can be O(N3/23/2), but still not good enuf), but still not good enuf doubling size of array quadruples timedoubling size of array quadruples time

Want something better!Want something better! Recursion offers a way upRecursion offers a way up

next time: merge sort, quick sortnext time: merge sort, quick sort

Documents

Searching and Sorting Searching: Sequential, Binary Sorting: Selection, Insertion, Shell