SortingOne of the most common operationsDefinition:

Arrange an unordered collection of elements into a increasing or decreasing order.Two categories of sorting

internalexternal

Sorting AlgorithmsComparison based

compare-exchangeO(n log n)Noncomparison based

Uses known properties of the elementsO(n) - bucket sort etc.

Parallel Sorting IssuesInput and Output sequence storage

Where?Local to one processor or distributedComparisons

How compare elements on different nodes elements per processor

One (compare-exchange --> comm.)Multiple (compare-split --> comm.)

Compare-Exchange

1.ps

Compare-Split

2.ps

Sorting NetworksSpecialized hardware for sorting

based on comparator

xy

xymax{x,y}min{x,y}

min{x,y}max{x,y}

Sorting Network

3.ps

Parallel Sorting AlgorithmsMerge SortQuick SortBitonic Sort

Merge SortSimplest parallel sorting algorithm?Steps

Distribute the elementsEverybody sort their own sequenceMerge the listsProblem

How to merge the lists

Bitonic SortKey operation:

rearrange a bitonic sequence to orderedBitonic Sequence

sequence of elements There exists i such that is monotonically increasing and is monotonically decreasing orThere exists a cyclic shift of indicies such that the above is satisfied.

Bitonic Sequences

First it increases then decreases i = 3

Consider a cyclic shifti will equal 3

Rearranging a Bitonic SequenceLet s =

an/2 is the beginning of the decreasing seq.Let s1= Let s2=In sequence s1 there is an element bi = min{ai, an/2+i}

all elements before bi are from increasingall elements after bi are from decreasingSequence s2 has a similar pointSequences s1 and s2 are bitonic

Rearranging a Bitonic SequenceEvery element of s1 is smaller than every element of s2Thus, we have reduced the problem of rearranging a bitonic sequence of size n to rearranging two bitonic sequences of size n/2 then concatenating the sequences.

Rearranging a Bitonic Sequence

4.ps

Bitonic Merging Network

5.ps

What about unordered lists?To use the bitonic merge for n items, we must first have a bitonic sequence of n items.Two elements form a bitonic sequenceAny unsorted sequence is a concatenation of bitonic sequences of size 2Merge those into larger bitonic sequences until we end up with a bitonic sequence of size n

Mapping onto a hypercubeOne element per processorStart with the sorting network mapsEach wire represents a processorMap processors to wires to minimize the distance traveled during exchange

Bitonic Merging Network

6.ps

Bitonic SortProcedure BitonicSortfor i = 0 to d -1 for j = i downto 0 if (i + 1)st bit of iproc jth bit of iproc comp_exchange_max(j, item)else comp_exchange_min(j, item)endif endforendfor

comp_exchange_max and comp_exchange_min compare and exchange the item with the neighbor on the jth dimension

Bitonic Sort Stages

7.ps