Divide and Conquer - SJTUgao-xf/algorithm/Document/Slide03... · 2016-03-10 · Divide-and-Conquer Applications Sorting Networks Divide and Conquer∗ Xiaofeng Gao Department of Computer

Divide-and-ConquerApplications

Sorting Networks

Divide and Conquer∗

Xiaofeng Gao

Department of Computer Science and EngineeringShanghai Jiao Tong University, P.R.China

X033533-Algorithm: Analysis and Theory

∗

Special thanks is given to Prof. Yijia Chen for sharing his teaching materials.

X033533-Algorithm@SJTU Xiaofeng Gao Divide and Conquer 1/73


Sorting Networks

Outline

1 Divide-and-ConquerBasic TechniqueAn Introductory Example: MultiplicationRecurrence Relations

2 ApplicationsSearch and SortMedianMatrix Multiplication

3 Sorting NetworksComparison NetworksZero-One PrincipleConstruction of a Sorting Network



Sorting Networks

Basic TechniqueAn Introductory Example: MultiplicationRecurrence Relations

Outline






Sorting Networks


Divide-and-Conquer Strategy

The divide-and-conquer strategy solves a problemP by:

(1) BreakingP into subproblems that are themselves smallerinstances of the same type of problem.

(2) Recursively solving these subproblems.

(3) Appropriately combining their answers.



Sorting Networks


Key Works

The real work to implement Divide-and-Conquer strategy is donepiecemeal, where the key works lay in three different places:

(1) How to partition problem into subproblems.

(2) At the very tail end of the recursion, how to solve the smallestsubproblems outright.

(3) How to glue together the partial answers.



Sorting Networks


Outline






Sorting Networks


Johann C.F. Gauss

Johann Carl Friedrich Gauss1777 - 1855



Sorting Networks


Johann C.F. Gauss

Johann Carl Friedrich Gauss1777 - 1855

1+ 2+ · · ·+ 100=100· (1+ 100)

2= 5050.



Sorting Networks


Multiplication for Complex Numbers

Gauss once noticed that although the product of two complexnumbers

(a + bi)(c + di) = ac− bd + (bc + ad)i

seems to involvefour real-number multiplications, it can in fact bedone with justthree: ac, bd, and(a + b)(c + d), since

bc + ad = (a + b)(c + d) − ac− bd.



Sorting Networks


Multiplication for Complex Numbers

Gauss once noticed that although the product of two complexnumbers

(a + bi)(c + di) = ac− bd + (bc + ad)i

seems to involvefour real-number multiplications, it can in fact bedone with justthree: ac, bd, and(a + b)(c + d), since

bc + ad = (a + b)(c + d) − ac− bd.

In our big-O way of thinking, reducing the number of multiplicationsfrom four to three seems wasted ingenuity. However, this modestimprovement becomesvery significant when applied recursively.



Sorting Networks


Multiplication for Integers

Supposex andy are twon-bit integers, and assume for conveniencethatn is a power of2.



Sorting Networks




Lemma: ∀n ∈ N, ∃ n′ with n ≤ n′ ≤ 2n such that n′ is a power of 2.



Sorting Networks





As a first step toward multiplyingx andy, we split each of them intotheir left and right halves, which aren/2 bits long:



Sorting Networks






x = xL xR = 2n/2xL + xR

y = yL yR = 2n/2yL + yR.



Sorting Networks








xy = (2n/2xL + xR)(2n/2yL + yR) = 2nxLyL + 2n/2(xLyR + xRyL) + xRyR.



Sorting Networks









The additions take linear time, as do the multiplications bypowers of2 (merely left-shifts).



Sorting Networks









The additions take linear time, as do the multiplications bypowers of2 (merely left-shifts). The significant operations are the four n/2-bitmultiplications; these we can handle byfour recursive calls.



Sorting Networks


Our method for multiplyingn-bit numbers starts by making recursivecalls to multiply these four pairs ofn/2-bit numbers, and thenevaluates the preceding expression inO(n) time.



Sorting Networks



Writing T(n) for the overall running time onn-bit inputs, we gettherecurrence relation:

T(n) = 4T(n/2) + O(n)



Sorting Networks




T(n) = 4T(n/2) + O(n)

Solution: O(n2).



Sorting Networks




T(n) = 4T(n/2) + O(n)

Solution: O(n2).

By Gauss’s trick, three multiplications,xLyL, xRyR, and(xL + xR)(yL + yR), suffice, as

xLyR + xRyL = (xL + xR)(yL + yR)− xLyL − xRyR.



Sorting Networks


A divide-and-conquer algorithm for integer multiplication



Sorting Networks


A divide-and-conquer algorithm for integer multiplication

MULTIPLY (x,y)

Input: positive integersx andy, in binaryOutput: their product

1: n = max(size ofx, size ofy) rounded as a power of 2.2: if n = 1 then3: returnxy.4: end if5: xL, xR = leftmostn/2, rightmostn/2 bits ofx6: yL, yR = leftmostn/2, rightmostn/2 bits ofy7: P1 = MULTIPLY (xL, yL)8: P2 = MULTIPLY (xR, yR)9: P3 = MULTIPLY (xL + xR, yL + yR)

10: returnP1× 2n + (P3− P1− P2)× 2n/2 + P2.



Sorting Networks


The time analysis



Sorting Networks


The time analysis

The recurrence relation: T(n) = 3T(n/2) + O(n)



Sorting Networks


The time analysis


⊲ The algorithm’s recursive calls form atree structure.



Sorting Networks


The time analysis



⊲ At each successive level the subproblems gethalvedin size.



Sorting Networks


The time analysis




⊲ At the (log2 n)th level, the subproblems get down to size 1, andso the recursion ends.



Sorting Networks


The time analysis





⊲ Theheight of the tree is log2 n.



Sorting Networks


The time analysis






⊲ Thebranching factor is 3: each problem recursively producesthree smaller ones, with the result that at depthk in the tree thereare3k subproblems, each ofsizen/2k.



Sorting Networks


The time analysis







For each subproblem, a linear amount of work is done in identifyingfurther subproblems and combining their answers.



Sorting Networks


The time analysis







For each subproblem, a linear amount of work is done in identifyingfurther subproblems and combining their answers. Therefore the total

time spent at depthk in the tree is3k × O( n

2k

)

=

(

32

)k

× O(n).



Sorting Networks


The time analysis (cont’d)

3k × O( n

2k

)

=

(

32

)k

× O(n).



Sorting Networks



3k × O( n

2k

)

=

(

32

)k

× O(n).

At the very top level, whenk = 0, we needO(n).



Sorting Networks



3k × O( n

2k

)

=

(

32

)k

× O(n).


At the bottom, whenk = log2 n, it is

O(

3log2 n) = O(

nlog2 3)



Sorting Networks



3k × O( n

2k

)

=

(

32

)k

× O(n).



O(

3log2 n) = O(

nlog2 3)

Between these two endpoints, the work done increasesgeometricallyfrom O(n) to O(nlog2 3), by a factor of 3/2 per level.



Sorting Networks



3k × O( n

2k

)

=

(

32

)k

× O(n).



O(

3log2 n) = O(

nlog2 3)


The sum of any increasing geometric series is, within a constantfactor, simply the last term of the series.



Sorting Networks



3k × O( n

2k

)

=

(

32

)k

× O(n).



O(

3log2 n) = O(

nlog2 3)


The sum of any increasing geometric series is, within a constantfactor, simply the last term of the series. Therefore the overall runningtime is

O(nlog2 3) ≈ O(n1.59).



Sorting Networks



3k × O( n

2k

)

=

(

32

)k

× O(n).



O(

3log2 n) = O(

nlog2 3)


The sum of any increasing geometric series is, within a constantfactor, simply the last term of the series. Therefore the overall runningtime is

O(nlog2 3) ≈ O(n1.59).

We can do even better!X033533-Algorithm@SJTU Xiaofeng Gao Divide and Conquer 13/73


Sorting Networks


Outline






Sorting Networks


Master Theorem



Sorting Networks


Master Theorem

IfT(n) = aT(⌈n/b⌉) + O(nd)

for some constantsa > 0, b > 1, andd ≥ 0,



Sorting Networks


Master Theorem

IfT(n) = aT(⌈n/b⌉) + O(nd)

for some constantsa > 0, b > 1, andd ≥ 0, then

T(n) =

O(nd) if d > logb aO(nd logn) if d = logb aO(nlogb a) if d < logb a.



Sorting Networks


Proof of Master Theorem



Sorting Networks





Sorting Networks



Assume thatn is a power ofb. This will not influence the final boundin any important way:n is at most a multiplicative factor ofb awayfrom some power ofb.



Sorting Networks




Next, notice that the size of the subproblems decreases by a factor ofb with each level of recursion, and therefore reaches the basecaseafterlogb n levels. This is the height of the recursion tree.



Sorting Networks





The branching factor of the recursion tree isa, so thekth level of thetree is made up ofak subproblems, each ofsizen/bk.



Sorting Networks





The branching factor of the recursion tree isa, so thekth level of thetree is made up ofak subproblems, each ofsizen/bk.

The total work done at this level is

ak × O( n

bk

)d= O(nd)×

( abd

)k.



Sorting Networks





Sorting Networks



The total work done islogb n∑

k=0

(

ak × O( n

bk

)d)

=

lognb

∑

k=0

(

O(nd)×( a

bd

)k)

.



Sorting Networks




k=0

(

ak × O( n

bk

)d)

=

lognb

∑

k=0

(

O(nd)×( a

bd

)k)

.

It’s the sum of a geometric series withratio a/bd .



Sorting Networks




k=0

(

ak × O( n

bk

)d)

=

lognb

∑

k=0

(

O(nd)×( a

bd

)k)

.


⊲ The ratio is less than 1. Then the series is decreasing, and its sumis just given by its first term,O(nd).



Sorting Networks




k=0

(

ak × O( n

bk

)d)

=

lognb

∑

k=0

(

O(nd)×( a

bd

)k)

.



⊲ The ratio is greater than 1. The series is increasing and its sum isgiven by its last term,O(nlogb a):

nd( a

bd

)logb n= nd

(

alogb n

(blogb n)d

)

= alogb n = a(loga n)(logb a) = nlogb a.



Sorting Networks




k=0

(

ak × O( n

bk

)d)

=

lognb

∑

k=0

(

O(nd)×( a

bd

)k)

.




nd( a

bd

)logb n= nd

(

alogb n

(blogb n)d

)


⊲ The ratio is exactly 1. In this case allO(logn) terms of the seriesare equal toO(nd).



Sorting Networks




k=0

(

ak × O( n

bk

)d)

=

lognb

∑

k=0

(

O(nd)×( a

bd

)k)

.




nd( a

bd

)logb n= nd

(

alogb n

(blogb n)d

)


⊲ The ratio is exactly 1. In this case allO(logn) terms of the seriesare equal toO(nd).



Sorting Networks

Search and SortMedianMatrix Multiplication

Outline






Sorting Networks


BinarySearch Algorithm

BINARY SEARCH(A[1 . . . n])

Input: An arrayA[1..n] in nondecreasing order and anx.Output: j if x = A[j], 1≤ j ≤ n, and 0 otherwise.

1: low← 1; high← n; j← 0;2: while low ≤ high and j = 03: mid ← ⌊(low + high)/2⌋;4: if x = A[mid] then j← mid break;5: else if x < A[mid] then high← mid − 1;6: else low← mid + 1;7: end while8: return j



Sorting Networks


The Time Analysis

To find a keyx in A[1, · · · , n] in sorted order, we first comparex withA[n/2], and depending on the result we recurse either on the first halfof the arrayA[1, · · · , n/2− 1], or on the second halfA[n/2, · · · , n].



Sorting Networks


The Time Analysis


The recurrence function is

T(n) = T(⌈n

2

⌉)

+ O(1),



Sorting Networks


The Time Analysis


The recurrence function is

T(n) = T(⌈n

2

⌉)

+ O(1),

By Master Theorem,a = 1, b = 2, d = 0, and thus the running timeshould beO(logn).



Sorting Networks


MergeSort Algorithm



Sorting Networks


MergeSort Algorithm

MERGESORT(a[1 . . . n])

Input: an array of numbersa[1 . . . n]Output: A sorted version of this array

1: if n > 1 then2: returnMERGE(MERGESORT(a[1 . . . ⌊n/2⌋]),3: MERGESORT(a[⌊n/2⌋+ 1 . . . n⌋]),4: else returna.



Sorting Networks


MergeSort Algorithm (Cont.)

MERGE(x[1 . . . k], y[1 . . . ℓ])

Input: two sorted arraysx andyOutput: A sorted version of the union ofx andy

1: if k = 0 then returny[1 . . . ℓ]2: if ℓ = 0 then returnx[1 . . . k]3: if x[1] ≤ y[1] then4: returnx[1]MERGE(x[2 . . . k], y[1 . . . ℓ])5: else returny[1]MERGE(x[1 . . . k], y[2 . . . ℓ]).



Sorting Networks


The Time Analysis



Sorting Networks


The Time Analysis

The recurrence relation:

T(n) = 2T(n/2) + O(n);



Sorting Networks


The Time Analysis

The recurrence relation:

T(n) = 2T(n/2) + O(n);

By Master TheoremT(n) = O(n logn).



Sorting Networks


An n logn Lower Bound for Sorting



Sorting Networks



Sorting algorithms can be depicted astrees.



Sorting Networks




Thedepth of the tree – the number of comparisons on the longestpath from root to leaf, is exactly the worst-case time complexity of thealgorithm.



Sorting Networks





Consider any such tree that sorts an array ofn elements.



Sorting Networks





Consider any such tree that sorts an array ofn elements. Each of itsleaves is labeled by apermutationof 1,2, . . . , n.



Sorting Networks






every permutation must appear as the label of a leaf.



Sorting Networks







This is a binary tree withn! leaves.



Sorting Networks







This is a binary tree withn! leaves. Thus, the depth of our tree – andthe complexity of our algorithm – must be at least

log(n!) ≈ log(

√

π (2n + 1/3) · nn · e−n)

= Ω(n logn),

where we useStirling’s formula.X033533-Algorithm@SJTU Xiaofeng Gao Divide and Conquer 25/73


Sorting Networks


A Sorting Permutation Tree

An example sorts fora1, a2, a3:



Sorting Networks


Outline






Sorting Networks


Median



Sorting Networks


Median

Themedian of a list of numbers is its 50th percentile: half thenumbers are bigger than it, and half are smaller.



Sorting Networks


Median


If the list haseven length, there are two choices for what the middleelement could be, in which case we pick the smaller of the two,say.



Sorting Networks


Median



The purpose of the median is to summarize a set of numbers by asingle, typical value.



Sorting Networks


Median




Computing the median ofn numbers is easy: just sort them. Thedrawback is that this takesO(n logn) time, whereas we would ideallylike somethinglinear.



Sorting Networks


Median




Computing the median ofn numbers is easy: just sort them. Thedrawback is that this takesO(n logn) time, whereas we would ideallylike somethinglinear.

We have reason to be hopeful, because sorting is doing far more workthan we really need – we just want the middle element and don’tcareabout the relative ordering of the rest of them.



Sorting Networks


Selection



Sorting Networks


Selection

Input: A list of numbersS; an integerK.Output: Thekth smallest element ofS.



Sorting Networks


A randomized divide-and-conquer algorithm for selection



Sorting Networks



For any numberv, imagine splitting listS into three categories:



Sorting Networks




elements smaller thanv, i.e.,SL;



Sorting Networks





those equal tov, i.e.,Sv (there might be duplicates);



Sorting Networks






and those greater thanv, i.e.,SR respectively.



Sorting Networks






and those greater thanv, i.e.,SR respectively.

selection(S, k) =

selection(SL, k) if k ≤ |SL|

v if |SL| < k ≤ |SL|+ |Sv|

selection(SR, k − |SL| − |Sv|) if k > |SL|+ |Sv|.



Sorting Networks


How to choosev?



Sorting Networks


How to choosev?

It should be picked quickly, and it should shrink the arraysubstantially, the ideal situation being

|SL|, |SR| ≈|S|2.



Sorting Networks


How to choosev?


|SL|, |SR| ≈|S|2.

If we could always guarantee this situation, we would get a runningtime of

T(n) = T(n/2) + O(n) = O(n).



Sorting Networks


How to choosev?


|SL|, |SR| ≈|S|2.


T(n) = T(n/2) + O(n) = O(n).

But this requires pickingv to be the median, which is our ultimategoal!



Sorting Networks


How to choosev?


|SL|, |SR| ≈|S|2.


T(n) = T(n/2) + O(n) = O(n).

But this requires pickingv to be the median, which is our ultimategoal!

Instead, we follow a much simpler alternative: we pickv randomlyfrom S.



Sorting Networks


How to choosev? (cont’d)



Sorting Networks



Worst-case scenario would force our selection algorithm to perform

n + (n− 1) + (n− 2) + · · ·+n2= Θ(n2)



Sorting Networks




n + (n− 1) + (n− 2) + · · ·+n2= Θ(n2)

Best-case scenario:O(n).



Sorting Networks




n + (n− 1) + (n− 2) + · · ·+n2= Θ(n2)


Where, in this spectrum fromO(n) to Θ(n2), does the averagerunning time lie?



Sorting Networks




n + (n− 1) + (n− 2) + · · ·+n2= Θ(n2)


Where, in this spectrum fromO(n) to Θ(n2), does the averagerunning time lie? Fortunately, it lies very close to the best-case time.



Sorting Networks


The efficiency analysis



Sorting Networks



v is goodif it lies within the 25th to 75th percentile of the array thatitis chosen from.



Sorting Networks




A randomly chosenv has a50% chance of being good,



Sorting Networks





Lemma: On average afair coin needs to be tossed two times before a“heads” is seen.



Sorting Networks






Proof: E := expected number of tosses before head is seen.



Sorting Networks






Proof: E := expected number of tosses before head is seen.We need at least one toss, and it’s heads, we’re done.



Sorting Networks






Proof: E := expected number of tosses before head is seen.We need at least one toss, and it’s heads, we’re done.If it’s tail (with probability 1/2), we need to repeat.



Sorting Networks






Proof: E := expected number of tosses before head is seen.We need at least one toss, and it’s heads, we’re done.If it’s tail (with probability 1/2), we need to repeat. Hence

E = 1+12

E,



Sorting Networks






Proof: E := expected number of tosses before head is seen.We need at least one toss, and it’s heads, we’re done.If it’s tail (with probability 1/2), we need to repeat. Hence

E = 1+12

E,

whose solution isE = 2



Sorting Networks


The efficiency analysis (cont’d)



Sorting Networks


The efficiency analysis (cont’d)

Let T(n) be theexpected running time on an array of sizen, we get

T(n) ≤ T(3n/4) + O(n) = O(n).



Sorting Networks


Outline






Sorting Networks


The product of twon× n matricesX andY is a thirdn× n matrixZ = XY,



Sorting Networks


The product of twon× n matricesX andY is a thirdn× n matrixZ = XY, with (i, j)th entry

Zij =

n∑

k=1

XikYkj



Sorting Networks



Zij =

n∑

k=1

XikYkj

That is,Zij is thedot productof the ith row of X with the jth columnof Y.



Sorting Networks



Zij =

n∑

k=1

XikYkj


In general,XY is not the same asYX; matrix multiplication is notcommutative.



Sorting Networks



Zij =

n∑

k=1

XikYkj


In general,XY is not the same asYX; matrix multiplication is notcommutative.

The preceding formula implies anO(n3) algorithm for matrixmultiplication.



Sorting Networks


Volker Strassen

Volker Strassen (1936 – )



Sorting Networks


Volker Strassen

Volker Strassen (1936 – )

In 1969, the German mathematicianVolker Strassen announced asurprisingO(n2.81) algorithm.



Sorting Networks


Divide and conquer



Sorting Networks


Divide and conquer

X =

[

A BC D

]

, Y =

[

E FG H

]



Sorting Networks


Divide and conquer

X =

[

A BC D

]

, Y =

[

E FG H

]

Then

XY =

[

A BC D

] [

E FG H

]

=

[

AE + BG AF + BHCE + DG CF + DH

]



Sorting Networks


Divide and conquer

X =

[

A BC D

]

, Y =

[

E FG H

]

Then

XY =

[

A BC D

] [

E FG H

]

=

[


]

To compute the size-n productXY, recursively compute eight size-n/2productsAE, BG, AF, BH, CE, DG, CF, DH and then do someO(n2)-time addition.



Sorting Networks


Divide and conquer

X =

[

A BC D

]

, Y =

[

E FG H

]

Then

XY =

[

A BC D

] [

E FG H

]

=

[


]

To compute the size-n productXY, recursively compute eight size-n/2productsAE, BG, AF, BH, CE, DG, CF, DH and then do someO(n2)-time addition.

The recurrence is

T(n) = 8T(n/2) + O(n2)

with solutionO(n3).X033533-Algorithm@SJTU Xiaofeng Gao Divide and Conquer 38/73


Sorting Networks


Strassen’s trick



Sorting Networks


Strassen’s trick

XY =

[

P5 + P4− P2 + P6 P1 + P2

P3 + P4 P1 + P5 = P3− P7

]

where

P1 = A(F − H) P5 = (A + D)(E + H)P2 = (A + B)H P6 = (B− D)(G + H)P3 = (C + D)E P7 = (A− C)(E + F)P4 = D(G− E)



Sorting Networks


Strassen’s trick

XY =

[

P5 + P4− P2 + P6 P1 + P2

P3 + P4 P1 + P5 = P3− P7

]

where

P1 = A(F − H) P5 = (A + D)(E + H)P2 = (A + B)H P6 = (B− D)(G + H)P3 = (C + D)E P7 = (A− C)(E + F)P4 = D(G− E)

The recurrence is

T(n) = 7T(n/2) + O(n2)

with solutionO(nlog2 7) ≈ O(n2.81).



Sorting Networks

Comparison NetworksZero-One PrincipleConstruction of a Sorting Network

Outline






Sorting Networks


Introduction

Previously, we examined sorting algorithms forserial computers(random-access machines, RAM’s) that allow only one operation tobe executed at a time.



Sorting Networks


Introduction


In this section, we investigate sorting algorithms based onacomparison-networkmodel of computation, in which manycomparison operations can be performed simultaneously.



Sorting Networks


Introduction


In this section, we investigate sorting algorithms based onacomparison-networkmodel of computation, in which manycomparison operations can be performed simultaneously.

Comparison NetworkVS RAM’s

⊲ Comparison network can only perform comparisons. (Cannotdeal with Counting Sort etc)

⊲ Comparison network run parallel operations.



Sorting Networks


Definition

A comparison network is composed solely ofwiresandcomparators.

A comparator is a device with two inputs,x andy, and two outputs,x′

andy′, that performs the following function:

x′ = minx, y, y′ = maxx, y.

Each comparator operates inO(1) time.



Sorting Networks


Wire

A wire transmits a value from place to place.

⊲ Connect the output of one comparator to the input of another;

⊲ The network input wires or output wires.



Sorting Networks


Wire




Assume a comparison networks containsn input wires< a1, a2, · · · , an >, through which the values to be sorted enter thenetwork, andn output wires< b1, b2, · · · , bn > , which produce theresults computed by the network.



Sorting Networks


Wire




Assume a comparison networks containsn input wires< a1, a2, · · · , an >, through which the values to be sorted enter thenetwork, andn output wires< b1, b2, · · · , bn > , which produce theresults computed by the network.

Draw a comparison network onn inputs as a collection ofn horizontallines with comparators stretched vertically.



Sorting Networks


An Example



Sorting Networks


An Example



Sorting Networks


An Example

Data move from left toright.



Sorting Networks


An Example


Interconnections must beacyclic.



Sorting Networks


An Example


Interconnections must beacyclic.

If a comparator has twoinput wires withdepthsdx

anddy, then its output wirehave depthmaxdx, dy+ 1. (Initiallyis 0)



Sorting Networks


Sorting Network

A sorting networkis a comparison network for which the outputsequence is monotonically increasing (b1 ≤ b2 ≤ · · · ≤ bn) for everyinput sequence.



Sorting Networks


Sorting Network

A sorting networkis a comparison network for which the outputsequence is monotonically increasing (b1 ≤ b2 ≤ · · · ≤ bn) for everyinput sequence.

We are discussing a family of comparison networks accordingto theinput size.



Sorting Networks


Outline






Sorting Networks


Zero-One Principle

Thezero-one principlesays that if a sorting network works correctlywhen each input is drawn from the set 0,1, then it works correctlyon arbitrary input numbers (e.g., integers, reals, or any linearlyordered set).



Sorting Networks


Zero-One Principle


This principle allow us to focus on the operations for input sequencesconsisting solely of 0’s and 1’s.



Sorting Networks


Zero-One Principle



Once we have constructed a sorting network and proved that itcansort all zero-one sequence, we shall appeal to 0-1 principleto proveits correctness on arbitrary values.



Sorting Networks


Zero-One Principle



Once we have constructed a sorting network and proved that itcansort all zero-one sequence, we shall appeal to 0-1 principleto proveits correctness on arbitrary values.

Note: the proof of 0-1 principle relies on the notion of monotonicallyincreasing function.



Sorting Networks


Lemma

Lemma: If a comparison network transforms the input sequencea =< a1, a2, · · · , an > into the output sequenceb =< b1, b2, · · · , bn >, then for any monotonically increasingfunction f , the network transforms the input sequencef (a) =< f (a1), f (a2), · · · , f (an) > into the output sequencef (b) =< f (b1), f (b2), · · · , f (bn) >.



Sorting Networks


Proof (by Induction)

Consider a comparator whose input values arex andy. The upperoutput is minx, y while the lower output is maxx, y.



Sorting Networks




If we apply f (x) andf (y) as the inputs, the operation of thecomparator yields the value of upper minf (x), f (y) and lowermaxf (x), f (y).



Sorting Networks




If we apply f (x) andf (y) as the inputs, the operation of thecomparator yields the value of upper minf (x), f (y) and lowermaxf (x), f (y).

Sincef is monotonically increasing,x ≤ y implies f (x) ≤ f (y).Thus we have

minf (x), f (y) = f (minx, y),

maxf (x), f (y) = f (maxx, y),

which completes the proof of the claim as the base case.



Sorting Networks


Proof (Continued)

We use induction on the depth of each wire in a general comparisonnetwork to prove a stronger result than the statement of the lemma:



Sorting Networks


Proof (Continued)


If a wire assumes the valueai when the input sequence isa, then itassumes the valuef (ai) when the input sequence isf (a).



Sorting Networks


Proof (Continued)


If a wire assumes the valueai when the input sequence isa, then itassumes the valuef (ai) when the input sequence isf (a).

Since the output wires are included in this statement, proving it willprove the lemma.



Sorting Networks


Proof (Continued)

Basis: A wire at depth 0 is an input wireai. Whenf (a) is applied tothe network, the input wire carriesf (ai).



Sorting Networks


Proof (Continued)


Induction: A wire at depthd ≥ 1 is the output of a comparator atdepthd, and the input wires to this comparator are at a depth strictlyless thand. By inductive hypothesis, if the input wires carry valuesai

andaj with input sequencea, then they carryf (ai) andf (aj) withinput sequencef (a).



Sorting Networks


Proof (Continued)


Induction: A wire at depthd ≥ 1 is the output of a comparator atdepthd, and the input wires to this comparator are at a depth strictlyless thand. By inductive hypothesis, if the input wires carry valuesai

andaj with input sequencea, then they carryf (ai) andf (aj) withinput sequencef (a).

By previous claim, the output wires of this comparator then carryf (minai, aj) andf (maxai, aj). Since the carry minai, aj andmaxai, aj when the input sequence isa, the lemma is proved.



Sorting Networks


An Example



Sorting Networks


Zero-One Principle

Theorem: If a comparison network withn inputs sorts all 2n possiblesequences of 0’s and 1’s correctly, then it sorts all sequences ofarbitrary numbers correctly.



Sorting Networks


Zero-One Principle

Theorem: If a comparison network withn inputs sorts all 2n possiblesequences of 0’s and 1’s correctly, then it sorts all sequences ofarbitrary numbers correctly.

Proof: (Contradiction)Suppose there exists a sequence of arbitrarynumbers that the network does not correctly sort. That is , there existsan input sequence< a1, a2, · · · , an > containing elementsai andaj,such thatai < aj, but the network placesaj beforeai in the outputsequence.



Sorting Networks


Proof (Continued)

Define a monotonically increasing functionf as

f (x) =

0 if x ≤ ai,1 if x > ai.



Sorting Networks


Proof (Continued)


f (x) =


Since the network placesaj beforeai, by previous lemma, it will placef (aj) beforef (ai) in the output sequence when< f (a1), f (a2), · · · , f (an) > is input.



Sorting Networks


Proof (Continued)


f (x) =


Since the network placesaj beforeai, by previous lemma, it will placef (aj) beforef (ai) in the output sequence when< f (a1), f (a2), · · · , f (an) > is input.

However, sincef (aj) = 1 andf (ai) = 0, the network fails to sort thezero-one sequence< f (a1), f (a2), · · · , f (an) > correctly.

A contradiction!



Sorting Networks


Outline






Sorting Networks


Construction of a Sorting Network

To construct a sorting network, we need three steps:



Sorting Networks




Step 1: Construct a Bitonic Sorter⇒ to sort bitonic sequence.



Sorting Networks





Step 2: Construct a Merger⇒ to merge two sorted sequence.



Sorting Networks





Step 2: Construct a Merger⇒ to merge two sorted sequence.

Step 3: Construct a Sorter⇒ to sort an arbitrary sequence.



Sorting Networks


Step 1: Construct a Bitonic Sorter

We start fromBitonic sequence.



Sorting Networks




A Bitonic Sequence is a sequence that monotonically increases andthen monotonically decreases, or can be circularly shiftedto becomemonotonically increasing and then monotonically decreasing.



Sorting Networks





Examples: <1,4,6,8,3,2>, <6,9,4,2,3,5>, <9,8,3,2,4,6>



Sorting Networks





Examples: <1,4,6,8,3,2>, <6,9,4,2,3,5>, <9,8,3,2,4,6>



Sorting Networks





Examples: <1,4,6,8,3,2>, <6,9,4,2,3,5>, <9,8,3,2,4,6>



Sorting Networks


Half-Cleaner

A half-cleaneris a comparison network of depth 1, in which input line

i is compared with linei +n2

for i = 1,2, · · · ,n2

(assumen is even).



Sorting Networks


Half-Cleaner

A half-cleaneris a comparison network of depth 1, in which input line

i is compared with linei +n2

for i = 1,2, · · · ,n2

(assumen is even).



Sorting Networks


Half-Cleaner (2)

When a bitonic sequence of 0’s and 1’s is applied as input to ahalf-cleaner, it produces an output sequence with smaller values in thetop-half, and larger values in the bottom-half. Both halvesare bitonic.



Sorting Networks


Half-Cleaner (2)

When a bitonic sequence of 0’s and 1’s is applied as input to ahalf-cleaner, it produces an output sequence with smaller values in thetop-half, and larger values in the bottom-half. Both halvesare bitonic.

In fact, at least one of the halves isclean, say, consists of either all 0’sor all 1’s.



Sorting Networks


Half-Cleaner Lemma

Lemma: If the input to a half-cleaner is a bitonic sequence of 0’s and1’s, then the output satisfies the following properties:

⊲ both the top half and the bottom half are bitonic;

⊲ every element in the top half is at least as small as every elementof the bottom half, and at least one half is clean.



Sorting Networks


Proof

The comparison networkHALF-CLEANER[n] compares inputsi andi + n/2 for i = 1,2, · · · , n/2. Without loss of generality, suppose thatthe input is of the form 00· · · 011· · · 100· · · 0 (the situation of11· · · 100· · · 011· · · 1 are symmetric).



Sorting Networks


Proof

The comparison networkHALF-CLEANER[n] compares inputsi andi + n/2 for i = 1,2, · · · , n/2. Without loss of generality, suppose thatthe input is of the form 00· · · 011· · · 100· · · 0 (the situation of11· · · 100· · · 011· · · 1 are symmetric).

There are three possible cases depending upon the block ofconsecutive 0’s and 1’s in which the midpointn/2 falls, and one ofthese cases is further split into two cases. In each of the cases, thelemma holds.



Sorting Networks


Case Analysis



Sorting Networks


Case Analysis (Cont.)



Sorting Networks


The Bitonic Sorter

By recursively combing half-cleaners, we can build abitonic sorter,which is a network that sorts bitonic sequences.

Half-

Cleaner[n]

Bitonic-

Sorter[n/2]

Bitonic-

Sorter[n/2]

Bitonic-Sorter[n]



Sorting Networks


An Example ofn = 8



Sorting Networks


DepthD(n)

The depthD(n) of BITONIC-SORTER[n] is given by the recurrence

D(n) =

0 if n = 1;D(n/2) + 1 if n = 2k andk ≥ 1,



Sorting Networks


DepthD(n)


D(n) =

0 if n = 1;D(n/2) + 1 if n = 2k andk ≥ 1,

Easy to see,D(n) = ln n.



Sorting Networks


DepthD(n)


D(n) =

0 if n = 1;D(n/2) + 1 if n = 2k andk ≥ 1,


Thus, a zero-one bitonic sequence can be sorted byBITONIC-SORTER[n], which has a depth of lnn.



Sorting Networks


DepthD(n)


D(n) =

0 if n = 1;D(n/2) + 1 if n = 2k andk ≥ 1,


Thus, a zero-one bitonic sequence can be sorted byBITONIC-SORTER[n], which has a depth of lnn.

By zero-one principle, any bitonic sequence of arbitrary numbers canbe sorted by this network.



Sorting Networks


Step 2: Construct a Merger

Merging Network can merge two sorted input sequences into onesorted output sequence.



Sorting Networks




Given two sorted sequences, if we reverse the order of the secondsequence and then concatenate the two sequences, the resultingsequence is bitonic.



Sorting Networks




Given two sorted sequences, if we reverse the order of the secondsequence and then concatenate the two sequences, the resultingsequence is bitonic.

For instance:

X = 00000111;

Y = 00001111;

YR = 11110000;

X YR = 0000011111110000.



Sorting Networks


MERGER[n]

Given two sorted sequences< a1, a2, · · · , an/2 > and< an/2+1, an/2+2, · · · , an >, we want the effect of bitonically sortingthe sequence< a1, a2, · · · , an/2, an, an−1, · · · , an/2+1 >.



Sorting Networks


MERGER[n]


Since the first half-cleaner ofBITONIC-SORTER[n] compares inputsiwith n/2+ i, for i = 1,2, · · · , n/2, we make the first stage of themerging network compare inputsi andn− i + 1.



Sorting Networks


MERGER[n]


Since the first half-cleaner ofBITONIC-SORTER[n] compares inputsiwith n/2+ i, for i = 1,2, · · · , n/2, we make the first stage of themerging network compare inputsi andn− i + 1.

The order of the outputs from the bottom of the first stage ofMERGER[n] are reversed compared with the order of outputs from anordinary half-cleaner.



Sorting Networks


Comparison betweenMERGER[n] andHALF-CLEANER[n]



Sorting Networks


MERGER[n]

Bitonic-

Sorter[n/2]

Bitonic-

Sorter[n/2]



Sorting Networks


Step 3: Construct a Sorter

The sorting networkSORTER[n] are composed by two copies ofSORTER[n/2] and oneMERGER[n] recursively.

Merger[n]

Sorter[n/2]

Sorter[n/2]



Sorting Networks


An Example withn = 8

Sorter[n/2]

Merger[n]



Sorting Networks


Performance Analysis

The depthD(n) of SORTER[n] is the depthD(n/2) of SORTER[n/2]plus the depth lnn of MERGER[n].



Sorting Networks




Consequently, the depth ofSORTER[n] is given by the recurrence

D(n) =

0 if n = 1;D(n/2) + ln n if k ≥ 1.



Sorting Networks




Consequently, the depth ofSORTER[n] is given by the recurrence

D(n) =

0 if n = 1;D(n/2) + ln n if k ≥ 1.

By Master’s Theorem, the solution isD(n) = Θ(ln2 n). Thus we cansortn numbers in parallel inO(ln2 n) time.


Documents

Divide and Conquer - SJTUgao-xf/algorithm/Document/Slide03... · 2016-03-10 · Divide-and-Conquer Applications Sorting Networks Divide and Conquer∗ Xiaofeng Gao Department of Computer