Divide-and-Conquer

Outline

Introduction Merge Sort Quick Sort Closest pair of points Large integer multiplication

Steps Divide the problem into a

number of subproblems. Conquer the subproblems by

solving them recursively. If the subproblem sizes are small

enough, however, just solve the subproblems in a straightforward manner.

Combine the solutions to the subproblems into the solution for the original problem.

Merge Sort

Steps in Merge Sort

Divide Divide the sequence to be sorted

into two halves. Conquer

Sort the two subsequences recursively using merge sort.

Combine Merge the two sorted subsequences

to produce the sorted answer.

Merge Sort: Example

36 86 45 12 84 77 51 64

36 86 45 12 84 77 51 64

36 86 45 12 84 77 51 64

12 45 77 84 51 64

36 8612 45 77 8451 64

36 8612 45 77 8451 64

divide

dividedivide

divide

36 86conquer

36 86combine

divide

45 12 conquer

combine

conquer

combine

84 77 51 64

Merge AlgorithmMERGE(A, p, q, r)

► initialize left and right arrays

n1 = q − p + 1

n2 = r − q

create array L[1 . . n1 + 1]

create array R[1 . . n2 + 1]

for i = 1 to n1

do L[i ] = A[p + i − 1]

for j = 1 to n2

do R[ j ] = A[q + j ]

L[n1 + 1]=∞

R[n2 + 1]=∞

i = 1

j = 1

► merge two arraysfor k = p to r

do if L[i ] ≤ R[ j ]

then A[k] = L[i ]

i = i + 1

else A[k] = R[ j ]

j = j + 1

Merge: ComplexityMERGE(A, p, q, r)

► initialize left and right arrays

n1 = q − p + 1

n2 = r − q

create array L[1 . . n1 + 1]

create array R[1 . . n2 + 1]

for i = 1 to n1

do L[i ] = A[p + i − 1]

for j = 1 to n2

do R[ j ] = A[q + j ]

L[n1 + 1]=∞

R[n2 + 1]=∞

i = 1

j = 1

► merge two arraysfor k = p to r

do if L[i ] ≤ R[ j ]

then A[k] = L[i ]

i = i + 1

else A[k] = R[ j ]

j = j + 1

(n) (1)

(n)

(n)

Merge Sort Algorithm

MERGE-SORT(A, p, r)

if p < r

then q = (p + r)/2MERGE-SORT(A, p, q)

MERGE-SORT(A, q + 1, r)

MERGE(A, p, q, r)

Merge Sort: Complexity

MERGE-SORT(A, p, r)if p < r

then q = (p + r)/2MERGE-SORT(A, p, q)MERGE-SORT(A, q + 1, r)MERGE(A, p, q, r)

If n1, T(n) = 2 T(n/2) + (n)If n=1, T(n) = (1)T(n) = (n lg n)

T(n/2) T(n/2)

(n)

(1)

Master TheoremLet a ≥ 1 and b > 1 be constants,

T(n) = aT(n/b) + f(n) be defined on the nonnegative integers, where f(n) = (nd)

Then T (n) can be bounded asymptotically as follows.

If a < bd, then T(n) = (nd). If a = bd, then T(n) = (nd lg n). If a > bd, then T(n) = (n logba ).

Solving Recurrence Equation

T(n) = 2 T(n/2) + (n) when n > 1 T(n) = (1) when n = 1

That is, T(n) = (n lg n)

… cn cn cn cn cn cn cn cn

… 2c 2c 2c2c2c

… 4c 4c

cn/2 cn/2

cn

lg n

Quicksort

Steps in Quicksort Divide

Partition (rearrange) the array A[p . . r] into two subarrays A[p . . q −1] and A[q +1 . . r] such that

each element of A[p . . q −1] A[q], A[q] each element of A[q + 1 . . r]

Compute the index q. Conquer

Sort A[p . . q−1] and A[q +1 . . r] by recursive calls.

Combine Since the subarrays are sorted in place,

nothing else need to be done.

Quicksort: Example

36 5572 61 77 4451 64

51 7236 44 77 6155 64

51 7236 44 77 6155 6444 7736 51 61 7255 6444 7736 51 61 7255 64

44 7736 51 64 7255 61

44 7736 51 64 7255 61

44 7736 51 64 7255 61

Quicksort Algorithm

QUICKSORT(A, p, r)

if p < r

then q = PARTITION(A, p, r)

QUICKSORT(A, p, q − 1)

QUICKSORT(A, q + 1, r)

Initial call:

QUICKSORT(A, 1, length[A]).

Partition: Example

36 5572 61 77 4451 64

36 5572 61 77 4451 64

72 5536 61 77 4451 6472 5536 61 77 4451 6451 5536 61 77 4472 64

51 5536 61 77 4472 64

51 5536 61 77 4472 64

51 5536 44 77 6172 64

51 7236 44 77 6155 64

Partition Algorithm

PARTITION(A, p, r)

x = A[r]

i = p − 1

for j = p to r − 1

do if A[ j ] ≤ x

then i = i + 1

exchange A[i] and A[ j]

exchange A[i + 1] and A[r]

return i + 1

PARTITION(A, p, r)

x = A[r]

i = p − 1

for j = p to r − 1

do if A[ j ] ≤ x

then i = i + 1

exchange A[i] and A[ j]

exchange A[i + 1] and A[r]

return i + 1

Partition: Complexity

O(1)

O(1)

O(n)

O(1)

Quicksort: Complexity

QUICKSORT(A, p, r)

if p < r

then q = PARTITION(A, p, r)

QUICKSORT(A, p, q − 1)

QUICKSORT(A, q + 1, r)

O(n)

T(n/?)

T(n/?)

Quicksort: Analysis

Worst case:

T(n) = T(n-1) + T(0) + (n)

= T(n-1) + (n)

T(n) (n2)

Best case:

T(n) = 2 T(n/2) + (n)

T(n) (n lg n)

Average-case Analysis

T(n) (n lg n)

Closest Pairs of Points

Problem

Given a set P of points in 2-dimensional space, find a closest pair of points.

Distance between two points are measured by Euclidean distance --(x2+y2)

Brute force method takes O (n2).

Closest Pair of Points: Example

divideagain

conquer

combine

Steps in Finding Closest Pair of Points Divide

Vertically divide P into PL and PR. PL is to the left and PR is to the right of the dividing line.

Conquer Recursively find closest pair of points among

PL and PR. dL and dR are the closest-pair distances

returned for PL and PR, and d = min(dL, dR). Combine

The closest pair is either the pair with distance d found by one of the recursive calls, or it is a pair of points with one point in PL and the other in PR.

Find a pair whose distance < d if there exists.

P

Closest point on the other side

PRPL

S

d d

Only 8 points need to be considered

P

Why 8 Points

PRPL

S

d/2

d/2

d/2

d/2

d/2

d/2

d d

Combine

Let PRY be the array of points in PR sorted by the value of Y.

For each point s in the d-width strip of PL, find a point t in PR such that dist(s, t) < d. To find t, look in the array PRY for 8

such points. If there is such a point, let d be the

dist(s, t).

Closest Pair of Points: Algorithm

Let px and py be the points in P, sorted by the value of x and y, respectively.

ClosestPair(px, py)if (|px|<=3) thenreturn(BruteForceCP(px))Using px, find the line L which vertically divides P into 2 halves.Using the dividing line L, split px into pxL and pxR. Using the dividing line L, split py into pyL and pyR.dL = ClosestPair(pxL, pyL)dR = ClosestPair(pxR, pyR)d = min(dL, dR)return ( combineCP(pyL, pyR, d) )

Closest Pair of Points: ComplexityLet px and py be the points in P, sorted by the value of x

and y, respectively.ClosestPair(px, py)

if (|px|<=3) thenreturn(BruteForceCP(px))Using px, find the line L which vertically divides P into 2 halves.Using the dividing line L, split px into pxL and pxR. Using the dividing line L, split py into pyL and pyR.dL = ClosestPair(pxL, pyL)dR = ClosestPair(pxR, pyR)d = min(dL, dR)return ( combineCP(pyL, pyR, d) )

O(1)

O(n)

2T(n/2)

O(1)

O(1)

O(n)

Solving the recurrence equation

T(n) = 2 T(n/2) + O(n)

T(n) Ο(n log n)

Large Integer Multiplication

Dividing Multiplication

x = (x1*2n/2 + x0) y = (y1*2n/2 + y0)

x y = (x1*2n/2 + x0) (y1*2n/2 + y0)

= x1 y1*2n + (x0 y1+ x1 y0)*2n/2 + x0 y0

(x1+ x0) ( y1+ y0) = x1 y1+ x0 y1+ x1 y0 + x0 y0

= (x0 y1+x1 y0)+(x1 y1+ x0 y0)

(x0 y1+x1 y0) = (x1+ x0)( y1+ y0) - (x1 y1+ x0 y0)

Algorithm

recMul(x, y)

n = maxbit(x, y)

if (n<k) then return (x*y)

x1 = x div 2n/2 x0 = x mod 2n/2

y1 = y div 2n/2 y0 = y mod 2n/2

p = recMul(x1+x0, y1+y0)

q = recMul(x1, y1)

r = recMul(x0, y0)

return(q*2n+(p-q-r)*2n/2+r)

Complexity

recMul(x, y)

n = maxbit(x, y)

if (n<k) then return (x*y)

x1 = x div 2n/2 x0 = x mod 2n/2

y1 = y div 2n/2 y0 = y mod 2n/2

p = recMul(x1+x0, y1+y0)

q = recMul(x1, y1)

r = recMul(x0, y0)

return(q*2n+(p-q-r)*2n/2+r)

O(1)

3T(n/2)

O(1)

Solving the recurrence equation

T(n) = 3 T(n/2) + O(n)

T(n) Ο(n log2 3)

T(n) Ο(n 1.59)