Multi-threaded Algorithm 3
Michael Tsai2011/6/17
2
Multithreaded matrix multiplicationP-SQUARE-MATRIX-MULTIPLY(A,B)n=A.rowslet C be a new n x n matrixparallel for i=1 to n
parallel for j=1 to n
for k=1 to n
return C
๐ 1(๐)=ฮ (๐3 )
๐ โ (๐)=ฮ ( log n )+ฮ (log๐)+ฮ (๐)=ฮ(๐)
๐ 1 (๐)๐ โ (๐ )
=ฮ (๐3 )ฮ (๐)
=ฮ (๐2)
3
Divide-and-conquer Multithreaded Algorithm for Matrix Multiplication ( ๅๅ้ท )
C T
4
P-MATRIX-MULTIPLY-RECURSIVE(C,A,B)n=A.rowsif n==1
else let T be a new n x n matrixpartition A,B,C, and T into n/2 x n/2
submatrices (spawn P-MATRIX-MULTIPLY-RECURSIVE( spawn P-MATRIX-MULTIPLY-RECURSIVE(spawn P-MATRIX-MULTIPLY-RECURSIVE(spawn P-MATRIX-MULTIPLY-RECURSIVE(spawn P-MATRIX-MULTIPLY-RECURSIVE(spawn P-MATRIX-MULTIPLY-RECURSIVE(spawn P-MATRIX-MULTIPLY-RECURSIVE(P-MATRIX-MULTIPLY-RECURSIVE(syncparallel for i=1 to n
parallel for j=1 to n
๐ 1 (๐)=8๐ 1(๐2 )+ฮ (๐2 )=ฮ (๐3 )
๐โ (๐)=๐โ(๐2 )+ฮ ( log๐ )+ฮ (log๐)=ฮ ( log2๐)
๐ 1 (๐)๐โ (๐ )
=ฮ (๐3 )ฮ ( log2๐)
=ฮ( ๐3
log2๐ )
5
How about Strassenโs method? Reading assignment: p.795-796.
Parallelism: , slightly less than the original recursive version!
6
Multithreaded Merge SortMerge-Sortโ(A,p,r)if p<r
q=spawn MERGE-SORTโ(A,p,q)MERGE-SORTโ(A,q+1,r)syncMERGE(A,p,q,r) ฮ(๐)
๐๐1โฒ (๐ )=2๐๐1
โฒ (๐2 )+ฮ (๐ )=ฮ (๐ log๐)
๐๐โโฒ (๐ )=๐๐โ
โฒ (๐2 )+ฮ (๐)=ฮ (๐ )
Parallelism=
Merge() is the bottleneck!
A: Array to be sortedp and r: Start and end index of the range to be sorted
7
1. ๆๅบ็ไธญไฝๆธ x2. ๆพๅบไธญ้ๅไฝ็ฝฎไฝฟๅพ้ฝ < x, ้ฝ x
ๆ Merge ไน็จ Divide & Conquer ไพ่งฃ !( ๆนไพฟไบค็ตฆไธๅ็ thread ๅปๅ )
3. Copy x ๅฐๆฐๅฐๆน ( ๆ นๆ x ๅ
4. ้ๅ่บซๅป merge ๅ , ๅ ๅๅ ฉๅๅๆฎต
Base case: ๅ้ฝๆฏ็ฉบ็
8
Multithreaded MergeP-MERGE(T,,, ,,A,)
if exchange exchange exchange
if return
else
BINARY-SEARCH(T[],T, ,)
A[]=T[]spawn P-MERGE(T,,, ,,A,)P-MERGE(T,,, ,,A,)sync
T: Array to be mergedA: Array to save the merged result,, ,: Start and end index of the range to be merged: The index of the median in A
9
,
ๆ็ณ็็ๆณไธ , x ๆฏๆๆ็้ฝๅคง ่ฆ merge ๅๅ ็ด
10
Multithreaded MergeP-MERGE(T,,, ,,A,)
if exchange exchange exchange
if return
else
BINARY-SEARCH(T[],T, ,)
A[]=T[]spawn P-MERGE(T,,, ,,A,)P-MERGE(T,,, ,,A,)sync
๐ ๐โ (๐ )=๐ ๐โ( 3๐4 )+ฮ (log๐)
ฮ (log๐)
, ๅ ็บ่ณๅฐ่ฆ copy n elements
, ่ฆ่ชฒๆฌ p.802
โ๐๐ 1 (๐)=ฮ (๐)
11
P-MERGE-SORT(A,p,r,B,s)n=r-p+1if n==1
B[s]=A[p]else let T[1..n] be a new array
spawn P-MERGE-SORT(A,p,q,T,1)P-MERGE-SORT(A,q+1,r,T,qโ+1)syncP-MERGE(T,1,qโ,qโ+1,n,B,s)
A: Array to be mergedp,r: Index of the range to be sortedB: Array to save the result
๐๐ ๐1 (๐)=2๐๐๐1(๐2 )+๐ ๐1 (๐)=2๐๐ ๐1(๐2 )+ฮ (๐)=ฮ (๐ log๐)
๐๐ ๐โ (๐)=๐๐๐โ(๐2 )+๐๐โ (๐)=๐๐๐โ(๐2 )+ฮ (log 2๐)=ฮ ( log3๐ )
Parallelism = Much better now!
12
ๆ้บผ่ชๅทฑๅญธๆผ็ฎๆณ ้ๅธธ้่ฆ ้ๅ ่ชฒๆฒ่พฆๆณๆๆๆโ้่ฆโ็ๆผ็ฎๆณ้ฝๆๅฎ ไฝๆฏๅธๆ้็จไธญไฝ ๅทฒ็ถๅปบ็ซไบ่ชๅทฑๅญธ็ฟๆผ็ฎๆณ็่ฝๅ
ๆบๅ็ ็ฉถๆ่่ฉฆ็ๆๅๅฏ่ฝๆ้่ฆ
13
ๆ็็ถ้ฉ ๆ้ไธๅนดไพไน่ท่ๅคงๅฎถไธ่ตทๅญธ็ฟ ่ณๆ็ตๆง ่ ๆผ็ฎๆณ ไธไบๅฅฝๆนๆณ ( ่ชไปฅ็บ ) ่ทๅคงๅฎถๅไบซ :
1. ็ซๅ , ็จ็ฐกๅฎ็ไพๅญๅ่งฃๆญฅ้ฉ2. ๅ ไบ่งฃๆฆๅฟต ( ๅคงๆนๅ ), ไธๆฅ่็่ค้็ๆธๅญธ
( ๆๅธธ่ชช : ๅฆๆไป้บผ้ฝๆฒๅผๆ , ๅ ๅผๆ้ๅ )3. ไธๆฌกไบ่งฃไธๅฐ้จๅๅฐฑๅฅฝ ( ๆจก็ตๅๅญธ็ฟ ), ไธๆฅ่ๅผๆๆ
ๆ็้จๅ4. ็ธไฟก่ชๅทฑไธๅฎๅฏไปฅๅผๆ ( ้ๅธธ้่ฆ )5. ไธ็ฅ้่ชๅทฑๅฐๅบๆไบๆฒ ? ็จไธไบๅคๆช็ไพๅญไพ่ฉฆ่ฉฆ
(boundary case)6. ็ทด็ฟ่ชๅทฑ็่ชฒๆฌ ( ้ๅธธ้่ฆ )
14
ๆๆซ่ๅ งๅฎน & ๅๅผ Closed book, 2 x A4 cheat sheets (double-
sided) ไฝๅญธๆๆ็ธพ 30% ๅ ๅซๆดๅญธๆไธ่ชฒๅ งๅฎน , ไฝไปฅๆไธญ่้ๅพ็ๅ งๅฎน็บไธป 180 minutes ้กๅ่ๆไธญ่้กไผผ ( ๆฏ้ + ้ธๆ + ่งฃ้ ) Extra office hour ??
15
ๅๆ , ๆๆๅญธ็ๆถฏ็็ฌฌไธ็ญๅญธ็~
ๅธๆไปฅๅพไน่ฝๅจ้ธไฟฎ่ชฒ / ๅฐ้ก่ชฒ็ๅฐไฝ ๅ ~
Iโll miss you
็ฅๅไฝๆๅไธๅคช็่็ๆๆซ + ๆๅฟซ็ๆๅ