1

Interconnection Networks

Charles E. Leiserson

6.896 Theory of Parallel SystemsMarch 29, 2004

2

“Ideal” Parallel Computer

P+MP+M P+MP+M

P+MP+M P+MP+M

P+MP+M

P+MP+M

P+MP+M

P+MP+M

3

“Ideal” Parallel Computer

…

PP

PP

PP

PP

MM

MM

MM

MM…

4

“Ideal” Parallel Computer

5

“Ideal” Parallel Computer

6

“Ideal” Parallel Computer

7

Mesh of Trees

8

Mesh of Trees

9

Hypercube

d = 0N = 1

d = 1N = 2

d = 2N = 4

d = 3N = 8

d = 4N = 16

10

Hypercube

010010 011011

000000 001001

110110 111111

100100 101101

11

Cube-Connected Cycles

12

Butterfly (FFT) Network0 1 2 0

000001010011100101110111

000001010011100101110111

13

Butterflies

14

Decomposing a Butterfly

15

Decomposing a Butterfly

16

Decomposing a Butterfly

17

Decomposing a Butterfly

18

Decomposing a Butterfly II

19

Decomposing a Butterfly II

20

Decomposing a Butterfly II

21

Decomposing a Butterfly II

22

Decomposing a Butterfly II

23

Decomposing a Butterfly II

24

Decomposing a Butterfly II

25

Routing on a Butterfly0 1 2 0

000001010011100101110111

000001010011100101110111

26

Tree in Butterfly0 1 2 0

000001010011100101110111

000001010011100101110111

27

Tree in Butterfly0 1 2 0

000001010011100101110111

000001010011100101110111

28

Beneš Network

29

n/2Benešn/2

Beneš

n/2Benešn/2

Beneš

Decomposing a Beneš Network

30

Routing on a Beneš Network

01234567

50746231

31

Routing on a Beneš Network

01234567

50746231

32

Routing on a Beneš Network

01234567

50746231

33

Routing on a Beneš Network

01234567

50746231

34

Routing on a Beneš Network

01234567

50746231

35

Routing on a Beneš Network

01234567

50746231

36

Routing on a Beneš Network

01234567

50746231

37

Routing on a Beneš Network

01234567

50746231

38

Routing on a Beneš Network

01234567

50746231

39

Routing on a Beneš Network

50746231

01234567

40

Bitonic Sorting Network

Hypercube connections!

41

Bitonic Sort on Butterfly

42

Bitonic Sort on Butterfly

43

Bitonic Sort on Butterfly

44

Bitonic Sort on Butterfly

45

Bitonic Sort on Butterfly

46

Bitonic Sort on Butterfly

47

Bitonic Sort on Butterfly

48

Bitonic Sort on Butterfly

49

Bitonic Sort on Butterfly

50

Bitonic Sort on Butterfly

51

Bitonic Sort on Butterfly

52

Bitonic Sort on Butterfly

53

Bitonic Sort on Butterfly