1
A H
isto
ry o
f Sup
erco
mpu
ting
Bur
ton
Smith
Mic
roso
ft
2
Out
line
!D
efin
ition
s!
Early
day
s: C
olos
sus a
nd E
niac
!M
ainf
ram
es: U
niva
c LA
RC
to C
DC
760
0!
SIM
D sy
stem
s"
Proc
esso
r arr
ays:
Illia
c IV
, PEP
E, a
nd B
SP"
One
-bit
arra
ys: S
tara
n, D
AP,
and
the
Goo
dyea
r MPP
"V
ecto
r pip
elin
ing:
TI-
ASC
, CD
C S
tar-
100,
and
Cra
y-1
"W
ide
inst
ruct
ions
: FPS
, Mul
tiflo
w, a
nd C
ydro
me
!M
IMD
syst
ems
"Sh
ared
mem
ory:
Cra
y X
-MP
to S
GI A
ltix
"D
istri
bute
d m
emor
y: C
osm
ic C
ube
to B
lue
Gen
e!
Japa
n-U
S co
mpe
titio
n!
Wha
t nex
t?
3
Def
initi
ons
!Su
perc
ompu
ter:
the
wor
ld�s
fast
est c
ompu
ter a
t the
tim
e!
SIM
D: S
ingl
e In
stru
ctio
n St
ream
, Mul
tiple
Dat
a St
ream
"O
ne in
stru
ctio
n at
a ti
me,
eac
h op
erat
ing
on a
n ar
ray
of d
ata
!M
IMD
: Mul
tiple
Inst
ruct
ion
Stre
am, M
ultip
le D
ata
Stre
am"
Mul
tiple
pro
cess
ors a
sync
hron
ousl
y is
suin
g in
stru
ctio
ns!
Shar
ed M
emor
y: M
IMD
com
pute
r in
whi
ch a
com
mon
m
emor
y is
acc
essi
ble
by a
ll pr
oces
sors
"U
MA
: Uni
form
mem
ory
acce
ss b
y al
l pro
cess
ors
"N
UM
A: N
on-u
nifo
rm m
emor
y ac
cess
, bas
ed o
n pl
acem
ent
!D
istri
bute
d M
emor
y: M
IMD
com
pute
r in
whi
ch th
e m
emor
y is
par
titio
ned
into
pro
cess
or-p
rivat
e re
gion
s!
RIS
C: R
educ
ed In
stru
ctio
n Se
t Com
pute
r"
Use
s fas
t and
sim
ple
inst
ruct
ions
to d
o co
mpl
ex th
ings
!Pi
pelin
ing:
Pro
cess
ing
in a
ssem
bly-
line
styl
e
4
Nam
es a
nd P
lace
s!
BM
D/A
TC: B
allis
tic M
issi
le D
efen
se A
dvan
ced
Tech
nolo
gy C
ente
r, H
unts
ville
, AL
!B
RL:
Bal
listic
s (no
w A
rmy}
Res
earc
h La
bora
tory
, Abe
rdee
n, M
D!
DA
RPA
: Def
ense
Adv
ance
d R
esea
rch
Proj
ects
Age
ncy,
VA
!N
ASA
-Am
es: A
mes
Res
earc
h C
ente
r, M
ount
ain
Vie
w, C
A!
NA
SA-G
odda
rd: G
odda
rd S
pace
Flig
ht C
ente
r, G
reen
belt,
MD
!Li
verm
ore:
Law
renc
e Li
verm
ore
Nat
iona
l Lab
orat
ory,
Liv
erm
ore,
CA
!Lo
s Ala
mos
: Los
Ala
mos
Nat
iona
l Lab
orat
ory,
Los
Ala
mos
, NM
!N
SA: N
atio
nal S
ecur
ity A
genc
y, F
ort M
eade
, MD
!Sa
ndia
: San
dia
Nat
iona
l Lab
orat
orie
s, A
lbuq
uerq
ue, N
M
5
Ear
ly d
ays:
Col
ossu
s!
Use
d at
Ble
tchl
ey P
ark,
Eng
land
dur
ing
WW
II"
for c
rypt
anal
ysis
of L
oren
z SZ
40/4
2 ro
tor s
yste
ms (
�Fis
h�)
"O
nly
rece
ntly
dec
lass
ified
by
Her
Maj
esty
�s G
over
nmen
t!
Con
cept
s by
Max
New
man
et a
l.!
Des
igne
d an
d bu
ilt b
y To
mm
y Fl
ower
s"
at th
e Po
st O
ffic
e R
esea
rch
Stat
ion,
Dol
lis H
ill!
Feat
ures
:"
Pape
r tap
e lo
op d
ata
inpu
t at 3
0 m
ph =
500
0 ch
arac
ters
/sec
"15
00 v
acuu
m tu
bes (
�val
ves�
) in
the
Mar
k I,
2400
in M
ark
II"
Prog
ram
med
by
switc
hes
"N
ot a
gen
eral
-pur
pose
com
pute
r
6
Col
ossu
s
7
Ear
ly d
ays:
Eni
ac!
Bui
lt at
Pen
n�s M
oore
Sch
ool f
or B
RL,
Abe
rdee
n, M
D"
The
mot
ivat
ing
appl
icat
ion
was
arti
llery
firin
g ta
bles
"Its
firs
t use
was
eva
luat
ing
H-b
omb
feas
ibili
ty!
Des
igne
d an
d bu
ilt b
y J.
Pres
per E
cker
t and
John
Mau
chly
!U
nvei
led
at th
e M
oore
Sch
ool o
n Fe
brua
ry 1
5, 1
946
!It
had
17,4
68 v
acuu
m tu
bes a
nd c
onsu
med
174
KW
"R
elia
bilit
y w
as a
con
cern
, so
it w
as le
ft on
con
tinuo
usly
!It
was
80
feet
long
and
wei
ghed
30
tons
!Pr
ogra
mm
ing
was
via
switc
hes a
nd p
atch
cab
les
!N
ot a
gen
eral
-pur
pose
com
pute
r
8
Eni
ac
9
Oth
er e
arly
supe
rcom
pute
rs
Zuse
Z3
(194
1)
The
IAS
mac
hine
s (19
52)
Man
ches
ter/F
erra
nti M
ark
I (19
51)
Uni
vac
1 (1
951)
10
Mai
nfra
mes
!Th
e 50
�s a
nd 6
0�s s
aw fa
st p
rogr
ess i
n co
mpu
ter t
echn
olog
y!
Mos
t of i
t was
firs
t use
d in
�m
ainf
ram
e� su
perc
ompu
ters
"Th
ese
syst
ems w
ere
used
for b
oth
busi
ness
and
scie
nce
"La
ter,
supe
rcom
pute
rs b
ecam
e m
uch
mor
e sc
ienc
e-or
ient
ed!
Exam
ples
of n
ew te
chno
logy
incl
uded
:"
Mag
netic
cor
e m
emor
y"
Tran
sist
or lo
gic
circ
uits
"Fl
oatin
g po
int h
ardw
are
"Pi
pelin
ing
"H
igh
leve
l lan
guag
es a
nd c
ompi
lers
11
Mai
nfra
mes
: LA
RC
!B
egun
in 1
955
for L
iver
mor
e an
d de
liver
ed in
196
0!
Had
dua
l pro
cess
ors a
nd d
ecim
al a
rithm
etic
!Em
ploy
ed su
rfac
e-ba
rrie
r tra
nsis
tors
and
cor
e m
emor
y
12
Mai
nfra
mes
: Str
etch
and
Har
vest
�IB
M 7
030
(STR
ETC
H)
�Del
iver
ed to
Los
Ala
mos
4/6
1�P
ione
ered
in b
oth
arch
itect
ure
and
impl
emen
tatio
n at
IBM
�IB
M 7
950
(HA
RV
EST)
�Del
iver
ed to
NSA
2/6
2�W
as S
TRET
CH
+ 4
boxe
s�I
BM
795
1 St
ream
uni
t�I
BM
795
2 C
ore
stor
age
�IB
M 7
955
Tape
uni
t�I
BM
795
9 I/O
Exc
hang
e
13
Mai
nfra
mes
: CD
C 6
600
!Se
ymou
r Cra
y an
d B
ill N
orris
left
Uni
vac
to st
art C
DC
!C
ray�
s CD
C 1
604
was
the
first
succ
essf
ul tr
ansi
stor
syst
em!
He
wan
ted
to b
uild
a sc
ient
ific
supe
rcom
pute
r!
The
6600
, bui
lt w
ith si
licon
tran
sist
ors,
ship
ped
Sept
. 196
4"
The
chie
f arc
hite
cts w
ere
Seym
our C
ray
and
Jim
Tho
rnto
n!
The
6600
had
seve
ral n
otab
le fe
atur
es"
A v
ery
sim
ple
inst
ruct
ion
set,
lend
ing
itsel
f to
mor
e sp
eed
#N
o de
cim
al a
rithm
etic
nee
ded
or w
ante
d"
10 a
rithm
etic
func
tion
units
abl
e to
pro
cess
in p
aral
lel
#Th
is o
verla
pped
exe
cutio
n st
yle
is n
ow c
alle
d su
pers
cala
r#
It w
as c
oord
inat
ed b
y a
fam
ous c
ircui
t cal
led
the
scor
eboa
rd"
Para
llel I
/O p
roce
ssor
s usi
ng a
new
idea
cal
led
mul
tithr
eadi
ng#
Som
e sa
y R
ISC
mea
ns �
Rea
lly In
vent
ed b
y Se
ymou
r Cra
y�"
A fu
turis
tic o
pera
tor�
s con
sole
was
pro
vide
d at
no
extra
cha
rge
14
CD
C 6
600
Con
sole
15
Mai
nfra
mes
: IB
M 3
60/9
1!
IBM
�s T
om W
atso
n w
as a
ngry
:"L
ast w
eek,
Con
trol D
ata
...an
noun
ced
the
6600
syst
em.
I und
erst
and
that
in th
e la
bora
tory
...th
ere
are
only
34
peop
le in
clud
ing
the
jani
tor.
Of
thes
e, 1
4 ar
e en
gine
ers a
nd 4
are
pro
gram
mer
s. C
ontra
stin
g th
ism
odes
t ef
fort
with
our
vas
t dev
elop
men
t act
iviti
es, I
fail
to u
nder
stan
d w
hy w
e ha
ve lo
st o
ur in
dust
ry le
ader
ship
pos
ition
by
letti
ng so
meo
ne e
lse
offe
r th
e w
orld
's m
ost p
ower
ful c
ompu
ter."
!Th
e 36
0/91
(AK
A 3
70/1
95) w
as IB
M�s
ans
wer
!It
was
del
iver
ed to
NA
SA-G
odda
rd in
Oct
ober
196
7!
Kill
er A
pp: P
asse
nger
Airl
ine
Res
erva
tion
Syst
ems (
1964
)"
Join
tly d
evel
oped
by
IBM
, Am
eric
an, a
nd U
nite
d A
irlin
es"
Orig
inal
ly w
ritte
n in
IBM
ass
embl
y (i.
e. m
achi
ne) l
angu
age
!It
was
ver
y cl
ose
to th
e C
DC
760
0 in
per
form
ance
!Th
e 36
0/91
als
o ha
d a
pret
ty im
pres
sive
con
sole
�
16
IBM
360
/91
Con
sole
17
Mai
nfra
mes
: CD
C 7
600
!Th
e 76
00 w
as d
eliv
ered
to L
iver
mor
e in
ear
ly 1
969
!It
was
hig
hly
com
patib
le w
ith th
e 66
00!
Bes
ides
bei
ng fa
ster
, its
arit
hmet
ic w
as p
ipel
ined
"Th
is m
eant
the
arith
met
ic u
nits
wer
e ne
ver b
usy
!It
was
so c
ompa
ct it
had
to b
e co
oled
with
freo
n!
It w
as a
lso
one
of th
e w
orld
s mos
t bea
utifu
l com
pute
rs
Seym
our C
ray
18
Two
CD
C 7
600s
19
SIM
D a
rray
s: Il
liac
IV!
By
the
late
60�
s, it
was
cle
ar m
ainf
ram
es w
eren
�t en
ough
!To
impr
ove
perf
orm
ance
, SIM
D a
rray
mac
hine
s wer
e bu
ilt
or p
ropo
sed
with
man
y ar
ithm
etic
pro
cess
ing
units
"So
lom
on w
as a
n ea
rly W
estin
ghou
se S
IMD
arr
ay p
roto
type
!Th
e Ill
iac
IV w
as a
Uni
vers
ity o
f Illi
nois
/Bur
roug
hs p
roje
ct"
Fund
ed b
y D
AR
PA fr
om 1
964
onw
ard,
it w
as u
sabl
e in
197
5"
The
chie
f arc
hite
ct, D
an S
lotn
ick,
cam
e fr
om W
estin
ghou
se!
It w
as to
hav
e 25
6 ar
ithm
etic
uni
ts, l
ater
cut
bac
k to
64
!Th
e th
in-f
ilm m
emor
y sy
stem
was
a m
ajor
hea
dach
e!
Afte
r stu
dent
dem
onst
ratio
ns a
t Illi
nois
in M
ay 1
970,
the
proj
ect w
as m
oved
to N
ASA
-Am
es!
Lang
uage
s lik
e IV
tran
(not
pron
ounc
ed fo
ur-tr
an) a
imed
to
use
para
llel l
oops
to e
xpre
ss th
e ne
cess
ary
para
llelis
m"
Muc
h co
mpi
ler e
xper
tise
was
dev
elop
ed w
orki
ng o
n Ill
iac
IV
20
ILLI
AC
IV
21
SIM
D A
rray
s: P
EPE
and
BSP
!B
oth
wer
e B
urro
ughs
des
igns
der
ived
from
Illia
c IV
!PE
PE h
ad 2
88 p
roce
ssin
g el
emen
ts a
nd w
as d
esig
ned
to
track
bal
listic
mis
sile
s"
It w
as d
eliv
ered
to B
MD
/ATC
in H
unts
ville
in 1
976
!B
SP h
ad 1
6 pr
oces
sing
ele
men
ts a
nd w
as a
imed
at
com
mer
cial
use
"R
icha
rd S
toke
s was
chi
ef a
rchi
tect
"Th
e fir
st c
usto
mer
was
Mar
atho
n O
il C
o. in
Den
ver,
CO
"Th
e pr
ojec
t was
can
celle
d in
198
0, b
efor
e its
firs
t del
iver
y
22
One
-bit
Arr
ays:
STA
RA
N, D
AP,
MPP
!Th
ese
wer
e ar
rays
of p
roce
ssor
s tha
t wer
e on
ly o
ne b
it w
ide
"Th
ey w
ere
inex
pens
ive
to im
plem
ent,
espe
cial
ly o
n on
e ch
ip"
Arit
hmet
ic w
as se
rial (
like
grad
e-sc
hool
, onl
y bi
nary
)#
It co
uld
use
wha
teve
r pre
cisi
on m
ade
sens
e fo
r the
pro
blem
!G
oody
ear A
eros
pace
dev
elop
ed S
TAR
AN
in 1
972
"It
was
a 2
56 b
y 25
6 ar
ray
with
lots
of d
ata
perm
utin
g po
wer
"K
en B
atch
er, a
n Ill
inoi
s gra
duat
e, w
as c
hief
arc
hite
ct"
It w
as g
reat
at i
mag
e pr
oces
sing
, tho
ugh
not i
nten
ded
for i
t "
The
ASP
RO
ver
sion
was
use
d on
Nav
y A
WA
CS
airc
raft
!Th
e G
oody
ear M
PP w
as a
sing
le-c
hip
STA
RA
N su
cces
sor
"Th
e ar
ray
size
was
redu
ced
to 1
28 b
y 12
8"
Del
iver
ed to
NA
SA-G
odda
rd in
198
4!
ICL
(UK
) dev
elop
ed th
e D
istri
bute
d A
rray
Pro
cess
or"
Stua
rt R
edda
way
was
the
arch
itect
23
One
-bit
Arr
ays:
Con
nect
ion
Mac
hine
!D
anny
Hill
is a
nd h
is c
olle
ague
s ini
tially
bas
ed th
e de
sign
on
a �
conn
ectio
nist
� ap
proa
ch to
arti
ficia
l int
ellig
ence
"Th
e de
sign
cam
e ou
t of D
anny
�s P
hD th
esis
at M
IT!
The
CM
-1 w
as a
hyp
ercu
be-c
onne
cted
one
-bit
arra
y"
Hig
h-le
vel l
angu
age
supp
ort w
as e
xcel
lent
"Th
e m
achi
ne p
rove
d us
eful
for s
cien
tific
com
puta
tion
!Th
e C
M-2
/CM
-200
add
ed so
me
32-b
it flo
atin
g po
int h
ardw
are
"M
any
HPC
use
rs li
ke N
SA a
ndLo
s Ala
mos
bou
ght t
hem
!Th
e C
M-2
had
an
arra
y of
ligh
t-em
ittin
g di
odes
(rig
ht)
"N
ot o
ne p
er p
roce
ssor
, sad
ly!
The
CM
-5 w
as M
IMD
, not
SIM
DC
M-2
24
Vec
tor p
ipel
inin
g: T
I ASC
!
The
ASC
was
des
igne
d to
exp
loit
Fortr
an lo
ops
"Th
is m
otiv
ated
the
vect
or p
ipel
inin
g de
cisi
on"
The
ASC
�s F
ortra
n co
mpi
ler w
as st
ate-
of-th
e-ar
t!
Seve
ral s
yste
ms s
hipp
ed, b
egin
ning
in 1
974
"G
eoph
ysic
al F
luid
Dyn
amic
s Lab
orat
ory,
Prin
ceto
n, N
J"
Bal
listic
Mis
sile
Def
ense
, Hun
tsvi
lle"
Nav
al R
esea
rch
Labo
rato
ry, A
naco
stia
, MD
!Th
e G
FDL
mac
hine
had
an
inte
rest
ing
perf
orm
ance
tool
: a
strip
-cha
rt re
cord
er p
erch
ed o
n to
p th
at p
lotte
d ve
ctor
leng
th"
The
reco
rdin
g w
as re
turn
ed to
the
user
with
the
job
outp
ut
25
Vec
tor P
ipel
inin
g: C
DC
Sta
r 100
!C
DC
use
d A
PL ra
ther
than
For
tran
loop
s as a
vec
tor m
odel
"A
PL is
a la
ngua
ge in
vent
ed b
y K
en Iv
erso
n of
IBM
Res
earc
h"
Nei
l Lin
coln
and
Jim
Tho
rnto
n w
ere
the
arch
itect
s"
Sidn
ey F
ernb
ach
at L
iver
mor
e w
as th
e ta
rget
cus
tom
er!
The
Star
-100
had
poo
r sca
lar (
non-
vect
or) p
erfo
rman
ce"
The
resu
lt w
as h
ighl
y lim
ited
appl
icab
ility
"Th
is e
ffec
t is c
alle
d Am
dahl
�s la
w!
Bot
h Li
verm
ore
and
Los A
lam
osw
ere
sche
dule
d to
get
one
"Lo
s Ala
mos
bac
ked
out
in fa
vor o
f the
Cra
y-1
"Li
verm
ore
got b
oth
Star
-100
syst
ems a
s a re
sult
Star
-100
26
Am
dahl
�s la
w!
If w
1w
ork
is d
one
at sp
eed
s 1an
d w
2at
spee
d s 2
,th
e av
erag
e sp
eed
sis (
w1+
w2)
/(w1/s
1+
w2/s
2)"
This
is ju
st th
e to
tal w
ork
divi
ded
by th
e to
tal t
ime
!Fo
r exa
mpl
e, if
w1=
9, w
2= 1
, s1=
100
, and
s 2=
1th
en s
= 10
/1.0
9 ≅
9"
This
is o
bvio
usly
not
the
aver
age
of s 1
and
s 2
Am
dahl
, Gen
e M
, �V
alid
ity o
f the
sing
lepr
oces
sor a
ppro
ach
to a
chie
ving
larg
esc
ale
com
putin
g ca
pabi
litie
s�,
Proc
. SJC
C, A
FIPS
Pre
ss, 1
967
27
Vec
tor P
ipel
inin
g: C
ray-
1!
Unl
ike
the
CD
C S
tar-
100,
ther
e w
as n
o de
velo
pmen
t co
ntra
ct fo
r the
Cra
y-1
"M
r. C
ray
disl
iked
gov
ernm
ent�s
look
ing
over
his
shou
lder
!In
stea
d, C
ray
gave
Los
Ala
mos
a o
ne-y
ear f
ree
trial
!A
lmos
t no
softw
are
was
pro
vide
d by
Cra
y R
esea
rch
"Lo
s Ala
mos
dev
elop
ed o
r ada
pted
exi
stin
g so
ftwar
e!
Afte
r the
yea
r was
up,
Los
Ala
mos
leas
ed th
e sy
stem
"Th
e le
ase
was
fina
nced
by
a N
ew M
exic
o pe
trole
um p
erso
n!
The
Cra
y-1
defin
itely
did
not
suff
er fr
om A
mda
hl�s
law
"Its
scal
ar p
erfo
rman
ce w
as tw
ice
that
of t
he 7
600
"O
nce
vect
or so
ftwar
e m
atur
ed, 2
x be
cam
e 8x
or m
ore
!W
hen
peop
le sa
y �s
uper
com
pute
r�, t
hey
thin
k C
ray-
1
28
Cra
y-1
29
VLI
W: F
PS, M
ultif
low
, and
Cyd
rom
e!
VLI
W st
ands
for V
ery
Long
Inst
ruct
ion
Wor
d"
It is
a k
ind
of S
IMD
syst
em in
whi
ch e
ach
arith
met
ic u
nit c
an
be d
oing
som
ethi
ng d
iffer
ent (
thus
the
need
for w
idth
)!
Floa
ting
Poin
t Sys
tem
s led
in 1
976
with
the
AP-
120B
"G
lenn
Cul
ler w
as th
e ch
ief a
rchi
tect
"Th
ese
syst
ems w
ere
embe
dded
in G
E C
AT
scan
ners
"Th
ey w
ere
also
use
d w
idel
y fo
r sei
smic
dat
a pr
oces
sing
!M
ultif
low
was
foun
ded
in 1
984
by Jo
sh F
ishe
r to
expl
oit
som
e co
mpi
ler t
echn
olog
y he
had
bee
n pu
rsui
ng a
t Yal
e"
The
first
ship
men
ts o
f the
Tra
ce 7
/200
wer
e in
198
7!
Bob
Rau
�s c
ompa
ny, C
ydro
me,
was
als
o fo
unde
d in
198
4 an
d de
mon
stra
ted
the
Cyd
ra-5
mac
hine
in 1
987
!B
oth
syst
ems h
ad in
stru
ctio
ns w
ith se
ven
field
s and
256
bits
!Te
chni
cally
, non
e of
thes
e m
achi
nes w
as a
supe
rcom
pute
r
30
Shar
ed M
emor
y: C
ray
Vec
tor S
yste
ms
!C
ray
Res
earc
h, b
y Se
ymou
r Cra
y"
Cra
y-1
(197
6): 1
pro
cess
or"
Cra
y-2
(198
5): u
p to
4 p
roce
ssor
s*!
Cra
y R
esea
rch,
not
by
Seym
our C
ray
"C
ray
X-M
P (1
982)
: up
to 4
pro
cs"
Cra
y Y
-MP
(198
8): u
p to
8 p
rocs
"C
ray
C90
: (19
91?)
: up
to 1
6 pr
ocs
"C
ray
T90:
(199
4): u
p to
32
proc
s"
Cra
y X
1: (2
003)
: up
to 8
192
proc
s!
Cra
y C
ompu
ter,
by S
eym
our C
ray
"C
ray-
3 (1
993)
: up
to 1
6 pr
ocs
"C
ray-
4 (u
nfin
ishe
d): u
p to
64
proc
s!
All
are
UM
A sy
stem
s exc
ept t
he X
1, w
hich
is N
UM
A
*One
8-p
roce
ssor
Cra
y-2
was
bui
lt
Cra
y-2
31
Shar
ed M
emor
y: M
ultit
hrea
ding
!Th
ese
syst
ems p
ipel
ine
MIM
D a
rchi
tect
ure
in th
e sa
me
way
ve
ctor
syst
ems p
ipel
ine
SIM
D a
rchi
tect
ure
"Li
ke v
ecto
r mul
tipro
cess
ors,
they
can
be
UM
A o
r NU
MA
!Ex
ampl
es in
clud
e"
Den
elco
r HEP
(198
0): u
p to
16
proc
s"
Tera
MTA
-1 (1
998)
: up
to 1
6 pr
ocs
"C
ray
MTA
-2 (2
002)
: up
to 2
56 p
rocs
!A
var
iant
form
of m
ultit
hrea
ding
isSi
mul
tane
ous M
ultit
hrea
ding
(SM
T)"
Inte
l®ca
lls S
MT
�Hyp
er-T
hrea
ding
�C
ray
MTA
-2
32
Shar
ed M
emor
y: C
C-N
UM
A!
Cac
he-c
oher
ent N
UM
A sy
stem
s wer
e re
sear
ched
at
Stan
ford
and
dev
elop
ed a
s pro
duct
s by
Silic
on G
raph
ics
"K
enda
ll Sq
uare
Res
earc
h us
ed a
var
iant
cal
led
CO
MA
, w
hich
stan
ds fo
r Cac
he-O
nly
Mem
ory
Arc
hite
ctur
e!
Thes
e sy
stem
s aut
omat
ical
ly m
ove
data
am
ong
cach
esat
ea
ch p
roce
ssor
to m
inim
ize
proc
esso
r dat
a ac
cess
tim
es!
SGI C
C-N
UM
A sy
stem
s:"
Orig
in 2
000
(199
7): u
p to
128
pro
cs"
Orig
in 3
000
(200
3): u
p to
512
pro
cs"
Alti
x (2
003)
: up
to 5
12 p
rocs
*!
Ken
dall
Squa
re C
OM
A sy
stem
s:"
KSR
1: (1
991)
: up
to 1
088
proc
s"
KSR
2: (1
994)
: up
to 5
000
proc
s
* �C
olum
bia�
at N
ASA
-Am
es c
lust
ers 2
0 of
thes
eSGI A
ltix
33
Dis
trib
uted
Mem
ory:
Cos
mic
Cub
e!
Chu
ck S
eitz
and
Geo
ffre
y Fo
x pi
onee
red
the
idea
at C
alte
ch"
The
first
syst
em (1
982)
was
cal
led
the
Cos
mic
Cub
e"
It ha
d 64
Inte
l®80
86 p
roce
ssor
s, ea
ch w
ith a
n 80
87 a
ttach
ed"
The
proc
esso
rs w
ere
conn
ecte
d in
a 6
-dim
ensi
onal
hyp
ercu
be"
They
com
mun
icat
ed b
y se
ndin
g m
essa
ges t
o ea
ch o
ther
"Th
is sy
stem
arc
hite
ctur
e no
w d
omin
ates
supe
rcom
putin
g!
Early
com
mer
cial
syst
ems i
nclu
ded:
"A
met
ek/S
14 (1
986)
: 680
20, o
ver 1
000
proc
s"
Inte
l®IP
SC/1
(198
5): 8
0286
, up
to 6
4 pr
ocs
"nC
UB
E/10
(198
5): c
usto
m, u
p to
102
4 pr
ocs
!V
aria
tions
sinc
e th
en a
re le
gion
nCU
BE/
10
34
Dis
trib
uted
Mem
ory:
Cla
ssic
MPP
s!
A c
lass
ic M
PP is
a d
istri
bute
d m
emor
y sy
stem
who
se
proc
essi
ng e
lem
ents
are
indi
vidu
al (m
icro
)pro
cess
ors
!Ex
ampl
es in
clud
e:"
Cra
y T3
D (1
994)
, T3E
(199
6), X
T3"
Inte
l Tou
chst
one
serie
s, Pa
rago
n (1
992)
"IB
M S
P-1
(199
2), S
P-2,
Dee
p B
lue,
Blu
e G
ene
"A
nd m
any
othe
rs
IBM
Blu
e G
ene/
LC
ray
XT3
35
Dis
trib
uted
Mem
ory:
Clu
ster
s!
A c
lust
er M
PP is
a d
istri
bute
d m
emor
y sy
stem
who
se
proc
essi
ng e
lem
ents
are
shar
ed m
emor
y m
ultip
roce
ssor
s!
Exam
ples
incl
ude:
"IB
M e
Serv
er p
575
"N
EC E
arth
Sim
ulat
or (2
002)
, SX
-8"
A v
ast m
ultit
ude
of h
ome-
mad
e �B
eow
ulf�
clu
ster
s#
Usu
ally
bas
ed o
n In
tel o
r AM
D p
roce
ssor
s#
Even
Gov
ernm
ent L
abs h
ave
done
this
Earth
Sim
ulat
or (a
bove
)Ea
rth S
imul
ator
(bel
ow)
36
The
Top
500
List
!Th
is is
a li
st o
f the
fast
est c
ompu
ters
in th
e w
orld
"w
ww
.top5
00.o
rg"
By
defin
ition
, it�s
a g
reat
pla
ce fo
r sup
erco
mpu
ter s
potti
ng!
Be
very
car
eful
abo
ut th
e te
rm �
fast
est�
, tho
ugh
"Sp
eed
is m
easu
red
by o
ne b
ench
mar
k pr
ogra
m, L
INPA
CK
"LI
NPA
CK
doe
sn�t
care
how
wel
l the
syst
em is
con
nect
ed"
Shar
ed m
emor
y is
n�t u
sefu
l for
goo
d LI
NPA
CK
resu
lts!
With
that
cav
eat,
it�s w
ell w
orth
a lo
ok
37
Japa
nese
Sys
tem
s: F
ujits
u, N
EC
, Hita
chi
!Ja
pane
se su
perc
ompu
ters
mos
tly u
se v
ecto
r pro
cess
ors w
ith
shar
ed o
r (m
ost r
ecen
tly) d
istri
bute
d m
emor
y ar
chite
ctur
e!
Fujit
su sh
ippe
d its
firs
t vec
tor s
yste
m (F
acom
230
) in
1976
:"
This
was
follo
wed
by
the
VP2
00 (1
982)
, VP4
00 (1
985)
,V
P200
0 (1
989)
, VP2
600(
1990
), an
d di
strib
uted
mem
ory
VPP
500(
1992
), V
PP70
0(19
96),
and
VPP
5000
(199
9)"
Rec
ently
, Fuj
itsu
has b
een
selli
ng sc
alar
NU
MA
syst
ems
!N
EC g
ot a
late
star
t on
vect
ors,
but c
augh
t up
very
qui
ckly
:"
SX-1
(198
3), S
X-2
(198
5), S
X-3
(199
0), S
X-4
(199
4),
SX-5
(200
0), a
nd d
istri
bute
d m
emor
y SX
-6 a
nd S
X-7
(200
2),
Earth
Sim
ulat
or (2
002)
, and
SX
-8 (2
006)
"Ta
dash
i Wat
anab
e is
the
chie
f arc
hite
ct o
f the
se sy
stem
s!
Hita
chi h
as so
ld fe
w su
perc
ompu
ters
out
side
Japa
n"
An
exce
ptio
n is
the
dist
ribut
ed m
emor
y SR
8000
(200
0)
38
Japa
nese
Nat
iona
l Pro
ject
s!
The
Nat
iona
l Sup
ersp
eed
Com
pute
r Pro
ject
(198
1)"
Abo
ut ¥
23 b
illio
n ov
er 9
yea
rs"
Inte
nded
to p
rodu
ce a
supe
rcom
pute
r"
Two
proj
ects
: arc
hite
ctur
e an
d te
chno
logy
"C
ompa
nies
: Fuj
itsu,
NEC
, Hita
chi,
Mits
ubis
hi, O
ki, T
oshi
ba!
The
Fifth
Gen
erat
ion
Proj
ect (
1982
)"
Abo
ut ¥
55 b
illio
n ov
er 1
0 ye
ars
"C
ente
red
at a
new
labo
rato
ry: I
CO
T"
Focu
sed
on a
rtific
ial i
ntel
ligen
ce: i
nfer
ence
, vis
ion,
spee
ch"
MC
C in
Aus
tin, T
X w
as a
US
indu
stria
l res
pons
e
39
DA
RPA
Str
ateg
ic C
ompu
ting
Prog
ram
!Th
is p
rogr
am w
as in
tend
ed to
acc
eler
ate
the
deve
lopm
ent
and
appl
icat
ion
of p
aral
lel c
ompu
ting
!It
fund
ed se
vera
l ind
ustri
al d
evel
opm
ent p
roje
cts o
ver t
he
perio
d 19
83-1
993:
"Th
inki
ng M
achi
nes:
Con
nect
ion
Mac
hine
(1-b
it SI
MD
arr
ay)
"B
olt,
Ber
anek
, and
New
man
: But
terf
ly (s
hare
d m
emor
y)"
Inte
l: To
uchs
tone
serie
s (di
strib
uted
mem
ory)
"Te
ra: M
TA (s
hare
d m
emor
y)"
Cra
y: T
3D (d
istri
bute
d m
emor
y)!
Gov
ernm
ent a
genc
ies w
ere
enco
urag
ed to
try
thes
e sy
stem
s
40
The
DO
E A
SCI P
rogr
am!
The
Dep
artm
ent o
f Ene
rgy
was
dire
cted
to u
se c
ompu
ter
sim
ulat
ion
to re
plac
e nu
clea
r wea
pons
test
ing
!It
has p
rocu
red
a se
ries o
f dis
tribu
ted
mem
ory
syst
ems:
Syst
em N
ame
Lab
orat
ory
Man
ufac
ture
rD
ate
Red
Sand
iaIn
tel
1995
Blu
e Pa
cific
Live
rmor
eIB
M19
96B
lue
Mou
ntai
nLo
s Ala
mos
SGI
1998
Whi
teLi
verm
ore
IBM
2000
QLo
s Ala
mos
Com
paq
2002
Red
Sto
rmSa
ndia
Cra
y20
04Pu
rple
Live
rmor
eIB
M20
06
!Th
e pe
rfor
man
ce ra
nge
is a
bout
100
-fol
d
41
Wha
t Nex
t?!
Para
llelis
m is
bec
omin
g m
ains
tream
"In
tel a
nd A
MD
are
put
ting
mul
tiple
pro
cess
ors o
n a
chip
"It�
s onl
y a
few
now
, but
just
wai
t!!
This
will
cha
nge
the
supe
rcom
pute
r pla
ying
fiel
d, a
gain
"Pa
ralle
l app
licat
ion
softw
are
will
em
erge
"Pa
ralle
l lan
guag
es a
nd c
ompi
lers
will
ena
ble
it"
Para
llel a
rchi
tect
ure
will
evo
lve
to m
atch
!Th
e w
orld
�s c
ompu
ters
bec
ome
one
gian
t clu
ster
"Th
e cl
oud
links
them
"W
e m
ust m
ake
them
safe
!W
hat a
n ex
citin
g fie
ld it
is w
ith su
ch c
halle
nges
!
42
Apo
logi
es: C
ompa
nies
Not
Men
tione
d!
Alli
ant
!C
onve
x!
HP
!M
asPa
r!
Mat
sush
ita!
Mei
ko!
Pars
ys!
Pars
ytec
!Q
uadr
ics
!Si
emen
s-N
ixdo
rf!
Sun