Embed Size (px)
Citation preview
Len
nar
t G
ust
afss
on pag
e
Cours
e note
s C
SE
2330
1
Pa
rt I
II C
yb
ern
etic
s
C
hap
ter
6 C
on
cep
t N
euro
ns
Ch
ap
ter
7 N
eura
l C
od
ing
Ch
ap
ter
8 O
ur
Fri
end
th
e L
imu
lus
Ch
ap
ter
9 S
up
ervis
ed L
earn
ing
Ch
ap
ter
10 A
ssoci
ati
ve
Mem
ory
Net
work
s
Thes
e ch
apte
rs p
rese
nt
clas
sica
l ar
tifi
cial
neu
ral
net
work
s an
d t
he
subst
ance
this
cours
e is
conce
ntr
ated
on.
Tw
o c
hap
ters
are
lac
kin
g i
n t
he
book,
one
on U
nsu
per
vis
ed L
earn
ing a
nd o
ne
on
Rei
nfo
rcem
ent
Lea
rnin
g w
hic
h i
n b
rain
model
ling a
re m
uch
more
im
port
ant
than
super
vis
ed
lear
nin
g, as
I s
ee i
t. W
e w
ill,
of
cours
e, d
iscu
ss t
hes
e to
pic
s to
o.
Len
nar
t G
ust
afss
on pag
e
Cours
e note
s C
SE
2330
2
Ch
ap
ter
6,
Co
nce
pt
Neu
ron
s T
he
McC
ull
och
-Pit
ts n
euro
n f
rom
1943 a
ccord
ing t
o L
ytt
on:
Len
nar
t G
ust
afss
on pag
e
Cours
e note
s C
SE
2330
3
Ch
ap
ter
6,
Co
nce
pt
Neu
ron
s W
e w
ill
dra
w n
euro
ns
as b
elow
The
synap
ses
are
dra
wn a
s ar
row
s, w
hic
h i
s our
sym
bol
for
exci
tato
ry s
ynap
ses.
The
wei
ghte
d s
um
of
the
affe
rents
is
call
ed a
ctiv
atio
n p
ote
nti
al.
The
nonli
nea
rity
, w
hic
h i
s a
par
t of
alm
ost
all
neu
ron m
odel
s, r
esid
es i
n t
he
circ
le.
Len
nar
t G
ust
afss
on pag
e
Cours
e note
s C
SE
2330
4
Ch
ap
ter
6,
Co
nce
pt
Neu
ron
s
A b
inar
y n
euro
n w
ill
rece
ive
bin
ary
inputs
fro
m o
ther
bin
ary
neu
rons.
The
inputs
coll
cted
in
vec
tors
can
be
illu
stra
ted i
n t
wo a
nd t
hre
e dim
ensi
ons.
Fro
m t
her
e on w
e m
ust
gen
eral
ize.
Ly
tton, fi
g 6
.6
Len
nar
t G
ust
afss
on pag
e
Cours
e note
s C
SE
2330
5
Ch
ap
ter
6,
Co
nce
pt
Neu
ron
s
A “
com
pu
tin
g”
neu
ron
(L
ytt
on,
fig
6.2
):
Len
nar
t G
ust
afss
on pag
e
Cours
e note
s C
SE
2330
6
Ch
ap
ter
6,
Co
nce
pt
Neu
ron
s R
eal
neu
ron
s h
ave
a sm
oo
th n
onli
nea
rity
(L
ytt
on
, fi
g 6
.3):
Ther
e ar
e se
ver
al d
iffe
rent
way
s to
des
crib
e th
e nonli
nea
rity
mat
hem
atic
ally
, th
e m
ain d
iffe
rence
bei
ng c
onven
ience
.
Len
nar
t G
ust
afss
on pag
e
Cours
e note
s C
SE
2330
7
Ch
ap
ter
6,
Co
nce
pt
Neu
ron
s
Dif
fere
nt
neu
ron
s ar
e co
nn
ecte
d i
n d
iffe
ren
t d
irec
tion
s in
dif
fere
nt
cort
ical
lay
ers:
Len
nar
t G
ust
afss
on pag
e
Cours
e note
s C
SE
2330
8
(Fro
m K
olb
& W
his
haw
: F
undam
enta
ls o
f H
um
an N
euro
psy
cholo
gy)
Ch
ap
ter
6,
Co
nce
pt
Neu
ron
s
We
shou
ld a
lso
rem
emb
er t
hat
th
ere
is a
co
lum
nar
org
aniz
atio
n (
Fro
m M
oun
tcas
tle:
”Th
e co
lum
nar
org
aniz
atio
n o
f th
e n
eoco
rtex
”, B
rain
199
7, p
p. 7
01
-72
2):
Len
nar
t G
ust
afss
on pag
e
Cours
e note
s C
SE
2330
9
Ch
ap
ter
6,
Co
nce
pt
Neu
ron
s
In a
rtif
icia
l n
eura
l n
etw
ork
s w
e h
ave
gen
eric
neu
ron
s, c
on
nec
ted
in
ver
y
sim
pli
fied
cir
cuit
s.
Dep
end
ing
on
th
e m
ain
fea
ture
s re
tain
ed i
n t
he
arti
fici
al n
eura
l n
etw
ork
s w
e
hav
e m
ain
ty
pes
of
net
wo
rks,
su
ch a
s fe
edfo
rwa
rd n
etw
ork
s an
d r
ecu
rren
t
net
wo
rks.
(in
Sec
tio
n I
V B
rain
s w
e w
ill
mak
e se
rio
us
atte
mp
ts t
o i
mp
rov
e o
ur
mo
del
s)
Len
nar
t G
ust
afss
on pag
e
Cours
e note
s C
SE
2330
10
C
ha
pte
r 6
, C
on
cep
t N
euro
ns
A
one
lay
er f
eedfo
rwar
d n
etw
ork
is
show
n b
elow
Just
one
lay
er o
f neu
rons
wit
h n
o i
nte
rdep
enden
ce –
not
much
of
a net
work
yet
.
Len
nar
t G
ust
afss
on pag
e
Cours
e note
s C
SE
2330
11
C
ha
pte
r 6,
Co
nce
pt
Neu
ron
s
A t
wo
lay
er f
eed
forw
ard
art
ific
ial
neu
ral
net
wo
rk (
Ly
tto
n f
ig 6
.7)
Th
ere
is a
hid
den
lay
er “
bef
ore
” th
e o
utp
ut
lay
er o
f n
euro
ns.
Ly
tto
n c
alls
th
is a
th
ree
lay
er n
etw
ork
, co
un
ting
th
e in
pu
ts a
s a
lay
er.
Len
nar
t G
ust
afss
on pag
e
Cours
e note
s C
SE
2330
12
C
ha
pte
r 6,
Co
nce
pt
Neu
ron
s
A p
arti
ally
connec
ted f
eedfo
rwar
d n
etw
ork
(f
rom
Hay
kin
: N
eura
l N
etw
ork
s A
Com
pre
hen
sive
Foundat
ion)
This
is
alre
ady
a b
iolo
gic
ally
more
rea
list
ic m
odel
than
the
full
connec
tivit
y m
odel
s.
Len
nar
t G
ust
afss
on pag
e
Cours
e note
s C
SE
2330
13
Ch
ap
ter
6,
Co
nce
pt
Neu
ron
s A
rec
urr
ent
net
work
is
show
n b
elow
. T
he
effe
rent
signal
s ar
e fe
d b
ack t
o b
ecom
e par
t of
the
affe
rent
signal
s. T
her
e ar
e lo
ops
in r
ecurr
ent
net
work
s! T
he
fill
ed c
ircl
es a
re i
nhib
itory
sy
nap
ses.
Len
nar
t G
ust
afss
on pag
e
Cours
e note
s C
SE
2330
14
Ch
ap
ter
6,
Co
nce
pt
Neu
ron
s
A r
ecu
rren
t n
etw
ork
of
neu
ron
s (L
ytt
on, fi
g 6
.4).
Th
ere
are
no
aff
eren
ts i
n t
his
ver
sio
n.
Len
nar
t G
ust
afss
on pag
e
Cours
e note
s C
SE
2330
15
Ch
ap
ter
6,
Co
nce
pt
Neu
ron
s
Sta
tes
kee
p c
ha
ng
ing
, at
lea
st f
or
som
e ti
me,
in
a r
ecu
rren
t n
etw
ork
(Ly
tto
n,
fig
6.5
; th
ere
is a
mis
tak
e in
th
e b
oo
k –
un
it a
sh
ou
ld a
lway
s b
e 0
):
