Click here to load reader

01 artificial neuron - · PDF file 1/1/2018  · ARTIFICIAL NEURON 2 Topics: connection weights, bias, activation function • Neuron pre-activation (or input activation): • Neuron

  • View
    2

  • Download
    0

Embed Size (px)

Text of 01 artificial neuron - · PDF file 1/1/2018  · ARTIFICIAL NEURON 2 Topics:...

  • Neural networks Feedforward neural network - artificial neuron

  • ARTIFICIAL NEURON 2

    Topics: connection weights, bias, activation function • Neuron pre-activation (or input activation):

    • Neuron (output) activation

    • are the connection weights • is the neuron bias • is called the activation function

    ... 1

    Feedforward neural network

    Hugo Larochelle D

    ´

    epartement d’informatique

    Universit

    ´

    e de Sherbrooke

    [email protected]

    September 6, 2012

    Abstract

    Math for my slides “Feedforward neural network”.

    • a(x) = b+ P

    i wixi = b+w > x

    • h(x) = g(a(x)) = g(b+ P

    i wixi)

    • x 1

    xd

    • w

    • {

    • g(·) b

    • h(x) = g(a(x))

    • a(x) = b(1) +W(1)x ⇣ a(x)i = b

    (1)

    i

    P j W

    (1)

    i,j xj

    • o(x) = g(out)(b(2) +w(2)>x)

    1

    Feedforward neural network

    Hugo Larochelle D

    ´

    epartement d’informatique

    Universit

    ´

    e de Sherbrooke

    [email protected]

    September 6, 2012

    Abstract

    Math for my slides “Feedforward neural network”.

    • a(x) = b+ P

    i wixi = b+w > x

    • h(x) = g(a(x)) = g(b+ P

    i wixi)

    • x 1

    xd

    • w

    • {

    • g(·) b

    • h(x) = g(a(x))

    • a(x) = b(1) +W(1)x ⇣ a(x)i = b

    (1)

    i

    P j W

    (1)

    i,j xj

    • o(x) = g(out)(b(2) +w(2)>x)

    1

    Feedforward neural network

    Hugo Larochelle D

    ´

    epartement d’informatique

    Universit

    ´

    e de Sherbrooke

    [email protected]

    September 6, 2012

    Abstract

    Math for my slides “Feedforward neural network”.

    • a(x) = b+ P

    i wixi = b+w > x

    • h(x) = g(a(x)) = g(b+ P

    i wixi)

    • x 1

    xd b w1 wd

    • w

    • {

    • g(·) b

    • h(x) = g(a(x))

    • a(x) = b(1) +W(1)x ⇣ a(x)i = b

    (1)

    i

    P j W

    (1)

    i,j xj

    • o(x) = g(out)(b(2) +w(2)>x)

    1

    Feedforward neural network

    Hugo Larochelle D

    ´

    epartement d’informatique

    Universit

    ´

    e de Sherbrooke

    [email protected]

    September 6, 2012

    Abstract

    Math for my slides “Feedforward neural network”.

    • a(x) = b+ P

    i wixi = b+w > x

    • h(x) = g(a(x)) = g(b+ P

    i wixi)

    • x 1

    xd b w1 wd

    • w

    • {

    • g(·) b

    • h(x) = g(a(x))

    • a(x) = b(1) +W(1)x ⇣ a(x)i = b

    (1)

    i

    P j W

    (1)

    i,j xj

    • o(x) = g(out)(b(2) +w(2)>x)

    1

    Feedforward neural network

    Hugo Larochelle D

    ´

    epartement d’informatique

    Universit

    ´

    e de Sherbrooke

    [email protected]

    September 6, 2012

    Abstract

    Math for my slides “Feedforward neural network”.

    • a(x) = b+ P

    i wixi = b+w > x

    • h(x) = g(a(x)) = g(b+ P

    i wixi)

    • x 1

    xd b w1 wd

    • w

    • {

    • g(·) b

    • h(x) = g(a(x))

    • a(x) = b(1) +W(1)x ⇣ a(x)i = b

    (1)

    i

    P j W

    (1)

    i,j xj

    • o(x) = g(out)(b(2) +w(2)>x)

    1

    Feedforward neural network

    Hugo Larochelle D

    ´

    epartement d’informatique

    Universit

    ´

    e de Sherbrooke

    [email protected]

    September 6, 2012

    Abstract

    Math for my slides “Feedforward neural network”.

    • a(x) = b+ P

    i wixi = b+w > x

    • h(x) = g(a(x)) = g(b+ P

    i wixi)

    • x 1

    xd b w1 wd

    • w

    • {

    • g(·) b

    • h(x) = g(a(x))

    • a(x) = b(1) +W(1)x ⇣ a(x)i = b

    (1)

    i

    P j W

    (1)

    i,j xj

    • o(x) = g(out)(b(2) +w(2)>x)

    1

    Feedforward neural network

    Hugo Larochelle D

    ´

    epartement d’informatique

    Universit

    ´

    e de Sherbrooke

    [email protected]

    September 6, 2012

    Abstract

    Math for my slides “Feedforward neural network”.

    • a(x) = b+ P

    i wixi = b+w > x

    • h(x) = g(a(x)) = g(b+ P

    i wixi)

    • x 1

    xd b w1 wd

    • w

    • {

    • g(·) b

    • h(x) = g(a(x))

    • a(x) = b(1) +W(1)x ⇣ a(x)i = b

    (1)

    i

    P j W

    (1)

    i,j xj

    • o(x) = g(out)(b(2) +w(2)>x)

    1

    Feedforward neural network

    Hugo Larochelle D

    ´

    epartement d’informatique

    Universit

    ´

    e de Sherbrooke

    [email protected]

    September 6, 2012

    Abstract

    Math for my slides “Feedforward neural network”.

    • a(x) = b+ P

    i wixi = b+w > x

    • h(x) = g(a(x)) = g(b+ P

    i wixi)

    • x 1

    xd b w1 wd

    • w

    • {

    • g(·) b

    • h(x) = g(a(x))

    • a(x) = b(1) +W(1)x ⇣ a(x)i = b

    (1)

    i

    P j W

    (1)

    i,j xj

    • o(x) = g(out)(b(2) +w(2)>x)

    1

    Feedforward neural network

    Hugo Larochelle D

    ´

    epartement d’informatique

    Universit

    ´

    e de Sherbrooke

    [email protected]

    September 6, 2012

    Abstract

    Math for my slides “Feedforward neural network”.

    • a(x) = b+ P

    i wixi = b+w > x

    • h(x) = g(a(x)) = g(b+ P

    i wixi)

    • x 1

    xd b w1 wd

    • w

    • {

    • g(·) b

    • h(x) = g(a(x))

    • a(x) = b(1) +W(1)x ⇣ a(x)i = b

    (1)

    i

    P j W

    (1)

    i,j xj

    • o(x) = g(out)(b(2) +w(2)>x)

    1

    Feedforward neural network

    Hugo Larochelle D

    ´

    epartement d’informatique

    Universit

    ´

    e de Sherbrooke

    [email protected]

    September 6, 2012

    Abstract

    Math for my slides “Feedforward neural network”.

    • a(x) = b+ P

    i wixi = b+w > x

    • h(x) = g(a(x)) = g(b+ P

    i wixi)

    • x 1

    xd b w1 wd

    • w

    • {

    • g(·) b

    • h(x) = g(a(x))

    • a(x) = b(1) +W(1)x ⇣ a(x)i = b

    (1)

    i

    P j W

    (1)

    i,j xj

    • o(x) = g(out)(b(2) +w(2)>x)

    1

  • ARTIFICIAL NEURON 3

    Topics: connection weights, bias, activation function

    2Réseaux de neurones

    -1 1

    -1

    1

    1 1

    1 1

    .5

    -1.5

    .7 -.4-1

    x1 x2

    x1

    x2

    z=+1

    z=-1

    z=-1

    0

    1 -1

    0

    1

    -1

    0

    1

    -1

    0

    1 -1

    0

    1

    -1

    0

    1

    -1

    0

    1 -1

    0

    1

    -1

    0

    1

    -1

    R2

    R2

    R1

    y1 y2

    z

    zk

    wkj

    wji

    x1

    x2

    x1

    x2

    x1

    x2 y1 y2

    sortie k

    entrée i

    cachée j biais

    (from Pascal Vincent’s slides)

    Feedforward neural network

    Hugo Larochelle D

    ´

    epartement d’informatique

    Universit

    ´

    e de Sherbrooke

    [email protected]

    September 6, 2012

    Abstract

    Math for my slides “Feedforward neural network”.

    • a(x) = bi P

    i wixi = bi +w > x

    • h(x) = g(a(x)) = g(bi P

    i wixi)

    • w

    1

    Feedforward neural network

    Hugo Larochelle D

    ´

    epartement d’informatique

    Universit

    ´

    e de Sherbrooke

    [email protected]

    September 6, 2012

    Abstract

    Math for my slides “Feedforward neural network”.

    • a(x) = bi P

    i wixi = bi +w > x

    • h(x) = g(a(x)) = g(bi P

    i wixi)

    • w

    • {

    1

    range determined 
 by

    bias only changes the position of the riff

    Feedforward neural network

    Hugo Larochelle D

    ´

    epartement d’informatique

    Universit

    ´

    e de Sherbrooke

    [email protected]

    September 6, 2012

    Abstract

    Math for my slides “Feedforward neural network”.

    • a(x) = b+ P

    i wixi = b+w > x

    • h(x) = g(a(x)) = g(b+ P

    i wixi)

    • w

    • {

    • g(·) b

    1

    Feedforward neural network

    Hugo Larochelle D

    ´

    epartement d’informatique

    Universit

    ´

    e de Sherbrooke

    [email protected]

    September 6, 2012

    Abstract

    Math for my slides “Feedforward neural network”.

    • a(x) = b+ P

    i wixi = b+w > x

    • h(x) = g(a(x)) = g(b+ P

    i wixi)

    • w

Search related