Upload
anirut-trakultritrung
View
43
Download
1
Embed Size (px)
DESCRIPTION
Neural Networks for Solving Quadratic Assignment Problems
Citation preview
Agenda Hopfield Neural Network Gaussian Machine Quadratic Assignment Problems How to Solve Problem Computation Results Conclusion
Artificial Neural Network (ANN) An mathematic model Inspired by the biological nervous systems Acquires knowledge through learning ANN’s knowledge is stored within inter-neuron
connection strengths (Synaptic weights)
Use continuous activation function Fully connected recurrent network Notion of energy Each state has an energy Computes recursively until a stable state
reached Converges to stable states
Hopfield Network Model:
Hopfield Neural Network
1
( ) ( ) ( )N
ii i j i i
j
du t u t T x t Idt =
= - + +å
0
( )1( ) ( ( )) 1 tanh2
ii i
u tx t f u ta
æ öæ ö= = +ç ÷ç ÷ç ÷è øè ø
Dynamic
Equation:
Output:
Hopfield Neural Network
Activation Function
a0 parameter determines behavior of the gain function Higher ~ gentle Lower ~ steep
0
( )1( ) ( ( )) 1 tanh2
ii i
u tx t f u ta
æ öæ ö= = +ç ÷ç ÷ç ÷è øè ø
ui
xi =f(ui)
0
1
Hopfield Network: Energy Function Quantification of current energy of the network Energy surface determines stable states Stable states are local minima
Each update converges to stable state Symmetric connections
2
1
0N
i
i
dxdEdt dt=
æ ö= - £ç ÷è ø
å
1 1 1
1( )2
N N N
i j i j i ii j i
E x T x x I x= = =
= - -åå å
, 0i j ji iiT T T= =
Energy
function:
Lyapunov
Condition:
Gaussian Machine:An Improvement of Hopfield NNGaussian Machine Objective: Allow the system to escape from local minima Added Gaussian noises that its power vary in time Vary the activation function gain in time
,1
( ) ( ) ( ) ( )N
ii i j j i i
j
du t u t T x t I tdt =
= - + + +å
1( ) ( ( )) 1 tanh2 ( )i ix t f u tæ öæ ö
= = +ç ÷ç ÷ç ÷è øè ø
Dynamic
Equation:
Output:
Gaussian Machine:An Improvement of Hopfield NNGaussian Machine Added Gaussian noises that its power vary in time
( ) (0, ( ))i it N th s=
Dynamic
Equation:Temperature
0 0.5 1 1.5 2 2.5 3-1.5
-1
-0.5
0
0.5
1
1.5
t
( )i th
,1
( ) ( ) ( ) ( )N
ii i j j i i
j
du t u t T x t I tdt =
= - + + +å
Gaussian Machine:An Improvement of Hopfield NNGaussian Machine Vary the activation function gain in time
Output:
ui
xi =f(ui)1
1( ) ( ( )) 1 tanh2 ( )i ix t f u tæ öæ ö
= = +ç ÷ç ÷ç ÷è øè ø
Quadratic Assignment Problems-QAP
12 23 13AB BC ACCost f d f d f d= + +
Assign N Facilities to N locations Minimum sum of product of
“flow between facilities” and
“distance between locations”
N! Possible Solutions
Computationally hard problem, grows exponentially
N = 12, 479001600 solutions
N = 20, 2432902008176640000 solutions
How to find a solution from this ocean ?
Quadratic Assignment Problems-QAP
Problem Representation: ~ the distance between location k and l and ~ the traffic flow between facility i and j and
A solution: Permutation Matrix (NxN Matrix)
Quadratic Assignment Problems-QAP
[ ]klD d=
[ ]ijF f=
0 ,ii ij jif f f= =
[ ]ikP x=
0 ,kk kl lkd d d= =
1 if is assigned to 0 otherwise ik
i kx ì
= íî
Objective: Assign N Facilities to N locations Minimize the total cost of assignment
The constraints
Quadratic Assignment Problems-QAP
1 1 1 1
1min2
N N N N
ij kl ik jlx i j k lC f d x x
= = = =
= åååå
1
1
1 for 1,...,
1 for 1,...,
[0,1] ,
N
ikiN
ikk
ik
x i N
x k N
x i k
=
=
= =
= =
Î "
å
å
: Only one location k is assigned in each facility i
: Only one facility i is assigned in each location k
: Output level boundary
Quadratic function QAP
QAP Example
Neural Network as QAP solver
Find a representation for the problem
Define a problem energy function
Derive T and I matrixes from the energy function
Construct the network using T and I matrixes
Representation of QAP Permutation matrix represents an assignment Rows ~ Facilities Columns ~ Locations
12 23 34 45 51DA BC CB BE EDCost f d f d f d f d f d= + + + +
[ ]ikP x= =1 if is assigned to 0 otherwise ik
i kx ì
= íî
Representation of QAP Use a neuron to represent each entry of the matrix ��If the entry is 1, neuron is on ( ≈ 1) ��If the entry is 0, neuron is off ( ≈ 0)
N-facilities problem represented using N2 neurons
ikx
ikx
Hopfield Neural Network for QAP Arrange the neuron in a matrix form Neurons addressed with double indices
,1 1 1 1 1 1
1( )2
N N N N N N
ik jl ik jl ik iki j k l i k
E x T x x I x= = = = = =
= - -åååå åå
,1 1
( ) ( ) ( )N N
ikik ik jl jl ik
j l
du t u t T x t Idt = =
= - + +åå
0
( )1( ) ( ( )) 1 tanh2
ikik ik
u tx t f u ta
æ öæ ö= = +ç ÷ç ÷ç ÷è øè ø
Dynamic
Equation:
Output:
Energy
function:
Gaussian Machine for QAP
Same notation as Hopfield network
,1 1 1 1 1 1
1( )2
N N N N N N
ik jl ik jl ik iki j k l i k
E x T x x I x= = = = = =
= - -åååå åå
Dynamic
Equation:
Output:
Energy
function:
( )1( ) ( ( )) 1 tanh2 ( )
ikik ik
u tx t f u tæ öæ ö
= = +ç ÷ç ÷ç ÷è øè ø
,1 1
( ) ( ) ( ) ( )N N
ikik ik jl jl ik ik
j l
du t u t T x t I tdt = =
= - + + +åå
Neural Network as QAP solver
Find a representation for the problem
Define a problem energy function
Derive T and I matrixes from the energy function
Construct the network using T and I matrixes
Energy Function for QAP Its minima must correspond to the valid solutions Shorter paths and flow must have lower energy So, break it down into
penalty cost( )E x E E= +
Energy Function for QAP Constraint Satisfaction:
Cost:
2 2
penalty1 1 1 1 1 1
1 1 (1 )2 2 2
N N N N N N
ik ik ik iki k k i k i
A A CE x x x x= = = = = =
æ ö æ ö= - + - + -ç ÷ ç ÷
è ø è øå å å å åå
cost, 1 , 12
N N
ij kl ik jli j k l
BE f d x x= =
= åå
Only one “1” in each row Only one “1” in each column Output level close to “1”
Neural Network as QAP solver
Find a representation for the problem
Define a problem energy function
Derive T and I matrixes from the energy function
Construct the network using T and I matrixes
Mapping QAP onto Neural Network
Quadratic terms for T values Linear terms for I values
,, 1 , 1 , 1
1( )2
N N N
ik jl ik jl ik iki j k l i k
E x T x x I x= = =
= - -åå åQuadratic term Linear term
Mapping QAP onto Neural Network QAP Energy Function
Network Energy Function
2 2
1 1 1 1 1 1
, 1 , 1
( ) 1 1 (1 )2 2 2
2
N N N N N N
ik ik ik iki k k i k i
N N
ij kl ik jli j k l
A A CE x x x x x
B f d x x
= = = = = =
= =
æ ö æ ö= - + - + -ç ÷ ç ÷è ø è ø
+
å å å å åå
åå
,, 1 , 1 , 1
1( )2
N N N
ik jl ik jl ik iki j k l i k
E x T x x I x= = =
= - -åå å
, , 1 , , 1 , , , 1 , 1 , 1
1 1
( )2 2 2 2
2
N N N N N
ik jl ik jl ik jl ij kl ik jli k l k i j i j k l i j k l
N N
ik ik ikk i
A A C BE x x x x x x x f d x x
CA x A x x
= = = = =
= =
= + - +
- - -
å å å åå
å å
Mapping QAP onto Neural Network QAP Energy Function
Network Energy Function
,, 1 , 1 , 1
1( )2
N N N
ik jl ik jl ik iki j k l i k
E x T x x I x= = =
= - -åå å
Mapping QAP onto Neural Network
Network Energy Function
( )E x T x x I x= - -
Linear term
Mapping QAP onto Neural Network
Network Energy Function
Derived T and I matrices
( )E x T x x I x= - -
Linear term
, , , , ,
22
ik jl i j k l i j k l ij kl
ik
T A A C Bf dCI A
d d d d= - - + -
= +1 ,0 ,ik
i ji j
d=ì
= í ¹î
Kronecker Delta
Neural Network as QAP solver
Find a representation for the problem
Define a problem energy function
Derive T and I matrixes from the energy function
Construct the network using T and I matrixes
Agenda Hopfield Neural Network Gaussian Machine Quadratic Assignment Problems How to Solve Problem Computation Results Conclusion
Computation Results
Conclusion