Upload
asher-sims
View
217
Download
0
Embed Size (px)
DESCRIPTION
3 A study of simulated annealing variants Motivation We aim to find the global solution of the nonlinear optimization problem Numerical methods Deterministic methods Stochastic methods Examples: Multistart, clustering, genetic algorithms, simulated annealing ….
Citation preview
A study of simulated annealing variants
Ana PereiraPolytechnic Institute of Braganca, Portugal
email: [email protected]
Edite FernandesUniversity of Minho, Braga, Portugal
email: [email protected]
SEIO’04 Cádiz, October 25-29, 2004
2A study of simulated annealing variants
Outline
• Motivation
• Simulated annealing algorithm
• Simulated annealing variants
• Computational results – Characterization of the presented variants
• Computational results – Comparison of the presented variants
• Simulated annealing
• Conclusions
3A study of simulated annealing variants
Motivation
We aim to find the global solution of the nonlinear optimization problem
Numerical methods
Deterministic methods
Stochastic methodsExamples: Multistart, clustering, genetic algorithms, simulated annealing ….
4A study of simulated annealing variants
Simulated annealing
In 1953, Metropolis proposed an algorithm to simulate the behavior of physical systems in the presence of a heat bath.
In 1983, based on ideas of Metropolis algorithm, Kirkpatrick, Gelatt and Vecchi, and in 1985 Cerny, proposed the simulated annealing (SA) algorithm to solve combinatorial optimization problems.
In 1986, Bohachevsky, Johnson and Stein applied the SA algorithm to solve continuous optimization problems.
Since then, the SA algorithm has been subject to various modifications and has been applied in many areas such as graph coloring, circuit design, data analysis, image reconstruction, biology, …
5A study of simulated annealing variants
Simulated annealing
Advantages:
Easily implemented.
Can be applied to any optimization problem.
Does not use derivative information.
Does not require specific conditions on the objective function.
Asymptotically converges to a global maximum.
Disadvantages:
Requires a great number of function evaluations.
Many authors have been proposing variants of the SA algorithm.
6A study of simulated annealing variants
Given an initial approximation , a control parameter and the number of iterations with the same control parameter
while stopping criterion is not reached do
Generate a new candidate point y
Analyze the acceptance criterion
end
Update
Reduce the control parameter
end
Simulated annealing algorithm
kcN
0c0t
1 kcj Nfor to do
kc
1 k k
kcN
7A study of simulated annealing variants
Simulated annealing algorithm
Generation of a new candidate point
The new point is found using the current approximation, , and the generating probability density function, .
Acceptance criterion
is the acceptance function and it represents the probability of accepting the point y when is the current point. The acceptance criterion has the following form
The most used acceptance function is
kt k
kt y
f c
kk
t yA c
kt
1 if , 0,1
otherwise
kk
k t y
k
y A ct U
t
min 1,
k
k
k
g t g y
k ct y
A c e
8A study of simulated annealing variants
Simulated annealing algorithm
Reduction of the control parameter
The function is called the control parameter and must be a decreasing function that verifies
Stopping criterion
The stopping criterion is based on the idea that the algorithm should terminate when no changes occur.
We propose that the algorithm stops when successive approximations to a global maximum are similar, i.e, the algorithm stops if the following condition is verified for successive iterations
where represents the previous approximation to an optimum value.
kc
lim 0
k
kc
*N* * antf f
*antf
9A study of simulated annealing variants
Simulated annealing variants
Standard SA variant (SSA)
Generation of a new point
Reduction of the control
parameter
Length of the chain: constant
, 1,1 kiy t U
1 , 0,1 k kc c
kcN
10A study of simulated annealing variants
Simulated annealing variantsCorana SA variant (CSA)
Generation of a new point
Reduction of the control
parameter
Length of the chain: constant
, 1,1 ,
is the euclidian vector and
k k k ki i i i
i
y t d e d U
e
1 , 0,1 k kc c
kcN
0.6* 1 * 0.60.4
0.4 0.6
0.40.41 *
0.4
ki i
k ki i
ki
i
rV r
r
rrV
where is constant throughout the processiV
11A study of simulated annealing variants
ASA variant
Main difference :
There are two control parameters: one associated with the generation of the new points and another associated with the acceptance function.
And it is possible the redefinition of the control parameters.
Simulated annealing variants
12A study of simulated annealing variants
ASA variant
Generation of a new point
Redefinition of the control parameters
Reduction of the control
parameters
Length of the chain: 1
Simulated annealing variants
2 1
.
Consider 0,1 , is given by
1 1sgn 1 1 .2 i
i
k k
ki
u
k ki Gk
G
y t b a
u U
u cc
1
1
0
0
1, for 1 and
1.
i i
nGi
i i
nA
G G
kkG G
A A
kkA A
k ki n
c c e
k k
c c e
kcN
13A study of simulated annealing variants
SALO variant
Similar to ASA algorithm except on generation of a new point.
Generation of a new point
Simulated annealing variants
Obtain doing a slight perturbation on t ky
y LocalSearch y
14A study of simulated annealing variants
ASALO variant
Based on ASA and SALO algorithms.
Generation of a new point Like ASA algorithm.
If y is infeasible, applies the reflection technique proposed by Romeijn and Smith
If the new point y is accepted
Simulated annealing variants
if if
if
i i i i i
i i i i i
i i i i i
a a y y ar y y a y b
b y b y b
y LocalSearch r y
15A study of simulated annealing variants
SSA
CSA
Computational results – Characterization of the presented variants
0 and .cCrucial parameters:
16A study of simulated annealing variants
ASA
SALO
ASALO
Computational results – Characterization of the presented variants
0 and .c Crucial parameters:
17A study of simulated annealing variants
Computational results – Comparison of the presented variants
Test functions n NFE NAP
B 2 1000 311 -0.3978875 -0.3978874
GP 2 1000 306 -3.0000004 -3.0000001
R2 2 26015 20745 -0.0394671 -0.0049240
R4 4 64262 4131 -0.0362470 -0.0230065
Me3 3 1000 122
H3 3 2068 374 3.8627819 3.8627821
Ra4 4 2680 174
ASA
algorithm
ASALO
algorithm
*g*mg
82.8 10
79.4 10
105.3 10
88.3 10
Test functions n NFE NAP
B 2 15531 284 -0.3978874 -0.3978874
GP 2 15944 285 -3 -3
R2 2 16671 294
R4 4 40923 577 -0.0265422 -0.0049288
Me3 3 5717 98
H3 3 15237 260 3.8627815 3.8627821
Ra4 4 10293 143
*g*mg
94.0 10
61.6 10
112.6 10
85.5 10
62.8 10 88.8 10
18A study of simulated annealing variants
Conclusions
SSA variant requires a large number of function evaluations.
SSA and CSA variants recognize more than one solution, when the problem has more than one global maximum.
CSA is the variant that has more accepted points.
ASA variant requires a reduced number of function evaluations.
19A study of simulated annealing variants
Conclusions
ASA variant did not converge to a global maximum in some runs of some problems.
CSA and ASALO variants provide better approximations to the global maximum.
SALO and ASALO variants have a small number of accepted points.
CSA and ASALO variants did not converge to a global maximum only in one run of a problem.
20A study of simulated annealing variants
A study of simulated annealing variants
Ana PereiraPolytechnic Institute of Braganca, Portugal
email: [email protected]
Edite FernandesUniversity of Minho, Braga, Portugal
email: [email protected]