UMDAs for Dynamic Optimization Problems

GECCO08 - AtlantaFernandes, Lima and Rosa – “UMDAs for DOPs”

UMDAs for Dynamic Optimization Problems

Carlos Fernandes1,2

Claudio Lima3

Agostinho C. Rosa1

1LaSEEB-ISR-IST, Technical Univ. of Lisbon (IST), Portugal2 Department of Architecture and Computer Technology, University of Granada, Spain3 Informatics Laboratory, University of Algarve, Portugal


Motivation and Objectives

•Design an update strategy for UMDAs to deal with Dynamic Optimization Problems (DOPs), based on ACO.

Problem: full convergence.When solving DOPs, finding the optima is not the main task; the algorithm must track the optima.Full convergence (without ways to escape it) is not suitable for DOPs (even if the population converges to the global optimum)

Solution: delay or avoid full convergence.Change probability distribution.Maintain probability distribution except when close to 0 and 1

Combination of strategies



•Proposal: Update strategy for UMDA based on ACO equations.

•Effects: Change probability distribution.

•Why? Reinforcement/evaporation equations allow us to control the convergence speed of the algorithm

ACO algorithms build solutions by travelling trough the nodes of a (combinatorial) problem. (TSP, for instance…)

After the evaluation of the solutions, edges that belong to good solutions are reinforced (pheromone)

Each generation, pheromone in all edges is evaporated – diversity, mutation.

In the following generation, solutions/paths are chosen according to the amount of pheromone in each connection between nodes.



ACO Univariate EDA

initialize pheromone in all edgesrepeat sample N solutions evaluate solutions update pheromoneuntil stop criterion

initialize probability modelrepeat sample N solutions evaluate solutions update model parametersuntil stop criterion



The Binary Ant Algorithm (presented in GECCO07). Based on Ant Colony Optimization (ACO), BAA builds pheromone trails between binary variables.

•BAA is a kind of Estimation of Distribution Algorithm (EDA)

There are similarities between EDAs and ACO. The pheromone trails are similar to the probability models, and reinforcement/evaporation is ACO’s update strategy.

0 0 0

1 1 1

0

1

0

1

Solution Dimension

BAA


Reducing Diversity LossBranke, J., Lode, C., and Shapiro, J. 2007. Adressing sampling errors and diversity loss in UMDA. Proceedings of the 2007 Genetic and Evolutionary Computation Conference, ACM, 508-515.

•Permutation Sampling: reduces loss due to sampling; used in all the experiments

•Loss Correction (LC)

•Laplace Correction

•Iterated Laplace Correction (iLaplace)

•Boundary Correction: changes probability distribution near 0 and 1

Changes probability distribution


Reducing Diversity Loss


UMDASet γi ← 1/2 for all i = 1 . . .L;repeat Sample N strings according to make a population D. Generate a new population Ds from D by selecting the f×N fittest strings. for i = 1 to L do update model:

end foruntil stop criterion

Replace by ACO-like equations


Reinforcement/Evaporation (RE) Update

Set γi ← 1/2 for all i = 1 . . .L; Set ← 0 for all i = 1 . . .L; Set α and βrepeat sample N strings according to make a population D. generate new population Ds by selecting the f×N fittest strings. update pheromone

evaporate for i = 1 to L do update model end foruntil stop criterion met

β = 1

+

+

If α = 1 and β = 1, we have the standard update strategy

1 0 1 1…


Results – Diversity Loss in Flat Landscape

β = 1; N = 20; f = 0.5; L = 100 α = 1; N = 20; f = 0.5; L = 100


Test Set Severity of change: This criterion establishes how strongly the problem is changing

Speed of change: This criterion establishes how often the environment changes

Yang and Yao’s dynamic problems generator*

•By using a binary mask, dynamic environments are created by applying the mask to each solution before its evaluation.

•Severity of change is controlled by setting the number of 1’s in the mask.

•Speed of change is controlled by defining the number of generations between the application of a different mask.

*Yang, S. and Yao, X. 2005. Experimental study on PBIL algorithms for dynamic optimization problems. Soft Computing 9(11), 815-834.


Test Set

•Functions

•Onemax

•Royal Road

Speed was set to = 10, 100 (generations)𝜏Severity was set to ρ = 0.05, 0.6 and 0.95

6 different scenarios

Performance is measured by the mean best-of-generation values, i.e., best fitness averaged over all generations, and then over all runs

30 runs for each configuration

Compared RE with Loss, Laplace, iterated Laplace and Boundary Correction


Results – RE Parameters

β = 1 β = 0.5

Royal Road

•Decreasing β improves performance when speed is high and severity is low

•β = 1 and α = 1 is standard UMDA’s update strategy


Results

Onemax Royal Road

τρ

100.05

100.6

100.95

1000.05

1000.6

1000.95

100.05

100.6

100.95

1000.05

1000.6

1000.95

1 2

RE1 iLap − + + − − − − + ~ ~ ~ ~

RE1 LC + + + − − − + + + + + +

RE2 iLap − + + ~ ~ + + + + + ~ ~

RE2 LC ~ + + ~ ~ + + + + + + +

iLap LC + + + ~ ~ ~ + + + + + +

Comparing strategies that do not avoid full convergence

RE1: α = 0.6, β = 1RE2: α = 1, β = 0.5


Results

Table 2. Statistical analysis of the results in table 1.

Onemax Royal Road

τρ

100.05

100.6

100.95

1000.05

1000.6

1000.95

100.05

100.6

100.95

1000.05

1000.6

1000.95

1 2

RE1+BC iLap+BC ~ + + ~ + + + + + + + +

RE1+BC Laplace − + + − + + + + ~ + + +

RE1+BC LC+BC − + + ~ + + + + ~ + + +

RE1+BC BC − + + ~ + + + + + + + +

RE2+BC iLap+BC − + + − − − + + ~ + + +

RE2+BC Laplace − + + − − − ~ + ~ + + +

RE2+BC LC+BC − + + − − − ~ ~ ~ ~ − ~

RE2+BC BC − + + − − − + + ~ + + +

Avoiding full convergence

RE1: α = 0.8, β = 1RE2: α = 0.9, β = 0.5

*Laplace performs well when compared to iL+BC, LC+BC and BC


Results

Dynamic Royal Road with = 100. 𝜏 β = 1 (α = 1 curves correspond to the standard UMDA update strategy)

ρ = 0.05 ρ = 0.6 ρ = 0.9

RE


Results

Table 2. Statistical analysis of the results in table 1.

Dynamic Royal Road with = 100. 𝜏 α = 0.8, β = 1

RE Laplace


ConclusionsRE is capable of outperforming other diversity loss correction techniques

•RE performs well when compared to Loss Correction and Iterated Laplace Correction

•RE with boundary correction outperforms other strategies in a wide range of scenarios

Laplace Correction attains better results than other techniques (except RE)

Diversity of the UMDA with RE may be controlled by α and β parameters

RE seems to work well with α between 0.6 and 0.9, depending on β and depending if we hybridize it with Boundary Correction.


Conclusions

RE strategy works well without needing to know when the fitness changes. There DOPs with changes that are not detectable (or that are too costly to detect).


Future work

In-depth study of the effects of parameter values on performance.

Understand how α and β affect UMDA’s behavior.

Tests on dynamic trap functions.

Extend the strategy to other EDAs.

Education

UMDAs for Dynamic Optimization Problems