Parameter Control for Evolutionary Algorithms,Constrained Problems and Constraint-Handling
Techniques
FIT4012 Advanced topics in computational science
September 1, 2014
Parameter control for Evolutionary Algorithms
I Parameters of Evolutionary Algorithms
I Parameter Control
I Adaptive Parameter Control
I Adaptive Range Parameter Control
Parameters of Evolutionary Algorithms
I Solution representation: the search-spaceI Strategy parameters: how to search
I Crossover operator: uniform, single-point, etc.I Crossover rate: [0.6, 1.0]I Mutation operator: uniform, single-point, etc.I Mutation rate: [0.001, 0.5]I Population size: [1, 200] etc.
Problem
I Setting algorithm parameters is computationally expensive.
I Different parameter values are required for different stages ofthe optimisation process.
Parameter Control
1. DeterministicI mr0 = 0.5, mrt = mrt−1 − 0.01, t = {10, 20, ..., n}
2. Self-adaptive
I 1 1 0 1 0 1 0 1 0 0 0.3
3. AdaptiveI Feedback mechanism: pt(υij), qt(υij)
Adaptive Parameter Control
A genetic algorithm
Stopping criterionTrue
False
Evolve solution(s)
Evaluate solution(s) Final solutionsInitial population
A genetic algorithm with adaptive parameter control
Stopping criterionTrue
False
Quality attribution
Effect assessment
Feedback collection
Selection Evolve solution(s)
Evaluate solution(s) Final solutionsInitial population
State-of-the-art Adaptive Parameter Control Strategies
I Probability Matching (PM),
pt(υij) = pmin + (1− n ∗ pmin)qt(υij)∑ns=1 qt(υis)
I Adaptive Pursuit (AP),
pt(υij) =
pt−1(υij) + β(pmax − pt−1(υij)) if j = j∗pt−1(υij) + β(pmin − pt−1(υij)) otherwise
I Dynamic Multi-Armed Bandits (DMAB).
Issues
I The choice of parameter assignment is made based on staticpredefined ranges.
I E.g. crossover rate: [0.6, 0.8], [0.8, 1.0]
I Defining narrow ranges leads to more accuracy but increasedcombinatorial complexity.
I Wider ranges entail a sampling inaccuracy.
Research question and approach
I What is an effective method for configuring real-valuedparameters during the optimisation process?
I Ideally, the ranges should be optimised by the parametercontrol process.
Research question and approach
I What is an effective method for configuring real-valuedparameters during the optimisation process?
I Ideally, the ranges should be optimised by the parametercontrol process.
Adaptive Range Parameter Selection
υi2υi1 p(e+|υi2)p(e+|υi1) { }−−−−−−−︸ ︷︷ ︸[υi1(min),υi1(max)]
−−−−−−−︸ ︷︷ ︸[υi2(min),υi2(max)]
υi2υi1
−−−−−−−︸ ︷︷ ︸[υi1(min),υi1(max)]
−−−−−−−︸ ︷︷ ︸[υi2(min),υi2(max)]
} p(e+|υi2){p(e+|υi1)
υi2υi11
−−−−︸ ︷︷ ︸[υi11 (min),υi11 (max)]
−−−−−−−︸ ︷︷ ︸[υi2(min),υi2(max)]
} p(e+|υi2){p(e+|υi11) υi12[υi12 (min),υi12 (max)]︷ ︸︸ ︷
−−−−
p(e+|υi12)
Adaptive Range Parameter Control
υi2υi11
−−−−︸ ︷︷ ︸[υi11 (min),υi11 (max)]
−−−−−−−︸ ︷︷ ︸[υi2(min),υi2(max)]
} p(e+|υi2){p(e+|υi11) υi,12[υi12 (min),υi12 (max)]︷ ︸︸ ︷
−−−
p(e+|υi12)
}
υ′i2υi1
−−−−︸ ︷︷ ︸[υi1(min),υi1(max)]
−−−−−−−−−−︸ ︷︷ ︸[υ′
i2(min),υ′
i2(max)]
}p(e+|υ′i2){p(e+|υi1)
Experimental Design
I Controlled parameters:I Mutation rate,I Crossover rate.
I Benchmark problems:I Royal Road,I Quadratic Assignment Problem (QAP),I Multiobjective Quadratic Assignment Problem (mQAP),I Component Deployment Problem (Scheduling length and
communication overhead).
I Benchmark adaptive parameter control methods:I Probability Matching (PM),I Adaptive Pursuit (AP),I Dynamic Multi-Armed Bandits (DMAB): statistical tests to
restart parameter rewards
I 30 trials.
Results
Royal Road QAP
0.486
0.4865
0.487
0.4875
0.488
0.4885
0.489
PM AP DMAB ARPS
Nor
mal
ised
fitn
ess
0.2655
0.266
0.2665
0.267
0.2675
0.268
PM AP DMAB ARPS
Nor
mal
ised
fitn
ess
mQAP Component deployment
0.84
0.86
0.88
0.9
0.92
0.94
0.96
PM AP DMAB ARPS
Hyp
ervo
lum
e
0.9
0.91
0.92
0.93
0.94
0.95
0.96
0.97
0.98
PM AP DMAB ARPS
Hyp
ervo
lum
e
Kolmogorov-Smirnov test
Royal Road QAP mQAP
d p d p d p
ARPS vs. DMAB 0.3414 0.000 0.5172 0.000 0.5172 0.000ARPS vs. AP 0.3793 0.000 0.5172 0.000 0.6207 0.000ARPS vs. PM 0.4759 0.000 0.5517 0.000 0.5172 0.000
Parameter ranges
Crossover rate
0.6
0.65
0.7
0.75
0.8
0.85
0.9
0.95
1
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
Cros
sove
r ra
te
Iterations
0.6
0.65
0.7
0.75
0.8
0.85
0.9
0.95
1
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
Cros
sove
r ra
te
Iterations
Mutation Rate
0
0.1
0.2
0.3
0.4
0.5
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
Mut
atio
n ra
te
Iterations
0
0.1
0.2
0.3
0.4
0.5
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
Mut
atio
n ra
te
Iterations
Predictive parameter control
The past effectiveness of parameter values provides the informationnecessary to project their chance of leading to good results atiteration t + 1
0 10 20 30 40 50
0.45
0.50
0.55
0.60
0.65
Iterations
Suc
cess
rate
PPC
I The effect of a parameter value is calculated as:
et(vij) =ns(υij)
n(υij)(1)
I The effect of a parameter value over time is the time series
e1(υij), e2(υij), ..., et(υij). (2)
I A predictive model in adaptive parameter control has thegeneral form:
qt(υij) = f (e1(υij), e2(υij), ..., et(υij), �) (3)
Predictive models
I Linear regression
qt(υij)− et(υij)σ(et(υij))
= r · t − tσ(t)
(4)
I Simple moving average
qt(υij) =et−1(υij) + ...+ et−k(υij)
k(5)
I Exponentially weighted moving average
qt(υij) =t∑
k=0
α(1− α)ket−(k+1)(υij) (6)
I Autoregressive Integrated Moving Average
qt(υij) = φ1et−1(υij) + �t (7)
Results
Assumptions of the forecasting models
I Linearity: can you fit a linear model?
I Normality: are the errors normally distributed?
I Independence: are the error terms independent?
I Homoscedasticity: is the variance in errors similar over time?
I Stationarity: does the mean and standard deviation remainthe same over time?
Linearity Normality Independence Homoscedasticity Stationarity
LR + + + + -
SMA - - - + +
EWMA - - - + +
ARIMA - + + + +
Statistical tests
I Linearity: fit a linear model and report the p − valueI Normality: Kolmogorov-Smirnov (KS) test
I Independence: Durbin-Watson (DW) statistical test
I Homoscedasticity: Breusch-Pagan test
I Stationarity: Kwiatkowski-Phillips-Schmidt-Shin (KPSS) test
Ranges/values of parameters analysed
Parameter Ranges/values
Mutation rate - 2 ranges [0.001,0.249], [0.25,0.49]
Mutation rate - 4 ranges [0.001,0.1249], [0.125,0.249], [0.25,0.3749], [0.375,0.49]
Crossover rate - 2 ranges [0.6,0.79], [0.8,0.99]
Crossover rate - 4 ranges [0.6,0.69], [0.7,0.79], [0.8,0.89], [0.9,0.99]
Population size - 2 ranges [20,60], [61,100]
Population size - 4 ranges [20,40], [41,60], [61,80], [81,100]
Mating pool size - 2 ranges [0.1,0.39], [0.4,0.69]
Mating pool size - 4 ranges [0.1,0.249], [0.25,0.39], [0.4,0.549], [0.55,0.69]
Mutation operator Single-point, Uniform
Crossover operator Single-point, Uniform
Results
Characteristics of the effects of the discrete parameter values. Thepercentages represent the percentage of times (of a total 30) H0was not rejected.
Parameter Problem Linear Normal Ind.t Hom. Stat.Single-point QAP 100% 100% 70% 93% 97%mutation MQAP 100% 97% 70% 83% 83%
Royal Road 90% 90% 73% 87% 87%Uniform QAP 100% 100% 77% 93% 93%mutation MQAP 100% 97% 80% 83% 90%
Royal Road 97% 87% 73% 87% 90%Single-point QAP 100% 100% 77% 87% 93%crossover MQAP 100% 97% 80% 83% 90%
Royal Road 97% 93% 73% 87% 87%Uniform QAP 100% 100% 77% 97% 93%crossover MQAP 100% 97% 77% 83% 93%
Royal Road 100% 90% 73% 87% 83%
Recommendations on the suitability of forecasting methods
Parameter LR SMA EWMA ARIMA
Mutation rate + + + +
Crossover rate + + + +
Population size ∼ ∼ ∼ ∼Mating pool size + + + +
Mutation operator + + + +
Crossover operator + + + +
Performance comparison
Average Quality Attribution (AQA), Extreme Quality Attribution(EQA) and Predictive Quality Attribution (PQA)Royal Road QAP
0.236
0.238
0.24
0.242
0.244
0.246
0.248
AQA EQA PQA
Nor
mal
ised
fitn
ess
0.804
0.8045
0.805
0.8055
0.806
0.8065
0.807
0.8075
0.808
AQA EQA PQA
Hyp
ervo
lum
emQAP Component deployment
0.19
0.195
0.2
0.205
AQA EQA PQA
Hyp
ervo
lum
e
0.78
0.79
0.8
0.81
0.82
0.83
0.84
0.85
AQA EQA PQA
Nor
mal
ised
fitn
ess
Means and standard deviations
The means and standard deviations over 30 runs of AverageQuality Attribution (AQA), Extreme Quality Attribution (EQA)and Predictive Quality Attribution (PQA).
Mean Standard Deviation
Problem AQA EQA PQA AQA EQA PQA
CHR20A 0.4909 0.4845 0.4972 1.627E-02 1.306E-02 1.920E-02
BUR26E 0.2416 0.2395 0.2422 2.049E-03 1.893E-03 2.365E-03
TAI30A 0.4909 0.4845 0.4982 1.627E-02 1.306E-02 1.870E-02
STE36B 0.7978 0.7984 0.8308 7.823E-03 9.038E-03 8.704E-03
Royal Road 0.0064 0.0063 0.0072 1.537E-04 6.3723E-03 1.473E-03
KC10-2fl-5rl 0.8062 0.8061 0.8067 1.040E-03 8.284E-04 9.555E-04
KC30-3fl-1rl 0.1984 0.1961 0.1995 3.255E-03 3.974E-03 3.806E-03
KC30-3fl-2rl 0.5199 0.5175 0.5223 7.312E-03 5.439E-03 4.821E-03
Statistical analysis
The Kolmogorov-Smirnov test values for the 30 runs of AverageQuality Attribution (AQA), Extreme Quality Attribution (EQA)and Predictive Quality Attribution (PQA)
PQA vs. AQA PQA vs. EQA
Problem d p d p
CHR20A 0.2333 0.042 0.3333 0.045
BUR26E 0.3000 0.048 0.3000 0.049
TAI30A 0.3294 0.046 0.3517 0.039
STE36B 0.9667 0.000 0.9667 0.000
Royal Road 0.6333 0.000 0.7333 0.000
KC10-2fl-5rl 0.3333 0.045 0.4000 0.011
KC30-3fl-1rl 0.2897 0.016 0.4333 0.005
KC30-3fl-2rl 0.3433 0.035 0.4333 0.005
Future research
Search Space
Global optimum
Local optima
Variable 1 Variable 2
FunctionvalueCharacterisation
Modality
Uniformity
Diversity
Learning
Bayesian Networks
Forecasting
Control Theory
Adaptation
Local search
EA
ACO
Constrained Problems
The component deployment problem
Example: Reliability Optimisation in Embedded Systems
ECU3
ECU1
ECU2
ECU5
ECU6
ECU7
ECU8Bus0 (CAN)
Bus2(Rear LIN)
ECU0
Bus1(Front LIN)
ECU4
EmergencyStopDetector
1
HMIOutputs
12DistanceCalc
13
14SpeedCalc
ObjectRecogn-ition
10
SpeedLimitter
8
ModeSwitch
9ABSMainUnit
0
LoadCompen-sator
3
5WSR-F
6WAC-R
7WAC-F
BrakePedalSensor
2
4WSR-R
ACCMainUnit
11
WAC : Wheel Actuator Controllers (Front and Rear) WSR : Wheel Sensor Readers (Front and Rear)
Reliability function
R ≈∏
i∈IRvc (ci )i ·
∏
i ,j∈IRvl (lij )ij (8)
I Reliability of a component: Ri = e−fr(d(ci ))·
wl(ci )
ps(d(ci ))
I Reliability of a link: Rij = e−fr(d(ci ),d(cj ))·
ds(ci ,cj )
dr(d(ci ),d(cj ))
I Expected number of visits for a component:vc(ci ) = q0(ci ) +
∑j∈I(vc(cj) · p(cj , ci ))
I Expected number of visits for a link:vl(lij) = 0 +
∑x∈{i}(vc(cx) · p(cx , lij))
Constraints
I Memory
mem(d) = ∀h ∈ H :∑
Ch∈d−1(h)
mc(Ch) ≤ mh(h) (9)
I Colocation constraints
coloc(d) = ∀c ∈ C : (h ∈ cr(ci , cj) ⇒ d(ci ) 6= d(cj) (10)
I Communication
com(d) = ∀c ∈ C : (h ∈ cm(ci , cj) ⇒ ld(ci ),d(cj ) ≥ 0) (11)
Constraint optimisation problem formulation
maximise R ≈ ∏i∈I Rvc (ci )i ·
∏i ,j∈I
subject to ∀h ∈ H : ∑Ch∈d−1(h)
mc(Ch) ≤ mh(h)
∀c ∈ C : (h ∈ cr(ci , cj) ⇒ d(ci ) 6= d(cj)
∀c ∈ C : (h ∈ cm(ci , cj) ⇒ ld(ci ),d(cj ) ≥ 0)
Constraint handling
I Eliminating infeasible candidates
I Penalizing functions
I Repairing infeasible candidates
Eliminating infeasible candidates: death penalty
Advantages
I The algorithm does not spend a significant amount of timeevaluating illegal individuals.
Disadvantages
I For some problems the probability of generating a feasiblesolution is relatively small
I Infeasible regions may serve as a bridge between feasibleregions.
I In this approach non-feasible solutions do not contribute tothe gene-pool of any population
Penalising functions
Generating potential solutions without considering the constraintsand then to penalize them by decreasing the ‘goodness’ of theevaluation function.
I assign a constant as a penalty measure
I assign a penalty measure depend on the degree of violation:the larger violation is, the greater penalty is imposed
I the growth of the penalty can be logarithmic, linear,quadratic, exponential, etc.
Example:
R ≈∏
i∈IRvc (ci )i ·
∏
i ,j∈IRvl (lij )ij − w (12)
Repair algorithms
‘Correct’ any infeasible solutions so generated.
Disadvantages
I Such repair algorithms might be computationally intensive torun and the resulting algorithm must be tailored to theparticular application.
I Moreover, for some problems the process of correcting asolution may be as difficult as solving the original problem.
Example:
I Change the allocation of a software component to a differenthardware unit.
I Swap the allocations of two software components.
Are constrained problems more difficult than unconstrainedproblems?
10% 25% 50% 75% 100%
5.0e-08
1.5e-07
Interactions
Pre
dict
or e
rror
Constrained problems can be easy
10% 25% 50% 75% 100%
0.999980
0.999990
Interactions
Fitness
The end