Embed Size (px)
DESCRIPTION
Presentation by Luigi Malago at the Optimization by Building and Using Probabilistic Models (OBUPM 2008) workshop at the Genetic and Evolutionary Computation Conference (GECCO-2008)
Citation preview
An Information Geometry Perspective on Estimation of Distribution Algorithms: Boundary Analysis
Luigi Malagò, Matteo Matteucci, Bernardo Dal Seno
OBUPM 2008, July 13GECCO’08, Atlanta, Georgia, USA
DEPARTMENT OF ELECTRONICS AND INFORMATION http:/www.dei.polimi.it
AIRLab Artificial Intelligence and Robotics Laboratoryhttp:/www.airlab.elet.polimi.it
L. Malagò, OBUPM 2008
Agenda
Motivations and Context
Estimation of Distribution Algorithms (EDAs)
Notions of Information Geometry (IG)
Different parametrization for a statistical manifold
Interpretation of the behavior of an EDA
Simple examples
Conclusions and future work
L. Malagò, OBUPM 2008
Context
Amari, Shun-ichi (2001). Information geometry on hierarchy of probability distributions. IEEE Transactions on Information Theory, 47(5), 1701-1711
Toussaint, Marc (2004). Notes on information geometry and evolutionary processes. Los Alamos pre-print nlin.AO/0408040.
Does Information Geometry provide (useful) insightsin the study of EDAs?
L. Malagò, OBUPM 2008
Estimation of Distribution Algorithms
a.k.a. Probabilistic Model-Building Optimization Genetic Algorithms (PMBGAs)
make use of a probabilistic model
replace crossover and mutation with estimation and sampling
L. Malagò, OBUPM 2008
Estimation of Distribution Algorithms
a.k.a. Probabilistic Model-Building Optimization Genetic Algorithms (PMBGAs)
make use of a probabilistic model
replace crossover and mutation with estimation and sampling
Notation
: Population of candidate solutions
: Random vector of binary variables
: Parametrized probability model
L. Malagò, OBUPM 2008
Estimation of Distribution Algorithms
Many different algorithms, according to
choice of the model
model building and model fitting
model evaluation
sampling techniques
L. Malagò, OBUPM 2008
Estimation of Distribution Algorithms
Many different algorithms, according to
choice of the model
model building and model fitting
model evaluation
sampling techniques
In the literature, EDAs are often classified as
Univariate: no dependencies
Bivariate: pairwise dependencies
Multivariate: higher order dependencies
The choice of can determine convergenceto local optimum!
L. Malagò, OBUPM 2008
EDA Classification
INDEPENDENT VARIABLES(PBIL, UMDA, cGA, DEUM)
CHAIN MODEL(COMIT)
TREE MODEL(MIMIC)
FOREST MODEL(BMDA)
UNDIRECTED GRAPH(FDA, LFDA, DEUM, MN-EDA)
INDEPENDENT CLUSTERS(eCGA)
BAYESIAN NETWORKS(BOA, EBNA)
Univariate Bivariate Multivariate
L. Malagò, OBUPM 2008
Information Geometry
studies properties of families of probability distributions by means of differential geometry
is a rather theoretical framework with an increasing number of applications
A parametric statistical model can be regarded as a manifold of distributions, where the Fisher Information matrix plays the role of metric tensor
L. Malagò, OBUPM 2008
Information Geometry
studies properties of families of probability distributions by means of differential geometry
is a rather theoretical framework with an increasing number of applications
A parametric statistical model can be regarded as a manifold of distributions, where the Fisher Information matrix plays the role of metric tensor
L. Malagò, OBUPM 2008
Contingency tables and log-linear models
Consider a vector of binary variables
L. Malagò, OBUPM 2008
Contingency tables and log-linear models
Consider a vector of binary variables
L. Malagò, OBUPM 2008
Contingency tables and log-linear models
Consider a vector of binary variables
Mean parameters
L. Malagò, OBUPM 2008
Contingency tables and log-linear models
Consider a vector of binary variables
Natural parameters Mean parameters
L. Malagò, OBUPM 2008
Contingency tables and log-linear models
Consider a vector of binary variables
Natural parameters
Due to orthogonality among mean and natural parameters, we can employ a k-cut mixed parametrization
Mean parameters
L. Malagò, OBUPM 2008
k-cut mixed parametrization
L. Malagò, OBUPM 2008
k-cut mixed parametrization
L. Malagò, OBUPM 2008
k-cut mixed parametrization
L. Malagò, OBUPM 2008
k-cut mixed parametrization
L. Malagò, OBUPM 2008
k-cut mixed parametrization
L. Malagò, OBUPM 2008
k-cut mixed parametrization
L. Malagò, OBUPM 2008
k-cut mixed parametrization
L. Malagò, OBUPM 2008
Analysis of the border of the manifold
L. Malagò, OBUPM 2008
Analysis of the border of the manifold
L. Malagò, OBUPM 2008
Analysis of the border of the manifold
L. Malagò, OBUPM 2008
Interpretation of EDA operatorswithin the manifold of distributions
L. Malagò, OBUPM 2008
A Simple Example 1/2
Independence model
L. Malagò, OBUPM 2008
A Simple Example 1/2
Independence model Model
L. Malagò, OBUPM 2008
A Simple Example 2/2
Independence model
L. Malagò, OBUPM 2008
A Simple Example 2/2
Independence model Model
L. Malagò, OBUPM 2008
Conclusions and Future Work
Information Geometry provides an interesting perspective on the study of the behavior of EDAs
Direction of research
Formalization of the ideas presented by means of mathematical proofs
Convergence results for specific classes of problems according to the model used by the EDA
L. Malagò, OBUPM 2008
Conclusions and Future Work
Information Geometry provides an interesting perspective on the study of the behavior of EDAs
Direction of research
Formalization of the ideas presented by means of mathematical proofs
Convergence results for specific classes of problems according to the model used by the EDA
From discrete to continuous EDAs
Proposal of new meta-heuristics based on IG principles
Extension of the framework to other meta-heuristics that employ statistical models
![Pandola - Geant4 - 1 [modalità compatibilità] · Geometry Geometryalso entersinto the tracking A stepcan never cross a geometry boundary Always stop the step when there is a boundary,](https://img.dokumen.tips/doc/110x75/6021c20221e951343b5096a4/pandola-geant4-1-modalit-compatibilit-geometry-geometryalso-entersinto.jpg)
