Let's get ready to rumble redux: Crossover versus mutation head to head on exponentially scaled problems
This paper analyzes the relative advantages between crossover and mutation on a class of deterministic and stochastic additively separable problems with substructures of non-uniform salience. This study assumes that the recombination and mutation operators have the knowledge of the building blocks (BBs) and effectively exchange or search among competing BBs. Facetwise models of convergence time and population sizing have been used to determine the scalability of each algorithm. The analysis shows that for deterministic exponentially-scaled additively separable, problems, the BB-wise mutation is more efficient than crossover yielding a speedup of (l logl), where l is the problem size. For the noisy exponentially-scaled problems, the outcome depends on whether scaling on noise is dominant. When scaling dominates, mutation is more efficient than crossover yielding a speedup of (l logl). On the other hand, when noise dominates, crossover is more efficient than mutation yielding a speedup of (l).
1.Lets Get Ready to Rumble Redux:Crossover vs. Mutation Head to Head on Exponentially-Scaled ProblemsKumara Sastry1,2 and David E. Goldberg1 1IllinoisGenetic Algorithms Laboratory 2Materials Computation Center University of Illinois at Urbana-Champaign, Urbana IL 61801 http://www.illigal.uiuc.edu firstname.lastname@example.org, email@example.com Supported by AFOSR FA9550-06-1-0096 and NSF DMR 03-25939. 2. Motivation Great debate between crossover and mutation When mutation works, its lightning quick When crossover works, it tackles more complex problems Compare crossover and mutation where both operators have access to same neighborhood information Local search literatureEmphasis on good neighborhood operators [Barnes et al, 2003;Watson, 2003; Hansen et al, 2001]Need for automatic induction of neighborhoodsLeads to adaptive time continuation operator [Lima et al 2005, 2006, 2007] 3. OutlineRelated work Assumption of known or discovered linkage Objective Algorithm Description Scalability analysis: Crossover vs. MutationKnown or discovered linkageExponentially scaled additively-separable problem with andwithout Gaussian noiseSummary and Conclusions 4. BackgroundEmprical studies comparing crossover and mutationScalability of GAs and mutation-based hillclimber [Mhlenbein, 1991 & 1992; Mitchell, Holland, and Forrest, 1994; Baum, Boneh, and Garett, 2001; Dorste, 2002; Garnier, 1999; Jansen and Wegener, 2002, 2005]Single GA run with large population vs. multiple GA runs with small population at fixed computational cost [Goldberg, 1999; Srivastava & Goldberg, 2001; Srivastava, 2002; Cant-Paz & Goldberg, 2003; Luke, 2001; Fuchs, 1999]Used fixed operators that dont adapt linkageDid not consider problems of bounded difficulty Linkage and neighborhood information is critical 5. Known or Discovered LinkageAssumption of known or induced linkageCan use linkage-learning techniques Linkage information is critical for selectorecombinative GA success Exponential Polynomial ScalabilityPelikan, Ph.D. Thesis, 2002Provide the same information for mutationMutation searches in the building-block subspace 6. Algorithm DescriptionSelectorecombinative genetic algorithm Population of size n Binary tournament selection Uniform building-block-wise crossover BBs #1 and #3 exchangedExchange BBs with probability 0.5Selectomutative genetic algorithm Start with a random individual Enumerative BB-wise mutationConsider BB partitions Arbitrary left-to-right orderChoose the best schemata Among the 2k possible ones 7. Crossover Versus Mutation: Uniform ScalingDeterministic fitness: Noisy fitness: Recombination Mutation is more efficient is more efficient[Sastry & Goldberg, 2004] 8. ObjectiveCrossover and mutation both have access to same neighborhood information Known or discovered linkage Recombination exchanges building blocks Mutation searches for the best BB in each partitionCompare scalability of crossover and mutation Additively separable problems with exponentially-scaled BBsWith and without additive Gaussian noise Where do they excel?Derive, verify, and use facetwise models Convergence time and population sizing 9. Scaling and Noise Cover Most ProblemsAdversarial problem design [Goldberg, 2002]FluctuatingRP NoiseDeceptionScaling Noisy BinInt 10. Convergence Time for Crossover: Deterministic Fitness FunctionsSelection-Intensity based model [Rudnick, 1992; Thierens et al, 1998] Derived for the BinInt problem Applicable to additively-separable problems Selection Intensity Problem size (mk ) 11. Population Sizing for Crossover:Deterministic Fitness FunctionsDomino convergence [Rudnick, 1992] ProportionMostLeastsalient salientBB convergence in order of salience...Drift bound dictates population sizingDrift time [Goldberg and Segrest, 1987]timeSize the population such that: Population size: 12. Scalability Analysis of Crossover & Mutation:Deterministic Fitness Functions Selectorecombinative GAPopulation size: Convergence time: Number of function evaluations:Selectomutative GAInitial solution is evaluated once2k 1 evaluations in each of m partitions 13. Crossover vs. Mutation:Deterministic Fitness FunctionsSpeed-Up: Scalability ratio of mutation to that of crossover 14. Convergence Time for Crossover:Noisy Fitness FunctionsAdditive Gaussian noise with variance 2NSet proportional to maximum fitness variance Scaling dominated: Noise dominated: 15. Population Sizing for Crossover:Noisy Fitness FunctionsScaling dominated: Noise dominated: 16. Scalability Analysis of Mutation: Noisy Fitness Functions Fitness should be sampled to average out noiseWhat should the sample size, ns, be?BB-wise decision making [Goldberg, Deb, & Clark, 1992] Square of the ordinate of a one-sided Gaussian deviate with specified error probability, 17. Scalability Analysis of Crossover & Mutation:Noisy Fitness Functions Selectorecombinative GASelectomutative GAFitness of each individual is sampled ns times2k 1 evaluations in each of m partitions 18. Crossover vs. Mutation: Noisy BinInt Speed-Up: Scalability ratio of crossover to that of mutation 19. SummaryDeterministic fitness: Noisy fitness: Recombination Mutation is more efficient is more efficient in noisedominated regime 20. ConclusionsGood neighborhood information is essential Quadratic scalability of crossover and mutation Exponential scalability of simple crossover [Thierens & Goldberg, 1994]ekmk scalability of simple mutation [Mhlenbein, 1991]Leads to a theory of time continuation Key facet of efficiency enhancementLeads to principled design and development of adaptive time continuation operators Promise of yielding supermultiplicative speedups