Parallel Genetic AlgorithmsBy Larry Hale and Trevor McCasland

Introduction to Genetic Algorithms

• Genetic algorithms are search algorithms that use the principles of natural selection to find more optimal solutions to modeling, simulation, and optimization problems.

• Generate “good but not optimal” solutions to brute-forceable problems

• They then recombine these in an attempt to find an optimal or near optimal solution

• These candidate solutions are then evaluated for “fitness”. • Fitness is computationally expensive to evaluate

Biological Background

• Individual - Any possible solution• Population - The working group of all individuals• Gene - A particular trait of an individual• Chromosome - A set of genes that make a model for an individual • Genome - Set of all chromosomes of an individual• Genotype - Particular set of genes in a genome of an individual• Phenotype - Physical and mental characteristics of an individual

Search Space

• All possible solutions

Solution

Fitness

Selection

• Deciding which members of the population that will be used to generate new solutions through either of the following• Random selection• Selection proportional to

fitness

Evaluation

• Fitness - each individual is evaluated for its effectiveness• For example, fitness of the following individual is 23 - 10 = 13

• 23 - best possible fitness• 10 - inferior genes• 13 - resulting fitness score

10010101000010101010100Solution

10100101010101010010101Individual

Evaluation Continued

• A good fitness evaluation gives a fitness that is proportional to how close to the optimal solution it is

• A more practical example,• 12 total possible

conflicts• 2 conflicts• fitness = 12 - 2 = 10

Recombination (or Crossover)

• Selected solutions are recombined in various ways to generate new solutions

• The most common solution to this is called a one-point crossover

10110001 1010010101001101

Parent 1 Parent 2

10011010 1010010101001101

Child 1 Child 2

10110001 0100101010100111

10011010 0100101010100111

Mutation

• Random changes introduced to aid in finding a better solution• Chance of mutation is usually between 1 and 2 tenths of a percent

101100011010010101001101Before:

101100011010110101001101After:

Genetic Algorithm Process

Initialize Population

Evaluate

SelectMutate

Recombine

End?

Solution

YesNoNext Generation

Advantages of Parallel Genetic Algorithms

• Shares all of the advantages of a serial GA• Higher efficiency and efficacy• Less likely to find suboptimal solutions• Parallel search from multiple points in search space• Can yield alternative solutions to a problem• Robust and not very restrictive because they run independently

Classification of Parallel Genetic Algorithms

• Ratio is low for fine grain parallel GAs

• Fine grain examples• Master-slave• Toroidal cluster

• Ratio is high for coarse grain parallel GAs

• Coarse grain examples• Ring GA (hypercube or

other topology)• Dual Species GA

• Classification relies of the computation/communication ratio

Master-Slave

• Global single-population• Evaluation of fitness is distributed among slaves• Selection, recombination, and mutation are all done in the master

processor

Slaves

Master Genetic Operators

Calculate Fitness

Calculate Fitness

Calculate Fitness

Calculate Fitness

FitnessIndividuals

Toroidal Cluster

• Separated single-population• Population is distributed

over processors of a 2D mesh

• Neighbors share members of their population for parent selection

Ring GA

• Multiple populations• Each processor is assigned

a simple GA and there is little communication between populations

• This also restricts the size of the selection and mating pools

Migration

Island

Island Island

Dual Species Genetic Algorithm

• Population 1 favors • crossovers of parents

that are high in similarity • low mutation rate

• Population 2 favors • crossovers of parents

that are low in similarity • high mutation rate

• A coarse grained GA with only two subpopulations • The populations exploit the use of different crossover probability and

mutation operators to prevent premature convergence• Migration occurs when one of the populations have converged

DSGA Process

Initialize Populations

Evaluate Populations

End? SolutionYes

Selection 1

Selection 2

Crossover 1

Crossover 2

Mutation 1

Mutation 2Migrate?

Migrate?

No

No

Migrate

Yes

Yes

No

Parallel GA Hybrids

Coarse and fine grain Coarse grain and master slave Coarse grain and coarse grain

Coarse Grained Vs Fine Grained (Rastrigin function)

• Generally coarse grained genetic algorithms perform better because of their efficiency and global population

• Fine grained genetic algorithms are favored when fitness calculation time is large

MAXSAT FGA Vs SGA

Number of fitness evaluations: 30000

Population size: 30

Offspring size: 200

Speedup: 43/40 = 1.08

MAXSAT FGA Vs SGA

Number of fitness evaluations: 30000

Population size: 30

Offspring size: 600

Speedup: 117/75 = 1.56

MAXSAT FGA Vs SGA

Number of fitness evaluations: 50000

Population size: 30

Offspring size: 1000

Speedup: 211/115 = 1.83

MAXSAT FGA Vs SGA

Number of fitness evaluations: 100000

Population size: 30

Offspring size: 2000

Speedup: 416/240 = 1.83

Sources

http://neo.lcc.uma.es/Articles/albatroyaxx_2.pdfhttp://tracer.uc3m.es/tws/cEA/documents/cant98.pdfhttp://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.45.8861&rep=rep1&type=pdf

http://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=4636604http://neo.lcc.uma.es/cEA-web/documents/hiroyasu.pdf