Upload
marjorie-cross
View
215
Download
0
Tags:
Embed Size (px)
Citation preview
Genetic Programming
Part 2
Jay Shaffstall
Genetic Programming
Review of Part 1
Program graphing
Program breeding
Program mutation
Fitness functions
Development life cycle
Real world applications
Conclusion
Review of Part 1
Genetic programming takes advantage of the power of evolution to grow a program
The fitness function determines which programs from any generation will survive to breed
The best programs out of every generation create the next generation
Review of Part 1
After millions of generations, we end up with a program that can solve the problem
But how does it all work?
Program Graphing
In order to breed programs, we have to represent them in some fashion. Genetic programming represents a program as a graph.
Programs are represented using a lisp like syntax, with the operation coming before the arguments
Program Graphing
Program Breeding
So we have two programs represented as program graphs. How do these two programs combine to form children?We randomly choose a node in each program. We delete the subtree from the first program and replace it with the subtree from the second program
Program Breeding -- Parents
Program Breeding -- Children
Program Mutation
Breeding is not the only way to change a program. We can also mutate a program. The program is changed in a random way, and the change will be passed on to any children.
Program Mutation
Types of mutationsReplace a random subtree with another randomly generated program
Duplicate a subroutine, and randomly half the calls to use the new subroutine
Duplicate arguments to a subroutine, and randomly change half the references to use the new argument
Program Mutation
Types of mutations (continued)Create a new subroutine
Delete an existing subroutine
Delete an existing argument
Fitness Functions
Eventually, between breeding and mutation, we can end up with some complex programs.
So how do we pick the ones that live and the ones that die?
Fitness Functions
The basic purpose of a fitness function is to determine the distance between one program’s solution and the ideal solution.
This allows us to pick the “closest” solutions for breeding.
Fitness Functions
An easy example is a program that must calculate the square of a number.
The fitness function can be something like this:
square (x) – program (x)
Fitness Functions
Those programs that score the lowest in the fitness function are those that are closest to the ideal solution.
Those are the programs that we choose to live and breed.
Fitness Functions
So how do we develop a fitness function for a complex problem, without running into the automated oracle problem?
For example, the problem of writing an order entry system?
Fitness Functions
The simple answer is that we cannot reliably create a fitness function for complex problems.
Does this mean that genetic programming is useless for more complex problems?
Not necessarily.
Fitness Functions
One possibility is to break the problem down into simpler modules that can be represented by a less complex fitness function.
We then evolve all the modules, which will work together to create the final product.
Fitness Functions
So this approach would be a combination of designing the system by a human, and evolving the pieces of the system by a computer.
Development Life Cycle
At first glance, genetic programming seems to require a waterfall method of development.
We’ve all seen that waterfall development only works for simple projects.
What life cycle do we use for complex projects using genetic programming?
Development Life Cycle
The best fit seems to be an iterative cycle, where a human designs the architecture down to simple modules that are then evolved to meet requirements.
As requirements change, only those modules that are affected need to be re-evolved.
Development Life Cycle
Since the modules are relatively simple, re-evolving them will not take as much time as re-evolving the entire program.
We also reduce the chance of getting unexpected features in the program, since the user-visible features will be specified in the human created design of the modules.
Development Life Cycle
In this type of life cycle, we use genetic programming as an entry level programmer, leaving humans free to do the higher level design work.
Real World Applications
So, how closely can genetic programming approach the results of a program written by a human?
John Koza, regarded as the inventor of genetic programming, took a number of problems in circuit design and evolved programs to see how competitive genetic programming was.
Real World Applications
The hardware he used to do this was a Beowulf style cluster of 1,000 Pentium II processors.
Clearly, for anything more than trivial problems, significant computing power is needed.
Real World Applications
What Koza found is that genetic programming does indeed produce results competitive with human designed circuits.
In fact, in several cases genetic programming produced results that were better than human created results
Real World Applications
Unfortunately, several of the genetically programmed designs also infringed on exiting U.S. patents.
This has led Koza to the idea that genetic programming can be used as an invention creator. Give it a problem you would like to solve, but have no idea how, and let it run.
Conclusion
We’ve seen how genetic programming works, and how it can produce competitive results for well defined problems.
Most of our problems in the business arena are not well defined, but I’ve suggested a way in which genetic programming could be a useful tool for implementing well defined pieces of a program.
Conclusion
While I don’t expect business to start using genetic programming, after doing the research for this presentation, I think it could very well have a place in a business environment.
ReferencesKoza, John, et al. (1999) Genetic Programming III Morgan Kaufmann PublishersJohn Koza's Home Page, includes basic information on genetic programming, available on the web at http://www.genetic-programming.com/johnkoza.html Koza, John, et al Human Competitive Machine Intelligence available on the web at http://www.genetic-programming.com/humancompetitive.html
ReferencesQureshi, Adil GPsys 2b: Java Genetic Programmin System available on the web at http://www.cs.ucl.ac.uk/staff/A.Qureshi/gpsys.html Genetic Programming Conference homepage, available on the web at http://www.genetic-programming.org/Genetic Programming Tutorial, available on the web at http://www.geneticprogramming.com/Tutorial/
References
Genetic Programming, available on the web at http://www2.informatik.uni-erlangen.de/~jacob/Evolvica/GP/Java/html/genprog/gp.html
Fundamentals in Genetic Programming, available on the web at http://www.dd.chalmers.se/~f96edfa/