EVOLVING BETTER COMPUTER GAME ALGORITHMS WITH THE GPROTOOLKIT GENETIC PROGRAMMING

SYSTEM

By: Gregory MirandaUniversity of Advancing Technology

Summary

• Introduction and Background• Statement of the Problem• Literature Review• Methodology and Results• Discussion, Implications, Recommendations• Q&A

Introduction and Background• Genetic Programming is a method for automatically creating

computer software– Capable of solving complex problems using an iterative approach to

find a solution– Can be used to create software functions that can solve new

problems and to find new solutions to existing problems which have already been solved

• By applying genetic programming to an existing problem like pathfinding, it may be possible to find a more efficient solutions– Efficiency can be measured by comparing an A* algorithm to genetic

programming’s best solution to see which performs the best and utilizes the least amount of computer resources

Introduction and Background (cont.)

• Genetic Programming still mainly an academic tool – Not used in mainstream software development– A number of genetic programming applications and libraries

have been created to further genetic programming research• The majority of the genetic programming applications are self

contained, requiring the use of scripting languages to define the primitive set and the testing environment.

• The outputs of some of these applications can be saved in mainstream programming languages such as C++ and Java, but others require translating the output or digging through log files to find the final output of the system.

• Neither of these methods is easy to use and require a large learning curve before even attempting to use them.

• Having to recreate testing environment requires a lot of duplication

Statement of the Problem

• Despite the number of genetic programming applications available, none of them really address the needs of mainstream software developers– They are difficult to use and, in many cases, require

duplication of effort to recreate the testing environment. • To fix this gap, a genetic programming library was

created that abstracted the majority of the genetic programming system and only exposed those areas that require integration into a developer’s software project. – Designed to be easy to integrate and use, the library should

appeal to mainstream software developers in any industry.

Statement of the Problem (cont.)

• To prove that the library was capable of being used in the game industry, it was applied to creating a pathfinding solution that was comparable to the industry standard A* algorithm– Pathfinding is a basic component of many 2D and 3D

computer games and simulations and is an area that receives a lot of attention as it can consume a large amount of computer resources when there are a large number of dynamic objects moving simultaneously which is becoming more common in today’s computer games

Literature Review

• Genetic Programming– Evolutionary computing technique for solving problems by

automatically generating computer programs• Developed by John Koza in 1992• Based on genetic algorithms (developed by John Holland, in 1992)• Uses computer programs for genome instead of DNA or byte string

– Modeled after several natural evolution principles• Inheritance• Mutation• Fitness – Survival of the fittest

Literature Review (cont.)

• Genetic programming genome – Each program in the population must compile and execute in

order to evaluate their fitness in solving the problem– Koza used a parse tree where the terminals and the

functions make up the program’s components• Nodes represents a function• Leaves represents a terminal - an independent variable, zero-

argument function, or a random constant• Each of the branches coming out of a node represents an argument

for the node’s function. • The order of the components of the computer program is

determined by the connections between the terminals and functions

Literature Review (cont.)

• Evolutionary Algorithm (Eggermont, 2007, p. 1)– Randomly create an initial population– Do

• Evaluate current population• Select parents• Generate offspring• Replace current population with offspring

– Until stop criterion met– Return best individual so far

Literature Review (cont.)

• Lisp was the first language used by Koza and others in the early research of genetic programming

• Since then genetic programming systems have been created with C, C++, Java, and even C#.

• Glaholt and Zhang (2004) believe that genetic programming is not used as a standard for solving complex problems because the tools are hard to use or do not produce useable output for programmers– Creators of GP-Lab

Literature Review (cont.)

• A number of genetic programming systems have been created in various programming languages– GP-Lab was Windows based application that could generate compiler ready C++

code from its genetic programming system and displayed it on the screen for the user to copy into their own applications

– GAPS was easy to setup and run. It used an ASCII text file for input and a text log for output. The output did not produce any code that could be used by a programmer

– lil-gp, which was flexible and comprehensive, containing all of the genetic programming necessary to evolve programs. To use, the function set and fitness functions need to be coded in C source files and compiled with the lil-gp source

– Open BEAGLE is a genetic programming framework built with C++ with a set of genetic programming class libraries. To use Open BEAGLE, the programmer must override some classes and methods and compile their changes with the Open BEAGLE system. The output of the system is in XML and must be converted to source code by the user

– Tiny-gp created for a competition. It was written in Java and only supports symbolic regression

Literature Review (cont.)

• Genetic Programming in Computer Games– Very little of the artificial intelligence research performed with

games has been genetic programming, most have been with neural networks, genetic algorithms, and other machine learning concepts

– It has been applied to a variety of strategy type games such as Othello, poker, draughts, chess, and backgammon along with some controller based games such as Ms. Pac-Man, robotic football, and radio controlled model car racing

– Agapitos, Togelius, and Lucas, (2007) used both genetic programming and neural networks to create a controller for simulated car racing and compared the results against each other

– Genetic programming was used to develop a high level strategy for training individual players in RoboCup soccer

Literature Review (cont.)

• Pathfinding in Computer Games– To figure out how to move a character from one point to

another, the artificial intelligence system uses a pathfinding algorithm

– The industry standard pathfinding algorithm used in computer games, and the basis for many newer algorithms, is A*• A* is a simple algorithm that uses graph theory for finding the shortest

path between two points• The further away the goal is from the starting point, the longer it takes

and the more memory is required

• Genetic Programming and Pathfinding– Strom (2006) created a pathfinding algorithm using a LISP based

genetic programming system to be more aesthetic than A*

Methodology and Results• GProToolkit Overview

– Genetic Programming System incorporated into a C# library– C# vs Java or C++– Object Oriented Design

• Primarily encapsulation– Tree-based representation– Variable number of automatically defined functions– Variable number of result producing branches– Strongly Typed GP

• Supports only primitive types (void, 32 bit integer, 64 bit integer, double, and boolean)

– Variable number of arguments– Supports multiple objectives using Pareto optimality

Methodology and Results (cont.)

• GProToolkit Overview cont.– Application programmer defines the primitive set

• Uses reflection for discovering the function and terminal set defined by the application programmer in the client application.

– Application programmer executes the Solution– Application programmer evaluates and sets Fitness values

Methodology and Results (cont.)

• Public Classes– Engine– Properties

• Internal Class– RNG

Methodology and Results (cont.)

• Internal Classes– Primitives– PrimitiveSet– PrimitiveList

• Public Class– PrimitiveBase

• Attributes– GPPrimitiveSet– GPConstant

Methodology and Results (cont.)

• Internal Classes– CodeElement– ConstantElement– MethodElement– PropertyElement– FieldElement– ArgumentElement– AdfElement

Methodology and Results (cont.)

• Internal Classes– Node– TreeInfo

Methodology and Results (cont.)

• Public Class– Solution

• Internal Classes– GPSolution– Program– FitnessResults– Result

Methodology and Results (cont.)

• Debugging Application (Unit Test)– nUnit Unit Test Framework– Symbolic Regression Test

• Create expression: x2 + x + 1

• Output:namespace Namespace1 { using System; public class Class1 { public virtual double ResultFunction1(

GProToolkitUnitTest.MathFunctions functionBase) { return this.adf1(functionBase.ArgX(), functionBase.ArgX(), functionBase); } private double adf1(double arg0, double arg1, GProToolkitUnitTest.MathFunctions functionBase) { return functionBase.Subtract(functionBase.Add(functionBase.Multiply(arg1, arg0), functionBase.Divide(arg0, arg0)),

functionBase.Subtract(functionBase.Multiply(functionBase.Subtract(arg1, functionBase.Add(functionBase.Multiply(arg1, arg0), functionBase.Divide(arg0, arg0))), functionBase.Subtract(arg0, arg0)), functionBase.Divide(functionBase.Add(functionBase.Multiply(arg1, arg0), functionBase.Divide(arg0, arg0)), functionBase.Divide(functionBase.Add(functionBase.Multiply(arg1, arg0), functionBase.Divide(arg0, arg0)), arg1))));

} }}

Methodology and Results (cont.)

• GPTester Application Demos– http://www.youtube.com/watch?v=Q3vxPygyFO4– http://www.youtube.com/watch?v=ShIw_Bxmsrg

• Results from successful run– 50 generations– Population of 200

A* and GP Results

A* Path Size(# tiles)

A* Run Time (ms)

GP Path Size(# tiles)

GP Run Time (ms)

Map 1 58 26.8 60 3.0Map 2 58 28.1 58 3.0Map 3 43 5.1 49 2.0Map 4 51 7.5 55 3.0Map 5 41 3.0 57 3.0Map 6 120 1.0 122 7.0

Methodology and Results (cont.)

• A* (left) and GP Solution (right) for Tile Map 1

• A* (left) and GP Solution (right) for Tile Map 2

Methodology and Results (cont.)

• A* (left) and GP Solution (right) for Tile Map 3

• A* (left) and GP Solution (right) for Tile Map 4

Methodology and Results (cont.)

• A* (left) and GP Solution (right) for Tile Map 5

• A* (left) and GP Solution (right) for Tile Map 6

Methodology and Results (cont.)

• Bad GP Solution for Tile Map 5 (but interesting)

Discussion, Implications, Recommendations

• Discussion– Easy to use– Hide GP details– Symbolic regression test

• solved a trivial problem domain– Sometimes a good solution could not be found in a single run

• Multiple runs using new initial populations were necessary– Adjusted the primitive set and the fitness results to find a ‘good’ solution– Issues with Pareto optimality

• Too many fitness results resulted in a large “best so far” pool that had a high proportion of unsuitable results being reused as parents.

• Too few fitness results though resulted in not having enough diversity and also produced unsuitable results. • Through a lot of trial and error testing, a middle ground was found that resulted in good trial solutions being

generated– The fact that the trial solution outperformed the A* algorithm in two-thirds of the test cases

also showed that genetic programming was capable of making a more optimized pathfinding algorithm than the basic A* implementation. • Given more time, and possibly some more adjustments to the fitness results and the primitive set, a trial

solution could be generated that completely mimicked the A* algorithm path instead of just getting very close

Discussion, Implications, Recommendations

• Implications– Limitations of genetic programming

• Length of the genetic programming runs• Difficulty in defining the proper primitive set

– Primitive set and the fitness results are areas are highly customized to specific problems sets

– Abstracting the genetic programming system into a library makes it easier integrate genetic programming

– Can be used in game development to find new and novel solutions to standard algorithms

Discussion, Implications, Recommendations

• Recommendations– The primitive set should be able to include any library, both system and

custom made• Combination of both methods would lead to a system capable of developing a

much wider range of computer software– Training a genetic program system can take a long time

• Most of the time is spent actually testing the solutions to measure their fitness– Allow for parallel processing and, if possible, distributed computing

– Offer options with the structure of the genetic programming system• Linear GP and graph GP in addition to tree-based GP

– Expanded to support more than just primitive types– Support COM interop

• The library would be available in any language on a Microsoft Window system that supports COM, such as C++, Java, and Visual Basic

Q&A