Evolving Agents in a Hostile Environment

Alex J. Berry

Training First Responders

VEnOM Labs is developing a suite to train First Responders.

Is the training effective? How can we make the training more effective? Environment lacks atonomous agents that can

interact with trainees in the environment.

Goal

Long Term To develop a system to allow for friendly and hostile

AI agents in the training environment.

Short Term To develop a system to evolve agents in a hostile

environment.

Simulation of Adaptive Agents in a Hostile Environment[HW95]

Thomas Haynes Used Genetic Programming Simple Agents Mines and Energy Experiments

Single Agent, Static and Random Environment Multiple Agent, Static and Random Environment

The Approach

Randomly Generated Map Environment Three Types of Agents:

First Responders Terrorists Victims

Genetic Programming to Evolve the Agents

Maps

Any Dimension Percentage walls Bit Array to Hold the

Data Used for memory

storage in the Agents

What’s an Agent to do?

Victims Move Randomly Remember Things Forget Things Survive

Terrorists Kill Victims Kill First Responders Lay Traps Not Get Caught

First Responders Help Victims Find and Disarm Traps Survive Catch Terrorists

Evolutionary Algorithm

Two Agents to Evolve First Responder Terrorist

Mutation and Crossover are the only operators changed. The individuals consist of expression and decision trees. Initialization was based on both random and created

individuals. Rank Based Selection was used. Elitist Competition was used.

What An Individual Looks Like

Terminals Current Grid Location (C) Surrounding Grid Locations

(S) Rand (R) Memory (M)

Non-Terminals If-Then-Else

Threat And, Or, Not Victim, First Responder,

Terrorist, Trap Valid Move

Actions Save Kill Move Place Trap Remove Trap

Sample Individual

MOVE

Genetic Programming Evaluation

First Responder Victims Helped Terrorists Caught Traps Removed Survival Time Amount of the Map

explored

Terrorist Kills using Traps Kills on Contact Survival Time Deduction for killing

other Terrorists

Experiments

Static Environment Evolution Random Environment Evolution Varying Ratios of First Responders, Victims, and

Terrorists Evolving one Population at a time