Swarm behaviour and traffic simulationsUsing stigmergy to solve algorithmic problems, predict and improve vehicle traffic

Overview (1)

Swarms in nature Social insects and Stigmergy Ant algorithms and application examples:

Foraging in ants Using foraging behaviour to solve the TSP

Labour division among social insects Mailmen using adaptive task allocation model

Overview (2)

Traffic simulation by cellular automata Adopting the stigmergic process Prediction of driver behaviour Traffic infrastructure optimization

Drivers as ant agents Signal lights as social insects

Other real world applications

Swarms in nature

What is a swarm ???

Pictures of swarms (1)

Pictures of swarms (2)

Pictures of swarms (3)

Characteristics of swarms Aggregation of animals with similar size and often

similar orientation

Interaction of animals leads to new intelligent forms of behaviour that are not inherited in the individuals

E.g. insects, birds, fish, bacteria

The main actor of the presentation

Social insects and stigmergy

Social insect societies are distributed systems with highly structered social organization

They can accomplish complex tasks that far exceed the individuals abilities

Here: focus on stigmergy as important means of indirect communication paradigm

Stigmergy Originally defined by Grassé:

Stimulation of workers by the performance they have achieved

Method of indirect communication in a self-organizing emergent system where its individual parts communicate with each other by modifying their local environment

Here: Pheromones diffusing chemical substance

Stigmergy example

Example: termites buildung nest pillars with soil pellets

Stimulus response

Autocatalytic process

Stigmergy behaviour of ants

Stigmergy behaviour in ants and their transfer to

computer algorithms:

Foraging and the TSP

Labour Division and adaptive task allocation

Foraging in ants Foraging means searching for food

Ants manage to find the shortest path between their nest and a food source

Achieved through trail-laying and trail–following behaviour with pheromones Stigmergy

Foraging example Two paths with

different lengths

Ants follow way with most pheromones

Autocatalytic process leads to „differential length effect“

The Travelling Salesman Problem Consists of a set of given cities

Goal is to visit all cities in a closed loop of shortest length

Every city must be visited only once!

E.g. 15 biggest cities of Germany

TSP represented by graph theoryTSP defined more generally by graph theory: Graphs consist of vertices V and edges E Cities are vertices, edges are connections between cities In the TSP each city is connected to each other! Each edge has a certain length

Example: 4 cities A,B,C,D

represented by vertices;

6 connnections with lengths

represented by edges

Ants solving the TSP Artificial ants exploring the TSP graph Artificial pheromones added by ants after completion of a

complete loop proportional to 1/length of route Probabilistic transition rule for ant k to next city j:

City j visited? Length of edge gives desirability measure ηij = 1/length

Amount of pheromones τij(t) on edge (i, j)

Evaporation of pheromones over time lets system forget bad information

Labour division in ants Fundamental in social insects: Division of

reproductive castes from worker castes

Further divisions: subcastes of workers specialists subcastes of age and morphology again dividing subcastes into behavioural castes

Plasticity: Workers switch tasks in response to internal and external pertubations

Labour division model

Based on idea of response threshold

Stimulus exceeds individuals response threshold individual engages in task performance

Stimulus plays role of Stigmergy here (can be pheromones, amount of encounters, …)

Extended labour division model

More realistic:

Extend previous model by threshold varying in time.

if an individual performs a certain task its threshold related to this task decreases

the thresholds related to all other tasks, not performed meanwhile, are increasing

Example: Express mail retrieval (1)

Group of mailmen has to

pick up letters in a city

Goal: Allocation of mailmen to appearing demands should be optimal realized with adaptive task allocation model

Each mailmen i reacts with a certain probability p to arising demands, depending on: response threshold Ө related to area j with demand the distance d to the area with demand the intensity s of the demand stimulus

Example: Express mail retrieval (2)

Figure (a): demand of a certain area over time

At t = 2000 the mailman that is specialized on this area gets removed

Figure (b): response threshold of another mailmen that is reacting to the loss of the specialist

Traffic simulations

Why would one do that?

to predict drivers behaviour in order to adjust dynamic traffic signs, or propose alternative routes in navigation devices or radio

to improve traffic infrastructure and traffic light plans in big, complicated traffic networks like cities

Cellular automata traffic models (1)

Two major approaches on traffic simulation: Fluid-dynamical, with continuous traffic macroscopic Discretized cellular automata model microscopic

Focus in presentation: discretized cellular automata models

Discretized: Street is diveded into fixed sites, cars have integer velocities Each site can be occupied by a car or can be empty

car 1 car 2

x meters

site n site n+1 site n +2 site n +3 site n + 4

e.g. one lane traffic

Cellular automata traffic models (2)

Assuming a simple one lane model:

One update of the system consists of the following steps performed with each car in parallel: Acceleration or slowing down:

Depending on maximal speed and distance to next car Randomization:

To contribute human behaviour and external influences Car motion:

The advancment of sites, according to the speed

Model shows nontrivial and realistic behaviour

Adopting Stigmergy Cars adopt the pheromone laying and sniffing behaviour of

ants

Leads to very realistic and dynamic system

Reduces communication between cars to local information creation and retrieval stigmergy

Computational costs can be reduced for collision checking

Still, information about traffic signs and other environmental signals are non-local

„cellular automata ant cars“

How cars behave like ants Each car leaves and sniffs pheromones on the road Pheromones fade over time, like with ants Faster cars leave longer trails then slower cars (a)

Additional pheromone dropping necessary for stopped cars (b)

quick deceleration lane changing (c)

like using signal andbrake lights

(a)

(b)

(c)

Traffic prediction Measurement devices like cameras determine the vehicles

entering an area Implementing foraging behaviour of ants leads to realistic

system of interacting drivers

drivers follow other drivers and try to escape jams Various types of driver support:

Adjustment of dynamic traffic signs to avoid congestions Knowledge of growth of traffic jams allows to give reasonable

redirections Through use of foraging behaviour alternative routes can be given

more effectively, cars spread more

Optimizing traffic light plans

Microscopic traffic model by individual cars with individual aims

1st Approach: Cars as agents facilitate change of light plans by voting Evolutionary process improves overall light plans

2nd Approach: Groups of ligths at an intersection behave like social

insects Adaptive task allocation is responsible for running plans

Cars voting for traffic ligths Each car keeps track of two variables:

total driving time dtot

total waiting time wtot

waiting measure:

Statistics give information about overall fitness overall waiting measure:

Cars that are stopped at a light vote for it

Lights with many votes are more probable to be mutated quicker adaption in the evolutionary process

Probabilistic mutations of traffic light timings Mutation changes length of correlated light phases at

an intersection (e.g. N–S and E-W)

Simulation of each branch of a new generation Survival of the fittest with least waiting time of

cars

Evolutionary process

simulation 1

simulation 2

simulation 3

choose fittest

mutate

. . . .

mutate

Traffic Simulation under SuRJESuRJE: Swarms Under R&J using Evolution Design environment to build, test and optimize traffic

scenarios Uses swarm based approach and

evolutionary adaption Features:

Enables to build multi-lane road maps Car seeding areas define the

input and output of cars Initial lights settings for starting point Evolutionary adaption parameters can be set

Simulation example in SuRJE

Example network: „Looptown“ (a) Figure (b) shows the decrease of overall waiting time

over generations

Traffic lights as social insects

Traffic lights are implemented as social insects

all lights at an intersection form one insect Insect has to perform one traffic light plan out of

several for its intersection Traffic can be modeled by any microscopic traffic

simulator Cars emit pheromones when crossing and waiting at

an intersection to provide stimulus

Use of adaptive task allocation Adaptive task allocation model is applied to insects

light-plans are chosen through stimulus – response strategies communication of intersections only by Stigmergy

Each insect/intersection has individual thresholds related to available light-plan

Stimulus for light-plan j provided by cars:

Reinforcement learning is used to specialize intersections: Threshold j of intersection i: Learning coefficient:

Example of stimulus evaluation 4 way intersection with two light-plans:

1st plan gives priority to North-South leading lanes 2nd plan has priority for West-East leading lanes

Traffic situation: Many cars are driving on West – East direction Few cars on North-South direction

Initially plan 1 is driven: Big amount of pheromones for W-E (many, waiting cars) Small amount for N-S (few, almost not waiting cars)

evaluation of stimulus yields higher value for 2nd plan!

N

W E

S

Traffic simulation - Recapitulation Traffic prediction:

Simulation into future by using swarm based approach Giving the driver useful information about choosing the best route

Traffic light timing improvment by cars as agents: Cars are modeled with swarm based approach Improvement of traffic light plans through evolution Good for simple traffic lights with static timing program

Dynamic traffic light adoption through social insects: Cars are modeled by any microscopic traffic simulation Intersections choose their light plan through adaptive task allocation Good for traffic lights with sensors for counting cars

Examples for real world applications

Scheduling Problems, e.g. subway, train

Vehicle Routing, e.g. bus, taxi

Connection-oriented network routing, e.g. internet, TCP/IP

Connection-less network routing, e.g. bluetooth, infrared

Optical networks routing

Thank you for your attention!