Ant-based Routing in Networks

Ant System: Routing Problem

• Idea– Ants dropping different pheromones used to

compute “shortest” path from source to destination(s);

– more flexible adaptation to failures and network congestion;

– use only local knowledge for routing and avoid costly communication of state to all network nodes.

Ant System: Why Routing?

• Conventional routing often relies on:– global state available at all nodes;– centralized control;– fixed “shortest path” (Dijkstra) algorithms;– limited ability to deal with congestion or

failure.• Ideally, would like to have network adapt

routing patterns to take advantage of free resources and move existing traffic if possible.

Ant System: Routing Research

• Approaches so far investigated:– [White et al, 97+]– [Schoonderwoerd et al, 97] (*)– [Dorigo et al, 97] (*)– [Guerin, Heusse, Snyers et al, 98+]

• Differences– Link cost metric constant in *– Point to point traffic only in *– AS Parameter settings constant in *

Schoonderwoerd:Ant-based Control (ABC)

• Circuit-switched routing• Ant-based

– Ants deposit pheromone which modify routing tables

• Network performance determined by ability to admit (or reject) calls

ABC Model

Links have capacity, Ci and

space capacity Si.

Contains routing table

giving shortest

distance to destination:

rin,d(t)

ABC Call Model

• Capacity is reserved for duration of call• Released when call terminates• Call has variable holding time• Calls set up deterministically, using routing

tables• Ants launched from nodes at any time

– Generation frequency a parameter of system– Destination chosen randomly

ABC Ant Model

• Ants move from node to node– Node choice is probabilistic– Route creation is deterministic

• Deleted from network when they reach destination• Routing tables are updated with pheromone

– rin,d(t),

– i = current node– n = next node– d = destination

Routing Table Updates

rrtr

tri

siisi ∂+

∂+=+ −

− 1)(

)1( ,1,1

1n ,1

)()1( ,

, −≠∂+

=+ irtr

tri

snisn

s = source node, i-1 = node just come from, i = current node

bTar +=∂

a,b,c and d are constants, T is the ant’s age, D is delay at each node

Reinforcement decays with distance

SdecD .. −=

Comments

• Normalized, r values can be considered as probabilities:– Ants ONLY choose viable links– No viable links, they die

• The rii-1,s(t) more reinforced when small

– Not on preferred route– Trail smoothing is via proportional update– New routes quickly discovered when others congested

Example

2

1

4

53

Source is 5, destination is 2, arrives at 4

0.80.10.30.150.10.80.40.130.10.10.30.815321

Destination nodes

Nei

ghbo

urno

des

Example continued

0.80.10.30.150.10.80.40.130.10.10.30.815321

Destination nodes

Nei

ghbo

urno

des

)1(1.0r∂+

)1(1.0r∂+

)1(8.0

rr

∂+∂+

a=0.08b=0.05c=80d=0.075S=100%

095.0)051.01(

1.0=

+=

810.0)051.01()051.08.0(=

++

=

095.0)051.01(

1.0=

+=

Comments• Requires initialization phase• Good routes can become “frozen”• Added exploration probability:

– (1-q) uses probabilistic exploration, q random

Routes of new calls

Load onnodes

Routingtables

Routes of ants

Expiring calls

Failing calls

Dead ants

New ants

Newcalls

Results

• Tested on real 30 node BT network– Max. capacity of nodes 40 calls– Call generation 1 per 170 time steps– P(emitter) = P(receiver) uniformily in

[0.01,0.07]

Results

0.541.99ABC (with noise)0.541.79ABC (no noise)

0.774.22Improved mobile agents0.789.19Mobile Agents2.1612.57Shortest Path

Std. Dev.Avg. CallFailures

Other scenariosAlso superior

Enhancements

• Guerin– Update all columns (in stead of just one)– Rather like dynamic programming– Principle is:

• If subroutes optimal, then route will be too– Requires ants to return to nest

• Smarter, but fewer ants required • L(L-1)/2 entries updated, compared to L for ABC

ants

Enhanced Model

1

4

2

5

3

6Ant gets to destination (6),Then reverses trail updatingrouting tables on all nodes: {6,4,5,1}

At 6: Updates for {4,5}

At 4: Updates for {5,6}

At 5: Updates for {1,4,6}

At 1: Updates for {5}

How it works

• Updating now uses “relative” rather than absolute age

rrtr

tri

fiifi ∂+

∂+=+ −

− 1)(

)1( ,1,1

1n ,1

)()1( ,

, −≠∂+

=+ irtr

tri

fnifn

bTT

arfi

+−

=∂ SdecD .. −=nodeith at time=iT

Network Tested

Swarm Intelligence: Bonabeau et al

Results and Comment

• Smart ants significantly better than simple ants (see Figure 2.20 in book)

• Relies on symmetric path costs– Generally not true– Heusse generalized to asymmetric case

• Subramanian applied algorithm to packet networks– Basically same algorithm

ABC Results

Swarm Intelligence: Bonabeau et al

Symmetric Links

2

Ant Movement

1

Routing table update is appliedto movement in opposite direction

AntNet

• Similar principles to ABC• Can be applied to connection-oriented and

connectionless networks• Collect information which builds parametric

models of network state– Compute reinforcements (changes) to probabilistic

routing tables• Performance compared to existing routing

algorithms

AntNet Model

• Two types of ants defined:– Forward: source->destination– Backward: destination->source– Ants transformed from Forward->Backward– Forward move at same priority as data– Backward move at higher priority

AntNet Model continued

• Forward ants launched periodically• Destinations chosen to mirror traffic

patterns• Forward ant chooses next hop from not-yet-

visited nodes probabilistically, proportional to ri

n,d(t)• Identifier of visited nodes (and visit time)

pushed onto stack

AntNet Model continued

• Cycle detected– Cycles nodes popped from stack

• At destination, backward ant generated and forward ant dies. Stack is transferred.

• On backward trip, routing table modified by incrementing ri

i-1,d(t) and decreasing rin,d(t).

• Trip time i->d used to compute increments– However, this value is “noisy”– Other things are going on in the network!

AntModel continued

• Signal is noisy, so do reinforcement:

• Value r defined as:

• Essentially, rii-1,d(t) increased in proportion

to signal received• Similar to Actor-Critic model from neural

networks

( ) rtrrtr idi

idi +−= −− )(.1)( ,1,1

( )idiTp Γ− ,

Comments

• All discovered paths are reinforced• Frequency of traversal is a factor

– Ant arrival rate• There’s a startup transient when routing is

essentially random• Remember: routing is probabilistic not

deterministic– Used non-linear distribution f(p)=pδ, δ=1.2

Experimental setup

• Used model networks:– NSFNET (14 nodes, 21 bi-directional links)– NTTnet (57 nodes, 162 bi-directional links)

• Not well balanced

• Several traffic profiles tried• Compared to

– Simplified open shortest path first (OSPF)– Asynchronous Bellman-Ford with dynamic cost– Shortest path first (SPF) with dynamic cost– Q routing (Boyan and Littman) – Predictive Q-routing (extension to Q routing)

AntNet Algorithm

Swarm Intelligence: Bonabeau et al

AntNet Networks Tested

Swarm Intelligence: Bonabeau et al

Results and Comment

• Throughput and delay significantly superior to existing algorithms (Figure 2.23, 2.24)

• Approximately 90% of “theoretical” bound– Perfect and instantaneous information update

• However:– No proof that it works in real networks– Higher routing cost per packet (probabilistic)

• Added nodal computation ignored

– Need lots of simulation work Good thesis workhere. Extensionsto AntNet to real

networks

AntNet Results

Swarm Intelligence: Bonabeau et al

AntNet Results II

Swarm Intelligence: Bonabeau et al

Thoughts on Routing …

• Improved accuracy of simulation– Nodal delays accurately simulated– Buffers modelled etc.

• Introduce constraints; e.g. QoS• Model different types of networks

– Ad hoc, transmission

• Introduce flow control into simulation– Couple routing and admission control considerations

Concerns for Routing

• Convergence to steady state• Adaptation to changing environments• Oscillation

Other forms of Routing

• Looking for information– Surfing on the web

Web Site

mammals

pigsWeb Site

Web Site

Web Site

Good thesis workhere. Annotation

of searches:AntWorld