Ant Colony Optimization: an introduction

Daniel Chivilikhin

03.04.2013

2

Outline

1. Biological inspiration of ACO

2. Solving NP-hard combinatorial problems

3. The ACO metaheuristic

4. ACO for the Traveling Salesman Problem

3

Outline

1. Biological inspiration of ACO

2. Solving NP-hard combinatorial problems

3. The ACO metaheuristic

4. ACO for the Traveling Salesman Problem

4

Biological inspiration:

from real to artificial ants

5

Ant colonies

• Distributed systems of social insects

• Consist of simple individuals

• Colony intelligence >> Individual intelligence

6

Ant Cooperation

• Stigmergy – indirect communication between individuals (ants)

• Driven by environment modifications

7

Denebourg’s double bridge experiments

• Studied Argentine ants I. humilis

• Double bridge from ants to food source

8

Double bridge experiments: equal lengths (1)

9

Double bridge experiments: equal lengths (2)

• Run for a number of trials

• Ants choose each branch ~ same number of trials

10

Double bridge experiments: different lengths (2)

11

Double bridge experiments: different lengths (2)

• The majority of ants follow the short path

12

Outline

1. Biological inspiration of ACO

2. Solving NP-hard combinatorial problems

3. The ACO metaheuristic

4. ACO for the Traveling Salesman Problem

13

Solving NP-hard combinatorialproblems

14

Combinatorial optimization

• Find values of discrete variables

• Optimizing a given objective function

15

Combinatorial optimization

Π = (S, f, Ω) – problem instance

• S – set of candidate solutions

• f – objective function

• Ω – set of constraints

• – set of feasible solutions (with respect to Ω)

• Find globally optimal feasible solution s*

SS ~

16

NP-hard combinatorial problems

• Cannot be exactly solved in polynomial time

• Approximate methods – generate near-optimal solutions in reasonable time

• No formal theoretical guarantees

• Approximate methods = heuristics

17

Approximate methods

• Constructive algorithms

• Local search

18

Constructive algorithms

• Add components to solution incrementally

• Example – greedy heuristics:

add solution component with best heuristic estimate

19

Local search

• Explore neighborhoods of complete solutions

• Improve current solution by local changes– first improvement– best improvement

20

What is a metaheuristic?

• A set of algorithmic concepts

• Can be used to define heuristic methods

• Applicable to a wide set of problems

21

Examples of metaheuristics

• Simulated annealing

• Tabu search

• Iterated local search

• Evolutionary computation

• Ant colony optimization

• Particle swarm optimization

• etc.

22

Outline

1. Biological inspiration of ACO

2. Solving NP-hard combinatorial problems

3. The ACO metaheuristic

4. ACO for the Traveling Salesman Problem

23

The ACO metaheuristic

24

ACO metaheuristic

• A colony of artificial ants cooperate in finding good solutions

• Each ant – simple agent

• Ants communicate indirectly using stigmergy

25

Combinatorial optimization problem mapping (1)

• Combinatorial problem (S, f, Ω(t))

• Ω(t) – time-dependent constraints Example – dynamic problems

• Goal – find globally optimal feasible solution s*

• Minimization problem

• Mapped on another problem…

26

Combinatorial optimization problem mapping (2)

• C = {c1, c2, …, cNc} – finite set of components

• States of the problem:

X = {x = <ci, cj, …, ch, …>, |x| < n < +∞}

• Set of candidate solutions: XS

27

Combinatorial optimization problem mapping (3)

• Set of feasible states:

• We can complete into a solution satisfying Ω(t)

• Non-empty set of optimal solutions:

XX ~

Xx~

SXS ~*

28

Combinatorial optimization problem mapping (4)

S* X~S

X• X – states

• S – candidate solutions

• – feasible states

• S* – optimal solutions

X~

29

Combinatorial optimization problem mapping (5)

• Cost g(s, t) for each

• In most cases – g(s, t) ≡ f(s, t)

• GC = (C, L) – completely connected graph

• C – set of components

• L – edges fully connecting the components (connections)

• GC – construction graph

SsSs~

30

Combinatorial optimization problem mapping (last )

• Artificial ants build solutions by performing randomized walks on GC(C, L)

31

Construction graph

• Each component ci or connection lij have associated: heuristic information pheromone trail

32

Heuristic information

• A priori information about the problem

• Does not depend on the ants

• On components ci – ηi

• On connections lij – ηij

• Meaning: cost of adding a component to the current

solution

33

Pheromone trail

• Long-term memory about the entire search process

• On components ci – τi

• On connections lij – τij

• Updated by the ants

34

Artificial ant (1)

• Stochastic constructive procedure

• Builds solutions by moving on GC

• Has finite memory for:– Implementing constraints Ω(t)– Evaluating solutions– Memorizing its path

35

Artificial ant (2)

• Has a start state x

• Has termination conditions ek

• From state xr moves to a node from the neighborhood – Nk(xr)

• Stops if some ek are satisfied

36

Artificial ant (3)

• Selects a move with a probabilistic rule depending on: Pheromone trails and heuristic information of

neighbor components and connections Memory Constraints Ω

37

Artificial ant (4)

• Can update pheromone on visited components (nodes)

• and connections (edges)

• Ants act: Concurrently Independently

38

The ACO metaheuristic

While not doStop():

ConstructAntSolutions()

UpdatePheromones()

DaemonActions()

39

ConstructAntSolutions

• A colony of ants build a set of solutions

• Solutions are evaluated using the objective function

40

UpdatePheromones

• Two opposite mechanisms: Pheromone deposit Pheromone evaporation

41

UpdatePheromones: pheromone deposit

• Ants increase pheromone values on visited components and/or connections

• Increases probability to select visited components later

42

UpdatePheromones: pheromone evaporation

• Decrease pheromone trails on all components/connections by a same value

• Forgetting – avoid rapid convergence to suboptimal solutions

43

DaemonActions

• Optional centralized actions, e.g.: Local optimization Ant elitism (details later)

44

ACO applications

• Traveling salesman• Quadratic assignment• Graph coloring• Multiple knapsack• Set covering• Maximum clique• Bin packing• …

45

Outline

1. Biological inspiration of ACO

2. Solving NP-hard combinatorial problems

3. The ACO metaheuristic

4. ACO for the Traveling Salesman Problem

46

ACO for the Traveling Salesman Problem

47

Traveling salesman problem

• N – set of nodes (cities), |N| = n

• A – set of arcs, fully connecting N

• Weighted graph G = (N, A)

• Each arc has a weight dij – distance

• Problem:

Find minimum length Hamiltonian circuit

48

TSP: construction graph

• Identical to the problem graph

• C = N

• L = A

• states = set of all possible partial tours

49

TSP: constraints

• All cities have to be visited• Each city – only once• Enforcing – allow ants only to go to new

nodes

50

TSP: pheromone trails

• Desirability of visiting city j directly after i

51

TSP: heuristic information

• ηij = 1 / dij

• Used in most ACO for TSP

52

TSP: solution construction

• Select random start city

• Add unvisited cities iteratively

• Until a complete tour is built

53

ACO algorithms for TSP

• Ant System

• Elitist Ant System

• Rank-based Ant System

• Ant Colony System

• MAX-MIN Ant System

54

Ant System: Pheromone initialization

• Pheromone initialization

Τij = m / Cnn,

where: m – number of ants Cnn – path length of nearest-neighbor

algorithm

55

Ant System: Tour construction

• Ant k is located in city i

• is the neighborhood of city i

• Probability to go to city :

kiNl

ilil

ijijkijp

kiN

kiNj

56

Tour construction: comprehension

• α = 0 – greedy algorithm

• β = 0 – only pheromone is at work quickly leads to stagnation

kiNl

ilil

ijijkijp

57

Ant System: update pheromone trails – evaporation

Evaporation for all connections∀(i, j) ∈ L:

τij ← (1 – ρ) τij,

ρ ∈[0, 1] – evaporation rate

Prevents convergence to suboptimal solutions

58

Ant System: update pheromone trails – deposit

• Tk – path of ant k• Ck – length of path Tk

• Ants deposit pheromone on visited arcs:

Ljim

k

kijijij

,,1

otherwise

TjiC kkkij

,0

,,/1

59

Elitist Ant System

• Best-so-far ant deposits pheromone on each iteration:

bestij

m

k

kijijij e

1

otherwise

TjiC bestbestbestij

,0

,,/1

60

Rank-based Ant System

• Rank all ants

• Each ant deposits amounts of pheromone proportional to its rank

61

MAX-MIN Ant System

1. Only iteration-best or best-so-far ant deposits pheromone

2. Pheromone trails are limited to the interval [τmin, τmax]

62

Ant Colony System

• Differs from Ant System in three points: More aggressive tour construction rule Only best ant evaporates and deposits

pheromone Local pheromone update

63

Ant Colony System

otherwiseSystemAntlike

qqifj

ililNl ki

,

,maxarg 0

1.Tour Construction

2. Local pheromone update:

τij ← (1 – ξ)τij + ξτ0,

64

Comparing Ant System variants

65

State of the art in TSP

• CONCORDE http://www.tsp.gatech.edu/concorde.html

• Solved an instance of 85900 cities

• Computation took 286-2719 CPU days!

66

Current ACO research activity

• New applications

• Theoretical proofs

67

Further reading

• M. Dorigo, T. Stützle. Ant Colony Optimization. MIT Press, 2004.

• http://iridia.ulb.ac.be/~mdorigo/ACO/

68

Next time…

• Some proofs of ACO convergence

69

Thank you!

Any questions?

This presentation is available at:http://rain.ifmo.ru/~chivdan/presentations/

Daniel Chivilikhin [mailto: chivdan@gmail.com]

70

Used resources• http://teamaltman.com/wp-content/uploads

/2011/06/Uncertainty-Ant-Apple-1024x1024.jpg • http://myrealestatecoach.files.wordpress.com/2012/04/ant.jpg• http://ars.els-cdn.com/content/image/1-s2.0-S1568494613000264-gr3.jpg• http://www.theorie.physik.uni-goettingen.de/forschung/ha/talks/stuetzle.pdf • http://www.buyingandsellingwebsites.com/wp-content/uploads/2011/12/Ant-150.jpg• http://moodle2.gilbertschools.net/moodle/file.php/1040/Event-100_Days/Ant_Hormiga

.gif• http://4.bp.blogspot.com/_SPAe2p8Y-kg/TK3hQ8BUMtI/AAAAAAAAAHA/p75K-GcT_

oo/s1600/ant+vision.GIF• http://1.bp.blogspot.com/-tXKJQ4nSqOY/UQZ3vabiy_I/AAAAAAAAAjA

/IK8jtlhElqk/s1600/16590492-illustration-of-an-ant-on-a-white-background.jpg