Ant Colony Optimization for Testing Concurrent Systems ... · Ant Colony Optimization for Testing Concurrent Systems: Analysis of Scalability Francisco Chicano and Enrique Alba. Introduction

1 / 27META’08, Hammamet, Tunisia, October 29-31, 2008

Ant Colony Optimization for Testing Concurrent Systems: Analysis of Scalability

Francisco Chicano

and Enrique Alba

Introduction Background Algorithmic Proposal Experiments

Conclusions & Future Work


• Concurrent software is difficult to test ...

• ... and it is in the heart of a lot of

critical systems

• Techniques for proving the correctness

of concurrent software are required•Model checking

→ fully automatic

• Traditional techniques for this purpose have problems with large models•We analyze here the scalability

of a new proposal: ACOhg-mc

MotivationMotivation




Intersection Büchi

automaton

• Objective: Prove that model

M satisfies the property :

• HSF-SPIN: the property f is an LTL formula

s0

s4s7

s6

s2

s1

s8s9

s5

s3

s0

s1s2

s3

s4

s5

Model M

s1

s0 s2

qp∧!q

!p∨q

LTL formula ¬

f(never claim)

∩ =

Explicit State Model CheckingExplicit State MC State Explosion Heuristic MC

Safety & Liveness




Intersection Büchi

automaton




s0

s4s7

s6

s2

s1

s8s9

s5

s3

s0

s1s2

s3

s4

s5

Model M

s1

s0 s2

qp∧!q

!p∨q

LTL formula ¬

f(never claim)

∩ =


Safety & Liveness




Intersection Büchi

automaton




s0

s4s7

s6

s2

s1

s8s9

s5

s3

s0

s1s2

s3

s4

s5

Model M

s1

s0 s2

qp∧!q

!p∨q

LTL formula ¬

f(never claim)

∩

Using Nested-DFS

=


Safety & Liveness




s0

s4s7

s6

s2

s1

s8s9

s5

s3

• Number of states very large

even for small models

• Example: Dining philosophers with n

philosophers

→ 3n

states20 philosophers → 1039 GB

for storing the states

•

Solutions: collapse compression, minimized automaton representation, bitstate hashing, partial order reduction, symmetry reduction

• Large models cannot be verified but errors can be found

Memory

State Explosion Problem



Explicit State MC

State Explosion

Heuristic MC

Safety & Liveness


• The search for errors can be directed by using heuristic information

• Different kinds of heuristic functions have been proposed in the past:

• Formula-based

heuristics

• Structural

heuristics

s0

s4

s7

s6

s2

s1

s8s9

s5

s32

0

3

5

1

2

40

76

Heuristic value



Heuristic Model Checking

• Deadlock-detection

heuristics

• State-dependent

heuristics

Explicit State MC

State Explosion Heuristic MC

Safety & Liveness


Safety property

• Counterexample ≡

path to accepting state

• Graph exploration algorithms can be used: DFS

and BFS

Liveness

property

• Counterexample

≡

path to accepting cycle

• It is not possible to apply DFS or BFS

s0

s4s7

s6

s2

s1

s8s9

s5

s3

Safety and Liveness

Properties



s0

s4s7

s6

s2

s1

s8s9

s5

s3

Explicit State MC

State Explosion Heuristic MC

Safety & Liveness


OPTIMIZATION/SEARCH TECHNIQUESOPTIMIZATION/SEARCH TECHNIQUES

EXACTEXACT APPROXIMATEDAPPROXIMATED

Ad Hoc

HeuristicsAd Hoc

Heuristics METAHEURISTICSMETAHEURISTICS

Optimization/Search

Techniques

• Newton• Gradient

Based on CalculusBased on Calculus

• Depth

First

Search• Branch

and

Bound

EnumerativesEnumeratives

• SA• VNS• TS

Trayectory-basedTrayectory-based

• EA• ACO• PSO

Population-basedPopulation-based



Metaheuristics

ACO

ACOhg

ACOhg-mc


•

Ant Colony Optimization (ACO) metaheuristic

is inspired by the foraging behaviour of real ants

• ACO Pseudo-code

ACO: Introduction



Metaheuristics

ACO ACOhg

ACOhg-mc


• The ant selects its next node stochastically

• The probability of selecting one node depends on the pheromone trail and theheuristic value (optional) of the edge/node

• The ant stops when a complete solution is built

i

j

l

m

k

Ni

τij

ηij

kTrail

Heuristic

ACO: Construction Phase



Metaheuristics

ACO ACOhg

ACOhg-mc


• Pheromone update

During the construction phase

After the construction phase

• Trail limits (particular of MMAS)

Pheromones are kept in the interval [τmin, τmax]

ACO: Pheromone Update



with

Metaheuristics

ACO ACOhg

ACOhg-mc

with


The length of the ant paths is limited by λant λant Objective node

What if…?

Starting nodes for path construction change

After σs

steps

Second stage Third stage

Initial node

ACOhg: Huge Graphs Exploration



Metaheuristics

ACO

ACOhg

ACOhg-mc


• The search is an alternation of two phases

First phase: search for accepting states

Second phase: search for cycles from the accepting states

ACOhg-mc



Metaheuristics

ACO

ACOhg

ACOhg-mc

ACOhg-mc

Pseudocode

First

phase





ACOhg-mc



Metaheuristics

ACO

ACOhg

ACOhg-mc

ACOhg-mc

Pseudocode

Second

phase





ACOhg-mc



Metaheuristics

ACO

ACOhg

ACOhg-mc

ACOhg-mc

Pseudocode

Tabu

list


•We used 4 scalable Promela

models

for the experiments

• Parameters for ACOhg-mc

• Formula-based

and finite state machine

heuristics• ACOhg-mc implemented in HSF-SPIN• 100

independent executions

Model LoC Processes Propertymarriersj 64 j+1 deadlock elevj 191 j+3 □(p → ◊q)giopj 740 j+6 deadlock and □(p → ◊q)phij 57 j+1 deadlock and □(p → ◊q)

Promela

Models



Models & parameters Results

Parameter msteps colsize λant σs ξ a ρ α β

1st phase100

10 404

0.75 0.2 1.0 2.0

2nd phase 20 4 0.5


•We used 4 scalable Promela

models

for the experiments

• Parameters for ACOhg-mc

• Formula-based

and finite state machine

heuristics• ACOhg-mc implemented in HSF-SPIN• 100

independent executions

Model LoC Processes Propertymarriersj 64 j+1 deadlock elevj 191 j+3 □(p → ◊q)giopj 740 j+6 deadlock and □(p → ◊q)phij 57 j+1 deadlock and □(p → ◊q)

Promela

Models



Models & parameters Results

j=2 to

30

Parameter msteps colsize λant σs ξ a ρ α β

1st phase100

10 404

0.75 0.2 1.0 2.0

2nd phase 20 4 0.5


Efficacy



Models & parameters

Results

0 4 8 12 16 20 24 28

phi-r

giop-r

elev-r

phi-d

giop-d

marriers-d

Mod

els

n

ACOhg-mc NDFS


Analysis of Scalability: Memory



Models & parameters

Results

0

2000

4000

6000

8000

10000

12000

14000

2 6 10 14 18 22 26 30

n

Mem

ory

(KB

)

marriers-d giop-d phi-d elev-r giop-r phi-rACOhg-mc


0

50000

100000

150000

200000

250000

300000

350000

400000

2 6 10 14 18 22 26 30

n

Mem

ory

(KB

)

marriers-d giop-d phi-d elev-r giop-r phi-r

Analysis of Scalability: Memory



Models & parameters

Results

NDFS


Analysis of Scalability: Length



Models & parameters

Results

0

100

200

300

400

500

600

700

800

2 6 10 14 18 22 26 30

n

Leng

th



0

1000

2000

3000

4000

5000

6000

7000

8000

9000

10000

2 6 10 14 18 22 26 30

n

Leng

th

marriers-d giop-d phi-d elev-r giop-r phi-r

Analysis of Scalability: Length



Models & parameters

Results

NDFS


Analysis of Scalability: CPU time



Models & parameters

Results

0

5000

10000

15000

20000

25000

30000

35000

2 6 10 14 18 22 26 30

n

Tim

e (m

s)



Analysis of Scalability: CPU time



Models & parameters

Results

0

10000

20000

30000

40000

50000

60000

70000

80000

90000

2 6 10 14 18 22 26 30

n

Tim

e (m

s)

marriers-d giop-d phi-d elev-r giop-r phi-rNDFS


• ACOhg-mc is able

to find errors in large models

for which NDFS fails

• The memory required by ACOhg-mc for the search is small and grows very slowly

• The length of the error trails increases linearly

in most of the cases

• Although ACOhg-mc is not always the fastest algorithm, the time required is small

Conclusions

Future Work

• Analysis of parameterization for reducing the parameters

• Include ACOhg-mc into Java PathFinder

for finding errors in Java programs

•

Combine ACOhg-mc with techniques for reducing the memory

required for the search such as partial order reduction






Thanks for your attention !!!

Ant Colony Optimization for Testing concurrent Systems: Analysis of Scalability

Documents

Ant Colony Optimization for Testing Concurrent Systems ... · Ant Colony Optimization for Testing Concurrent Systems: Analysis of Scalability Francisco Chicano and Enrique Alba. Introduction