Analysis of the Asynchronous Backtracking Algorithm

Drew Wicke

Abstract

The purpose of the paper is to explore the asynchronous backtracking algorithm, a

distributed constraint satisfaction algorithm, through the graph coloring problem. I study the

parallelism of the algorithm by comparing the number of messages sent and the runtime to the

number of vertices. Also, I discuss and analyze various means of prioritizing vertices in a

directed constraint cyclic graph.

Introduction

The asynchronous backtracking algorithm or ABT is used to solve distributed constraint

satisfaction problems, DCSP. The algorithm and the problem domain were originally studied by

Dr. Yokoo and colleagues and published in [1, 2]. The main motivation for the creation and

numerous extensions of the algorithm is the ability to solve constraint problems that would be

difficult or less intuitive to solve using a centralized algorithm [4]. One area that greatly

benefited from this algorithm is multiagent systems. “Many problems that arise in multiagent

systems can be reduced to a distributed constraint satisfaction problem and this approach has

lead to several successful multiagent applications” [3].

The rest of the paper is laid out as follows. First, I describe what constraint satisfaction is

and give example problems. I then give an overview of the ABT algorithm and present the

modifications I chose to implement. I provide the details of the graph coloring experiments and

show their results. Then I discuss the difficulty and the motivation for solving directed

constraint satisfaction problems. Finally, I conclude with a brief look at a current advance in

solving distributed constraint satisfaction problems and describe my future work.

Constraint Satisfaction Problem

A constraint satisfaction problem is a problem in which the goal is to satisfy the

constraints of the problem given a domain of values to choose from. This can be restated more

mathematically as a problem that consists of n variables , whose values are taken from

finite, discrete domains , respectively, and a set of constraints on their values. A

constraint is defined by a predicate. The constraint

is a predicate that is

defined on the Cartesian product × · · · ×

. The predicate can be any type of formula or

relation. “Solving a CSP is equivalent to finding an assignment of values for all variables such

that all constraints are satisfied” [3].

Similarly, distributed constraint satisfaction as defined by Yokoo is “a CSP in which the

variables and constraints are distributed among automated agents” [2]. I assume the agents use

the following communication model:

Agents communicate by sending messages. An agent can send messages to other agents if

and only if the agent knows the addresses of the agents.

The delay in delivering a message is finite, though random. For the transmission between

any pair of agents, messages are received in the order in which they were sent.

[2]

This pertains to how the underlying message passing system works. In my implementation of

the ABT algorithm Yokoo’s communication scheme conforms to the message passing interface

specifications that I used [9].

Now that we know what a CSP is I can provide some examples. “The most widely

studied instance of a constraint satisfaction problem is the graph coloring problem” [8]. The goal

of this problem is to assign a value to each variable such that it does not equal the value of

connected vertices. This is the problem used to explore ABT in this paper. Another classic CSP

problem is the n-queens problem. The problem is: given a chess board with n-queens place them

so that they are not in conflict as per the rules of chess. Both of these problems can be solved

using the asynchronous backtracking algorithm.

Asynchronous Backtracking Algorithm

The asynchronous backtracking algorithm or ABT is the asynchronous version of the

backtracking algorithm. Before the actual algorithm starts the constraint graph is created and

each vertex is given a priority. The connections and priority are then given to each of the

processes.

Essentially, ABT works by each agent first randomly picking a value for its variable from

its domain. Each agent then communicates its value to agents with lower priority via the OK

message type. Every agent except the highest priority agent receives the values of the higher

priority agents and adds the value to its agent view. The agent view is a list that contains what

the agent knows about the other agents’ value. Since the goal is to satisfy all of the constraints,

if a value that it receives does not satisfy a constraint and the agent cannot find a value in its

domain to satisfy the constraint the agent sends a NOGOOD. That is, the agent sends its entire

agent view to the least priority agent in the agent view and it removes that agent’s value from its

agent view. However, if the agent does find a value that satisfies the constraints it will then send

an OK message with its new value to its neighboring lower priority agents. When an agent

receives a NOGOOD it first checks if it matches its agent view. If the values match then it adds

the NOGOOD to its list of NOGOOD messages otherwise the NOGOOD is ignored. These

messages are treated as new constraints on the agent’s value. However, if an agent receives a

NOGOOD that is the empty set then the algorithm terminates with no solution. However, if

there are no messages being sent from any agent then the solution must have been found such

that all of the constraints have been satisfied. The pseudo code from Yokoo’s algorithm from [2]

is in the appendix. It provides a more functional view of the operations of the algorithm.

Analysis

The ABT algorithm is NP-complete since the CSP problem is also NP-complete [2].

This means that the ABT algorithm is “exponential in the number of variables n”[2]. This can be

seen by that fact that in the worst case the algorithm must try all possible combinations in order

to terminate. Another way to view the runtime is to look at the stage in which the algorithm

takes the longest which is checking its constraints against its value. The researchers in [3] found

that this area takes the longest so they tried fixing it by only calling it when necessary. Also,

“The worst-case space complexity of the algorithm is determined by the number of recorded

nogoods” [2].

Implementation

This paper explores a slightly modified version of Yokoo’s original algorithm created in

[3]. I do not add new neighbors to an agent and I also change my agent view when I receive a

valid nogood that has new values. The code is in C and uses the Open MPI library for the

message passing. It was created to solve the graph coloring problem. So, the constraints are

such that if x has a directed constraint going out to y then the value of x cannot equal y’s value.

For the directed version each process picks two random vertices to connect to and its

priority is assigned based on its process id. In the undirected implementation the root or process

zero created connections like the directed version, but it then made the connection bidirectional

to create an undirected graph. From there, each process then chooses a random value from its

domain and sends it to the lower priority agents. The algorithm continues like the pseudo code

until either it terminates due to an agent producing the empty set or no messages are sent from

any of the process.

Experiments and Results

The algorithm was run on a cluster and was created to solve the graph coloring problem

with three colors for each vertex. I measure the runtime, number of OK messages sent and the

number of NOGOOD messages sent. The reason for some of the discrepancies in the runtime

may be due to network latency.

First, I tested the ABT on 75 random directed graphs for 3, 4, 5 and 6 variables with a

domain of three values. The figure below shows the relation between the number of variables

and the number of ok messages sent. It can be seen that the median of the values is increasing

very quickly as the number of variables increases.

0

5

10

15

20

25

30

35

0 1 2 3 4 5 6 7

# o

f O

K M

essa

ges

Number of Variables

Directed GraphNumber of Variables vs. OK Messages

Three

Four

Five

Six

I also produced a graph for the number of NOGOOD messages sent for the directed

version. It is easy that when there are few variables the number of NOGOODs sent is small. It

is also dependant on the original choice for the value of the variable.

The table below shows the average time to complete the algorithm in micro seconds for

four, five and six variables. As is clear the more variables on average the length of time it takes

to complete increases.

# of variables 4 5 6

Time in micro

seconds

13046.63 13687.07 16739.75

Next, I experimented with the algorithm on undirected graphs. As described in the

implementation section I created 75 random undirected graphs and measured the same data only

for three, four and five variables. The first graph shows the relationship between the number of

OK messages and the number of variables. For the undirected graphs with three and four

variables there were no NOGOODs sent. However, for five variables there was a very small

percentage of the graphs sampled that had very large numbers of OK messages and NOGOOD

0

2

4

6

8

10

12

0 1 2 3 4 5 6 7

# o

f N

OG

OO

D M

ess

age

s

Number of Variables

Directed GraphNumber of Variables vs. NOGOOD Messages

Three

Four

Five

Six

messages sent. This may be due to an unsolvable graph that had a worst case configuration of

connections. Finally, I show the average runtimes for each of the variable sizes.

# of variables 3 4 5

Time in micro

seconds

10784.39 14513.92 14965.53

I believe the increase in the number of OK messages being sent when there are five

variables is due to the possible increase in the number of constraints since every time a variable

changes, all of the lower priority agents receive an ok message with the agent’s new variable

assignment.

Prioritization

I continue exploring the algorithm by trying various ways to prioritize the agents in the

ABT algorithm. In a directed constraint graph, each agent may not know all constraint

predicates relevant to its variable. To combat this problem a priority is given to each of the

agents before the algorithm is run. In some cases an incorrect priority of an agent will lead to an

invalid solution. The figure below shows a very simple example of a case where an invalid

0

5

10

15

20

25

30

0 1 2 3 4 5 6

# o

f O

K M

ess

age

s

Number of Variables

Undirected GraphNumber of Variables vs. OK Messages

Three

Four

Five

solution is obtained. If agent one has the highest priority and two has the least then the following

will cause the algorithm to complete as though this was a valid result.

In both [6 and 7] they develop a heuristic to reorder the agents while the algorithm is

running in order to solve the problem of prioritization. I tried two ways to reorder the agents

before the algorithm runs. However, neither of them fully solved the problem in certain cases.

First I tried prioritizing the agents based on their in-degree. Agents with a higher in-

degree received a higher rank. This solved the problem of types like the one shown in the figure

above. However, when ties occurred I did not do anything to resolve them. This caused the

problem where each node had the same in-degree creating an invalid solution. An example is

shown in the figure below.

To patch this I tried factoring in the number of bidirectional links an agent had. The

more bidirectional links the lower the priority the agent. Again this did not provide a complete

solution due to a graph like the one in the figure below.

The solvability of this problem is not essential to the correctness of the algorithm. The

algorithm would just require all constraints to be undirected. I should note that one of the

advantages of ABT is the privacy of the constraints. By allowing directed graphs and

centralizing the computation of the priority I lose this privacy since one agent knows all

constraints in order to provide a priority to each agent. However, this issue of assigning priority

is relevant if it is the only way to solve the constraint problem.

Extensions

The ABT algorithm has numerous extensions, but one that is most interesting and the

state of the art is the message management ABT or MMABT [3]. This algorithm improves ABT

by decreasing the number of messages sent. This greatly improved upon the original ABT

algorithm because it sent too many useless messages. They did this by handling received

messages as batches rather than individually. In so doing they found that this allowed the

agent’s view to stay up to date thus providing a more efficient result. They also found that it is

useless to add links to non-neighboring agents during the received nogood function. They also

show that it is beneficial to change all the values in the agent view to the values that an agent

receives in a nogood message.

Conclusion

In conclusion, the ABT algorithm provides a nice way to solve constraint satisfaction

problems when working in a distributed environment. ABT allows for possible privacy of

constraints from other agents and it promotes better utilization of current multiprocessor

technologies.

The results of my experiments are similar to those in [3] and they make sense. Future

research involves continuing researching a way to prioritize undirected graphs possibly using a

modified topological sorting with depth first search. Also, I would like to see the speed up if I

used the GPU to check the agent’s view and nogoods since this area of the program is the most

intensive [3].

Works Cited

[1] M. Yokoo, E.H. Durfee, T. Ishida, and K. Kuwabara. Distributed constraint satisfaction for

formalizing distributed problem solving. Proc. 12th International Conference on Distributed

Computing Systems, ICDCS'92, pages 614-621, 1992.

[2] M. Yokoo, E.H. Durfee, T. Ishida, and K. Kuwabara. The distributed constraint satisfaction

problem: formalization and algorithms. IEEE Transactions on Knowledge and Data Engineering,

10(5):673-685, 1998.

[3] Jiang, H. and Vidal, J. M. (2006). The message management asynchronous backtracking

algorithm. Journal of Experimental and Theoretical Artificial Intelligence. To appear.

[4] C. Bessiere, A. Maestre, I. Brito, and P. Meseguer. Asynchronous backtracking without

adding links: a new member in the abt family. Artificial Intelligence, 161:1-2:7–24, January

2005.

[6] M.-C. Silaghi. Framework for modeling reordering heuristics for asynchronous backtracking.

In IAT, 2006.

[7] M.-C. Silaghi, D. Sam-Haroud, and B. Faltings. ABT with asynchronous reordering. In IAT,

2001.

[8] José M. Vidal. Fundamentals of Multiagent Systems: Using NetLogo Models. , Unpublished,

2006. . http://www.multiagent.com

[9] http://www.mpi-forum.org/docs/mpi22-report/node54.htm#Node54

Appendix

Asynchronous backtracking algorithm as posed by [2].

Known Bugs

In my implementation of the ABT algorithm when testing using a large number of variables the

algorithm crashes. This bug is known and it has been test to ensure that it does not affect the results of

smaller number of variables. In the next version of this algorithm this bug is expected to be fixed.