Constraint Satisfaction Constraint Satisfaction ProblemsProblems

ContentsContents

RepresentationsRepresentationsSolving with Tree Search and HeuristicsConstraint PropagationTree Clustering

Posing a CSPPosing a CSP

A set of variables V1, …, Vn

A domain over each variable D1,…,Dn

A set of constraint relations C1,…,Cm between variables which indicate permitted combinations

Goal is to find an assignment to each variable such than none of the constraints are violated

Constraint GraphsConstraint Graphs Nodes = variables Edges = constraints

Example: map coloring

A

B

C

D

A

B C

D

N-ary Constraint GraphsN-ary Constraint Graphs Example:

– Variables: X=[1,2] Y=[3,4] Z=[5,6]– Constraints: X + Y = Z

(Roman Barták, 1998 )

X

Y Z

Hyper graph

X

Y Z

Primal constraint graph

Making a Binary CSPMaking a Binary CSP

Can convert n-ary constraint C into a unary constraint on new variable Vc

– Dc = cartesian product of vars in C

Can convert n-ary CSP into a binary CSP– Create var Vc for each constraint C (as above)– Domain Dc = cartesian product – tuples that violate C– Add binary equivalence constraints between new

variables Vc, Vc’:C,C’ share var X Vc,Vc’ must agree on X

Making a Unary ConstraintMaking a Unary Constraint Variables: X=[1,2] Y=[3,4] Z=[5,6] Constraints: X + Y = Z

X

Y Z

(Roman Barták, 1998 )

XYZ

[(1,3,5),(1,3,6), (1,4,5),(1,4,6), (2,3,5),(2,3,6) (2,4,5),(2,4,6)]

XYZ=

Making a Unary ConstraintMaking a Unary Constraint Variables: X=[1,2] Y=[3,4] Z=[5,6] Constraints: X + Y = Z

X

Y Z

(Roman Barták, 1998 )

XYZ

XYZ= [(1,4,5), (2,3,5), (2,4,6)]

Making a Binary CSPMaking a Binary CSP

Variables: X=[1,2] Y=[3,4] Z=[5,6] W=[1,3] Constraints: X + Y = Z, W<Y

(Roman Barták, 1998 )

XYZ

WY

Y

Dual constraint graph

X

Y Z

W

Making a Binary CSPMaking a Binary CSP

Variables: X=[1,2] Y=[3,4] Z=[5,6] W=[1,3] Constraints: X + Y = Z, W<Y

(Roman Barták, 1998 )

XYZ

WY

Y

Dual constraint graph

X

Y Z

W

ContentsContents

RepresentationsSolving with Tree Search and HeuristicsSolving with Tree Search and HeuristicsConstraint PropagationTree Clustering

Generate and TestGenerate and Test

Generate each possible assignment to the variables and test if constraints are satisfied– Exponential possibilities: O(d n)– Simple but extremely wasteful!

DFS and BacktrackingDFS and Backtracking

Depth first search– Levels represent variables – Branches off nodes represent a possible instantiations

of variables

Test against constraints after every variable instantiation and backtrack if violation– Incrementally attempts to extend partial solution– Whole subtrees eliminated at once

ExampleExample

red green blue

red red green

V1

V2 V3

(*,*,*)

ExampleExample

red green blue

red red green

V1

V2 V3

(*,*,*)

(r,*,*)

ExampleExample

red green blue

red red green

V1

V2 V3

(*,*,*)

(r,*,*)

(r,r,*)

ExampleExample

red green blue

red red green

V1

V2 V3

(*,*,*)

(r,*,*) (g,*,*)

(r,r,*) (g,r,*)

ExampleExample

red green blue

red red green

V1

V2 V3

(*,*,*)

(r,*,*) (g,*,*)

(r,r,*) (g,r,*)

(g,r,r) (g,r,g)

ExampleExample

red green blue

red red green

V1

V2 V3

(*,*,*)

(r,*,*) (g,*,*)

(r,r,*) (g,r,*)

(g,r,r)

(b,*,*)

(b,r,*)

(b,r,r)(g,r,g) (b,r,g)

Forward CheckingForward Checking

Backtracking is still wasteful– A lot of time is spent searching in areas where no

solution remains– Ex. setting V4 to value X1 eliminates all possible values

for V8 under the given constraints– Can cause thrashing

Forward checking removes restricted values from the domains of all uninstantiated variables– If a domain becomes empty backtracking is done

immediately

HeuristicsHeuristics

The search can usually be sped up by searching intelligently:– Most-constrained variable: Expand subtree of

variables that have the fewest possible values within their domain first

– Most-constraining variable: Expand subtree of variables which most restrict others first

– Least-constraining value: Choose values that allow the most options for the remaining variables first

ContentsContents

RepresentationsSolving with Tree Search and HeuristicsConstraint PropagationConstraint PropagationTree Clustering

Constraint PropagationConstraint Propagation

A preprocessing step to shrink the CSP– Constraints are used to gradually narrow down the

possible values from the domains of the variables

A singleton may result– If the domains of each variable contain a single value

we do not need to search

Arc ConsistencyArc ConsistencyArc (Vi,Vj) in a constraint graph is arc

consistent if for every value of Vi there is some value that is permitted for Vj

Algorithm:

Complexity O(ed3)

do foreach edge (i,j) delete values from Di that cause Arc(Vi,Vj) to failwhile deletions

ExampleExample

green

red green blue green blue

V1

V2 V3

Consider edge (1,3)

ExampleExample

green

red green blue green blue

V1

V2 V3

Consider edge (3,1)

ExampleExample

green

red green blue green blue

V1

V2 V3

Consider edge (2,1)

ExampleExample

green

red green blue green blue

V1

V2 V3

Consider edge (2,3)

ExampleExample

green

red green blue green blue

V1

V2 V3

Consistent and a singleton!

Levels of ConsistencyLevels of Consistency

Algorithms we have seen before are combinations of tree search and arc consistency:

Generate and TestBacktrackingForward CheckingPartial LookaheadFull LookaheadReally Full Lookahead

(Nadel, 1988)

TSBT = TS + AC 1/5FC = TS + AC 1/4PL = FC + AC 1/3FL = FC + AC 1/2RFL = FC + AC

BacktrackingBacktracking

function BT(i,var)for(var[i]=Di) CONSISTENT = true for(j=1:i-1) CONISITENT = check(i,var[i],j,var[j]) end

if CONSISTENT if i==n disp(var) else BT(i+1,var)end

Given:– check(i,Xi,j,Xj): true if Vi = Xi and Vj = Xj is permitted

by constraints– revise(i,j): true if Di is empty after making Arc(Vi,Vj)

= true

function BT(i)EMPTY_DOMAIN = check_backward(i)if ~EMPTY_DOMAIN for(var[i]=Di) Di = var[i] if i==n disp(var) else BT(i+1) end

function check_backward(i)for(j=1:i-1) if revise(i,j) return trueendreturn false

Forward CheckingForward Checkingfunction FC(i)EMPTY_DOMAIN = check_forward(i)if ~EMPTY_DOMAIN for(var[i]=Di) Di = var[i] if i==n disp(var) else FC(i+1) end

function check_forward(i)if i>1 for(j=i:n) if revise(j,i-1) return true end return false

Similar to backtracking except more arc-consistency

Levels of ConsistencyLevels of Consistency

Generate and TestBacktrackingForward CheckingPartial LookaheadFull LookaheadReally Full Lookahead

(Nadel, 1988)

TSBT = TS + AC 1/5FC = TS + AC 1/4PL = FC + AC 1/3FL = FC + AC 1/2RFL = FC + AC

A Stronger Degree of ConsistencyA Stronger Degree of Consistency

A graph is K-consistent if we can choose values for any K-1 variables that satisfy all the constraints, then for any Kth variable be able to assign it a value that satisfies the constraints

A graph is strongly K-consistent if J-consistent for all J < K– Node consistency is equivalent to strong 1-consistency – Arc consistency is equivalent to strong 2-consistency

Towards Backtrack Free Towards Backtrack Free SearchSearch

A graph that has strong n-consistency requires no search– Acquiring strong n-consistency is exponential

in the number of variables (Cooper, 1989)

For a general graph that is strongly k-consistent (where k < n) backtracking cannot be avoided

ExampleExample

Arc consistent, yet a search will backtrack!

red green

red green green blue

V1

V2 V3

(*,*,*)

(r,*,*)

(r,r,*) …

Constraint Graph WidthConstraint Graph Width

The nodes of a constraint graph can be ordered

V1

V2 V3

V1

V2

V3

V1

V3

V2

V2

V1

V3

V2

V3

V1

V3

V1

V2

V3

V2

V1

The width of a node in an ordered graph is equal to the number of incoming arcs from higher up nodes

The width of an ordered graph is the max width of its vertices

The width of a constraint graph is the min width of each of its orderings

1 1 1 2 1 2 1

Backtrack Free SearchBacktrack Free Search

Theorem: If a constraint graph is strongly K-consistent, and K is greater than its width, then there exists a search order that is backtrack free

– K>2 consistency algorithms add arcs requiring even greater consistency

– If a graph has width 1 we can use node and arc consistency to get strong 2-consistency without adding arcs

– All tree structured constraint graphs have width 1 (Freuder 1988)

ContentsContents

RepresentationsSolving with Tree Search and HeuristicsConstraint PropagationTree ClusteringTree Clustering

Tree Clustering MotivationTree Clustering Motivation Tree structured constraint graphs can be solved

without backtracking– We would like to turn non-tree graphs into trees by

grouping variables– The grouped variables themselves become smaller

CSP’s– Solving a CSP is exponential in the worst case so

reducing the number of variables we consider at once is also important

If we want the CSP for many queries it is worth investing more time in restructuring it

(Dechter, 1988)

RedundancyRedundancy Constraints in the dual graph are equalities

Variables: A, B, C, D, E, F Constraints: (ABC), (AEF), (CDE), (ACE)

ABC AEF

ACE CDE

A

CE

AC EC

AE

ABC AEF

ACE CDECE

AC AE

Join graph/tree

Tree ClusteringTree Clustering

If the dual graph cannot be reduced to a join tree we can still make it acyclic:

– Condition for acyclicity: A CSP is acyclic iff its primal graph is chordal and conformal

Given a primal graph its dual can be made acyclic:– Triangulate graph to make it chordal – The maximal cliques are constraints/nodes in the new

dual graph

(Beeri, 1983)

TriangulationTriangulation

Use maximum cardinality search (m-ordering) to order the nodes

Add an edge between any two nonadjacent nodes that are connected by nodes higher in the ordering

(Tarjan, 1984)

The AlgorithmThe Algorithm

Build the primal graph for the CSP– O(n2)

Triangulate– O(n2)

Use maximal cliques as new nodes in dual graph– O(n)

Remove any redundancies in the new graph– O(n)

ExampleExample Variables: A, B, C, D, E Constraints: (A,C), (A,D), (B,D), (C,E), (D,E)

AD

AC

BD

DE

CE

D D

A E

D

C

AD

AC

BD

DE

CE

D D

A E

C

Still cyclic!

ExampleExample Variables: A, B, C, D, E Constraints: (A,C), (A,D), (B,D), (C,E), (D,E)

E

C D

A B

E

C

D

A

B

Order: E, D, C, A, B

E

C D

A B

ExampleExample Variables: A, B, C, D, E Constraints: (A,C), (A,D), (B,D), (C,E), (D,E)

E

C D

A B ACD

BD

CDE

D D

CD ACD

BD

CDE

D

CD

Acyclic!

Solving the CSPSolving the CSP

Solve each node of the tree as a separate small CSP– This can be done in parallel– The solutions to each node constitute the domain of that

node in the tree– O(d m)

Use arc consistency to reduce the domains of each node

Solve the entire CSP without backtracking

AppendixAppendix

Example CSP’sExample CSP’s

N-queensMap coloring Cryptoarithmatic

Wireless network base station placementObject recognition from image features

Heuristic RepairHeuristic Repair

Start with a random instantiation of variables, choose a variable and reassign it so that fewest constraints are violated

Repeat this some number of times, if constraints still violated, restart with a new random instantiation

Similar to GSAT

Graham’s AlgorithmGraham’s Algorithm

Given a dual constraint graph– If Vi is a variable that appears in exactly one node then

remove Vi

– If the variables in Ni are a subset of variables in another node Nj then remove Ni

Repeat until neither applies Graph is acyclic if the result is the empty set Easy to verify

(Graham, 1979)

Graham’s AlgorithmGraham’s Algorithm Trees collapse to empty set

– Edges in dual graph are constraints on common vars– Nodes in a tree share vars only with their parents – Step 1 removes any unique vars from the children– After step 1 the children are removed by step 2 since

they are now subsets of the parent

Step-1

Step-2

[ ]

Step-1

Graham’s AlgorithmGraham’s Algorithm Cycles don’t collapse

– Step 1 will fail since cycles are created among nodes with vars that are shared among multiple nodes

– Step 2 must fail since if any node was a subset of another it would have to share all of its variables with at least one other node

Graham’s Algorithm Graham’s Algorithm Primal Primal Graph is ChordalGraph is Chordal

Assume Graham’s algorithm succeeds– Step 1 must have removed every node of the

primal graph– Assume there was a chordless cycle

Let z be a node on the cycle that is first eliminatedLet x, y be nodes on the cycle adjacent to zFor step 1 to apply z must have belonged to only one constraint (which are cliques/hyperedges)With z gone the constraint is left with x and yThus x and y are connected contradiction!

Why m-ordering?Why m-ordering?

We want an ordering that will not add any edges to chordal graphs

Property P (for zero fill-in): – If u < v < w– If (u,w) is an edge– If (v,w) is not an edge– Then exists vertex x

v < x (v, x) is an edge (w, x) is not an edge

w

uv

x

Proof of Property PProof of Property P Assumptions

– Given chordal graph G– Given an ordering with Property P

Define property Q:– Let V0, V1, …, Vk be an unchorded path for which (Vk)

is maximum– Vk > … > Vi+1 > Vi < … < V2 < V1 < Vk < V0

– Not possible!

w

v

u

…

…

…

- order: u < v < w

- if no edge (v,w) then v, u, w satisfies property Q

M-ordering Satisfies PM-ordering Satisfies P

Suppose u < v < w

w

uv

x