1

Chapter 5

Constraint Satisfaction Problems

2

OutlinesConstraint Satisfaction ProblemsBacktracking Search for CSPsLocal Search for CSPThe structure of problems

3

Constraint Satisfaction Problems (CSP)

What is a CSP?– Finite set of variables X1, X2, …, Xn

– Domain of possible values for each variable Dx1, Dx2, … Dxn

– {C1, C2, …, Cm} Set of constraint relations that limit the values the variables can take on, e.g., X1 ≠ X2

Solution: Assignment of value to each variable such that the constraints are all satisfied

Consistent, objective function.Applications: Scheduling the time of observations on

the Hubble Space Telescope, Floor planning, Map coloring, Cryptography

4

Search and CSPs Search:

– Each state is a black box with no internal structure– States support goal test, evaluation, successor

function– Path to goal is important

CSP:– State is defined by variables with values from

domain– Goal test is a set of constraints specifying

allowable combinations of values for subsets of variables

– Solution is more important than path to goal

5

ConstraintsGiven a set of variables, each with a set of

possible values (depends on domain), assign a value to each variable that:– satisfies some set of explicit constraints

“hard constraints”– minimizes some cost function, where each

variable assignment has a cost“soft constraints”

6

CSP Example: Map Coloring

[From Russell and Norvig’s AIMA slides]

7

Solution to Map Coloring

8

Map Coloring represented as a Constraint Graph

9

Incremental Formulation of CSPInitial state: {}Successor function: a value can be

assigned to any unassigned variable, subject to constraints

Goal test: current assignment is complete

Path cost: constant cost for every step

10

Complete-state formulationState is a complete assignmentMight or might not satisfy the constraintsLocal search method.

11

DomainDiscrete

– Finite domain: • Map coloring,

• 8-Queens problem: Q1, Q2, …, Q8

– Di={1, 2, …,8}, d=|Di|

• All possible assignment O(dm)• Exponential in the number of variables.• Boolean CSP: 3SAT problem (NP-complete

problem).

12

DomainDiscrete

– Infinite domain: D=N, D=R

– Constraint language• StratJob1 + 5 < StartJob3• Linear Constraints• Nonlinear constraint

13

DomainContinuous

– Operation research (OR)

– AP: Hubble Space Telescope

– Linear programming:

– Integer Linear Programming

– Quadratic programming

14

Types of ConstraintsHard Constraints:

– Unary: single variable– Binary: pairs of variables– Higher-Order: 3 or more variables

Soft constraints: preferences represented by cost for each variable assignment

15

Example of Higher-Order CSP: Cryptarithmetic

16

Auxiliary variablesX1, X2, X3 {0, 1}Constraint hypergraph.Every higher-order, finite-domain constraints can

be reduced to a set of binary constraints if enough auxiliary variables are introduced.

Types of constraints– Absolute constraints: can not be violated.– Preference constraints: timetable scheduling, may be

violated.

17

Turtle Puzzle

•What are your variables?•What are the domains of possible values?•What are the constraints?

18

Search Method to CSPApply BFS to CSPBranch factors

– Incremental state formulation.– First level: nd( d=|D|, n=# of variables)– Second level: (n-1)d( d=|D|, n=# of variables)– …– Size of Trees (n! dn)

19

Commutativity Property of CSPsCommutativity: Order of application of actions

has no effect on the outcome.All CSP search algorithms consider a single

variable assignment at a timeExample: We choose colors for Australian

territories one at a time. The number of leaves is dn.

20

Backtracking SearchDFS that chooses values for one variable at a time

and backtracks when a variables has no legal values left to assign.

21

Backtracking Example: Map Coloring

22

Backtracking Example: Map Coloring

23

Backtracking Example: Map Coloring

24

Backtracking SearchPlain backtracking is uninformed search.

25

Improving Backtracking EfficiencyWhich variable (or value) should be

assigned next?In what order should its values be tried?Can we detect inevitable failure early (and

avoid the same failure in subsequent paths)?

26

Most constrained variableMost constrained variable:

choose the variable with the fewest legal values

a.k.a. minimum remaining values (MRV) heuristic If there is a variable X with zero legal values remaining,

the MRV will select X and failure will be detected immediately (pruning the search tree)

–

27

Most constraining variableTie-breaker among most constrained variables

– Degree heuristic

Most constraining variable:– choose the variable with the most constraints on

remaining variables

–

5

28

Least constraining valueGiven a variable, choose the least constraining

value:– the one that rules out the fewest values in the

remaining variables

Combining these heuristics makes 1000 queens feasible

–

29

Backtracking Search

Plain backtracking is uninformed search.

30

Forward CheckingWhenever a variable X is designed, the forward

checking process looks at each unassigned variable Y that is connected to X by a constraint and deletes from Y’ domain any value that is inconsistent with the value chosen for X.

31

Forward checking Idea:

– Keep track of remaining legal values for unassigned variables

– Terminate search when any variable has no legal values

–

32

Forward checking Idea:

– Keep track of remaining legal values for unassigned variables

– Terminate search when any variable has no legal values

–

33

Forward checking Idea:

– Keep track of remaining legal values for unassigned variables

– Terminate search when any variable has no legal values

–

34

Forward checking Idea:

– Keep track of remaining legal values for unassigned variables

– Terminate search when any variable has no legal values

–

inconsistent

35

Constraint propagation Forward checking propagates information from assigned to

unassigned variables, but doesn't provide early detection for all failures:

NT and SA cannot both be blue! Constraint propagation repeatedly enforces constraints locally

Undetected

36

Arc consistency Simplest form of propagation makes each arc consistent X Y is consistent iff

for every value x of X there is some allowed y

–

37

Arc consistency Simplest form of propagation makes each arc consistent X Y is consistent iff

for every value x of X there is some allowed y

–

38

Arc consistency Simplest form of propagation makes each arc consistent X Y is consistent iff

for every value x of X there is some allowed y

If X loses a value, neighbors of X need to be rechecked

–

39

Arc consistency Simplest form of propagation makes each arc consistent X Y is consistent iff

for every value x of X there is some allowed y

If X loses a value, neighbors of X need to be rechecked Arc consistency detects failure earlier than forward checking Can be run as a preprocessor or after each assignment

–

40

Arc Consistency As a preprocessing Step. As a propagation Step (forward checking) after every assignment

during search (MAC algorithm, maintaining arc consistency). Queue for storing arcs.

41

Arc consistency algorithm AC-3

Time complexity: O(n2d3)

42

Arc consistencyK-consistent1-consistent=node consistency2-consistent=arc consistency3- consistency =path consistencyA graph is strongly consistency it is k-

consistency and is also (k-1)-consistency, …

43

Local search for CSPsHill-climbing, simulated annealing typically work

with "complete" states, i.e., all variables assigned– The initial state assigns a value to every one variables– Successor function works by changing the value of

one variable at a time.Example: 8-Queens problem

– Successor function = swap rowsTo apply to CSPs:

– allow states with unsatisfied constraints– operators reassign variable values–

–

44

Min-conflictVariable selection: randomly select any

conflicted variableValue selection by min-conflicts heuristic:

– choose value that violates the fewest constraints

– i.e., hill-climb with h(n) = total number of violated constraints

–

–

45

Min-Conflicts algorithm

46

Example

47

Example: 4-Queens States: 4 queens in 4 columns (44 = 256 states) Actions: move queen in column Goal test: no attacks Evaluation: h(n) = number of attacks

Given random initial state, can solve n-queens in almost constant time for arbitrary n with high probability (e.g., n = 10,000,000)

48

Local search for CSPMin-Conflicts is surprisingly effective for many CSPs.Particular when given a reasonable initial state.For n-Queens problem, (don’t count the initial state) the

runtime of min-conflicts is roughly independent of problem size.

Million-queens problem in 50 steps.Application:

– Schedule

– Plan schedule

– Repair the schedule with a minimum number of changes.

49

Performance measure

50

The structure of Problem The only that we can possibly hope to deal with the real world

problem is decompose it into many small problem. Decompose into independent subproblem.

51

Decompose

Each component CSPi.

If Si be the solution of SCPi, Ui Si is a solution of Ui CSPi.

52

Tree-structures CSP

53

54

55

Cycle Cutset

56

Worst caseC= (n-2)Finding the smallest cycle cutset is NP-hard.Approximation algorithm are known.Cutset conditioningApplication: reasoning about probabilities.

57

Tree-Decomposition

58

Tree-decompose

59

Tree-decomposedSolve each subproblem independently. If any one has no solution, then entire problem has no

solution. If all subproblems have solutions, then construct a global

solution as follows:– View each subproblem as a mega-variable whose domain is the

set of all solutions for the subproblem.• WA=red, SA=blue, NT=green

– Solve the constraints connecting the subproblems using tree-algorithm. (for share variable)

• WA=red, SA=blue, NT=green consist with SA=blue. NT=green, Q=red.

60

Tree-widthThe aim of tree-decomposed is to make the subproblem

as small as possible.Tree-width of a tree decomposition of a graph is one less

than the size of the largest subproblems.The tree-width of the graph is defined to be the minimum

tree width among all its tree decompositions. If a graph has tree-width with w, the problem can be

solved in O(ndw+1) time. (polynominal time solvable)Finding the decomposition with minimal tree width is

NP-hard. (heuristic method).

61

Summary CSPs are a special kind of problem:

– states defined by values of a fixed set of variables– goal test defined by constraints on variable values

Backtracking = depth-first search with one variable assigned per node

Variable ordering and value selection heuristics help significantly Forward checking prevents assignments that guarantee later failure Constraint propagation (e.g., arc consistency) does additional work

to constrain values and detect inconsistencies Iterative min-conflicts is usually effective in practice

–

–

62

Another Forward Checking Example: 4-Queens Problem

1

3

2

4

32 41

X1{1,2,3,4}

X3{1,2,3,4}

X4{1,2,3,4}

X2{1,2,3,4}

[4-Queens slides are adapted from Stanford’s CS 121, with major corrections]

63

Another Forward Checking Example: 4-Queens Problem

1

3

2

4

32 41

X1{1,2,3,4}

X3{1,2,3,4}

X4{1,2,3,4}

X2{1,2,3,4}

64

Another Forward Checking Example: 4-Queens Problem

1

3

2

4

32 41

X1{1,2,3,4}

X3{ ,2, ,4}

X4{ ,2,3, }

X2{ , ,3,4}

65

Another Forward Checking Example: 4-Queens Problem

1

3

2

4

32 41

X1{1,2,3,4}

X3{ ,2, ,4}

X4{ ,2,3, }

X2{ , ,3,4}

66

Another Forward Checking Example: 4-Queens Problem

1

3

2

4

32 41

X1{1,2,3,4}

X3{ , , , }

X4{ ,2,3, }

X2{ , ,3,4}

67

Another Forward Checking Example: 4-Queens Problem

1

3

2

4

32 41

X1{ ,2,3,4}

X3{1,2,3,4}

X4{1,2,3,4}

X2{1,2,3,4}

68

Another Forward Checking Example: 4-Queens Problem

1

3

2

4

32 41

X1{ ,2,3,4}

X3{1, ,3, }

X4{1, ,3,4}

X2{ , , ,4}

69

Another Forward Checking Example: 4-Queens Problem

1

3

2

4

32 41

X1{ ,2,3,4}

X3{1, ,3, }

X4{1, ,3,4}

X2{ , , ,4}

70

Another Forward Checking Example: 4-Queens Problem

1

3

2

4

32 41

X1{ ,2,3,4}

X3{1, , , }

X4{1, ,3, }

X2{ , , ,4}

71

Another Forward Checking Example: 4-Queens Problem

1

3

2

4

32 41

X1{ ,2,3,4}

X3{1, , , }

X4{1, ,3, }

X2{ , , ,4}

72

Another Forward Checking Example: 4-Queens Problem

1

3

2

4

32 41

X1{ ,2,3,4}

X3{1, , , }

X4{ , ,3, }

X2{ , , ,4}

73

Another Forward Checking Example: 4-Queens Problem

1

3

2

4

32 41

X1{ ,2,3,4}

X3{1, , , }

X4{ , ,3, }

X2{ , , ,4}