View
215
Download
1
Tags:
Embed Size (px)
Citation preview
1
On the Integration of Constraint Propagation
and Local Search
Pedro Barahona
Department of Computer ScienceFaculty of Science and Technology
New University of Lisbon
2
Summary
• Constraint Solvers for CSP and CSOP
– Pure Methods (Propagation, Local Search, ...)
– Hybrid Methods
• Integration of Propagation & Local Search
– Digital Circuits Testing
– Handling Continuous Domains
– Determination of Protein Structure
• Conclusions
3
CSP and CSOP
• CSP: Contraint Satisfaction Problems
Given variables V1 to Vn, find values in their respective domains D1 ... Dn that satisfy Constraints C1 to Cp.
Constraints Ci D1 x D2 x ... x Dn
• CSOP: Contraint Sat & Optimisation Problems
Given variables V1 to Vn, find values in their respective domains D1 ... Dn that satisfy Constraints C1 to Cp and, additionally, optimise some objective function F
Objective Function F: D1 x D2 x ... X Dn R
4
Constraint Solving Methods
• Algebraic Approach
– Available in some specific domains
– Examples: Boolean, Rational Numbers
• Search Methods
– General Purpose
– Finite Domains, Integers, Real Numbers, Intervals, Sets, Strings, ...
• Variety of Search Techniques
– Constructive vs. Repairing
5
Constructive Methods
• General Idea
– Iterative narrowing of the possible values of the variables within their domains, until a completely characterised solution is found
• Different Techniques for Narrowing– Assign Values to Variables (Finite Domains / Satisf.)
– Split Current Domains (Continuous Domains / Optim.)
• Key Issues– Early detection of dead ends
– Efficient Backtracking
6
Repairing Methods
• General Idea
– Starting with a completely characterised solution, somehow unsatisfactory, search for a better solution in its neighbourhood.
• Key issues
– Selection of neighbour
– Escaping from local optima
– Search the whole solution space
– Stopping criteria
7
Comparison of Methods
• Constructive Methods– Complete
– Inherently satisfaction methods• Optimisation considered in heuristics and deadends
(BB)
• More adequate for few and sparse feasible solutions
• Repairing Methods– Incomplete
– Adaptation of solutions from similar problems
– Inherently optimisation methods• Satisfaction encoded in the optimising function
• More adequate for many feasible solutions
8
• Complete Search: Backtracking
– Efficient Backtracking: dependency directed
Constructive Methods
1 3 4 2 5 4 5 3 5 1 2 3
•Analyse dependencies
•Backtrack in non-chronological order
•“greatest of the least”
– Inneficiency: late detection of deadends
9
Constructive Methods
– What kind of propagation?
1 1
1
1
1
1
11
1
1
1
1
1
12
2
2
2
2
2
2
2
2
2
2
3
3
3
3
3
3
4
3
4
4
44
•Propagate “Conflicts”
•Avoid irrelevant backtracking
• Complete Search: Backtracking + Propagation
– Early detection of deadends - propagation
10
Constructive Methods
What kind of propagation
– Trade-of: pruning vs. complexity
Backtracks Prunings RunTime (ms)N Queens 32 64 96 32 64 96 32 64 96
Node Consistency
11 382 503 609 9951 13594 30 200 370
Arc Consistency
8 221 255 588 7729 10345 701 10875 52707
Controlled Arc-Consistency
8 221 255 588 7729 10345 50 531 1252
11
Constructive Methods
• Difficulty with Backtracking
– Typically Chronological
– Costly backtracking of early errors
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1
0 1 0 1
0 1X1
X2
X3
X4
X
12
Repairing Methods
• Even treatment of all variables
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1
0 1 0 1
0 1X1
X2
X3
X4
0010
X1
1010x
X2
1110x
13
Repairing Methods
• Satisfaction via Optimization
• Modeling issue: Definition of Neighbourhood
Objective Function = 3
# Violations (attacks)
Neighbourhood:
Permutations
14
Repairing Methods• Difficulty - Escaping from Local Optima
• Restarts / Stochastic methods
2 3 1 3 0
15
Pure vs. Hybrid Approaches• Compete or Cooperate
• Many Examples of Cooperation
– Operations Research
– Constraint Programming
• Examples from Operations Research
– Integer Programming with Linear Programming
• LP provides Bounds for IP
– Nonlinear and Linear Programming
• Stepwise Linearization
16
Pure vs. Hybrid Approaches• Constraint Programming
– Global Constraints improve Propagation
• Graph Theory / OR Methods with Propagation
– Integer Programming and Propagation
• Cutting Planes (CPLEX and ECLiPse)
• Benders Decomposition
– Continuous Domains
• Interval Arithmetic (Newton)
• Quadratic Optimization
– Propagation and Local Search
17
Experiments with CP and LS
• Optimisation of Test Patterns (Digital Circuits)
– Francisco Azevedo
• Global-Hull Consistency (Continuous Domains)
– Jorge Cruz
• Determination of Protein Structure (NMR Data)
– Ludwig Krippahl
Interaction of Constraint Programming
and Local Searchfor Optimisation Problems
Francisco Azevedo & Pedro Barahona
19
Introduction
• Scope: Optimisation problems in the field of Automatic Test Pattern Generation (ATPG)
• Goal: For a given stuck-at fault, find a test (an input vector) with the maximum number of unspecified bits.
• Approach: Integration of branch-and-bound CLP with Local Search (in a cycle CP <-> LS) with multi-valued logics
• Key feature - Different encodings in CP and LS.
20
CP Modeling: 5-valued Logic
• 5 valued logic to encode dependency/ignorance– Values D/ D encode dependency on a
fault– Value X encodes “don’t know” situations
G s-a-011
D (1/0)
G s-a-010
0
Input 0 1 D D x S-buffer/0 output 0 D D 0 x S-buffer/1 output D 1 1 D x
• Encoding “stuck-at” faults (S-buffers)
G s-a-1
010
D (0/1)
21
CP Modeling: 5-valued Logic
_• Goal: An output D/D that maximises # X inputs
• Encoding normal behaviour in 5-valued Logic
D D
NOT0 11 0D D
D Dx x
AND 0 1 D D x
0 0 0 0 0 01 0 1 D D xD 0 D D 0 xD 0 D 0 D xx 0 x x x x
X0
0
DX
X
CombinationalCircuit
DPIs POs
D
maximise #{i PI: i = x}
x
22
Incompleteness of CP Modeling
• 5 Valued Logic does not detect the fault
• Test ab=x0 detects b s-a-1
x
0/1
_D
x x
x
x
x
0
0/1 0/1
0/0 1/1 0/1
0/0
0/1
x
1
0/1 0/1
1/1 0/0 0/0
0/1
0/1
x
23
Improving Completeness (1)
• Naming unspecified values, including inversion parity, improves detection of faults
NOT AND 0 1 id-0 id-1 XOR 0 1 id-0 id-10 1 0 0 0 0 0 0 0 1 id-0 id-11 0 1 0 1 id-0 id-1 1 1 0 id-1 id-0
id-0 id-1 id-0 0 id-0 id-0 0 id-0 id-0 id-1 0 1id-1 id-0 id-1 0 id-1 0 id-1 id-1 id-1 id-0 1 0
a-0
0 0 0
a-1 00
a-0
0 /1 1 a-0
a-1 a-11
(a) (b)
x
24
Improving Completeness (2)
• Modelling still incomplete• final output should have value 1
• Multiple Sources of Unspecified Values
Id G-0Id A -p A
Id B -p B
Id G
A B Z=A.B0 Arg 01 Arg Arg
Arg Arg Argid-0 id-1 0idA-pA idB-pB idZ-0
a-0
0 /1 1b-0
a-1 a-1w-0
x-0
y-0a
a-0
0 00
a-1 00
x-0
y-0a
b b-0 b-0b
w
x
25
Integration CP and LS
• Constraint Propagation (weaker then full arc-consistency) finds solutions efficiently
• (Still) Incomplete CP Model
• Maximisation in usual Branch and Bound would still be incomplete
• CP solutions as seeds for LS optimisation
• LS uses a complete logic
• Interesting neighbours heuristically detected with such logic
26
Annotations in Logic for LS
• Complete LS Model
• Heuristic for “good” neighbours
• Dependency on (single) unspecified values
{}:000
(a)
xy
z{x/ x-0}:00
1
(b)
z {x/ x-0, y/ y-0}:111
(c)
z
000
(a)
xy
z0
01
(b)
z1
11
(c)
z
27
Annotations in Logic for LS
• Simulation
/1
0a= x a-0
1
b=0
a-0
00
0
0
a-0
a-1a-1
1 0
a-1
/1
{b/b-0}:0a=0 {a/a-0}:0
{}:1
b=0
{a/a-0}:0
{b/b-0}:0{b/b-0}:0
{b/b-0}:0
{b/b-0}:0
{a/a-0}:0
{a/a-1}:1{a/a-1}:1
{}:1{}:0
{a/a-1}:1
0/1 output does not depend on inputa , which may flip to x
• Example
• Local Search (over original test ab=00)
28
Experimental Results (1)
• Evaluation of Maxx on ISCAS circuitsc17 c432 c499 c880 c1355 c1908 c2670 c3540 c5315 c6288 c7552
PI 5 36 41 60 41 33 233 50 178 32 207G 6 160 202 383 546 880 1193 1669 2307 2416 3512F 22 524 758 942 1574 1878 2746 3425 5350 7744 7550
Atalanta MTP MAXX%X %X Gain T/f %X Gain T/f Diff
c432 56.2 60.8 10.5 3.21 71.4 34.7 7.16 24.2c499 17.1 18.7 1.9 4.35 25.6 10.3 7.95 8.3c880 82.2 83.8 9.0 2.54 85.2 16.9 6.08 7.9c1355 13.3 13.7 0.5 9.12 20.0 7.7 7.93 7.3c1908 44.7 48.4 6.7 9.61 52.8 14.6 7.51 8.0c2670 92.0 92.4 5.0 10.99 94.1 26.2 5.90 21.2c3540 74.6 77.3 10.6 16.81 79.2 18.1 7.07 7.5c5315 92.6 92.9 4.1 9.34 93.6 13.5 7.53 9.5c6288 22.2 22.2 0.0 36.65 28.4 8.0 12.51 8.0c7552 86.9 86.9 0.0 17.46 90.9 30.5 10.17 30.5
• MTP100 and Maxx improvements over Atalanta
29
Experimental Results (2)
• Breakdown of Maxx PerformanceMTP LS1 CP1 LS2 CP2 LS3 CP3 LS4 CP4 LS5
c6288 52,874 10,749 2,126 303c7552 1,334,794 56,052 5,630 628 22 29 26 8 2 2
• MTP1000 and Maxx improvements over MTP100
MTP100 MTP1000 MAXX %X %X Gain T/f %X Gain T/f Diff
c432 60.8 64.1 8.4 23.83 71.3 26.7 7.03 18.2 c499 18.7 19.5 1.0 29.36 26.3 9.3 7.08 8.3 c880 83.8 85.6 11.1 19.80 86.1 14.3 6.00 3.1 c1355 13.7 15.2 1.7 55.74 16.5 3.2 7.44 1.5 c1908 48.4 51.0 5.0 63.83 57.5 17.6 7.50 12.6 c2670 92.4 93.0 7.9 72.47 94.2 23.3 5.57 15.4 c3540 77.3 n/a n/a n/a 80.6 14.5 6.95 n/a c5315 92.9 n/a n/a n/a 93.8 13.1 7.49 n/a c6288 21.4 n/a n/a n/a 26.8 6.8 12.34 n/a c7552 86.9 n/a n/a n/a 91.0 31.1 9.61 n/a
30
Maintaining Global Hull Consistency with
Local Search for Continuous CSPs
Jorge Cruz & Pedro Barahona
31
Introduction
• Scope: Continuous Constraint Satisfaction Problems for Decision Support
• Goal: Find an enclosure of the solution space where a decisor may look for “good” solutions.
• Approach: Maintaining Global Hull Consistency as the basic Constraint Propagation Method
• Key Feature: Local Search in Enforcement Algorithms
32
Under-Constrained CCSPs
• Representation of Continuous Domains
F-interval R
F
[r1..r2]
[f1 .. f2]
r
[r..r]
F-box
Canonical solution
33
Solving CCSPs
• Branch and Prune algorithms
• Prune: Constraint Propagation
• Branch: Isolate Solutions
34
• Based on Interval Arithmetic
• Hull-consistency
• Decomposes constraints into sets of primitive constraints
x3+xy+y5=3 { Z1= X3 ; Z2= XY ; Z3= Y5 ; Z1+Z2+Z3 =3 }
• Enforces consistency on the bounds of each domain
• Box-consistency
• Does not decompose constraints
• Enforces consistency on domain bounds combining the Interval Newton method with spliting
F(x,y) = x3+xy+y5-3=0 Xn = xm - F(xm, Y) Xn-
1
F’ (Xn-
1,Y)
Basic CP Methods for CCSPs
35
Higher Order Consistency
• Basic Methods are often Insufficient
• Higher Order Consistency Criteria
• 3B- consistency, nB consistency (Hull)• if n-2 bounds are fixed then the problem is Hull-
consistent
• Bound-consistency, nBound consistency (Box)
• if n-2 bounds are fixed then the problem is Box-consistent
• However, these higher order methods may still be insufficient
• Our Approach: Global Hull Consistency (GH)
36
Global Hull Consistency
Each bound of the domain of each variable is justified by a canonical solution for the global set of constraints.
• Decision Support Context (“many” solutions)
• Requirement: search for extreme solutions
37
Hull Consistency Enforcing Algorithms
• Several Algorithms were tried
• OS1, OS2: Independent search for each bound of each variable
• OS3: Alternate search for the bounds of all variables
• TSA: Alternate search keeping some tree search structure of the domains
• Common goal: search for canonical solutions
• Alternatives: Pure splitting or local search
38
Local Search in GH Enforcing
• Find a canonical solution within a box
canonical solution ?
x
y
upper bound of x
minimization
initial point
Newton’s vector
new point
Newton’s vector
minimization
new point
Stops when:convergesfind a solution
39
Hull Consistency Enforcing Algorithms
Newton’s Vector
f(x,y)=0
g(x,y)=0
h(x,y)=0
Constraints:
X
F
Current Point : X0 F(X0)0
goal: X F(X0+ X)=0
Newton’s vector
By the Taylor expansion (ignoring higher order terms):
F(X0+ X)= F(X0)+J·X
Jacobian0
X= - J-1 F(X0)
Singular Value Decomposition
40
Experimental Results (1)
• The need for Global Hull (toy examples):
x2 + y2 1
x2 + y2 2
Box-consistency
3B-consistency
Global Hull-consistencyØ
x2 + y2 + z2 2
x2 + y2 + z2 3Global Hull-consistency
4B-consistencyØ
Box-consistencyr=1.415
3B-consistencyr=1
41
Experimental Results (2)
• Determination of Protein Structure (6 atoms) 2B 3B GH
x4 -1.001 .. 1.415 -0.056 .. 0.049 -0.004 .. 0.004
y4 0.585 .. 3.001 1.942 .. 2.047 1.996 .. 2.004
z4 -1.415 .. 1.416 -1.415 .. 1.415 -1.415 .. 1.415
x5 -0.415 .. 2.001 0.998 .. 1.002 0.999 .. 1.001
y5 -0.001 .. 2.001 0.999 .. 1.001 0.999 .. 1.001
z5 -1.415 .. 1.416 -1.415 .. 1.415 -1.415 .. 1.415
x6 -2.001 .. 2.001 -1.110 .. 1.053 -1.008 .. -0.992
y6 -0.001 .. 2.001 -0.894 .. 1.169 0.999 .. 1.001
z6 -2.001 .. 2.001 -1.483 .. 1.483 -1.420 .. 1.402
t (ms) 10 7 380 62 540
1 in << 7 secs
42
Experimental Results (2)
• How useful is Local Search ?
Time (s) Max Storage (F-boxes)
k LS =10-1 =10-2 =10-3 =10-1 =10-2 =10-3
2 n 2.99 11.37 600+ 30 47 60
OS2 y 14.97 38.46 600+ 14 16 24
3 n 41.01 150.21 359.69 22 37 53
y 19.14 75.98 234.65 3 8 25
2 n 10.16 60.96 600+ 332 6786 76506
TSA y 44.79 47.59 600+ 221 621 16764
3 n 42.52 184.38 600+ 52 207 843
y 38.63 97.88 275.85 99 185 392
43
PSICO
Combining Constraint Programming and Optimisation to Solve Macromolecular
Structures
Ludwig Krippahl & Pedro Barahona
44
Introduction
• Scope: Fast Interpretation of data collected in Nuclear Magnetic Resonance (NMR) experiments.
• Goal: Determine Protein Structure from inter-atomic distances provided by NMR data.
• Approach: Combining CP with local search optimisation (in continuous domains)
• Key Feature: CP and Local Search acting on different models
45
Motivation (1)
• Protein (amino acid) sequence, coded in the genes, determines its structure (shape) which, in turn, determines much of its functionality
Catalysis Signaling
RegulationMotion
Function
Genes
Structure
46
Motivation (2)
• NMR avoids crystalisation and other costly procedures, but requires fast data post-processing (to correct interpretation errors)
What we want
What NMR provides
C
Glycin Leucin
Tyrosine
N
OH
W Y L L H W S T
Basic Knowledge
47
Overall Method
• Phase 1: Establish Distance Constraints
• From Basic Knowledge
• From NMR Data
In: | Xi - Xj | Uij Out: | Xi - Xj | Lij
• Phase 2: Approximate Solution
• Constraint Propagation
• Phase 3: Repair Solution
• Local Search
48
Phase 1: Domain Modeling
Domains modeled as 3-D boxes
Good
NoGoodDistance
Out
In
x1 , y1
x
y x2 , y2
a b c
Modeling allows “arc-consistency”
stronger than “bounds-
consistency”
49
Phase 2: Constraint Propagation
Good d
Max
Min
Intersection
In Constraint Out Constraint
d
d
Exclusion Zone
Good
GoodNoGood
Exclusion Zone
NoGood
50
• Change representation to torsion angles (i.e the chemical bond allow some rotation)
Phase 3: Local Search (Optimisation)
• Fit approximate solution from CP to the torsion angle model
• Optimise overall constraint violation (Conjugate Gradient Method)
51
• Fit CP solution to the torsion angle model
Phase 3: Local Search (Optimisation)
n
jjj ppF
1
*)(min
p* are CP positions and p() those determined by torsion angles
1
1 1,
1
)(NA
j
NA
jiij
NC
jj ACGmin
Cj() are penalties for violation of distance constraints, and Ajj() penalties associated to implicit atomic “collisions”.
• Optimise overall constraint violation (via Conjugate Gradient Method)
52
•Experimental Data (Desulforedoxin dimer, …)
Experimental Results
DYANA
Time: 10 hours
Solutions: 1Å
PSICO•Time: 9 min
•1 min (CP) •8 min (15 LS solutions)
•Solutions RMSD: 2.3 Å•Constraints Violated:
15%
• 520 atoms 8000 constraints where
•> 800 provided from NMR
•> 7000 from amino acid knowledge
53
Conclusions• Several Methods to solve CSP and CSOP
•Constraint Propagation
•Local Search
• Opportunities for Cooperation but...
•Careful Modeling
•Seek advantages from complementary nature
• Adequacy to satisfaction /optimisation
• Ease of modeling