A. Darwiche Knowledge Compilation Jinbo Huang NICTA and ANU Slides made by Adnan Darwiche and Jinbo Huang

A. Darwiche

Knowledge Compilation

Jinbo HuangNICTA and ANU

Slides made by Adnan Darwiche and Jinbo Huang

A. Darwiche

Propositional Logic

X YA CB

A okX BA okX B

B okY CB okY C

KB =

Is A, C a normal device behavior?

A. Darwiche

Propositional Reasoning

A okX BA okX B

B okY CB okY C

Is A, C a normal device behavior?

A okX BA okX B

B okY CB okY C

A, C, okX, okY

Satisfiability Algorithm

A. Darwiche

SAT Solvers: Significant growth in last decade; many solvers publicly available (source code); millions of clauses not uncommon.

Applications: Verification, planning, diagnosis, CAD, non-propositional reasoning (e.g., SMT), …

Satisfiability (SAT)

A. Darwiche

A okX BA okX B

B okY CB okY C

CompiledStructureCompiler

Evaluator(Polytime)

Queries


A. Darwiche

?Compiler

Evaluator(Polytime)

Queries


A okX BA okX B

B okY CB okY C

A. Darwiche

.....Prime Implicates

OBDD…

Compiler

Evaluator(Polytime)

Queries


A okX BA okX B

B okY CB okY C

A. Darwiche

Knowledge Compilation Map

What’s the space of possible target compilation languages? Can it be synthesized in a

semantically systematic way?

How do the languages compare? Succinctness (relative size) Operations they support in polytime

A. Darwiche

Diagnosis Is this a normal behavior? What are the possible faults?

Planning Can this goal be achieved? Generate plan with highest reward Generate plan with highest success probability

Probabilistic reasoning What is the probability of X given Y

Formal verification / CAD: Is it possible that the design will exhibit behavior X? Are two designs equivalent?

Applications

A. Darwiche

Knowledge Compilation Map

For a given application: identify needed operations

Choose most succinct language that supports desired operations

Compile knowledge base into chosen language

A. Darwiche

Agenda

Part I: Languages Part II: Operations Part III: Compilers Part IV: Applications

A. Darwiche

Part I: Languages

A. Darwiche

Succinctness

Polytime OperationsConsistency (CO)Validity (VA)Clausal entailment (CE)Sentential entailment (SE)Implicant testing (IP)Equivalence testing (EQ)Model Counting (CT)Model enumeration (ME)

Projection (exist. quantification)ConditioningConjoin, Disjoin, Negate

DecomposabilityDeterminismSmoothness

FlatnessDecision

Ordering

Negation Normal Form

A B B A C D D C

and and and and and and and and

or or or or

and and

or

A Knowledge Compilation MAP

A. Darwiche

Propositional Logic

Literal Clause Term CNF: Conjunctive Normal Form

DNF: Disjunctive Normal Form

XX ,

)( ZYX

)( ZYX

)(...)( WYZYX

)(...)( WZXZYX

A. Darwiche

Propositional Logic

falseWtrueZfalseYtrueX :,:,:,:

)(...)( WYZYX

trueWtrueZfalseYtrueX :,:,:,:

Truth assignment (TA)

TA satisfies sentence (model)

Following TA is not a model

A. Darwiche

Succinctness




FlatnessDecision

Ordering


A B B A C D D C


or or or or

and and

or


A. Darwiche

Queries Consistency (CO) Validity (VA) Sentential entailment (SE) Clausal entailment (CE): KB implies

clause Implicant testing (IP): term implies KB Equivalence testing (EQ) Model Counting (CT) Model enumeration (ME)

A. Darwiche

Transformations

Projection (existential quantification)

Conditioning Conjoin Disjoin Negate

A. Darwiche

Representation vs Compilation Languags

Representation Language (intuitive):CNFDNF

Target Compilation Language (tractable): Binary Decision Diagrams (BDDs)DNNF

A. Darwiche

A B B A C D D C


or or or or

and and

or

rooted DAG (Circuit)


A. Darwiche

A B B A C D D C


or or or or

and and

orDecomposability

DeterminismSmoothness

FlatnessDecision

Ordering


A. Darwiche

A B B A C D D C


or or or or

and and

or

A,B C,D

Decomposability

A. Darwiche

NNF

DNNF

CO, CE, ME

NNF Subsets

A. Darwiche

A B B A C D D C


or or or or

and and

or

Determinism

A. Darwiche

A B B A C D D C


or or or or

and and

or

Smoothness

A. Darwiche

NNF

d-NNF s-NNF

sd-DNNF

DNNF

CO, CE, ME

d-DNNF

VA,IP,CT

NNF Subsets

EQ?

A. Darwiche

XX YY ZZ

and

or

and andand

Nested vs Flat languages

(XY Z)(ZXY) (YZX) (XY Z)

Flatness

A. Darwiche

XX YY ZZ

and

or

and andand

Simple conjunction implies decomposability

Simple Conjunction

A. Darwiche

XX Y Z

or

and

oror

Simple Disjunction

A. Darwiche

NNF

d-NNF s-NNF f-NNF

sd-DNNF

DNNF

CO, CE, ME

d-DNNF

VA,IP,CT

EQ?

CNFDNF

NNF Subsets

A. Darwiche

Prime Implicates (PI)

A CNF such that: No clause subsumes another If a clause is implied by the CNF, it

must be implied by a clause in the CNF

CNF:

PI:

)()()( DCCBBA

)()()(

)()()(

DBDACA

DCCBBA

)(

)(),(

XX

Resolution

A. Darwiche

Prime Implicants (IP)

A DNF such that: No term subsumes another If a term implies the DNF, it must

imply a term in the DNF DNF:

IP:

)()( CBBA

)()()( CACBBA

)(

)(),(

XX

Consensus

A. Darwiche

NNF

d-NNF s-NNF f-NNF

sd-DNNF

DNNF

CO, CE, ME

d-DNNF

VA,IP,CT

EQ?

CNFDNF

IP PI

CO,CE,MEVA,IP,SE,EQVA,IP, SE,EQ

NNF Subsets

A. Darwiche

X X

and

or

and

Decision

Are decision nodes

A. Darwiche

or

and and

X1 X1or or

and and andand

X2 X2 X2 X2

and and andand

X3 X3 X3 X3

or or

true false

Decision

A. Darwiche

X X

and

or

and

X

Decision

A. Darwiche

X1

X2 X2

X3X3

1 0

or

and and

X1 X1or or

and and andand

X2 X2 X2 X2

and and andand

X3 X3 X3 X3

or or

true false

Binary Decision Diagrams(BDDs)

Decision implies determinism

A. Darwiche

NNF

d-NNF s-NNF f-NNF

sd-DNNF

DNNF

CO, CE, ME

d-DNNF

VA,IP,CT

EQ?

CNFDNF

IP PI


BDD

NNF Subsets

A. Darwiche

X1

X2 X2

X3X3

1 0

or

and and

X1 X1or or

and and andand

X2 X2 X2 X2

and and andand

X3 X3 X3 X3

or or

true false


Decision + decomposability = FBDDTest once property

A. Darwiche

NNF

d-NNF s-NNF f-NNF

sd-DNNF

DNNF

CO, CE, ME

d-DNNF

VA,IP,CT

EQ?

CNFDNF

IP PI


BDD

FBDD EQ?

MODSSE,EQ

NNF Subsets

A. Darwiche

X1

X2 X2

X3X3

1 0

or

and and

X1 X1or or

and and andand

X2 X2 X2 X2

and and andand

X3 X3 X3 X3

or or

true false


Decision + decomposability + ordering = OBDD

A. Darwiche

NNF

d-NNF s-NNF f-NNF

sd-DNNF

DNNF

CO, CE, ME

d-DNNF

VA,IP,CT

EQ?

CNFDNF

IP PI


BDD

FBDD EQ?

OBDD

SE,EQ

MODSSE,EQ

NNF Subsets

A. Darwiche

OBDD Example: Odd Parity

X1

X2

X3

X4

1

X2

X3

X4

0Symmetric Functions

A. Darwiche

Language Succinctness

Size nSize p(n)

L1 <= L2L1 at least as succinct as L2

L1 < L2L1 is more succinct than L2

A. Darwiche

A B B A C D D C


or or or or

and and

or

Odd Parity Function

A

B

C

D

1

B

C

D

0

NNF

DNNF

CNF

d-DNNFDNF

PIFBDD

OBDDIP

MODS

sd-DNNF

PI <= DNNFDNNF <=? PI

DNF <= OBDDOBDD <= DNF

=*

A. Darwiche

OBDD

FBDD

d-DNNF

DNNF

Space Efficiency (succinctness)

Tractable OperationsNNF

decomposability

determinism

decision

ordering

Diagnosis,Non-mon

Probabilisticreasoning

Tractability & Succinctness

A. Darwiche

Separating Functions

OBDD/FBDD: Hidden weighted bit function

hwb(x1,..,xn)

DNNF/DNF: odd parity function parity(x1,..,xn)

DNNF/OBDD: Distinct integers function

distinct(x1,..,xn)

A. Darwiche

Agenda


A. Darwiche

Part II: Operations

A. Darwiche

KB CompiledStructureCompiler

Evaluator(Polytime)

Queries


A. Darwiche

Queries Consistency (CO) Validity (VA) Sentential entailment (SE) Clausal entailment (CE): KB implies

clause Implicant testing (IP): term implies KB Equivalence testing (EQ) Model Counting (CT) Model enumeration (ME)

A. Darwiche

Transformations

Projection (existential quantification)

Conditioning Conjoin Disjoin Negate

A. Darwiche

Decomposability

A. Darwiche

A B B A C D D C


or or or or

and and

or

A,B C,D

Decomposability

A. Darwiche

X YA CB

A okX BA okX B B okY CB okY C

Example Knowledge Base

A. Darwiche

Decomposable or

and and

or or or or

A C~A ~C

~okX ~okY

B ~B

A. Darwiche

Decomposable or

and and

or or or or

A C~A ~C

~okX ~okY

B ~B

BA, okXC, okY

A. Darwiche

Decomposable or

and and

or or or or

A C~A ~C

~okX ~okY

B ~B

A. Darwiche

Satisfiability

or

and and

or or or or

A C~A ~C

~okX ~okY

B ~By y

y y

yy

y y

y y

y y y y

y

•SAT(A or B) iff SAT(A) or SAT(B)•SAT(A and B) iff SAT(A) and SAT(B)•SAT(X) is true•SAT(~X) is true•SAT(True) is true•SAT(False) is false

A. Darwiche

Satisfiability

or

and and

or or or or

A C~A ~C

~okX ~okY

B ~B

A. Darwiche

Satisfiability

or

and and

or or or or

false Cfalse ~C

false ~okY

B ~Bn y

y y

yn

n y

n n

n y n y

n

A. Darwiche

Partial Decomposability

or

and and

or or or or

A C~A ~C

~okZ okZ

B ~B

Decomposable except on okZ

A. Darwiche

KB entails L1 v L2 v … v Ln ?

KB L1 L2 … Ln SAT ?

Clausal Entailment

A. Darwiche

or

and and

or or or or

A C~A ~C

~okX ~okY

B ~B

and

A

Literal Conjoin

A. Darwiche

or

and and

or or or or

A C~A ~C

~okX ~okY

B ~B

A

Literal Conjoin

A. Darwiche

or

and and

or or or or

true C ~C

~okX ~okY

B ~B

A

false

Literal Conjoin

Conditioning

A. Darwiche

or

and and

or or or or

true C ~C

~okX ~okY

B ~B

and

A

false

Literal Conjoin

or

and and

or or or or

A C~A ~C

~okX ~okY

B ~B

A ~C okX okYX YA CB

or

and and

or or or or

false

B ~B

A

and

~C okX okYX YA CB

false false

falsetrue true

or

and and

or or or or

false

B ~B

A

and

~C okX okYX YA CB

false false

falsetrue true

y n

y y

nn

n y

n n

n

n y y n

n

A. Darwiche

Partial Decomposability

or

and and

or or or or

A C~A ~C

~okZ okZ

B ~B

Decomposable except on okZ

Clausal entailment test works as long as clause mentioned allvariables on which we don’t have decomposability!

or

and and

or or or or

A C~A ~C

~okZ okZ

B ~B

or

and and

or or or or

A C~A ~C

false true

B ~B

and

okZ

or

and and

or or or or

A C~A ~C

true false

B ~B

and

~okZ

A. Darwiche

Projection: Existential Quantification

DCCBBA ,,Knowledge Base

DACB )(

Existentially quantifying B,CForgetting B,C

Projecting on A,D

A. Darwiche


)|()|( XXX Formal Definition

•If Knowledge base is a CNF:•Close under resolution•Remove all clauses that mention X

A. Darwiche


or

and and

or or or or

A C~A ~C

~okX ~okY

B ~B

X YA CB

A. Darwiche


or

and and

or or or or

A~A true

~okX

B ~B

true

true

X YA CB A okX BA okX B

)(

))|()|((

))|(())|((

))|()|(())|()|((

)|)(()|)((

)(

X

XX

XX

XXXX

XX

X

I1I2

I3I4

I5I6

O

+ {O}project( , I1..I6)

A. Darwiche

Minimum Cardinality

13..

1

okY A B C

true truetrue truefalsetrue

.

.

false

.

.

true

.

.

false

.

.

false

.

.

okX

A okX BA okX B

B okY CB okY C

A. Darwiche

Minimum Cardinality

or

and and

or or or or

A C~A ~C

~okX ~okY

B ~B0 0

0 1

11

1 1

2 1

1 1 0 0

1

A. Darwiche

Minimizing: Requires Smoothness

A B B A C D D C


or or or or

and and

or

1 0 1 0 0 1 0 1

1 2 0 1 1 0 2 1

1 0 0 1

1 11

A. Darwiche

Minimizing

A B B A C D D C


or or or or

and and

or

1 0 1 0 0 1 0 1

1 2 0 1 1 0 2 1

1 0 0 1

1 11

A. Darwiche

Minimizing

A B B A C D D C


or or or or

and and

or

1 0 1 0 0 1 0 1

1 2 0 1 1 0 2 1

1 0 0 1

1 11

A. Darwiche

Minimizing

A B B A C D D C

and and and and and and

or or or or

and and

or

1 0 1 0 0 1 0 1

1 0 1 1 0 1

1 0 0 1

1 11

A. Darwiche

Minimizing

A B B A C D D C


or or or or

and and

or A,B,C,D

A. Darwiche

Minimizing

A B B A C D D C


or or or or

and and

or A, B,C,D

A. Darwiche

Minimizing

A B B A C D D C


or or or or

and and

or A, B,C, D

A. Darwiche

Minimizing

A B B A C D D C


or or or or

and and

or A, B, C,D

A. Darwiche

Decomposability

Query DNNF

CO: Consistency Yes

VA: Validity

CE: Clausal entailment Yes

SE: Sentential entailment

IP: Implicant testing

EQ: Equivalence testing

MC: Model Counting

ME: Model enumeration Yes

A. Darwiche

Decomposability

Transformation DNNF

CD: Conditioning Yes

SFO: Single variable Yes

FO: Multiple variable Yes

&: Conjoin

B&: Bounded Conjoin

|: Disjoin Yes

B|: Bounded Disjoin Yes

~: Negate

A. Darwiche

Determinism

A. Darwiche

A B B A C D D C


or or or or

and and

or

Determinism

A. Darwiche

Satisfiability

AA BB CC okXokX okYokY

T T T T T

T T T T F

… … ... … …

… … … … …

… … … … …

F F F F F

A okX BA okX B

B okY CB okY C

A, C, okX, okY

Satisfiability Algorithm

Is there a satisfying assignment?

A. Darwiche

Model Counting

AA BB CC okXokX okYokY

T T T T T

T T T T F

… … ... … …

… … … … …

… … … … …

F F F F F

A okX BA okX B

B okY CB okY C

A, C, okX, okY

Counting Algorithm

How many satisfying assignments?

A. Darwiche

Counting Models

A B B A C D D C


or or or or

and and

or

A. Darwiche

Counting Graph

* * * * * * * *

+ + + +

* *+

1 1 1 1 1 1 1 1

1 1 1 1 1 1 1 1

2 2 2 2

4 48

A B B A C D D C

A. Darwiche

Counting Models

A B B A C D D C


or or or or

and and

orS={A, B}

A. Darwiche

Counting Models

false false true true C D D C


or or or or

and and

orS={A, B}

A. Darwiche

Counting Graph

* * * * * * * *

+ + + +

* *+

0 0 1 1 1 1 1 1

0 0 0 1 1 1 1 1

1 2 0 2

2 02

false false true true C D D C

S={A, B}

A. Darwiche

Counting Graph

A B B A C D D C

* * * * * * * *

+ + + +

* *+

A. Darwiche

Counting Graph

VA VB V B VA VC V D VD V C

* * * * * * * *

+ + + +

* *+

A. Darwiche

Counting Graph


* * * * * * * *

+ + + +

* *+S={A, B}

0 0 1 1 1 1 1 1

0 0 0 1 1 1 1 1

1 2 0 2

2 02

A. Darwiche

Counting Graph


* * * * * * * *

+ + + +

* *+

A. Darwiche

Counting Graph


* * * * * * * *

+ + + +

* *+S={A, B,C}

0 0 1 1 1 1 1 0

1

1 1 1 1 1 0 1 1

A. Darwiche

Asserting Literals


* * * * * * * *

+ + + +

* *+S={A, B,C}

0 0 1 1 1 1 1 0

1

1 1 1 1 1 0 1 1

{ D} 1 + (-1)1 = 0

0

A. Darwiche

Retracting Literals


* * * * * * * *

+ + + +

* *+S={A, B,C}

0 0 1 1 1 1 1 0

1

1 1 1 1 1 0 1 1

\ { B} 1 + (+1)1 = 2

1

A. Darwiche

Flipping Literals


* * * * * * * *

+ + + +

* *+S={A, B,C}

0 0 1 1 1 1 1 0

1

1 1 1 1 1 0 1 1

\ { B} {B} 1 + (+1)1 + (-1)1 = 1

1 0

A. Darwiche

Testing Equivalence

.00864

.01944..

p

okY A B C

T TT TFT

.

.

F

.

.

T

.

.

F

.

.

F

.

.

okX

A okX BA okX B

B okY CB okY C

Pr(A)=.4Pr(B)=.7Pr(C)=.1Pr(okX)=.9Pr(okY)=.8

A. Darwiche

Testing Equivalence

Pr(A)=.50Pr(B)=.05Pr(C)=.74Pr(okX)=.33Pr(okY)=.80

KB1(A,B,C,okX,okY)

KB2(A,B,C,okX,okY)

: p

: q

If p <> q, then KB1 and KB2 not equivalentIf p = q, then Pr(KB1 and KB2 are equivalent)> 1/2

Run test 100 times error is < 10-30

A B B A C D D C


or or or or

and and

or

.1 .5 .5 .9 .3 .2 .8 .7

.05 .05 .45 .45 .06 .24 .14 .56

.5 .38 .50 .62

.19 .31

.50

A. Darwiche

Equivalence Testing

Map x y z to {1,2,3,4,5,6}

x2, y 3, z5

x y z F(xyz)

0 0 1 1

0 1 0 1

0 1 1 1

1 0 1 1

= -15(1-2) 3 5

= -202 (1-3) 5

= 12(1-2) 3 (1 - 5)

= 10(1-2) (1-3) 5

Add: p(F) = -13

p(F) = p(G) F & G are equivalent with probability ≥ (m – 1)n / mn ( > ½ for m ≥ 2n)

A. Darwiche

Projection under determinism

or

and and

or or or or

A C~A ~C

~okX ~okY

B ~B

A. Darwiche

Projection under determinism

or

and and

or or or or

A C~A ~C

~okX ~okY

true true

A. Darwiche

Determinism

Query d-DNNF

CO: Consistency Yes

VA: Validity Yes



IP: Implicant testing Yes

EQ: Equivalence testing ?

MC: Model Counting Yes


A. Darwiche

Determinism

Transformation d-DNNF


SFO: Single variable

FO: Multiple variable

&: Conjoin

B&: Bounded Conjoin

|: Disjoin

B|: Bounded Disjoin

~: Negate ?

A. Darwiche

Decision

A. Darwiche

X1

X2 X2

X3X3

1 0

or

and and

X1 X1or or

and and andand

X2 X2 X2 X2

and and andand

X3 X3 X3 X3

or or

true false

Decision

A. Darwiche

Decision

Query FBDD

CO: Consistency Yes

VA: Validity Yes




EQ: Equivalence testing ?



A. Darwiche

Decison

Transformation FBDD


SFO: Single variable


&: Conjoin

B&: Bounded Conjoin

|: Disjoin

B|: Bounded Disjoin

~: Negate Yes

A. Darwiche

Ordering

A. Darwiche

Ordering

Query OBDD

CO: Consistency Yes

VA: Validity Yes


SE: Sentential entailment Yes


EQ: Equivalence testing Yes



A. Darwiche

Ordering

Transformation OBDD


SFO: Single variable Yes


&: Conjoin

B&: Bounded Conjoin Yes

|: Disjoin

B|: Bounded Disjoin Yes

~: Negate Yes

A. Darwiche

Agenda


A. Darwiche

Succinctness




FlatnessDecision

Ordering


A B B A C D D C


or or or or

and and

or


A. Darwiche

NNF

d-NNF s-NNF f-NNF

sd-DNNF

DNNF

CO, CE, ME

d-DNNF

VA,IP,CT

EQ?

CNFDNF

IP PI


BDD

FBDD EQ?

OBDD

SE,EQ

MODSSE,EQ

NNF Subsets

A. Darwiche

Part III: Compilers

A. Darwiche

Building Compilers

To-down approaches: Based on exhaustive search

Bottom-up approaches: Based on transformations

A. Darwiche

x y ¬x ¬z

¬w z ¬vv w z

Terminating condition for recursion:

empty set (satisfied), or empty clause (contradiction)

SAT?

x y ¬x ¬z

w z

v = false

SAT?x y

¬x ¬z¬w z

v = true

SAT?

SAT by DPLL Search

•Unit resolution•Conflict-directed backtracking•Clause learning•Branching heuristics•Restarts

A. Darwiche

Recent Trend: Exhaustive DPLL

Count number of models: Model counters, e.g., relsat, cachet

Generate all/subset of models: Image computation in model checking SMT (non-propositional reasoning)

Variations on DPLL Search

A. Darwiche

KnowledgeCompiler

ExhaustiveDPLL

Record Trace

Variations Languages

The Language of Search

A. Darwiche

Trace of DPLL

X

Y

Z

unsat

unsat

sat

X Y

X Y Z

X Y Z

0

0 1

01

A. Darwiche

X

YY

Z Z

unsat

sat

unsat

unsat

satsat

0 1

0 1

01

0

01

1

X Y

X Y Z

X Y Z

Run to Exhaustion

Exhaustive DPLL

A. Darwiche

X

or

X

and

Y Y0

and

andand

or

Y Y

andand

or

1Z Z

X

YY

Z Z

unsat

sat

unsat

unsat

satsat

Trace of DPLL:a Formula

A. Darwiche

X

or

X

and

Y Y0

and

andand

or

Y Y

andand

or

1Z Z

Equivalent to original CNF

Tractable (e.g., count models)

Trace of DPLL: a Formula

A. Darwiche

X

YY

Z Z

unsatunsat

unsat sat

satsat

Level One: Do not record redundant portions of trace

Level Two: Try not to solve equivalent subproblems

Dealing with Redundancy

A. Darwiche

X

YY

Z Z

unsatunsat

unsat

sat


A. Darwiche

X

YY

Z Z

unsat sat

Do not createSimply point to existing node


A. Darwiche

X

YY

Z

0 1

This is an OBDD!

A. Darwiche

X

YY

Z

0 1

X

or

X

and

0

and

andand

or

YY

andand

or

1Z

This is an OBDD!

NNF + decision, decomposability, ordering

A. Darwiche

X Y

X Y Z

X Y Z

Compile

0

X

Y Y

Z

1

A Non-traditional OBDD Compiler

Exhaustive DPLL,Fixed variable order,Unique nodes

New complexity guarantees

A. Darwiche

0

X

Y Z

Z

1

Y

FBDD

Compile

Exhaustive DPLL,Dynamic variable order,Unique nodes

X Y

X Y Z

X Y Z

NNF + decision, decomposability

A. Darwiche

FBDD more succinct than OBDD (dynamic var ordering in SAT)

OBDD: equivalence test (canonical)

FBDD: probabilistic equivalence test

Both allow model counting

FBDD vs OBDD

A. Darwiche

Level One: Unique nodes (done)

Level Two: Avoid redundant compilation (search)


A. Darwiche

Redundant Compilation

x5 x6

x4 x5 x6

x1 x3 x4 x5

x2 x3

x1 x2 x3

X1

X2X2

X3 X3

0 1

1

1

0

1

x5 x6

x4 x5 x6

x5 x6

x4 x5 x6

Formula Caching: complexity guarantees

A. Darwiche

Formula Caching

Majercik and Litmman, 1998 Darwiche, 2002 Bacchus et al, 2003, 2004 Huang & Darwiche, 2004 Sang, Kautz, Beam, 2004, 2005 Thurley, 2006

A. Darwiche

Caching for DPLL

v1

OBDD(Δ|v1=0) OBDD(Δ|v1=1)

OBDD(Δ) =

v2

OBDD(Δ|v1=0,v2=0) OBDD(Δ|v1=1,v2=1)

= =

… Recursive calls may be made on equivalent CNFs

A. Darwiche

Caching for DPLLΔ = {v5 + v6

v4 + v5 + v6

v1 + v3 + v4 + v5

v2 + v3

v1 + v2 + v3}

OBDD(Δ)

… …

… … …

v1v2v3 Δ’

0 0 0 contradiction

0 0 1 contradiction

0 1 0 v5 + v6, v4 + v5 + v6, v4 + v5

0 1 1 v5 + v6, v4 + v5 + v6

1 0 0 contradiction

1 0 1 v5 + v6, v4 + v5 + v6

1 1 0 v5 + v6, v4 + v5 + v6

1 1 1 v5 + v6, v4 + v5 + v6

A. Darwiche

Caching for DPLL

v1 v2 v3 v4 v5 v6

Cutset_3

c1: v1 + v2 + v3

c3: v1 + v3 + v4 + v5

c2: v2 + v3

c4: v4 + v5 + v6

c5: v5 + v6

After instantiation of v1v2v3, Δ’ is either contradictory, or determined by clause c3 alone.

c3 can only be in one of two states: satisfied or shrunk to v4v5

A. Darwiche

Caching for Basic DPLL

In general, cutset_i is set of clauses mentioning a variable vi and one > vi

After instantiation of v1v2…vi, Δ’ is either contradictory, or determined by states of clauses cutset_i

Number of distinct Δ’ is 2 |cutset_i| + 1 Maintain a cache for each i, and use the

value of cutset_i—a bit vector—as key

A. Darwiche

CNF to OBDD

OBDD(Δ, i){if(contradiction) return 0-sinkif(satisfied) return 1-sinkkey = value(cutseti-1)lookup = cachei-1[key]if(lookup ≠ nil) return lookupresult = getnode_node(vi, OBDD(Δ|vi=0, i+1),

OBDD(Δ|vi=1, i+1))

cachei-1[key] = resultreturn result

}

A. Darwiche

Complexity

For each i, 2|cutset_i| bounds number of recursive calls OBDD(Δ, i+1) number of entries in cachei

number of OBDD nodes labeled with vi

Size of largest cutset is known as cutwidth of variable order

Time and space complexities of algorithm and size of OBDD are all linear in number of variables, and exponential only in cutwidth

Variable orders with small cutwidth can help

A. Darwiche

Complexity Theorems

size(OBDD) n2w + 2 n: number of variables w: cutwidth of variable order (size of largest

cutset) Time complexity = O(sn2w)

s: size of CNF Also hold for w = pathwidth, using a

slightly different caching scheme Cutwidth and pathwidth are incomparable

A. Darwiche

Plain DPLL FBDD

Fixed Variable Ordering OBDD

Beyond BDDs…

A. Darwiche

Combine as AND node

d-DNNF

Decomposition (Component Analysis)

Solve disjoint subproblems independently

A. Darwiche

A B C A B CA D E A D E

B CD E

D EB C

and

0

A

B

1

and

D B

C

D

E

Deterministic Decomposable Negation Norm Form

(d-DNNF)

A. Darwiche

or

and

A

and

Aand and

or

and

B

C

or

and

D

E

or or

B D

and and

Deterministic Decomposable Negation Norm Form

(d-DNNF)

A. Darwiche

Dynamically detect disjoint components Most effective, but very expensive

Static structural analysis Constructs a decomposition tree

(dtree) Does not detect all decompositions Low overhead at runtime

Decomposition Methods

A. Darwiche

d-DNNF more succinct than FBDD (effectiveness of decomposition)

Deterministic equivalence test open

Probabilistic equivalence test applies

Other queries same…

FBDD vs d-DNNF

A. Darwiche

Plain DPLL FBDD

Fixed Variable Ordering OBDD

Allowing Decomposition d-DNNF

The Language of Search

Other languages: deterministic DNF

A. Darwiche

Relation to AND/OR Search (CP)

AND/OR graphs are deterministic and decomposable

AND/OR search algorithms are doing enough work to compile networks into (multi-valued equivalent of) d-DNNF

Capable of more than answering a single query (model counting, belief revision, etc)

A. Darwiche

Implications SAT techniques harnessed for

knowledge compilation c2d compiler based on Rsat Solver (SAT

Competition 2007): uses dtrees

Language properties (succinctness/tractability) help characterize power and limitations of search

A. Darwiche

Understanding DPLL

Take any program X that runs exhaustive DPLL-style search

Examine traces, if traces L, then

X can answer all queries tractable for L

X is hopeless on any input having no polynomial-size representation in L

A. Darwiche

Power of DPLL

Traces of several model counters (Relsat, Cachet, e.g.) are in d-DNNF

Are doing enough work to compile formulas into d-DNNF solve tasks beyond model counting

(e.g., minimum cardinality, probabilistic equivalent testing)

A. Darwiche

or

and

X

and

X

Decision nodes(d-DNNF’)

Deterministic nodes(d-DNNF)

or

A B C

and and

andand

or

A B A B

Limitation of DPLL:General Determinism

A. Darwiche

Beyond DPLL:Decomposability (D) Without Determinism (d)

or

or

and

and

X1 X2

X3

DNNF:CO, CE, ME, exist quant

A. Darwiche

Approximate DNNF

dnnf( l

) dnnf( r

)

and

l r

U=

XY XY

A. Darwiche

Approximate Compilation

l

r

U

dnnf( l

) dnnf( r

)

and

XY

|XY |XY

or

and and and

~X~YX~Y~XY

A. Darwiche


l

r

U

dnnf( l

) dnnf( r

)

and

XY

|XY |XY

or

and and and

~X~YX~Y~XY

Sound, but not complete

A. Darwiche


dnnf( l

) dnnf( r

)

and

~X

dnnf( l

) dnnf( r

)

and

or

X

|X |X |~X |~X

l

r

U

Complete, but not sound

A. Darwiche

sat ?

sat

sat

n

ysat y

nsat n

ysat ?

SoundIncomplete

UnsoundComplete

A. Darwiche

Bottom-up Compilation

A. Darwiche

0 1

z

y

DEAD NODES

x

y

x

x

CNF: (x + y) (y + z)

Variable order: x, y, z

The Apply algorithm:

combines two OBDDs using any one of the 16 binary Boolean operators

x + y

y + z

Final OBDD

Bottom-up OBDD Construction

A. Darwiche

Bottom-Up OBDD Construction OBDD packages, such as CUDD

implement Apply (conjoin, disjoin, etc) garbage-collect dead nodes

Apply is efficient: quadratic in operand size Problem: intermediate OBDDs can be much

larger than final one—many dead nodes uf100-08 (32 models): OBDD has 176 nodes

under MINCE order; 30,640,582 intermediate nodes using CUDD; taking 25 mins

A. Darwiche

DPLL Based Construction

Δ|x=0 = y Δ|x=1= y + z

0 1

z

y y

xΔ = (x + y) (y + z)

A. Darwiche

Missing Opportunities

Bottom up construction methods for DNNF and d-DNNF

A. Darwiche

Agenda


A. Darwiche

Part IV: Applications

A. Darwiche

Applications

Model-based diagnosis Planning Probabilistic reasoning

A. Darwiche

CDA

YX

B

System model :

okX (A C)

okY (B C) D

Health variables: okX, okY

Observables: A, B, D

Nonobservable: C

Model-based Diagnosis

A. Darwiche

CDA

YX

B

Abnormal observation :

A B D

Diagnosis: Values of (okX, okY) consistent with :

(0, 0), (0, 1), (1, 0)


System model :

okX (A C)

okY (B C) D

A. Darwiche

CDA

YX

B

Abnormal observation :

A B D

Minimum cardinality Diagnoses:

(0, 0), (0, 1), (1, 0)


System model :

okX (A C)

okY (B C) D

A. Darwiche


Methods for characterizing diagnoses: Conflicts Kernel diagnoses

Methods for manipulating diagnoses: Find minimal diagnoses Find minimum-cardinality diagnoses Characterize lost functionality

A. Darwiche

A

C

E

B

D

F

G

1

2

3

4

5

Characterizing Diagnoses

A. Darwiche

A

C

E

B

D

F

G

1

2

3

4

5


A. Darwiche

A

C

E

B

D

F

G

1

2

3

4

5


low

high

high

low

A. Darwiche

Minimal Conflicts

)(

),(

5431

421

okokokok

okokok

An instantiation of ok1,…ok5 is a diagnosis iff it is consistent with(conjunction of) minimal conflicts

A. Darwiche

Kernel Diagnoses

452

321

),(

),(,

okokok

okokok

An instantiation of ok1,…ok5 is a diagnosis iff it is consistent with(disjunction of) kernel diagnoses

A. Darwiche

Characterizing Diagnosis

Device Model Observables Health Variables Others

nOO ,...,1

mHH ,...,1

kXX ,...,1

)(,...,,,..., 11 nk OOXXHealth Condition

1ok

1 0

4ok

2ok

3ok

5ok

OBDD

1ok

1ok 4ok

5ok

2ok

3ok

4ok

DNNF

A. Darwiche

CNF/PI

Minimal Conflicts

)(

)(

5431

421

okokokok

okokok

A. Darwiche

DNF/IP

Kernel Diagnoses

452

321

)(

)(

okokok

okokok

A. Darwiche

Characterizing Diagnosis

Device Model Observables Health Variables Others

nOO ,...,1

mHH ,...,1

kXX ,...,1

)(,...,,,..., 11 nk OOXXHealth Condition

Compile Model

SystemModel

Compile ?

Succinctness

Efficient computation of diagnoses

SystemModel

CompileOBDD

vs.DNNF

Succinctness

Efficient computation of diagnoses

A. Darwiche

DNNF is more succinct: can lead to smaller compilation

Smaller compilation of system model does not imply faster on-line diagnosis

Although less succinct, OBDD is more powerful (DNNF is a weaker form)

Is this extra power relevant?

Compiling System Models

A. Darwiche

CDA

YX

B

Observation: A B D

TF

F FTT

TT

System model :

okX (A C)

okY (B C) D

Diagnosis Using DNNF

A. Darwiche

CDA

YX

B

Observation: A B Dor

okX

okY


System model :

okX (A C)

okY (B C) D

A. Darwiche

Set observables Linear time

Project out nonobservables Linear time

Projection exponential for OBDD(can be made linear with particular orders)


A. Darwiche

Number of diagnoses can be large Some may be more preferable than

others Minimize number of faulty

components in diagnosis Again, easy for DNNF

Minimum Cardinality Diagnoses

A. Darwiche

or

okX okY

Smoothing: make disjuncts mention same set of variables; O(|’| • |H|)

0

1

0

0 0

1 1

11


A. Darwiche

or

okX okY0

1

0

0 0

1 1

11

Minimizing the DNNF

A. Darwiche

or

and

and

okX

okYokXokY


A. Darwiche

Create “filter” OBDD that asserts a given cardinality k

Conjoin it with OBDD that represents set of diagnoses

Repeat for k = 0, 1, 2... until result is nonempty

Minimum Cardinality Diagnoses Using OBDDs

A. Darwiche

Operation OBDD DNNF

Condition(’, )

O(|’|) O(|’|)

Project(, H) exponential O(||)

Minimize(d) O(mc • |d| • |H|2)

O(|d| • |H|)

Comparison

A. Darwiche

Characterizing Lost Functionality

))...,(( 1 mHHprojectMinmize

A. Darwiche

Hierarchical Diagnosis

A. Darwiche

Scalability

Requires a health variable for each component

c1908 has 880 gates; basic encoding fails to compile

New technique to reduce number of health variables

Preserves soundness and completeness w.r.t. min-cardinality diagnoses

Requires only 160 health variables for c1908

A. Darwiche


A. Darwiche


A. Darwiche


A. Darwiche

Identifying Cones

Gate G dominates gate X if any path from X to output of circuit contains G

All gates dominated by G form a cone

Dominators found by breath-first traversal of circuit

Treat maximal cones as blackboxes

A. Darwiche

Abstraction of Circuit

C = {T, U, V, A, B, C}

A. Darwiche

Top-level Diagnosis

Diagnosis: {A, B, C}

A. Darwiche

Diagnosis of Cone Need to set inputs/output of cone

according to top-level diagnosis

Rest is similar, but not a simple recursive call (to avoid redundancy)

Once cone diagnoses found, global diagnoses obtained by substitution

A. Darwiche

Diagnosis of Cone

Top-level diagnosis:{A, B, C}

3 diagnoses for cone A:{A}, {D}, {E}

3 global diagnoses by substitution:

{A, B, C}{D, B, C}{E, B, C}

A. Darwiche

Soundness Top-level diagnoses have same cardinality.

Substitutions do not alter cardinality (cones do not overlap).

Remains to show that cardinality of these diagnoses, d, is smallest. Proof by contradiction:

Suppose there is diagnosis |P| < d. Replace every gate in P with its highest dominator to obtain P’.

P’ is a valid top-level diagnosis, contradicting soundness of baseline diagnoser

A. Darwiche

Completeness Need to show every min-cardinality diagnosis is found

Given diagnosis P of min cardinality d, replace every gate in P with its highest dominator to obtain P’

P’ has cardinality d, and only mentions gates in top-level abstraction, and hence will be found by top-level diagnosis (by completeness of baseline diagnoser)

P itself will be found by substitution (by completeness of cone diagnosis)

A. Darwiche

Sequential Diagnosis

A. Darwiche

Sequential Diagnosis Set of min-cardinality diagnoses may still be large

Faults not identified with certainty

Take measurements, one at a time, until faults identified

Would like to minimize number of measurement

Use heuristic based on amount of information gain

A. Darwiche

Sequential Diagnosis Assume a probabilistic model of the system

Failure probabilities for components Output behavior of faulty component (e.g., 1 and 0 with

equal probability) These implicitly define a joint probability distribution

over all system variables

Encoding and compilation described later

Pick component with highest posterior probability of failure, measure variable with highest entropy in that component

θJ

or

or or

andand

okA

okJ

and

θAokA

and

θJokJ

and

J and

and

and

P D A

1 00.5 0.5 0.50.1

0.9 0.05

0.95

0.5

0.475

1

1 1 1

1

0.05

0.5

0.5

0.025

0.5

0.5

A. Darwiche

Hierarchical Sequential Diagnosis

To improve scalability, previous idea of abstraction applies

Treat each cone (blackbox) as a single “big” component

Need to compute a single failure probability that “summarizes” the failure behavior of the cone

Create copy of cone with all gates healthy, feed outputs of two cones into XOR gate, compute Pr(output = 1)

A. Darwiche

Planning

A. Darwiche

Slippery Gripper

Probabilistic action effects: Paint: paints block w. p. 1; makes gripper dirty w. p. 1 if it

holds block, w. p. 0.1 if not Pick-up: succeeds w. p. 0.5 if gripper wet, 0.95 if gripper dry Dry: dries wet gripper w. p. 0.8; doesn’t affect dry gripper

Probabilistic initial state: block not painted, not held gripper clean, but dry with probability 0.7

Goal: block painted and held, gripper clean

A. Darwiche

Conformant Probabilistic Planning

Probabilistic initial state and action effects

Conformant: action effects not observable Can’t decide next action by observing environment

Needs straight-line plan with max probability of success, for given horizon (number of steps)

Example 2-step plan: [paint, pickup] (succeeds with probability 0.7335)

A. Darwiche

Brute-force Approach

Compute success probability for all plans of length one

Given success probabilities of plans of length i, compute probability of success for plans of length i + 1

Iterate to planning horizon n

Pick n-step plan with max success probability

Exponential in planning horizon

A. Darwiche

Why is it hard? Consider decision version

Does there exist plan of success probability > p, for given horizon

Given plan, deciding if success probability > p is PP-complete Can be reduced to MAJ-SAT: does majority of assignments satisfy CNF Alternatively: Is CNF satisfiable with probability > ½

Finding plan with given property (that is free to test) is NP-complete, for given horizon

Conformant probabilistic planning for given horizon is NPPP-complete

NP PP NPPP

A. Darwiche

Propositional Encoding: Initial State

State space: BP (block-painted), BH (block-held), GC (gripper-clean), GD (gripper-dry)

Probabilistic initial state: {BP, BH, GC, p GD} Chance variable: p (labeled with 0.7) p = 1: {BP, BH, GC, GD} probability 0.7 p = 0: {BP, BH, GC, GD} probability (1 - 0.7)

Encoded as propositional sentence; some variables labeled with numbers (probabilities)

A. Darwiche

Propositional Encoding: Action Effects

Dry: dries wet gripper w. p. 0.8; doesn’t affect dry gripper dry GD (q GD’), dry GD GD’

Frame axiom dry (BP BP’), dry (BH BH’), dry (GC

GC’)

Each setting of chance variable selects one effect probability of effect is label of chance variable, or 1

minus that depending on value of variable multiple chance variables: multiply the numbers

Define Pr(e) where e is an event (setting of chance vars)

A. Darwiche

Propositional Encoding: Goal

Goal: {BP’, BH’, GC’}

A. Darwiche

Propositional Encoding: Horizon n

State variables: S0, S1, S2, …, Sn

Chance variables: P-1, P0, P1, …, Pn-1

Action variables: A0, A1, A2, …, An-1

Initial state: I(P-1, S0) Goal: G(Sn)

Plan step: Ak A(Sk, Ak, Pk, Sk+1)

Δn I(P-1, S0) A0 A1 … An-1 G(Sn)

A. Darwiche

Plan Assessment Given propositional encoding Δn and n-step plan

What’s probability of success Pr(Δn, )?

Plan is instantiation of action vars

Pr(Δn, ) is sum of Pr(e) for e consistent with Δn

Computation intractable (has to enumerate models)

A. Darwiche

Brute-force Algorithm Search through all plans (instantiations of action vars)

For each plan , compute Pr(Δn, )

Return plan with max Pr(Δn, )

Improve in two ways Efficient plan assessment Search space pruning

Both achieved by compiling Δn to d-DNNF

A. Darwiche

Compilation to d-DNNF Use existing compiler c2d,

http://reasoning.cs.ucla.edu/c2d/

Compilation intractable in general

For structured problems, exponential only in treewidth

Treewidth does not grow with horizon in this encoding

Can scale to large horizons

http://reasoning.cs.ucla.edu/c2d/

A. Darwiche

d-DNNF

or

and

and

pt’

and

s’p dry paint pickup r dry’ paint’pickup’

A. Darwiche

[paint, pickup’]

or

and

and

pt’

and

s’p dry paint pickup r dry’ paint’pickup’

Plan Assessment on d-DNNF

A. Darwiche

or

and

and

pt’

and

s’p dry paint pickup r dry’ paint’pickup’11 1 1 1 1


[paint, pickup’]

A. Darwiche

[paint, pickup’]

or

and

and

pt’

and

s’p dry paint pickup r dry’ paint’pickup’11 1 .9 1 1 1.5 .3 .7 .95


A. Darwiche

or

and

and

pt’

and


.665.15


[paint, pickup’]

A. Darwiche

[paint, pickup’]

or

and

and

pt’

and


.665.15

.815


A. Darwiche

[paint, pickup’]

or

and

and

pt’

and


.665.15

.815

.7335


A. Darwiche

Set action vars to constants according to plan

Set all state vars to 1 (existential quantification)

Turn chance vars and their negations into numbers

Turn or into summation, and into multiplication

Evaluate resulting arithmetic circuit

Value at root is Pr(Δn, )


A. Darwiche

Assessment of Partial Plan

Partial plan : instantiation of a subset of action vars

Pr(Δn, ): success probability of best completion of

Maximize over uninstantiated action vars

Special d-DNNF:

Turn corresponding or node into max

β x

or

and and

x γ

A. Darwiche

Assessment of Partial Plan

Summations over state vars, followed by maximizations over action vars

All max must be performed after sum

Not guaranteed in d-DNNF

Max and sum nodes mixed in any order

Some max performed too early: Result incorrect!

A. Darwiche

Depth-first Branch-and-Bound

Key observation: performing max too early can only increase result

Result is upper bound on true value

Partial plan can be pruned if upper bound <= value of best plan already found

Depth-first branch-and-bound: will find optimal plan

Tighter bounds leads to more pruning

A. Darwiche

Plan Search

C

B

C

B B

t’cb

.023

t’cb’

.051

t’c’b

.031

t’c’b’

.055

B

tc’b’

.053

tc’b

.117

tcb’

.079

tcb

.009

T

A. Darwiche

Plan Search

C

B

C

B B

t’cb

.023

t’cb’

.051

t’c’b

.031

t’c’b’

.055

B

tc’b’

.053

tc’b

.117

tcb’

.079

tcb

.009

T Best Score: 0.151

A. Darwiche

Plan Search

C

B

C

B B

t’cb

.023

t’cb’

.051

t’c’b

.031

t’c’b’

.055

B

tc’b’

.053

tc’b

.117

tcb’

.079

tcb

.009

T Best Score: 0.151

.135

A. Darwiche

Plan Search

C

B

C

B B

t’cb

.023

t’cb’

.051

t’c’b

.031

t’c’b’

.055

B

tc’b’

.053

tc’b

.117

tcb’

.079

tcb

.009

T Best Score: 0.151

.135

.127

A. Darwiche

Plan Search

C

B

C

B B

t’cb

.023

t’cb’

.051

t’c’b

.031

t’c’b’

.055

B

tc’b’

.053

tc’b

.117

tcb’

.079

tcb

.009

T Best Score: .009.151

.135

.127

A. Darwiche

Plan Search

C

B

C

B B

t’cb

.023

t’cb’

.051

t’c’b

.031

t’c’b’

.055

B

tc’b’

.053

tc’b

.117

tcb’

.079

tcb

.009


.135

.127

A. Darwiche

Plan Search

C

B

C

B B

t’cb

.023

t’cb’

.051

t’c’b

.031

t’c’b’

.055

B

tc’b’

.053

tc’b

.117

tcb’

.079

tcb

.009


.135

.120

A. Darwiche

Plan Search

C

B

C

B B

t’cb

.023

t’cb’

.051

t’c’b

.031

t’c’b’

.055

B

tc’b’

.053

tc’b

.117

tcb’

.079

tcb

.009


.135

.120

A. Darwiche

Plan Search

C

B

C

B B

t’cb

.023

t’cb’

.051

t’c’b

.031

t’c’b’

.055

B

tc’b’

.053

tc’b

.117

tcb’

.079

tcb

.009


.135

.120

A. Darwiche

Plan Search

C

B

C

B B

t’cb

.023

t’cb’

.051

t’c’b

.031

t’c’b’

.055

B

tc’b’

.053

tc’b

.117

tcb’

.079

tcb

.009


.101

A. Darwiche

Plan Search

C

B

C

B B

t’cb

.023

t’cb’

.051

t’c’b

.031

t’c’b’

.055

B

tc’b’

.053

tc’b

.117

tcb’

.079

tcb

.009


.101

A. Darwiche

Probabilistic Reasoning

A. Darwiche

Bayesian Networks

Battery Age Alternator Fan Belt

BatteryCharge Delivered

Battery Power

Starter

Radio Lights Engine Turn Over

Gas Gauge

Gas

Fuel Pump Fuel Line

Distributor

Spark Plugs

Engine Start

A. Darwiche

Bayesian Networks



Battery Power

Starter


Gas Gauge

Gas

Fuel Pump Fuel Line

Distributor

Spark Plugs

Engine Start

ON OFF

OK

WEAK

DEAD

Lights

Batt

ery

P

ow

er .99 .01

.20 .80

0 1

If Battery Power = OK, then Lights = ON (99%)

….

A

B

C

θb|a

θa

θc|a

A B C Pr(.)

a b c θa θb|a θc|a

a b ~c θa θb|a θ~c|a

a ~b c θa θ~b|a θc|a

a ~b ~c θa θ~b|a θ~c|a

. . . …

A. Darwiche

Pr(a) =Pr(a) = .03.03 + .27 = .3+ .27 = .3

false

false

B

.03

.27

A

.56

.14

truetrue

true

false

false

false

Pr(.)

false

true

Joint Distributions

A. Darwiche

Pr(~b) =Pr(~b) = .27.27 + .14 = .41+ .14 = .41

false

false

B

.03

.27

A

.56

.14

truetrue

true

false

false

false

false

true

Pr(.)

Joint Distributions

A. Darwiche.03λaλb + .27λaλ~b + .56λ~aλb + .14λ~a λ~b

false

false

B

.03

A

truetrue

true

false

false

false

false

true

.27

.14

.56

λa*λb * .03

λa*λ~b * .27

λ~a*λb * .56

λ~a*λ~b* .14

F(λ~a, λ~b, λa, λb) =

Pr(.)

Joint Distributions

=.03λaλb + .27λaλ~b + .56λ~aλb + .14λ~a λ~b

F(λ~a, λ~b, λa, λb)

Pr(a,~b)= F(λ~a:0, λ~b:1, λa:1 , λb:0) = .27

Pr(a)= F(λ~a:0, λ~b:1, λa:1 , λb:1) = .03+.27

A

B

C

θb|a

θa

θc|a

A B C Pr(.)

a b c θa θb|a θc|a

a b ~c θa θb|a θ~c|a

a ~b c θa θ~b|a θc|a

a ~b ~c θa θ~b|a θ~c|a

. . . …

A

B

C

θb|a

θa

θc|a

A B C Pr(.)

a b c λa λb λc θa θb|a θc|a

a b ~c λa λb λ~c θa θb|a θ~c|a

a ~b c λa λ~b λc θa θ~b|a θc|a

a ~b ~c λa λ~b λ~c θa θ~b|a θ~c|a

. . . …

F = λa λb λc θa θb|a θc|a + λa λb λ~c θa θb|a θ~c|a + λa λ~b λc θa θ~b|a θc|a +

λa λ~b λ~c θa θ~b|a θ~c|a

….

A

B

C

F = λa λb λc λd θa θb|a θc|a θd|bc +

λa λb λc λ~d θa θb|a θc|a θ~d|bc +

….

A

B

C

D

Each term has 2n variables (n indicators, n parameters)

Each variable has degree one (multi-linear function)

θa

θb|a

θc|a

θd|bc

A. Darwiche

Multi-Linear Functions Arithmetic Circuits

ababaababaababaababaf ||||

A B

**

* *

+

+ +

* * * *

a ab ba aab| ab | ab| ab |

Factoring

A. Darwiche

Compiler:http://reasoning.cs.ucla.edu/c2d

d-DNNFd-DNNFCNFCNF

Multi-Linear Function

ArithmeticCircuit

Encode Decode

Reduction to Logic

A. Darwiche

a c + a b c + cMulti-linear function:Propositional theory:

c ^ (a b) Encode

c

b 1

a 1Arithmetic Circuit

Decode

c

b b

a aSmooth d-DNNF

Compile

MLFsACsCNFsd-DNNF

A. Darwiche

Propositional Encoding of Multi-Linear Functions

Propositional theory:

Δ = c ^ (a b)

Encodes:F = a c + a b c + c

a b c Encodes

T T T abcT T F ab

T F T acT F F a

F T T bc

F T F b

F F T cF F F 1

A. Darwiche

Why Logic?

Encoding local structure is easy: Determinism encoded by adding

clauses:

CSI encoded by collapsing variables:

0| AC

BACABC ||

A. Darwiche

Global Structure:Treewidth w

))exp(( wnO

A. Darwiche



Battery Power

Starter


Gas Gauge

Gas

Fuel Pump Fuel Line

Distributor

Spark Plugs

Engine Start

Local Structure:CSI and Determinism

A. Darwiche



Battery Power

Starter


Gas Gauge

Gas

Fuel Pump Fuel Line

Distributor

Spark Plugs

Engine Start

Context Specific Independence (CSI)


A. Darwiche




Battery Power

Starter


Gas Gauge

Gas

Fuel Pump Fuel Line

Distributor

Spark Plugs

Engine Start

ON OFF

OK

WEAK

DEAD

Lights

Batt

ery

P

ow

er .99 .01

.20 .80

0 1

If Battery Power = Dead,

then Lights = OFF

Determinism

A. Darwiche

X

Y

Belief network

xx

xx

x yx|y

….

….

CNF Smooth d-DNNF

x y x|

yx

x y x|

y

Arithmetic Circuit

The Ace System:http://reasoning.cs.ucla.edu/ace

A. Darwiche

1st International Evaluation of Exact Probabilistic Reasoning Systems (UAI’06, Boston)

135 problem instances from speech, coding, bioinformatics, circuits, medical diagnosis,…

Each team given 4 days of computation time

UCLA: Only team to solve all problems in allotted time (solved all problems in 1 hr)

Failure rates of other teams: 10%-40%

A. Darwiche

Inference by Compiling to d-DNNF

Deterministic Planning Blai Bonet and Hector Geffner (KR 2006)

Probabilistic Conformant Planning Jinbo Huang (ICAPS 2006)

Model-based diagnosis Paul Elliott and Brian Williams (AAAI 2006) Anthony Barrett (IJCAI 2005)

Databases (query re-write) Yolife Arvelo, Blai Bonet and Maria Esther Vidal (AAAI 2006)

Inference in Bayesian Networks (2006 competition) Mark Chavira, Adnan Darwiche (IJCAI 2005)

Inference in Probabilistic Relational Models Mark Chavira, Adnan Darwiche and Manfred Jaeger (IJAR 2006)

c2d compiler: http://reasoning.cs.ucla.edu/c2d

Documents

A. Darwiche Knowledge Compilation Jinbo Huang NICTA and ANU Slides made by Adnan Darwiche and Jinbo Huang