Independence, Decomposability and functions which take values into an Abelian Group Adrian Silvescu...

Independence, Decomposability and functions which take values into an

Abelian Group

Adrian Silvescu Vasant Honavar

Department of Computer ScienceIowa State University

Decomposition and Independence

Decomposition renders problems more tractable.

Apply recursively Decomposition is enabled

by “independence” Decomposition and

independence are dual notions

A B

A B

A B

A B

Conditional Decomposition and Independence

Seldom are the two sub-problems disjoint

All is not lost Conditional

Decomposition / Independence

Conditioning on C C a.k.a. separator

CA B

A BA C C B

CA B

C

=

Formalization of the intuitions

Problem P = (D, S, solP) D = Domain, S = Solutions solP : D S

A BsolP

Example: Determinant_Computation(M2, R, det)

Conditional Independence / Decomposition Formalization (Variable Based)

P = (D = A X B X C, S, solP)

P1 = (A X C, S1, solP1), P2 = (B X C, S2, solP2)

solP(A, B, C) = solP1(A, C) solP2(B, C)

PPP 2,1

Probabilities

I(A, B|C) iff P(A, B| C) = P(A|C) P(B|C) Equivalently P(A, B, C) = P(A, C) P(B|C) P(A, B, C) = f1(A, C) f2(B, C) Independencies can be represented by a

graph where we do not draw edges between variables that are independent conditioned on the rest of the variables.

A C B

The Hammersley-Clifford Theorem: From Pairwise to Holistic Decomposability

)(

)()(GMaxCliquesCC CfVp

Outline

Generalized Conditional Independence with respect to a function f and properties

Theorems Conclusions and Discussion

Conditional Independence with respect to a function f - If(A,B|C)

solP(A, B, C) = solP1(A, C) solP2(B, C)

Assumptions: – S = S1 = S2 [= G]– .

– A, B, C is a partition of the set of all variables– Saturated independence statements – from now on

PPP 2,1

PPP 2,1

f(A, B, C) = f1(A, C) f2(B, C)

If(A,B|C)

Conditional Independence with respect to a function f If(A,B|C) – cont’d

A B C f

0 0 0 .25

0 0 1 .3

… … … …

=

A C f1

0 0 .5

0 1 .3

… … …

B C f2

0 0 .5

0 1 .3

… … …

If(A,B|C) iff

f(A, B, C) = f1(A, C) f2(B, C)

Examples of If(A,B|C )

Multiplicative (probabilities)

Additive (fitness, energy, value functions)

Relational (relations)

Properties of If(A,B|C )

1.Trivial Independence If(A, Φ|C) 2. Symmetry If(A, B|C) => If(B, A|C) 3. Weak Union If(A, B U D|C) => If(A, B|C U D) 4. Intersection If(A, B|C U D) & If(A, D|C U B) => If(A, B U D|C)

A CD

B

Abelian Groups

(G, +, 0, -) is an Abelian Group iff– + is associative and commutative– 0 is a neutral element – - is an inversion operator

Examples:– (R, + , 0, - ) - additive (value func.)– ((0, ∞), · , 1, -¹) - multiplicative (prob.)– ({0, 1}, mod2, 0, id) - relational (relations)

Outline

Generalized Conditional Independence with respect to a function f

Properties and Theorems Conclusions and Discussion

Markov Properties [Pearl & Paz ‘87]

If Axioms 1-4 then the following are equivalent

Pairwise – (α,β) G => If(α, β|V\{α,β})

Local -

If(α, V\(N(α)U{α})| N(α)) Global – If C=V\{A, B}

separates A and B in G If(A, B| C=V\{A, B})

α βV\{α,β}

N(α)α

A BC

Factorization – Main Theorem

The Factorization Theorem: From Pairwise to Holistic Decomposability

)()()(

CfVf CGMaxCliquesC

Particular Cases - Factorization

Probabilistic – Hammersley-Clifford

Additive Decomposability

Relational Decomposability

Graph Separability and Independence [Geiger & Pearl ‘ 93]

If Axioms 1-4 hold then

SepG(A, B|C) If(A, B|C)

for all saturated independence statements

Completeness

Axioms 1-4 provide a complete axiomatic characterization of independence statements for functions which take values over Abelian groups

Outline

Generalized Conditional Independence with respect to a function f

Properties and Theorems Conclusions and Discussion

Conclusions (1)

Introduced a very general notion of Conditional Independence / Decomposability.

Developed it into a notion of Conditional Independence relative to a function f which takes values into an Abelian Group If(.,.|.).

We proved that If(.,.|.) satisfies the following important independence properties:

– 1. Trivial independence, – 2. Symmetry, – 3. Weak union – 4. Intersection

Conclusions (2)

Axioms 1-4 imply the equivalence of the Global, Local and Pairwise Markov Properties for our notion conditional independence relation If(.,.|.)) based on the result from [Pearl and Paz '87].

We proved a natural generalization of the Hammersley-Clifford which allows us to factorize the function f over the cliques of an associated Markov Network which reflects the Conditional Independencies of subsets of variables with respect to f.

Completeness Theorem, Graph Separability Eq. Theorem The theory developed in this paper subsumes: probability

distributions, additive decomposable functions and relations, as particular cases of functions over Abelian Groups.

Discussion: Relation to Graphoids

(-) Decomposition

(-) Contraction

(+) Weak Contraction

Graphoids – No finite axiomatic charact. [Studeny ’92]

Intersection Discussion – noninvertible elms.

Discussion – cont’d

Graph Separability Independence Completeness Seems that

– Trivial Independence– Symmetry– Weak Union– Intersection

Strong Axiomatic core for Independence

Applications