A Graphical Model For A Graphical Model For Simultaneous Partitioning Simultaneous Partitioning
And LabelingAnd Labeling
Philip Cowans &Philip Cowans &
Martin Szummer Martin Szummer
AISTATS, Jan 2005AISTATS, Jan 2005
Cambridge
Motivation – Interpreting InkMotivation – Interpreting Ink
Hand-drawn diagramHand-drawn diagram
Machine Machine interpretationinterpretation
Graph ConstructionGraph ConstructionVertices are Vertices are grouped into grouped into partsparts..
Each part is Each part is assigned a assigned a
labellabelG
Edges, Edges, EE
Vertices, Vertices, VV
Labeled PartitionsLabeled Partitions
We assume:We assume:Parts are Parts are contiguous.contiguous.The graph is triangulated.The graph is triangulated.
We’re interested in We’re interested in probability distributionsprobability distributions over labeled partitions conditioned on over labeled partitions conditioned on observed data.observed data.
Conditional Random FieldsConditional Random Fields
CRFs (Lafferty CRFs (Lafferty et. al.et. al.) ) provide joint labeling of provide joint labeling of graph vertices.graph vertices.
Idea: define parts to be Idea: define parts to be contiguous regions with contiguous regions with same label.same label.
But…But… Large number of labels Large number of labels
needed.needed. Symmetry problems / Symmetry problems /
bias.bias.
+2+1
+2
-1 -1
+3 +3
A Better Approach…A Better Approach…
Extend the CRF framework to work Extend the CRF framework to work directlydirectly with labeled partitions. with labeled partitions.Complexity is improved – don’t need to deal Complexity is improved – don’t need to deal
with so many labels.with so many labels.No symmetry problem – we’re working directly No symmetry problem – we’re working directly
with the representation in which the problem with the representation in which the problem is posed.is posed.
ProjectionProjection
Projection maps labeled Projection maps labeled partitions onto smaller partitions onto smaller subgraphs.subgraphs.
If G If G µµ VV then, the then, the projection of projection of YY onto G is onto G is the unique labeled the unique labeled partition of G which is partition of G which is ‘consistent’ with ‘consistent’ with YY..
NotationNotation
YY Labeled partition of Labeled partition of GG
YY (A)(A) Labeled partition of the induced subgraph Labeled partition of the induced subgraph of A of A µµ VV
YYAA Projection of Projection of YY onto A onto A µµ VV
YYii Projection of Projection of YY onto vertex i. onto vertex i.
YYijij Projection of Projection of YY onto vertices i and onto vertices i and j.j.
PotentialsPotentials
The ModelThe Model
- Unary:Unary:
- Pairwise:Pairwise:
The ModelThe Model
TrainingTraining
Train by finding MAP weights on example Train by finding MAP weights on example data with Gaussian prior (BFGS).data with Gaussian prior (BFGS).
We require the value and gradient of the We require the value and gradient of the log posterior:log posterior:
Normalization
Marginalization
PredictionPrediction
New data is processed by finding the most New data is processed by finding the most probable labeled partition.probable labeled partition.
This is the same as normalization with the This is the same as normalization with the summation replaced by a maximization.summation replaced by a maximization.
InferenceInference
These operations require summation or These operations require summation or maximization over all possible labeled maximization over all possible labeled partitions.partitions.
The number of terms grows super-The number of terms grows super-exponentially with the size of exponentially with the size of GG..
Efficient computation possible using Efficient computation possible using message passing as distribution factors.message passing as distribution factors.
Proof based on Shenoy & Shafer (1990).Proof based on Shenoy & Shafer (1990).
Message PassingMessage Passing
44
88
77
99
11
22 33
55
66
Message PassingMessage Passing
1,2,3,41,2,3,4
2,3,4,52,3,4,5
2,92,91,7,81,7,8
4,5,64,5,6
‘‘Upstream’Upstream’Message summarizes Message summarizes
contribution from contribution from ‘upstream’ to the ‘upstream’ to the sum for a given sum for a given configuration of the configuration of the separator.separator.
Junction tree constructed from cliques on original
graph.
Message PassingMessage Passing
PartitionPartition LabelsLabels ValueValue
(2)(3)(4)(2)(3)(4) +,+,-+,+,- 0.0120.012
(2)(3)(4)(2)(3)(4) +,-,-+,-,- 0.0430.043
(2,3)(4)(2,3)(4) +,++,+ 0.1340.134
(2,3,4)(2,3,4) -- 0.2350.235
…… …… ……
1,2,3,41,2,3,4
2,3,4,52,3,4,5
2,92,91,7,81,7,8
4,5,64,5,6
PartitionPartition LabelsLabels ValueValue
(2)(2) ++ 0.430.43
(2)(2) -- 0.720.72
PartitionPartition LabelsLabels ValueValue
(1)(1) ++ 0.230.23
(1)(1) -- 0.570.57
x22x22
Message Update RuleMessage Update Rule
Update messages (for summation) according toUpdate messages (for summation) according to
Marginals found usingMarginals found using
Z can be found explicitlyZ can be found explicitly
ComplexityComplexity
Clique SizeClique Size 22 33 44 55
CRFCRF 1616 216216 40964096 1.0 1.0 ££ 10 1055
Labeled PartitionsLabeled Partitions 66 2222 9494 454454
1
1000
1E+06
1E+09
1E+12
1E+15
1 3 5 7 9 11
Labeled Partitions
CRF
Experimental ResultsExperimental Results
We tested the algorithm on We tested the algorithm on hand drawn hand drawn inkink collected using a Tablet PC. collected using a Tablet PC.
The task is to partition the ink fragments The task is to partition the ink fragments into perceptual objects, and label them as into perceptual objects, and label them as containerscontainers or or connectorsconnectors..
Training data set was 40 diagrams, from Training data set was 40 diagrams, from 17 subjects with a total of 2157 fragments.17 subjects with a total of 2157 fragments.
3 random splits (20 training and 20 test 3 random splits (20 training and 20 test examples).examples).
Example 1Example 1
Example 1Example 1
Example 2Example 2
Example 2Example 2
Example 3Example 3
Example 3Example 3
Labeling ResultsLabeling ResultsModelModel Labeling Labeling
ErrorErrorGrouping Grouping
ErrorError
Independent LabelingIndependent Labeling 8.5%8.5% --
Joint LabelingJoint Labeling 4.5%4.5% --
Labeled PartitionsLabeled Partitions 2.6%2.6% 8.5%8.5%
• Labelling error: fraction of fragments Labelling error: fraction of fragments labeled incorrectly.labeled incorrectly.
• Grouping error: fraction of edges locally Grouping error: fraction of edges locally incorrect.incorrect.
ConclusionsConclusions
We have presented a conditional model definied We have presented a conditional model definied over labeled partitions of an undirected graph.over labeled partitions of an undirected graph.
Efficient exact inference is possible in our model Efficient exact inference is possible in our model using message passing.using message passing.
Labeling and grouping simultaneously can Labeling and grouping simultaneously can improve labeling performance.improve labeling performance.
Our model performs well when applied to the Our model performs well when applied to the task of parsing hand-drawn ink diagrams.task of parsing hand-drawn ink diagrams.
AcknowledgementsAcknowledgements
Thanks to:Thanks to:
Thomas Minka, Yuan Qi and Michel Gagnet Thomas Minka, Yuan Qi and Michel Gagnet for useful discussion and providing for useful discussion and providing
software. software.
Hannah Pepper for collecting our ink Hannah Pepper for collecting our ink database.database.