Upload
terry-payne
View
125
Download
0
Embed Size (px)
Citation preview
Negotiating over Ontological Correspondences
with Asymmetric and Incomplete Knowledge
Terry Payne & Valentina TammaUniversity of Liverpool
[email protected]@liverpool.ac.uk
Negotiating over Ontological Correspondences with Asymmetric and Incomplete Knowledge
Terry Payne University of Liverpool
Open Systems, Ontologies and Alignment
• Agents can assume different ontological models• Modelled implicitly, or explicitly by defining entities (classes, roles etc), typically using
some logical theory, i.e. an Ontology
• Alignment Systems align similar ontologies
• If we assume that different alignments exist, how do agents choose which to use?
2
Alignment
Correspondence
Negotiating over Ontological Correspondences with Asymmetric and Incomplete Knowledge
Terry Payne University of Liverpool
Align Everything?
• What does the agent know?• Pre-computed alignments exist, and can be shared
• Different agents may possess different alignment fragments from different sources.
• Do we need everything to be aligned?• An agent may aggregate several ontologies for a variety of domains
• A task may be relevant to only a single module within an ontology
• Fragments of the ontological space may be confidential, or commercially sensitive.
3
Negotiating over Ontological Correspondences with Asymmetric and Incomplete Knowledge
Terry Payne University of Liverpool
Aims and Contribution• Correspondence Inclusion Dialogue (CID)
• Allows two agents to exchange knowledge about correspondences to agree upon a mutually acceptable final alignment AL.
• This alignment aligns only those entities in each agents’ working ontologies, without disclosing the ontologies, or all of the known correspondences.
• Assumptions1. Each agent knows about different correspondences from different sources
2. This knowledge is partial, and possibly ambiguous; i.e. more than one correspondence exists for a given entity
3. Agents associate a utility (Degree of Belief) κc to each unique correspondence
4. Correspondences with a joint utility below the admissibility threshold ϵ should be rejected
4
Negotiating over Ontological Correspondences with Asymmetric and Incomplete Knowledge
Terry Payne University of Liverpool
• Correspondences are modelled as Beliefs
• The degree of belief κc reflects the agent specific belief of the utility of c aligning its corresponding entities in the two working ontologies
• The beliefs of both agents for a correspondence can be combined into a joint belief
Modelling and Aggregating Beliefs
5
� = hc,ci,where correspondence c = he, e0,=i and e 2 W, e0 2 W 0
joint(c) =
8><
>:
avg(x
c
,
x̂
c
) if beliefs from both agents x and x̂ are disclosed
12 (
x̂
c
) if the beliefs of c is only known to x
avg(x
c
,
x
upper
) when x̂ ’s belief of c is not yet known
Negotiating over Ontological Correspondences with Asymmetric and Incomplete Knowledge
Terry Payne University of Liverpool
Knowledge Model
6
Working Ontology (a fragment of the agent’s ontology) contains the entities to be aligned
Alignment Store contains the correspondences known to the agent
Joint Belief Store tracks all beliefs
(commitments) disclosed by both
agents
As beliefs are shared, the agent
builds up a joint degree of belief of
the utility of the correspondence
Joint Belief Store JB
bob = hha, x,=i, 0.6i�
alice = hha, x,=i, 0.8i
�
alice = hhb, x,=i, 0.5i
bob = hhb, x,=i, 0.8i· · ·
Alignment Store �
Ontology O
joint(ha, x,=i) = 0.7joint(hb, x,=i) = 0.65�
alice = hhb, z,=i, 0.9i�
alice = hha, x,=i, 0.8i�
alice = hhb, w,=i, 0.6i�
alice = hhb, x,=i, 0.5i Alice
· · ·
W =
{a, b, a v b}
Public KnowledgePrivate Knowledge
Negotiating over Ontological Correspondences with Asymmetric and Incomplete Knowledge
Terry Payne University of Liverpool
Ambiguity and Attacks
7
publication article author
submittedPaper reviewedPaper paper editor
• Alignments typically consist of one-to-one mappings• Combining correspondences from different alignment fragments can
result in one-to-many correspondences; i.e. ambiguity
• Agents therefore need a mechanism for selecting the ‘best’ set of correspondences
Negotiating over Ontological Correspondences with Asymmetric and Incomplete Knowledge
Terry Payne University of Liverpool
Correspondence Inclusion Dialogue (CID)
• Inquiry Dialogue, use to determine mutually acceptable correspondences to facilitate some task
• Agents take turns to select and propose a belief they know of, that has not yet been asserted, based on its utility κc
• A shared, or asserted correspondence is:
• accepted based on their combined κc (i.e. joint(c))
• rejected if joint(c) < ϵ, the admissibility threshold • objected to if an agent believes a better correspondence exists for
one of the entities in the correspondence
• The dialogue is presented formally in the paper
8
Negotiating over Ontological Correspondences with Asymmetric and Incomplete Knowledge
Terry Payne University of Liverpool
Commitment Strategy
• Correspondences are selected (asserted) for disclosure:
1. Agents assert the best undisclosed correspondence (i.e. with the highest κc) in any round
2. Disclosed correspondences should be grounded in the working ontology
3. A correspondence should not be disclosed if the joint degree of belief is guaranteed to be less than the admissibility threshold.
9
Negotiating over Ontological Correspondences with Asymmetric and Incomplete Knowledge
Terry Payne University of Liverpool
Objecting Strategy
10
… 2 Alice -> Bob:ASSERT <0.9, {http://l#b = http://h#z}> 3 Bob -> Alice:OBJECT <0.8, {http://l#b = http://h#x}> to <0.0, {http://l#b = http://h#z}> 4 Alice -> Bob:OBJECT <0.8, {http://l#a = http://h#x}> to <0.5, {http://l#b = http://h#x}> 5 Bob -> Alice:ACCEPT <0.6, {http://l#a = http://h#x}> 6 Bob -> Alice:OBJECT <0.4, {http://l#b = http://h#w}> to <0.0, {http://l#b = http://h#z}> 7 Alice -> Bob:ACCEPT <0.6, {http://l#b = http://h#w}> …
a b c
z w x y
• Alternate correspondences are counter-proposed (through an objection):
1. If an agent has an undisclosed correspondence that shares an entity and:
2. It’s estimated κc is greater than the joint degree of belief for the current correspondence
• Each objection results in the agent’s actual degree of belief being disclosed.
Negotiating over Ontological Correspondences with Asymmetric and Incomplete Knowledge
Terry Payne University of Liverpool
Example DialoguePublic Knowledge Private Knowledge
Ontology O
W = {w, x, y, z,x v w}
Alignment Store �
�
bob = hhb, x,=i, 0.8i�
bob = hha, x,=i, 0.6i�bob = hhb, w,=i, 0.4i
Joint Belief Store JB
Public KnowledgePrivate Knowledge
Alignment Store �
�alice = hhb, z,=i, 0.9i�
alice = hha, x,=i, 0.8i
�
alice = hhb, x,=i, 0.5i�alice = hhb, w,=i, 0.6i
W = {a, b, c,a v b}
Ontology O Joint Belief Store JB
Alice asserts the correspondence with the highest κc
Negotiating over Ontological Correspondences with Asymmetric and Incomplete Knowledge
Terry Payne University of Liverpool
Example DialoguePublic Knowledge Private Knowledge
Ontology O
W = {w, x, y, z,x v w}
Alignment Store �
�
bob = hhb, x,=i, 0.8i�
bob = hha, x,=i, 0.6i�bob = hhb, w,=i, 0.4i
Joint Belief Store JB
alice = hhb, z,=i, 0.9i
Public KnowledgePrivate Knowledge
Alignment Store �
�alice = hhb, z,=i, 0.9i�
alice = hha, x,=i, 0.8i
�
alice = hhb, x,=i, 0.5i�alice = hhb, w,=i, 0.6i
W = {a, b, c,a v b}
Ontology O Joint Belief Store JB
hAlice, assert, hhb, z,=i, 0.9i, nili
�alice = hhb, z,=i, 0.9i
Alice asserts the correspondence with the highest κc
Negotiating over Ontological Correspondences with Asymmetric and Incomplete Knowledge
Terry Payne University of Liverpool
Example DialoguePublic Knowledge Private Knowledge
Ontology O
W = {w, x, y, z,x v w}
Alignment Store �
�
bob = hhb, x,=i, 0.8i�
bob = hha, x,=i, 0.6i�bob = hhb, w,=i, 0.4i
Joint Belief Store JB
alice = hhb, z,=i, 0.9i
Public KnowledgePrivate Knowledge
Alignment Store �
�alice = hhb, z,=i, 0.9i�
alice = hha, x,=i, 0.8i
�
alice = hhb, x,=i, 0.5i�alice = hhb, w,=i, 0.6i
W = {a, b, c,a v b}
Ontology O Joint Belief Store JB
hAlice, assert, hhb, z,=i, 0.9i, nili
�alice = hhb, z,=i, 0.9i
Bob realises that joint(⟨b,z,=⟩) = 0.45, and finds an alternative c with jointest(⟨b,x,=⟩) = 0.85
Negotiating over Ontological Correspondences with Asymmetric and Incomplete Knowledge
Terry Payne University of Liverpool
Example DialoguePublic Knowledge Private Knowledge
Ontology O
W = {w, x, y, z,x v w}
Alignment Store �
�
bob = hhb, x,=i, 0.8i�
bob = hha, x,=i, 0.6i�bob = hhb, w,=i, 0.4i
Joint Belief Store JB
alice = hhb, z,=i, 0.9i
Public KnowledgePrivate Knowledge
Alignment Store �
�alice = hhb, z,=i, 0.9i�
alice = hha, x,=i, 0.8i
�
alice = hhb, x,=i, 0.5i�alice = hhb, w,=i, 0.6i
W = {a, b, c,a v b}
Ontology O Joint Belief Store JB
hAlice, assert, hhb, z,=i, 0.9i, nili
�alice = hhb, z,=i, 0.9i
joint(hb, z,=i) = 0.45
hBob, object, hhb, x,=i, 0.8i, hhb, z,=i, 0.0ii
bob = hhb, z,=i, 0.0i�
bob = hhb, x,=i, 0.8i�bob = hhb, z,=i, 0.0i
Bob realises that joint(⟨b,z,=⟩) = 0.45, and finds an alternative c with jointest(⟨b,x,=⟩) = 0.85
bob = hhb, x,=i, 0.8iUsing the Upper Bound
Alice’s previous assertion was a belief with a utility of 0.9. Therefore the
utility of ⟨b,x,=⟩ ≤ 0.9.As Bob’s ⟨b,x,=⟩ is 0.8, he calculates
jointest(⟨b,x,=⟩) = avg(0.9, 0.8)
Negotiating over Ontological Correspondences with Asymmetric and Incomplete Knowledge
Terry Payne University of Liverpool
Example DialoguePublic Knowledge Private Knowledge
Ontology O
W = {w, x, y, z,x v w}
Alignment Store �
�
bob = hhb, x,=i, 0.8i�
bob = hha, x,=i, 0.6i�bob = hhb, w,=i, 0.4i
Joint Belief Store JB
alice = hhb, z,=i, 0.9i
Public KnowledgePrivate Knowledge
Alignment Store �
�alice = hhb, z,=i, 0.9i�
alice = hha, x,=i, 0.8i
�
alice = hhb, x,=i, 0.5i�alice = hhb, w,=i, 0.6i
W = {a, b, c,a v b}
Ontology O Joint Belief Store JB
hAlice, assert, hhb, z,=i, 0.9i, nili
�alice = hhb, z,=i, 0.9i
joint(hb, z,=i) = 0.45
hBob, object, hhb, x,=i, 0.8i, hhb, z,=i, 0.0ii
bob = hhb, z,=i, 0.0i�
bob = hhb, x,=i, 0.8i�bob = hhb, z,=i, 0.0i
Bob realises that joint(⟨b,z,=⟩) = 0.45, and finds an alternative c with jointest(⟨b,x,=⟩) = 0.85
bob = hhb, x,=i, 0.8i
joint(hb, z,=i) = 0.45
Negotiating over Ontological Correspondences with Asymmetric and Incomplete Knowledge
Terry Payne University of Liverpool
Example DialoguePublic Knowledge Private Knowledge
Ontology O
W = {w, x, y, z,x v w}
Alignment Store �
�
bob = hhb, x,=i, 0.8i�
bob = hha, x,=i, 0.6i�bob = hhb, w,=i, 0.4i
Joint Belief Store JB
alice = hhb, z,=i, 0.9i
Public KnowledgePrivate Knowledge
Alignment Store �
�alice = hhb, z,=i, 0.9i�
alice = hha, x,=i, 0.8i
�
alice = hhb, x,=i, 0.5i�alice = hhb, w,=i, 0.6i
W = {a, b, c,a v b}
Ontology O Joint Belief Store JB
hAlice, assert, hhb, z,=i, 0.9i, nili
�alice = hhb, z,=i, 0.9i
joint(hb, z,=i) = 0.45
hBob, object, hhb, x,=i, 0.8i, hhb, z,=i, 0.0ii
bob = hhb, z,=i, 0.0i�
bob = hhb, x,=i, 0.8i�bob = hhb, z,=i, 0.0i
Alice calculates joint(⟨b,x,=⟩) is 0.65. As jointest(⟨a,x,=⟩) is 0.9, which is greater than joint(⟨b,x,=⟩), she objects.
bob = hhb, x,=i, 0.8i
joint(hb, z,=i) = 0.45
hAlice, object, hha, x,=i, 0.8i, hhb, x,=i, 0.5ii
joint(hb, x,=i) = 0.65
Negotiating over Ontological Correspondences with Asymmetric and Incomplete Knowledge
Terry Payne University of Liverpool
Example DialoguePublic Knowledge Private Knowledge
Ontology O
W = {w, x, y, z,x v w}
Alignment Store �
�
bob = hhb, x,=i, 0.8i�
bob = hha, x,=i, 0.6i�bob = hhb, w,=i, 0.4i
Joint Belief Store JB
alice = hhb, z,=i, 0.9i
Public KnowledgePrivate Knowledge
Alignment Store �
�alice = hhb, z,=i, 0.9i�
alice = hha, x,=i, 0.8i
�
alice = hhb, x,=i, 0.5i�alice = hhb, w,=i, 0.6i
W = {a, b, c,a v b}
Ontology O Joint Belief Store JB
hAlice, assert, hhb, z,=i, 0.9i, nili
�alice = hhb, z,=i, 0.9i
joint(hb, z,=i) = 0.45
hBob, object, hhb, x,=i, 0.8i, hhb, z,=i, 0.0ii
bob = hhb, z,=i, 0.0i�
bob = hhb, x,=i, 0.8i�bob = hhb, z,=i, 0.0i
Bob calculates joint(⟨a,x,=⟩) is 0.7. As he has no further objections, he accepts this.
bob = hhb, x,=i, 0.8i
joint(hb, z,=i) = 0.45
hAlice, object, hha, x,=i, 0.8i, hhb, x,=i, 0.5ii
joint(hb, x,=i) = 0.65
�
alice = hhb, x,=i, 0.5i�
alice = hha, x,=i, 0.8i
hBob, accepts, hha, x,=i, 0.6ii
alice = hha, x,=i, 0.8i
joint(hb, x,=i) = 0.65
alice = hhb, x,=i, 0.5i
joint(ha, x,=i) = 0.7
Negotiating over Ontological Correspondences with Asymmetric and Incomplete Knowledge
Terry Payne University of Liverpool
Example DialoguePublic Knowledge Private Knowledge
Ontology O
W = {w, x, y, z,x v w}
Alignment Store �
�
bob = hhb, x,=i, 0.8i�
bob = hha, x,=i, 0.6i�bob = hhb, w,=i, 0.4i
Joint Belief Store JB
alice = hhb, z,=i, 0.9i
Public KnowledgePrivate Knowledge
Alignment Store �
�alice = hhb, z,=i, 0.9i�
alice = hha, x,=i, 0.8i
�
alice = hhb, x,=i, 0.5i�alice = hhb, w,=i, 0.6i
W = {a, b, c,a v b}
Ontology O Joint Belief Store JB
hAlice, assert, hhb, z,=i, 0.9i, nili
�alice = hhb, z,=i, 0.9i
joint(hb, z,=i) = 0.45
hBob, object, hhb, x,=i, 0.8i, hhb, z,=i, 0.0ii
bob = hhb, z,=i, 0.0i�
bob = hhb, x,=i, 0.8i�bob = hhb, z,=i, 0.0i
The agents continue until they have no more correspondences to consider.
bob = hhb, x,=i, 0.8i
joint(hb, z,=i) = 0.45
hAlice, object, hha, x,=i, 0.8i, hhb, x,=i, 0.5ii
joint(hb, x,=i) = 0.65
�
alice = hhb, x,=i, 0.5i�
alice = hha, x,=i, 0.8i
hBob, accepts, hha, x,=i, 0.6ii
�
bob = hha, x,=i, 0.6i
alice = hha, x,=i, 0.8i
joint(hb, x,=i) = 0.65
alice = hhb, x,=i, 0.5i
bob = hha, x,=i, 0.6i
joint(ha, x,=i) = 0.7joint(ha, x,=i) = 0.7
joint(hb, w,=i) = 0.5 joint(hb, w,=i) = 0.5
hBob, object, hhb, w,=i, 0.4i, hhb, z,=i, 0.0iihAlice, accepts, hhb, w,=i, 0.6ii· · ·
bob = hhb, w,=i, 0.4i�alice = hhb, w,=i, 0.6i
�bob = hhb, w,=i, 0.4i alice = hhb, w,=i, 0.6i
Negotiating over Ontological Correspondences with Asymmetric and Incomplete Knowledge
Terry Payne University of Liverpool
Modelling the objections as an attack graph
• The objections can form a Dungian attack graph• Attacks can be directed by the difference in the degree of belief
of each correspondence • Bi-directional attacks are resolved by random selection of one of the
alternatives
• Can then use grounded semantics to determine the extension
16
⟨a,x,=⟩ 0.7
⟨b,x,=⟩ 0.65
⟨b,w,=⟩ 0.5
⟨b,z,=⟩ 0.45
Negotiating over Ontological Correspondences with Asymmetric and Incomplete Knowledge
Terry Payne University of Liverpool
Empirical Analysis: Datasets
• Datasets taken from the Ontology Alignment Evaluation Initiative (2012) Conference Track• 7 Ontologies, each describing the conference domain
• Each included a reference ontology, used by OAEI to evaluate alignment performance.
• 17 Alignment systems used to generate pairwise alignments between the ontologies • 21 pairwise alignments tested
• 357 alignments in total
17
Negotiating over Ontological Correspondences with Asymmetric and Incomplete Knowledge
Terry Payne University of Liverpool
Experimental Method• Evaluate alignment generated through two agents
negotiating• Each agent starts with 8 of 17 randomly selected alignments
form the OAEI repository.
• Degree of Belief κc based on the probability of a correspondence appearing in the alignments
• Admissibility thresholds ϵ were varied between 1/16 (no filtering) to 1 in 1/16ths
• Each experiment was run 500 times for statistical significance
18
Negotiating over Ontological Correspondences with Asymmetric and Incomplete Knowledge
Terry Payne University of Liverpool
Hypothesis 1: Are we fit for purpose?
• Compare each alignment AL generated through dialogue with OAEI reference ontology
• Performance evaluated using Precision, Recall and F-measure
• Baseline result based on selecting a single alignment at random
19
Negotiating over Ontological Correspondences with Asymmetric and Incomplete Knowledge
Terry Payne University of Liverpool
Hypothesis 1: Are we fit for purpose?
20
-0.6
-0.5
-0.4
-0.3
-0.2
-0.1
0
0.1
0.2
2/16 4/16 6/16 8/16 10/16 12/16 14/16 16/16Diffe
renc
e in
F-m
easu
re fr
om R
ando
m A
lignm
ent A
vera
ge
Admissibility Threshold
Delta f-measure performance for all 21 Ontology Pairs
F-measure for each ontology pair found to be statistically higher than baseline for 3/16 ≤ ϵ < 14/16
Negotiating over Ontological Correspondences with Asymmetric and Incomplete Knowledge
Terry Payne University of Liverpool
Hypothesis 2: Can filtering out correspondences help?
• Calculate percentage of unique mappings disclosed/messages exchanged during the negotiation.
• Results averaged across all ontology pairs
• Evaluate the significance of the upper-bound κupper in the strategy
• Determine the percent of correspondences in final alignment AL
21
Negotiating over Ontological Correspondences with Asymmetric and Incomplete Knowledge
Terry Payne University of Liverpool
Hypothesis 2: Can filtering out correspondences help?
22
0
20
40
60
80
100
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
Perc
enta
ge o
f Com
bine
d U
niqu
e C
orre
spon
denc
es
Admissibility Threshold
Percentage of Correspondences Disclosed or in the Final Alignment
% in final alignment% disclosed (using the upper boud)
% disclosed (no upper bound)
jointest(c) ≈ 0.5 when κupper is fixed (i.e. not used) and ϵ ≤ 9/16
the number of ambiguous correspondences falls as ϵ increases
When ϵ > 9/16, low κc correspondences are filtered
Negotiating over Ontological Correspondences with Asymmetric and Incomplete Knowledge
Terry Payne University of Liverpool
Conclusions• Developed a formal Inquiry Dialogue that supports the
sharing of ontological correspondences between agents• Only those correspondences relating to the agents working ontology
are aligned, thus avoiding unnecessary alignment
• Implemented a full version of the dialogue for evaluation• The resulting alignment performs significantly better in most cases
than the average performance of other approaches, when tested with a reference alignment
• By modelling the opponent’s assertions, an agent can more accurately estimate the joint utility of the correspondence, resulting in an exponential drop in correspondences disclosed and messages exchanged as the threshold increases
23