Formalization of Context and Situation Awareness Mitch Kokar [email protected]

Formalization of Context and Situation Awareness

Mitch [email protected]

Outline• What is “context”?• Formalizations of context• Situations as contexts• Situation Theory• STO – Situation Theory Ontology• Merging knowledge with context• Examples

– Intelligence and context– METT-TC as context

13 November 2009 WPAFB 2


Context - dictionaryMain Entry: con·text

Etymology: Middle English, weaving together of words, from Latin contextus connection of words, coherence, from contexere to weave together, from com- + texere to weave

1 : the parts of a discourse that surround a word or passage and can throw light on its meaning2 : the interrelated conditions in which something exists or occurs


Context

• The same gray square looks whiter against a dark background

and blacker when placed in a bright surround

• The manner in which a stimulus is perceived depends not only on its own physical characteristics but also on those of surrounding stimuli and of stimuli previously experienced by the observer.

[Encyclopedia Britannica]


Another example

• “I am a philosopher.” says A to B.• “She is a philosopher” says B to C.• “So you are a philosopher” says C to A• Note that “I”, “she” and “you” all

refer to the same object, A.• Go figure, computer!


Other context definitions

• “By context, we mean information about physical characteristics (such as location and network elements), the system (such as applications running and available services), and the user (such as privacy and presence).” [Khedr, Karmouch]

• “SCOPES integrates several automated and semi-automated techniques … to facilitate an incremental construction of the knowledge, referred heretofore as context, necessary to translate a query posed against a local database into an equivalent one against a remote database. [Ouksel]

• “Lexical polysemy either arises due to the fact that words can be used in different part-of-speeches (syntactical polysemy) or due to words that have varying meanings in different contexts (semantical polysemy).”

[Lamparter et al.]


Why Use Context?

• No need to explicitly encode knowledge on everything (McCarthy’s criticism of MYCIN). Use generic rules to “lift” knowledge to appropriate context.

• Economy of representation (not one large database)• Inconsistent databases can co-exist, but in different contexts• Efficiency of reasoning – modularization by context• Efficiency in querying (short queries are interpretable in a

context)• Lexical ambiguity (same terminology can be used in modules,

context resolves ambiguity)• Flexible entailment (closed vs. open world)


Formalization of “context”• McCarthy: “Whenever we write an axiom, a critic can

say that the axiom is true only in a certain context. … usually can devise a more general context in which the axiom doesn’t hold.”

• Guha: Formalized ist(C,p) – “proposition p is true in context C”.

• Make context a first-class object – manipulate– Same context can be merged with different knowledgebases

• Lifting axioms – relating truth in different contexts (axioms about context)


Handling context formally

• Giunchiglia: LMS – Local Model Semantics – inference rules for reasoning about context

• Buvac: PLC – Propositional Logic of Context – lifting axioms


Barwise & Devlin• Context is just Situation Type• If Claire rubs her eyes then Claire is sleepy. Unless there is

pollen involved!


Situations – First Class Citizens

“One of the starting points for situation semantics was the promotion of realsituations from second class citizens to first class citizens.”

“By a situation, then, we mean a part of reality that can be comprehendedas a whole in its own right – one that interacts with other things.

By interacting with other things we mean that they have properties or relateto other things.”

- Jon Barwise, The Situation in Logic, 1989

Situation Theory (Barwise/Perry/Devlin)


s ⊨ ≪R, a1, …, an, 1≫ “situation s “supports”I.e., in situation s objects a1, …, an stand in relation R

s ⊨ ≪R, a1, …, an, 0≫ “situation s “supports”I.e., in situation s objects a1, …, an do not stand in relation R

“Information is always taken to be (…) about some situations, and is (…) built up from discrete information items known as infons.”“Infons are not things that in themselves are true or false. Rather aparticular item of information may be true or false about a situation.”

Problem: The symbol ⊨ means “supports” in Situation Theory but it means “satisfies” in model theory.


Situations

Representation (infons)

<<R, o1, o2, l1, t1, 1>>

<<type, s, Situation, l1, t1, 1>>

supports

ssupports

entails

AgentWorld

Formalization of Situation Theory

M. M. Kokar, C. J. Matheus, and K. Baclawski. “Ontology-based situation awareness.” Information Fusion, 10:83-98, 2009.


Barwise/Devlin Ontology (types)• SIT – situations• RELn – n-place relations• IND – individuals• POL – polarity (0 or 1)• LOC – locations• TIM – temporal locations• PAR – parameters• TYP – type of all types

Ontologies• What is an Ontology?

– Language (vocabulary) of discourse for a given domain

– Definition of concepts (classes), individuals and relationships relevant to the problem domain

– Captured in a formal language e.g., OWL– Computer processable semantics (so that

computers can “understand”)– Data annotation

– Markup of data in terms of the classes that entities represent and the values of their properties

13 November 2009 15WPAFB


Situation Theory Ontology (STO)


Situation Types


Relations in STO


Role of Observer• Since a situation is perceived by an

observer (or starts with an utterance), therefore a situation must include an utterance (or query), as well as background knowledge of the observer.

• Query then defines what should and what should not be included in the situation (and thus should, or should not, count as context).



Situation Type Examples• Chasing• Running away (from a lion)• Eating (because food was brought in, Rex

stops chasing Fluffy)• Still chasing (something unimportant

happens, e.g., Jim’s brother comes in)• Iraq invading Kuwait

Ontology Mapping & Integration• Still in the laboratory stage• IFF: Information Flow Framework -Robert Kent, Ontologos

Consortium (related to the SUO effort) http://www.ontologos.org/ • IF-Map – Kalfoglou and Schorlemmer

http://www.aktors.org/technologies/ifmap/ • MAFRA - A MApping FRAmework for Distributed Ontologies

http://tw.rpi.edu/wiki/Main_Page • The BUSTER Project, University of Bremen – not active?• Toward Semantic Information Integration - Eduard Hovy, ISI

University of Southern California, http://www.isi.edu/natural-language/people/hovy.html

• Monika Crubezy - http://www-sop.inria.fr/orion/personnel/Monica.Crubezy/me.html


Ontology Alignment: General Schema Using “colimit” of Category Theory


1. Identify what should be unified (morphisms)2. “Add” two ontologies (colimit)3. Insure integration is logically consistent (consistent maps, obey

signatures of operations, obey axioms).

From: “IF-Map: An Ontology-Mapping Method based on Information-Flow Theory,” by Yannis Kalfoglou and Marco Schorlemmer

Ontology Alignment: Example – Easy!


• O1 contains “Car”• O2 contains “Automobile”• And these are synonyms

Easy!

Ontology Alignment: Difficult!


1. River and Fleuve are close, but not the same.2. Stream and Riviere are close but not the same.3. Axioms would need to be preserved in order to translate correctly.

From: “IF-Map: An Ontology-Mapping Method based on Information-Flow Theory,” by Yannis Kalfoglou and Marco Schorlemmer

Example: Intelligence



Intelligence and Context• 80% of intelligence data today is from HUMINT• This is all about She-said, He-said …• If reifying – then all can be inferred is who said what• Need to integrate data in order to make inferences

about not who said what, but what is true and what’s not

• What should be integrated and what not?• Situation Theory of Barwise is all about

interpretation of speech acts


Example Scenario: Iran supplies weapons to Iraq Insurgents?

• Military officials provides evidence Iran is supplying EFPs, mortars, other deadly weapons to Iraqi insurgents; Qods forces members in custody

• The Telegraph reports a large quantity of Austrian made .50 caliber sniper rifles were seized in Baghdad

• IRNA reports that Austrian-made rifles have been found by US military in Iraq.


Mapping the Example to STO

• Utterance situation:– U1: Iran supplies weapons to Iraqi insurgents; – Attributes of u1: time=-070211, source=United Nations

• Individuals: Iran, Iranians, Rifle, Weapon, Austria, Austrians, Iraqi Insurgents

• Relations (just examples):– delivers(Supplier, Receiver, Product, Year); – purchases(Customer, From, Item, Year, Price)


Resource Situations

• R1: U.S. Military reports a large quantity of Iranian owned rifles were seized in Baghdad– Attributes: time = 070212, source=U.S. Military

• R2: Austrian sniper rifles that were exported to Iran have been discovered in the hands of Iraqi terrorists– Attributes: time=070213, source = The Telegraph

• R3 (I made it up): Islamic Republic News Agency (IRNA) has reported that Austrian made weapons were found with Iraqi Insurgents– Attributes: time=010213, source=IRNA

• Inferences depend on which of the situations, R1, R2 or R3, accepted as a “resource situation”


Inferences• R1: U.S. Military reports a large quantity of

Iranian owned rifles were seized in Baghdad– Does being in Baghdad mean same as being

with Iraq Insurgents?– Does Iran make such weapons?– Could infer only what was said by the U.S.

Military, but not based upon any background knowledge


Inferences (2)• R2: Austrian sniper rifles that were exported

to Iran have been discovered in the hands of Iraqi terrorists– If combined with R1 (and some more

background knowledge) would infer (independently of what US Military has said) that the weapons found in Iraq actually came from Iran


Inferences (3)• R3: Islamic Republic News Agency (IRNA) has

reported that Austrian made weapons were found with Iraqi Insurgents– Could not confirm the claim that Iran delivered the

weapons– It is possible that some background knowledge in the

(huge) knowledge repository of the system could allow to infer that the weapons actually came directly from Austria (?)

Intelligence: Another example


Examples of Reports(modified a bit)

• Report 1: Outside the Terminal boundary fence an employee witnessed a woman photographing terminal facility through the boundary fence from the public access area. Employee reported to security. Security approached woman who left scene in a maroon coloured car unable to ascertain registration no. Woman was 20-40 years old wearing blue coloured shorts and black short sleeve top. No other identification.

• Report 2: Re: vehicle at A gate. When approached, stated only looking & left immediately. 1 female approx 25- 35 y.o shoulder length brown/blonde hair, thin & average in height, wearing dark clothes. All is in order. Vehicle Description: Maroon 4 door hatch. Unknown badge. P plates displayed. Rego XXX.

• Report 3: Port Security Control Centre advised of a female of Asian appearance at the Bridge. She was looking around the area with a set of binoculars. The woman then crossed the bridge by foot. Guard pulled over and approached her to ascertain her business in the area. She informed Guard that she was a bird watcher. The woman was advised that she should not be crossing the bridge by foot. The person left without any further incident.

Source: GUIDANCE PAPER FOR REPORTING OF SECURITY INCIDENTS AND EVENTS BY MARITIME INDUSTRY PARTICIPANTS Office of Transport Security, Australian Government. January 2009.


What can be inferred?• Is the person the same in all reports?• Is the car the same?• Why not? Why yes?• If not, are these events linked? By what relations?• Are all events within the same time interval?• Is gate A part of Terminal?• Is 20 – 40 compatible with 25 – 35?• Is bird watching an illegal/suspicious activity?• What purpose can binoculars be used for?• Is “blue coloured shorts” and “black short sleeve top” same as “dark

clothes”?• If two reports are about the same person/event, how can information be

integrated (“added”)?• Many other questions …


Three kinds of inference• Meta-inference

– Alignment of concepts (“fence” vs. “boarder”, “approach” vs. “meet”)

– Alignment of individuals (“4 door hatch” vs. “maroon car”)• Integration (of separate reports into one; could be “virtually

one”)• Inferring implicit facts (materialization)

– Door is part of car; color of car in Report 1 is maroon, color of car in Report 2 is maroon, …


Conclusion• Tried to show that context can modify meaning• Mentioned approaches to formalize context• Tried to support use of Situation Theory to model use of

context• Outlined Situation Theory and Situation Theory Ontology• Indicated issues related to merging context with a set of given

facts• Suggested the merging can be done using category theory

(colimit)• Showed examples of inferences that a computer system could

possibly perform in order to support human analysts


Backup Slides



OWL Inference Engines

• Pellet http://www.mindswap.org/2003/pellet/• BaseVISor http://www.vistology.com (alpha testers welcome)• KAON2 http://kaon2.semanticweb.org/ • SNARK http://www.ai.sri.com/snark/tutorial/tutorial.html• JTP Java Theorem Prover• Racer http://www.racer-systems.com/index.phtml• CWM W3C Semantic Web's Closed World Machine• ConsVISor www.vistology.com (consistency checker)…

OWL+Rules Reasoners• JESS (Java) <- CLIPS (C++)<- OPS-5 (Lisp), rules4J,

Drools, etc…• general implementations of Rete Algorithm

• Jena• open source RDF/OWL parser/reasoners

• OWLIM/TRREE and BaseVISor • Rete Algorithm optimized for RDF triples

• Oracle OWL Prime • Forward chaining through iterative SQL queries• Supports OWL 2 RL

• Ivan Herman’s OWLRL• “proof of concept implementation” of OWL 2 RL reasoner• http://www.ivan-herman.net/Misc/2008/owlrl/


Documents

Formalization of Context and Situation Awareness Mitch Kokar [email protected]