11:00 Self-Introductions11:15 Report on ontology-based data integration work in DCGS-A--- Goals and methodology--- Practical experience and results so far--- Risks and problems12:15 Lunch (Brown Bag Lunch?)13:00 Discussion of NSA work and goals (to be expanded)--- ontology alignment / management--- OMaaS (Object Management as a Service)14:00-15:30 Discussion of how collaborative ontology effort (e.g. between I2WD and NSA) will work in practice--- how to ensure consistency (architectural and content)--- how to address governance--- business development and work flow
DSC Ontology Work
1. Goals and methodology2. Practical experience and results so far3. Challenges
02/05/13
Goal: To realize Horizontal Integration(HI) of intelligence data
HI =Def. the ability to exploit multiple data sources as if they are one Problem: the data coming onstream are out of our
control Any strategy for HI must be agile in the sense that
it can be quickly extended to new zones of emerging data according to need
Ontology can provide the needed agility and (incremental approach to) comprehensiveness
The Business Case• Huge resources are wasted as multiple different agencies
create lexicons, glossaries, data models, messaging and exchange standards with the same or closely overlapping coverage.
• Additional resources are wasted in creating mappings between these artifacts, and in maintaining them in light of new needs and challenges. These mappings always fail.
• A sensible solution must incorporate an evolutionary process which will ensure that artifacts used to manage data converge over time.
Methodology: Create an agile strategy for building ontologies within a Shared Semantic Resource (SSR) and apply and extend these ontologies
to annotate new source data as they come onstream
⁻ Strategy pioneered in biomedical and other scientific fields: leaves data as they are, and incrementally tags data sources with terms from a growing, consistent, non-redundant set of ontologies
⁻ Problem: Given the immense and growing variety of data sources, the development methodology must be applied by multiple different groups
⁻ How to manage collaboration? This is what this meeting is about
Current State2010-2011
– Architectural implementation (DRIF) to create the Dataspace (a cloud of intelligence data) = lossless representation of sources with their native semantics
– Initiated Semantic Enhancement (SE) = using ontologies to annotate these native semantics
– Demonstration of use of SE to index and query the content of the Dataspace 2012
– Created methodology and architecture for ontology development– Initiated Shared Semantic Resource (SSR); created suite of prototype ontologies
enabling SE also outside the Dataspace– CUBRC: demonstrated ability to leverage SE for analytics – Priming potential users of SE (in DoD CIO, NGA, JIEDDO, TRADOC …)
2013-– Milportal– Event Reporting Application– Initial negotiations with other agencies and groups
Ontologies We Have (Feb. 2013)• Physical Artifact, including Infrastructure, Facility,
Vehicle, Weapon
• Information Artifact, including Report, Image, Map
• Event, including Military, Criminal, Economic, Political, Religious, Social
• Human Physical Characteristics• Agent, including Person, Organization, Social Network
• Geospatial • Time
MilPortalhttp://milportal.ncor.buffalo.edu
slide from Margaret Storey
Next step
slide from Margaret Storey
Work plans 1: For DSC Cloud (on-going)
• Perform needs analysis: Review DCGS-A Logical Data Model and schemas of other DSCG-A data sets; in each case, examine content and establish what terms are needed to ensure sufficient coverage for SE;
• Where these terms already exist within the SSR, check that definitions exist and that these definitions are adequate
• Where the terms do not exist within the SSR, create new terms (or new ontologies), with appropriate definitions as necessary to fill gaps.
• Use the terms in 2. and 3. to annotate the corresponding entries in the data models to effect horizontal integration;
• With each new expansion in scope of DSGS-A data sets, iterate the above as needed.
• In addition, we are engaging in documentation of the methodology as here described, and in dissemination and training.
Work plans 2: For interagency collaboration
Step 1: Initiating interagency collaboration in the service of horizontal integration of intelligence dataIdentify candidate teams / agenciesEstablish collaboration with one or more specific teams
– Formulate and ratify agreements as appropriate– Create work plan and identify funding needs– Perform risk assessment
Step 2: Establishment of the inter-agency ontology development process Examples of types of work to be performed would include: • Create governance infrastructure• Establish needed technological support• Implement workflow• Conduct training in methodology where needed • Explore opportunities for inter-agency HI of data • Begin application to relevant agency data models of the SE strategy• Dissemination of results in a form which will allow improved systems to perform
enhanced analytics exploiting semantic interoperability.
The SSR methodology and governance is neutral as to how SSR-annotations are used• Currently SE of DSC enhancement only integration
through improved indexing/search capability• On CUBRC project, we have much more, including
ontology-based reasoning. • In the future we will have for DSC enhancement
applied – to multiple models such as LDM– to unstructured text
• see the various methodology documents provided so far
Event Reporting Application (EvR)
• Aggregates content from various report generating applications (CPOF, TIGR, TIGR with MAPHT extension)
• The underlying data model contains nearly 500 terms (e.g. ReportName, EventName, DeclassificationEvent)
• The semantics of the data model seems similar to that of a relational model
EvR Data Model as Relational Database• Hierarchies of events and units are referenced by EventType and UnitEchelon
columns, but this alone provides no capability for traversing these hierarchies• A report is related to both a report name and a report unit by the inclusion of
appropriately named columns, but what is the difference between these relationships?
• Our approach shows how to express the difference, e.g. between report-to-event and event-to unit relationships
Report
ReportId
ReportName
ReportUnit
ContainedEventIds
…
Event
ID
UnitIdentificationCode
Location
EventType
…
UnitInformation
UIC
UnitName
UnitAffiliation
UnitEchelon
…
Enhancing the vocabulary of the EvR Data Model
• Semantic enhancement (SE) amounts to associating a database field to a whole knowledge system enabling machines to process data … – “vertically” e.g. by traversing the echelon command hierarchy
and – “horizontally” e.g. by following specified relations between units
and events. • SE separates semantics from structure, reducing
maintenance costs as source databases no longer have to be modified each time we improve our understanding of reality
• The ontology tags move with the data …
Enhancing the vocabulary of the EvR Data Model
Progress to Date• Two techniques of SE are available
– Partial enhancement (now being used in the DSC) associating ontology label terms with EvR terms
– Full enhancement (not yet implemented in the DSC) aligning terms from the EvR to assertions using terms and relations from SSR ontologies
• At present the current ontologies… – provide 70% coverage for partial enhancement– provide 38% coverage for full enhancement– plan for extending the ontologies to raise the coverage for the EvR
to 90% and 60% respectively by end of February
Enhancing the vocabulary of the EvR Data Model
Event Reporting Term
Partial Enhancement via Ontology Label
Full Enhancement via Ontology Assertion(s)
Enhanced EvR Data as a Graph
Act of Reporting
Event
EventID
Report
ReportName
Report ID
Event Type
Unit Identification
Code
Unit
Command Unit
Service
designatespa
rt_of
describes
designates participates_in
agent_in
contro
ls
affiliated_w
ith
has_output
is_subcla
ss_of
UnitName
Report Unit
designates
designates
designates
shows how full annotation provides the semantics missing from the EvR relational model described above
Act of Reporting
Event
EventID
Report
ReportName
Report ID
Event Type
Unit Identification
Code
Unit
Command Unit
Service
designatespa
rt_of
describes
designates participates_in
agent_in
contro
ls
affiliated_w
ith
has_output
is_subcla
ss_of
UnitName
Report Unit
designates
designates
designates
relationships between • event type and echelon • report name and reporting unit
are made distinct and machine processable
Challenges • Challenges to Horizontal Integration in general
– Too many lexicons – The scope of the domain: signal, sensor, image, … intelligence
about the whole world• Our solution
– Incremental extraction and sanitization, and creation of content– Distributed collaborative development – Strong methodology
• Challenges for our solution– Governance and management of ontology development to
ensure consistent evolution– Lack of expertise
• For ontology development and management• For annotation
– Success will breed failure
We have these precursor ontologies
• The prototype ontologies in the existing SE (Malyuta-Salmen) helped indexing
• The ontologies we now have are much better and more than these prototypes
• How should we implement them re Event Reporting Ontology, global graph, etc. …?
• Next steps with I2WD
Our ideas are being heard
• 2 classes of collaborator: observers and partners
• companies such as DataTactics are realizing that in case of success this provides huge potential business benefits.
• (DT invited our team to talk at the Ontology Summit they have called on Feb. 12, devoted to the development of shared semantics)
Peter Morosoff and Bill Mandrick
• How to create a strategy?