Integrating Multi-Media with Geographical Information in the

BORG Architecture

R. George

Department of Computer Science

Clark Atlanta University

Atlanta, GA

Outline

• The BORG Architecture– Where We Fit In

• Spatio-Temporal Query Models• Applications

– The Weather Information and Tactical Support (WITS) System

– Integrating Multimedia with GIS

• Conclusion

Battlefield Organic Robotic Grid

• Ubiquitous Knowledge Environment for the Battlefield

• Built upon an ad-hoc computing environment configured as a single computing resource

BORG Experimental Architecture & Platform

IMPACT / PASTA / P2P

Java/C++ Sequal Server/Oracle UML Cognitive Models Sun JXTA

B3A

Imagery DB

Soldier

HighlightsJCDBGIS...Requirements,

CognitiveDeficiencies

Autonomous Fusionand NavigationDistinct Sources

Data Streams

Content Based MessagingSystem

RSSRSSRSS...

Advanced Interactions

Story

Visualizations

Commander/Analysts

Tactical OperationsCenter Workstations

Command and Control

Interactive Fusion andNavigationSummary of Source

Website

Plans

WWWHeterogeneousData sources

KnowledgeEngineering,

Cognitive Models,HCI

Data Related Issues

• Heterogeneous Data Sources

• Multiple Data Types

• Flexible Querying with Levels of Confidence

• Need for Semantic Queries

• Performance

Research Objectives

• Spatial, Temporal Dimensions Features of Data– Characterization of Spatio-Temporal Query Types

• Applications: – Support for Operational Planning

• Long term weather patterns• Detect weather anomalies not predicted by forecast models• Integration with External Knowledge Bases

– Situational Awareness in an Urban Environment• Integration of Heterogeneous Databases including Multi-Media

• Approach: Fuzzy Logic provides a expressive query mechanism for the spatial and temporal domains

Spatial Properties

• Minimum Bounding Rectangle basis for Query– The location of an object Ai is defined by the rectangular area (a

region) [(Xi, Yi), (Xj, Yj)] where Xi Xj, Yi Yj.

– The spatial property of an object (of interest) A is a tuple (R, I), where, R is a rectangular area (a region) which covers a minimal area in which object A appears during the time interval I = [ti, tf]. R is an approximation of a MBR.

– A static or moving in the rectangle R during the interval I. • When A is spatially static, R is the minimum bounding rectangle of A.

Relationships between Objects

• Defined using the Spatial Relationship between MBRs.

• Extension of Allens Temporal Interval Algebra [Li et al]

Relationships between Objects

Relation DefinitionA BOTTOM B Ax {b, bi, m, mi, o, oi, d, di, s, si, f, fi, e} Bx Ay {b, m} By

A TOP B Ax {b, bi, m, mi, o, oi, d, di, s, si, f, fi, e} Bx Ay {bi, mi} By

A LEFT B Ax {b, m} Bx Ay { b, bi, m, mi, o, oi, d, di, s, si, f, fi, e} By

A RIGHT B Ax {bi, mi} Bx Ay { b, bi, m, mi, o, oi, d, di, s, si, f, fi, e} By

A TOP-LEFT B (Ax {b,m} Bx Ay {bi, mi, oi} By ) (Ax {o} Bx Ay {bi,mi} By)

A TOP-RIGHT B (Ax {bi, mi} Bx Ay {bi, mi, oi} By ) (Ax {oi} Bx Ay { bi,mi }By)

A BOTTOM-LEFT B (Ax {b,m} Bx Ay {b,m,o} By ) (Ax {o} Bx Ay {b,m} By)

A BOTTOM-RIGHT B (Ax {b,m} Bx Ay {b,m,o} By ) (Ax {oi} Bx Ay {b,m} By)

A OVERLAPS B Ax {d, di, s, si, f, fi, o, oi, e} Bx Ay { d, di, s, si, f, fi, o, oi, e } By

A EQUAL B Ax {e} Bx Ay {e} By

A INSIDE B Ax {d} Bx Ay {d} By

A CONTAIN B Ax {di} Bx Ay {di} By

A TOUCH B (Ax {m, mi} Bx Ay {d, di, s, si, f, fi, o, oi, m, mi, e} By) (Ax {d, di, s, si, f, fi, o, oi, m, mi, e} Bx Ay {m, mi } By)

A DISJOINT B Ax {b, bi} Bx Ay {b, bi} By

Relationship between Objects

• The fuzzy spatio-temporal relationship during time interval, I is Ai (, , I) Aj is a relation; is the value of membership

and Ai ( )Aj is true during the interval I.– Ex: takes values of WEST, NORTH,

NORTH-WEST

Query Model

• Query is performed on regions within the underlying data structure that form a Minimum Bounding Rectangle.

• Query Model supports queries in the spatial and temporal dimensions – Spatial Search: S, retrieves all regions, Rj, whose area is equal to that of

the user selected MBR, Ri, and field values are similar to those of Rj at Time, Tk

S(MBR) : {Rj | Equal(Ri, R) Λ Disjoint(Ri, Rj) Λ (, ≥ ε)}Tk

– Temporal search :, retrieves region, R, within a time interval, (T0, Tn), to retrieve all instances in which the attribute values are similar. The spatial domain is constant in this operation.

(MBR) = {Tj, R | Equal(Ri, R) Λ (, ≥ ε)} T0-Tn

– Spatio-Temporal Search: ST, identifies similar regions in space and time. The MBRs adjacent to the original are examined to track the weather in the spatial domain. The coordinates of the adjacent MBRs are computed as in TableST(MDC) = {Tj, Rj | Similar(Ri, R) Λ Adj(Ri, Rj) Λ (, ≥ ε)} T0-Tn

Application: The Weather Information and Tactical Support System (WITS)

• Objective: Development of a Weather Data Repository for Operational Planning– Need to know long term weather conditions– Detect weather anomalies not predicted by forecast models– Integration with External Knowledge Bases

• Development of an OLAP Weather Repository– Sources: Georgia Weather (1981-2002)

• US National Weather Service, Georgia Environmental Network, ASOS• Modular Development of WITS

– Ad-hoc Querying (IQ)– Real time Analysis and Planning (TAPS)

• Effects on Operational Systems• Integration with External Knowledge Bases

– Data Mining (DM)

WITS System DesignUSER

INTERFACE

text

text

text

text

DATA WAREHOUSE

DATAMINING

MODULES

QUERYMODULES

KNOWLEDGEBASES

(IWEDA)

DATA CLEANING& TRANSFORMATION

DATAACQUISITION AGENTS

REAL TIME MODULE

TAPS MODULE

IQ MODULE

WITS: Information Query (IQ)

• Module for Spatio-Temporal Ad-Hoc Querying – GUI Driven

– Drill Down and Roll Up Capabilities

WITS/IQ: Drill Down

WITS: Tactical Analysis and Planning (TAPS)

• Integration with External Knowledge Bases– Understand Weather

Effects on Systems

– Logistical/Route Planning

WITS: Data Mining (DM)

• OLAP and Data Mining Module to show trends and artifacts in the data

• Detect local weather anomalies not predicted by weather forecasts

• Example: Trend Analysis of Winter weather in Georgia

WITS/DM

Application: Situational Awareness in an Urban Environment

• Heterogeneous Data Sources– Regional Planning, Census, Crime Statistics– Traffic Cams, Overflights– Weather Data (Real-Time, Historical)– Geographical Information

• Maps with Attributes

• Challenges– Multiple Formats, Scales– Co-ordinates – Missing Data

Conflation

• Conflation: integration of data from various data sources into digital maps

• Multiple steps– Feature Matching– Positional Re-alignment – Attribute Deconfliction

• Query Support

Web services

WEBSERVER

MAP WEBSERVICE

VIDEO DATASERVICE

DATASERVICES

AGENTS

Agent-Based Conflation

VideoGIS System

VideoGIS System: Crime Data

VideoGIS System: Video Tracking 1

VideoGIS System: Video Tracking 2

VideoGIS System: Query Results

Conclusions/Future Work

• Integration of Spatio-Temporal Data, with Multi-Media is challenging

• Several areas of theoretical development of in – Basic OLAP operations (approximations rollup, drill

down)

– Query Models

• Practical Applications in Earth Science and Scientific Data Management