VOYAGER Data Explorer: Architecture and Technologies See also the the Voyager Developer Website and...

VOYAGER Data Explorer:Architecture and Technologies

See also the the Voyager Developer Website and early Applications

Layered Map

Time Chart

Providers Users

Vector

GIS Data

XDim DataSQL Tables

WebImages

Voyager Web Services

Publish, Find, Bind

Data & Tool Catalog

Uniform Access

Scatter Chart

S u p p o r tCoord./Cooperation T e c h n o l o g i e s

Select, Overlay, Explore; Multidimensional data

Maintain Distributed Data; Heterogeneous coding , access

Connect providers to users; Homogenize data access

Voyager Spatio-Temporal Data Explorer Built and Used by a Virtual Community

Layered Map

Time Chart

Providers Users

Vector

GIS Data

XDim Data

SQL Table

Web

Images

Voyager Web Services

Publish, Find, Bind

Data & Tool Catalog

Uniform Access/Retrieval Scatter Chart

The Dvoy Project

DVOY is a graphic browser for distributed, heterogeneous, multidimensional datasets

The initial Dvoy infrastructure was developed at CAPITA, with NSF support

Further services for data access, processing and viewing are expected from the community

The project evolution is to ride 'web services wave‘ of the Internet

CAPITA Support: – NSF ITR Workgroup Collab. Tool: Aug 2001 - Aug 2004

– EPA Web-based Visibility: Aug 2001 - Apr 2003

– NOAA ASOS Visibility: Aug 2001 - May 2002

– MARAMA Chemical Trajectory Tool: Aug 2002 - July 2003

– EPA OAQPS Global Transport Analysis: Nov 2002 – Oct 2003

In-kind support by organizations participating in DVOY-based data sharing

Dvoy Data Model

Multi-Dimensional Data Model

Data can be distributed

over 1,2, …n dimensions

1 Dimensional e.g. Time dimension

ij

k

ji

Data Granule

i

1 Dimensional

e.g. Location & Time

1 Dimensional e.g. Location, Time &

Parameter

View 1

Data Space

View 2

Views are orthogonal

slices through data space

4 D Geo-Environmental Data Cube (X, Y, Z, T)

Environmental data represent measurements in the physical world which has space (X, Y, Z) and time (T) as its dimensions.

The inherent dimensions for geo-environmental data are: Longitude X, Latitude Y, Elevation Z and Time T.

Semi-Static Views (slices) through 4D Data Space

• Possible Cross-sections through the 4 D Data space

- data point..Temperature (xi, yi, zi, ti)

- image Temperature (xrange , yrange, zrange, trange)

XY MAP: Z,T fixed

Vertical Profile:XYT fixed

Time Chart: X,Y,Z fixed

Vertical Cross sect: YT fixed Vertical Cross sect: XT fixed

Vertical Profile Trend:

X,Y fixed

Web Services

Web Services Components and Actions

• Service providers publish services to a service broker.

• Service users find the needed service and get access key from a service broker• With the access key, users bind to the service provider• The result is a dynamic binding mechanism between the service users and providers

Service Broker

Service Provider

PublishFind

BindServiceUser

Components: Provider – User – Broker

Actions: Publish – Find - Bind

Interoperability through a Layered Protocol Stack

• Web Services are implemented on a layered stack of technologies and standards

• The lower layers enable binding and exchange of messages; higher levels enable interoperability

• Applications are formed dynamically from distributed components through publish-find-bind mechanisms

TCP/IP, HTTP, FTP

ASCII, XML, etc.

HTML, XML OGC -GML

OGC Coverage, CoordTransfom, WMS

HTTP, SOAP

WSDL

UDDI OGC Catalog

WSFL, XLANG

StandardsInteroperability

Comm. Protocols

Data Encoding

Data Schema

Data Binding

Web Service

Service Integr.

Service Discovery

Service Descript.

Connectivity

Web PublishHTTP, FTP

Data Access though a Web Service Adapter

Service Cataloger

Service Consumer

Ordinary web content can be delivered as a Web Service through a Proxy Server.

• The Wrapper Service converts HTTP/FTP service to XML Web service

• The Wrapper Service is published as a web service to the Broker

• The User finds the service from the broker and accesses the Wrapper to use the service

Service Wrapper

HTTP/FTP

ServiceService

User Chain

FindUDDI, WSDL

PublishUDDI, WSDL

AccessSOAP, XML

Web Application: Chained Web Services

• A Web Service Provider may also be a User of other services • Multiple web services can be chained into an interactive workflow system • The result is an agile application that can be created ‘just in time’ by the user for a specific need

Service Broker

Service Provider/User

PublishFind

BindServiceUser

Chain

Service Provider

Bind

Chain

Features of the DVOY XML Web Service Architecture

Interoperability: Platform and language independence; based on Web Services (XML,SVG)

Legacy Support: Encapsulating existing data and exposing them as Web Services. (Access to standard HTTP/FTP servers)

Just-in-time integration: Discovery, access to and ad-hoc chaining of services.

(Future: Agile app building)

DVOY Web Services

DVOY draws on three basic web services:

DataCatalogWrapping service is for data registration, finding and wrapping information

DataAccess service provides uniform access to heterogeneous, distributed, multidim. data

DataPortrayal service prepares input data in form suitable for rendering

Applications….

• Voyager Data Browser

Data Catalog

• All the data in the system are to be distributed on the Web and maintained by their custodians

• The purpose of the catalog is to help finding and and accessing the data

• Catalog would be limited to data that can be accessed/merged in DVOY

Catalog of Multidimensional Datasets

• Designed to Publish, Find(Select), Bind(Access) and Render distributed datasets• Publishing is though an open web interface for user/broker registration of datasets• Finding a dataset is aided by a metadata on Provider and Dataset• Binding (data access) information is contained in the Dimensional Tables• Rendering parameters are also contained in the Dimensional Tables

Geo-referenced Map Image: Image IndexedVariable Time , Var GeoRect, Var Sensor

SatGeoImg

CapImgID Universal ID

ProviderURL URL

SourceURL URL

ImageURL URL

ProviderAbbrev NASA GSFC

ImageDescription Text

Sensor Enumerated

DateTime DateTime

LatMin -65

LatMax 65

LonMin - 180

LonMax 180

ImageWidth For size verification

ImageHeight

MarginLeft For trimming

MarginTop

MarginRight

MarginBottom

SQL Table

<measure code=“SeaWiFS01" name=“SeaWiFS – Reflectance Images - Glob" descr=" SeaWiFS – Reflectance Images - Global">

<data_source service_url="http://capita.wustl.edu/dataservice/data.asmx"/>

<data_facts>< fact_attributes >

<data name ="SeaWiFS" type="map image" descript=“stuff stuff “/>

<granul_query handler_assembly= (access handler)…./>

<fact table, SQL same as CIRA >

</ fact_attributes > … see table left

</ data_facts>

</measure>

NO DIMENSION TABLES

XML Example: SeaWiFS

Geo-referenced Map Image: Time Indexed

Regular Time Series; Fixed Georect; Image Trim

<measure code="TOMS01" name="TOMS - Aerosol Index - Glob" descr="Earth probe TOMS aerosol index">

<data_source service_url="http://capita.wustl.edu/dataservice/data.asmx"/>

<data_facts>< fact_attributes >

<data name ="TOMS" type="map image" descript="/>

<granul_query handler_assembly= (access handler)…./>

<image width=“640" height=“480“ margin_left="69" margin_top="46"/>

< georect lat_ min="-65" lat_ max="65“ lon_min="-180“ lon_max="180“ unit ="DecDeg“ />

</ fact_attributes >

</ data_facts>

<data_dimensions>

<time_dimension

handler_assembly="CAPITA.data.index.impl"

handler_class="CAPITA.data.index.impl.datetime_interval"

start_time="1996-07-25"

end_time="yesterday">

sample_periodicity _unit = "day"

sample_periodicity_mult = “1"

</time_dimension>

</data_dimensions>

</measure>

TOMSFact

TimeIndex Int, Calc from date

DailyImage Img, Parsed from date

LatMin Fixed, -65

LatMax Fixed, 65

LonMin Fixed, - 180

LonMax Fixed, 180

TOMSTime

TimeIndex

DateTime

Virtual Tables (No physical StarTables) XML Example: TOMS

ftp://jwocky.gsfc.nasa.gov/pub/eptoms/images/aerosol/y2002/ea020510.gif

Astronaut Photos: Image IndexedVariable Time, Image Center, Platform SatImg

NASAImageID Universal ID

ProviderURL http://eol.jsc.nasa.gov/

SourceURL http://eol.jsc.nasa.gov/scripts/sseop/photo.pl?mission=ISS004&roll=E&frame=11079

ImageURL http://eol.jsc.nasa.gov/sseop/images/ESC/small/ISS004/ISS004-E-11079.JPG

ProviderAbbrev NASA JSC

ImageDescription Text

Platform ISS004

GeoRegion GUATEMALA

Features SMOKE, GULF OF HONDURAS

DateTime 20020501 212700

Lat 17.6

Lat -90.2

SQL Table

<measure code=“AstroPhoto01" name=“Astronaut Photographs " descr=" Astronaut Photographs ">

<data_source service_url="http://capita.wustl.edu/dataservice/data.asmx"

/>

<data_facts>< fact_attributes >

<data name ="SeaWiFS" type="map image" descript=“stuff stuff “/>

<granul_query handler_assembly= (access handler)…./>

<fact table, SQL same as CIRA >

</ fact_attributes > … see table left

</ data_facts>

</measure>

XML Example: ISS004

Atmospheric Photos: Image IndexedVariable Time, Image Center, PlatformAtmoImg

ImageID Universal ID

ProviderURL http://capita.wustl.edu/capita/people/RHusar/rhusar.html

SourceURL http://capita.wustl.edu/capita/people/RHusar/Pic/9812SanFranciscoAGU/9812SanFranciscoAGU.htm

ImageURL http://capita.wustl.edu/capita/people/RHusar/Pic/9812SanFranciscoAGU/Dcp01000Enh.jpg

Provider Abbrev R Husar

ImageDescription Text

Platform Airplane

GeoRegion NEVADA

Features DUST PLUME

DateTime 19981208 1800

Lat 36.75

Lat -115.15

SQL Table

<measure code=“AtmosPhoto01" name=“Atmospheric Photographs " descr=" Atmospheric Photographs ">

<data_source service_url="http://capita.wustl.edu/dataservice/data.asmx"

/>

<data_facts>< fact_attributes >

<data name ="SeaWiFS" type="map image" descript=“stuff stuff “/>

<granul_query handler_assembly= (access handler)…./>

<fact table, SQL same as CIRA >

</ fact_attributes > … see table left

</ data_facts>

</measure>

XML Example: Dust Plume Photo

DVOY State Maintenance 

This DVoyState data structure contains the state variables, which drive DVoy during browsing

It is created by the DataCatalogWarpping service using the Catalog and Wrapping information

User actions on Controllers modify the DVoyState structure directly but do not execute changes

Actions are executed by managers (e.g. ConMan for connections) based on changes in DVoyState

DVoyState resides on the client web page, hence it is unique for each Dvoy session

The session state can be saved as a web page and can be recreated at load time

Workbooks can be created from multiple pages of DVoyState (not yet implemented)

Add New Layer (New): Populate DVoyState.xml

WSGetDatasetListDatasetList.xml DVoyState.xml

WSGetDatasetInfo

DatasetInfo.xml

DatasetSelector

DVoy Catalog Database

On LoadCatalogPage On AddLayer

Init DVoyState Add layer/dataset_abbr to DVoyState Add Layer Description to DVoyState

DVoyState.xml

Event Handling in DVoy

Applications - Examples

Quebec Fires, July 6, 2002

SeaWiFS, METAR and TOMS Index superimposed

SeaWiFS satellite and

METAR surface haze shown in the Voyager distributed data

browser

Satellite data are fetched from NASA GSFC; surface data

from NWS/CAPITA servers

Trans-Atlantic Transport of Quebec Smoke

July 11: Smoke approaching Europe

July 10: Quebec smoke over Mid-Atlantic

SeaWiFS Reflectance

TOMS Absorbing Aerosol

SeaWiFS Reflectance

TOMS Absorbing Aerosol

Spain

Spain

E. US

Web Service Wrapper for TOMS Aerosol Index Fixed Geo-rectangle; Image Trim, daily

TOMS Image Metadata

Geo-rectangle (65, -180; -65, 180)

Image Size (640, 480)

Image Margins (40, 40, 30,30)

Transparent Colors (0,0,255 )

Image Access Metadata

URL template: :ftp://jwocky.gsfc.nasa.gov/pub/eptoms/images/aerosol/YYYYY/eaYYMMDD.gif

StartDate, EndDate

Browsing of Distributed Data from HTTP/FTP Servers

Land Reflectance from SeaWiFS Project, NASA GSFC

AVHRR Oceanic Aerosol

CAPITA, WashU, STL

Fire Pixels, Jan 1997, ESA Ionia Project

Voyager Spatio-Temporal Data Browser

NRL Forecast Model for Dust, Smoke and SulfateMETAR Surface Haze

• Real-time model and surface observations are compared spatially and temporally

Dust

Sulfate

Smoke

METAR HazeTime Selector