View
224
Download
3
Tags:
Embed Size (px)
Citation preview
State of the art State of the art Technologies for GIS Technologies for GIS
Management SystemsManagement Systems
Implementation through EuropeanCommunity Research Projects
Hellenic Center for Marine Research
Lykiardopoulos Angelos, Iona Athanasia, Lakes Vasilis, Balopoulos Efstathios, Kalkavouras Constantine
HCMR / Hellenic National Oceanographic Data Center
Sections of this presentation
• Data management and Web Services
• Quality control in HNODC data base
• Poseidon system, a short description
HCMR / Hellenic National Oceanographic Data Center
Data Management and Web Services
a Historical Brief
At the end of 80’s and early 90’s several Oceanographic Data centers had created Databases with mass storage of spatial Information which concerned Oceanographic research outcomes.
This was en enormous step of digital storage because big data volumes organized in Database Systems.
PASTPAST
Historical Brief
Centralized D.M.
From Centralized to Distributed Model
From Non Spatial to SpatialPRESENTPRESENT
Web Services
W.S. Architecture
Geospatial W.S.
WMS/WFS
WPS
HNODC General
HNODC Architecture
HNODC Interface
HNODC W.S.FUTUREFUTURE
The new architecture
An example
Conclusions
HCMR / Hellenic National Oceanographic Data Center
In other words the data centers owned databases – Based on various RDBMS mechanisms– With their own database Schemas– With their own data vocabulary (if existed)– With raw data in various formats– With Applied various processes of data quality control (or
with no data quality control)
This effort leaded into the creation of several data bases without any Interoperability among them.
Data Management Historical Brief
PASTPAST
Historical Brief
Centralized D.M.
From Centralized to Distributed Model
From Non Spatial to SpatialPRESENTPRESENT
Web Services
W.S. Architecture
Geospatial W.S.
WMS/WFS
WPS
HNODC General
HNODC Architecture
HNODC Interface
HNODC W.S.FUTUREFUTURE
The new architecture
An example
Conclusions
HCMR / Hellenic National Oceanographic Data Center
The centralized Data Management Idea…
But the oceanographic community had the need of homogenized and comprehensive data sets for various regions and even world wide.
Due to the limitations of information technologies this goal was achieved in the framework of several E.U. projects– Central accumulation of data from various Data Centers
took place– Data sets were processed in common data formats and
more or less same Q.C. procedures. – Finally the dissemination of these Data sets was carried
out mainly via digital media (CD-ROMs, etc).
PASTPAST
Historical Brief
Centralized D.M.
From Centralized to Distributed Model
From Non Spatial to SpatialPRESENTPRESENT
Web Services
W.S. Architecture
Geospatial W.S.
WMS/WFS
WPS
HNODC General
HNODC Architecture
HNODC Interface
HNODC W.S.FUTUREFUTURE
The new architecture
An example
Conclusions
HCMR / Hellenic National Oceanographic Data Center
The centralized Data Management Idea…
During this process: Diverse Data form a wide range of Data Centers
were gathered. Passed common homogenization and QC
procedures. And finally some comprehensive Data sets were
produced and disseminated.But… The produced Data Sets were stand alone
products stored in digital media. Without any dynamic mechanism of maintenance
and enhancement. And without any online access.
PASTPAST
Historical Brief
Centralized D.M.
From Centralized to Distributed Model
From Non Spatial to SpatialPRESENTPRESENT
Web Services
W.S. Architecture
Geospatial W.S.
WMS/WFS
WPS
HNODC General
HNODC Architecture
HNODC Interface
HNODC W.S.FUTUREFUTURE
The new architecture
An example
Conclusions
HCMR / Hellenic National Oceanographic Data Center
From Centralized to Distributed model
Thus, the idea of a distributed Oceanographic Data network among the Data Centers start growing up.
And since Web technologies were being developed rapidly, became apparent that the above network had to be web based.
PASTPAST
Historical Brief
Centralized D.M.
From Centralized to Distributed Model
From Non Spatial to SpatialPRESENTPRESENT
Web Services
W.S. Architecture
Geospatial W.S.
WMS/WFS
WPS
HNODC General
HNODC Architecture
HNODC Interface
HNODC W.S.FUTUREFUTURE
The new architecture
An example
Conclusions
HCMR / Hellenic National Oceanographic Data Center
Main obstacles…
The main obstacles that came through for this idea to be fulfilled were:
Data bases was in different Schemas and Technologies specific per Data center so there was not any common data interchange mechanism.
Data were not homogenized among Data centers. Data quality and Data vocabulary differed from
Database to Database
PASTPAST
Historical Brief
Centralized D.M.
From Centralized to Distributed Model
From Non Spatial to SpatialPRESENTPRESENT
Web Services
W.S. Architecture
Geospatial W.S.
WMS/WFS
WPS
HNODC General
HNODC Architecture
HNODC Interface
HNODC W.S.FUTUREFUTURE
The new architecture
An example
Conclusions
HCMR / Hellenic National Oceanographic Data Center
From Non Spatial to Spatial Enabled Databases…
At the same time another obstacle was that due to premature Data Base technology many RDMBS systems lacked spatial capabilities.
So it was very difficult for the end user: To query the Database with spatial criteria To represent geographically the spatial information and
produce geospatial products To work interactive with Geo Data through interfaces with
geospatial capabilities To combine geospatial data in synthetic data products
Hence, for a distributed Oceanographic Data network to be established, the necessity of GIS enabled Databases became apparent as well.
PASTPAST
Historical Brief
Centralized D.M.
From Centralized to Distributed Model
From Non Spatial to SpatialPRESENTPRESENT
Web Services
W.S. Architecture
Geospatial W.S.
WMS/WFS
WPS
HNODC General
HNODC Architecture
HNODC Interface
HNODC W.S.FUTUREFUTURE
The new architecture
An example
Conclusions
HCMR / Hellenic National Oceanographic Data Center
From Non Spatial to Spatial Enabled Databases…
At the end of 90’s most of the common used RDBMS enriched with spatial capabilities and became then powerful mechanisms of integration, exploitation and presentation of spatial information
Worldwide oceanographic data centers, exploit this advantage and started upgrading their databases
PASTPAST
Historical Brief
Centralized D.M.
From Centralized to Distributed Model
From Non Spatial to SpatialPRESENTPRESENT
Web Services
W.S. Architecture
Geospatial W.S.
WMS/WFS
WPS
HNODC General
HNODC Architecture
HNODC Interface
HNODC W.S.FUTUREFUTURE
The new architecture
An example
Conclusions
HCMR / Hellenic National Oceanographic Data Center
Interoperability obstacle is remaining…
But still the luck of interoperability was unsolved. The spatial databases made researchers’ life easier with regard to querying data or representing graphically geospatial information.But…
they were acting without any interconnection among the various Data Centers.
PASTPAST
Historical Brief
Centralized D.M.
From Centralized to Distributed Model
From Non Spatial to SpatialPRESENTPRESENT
Web Services
W.S. Architecture
Geospatial W.S.
WMS/WFS
WPS
HNODC General
HNODC Architecture
HNODC Interface
HNODC W.S.FUTUREFUTURE
The new architecture
An example
Conclusions
HCMR / Hellenic National Oceanographic Data Center
SEA DATA SEA DATA NETNET
SEASEARCSEASEARCHH
MEDAR/MEDAR/MEDATLAS IIMEDATLAS II
MEDATLAMEDATLASS
Summarizing the historical brief throughout important E.U. projects
1994
•Efforts in QC•Data homogenization•Centralized management•Distribution with CD ROMs
1998
•Enrich existing data sets•Establishment of common protocols in QC and formatting•Enhance communication facilities (www, ftp, e-mail etc)
2002
2006
•Development of a pan-european network for ocean and marine data & information management.•Improve the exchange, availability and accessibility of ocean & marine data and information.•Expand the online metadatabases with input of research institutes.•Improve the online accessibility of Sea-Search services & products.
•Develop a standardized distributed system for managing and disseminating the large and diverse data sets•Enhance the currently existing infrastructures with web services
PASTPAST
Historical Brief
Centralized D.M.
From Centralized to Distributed Model
From Non Spatial to SpatialPRESENTPRESENT
Web Services
W.S. Architecture
Geospatial W.S.
WMS/WFS
WPS
HNODC General
HNODC Architecture
HNODC Interface
HNODC W.S.FUTUREFUTURE
The new architecture
An example
Conclusions
HCMR / Hellenic National Oceanographic Data Center
Web Services…
At the same time with the efforts of the Oceanographic community, during the last five years new technologies aroused and became common known as:
Web ServicesWeb Services
PASTPAST
Historical Brief
Centralized D.M.
From Centralized to Distributed Model
From Non Spatial to SpatialPRESENTPRESENT
Web Services
W.S. Architecture
Geospatial W.S.
WMS/WFS
WPS
HNODC General
HNODC Architecture
HNODC Interface
HNODC W.S.FUTUREFUTURE
The new architecture
An example
Conclusions
HCMR / Hellenic National Oceanographic Data Center
Web Services fundamentals
First of all what is a Web Service?
• The hidden idea behind WS is the existence of a universal messaging mechanism, capable to deliver and translate messages for diverse and different systems.
• This feature came to open the road for the solution of interoperability.
• If different and diverse systems can interpret and process requests from each other let them stay different.
PASTPAST
Historical Brief
Centralized D.M.
From Centralized to Distributed Model
From Non Spatial to SpatialPRESENTPRESENT
Web Services
W.S. Architecture
Geospatial W.S.
WMS/WFS
WPS
HNODC General
HNODC Architecture
HNODC Interface
HNODC W.S.FUTUREFUTURE
The new architecture
An example
Conclusions
HCMR / Hellenic National Oceanographic Data Center
There are various theoretical definitions of What Web Service is such as:
“A software system designed to support interoperable machine-to-machine interaction over a network. It has an interface described in a machine - processable format (specifically WSDL). Other systems interact with the Web service in a manner prescribed by its description using SOAP messages, typically conveyed using HTTP with XML serialization in conjunction with other Web-related standards.” (W3C, 2004)
What is a Web Service?
PASTPAST
Historical Brief
Centralized D.M.
From Centralized to Distributed Model
From Non Spatial to SpatialPRESENTPRESENT
Web Services
W.S. Architecture
Geospatial W.S.
WMS/WFS
WPS
HNODC General
HNODC Architecture
HNODC Interface
HNODC W.S.FUTUREFUTURE
The new architecture
An example
Conclusions
HCMR / Hellenic National Oceanographic Data Center
What is a web Service?
or
“Web services are a distributed computing architecture. Only this particular architecture makes use of loosely coupled applications, as opposed to tightly coupled applications, to enable applications to communicate. This tightly coupled concept radically affects how information systems will work in the future”
(Clabby 2003)
PASTPAST
Historical Brief
Centralized D.M.
From Centralized to Distributed Model
From Non Spatial to SpatialPRESENTPRESENT
Web Services
W.S. Architecture
Geospatial W.S.
WMS/WFS
WPS
HNODC General
HNODC Architecture
HNODC Interface
HNODC W.S.FUTUREFUTURE
The new architecture
An example
Conclusions
HCMR / Hellenic National Oceanographic Data Center
What is a web Service?
Generalizing we may define that: “Web services provide a convenient and standardised way of
exposing business logic over a network (the Internet) by use of specific components of communication”
The basic components of WS are SOAP (Simple Object Access Protocol)
• which provides a Standardized messaging structure based on XML
UDDI (Universal Description, Discovery and Integration) • which provides the ability to dynamically select a
service at runtime WSDL (Web Services Description Language)
• which describes how the client communicates with a Web service or what a service does or where is located.
PASTPAST
Historical Brief
Centralized D.M.
From Centralized to Distributed Model
From Non Spatial to SpatialPRESENTPRESENT
Web Services
W.S. Architecture
Geospatial W.S.
WMS/WFS
WPS
HNODC General
HNODC Architecture
HNODC Interface
HNODC W.S.FUTUREFUTURE
The new architecture
An example
Conclusions
HCMR / Hellenic National Oceanographic Data Center
In the most simple form a web service architecture may be depicted as the diagram
Web Service Architecture…PASTPAST
Historical Brief
Centralized D.M.
From Centralized to Distributed Model
From Non Spatial to SpatialPRESENTPRESENT
Web Services
W.S. Architecture
Geospatial W.S.
WMS/WFS
WPS
HNODC General
HNODC Architecture
HNODC Interface
HNODC W.S.FUTUREFUTURE
The new architecture
An example
Conclusions
HCMR / Hellenic National Oceanographic Data Center
Geospatial Web Services
The rapid development of technologies towards WS concept has produced several sets of Web Services with regard to the area of the WS applicability.
Especially for Geospatial data, Open Geographical Consortium (OGC) has defined two basic sets of Geospatial Web Services.
PASTPAST
Historical Brief
Centralized D.M.
From Centralized to Distributed Model
From Non Spatial to SpatialPRESENTPRESENT
Web Services
W.S. Architecture
Geospatial W.S.
WMS/WFS
WPS
HNODC General
HNODC Architecture
HNODC Interface
HNODC W.S.FUTUREFUTURE
The new architecture
An example
Conclusions
HCMR / Hellenic National Oceanographic Data Center
The Web Map Services (WMS) which are concerned with transforming spatial data into maps (images).
The Web Feature Services (WFS) which are concerned with direct access to data - reading, writing, and updating data.
WMS and WFS…
PASTPAST
Historical Brief
Centralized D.M.
From Centralized to Distributed Model
From Non Spatial to SpatialPRESENTPRESENT
Web Services
W.S. Architecture
Geospatial W.S.
WMS/WFS
WPS
HNODC General
HNODC Architecture
HNODC Interface
HNODC W.S.FUTUREFUTURE
The new architecture
An example
Conclusions
HCMR / Hellenic National Oceanographic Data Center
• Furthermore, more Web services are under finalization process, one of the most important of them is WPS (Web Processing Service)
• WPS is designed to standardize the way that GIS calculations are made available to the Internet.
• Can describe any calculation (i.e. process) including all of its inputs and outputs, and trigger its execution as a Web Service.
• Supports simultaneous exposure of processes via GET, POST, and SOAP, thus allowing the client to choose the most appropriate interface mechanism.
Web Processing Services
PASTPAST
Historical Brief
Centralized D.M.
From Centralized to Distributed Model
From Non Spatial to SpatialPRESENTPRESENT
Web Services
W.S. Architecture
Geospatial W.S.
WMS/WFS
WPS
HNODC General
HNODC Architecture
HNODC Interface
HNODC W.S.FUTUREFUTURE
The new architecture
An example
Conclusions
HCMR / Hellenic National Oceanographic Data Center
HNODC…
The HNODC data management case.
PASTPAST
Historical Brief
Centralized D.M.
From Centralized to Distributed Model
From Non Spatial to SpatialPRESENTPRESENT
Web Services
W.S. Architecture
Geospatial W.S.
WMS/WFS
WPS
HNODC General
HNODC Architecture
HNODC Interface
HNODC W.S.FUTUREFUTURE
The new architecture
An example
Conclusions
HCMR / Hellenic National Oceanographic Data Center
Hellenic National Oceanographic Data Center (HNODC) is a National Public Oceanographic Data provider and at the same time a member of the International Net of Oceanographic Data Centers (IOC/IODE)
HNODC owns a very big volume of Data and Relevant information about the Marine Ecosystem
HNODC…PASTPAST
Historical Brief
Centralized D.M.
From Centralized to Distributed Model
From Non Spatial to SpatialPRESENTPRESENT
Web Services
W.S. Architecture
Geospatial W.S.
WMS/WFS
WPS
HNODC General
HNODC Architecture
HNODC Interface
HNODC W.S.FUTUREFUTURE
The new architecture
An example
Conclusions
HCMR / Hellenic National Oceanographic Data Center
For the efficient management and exploitation of these Data, the first step was a relational Data Base to be constructed.
After the necessary stage of data quality control took place, following the International standards, as they agreed during several EU Projects or International Organizations.
Finally a mass volume of over 300.000 station data concerning , physical, chemical and biological Oceanographic information, stored in a Relational Data Base.
HNODC…PASTPAST
Historical Brief
Centralized D.M.
From Centralized to Distributed Model
From Non Spatial to SpatialPRESENTPRESENT
Web Services
W.S. Architecture
Geospatial W.S.
WMS/WFS
WPS
HNODC General
HNODC Architecture
HNODC Interface
HNODC W.S.FUTUREFUTURE
The new architecture
An example
Conclusions
HCMR / Hellenic National Oceanographic Data Center
The next target aroused then, was an Application to be constructed capable to represent either
- the Geospatial aspects of this information - together with the non spatial information
The necessary analysis took place and a traditional Web Application was developed by the use of the convenient web tools and technologies.
Geographical representation was achieved by the use of a Middle Layer tool (ArcIMS) over the Data Base Mechanism.
HNODC…PASTPAST
Historical Brief
Centralized D.M.
From Centralized to Distributed Model
From Non Spatial to SpatialPRESENTPRESENT
Web Services
W.S. Architecture
Geospatial W.S.
WMS/WFS
WPS
HNODC General
HNODC Architecture
HNODC Interface
HNODC W.S.FUTUREFUTURE
The new architecture
An example
Conclusions
HCMR / Hellenic National Oceanographic Data Center
• The convenient Multilayer Architecture was followed – The Relational data base as the back end.– ArcIMS as the Geospatial representation Mechanism.– Apache and Tomcat in heart of the Middle-Tier System.– On the top a web Interface with selection criteria for
querying specific data sets in specific geographical areas.
HNODC - Architecture…PASTPAST
Historical Brief
Centralized D.M.
From Centralized to Distributed Model
From Non Spatial to SpatialPRESENTPRESENT
Web Services
W.S. Architecture
Geospatial W.S.
WMS/WFS
WPS
HNODC General
HNODC Architecture
HNODC Interface
HNODC W.S.FUTUREFUTURE
The new architecture
An example
Conclusions
HCMR / Hellenic National Oceanographic Data Center
Data Base
ArcIMS
TomcatApache
HNODC - Architecture…PASTPAST
Historical Brief
Centralized D.M.
From Centralized to Distributed Model
From Non Spatial to SpatialPRESENTPRESENT
Web Services
W.S. Architecture
Geospatial W.S.
WMS/WFS
WPS
HNODC General
HNODC Architecture
HNODC Interface
HNODC W.S.FUTUREFUTURE
The new architecture
An example
Conclusions
HCMR / Hellenic National Oceanographic Data Center
HNODC – Web page…PASTPAST
Historical Brief
Centralized D.M.
From Centralized to Distributed Model
From Non Spatial to SpatialPRESENTPRESENT
Web Services
W.S. Architecture
Geospatial W.S.
WMS/WFS
WPS
HNODC General
HNODC Architecture
HNODC Interface
HNODC W.S.FUTUREFUTURE
The new architecture
An example
Conclusions
HCMR / Hellenic National Oceanographic Data Center
HNODC…
Although it’s geospatial capabilities this Application stranded only as a data geographical presentation and queering mechanism – without any Data management and Data
processing abilities.– and also without any Data interchanging with other
applications– and with it’s own data vocabulary
PASTPAST
Historical Brief
Centralized D.M.
From Centralized to Distributed Model
From Non Spatial to SpatialPRESENTPRESENT
Web Services
W.S. Architecture
Geospatial W.S.
WMS/WFS
WPS
HNODC General
HNODC Architecture
HNODC Interface
HNODC W.S.FUTUREFUTURE
The new architecture
An example
Conclusions
HCMR / Hellenic National Oceanographic Data Center
• At the same time technical issues aroused, regarding the Application’s performance, due to the inefficiency of the Geographical mechanism to handle big volumes of data sets as fast as a Web Application needs.
• For HNODC this approach is today considered as the fist Version (V1) of an Integrated Web Based Environment for it’s Data Management and Data exploitation.
HNODC…
PASTPAST
Historical Brief
Centralized D.M.
From Centralized to Distributed Model
From Non Spatial to SpatialPRESENTPRESENT
Web Services
W.S. Architecture
Geospatial W.S.
WMS/WFS
WPS
HNODC General
HNODC Architecture
HNODC Interface
HNODC W.S.FUTUREFUTURE
The new architecture
An example
Conclusions
HCMR / Hellenic National Oceanographic Data Center
• Thus and at the time that Web Services became a fact, a second Version was planned
• The target of this effort was an Integrated Web Based environment to be developed, following at least the basic concepts of a Service Oriented Application (SOA) which are:– The interface contract to any Web Service is
platform-independent. – Any Web Service can be dynamically
located and invoked. – Any Web Service invoked is self-contained.
That is, the service maintains its own state.
HNODC…
PASTPAST
Historical Brief
Centralized D.M.
From Centralized to Distributed Model
From Non Spatial to SpatialPRESENTPRESENT
Web Services
W.S. Architecture
Geospatial W.S.
WMS/WFS
WPS
HNODC General
HNODC Architecture
HNODC Interface
HNODC W.S.FUTUREFUTURE
The new architecture
An example
Conclusions
HCMR / Hellenic National Oceanographic Data Center
For the development, state of the art software components and tools replaced the “old” ones:– Geospatial and no Spatial Web Services
mechanisms took the place in the Middle- Layer.– Geospatial open source tools and custom
development was employed in presentation layer.
HNODC…
PASTPAST
Historical Brief
Centralized D.M.
From Centralized to Distributed Model
From Non Spatial to SpatialPRESENTPRESENT
Web Services
W.S. Architecture
Geospatial W.S.
WMS/WFS
WPS
HNODC General
HNODC Architecture
HNODC Interface
HNODC W.S.FUTUREFUTURE
The new architecture
An example
Conclusions
HCMR / Hellenic National Oceanographic Data Center
At the same time the application may now interact with any other SOA application either in sending or receiving Geospatial Data, since it inherits the big advantage of interoperability between Web Services systems.
HNODC…
PASTPAST
Historical Brief
Centralized D.M.
From Centralized to Distributed Model
From Non Spatial to SpatialPRESENTPRESENT
Web Services
W.S. Architecture
Geospatial W.S.
WMS/WFS
WPS
HNODC General
HNODC Architecture
HNODC Interface
HNODC W.S.FUTUREFUTURE
The new architecture
An example
Conclusions
HCMR / Hellenic National Oceanographic Data Center
The Architecture can denoted as follows:– At the back End Open source PostgreSQL DBMS (and
not only..) stands as the data storage mechanism with more than one Data Base Schemas cause of the separation of the Geospatial Data and the non Geospatial Data.
– UMN Map Server and Geoserver are the mechanisms for
• Represent Geospatial Data via Web Map Service (WMS)
• Querying and Navigating in Geospatial and Meta Data Information via Web Feature Service (WFS)
• And in the near future Transacting and processing new or existing Geospatial Data via Web Processing Service (WPS)
HNODC - Architecture…PASTPAST
Historical Brief
Centralized D.M.
From Centralized to Distributed Model
From Non Spatial to SpatialPRESENTPRESENT
Web Services
W.S. Architecture
Geospatial W.S.
WMS/WFS
WPS
HNODC General
HNODC Architecture
HNODC Interface
HNODC W.S.FUTUREFUTURE
The new architecture
An example
Conclusions
HCMR / Hellenic National Oceanographic Data Center
– WhereGroup MapBender, a geospatial portal site management software for OGC and OWS architectures, acts as the integration module between the Geospatial Mechanisms.
– MapBender comes with an embedded data model capable to manage interfaces for displaying, navigating and querying OGC compliant web map and feature services (WMS and transactional WFS).
HNODC - Architecture…PASTPAST
Historical Brief
Centralized D.M.
From Centralized to Distributed Model
From Non Spatial to SpatialPRESENTPRESENT
Web Services
W.S. Architecture
Geospatial W.S.
WMS/WFS
WPS
HNODC General
HNODC Architecture
HNODC Interface
HNODC W.S.FUTUREFUTURE
The new architecture
An example
Conclusions
HCMR / Hellenic National Oceanographic Data Center
Apache and Tomcat stand again as the Web Service middle Layers
Apache Axis2 with it’s embedded implementation of the SOAP protocol acts as the No spatial data Mechanism of Web Services.
(These modules of the platform are still under development but their implementation will be fulfilled in the near future.)
Finnaly Web user Interface for the end user developed based on enhanced and customized version of a MapBender GUI.
HNODC - Architecture…PASTPAST
Historical Brief
Centralized D.M.
From Centralized to Distributed Model
From Non Spatial to SpatialPRESENTPRESENT
Web Services
W.S. Architecture
Geospatial W.S.
WMS/WFS
WPS
HNODC General
HNODC Architecture
HNODC Interface
HNODC W.S.FUTUREFUTURE
The new architecture
An example
Conclusions
HCMR / Hellenic National Oceanographic Data Center
PostGreSQLSpatial
PostGreSQLNon Spatial
UMN Map Server
Internet
GeoServer
Apache
Web Application
Back End
Geospatial Mechanism
Middle Layer
End User
External Web Services
Internal Web Services(Apache/Axis)
Non Spatial Mechanism
HNODC - Architecture…PASTPAST
Historical Brief
Centralized D.M.
From Centralized to Distributed Model
From Non Spatial to SpatialPRESENTPRESENT
Web Services
W.S. Architecture
Geospatial W.S.
WMS/WFS
WPS
HNODC General
HNODC Architecture
HNODC Interface
HNODC W.S.FUTUREFUTURE
The new architecture
An example
Conclusions
HCMR / Hellenic National Oceanographic Data Center
HNODC Web Interface PASTPAST
Historical Brief
Centralized D.M.
From Centralized to Distributed Model
From Non Spatial to SpatialPRESENTPRESENT
Web Services
W.S. Architecture
Geospatial W.S.
WMS/WFS
WPS
HNODC General
HNODC Architecture
HNODC Interface
HNODC W.S.FUTUREFUTURE
The new architecture
An example
Conclusions
HCMR / Hellenic National Oceanographic Data Center
HNODC Web Interface PASTPAST
Historical Brief
Centralized D.M.
From Centralized to Distributed Model
From Non Spatial to SpatialPRESENTPRESENT
Web Services
W.S. Architecture
Geospatial W.S.
WMS/WFS
WPS
HNODC General
HNODC Architecture
HNODC Interface
HNODC W.S.FUTUREFUTURE
The new architecture
An example
Conclusions
HCMR / Hellenic National Oceanographic Data Center
HNODC Web Interface PASTPAST
Historical Brief
Centralized D.M.
From Centralized to Distributed Model
From Non Spatial to SpatialPRESENTPRESENT
Web Services
W.S. Architecture
Geospatial W.S.
WMS/WFS
WPS
HNODC General
HNODC Architecture
HNODC Interface
HNODC W.S.FUTUREFUTURE
The new architecture
An example
Conclusions
HCMR / Hellenic National Oceanographic Data Center
• As conclusion we may denote that embedding these new technologies to data management model we can exploit the advantages to:– Inherit interoperability between among
Data Centers and Data Providers– Interchange and combine Data Sets with
minimum programming efforts– Produce comprehensive and self explained
new Data Products combing various and diverse data sources
– Help end user to navigate through a wide range of information
ConclusionsPASTPAST
Historical Brief
Centralized D.M.
From Centralized to Distributed Model
From Non Spatial to SpatialPRESENTPRESENT
Web Services
W.S. Architecture
Geospatial W.S.
WMS/WFS
WPS
HNODC General
HNODC Architecture
HNODC Interface
HNODC W.S.FUTUREFUTURE
The new architecture
An example
Conclusions
HCMR / Hellenic National Oceanographic Data Center
We are happy to say that HNODC joins the global scientific community by providing and consuming application Independent data products.
ConclusionsPASTPAST
Historical Brief
Centralized D.M.
From Centralized to Distributed Model
From Non Spatial to SpatialPRESENTPRESENT
Web Services
W.S. Architecture
Geospatial W.S.
WMS/WFS
WPS
HNODC General
HNODC Architecture
HNODC Interface
HNODC W.S.FUTUREFUTURE
The new architecture
An example
Conclusions
HCMR / Hellenic National Oceanographic Data Center
Quality Control in HNODC
The quality control procedures that HNODC uses to manage oceanographic data and information follow the international standards recommended by IOC, ICES and EU:
Documentation of the data sets
Format Conversions (Medatlas)
Quality Control on data and metadata
HCMR / Hellenic National Oceanographic Data Center
Quality Control in HNODC
Quality Control at Metadata and Data (automatic and visual):
• Format, codes and completeness of information
• Cruise and stations headers
• Observed data of vertical profiles and time series of fixed moorings
The results of the QC are added as quality flags to each numerical value for the location, date, bottom depth and the data points of vertical profiles and time series.
The quality controlled data set is archived in the
HNODC’s multidisciplinary database.
HCMR / Hellenic National Oceanographic Data Center
The main software tool deployed for QC is “Scoop” Scoop is a open source application aimed in
moderating and visual handling of Quality Control aspects like :– Checks in various data format– Duplication entries in datasets– Visual observation of vertical or time series profiles – …etc
Quality Control in HNODC
HCMR / Hellenic National Oceanographic Data Center
Format Checks
HCMR / Hellenic National Oceanographic Data Center
Vertical Profiles
duplicates cruises within a preset space-time radius
date of the stations
ship velocity between two
consecutive stations
bottom sounding (ETOPO5)
land position (GEBCO)
Time Series
duplicates entriesSensor depth check
Series duration check
land position (GEBCO)
Automatic Checks of Cruise and Stations headers
HCMR / Hellenic National Oceanographic Data Center
at least two parameters required for the qc (vertical
reference + measurement)
constant profiles or times series (sensor stuck)
data out of regional values
comparison with pre-existing climatological statistics
(LEVITUS, MODB, MEDATLAS). Time series are compared
with internal statistics.
decreasing reference parameter (pressure or time)
pressure must not exceed the bottom depth
spikes
vertical instability (for vertical profiles)
Observed data of vertical profiles and time series of
fixed moorings
Automatic Checks of Observed Data
HCMR / Hellenic National Oceanographic Data Center
Visual Checks of Cruise and Stations headers
HCMR / Hellenic National Oceanographic Data Center
Visual Checks of Observed Data
HCMR / Hellenic National Oceanographic Data Center
Poseidon System
Poseidon is a system of : Monitoring
Forecasting
Various Information production
For the Greek seas
HCMR / Hellenic National Oceanographic Data Center
Poseidon is based on an established network of observation buoys
And on a specialized operational center for the processing of the data collected and the production of forecasts,
POSEIDON system is an infrastructure at the leading edge of modern oceanography in Europe .
The network of observation buoys records continuously the physical, biological and chemical parameters of the Greek seas.
Those data are then transmitted to the operational center where they are sorted and fed into forecasting models.
Poseidon System
HCMR / Hellenic National Oceanographic Data Center
POSEIDON system is a unique planning tool in the endeavour for the protection of the marine environment.
It also provides a competitive advantage for the development of business activity, the prevention of disaster, and the safeguarding of human life.
In the frame of the world-wide trend for the development of operational oceanography, POSEIDON network places Greece among the leading countries in this field
Poseidon System
HCMR / Hellenic National Oceanographic Data Center
The system under its full functionality can provide :
immediate (on line) and dependable information
reliable forecast
protection of marine ecosystem
support of marine research
Poseidon System
HCMR / Hellenic National Oceanographic Data Center
http://www.poseidon.hcmr.gr
7 Seawatch buoys
3 Wavescan buoys
The “POSEIDON” buoy network system
The observation buoys are equipped with sensors that monitor:
Air-pressure
Air-temperature
Wind speed and direction
Wave height, period and direction
Sea surface salinity and temperature
Surface current speed and direction
Sea surface dissolved oxygen
Light attenuation with fluorescence
Salinity and temperature in depths 20-1000 m
Chlorophyll-A in depths 0-100 m
Dissolved oxygen in depths 20-100 m
Current speed and direction in depths 20-50 m
Nutrients
Radioactivity
HCMR / Hellenic National Oceanographic Data Center
Data are transmitted to the Operational Center of POSEIDON using 3
telecommunications systems:
- INMARSAT-C satellite
- GPRS
- Irridium
HCMR / Hellenic National Oceanographic Data Center
Exports – Decodes DATA
Creates a 24h-file with the previous day – Data which are
to pass the Quality Check process
Inputs file to
Quality Check
The data pass through some tests
HCMR / Hellenic National Oceanographic Data Center
QC checkMissing Data are replaced with 99.999
Sensor-range test
The data are tested against physical limitation of sensors
fail
Physical-range test
The data have to range within thresholds according to regional
climatology
Rate of change test
The difference between the test value with the previous and the next one may not exceed the
timeseries std or a pre-arranged threshold
fail
Stuck value test
The value may not remain constant compared with a number of previous values
fail
fail
Flag 9 assigned
Flag 2 assigned
Flag 4 assigned
Flag 3 assigned
Flag 3 assigned
Flag 1 assigned
Produce the file with flag indicators
pass
pass
pass
pass
Generate output file in Medatlas format
Update Database
•MERSEA
•Coriolis
•Ocean SITES
•SEPRISE …etc
Examples
HCMR / Hellenic National Oceanographic Data Center
Contains
Contains
The parameter data pass through a perl script into a mysql
database
Insitu Data DatabaseNormalized Metadata
Database
Metadata database normalization process applied to the insitu
metadata database
Metadata Database
Metadata cleaning (containing name uniformity, filling missing
values)
Parameters Measured Stored
into files
Server 1
Server 2
Normalization process applied to the insitu
data
Inputs data
Updates flagged data
Inputs Metadata
HCMR / Hellenic National Oceanographic Data Center POSEIDON, 2007
A. Telecommunications (Satellite, GSM modems)B. Computer infrastructure:
o Altix 3700 High-Performance Server 128 CPUs [email protected] GHz 128 GΒ RAM 20 TB Hard Disk Drives
10 Dual socket Linux Workstations
C. Information management (S/W, D/B)
DATA VIAINMARSAT-C
DATA VIA GSM
DATARECEPTIONODA LINK
POSEIDONDATA BASE
OCEAN CHECK(DATA QUALITY
CONTROL)
OCEAN INFO
NODC
HCMR / Hellenic National Oceanographic Data Center
Thank you for your attention!
The end…