Upload
nigel-malone
View
217
Download
0
Embed Size (px)
Citation preview
OBSERVATIONS
& PRÉVISIONS CÔTIÈRES
www.seadatanet.org
SeaDataNetPan-European Infrastructure
for Ocean and Marine Data Management Gilbert Maudire and the SeaDataNet Consortium
SeaDataNet mid-term review – Bologna – September 2008, 19
OBSERVATIONS
& PRÉVISIONS CÔTIÈRES
2www.seadatanet.org
SeaDataNet mid-term review – Bologna – September 2008, 19
Data is of vital importance for marine researchand support to public decision
Marine observation and data sets status
o observation conducted by more than 1100 active scientific laboratories in the countries of partners, using various sensors and platforms
o not always quality controlled using comparable procedures
o not always easily accessible
Necessity of large background work in order to : secure them in long term archivesdocument available observations and data sets (metadata) improve coherence of data sets (references, quality, redundancy) make data sets accessible easily (formats, on line distribution)
OBSERVATIONS
& PRÉVISIONS CÔTIÈRES
3www.seadatanet.org
SeaDataNet mid-term review – Bologna – September 2008, 19
What is SeaDataNet? EU Sixth Framework Programme
Integrated Research Infrastructure Initiative (I3)
1 April 2006 to 31 March 2011 (5 years)
Objectives
o To network existing oceanographic data centres already nationally funded
o To develop an efficient distributed pan-European marine data management Infrastructure (a “unique Virtual Data Centre”)
o To provide on-line access to distributed databases of standardised quality by using adapted communication & information technology
This project integrates and develops several EU past initiatives and international cooperation, in particular Medar/Medatlas, Edmed, Edios, Sea-Search etc.
OBSERVATIONS
& PRÉVISIONS CÔTIÈRES
4www.seadatanet.org
SeaDataNet mid-term review – Bologna – September 2008, 19
49 Partners National Oceanographic Data Centres
o divisions of major national marine research institutes in 35 countries
o experienced as leaders of data Management of many scientific projects and of concerted actions (e.g. MTPII-MATER, CANIGO, OMEX, EDIOS, MERSEA, MEDAR/MEDATLAS, SEA-SEARCH)
3 satellite data centres
3 scientific modelling centres
2 SMEs experts in software development
3 International organisations
with strong regional interest/cooperation
OBSERVATIONS
& PRÉVISIONS CÔTIÈRES
5www.seadatanet.org
SeaDataNet mid-term review – Bologna – September 2008, 19
Primary focus on water column parameterso Physical
o Chemical
o Bio-chemical (non species dependant)
Projects will be proposed for other domains by involved communities
in strong relationships with SeaDataNet
Several SeaDataNet partners are already involved in those domains
o Geophysical and geological data
o Biology (species dependant)
SeaDataNet managed data
OBSERVATIONS
& PRÉVISIONS CÔTIÈRES
6www.seadatanet.org
SeaDataNet mid-term review – Bologna – September 2008, 19
SeaDataNet services Online services : web portal and web services
o Discovery services • Metadata directories • Common vocabularies : parameters, units, …
o Delivery services • Security services : Authentication, Authorization & Administration (AAA)• Data download and access including reformatting in standardized formats
o Viewing services• Web map services, data visualization.
Offline services : « back office » serviceso Safeguarding within the national repositories (NODCs)
o Quality Control and desktop viewing services• ODV analysis software
o Product generation adapted to each regions
o Monitoring and statistics : usage & system performance
OBSERVATIONS
& PRÉVISIONS CÔTIÈRES
7www.seadatanet.org
SeaDataNet mid-term review – Bologna – September 2008, 19
Infrastructure versions & planning Version 0 – 2006-2007
Continuation and maintenance of existing Sea-Search system : the data access needs several different requests to each data centres and the data sets are delivered in different formats
o Operated at the start of the project
Version 1 – 2008-2010Setup of the integrated online data services to users :
networking of 10 “interoperable” data centres of the Technical Task Teamunique request to the interconnected data centresand the data sets are delivered with a unique format
o Presently under test and progressive integration of 10 data centres during 2008
Version 2 – 2010-2011Improved version of the system (ease of use, data visualisation, data access)and by step, all data centres connected for providing online trans-national access to the data, metadata and products
o Operational in 2010
OBSERVATIONS
& PRÉVISIONS CÔTIÈRES
8www.seadatanet.org
SeaDataNet mid-term review – Bologna – September 2008, 19
Discovery services
All catalogues (European metadata directories)• are compliant with ISO19115 standard family for spatial metadata• make use of common vocabularies and references• are accessible and linked through web services• ease data access using various selection criterions adapted to
different users (scientists, observatory and fleet managers, …)
OBSERVATIONS
& PRÉVISIONS CÔTIÈRES
9www.seadatanet.org
SeaDataNet mid-term review – Bologna – September 2008, 19
Common data index : gate to data access
online searchable directory
where?, when?, what?, …
tool for managing user’s“Shopping Basket”, that is :o To prepare a list of distributed datasets
matching user’s requesto To ask concerned data centres to serve the user,o To display the status of user’s requesto To route the requested data from the distributed
data centres to the user
OBSERVATIONS
& PRÉVISIONS CÔTIÈRES
10www.seadatanet.org
SeaDataNet mid-term review – Bologna – September 2008, 19
SeaDataNet data policy and data licenceinspired by INSPIRE Directive to facilitate the data flow takes into account the local rules and the SeaDataNet users needs
Metadatao free and open access, no registration required
o each data centre should provide the meta-data in standardized format to populate the catalogue services
Data and productso visualisation freely available
o data download, in general, free for academic purposes with mandatory registrationo a “SeaDataNet role” (partner, academic, commercial etc.) is attributed to individual user
using the Authentication, Authorization and Administration (AAA) Service• Each NODC attributes the roles to the users of its of country
• Out of the partnership, the roles are assigned by SeaDataNet user-desk
o a common data licence has been adopted and users must agree on when registering
o each data centre node delivers data according to the user’s role and its local regulationo each data centre should provide the data sets necessary to develop the common products
OBSERVATIONS
& PRÉVISIONS CÔTIÈRES
11www.seadatanet.org
SeaDataNet mid-term review – Bologna – September 2008, 19
QC procedures and tools
SeaDataNet protocol :
o Coherent automatic and visual checks are performed on the data by each NODC
o A unique quality flag scale has been adopted The automatic checks and the visualization could be performed
using either by Ocean Data View (ODV) or existing partner software (if the same protocol and checks are applied)
Example:Using derivatives for spike & discontinuity detection in ODV
OBSERVATIONS
& PRÉVISIONS CÔTIÈRES
12www.seadatanet.org
SeaDataNet mid-term review – Bologna – September 2008, 19
Data products SeaDataNet develops added-value products at regional
scales:o Mediterranean Seao Black Seao Baltic Seao Arctic Seaso Atlantic and global oceanThe preparation of these products allows the
checking of the compatibility of the QCs performed on the data in each NODC.
First products => gridded climatologies and trends
The comparability of the different regional products is assessed by using a new version of DIVA (Data Inverse Variational Analysis) developed (University of Liege)
OBSERVATIONS
& PRÉVISIONS CÔTIÈRES
13www.seadatanet.org
SeaDataNet mid-term review – Bologna – September 2008, 19
Capacity Building is a main issue Training program main objectives
to ensure that management procedures are coherent within the consortium
to transfer expertise and to train IT experts of the SeaDataNet data centers in the installation and operation of the SeaDataNet technical components
OBSERVATIONS
& PRÉVISIONS CÔTIÈRES
14www.seadatanet.org
SeaDataNet mid-term review – Bologna – September 2008, 19
www.seadatanet.org
Please contact : [email protected]
OBSERVATIONS
& PRÉVISIONS CÔTIÈRES
www.seadatanet.org
SeaDataNetProject management
SeaDataNet mid-term review – Bologna – September 2008, 19
OBSERVATIONS
& PRÉVISIONS CÔTIÈRES
16www.seadatanet.org
SeaDataNet mid-term review – Bologna – September 2008, 19
Role of partners (1/4) Coordination
o Ifremer : project coordinator
o Maris : technical coordinator animator of the Technical Task Group with the help of :
• NERC/ BODC : standard development
• AWI : data quality control and desktop viewing services
o University of Liege : scientific coordinator in charge of the regional products development with the help of :
• JRC : Product harmonization
• INGV : coordinator of Mediterranean products
• METU : coordinator of Black Sea products
• NERI : coordinator of Baltic products
• IMR & VLIZ : coordinators of Artic & North Seas products
• Ifremer : coordinators of Atlantic & Global Ocean products
OBSERVATIONS
& PRÉVISIONS CÔTIÈRES
17www.seadatanet.org
SeaDataNet mid-term review – Bologna – September 2008, 19
Role of partners (2/4) Coordination (cont.)
o HCMR : Coordinator of the networking activities i.e. metadata and data management, system monitoring
with the help for coordinating the distributed network of
• Maris : management of Common Data Index (CDI) and of Directories of Marine Projects (EDMERP) and Marine Organizations (EDMO)
• NERC/ BODC : management of Directories of Marine Environmental Datasets (EDMED) and of Ocean Observing System (EDIOS)
• BSH : management of Cruise summary reports
• Ifremer : Management of SeaDataNet users
OBSERVATIONS
& PRÉVISIONS CÔTIÈRES
18www.seadatanet.org
SeaDataNet mid-term review – Bologna – September 2008, 19
Role of partners (3/4) Technical task team (11 partners : Western Europe + Russia)
o Technical specifications,
o Check of the defined standards
o Integration and system setup, test and validation of technical components
o Running of System V1
Capacity building and training
o UNESCO/ IOC Project office with the help of Technical Task Team
Information and communication
o ENEA
OBSERVATIONS
& PRÉVISIONS CÔTIÈRES
19www.seadatanet.org
SeaDataNet mid-term review – Bologna – September 2008, 19
Role of partners (4/4)
All partners which are mostly NODC’s (except international organizations) have to
o generate metadata : organizations, projects, cruises, observing systems, data sets
o manage data from their country : quality, format, … according to SeaDataNet agreed standards
o provide access to their data repository, automatically using SeaDataNet system or manually as a first stage
o manage users of their respective countries by providing information and help
OBSERVATIONS
& PRÉVISIONS CÔTIÈRES
20www.seadatanet.org
SeaDataNet mid-term review – Bologna – September 2008, 19
External relationships / Collaborations
Advisory board
o MEDS Canada (represents IODE/WMO JCOMM, GTSPP and Argo international programs)
o US – NODC (represents North America initiatives like DMAC)
o EuroGOOS (represents Operational Oceanography community)
Standard summit (last January on behalf of COI)
o North America, Europe, Japan
“Memorandum of understanding” with other consortiums
o Unidata, Mersea-MyOcean (GMES), EuroBis-Marbef (FP6), GENESI (FP7)
International conference on Marine Data and Information Systems (IMDIS)
o April 2008, more than 200 participants from about 50 countries.
OBSERVATIONS
& PRÉVISIONS CÔTIÈRES
21www.seadatanet.org
SeaDataNet mid-term review – Bologna – September 2008, 19
Project management tools SeaDataNet web site www.seadatanet.org
o Distributed management, based on a Content Management system (EZPublish)
o General information : Description of Work, Meetings, Conferences, Education (several languages)
o Project services (discovery, data access, …)
o Finalised documents (applicable version of standards) and software tools
Mailing listso Based on Sympa software
o One list per group (technical, …) owned by the chair person
o + user desk (SeaDataNet users point of contact)
Document managemento Based on BSCW software (Basic Support for Cooperative Work - Germany)
o Cooperative work on documents (non finalised versions) and software
Software management (versionning tool)o Based on GForge
OBSERVATIONS
& PRÉVISIONS CÔTIÈRES
22www.seadatanet.org
SeaDataNet mid-term review – Bologna – September 2008, 19
System monitoring focused on system “availability”
Availability of SeaDataNet user serviceso availability of web pages and web services
general information, metadata browsing (EDMED, CSR, CDI), data selection users’ interfaces, web services
o availability of web data access servicesdata centre download services
o Service “available” if monitoring system get an answer• Search of keywords within web pages,• Web services can be reached : WSDL returned
Monitoring o done by IFREMER (coordinator) and HCMR
in order to avoid biases due to network troubles
o Using NAGIOS Open Source software• Polling of services (web pages, web services)
OBSERVATIONS
& PRÉVISIONS CÔTIÈRES
23www.seadatanet.org
SeaDataNet mid-term review – Bologna – September 2008, 19
Polling of system componentsType(*) Name Address (URL) Comment (**)
BODC
Web page (static)
BODC home page http://www.bodc.ac.uk/ Apache web server check (HEAD)
Web page (dynamic content)
http://www.bodc.ac.uk/cgi-bin/dbcheck Oracle database availability check (via Perl) (GET)
Web page (dynamic content)
http://www.bodc.ac.uk/data/online_delivery/gebco/ Tomcat availability check (HEAD)
ENEA
http MOON-VOS http://moon.santateresa.enea.itHCMR
User Interface
Oceanographic database
http://hnodc-data.hcmr.gr
Ifremer
Web page + User interface
Nautilus http://www.ifremer.fr/nautilus
Web page +User interface
Coriolis http://www.coriolis.eu.org
OGS
Web page (static)
NODC portal http://nodc.ogs.trieste.it/
BSH
Maris
Web site (static pages)
Sea-Search portal http://www.sea-search.net Will be integrated into SeaDataNet portal soon/ count only on static pages
Web page (dynamic content)
http://www.bodc.ac.uk/maps/
http://www.bsh.de/de/Meeresdaten/Beobachtungen/DOD-Datenzentrum/index.jsp
Map server availability check (HEAD)
Home page DOD Home page
OBSERVATIONS
& PRÉVISIONS CÔTIÈRES
24www.seadatanet.org
SeaDataNet mid-term review – Bologna – September 2008, 19
Monitoring board Example of one-month report for Ifremer http access
Electrical failure during 6 hours
OBSERVATIONS
& PRÉVISIONS CÔTIÈRES
www.seadatanet.org
SeaDataNetAdministrative and financial aspects
SeaDataNet mid-term review – Bologna – September 2008, 19
OBSERVATIONS
& PRÉVISIONS CÔTIÈRES
26www.seadatanet.org
SeaDataNet mid-term review – Bologna – September 2008, 19
Known issues IOC Project Office partner
o In charge of training courses and capacity buildingo Report at 0 up to now even if work done (3 training courses organised yet)
no agreement on US $ / € change rate between UNESCO and EC)
o IOC is an « Additional Cost partner »not well adapted to training course organization (additional personnel)
Transnational activities report
o Partners with very small web site and/or very little audience
o Audience measurment : KiloPageViews unit difficult to report accurately
o 80% of partners report less than expected for TA
OBSERVATIONS
& PRÉVISIONS CÔTIÈRES
27www.seadatanet.org
SeaDataNet mid-term review – Bologna – September 2008, 19
Transnational Activities amendment
Change reporting unit from KiloPageViews to KiloHits
o To take into account all hits on online services :web pages (for any usages), web services, ftp servers, …
o But, cost / unit must decrease (Project Officer comment)
However, most audiences will remain far less than expected
o Cost are not linearly proportional to the audience« Entry costs » to set up a web site funds transfer to JRA (web site maintenance)
o Who? How much?
All must remain in the same budget envelope
OBSERVATIONS
& PRÉVISIONS CÔTIÈRES
www.seadatanet.org
SeaDataNetPlanning
SeaDataNet mid-term review – Bologna – September 2008, 19
OBSERVATIONS
& PRÉVISIONS CÔTIÈRES
29www.seadatanet.org
SeaDataNet mid-term review – Bologna – September 2008, 19
Infrastructure versions & planning Version 0 – 2006-2007
Continuation and maintenance of existing Sea-Search system : the data access needs several different requests to each data centres and the data sets are delivered in different formats
o Operated at the start of the project
Version 1 – 2008-2010Setup of the integrated online data services to users :
networking of 10 “interoperable” data centres of the Technical Task Teamunique request to the interconnected data centresand the data sets are delivered with a unique format
o Presently under test and progressive integration of 10 data centres during 2008
Version 2 – 2010-2011Improved version of the system (ease of use, data visualisation, data access)and by step, all data centres connected for providing online trans-national access to the data, metadata and products
o Operational by 2010
OBSERVATIONS
& PRÉVISIONS CÔTIÈRES
30www.seadatanet.org
SeaDataNet mid-term review – Bologna – September 2008, 19
Project planning and budget
High level of effort during the first three years(80% of the budget)
o Common standard adoption, set up and training,
o Development / Adaptation of common tools
o System specifications, development and setup (V1 and then V2)
o Development of first release of regional products and initial harmonization
Lower level of effort during the last two yearso Continuous data management
o Exploitation of the system, system maintenance and upgrade
o Upgrade of regional products
OBSERVATIONS
& PRÉVISIONS CÔTIÈRES
31www.seadatanet.org
SeaDataNet mid-term review – Bologna – September 2008, 19
More information about budget
Funds breakdown / year / partner after second report
o Funds breakdown : https://www.ifremer.fr/bscw/bscw.cgi/0/110042
Transnational access audience
o Evolution for TA- Seadatanet.xls