© Crown copyright 2005 NESC Workshop 6th-8th September 2005Page 1
V-GISC – SIMDATGil Ross (Met Office UK)
NESC Workshop 6th to 8th September 2005
© Crown copyright 2005 NESC Workshop 6th-8th September 2005Page 2
Contents
Part 1 World Meteorological Organisation
Data - Communications now WMO Operational Meteorology
Real-Time Daily Data Flows
WMO Information System GISC - Global Information System Centre
data Portal data Mirror
Part 2 vGISC and SIMDAT
SIMDAT EU Project SIMDAT Meteorology
© Crown copyright 2005 NESC Workshop 6th-8th September 2005Page 3
WMO is….
UN Agency (1950)derived from IMO (1873)
Co-ordinate meteorologyweather
observations forecasts
climate changehydrologydisaster mitigation
187 Members
© Crown copyright 2005 NESC Workshop 6th-8th September 2005Page 4
WMO - Data Exchange
Now GTS Global Telecommunications System
Observations ForecastsWarnings
Private NetworkNode to Node networkNOT an Internet
Future WIS - WMO Information System
Replacement for GTSPortal for Operational MeteorologyCatalogue driven based on -
ISO 19115 Metadata WMO Core Profile
© Crown copyright 2005 NESC Workshop 6th-8th September 2005Page 5
GTS Real-Time Data in Met Office UK
Main Message Switch300,000 bulletins received per day5.2 million bulletins sent/day.
Main File Server17,000 files received/day27,000 files sent/day
Other routes – e.g. Satellite data ~ 27GB/day
Ephemeral and Persistent databases Ephemeral completely repopulated every dayPersistent < 1 TB new data per day
© Crown copyright 2005 NESC Workshop 6th-8th September 2005Page 6
Surface Observations 0000Z 3rd September
http://www.metoffice.gov.uk/research/nwp/observations/data_coverage/index.html
© Crown copyright 2005 NESC Workshop 6th-8th September 2005Page 7
Aircraft observations 000Z 3rd September
© Crown copyright 2005 NESC Workshop 6th-8th September 2005Page 8
WIS - Information collection data flow
Global Information System Centre GISC
GISC
GISC
GISC
GISC
NC NC
NC
NC
NC
NC
NC NC
NC
NC
Data Collection or Product Centre
National Centre NC
DCPC
DCPC
DCPC
DCPC
NC NC
Various communication networks
Global Information System Centre GISC GISC
GISC
GISC
GISC
GISC
NC NC
NC
NC
NC
NC
NC NC
NC
NC
Data Collection or Product Centre
National Centre NC
DCPC Data Collection or Product Centre
National Centre NC
DCPC
DCPC
DCPC
DCPC
NC NC
Various communication networks
© Crown copyright 2005 NESC Workshop 6th-8th September 2005Page 9
WIS - Information distribution data flow
National disseminationRequest/reply (Internet)
Nation
al u
sers
Nat
ion
al u
sers
Routine dissemination(Internet, private network,satellite, etc.)
GISC
GISC
GISC
GISC
NC
NC
NC
NCNC
NC
NCNC
DCPC
DCPC
DCPC
DCPC
Various communication networks
National disseminationRequest/reply (Internet)National disseminationRequest/reply (Internet)
Nation
al u
sers
Nat
ion
al u
sers
Routine dissemination(Internet, private network,satellite, etc.)
Routine dissemination(Internet, private network,satellite, etc.)
GISC
GISC
GISC
GISC
NC
NC
NC
NCNC
NC
NCNC
DCPC
DCPC
DCPC
DCPC
Various communication networks
SIMDAT / NESC Workshop Sept 05 12 SIMDAT
V-
GISC
vGISC
Instead of each National Met Service having a GISC (Global Information System Centre)
a “distributed” or “virtual” GISC
3 NMSs – Meteo France Deutcher Wetterdienst DWD Met Office UK – MetO
and two DCPCs Data Collection and Production Centre ECMWF EUMETSAT
SIMDAT / NESC Workshop Sept 05 13 SIMDAT
V-
GISC
SIMDAT
4 years project funded by the EU
Contract with EU was signed on 1 September 2004
Data Grids for Process and Product Development using Numerical
Simulation and Knowledge Discovery
SIMDAT focuses on 4 application area:
product design in automotive and aerospace,
process design in pharmacology
service provision in meteorology
Budget of 11 M € of which 10.5% for meteorological activity
project managed by ECMWF
SIMDAT / NESC Workshop Sept 05 15 SIMDAT
V-
GISC
SIMDAT
Seven Grid-technology areas have been identified to achieving SIMDAT objectives:
Integrated Grid infrastructure offering basic services to applications Access to data distributed on Grid sites Management of Virtual Organisation Workflows Ontologies Integration of analysis services Knowledge Services
Not all will be addressed in Meteo - SIMDAT
SIMDAT / NESC Workshop Sept 05 16 SIMDAT
V-
GISC
SIMDAT - Strategy
. Virtual Data Repository. Introduction of grid technologies research
The heart of the issue is data
Phase 1: Connectivity Phase 2: Interoperability Phase 3: Knowledge
. Deployment of Grid infrastructure with particular attention to data transport and management.. Distributed DB access
Workflows for next-generation aggregated knowledge capture, discovery and mining.
SIMDAT / NESC Workshop Sept 05 17 SIMDAT
V-
GISC
SIMDAT - vGISC
vGISC working groups adopted by SIMDAT (meteorology)
SIMDAT is a technical solution for vGISC type operations
will NOT deliver a vGISC
has specified requirements similar to vGISC
is investigating a vGISC Virtual Organisation
is developing catalogue operations
has developed message infrastructure
SIMDAT 18 month test system due - NOW!
SIMDAT / NESC Workshop Sept 05 20 SIMDAT
V-
GISC
VGISC Demonstrator: UKMO local db (1)
JEDDS
Internet
Internet
SIMDAT / NESC Workshop Sept 05 21 SIMDAT
V-
GISC
VGISC Demonstrator: UKMO local db (2)
Local database is a derivative of JEDDS – the ‘Joint Environmental Dynamic Data Server’
Originally developed by Met Office to provide data on demand to UK military
Features:
Live data feed; continuously refreshed
Serving global TAF, METAR & GRIB bulletins
Request-oriented SOAP web-service interface
SIMDAT / NESC Workshop Sept 05 22 SIMDAT
V-
GISC
VGISC Demonstrator: UKMO local db (3)
Database DMZ
EAL4Firewall
Internal
JEDDS ++
FROST
Application DMZ
Portal &Catalogue
(local)
Router
SIMDATDemonstratorDeployment
July 2005Jeremy Tandy
Push (ftp)
Internet
Pu
ll
User
Pull (http)
Portal &Catalogue(Remote)
Synchronisation
SIMDAT / NESC Workshop Sept 05 24 SIMDAT
V-
GISC
Working decisions (1)
Teleconference, wiki & email used for remote collaboration
focus on HOW the catalogue operates, not the minutiae of metadata standards
use a ‘random’ UID scheme and not try to ‘overload’ identifiers with semantics
use a simple ‘Google-like’ and/or directory-based search interface
3 classes of search – by Keyword, time and/or location
ignore internationalisation for now
Issue: WMO Core (& ISO19115) is TOO flexible – there are too many options for spatial & temporal descriptions … we need best practice ‘application schemas’
SIMDAT / NESC Workshop Sept 05 25 SIMDAT
V-
GISC
Working decisions (2)
Metadata
create linking schemas using the WMO Core Profile by using additional namespaces (i.e. <v:id>)
metadata documents should be split into a ‘hierarchy’ of XML ‘fragments’ are used within the catalogue
model for merging metadata fragments will uses Java templating rather than an XSLT-based approach … key issue to demonstrate the concept is WHAT the fragments are not the mechanics of HOW they are merged. This position may be revised post-demonstrator
Agreement: nomenclature …• a specific SYNOP (12:00UTC, 16/06/05 at Heathrow) is an instance• the SYNOP ‘product’ is a generalised instance that requires instantiation parameters to
identify a specific synop instance