Upload
meredith-gardner
View
221
Download
2
Tags:
Embed Size (px)
Citation preview
WMO
4.1 A critique of the existing data models for weather radar data exchange as presented in item
3.1 docsCBS/OPAG-IOS Workshop on Radar Data Exchange
Exeter, UK, 24-26 April 2013
Daniel Michelson, SMHI, Sweden
WMO
Data exchange model has four components:
1. File format – container for storing data in physical files.
2. Data/information model – the way in which information is organized/represented, either in computer memory or files, independently of file format.
3. Envelope – extra header announcing/describing the contents of the information being exchanged.
4. Protocol – mechanisms for communication over a network.
test footer 3
Content-Disposition: form-data; name="<baltrad_frame_xml/>"Content-Type: multipart/form-data; charset=UTF-8Content-Transfer-Encoding: 8bit<?xml version="1.0" encoding="UTF-8"?><baltrad_frame> <header mimetype="multipart/form-data" sender="local_production_system"/> <content channel="test_channel" name="pas1.h5" type="file"/></baltrad_frame>--s7Wj4EEwDrDPwOdPM30KMQzaEpfnylContent-Disposition: form-data; name="<baltrad_frame_file/>"; filename="pas1.h5"Content-Type: multipart/form-dataContent-Transfer-Encoding: binary�HDFÿ (… binary payload ODIM_H5 file …)--s7Wj4EEwDrDPwOdPM30KMQzaEpfnyl--
E.g. an early BALTRAD HTML exchange message
WMO
Different categories of file formats
1. Transmission – to facilitate data transfer from site to central facility. Can be optimized for network load balancing, e.g. ray-by-ray.
2. Production – contains all data and metadata required to derive higher-order outputs (e.g. QC:ed data, products) from input data.
3. Exchange – representation of data or product for sharing but not necessarily further processing, e.g. a radar composite or vertical profile.
4. Archive – e.g. to represent data from many different observation systems for storage.
WMO
Basic data exchange concepts
“Passive” Simple availability, e.g. “pull” Sender/recipient are aware of each other but
transact indirectly, e.g. “push” (GTS) Directory polling or file-system event monitoring
“Active” Sender/recipient are aware of each other and
interact/transact directly E.g. BALTRAD
WMO
GTS for radar data exchange?
Advantages+ Already exists+ Active networking
Disadvantage- Hierarchical organization causes delays
For example:
Exeter Offenbach Norrköping Riga
faster?
WMO
BALTRAD for radar data exchange?
Positives+ Already exists+ Active networking+ WIS integration already demonstrated (2010)
Challenges- Still being developed- Requires dedicated software- Not yet officially recognized as part of WIS
File format: GRIB
+ WMO standard+ Already in use with
national composite products in RA IV.
+ Demonstrated use for exchange. (GRIB1)
- No single official, centrally maintained, software.
- No mature information model representation.
File format: TITAN
+ Already in use in two regions (I and III)
+ Storm tracks also available, e.g. XML
+ Open software, centrally maintained
+ Efficient for transmission, production, exchange
- Documentation- No DQ preparedness- Prepared for dual-pol
moments?- Built-in compression?- Unknown outside
meteorology
File format: BUFR
+ WMO standard.+ Suitable for point and
profile data.+ Very flexible, “can encode
anything”.+ Demonstrated use in
Region VI, ODIM_BUFR
- “Unwieldy” for non-trivial data such as radar.
- Many mutually incompatible software and data model implementations.
- No single official, centrally maintained, software.
- Bottleneck to academia.- Not a marketable skill.- Largely unknown outside
meteorology.
File format: HDF5
+ Central, open, modern software developed/maintained by HDF Group.
+ Bundled for various OSes.+ Established in Earth
Sciences.+ ODIM_H5 prepared for
dual-pol and DQ.+ ODIM_H5 is “policed”.+ Efficient for production
and exchange.+ ODIM_H5 compatibility
between regions V and VI
- Steep learning curve.- Currently unsuitable as
transmission format.- ODIM_H5 not an orthodox
information model.- Ambiguities in ODIM_H5
risk incompatibilities.
File format: netCDF, version 4
+ Central, open, modern software developed/maintained by NCAR.
+ Bundled for various OSes.+ Established in Earth
Sciences through “CF Conventions”.
+ GIS-compliant.+ “CF Radial” prepared for
dual pol?
- Requires HDF5, added abstraction layer.
- No mature CF Convention yet for operational radar, although “CF Radial” for R&D.
- Unprepared for DQ?
WMO
ODIM in OPERA, February 2013
Input data Output products
Green = ODIM_H5Brown = ODIM_BUFRYellow = Both
WMO
Summary
Terminology needs clarifying. “Data model”, “exchange”, “file format”, etc.
Several file formats in use. No orthodox information model as such in use.
Not necessarily a disadvantage. Tight connections between file formats and
information models. Good solutions exist already.
www.wmo.int
Thank you for your attentionDaniel Michelson
Swedish Meteorological and Hydrological Institute
Norrköping, Sweden