WP8 Data and product delivery, exploitation and dissemination€¦ · The EUNADICS-AV project has received funding from the European Union’s Horizon 2020 research programme for

www.eunadics.euThe EUNADICS-AV project has received funding from the European Union’s Horizon 2020 research programme for Societal challenges - smart, green and integrated transport under grant agreement no. 723986.

WP8 Data and product delivery, exploitation and dissemination


Wim Som de Cerff, Saskia Wagenaar, Suzanne Kok, Nicolas Theys, Hugues Brenot, Matthieu Plu , Matthias Themessl, Martin Suttie, Fritz Zobl

EUNADICS-AV Final Review 05.11.2019 @EC, Brussels


Objectives (DoW)

• Assure pan European information accessibility for aviation stakeholders in a disaster situation

• Optimise dissemination and improve access to crisis data and information

• Data dissemination to NMS, MWOs, RSMCs, VAACs, SWIM, GEO/GEOSS, …

• Demonstration platform • Sustainable basis


WP8 Specific Objectives

• Dissemination and exploitation of project results – Project public website, news, workshops, etc. – Project internal website (supporting WP1)

• Demonstration platform: showcase data products – Data delivery chain for EUNADICS-AV products – Interfaces to VAAC, NMS, SWIM, web portal, web services

(GEO/GEOSS)


Activties in RP2

Task No.

Title Task Lead Completion state (%)

T8.1 Dissemination and outreach [M1-36] KNMI 99.9%

T8.2 Prototyping Data interfaces [M1-M10] KNMI 100%

T8.3 Prototyping the EUNADICS-AV portal [M6-M32] KNMI 100%

T8.4 VAAC interface to EUNADICS-AV [M6-M36] BIRA-IASB 100%

T8.5 NMS interface to EUNADICS-AV [M6-M36] ECMWF 100%

T8.6 4DWxCube interface to EUNADICS-AV [M6-M36] PLUS 100%


Activities in RP2

Task 8.1 Dissemination and outreach • www.eunadics.eu website (D39)

– 8 news items on relevant hazardous events • Project administration website

– Project repository (Redmine) • D40: The Dissemination plan

– Defined target users and key messages – Delivered, updated final version available

• EUNADICS promotion: – Movies, folder, doppers, https://youtu.be/XQT64N9Axi0 – 14 conferences (presentations, posters, papers) – stake holder workshop – 2 invited talks at AWS conference, case study (case study, movie)

http://www.eunadics.eu/

https://youtu.be/XQT64N9Axi0


Website statistics

number of views of the EUNADICS-AV promotion video


Activities in RP2Task 8.2 Prototyping Data interfaces

• Implemented the ingestion system for the EUNADICS data products


Activities in RP2

Task 8.2 Prototyping Data interfaces • Continuation of RP1 work • Provided NetCDF-CF

ADAGUC standard for formatting data products

• Provided an online data checker to enable data providers to check their data format

• Implemented data ingest routines


Activities in RP2

Task 8.3 Prototyping the EUNADICS-AV data portal

• User involvement in developing the user interface • demonstrations, training exercise, use of persona’s

• Iterative architecture design and implementation: • From using own components to cloud native • Aiming for optimal use of cloud

• Optimal use of Cloud technology in the implementation: • Ability to automatically scale up if an event occurs • Keeps running costs low (maintenance, development, running costs) • Using cloud native components improves maintainability • Enables a small team to do big developments


Architecture evolution

Home page: Portal showing realtime TropOMI aerosol index

Home page: Portal showing realtime TropOMI aerosol index and E-Profile level 2 data

http://data.eunadics.eu/

Portal showing the exercise data

http://data.eunadics.eu/


Activities in RP2

Task 8.4-6 Prototyping the EUNADICS-AV data interfaces

VAAC interface and NMS interface: • Provisioned by ECMWF ECPDS • Developed and operated by ECMWF • Operational in use for data dissemination to NMS (and thus VAAC) • Tested • Results reported in milestone reports (M36, M37, M39, M40)

SWIM interface: • Following the advisory board advise we did not investigate the 4DWxCube • Opted for SWIM implementation • Pragmatic implementation for feature data in GeoJSON agreed on • Demonstrator implementation developed and tested during the exercise • Requests for gridded data via SWIM, need for new standardised data formats • Results are in M38

All lessons learned are documented in D44, Integrated test report and lessons learned on interface prototyping


Highlights

• Successful Data architecture • Services based, convert data to standards, provide standards, checker for self servicing data providers

• Data provisioning during and after training event • Boosted developments and data delivery

• Many lessons learned (see D44), especially regarding visuals

• Cloud implementation fits all stated objectives • Accessibility, optimised data dissemination (able to scale up on event)

• Demonstration platform, sustainable basis

• Visualisations in the portal • Show model data, satellite data and observations in one interactive online viewer

• AWS Support and showcase • Invited talks at 2 AWS conferences

• SWIM interface demonstrator • Pragmatic approach


Deviations (if project relevant)

• Deviations reported in RP1 repaired: • Delayed deliverables are delivered • Data delivery interfaces are prototyped

• RP2: Milestone 35 Hackathon event • Hackathon will be joined after project end:

Copernicus Hackathon session at Deltares Software Days, 14-15 November, 2019

• Not possible to join similar community events earlier • Not seemed feasible or necessary to organise a separate event • Goals are largely covered by training event


Lessons learned

• Dissemination • Movies are beneficial but take time and effort to implement

• Data integration requires standardisation • Approach to use NetCDF CF, provide standards and a checker payed of

• Data platform to do the data plumbing

• but still not the answer to all issues

• Cloud (native) implementation • Provides robust, scalable solution

• Design iteratively to get the best result

• Visualisation • Integrating model, satellite and ground based observations requires more interaction between users, data providers and developers

• Agile approach to development • Learned how to successfully apply Scrum development in a EU project context

• SWIM interface design requires follow-up • Hard to get developer input, guidance or feedback

• Pragmatic ideas formed and tested

• Need to be formalised, but how?

• Gridded data requested, but no standardised SWIM data format yet

• Operational data delivery • Not all data operational available • Different, changing and hard to get access to the data streams

• And many more, see D44


Sustainability plan (see WP2)EUNADICS-AV product

categoryOffering / product TRL Public funding of R&D Public funding of

operational serviceRemarks (e.g. already in

use in...)

EUNADICS-AV Data platform

Platform integrating and harmonizing the different EUNADICS-AV products and provding these through standardized data distribution interfaces and visualizations

5, Component and/or breadboard critical function verification in relevant environment

EUNADICS-AV visualizations (ADAGUC)

Specific data backend and web-interface components, enabling visualization of the EUNADICS-AV data

6, Model demonstrating the critical functions of the element in a relevant environment

EUNADICS contributed to the development of ADAGUC is some components

EUNADICS-AV SWIM interface

Protoype SWIM interface for distribution of EUNADICS-AV products.

1, Basic principles observed and reported

EUNADICS-AV data distribution interface (ECPDS)

Data distribution services for distribution of data using standard FTP/SFTP to VAAC and NMSs

7, Model demonstrating the element performance for the operational environment

No, is is running as an operational service at ECMWF, available for Meteo partners


Data portal sustainability

• Provided for demonstration purposes until the end of 2020 (TBC) • TBC: depending internal funding being provided

• As demonstrator, showing realtime data: • TropOMI data (Aerosol index, CO, CF, CTH, NO2, SO2)

• E-Profile data (L2)

• Provides access to the Salzburg exercise data • Satellite-, ground based- and model data

• Visualisations

• Possible to scale up when an event happens • But not automatically

• Requires the partners to take action, especially around data availability

• Clear disclaimer stating that the informations cannot be used for operations

Note: in order to save costs, machines have been scaled down, making the portal slower than it used to be


Concluding remarks

• WP8 Objectives reached: • Dissemination and exploitation of project results • Demonstration platform: showcase data products • All milestones and deliverables delivered

• Many highlights & lessons learned • SWIM, visualisation, Standardisation, Cloud, Training event

• Portal Demonstrator remains available (TBC) for demonstration purposes until end 2020



Supporting safe air traffic

Documents

WP8 Data and product delivery, exploitation and dissemination€¦ · The EUNADICS-AV project has received funding from the European Union’s Horizon 2020 research programme for