High-Velocity Data Flow

VESTEC is developing and evaluating methods and interfaces to integrate high-performance data analytics processes into running simulations and real-time data environments.

Interactive SupercomputingInteractive ensemble management will launch new simu-lations for new data, building up statistically more and more accurate pictures of emerging, time-critical pheno-mena. Innovative data compression approaches, based on topological feature extraction and data sampling, will result in considerable reductions in storage and proces-sing demands by discarding domain-irrelevant data.

The use of extreme computing in real-time applications with high velocity data and live analytics is within reach. The availability of fast growing social and sensor net-works raise new possibilities in monitoring, assessing and predicting environmental, social and economic incidents as they happen. Add in grand challenges in data fusion, analysis and visualization, and extreme computing hardware has an increasingly essential role in enabling efficient processing workflows for huge hete-rogeneous data streams.

VESTEC is creating the software solutions needed to realize this vision for urgent decision making in various fields with high impact for the European community. VESTEC is building a flexible toolchain to combine mul-tiple data sources, efficiently extract essential features, enable flexible scheduling and interactive supercom-puting, and realize 3D visualization environments for

interactive explorations by stakeholders and decision makers.

HPC meets Real-Time Data

Topological ProxiesGiven the exascale nature of the simulations addressed by VESTEC, novel drastic data reduction schemes must be invented to be able to cope with the amount of generated data to allow for urgent decision making.

In-Situ RenderingThe visualization task can make use of same resour-ces (CPUs, memory, and fabric) as the code gene-rating the data to be visualized. In particular, having direct access to the huge memory holding the data allows for in-situ visualization with no or only little post-processing, reduction or compression. An exa-mple of the possibilities of in-situ visualization is ray-tracing to create volume renderings directly from the simulation data.

Architecture of VESTEC The Data Processing

Urgent Decision Making at DLR‘s Crisis Management Center Topological Proxies can help to extract Essential FeaturesData Flow: From Sensor Networks to the Rescue Forces

Pro

ject

Des

crip

tion

VEST

EC F

lyer, V

ersio

n: 20

19-11

-08

The PartnersGerman Aerospace Center (DLR), Cologne, Germany

EPCC at the University of Edinburgh, Edinburgh, UK

Kungliga Tekniska Hoegskolan (KTH), Stockholm, Sweden

Sorbonne Université, Paris, France

Kitware SAS, Villeurbanne, France

INTEL Deutschland Gmbh, Feldkirchen, Germany

Fondazione Bruno Kessler, Trento, Italy

Université Paul Sabatier, Toulouse, France

Tecnosylva SL, León, Spain

AcknowledgementThe research leading to the results which are presented in this flyer has received fun-ding from the European Union Horizon 2020 Programme (H2020-FETHPC-2017) under grant agreement n° 800904.

ContactHomepage: https://VESTEC-project.euEmail: vestec@dlr.deTwitter: @VESTECproject

The Three Use Cases

VESTECVisual Exploration and Sampling Toolkit for Extreme Computing

Urgent HPCInteractive Supercompu-ting for Urgent Decision Making

Wildfire Monitoring and Forcasting

Mosquito-borne Diseases Risk Analysis

Space Weather Effects

Wild Fire Propagation Forcast (left) and Uncertainty Depiction (right)

Spatio-Temporal Variation in the Epidemic Transmissibility Potential

Ray-tracing Particles of the Space Weather Use Case in Paraview