D5.2 EUXDAT e-Infrastructure · Document name: D5.2 EUXDAT e-Infrastructure Page: 8 of 38 Reference: D5.2 Dissemination: PU Version: 1.0 Status: Final 2. Introduction The EUXDAT End

This document is issued within the frame and for the purpose of the EUXDAT project. This project has received funding from the European

Union’s Horizon2020 Framework Programme under Grant Agreement No. 777549. The opinions expressed, and arguments employed herein

do not necessarily reflect the official views of the European Commission.

This document and its content are the property of the EUXDAT Consortium. All rights relevant to this document are determined by the

applicable laws. Access to this document does not grant any right or license on the document or its contents. This document or its contents are

not to be used or treated in any manner inconsistent with the rights or interests of the EUXDAT Consortium or the Partners detriment and are

not to be disclosed externally without prior written consent from the EUXDAT Partners.

Each EUXDAT Partner may use this document in conformity with the EUXDAT Consortium Grant Agreement provisions.

(*) Dissemination level.-PU: Public, fully open, e.g. web; CO: Confidential, restricted under conditions set out in Model Grant Agreement;

CI: Classified, Int = Internal Working Document, information as referred to in Commission Decision 2001/844/EC.

D5.2 EUXDAT e-Infrastructure

Keywords:

Data Analytics, Big Data, e-Infrastructure, Architecture, Design, EUXDAT

Document Identification

Status Final Due Date 31/10/2018

Version 1.0 Submission Date 06/11/2018

Related WP WP5 Document Reference D5.2

Related Deliverable(s) D3.2

D4.2

Dissemination Level (*) PU

Lead Participant Fabien Castel

(ATOS)

Lead Author Fabien Castel (ATOS)

Contributors Ugo LOPEZ,

Jessica

BRETAGNE,

Matthieu JEAN-

JACQUES,

François-Xavier

STEMPFEL

Reviewers Dimitrios Moshou,

CERTH

Walter Mayer, CoO

Document name: D5.2 EUXDAT e-Infrastructure Page: 2 of 38

Reference: D5.2 Dissemination: PU Version: 1.0 Status: Final

Document Information

List of Contributors

Name Partner

Fabien CASTEL Atos FR

Ugo LOPEZ Atos FR

Jessica BRETAGNE Atos FR

Matthieu JEAN-JACQUES Atos FR

François-Xavier STEMPFEL Atos FR

Document History

Version Date Change editors Changes

0.1 03/10/2018 Fabien Castel

(ATOS FR)

Table of Contents


(ATOS FR)

Updated table of content


(ATOS FR)

Version for partner review


(ATOS FR)

Updated version with outcomes from Basel

consortium meeting and CERTH review


(ATOS FR)

Version for quality review

0.6 06/11/2018 ATOS ES Quality

FINAL 06/11/2018 FINAL VERSION TO BE SUBMITTED

Quality Control

Role Who (Partner short name) Approval Date

Deliverable leader Fabien Castel (ATOS FR) 05/11/2018

Technical manager Fabien Castel (ATOS FR) 05/11/2018

Quality manager Susana Palomares (ATOSES) 06/11/2018

Project Manager Javier Nieto (ATOSES) 06/11/2018



Table of Contents

Document Information ............................................................................................................................ 2

Table of Contents .................................................................................................................................... 3

List of Tables ........................................................................................................................................... 4

List of Figures ......................................................................................................................................... 5

List of Acronyms ..................................................................................................................................... 6

1. Executive Summary ......................................................................................................................... 7

2. Introduction ...................................................................................................................................... 8

2.1 Relation to other project work ................................................................................................. 8

2.2 Structure of the document ....................................................................................................... 8

3. E-Infrastructure User Access ........................................................................................................... 9

3.1 Public APIs .............................................................................................................................. 9

Digital Elevation Model ...................................................................................................... 9

Sentinel data ........................................................................................................................ 9

3.2 Development environment .................................................................................................... 10

User Interface .................................................................................................................... 10

Notebook Examples........................................................................................................... 17

3.3 Frontend................................................................................................................................. 21

Methodology ..................................................................................................................... 21

Frontend Mock-up ............................................................................................................. 22

Partners Feedbacks ............................................................................................................ 26

4. Requirement Coverage ................................................................................................................... 28

4.1 Pilot specific requirements .................................................................................................... 28

Pilot 1: Land Monitoring and Sustainable Management ................................................... 28

Pilot 2: Energy efficiency analysis .................................................................................... 30

Pilot 3: 3D farming ............................................................................................................ 32

4.2 EUXDAT Platform requirements .......................................................................................... 33

5. Conclusions .................................................................................................................................... 37

6. References ...................................................................................................................................... 38



List of Tables

Table 1: Informational requirements .................................................................................................................... 28 Table 2: Functional/non-functional requirements ................................................................................................ 29 Table 3: Informational requirements .................................................................................................................... 30 Table 4: Functional/non-functional requirements ................................................................................................ 31 Table 5: Informational requirements .................................................................................................................... 32 Table 6: Functional/non-functional requirements ................................................................................................ 32 Table 7: Functional/non-functional requirements ................................................................................................ 33



List of Figures

Figure 1: DEM public API documentation page _________________________________________________ 9 Figure 2: Sentinel public API Swagger page ___________________________________________________ 10 Figure 3: Development environment login page _________________________________________________ 11 Figure 4: Single-sign-on delegated page ______________________________________________________ 11 Figure 5: Development environment waiting page _______________________________________________ 11 Figure 6: Development environment web interface ______________________________________________ 12 Figure 7: Administration Panel _____________________________________________________________ 12 Figure 8: File explorer panel _______________________________________________________________ 13 Figure 9: Tabs panel ______________________________________________________________________ 14 Figure 10: Notebook panel _________________________________________________________________ 15 Figure 11: Running panel __________________________________________________________________ 16 Figure 12: Terminal session ________________________________________________________________ 16 Figure 13: DEM notebook example step 1 _____________________________________________________ 17 Figure 14: DEM notebook example step 2 _____________________________________________________ 17 Figure 15: DEM notebook example step 3 _____________________________________________________ 18 Figure 16: DEM notebook example step 4 _____________________________________________________ 18 Figure 17: DEM notebook example step 5 _____________________________________________________ 19 Figure 18: DEM notebook example result _____________________________________________________ 19 Figure 19: Sentinel API notebook example step 1 _______________________________________________ 20 Figure 20: Sentinel API notebook example step 2 _______________________________________________ 21 Figure 21: Sentinel API notebook example step 3 _______________________________________________ 21 Figure 22: Login dialog. ___________________________________________________________________ 22 Figure 23: User workspace. ________________________________________________________________ 23 Figure 24: Services catalogue. ______________________________________________________________ 23 Figure 25: Service´s form. _________________________________________________________________ 24 Figure 26: History sidebar. _________________________________________________________________ 24 Figure 27: Service execution detail. __________________________________________________________ 25 Figure 28: Map viewer.____________________________________________________________________ 26 Figure 29: EUXDAT services overview _______________________________________________________ 27



List of Acronyms

Abbreviation /

acronym

Description

API Application Programming Interface

Dx.y Deliverable number y belonging to WP x

EC European Commission

HPC High Performance Computing

HTTP HyperText Transfer Protocol

JSON JavaScript Object Notation

NDVI Normalized Difference Vegetation Index

OGC Open Geospatial Consortium

OLU Open Land Use

REST Representational State Transfer

SAR Synthetic Aperture Radar

URL Uniform Resource Locator

WFS Web Feature Service, OGC standard for georeferenced object

WMS Web Map Service, OGC standard for georeferenced coverage

WP Work Package

WS Web Service



1. Executive Summary

This document goes along the first version of the EUXDAT platform, implemented during the first year

of the project. It aims at describing the ready to use features available on the EUXDAT e-infrastructure

from a user point of view rather than a technical and architectural point of view.

This document lists the different service that user can access on the end users' platform. First public

APIs to access specific datasets are described. Then the online development environment is presented,

and its user interfaces are detailed. The interaction between development environment and public APIs

are also highlighted through the presentation of use case examples. Lastly, a mock-up of the future

frontend application and its foreseen features are described.

The EUXDAT project comes at the end of its first step, with the release of the first online version. This

document aims at this point at providing a first assessment and recap of the requirement coverage status.

The technical and functional requirements defined in WP2 are extracted and a clear status is provided

for each of them.



2. Introduction

The EUXDAT End Users platform gathers a set of components that are directly accessed by the end

users. The objective of this document is to provide a description from the user point clear view of this

component, to describe what services are available and how they can be used. This document also aims

at giving an assessment of the requirement coverage at the end of the first step of the project constituted

by the first version of the e-Infrastructure

2.1 Relation to another project work

This document is articulated with D3.2 "End Users' Platform" and D4.2 "Infrastructure Platform" as all

together these documents provide insight on the actual status of the overall EUXDAT e-infrastructure:

• D3.2 describes the implementation of the end users' platform

• D4.2 describes the status of the infrastructure platform (for both cloud and HPC resources)

• D5.2 describes the status of the overall e-infrastructure focusing on the user point of view

2.2 Structure of the document

This document is structured in 2 major chapters

Chapter 3 presents the end users' platform component for the user point of view

Chapter 4 presents the requirement coverage



3. E-Infrastructure User Access

3.1 Public APIs

Digital Elevation Model

The DEM API can be accessed at http://<server_url>/dem/. The next figure shows the online

documentation.

Figure 1: DEM public API documentation page

It supports a single GET method used to get a subset of the DEM. The request must be sent to

http://<server_url>/dem/ /subset/{lat_min}/{lon_min}/{lat_max}/{lon_max} where the four

parameters are the bounds of the rectangle subset in EPSG:3035:

• lat_min is the minimum Y axis

• lon_min is the minimum X axis

• lat_max is the maximum Y axis

• lon_max is the maximum X axis

The output is returned as JSON data with the following top level keys:

- subset (the expected DEM array),

- nodata_value (the value for missing data),

- crs (the CRS of the subset).

Sentinel data

The API to access Sentinel data provides 3 methods:

• search

• download

• display (not fully implemented at the moment)



All methods are documented in a Swagger interface reachable at http://<server_url>/mundiapi/.

Figure 2: Sentinel public API Swagger page

The “search” method is used to look for all the products corresponding to a time and area of interest

settable by the user. This method needs all the following required parameters to be filled:

• latitude [-180, 180] and longitude [-90, 90] minimum and maximum in degrees

• date minimum and maximum, with “dd-mm-YYYY” format

• name of the satellite (Sentinel1, Sentinel2 or Sentinel3)

• index to start the list of product of the response (default value is 1)

The “download” method is used to download the product in GeoTIFF or jpeg2000 format to the local

workspace. This method needs the following two required parameters to be set:

• JSON metadata of the wanted product (as it is returned by the search method)

• path in the local workspace of the user

An optional parameter is used for Sentinel 2 data. As Sentinel 2 data contains several bands, the identifier

number for each band to return has to be provided by the user.

3.2 Development environment

User Interface

This section describes the basic features of the development environment from a user point of view. We

refer to the administration pages which is common to all users as the hub, and the actual development

environment of a user which is private, as the notebook.



When contacting the development environment URL, the user lands to the hub and is asked to sign in

into the EUXDAT infrastructure.

Figure 3: Development environment login page

He is redirected to the single-sign on system, where it enters its identifiers and credentials.

Figure 4: Single-sign-on delegated page

The user is then redirected to a wait page of the hub until its development environment is ready.

Figure 5: Development environment waiting page

The development environment is a web IDE with three main components:

• A menu bar

• A panel on the left containing a File explorer and other tools

• A tabbed work area that displays anything useful for development like code, terminals,

references, most types of file.



Figure 6: Development environment web interface

The user can access the administration panel from the notebook, selecting "Hub > Control Panel" in the

menu bar, to logout, stop its notebook server or logout from EUXDAT infrastructure.

Figure 7: Administration Panel

The left panel contains tools to navigate across the content of your IDE. The "Files" tab displays a file

navigator allowing the user to display the content of its filesystem. The root folder contains a "public"

folder where users have no writing permissions. It is intended to provide examples of script or other

useful resources. The user owns a "work" folder persisted across server IDE sessions where he can store

anything useful for its work.



Figure 8: File explorer panel

Common operations can be executed with right click. A set of actions is accessible through a shortcut

bar:

• New Launcher: Open a tab in the work area listing capabilities of the IDE. Creation of a new

python notebook, launch of a terminal or a python console, creation of text file...

• Create Folder

• Upload Files

• Refresh File List

A "Tabs" panel allows navigating quickly amongst the different tabs opened in your IDE.



Figure 9: Tabs panel

The main element of the IDE is the notebook document. It is composed of a set of cells where the user

can enter code, raw text or rich text using the Markdown format. Each code cell can be executed. The

resulting output is printed under the code cell. The comment cells can be edited in markdown language

and the resulting formatted text is compiled when the user starts the user command.



Figure 10: Notebook panel

The notebook document is linked to a kernel, i.e. a code interpreter session that executes the instructions

contained in the code cells. This kernel starts when a notebook is first opened, it can be restarted at any

time and managed through the "Running" tab of the tools panel.



Figure 11: Running panel

A terminal session can be started to install additional packages or use GDAL in command line. To

copy/paste in the terminal use "Maj + Right Click".

Figure 12: Terminal session



Notebook Examples

3.2.2.1 Digital Elevation Model

An example of how to use the Digital Elevation Model (DEM) API in a notebook is available in the

public folder: public/eu_dem/examples/dem_subset.ipynb

In this example, we get a subset of the whole dataset and display it over a map.

First step is to add the folder containing the API to the path. Then we can import the get_subset function

which takes a single parameter: the bounds of the subset in EPSG:3035 coordinates. They can be given

as a tuple or a list, in the following order: min y, min x, max y, max x. The output is a tuple containing:

- The subset as a NumPy array,

- The no data value which fills the pixels with missing values,

- The CRS of the resulting subset.

Figure 13: DEM notebook example step 1

The next cell converts the subset bounds from the native projection (EPSG:3035) to regular

latitude/longitude coordinates (EPSG:4326). This is done with the PyProj module.




The following step warps the original data from EPSG:3035 to EPSG:4326. We use GDAL Python

bindings for this purpose. We need to specify some parameters in order for GDAL to run this process:

the transform, the CRS and the no data value of the input array.


Next step is to use matplotlib to save the warped data with a specified colormap: here we replace no

data values with "NaN" values and we ask the "NaN" values to be displayed in blue and with 20%

opacity. We choose to use the predefined colormap which goes from green to yellow to red.


The last cell uses the python folium module to create a map and display our warped image on top of it.

The map only works with latitude/longitude coordinates which is why we had to warp the native data.




Figure 18: DEM notebook example result

3.2.2.2 Sentinel data

Mundi provides APIs to access Sentinel data through a catalogue. In particular, an OpenSearch API is

available. An intermediate API module has been developed in the frame of the EUXDAT project to

make the use of this API easier by hiding the complexity of making OpenSearch request over HTTP

(see [1] for more details on the implementation of this module).

A use case example of this API is implemented in a notebook available in the public folder to all users

of the development environment. This notebook allows users to look for Sentinel products

corresponding to their request (area and time of interest, product type), retrieve the metadata for one of

them (description of the product) and download them to its personal workspace.

First step is to run the first cell to have all the required python libraries.



Then, the user has to set all the parameters with right values (see the green rectangle on Figure 19)

depending on its needs. The starting index needs to be set to 1 if it is the first time the search request is

launched. That is then used as a pagination parameter to handle requests returning a large amount of

results. A list of product identifiers is then returned and displayed. A “nextIndex” parameter is also

displayed. It allows the user knowing which number to set for the “startingIndex” parameter in the next

search to have the following products for the same request.

Figure 19: Sentinel API notebook example step 1

On the next step, the user is able to see the metadata of a product by choosing an identifier in the

displayed identifier list (see Figure 20, an identifier needs to be pasted in the green rectangle). If the

requested product is not in the list, then another search request needs to be launched with another starting

index to have more displayed products.




Once the right product is selected and its metadata displayed, a download request can be launched. The

user will have to set two more parameters (see Figure 21the green rectangle on 19).


3.3 Frontend

Methodology

The platform´s user interface is an important component which allow the users interacting with the

service offered by the platform. It must fulfill the requirements described in the deliverable D2.1

"Description of Proposed Pilots and Requirements" [1]. These are high level requirements that do not

define the detailed features of this interface.

To better understand what the needs in terms of user interface are, a mockup has been developed, i.e.

GUI screens and panels on the client side with no corresponding features implemented on the server

side. This mockup is described in the next chapter and has been presented to partners during consortium

meeting in Basel on 23rd and 24th of October. Feedbacks have been gathered and are presented at the end

of the section.



Frontend Mock-up

The frontend´s mock-up can be divided in several components, the first one is the main navigation bar,

which shows the user profile, the contact point and the left menu handler (this menu can be hide for a

better display). The left menu shows all the application´s sections that can be accessed through this

menu. The image bellow (see Figure 22) shows the look and feel of the application and the login dialog.

Figure 22: Login dialog.

The ‘Workspace’ section (see Figure 23) is where the user will store its data. This data can be used as

an input of the services provided by the platform (the folder´s structure has to be defined in the future).

The workspace also will store the services´ output data. The user can manage their data and perform

several actions such as copy, remove, download and upload files into the platform.



Figure 23: User workspace.

The “Services” catalogue will show all the services offered by the platform (see Figure 24). The frontend

will provide a search engine which can evolve depending on the complexity of the service´s properties

and description. Every service will have a title and a brief description, if the user needs detailed

information about the service, a modal will be shown with all the information by clicking the “details”

button.

Figure 24: Services catalogue.



To execute a service, the user must click on the “run” button. A new page will be shown with a specific

form. The image bellow (Figure 25) shows a very basic example form (the inputs are related with the

service definition, so the form will be different for each service). The form must provide an appropriate

way to select the user data from their workspace as well as the Earth Observation data from a catalogue.

Figure 25: Service´s form.

The user will be able to see the history of executed services by clicking the ‘info’ button highlighted in

red (Figure 26). A sidebar will appear with all the executions made by the user, the list will show the

service identifier, the date and the execution status codified in different colors.

Figure 26: History sidebar.



For more detailed information about the service´s execution the user can click on the information button

(Figure 26). A new page will be shown with all the information about the service´s execution, the image

bellow shows an example (Figure 27).

This page is divided in three different parts. The first one shows all the information that can be interesting

for the user such us the service identifier, the disk usage and the progress. The user can also delete the

execution or either re-run the service. The second one shows the service´s output files in the workspace,

here the user will see easily the output generated by the service and will be able to perform the same

actions as the workspace section. At the bottom the user can see the service´s execution logs in real time,

this way the user can monitor the progress of the service.

Figure 27: Service execution detail.

Thanks to the “Data Visualization” section the user will be able to display all the layers generated by

the services into a map. The map will support all the OGC standards such as WMS, WFS... This way it

will be easy to interact with the platform backend. The map will provide all the features needed for the

user to manage the layers, perform geospatial operations, extract information, change the base map, etc.



Figure 28: Map viewer.

Partners Feedbacks

The main feedback about the user interface mock-up is that it is designed for scientists. It allows users

configuring and running algorithms and then exploiting the results through a generic visualization panel.

It is not very clear which class of user will use these features:

• Service developers does not need such advanced features as they have their own way to execute

their algorithms and visualize their data. Basic ways to interact with the platform should be

sufficient, through command lines and technical tools (orchestrator GUI, Jupyter development

environment…).

• Applications built on top of EUXDAT just needs APIs to access added value data.

• End users of these applications needs GUI more focused on a specific use case. This class of

users are not willing to run and configure complex algorithm with a generic tool, they want to

access the information they need as quickly and efficiently as possible.

The following diagram represents the outcome of the discussions triggered by the mockup presentation

and illustrate the global vision of the services offered by EUXDAT.

• On the bottom layer, data sources are represented. These are connectors that enable the

EUXDAT applications accessing the basic data they need for their analysis. This data can be

free and open or “for-fee” data, but they are available on internet to anyone. Connectors can be

APIs developed in the frame of EUXDAT (for EU-DEM data for instance) or existing APIs that

EUXDAT applications use just as consumers.

• The layer above contains the EUXDAT applications, i.e. pilots/scenario applications described

in WP2 deliverables. Those are the components using basic data as inputs and data processing

algorithms to generate EUXDAT specific added value datasets. This layer is in charge of

generating and storing these datasets. It also contains the data catalogue, referencing both basic

and added value datasets, and technical GUI used by the applications developers mainly to

execute applications on the platform (orchestrator, Jupyter development environment...).

• These added value datasets stored on the platform are served to users through an API layer.

These APIs should be as standardized as possible. Defining the right standards to use between



geospatial standards (OGC, INSPIRE), IoT standards or other standards will be key to make

APIs in line with the user expectations.

• The top-level layer contains all the graphical interface dedicated to end users. Several pilot

specific frontend interfaces are provided, with features focused on a well-defined use case. A

general graphical interface is provided by the platform. It is focused on a marketplace and a data

catalogue for users to browse the available services (APIs and specific frontend applications)

and datasets (basic and added value datasets).

Figure 29: EUXDAT services overview

Added value data API layerStandardized API

(OGC, INSPIRE, IoT...)OLU

Crop Status Monitoring

Climatic zones

Application layerProcessing applications

Data storage

User Frontend layerUser interfaces

Data VisualizationLand Use

Crop Monitoring

Climatic zones

Open Land Use Map improvement

Infection index computation

Climatic map generation

OLU database

Data sources layerEU-DEM Sentinel Meteorological

Stored index maps Climatic maps

Jupyter

Cloudify

Data catalogue API

Marketplace & Data Catalogue

UAV ...

Data Catalogue

...



4. Requirement Coverage

The following section aims at providing a first assessment of the requirement coverage at the end of the

first step of the project constituted by the first version of the e-Infrastructure.

The requirements are retrieved from D2.1 deliverable [1]. The same overall structure of this document

is kept: first the pilot specific requirements, with informational requirements (targeted datasets) and

functional requirements, and then the general requirements related to the platform itself.

4.1 Pilot specific requirements

Pilot 1: Land Monitoring and Sustainable Management

4.1.1.1 Informational requirements

Table 1: Informational requirements

ID Name Format Size Priority Covered Comment

EUXDAT-

REQ-Pilot1-

DATA-001

Level-1C multi-

spectral imaging

products from the

Sentinel-2

Raster

(Geotiff) Gb High Yes

EUXDAT-

REQ-Pilot1-

DATA-002

UAV-enabled

hyperspectral

imagery

Raster

(Geotiff)

Tb

Medium No

EUXDAT-

REQ-Pilot1-

DATA-003

Climate data Raster Gb High No

EUXDAT-

REQ-Pilot1-

DATA-004

Sentinel-1 GRD data

products from the

Sentinel-1 available

at field scale

Raster Gb Medium Yes

EUXDAT-

REQ-Pilot1-

DATA-005

Dynamic cropland

mask, crop type map

and LAI from Sen2-

Agri system

Raster Gb High No



ID Name Format Size Priority Covered Comment

EUXDAT-

REQ-Pilot1-

DATA-006

Copernicus

European Digital

Elevation Model

(EU-DEM), version

1.1

Raster

(Geotiff) Gb High Yes

EUXDAT-

REQ-Pilot1-

DATA-007

Land use map Raster Gb High Partially

Components

for the OLU

service are

deployed but

the actual

service still

needs to be

implemented

EUXDAT-

REQ-Pilot1-

DATA-008

Soil map Raster Gb High Yes

4.1.1.2 Functional/non-functional requirements

Table 2: Functional/non-functional requirements

ID Name Format Priority Covered Comments

EUXDAT-

REQ-Pilot1-

001

Atmospheric

correction of

Multispectral Sentinel

bands

Data

processing Medium No

EUXDAT-

REQ-Pilot1-

002

Calculation of NDVI

from the 12 Sentinel

multispectral bands

Data


EUXDAT-

REQ-Pilot1-

003

Calculation of

Hyperspectral indices

relevant for stress and

disease

Data





EUXDAT-

REQ-Pilot1-

004

Availability of

Sentinel-2 and

Sentinel-1m data at

field scale/for a given

polygon

Data

Management

Data

Processing

High Partially

A WMS service has

been tested on the

Mundi platform and

could fulfil this

requirement

EUXDAT-

REQ-Pilot1-

005

Availability of the

time-series of

Sentinel-1 and

Sentinel-2 data for a

given pixel/given

polygon.

Data

Management

Data

Processing

Medium No

EUXDAT-

REQ-Pilot1-

006

Dynamic data

analyses of EO data

and in-situ data

Data

analytics High No

EUXDAT-

REQ-Pilot1-

007

2D visualization of

time-series over

selected pixels,

provision of

interfaces, toolkits

Data

Visualization

and User

Interaction

High No

EUXDAT-

REQ-Pilot1-

008

Installation of

Sen2Agri system and

provision of Dynamic

cropland mask, crop

type map and LAI

Cloud and

High

Performance

Computing

(HPC)

High No

Pilot 2: Energy efficiency analysis



ID Name Format Size Priority Covered Comments

EUXDAT-

REQ-Pilot2-

DATA-001

Open Land Use Map Shapefile,

RDF GBs Medium Partially

Components

for the OLU

service are

deployed but

the actual




service still

need to be

implemented

EUXDAT-

REQ-Pilot2-

DATA-002

Land Parcel

Identification System

(LPIS)

Raster GBs Medium No

EUXDAT-

REQ-Pilot2-

DATA-003

Copernicus Sentinel 2

data GeoTiff TBs Medium Yes

EUXDAT-

REQ-Pilot2-

DATA-004

EU-DEM GeoTiff GBs High Yes




EUXDAT-

REQ-Pilot2-

001

Collecting machinery

tracking data

Data

Management High No

EUXDAT-

REQ-Pilot2-

002

Collecting of agro

meteorological data

Data

Management High No

EUXDAT-

REQ-Pilot2-

003

Calculation of yield

productivity zones

Data

Analytics High No



Pilot 3: 3D farming




EUXDAT-

REQ-Pilot3-

DATA-001

EU-DEM Raster

(Geotiff) GBs High Yes

EUXDAT-

REQ-Pilot3-

DATA-002

Hydrology for

EU Unknown

MBs-

GBs Medium No

EUXDAT-

REQ-Pilot3-

DATA-003

Actual weather Unknown Unknown Unknown No

EUXDAT-

REQ-Pilot3-

DATA-004

Historic weather Unknown Unknown Unknown No




EUXDAT-

REQ-Pilot3-

001

Zone related

morphometric

statistic

Cloud and High

Performance

Computing (HPC)

Data Analytics

High

No

EUXDAT-

REQ-Pilot3-

002

Water influence to

weather

conditions

Cloud and High

Performance

Computing (HPC)

Data Analytics

Medium No

EUXDAT-

REQ-Pilot3-

003

3D visualization Data Visualization

and User Interaction High No



4.2 EUXDAT Platform requirements



EUXDAT-

REQ-PLATF-

001

Support for

various HPC and

Cloud providers

Cloud and High-

Performance

Computing (HPC)

High Partially

A cloud platform is

deployed and

reachable through

APIs and the

development

environment

EUXDAT-

REQ-PLATF-

002

Monitor HPC and

Cloud resources

Cloud and High-

Performance

Computing (HPC)

High No

EUXDAT-

REQ-PLATF-

003

Applications

monitoring and

profiling

Cloud and High-

Performance

Computing (HPC)

Medium No

EUXDAT-

REQ-PLATF-

004

Adequate

operation of the

platform

Cloud and High-

Performance

Computing (HPC)

Low No

EUXDAT-

REQ-PLATF-

005

Optimize data

movement

Cloud and High-

Performance

Computing (HPC)

Data Management

High No

EUXDAT-

REQ-PLATF-

006

Support security

and privacy in

data management

Cloud and High-

Performance

Computing (HPC)

Data Management

Data Protection and

Security

Medium No

EUXDAT-

REQ-PLATF-

007

Automated

deployment and

execution of

applications

Cloud and High-

Performance

Computing (HPC)

Data Management

Medium No

EUXDAT-

REQ-PLATF-

008

API access to

pilots' data and

services

Data Management High Partially

APIs are available

for a partial set of

targeted data




EUXDAT-

REQ-PLATF-

009

User management Data Protection and

Security High Yes

EUXDAT-

REQ-PLATF-

010

Access sensor

observations

Data Management

Standards High No

EUXDAT-

REQ-PLATF-

011

Support

information

modelling

Data Management

Standards High No

EUXDAT-

REQ-PLATF-

012

Support

integration of

meta-information

Data Management

Standards High No

EUXDAT-

REQ-PLATF-

013

Compliance with

INSPIRE

specifications

Standards

Data Management High No

EUXDAT-

REQ-PLATF-

014

Compliance with

GEO/GEOSS

specifications

Standards

Data Management High No

EUXDAT-

REQ-PLATF-

015

Integrate Web

map services

Standards

Data Visualization

and User Interaction

Data Management

High No

EUXDAT-

REQ-PLATF-

016

Multiple Data

Centers in the

Cloud

Cloud and High-

Performance

Computing (HPC)

Data Management

Data Protection and

Security

Medium No

EUXDAT-

REQ-PLATF-

017

Cloud Data

Storage

Cloud and High-

Performance

Computing (HPC)

Data Management

Data Protection and

Security

High No

EUXDAT-

REQ-PLATF-

018

Dependability

Cloud and High-

Performance

Computing (HPC)

High No




EUXDAT-

REQ-PLATF-

0219

Big Data

Management

Cloud and High-

Performance

Computing (HPC)

Data Management

High No

EUXDAT-

REQ-PLATF-

020

Identity

Management &

Access control

Data Management

Data Protection and

Security

High Yes

EUXDAT-

REQ-PLATF-

021

Scalability –

Users growth

Cloud and High-

Performance

Computing (HPC)

Data Management

High No

EUXDAT-

REQ-PLATF-

022

Scalability – Data

growth and

complex analytics

Cloud and High-

Performance

Computing (HPC)

Data Management

Data Processing

Data Analytics

High No

EUXDAT-

REQ-PLATF-

023

Data

decentralization

Cloud and High-

Performance

Computing (HPC)

Data Management

Medium No

EUXDAT-

REQ-PLATF-

024

Parallel data

stream processing

Cloud and High-

Performance

Computing (HPC)

Data Management

Data Processing

Data Analytics

Medium No

EUXDAT-

REQ-PLATF-

025

Reduction in

energy

consumption by

improved

processing

algorithms

Cloud and High-

Performance

Computing (HPC)

Low No

EUXDAT-

REQ-PLATF-

026

Use of efficient

hybrid

architectures

Cloud and High-

Performance

Computing (HPC)

High No




EUXDAT-

REQ-PLATF-

027

Visualization of

large amounts of

data

Data Management

Data Analytics

Data Visualization


High No

EUXDAT-

REQ-PLATF-

028

Support of

different formats

for visualization

Data Visualization


Standards

High No

EUXDAT-

REQ-PLATF-

029

Provide rich user

interfaces for the

interactive

visualization

Data Visualization


EUXDAT-

REQ-PLATF-

030

Render high

resolution data in

N arbitrary

dimensions

Data Visualization


EUXDAT-

REQ-PLATF-

031

Personalised end-

user-centric

reusable data

visualisation

Data Visualization

and User Interaction Medium No

EUXDAT-

REQ-PLATF-

032

Detection of

abnormal sensor

measurements

Data Management

Data Processing Medium No

EUXDAT-

REQ-PLATF-

033

Use of high

performance

computing

techniques to the

processing of

extremely huge

amounts of data

Cloud and High-

Performance

Computing (HPC)

Data Management

Data Processing

Data Analytics

High No

EUXDAT-

REQ-PLATF-

034

Heterogeneous

data aggregation

and normalization

Data Management

Data Processing

Data Analytics

Data Visualization


Standards

High No

EUXDAT-

REQ-PLATF-

035

Verification of

data integrity and

veracity

Data Management

Data Processing Low No



5. Conclusions

This document has presented all the user components deployed in the first version of the EUXDAT end

users' platform.

This first version will serve as a basis for the next developments to come. In particular, APIs to access

other kind of data (meteorological, UAV, thematic land data) will be integrated on the next step of the

project.

The cloud and HPC processing capabilities have still to be fully exploited. In particular, the link between

cloud and HPC environment is a big construction work that will start to be implemented in for the next

version of the e-infrastructure.

Next to the development environment, a frontend component will be implemented providing high level

and easier to use features than the development environment, in particular for users having low technical

background.



6. References

[1] EUXDAT; "D2.1 Description of Proposed Pilots and Requirements”; Jedlička, Karel; 2018.

[2] EUXDAT, "D3.2 End Users' Platform", Fabien Castel, 2018

Documents

D5.2 EUXDAT e-Infrastructure · Document name: D5.2 EUXDAT e-Infrastructure Page: 8 of 38 Reference: D5.2 Dissemination: PU Version: 1.0 Status: Final 2. Introduction The EUXDAT End