Grid-based System for Flood Forecasting

Cracow Grid Workshop ’03, 27-29.10.2003

Grid-based System for Flood Forecasting

Ladislav Hluchy

Institute of Informatics SAS

in co-operation with Water Research Institute, Vah River Authority and Slovak Hydrometeorological Institute

Slovakia

[email protected]

http://www.cultivate-europe.org/funding.html


Outline

• Introduction

• Flood Forecasting

• Grid infrastructure for Flood Forecasting

• Use cases

• Grid-based Implementation

• Results

• Conclusion



Flood Forecasting one of the Geospatial Applications

• Applications that use data from Geographic Information System (GIS)

• Typical applications: flood forecasting, fire simulations, environmental risk management etc.



Flood Forecasting

• Topical problem: floods have caused widespread damages in the recent years

• Common interest: many countries threatened• Many potential users: governments, flood crisis

teams, insurance companies, public,• Requires Grid technology



ANFAS ArchitectureModelling ServerDatabase Server

ANFAS corecomponent

HTTP ProtocolHTTP Protocol

Data Preparation

ANFAS exchange bus (Middleware : RMI, CORBA, Http)

ANFAS bus extension (CORBA / RMI)

Model AccessComponent

Model AccessComponent

Model ManagerModel ManagerDataManager

DataManager

FileExchange

Component

FileExchange

ComponentAdministration

& user manager

Administration & user manager

ApplicationServer

ApplicationServer

Map ServerMap Server

GIS Accesscomponent

GIS Accesscomponent

GIS ArcView

GIS ArcView

ANFAS bus extension (CORBA / RMI)

Co-operative, ExplanatoryComponent

Co-operative, ExplanatoryComponent

ANFAS Client

Modellingpreparation

Modellingpreparation

Modelling activation

& follow-up

Modelling activation

& follow-upResults

exploitation

Resultsexploitation

NumericalModel

NumericalModel

ANFAS



Integration to ANFAS core

Machine 1ANFAS Core Server

currently hosted by EADS-MS&I

Web

Machine 2 hostingRPS Controller

i-cluster

LAN

Use of the i-clusterin the ANFAS system

ANFAS



SMS/FESWMS • FESWMS has been developed under funding by the U.S. Federal

Highways Administration (FHWA) • FESWMS is specifically suited for modeling regions involving flow

control structures, such as are encountered at the intersection of roadways and waterways. Specifically, the FESWMS model allows the user to include weirs, culverts, drop inlets, and bridge piers into a standard 2D finite element model.

• As there is highway planned at the Vah River pilot site in Slovakia, the choice of FESWMS model is important

• SMS provides graphical tools for defining these structures and controlling analysis using the FESWMS model. Both pre- and post-processing capabilities are included in the interface.

ANFAS



Detailed FESWMS structures

Input files

Finite element

Nonlinearsolver

write solutionto the file

Solution file

Updatesolution

Generating matrix

Linear solver

OK

Nonlinear solverSolution schema

ANFAS



Parallel matrix generation

Generating partial matrix



PARALLEL LINEAR SOLVER

Updating solution

Updating solution

Updating solution

ANFAS



Implementation issuesReal flood modeling software is much more complicated than its mathematical model:

– Mathematical model of flood is well-known (partial differential equations finite elements nonlinear solver linear solver)

– Real software has to deal with• Input processing: different types input data, different variations of each type,

different formats of each variation• Special cases: wetting/drying, raining/evaporation, special constructions

(bridges, dams, culverts), wind effect, …• Calibration of results• Graphical user interface (GUI), visualisation• Error checking, documentation

As the result, source code of real software may be hundreds times longer than source code of mathematical model

ANFAS



Remote processing

Pre-processing

Post-processing

Processing input data

Save solutions

Parallelcomputational

kernel R

emote processing

ANFAS



Planned highway in the Váh pilot site

ANFAS



Main part affected by highway

LIDAR+highway position

Bytca city

Predmiervillage

ANFAS



Predmier village in orthophotomap

ANFAS



Predmier village in LIDARANFAS



TIN network at PredmierANFAS



Scenario: Water level for current terrain situation (Q-100-year)

Water depth

ANFAS



Scenario: Water level for highway without bridges (Q-100-year)

Water depth

Water level is about 70cm

higher than for situation without

highway

ANFAS



Scenario: Water level for highway with 2 bridges (Q-100-year)

Water depth

Water level is about 30cm

higher than for situation without

highway

ANFAS



Why Grid?

• Cooperation: requires cooperation between many organizations (meteorological institutes, river authorities) from many countries

• Data management: needs large amount of data of different sources, different owners, different countries, different access right

• Computation power: forecasting require large computational power for modeling and simulation



Virtual Organization

• Purpose– Shared data and computational power for flood

forecasting– Cooperation between users for flood forecasting

• Requirements– Identify and define clear relationships between users– Authentication: certificate authorities– Authorization: access right for each data/resources– Collaborative tools– Security



Flood Forecasting VO

Storage systems

databases

surface automatic meteorological and hydrological stations

systems for acquisition and processing of satellite information

meteorological radars

External sources of informationGlobal and regional centers GTSEUMETSAT and NOAAHydrological services of other countries

Data sources

meteorological models

hydrological models

hydraulic models

High performance computers

Grid infrastructure

Flood crisis teams meteorologistshydrologistshydraulic engineers

Users

river authoritiesenergyinsurance companiesnavigation

mediapublic



Virtual Organization for Fire Simulation

Storage systems

databases

GISFuel type- vegetation- canopy cover

GISTopography-elevation-slope

Meteorological data source

Weather

- wind direction, speed

- temperature, rel. humidity

FIRE MODELDescriptive Numerical Parameters

Data sources

Grid infrastructure

Users

Fire ManagementCreation Decision Support system, prevention-Terrain, resources, - capacities

Fire suppression authorities- training- operation mode

Ecosystem authorities Universities,Insurance companies


Fire Modelling SystemFARSITE



Data management

• Typical data: satellite images, radar images, measured data from hydrological stations, topographical data, historical data, simulation results

• Different formats, different quality, different owners, different access right

• Metadata server:data description, security, replication



FloodVO data transfer

Users


Data sources

Storage systems

Databases

surface automatic meteorological and

hydrological stations

systems for acquisition and processing of

satellite information

meteorological radars

External sources of informationGlobal and regional centers GTSEUMETSAT and NOAAHydrological services of other countries

meteorological models

hydrological models

hydraulic models

Grid infrastructure

Flood crisis teams meteorologistshydrologistshydraulic engineers

river authoritiesenergyinsurance companiesnavigation

mediapublic



DataGrid

• EDG Replica Manager• EDG Local Replica Catalogue• EDG Replication Metadata Catalogue• EDG Replica Optimization Service



Storage Element

Storage Element

Storage Element

Storage control

Metadata

EDG RM

EDG RMC

EDG ROS

EDG LRC EDG LRC EDG LRC

DataGrid (cont.)



Grid computing

• Many multidisciplinary simulations are needed for flood forecasting

• For critical situations, short response times are very important

• Numerical simulations are computationally intensive

• Grid can offer the necessary computational power



Visualization

• Data are stored in many different formats• Unified visualization tools may simplify the user-

interface• Many data for flood forecasting has spatial

character

=> GIS software may be used as the unified visualization tool



Portal

• The unified user-interface• Allow users access to the VO remotely• Simple requirements on clients - based on

standard Web technologies



3 current portals

1. Based on GridPort using Globus grid toolkit

2. Based on Jetspeed portal framework using DataGrid/CrossGrid services

3. Migrating Desktop - java fat client using DataGrid/CrossGrid services



GridPort

• A set of Perl scripts that enable Perl based portal (its CGI scripts) to use grid services of underlying Globus toolkit

• Wraps Globus’ command line tools

• Provides session management

• Provides no additional portal infrastructure



Architecture of GridPort based portal

Portal(Apache

web server)

StorageG

ridP

ort

tool

kit

Glo

bus

tool

kit

(GSI

, MD

S,

JobM

anag

er,

Gri

dFT

P, …

)

Resource 1

Resource 2

Resource n

…

Storage & Portal Machine

User’s web browser

User’s web browser

…



GridPort screenshot



Job submission in Flood-VO

XML file (parameter description)

Config. file (default values of parameters)

New config file

Job script file

globus-job-submit machine job_script config_file



Flood-VO: Job list



Flood-VO: Field data

II SAS

SHMI

RDBMS

Time Value

00:00:00 102.00000 cm

01:00:00 126.00000 cm

02:00:00 103.00000 cm

03:00:00 80.00000 cm

04:00:00 70.00000 cm

05:00:00 65.00000 cm



Jetspeed

• Portal framework

• Server-side Java based engine (application server)

• Client services are plugged using software components called portlets.

• User can arrange portlets – position, size, visibility



Jetspeed - architecture



Application portal screenshot (Jetspeed)



Application Portal



Migrating Desktop (MD)

• Java application (applet) running at the client computer

• Provides interface to all basic grid services (authentication, job

management, file management)

• Application specific job parameter input and job submission is

supported via application plug-ins

• has built-in viewer for common picture formats (jpeg, gif, png) and

text files, advanced visualization of results via application specific

visualization plug-in

• Being developed in the context of the CrossGrid project



Migrating Desktop

Screenshot of MD with Job submission wizard dialog



Use case: Cascade simulationData sources

Hydrological simulation

Hydraulic simulation

Portal

Meteorological simulation



Model characteristics

• ALADIN (meteorological model)– Limited area model– Operated by 13 Euro-Mediterranean countries– ALADIN/SLOVAKIA operated by SHMI– More than 1M lines of source code (mainly F90)– Developed for 64 bit big-endian architecture– Proprietary - requires nondisclosure agreement




• ALADIN (meteorological model)– Type: MPI parallel task, possible parameter studies –

multiple executions– CPU time: approximately one hour on 8 processors– I/O size: 33/180 MB per run– Scalability: on fast Ethernet up to 8 processors– Input data: boundary conditions– Output data: quantitative precipitation forecast,

temperature




• HSPF (hydrological model)– Type: sequential task, multiple executions (high

throughput computing)– CPU time: very small (seconds - minute)– I/O size: 1-10 MB– Scalability: HTC– Input data: quantitative precipitation, temperature,

topographical data– Output data: hydrograph




• FESWMS (hydraulic model)– Funded by US Federal Highway Administration– Distributed in commercial package SMS by EMS-I– Source code available (direct cooperation with

developer)– Optimized and parallelized by II SAS




• FESWMS (hydraulic model)– Type: MPI parallel task, multiple executions with

different input data– CPU time: 10min to several hours per a task– I/O size: 10-100 MB– Scalability: good for smaller number of processor (to

16). – Input data: inflow, topographical data– Output data: water levels and velocities



DaveF model

• A time-explicit finite-volume model from the same developers as FESWMS. It is considered as the complement of FESWMS and it is best suitable for unsteady state with critical or super-critical flow (dam-breaking, flash flood, flood with wetting/drying in large expanses)

• DaveF uses the same graphical environment like FESWMS (SMS) and similar input/output format =>can be easily added into ANFAS system

• Parallel version of DaveF has been developed for clusters by II-SAS and shows good results



Architecture

PortalComputing

element

Resource broker

Storage element



Login to FloodVO

PortalComputing

element

Resource broker

Storage element



Choose simulation

PortalComputing

element

Resource broker

Storage element



Enter input parameters

PortalComputing

element

Resource broker

Storage element



Visualization

PortalComputing

element

Resource broker

Storage element



Visualization

PortalComputing

element

Resource broker

Storage element



Download simulation results

PortalComputing

element

Resource broker

Storage element



Cascade simulation

PortalComputing

element

Resource broker

Storage element

Call meteorology master scriptRun meteorological simulationExtract hydrological input from resultsCall hydrology master script Run hydrological simulationCheck results if (inflow > critical flow)If yes: call hydraulics master script run hydraulic simulation



Cascade simulation

PortalPortalComputing

element

Resource broker

Storage element

Resource broker

Computing element

Computing element

Storage element

Portal

Resource broker

Storage element




Cascade simulation

PortalComputing

element

Resource broker

Storage element




Cascade simulation

PortalComputing

element

Resource broker

Storage element




Cascade simulation

PortalPortalComputing

element

Resource broker

Storage element

Resource broker

Computing element

Computing element

Storage element

Portal

Resource broker

Storage element




Cascade simulation

PortalComputing

element

Resource broker

Storage element




Cascade simulation

PortalComputing

element

Resource broker

Storage element




Dynamic flood simulation for t=315 minutes






























Dynamic flood simulation for t=1905 minutesthe maximum water level















Dynamic flood simulation step 1 time 0



Dynamic flood simulation step 3 time 0:30















Future work

• Adding more models • Performance analysis and optimization• Improving data management (repositories)• Adding more information about data (metadata)• Adding collaborative tools



Use case: Simulation sequence

Hydrology

Hydrology

Hydrology

Hydrology

HydrologyHydrology

Hydraulics

MeteorologyCZ Meteorology

SK

MeteorologyA

MeteorologyD

MeteorologyH

MeteorologyCH



Danube River basin

Experts

Data providers

Navigation

Computingcenters

Flood crisisteams

Riverauthorities

MediaPublic

Virtual Organizationfor Flood Forecasting

FloodGridFloodGridEnergy

Insurancecompanies

simulations

man

agem

ent

information



Knowledge based Flood forecasting

Data Sources

Visualization/Output Processing

MeteorologicalSimulations

HydrologicalSimulations

HydraulicSimulations

Metadata & Knowledge Repository

Automatic or Semiautomatic

ReasoningExternalResources(Web services)

Grid infrastructure

Simulation Metadata (parameters, area, sim. method, etc)

User interaction(Semiautomatic)

Job Submission (based on evaluation of previous jobs outputs)

Information aboutjob run (relevance)

Actual data fromobservation stations(relevance)


Documents

Grid-based System for Flood Forecasting