Open Source Software as a construction set forcomplex digital models of technical systems.

Matvey Kraposhin

Ivannikov ISP RAS Open Conference � 2019

6 December, 2019

1/40

Overview1 Introduction

2 Finite volume analysis

3 Finite element analysis

4 Finite di�erence/AMR analysis

5 Lagrangian methods

6 High order analysis

7 Mixed methods libraries

8 Optimization software

9 Model reduction

10 Coupling software

11 Meshing software

12 Visualization

13 Integration platforms

14 Problem-oriented libraries

15 OpenFOAM forks

16 Demanded skills

17 Conclusions

2/40

Typical programmer's CV

3/40

Modern challenges

The majority of today's industrial & scienti�c problems are:

• Multiscale

• Multiphysical

• Resource-consuming (computational, human, time)

• Not solved only experimentally or analytically

• Interdisciplinary

Proprietary software:

• High costs

• Slow development cycle

• A sink of expertise from customer to developer

Alternative: Open-Source software

4/40

What does an Open-Source Softwareo�er?

• OSS can be regarded as an construction set

• Usage of open source sofrware from prosperous developers acts as asource of knowledge and experience

• Source code could be literally regarded as natural language

• Conservation (capitalization) of experience inside the team

5/40

OSS as an construction set

• Which models are present now?

• How could they be used?

• Which demands are imposed on the competences of engineer?

6/40

Finite volume analysis, I

General-purpose FVM unstructured CFD code, GPLv.3

https://github.com/OpenFOAM

A generalized unstructured massively parallellow Mach �ow code, Apache License v.2.0,https://github.com/NaluCFD/Nalu

An Open-Source Suite for MultiphysicsSimulation and Design, LGPL v.2.1https://github.com/su2code/SU2

A fundamentally research platform for R& D in the �eld of high-�delity (WENO)Computational Fluid Dynamics under GPL v.3.0https://github.com/ucns3d-team/UCNS3D

7/40

Finite volume analysis, II

Free Software program for the solution of partialdi�erential equations on adaptive Cartesianmeshes, published under GNU GPL licensehttp://basilisk.fr/

The fast Finite Volume simulator with UQsupport, published under GNU GPL v.3.0 licensehttps://github.com/alsvinn/alsvinn

8/40

Finite element analysis, I

CalculiX is a package designed tosolve �eld problems using �nite elementmethod. It is published under GPL license.http://www.calculix.de/

Parallel �nite element unstructured meshes generalpurpose library, published under BSD-3-Clausehttps://github.com/SCOREC/core

General Purpose Finite Element EmbeddedLanguage and Library in C++, published underLGPL https:

//github.com/feelpp/feelpp

9/40

Finite element analysis, II

Fluidity is an open source, general purpose,multiphase computational �uid dynamicsFEM code, published under LGPL v.2.1https://github.com/FluidityProject/

fluidity

A C++ software library supporting the creation of�nite element codes and an open community ofusers and developers, published under LGPL v.2.1https://github.com/dealii/dealii

A set of computational tools that enablesresearch & numerical analysis in �uiddynamics employing asynchronous parallelprogramming, published under own licensehttps://github.com/quinoacomputing/

quinoa

10/40

Finite element analysis, III

Elmer is the computational FEM tool formulti-physics problems developed by CSC incollaboration with Finnish universities, researchlaboratories and industry, licensed under GPLhttps://github.com/ElmerCSC/elmerfem

General-purpose FEM code for mechanical,vibroacoustical and termal analysisof structure, released under GPLhttps://github.com/ralic/Code_Aster

is an implicit, unstructured grid, �nite element codefor the solution and analysis of multiphysicsproblems, released under own OSS licensehttps://github.com/SNLComputation/Albany

Multiphysics Object-Oriented SimulationEnvironment � An open-source, parallel �niteelement framework, released under GNU LGPLhttps://github.com/idaholab/moose

11/40

Finite di�erence/AMR analysis

AMReX is a software frameworkcontaining all the functionality to writemassively parallel, block-structured adaptivemesh re�nement (AMR) applicationshttps://github.com/AMReX-Codes/amrex

IBAMR

is a distributed-memory parallel implementationof the immersed boundary (IB) method withsupport for Cartesian grid adaptive meshre�nement (AMR), published under it's own licensehttps://github.com/IBAMR/IBAMR

Structured Adaptive Mesh Re�nement ApplicationInfrastructure, published under GNU LGPLhttps://github.com/LLNL/SAMRAI

12/40

AMReX Family

Code suites:

• AMReX-Astro

• AMReX-Combustion

• AMReX-FHD

Individual codes:

• Castro

• IAMR

• MAESTROeX

• MFIX-Exa

• Nyx

• WarpX

• PeleLM

• PeleC

https://amrex-codes.github.io/

13/40

SAMRAI Applications

ALE-AMR multi-material detonation front

calculation

An electronic structure calculation of Be4

cluster

Three-dimensional AMR simulation of

cardiac �uid dynamics in the human heart

The use of AMR to resolve tip vortices of

an isolated V-22 rotor14/40

Lagrangian methods, I

Open-source implementation of vortexmethod for computational hydrodynamics ofviscous incompressible �uid, GNU GPL v.3.0https://github.com/vortexmethods/VM2D

Discrete Element Method Particle SimulationSoftware. It can be used for the simulation ofparticulate materials, and aims to for applicationsit to industrial problems, GNU GPL v.2.0https://github.com/CFDEMproject/

LIGGGHTS-PUBLIC

is Open Source software for particle-basednumerical modelling, implementing the DiscreteElement Method (DEM), for modellingprocesses involving large deformations, granular�ow and/or fragmentation, Apache Licencehttps://launchpad.net/esys-particle

15/40

Lagrangian methods, II

C++/CUDA/OpenMP based SmoothedParticle Hydrodynamics (SPH)Solver, released under GNU LGPL,https://github.com/DualSPHysics/

DualSPHysics

PySPH is a framework for SmoothedParticle Hydrodynamics (SPH) simulationsimplemented in Python and the performancecritical parts are implemented in Cythonand PyOpenCL, released under own licensehttps://github.com/pypr/pysph

stands for Stochastic PArallel Rare�ed-gasTime-accurate Analyzer and is a DirectSimulation Monte Carlo (DSMC) code designedto run e�ciently on parallel computers.https://github.com/sparta/sparta

16/40

Lagrangian methods, III

a fully-featured Monte Carlo particle transportcode based on modern methods: constructivesolid geometry, continuous-energy transport codeand HDF5 format support, released MIT Licensehttps://github.com/openmc-dev/openmc

LAGrangian High-Order Solver solves thetransient Euler equations of compressible gasdynamics in a moving Lagrangian frame usingunstructured high-order �nite element spatialdiscretization, released under BSD-2 Licensehttps://github.com/CEED/Laghos

17/40

High order analysis

A scienti�c software for the numerical simulation ofseismic wave phenomena and earthquake dynamics,released BSD 3-Clause "New"or "Revised"Licensehttps://github.com/SeisSol/SeisSol

A generalized unstructured massively parallellow Mach �ow code, Apache License v.2.0,https://github.com/NaluCFD/Nalu

Flexi o�ers a powerful framework, whichis tailored for high-�delity (Discontinuous-Galerkin) time-dependent simulations in HPCenvironments, published under GPLv.3.0https://github.com/flexi-framework/flexi

18/40

Mixed methods libraries, I

INMOST

is a software platform for development ofparallel numerical models on general meshes.It is a supplimentary tool for supercomputernumerical mathematical models characterizedby a maximum generality of supportedcomputational meshes, BSD 3-Clause Licensehttps://github.com/INMOST-DEV/INMOST

is a free, lightweight, scalable C++ libraryfor �nite element methods, arbitrary high-order �nite element meshes and spaces withconforming and nonconforming adaptivemesh re�nement, published under GNU LGPLhttps://github.com/mfem/mfem

general-purpose (but mainly for research andscienti�c studies) open-source computingplatform for solving partial di�erentialequations, published under GNU LGPL v.3.0https://fenicsproject.org/

19/40

Mixed methods libraries, II

general-purpose multiphysics computationalplatform developed by Von Karman Institute.It is published under the LGPL v3.0.https://github.com/andrealani/COOLFluiD

ENigMA

is an object-oriented C++ template library whichaim is to provide multi-physics simulation in amulti-domain environment : FDM, FVM, FEM,SPH, BEM, etc. It is published under GPLv2.https://github.com/bjaraujo/ENigMA

20/40

Optimization software, I

state-of-the-art research and robust,usable software for optimization andUQ. It is published under the LGPL.https://dakota.sandia.gov

is an open-source high-performance computingplatform for e�cient optimization, focusedon supporting gradient-based optimizationwith analytic derivatives. It is publishedunder the Apache License, Version 2.0.https://openmdao.org/

is a parametric aircraft geometry tool and allowsthe user to create a 3D model of an aircraftde�ned by common engineering parameters. It ispublished under the NASA Open Source Agreement.https://github.com/OpenVSP/OpenVSP

21/40

Optimization software, II

DAFoam contains a suite of discrete adjoint solversfor OpenFOAM and supports design optimizationsfor a wide range of disciplines such as aerodynamics,heat transfer, structures, hydrodynamics, andradiation. It is published under GNU GPL v.3.0.https://github.com/mdolab/dafoam

OpenAeroStruct is a lightweight tool that performsaerostructural optimization using OpenMDAO. Itcouples a vortex-lattice method (VLM) and a 6degrees of freedom 3-dimensional spatial beammodel. It is published under Apache License 2.0.https://github.com/mdolab/OpenAeroStruct

BlenderFOAM

Open-source Fluid Based Shape Optimization.It is published under GNU GPLv.3.0. https:

//nathanrooy.github.io/posts/2019-03-05/

blenderfoam-aerodynamic-shape-optimization/

22/40

OpenFOAM+Blender+Python

23/40

Model reduction

libROM

is a library to compute proper orthogonaldecomposition-based reduced order models (POD-based ROMs). It is published under Apache License2.0. https:

//github.com/LLNL/libROM

is an implementation in FEniCS of several reducedorder modelling techniques (and, in particular,certi�ed reduced basis method and ProperOrthogonal Decomposition-Galerkin methods) forparametrized problems. It is published under LGPLv.3.0. https:

//github.com/mathLab/RBniCS

In real Time Highly Advanced ComputationalApplications for Finite Volumes - ROMs forOpenFOAM. Versions for DGM and SEM areavailable. It is published under LGPL v.3.0.https://github.com/mathLab/ITHACA-FV

24/40

Coupling software

precICE stands for Precise Code InteractionCoupling Environment. Its main component is alibrary that can be used by simulation programs tobe coupled together in a partitioned way, enablingmulti-physics simulations, such as �uid-structureinteraction. It is published under LGPL v.3.0.https://github.com/precice/precice

25/40

Meshing software, I

Ani3D provides portable libraries for each step in thenumerical solution of systems of PDEs with variabletensorial coe�cients: (1) unstructured adaptivemesh generation, (2) metric-based mesh adaptation,(3) �nite element discretization and interpolation,(4) algebraic solvers. It is published under LGPLv.2.0. https:

//sourceforge.net/projects/ani3d/

is a high performance multiphysics �nite elementsoftware. It is published under LGPL v.2.1.https://github.com/NGSolve/netgen

the open source system for processing and editing 3Dtriangular meshes. It is published under GNU GPLv3.0. https:

//github.com/cnr-isti-vclab/meshlab

26/40

Meshing software, IIThe snappyHexMesh utility is a part ofOpenFOAM and generates 3-dimensionalmeshes containing hexahedra (hex) and split-hexahedra (split-hex) automatically fromtriangulated surface geometries, or tri-surfaces,in Stereolithography (STL) or Wavefront Object(OBJ) format. It is published under GNU GPL v3.0.https://www.openfoam.com/

TetWildis a library for tetrahderal mesh generation.It is published under GNU GPL v3.0.https://github.com/Yixin-Hu/TetWild

PyGeM is a python library using Free FormDeformation, Radial Basis Functions and InverseDistance Weighting to parametrize and morphcomplex geometries. It is ideally suited foractual industrial problems, since it allows tohandle 1) Computer Aided Design �les (in.iges and .stl formats); 2) Mesh �les (in .unvand OpenFOAM formats) 3) Output �les (in.vtk format) 4) LS-Dyna Keyword �les (.kformat) It is published under MIT License.https://github.com/mathLab/PyGeM

27/40

Meshing software, IIIG+Smo (pronounced gismo or gizmo) is aC++ library for isogeometric analysis (IGA).Geometry plus simulation modules aims at theseamless integration of Computer-aided Design(CAD) and Finite Element Analysis (FEA) Itis published under Mozilla Public License 2.0.https://github.com/gismo/gismo

The library provides a framework for the numericalsimulation of partial di�erential equationsusing arbitrary unstructured discretizationson serial and parallel platforms. A majorgoal of the library is to provide support foradaptive mesh re�nement (AMR) computationsin parallel. It is published under LGPL v2.1.https://github.com/libMesh/libmesh

mimmo is a open source C++ library for meshmanipulation and morphing developed by OptimadEngineering Srl. It is published under LGPL v3.0.https://github.com/optimad/mimmo

28/40

Visualization

ParaView is an open-source, multi-platformdata analysis and visualization application.It is published under BSD 3-clause License.https://github.com/Kitware/ParaView

VisIt is an open-source interactive parallelvisualization and graphical analysistool for viewing scienti�c data. It ispublished under BSD 3-clause License.https://github.com/visit-dav/visit

3D scienti�c data visualization and plotting inPython. Mayavi strives to be a reusable toolthat can be embedded in your applications indi�erent ways. It is published under BSD License.https://github.com/enthought/mayavi

29/40

GUI & Integration

FreeCAD is an general-purpose parametric platformfor 3D CAD/CAE based on Open CascadeTechnology. It is published under LGPL v.2.https://github.com/FreeCAD/FreeCAD

SALOME is an open-source software that providesa generic Pre- and Post-Processing and couplingplatform for numerical simulation. It is based onan open and �exible architecture made of reusablecomponents and is published under LGPL v.2.1https://www.salome-platform.org

Blender is the free and open source 3Dcreation suite. It supports the entirety of the3D pipeline�modeling, rigging, animation,simulation, rendering, compositing and motiontracking, even video editing and gamecreation and is published under GNU GPL.https://github.com/blender/blender

30/40

SALOME Integration platform

31/40

Problem-oriented libraries, IQGDsolver

OpenFOAM implementation of regularized gas- and hydrodynamuicsequations https://github.com/unicfdlab/QGDsolver

libAcousticsopen-source (OpenFOAM) library for the far-�eld acoustics https:

//github.com/unicfdlab/libAcoustics

OpenNOSEan open-source (OpenFOAM) procedure for the simulation ofnasal aerodynamics https://home.aero.polimi.it/quadrio/it/

Tesi/pesci/tesi-pesci.pdf

rheoToolToolbox to simulate GNF and viscoelastic �uid �ows in OpenFOAMhttps://github.com/fppimenta/rheoTool

SOWFAToolbox (OpenFOAM) for the windplant simulation https://github.

com/NREL/SOWFA

ExaWindECP library for precise simulation of windplants https://www.

exawind.org/

AMR-WindAMReX based library for wind-turbine simulations https://github.com/Exawind/amr-wind

EnergyplusOpen-source toolbox for simulation of house energy balance https:

//github.com/NREL/EnergyPlus

MACH-AeroThe aerodynamic shape optimization library based on OpenMDAO,Python, OpenFOAM https://github.com/mdolab/MACH-Aero

AeroSandBoxAircraft design optimization, fully-coupled viscous/inviscid 3Daerodynamics, and reverse-mode automatic di�erentiation https:

//github.com/peterdsharpe/AeroSandbox

CfdOFComputational Fluid Dynamics (CFD) for FreeCAD based onOpenFOAM solver https://github.com/jaheyns/CfdOF

32/40

Problem-oriented libraries, II

SeisSola scienti�c software for the numerical simulation of seismicwave phenomena and earthquake dynamics https://github.com/

SeisSol/SeisSol

MOM6

a next-generation open-source ocean model that combines thebest of GOLD (http://code.google.com/p/gold-omod/) and MOM5(https://mom-ocean.github.io/) https://github.com/NOAA-GFDL/

MOM6

WRFthe Weather Research and Forecasting (WRF) model https://

github.com/wrf-model/WRF

PyLitha �nite element code for the solution of dynamic and quasi-statictectonic deformation problems https://github.com/geodynamics/

pylith

peridigman open-source computational peridynamics code developed at SandiaNational Laboratories for massively-parallel multi-physics simulationshttps://github.com/peridigm/peridigm

OPMThe Open Porous Media (OPM) initiative encourages open innovationand reproducible research for modeling and simulation of porous mediaprocesses https://opm-project.org/

COOLFluiDThe object-oriented HPC platform for CFD, plasma and multi-physicssimulations developed at the Von Karman Institute for Fluid Dynamics,https://github.com/andrealani/COOLFluiD

enricoExascale Nuclear Reactor Investigative COde with coupling to Nek5000open-source code, https://github.com/enrico-dev/enrico

33/40

Application to real problems

• Study of rocket lift-o� processes: OpenFOAM+AMReX+...

• The integrated platform for �uid-structure interaction simulationand design variation: OpenFOAM+Code_Aster+SALOME

• Full-scale simulation of vibrations and acoustics inside an aircraft

34/40

Example: an aircraft's interior acoustics

¾Full frequency vibro-acoustic simulation in the aeronautics industry¿ byA. Pei�er, Airbus Group Innovations, Vibroacoustics and Dynamics

35/40

OSS solution to the problem

Elmer

UCNS3D

BEM

++

github.com/FRidh/seapy

Integration/GUI:SALOME, Blender,FreeCAD

github.com/ladybug-tools/honeybee

Model reduction: libROM, RBniCS

FeniCS

Op

tim

izati

on

Dat

a as

sim

ilati

on

: DA

PP

ER

• FEM: Code_Aster(includinganisotropy ofmaterials), Elmer(in spatial-frequenydoamin), MFEM

• FEM/RKDG:MFEM, Nektar++

• Aerodynamic noisesources/CFD:OpenFOAM,AMReX

• BEM: BEM++

• Optimization:Dakota/openMDAO

• Integration/GUI:SALOME/Blender

• Model reduction:RBniCS, ITHACA

• Coupling: precICE

36/40

OpenFOAM forks• blueCFD R©-Core is an Windows R© port of OpenFOAM R©.• RapidCFD is an OpenFOAM solvers ported to Nvidia CUDA.• DigiTEF/OpenDTEF is an ISP RAS build of OpenFOAM

Входныеданные

HCSМодуль

QGDМодуль

CAAМодуль

swak4Foam

PyFoam

Цифроваямодель

Персона-лизирова-

ныймодуль

RANS/LES

AMReX

BEM++

SALOME Code_Aster

Постановкастандартной задачи

Постановкауникальной

задачи

Как средствомоделирования

Как платформадля интеграции

Digital TEst Facility -

Специлизированнаяцифровая

модель

37/40

Demanded skills

• Engineer must be well trained at the intersection of at least 3knowledge areas:

• Applied physics• Programming• Mathematics

• Validation and veri�cation of algorithms & libraries

• Common libraries interfaces and methods for models coupling

• Libraries and area of their application

• High-performance computing theory and practice

• Stable application development

• Communication skills and team work

38/40

Conclusions

• Single-physics problems are more exception, than the rule

• Open-source sofrware covers almost all aspects of modernengineering & science

• The idea of OSS as a construction set is used by many teams

• OSS creates new demands to engineers and developers

• Each year the coupling procedure between di�erent libraries & codesrequires less e�orts and resources

• The shift of engineer's attitude to the problem solution: from"Which program"to "Which model"

• Changes in requirements to engineer's skills and education

39/40

Encouragement

As the conference is dedicated to open-source software, I encourage youto

• use as many OSS packages as possible and as much as possible;

• enhance existing OSS

• develop new OSS in novel areas of science & industry

Thank you for the attention!

40/40