(CC BY-SA 4.0)

EOF-Library: Open-source Elmer FEM and OpenFOAM

coupler for electromagnetics and fluid dynamics

Motivation for MHD - controlling liquid metals

F1

F2

F

Mechanically

Electromagnetically

piperotating permanent magnets

FL

FLFL

FL

Liquid metal pump

Required physics

Modelling coupled

● Electromagnetics● Fluid dynamics

Extra

● Free surface● Heat transfer● Solidification / melting Liquid metal pump - modelling requires two-way

coupled electromagnetics and fluid dynamics

1

Required numerical models

● 3D, transient

● Complex geometries, multi-regions

● Advanced pre- and post-processing tools

● Parallelization with good scaling

● User community, documentation

● Complex A−V magnetic vector potential formulation

● Conductivity dependence on temperature

● High Reynolds turbulence models

● Volume of Fluid (VOF) for free surface modelling

● Viscosity dependence on temperature

Fluid dynamics Electromagnetics

2

Existing Open Source codes

FEM FVM

getDP OpenFOAM

Elmer SU2

... ...

Best forElectromagnetics (EM)

Best forFluid dynamics (FD)

3

Choosing Electromagnetics solver (1)

GetDP

+ Designed for EM in mind, great for eddy-current problems

+ Uses PETSc iterative solvers, PETSc is MPI parallelized

- Matrix assembly is not parallel (neither MPI nor OpenMP)

- Weak integration with external post-processing tools

MaxFEM, FreeFem++, deal.II, …

● I have no opinionPictures from getdp.info

4

Choosing Electromagnetics solver (2)

+ Multiphysics out-of-the-box (>40 solvers)

+ Build-in linear & nonlinear iterative solvers, fully parallelized

+ Build-in computation of 3D electrical currents resulting from

VxB term - very important for MHD and rotating magnets

+ Extensive documentation, active forum, responsive developers

- Adding new functionality requires knowledge in FEM and Fortran

Elmer by CSC (Finland)

5

Choosing Fluid dynamics solver

● In terms of popularity and functionality OpenFOAM is leader+ Most important - it has Volume of Fluid (VOF)+ OpenFOAM = solvers + examples + community + impact

6

No one code solves our problem

.., but two coupled codes can!

Elmer + OpenFOAM

● Both can simulate real stuff● Common pre/post processing (Salome, Paraview)● Straight-forward mesh interpolation● Most important - Parallel MPI-based coupling is feasible 7

Elmer OpenFOAM = EOF-Library

Two-way coupling

File based

● SLOW and does not SCALE on parallel computers

MPI (Message Passing Interface) based

● Both codes already use MPI for parallelization

● Probably the most EFFICIENT coupling solution8

MPI - Message Passing Interface

Running on 16 cores:

mpirun -n 16 Elmer : -n 16 OpenFOAM

Elmer(16 processes)

OpenFOAM(16 processes)“Wrapper

that starts applicationsin parallel”

MPI_SEND(data,type,size,

to whom)

MPI_RECV(data,type,size,

from whom)

MPI_RECV(...) MPI_SEND(...)

9

In the code

Interpolating and sending fields from Elmer to OpenFOAM

Checks whether Elmer sends message,if not then sleeps 10ms (saves CPU resources) Projector matrix (interpolation) is applied on variable

10

Structure of EOF-Library

Patch for splitting MPI_COMM_WORLD on OpenFOAM side(every OF version has its own patch)

├── commSplit│ ├── allReduce.H│ ├── allReduceTemplates.C│ ├── Make│ │ ├── files│ │ └── options│ ├── PstreamGlobals.C│ ├── PstreamGlobals.H│ ├── UIPread.C│ ├── UOPwrite.C│ └── UPstream.C

Coupler’s OpenFOAM part

├── coupleElmer│ ├── Elmer.C│ ├── Elmer.H│ └── Make│ ├── files│ └── options

Coupler’s Elmer part

├── Elmer2OpenFOAM│ └── Elmer2OpenFOAM.F90└── OpenFOAM2Elmer └── OpenFOAM2Elmer.F90

11

Example for modified OpenFOAM solver

├── mhdInterFoam│ ├── alphaCourantNo.H│ ├── alphaEqn.H│ ├── alphaEqnSubCycle.H│ ├── correctPhi.H│ ├── createFields.H│ ├── Make│ │ ├── files│ │ └── options│ ├── mhdInterFoam.C│ ├── pEqn.H│ ├── setDeltaT.H│ ├── setRDeltaT.H│ └── UEqn.H

Field value when Elmer was called last time

Current field value

User is responsible about physics 12

OpenFOAM ParaviewSalome Elmer FEM

CAD & meshing Post-processing

EOF-Library

Simulation workflow using open-source software

OpenFOAM Elmer FEM

Liquid Gas

Rarified gas

Induction Microwaves

RadiationSteady & transient

Heat

Parametricdependence

f(T)

Particle tracking

Possible* coupling combinations* not all of them are tested

Application examples

Electromagnetic levitation melting Surface wave generation usinglow frequency EM field

Interpolation test - unit cube

Elmer & OpenFOAM meshes are different

Refinement level

● 0.1 (4.7k tets)

● 0.05 (32k tets)

● 0.025 (245k tets)

Reference scalar field

2 sin[2πx] cos[2πy]

Elmer-to-OpenFOAM

Tet size 0.1 0.05 0.025

Relative Error % 8.0 2.3 0.57

OpenFOAM-to-Elmer

Tet size 0.1 0.05 0.025

Relative Error % 11 4.0 1.5

Strong scaling test

13.1M cells / elements

Conclusion:Coupler makes a negligible performance overhead comparing to computation time

Finally - obtaining the code

https://github.com/jvencelsI typically use Salome for CAD and meshing

Useful salome script for exporting mixed (tet+hex) meshes directly to Elmer

Similar script for OpenFOAMhttps://github.com/nicolasedh/salomeToOpenFOAM

13

EOF-Library.com● Research● Development● Support

vencels@eof-library.com