Direct and iterative sparse linear solvers applied to groundwater flow simulations

Matrix Analysis and Applications

October 2007

Jocelyne Erhel INRIA Rennes

Jean-Raynald de Dreuzy CNRS, Geosciences Rennes

Anthony Beaudoin

LMPG, Le Havre

Partly funded by Grid’5000 french project

From Barlebo et al. (2004)

Dispersion

Flow

Injection of tracer

Tracer evolution during one year (Made, Mississippi)

Heterogeneous permeability

Physical context: groundwater flow

Flow governed by the heterogeneous permeability

Solute transport by advection and dispersion

Head

Numerical modelling strategy

NumericalStochasticmodels

Simulationresults

Physical model

natural system

Simulation of flowand solute transport

Characteriz

ation of

heterogeneity

Model validation

Uncertainty Quantification methods

Spatial heterogeneity

Stochastic models of flow and solute transport

-random velocity field-random solute transfer time and dispersivity

Lack of observationsPorous geological media

Solute dispersionHorizontal velocityPermeability field

HYDROLAB: parallel software for hydrogeoloy

Numerical methods

Physical models

Porous Media

Solvers

PDE solversODE solversLinear solversParticle tracker

Utilitaries

Input / OutputVisualizationResults structuresParameters structuresParallel and grid toolsGeometry

PARALLEL-BASED SCIENTIFIC PLATFORM HYDROLAB

Open source libraries

Boost, FFTW, CGal, MPI, Hypre, Sundials, OpenGL, Xerces-C

UQ methods

Monte-Carlo

FractureNetworks

Fractured-Porous Media

Object-oriented and modular with C++Parallel algorithms with MPIEfficient numerical libraries

Saturated medium: one water phaseSaturated medium: one water phase Constant density: no saltwaterConstant density: no saltwater Constant porosity and constant viscosityConstant porosity and constant viscosity Linear equationsLinear equations Steady-state flow or transient flowSteady-state flow or transient flow Inert transport: no coupling with chemistryInert transport: no coupling with chemistry No coupling between flow and transportNo coupling between flow and transport No coupling with heat equationsNo coupling with heat equations No coupling with mechanical equationsNo coupling with mechanical equations Classical boundary conditionsClassical boundary conditions Classical initial conditionsClassical initial conditions

Physical equations

Flow equations: Darcy law and mass conservationFlow equations: Darcy law and mass conservationTransport equations: advection and dispersionTransport equations: advection and dispersion

Flow and transport equations

Fix

ed

head

an

d C

=0

Fix

ed

head

an

d

C/

n=

0

Nul flux and C/ n = 0

Nul flux and C/ n=0

inje

ctio

n

• Advection-dispersion equationsBoundary conditionsInitial condition

• Flow equations

Monte-Carlo simulations

For j=1,…,M

Compute Vj

using a finite volume method

generate permeability field Kj

using a regular mesh

End For

Discrete flow numerical model

Linear system Ax=b

b: boundary conditions and source termA is a sparse matrix : NZ coefficientsMatrix-Vector product : O(NZ) opérations

Regular 2D mesh : N=n2 and NZ=5NRegular 3D mesh : N= n3 and NZ=7N

Need for parallel sparse linear solvers

Accuracy: condition number and variance

Estimation with MUMPS solverCond(A) in O(exp(2))

But in theory, cond(A) in O(Kmax/Kmin) thus in O(exp())

Accuracy: condition number and scaling

Estimation with Matlab without scaling and with scaling

Scaled condition number in O(exp(As expected

Componentwise condition number

Matlab condition number

residual Scaled condition number

Componentwise condition number

Solution error

1 6.2085e+004 3.0868e-015 1.6121e+004 2.4745e+003 6.7876e-015

2 1.1660e+006 5.7614e-015 6.8597e+004 4.6892e+003 1.8597e-014

3 3.4221e+007 6.9331e-015 1.7549e+005 1.2636e+004 8.4842e-015

4 2.1117e+009 8.8442e-015 5.0661e+005 4.2924e+004 5.2058e-013

5 2.0372e+011 2.1596e-014 1.6503e+006 1.6898e+005 4.7836e-014

Componentwise condition number estimated by|| |A-1| |A| |x| + |A-1| b ||1 / ||x||1

Solution error means ||x-xs|| / ||xs||

n=64

Accuracy: condition number and system size

Estimation with MUMPS for Cond(A) in O(N) as expected

Condition number not too large

for · 3 and for N up to 16 millions

Sparse direct linear solver

UMFPACK multifrontal solverRobust to variance but CPU time in O(N1.5)

As expected

Preconditioned Conjugate Gradient

PCG with IC(0) slightly sensitive to variance But very sensitive to size N

Need for a multilevel preconditioner

Geometric multigrid

HYPRE Solver SMGLinear CPU time in O(N) but sensitivity to variance

As expected

Algebraic multigrid

HYPRE Solver AMGRobust to variance and linear CPU time in O(N)

As expectedLess efficient than SMG for small variance

Algebraic multigrid with 3D domains

Robust to variance and CPU time in O(N)Same properties as in 2D

Parallel computing facilities

Numerical model

Clusters at Inria Rennes

Grid’5000 project

67.1 millions of unknowns

in 3 minutes

with 32 processors

Parallel performances with 2D domains

Parallel CPU time in O(N)SMG more efficient than AMG for small

AMG much more efficient than SMG for large

0 250 500 750 100010-1

100

101

2=9

2=6,25

2=4

2=2.25

2=1

2=0.25

DL(t

)

tN

0 500 1000-2

-1

0

1

2 2=9

2=6.25

2=4

2=2.25

2=1

2=0.25

DT(t

N)

tN

Longitudinal dispersion Transversal dispersion

Macro-dispersion analysis

Each curve represents 100 simulations on domains with 67.1 millions of unknowns

Conclusion

Summary• Efficient and accurate algebraic multigrid solver for groundwater flow in heterogeneous porous media• Good performances with clusters • Macro-dispersion analysis in 2D domains

Current and Future work• 3D heterogeneous porous media• Subdomain method with Aitken-Schwarz acceleration• Transient flow in 2D and 3D porous media • Grid computing and parametric simulations• UQ methods