Stochastic Finite Element Methods (FEMs) for PDE …...Stochastic Finite Element Methods (FEMs) for PDE Models with Uncertain Inputs Catherine Powell University of Manchester [email protected]

Stochastic Finite Element Methods (FEMs) for PDEModels with Uncertain Inputs

Catherine Powell

University of Manchester

[email protected]

February 1, 2018

Catherine Powell Stochastic FEMs 1 / 33

Uncertainty Quantification (UQ)

Many physical processes are modelled by PDE(s):

B fluid mechanics

B continuum mechanics

B thermodynamics

B etc...

If we know the geometry, boundary conditions, coefficients, initialconditions etc then we can compute often accurate appproximations usingfinite element methods (FEMs).

In many applications, we don’t know all the inputs. One possibility is torepresent uncertain inputs as random variables.

B We have to solve stochastic/parameter-dependent PDEs.

B We need tailored numerical methods (computer models)


Uncertainty Quantification (UQ)

Many physical processes are modelled by PDE(s):

B fluid mechanics

B continuum mechanics

B thermodynamics

B etc...

If we know the geometry, boundary conditions, coefficients, initialconditions etc then we can compute often accurate appproximations usingfinite element methods (FEMs).

In many applications, we don’t know all the inputs. One possibility is torepresent uncertain inputs as random variables.

B We have to solve stochastic/parameter-dependent PDEs.

B We need tailored numerical methods (computer models)


Finite Element Approximation

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1Q1 finite element subdivision


Types of Uncertainty

B Model UncertaintyUncertainty in: form of the model, scales of the physical process, missingphysics etc.

B Parameter/Input UncertaintyUncertainty in: coefficients, material parameters, boundary conditions,initial conditions, geometry etc.

B Numerical UncertaintyUncertainty (error) stemming from choice of discretisation, numericalapproximation etc.


Forward UQ

Given a probabilistic description of the uncertain inputs

B mean, covariance

B distribution

B . . .

compute an approximation to a quantity of interest (QoI) related to thesolution u, such as

B E[u], E[f (u)], Var(u)

B P(f (u) > tol)

B . . .

and an estimate of the error/uncertainty in that approximation.


1. Navier Stokes

Find u(x) (the velocity) and p(x) (the pressure) such that,

− ν∇2u + u · ∇u +∇p = f in D,

∇ · u = 0 in D,

u = g on ∂DD ,

ν∂u

∂n− np = 0 on ∂DN .

If the viscosity is uncertain, we can model it as a single random variable.For example, a Uniform random variable with

E[ν] = µ, Var[ν] = σ2.


Example: Flow over a step

µ = 1/50 and σ = 1/500. Streamlines of the mean flow (top) and meanpressure (bottom) computed with a stochastic (Galerkin) FEM.

−1 0 1 2 3 4 5−1

0

1

−0.3

−0.2

−0.1

0

0.1



Variance of the magnitude of velocity (top) and variance of the pressure(bottom) computed with a stochastic (Galerkin) FEM.

−1 0 1 2 3 4 5 −1

0

12

4

6

8

x 10−4


2. Groundwater Flow

A simple model for steady-state fluid flow in a porous medium is:

−a∇p = u in D,

∇ · u = f in D,

p = g on ∂DD ,

u · n = 0 on ∂DN = ∂D\∂DD .

Solution variables: p = p(x) (hydraulic head), u = u(x) (velocity field).

a(x) permeability coefficientf (x) source or sink termsg(x) boundary data

D ⊂ Rd spatial/computational domain


WIPP Case Study

Measurements a(xi ) are known at 41 spatial locations xi .

605 610 615 620

3570

3575

3580

3585

3590

3595

WIPP Site: Sandia Grid and Bore Locations

in thousands of mE

in th

ousa

nds

of m

N

H1H2H3

H4

H5H6

H7

H9

H10

H11

H12

H14

H15H16

H17

H18

DOE1

DOE2

P14

P15P17

P18

WIPP12WIPP13

WIPP18WIPP19WIPP21WIPP22

WIPP25

WIPP26

WIPP27

WIPP28

WIPP30

ERDA9

CB1

ENGLE

USGS1

D268

AEC7


Random Field Model

Let a(x) be a random variable at each point in space.

Given a mean µ(x) and covariance C (x, x′) for a(x), we want to makestatistical predications about the pressure p and velocity u.


WIPP (Monte Carlo FEM)

Realisations of the path travelled by a particle released into the flow at afixed location (left). Monte Carlo estimate of the expected flow field (right).

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2x 104

0

0.5

1

1.5

2

2.5

3

x 104

in mE

in m

N

Individial Particle Paths from Unconditioned Simulations

0 5 10 15 200

5

10

15

20

25

30

km

km

Monte Carlo Mean RT0 flux solution

H1H2H3

H4

H5H6

H7

H9

H10

H11

H12

H14

H15H16

H17

H18

DOE1

DOE2

P14

P15P17

P18

WIPP12WIPP13

WIPP18WIPP19WIPP21WIPP22

WIPP25

WIPP26

WIPP27

WIPP28

WIPP30

ERDA9

CB1

ENGLE

USGS1

D268

AEC7


Monte Carlo FEM

A popular black-box strategy for performing forward UQ in PDE models is

B choose a statistical model for the uncertain input(s)

B use random field generators to draw samples of the input(s)

B use FEM codes to solve the PDEs for each sample

B average results to obtain estimates of the mean, variance etc.

Commonly cited benefits include:

B allows reuse of existing FEM codes

B ‘trivially’ paralellisable

B no complicated mathematics!


What’s wrong with Monte Carlo FEM?

The error in approximating the expected solution is:

Error =

∥∥∥∥∥E [u]− 1

M

M∑i=1

uh(x, yi )

∥∥∥∥∥

and breaks up into FEM discretisation and MC statistical errors:

Error =

∥∥∥∥∥E [u − uh] +

(E [uh]− 1

M

M∑i=1

uh(x, yi )

)∥∥∥∥∥≤ ‖E [u − uh]‖︸︷︷︸

FEM error

+

∥∥∥∥∥(E [uh]− 1

M

M∑i=1

uh(x, yi )

)∥∥∥∥∥︸︷︷︸MC sampling error

≈ O(hα) + O(M−1/2).



The error in approximating the expected solution is:

Error =

∥∥∥∥∥E [u]− 1

M

M∑i=1

uh(x, yi )

∥∥∥∥∥and breaks up into FEM discretisation and MC statistical errors:

Error =

∥∥∥∥∥E [u − uh] +

(E [uh]− 1

M

M∑i=1

uh(x, yi )

)∥∥∥∥∥≤ ‖E [u − uh]‖︸︷︷︸

FEM error

+

∥∥∥∥∥(E [uh]− 1

M

M∑i=1

uh(x, yi )

)∥∥∥∥∥︸︷︷︸MC sampling error

≈ O(hα) + O(M−1/2).



To obtainError ≤ TOL,

we need to choose

h = O(TOL1/α), M = O(TOL−2)

Suppose an optimal solver for the FEM system is available. The cost toachieve Error ≤ TOL is:

Cost = M × O(h−d)︸︷︷︸Cost of FEM solve

= O(TOL−2−d/α)

So, even for a toy problem: −∇ · a∇u = f in 2d with linear FEM:

Cost = O(TOL−4).




we need to choose




= O(TOL−2−d/α)


Cost = O(TOL−4).




we need to choose




= O(TOL−2−d/α)


Cost = O(TOL−4).


Alternative Stochastic FEMs

There are many alternative schemes that can exploit existing FEM codes.

B Sampling methods- Multifidelity & Multilevel Monte Carlo methods.- Stochastic collocation FEMs (SC-FEMs)- Reduced basis FEMs (RB-FEMs)

B Not sampling methods- Stochastic Galerkin FEMs (SG-FEMs)

SC-FEMs and SG-FEMs are based on polynomial approximation.

RB-FEMs can be combined with any sampling method.


Stochastic Galerkin FEM

We compute an approximation of the form

u(x, y) ≈N∑j=1

uj(x)ψj(y)

where {ψ1, . . . , ψN} are polynomials in the parameters y (Polynomial

Chaos) by solving one linear system,

Au = f

of dimension NNh × NNh.

B Nh is the size of the FEM problem

B N depends on no. of parameters and chosen polynomial degree



PC component 0 norm=9.9995e−01

−1 0 1 2 3 4 5−1

0

1PC component 1 norm=2.5326e−02

−1 0 1 2 3 4 5−1

0

1


−1 0 1 2 3 4 5−1

0


−1 0 1 2 3 4 5−1

0

1


−1 0 1 2 3 4 5−1

0


−1 0 1 2 3 4 5−1

0

1


Estimating the vorticity

Components of the vorticity along the bottom channel wall, E[Re] ≈ 296.

0 5 10 15−1

−0.5

0

0.5

1

x

PC component 0

0 5 10 15−0.2

00.20.40.6

x

PC component 1

0 5 10 15−0.4

−0.2

0

0.2

0.4

x

PC component 2

0 5 10 15−0.2

0

0.2

x

PC component 3

0 5 10 15−0.04−0.02

00.020.040.06

x

PC component 4

0 5 10 15−0.02

0

0.02

x

PC component 5


Standard deviation of the vorticity

Here, just one parameter (viscosity), ν = µ+ σy . Quantity of interest:

Q(ν(y)) :=

∫ 7

3

ω(x, y)ds

The standard deviation of Q increases as we increase σ and we need morepolynomials to accurately compute the QoI.

N = 2 N = 3 N = 4 N = 5 N = 6σ = µ/10 0.6553 0.6492 0.6491 * *σ = 2µ/10 1.2639 1.2150 1.2139 1.2138 *σ = 3µ/10 1.7752 1.6197 1.6180 1.6167 1.6162σ = 4µ/10 2.1379 1.8133 1.8320 1.8287 1.8291


Reduced Basis FEM

If using a sampling method, for each y of interest, if we use standard FEM(high fidelity methods),

u(x, y) ≈ uh(x, y)

we have to solve a sparse linear system of Nh equations

A(y)︸︷︷︸Nh×Nh

u(y) = f.

If our UQ method (e.g. Monte Carlo) uses M samples, and we have anoptimal iterative solver for each system, then the cost is

M︸︷︷︸Live with this!

× O(Nh)︸︷︷︸Try to reduce this!


Reduced Basis FEM

If using a sampling method, for each y of interest, if we use standard FEM(high fidelity methods),

u(x, y) ≈ uh(x, y)

we have to solve a sparse linear system of Nh equations

A(y)︸︷︷︸Nh×Nh

u(y) = f.

If our UQ method (e.g. Monte Carlo) uses M samples, and we have anoptimal iterative solver for each system, then the cost is

M︸︷︷︸Live with this!

× O(Nh)︸︷︷︸Try to reduce this!


Basic Idea: Reduced Problem

Instead, for each y of interest we now seek to approximate:

uh(x, y) ≈ uR(x, y)

where uR(x, y) belongs to a lower-dimensional space.

B Run the expensive FEM code NR times with specially chosen sets ofinput parameters yi .

B Collect the solution vectors (snapshots) into a matrix

V = [u(y1),u(y2), · · · ,u(yNR)].

B Make the columns orthonormal to produce a new matrix Q

V → Q = [q1,q2, · · · ,qNR].

which has size Nh × NR (tall and skinny).


Basic Idea: Reduced Problem

Instead, for each y of interest we now seek to approximate:

uh(x, y) ≈ uR(x, y)

where uR(x, y) belongs to a lower-dimensional space.

B Run the expensive FEM code NR times with specially chosen sets ofinput parameters yi .

B Collect the solution vectors (snapshots) into a matrix

V = [u(y1),u(y2), · · · ,u(yNR)].

B Make the columns orthonormal to produce a new matrix Q

V → Q = [q1,q2, · · · ,qNR].

which has size Nh × NR (tall and skinny).


Standard Snapshot Basis

For each new y of interest, approximate u(y) ≈ QuR(y), where uR(y) isfound by solving the reduced system

(Q>A(y)Q

)︸︷︷︸NR×NR

uR(y) = Q>f

with uR(y) ∈ RNR .

The reduced problem can be used as a surrogate.


Test problem 1: Groundwater Flow

5

4

3

2

1

0.2

0.4

0.6

0.8

High-Fidelity Mixed FEM: Nh = 217, 464.


Timings: System Solves

Average time in seconds to assemble and solve the high fidelity FEMsystems (T) and reduced basis FEM systems (t):

ε1 NR T t1e-3 36 2.13e0 5.57e-41e-4 59 2.14e0 1.06e-31e-5 84 2.14e0 1.75e-31e-6 108 2.08e0 5.03e-3


High Fidelity vs Reduced Basis

Total time to compute mean and variance of FEM solution.

ε = 10−5 RB-FEM Standard FEMM NR offline online high fidelity

351 64 510.9 0.4 746.01471 84 795.0 2.6 3058.05503 96 1,002.1 14.3 11,523.5

Note: In this problem, we have only 5 parameters and a nice structure. Whenoffline times are taken into account, 10 times faster than standard FEM.


Test Problem 2

Now we have 12 parameters, and a more complicated PDE structure.

ε = 10−5 RB-FEM Standard FEMM NR offline online high fidelity

2,649 52 1548.9 8.8 3,182.517,265 53 9,327.1 57.6 20,746.193,489 53 53,685.8 300.0 116,049.5

When offline times are taken into account, only twice as fast.


Summary

B Applied mathematicians are mainly concerned with the accuracy ofoutputs of computer models with respect to the true solution of thecontinuous physical model.

B Standard Monte Carlo is infeasible for computer models consisting ofFEM discretizations for PDE models.

B Stochastic Galerkin FEM (polyomial chaos type methods) may beappropriate if there are a modest no. of input parameters and the truesolution is well behaved enough.

B Building surrogate FEM models using reduced basis ideas might offersome savings if time can be invested in the offline stage.


Standard Reduced Basis Methods

� A. Quarteroni, A. Manzoni, F. Negri, Reduced basis methods forPDEs: an introduction, Springer (2015)

� J. Hesthaven, G. Rozza B. Stamm, Certified Reduced Basis Methodsfor Parameterized PDEs, Springer (2015).

� M. Barrault, Y. Maday, N. Nguyen, A.T. Patera, An empiricalinterpolation method: application to efficient reduced basis discretizationof PDEs, Comptes Rendus Mathematique 339(9), 2004.

� G. Rozza, K. Veroy, On the stability of the reduced basis method forStokes equations in parametrized domains, Computer methods inapplied mechanics and engineering, 196(7), 2007.


Stochastic Collocation

� I. Babuska, F. Nobile, R. Tempone, A Stochastic Collocation Methodfor Elliptic Partial Differential Equations with Random Input Data ,SIAM. Journal on Numerical Analysis, 45(3), 2007.

� F. Nobile, R. Tempone, C. Webster , A Sparse Grid StochasticCollocation Method for PDEs with Random Input Data, SIAM. Journalon Numerical Analysis, 46(2), 2008.

� D. Xiu, J. Hesthaven, High-Order Collocation Methods for DifferentialEquations with Random Inputs, SIAM. Journal on ScientificComputing, 27(3), 2005.


RBMs + Stochastic Collocation

� H.C. Elman & Q. Liao, Reduced Basis Collocation Methods for PDEswith Random Coefficients, SIAM/ASA Journal on UQ, 1(1), 2013.

� P. Chen, A. Quarteroni & G. Rozza, Comparison Between ReducedBasis and Stochastic Collocation Methods for Elliptic Problems, Journalof Scientific Computing, 59(1), 2014.

� C. Newsum & C.E. Powell, Efficient reduced basis methods for saddlepoint problems with applications in groundwater flow. (2016). MIMSEPrint: 2016.60.


Offline vs Online

RB methods decompose the computational work into two stages.

� Offline: The reduced basis matrix Q is constructed, and any matrices &vectors that are too expensive to compute online.

Warning: Offline costs may be very expensive!

� Online: All the reduced systems are solved,(QTA(yi )Q

)uR(yi ) = f, i = 1, . . . ,N,

and quantities of interest computed.

Key Philosophy: Costs should be independent of Nh.


Standard Greedy Method

Training set Θtrain ⊂ Γ, error tolerance ε, error estimator ∆R(y).Initialise Q = [u(y0)] with some y0 ∈ Γ.while maxy∈Θtrain

∆R(y) > ε doFind yk = arg maxy∈Θtrain

∆R(y)Solve high-fidelity problem for u(yk)Set Q = [Q,u(yk)] & Orthonormalise

end

If the error estimator satisfies

‖uh(y)− uR(y)‖Vh≤ ∆R(y)

then we say the method is certified.


Documents

Stochastic Finite Element Methods (FEMs) for PDE …...Stochastic Finite Element Methods (FEMs) for PDE Models with Uncertain Inputs Catherine Powell University of Manchester [email protected]