High-Order Collocation Methods for Differential Equations With Random Inputs

8/2/2019 High-Order Collocation Methods for Differential Equations With Random Inputs

1/22

SIAM J. SCI. COMPUT. c 2005 Society for Industrial and Applied MathematicsVol. 27, No. 3, pp. 11181139

HIGH-ORDER COLLOCATION METHODS FOR DIFFERENTIAL

EQUATIONS WITH RANDOM INPUTS

DONGBIN XIU

AND JAN S. HESTHAVEN

Abstract. Recently there has been a growing interest in designing efficient methods for the so-lution of ordinary/partial differential equations with random inputs. To this end, stochastic Galerkinmethods appear to be superior to other nonsampling methods and, in many cases, to several samplingmethods. However, when the governing equations take complicated forms, numerical implementa-tions of stochastic Galerkin methods can become nontrivial and care is needed to design robust andefficient solvers for the resulting equations. On the other hand, the traditional sampling methods,e.g., Monte Carlo methods, are straightforward to implement, but they do not offer convergence asfast as stochastic Galerkin methods. In this paper, a high-order stochastic collocation approach isproposed. Similar to stochastic Galerkin methods, the collocation methods take advantage of anassumption of smoothness of the solution in random space to achieve fast convergence. However,the numerical implementation of stochastic collocation is trivial, as it requires only repetitive runsof an existing deterministic solver, similar to Monte Carlo methods. The computational cost of thecollocation methods depends on the choice of the collocation points, and we present several feasibleconstructions. One particular choice, based on sparse grids, depends weakly on the dimensionality of

the random space and is more suitable for highly accurate computations of practical applications withlarge dimensional random inputs. Numerical examples are presented to demonstrate the accuracyand efficiency of the stochastic collocation methods.

Key words. collocation methods, stochastic inputs, differential equations, uncertainty quantifi-cation

AMS subject classifications. 65C20, 65C30

DOI. 10.1137/040615201

1. Introduction. This paper is devoted to numerical solutions of differentialequations with random inputs. The source of random inputs includes uncertainty insystem parameters, boundary/initial conditions, etc. Such kinds of uncertainty areubiquitous in engineering applications and are often modeled as random processes.The classical approach is to model the random inputs as some idealized processeswhich typically are white noises, e.g., Wiener processes, Poisson processes, etc., andhas led to elegant mathematical analysis and numerical methods by using stochasticcalculus, e.g., Ito or Strantonovich calculus. The resulting differential equations aretermed stochastic ordinary/partial differential equations (SODE/SPDE) (see, forexample, [18, 25, 28, 43]). Recently, there has been a growing interest in studyingproblems with more correlated random inputs (colored noises) and, in case of ran-dom parameters, fully correlated random processes, i.e., random variables. In thiscontext, the classical stochastic calculus does not readily apply and other approachesare required.

Received by the editors September 16, 2004; accepted for publication (in revised form) August5, 2005; published electronically December 30, 2005.

http://www.siam.org/journals/sisc/27-3/61520.htmlDepartment of Mathematics, Purdue University, West Lafayette, IN 47907 ([email protected].

edu). The work of this author was partly supported by AFOSR contract F49620-01-1-0035.Division of Applied Mathematics, Brown University, Providence, RI 02912 ([email protected].

edu). The work of this author was partly supported by NSF Career Award DMS-0132967, byAFOSR contract FA9550-04-1-0072, and by the Alfred P. Sloan Foundation through a Sloan ResearchFellowship.

1118


2/22

COLLOCATION METHODS FOR STOCHASTIC EQUATIONS 1119

The traditional approach for differential equations with random inputs is theMonte Carlo method, which generates ensembles of random realizations for the pre-scribed random inputs and utilizes repetitive deterministic solvers for each realization.Monte Carlo methods have been applied to many applications and their implemen-

tations are straightforward. Although the convergence ratethe ensemble mean ofa brute-force Monte Carlo method with K realizations converges asymptotically ata rate 1/

Kis relatively slow, it is (formally) independent of the dimensionality

of the random space, i.e., independent of the number of random variables used tocharacterize the random inputs (e.g., [13]). To accelerate convergence, several tech-niques have been developed, for example, Latin hypercube sampling, [34, 48], thequasi-Monte Carlo (QMC) method, [14, 39, 40], and the Markov chain Monte Carlomethod (MCMC) [16, 22, 36]. These methods can improve the efficiency of the brute-force Monte Carlo method. However, additional restrictions are imposed based ontheir specific designs and their applicability is limited.

Another class of methods is the nonsampling methods, where no repetitive de-terministic solvers are employed. Methods along this line of approach include per-turbation methods [27] and second-moment analysis [31, 32]. These methods do not

result in convergent series of expansion and their applicability is restricted to systemswith relatively small random inputs and outputs. Such a condition is difficult to sat-isfy, especially for nonlinear problems where small random inputs may induce largerandom outputs (e.g., [56]). Stochastic Galerkin methods, whose convergence can beestablished for problems with relatively large magnitude of random inputs/outputs,have been developed. The stochastic Galerkin methods are generalizations of theWienerHermite polynomial chaos expansion, which was developed in [51] and hasbeen applied to various problems in mechanics [21]. The generalizations, also calledgeneralized polynomial chaos, employ non-Hermite polynomials to improve efficiencyfor a wider class of problems and include global polynomial expansions based on hy-pergeometrical polynomials [53, 52, 54], piecewise polynomial expansions [2, 12], andwavelet basis expansions [29, 30]. In general, these methods exhibit fast convergencerates with increasing order of the expansions, provided that solutions are sufficientlysmooth in the random space. However, the resulting set of deterministic equations isoften coupled and care is needed to design efficient and robust solver. (See [52, 55]for examples of diffusion equations.) The form of the resulting equations can becomevery complicated if the underlying differential equations have nontrivial and nonlinearforms. (See [7, 37, 57] for examples of hyperbolic systems.)

In this paper, we propose a class of stochastic collocation methods that combinesthe strength of Monte Carlo methods and stochastic Galerkin methods. By taking ad-vantage of the existing theory on multivariate polynomial interpolations (e.g., [5, 6]),the stochastic collocation methods achieve fast convergence when the solutions possesssufficient smoothness in random space, similar to stochastic Galerkin methods. Onthe other hand, the implementations of stochastic collocation methods are straight-forward as they only require solutions of the corresponding deterministic problems

at each interpolation point, similar to Monte Carlo methods at each sampling point.Such properties make stochastic collocation methods good alternatives to Monte Carlomethods and stochastic Galerkin methods. The core issue is the construction of theset of interpolation points, and it is nontrivial in multidimensional random spaces.For such a cases, point sets based on tensor products of one-dimensional quadraturepoints [37, 53] are not suitable, as the number of points grows too fast with increasingdimensions. Therefore, we investigate and propose two other choices. One is basedon Strouds cubature points [49] and the other on sparse grids from the Smolyak


3/22

1120 DONGBIN XIU AND JAN S. HESTHAVEN

algorithm [46]. The former approach is of relatively low-order accuracy but resultsin highly efficient algorithms, especially for problems with large dimensional randominputs. The latter offers true high-order accuracy with convergence rate dependingweakly on dimensionality. Both methods are more efficient than brute-force Monte

Carlo methods in a large range of random dimensionality. Compared to stochas-tic Galerkin methods, the collocation methods generally result in a larger number ofequations than a typical Galerkin method; however, these equations are easier to solveas they are completely decoupled and require only repetitive runs of a deterministicsolver. Such a property makes the collocation methods more attractive for problemswith complicated governing equations.

This paper is organized as follows. In section 2 we present the mathematicalframework of stochastic differential equationsthe functional spaces and the strongand weak forms are formulated in section 2.1 and we briefly review the Monte Carlomethods and stochastic Galerkin methods in section 2.2. The stochastic collocationmethods are discussed in section 3, where the construction of collocation points isdiscussed in detail. More technical details and numerical examples are presented insections 4 and 5, and we conclude the paper in section 6.

2. Random differential equations. Let (, A, P) be a complete probabilityspace, where is the event space, A 2 the -algebra, and P the probabilitymeasure. Consider a d-dimensional bounded domain D Rd (d = 1, 2, 3) withboundary D, and we study the following problem: find a stochastic function, u u(, x) : D R, such that for P-almost everywhere , the following equationholds:

L (, x; u) = f(, x), x D,(2.1)

subject to the boundary condition

B(, x; u) = g(, x), x D,(2.2)

where x = (x1, . . . , xd) are the coordinates in Rd, L is a (linear/nonlinear) differential

operator, and B is a boundary operator. The operator B can take various formson different boundary segments, e.g., B I on Dirichlet segments (where I is theidentity operator) and B n on Neumann segments (where n is the outward unitnormal vector). In the most general settings, the operators L and B, as well as thedriving terms f and g, can all have random components. Finally, we assume that theboundary D is sufficiently regular and the driving terms f and g are properly posed,such that (2.1)(2.2) is well-posed P-a.e. . (With a slight abuse of notation,(2.1)(2.2) can incorporate time-dependent problems, if we consider D to be in an(Rd+1)-dimensional space, with the extra dimension designated to time.)

2.1. Strong and weak forms. To solve (2.1)(2.2) numerically, we need to re-duce the infinite-dimensional probability space to a finite-dimensional space. This canbe accomplished by characterizing the probability space by a finite number of randomvariables. Such a procedure, termed the finite-dimensional noise assumption in [2],is often achieved via a certain type of decomposition which can approximate the tar-get random process with desired accuracy. One of the choices is the KarhunenLoevetype expansion [33], which is based on the spectral decomposition of the covariancefunction of the input random process. Following a decomposition and assuming thatthe random inputs can be characterized by N random variables, we can rewrite the


4/22


random inputs in abstract form, e.g.,

L(, x; u) = L(Y1(), . . . , Y N(), x; u), f(, x) = f(Y1(), . . . , Y N(), x),(2.3)

where {Yi}Ni=1 are real random variables with zero mean value and unit variance.Hence, following the DoobDynkin lemma [43], the solution of (2.1)(2.2) can bedescribed by the same set of random variables {Yi()}Ni=1, i.e.,

u(, x) = u(Y1(), . . . , Y N(), x).(2.4)

By characterizing the probability space with N random variables, we have effectivelyreduced the infinite-dimensional probability space to a N-dimensional space.

From a mathematical and computational point of view, it is most convenient tofurther assume that these random variables are mutually independent, and subse-quently, we can define the corresponding Hilbert functional spaces via simple tensorproduct rules. However, such an assumption could potentially introduce more errors

in approximating the input random processes (in addition to the errors induced bytruncating the infinite series during a decomposition procedure such as the KarhunenLoeve expansion). For example, for non-Gaussian processes, the KarhunenLoeve ex-pansion yields a set of uncorrelatedrandom variables that are not necessarily mutuallyindependent. To transform them into independent random variables, a (nonlinear)transformation is needed and its numerical construction can be nontrivial. In fact, amathematically rigorous transformation exists by using merely the definition of jointcumulative density function (CDF) [44]. However, such a transformation is of littlepractical use as it requires the knowledge of all the joint CDFs of the underlyingrandom process.

In this paper, we take the customary approach (see, for example, [2, 30, 52])of assuming that the random inputs are already characterized by a set of mutuallyindependent random variables with satisfactory accuracy. We emphasize that how to

(numerically) achieve this, i.e., to represent continuous non-Gaussian random inputsvia the KarhunenLoeve decomposition or any other representation and to ensurethat the random variables involved are independent, falls into the topic of numericalrepresentation of non-Gaussian processes. It remains an active research area (e.g.,[23, 45, 58]) and is beyond the scope of this paper. We also remark that it is possibleto construct multidimensional functional spaces based on finite number of dependentrandom variables [47]. However, such a construction does not, in its current form,allow straightforward numerical implementations.

Let us now assume that {Yi}Ni=1 are independent random variables with probabil-ity density functions i :

i R+ and their images i Yi() are bounded intervalsin R for i = 1, . . . , N . Then

(y) =

Ni=1

i(Yi

) y (2.5)

is the joint probability density of y = (Y1, . . . , Y N) with the support

Ni=1

i RN.(2.6)


5/22


This allows us to rewrite (2.1)(2.2) as an (N + d)-dimensional differential equationin the strong form

L (y, x; u) = f(y, x), (y, x) D,(2.7)

subject to the boundary condition

B(y, x; u) = g(y, x), (y, x) D,(2.8)where N is the dimensionality of the random space and d is the dimensionality ofthe physical space D.

Similar to the weak formulations in deterministic problems, we often seek tosolve (2.7)(2.8) in a weak form. For example, we can define a (finite-dimensional)subspace V L2(), the space of all square integrable function in with respect tothe measure (y)dy, and seek uV(y, x) V(y), such that

(y)L (y, x; uV) v(y)dy =

(y)f(y, x)v(y)dy v(y) V, x D,(2.9)

(y)B(y, x, uV)v(y)dy =

(y)g(y, x)v(y)dy v(y) V, x D.(2.10)

The resulting problem (2.9)(2.10) thus becomes a deterministic problem in the phys-ical domain D and can be solved by a standard discretization technique, e.g., finiteelements, finite volume.

2.2. Existing numerical methods. Several numerical methods can be appliedto problem (2.7)(2.8), or (2.9)(2.10). Here we briefly review two of the more popularmethodsMonte Carlo methods and stochastic Galerkin methodswhose strengthscomplement each other.

2.2.1. Monte Carlo methods. Monte Carlo simulation is one of the mostdeveloped methods for solving stochastic differential equations. Although the con-vergence rate is relatively slowa typical Monte Carlo simulation consisting of Krealizations converges asymptotically as 1/

K [13]it is independent of the number

of random variables {Yi()}Ni=1. The procedure of applying a Monte Carlo methodto problem (2.7)(2.8) takes the following steps:

1. For a prescribed number of realizations K, generate independent and iden-tically distributed (i.i.d.) random variables {Yij }Ni=1 {Yi(j)}Ni=1, for

j = 1, . . . , K ;2. For each j = 1, . . . , K , solve a deterministic problem (2.7)(2.8) with yj =

(Y1j , . . . , Y Nj ) and obtain solution uj u(yj , x), i.e., solve L(yj , x; uj) =

f(yj , x) in D, subject to B(yj , x; uj) = g(yj , x) on D;3. Postprocess the results to evaluate the solution statistics, e.g., u E[u] =

1MM

j=1 uj .Here E[u] =

(y)u(y)dy is the expectation. Note that for each realization j =

1, . . . , K , step 2 is a deterministic problem and can be solved by any suitable scheme.

2.2.2. Stochastic Galerkin methods. Stochastic Galerkin methods deal withthe weak form (2.9)(2.10), and they offer fast convergence when the solution issufficiently smooth in the random space. Forms of stochastic Galerkin methods differin the construction of the subspace V in (2.9). Here we briefly review the construction


6/22


based on global orthogonal polynomial expansions. Let us define one-dimensionalsubspaces for L2i(

i),

Wi,pi

v : i

R : v

spanm(Yi)

pi

m=0 , i = 1, . . . , N,(2.11)where {m(Yi)} are a set of orthogonal polynomials satisfying the orthogonality con-ditions

i

i(Yi)m(Y

i)n(Yi)dYi = h2mmn,(2.12)

with normalization factors h2m =i

i2mdY

i. The type of the orthogonal polyno-

mials {m(Yi)} in (2.12) is determined by i(Yi), the probability density function ofYi(), for i = 1, . . . , N . For example, uniform distributions are better approximatedby Legendre polynomials, and Hermite polynomials are more appropriate for Gaus-sian distributions. (See [53, 52] for a detailed list of correspondences.) We remarkthat such a correspondence is not mandatory, so long as

{m(Y

i)}

forms a completebasis.

The corresponding N-dimensional subspace V for can be defined as

WpN |p|p

Wi,pi ,(2.13)

where the tensor product is over all possible combinations of the multi-index p =(p1, . . . , pN) NN0 satisfying |p| =

Ni=1pi p. Thus, WpN is the space of N-

variate orthogonal polynomials of total degree at most p, and the number of basisfunctions in (2.13) is dim(WpN) =

N+pN

. Hereafter, we will refer to WpN as a complete

polynomial space. Such spaces are often employed in the (generalized) polynomialchaos expansions [21, 52, 54].

Another construction ofV is based on the tensor products of the one-dimensionalpolynomial spaces with fixed highest polynomial orders in all dimensions, i.e.,

ZpN =

Ni=1

Wi,pi , maxi

pi = p.(2.14)

Such a space will be denoted as tensor polynomial space hereafter. They are employedin [2, 15], and the number of basis functions in ZpN is dim(Z

pN) = (p + 1)

N.If the same one-dimensional subspaces (2.11) are employed, then WpN ZpN. The

numbers of basis functions in both spaces depend on the number of dimensions N andthe polynomial orders p, and they grow fast with increasing N and p. In particular,the total number of terms can grow exponentially fast with increasing order p whenN

1 (curse-of-dimensionality). We also remark that when N

1 and p is moderate

(e.g. p 10), dim(WpN) dim(ZpN).For low to moderate value ofN, stochastic Galerkin methods are preferred because

their high accuracy and fast convergence; for large value ofN 1, the rapidly growingnumber of basis functions effectively reduces its efficiency. In this case, the MonteCarlo method is preferred. However, if a highly accurate stochastic solution is desired,then stochastic Galerkin methods can be more efficient even for relatively large valueof N (e.g., [2]).


7/22


3. Stochastic collocation methods. In this section, we present high-ordercollocation methods for (2.1)(2.2). The objective is to combine the strength of thetwo existing approaches: the high resolution of stochastic Galerkin methods resultingfrom polynomial approximations in random spaces, and the ease of implementation

of Monte Carlo methods by sampling at discrete points in random spaces.3.1. Formulation. The construction of high-order stochastic collocation meth-

ods is based on polynomial interpolations in the multidimensional random space. Letus denote by y = (Y1, . . . , Y N) any point in the random space RN, by N thespace of all N-variate polynomials with real coefficients, and by pN the subspace ofpolynomials of total degree at most p. (Note that the space WpN defined in (2.13) isa subset of pN.) When derivatives are not interpolated, the Lagrange interpolationproblem can be stated in the following form.

Definition 3.1 (Lagrange interpolation). Given a finite number of distinctpoints y1, . . . , yM, some real constants b1, . . . , bM, and a subspace VI N, find apolynomial l VI such that

l(yj) = bj , j = 1, . . . , M .(3.1)

The points y1, . . . , yM are called interpolation nodes, and VI is the interpolation space.Definition 3.2 (poisedness). The Lagrange interpolation problem in Definition

3.1, for the points y1, . . . , yM RN, is called poised in VI if, for any given datab1, . . . , bM R, there exists a function f VI such that f(yj) = bj, j = 1, . . . , M .When the Lagrange interpolation problem for any M distinct points inRN is poisedin VI, then VI is called a Haar space of order M.

There is a well-developed and extensive classical theory of univariate Lagrangepolynomial interpolation. However, the multivariate case is more difficult. In par-ticular, the Haar spaces exist in abundance for N = 1 but not for N > 1. In fact,in this case there are no Haar spaces of dimension greater than onethe so-calledloss-of-Haar. (For more details and the refinement of this statement, see [11, 35].)Thus, research has focused on the selection of points {yi}Mi=1 such that one achievesa good approximation by a Lagrange interpolation.Let N = {yi}Mi=1 be a set of (prescribed) nodes such that the Lagrangeinterpolation (3.1) in the N-dimensional random space is poised in a correspondinginterpolation space VI. (For sufficient conditions on the distribution of the nodesto ensure a unique interpolation, see [8].) Subsequently, Lagrange interpolation ofa smooth function, f : RN R, can now be viewed as follows: find a polynomialI(f) VI such that I(f)(yi) = f(yi), i = 1, . . . , M .

The polynomial approximation I(f) can be expressed by using the Lagrangeinterpolation polynomials, i.e.,

I(f)(y) =M

k=1

f(yk)Lk(y),(3.2)

where

Li(y) VI, Li(yj) = ij , 1 i, j M,(3.3)are the Lagrange polynomials.

We equip the space VI with the supremum-norm and introduce

I = supf=0

Iff .(3.4)


8/22


The Lebesgue theorem states that the interpolation error is uniformly bounded as

f(y) f(y) f(y) If(y) (1 + ) f(y) f(y),(3.5)

where f

(y) is the best approximating polynomial, and = I = maxyK

k=1 |Li(y)|is the Lebesgue constant. We remark that the determination of f(y) is an unsolvedproblem for arbitrary (N > 1)-dimensional spaces, and the estimation of the Lebesgueconstant, which depends on the particular choice of nodal sets N, is a nontrivial taskeven for N = 1.

By denoting

u(y) Iu(y) =Mk=1

u(yk)Lk(y),(3.6)

the collocation procedure to solve the stochastic elliptic equation (2.7) is

R (u(y))

|yk

= 0

k = 1, . . . , M,(3.7)

where R(u) = L(u) f is the residual of (2.7). By using the property of Lagrangeinterpolation (3.3), we immediately obtain: for k = 1, . . . , M ,

L(yk, x; u) = f(yk, x), x D,(3.8)

with boundary condition

B(yk, x; u) = g(yk, x), x D.(3.9)

Thus, the stochastic collocation method is equivalent to solving M deterministic prob-lems (3.8)(3.9), the deterministic counterpart of problem (2.7)(2.8), at each nodalpoint yk, k = 1, . . . , M in a given nodal set N. Note that the problem (3.8)(3.9)for each k is naturally decoupled, and existing deterministic solvers can be readily

applied. This is in direct contrast to the stochastic Galerkin approaches, where theresulting expanded equations are, in general, coupled.

Once the numerical solutions of (3.8)(3.9) are obtained at all collocation points,the statistics of the random solution can be evaluated, e.g.,

E(u)(x) =Mk=1

u(yk, x)

Lk(y)(y)dy.(3.10)

The evaluations of such expectations require explicit knowledge of the Lagrange in-terpolation polynomials {Lk(y)}. For a given nodal set N, the polynomials can bedetermined numerically by inverting a Vandermonde-type matrix. (The matrix is in-vertible under the assumption that the Lagrange interpolation is poised on N.) In

multivariate cases, such a procedure can be cumbersome, but can be accomplishedonce and for all at the preprocessing stage.An alternative is to choose the set N to be a cubature point set. Recall that a

cubature rule approximates an integral by

f(y)dy Mi=1

f(yi)wi,(3.11)


9/22


where {wi} are the weights. It is called of order s if the approximation is exactfor all f rN, r s, and not exact for at least one f s+1N . When we choose theinterpolation point set N to be the same as a cubature point set, the evaluation of(3.10) is reduced to

E(u)(x) =

Mk=1

u(yk, x)wk.(3.12)

It should be noted that the use of cubature formula to evaluate the solution statisticswill introduce extra errors whenever the integrals defining these statistics, e.g., (3.10)for mean solution, cannot be evaluated exactly by an s-order cubature.

Construction of cubature formulas has long been an active and challenging re-search area, and we refer interested readers to [10, 9] for extensive reviews.

3.2. Choices of collocation points. The computational complexity of thestochastic collocation methods (3.8)(3.9) is M times that of a deterministic problem,where M is the total number of collocation points. Thus, we need to choose a nodal

set N with fewest possible number of points under a prescribed accuracy require-ment. In this section, we present several choices of such collocation points. The mainfocus here is on multidimensional random spaces, i.e., RN with N > 1. Withoutloss of generality, we assume that the bounded support of the random variables Yi isi = [1, 1] for i = 1, . . . , N and subsequently the bounded random space is a N-hypercube, i.e., = [1, 1]N. (Note that random variables with bounded support in[a, b] can always be mapped to [1, 1], and any domain = Ni=1[ai, bi] to [1, 1]N.)

3.2.1. Tensor products of one-dimensional nodal sets. A natural choice ofthe nodal set is the tensor product of one-dimensional sets. In interpolating smoothfunctions f : [1, 1]N R, much is known about good interpolation formulas forN = 1, i.e., for every direction i = 1, . . . , N , we can construct a good one-dimensionalinterpolation

Ui(f) =mik=1

f(Yik ) aik(3.13)

based on nodal sets

i1 = (Yi1 , . . . , Y

imi

) [1, 1],(3.14)

where aik ak(Yi) C([1, 1]). We assume that a sequence of formulas (3.13) isgiven. In the multivariate case N > 1, the tensor product formulas are

I(f) Ui1

U iN

(f) =

mi1

k1=1

miN

kN=1

f(Y

i1

k1 , . . . , Y

iN

kN ) (ai1

k1 aiN

kN ).

(3.15)

Clearly, the above product formula needs M = mi1 miN nodal points. If we chooseto use the same interpolating function (3.13) in each dimension with the same numberof points, i.e., mi1 = = miN m, the total number of points is M = mN. Thisnumber grows quickly in high dimensions N 1even for a poor approximationwith two points (m = 2) in each dimension, M = 2N 1 for N 1.


10/22


The tensor product interpolation is easy to constructthe interpolation pointsand space are obtained by tensor products of the univariate ones. The Lagrangeformula are easily extended to this case, as can be found in [4, 24]. However, becauseof the rapidly growing number of interpolation nodes in high dimensions, we will not

consider this approach extensively in this paper.3.2.2. Sparse grids. In this section we propose to solve the stochastic colloca-

tion problem (3.8)(3.9) on a sparse grid constructed by the Smolyak algorithm. TheSmolyak algorithm is a linear combination of product formulas, and the linear combi-nation is chosen in such a way that an interpolation property for N = 1 is preservedfor N > 1. Only products with a relatively small number of points are used andthe resulting nodal set has significantly less number of nodes compared to the tensorproduct rule (3.15). Much research has been devoted to the Smolyak algorithm sinceits introduction in [46]; see, e.g., [3, 41, 42].

Starting with the one-dimensional interpolation formula (3.13), the Smolyak al-gorithm is given by (see [50])

I(f) A(q, N) = qN+1|i|q

(1)q|i|1

N

1

q |i| Ui1 U iN ,(3.16)where i = (i1, . . . , iN) NN. To compute A(q, N), we only need to evaluate functionon the sparse grid

N H(q, N) =

qN+1|i|q

(i11 iN1 ).(3.17)

In this paper, we choose to use Smolyak formulas that are based on one-dimensionalpolynomial interpolation at the extrema of the Chebyshev polynomials. (Other choices,such as the Gauss quadrature points, can be considered as well.) For any choice ofmi > 1, these nodes are given by

Yij = cos (j 1)

mi 1 , j = 1 . . . , mi.(3.18)

In addition, we define Yi1 = 0 if mi = 1. The functions aij in (3.13) are characterized

by the requirement thatUi reproduce all polynomials of degree less than mi. We alsochoose m1 = 1 and mi = 2

i1 for i > 1. By doing so, the one-dimensional nodal setsi1 are nested, i.e.,

i1 i+11 , and subsequently H(q, N) H(q + 1, N).

It has been shown that if we set q = N + k, then A(N + k, N) is exact for allpolynomials in kN [42], and the total number of nodes for N 1 is [41]

M dim(A(N + k, N)) 2k

k!Nk, k fixed, N 1.(3.19)

On the other hand, dim(kN) =N+kN

Nk/k! for N 1. Thus, A(N + k, N)uses about 2k times as many points as the degree-of-freedom of kN. Since thisfactor is independent of N, the algorithm may be regarded as optimal. Formula(3.19) also indicates that the number of nodes from the Smolyak sparse grid is largerthan the number of expansion terms of a stochastic Galerkin method in the completepolynomial space WkN. Hereafter, we will refer k in A(N+ k, N) as the level of theSmolyak construction.


11/22


1 0.8 0.6 0.4 0.2 0 0.2 0.4 0.6 0.8 11

0.8

0.6

0.4

0.2

0

0.2

0.4

0.6

0.8

1

1 0.8 0.6 0.4 0.2 0 0.2 0.4 0.6 0.8 11

0.8

0.6

0.4

0.2

0

0.2

0.4

0.6

0.8

1

Fig. 3.1. Two-dimensional (N = 2) interpolation nodes based on the extrema of Chebyshevpolynomials (3.18). Left: sparse gridH(N+ k, N) from Smolyak algorithm, k = 5. Total number ofpoints is 145. Right: tensor product algorithm (3.15) from the same one-dimensional nodes. Total

number of nodes is 1, 089.

Table 3.1The degrees-of-freedom of the Smolyak sparse grid (H(N + k, N) = dim(A(N + k, N))), the

complete polynomial space WkN

, and the tensor polynomial space ZkN

, for various dimensions N andorder k.

N k H(N + k, N) dim(WkN

) dim(ZkN

)

2 1 5 3 42 13 6 93 29 10 164 65 15 25

10 1 21 11 1, 0242 221 66 59, 0493 1, 581 286 1, 048, 576

20 1 41 21 1, 048, 5762 841 231 3.5 109

50 1 101 51 1.1 1016

2 5, 101 1, 326 7.2 1023

In Figure 3.1, we show the two-dimensional (N = 2) interpolation nodes by thesparse grid H(N + k, N) described above, with k = 5. For comparison, the two-dimensional tensor product grid based on the same one-dimensional nodes are shownon the right of Figure 3.1, and we observe that the sparse grid has significantly fewernodes.

In Table 3.1 we list the number of nodes of the Smolyak sparse grid H(N+k, N) =dim(A(N + k, N)), the degrees-of-freedom of the complete polynomial space WkN(2.13), and the tensor polynomial space ZkN (2.14). (As shown in [42], the inter-polation by A(N + k, N) is exact in a polynomial space richer than WkN but not asrich as ZkN.) It can be seen that as N grows, the number of nodes from the Smolyaksparse grid is significantly less than dim(ZkN). Also, the relation (3.19) can be ob-served. This indicates that only at low dimensions, e.g., N < 5, should the tensorproduct construction of interpolation nodes be considered.

The interpolation error in a space FlN = {f : [1, 1]N R | |m|f continues, mi l, i}, where m NN0 and |m| is the usual N-variate partial derivative of order |m|,


12/22


is (cf. [3])

IN A(q, N) CN,l Ml (log M)(l+2)(N1)+1,(3.20)

where IN

is the identity operator in a N-dimensional space and M = dim(H

(q, N) isthe number of interpolation points.

Another intriguing property of the sparse grid constructed above is that it isalso the nodes of a good cubature formula (3.11). Thus, we can efficiently evaluatethe solution statistics in a similar way of (3.12). For analysis on integration errorestimates of Smolyak sparse grid for multivariate integrals, see [41].

3.2.3. Strouds cubature of degrees 2 and 3. For the N-hypercube spaces[1, 1]N considered in this paper, Stroud constructed a set of cubature points with(N + 1)-point that is accurate for multiple integrals of polynomials of degree 2, anda set of degree 3 with 2N-point [49]. The degree 2 formula, termed the Stroud-2method hereafter, consists of points {yk}Nk=0 such that

Y2r1

k =2

3 cos

2rk

N + 1 , Y2r

k =2

3 sin

2rk

N + 1 , r = 1, 2, . . . , [N/2],(3.21)

where [N/2] is the greatest integer not exceeding N/2, and if N is odd YNk =(1)k/3. The degree 3 formula, which will be called as Stroud-3 method, hasthe points {yk}2Nk=1 such that

Y2r1k =

2

3cos

(2r 1)kN

, Y2rk =

2

3sin

(2r 1)kN

, r = 1, 2, . . . , [N/2],

(3.22)

and if N is odd YNk = (1)k/

3.It was proved in [38] that the Stroud-2 and -3 methods employ the minimal num-

ber of points for their corresponding integration accuracy. However, to the authors

knowledge, no theory exists on the approximation properties of the Lagrange inter-polations constructed on these nodes. It is expected that the interpolation errors willbe lower than degree 2 and 3 for the Stroud-2 and -3 method, respectively.

In this paper, we will employ the Stroud-2 and -3 nodes for stochastic collocationmethods. For relatively low-order accuracy requirement in the solution statistics(expectations), the Stroud methods are highly efficient especially in high-dimensionalspaces (N 1) because the numbers of nodal points are minimal. However, theaccuracy cannot be improved any further and no error estimates can be obtained.

4. Applications to stochastic elliptic equations. In this section we use astochastic elliptic problem to illustrate the applications of stochastic collocation meth-ods. In particular, we consider for P-a.e. ,

(a(, x)u(, x)) = f(, x), in D,u(, x) = 0, on D,(4.1)

where a, f : D R are known stochastic functions. Standard assumptionsare made on the regularity of the input functions so that (4.1) is well-posed, e.g.,a L(, D) is strictly positive.

Again let us assume that the random input functions can be characterized byN independent random variables {Yi()}Ni=1, whose images are bounded intervals


13/22


and define an N-dimensional random space RN. Without loss of generality, let = [1, 1]N. Problem (4.1) can be then expressed in the strong form,

(a(y, x)u(y, x)) = f(y, x) (y, x) D,

u(y, x) = 0 (y, x) D,(4.2)

and weak variational form: find u L2() H10 (D) such that

K(u, v; ) =I(f, v) v L2() H10 (D),(4.3)where

I(v, w) =

(y)

D

v(y, x)w(y, x)dxdy

and

K(v, w; ) =

(y)

D

a(y, x)v(y, x) w(y, x)dxdy.

4.1. Formulations and error estimates. A stochastic collocation method,combined with a finite element method (FEM) in the physical space D Rd, takesthe following form: for a given set of collocation nodes N = {yk}Mk=1 on which theLagrange interpolation is poised in an interpolation space VI L2(), and a piecewise(continuous) FEM mesh Xhd H10 (D) (where h > 0 is its mesh spacing parameter)that satisfies all the standard requirements on a FEM mesh, find as follows: fork = 1, . . . , M , uhk(x) uh(yk, x) Xhd , such that

D

a(yk, x)uhk(x) (x)dx =D

f(yk, x)(x)dx Xhd .(4.4)

The final numerical solution takes the form

uh(y, x) =

Mk=1

uhk(x)Lk(y).(4.5)

To study error estimates, let us define an error in term of the deterministic energyerror, i.e.,

EG E

D

a((u uh))2dx1/2

.(4.6)

We can show that

EG =

(y)D

a(y, x)((u uh))2dxdy1/2

aL(D)1/2 minvVIXdh u vL2()H10(D) aL(D)

1/2

(ED + E),

(4.7)

where ED = minvL2()Xdh u vL2()H10 (D) is the standard H10 (D) FEM approxi-

mation error, and E is the L2() approximation error in the random space satisfying

E = minvVIH

1

0(D)

u vL2()H10(D) u I(u)L2()H10(D) u I(u) = EI.(4.8)


14/22


The last term is the interpolation error, which can be bounded by the Lebesguetheorem (3.5). For the interpolation by Smolyak sparse grid, this error is (3.20). Wehence have the following proposition.

Proposition 4.1 (separation of error estimates). The energy error defined in

(4.6) can be effectively separated into two parts,

EG C(ED + EI),(4.9)where ED is the standardH10 (D) FEM error in physical space D, EI is the interpolationerror in random space , and C is a constant independent of the discretizations inboth D and .

4.2. Numerical results. Numerical implementations of stochastic Galerkin meth-ods are not considered here as they have been considered in many earlier publications.For Galerkin methods in tensor polynomial spaces, see [12, 1, 2, 15]; in complete poly-nomial spaces, see [19, 20, 21, 52, 54, 55]. Here we focus on numerical examples ofstochastic collocation methods and consider the following problem in one spatial di-mension (d = 1) and N > 1 random dimensions:

d

dx

a(y, x)

du

dx(y, x)

= 0, (y, x) (0, 1),(4.10)

with boundary conditions

u(y, 0) = 0, u(y, 1) = 1, y .The use of one-dimensional physical space D allows us to focus on the properties ofthe stochastic collocation methods associated with the random space . Extensionsto two- and three-dimensional physical spaces are straightforward. We assume thatthe random diffusivity has the form

a(y, x) = 1 +

N

k=1

1

k22 cos(2kx)Y

k

(),(4.11)

where Yk() [1, 1], k = 1, . . . , N , are independent uniformly distributed randomvariables. The form of (4.11) is similar to those obtained from a KarhunenLoeveexpansion with eigenvalues delaying as 1/k4. Here we employ (4.11) to eliminate theerrors associated with a numerical KarhunenLoeve solver and to keep the randomdiffusivity strictly positive as N . The series in (4.11) converge as N andwe have

E(a(y, x)) = 1, 1 6

< a(y, x) < 1 +

6.

The spatial discretization is done on a p-type spectral element with high-order modalexpansions. The basis functions () in a standard interval

(

1, 1) are defined as

p() =

12 , p = 0,

12

1+2 P

1,1p1(), 0 < p < P,

1+2 , p = P,

(4.12)

where P,p () is the Jacobi polynomial of order p with parameters , > 1. Formore details of such a basis, see [26]. In all the following computations, the spatial


15/22


1 1.5 2 2.5 3 3.5 4 4.5 5 5.5 610

20

1018

1016

1014

1012

1010

108

106

104

102

100

Quadrature level

Errors

mean

variance

skewness

kurtosis

100

101

102

103

104

105

1016

1014

1012

1010

108

106

104

102

100

102

104

DOF

Errors

mean

variance

skewness

kurtosis

Fig. 4.1. Convergence of errors in moments with two-dimensional random input (N = 2).Left: convergence with respect to level of approximation k of Smolyak formula A(N + k, N); Right:convergence with respect to number of quadratures of different methods, Smolyak sparse grid (solidlines), Strouds cubature (dotted lines), and Monte Carlo simulation (dashed lines).

discretizations are conducted with sufficiently high-order basis such that the spatialerrors are subdominant.

The exact solution to this problem is

ue(y, x) = c1

1

a(y, x)dx + c2,

where the two constants c1, c2 are determined by the boundary conditions. However,the closed form expression for the exact solution is not readily available.

4.2.1. Low-dimensional random inputs: Accuracy. In this section, a low-dimensional random input is assumed. In particular, we choose N = 2. The exactsolution is obtained by using the Smolyak sparse grid with level 8, i.e., A(N+ 8, N) =

A(10, 2).In Figure 4.1, the errors in the first four moments of the solution are shown. Onthe left, the error of the Smolyak collocation method is shown. We observe that asthe approximation level k ofA(N+ k, N) increases, the errors converge exponentiallyfast before reaching saturate levels.

On the right of Figure 4.1, the error convergence of a few different methods isshown with respect to the degree-of-freedom, i.e., the total number of interpolationpoints. It can be seen that the Smolyak method has the best accuracy; the Stroud-2 and -3 methods approximate the lower moments (mean and variance) well, withconsiderable less degrees-of-freedom (only three and four points for the Stroud-2 and-3 method, respectively). On the other hand, the standard Monte Carlo method hasthe expected O(DOF1/2) convergence.

4.2.2. Moderate-dimensional random inputs. Here we choose a moderatelyhigh-dimensional random space with N = 8. Figure 4.2 shows the error convergencewith respect to degrees-of-freedom for different methods. Again, the Stroud-3 methodoffers solutions with accuracy comparable to the Smolyak method at level one ( k = 1for A(N + k, N)), and with fewer nodes. In this case, both the Stroud method andSmolyak method are more efficient than Monte Carlo methods. We also observethat there is no clear sign of accuracy improvement in kurtosis, when comparing theStroud-3 and Stroud-2 methods. (See also the results in Figure 4.1.) The reason is


16/22


100

101

102

103

104

105

106

1014

1012

1010

108

106

104

102

100

102

DOF

Errors

meanvariance

skewnesskurtosis

Fig. 4.2. Convergence of errors in moments with eight-dimensional random input (N = 8).Smolyak sparse grid A(N + k, N), k 4 (solid lines), Strouds cubature (dotted lines), and Monte

Carlo simulation (dashed lines).

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

x

mean

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

1

2

3

4

5

6

7

8

9x 10

7

x

2

Fig. 4.3. Stochastic solution of (4.10) with 50-dimensional (N = 50) random input. Left: meansolution. Right: variance.

Table 4.1Solutions at high-dimensional random input (N = 50), with Stroud-2 and -3 method, Smolyak

method A(N+k, N), and Monte Carlo method. Degrees-of-freedom for polynomial spaces: for k = 1,dim(Wk

N) = 51, dim(Zk

N) 1.1 1016; for k = 2, dim(Wk

N) = 1, 326, dim(Zk

N) 7.2 1023.

Methods DOF max 2u(x)

Stroud 2 51 8.67914(7)Stroud 3 100 8.67916(7)

Smolyak (k = 1) 101 8.67925(7)Smolyak (k = 2) 5101 8.67919(7)

MCS 100 8.16677(7)MCS 5000 8.73434(7)

that kurtosis is the fourth moment of the solution and neither of the Strouds methodsis precise beyond third-order polynomials.


17/22


4.2.3. High-dimensional random inputs: Efficiency. In this section, wefocus on problem (4.10) with high-dimensional random input, N = 50. In Figure 4.3,the solution profiles of mean and variance are shown. The maximums of variance aretabulated in Table 4.1. It can be seen that the Stroud-2, -3 method and Smolyak

method with k = 2 agree with each other well. With similar numbers of degrees-of-freedom, Monte Carlo methods show noticeable differences. In this particular case,both the Stroud methods (Stroud-2 and Stroud-3) and the Smolyak method (k = 1and k = 2) are superior to the Monte Carlo method, and the Stroud-2 method isto be preferred, as it gives sufficiently accurate results (in variance) with only 51nodes. The collocation points based on tensor products and Galerkin methods intensor polynomial spaces ZkN are clearly not practical, as dim(Z

kN) 7.2 1023. The

Galerkin methods in the complete polynomial spaces have number of expansion termsof 51 and 1, 326 for k = 1, 2, respectively. The choice between a Galerkin method anda Smolyak collocation method in this case will depend on users, i.e., whether to use adeterministic solver more times or to design a solver for a coupled system of equationswith less terms.

5. Time dependent problem: An illustrative example. Although thestochastic collocation algorithm is discussed in the context of boundary value prob-lems, it is straightforward to apply it to time-dependent problems. Let us consider adamped motion of a particle in a bistable potential in the presence of a noisy periodicforcing. In particular, we consider for

dx

dt(t, ) = x x3 + f(t, ), x(0, ) = x0,(5.1)

where f(t, ) is a 2-periodic random process with zero mean value and a 2-periodiccovariance function, i.e., E[f(t, )] = 0, and E[f(t1, )f(t2, )] =

2fCf(|t1 t2|),

where 2f is the variance and Cf(t) = Cf(t + 2). We remark that with proper noisyinput on top of a periodic driving force, the system (5.1) can exhibit complicatedbehavior such as stochastic resonance. (See, for example, [17].) However, the purpose

of this example is on computational aspects of the algorithm, and we will restrictourselves to cases with relatively simple dynamical behavior.To model the periodic random process f(t, ), we use a Fourier-type expansion

f(t, ) =K

n=K

fn()eint(5.2)

for its characterization by a finite number of random variables. Here, the coefficientsfn() = f

rn() + if

in() are complex random variables. It is straightforward to show

that if frn and fin are statistically independent for all n and have zero mean and

variance of Cn/4, where Cn =1

20 Cf(t)cos(nt)dt are the coefficients of the Fourier

cosine series of the 2-periodic covariance function Cf, then the random field f(t, )given by (5.2) approximates the prescribed 2-periodic covariance function Cf

Cf

as K . Let y() = (fr0 , fr1 , fi1, . . . , f rK , fiK), then f(t, ) f(t, y()) has (2K+1)random dimensions. (Note fi0 does not take effect in (5.2).) Obviously, f(t, y()) is anapproximation of f(t, ), as (5.2) is a finite-term summation. In practice, we choosethe number of expansion terms so that the contributions from the truncated termsare sufficiently small.

In this numerical example, we choose a 2-periodic covariance function Cf byextending the standard Gaussian covariance function CG(t) = exp(t2/b2) to the


18/22


0 5 10 150

0.05

0.1

0.15

0.2

0.25

n

Cn

(a)

0 1 2 3 4 5 6 70.2

0

0.2

0.4

0.6

0.8

1

1.2

t

(b)

Fig. 5.1. Construction of a 2-periodic covariance function Cf. (a) Decay of Fourier cosinecoefficients Cn, n = 0 , 1, . . . ; (b) reconstructed periodic covariance function (solid line) by extendingthe regular Gaussian covariance function (dashed line) CG(t) = exp(t

2/b2) with correlation length

b = 0.4.

periodic domain (0, 2), where b is the correlation length. Hereafter we choose amoderately short correlation length b = 0.4. In Figure 5.1(a), the decay of the Fouriercosine coefficient of Cf is shown. Due to the relatively fast decay of these coefficients,

we choose to use K = 10 in the truncated series (5.2). Thus, f(t, y) has N = 2K+1 =21 random dimensions. The constructed covariance function Cf(t) is shown in Figure

5.1(b), along with the standard nonperiodic Gaussian covariance function CG, whichwe extended to the periodic domain.

Figure 5.2(a) shows four (arbitrarily chosen) realizations of f(t, y), constructedby N = 21(K = 10) dimensions with correlation length b = 0.4 and = 0.1. Corre-sponding to these four realizations of forcing, the four realizations of solution to (5.1),

with initial condition x0 = 0.5, are shown in Figure 5.2(b). In this case, all realiza-tions are attracted to one of the stable wells (x = 1) and oscillate nonlinearly aroundit. Subsequently, E[u(t)] 1 and E[u2(t)] 2u = const. as t . The computa-tional results of the evolution of the standard deviation u(t) is shown in Figure 5.3.Stochastic collocation based on the Smolyak grid A(N+ k, N) is conducted for k = 1and k = 2. With N = 21, A(N+ 1, N) has 43 points and A(N+ 2, N) has 925 points.The results of standard deviation with k = 2 shows little visual difference comparedto k = 1, and is regarded as the well-resolved solution. Results by Monte Carlo sim-ulation are also shown in Figure 5.3. We observe the numerical oscillations resultedfrom the statistical errors, and the reduction of such errors from 100 realizations to1,000 realizations is rather modest.

Problem (5.1) can also be solved via a stochastic Galerkin method. In the stochas-

tic Galerkin method, we expand the solution as u(t, y) = Mi=1

ui(t)i(y), where{i(y)} is the multidimensional orthogonal basis defined in (2.13). The resultingequations for the expansion coefficients take the form

dumdt

(t) = xm 12mMi=1

Mj=1

Mk=1

xixjxkijkm + fm(t), m = 1, . . . , M,

(5.3)


19/22


0 1 2 3 4 5 6 71

0.8

0.6

0.4

0.2

0

0.2

0.4

0.6

0.8

1

t

f(t)

(a)

0 5 10 15 20 25 30 350.5

0.6

0.7

0.8

0.9

1

1.1

1.2

t

u(t)

(b)

Fig. 5.2. Realizations of problem (5.1). (a) Four realizations of the periodic random forcing

f(t, y()); (b) four realizations of the solution u(t, y()) corresponding to the four realizations of

forcing on the left.

0 5 10 15 20 25 30 350

0.01

0.02

0.03

0.04

0.05

0.06

t

STD

SC

MCS: 100

MCS: 1,000

(a)

24 25 26 27 28 29 30 31 32

0.024

0.026

0.028

0.03

0.032

0.034

0.036

t

STD

SC: k=1

SC: k=2

MCS: 100

MCS: 1,000

(b)

Fig. 5.3. Numerical results of the evolution of standard deviation (STD) of (5.1), computed bystochastic collocation (SC) and Monte Carlo simulations (MCS). (a) Solutions to 0 t 10; (b)close-up view near t 10.

where {fi(t)} are the expansion coefficients for the random forcing term. Solutionof each of the M number of equations requires a M3 triple summations and storage(or on-the-fly evaluations) of the constants ijkm of the size M4. In the casehere, when N = 21, M = 22 for first-order expansion, and M = 252 for second-orderexpansion. Neither the storage nor computing of these is negligible.

6. Summary. A class of high-order collocation methods is proposed in this pa-per. The method is a forge of several known mathematical and numerical components,e.g., sampling method, cubature formula/sparse grid, and collocational method. Itsformulation is similar to that of a Monte Carlo method, where a deterministic equationis solved repetitively at each discrete point in the random space. However, in colloca-tion methods the distribution of collocation points is fixed deterministically a prioriand is determined through the aid of existing theory of multivariate polynomial inter-


20/22


polation (or integration). Furthermore, by constructing the appropriate polynomialinterpolations, stochastic collocation methods can achieve fast convergence (exponen-tial convergence under sufficient smoothness conditions), similar to stochastic Galerkinmethods. Thus, the stochastic collocation methods can be regarded as high-order

deterministic sampling methods. Also, when the interpolation error of certain col-location nodes is known, as in the case of the Smolyak sparse grid, we can obtain errorestimates in strong form (e.g., the error estimate in (4.7)), whereas in cubature sam-pling or statistical sampling methods the interpolation errors are usually not known.

Several choices of the collocation nodal sets were investigated. It was found thatthe sparse grid based on Smolyak algorithm offers high accuracy and fast convergence,with a weak dependence on the number of random dimensions. An alternative, basedon the Stroud-2 and -3 cubature, is optimal at lower orders. However, their accuracyis fixed and cannot be improved. Numerical experiments were conducted on an ellip-tic equation with random inputs, and we demonstrate that the error estimate can beeffectively separated into two parts: one for the standard spatial discretization errorand one for the interpolation error in random space. It was found that for randomdimensions as high as 50, the stochastic collocation methods (including both Stroudsmethods and Smolyak methods) are more efficient than brute-force Monte Carlo meth-ods. With low accuracy, the Stroud-3 method is preferred to the Smolyak methodwith degree one (k = 1). On the other hand, the implementation of a collocationmethod is embarrassingly trivial, when compared to a stochastic Galerkin methodwhich requires a specialized solver for the resulting coupled set of equations. However,the high-order collocation method based on Smolyak sparse grid has more degrees-of-freedom than Galerkin methods in complete polynomial spaces. An example of a timedependent nonlinear ODE is also presented. The collocation method exhibits goodconvergence and accuracy for a 21-dimensional random input. This problem has a cu-bic nonlinearity, and a stochastic Galerkin approach, albeit with less expansion termscompared to the sparse grid collocation method, would incur nonnegligible extra com-putation cost due to the nonlinearity. The choice between a collocation method and

a Galerkin method is problem dependent, and the advantage of collocation methodswill be more noticeable for problems with more complicated forms of governing equa-tions. A complete comparison between collocation and Galerkin methods depends onmany factors, including detailed error estimates (particularly, in this paper we didnot study the aliasing error by collocation methods) and assessment of the algorithmefficiency for a Galerkin method (for solving the coupled set of equations). This isour ongoing research.

REFERENCES

[1] I. Babuska and P. Chatzipantelidis, On solving elliptic stochastic partial differential equa-tions, Comput. Methods Appl. Mech. Engrg., 191 (2002), pp. 40934122.

[2] I. Babuska, R. Tempone, and G. Zouraris, Galerkin finite element approximations ofstochastic elliptic differential equations, SIAM J. Numer. Anal., 42 (2004), pp. 800825.

[3] V. Barthelmann, E. Novak, and K. Ritter, High dimensional polynomial interpolation onsparse grid, Adv. Comput. Math., 12 (1999), pp. 273288.

[4] I. Berezin and N. Zhidkov, Computing Methods, Addison-Wesley, Reading, MA, 1965.[5] C. Bernardi and Y. Maday, Properties of some weighted Sobolev spaces and application to

spectral approximations, SIAM J. Numer. Anal., 26 (1989), pp. 769829.[6] C. Canuto and A. Quarteroni, Approximation results for orthogonal polynomials in Sobolev

spaces, Math. Comput., 38 (1982), pp. 6786.[7] Q.-Y. Chen, D. Gottlieb, and J. Hesthaven, Uncertainty analysis for the steady-state flows

in a dual throat nozzle, J. Comput. Phys., 204 (2005), pp. 387398.


21/22


[8] K. Chung and T. Yao, On lattices admitting unique Lagrange interpolations, SIAM J. Numer.Anal., 14 (1977), pp. 735743.

[9] R. Cools, Monomial cubature rules since Stroud: A compilation, part 2, J. Comput. Appl.Math., 112 (1999), pp. 2127.

[10] R. Cools and P. Rabinowitz, Monomial cubature rules since Stroud: A compilation, J.

Comput. Appl. Math., 48 (1993), pp. 309326.[11] P. Davis, Interpolation and Approximation, Dover, New York, 1975.[12] M. Deb, I. Babuska, and J. Oden, Solution of stochastic partial differential equations using

Galerkin finite element techniques, Comput. Methods Appl. Mech. Engrg., 190 (2001),pp. 63596372.

[13] G. Fishman, Monte Carlo: Concepts, Algorithms, and Applications, Springer-Verlag, NewYork, 1996.

[14] B. Fox, Strategies for Quasi-Monte Carlo, Kluwer, Dordrect, The Netherlands, 1999.[15] P. Frauenfelder, C. Schwab, and R. Todor, Finite elements for el liptic problems with

stochastic coefficients, Comput. Methods Appl. Mech. Engrg., 194 (2005), pp. 205228.[16] D. Gamerman, Markov Chain Monte Carlo: Stochastic Simulation for Bayesian Inference,

Chapman & Hall, London, 1997.[17] L. Gammaitoni, P. Hanggi, P. Jung, and F. Marchesoni, Stochastic resonance, Rev. Modern

Phys., 70 (1998), pp. 223287.[18] C. Gardiner, Handbook of Stochastic Methods: For Physics, Chemistry and the Natural Sci-

ences, 2nd ed., Springer-Verlag, New York, 1985.

[19] R. Ghanem, Ingredients for a general purpose stochastic finite element formulation, Comput.Methods Appl. Mech. Engrg., 168 (1999), pp. 1934.[20] R. Ghanem, Stochastic finite elements for heterogeneous media with multiple random non-

Gaussian properties, ASCE J. Engrg. Mech., 125 (1999), pp. 2640.[21] R. Ghanem and P. Spanos, Stochastic Finite Elements: A Spectral Approach, Springer-Verlag,

New York, 1991.[22] W. Gilks, S. Richardson, and D. Spiegelhalter , Markov Chain Monte Carlo in Practice,

Chapman & Hall, London, 1995.[23] M. Grigoriu, Simulation of stationary non-Gaussian translation processes, J. Engrg. Mech.,

124 (1998), pp. 121126.[24] E. Isaacson and H. Keller, Analysis of Numerical Methods, John Wiley, New York, 1966.[25] I. Karatzas and S. Shreve, Brownian Motion and Stochastic Calculus, Springer-Verlag, New

York, 1988.[26] G. Karniadakis and S. Sherwin, Spectral/hp Element Methods for CFD, Oxford University

Press, London, 1999.[27] M. Kleiber and T. Hien, The stochastic finite element method, John Wiley, New York, 1992.

[28] P. Kloeden and E. Platen, Numerical Solution of Stochastic Differential Equations,Springer-Verlag, New York, 1999.[29] O. Le Maitre, O. Knio, H. Najm, and R. Ghanem, Uncertainty propagation using Wiener-

Haar expansions, J. Comput. Phys., 197 (2004), pp. 2857.[30] O. Le Maitre, H. Najm, R. Ghanem, and O. Knio , Multi-resolution analysis of Wiener-type

uncertainty propagation schemes, J. Comput. Phys., 197 (2004), pp. 502531.[31] W. Liu, T. Belytschko, and A. Mani, Probabilistic finite elements for nonlinear structural

dynamics, Comput. Methods Appl. Mech. Engrg., 56 (1986), pp. 6181.[32] W. Liu, T. Belytschko, and A. Mani, Random field finite elements, Internat. J. Numer.

Methods Engrg., 23 (1986), pp. 18311845.[33] M. Loeve, Probability Theory, 4th ed., Springer-Verlag, New York, 1977.[34] W. Loh, On Latin hypercube sampling, Ann. Statist., 24 (1996), pp. 20582080.[35] G. Lorentz, Approximation of Functions, Holt, Rinehart and Winston, New York, 1966.[36] N. Madras, Lectures on Monte Carlo Methods, American Mathematical Society, Providence,

RI, 2002.[37] L. Mathelin and M. Hussaini, A stochastic collocation algorithm for uncertainty analysis,

Tech. Report NASA/CR-2003-212153, NASA Langley Research Center, 2003.[38] I. Mysovskih, Proof of the minimality of the number of nodes in the cubature formula for a

hypersphere, USSR Comput. Math. Math. Phys., 6 (1966), pp. 1527.[39] H. Niederreiter, Random Number Generation and Quasi-Monte Carlo Methods, SIAM,

Philadelphia, 1992.[40] H. Niederreiter, P. Hellekalek, G. Larcher, and P. Zinterhof, Monte Carlo and Quasi-

Monte Carlo Methods 1996, Springer-Verlag, New York, 1998.[41] E. Novak and K. Ritter, High dimensional integration of smooth functions over cubes, Nu-

mer. Math., 75 (1996), pp. 7997.[42] E. Novak and K. Ritter, Simple cubature formulas with high polynomial exactness, Con-


22/22


structive Approx., 15 (1999), pp. 499522.[43] B. Oksendal, Stochastic Differential Equations. An Introduction with Applications, 5th ed.,

Springer-Verlag, New York, 1998.[44] M. Rosenblatt, Remark on a multivariate transformation, Ann. Math. Statist., 23 (1953),

pp. 470472.

[45] S. Sakamoto and R. Ghanem, Simulation of multi-dimensional non-Gaussian non-stationaryrandom fields, Probab. Engrg. Mech., 17 (2002), pp. 167176.

[46] S. Smolyak, Quadrature and interpolation formulas for tensor products of certain classes offunctions, Soviet Math. Dokl., 4 (1963), pp. 240243.

[47] C. Soize and R. Ghanem, Physical systems with random uncertainties: Chaos representationswith arbitrary probability measure, SIAM. J. Sci. Comput., 26 (2004), pp. 395410.

[48] M. Stein, Large sample properties of simulations using Latin hypercube sampling, Technomet-rics, 29 (1987), pp. 143151.

[49] A. Stroud, Remarks on the disposition of points in numerical integration formulas, Math.Comput., 11 (1957), pp. 257261.

[50] G. Wasilkowski and H. Wozniakowski, Explicit cost bounds of algorithms for multivariatetensor product problems, J. Complexity, 11 (1995), pp. 156.

[51] N. Wiener, The homogeneous chaos, Amer. J. Math., 60 (1938), pp. 897936.[52] D. Xiu and G. Karniadakis, Modeling uncertainty in steady state diffusion problems via

generalized polynomial chaos, Comput. Methods Appl. Math. Engrg., 191 (2002), pp. 49274948.

[53] D. Xiu and G. Karniadakis, The Wiener-Askey polynomial chaos for stochastic differentialequations, SIAM J. Sci. Comput., 24 (2002), pp. 619644.[54] D. Xiu and G. Karniadakis, Modeling uncertainty in flow simulations via generalized poly-

nomial chaos, J. Comput. Phys., 187 (2003), pp. 137167.[55] D. Xiu and G. Karniadakis, A new stochastic approach to transient heat conduction modeling

with uncertainty, Internat. J. Heat Mass Trans., 46 (2003), pp. 46814693.[56] D. Xiu and G. Karniadakis, Supersensitivity due to uncertain boundary conditions, Internat.

J. Numer. Meth. Engrg., 61 (2004), pp. 21142138.[57] D. Xiu and S. Sherwin, Uncertainty modelling in hyperbolic systems and its applications in

reduced modelling of a human arterial network, in Proceedings of International Conferenceof High-Order and Spectral Methods (ICOSAHOM), Providence, RI, 2004.

[58] F. Yamazaki and M. Shinozuka, Digital generation of non-Gaussian stochastic fields, J.Engrg. Mech., 114 (1988), pp. 11831197.

Documents

High-Order Collocation Methods for Differential Equations With Random Inputs