A Parallel, Finite-Volume Algorithm for Large-Eddy …NASA/TM-1999-206570 A Parallel, Finite-Volume Algorithm for Large-Eddy Simulation of Turbulent Flows Trong T. Bui Dryden Flight

NASA/TM-1999-206570

A Parallel, Finite-Volume Algorithm forLarge-Eddy Simulation of Turbulent Flows

Trong T. BuiDryden Flight Research CenterEdwards, California

January 1999

The NASA STI Program Office . . . in Profile

Since its founding, NASA has been dedicatedto the advancement of aeronautics and space science. The NASA Scientific and Technical Information (STI) Program Office plays a keypart in helping NASA maintain thisimportant role.

The NASA STI Program Office is operated byLangley Research Center, the lead center forNASA’s scientific and technical information.The NASA STI Program Office provides access to the NASA STI Database, the largest collectionof aeronautical and space science STI in theworld. The Program Office is also NASA’s institutional mechanism for disseminating theresults of its research and development activities. These results are published by NASA in theNASA STI Report Series, which includes the following report types:

• TECHNICAL PUBLICATION. Reports of completed research or a major significantphase of research that present the results of NASA programs and include extensive dataor theoretical analysis. Includes compilations of significant scientific and technical data and information deemed to be of continuing reference value. NASA’s counterpart of peer-reviewed formal professional papers but has less stringent limitations on manuscriptlength and extent of graphic presentations.

• TECHNICAL MEMORANDUM. Scientificand technical findings that are preliminary orof specialized interest, e.g., quick releasereports, working papers, and bibliographiesthat contain minimal annotation. Does notcontain extensive analysis.

• CONTRACTOR REPORT. Scientific and technical findings by NASA-sponsored contractors and grantees.

• CONFERENCE PUBLICATION. Collected papers from scientific andtechnical conferences, symposia, seminars,or other meetings sponsored or cosponsoredby NASA.

• SPECIAL PUBLICATION. Scientific,technical, or historical information fromNASA programs, projects, and mission,often concerned with subjects havingsubstantial public interest.

• TECHNICAL TRANSLATION. English- language translations of foreign scientific and technical material pertinent toNASA’s mission.

Specialized services that complement the STIProgram Office’s diverse offerings include creating custom thesauri, building customizeddatabases, organizing and publishing researchresults . . . even providing videos.

For more information about the NASA STIProgram Office, see the following:

• Access the NASA STI Program Home Pageat

http://www.sti.nasa.gov

• E-mail your question via the Internet to [email protected]

• Fax your question to the NASA Access HelpDesk at (301) 621-0134

• Telephone the NASA Access Help Desk at(301) 621-0390

• Write to:NASA Access Help DeskNASA Center for AeroSpace Information7121 Standard DriveHanover, MD 21076-1320

NASA/TM-1999-206570

A Parallel, Finite-Volume Algorithm forLarge-Eddy Simulation of Turbulent Flows

Trong T. BuiDryden Flight Research CenterEdwards, California

January 1999

National Aeronautics andSpace Administration

Dryden Flight Research CenterEdwards, California 93523-0273

NOTICE

Use of trade names or names of manufacturers in this document does not constitute an official endorsementof such products or manufacturers, either expressed or implied, by the National Aeronautics andSpace Administration.

Available from the following:

NASA Center for AeroSpace Information (CASI) National Technical Information Service (NTIS)7121 Standard Drive 5285 Port Royal RoadHanover, MD 21076-1320 Springfield, VA 22161-2171(301) 621-0390 (703) 487-4650

A PARALLEL, FINITE-VOLUME ALGORITHM FOR LARGE-EDDY SIMULATION OF TURBULENT FLOWS

Trong T. Bui*

NASA Dryden Flight Research CenterEdwards, California

Abstract

A parallel, finite-volume algorithm has beendeveloped for large-eddy simulation (LES) ofcompressible turbulent flows. This algorithm includespiecewise linear least-square reconstruction, trilinearfinite-element interpolation, Roe flux-differencesplitting, and second-order MacCormack timemarching. Parallel implementation is done using themessage-passing programming model. In this paper, thenumerical algorithm is described. To validate thenumerical method for turbulence simulation, LES offully developed turbulent flow in a square duct isperformed for a Reynolds number of 320 based on theaverage friction velocity and the hydraulic diameter ofthe duct. Direct numerical simulation (DNS) results areavailable for this test case, and the accuracy of thisalgorithm for turbulence simulations can be ascertainedby comparing the LES solutions with the DNS results.The effects of grid resolution, upwind numericaldissipation, and subgrid-scale dissipation on theaccuracy of the LES are examined. Comparison withDNS results shows that the standard Roe flux-differencesplitting dissipation adversely affects the accuracy of theturbulence simulation. For accurate turbulencesimulations, only 3–5 percent of the standard Roe flux-difference splitting dissipation is needed.

Nomenclature

A area

Roe flux-difference splitting matrix

area of duct side walls

c local speed of sound

C SGS model constant

A

As

1American Institute of Aero

*Trong T. Bui, Aerospace Engineer, AIAA member, (805) 258-2645, e-mail: [email protected].

Copyright 1999 by the American Institute of Aeronautics andAstronautics, Inc. No copyright is asserted in the United States underTitle 17, U.S. Code. The U.S. Government has a royalty-free licenseto exercise all rights under the copyright claimed herein for Govern-mental purposes. All other rights are reserved by the copyright owner.

CFD computational fluid dynamics

CFL Courant number

Smagorinsky constant

specific heat at constant volume

d normal distance from a solid wall

d+ normal distance from a solid wall in wall

units,

D entire flow domain

hydraulic diameter

DNS direct numerical simulation

elemental surface area on the boundary of a control volume

elemental volume of a control volume

total energy/unit volume

normal component of the inviscid flux vector

flux vector of the Navier-Stokes equations

FDS flux-difference splitting

inviscid flux vector

viscous flux vector

G spatial filter used in the LES equations

I total number of grid points in the streamwise direction

j Jacobian determinant

J total number of grid points in the wall-normal direction

k conduction heat-transfer coefficient

K total number of grid points in the spanwise direction

LES large-eddy simulation

MPI Message-Passing Interface

Cs

cv

d+ ρuτd

µ------------=

DH

sd

vd

Et

f

F

Fi

Fv

nautics and Astronautics

normal unit vector

p static pressure

mean pressure gradient

PVM Parallel Virtual Machine

trace of the SGS Reynolds stress tensor

SGS term in the LES energy equation

R specific gas constant

S total area on the boundary of a control volume

SGS subgrid scale

velocity gradient tensor

time

T temperature

u x-component velocity

mean streamwise velocity

mean Reynolds stress

friction velocity,

state vector of the Navier-Stokes equations

v y-component velocity

V total volume of a control volume

w z-component velocity

x, y, z coordinates of the physical space

Kronecker delta

width of filter used in LES equations

sampling time

scaling factor for Roe flux-difference splitting

molecular viscosity coefficient

coordinates of the computational space

static density

SGS term in the LES momentum equations (the SGS Reynolds stress tensor)

viscous stress tensor

wall shear stress

Subscripts

a node index

L flow conditions to the left of a cell face

rms root mean square

R flow conditions to the right of a cell face

Superscript

n time level

cell-averaged quantities in the Navier-Stokes equations, or filtered or large-scale quantities in the LES equations

Favre-filtered (density-weighted) variables

vector quantity

Introduction

Turbulence dominates the internal flows in aircraft jetengine components such as inlets, ducts, and nozzlesand has been found to significantly influence enginenoise and performance. Analytical tools are thereforeneeded to provide accurate predictions of theseturbulent flows and allow engineers to explore theunderlying flow physics, which would allow betteraeropropulsion flow components to be designed andused in the aerospace industry.

Direct numerical simulation (DNS) of the turbulentflow inside of complete jet engines is presently notpossible because of the tremendous computationalresources required; however, technologies thatpotentially could make such a feat possible in the futureare available today. These technologies include large-eddy simulation (LES) of turbulent flows, unstructuredcomputational fluid dynamics (CFD) algorithms, andparallel computer systems. Large-eddy simulation hasbeen shown to provide accurate turbulent flowsimulation at a fraction of the cost of direct simulation.With unstructured CFD algorithms, complex three-dimensional aerodynamics shapes, including completeaircraft geometrics, have been modeled using a singlegrid. Large-eddy simulation and unstructured CFDalgorithms require large computing resources thatpotentially can be provided by the emerging parallelcomputer systems. By linking together hundreds or eventhousands of individual processor nodes, the parallelcomputer systems can deliver significant advances in

n

Pg

q2

ql

Skl

t

uave

u'v'–

uτ2

-----------

uτ uττw

ρ------=

U

δkl

∆

∆ts

ε1

µ

ξ η ζ, ,

ρ

σkl

τkl

τw

˜

2American Institute of Aeronautics and Astronautics

computational resources in terms of memory, storage,and computing speed.

The above three technologies have been the subjectsof ongoing intensive research, and a large body ofknowledge has been separately accumulated on each ofthese subjects. The objective of this research is todevelop a turbulence simulation tool using acombination of all of these technologies. The accuracyand efficiency of such a tool for turbulence simulationsare then examined in detail from the LES of fullydeveloped turbulent flow in a square duct.

Use of trade names or names of manufacturers in thisdocument does not constitute an official endorsement ofsuch products or manufacturers, either expressed orimplied, by the National Aeronautics and SpaceAdministration.

Numerical Algorithm

Development of the numerical algorithm haspreviously been described in detail.1 This algorithm haspreviously been validated for time-accurate inviscidEuler simulations2 and three-dimensional viscousNavier-Stokes simulations3 with good results. Todescribe the numerical algorithm, the Navier-Stokesequations are used in this section. These equations canbe written in vector form as

(1)

where is the state vector and is the flux vector ofthe Navier-Stokes equations.

The above equation is discretized using the finite-volume approach. In this approach, equation (1) isintegrated over a finite volume. Assuming the grid doesnot change with time and using the Gauss divergencetheorem, the resulting equation is

(2)

where

(3)

and

(4)

To numerically solve equation (2), the major steps ofthe solution procedure are reconstruction, fluxcomputation, and evolution. This standard, finite-volume solution procedure has been used in previousworks and has been described in detail by Barth.4 Thesteps are given below.

Step One: Reconstruction

For the first step, reconstruction, a cell-centeredscheme is used. The piecewise linear, least-squarereconstruction procedure used here is similar to thoseused by Barth4 and Coirier.5 Each of the five primitivevariables , u, v, w, and p is assumed to linearly varywithin a finite volume as:

(5)

where U can be any of the above variables. The bars inequation (5) denote cell-averaged values as defined inequation (3). When used for high-speed compressibleflow simulations, a gradient limiter is normally used inequation (5) to ensure that the reconstructionpolynomial does not produce new extrema near a flowdiscontinuity such as a shock wave. In this paper, thegradient limiter is not used because the test case is low–Mach number turbulent flow in a square duct.

Following Coirier,5 the gradients , , and inthe target cell are computed using a least-squareprocedure that minimizes the sum of the squares of thedifferences between the values computed using thereconstruction polynomial from the target cell and thecell averages of the support set. For a three-dimensionalhexahedron cell, the support set is the six neighboringcells that share their faces with the target cell.Algebraically, the minimization statement above can beexpressed as:

(6)

t∂∂U ∇ F⋅+ 0=

U F

tdd U

1V---- F sd⋅

S∫–=

U1V---- U vd

V∫=

F ds⋅ F nds⋅=

ρ

U x y z, ,( ) U Ux x x–( ) Uy y y–( )+ +=

+ U z z z– ( )

Ux Uy Uz

a1 b1 b2

b1 a2 b3

b2 b3 a3

Ux

Uy

Uz

c1

c2

c3

=


(7)

where

i

= 1–6 denotes the neighboring cells, and

i

= 0denotes the target cell.

Step Two: Flux Computation

With the piecewise linear reconstruction, theunknown variables are continuous and assumed tolinearly vary within a finite volume. However, noguarantee exists that the variables will be continuousacross adjacent finite volumes because a differentpolynomial is used in each finite volume. As a result, aflux formula is needed to compute a single flux at afinite-volume boundary using fluxes from the adjacentvolumes. In the numerical solution of the Navier-Stokesequations, splitting the total flux vector into the inviscidflux vector and viscous flux vector is convenient:

(8)

For the viscous flux, a simple arithmetic average isused. The normal component of the inviscid fluxvector, , is approximated using the Roe flux-difference splitting (FDS) method without the entropycorrection. The entropy correction is normally used toremove the nonphysical expansion shock at the sonic

transition point and is not needed here because of thelow–Mach number test cases.

Define the normal component of the inviscid fluxvector as

(9)

Then can be computed using the Roe FDS methodas

(10)

where the subscripts

L

and

R

denote the flow conditionsto the left and right sides of the cell face.

Figure 1 shows the definitions used for the left andright states. Consider a cell, A, and its neighbor, B,sharing a common face, 1–2. When the flux across face1–2 is computed for cell A, the left state (

L

) of the face1–2 is on the side of cell A, and the right state (

R

) is on

the side of cell B. This definition of the left and rightstates of a face is used because the face normal unitvector , which also serves as the locally one-dimensional coordinate system for the wavepropagation across face 1–2, points from cell A to cellB. The

L

and

R

states are reversed when the flux iscomputed for cell B.

Figure 1. Definition of the left and right states of a face.

Step Three: Evolution

The two-stage, second-order, MacCormack time-marching algorithm is used to advance the solution intime. This explicit predictor-corrector time-marchingmethod is accurate, efficient, and simple to implementon parallel computer systems.

Equation (2) can be rewritten as

(11)

a1 xi x0–( )2

i 1=

6

∑=

a2 yi y0–( )2

i 1=

6

∑=

a3 zi z0–( )2

i 1=

6

∑=

b1 xi x0–( ) yi y0–( )i 1=

6

∑=

b2 xi x0–( ) zi z0–( )i 1=

6

∑=

b3 yi y0–( ) zi z0–( )i 1=

6

∑=

c1 Ui U0–( ) xi x0–( )i 1=

6

∑=

c2 Ui U0–( ) yi y0–( )i 1=

6

∑=

c3 Ui U0–( ) zi z0–( )i 1=

6

∑=

F n⋅ Fi n⋅ Fv n⋅+=

Fi n⋅

f Fi n⋅=

f

f12--- fL fR+( ) 1

2--- A UR UL–( )–=

n

A

1

2

1

2

1

2A

B BRL R

Ln

n

980499

tdd U R=


When applied to equation (11), the MacCormack time-marching method gives

(12)

(13)

In addition to the major solution steps outlined above,describing how the volume and surface integrals inequations (2) and (3) are evaluated is important.Although the flow variables are approximated bydiscontinuous piecewise linear polynomials, the spatialcoordinates x, y, and z of a finite volume areapproximated by a continuous trilinear hexahedralelement.6 This approach is the same that finite-elementmethods use to approximate the spatial coordinates. Allof the integrations are then numerically evaluated usingthe one-point Gauss quadrature formula.

Each cell in the physical space is mappedto a trilinear hexahedral element in the spaceas shown in figure 2. The nodes, indexed 1 to 8, have thenodal coordinates in the space shown in table 1.

Figure 2. Local mapping between the physical finite-volume and the trilinear hexahedral element.

This type of mapping is different from the mappingused in generalized curvilinear finite-differencemethods. In the finite-difference methods, the mappingapplies to the entire computational block. In thisalgorithm, the local mapping applies to the local cellonly. Each cell has its own mapping function, which issimilar to a finite-difference multiblock method inwhich each cell is its own block. This ability givesconsiderable flexibility in the grid topology that can beused and is one of the strengths of this unstructuredfinite-volume algorithm.

The mapping from the space to the space isgiven by:

(14)

(15)

where the subscript a denotes the node index, rangingfrom 1 to 8. In the physical (x, y, z) coordinate system,node a has the coordinate . In thecomputational space, node a has thecoordinate . The coordinates vary from cell to cell, depending on the physical grid.The coordinates are the same for every celland are shown in table 1.

To evaluate the volume integral, the followingrelation6 is used:

(16)

where j is the Jacobian determinant, defined as

(17)

Table 1. Nodal coordinates in the space.

Node index (a)

1 –1 –1 –1

2 1 –1 –1

3 1 1 –1

4 –1 1 –1

5 –1 –1 1

6 1 –1 1

7 1 1 1

8 –1 1 1

Un 1+

Un

t∆ Rn

+=

Un 1+ 1

2--- U

n 1+U

nt∆ R

n 1++ +

=

x x y z, ,( )ξ ξ η ζ, ,( )

ξ

5 5

88

1

177

4 4

η

ξ

ζ

33

2

2

6 6

yx

z

980500

ξ = g (x, y, z)

x = f (ξ, η, ζ)

ξ

ξa ηa ζa

ξ x

x ξ( ) Na ξ( )xaa 1=

8

∑=

y ξ( ) Na ξ( )yaa 1=

8

∑=

z ξ( ) Na ξ( )zaa 1=

8

∑=

Na ξ( )18--- 1 ξaξ+( ) 1 ηaη+( ) 1 ζaζ+( )=

xa, ya, za( )ξ η ζ, ,( )

ξa ηa ζa, ,( ) xa, ya, za( )

ξa ηa ζa, ,( )

f x y z, ,( ) vdV∫

= f x ξ η ζ, , ( ) y ξ η ζ, , ( ) z ξ η ζ, , ( ) , , ( )

1–

1

∫

1–

1

∫

1–

1

∫

j

ξ η ζ, ,

( )

ξ

d

η ζ

dd

j det∂x

∂ζ------

det

xξ xη xζ

yξ yη yζ

zξ zη zζ

= =


The partial derivatives , , , and so forth can beobtained by differentiating equation (14). Evaluating thedeterminant in equation (17) results in the following:

(18)

To evaluate the integral in equation (16), the one-point Gauss quadrature formula is used. In onedimension, the Gauss quadrature formulas are optimal,which means that accuracy of order (2

n

) is achievedusing (

n

) integration points. Gaussian rules for integralsin several dimensions are constructed by employing theone-dimensional Gaussian rules on each coordinateseparately. In three dimensions, the one-point Gaussianrule is given as

(19)

Using the tools described above, the volume of the cellis computed as

(20)

(21)

Using equation (19),

(22)

so that the cell volume is approximately eight times theJacobian determinant evaluated at the center of the cell

. Note that equation (22) containstwo approximations: the physical coordinates (x, y, z) inthe cell are approximated by equation (14), and the one-point Gaussian rule given by equation (19) is used forthe numerical integration. Better approximation of thecell volume can be obtained using a higher-orderapproximation for the physical coordinates and aGaussian rule with more points.

Computing the centroid of the cell also requires theevaluation of the volume integral. The coordinates of acell centroid are given by . The numericalapproximation for the x coordinate of the centroid is

developed below. Approximations for the y and zcoordinates are made exactly the same way.

(23)

(24)

(25)

or

(26)

From equation (14),

(27)

With the approximations in this algorithm, thecoordinates of the cell centroid are simply the averagesof the coordinates of the eight nodes defining the three-dimensional hexahedron cell.

The task of evaluating the surface integrals isdescribed next. The surface integrals on the right side of

equation (2) are of the form . To develop the

procedure, one face of the hexahedron shown in figure 2is considered. The results for the other faces can beobtained using the same procedure.

Consider face 6–2–3–7 of the hexahedron in figure 2.Figure 3 shows this face redrawn for convenience.Because for the nodes 6, 2, 3, 7 as well as anyother point on this face, equation (14) reduces to thefollowing:

xξ yη zζ

j xζyξzη xξyζzη– xζyηzξ– xηyζzξ+=

+ x ξ y η z ζ x η y ξ z ζ –

f ξ η ζ, ,( ) ξd ηd ζd

1–

1

∫1–

1

∫1–

1

∫

= 8 f ξ 0= η 0= ζ 0= , , ( )

V 1 vdV∫=

V j ξ η ζ, ,( ) ξd ηd ζd

1–

1

∫1–

1

∫1–

1

∫=

V 8 j ξ 0= η 0= ζ 0=, ,( )=

ξ 0= η 0= ζ 0=, ,

x y z, ,( )

x

x vdv∫

V-----------=

x

x ξ η ζ, ,( ) j ξ η ζ, ,( ) ξd ηd ζd

1–

1

∫1–

1

∫1–

1

∫

j ξ η ζ, ,( ) ξd ηd ζd

1–

1

∫1–

1

∫1–

1

∫--------------------------------------------------------------------------------=

x 8x ξ 0= η 0= ζ 0=, ,( ) j ξ 0= η 0= ζ 0=, ,( )8 j ξ 0= η 0= ζ 0=, ,( )

--------------------------------------------------------------------------------------------------------------=

x x

ξ

0=

η

0=

ζ

0=

, ,

( )=

y y

ξ

0=

η

0=

ζ 0=, ,( )=

z z ξ 0= η 0= ζ 0=, ,( )=

x ξ 0= η 0= ζ 0=, ,( )18--- xa

a 1=

8

∑=

y ξ 0= η 0= ζ 0=, ,( )18--- ya

a 1=

8

∑=

z ξ 0= η 0= ζ 0=, ,( )18--- za

a 1=

8

∑=

F sd⋅S∫

ξ 1.0=


(28)

so that x, y, and z are functions of and only.

Figure 3. Coordinate systems for surface integralevaluation.

Following the development outlined in Greenberg,7

the tangent vectors and are tangents on theplane 6–2–3–7. is defined to be along the

= constant curve on the face, and is along the = constant curve. The vector may be expressed as

follows:

(29)

where because the entire plane 6–2–3–7 is an-constant plane, and because the vector

is defined to be along the = constant curve. So

(30)

and similarly

(31)

The elemental area vector , denoted by the shadedparallelogram, can be computed as

(32)

or

(33)

Note that the order of the cross product in equation (32)is chosen so that the elemental area vector is positivepointing out of the cell and negative pointing into thecell. With the vector defined as

(34)

the dot product is

(35)

Finally, using the one-point Gaussian rule, the surfaceintegral can be evaluated as

(36)

The surface integrals for the other faces can beapproximated in an analogous fashion. The results aregiven below.

(37)

(38)

(39)

x14--- 1 ηaη+( ) 1 ζaζ+( )xa

a 6 2 3 7, , ,=∑=

y14--- 1 ηaη+( ) 1 ζaζ+( )ya

a 6 2 3 7, , ,=∑=

z14--- 1 ηaη+( ) 1 ζaζ+( )za

a 6 2 3 7, , ,=∑=

η ζ

6 6

n77

33

η

ξ

ζ

22

980501

ξ = g (x, y, z)

x = f (ξ, η, ζ)

ds2

ds1

ds1 ds2ds1

η ds2ζ ds1

ds1 dxi dy j dzk+ +=

dx xξdξ xηdη xζdζ+ +=

dy yξdξ yηdη yζdζ+ +=

dz zξdξ zηdη zζdζ+ +=

dξ 0=ξ dη 0= ds1

η

ds1 xζ i yζ j zζk+ +( )dζ=

ds2 xη i yη j zηk+ +( )dη=

ds

ds ds2 ds1×=

ds yηzζ zηyζ–( )i zηxζ xηzζ–( ) j+=

+ x η y ζ y η x ζ – ( ) k] d η d ζ

ds

F

F Fxi Fy j Fzk+ +=

F ds⋅ yηzζ zηyζ–( )Fx zηxζ xηzζ–( )Fy+[=

+ x η y ζ y η x ζ – ( ) F z ] d η d ζ

F sd⋅6237( )∫ 4[ yηzζ zηyζ–( )Fx=

+ z η x ζ x η z ζ – ( ) F y

+ x η y ζ y η x ζ – ( ) F z ] ξ

1=

η

0=

ζ

0=

, ,

F sd⋅1584( )∫ 4[ zηyζ yηzζ–( )Fx=

+ x η z ζ z η x ζ – ( ) F y

+ y η x ζ x η y ζ – ( ) F z ] ξ

1–=

η

0=

ζ

0=

, ,

F sd⋅8734( )∫ 4[ zξyζ yξzζ–( )Fx=

+ x ξ z ζ z ξ x ζ – ( ) F y

+ y ξ x ζ x ξ y ζ – ( ) F z ] ξ

0=

η

1=

ζ

0=

, ,

F sd⋅1265( )∫ 4[ yξzζ zξyζ–( )Fx=

+ z ξ x ζ x ξ z ζ – ( ) F y

+ x ξ y ζ y ξ x ζ – ( ) F z ] ξ

0=

η

1–=

ζ

0=

, ,


(40)

(41)

Symbolically, the surface integrals of the oppositefaces are the negative of each other (for example, face

as given by equation (36) and face asgiven by equation (37)). However, for actual numericalvalues, each of the integrals will need to be separatelyevaluated because the integrands depend on thecoordinates of the nodal points and theGaussian point coordinates.

To calculate the Jacobian determinant of thecoordinate transformation, equations (14) and (15) areused. For example, the derivative can be evaluated asfollows:

(42)

Governing Equations for Large-Eddy Simulation

In LES, the large scale of turbulence is computeddirectly in the numerical simulation, and the effects ofthe small scale stresses are modeled using a subgrid-scale (SGS) model. The governing equations for LES ofturbulent flows can be obtained from filtering (localvolume–averaging) the compressible Navier-Stokesequations. From Moin et al.,

8

the LES equations forcompressible flows (using tensor notation) are given bythe following:

(43)

(44)

(45)

The bar in the LES equations (43) to (45) denotes afiltered or large-scale flow quantity, defined as

where G is a spatial filter and the integral is over theflow domain,

D

. The tilde in the LES equations denotesa Favre-filtered (density-weighted) variable, defined as

The filtered ideal gas equation of state is given by

Moin et al.

8

used an internal energy equation in theirderivation of the LES equations. The LES total energyequation, equation (45), is obtained from adding the dotproduct of the LES momentum equation, equation (44),and the filtered velocity field to the LES internalenergy equation in Moin et al.

8

The LES equations given by equations (43) to (45) areessentially the Navier-Stokes equations written for thefiltered variables plus the additional subgrid terms in themomentum and total energy equations. Thus, thenumerical algorithm developed in the last section can beused to solve the LES equations. The treatment of thesubgrid terms are discussed in the next section.

Subgrid-Scale Models

Detailed studies have previously been performed toassess the relative importance of the subgrid terms in thefiltered total energy equation for compressible turbulentshear flows at different Mach numbers.

9, 10

Thesestudies led to the conclusion that the energy subgridterms may be neglected if the Mach number of thesimulation is low. Because of the low Mach number ofthe turbulent square duct test case, this assumption isused.

F sd⋅5678( )∫ 4[ yξzη zξyη–( )Fx=

+ z ξ x η x ξ z η – ( ) F y

+ x ξ y η y ξ x η – ( ) F z ] ξ

0=

η

0=

ζ

1=

, ,

F sd⋅2143( )∫ 4[ zξyη yξzη–( )Fx=

+ x ξ z η z ξ x η – ( ) F y

+ y ξ x η x ξ y η – ( ) F z ] ξ

0=

η

0=

ζ

1–=

, ,

ξ 1= ξ 1–=

xa, ya, za( )

xξ

x ξ( ) Na ξ( )xaa 1=

8

∑=

xξ Na ξ, xaa 1=

8

∑=

t∂∂ ρ

xk∂∂ ρuk+ 0=

t∂∂ ρuk xl∂

∂ ρukul xk∂∂

pxl∂

∂τkl

xl∂∂σkl+–+ + 0=

t∂∂Et

xl∂∂

Etul xl∂∂

pul xl∂∂

uk τkl–+ +

– ∂∂ x

l ------- k ˜ ∂

∂

x

l ------- T ˜

u ˜ k ∂σ

kl

∂

x

l ----------- c v

x

l

∂∂

q

l ++ 0=

f G x x ′–( ) f x ′( ) x ′dD∫=

fρfρ------=

p ρRT=

uk


The subgrid term in the momentum equations,equation (44), is

To close the system of LES equations, this term needs tobe modeled. In Moin et al.,

8

this term is approximatedas follows:

(46)

where

is the trace of the SGS Reynolds stress tensor. Thefiltered velocity gradient tensor is

and

In equation (46), C is a constant to be determinedaccording to the particular SGS model used. For LES ofturbulent channel and duct flows using the SmagorinskySGS model,

11

a value of C = 0.01 is commonly usedwith good results. Note that the constant C inequation (39) is the square of the Smagorinsky constant

= 0.1.

Unlike LES of isotropic turbulence, C is not constantin wall-bounded flows and varies according to distancefrom the wall. The dynamic SGS model developed byGermano et al.

12

would correctly determine the valuefor C using a dynamic procedure; however, this model iscomputationally expensive because of the extra filteringoperations that must be done. Also, a question currentlyexists on the mathematical well-posedness of thismodel.

13

Finally, the dynamic model has been known tocompensate for the effects of numerical dissipation byautomatically varying the magnitude of the constant C.

One of the main objectives of this research is toquantify the effects of the upwind numerical dissipationon the accuracy of the turbulence simulations. Also, thesimpler Smagorinsky model has been found to work aswell as the dynamic SGS model for this simple testcase.

14

As the result, the Smagorinsky SGS model isused in this study with the constant C given by thefollowing:

(47)

where is the normal distance from the wall in wallunits, defined as

and the friction velocity is defined as

Because the turbulent flow in the corner of the squareduct encounters walls in two different directions,

d

istaken to be

(48)

Equation (48) is frequently used in the turbulencemodeling of flows in the vicinity of a wall corner. Thevariables y and z are the normal distances to the nearestwalls in the y and z directions. Note that

d

tends to y as(y/z) tends to 0, and

d

tends to z as (z/y) tends to 0,which are the intended results.

In LES, the width of the filter used in the process ofvolume-averaging the Navier-Stokes equations, , istypically chosen to be the grid spacing size. This studydefined the grid spacing size to be , where

V

is thecell volume.

The term q2 in equation (46) is the isotropic part ofthe SGS Reynolds stress tensor. Like the rest of thistensor, the term cannot be calculated directly in anLES and has to be modeled. A number of differentmodels for q2 has been proposed.15–17 However,results from recent studies indicated that this term isnot important for accurate LES of low–Mach number,low–Reynolds number compressible turbulent flows.

An evidence in support of the above conclusion waspresented by Squires,18 who compared two differentmodels of q2 in addition to setting q2 = 0. Squires foundessentially no difference in the results of LES ofcompressible isotropic turbulence at a low Machnumber and, in fact, observed that neglecting q2 slightlyimproved the agreement between the LES and DNSresults.

σkl ρ ukul˜ ukul–( )=

σkl 2Cρ∆2S Skl

13--- Smmδkl–

–13---q

2δkl+=

q2 σii=

Skl12---

xl∂∂uk

xk∂∂ul+

=

S 2SklSkl( )

12---

=

Cs

C 0.01 1.0d

+

25------

3

– exp–

=

d+

d+ ρuτd

µ------------=

uττw

ρ------=

d2yz

y z y2

z2

+( )

12---

+ +

-------------------------------------------=

∆

V3


Vreman et al.9 confirmed the above results withtheir simulations of a three-dimensional temporalcompressible mixing layer at a mean convective Machnumber of 0.2. In a priori tests, the SGS model thatneglects was found to give a better correlation withthe DNS results. Furthermore, LESes conducted withthe dynamic SGS model for were unstable for thecases that were studied.

For low–Mach number turbulence LES, neglectingthe term will not introduce large errors in the resultsand is actually desirable in some cases, as the abovefindings showed. As a result, the term is neglectedin this study. This assumption is analogous to theStokes assumption for the viscous stress tensor in theNavier-Stokes equations. With the term omitted, theSGS stresses can be included in the Navier-Stokesequations by simply replacing the laminar viscositycoefficient with where and

.

Large-Eddy Simulation of Turbulent Flow in a Square Duct

To validate the numerical method for turbulencesimulations of duct flows, LES of fully developedturbulent flow in a square duct is performed. For thepurpose of comparison, a low–Reynolds number, squareduct DNS solution is available. This DNS database wasused by Mompean et al.19 to evaluate nonlinear k-εturbulence models. Another DNS solution of the fullydeveloped turbulent square duct flow at a slightly lowerReynolds number is also available.20

Figure 4 shows the coordinate system and geometryfor the square duct flow. In this test case, the Reynoldsnumber based on the mean streamwise velocity andhydraulic diameter is 4800. Based on the frictionvelocity and hydraulic diameter, the Reynolds numberis 320.

Table 2 shows a summary of the flow properties forthe test case, assuming an average Mach number of 0.3and standard sea level properties for air. Thecomputational domain size used in the LES is12 × × . In choosing the size of thecomputational domain, care must be taken to ensure thatthe length of the computation domain is large enough toadequately contain the largest turbulence structure.Two-point velocity correlations for three differentcross-stream positions in the duct were computed fromthe DNS solution by Gavrilakis.20 The correlations forall three velocity components become essentially 0 at a

duct length of approximately 6 , so that a length of12 should be adequate to capture the streamwiseturbulence structures. Two different grids are used in thepresent LES, and table 3 shows the simulationparameters. The sampling time for the turbulentstatistics is large compared to the time step size, but issmall compared to the eddy turnover time in order tocapture the unsteadiness of the turbulent flow.

q2

q2

q2

q2

q2

µ µeff µeff µ µSGS+=µSGS Cρ∆2

S=

DH DH DH

Table 2. Flow properties for the turbulent square duct test case.

Flow properties Values

Average Mach number, 0.3

Average streamwise velocity, 102.4 m/sec

Average friction velocity, 6.83 m/sec

4800

320

Hydraulic diameter, 7.37 × 10–4 m

Mean pressure gradient, –289,646 Pa/m

Eddy turnover time, 5.4 × 10–5 sec

DHDH

Wall normal bisector

Cornerbisector

x

zDH

y

980502

12 DH

Figure 4. Coordinate system and geometry for thesquare duct flow.

uave

c----------

uave

uτ

ReρuaveDH

µ----------------------=

ReτρuτDH

µ-----------------=

DH

Pg

0.5DH

uτ----------------


The periodic boundary condition used for the inflowand exit boundary of the square duct is similar to the oneused by Coleman et al.21 With this boundary condition,all of the flow conditions are periodic at the duct inletand exit planes. The driving pressure gradient in theduct is specified in the flow equations as an extra bodyforce term.

To reduce the number of iterations required forconvergence, the initial conditions for the large eddysimulations were obtained from interpolating a DNSsolution provided by Gavrilakis. The DNS was doneusing a 768 × 127 × 127 grid. The current simulationswere done using two different grid sizes, 129 × 90 × 90(grid A) and 257 × 100 × 100 (grid B). The finer LESgrid B, which gave good results, is approximately20 percent of the total size of the DNS grid.

The convergence of the LES is determined bymonitoring the time history of the total wall shear stress.For fully developed turbulent flow in a straight squareduct, conservation of the mean streamwise momentumshows that the mean driving pressure gradient and thetotal wall shear stress are related by the following:

(49)

The surface integral is over the four side walls of thesquare duct, so that = 4 × 12 × 2. is the

mean driving pressure gradient, and V is the totalvolume of the duct, given by 12 × 3. Defining

and , the familiar relation

between the mean pressure gradient and wall shearstress in fully developed flow in a square duct can berecovered.

(50)

Figure 5 shows the time history of the total side wallshear force for the LES using grid B. The instantaneousside wall shear force level from the LES, shown as asolid line, is seen to fluctuate about a mean value,indicating that flow equilibrium has been reached in thecurrent simulation. The time average of the computedside wall shear force is 0.001387 N. This value is inexcellent agreement with the exact value of 0.001391 N,computed from equation (50) and shown as thehorizontal dashed line (fig. 5). Because the time step isconstant, the number of iterations shown in figure 5 isdirectly proportional to the time elapsed. The simulationwas conducted for 218,600 time iterations, which isapproximately 10 eddy turnover times (as defined intable 2).

Figure 5. Convergence history for the total wall shearstress.

The parallel implementation and the results of theparallel performance studies have previously beenpublished.1, 3 The code was implemented on parallelcomputer systems using the message-passingprogramming model and message-passing libraries suchas Message-Passing Interface (MPI) and Parallel Virtual

Table 3. Parameters for the LES

Parameters Grid A Grid B

Grid size (I × J × K) 129 × 90 × 90 257 × 100 × 100

Number of cells 1,013,888 2,509,056

Minimumresolution

(in wall units)30 × 1.88 × 1.88 15 × 1.69 × 1.69

Maximumresolution

(in wall units)30 × 4.86 × 4.86 15 × 4.37 × 4.37

Sampling time, (sec) 6.0 × 10–7 5.0 × 10–7

Time step size, (sec) 3.0 × 10–9 2.5 × 10–9

CFL number 0.98 0.93

∆ts

∆t

τw AdAs

∫ V Pg–=

As DH Pg

DH

τw1As------ τw Ad

As

∫= Pg xddp=

τw

DH

4--------

xd

dp–=

.00150

.00145

.00140

.00135

.00130500 100

Number of iterations (in thousands)150

LES

200

980503

∫ τwdA,

NAs

Equation 50


Machine (PVM). The parallel speedup was found to bevery good, especially for large numbers of grid points.3

Using 128 processors on the T3D computer (CrayResearch, Inc.; Eagan, Minnesota), the simulation ofthese 10 eddy turnover times (5.5 × 10–4 sec in physicalflow time) took 772 hr or approximately 1 month ofcentral processing unit time. The same simulationwould have taken approximately 150 hr on an SP2 (IBMCorporation, Austin, Texas) or T3E (Cray Research,Inc.) computer. Regardless of the computer platformused, this computational time is a large cost and showsthat even with the parallel computer systems availabletoday, turbulence simulation is still a formidable task.However, parallel CFD algorithms that can efficientlyscale up with extremely large numbers of processorsoffer the only real hope that turbulence simulations canbe done in a reasonable amount of time in the future.

The LES results shown in figures 5 to 14 are obtainedusing grid B and a modified Roe FDS with an valueof 0.03. The modification to the Roe FDS and thedefinition of the parameter will be discussed below.

Figure 6 shows the mean streamwise velocity profilealong a wall bisector. The LES solution (solid line) iscompared with the DNS solution (diamond dots)supplied by Gavrilakis. The mean velocity profile in theLES was averaged both in time and space. Theagreement can be seen to be very good.

Figure 6. Mean streamwise velocity profile for fullydeveloped turbulent flow in a square duct along the wallbisector.

Figure 7 shows the mean secondary velocity vectorsfrom the LES. In straight ducts of noncircular cross-sections, turbulence-driven secondary flows are knownto exist. These flows are different from the pressure-driven secondary flows found in curved ducts. In

straight square ducts, the turbulence-driven secondaryflows are directed from the center of the duct toward thecorners along the corner bisectors, and have been foundto be produced by the anisotropy of the Reynoldsstresses in the cross-sectional plane of the square duct.22

Although the magnitudes of these secondary velocitiesare extremely small compared to the mean averagestreamwise velocity (on the order of 2 percent in thissimulation), these velocities have been found to beimportant features of this flow.

Figure 7 shows that the corner vortices produced bythe secondary flows are captured in this simulation.Although some asymmetry is still evident in the plot,the overall features of the secondary flows are well-predicted by the current simulation.

To determine the accuracy of the simulation incapturing turbulence-driven secondary flows, the meansecondary velocity profiles along the linesz/(0.5 ) = 0.15, 0.30, 0.50, 0.70, and 0.80 arecompared with the DNS results in figures 8 to 12.Generally, good agreement is obtained between the LESand DNS mean secondary velocity profiles.

Figures 13 and 14 show the turbulence statistics. Infigure 13, the mean Reynolds stress profile along a wallbisector is compared with the DNS solution, and infigure 14, the mean turbulence intensities , ,and are plotted. These results have been quadrant-averaged as well as averaged in space and time.

ε1

ε1

1.0

.8

.6

.4

.2

0 .5 1.0 1.5

980504

y/0.5DH

u/uave

LESDNS

1.0

.5

0 .5 1.0

980505

y/DH

z/DH

Figure 7. Mean secondary velocity vectors from LESwith a 257 × 100 × 100 grid.

DH

urms vrmswrms


Figure 8. Mean secondary velocity profiles alongz/(0.5 ) = 0.15.

Although the agreements between the LES and DNSsolutions are seen to be very good for this simulation,the accuracy of the LES in capturing the turbulencevelocity fluctuations was found during the turbulencesimulations to be highly dependent on the numericaldissipation and the grid size used. The effects of the RoeFDS upwinding term and grid size on the computedturbulence velocity fluctuations are examined next.

Effect of the Roe Flux-Difference Splitting Term

Although the Roe FDS implemented in this code gavegood results for Euler2 and laminar Navier-Stokes3 testcases, the full Roe FDS term was found to be toodissipative for LES. Incorrect levels of turbulentvelocity fluctuations are obtained when the normal RoeFDS term is used in turbulence simulations. Thisproblem was solved with a simple modification to theRoe FDS algorithm. In equation (10), the inviscid fluxesnormal to a cell boundary is approximated as

Figure 9. Mean secondary velocity profiles alongz/(0.5 ) = 0.3).

This approximation can be interpreted to state that thenormal component of the inviscid flux at a cell boundaryis the sum of the central difference of the fluxes on the

left and right states, , plus the Roe

upwinding dissipation term, . If this

interpretation is used, then the amount of Roeupwinding dissipation can be controlled using amultiplying factor in front of the Roe FDS term, suchthat

(51)

where can range between 0 and 1. = 0 correspondsto central differencing only, and = 1 corresponds tothe full Roe FDS.

1.0

.8

.6

.4

.2

00 .3.1– .1– .2 .2 .4

y/0.5DH

v/uτ

LESDNS

1.0

.8

.6

.4

.2

0– .1– .2 0 .1 .2 .3

980506

y/0.5DH

w/uτ

DH

1.0

.8

.6

.4

.2

0– .2 – .1 0 .1 .2

y/0.5DH

v/uτ

LESDNS

0

1.0

.8

.6

.4

.2

0– .05– .10 .05 .10

980507

y/0.5DH

w/uτ

DH

f12--- fL fR+( ) 1

2--- A UR UL–( )–=

12--- fL f+

R( )

12--- A UR UL–( )

–

f12--- fL fR+( ) ε1

12--- A UR UL–( )

–=

ε1 ε1ε1





1.0

.8

.6

.4

.2

0– .15– .20 – .10 – .05 0 .05

y/0.5DH

v/uτ

LESDNS

1.0

.8

.6

.4

.2

0– .1– .2 0 .1 .2 .3

980508

y/0.5DH

w/uτ

DH

1.0

.8

.6

.4

.2

0– .2 0– .3 – .1 .1 .3.2

y/0.5DH

v/uτ

LESDNS

1.0

.8

.6

.4

.2

00– .05 .05 .10 .15

980509

y/0.5DH

w/uτ

DH

Figure 12. Mean secondary velocity profiles alongz/(0.5 ) = 0.8).

Figure 13. Mean Reynolds stress profile along the wallnormal bisector.

Figure 14. Turbulent intensities along the wall normalbisector.

Note that Lin et al., using the same interpretation ofthe Roe upwinding term as equation (51), alsoconcluded that the normal Roe upwinding termproduces too much numerical dissipation forcomputational aeroacoustics applications.23 Lin et al.found that using values of approximately 0.1 gavegood results for acoustics computations.23

In the LES conducted here, values of less than 0.1are needed to give good turbulence results. Omitting theRoe FDS term altogether ( = 0) causes all calculationsto be unstable, and the best turbulence solutions are

1.0

.8

.6

.4

.2

0– .2 – .1 0 .1 .2

y/0.5DH

v/uτ

LESDNS

1.0

.8

.6

.4

.2

0– .2 – .1 .1 .30 .2 .4

980510

y/0.5DH

w/uτ

DH

1.0

.8

.6

.4

.2

0 .2 .4 .6 .8 1.0

980511y/0.5DH

– u,v

,/u2

τ

LESDNS

4

3

2

1

0

1.0

.8

.6

.4

.2

0

1.2

1.0

.8

.6

.4

.2

0

urms/uτ

LESDNS

vrms/uτ

wrms/uτ

.2 .4 .6 .8 1.0

980512y/0.5DH

ε1

ε1

ε1


obtained using the smallest possible values of thatcan still provide stable calculations. In general, the finerthe grids, the smaller the minimum values of that canbe used. For a grid of 129 × 90 × 90, the minimum valueof for stable calculation is 0.05, and for the257 × 100 × 100 grid, the minimum value is 0.03. TheLES results presented in the preceding section wereobtained using = 0.03.

To study the effect of the Roe upwinding term on theturbulence simulation, LES are made for the same gridsize of 129 × 90 × 90 but with different values of .Figure 15 shows the effect of Roe FDS on the meanstreamwise velocity profile. Near the wall, using the fullRoe FDS term produces a mean velocity gradient that ismuch less than both the DNS solution and the LESsolution with the reduced Roe FDS. A similar effect isobserved in figure 16, where the mean Reynolds stressprofile obtained with = 1.0 is much lower thanexpected.

Figure 17 also shows the excessive numericaldissipation of Roe FDS in the turbulence solution. Inthis figure, the solution with = 1.0 gives asignificantly higher level of and lower levels of

and . Although they did not use the Roe FDS,Wang and Pletcher24 reported the same problem in theirLES of fully developed turbulent channel flow using anupwind CFD algorithm.

From studying the results shown in figures 15 to 17,the full Roe FDS upwinding dissipation can be seen tobe detrimental to the turbulence solution, and an

improvement in the solution quality can be obtained byreducing the contribution of the Roe upwinding term.However, continuing to reduce the contribution of theRoe term until a good agreement is achieved is notpossible. For a given grid size, a minimum amount ofRoe FDS upwinding dissipation is required for stability.For the 129 × 90 × 90 grid, the minimum value fornumerical stability is 0.05, and values smaller than thisminimum will cause the calculation to be unstable. Toimprove the accuracy of the turbulence simulation,using a finer grid that in turn allows a smaller valueto be used is necessary. In the next section, the effect ofa finer grid on the quality of the turbulence solution willbe studied.

Effect of the Grid Size

The previous section showed that reducing thecontribution of the Roe FDS term will improve thequality of the solution. But using a 129 × 90 × 90 grid,reducing to the minimum value of 0.05 still does notgive a good agreement with the DNS solution. In thissection, a finer grid with 257 × 100 × 100 points is used,and the minimum value of that can be used islowered to 0.03.

Figure 18 shows a comparison of the meanstreamwise velocity profiles. Using the finer grid in theLES produces an almost perfect agreement with theDNS solution. Figures 19 and 20 show the sameimprovement in the profiles of the turbulence intensitiesand the mean Reynolds stress, respectively.

ε1

ε1

ε1

ε1

ε1

ε1

1.0

.8

.6

.4

.2

0 .5 1.0 1.5

980513

y/0.5DH

u/uave

LES with ε1 = 1.00LES with ε1 = 0.05DNS

Figure 15. Effect of Roe FDS on the mean streamwisevelocity profile.

ε1urms

vrms wrms

ε1

ε1

– u,v,/u2

τ

1.0

.8

.6

.4

.2

0 .2 .4 .6 .8 1.0

980514y/0.5DH


Figure 16. Effect of Roe FDS on the mean Reynoldsstress profile.

ε1

ε1


Another effect of grid size can be observed byexamining the mean secondary velocity vectors in across section of the square duct. Figure 21 shows thesolution obtained using grid A and = 0.05. Thesecondary turbulence-induced velocity field is capturedby the coarser grid. However, comparing this result withthe finer grid B result in figure 7 shows that the cornervortices in figure 7 are somewhat smaller than those infigure 21. Figure 22 shows the streamwise-averaged

instantaneous secondary velocity vectors from the DNSsolution for reference. For the DNS grid, the near wallvortical structures are even smaller than either of theLES grids. This comparison shows that the near wallturbulent structures are better resolved with finer grids.

Effect of the Subgrid-Scale Model

The previous section showed that the LES solutionwith the fine grid gives the best agreement with the DNSsolution. Although the LES fine grid is only 20 percentof the size of the DNS grid, the LES grid density in thecrossflow plane is 60 percent of the DNS crossflowplane density. As the LES grid resolution in thecrossflow plane approaches the DNS resolution, theeffect of the SGS model on the turbulence solution forthis particular LES grid is interesting to see. Anadditional simulation was performed using grid B, thefiner LES grid, with no SGS model. This simulation iseffectively a coarse grid DNS. Figure 23 shows acomparison of the mean streamwise velocity profiles.The use of the SGS model makes essentially nodifference in the mean streamwise velocity solution.Small differences are also observed in the turbulentvelocity fluctuations shown in figure 24. Figure 25shows a comparison of the mean Reynolds stressprofiles. The biggest difference is at the peak of theReynolds stress profile, where the LES solution with noSGS model predicts a slightly higher peak than theDNS, and the LES solution with the SGS model predicts

4

2

0

1

3

urms/uτ

1.0

.8

.6

.4

.2

0

vrms/uτ

1.2

1.0

.8

.6

.4

.2

0

wrms/uτ

.2 .4 .6 .8 1.0

980515y/0.5DH


Figure 17. Effect of Roe FDS on the turbulentintensities.

ε1

1.0

.8

.6

.4

.2

0 .5 1.0 1.5

980516

y/0.5DH

u/uave

LES with 129 x 90 x 90 gridLES with 257 x 100 x 100 gridDNS

Figure 18. Effect of grid size on the mean streamwisevelocity profile.



Figure 19. Effect of grid size on the turbulent intensities.

Figure 20. Effect of grid size on the mean Reynoldsstress profile.

Figure 21. Mean secondary velocity vectors from LESwith a 129 × 90 × 90 grid.

4

2

0

1

3

urms/uτ

1.0

.8

.6

.4

.2

0

vrms/uτ

1.2

1.0

.8

.6

.4

.2

0

wrms/uτ

.2 .4 .6 .8 1.0

980517y/0.5DH


– u,v

,/u2

τ

1.0

.8

.6

.4

.2

0 .2 .4 .6 .8 1.0

980518y/0.5DH


1.0

.5

0 .5 1.0

980519

y/DH

z/DH

a slightly lower peak than the DNS. Note that the LESturbulence statistics were only computed for theresolved velocity field. As a result, the LES Reynoldsstress profile with the SGS model should be lower thanthe DNS solution because the SGS contribution wasnot included.

Figure 24. Effect of the SGS model on the turbulentintensities.

These results indicate that an SGS model is notneeded for an accurate simulation of this test case. Asdiscussed earlier, the grid resolution in the near wallregion has to be very fine to resolve the small energy-producing structures there. The fine grid LES conductedhere has effectively approached the DNS in the nearwall limit.

1.0

.5

0 .5 1.0

980520

y/DH

z/DH

Figure 22. Streamwise-averaged secondary velocityvectors from the DNS solution.

Figure 23. Effect of the SGS model on the meanstreamwise velocity profile.

1.0

.8

.6

.4

.2

0 .5 1.0 1.5

980521

y/0.5DH

u/uave

No SGS modelWith SGS modelDNS

4

2

0

1

3

urms/uτ

1.0

.8

.6

.4

.2

0

vrms/uτ

1.2

1.0

.8

.6

.4

.2

0

wrms/uτ

.2 .4 .6 .8 1.0

980522y/0.5DH



Figure 25. Effect of the SGS model on the meanReynolds stress profile.

Conclusion

A new, parallel, finite-volume computational fluiddynamics algorithm was developed for large-eddysimulation (LES) of turbulent flows using parallelcomputer systems. Major components of the algorithmincluded piecewise linear least-square reconstruction ofthe unknown variables, trilinear finite-elementinterpolation for the spatial coordinates, Roe flux-difference splitting (FDS), and second-orderMacCormack explicit time marching. The parallelimplementation was accomplished using the message-passing programming model.

For the first time, a parallel, unstructured, finite-volume numerical algorithm was used for LES ofturbulent flow in a square duct, and several conclusionshave been drawn regarding the accuracy and efficiencyof this numerical algorithm. Comparison with the directnumerical simulation (DNS) solution showed that thestandard Roe FDS upwind dissipation adversely affectsthe accuracy of the turbulence simulations. Amodification to the standard Roe FDS method wasproposed in which the inviscid flux is computed as thearithmetic average of the right and left fluxes plusthe product of the Roe FDS dissipation term and areduction factor. For accurate turbulence simulations,only 3–5 percent of the normal Roe FDS dissipationwas found to be needed.

The finer, 257 × 100 × 100 LES grid required less RoeFDS upwind dissipation for stability and produced amore accurate solution than the 129 × 90 × 90 LES grid.The near wall vortical structures were better simulated

by the finer grid LES, and the effect of the subgrid-scalemodel on the accuracy of the results was found to besmall for the fine grid LES, which is nearly as fine as theDNS grid in the near wall region.

Acknowledgments

The author would like to thank Dr. Spyros Gavrilakisof the Ecole Polytechnique Federale de Lausanne,Switzerland, for graciously providing the DNS solutionused in this study. The author also thanks Dr. Tom Lundof the University of Texas at Arlington for his helpfuladvice on the topic of LES.

References

1Bui, Trong Tri, A Parallel Finite Volume Algorithmfor Large-Eddy Simulation of Turbulent Flows, Ph. D.thesis, Department of Aeronautics and Astronautics,Stanford University, California, Mar. 1998.

2Bui, Trong T. and Reda R. Mankbadi, “DirectNumerical Simulation of Acoustic Waves InteractingWith a Shock Wave in a Quasi-1D Convergent-Divergent Nozzle Using an Unstructured Finite VolumeAlgorithm,” FED-Vol. 219, ComputationalAeroacoustics, ASME 1995.

3Bui, Trong T., “Numerical Simulation of 3-D LowMach Number Viscous Duct Flows Using an ExplicitMethod on Massively Parallel Computer Systems,”AIAA-97-0335, Jan. 1997.

4Barth, Timothy J., “Recent Developments in HighOrder K-Exact Reconstruction on UnstructuredMeshes,” AIAA- 93-0668, Jan. 1993.

5Coirier, William John, “An Adaptively-Refined,Cartesian, Cell-Based Scheme for the Euler and Navier-Stokes Equations,” NASA TM-106754, 1994.

6Hughes, Thomas J. R., The Finite Element Method:Linear Static and Dynamic Finite Element Analysis,Prentice-Hall, Inc., Englewood Cliffs, New Jersey,1987.

7Greenberg, Michael D., Foundations of AppliedMathematics, Prentice-Hall, Inc., Englewood Cliffs,New Jersey, 1978.

8Moin, P., K. Squires, W. Cabot, and S. Lee, “ADynamic Subgrid-Scale Model for CompressibleTurbulence and Scalar Transport,” Physics of Fluids A,vol. 3, Nov. 1991, pp. 2746–2757.

– u,v

,/u2

τ

1.0

.8

.6

.4

.2

0 .2 .4 .6 .8 1.0

980523y/0.5DH



9Vreman, A. W., B. J. Geurts, and J. G. M. Kuerten,“Subgrid Modelling in LES of Compressible Flow,” inP. R. Voke et al. (eds.), Direct and Large-EddySimulation I, Kluwer Academic Publishers, Jan. 1994,pp. 133–44.

10Vreman, Albertus Willem, “Direct and Large-EddySimulation of the Compressible Turbulent MixingLayer,” Ph. D. thesis, Universiteit Twente, TheNetherlands, Dec. 1995.

11Smagorinsky, J., “General Circulation Experimentswith the Primitive Equations,” Monthly Weather Review,vol. 91, Mar. 1963, pp. 99–164.

12Germano, Massimo, Ugo Piomelli, Parviz Moin,and William H. Cabot, “A Dynamic Subgrid-Scale EddyViscosity Model,” Physics of Fluids A, vol. 3, July 1991,pp. 1760–1765.

13Ghosal, Sandip, Thomas S. Lund, Parviz Moin, andKnut Akselvoll, “A Dynamic Localization Model forLarge-Eddy Simulation of Turbulent Flows,” Journal ofFluid Mechanics, vol. 286, Mar. 1995, pp. 229–255.

14Breuer, M. and W. Rodi, “Large-Eddy Simulationof Turbulent Flow Through a Straight Square Duct and a180 Degree Bend,” Proc. of the First ERCOFTACWorkshop on DNS and LES, Guildford, Surrey, U.K.,Kluwer Academic Publishers, 1994.

15Erlebacher, G., M. Y. Hussaini, C. G. Speziale, andT. A. Zang, “Toward the Large-Eddy Simulation ofCompressible Turbulent Flows,” Journal of FluidMechanics, vol. 238, May 1992, pp. 155–185.

16Squires, Kyle and Otto Zeman, “On the Subgrid-Scale Modeling of Compressible Turbulence,”Proceedings of the 1990 Summer Program.NASA/Stanford Center for Turbulence Research,Dec. 1990, pp. 47–59.

17Yoshizawa, A., “Statistical Theory for CompressibleTurbulent Shear Flows, with the Application to SubgridModeling,” Physics of Fluids A, vol. 29, July 1986,pp. 2152–2164.

18Squires, Kyle D., “Dynamic Subgrid-ScaleModeling of Compressible Turbulence,” AnnualResearch Briefs, Stanford University Center forTurbulence Research, Dec. 1991, pp. 207–223.

19Mompean, G., S. Gavrilakis, L. Machiels, andM. O. Deville, “On Predicting the Turbulence-InducedSecondary Flows Using Nonlinear k-ε Models,” Physicsof Fluids, vol. 8, no. 7, July 1996, pp. 1856–1868.

20Gavrilakis, S., “Numerical Simulation of Low-Reynolds-Number Turbulent Flow Through a StraightSquare Duct,” Journal of Fluid Mechanics, vol. 244,1992, pp. 101–129.

21Coleman, G. N., J. Kim, and R. D. Moser, “ANumerical Study of Turbulent Supersonic Isothermal-Wall Channel Flow,” Journal of Fluid Mechanics,vol. 305, Dec. 1995, pp. 159–183.

22Madabhushi, Ravi K., and S. P. Vanka, “Large EddySimulation of Turbulence-Driven Secondary Flow in aSquare Duct,” Physics of Fluids A, vol. 3, Nov. 1991,pp. 2734–2745.

23Lin, San-Yih, Yu-Fene Chen, and Sheng-ChangShih, “Numerical Study of MUSCL Schemesfor Computational Aeroacoustics,” AIAA-97-0023,Jan. 1997.

24Wang, W. P. and R. H. Pletcher, “Evaluation ofSome Coupled Algorithms for Large Eddy Simulationof Turbulent Flow Using a Dynamic SGS Model,”AIAA-95-2244, June 1995.


REPORT DOCUMENTATION PAGE Form ApprovedOMB No. 0704-0188

Public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering andmaintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information,including suggestions for reducing this burden, to Washington Headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, Arlington,VA 22202-4302, and to the Office of Management and Budget, Paperwork Reduction Project (0704-0188), Washington, DC 20503.

1. AGENCY USE ONLY (Leave blank) 2. REPORT DATE 3. REPORT TYPE AND DATES COVERED

4. TITLE AND SUBTITLE 5. FUNDING NUMBERS

6. AUTHOR(S)

8. PERFORMING ORGANIZATION REPORT NUMBER

7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES)

9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSORING/MONITORING AGENCY REPORT NUMBER

11. SUPPLEMENTARY NOTES

12a. DISTRIBUTION/AVAILABILITY STATEMENT 12b. DISTRIBUTION CODE

13. ABSTRACT (Maximum 200 words)

14. SUBJECT TERMS 15. NUMBER OF PAGES

16. PRICE CODE

17. SECURITY CLASSIFICATION OF REPORT

18. SECURITY CLASSIFICATION OF THIS PAGE

19. SECURITY CLASSIFICATION OF ABSTRACT

20. LIMITATION OF ABSTRACT

NSN 7540-01-280-5500 Standard Form 298 (Rev. 2-89)Prescribed by ANSI Std. Z39-18298-102

A Parallel, Finite-Volume Algorithm for Large-Eddy Simulationof Turbulent Flows

WU 529-50-04

Trong T. Bui

NASA Dryden Flight Research CenterP.O. Box 273Edwards, California 93523-0273

H-2285

National Aeronautics and Space AdministrationWashington, DC 20546-0001 NASA/TM-1999-206570

A parallel, finite-volume algorithm has been developed for large-eddy simulation (LES) of compressibleturbulent flows. This algorithm includes piecewise linear least-square reconstruction, trilinear finite-elementinterpolation, Roe flux-difference splitting, and second-order MacCormack time marching. Parallelimplementation is done using the message-passing programming model. In this paper, the numerical algorithmis described. To validate the numerical method for turbulence simulation, LES of fully developed turbulentflow in a square duct is performed for a Reynolds number of 320 based on the average friction velocity and thehydraulic diameter of the duct. Direct numerical simulation (DNS) results are available for this test case, andthe accuracy of this algorithm for turbulence simulations can be ascertained by comparing the LES solutionswith the DNS results. The effects of grid resolution, upwind numerical dissipation, and subgrid-scaledissipation on the accuracy of the LES are examined. Comparison with DNS results shows that the standardRoe flux-difference splitting dissipation adversely affects the accuracy of the turbulence simulation. Foraccurate turbulence simulations, only 3–5 percent of the standard Roe flux-difference splitting dissipation isneeded.

Finite-volume; Large-eddy simulation; Parallel computers; Turbulence; Upwindmethods.

A03

27

Unclassified Unclassified Unclassified Unlimited

January 1999 Technical Memorandum

Presented at 37th AIAA Aerospace Sciences Meeting and Exhibit, Reno, Nevada, January 11–14, 1999,AIAA-99-0789.

Unclassified—UnlimitedSubject Category 34

Documents

A Parallel, Finite-Volume Algorithm for Large-Eddy …NASA/TM-1999-206570 A Parallel, Finite-Volume Algorithm for Large-Eddy Simulation of Turbulent Flows Trong T. Bui Dryden Flight