58
A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR FLUID DYNAMICS Thesis Defense Anil Kumar Karanam Department Of Computer Science Rensselaer Polytechnic Institute Troy, New York Advisor: Prof. Kenneth E. Jansen Mechanical, Aerospace and Nuclear Engineering, Biomedical Engineering, Computer Science Rensselaer Polytechnic Institute Troy, New York —- SCOREC, CS, RPI —- 1

A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR …karana/talk.pdf · 2008. 5. 6. · A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR FLUID DYNAMICS Thesis Defense Anil Kumar

  • Upload
    others

  • View
    5

  • Download
    0

Embed Size (px)

Citation preview

Page 1: A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR …karana/talk.pdf · 2008. 5. 6. · A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR FLUID DYNAMICS Thesis Defense Anil Kumar

A P-ADAPTIVE STABILIZED FINITE ELEMENT

METHOD FOR FLUID DYNAMICS

Thesis Defense

Anil Kumar KaranamDepartment Of Computer ScienceRensselaer Polytechnic Institute

Troy, New York

Advisor:

Prof. Kenneth E. JansenMechanical, Aerospace and Nuclear Engineering, Biomedical Engineering,

Computer ScienceRensselaer Polytechnic Institute

Troy, New York

—- SCOREC, CS, RPI —- 1

Page 2: A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR …karana/talk.pdf · 2008. 5. 6. · A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR FLUID DYNAMICS Thesis Defense Anil Kumar

Overview of the presentation

• Motivation

• FEM formulation and Basis functions

• Preprocessing based FEM simulations

• Design of the P-Adaptivity

• Results

• Future work

—- SCOREC, CS, RPI —- 2

Page 3: A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR …karana/talk.pdf · 2008. 5. 6. · A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR FLUID DYNAMICS Thesis Defense Anil Kumar

Motivation

• There exists a class of problems in fluid dynamics for which p-adaptivitycan be used effectively to enrich the solution space with a cost lowerthan that of pure h-adaptivity (Whiting [15])

• Stabilized methods ( SUPG, GLS etc.) well suited for p-adaptivemethods Optimal convergence rates with respect to interpolation error(Hughes et al. [6], Franca and Frey [4], Johnson and Szepessy [9])

—- SCOREC, CS, RPI —- 3

Page 4: A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR …karana/talk.pdf · 2008. 5. 6. · A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR FLUID DYNAMICS Thesis Defense Anil Kumar

Available Advancements

• Abstract mesh data structure and mesh-model classification proposed by(Beall and Shephard [1])

– Very powerful for h-adaptivity– Model classification helps with ease in setting boundary conditions– Rich data structures make extensions easier

• Construction of hierarchic basis functions for variable p-order meshes(Shephard et al. [13]). These basis functions were shown to yield betterconditioned matrices (Carnevali et al. [2])

—- SCOREC, CS, RPI —- 4

Page 5: A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR …karana/talk.pdf · 2008. 5. 6. · A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR FLUID DYNAMICS Thesis Defense Anil Kumar

Stabilized FEM solver

• A Stabilized FEM solver for Navier-Stokes equations.

• Optimized over 10 years for linear basis computations

– Blocked element data structure– Shape functions which are precomputed and tabulated at quadrature

points– Parent domain integrals with efficient mapping

• Input data format and runtime data structures which are compact andlight weight, which support all the above features and provide goodcache performance.

• Excellent scalability

—- SCOREC, CS, RPI —- 5

Page 6: A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR …karana/talk.pdf · 2008. 5. 6. · A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR FLUID DYNAMICS Thesis Defense Anil Kumar

First Adaptive implementation

• Goal ( circa. 1998 ) was to develop an adaptive simulation strategy whichpreserved the computational efficiency of the core solver while takingadvantage of the flexibility provided by the advanced mesh representation.

• Whiting, Jansen, Karanam, created the first implementation based on atwo stage approach

– Pre-processor implementation and design driven by Karanam ([10])– Solver side of the extensions and implementations provided by Whiting

([15])

• Support for alternate and mixed topologies other than tetrahedra wasimplemented (Jansen, Karanam and Whiting [8])

—- SCOREC, CS, RPI —- 6

Page 7: A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR …karana/talk.pdf · 2008. 5. 6. · A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR FLUID DYNAMICS Thesis Defense Anil Kumar

Uniform P Implementation

• Uniform p-adaptivity can be used effectively to enrich the solution space(Whiting [15])

• While the global cost is less, p-adaptivity does incur additional costproportional to the number and polynomial order of the added modes

• Uniform p-adaptivity is hence not optimal for problems with localizedfeatures, and calls for a localized p-adaptivity

—- SCOREC, CS, RPI —- 7

Page 8: A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR …karana/talk.pdf · 2008. 5. 6. · A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR FLUID DYNAMICS Thesis Defense Anil Kumar

Generalized Connectivity Template (Key DS)

—- SCOREC, CS, RPI —- 8

Page 9: A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR …karana/talk.pdf · 2008. 5. 6. · A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR FLUID DYNAMICS Thesis Defense Anil Kumar

Element Blocking

PSfrag replacements

Nel(total) : linear

• All elements are of the same topology and polynomial order

—- SCOREC, CS, RPI —- 9

Page 10: A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR …karana/talk.pdf · 2008. 5. 6. · A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR FLUID DYNAMICS Thesis Defense Anil Kumar

Element Blocking cont..

PSfrag replacements

Nel(total)

Nelbk elements in a single block

• Block size can be changed to tune for cache performance

—- SCOREC, CS, RPI —- 10

Page 11: A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR …karana/talk.pdf · 2008. 5. 6. · A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR FLUID DYNAMICS Thesis Defense Anil Kumar

Element Formation

< Loop over Element blocks >

c Loop over quadrature points

do intp = 1, ngauss

c.... get the shape functions at this int point

call getshp( shape, shdrv, detJ)

do na = 1, nshl ! #of local shape functions

do nb = 1, nshl

TMP = shape(:,na)*shape(:,nb)*detJ

< Loop over Elements in a block >

A(:,na,nb)=A(:,na,nb)+TMP*C(:)

< end Loop over Elements in a block >

enddo

enddo

enddo ! end Loop over quadrature points

< end Loop over Element blocks >

—- SCOREC, CS, RPI —- 11

Page 12: A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR …karana/talk.pdf · 2008. 5. 6. · A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR FLUID DYNAMICS Thesis Defense Anil Kumar

Element Formation cont..

• Inner loops with no loop carried dependencies

• Outer loops accumulate

• Completely order independent

• Ideal candidates for threading

• Easy to pre compute shape functions and tabulate

• Element specific transformations handled by Jacobian

• Predictable cache performance

—- SCOREC, CS, RPI —- 12

Page 13: A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR …karana/talk.pdf · 2008. 5. 6. · A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR FLUID DYNAMICS Thesis Defense Anil Kumar

Changes for Uniform P

PSfrag replacements

Nel(total) : linear Nel(total) : quadratic Nel(total) : cubic

• All elements are of the same topology and polynomial order

—- SCOREC, CS, RPI —- 13

Page 14: A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR …karana/talk.pdf · 2008. 5. 6. · A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR FLUID DYNAMICS Thesis Defense Anil Kumar

Proposed extension to P-adaptivity

PSfrag replacements

Nlinearel

Nquadraticel

Ncubicel

Ntotalel = Nlinear

el + Nquadraticel

+ Ncubicel

• Every unique combination of topology and polynomial order is grouped together

• Element blocks are now subsets of these groups

• Preserves all previously mentioned benefits while enabling p-adaptivity

• Relative sizes of these blocks change with adaptation

• Total number of elements remains constant for a given mesh

—- SCOREC, CS, RPI —- 14

Page 15: A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR …karana/talk.pdf · 2008. 5. 6. · A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR FLUID DYNAMICS Thesis Defense Anil Kumar

Incompressible Navier-Stokes Equations

The strong form of the continuity and momentum equations written in theadvective form( Gresho and Sani [5] )

ui,i = 0

ui + ujui,j = −p,i + τij,j + fi (1)

where ui is the ith component of velocity, p the pressure divided by thedensity ρ (assumed constant), fi the prescribed body force, and τij theviscous stress tensor given by:

τij = ν(ui,j + uj,i) (2)

where ν = µρ

is the kinematic viscosity, and the summation convention is

used throughout (sum on repeated indices).

—- SCOREC, CS, RPI —- 15

Page 16: A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR …karana/talk.pdf · 2008. 5. 6. · A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR FLUID DYNAMICS Thesis Defense Anil Kumar

Semi-Discrete, Stabilized, Galerkin FEM formulation

Ω ⊂ RN

Ω = Ω ∪ Γ

Γ = Γg ∪ Γh

Sph = v|v(·, t) ∈ H

1(Ω)N, t ∈ [0, T ], v|x∈Ωe

∈ Pp(Ωe)N

, v(·, t) = g on Γg,

Wph = w|w(·, t) ∈ H

1(Ω)

N, t ∈ [0, T ], w|x∈Ωe

∈ Pp(Ωe)N

, w(·, t) = 0 on Γg,

Pph = p|p(·, t) ∈ H

1(Ω), t ∈ [0, T ], p|x∈Ωe

∈ Pp(Ωe)

Pp(Ωe) is a piecewise polynomial space, complete to order p, defined on the element Ωe.

Li = ui + ujui,j + p,i − τij,j − fi

—- SCOREC, CS, RPI —- 16

Page 17: A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR …karana/talk.pdf · 2008. 5. 6. · A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR FLUID DYNAMICS Thesis Defense Anil Kumar

Semi-Discrete, Stabilized, Galerkin FEM formulation

Find u ∈ Sph and p ∈ Pp

h such that

B(wi, q;ui, p) = 0

B(wi, q;ui, p) =

∫Ω

wi (ui + ujui,j − fi) + wi,j (−pδij + τij) − q,iuidx

+

nel∑e=1

∫Ωe

τM(ujwi,j + q,i)Li + τCwi,iuj,j dx

+

nel∑e=1

∫Ωe

wi∆ujui,j + τ

∆ujwi,j

∆ukui,k dx

+

∫Γh

wi (pδin − τin) + qun ds

The above formulations is based on that described by Taylor et al. [14].

—- SCOREC, CS, RPI —- 17

Page 18: A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR …karana/talk.pdf · 2008. 5. 6. · A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR FLUID DYNAMICS Thesis Defense Anil Kumar

Localizing Element Integrals

PSfrag replacements

Ωe → x1, x2, x3

2 → ξ1, ξ2, ξ3

x(ξ) ↔ ξ(x)

Ωe ↔ 2

dΩe ↔ d2

0 ≤ ξi ≤ 1

B(wi, q; ui, p) =

Z

Ω(NA(x)g1(x) + NA,i(x)g2(x))dΩ

nelX

e=1

Z

Ωe(Na(x)g1(x) + Na,i(x)g2(x))dΩe ↔

nelX

e=1

Z

2

(Na(ξ)g1(ξi) + Na,ξ(ξ)ξ,xg2(ξ)) det(J)d2

D(ξ) = det(J) = det(x,ξ)

f(ξ) = (Na(ξ)g1(ξi) + Na,ξ(ξ)ξ,xg2(ξ))D(ξ)

nelX

e=1

nintX

l=1

f(ξl)Wl

—- SCOREC, CS, RPI —- 18

Page 19: A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR …karana/talk.pdf · 2008. 5. 6. · A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR FLUID DYNAMICS Thesis Defense Anil Kumar

Hierarchic Basis Functions

• Individual functions consist of element blend (ψ) multiplying an entityfunction (φ)(Shepard et al. [13],Dey [3])

Na = ψa(ξ)φa(ξ)

• Blend is of fixed polynomial order and ensures correct support of basis,while the entity level function gives the desired polynomial behavior

• Basis functions are written in terms of local coordinates, given by

0 ≤ ξi ≤ 1, i ≤ im

ξim+1 = 1 −

im∑k=1

ξk

im = 1, 2, 3 for edge, face, and region

—- SCOREC, CS, RPI —- 19

Page 20: A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR …karana/talk.pdf · 2008. 5. 6. · A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR FLUID DYNAMICS Thesis Defense Anil Kumar

—- SCOREC, CS, RPI —- 20

Page 21: A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR …karana/talk.pdf · 2008. 5. 6. · A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR FLUID DYNAMICS Thesis Defense Anil Kumar

Shape Functions for Various Entities

P=2 Edge

P=3 Edge

P=3 Face P=4 Face

P=4 Face

p=1 Vertex

P=4 Edge

—- SCOREC, CS, RPI —- 21

Page 22: A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR …karana/talk.pdf · 2008. 5. 6. · A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR FLUID DYNAMICS Thesis Defense Anil Kumar

Remarks on Hierarchic Basis

• There are several important aspects of the hierarchic basis

– Based on Legendre Polynomials (from Carnevali et al. [2])

– The hierarchic basis has the property that the basis for polynomialorder p is a subset of p+ 1

– Hierarchic functions are associated with mesh entities where asLagrange functions are associated with nodal coordinates

– Lagrange basis coefficients correspond to solution values at the nodes.Hierarchic coefficients are a property of the entire entity; no particularlocation

– Possible to have elements of mixed order elements without resortingto hanging nodes

—- SCOREC, CS, RPI —- 22

Page 23: A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR …karana/talk.pdf · 2008. 5. 6. · A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR FLUID DYNAMICS Thesis Defense Anil Kumar

Models of Adaptivity & Implementation

• h-adaptivity: Adapting the FEM mesh size, either locally or

globally

• p-adaptivity: Changing the polynomial order of the basis

functions representing the trial and weight space

• hp-adaptivity: A combination of the above two

• r-adaptivity: Relocating the nodes but maintaining the same

number of global degrees of freedom and connectivity

—- SCOREC, CS, RPI —- 23

Page 24: A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR …karana/talk.pdf · 2008. 5. 6. · A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR FLUID DYNAMICS Thesis Defense Anil Kumar

Computing Framework h-adaptive

Classified Mesh

G.U.I.

Attributes

Rich Database

Compact Database

Flow Solver Error Indicator

Mesh Adaptor

Preprocessor

Adaptive Refinement?

Hierarchical Postprocessing

Geometric Model

Yes

No

Adaptive Loop

Visualization Software

Rich Database

—- SCOREC, CS, RPI —- 24

Page 25: A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR …karana/talk.pdf · 2008. 5. 6. · A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR FLUID DYNAMICS Thesis Defense Anil Kumar

Computing Framework p-adaptive

Classified Mesh

G.U.I.

Attributes

Rich

DatabaseCompact

Database

Flow Solver

Error Indicator

Preprocessor

Adaptive

Refinement?

Hierarchical

Postprocessing

Geometric

Model

Yes

No

P-Adaptive

Loop

Visualization

Software

Rich

Database

—- SCOREC, CS, RPI —- 25

Page 26: A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR …karana/talk.pdf · 2008. 5. 6. · A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR FLUID DYNAMICS Thesis Defense Anil Kumar

Pre-processing simulations

• Generate connectivity statically, generalized connectivity for aelement is all the non-zero modes of that element (Whiting [15],Karanam et al. [10])

• Evaluate boundary conditions via inheritence (Karanam et al. [10]) andtabulate shape-functions

• Partition and pre-compute parallel communication tasks

• Advantages:

– Compact efficient data structures, significant memory savings– Faster simulations, lesser function calls– Substantial amount of work is done in the preprocessing stage

• Disadvantages:

– Not suited for rapid h-adaptivity, difficult to automate– Substantial amount of work is done in the preprocessing stage

—- SCOREC, CS, RPI —- 26

Page 27: A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR …karana/talk.pdf · 2008. 5. 6. · A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR FLUID DYNAMICS Thesis Defense Anil Kumar

Parallel Communication

1

987

6 5 4

32

1 2

34

7

1 2 3 4

5

4

6

321

7 6 5

111213

8 9 10

5 6 7 8

2

3

4

1

(a) Mesh

1 2

3 4

(b) Graph

2

3

1

(c) overview

2

3

1

(d) type1

2

3

1

(e) type2

—- SCOREC, CS, RPI —- 27

Page 28: A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR …karana/talk.pdf · 2008. 5. 6. · A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR FLUID DYNAMICS Thesis Defense Anil Kumar

Objectives: for P-adaptive implementation

• Leverages the existing infrastructure

• Can co-exist with unrefined cases

• Should be capable of both refinement and coarsening as needed

• Does not cause inefficiency in the linear modes

• Extends the existing data structures

—- SCOREC, CS, RPI —- 28

Page 29: A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR …karana/talk.pdf · 2008. 5. 6. · A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR FLUID DYNAMICS Thesis Defense Anil Kumar

Strategy for P-Adaptivity

• Model for adaptivity is based on a feedback loop.

• Refinement is driven by simple error indicators based on the L2 normsof the residual of the three components of the momentum equation

∑MABe

iB = h2

∫Ω

NAL2idΩ (3)

• Entities are marked for refinement if the error value on and surroundingthem is higher than a mean error value

• For every level of refinement of an entity, the polynomial order associatedwith that entity goes up by 1 and consequently the modes associatedwith it are increased

—- SCOREC, CS, RPI —- 29

Page 30: A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR …karana/talk.pdf · 2008. 5. 6. · A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR FLUID DYNAMICS Thesis Defense Anil Kumar

Implementation Details

• Element quadrature rules based on local polynomial order.

– To perform templated integration, elements were grouped into blocksof common polynomial order.

• Maintaining templated higher order connectivity , to avoid having to dealwith special cases.

– Vertex connectivity plus higher order modes– The polynomial order of an element is set to that of the highest

polynomial order of any of its constituent entities or itself.– Higher order connectivity is generated for the element as if all the

constituent entities are of the polynomial order of the element

—- SCOREC, CS, RPI —- 30

Page 31: A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR …karana/talk.pdf · 2008. 5. 6. · A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR FLUID DYNAMICS Thesis Defense Anil Kumar

Mixed Order Elements

3

6

2

4

5

7

8 9

10

1

3

61

2

4

5

7

8

9

10

Linear Element, Quadratic Element

• Handling of mixed order elements

– Creates gaps in standard connectivity templates– Too many specific cases to identify uniquely– Introduction of constrained virtual mode

—- SCOREC, CS, RPI —- 31

Page 32: A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR …karana/talk.pdf · 2008. 5. 6. · A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR FLUID DYNAMICS Thesis Defense Anil Kumar

Mixed Order Elements, cont..

1

2 3

4

5

12

11

13

67

89

10

1

2

5

element1 = 1 , 2 , 3 , 4 element2 = 2, 3, 4, 5, 14, 14, 14, 14, 14, 11

element3 = 4, 3, 6, 5, 14, 9, 14, 13, 14, 11 element4 = 3, 7, 6, 5, 8, 10, 9, 13, 11, 12

Virtual Mode = 14

12

11

13

3

6

4

7

89

10

Linear Element, Quadratic Element

—- SCOREC, CS, RPI —- 32

Page 33: A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR …karana/talk.pdf · 2008. 5. 6. · A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR FLUID DYNAMICS Thesis Defense Anil Kumar

Structure of the FEM solver

Computational work in the FEM solver can be largely classified into twocategories.

• Element formation:

– This involves the computation of element level integrals for all thevarious terms of the FEM formulation.

– Forms the LHS tangent matrix and RHS residual vector.– Global equation entries are assembled from values locally computed

on each element

• Linear Algebra

– Stores matrices in sparse storage– Solves global equations using A p products

—- SCOREC, CS, RPI —- 33

Page 34: A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR …karana/talk.pdf · 2008. 5. 6. · A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR FLUID DYNAMICS Thesis Defense Anil Kumar

Cost Metrics

We define three cost metrics based on the main areas of work(Jansen,Karanam,Whiting[8])

• (LHS) tangent matrix: CL =∑3

l=1 nlel ∗ (nl

shp)2 ∗ nl

int

• (RHS) residual vector: CR =∑3

l=1 nlel ∗ n

lshp ∗ n

lint

• Linear Algebra Cost for the Iterative solver: CS = nk ∗ nnz

(nnz is the number of non-zeros in the LHS , nk is the

number of Krylov vectors used, is unpredictable prior to

solve)

—- SCOREC, CS, RPI —- 34

Page 35: A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR …karana/talk.pdf · 2008. 5. 6. · A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR FLUID DYNAMICS Thesis Defense Anil Kumar

Performance & Scalability

• Virtual mode contributions are carefully avoided when forming the sparsematrix for LHS

• Element block size is varied based on polynomial order and predictablememory footprint (current choice is in the ratio of number of elementshapefunctions 1 : 2.5 : 5)

• Interpartition communication traces had to account for virtual mode toavoid unnecessary communication.

• Weighted partitioning schemes implemented based on element formationcost metrics discussed.

—- SCOREC, CS, RPI —- 35

Page 36: A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR …karana/talk.pdf · 2008. 5. 6. · A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR FLUID DYNAMICS Thesis Defense Anil Kumar

Post processing p-adapted solution

• Post processing higher order solutions presents some difficulty as visualization packages

which typically need a linear solution and mesh. This creates the need to generate a

visualization mesh on to which the higher order solution needs to be interpolated.

• Visualization mesh was generated using tag based refinement of entities which had

higher order modes on them. Elements are taged for refinement once for every unit

increase in polynomial order.

left:visualization mesh, right: mesh used for simulation

—- SCOREC, CS, RPI —- 36

Page 37: A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR …karana/talk.pdf · 2008. 5. 6. · A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR FLUID DYNAMICS Thesis Defense Anil Kumar

Surface Mounted Cube: geometry

Re = 50, 0 ≤ x ≤ 10, 0 ≤ y ≤ 1, 0 ≤ z ≤ 1.5,

—- SCOREC, CS, RPI —- 37

Page 38: A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR …karana/talk.pdf · 2008. 5. 6. · A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR FLUID DYNAMICS Thesis Defense Anil Kumar

Reference Solution for Surface mounted Cube.

• Experimental results were not available for chosen problem setup.

• Performed a convergence study to establish a reference solution for thechosen problem.

• Isotropic size field based h-adaptivity was used to generate successivemeshes using earlier described error indicators.

—- SCOREC, CS, RPI —- 38

Page 39: A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR …karana/talk.pdf · 2008. 5. 6. · A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR FLUID DYNAMICS Thesis Defense Anil Kumar

Surface Mounted Cube: reference solution

103 104 105 106 10710−6

10−5

10−4

10−3

10−2

10−1Convergence plot

Number of modes

L2

norm

of e

rror

Error w.r.t reference mesh

Reference mesh used had 3729136 modes.

—- SCOREC, CS, RPI —- 39

Page 40: A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR …karana/talk.pdf · 2008. 5. 6. · A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR FLUID DYNAMICS Thesis Defense Anil Kumar

Surface Mounted Cube: Speed profile

—- SCOREC, CS, RPI —- 40

Page 41: A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR …karana/talk.pdf · 2008. 5. 6. · A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR FLUID DYNAMICS Thesis Defense Anil Kumar

Surface mounted Cube: Refinement Study

• Started with a Linear simulaton on an inital mesh of 17688 elements

• Successively p-adapted the mesh to add first quadratic and then cubicmodes, based on the error indicator values computed from the simulationon the previous mesh

• Two levels of h-adaptation were also performed, starting with the samebase mesh using a isotropic sizefield based approach

• Effort was made to keep the size of the h-adapted mesh at any stageapproximately equivalent in element formation cost metric to the firstp-adaptation of the previous level

—- SCOREC, CS, RPI —- 41

Page 42: A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR …karana/talk.pdf · 2008. 5. 6. · A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR FLUID DYNAMICS Thesis Defense Anil Kumar

Surface Mounted Cube: refinement study

• Hi is a h-adapted mesh, Pj is a p-adapted mesh

• Hi-Pj is a p-adated mesh starting from Hi

• H2-P1 ∼ H1-P2 in cost metric CL + CR

• H3-P1 ∼ H2-P2 in cost metric CL + CR

P1 P2 P317688 13314 12431

H1 0 4374 16310 0 3626

102345 76779 62195H2 0 25566 16277

0 0 23873611190 600032 576115

H3 0 11158 239170 0 11158

Table 1: linear elements, quadratic elements, cubic elements

—- SCOREC, CS, RPI —- 42

Page 43: A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR …karana/talk.pdf · 2008. 5. 6. · A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR FLUID DYNAMICS Thesis Defense Anil Kumar

Surface Mounted Cube: Element Formation

P1 P2 P3H1 4 27.74 160.66H2 23.14 161.9 1084.6H3 138.22 198.77 722.85

Table 2: CL cost measure: LHS formation

P1 P2 P3H1 1.0 3.2256 9.0561H2 5.7861 18.7946 61.6441H3 34.5539 40.2313 68.9599

Table 3: CR cost measure: RHS formation

—- SCOREC, CS, RPI —- 43

Page 44: A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR …karana/talk.pdf · 2008. 5. 6. · A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR FLUID DYNAMICS Thesis Defense Anil Kumar

Surface Mounted Cube: Element Formation

P1 P2 P3

H1 1 5.7 29.79

H2 5.79 33.25 201.31

H3 34.55 46.54 143.46

Table 4: Element Formation Cost Estimate normalized (for two timesteps)

P1 P2 P3

H1 1.000 3.4558 19.5857

H2 5.9801 32.4585 105.6861

H3 38.5047 44.3876 89.5624

Table 5: Element Formation Time (normalized)

P1 P2 P3

H1 21.1646 73.1412 414.5233

H2 126.5670 686.8922 2236.8030

H3 814.9367 939.4453 1895.5520

Table 6: Element Formation Time (seconds)

—- SCOREC, CS, RPI —- 44

Page 45: A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR …karana/talk.pdf · 2008. 5. 6. · A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR FLUID DYNAMICS Thesis Defense Anil Kumar

Analysis of Element formation cost.

• The estimate is more accurate on meshes of uniform polynomial order

• Proportionality constant which differs for various polynomial orders needsto be accounted for to make estimates more accurate for mixed ordermeshes.

• Based on the rapid increase in CL cost mertric as we add higher orderelements, it is clear that higher order simulations ( cubics in particular )would benefit from forming the LHS less frequently.

• Cost metric is by and large successful in predicting the trend of elementformation costs

—- SCOREC, CS, RPI —- 45

Page 46: A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR …karana/talk.pdf · 2008. 5. 6. · A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR FLUID DYNAMICS Thesis Defense Anil Kumar

Surface Mounted Cube: Linear Algebra

P1 P2 P3

H1 1.0 4.86 48.45

H2 9.47 42.06 337.96

H3 132.73 211.9 581.36

Table 7: Cost metric CS : nnz ∗ nk

P1 P2 P3

H1 1.0 3.7719 30.9834

H2 9.4749 33.4107 236.3840

H3 145.2266 213.3015 487.0264

Table 8: Linear Algebra Time (normalized)

P1 P2 P3

H1 7.1070 26.8071 220.1991

H2 67.3380 237.4496 1679.9812

H3 1032.1251 1515.9338 3461.2969

Table 9: Linear Algebra Time (seconds)

—- SCOREC, CS, RPI —- 46

Page 47: A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR …karana/talk.pdf · 2008. 5. 6. · A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR FLUID DYNAMICS Thesis Defense Anil Kumar

Analysis of Linear Algebra cost.

• Linear algebra costs are hard to charcterize in terms of cost per elementadded since the global system is solved with all elements of mixed anduniform p-order

• Cost metric used does provided reasonable corelation to to the observedsolver time.

• We see a large increase in the linear algebra cost as we add quadraticand cubic modes.

• Current study converged each simulation from the same simple initialcondition. An alternate strategy, where solution is transfered from theprevious adaptation, may substantially accelerate quadratic and cubicconvergence.

—- SCOREC, CS, RPI —- 47

Page 48: A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR …karana/talk.pdf · 2008. 5. 6. · A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR FLUID DYNAMICS Thesis Defense Anil Kumar

Surface Mounted Cube: Error vs Runtime

101 102 103 10410−4

10−3

10−2

10−1Convergence plot

Time in seconds

L2

norm

of e

rror

linear+quadratic+cubic

L2 norm of the error vs Total runtime in seconds

Quadratic modes seem to provide some cost benefit over pure linear meshes,when solving for solutions of similar quality

—- SCOREC, CS, RPI —- 48

Page 49: A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR …karana/talk.pdf · 2008. 5. 6. · A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR FLUID DYNAMICS Thesis Defense Anil Kumar

Surface Mounted Cube: Further p-adaptation

101 102 103 104 10510−4

10−3

10−2

10−1Convergence plot

Time in seconds

L2

norm

of e

rror

linear+quadratic+cubic++cubic

L2 norm of the error vs Total runtime in seconds

Since we are limited to cubic modes, further p-adaptiviy cause us to addsubstantial cost with a minimal gain in accuracy.

—- SCOREC, CS, RPI —- 49

Page 50: A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR …karana/talk.pdf · 2008. 5. 6. · A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR FLUID DYNAMICS Thesis Defense Anil Kumar

Partitioning for mixed p-order

• Cost metrics appear to be over estimating the cost for mixed p-ordermeshes. While this is a cause for further improving the metrics, italso points to some relative performance benefit of higher order modescompared to linears.

• Unlike uniform p-meshes, partitioning the mesh based on number ofelements will not provide optimal load balance and scalability for mixedorder meshes and will require weighted partitioning.

• The predictive nature of the element formation cost metrics lets us assignrelative weights to various elements (nodes in the dual graph) based onpolynomial order and accordingly partition the mesh.

• For unsteady problems where the cost of element formation wouldcontinue to be a large percentage of the cost, this is likely to be a goodmodel.

—- SCOREC, CS, RPI —- 50

Page 51: A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR …karana/talk.pdf · 2008. 5. 6. · A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR FLUID DYNAMICS Thesis Defense Anil Kumar

Partitioning for mixed p-order cont..

• For steady problems as we can reduce the cost of LHS formation byreducing its frequency, linear algebra costs would dominate the runtime.

• In such circumstances, since number of krylov vectors is hard to predict,an alternated partition weighting scheme which balances the number ofnon-zero entries in each part can be used.

—- SCOREC, CS, RPI —- 51

Page 52: A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR …karana/talk.pdf · 2008. 5. 6. · A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR FLUID DYNAMICS Thesis Defense Anil Kumar

Work to be completed

• Complete a refinement study of the Kovasznay flow.

Having an analytical solution, this problem would make a

good testcase for evaluating cost vs accuracy of p-adaptivity

• Complete a scalability study of a p-adapted, mixed order

mesh using partition schemes based on the various cost

metrics developed

• A more thorough optimization of block size for higher order

elements.

—- SCOREC, CS, RPI —- 52

Page 53: A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR …karana/talk.pdf · 2008. 5. 6. · A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR FLUID DYNAMICS Thesis Defense Anil Kumar

Summary & Conclusions

• A Pre-processing based framework was developed for adaptivity, with specific emphasis

on p-adaptivity

• The efficient input and runtime data structures were preserved and extended to support

mixed p-order

• The solver implementation preserves linear performance while adding the ability to

grade p-order

• Initial results look promising when high accuracy is needed.

• Pre-processor based adaptivity mode provides good trade off between flexibility of a full

mesh database and the efficiency of the FEM solver.

• p-adaptivity provides cost and accuracy benefits in smooth regions of the flow.

• Quadratic modes seem to show the best cost/benefit ratio

• Adjustments to reduce higher order costs should make them viable alternatives.

—- SCOREC, CS, RPI —- 53

Page 54: A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR …karana/talk.pdf · 2008. 5. 6. · A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR FLUID DYNAMICS Thesis Defense Anil Kumar

Future Work

• More research needed in error estimators to use

• Improvements to adaptation strategy should be considerd

– criterion for adding higher modes– smoothing of polynomial gradation

• Directional refinement needs to be considered

• A more complete comparison to h-adaptivity needs to be performed

• A truly h-p adaptive model needs to be developed

—- SCOREC, CS, RPI —- 54

Page 55: A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR …karana/talk.pdf · 2008. 5. 6. · A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR FLUID DYNAMICS Thesis Defense Anil Kumar

Acknowledgments

• Prof. Kenneth E. Jansen

• Prof. Mark Shephard, Prof. Jacob Fish, Prof. Christopher Carothers

• Dr. Onkar Sahni, Dr. Chris Whiting, Dr. Andres E. Tejada-Martınez,Saurabh Tendulkar

• Marge Verville, Tim Wickberg, Adam Todorski, Christophe Dupre,SCOREC

• Simmetrix Inc

• Dr. Farzin Shakib, Acusim Software.

• National Science Foundation under Grant No. 9985340.

—- SCOREC, CS, RPI —- 55

Page 56: A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR …karana/talk.pdf · 2008. 5. 6. · A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR FLUID DYNAMICS Thesis Defense Anil Kumar

References

[1] M. W. Beall and M. S. Shephard. A general topology-based mesh datastructure. Int. J. Numer. Meth. Engng., 40(9):1573–1596, 1997.

[2] P. Carnevali, R. B. Morris, Y. Tsuji, and G. Taylor. New basis functionsand computational procedures for p-version finite element analysis. Int.

J. Numer. Meth. Engng., 36:3759–3779, 1993.

[3] S. Dey. Geometry-based three-dimensional hp-finite element modelling

and computations. PhD thesis, Rensselaer Polytechnic Institute, 1997.

[4] L. P. Franca and S. Frey. Stabilized finite element methods: II. Theincompressible Navier-Stokes equations. Comp. Meth. Appl. Mech.

Engng., 99:209–233, 1992.

[5] P. M. Gresho and R. L. Sani. Incompressible Flow and the Finite

Element Method. Wiley, New York, NY, 1998.

—- SCOREC, CS, RPI —- 56

Page 57: A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR …karana/talk.pdf · 2008. 5. 6. · A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR FLUID DYNAMICS Thesis Defense Anil Kumar

[6] T. J. R. Hughes, L. P. Franca, and G. M. Hulbert. A new finiteelement formulation for fluid dynamics: VIII. The Galerkin / least–squares method for advective–diffusive equations. Comp. Meth. Appl.

Mech. Engng., 73:173–189, 1989.

[7] M. Suri I. Babu˘ ska. The optimal convergence rate of the p-versionof the finite element method. SJNA, 24:750–776, 1987.

[8] K. E. Jansen, A. K. Karanam, and C. H. Whiting. Element topologyconsiderations when employing a hierarchical basis for stabilized fem influid dynamics. In Proceedings of the ASME 2000 Fluids Engineering

Division Summer Meeting, FED. ASME, 2000.

[9] C. Johnson and A. Szepessy. On the convergence of a finite elementmethod for a nonlinear hyperbolic conservation law. Mathematics of

Computation, 1987.

[10] A. K. Karanam and K. E. Jansen C. H. Whiting. A geometry basedpreprocessor for parallel fluid dynamics simulations using a hierarchicalbasis. Engineering with Computers, 2007.

—- SCOREC, CS, RPI —- 57

Page 58: A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR …karana/talk.pdf · 2008. 5. 6. · A P-ADAPTIVE STABILIZED FINITE ELEMENT METHOD FOR FLUID DYNAMICS Thesis Defense Anil Kumar

[11] Anil Kumar Karanam. Hierarchical hexahedral element for fluiddynamic simulations using stabilized finite element methods. TechnicalReport SCOREC #4-2001, Scientific Computation Research Center,Troy,NY,12180, 2001.

[12] F. Shakib. Finite element analysis of the compressible Euler and

Navier-Stokes equations. PhD thesis, Stanford University, 1989.

[13] M. S. Shephard, S. Dey, and J. E. Flaherty. A straight forward structureto construct shape functions for variable p-order meshes. Comp. Meth.

Appl. Mech. Engng., 147:209–233, 1997.

[14] C. A. Taylor, T. J. R. Hughes, and C. K. Zarins. Finite elementmodeling of blood flow in arteries. Comp. Meth. Appl. Mech. Engng.,158:155–196, 1998.

[15] C. H. Whiting. Stabilized finite element methods for fluid dynamics

using a hierarchical basis. PhD thesis, Rensselaer Polytechnic Institute,Sep. 1999.

—- SCOREC, CS, RPI —- 58