A study of direct and Krylov iterative sparse solver ... 1 A study of direct and Krylov iterative sparse

  • View

  • Download

Embed Size (px)

Text of A study of direct and Krylov iterative sparse solver ... 1 A study of direct and Krylov iterative...

  • A study of direct and Krylov iterative sparse solver1

    techniques to approach linear scaling of the integration2

    of Chemical Kinetics with detailed combustion3


    Federico Perinia,∗, Emanuele Galliganib, Rolf D. Reitza,5

    aUniversity of Wisconsin–Madison Engine Research Center, 1500 Engineering Drive,6 Madison, WI 53706, U.S.A.7

    bDipartimento di Ingegneria Enzo Ferrari, Università di Modena e Reggio Emilia, strada8 Vignolese 905/A, 41125 Modena, Italy9


    The integration of the stiff ODE systems associated with chemical ki-11 netics is the most computationally demanding task in most practical com-12 bustion simulations. The introduction of detailed reaction mechanisms in13 multi-dimensional simulations is limited by unfavorable scaling of the stiff14 ODE solution methods with the mechanism’s size. In this paper, we com-15 pare the efficiency and the appropriateness of direct and Krylov subspace16 sparse iterative solvers to speed-up the integration of combustion chemistry17 ODEs, with focus on their incorporation into multi-dimensional CFD codes18 through operator splitting. A suitable preconditioner formulation was ad-19 dressed by using a general-purpose incomplete LU factorization method for20 the chemistry Jacobians, and optimizing its parameters using ignition delay21 simulations for practical fuels. All the calculations were run using a same ef-22 ficient framework: SpeedCHEM, a recently developed library for gas-mixture23 kinetics that incorporates a sparse analytical approach for the ODE system24 functions. The solution was integrated through direct and Krylov subspace25 iteration implementations with different backward differentiation formula in-26 tegrators for stiff ODE systems: LSODE, VODE, DASSL. Both ignition de-27 lay calculations, involving reaction mechanisms that ranged from 29 to 717128 species, and multi-dimensional internal combustion engine simulations with29 the KIVA code were used as test cases. All solvers showed similar robustness,30 and no integration failures were observed when using ILUT-preconditioned31

    ∗Corresponding author Email addresses: perini@wisc.edu (Federico Perini),

    emanuele.galligani@unimore.it (Emanuele Galligani), reitz@engr.wisc.edu (Rolf D. Reitz)

    Preprint submitted to Combustion and Flame November 16, 2013

  • Krylov enabled integrators. We found that both solver approaches, coupled32 with efficient function evaluation numerics, were capable of scaling computa-33 tional time requirements approximately linearly with the number of species.34 This allows up to three orders of magnitude speed-ups in comparison with35 the traditional dense solution approach. The direct solvers outperformed36 Krylov subspace solvers at mechanism sizes smaller than about 1000 species,37 while the Krylov approach allowed more than 40% speed-up over the direct38 solver when using the largest reaction mechanism with 7171 species.39

    Keywords: chemical kinetics, sparse analytical Jacobian, Krylov iterative40 methods, preconditioner, ILUT, stiff ODE, combustion, SpeedCHEM41

    1. Introduction42

    The steady increase in computational resources is fostering research of43 cleaner and more efficient combustion processes, whose target is to reduce44 dependence on fossil fuels and to increase environmental sustainability of the45 economies of industrialized countries [1]. Realistic simulations of combus-46 tion phenomena have been made possible by thorough characterization and47 modelling of both the fluid-dynamic and thermal processes that define local48 transport and mixing in high temperature environments, and of the chemical49 reaction pathways that lead to fuel oxidation and pollutant formation [2]. De-50 tailed chemical kinetics modelling of complex and multi-component fuels has51 been achieved through extensive understanding of fundamental hydrocarbon52 chemistry [3] and through the development of semi-automated tools [4] that53 identify possible elementary reactions based on modelling of how molecules54 interact. These detailed reaction models can add thousands of species and55 reactions. However, often simple phenomenological models are preferred56 to even skeletal reaction mechanisms that feature a few tens/hundreds of57 species in “real-world” multidimensional combustion simulations, due to the58 expensive computational requirements that their integration requires, even59 on parallel architectures, when using conventional numerics. The stiffness of60 chemical kinetics ODE systems, caused by the simultaneous co-existence of61 broad temporal scales, is a further complexity factor, as the time integrators62 have to advance the solution using extremely small time steps to guarantee63 reasonable accuracy, or to compute expensive implicit iterations while solv-64 ing the nonlinear systems of equations [5].65



  • Many extremely effective approaches have been developed in the last few67 decades to alleviate the computational cost associated with the integration of68 chemical kinetics ODE systems for combustion applications, targeting: the69 stiffness of the reactive system; the number of species and reactions; the70 overall number of calculations; or the main ordinary differential equations71 (ODE) system dynamics, by identifying main low-dimensional manifolds [6–72 16]. A thorough review of reduction methods is addressed in [2]. From the73 point of view of integration numerics, some recent approaches have shown74 the possibility to reduce computational times by orders of magnitude in com-75 parison with the standard, dense ODE system integration approach, without76 introducing simplifications into the system of ODEs, but instead improving77 the computational performance associated with: 1) efficient evaluation of78 computationally expensive functions, 2) improved numerical ODE solution79 strategy, 3) architecture-tailored coding.80

    As thermodynamics-related and reaction kinetics functions involve expen-81 sive mathematical functions such as exponentials, logarithms and powers, ap-82 proaches that make use of data caching, including in-situ adaptive tabulation83 (ISAT) [17] and optimal-degree interpolation [18], or equation formulations84 that increase sparsity [19], or approximate mathematical libraries [20], can85 significantly reduce the computational time for evaluating both the ODE sys-86 tem function and its Jacobian matrix. As far as the numerics internal to the87 ODE solution strategy, the adoption of sparse matrix algebra in treating the88 Jacobian matrix associated with the chemistry ODE system is acknowledged89 to reduce the computational cost of the integration from the order of the90 cube number of species, O(n3s), down to a linear increase, O(ns) [21–23].91

    Implicit, backward-differentiation formula methods (BDF) are among92 the most effective methods for integrating large chemical kinetics problems93 [24, 25] and packages such as VODE [26], LSODE [27], DASSL [28] are widely94 adopted. Recent interest is being focused on applying different methods that95 make use of Krylov subspace approximations to the solution of the linear sys-96 tem involving the problem’s Jacobian, such as Krylov-enabled BDF methods97 [29], Rosenbrock-Krylov methods [30], exponential methods [31]. As far as98 architecture-tailored coding is concerned, some studies [32, 33] have shown99 that solving chemistry ODE systems on graphical processing units (GPUs)100 can significantly reduce the dollar-per-time cost of the computation. How-101 ever, the potential achieveable per GPU processor is still far from full opti-102 mization, due to the extremely sparse memory access that chemical kinetics103 have in matrix-vector operations, and due to small problem sizes in compari-104


  • son to the available number of processing units. Furthermore, approaches for105 chemical kinetics that make use of standardized multi-architecture languages106 such as OpenCL are still not openly available.107


    Our approach, available in the SpeedCHEM code, a recently developed re-109 search library written in modern Fortran language, addresses both issues110 through the development of new methods for the time integration of the111 chemical kinetics of reactive gaseous mixtures [18]. Efficient numerics are112 coupled with detailed or skeletal reaction mechanisms of arbitrarily large113 size, e.g., ns ≥ 104, to achieve significant speed-ups compared to traditional114 methods especially for the small-medium mechanism sizes, 100 ≤ ns ≤ 500,115 which are of major interest to multi-dimensional simulations. In the code,116 high computational efficiency is achieved by evaluating the functions associ-117 ated with the chemistry ODE system throughout the adoption of optimal-118 degree interpolation of expensive thermodynamic functions, internal sparse119 algebra management of mechanism-related quantities, and sparse analytical120 formulation of the Jacobian matrix. This approach allows a reduction in121 CPU times by almost two orders of magnitude in ignition delay calculations122 using a reaction mechanism with about three thousand species [34], and was123 capable of reducing the total CPU time of practical internal combustion en-124 gine simulations with skeletal reaction mechanisms by almost one order of125 magnitude in comparison with a traditional, dense-algebra-based reference126 approach [35, 36].127 In this paper, we describe the numerics internal to the ODE system solution128 procedure. The flexibility of the software framework allows investigations of129 the optimal solution strategies, by applying different numerical algorithms130 to a numerically consistent, and computationally equally efficient, problem131 formulation. Some known effective and robust