Deterministic and stochastic acceleration techniques for RichardsonDeterministic and stochastic acceleration techniques for RichardsonDeterministic and stochastic acceleration techniques for RichardsonDeterministic and stochastic acceleration techniques for RichardsonDeterministic and stochastic acceleration techniques for RichardsonMassimiliano Lupo Pasini1, Michele Benzi1, Thomas M. Evans2, Steven P. HamiltonMassimiliano Lupo Pasini1, Michele Benzi1, Thomas M. Evans2, Steven P. HamiltonMassimiliano Lupo Pasini , Michele Benzi , Thomas M. Evans , Steven P. Hamilton

1Emory College, Department of Mathematics and Computer Science, 400 1Emory College, Department of Mathematics and Computer Science, 400 1Emory College, Department of Mathematics and Computer Science, 400 Emory College, Department of Mathematics and Computer Science, 400 2Oak Ridge National Laboratory, 1 Bethel2Oak Ridge National Laboratory, 1 Bethel2Oak Ridge National Laboratory, 1 Bethel

3Georgia Institute of Technology, School of Civil and Environmental Engineering, 790 Atlantic Drive NW, Atlanta, GA 303323Georgia Institute of Technology, School of Civil and Environmental Engineering, 790 Atlantic Drive NW, Atlanta, GA 30332Georgia Institute of Technology, School of Civil and Environmental Engineering, 790 Atlantic Drive NW, Atlanta, GA 30332

IntroductionIntroduction Alternating AndersonIntroduction Alternating AndersonIntroduction Alternating AndersonAAR uses approximate solutions computed• Scientific computing is moving to exascale AAR uses approximate solutions computed• Scientific computing is moving to exascale AAR uses approximate solutions computedRichardson’s steps to build the matrix• Applications require high level of concurrency Richardson’s steps to build the matrix• Applications require high level of concurrency Richardson’s steps to build the matrix• Applications require high level of concurrency

• Next generation computers will exhibit more hardware failures • Next generation computers will exhibit more hardware failures • Next generation computers will exhibit more hardware failures – applications must be resilient– applications must be resilient The Anderson mixing is defined as– applications must be resilient The Anderson mixing is defined as

• Standard Krylov subspace methods struggle to The Anderson mixing is defined as

• Standard Krylov subspace methods struggle to simultaneously obtain efficiency and concurrencysimultaneously obtain efficiency and concurrencysimultaneously obtain efficiency and concurrency

• Standard Krylov methods struggle to achieve resilience The vector is chosen so as to minimize• Standard Krylov methods struggle to achieve resilience The vector is chosen so as to minimize• Standard Krylov methods struggle to achieve resilience • Richardson’s schemes benefit computational and data locality • Multiple Richardson’s steps without optimization benefit • Richardson’s schemes benefit computational and data locality • Multiple Richardson’s steps without optimization benefit • Richardson’s schemes benefit computational and data locality

computational and data localitycomputational and data localitycomputational and data locality• Convergence on positive definite • Convergence on positive definite

Mathematical framework• Convergence on positive definite

Mathematical frameworkMathematical framework• Reformulate the sparse linear system of interest• Reformulate the sparse linear system of interest• Reformulate the sparse linear system of interest

as a fixed point schemeas a fixed point schemeas a fixed point scheme

Conclusions and future such that the Neumann series recasts the solution as Conclusions and future such that the Neumann series recasts the solution as Conclusions and future such that the Neumann series recasts the solution as

• Identified classes of matrices for which convergence is (1)

• Identified classes of matrices for which convergence is (1) guaranteed(1) guaranteedguaranteed

• Competitive performance compared to restarted GMRES for • Competitive performance compared to restarted GMRES for • One level fixed point schemes are renown for their deteriorated

• Competitive performance compared to restarted GMRES for different choices of preconditioner• One level fixed point schemes are renown for their deteriorated different choices of preconditioner• One level fixed point schemes are renown for their deteriorated

asymptotic convergence ratedifferent choices of preconditioner

asymptotic convergence rate Convergence analysis on specific classes of problemsasymptotic convergence rate• Multilevel schemes can accelerate convergence

Convergence analysis on specific classes of problems• Multilevel schemes can accelerate convergence Performance assessment at extreme• Multilevel schemes can accelerate convergence Performance assessment at extreme• Purely deterministic algorithms struggle to deal with inherently

Performance assessment at extreme Use of GPU accelerations• Purely deterministic algorithms struggle to deal with inherently Use of GPU accelerations

random faulty phenomena References Use of GPU accelerations

random faulty phenomena Referencesrandom faulty phenomena ReferencesReferences• “Anderson acceleration of the Jacobi iterative method: an efficientOne level relaxation scheme • “Anderson acceleration of the Jacobi iterative method: an efficientOne level relaxation scheme alternative to Krylov methods for large, sparse alternative to Krylov methods for large, sparse P. P. Pratapa, P. Suryanarayana, and J. E. Pask P. P. Pratapa, P. Suryanarayana, and J. E. Pask P. P. Pratapa, P. Suryanarayana, and J. E. Pask J. Comput. Phys., 306, pp. 4354, 2016.J. Comput. Phys., 306, pp. 4354, 2016.• “Converge analysis of Anderson-type accelerationDeterministic accelerations Stochastic accelerations • “Converge analysis of Anderson-type accelerationDeterministic accelerations

improve convergenceStochastic accelerations

enhance resilience• “Converge analysis of Anderson-type accelerationiteration”, M. Lupo Pasini – In preparationimprove convergence enhance resilience iteration”, M. Lupo Pasini – In preparationiteration”, M. Lupo Pasini – In preparation

Deterministic and stochastic acceleration techniques for Richardson-type iterationsDeterministic and stochastic acceleration techniques for Richardson-type iterationsDeterministic and stochastic acceleration techniques for Richardson-type iterationsDeterministic and stochastic acceleration techniques for Richardson-type iterationsDeterministic and stochastic acceleration techniques for Richardson-type iterations, Steven P. Hamilton2, Stuart R. Slattery2, Phanish Suryanarayana3, Steven P. Hamilton2, Stuart R. Slattery2, Phanish Suryanarayana3, Steven P. Hamilton , Stuart R. Slattery , Phanish Suryanarayana

Department of Mathematics and Computer Science, 400 Dowman Drive, Atlanta, GA 30322Department of Mathematics and Computer Science, 400 Dowman Drive, Atlanta, GA 30322Department of Mathematics and Computer Science, 400 Dowman Drive, Atlanta, GA 30322Department of Mathematics and Computer Science, 400 Dowman Drive, Atlanta, GA 30322Bethel Valley Rd, Oak Ridge, TN 37830Bethel Valley Rd, Oak Ridge, TN 37830Bethel Valley Rd, Oak Ridge, TN 37830

Georgia Institute of Technology, School of Civil and Environmental Engineering, 790 Atlantic Drive NW, Atlanta, GA 30332Georgia Institute of Technology, School of Civil and Environmental Engineering, 790 Atlantic Drive NW, Atlanta, GA 30332Georgia Institute of Technology, School of Civil and Environmental Engineering, 790 Atlantic Drive NW, Atlanta, GA 30332

Anderson-Richardson (AAR) Monte Carlo Linear Solvers (MCLS)Anderson-Richardson (AAR) Monte Carlo Linear Solvers (MCLS)Anderson-Richardson (AAR) Monte Carlo Linear Solvers (MCLS)MCLS use random walks defined on a transition matrix P and a computed by successive MCLS use random walks defined on a transition matrix P and a computed by successive MCLS use random walks defined on a transition matrix P and a sequence of weights w = H / P , the statistical estimator

computed by successive matrix sequence of weights wi,j = Hi,j / Pi,j , the statistical estimator matrix sequence of weights wi,j = Hi,j / Pi,j , the statistical estimator

redefines (1) asmatrix

redefines (1) asredefines (1) as

asasas

• Convergence guaranteed for strictly diagonally dominant, M • Convergence guaranteed for strictly diagonally dominant, M • Convergence guaranteed for strictly diagonally dominant, M matrices, generalized diagonally dominant matricesminimize the residual norm. matrices, generalized diagonally dominant matricesminimize the residual norm. matrices, generalized diagonally dominant matrices

• Choice of preconditioners affects convergenceMultiple Richardson’s steps without optimization benefit • Choice of preconditioners affects convergenceMultiple Richardson’s steps without optimization benefit • Choice of preconditioners affects convergence• We use adaptive methods to select the number of histories• We use adaptive methods to select the number of histories

positive definite matrices guaranteed Reference solution Tolerance = 0.5positive definite matrices guaranteed Reference solution Tolerance = 0.5positive definite matrices guaranteed

Tolerance = 0.1 Tolerance = 0.01Tolerance = 0.1 Tolerance = 0.01

Conclusions and future developmentsand future developments Conclusions and future developmentsand future developments Conclusions and future developmentsand future developments• Identified classes of matrices and preconditioners for which Identified classes of matrices for which convergence is • Identified classes of matrices and preconditioners for which Identified classes of matrices for which convergence is

MCLS are guaranteed to converge a prioriMCLS are guaranteed to converge a prioriMCLS are guaranteed to converge a priori• Difficulty in ensuring a priori convergence of MCLS for a general Competitive performance compared to restarted GMRES for • Difficulty in ensuring a priori convergence of MCLS for a general Competitive performance compared to restarted GMRES for • Difficulty in ensuring a priori convergence of MCLS for a general

problemCompetitive performance compared to restarted GMRES for

preconditioner problempreconditioner problempreconditioner Algorithm scalability has not yet been analyzed Convergence analysis on specific classes of problems Algorithm scalability has not yet been analyzed Convergence analysis on specific classes of problems Testing MCLS on large parallel architectures and evaluating extreme scale Testing MCLS on large parallel architectures and evaluating extreme scale Testing MCLS on large parallel architectures and evaluating

their resilience in presence of faults still to be addressedextreme scale

their resilience in presence of faults still to be addressedtheir resilience in presence of faults still to be addressed

ReferencesReferences ReferencesReferences ReferencesReferences ReferencesReferences• “A Monte Carlo synthetic-acceleration method for solving the thermalacceleration of the Jacobi iterative method: an efficient • “A Monte Carlo synthetic-acceleration method for solving the thermalacceleration of the Jacobi iterative method: an efficientradiation diffusion equation“ . T. M Evans, S. W. Mosher, S. R. Slattery, S. , sparse linear systems”, radiation diffusion equation“ . T. M Evans, S. W. Mosher, S. R. Slattery, S. P. Hamilton – J. Comp. Phys., Vol. 258, 2014, pp. 338-358.

, sparse linear systems”, P. P. Pratapa, P. Suryanarayana, and J. E. Pask – P. Hamilton – J. Comp. Phys., Vol. 258, 2014, pp. 338-358.P. P. Pratapa, P. Suryanarayana, and J. E. Pask – P. Hamilton – J. Comp. Phys., Vol. 258, 2014, pp. 338-358.

• “Analysis of Monte Carlo Accelerated Iterative Methods for Sparse Linear P. P. Pratapa, P. Suryanarayana, and J. E. Pask –

• “Analysis of Monte Carlo Accelerated Iterative Methods for Sparse Linear Systems”. M. Benzi, T. M. Evans, S. P. Hamilton, M. Lupo Pasini, S. R. acceleration for Richardson’s Systems”. M. Benzi, T. M. Evans, S. P. Hamilton, M. Lupo Pasini, S. R. acceleration for Richardson’s Systems”. M. Benzi, T. M. Evans, S. P. Hamilton, M. Lupo Pasini, S. R. Slattery – Numer. Linear Algebra Appl. Vol. 24, Issue 3, 2017.

acceleration for Richardson’s preparation. Slattery – Numer. Linear Algebra Appl. Vol. 24, Issue 3, 2017. preparation. Slattery – Numer. Linear Algebra Appl. Vol. 24, Issue 3, 2017. preparation.