Applied Linear Algebra - in honor of Hans SchneiderMay 25, 2010 A Look-Back Technique of Restart for the GMRES(m) Method Akira IMAKURA † Tomohiro SOGABE

Applied Linear Algebra - in honor of Hans SchneiderApplied Linear Algebra - in honor of Hans SchneiderMay 25, 2010May 25, 2010

A Look-Back Technique of Restart A Look-Back Technique of Restart for the GMRES(for the GMRES(mm) Method) Method

Akira IMAKURAAkira IMAKURA†† Tomohiro SOGABE Tomohiro SOGABE‡‡ Shao-Liang ZHANG Shao-Liang ZHANG† †

††Nagoya University, Japan.Nagoya University, Japan.‡‡Aichi Prefectural University, Japan. Aichi Prefectural University, Japan.

11/29/29

May 25, 2010May 25, 2010 Applied Linear Algebra - in honor of Hans SchneiderApplied Linear Algebra - in honor of Hans Schneider

IntroductionIntroduction

Main subject-- Nonsymmetric linear systems of the form

Krylov subspace methods-- Symmetric linear systems　　　 Based on Lanczos algorithm

: CG, MINRES, …

-- Nonsymmetric linear systems　　　 Based on two-sided Lanczos algorithm

: Bi-CG, Bi-CGSTAB, GPBi-CG…

　　　 Based on Arnoldi algorithm: GCR, GMRES, …

＋ restart → GMRES(m) method

22/29/29



GMRES(m) method [Y. Saad and M. H. Schultz:1986]

-- Algorithm (focus on restart)

Algorithm : GMRES(m)

Run m iterations of GMRESInput: , Output:

-- Update the initial guess

l : number of restart cycle

33/29/29

Motivation

1.0

0.0



GMRES(m) method [Y. Saad and M. H. Schultz:1986]

-- Residual polynomials

: residual polynomial

l : number of restart

1.0

0.0

44/29/29



GMRES(m) method with readjustment-- Numerical result compared with GMRES(m) method

: parameters

CAVITY05 XENON1

― GMRES(m) ― GMRES(m) with readjustment

Readjustment leads to be better convergenceReadjustment leads to be better convergence

Question

How does we readjust the function s.t. the root is moved?

55/29/29



Look-Back GMRES(m) method-- Algorithm (focus on update the initial guess)

Algorithm : Look-Back GMRES(m)


We can simply achieve the readjustmentWe can simply achieve the readjustment

66/29/29



Look-Back GMRES(m) method-- Extension of the GMRES(m) method

-- A Look-Back technique of restart

→ Analyze based on error equations

→ Analyze based on residual polynomials

77/29/29


Extension of the GMRES(Extension of the GMRES(mm) method) method-- Analysis based on error equation ---- Analysis based on error equation --

88/29/29


Extension of the GMRES(Extension of the GMRES(mm) method) methodAlgorithm : GMRES(m)



Algorithm : Extension of GMRES(m)

?

99/29/29


Extension of the GMRES(Extension of the GMRES(mm) method) method

Analysis based on error equations-- Introduction of error eq. and iterative refinement scheme

Definition : error equation

Let and be the exact solution and the numerical solution respectively. Then the error vector can be computed by solving the so-called error equation, i.e.,

where is residual vector corresponding to .

Definition : iterative refinement scheme

The technique based on solving error equations recursively to achieve the higher accuracy of the numerical solution is called the iterative refinement scheme.

1010/29/29


Extension of the GMRES(Extension of the GMRES(mm) method) methodAlgorithm : GMRES(m)



Algorithm : Extension of GMRES(m)

Algorithm : Iterative refinement


Mathematically equivalent Mathematically equivalent

Natural extension


Algorithm : Iterative refinement

1111/29/29


A Look-Back technique A Look-Back technique of restartof restart

-- Analysis based on residual polynomials ---- Analysis based on residual polynomials --

1212/29/29

Extension of GMRES(m) method

GMRES(m) method


A Look-Back technique of restartA Look-Back technique of restart

Difference between GMRES(m) and its Extension

l : number of restart residual polynomial

If we set . Then the rational function s.t.

is exist.

1313/29/29

l : number of restart


Look-Back GMRES(m) method

GMRES(m) method



Difference between GMRES(m) and its Extension

Set by some techniqueLook-Back technique

residual polynomial

1414/29/29





A Look-Back technique

Set by Look-Back technique

Look-Back strategy

→ Readjust s.t. root is moved

Look-Back at the past polynomials and rational functions

1515/29/29






Motivation

1.0

0.0

Root of the function is moved( 　　→　　 )

It is expected that readjustment leads to be high convergence

1616/29/29






e.g.

1717/29/29



Proposal of Look-Back GMRES(m) method -- Algorithm (focus on update the initial guess)


-- extra costs for Look-Back technique: 1 matrix-vector multiplication per 1 restart

Algorithm : Look-Back GMRES(m)

1818/29/29


Numerical experimentsNumerical experiments-- Comparison with GMRES(m) ---- Comparison with GMRES(m) --

1919/29/29


Numerical experimentsNumerical experiments

Compared methods (without preconditioner)-- GMRES(m) method-- Look-Back GMRES(m) method

Parameters-- right-hand-side :-- initial guess :-- stopping criterion :

Experimental conditions-- AMD Phenom II X4 940 (3.0GHz);-- Standard Fortran 77 using double precision.

Test problems [obtained from UF Sparse Matrix Collection]-- CAVITY05, CAVITY16, CHIPCOOL0,

MEMPLUS, NS3DA, RAJAT03,

RDB5000, XENON1, XENON2.

(m = 30, 100)

2020/29/29



Numerical results for m = 30CAVITY05 CAVITY16 CHIPCOOL0

MEMPLUS NS3DA RAJAT03

RDB5000 XENON1 XENON2

― GMRES(m) ― Look-Back GMRES(m)

2121/29/29



Numerical results for m = 30CAVITY05 CAVITY16 CHIPCOOL0



Time [sec.]

Total 1 Restart

Time [sec.]

Total 1 Restart

Time [sec.]

Total 1 Restart

Time [sec.]

Total 1 Restart

Time [sec.]

Total 1 Restart

Time [sec.]

Total 1 Restart

Time [sec.]

Total 1 Restart

Time [sec.]

Total 1 Restart

Time [sec.]

Total 1 Restart

1.86 E-02

1.87 E-02

7.30 E-02

7.33 E-02

5.16 E-02

5.13 E-02

2.30 E-01

2.35 E-01

1.93 E-01

1.93 E-01

1.29 E-02

1.29 E-02

2.44 E-01

2.51 E-01

8.36 E-01

8.39 E-01

2.94 E+00

1.25 E+00

†

3.15 E+01

†

7.83 E+01

1.14 E+01

3.93 E+00

1.78E +01

1.84 E+01

†

9.16 E+00

4.00 E-01

4.42 E-01

9.47 E+01

1.57 E+01

4.38 E+02

6.64 E+01

4.43 E-03

4.57 E-03

― GMRES(m) ― Look-Back GMRES(m)

2222/29/29



Numerical results for m = 100― GMRES(m) ― Look-Back GMRES(m)

CAVITY05 CAVITY16 CHIPCOOL0



2323/29/29



Numerical results for m = 100― GMRES(m) ― Look-Back GMRES(m)

Time [sec.]

Total 1 Restart

† 1.23 E-01

2.38 E+01

1.24 E-01

Time [sec.]

Total 1 Restart

† 5.47 E-01

1.28 E+02

5.52 E-01

Time [sec.]

Total 1 Restart

1.35 E+01

4.49 E-01

7.78 E+00

4.50 E-01

Time [sec.]

Total 1 Restart

2.11 E+01

1.07 E+00

2.11 E+01

1.08 E+00

Time [sec.]

Total 1 Restart

† 1.71 E-01

8.56 E+00

1.71 E-01

Time [sec.]

Total 1 Restart

4.18 E-01 1.10 E-01

3.93 E-01 1.10 E-01

Time [sec.]

Total 1 Restart

6.41 E+01

1.65 E+00

3.11 E+01

1.63 E+00

Time [sec.]

Total 1 Restart

2.70 E+02

5.49 E+00

1.23 E+02

5.43 E+00

Time [sec.]

Total 1 Restart

2.65 E+00

3.09 E-02

1.15 E+00

3.09 E-02

CAVITY05 CAVITY16 CHIPCOOL0



2424/29/29



Test problems-- From discretization of partial differential equation of the form

over the unit square .

2525/29/29



Compared methods (with ILU(0) preconditioner)-- GMRES(m) method-- Look-Back GMRES(m) method

Parameters-- initial guess :-- stopping criterion :

Experimental conditions-- AMD Phenom II X4 940 (3.0GHz);-- Standard Fortran 77 using double precision.

(m = 10)

2626/29/29



Numerical results ― GMRES(m) ― Look-Back GMRES(m)

α = 0 α = 0.5

α = 1.0 α = 2.0

2727/29/29


Conclusion and Future worksConclusion and Future works

2828/29/29


Conclusion and Future worksConclusion and Future works

Conclusion-- In this talk, from analysis based on residual polynomials, we proposed the Look-Back GMRES(m) method.

-- From our numerical experiments, we learned that the Look- Back GMRES(m) method shows a good convergence than the GMRES(m) method in many cases.

-- Therefore the Look-Back GMRES(m) method will be an efficient variant of the GMRES(m) method.

Future works-- Analyze details of the Look-Back technique.-- Compare with other techniques for the GMRES(m) method.

2929/29/29

Thank you for your kind attentionThank you for your kind attention



3131/29/29

Documents

Applied Linear Algebra - in honor of Hans SchneiderMay 25, 2010 A Look-Back Technique of Restart for the GMRES(m) Method Akira IMAKURA † Tomohiro SOGABE