On Numerical Properties of Accelerated MultiplePrecision Implicit Runge-Kutta Methods
Shizuoka Institute of Science and TechnologyTomonori Kouya
http://na-inet.jp/na/birk/
SciCADE2013 in Valladolid, SPAIN2013-09-16(Mon) – 20(Fri)
Abstarct
Abstract
Motivation
IRK method with simplified Newton method
Acceleration of inner iteration and stepsize selection
Performance check by solving linear ODE
Numerical experiments of Evolutionary PDEs
Conclusion and Future work
MotivationBNCpack
▶ provides double and multiple precision numerical algorithmsbased on MPFR/GMP.
▶ has simple explicit and implicit Runge-Kutta (IRK) methodsand extrapolation methods for solving ODEs.
⇓
In SciCADE 2007, a gentleman suggested to us thatKuramoto-Sivashinsky (K-S) equation is suitable for our multipleprecision ODE solvers because of one of chaotic, stiff and largescale examples of ODEs.
⇓
Accelerated Multiple precision IRK methods based on MPFR/GMPare neccesary to solve it.
The Features of Accelerated IRK methods
1. It uses Gauss formula, which is 2m-th order for m stages, Aand P-stable, and symplectic method.
2. Supporting mixed precision iterative refinement method insimplified Newton iteration in IRK process can drasticallyreduce computational time.
3. The parallelization by using OpenMP can be more highlyperformed.
IVP of n dimensional ODE to be solved
{dydt = f(t,y) ∈ Rn
y(t0) = y0
Integration Interval:[t0, α]
(1)
We suppose that this above ODE has the unique solution, soLipschize constant L > 0 exists to be satisfied such as
||f(t,v)− f(t,w)|| ≤ L||v −w|| (2)
for ∀v,w ∈ Rn, ∀t ∈ [t0, α].
⇓
1D Brusselator problem and K-S eq. has large L >> 1, so they arecalled “stiff problems. ”
Skeleton of m stages IRK methods
Discretization: t0, t1 := t0 + h0, ..., tk+1 := tk + hk...
When we calculate the approximation yk+1 ≈ y(tk+1) from theformer yk ≈ y(tk), the following two steps are executed:
(A) Inner iteration: Solve the nonlinear equation for unknownY = [Y1 ... Ym]T ∈ Rmn.
Y1 = yk + hk∑m
j=1 a1jf(tk + cjhk, Yj)...
Ym = yk + hk∑m
j=1 amjf(tk + cjhk, Yj)
⇕F(Y) = 0 (3)
(B) Calculate the next approximation yk+1 with the above Y.
yk+1 := yk + hk
m∑j=1
bjf(tk + cjhk, Yj)
Coefficients of m stages Runge-Kutta method
We use IRK coefficients such as:
c1 a11 · · · a1m...
......
cm am1 · · · amm
b1 · · · bm
=c A
bT (4)
Our IRK solver only uses Gauss formula family which is one of fullyIRK formulas (aij = 0 (i ≤ j)).
Simplified Newton method as inner iteration of IRKmethod
RADAU5 (by Hairer) and SPARK3(by Jay) use simplified Newtonmethod as inner iteration to solve the nonlinear equation (3).
Simplified Newton Method:
Yl+1 := Yl−(Im ⊗ In − hkA⊗ J)−1F(Yl) (5)
where In and Im are n× n and m×m unit matrix respectively, J= ∂f/∂y(tk,yk) ∈ Rn×n is the Jacobi matrix corresponding to f .
⇒ We must solve the following linear equation for each iteration ofsimplified Newton method (5):
(In ⊗ Im − hkA⊗ J)Z = −F(Yl) (6)
and then obtain the solution Z and calculate Yl+1 := Yl + Z.
Why do we select SPARK3 reduction, not RADAU5?RADAU5: Complex Diagonalization of A by Complex SimilarityTransformation Matrix S
(S ⊗ In)(Im ⊗ In − hA⊗ J)(S−1 ⊗ In) = Im ⊗ In − hΛ⊗ J
=
In − hλ1J. . .
In − hλmJ
.
SPARK3: Real Tridiagonalization of A by Real Similarity TransformationMatrix W
X = WTBAW =
1/2 −ζ1
ζ1 0. . .
. . .. . . −ζm−2
ζm−2 0 −ζm−1
ζm−1 0
where W = [wij ] = [Pj−1(ci)] (i, j = 1, 2, ...,m)
ζi =(2√
4i2 − 1)−1
(i = 1, 2, ...,m− 1)
B = diag(b), Im = WTBW = diag(1 1 · · · 1)
Condition numbers of two kinds of similaritytransformation matrices
m 3 5 10 15 20 50
κ∞(S) 22.0 388 3.× 105 3.× 108 2.× 1011 4.× 1028
κ∞(W ) 3.24 6.27 16.4 29.3 44.5 172
▶ RADAU5’s S has larger condition numbers(κ∞(S) = ∥S∥∞∥S−1∥∞) as the number of stages of IRKformulas.
▶ SPARK3’s W condition number (κ∞(W ) = ∥W∥∞∥W−1∥∞)become mildly larger.
=⇒ SPARK3 reduction is the only one selection for many stagesIRK formulas.
SPARK3 Reduction(1/3)
The coefficient matrix of the linear equation for SPARK3 reductionis:
(W TB ⊗ In)(Im ⊗ In − hkA⊗ J)(W ⊗ In)
= Im ⊗ In − hkX ⊗ J =
E1 F1
G1 E2 F2
. . .. . .
. . .
Gm−2 Em−1 Fm−1
Gm−1 Em
where
E1 = In − 1
2hkJ, E2 = · · · = Es = In
Fi = hkζiJ, Gi = −hkζiJ (i = 1, 2, ...,m− 1).
SPARK3 Reduction(2/3)
Jay proposed the left preconditioned matrix P for linear solver sushas:
P =
E1 F1
G1 E2 F2
. . .. . .
. . .
Gm−2 Em−1 Fm−1
Gm−1 Em
≈ Im ⊗ In − hkX ⊗ J
so the preconditioned linear equation to be solved for Z is
P−1(Im ⊗ In − hkX ⊗ J)Z = P−1(W TB ⊗ In)(−F(Y)).
SPARK3 Reduction (3/3)We use LU decomposed P such as
P =
In
G1H−11 In
. . .. . .
Gm−2H−1m−2 In
Gm−1H−1m−1 In
×
H1 F1
H2 F2
. . .. . .
Hm−1 Fm−1
Hm
where
Hi := In − (2(2i− 1))−1hJ (i = 1, 2, ...,m).
(cf.) ”A Parallelizable Preconditioner for the Iterative Solution of Implicit Runge-Kutta-type Methods”, Journal of
Computational and Applied Mathematics 111 (1999) P.63-76
Mixed precision iterative refinement method
Mixed precision iterative refinement method is to reducecomputational cost by combining short S digits arithmetic andlong L digits arithmetic (S << L).
The linear equation to be solved: Cx = d , C ∈ RN×N , d,x ∈ RN
=⇒
(L) Solve Cx0 = d for x0.
For ν = 0, 1, 2, ...
(L) rν := d− Cxν
(S) r′ν := rν/∥rν∥(S) Solve Cz = r′ν for z.(L) xν+1 := xν + ∥rν∥z
Check convergence.
=⇒ x := xνstop
(cf.) Buttari, Alfredo, et al. International Journal of High Performance
Computing Applications 21.4 (2007): 457-466.
The whole algorithm of accelerated IRK methodInitial guess: Y−1 ∈ Rmn
For l = 0, 1, 2, ... Simplified Newton iteration
(1) Yl := [Y(l)1 Y
(l)2 ... Y
(l)m ]T
(2) C := Im ⊗ In − hkX ⊗ J , Compute ||C||F(3) d := (W TB ⊗ In)(−F(Yl))(4) Solve Cx0 = d for x0 (S)
For ν = 0, 1, 2, ... Mixed precision iterative refinement
(5) rν := d− Cxν
(6-1) r′ν := rν/||rν || (S)(6-2) Solve Cz = r′ν for z (S)(6-3) xν+1 := xν + ||rν ||z(6-4) Check convergence ⇒ xνstop
(7) Yl+1 := Yl + (W ⊗ In)xνstop
Check convergence ⇒ Ylstop
Y := Ylstop = [Y1 Y2 ... Ym]T
yk+1 := yk + hk∑m
j=1 bjf(tk + cjhk, Yj)
Computational environment
H/W Intel Core i7 3820 (4 cores) 3.6GHz + 64GB RAM
OS Scientific Linux 6.3 x86 64
S/W Intel C++ 13.0.1, MPFR 3.1.1/GMP 5.1.1,BNCpack 0.8
▶ OpenMP in Intel C++ standard.
▶ Block Parallelization for capable parts of IRK methods.
▶ Except left preconditioning and direct method.
Performance check by solving 128th dimentional constantlinear ODE (50 decimal digits)
1.E-38
1.E-34
1.E-30
1.E-26
1.E-22
1.E-18
1.E-14
1.E-10
1.E-06
1.E-02
1.E+02
1.E+06
1.E+10
1.E+14
0
200
400
600
800
1000
1200
1400
1600
3 4 5 6 7 8 9 10 11 12
Relative ErrorComp.Time (s)
m
Iter.Ref-DM W-Trans. W-Iter.Ref-MM W-Iter.Ref-DM Max.Rel.Err
Iter.Ref-DM No reduction + quasi-Newton + Double Precision (DP) -Multiple Precision (MP) mixed precision iterativerefinement method (based on direct method)
W-Trans. SPARK3 reduction + MP direct methodW-Iter.Ref-MM SPARK3 + MP(S = L/2)-MP iterative refinementW-Iter.Ref-DM SPARK3 + DP-MP iterative refinement
Stepsize selection by embedded formula (1/2)
Embedded formula for IRK methods (by Hairer): The followingm+ 1 stages IRK formula for given contant γ0:
0 0 0T
c 0 A
γ0 bT
In order to extend A stable area, we select γ0 = 1/8 whereb= [b1 · · · bm]T is obtained by solving the following linear equationto be satisfied in simplified assumption B(m):
1 · · · 1c1 · · · cm...
...
cm−11 · · · cm−1
m
b1b2...
bm
=
1− γ01/2...
1/m
.
Stepsize selection by embedded formula (2/2)
By using this embedded formula , we can get yk+1 as following:
yk+1 := yk + hkγ0f(tk,yk) + hk
m∑j=1
bjf(tk + cjhk, Yj).
And we use the yk+1 for the following local error estimator errk.
||errk|| =
√√√√ 1
n
n∑i=1
(|y(k+1)
i − y(k+1)i |
ATOL+RTOLmax(|y(k)i |, |y(k+1)i |)
)2
where ATOL is set as absolute tolerance and RTOL as relativetolerance given by users.This estimator is used in next stepsize hk+1 prediction as following:
hk+1 := 0.9||errk||m+1hk
Numerical experiments of Evolutionary PDEs
▶ 1D Brusselator Problem (omit!){∂u∂t = 1 + u2v − 4 + 0.02 · ∂2u
∂x2
∂v∂t = 3u− u2v + 0.02 · ∂2v
∂x2
(7)
▶ 1D Kuramoto-Sivashinsky (K-S) equation
∂U
∂t= −∂2U
∂x2− ∂4U
∂x4− 1
2
∂U2
∂x(8)
1D Kuramoto-Sivashinsky Equation: Discretizationmethod(1/2)cf. Hairer & Wanner, Solving ODE II, Chap. IV, pp.148 - 149.
∂U
∂t= −∂2U
∂x2− ∂4U
∂x4− 1
2
∂U2
∂x
Periodic boundary condition: U(x+ L, t) = U(x, t)
Initial value: U(x, 0) = 16max(0,
min(x/L, 0.1− x/L),
20(x/L− 0.2)(0.3− x/L),
min(x/L− 0.6, 0.7− x/L),
min(x/L− 0.9, 1− x/L))
Parameters: L = 2π/q, q = 0.025
1D Kuramoto-Sivashinsky Equation: Discretizationmethod(2/2)
⇓ Discretization by using pseudospectral method
Uj(t) =1
L
∫ L
0U(x, t) exp(−iqjx)dx
U(x, t) =∑j∈Z
Uj(t) exp(iqjx)
dUj
dt= ((qj)2 − (qj)4)Uj −
iqj
2(U · U)j (j ∈ Z)
⇓ Truncating at N = 1024, we make ODE for y(t) = {yj(t)}dyjdt
= ((qj)2−(qj)4)yj−iqj
2FN (F−1
N y·F−1N y) (j = 1, 2, ..., N/2−1)
where FN , F−1N means FFT and inverse FFT, respectively.
⋆ Mutiple precision real FFT and inverse real FFT routines areoriginated by Ooura’s double precision C routines.http://www.kurims.kyoto-u.ac.jp/~ooura/fftman/ftmn2_12.htm.
K-S eq. : Numerical values by Multiple precision andRADAU5(Double precision)
K-S eq. : Relative Errors of RADAU5 (Double precision)
K-S eq.: Computational Times by using variable #stagesIRK formulas
4 threads, 80 stages formulas in 100 decimal digits as the truesolution, and t = 10
80 dec.digits RTOL = ATOL = 10−60
# stages(m) 20 30 40 50
Comp.Time(s) 130165.4 160601.8 133541.0 190131.4# steps 6911 2667 1103 856
Average (s) 18.8 60.2 121.1 222.1Max.Rel.Error 4.2E-38 2.7E-38 1.4E-38 2.0E-36Min.Rel.Error 1.1E-54 1.8E-50 1.7E-52 1.0E-63
RTOL = ATOL = 10−70
# stages(m) 20 30 40 50
Comp.Time(s) 100695.2 86331.4 137232.9 200454.8# steps 6978 1738 1175 918
Average (s) 14.4 49.7 116.8 218.4Max.Rel.Error 4.4E-48 1.9E-49 2.4E-47 5.1E-47Min.Rel.Error 2.7E-68 5.9E-68 1.3E-62 1.2E-68
Conclusion
▶ We can implement the accelerated multiple precision IRKmethods with DP-MP mixed precision iterative refinementmethod and SPARK3 reduction in inner simplified Newtoniteration.
▶ Parallelization can reduce the computational cost.
▶ Our implemented ODE solver is available for solving complexevolutionary PDEs such as Brusselator problem or 1DKuramoto-Sivashinsky equation.
Future work
We have the following plans to:
1. Seek higher performance ODE solver in massively parallelcomputation environment such as GPGPU or Intel MIC.
2. Implement stable double precision linear solvers such asGMRES(m) or other stable Krylov subspace methods.
3. Solve many other problems by our ODE solver.
A part of our implemented ODE solver is published asBIRK(extented Bncpack for Implicit Runge-Kutta methods) in ourWeb site.
http://na-inet.jp/na/birk/