Upload
others
View
22
Download
0
Embed Size (px)
Citation preview
Journal of Computational Mathematics
Vol.34, No.3, 2016, 300–316.
http://www.global-sci.org/jcm
doi:10.4208/jcm.1511-m2015-0299
POSITIVE DEFINITE AND SEMI-DEFINITE SPLITTINGMETHODS FOR NON-HERMITIAN POSITIVE DEFINITE
LINEAR SYSTEMS*
Na Huang and Changfeng Ma
School of Mathematics and Computer Science, Fujian Normal University, Fuzhou 350117, China
Email: [email protected]
Abstract
In this paper, we further generalize the technique for constructing the normal (or pos-
itive definite) and skew-Hermitian splitting iteration method for solving large sparse non-
Hermitian positive definite system of linear equations. By introducing a new splitting, we
establish a class of efficient iteration methods, called positive definite and semi-definite
splitting (PPS) methods, and prove that the sequence produced by the PPS method con-
verges unconditionally to the unique solution of the system. Moreover, we propose two
kinds of typical practical choices of the PPS method and study the upper bound of the
spectral radius of the iteration matrix. In addition, we show the optimal parameters such
that the spectral radius achieves the minimum under certain conditions. Finally, some
numerical examples are given to demonstrate the effectiveness of the considered methods.
Mathematics subject classification: 65F10, 65F30, 65F50.
Key words: Linear systems, Splitting method, Non-Hermitian matrix, Positive definite
matrix, Positive semi-definite matrix, Convergence analysis.
1. Introduction
Many problems in scientific computing give rise to a system of linear equations
Ax = b, A ∈ Cn×n, and x, b ∈ Cn, (1.1)
with A being a large sparse non-Hermitian but positive definite matrix.
We call a matrix B positive definite (or positive semi-definite), if B + B∗ is Hermitian
positive definite (or positive semi-definite), i.e., for all 0 = x ∈ Cn, x∗(B + B∗)x > 0 (or
x∗(B + B∗)x ≥ 0), where B∗ denotes the complex conjugate transpose of the B. Let D =
diag(a11, a22, · · · , ann) be the diagonal part of A and ei = (0, · · · , 0, 1, 0, · · · , 0)T . Since the
coefficient matrix A is positive definite, we have e∗i (A+A∗)ei = aii + aii > 0. This shows that
D is positive definite.
The linear system has many important practical applications, such as diffuse optical tomog-
raphy, molecular scattering, lattice quantum chromodynamics (see, e.g., [1,7,8,22,24,38,39,41]).
Many researchers have been devoted themselves to the numerical solution of (1.1) (see e.g.,
[2–4,10,11,18,21,25,27,28,36,37,40,42,45–47] and the references therein) and proposed kinds
of available iteration methods for solving the system (1.1), in which splitting iteration methods
(see e.g., [9,13–17,19,29–31,35,44]) and Krylov subspace methods (see e.g., [5,20,23,26,32,43])
* Received May 25, 2015 / Revised version received October 13, 2015 / Accepted November 4, 2015 /
Published online May 3, 2016 /
Splitting Methods for Non-Hermitian Positive Definite Linear Systems 301
attract a lot of attention. In [16], the authors presented a Hermitian and skew-Hermitian s-
plitting (HSS) iteration method for solving (1.1) and showed that the HSS method converges
unconditionally to the unique solution of the system. Then many researchers focused on the
HSS method and proposed kinds of iterations method based on Hermitian and skew-Hermitian
splitting (see e.g., [6, 9, 12, 13, 17]). In recent years, other kinds of splitting iteration methods
have also been studied (see e.g., [14,15,33–35,44]). Normal and skew-Hermitian splitting (NSS)
iteration methods for solving large sparse non-Hermitian positive definite linear system was
studied in [15]. Based on block triangular and skew-Hermitian splitting, a class of iteration
methods for solving positive-definite linear systems was established in [14]. Krukier et. al pro-
posed the generalized skew-Hermitian triangular splitting iteration methods to solve (1.1) and
applied the methods to solve the saddle-point linear systems (see [34]). In this work, we further
generalize the technique for constructing the normal (or positive definite) and skew-Hermitian
splitting iteration method for solving (1.1).
Throughout this paper, we use the following notations: Cm×n is the set of m× n complex
matrices and Cm = Cm×1. We use C and R to denote the set of complex numbers and real
numbers, respectively. For any a ∈ C, we write Re(a) and Im(a) to denote the real and
imaginary parts of a. For B ∈ Cn×n, we write B−1, ∥B∥2, Λ(B) and ρ(B) to denote the the
inverse, 2-norm, the spectrum and the spectral radius of the matrix B, respectively. I denotes
the identity matrix of size implied by context. i =√−1 denotes the imaginary unit.
The organization of this paper is as follows. In Section 2, we present the positive definite
and semi-definite splitting methods for solving non-hermitian positive definite linear systems
and study the convergence properties of the PPS iteration. In Section 3, we establish two kinds
of typical practical choices of the PPS method and study the upper bound of the spectral radius
of iteration matrix. Numerical experiments are presented in Section 4 to show the effectiveness
of our methods.
2. The Positive Definite and Semi-definite Splitting Method
In this section, we study efficient iterative methods for solving (1.1) based on the positive
definite and semi-definite splitting (PPS for short) of the coefficient matrix A, and establish
the convergence analysis of the new methods. For this purpose, we split A into positive-definite
and positive semi-definite parts as follows:
A =M +N, (2.1)
where M is a positive-definite matrix and N is a positive semi-definite matrix. Then it is easy
to see that
A = αI +M − (αI −N) = αI +N − (αI −M).
This implies that the system (1.1) can be reformulated equivalently as:
(αI +M)x = (αI −N)x+ b,
or
(αI +N)x = (αI −M)x+ b.
By the two above fixed point equations, we can get the following iteration method:(αI +M)x(k+
12 ) = (αI −N)x(k) + b,
(αI +N)x(k+1) = (αI −M)x(k+12 ) + b.
(2.2)
302 N. HUANG AND C.F. MA
For convenience, we call the scheme (2.2) positive-definite and semi-definite splitting (PPS for
short) iteration method. It is worth mentioning that the normal and skew-Hermitian splitting
iteration method proposed in [15] and the positive definite and skew-Hermitian splitting itera-
tion method proposed in [14] are both the special cases of the PPS iteration method. Actually,
if A =M +N with N being a skew-Hermitian matrix, then we can see that
M +M∗ = A−N + (A−N)∗ = A+A∗,
which shows that M is positive definite. This, along with the fact that all the skew-Hermitian
matrix are positive semi-definite, follows that both of the two splitting mentioned above belong
to the positive-definite and semi-definite splitting.
In the following, we shall establish the convergence theorem of the PPS method. To this
end, we rewrite the scheme (2.2) equivalently. By using (2.2), we can obtain
x(k+12 ) = (αI +M)−1(αI −N)x(k) + (αI +M)−1b, (2.3)
and
x(k+1) = (αI +N)−1(αI −M)x(k+12 ) + (αI +N)−1b, (2.4)
which follows that
x(k+1) = S(α)x(k) + T (α)b, (2.5)
where
S(α) = (αI +N)−1(αI −M)(αI +M)−1(αI −N), (2.6a)
T (α) = 2α[(αI +M)(αI +N)]−1. (2.6b)
Therefore, the PPS method is convergent if and only if the spectral radius of the iterative
matrix S(α) is less than 1. To this purpose, we present the following lemma.
Lemma 2.1. For any positive semi-definite matrix P ∈ Cn×n, it holds that
∥(αI − P )(αI + P )−1∥2 ≤ 1, ∀α > 0.
Furthermore, if P is positive definite, then it holds that
∥(αI − P )(αI + P )−1∥2 < 1, ∀α > 0.
Proof. Let σ(α) =: ∥(αI − P )(αI + P )−1∥2. By the definition of 2-norm and the similarity
invariance of the matrix spectrum, we have
σ(α)2 = ∥(αI − P )(αI + P )−1∥22= ρ
((αI + P ∗)−1(αI − P ∗)(αI − P )(αI + P )−1
)= ρ
((αI − P ∗)(αI − P )(αI + P )−1(αI + P ∗)−1
).
Note that the matrix (αI + P ∗)−1(αI − P ∗)(αI − P )(αI + P )−1 is Hermitian positive semi-
definite. It follows that all the eigenvalues of (αI − P ∗)(αI − P )(αI + P )−1(αI + P ∗)−1 are
Splitting Methods for Non-Hermitian Positive Definite Linear Systems 303
nonnegative. This, together with the fact
(αI − P ∗)(αI − P )(αI + P )−1(αI + P ∗)−1
= [α2I + P ∗P − α(P + P ∗)][α2I + P ∗P + α(P + P ∗)]−1
= [α2I + P ∗P + α(P + P ∗)− 2α(P + P ∗)][α2I + P ∗P + α(P + P ∗)]−1
= I − 2α(P + P ∗)[α2I + P ∗P + α(P + P ∗)]−1
= I − 2α(P + P ∗)(αI + P )−1(αI + P ∗)−1
=: I − L(α),
yields that all the eigenvalues of the matrix I − L(α) are nonnegative and
σ(α)2 = ρ(I − L(α)). (2.7)
For any λ ∈ Λ(L(α)), it is easy to see that 1 − λ ∈ Λ(I − L(α)), which shows that 1 − λ ≥ 0.
Then for any positive semi-definite matrix P and α > 0, we can know that 2α(P + P ∗) is
Hermitian positive semi-definite. This, along with the fact that L(α) is similar to 2α(αI +
P ∗)−1(P + P ∗)(αI + P )−1, leads to λ ≥ 0. Then we have
0 ≤ 1− λ ≤ 1.
Combining this with (2.7), it leads to
σ(α)2 = maxλ∈Λ(L(α))
1− λ ≤ 1.
Furthermore, if P is positive definite, then it is easy to see that 2α(αI+P ∗)−1(P+P ∗)(αI+P )−1
is Hermitian positive definite, which shows that λ > 0. At this case, we have 0 ≤ 1− λ < 1. It
leads to σ(α)2 < 1 immediately. Then the result follows.
Now we establish the convergence theorem for the PPS iteration method.
Theorem 2.1. Let A = M + N be a positive definite and semi-definite splitting of A. Then
for any positive constant α, the spectral radius ρ(S(α)) of the iteration matrix S(α) satisfied
ρ(S(α)) < 1, where S(α) is defined as in (2.6a). Therefore, the PPS iteration method is
convergent to the exact solution x∗ ∈ Cn of the linear system (1.1).
Proof. Note that N is a positive semi-definite matrix and M is a positive definite matrix.
Then by using Lemma 2.1, we can get
ρ(S(α)) = ρ((αI +N)−1(αI −M)(αI +M)−1(αI −N))
= ρ((αI −M)(αI +M)−1(αI −N)(αI +N)−1)
≤ ∥(αI −M)(αI +M)−1∥2∥(αI −N)(αI +N)−1∥2≤ ∥(αI −M)(αI +M)−1∥2 < 1,
which completes the proof. We shall emphasize that the scheme (2.5) can induce a class of preconditioners for solving
the system (1.1). Actually, let
R(α) =1
2α(αI −M)(αI −N),
304 N. HUANG AND C.F. MA
where M, N are defined as in (2.1). Then by using the definition of T (α), we can know that
A = T (α)−1 −R(α) (2.8)
defines a new splitting of the coefficient matrix A in (1.1). It is easy to see that the PPS iteration
method can also be induced by the matrix splitting (2.8). Moreover, the matrix T (α)−1 can be
alternatively used to precondition the Krylov subspace iteration methods for solving the system
(1.1).
3. Two Typical Practical Choices of the PP-splitting
In this section, we give two kinds of typical practical choices of the PPS method. One is
based on Hermitian and skew-Hermitian splitting, the other one is based on triangular splitting.
To this end, let
H(A) =1
2(A+A∗), S(A) =
1
2(A−A∗).
Then H(A), S(A) are the Hermitian and skew-Hermitian parts of the matrix A, respectively.
Let η be any real constant, let
M = H(A) + iηI, N = S(A)− iηI. (3.1)
It is easy to verify that M +N = A, M +M∗ = H(A) + iηI +H(A)− iηI = 2H(A) is positive
definite, and N +N∗ = S(A)− iηI − S(A) + iηI = 0 is positive semi-definite. Thereby, we can
get the following PPS iteration method: (αI +H(A) + iηI)x(k+12 ) = (αI − S(A) + iηI)x(k) + b,
(αI + S(A)− iηI)x(k+1) = (αI −H(A)− iηI)x(k+12 ) + b.
(3.2)
If η = 0, the iteration scheme (3.2) reduces to the Hermitian and skew-Hermitian splitting
(HSS) iteration method proposed in [16]. This implies that the HSS iteration method is a
special case of the PPS iteration method.
In the following, we shall establish the convergence analysis of (3.2). For this purpose, we
introduce the quantities
V (α, η) = (αI −H(A)− iηI)(αI +H(A) + iηI)−1,
W (α, η) = (αI − S(A) + iηI)(αI + S(A)− iηI)−1,
S(α, η) = (αI + S(A)− iηI)−1V (α, η)(αI − S(A) + iηI), (3.3)
T (α, η) = 2α[(αI +H(A) + iηI)(αI + S(A)− iηI)]−1. (3.4)
Then it is easy to verify that the iteration scheme (3.2) can be equivalent to
x(k+1) = S(α, η)x(k) + T (α, η)b. (3.5)
This shows that the method (3.2) is convergent if and only if the spectral radius of the iterative
matrix S(α, η) is less than 1.
Theorem 3.1. Let A ∈ Cn×n be a positive definite matrix, M and N be defined as in (3.1),
λ1 and λn be the maximum and minimum eigenvalues of H(A). Then for any η ∈ R and α > 0,
Splitting Methods for Non-Hermitian Positive Definite Linear Systems 305
(1) the sequence generated by the iteration scheme (3.2) converges to the exact solution x∗ ∈Cn of the system (1.1);
(2) the spectral radius ρ(S(α, η)) of the iteration matrix S(α, η) is bounded by
ρ(S(α, η)) ≤√1− φ(α, η), (3.6a)
where
φ(α, η) = min 4αλ1(α+ λ1)2 + η2
,4αλn
(α+ λn)2 + η2
. (3.6b)
Proof. Noticing that A and M are positive definite, and N is positive semi-definite, then by
Theorem 2.1, we can get the first result. Now we estimate the bound of ρ(S(α, ω)).
It is easy to see that (αI − S(A) + iηI)−1S(A) = S(A)(αI − S(A) + iηI)−1. Then we can
know that
W (α, η)∗W (α, η)
= (αI − S(A) + iηI)−1(αI + S(A)− iηI)(αI − S(A) + iηI)(αI + S(A)− iηI)−1
= (αI + S(A)− iηI)(αI − S(A) + iηI)−1(αI − S(A) + iηI)(αI + S(A)− iηI)−1
= I.
Similarly, we can get W (α, η)W (α, η)∗ = I. This shows that W (α, η) is a unitary matrix and
∥W (α, η)∥2 = 1. This, along with the fact that S(α, η) is similar to V (α, η)W (α, η), leads to
ρ(S(α, η)) = ρ(V (α, η)W (α, η)) ≤ ∥V (α, η)∥2∥W (α, η)∥2 = ∥V (α, η)∥2. (3.7)
On the other hand, by Rayleigh-Ritz theorem, it follows that
∥V (α, η)∥22 = ρ(V (α, η)∗V (α, η))
= ρ((αI +H(A)− iηI)−1(αI −H(A) + iηI)(αI −H(A)− iηI)(αI +H(A) + iηI)−1
)= max
λ∈Λ(H(A))
(α− λ+ iη)(α− λ− iη)
(α+ λ− iη)(α+ λ+ iη)= max
λ∈Λ(H(A))
(α− λ)2 + η2
(α+ λ)2 + η2. (3.8)
Noticing that for any λ ∈ Λ(H(A)), we can get
(α− λ)2 + η2
(α+ λ)2 + η2=α2 − 2λα+ λ2 + η2
α2 + 2λα+ λ2 + η2=α2 + 2λα+ λ2 + η2 − 4λα
α2 + 2λα+ λ2 + η2
= 1− 4λα
α2 + 2λα+ λ2 + η2= 1− 4α
(α2 + η2)/λ+ λ+ 2α. (3.9)
Since α > 0 and α2 + η2 > 0, we have
maxλ∈Λ(H(A))
α2 + η2
λ+ λ = max
α2 + η2
λ1+ λ1,
α2 + η2
λn+ λn
.
306 N. HUANG AND C.F. MA
This, together with α > 0, (3.8) and (3.9), yields that
∥V (α, η)∥22 = maxλ∈Λ(H(A))
[1− 4α
(α2 + η2)/λ+ λ+ 2α
]= 1− min
λ∈Λ(H(A))
4α
(α2 + η2)/λ+ λ+ 2α
= 1−min 4α
(α2 + η2)/λ1 + λ1 + 2α,
4α
(α2 + η2)/λn + λn + 2α
= 1−min
4αλ1(α+ λ1)2 + η2
,4αλn
(α+ λn)2 + η2
.
Then the second result follows.
From the above theorem, we can see that the optimal point (α∗, η∗) of ρ(S(α, η)) make
φ(α, η) maximum. This implies that we just need to solve the following optimization problem
maxα>0,η∈R
φ(α, η)
if we need to find the optimal parameters.
Theorem 3.2. Under the same settings and conditions as in Theorem 3.1, for any η ∈ R and
α > 0, it holds that
(α∗, η∗) ≡ arg maxα>0,η∈R
φ(α, η) = (√λ1λn, 0), (3.10)
φ(α∗, η∗) =4√λ1λn
(√λ1 +
√λn)2
. (3.11)
Proof. From (3.6b), we can see that the function φ(α, η) decreases monotonically with
respect to η2 for any α > 0. This shows that
η∗ ≡ argmaxα>0
φ(α, η) = 0, (3.12a)
φ(α, 0) = min 4αλ1(α+ λ1)2
,4αλn
(α+ λn)2
. (3.12b)
In order to find the maximum point of φ(α, 0), in the following, we derive α∗ such that minimize
the function 1/φ(α, 0). It follows from (3.12b) that
1
φ(α, 0)= max
(α+ λ1)2
4αλ1,(α+ λn)
2
4αλn
= max
α
4λ1+λ14α
+1
2,α
4λn+λn4α
+1
2
. (3.13)
Since H(A) is Hermitian positive definite, we have λ1 > 0 and λn > 0. Then it is easy to
see that α∗ =√λ1λn. This, along with the independence of α and η, leads to (3.10) which
immediately yields (3.11). This completes the proof.
Remark 3.1. We shall emphasize that (√λ1λn, 0) is only the minimum point of
√1− φ(α, η),
rather than that of ρ(S(α, η)). Therefore, from Theorem 3.3, we can see that the optimal point
(α∗, η∗) is the quasi-optimal parameter of ρ(S(α, η)), and the quasi-optimal spectral radius is
ρ(S(α∗, η∗)) ≤√1− φ(α∗, η∗) =
√λ1 −
√λn√
λ1 +√λn. (3.14)
Splitting Methods for Non-Hermitian Positive Definite Linear Systems 307
In the following, we shall study another typical practical choice of the PPS method. Let
A = D + L+ U, (3.15)
where D, L and U are the diagonal, the strictly lower-triangular, and the strictly upper-
triangular matrices of the matrix A. For any ω ∈ R, let
M = D + L+ U∗ + iωI, N = U − U∗ − iωI, (3.16a)
M = D + L∗ + U + iωI, N = L− L∗ − iωI, (3.16b)
it is easy to verify that A =M +N . Noticing that
M +M∗ = D + L+ U∗ + iωI +D∗ + L∗ + U − iωI = A+A∗, (3.17a)
M +M∗ = D + L∗ + U + iωI +D∗ + L+ U∗ − iωI = A+A∗, (3.17b)
we can know that the matrix M defined above is positive definite. On the other hand, from the
fact N +N∗ = 0, it follows that N is positive semi-definite. Then we can get the PPS iteration
method as follows: (αI +D + L+ U∗ + iωI)x(k+12 ) = (αI − U + U∗ + iωI)x(k) + b,
(αI + U − U∗ − iωI)x(k+1) = (αI −D − L− U∗ − iωI)x(k+12 ) + b,
(3.18)
or (αI +D + L∗ + U + iωI)x(k+12 ) = (αI − L+ L∗ + iωI)x(k) + b,
(αI + L− L∗ − iωI)x(k+1) = (αI −D − L∗ − U − iωI)x(k+12 ) + b.
(3.19)
Particularly, if ω = 0, then the above iteration formulas reduce to the triangular and skew-
Hermitian splitting splitting (TSS) iteration method proposed in [14]. This shows that the TSS
iteration method is a special case of the PPS iteration method. In addition, it is easy to see
that the iterative formula (3.18) and (3.19) have same structure and property. Hence we just
need to analyze the convergence of (3.18) in the following. Let
V (α, ω) = (αI −D − L− U∗ − iωI)(αI +D + L+ U∗ + iωI)−1,
W (α, ω) = (αI − U + U∗ + iωI)(αI + U − U∗ − iωI)−1,
S(α, ω) = (αI + U − U∗ − iωI)−1V (α, ω)(αI − U + U∗ + iωI), (3.20)
T (α, ω) = 2α[(αI +D + L+ U∗ + iωI)(αI + U − U∗ − iωI)]−1. (3.21)
Then the iteration scheme (3.18) can be reformulated equivalently as
x(k+1) = S(α, ω)x(k) + T (α, ω)b. (3.22)
Firstly, we propose the following lemmas.
Lemma 3.1. Let D, U, L be defined as in (3.15), and let t1 = α− iω, t2 = α+iω, Q = L+U∗.
Then it holds that
V (α, ω) = (t1I −D)(t2I +D)−1 + 2α(t2I +D)−1n−1∑j=1
(−1)j [Q(t2I +D)−1]j . (3.23)
308 N. HUANG AND C.F. MA
Proof. By the definition of t1, t2 and Q, we can get
V (α, ω) = (t1I −D −Q)(t2I +D +Q)−1. (3.24)
It is easy to verify that t2I +D +Q = [I +Q(t2I +D)−1](t2I +D). Consequently,
(t2I +D +Q)−1 = (t2I +D)−1[I +Q(t2I +D)−1]−1
= (t2I +D)−1∞∑j=0
(−1)j [Q(t2I +D)−1]j .
Noticing thatQ(t2I+D)−1 is a strictly lower triangular matrix, it follows that all the eigenvalues
of Q(t2I + D)−1 are 0. Thus its characteristic polynomial is f(λ) = λn. This, together with
the Hamilton-Cayley theorem, yields [Q(t2I +D)−1]n = 0. Then we have
(t2I +D +Q)−1 = (t2I +D)−1n−1∑j=0
(−1)j [Q(t2I +D)−1]j . (3.25)
Using the definition of t1, t2 and (3.24), we can obtain
V (α, ω) = I − [(t2 − t1)I+2(D +Q)](t2I+D +Q)−1 = I − 2(iωI+D +Q)(t2I+D +Q)−1
= I − 2(iωI +D +Q)(t2I +D)−1n−1∑j=0
(−1)j [Q(t2I +D)−1]j
= I − 2(iωI +D)(t2I +D)−1n−1∑j=0
(−1)j [Q(t2I +D)−1]j
− 2Q(t2I +D)−1n−1∑j=0
(−1)j [Q(t2I +D)−1]j
= I − 2(iωI +D)(t2I +D)−1 − 2(iωI +D)(t2I +D)−1n−1∑j=1
(−1)j [Q(t2I +D)−1]j
+ 2
n−1∑j=1
(−1)j [Q(t2I +D)−1]j
= (t1I −D)(t2I +D)−1 + 2α(t2I +D)−1n−1∑j=1
(−1)j [Q(t2I +D)−1]j , (3.26)
where the sixth equality holds by [Q(t2I +D)−1]n = 0. This completes the proof.
By Lemma 3.1, we can take (t1I − D)(t2I + D)−1 as a first-order approximation of the
matrix V (α, ω), then we get
∥V (α, ω)∥2 ≈ ∥(t1I −D)(t2I +D)−1∥2.
Lemma 3.2. For any α > 0 and ω ∈ R, the matrix W (α, ω) is a unitary matrix, which follows
that ∥W (α, ω)∥2 = 1
By the above analysis, we are now in a position to present the following theorem.
Theorem 3.3. Assume A ∈ Cn×n is a positive definite matrix. Let D = diaga11, a22, · · · , ann,L and U be the diagonal, the strictly lower-triangular, and the strictly upper-triangular matrices
of the matrix A. Then for any α > 0 and ω ∈ R,
Splitting Methods for Non-Hermitian Positive Definite Linear Systems 309
(1) the sequence generated by the iteration scheme (3.18) converges to the exact solution
x∗ ∈ Cn of the system (1.1);
(2) the spectral radius ρ(S(α, ω)) of the iteration matrix S(α, ω) is bounded by
ρ(S(α, ω)) ≤ max1≤i≤n
|α− iω − aii||α+ iω + aii|
, (3.27)
approximatively;
(3) If aii, i = 1, · · · , n are all real with amin and amax being the minimum and the maximum
elements of aii, i = 1, · · · , n, respectively, then it holds that
(α∗, ω∗) ≡ arg minα>0,ω∈R
ψ(α, ω) = (√aminamax, 0), (3.28a)
ψ(α∗, ω∗) =
√amax −
√amin√
amax +√amin
, (3.28b)
where
ψ(α, ω) = max1≤i≤n
|α− iω − aii||α+ iω + aii|
. (3.28c)
Proof. As the iteration scheme (3.18) is a special case of the PPS iteration method, from
Theorem 2.1, we get the first result.
Since A is a positive definite matrix, we have Re(aii) > 0, i = 1, · · · , n. It follows from
Lemmas 3.1 and 3.2 that
ρ(S(α, ω)) = ρ(V (α, ω)W (α, ω)) ≤ ∥V (α, ω)∥2∥W (α, ω)∥2≈ ∥(t1I −D)(t2I +D)−1∥2
= max1≤i≤n
|t1 − aii||t2 + aii|
= max1≤i≤n
|α− iω − aii||α+ iω + aii|
< 1, (3.29)
where the last equality holds by the definition of t1 and t2. Then the second result follows.
Furthermore, if aii, i = 1, · · · , n are all real, then aii > 0. This yields that
ψ(α, ω) = max1≤i≤n
|α− iω − aii||α+ iω + aii|
= max1≤i≤n
[ (α− aii)2 + ω2
(α+ aii)2 + ω2
] 12
. (3.30)
For any i = 1, · · · , n, since
(α− aii)2 + ω2
(α+ aii)2 + ω2=
(α+ aii)2 + ω2 − 4αaii
(α+ aii)2 + ω2= 1− 4αaii
(α+ aii)2 + ω2,
we can know that ψ(α, ω) increases monotonically with respect ω2 for any α > 0. Then
ω∗ ≡ argminω∈R
ψ(α, ω) = 0, (3.31)
which immediately follows that
ψ(α, ω∗) = max1≤i≤n
|α− aii||α+ aii|
= max |α− amin||α+ amin|
,|α− amax||α+ amax|
. (3.32)
If α∗ is a minimum point of ψ(α, ω∗), then it must satisfy that α−amin > 0, α−amax < 0, and
α− amin
α+ amin=amax − α
α+ amax.
310 N. HUANG AND C.F. MA
By some simple calculation, we can get
α∗ ≡ argminα>0
ψ(α, ω∗) =√aminamax.
This, along with the independence of α and ω, leads to (3.28a) and (3.28b).
Remark 3.2. We shall emphasize that the condition of the third result on the diagonal ele-
ments is necessary. Otherwise, we can not derive the specific formula of ψ(α, ω). For example,
let
A =
1 + i 0 0
0 2 0
0 0 1− 1
2i
.
Evidently, A is positive definite and a11 = 1 + i, a22 = 2, a33 = 1 − 12 i. Then for α = 1 and
ω = 0, we have
|1− a11||1 + a11|
=1√5,
|1− a22||1 + a22|
=1
3,
|1− a33||1 + a33|
=1√17.
This show that
ψ(1, 0) =|1− a11||1 + a11|
.
However, a11 does not have apparent characteristic point, such as maximum modulus or mini-
mum modulus. Therefore, for general problems, we can not determine the expression of ψ(α, ω).
And then we can not calculate the minimum of ψ(α, ω).
4. Numerical Experiments
In this section, we use two examples to numerically examine feasibility and effectiveness of
our new methods. All our tests are started from zero vector, and terminated when the current
iteration satisfies∥rk∥2∥r0∥2
< 10−5, (4.1)
where rk is the residual of the current, say k-th, iteration. We report the number of required
iterations (IT), the norm of the relative residual (4.1) (Res) when the process is stopped, the
required CPU time (CPU). All algorithms were coded by MATLAB 2011b and were run on a
PC with 2.20 GHz Pentium(R) Dual-Core CPU and 2.00 GB RAM.
Example 4.1. Consider the system of linear equations (1.1), for which
A = tridiag(−1 + i, 10, 1− i), b = (1, 1, · · · , 1)T .
Since A+A∗ = tridiag(2i, 20,−2i), A is non-Hermitian positive definite. We split A intoM+N
with
M = tridiag(0, 5, 1− i), and N = tridiag(−1 + i, 5, 0).
This shows that
M +M∗ = tridiag(1 + i, 10, 1− i), and N +N∗ = tridiag(−1 + i, 10,−1− i).
Splitting Methods for Non-Hermitian Positive Definite Linear Systems 311
Table 4.1: Numerical results of Example 4.1.
HSS TSS PPS
α IT CPU Res IT CPU Res IT CPU Res
3 19 14.0936 4.2169e-6 20 9.6555 4.3508e-6 5 0.9476 1.1832e-7
4 14 10.7445 3.0389e-6 15 8.0475 2.7010e-6 4 0.8178 1.6163e-8
5 11 8.8524 1.8959e-6 12 6.0794 1.5284e-6 3 0.6128 1.5982e-7
6 9 7.1883 9.6551e-7 9 4.9430 2.7557e-6 3 0.6843 1.2306e-7
7 7 5.6605 9.5964e-7 8 4.4507 8.3373e-7 4 0.8672 3.5326e-8
8 6 4.9500 2.2618e-7 7 3.9645 3.5884e-7 4 0.8936 5.0542e-7
9 4 3.4438 5.4925e-7 6 3.1593 3.0152e-7 5 0.9906 3.2121e-7
10 3 2.8982 6.9605e-7 5 2.9549 9.6901e-7 6 1.1582 2.2039e-7
11 4 3.5298 5.3756e-7 6 3.3052 1.5220e-7 6 1.1204 1.1160e-6
12 5 4.2154 6.6345e-7 6 3.1742 5.7696e-7 7 1.2235 6.9808e-7
13 6 4.7301 6.9013e-7 7 3.6064 3.6556e-7 8 1.4491 4.6709e-7
Fig. 4.1. ρ(M(α)) versus α for the PPS, the HSS and the TSS iteration matrices for Example 4.1.
Hence both M and N are positive definite. In our test, we compare the PPS method based on
this splitting with the HSS method and the TSS method. We take the dimension n = 1024.
The test results are listed in Table 4.1. And we depict the curves of ρ(M(α)) with respect to
α for all PPS, HSS and TSS iteration methods to intuitively show this functional relationship
in Figure 4.1.
From Table 4.1, we can see that all the methods stopped regularly and the optimal parameter
αopt of the PPS method is in the interval of [5, 6]. The optimal parameters of the HSS method
and the TSS method are both near by 10. This fact is further confirmed by Figure 4.1. As
shown in Table 4.1, whatever the method, the number of iterations and the actual computing
time vary seriously for the different parameter α. Furthermore, whatever the parameter α we
choose, the CPU time of the PPS method is always the least.
312 N. HUANG AND C.F. MA
Table 4.2: Numerical results of Example 4.2.
HPPS TPPS
η/ω αopt IT CPU Res αopt IT CPU Res
-8 11.7742 73 4.6751 7.7659e-006 11.2158 72 3.1349 8.3495e-006
-6 12.9086 66 4.1304 8.2780e-006 12.4248 65 2.9226 8.1309e-006
-4 13.6613 63 3.9267 7.2506e-006 13.2241 61 2.9510 8.0710e-006
-2 14.0936 61 3.6657 7.2436e-006 13.7097 59 2.4537 7.7419e-006
0 14.2683 60 0.5630 7.5525e-006 13.8493 59 0.5602 6.8548e-006
2 14.0936 61 3.6096 7.2436e-006 13.7097 59 2.6133 7.7419e-006
4 13.6613 63 3.8884 7.2506e-006 13.2241 61 3.0034 8.0710e-006
6 12.9086 66 4.2420 8.2780e-006 12.4248 65 2.9795 8.1309e-006
8 11.7742 73 4.6436 7.7659e-006 11.2158 72 3.1213 8.3495e-006
Example 4.2. (see [14]) Consider the system of linear equations (1.1), for which
A =
(W FΩ
−FT N
),
where W ∈ Rq×q, and N, Ω ∈ R(n−q)×(n−q), with 2q > n, b = Ae, e = (1, · · · , 1)T .In particular, the matrices W , N , F and Ω = diagν1, · · · , νn−q are defined as follows:
wkj =
k + 1, for j = k,
1, for |k − j| = 1,
0, otherwise,
k, j = 1, · · · , q,
nkj =
k + 1, for j = k,
1, for |k − j| = 1,
0, otherwise,
k, j = 1, · · · , n− q,
fkj =
j, for k = j + 2q − n,
0, otherwise,k = 1, · · · , q, j = 1, · · · , n− q,
νk =1
k, k = 1, · · · , n− q.
In this example, we compare the iteration scheme (3.2) (HPPS for short) with the iteration
scheme (3.19) (TPPS for short) with different parameter η or ω. In our computations, we
choose n = 200 and q = 910n. The test results are listed in Table 4.2. And we depict the
surfaces of ρ(S(α, η)) (or ρ(S(α, ω))) with respect to α and η (or ω) for HPPS (or TPPS)
iteration method to intuitively show this functional relationship in Figure 4.2 (or Figure 4.3).
Moreover, we depict the eigenvalue distributions of the HPPS (or TPPS) iteration matrices for
different parameter η (or ω) and the corresponding optimal parameter αopt in Figure 4.4 (or
Figure 4.5).
As can be seen from Table 4.2, when η = 0, the number of iterations and the CPU time
of the HPPS method are the best. And the HPPS method may be slowed by increasing the
module of η. Moreover, we can see that the numerical results of the HPPS method are much
the same when the module of η is equal. And the optimal parameter αopt is equal when the
module of η is equal. This fact is further confirmed by Figure 4.2. For TPPS method, we have
the same results as the HPPS method, which can be seen from Table 4.2 and Figure 4.3. As is
shown in Figure 4.4, the distribution domain of the eigenvalues of the HPPS iteration matrix
Splitting Methods for Non-Hermitian Positive Definite Linear Systems 313
510
1520
2530
−5
0
5
0.85
0.9
0.95
αη
ρ(S(
α,η)
)
Fig. 4.2. Surfaces of ρ(S(α, η)) versus α and η for the HPPS iteration matrix for Example 4.2.
510
1520
2530
−5
0
5
0.85
0.9
0.95
αω
ρ(S(
α,ω)
)
Fig. 4.3. Surfaces of ρ(S(α, ω)) versus α and ω for the TPPS iteration matrix for Example 4.2.
Fig. 4.4. The eigenvalue distributions of the HPPS iteration matrices when η = −4, 0, 4 for Example
4.2.
314 N. HUANG AND C.F. MA
Fig. 4.5. The eigenvalue distributions of the TPPS iteration matrices when ω = −4, 0, 4 for Example
4.2.
0 5 10 15 20 25 300.82
0.84
0.86
0.88
0.9
0.92
0.94
0.96
0.98
1
1.02
Parameter α
Spec
tral r
adiu
s
Take (n,η)=(200,0.25)
HPSSTPSS
Fig. 4.6. Curves of ρ(S(α)) versus α for the HPPS and the TPPS iteration matrices for Example 4.2
by η = ω = 0.25.
when η = 0 is considerably smaller than that when η = 0, in particular, along the direction of
the imaginary axis. From Figure 4.5, we can see the same result for the TPPS iteration matrix.
In addition, as you will see in Table 4.2, the numerical results of the TPPS method are a little
better than that of the HPPS method, which also can be seen from Figure 4.6.
Acknowledgments. The authors thank the anonymous reviewers for their valuable comments
and suggestions that helped improve the quality of this paper. The project was supported by
National Natural Science Foundation of China (Grant Nos. 11071041, 11201074) and Fujian
Natural Science Foundation (Grant No. 2016J01005).
References
[1] S.R. Arridge, Optical tomography in medical imaging, Inverse Probl., 15 (1999) 41-93.
[2] O. Axelsson, A generalized SSOR method, BIT, 12 (1972) 443-467.
Splitting Methods for Non-Hermitian Positive Definite Linear Systems 315
[3] O. Axelsson, Z.-Z. Bai and S.-X. Qiu, A class of nested iteration schemes for linear systems with
a coefficient matrix with a dominant positive definite symmetric part, Numer. Algorithms, 35
(2004) 351-372.
[4] K. Arrow, L. Hurwicz and H. Uzawa, Studies in Nonlinear Programming, Stanford University
Press, Stanford, CA, 1958.
[5] Z.-Z. Bai, Sharp error bounds of some Krylov subspace methods for non-Hermitian linear systems,
Appl. Math. Comput., 109 (2000) 273-285.
[6] Z.-Z. Bai, Optimal parameters in the HSS-like methods for saddle point problems, Numer. Linear
Algebra Appl., 16 (2009) 447-479.
[7] A. Bendali, Numerical analysis of the exterior boundary value problem for the time-harmonic
Maxwell equations by a boundary finite element method, Math. Comp., 43 (1984) 29-68.
[8] M. Benzi and D. Bertaccini, Block preconditioning of real-valued iterative algorithms for complex
linear systems, IMA J. Numer. Anal., 28 (2008) 598-618.
[9] Z.-Z. Bai, M. Benzi and F. Chen, Modified HSS iteration methods for a class of complex symmetric
linear systems, Computing, 87 (2010) 93-111.
[10] Z.-Z. Bai, M. Benzi and F. Chen, On preconditioned MHSS iteration methods for complex sym-
metric linear systems, Numer. Algorithms, 56 (2011) 297-317.
[11] Z.-Z. Bai, I. Duff and A.J. Wathen, A class of incomplete orthogonal factorization methods I:
Methods and theories, BIT, 41 (2001) 53-70.
[12] Z.-Z. Bai and G.H. Golub, Accelerated Hermitian and skew-Hermitian splitting iteration methods
for saddle point problems, IMA J. Numer. Anal., 27 (2007) 1-23.
[13] Z.-Z. Bai, G.H. Golub and C.-K. Li, Convergence properties of preconditioned Hermitian and skew-
Hermitian splitting methods for non-Hermitian positive semidefinite matrices, Math. Comp., 76
(2007) 287-298.
[14] Z.-Z. Bai, G.H. Golub, L.-Z. Lu and J.-F. Yin, Block triangular and skew-Hermitian splitting
method for positive-definite linear systems, SIAM J. Sci. Comput., 26 (2005) 844-863.
[15] Z.-Z. Bai, G.H. Golub and M.K. Ng, On successive-overrelaxation acceleration of the Hermitian
and skew-Hermitian splitting iterations, Numer. Linear Algebra Appl., 14 (2007) 319-335.
[16] Z.-Z. Bai, G.H. Golub and M.K. Ng, Hermitian and skew-Hermitian splitting methods for non-
Hermitian positive definite linear systems, SIAM J. Matrix Anal. Appl., 24 (2003) 603-626.
[17] Z.-Z. Bai, G.H. Golub and J.-Y. Pan, Preconditioned Hermitian and skew-Hermitian splitting
methods for non-Hermitian positive semidefinite linear systems, Numer. Math., 98 (2004) 1-32.
[18] M.A. Bochev and L.A. Krukier, On an iteration solution for strongly nonsymmetric systems of
linear algebraic equations, Comp. Math. Math. Phys., 37 (1997) 1241-1251.
[19] M. Benzi and D. Szyld, Existence and uniqueness of splittings of stationary iterative methods
with applications to alternating methods, Numer. Math., 76 (1997) 309-321.
[20] A. Bunse-Gerstner and R. Stover, On a conjugate gradient-type method for solving complex
symmetric linear systems, Linear Algebra Appl., 287 (1999) 105-123.
[21] Z.-Z. Bai and Z.-Q. Wang, On parameterized inexact Uzawa methods for generalized saddle point
problems, Linear Algebra Appl., 428 (2008) 2900-2932.
[22] S.H. Christiansen, Discrete Fredholm properties and convergence estimates for the electric field
integral equation, Math. Comp., 73 (2004) 143-167.
[23] M. Clemens and T. Weiland, Comparison of Krylov-type methods for complex linear systems
applied to high-voltage problems, IEEE Trans. Mag., 34 (1998) 3335-3338.
[24] D.D. Day and M.A. Heroux, Solving complex-valued linear systems via equivalent real formula-
tions, SIAM J. Sci. Comput., 23 (2001) 480-498.
[25] M. Eiermann, W. Niethammer and R.S. Varga, Acceleration of relaxation methods for non-
Hermitian linear systems, SIAM J. Matrix Anal. Appl., 13 (1992) 979-991.
[26] R.W. Freund, Conjugate gradient-type methods for linear systems with complex symmetric coef-
ficient matrices, SIAM J. Sci. Stat. Comput., 13 (1992) 425-448.
316 N. HUANG AND C.F. MA
[27] A. Hadjidimos, Accelerated overrelaxation method, Math. Comp., 32 (1978) 149-157.
[28] N. Huang and C.-F. Ma, Parallel multisplitting iteration methods based on M-splitting for the
PageRank problem, Appl. Math. Comput., 271 (2015) 337-343.
[29] N. Huang and C.-F. Ma, The BGS-Uzawa and BJ-Uzawa iterative methods for solving the saddle
point problem, Appl. Math. Comput., 256 (2015) 94-108.
[30] N. Huang and C.-F. Ma, The nonlinear inexact Uzawa hybrid algorithms based on one-step Newton
method for solving nonlinear saddle-point problems, Appl. Math. Comput., 270 (2015) 291-311.
[31] N. Huang and C.-F. Ma, Y.-J. Xie, An inexact relaxed DPSS preconditioner for saddle point
problem, Appl. Math. Comput., 265 (2015) 431-447.
[32] M.R. Hestenes and E. Stiefel, Methods of conjugate gradients for solving linear systems, J. Res.
Nat. Bur. Standards, 49 (1952) 409-436.
[33] L.A. Krukier, L.G. Chikina and T.V. Belokon, Triangular skew-symmetric iterative solvers for
strongly nonsymmetric positive real linear system of equations, Appl. Numer. Math., 41 (2002)
89-105.
[34] L.A. Krukier, B.L. Krukier and Z.-R. Ren, Generalized skew-Hermitian triangular splitting iter-
ation methods for saddle-point linear systems, Numer. Linear Algebra Appl., 21 (2014) 152-170.
[35] L.A. Krukier, T. Martynova and Z.-Z. Bai, Product-type skew-Hermitian triangular splitting
iteration methods for strongly non-Hermitian positive definite linear systems, J. Comput. Appl.
Math., 232 (2009) 3-16.
[36] J. Lu and Z. Zhang, A modified nonlinear inexact Uzawa algorithm with a variable relaxation
parameter for the stabilized saddle point problem, SIAM J. Matrix Anal. Appl., 31 (2010) 1934-
1957.
[37] T. Manteuffel, An incomplete factorization technique for positive definite linear systems, Math.
Comp., 34 (1980) 473-497.
[38] G. Moro and J.H. Freed, Calculation of ESR spectra and related Fokker-Planck forms by the use
of the Lanczos algorithm, J. Chem. Phys., 74 (1981) 3757-3773.
[39] B. Poirier, Effcient preconditioning scheme for block partitioned matrices with structured sparsity,
Numer. Linear Algebra Appl., 7 (2000) 715-726.
[40] Y. Saad, Iterative methods for sparse linear systems, (2nd edn), SIAM: Philadelphia, 2003.
[41] D. Schmitt, B. Steffen and T. Weiland, 2D and 3D computations of lossy eigenvalue problems,
IEEE Trans. Magnetics, 30 (1994) 3578-3581.
[42] T. Sogabe and S.-L. Zhang, A COCR method for solving complex symmetric linear systems, J.
Comput. Appl. Math., 199 (2007) 297-303.
[43] H. Van der Vorst, Iterative Krylov methods for large linear systems, Cambridge University Press,
Cambridge, 2003.
[44] L. Wang and Z.-Z. Bai, Skew-Hermitian triangular splitting iteration methods for non-Hermitian
positive definite linear systems of strong skew-Hermitian parts, BIT, 44 (2004) 363-386.
[45] W.-W. Li and X. Wang, A modified GPSS method for non-Hermitian positive definite linear
systems, Appl. Math. Comput., 234 (2014) 253-259.
[46] X. Wang, Y. Li and L. Dai, On Hermitian and skew-Hermitian splitting iteration methods for the
linear matrix equation AXB = C, Comput. Math. Appl., 65 (2013) 657-664.
[47] X. Wang, W.-W. Li and L.-Z. Miao, On positive-definite and skew-Hermitian splitting iteration
methods for continuous Sylvester equation AX + XB = C, Comput. Math. Appl., 66 (2013)
2352-2361.