Calcolo (2014) 51:141–149DOI 10.1007/s10092-013-0079-3

Convergence and comparison theorems for singleand double decompositions of rectangular matrices

Litismita Jena · Debasisha Mishra · Sabyasachi Pani

Received: 21 October 2011 / Accepted: 17 January 2013 / Published online: 1 February 2013© Springer-Verlag Italia 2013

Abstract Different convergence and comparison theorems for proper regular split-tings and proper weak regular splittings are discussed. The notion of double splitting isalso extended to rectangular matrices. Finally, convergence and comparison theoremsusing this notion are presented.

Keywords Moore-Penrose inverse · Double splitting · Proper splittings ·Non-negativity · Convergence theorem · Comparison theorem

Mathematics Subject Classification (2000) 15A09 · 65F15 · 65F20

1 Introduction

Many authors have studied the problem of characterizing inverse positive matri-ces (A−1 exists and A−1 ≥ 0, where the comparison is entry wise) in termsof matrix splittings. For a real square matrix A, a decomposition A = U − Vis a splitting if U is invertible. Any such splitting leads to the iterative methodxi+1 = U−1V xi + U−1b, i = 0, 1, 2, . . . for solving the linear system Ax = b,b ∈ R

n . It is well-known that this iterative scheme converges to a solution of Ax = b,

for any initial vector x0, if and only if the spectral radius ρ(U−1V ) of U−1V is strictly

L. Jena (B) · S. PaniSchool of Basic Sciences, Indian Institute of Technology Bhubaneswar, Bhubaneswar 751 013, Indiae-mail: litumath@gmail.com

S. Panie-mail: spani@iitbbs.ac.in

D. MishraInstitute of Mathematics and Applications, Bhubaneswar 751 003, Indiae-mail: kapamath@gmail.com

123

142 L. Jena et al.

less than 1. Standard iterative methods like the Jacobi, Gauss-Seidel and successiveover-relaxation methods arise from different choices of U and V . Schröder [12] andVarga [15] proposed the notion of a regular splitting as follows: A = U − V is calleda regular splitting if U is invertible, U−1 ≥ 0 and V ≥ 0. They showed that ifA = U − V is a regular splitting of A ∈ R

n×n , then A is invertible and A−1 ≥ 0 ifand only if ρ(U−1V ) < 1.

Ortega and Rheinboldt [11] then proposed the following splitting: A = U − V is aweak regular splitting if U is invertible, U−1 ≥ 0 and U−1V ≥ 0. They also provedthat for any weak regular splitting A = U − V , A−1 ≥ 0 if and only if ρ(U−1V ) < 1.

Csordas and Varga [5], Elsner [6] and Woznicki [16] have proved various comparisonresults for different matrix splittings. In particular, Csordas and Varga [5] proved thatfor any two regular splittings A = U1 − V1 = U2 − V2, the condition A−1 ≥ 0 andV2 ≥ V1 imply 1 > ρ(U−1

2 V2) ≥ ρ(U−11 V1). They again obtained another analogous

result replacing the condition V2 ≥ V1 by U−11 ≥ U−1

2 .In this paper, we extend the above results to the case of rectangular matrices where

A, U, V ∈ Rm×n are such that A = U − V . Denoting U † the Moore-Penrose inverse

of A, we consider the iteration

x (i+1) = U †V x (i) + U †b. (1)

The scheme (1) is said to be convergent if ρ(U †V ) < 1, and U †V is called theiteration matrix. A decomposition A = U − V of A ∈ R

m×n (the set of all real m × nmatrices) is called a proper splitting ([2]) if R(A) = R(U ) and N (A) = N (U ), whereR(A) and N (A) stand for the range space and the null space of A. Using this, theauthors of [2] showed that the scheme (1) converges to x = A†b for any initial vectorx0 if and only if ρ(U †V ) < 1.

Now, we are going to introduce the proposed extension of regular and weak regularsplitting for rectangular matrices. Earlier, Climent et al. [3] have also proposed theextension of regular splitting but they simply call this as regular splitting even forrectangular case (see Definition 1, [4] and Definition 2, [3]). Hence to make a differencebetween the regular splitting of a real square invertible matrix and real rectangularmatrix, and in a more general way, we call this as proper regular splitting and thedefinition is as follows.

Definition 1.1 A decomposition A = U − V of A ∈ Rm×n is called a proper regular

splitting if it is a proper splitting such that U † ≥ 0 and V ≥ 0.

The next definition is the corresponding analogue of weak regular splitting. Notethat Climent et al. [3] call such an extension as weak non-negative of the first typewhile Climent and Perea [4] name it as weak proper splitting of the first type. (SeeDefinition 2, [3] and Definition 1, [4].)

Definition 1.2 A decomposition A = U − V of A ∈ Rm×n is called a proper weak

regular splitting if it is a proper splitting such that U † ≥ 0 and U †V ≥ 0.

We recall that Climent et al. [3] have proved a convergence theorem (Theorem 2,[3]) for these splittings in the case of matrices of full rank. Moreover, Berman andPlemmons [2] have given the following result (Theorem 3, [2]).

123

Convergence and comparison theorems 143

Theorem 1.3 Let A = U − V be a proper regular splitting of A ∈ Rm×n. Then

A† ≥ 0 if and only if ρ(U †V ) < 1.

Note that the theorem above also holds for proper weak regular splittings. Com-parison theorems between the spectral radii of matrices are useful tools in theanalysis of rate of convergence of iterative methods or for judging the efficiencyof pre-conditioners. There is also a connection to population dynamics (see [7] andthe references cited there in).

Miao and Zheng [14] and Shen et al. [13] studied convergence and comparisontheorems of inverse positive matrices using double splittings. A decomposition A =P − R + S of A ∈ R

n×n is called double splitting if P is invertible. This notion wasfirst introduced by Woznicki [17], and was later extended by Neumann [10] for realm × n matrices which he called as 3-part splitting. Several comparison results fordouble splittings are available in the literature. Among them, Shu-Xin and Bing [14]presented the result given below for invertible matrices.

Theorem 1.4 (Theorem 3.1, [14]) Let A1 and A2 be two inverse positive matrices.Suppose that A1 = P1 − R1 + S1 and A2 = P2 − R2 + S2 be their weak regular doublesplittings. If P−1

1 A1 ≥ P−12 A2 and P−1

1 R1 ≥ P−12 R2, then ρ(W1) ≤ ρ(W2) < 1.

In this paper, we generalize the above results for rectangular or singular matrices.More specifically, we prove that for a proper regular splitting, the condition A† ≥ 0

implies that A† ≥ U † and ρ(A†V ) ≥ ρ(U †V ) = ρ(A†V )

1+ρ(A†V ), and we provide compar-

ison theorems for two proper regular splittings A = U1 − V1 = U2 − V2 of the samematrix A. In particular, we show that if A† ≥ 0 and either V2 ≥ V1 or U †

1 ≥ U †2 , then

ρ(A†V2) ≥ ρ(A†V1). Finally, we provide analogous results for double splittings.The paper is organized as follows. The next section contains notations definitions

and preliminary tools. Section 3 contains the main results.

2 Preliminaries

For a real m×n matrix A, the matrix G satisfying the four equations known as Penroseequations: AG A = A, G AG = G, (AG)T = AG and (G A)T = G A is called theMoore-Penrose inverse of A (BT denotes the transpose of B). It always exists andunique, and is denoted by A†. In case of invertible matrices A, we have A† = A−1.A ∈ R

m×n is non-negative if A ≥ 0, and B ≥ C if B − C ≥ 0. Let L and M becomplementary subspaces of R

n . Let also PL ,M be a projector on L along M . ThenPL ,M A = A if and only if R(A) ⊆ L and APL ,M = A if and only if N (A) ⊆ M .If L ⊥ M , then PL ,M will be denoted by PL . The following properties of A† ([1])will be used in the proofs of the next section: R(AT ) = R(A†); N (AT ) = N (A†);AA† = PR(A); A† A = PR(AT ).

Berman and Plemmons [2] showed that if A = U − V is a proper splitting ofA ∈ R

m×n , then I −U †V is invertible and A† = (I −U †V )−1U †. Similarly, the factU = A + V is a proper splitting implies that I + A†V and I + V A† are invertible, andU † = (I + A†V )−1 A† = A†(I + V A†)−1. (See Theorem 1, [2] and Theorem 3.1,[9] for the respective proofs.) Also, 1 is not an eigenvalue of I − U †V as I − U †V isinvertible. Hence 1 does notlie in the spectra of U †V . Similarly, −1 does not lie in the

123

144 L. Jena et al.

spectra of A†V . The next lemma shows a relation between the eigenvalues of U †Vand A†V .

Lemma 2.1 (Lemma 3.6, [9]) Let A = U − V be a proper splitting of A ∈ Rm×n. Let

μi , 1 ≤ i ≤ s and λ j , 1 ≤ j ≤ s be the eigenvalues of the matrices U †V and A†V ,respectively. Then for every j , we have 1 + λ j = 0. Also, for every i , there exists j

such that μi = λ j1+λ j

and for every j , there exists i such that λ j = μi1−μi

.

The next results deal with non-negativity and spectral radius.

Theorem 2.2 (Theorem 2.20, [15]) If B ≥ 0, then B has a non-negative real eigen-value equal to its spectral radius.

Theorem 2.3 (Theorem 2.21, [15]) If A ≥ B ≥ 0, then ρ(A) ≥ ρ(B).

Theorem 2.4 (Theorem 3.16, [15]) Let X ∈ Rn×n and X ≥ 0. Then ρ(X) < 1 if and

only if (I − X)−1 exists and (I − X)−1 = ∑∞k=0 Xk ≥ 0.

Lemma 2.5 (Corollary 3.2, [8]) If B ≥ 0 and x ≥ 0 is such that Bx − αx ≥ 0, thenα ≤ ρ(B).

Lemma 2.6 (Lemma 2.2, [13]) Let X =(

B CI O

)

≥ 0 and ρ(B + C) < 1. Then

ρ(X) < 1.

3 Main results

In this section, we present few generalizations of convergence and comparison resultsof regular and weak regular splittings. We then proceed to extend the notion of doublesplitting theory. The first result talks about a convergence theorem for the properregular splittings.

Theorem 3.1 Let A = U − V be a proper regular splitting of A ∈ Rm×n. If A† ≥ 0,

then(a) A† ≥ U †;(b) ρ(A†V ) ≥ ρ(U †V );

(c) ρ(U †V ) = ρ(A†V )

1+ρ(A†V )< 1.

Proof Given that A = U − V is a proper regular splitting and A† ≥ 0, so we haveA = U − V is a proper splitting with U † ≥ 0 and V ≥ 0.

(a) The fact A = U − V is a proper splitting yields A† = (I − U †V )−1U † so thatU † = (I − U †V )A†. Therefore A† − U † = U †V A† ≥ 0 i.e., A† ≥ U †.

(b) Post-multiplying V to A† ≥ U †, and then by Lemma 2.3, we get ρ(A†V ) ≥ρ(U †V ).

(c) We have A†V ≥ 0. Let λ be any eigenvalue of A†V . Let f (λ) = λ1+λ

, λ ≥ 0.Then f is a strictly increasing function. Let μ be any eigenvalue of U †V . Then byLemma 2.1, μ = λ

1+λ. So, μ attains its maximum when λ is maximum. But λ is

maximum when λ = ρ(A†V ). As a result, the maximum value of μ is ρ(U †V ).

Hence, ρ(U †V ) = ρ(A†V )

1+ρ(A†V )< 1. �

We next obtain comparison theorems for proper regular splittings.

123

Convergence and comparison theorems 145

Theorem 3.2 Let A = U1 − V1 = U2 − V2 be two proper regular splittings ofA ∈ R

m×n. If A† ≥ 0 and V2 ≥ V1, then

1 > ρ(U †2 V2) ≥ ρ(U †

1 V1).

Proof By Theorem 1.3, we have ρ(U †i Vi ) < 1 for i = 1, 2. Also, A†Vi ≥ 0 for

i = 1, 2 and V2 ≥ V1. Then A†V2 ≥ A†V1 ≥ 0. Let λi be the eigenvalues of A†Vi

for i = 1, 2. Since λ1+λ

is a strictly increasing function for λ ≥ 0, Theorem 2.3 yieldsρ(A†V2) ≥ ρ(A†V1). Hence

ρ(A†V2)

1 + ρ(A†V2)≥ ρ(A†V1)

1 + ρ(A†V1).

�The above result is true for proper weak regular splitting A = U − V . (Since

A†V = (I −U †V )−1U †V , and then using Theorems 1.3 and 2.4, we have A†V ≥ 0.)Note that Theorem 3.2 is also true if one is a proper regular splitting and the other oneis a proper weak regular splitting.

Theorem 3.3 Let A = U1 − V1 = U2 − V2 be two proper regular splittings ofA ∈ R

m×n. If A† ≥ 0 and U †1 ≥ U †

2 , then

1 > ρ(U †2 V2) ≥ ρ(U †

1 V1).

Proof By Theorem 1.3, we have ρ(U †i Vi ) = ρ(ViU

†i ) < 1 for i = 1, 2. Also

ρ(U †i Vi ), i = 1, 2 are strictly monotone increasing functions of ρ(A†Vi ), i = 1, 2,

so it suffices to show that

ρ(A†V2) ≥ ρ(A†V1).

The hypothesis A = Ui − Vi , i = 1, 2 are proper splittings imply I + A†V1

and I + V2 A† are both invertible. Also, both are non-negative. Now, the fact U †1 ≥

U †2 implies A†(I + V2 A†) ≥ (I + A†V1)A† i.e., A†V2 A† ≥ A†V1 A†. Then post-

multiplying by V2, and again by V1, we have

(A†V2)2 ≥ A†V1 A†V2 and A†V2 A†V1 ≥ (A†V1)

2.

Therefore, by Theorem 2.2, we have

ρ2(A†V2) ≥ ρ(A†V1 A†V2) = ρ(A†V2 A†V1) ≥ ρ2(A†V1).

Hence ρ(A†V2) ≥ ρ(A†V1).

123

146 L. Jena et al.

Theorems 3.1 and 3.3 do not hold for proper weak regular splittings unless weassume Vi ≥ 0. Next, we are going to present convergence and comparison results ofdouble splittings for rectangular matrices.

A decomposition A = P − R + S of A ∈ Rm×n is called double proper splitting

if R(A) = R(P) and N (A) = N (P). Applying the double proper splitting A =P − R + S to (1), we now propose the following iterative scheme spanned in threeiterates:

xi+1 = P† Rxi − P†Sxi−1 + P†b, i > 0. (2)

Motivated by Woznicki’s idea ([17]), we now propose

(xi+1

xi

)

=(

P† R −P†SI O

)(xi

xi−1

)

+(

P†bO

)

.

Setting yi+1 =(

xi+1

xi

)

, yi =(

xi

xi−1

)

, W =(

P† R −P†SI O

)

and d =(

P†bO

)

,

we get

yi+1 = W yi + d (3)

Then the above scheme is convergent if ρ(W ) < 1.

Let us introduce some new subclasses of double proper splittings.

Definition 3.4 A double decomposition A = P − R + S is called double properregular splitting if R(A) = R(P), N (A) = N (P), P† ≥ 0, R ≥ 0 and S ≤ 0.

Definition 3.5 A double decomposition A = P − R + S is called double proper weakregular splitting if R(A) = R(P), N (A) = N (P), P† ≥ 0, P† R ≥ 0 and P†S ≤ 0.

The above double decompositions reduce to respective single decompositions bywriting A = P − (R − S) = U − V .

Theorem 3.6 Let A† ≥ 0. If A = P − R + S is a double proper regular splitting (ordouble proper weak regular splitting) of A, then ρ(W ) < 1.

Proof Since A = P−R+S is a double proper regular splitting (or double proper weak

regular splitting) of A, so for both the cases W =(

P† R −P†SI O

)

≥ 0. Setting U = P

and V = R − S, we get A = U − V is a proper regular splitting (or a proper weakregular splitting) of A. Then by Theorem 1.3, we have ρ(P†(R − S)) = ρ(U †V ) < 1.By Lemma 2.6, it now follows that ρ(W ) < 1. �

Before presenting the next result, we would like to recall the notion of semi-monotonicity: an extension of inverse positive matrices to rectangular case. A ∈ R

m×n

is said to be semi-monotone if A† ≥ 0. For invertible matrices A, the notion semi-monotonicity reduces to the notion of monotonicity. (A real square matrix A iscalled

123

Convergence and comparison theorems 147

monotone if A is inverse positive.) In order to study convergence rate of two differentsystems of linear equations by iterative methods, we need to compare the spectralradius of their iteration matrices. The scheme with smaller spectral radius will con-verge faster. Our next result is in this direction.

Theorem 3.7 Let A1 and A2 be two semi-monotone matrices having the same nullspace. Suppose that A1 = P1 − R1 + S1 and A2 = P2 − R2 + S2 be their doubleproper weak regular splittings. If P†

1 A1 ≥ P†2 A2 and P†

1 R1 ≥ P†2 R2, then ρ(W1) ≤

ρ(W2) < 1.

Proof By Theorem 3.6, we have ρ(Wi ) < 1 for i = 1, 2. If ρ(W1) = 0, then ourclaim holds trivially. Suppose that ρ(W1) = 0. Since A1 and A2 possesses doubleproper weak regular splittings, so W1 ≥ 0 and W2 ≥ 0. Now, applying Theorem 2.2to W1, we get W1x = ρ(W1)x i.e.,

P†1 R1x1 − P†

1 S1x2 = ρ(W1)x1

x1 = ρ(W1)x2.

Then, we have

W2x − ρ(W1)x =(

P†2 R2x1 − P†

2 S2x2 − ρ(W1)x1x1 − ρ(W1)x2

)

=(

P†2 R2x1 − P†

2 S2x2 − P†1 R1x1 + P†

1 S1x20

)

=(

(P†2 R2 − P†

1 R1)x1 + 1ρ(W1)

(P†1 S1 − P†

2 S2)x1

0

)

:=(

�

0

)

.

Since A1 and A2 have the same null space, and also possess proper weak regularsplitting, it follows that P†

1 P1 = P†2 P2. As P†

1 R1 ≥ P†2 R2 and 0 < ρ(W1) < 1, so

� = (P†2 R2 − P†

1 R1)x1 + 1

ρ(W1)(P†

1 S1 − P†2 S2)x1

≥ 1

ρ(W1)(P†

2 R2 − P†1 R1)x1 + 1

ρ(W1)(P†

1 S1 − P†2 S2)x1

= 1

ρ(W1)[P†

2 (R2 − S2) − P†1 (R1 − S1)]x1

= 1

ρ(W1)[P†

2 (P2 − A2) − P†1 (P1 − A1)]x1

= 1

ρ(W1)[P†

1 A1 − P†2 A2)]x1.

123

148 L. Jena et al.

The condition P†1 A1 ≥ P†

2 A2 now yields that � > 0. Hence, W2x − ρ(W1)x ≥ 0,i.e. W2x ≥ ρ(W1)x . Thus, by Lemma 2.5, we have ρ(W1) ≤ ρ(W2) < 1. �

The above theorem admits the following Corollary.

Corollary 3.8 (Theorem 3.1, [14]) Let A1 and A2 be two monotone matrices. Supposethat A1 = P1−R1+S1 and A2 = P2−R2+S2 be their weak regular double splittings.If P−1

1 A1 ≥ P−12 A2 and P−1

1 R1 ≥ P−12 R2, then ρ(W1) ≤ ρ(W2) < 1.

The example given below demonstrates that the conditions P†1 A1 ≥ P†

2 A2 and

P†1 R1 ≥ P†

2 R2 can not be dropped.

Example 3.9 Let A1 =(

2 −1 0−3 2 0

)

and A2 =(

1 −1 0−1 2 0

)

. Then A1 and A2 are

semi-monotone matrices having the same null space. Set P1 =(

3 0 00 2 0

)

, R1 =(

1/3 0 03 0 0

)

and S1 =(−2/3 −1 0

0 0 0

)

. Again, P2 =(

2 0 00 2 0

)

, R2 =(

1/2 0 01 0 0

)

and S2 =(−1/2 −1 0

0 0 0

)

. So A1 = P1 − R1 + S1 and A2 = P2 − R2 + S2 are two

double proper weak regular splittings. Then P†1 A1 � P†

2 A2 and P†1 R1 � P†

2 R2, but1 > .9293 = ρ(W1) ≥ ρ(W2) = .8689.

The next example shows that the converse of Theorem 3.7 is not true.

Example 3.10 Let A1 =(

2 −1 0−1 2 0

)

and A2 =(

1 −1 0−1 2 0

)

. Here N (A1) =

N (A2) , A†1 ≥ 0 and A†

2 ≥ 0. Set P1 =(

4 0 00 3 0

)

, R1 =(

4/3 0 01 1 0

)

and S1 =(−2/3 −1 0

0 0 0

)

. Again, P2 =(

2 0 00 2 0

)

, R2 =(

1/2 0 01 0 0

)

and S2 =(−1/2 −1 0

0 0 0

)

.

Therefore A1 = P1 − R1 + S1 and A2 = P2 − R2 + S2 are two double properweak regular splittings. We then have .7829 = ρ(W1) ≤ ρ(W2) = .8689 < 1, butP†

1 A1 � P†2 A2 and P†

1 R1 � P†2 R2.

Acknowledgements The authors thank Professor Paola Favati and the referees for their helpful commentsand suggestions that have led to a much improved presentation.

References

1. Ben-Israel, A., Greville, T.N.E.: Generalized Inverses. Theory and Applications. Springer, New York(2003)

2. Berman, A., Plemmons, R.J.: Cones and iterative methods for best square least squares solutions oflinear systems. SIAM J. Numer. Anal. 11, 145–154 (1974)

3. Climent, J.-J.; Devesa, A.; Perea, C.: Convergence results for proper splittings. In: Mastorakis, N. (ed.)Recent Advances in Applied and Theoretical Mathematics, pp. 39–44. World Scienti1c, Singapore(2000)

4. Climent, J.-J., Perea, C.: Iterative methods for least square problems based on proper splittings. J.Comput. Appl. Math. 158, 43–48 (2003)

123

Convergence and comparison theorems 149

5. Csordas, G., Varga, R.S.: Comparisons of regular splittings of matrices. Numer. Math. 44, 23–35 (1984)6. Elsner, L.: Comparisons of weak regular splittings and multi splitting methods. Numer. Math. 56,

283–289 (1989)7. Li, C.-K., Schneider, H.: Applications of Perron-Frobenius theory to population dynamics. J. Math.

Biol. 44, 450462 (2002)8. Marek, I., Szyld, D.B.: Comparison theorems for weak splittings of bounded operators. Numer. Math.

56, 283–289 (1989)9. Mishra, D., Sivakumar, K.C.: Comparison theorems for a subclass of proper splittings of matrices.

Appl. Math. Lett. 25, 2339–2343 (2012)10. Neumann, M.: 3-part splittings for singular and rectangular linear systems. J. Math. Anal. Appl. 64,

297–318 (1978)11. Ortega, J.M., Rheinboldt, W.C.: Monotone iterations for nonlinear equations with application to Gauss-

Seidel methods. SIAM J. Numer. Anal. 4, 171–190 (1967)12. Schröder, J.: Operator Inequalities. Academic Press, New York (1980)13. Shen, S.-Q., Huang, T.-Z.: Convergence and comparison theorems for double splittings of matrices.

Comput. Math. Appl. 51, 1751–1760 (2006)14. Shu-Xin, M., Bing, Z.: A note on double splittings of monotone matrices. Calcolo 46, 261–266 (2009)15. Varga, R.S.: Matrix Iterative Analysis. Springer, Berlin (2000)16. Woznicki, Z.I.: Matrix splitting principles. Novi Sad J. Math. 28, 197–209 (1998)17. Woznicki, Z.I.: Estimation of the optimum relaxation factors in partial factorization iterative methods.

SIAM J. Matrix Anal. Appl. 14, 59–73 (1993)

123