Iterative Methods for Systems of Linear Equations

8/9/2019 Iterative Methods for Systems of Linear Equations

http://slidepdf.com/reader/full/iterative-methods-for-systems-of-linear-equations 1/17

Iterative methodsfor systems of linear

equationsAstrid Xiomara

Rodriguezwww.nebrija.es/.../MetodosMatem/sistemas_lin

eales_iterativos

Iterative methodsfor systems of linear

equationsAstrid Xiomara

Rodriguezwww.nebrija.es/.../MetodosMatem/sistemas_lin

eales_iterativos



Iterative methods for systems

of linear equations

Iterative methods for systems

of linear equations

Introduction

Heat EquationJacobi Method

Gauss-Seidel Method

Overrelaxation methodCapacitor Problem



Direct methods versus

iterative methods

Direct methods versus

iterative methodsDIRECT

Ax =b

x = A\ b

Moderate size

Alter the structure

Rounding error

ITERATIVE

x = Cx + d

x(k+1) = Cx(k) + d

Big size

Zeros preservedTruncation error



Iterative methods compared with direct, we donot guarantee a better approach, however, aremore efficient when working with large matrices

In the resolution for numerical partial differentialequations often appear linear equation systemswith even 100 000 unknowns, in these systemsthe coefficient matrix is sparse, ie a highpercentage of matrix elements are equal to 0. If

there are any patterns in the nonzero elements(eg tridiagonal systems), then an iterativemethod can be very effective.



Heat Equation

Heat Equation

System Eq. linear. Associated matrix

T (T T ) / 2T (T T ) / 2

T (T T ) / 2

T (T T ) / 2

1 0 2

2 1 3

3 2 4

n n-1 n+1

!

!

!

!

/

2 -

- 2 -

- 2

-

- 2

1

1 1

¨

ª

©©©©

©©

¸

º

¹¹¹¹

¹¹

T0 T1 T2 . . . Tn Tn+1



J acobi methodJ acobi method System of linear equations

a x

a x

a x

a x

a x a x a x

a x a x a x

a x a x a x

a x a x a x

b

b

b

b

11 1

21 1

31 1

n1 n

12 2 13 3 1n n

22 2 23 3 2n n

32 2 33 3 3n n

n2 2 n3 3 nn n

1

2

3

n

!

!

!

!

¾

¿

±±±

À

±±±

/

.

.

.

/ / /

.

/



Of all the iterative methods, Jacobi's is theeasiest to implement and understand, however,

is not very efficient in terms of obtainingsolutions.Consider the system:

we can write equivalently as:

Jacobi's method is to use the above formulas asfixed point iteration.

±°

±¯

®

!

!

!

1552

2184

74

z y x

z y x

z y x

±±±

°

±±±

¯

®

!

!

!

5

2158

421

4

7

y x z

z x y

z y x



Fixed point equationFixed point equation

x (bx (b

x (b

x (b

a x a x a x ) / aa x a x a x ) / a

a x a x a x ) / a

a x a x a x ) / a

1 1

1 2

2 3

n n

12 2 13 3 1n n 11

21 1 23 3 2n n 22

31 1 32 2 3n n 33

n1 1 n2 2 n,n 1 n 1 nn

! !

!

!

¾

¿

±±±

À

±±±

/

.

.

.

/ / 1 /

.



J acobi IterationJ acobi Iterationx (b

x (b

x (b

x (b

a x a x a x ) /a

a x a x a x ) /a

a x a x a x ) /a

a x a x a x )/a

1

(k+1)

1

2

(k+1)

2

3

(k+1)

3

n

(k+1)

n

12 2

(k)

13 3

(k)

1n n

(k)

11

21 1

(k)

23 3

(k)

2n n

(k)

22

31 1

(k)

32 2

(k)

3n n

(k)

33

n1 1

(k)

n2 2

(k)

n,n 1 n 1

(k)

nn

!

!

!

!

¾

¿

À

/

.

.

.

/ / 1 /

.



At each step of the Jacobi iteration yields a vectorwith n coordinates

P0 = (x1 (0), x2 (0), ..., xn (0)), ..., Pk = (x1 (k), x2(k), ..., xn (k))

where the initial estimate (x1 (0), x2 (0), ..., xn (0))should be chosen. When you do not have a clue aboutthe solution is usually taken xi (0) = bi / aii

In the example above, if we take P0 = (x (0), y (0), z(0)) = (1,2,2)In the first iteration is obtained

P1 = (1.75, 3375, 3.00)

Generating the sequence of iterations of Jacobi notesthat converges to (2, 4, 3).



Many times the Jacobi iteration method does notwork. Here is an example of rearranging theequations above example.Example:

Now the iteration formula is

and notes that the sequence of Jacobi diverges.Note that the system matrix is not strictlydiagonal dominant.

±

°

±¯

!

!

!

74

2184

1552

z y x

z y x

z y x

±±

±

°

±±±

¯

¡

!

!

!

y x z

z y y

z y x

47

8

421

2

515



G auss-Seidel iterationG auss-Seidel iteration

x (b

x (b

x (b

x (b

a x a x a x )/a

a x a x a x )/a

a x a x a x )/a

a x a x a x )/a

1

(k+1)

1

2

(k+1)

2

3

(k+1)

3

n

(k+1)

n

12 2

(k)

13 3

(k)

1n n

(k)

11

21 1

(k+1)

23 3

(k)

2n n

(k)

22

31 1

(k+1)

32 2

(k+1)

3n n

(k)

33

n1 1

(k+1)

n2 2

(k+1)

n,n 1 n 1

(k+1)

nn

!

!

!

!

¾

¿

±

/

.

.

.

/ / /

.

±±

À

±±±



The Gauss-Seidel method is a modification of theJacobi method which accelerates the convergence of the latter.Note that the Jacobi method generates a sequence foreach unknown(X1 (k)), ..., (xn (k)). Since xi (k +1) is probablybetter approximated by xi (k) instead of xi (k) in thecalculation of xi +1 (k +1) we use xi (k +1).Apply this strategy to the example 1 and comprubesethe speed of convergence.

The Gauss-Seidel substantially cut the number of iterations to make for some precision in thesolution. Obviously the convergence criteria aresimilar to those of Jacobi.



Overrelaxation method

Overrelaxation method

1)+(k

i

(k)

i

1)+(k

i

(k)

i

1)+(k

ii

i

(k)

i

1)+(k

i

i(k)i

1)+(ki

xww)x(1x

xxz

2<w<0 ;wzxx

:tionIverrelaxa

zxx

:SeidelGauss

Ö

Ö

Ö

!

!

!

!

xik

zi xik+1

ik+1x



Overrelaxation method reducesthe number of iterations in thecalculations of solutions of linear

systems by Gauss-Seidel. It isbased on each iteration to obtain aweighted average (only for vectorelements before the position

calculation) for the solution of theJacobi method and the solution of the Gauss-Seidel.



P ass

Overrelaxation

P ass

Overrelaxation

x (1 )x (b

x (1 )x (b

x (1 )x (b

x (1 )x (b

a x a x a x )/a

a x a x a x ) /a

a x a x a x ) /a

1

(k+1)

1

(k)

1

2

(k+1)

2

(k)

2

3

(k+1)

3

(k)

3

n

(k+1)

n

(k)

n

12 2

(k)

13 3

(k)

1n n

(k)

11

21 1

(k+1)

23 3

(k)

2n n

(k)

22

31 1

(k+1)

32 2

(k+1)

3n n

(k)

33

!

!

!

!

[ [

[ [

[ [

[ [

/

.

.

.

/ / 1 /

.a x a x a x )/an1 1

(k+1)

n2 2

(k+1)

n,n 1 n 1

(k+1)

nn

¾

¿

±±±±

À

±

±±±



SummarySummary Iterative methods are applied to large

and sparse matrices.

The cost per iteration is O (n2) or less if you take advantage of the dispersityAre expected to converge in less than nsteps.

The matrix has to fulfill certainconditions for the method to converge.

Documents

Iterative Methods for Systems of Linear Equations