School of Mechanical EngineeringChung-Ang UniversityNumerical Methods 2010-2

Iterative MethodsIterative Methods

Prof. HaeProf. Hae--Jin ChoiJin Choi

hjchoi@cau.ac.krhjchoi@cau.ac.kr

Part 3Part 3Chapter 12Chapter 12

1

School of Mechanical EngineeringChung-Ang UniversityNumerical Methods 2010-2

Chapter ObjectivesChapter Objectives

2

l Understanding the difference between the Gauss-Seidel and Jacobi methods.

l Knowing how to assess diagonal dominance and knowing what it means.

l Recognizing how relaxation can be used to improve convergence of iterative methods.

l Understanding how to solve systems of nonlinear equations with successive substitution and Newton-Raphson.

School of Mechanical EngineeringChung-Ang UniversityNumerical Methods 2010-2

GaussGauss--Seidel MethodSeidel Method

3

l The Gauss-Seidel method is the most commonly used iterative method for solving linear algebraic equations [A]{x}={b}.

l For a 3x3 system with nonzero elements along the diagonal, for example, the jth iteration values are found from the j-1th

iteration using:

�

x1j =b1 - a12x2

j-1 - a13x3j-1

a11

x2j =b2 - a21x1

j - a23x3j-1

a22

x3j =b3 - a31x1

j - a32x2j

a33

11 12 13 1 1

21 22 23 2 2

31 23 33 3 3

a a a x ba a a x ba a a x b

é ù ì ü ì üï ï ï ïê ú =í ý í ýê úï ï ï ïê úë û î þ î þ

11 1 1 12 2 13 3

22 1 2 21 1 23 3

33 1 3 31 1 32 2

a x b a x a xa x b a x a xa x b a x a x

= - -

= - -

= - -

School of Mechanical EngineeringChung-Ang UniversityNumerical Methods 2010-2

Jacobi IterationJacobi Iteration

4

l The Jacobi iteration is similar to the Gauss-Seidel method, except the j-1th information is used to update all variables in the jthiteration:a) Gauss-Seidelb) Jacobi

1 11 12 2 13 3

1111 1

2 21 1 23 32

221 1

3 31 1 32 23

33

j jj

j jj

j jj

b a x a xxa

b a x a xxa

b a x a xxa

- -

- -

- -

- -=

- -=

- -=

School of Mechanical EngineeringChung-Ang UniversityNumerical Methods 2010-2

ConvergenceConvergence

5

l The convergence of an iterative method can be calculated by determining the relative percent change of each element in {x}. For example, for the ith element in the jth iteration,

l The method is ended when all elements have converged to a set tolerance. �

ea,i =xij - xi

j-1

xij ´100%

School of Mechanical EngineeringChung-Ang UniversityNumerical Methods 2010-2

Example 12.1Example 12.1

6

Q. Use the Gauss-Seidel Method to solve this set of equations.

4.71 10 .203.03.193.07 1.085.7 0.21.03

321

321

321

=+--=-+

=--

xxxxxxxxx

[ ]{ } 3 2.5 7Tx = -Note : True solution is

Assume that x2 and x3 are zero in the first computation.

32.01.085.7 32

1xxx ++

=7

3.01.03.19 312

xxx +--=

102.03.04.71 21

3xxx +-

=

School of Mechanical EngineeringChung-Ang UniversityNumerical Methods 2010-2 7

1 12 3

17.85 0.1 0.2

3

j jj x xx

- -+ +=

11 3

219.3 0.1 0.3

7

j jj x xx

-- - +=

1 23

71.4 0.3 0.210

j jj x xx - +

=

616667.23

)0(2.0)0(1.085.71 =

++=x

794524.27

)0(3.0)616667.2(1.03.192 -=

+--=x

005610.710

)794524.2(2.0)616667.2(3.04.713 =

-+-=x

1st iteration

School of Mechanical EngineeringChung-Ang UniversityNumerical Methods 2010-2 8

990557.23

)005610.7(2.0)794524.2(1.085.71 =

+-+=x

499625.27

)005610.7(3.0)990557.2(1.03.192 -=

+--=x

000291.710

)499625.2(2.0)990557.2(3.04.713 =

-+-=x

2nd iteration

%5.12%100990557.2

616667.2990557.21, =´

-=ea

ea,2 = 11.8%; ea,3 = 0.076%;

School of Mechanical EngineeringChung-Ang UniversityNumerical Methods 2010-2

Diagonal DominanceDiagonal Dominance

9

l The Gauss-Seidel method may diverge, but if the system is diagonally dominant, it will definitely converge.

l Diagonal dominance means:

l Many engineering problems satisfy this requirement

�

aii > aijj=1j¹i

n

å

School of Mechanical EngineeringChung-Ang UniversityNumerical Methods 2010-2

MATLAB ProgramMATLAB Program

10

new2

33

32new1

33

31

33

3new3

old3

22

23new1

22

21

22

2new2

old3

11

13old2

11

12

11

1new1

xaax

aa

abx

xaax

aa

abx

xaax

aa

abx

--=

--=

--=

MATLAB M-file: GaussSeidel

}]{[}{}{ xCdx -=

ïþ

ïý

ü

ïî

ïí

ì=

333

222

111

///

}{ababab

dúúú

û

ù

êêê

ë

é=

0///0///0

][

33323331

22232221

11131112

aaaaaaaaaaaa

C

School of Mechanical EngineeringChung-Ang UniversityNumerical Methods 2010-2 11

School of Mechanical EngineeringChung-Ang UniversityNumerical Methods 2010-2

RelaxationRelaxation

12

l To enhance convergence, an iterative program can introduce relaxation where the value at a particular iteration is made up of a combination of the old value and the newly calculated value (update for the new one):

where λ is a weighting factor that is assigned a value between 0 and 2.

l 0< λ <1: underrelaxationl λ =1: no relaxationl 1< λ ≤2: overrelaxation

�

xinew = lxi

new + 1- l( )xiold

School of Mechanical EngineeringChung-Ang UniversityNumerical Methods 2010-2

Nonlinear SystemsNonlinear Systems

13

lNonlinear systems can also be solved using the same strategy as the Gauss-Seidel method - solve each system for one of the unknowns and update each unknown using information from the previous iteration.

lThis is called successive substitution.

School of Mechanical EngineeringChung-Ang UniversityNumerical Methods 2010-2

Example 12.2Example 12.2

14

Q. Use successive substitution to determine the roots ofthe following equation. A correct pair of roots is x1 = 2 and x2 = 3. Use the initial guesses of x1 = 1.5 and x2 = 3.5.

573

102212

2121

=+

=+

xxx

xxx

First iteration

2

21

110xxx -

= 2212 357 xxx -=

21429.25.3

)5.1(10 2

1 =-

=x 37516.24)5.3)(21429.2(357 22 -=-=x

Second iteration

20910.037516.24

)21429.2(10 2

1 -=--

=x 709.429)37516.24)(20910.0(357 22 =---=x

Seems to be diverging

School of Mechanical EngineeringChung-Ang UniversityNumerical Methods 2010-2 15

Use the same equation but with a different format

211 10 xxx -=1

22 3

57xxx -

=

First iteration

Second iteration

17945.2)5.3(5.1101 =-=x 86051.2)17945.2(3

5.3572 =

-=x

94053.1)86051.2(17945.2101 =-=x04955.3

)94053.1(386051.257

2 =-

=x

The approach is converging on the true values.

à The most serious shortcoming of substitution, which depends on the manner in which the equations are formulated.

School of Mechanical EngineeringChung-Ang UniversityNumerical Methods 2010-2

NewtonNewton--RaphsonRaphson

16

l Nonlinear systems may also be solved using the Newton-Raphson method for multiple variables.

l For a one-variable system, the Taylor series approximation and resulting Newton-Raphson equations are:

l For a two-variable system, )()()()( 11 iiiii xfxxxfxf ¢-+= ++ 1

( )( )i

i ii

f xx xf x+ = -¢

�

f1,i+1 = f1,i + x1,i+1 - x1,i( )¶f1,i

¶x1

+ x2,i+1 - x2,i( )¶f1,i

¶x2

x1,i+1 = x1,i -f1,i

¶f2,i

¶x2

- f2,i¶f1,i

¶x2

¶f1,i

¶x1

¶f2,i

¶x2

-¶f1,i

¶x2

¶f2,i

¶x1

f2,i+1 = f2,i + x1,i+1 - x1,i( )¶f2,i

¶x1

+ x2,i+1 - x2,i( )¶f2,i

¶x2

x2,i+1 = x2,i -f2,i

¶f1,i

¶x1

- f1,i¶f2,i

¶x1

¶f1,i

¶x1

¶f2,i

¶x2

-¶f1,i

¶x2

¶f2,i

¶x1

School of Mechanical EngineeringChung-Ang UniversityNumerical Methods 2010-2 17

1[ ]{ } { } i iZ x x f+ - = -

úúúúúúúú

û

ù

êêêêêêêê

ë

é

¶

¶

¶

¶

¶

¶

¶

¶

¶

¶

¶

¶¶

¶

¶

¶

¶

¶

=

n

ininin

n

iii

n

iii

xf

xf

xf

xf

xf

xf

xf

xf

xf

Z

,

2

,

1

,

,2

2

,2

1

,2

,1

2

,1

1

,1

][

L

MMM

L

L

1, 2, ,{ }Ti i i n ix x x xé ù= ë ûL

1 1, 1 2, 1 , 1{ }Ti i i n ix x x x+ + + +é ù= ë ûL

1, 2, ,{ }T i i n if f f fé ù= ë ûL

( ) ( )

( ) ( )

1, 1,1, 1 1, 1, 1 1, 2, 1 2,

1 2

2, 2,2, 1 2, 1, 1 1, 2, 1 2,

1 2

i ii i i i i i

i ii i i i i i

f ff f x x x x

x xf f

f f x x x xx x

¶ ¶¶ ¶¶ ¶¶ ¶

+ + +

+ + +

= + - + -

= + - + -

1[ ]{ } { } [ ]{ }, where [ ] i iZ x f Z x Z Jacobian matrix+ = - + =

School of Mechanical EngineeringChung-Ang UniversityNumerical Methods 2010-2 18

School of Mechanical EngineeringChung-Ang UniversityNumerical Methods 2010-2

Example 12.3 (1/2)Example 12.3 (1/2)

19

Q. Use the Newton-Raphson method to determine theroots of the equations. Use the initial guesses of x1 =1.5 and x2 = 3.5.

573

102212

2121

=+

=+

xxx

xxx

5.32)5.3)(5.1(6161 75.36)5.3(33

5.1 5.65.3)5.1(22

212

0,2222

1

0,2

12

0,121

1

0,1

=+=+=¶

¶===

¶

¶

==¶

¶=+=+=

¶

¶

xxxf

xxf

xxf

xxxf

6.5(32.5) 1.5(36.75) 156.125Jacobian = - =

School of Mechanical EngineeringChung-Ang UniversityNumerical Methods 2010-2

Example 12.3 (2/2)Example 12.3 (2/2)

20

5.210)5.3(5.1)5.1( 20,1 -=-+=f

625.157)5.3)(5.1(35.3 20,2 =-+=f

The values of the functions can be evaluated at the initial guesses as

These values can be substituted to give

03603.2125.156

)5.1(625.1)5.32(5.25.11 =--

-=x

84388.2125.156

)75.36)(5.2()5.6(625.15.32 =--

-=x

The computation can be repeated until an acceptable accuracyis obtained.

School of Mechanical EngineeringChung-Ang UniversityNumerical Methods 2010-2

MATLAB ProgramMATLAB Program

21