SOLVING SYSTEMS OF EQUATIONS, ITERATIVE METHODS SOLVING SYSTEMS OF EQUATIONS, ITERATIVE METHODS ELM1222

• View
2

0

Embed Size (px)

Text of SOLVING SYSTEMS OF EQUATIONS, ITERATIVE METHODS SOLVING SYSTEMS OF EQUATIONS, ITERATIVE METHODS...

• SOLVING SYSTEMS OF EQUATIONS,

ITERATIVE METHODS

ELM1222 Numerical Analysis

1

Some of the contents are adopted from

Laurene V. Fausett, Applied Numerical Analysis using MATLAB. Prentice Hall Inc., 1999

ELM1222 Numerical Analysis | Dr Muharrem Mercimek

• Today’s lecture

• 3 Common classical iterative techniques for linear equation systems

• Jacobi Method

• Gauss- Seidel Method

• Successive Over Relaxation

2 ELM1222 Numerical Analysis | Dr Muharrem Mercimek

• Iterative techniques for linear equation systems

• Systems of linear equations for which numerical solutions are needed are

often very large.

• Using general methods such as Gauss elimination is computationally

expensive.

• If the coefficient system is having a specific structure iterative techniques are

preferable.

3 ELM1222 Numerical Analysis | Dr Muharrem Mercimek

• Iterative techniques for linear equation systems

4 ELM1222 Numerical Analysis | Dr Muharrem Mercimek

For large, sparse systems (many coefficients whose value is zero)

iterative techniques are preferable.

• What is an equation System?

5 ELM1222 Numerical Analysis | Dr Muharrem Mercimek

When you have

derive the equivalent system

and solve it

• Generate a sequence of approximation , where

dCxx kk   )1()(

,..., )2()1( xx

3333132131

2323122121

1313212111

bxaxaxa

bxaxaxa

bxaxaxa







33

3 2

33

32 1

33

31 3

22

2 1

22

23 1

22

21 2

11

1 3

11

13 2

11

12 1

a

b x

a

a x

a

a x

a

b x

a

a x

a

a x

a

b x

a

a x

a

a x







BAx 

dCxx 

• • A 2x2 equation system

• 𝐀 = 2 1 1 2

𝐱 = 𝑥 𝑦 𝐛 =

6 6

• Simultaneous updating

• New values of the variables are not used until a new iteration step is begun

Jacobi Method

6 ELM1222 Numerical Analysis | Dr Muharrem Mercimek

4

11 3

4

1 3

2

1

4

11 3

4

1 3

2

1

)0()1(

)0()1(





xy

yx

62

62





yx

yx

3 2

1

3 2

1





xy

yx

2/1)0()0(  yx

𝐀𝐱 = 𝐛 Example 1:

𝐱 = 𝐂𝐱 + 𝐝

𝐂 has zeros in the diagonal

• Jacobi Method

7 ELM1222 Numerical Analysis | Dr Muharrem Mercimek

𝐀 = 2 1 1 2

𝐱 = 𝑥 𝑦 𝐛 =

6 6

4

11 3

4

1 3

2

1

4

11 3

4

1 3

2

1

)0()1(

)0()1(





xy

yx

2/1)0()0(  yx

Stopping Criteria:

Stop the iterations when

• The function 𝐀𝐱 (𝑘) − 𝐛 𝟐

less than a “𝑡𝑜𝑙” value.( . 𝟐 is 2-Norm or Euclidean Norm)

• The max number of iterations “𝑚𝑎𝑥 _𝑖𝑡𝑒𝑟” has reached

8

13 3

8

11 3

2

1

8

13 3

8

11 3

2

1

)1()2(

)1()2(





xy

yx

• Jacobi Method

8 ELM1222 Numerical Analysis | Dr Muharrem Mercimek

Example 2: Consider the 3x3 system

32

62

12

321

321

321







xxx

xxx

xxx

213

12

1

5.05.0

5.0

xxx

xx

x





5.1

0.35.0

5.05.05.0

3

32





x

xx

  

  

  

  

  

  

  

  

5.1

0.3

5.0

0.05.05.0

5.00.05.0

5.05.00.0

)0( 3

)0( 2

)0( 1

)1( 3

)1( 2

)1( 1

x

x

x

x

x

x

)0,0,0()0( x

  

  

  

  

  

  

  

  

5.1

0.3

5.0

5.1

3

5.0

0.05.05.0

5.00.05.0

5.05.00.0

)2( 3

)2( 2

)2( 1

x

x

x The method converges in 13 iterations

𝐱 (13) = [1.002 2.001 − 0.9997] T

• Example 3:

A necessary and sufficient condition for the convergence of the Jacobi method

“the magnitude of the largest eigenvalue of the iteration matrix C be less than 1”

A necessary condition (not sufficient) for the convergence of the Jacobi method

“ A should be diagonally dominant. Magnitude of diagonal element should be greater than

sum f magnitudes of other elements of the row

9 ELM1222 Numerical Analysis | Dr Muharrem Mercimek

𝐀 = 1 2 2 1

𝐱 = 𝑥 𝑦 𝐛 =

6 6

62

62





yx

yx

62

62





xy

yx

56162

56162

)0()1(

)0()1(





xy

yx

• When you run the other iterations you will see it diverges

Jacobi Method

• • A 2x2 equation system

• 𝐀 = 2 1 1 2

𝐱 = 𝑥 𝑦 𝐛 =

6 6

• Sequential updating

• New values of the variables are used in the same iteration step

Gauss-Seidel Method

10 ELM1222 Numerical Analysis | Dr Muharrem Mercimek

8

13 3

8

11 3

2

1

4

11 3

4

1 3

2

1

)1()1(

)0()1(





xy

yx

62

62





yx

yx

3 2

1

3 2

1





xy

yx

2/1)0()0(  yx

𝐀𝐱 = 𝐛

Example 4:

32

61 3

32

35 3

2

1

16

35 3

16

13 3

2

1

)2()2(

)1()2(





xy

yx

• 11

• Example 5: Consider the three-by-three system

• After 10 iterations 𝐱=[1.0001 1.9999 -1.0001]

32

62

12

321

321

321







xxx

xxx

xxx

 

   

     newnewnew

newnew

new

xxx

xx

x

213

12

1

5.05.0

5.0





5.1

0.35.0

5.05.05.0

)( 3

)( 3

)( 2





old

oldold

x

xx

)0,0,0()0( x

Gauss-Seidel Method

11 ELM1222 Numerical Analysis | Dr Muharrem Mercimek

Stopping Criteria:

Stop the iterations when

• The function 𝐀𝐱 (𝑘) − 𝐛 𝟐

less than a “𝑡𝑜𝑙” value.( . 𝟐 is 2-Norm or Euclidean Norm)

• The max number of iterations “𝑚𝑎𝑥 _𝑖𝑡𝑒𝑟” has reached

• • Discussion

• The Gauss-Seidel method is sensitive to the form of the coefficient

matrix A

• The Gauss-Seidel method typically converges more rapidly than the

Jacobi method

• The Gauss-Seidel method is more difficult to use for parallel

computation

12

Gauss-Seidel Method

12 ELM1222 Num

Documents
Documents
Documents
Documents
Documents
Documents
Documents
Documents
Documents
Documents
Documents
Documents