SOLVING SYSTEMS OF EQUATIONS, ITERATIVE METHODS SOLVING SYSTEMS OF EQUATIONS, ITERATIVE METHODS ELM1222

  • View
    2

  • Download
    0

Embed Size (px)

Text of SOLVING SYSTEMS OF EQUATIONS, ITERATIVE METHODS SOLVING SYSTEMS OF EQUATIONS, ITERATIVE METHODS...

  • SOLVING SYSTEMS OF EQUATIONS,

    ITERATIVE METHODS

    ELM1222 Numerical Analysis

    1

    Some of the contents are adopted from

    Laurene V. Fausett, Applied Numerical Analysis using MATLAB. Prentice Hall Inc., 1999

    ELM1222 Numerical Analysis | Dr Muharrem Mercimek

  • Today’s lecture

    • 3 Common classical iterative techniques for linear equation systems

    • Jacobi Method

    • Gauss- Seidel Method

    • Successive Over Relaxation

    2 ELM1222 Numerical Analysis | Dr Muharrem Mercimek

  • Iterative techniques for linear equation systems

    • Systems of linear equations for which numerical solutions are needed are

    often very large.

    • Using general methods such as Gauss elimination is computationally

    expensive.

    • If the coefficient system is having a specific structure iterative techniques are

    preferable.

    3 ELM1222 Numerical Analysis | Dr Muharrem Mercimek

  • Iterative techniques for linear equation systems

    4 ELM1222 Numerical Analysis | Dr Muharrem Mercimek

    For large, sparse systems (many coefficients whose value is zero)

    iterative techniques are preferable.

  • What is an equation System?

    5 ELM1222 Numerical Analysis | Dr Muharrem Mercimek

    When you have

    derive the equivalent system

    and solve it

    • Generate a sequence of approximation , where

    dCxx kk   )1()(

    ,..., )2()1( xx

    3333132131

    2323122121

    1313212111

    bxaxaxa

    bxaxaxa

    bxaxaxa

    

    

    

    33

    3 2

    33

    32 1

    33

    31 3

    22

    2 1

    22

    23 1

    22

    21 2

    11

    1 3

    11

    13 2

    11

    12 1

    a

    b x

    a

    a x

    a

    a x

    a

    b x

    a

    a x

    a

    a x

    a

    b x

    a

    a x

    a

    a x

    

    

    

    BAx 

    dCxx 

  • • A 2x2 equation system

    • 𝐀 = 2 1 1 2

    𝐱 = 𝑥 𝑦 𝐛 =

    6 6

    • Start with

    • Simultaneous updating

    • New values of the variables are not used until a new iteration step is begun

    Jacobi Method

    6 ELM1222 Numerical Analysis | Dr Muharrem Mercimek

    4

    11 3

    4

    1 3

    2

    1

    4

    11 3

    4

    1 3

    2

    1

    )0()1(

    )0()1(

    

    

    xy

    yx

    62

    62

    

    

    yx

    yx

    3 2

    1

    3 2

    1

    

    

    xy

    yx

    2/1)0()0(  yx

    𝐀𝐱 = 𝐛 Example 1:

    𝐱 = 𝐂𝐱 + 𝐝

    𝐂 has zeros in the diagonal

  • Jacobi Method

    7 ELM1222 Numerical Analysis | Dr Muharrem Mercimek

    𝐀 = 2 1 1 2

    𝐱 = 𝑥 𝑦 𝐛 =

    6 6

    4

    11 3

    4

    1 3

    2

    1

    4

    11 3

    4

    1 3

    2

    1

    )0()1(

    )0()1(

    

    

    xy

    yx

    2/1)0()0(  yx

    Stopping Criteria:

    Stop the iterations when

    • The function 𝐀𝐱 (𝑘) − 𝐛 𝟐

    less than a “𝑡𝑜𝑙” value.( . 𝟐 is 2-Norm or Euclidean Norm)

    • The max number of iterations “𝑚𝑎𝑥 _𝑖𝑡𝑒𝑟” has reached

    8

    13 3

    8

    11 3

    2

    1

    8

    13 3

    8

    11 3

    2

    1

    )1()2(

    )1()2(

    

    

    xy

    yx

  • Jacobi Method

    8 ELM1222 Numerical Analysis | Dr Muharrem Mercimek

    Example 2: Consider the 3x3 system

    • Start with

    32

    62

    12

    321

    321

    321

    

    

    

    xxx

    xxx

    xxx

    213

    12

    1

    5.05.0

    5.0

    xxx

    xx

    x

    

    

    5.1

    0.35.0

    5.05.05.0

    3

    32

    

    

    x

    xx

      

      

      

      

      

      

      

      

    5.1

    0.3

    5.0

    0.05.05.0

    5.00.05.0

    5.05.00.0

    )0( 3

    )0( 2

    )0( 1

    )1( 3

    )1( 2

    )1( 1

    x

    x

    x

    x

    x

    x

    )0,0,0()0( x

      

      

      

      

      

      

      

      

    5.1

    0.3

    5.0

    5.1

    3

    5.0

    0.05.05.0

    5.00.05.0

    5.05.00.0

    )2( 3

    )2( 2

    )2( 1

    x

    x

    x The method converges in 13 iterations

    𝐱 (13) = [1.002 2.001 − 0.9997] T

  • Example 3:

    A necessary and sufficient condition for the convergence of the Jacobi method

    “the magnitude of the largest eigenvalue of the iteration matrix C be less than 1”

    A necessary condition (not sufficient) for the convergence of the Jacobi method

    “ A should be diagonally dominant. Magnitude of diagonal element should be greater than

    sum f magnitudes of other elements of the row

    9 ELM1222 Numerical Analysis | Dr Muharrem Mercimek

    𝐀 = 1 2 2 1

    𝐱 = 𝑥 𝑦 𝐛 =

    6 6

    62

    62

    

    

    yx

    yx

    62

    62

    

    

    xy

    yx

    56162

    56162

    )0()1(

    )0()1(

    

    

    xy

    yx

    • When you run the other iterations you will see it diverges

    Jacobi Method

  • • A 2x2 equation system

    • 𝐀 = 2 1 1 2

    𝐱 = 𝑥 𝑦 𝐛 =

    6 6

    • Start with

    • Sequential updating

    • New values of the variables are used in the same iteration step

    Gauss-Seidel Method

    10 ELM1222 Numerical Analysis | Dr Muharrem Mercimek

    8

    13 3

    8

    11 3

    2

    1

    4

    11 3

    4

    1 3

    2

    1

    )1()1(

    )0()1(

    

    

    xy

    yx

    62

    62

    

    

    yx

    yx

    3 2

    1

    3 2

    1

    

    

    xy

    yx

    2/1)0()0(  yx

    𝐀𝐱 = 𝐛

    Example 4:

    32

    61 3

    32

    35 3

    2

    1

    16

    35 3

    16

    13 3

    2

    1

    )2()2(

    )1()2(

    

    

    xy

    yx

  • 11

    • Example 5: Consider the three-by-three system

    • Start with

    • After 10 iterations 𝐱=[1.0001 1.9999 -1.0001]

    32

    62

    12

    321

    321

    321

    

    

    

    xxx

    xxx

    xxx

     

       

         newnewnew

    newnew

    new

    xxx

    xx

    x

    213

    12

    1

    5.05.0

    5.0

    

    

    5.1

    0.35.0

    5.05.05.0

    )( 3

    )( 3

    )( 2

    

    

    old

    oldold

    x

    xx

    )0,0,0()0( x

    Gauss-Seidel Method

    11 ELM1222 Numerical Analysis | Dr Muharrem Mercimek

    Stopping Criteria:

    Stop the iterations when

    • The function 𝐀𝐱 (𝑘) − 𝐛 𝟐

    less than a “𝑡𝑜𝑙” value.( . 𝟐 is 2-Norm or Euclidean Norm)

    • The max number of iterations “𝑚𝑎𝑥 _𝑖𝑡𝑒𝑟” has reached

  • • Discussion

    • The Gauss-Seidel method is sensitive to the form of the coefficient

    matrix A

    • The Gauss-Seidel method typically converges more rapidly than the

    Jacobi method

    • The Gauss-Seidel method is more difficult to use for parallel

    computation

    12

    Gauss-Seidel Method

    12 ELM1222 Num

View more >