Embed Size (px)
Citation preview
12/7/2011
1
Titus Beu 2011
Titus BeuUniversity “Babes-Bolyai”Department of Theoretical and Computational PhysicsCluj-Napoca, Romania
8. 8. Systems of Linear Algebraic EquationsSystems of Linear Algebraic Equations
Titus Beu 2011
Bibliography
Introduction
Gaussian elimination method
Gauss-Jordan elimination method
Systems of linear equations with tridiagonal matrix
12/7/2011
2
Titus Beu 2011
Gh. Dodescu, Metode numerice în algebră, Editura tehnică, Bucureşti, 1979).
B.P. Demidovich şi I.A. Maron, Computational Mathematics (MIR Publishers, Moskow, 1981).
R.L. Burden şi J.D. Faires, Numerical Analysis, Third Edition (Prindle, Weber & Schmidt, Boston, 1985).
W.H. Press, S.A. Teukolsky, W.T. Vetterling şi B.P. Flannery, Numerical Recipes in C: The Art of Scientific Computing, Second Edition (Cambridge University Press, Cambridge, 1992).
Beu, T. A., Numerical Calculus in C, Third Edition(MicroInformatica Publishing House, Cluj-Napoca, 2004).
Titus Beu 2011
Solving systems of linear equations plays a central role in numerical analysis.
Direct methods – finite algorithms, round-off errors – Gauss, Crout, Choleski
Iterative methods – infinite processes, truncation errors – Jacobi, Gauss-Seidel.
12/7/2011
3
Titus Beu 2011
Basic idea – equivalent system with upper triangular matrix by elementary row transformations
Forward elimination – transformation of the system
Backward substitution – solving in reversed order the equivalent system
Titus Beu 2011
Example:
3 linear equations with 3 unknowns – A = [aij]33, b = [bi]3 , x = [xi]3
or
11 12 13 1 1
21 22 23 2 2
31 32 33 3 3
a a a x b
a a a x b
a a a x b
=
11 1 12 2 13 3 1
21 1 22 2 23 3 2
31 1 32 2 33 3 3
a x a x a x b
a x a x a x b
a x a x a x b
+ + = + + = + + =
12/7/2011
4
Titus Beu 2011
Forward elimination
1. Eliminate x1 from all equations except the first oneDivide 1st (pivot) equation to pivot element, a11
Subtract 1st equation multiplied by first coefficient from all other equations:
with
(1) (1) (1) (1) (1) (1)1 12 2 13 3 1 12 13 11
(1) (1) (1) (1) (1) (1)22 2 23 3 2 22 23 2 2
(1) (1) (1) (1) (1) (1)332 2 33 3 3 32 33 3
1
or 0
0
x a x a x b a a bx
a x a x b a a x b
xa x a x b a a b
+ + = + = = + =
(1)1 1 11
(1)1 1 11
(1) (1)1 1
(1) (1)1 1
/ , 1,2, 3
/
, 1,2, 3, 2, 3
j j
ij ij i j
i i i
a a a j
b b a
a a a a j i
b b a b
= = = = − = = = −
Titus Beu 2011
Forward elimination
2. Eliminate x2 from the last equationDivide 2nd (pivot) equation to pivot element, a22
(1)
Subtract 2nd equation multiplied by a32(1) from 3rd equation:
with
(1) (1) (1)12 13 11
(2) (2)23 2 2
(2) (2)333 3
1
0 1
0 0
a a bx
a x b
xa b
=
(2) (1) (1)2 2 22
(2) (1) (1)2 2 22
(2) (1) (1) (2)2 2
(2) (1) (1) (2)2 2
/ , 2, 3
/
, 2,3, 3
j j
ij ij i j
i i i
a a a j
b b a
a a a a j i
b b a b
= = = = − = = = −
12/7/2011
5
Titus Beu 2011
Forward elimination
3. Divide 3nd (pivot) equation to pivot element, a33(2) :
with
(1) (1) (1)12 13 11
(2) (2)23 2 2
(3)3 3
1
0 1
0 0 1
a a bx
a x b
x b
=
(3) (2) (2)3 3 33/b b a=
Titus Beu 2011
Backward substitution
Express solution components recurrently in reversed order:
Determinant
Due to successive divisions to the pivot elements:
(3)3 3
(2) (2)2 2 23 3
(1) (1) (1)1 1 12 2 13 3( )
x b
x b a x
x b a x a x
= = − = − +
AA(3)
(1) (2)11 22 33
detdet 1
a a a= =
A(1) (2)
11 22 33det a a a=
12/7/2011
6
Titus Beu 2011
Generalization:
n linear equations with n unknowns – A = [aij]nn, b = [bi]n , x = [xi]n
Forward eliminationBefore step k (k = 1,2,...,n−1):
( 1) ( 1) ( 1)1
( 1
(1) (1) (1) (1)12 1 1 1 1 1
(2) (2) (2
) ( 1) ( 1)1 1 1 1
)2 2 1
( 1) ( 1) ( 1
2
)1
2
1
0 1
0 0
0 0
0 0
k k k
kk kk kn
k k k
k k k k
k k n
k k n
k n
k k knk nk nn
a a a
a a a
a a a a x
a a a x
a a a
− − −
+
− − −
+ + + +
− − −+
+
+
�
�
� � � �
�
� �
� �
� � � � � � �
�
�
� �
�
( 1
(1)1
)
( 1)1
( 1)
(2)2
1
kk
k
kk
nkn
k
b
b
x
x
b
b
bx
+
−
−
+
−
=
�
� �
Titus Beu 2011
Forward eliminationStep k – eliminate xk from all equations below equation k:
New elements of pivot line k:
( ) ( )
(1) (1) (1) (1)12 1 1 1 1 1
(2
1
) (2) (2)2 2
( ) ( )
1 1
( ) (
1
1
1
2 2
1
)
0
0
1
0 1
0 0 1
0 0
0 0
k k
k k k n
k
k
knk
k
kk k
k k n
k k n
nn n
n k
k
a a a a x
a a a
a
x
x
x
xa
a
a
a
a
+
+
+
+
+ +
+
+
� �
� �
� � � � � � �
�
�
�� �
��
�
� � � �
�
(1)1
( )1
(
( )
2
)
( )2
k
k
k
kk
n
b
b
b
b
b
+
=
�
�
( )
( )
(
( 1) ( 1)
( 1) () 1)
1
/ , 1, ,
/
kkk
k
k
k
j
k
k
kj kk
k k
k kkk
a
a
b
a a j k n
b a
− −
− −
= = = + =
…
12/7/2011
7
Titus Beu 2011
Forward eliminationStep k – eliminate xk from all equations below equation k:
New elements of non-pivot lines below pivot line k:
( ) ( )
(1) (1) (1) (1)12 1 1 1 1 1
(2
1
) (2) (2)2 2
( ) ( )
1 1
( ) (
1
1
1
2 2
1
)
0
0
1
0 1
0 0 1
0 0
0 0
k k
k k k n
k
k
knk
k
kk k
k k n
k k n
nn n
n k
k
a a a a x
a a a
a
x
x
x
xa
a
a
a
a
+
+
+
+
+ +
+
+
� �
� �
� � � � � � �
�
�
�� �
��
�
� � � �
�
(1)1
( )1
(
( )
2
)
( )2
k
k
k
kk
n
b
b
b
b
b
+
=
�
�
( 1) ( 1
(
) ( )
( 1)
)
( )
( ) ( 1) ( )
0
, 1, , , 1, ,k k k
ij ik k
kik
j
k k k
i ik k
k
ij
k
i
a a a j k n i k n
b a b
a
a
b
− −
− −
= = − = + = + = −
… …
Titus Beu 2011
Forward eliminationStep n – divide last line to last pivot ann
(n−1):
or
(1)(1) (1) (1) (1)112 1 1 1 1 1(2)(2) (2) (2)
2 22 2 1 2
(( ) ( )1
( 1)11
1
0 1
0 0 1
0 0 0 1
0 0 0 0 1
k k n
k k n
k kk kkk kn
kkk n
n
ba a a a x
x ba a a
x ba a
xa
x
+
+
+
+++
=
� �
� �
� �� � � � � �
� �
� �
�� � � � � �
� �
)
( 1)1
( )
k
k
k
nn
b
b
+
+
�
A x b( ) ( )n n⋅ =
12/7/2011
8
Titus Beu 2011
Backward substitution
Express solution components recurrently in reversed order:
Determinant
Due to successive divisions to the pivot elements:
AA( )
(1) ( 1)11 22
detdet 1n
nnn
a a a −= =
�
A(1) ( 1)
11 22det nnn
a a a −= �
( )
( ) ( )
1
, 1, ,1
nn n
n
k k
k k ki i
i k
x b
x b a x k n
= +
= = − = −
∑ …
Titus Beu 2011
Generalization:
Matrix equation – A = [aij]nn, B = [bij]nm , x = [xij]nm
Forward elimination
Backward substitution
Matrix inversion
( ) ( 1) ( 1)
( ) ( 1) ( 1)
( ) ( 1) ( 1) ( )
( ) ( 1) ( 1) ( )
/ , 1, ,
/ , 1, ,
, 1, , , 1, ,
, 1, ,
k k k
kj kj kk
k k k
kj kj kk
k k k k
ij ij ik kj
k k k kij ij ik kj
a a a j k n
b b a j m
a a a a j k n i k n
b b a b j m
− −
− −
− −
− −
= = + = = = − = + = + = − =
…
…
… …
…
( )
( ) ( )
1
, 1, ,
, 1, , , 1, ,1
n
nj nj
n
k k
kj kj ki ij
i k
x b j m
x b a x j m k n
= +
= = = − = = −
∑
…
… …
B E X A 1[ ]n ij nnδ −= ≡ ⇒ =
12/7/2011
9
Titus Beu 2011
Pivoting
Rearrangement of lines and/or columns to have a maximum pivot element at each elimination step
Merits
Avoids divisions by 0
Minimizes round-off errors
Variants
Partial pivoting (on columns) – at step k one seeks the maximum element alk(k-1)
on column k and lines l ≥ k and interchanges lines k and l.
Total pivoting (method of principal element) – one seeks maximum pivot on all
lines and columns of A(k−1) on which no pivoting was done yet
Titus Beu 2011
//===========================================================================
int Gauss(float **a, float **b, int n, int m, float &det)//---------------------------------------------------------------------------
// Solves matrix equation a x = b by Gaussian elimination, replacing on// output b by x. Uses partial pivoting on columns.
// a - matrix of system (n x n)// b - matrix of r.h.s. terms (n x m); solution x on output
// det – determinant of the system matrix (output)// Error flag: 0 – normal execution
// 1 – singular matrix//---------------------------------------------------------------------------
{#define Swap(a,b) { t = a; a = b; b = t; }
float amax, sum, t;int i, imax, j, k;
det = 1.0;
for (k=1; k<=n; k++) { // ELIMINATIONamax = 0.0; // determines pivot line having
for (i=k; i<=n; i++) // maximum element on column kif (amax < fabs(a[i][k])) {amax = fabs(a[i][k]); imax = i;}
if (amax == 0.0){printf("Gauss: singular matrix !\n"); return 1;}
// interchanges lines imax and kif (imax != k) { // to put pivot on diagonal
det = -det;for (j=k; j<=n; j++) Swap(a[imax][j],a[k][j])
for (j=1; j<=m; j++) Swap(b[imax][j],b[k][j])}
//===========================================================================
int Gauss(float **a, float **b, int n, int m, float &det)//---------------------------------------------------------------------------
// Solves matrix equation a x = b by Gaussian elimination, replacing on// output b by x. Uses partial pivoting on columns.
// a - matrix of system (n x n)// b - matrix of r.h.s. terms (n x m); solution x on output
// det – determinant of the system matrix (output)// Error flag: 0 – normal execution
// 1 – singular matrix//---------------------------------------------------------------------------
{#define Swap(a,b) { t = a; a = b; b = t; }
float amax, sum, t;int i, imax, j, k;
det = 1.0;
for (k=1; k<=n; k++) { // ELIMINATIONamax = 0.0; // determines pivot line having
for (i=k; i<=n; i++) // maximum element on column kif (amax < fabs(a[i][k])) {amax = fabs(a[i][k]); imax = i;}
if (amax == 0.0){printf("Gauss: singular matrix !\n"); return 1;}
// interchanges lines imax and kif (imax != k) { // to put pivot on diagonal
det = -det;for (j=k; j<=n; j++) Swap(a[imax][j],a[k][j])
for (j=1; j<=m; j++) Swap(b[imax][j],b[k][j])}
12/7/2011
10
Titus Beu 2011
det *= a[k][k]; // multiplies determinant with pivot
t = 1.0/a[k][k]; // divides pivot line by pivot
for (j=k+1; j<=n; j++) a[k][j] *= t;for (j= 1; j<=m; j++) b[k][j] *= t;
for (i=k+1; i<=n; i++) { // reduces non-pivot lines
t = a[i][k];for (j=k+1; j<=n; j++) a[i][j] -= a[k][j]*t;
for (j= 1; j<=m; j++) b[i][j] -= b[k][j]*t;}
}
for (k=n-1; k>=1; k--) // BACK SUBSTITUTIONfor (j=1; j<=m; j++) {
sum = b[k][j];for (i=k+1; i<=n; i++) sum -= a[k][i]*b[i][j];
b[k][j] = sum;}
return 0;}
det *= a[k][k]; // multiplies determinant with pivot
t = 1.0/a[k][k]; // divides pivot line by pivot
for (j=k+1; j<=n; j++) a[k][j] *= t;for (j= 1; j<=m; j++) b[k][j] *= t;
for (i=k+1; i<=n; i++) { // reduces non-pivot lines
t = a[i][k];for (j=k+1; j<=n; j++) a[i][j] -= a[k][j]*t;
for (j= 1; j<=m; j++) b[i][j] -= b[k][j]*t;}
}
for (k=n-1; k>=1; k--) // BACK SUBSTITUTIONfor (j=1; j<=m; j++) {
sum = b[k][j];for (i=k+1; i<=n; i++) sum -= a[k][i]*b[i][j];
b[k][j] = sum;}
return 0;}
Titus Beu 2011
Basic idea – equivalent system with diagonal matrix by elementary row transformations
Forward elimination – transformation of the system – more elaborate
Backward substitution – inexistent
Variant – system matrix is transformed into the unit matrix
Enables simultaneous calculation and storage of inverse
12/7/2011
11
Titus Beu 2011
Forward eliminationBefore step k (k = 1,2,...,n−1):
Relevant information of matrix A(k) – exclusively in columns j = k+1,...,n
Columns j = 1,...,k contain trivial data – can be used to produce the inverse A−1
( 1) ( 1) ( 1)1 1 1 1
( 1) ( 1) ( 1)2 2 1 2
( 1) ( 1) ( 1)1
( 1) ( 1) ( 1)1 1 1 1
( 1) ( 1) ( 1)1
1 0
0 1
0 0
0 0
0 0
k k k
k k n
k k k
k k n
k k k
kk kk kn
k k k
k k k k k n
k k knk nk nn
a a a
a a a
a a a
a a a
a a a
− − −
+
− − −
+
− − −
+
− − −
+ + + +
− − −+
�
�
� � �
�
�
�
�
� � �
�
�
� � � �� �
� �
( 1)1
( 1)2
( 1)
(
1
2
1)
(
1 1
1)
k
k
k
k
k
k k
n
k
n
k
b
b
b
b
x
x
b
x
x
x
−
−
−
−
+
−
+
=
�
�
�
�
Titus Beu 2011
Forward eliminationStep k – eliminate xk from all equations:
New elements of pivot line k:
( ) ( ) ( )1 1 1 1( ) ( ) (2 1 2 2
( ) ( )1 1 1
( ) ( )1
( ) ( )1
1
2
1
1 0
0 1
0
0 0
0 0
0 0
0
0
0
1
k k kk n
k k
k n
k k
k k k n
k k
kk
k knk
kn
nn
k
k
n
a a b
a a b
a a
a
x
x
xa a
xa
x
+
+
+ + +
+
+
+
=
�
�
� � � �
�
�
��
�
�
� � �
�
�
�
� ��
�
�
�
)
( )1
(
(
)
)
k
k
kk
k
kn
b
b
b
+
�
�
( )
( )
(
( 1) ( 1)
( 1) () 1)
1
/ , 1, ,
/
kkk
k
k
k
j
k
k
kj kk
k k
k kkk
a
a
b
a a j k n
b a
− −
− −
= = = + =
…
12/7/2011
12
Titus Beu 2011
Forward eliminationStep k – eliminate xk from all equations:
New elements non-pivot lines:
( ) ( ) ( )1 1 1 1( ) ( ) (2 1 2 2
( ) ( )1 1 1
( ) ( )1
( ) ( )1
1
2
1
1 0
0 1
0
0 0
0 0
0 0
0
0
0
1
k k kk n
k k
k n
k k
k k k n
k k
kk
k knk
kn
nn
k
k
n
a a b
a a b
a a
a
x
x
xa a
xa
x
+
+
+ + +
+
+
+
=
�
�
� � � �
�
�
��
�
�
� � �
�
�
�
� ��
�
�
�
)
( )1
(
(
)
)
k
k
kk
k
kn
b
b
b
+
�
�
( 1) ( 1) ( )
( 1) ( 1) ( )
( )
( )
( )
0
, 1, , , 1, , ,k k k
ij ik kj
k k k
i ik
kik
k
ij
i k
k
a
a
b
a a a j k n i n i k
b a b
− −
− −
= = − = + = ≠ = −
… …
Titus Beu 2011
Forward eliminationStep n – divide last line to last pivot ann
(n−1):
Solution
Determinant
( )11( )
2 2
( )
( )1 1
( )
1 0 0 0 0
0 1 0 0 0
0 0 1 0 0
0 0 0 1 0
0 0 0 0 1
n
n
nk k
nk k
nn n
bx
x b
x b
x b
x b
+ +
=
� �
� �
� �� � � � � �
� �
� �
� � � � � � � �
� �
x b( )n=
A(1) ( 1)
11 22det nnn
a a a −= �
12/7/2011
13
Titus Beu 2011
//===========================================================================
int GaussJordan0(float **a, float **b, int n, int m, float &det)//---------------------------------------------------------------------------
// Solves matrix equation a x = b by Gauss-Jordan elimination, replacing on// output b by x. Uses partial pivoting on columns.
// a - matrix of system (n x n)// b - matrix of r.h.s. terms (n x m); solution x on output
// det – determinant of the system matrix (output)// Error flag: 0 – normal execution
// 1 – singular matrix//---------------------------------------------------------------------------
{#define Swap(a,b) { t = a; a = b; b = t; }
float amax, t;int i, imax, j, k;
det = 1.0;
for (k=1; k<=n; k++) { // ELIMINATIONamax = 0.0; // determines pivot line having
for (i=k; i<=n; i++) // maximum element on column kif (amax < fabs(a[i][k])) {amax = fabs(a[i][k]); imax = i;}
if (amax == 0.0){printf("GaussJordan0: singular matrix !\n"); return 1;}
// interchanges lines imax and kif (imax != k) { // to put pivot on diagonal
det = -det;for (j=k; j<=n; j++) Swap(a[imax][j],a[k][j])
for (j=1; j<=m; j++) Swap(b[imax][j],b[k][j])}
//===========================================================================
int GaussJordan0(float **a, float **b, int n, int m, float &det)//---------------------------------------------------------------------------
// Solves matrix equation a x = b by Gauss-Jordan elimination, replacing on// output b by x. Uses partial pivoting on columns.
// a - matrix of system (n x n)// b - matrix of r.h.s. terms (n x m); solution x on output
// det – determinant of the system matrix (output)// Error flag: 0 – normal execution
// 1 – singular matrix//---------------------------------------------------------------------------
{#define Swap(a,b) { t = a; a = b; b = t; }
float amax, t;int i, imax, j, k;
det = 1.0;
for (k=1; k<=n; k++) { // ELIMINATIONamax = 0.0; // determines pivot line having
for (i=k; i<=n; i++) // maximum element on column kif (amax < fabs(a[i][k])) {amax = fabs(a[i][k]); imax = i;}
if (amax == 0.0){printf("GaussJordan0: singular matrix !\n"); return 1;}
// interchanges lines imax and kif (imax != k) { // to put pivot on diagonal
det = -det;for (j=k; j<=n; j++) Swap(a[imax][j],a[k][j])
for (j=1; j<=m; j++) Swap(b[imax][j],b[k][j])}
Titus Beu 2011
det *= a[k][k]; // multiplies determinant with pivot
t = 1.0/a[k][k]; // divides pivot line by pivot
for (j=k+1; j<=n; j++) a[k][j] *= t;for (j= 1; j<=m; j++) b[k][j] *= t;
for (i=1; i<=n; i++) // reduces non-pivot lines
if (i != k) {t = a[i][k];
for (j=1; j<=n; j++) a[i][j] -= a[k][j]*t;for (j=1; j<=m; j++) b[i][j] -= b[k][j]*t;
}}
return 0;}
det *= a[k][k]; // multiplies determinant with pivot
t = 1.0/a[k][k]; // divides pivot line by pivot
for (j=k+1; j<=n; j++) a[k][j] *= t;for (j= 1; j<=m; j++) b[k][j] *= t;
for (i=1; i<=n; i++) // reduces non-pivot lines
if (i != k) {t = a[i][k];
for (j=1; j<=n; j++) a[i][j] -= a[k][j]*t;for (j=1; j<=m; j++) b[i][j] -= b[k][j]*t;
}}
return 0;}
12/7/2011
14
Titus Beu 2011
//===========================================================================
int GaussJordan1(float **a, float **b, int n, int m, float &det)//---------------------------------------------------------------------------
// Solves matrix equation a x = b by Gauss-Jordan elimination, replacing on// output a by a^(-1) and b by x. Uses partial pivoting on columns.
// a - matrix of system (n x n)// b - matrix of r.h.s. terms (n x m); solution x on output
// det – determinant of the system matrix (output)// Error flag: 0 – normal execution
// 1 – singular matrix//---------------------------------------------------------------------------
{#define Swap(a,b) { t = a; a = b; b = t; }
float amax, t;int i, imax, j, k;
int *ipivot;
ipivot = IVector(1,n); // stores pivot line
det = 1.0;for (k=1; k<=n; k++) { // ELIMINATION
amax = 0.0; // determines pivot line havingfor (i=k; i<=n; i++) // maximum element on column k
if (amax < fabs(a[i][k])) {amax = fabs(a[i][k]); imax = i;}if (amax == 0.0)
{printf("GaussJordan1: singular matrix !\n"); return 1;}ipivot[k] = imax; // stores pivot line
//===========================================================================
int GaussJordan1(float **a, float **b, int n, int m, float &det)//---------------------------------------------------------------------------
// Solves matrix equation a x = b by Gauss-Jordan elimination, replacing on// output a by a^(-1) and b by x. Uses partial pivoting on columns.
// a - matrix of system (n x n)// b - matrix of r.h.s. terms (n x m); solution x on output
// det – determinant of the system matrix (output)// Error flag: 0 – normal execution
// 1 – singular matrix//---------------------------------------------------------------------------
{#define Swap(a,b) { t = a; a = b; b = t; }
float amax, t;int i, imax, j, k;
int *ipivot;
ipivot = IVector(1,n); // stores pivot line
det = 1.0;for (k=1; k<=n; k++) { // ELIMINATION
amax = 0.0; // determines pivot line havingfor (i=k; i<=n; i++) // maximum element on column k
if (amax < fabs(a[i][k])) {amax = fabs(a[i][k]); imax = i;}if (amax == 0.0)
{printf("GaussJordan1: singular matrix !\n"); return 1;}ipivot[k] = imax; // stores pivot line
Titus Beu 2011
// interchanges lines imax and k
if (imax != k) { // to put pivot on diagonaldet = -det;
for (j=1; j<=n; j++) Swap(a[imax][j],a[k][j])for (j=1; j<=m; j++) Swap(b[imax][j],b[k][j])
}
det *= a[k][k]; // multiplies determinant with pivot
t = 1.0/a[k][k]; // divides pivot line by pivota[k][k] = 1.0; // diagonal element of unit matrix
for (j=1; j<=n; j++) a[k][j] *= t;for (j=1; j<=m; j++) b[k][j] *= t;
for (i=1; i<=n; i++) // reduces non-pivot lines
if (i != k) {t = a[i][k];
a[i][k] = 0.0; // non-diagonal element of unit matrixfor (j=1; j<=n; j++) a[i][j] -= a[k][j]*t;
for (j=1; j<=m; j++) b[i][j] -= b[k][j]*t;}
}
for (k=n; k>=1; k--) // rearrange columns of inverseif (ipivot[k] != k)
for (i=1; i<=n; i++) Swap(a[i][ipivot[k]],a[i][k])
FreeIVector(ipivot,1);return 0;
}
// interchanges lines imax and k
if (imax != k) { // to put pivot on diagonaldet = -det;
for (j=1; j<=n; j++) Swap(a[imax][j],a[k][j])for (j=1; j<=m; j++) Swap(b[imax][j],b[k][j])
}
det *= a[k][k]; // multiplies determinant with pivot
t = 1.0/a[k][k]; // divides pivot line by pivota[k][k] = 1.0; // diagonal element of unit matrix
for (j=1; j<=n; j++) a[k][j] *= t;for (j=1; j<=m; j++) b[k][j] *= t;
for (i=1; i<=n; i++) // reduces non-pivot lines
if (i != k) {t = a[i][k];
a[i][k] = 0.0; // non-diagonal element of unit matrixfor (j=1; j<=n; j++) a[i][j] -= a[k][j]*t;
for (j=1; j<=m; j++) b[i][j] -= b[k][j]*t;}
}
for (k=n; k>=1; k--) // rearrange columns of inverseif (ipivot[k] != k)
for (i=1; i<=n; i++) Swap(a[i][ipivot[k]],a[i][k])
FreeIVector(ipivot,1);return 0;
}
12/7/2011
15
Titus Beu 2011
//===========================================================================
int GaussJordan(float **a, float **b, int n, int m, float &det)//---------------------------------------------------------------------------
// Solves matrix equation a x = b by Gauss-Jordan elimination, replacing on// output a by a^(-1) and b by x. Uses total pivoting.
// a - matrix of system (n x n)// b - matrix of r.h.s. terms (n x m); solution x on output
// det – determinant of the system matrix (output)// Error flag: 0 – normal execution
// 1,2 – singular matrix//---------------------------------------------------------------------------
{#define Swap(a,b) { t = a; a = b; b = t; }
float amax, t;int i, imax, j, jmax, k;
int *ipivot, *jpivot, *npivot;
ipivot = IVector(1,n); // stores pivot linejpivot = IVector(1,n); // stores pivot column
npivot = IVector(1,n); // marks used pivot column
det = 1.0;for (i=1; i<=n; i++) npivot[i] = 0;
//===========================================================================
int GaussJordan(float **a, float **b, int n, int m, float &det)//---------------------------------------------------------------------------
// Solves matrix equation a x = b by Gauss-Jordan elimination, replacing on// output a by a^(-1) and b by x. Uses total pivoting.
// a - matrix of system (n x n)// b - matrix of r.h.s. terms (n x m); solution x on output
// det – determinant of the system matrix (output)// Error flag: 0 – normal execution
// 1,2 – singular matrix//---------------------------------------------------------------------------
{#define Swap(a,b) { t = a; a = b; b = t; }
float amax, t;int i, imax, j, jmax, k;
int *ipivot, *jpivot, *npivot;
ipivot = IVector(1,n); // stores pivot linejpivot = IVector(1,n); // stores pivot column
npivot = IVector(1,n); // marks used pivot column
det = 1.0;for (i=1; i<=n; i++) npivot[i] = 0;
Titus Beu 2011
for (k=1; k<=n; k++) { // ELIMINATION
amax = 0.0; // determines pivotfor (i=1; i<=n; i++) { // loop over lines
if (npivot[i] != 1)for (j=1; j<=n; j++) { // loop over columns
if (npivot[j] == 0) // pivoting not yet done?if (amax < fabs(a[i][j]))
{amax = fabs(a[i][j]); imax = i; jmax = j;}else if (npivot[j] > 1)
{printf("GaussJordan: singular matrix 1 !\n"); return 1;}}
}ipivot[k] = imax; // stores pivot line
jpivot[k] = jmax; // stores pivot column++(npivot[jmax]); // mark used pivot column
if (amax == 0.0){printf("GaussJordan: singular matrix 2 !\n"); return 2;}
// interchanges lines imax and jmaxif (imax != jmax) { // to put pivot on diagonal
det = -det;for (j=1; j<=n; j++) Swap(a[imax][j],a[jmax][j])
for (j=1; j<=m; j++) Swap(b[imax][j],b[jmax][j])}
det *= a[jmax][jmax]; // multiplies determinant with pivot
for (k=1; k<=n; k++) { // ELIMINATION
amax = 0.0; // determines pivotfor (i=1; i<=n; i++) { // loop over lines
if (npivot[i] != 1)for (j=1; j<=n; j++) { // loop over columns
if (npivot[j] == 0) // pivoting not yet done?if (amax < fabs(a[i][j]))
{amax = fabs(a[i][j]); imax = i; jmax = j;}else if (npivot[j] > 1)
{printf("GaussJordan: singular matrix 1 !\n"); return 1;}}
}ipivot[k] = imax; // stores pivot line
jpivot[k] = jmax; // stores pivot column++(npivot[jmax]); // mark used pivot column
if (amax == 0.0){printf("GaussJordan: singular matrix 2 !\n"); return 2;}
// interchanges lines imax and jmaxif (imax != jmax) { // to put pivot on diagonal
det = -det;for (j=1; j<=n; j++) Swap(a[imax][j],a[jmax][j])
for (j=1; j<=m; j++) Swap(b[imax][j],b[jmax][j])}
det *= a[jmax][jmax]; // multiplies determinant with pivot
12/7/2011
16
Titus Beu 2011
t = 1.0/a[jmax][jmax]; // divides pivot line by pivot
a[jmax][jmax] = 1.0; // diagonal element of unit matrixfor (j=1; j<=n; j++) a[jmax][j] *= t;
for (j=1; j<=m; j++) b[jmax][j] *= t;
for (i=1; i<=n; i++) // reduces non-pivot linesif (i != jmax) {
t = a[i][jmax];a[i][jmax] = 0.0; // non-diagonal element of unit matrix
for (j=1; j<=n; j++) a[i][j] -= a[jmax][j]*t;for (j=1; j<=m; j++) b[i][j] -= b[jmax][j]*t;
}}
for (k=n; k>=1; k--) // rearrange columns of inverse
if (ipivot[k] != jpivot[k])for (i=1; i<=n; i++) Swap(a[i][ipivot[k]],a[i][jpivot[k]])
FreeIVector(ipivot,1);
FreeIVector(jpivot,1);FreeIVector(npivot,1);
return 0;}
t = 1.0/a[jmax][jmax]; // divides pivot line by pivot
a[jmax][jmax] = 1.0; // diagonal element of unit matrixfor (j=1; j<=n; j++) a[jmax][j] *= t;
for (j=1; j<=m; j++) b[jmax][j] *= t;
for (i=1; i<=n; i++) // reduces non-pivot linesif (i != jmax) {
t = a[i][jmax];a[i][jmax] = 0.0; // non-diagonal element of unit matrix
for (j=1; j<=n; j++) a[i][j] -= a[jmax][j]*t;for (j=1; j<=m; j++) b[i][j] -= b[jmax][j]*t;
}}
for (k=n; k>=1; k--) // rearrange columns of inverse
if (ipivot[k] != jpivot[k])for (i=1; i<=n; i++) Swap(a[i][ipivot[k]],a[i][jpivot[k]])
FreeIVector(ipivot,1);
FreeIVector(jpivot,1);FreeIVector(npivot,1);
return 0;}
Titus Beu 2011
Tridiagonal matrices – sparse matrices – most of the non-diagonal elements 0.
General methods are not efficient.
Typically – discretization of differential equations by finite difference schemes.
General form:
A x d⋅ =
1 1 1 1
2 2 2 2 2
1 1 1 1 1
1 1 1 1 1
0
0
i i i i i
i i i i i
n n n n n
n n n n
b c x d
a b c x d
a b c x d
a b c x d
a b c x d
a b x d
− − − − −
− − − − −
=
� � � � �
� � � � �
12/7/2011
17
Titus Beu 2011
Factorization of matrix A:
Elements of matrices L and U – by identification (βi ≠ 0):
A L U= ⋅⋅⋅⋅
1 1
2 2 2
1 1 1
1 1 1
1
0 1 0
1
1
0 0 1
1
i i i
i i i
n n n
n n
β γ
α β γ
α β γ
α β γ
α β γ
α β
− − −
− − −
= ⋅
A
� � � �
� � � �
1 1 1 1 1
1
1
,
, , , 2,3, , 1
,i i i i i i i i i
n n n n n n
b c
a b c i n
a b
β γ β
α β α γ γ β
α β α γ
−
−
= = = = − = = − = = −
…
Titus Beu 2011
Initial system becomes:
First system:
Intermediate solution y – by identification (βi ≠ 0):
L y dL U x d
U x y( )
== ⇔ =
⋅⋅⋅⋅⋅ ⋅⋅ ⋅⋅ ⋅⋅ ⋅
⋅⋅⋅⋅
1 1 1
2 2 2 2
1 1 1 1
1 1 1 1
0
0
i i i i
i i i i
n n n n
n n n n
y d
a y d
a y d
a y d
a y d
a y d
β
β
β
β
β
β
− − − −
− − − −
=
� � � �
� � � �
1 1 1
1
/
( ) / , 2, 3, ,i i i i i
y d
y d a y i n
β
β−
= = − =
…
12/7/2011
18
Titus Beu 2011
Initial system becomes:
Second system:
Final solution x – by identification:
L y dL U x d
U x y( )
== ⇔ =
⋅⋅⋅⋅⋅ ⋅⋅ ⋅⋅ ⋅⋅ ⋅
⋅⋅⋅⋅
1 1 1
2 2 2
1 1 1
1 1 1
1
1 0
1
1
0 1
1
i i i
i i i
n n n
n n
x y
x y
x y
x y
x y
x y
γ
γ
γ
γ
γ
− − −
− − −
=
� � � �
� � � �
1, 1, ,1n n
i i i i
x y
x y x i nγ+
= = − = −
…
Titus Beu 2011
Algorithm
Factorization + solution of system L ⋅ y = d:
Back substitution – solution of system U ⋅ x = y:
Four arrays (a, b, c and d) are sufficient in the implementation
Elements γi can be stored in array c, overwriting elements ci
For di, yi and xi the same array d may be used, which finally returns the solution
No need for pivoting – diagonal dominance.
1 1 1 1 1 1 1 1
1 1
, / , /
, / , ( ) /
2,3, , 1i i i i i i i i i i i i
b c y d
b a c y d a y
i n
β γ β β
β γ γ β β− −
= = = = − = = − = −
…
1 1
1
( ) / ( )
, 1, ,1n n n n n n n
i i i i
x d a y b a
x y x i n
γ
γ
− −
+
= − − = − = −
…
12/7/2011
19
Titus Beu 2011
//===========================================================================
void TriDiag(float a[], float b[], float c[], float d[], int n)//---------------------------------------------------------------------------
// Solves by LU factorization a linear system with tridiagonal matrix// a - lower co-diagonal (i=2..n)
// b - main diagonal// c - upper co-diagonal (i=1..n-1)
// d - r.h.s. terms; solution on output// n - order of system
//---------------------------------------------------------------------------{
float beta;int i; // factorization
c[1] /= b[1]; d[1] /= b[1];
for (i=2; i<=(n-1); i++) {beta = b[i] - a[i]*c[i-1];
c[i] /= beta;d[i] = (d[i] - a[i]*d[i-1])/beta;
}d[n] = (d[n] - a[n]*d[n-1])/(b[n] - a[n]*c[n-1]);
// back substitutionfor (i=(n-1); i>=1; i--) d[i] -= c[i]*d[i+1];
}
//===========================================================================
void TriDiag(float a[], float b[], float c[], float d[], int n)//---------------------------------------------------------------------------
// Solves by LU factorization a linear system with tridiagonal matrix// a - lower co-diagonal (i=2..n)
// b - main diagonal// c - upper co-diagonal (i=1..n-1)
// d - r.h.s. terms; solution on output// n - order of system
//---------------------------------------------------------------------------{
float beta;int i; // factorization
c[1] /= b[1]; d[1] /= b[1];
for (i=2; i<=(n-1); i++) {beta = b[i] - a[i]*c[i-1];
c[i] /= beta;d[i] = (d[i] - a[i]*d[i-1])/beta;
}d[n] = (d[n] - a[n]*d[n-1])/(b[n] - a[n]*c[n-1]);
// back substitutionfor (i=(n-1); i>=1; i--) d[i] -= c[i]*d[i+1];
}
