03. Matrices and Matrix Algebra - Yonsei University · 2014. 12. 29. · Solving Linear Systems by Matrix Inversion (a) To prove the invertibility of A , it suffices to show that

E-mail: [email protected]://web.yonsei.ac.kr/hgjung

Matrices and Matrix AlgebraMatrices and Matrix Algebra


3.1. Operations on Matrices3.1. Operations on Matrices

Matrix: a rectangular array of numbers, called entries.

A matrix with m rows and n columns m×n

Matrix Notation and Terminology

A n×n matrix : a square matrix of order n



Matrix Notation and Terminology

(A)ij: the entry in row i and column j of a matrix A.

(A)12=-3



Operations on Matrices

Example 1Example 1



Operations on Matrices



Row and Column Vectors



Row and Column Vectors



The Product Ax



The Product Ax



The Product Ax



The Product AB



The Product AB

Example 5Example 5



Finding Specific Entries in A Matrix Product



Finding Specific Rows and Columns of A Matrix Product

the column rule for matrix multiplication

the row rule for matrix multiplication



Matrix Products as Linear Combinations



Matrix Products as Linear Combinations

Example 9Example 9



Transpose of a Matrix

Example 10Example 10



Trace



Inner and Outer Matrix Products








Keep in mind, however, that these formulas apply only when u and v are expressed as column vectors.


3.2. Inverses; Algebraic Properties of Matrices3.2. Inverses; Algebraic Properties of Matrices

Properties of Matrix Addition and Scalar Multiplication



Properties of Matrix Multiplication



Properties of Matrix Multiplication

The commutative law does not hold for matrix multiplication; that is AB and BA need not be equal matrices.

Example 1Example 1

≠



Zero Matrices

A matrix whose entries are all zero is called a zero matrix.



Zero Matrices

Example 2Example 2

The cancellation law for real numbers: If ab=ac, and if a≠0, then b=c.

The cancellation law does not hold, in general, for matrix multiplication.



Zero Matrices

Example 3Example 3

If c and a are real numbers such that ca=0, then c=0 or a=0.

Nonzero matrices can have a zero product.

Hear CA=0, but C≠0 and A≠0.



Identity Matrices

A square matrix with 1’s on the main diagonal and zeros elsewhere is called an identity matrix.



Identity Matrices



Inverse of A Matrix

REMARK

Observe that the condition that AB=BA=I is not altered by interchanging A and B. Thus, if A is invertible and B is an inverse of A, then it is also true that B is invertible and A is an inverse of B. Accordingly, when the condition AB=BA=I holds, it is correct to say that A and B are inverse of one another.



Inverse of A Matrix

Example 4Example 4



Inverse of A Matrix

Example 5Example 5

In general, a square matrix with a row or column of zeros is singular.



Properties of Inverses




Example 6Example 6




Example 7Example 7

Because the coefficients of the unknowns are literal rather than numerical, Gauss-Jordan elimination is a little clumsy.




Example 8Example 8

What should be the lengths of the arms be in order to position the tip of the working arm at the point (x, y) shown in the figure?






Powers of Matrix

If A is a square matrix, then we define the nonnegative integer powers of A to be

and if A is invertible, then we define the negative integer powers of A to be



Powers of Matrix



Matrix Polynomials

If A is a square matrix, say n×n, and if

is any polynomial, then we define the n×n matrix p(A) to be

It is called a matrix polynomial in A.

If p(x)=p1(x)p2(x), then



Properties of the Transpose



Properties of the Transpose



Properties of the Trace



Properties of the Trace




Transpose and Dot Product

In expressions of the form Au∙v or u·Av, the matrix A can be moved across the dot product sign by transposing A.

If u and v are column vectors, then their dot product can be expressed as the matrix product u·v=vTu.


3.3. Elementary Matrices; A Method for Finding A3.3. Elementary Matrices; A Method for Finding A--11

Elementary Matrices

Elementary row operations

1. Multiply an equation through by a nonzero constant.

2. Interchange two equations.

3. Add a multiple of one equation to another.

Elementary matrix: a matrix that results from applying a single elementary row operation to an identity matrix.



Elementary Matrices

In short, this theorem states that an elementary row operation can be performed on a matrix Ausing a left multiplication by an appropriate elementary matrix.



Elementary Matrices

Example 1Example 1



Elementary Matrices

Example 2Example 2



Characterization of Invertibility



Row Equivalence

In general, two matrices that can be obtained from one another by finite sequences of elementary row operations are said to be row equivalent.



An Algorithm for Inverting Matrices




Example 3Example 3




Example 4Example 4



Solving Linear Systems by Matrix Inversion




Example 5Example 5







(a) To prove the invertibility of A, it suffices to show that the homogeneous system Ax=0 has only the trivial solution. If x is any solution of this system,

Thus, the system Ax=0 has only the trivial solution, which establishes that A is invertible.

(b)



A Unifying Theorem



Consistency of Linear Systems

Example 8Example 8


3.4. Subspaces and Linear Independence3.4. Subspaces and Linear Independence

Subspaces of Rn

In general, if W is a nonempty set of vectors in Rn,

then we say that W is closed under scalar multiplication if any scalar multiple of a vector in W is also in W,

And we say that W is closed under addition if the sum of any two vectors in W is also in W.

Let W be the plane through the origin of Rn whose equation is



Subspaces of Rn

The zero subspace and Rn are called the trivial subspaces of Rn.

Example 1Example 1



Subspaces of Rn

The subspace W of Rn whose vectors satisfy (3) is called the span of v1, v2, …, vs and is denoted by

We also say that the vector v1, v2, …, vs span W.

The scalars in (3) are called parameters, and we can think of span{v1, v2, …, vs} as the geometric object in Rn that results when the parameters in (3) are allowed to vary independently from -∞ to ∞.



Subspaces of Rn

Example 2Example 2

Thus, span{e1, e2, …, en}=Rn; that is, Rn is spanned by the standard unit vectors.



Subspaces of Rn

Example 4Example 4

LOOKING AHEAD

We will eventually show that every subspace of Rn is the span of some finite set of vectors, and, in fact, is the span of at most n vectors.

All subspaces of R2 fall into one of three categories:1. The zero subspace2. Lines through the origin3. All of R2

All subspaces of R3 fall into one of four categories:1. The zero subspace2. Lines through the origin3. Planes through the origin4. All of R3



Solution Space of A Linear System

Since x=0 is a solution of the system, we are assured that the solution set is nonempty.

If x0 is any solution of the system, A(kx0)=k(Ax0)=k0=0

If x1 and x2 are solutions of the system, A(x1+x2)=Ax1+Ax2=0+0=0

The solution set of a homogeneous linear system is a subspace, we will refer to it as the solution space of the system.

The solution space, being a subspace of Rn, must be expressible in the form

which we call a general solution of the system.




Example 5Example 5

The solution space can also be denoted by span{v1, v2, v3}, where

v1=(-3,1,0,0,0,0), v2=(-4,0,-2,1,0,0), v3=(-2,0,0,0,1,0)




Example 7Example 7

The solution space of a homogeneous linear system in three unknowns is a subspace of R3.






Linear Independence



Linear Independence


Two vectors v1 and v2 in Rn are linearly dependent if and only if there are scalars c1 and c2, not both zero, such that

c1≠0 and c2≠0

Two vectors in Rn are linearly dependent if they are collinear and linearly independent if they are not.



Linear Independence


Three vectors in Rn are linearly dependent if they lie in a plane through the origin and are linearly independent if they do not.



Linear Independence and Homogeneous Linear Systems

We can rewrite Ax=0 as

Consider the n×s matrix





In Example 6 of Section 3.3, where we showed that it has only the trivial solution. Thus, the vectors are linearly independent.

In Example 4 of Section 3.3, where we showed that the coefficient matrix for this system is not invertible. This implies that the system has nontrivial solutions (Theorem 3.3.7), and hence that the vectors are linearly dependent.





This system has more unknowns than equations, so it must have nontrivial solutions by Theorem 2.2.3. This implies that the vectors are linearly dependent.



Translated Subspaces

If x0, v1, v2, …, vs are vectors in Rn, then the set of vectors of the form

can be viewed as a translation by x0 of the subspace

We call it the translation of W by x0 and denote it by

Translations of subspaces have various names in the literature, the most common being linear manifolds, flats, and affine spaces.

We will call them linear manifolds.



A Unifying Theorem


3.5. The Geometry of Linear Systems3.5. The Geometry of Linear Systems

The Relationship Between Ax=b and Ax=0

If nonhomogeneous system Ax=b is consistent,

we will call Ax=0 the homogeneous system associated with Ax=b.specific solution







The solution of a consistent nonhomogeneous linear system is expressible in the form

where

particular solutiongeneral solution




Example 1Example 1

Since the solution set of a consistent nonhomogeneous linear system is the translation of the solution space of the associated homogeneous system, the solution set of a consistent nonhomogeneous linear system in two or three unknowns must be one of the following:






Consistency of a Linear System from the Vector Point of View

If the successive column vectors of A are a1, a2, …, an, Ax=b is rewritten as

The linear system is consistent if and only if b can be expressed as a linear combination of the column vectors of A.

If A is an m×n matrix, then to say that b is a linear combination of the column vectors of A is the same as saying that b is in the subspace of Rm spanned by the column vectors of A. This subspace is called the column space of A and is denoted by col(A).



Consistency of a Linear System from the Vector Point of View

Example 2Example 2

Determine whether the vector w=(9,1,0) can be expressed as a linear combination of the vectors

v1=(1,2,3), v2=(1,4,6), v3=(2,-3,-5)

and, if so, find such a linear combination.



Hyperplanes

The set of points (x1, x2, …, xn) in Rn that satisfy a linear equation of the form

is called a hyperplane in Rn.

The hyperplane passes through the origin.



Hyperplanes

The hyperplane consists of all vectors x in Rn

that are orthogonal to the vector a.

Hyperplane through the origin with normal a

Orthogonal complement of a

α (read, “a perp”)



Geometric Interpretations of Solution Spaces


3.6. Matrices with Special Forms3.6. Matrices with Special Forms

Diagonal Matrices

A square matrix in which all entries off the main diagonal are zero is called a diagonal matrix.

A diagonal matrix is invertible if and only if all of its diagonal entries are nonzero.



Diagonal Matrices

If k is a positive integer, then the kth power of the diagonal matrix is



Triangular Matrices

A square matrix in which all entries above the main diagonal are zero is called lower triangular and a square matrix in which all the entries below the main diagonal are zero is called upper triangular. A matrix that is either upper triangular or lower triangular is called triangular.

Example 2Example 2

, 0 if ij ijA a a i j , 0 if ij ijA a a i j



Properties of Triangular Matrices

Part (b)If i<j,



Properties of Triangular Matrices

Example 4Example 4

invertible noninvertible



Symmetric and Skew-symmetric Matrices

A square matrix A is called symmetric if AT=A and skew-symmetric if AT= -A.







Let A and B be symmetric matrices with the same size.

If the product of A and B is symmetric, (AB)T=AB

The product AB is symmetric if and only if AB=BA.

Example 5Example 5



Invertibility of Symmetric Matrices

If A is symmetric and invertible,



Matrices of the Form AAT and ATA

The products AAT and ATA are always symmetric, since

If A is invertible, AT is invertible, so the products AAT and ATA are invertible.

(A-1)T=(AT)-1



Fixed Points of a Matrix

If A is a square matrix and Ax=x,

the solutions of this equation, if any, are called the fixed points of A.

Example 6Example 6



A Technique for Inverting I-A When A Is Nilpotent

If Ak=0,

A square matrix A with the property that Ak=0 for some positive integer k is said to be nilpotent, and the smallest positive power for which Ak=0 is called the index of nilpotency.



A Technique for Inverting I-A When A Is Nilpotent

Example 7Example 7



Inverting I-A by Power Series

If 0<x<1 and k∞,



Inverting I-A by Power Series

It is called a power series representation of (I-A)-1.


3.7. Matrix Factorizations; LU3.7. Matrix Factorizations; LU--DecompositionDecomposition

Solving Linear Systems by Factorization

If a square matrix A is in the form A=LU (1)

where L is lower triangular and U is upper triangular,

Step 1. Rewrite the system Ax=b as

LUx=b (2)

Step 2. Define a new unknown y by letting

Ux=y (3)

and rewrite (2) as Ly=b.

Step 3. Solve the system Ly=b for the unknown y.

Step 4. Substitute the now-known vector y into (3) and solve for x.

This procedure is called the method of LU-decomposition.

Notice that Ly=b and Ux=y are easy to solve because their coefficient matrices are triangular.




Example 1Example 1Assume that

Solve the following linear system




In general, not every square matrix A has an LU-decomposition, nor is an LU-decomposition unique if it exists.




Suppose that A is an n×n matrix that has been reduced by elementary row operations without row interchanges to the row echelon form U.

There is a sequence of elementary matrices E1, E2, …, Ek such that

Since elementary matrices are invertible, we can solve (8) for A as

(8)

where (10)




U is upper triangular because it is a row echelon form of the square matrix A.

Notice that no row interchanges are used to obtain U from A and that in Gaussian elimination zeros are introduced by adding multiples of rows to lower rows.

Each elementary matrix in (8) arises either by multiplying a row of the n×n identity matrix by a scalar or by adding a multiple of a row to a lower row. In either case the resulting elementary matrix is lower triangular.

Each of the matrices on the right side of (10) is lower triangular, so their product L is also lower triangular.




A Procedure for finding an LU-decomposition of the matrix A:

1. Reduce A to a row echelon form U without using any row interchanges.

2. Keep track of the sequence of row operations performed, and let E1, E2, …, Ek be the sequence of elementary matrices that corresponds to those operations.

3. Let

4. A=LU is an LU-decomposition of A.

Try to figure what entry to put into L, so that L will be reduced to I.




Four step for finding an LU-decomposition of the matrix A:

1. Reduce A to row echelon form U without using row interchanges, keeping track of the multipliers used to introduce the leading 1’s and the multipliers used to introduce zeros below the leading 1’s.

2. In each position along the main diagonal of L, place the reciprocal of the multiplier that introduced the leading 1 in that position in U.

3. In each position below the main diagonal of L, place the negative of the multiplier used to introduce the zero in that position in U.

4. From the decomposition A=LU.




Example 2Example 2

Find an LU-decomposition of



The Relationship between Gaussian Elimination and LU-Decomposition

Example 3Example 3

Find an LU-decomposition of



Matrix Inversion by LU-Decomposition

Many of the best algorithms for inverting matrices use LU-decomposition.

Then, AA-1=I can be expressed as

Let

This can be done by finding an LU-decomposition of A.



LDU-Decomposition

U has 1’s on the main diagonal as it is a row echelon form of A, but L need not.

We can “shift” the diagonal entries of L to a diagonal matrix D and write L as

L=L’D

where L’ is a lower triangular matrix with 1’s on the main diagonal.



LDU-Decomposition

If A is a square matrix that can be reduced to row echelon form without row interchanges, then A can be factored uniquely as

A=LDU

where L is a lower triangular matrix with 1’s on the main diagonal, D is a diagonal matrix, and U is an upper triangular matrix with 1’s on the main diagonal.

This is called the LDU-decomposition (or LDU-factorization) of A.



Using Permutation Matrices to Deal With Row Interchanges

A matrix P is formed by multiplying in sequence those elementary matrices that corresponds to the row interchanges, and then execute all of these row interchanges on A by forming the product PA. Since all of the row interchanges are out of the way, the matrix PA can be reduced to row echelon form without row interchanges and hence an LU-decomposition

PA=LU

Since the matrix P is invertible (being a product of elementary matrices), the system Ax=b and PAx=Pb have the same solutions, and the latter system can be solved by LU-decomposition.

P is called permutation matrix.

This is called a PLU-decomposition of A.



Cost Estimates for Solving Large Linear Systems

Neither method has a cost advantage over the other. However, LU-decomposition has other advantages that make it the method of choice: …


3.8. Partitioned Matrices and Parallel Processing3.8. Partitioned Matrices and Parallel Processing

General Partitioning

A matrix can be partitioned (subdivided) into submatrices (also called blocks) in various ways by inserting lines between selected rows and columns.

where




If the sizes of the blocks conform for the required operations, then the block version of the row-column rule of Theorem 3.1.7 yields

This procedure is called block multiplication.




Example 1Example 1







It is sometimes called outer product rule.




Example 2Example 2

Here is the proof of Therem 3.2.12(e). tr(AB)=tr(BA)



Block Diagonal Matrices

A partitioned matrix A is said to be block diagonal if the matrices on the main diagonal are square and all matrices off the main diagonal are zero.

where the matrices D1, D2, …, Dk are square.

It can be shown that the matrix A is invertible if and only if each matrix on the diagonal is invertible, in which case



Block Diagonal Matrices

Example 3Example 3

Consider the block diagonal matrix



Block Upper Triangular Matrices

A partitioned matrix A is said to be block upper triangular if the matrices on the main diagonal are square and all matrices below the main diagonal are zero; that is, the matrix is partitioned as

where the matrices A11, A22, …, Akk are square.

The definition of a block lower triangular matrix is similar.




This formula allows the work of inverting A to be accomplished by parallel processing, that is by using two individual processors working simultaneously to compute the inverse of the smaller matrices, A11 and A22.




Example 4Example 4




Example 4Example 4

Documents

03. Matrices and Matrix Algebra - Yonsei University · 2014. 12. 29. · Solving Linear Systems by Matrix Inversion (a) To prove the invertibility of A , it suffices to show that