the concept of vector spaces vectors and matrices linear ...vps/ME505/IEM/03 01.pdfcontinuously differentiable on the interval (a,b). The notion of vector spaces helps to obtain and

Chapter III Linear Algebra August 15, 2020

161

CCHHAAPPTTEERR IIIIII LLIINNEEAARR AALLGGEEBBRRAA Objectives: After the completion of this section the student should recall

- the concept of vector spaces

- the operations with vectors and matrices

- the methods of solution of linear systems of algebraic equations

- the methods of solution of the eigenvalue problem

Contents: 1. Vector Spaces

2. Linear Combination, Linear Independence, Basis

3. Vectors

4. Matrices

5. Linear Transformations with the Help of Matrices

6. Determinants

7. Matrix Inverse

8. Row Reduction

9. Linear Systems

10. Eigenvalue Problem

11. Linear Algebra with Maple

12. Review questions and exercises

Chapter III Linear Algebra August 15, 2020 162

1. VECTOR SPACES A vector space { }, , ,V = u v w is a set of objects called vectors, for which two operations are defined: the operation of addition of vectors +u v and the operation of scalar multiplication ku by the scalars from the field of real numbers k ∈ , under the condition that they satisfy the special axioms. In the printed text, vectors are denoted by bold letters , ,...u v and in handwriting they are denoted by arrows above letters u,v ,... . Scalars are denoted by lower case letters a,b,... .

Examples of vector spaces include the Euclidian space n∈x (column vectors of dimension n ), the space of all m n× matrices mnM , the space of all continuous functions ( ) [ ]f x C a,b∈ defined on the interval [ ]a,b , the space of

all functions ( ) ( )1f x C a,b∈ continuously differentiable on the interval ( )a,b . The notion of vector spaces helps to obtain and to study the general properties of solutions of algebraic and differential equations and systems of equations.

The formal definitions of a vector space and a vector subspace are the following:

Definition 1 A linear vector space V over the field of real numbers is a set of elements

(called vectors) with two operations:

i) addition of vectors:

if , V∈u v , then V+ ∈u v closure for sum

ii) multiplication by a scalar:

if V∈u and k ∈ , then k V∈u closure for scaling

which satisfy the following axioms:

1) if , V∈u v , then + = +u v v u commutative law

2) if , , V∈u v w , then ( ) ( )+ + = + +u v w u v w associative law

3) there exists a vector V∈0 , such that existence of + = + =u 0 0 u u for any V∈u zero vector

4) for any V∈u there exists a vector V− ∈u existence of

such that ( )+ − =u u 0 negative vector

5) if V∈u and a,b ∈ , then ( ) ( )a b ab=u u associative law

6) if , V∈u v and k ∈ , then ( )k k k+ = +u v u v distributive law

7) if V∈u and a,b ∈ , then ( )a b a b+ = +u u u distributive law 8) if V∈u , then 1 =u u

Definition 2 Let V be a vector space, then any subset U V⊂ which is a vector space itself

with the operations induced by V is called to be a subspace of V


163

Comments: 1) Conditions i) and ii) are the closure axioms. They mean that the sum of any two vectors from the space is a vector which also belongs to this space, and any vector multiplied by any scalar is a vector which also belongs to the space. Therefore, the defined operations can always be performed without going outside of the vector space, because vector space V contains all required vectors.

2) Axioms 1-8) are the vector axioms; the set of objects which satisfies these axioms is called the vector set, and its elements are called vectors.

3) We defined a vector space over the field of real numbers , therefore it can be referred to as a real vector space. Similarly, a complex vector space over the field of complex numbers also can be defined.

4) The zero vector V∈0 is unique. It means that if there exists a vector V∈θ with the same property + = + =u θ θ u u then =θ 0

5) To show that some set V is a vector space, we need to demonstrate that all ten axioms i-ii) and 1-8) are satisfied.

6) To show that some subset U V⊆ of the vector space V is a subspace, we need to check only the closure axioms i) and ii), because elements of the subset of the vector space are already the vectors.

To prove the opposite, that U V⊆ is not a subspace, very often it is sufficient to show that the zero vector V∈0 does not belong to the subset U∉0 , and, therefore, it cannot be a subspace.

7) The negative element V− ∈u is unique. It means that if there exists a vector V− ∈w with the same property ( )+ − =u w 0 then − = −u w

8) 0 =u 0 for any V∈u

9) ( )1− = −u u for any V∈u

10) a =0 0 for any a ∈

11) a =u 0 if and only if a 0= or =u 0

12) Vector space can be defined over an arbitrary field (scalars) for which operation of multiplication of a vector by a scalar is defined:

vector ku is defined for all V∈u and k ∈

If is , then V is referred to as a real vector space. If is , then V is referred to as a complex vector space.

space V subspace U

0


EXAMPLES OF VECTOR SPACES Space of real numbers 1. The field of all real numbers with usual algebraic operations of addition

of real numbers and multiplication by a real number is a vector space. The field axioms (see Chapter I) and algebraic rules provide satisfaction of the vector space axioms. The vector space is a particular case of a more general n-dimensional Euclidian vector space n .

Euclidean space n 2. The elements of the Euclidian vector space n are n-tuples ( )1 2 nx ,x ,...,x=x ix ∈ which can be treated as the coordinates of the points in n-dimensional

geometrical space. The operations are defined in the following way: ( ) ( ) ( )1 2 n 1 2 n 1 1 2 2 n nx ,x ,...,x y , y ,..., y x y ,x y ,...,x y+ = + = + + +x y

( )1 2 nk kx ,kx ,...,kx=x The zero vector is ( )0,0,...,0=0 and the negative vector is ( ) ( )1 2 n1 x , x ,..., x− = − = − − −x x

It is easy to verify the satisfaction of the vector space axioms for n . Euclidean space 3 is a particular case of n : ( )1 2 3x ,x ,x=x ix ∈

It is a physical 3-dimentional Euclidian space which is used in mathematical modeling physical and engineering problems.


165

Column vectors The vector space n is the set of all column vectors

1

2

n

xx

x

=

x

ix ∈

for which operations are defined in the similar way:

1 1 1 1

2 2 2 2

n n n n

x y x yx y x y

x y x y

+ + + = + =

+

x y

,

1

2

n

kxkx

k

kx

=

x

The column vectors are just a different form of writing the real n-tuples. In their turn, the column vectors are a particular case of the m n× real matrices.

Space of m n× matrices mn 3. The set of all real matrices with m rows and n columns

( )11 12 1n

21 22 2ni 1,..,mmn ijj 1,...,n

m1 m2 mn

a a aa a a

a

a a a

==

= =

A

ija ∈

with operations

11 11 12 12 1n 1n

21 21 22 22 2n 2n

m1 m1 m2 m2 mn 2n

a b a b a ba b a b a b

a b a b a b

+ + + + + + + =

+ + +

A B

11 12 1n

21 22 2n

m1 m2 mn

ka ka kaka ka ka

k

ka ka ka

=

A

is a vector space with zero vector

m,n

0 0 00 0 0

0 0 0

=

0


Space of continuous functions [ ]C a,b 4. The set of all real-valued continuous functions

[ ] ( ) [ ] [ ]{ }C a,b f x : a,b f is continuous on a,b= →

The operations addition of vectors and multiplication by a scalar are defined as the point-wise operations with functions

( ) ( )f g f x g x+ ≡ + [ ]x a,b∈ for [ ]f ,g C a,b∈

( )kf kf x≡ [ ]x a,b∈ for [ ]f C a,b∈ , k ∈

Obviously, a sum of two continuous functions on [ ]a,b is a

continuous function on [ ]a,b , and a continuous function multiplied by a scalar is still continuous. Therefore, the closure axioms i-ii) of a vector space are satisfied. The zero vector [ ]0 C a,b∈ is a function identically equal to

zero on the interval [ ]a,b . Other vector spaces of continuous functions are:

( ) ( )C a,b , C ,−∞ ∞

Space of differentiable functions ( )1C a,b 5. The set of all real valued functions continuously

differentiable on the open interval ( )a,b

( ) ( ) ( ) ( ){ }1C a,b f x : a,b f is continuous on a,b′= →

with operations identical to operations in the space [ ]C a,b . Space of polymonials ( )nP x 6. The set of all polynomials with degree not greater than n:

( ) { }n n 1n n n 1 1 0 iP x a x a x a x a a−

−= + + + + ∈

is a vector space with operations identical to operations with functions.

Zero subspace 7. The simplest but important example of a vector subspace of

any space V over the field of real numbers is its subspace consisting of one element { } V⊂0 , where vector V∈0 is a zero vector of the vector space V :

{ }V0 = 0

It also is called the null-subspace of V.

V

{ }0


167

Solution space 8. Consider the 1st order linear ordinary differential equation

y y 0′ − =

The set of all solutions of this equation (provided that they exist) is a subset of the vector space ( )1C (The set of all solutions is not empty – because it obviously possess the trivial solution ( )y x 0= ):

( ) ( ){ }1V y x C y y 0 ′= ∈ − = ( )1C⊂

We can verify that the set V is a vector space. For this, according to comment (6), we have to verify only the conditions i-ii) of the Definition 1:

i) Assume that 1 2y , y V∈ are solutions, then show that the sum of them 1 2y y+ is also a solution. Indeed,

( ) ( )1 2 1 2 1 1 2 2

0 0

y y y y y y y y 0′ ′ ′+ − + = − + − =

ii) Assume that 1y V∈ is a solution, then show that the scalar multiple of it 1ky , k ∈ is also a solution. Indeed,

( ) ( ) ( )1 1 1 1 1 1ky ky ky ky k y y k 0 0′ ′ ′− = − = − = ⋅ =

Therefore the set of all solutions ( )1V C⊂ is a vector space.

We investigated the structure and the properties of the set of all solutions of the differential equation and showed that they constitute the vector space even without solving the equation. In fact, the general solution of this equation is given by

( ) xy x ce= , c ∈

They are infinitely many times differentiable functions, and therefore, they belong to the space ( ) ( ) ( )1y x C C∞∈ ⊂ .

Intersection of vector spaces Let iU V⊂ be the subspaces of the vector space V , then

{ }n

i ii 1

U = V U , i=1,...,n=

∈ ∈u u

is a subspace of V (Exercise 2, p.201).

V

3U

2U

1U


2. LINEAR COMBINATION, LINEAR INDEPENDENCE, BASIS

We need a convenient way of constructing and describing vector spaces. In the plane any point can be uniquely defined by a pair of its coordinates ( )x, y . In vector spaces the role of coordinates is played by the coefficients in the linear combination representation of the vectors with the help of the basis vectors. For example, the set of monoms { }21,x,x is a basis of the vector space of all

quadratic polynomials ( )2P x where each polynomial is represented uniquely as

a linear combination of basis vectors 20 1 2a 1 a x a x⋅ + ⋅ + ⋅ with coefficients

0 1 2a ,a ,a ∈ . Therefore, the polynomials are defined just by these coefficients. Let V be a vector space. Recall some definitions and facts:

Linear Combination Let 1 2 n, ,..., V∈u u u . Then a vector 1 1 2 2 n nc c ... c+ + +u u u is called a linear

combination of vectors 1 2 n, ,...,u u u with coefficients 1 2 nc ,c ,...,c ∈ . It is obvious that any linear combination of vectors from V also is a vector of V

1 1 2 2 n nc c ... c V+ + + ∈u u u Span The set of all linear combinations of vectors 1 2 n, ,..., V∈u u u is called a span of

vectors 1 2 n, ,...,u u u

{ } { }1 2 n 1 1 2 2 n n ispan , ,..., c c ... c c≡ + + + ∈u u u u u u

In this definition coefficients ic attain all real values generating infinitely many

vectors. For example, in the Euclidian space 3 the span

1 1

span 1 c 1 c 0 0

= ∈

generates a line in the xy-plane. And the span

1 2 1 2

1 1 1 1span 0 , 1 c 0 c 1 c ,c

0 0 0 0

= + ∈

generates the xy-plane.

The ( ) ( ){ }span sin x ,cos xλ λ of two vectors ( ) ( ) ( )sin x ,cos x Cλ λ ∞∈

generates the general solution of the linear ODE 2y y 0λ′′ + = , λ ∈ .

The { }mx mxspan e ,e− − of two vectors ( )mx mxe ,e C− − ∞∈ generates the general

solution of the linear ODE 2y m y 0′′ − = , m ∈ . The same general solution

also can be generated by the ( ) ( ){ }span sinh mx ,cosh mx .

Because each of these linear combinations is a vector of space V , the span is a subset of vector space V

{ }1 2 nspan , ,..., V⊂u u u

We can verify that the span is a vector space itself, therefore { }1 2 nspan , ,...,u u u is a subspace of V .


169

The question arises, if it is possible to specify some set of vectors 1 2 n, ,..., V∈u u u such that their span generates the whole vector space V ? And

if yes, then what is the minimal set of vectors with this property? Then such a set can be used as a basis for vector space V .

Linear Independence The set of vectors 1 2 n, ,..., V∈u u u is called linearly independent if their linear

combination is equal to the zero vector with only all zero coefficients:

1 1 2 2 n nc c ... c + + + =u u u 0 ⇒ iall c 0=

If a set of vectors 1 2 n, ,..., V∈u u u is not linearly independent, then it is called linearly dependent. Formally, it can be stated as follows:

The set of vectors 1 2 n, ,..., V∈u u u is called linearly dependent if there exist coefficients 1 2 cc ,c ,...,c ∈ not all equal to zero such that the linear combination with them is equal to the zero vector:

1 1 2 2 n nc c ... c + + + =u u u 0

Consider the following useful facts about linear independence of vectors:

1. If in the set of vectors 1 2 n, ,..., V∈u u u one of vectors k ≠u 0 can be represented as a linear combination of the other vectors

k 1 1 k 1 k 1 k 1 k 1 n n c ... c c ... c− − + += + + + + +u u u u u then the set 1 2 n, ,..., V∈u u u is linearly dependent. Indeed, rewrite the equation as ( )1 1 k 1 k 1 k k 1 k 1 n nc ... c 1 + c ... c− − + ++ + + − + + =u u u u u 0 where at least one coefficient is non-zero. 2. If { }1 2 n, ,...,∈0 u u u then the set { }1 2 n, ,...,u u u is linearly dependent.

Linear Independence in m Let

11

211

m1

aa

a

=

u

,

12

222

m2

aa

a

=

u

,…,

1n

2nn

mn

aa

a

=

u

m∈

Then the set 1 2 n, ,...,u u u is linearly dependent if there exist coefficients

1 2 nc ,c ,...,c ∈ not all equal to zero such that

1 1 2 2 n nc c ... c + + + =u u u 0 In the component form this vector equation can be written as

11 12 1n

21 22 2n1 2 n

m1 m2 mn

a a a 0a a a 0

c c ... c

a a a 0

+ + + =

It means that the homogeneous system of algebraic equations for 1 2 nc ,c ,...,c

11 12 1n 1

21 22 2n 2

m1 m2 mn n

a a a c 0a a a c 0

a a a c 0

=

or =Ac 0

has non-trivial solutions.

If the number of vectors n m> then the set 1 2 n, ,...,u u u is linearly dependent. If n m= then the set 1 2 n, ,...,u u u is linearly independent only if det 0=A .


Linear independence of functions See the Section 5.3 Theory of Linear ODE in Chapter 5. Basis of vector space V Let V be a vector space. Then the set of vectors

1 2 n, ,..., V∈u u u is called a basis for the vector space V if i) the set 1 2 n, ,...,u u u is linearly independent ii) { }1 2 nspan , ,..., V=u u u Any vector V∈v is uniquely represented as a linear

combination of the basis vectors:

1 1 2 2 n n c c ... c= + + +v u u u

That means that the span of 1 2 n, ,...,u u u generates the space V. Theorem 1 Every vector space possesses a basis. A basis for the vector space is not unique, in general there can

be many different bases, but the number of vectors in the different bases for the vector space is the same:

if the set 1 2 n, ,...,u u u is a basis for the vector space V , then any other basis for V also consists of n vectors.

The number of vectors in the basis is called the dimension of

the vector space V . Not all vector spaces have a finite dimension. For example,

the space of continuous functions [ ]C a,b has an infinite dimension.

Examples: 1. The set of column vectors

1

10

0

=

e

, 2

01

0

=

e

,…, n

00

1

=

e

is a standard basis for n . In particular, the standard basis for 3 is denoted by

100

=

i , 010

=

j ,001

=

k

2. The set of monoms { }2 n1,x,x ,...,x is a standard basis for

the vector space of polynomials of order n ( )nP x : ( ) { }2 n

n 0 1 2 n iP x a a x a x ... a x , a= + + + + ∈

100

=

i

010

=

j

001

=

k

x

y

z

3

standardbasis in


171

Functions defined on vector spaces Let U and V be two vector spaces. Then the function f :U V→ defined between these vector spaces is called a

linear map or a homomorphism if for all , U∈u w and c ∈ it satisfies the following conditions:

i) ( ) ( ) ( )f f f+ = +u w u w

ii) ( ) ( )f c cf=u u

For example, a linear map can be defined by a matrix as a linear transformation of Euclidian vector space: Let mnA∈ be a m n× real matrix, then the transformation defined by

A=v u for all m∈u is a linear map from m to the vector space n . This linear map includes particular transformations such as dilation and rotation of the vector space. A linear map in the vector spaces of functions can be defined by the familiar operations such as definite and indefinite integration and differentiation. Let f :U V→ be a linear map (homomorphism). Then

Image (Range) ( ) ( ){ }im f f U= ∈u u is called the image (or range) of f .

It can be shown that ( )im f V⊂ is a subspace of V . Rank ( ) ( )( )rank f dim im f= is called the rank of f . Rank is a dimension of the image of f. Kernel ( ) ( ){ }ker f U f= ∈ =u u 0 is called the kernel of f .

Kernel is a subset of all vectors in space U which are mapped into the zero vector of the space V. It can be shown that

( )ker f U⊂ is a subspace of U .

Theorem 2 ( )rank f dimU dim ker( f )= − The linear maps help to obtain and to study the properties of solutions of algebraic and differential equations.

n

1

n

u

u

=

u

=Au v

1

m

v

v

=

v

m

U

u

( )f=v u

v

V


3. VECTORS Recall the definition of the column-vectors in n

1

2

n

xx

x

=

x

ix ∈

for which operations of vector addition and multiplication by a scalar are defined by the operations with their components:

1 1 1 1

2 2 2 2

n n n n

x y x yx y x y

x y x y

+ + + = + =

+

x y

,

1

2

n

kxkx

k

kx

=

x

The set of all column-vectors n is a vector space of dimension n with the standard basis

1

10

0

=

e

, 2

01

0

=

e

,…, n

00

1

=

e

4. MATRICES Recall the vector space of all real matrices with m rows and n columns mn

( )11 12 1m 1n

21 22 2m 2ni 1,..,mm,n ijj 1,...,n

m1 m2 mm nn

a a a aa a a a

a

a a a a

==

= =

A

ija ∈

The set of elements with equal indices { }kka is called the main diagonal. If the number of rows is equal to the number of columns, n m= , then the matrix is called a square matrix. A matrix can be interpreted as a combination of its column-vectors:

( )mn 1 2 n=A a a a with column-vectotrs

1 j

2 ji

mj

aa

a

=

a

j 1,2,...,n=

or their row vectors:

1

2mn

m

AA

A

=

A

with row-vectors ( )i i1 i2 inA a a a= i 1,2,...,m=

Column-vectors n∈x can be treated as the n 1× matrices.

Equality =A B Equality of matrices: two matrices of the same size mn, ∈A B are equal if

and only if all the corresponding matrix elements are equal: =A B if ( ) ( )i 1,..,m i 1,..,mij ij

j 1,...,n j 1,...,na b= =

= =

=

main diagonal


173

Multiplication by a scalar kA Let mn∈A , k ∈ , then

11 12 1n

21 22 2n

m1 m2 mn

ka ka kaka ka ka

k

ka ka ka

=

A

Addition +A B Addition of two matrices of the same size mn mn, ∈ → + ∈A B A B

11 11 12 12 1n 1n

21 21 22 22 2n 2n

m1 m1 m2 m2 mn 2n

a b a b a ba b a b a b

a b a b a b

+ + + + + + + =

+ + +

A B

Transpose TA Transpose of a matrix T

mn nm∈ → ∈A A

( ) ( )

T11 12 1n 11 21 m1

T 21 22 2n 12 22 m2Ti 1,..,m i 1,..,mij jij 1,...,n j 1,...,n

m1 m2 mn 1n 2n mn

a a a a a aa a a a a a

a a

a a a a a a

= == =

= = ≡ =

A

This operation interchanges the components ij jia a↔ .

Properties of transpose: ( )TT =A A

( )T T T+ = +A B A B

( )T Tk k=A A

( ) ( )T 11 T −− =A A for invertible matrices

Symmetric Matrix The square matrix A is called symmetric if T =A A . The square matrix A is called skew-symmetric if T = −A A . Every square matrix can be separated into a sum of a symmetric matrix and a skew-symmetric matrix:

( ) ( )T Ts a

1 12 2

= + = + + −A A A A A A A

Matrix multiplication AB Matrix Multiplication: the product of two matrices AB is defined if the number of the columns of the matrix A is equal to the number of rows of the matrix B : for mn np, ∈ ∈ →A B mp∈AB

Let ( )11 12 1n

21 22 2ni 1,..,mmn ijj 1,...,n

m1 m2 mn

a a aa a a

a

a a a

==

= = =

A A

( )1 2 n= a a a

( )11 12 1 p

21 22 2 pi 1,..,nnp ijj 1,...,p

n1 n2 np

b b bb b b

b

b b b

==

= = =

B B

( )1 2 p= b b b


The product AB yields an m p× matrix which can be defined in one of the following ways

AB

n n n

1k k1 1k k 2 1k kpk 1 k 1 k 1

n n n

2k k1 2k k 2 2k kpk 1 k 1 k 1

n n n

mk k1 mk k 2 mk kpk 1 k 1 k 1

a b a b a b

a b a b a b

a b a b a b

= = =

= = =

= = =

=

∑ ∑ ∑

∑ ∑ ∑

∑ ∑ ∑

AB ( )1 2 3= Ab Ab Ab ( )ijAB i1 1 j i2 2 j in nj a b a b ... a b= + + +

The element in the intersection of the thi row and the thj column ( )ijAB of the

matrix AB is obtained by a scalar product of the thi row of the matrix A with the thj column of the matrix B

Properties of matrix multiplication: Let mn np 1 np 2 np pr, , , ∈ ∈ ∈ ∈ ∈A B B B C , m,n, p,r ∈

1. ( ) ( )c c c= =AB A B AB

2. ( ) ( )=AB C A BC associative law

3. ( )1 2 1 2+ = +A B B AB AB distributive law

4. ( )1 2 1 2+ = +B B C B C B C distributive law

5. =AB 0 ⇒ =A 0 or =B 0

6. 1 2=AB AB ⇒ 1 2=B B

7. ( )T T T=AB B A transpose of a product

8. =A0 0 zero matrix

9. =AI A unit matrix

10. ≠AB BA do not commute

i

j

i

j ABBA

( )ijAB

=


175

Example:

( )3 2 2 4 3 4× × ×=A B AB

1 23 45 6

1 2 3 45 6 7 8

11 14 17 2023 30 37 4435 46 57 68

=

Particular case: product of a matrix and a vector m n n 1 m 1× × ×=A x b

System of algebraic equations: System of m algebraic equations for n unknowns 1 2 nx ,x ,...,x :

matrix form

m n n 1 m 1× × ×=A x b

11 12 1n 1 1

21 22 2n 2 2

m1 m2 mn n m

a a a x ba a a x b

a a a x b

=

11 1 12 2 1n n 1

21 1 22 2 2n n 2

m1 1 m2 2 mn n m

a x a x ... a x ba x a x ... a x b

a x a x ... a x b

+ + + =+ + + =

+ + + =

Let n, ∈x y be vectors. Two types of vector product can be defined

Inner product of column-vectors (dot product) is defined as

Inner product (dot product) [ ]1

2T1 2 n 1 1 2 2 n n

n

yy

x x x x y x y ... x y

y

⋅ ≡ = = + + +

x y x y

The results is a number, ⋅ ∈x y .

Outer-product (dyadic product) of column-vectors is defined as

Outer-product [ ]1 1 1 1 2 1 n

2 2 1 2 2 2 nT1 2 n

n n 1 n 2 n n

x x y x y x yx x y x y x y

y y y

x x y x y x y

= =

xy

The result is a matrix, T

nn ∈xy .


5. LINEAR TRANSFORMATIONS WITH THE HELP OF MATRICES

Let ( )mn 1 2 n mn= ∈A a a a where

1i

2i mi

mi

aa

a

= ∈

a

i 1,2,...,n=

Matrix multiplication of vectors generates a linear transformation n m→ :

→x Ax A vector n∈x is transformed to a vector m∈Ax The resulting vector can be written as

m n n 1 m 1

= × × ×

A x b m∈

( )

1

2 m1 2 n 1 1 2 2 n n

n

xx

x x +...+x

x

= = + ∈

Ax a a a a a a

ix ∈

Therefore, the image of a linear map is a set of all linear combinations of the columns of the matrix A which by the definition is a span. Therefore, the image is a subspace of the vector space m ; let us call it the column space of the matrix A and denote it in the following way:

column space { }1 2 ncol span , ,...,=A a a a , mcol ⊂A

rank rank dimcol=A A

Recall the definition of the kernel of a linear map denoted by

{ }nker = ∈ =A x Ax 0 nker ⊂A

Therefore, the kernel of a linear transformation defined with the help of the matrix A is a set of all solutions of a homogeneous system of linear algebraic equations =Ax 0 . The column space of the matrix A is a set of all vectors m∈b which are images of some vectors n∈x , i.e. for which there exists a solution of the system of non-homogeneous system of linear algebraic equations =Ax b .

n

1

n

x

x

1

m

b

b

linear transformation=Ax b m


177

m0

b

x c

Theorem 3 ( )dim ker rank n+ =A A ( )rank n dim ker m= − <A A

Consider transformation n m→ given by

m n n 1

× ×

A x m∈ .

This important theorem of linear algebra allows one to make some conclusions about the solutions of the system of linear algebraic equations: If col∈b A then there exists n∈x such that =Ax b . If col∉c A then there is no n∈x such that =Ax c . If the homogeneous system =Ax 0 has only the trivial solution =x 0 , then the

{ }ker =A 0 and ( )dim ker 0=A and therefore rank n=A .

Then for any col∈b A there exists the unique n∈x such that =Ax b . If A is a square matrix (m=n), then it means that mcol =A is the whole space

m , and therefore, for any m∈b there exists n∈x such that =Ax b . If columns of A are linearly independent then the dimension of the vector space spanned by n linearly independent vectors is n, rank n=A and ( )dim ker 0=A . Therefore, there is only the trivial solution for =Ax 0 .

nvector space

ker A

=Ax 0

=Ax b

col A

n0

mvector space

( )dim ker A rank A

mn

Axtransformation


6. DETERMINANTS Determinants are defined only for square matrices nn∈A ( n n× matrices) . The determinant of a matrix nn∈A is a number calculated according to special rules. The determinant is denoted in one of the following ways:

det A

11 12 1n 11 12 1n

21 22 2n 21 22 2n

m1 m2 mn m1 m2 mn

a a a a a aa a a a a a

det det

a a a a a a

= =

A

Definition 1 One possible definition of the det A is based only on its properties (it does not show actually how to calculate the determinant):

Determinant of the square matrix nn∈A is defined as a function nndet : →A which satisfies the following conditions:

1. the determinant is linear in every row

i i i idet A B det A det B + = +

for every i 1,2,...,n=

i idet cA c det A =

where the dots indicate unchanged rows.

2. If matrix A contains two equal rows i jA A= then det 0=A .

3. Determinant of the unit matrix

1 0 00 1 0

det 1

0 0 1

= =I

Definition 2 The other definition demonstrates the structure of the determinant and provides the method of its calculation:

Determinant of the square matrix nn∈A is a number equal to the sum of all possible products of n matrix entries such that every term contains exactly one entry of each row and each column of the matrix weighted by 1 or 1− in the following way:

22∈A a b

ad bcc d

= −

33∈A 11 12 13

21 22 23

31 32 33

a a aa a aa a a

11 22 33 11 32 23 12 31 23

12 21 33 13 21 32 13 31 22

a a a a a a a a a

a a a a a a a a a= − +

− + −

( ) ( )11 22 33 32 23 12 21 33 31 23a a a a a a a a a a= − − −

( )13 21 32 31 22a a a a a+ −

22 23 21 23 21 2211 12 13

32 33 31 33 31 32

a a a a a aa a a

a a a a a a= − +

11 11 12 12 13 13a A a A a A= − +

11 11 12 12 13 13a C a C a C= + +


179

Where the minor ijA is defined as a matrix obtained from the matrix A by

deleting the thi row and the thj column

the cofactor ijC ( )i jij1 A+= − ( )i j

ij1 det A+= − Then the definition of the determinant of the matrix nn∈A can be formalized in the following expansion over the 1st row form:

( )n n1 j

1 j 1 j 1 j 1 jj 1 j 1

det 1 a A a C+

= =

= − =∑ ∑A

Theorem 4 (the cofactor expansion theorem)

The determinant can be calculated by the cofactor expansion

over any row:

( )n ni k

ik ik ik ikk 1 k 1

det 1 a A a C+

= =

= − =∑ ∑A i 1,2,...,n=

or any column:

( )n nk j

kj kj kj kjk 1 j 1

det 1 a A a C+

= =

= − =∑ ∑A j 1,2,...,n=

In practice, for calculation of the determinants the cofactor expansion method is used only for small matrices ( 2 2× or 3 3× ) or matrices of special form (diagonal, triangle, sparse, etc). Theorem 5 (the determinant of the triangle matrices)

The determinant of the upper-triangle, the low-triangle and the diagonal matrices are equal to the product of the elements on the main diagonal:

11 11 11

22 22 2211 22 nn

nn nn nn

d d 0 0 d 0 00 d d 0 0 d 0

d d d

0 0 d d 0 0 d

∗ ∗∗ ∗

= = =

∗ ∗

Theorem 6 (properties of the determinant)

Let nn, ∈A B then

1. ( ) ndet c c det=A A

2. ( )det det det=AB A B

3. ( )Tdet det=A A

4. det 0=A ⇒ rank n<A

5. ( )1 1det det

− =AA

for invertible A

6. If two rows of A are the same, then det 0=A

7. If A has a row(or a column) all of zeroes, then det 0=A

j

i

ijA


7. MATRIX INVERSE Definition Let matrix nn∈A be a square matrix. If there exists a matrix nn∈C such that

=CA I left inverse

=AC I right inverse

then the matrix A is said to be invertible, and the matrix C is called an inverse of A . It is denoted by 1−A :

inverse matrix 1−A 1 1− −= =A A AA I

Theorem 7 If the inverse of nn∈A exists then it is unique Theorem 8 Let nn, ∈A B be invertible matrices, then

a) ( ) 11 −− =A A ( 1−A is also invertible)

b) ( ) 1 1 1− − −=AB B A

c) ( ) ( )1 TT 1− −=A A Theorem 9 If nn∈A is invertible then A is row equivalent to I Theorem 10 If nn∈A is invertible then det 0≠A Theorem 11 If nn∈A is not invertible then det 0=A .

singular matrix Matrix with zero determinant det 0=A is called singular. Theorem 12 If nn∈A is invertible then for any n∈b the linear system

=Ax b has a unique solution 1−=x A b

Proof: a) 1−=x A b is a solution: b) Let n∈u be another solution

( )1− =A A b b =Au b

( )1− =AA b b ( )1 1− −=A Au A b

=Ib b ( )1− =A A u x =b b =u x ■

Inversion by row reduction: The algorithm for finding A (for row reduction procedure see below):

1) start with augmented matrix [ ]A I

2) by Gaussian elimination reduce to 1− I A

Inverse of 2 2× matrix: Let a bc d

=

A , then 1 d b d b1 1

c a c adet ad bc− − −

= = − −− A

A

Cofactor formula for inverse: Define the adjoint of matrix nn∈A as the matrix which entries are cofactors:

11 21 n1

12 22 n2

1n 2n nn

C C CC C C

adj

C C C

=

A

, ijC ( )i jij1 det A+= −

then the inverse of matrix A is defined by

1 adjdet

− =AAA


181

Example: Let 2 15 3

=

A

1) Row reduction (inversion of a unit matrix):

2 1 1 0

5 3 0 1

=

A I

1 11 0

2 25 3 0 1

1R2

1 11 02 2 1 50 12 2

−

2 1 2R 5R R− →

1 0 3 1

1 50 12 2

− −

1 2 1R R R− →

1 0 3 1

0 1 5 2

− −

1 2 1R R R− →

1 3 15 2

− − = −

A

2) 2 1

det 2 3 1 5 15 3

= = ⋅ − ⋅ =

A

1 d b 3 1 3 11 1

c a 5 2 5 2det 1− − − −

= = = − − − A

A

3) Cofactor formula for inverse:

[ ] ( )1 111 11 11A 3 detA 3 C 1 3 3+= = ⇒ = − =

[ ] ( )1 212 12 12A 5 detA 5 C 1 5 5+= = ⇒ = − = −

[ ] ( )2 121 12 21A 1 detA 1 C 1 1 1+= = ⇒ = − = −

[ ] ( )2 222 22 22A 2 detA 2 C 1 2 2+= = ⇒ = − =

1 adjdet

− =AAA

3 13 15 25 2det

− −− = = − A


8. ROW REDUCTION: The following elementary operations with matrix rows, their symbolic designation and affiliated with them elementary matrices are introduced as:

Interchange i jR R↔ interchange rows i and j 1

i j1

0 1 iE

1 0 j1

=

1det E 1= −

Scaling ikR multiply row i by k ∈ 2

11

E k i

1

=

2det E k=

Replacement i j iR kR R+ → replace row i by the sum of row i

and row j multiplied by k

With the help of the elementary matrices, the elementary row operations can be performed by matrix multiplication (with the corresponding acting on the determinant for square matrices):

operation interchange 1E A ( )1det E det= −A A

operation scaling 2E A ( )2det E k det=A A

operation replacement 3E A ( )3det E det=A A

Two matrices B and A are called row equivalent if one of them is obtained from the other by a sequence of elementary operations

s 2 1k k kE ...E E=B A

Row equivalent matrices have the same kernel. Row Reduced Echelon Form: Row Reduced Echelon Form (RREF) is a matrix which entries satisfy the

following conditions:

1. All non-zero rows are above any row with all zeroes.

2. Each leading entry (first non-zero element) of a row is to the right of the leading entry of the row above.

3. All entries in the column below the leading entry are zero.

4. All leading entries are 1 .

5. Leading entry (pivot) is the only non-zero entry in the column.

Theorem 13 Every matrix A is row equivalent to the unique RREF U

pivot

1 0 0 pivot rows 0 0 1 0 0 0 0 1

0 0 0 0 0

pivot columns

↓

∗ ∗ ←

← ∗ ←∗

↑ ↑ ↑

3

j 1

1E

k 1 i1

=

3det E 1=


183

Row Reduced Algorithm – Gaussian Elimination

Row operations: Step 1 Find the first pivot column

(leftmost non-zero column) A Step 2 Select any non-zero entry in the 1 2R R↔ pivot column as a pivot. If necessary interchange rows to move pivot into pivot position Step 3 Use row replacement operations to 2 1 2R 3R R− →

create zeroes in all positions 3 1 3R 2R R+ → below the pivot 4 1 4R R R− →

Step 4 Ignore the pivot rows and repeat 3 2 3R R R+ →

steps 1-3 to the remaining matrix 4 2 4R 2R R− →

2R2−

3 4R R↔

3R20

4R2

Step 5 Starting from the rightmost pivot 3 4 31R R R

10− →

use row replacement operations 2 4 21R R R2

+ →

to create all zeroes above pivot 1 4 1R R R− →

(back elimination) 2 3 22R R R2

+ →

1 3 1R 3R R− → 1 2 1R 2R R− → We obtained the RREF

3 3 4 0 21 1 2 3 12 2 2 3 1

1 1 2 5 1

− − − −

− − ↑

1 1 2 3 13 3 4 0 22 2 2 3 1

1 1 2 5 1

−

− − − − −

1 1 2 3 10 0 2 9 10 0 2 9 30 0 4 2 0

−

− − − −

1 1 2 3 19 10 0 12 2

0 0 0 0 20 0 0 20 2

− − −

1 1 2 3 19 10 0 12 2

10 0 0 110

0 0 0 0 1

− − −

1 1 0 0 0

0 0 1 0 0

0 0 0 1 0

0 0 0 0 1

−

1 1 0 0 0

0 0 1 0 0

0 0 0 1 0

0 0 0 0 1

−


9. LINEAR SYSTEMS: We introduced all necessary foundation and tools to solve and to determine the properties of a system of algebraic equations. The linear system of m algebraic equations for n unknowns 1 2 nx ,x ,...,x can be defined in the following form:

standard form

11 1 12 2 1n n 1

21 1 22 2 2n n 2

m1 1 m2 2 mn n m

a x a x ... a x ba x a x ... a x b

a x a x ... a x b

+ + + =

+ + + =

+ + + =

where the system coefficients ija ∈ and the system constants jb ∈ are fixed real numbers.

System coefficients can be organized in the matrix component form or in the column-vector form:

11 12 1n

21 22 2nmn

m1 m2 mn

a a aa a a

a a a

= ∈

A

, ( )1 2 n=A a a a ,

1i

2i mi

mi

aa

a

= ∈

a

The vector of constants and the vector of unknowns can be written as:

1

2 m

m

bb

b

= ∈

b

1

2 n

n

xx

x

= ∈

x

Augmented matrix (matrix of coefficients with additional column of constants):

augmented matrix

11 12 1n 1

21 22 2n 2#m 1,n

m1 m2 mn m

a a a ba a a b

a a a b

+

= ∈

A

( )#1 2 n =A a a a b

With the help of these notations, the linear system can be written in the matrix form:

non-homogeneous system =Ax b

or in the vector form: 1 1 2 2 n nx x ... x+ + + =a a a b

Associated with this system is the homogeneous system ( =b 0 ):

homogeneous system =Ax 0

Solution of the non-homogeneous system =Ax b is any vector n∈x which satisfies this system. If the linear system has at least one solution, then it is called consistent, otherwise it is inconsistent.

The homogeneous system =Ax 0 has at least one solution =x 0 (trivial solution), and therefore, it is always consistent. The non-zero vector ≠x 0 satisfying =Ax 0 is called a non-trivial solution.

For the linear system we want to get the answer to the following questions: – Is the system consistent? – And if yes, then how many solutions does it have and how are they found? The main method of solution of the liner system is Gaussian Elimination:

row reduction of the matrix A for a homogeneous system, and row reduction of an augmented matrix #A for a non-homogeneous system.


185

Row reduction of A for homogeneous system =Ax 0 . Solution space of homogeneous system ker A .

If in the RREF of matrix A there exists a non-pivot column, then this column corresponds to a free variable. In this case, there exists a non-trivial solution of the homogeneous system. For example, let the RREF have the following form:

This row reduced echelon form represents the linear system =Ax 0 : 1 2 5x 2x 3x 0+ + =

3 5x 4x 0+ =

4 5x x 0+ = Then the corresponding linear system with arbitrary values for free parameters

2x and 5x can be written as

1 2 5x 2x 3x 0+ + =

2 2x x=

3 5x 4x 0+ =

4 5x x 0+ =

5 5x x= Solve each equation in terms of free variables

1 2 5x 2x 3x= − −

2 2x x=

3 5x 4x= −

4 5x x= −

5 5x x= and rewrite the system in column-vector form; and then expand the right hand side into component form with free variables as coefficients

1 2 5

2 2

3 2

4 5

5 5

x 2x 3xx x

x 4xx xx x

− − = = −

−

x 2 5

2 31 0

x x4 00 10 1

− − = +−

−

2 5x ,x ∈

For any values of free parameters 2 5x ,x ∈ , this equation provides the solution x of the homogeneous system. Therefore, the solution vector is a linear combination of two vectors or a span of these two vectors.

c ker=x A = the solution space of the homogeneous system =Ax 0

(also called a complimentary solution)

c ker=x A

1

2

3

4

5

xxxxx

=

{ }1 2

2 31 0

span , span ,4 00 10 1

− − = =−

−

u u

free varibles

1 0 0 pivot rows 0 0 1 0 0 0 0 1

0 0 0 0 0

pivot columns

↓ ↓

∗ ∗ ←

← ∗ ←∗

↑ ↑ ↑

1 2 3 4 5x x x x x

1 2 0 0 3 pivot rows 0 0 1 0 4 0 0 0 1 1

0 0 0 0 0

basic variables

←

← ←

↑ ↑ ↑

2 31 0

span ,4 00 10 1

− − −

−

0

5

ker =solution spaceof =

A

Ax 0


This solution set of the homogeneous system =Ax 0 is a vector space which we called the kernel of the matrix A . Here, we explicitly constructed the vector space ker A . Because it is spanned by two linearly independent vectors, it has a dimension two: ( )dim ker 2=A . Then according, to the Theorem, the dimension of the set spanned by the column vectors of the matrix A is determined as

( )dim ker rank n+ =A A ( )rank n dim ker 5 2 3= − = − =A A

This dimension corresponds to the number of pivot columns in the RREF of A . If all columns in RREF are pivot columns then there are no free variables, and the homogeneous system in this case has a trivial solution =x 0 . Here, we demonstrated the technique of explicit construction of the solution space ker A of the homogeneous system =Ax 0 .

There are some facts about the consistency of linear systems:

Theorem14 If the number of unknowns n is greater than the number of equations m , then homogeneous linear system has the non-trivial solutions:

Let

11 12 1n

21 22 2nmn

m1 m2 mn

a a aa a a

a a a

= ∈

A

with m n< , then

there always exists

1

2

n

xx

x

= ≠

x 0

such that =Ax 0

Theorem15 The dimension of the solution space ker A of the homogeneous

linear system =Ax 0 is equal to the number of free variables in the row reduced echelon form of the matrix A .

If m n< , then the dimension of solution space is at least n m− . Theorem16 The non-homogeneous system =Ax b is consistent if and only if

the RREF of the augmented matrix ( )#1 2 n =A a a a b

has no pivot in the last column (no row of the form

( )0 0 0 1 :

The solution set of the consistent linear system =Ax b consists of - a unique solution if there are no free variables in RREF;

- infinitely many solutions when there are free variables. In this case the dimension of the solution space is equal to the dimension of the ker A (number of free variables).

no pivot in last column

pivot rows

pivot columns

1 * 0 0 * * 0 0 1 0 *

0 0 0 1 *00 0 0 0 0

↓

←

←∗ ∗ ←

↑ ↑ ↑


187

Theorem17 (Solution of the non-homogeneous system =Ax b ) Let the vector p n∈x be some particular solution of =Ax b .

Then any solution of the non-homogeneous system =Ax b is represented as

c p= +x x x

where c ker=x A is the solution of homogenous system =Ax 0 (complimentary solution).

Proof: 1) Show that c p= +x x x is a solution of =Ax b . Indeed

( )c p c p= + = + = + =Ax A x x Ax Ax 0 b b

2) Show that for any solution of =Ax b , the vector cx defined by the equation c p= +x x x is a solution of the homogeneous system. Indeed

c p= +x x x c p= −x x x

( )c p p= − = − = − =Ax A x x Ax Ax b b 0 ■

Row reduction of the augmented matrix #A – procedure for solution of non-homogeneous system =Ax b :

The solution of non-homogeneous system =Ax b consists of the following steps:

1) Write an augmented matrix ( )#1 2 n =A a a a b

2) Use Gaussian elimination to obtain RREF – determine if linear system is consistent

3) Solve the system for basic variables 4) Rewrite solution in a vector form using free variables as the parameters

nvector space

general solution ofnon-homogeneous system =Ax b

ker = general solution ofhomogeneous system =

AAx 0

p

particular solution =Ax b px

cx

c p= +x x x

0

b

=Ax 0

=Ax b

col A0

mvector space Ax


Example: Find the solution of the linear system 1 2x 2x 3+ = 1 23x 6 x 9+ = 1) augmented matrix:

# 1 2 3A

3 6 9

=

2) Gaussian elimination:

1 2 3

3 6 9

2 1 2R 3R R− →

31 2 00 0

RREF 1

2

x basic variablex free parameter

3) Equivalent system:

1 2x 2x 3+ = 2 2x x=

Solve for basic variables: 1 2x 2x 3= − + 2 2x x= 4) Rewrite in a vector form:

1 2

2 2

x 2x 3x x

− + =

From which the general solution follows as:

12

2

x 2 3x

x 1 0−

= +

2x ∈

The general solution of the non-homogeneous system can be written in a vector-parametric form:

pt= +x v x where p

2 3,

1 0−

= =

v x

The solution has a geometrical visualization in the 1 2x x -plane:

1

2

x 2t

x 1−

=

is a parametric equation of the line along the

vector 2

1−

=

v , representing the solution set of the

homogeneous system (kernel of the matrix A ).

Then by translation of this set by a constant vector p

3

0

=

x ,

the general solution of a non-homogeneous linear system is obtained as the parametric equation of the line:

1

2

x 2 3t , t

x 1 0−

= + ∈

21

− =

v

p

3

0

=

x

1

2

x 2 3t , t

x 1 0−

= + ∈

2 x

1 x00


189

LINEAR SYSTEMS WITH SQUARE MATRICES: Theorem18 Let nn∈A . The homogeneous linear system =Ax 0 has a

non-trivial solution if and only if

det 0=A

Theorem19 If nn∈A is invertible then for any n∈b the linear system =Ax b has the unique solution

1−=x A b

Theorem 20 (Cramer’s Rule)

If nn∈A is invertible then for any n∈b the unique solution of the linear system =Ax b is the vector

1

2 n

n

xx

x

= ∈

x

the components of which are determined by the

following formula:

kk

detx

det=

BA

k 1,2,...,n=

where matrices kB are obtained from the matrix A by

replacing the column-vector ka by the vector of constants b : [ ]k 1 k 1 k 1 n− +=B a a b a a k 1,2,...,n=

Example: Solve the linear system

1 2 33x 2x 4x 1+ + =

1 2 32x x x 0− + =

1 2 31x 2x 3x 1+ + = Rewrite the system in the matrix form:

=Ax b where 3 2 42 1 11 2 3

= −

A , 101

=

b , 1

2

3

xxx

=

x

Determinant of the matrix of coefficients det 5= −A , therefore:

1) The solution of the linear system is given by

1−=x A b

13 2 4 12 1 1 01 2 3 1

− = −

2 61 15 51 1 1 0

4 7 115 5

− − = − − −

15

025

−

=

2) The same solution can be obtained with Cramer’s Rule:

1

1 2 40 1 11 2 3 1x3 2 4 52 1 11 2 3

−

= = −

−

, 2

3 1 42 0 11 1 3

x 03 2 42 1 11 2 3

= =

−

, 3

3 2 12 1 01 2 1 2x3 2 4 52 1 11 2 3

−

= =

−


INVERSE MATRIX THEOREM: The following theorem combines some previous results and reveals the connection between different aspects of linear algebra:

Theorem 21 (the inverse matrix theorem)

Let nn∈A be a square n n× matrix, ( )1 2 n =A a a a .

Then the following statements are equivalent: 1. A is invertible

2. There exists 1nn

− ∈A such that 1 1− −= =A A AA I

3. =Ax 0 has only the trivial solution =x 0

4. For any n∈b there exists n∈x such that =Ax b

5. The set of column-vectors { }1 2 n, ,...,a a a is linearly independent

6. { } n1 2 nspan , ,..., =a a a

7. ncol =A

8. rank n=A

9. A has n pivots

10. A is row equivalent to the identity matrix I

11. { }ker =A 0

12. ( )dim ker 0=A

13. det 0≠A


191

10. EIGENVALUE PROBLEM: We have seen that linear transformation of a vector space can be defined with the help of matrices. Consider the linear transformation

( ) n nT : →x defined by an n n× matrix A as ( )T =x Ax . There is the following question concerning the output of the linear transformation: are there some vectors n∈x which only are scaled under the given transformation but not rotated:

λ=Ax x

Such vectors if they exist are called the eigenvectors and the corresponding scaling coefficients are called the eigenvalues of A . They can be found by solution of vector equation λ=Ax x . It appears that these eigenvalues, in general, are the real or complex, or λ ∈ , and the corresponding eigenvectors are the vectors over or .

Definition 1 (eigenvalues and eigenvectors)

Let nn∈A be a n n× matrix. The value or λ ∈ for which equation

λ=Ax x

has a non-trivial solution n n,∈x , ≠x 0 is called an eigenvalue of the matrix A . Non-zero solution vector x is called an eigenvector corresponding to eigenvalue λ .

It is obvious that if x is an eigenvector corresponding to the eigenvalue λ , then any scalar multiple cx also is an eigenvector corresponding to the eigenvalue λ . Indeed:

( ) ( )c cλ=A x x ⇒ c cλ=Ax x ⇒ λ=Ax x provided c 0≠ . And if 1 k,...,x x are the eigenvectors corresponding to eigenvalue λ , then any their linear combination 1 1 k kc ... c+ +x x also is an eigenvector corresponding to eigenvalue λ . Definition 2 eigenspace ( )Eig λA

The set of all eigenvectors corresponding to eigenvalue λ completed with the zero vector (which is the trivial solution of λ=Ax x ) is called the eigenspace of the matrix A corresponding to λ :

( ) { }Eig λ λ= ≠ =A x 0 Ax x { }∪ 0

Eigenvectors and eigenvalues of the matrix A can be found by solving the vector equation

λ=Ax x

λ− =Ax x 0

λ− =Ax Ix 0

eigenvalue problem ( )λ− =A I x 0

This is the homogeneous linear system which has a non-trivial solution only if the matrix of the system has the zero determinant

( )det 0λ− =A I

Ax

0

x

λx

1 1 1λ=Ak k

0

1k

( )1

eigenspaceEig λA

( )2

eigenspaceEig λA

2 2 2λ=Ak k


Therefore, the eigenvalues of the matrix A have to satisfy this equation, which after evaluation of the determinant becomes an algebraic equation in λ of degree n:

n n 1n n 1 1 0... 0α λ α λ α λ α−

−+ + + + = with real coefficients iα ∈ According to the Fundamental Theorem of Algebra there exist exactly n roots of this equation which can be real distinct, repeated, or complex conjugate. This equation is called the characteristic equation:

Characteristic equation ( )

11 12 1n

21 22 2n n n 1n n 1 1 0

n1 n2 nn

a a aa a a

det ... 0

a a a

λλ

λ α λ α λ α λ α

λ

−−

−−

− = = + + + + =

−

A I

Theorem22 The scalar λ is an eigenvalue of the matrix nn∈A if and only if it satisfies the characteristic equation ( )det 0λ− =A I

Theorem23 If 1 2 m, ,...,λ λ λ are distinct eigenvalues of the matrix nn∈A , then

the corresponding eigenvectors 1 2 m, ,...,x x x are linearly independent .

Theorem24 If λ is a root of multiplicity k of the characteristic equation

( )det 0λ− =A I and therefore it is an eigenvalue of the matrix nn∈A , then the dimension of the corresponding eigenspace is ( )dim Eig kλ ≤A . So, if λ is a root of multiplicity 1 , then ( )dim Eig 1λ =A It means that only one linearly independent eigenvector corresponds to λ .

Theorem25 If matrix nn∈A is symmetric, T =A A , then its eigenvalues are real.

Theorem26 If matrix nn∈A is invertible then 0 is not its eigenvalue. If 0 is an eigenvalue of A then det 0=A . PROCEDURE FOR SOLVING AN EIGENVALUE PROBLEM: 1. Construct the characteristic equation ( )det 0λ− =A I 2. Find the roots of the characteristic equation kλ eigenvalues (can be distinct, repeated, complex conjugate) 3. For each eigenvalue kλ solve the linear system ( )kλ− =A I u 0 to find eigenvectors ku corresponding to kλ .


193

EXAMPLES: 1. (distinct real eigenvalues)

Solve the eigenvalue problem for 2 0

A1 3

=

.

1) Characteristic equation:

22 0det 5 6 0

1 3λ

λ λλ

− = − + = −

2) The roots are: 1 2λ = 2 3λ = 3) Vector equation for 1 2λ = : ( )1 1λ− =A I u 0

1

2

x0 0 0x1 1 0

=

Row reduction of matrix of coefficients:

1 20 0 R R

1 1↔

1 1

0 0 row reduced echelon form

Corresponding equation includes one basic variable and one free variable:

1 2x x 0+ = Solve for the basic variable 1 2x x= − 2x ∈

Any value of free parameter 2x provides a solution, choose 2x 1= − , then 1x 1= . Therefore, the solution vector (eigenvector) is

11

2

x 1x 1

= = −

u

Vector equation for 2 3λ = : ( )2 2λ− =A I u 0

1

2

x1 0 0x1 0 0

− =


1 2 21 0 R R R

1 0− + →

11 0 R

0 0− −

1 0

0 0 row reduced echelon form

The corresponding equation includes one basic variable and one free variable:

1 2x 0 x 0+ ⋅ = Solve for the basic variables 1 2x 0 x= ⋅ 2x ∈

Any value of the free parameter 2x provides a solution, choose 2x 1= , then 1x 0= . Therefore, the solution vector (eigenvector) is

12

2

x 0x 1

= =

u


2. (real eigenvalues, repeated eigenvalue)

Solve the eigenvalue problem for 2 1 10 3 10 1 3

− = − −

A .


2 1 1

det 0 3 1 00 1 3

λλ

λ

− − − − = − −

2) The roots are: 1 4λ = 2 3 2λ λ= = eigenvalue of multiplicity 2

3) Vector equation for 1 4λ = :

( )1 1λ− =A I u 0

1

2

3

2 4 1 1 x 00 3 4 1 x 00 1 3 4 x 0

− − − − = − −

1

2

3

2 1 1 x 00 1 1 x 00 1 1 x 0

− − − − = − −


1 2 1

3 2 3

2 1 1 R R R0 1 1 0 1 1 R R R

− − + → − − − − − →

1

2

2 0 2 R 20 1 1 R0 0 0

− − − − − −

1 0 10 1 1 0 0 0 row reduced echelon form

Corresponding system of equation includes two basic variables 1 2x ,x and one free variable 3x :

1 3x x 0+ = 2 3x x 0+ =

Solve for the basic variables

1 3x x= − 2 3x x= − 3x ∈

Any value of the free parameter 3x provides a solution, choose

3x 1= − , then 1 2x x 1= = . Therefore, the solution vector (eigenvector) is

1

1 2

3

x 1x 1x 1

= = −

u


195

Vector equation for 2 2λ = :

( )2 2λ− =A I u 0

1

2

3

2 2 1 1 x 00 3 2 1 x 00 1 3 2 x 0

− − − − = − −

1

2

3

0 1 1 x 00 1 1 x 00 1 1 x 0

− − = −


2 1 2

3 1 3

0 1 10 1 1 R R R0 1 1 R R R

− − − → − + →

0 1 10 0 0 0 0 0 row reduced echelon form

−

The corresponding equation includes one basic variable 1x and two free variables 2 3x ,x (the dimension of kernel is 2):

1 2 30 x x x 0⋅ + − = ⇒ 2 3x x= 2 3x ,x ∈

It is possible to construct two independent vectors with arbitrary 1x ∈ and 2 3x x= . The simplest non-zero vectors satisfying these

conditions are

1

2 2

3

x 1x 0x 0

= =

u , 1

3 2

3

x 0x 1x 1

= =

u

Therefore, for the given matrix, there are two linearly independent vectors corresponding to eigenvalue of multiplicity two.

3. (complex eigenvalues)

If a biλ = + ∈ is an eigenvalue of the matrix with real coefficients nn∈A with the corresponding eigenvector n∈u , then the complex

conjugate of it a biλ = − is also an eigenvalue of nn∈A with the eigenvector u . Indeed, let

λ=Au u

____ ____

λ=Au u

λ=Au u

Example: Solve the eigenvalue problem for 1 5

A2 3

= −

.


21 5det 4 13 0

2 3λ

λ λλ

− = − + = − −


2) The roots are: 1 2 3iλ = + 2 2 3iλ = − 3) Vector equation for 1 2 3iλ = + :

( )kλ− =A I u 0

( )

( )1

2

1 2 3i 5 x 02 3 2 3i x 0

− + = − − +

1

2

x1 3i 5 0x2 1 3i 0

− − = − −


( ) 11 3i 5 -1+3i R

2 1 3i− −

− −

110 5 15i R 5

2 1 3i− +

− −

2 1 2

2 1 3i R R R2 1 3i

− + + →− −

2 1 3i

0 0 row echelon form

− +

The corresponding equation includes one basic variable and one free variable:

( )1 22x 1 3i x 0+ − + =

Solve for the basic variable

( )1 22x 1 3i x= − 2x ∈

Any value of free parameter 2x provides a solution, choose 2x 2= , then 1x 1 3i= − . Therefore, the solution vector (eigenvector) is

11

2

x 1 3i 1 3i

x 2 2 0−

= = = −

u

Then the eigenvector corresponding to the second eigenvalue 1λ is

___________________

2 1

1 3 1 3i i

2 0 2 0

= = − = +

u u

4. (system of ODEs) For application of the eigenvalue problem for solution of the System of

Linear Ordinary Differential equations see Chapter V (Section 5.5).


197

11. LINEAR ALGEBRA WITH MAPLE: Many linear algebra operations can be performed only using linalg package which is downloaded by the command:

> with(linalg):

Entering vectors and matrices: can be defined by array:

> u:=array([3,2,-3]);

:= u [ ], ,3 2 -3

> A:=array([[-1,2,1],[3,2,1]]);

:= A

-1 2 13 2 1

by command: > v:=vector([1,2,-1]);

:= v [ ], ,1 2 -1

> B:=matrix(3,2,[1,2,3,4,5,6]);

:= B

1 23 45 6

Operations with vectors:

> w:=vector([1,0,1]);

:= w [ ], ,1 0 1 dot product:

> dotprod(u,v); 10

cross product: > crossprod(u,v);

[ ], ,4 0 4

angle between two vectors: > beta:=evalf(angle(u,v));beta:=evalf(convert(beta,degrees));

:= β 0.5148059555

:= β 29.49620851 degrees Operations with matrices:

Arithmetic operations are performed by applying command evalm

addition : > c:=evalm(u+v);

:= c [ ], ,4 4 -4 scalar multiplication :

> C:=evalm(2*B);

:= C

2 46 8

10 12

matrix multiplication : > F:=evalm(A&*B);

:= F

10 1214 20

transpose : > transpose(C);

2 6 104 8 12


determinant: > det(F);

32 inverse:

> inverse(F);

58

-38

-716

516

adjoint: > adjoint(F);

20 -12-14 10

row reduced echelon form (RREF): > rref(A);

1 0 0

0 1 12

rank of matrix > rank(A);

2 kernel of matrix:

> kernel(A); { }[ ], ,0 1 -2

column space of a matrix: > colspan(A);

{ },[ ],-1 3 [ ],0 -8

Solution of linear system of algebraic equations

> A:=matrix(2,2,[2,4,-1,3]);

:= A

2 4-1 3

> b:=array([[1],[-2]]);

:= b

1-2

solution with linsolver: > x:=linsolve(A,b);

:= x

1110-310

solution with inverse matrix: > x:=evalm(inverse(A)&*b);

:= x

1110-310

The Eigenvalue Problem:

eigenvalues: > G:=matrix(3,3,[1,2,1,6,-1,0,-1,-2,-1]);

:= G

1 2 16 -1 0

-1 -2 -1

> eigenvalues(G); , ,0 3 -4


199

> eigenvectors(G);

, ,[ ], ,0 1 { }[ ], ,1 6 -13 [ ], ,-4 1 { }[ ], ,-1 2 1

, ,3 1 { }

, ,-1 -3

2 1

with notations: [eigenvalue, multiplicity of eigenvalue, {[corresponding eigenvector]}, ] Differentiation of matrices (each entry is differentiated):

> A:=matrix(2,2,[3*t,sin(3*t),exp(2*t),2*t*exp(4*t)]);

:= A

3 t ( )sin 3 t

e( )2 t

2 t e( )4 t

> Ap:=map(diff,A,t);

:= Ap

3 3 ( )cos 3 t

2 e( )2 t

+ 2 e( )4 t

8 t e( )4 t

Integration of matrices (each entry is integrated)

> Ai:=map(int,A,t);

:= Ai

3 t2

2 −13 ( )cos 3 t

12 e

( )2 t −

12 t e

( )4 t 18 e

( )4 t

Evaluation of values of the functions inside of the matrix A:

> A:=matrix([[sin(x),cos(x)],[exp(x),2*x]]);

:= A

( )sin x ( )cos xex 2 x

> for n from 1 to 2 do A[n,1]:=subs(x=Pi/2,A[n,1]) od;

:= A ,1 1

sin π

2

:= A ,2 1 e

π2

> for n from 1 to 2 do A[n,2]:=subs(x=Pi/2,A[n,2]) od;

:= A ,1 2

cos π

2

:= A ,2 2 π

> evalm(A);

sin π

2

cos π

2

e

π2

π

> evalf(evalm(A));

1. -0.2051033808 10-9

4.810477382 3.141592654

> map(evalf,A);

1. -0.2051033808 10-9

4.810477382 3.141592654


12. REVIEW QUESTIONS: 1) What is a vector space?

2) What operations with vectors are defined in vector spaces?

3) What is the meaning of closure axioms for vector spaces i) and ii)?

4) What is a vector subspace?

5) What conditions should be satisfied by the subset of a vector space to be a

subspace?

6) Recall examples of vector spaces.

7) Is the union of vector subspaces of the vector space V a vector space?

9) Is the intersection of vector subspaces of the vector space V a vector space?

10) What is a linear combination of vectors?

11) What is a span of a set of vectors?

12) What vectors are called linearly independent?

13) Is the set of vectors 1 1 0

, ,0 2 1

linearly independent?

14) What is the basis of a vector space?

15) What is the dimension of a vector space?

16) What is a linear map between vector spaces?

17) What is the image of a linear map?

18) What is the kernel of a linear map?

19) What is the rank of a linear map?

20) What is the relationship between the rank and the dimension of the kernel of

a linear map?

21) When are two matrices equal?

22) How is the sum of two matrices defined?

23) What is a symmetric matrix?

24) For what matrices is the product of matrices defined?

25) Give an example of two non-zero 2 2× matrices such that =AB 0 ?

26) Show that if =AB 0 , for nn, ∈A B , then ( )2 =BA 0 .

27) How is the transformation of vector space defined with the help of

matrices?

28) What is the determinant?

29) What is the matrix inverse?

30) What are the elementary row operations?

31) What is the Row Reduced Echelon Form (RREF)?

32) What is a consistent linear system?

33) When does a homogeneous system =Ax 0 have a non-trivial solution?

34) What is the method of solution of linear systems of equations?

35) What are eigenvalues and eigenvectors?


201

EXERCISES: 1) Prove the following statements for the vector space V : a) The negative element V− ∈u is unique.

b) Zero vector V∈0 is unique.

c) 0 =u 0 for any V∈u .

d) ( )1− = −u u for any V∈u .

2) Let iU V⊂ be the subspaces of the vector space V . Show that the

intersection of subspaces

{ }n

i ii 1

U = V U , i=1,...,n=

∈ ∈u u

is a subspace of V . 3) Let U be a vector space, and V and W be subspaces of U . Prove or disapprove the following statements:

a. Intersection V W∩ is a subspace of U . b. Union V W∪ is a subspace of U .

4) Let V be a vector space and let 1 2 n, ,..., V∈u u u .

Show that { }1 2 nspan , ,...,u u u is a subspace of V .

5) Show that if { }1 2 n, ,...,∈0 u u u then the set { }1 2 n, ,...,u u u is linearly dependent.

6) Determine if the following sets are linearly independent or dependent:

a) 1 2 1

, ,2 1 0

b) 2 1

,1 0

7) What is the standard basis for the vector space

22

a ba,b,c,d

c d = ∈

8) Let V be an n dimensional vector space then (fill in the blank with

, ,> = < ):

a) if { }1 p,...,u u spans V , then p ____ n

b) if { }1 p,...,u u is a basis for V , then p ____ n

c) if { }1 p,...,u u is a linearly independent set in V , then p ____ n

9) Show that if { },u v is a linearly independent subset of n , then

{ },+ −u v u v is also linearly independent. 10) Suppose that 1 2 m, ,...,v v v are linearly independent vectors in n and let u

be any vector that is not in { }1 2 mspan , ,...,v v v . Show that 1 2 m, ,..., ,v v v u are linearly independent.


11) Let A be a square matrix, nn∈A . Show that

( )Ts

12

= +A A A is a symmetric matrix and

( )Ta

12

= −A A A is a skew-symmetric matrix (antisymmetric).

12) Fill in the blank with the best symbol or number:

a) If nn∈A is an invertible n n× matrix, then rank _____=A

b) If nn∈A is an invertible n n× matrix and det d=A , then 1det = _____−A

13) True – False questions: a) If nn, ∈A B are invertible n n× matrices, then ( ) 1 1 1− − −=AB A B ___

b) For any m n× matrix mn∈A , the matrix TAA is symmetric ___

c) For any m n× matrix mn∈A , the matrix TA A is symmetric ___

d) If nn∈A is invertible, the columns of A are linearly independent __

e) All eigenvalues of a symmetric matrix are real ___

f) The set of all invertible n n× matrices is a subspace of the vector space

nn of all n n× matrices ___

14) Prove the following statement by mathematical induction:

Let 1 2 m nn, ,..., ∈A A A be invertible n n× matrices. Then 1 2 mA A A is

invertible and ( ) 1 1 1 11 2 m m m 1 1

− − − −−=A A A A A A .

15) Let 3 1 42 1 02 1 1

= −

A and 1 3 02 1 30 1 2

= − −

B , then compute

a) 3A b) +A B

c) 2 −A B d) AB

e) TA B f) ( )TBA

16) Let 2 1 01 0 3

= −

A and 1 01 20 1

=

B . Determine if it is possible to

perform the following operations, and if yes, find the result:

a) AB b) BA

c) TBA d) T TA B

e) ( )T+A B B f) TAA


203

17) Let

1 12 21 12 2

− =

−

A . Compute 2 =A AA , 3 2=A A A . What will be nA ?

18) Find the Row Reduced Echelon Form of the following matrices:

a)

2 0 30 1 24 2 05 3 15 1 0

b) 2 1 2 0 24 2 0 1 04 2 4 1 0

− −

c)

1 2 1 22 2 1 4

3 2 2 53 8 5 17

− − −

−

d)

2 1 3 1 21 2 0 3 14 2 6 2 4

3 1 0 1 0

− − − − −

−

19) Solve the following linear system of equations:

a) 1 2 3

1 2 3

2x 4x 3x 0x x 2x 0

− + =

+ − = b) 1 2 3

1 2 3

x 2x 3x 02x x 2x 0

+ − =

+ − =

c) 1 2 3 4

1 2 3 4

1 2 3 4

x x 2x x 0x 3x x x 02x x 3x x 0

− + + =

+ − − =

+ − − =

d) 1 2

1 2 3

1 3

3x 3x 02x x x 0

3x x 0

+ =− + + =

+ =

e) 1 2 3 4

1 2 3 4

1 2 3 4

3x 5x 2x 4x 27x 4x x 3x 55x 7x 4x 6 x 3

− + + =

− + + =

+ − − =

f) 1 2 3 4

1 2 3 4

1 2 3 4

3x 2x 5x 4x 26 x 4x 4x 3x 39x 6 x 3x 2x 4

− + + =

− + + =

− + + =

g) 1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

3x 9x 2x 2x x 0x 6 x x x 2x 0x 16 x 6 x 6 x 7x 0

+ + − + =

+ − + + =

+ − + + =

h) 1 2 3 4 5

1 2 3 4

1 2 3 4 5

4x 11x 5x 2x 3x 73x 8x x 2x 52x 5x 3x 2x 3x 3

− + + + =

− + + =

− − + − =

20) Find the determinant of the following matrices:

a) 2 6

1 52

− − =

A b) 1 05 9

=

A

c) 2 1 04 2 5

2 3 1

= − −

A d) 8 5 14 1 21 2 3

− = − −

A

e)

2 0 1 34 0 0 00 1 4 12 0 2 3

= −

−

A f)

1 3 0 05 1 2 10 1 0 53 1 0 0

− =

−

A


21) Find the inverse of the following matrices:

a) 1 52 3

=

A b)

2 02 3

=

A

c) 1 2 10 3 10 2 1

=

A d) 0 1 01 0 12 0 1

=

A

22) Solve the following linear systems of equations using matrix inverse and

Cramer’s Rule:

a) 1 2

1 2

2x x 1x x 2

− =+ =

b) 1 2

1 2

4x 5x 35x x 2

+ = −− + =

c) 1 2

2 3

1 2 3

2x 4x 12x 4x 2

x x x 0

+ =+ =

− − + =

d) 1 3

1 2 3

1 3

2x 5x 23x x 5x 1x 2x 1

+ = −

+ + =

+ = −

23) Solve the eigenvalue problem for the following matrices:

a) 1 02 0

=

A b)

5 13 1

− =

A

c) 0 11 0

− =

A d)

2 33 2

= −

A

e) 3 1 11 5 1

1 1 3

− = − − −

A f) 1 0 02 1 23 2 1

= −

A

g) 3 1 13 2 13 1 0

= − −

A h) 0 0 11 0 01 1 1

− = −

A

i)

1 0 0 00 2 0 00 0 3 00 0 0 5

− =

A k)

1 2 0 10 1 0 10 0 2 00 0 0 2

− =

A

24) Show that the matrix (the rotation matrix)

cos sinsin cos

θ θθ θ

− =

A

has complex eigenvalues when θ is not a multiple of π


205


Documents

the concept of vector spaces vectors and matrices linear ...vps/ME505/IEM/03 01.pdfcontinuously differentiable on the interval (a,b). The notion of vector spaces helps to obtain and