General Vector Spaces

Lecture 16

General Vector Spaces

Basis of Row Space and Column Space

Rank and Nullity

Covered Range : 4.8 +４.9

2021/11/17Lec16_LA21_ Singling Lee 1

Terminologies

Row Space ;

Column Space ;

Null Space ;


Recap Row(A), Col(A)and Null(A)

Definition 2 : A is an m×nmatrix A=[aij]

Row(A) ⊆ Rn: Row space of A ⇒ span{r1,r2,…,rm}

Col(A) ⊆ Rm: Column space of A ⇒ span{c1,c2,…,cn}

Null(A) ⊆ Rn: Null space of A ⇒ Solution of Ax=0

Note : Ax=0 always has a solution x=0 (Orthogonal to each row vector)

The solution set is a vector subspace of Rn


Recap Col (A) and the Solution of Ax=b

Theorem4.7.1 Ax = b is consistent if and only if b is in the column space of A.

Means b ∈ Col(A) = span {c1, c2, … ,cn}.

Ax : Linear combinations of column vectors of A .


Recap Orthogonality of Row(A) and Null(A)

Theorem 3.4.3 : If 𝐴 is an 𝑚 × 𝑛 matrix, then the solution set of the homogeneous linear system 𝐴𝐱 = 𝟎 consists of all vectors in 𝑅𝑛 that are orthogonal to every row vector of 𝐴.


Recap Relation Between Ax = 0 and Ax = b

Theorem 4.8.2 : If x0 is any solution of a consistent linear system of Ax = b and ifS={v1 ,v2 , … , vk} is a basis for the null pace of A, then every solution of Ax = b can be expressed in the form

x = x0 + c1v1 + c2v2 + … + ckvk


Recap Subspace of Ax = 0 by Spanning (II)

Example 6 :

x2 = r, x4 = s, x5 = t

Solution : 𝐱 = (−3𝑟 − 4𝑠 − 2𝑡, 𝑟, −2𝑠, 𝑠, 𝑡, 0)

parametric equation : (x1, x2, x3, x4, x5 , x6) =

r (-3, 1, 0, 0, 0, 0) + s (-4, 0, -2, 1, 0, 0) + t (-2, 0, 0, 0, 1, 0)

⇒ 𝐯𝟏 = −3,1,0,0,0,0 , 𝐯2 = −4,0, −2,1,0,0 , 𝐯3 = (−2,0,0,0,1,0)

linearly ind. span the solution space and dimension = 3


1 3 0 4 2 0 00 0 1 2 0 0 00 0 0 0 0 1 00 0 0 0 0 0 0

Recap Row Operations : Row(A) ,Null(A)

Theorem 4.8.3 :

a) Row equivalent matrices have same row space

b) Row equivalent matrices have same null space

Elementary row operations do not change the row space and null space of a matrix


Recap Bases for Row(A) and Col(A) on ref Matrix

Theorem 4.8.4 :

If a matrix R is in row echelon form, then the rowvectors with the leading 1's (the nonzero rowvectors) form a basis for the row space of R, and thecolumn vectors with the leading 1's of the rowvectors from a basis for the column space of R.


Recap Bases for Row(A) and Col(A) on ref Matrix

Example 3 pp 268 : (Row-Echelon Matrix)

𝑅 =

1 −2 5 0 30 1 3 0 00 0 0 1 00 0 0 0 0 Theorem 4.8.4

𝐫𝟏 = 1 −2 5 0 3𝐫𝟐 = 0 1 3 0 0𝐫𝟑 = 0 0 0 1 0

𝐜𝟏 =

1000

, 𝐜𝟐 =

−2100

, 𝐜𝟒 =

0010

⇒


Recap Find the Basis of Row(A)

Find the basis of Row(A):

1. Use Gaussian Elimination to construct R : R : ref matrix of A,

2. Basis of Row Space of A are formed by “rows with leading 1's” in R.


Recap Basis for Column Space Theorem 4.8.5 : If A and B are row equivalent matrices,

1) A given set of column vectors of A is linearly independent iff the corresponding column vectors of Bis linearly independent

2) A given set of column vectors of A forms a basis of the column space of A iff the corresponding column vectors of B forms a basis for the column space of B

How to use this theorem?


Recap Examples for Basis of Col(A) Example 5 pp.270 Basis for Col(A)

𝐴 =

1 −3 4 −2 5 42 −6 9 −1 8 22 −6 9 −1 9 7

−1 3 −4 2 −5 −4

⇒

R =

1 −3 4 −2 5 40 0 1 3 −2 −60 0 0 0 1 50 0 0 0 0 0

The correspond vectors in A :

c1, c2, c5 is a basis of Col(A)


Reduce A to a row-echelon matrix (Gaussian Elimination) R

𝐜1′ =

1000

𝐜3′ =

4100

𝐜5′ =

5−210

⇒

c1’, c３’, c5’,

basis of Col(R)

𝐜𝟏 =

122

−1

𝐜𝟑 =

499

−4

𝐜5 =

589

−5

Lecture 16



Rank and Nullity



Basis from the Rows of AExample 6 pp. 270: Find a basis for row(A) spanned by r1, r2, r3, r4 of A

Consisting entirely of row vectors from A {使用原來的vectors r1, r2, r3, r4 }

Use AT:

→ column 1, 2, 4 for AT


1 2 0 0 3

2 5 3 2 6

0 5 15 10 0

2 16 18 8 6

A

1 2 0 2 1 2 0 2

2 5 5 6 0 1 5 10

0 3 15 18 0 0 0 1

0 2 10 8 0 0 0 0

3 6 0 6 0 0 0 0

TA

1 2 31, 2,0,0,3 2, 5, 3, 2,6 2,6,18,8,6 r r r

→ consists entirely of row vectors from A

Find Basis Vectors and Linear Comb. of Nonbasis Vectors

Example 8 pp. 272 :

Find a subset of vectors v1=(1, -2, 0, 3) v2=(2, -5,-3, 6) v3=(0, 1, 3, 0) v4=(2, -1, 4, -7) v5=(5, -8, 1, 2) that forms a basis

1) Use original vector as column vectors Column Space

2) Express each vector by the basis


1 2 0 2 5−2 −5 1 −1 −80 −3 3 4 13 6 0 −7 2

⇒

1 0 2 0 10 1 −1 0 10 0 0 1 10 0 0 0 0

𝐯𝟏 𝐯𝟐 𝐯𝟑 𝐯𝟒 𝐯𝟓 𝐰𝟏 𝐰𝟐 𝐰𝟑 𝐰𝟒 𝐰 𝟓

{𝐯𝟏 , 𝐯𝟐 , 𝐯𝟒 } ⇐ {𝐰𝟏 , 𝐰𝟐 , 𝐰𝟒 }

Find Basis Vectors and Linear Comb. of Nonbasis Vectors

Example 8 pp. 272 :

2) Express each vector by the basis

(b) w3 = 2w1 - w2 v3 = 2v1 - v2

w5 = w1 + w2 + w4 v5 = v1 + v2 + v4


0130

= 2

1−203

−

2−5−36

𝐯3 𝐯1 𝐯2

5−812

=

1−230

+

2−5−36

+

2−147

𝐯5 𝐯1 𝐯2 𝐯4

1 2 0 2 5−2 −5 1 −1 −80 −3 3 4 13 6 0 −7 2

⇒

1 0 2 0 10 1 −1 0 10 0 0 1 10 0 0 0 0

𝐯𝟏 𝐯𝟐 𝐯𝟑 𝐯𝟒 𝐯𝟓 𝐰𝟏 𝐰𝟐 𝐰𝟑 𝐰𝟒 𝐰 𝟓

{𝐯𝟏 , 𝐯𝟐 , 𝐯𝟒 } ⇐ {𝐰𝟏 , 𝐰𝟐 , 𝐰𝟒 }

Since 𝐸𝐯𝒋 = 𝑐1𝐸𝐯𝟏 + ⋯ + 𝑐𝑘𝐸𝐯𝒌 ⇒ 𝐯𝒋 = 𝑐𝟏𝐯𝟏 + ⋯ + 𝑐𝒌𝐯𝒌!

Steps to Find a Basis of Vectors and Linear Comb. Of Nonbasis Vectors

Given S = {v1, v2, …, vk } ⊆ Rn

Find a basis ⊆ S for span(S)

1) Matrix A has v1, v2, …, vk as it column vectors

2) Reduce A to reduced row-echelon form R.

3) Columns that contain leading 1’s in R⇒ The corresponding vectors of A are the basis

4) Express each nonbasis column vector of R as a linear combination of the leading 1 columns in R!

5) Replace the linear combination of nonbasis column vectors in R by the corresponding column vectors in A


Recap General Questions

Question 1 : What are the relationships between Ax = b and Row Space, Column Space and Null Space of the coefficient matrix A ?

Question 2: What are the relationships among Row(A), Col(A) and Null(A) of a given matrix A ?


Conclusion

Ax = b is consistent if and only if b ∈ Col(A),

Use row echelon form (ref) to find basis of Row(A) and Col(A).

Use reduced row echelon form (rref) to find basis of Null(A).

General solutions of Ax = b : Special solution + Span (Basis of Null(A))

Orthogonality of Row(A) and Null(A).


Lecture 16



Rank and Nullity



dim(Row(A)) = dim(Col(A))Theorem 4.9.1 : The row space and the column space

have the same dimension !

Basis of Row(A) : The leading one rows in R (row-echelon matrix)!

Basis of Col(A) : Column vectors in A corresponding to the leading one columns in R (row-echelon matrix)!


Rank and Nullity

Definition 1 : Given a matrix A : m×n

Rank(A) = dim(row space of A) = dim(column space of A)

Nullity(A) = dim(null space of A)

Rank(A) = Rank(AT)

Rank(A) ≦min(m , n)


Examples for Rank and Nullity (I) Example 1 pp 277 : Rank and Nullity

The solution space of Ax=0


A=

−1 2 0 4 5 −332

−7−5

22

04

16

41

4 −9 2 −4 −4 7

->

1 0 −4 −28 −37 1300

10

−20

−120

−160

50

0 0 0 0 0 0

From Row1: 𝑥1−4𝑥3 −28 𝑥4 −37 𝑥5 + 13𝑥6 = 0

Row2: 𝑥2 −2𝑥3 − 12𝑥4 −16 𝑥4 + 5𝑥6 = 0

𝑥1 =4𝑥3 + 28𝑥4 + 37𝑥5 − 13𝑥6 ; 𝑥2 = 2𝑥3 +12𝑥4 + 16𝑥5 − 5𝑥6

𝑥3= r ， 𝑥4 = 𝑠， 𝑥5 = 𝑡， 𝑥6 = 𝑢𝑥1= 4𝑟 + 28𝑠 + 37𝑡 − 13𝑢𝑥2= 2𝑟 + 12𝑠 + 16𝑡 − 5𝑢

Examples for Rank and Nullity (II)

The Solution of Ax= 0 with 4 parameters

Rank =2 , dim(Null(A)) = 4; Nullity = 4


𝑥3= r ， 𝑥4 = 𝑠， 𝑥5 = 𝑡， 𝑥6 = 𝑢

A=

−1 2 0 4 5 −332

−7−5

22

04

16

41

4 −9 2 −4 −4 7

->

1 0 −4 −28 −37 1300

10

−20

−120

−160

50

0 0 0 0 0 0

𝑥1= 4𝑟 + 28𝑠 + 37𝑡 − 13𝑢𝑥2= 2𝑟 + 12𝑠 + 16𝑡 − 5𝑢

𝑥1

𝑥2

𝑥3

𝑥4

𝑥5

𝑥6

= r

421000

+ s

28120100

+ t

37160010

+ u

−13−50001

Find the basis of Null(A) (I)

Find the basis of Null(A):

1. Use Gaussian Elimination to construct RR : RR : Reduced row-echelon matrix of A,

2. Basis of Null(A) (Rn) are formed by :

Finding Free variables (parameters) xk : without leading 1's columns in R


Find the basis of Null(A) (II)

Find the basis of Null(A) (⊆Rn) :

Finding Free variables (parameters) xk : without leading 1's columns in R

Basis of Null (A) in Rn :

(a) having 1 at the position k for the corresponding vector,

(b) reverse the sign in column k of R at the appropriate positions in the vector,

(c) zeros at the positions for the other free variables!


Basis of Null(A) (II)

The Solution of Ax= 0 with 4 parameters

Rank =2 , dim(Null(A)) = 4; Nullity = 4


𝑥3= r ， 𝑥4 = 𝑠， 𝑥5 = 𝑡， 𝑥6 = 𝑢

A=

−1 2 0 4 5 −332

−7−5

22

04

16

41

4 −9 2 −4 −4 7

->

1 0 −4 −28 −37 1300

10

−20

−120

−160

50

0 0 0 0 0 0

𝑥1= 4𝑟 + 28𝑠 + 37𝑡 − 13𝑢𝑥2= 2𝑟 + 12𝑠 + 16𝑡 − 5𝑢

𝑥1

𝑥2

𝑥3

𝑥4

𝑥5

𝑥6

= r

421000

+ s

28120100

+ t

37160010

+ u

−13−50001

reduced row echelon matrix

← Linearly Ind.

Dimension Theorem for MatrixTheorem 4.9.2: If A is a m×n matrix then Rank(A) + Nullity(A) = n

From Reduced Row-Echelon form.

Rank(A) + = n

Rank(A) + Nullity(A) = n

Similarly : Rank(AT) + Nullity(AT) = m

Rank(A) + Nullity(AT) = m

→ If Rank(A) = r then , Nullity(A) = n-r ; Nullity(AT) = m - r

number of free

variables


Nullity and Number of Parameters

Theorem 4.9.3: If A is an m×n matrix then

(a) Rank(A) = number of leading one variables in rref of matrix A;

(b) Nullity(A) = number of parameters (free variables)

in general solution of Ax=0


Rank, Nullity, and Linear Systems

Example 4 pp. 279

a) Find the number of parameters in general solution of Ax = 0 if A is a 5 x 7 matrix of rank 3!

b) Find the rank of a 5 x 7 matrix for which Ax=0 has a two-dimensional solution space.

Sol:

a) Nullity(A) = n- rank(A) = 7-3 = 4

b) Nullity(A) = 2. rank(A) = 7-2 = 5


Equivalent Statements

Theorem 4.9.4 : If 𝐴𝐱 = 𝐛 is a consistent linear system of 𝑚 equations in n unknown, and if 𝐴 has rank 𝑟, then the general solution of the system contains 𝑛 − 𝑟 parameters.


Fundamental Spaces of a Matrix

Theorem 4.9.5: If A is any matrix, rank(A)=rank(AT)

If A : m × n matrix

rank(A) + nullity(A) = n

rank(A) + nullity(AT) = m

dim[row(A)] = dim[col(A)]= r = rank(A)

dim[null(A)]=n-r dim[null(AT)]=m-r


The Consistence Theorem

If Ax = b and A is m×n , then the following are equivalent.

a) Ax = b is consistent.

b) b ∈ Col (A)

c) Rank(A) = Rank( [A | b] )


Condition for Ax = b has ≦1 Solution

A is m×n, the following are equivalent.

1) Ax = 0 has only the trivial solution.

2) The column vectors of A are linearly independent

3) Ax = b has at most one solution (none or one)for every m×1 vector b


Orthogonal Complement

Definition 2: W is a subspace of Rn, the set of all vectors in Rn that are orthogonal to every vector in W is called the orthogonal complement(正交補

餘) of W, denote W⊥


Properties of Orthogonal Complements (I)

Theorem 4.9.6: W: subspace of Rn

a) W⊥ is a subspace of Rn

b) The only vector common to W and W⊥ is 0

c) The orthogonal complement of W⊥ is W


Examples for Orthogonal Complements

1. In R2 , what is the orthogonal complement W⊥ of a line W through the origin?

2. In R3 , what is the orthogonal complement W⊥ of a plane W through the origin ?

3. In R3 , what is the orthogonal complement of zerovector ?

4. In R3 , what is the orthogonal complement W⊥ of subspace W spanned by vectors (1,5,1) and (2,2,2)?


Orthogonal Comp. for row(A) and null(A)

Theorem 4.9.7: A: m × n matrix

a) Null(A) and Row(A) are orthogonal comp. in Rn

from Ax=0 : each element in row space is orthogonal to any solution x;

b) Null(AT) and Col(A) are orthogonal comp. in Rm

from ATy=0


Fundamental Theorem of Linear Algebra


Solving Ax = b for an 𝑚 × 𝑛 matrix A with rank(A) = r

Equivalent StatementsTheorem 4.9.8: If A is an n×n matrix, no duplicate

rows and columns, then the following are equivalent.

a) A is invertible.

b) Ax=0 has only the trivial solution.

c) The reduced row-echelon form of A is In.

d) A is expressible as a product of elementary matrices.

e) Ax=b is consistent for every n×1 matrix b.

f) Ax=b has exactly one solution for every n×1 matrix b.


Equivalent Statements

g) det(A) ≠0.

h) The column vectors of A are linearly ind.

i) The row vectors of A are linearly ind.

j) The column vectors of A span Rn. (col(A) = Rn )

k) The row vectors of A span Rn. (row(A) = Rn )

l) The column vectors of A form a basis for Rn.

m) The row vectors of A form a basis for Rn.


Equivalent Statements [cont.]

n) A has rank n.

o) A has nullity 0

p) The orthogonal complement of the null space of A is Rn.

q) The orthogonal complement of the row space of A is {0}.


Documents

General Vector Spaces