Column and row space of a matrix

• Recall that we can consider matrices as concatenation of rows or columns.

𝐴 =

𝑎11 𝑎12 𝑎13𝑎21 𝑎22 𝑎23𝑎31 𝑎32 𝑎33

• The space spanned by columns of a matrix is called “Column Space”, and denoted by Col(A).

• The space spanned by rows of a matrix is called “Row Space”, and denoted by Row(A).

𝒓1𝒓2𝒓3

𝒄1 𝒄2 𝒄3

Column and row space of a matrix

• If matrix 𝐴 is 𝑚 × 𝑛, its columns are 𝑚-dimensional (ℝ𝑚).

• The column space of 𝐴 is a subspace of ℝ𝑚.

• If matrix 𝐴 is 𝑚 × 𝑛, its rows are 𝑛-dimensional (ℝ𝑛).

• The row space of 𝐴 is a subspace of ℝ𝑛.

• Example:

• 𝐴 =1 20 1−1 0

.

• 𝐴 has two columns 10−1

and 210

. These two columns span a plane in ℝ3

• 𝐴 has three rows 1 2 , 0 1 , and −1 0 . These rows are linearly dependent (because they are 3 2-d rows), but they span the ℝ2 space.

Column and row space of a matrix

• The equation 𝐴𝒙 = 𝒃 could be re-written as combination of columns of 𝐴:

𝐴𝒙 = 𝒃 ⇒ 𝑥1𝒄1 + 𝑥2𝒄2 +⋯+ 𝑥𝑛𝒄𝑛 = 𝒃

• Or as dot product of rows of matrix 𝐴:

𝒓1 ∙ 𝒙 = 𝑏1𝒓2 ∙ 𝒙 = 𝑏2

⋮𝒓𝑚 ∙ 𝒙 = 𝑏𝑚

Column space of a matrix

• The equation 𝐴𝒙 = 𝒃 is solvable if and only if 𝒃 is in the column space of 𝐴.

• In other words, 𝒃 should be in the span of columns of 𝐴, in order to 𝐴𝒙 = 𝒃 have solution.

• Example:

• 𝐴 =1 20 1−1 0

, 𝒃1 =41−2

, 𝒃2 =40−2

• 𝐴𝒙 = 𝒃1 has solution because 𝒃1 is in the column space of 𝐴.(𝒃1 = 2𝒄1 + 𝒄2)

• 𝐴𝒙 = 𝒃2 has NO solution because 𝒃2 is NOT in the column space of 𝐴.

Column space of a matrix

• Example:

•2 55 10 3

𝑥1𝑥2

=−408

• 𝒄1 =250

and 𝒄2 =513

are linearly

independent and span a plane in ℝ3.

• But 𝒃 =−408

is NOT in that plane.

• So, the equation has NO solution.

Column space of a matrix

• Example:

•2 55 10 3

𝑥1𝑥2

=−7−6−3

• Now 𝒃′ =−7−6−3

is in that plane.

• So, the equation has a solution.

• 𝒃′ = −𝒄1 − 𝒄2, so 𝒙 =−1−1

𝒃′

Null space of a matrix

• In equation 𝐴𝒙 = 𝒃, if 𝒃 = 𝟎 then 𝐴𝒙 = 𝟎 is called a homogenous equation.

• This equation always has a trivial solution which is 𝒙 = 𝟎.• We are interested in non-trivial solutions for homogenous equations.

𝐴𝒙 = 𝟎 ⇒ 𝑥1𝒄1 + 𝑥2𝒄2 +⋯+ 𝑥𝑛𝒄𝑛 = 𝟎

• The above equation has non-trivial solutions if and only if the 𝒄1, 𝒄2, …, 𝒄𝑛are linearly dependent.

• The space spanned by the solution set of 𝐴𝒙 = 𝟎 is called the “Null space” of 𝐴, and denoted as Nul(A).

Trivial vs. Non-trivial solutions

• 𝒂 is a non-zero vector and it can span the line along its direction.

• In other words, any point along its direction can be reached by 𝑐𝒂, where 𝑐 is a scalar.

• 𝑐𝒂 can be zero only when 𝑐 = 0.

• So, there is NO non-trivial

solution for 𝑐𝒂 = 0.𝒂

𝑜𝑟𝑖𝑔𝑖𝑛

𝑐𝒂

Trivial vs. Non-trivial solutions

• 𝒂 and 𝒃 are linearly independent vectors.

• So, they span the whole ℝ2 plane.

• In other words, any point in the plane in which they spanned can be reached by 𝑐1𝒂 + 𝑐2𝒃, where 𝑐1 and 𝑐2 are scalars.

• 𝑐1𝒂 + 𝑐2𝒃 can be zero only when 𝑐1 = 0 and 𝑐2 = 0.

• So, there is NO non-trivial

solution for 𝑐1𝒂 + 𝑐2𝒃 = 0.

𝒂

𝑜𝑟𝑖𝑔𝑖𝑛

𝒃

Trivial vs. Non-trivial solutions

• 𝒂, 𝒃, 𝒄 are linearly dependent vectors in ℝ2.

• So, the span is still the whole ℝ2plane.

• There are infinite set of (𝑐1, 𝑐2, 𝑐3) that can satisfy 𝑐1𝒂 + 𝑐2𝒃 + 𝑐3𝒄 =𝟎, besides (0,0,0).

• Example:

• 𝒂 =12, 𝒃 =

21, 𝒄 =

−1−1

• Non-trivial solution for 𝑐1𝒂 + 𝑐2𝒃 + 𝑐3𝒄 = 𝟎

• 𝑘(1,1,−3) for any 𝑘 ∈ ℝ − 0

𝒂

𝑜𝑟𝑖𝑔𝑖𝑛

𝒃𝒄

Null space of a matrix

• So, matrix 𝐴 has a non-trivial null space if and only if its columns are linearly dependent.

• Example:

• 𝐴 =2 55 10 3

, 𝒄1and 𝒄2are linearly independent. So, Null space of 𝐴 has only the zero vector.

• 𝐴 =2 5 75 1 60 3 3

, 𝒄1, 𝒄2, and 𝒄3 are linearly dependent. So, Null space of 𝐴 is

the span of −1−11

.

Null space of a matrix

• Recall that we can re-write 𝐴𝒙 = 𝒃 as following:𝒓1 ∙ 𝒙 = 𝑏1𝒓2 ∙ 𝒙 = 𝑏2

⋮𝒓𝑚 ∙ 𝒙 = 𝑏𝑚

• Now if we substitute 𝒃 = 𝟎:𝒓1 ∙ 𝒙 = 0𝒓2 ∙ 𝒙 = 0

⋮𝒓𝑚 ∙ 𝒙 = 0

Null space of a matrix

• The non-trivial solution for 𝐴𝒙 = 𝟎 will be perpendicular to all rows of 𝐴.

• In other words, the null space of 𝐴 is perpendicular to row space of 𝐴.

• Example:

• 𝐴 =2 5 75 1 60 3 3

, Null space of 𝐴 is the span of −1−11

.

2 5 7−1−11

= 0, 5 1 6−1−11

= 0, 0 3 3−1−11

= 0

Null space of a matrix

• Cont. Example :• In Figure below, you can see all the rows

are in the same plane.

• In right Figure, you can see the row space and null space

from an appropriate perspective that shows they

are perpendicular.

Null space vs. Row space

• Null space and row space, both are subset of ℝ𝑛.

• Null space and row space, are perpendicular to each other.

• Null space and row space, are complement of each other in ℝ𝑛.

• Example:• 𝐴 is 4 × 3 matrix. It’s row space span a plane. What is the dimension of its

null space?

• Null space and row space are bases for ℝ3. Since its row span a plane, then dim(Row(A))=2. Null(A)=3-2=1.

How to compute null space

• Example: Find a spanning set for the null space of the matrix

𝐴 =−312

6−2−4

−125

138

−7−1−4

Solution: The first step is to find the general solution of 𝐴𝑥 = 0 in terms of free variables. Row reduce the augmented matrix 𝐴 0 to reduce echelon form in order to write the basic variables in terms of free variables.

100

−200

010

−120

3−20

000

𝑥1 − 2𝑥2 − 𝑥4 + 3𝑥5 = 0𝑥3 + 2𝑥4 − 2𝑥5 = 0

0 = 0

How to compute null space

• The general solution is 𝑥1 = 2𝑥2 + 𝑥4 − 3𝑥5, 𝑥3 = −2𝑥4 + 2𝑥5 with 𝑥2, 𝑥4 and 𝑥5 free. Next, decompose the vector giving the general solution into a linear combination of vectors where the weights are the free variables.

𝑥1𝑥2𝑥3𝑥4𝑥5

=

2𝑥2 + 𝑥4 − 3𝑥5𝑥2

−2𝑥4 + 2𝑥5𝑥4𝑥5

= 𝑥2

21000

+ 𝑥4

10−210

+ 𝑥5

−30201

= 𝑥2𝑢 + 𝑥4𝑣 + 𝑥5𝑤

Every linear combination of 𝑢, 𝑣 and 𝑤 is an element of null(A). Thus 𝑢, 𝑣, 𝑤 is the spanning set for null(A).

𝒖 𝒗 𝒘