3.2 Vector Space Properties

on .. = I

•

[X, ]13. IV = Ix: x = . XI = -3x~.Xl

.'"2 any real number}

14. W = {w: w = [ ~ ]. b any real number}

15. Il' = (u: u = [ : ]. C +(1 ~ OJ

16. Il' = {x: x =t [ ~ ]. I any real number}

[a] , ,17.1l'={X:x= b . a-+b-=4j

In Exercises 18-21. interpret the subset IV of RJ geometrically by sketching a graph for IV.

18. IV ~ {~ , ~ [ ~ l a > OJ

19. IV ~ I" ~ [:J x, ~ -x, - h,j

3.2 Vector Space Properties or Rtf 167

20. IV ~ {w, w ~ , [ ~ ]. 'aay «al "mbe,}

21. IV ~ lu, " ~ [ : ]. a' + b' +0' ~ I ,"d

c ~ OJ

In Exercises 22-26. give a set-theoretic description ofthe given points as a subset IV of R2.

22. The points on the line X - 2)' = I

23. The points on the ,r-axis

24. The points in the upper half-plane

25. The points on the line)" = 2

26. The points on the parabola y = x::!

In Exercises 27-30. ghe a set·theoretic description ofthe given points as a subset IV of R3 .

27. The points on the plane X .;.. -' - 2;: = 0

28. The points on the line .... ith parametric equationsx = 21. y = -31. and;: = t

29. The points in the y;:-plane

30. The points in the plane ). = 2

VECTOR SPACE PROPERTIES OF Rn

Recallihat R" is the se.t..of aU n-dimensional \ectors with real components:

r~

'"'f- " '>

[

X, ]

R~ = {x: x = '7 .x.

XI.X~••••• X.. real numbers}.

f R~ geo-

If x and .y are elements of W with

[

X, ] [ ", ]

'~:: andY~::'

168 Chapter 3 The Vector Space Nil

A~ -YftMJ A then (see Section 1.5) the \ ector x _'j' is defined by

,. ~1"'M.,(.,J.. ~'0~ [ x, _ y, ]

~6'1 ~~.r.?~;~.4tdf x-y= X2+)2 .

.l'W ta'I1 !( M @It b(~~ x. +).01 YVlMiA.4d I,.. :-~.w., . d fi d be

.. '1 and if a is a real number. then the vector ax IS e ne to

1hi IJU;(,.n~ ,1M.<! [ ax, ]

1<1M.(.{~~WIo~ ax= a~2 .1~1AJ"" ,,/1,.._.. .

. - I ..... ~~IN, ax"

In the context of R~. scalars are always real numbers. In panicular. throughout thl~

chapter. the tenn scalar always means a real number.The following theorem gives die anthmeiic propenies of \'ector addition and scalar

multiplication. Note that the statements in this theorem are already familiar from Section1.6. which discusses the arithmetic properties of matrix operations (a vector in Rn is an(n x I) matrix, and hence the propenies of matrix addition and scalar multiplicationlisted in Section 1.6 are inherited by vectors in R"),

~ As we will see in Chapter 5. any set that satisfies the properties of Theorem 1 iscalled a "trtor space: thus for each positi,-e integer n, R~ is an example of a vector space-

•

Closure properties:

If x. y. and z are vectors in R~ and a and b are scalars. then the following propenie:.

hold, v -oftn '<""1« ~ \Wo~(" ;0~

-ltI£olU \\ 1

~J1~~i/.ut.n wrur W/n M (el) x + y i, io R".~ (e2) oxisinR~.

£~11 nIl, ( Properties ofadditiOIl:

~1MW. tH.;1w (81) x + y ~ y + x.• (a2) X..j.. ()' + z) = (x ~y)...j.. z.

~. ~1l ~1-i", ItJ (a3) RIt contains the zero vector. 8. arid x ~ 8 = x for all x in R",.rf /) '11 . ['1.1 1 ~ J \ (a4) For each "ector x in R". there is a vector -x in R" such that x + (-x) == 8.

If ,e..) lit . J I S Properties ofscalar multiplication:

') ~ t '81 (ml) a(bx) = (ab)x.(m2) a(x + y) = ax + ay.

II vO" I "1 "'>1m // r) (013) (a + b)x = ax + bx.WJtI --'-"\ (mol) Ix = x for all x In R",

~I"..t~ vfif' Subspaces of R"

In this chapter we are interested in subsets. \V, of R" that satisf) all the properties ofTheorem I (with R" replaced by \V throughout). Such a subset W is called a subspou

3.2 Vector Space Properties of RIO 169

OI\IGI W Of- tt IGttEI\-DI \ \E~fIO~ALfPACFJ In addition to Gtassmaon(see Section 1.7). Sir William Hamilton (1805-1865) also envisioned algebras of /I-tuples (which hecalled po/}plets). In 1833. Hamilton gave rules for the addition and multiplication of ordered pairs,(a. b), which became the algebra of complex numbers, z = a + hi. He searched for years for anextension to 3-tuples. He finally discovered, in a flash of inspiration while crossing a bridge, lhattheextension was possible if he used 4-lUples (a. b. c, d) = a +hi +cj +dk. In this algebra of quaternions,however, multiplication is not commutative; for example. ij = k. but ji = -k. Hamilton stopped andcarved the basic formula. i 2 = j2 = k2 = ijk. on the bridge. He considered the quaternions his greatestachievement. even lhough his so-called Hamiltonian principle is considered fundamentallo modemphysics.

TllIL\I~t-\\ 2

Figurf' 3.5\4-. as a subset of Rl

of R". For example, consider the subset W of R) defined by

IV = {so x = [ ~ ]. x, aod x, "aI n"mbe,,}.

Viewed geometrically, 1V is the .1")·-plane (see Fig. 3.5). so it can be represented by R2.Therefore. as can be easily shown. W is a subspace of R 3.

The following lheorem provides a convenient way of delermining when a subset IVof R" is a subspace of R".

A subset IV of R" is a subspace of R" if and only if the following condilions are mel:

(51)" The zero vector. 8. is in U'.

(52) x + y is in W whenever x and " are in tv.

(53) ax is in lV whenever x is in 1V and a is any scalar.

Suppose that lV is a subset of R" Ibat satisfies conditions (sIHs3). To show that 1V isa subspace of R". we must show that the 10 properties of Theorem 1 (with R" replacedby IV throughout) are satisfied. But properties (al). (a2). (ml). (m2). (mJ). and (m4)are satisfied by e\'ery subsel of R- and so hold in 1V. Condition (aJ) is satisfied by 1Vbecause the hypothesis (s I) guarantees thai 8 is in 1V. Similarly. (c 1) and (c2) are givenby the hypotheses (s2) and (s3). respecth'ely. The only remaining condition is (a4). andwe can easily see that -x = (-l)x. Thus if x is in W. then. by (s3). -x is also in W.Therefore. all the conditions of Theorem I are satisfied by 1V. and 1V is a subspace ofR".

For the con\erse. suppose 1V is a subspace of R". The conditions (a3). (e1). and(c2) of Theorem 1 imply that properties (sl). (s2), and (s3) hold in IV. •

The neXI example illustrates the use of Theorem 2 to verify that a subset IV of R"is a subspace of R".

&,.,.

/,1

/1,

'.lM 1"~1"'1

~Ff u, il:l (I ~ Proof

o1JrOlA~1).'tl4

fig properties

•

iIhroughout Ihis

tion and scalar

~from Section

or in RIO is anmUltiplication

i.l. -x)=8.

rTheorem I isIII \eclor space.

properties ofa subspace "TIle usual stalemem orThcorem 2li.l. only conditions (s2) and (.3) bul as~ume. thaI the subset IV is nonemplY.

Thus (51) replaces the assumplion thal IV is nonempty. The twO \ersions are equivalenl (su Exercise 34).

170 Chapter 3 The Vector Space R"

E\. "'PI r 1 Lei IV be lhe subset of RJ defined by

IV == fx; x = [ :~ ]. XI =.\"2 - X~. .\"1 and .\') any real numbers}.

XJ

Verify that IV is a subspace of RJ and give il geometric interpretation of IV.

Solutioll To show that IV is a subspace of RJ , we must check that properties (51)-(s3) of Theorem2 afC satisfied by IV. Clearly the zero vector, 6. salisfies the condition .t1 = X2 - .\'].

Therefore. 8 is in IV. showing that (sl) holds. Now let u and \' be in IV. where

['" ]U = 112

"3[" ]and \. = t'! •

'3

and let a be an arbitrary scalar. Since u and \' are in 1V.

1/1 = 112 - 113 and VI = t'2 - t.'3·

The sum u + v and the scalar product au are given by"

['" + v, 1

u + \' = /12 + Vz

Ilj + U3 J

To see Ihal u + \' is in IV. note that (1) giles

[

au, ]and au = (/l/2 .

all)

"l ..... t'. = (112 - If) + (1'2 - 1'3) = (uz + t'2) - (113 T V3)·

Thus if the components of u and " salisf) the condition XI = X2 - XJ. then so do thecomponents of the sum U T '>. This argument shows that condilion (s2) is mel by IV.

../ Similarl). from (I).

alii = a(lll - IIJ) = alll - allJ. .'so au i!'i in IV. Therefore. nr is a sub~pace of R3.

Geometrically. IV is the plane whose equation is.r - y +.: = 0 (see Fig. 3.6). •

:=y-x

')(, /

.(0. I. I)

,·(\.1.0)

x

Figure 3.6 A portion oft~ plane.r - y +.: = 0

Step 4.

Step 5.

~

'"TheoremkX1 - .l'J.

(I,

3.2 Vector Space Properties of R ff 171

Verifying that Subsets are Subspaces

Example 1 illustrates the typical procedure for verifying thaI a subset IV of R" is asubspace of RI!, In general such a verification proceeds along the following lines:

Verifying that W Is a Subspace of Rn

Step J. An algebraic specification for the subset \V is given, and thisspecification serves as a test for determining whether a vector in R" isor is not in IV.

Step 2. Test the zero vector. 9, of R" to see whether it satisfies the algebraicspecification required to be in W. (This shows that IV is nonempty.)

Step 3. Choose two arbitrary \"e<:tors x and J from IV. Thus x and yare in R",and both vectors satisfy the algebraic specification of \Y.

Test the sum x .... )' to see whether it meets the specification of IV,

For an arbilfary scalar, Q. test the scalar multiple ax to see whether itmeets the specification of W.

The next example illustrates again the use of the procedure described above to verifythat a subset lV of R" is a subspace.

.0 do t:hellel by lV.

E'..... '\I'I' l lei IV be the subset of RJ defined by

IV ~ I>' X~ [ :: ]. x, ~ 2.". x, ~ 3x,. x, "'y ""I numbe,) .

Verify thai IV is a subspace of R J and give a geometric interpretation of IV.

Solution For clarity in this initial example. we explicitly number the five steps used 10 verify thatIV is a subspace.

:;.6). • • I. The algebraic condition for x to be in W is

X1 = 2Xt and XJ = 3XI. H)

t/ In \\'oreis. x is in W if and only if the second component of x is twice the firstcomponent and the third component of x is three times the first.

2. Note that the zero vector, 9, clearly satisfies (4). Therefore, 8 is in W.

3. Next, let u and v be two arbitrary vectors in W:

['" ]U = 1/1

"J"ndV~[::]

172 Chapler 3 The \'ector Space R"

Because U and v are in W. each musl satisfy the algebraic specification of \VThat is.

1/2 = 2uI and u) = 3// 1

tI:2 = 2vI and v) = 3vl.

(Sal

(5b]

4. Next, check whether the sum, U + v, is in W. (That is, does the veclor u + \satisfy Eq. (4)1) Now. the sum u + v is given by

[u,+u,]

u+v= 112+ti2

1I:1+ V)

By (5a) and (5b). we have

U2 + Vl = 2(ul + vd and (U) + v) = 3(//1 + Vt).

Thus u + v is in W whenever u and v are both in 1V (see Eq. (4».

5. Similarly. for any scalar a. Ibe scalar multiple au is given by

[

au, ]au = aU2 .

au,

Using (5a) gives aUl = a(2111) = 2(a/ll) and aUJ = 0(3/11) = 3(OUl). Therefore. au is in W whene\'er u is in W (see Eq. (4)).

:u Thus. by Theorem 2. \V is a subspace of RJ . G....:.?me.!!!cally. W is a line-lhrough theorigin with parametric equalions

x = XI

y=2x1

:.= 3.1'1.

The graph of the line is given in Fig. 3.7. •

I j

Exercise 29 shows that any line in three-space through the origin is a subspace ofRJ • and Example 3 of Section 3.3 shows Ibat in three-space any plane through the originis a subspace. Also note thai for each positive integer n. R" is a subspace of itself and{OJ is a subspace of R". We conclude this section with examples of subsels that are nOI

subspaces.

E,-\\lPL[ 3 Let \V be the subset of RJ defined by

IV ~ (x," ~ [ ~: l x, ,"dx, any real numbe,,}.

Show that \V is not a subspace of R).

E-

tication of w.

(5a)

(5h)

e vector u + v

3.2 Vector Space Properties or R" 173

"

, (I. 2. 3),,,,,,y ,

r :~, .,---------. (I. 2. 0)

x

Figun 3.7 A geometric representation of the subspace IV(see Example 2)

Solution To show that IV is not a subspace of R', we need only verify that at least one of theproperties (~IHs3) of Theorem 2 fails. NOle Ihal geometrically IV can be interpretedas the plane:; = I, which does not contain the origin. In other words, the zero vector.8, is not in IV. Because condition (sl) of Theorem 2 is not mel. IV is not a subspace ofR', Although it is nOI necessary to do so. in this example we can also sho,"" that bothconditions (s2) and (s3) of Theorem 2 fail. To see this. let x and y be in IV, where

iiJUd. There-[ x, ] ond'~[~,:l

~ghthe

x= "7Then x + ~' is given by

In particular, x - ~' is nOI in W _because the third component of x +Ydoes not have thevalue 1. Similarly.

•I~ubspace ofIh the origin,f 1l~lf andthat are not

[

X, - ,., ]

x+~'= X1~Y1 '

[

ax, ]ax = a;1 .

So if a i= I. then ax is not in IV.

E\A\\PLE ,I Let IV be the subset of R2 defined by

\V = Ix: x = [ :: ]. Xl and Xl any integers}.

Demonstrate that IV i<; not a subspace of R2•

•

174 Chaprer 3 The Veclor Space R"

Solution In this case 8 is in IV. and it is easy to see that if x and }' are in IV. then so is x + y, Ifwe set

=-and a = J/2. then x is in IV but ax is not. Therefore. condition (s3) of Theorem 2 is nOi--met by IV. and hence IV is not a subspace of R2 , •

x~[:]

r \A \\[JLI::: S Let IV be the subspace of R2 defined by

[X, ]

IV = {x; x = X2 ' where either XI = 0 or X2 = OJ.

= "=il:

=11:

~&'=I:I:

=J:I:

= x!..e Ji ~

"'",

tct ". beShow that IV is not a subspace of R2.

Solution Let x and y be defined by

x=[~] ,nd J~[~l G,,"e.~

1lell" tetThen x and)' are in IV, Bur

X+ J'=[:]

~

EXERCISES

In Exercises 1--8. IV is a subset of R~ consisting of"ec·lars of lhe fonn

LI

~"=L

~~.• =

:3.• =

p,,,,,

EUT:'1E.ter.:!<.e ~ I. f

Je-.;.."'n;Oor

:2.. =

Gne3~

:1. let. aOCd.

a'

X~[::].x,

In each case. detennine whethc,r IV is a subspace of R),If IV is a subspace. then give a geomclric descriptionof IV.

9. IV = Ix: X] = lxl - X2)

10. IV = Ix: X2 =X] +Xl}

11. IV = Ix: XIX! = x31

7. IV = {x: xi +X2 = I}

8. IV = Ix: XjXl = OJ

In Exercises 9-17. IV is a subset of R3 consisting of\cctors of the fonn

is not in IV. so IV is nor a subspace of R2. Note that 8 is in Uf. and for any "ector x inIV and any scalar a. ax is again in IV. Geometrically, \V is the sel of points in the planethai lie either on the x-axis or on lhe J.axis. Eilher of lhese axes alone is a subspace ofR2

• bul. as lhis example demonstrates. their union is not a subspace. •

~"31

.~ [ x, ]X •x,

In each case detennine whether W is a subspace of Rl .

If '" is a subspace. Ihen give a geometric descriplionof IV.

I. IV = Ix: .(1 = 2x~J

2. IV = Ix: Xl - X2 = 2JJ. IV = {lo:: Xl = X2 or .tj = -X2}

4. IV = {x: Xl and X2 are rational numbers}

5. IV = {x: Xl = OJ

6. IV = Ix: lxll + IXII = O}

~ '\: + y. If

em2isnot

•

f-IOCX ;n

the planep3Ce of

•

~lOg of

"eof RJ.~nption

12. \V = {x: Xl = 2x3113. IV = {x: .If =.11 +.121

14. \V = {x: .12 = 0115. \V = {x: Xl = 2Xl, .12 = -.Ill

16, IV = Ix: Xl = .12 = 2Xli

17. IV = {x: .12 =Xl =OJ18. Let a be a fixed vector in RJ , and define IV to be the

subset of Rl given by

IV = {x: aTx =OJ,

Prove that IV is a subspace of R3.

19, Let IV be the subspace defined in Exercise 18. where

a~ [: lGive a geometric description for \V.

20. Let \V be the subspace defined in Exercise 18. where

a=UlGive a geometric description of IV.

21. Let a and b be fixed VKtOrs in R3, and let IV be the

subset of Rl defined b)

\V = {x: aTx = 0 and bTx =O}.

Prove thaI W is a subspace of RJ•

In Exercises 22-25. W is the subspace of Rl defined inExercise 21. For each choice of a and b. give a geo~tric description of IV.

11a~[-J b~[-:]

n.a~Ul b~[i]

~.a~[:J b~m

15. a = Ul b ~ [ -n

3.2 Yectol' Space Properties of R" 175

26. In R4, let x = [I. -3. 2. I)T, Y=[2, I. 3. 2jT. and

z = [-3. 2. -1,4jT. Seta = 2 andb = -3. illustrate that the ten properties ofTheorero I are satisfiedby x, y. z, a. and b.

27. In R2. suppose that scalar multiplication were defined by

ax ~ a [ X, ] ~ [ z"x, ].12 2aX2

for e\'ery scalar a. Illustrate with specific examplesthose properties of Theorem I that are not satisfied.

28. Let

IV = {x x = [ :: l .12 ~ 01

In the statement of Theorem I, replace each occurrence of R~ with W. Illustrate with specific examples each of the ten properties of Theorem 1 that arenot satisfied.

29. In R]. a line through the origin is the set of all pointsin R] whose coordinates satisfy XI = at, .12 = bt,and .13 = ct.whe~ r is a variable and u. b, and c arenot aU zero. Show that a line through the origin is asubspace of RJ •

30. [f U and V are subsets of R". then the sel U + V isdefined by

U+V={x:x=u-,', uinU and ,'in VI .Prove that if U and V are sUbspaces of R". thenU + V is a subspace of R".

31. leI U and V be sUbspaces of R". Pro'e thai theintersection. U n V. is also a subspace of R~.

32. Let U and V be the sUbspaces of Rl defined by

U = Ix: aTx = OJ and V = Ix: bT" = 01.whe~

'~[i] ~d b~[JDemonstrate that the union. U U V, is not a subspace ofR3 (sec Exe,rcise 18).

33. Let U and V be subspaces of Rn .

a) Show that the union, U U V. satisfies properties(sl) and (s3) of Theorem 2.

b) If neither U nor V is a subset of the olher, showthat U U V does not satisfy condition (s2) of


Theorem 2. [Hint: Choose vectors u and y suchthat u is in U but not in V and y is in V but notin U. Assume that u + \' is in either U or V andreach a contradiction.]

34. Let W be a nonempty subsel of R" thai satisfies conditions (52) and (s3) of Theorem 2. Prove that 8 is inWand conclude that W is a subspace of R". (Thusproperty (s I) of Theorem 2 can be replaced with theassumption thaI W is nonempIY.)

EXAMPLES OF SUBSPACES

In this section we introduce several important and particularly useful examples of subspaces of R".

The Span of a Suhset

To begin, recall that if VI ...• , \', are vectors in R", then a vector y in R" is a linearcombination of VI, ...• y" provided that there exist scalars al •... ,0, such that

y = alYI + ... +a,v,.

The next theorem shows that the set of all linear combinations of VI •...• V, is a

subspace of R".

Til rORE\\ 1 If VI •••.• V, are \'ectors in R". then the set \V consisting of all linear combinations ofVI ••••• V, is a subspace ofR".

Proof To show that \V is a subspace of R", we must verify that the three conditions ofTheorem 2 are satisfied. Now 8 is in W because

8 =OVI + ···+Ov,.

Jext, suppose that)' and 1 are in \V. Then lhere exist scalmal .... , a" bl ..... b, suchthat

~'= a!vl + ... +a,v,

and

l=bIVI+···+b,v,.

Thus.

Y+ 1 = (al + bdvi + ... + (a, + b,)v,.

so Y+ 1 is a linear combination of VI •...• v,: that is,)' + 1 is in W. Also, for any scalarc.

cy = (cailvi +... + (ca,)v,.

In particular, C)' is in W. It follows from Theorem 2 that W is a subspace of R". •

If S = {VI t •••• v, I is a subset of R" , then the subspace W consisting of all linearcombinations of Vl •... t V, is called the subspace spanned by S and will be denoted by

SpeS) or Sp{VI .... ,V'}.

oil satisfies conrmethat8isine of R~. (Thus- aced with the

t'Jples of sub-

is a linearthat

.. \', is a

'inations of

lflditions of

. b, such

~ any scalar

R~. •

alllinear~noted by,

,

,

Figure 3.8

SpIv}

av'

3.3 Examples of Subspaces 177

For a single vector \' in RI!. Spiv} is the subspace

Spiv} = {a\': a is any real number}.

If v is a nonzero vector in R2 or R3• then Sp{\'} can be interpreted as the line determinedby v (see Fig. 3.8). As a specific example, consider

,~[JThen

Spiv} ~ {, [ ~ l' i, ,ny ,,,1 numbe,}.

Thus Spiv} is the line with parametric equations

x = r

)' = 2r

z = 3r.

Equivalently, SpIv} is the line that passes through the origin and through the point withcoordinates J. 2, and 3 (see Fig. 3.9).

: l

" (I. 2. 3),,,,,,V :_

r : J,-~~------ .. (I. 2. 0)

x

Hgu" 39 SpI[i ]IIf u and \' are noncollinear geometric vectors, then

Sp{u. v} = {au + bv: a. b any real numbers}

is the plane containing u and \' (see Fig. 3.10). The following example illustrates thiscase with a subspace of R3.

"

178 Chapter 3 The Vector Space Rn

,.4 ,, ,, ,, ,

au/ (lu+bv~ ,-,,,,,

",,,Ie' "o • • -------,

\' bv

Figure 3.10 Sp[u. v)

EAM"IPLI::: 1 Let u and v be the three-dimensional vectors

U~[n and V~[:JDetennine IV = Sp{u. v) and give a geometric interpretation of W.

Solutio" Let y be an arbitrary vector in R3, where

y~ [~JThen y is in IV if and only if there exist scalars Xl and X2 such that

y = Xlll +X2V,

That is, y is in W if and only if there ex.ist scalars Xl and Xl such that

Yl = 2Xl

)'2 = Xl + X2

)'3 = 2X2.

The augmented matrix for linear system (2) is

[20"']I I Y2 ,

a 2 }'3

PtlYflCAL I\tPI\Eft~rrATIO~fOf'VECTOrv Tho v"'"' space wo,k ofGrassmann and Hamilton was distilled and popularized for the case of R3 by a Yale University physicist,Josiah Willard Gibbs (1839-1903). Gibbs produced a pamphlet, "Elements of Vector Analysis," mainlyfor the use of his students. In it, and subsequent articles, Gibbs simplified and improved Hamilton's workin multiple algebras with regard to three-dimensional space. This led to the familiar geometricalrepresentation of vector algebra in terms of operations on directed line segmenlS,

(I)

(2)

,

A

~

13)

(4)

(5)

(1 )

(ll

,,.

•y

"x

Figure 3.11A ponion of the planex~2)'+;:=O

O£:t-!" 1110' I

£ lill ()I~1 \I ~Proof


and this matrix is row equivalent to the matrix

[

1 0 (I/2)Y,]o 1 )'2 - (1/2»)',

o 0 0/2))'3 + (l/2)YI - J2

in echelon form. Therefore, linear system (2) is consistent if and only if (1/2).'1'1 - }'2 +(1/2))'3 = 0, or equivalently, if and only if

)'1 - 2n + )'3 = O.

Thus It! is the subspace given by

[

y, ]\V = Iy = )'2 :)'1 - 2)'2 + Y3 = OJ.

nIt also follows from Eq. (5) that geomelrically \tI is the plane in three-space wilh equation:c - 2)'+:: = o(see Fig. 3.11). •

The Null Space of a Matrix

We no\\, introduce two subspaces that ha\e panicular relevance 10 the linear system ofequations Ax = b. where A is an (m x n) matrix. The first of these subspaces is calledthe null space of A (or the kernel of A) and consists of all solutions of Ax = 8.

Let A be an (m x n) matrix. The null spau of A (denoted X(A» is the set of\'ectors in R" defined by

."(A) = {x: Ax = 8. x in R"}.

In words. the null space consists of allth~x§\lCh that Ax is the zero vector.The next theorem shows that the null space of an (m x n) matrix A is a subspace of R".

If A is an (m x n) matrix. then.'~(A) is a subspace of R"":'--TOSilOWlliat Ii. (Alis a subspace of R". we must verify that the three conditions ofTheorem 2 hold. Let 8 be the zero vector in R". Then

and so 8 is in N(A). (Nole: In Eq. (6). the left 8 is in R" but the right 8 is in R"'.) Nowlet u and" be vectors in N(A). Then u and v are in R" and

'ofysicist,

mainly's work

A8 =8.

Au=8 and Av=8.

(6)

(1)

To see that u + v is in !'I(A). we must test u + v against the algebraic specification ofN(A); that is, we must show that A(u +,"') -= 8. But it follows from Eq. (7) that

A(u +v) -= Au + A\' -= 8 + 8 -= 8,


and therefore u + v is in N(A). Similarly, for any scalar a. it follows from Eq. (7) that

A(au) = aAu = a8 = 9.

Therefore, au is in N(A). By Theorem 2. N(A) is a subspace of Rn • •

1

1

2A~[i

EX,.\,\1PLf 2 Describe N(A). where A is the (3 x 4) matrix

3 1]5 4 .

4 -1

Solution N(A) is determined by solving the homogeneous system

Ax = 9. (8)

This is accomplished by reducing the augmented matrix [A 18]10 echelon form. It iseasy to verify that LA 181 is row equivalent to

uo1

o

2 3

1 -2

o 0 nSolving the corresponding reduced system yields

Xl = -2x) - 3x-t

X2 = - X) + 2X4

as the solution to Eq. (8). Thus a veclor x in R4•

x ~ [ ~~1is in N(A) if and only if x can be wriuen in the form

[

-2X;-3X,] [_2] [-3]-x) +2X4 -I 2

x= =x, +X4 ..\") . 1 0

401

where.\"3 and.\"4 are arbitrJry; thai is.

N(A) ~ Ix x ~ X; [

-2

-1

1

o]+x'[-~l .\") and.\"4 any real numbers}. ..

f7Hhar

•


As the next example demonstrates. the fact that ...\"(A) is a subspace can be used toshow that in three-space e\'el) plane through the origin is a subspace.

L\.\\\PLE] Verify that any plane through the origin in R3 is a subspace of R3.

Solution The equation of a plane in three-space through the origin is tax + by +c.;; = 0,- (9,

II i~

•

OH1'ITIO' 2.

where a. b. and c are specified constants not all of which are zero. Now. Eq. (9) can be~ritten as

Ax =8.

where A is a (I x 3) matrix and x is in R3:

A ~ I. b cJ ood x = [ JThus x is on the plane defined by Eq. (9) if and only if x is in N(A). Since N(A)is a subspace of R3 by Theorem 4. any plane through the origin IS a subspace~R3. •

The Range of a Matrix

Another important subspace associated with an (m x n) matrix A is the rallge of A.defined as follows.

Let A be an (m x 11) matrix. The ra1lge of A (denoted 'R(A)] is the set of \'ectorsin Rill defined by

'R(A) = b:)' = Ax for some x in R~J.

In words. the range of A consists of the set of all vectors y in R'" such that the linearsystem

Ax = Y

is consistent. As another way to view R(A). suppose that A is an (m x n) matrix. Wecan regard multiplication by A as defining a function from W to Rm. In this sense. as xvaries through R~. the set of all vectors

y = Ax

produced in R'" constitutes the "range" of the function.

182 Chapter 3 The Vector Space R n

We saw in Section 1.5 (see Theorem 5) that if the (m x II) matrix A has coluAt, A2, .... A" and if

X~[Jthen the matrix equation

Ax =y

is equivalent to the vector equalion

XtAt + X2A2 + ... + xnA" = y.

Therefore, it follows that

R(A) = Sp{A t .A2..... All}'

By Theorem 3, Sp{AJ, A2, ... ,All} is a subspace of R m . (This subspace is also calledthe columll space of matrix A.) Consequently, R(A) is a subspace of Rm , and we ha\~

proved the following theorem.

+-J Til I OIU- \\ ~ If A is an (III x II) matrix and if R(A) is the range of A. then R(A) is a subspaceof~. •

The next example illustrates a way to give an algebraic specification for R(A).

I \, ,\PI r I Describe the range of A. where A is the (3 x 4) matrix

A~[~

Solurion Let b be an arbitrary vector in R 3.

1

1

2

3 1]5 4 .

4 -1

b~ [:JThen b is in R.(A) if and only if the system of equations

Ax = b

is consistent. The augmented matrix for the system is

[AlbJ~[~1

1

2

3 1

5 4

4 -1

b, ]:: '

4 has columns

~ also calledj.nd we ha\e

• ~ubspace-t 'CIAI.

~~'LJl"'Ao

f/vJ ~tA '"•.Jo~wl1

') .""--

3.3 Examples of Subsp8ces 183

which is equivalent to

[~0 2 3 b,-b, ]1 1 -2 2b1 -b2 .

0 0 0 -3bl +b2+ b3

It follows that Ax = b has a solution [or equivalently, b is in 'R(A)} if and only if-3b l + b~ + bJ =O. or b3 =3b 1 - b2, where hi and b2 are arbitrary. Thus

7«A) = Ib, b = [ :: ]3b 1 - b1,

= b, [ ~ ] + b, [ -:J b, ,nd b, aoy ",1 n"m~,,). -

The Row Space of a Matrix

If A is an (m x n) matrix with columns AI. A2..... A~. then we have already definedthe column space of A 10 be

Sp{AI.A2..... A~} .

In a similar fashion. the rows of A can be regarded as vectors ai, a2' ... , a.. in R~, andthe row space of A is defined to be

Sp{al.a2..... a..}.

For example. if

A=[:~~lthen the row space of A is Sp{a\. all. where

al = II 2 3J and 82 =[1 0 1].

The following theorem shows that row-equivalent m8lrices have the same row space.

TUEOI\(\\ b Let A be an (m x n) matrix. and suppose thai A is row equivalent to the (m x 11)

matrix B. Then A and B have the same row space. -

The proof of Theorem 6 is given at the end of this section. To illustrate Theorem 6,let A be the (3 x 3) matrix

[

1 -1

A = ~ -: :]

184 Chapter 3 The Vector Space R-

By perfonning the elementary row operations R2 - 2R1, R3- RI. R1+R2•and R3-2R:we obtain the matrix

[

1 0 J]B ~ 0 1 2

o 0 0

By Theorem 6, matrices A and 8 have the same row space. Clearly the zero row of Econtributes nothing as an element of the spanning SCI, so the row space of B is Sp{b l, b:where

bl ::: [1 0 3] and b2::: [0 I 2].

If the rows of A are denoted by 3]. 82. and 83. then

Sprat. 82. 83}::: Sp{b]. b2J.

More generally. given a subset S ::: {"j ....• "",1of Rn , Theorem 6 allows us to obtai.::.a "nicer" subset T ::: {WI •••.. Wk} of R" such that Sp(S)::: Sp(T). The next exampleillustrates mis.

\ \ ,\PI Ie") Let S ::: {VI, \'2, \'3, V4} be a subset of R3. where

"I ~ [ : l v, = [ : l v, ~ [-:l ~d v, = [ _; lShow that there exists a set T = {WI. W2) consisting of two veclQrs in R3 such thatSpeS) = Sp(T).

Solutio" Let A be the (3 x 4) matrix

A = [VI, v2. vJ, V41:

that is.

A=[~The matri:'l: AT is the (4 x 3) matrix

2

J

5

1

4

-5~ ].

-I

[I 2 1]

T _ 2 3 5A _ ,I 4-5

2 5-1

and the row vectors of AT are precisely the vectors "f, ,'r t vf, and vI. It is straightforward to see that AT reduces to Ihe matri:'l:

[1 0 7]

T _ 0 1-3B - 0 0 0 .

000

and R3-2R1.

3.3 Examples or Subspaces 185

So. by Theorem 6. AT and BT have the same row space. Thus A and B have the samecolumn space, where

In particular. 5p(S) = Sp(T), where T = {WI. W2},

uro row of Bis 5p{b]. b:ll.

B~[~o1

~3

ooo n

w, ~ U] aod W, ~ [ -;] ~

~ us to obtainpt'u example

l:].-I

, such that

~ght(or.

Proof of Theorem 6 (Optional)

Assume that A and Bare row·equivalenl (m x n) matrices. Then there is a sequence ofmatrices

A = AI.A1, ... ,Ak_I,A,t = B

such that for 2 :5 j ~ k, A j is obtained by performing a single elementary row operalionon Aj _ l . It suffices, then. 10 show that Aj_1 and A j have the same row space for eachj. 2 :5 j :5 k. This means thai it is sufficient to consider only the case in which B isoblained from A by a single elementary row operation.

Let A have rows 81 .... , a",; that is, A is the (m x n) mauix

a,

a,A~

a•

..where each ai is a (l x n) row vector;

a; = [ail ai2 ... a,~].

Clearly the order of the rows is immaterial: that is, if B is obtained by interchanging thejth and kth rows of A,

a,

a.B~ ,

aJ

am

186 Chapter 3 The Yeetor Space R-

then A and B ha\ e the same row space because

Sp{al' .... aJ ..... 8t- .... a",} = Sp{al' .... at- .... aj' .... a",}.

Next. suppose that B is oblained by performing the row operalion Rt +CRj on A; thu~

a,

aj

B=at + C8j

am

If lhe vector x is in Ihe row space of A. then there exist scalars bl ..... b". such that

10x = b18, + ... .,-bJaJ

+ ... -bt 8j; _ ... +h"'a....

The vector equation (10) can be rewritten as

x = biB. _ ... + (bJ - cbt)aJ _ ••. +bj;(llt - caJ ) ... ••• + b",a",.

and hence x is in the row space of B. Conversel)'. if the \'eetor)' is in the row space ofB, then theTe exist scalars d., ' ... d", such that

.Y = dl81 + ... + djaJ + ... -dt(at + eaj) +, .. + dill&"'. (1~

But Eq. (12) can be rearranged as

y = dial + ···+(dJ +cdt)aj +···-dt84 + ... +d",8", , (U,

so Yis in the row space of A. Therefore, A and B have the same row space.The remaining case is the one in which B is obtained from A by multiplying the jth

row by the nonzero scalar c. This case is left as Exercise 54 at the end of this section.

')

EXERCISES

C141

(IS)

8. S = {a. b. d}

10. S = lao b. e}

v~ [J w~ [ -:J x~ [ Jy~ [=n ,~U]

7. S = {b. e}

9, S= {b,c,d}

11, S = {a. c, e}

Exercises 12-19 refer (0 the \CClors in Eq. (15).

a ~ [ _: l b ~ [ -: l ,~[ -: ld~[~l·~[a

Exercises 1-11 refer to lhe \'eclOl"S in Eq. (14).

In Exercises 1-11. either shoy. that SpeS) = R~ or gi,-ean algebraic specification for SpeS). If Sp(S) #= R~.

then give a geometric description of SpeS).

I. s ~ {al 2. S ~ {bl 3. s ~ I.)4. S = {a. b} 5. S = lao dl 6. S = lao cl

3.3 Examples or Subspaces 187

In Exercises 26-37. give an algebraic specification forthe null space and the range of the given matrix A.

[ 1-2 ] [-I 3]26. A = 27. A =-3 6 2 -6

38. Let A be the matrix given in Exercise 26.

a) For each "ector b that follo....s. determine.... hether b is in 'R(A).,

b) Ifb is in "R.(A). then exhibit a vector x: in R 2

such that Ax: = b.

c) If b is in R(A). then write b as a linearcombination of the columns of A.

JU)b: [ : ]

])b=[_:] ~b~[-:]

~b~[~]

~b=[:]

29.A=[~~]

[1 2 ']31. A =3 6 4

33A=[~~]

[I 2 3]

35.A=131

2 2 10

:]

:]

')b~[_~]

[

I 2

37. A = 2 5

I 3

36.A=[ -;

[

1 -I30.A= 2-1

28.A=[: ~]

32.A~ [i ;]

[

I -, ']34. A = 2 -3 5

1 0 7

o -I ]1 2

2 2

23. Determine which of the "ectOT'llisted in Eq. (1-1) isin the null space of the matrix

A~[: :]

A~U :l25. Determine which of the vectors listed in Eq. (15) is

in the null space of the matrix

In Exercises 12-19. either show that Sp(S) = R) or givean algebraic specification for SpeS). If Spes) "1= R).then gi\'e a geometric description of SpeS).

12. s= (vI 13. S= (wI

14.S=(v.wl IS.S=(v.xl

16.S=I\'.w.xl 17.S=(w,x.zl

IS.S=(v.w.z} 19.5=(w,x.YI

20. Let S be the set given in Exercise 14. For each vector given below. determine whether the vector is inSpeS). Express those vectONi lhilt are in SpeS) as alinear combination of "and w.

.) [ : ] b) [ J ,) [~ ]d)[:] ')[-~] 0[:]

[1-I 0]

A = 2 -I I

3 -5 -2

21. Repeat Exercise 20 for the set 5 gh en in Exercise 15.

H. Determine which oflhe \ectors listed in Eq. (14) isin the null space of the matrix

A = [-2 I I).

24. Determine which of the vectors listed in Eq. (15) isin the null space of the matrix

( II)

space of

(lOj

(13

till

uch that

15).

J

,.mgthejthseCtion.

l .. on A; thus.

( 15)


Exhibit a (3 x 3) matrix A such that W = R(A).Conclude that W is a subspace of R'.

43. Lei

41. Repeat Exercise 40 for the matrix A given in Exercise 35.

42. Let

i)b ~ U] ii)b~ [Jiii)b~[;] i,) b~ [ ; ]

,)b~ U] ,i)b~ [n

W={x= [:: ]:3XI-4X2+2X3=O}.

X3

Exhibit a (I x 3) matrix A such that IV = N(A).Conclude that W is a subspace of R 3.

44. Let S be the set of vectors given in Exercise 16.Exhibit a matrix A such that Sp(S) = R(A) .

SO. Identify the range and the null space for each of thefollowing.

a) The (n x II) identity matrix

b) The (II x II) zero matrix

c) Any (II x n) nonsingular matnx A

51. Let A and B be (n x 11) matrices. Verify thatN(A) nN(B) S; N(A + B).

52. Let A be an (m x r) matrix and B an (r x II) matrix.

a) Show that N(B) S; N(AB).

b) Show that R(AB) S; R(A).

53. Let W be a subspace of Rn . and let A be an (m x II)matrix. Let V be the subset of R ffl defined by

V = {y: y = Axforsomexin WI.

Prove that V is a subspace of Rffl•

54. Let A be an (III x II) matrix, and let B be obtained bymultiplying the kth row of A by the nonzero scalarc.Prove that A and B have the same row space.

In Exercises 46--49. use the technique illustrated in Example 5 to find a set T = (WI. W2) consisting of twovectors such that SpeS) = Sp(T).

46.s~!UHn[nl

47 s ~ \ [ -;H_~H-~ ]I48. h ![~H~~H;H-: ]I49. S ~ l[ ~ H:H:J[-: ]I

45. Let 5 be the set of vectors given in Exercise 17.Exhibit a matrix A such that Sp(S) = R(A).

2Xl - 3X2 + X, ]

-Xl + 4X2 - 2X3 : Xl, X2, X, real}.

2XI+ X2+ 4x,

\V~{y~[

39. Repeat Exercise 38 for the matrix A given in Exercise 27.

40. Let A be the matrix given in Exercise 34.

a) For each vector b that follows. detenninewhether b is in R(A).

b) If b is in R(A), then exhibit a vector x in R 3

such that Ax = b.c) If b is in R(A), then write b as a linear

combination of the columns of A.

• BASES FOR SUBSPACES

Two of the most fundamental concepts of geometry are those of dimension and theuse of coordinates to locate a point in space. In this section and the nex.t, we extendthese notions to an arbitrary subspace of Rn by introducing the idea of a basis for a

3.4 Bases for Subspaces 189

Clearly the vector \' in (1) can be expressed uniquely as a linear combination of e, and

e2:

can be interpreted geometrically as the point with coordinates a and b. Recall that in R2

the vectors el and e2 arc defined by

e,~[~] .nd e'~[~l

(I)v ~ [ : l

v = ael + b~. (2)

As we will see later. the set Ie" e2} is an example of a basis for R2 (indeed. it is calledthe lIatural basis for R2). In Eq. (2), the vector \' is detennined by the coefficientsa and b (see Fig. 3.12). Thus the geometric concept of characterizing a point by itscoordinates can be interpreted algebraically as detennining a vector by itS coefficientSwhen the ,ector is expressed as a linear combination of "basis" vectors. (In facl. thecoefficients obtained are often referred to as the coordinates of the vector. This ideawill be developed fuoher in Chapter 5.) We rum now to the task of making these ideasprecise in the context of an arbitrary subspace \V of R" .

subspace. The first pan of this section is devoted to developing the definition of a basis.and in the latter part of the section. we present techniques for obtaining bases for thesubspaces introduced in Section 3.3. We will consider the concept of dimension inSection 3.5.

An example from R2will serve to illustrate the transition from geometry 10 algebra.We have already seen that each vector \' in R2

•

-h of tl1c

~Urclse ,-":t I

rated in Exling of [\\0

(J),

])

main\[.

,.

yi(b, L ++~a. b)

~~~~·a ~

.m XII)

tty

b)'alare.

•ae,

Figure 3.12 \' = ael ..... be~

lnd theIC1[Cend~ for a

Spanning Sets

Let \V be a subspace of R". and let S be a subset of \V. The discussion above suggeststhat the first requirement for 5 to be a basis for \V is that each vector in \V be expressibleas a linear combination of the vectors in S. This leads to the following definition.

,


DEFt" n 10' J Let W be a subspace of Rn, and let S = {WI ••.. , wml be a subset of W. We saythat S is a spanning set for \V. or simply that S spans W. if every ..ector w in Wcan be expressed as a linear combination ofveclors in S:

w=a\,,', +··'TQ",W.,.

A restatement of Definicion 3 in the notation of the previous section is that 5 is aspanning set of W provided that SpeS) = W. It is evident that the set S = feJ, e2. e31.consisting of the unit vectors in R J• is a spanning set for R J • Specifically. if v is in RJ.

"=[:lthen v = ael + be2 + eeJ. The next two examples consider other subsets of RJ.

(.\1

f\ \\\NL 1 In R).let S = {UI. U2. u31. where

"1= [ -] ", = [ -n, and ", = [ ~ lDetennine whether S is a spanning set for R3.

SQll/tiol/ We must detennine whether an arbitrary vector ,. in RJ can be expressed as a linearcombination of Ul. U2. and U3. in other words. we must decide whether the veclorequation

X1Ul + X2U~ + X3U3 = V. "where v is the vector in (3), always has a solution. The vector equation (4) is equivalentto the (3 x 3) linear system with the matrix equation

Ax = v. (5)

where A is the (3 x 3) malrix A = [UI' U2. U3]. The augmented matrix for Eq. (5) is

[AI'l~[ -l -2 I :J3 2

I 4

and this matrix is row equivalent to

U0 0 lOa+9b-7C]I 0 4a+4b-3c .

0 I -a -b+c

· U'. We say'torwin W

riSlhatSisa= lel.e•. eJ}.IfvlsinRJ•

(3)

of RJ.


Therefore.

Xl = lOa + 9b - 7c

X2= 4a+4b-3c

XJ = -a - b + c

is the solution of Eq. (4). In particular, Eq. (4) always has a solution, so 5 is a spanningm~RJ. _

E\\\\PLl 2 LetS = {\'I.Vl.VJJ be the subsetofRJ defined by

v, = Ul v, ~ [~l and v, ~ [ ilDoes S span RJ?

Solutioll Let \' be the vector given in Eq. (3). As before, the vector equation

XlVI -l- Xl\'l + xJ\'J = \'

is equivalent to the (3 x 3) system of equations

Ax = \'.

where A = [vI. V2, \'J]' The augmented matrix for Eq. (7) is

(6)

(7)

Fd as a IinearItt the \-eclor

[AI'I~ Uand the matrix [A I\'1is row equivalem to

-I

o-7

2

7

o :l'"

equivaJenl[

I 0 7/2

o 1 3/2

o 0 0

b/2 ]-0 ...... (I/2)b .

-70 +2b-c

(5)

Eq. (5) is

It follows that Eq. (6) has a solution if and only if -70 - 2b - c = O. In panicular, Sdoes not span R3. Indeed.

SpeS) ~ {,.". ~ [ : l wh,.e - 7. + 2b + c = O}.

For example, the vector

w~[:]is in R3 but is not in SpeS); that is. w cannot be expressed as a linear combination of VI.

~,and~. _

192 Chapter 3 The Vector Space RII

The next example illustrates a procedure for constructing a spanning set for the nullspace. N(A), of a matrix A.

E\ \ '\Ill r 3 Let A be the (3 x 4) matrix

A=U1

1

2

) I]54.

4 -I

Exhibit a spanning set for ...'~(A). the null space of A.

Solutio" The firs[ Step toward obtaining a spanning set for N(A) is 10 obtain an algebraicspecification for N(A) by solving the homogeneous system Ax = 9. For the givenmatrix A, this was done in Example 2 of Section 3.3. Specifically,

[

-2X3-)X']-XJ + 2x4

.V(A) = {x: x = XJ . Xl and x~ any real numbers}.

x,

Thus a vector x in .,r(A) is totally determined by me unconstrained parameters XJ andX.t. Separating mose parameters gives a decomposition of x:

[

-2-" -)x, ] [ -2x, ] [ -lx, ] [ -2 ] [ -) ]-XJ.U4 - XJ h 4 -I 2x= = + =X3 +X4 .(8)

x3 X3 0 I 0

X4 0 X.t 0 I

Let Ul and U2 be the vectors

[

-2 ] [ -) ]u, = -l ~d u, ~ ;.

By setting X3 = I and .1'4 = 0 in Eq. (8). we obtain u" so Ul is in X(A). Similarly,U2 can be obtained by setting X3 = 0 and X4 = I. so U2 is in Ar(A). Moreover, it is animmediate consequence of Eq. (8) that each vector x in }leA) is a linear combinationof Ul and U2. Therdore. N(A) = Sp{Uj. U2}: that is. {Ul. til} is a spanning set forN(A). •

The remaining subspaces introduced in Section 3.3 were either defined or characterized by a spanning set. If S = {VI ••••• "r} is a subset of R". for instance, men obviouslyS is a spanning set for SpeS). If A is an (m x n) matrix,

A = [At •...• AIlJ.

et for the null

1m algebraicIII" the given

3,4 Bases for Subspaces 193

then. as we saw in Section 3.3, (AJ, ... , An} is a spanning set for'R.(A), the range of A.Finally, if

A~[:~lwhere a/ is the ;th-row vector of A, then, by definition, {at. ... , am} is a spanning setfor the row space of A.

Minimal Spanning Sets

If lV is a subspace of R", W i= {O}, then spanning sets for W abound. For example. avector v in a spanning set can always be replaced by av, where a is any nonzero scalar.It is easy 10 demonstrate, however, that not all spanning sets are equally desirable. Forexample, define u in R2 by

u~ [ : lThe set S = {el,~, u} is a spanning set for R2. Indeed, for an arbitrary vector" in Rl ,

,= [:l\' = (0 - c)el - (b - c)el -r CU, where C is any real number whatsOever. But the subset{el. ell already spans R1, so the vector u is unnecessary.

Recall that a set {VI •...• v",} of vectors in R" is linearly independent if the vectorequation

-ers Xj and

-3]~ ,',

XI\'I-r"'+X",Y", =9 C9,

I \,.\\lPll ~ LetS={VI,Vl,Vj}bethesubsetofR3,whereISimilarly.er, it is an

'ination:g set for-

~h.aracter

Pb\iously

Solution

has only the trivial solution Xl = ... = x'" = 0; ifEq. (9) has a nontrivial solution. thenthe set is linearly dependent. The set S = lei, el. ul is linearly dependent because

el-r e2- u =9.

Our nexl example illustrates that a linearly dependent set is not an efficient spanning set:Ihat is, fewer vectors will span the same space.

v, = [:l v, ~ [ ; J'nd v, ~ [ :lShow that S is a linearly dependent set. and exhibit a subset T of S such that T containsonly two vectors but Sp(T) = SpeS).

The vector equation

Xl VI +X2V! + X3V, = 8 ( 10)

194 Chapter 3 The Vector Space R~

is equivalent to the (3 x 3) homoge,neous system of equations with augmented matrix

['230]

A= I 3 5 0 .

I I I 0

Matrix A is row equivalent to

B=[~o -I

1 2

o 0 n(11 )

in echelon form. Solving the system with augmented matrix B gives

Xl = X3

X2 = -2X3.

Because Eq. (10) has nontrivial solutions. the set S is linearly dependent. Taking X3 = I.for example. gi\'es x. = I. Xl = -2. Therefore.

\'1 - 2\'2 + v) = 8.

Equation (11) allows us to express \'3 as a linear combination of VI and \'2:

\'3 = -VI + 2\'2'

(!'\OIe lhal we could juSt as easily have soh'ed Eq. (11) for either \', or \'2.) It now followsthaI

Sp{v], \'2) = SP{VI. "2, v31.

To illustrate. let \' be in the subspace SP{VI. vl. V3}:

v = o,v. +al"l +a3\'3.

~1aking the substitution V3 = -\', + 2Vl. yields

\' = aivi +0lV2 +a3(-VI + 2\'2).

This expression simplifies to

v = (al - a3)v. T (al + 2a3}Vl:

in panicular. \' is in SP{VI. \'2). Clearly any linear combination of \'1 and \'2 is in Sp(S)because

b,vI + b2V2 = blv, + b 2\'2 + 0"3.

Thus if T = {VI. \'l}. then Sp(T) = SpeS). •The lesson to be drawn from Example 4 is thai a linearly dependent spanning set

contains redundant infonnation. That is, if 5 = (w, .... , w,} is a linearly dependentspanning set for a subspace W, then at least one vector from S is a linear combination ofthe other r - I vectors and can be discarded from S to produce a smaller spanning set.On the other hand. if B = {\'" ... , VIII} is a linearly independent spanning set for W.lhen no vector in B is a linear combination of the other m - 1 \"ectO~ in B. Hence if a

~nted matrix

gX3 = I,

(II)

follows

in SpeS)

•

Dft-1'\lrlO\, ,I


vector is removed from B, this smaller sel cannOI be a spanning sel for \V (in particular.the veclor removed from B is in \V but cannot be expressed as a linear combinalionof the vectors retained). In this sense a linearly independent spanning sel is a minimalspanning sel and hence represents the mosl efficient way of characterizing the subspace.This idea leads to the following definition.

lei W be a nonzero subspace of R". A basis for W is a linearly independentspanning sel for IV.

Note Ihat the zero subspace of R". IV = {B}. contains only the vector B. Althoughit is the case that (B) is a spanning set for IV, Ihe set {B} is linearly dependent. Thus theconcept of a basis is not meaningful for \V = {B}.

Uniqueness of Representation

lei B = {"I. "2..... vp } be a basis for a subspace W of R~. and lei x be a veclor in W.Because B is a spanning set. we know that then: are scalars 01. a2 • •.. . op such Ihat

x = al"\ + 02"2 + ... + 0l"'p- 12)

Because B is also a linearly independent set. v. e can show Ihal the representation of x inEq. (12) is unique. That is. if we have any n:prescmalion of the fonn x = hi VI + h2"2 +... + bp"p.lhen al = bl. 02 =~..... ap = hp- To eSlablish lhis uniqueness. supposethai bJ, b2 . .... bp are any scalars such thai

x = h1"l + h2"2 + ... + bp"p,

Subtracting the preceding equalion from Eq. (12). we obtain

8 = (al - hl)"1 + (a2 - b2)V2 + ... + (a p - bp)"p,

Then, using the factthat {"~I. "2 .... , vpJ is linearly independent. we see thai OJ -hi = O.a2 - ~ = O..... ap - bp = O. This discussion of uniqueness leads 10 the followingremark.

Remork lei B = (VI. v2 ..... vp ) be a basis for W, where \V is a subspace of RI!. Ifx is in W. then x can be represented uniquely in terms of the basis B. Thai is. there areunique scalars al. 02•.... 01' such that

x = 01"1 -02"2 + ... -0,."1"

As we see later. these scalm are called Ihe coordinates of x with respect to the basis.

Examples of Bases

II is easy 10 show that Ihe unil "ceIOTS

·,=Ul "~ [!l 'Od "~ [n


constitute a basis for R3 . In generaL the II-dimensional vectors el. e2 .... , e" form abasis for R", frequently called the natural basis.

In Exercise 30, the reader is asked to use Theorem 13 of Section 1.7 to prove thatany linearly independent subset B = {Vj. V2. ,')} of RJ is actually a basis for R3, Thus.for example, the vectors

"~Ul '2 ~ [il ond "~ [ : ]

provide another basis for R3 .

In Example 3. a procedure for detennining a spanning set for N(A), the null spaceof a matrix A, was illustrated. Note in Example 3 that the spanning set {Ul. Ul} obtainedis linearly independent so it is a basis for NCA). Oftentimes. if a subspace W of R" hasan algebraic specification in tenns of unconstrained variables. the procedure illustratedin Example 3 yields a basis for W. The next example provides another illustration.

EXA\\PLf 5 Let A be the (3 x 4) matrix given in Example 4 of Section 3.3. Use the algebraicspecification of R(A) derived in that example to obtain a basis for R(A).

Solution In Example 4 of Section 3,3, the range of A was determined to be

R(A) = {b b = [ :: ]. b l and b2 any real numbers].

3b l - b2

Thus b l and b2 are unconstrained variables, and a vector b in R(A) can be decomposed

"•

b ~ [ :: ] ~ [ ~ ] + [ ~,] = b, [ ~ ] + b, [ ~] .3b1 - b2 3b l -bl 3 -1

If Ul and U2 are defined by

u, ~ U] ond U~ [ -:J

(BI

then UI and U:l are in R(A). One can easily check that {Uj. U2} is a linearly independentset. and it is evident from Eq. (13) that R(A) is spanned by Ul and U2. Therefore,[UI. U:l} is a basis for R(A). •

The previous example illustrates how to obtain a basis for a subspace W, givenan algebraic specification for W. The last two examples of this seclion illustrate twodifferent techniques for constructing a basis for W from a spanning set.

3.4 Bases Cor Subspaces 197

· e~ fonn a D..\\\PlC b lei \V be the subspace of R4 spanned by the set S = (\'\. \'2. V). V4. v51, where

Find a subset of S that is a basis for \V.

Solution The procedure is suggested by Example 4. The idea is to solve the dependence relation

v, ~ [ ) l v, ~ [ tl "= [ -Jv, = [ _~l 'nd v, ~ [n

to prove Ihalfor R J

. Thus.

algebraic

XIVI-~~+~~+~~+~~=8

and then determine which of the Vj'S can be eliminaled. If V is the (4 x 5) matrix

114)

, - 0where xJ and Xs are unconstrained variables. In panicular. the set S is linearly dependent.:\1oreover. taking XJ = J and Xj = 0 yields x\ = 2, X2 = -3. and X4 = O. ThusEq. (14) becomes

v = [\'1. V2. v). V4. vs1.

then the augmemed matrix (V 8J reduces 10

[~0 -2 0 1

f]1 3 0 2

0 0 I -I(IS)

0 0 0 0

X2 = -3x) - 2,rs

(171

(16)

2x) - Xs

X4 = xs.

XI =

2\'1 - 3\"2 + \') = 9,

The system of equations with augmented matrix (15) has solution

?

31.j""O b

'omposed

1Since Eq. (17) can be solved for "J.

\'J = ~2"1 ..j.. 3"2.

it follows that v) is redundam and can be remo\ed from Ihe spanning set. Similarly.setting x) = 0 and Xs = I gh'es .1'\ = ~ L X2 = -2. and .1'4 = I. In this case, Eq. (14)becomes

-\'1 ~ 2"2 + V4 + v! = 8.

198 Chapter 3 The Veclor Space R"

and hence

V5 = \'1 T 2\"2 - \'4.

Since both V3 and Vs are in Splvi' V2. v~l. it follows (as in Example 4) thai VI, \'2. anc.\'~ span w.

To see that the set {"J, V2. v~1 is linearly independent. nOie that the dependen,,;crelation

Xl\'] T XlV! + .t~\'4 = 8

is JUSt Eq. (14) with \'~ and Vs removed. Thus the augmented matrix ["I. V2. v418]. forEq. (18) reduces to

[1000]o J 0 0

o 0 1 0 .

o 000which is matrix (15) with the third and fifth columns removed. From matrix (19). it i"clear that Eq. (18) has only the lrivial solution: so {\'l' \'2. \'~} is a linearly independentset and therefore a basis for W. •

The procedure demonstrated in the preceding example can be outlined as follo\\-,,:

I. A spanning set 51\'1 ... " \'",} for a subspace IV is ghen.

1. Solve the ,ector equation

XlVI + ",+x.v", =8.

,

3. If Eq. (20) has only the trivial solution XI = ... = x'" = O. then 5 is a Iinearl~

independent set and hence is a basis for tv.4. If Eq. (20) has nontrivial solutions. then there are unconstrained variables. For

each:rJ thai is designated as an unconstrained \'anable, delete the \'ector \'j fromthe set S. The remaining vectors constitute a basis for w.

Our final technique for constructing a basis uses Theorem 7.

TIIEORl\\ - If the nonzero matrix A is row equivalent to the matrix B in echelon form, then thenonzero rows of B form a basis for the row space of A.

Proof B} Theorem 6. A and B ha\e the same row space, II follows that the nonzero rows ofB span Ihe row space of A. Since the nonzero rows of an echelon matrix are Iinearl}independent vectors. it follows thallhe nonzero rows of B form a basis for the row spaceofA. •

r \.\ \\PLE - Let IV be the subspace of R4 gi\cn in Example 6. Use Theorem 7 to construct a basisfor IV.

, . "~. and

ikpendence

",~ 8J. for

'9

119). it ispendent

•iollO\H:

la Iinearl~

,Ie, For,. from

•

Solution

3.4 Bases (or Subspaces 199

As in Example 6. let V be the (4 x 5) matrix

y = [VI. V2. "3. V~. "3].

Thus \V can be viewed as the row space of the matrix yT. where

1 1 2 -1

1 2 1 1

V' = I I 4 -1 5

1 0 4 -1

2 5 0 2

Since yT is row equivalenllo the matrix

1 0 0 -9

0 I 0 4

B' ~ I 0 0 1 2

0 0 0 0

0 0 0 0

in echelon form. it follows from Theorem 7 that tbe nonzero rows of BT form a basisfor the row space of yT. Consequently the nonzero columns of

[

'0000]o I 0 0 0B~

o 0 I 0 0

~9 4 2 0 0

are a basis for W. Specifically. the set {Ul. u~. U3) is a basis of W. where

The procedure used in the preceding example can be summarized as follows:

1. A spanning set 5 := {"I ..... VIII} for a subspace W of R" is given.

2. Let Y be the (n x m) matrix Y := l"! .... \'",]. Use elementary row operationsto transform V T to a matrix aT in echelon form.

3. The nonzero columns of B are a basis for lv.

then the

fOVo.) ofIinearl}

'p3.ce

•~ 3 basi!>

u, ~ [ j1u, ~ [ ~1~d u, ~ [!1 •


D..± EXERCISES

In Exercises 1-8. let W be the subspace of R4 consistingof vectors of the fonn

,=[JFind a basis for IV when the components of x satisfy thegiven conditions.

1. XI TX2-X3 =0X2 -X4=0

2. XI + X2 - X3 +X4 = 0Xl - 2.\) -X4 = 0

3. XI -Xl +X3 -3X4 =0

4. x] - x2 - x) = 0

5.x] +X2 =0

6,Xl-Xl =0X2-2x) =0

X3- X4=0

7. -x] +2xl -X4 =0

Xl-X' =0

8,XI- X2- X3- X4=0

X2+X) =0

9. Let IV be the subspace described in Exercise I. Foreach veclor Xthai follows, determine if x is in \V. Ifx is in \V, then express x as a linear combination ofthe basis vectors found in Exercise I.

0) , ~ [ i] b) ,= [ -i],) , = [j d), = [ iJ

10. Let \V be the subspace deo;crjbed in Exercise 2. Foreach vector x that follows, delermine if x is in IV.If x is in IV, then express x as a linear combinationof the basis vectors found in Exercise 2.

0) ,= [ -~] b), = [ _i],) , = [ ~ ] d) , = [ ~~ ]

In Exercises 11-16:

a) Find a matrix 8 in reduced echelon form such that8 is row equivalent to the given matrix A.

b) Find a basis for the nulls~ of A.

c) As in Example 6. find a basis for the range of A thatconsists of columns of A. For each column, Aj. ofA that does not appear in the basis. express AJ as alinear combination of the basis vectors.

d) Exhibit a basis for the row space of A.

1I.A~[:2

3 -I ]5 8 -2

1 2 0[I 1:]12, A = I I

2 3

13. A=U2 1 0]5 3 -I

2 o 2

1 1-I[22 0]14. A = 2 I 1

2 3 0

[' , I]IS, A = 2 4 1

362


26. Find a basis for the range of each matrix in Exercise 25.

25. Find a basis for the null space ofeach of the following matrices.

Find every subset of S thaI is a basis for R2.

29. Let 5 = (VI. \'2, V3, "..1, .... here

" = [ ~ l '0 = [ ~ l "d

.,=[ -: l

32. h ![J [: H=n!

Show thaI S is a linearly dependent sel. and verifythatSp{\'J. \'2· \'31 = Sp{vl,v2l,

28, Let S = {v]. \'2, v31. where

"~[il '0 = [ =:J"= [-;J '"d ", = [ =: l

27. Let S = {\'J. \'2. "Jl. where

" = [ il '0 =[ =:J "d

'0 = [ -J

Find e\ery subset of 5 that is a basis for R).

30, ut B = {VI, "2, ")1 be a seloflinearly independentvectors in R), Prove that B is a basis for R3. [Hint:Use Theorem 13 of Section 1.7 to show that B is aspanning set for R3,J

31. ut B = {"J, "2. V3} be a subset of R3 such thatSp(B) = R3. Prove that B is a basis for R3. [Hint:esc Theorem 13 of Section 1.7 to show thaI B is alinearly independent set.)

In Exercises 32-35, detennine whether the given set Sis a basis for R3.

,

-nlI I 0]I I 0~ ~] b) [

~ ]'J [

<J [

[2 I 'J16..-1.= 2 2 1

2 3 0-. Cse the technique illustrated in Example 7 10 obtain

a basis for the range of A. where A is the matrixgiven in Exercise 11

Repeat Exercise 17 for the matrix given in Exercise 12.

19. Repeat Exercise 17 for the matrix given in Exercise 13.

~. Repeat Exercise 17 for the matrix gi\'en in Exercise 14.

Exercises 21-24 for the given set 5:

.' Find a subset of 5 that is a basis for Sp(S) using Ihetechnique illustrated in Example 6.

b' Find a basis for Sp(S) using the techniqueillustrated in Example 7.

Ol.h![: H~]l

1Lh![:H:H:]1D.s=![iH][;JtJ!~s=l[ -iH~;Hj[

J,

~_SUCh that

eeOfAth31• A .ofA. asa


33.hlUH1]U])34. 5 ~I[=; H;H=: H:])3;,s~I[J[1])36. Find a vector w in R3 such that w is not a linear

combination of \'J and "2:

"~ [J ood "~ [ J

37. Prove that every basis for R2 contains exactly twovectors. Proceed by showing the following:

a) A basis for R2 cannot have more than twovectors.

b) A basis for R2 canriot have one vector. [Hilll:Suppose that a basis for R2 could contain onevector. Represent ej and e2 in terms of the basi~

and obtain a contradiction.J

38. Show that any spanning set for Rn must containat least n vectors. Proceed by showing that ifUj, U2 .....up are vectors in R". and if p < n.then there is a nonzero vector v in Rn such thatvTu, = O. 1 sis p. [Him: Write the conSlrainli.as a (p x II) system and use Theorem 4 of Section1.3.J Given v as above. can v be a linear combinationofuj. Ul, ... up?

39. Recalling Exercise 38, prove that every basis for R"contains exactly 11 vectors.

~ DIMENSION

In this section we translate the geometric concept of dimension into algebraic terms.Clearly R2 and R3 have dimension 2 and 3. respectively. since these vector spaces aresimply algebraic interpretations of two-space and three-space. It would be natural toextrapolate from these two cases and declare that Rn has dimension f1 for each positiveinteger II: indeed, we have earlier referred to elements of R" as II-dimensional vectors.But if IV is a subspace of Rn, how is the dimension of IV to be detennined? Anexamination of the subspace. IV. of R3 defined by

. [,,-2 X)]

\V = {x: x = :; ,Xl and X3 any real numbers}

suggests a possibility. Geometrical.ly. IV is the. plane with equation x = J - 2;:. sonaturally the dimension of IV is 2. The techniques of the previous section show that IVhas a basis {VI, V2} consisting of the two vectors

'. ~ [i] and "~ [ -nThus in this case the dimension of IV is equal to the number of vectors in a basis for W.

The Definition of Dimension

More generally. for any subspace \V of W, we wish to define the dimension of \V 10be the number of vectors in a basis for IV. We have seen, however. that a subspace IV

3.5 Dimension 203

To ~how Ihat (SI, 52 .. , .• Sill) is linearly dependent. we must show that there is a nontrivialsolution of

Let W be a subspace of W. and let B = {"'I. "'2, .... ",pI be a spanning set for Wcomaining p vectors. Then any set of p + 1 or more vcctors in W is linearly dependcllt.

Let {Sl. S2, .... Sill} be ally set of 111 vectors in W. where III > p. To show that this set islinearly dependent, we first express each Sj in terms of the spanning set B:

(4

(I)

'21

(3a)

13hl

allCt ..l. UI2C2 + ... +UI",C. = 0

U21CI Ta~Jc2--1.···+a:lmc .. =0

Sill = al",wl + U:lmW2 + ... + upmw p'

Sj = 011"'1 + a21"'2 + +apl"'p

S2 = 012"'1 + 022W2 + +Op:lwp

Equation (3a) can be regrouped as

Now using system (I), \\Oe can rewrite Eq. (2) in terms of the vectors in B as

CI(OIl \~ I ..&.- a21 \\'2 - + Opl W p ) +C2(OnWI - 022w2 + - ap 2\\'p) .,..

... + c.. (aj .. wl - U:!mw2 + up"'Wp ) = 8.

CjSI ..l. C2S2 + ... + C"'S'" = 8.

(CIUIl +C2UI2 - +e",aj.)wj +(CIU21 + C2U22 T + C...U~)w2 T

... - (CIUpl + C2Up2 + - c..u.-)wp = 8.

Nov. finding Cj. C2 • •••• c.. to satisfy Eq. (2) is the same as finding CI. C2 • ...• C. tosatisfy Eq. (3b). Funherrnore. we can clearly satisfy Eq. (3b) if v.e can choose zero foreach coefficient of each w,. Therefore. 10 obtain one solution of Eq. (3b). it suffices 10

sohe the system

may have many different bases. In facl. Exercise 30 of Section 3.4 shows Ihat any sel ofthree linearly independent vectors in R 3 is a basis for Rl . Therefore. for the concept ofdimension to make sense, we must show that all bases for a given subspace IV containthe same number of vectors. This fact will be an easy consequence of the followingtheorem.

Prouf

litt Oil.! \I "l

thatU

'.or {Him:lam one.fthe ba!oh

n.actly IWO

a-lJ1g

two

- ~.:. so

biorR

U'-I ContalOing that iflJ P < fl.

. ,-uch thaiconstram~

r. of Section~binatlon

.. for n'

of IV 10,j).lce U'

aplCI + Up2C2 + ... + 0p",e", = O.

[Recall thai each ail is a specified constant determined by system (1), whereas eachCi is an unknown parameter of Eq. (2).] The homogeneous system in (4) has moreunknowns than equations. so by Theorem 4 of Seclion 1.3 there is a nontrivial solutionto system (4). But a solution to system (4) is also a solution to Eq. (2). so Eq. (2) has anontrivial solution, and the theorem is established, ....

•

204 Chapter 3 The Vector Space Rfl

As an immediate corollary of Theorem 8. we can show that all bases for a subspa.;ccontain the same number of \'ectors.

I t COI\Oll.\I\' Let W be a subspace of Rfl• and let B = {WI. W2..... w p } be a basis for IV containin,:,:

I p veclOrs. Then every basis for \V contains p "ectors.

Proof Let Q = {Ul. U2, .. , , UrI be any basis for IV. Since Q is a spanning set for IV. b)Theorem 8 any set of r + I or more vectors in \f is linearly dependent. Since 8 is.linearly independem set of p vecwrs in IV. we know that p :::: r. Similarly, since B ia spanning set of p vectors for W, any set of p + I or more vecwrs in IV is linear!.dependent. By assumption, Q is a set of r linearly independent vectors in IV; so r :::: pNow. since we have p :::: rand r S p, it must be that r = p. •

Given that e\'ery basis for a subspace comains the same number of vectors, we canmake the following definition without any possibility of ambiguity.

DH1 .... llIO .... :) Let IV be a subspace of Rfl• If IV has a basis B = {WI. "'2..... wp } of

p vectors, then we say that \V is a subspace of dimension p. and we writedim(IV) = p.

In Exercise 30. the reader is asked to show that every nonzero subspace of R fl doesha\'e a basis. Thus a value for dimension can be assigned to any subspace of Rfl

• wherefor completeness we define dime W) = 0 if IV is the zero subspace.

Since R3 has a basis lei. e2. e3} containing three vectors, we see that dim(R3) = 3.In general. Rfl has a basis lei, e2..... efl} that contains n veclOrs: sodim(R fI

) = n. Thusthe definition of dimension-the number of vectors in a basis-agrees with the usualtenninology: R3 is three-dimensional. and in general. Rfl is Il..-dimensional.

F\,\\\PII: I Let W be the subspace of R3 defined by

[

X, ]IV = {x: x = X2 . XI = -2X3. -"2 = X3. X3 arbitrary}.

x,Exhibit a basis for IV and detennine dim(1V).

50lulion A vector Xin \V can be wri!ten in the fonn

[

-lx, ] [ -2 ]x = :: = XJ : •

Therefore. the set {u} is a basis for \V. where

u~ [-:J

3.5 Dimension 205

Let IV be the subspace of R3• IV = span[u]. U2, U3. u41. where

Use the techniques illustrated in Examples 5. 6. and 7of Section 3.4 to find three differentbases for IV. Give the dimension of IV.

s..

u; ~ [ : l anh = [ _: ]

XIUj + .'f2U2 - X3U3 +.'f414 = b

is consistenl. The mauix equalion for (5a) is Ux = b. "" here U is the (3 x -I.)matrix U = [Ul. U2. U3. 14]. Now. the augmemed matrix [U I b] is TO\It equivalent to the matrix

Then b is in W if and only if the veclor equation

b~ [: l

u, = [] u, ~ [ ~ l(a) The lechnique used in Example 5 consisted of finding a basis for IV by using

the algebraic specificalion for IV. In panicular.let b be a vector in R3:

It follows that dim(IV) = I. Geometrically. IV is the line through the origin and throughthe point with coordinates (-2. I. I). so again thedefinition of dimension coincides withour geometric intuition. •

The next example illuSlTates the importance of the corollary 10 Theorem 8.

Solution

EX,\MPI r 2

w of,e wme

R') = 3= n. Thlh<u

1\ comainin!

:-tors.\lte

f~)r a subspa~

iott for 1\'. b:ince B i\

'~. since B.n is Jinearl.

I";sor:::;:.

[

1 0 1 -1

o 1 1 3/2

o 0 0 0

20 - b ]-0/2 + b/2 .

--ta+2b ...... c

(Sh'

Thus b is in 1V if and only if -40 +2b +c = 0 or. equivalently. c = -ta - lb.The subspace IV can then be described by

IV = (b: b = [ : ]. 1I and b any real numbers).

40 - 2b

From Ihis description il follows that IV has a basis {VI. \'2}. where

"~ U] ond "~ [ _: l~


(b) The technique used in Example 6 consisted of discarding redundam vectorsfrom a spanning set for W. In particular since {Ul' U2. U3. u.1I spans lV, thistechnique gives a basis for \V that is a subset of {Ul. U2. U3. ~}. To obtain sucha subset, solve the dependence relation

X1Ul +X2U2 +X3U3 +X414 = 8. eSc

Note that Eq. (5c) is just Eq. (5a) with b = 8. It is easily seen from matri't(5b) that Eq. (5c) is equivalent \0 the reduced system

x,

Backsolving (5d) yields

-4X3 - X4

.1"2 + X3 + (3/2)X4

Xl = -X3 + X4

~O

~O.(Sd I

X2 = -X3 - (3/2)X4.

"'here X3 and X4 are arbilrary. Therefore. the vectors U3 and 14 can be deletedfrom the spanning set for \Y. lea\"ing {UI. u21 as a basis for W.

(c) Let U be the (3 x 4) matrix whose columns span lV. U = [Ul' U2. U3.14).Following the technique of Example 7. reduce UT to the matrix

[I 0 4]

CT = 0 1-2o 0 0

000in echelon form. In this case the nonzero columns of

[

I 0

C= 0 I

4 -2

ooo n

fonn a basis for W; that is. (WI. w21 is a basis for W. where

W, ~ [ ~] 'nd w, ~ [ -!JIn each case the basis obtained for \V contains two vectors. sodim(W) = 2. Indeed.

viewed geometrically. W is the plane with equation -4x + 2)' + z = O. •

Properties of a p.Dimensional Subspace

An imponam feature of dimension is that a p-dimensional subspace W has many of thesame properties as R'. For example. Theorem 11 of Section 1.7 shows that any set ofp + I or more vectors in R' is linearly dependent The following theorem shows thatthis same property and others hold in W when dim(1V) = p.

IHI OK.r\\ 9\ector.

n. tJuosuch

~..:

marnx

'"

deleted

u Q;J.

3.5 Dimension 207

Let W be a subspace of R" wilh dim(W) = p.

1. Any set of p + 1or more vectors in W is linearly dependent.

2. Any set of fewer than p vectors in W does not span W.

3. Any set of p linearly independent \'ectors in W is a basis for W.

4. Any set of p vectors that spans W is a basis for w.Proof Property I follows immediately from Theorem 8. because dim(W) = p means that W

has a basis (and hence a spanning set) of p vectors.Property 2 is equivalent to the statement that a spanning set for W must contain at

least p veclOrs. Again, this is an immediate consequence of Theorem 8.Toestablishpropeny 3, let {UI. U!..... up) be a setof p linearly independent vectors

in W. To see that the given set spans W.let v be any veclOr in W. By propeny 1. the set{\'. UI, U!, .... up} is a linearly dependent set of vectors because the set contains p + Ivectors. Thus there are scalars ao. al ..... ap (nol all of which are zero) such that

00\' + alUI + a2U2 + ... + opup = 8. (6)

In addition. in Eq. (6). Go cannot be zero because lUi. U!..... up} is a Ijnearly independent set. Therefore, Eq. (6) can be rewritten as

\' = (-I/ao)[a]ul + 02"2 + ... +opup]. (7)

It is clear from Eq. (7) that any vector in W can be expressed as a linear combination ofUI. U2 ..••• up, so the given linearly independent set also spans W. Therefore, the set isa basis.

The proof of property 4 is left as an exercise. -

E\-\ \\1>1 E J Let W be the subspace of R3 given in Example 2. and let lVI. v2. V3) be the subset of Wdefined by

VI~ [ -J v, ~ [ ~ l "d v) ~ [ ]

lndeed.

•

I~ of theset of~ that

Determine which of the subsets {\'d. {'"21, {Vl. v21, {v], v31. {v2' v31, and {v], V2, "3) isa basis for W.

Solution In Example 2, the subspace \V was described as

W = {b: b = [ : ]. a and b any real numbers}, (8)4a - 2b

Using Eq, (8), we can easily check thai v], v2. and v3 are in W. We saw further inExample 2 that dim(W) = 2. By Theorem 9, propeny 2. neither of the sets {,,]} or {V2}spans IV. By Theorem 9, propeny 1, the set {v]. V2, v3} is linearly dependent We caneasily check that each of the sets {v]. "21, {VI, V3}, and {V2. V3}. is linearly independent.so by Theorem 9. property 3. each is a basis for IV. •


The Rank of a Matrix

In this subsection we use the concept of dimension 10 characterize nonsingular matricesand 10 detennine precisely when a system of linear equations Ax = b is consistent. Foran (m x n) matrix A, the dimension of the null space is called the nullity ofA. and thedimension of the range of A is called the rank ofA. The following example will illustratethe relationship between the rank of A and the nullity of A. as well as the relationshipbetween the rank of A and the dimension of the row space of A.

-Ex"" \\PLI:: I Find the rank. nullity, and dimension of the row space for the matrix A, where

A ~ [ -~1

o4

1 2]2 -3 .

8 5

Solution To find the dimension of the row space of A, observe that A is row equivalent to thematrix

B= [i o -2

I 3

o 0 nand B is in echelon form. Since the nonzero rows of B fonn a basis for the row spaceof A, the row space of A has dimension 3.

To find the nullity of A, we must detennine the dimension of the null space. Sincethe homogeneous system Ax = 6 is equivalent to Bx = 6, the null space of A can bedetennined by solving Bx = 6. This gives

Xl = 2x)

X2 = -3X3

X~ = O.

Thus N(A) can be described by

[

2Xl]-3x)N(A) ~ Ixo> = OXl' Xl ,"y ",I ,umbe'l·

It now follows that the nullity of A is 1 because the vector

v ~ [ -r]forms a basis for N(A).

To find the rank of A, we must detennine the dimension of the range of A. Recallthat 'R(A), the range of A, equals the column space of A, so a basis for 'R(A) can be

gular matrices)Osistenl. For

'ofA, and the~ \\.i11 illustrate~ relationship

re

inlent 10 the

3.5 Dimension 209

found by reducing AT to echelon form. It is straightforward to show that AT is rowequivalent to the matrix C T , where

[I 0 0]

CT = 0 I 0 .o 0 I

o 0 0

The nonzero columns of the matrix C,

C~[~0 0 nI 0

0 I

fonn a basis for neAl. Thus the rank of A is 3. -

The proof of Theorem 10 will be gi\'en at the end of this section. Note that the rangeof AT is equal to the column space of AT. But the column space of AT is precisely therow space of A. so the following corollaJ) is actually a restatement of Theorem 10.

Note in the previous example that the row space of A is a subspace of R~. whereasthe column space (or range) of A is a subspace of R3, Thus they are entirely differentsubspaces: even so, the dimensions are the same, and the next theorem states that this isahl,'3)'s the case.

~ row space

fpace. Sincel'fA can be

T+'t:OI~t:\\ 10 If A is an (III x II) matrix. then the rank of A is equal to the rank of AT. -

.-\. Recall.-\) can be

COl\llll \In If A is an (m x II) matri~. then the row space and the column space of A ha\e the samedimension. _

This corollary pro\'ides a useful way to determine the rank of a matrix A. Specifi..call}. if A is row equh-alent to a matriJi: B in echelon form. then the number. r. of nonzerorows in B equals the rank of A.

The null space of an (m x /I) matrix A is determined by soh ing the homogeneouss},tem of equations Ax = 8. Suppose the augmented matrix [A 18J for the system isrow equivalent to the matri~ IB 8J. which is in echelon form. Then clear!} A is rowequivalent to B. and the number. r. of nonzero rows of B equals the rank of A. But r

is also the number of nonzero rows of [B 18J. It follows from Theorem 3 of Section 1.3that there are " - r free variables in a solution for Ax = 8. But the number of vectorsin a basis for ..\~(A) equals the number of free variables in the solution for Ax = 8(see Example 3 of Section 3.4): that is. the nullity of A is II - r. Thus we ha\'e show n.informally. Ihat the following formula holds.

Remark If A is an (Ill x II) malrix.lhen

Il = rank(A) + nullily(A) .

This remark will be proved formally in a more general COntexl in Chapter 5.


Example 4 illustrates the argument preceding the remark. If A is the matrix givetlin Example 4,

A ~ [ -;

I I

-~ l0 2

4 8

then the augmented matrix fA 19] is row equivalent 10

rBle)~ U 0 -2 0 n1 3 0

0 0 1

Since A is row equivalent 10 8. the corollary to Theorem 10 implies that A has rank3. Further. in the notation of Theorem 3 of Section 1.3. the system Ax = 9 has 11 = ~

unknowns. and the reduced matrix [8181 has r = 3 nonzero rows. Therefore, tht'solution for Ax = 8 has n - r = 4 - 3 = I independent variables, and it follows thaithe nullity of A is I. In particular,

rank(A) + nullity(A) = 3 + I = 4,

as is guaranteed by the remark.The following theorem uses the concept of the rank of a matrix to establish necessal)

and sufficient conditions for a system of equations. Ax = b. 10 be consistent.

TH-EOll,t\\ 11 An-(m x II) system of linear equations. Ax = b. is consistent if and only if

rank(A) = rank([A I bJ).

Proof Suppose that A = fA J • A 2, .... An]. Then the rank of A is the dimension of the columnspace of A. that is. the subspace

Sp{A j .A2, ... ,An )·

Similarly, the rank of fA I b1 is the dimension of the subspace

SprAt. A2 •... ,An' b}.

(9'

(lOJ

But we already know that Ax = b is consistent ifand only ifb is in the column space ofA. It follows that Ax = b is consistent if and only if the sub:o>paces given in Eq. (9) andEq. (10) are equal and consequently have the same dimension. _

Our final theorem in this section shows that rank can be used 10 detennine nonsin·gular matrices.

-l~! rORr \\ 11 An (11 x 11) matrix A is nonsingular if and only if the rank of A is I!.

Proof Suppose that A = fA J • A2' .... A"]. The proof of TIleorem 12 rests on the observationthat the range of A is given by

R(A)~ Sp{A"A"""A"I, (11

If A is nonsingular then. by Theorem 12 of Section 1.7, the columns of A are linearl}independent. Thus {A J • A2..... All} is a basis for R(A). and the rank of A is II.

matrix given

~h" "'nk

hasn=4fore. the

)lJows Ihal

'1eCess<U)

column

3.5 Dimension 211

Conversely. suppose that A has rank n: that is, R.(A) has dimension n. It is animmediate consequence of Eq. (II) and Theorem 9. property 4. that {AI. A2... " A~}

is a basis for 'R,(A). In particular, the columns of A are linearly independent. so. byTheorem 12 of Section 1.7. A is nonsingular. ~


To prove Theorem 10. let A = (au) be an (m x n) matrix. Denote Ihe rows of A by

Ill. a2_···. am' Thus,

a, = [ail.ai2 •.. .. ai~l.

Similarly, lei AI. A:l .... , A~ be the columns of A. where

[

aJ) ]

Aj

= a2J •

a.)

Suppose thai AT has rank k. Since the columns of AT are af. aI ..... a~. it follows that

if

\V = Sp{al.32..... 3"'1.

then dim(W) = k. Therefore, W has a basis {WI. W2 •.••• w.J and. by Theorem 9.property 2, III ::: k. For I .::: j .::: k. suppose thai Wj is the (I x n) ,ector

Wj = [Wjl. Wj2···· .U.'i"]·

Writing each a, in terms of the basis yields

Equating the jth component of the left side of system (12) with the jth componentof the right side yields

•10

pace of«9) and

•, nonsin-

~nation

l 11r linear!)

[all. au al~J = 81 = CIlWI + CI2,",'2 -r-'" + CltWt

(all. an a2.llJ = 32 = C21wI + C22W2 + ... + C2tWt

(a.. l. a..2, ...• a..~J = a", = C.. lwl +C",lw2 + ...... C",t,",'t.

[

aJ) ] [ c" ] [ CJ2 ] [ C" ]a. C21 C22 C2t~J = Wlj . + W:2J • + ... + Wtj .. . . .. . . .

alii; C",l e",2 C",J:

for I .::: j .::: n. For I .::: i .::: k, define c, to be lhe (m x I) column vector

[

C" ]C.c., = .

c.,

(Ill

(1J)


Then system (13) becomes

'R.(A) = Sp{Al . A1.... AII }:; SpfCI' C2· .. ·. Ctl·

It follows from Theorem 8 that the subspace

V = Sp(CI. C2 ..... cd

has dimension k. at most. By Exercise 32. dim[R(A)) .:s dim(V) ::: k: thtlt is.rank(A) .:s rank(A T ).

Since (AT)T = A. the same argument implies that rank(A T) .:s rank(A). TIm"rank(A) = rank{A T ). _

AJ = WljCI + lL'2jC2 + ... + WtJCb l.:s j ::: n.

It follows from the equations in (14) that

"..

•

14

~ ][

1 ,22. A = 2 -.5

+ .\"3-2t 4=0.1'2+2xJ -3x4=0

20. .1'1-.1'2 =0X2-21"J =0

·1"3- X4=0

18. Xl

Tn Exercises 21-24. find a basis for N(A) and give thenullity and the rank of A.

[

X, ]_ X2x_ .x,x,

21. A _ [ _~ _~ ]

10. S = lUI. Ul} for R 2

11. 5 = {U2. U3} for R2

12.5= IVI. V2. v31 for RJ

13.5= IVI. "2. v~} for RJ

!-t. 5 = {"2. "3. "41 for R3

In Exercises 15-20. \V is a subspace of R4 consisting ofvectors of the fonn

Determine dim("') when the components of x satisfythe given conditions.

IS. Xl - 2.t2 + XJ -.\"4 = 0

16,XI-2t] =0

17. Xl = -X2 - 2x4

X3 = -X4

19. Xl = -X4

X2 = 3X4

X3 = 2x4

u, ~ [ :1u, ~ [ :1u, = [ -: 1"'=[a u'~[:l v,~[ -;]v,=[;] "~[ -] ,.,~[-:]

"In Exercises 1-6. detennine by inspection .... hy the givenset 5 is not a basis for R2• (That is. either S is linearlydependent or S does not span R2.)

LS={uJ! 2.S={Ul}

3. S = {Ul' U2. U3} 4. S = {U2. U3. us}

5. S = {U!.14} 6. S = {UI. usl

~ EXERCISES

In Exercises 7-9. detennine b} inspection .... hy the givenset S is nOl a basis for R3 . (That is. either S is linearlydependent or S does not span R3 .)

7. S= {Vl."2} 8. S= {VI. V)}

9. S = {"l. V2. "J. V4}

In Exercises 10-14. use Theorem 9. propeny 3. to detennine .... hether the gi,en set is a basis for the indicated\ector space.

Ex.ercises 1-14 referto the vectors in (l5l.

gi\'e the

.-4, Thu_~

•

-:!..r~ =0Jr~ = 0

~O

~O

-_f~=O

that ie;:.

",-"'tingof

\ salisfy

,~ ].;

[ I-I :]13.A= 2-1

-I •

"A~ [;2 0 ;]3 1

3 -I

In Exercises 25 and 26. find a basis fOT R(A) and givethe nullity and Ihe rank of A .

25. A~ [ -: 21]o 3

1 5 7

'6'A=[~1 2 0]• , .1 5 -2

27. ut W be a subs~. and let S be- a spanning sel forW. Find a basis for \Y. and calculate dim(W) foreach sel S.

.) h I[J [=;H_][-~])b)S=I[ -i]{H~iH -f]l28. Let W be the subspace of R~ defined by \V =

fx: \,T x = 01. Calculate dim(\V). where

,.~[J29. Let W be- !he subspace of R~ defined by IV = Ix:

aTx = 0 and bTx = 0 and eTx = 01. Calculatedim{W) for

3.5 Dimension 213

.=[ -J b~[ _] ood

,~[ -:J30. LeI IV be a nonzero subspace of R~. Show Ihal IV

has a basis. [Hint: LeI WI be any nonzero \eclOl" in\Y. If (WI I is a spanning set for \V. then we are done.IfnOl.lhere isa \ectOr""! in IV such that (\II. \Illis linearly independent. Why? Continue by askingwhether this is a spanning set for IV. Why must thisprocess eventually stop?]

31. Suppose thaI {UI. U2 •.•.. up} is a basi~ for a subspace IV. and suppose that x is in IV with x =0IU] + a2D! + ... - OpDp- Show thallhis representation for x in lenns of !he basis is uniqueIhal is. if x = blUI + b1U2 + ... - bpup. thenb l = al. b1 = Ul • •... bp = up-

32. LeI U and V be subspaces of Rn• and suppose thatU is a subset of V. Prove Ihat dim(U) ::: dim(V).If dim(U) = dim(V). prove Ihal V is contained inU. and thus conclude thaI U = v.

33. For each of the following. determine lhe largest possible \'alue for the rank of A and the smallest possible\'alue for Ihe nullily of A.

a) A is(3 x 3)b) A is(3 x 4)

c) A is(5 x 4)

34. If A is a (3 x 4) matrix. prove that the columns ofA are linearly dependent.

35. If A is a (4 x 3) m:urh:. prove Ihal the rows of A arelinearly dependenl.

36. Let A bean (m x n) matrix. Pro\e Ihal rank(A) .::: mand I1lnk(A) ::: n.

37. Let A be a (2 x 3) matrix with Tank 2. Show thai the(2 x 3) system of equations Ax = b is consistentfor e\ cry choice of b in R2.

38. lei A be a (3 x 4) matrix wilh nullity 1. Prove Ihatthe(Jx4)systemofequationsAx = bisconsislenlfore\erychoiccofbin RJ .

214 Chapter 3 The Vector Space W

39. Prove that lin (n x II) matri;( A is nonsingular if andonly if the nullil}' of A is zero.

40. lei A be an (m x m) nonsingular mlltrix, and let Bbe an (m x II) mlltrix. Prove that X(AB) = ;\'(B)and conclude that rank (A B) = rank (B).

41. Prove propeny 4 of Theorem 9 as follows: Assumethat dim(W) = p and let S = ("'·1 ..... '""pI be a setof p vectors that spans W. To see that S is linearlyindependent. suppose that CIWI + ... + cpwp =8. If Cj :F O. show that W = Sp{Wl ..... WI-J.

'""'-0-1 •...• wpl· Finall). use Theorem 8 to reach acomrndiction.

-U. Suppose that S = {Ul. u!..... up} is a set of linearly independent vectors in II subspace IV. wheredim(W) = 11/ and If/ > p. Prove that there is II

\-ector Up_I in IV such that {UI. U2 ..... up. Up_IIis linearl) independelll. Use this proof to shO\\ thatII basis including all the veclOrs in S can be constructed for W.

i3,E, ORTHOGONAL BASES FOR SUBSPACES

We have seen that a basis provides a very efficient way to chamcterize a subspace. Also.given a subspace IV. we know that there are many different ways to construct a basis forIV. In this '\eCtion we focus on a particular type of basis called an onllOgollaf basis.

Orthogonal Bases

The idea of onhogonality is a generalization of the vector geometT) concept of perpendicularit). If u and \' are twO vectors in R2 or R3• then ",e know that u and \' areperpendicular if ur \' = a (see Theorem 7 in Section 2.3). For example. consider thevectors U and v given by

u ~ [ _~] and "= [ : ],Clearly u r v = 0, and these two vectors are perpendicular when viewed as directed linesegmems in the plane (see Fig. 3.13).

F;gu,~J.IJ In R~. non.tero vectors u and ,'are perpendicular if andonl)ifur,,=O.

In general. for vectors in Rn. we use the te.rm orthogonal mther than the tennperpendicular. Specifically. if u and \' are vectors in Rn. we say that U and \' areorthogonal if

uTv=O.

We will also find the concept of an onhogonal set of vectors to be useful.

3.6 Orthogonal Bases ror Subspaces 215

~m 8 to reach a

1~ a set of lin,pace IV. where

,; that there is aI: ... up. up...d1'\'01' to show thata 5 can be con-

Dt:t-l'lTIo, b Let 5 = {Ut. U2 ••... up} be a set of \'ectors in R~. The set S i!. said to bean orthogollal set if each pair of distinct \'ectors from S is orthogonal; that is.u'["Uj =Owheni f:.j.

E,\\\\PI r I Verify that S is an orthogonal set ofveclOrs. where

Therefore. 5 = {u]. U1. uJ} is an orthogonal set of "ectors in R4•

Soilltion If we use the notation S = {Ut. U2. uJ}, then

s~l[rH -iH =iJ!

•

1 ]-2= 1-2"t"I+O=O.

-I

o

21[ _i]~'_o-,+o=o

21[ =i]~'_o-,+o=o

01 [-IuIU; = [I

u[u1=[1 0 I

ur UJ = (I 0 I

directed line

~~pace. Also.'t a basis for

#U11 basis.

ocept of peru and \' are

,. .:on~ider the

An important property of an onhogonal set 5 is that S is necessarily linearly independent (so long as S does not contain the zero vector).

IllrOR.r \I 13 Let S = lUI. U2 ••••• upl be a set of nonzero vectors in R~. If S is an orthogonal set ofvectors. Ihen S is a linearly independent set of veclOrs.

Proof leI (1. q ..... Cp be any scalars that satisfythe termII and v are

CtUt - C1U1 + ... + CpU p = O. 11

Form the scalar product

UiCctUI +C2U1 +.. ·c"Up ) = uio

2J6 Chapter 3 The Vector Space Nit

0'

-- --,

Cj(U[ lIl) + C~(Ur U~) +, .. -l- Cp(u'i Up) = O.

Since uf llj = 0 for 2.::: j :5 p. the expression abo\c reduccslo

("j (Uf"l) = O. '-

DHI'drro, -

Kexi. !>e1:al.L<;e uj lit > 0 when"j is nonzero. we see from Eq. (2) that Cl = O.Similarly. forming the scalar product of both sides of Eq. (I) with lli. we sec that

cs(u; ll,) = 0 or Ci = 0 for I .::: i .::: p. Thus S is a linearly independent sel 01'·ecto~. •

By Theorem 13, any orthogonal SCI S containing p nonzero vectors from a pdimensional subspace \V will be a basis for W (since S is a linearly independent subsetof p ,'eeton; from W. where dim{lV) = pl. Such a bash. is called an orthogonal basis. Inthe following definition. recall that the symbolll'-U denotes the length oC\'.II\'1I =~

Let IV be a sub~pace of RI/. and let B = {UJ. "2, .... Up} be a basis for W. IfB is an orthogonal set of \cctors. then B is called an orthogonal basis for W.

Funhennore. if u, I = I for I .::: i .::: p. then B i<; said to be an orthonormalbasis for HI.

The "ord ort/wnonl/o! suggesls both orthogonal and normalized. Thus an orthonormal basis is an orthogonal basis consisling of vectors having length I. where alector of length 1 is a unit vector or a nonnaliLed vector. Observe that Ihe unit lectorsc,. C1 ..... ell fonn an onhononnal basis for R·.

E\--\ \IPlt: 2 Verif) that the set B = {\ I. \'2. \'J} is an onhogonal basis for R3• where

v, ~ [il v,= [=:J and ",= [ -]

Solution We first verify that B is an orthogonal sel by calculating

vi"1 = 3 - 2 - I = 0

vi \'J = I - 8 ..j.. 7 = 0

vf \"3 = 3 + 4 - 7 = O.

Now. R3 has dimemion 3. Thus. since 8 is a set of three \ectors and is also a linearlyindependent sel (see Theorem 13). it follows that B is an orthogonal basis for RJ • •

These observations are stated fOrolany in the following corollary of Theorem 13.

(2)

c = O.:h U , we see that.:Iependent set of

•XlOrs from a p_.:Iependent subsetbot!-onal basis. In\-.- vII = JvTv.

c'b for IV. Ifsis for IV.

onhollormal

3.6 Orthogonal Bases for Subspaces 217

COl\OLL "I' Let IV be a subspace of R". where dim(lV) = p. If S is an orthogonal set of p nonzerovectors and is also a subset of IV. then S is an orthogonal basis for W. •

Orthonormal Bases

If B = {UI, U2' ...• upl is an orthogonal set. then C = l{/j Ul. {/2U2 ....• apup } is alsoan orthogonal set for any scalars al. a2 ..... ap ' If B contains only nonzero vectors andif wc define the scalars (/, by

aj = J T .'u, u,

then C is an orthol/ormal set. That is. we can convert an orthogonal set of nonzerovectors into an orthonormal set by dividing each vector by its length.

1 \+\ "PLE J Recall that the set B in Example 2 is an orthogonal basis for R3. Modify B so that it isan orthonormal basis.

Soilltion Given that B = {v). V2. V3} is an orthogonal basis for R3. we can modify B to be anorthonormal basis by dividing each vector by its length. In particular (see Example 2).the lengths of VI, "2. and V3 arc

IlvJlI = J6. IIv211 =./!T, and IIv311 = ../66.

Therefore. the set C = lWI. ""2. W3} is an orthononnal basis for R3• where

__1 V3 =w3 - J66

d. Thus an or~[lgth I, where athe unit vectors

I \' =WI = .J6 1

[

1/)6 ]2/)6 .

1/)6

[1/./66]

-4/./66 .7/./66

1 \', =W2 = ji'1 .[

3/JIT]-l/JIT .-l/JIT

'nd

•

• also a linearly_n for R 3• ~

.Theorem 13.

Determining Coordinates

Suppose that IV is a p-dimensional subspace of R". and B = {WI. ""2. , ... wpl is abasis for IV. If v is any vector in IV. then v can be written uniquely in the form

v=al\\'1 +{/2w2+···+{/p"",,·

(In Eq. (3), the fact that the scalars al. a2' .... (/p are unique is proved in Exercise 31 ofSection 3.5.) The scalars {/J. {/2 •.... {/p in Eq. (3) are called the coordinates of I' withrespect to the basis B.

As we will see, it is fairly easy to determine the coordinates of a vector with respect toan orthogonal basis. To appreciate the savings in computation. consider how coordinatesare found when the basis is not orthogonal. For instance. the set B, = {VI. v2_ \lJ i" a


basis for R;. where

", ~[-:J '" = [ - ~ l ..d "= [ -;]

As can be seen. "j"3 i= O. and so 8 1 is not an onhogonal basis. Nexl. suppose we wi...r..10 express some \'ector \ in R3• say \ = [5, -5. -lJT, in term5 of BI , We musl sohethe (3 x 3) system: 0IV! +02v2 +OJ"3 = v, In matrix terms thecoordinatesol. {/2. ana

OJ are found by solving the equation

[ 1-I 2] [a,] [ 5]I 2 -2 0' = -5 .

-I I I (13 -2

(B) Gau~sian elimination. the solution is 01 = I. 02 = -2.03 = I.)Bycomf3st. if 82 = {WI. W2. w31 is an onhogonal basis for R3. it is easy to determine

°1.°2. and (13 SO that

\' = 01 WI +02\\'2 .L. 0J"3.

To find the coordinate (II in Eq. (4). we form the scalar product

\\'T \' = wT(a,., + (I,\\', + 0-",,1I I . - ~

= 0t (wj wI> + G2(" j W2) -l- 03(Wj W3)

= ol(wj WI).

The last equalil} follows because wj W2 = 0 and wjw, = O. Therefore, from abo\e

wTv{/I=-'-,

wjwl

Similarly.

Wry02 = ---t

\\'2 "'2and

WryGJ = ",T\\'J,

-

(Note: Since B2 is a basis. w;w, > O. I ::: i .::; 3.)

F\ \\\1'11: I Express the "ector \' in teons of the onhogonal basis B = {WI. "'2. "'3}. where

,,= [ ~n w, =[] " =[J ..d w, = [ -JSolution Beginning with the equation

\' = 01"" +02"'2 +03"'3·

Thus GI = 2. (/2

3W2 + w3·

-.e ~e wish'"lUSt solve

• G1. and


we form scalar products to obtain

wfv=al(w[wI). or 12= 6a1

wf v =a2(wf "'2). or 33 = 1IG2

wf v = Q3(wf "'3). or 66 = 6603.

= 3. and a) = I. Therefore, as can be verified directly, v = 2Wl +~

In general, let IV be a subspace of Rn • and let B = {WI. W2•.... wpl be an orIhogonal basis for IV. [f \' is any vector in IV, then ,. cun be expressed uniquely in theform

tennine

where

v = (/1"'1 + (/2"'2 + ... +Gpwp •

WTv, I·a, - -T-' :::: I ::: p.

Wi W,

(Sa)

(5b)

~bo'e

~

fllrcW.l \\ I ~

Constructing an Orthogonal Basis

The next theorem gives a procedure that can be used 10 genenue an onhogonal basis from any given basis. This procedure. called the Gram-Schmidt process, is quitepractical from a computational standpoim (although some care must be exercised whenprogramming Ihe procedure for the computer). Generaling an onhogonal basis is oflen the firsl step in solving problems in least-squares approximation; so Gram-Schmidtorthogonalization is of more Ihan theorelical iDleresl.

Gram-Schmidl Lei W be a p-dimen::.ional subspace of R". and let {WI, "'2.... , wplbe any basis for W. Then Ihe sel of'ectors {u\. U2 ..... up} is an onhogonal basis forW. where

UI =WI

ujW!U2 = "'2 - -T-U,

U, Ul

uj"'3 Ur",)U) = W) - -T-U1 - -Y---U2.

u1 UI u2 U2

and ~here. in general.

i-I r""' Uk Wi

Uj = W, - L -T-U,.,1,=1 Ul Uk

2::: i ::: p. 161

•The proof of Theorem I~ is somewha' technical. and we defer it 10 the end of this section.

In Eq. (6) we have explidt expre,ssions that can be used to generate an orthogonalset of vectors {UI. U2 ••.. , up} from a given set of linearly independent vectors. These


explicit expressions are especially useful if .... e have reason to implement the GramSchmidt process on a compUier.

Ho.... e'er, for hand calculations. it is not necessary to memorize formula (6). AU .... eneed to remember is the form or the general pattem of tbe Gram-Schmidt process. Inparticular. the Gram-Schmidt process smrts wilh a basis {WI. w2....• wp ) and generatesnew vectors Ul. U2. U3 ..•• according 10 the following patlern:

UI = WI

U2 = "'2+aUI

U3 = "'3 + bUI + CU2

u~ = "'~ + dUI + eu! + [U3

Ur = Wj + alul + a2U2 _ ... + ar_1U,_1

In this sequence. the scalars can be determined in a step-b)'-step fashion from the orthogonality conditions.

For instance. to determine the scalar a in the definition of U2 ..... e use the conditionufU2 = 0:

0= UfU2 = UfW2 +oufu\:

Therefore: a = -(ufw2)/(ufUl).

To determine the IWO scalars band c in the definition of U3. we use the two condilionsuf U3 = 0 and ur U3 = O. In particular.

0 - T _ T b T + T- u j U3 - u j "'3 + Ul Ul CUI U2

= uf "'3 + buf Uj (since uf U2 = 0 by Eq. (7»Therefore: b = -(uf"'3)/(U;ud·

Similarly.

0= ur U3 = uf"'3 + buI UI -r- cuI U2

= uIw, + CuI U2 (since uIUI = 0 by Eq. (7»Therefore: c = -(ufw3)/(uI U2).

The examples that follow illustrate the previous calculations.Finall). to use the Gram-Schmidt orthogonalization process to find an orthogonal

basis for \V. ....e need some basis for \V as a staning poim. In man) of the applicationsthat require an orthogonal basis for a subspace lV. it is relathely easy to produce thisinitial basis-we will gi\c some examples in a later section. Ghen a basis for lV, theGram-Schmidt process proceeds in a meehanical fashion using Eq. (6). (Note: It wasshown in Exercise 30 of Section 3.5 that every nonzero subspace of R" has a basis.Therefore. by Theorem 14. every nonzero subspace of R n has an orthogonal basis.)


111 =Wl

EX-i-\I-.IPLE (, Use the Gram-Schmidt onhogonalization process to generate an onhogonal basis forIV = Sp{Wi. W2. W3), where

Use the Gram-Schmidt process to construct an orthogonal basis for W.

Solution We define vectors 111 and 111 of the form

•u, ~ [ :] ,nd u, ~ [ _: lFor convenience in hand calculations. we can always eliminate fractional compo

nents in a set of orthogonal vectors. Specifically, if x and yare onhogonaL then so areax and)' for any scalar a:

If xTy = 0, then (ax}T)' = a(xTy} = O.

We will make use of this observation in the following example.

W, ~ [ ] W, ~[] oed w, ~ [J

112 = Wl +auj.

where the scalar a is found from the condition llf 112 = O. Now, 111 = [I. 1, 2]T and thus111112 is given by

u[ 112 = 111<W2 +aul) = ll1w2 +aufuj = -6+6{/.

Therefore. to have uf 112 = O. we need a = I. With a = I. 112 is given by 112 = "'2+111

[I. 3, -2f.In detaiL an orthogonal basis for W is B = {Ut, U2}, where

-EXAM!"l E 5 Let W be the subspace of R3 defined by \V ::;; Sp(Wj, W2}, where

w, ~ [ :] ,nd W, ~ [ -n6

..ond.ltiOD

J generaL.-.....

lb< or-

lb< Grnm-

iiUOlh

::'rtll.:e<.",

.;onaJlQtiOfu,

llLe this"'.theII was

a basis.'h.)

Solution First we should check to be sure that {WI. w2. W3} is a linearly independent set. Acalculation shows that the vectors are linearly independent. (Exercise 27 illustrates whathappens when the Gram-Schmidt algorithm is applied to a linearly dependent set.)

To generate an onhogonal basis lUI. U2. U3} from {WI. "'2, "'3}, we first set

UI =Wl

U2 ="'2 +aUj

U3 ="'3 + bUI + CU2·

222 Chapter 3 The Vector Space R~

With UI = LO. I. 2. IV. the orthogonality condition uj u2 = 0 leads to uj W2 +auj Ul =0, or 8 + 6a = 0, Therefore. a = -4/3 and hence

U2 = W2 - (4/3)u] = [0. -1/3. 1/3, _1/3]7.

Next, the conditions uj U3 = 0 and uI u, = 0 lead to

0= uj(w, +bUl +CU2) = 3+6b

0= uI (w, + bUI + CU2) = 0 + 0/3)c.

Therefore. b = -1/2 and c = O. Having the scalars band c,

U3 = W3 - 0/2)uI - (0)U2 = Ll.l/2.0. -1/2f.

For convenience, we can eliminate the fractional components in U2 and u, and obtain anorthogonal basis {v), V2, ",}, where

v, ~ [] v, = [J 'nd v, ~ [ J1 •

(Note: In Example 6. we could have also eliminated fractional components in the middleof the Gram-Schmidt process, That is. we could have redefined U2 to be the vectorU2 = [0, -I. 1, -If and then calculated u, with this new. redefined multiple of U2.)

As a final example. we use MATLAB to construct orthogonal bases.

~ [ '-"\PI r - Let A be the (3 x 5) matrixI

[12132]

A= 4 I 0 6 I

I I 2 4 5

Find an orthogonal basis for R(A) and an orthogonal basis for N(A).

Solutio" The MATLAB command orth (A) gives an orthonormal basis for the range of A.The command null (A) gives an orthonormal basis for the null space of A. The results are shown in Fig. 3.14. Observe that the basis for R(A) has three vectors; thatis, the dimension of R(A) is three or. equivalently, A has rank three. The basis forN(A) has two vectors; that is, the dimension of JV(A) is two, or equivalently, A has

nullity two. .'


We first show that the expression given in Eq. (6) is always defined and that the vectorsUI. U2 .... , up are all nonzero. To begin, UI is a nonzero vector since UI = WI. Thusuj III > O. and so we can define U2. Furthennore. we observe that U2 has the fonnu2 = w2 - hUl = w2 - blw]; so U2 is nonzero since it is a nomriviallinear combination

.au u ==


A.

1

•1

»orth{A)

ana.

211

1o2

J

••215

ob<.un an

•

0.38410.76820.5121

»null (A)

ana_-0.7528-0.2063-0.10690.5736

-0.2243

·0.11730.5908

·0.7983

-0.06900.1800

-0.9047-0.OU9

0.3772

·0.91580.24660.3170

(

;-, r )T r Uk W,UjUj = Uj Wi - L-r-u,

t=1 Uk Ut

middl(

'«torof u~.)

.ge of ATh< l<

:ors; thatbasis for~. A has

•

\ectorsI,," . Thus

the fonnmation

Figure 3.14 The ~IATLAB command arth IA) produces anorthonormal basis for the range of A. The command null (A) givesan orthononnal basis for the null space of A.

ofw) and Wl. Proceeding inductively. suppose thai ul. "1_ .... U,_I have been generatedby Eq. (6): and suppose that each Ut has (he form

Ut = Wt - CIWI - C1Wl - ... - Ci_IWt_l_

From this equation, each Uk is nonzero: and it follows thai Eq. (6) is a well-definedexpression lsince uf lit > 0 for 1 ~ k ::: (i - OJ. Finally, since each Uk in Eq. (6) is alinear combinalion of "'\. \\'2 •.••• "'t. we see thal Ui is a nontrivial linear combinationof "'\. w~ ..... w,: and therefore u, is nonzero.

All thai remains 10 be proved is Ihat the vectors generated by Eq. (6) are orthogonal.Clearl) urU2 = O. Proceeding inductivel) again. suppose thai uJ Ut = 0 for any j andk. where j t- k and I :::: j. k :::: i-I. From (6) we have

;-, (r )r Uk W, r= U J Wi - L -r- (Uj ud

i_I UkUt

r (uJ",) r= UJ

Wi - -r- (Uj Uj) =O.U j uJ

Thus Ui is orthogonal to uJ for I :::: j :::: i - 1. Having this result. we have shownthal {UI. U2 ..... up} is an orthogonal set of p nonzero veClors. So. by the corollary ofTheorem 13. the \ectoT'S Ul. U1 ..... up are an orthogonal basis for \V.


3.(, EXERCISES

12'~[!]In Exercises 13-18. use the Gram-5chmidt process togenerate an orthogonal set from the given linearly independent \,ectors.

11.' ~ [ nl.u,~ [:J u,~ [-~ l u,~[!]

2'U'~[ ~ l u,~[ -~ l u'~[:J

3. u, ~ [J u, ~ Ul u, ~ [ -;]

4. u, ~ [;] u, ~ Ul u, ~ [ -n

In Exercises 1--4. verify that {UI. U1. U3} is an orthogonalset for the given vectors.

I3tH;H~]

14. [ iH;H-~]In Exercise~ 5-8. tind values a. b. and c such that 15. [ 1 ] [0] [1 ](UI.U2.U3) IS an orthogonal set. 1.2. I

;. u, ~ [ : l u, ~ [J u, ~ [n 16. [ ; H;H~~]6 u, ~ [ ~ l u, ~ [-J u, ~ [ : ] 17. [ ; ] . [ : ] . [ ~ ;]

[

1 ] [ -2 ] [ 4 ] 0 0 17. UI = I . U! = ~ I . U3 = b I 0 0

1 a c 18. [ 1] [0] [0]8. u, ~ [J u, ~ [J u, ~ [ :J ~ . : . ~

in Exercises 9-12. express the given vector v in termsof the orthogonal basis B = (UI. "2. U3). where UI. U2.

and U3 are as in Exercise T.

9'~[l] 1O"~[;]

In Exercises [9 and 20. find a basis for the null space andthe range of the given matrix. Then use Gram-Schmidtto obtain orthogonal bases.

19. [ 1 -2 1 -; ]2 I 7 5

I -1 2-2

21. Argue that any set of four or more nonzero veCIQrsin R) cannOI be an orthogonal set.

2!. Let S = {Ut. "2, U31 bean orthogonal set of nonzerovectors in RJ. Define the (3 x 3) matrix A byA = [Ul. "2. U3]. Show thaI A is nonsingu!ar andATA = D. where D is a diagonal matrix. Calculatethe diagonal matrix D when A is created from theorthogonal vectOrs in Exercise I.

B. Let IV be a p-dimensional subspace of Rn . If v is avector in IV such Ihal \,7 w = 0 for every w in W,show that v = 8. [Hint: Consider w = v.1

2-1. The Callch)'-Schwa~ ineqlloliry. Let x and y be vectors in R~. Prove thai I"TYI .:5 IIx JI: YII. [Hint: Observe that IIx - cyU2 ~ 0 for any scalar c. If y #- 8.

,]Idl process tolinearly inde-

20. [1

-I

2

3 10 11

2 5 4

-I -I 1;]

3.7 Linear Transformations from R" to R- 225

letc = xT~'/yTy and expand (x _cy)T (x -Q') ~ O.Also treal the case y = 8.]

25. The triangle inequality. lei x and y be \ ectors inR~. Provethatllx+yl!::: Jlxll+IIYII. [Him: ExpandIIx + )"1[2 and use Exercise 24.)

26. LeI x and y be vectors in R~. Prove that IlIxl lIyUI ::: IIx - Yl!. [Him: For one part considerIIx + (y - x)11 and Exercise 25.]

27. If the hypotheses for Theorem 14 were altered so

that {Wi If:/ is linearly independent and (Wj Ir.1 islinearly dependent. use Exercise 23 to show thatEq. (6) yields up = 8.

28. Let B = (UI. U2 •.••• upl be an orthonormal basisfor a subspace W. Let \' be any vector in IV. wherev = 01111 +02U2 + ... + opup. Show that

lIvll2 = af +oi + ... +a~.

IJI 'p3Ceand~Schmidl

LINEAR TRANSFORMATIONS FROM Rn TO Rm

In lhis section we consider a special class of funclions. called linear transfomwlions.that map veclors to veClors. As we will presenlly obsen e. linear transfonnalions arisenaturally as a generalization of matrices. Moreover. linear uansfonnations have important applications in engineering science. the social sciences. and various branches ofmathemalics.

The notation for linear transformations follows the usual notation for functions. IfV is a subspace of R" and \V is a subspace of R"'. then the notation

F:V __ W

\loill denote a function, F. whose domain is the subspace V and whose range is containedin W. Furthennore. for \' in V we write

w = F(v)

10 indicate that F maps \' to w. To illustrate. let F: R3 -+ R2 be defined by

[

X, - x, ]F(x)= xl+X3 •

where

x~ [:J

226 Chapter 3 The Veclor Space R II

In this case if, for example. v is the vecwr

v~ [~ lthen F(\» = w, where

w ~ [ -:]

In earlier seclions we have seen that an (m x n) matrix A detemlines a functionfrom R" to R"'. Specifically for x in R". the formula

T(x) = Ax

defines a function T: W --+ R"'. To illustrate. let A be the (3 x 2) matrix

[1-I]A = ~ ~ .

In this case Eq. (I) defines a function T: R2 --+ RJ• and the formula for T is

T(,) = T ([ :: ]) ~ [~ -~] [ :: ] ~ [ X'2~:V2 ]:

3 I 3Xl + X2

for instance.

T ([ : ]) ~ [; l

II

,

DEFI'lTlO' 5

Returning to the general case in which A is an (m x n) matrix. note that the functionT defined by Eq. (I) salisfies the following linearity properties:

T(v + w) = A(\' + w) = Av + Aw = T(v) + T(w)

T(ev) = A(e\') = eAv = eT(v).

\\here v and w are any vectors in R" and e is an arbitrary scalar. We next define alinear transformation to be a function that satisfies the two linearity properties given inEq. (2).

Let \I and \V be subspaces of Wand R"', respectively, and let T be a functionfrom \' to lV, T: V --+ IV. We say that T is a linear transformation if for all uand \' in V and for all scalars Q

T(u + v) = T(u) + Tv)

andT(au) = aT(u). (~)

j1e~ a funclion

(I


II is apparent from Eq. (2) that the funclion T defined in Eq. (I) by matrix muhiplication is a linear transformation. Conversely, if T: R" ... R'" is a linear transformation.then (see Theorem 15 on page 232) there is an (m x 11) matrix A such that T is definedby Eq. (I). Thus linear transformations from R" to R'" are precisely thosc functions thatcan be defined by matrix multiplication as in Eq. (I). The situation is not so simple forlinear transformations on arbitrary vector spaces or even for linear transformations onsubspaces of R". Thus the concept of a linear transformation is a convenient and usefulgeneralization to arbitrary subspaces of matrix functions defined as in Eq. (I).

Examples of Linear Transformations

Most of the familiar functions from the reals to the reals are not linear transformations.For example. none of the functions

f(x)=x+l. 8(X)=X 2• h(x) = sin.r. k(.t)=e~

is a linear transformation. Indeed, it will follow from the exercises that a functionf: R -l- R is a linear transformation if and only if f is defined by f(x) = ax for somescalara.

We now give several examples to illustrate the use of Definition 8 in verifyingwhether a function is or is not a linear transformation.

E\..'I,\\PLL I Lei F: R3 _ R2 be the function defined by

[

X, - x, ]F(x)= X2+ X3 [." ]where x = X2 '

."

function

"C:ll define alIe>. gi\en in

1.Jlk:t1on)f all u

Determine whether F is a linear transformation.

Solution We must determine whether the 1\\'0 linearity propenies in Eq. (3) are satisfied by F.Thus let u and \' be in R3.

u ~ [ ::] and "~ [ : ] .143 (/3

and let c be a scalar. Then

[

u, - u, ]u.J,.\,= U2+l/2

"3 T V3

Therefore. from the rule defining F,

[

(u, + u,J - (u, + ",) ]F(u + v) =

(142 + (/2) + (113 + (/3)

~ [ u, - u, ]+ [ u, - u, ]142 + 143 (/2 + (/3

= F(u) + F(v).

228 Chapter 3 The "edor Space R"

SimiLarly,

[cu, -cu, J [u, -u, JF(cu) = = c = cF(u).CII2 + CU) 112 +II)

so F is a linear lransfonnation.Note that F can also be defined as F(x) = Ax. "here A is the (2 x 3) matrix

[1 -I 0 JA= . •o 1 1

l .h"ex~ [:: lE.\.\.\\PLE 2

Solution

Define H: R2 -+ R2 by

[

xl-x2+1H(x) =

3X2

Determine 'Whether H is a linear transformation.

Let u and v be in R2:

Then

while

[ u'J ["'Ju= 112 and v= L'2 .

[

(ul-t'Il-(1I2+t'l)+1 JH(u~v)= .

3(112 ~ L'l)

[ u, - '" + 1 J ["' - '" + 1 JH(u) + H(",) = +3112 3V2

= [ (/ll + L'l) - (U2 + L'!) + 2 J.3(112 + l'2)

Thus v.e see that H(u +v) ¥= H(u) +H(v). Therefore. H is not a linearlransformation.Although. it is nOI necessary. it can also be \erified easily thai if C ¥= I. then H(cu) rcHeu). •

E.X+\\lPI E 3 Let \V be a subspace of Rn such Ihatdim{W) = p.and let 5 = (WI. ""2 ..... Wpl beanorthonormal basis for IV. Define T: R~ ....... IV by

T(v) = {\ TWd""t + (\,T w2)W2 -+- .•• _ (\,T ""p)""p' {4,

Pro\e that T is a linear transformation.

Solution If u and \' are in R~. then

T(u + \') = [(u + vlwiJwr + [(u + V)T w21w2 +... + [(u + v)T wp]w/!

= HUT + \'T)\\'11wl + [CUT +VT)W2JW2 + ... + [CUT + vT)\\'p]Wp

= (u T WI)""I + (uT w2)\\'2 + + (u Twp)Wp

+(\,T wI)\\ I + (\,T W2)W2 + -.,.. (vTWp)W p

= T(u) + T{\).

ilj maui"

•

3.7 Linear Transformations from R" 10 R'" 229

II can be shown similarly that Tku) =:: cT(u) for each scalar c, SO T is a linearuansformation. •

The \'ector T(\'> defined by Eq. (.J) is called the onhogonal projection of v ontotV and will be considered funher in Sections 3.8 and 3.9. As a specific illustration ofExample 3. let tV be the subspace of R) consisting of all vectors of the form

[

X, ]

X= ~

Thus tV is the .\)'-plane. and the set tel. e21 is an orthonormal basis for tV. For x in R),

[." ]x = :: •

the formula in Eq. (.J) yields

T(x) = (xTedel + (x TC2)C2 =Xle, + X2C2·

Thus.

rex) ~ [ ~ lThis transformation is illustrated geomeuically by Fig. 3.15.

,.(.l"\.X2·.l"))

y

ormation.H(cu) =::

•14',}bean

x

x

T(x)

,,,,~

(.l! • .l"2' 0)

J" ,

FigUff 3.15 Onhogonal projection onto the .l)·-plane

E\ \\\PLE 4- Let \V be a subspace of R~. and lei a be a scalar. Define T: lV --,. \V by T(w) = aw.Demonstrate that T is a linear transformation.

Solution If \' and 14' are in tV. then

T(,· + w) = a(v + 14') = a\' + a 14' = T(v) + T(w).


Like\l,'ise. if e is a scalar. then

T(ew) = a(ew) = c(ow) = cT(w).

It foUows that T is a linear transfonnation. •The linear transfonnation defined in Example 4 is called a dilation when 0 > I arx.

a COlllroction when 0 < (/ < I. These cases are illustrated geometrically in Fig. 3.16.

a > I. dilalion

(,)

Figun 3.16 Dilations and conlfaCtions

o < a < I. conlfaClion

(b)

The mapping I: \V --+ \V defined by I(w) = w is the special case of Example 4 inwhich a = I. The linear transfonnation I is called the identity trans/onnation.

t:\.\ \\1111:> Let \V be a subspace of R~. and let (J be the zero vector in R"'. Define T: IV --+ R'" byT(w) = 9 for each w in IV. Show that T is a linear transfonnation.

Solutio" Let v and w be vectors in IV. and let c be a scalar. Then

T(v +w) = (J = (J + 9 = T(v) +T(w)

and

T(cv) = 9 = c9 = eT(v).

so T is a linear transfonnation. •The linear transfonnation T defined in Example 5 is called the :uo trallS/ormatioll.Later in this section we will consider other examples when",e study a particular class

of linear tralLSfonnations from R" to R2. For the present. we tum to further propertiesof linear transfonnations.

The Matrix of a Transformation

Let V and IV be subspaces. and let T: V __ IV be a linear transfonnation. If u and \'are vectors in V and if a and b are scalars. then the linearity properties (3) yield

T(au + bv) = T(au) + T(b\') = aT(u) + bT(,-). "

3.7 Linear Transformations from R" to Rill 231

Inducti\ely we can extend Eq. (5) to any finite subset of V. That is. if VI. \'2 ..... Vr areveclors in V and if CI. C! •••.. Cr are scalars. then

•lena> land

Fig. 3.16.

nCI\'l + ell'! +... + crvr ) = Cl T(l'l) +clT(l'l) +... +Cr T(vr ).

The following example illustrates an application of Eq. (6).

E'.\\\PI r b Let \V be the subspace of R3 defined by

[X,+2X,]

W = Ix: x = x! . X2 and X3 any real numbers}.

x,Then {WI. w:d is a basis for W. where

w, = [i] 'lld w, = [ ~ l

,6,

ple..J in'n

- R·b~

Suppose that T: W - Rl is a linearlransformation such Ihat T(wd = ul and T(wl) =Ul. "'here

u, ~ [ :] 'lld u, = [ _: llei the vector W be: gi\ en by

w ~ [: ]

Show that" is in 1V. express w as a linear combinalion of WI and w2. and use Eq. (6)10 determine Trw).

Solution Ii folio", s from the description of 1V that W is in W. Funhennore. it is easy to see that

W= 3\\"1 - 2"'2.

Example 6 illustrates that Ihe action of a linear transfonnation T on a subspace 1Vis complelely determined once the action of T on a basis for W is known. Our nextexample provides yet another illuSiration of this fact.

B) Eq. (6)•

Trw) = 3T(wd - 2T(W2) = 3u\ - 2U2 = 3 [ : ] - 2 [ _: ].

•Innation.vhrcl'b~

>~nie)

u and,

~

Thus.

T,w) ~ [a •


Let T: R3 ---+ R2 be a linear transformation such thatI\A\\PLf -

TC',) ~ [ : lFor an arbitrary vector x in R3 •

TC',) ~ [ -: l 'nd

[

X, ]

x = :: '

T(,,) ~ [ a

give a formula for T(x).

Solution The vector x can be written in the form

x = Xlet + X2e2 + X3e3,

so by Eq. (6).

T(x) = Xl T(ed +X2T(e2) + X3T(e3).

Thus,

[1] [ -1 ] [2] [x, -xd 2x, ]T(X)=Xl +X2 +x3 = - .2 I 3 2XI+X2+3x3

(7'

•Continuing with the notation of the preceding example, leI A be the (2 x 3) matrix

with columns T(el). T(e2), T(e): thus.

A = [T(ed. T(e2), T(e))] = [ ~ -1

1 a-i

It is an immediate consequence ofEq. (7) and Theorem 5 of Section 1.5 that T(x) = Ax.Thus Example 7 illustrates the following theorem.

111 EORf\\ 15 Let T: Rn ---+ Rm be a linear transformation, and let el. e2, .... en be the unit vectors inRn. If A is the (m x n) matrix defined by

A = [T(el)' T(e2),"" T(en )],

then T(x) = Ax for all x in Rn.

Proof [f x is a vector in Rn,

[

X, ]x,x = .

.;nthen x can be expressed in the form

x = Xlel + X2e2 +.. + xnen.

3.7 Linear Transformations from RII to R'" 233

It now follows from Eq. (6) that

rex) = Xl T(el) +X2T(e2) -r- ... + x"T(e,,). ",if A = [T(ed. T(e!) . .... T(e,,)].lhen by Theorem 5 of Section 1.5. the right-hand sideof Eq. (8) is simply Ax. Thus Eq. (8) is equivaJenllo rex) = Ax. •

E,\.\\\Pl L~ Let T: R2 -+ R3 be the linear transformation defined by the formula

T ([ x, ]) = [ _:' + 2x, ]X2 1+-"2·

2.\'1 - Xl

Find a matrix A such that T(x) = Ax for each x in R!.

Solution By Theorem 15. A is the (3 x 2) matrix

A = [T(el). T(e!)).

,nd T(e,l = [ JII is an easy calculation that

T(e,) = [ - ~ ]•Therefore,

...A=[-: :]

2 -1

One can easily verify that rex) = Ax for each x in R 2, •:\'u11 Space and Range

Associated with a linear transformation. T. are two imponant and useful subspaces calledthe null space and the range of T. These are defined as follows.

D-I:::fl" ITIOr-.. t) Let V and W be subspaces. and let T: V --+ \V be a linear transformation.The null space of T, denoted by N(T). is the subset of V given by

..'\'(T) = Iy: ,. is in V and TCY} = O}.

The range of T, denoted by 'R(T). is the subset of \V defined by

'R(T) = {w: w is in Wand w = T(y} for some " in V}.

That N(T) and R(T) are subspaces will be proved in (he more general COnlex[ ofChapter 5. If T maps RII into R"'. then by Theorem 15 there exists an (m x n) matri'\


A such that T(x) = Ax. In this case il is clear that the null space of T is the null spa,~

of A and the range of T coincides with the range of A.As with matrices. the dimension of the null space of a linear transfonnation T I'

called the lIullity of T. and the dimension of the range of T is called the rank of T. I:T is defined by matrix multiplication. T(x) = Ax. then the transformation T and Ill.:matrix A have the same nullity and the same rank. Moreo\"er. if T: Rn __ Riff. then A.is an (m x n) matrix. so it follows from the remark in Seclion 3.5 that

rank(T) +nullily(T) = /1.

Formula (9) will be pro\'ed in a more general setting in Chapter 5.The next twO examples illustrate the use of the matrix of T 10 determine the null

space and the range of T.

I \ \ \\PU: 9 Let F be the linear transformation gken in Example I. F: R3 -+ R~. Describe the nullspace and Ihe range of F. and delennine the nullity and the rank of F.

Solution It follows from Theorem 15 that F(x) = Ax. where A is the (2 x 3) matri:<

[

1 -1A = (F(e,). F(e2). F(e)) = 0 I a

Thus the null space and the range of F coincide. respectivel). wilh the null space and therange of A. The null space of A is determined by backsol\'ing the homogeneous systemAx = 8. where x is in R3:

[

X, ]x = .~~ .

."This gives

..\'(F) = ;\"(A) = Ix: Xl = -X3 and X2 = -x,l.

Using the techniques of Section 3.4. we can easily see that the \ector

u = [ =: ]is a basis for .\'(Fl, so F has nullilY I. By Eq. (9),

rank(F) = II - nullity(F) = 3 - 1 = 2.

Thus 'R( F) is a two-dimensional subspace of R~. and hence 'R(F) = R~.

Alternatively. note that Ihe system of equations Ax = b has a solution for each b inR~. so R(F) = R(A) = R". •

['"","flU: 10 Let T: R1-- R3 be Ihe linear transformation ghen in Example 8. Describe the null

space and the range of T, and determine the nullity and the rank of T.

(10)

Ie null space

'I'lUtion T i~

m!ofT.1fD T and theR" . then A

the null

the null

and the) 'tem

3.7 Linear Transformations from R It to R- 235

SOlllliO/I In E:<.ample 8 it was shown that T(x) = Ax. where A is the (3 x 2) matrix

A~[~: ~].2 -I

Ifb is the (3 x I) vector.

b~ [:Jthen the augmented matrix (A I bj for the linear system Ax = b is row equivalent to

[

I 0 (1/3)b, - (2/3)b, ]

o 1 (1/3)b, + (l/3)b,

o 0 (-lj3)b l + (5j3)b2 +b)

Therefore. T(x) =Ax =b can be solved if and only if 0 = (-I j3)b l + (5j3)b2 + b3•

The range of T can thus be described as

'R(T) ~ 'R(A)

= fb: b = [ :: ]. b l and b2 any real numbersl.

(1/3)b, - (5/3)b,

A basis for 'R(T) is lUI. U2} where

u, ~ [ ~ ] and U, ~ [ ~]1/3 -5/3

for all v in R2. Transfonnations that satisfy Eq. (II) are called orthogonal transforma.tiolls. We begin by giving some examples of orthogonal transformations.

Thus T has rank 2. and by Eq. (9).

nullity(T) = n - rank(T) = 2 - 2 = O.

II follows that T has null space {9}. Altemathely. it is clear from matrix (10). \lo'ithb = 9. that the homogeneous system of equations Ax = 9 ha... only the trivial solution.Therefore...qT) = .A'(A) = 19}. •

Orthogonal Transformations on R2 (Optional)

" is often infonnati\'e and useful to view linear transfonnations on either R2 or R3 froma geometric point of view. To illustrate this general notion. the remainder of this sectionis de\oted to determining those linear tmnsfonnations T: R2 -+ R2 that preserve thelength of a vector: that is. we are inte,rested in linear transformations T such thatb ..

•n

IIT(v)1I ~ IIvll (II)


-i -EX·MIPl£ 11 Let 8 be a fixed angle. and let T: R" -+ R" be the linear transformation defined ~T(v) = Av. where A is the (2 x 2) matrix

_ [co,O -'inO]A_ .sin (} cos 8

Give a geometric interpretation of T. and show that T is an orthogonallransfonnation.

Solution Suppose that v and T(v) are given by

v ~ [ :] "d T(v) ~ [ ~ ] .

Then rc\') = A \', so

--(J.l

{ll[~ ]~ [CO,O -'inO] [a] ~ ["OOSO -b"nO ]sinO cos8 b asm8+bcosO

We proceed now to show thai T(v) is obtained geometrically by rotating the vector,through the angle 8. To see this, let ¢ be the angle between" and the positive x-axis(see Fig. 3.17), and set r = II vII. Then the coordinates (/ and b can be wrilten as

a=rcos¢. b=rsin¢.

Making the substitution (13) for a and b in (12) yields

c = ,. cos rP cos 8 - r sin ¢ sin 8 = r cos(¢! + 8)

nndd = r cos¢sin8 + r sin¢cos8 = r sin(¢ + 8).

Therefore, c and d are the coordinates of the point obtained by rotating the point (a. b)through the angle O. Clearly then, IIT(v)1I = IIvll. and T is an orthogonal lineartransformation. _

y

c "x

d(c. d)

""'",

",, (a. b)

Figure 3.17 Rotation through the angle 8

The linear transformation T defined in Example II is called a rotation. Thus if Ais a (2 x 2) matrix.

A~ [a -b].b "

3.7 Linear Transformations from R" to R" 237

1 defined b~ where a~ + b1 = 1. then Ihe linear transfonnalion T(v) = Av is the rOlalion throughthe angle 8. 0;5 8 < 2JT. wherecos8 = a and sin8 = b.

fXi\<\IIJU: 12 Define T: R~ _ R1 by T(\') = A". where

Ormation.

[

~1/2

A ~ -./3/2./3/2 ] .-1/2

Give a geometric imerpretation of T.

Solution Since cos(4iT13) = - t/2 and sin(4rr13) = -fl/2. T is the rotation through the angle4JT/J· •

EM\\PI t: 13 Lei T: R~ ...... R~ be defined by T(\') = Av. where A is the (2 x 2) matrix

Figun3.18Reflection about a line

./3;2 ].-1'2

"08 ]-cos8 .

1/2

./3/2A~[

A~[CO'8sin 8

Now leI I be a line in the plane Ihat passes through the origin. and leI " be a vectorin the plane. If we define T(v) to be the symmetric image of v relntive to J (see Fig.3.18). lhen clearly T preserves the length of v'. It can be shown Ihat T is mulliplicationby the matrix

where (1/2)8 is the angle between I and the positive x-axis. Any such lran!>formation iscalled a rejlecriolJ. Note that a reflection T is also an orthogonal linear lransformation.

\-f

,,,,,

,,,,,,,,,

T(v)

abli"""

•

\e.,:lor \

-e :r·a"l

.'

Ghe a geometric imerpretalion of T.

Solution Sincecos(iT/J) = 1/2 and sin(rr 3) = ./3/2. T is the reflection about the line I. whereI is the line thai passes through the origin at an angle of 30 degrees. _

The next theorem give, a characterization of orthogonal transformations on R1. Aconsequence of Ihis theorem will be that every orthogonal tranSformluion is either arotation or a reflection.

THl:OI~l:\\ Ie:, Let T: R~ -+ R~ be a linear transformation. Then T is an orthogonal transformation ifand only if IIT(edll = UT(e2 HI = 1 and T(et> is perpendicularto T(e1)·

itA Proof If T is an orthogonal tmosformalion. then IIT(v)lI = IIvl: for every vector \' in R2.In particular. IlT(elHI := I,edl := I. and similarly IIT(e~)1I = I. Set u\ = T(ell.U1 = T(e~). and \. = rl. IV = el + e~. Then

2 = ","' ~ IIT('lIl' ~ IIT(o, +o,lll' ~ llreo,) +T(o,lll'.


Thus.

2 = lIuI + u211 2

= (Uj + U2)T (Uj + U2)

= (u; + uiHul + U2)

-'+'+'+'- Ul Uj Uj U2 U1 Ul U2 U2

= !lUI 11 2 + 2ur U2 + lIu2112

=2+2U;U2'

It follows that uf U2 = 0, so Ul is perpendicular 10 U2.The proof of the converse is Exercise 47. •We can now use Theorem 16 to give a geometric description for any orthogonal

linear transformation. T, on R2. Firsl, suppose that T(ed = Ul and T(e2) =U2. If

u, ~ [: lthen 1 = lIullI" = a" + b2

• Since lIu211 = I and U2 is perpendicular 10 Ul. there are twochoices for U2 (see Fig. 3.19): either

u, ~ [ -:] m u, ~ [ _: ]

In either case. it follows from Theorem 15 thai T is defined by T(v) = Av, where A isthe (2 x 2) matrix A = [Ul. u21. Thus if

u, ~ [ -: l

L

.,T

c>Tth,n

A ~ [ :

y

-: lMod

, ,,,,,,,,/

-----" (0. b)

(-b,o)

u,<,

,,,,

•

c

'-'-"

Figure 3.19 Choices for U2

(b. -0)

3.7 Linear Transformations from Hit to Rm 239

so T is a rotation. If

u'~[_~lth'"

A~[: -~l

•

and T is a reflection. In either case note that ATA = f, so AT = A -I (see Exe,rcise 48).An (II x II) real matrix with the propeny that ATA = I is called an orthogonal matrix.Thus we have shown mal an onhogonal transformation on R1 is defined by

<

IfT(x) = Ax.

where A is an orthogonal matrix.

2. !kline T: R" -+ R1 by T(x) = Ax. where

_[ 1-1]A_ .-3 3

Find each of the following.

aj T ([ ~]) bj T ([ : ])

,j T ([ ~]) d) T ([ -~ ])

aj T ([ ~]) bj T ([ : ])

,j T ([ ~]) dj T ([ ~ ])

b~[ -a

b~[ -a

Which of the following vectors are in the null spaceofT?

oj [n bj [ -n,j [ ;] dj [ -:/2 ]

I -1/2

4. lei T: R" -+ R1 be the fUJ1Clion defined in Exercise I. Find x in R" such thai T(x) = b. where

5. Let T: R" _ R2 be lhe function given in Exercise I. Show lhal for each b in R2, there is an x inR2 such that T(x) = b.

6. Let T be the linear transformation gi\en in Exercise 2. Find x in R2 such mat T(x) = b. where

7. Let T be the linear transformalion given in Exercise 2. Show mal mere is no x in R2 such that

2x, - 3x, ].-Xl + X2

EXERCISES3


I. Define T: R2 ...... R1by

T ([ ;; ]) ~ [

3. L=l T: R3 -- R" be the linear lransfonnation defined by

([ X']) [ ]Xl +2r1 + 4-':3T .f> = .x: 2\"1 +3X1 + 5X3

A

""


T(x) = b for

b ~ [ : ]

16. F: R2 ...... R defined by

F ([ :: ]) ~ 2x, + lx,

t"..

.J T

In Exercises 8-17. detennine whether the function F isa linear transfonnation.

8. F: R2 ...... R2 defined by

F([ :: ]) ~ [ :: ]10. F: R2 ...... R2 defined by

F([::]) ~[x,~x,]

II. F: R2 -+ R2 defined by

F ([ :: ]) ~ [ x~:, ]12. F: R3 -+ R2 defined by

F ([ :: ]) ~ [

-

T

T

F ([ :: ]) ~ Ix,1 + Ix,l


,~[ ~ l

17. F: R2 ...... R defined by

18. Let W be the subspace of R 3 defined by

IV ~ I" ~ [;:] x, ~ x) ~ 0).

Find an orthononnal basis for W. and use Eq, (.t)of Example 3 to give a formula for the orthogonalprojection T: RJ -+ W; that is. determine T(v) forarbitrary \' in R3;

Give a geometric interpretation of IV, v. and T (\').

19. Let T: R2 -+ R3 be a linear transformation suchthat T(el) = U\ and T(e2) = U2, where

", ~ U] ood ", ~ [ ~ ]

2x, - ." ]

Xl + 3x2

Xl - X2 + XJ ]

-Xl + 3X2 - 2xJF ([ :: ]) ~ [

9. F: R2 -+ R2 defined by

F ([ :: ]) ~ [ ~ ]

13. F: R3 -+ R:' defined by

F([::])14. F: R:' -+ RJ defined by

F ([ :: ]) ~ [

15. F: R:' -+ R3 defined by

~ [:: ]x, - x, ]

-XI + X2

x,

,j T ([ : ])

b)T([_~])

,jT([:])20. Let T: R2 -+ R2 be a linear transfonnation such

that T(vl) = Ut and T(\'2) = U2' where

"~[~ 1 "~[-: 1", ~ [ ~ ]. "d ", ~ [ : ].

(

I

--

~OJ

'" ,J:onal

T'\ ior

T •

~u.:h

Find each of the roIlO'o\'ing.

aJ T ([ : ])

b)T([_~])

oj T ([ ~ ])

In E'l:crcises 21-24. the action of a linear trnnsformationT on a basis for either Rl or RJ is gi\en. In each casey,.e Eq. (6) to dcrh'e a formula for T.

'I.T([: ])=[ _~] ~d

T ([ _: ]) ~ [ ~ ]

11. T ([ : ]) ~ [ :] ~d

T ([ -: ]) ~ [ nD T ([ ~ ]) = [ ~ l

T ([ -:]) ~ [a

T([ -l]) ~[nU T ([ ~ ]) = [ -:]


T ([ -:]) ~ [JT([ -i])~[n

Tn Exercises 25-30. a linear transformation T is given.In each case find a matrix A ~uch that T (x) = Ax. Alsode!>Cribe the null space and the range of T and give thenmk and the nullity of T.

25. T: Rl ....... R~ defined by

T ([ x, ]) ~ [ ;" + 3x, ]Xl _1"1 + Xl

26. T: Rl --. R) defined by

T ([ :: ]) ~ [ ::::: ]

27. T: R" - R defined by

T ([:: ]) ~ 3" +'x,28. T: R3 -.. RJ defined by

([X']) [X' +x, ]TX2=X)

x~ Xl

29. T: RJ .....,. R2 defined by

T([::]) =[ :;=::]30. T: RJ ...... R ddined b)'

T ([ :J) = 2<, -x,+4x;

31. Let a be a real number. and define. j: R _ R byf(x) = ax for each.1" in R. Show lhal f is a lineartransformation.

242 Chapter 3 The Vector Space RR

v ~ [ : ]

44. Let a be a real number and define T: R" -+ Rn byT(x) = ax (see Example 4). Detennine the matrixA such that T (x) = Ax for each x in Rn .

a) 9 = ]I" 12 b) 9 = Jr13 c) 9 = 27r 1346. Let T: R'- -+ R2 be the reflection with respect to

the line I. In each of the following cases, exhibit

Exercises 45-49 are based on the optional materia1.

45. Let T: R2 -+ R2 be a rotation through the angle 9.In each of the following cases, exhibit the matrix forT. Also represent v and T(v) geometrically, where

•

•

~.,

-Xl+ 2x2- 4x3 ]

2XI+5x2+ X3

c([::]) ~ [~:::~::]

F([::J)~["d

a) By Exercise 39. G 0 F: R3 -4 R3 is a lineartransformation. Give a formula for G 0 F.

b) Find matrices A. B. and C such that F(x) =Ax, G(x) = Bx, and [G 0 FJ(x) = Cx.

c) Verify that C = BA.

41. Let B be an (m x n) matrix, and let T: R" -4 R'"be defined by T(x) = Bx for each x in Rn. If Ais the matrix for T given by Theorem 15, show thatA = B.

42. Let F: Rn -+ RP and G: RP -+ R'" be linear transformations. and assume that Theorem 15 yields matrices A and B. respectively. for F and G. Show thatthe matrix for the composition G 0 F (sec Exercise39) is BA. [Him: Show that (G 0 F)(x) = BAx forx in Rn and then apply Exercise 41.]

43. Let I: R" -+ Rn be the identity transformation. Determine the matrix A such that / (x) = Ax for eachx in Rn .

defining by [G 0 Fj(u) = G(F(u» for each u in V,is a linear transformation.

40. Let F: R3 -+ R" and G: R" -+ R3 be linear transformations defined by

(["']) [ ]2xI - 3_~2 + X3

F :: = 4Xt + 2X2 - 5X3

and

(["DI _ XI +4X2 +2X3c:: ~ [-2x, +]" +3XJa) Give a formula for the linear transformation

F + G (see Exercise 35).b) Find matrices A, B. and C such that F(x) =

Ax, G(x) = Bx. and (F +G)(x) = Cx.

c) Verify that C = A + B.

37, Let V and IV be subspaces, and let T: V -+ IV bea linear transformation. If a is a scalar. define aT:V -+ IV by [aTl(v) = a[T(v)] for each v in V.Show that (./ T is a linear transformation.

38. Let T: RJ -+ R2 be the linear transformation defined in Exercise 29. The linear transformation [3T]:R3 -+ R2 is defined in Exercise 37.

a) Give a formula for the transformation 3T.

b) Find matrices A and B such that rex) = Axand [3T](x) = Bx.

c) Verify thai B = 3A.

39, Let U, V. and IV be subspaces. and let F: V -+ Vand G: V -+ IV be linear transformations. Provethat the composition G 0 F: U -+ IV of F and G,

32. Let T: R ---+ R be a linear transformation. and suppose that T( I) = a. Show that T (x) = ax for eachx in R.

33. Let T: R2 -"" R" be the function that maps eachpoint in R" to its reflection with respect to the~-axis. Give a formula for T and show that T isa linear transformation.

34. Let T: R" -+ R" be the function that maps eachpoint in R2 to its reflection with respect to the line)' = x. Give a formula for T and show that T is alinear transformation.

35. Let V and IV be subspaces, and let F: V -+ IVand G: V -+ IV be linear transformations. DefineF + G: V -+ IV by [F + G](v) = F(v) + G(\·)for each v in V. Pro\'e that F + G is a lineartransformation.

36, Let F: R3 -"" R" and G: R3 -+ R" be defined by

3.8 Least-Squares Solutions to Inconsistent S)·stems. with Applications to Data Fitting 243

e.lC'h u in U.

near lrans_

"" ]x,

~

the malrix for T. Also re,present el. <:2. T{el). andT{e2) geometrically.a) I is the x-axis. b) 1 is the y-axis.c) I is the line with equation.'" = x.d) I is the line with equation y = .J3x.

47. Lei T: R2 -+ R2 be a linear transformation thatsatisfies the conditions of Theorem 16, Show thatT is orthogonal. (Him: If v = [Il, bV, then\' = ael + be1. Now use Eq. (6).J

48. Let T: R2 -+ R2 be an orthogonal linear transfonnation. and let A be the corresp::mding (2 x 2)malrix. Show that ATA = f. [Him: Use Theorem16.J

49, Let A = [AI. A1J be a (2 x 2) matrix such thatATA = f, and define T: R2 _ R" by T(x) =Ax.

a) Show that fA I, A11 is an orthononnal Set.

b) Use Theorem 16 to show that T is an orthogonal transformation.

linearF.

XI =,

0..'fore~

,

LEAST-SQUARES SOLUTIONS TO INCONSISTENTSYSTEMS, WITH APPLICATIONS TO DATA FITTING

When faced wilh solving a linear syslem of the form Ax = b. our procedure has been todescribe all solutions iflhe system is consistent but merely 10 say "there are no solutions"if the system is inconsistent. We now want 10 go a step further with regard 10 inconsistentsystems. If a given linear system Ax = b has no solution. then .... e .....ould like to dothe next best thing-find a vector x· such Ihat the residual "ector. r = Ax· - b. is assmall as possible. In terms of praclical applications. we shall see that any technique forminimizing Ihe residual "ector can also be used to find best least-squares fits to data.

A common source of inconsistent syslems are ol'f!rdf!ttrmined s)'sttms (that is.systems wilh more equations than unkn()\\'ns). The syslem that fo11o....s is an e>;ample ofan o\'erdetennined system:

XI + 4X2 = -2

XI +2(1 = 6

21"1 + 3X1 = I.

OH=rdelermined systems are often inconsistent. and the preceding example is no exception. Given that the above system has no solution. a reasonable goal is to find values forXl and X2 that come as close as possible 10 satisfying all three equations. ~ethods forachieving that goal are the subjecl of this section.

Least·Squares Solutions to Ax = b

Consider the linear system Ax = b where A is (m x 1/). If x is a vector in Rn, then the

vector r = Ax - b is called a residual ~·f!ClOr. A \'ector x' in R~ Ihat yields Ihe smalleslpossible residual ,ector is called a least-squares solution to Ax = b. More precisely. x'is a least-squares sofutioll to Ax = b if

IIAx· - bll ::: IIAx - bl!. for all x in Rn•

(If Ax = b happens to be consistent. then a least~squares solution x' is also a solutionin the usual sense since IIAx' - bll = 0.)

244 Chapter 3 The Vector Space RM

The special case of an inconsistent (3 x 2) system Ax = b suggests how we cancalculate least-squares solutions. In particular. consider Fig. 3.20 which illustrates avector b that is not in R(A); that is. Ax = b is inconsistent.

b

~ .~1/ I y y,

R(A)

.,Figure 3.20 )" is the closest vector in R(A) to b

Let the vector y. in R(A) be the closest vector in R(A) to b: that is

II,Y~ - bll ::; IIY - bl!. for all y in R(A).

Geometry suggests (see Fig. 3.20) that the vector y. - b is orthogonal to every vectorin R(A). Since the columns of A form a spanning set for R(A). this orthogonalitycondition leads to

A;(,Y· - b) =0

ArC,Y- - b) = 0

or. in matrix-vector terms.

AT(,Y--b)=(}.

Since,Y° = Ax~ for some x- in R2• the preceding equation becomes AT (Ax~ - b) = 8.m

ATAx· = Arb.

Thus, the geometry of the (3 x 2) system. as illustrated in Fig. 3.20, suggests thatwe can find least-squares solutions by solving the associated system (I):

ATAx = Arb.

As the following theorem asserts, this solution procedure is indeed valid.

(a) The associated system ATAx = ATb is always consistent.

(b) The least-squares solutious of Ax = bare prel.:isely the solutions of ATAx =ATb.

~ TttEOREM 17 Consider the (m x II) system Ax = b.

(c) The least-squares solution is unique if and only if A has rank n. •

3.8 Least·Squares Solutions to Inconsistent S)'stems, with Applications to Data "'ltting 245

Forming ATA and Arb. we obtain

E\.\\\Pll: 1 Find the least-squares solutions to the inconsistent system

One of the major applications for least-squares solutions is to the problem of detenniningbest least-squares fits to data. To introduce this imp:mant topic. consider a table of datasuch as the one displayed next.

•x'~ [-a

Table 3.1

H " " " ")'0 y, )'2 ,-.

Least-Squares Fits to Data

Solving the syslcm ATAx :::: Arb. we find the least.squares solUlion

ATA~[ 612] "d ATb=[6].12 29 7

A ~ [: ~] "d b = [ -~ ]

Suppose. when we plot the data in Table 3.1. that it has a distribution such as the onesho\.\n in Fig. 3.21. When we ex:amine Fig. 3.21. it appe~ that the data (Ktints nearlyfall along a line of the form y = ml + c. A logical question is: "What is the best linethat \.\e can draw through the data. one mat comes closest to representing the dataT

We will give the proof of Theorem 17 in the next seelion. For now, we will illustraleIhe use of Theorem 17 and also gi\c some examples showing Ihe connections betweendata-filting problems and leasl-squares solutions of inconsi!>lcm systems. (In parts (a)and (b) of Theorem 17. [he associated equalions ATAx :::: Arb are called the normalf'quQlions.)

Xl +4X2::::-2

Xl + 2X2 = 6

2fl +3X2 = I.

Solution By Theorem 17. we can find the leasl.squares solUlions by solving Ihe normal equalions.ATAx = Arb..... here

•

=

~

246 Chapter 3 The Vector Space R fl

y

•••••

••

Figure 3.21 A nearly linear distribution of data points

In order (0 answer this question. we need a way to quantify the terms best anocfose.n, There are many different methods we might u!>c to measure best. but one oflhemost useful is the least-squares criterion:

"Find 11/ and c to minimize L {(lilt; + c) - )'1 J1,,,,,

The panicular linear pol) nomial J = lilt + c that minimizes the sum of squares il'Eq. (2) is called the b~st least·squar~s linearjit to the data in Table 3.1. (We see in thtnext section that best least-squares linear fits always exist and are unique.)

Similarl). suppose the set of data points from Table I has a distribution such as tMone displayed in Fig. 3.22. In this case. it appears that the data might nearly fall alonglhe graph of a quadratic polynomial J = all + bt + C. As in Eq. (2). we can use aleast-squares criterion to choose the besr leasr-sqllan's quadraric fit:

"Find a. b. and c to minimize L [(at? + btl + c) - J,ll.,,,,,

In a like manner. we can consider fitting data in a least-squares sense with pol)nomiahof an) appropriate degree.

,.•

••

•• •

••

•

Figure 3.11 A nearly parabolic disuibution of data points

In the next several subsections. we examine the connection between least-squaresfits to data and least-squares solutions to Ax = b,

3.8 Least-Squares Solutions (0 Inconsistent S:rstems. with Applications to Data Fining 247

Least-Squares Linear Fits to Data

Suppose the laws of physics tell us that two measurable quantities, / and)". are relatedin a linear fashion:

y=m(+c. -',

Find the least·squares linear fit to the data.

E, \ \\PUc:2 Con~ider the experimental obsenations gi\en in the following table:

b is a vector x· = 1m". c·f that

X~[:l ond b~[:::]'~'k

m [to I][m]~[)'o]./1 I C )"1

,II

4

8

mto+c=,\'o

nltl +c = ,n

2"

[to ']t, ,

A = . . .

I, ,

~y I I

In this contexi. a least-squares solution to Axminimizes liAx - bl:, where

•, " 'IIAx - b ,- = L.., [(mt, + c) - y;]-.,...,

Comparing the equation abo\'e with Ihe leasl-squares criterion (2). we see thai the bestleast-squares linear fil. y = m-' + c·, can be detennined by finding the leasl-squaressolution of Ax = b.

mli-C=Yt·

In matrix lenns. this o\'erdelennined S)Slem can be expressed as Ax = b. where

1/I10-C = Yo

mIl -l- c =)"1

Usually we must be resigned to experimental errors or to imperfections in the modelgiven by Eq, (3). In Ihis case. we would probably make a number of experimenlalobservations. (tj.)"i) for i = O. I..... k. Using these observed values in Eq. (3) leadsto an o\erdetennined system of the form

Suppose also that we wish to determine experimentally the values of 1/1 and c. If we knowthat Eq. (3) models the phenomena exactly and that we have made no experimental error.then we can determine m and c with only two experimental observations, For instance.if y = Yo when / = to and if y = .n when / = (I, we can solve for 1/1 and c from

~Sl ~

o'lb<

,


SO/l/Iion For the function defined by )' = lilt + c, the data lead to the overdetermined system

m+c= I

4m+c=2

811l+c=4

11m +c= 5.

In matrix terms, the system is Ax = b, where

A~U il X~[';l "d b~[~]The least-squares solution, x~. is found by solving ATAx = ATb, where

ATA = [ 20224

2: ] ",d A'b~ [96]12 .

There is a unique solution to ATAx = ATb because A has rank 2, The solution is

X'~[ 12/29].15/29

Thus the least-squares linear fit is defined by

12 15)' - t +

- 29 29'

The data points and the linear fit are sketched in Fig. 3.23,

r

6

4

2

•

2 4 6 8 10 12 r

Figure 3.23 The least-squares linear fit to the data in Example 2

Using MATLAB to Find Least-Squares Solutions

Up to now we have been finding least-squares solutions to inconsistent systems bysolving the normal equations ATAx = ATb. This method is fine in theory but (becauseof roundoff error) it is not reliable for machine ca1culations---especial1y for large systems

3.8 Least·Squares Solutions to Inconsistent Systems, "'ith Applications to Data fitting 2~9

"~'Iem. Ax = b. r-.IATLAB has several reliable altemalhes for finding least-squares solutionsto inconsistent systems: these methods do not depend on solving the nonnal equations.

If A is not square. the simple MATLAB command x = A\b produces a least·squares solution to Ax = b using a QR-factorization of A. (In Chapter 7. we give athorough discussion of how to find least~squares solutions using QR-factorizations andHouseholder transformations.) If A is ~quare but inconsistent. then the command x =

A\b results in a warning but does not return a least-squares solution. If A is not square.a warning is also issued when A does not ha\e full rank. In the next section we willgive more details about these matters and about using MATLAB to find least·squaressolutions.

The least~squaressolution to Ax = b is found from the MATLAB commands in Fig, 3.24,

Detennine the least-squares linear fit from these readings and use it 10 delennine anoperating temperature that should limit bearing wear 10 7 gm/IO.000 hr of operation.

L\ \ \\ PI r.3 Lubricating characterislics of oils deteriorate at elevated temperatures. and the amountof bearing wear. y. is nonnally a linear function of the operating temperature. I. That is..r = nil + b. By weighing bearings before and after operation at various temperatures.the following table was constructed:

371;1] , ;

316 3431 1

12] , ;

260 2881 1

10 11.1

148 175 204 2321 1 1 1

5 5.5 6 7.5 8.8

»A.. [UO1

»b.. [3 4

»x..A\b

SOllllioll For the system Ax = b. v.e see Ihal A and b are gi\'en by

A = [ 12~ 1.8 175 20-l 232 160 288 316 343 37:]'• - 1 1 1 1 I 1 1 1

b ~ 13. 4. 5. 5.5. 6. 7.5, 8.8. 10. I I. I. 12f'.

Operntingtempernture. "e 120 1'8 175 2l" 232 160 288 316 3·" 311---

b - Amount orwear. gmllO.OOO hr 3 • 5 5.5 6 7.5 8.8 10 ILl 11

x.

0.0362·1.6151

Figure 3.14 The MATLAB commands for E~ample 3


From Fig. 3.24 we see that the least-squares linear fit is

y = (0.0362)1 - 1.6151.

Setting y = 7 yields t = 237.986. Hence an operating temperature of about 238c Cshould limit bearing wear to 7 gm/lO.000 hr. _

General Least-Squares Fits

Consider the following table of data:

)'

'0)0

".n

".'"2

'.,.'

When the data points (Ii. Yi) are plotted in the ty·plane. the plot may reveal a trend that isnonlinear (see Fig. 3.25). For a set of data such as that sketched in Fig. 3.25. a linear filwould not be appropriate. However, we might choose a polynomial function. y = p(t).

where

p(t,):::::.\',.O:::;i:::;m.

In particular, suppose we decide to fit the data with an 11th-degree polynomial:

p(t) = lIntn + lIn_lt n- 1+ ... + alt + ao. Ill::: II.

,

I• •• • • •••

•,

'0 " "Figure 3.25 Nonlinear data

As a measure for goodness of fit. we can ask for coefficients 00. a I..... an thatminimize the quantity Q(ao, al ..... an)' where

•Q(ao·(/j·· ... (/,,) = L [p(I,) - )'if

i=O

•= L [(00 +al1i + ... +a"t7) - yilt.

; ..0

"

3.8 Least·Squares Solutions to Inconsistent S~·stems. with Applications to Data Fitting 251

As can be seen b} in:-.pection. minimizing Q(ao. at ..... all) is the same as minimizinglAx - bl:!. "here

A ~ [ ;

10 tJ ...I~ ]

[ Go ]t .:!3!'1 C j I, I' I", , G,• :.:= : . and b=

I. I' r" a"• •[

YO]

:~ (5)

As before. we can minimize I1A:.: - bill = Q(ao. {Jt ....• (J~) by solving ATAx = ATb.The 11th-degree polynomial p. that minimizes Eq. (4) is called the least-squares nthdegree fit.

f, \\\1'1 I ~ Consider the data from the following table:1M

Iule.u fil I ~ -2 -I 0 I ,=Dt.

J 12 5 3 2 4

Find the least-squares quadratic fit to the data.

Solutio" Since we want a quadratic fit. "e are trying to match the fonn y = ao + alt + 02t2 tothe data. The equations are

ao - 2a 1 +-Wl2 = 12

ao - al + a2 = 5

ao = 3

ao + al + 02 = 2

(10 + 201 +4a2 = 4.

This overdetermined sy1itcm can be shown to be inconsistent. Therefore. we look for aleast-squares solution 10 Ax = b. where A and b are as in system (5). with /I = 2 andIII = 4.

The matrix A and the vectors x and b are

-2 4 12

-I 1

X~[::l5

A ~ I 1 0 0 and b = 3

a. tbx • I • 1 1 2

2 4 4

The least-squares solution of Ax = b is found by solving ATAx = ATb. where

[5 0 10] [ 26]

ATA = 0 10 0 and ATb= -19 .

10 0 34 71


•19, 19 87

p(t) = -r - -I +-.I-\. 10 35

A graph of)' = p(l) and the data poinls are sketched in Fig. 3.26.

The solution is x· = [87/35. -19/10. 19/14]. and hence the least-squares quadratic fi:is

10

y

., ,._.!22_'!! +!!)-IJ 1 JOt 35

-2 -, 2

Figure 3.26 Least-squares quadratic fit for the data in Example 4

The same principles apply when we decide to fit data with any linear combinationof functions. For example. suppose)' = f(t) is defined by

f(t) = (/181 (t) + (/282(t) + ... + (/ng,l(t).

whele 81.82 ..... 8n are given functions. We can use the method of least squares todetermine scalars al. (/2 •.... an thaI will minimize

•Q(al.a2 .. · .• a,,) = L If(t,) - y,i

i_I

•,.,

= L {Ialg\(t,) + alg2(t;) + ... + all8" (1/» _ y,}2 ...,The ideas associated with minimizing Q(a\. a1 . .... all) are explored in the exercises.

Rank Deficient Matrices

In each of Examples I--t the least-squares solution to Ax = b was unique. Indeed. ifA is (m x n). then pan (c) of Theorem 17 states that least-squares solutions are uniqueif and only if the rank of A is equal to n. If the rank. of A is less than 11. then we say thatA is rank deficiellt. or A does not havefull rank.

" quadralic fil

•

bination

3.8 Least·Squares Solutions to Inconsistent S)'stems, with Applications to Data Filling 253

Therefore. when A is rank deficient. there is an infinite family of least·squaressolutions to Ax = b. Such an example is gi\cn next. This example is worked usingMATLAB. and we note thai MATLAB produces only a single least-squares solution butdoes give a warning that A is rank deficient. In Section 3.9 we will discuss this topic inmore demil.

E:\+\,\\Pl r j For A and b as given. Ihe system Ax = b has no solution. Find all the least·squaressolutions

A ~ [ ~ ~ ~] . b = [ -: ] .-I I -I 0

-I 2 0 -3

Solution The MATLAB calculation is displayed in Fig. 3.27(a). Notice that MATLAB warns usthat A is rank deficient. having rank two. In Exercise 18 we ask you to verify that Adoes indeed have rank two. _

»A_[1 0 210 2 21-1 1 -11-1 2 0];»b_[3 -30 -31 'I»x_A\b

..quares to

Warning: Rank deficient, rank _ 2 to1_

..o

-1. 50000.5000

2.664.5.-15

~l-.e~_

Indeed. itunique

e..cl) that

Figure J.27(a) The MATLAB commands for Example 5

Since A is rank deficient. there are infinitely many least-squares solutions 10 theinconsistent system Ax = b. MATI...AB returned just one of these solutions. namelyx = [0. -1.5. O.5f. We can find all the solutions by solving the normal equationsATAx = ATb.

Fig. 3.27(b) shows the result of using MATLAB to solve the normal equations forthe original system (since A and b have already been defined. in Fig. 3.27(a). MATLAB


»NormBqn_ (A' *A. A' *b]

NormBqn _

J -J J ,-J , J -12

J J , 0

»rret (NormBqn)

llnll _

1 0 2 10 1 1 -l0 0 0 0

Figllre 3.17(b) Setting up and ~I\'ing the normal equations forExampleS

makes it very easy to define the augmented matrix for ATAx = ATb), The completesolution is Xl = I - 2t). Xl =- I ~ X3. or in vector fonn:

[1 - 2.<'] [ '] [ -2 ]

x· = -lx~X3 = -~ +x3 -:

As can be seen from the complete solUlion just displayed. the particular MATLAB least·squares solution can be reco\'ered by setting X3 = 0.5.

3.S EXERCISES

In Exercises 1-6. find all \'ectors x· that minimizelAx-bU. where A and bareasgi\en. Use the proceduresuggested by Theorem 17. as illuSlrated in Examples Iand 5.

I.A~[-::J b=[:]2A~[ -:-:-;] b~[;]

[ 121] [I]3.A= 354. b= 3

-1 I -4 0

[

I 2 -I ] [ I ]2 3 1 04.A= . b=

-I -1 -2 I

3 5 0 0

5. A~ [: ] b~ [ In

~

3.9 Theor)' and Practice or Least Squares 255

[ '00] ["]6.A= ~~~. b= ~

In Exercises 7-10. find the least-squares linear fit to thegiven data. In each exercise. plot the data points and thelinear approximation.

For given functions 81 and g~. consider a functionf defined by f(1) = al81 (I) +(/282(r). Show that

•" ' ,L [/(1,) - yd' ~ IIAx - bll',,.,where

~ -1 0 1 27,Y 0 I 2 4

in Exercises II-I·t find the least-squares quadratic fitto the given data. In each exercise. plot the data poinlsand the quadratic approximation.

x~ [:: 1"d

g,(I,) ]81(f2)

81Um)

[

y, ]",b - ..- .

Y.

[

g,(I,)

A = 81~(1)

81 (1m )

16. Let 81(r) = ..Ii and 82(t) = COSJU. and considerthe data points (t;. J;). I ::: j ::: 4. defined by

1 4 9 16

3

3

107

2

2

3

of -I9. ;r-:.:;

t -2 0 I 28, ;:l

Y 2 I 0 0

2

10. t 0y -2

IILompl~

•f __-.=2_.::-:.:'_.::0~~0. -y -3 -1 0 3

:J -2 0 I 2U,

y 5 I I 5

y 0 2 4 5

As in Eq. (6). lei Q(al. (/2) = L::.I(aI81(t;)(/282(t;) - -,",f ......here 8t(t,) = JT, and 82(t,) =cos Jrt,.

a) Use the resull of Exercise 15 to determine A. x.and b so that Q(al. (2) = IIAx - b 12.

b) Find the coefficients for f(l) = (/I..Ii .,..a2 cos;rt thai minimize Q(al. a~).

11. Consider the ((m - I) x (n - I)] matri-.: A in Eq. (5).\\here m ~ n. Show that A has rank" - 1. (Hint:Suppose that Ax = 8. \\here x = lao. al ..... an)T.What can you say about the polynomial pet) =ao -l- alt + ... + a.tn?)

18. Find the rank of the matrix A in Example 5.f.

2

2

,.. '

o-I

")'2

2

-2

y11.

_f _,-0_,-1_.::2_.::312.

, 0 0 I 2

15. Consider the following table of data:

~yin

\8/,

JjJ THEORY AND PRACTICE OF LEAST SQUARES

In the previous section. we discussed leasHquares solutions to Ax = b and the closelyrelated idea of best least-squares fits todata. In this section. we have IWO majorobjecti\'es:

(a) Develop the theory for the least-squares problem in Rn

,


(b) Usc the theory to explain some of the technical language associated with le~·

squares so that we can become comfortable using computational packages su~

as MATLAB for least-squares problems.

The Least-Squares Problem in R"

The theory necessary for a complete understanding of least squares is fairly conei;,,;and geometric. To ensure our development is completely unambiguous, we begin b'reviewing some familiar terminology and notation. In particular, let x be a vector in R

x~[::]x"

We define the distallce between two vectors x and y to be the length of the vector x - ~

recall thaI the length of x - Yis the number IIx - YII where

IIx - YII ~ lex - y)'ex - y)

= J(X\ ~ .n)2 + (X2 - Y2)2 + ... + (x" - )',,)2.

The problem we wish to consider is stated next.

The Least-Squares Problem in R"Let IV be a p-dimensional subspace of R". Given a vector y in R", find a vectorw' in IV such that

1I"-w'II:::lIv-wll, for all win IV,

The vector w¥ is called the best least-squares approximation to v.

That is, among all vectors w in IV. we wanl to find the special vector w¥ in IV thatis closest to v. Although this problem can be extended to some very complicated andabstract settings, examination of the geometry of a simple special case will exhibit afundamental principle that extends to all such problems.

Consider lhe special case where IV is a two-dimensional subspace of R3 • Geometrically, we can visualize IV as a plane through the origin (see Fig. 3.28). Given a point \not 011 IV, we wish to find the point in the plane. w', that is closest to Y. The geometI)of this problem seems to insist (see Fig, 3.28) that w' is characterized by the fact thatthe vector v - w* is perpendicular to the plane W.

The next theorem shows that Fig. 3,28 is not misleading. That is, if Y - w' isorthogonal to every vector in IV. then w' is the best least-squares approximation to v,

I TH EOI\ E\\ I ~ Let IV be a p-dimensional subspace of R". and let v be a vector in R". Suppose thereis a vector w' in \V such that (y - W*)T W = 0 for every vector w in IV. Then w' is thebest least-squares approximation to Y.

3.9 Theor)' and Practice of Least Squares 257

kd with least"kages such "

'airly concisee begin b~

ector in R".

v '--"ib,_w.r .'w_

)'

IV

x

Figure 3.18 w* is the closesl point in the plane IV 10 \'

(1)

Ilv - wll 2 = n(v - w*) -l- (w* - w)11 2

= (,' - w·)T (\. _ w*) + 2(\' - w'l (w' - w)

-l- (w· - w)T (w· - w)

= nv - w*p2 + hw· - w11 2•

(The last equality follows because w· -w is a veclOr in W. and therefore ("_W")T(w·w) = 0.) Since nw" - w02 :: O. it follows from Eq. (I) that Ill' - wf 2: nv - w·1I 2

.

Therefore. w' is lhe best approximation to ". •

The equality in calcu!ation(1).I,v-w,1 2 = ,'_w·1l 2+Ow*-wIl2. is reminiscent oflhe Pylhagorean theorem. A schematic "iew of this connection is sketched in Fig. 3.29.

Proof Let w be any vector in Wand consider the fol1owing calculation for the distance from \.to w:

'e..-Ior x - ~

:a "ector

, - "...

dw " .•II'

Figun 3.19 A geometric interpretation of !he ,ector w' in W closest

'0'

In a later theorem. \\e will show thatlhere is always one. and only one. ,ector w'in W such that \' - w* is onhogonallO e"ery vector in W. Thus it will be establishedthat the beSt approximation always exists and is always unique. The proof of Ihis factwill be constructi\e. so \\e now concentrate on methods for finding w·.

•

Finding Best Approximations

Theorem 18 suggests a procedure for finding the best approximation W". In particular.we should search for a ,'ector w· in \V that satisfies the following condition:

If w is any vector in W, then (v - w*)T w = O.


The search for w· is simplified if we make the following observation: If v - w· is orthogonal to every vector in IV. then v - w· is also onhogonal to every "ector in a baS!for IV. In facL see Theorem 19. the condition that v - w· be onhogonalto the bas!vectors is both necessary and sufficient for" ~ w· to be onhogonal to every vector in n

TtH:OHE:\\ It) Let IV be a p-dimensional subspace of R n • and lelluJ. U2•.... up) be a basis for \\Let " be a vector in W. Then (,' - w·)r w = 0 for all w in IV if and only if

(V_W·)TU/ =0, :::: i:::: p. •The proof of Theorem 19 is left as Exercise 17.

As Theorem 19 states. the beSt approximation w· can be found by solving the pequations:

(v - W·)T UI ::; 0

(v - w·)T U2 = 0

(v - W·)T Up =0

Suppose we can show that these p equations always have a unique solution.Theorem 18, it will follow that the best approximation exists and is unique.

"

Then. b~

Existence and Uniqueness of Best Approximations

We saw above that w· is a best least-squares appro~imation to \' if the vector v - ""satisfies system (2). We now use this result to prove that best approximations always existand are always unique. In addition. we will gi"e a formula for the best approximation.

I t I rORr" 20 Let IV be a p-dimensionaJ subspace of R" and let \' be a vector in R". Then there is oneand only one best least-squares approximation in IV to \'.

Proof The proof of existence is based on finding a solution to the system of Eq. (2). KQ\\.system (2) is easiest 10 analyze and sohe if we assume the basis vectors are onhogonaL

In panicular.lel {u]. U2••••• up) be an onhogonal basis for IV (in Section 3.6 \Ioeobserved that every subspace of R" has an onhogonal basis). Let w· be a "eclor in nwhere

w· = a]u] - U2U2 + ... +apup.

Using Eq. (3). the equations in system (2) become

(v - (aJu] + 0202 + ... + UpUp»TUf = O. for i = 1.2..... p.

•

Then. because the basis veclors are onhogonal. the preceding equations simplify con·siderably:

T T 0V u, - a;u, Uj::: .

Solving for the coefficienlS (II. we obtain

fori = 1.2..... p.

vT u,OJ = ~T~·

uf Uj

(4)

\- - w' is or·\.'1.or in a ba~is

10 the basis: vector in W

• ba:.is for \\,i

•hing the p

"Then. b)

Ittor v - w·'3) S e,i~t

-unauoo.

here is ODe

~ :-:ov.ortbogolL

i)Q 3.6 ~e

~ector In "

f) COD-

3.9 Theory and Practice of Least Squares 259

Note that the preceding expression for Ui is well defined since u, is a basis vectOr. andhence the denominator uT u, cannot be zero.

Ha\ ing solved the system (2). we can write down an expression for a vector w'such that (v - w·)TW = 0 for all w in IV. By Theorem 18. this vector w· is a bestapproximation to v:

P vT U. " 'w=L-r-. Uj •

1=1 "i U,

Having established the existence of best approximations with formula (4). we turnnow to the question of uniqueness, To begin, suppose w is any best approximation to\'. and w· is the best approximation defined by Eq. (4). Since the vector \' - w· wasconstructed so as to be orthogonal to every vector in IV, we can make a calculationsimilar to the one in Eq, (I) and conclude the following:

11\' - wf = 11\' - w'1l2 + Iw' - w11 2.

But. if wand w' are both best approximations to v. then it follows from the equationabove that Ilw' _"'11 2 = O. This equality implies that w· -w = 8or w· = w. Therefore.uniqueness of best approximations is established. -

The following example iIIu~trates how a best approximation can be found fromEq. (4).

[\" \\1'1 l I Let IV be the subspace of R3 defined by

[X, ]

IV = {x: x = X2 • Xl +.1"2 - 3.(3 = O}.

x,

Let \' be the ,ecIOT\' = (I. ~2. -4f. Use Eq, (4) to find the best least·squares approximation to \-'.

Solutio" Our first task is 10 find an orthogonal basis for W. We will use the Gram-Schmidt processto find such a basis.

To begin. x is in IV if and onl) if XI = ~.1"2 + 3.1"3. That is. if and onl) if x has thelonn

[-X,+3X,] [-1] [3]

x= : =.1"2 ~ +~ ~ .

Therefore. a nalUral basis for W consists of the tWO vectors WI = (-I. I, of and

"'2 = (3.0. If·We now use the Gram-Schmidt process to derive an onhogonal basis {UI. U2} from

the natural basis {WI. \Il2}. In panicuiar.

(a) Let Ul = \\'1·

(b) Choose a scalar {/ so that U2 ="'2 + QUI is orthogonal (0 UI.

260 Chapter 3 The Vector Space Nil

To find the scalar II in (b). consider

uf U2 = uf (\\'2 +aUl), ,= u 1 W2 +aUI UJ.

Thus. to have ufu~ = O. we need uf \\'2 +aufUI = O. or

Uf W2a=-~,~

U1 u,

-3=~2

= 1.5.

Having found a. we calculate the second vector in the orthogonal basis for \V. findiU2 = w~ + l.5uJ =13. O. IV + !.S[-I. l. OlT = [1.5. 1.5. I]T.

Next. let \1· = alUI + a2U2 denote the best approximation. and determine lL

coefficients of \I using Eq. H):

vTUI ~3

al=-,-=~=-1.5ul

UI _

v T U2 -5.5a, = ~- ~ -- =-1.

- UfU2 5.5

Therefore. the best approximation is gh'en by

\1' = -1.5uI - U2 = -1.51-1. J. OJT - [1.5. 1.5. If = 10. -3. -If.

(As a check for the calculations. ""e can form "- w· = (I. I. -3f and verify tha.\ - \I i~ orthogonal to each of the original basis vectors. WI = [-I. I. OJT and W2 =[3.0. lIT.) •

Least-Squares Solutions to Inconsistent Systems Ax = b

In Section 3.8 we ""ere imerested in a special case of least-squares approximationsfinding least-squares solUlions to inconsistent systems Ax = b. Recall that our methoofor finding least-squares solutions consisted of soh ing the normal equations ATAx =ATb. In tum. the validity of the nonnal equations approach ""as based on Theorem 1\\ hich said:

(8) The normal equations are ah\ays consistent.

(b) The solutions of the normal equations are precisely the least-squares solution,of Ax = b.

(c) If A is (m x Ill. then least~squares solutions of Ax = b are unique if and onl~

if A has rank 11.

We arc now in a po~ition 10 sketch a proof of Theorem 17. The basic ideas supporting Theorem 17 are very important 10 a complete understanding of least-squares solutions of inconsistent systems. These ideas are easy 10 explain and are illustrated inFig. 3.30.

Figure 3.30 A gro~lric \·isualization of lbeorem 17

In Fig. 3.30. we think of Ihe (m x n) matrix A as defining a function of thefonn)' = Ax from R" to R"'. The subspace 'R(A) represents the range of A: it is ap-dimensional subspace of Rm . We have drawn the vector b so that it is not in R(A),illustrating the case where the system Ax = b is inconsistent. The vector }'- representsthe (unique) best approximalion in 'R(A) to b.

Rm

~ =Ax-------.


R'

n°. findin.:-

'[ttmine the

:S)

In addition.

_ ,T

'mtvand 1Ir; =

•

ProofofTheorem 17 Because ),.- is in 'RCA). Ihere must be veclOrs x in R" such Ihat Ax = )".because ya is Ihe c10sesl point in 'R(A) 10 b. we can say:

A vector x in 'R" is a best least-squares solution to Ax = bif and only if Ax = y•.

In order to locate y~ in IV. we note that ).• is characterized by wT(y. - b) = 0 forany veclOr w in R.(A). Then. since Ihe columns of A fonn a spanning set for 'RCA), y'can be characterized by Ihe condilions:

AJ()·· - b) =O. fori = I. 2..... /1.

Finally. since ya is in 'R(A). finding )'~ to solve Eq. (7) is the same as finding vectors xin R" that salisfy Ihe nonnal equalions:

The orthogonality conditions abo\e can be rewritten in matrix/vector terms as

AT(y·-b)~6. (7)

AT(Ax - b) =(J.

ond

We can nov. complete Ihe proof of Theorem 17 by making the observation thatEq. (6) and Eq. (8) are equivalent in the following sense: A veclor x in R" satisfiesEq. (8) if and only if the vector y. satisfies Eq. (6), where)'- = Ax.

To establish part (a) of Theorem 17. we note that Eq. (6) is consistent. and hencelhe nonnal equations given in Eq. (8) are consistent as well. Part (b) of Theorem17 follows from rule (5) and (he equivalence of equations (6) and (8). Pan (c) ofTheorem 17 follows because Ax = y- has a unique solUlion if and only if the columnsof A are linearly independent. _


Uniqueness of Least-Squares Solutions to Ax = b

Best leasl-squares approximations are always unique but least-squares solutions to A:\ =b mighl or might not be unique. The preceding statement is somewhat confusing becauthe term least-squares is being used in two different contexts. To clarify this widelaccepted, but somewhat unfortunate. choice of terms, we can refer to Fig. 3.30.

In Fig. 3.30. the best least-squares approximation. y•. is unique (uniqueness y,~

proved in Theorem 10). A best least-squares solution to Ax = b. however. is a vector xsuch that Ax = y·. and there might or might not be infinitely many solutions to Ax = ~

(The equation Ax = y' is always consistent because y. is in R(A): the equation hasunique solution if and only if the columns of A are linearly independent.)

Recall from the previous section that an (m x n) matrix A is called raukdejiciellt if:has rank less than 11 (that is. if the columns of A are linearly dependent). When A is randeficient. there are infinitely many least-squares SolUlions to Ax = b. In this instancewe might want to seleci the minimum norm solution as the least-squares solution y,e u...:in our application. To explain. we say x' is the minimum norm least-squares solUliQrl10 Ax = b if Ux·11 minimizes IIxli among al1leasl-squares solUlions. That is.

!lx"lI = minfl.x'i : Ax = yOlo

II can be shown that the minimum norm solution always exiSIS and is always unique.The minimum nonn solution is associaled with another least-squares concept. Ih

of the pseudoinverse of A. The pseudoin\erse of A is. in a sense. the closest thing (,an inverse (hal a rectangular matrix can have. To explain the idea. "e first introdu<.:ethe Frobenius noml for an (m x n) matrix A. The Frobenius nonn, denOled HA [IF,defined by the following:

• •lAI:, ~ II:L>~·

;=1 j=1

Just as I x measures the size of a \'eclor x. IIA !IF measures lite size of a matrix A.Now. let A be an (m XII) mauix. Thepseudoim'erseof A. denoled A+, is the (n xm

matrix thai minimizes lAX - IIIF where I de,notes the (m X m) idemit) matrix. It canbe shown that such a minimizing matri~ always e~ists and is alwa):, unique. As can beseen from Ihe definilion of the pseudoinw.rse. it is Ihe closest thing (in a least-squaresense) 10 an inverse for a reclangular matrix. In the event that A is square and in\enible.then Ihe pseudoinverse coincides with the usual inverse. A-I. It can be shown Ihat 1Mminimum nonn least-squares solution of Ax = b can be found from

x· = A-b,

An actual calculation of the pseudoin\oerse is usually made with the aid of anolher typeof decomposilion. the singular-value decomposition. A discussion of the singular-valuedecomposition would lead us too far afield. and so we ask the interested reader to consulta reference. such as Golub and Van Loan. Matrix Compillatiolls.

MATLAB and Least-Squares Solutions

As we noted in the previous section. there are seveml ways to sohe least-squares problems using MATLAB.

3.9 Theor)' and Practice of leaSI Squares 263

a = 1.05.b = 2.00.c= -4.10.

A least-squares solUlion.

(a) If A is (m x II) wilh III :F 11. then the MATLAB command A\b retums aleast·squares solution to Ax = b. If A happens to be rank deliciem. thenMATLAB selects a least-squares solution with no more than /) nonzero emries(where p denotes the rank of A). The least-squares solution is calcul.ated usinga QR-factorizatioll for A (see Chapter 7).

(b) If A is square and inconsistent. then the MATLAB command A\b will producea warning that A is singular or nearly singular. but will not give a least-squaressolution. One way to use MATLAB to find a least-squares solution for a squarebut inconsistent system is to set up and solve the normal equations.

(c) Whether A is square or rectangular. the MATLAB command x = piny (A)·bwill give the minimum norm least-squares solution: the command piny (A)generates the pseudoim'erse A +.

The following sample values from the function =- = f(x. y) were obtained from experimentalobservations:

[112] [I]

A~ : ~ ~ . b~ : .

to this o\erdetennined system Ax = b was found using MATLAB. see Fig. 3.31. SinceMATLAB did not issue a rank deficient warning. we can assume that A has full rank(rank equal to 3) and therefore thai the least-squares solulion is unique. _

f(1. 1) ~ -1.1 f(1.21 ~ 0.9

f(2. I) ~ 0.2 f(2. 2) ~ 2.0

f(3. I) ~ 0.9 f(3. 2) ~ 3.1

We "auld Iil.e to approximate Ihe surface:: = f(:c. y) by a plane of the form:: =ax + by + c. Use a least-squares criterion to choose the parameters a. b. and c.

Solution The conditions implied b} the experimental observalions are

a- b+c=-I.I

2a- b+c= 0.2

3a- b,c= 0.9

a - 2b +c = 0.9

2a-2b+c= 2.0

3a-l-2b+c= 3.1.

C\-\\\PI r 3 Find a least-squarei> solution 10 the equation Ax = b where

I \.\\\PlC2

uenes\ \\ j,:\

I" a '-ector I

loA). =.\

uation h:..,

m,loA). =ing becau""this wickl

:UO.

•

r ~ - .

264 Chapler 3 The 'ector Space R"

»A_fl,l,l;2,l,1;3,1,l,1,2,l;2,2,l;3,2,l];»b_[_l.l, .2, .9, .9,2.,3.1] 'i

»x_A\b

x •

1.05002.0000

-4.1000

)

Figure 3.31 The result:. of Example 2

Solutioll The re~uhs art shown in Fig. 3.32(a}. Note that ~1ATLAB has issued a rank deficiemwarning and concluded that A has rank 2. Because A is not full rank. least-squares solutions to Ax = b are not unique. Since A has rank 2. the MATLAB command A\b ~elecb

a solution with no more than 2 nonzero components. namely Xl = [0.0. -0.8, uf.As an altemative. we can use the pseudoinverse to calculate the minimum-nonn

least-squares solution (see Fig. 3.32(b)). As can be seen from Fig. 3.32(b). the MATLABcommand piny (A) *b has produced the least-squares solution X2 = [0.6. -0.2. OAfA calculation shows that IIxll! = 1.2806. while the minimum norm solution inFig. 3.32(b) has IIx211 = 0.7483.

Finally. to complete this example. we can find all possible least-squares solutionsby solving the nonnal equations. We find. using the MATLAB command rref (8 .

»x_A\b

warning: Rank deficient, rank _ 2

x •

o-0.80001.1000

(x)

x •

x_piny (A) *b

0.6000-0.20000.4000

(bl

Figure 3.31 (aj L.:smg the command A.o 10 find a least-squaresr.olution for Example 3. (b) Using the pseudoin\eT'Se to find aleast.squares solunon for Exampk 3.

II

I

~ deficient~uares solu

;. bselem... uf.mum-norm.'IATlAS

-f>:!. OAf,Iulion in

~~Iution- ::-::-ef E

11.,0,


that the augmented malrix B = [AT A IATb] is rov. equhalent to

[10 1 1]o I I .2

o 0 0 0

Thus, the set of aillenst-squares solutions are found from x = [I - ,\"3. 0.2 - ,\"3. XJ f =[I.O.2.0V +X3[-1, -I, IV· •

I \-\ \\1'1 L ~ As a final example 10 illustrate how MATLAB treats inconsistent square systems. find aleast-squares solution 10 Ax = b where

A~[: ~ n b~[:JSoilltioll The results are given in Fig. 3.33 \\, here. for clarity. we used Ihe rlllional formal to display

the calculations. As can be seen. the MATLAB command A\b results in a \\ aming thatA may be ill conditioned and ma~ have a solution vector with "ery large components.

Then. a least-squares solution is calculmed using the pseudoinverse. The leasl-squares solution found is x = [2/39. -2/13. 8/39J1. •

/' '"»A_[~,3,5;l,O,3;3,3,8};

»b_ [1,1,11';»x_A\b

Warning: Matrix is close to singular or badly scaled.Results may be inaccurate. RCOND _ 6.405133.·18

x •

-67553994410557447S0S9gg3789508~

~~S1799813685248

»x_pinv(A)*b

x •

2/39-2/13

8139

Figure J.JJ The results from Example 4

266 Chapter 3 The Vector Space RR

3.':11 EXERCISES

Exercises 1-16 refer to the following subspaces:

a) IV =

(x x~ [:J x, - 2" +xJ ~ 0)

•

c

1. IV given by (a). \'= [1.2.6f

2. IV given by (a). \' = [3, O. 3V

3. IV giYen by (a). \' = [1. I. If

4, IV given by (b). v = [l. I. 6f

5. IV given by (b). v = [3, 3. 3]T

6. IV given by (b). \' = [3. O. 3V

7. IV given by (c). \' = [2. O. 4V

8. IV given by (c). v = [4.0. -IV

9. IV given by (d). v = [1. 3. If

10, IV given by (d), v = [3.4. OJT

In Exercises 11-16. find an onhogonal basis for the indicated subspace IV. Use Eq. (4) to detennine the beStapproximation w* for the given vector v.

II. IV and v as in Exercise 1

12. IV and v as in Exercise 213. IV and v as in Exercise 4

14. IV and v as in Exercise 5

15. IV and \' as in Exercise 7

16. IV and v as in Exercise 8

17. Prove Theorem 19,

2 4]o -2

1 3

B~[~:]

B~ [ -I

(x x~ [:Joj IV ~ R(B).

b) IV ~ R(8).

d) IV ~

Xl +Xl + .'1'3 = 0 IX\ - Xl - X3 = 0

In Exercises 1- [0, find a basis for the indicated subspaceIV. For the given vector v, solve the nonnal equations(2) and determine the best approximation w·. Verifythat v - w· is onhogonalto the basis vectors.

SUPPLEMENTARY EXERCISES

IV = {x: x = [ x, ]. x\ ~ 0, X2 ~ OJ.X2

Verify that IV satisfies propenies (sl) and (s3) ofTheorem 2. l11ustrate by example that IV docs notsatisfy (s2).

2. Let

1. Let

[X, ]IV = jx: x = .x2

XIX2 = OJ.

3. Let

['-I I]A = 1 4-1

2 2 I

and

[" ]IV = {x: x = X2 • Ax = 3x}.

XJ

Verify that IV satisfies prope,nies (51) and (s2) ofTheorem 2. lllustrate by example that IV does notsatisfy (53).

a) Show that IV is a subspace of R3•

b) Find a basis for Wand determine dim(IV).

~

Supplementary Exercises 267

~. If

2" ]3"

(m + 1)/1

b~ [: l

o 8 -5a"'t"3b].I -3 2a - b[~

and suppose that T: R~ - R! is a linear transformation defined by T (x) = Ax. "here A is a (2 )( 3)matrix such that the augmented malrix (A bl reduces to

T(xd = [ :] and T(X2) = [ _~ l

a) Find \ectors Xl and X1 in R' such (hat T(xI) =Cl and T(xJ) = C1_ where CI and C2 are the unit\-cetoTS in R1•

b) Exhibit a nonzero \eClor X3 in R1 such that X3 isinX(T).

c) Show that B = {XI. Xl. x31 is a basis for R3.

d) Express each of the unit \CCtors el_ C2_ el of R)as a linear combination of the vectors in B.Now calculate T(c,). i = 1.2.3. and determine the matrix A.

Find a basis for the row space of A. and dctenninethc rank and the nullity of A.

8. In a)--c). use the givcn infonnation to detennine thenullity of T.

a) T: R' _ R2 and the rank of T is 2.

b) T: R 3 ...,.. Rl and the rank of T i~ 2,c) T: R3 _ R3 and the rank of Tis 3.

9. In a)--c). use the given information to delennine therank of T.

a) T: R3 ...,.. R2 and the nullity of T i~ 2.b) T: R·1 __ R1 and the nullity ofT is I.

c) T: R2 _ Rl and the nullity of T is O.

10. Let B = (XI.X~J be a basis for R2. and let T:R2 _ R2 be a linear transfonnation such thai

If e\ = XI ~ 2X2 and C1 = 21' + 1'2. y,here CI andC2 are the unit \ectors in R 2• then find the matrixofT.

11. Le,t

a) Reduce the matrix A to echelon fonn. anddetennine lhe rank and the nullity of A.

b) Exhibit a basis for thc roy, ~pace of A.e) Find a basis for the column space of A (that i!>.

for R.(A» con... l ...ting of columns of A.

d) Use the answers obtained in pan... bl and c) toexhibit bases for the to\\ ...pace and the columnspace of AT.

e) Find a basis for.'·(A).

then show that Sp(S) = Sp(T), [Him: Obtain an algebraic specification for cach of Sp(S) and Sp(T).}

S. Let

s~IUH;]1"d

T~I[~lUHnl

6. leI S = {";. "~. ')}. y,here

"~ [-:J "~ [J ,00

"~[Ja) Find a subset of S that is a ba... is for Sp( S).

b) Find a basis for SpCS) by setting A =[VI. "2. "3J and reducing AT to echelon fonn.

e) Ghe an algebraic specification for Sp(S). anduse that specification to obtain a basis for SpeS).

7. Let A be the (m )( n) matrix dcfined by

[

,,_I ,,+2 2,,-1

211-1211+2 3/1-1•-1. =

ml1:+111II1'+2 (111+1)/1-1

[1-I 2 3]

A = 2 -2 5 4

I -I 0 7

268 Chapter 3 The Vedor Space R"

In Exercise~ 12-18, b = [(I.b.c.dJT, T: R6 ....... R4 isa linear transfonnation defined by rex) = Ax. and A isa (4 x 6) malrix such that the augmemed malrix [A blreduces to

12. Exhibit a basis for the row space of A. and deh~nnine

the rank and the nullity of A.

13. Dctennine \\hich of the follo\\ing \ector'S are in'R(T). Explain hO\\ you can tell.

14. For each vector WI, i = 1.2.3.4 listed in E:t.::ercise 13. if the system of equations Ax = W; i!> consistent. then exhibit a solution.

15. For each vector w . ; = 1.2.3. -I listed in Exer~

cise 13. ifw, i!> in "R(T). then find a \ectOT x in ~such thaI rex) = ",.

W, ~ [-] Wl = [; lW, = [-] ". = [ 1]

16. Suppose that A = lAt. Az. A3. A4. As. A61.

a) For each \ector w,. ; = I. 2. 3. 4. listed in E..;ercise 13. if w is in the column space of A. the'"express ". as a linear combination of thecolumns of A.

b) Find II subset of (AI. A:!. A3. A..!. As. ~l thilta basis for the column space of A.

c) For each column. A). of A thai does nOI appe:trin the basis obtained in part b). express Aj as alinear combination of the basis \ ectors.

d) Letb = (I. -2. I. -7f. Shay, that bisinthecolumn space of A. and expres~ b as a linearcombination of the basis vectors found inpart b).

e) If.!!: = [2,3.1. -I. I. If.lhenexpress Ax asalinear combination of the ba;,is vectors found inpari b).

17. a) Give an algebraic specification for "R(T). anduse that ~pecification to detennine a basis forR(T).

b) Sho", thaI b = (I. 2. 3. 3f is in "R(T). andexpre!>s b a.. a linear combination of the basiswctors found in part a).

18. a) Exhibit a basis for X'(T).

b) Show that x = [6. I. 1, ~2. 2. _2)T isin. \'(T). and expres~ x as a linearcombination of the basis vectors foundin pan a).

-la+ b-2, ]I~ + 5b -7c-5a-2h+3c .

-1&1-7b -r9c +d

0-3

o 2 2

I -I -2

o 0 0

o 2

I -I

o 0

o 0[~

CONCEPTUAL EXERCISES

In Exercises 1-12. answer true or fal ..e. Justify your ans"'er by prO\'iding a counterexample if the stalement isfalse or an outline of a proof if the statement is true.

1. If IV is a subspace of R" and x and ). are \ ectors inR~ such that x +y is in IV. then x is in IV and )' isin IV.

2. If IV is II subspace of R" and (IX is in IV. where a isa nonzero scalar and x is in R". then x is in IV.

3. If S = /XI ..... x~l is a subset of R· and k ::: n.then S is a linearly independent .set.

·t If S = {xl •.... x.tl is a ..ubsetof R· andk > II.then S i!> a linearly dependent .set.

5. If S = {Xl ....• xd is a subset of R· and k < n.then S j;, not a spanning ..et for R".

6. If 5 = {Xt ..... xd is a subsel of Wand k :::: n.then 5 is l\ spanning set for R".

7. If 51 and S2 are linearly independent subsets of R·.then the sel SI U S~ is also linearly independent.

8. If IV is a subspace of R-. then IV has exactly onebasis.

Conceptual Exercises 269

, A~lth:l1l'

,-lox 3.lI a

found "

Illo ..\"j.

,ted in Exo;:e of A.llIenlillie

Xow sel W = CIW + ... + C,,~I"II_I and usepan c).)

19. Let V and W be ~ubspacesof R" such that V n IV ={9\ and dim(V) + dim(W) = n.

a) If\ +\1 = 8. where \. is in V and wisin W.show that \' = (J and II = 8.

b) If 8 1 is a basb for V and 8: is a basis for W.show thaI 8 1 U B~ b a basis for Rn . [Hillt:Use part a) 10 show Ihat 8j U B2 is linearlyindependent.]

e) If x i~ in R". ~how Ihat x can be ""TIllen in thefonn x = \' + w. where \' is in V and \I is in IV.[Him: Fir<it note that \: can be wriuen as a hnearcombination of the vector<; in 8 1 U B~.l

d) Sho\l that Ihe representation obtained in part c)

is unique; Ihal is. if x = 'I - ""I. where \', is in~'and "I is in \\.. then \ = 'I and w = ""\.

20. A linear lransformalion T: R" -+ R" is 01110 pro\-idcd that "R(T) = R'. PrO' e each oflbe follov. ing.

a) If the rank of T is n. tben T is onto.

b) If the nullny of T is zero. then T is onlo.

e) If T is onto. then lhe rank of Tis n and Ihenullit) of T is zero.

21. [f T: R" - R'" i~ a linear tramfonnation. thenshow that Tr9n ) = 9",. where 9" and 9", are the 7.ero\'ectol'S in R" and R"'. respectively.

22. Let T; R" -+ R'" be a linear transfOrmation. andsuppo~e that 5 = (x) ..... xd is a subset of Wsuch Ihal{T(xl) ..... T(Xll} is a linearly independent subset of R"'. Show tha! thc set 5 i~ linearlyindependent.

23. Let T: Rn _ R'" be a linear lran~fonnalion

with nullit} zcro. If S = Ix\ ..... XlJ is a linearly independent sub~1 of R". Ihen show that{T(xi I..... T(l:l}} is a linearl~ indepe,ndem subsetof R".

In Exerl'ises 13-23. give a brief answer.

13. Lei IV be a subspace of R~. and set V = Ix: x is inR~ bul x is not in I\' ,. Delcrmine if V is a sub~pace

of R~.

I·t Explain .... hat is wrong with the following argument:Let \\' be asubspace of R~. and let B = {el ..... c, \be the basis of R- consisting of lhe um! \ecIOr...Since 8 is hnearl~ indepcndem and since e'-er) \eclOr \I in W can be wrilten as a linear combmation ofthe \'ectors in 8. it follows Ihal 8 is a basis for It'.

IS. If 8 = {xl.x~.x;1 is a basis for RJ• ~how that8' = {XI. X + X!. Xl + x~ - x~1 is also a basisfor R1

•

16. Let IV be a subspace of R". and leI S ={WI' .••. \Ill be a linearly indepcndcnlsubsct of \I'such that {"''t ..... \It- II} is Iinearl) dependent fore\'er)' \I in IV. Prove that 5 is a basis for IV.

17. Lei [Uj ••.. , u,,} be a linearly independent .subset ofR~.andlctxinRnbesuchlhaturx=... = u:x =O. Sho.... that x = (J.

18. LeI u be a nonzero vector in R~. and let IV be thesubset of Rn defincd by IV = {x: u r x = OJ.

a) Pro\C Ihal IV is a subspace of R~.

b) Show Ihal dim/IV) = 11 - I.e) If9 = w+cu. wherew is in IV andcisa

scalar. show Ihal "'. = 9 and c = O. fHim:Consider u' (w + cu).)

d) If I" w,,_.}isabasisforn'.~howlhal

("I \1',,-1' uJ is a basi~ for R". [Him:Supposelhalcl\lI+"'+C__ 1W,,_ +cu=9.

9. If \1' is a subspace of R'. and dimllV) = k. then 1\comains e"actly I. \'ttlors.

10. If B is a ba~i~ for R~ and \1/ is a .subspace of R .Ihcn some subset of B i~ a basis for IV.

II. If IV is a ~ubspace of W. and dim(lV) = II. thcnIV = Rn•

12. Let IVI and IVz be ~ub~paces of R" with base~ Bland 8~. respectively. Then Bj n Hz b a basis forIV1n IVl .

not 3ppear" ... AJasa

'"bbinthe"" a linear

m

21. R = (-3. -I)

Y

8

,"

-8 ~' x-4 w 4 8

-,-8

23 ", ~ [ : ]. ", ~ [ _~ ]

H. ", ~[] ", ~ [ _~ ]

33. [ -:J35.3i-j-5k

Answers to Selected Odd-Numbered Exercises AN9

3.x=t.y=4-2/ . .:=1+31

5. The lines are parallel.

7. The lines are nOf parallel.

9.x= 1+31.)'=2+41.:= l-t

II. The line intersects the plane atP = (-1.4.1).

13. The line intersects the plane atP = (-8. -13. 36).

15.6x..;..y-.:=16

17.-1.l-y+4.:=5

19. 2.l - 7)' - 3.: = I

[

2/3]21. n = 1/3

-2/3

23. .1:-1-2.1'-2.::= 17

25. _l =4 -t.y = 5+1. .: = I

CHAPTER 3

Exercises 3.1, p. 166

I. ."

37. [ =: ]

39. [ -1)41. [J

-u(-3. -I)

3.

u

) .

(3. 1)

x

43.4..16 square units

45. 3JTT square units

47. 24 cubic units

49. not coplanar

Exercises 2.4. p. 157I.x=2+31.)'=4+2I.

z = -3+41

(-9. -3)

u (3. I)-x

I,~W'" '.W ,,, "'" [ ~ ]I~ AI"

10 = '" - "+ " [ :: ] I= AI '"

{J~WnU rC~.L':U-e D :[ ~ ]} = A\ 'S'Z

l""W," ~w ,,,. [ ~ ] I= ,II "

(0'[ 'l)

'Il

·I=.~-,·\+tr

~ds 0Xj1 JO jlllll mdn ~I UO SlU!od jO j;/S ~ S! i" 'I,

'0 = ::, +.\ .... r UO!1tm~ !P!.\\. ;Jtrnjd;HJl S! ,n '61

t--O-'V ~~~~.,

" .l-, .l- n

"

n

,,,

(

([ 'f)

'LT"-

(l'l)

'L

",,,,,,,

(f'r)

., '£I ,,\ 'S

OI.\rl:V

Answers to Selected Odd·Number«! Exercises A..'dl

-1.:: "" 0,

IWf of the sphere

,I

-./

Exercises 3.2, p. 174I. \V is a subspace. \V is the sct of poillls on the line with

equation x = 1.\'.

3. II' is not &subspace.

5, II' is the subspace consi~ting of the points on the y-allis.

7. II' is 110I a subspace.

9. II' is the subspace consisting of the points on the plane2f-y-:=0.

11. II' is not a subspace.

13. \V is DOl a subspace.

IS. II' is the SUbspace consisting of the points on the linewith parametric equations X "" 2/. -" "" -t . .:::::: t.

17. II' is the SUbspace consisting of the points on the x-allis.

19. II' is the sct of points on the plane x + 2.1' - 3~ :::: O.

23. II' IS the line formed by the two intersecting pl:lnes.f +2)'+2: = 0 :lndx +3)' = O. The line has p:lrametricequ&tionsx = -6/.)' = 2t.: = I.

25. II' is the set of points on the plane x - : = O.

Exercises 3.3, p. 1861. SpeS) = Ix: x[ -I-Xl = OJ: Spes) isthe line witheqllation

x+)'=O.

3, SpeS) = Ie}: Sp(S) is the poim (0. 0).

5, SpeS) = R1

7. SpeS) = {x: 3x[ + 2x2 = OJ: Spes) is the line withequation 3x + 2)' = O.

9, SpeS) = R2

II. Spes) = {x: Xl +X2 = 0): SpeS) is thc line withequalionx+y=O.

13. SpeS) = Ix: X2 + X, = 0 and XI = OJ: Sp(S) is the linethrough (0. O. 0) and (0. -I. J). The parametric equationsforthe line are x = 0.)' = -t.: = I.

15. SpeS) = {x: Ix[ - X1 +Xl = OJ: Sp(S) is the plane withequation 2x - )' -I- .:: = o.

17. Spes) = Rl

19. Spes) = {x: X2 + XJ = OJ: Sp(S) is the plane withequation)' + .:: =O.

21. Thnectors u in b), c), and e) are in SpeS): for b)_ u = x:for c), u = ": for e). u = ],. - 4x.

23. dande

25. x and Y

27. -"(A) = Ix in R1: -Xl -lf2 = 01:"R(A) = Ix in R1: lx[ +X2 = O}

29. N(A) = {S): "R(A) = Rl

31. ..V(A) = Ix in R.l: X[ +21:2 = 0 and X.l = 0):"R(A) = R2

33. _\'(A) = Ix in R2: X2 = OJ:

"R(A) = Ix in R.l: .12 = lx] and x, = 3x[1

35. Ar(A) = Ix in R.l: .f[ = -7xJ and Xl = 2_fll:"R(A) = Ix in R.l: -4.f[ + 2X2 + Xl = O}

37. ~\'(A) = {I}: "R(A) = R'

39. a) The vectors b in ii). v). and vi) arc in R{A).

b) For ii), x= (1. of is one choice: for v). x= 10. I)is one choice: for vi), x = 10. OJT is one choice

c) For ii), b = AI: fonl). b = A~. fon-il,b=OA[+OA2.

41. a) The vectors b in i). iii). v). and vi) are in RCA I.

b) For i), x = [-I. 1. Olr is one choice: for iii I.x = (-2. 3, O)T is one choice: for \).

x = 1-2. I. OJT is one choice: for \i).

x = 10. O. Olr is one choice.

c) For i), b = -A[ T A!; for iii). b = -2A[ T JA; f\'), b = -2AI +A2: fO£ \i), b = OA I -OA~ - ,\

47. w[ = (-2. 1. 3JT. W2 = [0. 3. 2J'

49, WI =ll.2.2]T. W2 = 10.3. IV

Exercises 3.4. p. 2001, II I. O. I. OV. [-1. I. O. liT}

3, [[ 1. l. O. O]T. [-1. O. l. OJ' .13. o. O. liT}

S.I[-I.1.0.01T • [0,0.1.01T , 10.0.0. I]TI

7. H2. I. -I. OIT, (-I. 0, O. I]T}

")X~2[t]-[-] b) x;,oo" II

<) x ~ -3 [ - ~l d) x ~ 2 [ t]

[

I 0 I ']Il.a)8= 0 I I-I

o 0 0 0

b) A basis for .V(A) is {[- L -1. I. 0]1.(-1.1.0. lIT}.

c) {AI. A2} is a basis for the column space ofA; = AI +A2 and Ai = A[ - A1.

ANI2 Answers to Selected Odd-Numbered Exercises

d) ([LO.!. I). to. I. L-I]) is a basis forthc rowspacc of A.

[1 °-1 2]o I I-I

13. a) B=o 0 0 0

o 0 0 0

b) A basis for N(A) is ([I. -1. 1, of. [-2. 1. 0, WI.c) (AI. A2} is a basis for the column space of A:

A] = -AI +A2 andA4 =2A I -A, .

d) Il I. O. -I, 2], [0. 1. 1. -I JJ is a basis forthe rowspace of A.

[

1 2 0 ]15.a)8= 001

000

b) A basis for N(A) is ([ -2, I. Of I·

c) (A I' A31 is a basis for the column space of A:A2 = lA I.

d) HI. 2. 0). [0. o. III isa basis for the row space of A.

17. Ill. 3. W. [0. -1. -IfI is a basis for R(A).

19. Ill. 1, 2. Of. [0. I. -2, IjT} is a basis for R(A).

2La) {[1.2jT}: b) (II.WI

23. a) ([1.2.lf.[2,5.0jTj:b) IlL2.ljT.[0.1.-2f)

25. a) {l0.1.0n: b) {[-I. 1.0jT.[0.0. 1jT1c) ([-l.l.ofl

27. -2v] - 3V2 + \', = 9. so 5 is linearly dependent. Since\'] = 2v] + 3V2. if" = a) VI + a,l', + aJ"] is inSp{\'I. v,. '·Jl. then v = (a] + 2a3)VI + (a, + 3aJ)\'l.Therefore ,. is in SP(vl. \·l}.

29. The subsets are {Vj. "2. \'J}. (VI. v,. V4),{vl. V). '·4}.

33. 5 is not a basis.

35. 5 is not a basis.

Exercises 3.5, p. 212L 5 does not span R' .

3. S is linearly dependent.

5. 5 is linearly dependent and does not span R',

7. 5 does not span R3.

9. 5 is linearly dependent.

II. 5 is a basis.

13. Sis nOl a basis.

15. dim(W) = 3

17. dim(W) = 2

19. dim(lV) = I

21. (( -2. WI is a basis for ..V(A): nullity(A) = I:rank(A) = 1.

23. {[-5. -2, If} is a basis for N(A): nullity(A) = I:rank(A) = 2.

25. HI. -I. If. [0. 2. 3)T) is a basis for R(A): rank(A) =2; nullity(A) = I.

27. a) ([I. 1. -2). [0. -1. If. [0. o. W} is a basis for IV:dim(W) = 3,

b) (II, 2. -I. If. [0.1, -1. If. [0.0. -1. 4n is abasis for W: dim(W) = 3.

29. dim(W) = 2

(33. a) rank(A) ~ 3 and nullity(A) ~ O.

b) rank(A) ~ 3 and nullity(A) ~ 1.c) rank(A) ~ 4 and nullity(A) ~ O.

Exercises 3.6, p. 224S. ur U3 = 0 requires a - b + c = 0: ur U3 = 0 requires

la + lb - 4c = 0: therefore c = Oanda + b = O.

7. ur U2 = 0 forces a = 3: then ur U3 = 0 requires-8 - b + 3c = O. while ui U3 = 0 requires4 +b +c = 0: therefore b = -5. c = 1.

9. V = (2/3)u] - (1/2)U2 + (1/6)U3

11." = 3uI

13. u, = [n u, = [ ; l u, = [ -:~:]

15 u, =[J u, = [ J u, = [ -;]

[0] [-1] [-213]

17.u,= ~ . u,= -~ . u,= _:::

[-3] [ 7/11]-I -27/11

19. For the null space: • ,I -6/11

° 1

f~"

Exerc~

1..) [

d1 !3. c) i:;,

9. F is

11. F i~

13. F i~

15. F j,

17. F i,

19. al

21. T (

23. T (

25. A,

r.m

27. A'R

29..'"

""'"

Exerci

1. x'

3. x·

Answers to Selected Odd-Numbered Exercises ANI3

[ '] [-11/6]for the range space: 2. 8/6

I -5/6

Exercises 3.7. p. 239

I..) [a b) [ -a ,) [-: ld) [ -: ]

] ,"0

;.=3.x=a[ ~]. a#O

: ]. a #0:

).3,.,[ -: l ,"03.).• 1.'.,[

Exercises 4.1. p. 279

[

-I1.;.=l.x=a I

CHAPTER 4

[2x'+'6f1 ] .5. X' = . .11 arbltrary-x,

7. )' = 1.3t T 1.1 9. y = 1.51

11. Y = 0.5t1 +a.lt 13. y = 0.251:+2.151+0.45


1. w' ~ [ IE, ] ,.'. [ :]

5. w' • [ : ] 7. w' • [ -: ]

9w'. [-:]

['] [-1/']11. w"" = i ~ + 121 2~5

13.W'.[~]-4[ ~~',]

15.'-.'[ -; ].[ ~]

]. x, ,m;"."


1. x. = [ -5{13 ]7/13

[

(28/74) - 3xJ

3. x· = (27/74) + XJ

x,

3. c) is not but a). b). :lnd d) nre.

9. F is a linear Iransfonn31ion.

11. F is not a linear transformation.

13. F is a linear transformation.

15. F is a linear transformation.

17. F is nOl a linear transformation.

19.•j [ _: Jbj [ =: J,) [_~ ]([ x,]) [x, + x, ]

21. T .T1 = XI _ h2

[

1 I ']13. T ([ ~: ]). -;x, -;x, + ;x,;c. -:c, + -.(, + -;c.

J 2 2" 2~

25. A = [~ ~]; .\~(n = lSI: 'R.(T) = R2:

rank(T) = 2: nullity(T) = 0

21. A = (3 2]: X'(T) = Ix in R2; 3xI + 2t2 = Ol:'R(T) = Rl

: rank(r) = 1: nullity(T) = I

[1-I 0]29. A = : A"CT) = (x in Rl; Xl = .1)o I-I

and.1"1 = Xl}: 'RCT) = R2; rank(Tl = 2:nullit)(T) = I

, •

, •

.'- =

••

1= u ~wre~

•

-::]-~ ]

-: ; ]I J

-1:3

I: ],II

Documents

3.2 Vector Space Properties