Lecture 12 orhogonality - 6.1 6.2 6.3

6

6.1

© 2012 Pearson Education, Inc.

Orthogonality

INNER PRODUCT, LENGTH,

AND ORTHOGONALITY

Slide 6.1- 2© 2012 Pearson Education, Inc.

INNER PRODUCT

If u and v are vectors in Rn, then we regard uand v as nx1 matrices

The number uTv is called the inner product of u and v, and it is written as u∙v

aka dot product

Scalar


INNER PRODUCT

If 𝐮 =

𝑢1

𝑢2

⋮𝑢𝑛

and 𝐯 =

𝑣1

𝑣2

⋮𝑣𝑛

then the inner product of u and v is

𝑢1 𝑢2 ⋯ 𝑢𝑛

𝑣1

𝑣2

⋮𝑣𝑛

= 𝑢1𝑣1 + 𝑢2𝑣2 + ⋯+ 𝑢𝑛𝑣𝑛

Inner Product - Example

Find u∙v & v∙u, for:

𝐮 =2−5−1

, 𝐯 =323


INNER PRODUCT - Properties

Theorem 1: Let u, v, and w be vectors in Rn, and let c

be a scalar. Then

a. u•v = v•u

b. (u + v)•w = u•w + v•w

c. (cu)•v = c(u•v) = u•(cv)

d. u•u ≥0, and =0 iff u = 0

Properties (b) and (c) can be combined several

times to produce the following useful rule:

(c1u1 + c2u2 + … + cnun)•v = c1(u1•v) + … + cp(up•v)


THE LENGTH OF A VECTOR

Definition: The length (or norm) of v:

𝐯 = 𝐯 ∙ 𝐯 = 𝑣12 + 𝑣2

2 + ⋯+ 𝑣𝑛2

||cv|| = |c| ||v||

Unit vector has length 1

Any vector divided by its length is a unit vector in same direction normalizing


Normalization – Example 1

Example 1: Let v = (1, -2, 2, 0). Find a unit vector

u in the same direction as v.


Normalization – Example 2

Example 2: Let W be a subspace of R2 spanned

by x = 2/31

. Find a unit vector z that is a basis for

W.


DISTANCE IN Rn

Definition: For u and v in Rn, the distance between u

and v, written as dist (u, v), is the length of the vector

u-v. i.e.,

dist(u, v) = ||u – v||

Example: Find the distance between u=(7,1) & v=(3,2)

Orthogonal Vectors

Definition: 2 vectors u and v are orthogonal

if u•v = 0.

Theorem 6-2: Pythagorean Theorem: 2

vectors u and v are orthogonal iff

𝐮 + 𝐯 2 = 𝐮 2 + 𝐯 2



Orthogonal Complement

If a vector z is orthogonal to every vector in a

subspace W of Rn, then z is said to be

orthogonal to W.

The set of all vectors z that are orthogonal to W

is called the orthogonal complement of W and

is denoted by W┴ (and read as “W

perpendicular” or simply “W perp”).

ORTHOGONAL COMPLEMENTS

Theorem 3: Let A be an mxn matrix. The orthogonal

complement of the row space of A is the null space of

A, and the orthogonal complement of the column

space of A is the null space of AT:

Proof: ai = ith row of A

Row(A) = Span{rows of A}

Nul(A) = x:Ax = 0

ai • x = 0 for every i

x is orthogonal to each row of A

(Row ) Nul A A

(Col ) Nul TA A


ANGLES & Inner Products

u•v = ||u|| ||v|| cosϑ


ANGLES & Inner Products - Example

Find the angle between 110

,011

6

6.1


Orthogonal Sets


ORTHOGONAL SETS

A set of vectors {u1,…,up} in Rn is said to be an

orthogonal set if each pair of distinct vectors

from the set is orthogonal, i.e, if

ui •uj = 0 if i ≠ j.

Orthogonal Sets - Example

Show that S={u1,u2,u3} is an orthogonal set.

𝐮1 =311

, 𝐮2 =−121

, 𝐮1 =−1/2−27/2


Theorem 6-4 – Orthogonal Set as a Basis

Theorem 4: If S = {u1, …, up} is an

orthogonal set of nonzero vectors in Rn ,

then S is linearly independent and hence is

a basis for the subspace spanned by S.



ORTHOGONAL Basis

Definition: An orthogonal basis for a subspace W

of Rn is a basis for W that is also an orthogonal set.

Theorem 5: Let {u1,…,up} be an orthogonal basis for

a subspace W of Rn. For each y in W, the weights in

the linear combination

are given by y = c1u1+ … + cpup

𝑐𝑗 =𝐲∙𝐮𝒋

𝐮𝒋∙𝐮𝒋(𝑗 = 1,… , 𝑝)

Orthogonal Basis – Example 2

Same S as in Example 1:

𝐮1 =311

, 𝐮2 =−121

, 𝐮1 =−1/2−27/2

write 𝐲 =61−8

as a linear combination of the

vectors in S.



AN ORTHOGONAL PROJECTION

Decompose y into 2 vectors:

One in direction of u ŷ = αu

One orthogonal to u z = y - ŷ

Orthogonal Projections

ŷ is the projection of y onto u

If L is subspace spanned by u

ŷ is the projection of y onto L

𝐲 = proj𝐿𝐲 =𝐲 ∙ 𝐮

𝐮 ∙ 𝐮𝐮



ORTHOGONAL PROJECTION - Example

Example 1: 𝐲 =76

, 𝐮 =42

a) Find the orthogonal projection of y onto u.

b) Write y as the sum of two orthogonal vectors,

one in Span {u} and one orthogonal to u.

ORTHOGONAL PROJECTION - Example

7 8 1

6 4 2

yy ˆ(y y)


AN ORTHOGONAL PROJECTION

Check: ŷ•z = 0?

Distance from y to L?

Distance from y to u?


ORTHONORMAL SETS

A set {u1,…,up} is an orthonormal set if it is an

orthogonal set of unit vectors (||u||=1).

If W is the subspace spanned by such a set, then

{u1,…,up} is an orthonormal basis for W, since

the set is automatically linearly independent, by

Theorem 4.

The simplest example of an orthonormal set is the

standard basis {e1,…,en} for Rn.


ORTHONORMAL SETS - Example

Example 2: Show that {v1, v2, v3} is an

orthonormal basis of R3, where

1

3 / 11

v 1/ 11

1/ 11

2

1/ 6

v 2 / 6

1/ 6

3

1/ 66

v 4 / 66

7 / 66


ORTHONORMAL SETS

When the vectors in an orthogonal set of

nonzero vectors are normalized to have unit

length, the new vectors will still be orthogonal,

and hence the new set will be an orthonormal

set.


ORTHONORMAL SETS

Theorem 6: An mxn matrix U has orthonormal columns if and only if UTU = I


ORTHONORMAL SETS

Theorem 7: Let U be an mxn matrix with

orthonormal columns, and let x and y be in Rn.

Then

a. ||Ux|| = ||x||

b. (Ux)•(Uy) = x•y

c. (Ux)•(Uy) = 0 iff x•y=0

Properties (a) and (c) say that the linear

mapping x Ux preserves lengths and

orthogonality.

Orthonormal Sets - Example

𝑈 =

1 2 2 3

1 2 2/30 1/3

, 𝐱 = 23


Orthogonal Matrix

If U is square and U-1=UT

Called “Orthogonal Matrix”

Really should be Orthonormal matrix

Orthonormal columns

Orthonormal rows as well


6

6.1


Orthogonal Projections


ORTHOGONAL PROJECTIONS

Extend ideas of previous section from R2 to Rn

Given a vector y and a subspace W in Rn, there

is a vector ŷ in W such that

ŷ is the unique vector in W for which y - ŷ is

orthogonal to W

ŷ is the unique vector in W closest to y. See the

following figure. y


THE ORTHOGONAL DECOMPOSITION

THEOREM

Theorem 8: Let W be a subspace of Rn . Then

each y in Rn can be written uniquely in the form

y = ŷ + z

where ŷ is in W and z is in W┴ .

If {u1,…,up} is any orthogonal basis of W, then

𝐲 =𝐲 ∙ 𝐮1

𝐮1 ∙ 𝐮1𝐮1 + ⋯+

𝐲 ∙ 𝐮𝑝

𝐮𝑝 ∙ 𝐮𝑝𝐮𝑝

and

z = y - ŷ



THEOREM

The vector ŷ is called the orthogonal

projection of y onto W and often is written as

projWy. See the following figure.



THEOREM - Example

Example 1: Write y as the sum of a vector in W =

span{u1,u2}, and a vector orthogonal to W.

𝐮1 =25−1

, 𝐮2 =−211

, 𝐲 =123


PROPERTIES OF ORTHOGONAL

PROJECTIONS

If {u1,…,up} is an orthogonal basis for W and if y

happens to be in W, then the formula for projWy

is exactly the same as the representation of y

given in Theorem 5 in Section 6.2.

If y is in W = Span {u1,…,up}, projwy = y


THE BEST APPROXIMATION THEOREM

Theorem 9: Let W be a subspace of Rn, let y be any

vector in Rn, and let ŷ be the orthogonal projection of

y onto W. Then ŷ is the closest point in W to y, in the

sense that

||y – ŷ|| < ||y – v||

for all v in W distinct from ŷ.

The Best Approximation Theorem

ŷ in Theorem 9 is called the best

approximation to y by elements of W.

The distance from y to v, given by ||y – v||,

can be regarded as the “error” of using v in

place of y.

Theorem 9 says that this error is minimized

when v=ŷ

Note – does not depend of basis of W used



THE BEST APPROXIMATION THEOREM


Best Approximation - Example

Example 2: Find the distance from y to W = Span

{u1,u2} for:

𝐮1 =5−21

, 𝐮2 =12−1

, 𝐲 =−1−510