58
Inner Product Spaces 1. Length and Dot Product in R n 2. Inner Product Spaces 3. Orthonormal Bases: Gram-Schmidt Process

Lecture5, Inner Product Spaces

Embed Size (px)

DESCRIPTION

Lecture 5

Citation preview

Inner Product Spaces1.Length and Dot Product in Rn 2.Inner Product Spaces3. Orthonormal Bases: Gram-Schmidt Process 1. Length and Dot Product in Rn Length: The length of a vector in Rn is given by 2 2 21 2|| ||( || ||is a real number)nv v v = + + + v v Properties of length (or norm) (1)0(2)1 (3)0iff(4) (proved in Theoerm 5.1) c c>= = ==vv vv v 0v vis called a unit vector ) , , , (2 1 nv v v = v Notes: The length of a vector is also called its norm Ex 1: (a) In R5, the length ofis given by (b) In R3, the length ofis given by) 2 , 4 , 1 , 2 , 0 ( = v5 25 ) 2 ( 4 1 ) 2 ( 0 || ||2 2 2 2 2= = + + + + = v11717173172172|| ||2 2 2= = |.|

\|+ |.|

\| + |.|

\|= v) , , (173172172 = v(If the length of v is 1, then v is a unit vector) A standard unit vector in Rn: only one component of the vector is 1 and the others are 0 (thus the length of this vector must be 1) (1) 0(2) 0cc > < u and v have the same direction u and v have the opposite directions Notes: Two nonzero vectors are parallel if{ } ( ) ( ) ( ) { }1 2: , , , 1, 0, , 0 , 0,1, , 0 , 0, 0, ,1nnR = e e e{ } ( ) ( ) { }21 2: , 1, 0 , 0,1 R = e e{ } ( ) ( ) ( ) { }31 2 3: , , 1, 0, 0 , 0,1, 0 , 0, 0,1 R = e e ec = u v Theorem 1: Length of a scalar multiple Let v be a vector in Rn and c be a scalar. Then || || | | || || v v c c =|| || | || |) () ( ) ( ) (|| ) , , , ( || || ||2 22212 222122 22212 1vvcv v v cv v v ccv cv cvcv cv cv cnnnn=+ + + =+ + + =+ + + ==Pf: ) , , , (2 1 nv v v = v) , , , (2 1 ncv cv cv c = v Theorem 2: How to find the unit vector in the direction of v If v is a nonzero vector in Rn, then the vector has length 1 and has the same direction as v. This vector u is called the unit vector in the direction of v || || vvu =Pf: v is nonzero 010 > = vvIf=1u vv(u has the same direction as v) || || | | || ||1|| || || || 1|| || || ||c c == = =v vvu vv v(u has length 1) Notes: (1) The vector is called the unit vector in the direction of v (2) The process of finding the unit vector in the direction of v is called normalizing the vector v || || vv Ex : Finding a unit vector Find the unit vector in the direction of v = (3, 1, 2), and verify that this vector has length 1 ( )22 22 2 22 2 2(3 , 1, 2) 3 1 2 14(3 , 1, 2) 1(3 , 1, 2)|| ||143 ( 1) 23 1 2, ,14 14 143 1 2 1411414 14 14 is a unit vector= = + + = = = + + | |=|\ . | | | | | |+ + = = |||\ . \ . \ .

v vvvvv Sol: Distance between two vectors: The distance between two vectors u and v in Rn is|| || ) , ( v u v u = d Properties of distance (1) (2) if and only if u = v (3) 0 ) , ( > v u d0 ) , ( = v u d) , ( ) , ( u v v u d d = (commutative property of the function of distance) Ex 3: Finding the distance between two vectors The distance between u=(0, 2, 2) and v=(2, 0, 1) is 3 1 2 ) 2 (|| ) 1 2 , 0 2 , 2 0 ( || || || ) , (2 2 2= + + = = = v u v u d Dot product in Rn: The dot product ofandis a scalar quantity Ex 4: Finding the dot product of two vectors The dot product of u=(1, 2, 0, 3) and v=(3, 2, 4, 2) is 7 ) 2 )( 3 ( ) 4 )( 0 ( ) 2 )( 2 ( ) 3 )( 1 ( = + + + = v u1 1 2 2 (isa real number)n nu v u v u v = + + + u v u v) , , , (2 1 nu u u = u ) , , , (2 1 nv v v = vMatrix Operations in Excel SUMPRODUCT: calculate the inner product of two vectors(The dot product is defined as the sum of component-by-component multiplications) Theorem 3: Properties of the dot product Ifu, v, and w are vectors in Rn and c is a scalar,then the following properties are true (1) (2) (3) (4) (5),andif and only if u v v u = w u v u w v u + = + ) () ( ) ( ) ( v u v u v u c c c = = 2|| || v v v = 0 > v v0 = v v= v 0 The proofs of the above properties follow easily from the definition of dot product (commutative property of the dot product) (distributive property of the dot product over vector addition) (associative property of the scalar multiplication and the dot product) Euclidean n-space: Rn was defined to be the set of all order n-tuples of real numbers When Rn is combined with the standard operations of vector addition, scalar multiplication, vector length, anddot product, the resulting vector space is called Euclidean n-space Sol: 6 ) 8 )( 2 ( ) 5 )( 2 ( ) a ( = + = v u) 18 , 24 ( ) 3 , 4 ( 6 6 ) ( ) b ( = = = w w v u12 ) 6 ( 2 ) ( 2 ) 2 ( ) c ( = = = v u v u25 ) 3 )( 3 ( ) 4 )( 4 ( || || ) d (2= + = = w w w) 2 , 13 ( ) 6 8 , ) 8 ( 5 ( 2 ) e ( = = w v22 4 26 ) 2 )( 2 ( ) 13 )( 2 ( ) 2 ( = = + = w v u Ex : Finding dot products ) 3 , 4 ( ), 8 , 5 ( , ) 2 , 2 ( = = = w v u(a)(b) (c)(d) (e) v u w v u ) ( ) 2 ( v u 2|| || w) 2 ( w v u Ex : (Using the properties of the dot product) Given 39, 3, 79, = = = uu u v v v) 3 ( ) 2 ( v u v u + + Sol: ) 3 ( 2 ) 3 ( ) 3 ( ) 2 ( v u v v u u v u v u + + + = + +Find. 254 ) 79 ( 2 ) 3 ( 7 ) 39 ( 3 = + + =v v u v v u u u + + + = ) 2 ( ) 3 ( ) 2 ( ) 3 () ( 2 ) ( 6 ) ( 3 v v u v v u u u + + + =) ( 2 ) ( 7 ) ( 3 v v v u u u + + = Theorem 4: The Cauchy-Schwarz inequality If u and v are vectors in Rn, then (denotes the absolute value of)|| || || || | | v u v u s | | v uv uv u v uv v u u v uv us = = == = 55 5 111 11, 11, 5 = = = uv uu vv Ex : An example of the Cauchy-Schwarz inequality Verify the Cauchy-Schwarz inequality for u=(1, 1, 3) and v=(2, 0, 1) Sol: (The geometric proof for this inequality is shown on the next slide) Dot product and the angle between two vectors To find the angle between two nonzero vectors u = (u1, u2) and v = (v1, v2) in R2, the Law of Cosines can be applied to the following triangle to obtain

) 0 ( t u u s su cos 22 2 2u v u v u v + = u vv uu vuvu v=+= + =+ = + = 2 2 1 1222122221222 221 12cos ) ( ) (v u v uu uv vv u v uu You can employ the fact that |cos | s 1 to prove the Cauchy-Schwarz inequality in R2 Note: The angle between the zero vector and another vector is not defined (since the denominator cannot be zero) The angle between two nonzero vectors in Rn: t u u s s= 0 ,|| || || ||cosv uv ucos 1u tu == 0cos 1uu ==2cos 0tu tu< v v 0 , = v v= v 0 Inner product: represented by angle brackets Let u, v, and w be vectors in a vector space V, and let c be any scalar. An inner product on V is a function that associates a real number with each pair of vectors u and v and satisfies the following axioms (commutative property of the inner product) (distributive property of the inner product over vector addition) (associative property of the scalar multiplication and the inner product) , u v , u v Note: dot product (Euclidean inner product for) , general inner product for a vector space nRV =< >=u vu v Ex 1: The Euclidean inner product for Rn Show that the dot product in Rn satisfies the four axioms of an inner product n nv u v u v u + + + = = 2 2 1 1, v u v u ) , , , ( , ) , , , (2 1 2 1 n nv v v u u u = = v uSol: By Theorem3, this dot product satisfies the required four axioms. Thus, the dot product can be a kind of inner product in Rn Ex 2: A different inner product for Rn Show that the following function defines an inner product on R2, where and 2 2 1 12 , v u v u + = v u) , ( ) , (2 1 2 1v v u u = = v u Sol: 1 1 2 2 1 1 2 2(1) , 2 2 , u v u v v u v u = + = + = u v v u w u v uw v u, ,) 2 ( ) 2 (2 2) ( 2 ) ( ,2 2 1 1 2 2 1 12 2 2 2 1 1 1 12 2 2 1 1 1+ =+ + + =+ + + =+ + + = +w u w u v u v uw u v u w u v uw v u w v u 1 2(2)( , ) w w = w Note: Example 2 can be generalized such that can be an inner product on Rn 0 , ,2 2 2 1 1 1> + + + =i n n nc v u c v u c v u c v u1 1 2 2 1 1 2 2(3), ( 2 ) ( ) 2( ) , c c u v u v cu v cu v c = + = + = u v u v 2 21 2(4), 2 0 v v = + > v v 2 21 2 1 2, 0 2 0 0 ( ) v v v v = + = = = = v v v 0 Ex 3: A function that is not an inner product Show that the following function is not an inner product on R3

1 1 2 2 3 32 u v u v u v = + u v Sol: Let) 1 , 2 , 1 ( = vThen (1)(1) 2(2)(2) (1)(1) 6 0 = + = < v v Axiom 4 is not satisfiedThus this function is not an inner product on R3 Theorem 7: Properties of inner products Let u, v, and w be vectors in an inner product space V, and let c be any real number (1) (2) (3) , , 0 = = 0v v0 , , , + = + u vw uw vw , , c c = u v uv To prove these properties, you can use only the four axioms in the definition of inner product Hint: (1)(2) (3)0 , 0 , , 0 = = = uv uv 0v , ,and , , , = + = uv vu w u v w u wv + , ,and, , c c = = uv vu uv uv u and v are orthogonal if Distance between u and v: ) ( = = v u v u v u v u , || || ) , ( d Angle between two nonzero vectors u and v: t u u s s = 0 ,|| || || ||,cosv uv u Orthogonal: 0 , = v u) ( v u Norm (length) of u: u u u , || || = The definition of norm (or length), distance, angle, orthogonal, and normalizing for general inner product spaces closely parallel to those for Euclidean n-space Normalizing vectors (1) If, then v is called a unit vector (2) 1 || || = v= v 0Normalizingvv(the unit vector in the direction of v) (if v is not a zero vector) Ex 6: Finding inner product 2 22Let( ) 1 2 , ( ) 4 2be polynomials inp x x q x x x P = = +0 0 1 1,n np q a b a b a b + + + is an inner product (a), ? p q = (b)|| || ? q = (c)( , ) ? dp q =Sol: (a), (1)(4) (0)( 2) ( 2)(1) 2 p q = + + =2 2 2(b)|| || , 4 ( 2) 1 21 q q q = = + + =22 2 2(c)3 2 3( , ) || || , ( 3) 2 ( 3) 22p q x xd p q p q p qp q = + = = ( )= + + =, and For 1 0 1 0nnnnx b x b b q x a x a a p + + + = + + + = and Properties of norm: (the same as the properties for the dot product in Rn) (1) (2) if and only if(3) Properties of distance: (the same as the properties for the dot product in Rn) (1) (2)if and only if(3) 0 || || > u0 || || = u 0 u =|| || | | || || u u c c =0 ) , ( > v u d0 ) , ( = v u d v u =) , ( ) , ( u v v u d d = Theorem 8 Let u and v be vectors in an inner product space V (1) Cauchy-Schwarz inequality:

(2) Triangle inequality:

(3) Pythagorean theorem: u and v are orthogonal if and only if|| || || || || || v u v u + s +Theorem 5 2 2 2|| || || || || || v u v u + = +Theorem 6 || || || || | , | v u v u s Theorem 4 For inner product spaces:Let u and v be two vectors in an inner product space V. If, then the orthogonal projection of u onto v is given by 0 v =,proj,( )=( )vuvu vvv2Consider0, cos| || || || cos | || || ||

|| || || ||proja a a aauu> = = = = = = = = = vv v v uu v u v uv uvv v u v vuv uv uvu vvv vvvuv0 , proj > = a av uvu Orthogonal projections: For the dot product space in Rn, we define the orthogonal projection of u onto v to be projvu = av (a scalar multiple of v), and the coefficient a can be derived as follows Ex 10: Finding an orthogonal projection in R3 Use the Euclidean inner product in R3 to find the orthogonal projection of u=(6, 2, 4) onto v=(1, 2, 0) Sol: 10 ) 0 )( 4 ( ) 2 )( 2 ( ) 1 )( 6 ( , = + + = ) ( v u 5 0 2 1 ,2 2 2= + + = ) ( v v, 10proj (1, 2 , 0) (2 , 4 , 0), 5( ) = = = =( ) vu v u vu v vv v v v Theorem 9: Orthogonal projection and distance Let u and v be two vectors in an inner product space V, and if v 0, then,( , proj ) ( , ) , ,d d c c( )< =( )vu vu u u vv v) proj , ( u uvduvv c) , ( v u c duvuvproj Theorem 9 can be inferred straightforward by the Pythagorean Theorem, i.e., in a right triangle, the hypotenuse is longer than both legs 3.Orthonormal Bases: Gram-Schmidt Process Orthogonal set: A set S of vectors in an inner product space V is called an orthogonal set if every pair of vectors in the set is orthogonal Orthonormal set: An orthogonal setin which each vector is a unit vector is called orthonormal set { }1 22, , ,For, , , 1For, , 0ni j i i ii jS Vi ji j= _= ( ) = ( ) = == ( ) =v v vv v v v vv v{ }j iV Sj in= = ) (_ =for, 0 ,, , ,2 1v vv v v Note: If S is also a basis, then it is called an orthogonal basis or an orthonormal basis The standard basis for Rn is orthonormal. For example, is an orthonormal basis for R3 This section identifies some advantages of orthonormal bases, and develops a procedure for constructing such bases, known as Gram-Schmidt orthonormalization process { } ) 1 0 0 ( ) 0 1 0 ( ) 0 0 1 ( , , , , , , , , S = Ex 1: A nonstandard orthonormal basis for R3 Show that the following set is an orthonormal basis )`|.|

\|||.|

\||.|

\|=31,32,32,32 2,62,62, 0 ,21,213 2 1Sv v v Sol: First, show that the three vectors are mutually orthogonal092 292920 02 322 320 03 23 161612 1= + = = + = = + + = v vv vv vSecond, show that each vector is of length 1Thus S is an orthonormal set 1 || ||1 || ||1 0 || ||9194943 3 3983623622 2 221211 1 1= + + = == + + = == + + = =v v vv v vv v vBecause these three vectors are linearly independent (you can check by solving c1v1 + c2v2 + c3v3 = 0) in R3 (of dimension 3), by Theorem 4.12, they form a basis for R3. So S is a (nonstandard) orthonormal basis for R3 the standard basis is orthonormal Ex 2: An orthonormal basis for P3(x) In, with the inner product, 2 2 1 1 0 0, b a b a b a q p + + = ) (} , , 1 {2x x B =) (3x PSol: , 0 0 121x x + + = v , 0 022x x + + = v , 0 023x x + + = v1 21 32 3 , (1)(0) (0)(1) (0)(0) 0 , (1)(0) (0)(0) (0)(1) 0 , (0)(0) (1)(0) (0)(1) 0( ) = + + =( ) = + + =( ) = + + =v vv vv vThen ( )( ) ( )( ) ( )( )( )( ) ( )( ) ( )( )( )( ) ( )( ) ( )( )1 1 12 2 23 3 31 1 0 0 0 0 10 0 1 1 0 0 10 0 0 0 1 1 1= ( ) = + + == ( ) = + + == ( ) = + + =v v , vv v, vv v, v Theorem 10: Orthogonal sets are linearly independent If is an orthogonal set of nonzero vectors in an inner product space V, then S is linearly independent{ }nS v , , v , v 2 1=Pf: Sis an orthogonal set of nonzero vectors, i.e.,0 for,and0i j i ii j ( ) = = ( ) > v , v v , v1 1 2 21 1 2 2For, , 0n nn n i ic c cc c c i+ + + = ( + + + ) = ( ) = v v v 0v v v v 0 v0

2 2 1 1= ) ( =) ( + + ) ( + + ) ( + ) ( i i ii n n i i i i icc c c cv , vv , v v , v v , v v , v 00 is linearly independenti i ic i S ( ) = = v , v(If there is only the trivial solution for cis, i.e., all ci = 0, S is linearly independent)(because S is an orthogonal set of nonzero vectors) Corollary to Theorem 10: IfVisaninnerproductspaceofdimensionn,thenany orthogonal set of n nonzero vectors is a basis for V 1. By Theorem 5.10, if S = {v1, v2, , vn} is an orthogonal set of n vectors, then S is linearly independent 2. According to Theorem 4.12, if S = {v1, v2, , vn} is a linearly independent set of n vectors in V (with dimension n), then S is a basis for V Based on the above two arguments, it is straightforward to derive the above corollary to Theorem 5.10 Ex 4: Using orthogonality to test for a basis Show that the following set is a basis for 4R)} 1 , 1 , 2 , 1 ( , ) 1 , 2 , 0 , 1 ( , ) 1 , 0 , 0 , 1 ( , ) 2 , 2 , 3 , 2 {(4 3 2 1 = Sv v v v Sol:

0 2 2 6 20 2 4 0 20 2 0 0 24 13 12 1= + = = + + = = + + = v vv vv v1 2 3 4, , , : nonzero vectors v v v v0 1 2 0 10 1 0 0 10 1 0 0 14 34 23 2= + + = = + + + = = + + + = v vv vv vorthogonal is S 4forbasis a is R S (by Corollary to Theorem 10) The corollary to Thm. 5.10 shows an advantage of introducing the concept of orthogonal vectors, i.e., it is not necessary to solve linear systems to test whether S is a basis (e.g., Ex 2 in Section 4.5) if S is a set of orthogonal vectors Theorem 11: Coordinates relative to an orthonormal basis If is an orthonormal basis for an inner product space V, then the unique coordinate representation of a vector w with respect to B is } , , , {2 1 nB v v v =1Since, ,then0i ji ji j=( ) = =v vV k k kn ne + + + = v v v w 2 2 1 1(unique representation from Thm. 4.9) Pf: isanorthonormalbasisfor V } , , , {2 1 nB v v v =1 1 2 2, , ,n n= ( ) +( ) + +( ) w wv v wv v wv v The above theorem tells us that it is easy to derive the coordinate representation of a vector relative to an orthonormal basis, which is another advantage of employing orthonormal bases 1 1 2 21 1, ( ),, , ,for = 1 to i n n ii i i i n n iik k kk k kk i n= + + += + + + +=wv v v v vv v v v v vn nv v w v v w v v w w ) ( + + ) ( + ) ( = , , ,2 2 1 1 Note: Ifis an orthonormal basis for V and , } , , , {2 1 nB v v v = V e wThen the corresponding coordinate matrix of w relative to B is||(((((

) () () (=nBv wv wv ww,,,21 Ex For w = (5, 5, 2), find its coordinates relative to the standard basis for R3

2 ) 1 , 0 , 0 ( ) 2 , 5 , 5 ( ,5 ) 0 , 1 , 0 ( ) 2 , 5 , 5 ( ,5 ) 0 , 0 , 1 ( ) 2 , 5 , 5 ( ,3 32 21 1= = = ) ( = = = ) (= = = ) (v w v wv w v wv w v w((((

= 255] [Bw In fact, it is not necessary to use Thm. 5.11 to find the coordinates relative to the standard basis, because we know that the coordinates of a vector relative to the standard basis are the same as the components of that vector The advantage of the orthonormal basis emerges when we try to find the coordinate matrix of a vector relative to an nonstandard orthonormal basis (see the next slide) Ex 5: Representing vectors relative to an orthonormal basis Find the coordinates ofw = (5, 5, 2) relative to the following orthonormal basis for)} 1 , 0 , 0 ( , ) 0 , , ( , ) 0 , , {(53545453 = B3R Sol: 2 ) 1 , 0 , 0 ( ) 2 , 5 , 5 ( ,7 ) 0 , , ( ) 2 , 5 , 5 ( ,1 ) 0 , , ( ) 2 , 5 , 5 ( ,3 353542 254531 1= = = ) ( = = = ) ( = = = ) (v w v wv w v wv w v w((((

= 271] [Bw1v2v3v The geometric intuition of the Gram-Schmidt process to find an orthonormal basis in R2

1v2v{ }22 1forbasis a is , R v v1 1= w v2v1 12 2 2to orthogonalis proj1v w v v ww= =2w21proj vw222forbasis l orthonorma anis } , { Rwwww11 Gram-Schmidt orthonormalization process: is a basis for an inner product space V} , , , {2 1 nB v v v =1 1Let v w =1span({ }) S =1w2span({ , }) S =1 2w w} , , , { '2 1 nB w w w = 1 21 2'' { , , , }nnB =w w ww w wis an orthogonal basis is an orthonormal basis 111 projnnn in n S n n iii i== = v , ww v v v ww, w23 1 3 23 3 3 3 1 21 1 2 2projS= = v , w v , ww v v v w ww, w w, w12 12 2 2 2 11 1projS= = v, ww v v v ww , wThe orthogonal projection onto a subspace is actually the sum of orthogonal projection onto the vectors in an orthogonal basis for that subspace (I will prove it on Slides 5.67 and 5.68) Sol: ) 0 , 1 , 1 (1 1= = v w) 2 , 0 , 0 ( ) 0 ,21,21(2 / 12 / 1) 0 , 1 , 1 (21) 2 , 1 , 0 (22 22 311 11 33 3= = = ww ww vww ww vv w Ex 7: Applying the Gram-Schmidt orthonormalization process Apply the Gram-Schmidt process to the following basis for R3

)} 2 , 1 , 0 ( , ) 0 , 2 , 1 ( , ) 0 , 1 , 1 {(3 2 1= Bv v v) 0 ,21,21( ) 0 , 1 , 1 (23) 0 , 2 , 1 (11 11 22 2 = = = ww ww vv w} 2) 0, (0, 0), ,21,21( 0), 1, (1, { } , , { '3 2 1= = w w w BOrthogonal basis} 1) 0, (0, 0), ,21,21( 0), ,21,21( { } , , { ' '3322= = wwwwww11BOrthonormal basis Ex 10: Alternative form of Gram-Schmidt orthonormalization process Find an orthonormal basis for the solution space of the homogeneous system of linear equations 0 6 2 20 74 3 2 14 2 1= + + += + +x x x xx x x Sol:

G.-J.E1 1 0 7 0 1 0 2 1 02 1 2 6 0 0 1 2 8 0 (( (( (((((

+(((((

=(((((

+ =(((((

108101228 22

4321t stst st sxxxxThus one basis for the solution space is)} 1 , 0 , 8 , 1 ( , ) 0 , 1 , 2 , 2 {( } , {2 1 = = v v B( )( ) ( )( )11 1 112 12 2 2 1 1 2 2 1 1 11 12221 2 2 1 and2,2, 1, 0 , , , 03 3 3 3,,(due to and, 1),2 2 1 2 2 11,8, 0, 1 1,8, 0, 1 , , , 0 , , , 03 3 3 3 3 33, 4,2,1130| |= = = =|\ .( )= ( ) = ( ) =( ) ( | | | |= || (\ . \ . = = =ww v uwv uw v v u u w v u u uu uwuw( )3, 4,2,1 )`|.|

\| |.|

\|= 301,302,304,303, 0 ,31,32,32' ' B In this alternative form, we always normalize wi to be ui before processing wi+1 The advantage of this method is that it is easier to calculate the orthogonal projection of wi+1 on u1, u2,, ui Alternative form of the Gram-Schmidt orthonormalization process: is a basis for an inner product space V } , , , {2 1 nB v v v =1 111 122 2 2 2 1 1233 3 3 3 1 1 3 2 23111 2,where ,where ,where { , , , } is an orthonormal basis for nnn n n n i iinnV== == = = = = = w vuw vwu w v v, u uwwu w v v, u u v, u uwwu w v v , u uwu u u