02 Handout Linear Algebra

7/28/2019 02 Handout Linear Algebra

1/45

TU Munchen

1. Foundations of Numerics from Advanced Mathematics

Linear Algebra

Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics

Linear Algebra, October 23, 2012 1
http://www5.in.tum.de/http://www5.in.tum.de/


2/45

TU Munchen

1.1. Linear Algebra

Mathematical Structures

a mathematical structure consists of one or several sets and one or severaloperations defined on the set(s)

special elements:

neutral element (of an operation)

inverse element (of some element x) a group: a structure to add and subtract

a field: a structure to add, subtract, multiply, and divide

a vector space: a set of vectors over a field with two operations: scalarmultiplication, addition of vectors, obeying certain axioms (which?)

note: sometimes, the association with classical (geometric) vectors is helpful,sometimes it is more harmful




3/45

TU Munchen

Exercise Mathematical Structures

Show that the possible manipulations ofthe Rubiks Cube with the operation ex-ecute after are a group.




4/45

TU Munchen

Exercise Mathematical Structures Solution

Show that the possible manipulations ofthe Rubiks Cube with the operation ex-ecute after are a group.

Closure: executing any two manipulations after oneanother is a Cube minipulation, again.

Associativity: the result of a sequence of threemanipulations is obviously always the same no matter howyou group them (the first two or the last two together).

Identity: obviously included (just do nothing). Invertibility: execute a manipulation in backward direction.




5/45

TU Munchen


Show that the rational numbers with the operations + (add) and (multiply) are a field.




6/45

TU Munchen


Show that the rational numbers with the operations + (add) and (multiply) are a field.

Closure: obviously closed under + and .

Identity: 0 for +, 1 for .

Invertibility: each element q has an inverse q under +and 1q under . The latter holds for all elements except from

the neutral element of +, i.e., 0.

Associativity: well-known for both + and .

Commutativity: also known from school (a+ b = b+ a,a b = b a).

Distributivity: dito (a (b+ c) = a b+ a c).




7/45

TU Munchen


Is the set of NN matrices (N N) matrices with real numbersas entries over the field of real numbers a vector space?




8/45

TU Munchen


Is the set of NN matrices (N N) matrices with real numbersas entries over the field of real numbers a vector space?

The answer is yes. Look up the axioms and show that they holdfor the xample on your own.




9/45

TU Munchen

Vector Spaces

a linear combination of vectors

linear (in)dependence of a set of vectors

the span of a set of vectors

a basis of a vector space

definition?

why do we need a basis?

is a vectors basis representation unique?

is there only one basis for a vector space?

the dimension of a vector space

does infinite dimensionality exist?

important applications: (analytic) geometry

numerical and functional analysis: function spaces are vector spaces(frequently named after mathematicians: Banach spaces, Hilbert spaces,Sobolev spaces, ...)




10/45

TU Munchen

Exercise Vector Spaces

Is the set of vectors

10

,

01

,

13

linearly

independent?




11/45

TU Munchen

Exercise Vector Spaces Solution

Is the set of vectors

10

,

01

,

13

linearly

independent?

The set of vectors is not linearly independent, since the thirdelement can easily be written as a linear combination of the firsttwo:

13

= 1

10

+ 3

01

.




12/45

TU Munchen


span

100

,

001

= ?




13/45

TU Munchen


span

100

,

001

=

a0b

; a, b R

.




14/45

TU Munchen


Consider the set of all possible polynomials with realcoefficients as a vector space over the field of real numbers.Whats the dimension of this space? Give a basis.




15/45

TU Munchen


Consider the set of all possible polynomials with real

coefficients as a vector space over the field of real numbers.Whats the dimension of this space? Give a basis.

The space is infinite dimensional, a basis is for example

1, x, x2, x3, . . ..




16/45

TU Munchen

Linear Mappings

definition in the vector space context; notion of a homomorphism

image and kernel of a homomorphism

matrices, transposed and Hermitian of a matrix

relations of matrices and homomorphisms

meaning of injective, surjective, and bijective for a matrix; rank of a matrix

meaning of the matrix columns for the underlying mapping matrices and systems of linear equations

basis transformation and coordinate transformation

mono-, epi-, iso-, endo-, and automorphisms



TU M h


17/45

TU Munchen

Exercise Linear Mappings

Is the mapping f : R3 R3, x 5 x +

123

linear?



TU Munchen


18/45

TU Munchen

Exercise Linear Mappings Solution

Is the mapping f : R3 R3, x 5 x + 1

23

linear?

f is not linear, since f(x) = f(x).



TU Munchen


19/45

TU Munchen


Whats the linear mapping f : R2 R2 corresponding to the

matrix

4 03 2

?



TU Munchen


20/45

TU Munchen


Whats the linear mapping f : R2 R2 corresponding to the

matrix

4 03 2

?

fxy =

4x3x + 2y .



TU Munchen


21/45

TU Munchen


Give the rank of the matrix

1 0 0 0

0 1 0 00 0 0 00 0 0 1

.Is the corresponding linear mapping injective, surjective,bijective?



TU Munchen


22/45


Give the rank of the matrix

1 0 0 00 1 0 00 0 0 0

0 0 0 1

.

Is the corresponding linear mapping injective, surjective,bijective?

The rank is three. Thus, the corresponding linear mapping isneither injective, nor surjective or bijective.



TU Munchen


23/45

Examples Linear Mappings

Monomorphism:

0 10 01 0

Epimorphism: 1 0 0

0 1 00 0 0

Iso-/Automorphism:

0 11 0

Endomorphism:

2 1 00 1 21 0 1

.



TU Munchen


24/45

Determinants

definition

properties

meaning

occurrences

Cramers rule



TU Munchen


25/45

Determinants Definition

det(A) =

a1,1 a1,2 a1,Na2,1 a2,2 a2,N

.

.

.. . .

.

.

.

aN,1 aN,N

=

a1,1

a2,2 a2,N...

.

.

.

.

.

....

aN,2 aN,N

a1,2

a2,1 a2,3 a2,Na3,1 a3,N

.

.

....

aN,1 aN,3 aN,N

+ . . .

Miriam Mehl: 1. Foundations of Numerics from Advanced MathematicsLinear Algebra, October 23, 2012 25

TU Munchen


26/45

Exercise Determinants

det(A) = 0 A defines a . . .morphism.

det(A) = 0 A defines a . . .morphism.


TU Munchen


27/45

Exercise Determinants Solution

det(A) = 0 A defines an Endomorphism.

det(A) = 0 A defines an Automorphism.


TU Munchen


28/45


det(A B) =?

det

A1

=?

det AT =?


TU Munchen


29/45


det(A B) = det(A) det(B).

det

A1

= det(A)1.

det AT = det(A).


TU Munchen


30/45


Determine the solution of the linear system

2x1 + x2 = 42x2 + x3 = 0

x1 + x2 + x3 = 3

with the help of determinants.


TU Munchen


31/45


Determine the solution of the linear system

2x1 + x2 = 42x2 + x3 = 0

x1 + x2 + x3 = 3with the help of determinants.

x1 =

4 1 00 2 13 1 1

2 1 00 2 11 1 1

= 73

; x2 =

2 4 00 0 11 3 1

2 1 00 2 11 1 1

= 73

; x3 =

2 1 40 2 01 1 3

2 1 00 2 11 1 1

= 43

.


TU Munchen


32/45

Eigenvalues

notions of eigenvalue, eigenvector, and spectrum

similar matrices A,B:S : B = SAS1

(i.e.: A and B as two basis representations of the same endomorphism)

resulting objective: look for the best / cheapest representation (diagonal form)

important: matrix A is diagonalizable iff there is a basis consisting of

eigenvectors only

characteristic polynomial, its roots are the eigenvalues

Jordan normal form

important:

spectrum characterizes a matrix

many situations / applications where eigenvalues are crucial


TU Munchen


33/45

Exercise Eigenvalues

Diagonalize the matrix

3 22 3

. Give both eigenvalues and

eigenvectors and the basis transformation matrix transformingthe given matrix in diagonal form.


TU Munchen


34/45

Exercise Eigenvalues Solution


3 22 3


eigenvectors and the basis transformation matrix transforming

the given matrix in diagonal form.

Eigenvalues:

3 22 3

= 9 6 + 2 4 = 5 6 + 2

1,2 =63620

2= 3 2 1 = 5, 2 = 1.


TU Munchen


35/45



3 22 3


eigenvectors and the basis transformation matrix transforming

the given matrix in diagonal form.

Eigenvector for 1 = 5:

2 2

2 2

xy

=

00

x = y x1 =

11

Eigenvector for 2 = 1: 2 2

2 2 x

y

= 0

0

x = y x2 = 1

1


TU Munchen


36/45



3 22 3


eigenvectors and the basis transformation matrix transformingthe given matrix in diagonal form.

The basis transformation matrix thus is

1 11 1

and results in the diagonal matrix

5 00 1

.


TU Munchen


37/45

Scalar Products and Vector Norms

notions of a linear form and a bilinear form

scalar product: a positive-definite symmetric bilinear form

examples of vector spaces and scalar products

vector norms:

definition: positivity, homogeneity, triangle inequality

meaning of triangle inequality

examples: Euclidean, maximum, and sum norm

normed vector spaces

Cauchy-Schwarz inequality

notions of orthogonality and orthonormality

turning a basis into an orthonormal one: Gram-Schmidt orthogonalization


TU Munchen


38/45

Exercise Scalar Products and Vector Norms

Are the following operators scalar products in the vector spaceof continuous functions on the interval [a; b]?

f, g1 :=

ba f(x) g(x)dx

f, g2 :=

b

af(x) g(x)2dx

f, g3 :=b

af+(x) g(x)dx


TU Munchen


39/45

Exercise Scalar Products and Vector Norms Solution

Are the following operators scalar products in the vector spaceof continuous functions on the interval [a; b]?

f, g1 :=

ba f(x) g(x)dx Yes!

f, g2 :=

b

af(x) g(x)2dx No! (not linear in g)

f, g3 :=b

af+(x) g(x)dx No! (not positive definite)



TU Munchen


40/45


Proof that a set {x1, x2, . . . , xN} of non-zero orthogonal vectorsin a vector space with scalar product (, ) always is a basis ofits span.



TU Munchen


41/45


Proof that a set {x1, x2, . . . , xN} of non-zero orthogonal vectorsin a vector space with scalar product (, ) always is a basis ofits span.

Proof by contradiction:

Assume that the set is not linearly independent. Then, there is a element xi taht can bewritten as a linear combination xi =

kIkxk of other elements, where the index set

I {1,2, . . . ,N} does not contain i. With this, we get

0 = (xi, xi) = xi,kIkxk = k

Ik(xi, xk) = 0.

Contradiction. Thus, the vector set is linearly independent and, thus, is a basis of itsspan.



TU Munchen


42/45


Transform

111

,

110

,

100

into an orthogonal basis

of R3.



TU Munchen

E i S l P d d V N S l i


43/45


Transform 1

11

, 1

10

, 1

00

into an orthogonal basis

of R3.

Gram-Schmidt orthogonalization:

x1 =

11

1

, x2 =

11

0

x1,

110

(x1,x1)x1 =

11

0

2

3x1 =

1313

23

,

x3 =

10

0

x1,

100

(x1,x1)x1

x2,

100

(x2,x2)x2 =

715

815

115

.



TU Munchen

M t i N


44/45

Matrix Norms

definition:

properties corresponding to those of vector norms plus sub-multiplicativity:

AB A B

plus consistencyAx A x

matrix norms can be induced from corresponding vector norms: Euclidean,maximum, sum

A := maxx=1

Ax

alternative: completely new definition, for example Frobenius norm (considermatrix as a vector, then take Euclidean norm)



TU Munchen

Cl f M t i


45/45

Classes of Matrices

symmetric: A = AT

skew-symmetric: A = AT

Hermitian: A = AH = AT

s.p.d. (symmetric positive definite): xTAx > 0 x = 0

orthogonal: A1 = AT (the whole spectrum has modulus 1)

unitary: A1 = AH (the whole spectrum has modulus 1)

normal: AAT = ATA or AAH = AHA, resp. (for those and only those matricesthere exists an orthonormal basis of eigenvectors)



Documents

02 Handout Linear Algebra