1. Foundations of Numerics from Advanced Mathematics ... · another is a Cube minipulation, again. Associativity: the result of a sequence of three ... image and kernel of a homomorphism

TU Munchen

1. Foundations of Numerics from Advanced Mathematics

Linear Algebra

Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics

Linear Algebra, October 23, 2012 1

http://www5.in.tum.de/

TU Munchen

1.1. Linear Algebra

Mathematical Structures

• a mathematical structure consists of one or several sets and one or severaloperations defined on the set(s)

• special elements:

– neutral element (of an operation)– inverse element (of some element x)

• a group: a structure to add and subtract

• a field: a structure to add, subtract, multiply, and divide

• a vector space: a set of vectors over a field with two operations: scalarmultiplication, addition of vectors, obeying certain axioms (which?)

• note: sometimes, the association with classical (geometric) vectors is helpful,sometimes it is more harmful




TU Munchen

Exercise Mathematical Structures

Show that the possible manipulations ofthe Rubik’s Cube with the operation ’ex-ecute after’ are a group.




TU Munchen

Exercise Mathematical Structures – Solution

Show that the possible manipulations ofthe Rubik’s Cube with the operation ’ex-ecute after’ are a group.

• Closure: executing any two manipulations after oneanother is a Cube minipulation, again.

• Associativity: the result of a sequence of threemanipulations is obviously always the same no matter howyou group them (the first two or the last two together).

• Identity: obviously included (just ’do nothing’).• Invertibility: execute a manipulation in backward direction.




TU Munchen


Show that the rational numbers with the operations + (add) and∗ (multiply) are a field.




TU Munchen


Show that the rational numbers with the operations + (add) and∗ (multiply) are a field.

• Closure: obviously closed under + and ∗.• Identity: 0 for +, 1 for ∗.• Invertibility: each element q has an inverse −q under +

and 1q under ∗. The latter holds for all elements except from

the neutral element of +, i.e., 0.• Associativity: well-known for both + and ∗.• Commutativity: also known from school (a + b = b + a,

a ∗ b = b ∗ a).• Distributivity: dito (a ∗ (b + c) = a ∗ b + a ∗ c).




TU Munchen


Is the set of N ×N matrices (N ∈ N) matrices with real numbersas entries over the field of real numbers a vector space?




TU Munchen


Is the set of N ×N matrices (N ∈ N) matrices with real numbersas entries over the field of real numbers a vector space?

The answer is yes. Look up the axioms and show that they holdfor the xample on your own.




TU Munchen

Vector Spaces

• a linear combination of vectors

• linear (in)dependence of a set of vectors

• the span of a set of vectors

• a basis of a vector space

– definition?– why do we need a basis?– is a vector’s basis representation unique?– is there only one basis for a vector space?

• the dimension of a vector space

• does infinite dimensionality exist?

• important applications:

– (analytic) geometry– numerical and functional analysis: function spaces are vector spaces

(frequently named after mathematicians: Banach spaces, Hilbert spaces,Sobolev spaces, ...)




TU Munchen

Exercise Vector Spaces

Is the set of vectors{(

10

),

(01

),

(13

)}linearly

independent?




TU Munchen

Exercise Vector Spaces – Solution

Is the set of vectors{(

10

),

(01

),

(13

)}linearly

independent?

The set of vectors is not linearly independent, since the thirdelement can easily be written as a linear combination of the firsttwo:(

13

)= 1 ·

(10

)+ 3 ·

(01

).




TU Munchen


span

{( 100

),

( 001

)}= ?




TU Munchen


span

{( 100

),

( 001

)}=

{( a0b

);a,b ∈ R

}.




TU Munchen


Consider the set of all possible polynomials with realcoefficients as a vector space over the field of real numbers.What’s the dimension of this space? Give a basis.




TU Munchen


Consider the set of all possible polynomials with realcoefficients as a vector space over the field of real numbers.What’s the dimension of this space? Give a basis.

The space is infinite dimensional, a basis is for example{1, x , x2, x3, . . .

}.




TU Munchen

Linear Mappings

• definition in the vector space context; notion of a homomorphism

• image and kernel of a homomorphism

• matrices, transposed and Hermitian of a matrix

• relations of matrices and homomorphisms

• meaning of injective, surjective, and bijective for a matrix; rank of a matrix

• meaning of the matrix columns for the underlying mapping

• matrices and systems of linear equations

• basis transformation and coordinate transformation

• mono-, epi-, iso-, endo-, and automorphisms




TU Munchen

Exercise Linear Mappings

Is the mapping f : R3 → R3, ~x 7→ 5 · ~x +

( 123

)linear?




TU Munchen

Exercise Linear Mappings – Solution

Is the mapping f : R3 → R3, ~x 7→ 5 · ~x +

( 123

)linear?

f is not linear, since f (α~x) 6= αf (~x).




TU Munchen


What’s the linear mapping f : R2 → R2 corresponding to the

matrix(

4 03 2

)?




TU Munchen


What’s the linear mapping f : R2 → R2 corresponding to the

matrix(

4 03 2

)?

f((

xy

))=

(4x

3x + 2y

).




TU Munchen


Give the rank of the matrix

1 0 0 00 1 0 00 0 0 00 0 0 1

.

Is the corresponding linear mapping injective, surjective,bijective?




TU Munchen


Give the rank of the matrix

1 0 0 00 1 0 00 0 0 00 0 0 1

.

Is the corresponding linear mapping injective, surjective,bijective?

The rank is three. Thus, the corresponding linear mapping isneither injective, nor surjective or bijective.




TU Munchen

Examples Linear Mappings

Monomorphism:

( 0 10 01 0

)

Epimorphism:

( 1 0 00 1 00 0 0

)

Iso-/Automorphism:(

0 11 0

)

Endomorphism:

( 2 1 00 1 21 0 1

).




TU Munchen

Determinants

• definition

• properties

• meaning

• occurrences

• Cramer’s rule




TU Munchen

Determinants – Definition

det(A) =

∣∣∣∣∣∣∣∣∣a1,1 a1,2 · · · a1,Na2,1 a2,2 · · · a2,N

.... . .

...aN,1 aN,N

∣∣∣∣∣∣∣∣∣ =

a1,1

∣∣∣∣∣∣∣∣∣∣∣

a2,2 · · · · · · a2,N...

......

...aN,2 · · · · · · aN,N

∣∣∣∣∣∣∣∣∣∣∣− a1,2

∣∣∣∣∣∣∣∣∣a2,1 a2,3 · · · a2,Na3,1 a3,N

......

aN,1 aN,3 · · · aN,N

∣∣∣∣∣∣∣∣∣+ . . .




TU Munchen

Exercise Determinants

det(A) = 0⇒ A defines a . . .morphism.

det(A) 6= 0⇒ A defines a . . .morphism.




TU Munchen

Exercise Determinants – Solution

det(A) = 0⇒ A defines an Endomorphism.

det(A) 6= 0⇒ A defines an Automorphism.




TU Munchen


det(A · B) =?

det(A−1) =?

det(AT ) =?




TU Munchen


det(A · B) = det(A) · det(B).

det(A−1) = det(A)−1.

det(AT ) = det(A).




TU Munchen


Determine the solution of the linear system

2x1 + x2 = 42x2 + x3 = 0

x1 + x2 + x3 = 3

with the help of determinants.




TU Munchen


Determine the solution of the linear system

2x1 + x2 = 42x2 + x3 = 0

x1 + x2 + x3 = 3

with the help of determinants.

x1 =

∣∣∣∣∣∣∣∣4 1 00 2 13 1 1

∣∣∣∣∣∣∣∣∣∣∣∣∣∣∣∣2 1 00 2 11 1 1

∣∣∣∣∣∣∣∣= 7

3 ; x2 =

∣∣∣∣∣∣∣∣2 4 00 0 11 3 1

∣∣∣∣∣∣∣∣∣∣∣∣∣∣∣∣2 1 00 2 11 1 1

∣∣∣∣∣∣∣∣= −7

3 ; x3 =

∣∣∣∣∣∣∣∣2 1 40 2 01 1 3

∣∣∣∣∣∣∣∣∣∣∣∣∣∣∣∣2 1 00 2 11 1 1

∣∣∣∣∣∣∣∣= 4

3 .




TU Munchen

Eigenvalues

• notions of eigenvalue, eigenvector, and spectrum

• similar matrices A,B:∃S : B = SAS−1

(i.e.: A and B as two basis representations of the same endomorphism)

• resulting objective: look for the best / cheapest representation (diagonal form)

• important: matrix A is diagonalizable iff there is a basis consisting ofeigenvectors only

• characteristic polynomial, its roots are the eigenvalues

• Jordan normal form

• important:

– spectrum characterizes a matrix– many situations / applications where eigenvalues are crucial




TU Munchen

Exercise Eigenvalues

Diagonalize the matrix(

3 22 3

). Give both eigenvalues and

eigenvectors and the basis transformation matrix transformingthe given matrix in diagonal form.




TU Munchen

Exercise Eigenvalues – Solution


3 22 3



Eigenvalues:∣∣∣∣ 3− λ 22 3− λ

∣∣∣∣ = 9− 6λ+ λ2 − 4 = 5− 6λ+ λ2

⇒ λ1,2 = 6±√

36−202 = 3± 2⇒ λ1 = 5, λ2 = 1.




TU Munchen



3 22 3



Eigenvector for λ1 = 5:(−2 22 −2

)(xy

)=

(00

)⇔ x = y ⇒ ~x1 =

(11

)Eigenvector for λ2 = 1:

(2 22 2

)(xy

)=

(00

)⇔ x = −y ⇒ ~x2 =

(1−1

)




TU Munchen



3 22 3



The basis transformation matrix thus is(

1 11 −1

)and results in the diagonal matrix(

5 00 1

).




TU Munchen

Scalar Products and Vector Norms

• notions of a linear form and a bilinear form

• scalar product: a positive-definite symmetric bilinear form

• examples of vector spaces and scalar products

• vector norms:

– definition: positivity, homogeneity, triangle inequality– meaning of triangle inequality– examples: Euclidean, maximum, and sum norm

• normed vector spaces

• Cauchy-Schwarz inequality

• notions of orthogonality and orthonormality

• turning a basis into an orthonormal one: Gram-Schmidt orthogonalization




TU Munchen

Exercise Scalar Products and Vector Norms

Are the following operators scalar products in the vector spaceof continuous functions on the interval [a;b]?

〈f ,g〉1 :=∫ b

a f (x) · g(x)dx

〈f ,g〉2 :=∫ b

a f (x) · g(x)2dx

〈f ,g〉3 :=∫ b

a f +(x) · g(x)dx




TU Munchen

Exercise Scalar Products and Vector Norms – Solution

Are the following operators scalar products in the vector spaceof continuous functions on the interval [a;b]?

〈f ,g〉1 :=∫ b

a f (x) · g(x)dx Yes!

〈f ,g〉2 :=∫ b

a f (x) · g(x)2dx No! (not linear in g)

〈f ,g〉3 :=∫ b

a f +(x) · g(x)dx No! (not positive definite)




TU Munchen


Proof that a set {~x1, ~x2, . . . , ~xN} of non-zero orthogonal vectorsin a vector space with scalar product (·, ·) always is a basis ofits span.




TU Munchen


Proof that a set {~x1, ~x2, . . . , ~xN} of non-zero orthogonal vectorsin a vector space with scalar product (·, ·) always is a basis ofits span.

Proof by contradiction:

Assume that the set is not linearly independent. Then, there is a element ~xi taht can bewritten as a linear combination ~xi =

∑k∈I αk~xk of other elements, where the index set

I ⊂ {1, 2, . . . ,N} does not contain i . With this, we get

0 6= (~xi , ~xi ) =(~xi ,∑

k∈I αk~xk)

=∑

k∈I αk (~xi , ~xk ) = 0.

Contradiction. Thus, the vector set is linearly independent and, thus, is a basis of itsspan. �




TU Munchen


Transform

{( 111

),

( 110

),

( 100

)}into an orthogonal basis

of R3.




TU Munchen


Transform

{( 111

),

( 110

),

( 100

)}into an orthogonal basis

of R3.

Gram-Schmidt orthogonalization:

~x1 =

111

, ~x2 =

110

−~x1,

110

(~x1,~x1)~x1 =

110

− 23~x1 =

1313− 2

3

,

~x3 =

100

−~x1,

100

(~x1,~x1)~x1 −

~x2,

100

(~x2,~x2)~x2 =

715− 8

151

15

.




TU Munchen

Matrix Norms

• definition:

– properties corresponding to those of vector norms– plus sub-multiplicativity:

‖AB‖ ≤ ‖A‖ · ‖B‖

– plus consistency‖Ax‖ ≤ ‖A‖ · ‖x‖

• matrix norms can be induced from corresponding vector norms: Euclidean,maximum, sum

‖A‖ := max‖x‖=1

‖Ax‖

• alternative: completely new definition, for example Frobenius norm (considermatrix as a vector, then take Euclidean norm)




TU Munchen

Classes of Matrices

• symmetric: A = AT

• skew-symmetric: A = −AT

• Hermitian: A = AH = AT

• s.p.d. (symmetric positive definite): xT Ax > 0 ∀x 6= 0

• orthogonal: A−1 = AT (the whole spectrum has modulus 1)

• unitary: A−1 = AH (the whole spectrum has modulus 1)

• normal: AAT = AT A or AAH = AHA, resp. (for those and only those matricesthere exists an orthonormal basis of eigenvectors)




Documents

1. Foundations of Numerics from Advanced Mathematics ... · another is a Cube minipulation, again. Associativity: the result of a sequence of three ... image and kernel of a homomorphism