1. Foundations of Numerics from Advanced Mathematics ... · another is a Cube minipulation, again....

TU Munchen

1. Foundations of Numerics from Advanced Mathematics

Linear Algebra

Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics

Linear Algebra, October 23, 2012 1

TU Munchen

1.1. Linear Algebra

Mathematical Structures

• a mathematical structure consists of one or several sets and one or severaloperations defined on the set(s)

• special elements:

– neutral element (of an operation)– inverse element (of some element x)

• a group: a structure to add and subtract

• a field: a structure to add, subtract, multiply, and divide

• a vector space: a set of vectors over a field with two operations: scalarmultiplication, addition of vectors, obeying certain axioms (which?)

• note: sometimes, the association with classical (geometric) vectors is helpful,sometimes it is more harmful

TU Munchen

Exercise Mathematical Structures

Show that the possible manipulations ofthe Rubik’s Cube with the operation ’ex-ecute after’ are a group.

TU Munchen

Exercise Mathematical Structures – Solution

Show that the possible manipulations ofthe Rubik’s Cube with the operation ’ex-ecute after’ are a group.

• Closure: executing any two manipulations after oneanother is a Cube minipulation, again.

• Associativity: the result of a sequence of threemanipulations is obviously always the same no matter howyou group them (the first two or the last two together).

• Identity: obviously included (just ’do nothing’).• Invertibility: execute a manipulation in backward direction.

TU Munchen

Show that the rational numbers with the operations + (add) and∗ (multiply) are a field.

TU Munchen

Show that the rational numbers with the operations + (add) and∗ (multiply) are a field.

• Closure: obviously closed under + and ∗.• Identity: 0 for +, 1 for ∗.• Invertibility: each element q has an inverse −q under +

and 1q under ∗. The latter holds for all elements except from

the neutral element of +, i.e., 0.• Associativity: well-known for both + and ∗.• Commutativity: also known from school (a + b = b + a,

a ∗ b = b ∗ a).• Distributivity: dito (a ∗ (b + c) = a ∗ b + a ∗ c).

TU Munchen

Is the set of N ×N matrices (N ∈ N) matrices with real numbersas entries over the field of real numbers a vector space?

TU Munchen

Is the set of N ×N matrices (N ∈ N) matrices with real numbersas entries over the field of real numbers a vector space?

The answer is yes. Look up the axioms and show that they holdfor the xample on your own.

TU Munchen

Vector Spaces

• a linear combination of vectors

• linear (in)dependence of a set of vectors

• the span of a set of vectors

• a basis of a vector space

– definition?– why do we need a basis?– is a vector’s basis representation unique?– is there only one basis for a vector space?

• the dimension of a vector space

• does infinite dimensionality exist?

• important applications:

– (analytic) geometry– numerical and functional analysis: function spaces are vector spaces

(frequently named after mathematicians: Banach spaces, Hilbert spaces,Sobolev spaces, ...)

TU Munchen

Exercise Vector Spaces

Is the set of vectors{(

)}linearly

independent?

TU Munchen

Exercise Vector Spaces – Solution

Is the set of vectors{(

)}linearly

independent?

The set of vectors is not linearly independent, since the thirdelement can easily be written as a linear combination of the firsttwo:(

)= 1 ·

)+ 3 ·

TU Munchen

{( 100

TU Munchen

{( 100

{( a0b

);a,b ∈ R

TU Munchen

Consider the set of all possible polynomials with realcoefficients as a vector space over the field of real numbers.What’s the dimension of this space? Give a basis.

TU Munchen

Consider the set of all possible polynomials with realcoefficients as a vector space over the field of real numbers.What’s the dimension of this space? Give a basis.

The space is infinite dimensional, a basis is for example{1, x , x2, x3, . . .

TU Munchen

Linear Mappings

• definition in the vector space context; notion of a homomorphism

• image and kernel of a homomorphism

• matrices, transposed and Hermitian of a matrix

• relations of matrices and homomorphisms

• meaning of injective, surjective, and bijective for a matrix; rank of a matrix

• meaning of the matrix columns for the underlying mapping

• matrices and systems of linear equations

• basis transformation and coordinate transformation

• mono-, epi-, iso-, endo-, and automorphisms

TU Munchen

Exercise Linear Mappings

Is the mapping f : R3 → R3, ~x 7→ 5 · ~x +

)linear?

TU Munchen

Exercise Linear Mappings – Solution

Is the mapping f : R3 → R3, ~x 7→ 5 · ~x +

)linear?

f is not linear, since f (α~x) 6= αf (~x).

TU Munchen

What’s the linear mapping f : R2 → R2 corresponding to the

matrix(

4 03 2

TU Munchen

What’s the linear mapping f : R2 → R2 corresponding to the

matrix(

4 03 2

3x + 2y

TU Munchen

Give the rank of the matrix

1 0 0 00 1 0 00 0 0 00 0 0 1

Is the corresponding linear mapping injective, surjective,bijective?

TU Munchen

Give the rank of the matrix

1 0 0 00 1 0 00 0 0 00 0 0 1

Is the corresponding linear mapping injective, surjective,bijective?

The rank is three. Thus, the corresponding linear mapping isneither injective, nor surjective or bijective.

TU Munchen

Examples Linear Mappings

Monomorphism:

( 0 10 01 0

Epimorphism:

( 1 0 00 1 00 0 0

Iso-/Automorphism:(

0 11 0

Endomorphism:

( 2 1 00 1 21 0 1

TU Munchen

Determinants

• definition

• properties

• meaning

• occurrences

• Cramer’s rule

TU Munchen

Determinants – Definition

det(A) =

∣∣∣∣∣∣∣∣∣a1,1 a1,2 · · · a1,Na2,1 a2,2 · · · a2,N

.... . .

...aN,1 aN,N

∣∣∣∣∣∣∣∣∣ =

∣∣∣∣∣∣∣∣∣∣∣

a2,2 · · · · · · a2,N...

......

...aN,2 · · · · · · aN,N

∣∣∣∣∣∣∣∣∣∣∣− a1,2

∣∣∣∣∣∣∣∣∣a2,1 a2,3 · · · a2,Na3,1 a3,N

......

aN,1 aN,3 · · · aN,N

∣∣∣∣∣∣∣∣∣+ . . .

TU Munchen

Exercise Determinants

det(A) = 0⇒ A defines a . . .morphism.

det(A) 6= 0⇒ A defines a . . .morphism.

TU Munchen

Exercise Determinants – Solution

det(A) = 0⇒ A defines an Endomorphism.

det(A) 6= 0⇒ A defines an Automorphism.

TU Munchen

det(A · B) =?

det(A−1) =?

det(AT ) =?

TU Munchen

det(A · B) = det(A) · det(B).

det(A−1) = det(A)−1.

det(AT ) = det(A).

TU Munchen

Determine the solution of the linear system

2x1 + x2 = 42x2 + x3 = 0

x1 + x2 + x3 = 3

with the help of determinants.

TU Munchen

Determine the solution of the linear system

2x1 + x2 = 42x2 + x3 = 0

x1 + x2 + x3 = 3

with the help of determinants.

∣∣∣∣∣∣∣∣4 1 00 2 13 1 1

∣∣∣∣∣∣∣∣∣∣∣∣∣∣∣∣2 1 00 2 11 1 1

∣∣∣∣∣∣∣∣= 7

3 ; x2 =

∣∣∣∣∣∣∣∣2 4 00 0 11 3 1

∣∣∣∣∣∣∣∣∣∣∣∣∣∣∣∣2 1 00 2 11 1 1

∣∣∣∣∣∣∣∣= −7

3 ; x3 =

∣∣∣∣∣∣∣∣2 1 40 2 01 1 3

∣∣∣∣∣∣∣∣∣∣∣∣∣∣∣∣2 1 00 2 11 1 1

∣∣∣∣∣∣∣∣= 4

TU Munchen

Eigenvalues

• notions of eigenvalue, eigenvector, and spectrum

• similar matrices A,B:∃S : B = SAS−1

(i.e.: A and B as two basis representations of the same endomorphism)

• resulting objective: look for the best / cheapest representation (diagonal form)

• important: matrix A is diagonalizable iff there is a basis consisting ofeigenvectors only

• characteristic polynomial, its roots are the eigenvalues

• Jordan normal form

• important:

– spectrum characterizes a matrix– many situations / applications where eigenvalues are crucial

TU Munchen

Exercise Eigenvalues

Diagonalize the matrix(

3 22 3

). Give both eigenvalues and

eigenvectors and the basis transformation matrix transformingthe given matrix in diagonal form.

TU Munchen

Exercise Eigenvalues – Solution

3 22 3

Eigenvalues:∣∣∣∣ 3− λ 22 3− λ

∣∣∣∣ = 9− 6λ+ λ2 − 4 = 5− 6λ+ λ2

⇒ λ1,2 = 6±√

36−202 = 3± 2⇒ λ1 = 5, λ2 = 1.

TU Munchen

3 22 3

Eigenvector for λ1 = 5:(−2 22 −2

)⇔ x = y ⇒ ~x1 =

)Eigenvector for λ2 = 1:

(2 22 2

)⇔ x = −y ⇒ ~x2 =

(1−1

TU Munchen

3 22 3

The basis transformation matrix thus is(

1 11 −1

)and results in the diagonal matrix(

5 00 1

TU Munchen

Scalar Products and Vector Norms

• notions of a linear form and a bilinear form

• scalar product: a positive-definite symmetric bilinear form

• examples of vector spaces and scalar products

• vector norms:

– definition: positivity, homogeneity, triangle inequality– meaning of triangle inequality– examples: Euclidean, maximum, and sum norm

• normed vector spaces

• Cauchy-Schwarz inequality

• notions of orthogonality and orthonormality

• turning a basis into an orthonormal one: Gram-Schmidt orthogonalization

TU Munchen

Exercise Scalar Products and Vector Norms

Are the following operators scalar products in the vector spaceof continuous functions on the interval [a;b]?

〈f ,g〉1 :=∫ b

a f (x) · g(x)dx

〈f ,g〉2 :=∫ b

a f (x) · g(x)2dx

〈f ,g〉3 :=∫ b

a f +(x) · g(x)dx

TU Munchen

Exercise Scalar Products and Vector Norms – Solution

Are the following operators scalar products in the vector spaceof continuous functions on the interval [a;b]?

〈f ,g〉1 :=∫ b

a f (x) · g(x)dx Yes!

〈f ,g〉2 :=∫ b

a f (x) · g(x)2dx No! (not linear in g)

〈f ,g〉3 :=∫ b

a f +(x) · g(x)dx No! (not positive definite)

TU Munchen

Proof that a set {~x1, ~x2, . . . , ~xN} of non-zero orthogonal vectorsin a vector space with scalar product (·, ·) always is a basis ofits span.

TU Munchen

Proof that a set {~x1, ~x2, . . . , ~xN} of non-zero orthogonal vectorsin a vector space with scalar product (·, ·) always is a basis ofits span.

Proof by contradiction:

Assume that the set is not linearly independent. Then, there is a element ~xi taht can bewritten as a linear combination ~xi =

∑k∈I αk~xk of other elements, where the index set

I ⊂ {1, 2, . . . ,N} does not contain i . With this, we get

0 6= (~xi , ~xi ) =(~xi ,∑

k∈I αk~xk)

k∈I αk (~xi , ~xk ) = 0.

Contradiction. Thus, the vector set is linearly independent and, thus, is a basis of itsspan. �

TU Munchen

Transform

{( 111

)}into an orthogonal basis

of R3.

TU Munchen

Transform

{( 111

)}into an orthogonal basis

of R3.

Gram-Schmidt orthogonalization:

, ~x2 =

−~x1,

(~x1,~x1)~x1 =

− 23~x1 =

1313− 2

−~x1,

(~x1,~x1)~x1 −

(~x2,~x2)~x2 =

715− 8

TU Munchen

Matrix Norms

• definition:

– properties corresponding to those of vector norms– plus sub-multiplicativity:

‖AB‖ ≤ ‖A‖ · ‖B‖

– plus consistency‖Ax‖ ≤ ‖A‖ · ‖x‖

• matrix norms can be induced from corresponding vector norms: Euclidean,maximum, sum

‖A‖ := max‖x‖=1

‖Ax‖

• alternative: completely new definition, for example Frobenius norm (considermatrix as a vector, then take Euclidean norm)

TU Munchen

Classes of Matrices

• symmetric: A = AT

• skew-symmetric: A = −AT

• Hermitian: A = AH = AT

• s.p.d. (symmetric positive definite): xT Ax > 0 ∀x 6= 0

• orthogonal: A−1 = AT (the whole spectrum has modulus 1)

• unitary: A−1 = AH (the whole spectrum has modulus 1)

• normal: AAT = AT A or AAH = AHA, resp. (for those and only those matricesthere exists an orthonormal basis of eigenvectors)

1. Foundations of Numerics from Advanced Mathematics ... · another is a Cube minipulation, again....

Documents

The V-Way Cache: Demand-Based Associativity via Global

Importing Foreign Data and Maintaining Associativity

Title: The role of extension in strengthening the associativity to … · 2019-07-15 · Presentation Outline “CFCS 55rd Meeting – Strengthening the associativity to export high

Adjoint associativity: an invitation to algebra in -categorieslibrary.msri.org/books/Book68/files/150125-Lipman.pdfAdjoint associativity: an invitation to algebra in 1-categories JOSEPH

1 CMPE 421 Parallel Computer Architecture PART4 Caching with Associativity

Reducing Power Consumption for High-Associativity Data ...Title Reducing Power Consumption for High-Associativity Data Caches in Embedded Processors Author Dan Nicolaescu, Alex Veidenbaum,

The Complexity of the Minimum Cost Homomorphism Problem ...kim/journals/minhomsemic2010.pdf · The Complexity of the Minimum Cost Homomorphism Problem for Semicomplete Digraphs with

Johnson-Morita homomorphism and homological ﬁbered knots

Cache Associativity

Evaluating Associativity in CPU Cacheszz124/cs671_fall2013/lectures/jan_ca.pdf · Evaluating Associativity in CPU Caches Mark D. Hill, Alan Jay Smith IEEE Transactions on Computers(1989)

Elliptic curves over finite fields - Roma Tre · Elliptic curves over F q Formulas for Addition Computer assited proof of associativity ... Pappus Theorem Pascal’s Theorem Associativity

The Complexity of Parity Graph Homomorphism: An Initial ...regev/toc/articles/v011a002/v011a002.pdf · THE COMPLEXITY OF PARITY GRAPH HOMOMORPHISM: AN INITIAL INVESTIGATION the number

Parsing: Derivations, Ambiguity, Precedence, Associativity

Associativity between Weak and Strict Sequencing

Ambiguity, Precedence, Associativity & Top-Down Parsing

Associativity and other Wurban Things

ASSOCIATIVITY AND COOPERATION BASED ON THE POPULAR …

Associativity of the Secant Method

Skewed associativity enhances performance predictability

HOMOTOPY ASSOCIATIVITY OF ^-SPACES. II · 2018. 11. 16. · HOMOTOPY ASSOCIATIVITY OF ^-SPACES. II BY JAMES DILLON STASHEFF(i) 1. Introduction. This paper is a sequal to "Homotopy