Linear Algebrace.sharif.edu/courses/99-00/1/ce425-1/resources/root/...Introduction to Linear Algebra. 6.5. • Gilbert Strang. Introduction to Linear Algebra. 10.2. Hamid Reza Rabiee

Linear Algebra

Slide 19:Positive Definite Matrices and Matrix in Engineering

Fall 2020

Sharif University of Technology

Fall 2020Hamid Reza Rabiee

Positive Definite Matrices


Outline

• Positive Definite Matrices

• Positive Semidefinite Matrices

• Axes of Ellipse, Eigenvalues and Eigenvectors


Positive Definite Matrices

• If symmetry makes a matrix important, this property (all 𝜆> 0) makes it truly special.

• Symmetric matrices with positive eigenvalues are called positive

definite.

• How to find positive definite matrices?

× Find all the eigenvalues and test 𝜆 > 0✓ Find quick tests on a symmetric matrix that guarantee positive

eigenvalues.


Positive Definite Matrices(contd.)

Recap.

• Every eigenvalue is real because the matrix is symmetric.

Start with 2 by 2. When does have 𝜆1 > 0 and 𝜆2> 0?• Test 1:

The eigenvalues of S are positive if and only if a > 0 and 𝑎𝑐 − 𝑏2 > 0

• Example.

is not positive definite because 𝑎𝑐 − 𝑏2 = 1 − 4 < 0



Proof.

• The 2 by 2 test is passed when 𝜆1 > 0 and 𝜆2 > 0. Their product 𝜆1𝜆2 is the determinant so 𝑎𝑐 − 𝑏

2 > 0. Their sum 𝜆1+ 𝜆2 is the trace so 𝑎 + 𝑐 > 0. Then 𝑎 and 𝑐 are both positive.(why?)

• This test uses the 1 by 1 determinant 𝑎 and the 2 by 2 determinant 𝑎𝑐 − 𝑏2. When 𝑆 is 3 by 3, det 𝑆 > 0 is the third part of the test.

• In general, a matrix is positive definite if and only if all n

upper left determinants are positive.



3 by 3 example.

• Eigenvalues are 1, 1, 4 • Determinants are 2, 3, 4

• Pivots 2 and 3

2and

4

3

𝑆 − 𝐼 will be semidefinite: eigenvalues 0, 0, 3



• Test 3:

S is positive definite it 𝑥𝑇𝑆𝑥 > 0 for every nonzero eigenvector x:

• From 𝑆𝑥 = 𝜆x, multiply by 𝑥𝑇 to get 𝑥𝑇𝑆𝑥 = 𝜆𝑥𝑇𝑥.• If 𝜆 is positive then right-hand side is also positive. (why?)• So the left side 𝑥𝑇𝑆𝑥 is positive for any eigenvector.➢ The number 𝑥𝑇𝑆𝑥 is called the energy in the system, and has

many applications.



Theorem.

• If 𝑆 and 𝑇 are symmetric positive definite, so is 𝑆 + 𝑇Proof.

• 𝑥𝑇 𝑆 + 𝑇 𝑥 = 𝑥𝑇𝑆𝑥 + 𝑥𝑇𝑇𝑥• Both right terms are positive(why?) so 𝑆 + 𝑇 is also positive

definite.



• Test 4:

• If the columns of 𝐴 are independent, then S = 𝐴𝑇𝐴 is positive definite.

𝑥𝑇𝑆𝑥 = 𝑥𝑇𝐴𝑇𝐴𝑥 = 𝐴𝑥 𝑇 𝐴𝑥 = 𝐴𝑥 2

• 𝐴𝑥 is not zero when 𝑥 ≠ 0 (why?)• 𝑥𝑇𝑆𝑥 is positive so S is positive definite.



Conclusion:

When a symmetric matrix S has one of these five properties, it has

them all:

• All 𝑛 pivots of 𝑆 are positive.• All 𝑛 upper left determinants are positive.• All 𝑛 eigenvalues of 𝑆 are positive.• 𝑥𝑇𝑆𝑥 is positive except at x = 0. This is the energy-based

definition.

• S equals 𝐴𝑇𝐴 for a matrix 𝐴 with independent columns.


Positive Semidefinite Matrices

• The determinant is zero.

• The smallest eigenvalue is zero.

• The energy in its eigenvector is 𝑥𝑇𝑆𝑥 = 𝑥𝑇0𝑥 = 0• These matrices, on the edge of being positive definite, are called

positive semidefinite.


Positive Semidefinite Matrices(contd.)

Example.

𝑆 has eigenvalues 5 and 0. Its upper left determinants are 1 and 0. Its rank is only 1. This matrix 𝑆 factors into 𝐴𝑇𝐴 with dependent columns in 𝐴:

Dependent columns in 𝐴Positive semidefinite 𝑆


Positive Semidefinite Matrices(contd.)

Example.

𝑇 also has zero determinant. T is positive semidefinite.The eigenvector x = (l, 1, 1) has 𝑇𝑥 = 0 (x is in nullspace of T) and energy 𝑥𝑇𝑇𝑥 = 0:


Axes of Ellipse, Eigenvalues and Eigenvectors

Think of a tilted ellipse 𝑥𝑇𝑆𝑥 = 1. Its center is (0, 0)

Turn it to line up with the coordinate axes (X and Y axes)


Axes of Ellipse and eigenvalues and eigenvectors(contd.)

Those two pictures show the geometry behind the factorization

S = QΛ𝑄−1 = QΛ𝑄𝑇

• The tilted ellipse is associated with S. Its equation is:

𝑥𝑇𝑆𝑥 = 1• The lined-up ellipse is associated with A. Its equation is:

𝑥𝑇Λ𝑥 = 1• The rotation matrix that lines up the ellipse is the eigenvector

matrix Q.


Axes of Ellipse, Eigenvalues and Eigenvectors(contd.)

Example.

• Find the axes of this tilted ellipse 5𝑥2 + 8𝑥𝑦 + 5𝑦2 = 1Sol.

• Start with the positive definite matrix that matches this

equation:

• The matrix is , and the eigenvectors are



Sol(contd.)

Divide by 2 for unit vectors. Then S = QΛ𝑄𝑇:

Now multiply by 𝑥 𝑦 on the left and 𝑥𝑦 on the right to get

𝑥𝑇𝑆𝑥 = (𝑥𝑇𝑄)Λ(𝑄𝑇𝑥)

So we have 𝑥𝑇𝑆𝑥 = 5𝑥2 + 8𝑥𝑦 + 5𝑦2 = 9(𝑥+𝑦

2)2+1(

𝑥−𝑦

2)2



Sol(contd.)

• The axes of the tilted ellipse point along those eigenvectors

S = QΛ𝑄𝑇 is called the "principal axis theorem “• It displays the axes. Not only the axis directions

(from the eigenvectors) but also the axis lengths

(from the eigenvalues):

Lined up 𝑥+𝑦

2=X and

𝑥−𝑦

2= 𝑌 and 9𝑥2 + 𝑦2 = 1



Sol(contd.)

S = QΛ𝑄𝑇 is positive definite when all 𝜆𝑖 > 0. The graph of S = 𝑥𝑇𝑆𝑥 = 1 is an ellipse

= 𝜆1𝑥2 + 𝜆2𝑦

2 = 1The axes point along eigenvectors of S. The half-lengths are 1

𝜆1and

1

𝜆2


Matrix in Engineering


Outline

• Differential Equation to Matrix Equation

• Differences Replace Derivatives

• Fixed End and Free End and Variable Coefficient c(x)

• Free-free Boundary Conditions

• A Line of Springs and Masses


Description

• We will see how engineering problems produce symmetric

matrices K (often K is positive definite)

• The "linear algebra reason" for symmetry and positive

definiteness is their form, 𝐾 = 𝐴𝑇𝐴 and 𝐾 = 𝐴𝑇𝐶𝐴

• The "physical reason" is that the expression 1

2𝑢𝑇𝐾𝑢

represents energy, and energy is never negative.


Differential Equation to Matrix Equation

• Consider the fixed boundary value problem:

−𝑑2𝑢

𝑑𝑥2= 1 where 𝑢 0 = 0 and 𝑢 1 =0

• Complete solution to −𝑑2𝑢

𝑑𝑥2= 1 is:

𝑢 𝑥 = −1

2𝑥2 + 𝐶 + 𝐷𝑥

• Using boundary conditions we shall get:

𝑢 𝑥 = −1

2𝑥2 +

1

2𝑥 =

1

2(𝑥 − 𝑥2)


Differences Replace Derivatives• To get matrices instead of derivatives, we have three basic choices:

forward or backward or centered differences.

• First derivatives and first differences

• Between 𝑥 = 0 and 𝑥 = 1, we divide the interval into 𝑛 + 1 equal

pieces. Each piece have width Δ𝑥 =1

(𝑛+1)

• The values of 𝑢 at breakpoints Δ𝑥, 2Δ𝑥,…• Solution to compute:

• Zero values 𝑢0 = 𝑢𝑛+1 = 0 come from the boundary conditions 𝑢 0 = 𝑢 1 = 0


Differences Replace Derivatives(contd.)

• 𝑛 = 3

• The equation is 𝐴𝑇𝐴𝑢 = (Δ𝑥)2𝑓

• For ∆𝑥 =1

4,

• Solutions to −𝑑2𝑢

𝑑𝑥2= 1 and K0𝑢 = (Δ𝑥)

2𝑓

with fixed-fixed boundaries, on a graph:


A Line of Springs and Masses

• 𝑢 = (𝑢1 + 𝑢2 + 𝑢3) = movements of masses (down is positive)• 𝑦 = 𝑦1 + 𝑦2 + 𝑦3 + 𝑦4 𝑜𝑟(𝑦1 + 𝑦2 + 𝑦3) = tensions in springs


Fixed End and Free End and Variable Coefficient c( x)

• −𝑑

𝑑𝑥1 + 𝑥

𝑑𝑢

𝑑𝑥= 𝑓(𝑥) with u(0) = 0 and

𝑑𝑢

𝑑𝑥(1) = 0

• ∆𝑥 =1

4: 𝑐(𝑥) = 1 + 𝑥 becomes a diagonal matrix 𝐶.

Which multiplies by 1 + ∆𝑥,… , 1 + 4∆𝑥 at the meshpoints:


Fixed End and Free End and Variable Coefficient c(x)

• K = 𝐴𝑇𝐶𝐴 will be symmetric and positive definite:


Free-free Boundary Conditions

• Now 𝑑𝑢

𝑑𝑥= 0 at both 𝑥 = 0 and 𝑥 = 1

• Free-free examples; unknown 𝑢0, 𝑢1, 𝑢2 𝑎𝑛𝑑 Δ𝑥 = 0.5

•

The vector (1, 1, 1) is in both null spaces. This matches 𝑢 𝑥 = 1 in the continuous problem.

• Free-free 𝐴𝑇𝐴𝑢 = 𝑓 and 𝐴𝑇𝐶𝐴𝑢 = 𝑓 are generally unsolvable.


Free-free Boundary Conditions(contd.)


A Line of Springs and Masses(contd.)• When a mass moves downward, its displacement is

positive (𝑢𝑗 > 0). For the springs, tension is positive and

compression is negative (𝑦𝑖 < 0). • In tension, the spring is stretched, so it pulls the masses

inward.

• (stretching force y) = (spring constant c) × (stretching distance e )

• Our job is to link these one-spring equations 𝑦 = 𝑐 𝑒 into a vector equation 𝐾𝑢 = 𝑓


A Line of Springs and Masses(contd.)• f = (𝑚1𝑔,𝑚2𝑔,𝑚3𝑔)• To find the stiffness matrix (fixed-fixed and fixed-free):

• Create K in three steps. Instead of connecting the

movements 𝑢𝑗 directly to 𝑓𝑖, it is much better to connect

each vector to the next in this list:

𝑢 = Movements of 𝑛 masses = (𝑢1, … , 𝑢𝑛)𝑒 = Elongations of 𝑚 springs = (𝑒1, … , 𝑒𝑛)𝑦 = Internal forces in 𝑚 springs = (𝑦1, … , 𝑦𝑛)𝑓 = External forces on 𝑛 masses = (𝑓1, … , 𝑓𝑛)


A Line of Springs and Masses(contd.)• We will prove the fixed-fixed case. The other one is almost

the same and we’ll leave it to you:

The masses move down by distances 𝑢1, 𝑢2, 𝑢3. Each spring is stretched or compressed by 𝑒𝑖 = 𝑢𝑖 − 𝑢𝑖−1𝑒1 = 𝑢1 , 𝑒2 = 𝑢2 − 𝑢1, 𝑒3 = 𝑢3 − 𝑢2, 𝑒4 = −𝑢3


A Line of Springs and Masses(contd.)

• If both ends move the same distance, that spring is not

stretched: 𝑢𝑖 = 𝑢𝑖−1and 𝑒𝑖 = 0• The matrix in those four equations is a 4 by 3 difference

matrix A, and 𝑒 = 𝐴𝑢 :

• The next equation 𝑦 = 𝑐𝑒 connects 𝑒 with 𝑦. • This is Hooke's Law 𝑦𝑖 = 𝑐𝑖𝑒𝑖 for each separate spring.


A Line of Springs and Masses(contd.)• Hooke’s Law 𝑦1 = 𝑐1𝑒1 and 𝑦2 = 𝑐2𝑒2and 𝑦3 = 𝑐3𝑒3 and

𝑦4 = 𝑐4𝑒4

• Combining 𝑒 = 𝐴𝑢 with 𝑦 = 𝐶𝑒, the spring forces (tension forces) are 𝑦 = 𝐶𝐴𝑢.

• The internal forces from the springs balance the external

forces on the masses.

• Each mass is pulled or pushed by the spring force 𝑦𝑗 above

it. From below the spring force is 𝑦𝑗+1 plus 𝑦𝑗 from

gravity, thus 𝑦𝑗 = 𝑦𝑗+1 + 𝑓𝑗 or 𝑓𝑗 = 𝑦𝑗 − 𝑦𝑗+1.Fall 2020Hamid Reza Rabiee

A Line of Springs and Masses(contd.)

• 𝑓1 = 𝑦1 − 𝑦2 and 𝑓2 = 𝑦2 − 𝑦3 and 𝑓3 = 𝑦3 − 𝑦4 or

• The equation for balance of forces is f = 𝐴𝑇𝑦• 𝑒 = 𝐴𝑢 and y = 𝐶𝑒 and f = 𝐴𝑇𝑦 combine into 𝐴𝑇𝐶𝐴𝑢

= 𝑓 or 𝐾𝑢 = 𝑓• 𝐾 = 𝐴𝑇𝐶𝐴 is the stiffness matrix(mechanics)• 𝐾 = 𝐴𝑇𝐶𝐴 is the conductance matrix(networks)


Source/ for more details

• Gilbert Strang. Introduction to Linear Algebra. 6.5.

• Gilbert Strang. Introduction to Linear Algebra. 10.2.

Hamid Reza Rabiee Fall 2020

Special Thanks to the following TAs for this presentation:

• Sabrineh Mokhtari

Hamid Reza Rabiee Fall 2020

Documents

Linear Algebrace.sharif.edu/courses/99-00/1/ce425-1/resources/root/...Introduction to Linear Algebra. 6.5. • Gilbert Strang. Introduction to Linear Algebra. 10.2. Hamid Reza Rabiee