Lecture 1. System of Linear Algebraic Equa-tions

1. Introduction to system of linear equations

2. Augmented matrix and row operations

3. System in row echelon form and reduced row echelonform4. System of linear equations with unique solution

5. System of linear equations with no solution

6. System of linear equations with infinitely many solu-tions

7. Gaussian elimination method for solving linear equa-tions

8. Gaussian-Jordan elimination method for solving linearequations

1

1. Introduction to system of linear algebraicequations

Many problems in science and engineering can finallybe described as system of linear algebraic equations.

Solving the problems in science and engineering can fi-nally become solving the system of linear algebraic equa-tions.

Linear equations

A linear equation is an equation in variables that occuronly to the first power.

(1) Linear equation in two variables x, y:

ax + by = c (a, b 6= 0) (1)

2

(2) Linear equation in three variables x, y, z:

ax + by + cz = d (a, b, c 6= 0) (2)

(3) Linear equation in n variables x1, x2, , xn:

a1x1 + a2x2 + . . . + anxn = c (ai 6= 0) (3)

If c = 0, then the equation

a1x1 + a2x2 + . . . + anxn = 0 (ai 6= 0) (4)is called homogenous linear equation in variables x1, x2, , xn.

3

Examples about the definition of linear equations

Linear equations:

x + 3y = 7

1

2x y + 3z = 1

x1 2x2 3x3 + x4 = 0x1 + x2 + . . . + xn = 1

Nonlinear equations:

x + 3y2 = 4

3x + 2y xy = 5sinx + y = 0x1 + 2x2 + x3 = 1

4

System of linear equations

A finite set of linear equations is called a system oflinear equations. The variables in the system of linearequations are called unknowns.

Example 1.1 Two equations in two unknowns:5x + y = 32x y = 4

This system has the solution x = 1, y = 2.

This solution can be written as (x, y) = (1,2).

(1,2) is called order 2-tuple.

5

Example 1.2 Two equations in three unknowns:

4x1 x2 + 3x3 = 13x1 + x2 + 9x3 = 4

One solution of this system is x1 = 1, x2 = 2, x3 = 1.

This solution can be written as (x1, x2, x3) = (1, 2,1).

(1, 2,1) is called order 3-tuple.

6

Example 1.3 m equations in n unknowns:

a11x1 + a12x2 + . . . + a1nxn = b1a21x1 + a22x2 + . . . + a2nxn = b2. . . . . .am1x1 + am2x2 + . . . + amnxn = bm

If this system has the solution

x1 = s1, x2 = s2, . . . , xn = sn.

This solution can be written as

(x1, x2, . . . , xn) = (s1, s2, . . . , sn).

(s1, s2, . . . , xn) is called order n-tuple.

7

Geometric properties and solutions of linear equations

Consider the following two equations in two variables.

a1x + b1y = c1a2x + b2y = c2

(1) Each equation expresses a line in x-y plane.

(2) The lines may be parallel and distinct, in which casethere is no intersection and the consequently no solution.

(3) The lines may intersect at only one point, in whichcase the system has exactly one solution.

(4) The lines may coincide, in which case there are in-finitely many points of intersection and consequently in-finitely many solutions.

If a linear system has at least one solution, the linearsystem is called consistent linear system.

If a linear system has no solutions, the linear system iscalled inconsistent linear system.

A system of linear equations has no solutions, one so-lution, or infinitely many solutions. There are no otherpossibilities.

8

Example 1.4 A linear system with one solution

x y = 12x + y = 6

-2Eq. (1)+Eq.(2) Eq. (2) givesx y = 13y = 4

from which we get

y =4

3

x = 1 + y = 1 +4

3=

7

3

So the solution of this linear system is (43,73)

9

Example 1.5 A linear system with no solutions

x + y = 43x + 3y = 6

-3Eq. (1)+Eq.(2) Eq. (2) givesx + y = 40 = 6

The second equation is contradictory, so the given systemhas no solution.

Geometrically, the two equations represents two parallellines without intersection.

10

Example 1.6 A linear system with infinitely many solu-tions

4x 2y = 116x 8y = 4

-4Eq. (1)+Eq.(2) Eq. (2) gives4x 2y = 10 = 0

The second equation does not impose any restrictions onx and y and hence it can be omitted. Thus the solutionsof the system are those values of x and y that satisfy thefirst equation

4x 2y = 1

geometrically, this means that the lines corresponding tothe two equations in the original system coincide.

The solution of this system can be expressed as

x =1

4+1

2y

This can be expressed by assigning an arbitrary param-eter t to y by

x =1

4+1

2t y = t

11

For any given value of t, there is a solution (14 +12t, t).

Hence there are infinitely many solutions for this system.

12

2. Augmented matrix and row operations

Augmented matrix

The linear system

a11x1 + a12x2 + . . . + a1nxn = b1a21x1 + a22x2 + . . . + a2nxn = b2. . . . . .am1x1 + am2x2 + . . . + amnxn = bm

can be written in augmented matrix form as

a11 a12 . . . a1n b1a21 a22 . . . a2n b2. . . . . .am1 am2 . . . amn bm

Rules on the row operation of linear system:

After the following operations, the solution of the linearsystem can not be changed.

(1) Multiply an equation through by a nonzero constant.

(2) Interchange two equations.

(3) Time one equation by a constant and add to another.

13

Since the rows (horizontal lines) of an augmented matrixcorresponding to the equations in the associated system,these three operations correspond to the following opera-tions on the rows of the augmented matrix:

(1) Multiply a row through by a nonzero constant.

(2) Interchange two rows.

(3) Time one row by a constant and add to another.

These are called elementary row operations on amatrix.

14

Example 2.1 Practice on using elementary row opera-tions

1 1 2 92 4 3 13 6 5 0

-2 row (1)+row(2) row(2) gives1 1 2 90 2 7 173 6 5 0

-3 row (1)+row(3) row(3) gives1 1 2 90 2 7 170 3 11 27

12 row (2) row(2) gives

1 1 2 90 1 72

172

0 3 11 27

-3 row (2)+row(3) row(3) gives

1 1 2 90 1 72 1720 0 12 32

15

-2 row (3) row(3) gives1 1 2 90 1 72 1720 0 1 3

-1 row (2)+row(1) row(1) gives

1 0 112352

0 1 72 1720 0 1 3

112 row (3)+row(1) row(1) gives1 0 0 10 1 72 1720 0 1 3

72 row (3)+row(2) row(2) gives

1 0 0 10 1 0 20 0 1 3

The augmented matrix can be written back to linear sys-tem as

x1 = 1

x2 = 2

x3 = 3

16

which is the solution of the linear system because the rowoperations can not influence the solution of the linear sys-tem.

The matrix in the above procedure1 1 2 90 1 72 1720 0 1 3

is called row echelon matrix in which the entries belowthe 1s are all zero.

The matrix 1 0 0 10 1 0 20 0 1 3

is called reduced row echelon matrix in which the entriesbelow and above the 1s are all zero.

17

3. System in row echelon form and reducedrow echelon form

To be a linear system in reduced row echelon form,a matrix must have the following properties:

(1) If a row does not consist entirely of zeros, then thefirst nonzero number in the row is a 1 which is called aleading 1.

(2) If there are any rows that consist entirely of zeros,then they are grouped together at the bottom of the ma-trix.

(3) In any two successive rows that do not consist en-tirely of zeros, the leading 1 in the lower row occurs fartherto the right than the leading 1 in the higher row.

(4) Each column that contains a leading 1 has zeros ev-erywhere else in that column.

A matrix that has the first three properties (1)(3) issaid to be in row echelon form.

A matrix that has the four properties (1)(4) is said tobe in reduced row echelon form.

18

4. System of linear equations with unique so-lution

The system of linear equations formulated in civil en-gineering or other areas of science and engineering hasunique solution in most cases.

If the number of the unknowns equals the number ofleading 1s and equals the number of nonzero rows in alinear system in reduced row echelon form, the system hasunique solution.

For example 1 0 0 10 1 0 20 0 1 3

which unique solution is (1, 2, 3)

19

5. System of linear equations with no solution

If there is a contradictory equation being 0 = nonzeroor the number of leading 1s is less than the number ofnonzero rows in a linear system in reduced row echelonform, the system has no solution.

For example, the systems corresponding to the follow-ing systems in reduced row echelon form have no solutions.

1 0 0 10 1 2 20 0 0 3

or 1 0 0 10 1 0 20 0 0 3

The number of unknowns equals 2 which is less than thenumber of nonzero rows 3 in the linear system in reducedrow echelon form.

20

6. System of linear equations with infinitelymany solutions

If the number of unknowns is greater than the numberof nonzero rows in a linear system in reduced row echelonform, the system has infinitely many solutions.

For example

1 0 3 10 1 4 20 0 0 0

The number of unknowns equals 3 which is greater thanthe number of nonzero rows 2 in the linear system in re-duced row echelon form.

There are infinitely many solutions for this system.

The system can be written as

x1 + 3x3 = 1x2 4x3 = 2

The solution of this system is (1 3x3, 2 + 4x3, x3).

21

In the linear system, the variables multiplying the lead-ing 1s are called leading variables. The remainingvariables are called free variables.

The x1 and x2 in the above example are leading vari-ables and the x3 is free variable.

22

7. Gaussian elimination method for solving lin-ear equations

Gaussian elimination method is one of the popularlyused method for solving the system of linear equations.

Computer program can be implemented for Gaussianelimination method to solve large linear equations.

There are two steps with Gaussian elimination methodin solving linear equations.

Step 1: Elimination

In this step, the augmented matrix is changed to rowechelon form with the row operations.

Step 2: Back substitution

In this step, the solution is obtained by step-by-stepsubstitution.

Gaussian elimination procedure for the system withunique solution:

Suppose there are n unknowns and n equations, i.e.,

23

a11 a12 . . . a1n b1a21 a22 . . . a2n b2... ... . . . ... ...an1 an2 . . . ann bn

Step 1: Elimination (calculate in sequence as follows:)

akj =akjakk

aij = akjaik + aij

bi = bkakk

aik + bi

bk =bkakk

akk = 1

aik = 0(k = 1, 2, . . . , n; i, j = k + 1, k + 2, . . . , n)

1 a12 . . . a1n b

1

0 1 . . . a2n b2

... ... . . . ... ...0 0 . . . 1 bn

which is in row echelon form.

24

Step 2: Back substitution

xn = bn

xi = bi

nk=i+1

aikxk i = n 1, n 2, . . . , 1

25

8. Gaussian-Jordan elimination method for solv-ing linear equations

Gaussian-Jordan elimination method is one of thepopularly used method for solving the system of linearequations.

Computer program can be implemented for Gaussian-Jordan elimination method to solve large linear equations.

There are two steps with Gaussian-Jordan eliminationmethod in solving linear equations.

Step 1: Forward Elimination

In this step, the augmented matrix is changed to rowechelon form with the row operations.

Step 2: Backward phase

In this step, the augmented matrix is changed to re-duced row echelon form with the row operations based onthe result from step 1.

Gaussian-Jordan elimination procedure For the sys-tem with unique solution:

26

Suppose there are n unknowns and n equations, i.e.,

a11 a12 . . . a1n b1a21 a22 . . . a2n b2... ... . . . ... ...an1 an2 . . . ann bn

Step 1: Elimination (calculate in sequence as follows:)

akj =akjakk

aij = akjaik + aij

bi = bkakk

aik + bi

bk =bkakk

akk = 1

aik = 0(k = 1, 2, . . . , n; i, j = k + 1, k + 2, . . . , n)

1 a12 . . . a1n b

1

0 1 . . . a2n b2

... ... . . . ... ...0 0 . . . 1 bn

which is in row echelon form.

27

Step 2: Backward phase

bi = bi bjaij

(j = n, n 1, . . . , 2; i = j 1, j 2, . . . , 1)

1 0 . . . 0 b10 1 . . . 0 b2... ... . . . ... ...0 0 . . . 1 bn

which is in reduced row echelon form.

Then, the solution of the linear system is (b1, b2, . . . , b

n).

28

Assignments 1

Exercise Set 1.1:

Problem 2. (20%), 4. (20%), 13. (20%)

True-False Exercises (40%)

Assignments 2

Based on Gaussian-Jordan Elimination procedure, writea computer program for solving the system of linear equa-tions with unique solution. Select five systems by yourselfand try to solve five systems with your program. It isrequired that

the first is a 1 1 system (20%),the second is a 2 2 system (20%),the third is a 3 3 system (20%),the fourth is a 4 4 system (20%),the fifth is a 5 5 system (20%).

List the systems in augmented matrix form and give thesolutions from your program only.

29