linear and nonlinear equations analysis using matlab

8/10/2019 linear and nonlinear equations analysis using matlab

1/29

Chapter One

Numerical analysis:

Numerical analysisis the study of algorithmsthat usenumerical approximation(as opposed to general symbolic manipulations) for

the problems of mathematical analysis(as distinguished from discrete

mathematics).

One of the earliest mathematical writings is a Babylonian tablet from

the Yale Babylonian Collection(YBC 72!)" which gi#es a hexadecimal

numerical approximationof " the length of the diagonalin a unit s$uare.

Being able to compute the sides of a triangle (and hence" being able to

compute s$uare roots) is extremely important" for instance"

in astronomy" carpentryand construction.

%umerical analysis continues this long tradition of practical

mathematical calculations. &uch li'e the Babylonian approximation of "

modern numerical analysis does not see' exact answers" because exact

answers are often impossible to obtain in practice. nstead" much of

numerical analysis is concerned with obtaining approximate solutions while

maintaining reasonable bounds on errors.%umerical analysis naturally finds applications in all fields of

engineering and the physical sciences" but in the 2st century also the life

sciences and e#en the arts ha#e adopted elements of scientific

computations. Ordinary differential e$uationsappear in celestial

mechanics(planets" stars and galaxies)* numerical linear algebrais important

for data analysis* stochastic differential e$uationsand &ar'o# chainsare

essential in simulating li#ing cells for medicine and biology.

Before the ad#ent of modern computers numerical methods often

depended on hand interpolationin large printed tables. +ince the mid 2,th

century" computers calculate the re$uired functions instead. -hese same

interpolation formulas ne#ertheless continue to be used as part of the

software algorithmsfor sol#ing differential e$uations.
http://en.wikipedia.org/wiki/Algorithmhttp://en.wikipedia.org/wiki/Approximationhttp://en.wikipedia.org/wiki/Symbolic_computationhttp://en.wikipedia.org/wiki/Mathematical_analysishttp://en.wikipedia.org/wiki/Discrete_mathematicshttp://en.wikipedia.org/wiki/Discrete_mathematicshttp://en.wikipedia.org/wiki/Yale_Babylonian_Collectionhttp://en.wikipedia.org/wiki/Numerical_approximationhttp://en.wikipedia.org/wiki/Diagonalhttp://en.wikipedia.org/wiki/Unit_squarehttp://en.wikipedia.org/wiki/Astronomyhttp://en.wikipedia.org/wiki/Carpentryhttp://en.wikipedia.org/wiki/Ordinary_differential_equationhttp://en.wikipedia.org/wiki/Celestial_mechanicshttp://en.wikipedia.org/wiki/Celestial_mechanicshttp://en.wikipedia.org/wiki/Numerical_linear_algebrahttp://en.wikipedia.org/wiki/Stochastic_differential_equationhttp://en.wikipedia.org/wiki/Markov_chainhttp://en.wikipedia.org/wiki/Interpolationhttp://en.wikipedia.org/wiki/Algorithmshttp://en.wikipedia.org/wiki/Differential_equationshttp://en.wikipedia.org/wiki/Algorithmhttp://en.wikipedia.org/wiki/Approximationhttp://en.wikipedia.org/wiki/Symbolic_computationhttp://en.wikipedia.org/wiki/Mathematical_analysishttp://en.wikipedia.org/wiki/Discrete_mathematicshttp://en.wikipedia.org/wiki/Discrete_mathematicshttp://en.wikipedia.org/wiki/Yale_Babylonian_Collectionhttp://en.wikipedia.org/wiki/Numerical_approximationhttp://en.wikipedia.org/wiki/Diagonalhttp://en.wikipedia.org/wiki/Unit_squarehttp://en.wikipedia.org/wiki/Astronomyhttp://en.wikipedia.org/wiki/Carpentryhttp://en.wikipedia.org/wiki/Ordinary_differential_equationhttp://en.wikipedia.org/wiki/Celestial_mechanicshttp://en.wikipedia.org/wiki/Celestial_mechanicshttp://en.wikipedia.org/wiki/Numerical_linear_algebrahttp://en.wikipedia.org/wiki/Stochastic_differential_equationhttp://en.wikipedia.org/wiki/Markov_chainhttp://en.wikipedia.org/wiki/Interpolationhttp://en.wikipedia.org/wiki/Algorithmshttp://en.wikipedia.org/wiki/Differential_equations


2/29

General Introduction:

-he o#erall goal of the field of numerical analysis is the design and analysis

of techni$ues to gi#e approximate but accurate solutions to hard problems"

the #ariety of which is suggested by the following

/d#anced numerical methods are essential in ma'ing numerical

weather predictionfeasible.

Computing the tra0ectory of a spacecraft re$uires the accurate

numerical solution of a system of ordinary differential e$uations.

Car companies can impro#e the crash safety of their #ehicles by using

computer simulations of car crashes. +uch simulations essentially

consist of sol#ingpartial differential e$uationsnumerically.

1edge funds (pri#ate in#estment funds) use tools from all fields ofnumerical analysis to attempt to calculate the #alue of stoc's and

deri#ati#es more precisely than other mar'et participants.

/irlines use sophisticated optimiation algorithms to decide tic'et

prices" airplane and crew assignments and fuel needs. 1istorically" such

algorithms were de#eloped within the o#erlapping field of operations

research.

nsurance companies use numerical programs for actuarialanalysis.

-he rest of this section outlines se#eral important themes of numerical

analysis.

History:

-he field of numerical analysis predates the in#ention of modern

computers by many centuries. 3inear interpolationwas already in use more

than 2,,, years ago. &any great mathematicians of the past were

preoccupied by numerical analysis" as is ob#ious from the names of

important algorithms li'e%ewton4s method" 3agrange interpolation

polynomial" 5aussian elimination" or 6uler4s method.

-o facilitate computations by hand" large boo's were produced with

formulas and tables of data such as interpolation points and function
http://en.wikipedia.org/wiki/Numerical_weather_predictionhttp://en.wikipedia.org/wiki/Numerical_weather_predictionhttp://en.wikipedia.org/wiki/Ordinary_differential_equationhttp://en.wikipedia.org/wiki/Partial_differential_equationhttp://en.wikipedia.org/wiki/Hedge_fundhttp://en.wikipedia.org/wiki/Operations_researchhttp://en.wikipedia.org/wiki/Operations_researchhttp://en.wikipedia.org/wiki/Actuaryhttp://en.wikipedia.org/wiki/Linear_interpolationhttp://en.wikipedia.org/wiki/Newton's_methodhttp://en.wikipedia.org/wiki/Lagrange_polynomialhttp://en.wikipedia.org/wiki/Lagrange_polynomialhttp://en.wikipedia.org/wiki/Gaussian_eliminationhttp://en.wikipedia.org/wiki/Euler's_methodhttp://en.wikipedia.org/wiki/Numerical_weather_predictionhttp://en.wikipedia.org/wiki/Numerical_weather_predictionhttp://en.wikipedia.org/wiki/Ordinary_differential_equationhttp://en.wikipedia.org/wiki/Partial_differential_equationhttp://en.wikipedia.org/wiki/Hedge_fundhttp://en.wikipedia.org/wiki/Operations_researchhttp://en.wikipedia.org/wiki/Operations_researchhttp://en.wikipedia.org/wiki/Actuaryhttp://en.wikipedia.org/wiki/Linear_interpolationhttp://en.wikipedia.org/wiki/Newton's_methodhttp://en.wikipedia.org/wiki/Lagrange_polynomialhttp://en.wikipedia.org/wiki/Lagrange_polynomialhttp://en.wikipedia.org/wiki/Gaussian_eliminationhttp://en.wikipedia.org/wiki/Euler's_method


3/29

coefficients. sing these tables" often calculated out to 8 decimal places or

more for some functions" one could loo' up #alues to plug into the formulas

gi#en and achie#e #ery good numerical estimates of some functions. -he

canonical wor' in the field is the%+-publication edited by /bramowit

and +tegun" a ,,,9plus page boo' of a #ery large number of commonly used

formulas and functions and their #alues at many points. -he function #alues

are no longer #ery useful when a computer is a#ailable" but the large listing

of formulas can still be #ery handy.

-he mechanical calculatorwas also de#eloped as a tool for hand

computation. -hese calculators e#ol#ed into electronic computers in the

!:,s" and it was then found that these computers were also useful for

administrati#e purposes. But the in#ention of the computer also influencedthe field of numerical analysis" since now longer and more complicated

calculations could be done.

Direct and iterative methods:

;irect methods compute the solution to a problem in a finite number of

steps. -hese methods would gi#e the precise answer if they were performed

in infinite precision arithmetic. 6xamples include 5aussian elimination"

the acobi iteration. n

computational matrix algebra" iterati#e methods are generally needed for

large problems.
http://en.wikipedia.org/wiki/NISThttp://en.wikipedia.org/wiki/Abramowitz_and_Stegunhttp://en.wikipedia.org/wiki/Abramowitz_and_Stegunhttp://en.wikipedia.org/wiki/Mechanical_calculatorhttp://en.wikipedia.org/wiki/Computer_numbering_formatshttp://en.wikipedia.org/wiki/Gaussian_eliminationhttp://en.wikipedia.org/wiki/QR_algorithmhttp://en.wikipedia.org/wiki/System_of_linear_equationshttp://en.wikipedia.org/wiki/Simplex_methodhttp://en.wikipedia.org/wiki/Linear_programminghttp://en.wikipedia.org/wiki/Floating_pointhttp://en.wikipedia.org/wiki/Numerically_stablehttp://en.wikipedia.org/wiki/Iterative_methodhttp://en.wikipedia.org/wiki/Limit_of_a_sequencehttp://en.wikipedia.org/wiki/Residual_(numerical_analysis)http://en.wikipedia.org/wiki/Newton's_methodhttp://en.wikipedia.org/wiki/Bisection_methodhttp://en.wikipedia.org/wiki/Jacobi_iterationhttp://en.wikipedia.org/wiki/NISThttp://en.wikipedia.org/wiki/Abramowitz_and_Stegunhttp://en.wikipedia.org/wiki/Abramowitz_and_Stegunhttp://en.wikipedia.org/wiki/Mechanical_calculatorhttp://en.wikipedia.org/wiki/Computer_numbering_formatshttp://en.wikipedia.org/wiki/Gaussian_eliminationhttp://en.wikipedia.org/wiki/QR_algorithmhttp://en.wikipedia.org/wiki/System_of_linear_equationshttp://en.wikipedia.org/wiki/Simplex_methodhttp://en.wikipedia.org/wiki/Linear_programminghttp://en.wikipedia.org/wiki/Floating_pointhttp://en.wikipedia.org/wiki/Numerically_stablehttp://en.wikipedia.org/wiki/Iterative_methodhttp://en.wikipedia.org/wiki/Limit_of_a_sequencehttp://en.wikipedia.org/wiki/Residual_(numerical_analysis)http://en.wikipedia.org/wiki/Newton's_methodhttp://en.wikipedia.org/wiki/Bisection_methodhttp://en.wikipedia.org/wiki/Jacobi_iteration


4/29

terati#e methods are more common than direct methods in numerical

analysis. +ome methods are direct in principle but are usually used as though

they were not" e.g. 5&=6+and the con0ugate gradient method. ?or these

methods the number of steps needed to obtain the exact solution is so large

that an approximation is accepted in the same manner as for an iterati#e

method.

Direct vs iterative methods:

Consider the problem of sol#ing

@x@A : 2

for the un'nown $uantityx.

;irect method@x@A : 2.

Subtract 4 @x@ 2:.

Divide by 3 x@ .

Take cube roots x 2.

?or the iterati#e method" apply the bisection methodtof(x) @x@ 2:.

-he initial #alues are a ," b @"f(a) 2:"f(b) D7.

terati#e method

a b mid f(mid)

, @ .D @.7D

.D @ 2.2D ,.7...

.D 2.2D .7D :.22...

.7D 2.2D 2.,82D 2.@2...

Ee conclude from this table that the solution is between .7D and2.,82D. -he algorithm might return any number in that range with an error

less than ,.2.

Discretization :
http://en.wikipedia.org/wiki/GMREShttp://en.wikipedia.org/wiki/Conjugate_gradient_methodhttp://en.wikipedia.org/wiki/Bisection_methodhttp://en.wikipedia.org/wiki/GMREShttp://en.wikipedia.org/wiki/Conjugate_gradient_methodhttp://en.wikipedia.org/wiki/Bisection_method


5/29

?urthermore" continuous problems must sometimes be replaced by a

discrete problem whose solution is 'nown to approximate that of the

continuous problem* this process is called discretization. ?or example" the

solution of a differential e$uationis a function. -his function must be

represented by a finite amount of data" for instance by its #alue at a finite

number of points at its domain" e#en though this domain is a continuum.

Area of Numerical Analysis:

-he field of numerical analysis includes many sub9disciplines. +ome of

the ma0or ones are

! Computin" values of functions:

One of the simplest problems is the e#aluation of a function at a gi#enpoint. -he most straightforward approach" of 0ust plugging in the

number in the formula is sometimes not #ery efficient. ?or

polynomials" a better approach is using theHorner scheme" since it

reduces the necessary number of multiplications and additions.

5enerally" it is important to estimate and controlround!off

errorsarising from the use offloatin" pointarithmetic.

#! Interpolation$ e%trapolation$ and re"ression:Interpolationsol#es the following problem gi#en the #alue of some

un'nown function at a number of points" what #alue does that function

ha#e at some other point between the gi#en points.

&%trapolationis #ery similar to interpolation" except that now we want

to find the #alue of the un'nown function at a point which is outside

the gi#en points.

'e"ressionis also similar" but it ta'es into account that the data is

imprecise. 5i#en some points" and a measurement of the #alue of some

function at these points (with an error)" we want to determine the

un'nown function. -heleast suares9method is one popular way to

achie#e this solvin" euations and systems of euations
http://en.wikipedia.org/wiki/Discretizationhttp://en.wikipedia.org/wiki/Differential_equationhttp://en.wikipedia.org/wiki/Function_(mathematics)http://en.wikipedia.org/wiki/Continuum_(set_theory)http://en.wikipedia.org/wiki/Horner_schemehttp://en.wikipedia.org/wiki/Round-off_errorhttp://en.wikipedia.org/wiki/Round-off_errorhttp://en.wikipedia.org/wiki/Floating_pointhttp://en.wikipedia.org/wiki/Interpolationhttp://en.wikipedia.org/wiki/Extrapolationhttp://en.wikipedia.org/wiki/Regression_analysishttp://en.wikipedia.org/wiki/Least_squareshttp://en.wikipedia.org/wiki/Discretizationhttp://en.wikipedia.org/wiki/Differential_equationhttp://en.wikipedia.org/wiki/Function_(mathematics)http://en.wikipedia.org/wiki/Continuum_(set_theory)http://en.wikipedia.org/wiki/Horner_schemehttp://en.wikipedia.org/wiki/Round-off_errorhttp://en.wikipedia.org/wiki/Round-off_errorhttp://en.wikipedia.org/wiki/Floating_pointhttp://en.wikipedia.org/wiki/Interpolationhttp://en.wikipedia.org/wiki/Extrapolationhttp://en.wikipedia.org/wiki/Regression_analysishttp://en.wikipedia.org/wiki/Least_squares


6/29

/nother fundamental problem is computing the solution of some gi#en

e$uation. -wo cases are commonly distinguished" depending on whether the

e$uation is linear or not. ?or instance" the e$uation is linear

while is not.

&uch effort has been put in the de#elopment of methods for

sol#ing systems of linear e$uations. +tandard direct methods" i.e." methods

that use some matrix decompositionare 5aussian elimination" 3

decomposition" Choles'y decompositionfor symmetric(or hermitian)

andpositi#e9definite matrix" and acobi method" 5aussF+eidel

method" successi#e o#er9relaxationand con0ugate gradient methodareusually preferred for large systems. 5eneral iterati#e methods can be

de#eloped using a matrix splitting.

'oot!findin" al"orithmsare used to sol#e nonlinear e$uations (they

are so named since a root of a function is an argument for which the function

yields ero). f the function is differentiableand the deri#ati#e is 'nown"

then%ewton4s methodis a popular choice. 3ineariationis another techni$ue

for sol#ing nonlinear e$uations.
http://en.wikipedia.org/wiki/Systems_of_linear_equationshttp://en.wikipedia.org/wiki/Matrix_decompositionhttp://en.wikipedia.org/wiki/Gaussian_eliminationhttp://en.wikipedia.org/wiki/LU_decompositionhttp://en.wikipedia.org/wiki/LU_decompositionhttp://en.wikipedia.org/wiki/Cholesky_decompositionhttp://en.wikipedia.org/wiki/Symmetric_matrixhttp://en.wikipedia.org/wiki/Hermitian_matrixhttp://en.wikipedia.org/wiki/Positive-definite_matrixhttp://en.wikipedia.org/wiki/QR_decompositionhttp://en.wikipedia.org/wiki/Iterative_methodhttp://en.wikipedia.org/wiki/Jacobi_methodhttp://en.wikipedia.org/wiki/Gauss%E2%80%93Seidel_methodhttp://en.wikipedia.org/wiki/Gauss%E2%80%93Seidel_methodhttp://en.wikipedia.org/wiki/Successive_over-relaxationhttp://en.wikipedia.org/wiki/Conjugate_gradient_methodhttp://en.wikipedia.org/wiki/Matrix_splittinghttp://en.wikipedia.org/wiki/Root-finding_algorithmhttp://en.wikipedia.org/wiki/Derivativehttp://en.wikipedia.org/wiki/Newton's_methodhttp://en.wikipedia.org/wiki/Linearizationhttp://en.wikipedia.org/wiki/Systems_of_linear_equationshttp://en.wikipedia.org/wiki/Matrix_decompositionhttp://en.wikipedia.org/wiki/Gaussian_eliminationhttp://en.wikipedia.org/wiki/LU_decompositionhttp://en.wikipedia.org/wiki/LU_decompositionhttp://en.wikipedia.org/wiki/Cholesky_decompositionhttp://en.wikipedia.org/wiki/Symmetric_matrixhttp://en.wikipedia.org/wiki/Hermitian_matrixhttp://en.wikipedia.org/wiki/Positive-definite_matrixhttp://en.wikipedia.org/wiki/QR_decompositionhttp://en.wikipedia.org/wiki/Iterative_methodhttp://en.wikipedia.org/wiki/Jacobi_methodhttp://en.wikipedia.org/wiki/Gauss%E2%80%93Seidel_methodhttp://en.wikipedia.org/wiki/Gauss%E2%80%93Seidel_methodhttp://en.wikipedia.org/wiki/Successive_over-relaxationhttp://en.wikipedia.org/wiki/Conjugate_gradient_methodhttp://en.wikipedia.org/wiki/Matrix_splittinghttp://en.wikipedia.org/wiki/Root-finding_algorithmhttp://en.wikipedia.org/wiki/Derivativehttp://en.wikipedia.org/wiki/Newton's_methodhttp://en.wikipedia.org/wiki/Linearization


7/29

Chapter t*o

+inear and nonlinear euations

n mathematics" a linearfunction(or map) is one which satisfiesboth of the following properties

Additivity(+uperposition)"

Homo"eneity"

(/dditi#ity implies homogeneity for any rational" and" for continuous

functions" for any real. ?or a complex" homogeneity does not follow from

additi#ity* for example" an antilinear mapis additi#e but not homogeneous.)

-he conditions of additi#ity and homogeneity are often combined in

the superposition principle

/n e$uation written as is called linearif is a linear map

(as defined abo#e) and nonlinearotherwise.

-he e$uation is called homogeneousif .

-he definition is #ery general in that can be any sensible

mathematical ob0ect (number" #ector" function" etc.)" and the function

can literally be any mapping" including integration or differentiation with

associated constraints (such asboundary #alues). f

contains differentiationwith respect to " the result will be a differential

e$uation.

,he Difference -et*een +inear . Nonlinear &uations:

n the world of mathematics" there are se#eral types of e$uations that

scientists" economists" statisticians and other professionals use to predict"

analye and explain the uni#erse around them. -hese e$uations relate
http://en.wikipedia.org/wiki/Mathematicshttp://en.wikipedia.org/wiki/Linear_maphttp://en.wikipedia.org/wiki/Function_(mathematics)http://en.wikipedia.org/wiki/Superposition_principlehttp://en.wikipedia.org/wiki/Rational_numberhttp://en.wikipedia.org/wiki/Continuous_functionhttp://en.wikipedia.org/wiki/Continuous_functionhttp://en.wikipedia.org/wiki/Real_numberhttp://en.wikipedia.org/wiki/Complex_numberhttp://en.wikipedia.org/wiki/Antilinear_maphttp://en.wikipedia.org/wiki/Superposition_principlehttp://en.wikipedia.org/wiki/Map_(mathematics)http://en.wikipedia.org/wiki/Boundary_valueshttp://en.wikipedia.org/wiki/Derivativehttp://en.wikipedia.org/wiki/Differential_equationhttp://en.wikipedia.org/wiki/Differential_equationhttp://en.wikipedia.org/wiki/Mathematicshttp://en.wikipedia.org/wiki/Linear_maphttp://en.wikipedia.org/wiki/Function_(mathematics)http://en.wikipedia.org/wiki/Superposition_principlehttp://en.wikipedia.org/wiki/Rational_numberhttp://en.wikipedia.org/wiki/Continuous_functionhttp://en.wikipedia.org/wiki/Continuous_functionhttp://en.wikipedia.org/wiki/Real_numberhttp://en.wikipedia.org/wiki/Complex_numberhttp://en.wikipedia.org/wiki/Antilinear_maphttp://en.wikipedia.org/wiki/Superposition_principlehttp://en.wikipedia.org/wiki/Map_(mathematics)http://en.wikipedia.org/wiki/Boundary_valueshttp://en.wikipedia.org/wiki/Derivativehttp://en.wikipedia.org/wiki/Differential_equationhttp://en.wikipedia.org/wiki/Differential_equation


8/29

#ariables in such a way that one can influence" or forecast" the output of

another. n basic mathematics" linear e$uations are the most popular choice of

analysis" but nonlinear e$uations dominate the realm of higher math and

science.

,ypes of &uations:

6ach e$uation gets its form based on the highest degree" or exponent"

of the #ariable. ?or instance" in the case where y xG 99 8x A 2" the degree of

@ gi#es this e$uation the name Hcubic.H /ny e$uation that has a degree no

higher than recei#es the name Hlinear.H Otherwise" we call an e$uation

Hnonlinear"H whether it is $uadratic" a sine9cur#e or in any other form.

Input!Output 'elationships:

n general" HxH is considered to be the input of an e$uation and HyH is

considered to be the output. n the case of a linear e$uation" any increase in

HxH will either cause an increase in HyH or a decrease in HyH corresponding to

the #alue of the slope. n contrast" in a nonlinear e$uation" HxH may not

always cause HyH to increase. ?or example" if y (D 99 x)I" HyH decreases in

#alue as HxH approaches D" but increases otherwise.

Graph Differences:

/ graph displays the set of solutions for a gi#en e$uation. n the case of

linear e$uations" the graph will always be a line. n contrast" a nonlinear

e$uation may loo' li'e a parabola if it is of degree 2" a cur#y x9shape if it is

of degree @" or any cur#y #ariation thereof. Ehile linear e$uations are always

straight" nonlinear e$uations often feature cur#es.

&%ceptions:

6xcept for the case of #ertical lines (x a constant) and horiontal lines

(y a constant)" linear e$uations will exist for all #alues of HxH and Hy.H

%onlinear e$uations" on the other hand" may not ha#e solutions for certain

#alues of HxH or Hy.H ?or instance" if y s$rt(x)" then HxH exists only from ,


9/29

and beyond" as does Hy"H because the s$uare root of a negati#e number does

not exist in the real number system and there are no s$uare roots that result in

a negati#e output.

-enefits:

3inear relationships can be best explained by linear e$uations" where

the increase in one #ariable directly causes the increase or decrease of

another. ?or example" the number of coo'ies you eat in a day could ha#e a

direct impact on your weight as illustrated by a linear e$uation. 1owe#er" if

you were analying the di#ision of cells under mitosis" a nonlinear"

exponential e$uation would fit the data better.

Ee can compare the answers to the two main $uestions in differential

e$uations for linear and nonlinear first order differential e$uations.

=ecall that for a first order linear differential e$uation y4 A p(x)y g(x)

we had the solution

A C

=ecall that if a function is continuous then the integral always exists. f we

are gi#en an initial #alue y(x,) y, " then we can uni$uely sol#e for C to

get a solution. -his immediately shows that there exists a solution to all first

order linear differential e$uations. -his also establishes uni$ueness since the


10/29

deri#ation shows that all solutions must be of the form abo#e. %otice that if

the constant of integration foris chosen to be different from ," then the

constant cancels itself from the negati#e exponent outside the integral and the

positi#e exponent inside. -his pro#es that the answers to both of the 'ey

$uestions are affirmati#e for first order linear differential e$uations.

/or e%ample: to determine where the differential e$uation

(cos x)y4 A (sin x)y x2 y(,) :

has a uni$ue solution. Ee can say

;i#iding by cos x to get it into standard form gi#es

y4 A (tan x)y x

2

sec x+ince tan x is continuous for

9JK2 L x L JK2

and this inter#al contains ," the differential e$uation is guaranteed to ha#e a

uni$ue solution on this inter#al

%ow consider the differential e$uation

dyKdx xKy y(D) 9@

%otice that the theorem does not apply" since the differential e$uation isnonlinear. Ee can separate and sol#e.

ydy xdx

y2 x2A C

%ow plugging in the initial #alue" we get

! 2D A C

C 98

y2 x29 8

-a'ing the s$uare root of both sides" we would get a plus or minus solution"

howe#er the initial condition states that the #alue must be negati#e so the

solution is

y 9(x29 8)K2

-his e$uation is only #alid for MxM N :


11/29

%otice that the original e$uation is not continuous at y ," but the inter#al

where the solution is #alid could not ha#e been guessed without sol#ing the

differential e$uation.

Ho* to 0olve a 0ystem of +inear &uations:

-here are se#eral ways to sol#e a system of linear e$uations. -his article

focuses on : methods

. 5raphing

2. +ubstitution

@. 6limination /ddition

:. 6limination +ubtraction

0olve a 0ystem of &uations 1y Graphical 2ethod:Ee can found the solution of the following system of e$uations

yxA @

y 9x9 @

Note: Since the equations are inso!e"interce!t form# solving by graphingis

the best method$

Graph 1oth euations

# 3here do the lines meet4(9@" ,)

5 6erify that your ans*er is correct 7lu"x8 !5 andy8 9 into theeuations

yxA @

(,) (9@) A @

, ,

Correct

y 9x9 @

, 9(9@) 9 @

, @ 9 @

, ,

Correct
http://math.about.com/od/algebra1help/tp/Slope_Intercept.htmhttp://math.about.com/od/algebra1help/tp/Slope_Intercept.htm


12/29

# 0olve a 0ystem of &uations 1y 0u1stitution:

%ow we can found the intersection of the following e$uations. (n other

words" sol#e forxandy.)

@xAy 8

x 9@y

Note: %se the Substitution method because one of the variabes# x# is

isoated$

0incexis isolated in the top euation$ replacexin the top euation

*ith ! 5y

@ ( ; 5y) Ay 8

# 0implify

D: F !yAy 8D: F y 8

5 0olve

D: F yF D: 8 F D:

9y 9:

9yK9 9:K9

y 8


13/29

Note: This method is usefu &hen ' variabes are on one side of the equation#

and the constant is on the other side$

0tac? the euations to add

# 2ultiply the top euation 1y !5

9@(x A y ,)

5 3hy multiply 1y !54 Add to see

9@x A 9@y 9D:,

A @x A 2y ::

, A 9y 928

Notice that x is eiminated$

7lu" iny8 #= to findx

xAy ,

xA 28 ,

x D:

= 6erify that (>


14/29

#x@y8

5x;y8 =

%ote that" if add down" they4s will cancel out. +o 4ll draw an He$ualsH

bar under the system" and add down

2xAy !

@xFy 8

Dx 2D

%ow can di#ide through to sol#e forx D" and then bac'9sol#e" using

either of the original e$uations" to find the #alue ofy. -he first e$uation

has smaller numbers" so 4ll bac'9sol#e in that one

2(D) Ay !

, Ay ! y F

,hen the solution is (x$y) 8 (>$ ;)

t doesn4t matter which e$uation you use for the bac'sol#ing* you4ll get

the same answer either way. f 4d used the second e$uation" 4d ha#e gotten

@(D) Fy 8

D Fy 8

Fy y F

...which is the same result as before.

0olve the follo*in" system usin" addition

x; #y8 ;

x@ 5y8 =

%ote that thex9terms would cancel out if only they4d had opposite

signs. can create this cancellation by multiplying either one of the

e$uations by F" and then adding down as usual. t doesn4t matter which

e$uation choose" as long as am careful to multiply the F through the

entire e$uation. (-hat means bothsides of the He$ualsH sign)

4ll multiply the second e$uation.


15/29

-he HF(2H notation o#er the arrow indicates that multiplied

row 2 by F. %ow can sol#e the e$uation HFDy F2DH to gety D.Bac'9sol#ing in the first e$uation" get

xF 2(D) F!

xF , F!

x

,hen the solution is (x$y) 8 ($ >)

/ #ery common temptation is to write the solution in the form H(first

number found" second number found)H. +ometimes" though" as in this

case" you find they9#alue first and then thex9#alue second" and of course inpoints thex9#alue comes first. +o 0ust be careful to write the coordinates for

your solutions correctly.


16/29

yA 2 @

y 9!


17/29


18/29

Often the coefficients and un'nowns are realor complex numbers"

but integersand rational numbersare also seen" as are polynomials and

elements of an abstract algebraic structure.

6ector euation:

One extremely helpful #iew is that each un'nown is a weight for

a column #ectorin a linear combination.

-his allows all the language and theory of vector s!aces(or more

generally" modues) to be brought to bear. ?or example" the collection of all

possible linear combinations of the #ectors on the left9hand side is called

theirs!an" and the e$uations ha#e a solution 0ust when the right9hand #ector

is within that span. f e#ery #ector within that span has exactly one

expression as a linear combination of the gi#en left9hand #ectors" then any

solution is uni$ue. n any e#ent" the span has a basisof linearly

independent#ectors that do guarantee exactly one expression* and the

number of #ectors in that basis (its dimension) cannot be larger than mor n"

but it can be smaller. -his is important because if we ha#e mindependent

#ectors a solution is guaranteed regardless of the right9hand side" and

otherwise not guaranteed.
http://en.wikipedia.org/wiki/Real_numberhttp://en.wikipedia.org/wiki/Complex_numberhttp://en.wikipedia.org/wiki/Integerhttp://en.wikipedia.org/wiki/Rational_numberhttp://en.wikipedia.org/wiki/Algebraic_structurehttp://en.wikipedia.org/wiki/Column_vectorhttp://en.wikipedia.org/wiki/Linear_combinationhttp://en.wikipedia.org/wiki/Vector_spacehttp://en.wikipedia.org/wiki/Module_(mathematics)http://en.wikipedia.org/wiki/Span_(linear_algebra)http://en.wikipedia.org/wiki/Basis_(linear_algebra)http://en.wikipedia.org/wiki/Linearly_independenthttp://en.wikipedia.org/wiki/Linearly_independenthttp://en.wikipedia.org/wiki/Dimension_(linear_algebra)http://en.wikipedia.org/wiki/Real_numberhttp://en.wikipedia.org/wiki/Complex_numberhttp://en.wikipedia.org/wiki/Integerhttp://en.wikipedia.org/wiki/Rational_numberhttp://en.wikipedia.org/wiki/Algebraic_structurehttp://en.wikipedia.org/wiki/Column_vectorhttp://en.wikipedia.org/wiki/Linear_combinationhttp://en.wikipedia.org/wiki/Vector_spacehttp://en.wikipedia.org/wiki/Module_(mathematics)http://en.wikipedia.org/wiki/Span_(linear_algebra)http://en.wikipedia.org/wiki/Basis_(linear_algebra)http://en.wikipedia.org/wiki/Linearly_independenthttp://en.wikipedia.org/wiki/Linearly_independenthttp://en.wikipedia.org/wiki/Dimension_(linear_algebra)


19/29


20/29

solution set of their e$uations is empty* the solution set of the e$uations of

three planes intersecting at a point is single point* if three planes pass through

two points" their e$uations ha#e at least two common solutions* in fact the

solution set is infinite and consists in all the line passing through these points

?or n#ariables" each linear e$uation determines a hyperplanein n9

dimensional space. -he solution set is the intersection of these hyperplanes"

which may be a flatof any dimension.

General 1ehavior:

n general" the beha#ior of a linear system is determined by the

relationship between the number of e$uations and the number of un'nowns sually" a system with fewer e$uations than un'nowns has infinitely

many solutions" but it may ha#e no solution. +uch a system is 'nown as

an undetermined.

sually" a system with the same number of e$uations and un'nowns

has a single uni$ue solution.

sually" a system with more e$uations than un'nowns has no solution.

+uch a system is also 'nown as an o#er determined system.

n the first case" the dimensionof the solution set is usually e$ual

to n m" where nis the number of #ariables and mis the number of

e$uations.

-he following pictures illustrate this trichotomy in the case of two #ariables

One e$uation -wo e$uations -hree e$uations
http://en.wikipedia.org/wiki/Hyperplanehttp://en.wikipedia.org/wiki/N-dimensional_spacehttp://en.wikipedia.org/wiki/N-dimensional_spacehttp://en.wikipedia.org/wiki/N-dimensional_spacehttp://en.wikipedia.org/wiki/Flat_(geometry)http://en.wikipedia.org/wiki/Overdetermined_systemhttp://en.wikipedia.org/wiki/Dimensionhttp://en.wikipedia.org/wiki/File:Three_Lines.svghttp://en.wikipedia.org/wiki/File:Two_Lines.svghttp://en.wikipedia.org/wiki/File:One_Line.svghttp://en.wikipedia.org/wiki/Hyperplanehttp://en.wikipedia.org/wiki/N-dimensional_spacehttp://en.wikipedia.org/wiki/N-dimensional_spacehttp://en.wikipedia.org/wiki/Flat_(geometry)http://en.wikipedia.org/wiki/Overdetermined_systemhttp://en.wikipedia.org/wiki/Dimension


21/29


22/29

are not independent V they are the same e$uation when scaled by a factor of

two" and they would produce identical graphs. -his is an example of

e$ui#alence in a system of linear e$uations.

?or a more complicated example" the e$uations

are not independent" because the third e$uation is the sum of the other two.

ndeed" any one of these e$uations can be deri#ed from the other two" and

any one of the e$uations can be remo#ed without affecting the solution set.

-he graphs of these e$uations are three lines that intersect at a single point.

Consistency:

-he e$uations @xA 2y 8 and@xA 2y 2 are inconsistent.

/ linear system is inconsistentif it has no solution" and otherwise it is said to

be consistent. Ehen the system is inconsistent" it is possible to deri#e

a contradictionfrom the e$uations" that may always be rewritten such as the

statement , .?or example" the e$uations

are inconsistent. n fact" by subtracting the first e$uation from the second one

and multiplying both sides of the result by K8" we get , . -he graphs of

these e$uations on thexy9plane are a pair ofparallellines.
http://en.wikipedia.org/wiki/Contradictionhttp://en.wikipedia.org/wiki/Parallel_(geometry)http://en.wikipedia.org/wiki/File:Parallel_Lines.svghttp://en.wikipedia.org/wiki/Contradictionhttp://en.wikipedia.org/wiki/Parallel_(geometry)


23/29


24/29

the e$uations in the first system" and #ice9#ersa. -wo systems are e$ui#alent

if either both are inconsistent or each e$uation of any of them is a linear

combination of the e$uations of the other one. t follows that two linear

systems are e$ui#alent if and only if they ha#e the same solution set.

-here are se#eral algorithmsfor sol#inga system of linear e$uations.

Descri1in" the solution:

Ehen the solution set is finite" it is reduced to a single element. n this

case" the uni$ue solution is described by a se$uence of e$uations whose left

hand sides are the names of the un'nowns and right hand sides are the

corresponding #alues" for example . Ehen an order

on the un'nowns has been fixed" for example the alphabetical orderthesolution may be described as a #ectorof #alues" li'e for the

pre#ious example.

t can be difficult to describe a set with infinite solutions. -ypically"

some of the #ariables are designated as free(or independent" or

as parameters)" meaning that they are allowed to ta'e any #alue" while the

remaining #ariables are dependenton the #alues of the free #ariables.

?or example" consider the following system

-he solution set to this system can be described by the following e$uations

1erezis the free #ariable" whilexandyare dependent onz. /ny point

in the solution set can be obtained by first choosing a #alue forz" and then

computing the corresponding #alues forxandy.

6ach free #ariable gi#es the solution space one degree of freedom" the

number of which is e$ual to the dimensionof the solution set. ?or example"

the solution set for the abo#e e$uation is a line" since a point in the solution

set can be chosen by specifying the #alue of the parameter z. /n infinite

solution of higher order may describe a plane" or higher9dimensional set.
http://en.wikipedia.org/wiki/Algorithmhttp://en.wikipedia.org/wiki/Equation_solvinghttp://en.wikipedia.org/wiki/Alphabetical_orderhttp://en.wikipedia.org/wiki/Vector_spacehttp://en.wikipedia.org/wiki/Degrees_of_freedom_(statistics)http://en.wikipedia.org/wiki/Dimensionhttp://en.wikipedia.org/wiki/Algorithmhttp://en.wikipedia.org/wiki/Equation_solvinghttp://en.wikipedia.org/wiki/Alphabetical_orderhttp://en.wikipedia.org/wiki/Vector_spacehttp://en.wikipedia.org/wiki/Degrees_of_freedom_(statistics)http://en.wikipedia.org/wiki/Dimension


25/29

;ifferent choices for the free #ariables may lead to different

descriptions of the same solution set. ?or example" the solution to the abo#e

e$uations can alternati#ely be described as follows

1erexis the free #ariable" andyandzare dependent.

&limination of varia1les:

-he simplest method for sol#ing a system of linear e$uations is to

repeatedly eliminate #ariables. -his method can be described as follows

. n the first e$uation" sol#e for one of the #ariables in terms of the

others.

2. +ubstitute this expression into the remaining e$uations. -his yields a

system of e$uations with one fewer e$uation and one fewer un'nown.

@. Continue until you ha#e reduced the system to a single linear e$uation.

:. +ol#e this e$uation" and then bac'9substitute until the entire solution is

found.

?or example" consider the following system

+ol#ing the first e$uation forxgi#esx D A 2z @y" and plugging this into

the second and third e$uation yields

+ol#ing the first of these e$uations foryyieldsy 2 A @z" and plugging this

into the second e$uation yieldsz 2. Ee now ha#e

+ubstitutingz 2 into the second e$uation gi#esy " and substitutingz

2 andy into the first e$uation yieldsx D. -herefore" the solution set is

the single point(x"y"z) (D" " 2).


26/29


27/29

*ain artice: ,ramer-s rue

CramerBs ruleis an explicit formula for the solution of a system of

linear e$uations" with each #ariable gi#en by a $uotient of two determinants.

?or example" the solution to the system

is gi#en by

?or each #ariable" the denominator is the determinant of the matrix of

coefficients" while the numerator is the determinant of a matrix in which one

column has been replaced by the #ector of constant terms.

-hough Cramer4s rule is important theoretically" it has little practical

#alue for large matrices" since the computation of large determinants is

somewhat cumbersome. (ndeed" large determinants are most easilycomputed using row reduction.) ?urther" Cramer4s rule has #ery poor

numerical properties" ma'ing it unsuitable for sol#ing e#en small systems

reliably" unless the operations are performed in rational arithmetic with

unbounded precision.

2atri% solution:

f the e$uation system is expressed in the matrix form " the

entire solution set can also be expressed in matrix form. f the matrix )iss$uare (has mrows and nmcolumns) and has full ran' (all mrows are

independent)" then the system has a uni$ue solution gi#en by

where is the in#erseof). &ore generally" regardless of

whether mnor not and regardless of the ran' of)" all solutions (if any exist)
http://en.wikipedia.org/wiki/Cramer's_rulehttp://en.wikipedia.org/wiki/Determinanthttp://en.wikipedia.org/wiki/Matrix_inversehttp://en.wikipedia.org/wiki/Cramer's_rulehttp://en.wikipedia.org/wiki/Determinanthttp://en.wikipedia.org/wiki/Matrix_inverse


28/29

are gi#en using the &oore9Wenrose pseudo in#erseof)" denoted " as

follows

where is a #ector of free parameters that ranges o#er all possible nT

#ectors. / necessary and sufficient condition for any solution(s) to exist is

that the potential solution obtained using satisfy V that is"

that f this condition does not hold" the e$uation system is

inconsistent and has no solution. f the condition holds" the system is

consistent and at least one solution exists. ?or example" in the abo#e9

mentioned case in which)is s$uare and of full ran'" simply e$uals

and the general solution e$uation simplifies

to as pre#iouslystated" where has completely dropped out of the solution" lea#ing only a

single solution. n other cases" though" remains and hence an infinitude of

potential #alues of the free parameter #ector gi#e an infinitude of solutions

of the e$uation.

Other methods:

Ehile systems of three or four e$uations can be readily sol#ed by hand"

computers are often used for larger systems. -he standard algorithm forsol#ing a system of linear e$uations is based on 5aussian elimination with

some modifications. ?irstly" it is essential to a#oid di#ision by small numbers"

which may lead to inaccurate results. -his can be done by reordering the

e$uations if necessary" a process 'nown as!ivoting. +econdly" the algorithm

does not exactly do 5aussian elimination" but it computes the 3

decompositionof the matrix). -his is mostly an organiational tool" but it is

much $uic'er if one has to sol#e se#eral systems with the same matrix)but

different #ectors 1.

f the matrix) has some special structure" this can be exploited to

obtain faster or more accurate algorithms. ?or instance" systems with

a symmetricpositi#e definitematrix can be sol#ed twice as fast with

the Choles'y decomposition. 3e#inson recursionis a fast method for -oeplit
http://en.wikipedia.org/wiki/Moore-Penrose_pseudoinversehttp://en.wikipedia.org/wiki/Pivot_elementhttp://en.wikipedia.org/wiki/LU_decompositionhttp://en.wikipedia.org/wiki/LU_decompositionhttp://en.wikipedia.org/wiki/Symmetric_matrixhttp://en.wikipedia.org/wiki/Positive-definite_matrixhttp://en.wikipedia.org/wiki/Cholesky_decompositionhttp://en.wikipedia.org/wiki/Levinson_recursionhttp://en.wikipedia.org/wiki/Toeplitz_matrixhttp://en.wikipedia.org/wiki/Moore-Penrose_pseudoinversehttp://en.wikipedia.org/wiki/Pivot_elementhttp://en.wikipedia.org/wiki/LU_decompositionhttp://en.wikipedia.org/wiki/LU_decompositionhttp://en.wikipedia.org/wiki/Symmetric_matrixhttp://en.wikipedia.org/wiki/Positive-definite_matrixhttp://en.wikipedia.org/wiki/Cholesky_decompositionhttp://en.wikipedia.org/wiki/Levinson_recursionhttp://en.wikipedia.org/wiki/Toeplitz_matrix


29/29

matrices. +pecial methods exist also for matrices with many ero elements

(so9called sparse matrices)" which appear often in applications.

/ completely different approach is often ta'en for #ery large systems"

which would otherwise ta'e too much time or memory. -he idea is to start

with an initial approximation to the solution (which does not ha#e to be

accurate at all)" and to change this approximation in se#eral steps to bring it

closer to the true solution. Once the approximation is sufficiently accurate"

this is ta'en to be the solution to the system. -his leads to the class

of iterati#e methods.
http://en.wikipedia.org/wiki/Toeplitz_matrixhttp://en.wikipedia.org/wiki/Sparse_matrixhttp://en.wikipedia.org/wiki/Iterative_methodhttp://en.wikipedia.org/wiki/Toeplitz_matrixhttp://en.wikipedia.org/wiki/Sparse_matrixhttp://en.wikipedia.org/wiki/Iterative_method

Documents

linear and nonlinear equations analysis using matlab