Methods for Ordinary Differential Equations

Lecture 10

Alessandra Nardi

Thanks to Prof. Jacob White, Deepak Ramaswamy Jaime Peraire, Michal Rewienski, and Karen Veroy

Outline

• Transient Analysis of dynamical circuits– i.e., circuits containing C and/or L

• Examples

• Solution of Ordinary Differential Equations (Initial Value Problems – IVP)– Forward Euler (FE), Backward Euler (BE) and

Trapezoidal Rule (TR)– Multistep methods– Convergence

Ground Plane

Signal Wire

LogicGate

LogicGate

• Metal Wires carry signals from gate to gate.• How long is the signal delayed?

Wire and ground plane form a capacitor

Wire has resistance

Application ProblemsSignal Transmission in an Integrated Circuit

capacitor

resistor

• Model wire resistance with resistors.• Model wire-plane capacitance with capacitors.

Constructing the Model• Cut the wire into sections.

Application ProblemsSignal Transmission in an IC – Circuit Model

Nodal Equations Yields 2x2 System

C1

R2

R1 R3 C2

Constitutive Equations

cc

dvi C

dt

1R Ri v

R

Conservation Laws

1 1 20C R Ri i i

2 3 20C R Ri i i

1

1 2 21 1

2 22

2 3 2

1 1 1

0

0 1 1 1

dvR R RC vdt

C vdv

R R Rdt

1Ri

1Ci

2Ri

2Ci

3Ri

1v 2v

Application ProblemsSignal Transmission in an IC – 2x2 example

eigenvectors

1

1 2 21 1

2 22

2 3 2

1 1 1

0

0 1 1 1

dvR R RC vdt

C vdv

R R Rdt

1 2 1 3 2Let 1, 10, 1C C R R R 1.1 1.0

1.0 1.1

A

dxx

dt

11 1 0.1 0 1 1

1 1 0 2.1 1 1A

Eigenvalues and Eigenvectors

Eigenvalues

Application ProblemsSignal Transmission in an IC – 2x2 example

1Change of variab (le ) ( ) ( ) (s ): Ey t x t y t E x t

1

1 2 1 2

10 0

0 0

0 0n n

n

A E E E E E

E

E

Eigendecomposition:

0

( )Substituting: ( ), (0)

dEy tAEy t Ey x

dt

11Multiply by ( ) : ( )dy t

E AE tdt

E y 0 0

10 0

0 0

( )n

y t

0Consider an ODE( )

( ), (0): dx t

Ax t x xdt

An Aside on Eigenanalysis

( )Decoupling: ( ) ( ) (0)iti

i i i

dy ty t y t e y

dt

1 0 0

0 0

0 0

From last slide:( ) ( )

n

dy tdt

y t

DecoupledEquations!

1) Determine , E 1 0 0

3) Compute ( ) 0 0 (0)

0 0 n

t

t

e

y t y

e

102) Compute (0) y E x

4) ( ) ( ) x t Ey t

0Steps for solvi( )

( ), (0)ng dx t

Ax t x xdt

An Aside on Eigenanalysis

1(0) 1v

2 (0) 0v

Notice two time scale behavior• v1 and v2 come together quickly (fast eigenmode).• v1 and v2 decay to zero slowly (slow eigenmode).

Application ProblemsSignal Transmission in an IC – 2x2 example

Circuit Equation Formulation

• For dynamical circuits the Sparse Tableau equations can be written compactly:

• For sake of simplicity, we shall discuss first order ODEs in the form:

riablescircuit va of vector theis where

)0(

0),,)(

(

0

x

xx

txdt

tdxF

),()(

txfdt

tdx

Ordinary Differential EquationsInitial Value Problems (IVP)

Typically analytic solutions are not available

solve it numerically

.condition initial given the intervalan in

)(

),()(

:(IVP) Problem Value Initial Solve

00

00

x,T][t

xtx

txfdt

tdx

• Not necessarily a solution exists and is unique for:

• It turns out that, under rather mild conditions on the continuity and differentiability of F, it can be proven that there exists a unique solution.

• Also, for sake of simplicity only consider

linear case:

0),,( tydt

dyF

We shall assume that has a unique solution 0),,( tydt

dyF

00 )(

)()(

xtx

tAxdt

tdx

Ordinary Differential Equations Assumptions and Simplifications

First - Discretize Time

Second - Represent x(t) using values at ti

ˆ ( )llx x t

Approx. sol’n

Exact sol’n

Third - Approximate using the discrete ( )ldx t

dtˆ 'slx

1 1

1

ˆ ˆ ˆ ˆExample: ( )

l l l l

ll l

d x x x xx t or

dt t t

Lt T1t 2t 1Lt 0

t t t

1t 2t 3t Lt0

3x̂ 4x̂1x̂

2x̂

Finite Difference MethodsBasic Concepts

lt 1lt t

x

1( ) ( )slope l lx t x t

t

slope ( )ldx t

dt

1( ) ( ) ( )l l lx t x t t A x t

1

1

( ) ( )( ) ( )

or

( ) ( ) ( )

l ll l

l l l

x t x tdx t A x t

dt t

x t x t t A x t

Finite Difference Methods Forward Euler Approximation

1t 2t t

x

(0)tAx

11 ˆ( ) (0) 0x t x x tAx

3t

2 1 12 ˆ ˆ ˆ( )x t x x tAx

1ˆtAx

1 1ˆ ˆ ˆ( ) L L LLx t x x tAx

Finite Difference Methods Forward Euler Algorithm

lt 1lt t

x 1slope ( )l

dx t

dt

1( ) ( )slope l lx t x t

t

1 1( ) ( ) ( )l l lx t x t t A x t

11 1

1 1

( ) ( )( ) ( )

or

( ) ( ) ( )

l ll l

l l l

x t x tdx t A x t

dt t

x t x t t A x t

Finite Difference Methods Backward Euler Approximation

1t 2t t

x

1ˆtAx

2ˆtAx

1 11 ˆ ˆ( ) (0)x t x x tAx

Solve with Gaussian Elimination

1ˆ[ ] (0)I tA x x

1 1ˆ ˆ( ) [ ]L LLx t x I tA x

2 1 12 ˆ ˆ( ) [ ]x t x I tA x

Finite Difference Methods Backward Euler Algorithm

1

1

1

1 1

1( ( ) ( ))

21

( ( ) ( ))2

( ) ( )

1( ) ( ) ( ( ) ( ))

2

l l

l l

l l

l l l l

d dx t x t

dt dt

Ax t Ax t

x t x t

t

x t x t tA x t x t

t

x

1( ) ( )slope l lx t x t

t

slope ( )ldx t

dt

1slope ( )l

dx t

dt

1 1

1 1( ( ) ( )) ( ( ) ( ))

2 2l l l lx t tAx t x t tAx t

Finite Difference Methods Trapezoidal Rule Approximation

1t 2t t

x

1ˆ (0)2 2

t tI A x I A x

Solve with Gaussian Elimination

1 11 ˆ ˆ( ) (0) (0)

2

tx t x x Ax Ax

12 1

2

11

ˆ ˆ( )2 2

ˆ ˆ( )2 2

L LL

t tx t x I A I A x

t tx t x I A I A x

Finite Difference Methods Trapezoidal Rule Algorithm

1

1( ) (( ) )) )( (l

l

t

l l t

dx t x t x t A dx t xA

dt

lt 1lt

1

( )l

l

t

tAx d

1( )ltAx t BE

( )ltAx t FE

( ) ( )2 l l

tAx t Ax t

Trap

Finite Difference Methods Numerical Integration View

Trap Rule, Forward-Euler, Backward-Euler Are all one-step methods

Forward-Euler is simplest No equation solution explicit method. Boxcar approximation to integralBackward-Euler is more expensive Equation solution each step implicit method Trapezoidal Rule might be more accurate

Equation solution each step implicit method

Trapezoidal approximation to integral

1 2 3ˆ ˆ ˆ ˆ is computed using only , not , , etc.l l l lx x x x

Finite Difference Methods Summary of Basic Concepts

( ) ( ( ), ( ))dx t f x t u t

dtNonlinear Differential Equation:

k-Step Multistep Approach: 0 0

ˆ ˆ ,k k

l j l jj j l j

j j

x t f x u t

Solution at discrete points

Time discretization

Multistep coefficients

2ˆ lx

lt1lt 2lt 3lt l kt

ˆ lx1ˆ lx

ˆ l kx

Multistep Methods Basic Equations

Multistep Equation:

1 1 1,l l l lx t x t t f x t u t

FE Discrete Equation: 1 11ˆ ˆ ˆ ,l l l

lx x t f x u t

0 1 0 11, 1, 1, 0, 1k

Forward-Euler Approximation:

Multistep Coefficients:

Multistep Coefficients:

BE Discrete Equation:

0 1 0 11, 1, 1, 1, 0k 1ˆ ˆ ˆ ,l l l

lx x t f x u t

Trap Discrete Equation: 1 11ˆ ˆ ˆ ˆ, ,

2l l l l

l l

tx x f x u t f x u t

0 1 0 1

1 11, 1, 1, ,

2 2k Multistep Coefficients:

0 0

ˆ ˆ ,k k

l j l jj j l j

j j

x t f x u t

Multistep Methods – Common AlgorithmsTR, BE, FE are one-step methods

Multistep Equation:

01) If 0 the multistep method is implicit 2) A step multistep method uses previous ' and 'k k x s f s

03) A normalization is needed, 1 is common

4) A -step method has 2 1 free coefficientsk k

How does one pick good coefficients?

Want the highest accuracy

0 0

ˆ ˆ ,k k

l j l jj j l j

j j

x t f x u t

Multistep Methods Definition and Observations

Definition: A finite-difference method for solving initial value problems on [0,T] is said to be

convergent if given any A and any initial condition

0,

ˆmax 0 as t 0lT

lt

x x l t

exactx

tˆ computed with

2lx

ˆ computed with tlx

Multistep Methods – Convergence Analysis

Convergence Definition

Definition: A multi-step method for solving initial value problems on [0,T] is said to be order p

convergent if given any A and any initial condition

0,

ˆmaxpl

Tl

t

x x l t C t

0for all less than a given t t

Forward- and Backward-Euler are order 1 convergentTrapezoidal Rule is order 2 convergent

Multistep Methods – Convergence Analysis

Order-p Convergence

Multistep Methods – Convergence Analysis

Two types of error

made.been haserror previous no assuming ,solution theof value

exact theand ˆ valuecomputed ebetween th difference theis

at methodn integratioan of (LTE)Error Truncation Local The

1

1

1

)x(t

x

t

l

l

l

exactly.known iscondition

initial only the that assuming ,solution theof value

exact theand ˆ valuecomputed ebetween th difference theis

at methodn integratioan of (GTE)Error Truncation Global The

1

1

1

)x(t

x

t

l

l

l

• For convergence we need to look at max error over the whole time interval [0,T]– We look at GTE

• Not enough to look at LTE, in fact:– As I take smaller and smaller timesteps t, I would

like my solution to approach exact solution better and better over the whole time interval, even though I have to add up LTE from more timesteps.

Multistep Methods – Convergence Analysis

Two conditions for Convergence

1) Local Condition: One step errors are small (consistency)

2) Global Condition: The single step errors do not grow too quickly (stability)

Typically verified using Taylor Series

All one-step methods are stable in this sense.

Multistep Methods – Convergence Analysis

Two conditions for Convergence

Definition: A one-step method for solving initial value problems on an interval [0,T] is said to be consistent if for any A and any initial condition

1ˆ0 as t 0

x x t

t

One-step Methods – Convergence Analysis

Consistency definition

Forward-Euler definition

22

2

0()

2 0

dxtdxxtxt

dtdt

Expanding in about zero yields t

1ˆ00 xxtAx

Noting that (0)(0) and subtractingdxAx

dt

22

12 ˆ

2

tdxxxt

dt

Proves the theorem if

derivatives of x are bounded

0,t

One-step Methods – Convergence Analysis

Consistency for Forward Euler

Forward-Euler definition

1l

xltxlttAxlte

Expanding in about yields tlt 1

ˆˆ ˆlllxxtAx

where is the "one-step" error bounded byle

2

2

[0,]2 , where 0.5maxl

T

dxeCtC

dt

One-step Methods – Convergence Analysis

Convergence Analysis for Forward Euler

ˆ Define the "Global" errorll

Exxlt

1ˆˆ 1lllxxltItAxxlte

Subtracting the previous slide equations

Taking norms and using the bound on le

1 lll

EItAEe

2 1

21

ll

l

EItAECt

tAECt

One-step Methods – Convergence Analysis

Convergence Analysis for Forward Euler

10

1,0, If 0ll

uubu

To prove, first write as a power series and sum l

u

1

0

111

11

l lj l

j

ubb

Then l

leub

A lemma bounding difference equation solutions

One-step Methods – Convergence Analysis

A helpful bound on difference equations

To finish, note (1)(1) ll

ee

1111

11

lllle

ubbb

2 1

1ll

EtAECt

b

Mapping the global error equation to the lemma

One-step Methods – Convergence Analysis

A helpful bound on difference equations

2 1

1ll

EtAECt

b

Applying the lemma and cancelling terms

2

ltAe

CttA

Finally noting that , ltT

0, maxAT l

lL

CEet

A

One-step Methods – Convergence Analysis

Back to Convergence Analysis for Forward Euler

0, maxAT l

lL

CEet

A

• Forward-Euler is order 1 convergent• The bound grows exponentially with time interval• C is related to the solution second derivative• The bound grows exponentially fast with norm(A).

One-step Methods – Convergence Analysis

Observations about Convergence Analysis for FE

Summary

• Transient Analysis of dynamical circuits– i.e., circuits containing C and/or L

• Examples

• Solution of Ordinary Differential Equations (Initial Value Problems – IVP)– Forward Euler (FE), Backward Euler (BE) and

Trapezoidal Rule (TR)– Multistep methods– Convergence