1

EE 616 Computer Aided Analysis of Electronic Networks

Lecture 5

Instructor: Dr. J. A. Starzyk, ProfessorSchool of EECSOhio UniversityAthens, OH, 45701

09/19/2005

Note: materials in this lecture are from the notes of EE219A UC-berkeleyhttp://www- cad.eecs.berkeley.edu/~nardi/EE219A/contents.html

2

Outline

Nonlinear problems Iterative Methods Newton’s Method

– Derivation of Newton– Quadratic Convergence– Examples– Convergence Testing

Multidimensonal Newton Method– Basic Algorithm – Quadratic convergence– Application to circuits

3

Need to Solve

( 1) 0d

t

VV

d sI I e

0

1

IrI1 I

d

0)1(1

0

11

1

1

IeIeR

III

tV

e

s

dr

11)( Ieg

DC Analysis of Nonlinear Circuits - Example

4

Nonlinear Equations

Given g(V)=I It can be expressed as: f(V)=g(V)-I

Solve g(V)=I equivalent to solve f(V)=0

Hard to find analytical solution for f(x)=0

Solve iteratively

5

Nonlinear Equations – Iterative Methods

Start from an initial value x0

Generate a sequence of iterate xn-1, xn, xn+1 which hopefully converges to the solution x*

Iterates are generated according to an iteration function F: xn+1=F(xn)

Ask• When does it converge to correct solution ?• What is the convergence rate ?

6

Newton-Raphson (NR) Method

Consists of linearizing the system.Want to solve f(x)=0 Replace f(x) with its linearized version and solve.

Note: at each step need to evaluate f and f’

functionIterationxfxdx

dfxx

xxxdx

dfxfxf

iesTaylor Serxxxdx

dfxfxf

kkkk

kkkkk

)()(

))(()()(

))(()()(

11

11

***

7

Newton-Raphson Method – Graphical View

8

Newton-Raphson Method – Algorithm

Define iteration

Do k = 0 to ….?

until convergence

How about convergence? An iteration {x(k)} is said to converge with order q if there exists a

vector norm such that for each k N:

qkk xxxx ˆˆ1

)()(1

1 kkkk xfxdx

dfxx

9

2* * * 2

2

*

0 ( ) ( ) ( )( ) ( )( )

some [ , ]

k k k k

k

df d ff x f x x x x x x x

dx dx

x x x

Mean Value theoremtruncates Taylor series

But

10 ( ) ( )( )k k k kdff x x x x

dx by Newton

definition

Newton-Raphson Method – Convergence

10

Subtracting

21 * 1 * 2

2 ( ) [ ( )] ( )( )k k kdf d f

x x x x x xdx d x

Convergence is quadratic

21 * * 2

2( )( ) ( )( )k k kdf d fx x x x x x

dx d x

Dividing

21

2

21 * *

Let [ ( )] ( )

then

k k

k k k

df d fx x K

dx d x

x x K x x

Newton-Raphson Method – Convergence

11

Local Convergence Theorem

If

Then Newton’s method converges given a sufficiently close initial guess (and convergence is quadratic)

boundedisK

boundeddx

fdb

roay from zebounded awdx

dfa

)

)

2

2

Newton-Raphson Method – Convergence

12

0 Initial Guess,= 0x k

Repeat { 1

k

k k kf x

x x f xx

} Until ?

1 1? ?k k kf x thresholdx x threshold

1k k

Newton-Raphson Method – Convergence

13

Need a "delta-x" check to avoid false convergence

X

f(x)

1kx kx *x

1 a

kff x

1 1 a r

k k kx xx x x

Newton-Raphson Method – Convergence Check

False convergence since increment in f small but with large increment in X

14

Also need an " " check to avoid false convergencef x

X

f(x)

1kx kx

*x

1 a

kff x

1 1 a r

k k kx xx x x

Newton-Raphson Method – Convergence Check

False convergence since increment in x small but with large increment in f

15

Newton-Raphson Method – Convergence

16

Convergence Depends on a Good Initial Guess

X

f(x)

0x

1x2x 0x

1x

Newton-Raphson Method – Local Convergence

17

Convergence Depends on a Good Initial Guess

Newton-Raphson Method – Local Convergence

18

Nodal Analysis

1 2At Node 1: 0i i

NonlinearResistors

i g v

1bv

+

-

1i2i 2

bv

3i+ -

3bv

+

-

1v2v

1 1 2 0g v g v v

3 2At Node 2: 0i i

3 1 2 0g v g v v

Two coupled nonlinear equations in two unknowns

Nonlinear Problems – Multidimensional Example

19

* *Problem: Find such tha 0t x F x

Multidimensional Newton Method

functionIterationxFxJxx

atrixJacobian M

x

xF

x

xF

x

xF

x

xF

xJ

iesTaylor SerxxxJxFxF

kkkk

N

NN

N

)()(

)()(

)()(

)(

))(()()(

11

1

1

1

1

***

20

)()()( :solve Instead

sparse).not is(it )( computenot Do

)()(:

1

1

11

kkkk

k

kkkk

xFxxxJ

xJ

xFxJxxIteration

Each iteration requires:

1. Evaluation of F(xk)

2. Computation of J(xk)

3. Solution of a linear system of algebraic equations whose coefficient matrix is J(xk) and whose RHS is -F(xk)

Multidimensional Newton Method – Computational Aspects

21

0 Initial Guess,= 0x k

Repeat {

1 1Solve for k k k k kFJ x x x F x x

} Until 11 , small en ug o hkk kx x f x

1k k

Compute ,k kFF x J x

Multidimensional Newton Method – Algorithm

22

If

1) Inverse is boundedkFa J x

) Derivative is Lipschitz ContF Fb J x J y x y

Then Newton’s method converges given a sufficiently close initial guess (and convergence is quadratic)

Local Convergence Theorem

Multidimensional Newton Method – Convergence

23

Application of NR to Circuit EquationsCompanion Network

Applying NR to the system of equations we find that at iteration k+1:– all the coefficients of KCL, KVL and of BCE of the linear

elements remain unchanged with respect to iteration k– Nonlinear elements are represented by a linearization of

BCE around iteration k

This system of equations can be interpreted as the STA of a linear circuit (companion network) whose elements are specified by the linearized BCE.

24

General procedure: the NR method applied to a nonlinear circuit (whose eqns are formulated in the STA form) produces at each iteration the STA eqns of a linear resistive circuit obtained by linearizing the BCE of the nonlinear elements and leaving all the other BCE unmodified

After the linear circuit is produced, there is no need to stick to STA, but other methods (such as MNA) may be used to assemble the circuit eqns

Application of NR to Circuit EquationsCompanion Network

25

Note: G0 and Id depend on the iteration count k G0=G0(k) and Id=Id(k)

Application of NR to Circuit EquationsCompanion Network – MNA templates

26

Application of NR to Circuit EquationsCompanion Network – MNA templates

27

Modeling a MOSFET(MOS Level 1, linear regime)

d

28

Modeling a MOSFET(MOS Level 1, linear regime)

gds

29

DC Analysis Flow Diagram

For each state variable in the system

30

Implications

Device model equations must be continuous with continuous derivatives and derivative calculation must be accurate derivative of function

Watch out for floating nodes (If a node becomes disconnected, then J(x) is singular)

Give good initial guess for x(0)

Most model computations produce errors in function values and derivatives.

– Want to have convergence criteria || x(k+1) - x(k) || < such that > than model errors.

31

Summary

Nonlinear problems Iterative Methods Newton’s Method

– Derivation of Newton– Quadratic Convergence– Examples– Convergence Testing

Multidimensional Newton Method– Basic Algorithm – Quadratic convergence– Application to circuits

32

Improving convergence

Improve Models (80% of problems) Improve Algorithms (20% of problems)

Focus on new algorithms:Limiting Schemes

Continuations Schemes

33

Outline

Limiting Schemes– Direction Corrupting– Non corrupting (Damped Newton)

Globally Convergent if Jacobian is NonsingularDifficulty with Singular Jacobians

Continuation Schemes– Source stepping– More General Continuation Scheme– Improving Efficiency

Better first guess for each continuation step

34

At a local minimum, 0f

x

LocalMinimum

Multidimensional Case: is singularFJ x

Multidimensional Newton MethodConvergence Problems – Local Minimum

35

f(x)

X0x

1x

Must Somehow Limit the changes in X

Multidimensional Newton MethodConvergence Problems – Nearly singular

36

Multidimensional Newton MethodConvergence Problems - Overflow

f(x)

X0x 1x

Must Somehow Limit the changes in f(x)

37

0 Initial Guess,= 0x k Repeat {

1 1Solve for k k k kFJ x x F x x

} Until 1 1 , small enoughkkx F x 1k k

Compute ,k kFF x J x

Newton Algorithm for Solving 0F x

1 1limitedk k kx x x

Newton Method with Limiting

38

1limited k

ix

• NonCorrupting

1 1 if k ki ix x

1 otherwisekisign x

• Direction Corrupting

1 1limited k kx x

1min 1,

kx

1kx 1limited kx

1kx 1limited kx

Heuristics, No Guarantee of Global Convergence

Newton Method with LimitingLimiting Methods

Scales proportionally In all dimensions

Scales differently different dimensions

39

General Damping Scheme

Key Idea: Line Search

21

2Pick to minimize k k k kF x x

Method Performs a one-dimensional search in Newton Direction

21 1 1

2

Tk k k k k k k k kF x x F x x F x x

1 1k k k kx x x

1 1Solve for k k k kFJ x x F x x

Newton Method with LimitingDamped Newton Scheme

40

If

1) Inverse is boundedkFa J x

) Derivative is Lipschitz ContF Fb J x J y x y

Then

There exists a set of ' 0,1 such thatk s

1 1 with <1k k k k kF x F x x F x

Every Step reduces F-- Global Convergence!

Newton Method with LimitingDamped Newton – Convergence Theorem

41

X0x1x

2x1Dx

Damped Newton Methods “push” iterates to local minimumsFinds the points where Jacobian is Singular

Newton Method with LimitingDamped Newton – Singular Jacobian Problem

42

Solve , 0 where:F x

a) 0 ,0 0 is easy to solveF x

b) 1 ,1F x F x

c) is sufficiently smoothx

Starts the continuation

Ends the continuation

Hard to insure!

Newton with Continuation schemes Basic Concepts - General setting

Newton converges given a close initial guess

Idea: Generate a sequence of problems, such that a problem is a good initial guess for the following one

0)(xF

43

Solve 0 ,0 , 0prevF x x x

While 1 {

}

0prevx x

Try to Solve , 0 with NewtonF x

If Newton Converged

, 2,prevx x Else

,1

2 prev

Newton with Continuation schemes Basic Concepts – Template Algorithm

44

Newton with Continuation schemes Basic Concepts – Source Stepping Example

45

Diode

1, 0diode sf v i v v V

R

+-Vs

R v

, 1Not dependent!diodef v i v

v v R

Source Stepping Does Not Alter Jacobian

Newton with Continuation schemes Basic Concepts – Source Stepping Example

46

0 ,0= 0 00 F x x Observations

0 ,0F xI

x

=1 1 ,1 1F x F x

0 ,0 1F x F x

x x

Problem is easy to solve and Jacobian definitely

nonsingular.

Back to the original problem and original Jacobian

, 1F x F x x

Newton with Continuation schemes Jacobian Altering Scheme

(1),1)

47

Summary

Newton’s Method works fine: – given a close enough initial guess

In case Newton does not converge:– Limiting Schemes

Direction CorruptingNon corrupting (Damped Newton)

– Globally Convergent if Jacobian is Nonsingular– Difficulty with Singular Jacobians

– Continuation SchemesSource steppingMore General Continuation Scheme Improving Efficiency

– Better first guess for each continuation step