The Sensitivity Analysis of Mathematical Models Described ...hzp/index_files/presentations/ST1hossein.pdf · The Sensitivity Analysis of Mathematical Models Described by Differential

The Sensitivity Analysis of Mathematical ModelsDescribed by Differential Equations

Hossein Zivaripiran

Department of Computer Science

University of Toronto

Sensitivity Analysis – p.1/15

Outline

� Modeling with Differential Equations (IVPs, DDEs)

� Modeling and Sensitivity Analysis

� Numerical Sensitivity Analysis of IVPs

� DDEs and Sensitivity Analysis


Modeling with DE - Formulation

� An Initial Value Problem (IVP) for Ordinary Differential Equations (ODEs)

y′(t) = f(t, y(t))

y(t0) = y0




y′(t) = f(t, y(t))

y(t0) = y0

� Retarded Delay Differential Equations (RDDEs)

y′(t) = f(t, y(t), y(t− σ1), · · · , y(t− σν)) for t0 ≤ t ≤ tF

y(t) = φ(t), for t ≤ t0

σi = σi(t, y(t)) ≥ 0 delay (constant / time dependent / state dependent)

φ(t) history function (constant / time dependent)




y′(t) = f(t, y(t))

y(t0) = y0

� Retarded Delay Differential Equations (RDDEs)

y′(t) = f(t, y(t), y(t− σ1), · · · , y(t− σν)) for t0 ≤ t ≤ tF

y(t) = φ(t), for t ≤ t0

σi = σi(t, y(t)) ≥ 0 delay (constant / time dependent / state dependent)

φ(t) history function (constant / time dependent)

� Neutral Delay Differential Equations (NDDEs)

y′(t) = f(t, y(t), y(t− σ1), · · · , y(t− σν),

y′(t− σν+1), · · · , y′(t− σν+ω)) for t0 ≤ t ≤ tF

y(t) = φ(t), y′(t) = φ′(t), for t ≤ t0,


Modeling with DE - Some Areas of Application

Area Example

Ecology predator-prey

Epidemiology spread of infections

Immunology immune response models

HIV infection

Physiology human respiration system

Neural Networks

Cell Kinetics

Chemical Kinetics The Oregonator

Physics Ring Cavity Lasers , two-body problem of electrodynamics


Modeling with DE - Examples

� The Van der Pol Oscillator,

d2x

dt2− µ(1− x2)

dx

dt+ x = 0

using y1 = x and y2 = dxdt

,

y′

1(t) = y2(t)

y′

2(t) = µ(1− y21)y2 − y1



� The Van der Pol Oscillator with µ = 1, y1(0) = 2, y2(0) = 0

0 5 10 15 20−2.5

−2

−1.5

−1

−0.5

0

0.5

1

1.5

2

2.5

t

y1

0 5 10 15 20−3

−2

−1

0

1

2

3

t

y2



� An example of the famous Mackey-Glass DDEs proposed as a model for theproduction of white blood cells [Mackey and Glass 1977].

y′(t) =2y(t− 2)

1 + y(t− 2)9.65− y(t),

for t in [0, 100]. The history function is

φ(t) = 0.5 for t ≤ 0.



� The Mackey-Glass model

0 20 40 60 80 1000.2

0.4

0.6

0.8

1

1.2

1.4

1.6

t

y



� A neutral delay logistic Gause-type predator-prey system [Kuang 1991]

y′

1(t) = y1(t)(1− y1(t− τ)− ρy′

1(t− τ))−y2(t)y1(t)

2

y1(t)2 + 1

y′

2(t) = y2(t)

(

y1(t)2

y1(t)2 + 1− α

)

where α = 1/10 , ρ = 29/10 and τ = 21/50, for t in [0, 30]. The history functionsare

φ1(t) =33

100−

1

10t

φ2(t) =111

50+

1

10t

for t ≤ 0.



� The predator-prey model

0 5 10 15 20 25 300.31

0.32

0.33

0.34

0.35

0.36

0.37

t

y1

0 5 10 15 20 25 302.21

2.215

2.22

2.225

2.23

2.235

2.24

2.245

2.25

2.255

t

y2


Modeling with DE - Numerical Simulation

� Classical Theory of Step by Step Integration for IVPs.

� Runge-Kutta (RK).

� Linear Multistep (LM).






� Continuous Solution using Polynomial Approximation.

� Continuous Runge-Kutta (CRK).

� Linear Multistep methods have natural approximating polynomials.






� Continuous Solution using Polynomial Approximation.

� Continuous Runge-Kutta (CRK).

� Linear Multistep methods have natural approximating polynomials.

� DDEs : Combining an “interpolation” method (for evaluating delayed solutionvalues) with an ODE integration method (for solving the resulting “ODE”).


Modeling with DE - Special Difficulties with DDEs

� Derivative DiscontinuitiesIn general

φ′(t0) 6= f(t0, φ(t0), φ(t0 − σ1), · · · , φ(t0 − σν))

Due to the existence of delays, discontinuities propagate along the integrationinterval.Solution is smoothed for RDDEs but in general not for NDDEs.The RK and LM methods fail in presence of discontinuities.Treatment : Tracking Discontinuities and forcing them to be mesh points.

� Vanishing Delays

limt→t⋆

σi(t, y(t)) = 0

Causes the solver to choose a sequence of very small steps near t⋆. (h ≤ σi)Treatment : Using Special Interpolants to pass t⋆


Modeling and Sensitivity Analysis - Definitions

� Parameterized Models

� A parameterized IVP

y′(t;p) = f(t, y(t;p);p)

y(t0) = y0(p)

For example p = [µ] in the Van der Pol oscillator

y′

1(t) = y2(t)

y′

2(t) = µ(1− y21)y2 − y1

� A simple parameterized DDE

y′(t;p) = f(t, y(t;p), y(t− σ(t;p));p) for t0(p) ≤ t

y(t;p) = φ(t;p), for t ≤ t0(p)



� Forward Sensitivity Analysis

� The (first order) solution sensitivity with respect to the model parameter pi isdefined as the vector

si(t;p) = {∂

∂pi

}y(t;p), (i = 1, . . . ,L)

� The second order solution sensitivity with respect to the model parameterspi and pj is defined as the vector

rij(t;p) = {∂

∂pj

}si(t;p) = {∂2

∂pj∂pi

}y(t;p), (i, j = 1, . . . ,L)



� Adjoint Sensitivity Analysis

� We wish to evaluate the gradient ∂G∂pi

of

G(p) =

∫ tf

t0

g(t, y,p)dt

� or, alternatively, the gradient ∂g∂pi

of

g(t, y,p)

at time tf .

� More efficient if dim(g) < dim(y) and we need ∂∂p

for many parameters.


Modeling and Sensitivity Analysis - Importance

Sensitivity information can be used to:

� Estimate which parameters are most influential in affecting the behavior of thesimulation. Such information is crucial for

� Experimental Design

� Data Assimilation

� Reduction of complex nonlinear models

� Study of Dynamical Systems : Periodic orbits, the Lyapunov exponents, chaosindicators, and bifurcation analysis are fundamental objects for the completestudy of a dynamical system, and they require computation of the sensitivitieswith respect to the initial conditions of the problem.

� Evaluate optimization gradients and Jacobians in the setting of

� Dynamic Optimization

� Parameter Estimation


Numerical Sensitivity Analysis of IVPs - Forward

� Finite Difference Approach

{ ∂

∂pi

}y(t;p) ≈ y(t;p + ei∆pi) − y(t;p)

∆pi

Due to the rounding errors, the approximation is only O(√

Tol) with the best choice for∆pi.




{ ∂

∂pi

}y(t;p) ≈ y(t;p + ei∆pi) − y(t;p)

∆pi



� Internal Differentiation

y′(t;p) = f(t, y(t;p);p), y(t0) = y0(p)

⇓

Differentiation + Chain Rule + Clairaut’s Theorem

⇓

s′

i =∂f

∂ysi +

∂f

∂pi

, si(t0) =∂y0(p)

∂pi

, (i = 1, . . .L)




{ ∂

∂pi

}y(t;p) ≈ y(t;p + ei∆pi) − y(t;p)

∆pi



� Internal Differentiation

y′(t;p) = f(t, y(t;p);p), y(t0) = y0(p)

⇓


⇓

s′

i =∂f

∂ysi +

∂f

∂pi

, si(t0) =∂y0(p)

∂pi

, (i = 1, . . .L)

� Taylor Series method using extended rules of AD [Barrio 2006].⇒ second(or higher)-order sensitivities.


Numerical Sensitivity Analysis of IVPs - An Example

� The Van der Pol oscillator

y′

1(t) = y2(t)

y′

2(t) = µ(1− y21)y2 − y1

⇓

s′1(1)(t)

s′1(2)(t)

=

0 1

−2µy1y2 − 1 µ(1− y21)

s1(1)

s1(2)

+

0

(1− y21)y2

⇓

s′1(1)(t) = s1(2)

s′1(2)(t) = (−2µy1y2 − 1)s1(1)

+ µ(1− y21)s1(2)

+ (1− y21)y2


Numerical Sensitivity Analysis of IVPs - An Example

� The Van der Pol oscillator (∆p = 0.2). The decrease of y1 at t = 20 can be explained as second

order sensitivities being dominant ( ∂y1∂µ

(t = 20) ' 0 , ∂2y1∂µ2 (t = 20) < 0).

0 5 10 15 20−2.5

−2

−1.5

−1

−0.5

0

0.5

1

1.5

2

2.5

t

y1

0 5 10 15 20−8

−6

−4

−2

0

2

4

6

t

∂y1

∂µ

0 5 10 15 20−2.5

−2

−1.5

−1

−0.5

0

0.5

1

1.5

2

2.5

t

y 1(p

+∆

p)

0 5 10 15 20−2.5

−2

−1.5

−1

−0.5

0

0.5

1

1.5

2

2.5

t

y 1(p−

∆p)


DDEs and Sensitivity Analysis - Governing Equations

y′(t;p) = f(t, y(t;p), y(α(t, y;p);p), y′(α(t, y;p);p);p)

y(t;p) = φ(t;p), for t ≤ t0(p)

⇓


⇓

s′i(t) =∂f

∂ysi(t) +

∂f

∂y(αk)

(

y′(αk)

(

∂α

∂ysi(t) +

∂α

∂p

)

+ si(α)

)

+∂f

∂y′(α)

(

y′′(α)

(

∂α

∂ysi(t) +

∂α

∂p

)

+ s′i(α)

)

+∂f

∂p


DDEs and Sensitivity Analysis - Proper Handling of Jumps

� Hybrid ODE systems [Tolsma & Barton]

� Continuous transition at t = λ,

y(λ+) = y(λ−)

⇓

∂y

∂pl

(λ+) =∂y

∂pl

(λ−) +(

y′(λ−)− y′(λ+)) ∂λ

∂pl

� Triggered by,

g(t, y, y′;p) = 0

⇓

∂g

∂y′

(

∂

∂t

(

∂y

∂pl

)

+ y′′∂λ

∂pl

)

+∂g

∂y

(

∂y

∂pl

+ y′∂λ

∂pl

)

+∂g

∂pl

+∂g

∂t

∂λ

∂pl

= 0



� DDEs

� Discontinuity points of y′(t;p),

Λ(p) ≡ {λ1(p), λ2(p), . . .}.

� Identified by,

α(λr+1(p), y;p) = λr(p) r = 1, 2, . . .

� Can be viewed as λr+1(p) being a solution of,

g(t, y;p) = α(t, y;p)− λr(p) = 0.

� Using the result for hybrid ODEs,

∂λr+1(p)

∂pl

= −

∂α∂y

∂y∂pl

+ ∂α∂pl− ∂λr(p)

∂pl

∂α∂y

y′ + ∂α∂t

for r ≥ 1

∂λ1(p)

∂pl

=∂t0(p)

∂pl



� Integration (first order sensitivities)

1. Initialize (λ1 = t0(p)).

2. r ← 1.

3. Integrate the equations up to a C1-discontinuity point (λr+1).

4. Update the state variables (sensitivities) using

∂y

∂pl

(λ+r+1) =

∂y

∂pl

(λ−

r+1) +(

y′(λ−

r+1)− y′(λ+r+1)

) ∂λr+1(p)

∂pl

5. r ← r + 1 and restart (3).


DDEs and Sensitivity Analysis - Examples

� Mackey-Glass model

y′(t) =2y(t− 2)

1 + y(t− 2)9.65− y(t),

for t in [0, 100]. The history function is

φ(t) = 0.5 for t ≤ 0.

Parameters are

p = [τ, n, A]

where

y′(t) =2y(t− τ)

1 + y(t− τ)n− y(t),

and

φ(t) = A for t ≤ 0.



� Mackey-Glass model

0 20 40 60 80 1000.2

0.4

0.6

0.8

1

1.2

1.4

1.6

t

y

0 20 40 60 80 100−0.5

−0.4

−0.3

−0.2

−0.1

0

0.1

0.2

0.3

0.4

0.5

t

∂y

∂τ

0 20 40 60 80 100−0.1

−0.08

−0.06

−0.04

−0.02

0

0.02

0.04

t

∂y

∂n

0 20 40 60 80 1000

0.5

1

1.5

2

2.5

t

∂y

∂A



� The predator-prey model

y′

1(t) = y1(t)(1 − y1(t − τ) − ρy′

1(t − τ)) −y2(t)y1(t)2

y1(t)2 + 1

y′

2(t) = y2(t)

(

y1(t)2

y1(t)2 + 1− α

)

where α = 1/10 , ρ = 29/10 and τ = 21/50, for t in [0, 30]. The history functions are

φ1(t) =33

100−

1

10t

φ2(t) =111

50+

1

10t

for t ≤ 0.

Parameters are

p = [τ, ρ, α, a, b, c, d]

where

φ1(t) = a + b t

φ2(t) = c + d t



� The predator-prey model (structure-related parameters, y1)

0 5 10 15 20 25 300.31

0.32

0.33

0.34

0.35

0.36

0.37

t

y1

0 5 10 15 20 25 30−2.5

−2

−1.5

−1

−0.5

0

0.5

1

1.5

2

2.5

t

∂y1

∂τ

0 5 10 15 20 25 30−0.08

−0.06

−0.04

−0.02

0

0.02

0.04

0.06

0.08

t

∂y1

∂ρ

0 5 10 15 20 25 300

0.5

1

1.5

2

2.5

t

∂y1

∂α



� The predator-prey model (structure-related parameters, y2)

0 5 10 15 20 25 302.21

2.215

2.22

2.225

2.23

2.235

2.24

2.245

2.25

2.255

t

y2

0 5 10 15 20 25 30−0.2

−0.1

0

0.1

0.2

0.3

0.4

0.5

t

∂y2

∂τ

0 5 10 15 20 25 30−0.005

0

0.005

0.01

0.015

0.02

0.025

0.03

t

∂y2

∂ρ

0 5 10 15 20 25 30−18

−16

−14

−12

−10

−8

−6

−4

−2

0

t

∂y2

∂α



� The predator-prey model (history-related parameters, y1)

0 5 10 15 20 25 30−0.2

0

0.2

0.4

0.6

0.8

1

1.2

t

∂y1

∂a

0 5 10 15 20 25 30−0.5

−0.4

−0.3

−0.2

−0.1

0

0.1

0.2

t

∂y1

∂b

0 5 10 15 20 25 30−0.14

−0.12

−0.1

−0.08

−0.06

−0.04

−0.02

0

0.02

t

∂y1

∂c

0 5 10 15 20 25 30−0.01

0

0.01

0.02

0.03

0.04

0.05

0.06

t

∂y1

∂d



� The predator-prey model (history-related parameters, y2)

0 5 10 15 20 25 30−0.5

0

0.5

1

1.5

2

t

∂y2

∂a

0 5 10 15 20 25 30−0.8

−0.7

−0.6

−0.5

−0.4

−0.3

−0.2

−0.1

0

t

∂y2

∂b

0 5 10 15 20 25 30−0.2

0

0.2

0.4

0.6

0.8

1

1.2

t

∂y2

∂c

0 5 10 15 20 25 30−0.45

−0.4

−0.35

−0.3

−0.25

−0.2

−0.15

−0.1

−0.05

0

0.05

t

∂y2

∂d


The End

Thank you for your attention!

Questions?


Documents

The Sensitivity Analysis of Mathematical Models Described ...hzp/index_files/presentations/ST1hossein.pdf · The Sensitivity Analysis of Mathematical Models Described by Differential