Prediction of Financial Processes

Prediction of Financial Processes

Parameter Estimation in Stochastic Differential Equations

by Continuous Optimization

4th International Summer SchoolAchievements and Applications of Contemporary Informatics, Mathematics and PhysicsNational University of Technology of the UkraineKiev, Ukraine, August 5-16, 2009

GerhardGerhardGerhardGerhardGerhardGerhardGerhardGerhard--------Wilhelm Weber Wilhelm Weber Wilhelm Weber Wilhelm Weber Wilhelm Weber Wilhelm Weber Wilhelm Weber Wilhelm Weber **

Vefa Gafarova, Nüket Erbil, Cem Ali Gökçen, Azer Kerimov

Institute of Applied Mathematics Institute of Applied Mathematics Middle East Technical University, Ankara, TurkeyMiddle East Technical University, Ankara, Turkey

** Faculty of Economics, Management and Law, Universi ty of Siegen, GermanyFaculty of Economics, Management and Law, Universi ty of Siegen, GermanyCenter for Research on Optimization and Control, Univ ersity of Aveiro, Portugal

Pakize Taylan Dept. Mathematics, Dicle University, Diyarbakır, Tu rkeyDept. Mathematics, Dicle University, Diyarbakır, Tu rkey

by Continuous Optimization

• Stochastic Differential Equations

• Parameter Estimation

• Various Statistical Models

• C-MARS

Outline

• Accuracy vs. Stability

• Tikhonov Regularization

• Conic Quadratic Programming

• Nonlinear Regression

• Portfolio Optimization

• Outlook and Conclusion

Stock Markets

drift and diffusion term

( , ) ( , )= +t t t tdX a X t dt b X t dW

Stochastic Differential Equations

Wiener process

(0, ) ( [0, ])∈tW N t t T




Wiener process

(0, ) ( [0, ])∈tW N t t T

Ex.: price , wealth , interest rate , volatility

processes

Input vector and output variable Y ;

linear regression :

( )1 2, ,...,T

mX X X X=

1 01

( ,..., ) ,ε β β ε=

= + = + +∑m

m j jj

Y E Y X X X

Regression

which minimizes( )0 1, ,...,T

mβ β β β=

( ) ( )2

1

:N

Ti i

i

RSS y x=

= −∑β β

( ) 1ˆ ,T TX X X y−

=β

( ) 1 2ˆCov( ) Tβ X X σ−

=

are estimated by a smoothing on a single coordinate.jf

Generalized Additive Models

( ) ( )1 2 01

, ,..., β=

+= ∑i i i i m ij j

m

j

E x fx x xY

Standard convention .

• Backfitting algorithm (Gauss-Seidel)

it “cycles” and iterates.

( )0ˆ ,β

≠

= − −∑i i kjj

ikk

r y f x

( )( ): 0=j ijE f x

• Given data

• penalized residual sum of squares

22''

0 0( , ,..., ) : ( ) ( )β β

= − − + ∑ ∑ ∑ ∫bN m m

1 m i j ij j j jPRSS f f y f x f t dtjµ

( , ) ( = 1,2,..., ),i iy x i N


• New estimation methods for additive model with CQP :

0 01 1 1

( , ,..., ) : ( ) ( )β β= = =

= − − +

∑ ∑ ∑ ∫1 m i j ij j j ji j j a

PRSS f f y f x f t dtjµ

0.µ ≥j

0, ,

2

20

1 1

2''

min

subject to ( ) , 0,

( ) ( 1,2,..., ),

=

− − ≤ ≥

≤ =

∑ ∑

∫

t β f

N m

i j iji= j

j j j j

t

y β f x t t

f t dt M j m

jdj jθ=∑


splines:

By discretizing, we get

1

( ) ( ).j jj l l

l

f x h xθ=

=∑

0, ,

2 20 2

2

0 2

min

subject to ( , ) , 0,

( , ) ( 1,..., ).

β θ

β θ

≤ ≥

≤ =

t β f

j j

t

W t t

V M j m

0, ,

2

20

1 1

2''

min

subject to ( ) , 0,

( ) ( 1,2,..., ),

=

− − ≤ ≥

≤ =

∑ ∑

∫

t β f

N m

i j iji= j

j j j j

t

y β f x t t

f t dt M j m

jdj jθ=∑


splines:


1

( ) ( ).j jj l l

l

f x h xθ=

=∑

0, ,

2 20 2

2

0 2

min


( , ) ( 1,..., ).

β θ

β θ

≤ ≥

≤ =

t β f

j j

t

W t t

V M j m

0, ,

2

20

1 1

2''

min

subject to ( ) , 0,

( ) ( 1,2,..., ),

=

− − ≤ ≥

≤ =

∑ ∑

∫

t β f

N m

i j iji= j

j j j j

t

y β f x t t

f t dt M j m

jdj jθ=∑


splines:


1

( ) ( ).j jj l l

l

f x h xθ=

=∑

0, ,

2 20 2

2

0 2

min


( , ) ( 1,..., ).

β θ

β θ

≤ ≥

≤ =

t β f

j j

t

W t t

V M j m


: ( ) ( )⋅j j j j jInd = d D v V

MARS

y

• ••

•

••

••

•

y

••

•

••

••

•

τ x

• ••

••

•• •

••

••

•••

+( , )=[ ( )]c x x +τ + −τ( , )=[ ( )]-c x x +τ − −τ

τ x

• ••

••

•• •

••

••

•••

+( , )=[ ( )]c x x +τ + −τ( , )=[ ( )]-c x x +τ − −τ r egression w ith

( )max

1 2

2 22 2,

1 1 1, ( )( , )

: ( ) ( )α

αα α α

θ ψ= = = <

∈=

= − + ∑ ∑ ∑ ∑ ∫MN

m mi i m r s m

i m r sr s V m

PRSS y f D dmx t tµ

C-MARS

Tradeoff between both accuracy and complexity.

{ }

{ }

1 2

1 2

1 2 1 2

( ) : | 1,2,...,

: ( , ,..., )

( , )

: , , 0,1

Km

mj m

m Tm m m

V m j K

t t t

κ

α α αα α α α α

= =

== + ∈

t =

where

( )1 2, ( ) : ( )m m m m

r s m m r sD t tα αα αψ ψ= ∂ ∂ ∂t t

Tikhonov regularization:

2 2

22( )= − +PRSS y d Lθθθθ µµµµψ θψ θψ θψ θ

2θL

C-MARS

Conic quadratic programming:

,

2

2

subject to

min ,

( ) ,

θ

≤

≤

tt

td y

ML

ψ θ −ψ θ −ψ θ −ψ θ −

θθθθ

2( )−ψ θψ θψ θψ θy d

Tikhonov regularization:

2 2

22( )= − +PRSS y d Lθθθθ µµµµψ θψ θψ θψ θ

2θL

C-MARS

Conic quadratic programming:

,

2

2

subject to

min ,

( ) ,

θ

≤

≤

tt

td y

ML

ψ θ −ψ θ −ψ θ −ψ θ −

θθθθ

2( )−ψ θψ θψ θψ θy d

cluster

C-MARS

cluster

robust optimization



Stochastic Differential Equations Revisited

Wiener process

(0, ) ( [0, ])∈tW N t t T

Ex.: price , wealth , interest rate , volatility ,

processes




Wiener process

(0, ) ( [0, ])∈tW N t t T

bioinformatics, biotechnology(fermentation, population dynamics)

Universiti Teknologi Malaysia

Ex.:



Stochastic Differential Equations Revisited

Wiener process

(0, ) ( [0, ])∈tW N t t T

Ex.: price , wealth , interest rate , volatility ,

processes

Milstein Scheme :

( )21 1 1 1 1

1ˆ ˆ ˆ ˆ ˆ( , )( ) ( , )( ) ( )( , ) ( ) ( )2+ + + + +′= + − + − + − − −j j j j j j j j j j j j j j j jX X a X t t t b X t W W b b X t W W t t


and, based on our finitely many data:

2

2( )( , ) ( , ) 1 2( )( , ) 1 .

∆ ∆′= + + −

& j jj j j j j j j

j j

W WX a X t b X t b b X t

h h

• step length 1 :j j j jh t t t+= − = ∆

1

1

, if 1,2,..., 1

:

, if

j

j j

j

N N

N

X Xj N

hX

X Xj N

h

+

−

−= −

=− =

&


• (independent),

•

∆ jW Var( )∆ = ∆j jW t

( )21( , ) ( , ) ( )( , ) 1

2′= + + −& j

j j j j j j j j

j

ZX a X t b X t b b X t Z

h

(0, ),tW N t

, (0,1)∆ = ∆j j j jW Z t Z N

• More simple form:

where

( ): ( , ) , : ( , ),= =j j j j j jG a X t H b X t

( ) ,j j j j j j jX G H c H H d′= + +&


• Our problem:

is a vector which comprises a subset of all the parameters.

( )2: , : 1 2 1 .= = −j j j j jc Z h d Z

y

( )2

21

min ( ( ) )=

′− + +∑ &N

j j j j j j jy

j

X G H c H H d

2 2

0 , 0 ,1 1 1

2 2

0 , 0 ,1 1 1

2 2

0 , 0 ,1 1 1

( , ) ( ) ( )

( , ) ( ) ( )

( , ) ( ) ( )

gp

hr

fs

dl l

j j j p j p p p j pp p l

dm m

j j j j j r j r r r j rr r m

dn n

j j j j j s j s s s j ss s n

G a X t f U B U

H c b X t c g U C U

F d b b X t d h U D U

α α α

β β β

ϕ ϕ ϕ

= = =

= = =

= = =

= = + = +

= = + = +

′= = + = +

∑ ∑∑

∑ ∑∑

∑ ∑∑


where

• k th order base spline : a polynomial of degree k − 1, with knots, say

( ) ( ),1 ,2, : , ;j j j j jU U U X t= =

,kBη ,xη

1,1

, , 1 1, 11 1

1,( )

0, otherwise

( ) ( ) ( )kk k k

k k

x x xB x

x x x xB x B x B x

x x x x

η ηη

η ηη η η

η η η η

+

+− + −

+ − + +

≤ <=

− −= +

− −

• penalized sum of squares PRRS

( ){ }[ ] [ ]

22 2

1 1

2 22 2

1 1

( , , ) : ( )

( ) ( )

N

j j j j j j p p p pj p

r r r r s s s sr s

PRSS f g h X G H c F d f U dU

g U dU h U dU

θ λ

µ ϕ

= =

= =

′′ , = − + + +

′′ ′′+ +

∑ ∑ ∫

∑ ∑∫ ∫

&


• (smoothing parameters ),

• large values of yield smoother curves,smaller ones allow more fluctuation

( ){ }2

1

22 2 2

0 , 0 , 0 ,1 1 1 1 1 1 1

( ) ( ) ( )

h fgp sr

N

j j j j j jj

d ddNl l m m n n

j p p j p r r j r s s j sj p l r m s n

X G H c F d

X B U C U D Uα α β β ϕ ϕ

=

= = = = = = =

− + + =

− + + + + +

∑

∑ ∑∑ ∑∑ ∑∑

&

&

, , 0p r sλ µ ϕ ≥

, ,p r sλ µ ϕ

( , , )κ

κ

κ= =∫ ∫b

a

p r s

( ) ( ) ( )( ) ( )( ) ( )

1 20 1 2

1 20 1 2

1 20 1 2

, , , , , , , ,..., ( 1,2),

, , , , ,..., ( 1,2),

, , , , ,..., ( 1,2).

gp

hr

fs

TT T dT T T T Tp p p p

TT dT Tr r r r

TT dT Ts s s s

p

r

s

θ α β ϕ α α α α α α α α

β β β β β β β β

ϕ ϕ ϕ ϕ ϕ ϕ ϕ ϕ

= = = =

= = =

= = =

{ } 22N ( )T


• Then,

• Furthermore,

{ } 22

21

.N

j jj

X A X Aθ θ=

− = −∑ & & ( )( )

1 2

1 2

, ,...,

, ,...,

TT T TN

T

N

A A A A

X X X X

=

=& & & &

12 2

1,1

21

1 1

( ) ( ) ( )

( ) .

gp

b N

p p p p jp j p jpja

dNl lp p jP j

j l

f U dU f U U U

B U uα

−

+=

−

= =

′′ ′′≅ −

′′=

∑∫

∑ ∑

12 2 2

21

( ) ( 1,2)b N

p Bp p p j j p P p

ja

f U dU B u A pα α−

=

′′ ′′≅ = = ∑∫

( )1 1 2 2 1 1

1, ,

: , ,...,

: ( 1,2,..., 1).

TB p T p T p Tp N N

j j p j p

A B u B u B u

u U U j N

− −

+

′′ ′′ ′′=

= − = −

[ ]1 2 22

( ) ( 1,2)b N

r Cg U dU C v A rβ β− ′′′′ ≅ = =∑∫

Appendix Stochastic Differential Equations

[ ]2

21

( ) ( 1,2)r Cr r r j j r r r

ja

g U dU C v A rβ β=

′′′′ ≅ = = ∑∫

( )1 1 2 2 1 1

1, ,

: , ,...,

: ( 1,2,..., 1).

TC r T r T r Tr N N

j j r j r

A C v C v C v

v U U j N

− −

+

′′ ′′ ′′=

= − = −

12 2 2

21

( ) ( 1,2)b N

s Ds s s j j s s s

ja

h U dU D w A sϕ ϕ−

=

′′ ′′≅ = = ∑∫

( )1 1 2 2 1 1

1, ,

: , ,...,

: ( 1,2,..., 1).

TD s T s T s Ts N N

j j s j s

A D w D w D w

w U U j N

− −

+

′′ ′′ ′′=

= − = −

• Let us assume that

2 2 22 2 2 2

2 2 22 1 1 1

( , , )θ θ λ α µ β ϕ ϕ= = =

, = − + + +∑ ∑ ∑& B C Dp p p r r r s s s

p r s

PRSS f g h X A A A A

2: :λ µµ ϕ δ= = = =p r s

2 22( , , ) ,θ θ δ θ, ≈ − +&PRSS f g h X A L


where is a matrix:

22

22( , , ) ,θ θ δ θ, ≈ − +&PRSS f g h X A L

( ), , .θ α β ϕ=TT T T

1

2

1

2

1

2

0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0: ,

0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0

=

B

B

C

C

D

D

A

A

AL

A

A

A

L 6( 1)N m− ×

2 2

22min

θθ θ− +&X A Lµ

Tikhonov regularization


,

2

2

subject to

min ,

,

θ

θ −

θ

≤

≤

&

tt

A X t

L M

Conic quadratic programming


,

6( 1)6( 1)

1 6( 1) 1

min

0subject to : ,

1 0 0

00: ,

0 0

,

θ

χθ

ηθ

χ η

−−

+ − +

−= +

= +

∈ ∈

&

t

NTm

NNTm

N N

t

tA X

t

M

L L

L

primal problem

,χ η∈ ∈L L

{ }1 1 2 2 21 2 1 1 2: ( , ,..., ) | ...+ += = ∈ ≥ + + +N T N

N N+ NL x x x x x x x xR

( )

6( 1) 1

1 6( 1) 2

6( 1)1 2

11 2

max ( ,0) 0 ,

10 1 0 0subject to ,

00 0

, − +

−

−

+

+ −

+ =

∈ ∈

&

N

T TN

T TN N

T Tmm m

N

X M

A

L L

L

κ κκ κκ κκ κ



dual problem

is a primal dual optimal solution if and only if 1 2( , , , , , )t θ χ η κ κ

0: ,

1 0 0

00

NTm

tA X

t

χθ

−= +

&

L


6( 1)6( 1)

6( 1)1 2

1 2

1 6( 1) 11 2

1 6( 1) 1

00:

0 0

10 1 0 0

00 0

0, 0

,

, .

NNTm

T TN NT T

mm m

T T

N N

N N

t

M

A

L L

L L

ηθ

χ η

χ η

−−

−

+ − +

+ − +

= +

+ =

= =

∈ ∈

∈ ∈

L

Lκ κκ κκ κκ κ




Ex.:

( ) ( ), , , ,µ σ= +t t t t t tdX t X Z dt t X Z dW .

( )( ) + ,θ − θ = − + T T

t t t t t t t t tdV V dt cr dr t V dWµ σ

( ) ,σ= ⋅ − + ⋅ ⋅t t t t td R r dt rr dWτα

non linear regression

( ) ( ), , , ,µ σ= +t t t t t tdX t X Z dt t X Z dW .

( ) ( )

( )

2

,

1

2

1

min

:

β β

β

=

=

= −

=

∑

∑

N

j jj

N

jj

f d g x

f

Nonlinear Regression

min ( ) ( ) ( )β β β=

Tf F F

( )1( ) : ( ),..., ( )β β β= T

NF f f

• Gauss-Newton method :

( ) ( ) ( ) ( )β β β β∇ ∇ = −∇T qF F F F

1 :β β+ = +k k kq


• Levenberg-Marquardt method :

( )( ) ( ) I ( ) ( )β β λ β β∇ ∇ + = −∇Tp qF F F F

0λ ≥

( ) ( ),

min ,

subject to ( ) ( ) I ( ) ( ) , 0,β β λ β β∇ ∇ + − −∇ ≤ ≥

t

T

qt

F F F Fq t t

alternative solution


( ) ( )2

2

subject to ( ) ( ) I ( ) ( ) , 0,

|| ||

β β λ β β∇ ∇ + − −∇ ≤ ≥

≤

TpF F F F

qL

q t t

M

conic quadratic programming

( ) ( ),

min ,


t

T

qt

F F F Fq t t



( ) ( )2

2

subject to ( ) ( ) I ( ) ( ) , 0,

|| ||

β β λ β β∇ ∇ + − −∇ ≤ ≥

≤

TpF F F F

qL

q t t

M

interior point methods

conic quadratic programming

( ) ( ),

min ,


t

T

qt

F F F Fq t t



( ) ( )2

2

subject to ( ) ( ) I ( ) ( ) , 0,

|| ||

β β λ β β∇ ∇ + − −∇ ≤ ≥

≤

TpF F F F

qL

q t t

M

2

1min ( ) := ( ) + ( ) ( ) + ( ) ( )

2

subject to

β β β β β ∇ ∇ ∇ ≤ ∆

T T T

q

Q q f q F F q F F q

q

trust regiontrust region

max utility ! or

min costs !

martingale method:

Portfolio Optimization

Optimization ProblemOptimization Problem

Representation ProblemRepresentation Problem

or or stochastic control

max utility ! or

min costs !

martingale method:

Parameter Estimation





max utility ! or

min costs !

martingale method:






max utility ! or

min costs !

martingale method:






Aster, A., Borchers, B., and Thurber, C., Parameter Estimation and Inverse Problems, Academic Press, 2004.

Boyd, S., and Vandenberghe, L., Convex Optimization, Cambridge University Press, 2004.

Buja, A., Hastie, T., and Tibshirani, R., Linear smoothers and additive models, The Ann. Stat. 17, 2 (1989) 453-510.

Fox, J., Nonparametric regression, Appendix to an R and S-Plus Companion to Applied Regression, Sage Publications, 2002.

Friedman, J.H., Multivariate adaptive regression splines, Annals of Statistics 19, 1 (1991) 1-141.

Friedman, J.H., and Stuetzle, W., Projection pursuit regression, J. Amer. Statist Assoc. 76 (1981) 817-823.

References

Hastie, T., and Tibshirani, R., Generalized additive models, Statist. Science 1, 3 (1986) 297-310.

Hastie, T., and Tibshirani, R., Generalized additive models: some applications, J. Amer. Statist. Assoc.82, 398 (1987) 371-386.

Hastie, T., Tibshirani, R., and Friedman, J.H., The Element of Statistical Learning, Springer, 2001.

Hastie, T.J., and Tibshirani, R.J., Generalized Additive Models, New York, Chapman and Hall, 1990.

Kloeden, P.E, Platen, E., and Schurz, H., Numerical Solution of SDE Through Computer Experiments, Springer Verlag, New York, 1994.

Korn, R., and Korn, E., Options Pricing and Portfolio Optimization: Modern Methods of Financial Mathematics, Oxford University Press, 2001.

Nash, G., and Sofer, A., Linear and Nonlinear Programming, McGraw-Hill, New York, 1996.

Nemirovski, A., Lectures on modern convex optimization, Israel Institute of Technology (2002).

Nemirovski, A., Modern Convex Optimization, lecture notes, Israel Institute of Technology (2005).

Nesterov, Y.E , and Nemirovskii, A.S., Interior Point Methods in Convex Programming, SIAM, 1993.

Önalan, Ö., Martingale measures for NIG Lévy processes with applications to mathematical finance,presentation in: Advanced Mathematical Methods for Finance, Side, Antalya, Turkey, April 26-29, 2006.

Taylan, P., Weber G.-W., and Kropat, E., Approximation of stochastic differential equations by additive models using splines and conic programming, International Journal of Computing Anticipatory Systems 21(2008) 341-352.

Taylan, P., Weber, G.-W., and A. Beck, New approaches to regression by generalized additive modelsand continuous optimization for modern applications in finance, science and techology, in the special issue in honour of Prof. Dr. Alexander Rubinov, of Optimization 56, 5-6 (2007) 1-24.

References

in honour of Prof. Dr. Alexander Rubinov, of Optimization 56, 5-6 (2007) 1-24.

Taylan, P., Weber, G.-W., and Yerlikaya, F., A new approach to multivariate adaptive regression splineby using Tikhonov regularization and continuous optimization, to appear in TOP, Selected Papers at theOccasion of 20th EURO Mini Conference (Neringa, Lithuania, May 20-23, 2008) 317- 322.

Seydel, R., Tools for Computational Finance, Springer, Universitext, 2004.

Stone, C.J., Additive regression and other nonparametric models, Annals of Statistics 13, 2 (1985) 689-705.

Weber, G.-W., Taylan, P., Akteke-Öztürk, B., and Uğur, Ö., Mathematical and data mining contributionsdynamics and optimization of gene-environment networks, in the special issue Organization in Matterfrom Quarks to Proteins of Electronic Journal of Theoretical Physics.

Weber, G.-W., Taylan, P., Yıldırak, K., and Görgülü, Z.K., Financial regression and organization, to appear in the Special Issue on Optimization in Finance, of DCDIS-B (Dynamics of Continuous, Discrete andImpulsive Systems (Series B)).

Education

Prediction of Financial Processes