Isotonic Regression in Sobolev Spacesartax.karlin.mff.cuni.cz/~macim1am/pub/teach/michal.pdf · 2006. 11. 14. · Isotonia Isotonic Regression in Sobolev Spaces Michal Pešta Department

IntroductionAbstract and Motivation

Sobolev SpacesIsotonia

Isotonic Regression in Sobolev Spaces

Michal Pešta

Department of Probability and Mathematical Statistics

Supervisor:Mgr. Zdeněk Hlávka, Ph.D.

Robust 2006

Michal Pešta Isotonic Regression in Sobolev Spaces

DAX Calloptions – Multiple Observations

6000 6500 7000 7500

010

020

030

040

050

060

070

0Calloptions with Point−Wise Confidence Intervals

Strike price

Opt

ion

pric

e

Regression CurveAsymtotic CIBootstrap CIWild Bootstrap CI

Figure: DAX Calloptions Data – Monotonic (Decreasing) and Convex Regression Curve in SobolevSpace of rank m = 4 with various types of 95% Confidence Intervals.

we would like to control the value of first m derivatives at observation points

Aim of Paper and Main Steps of Procedure

Searching for Regression Function with Specific Constraints

nonparametric estimation takes place over balls of functions which are elementsof suitable Sobolev space (sufficiently smooth sets of functions)

the Sobolev spaces are special types of Hilbert spaces ⇒ facilitate calculation ofleast square projection ⇒ decomposition into mutually orthogonal complementstransform the problem of searching for the best fitting function in an infinitedimensional space into a finite dimensional optimization problem

isotonic regression ≡ imposition of additional constraint —isotonia— onnonparametric regression estimation, e.g. monotonicity, convexity or concavity

confidence intervals based on asymptotic behaviour and bootstrap techniques

tests of monotonicity, concavity, convexity, . . .

Least SquaresEstimators

Infinite to Finite

Theorem (Infinite to Finite)

Let y = (y1, . . . , yn)′ and define

σ̂2 = minf∈Hm

1n

nXi=1

[yi − f (xi )]2 s.t. ‖f‖2Sob,m ≤ L, (1)

s2 = minc∈Rn

1n

[y −Ψc]′ [y −Ψc] s.t. c′Ψc ≤ L (2)

where c is a n × 1 vector and Ψ is the representor matrix. Then σ̂2 = s2. Furthermore,there exists a solution of optimizing problem of the form

bf = nXi=1

bciψxi (3)where bc = ( bc1, . . . , bcn)′ solves σ̂2. The estimator bf is unique a.s.

Construction of Regression Estimator

observations X1, . . . ,Xnestimator

bf (x) = nXj=1

bcj 2mXk=1

exph<

`eiθk

´x

i

I[x≤Xj ]γk cosh=

`eiθk

´x

i+ I[x>Xj ]γ2m+k sin

h=

`eiθk

´x

i ff(4)

bf is NOT estimated using goniometric splines NEITHER kernel functionsquadratic optimizing with constraints provides bcjRiesz Representation Theorem gives us θkboundary conditions of specific differential equation provides γk

Construction of Regression Estimator

observations X1, . . . ,Xnestimator

bf (x) = nXj=1

bcj 2mXk=1

exph<

`eiθk

´x

i

I[x≤Xj ]γk cosh=

`eiθk

´x

i+ I[x>Xj ]γ2m+k sin

h=

`eiθk

´x

i ff(4)

bf is NOT estimated using goniometric splines NEITHER kernel functionsquadratic optimizing with constraints provides bcjRiesz Representation Theorem gives us θkboundary conditions of specific differential equation provides γk

Choosing of Smoothing Parameter

Cross–Validation

0.2 0.4 0.6 0.8

0.3

0.4

0.5

0.6

0.7

0.8

0.9

Monotonic Regression in Sobolev Space

Independent X data

Dep

ende

nt Y

dat

a

−4.5 −4.0 −3.5 −3.0 −2.5 −2.0

0.04

00.

045

0.05

00.

055

Cross−Validation for Smoothing Parameter

log(χ)C

V(χ

)

Figure: Monotonic Regression Curve in Sobolev Space of rank m = 2 for the best value ofSmoothing Parameter χ (left) according to Cross–Validation function CV (right).

Cross–Validation gives us optimal smoothing parameter

IntroductionAbstract and MotivationAim of Paper and Main Steps of Procedure

Sobolev SpacesLeast SquaresEstimators

IsotoniaChoosing of Smoothing Parameter