Native Hilbert Spaces for Radial Basis Functions I

Robert Schaback

Abstract

This contribution gives a partial survey over the native spaces associatedto (not necessarily radial) basis functions. Starting from reproducing kernelHilbert spaces and invariance properties, the general construction of nativespaces is carried out for both the unconditionally and the conditionally posi-tive definite case. The definitions of the latter are based on finitely supportedfunctionals only. Fourier or other transforms are not required. The depen-dence of native spaces on the domain is studied, and criteria for functionsand functionals to be in the native space are given. Basic facts on optimalrecovery, power functions, and error bounds are included.

1 Introduction

For the numerical treatment of functions of many variables, radial basis functionsare useful tools. They have the form φ(‖x − y‖2) for vectors x, y ∈ IRd with aunivariate function φ defined on [0,∞) and the Euclidean norm ‖ · ‖2 on IRd.This allows to work efficiently for large dimensions d, because the function boilsthe multivariate setting down to a univariate setting. Usually, the multivariatecontext comes back into play by picking a large number M of points x1, . . . , xM

in IRd and working with linear combinations

s(x) :=

M∑

j=1

λjφ(‖xj − x‖2).

1

2 R. Schaback

In certain cases, low–degree polynomials have to be added, but we give detailslater. Typical examples for radial functions φ(r) on r = ‖x− y‖2, x, y ∈ IRd are

thin–plate splines: rβ log r, β > 0, β ∈ 2 IN [1]

rβ , β > 0, β /∈ 2 IN [1]

multiquadrics: (r2 + c2)β/2, β > 0, β /∈ 2 IN [6]

inverse multiquadrics: (r2 + c2)β/2, β < 0, [6]

Gaussians: exp(−βr2), β > 0,

Sobolev splines: rk−d/2Kk−d/2(r), k > d/2

Wendland function: (1 − r)4+(1 + 4r), d ≤ 3

Another important case are zonal functions on the (d − 1)–dimensional sphereSd−1 ⊂ IRd. These have the form φ(xT y) = φ(cos(α(x, y))) for points x, y onthe sphere spanning an angle of α(x, y) ∈ [0, π] at the origin. Here, the symbol T

denotes vector transposition, and the function φ should be defined on [−1, 1]. Peri-odic multivariate functions can also be treated, e.g. by reducing them to productsof univariate periodic functions.

All of these cases of basis functions share a common theoretical foundationwhich forms the main topic of this paper. The functions all have a unique associated“native” Hilbert space of functions in which they act as a generalized reproducingkernel. The different special cases (radiality, zonality) are naturally related togeometric invariants of the native spaces. The paper will thus start in section 2with reproducing kernel Hilbert spaces and look at geometric invariants later insection 3.

But most basis functions are constructed directly and do not easily provideinformation on their underlying native space. Their main properties are symme-try and (strict) positive definiteness (SPD) or conditionally positive definiteness(CPD). These notions are defined without any relation to a Hilbert space, and wethen have to show how to construct the native space, prove its uniqueness, andfind its basic features. We do this for SPD functions in section 4 and for CPDfunctions in section 5. The results mostly date back to classical work on repro-ducing kernel Hilbert spaces and positive definite functions (see e.g. [12], [17]).We compile the necessary material here to provide easy access for researchers andstudents. Some new results are included, and open problems are pointed out. Inparticular, we show how to modify the given basis function in order to go overfrom the conditionally positive definite case to the (strictly) positive definite case.There are different ways to define native spaces (see [10] for comparisons), buthere we want to provide a technique that is general enough to unify different con-structions (e.g. on the sphere [3] or on Riemannian manifolds [2],[13]). We finish

Native Spaces 3

with a short account of optimal recovery of functions in native spaces from givendata, and provide the corresponding error bounds based on power functions.

The notation will strictly distinguish between functions f, g, . . . and function-als λ, µ, . . . as real–valued linear maps defined on functions. Spaces of functionswill be denoted by uppercase letters like F,G, . . ., and calligraphic letters F ,G, . . .occur as soon as the spaces are complete. Spaces with an asterisk are dual spaces,while an asterisk at lowercase symbols indicates optimized quantities.

2 Reproducing Kernel Hilbert Spaces

Let Ω ⊆ IRd be a quite general set on which we consider real–valued functionsforming a real Hilbert space H with inner product (., .)H. Assume further that forall x ∈ Ω the point evaluation functional δx : f → f(x) is continuous in H, i.e.

δx ∈ H∗ for all x ∈ Ω (2.1)

with the dual of H denoted by H∗. This is a reasonable assumption if we want toapply numerical methods using function values. Note, however, that techniques likethe Rayleigh–Ritz method for finite elements work in Hilbert spaces where pointevaluaton functionals are not continuous. We shall deal with this more generalsituation later.

If (2.1) is satisfied, the Riesz representation theorem implies

Theorem 2.1 If a Hilbert space of functions on Ω allows continuous point eval-uation functionals, it has a symmetric reproducing kernel Φ : Ω × Ω → IR withthe properties

Φ(x, ·) ∈ H

f(x) = (f,Φ(x, ·))HΦ(x, y) = (Φ(x, ·),Φ(y, ·))H = Φ(y, x)

Φ(x, y) = (δx, δy)H∗

(2.2)

for all x, y ∈ Ω, f ∈ H.

The theory of reproducing kernel Hilbert spaces is well covered in [12], forinstance. In the terminology following below, a reproducing kernel Hilbert spaceis the native space with respect to its reproducing kernel. This is trivial as longas we start with a Hilbert space, but it is not trivial if we start with a functionΦ : Ω× Ω → IR.

3 Invariance Properties

In many cases, the domain Ω of functions allows a group TT of geometric transfor-mations, and the Hilbert space H of functions on Ω is invariant under this group.

4 R. Schaback

This meansf T ∈ H

(f T, g T )H = (f, g)H(3.1)

for all f, g ∈ H, T ∈ TT. The following simple result has important implicationsfor the basis functions on various domains:

Theorem 3.1 If a Hilbert space H of functions on a domain Ω is invariant undera group TT of transformations on Ω in the sense of (3.1), and if H has a reproducingkernel Φ, then Φ is invariant under TT in the sense

Φ(x, y) = Φ(Tx, T y) for all x, y ∈ Ω, T ∈ TT .

Proof. The assertion easily follows from

f(x) = (f,Φ(x, ·))H= (f T−1)(Tx) = (f T−1,Φ(Tx, ·))H

= (f T−1 T,Φ(Tx, T ·))H= (f,Φ(Tx, T ·))H

for all x ∈ Ω, T ∈ TT, f ∈ H.

By some easy additional arguments one can read off the following invarianceproperties inherited by reproducing kernels Φ from their Hilbert spaces H on Ω:

• Invariance on Ω = IRd under translations from IRd leads to translation–

invariant functions Φ(x, y) = φ(x − y) with φ(x) = φ(−x) : IRd → IR.

• In case of additional invariance under all orthogonal transformations we getradial functions Φ(x, y) = φ(‖x − y‖2) with φ : [0,∞) → IR. Thus radialbasis functions arise naturally in all Hilbert spaces on IRd which are invariantunder Euclidean rigid–body motions.

• Invariance on the sphere Sd−1 under all orthogonal transformations leads tozonal functions Φ(x, y) = φ(xT y) for φ : [−1, 1] → IR.

• Spaces of periodic functions induce periodic reproducing kernels.

See [5] for basis functions on topological groups, and see [3] for a review of resultson the sphere. The paper [13] introduces the theory of basis functions on generalmanifolds, and corresponding error bounds are in [2].

Native Spaces 5

4 Native Spaces of Positive Definite Functions

Instead of a single point x ∈ Ω with a single evaluation functional δx ∈ H∗ wenow consider a set x1, . . . , xM of M distinct points in Ω and look at the pointevaluation functionals δx1

, . . . , δxM.

Theorem 4.1 In a real vector space H of functions on some domain Ω the fol-lowing properties concerning a set X = x1, . . . , xM of M distinct points areequivalent:

1. There are functions f ∈ H which attain arbitrary values at the points xj ∈x1, . . . , xM.

2. The points in X can be separated, i.e.: for all xj ∈ X there is a functionfj ∈ H vanishing on X except for xj .

3. The point evaluation functionals δx1, . . . , δxM

∈ H∗ are linearly independent.

Now let H be a reproducing kernel Hilbert space of real–valued functions onΩ. Furthermore, let one of the properties in Theorem 4.1 be satisfied for all finitesets X = x1, . . . , xM ⊆ Ω. Then the matrix

AΦ,X = (Φ(xk, xj))1≤j,k≤M =(

(δxj, δxk

)H∗

)

1≤j,k≤M(4.1)

is a Gramian matrix formed of linearly independent elements. Thus it is symmetricand positive definite.

But the above property can be reformulated independent of the Hilbert spacesetting:

Definition 4.2 A function Φ : Ω × Ω → IR is symmetric and (strictly)

positive definite (SPD), if for arbitrary finite sets X = x1, . . . , xM ⊆ Ω ofdistinct points the matrix AΦ,X = (Φ(xk, xj))1≤j,k≤M is symmetric and positivedefinite.

Definition 4.3 If a symmetric (strictly) positive definite function Φ : Ω×Ω → IRis the reproducing kernel of a real Hilbert space H of real–valued functions on Ω,then H is the native space for Φ.

We can collect the above arguments into

Theorem 4.4 If a real Hilbert space H of real–valued functions on some domainΩ allows continuous point evaluation functionals which are linearly independentwhen based on distinct points, the space has a symmetric and (strictly) positivedefinite reproducing kernel Φ and is the native space for Φ.

Except for the unicity stated above, this is easy since we started from a givenHilbert space. Things are more difficult when we start with an SPD function Φand proceed to construct its native space. The basic idea for this will first appearin the uniqueness proof for the native space:

6 R. Schaback

Theorem 4.5 The native space for a given SPD function Φ is unique if it exists,and it then coincides with the closure of the space of finite linear combinations offunctions Φ(x, ·), x ∈ Ω under the inner product defined via

(Φ(x, ·),Φ(y, ·))H = Φ(x, y) for all x, y ∈ Ω. (4.2)

Proof. Let H be a Hilbert space of functions on Ω which has Φ as a symmetricpositive definite kernel. Clearly all finite linear combinations of functions Φ(x, ·)are in H, and the inner product on these functions depends on Φ alone because of(2.2). We can thus use (4.2) as a redefinition for the inner product on the subspaceof linear combinations of functions Φ(x, ·). If H were larger than the closure of thespan of these functions, there would be a nonzero f ∈ H which is orthogonal toall Φ(x, ·). But then f(x) = (f,Φ(x, ·))H = 0 for all x ∈ Ω.

Theorem 4.5 shows that the native space is the closure of the functions we workwith in applications, i.e.: the functions Φ(x, ·) for x ∈ Ω fixed. Everything that canbe approximated by functions Φ(x, ·) is in the native space. But the above resultleaves us to show existence of the native space for any SPD function Φ. To do this,we mimic (4.2) to define an inner product

(Φ(x, ·),Φ(y, ·))Φ = Φ(x, y) for all x, y ∈ Ω. (4.3)

on functions Φ(x, ·). This inner product depends on Φ alone, and we thus use aslightly different notation. It clearly extends to an inner product on the space

FΦ(Ω) :=

M∑

j=1

λjΦ(xj , ·) λj ∈ IR, M ∈ IN, xj ∈ Ω

(4.4)

of all finite linear combinations of such functions, because Φ is an SPD function.We keep Φ and Ω in the notation, because we want to study later how native spacesdepend on Φ and Ω. The abstract Hilbert space completion FΦ(Ω) of this spacethen is a Hilbert space with an inner product that we denote by (·, ·)Φ again, butwe still have to interpret the abstract elements of FΦ(Ω) as functions on Ω. Butthis is no problem since the point evaluation functionals δx extend continuouslyto the completion, and the equation

δx(f) = (f,Φ(x, ·))Φ for all x ∈ Ω, f ∈ FΦ(Ω) (4.5)

makes sense there. We just define f(x) to be the right–hand side of (4.5). Alto-gether we have

Theorem 4.6 Any SPD function Φ on some domain Ω has a unique native space.It is the closure of the space FΦ(Ω) of (4.4) under the inner product (4.3). Theelements of the native space can be interpreted as functions via (4.5).

Native Spaces 7

There are various other techniques to define the native space. See [10] for acomparison and embedding theorems.

It is one of the most challenging research topics to deduce properties of thenative space from properties of Φ. For instance, Corollary 8.3 below will showthat continuity of Φ on Ω × Ω implies that all functions in the native space arecontinuous on Ω. Other interesting topics are embedding theorems and densityresults for native spaces. See [10] for a starting point.

5 Native Spaces of Conditionally Positive Defi-

nite Functions

Several interesting functions of the form Φ(x, y) given in section 1, e.g.: Duchon’sthin–plate splines or Hardy’s multiquadrics are well–defined and symmetric onΩ = IRd but not positive definite there. The quadratic form defined by the matrixin (4.1) is only positive definite on a certain subspace of IRM . For later use, wemake the corresponding precise definition somewhat more technical than necessaryat first sight.

Let P be a finite–dimensional subspace of real–valued functions on Ω. In ap-plications on Ω ⊆ IRd we shall usually consider P = IPd

m, the space of polynomialsof order at most m, while in periodic cases we use trigonometric polynomials,or spherical harmonics on the sphere. Then LP(Ω) denotes the space of all linearfunctionals with finite support in Ω that vanish on P . For convenience, we describesuch functionals by the notation

λX,M : f →M∑

j=1

λjf(xj), λX,M (P) = 0 (5.1)

for finite sets X = x1, . . . , xM ⊆ Ω and coefficients λ ∈ IRM . Note that thesefunctionals form a vector space over IR under the usual operations.

Definition 5.1 A function Φ : Ω× Ω → IR is symmetric and conditionally

positive definite (CPD) with respect to P, if for all λX,M ∈ LP(Ω) \ 0 thevalue of the quadratic form

M∑

j,k=1

λjλkΦ(xj , xk) = λxX,Mλy

X,MΦ(x, y)

is positive.

Here, the superscript x denotes application of the functional with respect to thevariable x. The classical definition of conditional positive definiteness of some orderm is related to the special case P = IPd

m on Ω ⊆ IRd.

8 R. Schaback

From now on we assume Φ : Ω × Ω → IR to be CPD with respect to afinite–dimensional space P of functions on Ω. Because Φ is conditionally positivedefinite, we can define an inner product

(λX,M , µY,N )Φ :=M∑

j=1

N∑

k=1

λjµkΦ(xj , yk) = λxX,Mµy

Y,NΦ(x, y) (5.2)

on the space LP(Ω). Furthermore, we can complete LP(Ω) to a Hilbert spaceLΦ,P(Ω), and we denote the extended inner product on LΦ,P (Ω) by (·, ·)Φ again.Note that LP(Ω) as a vector space does not depend on Φ, but the completionLΦ,P(Ω) does, because Φ enters into the inner product. Now we can form innerproducts (λ, µ)Φ for all abstract elements λ, µ of the space LΦ,P(Ω), but we stillhave no functions on Ω, because we cannot evaluate (λ, δx)Φ since δx is in generalnot in LΦ,P (Ω).

A simple and direct, but not ultimately general workaround for this problemuses brute force to construct for all x ∈ Ω a substitute δ(x) ∈ LP(Ω) for a pointevaluation functional. We start with the assumption of existence of a fixed setΞ = ξ1, . . . , ξq ⊆ Ω of q = dimP points of Ω which is unisolvent for P . Thismeans that any function p ∈ P can be uniquely reconstructed from its values on Ξ.This is no serious restriction to any application. In case of classical multiquadricsor thin–plate splines in two dimensions it suffices to fix three points in Ω whichare not on a line. By picking a Lagrange–type basis p1, . . . , pq of P one can writethe reconstruction as

p(x) =

q∑

j=1

pj(x)p(ξj), for all p ∈ P , x ∈ Ω. (5.3)

This defines for all x ∈ Ω a very useful variation

δ(x)(f) := f(x)−q∑

j=1

pj(x)f(ξj) =

δx −q∑

j=1

pj(x)δξj

(f) (5.4)

of the standard point evaluation functional at x defined on all functions f on Ω.This functional annihilates functions from P and lies in LP(Ω) because it is finitelysupported. Furthermore, we have

δ(ξj) = 0 for the points ξj ∈ Ξ. (5.5)

We now can go on with our previous argument, because we can look at thefunction

RΦ,Ω(λ)(x) := (λ, δ(x))Φ, x ∈ Ω (5.6)

which is well–defined for all abstract elements λ ∈ LΦ,P(Ω). It defines a map RΦ,Ω

from the abstract space LΦ,P (Ω) into some space of functions on Ω. We have chosen

Native Spaces 9

the notation RΦ,Ω because the mapping will later be continuously extended to theRiesz map on the dual of the native space. Let us look at the special situation forλ = λX,M ∈ LP(Ω). Then

RΦ,Ω(λX,M )(x) = (λX,M , δ(x))Φ

=

M∑

j=1

λj

(

Φ(x, xj)−q∑

k=1

pk(x)Φ(ξk , xj)

)

= λyX,MΦ(x, y)−

q∑

k=1

pk(x)λyX,MΦ(ξk, y)

(5.7)

shows the fundamental relation

µY,NRΦ,Ω(λX,M ) = (λX,M , µY,N)Φ (5.8)

for all λX,M , µY,N ∈ LP(Ω). It shows immediately that RΦ,Ω is injective on LP(Ω).We want to generalize this identity to hold on the completion LΦ,P(Ω), but forthis we have to define a norm or inner product on the range of RΦ,Ω.

This is easy, since RΦ,Ω is injective on LP(Ω). We can define an inner producton the range by

(RΦ,Ω(λ), RΦ,Ω(µ))Φ := (λ, µ)Φ for all λ, µ ∈ LΦ,P (Ω),

where we extended the definition already to the completion LΦ,P(Ω) and used thesame notation again, because we will never mix up functions with functionals here.

The space FΦ,P(Ω) := RΦ,Ω(LP(Ω)) for a CPD function Φ with respect toa space P of functions on Ω will be the major part of the native space to beconstructed. It is a Hilbert space by definition, and we have

RΦ,Ω : LΦ,P(Ω) := LP(Ω) → FΦ,P(Ω) (5.9)

as the Riesz mapping and can generalize (5.8) to

µ(RΦ,Ω(λ)) = (RΦ,Ω(µ), RΦ,Ω(λ))Φ = (µ, λ)Φ for all λ, µ ∈ LΦ,P (Ω) (5.10)

by going continuously to the completions. We thus have an interpretation of theHilbert space FΦ,P(Ω) as a space of functions and the Hilbert space LΦ,P (Ω) asa space of functionals on FΦ,P(Ω). Furthermore, RΦ,Ω is the Riesz map and thespaces form a dual pair.

But we still do not have a reproduction property like (2.2), and the spaceFΦ,P(Ω) has the additional and quite superficial property that all its functionsvanish on Ξ due to (5.5) and (5.6). But the latter property shows that P andFΦ,P(Ω) form a direct sum of spaces.

Definition 5.2 The native space NΦ,P(Ω) for a conditionally positive definitefunction Φ on some domain Ω with respect to a finite–dimensional function spaceP consists of the sum of P with the Hilbert space FΦ,P(Ω) from (5.9).

10 R. Schaback

Note that this coincides with Definition 4.3 for P = 0, if we take Theorem4.6 into account. To derive further properties of the native space, including ageneralized notion of the reproduction equation (4.5), we use (5.3) to define aprojector

ΠP(f)(x) :=

q∑

j=1

pj(x)f(ξj) for all f : Ω → IR, x ∈ Ω

onto P with the property that f − ΠP (f) always vanishes on Ξ. Thus Id − ΠP

projects functions in the native space onto the range of RΦ,Ω, i.e. on FΦ,P(Ω).For all functions f ∈ NΦ,P (Ω) there is some λf ∈ LΦ,P (Ω) such that we havef − ΠPf = RΦ,Ω(λf ), and the value of this function at some point x ∈ Ω is(λf , δ(x))Φ = (RΦ,Ω(λf ), RΦ,Ω(δ(x)))Φ by definition. This implies

Theorem 5.3 Every function f in the native space of a conditionally positivedefinite function Φ on some domain Ω with respect to a finite–dimensional functionspace P has the representation

f(x) = (ΠPf)(x) + (f −ΠPf,RΦ,Ω(δ(x)))Φ for all x ∈ Ω (5.11)

which is a generalized Taylor–type reproduction formula.

Definition 5.4 The dual N ∗Φ,P(Ω) of the native space consists of all linear func-

tionals λ defined on the native space NΦ,P(Ω) such that the functional λ− λ ΠP

is continuous on the Hilbert subspace FΦ,P(Ω).

Note that the functionals λ ∈ N ∗Φ,P(Ω) are just linear forms on NΦ,P(Ω) in the

sense of linear algebra. They are not necessarily continuous on FΦ,P(Ω) with re-spect to the norm ‖ · ‖Φ, because in that case they would have to vanish on P .This would rule out point–evaluation functionals, for instance. Our special con-tinuity requirement avoids this pitfall and makes point–evaluation functionals δxwell–defined in the dual, but in general not continuous. However, the functionalδ(x) = δx − δx ΠP is continuous instead.

For all λ ∈ N ∗Φ,P (Ω) we can consider the function fλ := RΦ,Ω(λ− λ ΠP ) in

FΦ,P(Ω). By (5.6) and (5.2) it can be explicitly calculated via

fλ(x) = RΦ,Ω(λ− λ ΠP)(x)

= (λ− λ ΠP , δ(x))Φ

= (λ− λ ΠP)yδz(x)Φ(y, z)

(5.12)

By (5.8) for f = RΦ,Ω(µ), it represents the functional λ in the sense

(λ− λ ΠP )(f) = (f, fλ)Φ for all f ∈ FΦ,P(Ω). (5.13)

Altogether, the action of functionals can be described by

Native Spaces 11

Theorem 5.5 Each functional λ in the dual N ∗Φ,P(Ω) of the native space NΦ,P(Ω)

of a conditionally positive definite function Φ on some domain Ω with respect to afinite–dimensional function space P acts via

λ(f) = (λ ΠP )f + (f −ΠPf,RΦ,Ω(λ− λ ΠP ))Φ

= (λ ΠP )f + (f −ΠPf, λxRΦ,Ω(δ(x)))Φ

on all functions f ∈ NΦ,P (Ω).

Proof. The first formula follows easily from (5.12) and (5.13). For the second, weonly have to use (5.6), (5.5), and (5.8) to prove

λxRΦ,Ω(δ(x))(t) = λxRΦ,Ω(δ(t))(x)

= (λ− λ ΠP)xRΦ,Ω(δ(t))(x)

= (δ(t), RΦ,Ω(λ− λ ΠP))Φ

= RΦ,Ω(λ− λ ΠP)(t) = fλ(t)

(5.14)

for all t ∈ Ω.

Note how Theorem 5.5 generalizes Theorem 5.3 to arbitrary functionals from thedual of the native space. The second form is somewhat easier to apply, becausethe representer fλ of λ can be calculated via (5.14).

6 Modified kernels

The kernel function occurring in Theorem 5.3 is by definition

RΦ,Ω(δ(x))(y) = (δ(x), δ(y))Φ =: Ψ(x, y), (6.1)

and we call Ψ the reduction of Φ. It has the explicit representation

Ψ(x, y) = Φ(x, y)−q∑

j=1

pj(x)Φ(ξj , y)−q∑

k=1

pk(y)Φ(x, ξk)

+

q∑

j=1

q∑

k=1

pj(x)pk(y)Φ(ξj , ξk)

= (Id−ΠP )x(Id−ΠP)

yΦ(x, y)

(6.2)

for all x, y ∈ Ω because we can do the evaluation of (6.1) via (5.2) and (5.4).Furthermore, equation (6.1) implies

Ψ(x, y) = (RΦ,Ωδ(x), RΦ,Ωδ(y))Φ = (Ψ(x, ·),Ψ(y, ·))Φ

12 R. Schaback

as we would expect from (2.2). A consequence of (5.5) is

Ψ(ξj , ·) = Ψ(·, ξj) = 0. (6.3)

The bilinear form

(f, g)Ψ := (f −ΠPf, g −ΠPg)Φ for all f, g ∈ NΦ,P(Ω) (6.4)

is positive semidefinite on the native space NΦ,P(Ω) for Φ. Its nullspace is P , andthe reproduction property (5.11) takes the simplified form

f(x) = (ΠPf)(x) + (f,Ψ(x, ·))Ψ for all x ∈ Ω, (6.5)

where we used (6.3). The representer fλ of a functional λ ∈ N ∗Φ,P(Ω) in the sense

of (5.13) takes the simplified form

fλ(x) = λyΨ(x, y), x, y ∈ Ω

and Theorem 5.5 goes over into

Theorem 6.1 Each functional λ in the dual N ∗Φ,P(Ω) of the native space NΦ,P(Ω)

of a conditionally positive definite function Φ on some domain Ω with respect to afinite–dimensional function space P acts via

λ(f) = (λ ΠP)f + (f, λxΨ(x, ·))Ψon all functions f ∈ NΦ,P (Ω).

Note that the reduction is easy to calculate. It coincides with the originalfunction Φ if the latter is unconditionally positive definite. There also is a connec-tion to the preconditioning technique of Jetter and Stockler [7].

Theorem 6.2 The reduction Ψ of Φ with respect to Ξ is (strictly) positive definiteon Ω \ Ξ.

Proof. Let λX,M be a functional with support X = x1, . . . , xM ⊆ Ω \ Ξ. Thenthe functional λX,M (Id−ΠP) is finitely supported on Ω and vanishes on P . Thuswe can use the conditional positive definiteness of Φ and get from (6.2) that

λxX,Mλy

X,MΨ(x, y) = (λX,M (Id−ΠP))x(λX,M (Id−ΠP))

yΦ(x, y)

is nonnegative and vanishes only if λX,M (Id−ΠP ) is the zero functional in LP(Ω).Its representation is

λX,M (Id−ΠP)(f) =

M∑

j=1

λj

(

f(xj)−q∑

k=1

pk(xj)f(ξk)

)

=

M∑

j=1

λjf(xj)−q∑

k=1

f(ξk)

M∑

j=1

λjpk(xj),

Native Spaces 13

and since the sets X = x1, . . . , xM and Ξ are disjoint, the coefficients mustvanish.

There is an easy possibility used in [9] to go over from here to a fully positivedefinite case. Using an early idea from Golomb and Weinberger [4] we form a newkernel function K : Ω× Ω → IR by

K(x, y) := Ψ(x, y) +

q∑

j=1

pj(x)pj(y) (6.6)

and a new inner product

(f, g)Φ :=

q∑

j=1

f(ξj)g(ξj) + (f −ΠPf, g −ΠPg)Φ (6.7)

on the whole native space NΦ,P (Ω).

Theorem 6.3 Under the new inner product (6.7) the native space NΦ,P(Ω) for aCPD function Φ on Ω is a Hilbert space with reproducing kernel defined by (6.6).In other words NΦ,P (Ω) = NK(Ω) as vector spaces but with different though verysimilar topologies.

Proof. It suffices to prove the reproduction property for some f ∈ NΦ,P(Ω) atsome x ∈ Ω via

(f,K(x, ·))Φ =

q∑

j=1

f(ξj)pj(x) + (f −ΠPf,Ψ(x, ·))Φ

= (ΠPf)(x) + (f −ΠPf)(x)

= f(x).

Theorem 6.3 is the reason why we do not consider the CPD case to be morecomplicated than the SPD case. We call K the regularized kernel with respectto the original CPD function Φ.

7 Numerical treatment of modified kernels

Here, we want to describe the numerical implications induced by reduction orregularization of a kernel Φ. Consider first the standard setting of interpolation ofpoint–evaluation data s|X ∈ IRM in some set X = x1, . . . , xM ⊂ Ω by a CPDfunction Φ, where we additionally assume that point–evaluation of Φ is possible.

14 R. Schaback

A generalization to Hermite–Birkhoff data will be given in section 10 below. Theinterpolant takes the form

s = pX +RΦ,Ω(λX,M ), pX ∈ P , λX,M ∈ LP(Ω) (7.1)

and this gives the linear system

(

AΦ,X PX

PTX 0

)(

λρ

)

=

(

s|X0

)

, (7.2)

where PX contains the values pj(xk), 1 ≤ j ≤ q, 1 ≤ k ≤ M for some arbitrary

basis p1, . . . , pq of P . The set X = x1, . . . , xM must be P–unisolvent to makethe system nonsingular, and thus we can assume Ξ ⊂ X and number the points ofX such that xj = ξj , 1 ≤ j ≤ q = dimP . This induces a splitting

A11 A12 P1

AT12 A22 P2

PT1 PT

2 0

λ1

λ2

ρ

=

s|Ξs|X\Ξ

0

with q× q matrices A11 and P1 and an (M − q)× (M − q) matrix A22. Passing toa Lagrange basis on Ξ then means setting σ = P1ρ and

A11 A12 IAT

12 A22 P2P−11

I (P−11 )TPT

2 0

(

λ1λ2

σ

)

=

s|Ξs|X\Ξ

0

. (7.3)

Now we take a closer look at the formula (6.2) and relate it to the above matrices.Denoting the identity matrix by I, we get the result

AΨ,X =

(

A11 A12

AT12 A22

)

−(

IP2P

−11

)

(A11, A12)

−(

A11

AT12

)

(

I, (P−11 )TPT

2

)

+

(

IP2P

−11

)

A11

(

I, (P−11 )TPT

2

)

and see that everything except the lower right (M − q)× (M − q) block vanishes.Setting Y := X \ Ξ we thus have proven

AΨ,Y = A22 − P2P−11 A12 −AT

12(P−11 )TPT

2 + P2P−11 A11(P

−11 )TPT

2

and this matrix is symmetric and positive definite due to Theorem 6.2. If weeliminate λ1 and σ in the system (7.3) by

λ1 = −(P−11 )TPT

2 λ2

σ = s|Ξ −A11λ1 −A12λ2

= s|Ξ +(

A11(P−11 )TPT

2 −A12

)

λ2

(7.4)

Native Spaces 15

we arrive at the system

AΨ,Y λ2 = s|Y − P2P−11 s|Ξ = s|Y − p|Y (7.5)

if p is calculated beforehand from the values of s on Ξ. The algebraic reductionto the above system and the transition to the case of Theorem 6.3 take at mostO(qM2) operations and thus are worth while when compared to the complexity ofa direct solution. Note that the transformations may spoil sparsity properties of theoriginal matrix AΦ,X , but they are unnecessary anyway, if compactly supported

functions on IRd are used, because these are SPD, not CPD.Finally, let us look at the numerical effect of a regularized kernel as in (6.6).

Since we used a Lagrange basis of P there, and since we have a special numbering,we get

AK,X =

(

I PTY

PY AK,Y

)

with PY = P2P−11 and AK,Y = AΨ,Y + PY P

TY , AK,Ξ = AΨ,Ξ + I = I.

Note that the representation (7.1) is different from (5.11). In particular, iftwo interpolants sY , sX based on different sets Y ⊇ X ⊇ Ξ coincide on X , thennecessarily ΠsY = ΠsY , but not pX = pY .

8 Properties of Native Spaces

Due to the pioneering work of Madych and Nelson [11], the native space can bewritten in another form. It is the largest space on which all functionals from LP(Ω)(and its closure LΦ,P(Ω)) act continuously:

Theorem 8.1 The space

MΦ,P(Ω) := f : Ω → IR : |λ(f)| ≤ Cf‖λ‖Φ for all λ ∈ LP(Ω)

coincides with the native space NΦ,P (Ω). It has a seminorm

|f |M := sup |λ(f)| : λ ∈ LP(Ω), ‖λ‖Φ ≤ 1 (8.1)

which coincides with |f |Ψ defined via (6.4).

Proof. For all functions f = pf + RΦ,Ω(λf ) ∈ NΦ,P(Ω) with λf ∈ LΦ,P(Ω)and pf ∈ P we can use (5.10) to prove that f lies in the space MΦ,P(Ω) withCf ≤ ‖λf‖Φ. To prove the converse, consider an arbitrary function f in MΦ,P(Ω)and define a functional Ff ∈ LP(Ω)

∗ by Ff (λ) := λ(f). The definition ofMΦ,P(Ω)makes sure that Ff is continuous on LP(Ω), and thus Ff can be continuouslyextended to LΦ,P (Ω). By the Riesz representation theorem, there is some λf ∈LΦ,P(Ω) such that

Ff (µ) = (µ, λf )Φ for all µ ∈ LΦ,P (Ω).

16 R. Schaback

Then we can define the function f − RΦ,Ω(λf ) and apply arbitrary functionalsµ ∈ LP(Ω) to get

µ(f −RΦ,Ω(λf )) = µ(f)− µ(RΦ,Ω(λf )) = µ(f)− (µ, λf )Φ = µ(f)− Ff (µ) = 0.

Specializing to µ = δ(x) for all x ∈ Ω we see that f − RΦ,Ω(λf ) coincides on Ωwith a function pf from P . Thus f = pf + RΦ,Ω(λf ) is a function in the nativespace NΦ,P (Ω). This proves that the two spaces coincide as spaces of real–valuedfunctions on Ω.

To prove the equivalence of norms, we use the above notation and first obtain|f |M ≤ ‖λf‖Φ from (8.1). But since we can replace LP(Ω) by its closure LΦ,P(Ω)in (8.1) and then use λf as a test functional, we also get

|f |M ≥ |λf (f)|/‖λf‖Φ = ‖λf‖Φ.

Due to |f |Ψ = ‖λf‖Φ this proves the assertion.

We now want to give a sufficient criterion due to Mark Klein [8] for differ-entiability of functions in the native space NΦ,P(Ω) of a CPD function Φ with

respect to some space P on Ω ⊆ IRd. Since this is a disguised statement aboutfunctionals in the dual space LΦ,P(Ω), we first look at functionals:

Theorem 8.2 Let an arbitrary functional λ be defined for functions on Ω andhave the properties

1. The real number λxλyΦ(x, y) is well–defined and

2. obtainable as the “double” limit of values λxnλ

ymΦ(x, y) for a sequence λkk

of finitely supported linear functionals λk from LP(Ω).

3. For any finitely supported linear functional ρ ∈ LP(Ω) the value ρxλyΦ(x, y)exists and is the limit of the values ρxλy

nΦ(x, y) for n → ∞.

Then the functional λ has an extension µ in the space LΦ,P(Ω) such that allappearances of λ in the above properties can be replaced by µ. All functionals inLΦ,P(Ω) can be obtained this way.

Proof. The second property means that for any ǫ > 0 there is some N ∈ IN suchthat

|λxλyΦ(x, y)− λxnλ

ymΦ(x, y)| < ǫ

for n,m ≥ N . Then for c := λxλyΦ(x, y) we get

‖λn − λm‖2Φ = ‖λn‖2Φ + ‖λm‖2Φ − 2(λn, λm)Φ

≤∣

∣‖λn‖2Φ − c∣

∣+∣

∣‖λn‖2Φ − c∣

∣+ 2 |(λn, λm)Φ − c|

< 4ǫ,

Native Spaces 17

proving that λkk is a Cauchy sequence. It has a limit µ ∈ LΦ,P (Ω), and bycontinuity we have c = (µ, µ)Φ. For any finitely supported functional ρ ∈ LP(Ω)we get

ρxλyΦ(x, y) = lim ρxλynΦ(x, y) = lim(ρ, λn)Φ = (ρ, µ)Φ.

Thus the action of λ on a function RΦ,Ω(ρ) coincides with the action of µ. Thefinal statement concerning necessity of the conditions is a simple consequence ofthe construction of LΦ,P (Ω).

The advantage of this result is that it does only involve limits of real num-bers and values of finitely supported functionals (except for λ itself). A typicalapplication is

Corollary 8.3 Let Ω1 ⊆ Ω ⊆ IRd be an open domain, and let derivatives of theform (Dα)x (Dα)yΦ(x, y) exist and be continuous on Ω1×Ω1 for a fixed multiindexα ∈ INd. Furthermore, assume P = IPd

m for m < |α| such that Dα(P) = 0. Thenall functions f in the native space FΦ,P(Ω) have a continuous derivative Dαf onΩ1.

Proof. Any pointwise multivariate derivative of order α at an interior point xcan be approximated by finitely and locally supported functionals which vanishon P = IPd

m if m < |α|. Thus there is a functional δαx ∈ LΦ,P (Ω) which acts likethis derivative on the functions in the space RΦ,Ω(LP(Ω)). Its action on generalfunctions RΦ,Ω(ρ) ∈ FΦ,P(Ω) with ρ ∈ LΦ,P(Ω) is also obtainable as the limit ofthe action of these functionals. This proves that the pointwise derivative exists forall functions in the native space.

To prove continuity of the derivative, we first evaluate

‖δαx − δαy ‖2Φ = ‖δαx‖2Φ + ‖δαy ‖2Φ − 2(δαx , δαy )Φ

= (Dα)u|x(Dα)v|xΦ(u, v) + (Dα)u|y (D

α)v|yΦ(u, v)

−2(Dα)u|x(Dα)v|yΦ(u, v)

and see that the right–hand side is a continuous function. Then

|(Dαf)(x)− (Dαf)(y)|2 = (δαx − δαy , R−1Φ,Ωf)

2Φ ≤ ‖δαx − δαy ‖2Φ‖f‖2Φ

proves continuity of the derivative of functions f in the native space.

Lower order derivatives and more general functionals λ can be treated simi-larly, applying Theorem 8.2 to approximations of λ− λ ΠP .

18 R. Schaback

9 Extension and Restriction

We now study the dependence of the native space NΦ,P (Ω) on the domain. To thisend, we keep the function Φ and its corresponding domain Ω fixed while lookingat subdomains Ω1 ⊆ Ω. In short,

Theorem 9.1 Each function from a native space for a smaller domain Ω1 con-tained in Ω has a canonical extension to Ω with the same seminorm, and therestriction of each function in NΦ,P(Ω) to Ω1 lies in NΦ,P(Ω1) and has a possiblysmaller norm there.

To prove the above theorem, we have to be somewhat more precise. Considera subset Ω1 with

Ξ ⊆ Ω1 ⊆ Ω ⊆ IRd

and first extend the functionals with finite support in Ω1 trivially to functionalson Ω. This defines a linear map

ǫΩ1: LP(Ω1) → LP(Ω)

which is an isometry because the inner products are based on (5.2) in both spaces.The map extends continuously to the respective closures, and we can use the Rieszmaps to define an isometric extension map

eΩ1: FΦ,P(Ω1) → FΦ,P(Ω), eΩ1

:= RΦ,Ω ǫΩ1RΦ,Ω1

−1

between the nontrivial parts of the native spacesNΦ,P(Ω1) andNΦ,P(Ω). The mainreason behind this very general construction of canonical extensions of functionsfrom “local” native spaces is that the variable x in (5.7) can vary in all of Ωwhile the support X = x1, . . . , xM of the functional λX,M is contained in Ω1.Of course, we define eΩ1

on P by straightforward extension of functions in P , andthus have eΩ1

well–defined on all of NΦ,P(Ω).At this point we want to mark a significant difference to the standard tech-

nique of defining local Sobolev spaces. On Ω = IRd one can prove that the globalSobolev space W k

2 (IRd) for k > d/2 is the native space for the radial positive

definite function

Φ(x, y) = φ(‖x− y‖2), φ(r) = rk−d/2Kr−d/2(r)

with a Bessel or Macdonald function. If we go over to localized versions of thenative space, we can do this for very general (even finite) subsets Ω1 of Ω = IRd,and there are no boundary effects. Furthermore, any locally defined function hasa canonical extension. This is in sharp contrast to the classical construction oflocal Sobolev spaces, introducing functions with singularities if the boundary hasincoming edges. These functions have no extension to a Sobolev space on a largerdomain. Functions from local Sobolev spaces only have extensions if the domain

Native Spaces 19

satisfies certain boundary conditions. Our definition always starts with the globalfunction Φ and then does a local construction. Is our construction really “local”?To gain some more insight into this question, we have to look at the restrictionsof functions from global native spaces.

Curiously enough, the restriction of functions from NΦ,P(Ω) to Ω1 is slightlymore difficult to handle than the extension. If we define rΩ1

(f) := f|Ω1for f ∈

NΦ,P(Ω), we have to show that the result is a function in NΦ,P(Ω1).

Lemma 9.2 The restriction map

rΩ1: NΦ,P(Ω) → NΦ,P(Ω1)

is well–defined and coincides with the formal adjoint of eΩ1. For any function f

in NΦ,P (Ω) the “localized” seminorm |rΩ1f |Ψ,Ω1

depends only on the values of fon Ω1. It is a monotonic function of Ω1.

Proof. From the extension property of ǫΩ1and the restriction property of rΩ1

wecan conclude

(ǫΩ1λΩ1

)(f) = λΩ1(rΩ1

f) for all λΩ1∈ LΦ,P(Ω1), f ∈ NΦ,P(Ω). (9.1)

This is obvious for finitely supported functionals from LP(Ω1) and holds in generalby continuous extension.

We use this to prove that rΩ1(f) := f|Ω1

lies in MΦ,P(Ω1) for each f inMΦ,P(Ω). This follows from

|λΩ1(rΩ1

f)| = |(ǫΩ1λΩ1

)f | ≤ |f |Ψ,Ω‖ǫΩ1λΩ1

‖Φ,Ω = |f |Ψ,Ω‖λΩ1‖Φ,Ω1

for all λΩ1∈ LΦ,P(Ω1) and all f ∈ FΦ,P(Ω). This also proves the inequality

|rΩ1f |Ψ,Ω1

≤ |f |Ψ,Ω. Both inequalities are trivially satisfied for f ∈ P . The mono-tonicity statement can be obtained by the same argument as above. Thus thenorm of the restriction map is not exceeding one. Our detour via MΦ,P(Ω1) im-plies that |rΩ1

f |Ψ,Ω1only depends on the values of f on Ω1. For all f ∈ FΦ,P(Ω)

and fΩ1∈ FΦ,P(Ω1) equation (9.1) yields

(eΩ1fΩ1

, f)Φ,Ω = (RΦ,ΩǫΩ1RΦ,Ω1

−1fΩ1, f)Φ,Ω

= (ǫΩ1RΦ,Ω1

−1fΩ1)(f)

= (RΦ,Ω1

−1fΩ1)(rΩ1

f)

= (fΩ1, rΩ1

f)Φ,Ω1.

(9.2)

This is the nontrivial part of the proof that that rΩ1is the formal adjoint of eΩ1

.

20 R. Schaback

Lemma 9.3 The above extension and restriction maps satisfy

rΩ1 eΩ1

= IdNΦ,P (Ω1).

Proof. The assertion is true on P . On FΦ,P(Ω1) we use (9.2) and the factthat eΩ1

is an isometry. Then for all fΩ1, gΩ1

∈ FΦ,P(Ω1) we have

(fΩ1, gΩ1

)Φ,Ω1= (eΩ1

fΩ1, eΩ1

gΩ1)Φ,Ω

= (fΩ1, rΩ1

eΩ1gΩ1

)Φ,Ω1

Thus gΩ1− rΩ1

eΩ1gΩ1

must be zero, because it is in FΦ,P(Ω1).

An interesting case of Lemma 9.2 occurs when Ω1 = X = x1, . . . , xM isfinite and contains Ξ. Then the functions in NΦ,P(Ω1) are of the form

s = p+RΦ,Ω1(λX,M ), p ∈ P , λX,M ∈ LP(Ω1)

and their seminorm is explicitly given by

|s|Ψ,Ω1= ‖λX,M‖Φ.

Due to Lemma 9.2 this value depends only on the values of s on Ω1 = X =x1, . . . , xM, and we can read off from (7.2) how this works. The value of thenorm is numerically accessible. See [14] for the special case of thin–plate splines.

If we look at the extension eΩ1s of s to all of Ω we see that it has the same

data on X . From

eΩ1(s−ΠPs) = RΦ,ΩǫΩ1

RΦ,Ω1

−1(s−ΠPs)

RΦ,Ω−1eΩ1

(s−ΠPs) = ǫΩ1RΦ,Ω1

−1(s−ΠPs)

one can read off that this functional has support in X = Ω1 and thus is the globalform of the interpolant.

This holds also for general transfinite interpolation processes. Consider anarbitrary subset Ω1 of Ω with Ξ ⊆ Ω1. On such a set, data are admissible if theyare obtained from some function f ∈ NΦ,P (Ω). Then eΩ1

rΩ1f has the same data

on Ω1. Furthermore, we assert that it is the global function with least seminormwith these data. By standard arguments this boils down to proving the variationalequation

(eΩ1rΩ1

f, v)Ψ,Ω = 0 for all v ∈ NΦ,P(Ω) with rΩ1v = 0.

But this is trivial for f ∈ P and follows from

(eΩ1rΩ1

f, v)Ψ,Ω = (RΦ,ΩǫΩ1RΦ,Ω1

−1rΩ1f, v)Ψ,Ω

= (ǫΩ1RΦ,Ω1

−1rΩ1f)(v)

= (RΦ,Ω1

−1rΩ1f)(rΩ1

v)

Native Spaces 21

for all f ∈ FΦ,P(Ω). Now we can generalize a result that plays an important partin Duchon’s [1] error analysis of polyharmonic splines:

Theorem 9.4 For all functions f ∈ NΦ,P(Ω) and all subsets Ω1 of Ω with Ξ ⊆Ω1 ⊆ Ω the function eΩ1

rΩ1f ∈ NΦ,P(Ω) has minimal seminorm in NΦ,P(Ω) under

all functions coinciding with f on Ω1.

Theorem 9.5 The orthogonal complement of eΩ1(NΦ,P (Ω1)) in NΦ,P(Ω) is the

space of all functions that agree on Ω1 with a function in P.

Proof. Use the above display again, but for v ∈ NΦ,P (Ω) being orthogonal to allfunctions g = rΩ1

f .

10 Linear recovery processes

For applications of native space techniques, we want to look at general methods forreconstructing functions in the native spaceNΦ,P(Ω) from given data. The data arefurnished by linear functionals λ1, . . . , λN from N ∗

Φ,P(Ω), and the reconstructionuses functions v1, . . . , vN from NΦ,P(Ω). At this point, we do not assume any linkbetween the functionals λj and the functions vj . A function f ∈ NΦ,P(Ω) is to bereconstructed via a linear quasi–interpolant

sf (x) :=N∑

j=1

λj(f)vj(x), x ∈ Ω, f ∈ NΦ,P(Ω)

which should reproduce functions from P by

p(x) = sp(x) =

N∑

j=1

λj(p)vj(x), x ∈ Ω, p ∈ P . (10.1)

Then the error functional

ǫx : f 7→ f(x)− sf (x), ǫx = δx −N∑

j=1

vj(x)λj(f)

is in LΦ,P (Ω) for all x ∈ Ω and is continuous on FΦ,P(Ω). This setting covers awide range of Hermite–Birkhoff interpolation or quasi–interpolation processes.

Theorem 10.1 With the power function defined by

P (x) := ‖ǫx‖Φthe error is bounded by

|f(x)− sf (x)| = |ǫx(f)| ≤ P (x)|f |Ψ for all f ∈ NΦ,P(Ω), x ∈ Ω.

22 R. Schaback

Proof. Just use (10.1) and evaluate

|f(x)− sf (x)| = |ǫx(f)| = |ǫx(f −ΠPf)|

≤ ‖ǫx‖Φ‖f −ΠPf‖Φ = P (x)|f |Ψ.

Theorem 10.2 The power function can be explicitly evaluated by

P 2(x) := ‖ǫx‖2Φ

= Ψ(x, x)− 2

N∑

j=1

vj(x)λyjΨ(y, x)

+

N∑

j=1

N∑

k=1

vj(x)vk(x)λyj λ

zkΨ(y, z).

(10.2)

Proof. Use

ǫx(f) =

δ(x) −N∑

j=1

vj(x)λj

(f −ΠPf)

for all f ∈ NΦ,P (Ω) and evaluate ‖ǫx‖2Ψ = ‖ǫx‖2Φ = P 2(x).

Note that it is in general not feasible to evaluate λxΦ(x, ·) for functionalsλ ∈ N ∗

Φ,P(Ω) unless they are plain point evaluations or vanish on P . This is whywe make a detour via Ψ here and use a different representation of ǫx. But in manystandard cases one can replace Ψ in (10.2) by Φ.

Altogether, each linear recovery process has a specific power function thatdescribes the pointwise worst–case error for recovery of functions from the nativespace. The power function can be explicitly evaluated, and it would be interestingto see various examples. Now we add more information on the relation betweenthe functionals λj and the functions vj :

Theorem 10.3 If the recovery process is interpolatory, i.e.

λj(vk) = δjk, 1 ≤ j, k ≤ N, (10.3)

the error is bounded by

|f(x)− sf (x)| = |ǫx(f)| ≤ P (x)|f − sf |Ψ for all f ∈ NΦ,P(Ω), x ∈ Ω.

Furthermore, we have

λj(P ) := λxj λ

yj (ǫx, ǫy)Ψ = 0, 1 ≤ j ≤ N. (10.4)

Native Spaces 23

Proof. The interpolation property implies sf−sf = 0, and then Theorem 10.1 canbe applied to the difference f − sf . This proves the error bound, and the secondassertion follows directly from (10.3) when applied to (10.2) written out in twovariables as required for (10.4).

Note that the special definition (10.4) for evaluation of λj(P ) is necessarybecause P is in general not in the native space. The new kernel function Q(x, y) :=(ǫx, ǫy)Ψ has a form similar to the reduction of Φ. It has interesting additionalproperties and can in particular be used for recursive construction of interpolantsand orthogonal bases. See [15] for details and [14] for a special case.

11 Optimal recovery

We now want to let the functions vj vary freely, but we keep the functionals λj

and the evaluation point x ∈ Ω fixed. Since the vj(x) influence only the powerfunction part of the error bound, we can try to minimize the quadratic form(10.2) with respect to the N real numbers vj(x), 1 ≤ j ≤ N under the linearconstraints imposed by (10.1). We do this by application of standard techniques ofoptimization. Picking a basis p1, . . . , pq of P and introducing Lagrange multipliersqm(x), 1 ≤ m ≤ q for the q linear constraints from (10.1), we get that any criticalpoint of the constrained quadratic form is a critical point of the unconstrainedquadratic form

Ψ(x, x)− 2

N∑

j=1

vj(x)λyjΨ(y, x)

+

N∑

j=1

N∑

k=1

vj(x)vk(x)λyjλ

zkΨ(y, z)

− 2

q∑

m=1

qm(x)

(

pm(x)−N∑

k=1

vk(x)λk(pm)

)

to be minimized with respect to the values vj(x). The equations for a critical pointof the constrained quadratic form thus are

λykΨ(x, y) =

N∑

j=1

λyjλ

zkΨ(y, z)v∗j (x) +

q∑

m=1

q∗m(x)λk(pm)

pm(x) =

N∑

j=1

λj(pm)v∗j (x), 1 ≤ k ≤ N, 1 ≤ m ≤ q.

(11.1)

24 R. Schaback

We want to prove that this system has a unique solution. Taking a solution of thehomogeneous system

0 =

N∑

j=1

λyjλ

zkΨ(y, z)aj +

q∑

m=1

bmλk(pm)

0 =

N∑

j=1

λj(pm)aj , 1 ≤ k ≤ N, 1 ≤ m ≤ q,

we see that the functional

χ :=

N∑

j=1

ajλj

vanishes on P and thus is in LΦ,P(Ω). Applying it to the first N equations of(11.1) yields

0 =

N∑

j=1

N∑

k=1

λyjλ

zkΨ(y, z)ajak +

q∑

m=1

bm

N∑

k=1

λk(pm)ak

=

N∑

j=1

N∑

k=1

λyjλ

zkΨ(y, z)ajak + 0

= χyχzΨ(y, z)

= χyχzΦ(y, z).

Since Φ is a CPD function, we conclude that the functional χ must vanish on thenative space. If we make the reasonable additional assumption that the functionalsλj are linearly independent, we see that the coefficients aj must vanish. This leavesus with the system

0 =

q∑

m=1

bmλk(pm), 1 ≤ k ≤ N,

and we can conclude that all the coefficients bm are zero, if we assume that thereis no nonzero function p in P for which all data λk(p), 1 ≤ k ≤ N vanish.

Now we know that (11.1) has a unique solution v∗j (x), q∗m(x) for indices1 ≤ j ≤ N, 1 ≤ m ≤ q = dimP . Thus the augmented unconstrained quadraticform has a unique critical point, and the same holds for the constrained quadraticform. Since the latter is nonnegative and positive definite, the critical point mustbe a minimum. Furthermore, we see that the optimal solution values v∗j (x), q

∗m(x)

are linear combinations of the right–hand values pm(x), 1 ≤ m ≤ q and λykΨ(x, y)

for 1 ≤ k ≤ N . If we apply the functional λj to the system, we see that the j–thcolumn coincides with the right–hand side, and this proves that the solution mustsatisfy the interpolation property (10.3). Furthermore, we get λj(qm) = 0 for allindices 1 ≤ j ≤ N, 1 ≤ m ≤ q.

Native Spaces 25

But note that our solution is not just any interpolatory set of functionsmatching the data functionals. It is uniquely determined by the λj , and it iscomposed of functions λy

jΨ(y, ·) acting as the Riesz representers of the λj in thesense of Theorem (6.1) plus functions from P . We summarize:

Theorem 11.1 Assume that there is no nonzero function from P on which allfunctionals λj vanish, and that the λj are linearly independent. Then the powerfunction at any point x ∈ Ω can be minimized among all other power functionsusing the same functionals λj, but possibly different functions vj. The minimum isattained for a specific set v∗j of functions that satisfy the interpolation conditions(10.3) and are linear combinations of functions from P and generalized represen-ters of the functionals λj .

We add an intrinsic characterization of the optimal power function:

Theorem 11.2 Under the above assumptions, the optimal power function P ∗(x)describes the maximal value that any function f ∈ NΦ,P(Ω) can attain at x ∈ Ω ifit satisfies the restrictions λj(f) = 0, 1 ≤ j ≤ N and |f |Ψ ≤ 1.

Proof. For any such function, Theorem 10.3 implies |f(x)| ≤ P ∗(x) because ofsf = 0. For fixed x ∈ Ω, the maximal value is attained for the special function

fx = Ψ(·, x)−N∑

j=1

v∗j (x)λyjΨ(y, ·)−

q∑

m=1

q∗m(x)pm(·)

after rescaling. This is due to

fx(x) = (P ∗(x))2 = |fx|2Ψwhich needs some elementary calculations and the identity

N∑

k=1

v∗k(x)λykΨ(x, y) =

N∑

j=1

N∑

k=1

v∗j (x)v∗k(x)λ

yjλ

zkΨ(y, z) +

q∑

m=1

q∗m(x)pm(x)

following from (11.1).

Under the assumptions of Theorem 11.1 one can try to rewrite the recoveryfunction as

s∗f(x) =

N∑

j=1

λyjΨ(y, x)aj +

q∑

m=1

pm(x)bm, (11.2)

where the coefficients aj satisfy

N∑

j=1

λj(pm)aj = 0, 1 ≤ m ≤ q = dimP

26 R. Schaback

in order to form a functional from LP(Ω). The equations for a generalized inter-polation of a function f then are

λk(f) = λk(s∗f ) =

N∑

j=1

λyjλ

zkΨ(y, z)aj +

q∑

m=1

λk(pm)bm

0 =

N∑

j=1

λj(pm)aj , 1 ≤ k ≤ N, 1 ≤ m ≤ q,

(11.3)

and the coefficient matrix coincides with the matrix in (11.1). There is muchsimilarity to (7.2), but we are more careful here and put Ψ instead of Φ into thedefinition (11.2) of the recovery function, because λy

jΦ(y, ·) may not make sensewhile δyxj

Φ(y, ·) in (7.1) always is feasible.If we take the Ψ–inner product of s∗f with an arbitrary function g from the

native space, we get

(s∗f , g)Ψ =

N∑

j=1

aj(

λyjΨ(y, ·), g

)

Ψ+ 0

=

N∑

j=1

ajλyj (g(y)− (ΠPg)(y))

=

N∑

j=1

ajλj(g).

By a standard variational argument this implies an extension of Theorem 9.4 tomore general data functionals:

Theorem 11.3 Under the hypotheses of Theorem 11.1 the function s∗f solves theminimization problem

min |g|Ψ : g ∈ NΦ,P(Ω), λj(f − g) = 0, 1 ≤ j ≤ N .Furthermore, the orthogonality relation

(f − s∗f , sf )Ψ = 0

holds and implies|f |2Ψ = |f − s∗f |2Ψ + |s∗f |2Ψ.

From section 9 we already know that the second term on the right–hand sidecan be calculated explicitly. The minimization principle of Theorem 11.3 impliesthat this value increases when more and more data functionals are used to recoverthe same function f . In case of reconstruction of a function from the native space,there is the upper bound |f |2Ψ for these values, but for general given functionsthere might be no upper bound. However, the following related characterizationof native spaces was proven in [16]:

Native Spaces 27

Theorem 11.4 The native space for a continuous CPD function Φ on Ω can becharacterized as the set of all real–valued functions f on Ω for which there is a fixedupper bound Cf ≥ |s∗f |Ψ for all interpolants s∗f based on arbitrary point–evaluationdata λj = δxj

, xj ∈ Ω.

12 Connection to L2 spaces: Overview

This section starts an analysis of native spaces directed towards the well–knownrepresentation of the “energy inner product” of classical splines in the form

(f, g)Φ = (Lf, Lg)L2(Ω) =: (Lf, Lg) (12.1)

with some linear differential operator L. Natural univariate splines of odd degree2n−1 are related to L = dm/dxm on Ω = [a, b] ⊂ IR. Furthermore, the fundamentalwork of Duchon on thin–plate and polyharmonic splines is based strongly on theuse of L = ∆m. For general (not necessarily radial) basis functions, there is noobvious analogue of such an operator. However, we want to take advantage of(12.1) and thus proceed to work our way towards a proper definition of L.

Since the procedure is somewhat complicated, we give an overview here, andpoint out the reasons for certain arguments that may look like unnecessary detours.We first have to relate the native space somehow to L2(Ω). To achieve this, wesimply imbed the major part FΦ,P(Ω) of the native space NΦ,P(Ω) = FΦ,P(Ω)+Pinto L2(Ω). Then we study the adjoint C of the embedding, which turns out tobe a convolution–type integral operator with kernel Φ that finally will be equal to(L∗L)−1. We thus have to form the “square root” of the operator C and invertit to get L. Taking the square root requires nonnegativity of C in the sense ofintegral operators. This is a property that is intimately related to (strict) positivedefiniteness of the kernel Φ, and thus in section 16 we take a closer look at therelation of these two notions. In between, section 15 will provide a first applicationof the technique we develop here. In the notation we use (·, ·) to denote the innerproduct in L2(Ω).

13 Embedding into L2

There is an easy way to imbed a native space into an L2 space.

Lemma 13.1 Let Φ be SPD on Ω, and let Ψ be the normalized kernel with respectto Φ as defined in section 6. Assume

C22 :=

∫

Ω

Ψ(x, x)dx < ∞. (13.1)

Then the Hilbert space FΦ,P(Ω) ⊆ NΦ,P(Ω) for Φ has a continuous linear embed-ding into L2(Ω) with norm at most C2.

28 R. Schaback

Proof: Conditional positive definiteness clearly implies that the integrand Ψ(x, x) =(δ(x), δ(x))Φ = ‖δ(x)‖2Φ is positive when forming (13.1).

Now for all f ∈ FΦ,P(Ω) and all x ∈ Ω we can use the reproduction property(5.11) to get

f(x)2 = (f,Ψ(x, ·))2Φ≤ ‖f‖2Φ‖Ψ(x, ·)‖2Φ= ‖f‖2ΦΨ(x, x),

where we used ΠPf = 0 for the functions f ∈ FΦ,P(Ω). Then the assertion followsby integration over Ω.

By the way, the above inequality shows in general how upper bounds forfunctions in the native space can be derived from the behaviour of Ψ on thediagonal of Ω× Ω.

14 The convolution mapping from L2 into FΦ,P(Ω)

We now go the other way round and map L2(Ω) into the native space.

Theorem 14.1 Assume (13.1) to hold for a CPD function Φ on Ω. Then theintegral operator

C(v)(x) :=

∫

Ω

v(t)Ψ(x, t)dt (14.1)

of generalized convolution type maps L2(Ω) continuously into the Hilbert spaceFΦ,P(Ω) ⊆ NΦ,P(Ω). It has norm at most C2 and satisfies

(f, v) = (f, C(v))Φ for all f ∈ FΦ,P(Ω), v ∈ L2(Ω), (14.2)

i.e. it is the adjoint of the embedding of the Hilbert subspace FΦ,P(Ω) of the nativespace NΦ,P (Ω) into L2(Ω).

Proof: We use the definition of MΦ,P(Ω) in Theorem 8.1 and pick somefinitely supported functional λ ∈ LP(Ω) to get

λ(C(v)) =

∫

Ω

v(t)λxΨ(x, t)dt

≤ ‖v‖‖λxΨ(x, ·)‖≤ C2‖v‖‖λ‖Φ

for all v ∈ L2(Ω). In case of f(t) := Ψ(x, t) with arbitrary x ∈ Ω, equation (14.2)follows from the definition of the operator C and from the reproduction property.The general case is obtained by continuous extension.

Of course, equation (14.2) generalizes to

(f −ΠPf, v) = (f −ΠPf, C(v))Φ = (f, C(v))Ψ for all f ∈ NΦ,P (Ω), v ∈ L2(Ω)

Native Spaces 29

on the whole native space NΦ,P (Ω).We add two observations following from general properties of adjoint map-

pings:

Corollary 14.2 The range of the convolution map C is dense in the Hilbert spaceFΦ,P(Ω). The latter is dense in L2(Ω) iff C is injective.

Criteria for injectivity of C or, equivalently, for density of the Hilbert spaceFΦ,P(Ω) in L2(Ω) are an open problem (in general). The main obstacle is toconstruct sufficiently many test functions in the native space. For classical radialbasis functions on Ω = IRd one can use Fourier transform techniques to proveexistence of tempered or C∞

0 test functions in the native space on IRd. Then oneuses the restriction technique to prove that restrictions of these functions are inthe local native spaces. This is how the problem can be attacked from the globalsituation.

We finally remark that the above problem is related to the specific way ofdefining an SPD or CPD function via finitely supported functionals. Section 16will shed some light on another feasible definition.

15 Improved convergence results

The space C(L2(Ω)) allows an improvement of the standard error estimates forreconstruction processes of functions from native spaces. Roughly speaking, theerror bound can be “squared”.

Theorem 15.1 labelTEBthree If an interpolatory recovery process in the sense ofTheorem 11.1 is given, then there is a bound

|f(x)− s∗f (x)| ≤ P ∗(x)‖P ∗‖‖v‖for all f − ΠPf = C(v) ∈ NΦ,P(Ω), x ∈ Ω, v ∈ L2(Ω). Here, we denote theoptimized power function for the special situation in Theorem 11.1 by P ∗.

Proof: Taking the L2 norm of the standard error bound in 10.3, we get

‖f − s∗f‖ ≤ ‖P ∗‖‖f − s∗f‖Ψ.Now we use (14.2) and the orthogonality relation from Theorem 11.3:

‖f − s∗f‖2Ψ = (f − s∗f , f − s∗f )Ψ= (f − s∗f , f)Ψ

= (f − s∗f , C(v))Ψ= (f − s∗f , v)

≤ ‖f − s∗f‖‖v‖≤ ‖P ∗‖‖f − s∗f‖Φ‖v‖.

30 R. Schaback

Cancelling ‖f − s∗f‖Φ and inserting the result into the error bound of Theorem10.3 proves the assertion.

An earlier version of this result, based on Fourier transforms and restrictedto functions on Ω = IRd was given in [19].

16 Positive integral operators

We now look at the operator C from the point of view of integral equations. Thecompactness of C as an operator on L2(Ω) will be delayed somewhat, because wefirst want to relate our definition of a positive definite function to that of a positiveintegral operator. The latter property will be crucial in later sections.

Definition 16.1 An operator C of the form (14.1) is positive (nonnegative),if the bilinear form

(w,C(v)), v, w ∈ L2(Ω)

is symmetric and positive (nonnegative) definite on L2(Ω).

In our special situation we can write

(w,C(v)) = (C(w), C(v))Φ , v, w ∈ L2(Ω)

and get

Theorem 16.2 If a symmetric and positive semidefinite function Φ on Ω satis-fies (13.1), then the associated integral operator C is nonnegative. If this holds,positivity is equivalent to injectivity.

Theorem 16.3 Conversely, if C is a nonnegative integral operator of the form(13.1) with a symmetric and continuous function Φ : Ω × Ω → IR, then Φ ispositive semidefinite on Ω.

Proof: We simply approximate point evaluation functionals δx by functionalson L2(Ω) that take a local mean. Similarly, we approximate finitely supportedfunctionals by linear combinations of the above form. The rest is standard, butrequires continuity of Φ.

Unfortunately, the above results do not allow to conclude positive definitenessof Φ from positivity of the integral operator C. However, due to the symmetry ofΦ, the integral operator C is always self–adjoint.

Native Spaces 31

17 Compact nonnegative self–adjoint integral op-

erators

To apply strong results from the theory of integral equations, we still need that Cis compact on L2(Ω). This is implied by the additional condition

∫

Ω

∫

Ω

Φ(x, y)2dxdy < ∞ (17.1)

which is automatically satisfied if our SPD function Φ is continuous and Ω iscompact. Note the difference to (13.1), which is just enough to ensure embeddingof the native space into L2(Ω).

From now on, we assume Φ to be an SPD kernel satisfying (13.1) and (17.1).Then C is a compact self–adjoint nonnegative integral operator. Now spectraltheory and the theorem of Mercer [18] imply the following facts:

1. There is a finite or countable set of positive real eigenvalues

µ1 ≥ µ2 ≥ . . . > 0 and eigenfunctions ϕ1, ϕ2, . . . ∈ L2(Ω) such that

C(ϕn) = µnϕn, n = 1, 2, . . . .

2. The eigenvalues µn converge to zero for n → ∞, if there are infinitely many.

3. There is an absolutely and uniformly convergent representation

Φ(x, y) =∑

n

µnϕn(x)ϕn(y), x, y ∈ Ω.

4. The functions ϕn are orthonormal in L2(Ω).

5. Together with an orthonormal basis of the kernel of C, the functions ϕn forma complete orthonormal system.

6. There is a nonnegative self–adjoint operator ∗√C such that C = ∗

√C ∗√C and

with an absolutely and uniformly convergent kernel representation

∗√Φ(x, y) :=

∑

n

√µnϕn(x)ϕn(y), x, y ∈ Ω,

where∗√C(v)(x) :=

∫

Ω

v(t)∗√Φ(x, t)dt, x ∈ Ω, v ∈ L2(Ω).

We use the symbol ∗√Φ to denote the “convolution square–root”, because

Φ(x, y) =

∫

Ω

∗√Φ(x, t)

∗√Φ(t, y)dt

32 R. Schaback

is a generalized convolution. We remark that this equation can be used for con-struction of new positive definite functions by convolution, and we hope to findroom for details later.

The situation of finitely many eigenvalues cannot occur for the standard caseof continuous SPD kernels on bounded domains with infinitely many points andlinearly independent point evaluations. Otherwise, the rank of matrices of the form(Φ(xj , xk))1≤j,k≤N would have a global upper bound.

18 The native space revisited

The action of C on a general function v ∈ L2(Ω) can now be rephrased as

C(v) =∑

n

µn(v, ϕn)ϕn,

and it is reasonable to define an operator L such that (L∗L)−1 = C formally by

L(v) =∑

n

(µn)−1/2(v, ϕn)ϕn. (18.1)

We want to show that this operator nicely maps the native space into L2(Ω),but for this we have to characterize functions from the native space in terms ofexpansions with respect to the functions ϕn.

Theorem 18.1 The native space for an SPD function Φ which generates a non-negative compact integral operator on L2(Ω) can be characterized as the space offunctions f ∈ L2(Ω) with L2(Ω)–expansions

f =∑

n

(f, ϕn)ϕn

such that the additional summability condition

∑

n

(f, ϕn)2

µn< ∞

holds.

Proof: We first show that on the subspace C(L2(Ω)) of the native spaceNΦ,P(Ω) we can rewrite the inner product as

(C(v), C(w))Φ = (v, C(w))

=∑

n

(v, ϕn)(C(w), ϕn)

=∑

n

(C(v), ϕn)(C(w), ϕn)

µn

Native Spaces 33

But this follows from (C(v), ϕn) = µn(v, ϕn) for all v ∈ L2(Ω). Since C(L2(Ω))is dense in NΦ,P (Ω) due to Corollary 14.2, and since NΦ,P(Ω) is embedded intoL2(Ω), we can rewrite the inner product on the whole native space as

(f, g)Φ =∑

n

(f, ϕn)(g, ϕn)

µnfor all f, g ∈ NΦ,P(Ω). (18.2)

Corollary 18.2 The functions√µnϕn are a complete orthonormal system in the

native space NΦ,P(Ω).

Proof: Orthonormality immediately follows from (18.2), and Theorem 18.1allows to rewrite all functions from the native space in the form of an orthonormalexpansion

f =∑

n

(f,√µnϕn)Φ

√µnϕn

with respect to the inner product of the native space.

Corollary 18.3 The operator L defined in (18.1) maps the native space NΦ,P(Ω)into L2(Ω) such that (12.1) holds. It is an isometry between its domain NΦ,P(Ω)and its range L2(Ω)/ker C = clos( span ϕnn).

Acknowledgements: Special thanks go to Holger Wendland for carefulproofreading and various improvements.

References

[1] J. Duchon, Interpolation des fonctions de deux variables suivant le principede la flexion des plaques minces, Rev. Francaise Automat. Informat. Rech.Oper., Anal. Numer. 10 (1976), 5–12

[2] N. Dyn, F. Narcowich and J. Ward, Variational principles and Sobolev–typeestimates for generalized interpolation on a Riemannian manifold, to appearin Constr. Approx.

[3] W. Freeden, R. Franke and M. Schreiner, A survey on spherical spline ap-proximation, Surveys on Mathematics for Industry 7 (1997), 29–85

[4] M. Golomb, H.F. Weinberger, Optimal approximation and error Bounds, in:On Numerical Approximation, The University of Wisconsin Press, Madison,R.E. Langer (1959), 117–190

34 R. Schaback

[5] T. Gutzmer, Interpolation by positive definite functions on locally compactgroups with application to SO(3), Resultate Math. 29 (1996), 69–77

[6] R.L. Hardy,Multiquadric equations of topography and other irregular surfaces,J. Geophys. Res. 76 (1971), 1905–1915

[7] K. Jetter, J. Stockler, A Generalization of de Boor’s stability result and sym-metric preconditioning, Advances in Comp. Math. 3 (1995), 353–367

[8] M. Klein, Spezielle Probleme der Rekonstruktion multivariater Funktionen,Diplom thesis, University of Gottingen, 1998

[9] W.L. Light, H. Wayne, On power functions and error estimates for radialbasis function interpolation, J. Approx. Theory 92 (1998), 245–267

[10] Lin–Tian Luh, Native spaces for radial basis functions, Dr. rer. nat. disserta-tion, Gottingen 1998

[11] W.R. Madych, S.A. Nelson, Multivariate Interpolation and conditionally pos-itive definite functions, Approx. Theory Appl. 4 (1988), 77–89

[12] H. Meschkowski, Hilbertsche Raume mit Kernfunktion, Springer, Berlin(1962)

[13] F.J. Narcowich, Generalized Hermite interpolation by positive definite kernelson a Riemannian manifold, J. Math. Anal. Appl. 190 (1995), 165–193

[14] M.J.D. Powell, ????? IDoMat 98

[15] R. Schaback, Reconstruction of multivariate functions from scattered data,manuscript, available via

http://www.num.math.uni-goettingen.de/schaback/research/group.html

[16] R. Schaback, H. Wendland, Inverse and saturation theorems for radial basisfunction interpolation, Preprint 1998, Univ. Gottingen

[17] J. Stewart, Positive definite functions and generalizations, an historical sur-vey, Rocky Mountain J. Math. 6 (1976), 409–434

New papers added for part two:

[18] Porter, D., and D.G. Stirling, Integral equations: a practical treatment fromspectral theory to applications, Cambridge Texts in Applied Mathematics,Cambridge University Press 1990

[19] Schaback, R., Improved error bounds for radial basis function interpolation,to appear in Math. Comp.

Native Spaces 35

Robert SchabackUniversitat Gottingen, Lotzestraße 16-18D-37083 Gottingen, GermanyEmail address: schaback@math.uni-goettingen.de