DAMTP /1993/NA6

Some algorithms for thin plate spline interpolationto functions of two variables1

M.J.D. Powell

Abstract: A disadvantage of thin plate spline interpolation to n values of afunction of two variables is that all but a few elements of the interpolation matrixare nonzero. Nevertheless, the calculation enjoys some structure that usuallyallows the required coefficients to be computed in much less than O(n3) operationsfor general positions of the interpolation points. In particular, linear combinationsof several thin plate spline terms can be replaced by a single truncated Laurentexpansion, when values of the linear combination are required at points thatare sufficiently far from the centres of the thin plate spline terms. Further, theinterpolation equations can be reduced to an (n - 3) x (n - 3) positive definitesystem, in order to employ a pre-conditioned conjugate gradient method. Thesetechniques are described briefly and are applied to some numerical examples. vVe

find that our implementations of the ~!.~re1}1and .£2gJ.]g!~.<l£~~ent.~~Lh,,<?~provide substantial reductions in the amount of computation when n 2:: 1000 andn;:::: 100, respectively.

Department of Applied Mathematics and Theoretical Physics,University of Cambridge,Silver Street,Cambridge CB3 9EvV,England.

March, 1993.

1Presented at the International Conference on Advances in Computational Mathematics

(New Delhi, January, 1993)

Thin plate spline interpolation

1. Introduction to thin plate spline interpolation

3

The general interpolation problem in two dimensions is to pick a function s from"'C'~d" •. "-'·~·'·~ .' - .• ~~~--~_~;~':""~:"O7'~::::""

n2 to n that satisfies the equations --- ------,,---'-

where the interpolation points L~i: i = 1,2, ... , n} and the right hand sides{Ii : i= 1,2, ... , n} are given. All the interpolation points have to be different,and we assume also that they do not lie on a straight line, which implies n ~ 3,but we impose no other restrictions on their positions.

The survey papers of Sabin (1980) and Franke (1982) show that several choicesof s have been proposed. The method that seems to be used most in practice ispiecewise linear interpolation. Here the convex hull of the interpolation points inR} is divided into triangles, such that {.f.i : i= 1, 2, ... , n} is the set of vertices ofthe triangles and no vertex is also the interior point of an edge of a triangle. Thens is defined on each triangle by linear interpolation to the data at the vertices. Itfollows that s is continuous across the edges, but we expect discontinuities to occurin the first derivatives of s. The method is excellent when these discontinuities donot matter. Indeed, it is usual in such cases for n to be large, and then it is highlyvaluable that the amount of computation need only be of magnitude n log n.

On the other hand, when there are fewer than 100 interpolation points and thecurvature of the underlying function is important, then usually piecewise linearinterpolation is too crude. Further, for much larger values of n, the domain ofthe data may be so extensive that curvature is important on each typical sub­domain that contains about 100 interpolation points. Therefore there is a needfor interpolation algorithms that provide good smoothness when n = 104, say,without an excessive amount of computation. Consequently, an algorithm thatrequires O( n2) operations could be useful, but O( n3) complexity would probablybe unacceptable.

This demand can be met by interpolating functions s that are composed ofquadratic or cubic polynomial pieces. On the other hand, when n is such thatO( n3) operations are tolerable, then the use of thin plate splines is highly success­ful in several applications (see Barrodale, Kuwahara, Poeckert and Skea, 1992,for example). Further, these interpolants have two strong advantages over piece­wise polynomials. Specifically, there are no artificial divisions of the data intosubregions for the separate pieces of s, and thin plate spline interpolants providethe solution to a very pertinent variational problem. Therefore, we are going toconsider some ~~~§ __oL~~i~_l?!a.~~sp!ineinterpola,nts that may allow theircoefficients to be calculated in f~:ver than,<2(1"!c~)operations.

In the variational problem, -;--1s required to have square integrable secondderivatives, and the freedom in s after satisfying the interpolation equations (1.1)

S(~i) = Ii, i= 1, 2, ... , n, (1.1)

4

is taken up by minimizing the expression

M.J.D. Powell

V( )=~ (82S(~))2 2(82S(~))2 (82S(~))2d ( 'J)s 8 2 + 8 8 + 8 2~' 1.~1<.2 X X Y Y

where we reserve x and Y for the components of ±. It is proved by Duchon (1977)that s has the form

n

s(±) = L Aj II±-±j 112log II±-±j II + Co+CIX+C2Y,j=l

(1.3)

the vector norm being Euclidean. Further, the coefficients {Aj : j = 1,2, ... , n}are constrained by the conditions

n n n

LAj = 2: AjXj = 2: AjYj = 0,j=l j=l j=l

(1.4)

where Xj and Yj are the components of ~j' in order that the integral (1.2) is finite.We see that the equations (1.1) and (1.4) provide a square linear system in theunknowns {Aj : j =1, 2, ... , n} and {Co, Cl, C2}'

We write this system in the form

(1.5)

Here <I>is n x n and has the elements {1I±i - ±j 112 log lI±i - ±jII : i, j = 1,2, ... , n},while P is n x 3, its i-th row being the vector (1 Xi yJ Further,~, ~ and farethe vectors that have the components {Aj: j=1,2, ... ,n}, {Co,Cl,C2} and-{fi:i = 1, 2, ... ,n}, respectively. One can deduce from the above variational problemthat the (n + 3) x (n + 3) matrix of the system (1.5) is nQJ1~ipg~19:!'Therefore V-the data define the function (1.3) .£,niqJMilir.This choice of s is the "thin platespline" interpolant, so our aim is to solve the equations (1.5) in fewer than O(n3)operations.

Our work will depend on a property of <I>that we establish by integration byparts. In order to begin this task, we note that the function

(1.6)

has the derivatives

"9 2qS(;~:) = 4 (1+log lI.flD },"9 4qS(;~.) = 87r 8(.f)

where "92 is the Laplacian operator

.f E 1?..2 , (1. 7)

(1.8)

Thin plate spline interpolation 5

and where 8 is the delta function in R}. Further, we let ft be any nonzero vectorin Rn that satisfies the equations

n n n

Lej = LejXj = LejYj = 0,j=l j=l j=l

and we introduce the functionn

s(~) = LOjll~-~jI12log II~-~jll, ~ER2.j=l

(1.9)

(1.10)

The conditions (1.9) imply that V(S) is finite, and two applications of integrationby parts to the definition (1.2) give the identity

V( s) = k2 s(~) \7 4 s(~) d~n n

- ~ {L Oill~-~iIl21og 11.;r.-3:.ill}{811" L Oj8(.if.-.if.j)} d~n2 i=l j=ln n

811" L L Oill.f.i-.f.j112log lI~i-iJ OJ = 811" Ii? if> ft,i=l j=l

(1.11)

which depends on equations (1.6)-(1.10), the conditions (1.9) being importantbecause they imply that the boundary terms of the integrations by parts are zero.Further ft =I 0 and the definition (1.2) imply that V( s) is positive. Thus we reachthe conclusion

Q=lO and pTQ=O :::;. ~T<I>Q>O. (1.12)

Another consequence of formula (1.7) will also be useful later. It depends on

the remark that \7E.2rjJ(~-JlJ and \7y2rjJ(.;r._Jl) are identical, where \7i and \7y2 denotethe Laplacian operator applied to- functions of ~ and y, respectively. Specifically,we will make use of the property -

(1.13)

2. Advantages of conjugate gradient methods

Conjugate gradient methods are highly useful for solving systems of linear equa­tions whose matrices are symmetric and positive definite, but the matrix of the

system (1.5) has some negative eigenvalues, because all its diagonal elements arezero. On the other hand, expression (1.12) shows a conditional positive definiteproperty of if>. Fortunately, this property allows the calculation of ~ to be reducedto the solution of an (n - 3) x (n - 3) positive definite system of linear equations.Details of this construction are given by Sibson and Stone (1991), and they are.-=._----_ ..",.~-,-.._...,-_._-~._' .~._~<~_.,,-

as follows.

6 M.J.D. Powell

The second half of the system (1.5) states that ~ is in the null space of P T,which is an (n - 3)-dimensional subspace of nn. Therefore, if we let Q be anynx(n-3) matrix whose columns span this subspace, then we can express ~ in theform

(2.1)

for some vector f-l in nn-3. Further, we pre-multiply the first half of the system(1.5), which is the equation <I>~+P.f= f, by QT. Thus, substituting for ~ andnoting that QTp is zero, we deduce the-;elation

(2.2)

(2.3)

The positive definiteness of Q T <I> Q is an elementary consequence of the property(1.12). Therefore we recommend that f-l be calculated by applying a conjugategradient method to the system (2.2). Then formula (2.1) provides the vector .6..

Further, the three components of f can be derived from the identity Ps;.= f - <I>~j

which requires much less work than the calculation of ~ for n ~ 10. -

There is a convenient way of £hoosi~4£. implicitly using l!£..1.!.§l.~b~dertr.CLll~­formati<2-!ls.Specifically, we define PI = P, and then, for j = 1,2,3, we calculate:!l.j Enn such that the matrix

( 2uou:f)Pj+! = I - II~:II~Pj

has zeros below the diagonal in its first j columns. Thus n Tp is upper triangular,where n is the orthogonal matrix

(2.4)

It follows that the last (n-3) columns of n span the null space of pT, so we letQ have these columns. Consequently, QT<I>Q is the bottom right (n-3) x (n-3)submatrix of nT <I> n. It can be calculated in O( n2) operations from expression(2.4) after the components of :!l.1, :!l.2 and lli have been found, which requires onlyO(n) operations. Further, the numbers on the right hand side of the system (2.2)

are just the last (n-3) components of nTL which can alsobe computed in O(n)operations. It is important to efficiency that Q is not formed explicitly. Thereforewe write equation (2.1) as the product ~=np, where p is the vector in nn whosecomponents are three zeros followedby the c;-mponent-;of l!:.: Thus the calculationof .6. from 1:: also has O(n) complexity. All of these details can be useful when nis at most a few hundred, because then it may be best to generate the matrixQ T <I> Q explicitly, in order to solve the system (2.2) by a direct method such asCholeili- factorization.

+~

'When n is large, however, then there are two ways in which a conjugate gra-dient procedure can improve substantially on the O(n3) work of a direct method.

Thin plate spline interpolation 7

(2.5)

(2.6)

Firstly, the number of operations of each iteration may be less than O(n2), andsecondly the total number of iterations to achieve enough accuracy in f-L may bemuch less than n. The remainder of this section considers the first possibility,while the number of iterations is the subject of Section 3.

The only task of an iteration of the conjugate gradient method that may requiremore than O( n) operations when solving the system (2.2) is the multiplicationof a vector by the matrix QT<J? Q (see Golub and Van Loan, 1983, for instance).As in the paragraph that includes equation (2.4), our choices of Q will be suchthat the work of multiplying a vector by Q or by QT is only O(n). Therefore weaddress the calculation of <J?~, where ~ is a general vector in nn. Fortunately, ifone can accept an approximation to this product whose accuracy is specified inadvance, then the following technique of Beatson and Newsam (1992) providesthe approximation in about O( n log n) operations.

The required components of <J?~ are the values of the functionn

s(~) = L'\jll~-~jIl2log II~-~jll,j=l

at the interpolation points {~i : i= 1,2, ... , n}. Therefore we seek a sufficientlyaccurate approximation, s say, to s such that the work of generating s is ofmagnitude n log n, and such that the calculation of S(~i) for a typical ~i requiresonly O(1og n) operations. The technique takes advantage of the smoothness of the

term {1I;£-;£jIl2log 11;£-;£jll : ;£ E n2} when;£ is bounded away from ;£j' Indeed,if .:T is a subset of the integers {I, 2, ... , n} such that the convex hull of the set

{.f.j : j E .:T} contains no other interpolation points, then it can be helpful toemploy the approximation

.€J(~) ~ L "\jll~-;£i112log II~-~jll,jeJ

where .f:J is a truncated Laurent expansion of the right hand side about a suitablepoint of the convex hull, ~ say. The error of this approximation is of magnitude1I.f.~JII-T-l when ~ is sufficiently far from.f.J' where r is the order of the truncatedexpanSIOn.

The method generates the coefficients of .€J for several sets.:T. In the ~Lrp,..gL~~,tcase, the dis2:~L~~tion<:>fdata points iSE~~:l~~Ei~~E1E:<?J:l~~~UIlit square, and arecursive procedure divlcIes-afCsqu-aresatlevel k into four equafsquares at levelk+l, where ,k is the index of the recursion. The procedure begins by letting theunit square be the only square at level k = 1, and it ends when each square atlevel k+ 1encloses ab~intYmQJ~t!()I!l?g!nts. Each set .:T contains the indicesof all the interpolation points in a square at some level, and we include every set.:T that can be constructed in this way. Thus the number of different sets is aboutn115, each interpolation point is in about log4(n/5) different sets, and the workto calculate all the coefficients of all the Laurent expansions is bounded above bya multiple of n log n if the order r is a constant.

8 !vI.J.D. Powell

n €= 10-6E= 10-10Direct

200

0.300.400.25400

0.680.991.001000

1.973.216.252000

4.437.8325.004000

9.7318.76100.0010000

27.3055.4 7625.00

Table 1:

Timings in seconds for a Laurent approximation to <I>l

When an estimate of the function value SC~i) is required, we ask whether theapproximation (2.6) is sufficiently accurate when .:J is the set at level one thatcontains all the indices {I, 2, ... , n}. Almost certainly the answer is negative,but this is a convenient way to begin a recursive procedure that provides s(;r;).Indeed, for each set .:J that occurs in the recursion, we either employ £.:T(~i) if itis accurate enough, or we introduce the four descendants of .:J at the next levelinto the procedure, except that the final level may have been reached. Then allterms of the sum (2.5) that have not yet been covered by Laurent expansions arecalculated explicitly. One can deduce that, if the required accuracy is a givenpositive constant, then usually the number of sets that occur is of magnitudelog n. Further, the number of terms that have to be treated explicitly tends tobe independent of n for large n. Thus the complexity of calculating S(;ri) by thismethod is only about O(1ogn).

A more sophisticated procedure is needed to achieve the required complexitywhen the distribution of the interpolation points is highly irregular. This problemis addressed by Powell (1992), and he gives details of a truncated Laurent expan­sion algorithm that calculates ~.lto prescribed accuracy for general positions ofthe interpolation points. He includes execution times of a Fortran implementationof the algorithm, when the points {~i : i= 1, 2, ... , n} are very close to a squaregrid and all components of.l are one. Some of his timings are shown in the secondand third columns of Table 1, where € is the prescribed tolerance on the error ofthe approximation s ~ s. Further, the last column of the table gives the times thatare required to compute all of the numbers {sC~J : i= 1,2, ... ,n} directly. Thecalculations were run on a Sparc 2 workstation in double precision arithmetic. Wesee that the Laurent expansion technique can reduce the work of each iterationof the conjugate gradient algorithm from O(n2) to about O(nlogn). Any savingsare rather slight, however, until n exceeds 1000.

Thin plate spline interpolation

3. Numerical experiments with pre-conditioners

9

The conjugate gradient algorithm would solve the system (2.2) in one iteration ifthe matrix Q were such that Q T <I>Q were a multiple of the unit matrix. Thusthe actual choice of Q is very important to the efficiency of the solution of theinterpolation equations. Therefore the purpose of this section is to investigatesome choices numerically. vVe employ two distributions of interpolation points

and three different right hand side vectors L these quantities being defined in away that allows an infinite sequence of values of n.

Specifically, n is always the square of an integer, in order that the first dis­tribution is a regular square grid that covers the unit square [0,1] x [0,1]. In theother distribution, the points {;fi : i= 1,2, ... , n} are given the coordinates

Xi = sin(i1l"jn) and Yi=1[1+sin(2i1l"jn)], i=1,2, ... ,n. (3.1)

Vve call these distributions R (regular) and C (curve), respectively. We see thatthe points (3.1) lie on a closed curve that is enclosed by and touches the four sidesof the unit square. Both of these kinds of distribution are of interest in practice.

We let the right hand sides have the components

(01)

(xy): Ii = XiYi

Ii = { 0,1,

X·<l1_ 2

Xi>l 2i= 1, 2, ... , n, (3.2)

(sine)

the entries on the left hand side being the names that we use to distinguish thesechoices. Thus the cases (xy), (01) and (sine) provide interpolation to an easysmooth function, a discontinuous function, and an oscillatory smooth functionrespecti vely.

We apply a straightforward implementation of the conjugate gradient algo­

rithm to the system (2.2). In order to describe it briefly, we let !:!:..kbe the valueof !!:.. at the beginning of the k-th iteration, while '£..1. denotes the correspondingresidual vector

'£..1. = Q Tt - Q T <PQ !:!:..k' k = 1, 2, 3, .... (3.3)

The calculation ends if the condition 11'£..1. II s. ACC is satisfied, where ACC is a givenpositive number. Otherwise, the k-th iteration chooses a search direction 511. E

nn-3, and it sets !:!:..k+1 to the vector !:!:..k+CikSlk, where the value of CYk is calculatedso that the new residual vector '£..k+1 is orthogonal to 11.' Specifically, the initialsearch direction is 11= '£..1' and, on all other iterations, !1k is a linear combinationof '£..k and 11.-1 that satisfies the conjugacy condition

(3.4)

10 M.J.D. Powell

ProblemACC= 10-6

ACC= 10-10

n=49

100196400n=49100196400

R(xy)

6213783193892206

R(Ol)

253371150 45108261715

R(sine)305186161 4472153401

C(xy)

93683175 4279190519

C(Ol)

3755116247491814451209

C(sine)397513726072132270709

Table 2:

Iteration counts for Householder Q (no diagonal scaling)

It is well known that, in exact arithmetic, the residual vectors {1:k : k = 1,2,3,...}

are mutually orthogonal, so termination occurs after at most n - 3 iterations.Further, for k ~ 2, each !!:.k is the value of l!:. that minimizes the strictly convexquadratic function

subject to the restriction that f.L is in the linear space that is spanned by theprevious residuals {1:1' 1:2' ... ,1:k:=-l}' Thus one can deduce the property

(3.6)

In practice, however, it is not unusual for the algorithm to run for more thann-3 iterations, due to the effects of computer rounding errors. Nevertheless, theinequality (3.6) is achieved on every iteration unless the matrix Q T ~ Q is veryill-conditioned. Therefore we expect termination to occur eventually for any valueof ACC.

Table 2 presents the numbers of iterations that are required by our implemen­tation when Q is defined implicitly by the Householder transformations of thethird paragraph of Section 2. We see that four values of n and two values of ACCwere tried for each of our six test problems. We see also that rounding errors arecausing severe inefficiencies in many of the ACC= 10-10 experiments.

Therefore we wish to promote some other ways of choosing the matrix Q T <I>Q.Ian Barrodale (private communication) recommends a simple technique that ishigh~y useful for the system (1.5). It is to apply a constant scaling factor through­out n2 so that the greatest of the distances {11~i-~jII: i, j = 1, 2, ... , n} is one, inorder that the corresponding element of <l> is zero. We see that this scaling makes

Thin plate spline interpolation

ProblemACC= 10-6

ACC = 10-10

n=49

100196400n=49100196400

R(xy)

33601172303977152308

R(Ol)387414228961149346898

R(sine)39771523084978170437

C(xy)

509518738457115223466

C(Ol)54109221460892245541441

C(sine)5711522346674132284739

Table 3:

Iteration counts for Householder Q (with diagonal scaling)

the change

11

(p.. = Ilx,-x'11210g Ilx,-x'll~J -~ -J -'-J a2(P .. + Ilx,-x'11210ga~J -~-J (3.7)

to each element of <1.>,where a is the scaling factor. Now the three n x n matriceswhose (i,j)-th elements are IliJ2, -2;rT;rj and II;rjIl2 are annihilated by post­multiplication by Q, post-multiplication by Q (or pre-multiplication by QT) andpre-multiplication by Q T, respectively. Therefore the change (3.7) only causesQ T (p Q to be replaced by a2 Q T (p Q, so it does not provide a suitable solution toour task.

Another technique that is sometimes helpful to the conjugate gradient methodis scaling the rows and columns of the matrix of the system, so that symmetryis preserved and so that the matrix of the new system has ones on its diagonal.Therefore we applied this scaling to the Q T <l? Q matrices that were generated inthe calculations of Table 2. The new numbers of iterations are reported in Table3. It seems, unfortunately, that nearly all of the inefficiencies have become worse,and not all of the loss is due to the fact that the termination condition becomesharder to satisfy when the scaling factors exceed one.

Therefore we give attention to the suggestion at the beginning of this section,that we should try to choose Q so that Q Tip Q is close to a multiple of the unitmatrix. Some guidance towards this objective is given by the identity (1.13).Indeed, following Dyn and Levin (1983) and Dyn, Levin and Rippa (1986),wecan pick the elements in the j-th column of Q to provide the approximation

n

~ Qtj g(bJ ~ 'J2g(;rj)(=1

(3.8)

12 M.J.D. Powell

(3.11)

(3.12)

when {g(~) : ~ E R2} is any smooth function. If both the i-th and j-th columnsof Q are chosen in this way, then we have the estimate

n n

(QT<pQ)ij = I:I:QkiQlj<P(~k-b)k=ll=l

n

~I:Qki ['V!2<p(~k-lL) ]y=x ~ ['V~2'V!2<p(~-lL)L=x, y=x' (3.9)k=l - -) - -"- -)

where <P is still the function (1.6). It follows from equation (1.13) that suchcolumns of Q may cause the resulting part of Q T <pQ to resemble a multiple ofthe corresponding part of the (n-3) x (n-3) identity matrix. In particular, weexpect I( Q T <pQ )ij I to be very small when ~j is far from ~i'

We applied this idea to our regular grid test problems. The well-known ap­proximation

gives a column of Q for every interpolation point that is i,nterior to the grid, wheref:.x and f:.y are the coordinate vectors in R2 and h is the grid size, but we scaledthese columns by h2 in order that the nonzero elements of Q are of magnitudeone. Further, if ~j is on the edge of the grid without being a corner point, thenwe let the j-th column of Q have the property

n

LQij 9(b) = g(~j-h§.t) - 2g(;£j) + g(:fj+hf:.t),i=1

where §.t is the coordinate direction that is parallel to the edge. Assuming thatthe interpolation points are ordered so that the corners of the grid are last, wedefine the final column of Q by the equation

n

I:Qln-3 g(~) = g(O) + g(.f.x+§.y) - g(§.:c) - g(§.y).i=l

Thus each of the columns of Q has only three, four or five nonzero elements, andit can be proved that they span the null space of P T in nn.

The number of iterations of the conjugate gradient method for this choice ofQ is given in Table 4. Further, even better results are obtained if we scale thecolumns of Q so that the diagonal elements of Q T <pQ are equal to one, as shownin Table 5. It is particularly promising that, in every case, the growth in thenumber of iterations as n increases is much slower than a multiple of n.

For the curve data, we try letting each column of Q have just four nonzeroelements, which are consecutive. Therefore we require the functionals

i+3

2:: Qij 9(b), j = 1, 2, ... , n-3,i=j

(3.13)

Thin plate spline interpolation

Problem

ACC=10-6ACC= 10-10

n=49

100196400n=49100196400

R(xy)

6121520 6152228

R(Ol)

2019253327253343

R(sine)

2229364428374860

Table 4:

Iteration counts when Q is derived from \7 2 (no diagonal scaling)

ACC=1O-6 ACC=10-1OProblem n=49

100196400n=49100196400

R(xy)

691213 6131720

R(Ol)

1415182120192328

R(sine)

1620232921263137

Table 5:

Iteration counts when Q is derived from V2 (with diagonal scaling)

13

to vanish when 9 is any linear polynomial, because this property is equivalent tothe requirement pTQ = O. Further, we scale the columns of Q so that again thediagonal elements of Q Tit> Q are all one. Since the points (3.1) lie on a strictlyconvex curve, it follows that, apart from the signs of the columns which areunimportant, these conditions define Q uniquely. Indeed, it is straightforward toshow that the nonzero part of the j-th column of Q is a multiple of the vector

sin ..J!.. [sin U+l)1I" + sin U+2)1I" + sin U+3)1I" + sin (3j+6)1I" ]2n n n n n

_ sin 311" [sin U+2)11" + sin U+3)11" + sin j1l" + sin (3j+5)11" ]2n n n n n

sin 311" [sin (j+3)1I" + sin j1l" + sin U+1)1I" + sin (3j+4)1I" ]2n n n n n

_ sin ..J!.. [sin j1r + sin U+l)1I" + sin U+2)1I" + sin (3j+3)1I" ]2n n n n n

(3.14)

We see that, when n is large, the ratios of these components are approximately

1 : -3 : 3 : -1 for most values of j. Therefore expression (3.13) can be regardedas a multiple of a third derivative of 9 along the curve, which we write in the form

(3.15)

14 M.J.D. Powell

ProblemACC= 10-6

ACC= 10-10

n=49

100196400n=49100196400

C(xy)

46122337727511784841437

C(Ol)

48137368784581805571726

C(sine)471373851146 602136041792

Table 6:

Iteration counts when Q is derived from curve points (no tri-diagonal scaling)

where c denotes the direction of the curve. Thus the "curve" analogue of the"regular grid" expression (3.9) is the approximation

(3.16)

Hence, as before, the smoothness of </> away from its singularity gives the propertythat I(QT<J> Q)iil is tiny when n is large and {fi is not close to ;!;..j'

The results in the "curve" case are given in Table 6, and we find that theyare bad. Indeed, the third derivatives of expression (3.16) seem to be far lessefficacious than the second derivative approximations that were employed earlier.Therefore we are going to develop a successful pre-conditioner from the view thatexpression (3.16) is analogous to the approximation

where 6i and 6j are forward difference operators that act on the indices i and j

respectively. Now, if the second derivative contributions to this approximation aresuitably large when i= j, then the difference operators cause the large elementsof Q T <P Q to occur not only on the diagonal but also adjacent to the diagonal.Therefore we apply the following technique, because it reverses the action of thedifference operators.

We let Q be the nx(n-3) matrix such that, for j = 1, 2, ... , n-3, the elements{Qej : .e=j,j+1,j+2,j+3} are the components of the vector (3.14), while all

other elements of Cd are set to zero. vVedefine 9.1 to be the multiple of ~1 thatsatisfies the condition 9.f<P!il = 1, where !Ij and ~j denote the j-th columns of Q

and Q respectively. Further, for j =2,3, ... , n-3, we define q. to be the multiple-)

Thin plate spline interpolation

ACC= 10-6ACC= 10-10Problem n=49

100196400n=49100196400

C(xy)

1729456822385888

C(Ol)

19335176244062102

C(sine)20325077244065101

15

Table 7:

Iteration counts when Q is derived from curve points (with tri-diagonal scaling)

of the vector 1j-(!1f-l <l>1)!1j-l that has the normalization!1f <l>!1j=1. The reasonfor these definitions is that the new matrix Q T <l>Q has ones on its diagonal andzeros adjacent to its diagonal, but Q is not formed explicitly because most of itselements are nonzero.

Instead the matrix Cd T <l>Cd is calculated, taking advantage of the structure ofQ. Then, for j =1, 2, ... , n-3, we let Uj be a very sparse (n-3)x(n-3) matrix suchthat the first j columns of the product Cd U1U2 ••• Uj are the same as the first jcolumns of Q, and, for j <n-3, the (j+1)-th column of the product is a multiple of

~i+l' Fortunately, it follows from the previous paragraph that (Uj )jj and (Uj)j HIare the only elements of Uj that may have to be different from elements of the(n-3) x (n-3) identity matrix. Further, it is easy to obtain the values of theseelements. Thus, by applying the formula

(3.18)

we generate Q T <I> Q in only O( n2) computer operations. Similarly, the right handside of the system (2.2) is constructed by pre-multiplying Cd Tf by the productU!-3U!-2'" U{. -

Table 7 shows the numbers of iterations of the conjugate gradient algorithmwhen it is applied to this new system of equations. We see that the tri-diagonalpre-conditioning provides an excellent improvement over the results of Table 6.Indeed, in every test problem of Table 7 the number of iterations is substantiallyless than n.

4. Conclusions

The numerical results of Tables 1, 5 and 7 suggest that the thin plate splineinterpolation equations can be solved in much less than O( n3) operations for

16 M.J.D. Powell

large n. There is a fundamental reason for this conclusion, which is eXplainedbelow because of its relevance to future research.

vVewrite the interpolant in the cardinal form

n

s(~) = Lfjfj(~),j=l

(4.1)

where each £j is independent of the function values {fi : i= 1,2, ... , n}. Therefore£j satisfies the conditions

(4.2)

and all the remaining freedom in £j is taken up by minimizing the functional V(£j),

where V is defined by equation (1.2). Thus the second derivatives of £j are nolarger than necessary to achieve the cqnstraints (4.2). Further, these constraintshelp l£j(;f) I to be small when ~ is far from !fj' Thus the contribution from 1;to s(~) in expression (4.1) is relatively small when ~ is in the convex hull ofall the interpolation points but is outside some neighbQurhood of ~j' A typicalneighbourhood of ~j might contain about twenty of the points {~i : i=1,2, ... , n}.Therefore, if the value of s(~) is required to moderate accuracy, then often, at leastin theory, one can ignore data at points that are remote from~. Further, we expectthe number of data that are important to s(~) to be practically independent ofn when n is very large. In other words, there is much implicit sparsity in theestimation of s(~) that may be accessible through the cardinal functions {fj :

j = 1,2, ... , n}. Thus the variational problem has some structure, which suggeststhat the solution of the interpolation equations need not be very expensive.

We ask whether the findings of Sections 2 and 3 are of much value in practice.Tables 5 and 7 suggest that a careful choice of Q can provide substantial gainsin efficiency when n ~ 100, but Table 1 indicates that the complications of theLaurent expansions are worthwhile only for n 2: 1000, approximately. Thereforeautomatic methods for pre-conditioning would probably be much more valuablethan the development of computer software that applies the Laurent technique.On the ot,her hand, many values of s(~) may be required after the coefficients{Aj : j = 1,2, ... , n} and {Co, Cl, C2} of the thin plate spline function (1.3) have beendetermined, and it is shown by Beatson and Newsam (1992) that this need can beanswered by extending the Laurent approximations to Taylor series expansions.Alternatively, a single bicubic spline approximation to s may be more convenientthan several Taylor series expansions. In this case it would probably be importantto choose enough knots to preserve the shape of s between interpolation points,because the shape seems to be of particular merit in many practical applications.

Two techniques are used to construct pre-conditioners in Section 3. Firstly,some columns of Q are derived from the approximation (3.8). Dyn et al (1986)

show that this technique is also useful when the interpolation points are in general

Thin plate spline interpolation 17

position. Secondly, expressions (3.11)-(3.14) are less ambitious, because theirpurpose is merely to generate columns of Q with few nonzero elements that arein the null space of P T, this condition being the equation

n

L. Qlj g(~) = 0, gE III,l=1

(4.3)

where III is the 3-dimensional space of constant and linear polynomials from n2to n. Sibson and Stone (1991) recommend a convenient way of satisfying equation(4.3) for general positions of the interpolation points. Specifically, they form theDirichlet tessellation of {;£i : i = 1,2, ... , n}, in order to express each interiorinterpolation point as a convex combination of its neighbours. Thus they obtainrelations of the form n

:£.j = L. (}lj ~ (4.4 )i=lli-j

for several values of j, where the coefficients {(}lj : £ E {1,2"H,n} \ {j}} arenonnegative and sum to one, and where (}lj is zero if ~ is not a neighbour of:£.j in the tessellation. Then they achieve the condition (4.3) by setting Qjj = 1and Qlj = -(}lj, £ f- j. Further, they present a special procedure for the casewhen :£.j is on the boundary of the convex hull of the interpolation points. Thusthey construct suitable matrices Q whose columns each have about seven nonzeroelements. Methods that require triangulations or tessellations are unattractive,however, when the interpolation points lie on a curve.

We recall from Table 6 that our first shot at choosing Q for the curve distri­bution (3.1) was unsuccessful. Further, if the curve had not been strictly convex,then the columns of Q might not have been linearly independent. For example,suppose that :£'2, !f.3 and ;£4 lie on a straight line and that :£.1 and :£.5 are not onthis line. Then, if only {Qel : R.= 1,2,3,4} and {Qe2 : R.= 2,3,4, 5} are allowedto be nonzero in the first and second columns of Q, respectively, it follows that

both Qn and Q52 are zero and that 9.2 is a multiple of 9.1' where 9.j is still the j-thcolumn of Q. It might be very worthwhile, therefore, to give more attention tothe problem of choosing Q automatically.

The second paragraph of this section suggests another way of solving the in­terpolation equations efficiently when n is large. It is to form approximations,{Ij : j = 1,2, ... ,n} say, to the cardinal functions {£j : j = 1,2, ... , n} of ex­pression (4.1), in order to calculate s by an iterative refinement procedure thatcorrects the errors of the approximations. It should be adequate to define each Ij

by minimizing V(lj) subject to the equations {Ij(:£.i) = Oij : i E Jj}, where Jj isa carefully chosen subset of {I, 2, ... , n}. Typically, Jj might include the indicesof about ten interpolation points from a neighbourhood of :£.j and about four dis­tant interpolation points. Thus the amount of work to generate the coefficientsof each Ij would be independent of n, and perhaps the number of iterations of

18 lVI.J.D. Powell

the refinement procedure would also be independent of n for most distributions ofinterpolation points. The author is investigating these ideas with Rick Beatson.

Our numerical results and remarks indicate that it may be possible to enjoy

the advantages of thin plate splines in applications that require interpolation tothousands of values of a function of two variables. The author intends to providesome Fortran software for this task.

References

1. Barrodale, R. Kuwahara, R. Poeckert and D. Skea (1992), "Processing side­scan sonar images using thin plate splines", presented at the NATOAdvanced Research Workshop on Algorithms for Approximation (Oxford,July, 1992).

R.K. Beatson and G.N. Newsam (1992), "Fast evaluation of radial basis func­tions: I", Comput. Math. Applic., Vol. 24, pp. 7-19.

J. Duchon (1977), "Splines minimizing rotation-invariant seminorms in Sobolevspaces", in Constructive Theory of Functions of Several Variables) LectureNotes in Mathematics 571, eds. W. Schempp and K. Zeller, Springer-Verlag

(Berlin), pp. 85-100.

N. Dyn and D. Levin (1983), "Iterative solution of systems originating from in­tegral equations and surface interpolation", SIAM J. Numer. Anal., Vol. 20,pp. 377-390.

N. Dyn, D. Levin and S. Rippa (1986), "Numerical procedures for surface fittingof scattered data by radial functions", SIAM J. Sci. Statist. Comput., Vol. 7,pp. 639-659.

R. Franke (1982), "Scattered data interpolation: tests of some methods", Math.V Comp., Vol. 38, pp. 181-200.

G.H. Golub and C.F. Van Loan (1983), Matrix Computations, The Johns HopkinsUniversity Press (Baltimore).

M.J.D. Powell (1992), "Truncated Laurent expansions for the fast evaluation ofthin plate splines", Report DAMTP 1992jNAI0, University of Cambridge.

M.A. Sabin (1980) "Contouring - a review of methods for scattered data", in~Mathematical Methods in Computer Graphics and Design, ed. K.vV. Brodlie,Academic Press (London), pp. 63-86.

R. Sibson and G. Stone (1991) "Computation of thin plates splines", SIAM J.Sci. Statist. Comput., Vol. 12, pp. 1304-1313.