University of Cambridge Department of Physics ...rachael/compphys/CompPhys2.pdf · 2 Computational Physics 1.1 Interpolation and Extrapolation Let us consider a set of N data points

University of CambridgeDepartment of Physics

Computational Physics

Handout 2The Physics of Computational Physics

Dr. Rachael Padman

Michaelmas 2007

ii

Contents

1 Working with Data: Interpolation, Transforms and Fitting 11.1 Interpolation and Extrapolation . . . . . . . . . . . . . . . . . . . . . . . . 2

1.1.1 Linear Interpolation . . . . . . . . . . . . . . . . . . . . . . . . . . 21.1.2 Spline Interpolation . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.1.3 Extrapolation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

1.2 Transforms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81.2.1 The Fourier Transform . . . . . . . . . . . . . . . . . . . . . . . . . 81.2.2 Zero filling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101.2.3 Uses of Discrete Transforms . . . . . . . . . . . . . . . . . . . . . . 10

1.3 Fitting Models to Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121.3.1 Fitting Data to a Straight Line . . . . . . . . . . . . . . . . . . . . 121.3.2 General Least-Squares Fitting . . . . . . . . . . . . . . . . . . . . . 131.3.3 Minimising and Maximising Functions . . . . . . . . . . . . . . . . 141.3.4 More than One Dimension . . . . . . . . . . . . . . . . . . . . . . . 161.3.5 Worked example using NAG . . . . . . . . . . . . . . . . . . . . . . 18

2 Ordinary Differential Equations 212.1 The Euler Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222.2 Runge-Kutta Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242.3 Adaptive Step Size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252.4 Scaling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 272.5 Two-point Boundary Problems . . . . . . . . . . . . . . . . . . . . . . . . 292.6 Methods for Specific Problems . . . . . . . . . . . . . . . . . . . . . . . . . 30

3 Simulation I: N-body Simulations 313.1 Simulations of Galaxies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

3.1.1 Basic structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323.1.2 Force Law . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323.1.3 Gas Particles and Collisions . . . . . . . . . . . . . . . . . . . . . . 333.1.4 Initial Conditions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

3.2 Molecular Dynamics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353.2.1 The Force Equations . . . . . . . . . . . . . . . . . . . . . . . . . . 363.2.2 Simulations in Different Ensembles . . . . . . . . . . . . . . . . . . 37

iii

Contents

3.3 Integration of the Equations . . . . . . . . . . . . . . . . . . . . . . . . . . 383.4 Finding Neighbours . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 393.5 Evaluating the Gravitational Force . . . . . . . . . . . . . . . . . . . . . . 43

4 Linear Algebra and Eigen Problems 454.1 Brief Review of Matrices and Matrix Operations . . . . . . . . . . . . . . . 45

4.1.1 Definition and Basic Operations . . . . . . . . . . . . . . . . . . . . 464.1.2 Advanced Aside . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 474.1.3 Inverse . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 474.1.4 Transpose and Orthogonality . . . . . . . . . . . . . . . . . . . . . 484.1.5 Similarity Transformations . . . . . . . . . . . . . . . . . . . . . . . 484.1.6 Complex Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . 484.1.7 Linear Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 494.1.8 Eigen Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

4.2 Shapes of Matrices and Choice of Routine . . . . . . . . . . . . . . . . . . 494.3 Applications of Linear Algebra . . . . . . . . . . . . . . . . . . . . . . . . . 53

4.3.1 Diffusion in a Porous Material: Tortuosity . . . . . . . . . . . . . . 534.3.2 Normal Modes of a Material and Spectroscopy . . . . . . . . . . . . 564.3.3 Quantum Mechanics . . . . . . . . . . . . . . . . . . . . . . . . . . 58

5 Simulation II: Abstract Simulation 635.1 Cellular Automata . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64

5.1.1 Ising Model of Ferromagnetism . . . . . . . . . . . . . . . . . . . . 655.1.2 The Annealing Rule . . . . . . . . . . . . . . . . . . . . . . . . . . 665.1.3 The HPP rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

5.2 Percolation and Cluster Animals . . . . . . . . . . . . . . . . . . . . . . . . 685.2.1 Forest fires and Percolation . . . . . . . . . . . . . . . . . . . . . . 685.2.2 Cluster Animals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

5.3 Cluster Animals and Monte Carlo Sampling . . . . . . . . . . . . . . . . . 725.3.1 Configurations of a Polymer Chain . . . . . . . . . . . . . . . . . . 72

Acknowledgement:This handout was originally prepared by Dr. Paul Alexander, Michaelmas Term 2000, andhas since been updated and amended by Dr. Peter Haynes.

iv

Chapter 1

Working with Data: Interpolation,Transforms and Fitting

We begin our discussion of computational physics by considering a number of topics whichfall into the general category of “working with data”. By data in this context we mean aset of data points of the form (xi, yi), forming a one-dimensional “vector” of N values {yi}obtained at a set of N data points {xi}. These data may be experimental data, or datavalues generated within a program. The important point is that the data are availableonly at certain values of x, which may or may not be equally spaced.

There are very many different ways in which you may wish to treat these data. Forexample, if they are experimental data you may wish to analyse them using the sorts oftechniques discussed last year in the “Experimental Methods” course to determine variousstatistical information (mean, standard deviation etc.). Implementing such techniques isrelatively straightforward and we shall not discuss them further here (indeed they may alsobe carried out in packages such as Excel).

There are, however, many other computational techniques which a physicist may wish toemploy when handling data. This is (as are many of the subjects we shall discuss), a verylarge topic, and there are complete texts devoted to the topics of signal processing (mainly1D data) and image processing (2D and some 3D data). Here we shall concentrate onlyon 1D data and look at three very important ideas which have general applicability:

1. Interpolation: estimation of data values between the tabulated points.

2. Transforms: applying transforms to the data, in particular Fourier transforms.

3. Fitting a model to the data: this is closely related to the topic of finding the minimumof a function.

1

2 Computational Physics

1.1 Interpolation and Extrapolation

Let us consider a set of N data points of the form (xi, yi). These data are discrete, i.e.we only have values {yi} at a few tabulated points {xi}. The data may be experimentaldata, output of a program or a representation of some unknown function. If however webelieve the data to be “samples” of some underlying continuous distribution then it is avalid question to ask how we can estimate the value of this distribution at some generalpoint x. Last year in the “Experimental Methods” course you discussed sampling in somedetail and you should revise this material for completeness; although it is not needed tounderstand the material here, it is closely related.

We shall now consider some ways of solving this problem.

1.1.1 Linear Interpolation

The simplest solution to this problem is to use what is called linear interpolation betweenadjacent points. First we identify adjacent points xi and xi+1 such that the point at whichwe seek a value satisfies xi < x < xi+1.

y

xx

y

Figure 1.1: Left: illustration of linear interpolation. Right: the resulting piece-wise continuousfit to the data.

We obtain an estimate of the value, y, of the continuous function at the point x by fitting

1. Working with Data: Interpolation, Transforms and Fitting 3

a straight line between the adjacent points to give:

y = yi +yi+1 − yi

xi+1 − xi

(x− xi) .

If we imagine doing this for all points x1 ≤ x ≤ xN then we obtain a complete representationof the function. We call the function used to do the interpolation between the points, inthis case a straight line, the interpolating function. Over the complete range of the data theinterpolating function is built up of many straight line segments joining adjacent points.

1.1.2 Spline Interpolation

In many cases straight line interpolation is a reasonable approximation to obtaining anestimate of the true function, and it is certainly both easy to implement and efficient interms of the computer time used in calculation. However the function has some unpleasantproperties:

• The function is only piece-wise continuous.

• The first derivative of the function is discontinuous at each tabulated point.

• Second and higher derivatives are not defined at each tabulated point.

If we have reason to believe the underlying function is smooth then we can ask whetherwe can invent an interpolation scheme which produces a smoother result. Now a straightline is an example of a polynomial. A polynomial of order m such as Pm =

∑mk=0 akx

k

will require m + 1 values to determine the coefficients. One could therefore fit up to an(N − 1)’th order polynomial to our N data points; although the result would be veryunsatisfactory. We would find that the fitted function, while smooth and going through allof the points, oscillated wildly between the tabulated data values. A much better approachis to use a lower-order polynomial and fit it again piece-wise to the data for each pair ofadjacent tabulated points. We can then adjust the polynomial coefficients to constrain theresulting piece-wise function to be as smooth as possible.

A common choice is to use a cubic polynomial or spline; this has four coefficients to bedetermined. We obtain smoothness by demanding not only that the function goes throughfour points, but also that adjacent regions of the interpolation function have the propertythat their first and second derivatives are continuous.

We define the interpolating function, a cubic spline Pi between points xi and xi+1 in thedata, according to:

Pi(x) =3∑

j=0

aijxj for xi ≤ x ≤ xi+1, i = 1, 2, . . . (N − 1)


We must find in total 4(N − 1) coefficients since there are N − 1 steps between the firstand last tabulated points. This is done as follows:

1. The polynomial must pass through the points (xi, yi) and (xi+1, yi+1):

⇒ Pi(xi) = yi, Pi(xi+1) = yi+1, i = 1, 2, . . . (N − 1)

[2(N − 1) equations]

2. First and second derivatives are continuous at the tabulated points:

⇒ P ′i−1(xi) = P ′

i(xi), P ′′i−1(xi) = P ′′

i(xi), i = 2, . . . , (N − 1)

[2(N − 2) equations]

3. Apply boundary conditions to the first or second derivatives at the end points x1

and xN . A natural spline is one for which the second derivatives vanish at the endpoints:

⇒ P ′′1(x1) = 0, P ′′

N−1(xN) = 0

[2 equations]

We could write a routine to solve these equations, but in fact the work has been done forus. The NAG library provides a routine called E01BAF. As this is the first NAG routinewe have discussed we will look in a little detail at a program which uses this routine.

Some things to note:

(i) NAG routines require double precision for all real variables. Failure to supply doubleprecision values will cause the routine to crash or produce nonsense.

(ii) We do not need fully to understand the details of how the NAG routines store theinformation about the spline, as we will use another NAG routine to evaluate thespline when needed.

(iii) NAG routines often need work arrays. You must check the documentation carefullyto make sure you supply the size of work array required.

(iv) Almost all NAG routines have an ifail parameter which is used to communicateerror information. It is essential you check its value on exit from the NAG routine ifthere is any possibility of failure.

The following simple program illustrates the use of the NAG library:


program naginfo

use nag_f77_a_chapter

implicit none

! output information about the NAG library

write(*,*) ’Calling NAG identification routine:’

call a00aaf

end program naginfo

The use statement tells the compiler about some of the routines called by the programwhich are contained in another file (chapter A of the NAG library in this case) rather thanin our program. The line is not compulsory, but the compiler will be able to check for moreerrors if it is present, so you should always include it. If the above program is contained inthe file ex.f90 then it can be compiled with the NAG routine included and then executedas follows:

spqr1@pcphy117:~> f95 -o ex ex.f90 -lnag

spqr1@pcphy117:~> ./ex

Calling NAG identification routine:

*** Start of NAG Library implementation details ***

Implementation title: Linux (Intel) NAGWare f95 (RedHat 7.x)

Precision: Double

Product Code: FLLUX20D9

Mark: 20B

*** End of NAG Library implementation details ***

Returning to our implementation of cubic spline interpolation, the documentation for theNAG library is available on the course web site. In summary, for E01BAF it gives thefollowing specification:

subroutine E01BAF(m, x, y, lambda, c, lck, wrk, lwk, ifail)

integer m, lck, lwrk, ifail

real x(m), y(m), lambda(lck), c(lck), wrk(lwk)

where real always means the double precision real type.

m is the number of data points with values x(m) and y(m). The results of the spline fittingprocedure are stored in the arrays lambda and c each of size lck=m+4. The NAG routinealso requires some workspace in the form of the array wrk of size lwk=6*m+16.


The ifail parameter must be set before the NAG routine is called (zero is a safe choice)and it determines how the routine behaves if an error occurs, and may contain errorinformation when the routine returns.

We do not need to understand the details of the information returned in the lambda andc arrays since another routine E02BBF is provided which finds the value of the fit yv at agiven location xv:

subroutine E02BBF(lck, lambda, c, xv, yv, ifail)

integer m, ifail

real lambda(lck), c(lck), xv, yv.

Even if you do not yet understand all the details, you should be able to follow the exampleprogram on the following page (which can be downloaded from the course web site).

Other NAG routines are available for interpolation. In particular the routine E01BEFimplements interpolation using cubic Hermite polynomials; this is very similar to usingcubic splines. Further routines exist to evaluate the interpolated function and its derivativesand integrals.

1.1.3 Extrapolation

The process of extrapolation is closely related to interpolation; it is simply estimatingvalues beyond the range of the supplied tabulated data. In general this is a very riskybusiness and therefore should be avoided if at all possible.


! program to demonstrate NAG routines for cubic spline interpolation

program test_spline

use nag_f77_e_chapter

implicit none

! define sizes of arrays

integer, parameter :: m = 20

! size of arrays used by NAG to store spline

integer, parameter :: lck = m + 4

! size of NAG work array

integer, parameter :: lwk = 6 * m + 16

! find the kind of a high precision variable, by finding kind(1.0d0)

integer, parameter :: dp = kind(1.0d0)

integer :: i, ifail

! NAG requires high precision variables

real(dp) :: x(m), y(m), &

lambda(lck), c(lck), work(lwk), &

xv, yv

! set up a model function

do i = 1, m

x(i) = (i - 1) * 0.1d0

y(i) = sin(x(i)) * cos(x(i))

end do

! fit spline using NAG routine: if ifail == 0 on entry, then NAG will

! print an error message and stop the program if an error occurs

ifail = 0

call E01BAF(m, x, y, lambda, c, lck, work, lwk, ifail)

! as a test print out a value of the interpolated function

! use NAG routine to evaluate the spline

xv = 0.31372d0

ifail = 0 ! strictly unnecessary here, but good practice...

call E02BBF(lck, lambda, c, xv, yv, ifail)

write(*,*) ’At xv = ’, xv, ’, interpolated value = ’, &

yv, ’, function = ’, sin(xv) * cos(xv)

end program test_spline


1.2 Transforms

You are already familiar with the importance of various forms of transforms in physics, inparticular Fourier transforms. This still holds true in the field of computational physics.In this section we aim to do two things: firstly introduce the idea of a numerical Fouriertransform (or more strictly Fourier series), and then look at uses of Fourier techniqueswhich are more specific to computational problems, and especially to data processing.

Once you have mastered the use of Fourier transforms you will of course be able to usesimilar techniques to perform a variety of other transforms. In all cases we will use libraryroutines to do the hard work, and in particular the NAG library once again.

1.2.1 The Fourier Transform

The Fourier transform pairs can be written in the form:

H(f) =∫ ∞

−∞h(t) exp(−2πift)dt

and for the inverseh(t) =

∫ ∞

−∞H(f) exp(2πift)df.

However, in practice we will only be able to obtain a sample over a finite interval of time0 ≤ t < T . If we assume that our function repeats itself with period T , then we can usethe results for Fourier series. In this case, the Fourier transform consists of components ata discrete set of equally-spaced frequencies {fn} given by:

fn =n

T

where n is an integer. As the sampling interval T increases, the frequencies {fn} becomemore closely spaced, and we approach the limit of the Fourier transform. Denoting thecomponent of the Fourier series at frequency fn by Hn we obtain:

Hn =∫ T

0h(t) exp(−2πifnt)dt =

∫ T

0h(t) exp(−2πint/T )dt,

h(t) =1

T

∞∑

n=−∞Hn exp(2πifnt) =

1

T

∞∑

n=−∞Hn exp(2πint/T ).

Now we wish to approximate this transform for discrete data. To do this we consider ourinput data to be sampled at discrete values (or provided at discrete time intervals):

tk = k∆, k = 0, 1, . . . N − 1


for N data values (T = N∆). Note that the time steps are all equally spaced. This samplingimposes a limit upon the maximum frequency for an oscillating function or signal that canbe properly represented: the Nyquist frequency which is given by:

fc =1

2∆.

Our sample now contains components up to this maximum frequency i.e.

fn =n

T=

n

N∆for n = −N/2, . . . N/2.

Note that the limits correspond to the Nyquist frequency. Defining hk = h(tk), the discreteFourier transform Hn ≈ ∆Hn is given by:

Hn =N−1∑

k=0

h(tk) exp(−2πifntk)

=N−1∑

k=0

hk exp(−2πikn/N).

Now since n is periodic with period N (check you can show that Hn+N = Hn), we can infact think of n running from 0 to N − 1, and in this case we have:

• zero frequency is for n = 0,

• positive frequency is for 1 ≤ n ≤ N/2,

• negative frequency is for N/2 + 1 ≤ n ≤ N − 1.

The inverse transform is given by:

hk =1

N

N−1∑

n=0

Hn exp(2πikn/N).

To evaluate these sums we again turn to routines from the NAG library. The routineC06ECF evaluates the following:

Zn =1√N

N−1∑

k=0

zk exp(−2πikn/N)

Note therefore that we must scale the output of this routine if we require a quantitativeestimate of the Fourier Transform. This particular routine takes a set of complex data asinput and outputs a set of complex data. In many cases the type of input data, or othersymmetries of the data, will enable specific forms of the transform to be exploited, and


NAG provides a large number of routines for specific cases which are likely to be moreefficient than this general purpose routine.

Using this routine is very easy. If the real arrays (of kind=dp) holding the real andimaginary parts of the input sampled data are x and y respectively, and have length n,then the transform is found by the subroutine call:

call C06ECF(x, y, n, ifail)

and the inverse transform is easily accomplished by taking the complex conjugate beforeand after calls to the same routine; using another NAG routine to perform the conjugation:

call C06GCF(y, n, ifail)

call C06ECF(x, y, n, ifail)

call C06GCF(y, n, ifail)

These examples have shown only one-dimensional transforms. It is possible to perform thetransforms for 2D and even 3D data, and library routines exist to help accomplish thisalso.

1.2.2 Zero filling

If you implement the above transform on some data you are likely to find that the trans-formed function looks somewhat coarse since it is only sampled at frequency points spacedby 1/(N∆). We can employ a simple method of improving the sampling of the transformeddata, although it adds no more information to the transform. This technique is called zerofilling. Suppose we have data in arrays of length N . We can double the frequency sam-pling by performing the transform on arrays of length 2N say, filling these new arrays fromthe (N + 1)’th element to the 2N ’th element with zeros. This procedure is common andprovides a sampling of the Fourier space data which is greater than the Nyquist samplingprovided without zero filling. Of course this procedure does not improve the sampling ofthe original data.

1.2.3 Uses of Discrete Transforms

How can we use discrete transforms? The short answer is in the same way as your use ofFourier transforms in any physics problem. For example one could easily write a programto compute the Fraunhofer diffraction pattern for an arbitrary aperture by supplying the


aperture function in numerical form and performing a discrete transform. Other examplesof the use of the discrete Fourier transform are more specific to the analysis of data andare worth mentioning briefly.

1. Smoothing of noisy data. We can make use of the fact that the multiplication inFourier space can be used to obtain a convolution in real space:

f ∗ g = FT−1(FG) where F = FT(f) and G = FT(g)

In this case a good smoothing function in real space might be a Gaussian. Thiscan be implemented numerically by taking the Fourier transform of the input dataand multiplying by an appropriate Gaussian function in Fourier space (the FT ofa Gaussian is a Gaussian and is known analytically). Taking the inverse Fouriertransform gives back the smoothed data. Care must be taken when dealing with thenegative and positive frequencies in the transform. The procedure is often given theterm apodization, and many functions other than Gaussians are used.

2. A related topic is to apply high- and low-pass filters to the data to remove noise. Thiscan amount simply to multiplying the transform by a simple top-hat like function.

3. Another class of problems concerns spectral analysis. Often experimental data areobtained as a time sequence and from these data we wish to estimate some form ofspectrum. This is accomplished numerically by taking the Fourier transform. Oftenwe are not interested in the phase of the resulting spectral signal, only its amplitude,and therefore we can form what is called the power spectrum |Hn|2 to eliminate thephase altogether.

4. A final class of problems concerns the search for possible periodic information in adataset, or to see if there is some specific time-constant hidden within the data. Apowerful tool is then the “correlation”. This is closely related to convolution. Wesimply compare one function against another with a “lag”, τ . The correlation is largewhen the functions, displaced by the lag, are similar to one another. Mathematicallythis looks like:

corr(g, h; τ) =∫ ∞

−∞g(t + τ)h(t)dt

= FT−1(GH∗)

i.e. the correlation can be written in terms of the inverse Fourier transform of theproduct of the FT of one function with the complex conjugate of the FT of theother. This resembles closely the convolution theorem and the proof is very similar.This can therefore easily be implemented using discrete transforms. When the twofunctions are the same we call this an auto correlation, and the auto correlation is


then simply the inverse Fourier transform of the power spectrum of the input data:

corr(g, g; τ) =∫ ∞

−∞g(t + τ)g(t)dt

= FT−1(|G|2

)

1.3 Fitting Models to Data

A very important task we are likely to have to perform is to fit some form of a model to aset of data. The model is likely to be some analytical function or perhaps even the resultof a numerical calculation of a physical model. The task is to ask how well does the modelfit some given data, and usually to estimate some input parameters of the model. In thissection we will give some consideration to this problem; however this is a very large topicindeed and it is impossible to discuss this in great detail. Instead we will concentrate onone topic, namely the use of a single statistical test of how well a model fits some data,the so-called Chi-squared (χ2) test. We shall start by discussing a familiar problem: thestraight line fit. Again, you met these ideas last year in the experimental methods course.

1.3.1 Fitting Data to a Straight Line

Suppose again we have N data points (xi, yi), and associated with each data point is anerror, σi. To these data we wish to fit a straight line of the form:

y = a + bx.

An appropriate measure of the goodness of fit of this model to the data is the χ2 statistic:

χ2 =N∑

i=1

(yi − a− bxi

σi

)2

.

If the errors are normally distributed this will give a “maximum likelihood” estimator forthe model parameters; if the errors are not normally distributed (or we don’t know, as isusually the case) then this can be regarded as a useful practical measure of the goodnessof fit. For the case of normally distributed errors we will be able to obtain a statisticalsignificance for the resulting fit; in other cases other statistical tools are needed to give suchestimates, such as the Bayesian or other non-parametric techniques which were mentionedbriefly last year.

In the present context we will simply regard χ2 as a good measure of how well the modelfits. Notice the form of this equation; we look at the squared difference between model


prediction and data, and weight these estimates by the inverse of the squared error. Our jobis now to minimise this measure to obtain estimates for the best fitting model parameters.Given the form of the problem we often refer to this method as “least squares fitting”. Forthe case of a straight line we can proceed analytically by differentiating χ2 with respectto each model parameter, a and b, obtaining two linear equations to solve by setting thesepartial derivatives equal to zero:

0 =∂χ2

∂a= −2

N∑

i=1

yi − a− bxi

σ2i

0 =∂χ2

∂b= −2

N∑

i=1

xi(yi − a− bxi)

σ2i

Solution of these equations gives:

a =SxxSy − SxSxy

∆

b =SxyS − SxSy

∆

where:∆ = SxxS − (Sx)

2,

and:

S =N∑

i=1

1

σ2i

, Sx =N∑

i=1

xi

σ2i

, Sy =N∑

i=1

yi

σ2i

,

Sxx =N∑

i=1

x2i

σ2i

, Sxy =N∑

i=1

xiyi

σ2i

.

It is a relatively straightforward task to implement these results. Of course it has alreadybeen done and routines are available in the NAG library.

Unfortunately it is not possible to generalise this analytical treatment. Even the compara-tively simple problem of having error bars in both coordinates significantly complicates theanalysis, and for polynomial-type models the equations immediately become non-linear.We need to look at more general methods.

1.3.2 General Least-Squares Fitting

We can easily generalise the formalism at the start of the previous section to consider anyform of model function as follows:

χ2 =N∑

i=1

(yi − f(X1, . . . XM ; xi)

σi

)2

,


where the model is represented by the function f(X1, . . . XM ; x) which is itself a functionof the model parameters X1, . . . XM and is evaluated at the point x. Our task is to findthe minimum of χ2 so as to determine the model parameters at this minimum which givethe best fit of the model to the data. Computationally the problem may be an extremelydifficult one to solve. The problem in fact reduces to one of minimising some function, inthis case χ2, which can be thought of as a function of some number of variables, in thisexample the M variables X1, . . . XM . A further complication may be that the physicalmodel imposes constraints on the possible values of the model parameters. At this pointour discussion of model fitting has turned into one of minimisation of a function, albeitone of a fairly generic form. We shall therefore conclude this first section by an importantdiscussion on the minimisation of functions. Our aim is not to discuss particular techniquesin detail (this quickly becomes complex numerical analysis), but instead to introduce thebasic ideas.

1.3.3 Minimising and Maximising Functions

In this section we will discuss in general terms the problem of minimising or maximising afunction. For brevity we will consider the problem of minimisation, since maximisation isclearly closely related. The general problem can be stated as:

min (F (X1, X2, . . . XM))

i.e. finding the set of M parameters, X1, X2, . . . XM , which minimises the value of thefunction F . In general we seek a global minimum; however the parameters may haveadditional constraints, which may be linear and take the form of equalities or inequalities(e.g. X1 + 2X2 + X3 = 0; or X1 + X2 > 0), or non-linear. We can write the constraints inthe form:

Ci(X) = 0 or Ci(X) > 0

where the Ci are the constraint functions.

To introduce the problem let’s consider a simple case in 1D (i.e. one variable), and a singleglobal minimum: In general methods of solution fall into two classes:

• Methods in which only the function is evaluated.

• Methods in which function and derivatives are evaluated.

The first class of methods is illustrated by the left-hand panel of Fig. 1.2 which shows themethod of bracketing. At each step we have three points which bracket the minimum; therule is that the middle point is lower than the outer two. For example we start with points1, 2, and 3. We then look at the point midway between two of them, say 2 and 3, and take


4

F

X 1

F

X 1

2

3

5

1

6

Figure 1.2: Left: minimisation of a 1D function without derivatives by the bracketing method.Right: how knowledge of derivatives aids minimisation.

it as the next outer or middle point depending on which existing point it is lower than. Inthis case the outer point becomes point 4. We then repeat with 3 and 1 and find a new leftouter point 5, then repeat with 3 and 4 and find a new right outer point 6. The procedurecontinues until convergence.

The second class of methods makes use of the gradient to give an indication of whichdirection is “downhill”, as in the right-hand panel of Fig. 1.2. Starting from general pointsand following the gradient we can therefore in general move towards the minimum.

What happens if there are many local minima in addition to the global minimum, as inFig. 1.3?

How do we find the global minimum? Two possible approaches are:

• Locate minima starting from many different points and accept the smallest of these.

• Find a minimum and then take a big step away from the minimum and see if anotherminimum is found which is lower.

The algorithms for such approaches are often rather ad hoc. One relatively new techniqueis called simulated annealing, which deliberately attempts to treat the problem in a morephysical way by introducing a probabilistic element which has the form of a Boltzmannfactor. The function to be minimised plays the role of energy, and we imagine starting at


1

F

X

Figure 1.3: Distinction between the global minimum (filled circle) and local minima (opencircles).

some point and taking steps and evaluating the function again. Naively one might expect toaccept a step if the step led to a state of lower energy, but using the probability distributionwe also allow ourselves the possibility of accepting an energetically unfavourable step.As the minimisation proceeds we steadily reduce the “temperature” in the Boltzmanndistribution. Initially, at high temperature, we allow many unfavourable moves and therebyexplore the function; the hope is to explore enough of the function so that we get to theglobal minimum. As the temperature cools we are forced more and more only to allowmoves which are towards the (hopefully global) minimum.

1.3.4 More than One Dimension

The generalisation to more than one dimension follows the same general procedures out-lined for the 1D case. Importantly, methods still separate into those for which gradientinformation is known and those for which it is not. Now we have not just a single gradient,but a gradient vector:

g(X) =

[∂F

∂X1

,∂F

∂X2

, . . .

]T

.

For example if the function is:

F (X) = exp(X1)(4X2

1 + 2X22 + 4X1X2 + 2X2 + 1

),


then

g(X) =

[F (X) + exp(X1) (8X1 + 4X2)

exp(X1) (4X2 + 4X1 + 2)

].

If second partial derivatives of the function are available we will often need the Hessianmatrix. This matrix has elements Hij given by:

Hij =∂2F

∂Xi∂Xj

.

Figure 1.4 illustrates this sort of problem for a 2D function (in fact the one shown above).The point X∗ is a global minimum, but we may also have to consider constraints on thevariables. The “contours” are lines of constant function value.

Figure 1.4: Contour plot for the function F (X) = exp(X1)(4X2

1 + 2X22 + 4X1X2 + 2X2 + 1

).

To solve problems we once again turn to NAG. The choice of routine depends criticallyon what we know about the function we are trying to minimise, and the constraints. TheNAG documentation tries to lead you through such choices. One routine that it is worthnoting is called E04FYF. This attempts to solve the general non-linear problem withoutconstraints when only the function itself (and not its derivatives) are known. This is avery useful routine to use in problems of a reasonably straightforward nature.


1.3.5 Worked example using NAG

Here we consider the one-dimensional example of finding the maximum of the functionx exp(−x) using the NAG library. Consulting the documentation shows that the NAGlibrary has many routines for finding minima. Our problem is of course equivalent tominimising −x exp(−x).

A suitable choice of routine is E04ABF which searches for a minimum using function valuesonly. Of course, the NAG library does not know which function we wish to minimise, sowe need to be able to pass the function to the routine, in the form of a subroutine funct.The summary specification is:

subroutine E04ABF(funct, e1, e2, a, b, maxcal, x, f, ifail)

integer maxcal, ifail

real e1, e2, a, b, x, f

• e1 and e2 specify the accuracy required.

• a and b define the interval to search.

• maxcal is the maximum number of function evaluations allowed.

• ifail performs its usual role.

• On return, x will be set to the position of the minimum, and f to the function valuethere.

• funct is the subroutine containing the function to minimise. The NAG documenta-tion also specifies the form of this subroutine.

subroutine funct(xc, fc)

real xc, fc

If you read the NAG documentation yourself, you will see that it also says that funct mustbe declared as external in the program which calls E04ABF. While this is true for olderversions of the Fortran language, it is no longer necessary for Fortran95 when youinclude the use nag f77 e chapter line and put the subroutine to be passed in a module.The code on the following page shows how this could be implemented.

spqr1@pcphy117:~> cp $PHYTEACH/part_2/examples/minimize.f90 .

spqr1@pcphy117:~> f95 minimize.f90 -lnag

spqr1@pcphy117:~> ./a.out

Minimum value of -0.3678794411714423 at 0.9999999996840286

The numerical result compares well with the exact result of −1/e at x = 1.


! this module contains a subroutine which evaluates the function

! -x*exp(-x) which we wish to minimize

module x_ex_mod

contains

subroutine x_ex(x,f)

real(kind(1.0d0)) :: x,f

f = -x*exp(-x)

end subroutine

end module

! this program controls the minimization

program minimize

use nag_f77_e_chapter

use x_ex_mod

implicit none

integer :: ifail,maxcal

real(kind(1.0d0)) :: e1,e2,a,b,x,f

! define problem for NAG library

e1 = 0.0d0 ; e2 = 0.0d0 ! NAG routine will use default values

a = 0.5d0 ; b = 1.5d0 ! let’s restrict the search to 0.5 <= x <= 1.5

maxcal = 50 ! allow at most 50 calls to x_ex

ifail = 0

! call NAG

call e04abf(x_ex,e1,e2,a,b,maxcal,x,f,ifail)

! report results

if (ifail == 0) then

write(*,*) ’Minimum value of ’,f,’ at ’,x

else

write(*,*) ’NAG routine E04ABF failed with ifail ’,ifail

end if

end program


Chapter 2

Ordinary Differential Equations

We can often reduce a physical problem to a description in terms of ordinary differentialequations; i.e. equations with one independent and one dependent variable. In this sectionwe shall consider methods of solution for ordinary differential equations. Our problem ismade slightly simpler, from a computational point of view, because we can always reducesecond and higher order differential equations to a system of first order equations. Forexample by introducing the dependent variable z, the equation:

d2y

dx2+ q(x)

dy

dx= r(x),

can be written in the form:

dy

dx= z(x)

dz

dx+ q(x)z(x) = r(x).

These are coupled first order equations. Computationally, therefore, we need to consideronly the solution of first order equations. An important example of this kind which weshall return to later are Newton’s laws of motion:

md2x

dt2= F (x).

The reduction to first order equations is accomplished in this case by introducing themomentum, p:

p(t) = mdx

dt,

to obtain Hamilton’s equations:

dx

dt=

p

m;

dp

dt= F (x).

21


The general form of the first order equation that we shall consider can be written:

dy

dx= f(x, y),

and all that remains is to consider the boundary conditions. We will still have as manyboundary conditions as the order of the original differential equation, N , since we nowhave N coupled first order equations. The boundary conditions are of two general formswhich will relate to quite different computational problems:

• All boundary conditions are specified at the same point (e.g. we know z and p att = 0). These are called initial value problems and are the easiest to solve.

• Boundary conditions are given at different points, usually the extrema of the regionover which we are seeking a solution to the differential equation. Such conditionslead to what are called two-point boundary value problems.

We will mostly consider initial value problems and only discuss two-point boundary valueproblems at the end of this section.

2.1 The Euler Method

This is the simplest method of solution, but also the least accurate. The Euler method doeshowever illustrate the main features of the solution of ordinary differential equations andtherefore we shall consider it in some detail; we shall also introduce the idea of differenceequations.

Let’s consider the first order equation of the form:

dy

dx= f(x, y),

and we seek a solution to this equation in the range x0 to xf . To do this we break theinterval up into N steps of finite size h:

h =xf − x0

N; xn = x0 + nh.

Our first job is to re-write the differential in terms of values of y at the points {xi}. To dothis we first look at the Taylor expansion of y(xn + h):

y(xn + h) = y(xn) + hdy

dx

∣∣∣∣∣xn

+1

2h2 d2y

dx2

∣∣∣∣∣xn

+ O(h3),

2. Ordinary Differential Equations 23

where the symbol “big O”, O(g), means “terms of order g”. Re-arranging, we find:

dy

dx

∣∣∣∣∣xn

=y(xn + h)− y(xn)

h− 1

2h

d2y

dx2

∣∣∣∣∣xn

+ O(h3)

≈ y(xn + h)− y(xn)

h

where the error associated with the final line is O(h). Labelling the value of y at xn as yn

etc., we can write the original differential equation in difference form:

yn+1 = yn + hf(xn, yn).

x

y

y3

x x x x

y

y

y

0 21 3

2

0

1

Figure 2.1: Illustration of the Euler method.

The method of solution is then straightforward for an initial value problem. We use thevalues at xn to predict the solution at xn+1, i.e. yn+1. We then step through the entirerange to find the complete solution. The hope is that the step length is sufficiently smallthat significant errors do not occur.

There are a number of problems with this simple method:

1. Comparatively low accuracy for a given step size: errors in the Euler method areO(h2) because we neglect terms of order h2 in the Taylor expansion.

2. Not symmetric: we advance the solution by h, but only use information at one endof the step. This makes the solution unstable and propagates the error from one endof the integration range through to the other.


3. Overall the scheme is not particularly stable.

4. We have a fixed step size. In some regions the solution may be changing so quicklythat our step size leads to very inaccurate results and indeed the solution may diverge,whereas in others the step size may be so small that we waste computational cyclesnot advancing the solution as far as we could have done with a much larger step size.

2.2 Runge-Kutta Method

The Runge-Kutta method addresses many of the problems associated with the Eulermethod discussed in the previous section. In fact it is not one method, but a wholefamily of ways of solving the problem.

xx

0

x x20

y

y1

1

y2

y

Figure 2.2: Illustration of the Runge-Kutta method.

The idea is to use an Euler step as an initial guess to the midpoint of the interval, then makeuse of values at the midpoint to calculate the full step. This will then have the advantageof adding symmetry to the solution method; which leads to higher accuracy, and greaterstability. Figure 2.2 illustrates the method. Open symbols are at the midpoints and areused only for intermediate calculations.

The method can be written as:

k1 = hf(xn, yn)

k2 = hf(xn + h/2, yn + k1/2)


yn+1 = yn + k2 + O(h3)

The error has now improved significantly (h3 is much smaller for small h!). This hasresulted from the symmetrization which has led to cancellation of the second-order errorterms. In general we can extend this idea to higher order. We call a Runge-Kutta methodof “order n” if its associated error is O(hn+1). The method described above is calleda second-order Runge-Kutta, or commonly, a midpoint method. Higher order methodsrequire more calculations per step and it is therefore not clear for a given problem whethera high order method with large step size, or lower order method of smaller step size isbetter. We only really have two ways forward to make the decision, either to do a detailednumerical analysis of our particular problem, or to investigate different methods and doa quantitative comparison of them based on results as applied to our problem (perhapsslightly simplified). The latter is more often adopted.

Many physical problems are in fact well suited to a fourth order Runge-Kutta, which hasthe following form (we quote it only for completeness!):

k1 = hf(xn, yn)

k2 = hf(xn + h/2, yn + k1/2)

k3 = hf(xn + h/2, yn + k2/2)

k4 = hf(xn + h, yn + k3)

yn+1 = yn + k1/6 + k2/3 + k3/3 + k4/6 + O(h5)

Again we can use a NAG routine to do the hard work for us. There are a number ofroutines to deal with slightly different problems and requirements. The simplest routine isD02BJF.

2.3 Adaptive Step Size

Both the Euler and Runge-Kutta methods we have described have a fixed step size. Formany physical problems this immediately causes problems. An example of such a prob-lem is motion under an inverse-square central force (either gravity or electromagnetism),and especially when the problem resembles scattering. Let us consider the situation forRutherford scattering in Fig. 2.3.

The equations of motion are very easy to write down:

md2r

dt2=

Ze2

4πε0r3r

and the problem is characterised by an impact parameter b and an initial incident particlevelocity v. We take the central charge as fixed.


b

e, m

v

Ze

Figure 2.3: Diagram of Rutherford scattering.

As the particle approaches the central charge the curvature of the orbit demands that weuse a very much smaller time step than at large distances.

We require a routine which detects the need to change the step according to some criterion.We shall illustrate the method by reference to the fourth order Runge-Kutta mentionedin the previous section. What we require is an estimate of the error in the solution for agiven step length. One method of assessing this is to consider getting to the point x + 2hfrom the point x by taking either one step of 2h or two steps of size h. If we performedthis using the Runge-Kutta method we would obtain two estimates of the solution at thispoint which we shall call y1 and y2 respectively. The difference between these we shall callthe truncation error, ∆, and is a measure of the accuracy of the scheme:

∆ = y2 − y1

We now use this truncation error to estimate whether a change in step size is required.Suppose the required accuracy is ∆0. Since the accuracy of the Runge-Kutta method weare employing is O(h5), we can determine a modified step length according to:

hnew = h(

∆0

∆

)1/5

In this way the step length continuously adjusts as we step through the solution.

This is of course a simplified description of the algorithm, mainly due to the fact that foranything other than a single first order equation we must consider the step size in relationto the set of coupled first order equations we are actually solving. The implementation ofthe algorithm is also made more complex in a real application so as to avoid undertakingmultiple Runge-Kutta calculations for each step.

Adaptive step algorithms are the most important for the majority of physical problems.The NAG library provides a number of easy to use routines which you can employ.


2.4 Scaling

One question we have not addressed is what values should our variables in a numericalscheme actually be? We phrase our physical problem in terms of quantities which areusually related to a standard set of dimensions, SI. This means that in many problemsvariables would take on extreme values, either very small (e.g. quantum problems) orvery large (astrophysics). From a numerical standpoint this is a bad idea. Algorithmsare developed so that an “average value” for a variable would be ∼ 1. There is a goodreason for this, and that is that variables have limited numerical accuracy and workingwith variables which naturally take on either extremely large or small values may lead tonumerical instabilities.

The solution to this problem is to recast the physics in terms of dimensionless quantities.Not only does this help the numerical analysis, it also leads to a set of equations where thephysical scalings present are much clearer and are represented in terms of dimensionlessgroups and dimensional constants.

We shall look at one example and see how we can recast the problem in terms of dimen-sionless quantities. You will be meeting other examples in various courses this year.

First, consider a cloud of gas which has spherical symmetry. The gas is held together by itsown gravity, and its own pressure acts to support it against gravitational collapse. This isan important starting point for many astrophysical problems; for example stellar structureor the distribution of gas in clusters of galaxies. We can easily derive the equation whichdetermines this equilibrium state by considering a shell of gas between radii r and r + drshown in Fig. 2.4.

The excess pressure must balance the gravitational attraction:

4πr2drρM(r)G

r2= −4πr2dp

therefore:dp

dr= −ρM(r)G

r2.

In these equations M(r) is the mass inside a radius r, therefore we can also write:

dM

dr= 4πr2ρ.

These equations are subject to the following boundary conditions: the cloud is of constant,but unknown, temperature T , radius R0, and is embedded in an external pressure P0. Thisrequires that the surface pressure of the cloud is also P0.


p

r + dr

r

p + dp

Figure 2.4: Definition of variables for analysing a gas cloud.

From the ideal gas law:

p =kBT

µρ =

c2s

γρ

where cs is the sound speed, which is a constant since the gas has a constant temperature.Eliminating the density from the above equations we are left with:

dp

dr= −γpM(r)G

c2sr

2,

dM

dr= 4πr2γp

c2s

.

To write these equations in dimensionless form we first identify the constants and boundaryconditions, which are:

T, R0 and P0.

The variables in the problem are:

r, p and M.

Therefore we need to find quantities characteristic of this problem which have units ofdistance, pressure and mass. Clearly two of these are R0 and P0. For a characteristic massit is easy to show that the following has the appropriate units:

M0 =c2sR0

G

and the sound speed (and hence temperature) must satisfy:

R0 =c2s

(GP0)1/2

.


We can then introduce dimensionless variables r, p and M :

r = rR0; p = pP0; M = MM0

and with these definitions it is easy to show that the equations become:

dp

dr= −γ

Mp

r2;

dM

dr= 4πγ r2 p.

How does this help us? In two ways. First the equations to solve have dimensionless vari-ables which are sensibly scaled, and large and small values have real physical significance.Secondly we can use the dimensional constants we have introduced to see how the phys-ical results scale. Note that the dimensionless equations do not change as we change theboundary values. Hence the total mass of the cloud must be proportional to M0. In factto within a constant which is of order unity, the total mass of the cloud is M0. The reasonis that:

Mtot = M0γ∫ 1

04πr2 p dr,

and since everything is scaled, the integral is of order unity.

2.5 Two-point Boundary Problems

Not all problems can be phrased in terms of initial value boundary conditions. Indeed theexample we have just considered in the previous section is one such case. The boundaryconditions for this problem (in dimensionless coordinates) are:

p = 1 at r = 1,

M = 0 at r = 0.

The solution to problems of this kind is considerably more complicated. One method istermed “shooting”, and proceeds as follows:

1. Guess values for all the variables at one boundary.

2. Integrate the equations as for initial value problems to the second boundary.

3. Use the error in the solution at the second boundary to modify the guess at the firstboundary. This might take the following form. Find two initial guesses which whenintegrated bracket the correct solution on the far boundary (i.e. one initial guess givesa solution above, and one below, the value on the second boundary). Use a bisectionmethod to iterate towards an initial guess which matches the boundary condition onthe second boundary to sufficient accuracy.


The importance of good initial guesses is clear. Scaling and physical arguments are crucialin this case. Take the example above. The value of M on the outer boundary is just thetotal mass. Physically the total mass must be close to M0 and hence our initial guess forM on the outer boundary must be close to unity. Without scaling the problem in the firstcase the guess would have been much more difficult to make.

2.6 Methods for Specific Problems

The methods we have discussed in this section apply quite generally; however it shouldbe clear that there is a lot of freedom in the selection of a method to solve a differentialequation. In many common cases specific schemes have been developed which are veryefficient and can be used in one particular context to great effect. We shall meet anexample of this in the next section when we consider simulations based on N -body models,in which we have to integrate Newton’s laws of motion many many times!

Chapter 3

Simulation I: N-body Simulations

How do we model very complex physical systems? For example:

• Galaxies containing 1011 stars and gas.

• Motion of atoms and molecules and chemical reactions.

• Plasma physics phenomena.

We shall only consider classical physical models in this section, in which the governingequations are simply Newton’s laws of motion:

mid2ri

dt2= Fi(r1, r2, . . . rN).

The approach to these complex problems is to use a simulation in which we keep track ofmany particles. These particles represent for example groups of stars, or individual atoms,molecules, electrons or ions. The problem is set in terms of basic physical equations andwe use the simulation much like a computer experiment to investigate how the systemresponds under specific conditions. The particles move according to Newton’s laws andperhaps other interactions. By following the simulation in time we can learn how thesystem evolves and begin to understand the phenomena that occur.

In all of these models we have to consider:

1. Correct form for the force laws.

2. Integration of the equations of motion.

3. Methods for improving efficiency.

31


In this section we will look at two of these types of simulation in some detail: galaxysimulations and molecular dynamics (MD).

3.1 Simulations of Galaxies

3.1.1 Basic structure

Figure 3.1: A real galaxy.

A galaxy consists of stars, gas and so-called dark-matter. How can we model such a system?

• Galaxy contains ∼ 1011 stars. We cannot therefore have one particle per star; insteadwe use ∼ 106 “super particles” representing say 105 stars each.

• Gas can be modelled as a fluid or, as we shall discuss here, a set of discrete clouds.

• The dark matter interacts via gravity, and again we include this by having particlesrepresenting the dark matter.

3.1.2 Force Law

In principle it is a straightforward job to write down the force law between particles; it isjust derived from Newtonian gravity:

Fi =∑

j

Gmimjrij

r3ij

.

3. Simulation I: N -body Simulations 33

However, since each particle represents ∼ 105 stars, then close encounters between twosimulation particles (a scattering event) lead to an unphysically large change in the mo-mentum of each particle. The effect is to randomise the particle velocities. We can thinkof this as an effective heating process. In the real system, stars are so small that directcollisions are exceedingly rare indeed. To avoid this problem we modify the force law to“soften” the gravitational interaction:

Fi =∑

j

Gmimjrij

(r2ij + ε2)3/2

.

This expression introduces a softening length, ε. By making this of order the size of asuper particle (in this case a region containing 105 stars) we remove the problem.

In principle this force law is all we need, and for small numbers of particles we can computethe forces directly using the above formula. However for large numbers of particles this is avery inefficient process since for N particles we would perform ∼ N2 force determinationsper time step. This soon becomes computationally impossible. We will return to thisproblem later.

3.1.3 Gas Particles and Collisions

Gas particles differ from star particles in the following way:

• Stars in a galaxy are to a very good approximation collisionless, i.e. direct collisionsdo not occur. The stellar system therefore preserves energy exactly.

• Gas clouds however have a significant physical size, typically ∼ 1− 2% of the size ofthe galaxy itself. Therefore direct collisions between gas clouds do occur. Further,in these collisions the gas itself is heated and cools via EM radiation. The collisionsare therefore dissipative.

When modelling the gas clouds we must therefore search for collisions. Similar techniquesare required in other N -body simulations. Consider two clouds with their relative motionas shown in Fig. 3.2, and take the cloud radius R ∝ m1/3.

During a time step, ∆t, the trajectory of the cloud is approximately given by a straightline. The position of the centre of each cloud is given by:

ri = r0i + viδt, δt < ∆t, i = 1, 2.

The clouds collide if:|r1 − r2| ≤ (R1 + R2).


r

1 1m , R , r

2

1 , v1

m , R ,2 2 2 , v

Figure 3.2: Diagram of gas cloud collision.

After squaring both sides, a little algebra reduces this to:

(∆v)2δt2 + 2∆r0 ·∆vδt + (∆r0)2 − (R1 + R2)2 ≤ 0

where ∆r0 = r01 − r0

2, and ∆v = v1 − v2.

This can easily be solved analytically. Note how, as far as possible, we use analysis toreduce the computational overhead of the problem. Having found that a collision occurswe can take an appropriate action, for example (i) merger of the clouds for a slow collision;(ii) dissipation for a normal collision; (iii) break up of the clouds for a high-velocity collision.Again we use analytical models to decide upon the outcome.

We still have a computational problem: should we check all pairs of clouds for possiblecollision? At the very least we would need to compute all cloud-cloud distances, whichwould quickly lead to a severe computational overhead as this operation would scale asN2. We will see later one method for making this problem tractable.

3.1.4 Initial Conditions

For each galaxy in the simulation we need initial conditions. For this we take an analyticalapproximation to the observed mass distribution, M(r), within a galaxy. We shall considerhere disc galaxies.


dark matter halo

bulge

disc

Figure 3.3: Diagram of a model disc galaxy.

The galaxy consists of three components, a disc, bulge and dark-matter halo. For eachmass component we use particles to represent the mass. Gas particles are only presentin the disc. In all cases we set the particles in motion with approximately circular orbitsaround the centre of symmetry with:

v2

r≈ GM(r)

r2.

In addition we add a small random velocity component. For particles in the disc all orbitsare confined (mainly) to a single plane. The distribution of particles is consistent with thesupplied mass distribution.

The next stage is called “relaxation”. The simulation is now run, moving the particlesaccording to the equations of motion, and various model parameters are monitored. Ourinitial conditions are not perfect, and initially the system will evolve substantially awayfrom its initial state; the hope is that the initial conditions are sufficiently good thateventually the system will settle down into a steady state. Once this is achieved we areable to begin the numerical experiment proper; for example the simulation of interactinggalaxies.

3.2 Molecular Dynamics

Molecular Dynamics (MD) is a simulation technique widely used to study the motion ofmolecules and atoms, and has wide applicability to problems in condensed matter physicsand chemistry. Although full quantum-mechanical N -body simulations are now possible,here we only consider models based upon classical approximations for the force laws. Theresulting equations are integrated using Newton’s laws, and there is therefore a clear re-semblance to N -body simulations of galaxies.


Figure 3.4: The interacting galaxies NGC 3395 and NGC 3396.

3.2.1 The Force Equations

We can use simulation particles to represent either:

• Whole molecules.

• Individual atoms or structural groups (in say a polymer molecule).

In each case forces of different types can be included:

• Non-bonding forces between molecules.

• Covalent-like bonding forces within a molecule.

• Electrostatic, ionic forces either represented as whole charges or residual dipoles,quadrupoles etc.

We shall not go into detail concerning different force models, but consider the simplestcase of forces between spherical molecules forming the so-called Lennard-Jones fluid.

The Lennard-Jones potential has the form:

ULJ(r) = 4U0

[(r

r0

)−12

−(

r

r0

)−6]

and:

FLJ = 12U0r

r2

[(r

r0

)−12

− 1

2

(r

r0

)−6].


This force clearly falls off rapidly, and therefore to avoid the calculation of increasinglysmall long-range interactions we shall choose to truncate the force at a critical radius rc.A common choice is rc = 2.5r0, at which radius the force has fallen to 1.6 × 10−3 of itsvalue at r0. This sounds like a reasonable approximation, but let us consider the errorintroduced into the energy calculation based on this truncated Lennard-Jones potential.

We can estimate the average error in the energy per particle by integrating from the criticalradius to infinity and including all possible interactions:

U+ =1

2

∫ ∞

rc

4πr2ρ(r)ULJ(r)dr

≈ 2πρ∫ ∞

rc

r2ULJ(r)dr

=8πρU0r

30

3

[1

3

(rc

r0

)−9

−(

rc

r0

)−3]

where the factor of 1/2 comes from not counting interactions twice, and we have treatedthe density of molecules as constant with radius beyond the critical radius. Again takingrc = 2.5r0, we find U+ ∼ −0.535U0, which is not negligible. More importantly, as a particlepasses through the critical radius it will see a step change in the potential; when averagedover the entire distribution, this will appear as a false pressure of magnitude of order ρU+.To avoid this problem we must use a shifted truncated potential of the form:

U(r) =

{ULJ(r)− ULJ(rc) r ≤ rc

0 r > rc

Once again we see a computational problem. Even using truncated potentials in order tocalculate the non-bonded interactions we must check the separation of all particles in thesimulation, which is an operation which again scales as N2. The solution to this problemwill involve the same computational tricks as we employ for the galaxy simulations, andwill be discussed below.

3.2.2 Simulations in Different Ensembles

The implicit algorithm we have considered above will lead to the total energy of the sys-tem being conserved. In the language of statistical mechanics such a system is called amicrocanonical ensemble. This is not the only possibility. Suppose the system is not in factcompletely isolated, but may exchange energy (in some form) with a heat bath. In thatcase we would expect the system to have a constant thermodynamic temperature. Sucha case is termed a canonical ensemble. We can simulate this scenario easily if we assumethat the heat transfer is a very efficient process. In that case we arrange for the systemto have a constant kinetic energy (isokinetic). To do this we simply calculate after eachtime step the total kinetic energy of the system, and scale the velocities so that the kineticenergy is kept fixed with respect to some pre-determined value.


3.3 Integration of the Equations

We have not yet discussed integration of the equations of motion, which is the main partof the simulation! Since integration of Newton’s laws is such a common problem (and acrucial one in N -body simulations) special algorithms have been developed for the task,although they turn out to be very similar to the Runge-Kutta method we discussed earlier,and have the same desirable properties.

Consider a time step of length h. We start by expanding the position in a Taylor series:

r(t + h) = r(t) + v(t)h +F(t)

2mh2 +

h3

3!

d3r

dt3+ O(h4),

similarly:

r(t− h) = r(t)− v(t)h +F(t)

2mh2 − h3

3!

d3r

dt3+ O(h4).

Summing these equations gives:

r(t + h) = 2r(t)− r(t− h) +F(t)

mh2 (3.1)

which has an error O(h4). This formula doesn’t include the velocity, which is given by:

v(t) =r(t + h)− r(t− h)

2h. (3.2)

This system of equations defines the Verlet method. It is symmetric and is closely relatedto the midpoint method, with an error O(h4). The apparent problem of having the velocitydefined at a different time to the positions, and needing the positions to be stored at twodifferent time steps, can be overcome by simply recasting the problem as follows. Firstwrite equations (3.1) and (3.2) for the next time step:

r(t + 2h) = 2r(t + h)− r(t) +F(t + h)

mh2, (3.3)

v(t + h) =r(t + 2h)− r(t)

2h. (3.4)

We now eliminate r(t + 2h) and r(t− h) from equations (3.1) to (3.4) to obtain:

r(t + h) = r(t) + v(t)h +F(t)

2mh2

v(t + h) = v(t) +h

2m[F(t) + F(t + h)] .

These equations clearly retain all the properties of the original set, but are in a very usefulform. The problem could have been re-phrased in terms of midpoints of a single step whichwould have led to an equivalent set of equations (in terms of errors etc.); the method inthis case is known as the leapfrog method. The method we have described to integrate theequations of motion is very commonly used in N -body simulations.


3.4 Finding Neighbours

We have seen that a significant computational cost in the two examples of N -body simu-lations we have considered is the identification of nearby particles, either for determiningif collisions have occurred or for the calculation of forces. The problem is worth restating.If we take the simple approach and determine the separation between all particles to findthose which are close, this will involve N2/2 determinations of the form:

d2 = (x1 − x2)2 + (y1 − y2)

2 + (z1 − z2)2,

and even if we work with d2 and not d (thereby avoiding the costly job of calculating asquare root) the whole calculation will still be very expensive and will strongly limit thenumber of particles we can have in our simulation. What we require is an algorithm thatscales with the number of particles N . One method to achieve this is to use “cell lists”.Consider superposing upon the simulation a grid of uniformly sized cubic cells.

Figure 3.5: Diagram illustrating the cell list method.

The two-dimensional diagram in Fig. 3.5 illustrates the method. What we do is to deter-mine which cell a given particle occupies, and for each cell keep a list of all particles withinit. By an appropriate choice of cell we can ensure that to find neighbours we only need tolook in the cell itself and adjoining cells. For a large problem the time saving is very greatsince the density of particles in the cells can be kept constant and therefore the searchprocess does indeed scale only as the total number of particles N .


It is worth considering how we might implement this in Fortran95 as it is the firstoccasion we have really met a non-numerical algorithm. Suppose we have npart particlesand a cell grid of ncell × ncell × ncell cells, with a cell size of cell size. Theposition of the i’th particle is held in posn(1, i), posn(2, i) and posn(3, i). First,appropriate variable definitions would be something like:

implicit none


integer, parameter :: npart = 100000

real(dp) cell_size

real(dp) posn(3, n)

integer i, part, cell(3)

The code to determine the indices of the cell in which particle part lies (cell(:)) is:

do i = 1, 3

cell(i) = 1 + posn(i, part) / cell_size

cell(i) = min(cell(i), ncell)

cell(i) = max(cell(i), 1)

end do

The min and max expressions take care of particles straying outside the boundaries. Thecode could be simplified into a single line using an implied do loop as follows:

cell(:) = max(min(1 + posn(:, part) / cell_size, ncell), 1)

but although more concise, this is rather less clear!

Having determined which cell the particle is in, we need to add it to a list of particles atthat cell. We can do this in Fortran95 using two integer arrays: cell num and cell list

defined as follows:

integer cell_num(ncell, ncell, ncell)

integer cell_list(max_list, ncell, ncell, ncell)

The array cell num holds the number of particles in each cell, and the array cell list isan index listing the particles found in each cell. max list is a parameter corresponding tothe maximum number of particles in any one cell.

During each iteration we do the following:


(a) Initialise the cell list array.

(b) Determine in which cell each particle is.

(c) Use the list in subsequent operations.

Here is a program fragment which sets up the cell num and cell list arrays. Note thatthis program stops if the number of particles in any cell exceeds the max list parameter.This behaviour would be rather undesirable if it were to occur half-way through the simu-lation. In order to rectify this situation, it would be necessary to use dynamically allocatedarrays and to update max list at each step.

program simulation

implicit none


integer, parameter :: npart = 100000 ! number of particles

integer, parameter :: max_list = 100 ! max number of particles per cell

integer, parameter :: ncell = 20 ! number of cells in one direction

real(dp) cell_size ! cell size

real(dp) posn(3, npart) ! particle positions

integer cell_num(ncell, ncell, ncell) ! number of particles in each cell


! index of particles in each cell

! set cell size

cell_size = 0.1d0

! randomise particle positions

call random_number(posn)

posn = posn * cell_size * ncell ! this is an array operation

! initialise cell list

call init_cell_list(npart, ncell, max_list, cell_size, posn, &

cell_num, cell_list)

! now go and do something interesting with it...

end program simulation


subroutine init_cell_list(npart, ncell, max_list, cell_size, posn, &

cell_num, cell_list)

implicit none


integer npart ! number of particles

integer ncell ! number of cells in one direction

integer max_list ! max number of particles per cell

real(dp) cell_size ! cell size

real(dp) posn(3, npart) ! particle positions

integer cell_num(ncell, ncell, ncell) ! number of particles in each cell


! index of particles in each cell

integer i, part, k, cell(3) ! local variables

! initialise the cell_num array

cell_num = 0

! loop over all particles

do part = 1, npart

do i = 1, 3

cell(i) = 1 + posn(i, part) / cell_size

! check for edges

cell(i) = min(cell(i), ncell)

cell(i) = max(cell(i), 1)

end do

! k is the number of particles already in this cell

k = cell_num(cell(1), cell(2), cell(3))

! check we don’t exceed the array bounds

if (k == max_list) stop ’Increase max_list!’

! add the new particle to the list

k = k + 1

cell_list(k, cell(1), cell(2), cell(3)) = part

cell_num(cell(1), cell(2), cell(3)) = k

end do

end subroutine init_cell_list


3.5 Evaluating the Gravitational Force

For the galaxy simulations we are left with one more computational problem. For largenumbers of particles, calculating the force is a major computational task. Unlike the MDcase we cannot truncate the gravitational force as it is strictly long-range. A number oftechniques exist to solve this problem, but we shall discuss briefly just one example whichmakes use of the cell list array we introduced in the previous section.

Instead of calculating the forces directly, we could attempt to calculate the gravitationalpotential and then define the force as the gradient of the potential. There is a trick we canuse to do this. The gravitational potential is determined by Poisson’s equation:

∇2φG = 4πGρ.

If we take the Fourier transform of this equation (for simplicity we consider a 1D case, butit is trivially generalised), then:

−k2φG = 4πGρ, where a = FT(a).

It then follows that:φG = −4πG FT−1(ρ/k2)

and the force is then easily calculated. Now if we have gone through the procedure of settingup a cell list array, we have an estimate of the mass distribution on a regular grid whichwe can take the FT of numerically. Hence we can solve Poisson’s equation and so obtain theforce without summing the individual interactions. Filling the cell list array scales asN , and provided we have an efficient numerical Fourier transform routine (which we do) wehave a good method of solution. This method gives a good estimate of the force due to theoverall distribution of mass in the simulation, and we can find the force at the location of agiven mass by interpolation. However, it is not good for interactions of close-by neighbours.Therefore the overall approach mixes the calculation of the background gravitational fieldusing a FT method with the direct summation approach for nearby particles (we knowwhich these are again using the cell list). Such approaches are called Particle-Particle-Particle-Mesh, or P3M methods. There are in fact other very elegant methods for solvingthis problem which have different error behaviour and are well suited for some types ofproblem, but unfortunately we do not have time to discuss them here.


Chapter 4

Linear Algebra and Eigen Problems

A particularly rich set of numerical techniques is available for dealing with matrices andvectors. If, therefore, we can attack a physics problem in such a way that we can reduce it toa system of linear equations, an eigen-problem, or a sequence of matrix-vector operations,then the chances are that we will be able to find a relatively efficient numerical approachto its solution even for large problems.

In this chapter we will:

1. Briefly review the mathematical properties of matrices and consider their implemen-tation in Fortran95.

2. Look at the range of numerical tools provided in libraries and how to choose betweenthem.

3. Consider a number of physical problems which lend themselves to solution in termsof a Linear Algebra approach.

At a first glance there are a bewildering number of routines in any library which deals withLinear Algebra. This is because most algorithms for, say, solving a set of linear equationsor finding eigenvalues are designed to work with particular types of matrices and matricesof a particular shape; where by the shape of a matrix we mean that region which hasnon-zero elements.

4.1 Brief Review of Matrices and Matrix Operations

In this section we consider the mathematics of matrices and, more importantly, remindyou of the terminology. Where appropriate we will consider particular aspects of how to

45


implement matrices using Fortran95.

4.1.1 Definition and Basic Operations

A matrix is an ordered array of, in general, n ×m numbers. Mathematically this can bewritten as A or Aij. It matches very well on to an Fortran95 array of rank 2 defined by:

real(dp) a(n,m)

where n and m are constants or integer, parameters.

Conventionally we take the first index as a row index and the second as a column index(i.e. in the same order as the suffices). We can perform basic mathematical operationsof equality, addition and subtraction between matrices of the same size, and this is easilyimplemented in Fortran95.

Multiplication requires a little more thought. We will consider multiplication of one matrixby another, but this is easily extended to include vector multiplication by a matrix.

For two matrices of size n×m and m× p, the matrix product is written:

Cij =m∑

k=1

AikBkj.

A simple Fortran95 implementation of this can be written as:

do i = 1, n

do j = 1, p

c(i,j) = 0.0d0

do k = 1, m

c(i,j) = c(i,j) + a(i,k) * b(k,j)

end do

end do

end do

Alternatively we can make use of a Fortran95 intrinsic array function, in which case thisbecomes just:

c = matmul(a,b)

4. Linear Algebra and Eigen Problems 47

4.1.2 Advanced Aside

The elements of a 2D array in Fortran95 are stored consecutively in the computermemory. The order in which these elements are stored is such that the first index is thefaster changing index as we move from one memory location to the next. That is, the orderis:

a(n,m)a(1,1)a(2,1)a(3,1) a(n,1)a(1,2)a(2,2)a(3,2)

Figure 4.1: Schematic representation of array data layout in Fortran95.

Occasionally we need to be aware of this ordering when considering the efficiency of arrayoperations. However for smaller problems we do not have to concern ourselves with this.The place you are most likely to meet this is when using some NAG routines which specifythe ordering of elements of an array explicitly.

4.1.3 Inverse

From now on we will assume all matrices are square of size n× n.

The inverse of a matrix, A−1, is such that:

A−1A = AA−1 = I

where I is the identity matrix. For a general matrix, one numerical technique which isguaranteed to find the inverse if it exists is the Gauss-Jordan technique.

It is possible to find matrices Mi which will have one of the following actions when multi-plying a matrix A (i.e. MiA):

(i) One row is multiplied by a scalar.

(ii) One row is replaced by the original minus a multiple of another row.

(iii) Two rows are interchanged.

Given these elementary operations it is easy to convince yourself that successive applicationof operations of this form can be used to reduce the magnitude of all non-diagonal elementsof the matrix to zero (the operations can clearly be used to make a given element zero).


Once this is done we can scale each row to get values of unity along the diagonal. Hencewe have:

MNMN−1 . . .M3M2M1A = I.

Clearly this product, M = MNMN−1 . . .M3M2M1, must be the inverse of A. It is possibleto implement such a scheme fairly easily numerically.

4.1.4 Transpose and Orthogonality

We define the transpose of a matrix A as AT:(AT

)ij

= Aji

i.e. rows and columns are interchanged. The matrix is orthogonal if:

AT = A−1.

The matrix is called symmetric if:AT = A,

and anti-symmetric (or skew-symmetric) if:

AT = −A.

4.1.5 Similarity Transformations

A similarity transformation of A is defined by:

A′ = BAB−1.

4.1.6 Complex Matrices

We can form the complex conjugate of a matrix on an element by element basis:

(A∗)ij = A∗ij.

We define the adjoint matrix A†:

A† =(AT

)∗= (A∗)T .

The matrix is self-adjoint or Hermitian if:

A† = A,

and unitary if:A† = A−1.


4.1.7 Linear Equations

A set of coupled linear equations can clearly be written in the form:

Ax = b

where x is an unknown vector and b is a known vector. The formal solution to this set ofequations is:

x = A−1b.

Numerically we would not find the inverse of A to solve this set of equations, but insteaduse a direct method of solution, for example using the Gauss-Jordan method applied tothis equation. We again find the matrix M such that:

MAx = Ix = Mb.

4.1.8 Eigen Equations

An eigenproblem has the general form:

Ax = λx

where the scalar λ is the eigenvalue and the vector solutions are the eigenvectors. We shallreturn to problems of this type below.

4.2 Shapes of Matrices and Choice of Routine

In choosing a specific routine from a library of routines (such as NAG) we need to considertwo things:

(i) The type of the matrix: is it square? symmetric? Hermitian? etc.

(ii) The shape of the matrix: in this context what we mean by shape is the distributionof non-zero elements of the matrix.

A matrix with a significant number of zero elements is termed sparse and depending onthe distribution of the zeros the matrix may have a particular form which lends itself toparticular algorithms. We start by considering the different shapes of sparse matrices andthen mention just one specific type of problem.


Figure 4.2: Schematic representation of a diagonal matrix.

Figure 4.3: Schematic representation of a tridiagonal matrix.


One shape of matrix you have met before is a diagonal matrix with non-zero values on thediagonal elements only. Schematically we can draw this as in Fig. 4.2. Another importantshape is the so-called tridiagonal matrix, shown in Fig. 4.3. This has non-zero elementson the main diagonal and each diagonal above and below the main diagonal of the matrix;therefore except for the first and last rows there are three non-zero values per row. Forthe first and last rows there are only two non-zero values. Non-zero elements may also bearranged in other patterns; for example a triangular matrix only has non-zero elementsalong the main diagonal and also in either the upper or lower triangle of the matrix, as inFig. 4.4.

Figure 4.4: Schematic representation of an upper triangular matrix.

We do not have the time to discuss in detail the various approaches applicable to differentshapes and types of matrix. However to illustrate the importance of shape we shall discussperhaps the simplest example, that of a triangular matrix.

Consider the following example the set of linear equations:

Tx = b.

For a 3× 3 triangular matrix:

T11 T12 T13

0 T22 T23

0 0 T33

x1

x2

x3

=

b1

b2

b3

.

The method of solution is trivial. Consider the last row of the matrix; this gives immedi-ately:

x3 = b3/T33.

Substituting this result into the equation from the row above (back substitution) we find:

x2 = (b2 − T23x3)/T22


(d)

zeros

(c)(b)(a)

(k)(j)

(e) (f)

zeros

zeros

(g) (h) (i)

Figure 4.5: Other shapes of sparse matrices. (a) band diagonal; (b) block diagonal; (c) blocktridiagonal; (d) singly bordered block diagonal; (e) doubly bordered block diagonal; (f) singlybordered block triangular; (g) bordered band-triangular; (h) and (i) singly and doubly borderedband diagonal; (j) and (k) other.


and so on up the matrix. This is clearly straightforward to implement computationally.Now consider a more general matrix A which can be written as the product of two trian-gular matrices: one upper triangular, U, and the other lower triangular, L:

Ax = LUx = b.

We can write this in the following form by introducing a new vector y:

Ly = b

Ux = y

The first equation can be solved trivially for y and then the second for x. Provided we areable to decompose the original matrix into such a product (the LU-decomposition) we cansolve equations of this form efficiently even for large problems; and indeed other matrixoperations are greatly simplified. There are therefore numerical techniques designed toperform this decomposition and then make use of it in efficient algorithms.

This is just one example of how the shape of a sparse matrix/array can be used to helpus implement very efficient algorithms. However, if the matrix is very sparse it is clearlyinefficient to store information about many zero-valued elements. There are thereforeschemes to enable us to store information about a sparse array in an efficient way. Onesimple scheme is as follows. Suppose we have n non-zero elements of the matrix. We onlyneed to store (a) the values of the non-zero elements and (b) where they are in the matrix.We could do this with one real(dp) and one integer array as follows:

real(dp) a(n) ! values of non-zero elements

integer idx(2,n) ! stores ’i’ and ’j’ for each non-zero element

4.3 Applications of Linear Algebra

In this section we discuss a number of problems which can be reduced to a descriptioninvolving the solution of a Linear Algebra problem. The aim is to show the wide rangeof problems which can be expressed in this form and to illustrate some of the generaltechniques used.

4.3.1 Diffusion in a Porous Material: Tortuosity

Many flow problems in porous materials can be modelled, to a first approximation, usinga model in which we reduce the complexity of the material to a series of interconnecting


“pores” joined at “nodes”, much like a network of inter-connecting pipes. Let us considerthe problem of diffusion. The diffusion equation can be written in the form:

J = −D∇c, (4.1)

where J is the flux of material, D the diffusion coefficient and c the concentration.

c1

l

Jr

c2

Figure 4.6: Diagram of diffusion along a cylindrical tube.

Consider diffusion in a restricted geometry, namely a long, thin cylindrical tube (witha radius, r, much larger than the mean free path). In steady state, and with a fixedconcentration difference between the ends of the tube, the flux must be constant down thetube as no material is lost or gained within the tube. In this case the total flow of materialN = πr2 |J| is also a constant and must be given (from integrating equation 4.1) by:

N = πr2D(c1 − c2)/l,

where l is the length of the tube.

Rc cL

Figure 4.7: Diagram of the porous material.

We now take this as our governing equation for a pore in a porous material, and modelthe material as an interconnecting set of such pores of potentially different radii. The


distribution of pores can be either regular or random. An overall concentration gradient isapplied to this model material as shown in Fig. 4.7. To determine the net flow of materialfrom left to right in the diagram, we must determine the flow through the network of pores.

To do this we label each node (intersection of pores) with an index, say i. For the porejoining nodes i and j we have a length lij, radius rij and net flow of material Nij given by:

Nij = πr2ijD(cj − ci)/lij,

where ci and cj are the concentrations at nodes i and j respectively. For all nodes withinthe bulk of the material (internal nodes) there can be no accumulation of material at thenode, hence for node i: ∑

j

Nij = 0.

For a node at the surface, there is a net flux. For surface nodes we must have:

Ntot =left surface∑

k

∑

j

πr2kjD (cL − cj)/lkj =

right surface∑

k

∑

j

πr2kjD (cj − cR)/lkj, (4.2)

where the sum is over left and right surface nodes in each sum over k respectively, and thenodes j to which they are joined. The unknowns in this problem are the concentrations,and we can re-write the problem in a matrix form:

Ac = z (4.3)

where:

Aij =

∑

k

πr2ik

likD i = j

−πr2ij

lijD i 6= j, but i joined to j

0 otherwise

and

zi =surface∑

k

πr2ik

likDck.

Hence we can solve for all the concentrations within the network by solving the coupledlinear equations given by (4.3).

How should we scale this problem?

• Note that D cancels from the equations; it has to since we have no other quantitywhich has the dimensions of time!

• Only the concentration difference across the network is important, hence we can takecR = 0 and cL = c0. Dimensionless concentrations are then Xi = ci/c0.


• If the width of the network is L, then dimensionless pore lengths can be writtenζij = lij/L; and further if it has cross section A, and the ratio of pore volume to totalvolume is ε, then we can write for pore cross sections πr2

ij = εAaij.

Then the net flux is equal to:

Ntot =AεDc0

L

∑

kj

akj (1−Xj)/ζkj

=ADeffc0

L

where the second line compares the result to a uniform material with an effective diffusioncoefficient. The effective diffusion coefficient is written as Deff = εD/τ , where τ , thetortuosity, is a measure of the tortuous path through the pore space. Clearly the tortuositycan be determined from the model results.

Many variants of this basic flow problem are possible, including leaky pores, bulk flow,diffusion and chemical reaction. The model is also a good starting point for other problemssuch as modelling oil recovery from rocks.

4.3.2 Normal Modes of a Material and Spectroscopy

An important problem in condensed-matter physics is to determine the normal modes ofvibration of a material. These modes will correspond to the phonon spectrum and maybe probed via a number of experimental techniques, for example IR spectroscopy in manycases. By material we mean something quite general. For example an ionic solid, or acovalently bonded solid or a finite molecule with bond-stretching and bond-bending forces.Note the similarity in one sense to Molecular Dynamics discussed in the previous chapter:the types of inter-atomic and inter-molecular forces are much the same. Now, however, weare seeking the normal modes of vibration of the system.

In general it is easy to write down the equations of motion for each particle in the model(again much as in the MD case). For each of the N atoms in our problem:

mj rj = −∑

k

∇ujk(r)|rj,

where the sum is over all the atoms (and/or molecules) with which atom j interacts, andujk is the energy of interaction between atoms j and k. This is just the same problemas MD, but now we are looking to solve not the motion of individual atoms as a functionof time, but determine the large-scale coherent motions of the material. We proceed bymaking the following assumptions:


• That we know the equilibrium configuration of the material.

• That we can linearise the equations of motion by expanding the potential aboutequilibrium.

We therefore proceed by expanding the gradient of the potential in a Taylor expansionabout equilibrium. In the linearised form the equation will look like:

mjxj = −∑

k

Fjkxk,

where the Fjk are 3×3 matrices giving the coefficients of the linearised interaction betweenatoms j and k, and the vectors x are displacements from equilibrium. From the Taylorexpansion we find:

(Fjk)pq =∂2ujk

∂rp∂rq

∣∣∣∣∣rj

.

We can turn this into a standard Linear Algebra problem as follows:

(i) Write the 3N linearised displacements as a single vector with three-dimensional sub-vectors Rk =

√mk xk.

(ii) Construct a single 3N × 3N matrix, H, with sub-matrices Hkj = Fkj/√

mkmj.

The equations of motion now reduce to:

R = −HR.

We can cast this in the form of a standard eigenproblem by seeking harmonic solutions ofthe form:

R(t) = R0 exp(iωt),

which gives an eigenvalue equation of the form:(H− ω2I

)R0 = 0.

If we know the nature of the forces we can work out the linearised forms of the equationsanalytically and construct the H matrix for any size of problem we choose. At this pointwe need to do two things:

(a) Scale the equations (this is now straightforward).

(b) Choose a suitable library routine to solve the problem.

The choice of library routine should take note of the following:


(i) The matrix H is clearly symmetric by Newton’s Third Law.

(ii) If only near-neighbour interactions are included, and the material has a regular struc-ture, then by appropriate ordering of the atom indices the matrix H should be banddiagonal in shape.

(iii) Even for longer range forces, the matrix should be highly sparse.

4.3.3 Quantum Mechanics

I have left perhaps the most obvious application of linear algebra techniques to the last.You have already seen, both last year and this year, how matrices occur naturally in theformulation of various quantum problems via “matrix elements” formed by integrals oversome operator, say A, and basis states, {φi} say:

Aij = 〈φi|A|φj〉.

Application to many of the problems that you have seen in quantum mechanics is (fairly)clear, provided the integrals implied by the formation of these matrix elements are analyt-ically or numerically tractable, given known basis states.

As an illustration let us consider one technique, a generalisation of the variational principleyou have already met. Suppose we seek the solution to Schrodinger’s equation which wewrite as:

H|ψ〉 = E|ψ〉. (4.4)

We also have a set of N functions, |φi〉. These are not necessarily orthonormal, but theyare required to match the boundary conditions. We can use this as a basis set:

|ψ〉 =∑

i

ai|φi〉. (4.5)

This is similar to the sorts of expansion you have made before, but we are now using alimited number of basis functions. We can obtain a form of eigenproblem as follows:

1. Insert the expansion in terms of the basis set (4.5) into (4.4):

H|ψ〉 =∑

i

aiH|φi〉 = E∑

i

ai|φi〉.

2. Take the inner product with |φk〉:∑

i

ai〈φk|H|φi〉 = E∑

i

ai〈φk|φi〉.


This is just a set of linear equations of the form:

Ha = ESa, (4.6)

where a is a vector and H and S are matrices given by:

Hij = 〈φi|H|φj〉, Sij = 〈φi|φj〉.Note that the basis functions are not necessarily orthonormal, and therefore S is not ingeneral a trivial matrix. The equations (4.6) can also be derived by a variational argumentwhich can be used to “prove” that in some sense the results of the solution of this set ofequations are a “good estimate” of the true solution.

Equations of the form of (4.6) are called generalised eigenvalue equations and routines existto solve them numerically!

So how would we use this approach? As an example consider the infinitely deep potentialwell in the region −l to l. We shall assume the problem is scaled so that l = 1 andh2/(2m) = 1. We know the solutions to this problem are just of the form cos(knx) (for nodd) and sin(knx) (for n even), where kn = nπ/2 and n > 0. The energies are En = n2π2/4.

We can find very good estimates of these exact results using the trial basis functions (whichare not orthonormal):

φn(x) = xn(x− 1)(x + 1), n = 0, 1, 2, . . .

The matrix elements are:

Hmn =4 [1 + (−1)m+n] (2mn + m + n− 1)

(m + n + 3)(m + n + 1)(m + n− 1),

Smn =8 [1 + (−1)m+n]

(m + n + 5)(m + n + 3)(m + n + 1).

A suitable routine for solving the generalised eigenvalue problem is F02FDF:

subroutine F02FDF(itype, job, uplo, n, a, lda, b,ldb, w, wrk, lwk, ifail)

integer itype, n, lda, ldb, lwk, ifail

real a(lda,*), b(ldb,*), w(*), wrk(lwk)

character job, uplo

• itype specifies the type of problem: for Az = λBz set itype = 1

• job specifies the job: to find eigenvalues only set job = ’N’

• since the matrices are symmetric, it is only necessary to define half of the arraypassed: setting uplo = ’U’ means the upper triangle should be used


• n is the order of the matrices A and B

• the contents of the arrays a and b are overwritten by the NAG routine

• for flexibility, NAG allows arrays larger than n × n to be used: lda and ldb specifythe size of the first dimension of the arrays a and b

• * indicates that an array size is unknown: in this case all the array dimensionslabelled * must be at least n but there is no upper limit

• the eigenvalues are returned in the array w

• workspace wrk is required of size lwk = 3*n

The code on the following page gives these results:

spqr1@pcphy117:~> cp $PHYTEACH/part_2/examples/quantum.f90 .

spqr1@pcphy117:~> f95 quantum.f90 -lnag

spqr1@pcphy117:~> ./a.out

Enter number of basis functions:

5

Computed % error

1 2.467401108747 0.000000343449

2 9.875388202502 0.058602160507

3 22.293405912300 0.390856642372

4 50.124611797498 26.967124923380

5 87.739192978953 42.237422151216

Even with just 5 basis functions we obtain estimates for E which are within 0.4% or betterfor the first 3 energy levels.


program quantum

use nag_f77_f_chapter

implicit none


integer :: nbasis

integer :: n, m, lwk,ifail

real(dp), parameter :: pisqon4 = 2.4674011002723397d0

real(dp), allocatable :: h(:,:),s(:,:),e(:),wrk(:)

write(*,*) ’Enter number of basis functions: ’

read(*,*) nbasis

! allocate arrays

lwk = 3*nbasis

allocate(h(nbasis,nbasis), s(nbasis,nbasis), e(nbasis), wrk(lwk))

! set up matrices: basis functions are labelled from zero whereas

! array indices start at one

h = 0.0d0 ; s = 0.0d0

do m=0,nbasis-1

do n=m,nbasis-1 ! fill upper triangle only

if (mod(m+n,2) == 0) then

h(m+1,n+1) = (8.0d0*(2*m*n+m+n-1))/((m+n+3)*(m+n+1)*(m+n-1))

s(m+1,n+1) = 16.0d0/((m+n+5)*(m+n+3)*(m+n+1))

end if

end do

end do

! call NAG routine to solve generalised eigenvalue problem

ifail = 0

call f02fdf(1,’N’,’U’,nbasis,h,nbasis,s,nbasis,e,wrk,lwk,ifail)

! report results and compare against known exact solution

if (ifail == 0) then

write(*,’(11x,a,11x,a)’) ’Computed’,’% error’

do n=1,nbasis

write(*,’(i6,f18.12,f6.2)’) n,e(n),(e(n)/(n*n*pisqon4)-1)*1.0d2

end do

else

write(*,*) ’NAG routine F02FDF failed with ifail = ’,ifail

end if

end program


Chapter 5

Simulation II: Abstract Simulation

Figure 5.1: The physicist’s horse.

You are familiar with the idea of creating a physical model in which we attempt to distilthe main aspects of a problem so that we can understand the behaviour of the systemwithout over-complicated detail.

However, the types of problem we have considered in this course so far fall into two classes:

(a) Numerical solution to problems that are not analytically soluble

(b) Simulations of complex systems by including detailed (and reasonably exact) physics.

In this section we shall consider a third class of problems, ones where again we investigatea more abstract problem which we hope contains the main ideas behind the behaviour ofa physical system, but this time using a purely computational approach.

63


There are many such types of simulation in computational physics; but here we shall discussone particular approach, where a lattice is used in one form or another.

5.1 Cellular Automata

Cellular automata are a simulation technique in which we regard time, location and allphysical quantities as discrete and only able to take on certain (integer) values. A systemis defined which is allowed to evolve according to a set of remarkably simple rules. Whatis perhaps surprising, and is the motivation for using cellular automata to model physicalsystems, is that very simple rules can lead to complex behaviour. In other words wecan hope to understand the complexity in many physical systems by using much simplerconsiderations.

Firstly we define a lattice; for example a regular 2D or 3D lattice with or without periodicboundary conditions. We shall call the state of the system Ψ, and the state of a particularcell ψ(i, j, . . .). The basic idea is that given the state of the system at some iteration step,Ψi, and a rule, we can obtain the state of the system at the next iteration step Ψi+1.

Let us consider a simple example. We take a 2D lattice with periodic boundary conditions.The cells have a single “property” which is that they can take on the value 0 or 1. Thesimple rule is:

(i) For site (i, j) find the sum over adjacent sites.

(ii) If the sum is even set ψ(i, j) to 0, otherwise set ψ(i, j) to 1.

The evolution of this system is illustrated in Fig. 5.2. Note how the system develops

Figure 5.2: The evolution of the system from the initial state (a) after (b) 93 iterations and (c)110 iterations.

complex geometrical structure, i.e. large-scale complexity, from simple initial conditionsand a simple (local) rule.

5. Simulation II: Abstract Simulation 65

The nature of the cellular automata rules is such that we only need to use values inthe vicinity of a given cell to update its value at the next iteration. We call this theneighbourhood of the cell. Two neighbourhoods are commonly used:

(a) Von Neumann; nearest neighbours.

(b) Moore; first and second nearest neighbours.

(b)(a)

Figure 5.3: (a) Von Neumann and (b) Moore neighbourhoods.

Figure 5.3 illustrates these neighbourhoods in 2D; they can easily be generalised to higherdimensions.

We are not confined to cubic lattices, and indeed a totally general lattice could be definedin which for each cell we specify a list of other cells to which it is joined.

We shall now consider a number of different physical models and the rules we can inventto model the complexity of the system with our abstract lattice model.

5.1.1 Ising Model of Ferromagnetism

Ferromagnetism at a microscopic level is a phenomenon driven by the distribution of elec-trons in atoms governed by quantum mechanics. However, on the macroscopic scale whatis most important is how the magnetic moments of individual atoms are arranged locallyin domains. We can abstract this model by considering just the direction of the spin, asup (represented by the value s = 1) or down (represented by the value s = 0). A spinwill flip from up to down or vice versa according to the local energy gain this transitioncorresponds to. This is clearly well suited to the cellular automata approach and we havealready defined the possible state of each cell. For the simulation consider a 2D regularlattice; we need to determine a local rule and for this we use energy conservation.

The energy comes from pair-wise contributions in the local, von Neumann neighbourhood;for two such spins the energy has a value J when they are aligned and −J when they are


not aligned. Our rule must be such that a spin can flip at time t only if the energy is notchanged. This would suggest that the rule should be that a spin can flip if it has equalnumbers of up and down neighbours, as this would not alter the energy.

Figure 5.4: Diagram of the two types of site.

However, this does not allow for the fact that the neighbours may also be changing betweent and t + 1 (they must obey the same rule). To overcome this we introduce a secondproperty and a second rule. The idea will be to avoid the problem by regarding the cellsas a checkerboard and only update one type of site (white or black) at a time. To do thiswe introduce the property b such that the “black” cells are labelled 0 and the “white” cellsare labelled 1.

The rule for the spin in cell (i, j) at time t, st(i, j), can be written:

• if bt(i, j) = 1 and st(i− 1, j) + st(i + 1, j) + st(i, j − 1) + st(i, j + 1) = 2 thenst+1(i, j) = 1− st(i, j)

otherwisest+1(i, j) = st(i, j)

• bt+1(i, j) = 1− bt(i, j)

This rule is often called “Q2R”. Note that the rule has been written such that spin down= 0 and spin up = 1; the spin value here is simply a label and is not related to a physicalquantity, unlike what we have met with in previous simulations.

5.1.2 The Annealing Rule

The annealing rule is also often called the majority rule. Cells can take on the value 0 or1. The idea is to modify the cell value depending (mostly) on the value of the majority oflocal cells. The simple rule is to take a Moore neighbourhood and the cell itself and sumthe values; depending on the value of the sum we obtain the new value as follows:

sum 0 1 2 3 4 5 6 7 8 9ψt+1(i, j) 0 0 0 0 1 0 1 1 1 1


Figure 5.5: Evolution of a system obeying the Q2R Ising model rule.

Note the switch to the majority rule between 4 and 5. This rule models the observedgrowth of the interface between two phases.

Figure 5.6: Growth of phases. The grey levels indicate regions changed in recent iterations(dark grey to have been eroded to white and light grey changed to black).

5.1.3 The HPP rule

The cellular automata approach can clearly model dynamical systems, and in particularwe can ask if we can abstract the behaviour of the random motion of particles in a largesystem (e.g. diffusion in a gas, etc.).

The HPP (Hardy, de Pozzis, Pomeau) rule is designed to model randomly moving, collidingparticles. We now think of particles at the nodes of a 2D lattice with motion allowedbetween the nodes. We conserve momentum and particle number during the simulation.

For each site we have 4 numbers each taking on the value 0 or 1, which indicate whether


Figure 5.7: Illustration of the HPP rule.

a particle is moving towards the nodes from directions 1, 2, 3 or 4 (West, North, East andSouth respectively). For example ψt(i, j) = (1, 0, 1, 1) means that we have particles movinginto the site (i, j) from directions 1, 3 and 4, but not 2, as shown in Fig. 5.7.

We break the rule into two parts:

(a) collision,

(b) motion.

For example, head-on collision at a node must give rise to the following change of state:

(1, 0, 1, 0) 7→ (0, 1, 0, 1) and (0, 1, 0, 1) 7→ (1, 0, 1, 0)

all other states not being changed.

The motion phase involves updating neighbouring nodes as the particles move from onenode to the next. Using the notation ψt(i, j; n) to indicate the n’th number at node (i, j)at time t we have:

ψt+1(i + 1, j; 1) = ψt(i, j; 1)

ψt+1(i, j − 1; 2) = ψt(i, j; 2)

ψt+1(i− 1, j; 3) = ψt(i, j; 3)

ψt+1(i, j + 1; 4) = ψt(i, j; 4)

5.2 Percolation and Cluster Animals

5.2.1 Forest fires and Percolation

We can use cellular automata to begin to understand large-scale phenomena. One suchphenomenon is called percolation. It can be illustrated by a well-known example, that of


Figure 5.8: Example of HPP evolution.

a cellular automata simulation of a forest fire.

The simulation consists of setting up a 2D array of cells, but now we associate the physicalcharacteristic of a cell being occupied by a tree using a random approach. We determineif a given cell should have a tree in it by generating a random number in the range 0 to 1.If the random number is less than p we say the cell contains a tree, and we do this for allcells. Some fraction of cells (1− p) therefore do not contain trees.

The cells which do contain trees can be in one of three states:

• unburnt,

• burning,

• burnt.

We start by setting all trees in the top row of the simulation to state burning. Thesimulation now proceeds with the rule that a cell to the right or below a burning treeburns if it is a tree; a burning tree becomes burnt at the next time step. Figure 5.9illustrates this sequence.

Note that the fire extinguishes itself 8 iterations after the initial state. If we plot the timeto extinction for the fire as a function of p for a large lattice we find a critical value of pof approximately 0.5928 at which the lifetime of the fire diverges. Above the critical valuethe forest is so full of trees the fire spreads easily. Well below the critical value the fire diesout because it leaves islands of untouched trees. Close to the critical value the fire can just


= 0 1 3

6

2

4 5 7

t

Figure 5.9: Illustration of the forest fire simulation: white = no tree, light grey = tree, darkgrey = burning, black = burnt.

keep propagating (in fact until it just reaches the far side of the grid for a finite grid). Wecall this critical value the percolation threshold, pc. Many similar examples of this sort ofbehaviour exist and it is closely related to the behaviour of complex physical systems asthey undergo second order phase transitions.

p0.4 0.5 0.6 0.7

time

to e

xtin

ctio

n

Figure 5.10: Plot of time to extinction against filling fraction p.

At the percolation threshold we find that there is one large cluster of joined cells, in thisexample of “burnt” sites, which spans the network of cells. The size of this cluster thereforediverges at the percolation threshold for large networks of cells.

Obviously this is not the only cluster present in the lattice, and we can go a long wayto understanding the nature of percolation by examining the structure of these so-called


lattice animals: clusters of adjoining cells. As we shall see, we can use similar ideas toexamine other physical phenomena.

In general, what we find is that many properties of the system change drastically as weapproach the percolation threshold. Let us consider the behaviour of cluster animals. Inparticular consider the largest cluster in a number of simulations with a size of L cells ineach dimension, and of dimension d (where d = 2 or 3).

• For p ¿ pc we find the largest cluster scales with L as log L.

• For p À pc we find the largest cluster scales with L as Ld; i.e. as for normal objects.

• For p ≈ pc we find the largest cluster scales with L as LD, where D is not equal to d.D is a fractal dimension and we say the largest cluster is fractal near the percolationthreshold. It has to be the case that D < d.

5.2.2 Cluster Animals

Generally we see that it is the growth of regions of connected burnt cells in the simulationwhich is important. Each separate region of connected cells is called a cluster (in theexample above the connection was defined using the von Neumann neighbourhood). Inprinciple, if we could determine all such possible clusters then we would have the statisticalinput to enable us to make predictions. Each different cluster configuration is called acluster animal.

For small clusters it is easy to enumerate the possible cluster animals. For example, forcluster animals of size 3 cells, we would have the two possible types of distribution shownin Fig. 5.11, and of course all the symmetrically related animals. Direct counting of cluster

Figure 5.11: Cluster animals of size 3 cells.

animals of larger size is however problematic as the possible number of cluster animalsdiverges exponentially with the cluster size. In the next section we consider this problem.


5.3 Cluster Animals and Monte Carlo Sampling

In many problems we are faced with the difficulty of not being able to investigate all possibleoutcomes of a system. The counting of cluster animals is one such example which we metin the previous section. How can we proceed? The answer is to fall back on ideas fromstatistical physics. Instead of trying to count every possibility, we could try to determinethe probability of some possibility arising in the configuration of our system, and if thisconfiguration is truly representative then we may infer properties of the distribution fromthis small sample (appropriately weighted by the probability of obtaining the sample thatwe have).

For example, for the case of cluster animals, suppose we have generated a cluster animalof size N , and know by some means that the probability of getting this cluster animal isPN ; then if all else is equal, the total number of cluster animals of size N must be:

wN = 1/PN .

Of course we can determine average properties of the system. Suppose we have M reali-sations of cluster animals of size N , each with probabilities PN,i for i = 1, . . .M . Then wecan estimate the sample mean of some quantity X simply using:

〈X〉 =M∑

i=1

PN,iXi.

The difficulty is how do we generate sample realisations with a known probability? Thedetails are somewhat problem dependent, and are well illustrated by the next example.However, the general idea is that we try to obtain representative realisations using a prob-abilistic argument, i.e. generate them in a purely random fashion. We call this techniqueMonte Carlo sampling.

5.3.1 Configurations of a Polymer Chain

An interesting question is what are the number of configurations that a long polymerchain can take up? Clearly this is related to the entropy of the polymer. To investigatethis problem we can use the idea of cluster animals and create a model polymer on aregular grid of cells. To model the polymer we introduce additional constraints over thosediscussed above for the simpler cluster animals:

(i) The polymer is made up of N cells forming a single chain where each non-end-of-chaincell is joined to two others using the von Neumann neighbourhood.


(ii) The polymer chain is space filling, so that a given cell can only be occupied once bythe polymer chain.

For this example, let us consider a model system in 2D (one would naturally work in 3Dfor this, but the diagrams are easier in 2D!). We can imagine a simulation in which westart with a polymer chain of 2 cells, and add cells continuously to one end exploring allof the configurations as we proceed.

Figure 5.12: Simulation of polymer configurations.

It is easy to see that there are three ways of adding the next cell to an existing 2-monomerpolymer, and then in the example above three ways to add the next, and then the next(each time we only add to the same end of the polymer). Naively we might thereforeguess that for a chain of length N , and allowing for the obvious symmetries of the initialconfiguration, there should be a total of 4× 3N−2 configurations. This argument howeverignores the disallowed configurations which would wind around and overlap the existingpolymer chain. Although this is an overestimate, we may guess that it provides a goodworking estimate for the number of configurations. We see immediately that for N of evenmoderate size the number of configurations is very large indeed!! Disallowing configurationsin which the polymer chain winds back on itself we call a multi-occupancy constraint.

So how do we count them in practice? To do this we use Monte Carlo sampling. The ideais that we shall try to generate some number of realisations, M , of the configuration asdiscussed above. One algorithm to do this is as follows:

1. Set up a grid of cells of size 2N × 2N .

2. Begin with a polymer of length 2 and determine one cell to be the end of the polymerchain. Indicate a cell filled with polymer by a value 1 and an empty cell by a value0.

3. Use a random number generator to select with equal probability which direction togrow the chain in; we must choose between the three possible directions, and viewedfrom the addition of the last monomer cell of the polymer we can label them ahead,a = 0; left, l = 1; right, r = 2.


4. If the selected cell is not occupied (value 0) add a monomer unit to that cell andcontinue by going back to stage 3 if the length of the chain is less than N ; otherwisego to stage 2.

5. If the cell is already occupied we have failed; clear the grid and go back to stage 2.

We continue until we have tried to create M realisations, however only some fraction f(N)of these will have succeeded. Now we must assign a probability to each attempt. Clearlyfor a chain of length N we have already seen that ignoring the multi-occupancy constraintwe would generate 4×3N−2 configurations. However, the multi-occupancy constraint leadsto only f(N) being successful. Therefore we can use these results to estimate the totalnumber of possible configurations as:

wN = f(N)× 4× 3N−2

and of course we may estimate a probability for each realisation by taking the inverse ofthis.

This procedure is good for moderate N . For large N we may find that most configurationsare not allowed, and we need a large number of realisations, M , to adequately constrainf(N). Can we improve on this algorithm? We can, although the coding can become muchmore complicated. One method would be to determine at each stage of the building processhow many ways of adding the next monomer cell do not lead to the chain winding back onitself. In our 2D example we may find 3, 2, 1 or zero possible additions. We could allowpolymers to grow into empty cells, taking note of all possible ways of failing, and carefullydetermine the probabilities from this information. This will be hard to work out and hardto code. We therefore finish with a very important message:

Remember that our first requirement is to understand the physics of the prob-lem we are trying to solve. Therefore we start by using the simplest (butprobably clearest and often most accurate) computational approach. We then“let the computer take the strain”. Only if the problem proves too large tocompute in a reasonable time should we consider more complicated solutions.

Documents

University of Cambridge Department of Physics ...rachael/compphys/CompPhys2.pdf · 2 Computational Physics 1.1 Interpolation and Extrapolation Let us consider a set of N data points