Integrated Design of Hydrological Networks (Proceedings of the Budapest Symposium, July 1986). IAHS Publ. no. 158,1986.

NOTATION

A robust version of classical D-op t ima l design applied to dissolved-oxygen sag-curve calibration

Une solide interprétation de la planification optimale classique D appliquée à la calibration de la courbe concave de 1'oxygène dissous

E.A. CASMAN Interstate Commission on the Potomac River Basin, 6110 Executive Boulevard, Rockville, Maryland, 20852-3903, U.S.A.

ABSTRACT This paper presents a modification to classical optimal-design theory for parameter precision intended to make this theory more relevant to water-quality modeling. A discussion of the classical techniques and their shortcomings precedes the development of the robust method. The appropriateness of this design technique for fitting river models is examined in the context of an example from the water-quality modeling literature: fitting a simple one-dimensional steady-state model of dissolved-oxygen kinetics in rivers.

Ejj Covariance matrix of the parameter estimates k, BOD decay coefficient (day ) ko Reaeration rate (day ) t Time of travel (days) a 2 Error variance of the observations f Fitted function X Matrix of first partial derivatives of f with respect to

its parameters p Number of unknown parameters in f p() Probability distribution function n Number of samples in experimental design Uj. Expected value of k a k Variance of k

INTRODUCTION TO D-OPTIMAL DESIGN THEORY

Optimal-sampling design methods are mathematical tools for identify­ing the ideal settings of the independent variables in an experiment before the data are collected. The purpose of these methods is to improve the information content of experiments while minimizing the sampling effort. Optimal designs are typically narrow in scope, emphasizing a single goal for an experiment. One such goal is the improvement of parameter-estimate precision. A major class of

105

106 E.A.Casman

parameter-precision design criteria focuses on reducing the variance of the parameter estimates. This information resides in the covariance matrix of the parameters estimates, 2k.(k.,t_), w n i c n is

commonly approximated by the following expression̂ :

£k(k,t) = a2(X'X)-1 (1)

where the circumflex signifies an estimated quantity, underlining indicates vectors, o is the error variance of the observations, and X is the matrix of first partial derivatives of the fitted function, f, with respect to its parameters, k. The vector, _t, representing the independent variable settings of the experiment, is included as the argument of £]<. to remind the reader that the approximate covariance matrix-of the parameter estimates is a function of the sampling design. t^ is also a function of k_, but in the rest of this paper the argulnent _k will be omitted for notational simplicity. This matrix approaches the true covariance matrix of the parameter estimates asymptotically as the number of samples becomes large. The determinant of the covariance matrix is proportional to the volume of the usual 1 - a confidence ellipsoid of the parameter estimates (Cramer, 1966). This property is exploited by D-optimal designs, which are composed of samples which minimize the determinant of 2^. Actually, the D-optimal criterion (Box & Lucas, 1957) maximizes tïïe determinant of X'X, which, when a2 is constant, is equivalent to minimizing the determinant of S^.

When f is linear in its parameters, the parameters, 1c, do not appear in the X matrix of equation (1), which makes solving the D-optimality problem for a linear model a relatively simpler minimi­zation of a function of the independent variable settings. However, for f nonlinear in the parameters, a unique solution usually does not exist because both the parameter values and the independent variable settings are unknown. If estimates of the unknown parameters can be supplied, an approximate D-optimal solution can be obtained. The solution of such a problem is called a "local" D-optimal solution to emphasize its dependence on an estimated quantity.

Optimal designs represent supreme economy of sampling. Therefore, the number of samples to be taken must be the fewest possible. The minimum number of samples required to fit a curve by nonlinear least-squares regression is equal to the number of unknown parameters (p) in the function being calibrated (Atkinson & Hunter, 1966). Taking replicate samples at the optimal design settings will improve the parameter precision. The D-optimal design will be superior (in terms of volume of the confidence ellipsoid) to any non-optimal design with the same number of samples, by definition.

As alluded to previously, D-optimal designs locate the n design points (where n = p) that cause the determinant of the matrix X'X to be maximized over the range of feasible sampling points. When there are two independent response variables in the model, described by the functions f\ and f2, the D-optimal design is found by minimizing the determinant of the approximate covariance matrix of the parameters. This approximate covariance matrix is:

\(t) = a21(X1'X1) ! + a

22(X2'X2)"

1 (2)

D-optimal design and D-0 sag-curve 107

the sum of the X'X matrices for the two equations weighted by the variances of their respective dependent variables (Draper & Hunter, 1966).

A criterion for measuring the influence of the experimental design on parameter-estimate precision is necessary to the development and evaluation of optimal designs. Several authors have proposed functions for quantifying the ability of a non-optimal design to reduce a parameter-estimate variance (Ehrenfeld, 1955; Atwood, 1969; Nalimov et al., 1970; Newhardt & Bradley, 1971; and Lucas, 1974). Berthoeux & Hunter (1971) defined the efficiency of a design as the determinant of X'X evaluated at the non-optimal design divided by the value of the determinant of X'X evaluated at the D-optimal design. This (loss of) efficiency function is:

L(k,_t) = det X'XCk,_t) / det X'X(k,t*) (3)

The loss of efficiency, L, of an experimental design t_ is defined as the D-optimality criterion evaluated at _t_ divided by that criterion evaluated at the D-optimal design, t_*. This function takes on values from 0 to 1 with 1 corresponding to the D-optimal design. Berthouex & Hunter (1971) suggested using this equation to identify regions in the design space within which the basic optimal design could be modified without undue concern for loss in experimental efficiency.

An example of the contours produced by plotting the efficiency function for some f(lc,_t_) for a fixed k_ over the design space is given in Fig. 1. The D-optimal criterion is maximized for a unique experi­mental design denoted by an asterisk. Designs (points in the two-dimensional independent variable space, the design space) further from the optimal design are characterized by diminished efficiency.

Since nearly every point in the design space corresponds to an optimal design for some parameter values, the existence of this flexibility in the D-optlmal result has further implications: namely, that a design that' is optimal for one set of parameter values may simultaneously have optimal properties for other sets of parameter values.

This has been a very brief introduction to classical D-optimal design theory; however, these rudiments suffice for understanding the development and application that follows.

APPLYING D-OPTIMAL DESIGN TO A WATER-QUALITY MODEL

D-optimality is based on some very restrictive assumptions: that the underlying mechanism of the phenomenon being modeled is known, that there is error only in the dependent variable observations, that those errors are independently distributed with mean zero and constant variance, and that the true parameter values are known. Water-quality models are invariably inaccurate simplifications of the processes they represent and are subject to error of unknown distri­bution in model structure, parameter values, independent variable measurements, as well as in the observations. This paper will demonstrate that, though the magnitudes of these kinds of inter­ference are large, they can be tolerated when fitting a certain type of water-quality model by a D-optimal design. Furthermore, a method

108 E.A. Casman

2.00 2.50 3.00 3.50 4.00 4.50 5.00 5.50 6.00 6.50 days

FIG.l Efficiency contours on a two-dimensional design space (from Casman et al., 1986).

for improving the robustness to parameter uncertainty of the D-optimal design will be presented.

One of the simplest water-quality models is used for illustrative purposes in the following development of a robust D-optimal design. It was proposed by Streeter & Phelps (1925) to predict the dissolved-oxygen (DO) concentration and biological-oxygen demand (BOD) at any point in a one-dimensional river.

The amount of oxygen dissolved in the water column is a crude

D-optimal design and D-0 sag-curve 109

indicator of the general health of the ecosystem and has been used to describe stream quality for years. The Streeter-Phelps model states that the rate of change in the BOD concentration is proportional to the BOD present, and that the change in the dissolved-oxygen concen­tration is controlled by the decay of BOD and by the movement of gaseous oxygen into the stream from the atmosphere. The latter is a first-order process relative to the DO deficit, which is the satura­tion concentration of oxygen in the water column minus the actual DO concentration. It assumes a reach of stream with homogeneous physical features, a single constant point source of BOD, and steady-state processes. In differential form, the model reads:

dB0D(t)/dt = -kiBOD(t)

dD0(t)/dt = -k!B0D(t) + k2(D0(SAT) - D0(t))

(4a)

(4b)

,-1 The units of k, and k2 , the unknown parameters, are days l, and the units of BOD and DO are mg/L. DO(SAT) is the saturation concentra­tion of dissolved oxygen for the reach, and t is traveltime (in days). t is used interchangeably with distance downstream of the point source BOD input when constant stream velocity is assumed. Time of travel is transformed into distance by multiplying it by the velocity of the stream.

In explicit form, the Streeter-Phelps model is:

BOD(t) = BOD(0) e-k-lt

kit -k0^ D0(t) = DO(SAT) - k1BOD(0) (k2 - kj) 1 (e Kl L - e K2L)

- (DO(SAT) - D0(0)) e"k2t

(5a)

(5b)

where the argument "0" refers to an initial condition. This two-equation model is nonlinear in the parameters so entries of the X matrices will be functions of the unknown parameters. By calling the BOD equation "f]/' and the DO equation "f2," the X matrices for an n-sample experiment (samples taken at unknown times t\, ..., tn) are constructed:

Xf,

afiCt^/akx

3fl(tn)/3ki

and

Xf.

8f2(t1)/3k1

3f2(tI1)/3k1

3f2(t1)/3k2

3f2(tn)/3k2 kl'k2

110 E.A. Casman

These matrices are evaluated at k̂ and k2, the initial parameter estimates.

D-optimal designs offer a method for improving parameter precision only if the model and true parameter values are known with certainty. In water-quality modeling, one can be assured that these two require­ments will not be met. All river models are simplifications and approximations of the processes actually taking place and can never be considered precisely correct. Furthermore, the true parameter values are never known. Appropriately, the only direct applications of D-optimality suggested in the water-resources literature have been for those models linear in the parameters, models whose D-optimal design criterion is independent of the parameters, namely simple linear time-series models (Watts & Minich, 1972).

One practical way to overcome the requirement that the parameters be known in advance is by a "sequential design" (Fedorov, 1972). In

FIG.2 Map of Antietam Creek in Maryland (adapted from Maryland Geological Survey, 1971).

D-optimal design and D-0 sag-curve 111

this method, sampling proceeds in stages: a few samples are collected for calculating initial parameter estimates. A local D-optimal sampling design is determined for the parameter estimates, and more samples are collected according to that design. The process proceeds iteratively, converging upon the true parameter values and the global D-optimal design simultaneously.

This paper will outline a modification of this method that abbreviates the sequential design effort. This modification is based on the optimization of a function of the D-optimal efficiency criterion of equation (3).

DEMONSTRATION OF A D-OPTIMAL DESIGN

Before proceeding with the development of a novel experimental design, a demonstration that the standard D-optimal design can be applied to the simple BOD/DO model given in equations (4a) and (4b) is warranted.

The data for this demonstration were collected on Antietam Creek in the United States. This creek serves as a major drainage system for the Hagerstown Valley in west-central Maryland. It flows through a limestone valley in Maryland's valley and ridge physiographic province to join the Potomac River (Fig.2). The 35.7-km portion of Antietam Creek below the town of Hagerstown can be considered a homogeneous reach for three reasons: (1) the tributary input south of Hagerstown is negligible at low-flow conditions, (2) all sewage-treatment plants on Antietam Creek are located above this reach, and (3) the reach is characterized by small riffles and pools over its entire length.

BOD and DO data were collected on three consecutive days in August 1968 by the Maryland Department of Natural Resources (unpublished data). The data are presented in Table 1. A total of 30 samples at six fixed sites we're collected. By using the following constants: D0(0) = 3 mg/L; DO(SAT) = 8 mg/L; and BOD(0) = 15 mg/L, the Streeter-Phelps model was fit to the data in Table 1 by nonlinear least-squares regression. The results of this regression were k̂ = 1.47 day-1; k2 = 3.64 day"

1; ° 20 D = 14.51; and <3

2DO = 1.42.

Various algorithms have been devised for locating D-optimal designs (Fedorov, 1972; Kiefer, 1971; Mitchell & Miller, 1970; Mitchell, 1974; and Wynn, 1972). None of these algorithms are guaranteed to give a D-optimal design. At present, it appears that an optimum can be guaranteed only by an exhaustive search of all possible designs. When the dimension of the problem is small and the design space limited, as is the case with the Streeter-Phelps example, a search of the design space is not only feasible, but also instructive. As pointed out by Fedorov (1972), such a search can reveal details of the objective function surface that would not ordinarily be discovered by some other method of solution, such as ridges or multiple-local optima, which may have practical implica­tions.

A simple search method was used to locate the D-optimal designs in this paper. The search was accomplished by representing the design space (sampling times) by a network of equidistant discrete points and identifying the values the D-optimality criterion at these

112 E.A.Casman

TABLE 1 Antietam Creek water-quality data

River BOD DO Traveltime n (mile) (mg/L) (mg/L) (days)

1 2 3 4 5 6

7 8 9 10 11 12 13 14 15 16 17 18

19 20 21 22 23 24 25 26 27 28 29 30

22.3 20.3 17.7 15.1 13.2 4.4

22.3 22.3 20.3 20.3 17.7 17.7 15.1 15.1 13.2 13.2 4.4 4.4

22.3 22.3 20.3 20.3 17.7 17.7 15.1 15.1 13.2 13.2 4.4 4.4

August 19,

13.5 1.7 1.0 3.8 4.7 4.8

August 20,

14.0 22.0 9.6 6.0 5.0 3.0 9.8 2.7 3.9 1.0 4.3 1.0

August 21,

7.4 16.0 3.6 13.0 12.0 8.0 0.8 2.6 0.6 5.0 0.9 2.3

1968

4.1 3.2 4.4 7.4 7.9 8.0

1968

1.2 3.3 3.7 3.5 3.2 3.8 4.6 4.8 8.5 7.2 8.1 8.0

1968

3.1 0.8 2.6 5.0 3.3 4.2 4.6 6.1 7.2 8.6 7.2 9.5

0.00 0.29 0.75 0.92 1.38 2.08

0.00 0.00 0.29 0.29 0.75 0.75 0.92 0.92 1.38 1.38 2.08 2.08

0.00 0.00 0.29 0.29 0.75 0.75 0.92 0.92 1.38 1.38 2.08 2.08

points. In the neighborhood of the maximum D-optimality value, a smaller subspace was defined and searched at a finer scale. The process was repeated until the desired resolution was achieved. This method is satisfactory for up to three-dimensional problems, but becomes unwieldly in terms of computational effort in higher dimen­sions.

D-optimal design and D-0 sag-curve 113

The D-optimal design for Antietam Creek located by this search method was composed of two sample times, both at 0.60 days. The sampling station corresponding to traveltime 0.75 days was the closest station to the D-optimal sampling time and was used as its surrogate. The D-optimal efficiency, as defined by equation (3), of a design consisting of samples at 0.75 days is 88%:

det Sk(0.60, 0.60) / det £k(0.75, 0.75) = 0.88

Here the approximate covariance matrix of the parameter estimates evaluated at k_ substitutes for X'X in the efficiency measure.

The data set consists of five samples taken at each of the six sampling stations. Thus, the largest surrogate D-optimal design permitted by the data set has five samples, all taken at 0.75 days. The Streeter-Phelps model was fitted to each of 14 five-point subsets. Because of the limited data, the criterion for judging the success of a design was its D-optimal efficiency relative to the five-point design with highest D-optimal efficiency (as opposed to its efficiency relative to the true or local D-optimal design). This measure will be called the "relative" D-optimal efficiency.

L(tj) = det 2k(_ti)mln / det £k(tj)

i,j = 1,2, ..., 14 (6)

The experimental designs, the entries of the covariance matrix of the parameter estimates they engendered, and the relative efficiency of each design are listed in Table 2. The five numbers in paren­theses under the column heading "data" identify each datum according to the first column of Table 1.

As anticipated, the approximate D-optimal design (design 1 of Table 2) achieved the smallest determinant of Ek of any of the designs tested. Recall that this determinant is" proportional to the volume of the confidence region for the parameter estimates. D-optimal design methodology was capable of identifying the samples that produced the best parameter estimates, even in a system that did not conform strictly to the D-optimality assumptions.

This demonstration is suggestive, though not definitive. It should be remembered that because of the small sample size (n = 5), these relative coefficients are provisional and will change with more data.

The success of a D-optimal design is determined in some part by the degree to which reality conforms to the assumptions made when solving for the D-optimal design. Violation of these assumptions invalidates both the form of the covariance matrix of the parameter estimates approximation and the D-optimal design derived from maximizing the determinant of X'X. Of all the D-optimal assumptions, a D-optimal design is especially sensitive to uncertainty in the parameter estimates and model structure. A design criterion is now offered for improving the robustness of D-optimal designs to uncertainty in the parameter values.

The term "robust" is used to describe procedures that give good results when underlying assumptions are violated. Because the

114 E.A. Casman

TABLE 2 Regression results for five sample designs

Design

lb 2C

3d 4e

5d 6d 7 8 9 10 IIe

12 13 14

Data3

(3,11,12,23,24) (4,13,14,25,26) (1,7,8,9,20) (2,9,10,21,22) (5,15,16,27,28) (6,17,18,29,30) (11,13,15,18,20) (22,24,26,28,30) (1,10,13,16,23) (2,3,4,5,19) (2,5,9,16,17) (2,3,7,8,9) (4,12,18,19,21) (2,8,9,14,29)

VAR(kp

0.547 0.145

OO

0.862 0.037 0.019 0.039 0.036 0.271 1.829 0.885 3.091 1.302

35.528

VAR(k2)

0.894 0.738

OO

4.207 12.242 154.284 0.950 2.281 0.978 3.977 3.845 18.370 2.533 0.128

C0V(k1,k2)

0.689 0.254 0

1.852 0.000 0.092 -0.088 0.102 0.475 2.562 1.826 7.412 1.695 0.124

Relative efficiency

100 30 -7 3 0 48 19 19 2 20 1 3 0

aFrom Table 1. "Surrogate D-optimal design. cSurrogate minimum-risk D-optimal design. "No convergence. optimization halted at k2 upper bound.

correctness of the assumptions determines the validity of the D-optimal design, questions of robustness are especially relevant.

DESIGN ROBUST TO ERROR IN THE PARAMETER ESTIMATES

As mentioned earlier, one way of dealing with uncertainty in the parameter values is to plan experiments that are carried out in stages, with each successive experiment being based on the results of the preceding experiments. An alternative to the iterative method is now proposed. This method combines the probability distribution function of the parameters of the parameter estimates with the concept of D-optimality to derive a new design criterion that will shorten the sequential process and produce a design that is more robust to uncertainty in the parameter values.

A plausible prior probability distribution for the unknown parameters of the model is a multivariate normal distribution with the mean and variance of the initial parameter estimates as its parameters. The mean and variance of the parameter estimates are generated by least-squares regression on a small set of data.

Recall the D-optimal efficiency function LOO of equation (3). It describes the loss of efficiency resulting from sampling at any n points (where n is the dimension of t) other than the n-point D-optimal design. The expected risk of designing for the wrong

D-optimal design and D-0 sag-curve 115

parameter values k_ for any sampling design, _t_, is the expected value of the loss function, equation (3), evaluated at t_:

R(t) = / L(k,_t) pQO dk_ (7)

This is the integral with respect to the parameter values of the loss function times the probability of the parameter values. It is called the "risk function." The discrete version of equation (7) is

„m R(t) = I Lfe.t) pCkO (8)

j=l J

which would be employed when m sets of parameter vectors kj and corresponding weights p(_k_-j) are chosen to represent the pdf (probability distribution function) of le, where

„m

I p(k-j) = 1 and 0 < pCy) < 1 j=l

A design _t** that maximizes the risk function (equation (7) or (8)) over the design space will be called a "minimum risk" D-optimal design. A schematic representation of this concept is given in Fig.3, in which k is some parameter having the univariate proba­bility distribution function of p(k). For the Streeter-Phelps model, p(_k), a bivariate normal distribution replaces p(k), resulting in a four-dimensional design problem. Each horizontal level of the figure's three-dimensional array depicts an efficiency map for a different parameter vector, _k-j, as in Fig. 1. The horizontal axes of each map are labeled t]̂ and t2, defining the design space. Verti­cally summing the j maps weighted by their probabilities at each point (t^,t2) produces an expected risk surface over t_. This approach requires the same assumptions as D-optimality with the additional assumption that p(k) is known.

DEMONSTRATION OF THE MINIMUM-RISK DESIGN USING SIMULATION

By way of example, a minimum-risk design will now be calculated and evaluated by simulation. First some data (with which to calculate the least-squares parameter estimates and their covariance matrix) are generated. Each datum is composed of the sum of the evaluation of equation (5a) or (5b) at some experimental design point, t, and a random error term, e-̂ .

Yi(t) = BOD(t) + ei (9a)

Y2(t) = DO(t) + e2 (9b)

The experimental design consists of 10 samples taken at 1-day intervals, (t = 1,2, ..., 10 days), and the errors, e-̂ , are distri­buted normally, e± ~ N(0,a2i), where a21 = 1.0 and a2

2 = 0.25. The observation variances are:

116 E.A. Casman

2± M,

^ ^

FIG.3 The concept of a minimum-risk D-optimal design (from Casman et al., 1986).

1.00

0.00

0.00

0.25

and the boundary conditions and true parameter values are:

BOD(0) = 15 mg/L D0(0) = 7 mg/L DO(SAT) = 8 mg/L k1 = 0.23 day k2 = 0.50 day 1

These constants were chosen as representative of typical settings for small streams of moderate velocity and small pollution load. The least-squares parameter estimates and their variances calculated from these data became the parameters of a bivariate normal distribution

D-optimal design and D-0 sag-curve 117

for kj and k2 with marginal distributions: k̂ ~ N(0.238, 0.0006) and k2 ~ N(0.526, 0.0016). The discrete form of the risk function, equation (8), was used to calculate the risk surface with p(k_) being a discrete approximation, the bivariate normal distribution covering a range of ^ ^ ± 3a2

k , i = 1,2 (Fig.4). In this figure, the height of each block is proportional to the probability of a different pair of kj and k̂ . The maximum of the risk function was located by evaluating the function over the entire design space.

P(k,.k,)

FIG.4 Discrete approximation of a bivariate normal distribution.

The minimum-risk D-optimal design for this pdf is _t** = (4.5, 4.5) days. The test of a minimum-risk design is how well it performs over the possible range of the parameter estimates. The efficiency of t_** relative to the D-optimal design for each different parameter setting is presented in Table 3. The efficiency of the minimum-risk design is calculated relative to the D-optimal design specific to the parameter pair under consideration, t*k

L(k-j,0 = 2k (t*k ) / lk (4.5,4.5,4.5,4.5,4.5,4.5,4.5,4.5) (10) -j -j -j

A design size of n = 8 was chosen to accommodate a comparison with a non-optimal design consisting of _t_ = (1,2, ..., 8) days. The minimum-risk design provided 90% D-optimal efficiency to 84% of the values listed in Table 3 and 95% D-optimal efficiency for nearly 68% of them. By contrast, a design consisting of _t_ = (1,2, ..., 8) was less than 85% efficiency for all parameter values.

118 E.A.Casman

TABLE 3a D-optimal efficiencies of the minimum-risk design, _t_= (4.5, ..., 4.5) days for a range of parameter settings

0.4060 0.4660 0.5260 0.5860 0.6460

(k2 day-1)

0.1645 0.2013 0.2382 0.2570 0.3118 (kj day l)

0.79 0.90 0.97 1.00 0.98 0.86 0.95 0.99 1.00 0.96 0.92 0.98 1.00 0.98 0.93 0.95 1.00 0.99 0.95 0.88 0.99 1.00 0.97 0.92 0.83

TABLE 3b D-optimal efficiency of non-optimal design, _t_ = (1,2,3, ..., 8) days for a range of parameter settings

0.4060 0.4660 0.5260 0.5860 0.6460

(k2 day-1)

0.1645 0.2013 0.2382 0.2570 0.3118 (kl day *)

0.59 0.66 0.72 0.82 0.80 0.59 0.64 0.67 0.75 0.71 0.60 0.62 0.63 0.69 0.64 0.60 0.61 0.60 0.64 0.58 0.61 0.59 0.58 0.61 0.54

In this example, the correct model and parameter values were known in advance and the error was controlled to conform with the least-squares assumptions. A demonstration of the minimum-risk D-optimal design on actual river data follows, using the Antietam Creek data from Table 1.

MINIMUM-RISK D-OPTIMAL DESIGN FOR ANTIETAM CREEK

A minimum-risk design for this system was based on the parameter estimates and error distributions calculated from the entire 30-point data set. The minimum-risk design consists of samples taken at 0.88 days. Such a design has a theoretical D-optimal efficiency of 70%, as calculated below:

det E\(0.60, 0.60) / det 2k(0.88, 0.88) = 70%

The design most closely approximating the correct minimum-risk design composed of data from Table 1 consists of five samples taken at 0.92

D-optimal design and D-0 sag curve 119

days. The theoretical D-optimal efficiency of this design is 63%:

det fk(0.60, 0.60) / det £k(0.92, 0.92) = 63%

a difference of 7% efficiency from the correct minimum-risk design. Thus, the surrogate design should resemble the correct design reasonably well (though not perfectly). The results of the compari­son between 14 five-sample designs have already been presented in Table 2. The smallest determinant of the covariance matrix of the parameter estimates was achieved with the surrogate D-optimal design, design 1. The surrogate minimum-risk design, design 2, was only third best by this criterion.

We can look at the results in another way, however. Given two n x p matrices, matrix M is said to be "smaller" than a matrix N (M < N) if every element of M is smaller than the corresponding element in N (m-y < n-jj for all i,j). By this criterion, the covariance matrix of the parameter estimates produced by the surrogate robust design was smaller than that produced by the surrogate D-optimal design (design 1).

CONCLUSION

It should be noted that the ranking of these designs according to their D-optimal efficiency is provisional at best. A sample size of 5, considering the magnitude of the observation error and the asymptotic nature of the covariance matrix approximation, is too small to assure stable ranking. However, even with these caveats, these results, in addition to showing that local D-optimal and robust D-optimal designs are of value when fitting the Streeter-Phelps model, demonstrate that the error inherent in a river system and the violations of D-optimal design assumptions of the kind and magnitude encountered with real-river data are tolerable in practice.

REFERENCES

Atkinson, A.C. & Hunter, W.G. (1968) The design of experiments for parameter estimation. Technometrics 10, 271-281.

Atwood, C.L. (1969) Optimal and efficient designs of experiments. Ann. Math. Stat. 40(5), 1570-1602.

Berthouex, P.M. & Hunter, W.G. (1971) Statistical experimental design: BOD tests. JEED Proc. ASCE, Proc. Paper 8286, 97(SA4), 393-407.

Box, G.E.P. & Lucas, H.L. (1959) Design of experiments in nonlinear situations. Biometrika (45), 77-90.

Casman, E.A., Naiman, D.Q. & Chamberlin, C.E. (1986) Two uses of a D-optimality efficiency function for designing experiments for dissolved oxygen sag curve calibration. Submitted to Wat. Resour. Res.

Cramer, H. (1966) Mathematical Methods of Statistics, 2nd edition. Princeton Univ. Press, Princeton.

Draper, N.R. & Hunter, W.G. (1966) Design of experiments for

120 E.A. Casman

parameter estimation in multiresponse situations. Biometrika (53), 525-533.

Ehrenfeld, S. (1955) On the efficiencies of experimental designs. Ann. Math. Stat. (26), 247-255.

Fedorov, V.V. (1972) Theory of Optimal Experiments. Academic Press, New York.

Kiefer, J. (1971) The role of symmetry and approximation in exact design optimality. In: Statistical Decision Theory and Related Topics, S.S. Gupta & J. Yachel, eds., Academic Press, New York.

Lucas, J.M. (1974) Optimum composite designs. Technometrics 16, 561-567.

Maryland Geological Survey (1971) Traveltime and concentration attenuation of a soluble dye in Antietam and Conococheague Creeks, Maryland. Information Circular 12. The Johns Hopkins Univ., Baltimore.

Mitchell, T.J. (1974) An algorithm for the construction of D-optimal experimental designs. Technometrics 16, 203-211.

Mitchell, T.J. & Miller, F.L., Jr. (1970) Use of design repair to construct designs for special linear models. Math. Div. Ann. Prog. Rept. (ORNL-4661), Oak Ridge National Laboratory, Oak Ridge, Tennessee, 130-131.

Nalimov, V.V., Golikova, T.I. & Mikeshina, N.G. (1970) On practical use of the concept of D-optimality. Technometrics 12(4), 799-812.

Newhardt, J.B. & Bradley, H.E. (1971) Selection of multi-factor experimental arrangements with resource constraints. J. Am. Stat. Assoc. (66), 618-621.

Streeter, H.W. & Phelps, E.B. (1925) A study of the pollution of the Ohio River, part III, Factors concerned in the phenomena of oxidation and reaeration. U.S. Public Health Service,., Public Health Bulletin, 146.

Watts, D.G. & Minich, G.M. (1972) Design of experiments for estimating process dynamics. In: International Symposium on Uncertainties in Hydrologie and Water Resource Systems, C.C. Kiesel & L. Duckstein, eds., Tucson, Arizona, 667.

Wynn, H.P. (1972) Results in the theory and construction of D-optimum experimental designs. J. Roy. Stat. Soc. , B-34, 133-147.