CBMR

76261985197

IIIII I IIII I INI II I II II III II I II IIIII INI I lul N I Ifaculteit der economische wetenschappen

RESEARCH MEMORANDUM

ILBURGr UNIVERSITY

EPARTMENT OF ECONOMICS

istbus 90153 - 5000 LE lilburgetherlands

f~~i K.U.E3.~-f~~ F~~BL~OTHEEK

~ TJLBURG

FEW197

VARIANCE HETEROGENEITY

IN

EXPERIMENTAL DESIGN

Dr. Jack P.C. Kleijnen

Dr. Jack P.C. KleijnenProfessor of Simulation and Information Systems

Department of Information System and Accountancy

School of 8usiness and Economícs

Catholic University Tilburg (Katholieke Hogeschool Tilburg)

5000 LE Tilburg, Netherlands

September 1985

Keyworda: replication; eatimated weighted least aquarea; optimalexperimental deaign; 2k p deaigna; aequentialization; two-atagedesign.

1

VARIANCE HETEROGENEITY IN EXPERIMENTAL DESIGN

Dr. Jack P.C.KleijnenProfesaor of Simulation and Information Syetema

Department of Informatíon Systems and AccountancySchool of Business and Economícs

Catholic University Tilburg (Katholieke Hogeachool Tilburg)5000 LE Tilburg, Netherlands

ABSTRACT

In many experiments (especially simulation experiments)the response variances ai2 differ substantially. These variancesa12 can be estimated through replication (ín simulation throughdifferent random numbera seeds). The experiment can be analyzedthrough Estimated Weighted Least Squares, or Corrected LeastSquares, which use the variance estimators ai2. Even if the de-sign matrix is orthogonal (like in 2k p designe), the resultingestimatora of the effects s become mutually dependent. Varianceheterogeneity means that classical designs are not necessarilyoptimal. Fortunately, optimality is not really important inpractice. More important is to have designs with a small numberof factor-level combinations (which still yield unbiased esti-mators of g) , permitting validation of the linear regresaionmodel, and providing flexibilíty including sequential experimen-tation. A heuristic two-stage procedure is proposed, which com-binea classical desígns with a number of replicatione euch thatthe variancea are approximately conetant per average response.

z

1. INTRODUCTION

It is amazing that in the vol~inous literature on the

design of experiments, virtually no attention is paid to theheterogeneity of the variances of the responsea yi(i-1,...,n) :

oi2 s a2. For example, a recent review on experimental design bySteinberg and Hunter (1984) doea not even mention the issue of

variance heterogeneity. Our experience (in the design of experi-

ments with simulation models, not real-life systems) is that theresponse variancea differ substantially, for example, Rleijnen,

Van den Burg, Van der Ven (1979, p.60) report a 26-Z experiment

where aiz (1~1,...,16) ranges between 64 and 93,228. In simula-

tion, estimating response variances is easy when compared toexperimenta with real-life technical systema (chemical plants,

agricultural plots) and eocio-technical systems (organisations

like business enterprises): Dykstra (1959, p. 63) points outthat in real (non-simulated) systems true replication maq be

difficult. In simulation, replication meana that one and thesame eimulation program is executed m times, with m different

random number seeds. Then

Ei (Yir - Yi )2Q12 ~ r:l (i~l,...,n) (1)

mi - 1

is an unbiased estimator of o12 - var (yir) - var (yi).

When simulating dynamic systems (Eor example, queuingsystems) other estimators of ai2 are poasible (using subruns,spectral analysis, and so on; see Kleijnen, 1975, 1986). Theseestimators are more complicated, and they may be biased. There-fore we discuss only the estimators of eq. (1). In real-lifeexperiments the variance estimators are also of the type dis-played in eq. (1). In non-experimental sciencee (like economics)

3

variances are estimated from reaiduals; see Horn, Horn, Duncan(1975); we do not discuss theae eatimators (which may again bebiased and complicated).

As we mentioned above, the variance eatimatea in simula-tion usually indicate subatantial heterogeneity. Apart fromthese empirical reaults, it seema atrange to assume that theexpected responae depends on the input vector xi, or E(yi) eE(y ~xi) ~ B' xi, but the variance, or var (y1 )~ var (y ~xi) ~oi2, does not depend on the input (so that oiZ ~ 02).

We note that in determiniatic aimulation the assumptionof common variances, seems more realíatic; see Rleijnen (1986).

2. VARIANCE HETEROGENEITY IN 2k p DESIGNS

In case of variance heterogeneity we may decide to con-tinue using classical experimental designs such as 2k-p deaigna.For the analysis of the experimentaltiona:(a) Ordinary Lesetvariance differences.(b) Corrected Leastpoint estimator

Squares

Squarea

(OLS),

S a(7~' x)-I x' Y

is combined with the correct covarlance matrix

szs ~ (x'x)-I X, ny X (X,X)-I

data there are aeveral op-

i.e., simply ignore the

i.e., the (unbiased) OLS

(z)

(3)

where X is the N x q matrix of independant variables, i.e.,thereare q independant variables, combined in n different ways, eachcombination i(i-1,...,n) replicated mi times so that

4

The vector y has N components: Y~(yll'~~~'ylm '" ~'ynl'~~~Ynm )~ 1

n(c) Estimated Weighted Least Squares (EWLS), i.e., the estimatedresponse variancea ( see eq.l) yield the (dlagonal) covariancematrix n(the first ml diagonal elementa

Y012,..., the last mn elementsunbiased point estimator

are all equal toare an2 ) which results ín

s~ (X, ity 1 7C)-1 R, IIy 1 y

wíth asymptotic covariance matrix

n., s (X, ny 1X)-1s

the

(a)

(5)

provided certain mild technical aasumptions hold; eee Schmidt(1976, p.71).OLS yields conservative tests; EWLS reaults in valid tests pro-vided the ntmmber of replications in large, say mi ~ 25, and CLSgives valid tests; see Kleijnen, Cremers, Van Belle (1985). Inthe case-atudy reported in Kleijnen, Van den Burg, Van der Ham(1979) both CLS and EWLS were applied; these two analysis tech-niques gave different quantitative resulta (point estimates ofS) but identical qualitative resulta (which factora are reallyimportant?).

What happens if we apply CLS and EWLS to claseical de-

signs like 2k-p designa? We consider a simple aituation, namelythe (N x q) matrix of independent variablea X in orthogonal.

Such a situation occurs Sf we use a 2k p design with a conatantnumber of replications (mi - m~ 1, 2,...), i.e., in that si-tuation we have X' X- N I(with N ~ n m). If the number of re-plicattona ís constant (mi ~ m), then

E Ei (Yir - yir)2 ~i~l ral

5

1 r(yir - yi)Z f m i(Yi - yi)2 (6)

so that identical least squares point estimatore (s) result if

we fit the regression model to the average responaes (yi). Let

the vector of average responses be y~(yi) and let the matrix

of independent variables with elímination of identical (repli-

cated) rows be X~(xi~) where i~ 1,...,n and j a 1,...,q and

n~ q. Then eq. (2) becomes

s 3(x~x)-1 X~ Y a(n E X1 ri) (~-1,...,q) (~)i-1 ~

and eq. (3) reduces to

Sts ~ 12 (E zi~ ai2 xij,) (j'~1,...,q)n i

(8)

where oi2 ~ var(yi). Then S2S is nót a diagonalwmatrix, eventhoug X is orthogonal! (All effect estimators gj have a commonvariance, namely E ai2~n2.) EWLS alao results in mutually depen-dent estimators; see eq. (5).

3. OPTIMAL VERSUS SATISFICING DESIGNS

If the responaes (yir) had constant variances (oi2 ~a2), then OLS would yield the Best Líneac Unbiased Estimator(RLUE), assuming the other claesical ass~ptions also hold(namely independence, and correct model specifícation, which arediscussed in Kleijnen, 1986). If indeed Sty ~ a2 i, then eq. (3)reduces to the classical formula:

52S 3 02 (X'X)-1 (9)

6

Further Box (1952, p. 50) proved that the diagonal elementa of

eq. (9) (the variances of g~) are minimal, if X is orthogonal(X' X ~ N I).

In (simulation) practice, the reaponaea yir have (sub-

stantially) different variances (oi2 ~ a2). Then we do not know

whether an orthogonal design still minimizes the variancea of

the S. And an orthogonal design does~

not make the estimatorsmay be dependent, then therelevant, and different cri-

g, independent. If the estimatoraJ

off-diagonal elements of n„ becometeria result for definings theJohn and Draper, 1975):

optimality of designs ( aee St.

(i) D-optimality: minimize the "generalized variance", orthe Determinant of its.

(11) A-optimality: minimize the Average varíance or the trace

oE 52~.

(iii) E-optimality: minimize the maximwa F.igenvalue of S2S.(iv) G-optimality: minimize the maximum value of the mean

squared error between the true responae surface and the

least-squares estimated eurface.

Hedayat (1978) gives a bibliography on optimal design theorywith 312 publications. However, in general this theory asaumeaconstant variances. Besides this theory, there is a sizable

(leas theoretical) literature on Response Surface Methodology(RSM) designs which uses second-order polynomial regreeaion mo-dels. RSM designs may be "rotatable" ( the variance of the eati-mated response is constant at a fixed distance from the centerof the design). The designs may alao minímize bias caused bypossible third-order effects. Unfortunately theae design proper-

ties hold only if the responae variances are constant. See Dra-per (1982). Kleijnen (1986).

In other words, many deaigns are optimal, only underunrealistic assumptions. Simon (1960) emphasizes that in prac-

7

tice humans do not try to optimize ( for different reaeons, forexample, they have no unique criterion); instead they try tosatisfice, i .e., they have a number of restrictiona or aspira-tions that they try to meet. Box and Draper (1975) formulate 14criteria. We highlight the following practical requirements.

(i) A small number of factor-level combinattons

The number of factor-level combinationa (n) dependa onthe number of factora ( k) and the number of levele per factor(L j with j ~ 1, ... ,k) . The number of levels depends on the mo-del: a first-order model requires two or more levels, and asecond-order model with pure quadratic effects requires three ormore levels.

In the pilot phase of a(simulation) experiment the num-ber of. conceivably Smportant Eactors may be very high. Groupscreenínq enables us to investiRate ( say) hundrede oE factors ina small number of runs (n ~~ k); see Kleijnen ( 1975, 1986).

Zf the number of factora k is reasonably amall, then we

formulate a model with q parameters, where q equals lfk in a

first-order model ( qualitatíve and quantitative factora); q iYr-

creases with k(k-1)~2 if we include two-factor interactions

(qualitative factors); and q further íncreases with k if we use

a second-order polynomial ( quantitative factors). The number of

combinations cannot be smaller than the number of parameters, if

we desire ( unbiased) least squares parametera. For firet-order

models this condition is satisfied by rcaolutíon III designs

such as Plackett-Burman designs. For higher-order modela we uae

resolution V designs for qualitative factors, and RSM designe

for quantitative factors. These designs are tabulated in several

publications. Under variance heterogeneity, these designs are

not "optimal" but they certainly provide matrices of independent

8

variables (X) that are not collinear, i.e., they yield unbiasedestimators.

(ii) Model validation

A scientist should always check whether the model uaedto derive conclusions, is correct. We recommend to calibrate thelinear regression model (i.e., quantify the model parametera)from (say) nl runs and to validate the calibrated model from(say) n2 other runa; see Kleíjnen (1983) for a test procedure.

We can explain w~ a first-order model ia inadequate, ifthese n2 runs together with the nl old rune, form a resolutionIV design. We also refer to the criterlon of "desirable confoun-ding patterns", atated in the literature. The latter criterionmeans that in general low-order effecta should be confounded,not with other low-order effects but wíth high-order effecta.For many designs we can apriori determine how effects arealiased.

(iíi) Flexibility of the design: sequential experimentation

We prefer sequential (etagewise) experimentation; aeeKleijnen (1986). Obvioualy, ad hoc experimenta are more flexiblethan are atandard statistical designs. However, ad hoc experi-mentation has other diasadvantages: bias, too many runa, lack offirm guidelines, and so on. Many classical deaigna have certainrestrictions: the number of runs muat be a power of two or amultiple of four; the next step in the staRewise design is big;all factors have to levels; and so on. These restrictions arerelaxed, if we are prepared to use more sophisticated deaigne(mixed designs, nested deaigns, mixture designs). In the nextsection we shall propose a sequentíal approach combined withclassical designs.

9

4. A HEURISTIC APPROACN

We temporarily assume that the responae variances ai2are known. Intuitively it seems wise to obtain more replicatlonsof those experimental conditions that show high variabilíty. If,more apecifically, we take

m - c a 2 (i~l,...,n)i i (10)

then var (yi) - l~c where c ie a positive constant ( we neglectthe condition that mi should be integer). Next we fit the re-gression model to the average responsea yi uaing least squares(Ordinary and Weighted Least Squares point estimatora coincide).If the n combinationa yield an orthogonal matrix X, then eq.(8) reduces to

1S2S ~ nc I

Thereforesee eq. ( 1). These estimatora oi2 yield estima-eq. (10). Further reaearch is neceasary toconfidence intervals for B reaulting from this

itake a pilot sample (m10- 5?)wwhich yields the

Actually the response variances a 2 are unknown.we~ 2estimators oi ;

tors of mi in

determine theheuristic two-stage approach ( and multi-stage variations).

A problem arises, if we apply this heuristíc rule to asituation with mutiple responses. If different responee typeahave substantially different variancea, then our approach breakedown. And we cannot find much help in the literature either,since experimental designs for multiple responses are virtuallynon-existent. We can use the standard designs as if there isonly a single response; these deaigns are then certainly not"optimal"; for their analysis we might use Corrected and Esti-

lo

mated Weighted Least Squares per responae type, combined withthe Bonferroni inequality.

(Estimated) Weighted Least Squares and (Ordinary) LeastSquares point estimators become identical (and equal tonX 1 y) if the design is saturated: q a N~ E mi or mi ~ 1(i.e.

saturated designs yield point estimators ins~ensitive to variance

inequality). However, it doea not seem wise to obtain no esti-

mators of the variance per factor combination (mi~ 1). Our

heuriatic implies that standard deaiRns are ueed to determine

which factor combinations to investigate (strategic iasue),while classical variance estimatora are used to determine how

long each factor combination should be inveatigated (tactical

issue). Balancing the (computer) time devoted to these two is-

sues, requires more research (the current literature asaumesconstant variances; see St. John and Draper, 1975 and Welch,

1985).

We no[e [hat our heuristic deviates from the rule which

prescribes an equal nwnber of replications (mi-m) so that the

power of the ordinary Analysis of Variance F etatistic is, hope-

fully, not affected; see Scheffé (1964, pp. 345, 350, 352).

Neither does our heuristic involve variance atabilizing trans-

formatíons. We emphasize that the interpretation of the experi-

mental data should be in terms of the original (non-transformed)

obaervations. See Scheffé (1964, pp. 364-368) and the references

in Hoyle (1973) and Kleijnen (1986).

5. CONCLUSIONS

Variance heterogeneity is often substantíal in (eimula-tion) experiments. Therefore we ahould always obtain estimatesof the response variances ai2 (i~l,...,n) uaing replication. Weuse these estimators ai2 to analyze the experiment, applying

11

F.stimated Weighted Least Squares and Corrected Leas[ Squarea.

Classical designs are no[ optimal anymore. We proposed a two-

stage heuristic procedure, replicattnR experimental condition i

so often that the averaKe reaponae yi has a constant variance

approximately. The point estimators of the regression parame-

ters S become computationally simple, and statistically inde-

pendent, if we combíne our heuriatic with (clasaical) orthogonal

designs.

REFERENCES

Box, G.E.P. (1952). Multi-factor designs of first order.Biometrika, 39,no. 1: 49-57.

Box, G.E.P. and N.R. Draper (1975). Robust designs. Bíometrika,

62, no. 2: 347-352.Draper, N.R. (1982). Center points in second-order response

surface designs. Technometric, 24, no. 2: 127-133.

Dykstra, 0. (1959). Partial duplication of factoria] experi-

ments. Technometrics, 1: 63-75.Hedayat, A., editor(1978). Optimal design theory. Communicationa

in Statistics, Theory and Methods, 7, no 14: 1295-1412.

Horn, S.D., R.A. Horn and D.E. Duncan (1975). Estimating

heteroscedastic variances in linear modela. Journal Ameri-can Statistical Association, 70, no. 350: 380-38.5.

Hoyle, M.H. (1973). Transformations - an introduction and abibliography. International Statistical Review, 14, no. 2:203-223.

Kleijnen, J.P.C. (1974~1975). Statiatical Techniques in

Simulation. Volumes I and II. Marcel Dekker, Inc., New

York. (Russian translation: Publishink House "Sta[istícs",

Moscow, 1978.)

Kleijnen, .1.P.C. (1983). Crosa-validation using the t statistic.

European Journal Operational Research, 13, no. 2: 133-141.

12

Kleijnen, J.P.C. (1986). Statístical Tools for SimulationPractioners. Marcel Dekker, Inc., New York, (forthcoming)

Kl.eijnen, ,1.P.C., P. Cremers and F. ~n Belle (1985). The powerof welghted and ordinary leas[ squares with estimatedunequal variances ín experimental design. Communications inStatistics, Simulation and Computation, 14, no 1: 85-102.

Kleijnen, J.P.C., A.J. Van den Burg and R.T. Van der Ham (1979).

Generalization of simulation results: practicality of ata-tistical methods. European Journal of Operational Research,

3: 50-64.St. John, R.C. and N.R. Draper (1975). D-optimally for regres-

sion designs: a review. Technometrícs, 17, no. 1: 15-23.

Scheffé, H. (1964). The Analysis of Variance. John Wiley 6 Sons,Inc., New York. (Fourth printing.)

Schmidt, P. (1976). Econometrics. Marcel Dekker, Inc., New York.

Símon, H.A.(1960). The New Science of Management Decision.

Harper b Row, New York.

Steinberg, D.M. and W.G. Hunter (1984). Experimental deaign:

review and comment. (And discussion.) Technometrics, 26,no. 2: 71-130.

Welch, W..i. (1984). Computer-aided design of experiments forresponse estimation. Technometrics, 26, no. 3.

i

IN 1984 REEDS VERSCHENEN

138 (;.J. Cuypers, J.P.C. Kleijnen en J.W.M. van RooyenTestinK the Mean of an Asymetric Popula[ion:Four Prucedures Evaluated

139 T. Wansbeek en A. KapteynEstimation in a linear model with serlally correlated errors whenobservations are missing

140 A. Kapteyn, S. van de Geer, H. van de Stadt, T. WansbeekInterdependent preferences: an econometric analysis

141 W.J.H, van GroenendaalDiscrete and continuous univariate modelling

142 J.P.C. Kleijnen, P. Cremers, F. van BelleThe power of weighted and ordinary leaet squares with estimatedunequal variances in experimental design

143 J.P.C. KleijnenSupereffícíent estlmatlon of power functlons in aimula[ionexperiments

144 P.A. Bekker, D.S.G. PollockIdentification of linear stochastic models with covariancerestrictions.

145 Max ll. Merbis, Aart J. de ZeeuwFrom structural form to state-apace form

146 T.M. Doup and A.J.J. TalmanA new variable dimension simplicial algorithm to find equilibria onthe product space of unit simplices.

l47 G, van der Laan, A.J.J. Talman and L. Van der HeydenVariable dimension algorithms for unproper labellings.

148 G.J.C.Th. van SchijndelDynamic firm behaviour and financial leverage clienteles

149 M. Plattel, J. PeilThe ethico-political and theuretical recontitruction of contemporaryeconomic doctrines

i50 F.J.A.M. Hoes, C.W. VroomJapanese Business Policy: The Cash Flow Trianglean exercíse in sociological demystification

151 T.M. Doupl, G, van der Laan and A.J.J. TalmanThe (2~ -2)-ray algorithm: a new simplicial algorithm to computeeconomic equilibria

11

IN 1984 REEDS VERSCHENEN (vervolg)

l52 A.L. Hempenius, P.(~.H. MulderTotal Mortality Analysis of the Rotterdam Sample of the Kaunas-Rotterdam Intervention Study (KRIS)

153 A. Kapteyn, P. KooremanA dísaggregated analysia of the allocation of time within thehousehold.

154 T. Wansbeek, A. KapteynStatistically and Computationally Efficient Estimation of theGravíty Model.

155 P.F.P.M. Neders[igtOver de kosten per ziekenhuisopname en levenaduurmodellen

156 B.R. MeijboomAn input-output like corporate model including multipletechnologies and make-or-buy decisiona

157 P. Kooreman, A. KapteynEstimation of Rationed and Unrationed Household Labor SupplyFunctions Using Flexible Functional Forms

158 R. Heuts, J. van LieshoutAn implementation of an inventory model with stochastic lead time

159 P.A. BekkerComment on: Identification in the Linear Errors in Variablea Model

160 P. MeysFuncties en vormen van de burgerlijke staatOver parlementariame, corporatiame en sutoritair etatisme

161 J.P.C. Kleijnen, H.M.M.T. Denis, R.M.G. KerckhoffsEfficient estimation of power functiona

162 H.L. TheunsThe emergence of research on third world tourism: 1945 to 1970;An introductory essay cum bibliography

163 F. Boekema, L. VerhoefDe "Grijze" sector zwart op witWerklozenprojecten en ondersteunende instanties in Nederland inkaart gebracht

164 G, van der Laan, A.J.J. Talman, L. Van der HeydenShortest paths for aimplicial algorithma

165 J.H.F. SchilderinckInterregional structure of the European CommunityPart II:Interregional input-output tablea of the European Com-

munity 1959, 1965, 1970 and 1975.

iii

IN (1984) REEDS VERSCHENEN ( vervolg)

166 P.J.F.G. MeulendijksAn exercise ín welfare economice (I)

167 L. Elsner, M.H.C. PaardekooperOn measures of nonnormality of matrices.

ív

I;1 1985 REEllS VERSCHENEN

16k1 T.M. Uoup, A.J.J. TalmanA continuous deformation algorithm on the pruduct space of unitsimplices

169 P.A. BekkerA note on the identification of restricted factor loading matrices

170 J.H.M. Uonders, A.M. van NunenEconomische politíek in een twee-sectoren-model

171 L.H.M. Bosch, W.A.M. de LangeShift work in health care

172 B.B. van der GenugtenAsymptotic Normality of Least Squares Estimators in AutoregressiveLinear Regression Models

173 N.J. de Groof~:eisoleerde versus gecoárdineerde economiache politiek in een twee-regiomodel

174 G, van der Laan, A.J.J. TalmanAdjustment processes for finding economic equilibria

175 B.R. MeijboomHorizontal mixed decomposition

176 F. van der Ploeg, A.J. de ZeeuwNon-cooperative strategies for dynamic policy games and the problemof time inconsistency: a comment

177 B.R. MeijboomA two-level planning procedure with reapect to make-or-buy deci-sions, including cost allocations

178 N.J. de BeerVoorspelprestaties van het Centraal Planbureau in de periode 1953t~m 1980

178a N.J, de BeerBIJLAGEN bij Voorspelprestaties van het Centraal Planbureau in deperiode 1953 t~m 198U

179 R.J.M. Alessie, A. Kapteyn, W.ti.J. de FreytasDe invloed van demografische factoren en lnkomen op consumptieveuitgaven

1tSU P. Kooreman, A. KapteynEstimation of a game theoretic model of hou5ehold labor supply

181 A.J. de "Leeuw, A.C. MeijdamOn Expectations, Information and Dynamíc Game Equilibria

V

18'L Cristina PennavajaPeriodízation approaches of capitalíst development.A critical survey

183 J.P.C. Kleijnen, G.L.J. Kloppenburg and F.L. MeeuwaenTesting the mean of an asymmetric population: Johnson's modified Ttest revisited

184 M.U. Nijkamp, A.M, van NunenFreia versus Vintaf, een analyse

185 A.H.M. GerardsHomomorphisms of graphs [o odd cycles

186 P. Bekker, A. Kapteyn, T. WansbeekConsistent sets of estimates for regresaíons with correlated oruncorrelated measurement errors ín arbitrary subsets of allvariables

187 P. eekker, J. de LeeuwThe rank of reduced dispersion matrices

188 A.J. de "Leeuw, F. van der PloegConsistency of conjectures and reactions: a critique

189 E.N. KertzmanBelastingstructuur en privatisering

190 J.P.C. KleijnenSimulation with too many factors: review of random and group-screening desígns

191 J.P.C. KleijnenA Scenario Eor Sequentfal Experimentation

L92 A. DortmansDe loonvergelijkingAfwenteling van collectieve lasten door loontrekkers?

193 R. Heuts, J, van Lieahout, K. BakenThe quality of some approximation formulas in a continuous reviewinventory model

194 J.P.C. KleijnenAnalyzing simulation experiments with common random numbera

195 P.M. KortOptimal dynamic investment policy under fínancial reatrictions andadjustment costs

196 A.H. van den Elzen, G. van der Laan, A.J.J. 'LalmanAdjustment procesaes Eor firtding equillbria on the simplotope

w i ~u~iw~~~ wi~m~~~~~ ~ i M